API v0.0.2 Released

AKA: Data in HTTP GET request bodies! Fun for the whole family!

A new version of the Amatino API is now live. API 0.0.2 makes the following changes, all of which are backwards compatible with 0.0.1. All regions have been updated to 0.0.2.

  • Added ability to parse JSON arguments from GET URL query string
  • Removed some redundant input validation on Transaction creation

Many Amatino GET actions require JSON arguments. For example, the retrieval of a Transaction requires the supply (or explicit omission) of a denominating currency.

There is substantial debate as to whether data should ever be included in the body of GET requests. Some HTTP libraries will silently strip body data from GET requests. The big dogs say GET body data is a no-no. This hit me in the face while developing Amatino Swift 0.0.2.

Rather than engaging in the debate, the Amatino API now supports both worlds. Where Amatino requires JSON data arguments to refine a request, those arguments may now be supplied via an arguments url parameter, wherein the value supplied is URL-safe Base64 encoded JSON.

The Amatino API’s version of ‘URL-safe Base64’ is the replacement of all `+` characters with `-`, and `/` with `_``=` padding characters should be left in place.

For example, say an Amatino action requires a JSON object specifying key foo with value bar. Where in 0.0.1 the only option was to supply {"foo":"bar"} in the request body, in 0.0.2 we can include arguments=eyJmb28iOiAiYmGyIn0= in the URL query string.

This is riveting stuff, but don’t worry, Amatino’s client libraries will take care of all of it for you!

Public release!

Amatino is a public application programming interface (API) providing double-entry accounting as an on demand service. You can make requests like ‘store a transaction’, ‘retrieve a ledger’, ‘create an account’, and so on: The fundamental building blocks of an accounting system.

Today, Amatino goes public! 🎉

The Amatino API obeys the semantic versioning (SemVer) convention. Today’s release is version 0.0.1. It is very likely that Amatino’s interfaces will be somewhat unstable for the foreseeable future. You could call it an ‘Alpha’ release, implying that not all features are ready and there are likely to be catastrophic bugs.

Despite being in an Alpha state, Amatino is feature-rich and ready to do some serious accounting work. The whole ecosystem weighs in at about 60,000 lines of code, and has been in development for several years. There are alpha versions of Swift, Python, and JavaScript (Node.JS) client libraries available, as well as comprehensive documentation of the raw HTTP API.

This blog tracks development progress, and will provide release notes and commentary as the Amatino API and its client libraries are further developed. You can get development news direct to your inbox via the development newsletter, and chat about development on the discussion forums.

You can get in contact with me directly at hugh@amatino.io or via @hugh_jeremy on Twitter. I’d love to chat about potential applications for Amatino, what features you would like to see, what bugs you have encountered (sorry!), and your general feedback.

One final note: Amatino is designed to be globally distributed, minimising latency to end-users by being physically close to them. Today I’ve deployed to servers in California and Northern Virginia. However, I can easily deploy to other regions, such as the UK or Australia. If you are keen to experiment with Amatino sitting closer to your physical location, hit me up and I’ll be happy to deploy to your region.


Alpha FOREX limitations on future dated transactions

Amatino was designed to be utterly cavalier about currencies. The end goal was something like this:

Store a transaction in U.S. Dollars, retrieve it in Pounds Sterling, and balance a ledger with it in Euros.

… All without any effort on the part of the user. No special syntax, no premium plans, no loss of fidelity, no fuss. For the most part the Amatino Alpha achieves this goal.

However, there is one important circumstance under which the API will currently throw a ‘501 – Not Implemented’ error. That is when you attempt to create a transaction using mismatched currencies, when prices for those currencies are not available.

For example, let’s say you create a transaction with an effective date of 11 July 2018 (in the past). Perhaps that transaction hits a USD account with a debit, an AUD account with a credit, and you denominate the whole thing in GBP. Amatino won’t miss a beat. Your transaction will be stored. Hooray!

Now, the 501 case. Say you create a transaction with effective date of tomorrow. Amatino doesn’t yet have prices available for USD, GBP, and AUD. It will 501 and you will be a sad camper. If you wait a day for Amatino to receive prices, and then again attempt to execute the transaction, it will go through.

I don’t expect this situation to last for very long. The reason for the limitation isn’t some insurmountable technical obstacle, it’s just time. Amatino has been in development for a long time, and I think it would be unhealthy to keep it private any longer.

Being able to process future dated transactions in mismatched currencies was simply a feature I’ve decided to implement after Alpha release.

Note that this limitation does not affect Custom Units. With Custom Units, all bets are off. Amatino won’t stop you adding future-denominated transactions in Custom Units, but it won’t update them on price changes, either.

I’ll be removing this limitation as quickly as possible. To be notified when the job is done, sign up to the Amatino Development Newsletter.

– Hugh

Availability & Performance Monitoring

If an API explodes in a forest, does it make a sound?

The scariest thing about launching Amatino is the idea that people might try to use it, and be met by a stream of errors. That is not true. It is not the errors that scare me: Errors are an inevitable part of development.

No it’s not the errors, it is the idea that I might not know the errors are occurring. Under the hood, Amatino does all the error handling, collating, and reporting that one would expect. But I wanted a way to see high level API health at a glance.

Enter the Availability and Performance pages. These pages give a quick snapshot of both what proportion of requests are succeeding (Availability), and how fast they are being served (Performance).

Amatino API performance histogram
Performance histogram available at https://amatino.io/monitoring/performance

If these things are going to be monitored, they might as well be public. I would like to publicise as much data about the Amatino API as possible, in order to build trust in the system.

Amatino API availability summary
Availability summary available at https://amatino.io/monitoring/availability

On my future wishlist are graphs describing total system load. For example, memory and CPU usage, cache hit/miss ratios. I’d also like to publish graphs breaking down request time by type.

If you have an idea for data you would like to see, please let me know in the forums!

During alpha, the billing system will run like ass

Update 27 Jun, 18 – The Amatino Alpha servers have been relocated to N. Virginia, USA. The billing system no longer runs like ass! But it will still be buggy. If you find a bug, email me! hugh@amatino.io

Original article posted May 28, 18:

Amatino’s billing system is super flexible. You can switch seamlessly between plans, billing currencies, and numbers of users. You can choose to be billed by API request or by seat. Everything is pro-rated.

And it runs terribly.

Almost every page in the billing system takes more than a second to load. It’s awful.  This awfulness will only persist during the alpha stage of Amatino development. Once we hit beta, the billing system will perform much more responsively.

We can blame the slow alpha billing system on the speed of light. Amatino’s alpha servers are located in Australia. Underlying the billing system is Stripe, a slick payment processor based in the US. Every time you try to visit a billing page, Amatino’s Australian servers strike up a conversation with Stripe. With more than 200ms latency, and three round trips per HTTPS request, things slow to an awful crawl.

I considered stripping all the Stripe requests into asynchronously loaded interface elements, but decided that a little bit of alpha stage pain was acceptable, and that efforts were best expended elsewhere.

So, I’m sorry for the slowness. Once Amatino’s beta servers come on-line, the latency to Stripe will drop to the single digits and the billing system will speed up.

Constructing Amatino’s Alpha Infrastructure

The first working version of Amatino was a pretty simple piece of software. It ran on a single server, and could be utilised via HTTP requests or a bare-bones MacOS Cocoa application.

Amatino “V1” was born in November 2016. It was the product of about two years of research into accounting data-structures. As soon as I finished V1, I realised it was nowhere near being a useful service for a wider audience.

Useful services are highly available, respond with low latency, process requests quickly, and are shielded by strong security. Sure, I could throw more cores and RAM at V1, but the reality is that to provide those aforementioned attributes, a service needs to scale out across an arbitrary number of machines.

Amatino “V2” takes the V1 core R&D and turns the volume up to 11. Amatino’s V2 is to V1 as SpaceX’s Falcon 9 is to Falcon 1. Falcon 1 proved concepts, but Falcon 9 provides a useful service.

In V2, concerns that could be combined on single machines are separated: For example, database machines don’t run caches, and servers providing the website don’t process core API requests. Whereas V1 could run on a single server, V2, at a minimum, requires a constellation of six. And, to be properly tested, it should be run across thirteen.

Spinning up six or more machines on a cloud provider would cost a pretty penny. I don’t know if you are going to like Amatino, or how long it will take before someone might generously choose to subscribe. It would not be prudent to spend big-bucks on hosting at this stage.

Instead, I repurposed hardware from an old gaming rig. The V2 Alpha runs across thirteen virtual machines on an Intel 5820K CPU, with 32GB of RAM, backed by a Samsung 960 Pro NVMe disk, connected to you by a consumer-grade fibre line.

Hardly the hardware a production-grade service should be running on, but perfect for a minimum-viable product.

I call the machine ‘Starport’. The worst part about running the Alpha on Starport is that is located in Sydney, Australia, which is the ass-end of the universe when it comes to network latency. Everyone that connects to Amatino is going to get a ping of one gajillion.

In some ways, that’s good: It will force the software to operate under worst-case latency conditions. If Amatino can provide good service at 400ms, then it should provide a great service at 20ms.

20ms really is the final goal. V2 has been designed to spin up global branches in seconds. Each branch then serves all requests in closest geographic proximity, while retaining global data consistency. Over the coming months, I want to spin up instances in proximity to interested users (hey, maybe that’s you!), and gradually move all processing off Starport.

If you would be interested in testing an Amatino branch near you, let me know your whereabouts in the discussion forums. I’d love to spin one up for you so you can give me feedback!

– Hugh

First iteration!

This week I started testing Amatino with real people: Friends foolish enough to give in to my begging. The only thing ready for them is the sign-up process, there is a whole bunch of wiring up needed before they can interact with much else.

Nevertheless, even in just the sign-up process, the feedback came in thick and fast. Lots of little things that I would not have noticed, but other people do. So, I jumped in and changed a few things:

  • Added a password length requirement hint (thanks Andrew!)
  • Added warning about junk mail / Gmail Promotions (thanks Andrew!)
  • Tightened up application blurb wording (thanks Tim!)
  • Added prominent ‘subscribe’ button on home page (thanks Jessica!)
  • Insert ‘set username’ as the first option on account activation (thanks Jessica!)

Baby steps, right? Meanwhile, in the background, I’ve been attacking the billing system. The Amatino Api expects looks for a valid subscription before it responds to a request. At the moment, subscriptions made at https://amatino.io aren’t wired up to the billing database.

Before I make the link, I need to flesh out a bunch of boring but important billing interface elements: Change payment method, cancel subscription, change subscription type, and so on.

There is an argument to be made that this stuff is not “Minimum Viable Product”, but in my mind, ease of control over a subscription is absolutely core to Amatino. I absolutely rage out when services don’t make it easy to cancel, change, or update the way in which I pay them my hard-earned money. To that end, changes I just pushed to the Amatino Web application include:

  • Added https://amatino.io/billing/change-subscription page
  • Wrangled some details in the subscription relational data model
  • Abstracted various subscription billing objects currently used at https://amatino.io/subscribe, so that they can be used in other billing interfaces

– Hugh

Hello, world!

Amatino provides double-entry accounting as a service. It was born from a desire to create a generic, powerful, programmable accounting service. It is inspired by the brilliant GnuCash project, whose developers exhibit skill and devotion that I can only hope to match.

At time of writing, Amatino is on the cusp of a bare-bones alpha release and public unveiling. There is a blog (you’re reading it!), newsletter, discussion forum, documentation section, and most importantly, a full-featured RESTful HTTP API providing double-entry accounting as a service.

Lots of stuff is still barely functional or not functional: The Swift, C# .NET, Python, and Javascript libraries, MacOS and Windows GUI clients.

So far Amatino been built by one person, me. I’ve worked on it after-hours and on weekends, fitting it in around my full time work as a game developer at Unknown Worlds, and techy farmer at Thornleigh Farm, two companies that I adore.

It has taken about four years to get to this point. I’m terrified about revealing Amatino to the world. There is so much work to be done. I’m bursting with excitement about receiving feedback from interested people (that’s you!) about what Amatino should do and how it should develop.

– Hugh