blog

Let's talk about testing

Web API Versioning

This entry is part 2 of 4 in the series Code Design

Note: this post was not originally published as part of the Code Design series, but fits quite well.

Today’s CommitStrip summarizes quite neatly why sensible API versioning is a must if you don’t want to annoy your customers. It goes without saying that annoying your customers is a bad idea.

When I finally manage to master an API or framework

There are reasons this happens a lot — often enough to warrant a comic strip — and most of them are bad.

Indifference

By far the worst reason is the indifference of API vendors. To analyse where this indifference stems from would take far too long; suffice to say that if you expose an API to third parties, your API is your product, and you should treat it and it’s users accordingly.

Indifference is also one of the more often cited reasons for breaking web APIs, something that didn’t occur so much in the times where web APIs were not the predominant type of API a vendor would provide.

History

Historically you would provide an API in the form of a client library, or module, that your clients import into their code. Breaking such an API is still not a great idea, which led to a plethora of best practices on how to label changes to an API that breaks compatibility (major revision) and those changes that don’t (minor revision).

A lot of these distinctions between breaking/non-breaking changes are related to the technologies used. If, for example, you understand how hard it is to change a C++ library such that programs linked against it do not need to be recompiled, you understand where all that trouble comes from.

But in todays world, most APIs that are offered as products are implemented in dynamic languages and recompilation is not much of an issue. So does any of this matter today?

Drop-In Replacement

The more fundamental problem remains even with dynamic languages; an API will not cause headaches on the side of the API consumer if and only if an update to it acts as a drop-in replacement, with no code changes required by the consumer.

That’s unlikely to be the case for most changes. Still, here’s a list of changes that will usually not affect the consumers of your API:

  • Additional functions.
  • Additional optional parameters to a function, with the function defaulting to previous behaviour if the parameters are not specified.

That’s pretty much it. Note that bugfixes are not listed, because bugfixes are a tricky beast in this context.

Viewed from a certain perspective, a function that produces an error for one set of parameters and some values for another set of parameters is just fine. As a user of the function, you just avoid that one set of parameters that produces errors. If a bugfix changes the behaviour of the function such that this set of parameters no longer produces errors, it can be argued — and it has been argued — that the bugfix breaks compatibility of the API.

The rationale is that if an API consumer has found a workaround to the set of parameters that produces errors, and fixing the function breaks the workaround, then the update clearly breaks the implied interface.

Such a rationale is rather more strict than is usually useful. However, it does lead to a distinction between “safe” and “unsafe” bugfixes that makes sense, and is similar to the definition above:

  • Bugfixes that do not alter the behaviour of successful function calls, but only add to the possible parameters that return successfully, can be deemed compatible.

Rather complicated, huh? And we’re still only talking about dynamic languages, where everything is (usually) easier.

Code Duplication

So assume that you have to provide an update to your API that fixes a critical bug for some consumers, and that does not fall under the above classification. How then do you proceed to publish this update without forcing everyone to upgrade?

The naive method would be to deploy a separate API instance in a different namespace (such as under a different base URL) that contains the incompatible bugfix, and suggest users upgrade to this new version. And this method works, were it not for the fact that you are effectively now supporting two slightly divergent code bases. Multiply this by a large number of such bugfixes, and you live in a form of Dependency Hell.

And that, in a nutshell, is what makes API versioning so hard in practice that few people ever bother with it.

Per-Function Versioning

The sad thing is that this particular view of the problem makes the implicit assumption that one can only upgrade an entire API at once, and never individual functions. Granted, when you imagine changing your versioning scheme to cover individual functions only, you’re entering a different kind of Dependency Hell.

Does version 1 of function foo()‘s output work as input for version 2 of function bar()? If you’re a good software vendor you may take care of answering this kind of question before releasing your next API version, at costs that are likely to be prohibitive. On the other hand, offloading this responsibility on the API consumer is the worse choice.

Clearly, this is not a good idea.

But if per-API versioning and per-function versioning both have benefits and drawbacks, what about some middle ground?

Possible Solution

What if one could combine the simplicity of maintaining per-function versioning, with the simplicity of using strict per-API versions? Well it’s not as hard as you might think.

Consider the following functions that are part of a node.js module:

exports.foo = function() {
  // Do foo
};
 
exports.bar = function() {
  // Do bar
};

Further assume that for an incompatible bugfix, you need to change the behaviour of bar(), but not the behavior of foo(). So let’s re-implement a fixed version of bar().

exports.foo = function() {
  // Do foo
};
 
exports.bar = function() {
  // Do bar
};
 
exports.bar2 = function() {
  // Do bar better
};

That’s not as uncommon a pattern as you might think. Quite a few libraries use a numbering scheme following function names, so users can choose to call either of bar() or bar2(). So far, this is exactly the per-function versioning scheme above, with all it’s problems.

So let’s look at these function names from a different perspective. What if, instead, they were named like this:

exports.v1_foo = function() {
  // Do foo
};
 
exports.v1_bar = function() {
  // Do bar
};
 
exports.v2_bar = function() {
  // Do bar better
};

So far, the situation isn’t any better. The magic happens — and this only works in some languages — when you add this line:

exports.v2_foo = exports.v1_foo;

If you assume your API is reachable under some base URL, and further assume that you specify a path as a version namespace, then version 1 of the API might be reachable under e.g. http://your-api/v1/ and version 2 under e.g. http://your-api/v2/.

If additionally your application server routed everything under those base URLs to functions with the matching version prefix, e.g. http://your-api/v1/foo to v1_foo(), etc., you’re combining the best of per-function versioning with the best of per-API versioning.

The consumer only sees the API version, and upgrades if and when they see fit. On the other hand, you can easily create a new, improved API version without implicitly creating diverging code bases.

Application Frameworks

The above scheme has served us well for a long time, but unfortunately it is not widely supported by existing FOSS packages such as Ruby on Rails — not that it’s impossible to bend such an application framework to your will, but it’s not exactly intuitive.

In the end, we ended up implementing our own miniature application framework to do this and a number of other API-related things well, and only those. But that’s a topic for a different blog post.


Comments are closed.

Reputation. Meet spriteCloud

Find out today why startups, SMBs, enterprises, brands, digital agencies, e-commerce, and mobile clients turn to spriteCloud to help improve their customer experiences. And their reputation. With complete range of QA services, we provide a full service that includes test planning, functional testing, test automation, performance testing, consultancy, mobile testing, and security testing. We even have a test lab — open to all our clients to use — with a full range of devices and platforms.

Discover how our process can boost your reputation.