Versioning RESTful web APIs

When I was recently tasked with designing and building a new set of public-facing REST APIs for a popular online platform, one of the design challenges I faced was whether we should version our web APIs, and if so, how?

Historically there has been a lack of consensus in the industry on best practice in this area of web API design. And on initially researching the problem area I couldn’t find the answers I was looking for, or advice that I felt I could rely on.

This post distills the things I learned from further research and the experience of designing our own web APIs, with the aim of helping those of you that need to tackle a similar design problem. It covers:

  • Reasons to consider versioning your web APIs.
  • The aspects of a RESTful web API which do and don’t require versioning.
  • A classification of the most popular API versioning solutions that are in use today, including which aspect(s) of a RESTful web API they aim to version;
  • Points to consider when choosing a versioning solution(s).

It concludes by explaining my current thinking on the best approach to versioning and what solution we adopted for our APIs. I hope you find it useful!

1. The case for API versioning

Resources and the representations that a web API accepts or serves need to evolve to accommodate new or changed business requirements. However, most businesses will deem it unacceptable to release changes to APIs that break their customer’s existing API clients and their associated integrations.

One way to address this problem is to ensure that API changes are always designed and implemented in a backward compatible (non-breaking) manner. For resource representations this can be achieved by adding new fields whilst maintaining old or defunct ones, and living with the duplicated values. Changes to the capability or behaviour of a resource can also be accommodated in a non-breaking way.  For example, if you want to modify how a resource handles a query parameter, you would create a new request parameter instead of changing the existing one, or create a sibling resource with a different name and URL that takes the alternate query parameters.

There is a school-of-thought that this approach of making all changes non-breaking is sufficient to support the evolution of APIs. However, there may come a time in the future when your APIs become overburdened with the resulting add-ons and the cost of supporting and maintaining them, including explaining legacy features to new customers, becomes too onerous. Whether and how quickly this point is reached depends on the lifetime of your APIs and the rate (and the extent) of changes the APIs need to accommodate – something you’re unlikely to be control of. If your APIs do become overburdened then you’ll really want to start afresh, from a clean piece of paper. This is where the benefit of versioning your APIs comes into play.

Versioning your web APIs, and supporting more than one version concurrently, allows you to release incompatible (‘breaking’) changes without immediately impacting existing API clients, allowing you and your API customers to work to different development schedules. It also gives you the added flexibility to redesign your API resource model and resource representations from scratch, jettisoning the need to support legacy aspects of the previous design, without needing to create and re-launch an entirely new API service (hosted on a different domain).

1.1 Summary Recommendations

Version your public web APIs from the outset to support the option of completely redesigning them from scratch in the future, without needing to resort to creating an entirely new API service.

Avoid making breaking changes to your public web APIs wherever possible. The first rule of API versioning should be to only create a new API version when absolutely necessary. Instead, in the interest of keeping costs down, implement the changes in a backwards compatible (non-breaking) manner, even though this will likely mean making concessions in purity of the API design.

Only bump your API version when the cost / burden of maintaining the current design outweighs the cost (to you and your customers) of making a breaking change, If your domain is reasonably stable and you’ve spent some time doing some upfront design of your APIs then this shouldn’t happen often.

2. Versionable aspects of REST APIs

When considering the design of API versioning, only changes to the public interface of a web API are relevant, since only these have the potential to break existing API clients. Changes to the underlying implementation of an API are irrelevant.

There are two distinct categories of changes to consider when designing a solution for versioning RESTful web APIs –

Representations – This category concerns changes to the representation of an API resource for a supported data-format (XML, JSON, etc). This includes renaming or removing fields (elements or attributes) or changing the way the fields are structured – anything which changes how the representation is consumed by API clients.

Resources and resource capability – This broader category covers changes to the URL of the resource (including its request parameters), the set of supported HTTP methods for manipulating the resource, the HTTP response codes used to signify the result (both success and error) of an operation on the resource, and also the different error responses (API specific error codes).

Both of these categories of changes have the potential to break existing API clients, and hence both representation and resources may benefit from versioning.  Complementary solutions exist for both.

3. Evaluation Criteria for Versioning Solutions

This section highlights points to consider when evaluating candidate solutions for versioning web APIs.

3.1 API Client Platform Support

When choosing a versioning solution for your APIs consider whether it’s supported by those platforms on which your customers are most likely to develop and run their API clients.

For example, if you’re developing a set of data APIs for a B2B application to support reporting to targeted back-end platforms or services then your customers are likely to be building their API clients using server-side programming languages / platforms  (e.g. PHP, Java, Apex, C#). In which case they’re most likely capable of supporting any of the candidate versioning solutions. However, you might, for example, also want to check if your targeted PaaS platforms impose any relevant restrictions or limits.

Alternatively, if you’re developing a set of APIs for mobile or web clients then consider whether the candidate solution can be supported on client-side platforms (e.g. JavaScript, iOS, Flash).

3.2 Support for HTTP Caching

If you have public (unauthenticated, for shared resource) APIs which don’t run over SSL then utilising the caching provided by public web servers (proxies) that exist between your API and clients may be important to you, to reduce load on your app server and improve client response times. If so, you need to consider whether your versioning solution is going to be compatible with the HTTP caching implemented by majority of web proxies. Specifically, how will the web proxies be able to distinguish between two different versions of a resource so that they can correctly serve them from different cache entries?

3.3 API Client Complexity

When evaluating versioning solutions we also need to consider how much extra complexity it adds for API client developers. Is the solution widely known and understood? How hard is it to implement? How much extra effort will it require on your part to explain and support?

4. Candidate Versioning Solutions

This section outlines the most popular versioning solutions for RESTful web APIs in use today, including identification of the aspect(s) of APIs they aim to version.

4.1 Create New API Service

One way of making a breaking change to your API resource model whilst continuing to support existing API clients is to host a new service on a different URL (hostname), whilst sun-setting the existing service. For example, replacing api.example.com with graph.example.com.

Following such a change, API clients may have to adjust their outbound firewall rules. If the APIs runs over HTTPS, new SSL certificates may be required and need to be re-accepted by API clients. Any marketing or branding of the APIs that is based on the old domain name would be affected.

This is therefore a fairly radical solution and something you’d probably want to avoid unless the APIs were being totally re-engineered and the resource model was changing drastically.

4.2 Version Resources in URL

This is a resource versioning solution – it supports making breaking changes to resources and their capabilities (i.e. whole APIs) and allows the resource model to evolve, without needing to resort to creating a new API service.

Resources are versioned by incorporating a version token (of various possible formats) in their URL.

Incorporating the token in the URL path is the most common approach. If this approach is taken then the token is best placed in the leftmost part of the URI path, e.g. api.example.com/v1/foo/123/bar/456, as this indicates that all resources in the hierarchy below the token are part of this version. The token can alternatively be specified as a request parameter, e.g. api.example.com/foo/123/bar/456?v=1 Personally I prefer using the URL path over a request parameter for the following reasons:

  • From a readability perspective it’s not as obvious that the resource is being versioned when using a request param. Using the leftmost part of a URL path makes versioning more prominent.
  • Request parameters can be optional. Versioning in the URL path is a clearer way of expressing that a version is mandatory, if that’s what is required.

4.2.1 Permanency of Resources

When the Version in URL solution is used and a version no. is bumped a key point to understand is that it actually results in a whole new set of distinct resources being defined which have no relationship to the existing resources in the previous version. For example, if the current version no. is 1 and there is a Foo resource with URL /v1/foo, and the version no. is bumped to 2, there may not necessarily be a Foo resource, and if there is and its URL is/v2/foo it has no relationship to /v1/foo. This is significant because the Version in URL solution gives you the ability to start afresh when designing your resource model.

When Versioning in URL, the potential downsides (cons) of creating a whole new tranche of what might be viewed as duplicate resources when a version no. is bumped is also worth considering.

Creates multiple, non-permanent URLs for resources – This argument asserts that versioning in the URL leads to the creation of URL aliases for a resource each time the API is versioned. For example Foo 1234 is accessible via /v1/foo/1234 and /v2/foo/1234, etc.  And that this creation of URL aliases is generally best avoided as it reduces effectiveness of web caches and SEO. However, as explained, /v1/foo/1234 and /v2/foo/1234 are in fact considered to be totally different resources (even though they may resolve to the same entity).  And in this case we actively want web caches to serve the content for the two resources separately. So I don’t consider this to be a downside of the solution, but a benefit.

Troublesome for API clients which persist URLs – Any URLs which an API client has hard-coded into their app or stored (bookmarked) are no longer valid in the new version of the API, and as part of upgrading they’ll need to munge (search and replace) these URLs. Versioning the resources collectively on a per API service basis minimises the impact of this issue as an API client only needs to maintain a single API version for the collection of APIs, rather than one per API.

4.3 Version Representations in Media Type

There is an argument that resource representations are more likely to change than resources, and therefore that there is potential value in allowing them to evolve independently by giving them their own version identifier, distinct from but related to the resource version.  For example, version 1 of the Foo resource may support both version 1 and 2 of its XML or JSON data-format.

The Version Representations in Media Type solution provides the ability to completely redesign (evolve) a representation for resource(s) in the future without making a breaking change, and does so in a manner which has less impact on API clients than the alternative of bumping the resource API version. It avoids the need to create a completely new set of resources and give all API clients notice that they need to upgrade within the sunset period. Instead, API clients are informed there is a new version of the representation for a particular resource/API and if they wanted to benefit from it they could upgrade.

The majority of examples and proponents of versioning representations I’ve researched advocate maintaining a representation version per resource (with the same one being reused for all supported data-formats e.g. XML, JSON).

In the Version Representation in Media Type solution, API clients request the version of the representation of the resource they want by including a version identifier in the media-type that is specified in the standard Accept or Content-Type HTTP request header used in HTTP content negotiation.

The are a couple of possible ways for the version identifier to be included in the media-type.

4.3.1 Version in custom media type

Custom (vendor-specific) media (content) types are generally used as a way of adding application specific semantics to a standard data-format, for example associating meaning to named link relations.

This approach to versioning requires a custom media type to be defined for each version of a representation that is supported for a particular resource, e.g.

  • application/vnd.acme.foo-v1+xml
  • application/vnd.acme.foo-v1+json
  • application/vnd.acme.bar-v1+xml
  • application/vnd.acme.bar-v2+xml

These replace (or supplement) the use of standard media types such as application/xml, and each has a separately documented schema (specified either by example or formally using e.g. XSD). The versioned media types are specific to resources, allowing them to change independently.

For example, to request version 1 of the representation of a Foo resource in XML format, the request and response would be:

-->
GET /foo/1234 HTTP/1.1
Accept: application/vnd.acme.foo-v1+xml
<--
HTTP/1.1 200 OK
Content-Type: application/vnd.acme.foo-v1+xml
<foo>
  <owner>
    <email>jbloggs@example.com</email>
    ...
  </owner>
  ...
</foo>

When a breaking change(s) to a representation is unavoidable this is handled by creating a new version of the media type and associating the new/changed representation with it. The API documentation specifies the difference in the resource representation between versions, e.g. v1 has a group of optional properties, while in v2 these same properties are mandatory.

Old and new clients are supported simultaneously based on the different versioned media-types they specify in Accept or Content-Type headers, e.g.

-->
GET /foo/1234 HTTP/1.1
Accept: application/vnd.acme.foo-v2+xml
<--
HTTP/1.1 200 OK
Content-Type: application/vnd.acme.foo-v2+xml
<foo>
 <owner>
   <emails>
     <email>jbloggs@example.comjoe.bloggs@home.com</email>
   </emails>    
   ...
 </owner>
 ...
</foo>  

GitHub use this approach to version their resource representations, as of V3 of their API, as described in GitHub Developer > Overview > Media Types. Interesting points to note:

  • The media type name (vnd.github[.version].param[+json]) doesn’t include the resource name as in the example above, instead only differing by the resource representation version and data format. This looks simpler for an API client, but it may be less clear there is still a distinct, potentially different, version for each resource.
  • Versionless media-types are still supported, including both the standard content-type for the data format – application/json – and the custom one – application/vnd.github+json.  In this case, GitHub, returns the latest representation. API clients which rely on this rather than versioned media types are more prone to break in the future, hence they do recommend a version be specified. The versionless media-types probably only exist to make it easier to explore the API using a browser (without needing a REST client extension).

At this time I have limited experience of applying the use of custom media types in APIs and so remain wary of the extra issues or complexity their use may bring.  One possible downside is that customers may need to reconfigure security rules on firewalls or their web/application servers to allow the use of non-standard media-types. The no. of customers applying such protocol level security (HTTP header inspection) may be small. Also, it’s not a concern for encrypted APIs, as the headers can’t be inspected anyway. So, this is not necessarily a showstopper, but another possible hurdle to a customer using your APIs.

4.3.2 Version as custom parameter to standard media type

If it turns out there is good reason to avoid introducing custom media types then another possible solution is to specify the representation version identifier as a custom parameter to a standard media type, e.g.

-->
GET /foo/1234 HTTP/1.1
Accept: application/xml; version=1
<--
HTTP/1.1 200 OK
Content-Type: application/xml; version=1
...

In terms of the extent of API client support alone there is unlikely to be any significant benefit of using this approach, as if an API client is able to specify a custom parameter on a standard media-type then they’re just as likely to be able to also use a custom media type.

4.4 Version in Custom Header

Another solution for versioning web APIs that I’ve seen used by a few well known API vendors is to specify a version identifier in a custom HTTP request header e.g. Acme-Api-Version

-->
GET /foo/1234 HTTP/1.1
Accept: application+xml
Acme-Api-Version: 1
...
<--
HTTP/1.1 200 OK
Content-Type: application/xml
...

It’s not clear to me what aspect of the APIs are being versioned when using this solution because the version is being tunnelled over HTTP and interpreted by the API service. It might support versioning of resources, but that does not manifest itself in the URL. It could support versioning of the representation but uses a custom header rather than using standard HTTP content negotiation (Accept and Content-Type) request headers.

Google, for example, used this approach to version their GData Atom-style APIs. The GData API Developer Guide states “the custom HTTP request header GData-Version is used to specify the version of the Google Data Protocol”. It’s not clear to me what this means.

4.4.1 Permanency of Resources

In contrast to Version Resource in URL, the Version in Custom Header solution does not change the existing set of resources when the version no. is bumped – the existing resources all remain, at the same URL. For this reason this solution is not considered to support versioning of resources. The only way to introduce a breaking change to a resource is to create a new, sibling resource with a different name.  It’s possible to retire a resource or one of its method by re-implementing it to return an appropriate HTTP response status code (e.g. 404 Not Found or 410 Gone). But you can’t use the same resource name. You can’t redesign the resource model from scratch without resorting to creating a new API service.

4.5 Summary Comparison of Solutions

Version Resources in URL provides a resource versioning solution.

Version Representations in Media Types supports finer grained versioning of resource representations, and can be used in addition to Version Resources in URL.

Version in Custom Header versions an API in an opaque, application-specific manner.

The remainder of this section summarises how these versioning solutions compare using the previously described evaluation criteria.

4.5.1 API Client Support

All three of the identified versioning solutions will be widely supported across most client-side platforms since they only require either specifying the version identifier in the URL, a standard request header or a custom one. The only caveat I’ve come across is the Version in Custom Header solution is not supportable by Flash as it doesn’t support setting custom headers for GET requests, only POST. (The ActionScript API docs state that “Due to browser limitations, custom HTTP request headers are only supported for POST requests, not for GET requests”).

The Version Representations in Media Types solution requires API clients use custom media-types in conjunction with the standard HTTP content negotiation (Accept and Content-Type) request headers. It’s worth considering whether your customers will be able to support requesting and consuming custom media-types.

If you want to make it easy for developers to explore your APIs using just a browser, rather than a REST client plug-in or an API console, Version Resources in URL supports this without any additional work. If using Version Representations in Media Types this requires adding extra support for versionless, standard media-types (e.g. application/xml). It can also be supported when using Version in Custom Header by additionally supporting client’s specifying the version identifier via a request param instead. (This is also a workaround to the previously mentioned Flash platform limitation).

4.5.2 HTTP Caching

With regards support for caching of non-private web APIs by public HTTP proxies –

Version Resources in URL causes each version of a resource to be a distinct resource with its own URL.  As a result, this solution is compatible with (does not impinge on) proxy-based caching support – all public web proxies will maintain separate caches for  different versions of a resource if the appropriate HTTP headers are set.

When a version is specified in an HTTP request header, as is the case for the Version Representations in Media Types and Version in Custom Header solutions, web caches must be informed that requests with different values of the header result in different responses and shouldn’t be served from the same cache entry. The HTTP/1.1 spec. supports this by including the name of the header in the Vary response header. (The Apache HTTPD manual contains a useful explanation of how it supports the use of the Vary header as part of Content Negotiation).

Version Representations in Media Types and Version in Custom Header therefore both rely on a proxy supporting the use of request headers in the Vary response header. Despite being part of the HTTP spec. the extent of support for the Vary header across the web is still questioned. The standard headers used by Version Representations in Media Types are more likely to be supported, but proxy support cannot be relied upon for either solution. When not supported, in the best case this will result in a proxy not caching any of the resources.

4.5.3 Client Complexity

Version Resources in URL is a commonly used, highly visible versioning solution which is easy for API developers to understand and simple to implement.

Version in Custom Header is also simple for an API client to implement. It will however be unclear to your API users exactly what is being versioned, which may lead to questions.

Version Representations in Media Types adds complexity for API client developers. It requires an understanding of custom media types, management of additional version identifiers over and above those used for resources, on a per resource basis, and the ability to produce and consume bodies with non-standard media types.

5. My Chosen Approach

I’ve chosen to version the resources in our public-facing APIs. Whilst it’s feasible to evolve the resource model and resource capability without it, doing so provides the  flexibility to redesign the model from scratch in the future, including reusing some resource names/types, without changing the API service’s domain/hostname. This seems prudent given the solution has little if any cost (it is fully supported by API clients and doesn’t make them any more complex) and has no major trade-offs (it does not impinge on support for proxy based caching).

We chose to version resources in the URL rather than via a custom HTTP request header solution, as for non-private APIs it offers better support for HTTP caching. Equally importantly it allows the whole resource model for the API to be re-designed from scratch (including removing whole resources), without changing the root  URL (domain name and context) for the API service.

The Version Representations in Media Types solution provides a useful means of evolving another aspect of APIs in a backwards compatible manner, which can complement a resource versioning solution. However, it adds some complexity to API clients and for this reason we’ve chosen to defer its adoption for now, subject to further trialling and evaluation. It’s possible to defer the decision until it becomes essential to make a breaking change to a representation. It can be introduced retrospectively to both new and existing APIs as long as you continue to support the standard, versionless media-types (e.g. application/xml) which for backwards compatibility must then be treated as requests for the oldest supported representation version.

Leave a comment

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s