My team and I design and build a lot of REST APIs during the course of our work, to expose functionality and data to client systems, and to integrate our back-end services with one another. I was therefore interested in learning more about Amazon’s recently launched API Gateway service. This is a short post summarising the role and major features of Amazon API Gateway based on my own preliminary research and trialling. I also briefly cover what the service is not designed to support, and consider possible future use-cases.

Amazon API Gateway in a Nutshell

The following summary of Amazon API Gateway (AAG) is based on one provided by Amazon, with some marketing spin removed, and the emphasis added by me –

“…a fully managed service that [allows] developers to create, …monitor, and secure APIs at any scale; …Using the service’s web console you can create an API that acts as a “front door” for applications to access data, business logic, or functionality from your back-end services, such as workloads running on …Amazon EC2, code running on AWS Lambda, or any web application….”

In essence, AAG gives you the ability to create a tier of public REST APIs, that is deployed between your API clients and back-end services, without needing to write or deploy any code. By acting as a proxy between your API clients and back-end services, the AAG (like any other API Gateway product) can apply some common services to all your APIs. Amazon also take care of scaling the servers on which the API tier runs to meet any load generated by concurrent API requests. This is all summarised in the following diagram –

Major Features

As of today (Dec 2015), the major features of AAG are as follows –

Traffic Management

AAG has features to protect your back-end service from unwanted and unmanageable volumes of API requests.

It can protect against DDoS attacks – minimising unwanted load on your back-end services.

Requests can be throttled by defining rate limits on a per API (URL and method) basis, to prevent your back end services being overwhelmed. When a limit is exceeded the requests are dropped.

Security – Authentication & Authorisation

The REST APIs you deploy on AAG have to run over HTTPS. They can either be open to all, or require authentication. The need for authentication can be enabled/disabled per API method.

The only authentication protocol supported is a hash-based message authentication code (HMAC) i.e. signing requests using a pair of public and private keys. This is the same proprietary authentication solution Amazon use for their own APIs. It requires you manage your API keys in the AWS Identity and Access Management (IAM) service.

To date, we’ve always used Basic Auth to secure our web APIs. When used over HTTPS / SSL, it provides simple but effective security. Given Amazon already support signing of requests I can understand why they wouldn’t bother supporting Basic Auth. One of the advantages it has over Basic Auth + SSL is that it also checks data integrity – that your requests haven’t been tampered with by a man-in-the-middle attack.

Service Integration

AAG supports invoking the following types of back-end services in response to the REST API calls:

• AWS Lambda – There is explicit support for integrating your APIs with the AWS Lambda service. You can invoke a named synchronous (request/response) Lambda function that you’ve previously deployed to the service.
• HTTP Proxy – You can use the API Gateway to proxy any other web API. This could be used to adapt / convert your own service’s legacy (e.g. XML/RPC) APIs.
• AWS Service Proxy – There is also support for generically invoking any other AWS service which has its own web APIs.
• Mock Integration – You can mock-out the back-end API call, returning either an entirely static (‘canned’) response, or a dynamic one, which varies based on aspects of the HTTP request.

Request and Response Transformation

There’s support for adapting your public-facing REST API requests and responses to your back-end service calls. You can transform the HTTP request body before a call is made to the back-end service; and also the HTTP response received from your back-end service before it is returned to your API client. This is supported using the Apache Velocity template engine to create template response bodies e.g. in JSON or XML.

Deployment Environments (‘Stages’)

AAG requires you to create named (e.g. “QA” and “production”) environments of your choosing to support the deployment of named versions (releases) of your API.

There is support for viewing and reverting (rolling-back) to previously deployed versions in a given Stage.

A Stage also supports environment specific configuration for API including e.g. settings for caching and throttling, and environment variables.

Caching

You can optionally configure AAG to cache API responses to reduce the hits on your back-end services. (It’s worth keeping in mind that this cache would be in addition to, and behind, any edge-caching provided by your CDN, if you use one).

The AAG cache is enabled and sized on a per Stage (environment) basis. Caching costs extra. It is priced at an hourly rate, which increases with the size of the cache.

Strangely the AAG web console doesn’t currently appear to provide any options for configuring the cache (e.g. TTL / cache expiry times for different resources) beyond specifying its size.

API Versioning

API versioning is supported by cloning a whole API application (i.e. all the resources and their methods) and making it available on a new domain.

This approach to versioning – a whole API application as opposed to individual resources – has its pros and cons. You may want to avoid changing the hostname your customer’s use to access the new version of your APIs. Regardless of this, my prior conclusions on this subject of versioning APIs remain unchanged – versioning an API has a cost and should be avoided where possible by using alternative solutions. Versioning should be a last resort.

Monitoring

AAG provides a dashboard for each API application to visualise (graph) metrics over a selected period (days). Available metrics include e.g. total API calls,  latency (time taken to return response); no. of error responses;

What it’s not

AAG is not designed to support the following requirements –

Load Balancing

AAG doesn’t provide load balancing. All API requests are forwarded to a single configured endpoint. If you need to distribute the requests across multiple instances of your back-end service then you need to point API Gateway at a load balancer.

API Gateway for Private Services

AAG can only be used to publish APIs that are accessible over the Internet, and the back-end services it proxies must also be so. Use of VPCs to proxy ‘internal’ or ‘private’ services not connected to the Net is not currently supported. This may rule out use of AAG for highly secure applications. However, use of client (SSL) certificates is supported, so although you have to expose your services you do have the option to limit access to just AAG.

API Quotaring

A common requirement when building public APIs is to apply quotas to individual API users to ensure fair usage of the service across the whole client/user base, avoiding any one client hogging resources. This is typically implemented by setting rate limits (e.g. no. of requests per minute) per API user. Although AAG supports configuring a rate limit for a given API this applies globally to all API requests, for the purpose of throttling total load on one endpoint. Therefore if you need to apply quotas to your API users you’ll still need to implement it yourself in your API back-end. This would be a useful additional feature for Amazon to add to AAG.

Cost

Like all other Amazon web services, AAG is based on a pay-per-use model, with no min spend or upfront costs. With AAG the cost is based on the number of API calls (requests), and outbound data transferred (no. x size of responses).  As noted above, caching, which is an optional extra, is an additional cost. See Amazon’s pricing page for more details.

Likely Use Cases

Subject to further evaluation my first thoughts are that this is not going to change the way we build and deploy the majority of our RESTful APIs in the short to mid term. One of the reasons is that adoption of AAG requires you to develop and deploy your APIs separately from your back-end services – introducing a separate API tier. We currently build and deploy our APIs as a tightly integrated, component part of our back-end (micro) services. Whilst, the design process for these APIs can and is decoupled from business logic, in practice the development is not. We build our back-end services on top of a single application development framework / stack that includes excellent support for developing RESTful web APIs. This offers many development benefits including debugging, first-class support for unit and (component-level) integration testing, whilst at the same time supporting cross cutting concerns (e.g. dependency injection) and a consistent programming model (patterns, annotations etc).  Also, AAG doesn’t (and never will) offer the same flexibility you get from coding your own RESTful web API controller, on top of your preferred REST API or application framework. One current example is that it doesn’t support orchestrating back-end service calls. In all cases, you’re limited to invoking a single back-end service method per REST API call.  

Having said the above, there are still cases where we will consider using AAG in the future – 

We may start by using AAG to provide traffic management features (DDoS protection and request throttling) for our back-end services that are already deployed in the Cloud.  This will likely be more scalable and cost-effective than licensing and deploying our existing traffic manager product in the Cloud. It would also be a simple, relatively low risk way, to get some production experience of using AAG.

Another way to gain familiarity with AAG is to use it to mock-out APIs you intend to build by other means, using the service’s support for mock back-ends.

I suspect a big driver for the future adoption of AAG will be providing REST API for back-end services built on AWS Lambda, driven by the latter’s potential for building highly available, scalable back-end services at lower cost, and the combined benefits of what is being referred to as a ‘server-less’ architecture. This will be an area we’ll evaluate when we next have the need to build a highly scalable, public facing API, and the freedom to chose how the back-end service is implemented.

Using AAG to provide authentication for new public APIs is also an option. Although this will lock you further into AWS, based on having to use Amazon’s method for digitally signing requests, and IAM to manage your keys.

Next Steps

Having already worked through the tutorials in the AAG developer guide, I’ll be looking  to gain more experience of AAG by building a prototype / proof of concept for one of the use-cases identified above. This will also serve as a vehicle for investigating how the use of AAG fits in with our development (and maintenance) process, and realistically how that process will need to be adapted to accommodate its use, in the areas of e.g. multi-person dev teams; testing; version control, deployment etc.

Subscribe to the blog to get notified of future posts on this subject.

Thanks for reading.