mark nottingham

Five Reasons to Considering Linking in Your HTTP APIs

Sunday, 23 June 2013

HTTP APIs

There’s been a lot of interest in and effort expended upon “hypermedia APIs” recently. However, I see a fair amount of resistance to it from developers and ops folks, because the pragmatic benefits aren’t often clear. This is as it should be, IMO; if you’re not able to describe concrete benefits without hand-waving about the “massive scale of the Web.”

The trick, is that those benefits are often subtle and situational. A while back I talked about different types of APIs in terms of complexity, and that applies here; if you’re doing a one-off integration, or if there’s a single server that calls all of the shots for its captured clients, linking might not be as interesting. That said, here are a few concrete benefits I’ve run across when looking at link-based APIs:

1. Mix and Match

“Modern” APIs are often defined with static URLs in-documentation. This has the benefit of being straightforward, and it works well for single-server APIs (e.g., Twitter).

However, once an API makes the jump to multi-server, this gets unwieldy. Any successful “standard” API – whether it be out of the IETF, or just an Open Source project with an API – will have many instances deployed, and this creates some tension.

For example, I might want to mix several APIs onto one host and serve them all on “api.mydomain.com”. If two of them specify that their URLs start with “/v1”, I’ve got a problem.

Or, I might want to split an API across several hosts for operational considerations, or put it in a subdirectory of an existing server to test it, or even delegate service of part of an API to a third-party vendor (“federation”). I can’t do any of this if the URLs are baked into documentation.

Or, larger companies often have their own standards for URI conventions or want to mix in their own tracking or authentication identifiers.

More trivially, my server might not have the same conventions as yours; just as Apache likes “.html” and IIS likes “.htm”, there are idiosyncrasies that are specific to each environment. People shouldn’t have to work against their Web server.

The underlying principle here is fundamental to the Web. Servers need to retain control of their URI space; once that starts being impinged upon, things get messy, ossified, and we risk conflicts. Link-based APIs help avoid this. By having the client fetch a “directory” of the APIs, the server retains control of its URI space, and can incrementally mix together different APIs, each one identified by the link relations it exposes, and configured to specific URLs (or templates for them).

2. Versioning

The usefulness of this flexibility in URI design extends over time as well. When you need to add new features to an API, the most common approach is adding new resources (see more about this in “Evolving HTTP APIs”).

However, when you add new resources, clients need some way to know that they’re there. While you could have some flag in a payload or a header, the most Web-like and flexible way to do this is to include a URI that types the resource with a link relation.

3. Discovery

Extending upon that, linking can become a general mechanism for discovering things about your API, such as what formats a resource supports, whether the resource has been deprecated, whether authentication is required, and what methods it supports. This is the idea behind a new draft I’ve written, HTTP Link Hints.

Fundamentally, discovery is learning stuff about something. Rather than coming up with your own scheme for referring to something, or baking it statically into documentation, why not just use links and attach the metadata to them?

4. Role-Based Access Control

Yet again extending that concept of discovery, a common problem in API design is describing what aspects of the API are available to a particular user. By simply linking to a resource (or not), you can do this. Of course, you still have to enforce authentication on the back end, but linking allows you to guide the decisions that the client is making based upon the role they hold.

5. Cache Invalidation

Another common problem with HTTP APIs is cache invalidation; while you often want to be able to use a cache, figuring out what the appropriate freshness lifetime is a well-known Hard Problem. One simple solution for this on the Web is to change the URI whenever the content changes; this allows you to version things in lockstep, and assures that the client has a single view of your resources, while still getting the (considerable) benefits of caching. CDNs have been doing this for years, and it’s become pretty standard in Web apps too. Props to Michael Mahemoff for drawing the line to APIs explicitly.

We’re Not There Yet

I see a lot of people who are interested in Link-based APIs (“Hypermedia” if you must), but a common question is “who’s using them?”

This chicken-and-egg problem is further worsened by the lack of good tools – especially on the client side – to do anything more than rudimentary HTTP for APIs. However, it’s not like nobody is doing it. Take a look at CIMI for an example; while it’s a big-company effort and it’s not perfect, somebody is doing it. Last time I checked, OpenStack was also flirting with using links for RBAC and deployment flexibility in its APIs.

Beyond that, though, the state of the art is to embed URLs into documentation when defining an API. While that’s OK for simpler APIs (see again the post on HTTP API complexity) – because they can get some of these benefits still using link templates or even just query strings – more complex APIs will suffer. This is why I’ve started pushing back on any standard (e.g., in the IETF) that bakes URLs into specs, unless they use the (very limited) escape hatch of .well-known.

We need tools, and to get the tools, we need shared conventions. I’m still not convinced that we need a general convention for linking in JSON (although I’ve probably softened about it since I last wrote on the topic), but we do need a way for an API to define a “home page” that clients can use to coordinate their interaction with the API, smoothing the path to getting the benefits above. This is the idea behind json-home, which is still a work in progress (and I’ll talk more about it, hopefully soon).

One last benefit of linking in APIs that’s less concrete, but perhaps most important of all in the long run. I think the state of HTTP APIs is pretty horrid; we’re all re-inventing slightly different styles of interaction, forcing the client side to do a lot more work. The level of interop and reuse is woefully low, and we can do better. Defining common formats, link relations and patterns of use gives us a chance to build a better, shared idea of what a HTTP API is.