Ruby on Rails has recently announced that the HTTP patch method will become the new primary HTTP method for updates. A lot of discussion as well as general Rails bashing ensued all over the internet, as well as a storm of tweets.
My favorite however, was from Ryan Dahl, the creator of node.js:
And so to the possible annoyance of Mr Dahl, I would like today to delve deep into realms of Http nerdom, and discuss the new hotness in REST: The Hypermedia API.
I have seen it being mentioned more often lately and I decided to try and understand it more deeply. This post thus serves as an exploration of the idea to me just as much as it is for the reader, so bear with me if I get it any or all of it wrong, and please feel free to bash me below by leaving a comment. Praise would be surprising, but also welcome. So without further ado, let’s get into it:
What the hell is a Hypermedia API?
Roy Fielding might have been the one who wrote the seminal work on REST, and how to leverage the power of HTTP to build robust and powerful distributed web applications, but I would dare to say that Ruby on Rails has been the most instrumental in spreading the word, when it adopted REST as it’s default mechanism for web services and controllers. But as much as Rails did to evangelize Fieldings’ ideas, a big part has also been missing in most developers minds, including myself. In a way, this is also a post about how many web developers, again including myself, do REST wrong, despite their best intentions.
For a long time, people have equated a RESTful API to consist of Resources representing your domain, each identified by a unique URI. The resources can be queried and manipulated by using HTTP verbs, usually GET, POST, PUT and DELETE (and soon for Rails developers, PATCH). In Rails, these resource URLs usually equate to your Controllers and their representations usually maps to your models.
Rails also introduced many people to content negotiation, the idea that each resource can have many representations, depending on the format requested by the user agent. In the Rails world, this means rendering a view when HTML is requested, or calling some serialization mechanism, like to_json for json, etc.
Throw in some pretty URLs (which actually don’t have anything to do with REST), and for a lot people, you’re done! You are now the proud owner of a RESTful API and everything will be rainbows and unicorns from here on!
Well it turns out that ‘s not enough. A lot of ‘REST’ APIs in Rails are just exposing their back end database using HTTP verbs to map to common CRUD operations.
This is wrong and doesn’t use the full power you have at disposal when using HTTP as a protocol for your API. The key thing to understand is that your resources are not your domain objects, but representations of your domain that live within the context of your API. Therefore, they don’t just contain data pertaining to your domain, but can and should also contain metadata describing their possible states and actions that can be performed on them. This is hypermedia.
So you might ask, as I did initially, how are we supposed to parse and understand this metadata? This comes back to Media Types, or more specifically, MIME Types. The MIME standard allows you to easily define the structure of HTML documents, but despite the standard types like text/html, or text/csv, you can and should define your own custom types specifically for your API or data format. A lot of domain agnostic types can/are also defined, for instance, checkout Githubs’ amazing docs on their custom types and browse a multitude of custom types here.
As it turns out, there is an exist Media Type that already has a lot of semantics built in for including resource metadata: HTML, of course! Using HTML in your Hypermedia API also means you can have a “browseble” API, so that both humans and API clients can digest the same resource representation.
Why should I care?
So all of this is nice and abstract but it’s still hard to grasp what this all means and what the benefits are of going through all this trouble. But there is an easy way to understand what a Hypermedia API is, by using something we’re all familiar with and that you use every single day, the web itself.
Imagine I visit the homepage of your site. As a consumer of your site, at this point, the only piece of information i have and need is it’s URL. That is the only entry point needed to navigate to any page, or execute any possible action on the rest of your website.
As soon as I receive the HTML representation of the homepage, I can begin scanning it for any hyperlinks or form inputs. These are usually labelled in a such manner that I can understand their purpose and thus use them to further drive my interaction with the site. Each subsequent page I visit will in turn contain more links which I can follow.
They key thing is that as a user of any website, I only have to learn what a hyperlink or html form does once. After that I know they will all behave in the same manner across all websites, and I can use the ‘hints’ provided as to which actions I want to perform or where I would like to navigate on any particular website.
This is an extremely powerful concept is what has made the web into the success that it is. In essence, it’s at the heart of the its distributed nature. We don’t need to go look up some central repository of links that lists all the possible web addresses in the world. Instead, HTML documents contain these themselves.
This is old news and the basis of how the web work, and for years humans have been enjoying this benefit in web browsers. But why have we been neglecting clients to our APIs for so long? Most REST APIs client need to have a whole lot of knowledge about the API upfront before they can do anything useful. Usually that means having to know at which URI to find objects, or how to craft URIs, how to uniquely identify resources, which states resource can be in and how they flow from one to the next and which actions are permitted on resources.
It would be nice if we could use the same mechanism when dealing with RESTful APIs, and of course it is. It’s called HATEOAS (Hypermedia as the Engine of Application State) and despite its rather unfortunately named acronym, it’s a very powerful idea. From the wikipedia entry above:
“The principle is that a client interacts with a network application entirely through hypermedia provided dynamically by application servers. A REST client needs no prior knowledge about how to interact with any particular application or server beyond a generic understanding of hypermedia.”
“The HATEOAS constraint serves to decouple client and server in a way that allows the server to evolve functionality independently.”
Can you imagine a world where you don’t need to rewrite your client each time the API changes? Where the whole state of the application doesn’t need to be carefully mirrored in the client and server. Where the client never has to build URLs, and can discover API functionality automatically. With hypermedia this is possible.
Where to from here?
Despite the fact that a lot of these ideas are quite old, in many ways, we’re still at the frontier of Hypermedia.
One question I’ve had: How do you do this stuff easily in Rails/Sinatra? The answer is, you can’t (yet). I’ve been playing around with Webmachine lately, which is a Ruby port of the Erlang framework with the same name. The framework and the need for it is well described in his excellent Rubyconf talk, but it basically provides a toolkit for building HTTP applications by proving a declarative way of describing your resources. I’m planning on building a small project and blogging about it, but I encourage you to check it out.
I’ve found a lot of inspiration and thoughts by reading some of Steve Klabniks writing, especially his post on why nobody understands REST or HTTP. Steve is also writing a book entitled
Get some REST (just renamed to Designing Hypermedia APIs), which I’m signed up for and very excited about.
Another great book I’ve read recently is Building Hypermedia APIs with HTML5 and Node. It’s a bit dry, but it gives the subject a thorough treatment. It dives deep into the building blocks of Hypermedia APIs, and then proceeds to build three example APIs using XML, JSON and HTML5.
Here are some practical suggestions that you can start off with today to start making your REST APIs better:
- STOP USING IDs and start using URLs. Whenever you have a way to unique identify an resource it should be it’s URL. Not the database unique key or anything else.
- Stop treating your resources as domain object and start treating them as resources. Consider adding metadata the HTTP requests that describe their possible states, and links to their possible actions.
- Start designing your REST APIs from a client perspective.