APIs

In response to: Amazon proves that REST doesn’t matter for Cloud APIs

A monkey from Spatially AdjustedI posted this in response to: Amazon proves that REST doesn’t matter for Cloud APIs. Some people really frustrate me. I know I’m a RESTafarian (so coined for people who have an unhealthy enjoyment of REST), but really, Amazon haven’t proven anything, other than being arrogant and disrespecting developers.

Anyway…

—-

It seems strange that someone with such a clear idea of what a RESTful web service is for and why it’s there makes such a ridiculous statement as ‘… REST doesn’t matter for Cloud APIs’.

At it’s simplest, REST is a way for communicating; a ruleset or a bunch of guidelines which, when adhered to, make being a consumer of these APIs much easier. I’ll address some points in the post:

Has this lack of REStfulness stopped anyone from using it? Has it limited the scale of systems deployed on AWS? Does it limit the flexibility of the Cloud offering and somehow force people to consume more resources than they need? Has it made the Amazon Cloud less secure? Has it restricted the scope of platforms and languages from which the API can be invoked? Does it require more experienced engineers than competing solutions?

1) Your set of questions, all of which answer ‘No’, is missing one and a pretty fundamental one at that. Does the fact that AWS use their own implementation of an API instead of a standard like, oh, I don’t know, REST, frustrate developers who really don’t want to have to learn another method of communicating with AWS? Yes.

Here’s a rule of thumb. If most invocations of your API come via libraries for object-oriented languages that more or less map each HTTP request to a method call, it probably doesn’t matter very much how RESTful your API is.

2) The rule of thumb: Who writes these libraries for the Object Oriented Languages you speak of? Oh, yes – developers. See Point 1.

The Rackspace people are technically right when they point out the benefits of their API compared to Amazon’s. But it’s a rounding error compared to the innovation, pragmatism and frequency of iteration that distinguishes the services provided by Amazon. It’s the content that matters.

3) If Rackspace are ‘technically’ right, then they’re right. There’s no grey area. Morally, they’re also right and mentally, physically and spiritually, they’re right.

If you think it’s rich, for someone who wrote a series of post examining “REST in practice for IT and Cloud management” (part 1part 2 and part 3), to now declare that REST doesn’t matter

4) Yes, it is rich.

As commenter John Leach says, it’s disrespectful to developers to implement and entirely different method of communicating with your API just because you’re amazon and you can do what you like.

Put it this way. If, every time you hired a car, the cockpit setup was the same; the pedals were in the same place, the wheel turned left and right and the speedo was in front of you, you’re comfortable and happy. Then, one day, you hire a car from Amazon Car Hire. The pedals are now handles and they’re all identical, the wheel is a yoke and the speedo is above your head. You’d be pretty pissed off. Especially if Amazon Car Hire were the only people you could hire a car from.

Not a perfect analogy, but the bottom line is, just because you’re amazon, doesn’t give you the right to buck the standard, regardless of how loose or ill-defined that standard might be.

What happens when Microsoft do it? Everyone hates IE8 and there furore.

Why do I even care what version your API is? Versioning your API with HATEOAS

Original image by zenera

I HATE OA(R)S. Original image by zenera

This is another in my unofficial series on stuff to do with REST. You can see the others by clicking here for API Anti-Patterns or here for Cookies and the RESTful API.

Notice what I did there? Think hard. I linked some other documents that you can GET. Click’em and you’ll:

GET: http://blog.mikepearce.net/2010/08/24/cookies-and-the-restful-api/

What I did there was provide you with two hypertext links, one to each of the other posts about REST that I’ve written. This is HATEOAS. You see,  you don’t actually need to know those two links above to be able to find them, you don’t even really need to know this link to THIS blog post, you only need to know my endpoint, which is http://blog.mikepearce.net. Navigate to there, and then you can get to each and every other page on my blog without having to know the URLs.

Even if you DID know my URLs (and by ‘know’ I mean bookmark, only the most diehard Mike Pearce fans memorise all my blog post URLs, dontcha Jon?) I might, one day, decide to change the schema, so they wouldn’t work. However, you’d get a 404 and a list of possible URLs, or I might even 301 redirect you if I was feeling nice (and this wasn’t hosted on wordpress.com). The thing about this hypermedia is that it’s so easy to get anywhere you like, the very definition of surfing.

CAN HAZ ENDPOINT. (Image by kwerfeldein)

So, now you’ve had a short tutorial on what HATEOAS means, Hypermedia As The Engine Of Application State, essentially means that your clients shouldn’t be building their own URIs. They should be requesting one of your endpoints and retrieving a list of URI with which they can interact with with one of the HTTP verbs (GET, PUT, DELETE, UPDATE, POST). The idea (which I illustrated above) is, that as an API provider, should you need to, you can change your URIs when you like and your clients will still know how to access all the resources of your API (if they’ve written their client to support that kind of thing). You could get away with one endpoint, the root of your API URI scheme. But it might be nice to provide one or two more which aren’t every going to change, perhaps:


http://api.yourcompany.com/users


http://api.yourcompany.com/links

Further, you can then provide your users with a list of properties that the API response will return at each of these endpoints, that links to the URIs they need to access to interact with your service. So, if the user were to:

GET: http://api.yourcompany.com/users

The response might be:

{
	"userMutateActions":
	{
		"add": "/users/add",
  		"update": "/users/updateUser",
  		"delete": "/users/deleteThatSucka"
  	},
  	"userReadOnlyActions"
  	{
  		"get.Details": "/users/$username",
  		"get.Email": "/users/$username/email",
  		"get.Dob": "/users/$username/dob
  	}
}

Now you’ve a contract with your clients that means they can visit any of the three endpoints and be able to navigate their way around the rest of your API. If you were a nice API provider you could provide examples of code for your clients to integrate into the software that interrogated your API for these endpoint URI indexes…

You see, I think that here it is a little fuzzy. The idea is that a client can and should be able to bookmark a URI it knows about and be able to call that URI and have your API still respond is one of the tenets of a REST API, but it’s at odds with HATEOAS.

Eggheads and Philosopers (Image by Dunechaser)

Eggheads and Philosopers (Image by Dunechaser)

Anyway, that’s for eggheads and philosophers. Use your best judgment to decide when and where to do this kind of thing, how many endpoints you have and what you provide as URIs to your clients.

I should really get to the point described in the title. But you’ve probably worked it out by now…

There are a couple of ways of versioning an API, check out slides 88 – 93 of my presentation in the API AntiPatterns presentation here, either on the URL, or in the body of a request or with primary and secondary URIs. The thing is, after I wrote that and while I spent a bit of time reading more about cookies, I decided that none of those methods of versioning were any good, the answer is…

… don’t actually bother with versioning. If you’re adhering to the HATEOAS contraint and you’ve created some good, solid endpoints, some documentation that creates contracts as to nomenclature of your URI properties with your clients, then you don’t need to version. Just change the URIs and the values of the URI properties at each endpoint and the client won’t even know you’ve done it.

Shhh, it’ll be our little secret!

Cookies and the RESTful API

Right, after my presentation at PHPLondon this month, the most contentious issue was that of using cookies with your REST API. I said, in no uncertain terms, that you shouldn’t do it. There were a few cries from the audience which were akin to the flapping you hear in a parliamentary broadcast, Derick Rethans didn’t agree but had the grace not to publicly embarrass me* and one comment on the original post requesting a clarification of my statement.

So, to clarify!

One of the most important constraints of REST is that it should be stateless, that is, every request made to API should contain everything the application needs in order to service the request. Now, at it’s most terse, that is my clarification, however, the quicker witted and cleverer among you will be proud to announce that a cookie is part of a HTTP request, and you’d be right, so, more clarification is needed.

In order to get to my decision that cookies shouldn’t be used with a REST API, you have to consider HATEOAS (Hypertext As The Engine Of Application State). What this means is that, a client only needs to know one, well published, end point to your entire API and from there, they can navigate the whole damn thing. Many people have blogged on this (including the big man himself), so I shan’t go into it right now, take a look at those links, but it’s safe to say that your clients shouldn’t be building their own URIs, they should be given the URIs from the API. Following this through to it’s logical conclusion means that the client shouldn’t be storing anything that isn’t the well published end point, because you, as the service provider, could change the requirement of whatever it is the client has stored as a cookie on a whim and then the client is banjaxed.

An example you say? OK. Let’s assume for a moment that we all agree that storing login information on the client is NOT GOOD. Right? (Consider XSS briefly and you’ll understand why – although less relevant nowadays, it’s still relevant (just)). OK, so the only other logical thing we might want to store in a cookie is an authentication token. That seems fairly harmless in it’s implementation, no? Yes it is and really, we could end the post here as, at a push, this is probably the one thing that IS OK to store in a cookie.

But I don’t sanction it.

Why? Because it doesn’t adhere to the constraints of HATEOAS and statelessness. Now the client has a cookie with an authentication token in it, you cannot change the way you authenticate, or the way the token is created without breaking the link between your API and each client that is using that token. If you publish a change to your REST API which handles the tokens differently, each and every cookie token will be useless.

Granted, this example doesn’t really offer anything that couldn’t be fixed with another login, but that’s not the point. The point is, it’s not RESTful.

To wit; this is a rule you can break and, probably, get away with it. However, you’ll feel dirty as it won’t be REST; try not to do it, make Roy happy.

*Although, he did heckle me on many other points. ;)

API Anti-Patterns (how NOT to write a RESTful API)

[Vimeo http://vimeo.com/13922981 w=640&h=385]

I had the honor of giving another talk at PHPLondon this month. Although I only had two weeks notice to research and write the thing, I think I managed to pull it off!

The talk was on API Anti-Patterns. I’d originally thought about doing a talk on How To Write a RESTful API, but the topic is enormous and sprawling and I only had 30 mins. So, I flipped the idea on it’s head and wrote about the things which we find with supposedly RESTful APIs which really aren’t RESTful. It’s shorter and, more importantly, funnier. So, below you’ll find the video and the slides from the night. Get in touch if you have any questions.

(more…)

The one about HTTP Request Methods and the RESTful API

Nobody Nose The Trouble I've Seen - BrittneyBush

Clearly a RESTful dog (photo by BrittneyBush on Flickr)

What HTTP Methods should I use for what API methods? A quesion which, until I began working on a RESTful API, I’d never needed to ask. I’ve been writing an ACL service which required an API. I thought “I don’t know how to write a RESTful API, this is a good opportunity to learn something you reprobate.” so, these are my collected musing on the subject, based on my recent experience.

I started with POSTing to URIs like /v1/adduser and /v1/getuser, which worked OK, it just didn’t feel right. I read through the O’Reilly book and decided I needed to up my game and make some changes.

You see, at first glance PUT and POST seem the same and, for a large part, they are, they RFC says:

The fundamental difference between the POST and PUT requests is reflected in the different meaning of the Request-URI. The URI in a POST request identifies the resource that will handle the enclosed entity. That resource might be a data-accepting process, a gateway to some other protocol, or a separate entity that accepts annotations. In contrast, the URI in a PUT request identifies the entity enclosed with the request — the user agent knows what URI is intended and the server MUST NOT attempt to apply the request to some other resource. If the server desires that the request be applied to a different URI,

Which is a but dull and dry, like beige. Anyway, when writing a RESTful API, you really want to adhere to the proper use of verbs and methods, it all seems a bit airy fairy until you’re given some context and some examples.

__construct()

Let’s say you’re writing an ACL (Access Control Layer) and the ACL needs to have an API which you can use to ask if a user has access to a resource, to add users, resources and roles, perform delete commands and modify stuff. You’ve also decided that the API should be RESTful as it’s a useful way to learn something new.

Your API is going to be located at: http://api.awes.ome/v1/ as it’s good to version APIs, but that’s for another post. For the rest of this article, I’ll be missing out the hostname and just using the URI, because that’s what REST is all about.

REST URI and methods looks like this:

HTTP_METHOD {version}/{resource}/{modifiers}
{body}

Don’t forget, you’ll only have a {body} depending on the HTTP_METHOD you use (GET, for example, doesn’t have a body), speaking of GET…

GET

So, you’ll need to have the concept of “users” (or, in whatever parlance you’d like, “people” or “persons” if you’re a bit PC or even “accounts”) these are a fixed resource. Something that will always be and can be counted as 1 or n. Now we get right down to it, a user is a  “resource”, possibly not in the context of your application, the ACL, but definately in the context of your API, “resource”‘s should have their own URL, like so:

GET /v1/user/mikepearce

What this URL is saying is this: From version one of the api, get me all the data about the user mikepearce. I’m just getting data, it’s read only and the only filter, or where clause of my “get me..” statement is the username. But it is also the resource, doing this:

GET /v1/user

Is pretty useless, what would it return? Not multiple users as it’s not plural. So, the real resource is /user/mikepearce

If you wanted to get multiple users, then “users” is also a resource (not the use of “IS” not “ARE”, you’re not interested in the ACTUAL users, but the concept of all of them or one of them, see? No? Well, you will; keep reading).

GET /v1/users

That will get me all the users, if I want a filter, I can add it on in multiple ways, again, for another post, but you could do it explicitly like this:

GET /v1/users/active/1/gender/male/newsletter/1

or implicitly, like this (your application should know what to do with each URI segment):

GET /v1/users/1/male/1

DELETE

Delete, this one is easy. If you’re deleting a resource (in this case a user) then you’d send this:

DELETE /v1/user/mikepearce
DELETE /v1/users/active/0/ (although, this is a little dangerous).

What is the difference between PUT and POST?

Ah now, here’s the rub! Essentially it boils down to this:

“The client uses PUT when it’s in charge of deciding which new URI the resource should have. The client uses POST when the server is in charge…” – O’Reillys RESTful web services

See? Easy, right. But that doesn’t quite cover everything. Let’s give you some context and then some more context:

PUT

You want to add users to your ACL, who wouldn’t? Afterall an ACL is useless without users, right? Easy and probably finished a long, long time ago. But I digress:

PUT /v1/users/bobmonkhouse

That will add a user called “bobmonkhouse” to your ACL. You’re “PUTting” something somewhere. Imagine that your service is like a shelf and you’ve got a shelf labelled “users” and your arm is the HTTP PUT method and … yeah, perhaps that analogy is a bit thin. Anyway, simply stated, you’re putting a resource at a location.

“Wait!” you cry, “How do I add email addresses and date of births and stuff to the user!?” you sit down in a huff with your arms crossed and eyeball me.

Well, you would still use put, y’see – PUT has a body, rather like POST does. So, if you wanted to PUT extra data along with the resource you’d do this:

PUT /v1/users/tedrogers
email=tedrogers@threetwoone.co.uk&
dob=1935-07-20&
friend=Dusty%20Bin&
password=j0kersw1ld
 
408 Request Timeout by Ape Lad at Flickr

PUT: LIKE A BASKETBALL IN THE FACE .. THE FACE! (image by Ape Lad)

I’ve URL encoded the body of the PUT, but it could be anything, some of you youngsters might even want to use JSON. From here, you can use the URI to create the resource and the body data to populate it.

Updating a resource, for example if a user wants to change their password, or their email address, is as simple as sending the same request as above. You’re PUTting data onto the server, or PUTing document changes to the server. You should, in all instances, send the entire document. So, even if the user was just changing his email address, you should  still send everything you sent with the original PUT to create the resource.

You don’t have to, but it’s the correct way to do things (also, it means you don’t have to check for the existence of all the fields and do if..else wrapped update statements, you can assume that all the fields are there (although that doesn’t mean you shouldn’t validate them!)).

PUT requests to a URI should be idempotent, that is, PUTing the same thing, multiple times should have exactly the same affect as sending it once. If you make the above PUT request three or four times, it won’t create three of four users called tedrogers, it’ll only create one, then, each subsequent PUT updates the resource, whereas a POST will create the user the first time, then, when you make the request again, depending on how you’ve set up your application will either give you an error stating the user exists, or an internal server error, or something entirely different.

POST

Having read the above, you’re possibly wondering, “What would I want to do with POST? I can do everything with I need with PUT”. Well, maybe you can, but probably you can’t.

As I mentioned above, POSTing the same thing multiple times will result in possible multiple copies of your resource, remember, you’re letting the server decide where to put this resource and if it’s something there CAN be multiples of, such as a blog post, comment or message board post, it will create multiples.

Another use for POST is when you’re authenticating a user on you ACL. As well as the ACL informing you of whether a user can access a resource, it’s also got an authenticate method, letting you send a username and password and receiving a response as to whether the user can log in, for example.

In this instance, you’d send a POST:

POST /v1/authenticate
username=jimbowen&
password=bulls3y3

/v1/authenticate/jimbowen isn’t really a resource, it could be, I guess, but just doesn’t feel like a resource. So, you send the username and password as part of the POST body.

“Why can’t I use GET? I’m not actually asking for anything to be saved or stored on the server.” you ask, having read the all the above and taken it in, go you! The problem with using this:

GET /v1/authenticate/username/jimbowen/password/bulls3y3/

Is that the password is now clearly visible in the URI and will be recorded in the server logs. This is NOT a “good thing”, this is a very bad thing. The content of the POST body isn’t recorded anywhere (unless you actually record it) so is ideal for this kind of request.

__destruct()

So, in conclusion, what have we learnt? GET, DELETE and PUT are the three things you need for adding, reading and deleting resources when the client has control over the resource URI and POST is used for when the server decides what to do with the request, whether to create a resource, or to return a value based on criteria you want to remain hidden.

There are, probably, other (better) uses for POST and please leave some extra examples in the comments for others. The above is really just my observations from creating a RESTful API for my ACL.

Making sure your API doesn’t suck

A Phalanger - a type of possum. Awesome.

I’ve recently had the displeasure of working with an API published by one of our service providers. I won’t name names as it’d be a little embarassing, but we provision a lot of something from then and then those somethings are used and the provider records their use. The API exposes both the ability to manage these somethings and the ability to report on their use.

There’s a great REST book by the lovely people at O’Reilly (it has a Phalanger on the cover) which describe the best practices for building a RESTful API. There’s a lot covered in the book and I won’t go into it all here. However, there are some lessons to be learned from the book and from using APIs in general about writing an API, even if it isn’t restful.

Here is a (non-exhaustive) list of things to do to make sure your API doesn’t suck:

  1. Don’t return HTTP 200 OK for EVERY response. If the query to the API fails, send an appropriate response code.
  2. If you are COMPLETELY unable to send different response codes to describe the outcome of the request, make sure you send SOMETHING to the client so they know the status.
  3. Make sure this SOMETHING is documented.
  4. Try and return JUST ONE type of response. If XML, then just send XML. (Bonus tip: Don’t send XML with a plain text header, it’ll annoy your clients).
  5. Don’t send mixed responses. For example; if your documentation states the client can request a CSV, send a CSV, don’t send the error as XML and the CSV as success. If it doesn’t work, send an appropriate HTTP response … oh, right…
  6. Document stuff.
  7. Don’t describe methods in the URL: http://apiurl.com/_functionName(param1, param2, paramn) is NOT RESTful (and is stretching the description of API).
  8. Use POST, GET, UPDATE and DELETE properly and in context.
  9. Send API auth credentials with HTTP auth, not in the URL. They can be read in the logs.
  10. Try and make sure that upstream errors are reported faithfully by your API. No point in the API returning OK when the request IT made upstream failed. There’s no confidence in Confidence-Free-Town

There’s probably more, but that’s all I’ve uncovered when working with client APIs. Avoid those 10 items and you’ll be well on your way to writing an API that doesn’t suck!