PHP Web Services (2013)
Chapter 8. REST
REST stands for REpresentational State Transfer, and in contrast to protocols such as SOAP or XML-RPC, it is more a philosophy or a set of principles than a protocol in its own right. REST is a set of ideas about how data can be transferred elegantly, and although it’s not tied to HTTP, it is discussed here in the context of HTTP. REST takes great advantage of the features of HTTP, so the earlier chapters covering this and the more detailed topics of headers and verbs can all come together to support a good knowledge of REST.
In a RESTful service, four HTTP verbs are used to provide a basic set of CRUD (Create, Read, Update, Delete) functionality: POST, GET, PUT and DELETE. It is also possible to see implementations of other verbs in RESTful services, such as PATCH to allow partial update of a record, but the basic four provide the platform of a RESTful service.
The operations are applied to resources in a system. The “Representational State Transfer” name is accurate; RESTful services deal in transferring representations of resources. A representation might be JSON or XML, or indeed anything else. So what is a resource? Well, everything is. Each individual data record in a system is a resource. At the first stage of API design, a starting point could be to consider each database row as an individual resource. Think of an imaginary blogging system as an example: resources might be posts, categories, and authors. Every resource has a URI, which is the unique identifier for the record.
A collection contains multiple resources (of the same type); usually this is a list of resources or the result of a search operation. A blog example might have a collection of posts, and another collection of posts limited to a particular category.
RESTful URLs
RESTful services are often thought of as “pretty URL” services, but there’s more than prettiness to the structures used here. In Chapter 5, the GitHub API was used as an example of an API using JSON; it is also a nice example of a RESTful API belonging to a system that developers may already be familiar with. Take a look at some of the URLs in this API:
§ https://api.github.com/users/lornajane/
§ https://api.github.com/users/lornajane/repos
§ https://api.github.com/users/lornajane/gists
These delightful, descriptive URLs allow users to guess what will be found when visiting them, and to easily navigate around a predictable and clearly designed system. They describe what data will be found there, and what to expect. A key characteristic of RESTful URLs is that they only contain information about the resource or collection data—there are no verbs in these URLs. The best of API designs will have URLs that are “hackable”—that is to say that they are predictable enough to successfully guess where to find things. This links closely to the idea of hypermedia, which we’ll discuss shortly.
In order to alter how a collection is viewed (for example, to add filtering or sorting to it), it is common to add query parameters to the URL, like so:
§ http://api.joind.in/v2.1/events for all events
§ http://api.joind.in/v2.1/events?filter=past for events that happened before today
§ http://api.joind.in/v2.1/events?filter=cfp for events with a Call for Papers currently open
Notice that the URLs are not along the lines of /events/sortBy/Past or any other format that puts extra variables in the URL, but they use query variables instead. This data set, in both cases, still utilizes the /events/ collection, but sorted and/or filtered accordingly.
Resource Structure and Hypermedia
Exactly how the resource is returned can vary hugely; REST doesn’t dictate how to structure the representations sent. For example, a GitHub gist in JSON format looks like this:
{
"created_at": "2012-12-20T15:37:51Z",
"commits_url": "https://api.github.com/gists/4346013/commits",
"description": "Gist created by API",
"public": true,
"html_url": "https://gist.github.com/4346013",
"url": "https://api.github.com/gists/4346013",
"forks_url": "https://api.github.com/gists/4346013/forks",
"history": [
{
"change_status": {
"additions": 1,
"total": 1,
"deletions": 0
},
"committed_at": "2012-12-20T15:37:51Z",
"url": "https://api.github.com/gists/4346013/f85e23d3443d0547292b202c3cd48881a28ebe9a",
"version": "f85e23d3443d0547292b202c3cd48881a28ebe9a",
"user": {
"type": "User",
"organizations_url": "https://api.github.com/users/lornajane/orgs",
"gists_url": "https://api.github.com/users/lornajane/gists{/gist_id}",
"followers_url": "https://api.github.com/users/lornajane/followers",
"login": "lornajane",
"events_url": "https://api.github.com/users/lornajane/events{/privacy}",
"repos_url": "https://api.github.com/users/lornajane/repos",
"following_url": "https://api.github.com/users/lornajane/following",
"received_events_url": "https://api.github.com/users/lornajane/received_events",
"gravatar_id": "372a6ef44baaa7291f1a6698348d2e98",
"subscriptions_url": "https://api.github.com/users/lornajane/subscriptions",
"starred_url": "https://api.github.com/users/lornajane/starred{/owner}{/repo}",
"url": "https://api.github.com/users/lornajane",
"avatar_url": "https://secure.gravatar.com/avatar/372a6ef44baaa7291f1a6698348d2e98?d=https://a248.e.akamai.net/assets.github.com%2Fimages%2Fgravatars%2Fgravatar-user-420.png",
"id": 172607
}
}
],
"forks": [
],
"user": {
"type": "User",
"organizations_url": "https://api.github.com/users/lornajane/orgs",
"gists_url": "https://api.github.com/users/lornajane/gists{/gist_id}",
"followers_url": "https://api.github.com/users/lornajane/followers",
"login": "lornajane",
"events_url": "https://api.github.com/users/lornajane/events{/privacy}",
"repos_url": "https://api.github.com/users/lornajane/repos",
"following_url": "https://api.github.com/users/lornajane/following",
"received_events_url": "https://api.github.com/users/lornajane/received_events",
"gravatar_id": "372a6ef44baaa7291f1a6698348d2e98",
"subscriptions_url": "https://api.github.com/users/lornajane/subscriptions",
"starred_url": "https://api.github.com/users/lornajane/starred{/owner}{/repo}",
"url": "https://api.github.com/users/lornajane",
"avatar_url": "https://secure.gravatar.com/avatar/372a6ef44baaa7291f1a6698348d2e98?d=https://a248.e.akamai.net/assets.github.com%2Fimages%2Fgravatars%2Fgravatar-user-420.png",
"id": 172607
},
"updated_at": "2012-12-20T15:37:51Z",
"git_pull_url": "https://gist.github.com/4346013.git",
"comments": 0,
"id": "4346013",
"comments_url": "https://api.github.com/gists/4346013/comments",
"files": {
"text.txt": {
"type": "text/plain",
"filename": "text.txt",
"size": 18,
"language": null,
"raw_url": "https://gist.github.com/raw/4346013/336516c8e23e55265245bf589ae56aafa9cbbcf2/text.txt",
"content": "Some riveting text"
}
},
"git_push_url": "https://gist.github.com/4346013.git"
}
Whereas a talk from Joind.in, also in JSON, would look like this:
{
"talks": [
{
"talk_title": "Everything You Ever Wanted to Know About Deployment But Were Afraid to Ask",
"talk_description": "Deployment can be a real bugbear for many web developers. From building something easy to deploy and manage; to coming up with a repeatable, consistent process; to continuous deployment… deployment can keep you up at night for months on end. In this talk I’ll cover the following topics:\n- The deployment maturity model\n- How to build a deployable application, from technology choice to instrumentation\n- Deployment velocity: Why your process matters more than how often you deploy\n- Deployment tools and processes: How to automate your troubles away\n- CI/Automated testing: Know you’re deploying something good, or at least how worried you should be about it\n- Automated testing vs monitoring: How they converge\n- When are you ready to deploy continuously? How do you make the jump?",
"start_date": "2012-11-08T13:00:00-05:00",
"average_rating": 5,
"comments_enabled": 1,
"comment_count": 4,
"speakers": [
{
"speaker_name": "Laura Thomson",
"speaker_uri": "http://api.joind.in/v2.1/users/20041"
}
],
"tracks": [],
"uri": "http://api.joind.in/v2.1/talks/7660",
"verbose_uri": "http://api.joind.in/v2.1/talks/7660?verbose=yes",
"website_uri": "http://joind.in/talk/view/7660",
"comments_uri": "http://api.joind.in/v2.1/talks/7660/comments",
"verbose_comments_uri": "http://api.joind.in/v2.1/talks/7660/comments?verbose=yes",
"event_uri": "http://api.joind.in/v2.1/events/1056"
}
],
"meta": {
"count": 1,
"this_page": "http://api.joind.in/v2.1/talks/7660?start=0&resultsperpage=20"
}
}
The two formats are quite different, and in fact the fields and formats available in a RESTful service will differ between each and every kind of service you could wish to encounter, but there are some common features, as can be seen even from this small sample size. Both responses include some nested information and some links out to other resources or collections.
The links to other resources/collections are called hypermedia and are an excellent inclusion in RESTful services; since every resource is identified by its URI, this data can be given as part of the response data. In this way, consuming clients can follow links, rather like a user clicking links on the Web, instead of assembling the next URL from the instructions and concatenating ID fields into it. Hypermedia makes the whole experience smoother and easier for consumers by offering the ability to find their way around easily. For example, using the previous data set, the following actions are available:
1. Look at this resource, and then visit the comments_uri to see the comments made on this talk.
2. See more information about the event this talk belongs to by visiting the events_uri.
3. From there, follow another piece of hypermedia in the talks_uri field to see a list of other talks at the event.
Another consideration when designing and working with RESTful APIs is whether or not it is useful to send additional nested data with the response to avoid a consumer having to make too many calls to the server. While GitHub and Joind.in both offer user information at their own locations, they also include some nested data in the responses shown here, which the consumer is likely to need.
On the other hand, sometimes too much information can lead to unnecessarily large amounts of data to transfer, and different APIs handle this in different ways. One common pattern is that, by default, a subset of the information is returned, but functionality to retrieve more information is also offered—this is what the Joind.in verbose_uri offers. Alternatively, the extra information may be made available as a separate resource, such as offering /article/42 as the data about a blog post, but excluding the (potentially large) body of the post, which can then be found at/article/42/body. Either approach shows consideration to the consumer, but which one is the right fit will depend on any particular scenario.
Data and Media Types
A RESTful service can offer a selection of data types, and it’s very common to offer multiple types. Often, these will be JSON or XML, but there can be others; for example, Joind.in will respond to GET requests with an HTML data type if the Accept header requests it. The format decision will be made on the server, usually on the basis of the Accept header (you can read more about content negotiation in Headers for Content Negotiation).
Some services will allow a content indicator to be present in the URL itself, but this mixes up the identification of the resource with information about the representation desired. In general, the Accept header is the “right” way to indicate the preferred format, and a URL parameter may lower the barrier of entry, depending on your consumers.
Including a version number in your URL is a matter of taste. It is a very practical way to offer a service while identifying the current version of that service and opening the door to offering new versions of the service in the future. However, there are alternatives, and an elegant alternative is to use media types. These are invented content types that specifically describe the structure of the resource that will be returned, and can also include version information, so if the structure of a particular resource changes between versions, that change can be conveyed without a URL change.
Not all APIs will support media types, but they can be a good way to version representation structures for users who have a requirement to keep them predictable, and are happy to work with such specific content negotiations. GitHub does have some media type support (their reference pageexplains the detail very well) that goes beyond the usual application/json levels. They support media types specific to GitHub (application/vnd.github+json) and also support using the media type to specify the version of representation that should be returned (application/vnd.github.v3).
HTTP Features in REST
REST makes the most of HTTP’s best features, placing all the metadata about the request and response into the headers, and reserving the main body of the communications for the actual content. This means that a correctly-implemented RESTful service will make use of verbs, status codes, and headers so that all the extra information goes in the “envelope” of the request, and only the content is in the body. See Appendix A and Appendix B for tables of common status codes and headers. To look at how these various pieces go together, the next few sections take a walk through some examples of actual CRUD operations.
Create Resources
Resources are created by making a POST request to the collection to which the new resource will belong. The body of the request will contain a representation of the new resource, with the Content-Type header set appropriately so that the server will know how to understand it. When the resource has been successfully created, a successful status code will be included with the response.
It’s common to choose a status code of 201 (which means “Created”) when a new resource has been made, and to either return a representation of the new resource in the body, or to set a Location header, redirecting the consumer to the URI of the new record. It’s perfectly valid to return a 200 (“Accepted, but not completed”), however, and helpful to return a representation of the resource (appropriately formatted according to the Accept header) including information about the URI of this new item.
In the event that the resource cannot be created, an informative status code and error message should be returned to the user. There is more in-depth discussion of error handling appropriate status codes in Chapter 12. In general, a 400 “Bad request” or 406 “Not acceptable” status code would be appropriate for a request that either wasn’t understood, or didn’t pass validation rules. There are also a very large number of other status codes to choose from (see Appendix A), depending on what exactly went wrong.
An alternative approach to using POST on a collection to create a new resource is appropriate in the situation when the consumer, rather than the server, sets the identifier of the new record. In this scenario, the representation of the new resource can instead be sent in a PUT request directly to the new URI. Care must be taken, when designing a system like this, to ensure that multiple consumers do not pick the same URIs, either causing conflicts or overwrites. At least make sure that these are dealt with in a sane way, perhaps using the 409 status code, which means “Conflict.”
Read Records
To fetch representations of resources, use the GET verb applied to either a collection or an individual resource without sending any body content with the GET request. The resources will usually appear with exactly the same structure, regardless of whether they were requested within a collection or on their own. The status code will be 200 if the record(s) were successfully retrieved, although other “good” status codes may also be used here such as 302 “Found” or 304 “Not modified” (more about caching in the next section when we discuss how to update records).
If, however, the record isn’t successfully found, a status code describing the problem will be returned. In a vast number of cases, this will be a 404 status code, to indicate that the record wasn’t found or doesn’t exist. If the user isn’t authenticated, a 401 “Not Authorized” status code may be returned; a user who has identified herself but doesn’t have permissions to see this item may receive a 403 “Forbidden” instead. Any one of a number of other possible failure cases could also occur, and these should have the appropriate status codes associated with them.
If your API implements rate limiting, then it might be that the resource exists and the user has permission to see it, but she has exceeded her allotted number of requests in a given time frame. In this situation, either a 420 “Enhance your calm” or 429 “Too many requests” would be good statuses to return.
Some APIs (this includes GitHub) will return a 404 to indicate that the record exists but the requesting user does not have access to it. This makes it impossible to deduce the existence (or nonexistence) of a record without the rights to see it! Exposing such details is known as “leaking information” and in many settings it is something of which to be wary.
Update Records
To edit records RESTfully is a multistep process. First, the resource should be retrieved by GET. Then, the representation of the resource can be altered as needed, and that resource should be PUT back to its original URI. Even if only a small part of the record needs to be changed, REST deals with representations of resources so the whole resource will be fetched and sent back for the update. Identical to when a resource was created using POST, the PUT request will include the resource representation in the body and the appropriate Content-Type in the header.
It’s quite common to include some identifying information for the contents of the resource, such as a Last-Modified header or an ETag, to allow for checking of whether the resource changed as a result of something else between the GET and PUT, as this isn’t an atomic operation. This is closely linked to how cacheable different URIs are, which we’ll cover later in this chapter (see Caching Headers).
For a newcomer to REST, updating a representation of a whole resource can seem cumbersome when only a tiny part of it is actually changing, but don’t be tempted to diverge from this approach and break the RESTfulness of the design. If it really does seem like an alternative approach would be better, then you have two options: either create a sub-resource or use the PATCH verb.
Creating a sub-resource is simplest, if you want to change one field of a resource, and make that field available at its own URI. For example, if it seems like overkill to update a whole user record just to change an email address, then instead create a resource /user/42/email. This smaller resource can then be subject to GET, change, and PUT instead of fetching and then pushing back a whole user profile.
The alternative is to use PATCH to make a small change to an existing record. This isn’t commonly-supported (support for PATCH is relatively uncommon even in modern APIs, but is also uncommon at an infrastructure layer, so beware that not all networks will allow PATCH), but an example is available because once again, GitHub has it. GitHub allows the user to make changes to individual fields, in a record by supplying the data you want to change and making a PATCH request instead of a PUT request to the existing resource’s URI.
Delete Records
This is the most damaging move but it’s also the simplest. The DELETE verb is sent with a request to the URI of the item to be deleted, with no body content necessary. Many services will return 200 for “OK”—or simply a 204 for “No content”—when an item was successfully deleted, and a 404 “Not found” if the item didn’t exist. However, if the request was made to delete something, and the record doesn’t exist, many services see that as “success” and will return 200 or 204, regardless of what really happened (unless the record couldn’t be deleted for some reason, such as the user does not have the proper permission). This idea of always behaving in the same way each time the action is called is known as idempotency and is expected behavior for both GET and DELETE requests.
Additional Headers in RESTful Services
As seen in Chapter 3, it is possible to convey a wide selection of information using HTTP headers. There are a couple things that are relevant to the majority of RESTful services, which will now be examined in more detail: authorization and caching. These are both areas in which best practice for APIs differs very little from the best practices when building websites, but they bear revisiting in an API context.
Authorization Headers
A common header that has been seen earlier in this book is the Authorization header. This can be used with a variety of different techniques for authenticating users, all of which will be familiar to web developers.
The simplest approach to authorization is HTTP Basic authentication (for more detail, see the RFC), which requires the user to supply a username and password to identify himself. Since this approach is so widespread, it is well supported in most platforms, both client and server. Do beware, though, that these credentials can easily be inspected and reused maliciously, so this approach is appropriate only on trusted networks or over SSL. When the user tries to access a protected resource using basic authentication, he will receive a 401 status code in response, which includes aWWW-Authenticate header with the value Basic followed by a realm for which to authenticate. As users, we see an unstyled pop up for username and password in our browser; this is basic authentication. When we supply the credentials, the client will combine them in the formatusername:password and Base64 encode the result before including it in the Authorization header of the request it makes.
Similar to basic authentication, but rather more secure, is HTTP Digest authentication (the Wikipedia page includes a great explanation with examples). This process combines the username and password with the realm, a client nonce (a nonce is a cryptographic term meaning “Number Used Once”), a server nonce, and other information, and hashes them before sending. It may sound complicated to implement but this standard is well understood and widely implemented by both clients and servers.
Other applications may have alternative approaches, including using cookies and sessions to record a user’s information after he has supplied credentials to a login endpoint, for example. Others will implement solutions of their own making, and many of these will use a simple API keyapproach. In this approach, the user acquires a key, often via a web interface or other means, that she can use when accessing the API. A major advantage of this approach is that the keys can be deleted by either party, or can expire, removing the likelihood that they can be used with malicious intent. This is nicer than passing actual user credentials, as the details used can be changed. Sometimes API keys will be passed simply as a query parameter, but the Authorization header would also be an appropriate place for such information.
An even better solution has emerged in the last few years: OAuth (version 2 is much better than version 1). OAuth arises as a solution to a very specific and common problem: how do we allow a third party (such as an external application on a mobile device) to have secure access to a user’s data? This problem is solved by establishing a three-way relationship, so that requests coming to the providing API from the third-party consumer have access to the user’s data, but do not impersonate that user. For every combination of application and user, the external application will send the user to the providing API to confirm that she wants access to be granted. Once the relationship is established, the user can, at any time, visit the providing API (with which she originally had the relationship of trust) to revoke that access. Newer versions of OAuth are simple to implement but once again should always be used over SSL.
Caching Headers
Issues of caching are not specific to REST, or even to APIs, but they can help enormously when an API server needs to handle a lot of traffic. Requests that perform actions cannot be cached, as they must be processed by the server each time, but GET requests certainly can be, in the right situation. Caching can either be done by the server, which makes a decision about whether to serve a previous version of a resource, or by clients storing the result of previous requests and allowing us to compare versions.
Giving version information along with a resource is a key ingredient in client-side caching, and also links with the non-atomic update procedures in REST as was mentioned in Update Records. When returning a resource, either an ETag (usually a hash of the representation itself) or a Last-Modified (the date this record last changed) is included with the response. Clients that understand these systems can then store these responses locally, and when making the same request again at a later point, they can tell us which version of a resource they already have. This is very similar to the way that web browsers cache assets such as stylesheets and images.
When a resource is served with an ETag header, some textual representation of the resource, perhaps a hash of the resource or a combination of file size and timestamp. When requesting the resource at a later date, the client can send an If-None-Match header with the value of the ETag in it. If the current version of the resource has a non-matching ETag, then the new resource will be returned with its ETag header. However if the ETag values do match, the server can simply respond with a 304 “Not modified” status code and an empty body, indicating to the client that it can use the version it already has without saving transferring the new version. This can help reduce server load and network bandwidth.
In exactly the same way, a resource that is sent with a Last-Modified header can be stored with that header information by the client. A subsequent request would then have an If-Modified-Since header, with the current Last-Modified value in it. The server compares the timestamp it receives with the last update to the resource, and again either serves the resource with new metadata, or with the much smaller 304 response.
RESTful versus Useful
REST is truly an elegant way to build services, and a nice way to work with data over HTTP. Not every application has requirements that are best met by a RESTful service, so don’t be tempted to make architectural decisions based on the current fashionable technologies. Standards are always an excellent thing to follow; they’ve been created by people who have implemented this several times and learned from their mistakes. That said, don’t be afraid to break the rules just as you would for any other architectural decision in software engineering. Many APIs are criticized because they are deemed “not RESTful.” While I recommend that you follow the strategies in this chapter, it’s acceptable for you to take inspiration from REST, rather than implementing it to the letter. Do make sure, though, that your API is still well documented, robust, and most of all: useful.