Designing Great Web APIs (2015)

Chapter 4. API Design Details

While API modeling is focused on mapping requirements to the API, the API design process maps the API model to HTTP, the language of the Web.

Transitioning from the model to design phase will require a variety of decisions. Some of these decisions will become obvious, while others may require some careful thought and planning. The more decisions you leave until the development phase, the more likely your API design will be compromised due to your delivery schedule.

Rather than making these decisions quickly during the development phase, we encourage you to spend sufficient time with the API design process to ensure that your API design is complete. This will help you focus on building a great web API and avoiding too many changes after your API has been released.

An HTTP Primer

As we move from the API design model to the details of how the web API will be realized using HTTP, it is important to review how HTTP works. If you are familiar with HTTP, feel free to skip this section.

HTTP Is Request/Response

HTTP is a request/response protocol. The HTTP client contacts a server and sends a request. The server processes the request and returns a response indicating a success or failure. It is important to note that HTTP is stateless, which means that every request must provide all of the details necessary to process the request on the server.

Uniform Resource Locators (URLs)

Uniform resource locators, or URLs, provide a address of where to locate a resource, such as a web page, image, or data, from an API. A URL is divided into the following parts:

Scheme

How we want to connect, (e.g., HTTP [unsecure] or HTTPS [secure]).

Hostname

The server to contact (e.g., api.example.com).

Port number

A number ranging from 0 to 65535 that identifies the process on the server where the request is to go (e.g., 443 [optional, defaults to 80 for HTTP and 443 for HTTPS]).

Path

The path to the resource being requested (e.g., /projects [default is /, which indicates the homepage]).

Query string

Contains data to be passed to the server. Starts with a question mark and contains name/value (e.g., foo=bar) pairs, using an ampersand as a separator between (e.g., ?page=1&per_page=10).

HTTP Verbs

HTTP request verbs indicate the type of action being requested from the URL. For APIs, they are often one of the following:

GET

Retrieve a collection or individual resource.

POST

Create a new resource or request custom action.

PUT

Update an existing resource or collection.

DELETE

Delete an existing resource or collection.

NOTE

There are more HTTP request verbs in the HTTP specification. This list contains the majority of HTTP verbs useful for a modern web API.

HTTP Requests

A client request is composed of the following parts:

Request verb

Informs the server about the type of request being made (e.g., retrieve, create, update, delete, etc.)

URL

The universal address of the information being requested

Request header

Information about the client and what is being requested in name:value format

Request body

The request details (may be empty)

The following is an example HTTP request with no request body:

GET http://www.oreilly.com/ HTTP/1.0

Proxy-Connection: Keep-Alive

User-Agent: Mozilla/5.0 [en] (X11; I; Linux 2.2.3 i686)

Host: oreilly.com

Accept: image/gif, image/x-xbitmap, image/jpeg, */*

Accept-Encoding: gzip

Accept-Language: en

Accept-Charset: iso-8859-1, *, utf-8

The following are examples of API requests:

GET /accounts

Retrieves all accounts in the accounts resource collection

GET /accounts/{id}

Retrieves a specific account by the given ID

POST /accounts

Creates a new acccount

PUT /accounts/{id}

Updates an account by the given ID

DELETE /accounts/{id}

Deletes an account by the given ID

HTTP Responses

A server response is composed of the following parts:

Server response code

A number indicating if the request was successful or not

Response header

Information about what happened in name:value format

Response body

Contains the response payload, often HTML, XML, JSON, or an image (may be empty)

The following code is an example HTTP request with an HTML response body:

HTTP/1.1 200 OK

Date: Tue, 26 May 2015 06:57:43 GMT

Content-Location: http://oreilly.com/index.html

Etag: "07db14afa76be1:1074"

Last-Modified: Sun, 24 May 2015 01:27:41 GMT

Content-Type: text/html

Server: Apache

<html>...</html>

Response codes are grouped into families, with the 2xx response codes indicating success, 4xx response codes indicating that the client failed to format the request properly, and 5xx response codes indicating a server error. The following are the most common server response codes used for web APIs:

200 OK

The request has succeeded.

201 Created

The request has been fulfilled and resulted in a new resource being created.

202 Accepted

The request has been accepted for processing, but the processing has not been completed.

204 No Content

The server has fulfilled the request but does not need to return a body. This is common for delete operations.

400 Bad Request

The request could not be understood by the server due to malformed syntax.

401 Unauthorized

The request requires user authentication.

403 Forbidden

The server understood the request, but is refusing to fulfill it.

404 Not Found

The server has not found anything matching the requested URI.

500 Internal Server Error

The server encountered an unexpected condition which prevented it from fulfilling the request.

Building Your Resource Ontology

An ontology is simply a classification of concepts. An API ontology captures the set of resources you will be offering, and their relationships to other resources. It is generally realized through your API’s URL structure.

If you modeled your API already, you likely have a list of candidate resources that will be part of it. If not, take some time and model your API to help you identify your resources. To build your ontology, begin by creating a list of the resources, placing them at the top of the URL structure (e.g., /projects and /tasks).

Defining URLs Through Relationships

Next, you will need to determine if your resources all belong at the top level, or if some of them should be nested under parent resources. To do this, we first need to understand the relationships between each of the resources.

Relationships between resources can be categorized into three types: independent, dependent, and associative. Those familiar with database design will recognize these relationship types and will quickly understand them. For those not familiar, the following list contains a description of each of the three types, with further information in Table 4-1:

Independent

Independent resources can exist stand alone without the other’s existence, but may reference each other. The URLs for both resources often exist at the top level.

Dependent

Dependent resources cannot exist without the existence of the parent resource. The URL for the dependent resource exists as a nested resource of its parent.

Associative

Associative resources have a relationship that contains or requires additional properties to describe it. Associative resources may be nested under one parent or may be placed as a top-level resource and treated as an independent resource.

In our project-management API example, we have to make a critical decision: whether tasks exist outside of a project or not. If they can, then both resources are independent and therefore both exist at the top level of the URL structure (e.g., /projects and /tasks). However, if tasks must belong to a project, then tasks are dependent on a project and must exist as a nested resource under the specific project instance (e.g., /projects/{id}/tasks).

Understanding and applying resource relationships is critical to a great API design. Weigh your resource URL designs carefully and understand the impact of your decisions.

Mapping Resource Lifecycles to HTTP Verbs

Once you determine your resource URL structure, you can then map your resource lifecycles to the necessary HTTP verb or verbs. We have four core HTTP verbs that we will focus on, though a few others exist when you need them for uncommon situations.

Your API model will provide insight into the lifecycle requirements of your resources. Review your model and notice the verbs you used for each resource. Some resources may require all verbs in our lifecycle: search, create, read, update, and delete. However, other resources may not require update or delete actions. Other resources may be read-only. Therefore, the requirements identified during the modeling phase will inform your API design.

As you model your API, you will notice a common pattern between the verbs you choose and the eventual resource lifecycle they require. Table 4-2 is a common mapping between verbs used in modeling and the verbs in HTTP.

While you may use different verbs during modeling, they will likely map to one of the common HTTP verbs. If they don’t, you may need to revisit the concept and see if it can be broken down into a resource with a specific lifecycle. Otherwise, you may need to consider a custom POST action on a particular resource instance (e.g., POST /projects/{id}/approve).

Mapping Response Codes

For each API endpoint you identified in the previous step, you will need to consider what response code(s) to return. While we hope that most responses will indicate a success, sometimes the client will fail to provide all of the correct details necessary to fulfill a request.

It is important to map both success and error codes in the design phase, as it will be part of our documentation delivered to developers consuming our API. It will also inform your team in the complexity of each API endpoint prior to development, to help with the estimation process (see Table 4-3).

Validating Design Through Documentation and Prototyping

As your API design starts to emerge, you should start the API documentation process. At this stage, you may not have all of the details of what your resource representations will look like—that is expected.

By documenting your API design early, it will encourage the team to focus on documentation throughout the development process. It will also encourage validation through feedback from internal or external developers by sharing your API design with them early rather than waiting until launch.

You can begin to document the high-level design using one of your favorite API definition formats, such as Swagger, RAML, Blueprint, or IO Docs. Each of these formats can convert your API design into beautiful interactive docs, one of our key characteristics of a great API design. As your resource structures begin to take shape, you can capture those additional details, along with examples to complete your documentation.

In addition to documentation, prototyping is another effective way to validate your API design. Prototypes come in two common forms: a static prototype and a working prototype.

A static prototype is just a method of returning resource representations in one or more formats, such as XML or JSON. The static prototype is either stored on the local filesystem or served via a web server.

Static prototypes allow developers to begin to integrate the search (i.e., GET collection) and read (i.e., GET a resource instance) portions of the lifecycle. However, the filesystem or web server cannot process POST, PUT, or DELETE requests, so static prototypes are limited. To get beyond this limitation, a working prototype is required.

Working prototypes offer more functionality than static prototypes, allowing any or all functionality to be delivered. The focus of a working prototype is to simplify more complex interactions, such as third-party integrations or connecting to existing SOAP-based services or legacy systems.

Working prototypes are not meant to be production-ready implementations, so they can take shortcuts or flatten complex data structures for ease of implementation. You can use the programming language and framework you plan to use for the production implementation or select something simple that provides the minimal functionality required.

Putting It All Together

APIs can offer several advantages for businesses, including faster innovation and increased revenue. Every member of the organization, including executives, product managers, and developers, must be willing to see APIs as an investment that will create business value.

A significant portion of your investment must be in the design of the API to provide a great developer experience. Only then will you produce APIs that developers will love.

About the Author

James Higginbotham is a seasoned API consultant with experience in architecting, building, and deploying APIs. He is also a speaker and trainer. As an API consultant, he enjoys helping businesses balance great API design and product needs. As a trainer, he enjoys equipping cross-functional teams to integrate their talents toward building first-class APIs for their product or enterprise systems.

Acknowledgments

I would first like to thank D. Keith Casey, Jr., whose generosity knows no bounds.

I would also like to thank Mike Amundsen for his feedback on an early draft of this book.

And finally, my sincere thanks to the many people that have directly and indirectly influenced this book through lively discussions, both online and offline.