Build APIs You Won't Hate: Everyone and their dog wants an API, so you should probably learn how to build them (2014)

9. Authentication

9.1 Introduction

Understanding authentication for an API can be one of the largest hurdles for many developers, partially because there are a lot of different methods, but mostly because none of them are anything like authentication in an average “web app”.

When building an admin dashboard, CMS, blog, etc it is widely accepted as standard behavior to use sessions with a data store such as cookies, Memcache, Redis, Mongo, or some SQL platform. Regardless of the data store, sessions are used so that - once logged in - the browser remembers who the user is. To login the user is presented with a form in HTML showing two fields: one for the username and/or email address of the user and password. Once the end-user closes the browser or is inactive for a certain period of time, they will be forgotten.

This is the standard way to handle logins for the vast majority of sites built with a server-side language, but it is not at all how you handle authentication for an API.

In this chapter we will look at some of the most popular authentication methods, and explain some pros and cons of each.

9.2 When is Authentication Useful?

Authentication allows APIs to track users, give endpoints user-context (“find all of my posts”), limit users access to various endpoints, filter data or even throttle and deactivate accounts. This is all very useful for many APIs, but some may never need to implement authentication.

Read-only APIs

If your API is entirely read-only and the data is not sensitive, then you can just make it available and not worry at all about authentication. This is perfectly acceptable.

There is the concern that people could be attacking your API with DDoS attacks (flooding your API with an unreasonable number of requests with malicious intent) and using some form of authentication would limit the vectors of attack. To get a response from the API they would need to be a valid user, and therefore the users account could be throttled or deactivated if malicious activity was detected.

This does not entirely negate DDoS attacks but it can help your API do less work, as the request will terminate much sooner if an invalid user is found. So if DDoS issues are still a concern with or without authentication, then using a self-improving firewall, or implementing other security barriers may well be the solution. Generally speaking having anyone spamming any of your servers is not ideal, so this may certainly be a stronger move than implementing authentication purely to avoid these attacks.

Either way you could quite easily release your API without authentication then implement piece-meal later on.

Internal APIs

If your API runs over a private network or is locked down with firewall rules and you do not require user-context for your API then you could probably skip authentication.

One concern with just leaving all the security up to the network is that if the network is breached then hackers would be able to do rather a lot of damage, but if hackers are “all up in your networks” then you probably have a lot of security issues already.

Keep it in mind.

9.3 Different Approaches to Authentication

Approach #1: Basic Authentication

The first approach that many developers go to is HTTP Basic, which is the most like the standard username/password approach they’ve grown to know and love, but instead implemented on the HTTP Request level and respected by the browser.

Here is what Wikipedia has to say:

HTTP Basic authentication (BA) implementation is the simplest technique for enforcing access controls to web resources because it doesn’t require cookies, session identifier and login pages. Rather, HTTP Basic authentication uses static, standard HTTP headers which means that no handshakes have to be done in anticipation.
– Source: Wikipedia

Pros

· Easy to implement

· Easy to understand

· Works in the browser and any other HTTP client

Cons

· Is ludicrously insecure over HTTP

· Is fairly insecure over HTTPS

· Passwords can be stored by the browser, meaning a honey-pot of user data is sitting around waiting to be gobbled up.

Browsers Storing Passwords

With Chrome not even protecting these plain-text passwords with a master password you really are leaving your users wide-open to attack if you let HTTP Basic be an option.

Elliott Kember publicly outed Chrome on this. The Guardian cared. Sir Tim Berners-Lee cared. Google didn’t.

More Plain-Text Woe

Another security issue with Basic authentication is that it is ludicrously insecure when running over HTTP.

In the example provided by Wikipedia a header will be placed in the HTTP Request that looks like this:

Authorization: Basic QWxhZGRpbjpvcGVuIHNlc2FtZQ==

If a request is made that goes over the wire (such as a JS-based API request from a user sat in a coffee shop) then that request could easily be intercepted. Taking that header as an example, it is insanely simple to find the username and password.

1 $ php -a

2 php > echo base64_decode('QWxhZGRpbjpvcGVuIHNlc2FtZQ==');

3 Aladdin:open sesame

This is no more or less secure than a HTML login form, but is certainly not secure enough for any API with sensitive data.

Using SSL improves the concerns greatly, but as the password is sent in every single HTTP Request there is still the potential for cracking it - but at this point somebody has to really want to get in.

HTTP Basic Auth may be a good fit for a relatively unimportant internal API, which needs some basic protection and needs to be implemented quickly, but certainly is not any good for anything that handles money, air traffic or nuclear weapons.

Approach #2: Digest Authentication

Digest is a similar approach to authentication as Basic, but is designed to improve on the security concerns.

Instead of transmitting passwords in plain-text, it will calculate a MD5 hash and send that. Unlike the Base64-based passwords used in the basic auth, MD5 is a one-way hash - meaning you cannot simply take the hash and calculate the original password without trying out a lot of different combinations.

HA1 = MD5(A1) = MD5(username:realm:password) HA2 = MD5(A2) = MD5(method:digestURI) response = MD5(HA1:nonce:HA2)

The nonce is a unique number, which can contain (but should not be only) a timestamp. This helps to avoid replay attacks as the same hash will not be usable later on.

Pros

· Password is not transmitted in plain text

· The use of nonce helps negate rainbow table attacks

· Generally speaking is more secure than basic auth

· Easier to implement than some approaches

Cons

· Harder than basic auth to implement well

· Easy to implement badly

· Still insecure over HTTP

· Just like basic auth, passwords can still be stored by the browser

· Uses MD5

MD5… 4… 3… 2… 1… HACKED

MD5 is well accepted by many people today to be extremely crackable in most scenarios. Digest authentication has not improved over time since its creation in 1993, and while the calculation process should help negate many of these issues, a lousy implementation of digest authentication will be open to some weird attack vectors that you don’t know about until after the fact.

Digest is more secure than basic certainly. It is great over SSL, definitely a good choice for an internal API if you have more time to spend implementing, but still requires the username and password to be sent repeatedly, meaning it is potentially hackable, if the hacker has enough encrypted requests available to process.

Approach #3: OAuth 1.0a

Not quite as popular these days, but OAuth 1.0a was a big player on the web-based authentication scene, used by services such as Dropbox, Flickr, Twitter, Google, LinkedIn and Tumblr. Since then most have moved over to OAuth 2 which we will discuss next. The two are very different beasts and should not be conflated.

OAuth provides a method for clients to access server resources on behalf of a resource owner (such as a different client or an end- user). It also provides a process for end-users to authorize third- party access to their server resources without sharing their credentials (typically, a username and password pair), using user- agent redirections.
– Source: Wikipedia

Previously we have looked at authentication technologies that were essentially “built into the browser” and were not particularly flexible in their usages. OAuth 1.0 was a great way for services such as social networks to implement web-based HTML login forms that looked the same as any other login form (were branded with logos, color schemes, etc), but could then send you back to the third-party website for all sorts of awesome integration purposes.

For example, when Twitter swapped from HTTP Basic integration to OAuth 1.0 it meant that instead of third-parties (iPhone apps, other websites, CMSs, whatever) asking end-users to enter their username and password (which would be saved somewhere in plain-text), the third-party could redirect the user to the Twitter website, get them to log in, have them come back to their service and save a special token, instead of saving a password. OAuth 1.0a called these tokens a “OAuth Token” and a “OAuth Token Secret”.

OAuth 1.0a was built to be very secure even when not running over SSL. That meant of course that it was incredibly complicated, having to set up signatures (of which there were a few different algorithms, including HMAC-SHA1 and RSA-SHA1 or just plaintext). That got a bit tricky when trying to write client code, as you had to make sure you supported the right signature and most of the PHP implementations out there (including my own old CodeIgniter Spark) didn’t support all of them.

An average OAuth 1.0a signed HTTP request would look a little something like this:

1 POST /moments/1/gift HTTP/1.1

2 Host: api.example.com

3 Authorization: OAuth realm="http://sp.example.com/",

4 oauth_consumer_key="0685bd9184jfhq22",

5 oauth_token="ad180jjd733klru7",

6 oauth_signature_method="HMAC-SHA1",

7 oauth_signature="wOJIO9A2W5mFwDgiDvZbTSMK%2FPY%3D",

8 oauth_timestamp="137131200",

9 oauth_nonce="4572616e48616d6d65724c61686176",

10 oauth_version="1.0"

11 Content-Type: application/json

12

13 { "user_id" : 2 }

Ow.

Another complication was that there were different implementations. Two-legged (proper, and not proper) and three-legged. This is incredibly confusing, so I will let Mashape explain in the OAuth Bible: OAuth Flows.

There was also xAuth, which is still OAuth 1.a, which is designed for mobile and desktop applications that do not have easy access to a browser. It’s much easier for a web-application to spawn a popup with JavaScript or redirect a user than it is for a mobile app, so this was a handy way to get OAuth Token than the other implementations.

In the end, however you got the OAuth Token and Secret, you would place the OAuth Token in the request as a header, and use the secret to sign the signature, which would encrypt the request - making the whole thing nice and secure. If you can shove SSL on top of that then you’ve got yourself a very secure setup… except for the fact that tokens would stay the same once created, so over time their security could be compromised. Somebody could recover the data off of a laptop you sold to them on eBay, or a potential hacker could packet sniff enough traffic signed with your signature to eventually programmatically guess the token and secret.

Pros

· Super secure, even without SSL

· Does not send username/password in every request - plaintext or hashed

· Stops third-party applications wanted or storing your username and password

· An attacker gaining an OAuth Token and even a Secret should still never be able to change your password, meaning you should be safe from account hijack

Cons

· Rather complicated to interact with, even if you have a well built client library. PHP never really had one, but The League of Extraordinary Packages has recently built a decent one

· Limited number of ways to grant access. xAuth and Two/Three-legged flows ended up being rather restrictive

· Tokens never changed, so security was essentially just a matter of how long and how much you used the service

OAuth 1.0a would be a great technology to implement if you were building a website with a public user-based API… and you were building it in 2009-2010. Now, probably not.

Approach #4: OAuth 2.0

OAuth 2 dropped the “secret token” so users are simply getting an “Access Token” now, and dropped signature encryption. This was seen by many as a massive step backwards in security, but it was actually rather a wise move. The OAuth 1.0a spec made SSL optional, but OAuth 2.0 requires it. Relying on SSL to handle the encryption of the request is just logical, and drastically improves the implementation.

Even a basic GET request in OAuth 1.0a was horrendous, as you’d always need to set up your consumers, signatures, etc, but with OAuth 2.0 you can simply do this:

1 file_get_contents('https://graph.facebook.com/me?access_token=DFGJKHDFGHDIFHGFKDJGHIU');

Or, as we saw back in Chapter 3, you can usually pass Access Tokens to the server as a HTTP request header:

1 POST /moments/1/gift HTTP/1.1

2 Host: api.example.com

3 Authorization: Bearer vr5HmMkzlxKE70W1y4MibiJUusZwZC25NOVBEx3BD1

4 Content-Type: application/json

5

6 { "user_id" : 2 }

That looks a little easier to work with than OAuth 1.0a, right?

“Short”-life Tokens

As discussed OAuth 1.0a also used the same tokens essentially forever. OAuth 2.0’s access tokens will (can) expire after an arbitrary period of time, defined by the OAuth server. When you request an access token you will usually be provided with a “Refresh Token” and an expiry offset, which is the number of seconds until the token expires. Some servers send you a unix time at which it expires. Folks like to do things different for some reason, but if you know what to look out for it is not so bad.

Using the expire time you know when your access token will not be valid, so you can proactively create a cron job that refreshes the access tokens, or you can wrap your HTTP requests in an exception handler that looks for a “Not Authorized” error and refresh them then as the OAuth 2.0 spec recommends.

This extra “access tokens expire and you have to refresh them” step initially seems confusing and annoying, especially when you are used to “once I have this token it works forever” but its much more secure. OAuth 1.0a stopped you handing out your username and password by essentially giving you another username and password (the token and the secret) which worked for one specific client. Any good network admin will tell you that you should regularly change your password (at least every month), and OAuth is no different, as the more you use the same password/token the greater your chance of somebody finding out what it is.

Grant Types

One further massive benefit OAuth 2.0 provides over OAuth 1.0a is the ability to have multiple (even custom) grant types. Grant types are essentially a “mode” in which the OAuth 2.0 server will run, expecting different inputs and maybe providing different outputs. With this flexibility you can create some amazing implementations.

The most common OAuth 2.0 Grant Type that a user will be familiar with is authorization_code, which is a very OAuth 1.0a-like flow.

A client web-app creates a link to the OAuth Server of the service they would like to log into (e.g: Facebook) and the user logs in. Facebook redirects the user back to the client web-app’s “Callback URL”, with a ?code=FOO variable in the query string. The web-app then takes that code and makes a second request (usually a POST, but sometimes a GET depending on which popular API you look at…) to Facebook and Facebook then offers up an access token in the response. Some other popular APIs - like Google Apps - then provide expires and a refresh token too.

This is just one approach, and there are more. Due to this flexibility OAuth 2.0 is good for pretty much any scenario when authenticating an API, be it a basic username password login on a single-page JavaScript app, a CRON job that has no database access or a full-blown user-redirect flow between different websites, the flexibility of custom grant-types allows absolutely anything to be done.

More on this in the “Understanding OAuth 2.0 Grant Types” section below.

Erin Hammer

Often I am asked why anyone would still use OAuth 2.0 after Erin Hammer (lead author and editor of the OAuth 2.0 standard) withdrew his name from the specification. It certainly sent a ripple through the Internet but I personally disagree wholeheartedly with the issues he raises.

1. OAuth 2.0 is less secure if you do not use SSL/TSL. Correct. So use them.

2. People have implemented OAuth 2.0 badly (looking at you Facebook/Google/most providers), but when implemented well it is lovely. Use a pre-built standard compliant implementation, like this one for PHP.

3. He thinks Refresh Tokens are annoying, but I think they are great.

Generally speaking his departure from the project is no major loss. I’m sure the IETF are bike-shedding hard, but after using both for years I am much happier with OAuth 2.0 and really wish Twitter would get on with a full upgrade so I never have to use OAuth 1.0a again.

Generally speaking OAuth 2.0 is a good fit for most situations, providing that you use SSL and implement a well tested existing solution for your OAuth 2.0 Server. Trying to do this yourself can be incredibly hard and may well lead to you getting super-hacked. Even Facebook have trouble with this to this day because they rolled their own solution based on a really early draft of the specification.

Other Approaches

· OpenID - https://openid.net/

· Hawk - https://github.com/hueniverse/hawk

· Oz - https://github.com/hueniverse/oz

9.4 Implementing an OAuth 2.0 Server

Implementation by hand of a OAuth 2.0 server (or any of these authentication methods for that matter) can be very difficult. This chapter aimed to explain the pros, cons and use-cases for each, and implementation is sadly out of its scope. Here are a few existing implementations that you could look into using.

PHP Implementations

One implementation stands out above the rest in PHP-land, and not just because it is written by a friend of mine: Alex Bilbie. We originally became friends because not only is he an all-round good guy, but he has studied both OAuth specs religiously, and built some great tools for them over the years which I have used many times.

In his last job he worked at University of Lincoln, using OAuth for all sorts of cool things. He then received funding for a research project to build awesome open-source code for improving authentication and interoperability. That project resulted in a few great packages, including the PHP OAuth 2.0 Server, which now has a home with The League of Extraordinary Packages. It is the only PHP package to implement the entire OAuth 2.0 spec, so is very worth trying out.

OAuthello

Alex is writing a book covering the implementation of this server in great detail amongst all other OAuth things, so you should definitely pick up a copy. That URL contains a coupon code apisyouwonthate, which should knock a significant chunk off the price.

There is another PHP OAuth 2.0 server implementation which also probably works.

Python Implementations

There are two implementations for Python which look pretty good. One is oauth2lib which is a fork of pyoauth2. The original authors gave up, then the new ones had to rename it, or something.

Another is python-oauth2 which was developed by SimpleGeo, which was a great geo-location/place SaaS, but has since been bought out and shut down and was last committed to two years ago… so… maybe somebody needs to take that one over too.

Ruby Implementations

The only active and documented Ruby OAuth 2.0 Server I found was a Rack module named Rack::OAuth2::Server. It is well documented with examples of implementations in Rails, Sinatra and Padrino.

9.5 Where the OAuth 2.0 Server Lives

Many assume that the OAuth 2.0 server should be part of their API server. While it certainly could, it definitely does not need to.

An OAuth server usually has a web interface, which has HTML forms, form validation, and all sorts of static resources like images, CSS, JavaScript, etc. That makes it more fitting with a general website, so if your API and web-site are different servers then the OAuth server would be more suitably placed on the website.

Generally speaking it is better to keep all of these things autonomous, as if you decide to build a new version of your website in AngularJS instead of server-side code then it would be a pain to have to switch your OAuth server implementation too. If the OAuth server is on its own server, or at very least its own code-base, then you do not have this concern.

The only thing your API needs to do is look for an Access Token (as a header or query string parameter) then hit whichever datastore (SQL database, Mongo, etc) that contains the access tokens. Check it is valid (in the DB and not expired) then grab whichever user is tied to it, and pull that record for use throughout the API code.

None of that is complicated, so trying to tie the API server and OAuth server together in the same application code-base out of some misplaced perception of belonging is just not required.

9.6 Understanding OAuth 2.0 Grant Types

The four grant types discussed in the specification are:

Authorization Code

Authorization Code is the full user-flow with redirects discussed earlier in the chapter.

This is most useful if you have multiple sites (like a network of sites for games, movies, books, etc) or just want to share logins with other partners. This is also the grant type you will most likely use to log users into Facebook or Google.

Section 4.1 in the spec

Refresh Token

Refresh Tokens are supported by most popular OAuth 2.0 providers. Basically, you notice that your old access token does not work anymore when you receive a HTTP 401 status code, so you request a new one using your refresh token. The OAuth 2.0 server will then either give you a new access token, or the server will refuse. At that point you will have to send your user an email saying “Your account is no longer connected to Example.com, please click here to reconnect.” This is not common, and usually means that the user has disconnected access for that account anyway so a manual request is literally the only option.

This sounds like a bit of a run around, but it is quite simple and has a few advantages.

Basically, using the same Access Token over and over again forever then there is a fairly strong chance of somebody finding it. There are an array of reasons for this, from the the site not implementing SSL, the site getting hacked, the sys admins accidentally exposing some of their access logs, or - more likely - the Access Token is stored in the browser.

Storing the access token in the browser is fine if the access token is going to expire soon, as it means the hacker has a very short window of opportunity to do anything if they find it. If they get the current access token then fine, but if there is a 5 minute expiry then getting that token would be much more difficult, and probably require the hacker to be physically on the device you were using, or SSHing in - at which point you have much greater concerns.

Not all APIs will expire their access tokens, so some do live forever. Normally they either last forever, or they will give you an expiry time and expect you to refresh them. One exception to that is Facebook, who do neither. Facebook’s whole approach is that they want you to be forced to send a user back to facebook.com on a login.

It is frustrating that once again Facebook have decided to flagrantly disregard the OAuth 2.0 spec to suit their own needs, hurting the user flow and confusing developers in the process. Working with these popular APIs you will notice a lot of things like this which wind you up, but the differences are much less problematic then if they were not even slightly OAuth 2.0 based. At least they have some common ground.

Section 6 in the spec

Client Credentials

Client credentials can be useful for saying:

I am an application, you know that I am an application because here are my client_id and client_secret values. Let me in now please.

This is useful for CRON jobs, worker processes, daemons or any other sort of background process. The application will not have any context of a “user”, but it will be able to interact with your API. They have an access token which they will keep on using, and if it happens to expire then the background process will know how to refresh it.

Twitter - as mentioned - have been OAuth 1.0a only for years, but they added an OAuth 2.0 endpoint which would accept client_credentials as the only grant type. Their documentation explains further.

This is handy for public crawling of tags or public tweets, but is not able to handle posting statuses or anything that relates to a user. This is a handy compromise for now, and hopefully is a sign that they intend to roll out support for more grant types in the future.

Section 2.3.1 in the spec

Password (user credentials)

User Credentials are possibly the easiest way to get an access token for a user. It skips the whole redirect-flow that “Authentication Code” provides, and the “user peace-of-mind” that comes with it, but does offer simplicity. If Twitter had offered User Credentials OAuth 2.0 login as a replacement for HTTP Basic then the “Twitter Authpocolypse” a few years ago would have been far less drastic.

All you need to do is provide a username and password to the OAuth 2.0 server, and it gives you back an access token (and of course maybe a refresh token). Simple.

An example of this being extremely useful would be creating a single-page application with AngularJS/EmberJS/WhateverJS and wanting to provide a login. Clearly redirecting users around would be unnecessary because they are already on “your site”, and the login box can be styled however you like already.

The trouble is, if you try and do all of this in JavaScript code you run into a problem. You need to send the client_id and client_secret along with the username and password, but if you are using JavaScript then putting your client_secret into the JavaScript means it is readable in the browser…

HACKHACKHACK!

Do not do that.

It is easily avoidable, simply make a proxy script which will take a username and password as POST items, then pass them onto the OAuth 2.0 server with the client_id and client_secret too, which both probably come from some secret config file on the server.

Basic access token proxy script written in Python using Flask