HTTP Content Negotiation - REST and the JAX-RS Standard - RESTful Java with JAX-RS 2.0 (2013)

RESTful Java with JAX-RS 2.0 (2013)

Part I. REST and the JAX-RS Standard

Chapter 9. HTTP Content Negotiation

Within any meaningfully sized organization or on the Internet, SOA (service-oriented architecture) applications need to be flexible enough to handle and integrate with a variety of clients and platforms. RESTful services have an advantage in this area because most programming languages can communicate with the HTTP protocol. This is not enough, though. Different clients need different formats in order to run efficiently. Java clients might like their data within an XML format. Ajax clients work a lot better with JSON. Ruby clients prefer YAML. Clients may also want internationalized data so that they can provide translated information to their English, Chinese, Japanese, Spanish, or French users. Finally, as our RESTful applications evolve, older clients need a clean way to interact with newer versions of our web services.

HTTP does have facilities to help with these types of integration problems. One of its most powerful features is a client’s capability to specify to a server how it would like its responses formatted. The client can negotiate the content type of the message body, how it is encoded, and even which human language it wants the data translated into. This protocol is called HTTP Content Negotiation, or conneg for short. In this chapter, I’ll explain how conneg works, how JAX-RS supports it, and most importantly how you can leverage this feature of HTTP within your RESTful web services.

Conneg Explained

The first part of HTTP Content Negotiation is that clients can request a specific media type they would like returned when querying a server for information. Clients set an Accept request header that is a comma-delimited list of preferred formats. For example:

GET http://example.com/stuff

Accept: application/xml, application/json

In this example request, the client is asking the server for /stuff formatted in either XML or JSON. If the server is unable to provide the desired format, it will respond with a status code of 406, “Not Acceptable.” Otherwise, the server chooses one of the media types and sends a response in that format back to the client.

Wildcards and media type properties can also be used within the Accept header listing. For example:

GET http://example.com/stuff

Accept: text/*, text/html;level=1

The text/* media type means any text format.

Preference Ordering

The protocol also has both implicit and explicit rules for choosing a media type to respond with. The implicit rule is that more specific media types take precedence over less specific ones. Take this example:

GET http://example.com/stuff

Accept: text/*, text/html;level=1, */*, application/xml

The server assumes that the client always wants a concrete media type over a wildcard one, so the server would interpret the client preference as follows:

1. text/html;level=1

2. application/xml

3. text/*

4. */*

The text/html;level=1 type would come first because it is the most specific. The application/xml type would come next because it does not have any MIME type properties like text/html;level=1 does. After this would come the wildcard types, with text/* coming first because it is obviously more concrete than the match-all qualifier */*.

Clients can also be more specific on their preferences by using the q MIME type property. This property is a numeric value between 0.0 and 1.0, with 1.0 being the most preferred. For example:

GET http://example.com/stuff

Accept: text/*;q=0.9, */*;q=0.1, audio/mpeg, application/xml;q=0.5

If no q qualifier is given, then a value of 1.0 must be assumed. So, in our example request, the preference order is as follows:

1. audio/mpeg

2. text/*

3. application/xml

4. */*

The audio/mpeg type is chosen first because it has an implicit qualifier of 1.0. Text types come next, as text/* has a qualifier of 0.9. Even though application/xml is more specific, it has a lower preference value than text/*, so it follows in the third spot. If none of those types matches the formats the server can offer, anything can be passed back to the client.

Language Negotiation

HTTP Content Negotiation also has a simple protocol for negotiating the desired human language of the data sent back to the client. Clients use the Accept-Language header to specify which human language they would like to receive. For example:

GET http://example.com/stuff

Accept-Language: en-us, es, fr

Here, the client is asking for a response in English, Spanish, or French. The Accept-Language header uses a coded format. Two digits represent a language identified by the ISO-639 standard.[8] You can further specialize the code by following the two-character language code with an ISO-3166 two-character country code.[9] In the previous example, en-us represents US English.

The Accept-Language header also supports preference qualifiers:

GET http://example.com/stuff

Accept-Language: fr;q=1.0, es;q=1.0, en=0.1

Here, the client prefers French or Spanish, but would accept English as the default translation.

Clients and servers use the Content-Language header to specify the human language for message body translation.

Encoding Negotiation

Clients can also negotiate the encoding of a message body. To save on network bandwidth, encodings are generally used to compress messages before they are sent. The most common algorithm for encoding is GZIP compression. Clients use the Accept-Encoding header to specify which encodings they support. For example:

GET http://example.com/stuff

Accept-Encoding: gzip, deflate

Here, the client is saying that it wants its response either compressed using GZIP or uncompressed (deflate).

The Accept-Encoding header also supports preference qualifiers:

GET http://example.com/stuff

Accept-Encoding: gzip;q=1.0, compress;0.5; deflate;q=0.1

Here, gzip is desired first, then compress, followed by deflate. In practice, clients use the Accept-Encoding header to tell the server which encoding formats they support, and they really don’t care which one the server uses.

When a client or server encodes a message body, it must set the Content-Encoding header. This tells the receiver which encoding was used.

JAX-RS and Conneg

The JAX-RS specification has a few facilities that help you manage conneg. It does method dispatching based on Accept header values. It allows you to view this content information directly. It also has complex negotiation APIs that allow you to deal with multiple decision points. Let’s look into each of these.

Method Dispatching

In previous chapters, we saw how the @Produces annotation denotes which media type a JAX-RS method should respond with. JAX-RS also uses this information to dispatch requests to the appropriate Java method. It matches the preferred media types listed in the Accept header of the incoming request to the metadata specified in @Produces annotations. Let’s look at a simple example:

@Path("/customers")

public class CustomerResource {

@GET

@Path("{id}")

@Produces("application/xml")

public Customer getCustomerXml(@PathParam("id") int id) {...}

@GET

@Path("{id}")

@Produces("text/plain")

public String getCustomerText(@PathParam("id") int id) {...}

@GET

@Path("{id}")

@Produces("application/json")

public Customer getCustomerJson(@PathParam("id") int id) {...}

}

Here, we have three methods that all service the same URI but produce different data formats. JAX-RS can pick one of these methods based on what is in the Accept header. For example, let’s say a client made this request:

GET http://example.com/customers/1

Accept: application/json;q=1.0, application/xml;q=0.5

The JAX-RS provider would dispatch this request to the getCustomerJson() method.

Leveraging Conneg with JAXB

In Chapter 6, I showed you how to use JAXB annotations to map Java objects to and from XML and JSON. If you leverage JAX-RS integration with conneg, you can implement one Java method that can service both formats. This can save you from writing a whole lot of boilerplate code:

@Path("/service")

public class MyService {

@GET

@Produces({"application/xml", "application/json"})

public Customer getCustomer(@PathParam("id") int id) {...}

}

In this example, our getCustomer() method produces either XML or JSON, as denoted by the @Produces annotation applied to it. The returned object is an instance of a Java class, Customer, which is annotated with JAXB annotations. Since most JAX-RS implementations support using JAXB to convert to XML or JSON, the information contained within our Accept header can pick which MessageBodyWriter to use to marshal the returned Java object.

Complex Negotiation

Sometimes simple matching of the Accept header with a JAX-RS method’s @Produces annotation is not enough. Different JAX-RS methods that service the same URI may be able to deal with different sets of media types, languages, and encodings. Unfortunately, JAX-RS does not have the notion of either an @ProduceLanguages or @ProduceEncodings annotation. Instead, you must code this yourself by looking at header values directly or by using the JAX-RS API for managing complex conneg. Let’s look at both.

Viewing Accept headers

In Chapter 5, you were introduced to javax.ws.rs.core.HttpHeaders, the JAX-RS utility interface. This interface contains some preprocessed conneg information about the incoming HTTP request:

public interface HttpHeaders {

public List<MediaType> getAcceptableMediaTypes();

public List<Locale> getAcceptableLanguages();

...

}

The getAcceptableMediaTypes() method contains a list of media types defined in the HTTP request’s Accept header. It is preparsed and represented as a javax.ws.rs.core.MediaType. The returned list is also sorted based on the “q” values (explicit or implicit) of the preferred media types, with the most desired listed first.

The getAcceptableLanguages() method processes the HTTP request’s Accept-Language header. It is preparsed and represented as a list of java.util.Locale objects. As with getAcceptableMediaTypes(), the returned list is sorted based on the “q” values of the preferred languages, with the most desired listed first.

You inject a reference to HttpHeaders using the @javax.ws.rs.core.Context annotation. Here’s how your code might look:

@Path("/myservice")

public class MyService {

@GET

public Response get(@Context HttpHeaders headers) {

MediaType type = headers.getAcceptableMediaTypes().get(0);

Locale language = headers.getAcceptableLanguages().get(0);

Object responseObject = ...;

Response.ResponseBuilder builder = Response.ok(responseObject, type);

builder.language(language);

return builder.build();

}

}

Here, we create a Response with the ResponseBuilder interface, using the desired media type and language pulled directly from the HttpHeaders injected object.

Variant processing

JAX-RS also has an API to dealwith situations in which you have multiple sets of media types, languages, and encodings you have to match against. You can use the interface javax.ws.rs.core.Request and the class javax.ws.rs.core.Variant to perform these complex mappings. Let’s look at the Variant class first:

package javax.ws.rs.core.Variant

public class Variant {

public Variant(MediaType mediaType, Locale language, String encoding) {...}

public Locale getLanguage() {...}

public MediaType getMediaType() {...}

public String getEncoding() {...}

}

The Variant class is a simple structure that contains one media type, one language, and one encoding. It represents a single set that your JAX-RS resource method supports. You build a list of these objects to interact with the Request interface:

package javax.ws.rs.core.Request

public interface Request {

Variant selectVariant(List<Variant> variants) throws IllegalArgumentException;

...

}

The selectVariant() method takes in a list of Variant objects that your JAX-RS method supports. It examines the Accept, Accept-Language, and Accept-Encoding headers of the incoming HTTP request and compares them to the Variant list you provide to it. It picks the variant that best matches the request. More explicit instances are chosen before less explicit ones. The method will return null if none of the listed variants matches the incoming accept headers. Here’s an example of using this API:

@Path("/myservice")

public class MyService {

@GET

Response getSomething(@Context Request request) {

List<Variant> variants = new ArrayList<Variant>();

variants.add(new Variant(

MediaType.APPLICATION_XML_TYPE,

"en", "deflate"));

variants.add(new Variant(

MediaType.APPLICATION_XML_TYPE,

"es", "deflate"));

variants.add(new Variant(

MediaType.APPLICATION_JSON_TYPE,

"en", "deflate"));

variants.add(new Variant(

MediaType.APPLICATION_JSON_TYPE,

"es", "deflate"));

variants.add(new Variant(

MediaType.APPLICATION_XML_TYPE,

"en", "gzip"));

variants.add(new Variant(

MediaType.APPLICATION_XML_TYPE,

"es", "gzip"));

variants.add(new Variant(

MediaType.APPLICATION_JSON_TYPE,

"en", "gzip"));

variants.add(new Variant(

MediaType.APPLICATION_JSON_TYPE,

"es", "gzip"));

// Pick the variant

Variant v = request.selectVariant(variants);

Object entity = ...; // get the object you want to return

ResponseBuilder builder = Response.ok(entity);

builder.type(v.getMediaType())

.language(v.getLanguage())

.header("Content-Encoding", v.getEncoding());

return builder.build();

}

That’s a lot of code to say that the getSomething() JAX-RS method supports XML, JSON, English, Spanish, deflated, and GZIP encodings. You’re almost better off not using the selectVariant() API and doing the selection manually. Luckily, JAX-RS offers thejavax.ws.rs.core.Variant.VariantListBuilder class to make writing these complex selections easier:

public static abstract class VariantListBuilder {

public static VariantListBuilder newInstance() {...}

public abstract VariantListBuilder mediaTypes(MediaType... mediaTypes);

public abstract VariantListBuilder languages(Locale... languages);

public abstract VariantListBuilder encodings(String... encodings);

public abstract List<Variant> build();

public abstract VariantListBuilder add();

}

The VariantListBuilder class allows you to add a series of media types, languages, and encodings to it. It will then automatically create a list of variants that contains every possible combination of these objects. Let’s rewrite our previous example using a VariantListBuilder:

@Path("/myservice")

public class MyService {

@GET

Response getSomething(@Context Request request) {

Variant.VariantListBuilder vb = Variant.VariantListBuilder.newInstance();

vb.mediaTypes(MediaType.APPLICATION_XML_TYPE,

MediaType.APPLICATION_JSON_TYPE)

.languages(new Locale("en"), new Locale("es"))

.encodings("deflate", "gzip").add();

List<Variant> variants = vb.build();

// Pick the variant

Variant v = request.selectVariant(variants);

Object entity = ...; // get the object you want to return

ResponseBuilder builder = Response.ok(entity);

builder.type(v.getMediaType())

.language(v.getLanguage())

.header("Content-Encoding", v.getEncoding());

return builder.build();

}

You interact with VariantListBuilder instances by calling the mediaTypes(), languages(), and encodings() methods. When you are done adding items, you invoke the build() method and it generates a Variant list containing all the possible combinations of items you built it with.

You might have the case where you want to build two or more different combinations of variants. The VariantListBuilder.add() method allows you to delimit and differentiate between the combinatorial sets you are trying to build. When invoked, it generates a Variant list internally based on the current set of items added to it. It also clears its builder state so that new things added to the builder do not combine with the original set of data. Let’s look at another example:

Variant.VariantListBuilder vb = Variant.VariantListBuilder.newInstance();

vb.mediaTypes(MediaType.APPLICATION_XML_TYPE,

MediaType.APPLICATION_JSON_TYPE)

.languages(new Locale("en"), new Locale("es"))

.encodings("deflate", "gzip")

.add()

.mediaTypes(MediaType.TEXT_PLAIN_TYPE)

.languages(new Locale("en"), new Locale("es"), new Locale("fr"))

.encodings("compress");

In this example, we want to add another set of variants that our JAX-RS method supports. Our JAX-RS resource method will now also support text/plain with English, Spanish, or French, but only the compress encoding. The add() method delineates between our original set and our new one.

You’re not going to find a lot of use for the Request.selectVariant() API in the real world. First of all, content encodings are not something you’re going to be able to easily work with in JAX-RS. If you wanted to deal with content encodings portably, you’d have to do all the streaming yourself. Most JAX-RS implementations have automatic support for encodings like GZIP anyway, and you don’t have to write any code for this.

Second, most JAX-RS services pick the response media type automatically based on the @Produces annotation and Accept header. I have never seen a case in which a given language is not supported for a particular media type. In most cases, you’re solely interested in the language desired by the client. You can obtain this information easily through the HttpHeaders.getAcceptableLanguages() method.

Negotiation by URI Patterns

Conneg is a powerful feature of HTTP. The problem is that some clients, specifically browsers, do not support it. For example, the Firefox browser hardcodes the Accept header it sends to the web server it connects to as follows:

text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8

If you wanted to view a JSON representation of a specific URI through your browser, you would not be able to if JSON is not one of the preferred formats that your browser is hardcoded to accept.

A common pattern to support such clients is to embed conneg information within the URI instead of passing it along within an Accept header. Two examples are:

/customers/en-US/xml/3323

/customers/3323.xml.en-US

The content information is embedded within separate paths of the URI or as filename suffixes. In these examples, the client is asking for XML translated into English. You could model this within your JAX-RS resource methods by creating simple path parameter patterns within your @Pathexpressions. For example:

@Path("/customers/{id}.{type}.{language}")

@GET

public Customer getCustomer(@PathParam("id") int id,

@PathParam("type") String type,

@PathParam("language") String language) {...}

Before the JAX-RS specification went final, a facility revolving around the filename suffix pattern was actually defined as part of the specification. Unfortunately, the expert group could not agree on the full semantics of the feature, so it was removed. Many JAX-RS implementations still support this feature, so I think it is important to go over how it works.

The way the specification worked and the way many JAX-RS implementations now work is that you define a mapping between file suffixes, media types, and languages. An xml suffix maps to application/xml. An en suffix maps to en-US. When a request comes in, the JAX-RS implementation extracts the suffix and uses that information as the conneg data instead of any incoming Accept or Accept-Language header. Consider this JAX-RS resource class:

@Path("/customers")

public class CustomerResource {

@GET

@Produces("application/xml")

public Customer getXml() {...}

@GET

@Produces("application/json")

public Customer getJson() {...}

}

For this CustomerService JAX-RS resource class, if a request of GET /customers.json came in, the JAX-RS implementation would extract the .json suffix and remove it from the request path. It would then look in its media type mappings for a media type that matched json. In this case, let’s say json mapped to application/json. It would use this information instead of the Accept header and dispatch this request to the getJson() method.

Leveraging Content Negotiation

Most of the examples so far in this chapter have used conneg simply to differentiate between well-known media types like XML and JSON. While this is very useful to help service different types of clients, it’s not the main purpose of conneg. Your web services will evolve over time. New features will be added. Expanded datasets will be offered. Data formats will change and evolve. How do you manage these changes? How can you manage older clients that can only work with older versions of your services? Modeling your application design around conneg can address a lot of these issues. Let’s discuss some of the design decisions you must make to leverage conneg when designing and building your applications.

Creating New Media Types

An important principle of REST is that the complexities of your resources are encapsulated within the data formats you are exchanging. While location information (URIs) and protocol methods remain fixed, data formats can evolve. This is a very important thing to remember and consider when you are planning how your web services are going to handle versioning.

Since complexity is confined to your data formats, clients can use media types to ask for different format versions. A common way to address this is to design your applications to define their own new media types. The convention is to combine a vnd prefix, the name of your new format, and a concrete media type suffix delimited by the “+” character. For example, let’s say the company Red Hat had a specific XML format for its customer database. The media type name might look like this:

application/vnd.rht.customers+xml

The vnd prefix stands for vendor. The rht string in this example represents Red Hat and, of course, the customers string represents our customer database format. We end it with +xml to let users know that the format is XML based. We could do the same with JSON as well:

application/vnd.rht.customers+json

Now that we have a base media type name for the Red Hat format, we can append versioning information to it so that older clients can still ask for older versions of the format:

application/vnd.rht.customers+xml;version=1.0

Here, we’ve kept the subtype name intact and used media type properties to specify version information. Specifying a version property within a custom media type is a common pattern to denote versioning information. As this customer data format evolves over time, we can bump the version number to support newer clients without breaking older ones.

Flexible Schemas

Using media types to version your web services and applications is a great way to mitigate and manage change as your web services and applications evolve over time. While embedding version information within the media type is extremely useful, it shouldn’t be the primary way you manage change. When defining the initial and newer versions of your data formats, you should pay special attention to backward compatibility.

Take , for instance, your initial schema should allow for extended or custom elements and attributes within each and every schema type in your data format definition. Here’s the initial definition of a customer data XML schema:

<schema targetNamespace="http://www.example.org/customer"

xmlns="http://www.w3.org/2001/XMLSchema">

<element name="customer" type="customerType"/>

<complexType name="customerType">

<attribute name="id" use="required" type="string"/>

<anyAttribute/>

<element name="first" type="string" minOccurs="1"/>

<element name="last" type="string" minOccurs="1"/>

<any/>

</complexType>

</schema>

In this example, the schema allows for adding any arbitrary attribute to the customer element. It also allows documents to contain any XML element in addition to the first and last elements. If new versions of the customer XML data format retain the initial data structure, clients that use the older version of the schema can still validate and process newer versions of the format as they receive them.

As the schema evolves, new attributes and elements can be added, but they should be made optional. For example:

<schema targetNamespace="http://www.example.org/customer"

xmlns="http://www.w3.org/2001/XMLSchema">

<element name="customer" type="customerType"/>

<complexType name="customerType">

<attribute name="id" use="required" type="string"/>

<anyAttribute/>

<element name="first" type="string" minOccurs="1"/>

<element name="last" type="string" minOccurs="1"/>

<element name="street" type="string" minOccurs="0"/>

<element name="city" type="string" minOccurs="0"/>

<element name="state" type="string" minOccurs="0"/>

<element name="zip" type="string" minOccurs="0"/>

<any/>

</complexType>

</schema>

Here, we have added the street, city, state, and zip elements to our schema, but have made them optional. This allows older clients to still PUT and POST older, yet valid, versions of the data format.

If you combine flexible, backward-compatible schemas with media type versions, you truly have an evolvable system of data formats. Clients that are version-aware can use the media type version scheme to request specific versions of your data formats. Clients that are not version-aware can still request and send the version of the format they understand.

Wrapping Up

In this chapter, you learned how HTTP Content Negotiation works and how you can write JAX-RS-based web services that take advantage of this feature. You saw how clients can provide a list of preferences for data format, language, and encoding. You also saw that JAX-RS has implicit and explicit ways for dealing with conneg. Finally, we discussed general architectural guidelines for modeling your data formats and defining your own media types. You can test-drive the code in this chapter by flipping to Chapter 23.


[8] For more information, see the w3 website.

[9] For more information, see the ISO website.