Responsive Web Design, Part 2 (2015)
Responsive Images
Deployment Concerns
I know what you’re thinking — all of the above means a lot of work. Resizing and optimizing every image and then adding multiple resource URLs in the markup to match them can be very cumbersome.
Fortunately, that’s not the only way to provide your users with the best performance. One might even say that it’s a sure way to a) go mad and b) mess it up from time to time.
We humans are not very good at tedious, repetitive tasks. Computers, on the other hand, are great at just that. What you need to create is an automatic process that would make sure all your images are responsive and optimized, and let the computers take care of that from that point on.
Now, there’s no one-size-fits-all solution here, since what you need to automate your image pipeline varies according to your choice of back-end, the source of your images, and what your site even does.
But there are some rules to live by:
•Static assets should be optimized using a build process. There are many build frameworks out there, such as Grunt and Gulp. Many of them have rich ecosystems of plugins to enable you to add an image optimization step into your build process with very little work. Use them.
•User-uploaded images should be optimized as part of the upload process. Keeping the original around is probably a good idea, in case you’d one day want to reprocess them to bump up your image quality.
•Markup creation of srcset should be automatic. It can be part of a templating framework, a build-time solution, or part of your dynamic HTML generation, but adding multiple resource URLs to the HTML by hand is nobody’s idea of a good time. Automate it away.
•sizes still requires manual work. sizes is brand-new, and we don’t yet have the tools to automatically generate sizes on the server side. Until we do, the extra optimization that sizes provides requires manual intervention, at least in classifying the different images on your page to their corresponding sizes value.
•Art direction using <picture> requires manual intervention as well, at least for now. Hopefully with time, CMSs and back-end frameworks will find the UI language to enable their users to define art direction with simple means.
If you are using a CMS, the RICG is working with Drupal as well as WordPress to integrate responsive images support into the CMSs themselves, to make sure that CMS users get their benefits with as little work on their part as possible. Hopefully, other CMS and server-side frameworks will soon have the proper plugins in place to enable easy responsive images deployment.
I believe the simplest way for CMSs to support the basic use cases of user-generated responsive images would be to ask the users and editors to upload the highest-quality images possible, and have back-end code that resizes these images automatically and creates the srcset syntax that comes with them. I’m hoping that this part will soon be a standard part of major CMSs.
If art direction is required, you’d probably have to get creative and set up a process in which users can define different crops or points of interest for each uploaded image, so that you could then automate the creation of the various crops. At the moment we have not yet created such easy-to-use tools, but I hope they’ll show up soon and get integrated into the major CMSs.
As far as automation goes, there are many open source utilities that can help you. To name a few, grunt-responsive-images36 and gulp-responsive37 seem very useful to get a build process in place and automate the responsive asset creation of your site’s static assets.
Common Pitfalls
As with any new piece of technology, there are a few things that may confuse you at first. Here’s a list of things you should look out for when starting to use srcset and <picture>.
SOURCE ORDER MATTERS
Both the source selection and the source size selection algorithms are first-match algorithms. That means that the first <source> that matches is the one that gets picked. The same goes for the first media condition that matches inside the sizes value.
Therefore, you should be careful to order media queries and media conditions included there from the narrowest to the widest. If you use max-width media queries, you should arrange them from the smallest value to the largest. If you use min-width media queries, it should be the other way around, from the largest value to the smallest. Failing to do so would mean that the wider media query will always get picked, while the narrower ones below it never will.
SRCSET DOES NOT PROVIDE GUARANTEES
Many developers try out srcset and are disappointed when browsers don’t react to their resizing of the screen and reload the image. The thing is, that’s exactly the point of srcset.
Browsers may reload the resource when the user’s environment changes (browser is resized or reoriented, usually), but they don’t have to. And in some cases, it makes no sense to do so (think about what should happen if a browser is downsized, and the image is as well). That behavior is not really documented anywhere outside of browsers’ source code, and as we said, may change between browser versions.
In short, you should not assume anything about any browser’s resource download behavior when it comes to srcset.
DON’T LIE TO THE BROWSER
If you provide a browser with the wrong information regarding srcset’s resources, the browser will act on that false information. That means the wrong resource may get picked to be downloaded and displayed, but it also means that the intrinsic size of the downloaded resource – which is the actual resource dimensions, corrected according to the provided descriptors – will get skewed. If the image’s display dimensions are not defined in CSS, the image’s display size will be incorrect. Therefore, make sure you provide the correct values of w and x descriptor when writing out srcset.
Should I Use <picture> Or srcset?
We’ve already discussed this, but since it’s a question that comes up a lot, I think it’s a point worth repeating. <picture> is for art direction; it should be used when you need absolute control over which image resource is loaded at each design breakpoint. Otherwise, you’re better off using srcset to provide the browser with multiple alternative resources, and let it pick the one that best fits the user’s current conditions. Of course, <picture> can also be used for client-side MIME type fallback, regardless of responsiveness.
Can I Use It Today?
Native support for most of the features we’ve discussed is not yet complete across all browsers, but you can certainly start using these features right now.
Full support for <picture>, srcset and sizes has shipped in Chrome and Opera since versions 38 and 25 (released in October 2014) respectively.
Firefox has implemented the responsive images features alongside the implementation in Blink, and it has reached the stable branch with Firefox 38.
The srcset x descriptor is already supported in Safari 8. The implementation for sizes and srcset’s w descriptor is complete in WebKit (the rendering engine behind Safari) but, unfortunately, it did not ship in Safari 8. Since it’s already implemented, there’s reason to be hopeful that it will get shipped in a future version. Support for <picture> is not yet in WebKit (due to some missing infrastructure), so it is likely that having full <picture> support in Safari will take longer.
Regarding IE, srcset x is in development and hopefully will be part of a preview release soon. <picture> is still under consideration, but the IE development team has shown positive signs regarding the possibility of starting its development. Let’s hope its status will switch to in development shortly.
<picture> element support has also been implemented in the W3C’s validator38, so you can use that to spot mistakes in your markup, should you run into trouble with it.
On top of that, the feature has a standards-compliant polyfill called picturefill39, which you may want to use if you’re using variable-width images or art direction and want to port the benefits of that back to legacy browsers. For fixed-width images (so the x descriptor of srcset), native support is widespread enough that a polyfill is often not necessary.
Even without picturefill, the inherent <img> fallback makes it so that legacy browsers will still download and display the fallback image, meaning that the user experience in these browsers won’t be any different than if you simply use an <img> element.
The Proposal Left Behind
Those of you who followed the responsive images saga closely may remember yet another proposal, called Client Hints. That proposal suggested solving some of the use cases by using HTTP request headers to tell the server about a browser’s environmental conditions, and letting the server adapt the images it sends accordingly. That kind of solution is generally referred to as content negotiation.
For a long time, this proposal stagnated because some browser folks were reluctant to add new content negotiation-based solutions, owing to past bad experience40 with this kind of solution. Without support from these browser vendors, progress on Client Hints stalled for a long while. However, recent interest in the proposal have lead to its revival. In fact, it’s currently one of the issues I’m working on in Blink, and hopefully work in Firefox, WebKit and IE won’t lag behind.
I’m hopeful that eventually we’d be able to revive at least parts of the original proposal, and get it adopted among browsers.
Community
We cannot sum up the responsive images effort without mentioning that unlike the development process behind most web platform features, it was a community-driven one. It was championed by the RICG, supported by the developer community, and taken home when browser folks got involved too. One big happy standards-writing family.
In the same spirit, the feature’s implementation in Blink (the rendering engine behind Chrome and Opera) also set something of a precedent.
To defuse initial concerns41 from the Blink project, I devised an implementation plan42 with the Blink team and started to work on related infrastructure.
At first, I worked on that during my free time, but later on, after I realized how long that was going to take, I completed the feature’s implementation as my day job. The effort was financed by the community via a crowdfunding campaign43 which was put together with the help of the RICG gang (particularly Mat Marquis44 and Geri Coady45).
Web developers and agencies, as well as Google and Opera, contributed to the campaign, making it possible for me to work full-time on the implementation, and bring it to where it is today.
Overview
In this chapter, we went over the various responsive images use cases and how to solve them with the native tools provided by browsers.
The main things you need to remember from it all are:
•If you need art direction or specific control over the images loaded in different environments, use <picture>. Otherwise, srcset is better, since it delegates that control to browsers, and lets browsers optimize the resource downloaded to the user’s conditions.
•sizes is an optimization, hinting to browsers about the image’s final display dimensions. It doesn’t have to be there, nor does it have to be extremely accurate. But when it’s there, it enables browsers to download even smaller images.
•Optimize your images! Images are the number one resource when it comes to byte size. Don’t send excessively large images for no good reason. But also, make sure that regardless of dimensions, your images are optimized, in the appropriate format, and don’t contain excessive metadata.
•No one can remember to do all of the above manually for each image on a site. Automate these things as far as you can. Bake them into your CMS, your image upload process, or your build system. Make sure that once you’ve done the hard work of getting that in place, it will be automatically applied to all future images in your site.
One more thing: if you find the entirety of the subject slightly overwhelming, that’s probably because it’s a complicated subject, and it’s not easy to get it all in one sitting. If that’s the case, you’ll probably do better by simply starting out with srcset, get some savings in place and extend your solution from there. sizes can be added later on, as an extra optimization, and <picture> can be added when your content needs force you to.
Responsive images have been a pain point when developing responsive websites for quite some time. Now, finally, a native solution is here, shipped in Chrome and Opera.
You can start using it today, with or without picturefill, and start saving your users time and money!
This chapter is loosely based on an article46 commissioned by Opera’s developer relations team and includes screenshots from www.worldwildlife.org.