Responsive Web Design, Part 2 (2015)

Performance Optimization Roadmap

Responsive Images

While images don’t block page rendering and hence aren’t on the critical path, we wanted to deliver images fast, so that the entire content of an article would be displayed right away. Obviously, this is what a CDN would be used for, but it also depends on the weight of images, and we weren’t doing particularly well in that regard. We wanted images to render crisply at all pixel density displays, and we wanted to use all available space when necessary, without penalizing low-density screens with large and heavy images.

From early on, we started consistently used mid-sized images of 500px width, which worked well in our layouts and remained unchanged over all these years. We compressed images as well as we could, using progressive JPEGs whenever possible and running images through lossless compression tools like TinyPNG and ImageOptim within a Grunt task. Most images had a fixed width and were rescaled on small screens while leaving a lot of white space on large screens. With our current optimizations in place, this approach didn’t really sound like smart content choreography, and we felt we were missing a good opportunity to reduce bandwidth and enhance the reading experience, on small and large screens. We weren’t serving high-quality images to Retina screens either, so there was a lot of room for improvement — and the devil was in details.

There wasn’t much we could do to improve the experience of our legacy articles on larger screens. For many of those articles we just didn’t have high-resolution images, or the original images were provided in 500px width. The only option we had was to resize or crop existing images to reduce loading times on mobile screens. By default, all those images were resized anyway (although a full 500px image would be downloaded by browsers), so we could just go ahead and rescale all images to 280–300px width and provide that as an option for browsers. Going through each image and cropping to ensure a fine level of detail for every image would require a significant effort from our side and was neither practical nor feasible.

It didn’t mean we couldn’t prepare images properly for upcoming articles, though. In fact, back in 2013 we started collecting multiple variations of images (usually just rescaled) for our editorial library, as we started using high-resolution images in our ebooks. But we didn’t use them on the website, and we didn’t consider the mobile view use case at this point. The foundation was there, it just had to be tweaked a little bit.

Delivering different images to different screens without a performance hit is often not really a technological matter, but a content problem and a workflow issue. The CMS has to be prepared for different image variants, and those variants have to be thoroughly processed, optimized and supplied, and the markup for responsive images has to be generated and embedded in the page as well. With our comfortable user base, we didn’t have to concern ourselves too much with legacy browsers, but we had to decide which image should be loaded by default if the browser didn’t support responsive images markup.

We had a few options:

•Use the smallest image from the stack as the default (in the image <src> attribute) and then use Picturefill to help legacy browsers deal with responsive images (particularly for users with large screens.

•Use a mid-size image (500px wide) from the stack by default (in the image <src> attribute), so legacy browsers always get a good regular view and we’d have no dependency on Picturefill (a performance hit).

At the time of writing we are experimenting with both approaches and haven’t decided on one of them just yet; but in the first case, the benefits for all browsers seem to outweigh the drawbacks that a reliance on Picturefill brings.

Prompted by the nature of our layout, our image stack would consist of three to five images with average sizes of 300px, 500px, 850px and 1,200px, plus the maximum size we can get from authors and potentially 1,600px — all derived from the layout width with a font size high enough for comfortable reading. Looking at the stats, our visitors tend to read articles on screens between 900px and 1,600px, and the most problematic areas are either very small screens (images too heavy), or very large screens (too much empty space, images too small).

For instance, the article area on (very) large screens (2,100px) can take up to 810px in width. To fill the entire space for @2x Retina displays would require at most a 1,600px-wide image (2×800px). These image variations obviously wouldn’t be enough for @3x or @4x displays, but images at such resolutions usually aren’t available in our articles, and we also have to find a balance between the effort involved and the result achieved. Besides, not all authors are able to deliver high-resolution images, so we have to be quite flexible on the markup we generate and the image versions we provide. Obviously, the more time and effort you put into crafting just the right images for just the right resolutions, the better the result would be, but it’s neither practical nor affordable in many situations.

What happens if the layout changes in a few years? Well, we don’t know. We do have a variety of images prepared for the upcoming changes, and the constraints of the image library would have to define the constraints of the layout we will design and build in the future. But at this point we sought a good balance and compromise between the ongoing extra effort and the achievable result.

What was pretty clear from the beginning was that we wouldn’t be using an art direction use case. We don’t need full control of how images look across specific breakpoints; srcset should be enough for our purposes, with sizes for small galleries or side-by-side image comparison. But most importantly, we wanted to give browsers options to make informed, meaningful decisions on what images to pick on their own. Should the user with a Retina display turn on the low bandwidth mode in a browser at some point in future, the browser could load more appropriate images, not necessarily the Retina images we’d define within the <picture> element. The only use case when we’d consider using <picture> would be to specifically address landscape and portrait device orientations, or serving WebP images to browsers that support them.

Examples: using <picture> to target different device orientations and WebP support, with a JPEG fallback.

The first use case hasn’t come up just yet, and the second use case would again introduce another level of complexity — creating a WebP version for every single variation of every single image in every single article — that felt a bit too much overboard. We also didn’t find a good regular use case for using sizes, since our images usually fill in the entire article space; a nice side effect of this was that we didn’t have to mix the content and the presentation. And we didn’t have to deal with background images because all content images in articles are regular foreground images.

Introducing responsive images meant introducing significant additional effort into the editorial workflow. On average, Smashing Magazine’s articles contain six to seven images, and each of the images would have four to five variations, resulting in around 30 to 35 images that would have to be processed, uploaded and marked up per article. Completing it manually would require a lot of work, so the only option was to automate the process. In fact, as Yoav mentions in his chapter, the Responsive Images Community Group is working on integrating the markup generation for responsive images into Drupal³⁹ and WordPress⁴⁰, so we wanted to be strategically prepared for that, but also experiment with different use cases using our own tools.

Resizing images wasn’t difficult with the responsive images Grunt task⁴¹ by Andi Smith, but defining a focal point and cropping images appropriately for smaller views proved to be a bit more involved. We started playing around with responsive images markup and specifically srcset, writing markup manually at first to get a good feeling of how it works, and automating the generation later. For cropping we tried out sizzlepig⁴² which worked quite well, despite the fact that it’s an online service that can’t be accessed from the terminal, so the process was a bit slower than it should be.

The average markup for an article looked very much like this:

<img src="perf-500-opt.jpg" alt="Optimization techniques"

srcset="perf-300-opt.jpg 300w,

perf-500-opt.jpg 500w,

perf-850-opt.jpg 850w,

perf-1200-opt.jpg 1200w,

perf-1600-opt.jpg 1600w">

The first articles with the srcset markup were published in January 2015, and on average they took 30 more minutes to prepare (without proper tools involved). To have a clear indicator of the image’s size, we add either the key “-opt” and the width to the file name, so everybody on the team knows if the image has been run through an optimization routine and what screen it’s for.

We’ve been providing links to larger views in captions under the images, so in some cases we could actually just play with images we already had without creating new ones first. The performance has improved since Chrome’s share in our stats is more than 53%, and it will keep improving gradually and significantly over time as more and more browsers support the new attribute. On average, by considering responsive images within the editorial workflow, we were able to reduce the bandwidth of an average article by around 40% (!) just by using srcset alone, with four variants provided per image.

With Picturefill in place, most users will benefit from faster loading times and reduced bandwidth. Again, the move wasn’t difficult from a technological perspective, but it took a lot of strategic consideration, long-term planning and additional ongoing editorial work.

If you’d like to jump right into responsive images, you might find the following resources particularly helpful:

•“Responsive Images in Practice⁴³” by Eric Portis

•“Responsive Images: Use Cases and Documented Code Snippets to Get You Started⁴⁴” by Andreas Bovens

•grunt-responsive-images-converter⁴⁵, a Grunt plugin for converting image tags in Markdown files to <picture> elements.