Multimedia: Preparing Your Media - JUMP START HTML5 (2014)

JUMP START HTML5 (2014)

Chapter 6 Multimedia: Preparing Your Media

Before we begin talking about HTML5's multimedia elements, let's talk about preparing your media. Though we can now use native HTML5 elements in most browsers, we can't (yet) use the same files in every browser. We touched on this in the introduction, but a fuller explanation is appropriate. I've tried my best to be fair about it.

So why can't we use the same files in every browser? The short answer is: file formats.

Codec Showdown

The longer answer is this: browser vendors have not yet agreed on a single format for audio and video on the web.

When multimedia was first added to the HTML5 specification, Ogg—Vorbis for audio and Theora for video—was proposed as a standard, default format. The Ogg specification is considered public domain. Most of its tools have open source licenses. It's believed to be patent-free, and non-infringing on other patents. For those reasons, it seemed like a fairly good choice for a default codec.

Some browser vendors—primarily Apple and Nokia—disagreed with the choice of Ogg and flatly refused to support it. Among their concerns was an uncertain patent landscape. After all, just because Ogg is believed to be non-infringing, doesn't mean that it is. Plus Ogg lacked the robust hardware support of formats such as H.264 video.

Speaking of H.264 video, Apple and Nokia made it their video format of choice. H.264 is widely supported by hardware and software, particularly on mobile. Indeed much of the video used on the web today relies on it. However, H.264 is what's known as encumbered: it's a patented format. Adding H.264 support to your software requires paying hefty royalty fees. Opera and Mozilla (developers of Firefox), balked at that. Both refused to add H.264 support to their browsers.

Google, to its credit, chose to support H.264 and Ogg in its Chrome browser. But Chromium, the open source codebase underlying Chrome, lacks H.264 support. Microsoft largely avoided the fray but then also settled on H.264.

I should mention here that H.264 is a video codec, and largely what the fuss was about. Audio formats are less contested, in part, because there are just so many to choose from.

All that said: in 2009, specification authors removed the codec requirement. Browser vendors implemented the codecs they wanted to. As a result, we have to encode our audio and video in multiple formats. For all of the sordid details and angry posts, comb through the WHATWG mailing list archives from about mid-2009.

The Current Landscape

Since the codec requirement was dropped from HTML5, things have changed slightly in browser land. Mozilla and Opera now support H.264, but only if the hardware or third-party software provides access—on Android devices, for example.

A new format—WebM—also joined the party. WebM is sponsored by Google, and is a container format for the VP8 and VP9 video codecs. Not surprisingly, Google's Chrome supports WebM natively. So does Opera as of version 10.60, and Firefox as of version 4.0 (though only the VP8 codec at this time). Internet Explorer doesn't support WebM natively, but will if third-party software is installed. Apple's Safari is still H.264 only.

At a minimum, we'll need to encode our video files in at least two formats: H.264 and WebM. For audio files, use MP3 and WebM. These formats enjoy the widest support among currently-used browsers. To support Firefox 3 or Opera 10.50 users encode your files using Ogg ― Ogg Theora for video and Ogg Vorbis for audio.

Luckily for us, basic media transcoding is easy and there are free and open-source tools for the job. In this chapter, we'll discuss two of them: the command-line tool FFmpeg and the graphical user interface, Miro Video Converter.

Note: Beware Hosting Costs

Storing multiple files in multiple formats may affect your hosting costs, particularly as the number of videos stored increases. Check the details your web hosting and storage plan.

Converting Files Using Miro Video Converter

Miro Video Converter provides a graphical user interface for FFmpeg, the open source, command line media conversion tool. It's less flexible than FFmpeg. But it's also easier to use if the command line makes you queasy.

To convert a file using Miro, add it to the queue. You can do this by dragging and dropping a file into the application. Or click "Choose File", and navigate to it. You can add multiple files to the queue. Miro will process them sequentially. Once our files are queued, we can choose our export format.

The conversion queue for Miro Video Converter for OS X

Figure 6.1. The conversion queue for Miro Video Converter for OS X

Miro offers a slew of presets for targeting specific devices. Using the Apple > Apple Universal preset is a good choice for H.264-capable browsers. For WebM and Ogg, look under the Format menu instead.

Under the settings menu (the gear icon on the right), you'll also see options to create thumbnails, resize your video, or adjust the aspect ratio. Click the gear icon again to return to the main settings screen.

The settings panel in Miro Video Converter

Figure 6.2. The settings panel in Miro Video Converter

Once you've selected a format and (optionally) adjusted your settings, click the green button at the bottom of the screen to begin transcoding.

We're okay to convert! What Miro looks like once we've chosen an output format

Figure 6.3. We're okay to convert! What Miro looks like once we've chosen an output format

Miro Video Converter will output your converted file to its output folder. Its location varies by operating system, but you can find a link in the settings menu.

Converting Media Using FFmpeg

If you want a bit more flexibility, try using FFmpeg. It's open source, and primarily for Linux systems. But binaries for Mac OS X and Windows are available via the FFmpeg web site. If you're using a Mac, you can also install FFmpeg using Homebrew or Macports.

FFmpeg supports dozens of formats, including the ones we need for the web: Ogg Vorbis and Theora, MPEG-3, MPEG-4/H.264, and WebM. You can also use it to export still frames, which comes in handy for generating poster images.

Let's take a look at what it takes to convert a video from QuickTime (.mov) to WebM using FFmpeg. Navigate to the directory that houses the file you want to convert. Then open a terminal window.

FFmpeg conversions are straightforward. A simple version looks like this.

ffmpeg -i /path/to/input.file /path/to/output.file

FFmpeg will determine which codec to use based on the output file's extension.

In this case, our file is named tracy-sings.mov. To convert it to WebM, we'll type the following.

ffmpeg -i tracy-sings.mov tracy-sings.webm

That's it. Your terminal window will then fill up with all sorts of details about the input and output files.

Note: Using FFmpeg Binary

If you're using an FFmpeg binary, you may also need to include the path to FFmpeg (for example: /usr/local/ffmpeg).

An action shot of the FFmpeg conversion process

Figure 6.4. An action shot of the FFmpeg conversion process

A few moments later you'll have a transcoded file in the output format you requested.

Resizing Video Files

Serving a 1080p (typically 1920 pixels by 1080 pixels) video may be a larger file than you need online. FFmpeg also includes tools to scale or resize video files. One way to do this is with FFmpeg's scaling filter.

Let's try scaling our movie to a smaller 720p (1280 pixels by 720 pixels) resolution. We'll also transcode it to WebM.

ffmpeg -i tracy-sings.mov -vf scale=-1:720 tracy-sings-720p.webm

What's new here is the -vf flag, which is a flag indicating we'd like to use a filter. That's followed by our filter name (scale) and our desired ratio (width:height). Though we could specify both dimensions, here we've used -1 instead. This indicates to FFmpeg that we want to preserve our aspect ratio as it scales. As in our previous example, the last part of our command is the output file name and extension.

Using FFmpeg to Generate a Poster Image

FFmpeg isn't limited to media conversions. We can also use it to export frames from video files. This will come in handy when we discuss adding a poster image to our videos. The basic command follows this pattern:

ffmpeg -ss 30 -t 2 -i /path/to/input.file -r 1

↵/path/to/output-%02d.jpg

This example involves a few more command line flags than our previous ones. So let's run through it.

The -ss flag instructs FFmpeg to seek the specified position, as expressed in seconds. In this case, we'll start exporting frames from 30 seconds into our video.

Next is the -t flag or duration, or how many seconds of video we'd like to extract as images. In this case, we'll capture two seconds worth of frames. As you may have guessed by now the -i flag stands for "input," and we use it to specify our source file.

The -r flag sets the frame rate, or number of frames per second in our video export. In this case, it's 1 frame per second. This means we'll end up with two stills from this video, one for each second specified by the -t flag. If we used a higher frame rate, such as 24 frames per second, we'd extract 24 images.

Finally, we need to tell FFmpeg where to put the files. The %02d is a place holder or wildcard of sorts. It tells FFmpeg to name our images in sequential order, padding to 2 digits long where necessary. In this case, our images would be named output-01.jpg, output-02.jpg and so on. If we used%03d, our files would be named output-001.jpg, output-002.jpg instead.

Using a Hosted Service

Miro Video Converter and FFMpeg are great for audio and video that you want to host yourself. But there are a few services that offer hosting and transcoding. Archive.org offers free hosting and transcoding for audio and video works released under a Creative Commons license or to the public domain. Vid.ly, Viddler, and Zencoder are three paid options, and also provide configurable players. Details and pricing vary.

Quality Versus File Size

This chapter isn't a comprehensive look at Miro Video Converter, FFmpeg, or optimizing media for the web. But it will get you started.

You should, however, consider bitrate and frames per second (FPS) for your audio and video files. There are few hard-and-fast guidelines here. Which values you choose will depend your ― and your audience's ― tolerance for quality versus file size, bandwidth consumption and download times.

Popular video sites such as YouTube, Vimeo, and Dailymotion offer some guidance for compressing media. Use FFmpeg to make these adjustments. Your best bet is to start with these guides and experiment with different rates until you strike the right balance.

Now that you know have some tools for converting media, let's talk about how to add it to your page.