JUMP START HTML5 (2014)

Chapter 25 Offline Apps: Storing Data With Client-side Databases

Web storage (localStorage and sessionStorage) is fine for small amounts of data such as our to-do list, but it’s an unstructured data store. You can store keys and values, but you can’t easily search values. Data isn’t organized or sorted in a particular way, plus the 5MB storage limit is too small for some applications.

For larger amounts of structured searchable data, we need another option: web databases. Web databases such as Web SQL and IndexedDB provide an alternative to the limitations of web storage, enabling us to create truly offline applications.

The State of Client-side Databases

Unfortunately, this isn’t as easy as it sounds. Browsers are split into three camps in the way they support client-side databases:

· both IndexedDB and Web SQL (Chrome, Opera 15+)

· IndexedDB exclusively (Firefox, Internet Explorer 10+)

· Web SQL exclusively (Safari)

IndexedDB is a bit of a nonstarter if you plan to support mobile browsers. Safari for iOS and Opera Mobile 11-12 support Web SQL exclusively; same for the older versions of Android and Blackberry browsers.

If you want your application to be available across desktop browsers, you’re about halfway there with Web SQL. It’s available in Safari, Chrome, and Opera 15+, but Firefox and Internet Explorer 10+ have no plans to add support.

Here’s the thing: the World Wide Web Consortium has stopped work on the Web SQL specification. As a result, there is a risk that browsers will drop Web SQL support, or that further development will proceed in nonstandard ways. Relying on it is risky, so for that reason we will focus on IndexedDB in this chapter, and use a polyfill to support Safari and older versions of Opera.

The bright side is that there are a few JavaScript polyfills and libraries available to bridge the gap between Web SQL and IndexedDB. Lawnchair supports both, and will use localStorage if you prefer. There’s also PouchDB, which uses its own API to smooth over the differences between Web SQL and Indexed DB. PouchDB also supports synchronization with a CouchDB server, though CouchDB isn’t necessary for building an app with PouchDB.

In this chapter, we’ll focus on IndexedDB, and use IndexedDBShim for other browsers.

About IndexedDB

IndexedDB is a schema-less, transactional, key-value store database. Data within an IndexedDB database lacks the rigid, normalized table structure as you might find with MySQL or PostgresSQL. Instead, each record has a key and each value is an object. It’s a client-side “NoSQL” database that’s more akin to CouchDB or MongoDB. Objects may have any number of properties. For example, in a to-do list application, some objects may have a tags property and some may not, as evident in Figure 25.1.

Objects with and without a tags property

Figure 25.1. Objects with and without a tags property

Two objects in the same database can even have completely different properties. Usually, we’ll want to use a naming convention for our object property names.

IndexedDB is also transactional. If any part of the read or write operation fails, the database will roll back to the previous state. These operations are known as transactions, and must take place inside a callback function. This helps to ensure data integrity.

IndexedDB has two APIs: synchronous and asynchronous. In the synchronous API, methods return results once the operation is complete. This API can only be used with Web Workers, where synchronous operations won't block the rest of the application. There is no browser support for the synchronous API at the time of writing.

In this chapter, we’ll cover the asynchronous API. In this mode, operations return results immediately without blocking the calling thread. Asynchronous API support is available in every browser that supports IndexedDB.

Examples in this book use the latest version of the IndexedDB specification. Older versions of IndexedDB in Chrome (23 and earlier) and Firefox (16 and earlier) required a vendor prefix. These experimental versions had several inconsistencies between them, which have been largely worked out through the specification process. Since the standardized API has been available for several versions in both Firefox and Chrome, and is available in Internet Explorer 10+, we won’t bother with older versions.

Note: Inspecting IndexedDB Records

If you’d like to inspect your IndexedDB records, use Chrome or Opera 15+. These browsers currently have the best tools for inspecting IndexedDB object stores. With IndexedDBShim, you can use Safari’s Inspector to view this data in Web SQL, but it won’t be structured in quite the same way. Internet Explorer offers an IDBExplorer package for debugging IndexedDB, but it lacks native support in its developer tools. Firefox developer tools are yet to support database inspections.

Now, let’s look at the concepts of IndexedDB by creating a journaling application to record diary entries.

Setting up Our HTML

Before we dive into IndexedDB, let’s build a very simple interface for our journaling application:

<!DOCTYPE html>

<head>

<title>IndexedDB Journal</title>

</head>

<body>

<form>

<p>

<label for="tags">How would you like to tag this entry?</label>

<span class="note">(Optional. Separate tags with a comma.)</span>

</p>

<p>

<label for="entry">What happened today?</label>

</textarea>

<button type="submit" id="submit">Save entry</button>

</p>

</form>

<h1>View all entries</h1>

</section>

</body>

</html>

This gives us a very simple interface consisting of two form fields and a Save entry button, as shown in Figure 25.2. We've also added a view that displays all entries after we've saved our latest one. A production-ready version of this application might have a few more screens, but for this demo this is fine.

Our simple journal interface as shown in Firefox.

Figure 25.2. Our simple journal interface as shown in Firefox.

Creating a Database

To create a database, use the open() method. It accepts two arguments: the name of your database and an optional integer value that sets the version of the database:

var idb = indexedDB.open('myIDBJournal',1);

Your database name may be any string value, including the empty string (''). The name just ties the database to its origin. As with localStorage, IndexedDB databases can only be read to from—or written to by—scripts that share its origin. Remember that an origin consists of the scheme (such as http:// or https://), the hostname, and port number). These must match exactly, meaning that a database created at http://example.com can't be accessed by scripts at http://api.example.com and vice versa.

Version numbers can be any integer from 1 to 2^53 (that’s 9,007,199,254,740,991). Floats, or decimal values, won’t work. Floats will be converted to integers. For example, 2.2 becomes 2 and 0.8 becomes 0 (and throws a TypeError). If you don’t include a version number, the browser will use 1.

If the open operation is successful, the browser will fire a success event. We can use the onsuccess callback to perform transactions or, as we’ve done here, use it to assign our database object to a global variable:

// initialize the variable.

var databaseObj;

idb.onsuccess = function(successEvent){

// set its value.

databaseObj = successEvent.target.result;

}

As with all DOM events, our successEvent object contains a target property. In this case, the target is our open request transaction. The target property is also an object, and contains a result property, which is the result of our transaction. In this case, the value of result is our database.

This is a really important point to understand. Every transaction requires an onsuccess handler. The results of that transaction will always be a child property of the target object, which is a property of the event object (event.target.result).

Now that we’ve set our global databaseObj variable to our database object, we can use databaseObj for our transactions.

Adding an Object Store

Creating a database won’t automatically make it do anything by itself. To begin storing data, you’ll have to create at least one object store. If you are familiar with SQL databases, an object store is analogous to an SQL table. Object stores are where we store our records, or entries. IndexedDB databases can contain multiple object stores.

To add an object store, we first need to trigger a version change. To trigger a version change, the version argument (the second parameter) needs to be greater than the database’s current version value. For the first release of your application, this value can be 1.

When the database version number increases (or is set for the first time), the browser will fire an upgradeneeded event. Any structural changes―adding an object store, adding an index to an object store―must be made within the onupgradeneeded event handler method:

idb.onupgradeneeded = function(event){

// Change the database.

}

Let’s create an object store named entries. To do this, we need to use the createObjectStore method of the database object:

idb.onupgradeneeded = function(event){

try {

event.target.result.createObjectStore(

['entries'],

{ keyPath: 'entrydate' }

);

} catch(error) {}

}

If the object store doesn’t exist, it will be created. If it does, the browser will fire a constraint error. We’ve wrapped this in a try-catch block so that we can silently handle an error should one occur. But we could also set an event handler using the onerror event attribute:

idb.onerror = function(event){

// Log the error to the console, or alert the user

}

At a minimum, createObjectStore requires a name argument. It must be a string, but this string can’t contain any spaces. The second argument is optional, but it must be a dictionary. Dictionaries are objects that have defined properties and values. For createObjectStore, those properties and values are as follows:

· autoIncrement: Must be either true or false; auto-generates keys and indexes for each record in the store. Default value is false.

· keyPath: Specifies which object property to use as an index for each record in the store. Default value is null. Makes the named field a required one.

Here we’ve chosen to set a keyPath. This means that every object we add to the database will need to contain an entrydate property, and the value of the entrydate property will become our key.

You don’t have to set autoIncrement or keyPath. It’s possible to add our object store without it. If you choose not to set either, you must set a key for every record stored. We’ll discuss this in the next section.

Notice that we didn’t use our databaseObj variable with our onupgradeneeded callback? That’s because the onupgradeneeded event is fired before the onsuccess event when working with IndexedDB. It won’t be defined when we need it.

Adding a Record

Adding records is a little more complicated. We need to create a transaction, and then take an action once the transaction completes. The process is roughly as follows:

1. Open a transaction connection to one or more object stores.

2. Select which object store to use for the transaction request.

3. Create a request by invoking the put, add, delete or get methods.

4. Define an onsuccess handler to process our results.

These steps need to happen within a callback or event handler of some kind. Here we’ll do it when our journal entry form is submitted:

document.forms[0].onsubmit = function(submitEvent){

var entry, i, transaction, objectstore, request, fields;

fields = submitEvent.target;

// Prevent form submission and page reload.

submitEvent.preventDefault();

/* Build our record object */

entry = {};

entry.entrydate = new Date().getTime();

for( i=0; i < fields.length; i++){

if( fields[i].value !== undefined ){

entry[fields[i].name] = fields[i].value;

}

/* Set our success handler */

request.onsuccess = showAllEntries;

/* Start our transaction. */

transaction = databaseObj.transaction(['entries'],'readwrite');

/* Choose our object store (the only one we've opened). */

objectstore = transaction.objectStore('entries');

/* Save our entry. We could also use the add() method */

request = objectstore.put(entry);

}

There’s a lot happening in this function, but the most important parts are the following three lines:

/* Start our transaction. */

transaction = databaseObj.transaction(['entries'],'readwrite');

/* Choose our object store (the only one we've opened). */

objectstore = transaction.objectStore('entries');

/* Save our entry */

request = objectstore.put(entry);

In these lines, we’ve first created a transaction by calling the transaction method on our database object. transaction accepts two parameters: the name of the object store or stores we’d like to work with, and the mode. The mode may be either readwrite or readonly. Use readonly to retrieve values. Use readwrite to add, delete, or update records.

Next, we’ve chosen which object store to use. Since the transaction connection was only opened for the entries store, we’ll use entries here as well. It’s possible to open a transaction on more than one store at a time, however.

Let’s say our application supported multiple authors. We might then want to create a transaction connection for the authors object store at the same time. We can do this by passing a sequence—an array of object store names—as the first argument of the transaction method, as shown here:

trans = databaseObj.transaction(['entries','authors'], 'readwrite');

This won’t write the record to both object stores. It just opens them both for access. Which object store will be affected is determined by the objectStore() method.

Note: Use Square Brackets

In newer versions of Chrome (33+) and Firefox, you may pass a single object store name to the transaction method without square brackets; for example, databaseObj.transaction('entries','readonly'). For the broadest compatibility, though, use square brackets.

req = objectstore.put(entry); is the final line. It saves our entry object to the database. The put method accepts up to two arguments. The first is the value we’d like to store, while the second is the key for that value: for example: objectStore.put(value, key).

In this example, we’ve just passed the entry object to the put method. That’s because we’ve defined entrydate as our keyPath. If you define a keyPath, the property name you’ve specified will become the database key. In that case, you don’t need to pass one as an argument. If, on the other hand, we didn’t define a keyPath and autoIncrement was false, we would need to pass a key argument.

When the success event fires, it will invoke the showAllEntries function.

Note: put Versus add

The IndexedDB API has two methods that work similarly: put and add. You can only use add when adding a record. But you can use put when adding or updating a record.

Retrieving and Iterating Over Records

You probably noticed the line request.onsuccess = showAllEntries in our form’s onsubmit handler. We’re yet to define that function, but this is where we’ll retrieve all our entries from the database.

To retrieve multiple records there are two steps:

1. run a cursor transaction on the database object

2. iterate over the results with the continue() method

Creating a Cursor Transaction

As with any transaction, the first step is to create a transaction object. Next, select the store. Finally, open a cursor with the openCursor method. A cursor is an object consisting of a range of records. It’s a special mechanism that lets us iterate over a collection of records:

function showAllEntries(){

var transaction, objectstore, cursor;

transaction = databaseObj.transaction(['entries'],'readonly');

objectstore = transaction.objectStore('entries');

cursor = objectstore.openCursor();

};

Since we want to show our entries once this transaction completes, let’s add an onsuccess handler to our cursor operation:

function showAllEntries(){

var transaction, objectstore, cursor;

transaction = databaseObj.transaction(['entries'],'readonly');

objectstore = transaction.objectStore('entries');

cursor = objectstore.openCursor();

cursor.onsuccess = function(successEvent) {

var resultset = successEvent.target.result;

if( resultset ){

buildList( resultset.value );

}

resultset.continue();

};

Within this handler, we’re passing each result of our search to a buildList function. We won’t discuss that function here, but it is included in this chapter’s code sample.

That final line―resultset.continue()―is how we iterate over our result set. The onsuccess handler is called once for the entire transaction, but this transaction may return multiple results. The continue method advances the cursor until we’ve iterated over each record.

Retrieving a Subset of Records

With IndexedDB, we can also select a subset or range of records by passing a key range to the openCursor() method.

Key ranges are created with the IDBKeyRange object. IDBKeyRange is what’s known as a static object, and it’s similar to the way the Math() object works. Just as you’d type Math.round() rather than var m = new Math(), with IDBKeyRange, you must always use IDBKeyRange.methodName().

In this example, we’re only setting a lower boundary using the lowerBound method. Its value should be the lowest key value we want to retrieve. In this case, we’ll use 0 since we want to retrieve all the records in our object store, starting with the first one:

objectstore.openCursor(IDBKeyRange.lowerBound(0));

If we wanted to set an upper limit instead, we could use the upperBound method. It also accepts one argument, which must be the highest key value we want to retrieve for this range:

objectstore.openCursor(IDBKeyRange.upperBound(1385265768757));

By default, openCursor sorts records by key in ascending order. We can change the direction of the cursor and its sorting direction, however, by adding a second argument. This second argument may be one of four values:

· next: puts the cursor at the beginning of the source, causing keys to be sorted in ascending order

· prev: short for “previous”, it places the cursor at the end of the source, causing keys to be sorted in descending order

· nextunique: returns unique values sorted by key in ascending order

· prevunique: returns unique values sorted by key in descending order

To display these entries in descending order (newest entry first), change objectstore.openCursor(IDBKeyRange.lowerBound(0)) to objectstore.openCursor(IDBKeyRange.lowerBound(0), 'prev').

Retrieving or Deleting Individual Entries

But what if we wanted to retrieve just a single entry instead of our entire store? For that, we can use the get method:

var transaction, objectstore, request;

transaction = databaseObj.transaction(['entries'],'readonly');

objectstore = transaction.objectStore('entries');

request = objectstore.get(key_of_entry_to_retrieve);

request.onsuccess = function(event){

// display event.target.result

}

When this transaction successfully completes, we can do something with the result.

To delete a record, use the delete method. Its argument should also be the key of the object to delete:

var transaction, objectstore, request;

transaction = databaseObj.transaction(['entries'],'readwrite');

objectstore = trans.objectStore('entries');

request = objectstore.delete(1384911901899);

request.onsuccess = function(event){

// Update the user interface or take some other action.

}

Unlike retrieval operations, deletions are write operations. Use readwrite for this transaction type, instead of readonly.

Updating a Record

To update a record, we can once again use the put method. We just need to specify which entry we’re updating. Since we’ve set a keyPath, we can pass an object with the same entrydate value to our put method:

var transaction, objectstore, request;

transaction = databaseObj.transaction(['entries'],'readwrite');

objectstore = trans.objectStore('entries');

request = objectstore.put(

{

entrydate: 1384911901899,

entry: 'Updated value for key 1384911901899.'

}

);

request.onsuccess = function(event){

// Update the user interface or take some other action.

}

The browser will update the value of the record whose key is 1384911901899, as shown in Figure 25.3.

The updated value for our key

Figure 25.3. The updated value for our key

Updates, like additions, are write operations. When creating an update transaction, the second parameter of the transaction method must be readwrite.

Be careful when using put for updates. Updates replace the entire object value with the new one. You aren’t simply overwriting individual object properties; you’re actually replacing the whole record.

Deleting a Database

Deleting a database is just as easy as opening one. It doesn’t require using a callback. Just call the deleteDatabase method of the IndexedDB database object:

idb.deleteDatabase('myIDBJournal');

It accepts one argument: the name of the database you wish to delete. Use it with caution.

We’ve covered enough to get you started with IndexedDB, but there’s more to the API than can be covered in a single chapter. The Mozilla Developer Network has the most robust documentation available to date, including coverage of older versions of the API.

Wrapping Up and Learning More

With offline web technologies, you can develop applications that work without an internet connection. We’ve covered the basics of offline applications in this book. If you’d like to learn more about any of these topics, a good place to start is with the W3C’s WebPlatform.org. It features tutorials and documentation on HTML5, CSS, JavaScript and web APIs.

Vendor-specific sites also have a wealth of information, particularly for documenting browser-specific quirks. Yet Google’s HTML5Rocks.com and the Mozilla Developer Network, mentioned above, are particularly good at documenting new technologies in their browsers while also addressing other vendors’ implementations. The Microsoft Developer Network includes a wealth of documentation about web technologies, especially as supported in Internet Explorer.

To track specifications, pay attention to the World Wide Web Consortium (W3C) and the Web Hypertext Application Technology Working Group (WHATWG). These bodies develop specifications and documentation for current and future APIs.

And of course, you can always visit Learnable.com or SitePoint.com to stay up to date on web technologies.