Understanding asynchronous flow control methods in JavaScript - Managing complexity - JavaScript Application Design: A Build First Approach (2015)

JavaScript Application Design: A Build First Approach (2015)

Part 2. Managing complexity

Chapter 6. Understanding asynchronous flow control methods in JavaScript

This chapter covers

· Understanding callback hell and how to avoid it

· Making Promises and keeping them in JavaScript

· Using asynchronous control flow

· Learning event-based programming

· Using Harmony (ECMAScript 6) generator functions

Chapter 5 taught the importance of building your components in a modular fashion, and you learned a great deal about scoping, hoisting, and closures, all of which are necessary to understand asynchronous JavaScript code effectively. Without a modest understanding of asynchronous development in JavaScript, it becomes harder to write quality code that’s easy to read, refactor, and maintain.

One of the most frequently recurring issues for JavaScript development beginners is dealing with “callback hell,” where many functions are nested inside each other, making it hard to debug or even understand a piece of code. This chapter aims to demystify asynchronous JavaScript.

Asynchronous execution is when code isn’t executed immediately, but rather in the future; such code isn’t synchronous because it doesn’t run sequentially. Even though JavaScript is single-threaded, user-triggered events such as clicks, timeouts, or AJAX responses can still create new execution paths. This chapter will cover different ways you can handle asynchronous flow in a tolerable and painless way, by applying a consistent style to asynchronous code flows. Much like chapter 5, this chapter has many practical code examples for you to follow!

To kick things off, we’ll look at one of the oldest patterns available: passing a callback as an argument so the caller of a function can determine what happens in the future when the callback is invoked. This pattern is referred to as continuation-passing style, and it’s the bread and butter of asynchronous callbacks.

6.1. Using callbacks

A prime example of using callbacks is found in the addEventListener API, which allows us to bind event listeners on DOM (Document Object Model) nodes. When those events are triggered, our callback function gets called. In the following trivial example, when we click anywhere in the document, a log statement will be printed to the console:

document.body.addEventListener('click', function () {

console.log('Clicks are important.');

});

Click event handling isn’t always that trivial. Sometimes you end up looking at something that resembles the following listing.

Listing 6.1. Callback soup using logic noodles

What’s going on? My thoughts exactly. You’ve been dragged through callback hell, that friendly name that describes deeply nested and indented callbacks on top of more callbacks, which make it pretty difficult to follow the flow and understand what’s going on. If you can’t make sense of the code presented in listing 6.1, that’s good. You shouldn’t have to. Let’s dig deeper into the subject.

6.1.1. Avoiding callback hell

You should understand how a piece of code flows at a glance, even if it’s asynchronous. If you need to spend more than a few seconds to understand how it flows, then there’s probably something wrong with that piece of code. Each nested callback means more nested scopes, as observed inchapter 5, and indentation one level deeper, which consumes a little more real estate in your display, making it harder to follow the code.

Callback hell doesn’t happen overnight, and you can prevent it from ever happening. Using an example (named ch06/01_callback-hell in the samples), let’s see how it might slowly creep through the cracks of your code base over time. Suppose you have to make an AJAX request to fetch data, and then show that to a human. You’ll use an imaginary http object to simplify the AJAX-foo. Let’s also assume you have a record variable holding a reference to a particular DOM element.

record.addEventListener('click', function () {

var id = record.dataset.id;

var endpoint = '/api/v1/records/' + id;

http.get(endpoint, function (res) {

record.innerHTML = res.data.view;

});

});

That’s still easy to follow! What if you need to update another component after the GET request succeeded? Consider the following listing. Let’s assume there’s a DOM element in the status variable.

Listing 6.2. Callback creep

function attach (node, status, done) {

node.addEventListener('click', function () {

var id = node.dataset.id;

var endpoint = '/api/v1/records/' + id;

http.get(endpoint, function (res) {

node.innerHTML = res.data.view;

reportStatus(res.status, function () {

done(res);

});

});

function reportStatus (status, then) {

status.innerHTML = 'Status: ' + status;

then();

}

});

}

attach(record, status, function (res) {

console.log(res);

});

Okay, that’s starting to get bad! Nested callbacks add complexity every time you add a nesting level into the piece of code, because now you have to keep track of the context of the existing function as well as the context of the deeper callback. Take into account that in a real application each of these methods would probably have more lines in them, making it even harder to keep all of that state in your memory.

How do you fight the callback creep? All that complexity can be avoided by reducing the callback nesting depth.

6.1.2. Untangling the callback jumble

You have ways to untangle these innocent pieces of code. Here’s a list of things you should take into account and fix:

· Name anonymous functions, to improve their readability, and give hints as to what they’re doing. Named anonymous callbacks provide two-fold value. Their names can be used to convey intent, and it also helps when tracking down exceptions, as the stack trace will show the function name, instead of showing up as “anonymous function.” A named function will be easier to identify and save you headaches when debugging.

· Remove unnecessary callbacks, such as the one after reporting the status in the example. If a callback is only executed at the end of the function, and not asynchronously, you can get rid of it. The code that used to be in the callback could come right after the function call.

· Be careful about mixing conditionals with flow control code. Conditionals hinder your ability to follow a piece of code, because new possibilities are introduced, and you need to think of all the possible ways in which the code might be followed. Flow control presents a similar problem. It makes it harder to read through the code, because the next instruction isn’t always the following line. Anonymous callbacks containing conditionals make it particularly hard to follow the flow, and they should be avoided. The first example in section 6.1 is a good demonstration of how this mixture is a recipe for disaster. You can mitigate this problem by separating the conditionals from the flow control. Provide a reference to the function, instead of an anonymous callback, and you can keep the conditionals as they were.

After making the changes suggested in the previous list, the code ends up like the following listing.

Listing 6.3. Cleaning up the jumble

That’s not that bad; what else?

· The reportStatus function now seems pointless; you could inline its contents, move them to the only call site, and reduce the mental overhead. Simple methods that aren’t going to be reused can be replaced with their contents, reducing cognitive load.

· Sometimes it makes sense to do the opposite, too. Instead of declaring the click handler inline, you could pull it into a named function, making the addEventListener line shorter. This one is mostly a matter of preference, but it can help when lines of code get longer than the 80 character mark.

The next listing shows the resulting code after applying these changes. Although the code is functionally equivalent, it’s gotten much easier to read. Compare it with listing 6.2 to get a clearer picture.

Listing 6.4. Pulling functions

function attach (node, status, done) {

function handler () {

var id = node.dataset.id;

var endpoint = '/api/v1/records/' + id;

http.get(endpoint, updateView);

}

function updateView (res) {

node.innerHTML = res.data.view;

status.innerHTML = 'Status: ' + res.status;

done(res);

}

node.addEventListener('click', handler);

}

attach(record, status, function done (res) {

console.log(res);

});

What you did was make your code read as it flows. The trick is to keep each function as small and focused as you can get away with, as covered in chapter 5. Then it’s a matter of giving the functions proper, descriptive names that clearly state the purpose of the method. Learning when to inline unnecessary callbacks, as you did with report-Status, is a matter of practice.

In general, it won’t matter if the code itself becomes a bit longer, as long as its readability improves. Readability is the single most important aspect of the code you write, because that’s how you’ll spend most of your time: reading code. Let’s go over one more example before moving on.

6.1.3. Requests upon requests

In web applications, it’s not uncommon to have web requests that depend on other AJAX requests; the back end might not be suited to give you all the data you need in a single AJAX call. For example, you might need to access a list of your customer’s clients, but to do that you must first get the customer ID, using their email address, and then you need to get the regions associated with that customer before finally getting the clients associated with that region and that customer.

Let’s look at the following listing (found as ch06/02_requests-upon-requests in the samples) to see how this AJAX-fest might look.

Listing 6.5. Using AJAX for callback nesting

http.get('/userByEmail', { email: input.email }, function (err, res) {

if (err) { done(err); return; }

http.get('/regions', { regionId: res.id }, function (err, res) {

if (err) { done(err); return; }

http.get('/clients', { regions: res.regions }, function (err, res) {

done(err, res);

});

});

});

function done (err, res) {

if (err) { throw err; }

console.log(res.clients);

}

As you’ll learn in chapter 9 while analyzing REST API service design, having to jump through so many hoops to get to the data you need is usually a symptom of client-side code conforming to whatever API the back-end server offers, rather than having a dedicated API that’s specifically built for the front end. In the case I described, it would be best if the server did all that work based off a customer email, rather than making that many round-trips to the server.

Figure 6.1 shows the repeated round-trips to the server, compared with an API dedicated to the front end. As you can see in the figure, with a preexisting API, chances are it won’t fit the needs of your front end, and you’ll have to massage inputs in your browser before handing them off to the API. In the worst-case scenario, you might even have to make multiple requests to get the desired result, meaning extra round-trips. If you had a dedicated API, it would be up for whatever task you ask of it, allowing you to optimize and reduce the number of requests made against the server, reducing server load and eliminating unnecessary round-trips.

Figure 6.1. The trade-offs between resorting to an existing API or using one dedicated to the front end

If you take into account that this code might be inside a closure and also inside an event handler, the indentation becomes unbearable: it’s too hard to follow code through all of those nesting levels, particularly if the methods are long. Naming the callback functions and extracting them, rather than using anonymous functions, is good enough to start refactoring the functionality so it’s easier to understand.

The following listing shows the refactored code as an example of how you might break down nesting.

Listing 6.6. Nesting no more

You can already see how this is easier to understand; the flow is much clearer now that everything is at the same nesting level. You might’ve noticed the pattern where every method checks for errors to ensure the next step isn’t met with any surprises. In the next few sections we’ll look at different ways to handle asynchronous flow in JavaScript:

· Using callback libraries

· Promises

· Generators

· Event emitters

You’ll learn how each of those solutions simplifies error handling. For now, you’ll build on the current sample, figuring out how to get rid of those error checks.

6.1.4. Asynchronous error handling

You should plan for errors, rather than ignore them. You should never let errors go unnoticed. That being said, when using either the callback hell or the named functions approach, it’s tedious to do any sort of error handling. Surely there’s a better way to go about it than adding an error handling line to each of your functions.

In chapter 5 you learned about different ways you can manipulate function invocation, such as using .apply, .call, and .bind. Imagine you could get away with writing a line such as the following code and have that get rid of repeated error-checking statements, while still checking for them, but in one place. Wouldn’t that be great?

flow([getUser, getRegions, getClients], done);

In the previous statement, the flow method takes an array of functions and executes each one in turn. Each function is passed a next argument that it should invoke when it’s done. If the first argument passed to next is “truthy” (JavaScript slang for any value that’s not false, 0, '', null, or undefined), then done gets called immediately, interrupting the flow.

The first argument is reserved for errors. If that argument is truthy, then you’ll short-circuit and call done directly. Otherwise, the next function in the array is called, and it gets passed all the arguments provided to next, except for the error, plus a new next callback function that allows the following method to continue chaining. Pulling that off does seem difficult.

First, you’ll ask consumers of the flow method to call a next callback when the method is done working. That’ll help with the flow part. You’ll have to provide that callback method and have it call the next function in the list, passing it all the arguments that were used to call next. You’ll append a new next callback, which will call the following method, and so on.

Figure 6.2 explains the flow function you’re going to implement.

Figure 6.2. Understanding an asynchronous flow method

Before you implement your flow method, let’s look at a full usage example. This is what you were doing previously, finding clients for a particular customer, but you’re not doing the error checking in every step anymore; the flow method will take care of that. The following listing shows what using flow would look like.

Listing 6.7. Using the flow method

Keeping in mind what we’ve discussed, let’s look at the implementation of the flow function. Adding a guard clause ensures that calling next multiple times on any given step doesn’t have a negative effect. Only the first call to next will be taken into consideration. A flowimplementation can be found in the following listing.

Listing 6.8. Implementing the asynchronous series flow method

Experiment and follow the flow on your own, and if you get lost, keep in mind that the next() method merely returns a function that has an effect once. If you didn’t want to include that safeguard, you could reuse that same function every step of the way. This approach, however, accounts for programming mistakes by the consumers where they might call next twice during the same step.

Maintaining methods such as flow to keep them up-to-date and bug-free can be cumbersome if all you want is to avoid the nesting hell of a callback-based asynchronous flow and get error handling to go with that. Luckily, smart people have implemented this and many other asynchronous flow patterns into a JavaScript library called async, and also baked it into popular web frameworks such as Express, too. We’ll go over control flow paradigms in this chapter, such as callbacks, Promises, events, and generators. Next up, you’ll get acquainted with async.

6.2. Using the async library

In the world of Node, many developers find it hard not to use the async control flow library. Native modules, those that are part of the Node platform itself, follow the pattern where the last argument taken by a function is a callback that receives an error as its first argument. The following code snippet illustrates the case in point, using Node’s file system API to read a file asynchronously:

require('fs').readFile('path/to/file', function (err, data) {

// handle the error, use data

});

The async library provides many asynchronous control flow methods, much like the one in section 6.1.3, when you built the flow utility method. Your flow method is much the same as async.waterfall. The difference is that async provides tens of these methods that can simplify your asynchronous code if applied correctly.

You can get async from either npm or Bower, or from GitHub.[1] While you’re on GitHub, you might want to check the excellent piece of documentation that Caolan McMahon (async’s author) has written.

1 You can download async from GitHub at https://github.com/caolan/async.

In the following subsections we’ll go into detail about the async control flow library, discussing problems you might run into and how async can solve those for you, making it easier for you, and everyone else, to read the code. To get started, you’ll look at three slightly different flow control methods: waterfall, series, and parallel.

6.2.1. Waterfall, series, or parallel?

One of the most important aspects of mastering asynchronous JavaScript is learning about all the different tools at your disposal, and you certainly will in this chapter. One such tool is common control flow techniques:

· Do you want to run tasks asynchronously so they don’t depend on each other to do their thing? Run them concurrently using .parallel.

· Do your tasks depend on the previous ones? Run them in series, one after the other, but still asynchronously.

· Are your tasks tightly coupled? Use a waterfall mechanism that lets you pass arguments to the next task in the list. The HTTP cascade we discussed earlier is a perfect use case for waterfall.

Figure 6.3 compares the three alternatives in greater detail.

Figure 6.3. Comparison of parallel, series, and waterfall in the async library.

As you can see in the figure, subtle differences exist between these three strategies. Let’s break them down.

Concurrent

Concurrent task execution is most helpful when you have a few different asynchronous tasks that have no interdependencies, but where you still need to do something when all of them finish; for example, when fetching different pieces of data to render a view. You can define a concurrency level, or how many tasks can be busy while the rest are queued up:

· Once a task finishes, another one is grabbed from the queue until the queue is emptied.

· Each task is passed a special next method that should be invoked when processing completes.

· The first argument passed to next is reserved for errors; if you pass in an error, no further tasks will be executed (although the ones already executing will run to completion).

· The second argument is where you’ll pass in the results for your task.

· Once all tasks end, the done callback is invoked. Its first argument will be the error (if any), and the second one will have the results sorted by the tasks, regardless of how long they took to finish.

Series

Sequential execution helps you connect correlative tasks, meant to be executed one by one, even if the code execution happens asynchronously, outside of the main loop. Think of the series flow as the concurrent flow with its concurrency level set to 1. In fact, that’s exactly what it is! The same conventions of next(err, results) and done(err, results) apply.

Waterfall

The waterfall variant is similar to sequential execution, but it allows you to easily roll the arguments from one task to the next in a cascade. This type of flow is most useful when tasks can only be initiated using the data provided by the response of other tasks. In the case of waterfall the flow is different in that next takes in an error followed by any number of result arguments: next(err, result1, result2, result...n). The done callback behaves in this exact same way, giving you all of the arguments that were passed to the last next callback.

Next, let’s get more insight into how series and parallel work.

Flow control in series

You’ve already seen waterfall in action, in the flow method you implemented. Let’s talk about series, which is a slightly different approach from what waterfall does. It executes the steps in series, one at a time the way waterfall does, but it doesn’t fiddle with the arguments of each step function. Instead, each step only receives a next callback argument, expecting the (err, data) signature. You might wonder, “How does that help me?” The answer to that is sometimes the consistency of having a single argument, and having that argument be a callback, is useful. Consider the following listing as an illustrative example of how async.series works.

Listing 6.9. Using async.series

async.series([

function createUser (next) {

http.put('/users', user, next);

},

function listUsers (next) {

http.get('/users/list', next);

},

function updateView (next) {

view.update(next);

}

], done);

function done (err, results) {

// handle error

updateProfile(results[0]);

synchronizeFollowers(results[1]);

}

Sometimes the results need to be manipulated individually, the way you did in the previous listing. In those cases, it makes more sense to use an object to describe the tasks rather than an array. If you do that, the done callback will get a results object, mapping results to the property name for each task. This sounds complicated, but it isn’t, so let’s modify the code in the following listing to illustrate the point.

Listing 6.10. Using the done callback

async.series({

user: function createUser (next) {

http.put('/users', user, next);

},

list: function listUsers (next) {

http.get('/users/list', next);

},

view: function updateView (next) {

view.update(next);

}

}, done);

function done (err, results) {

// handle error

updateProfile(results.user);

synchronizeFollowers(results.list);

}

If a task merely involves calling a function that takes arguments and the next callback, you could use async.apply to shorten your code; that’ll make it easier to read. The apply helper will take the method you want to call and the arguments you want to use and return a function that takes a next callback and appends that to your argument list. The two approaches shown in the following code snippets are functionally equivalent:

function (next) {

http.put('/users', user, next);

}

async.apply(http.put, '/users', user)

// <- [Function]

The following code is a simplified version of the task flow you put together previously, using async.apply:

async.series({

user: async.apply(http.put, '/users', user),

list: async.apply(http.get, '/users/list'),

view: async.apply(view.update)

}, done);

If you used waterfall, this kind of optimization wouldn’t have been possible. The function created by async.apply expects only a next argument but nothing else. In waterfall flows, tasks can get passed an arbitrary number of arguments. In contrast, in a series, tasks always receive exactly one argument, the next callback.

Concurrent flow control

Then there’s async.parallel. Running tasks concurrently works exactly like running tasks in series does, except you don’t chip away at tasks one at a time, but rather run them all at the same time. Concurrent flows result in faster execution time, making parallel the favorite when you don’t have any specific requirements for your workflow other than asynchronicity.

The async library also provides functional methods, allowing you to loop through lists, map objects to something else, or sort them. Next we’ll look at these functional methods and an interesting task queue functionality built into async.

6.2.2. Asynchronous functional tasks

Suppose you need to go through a list of product identifiers and fetch their object representations over HTTP. That’s an excellent use case for a map. Maps transform input into output using a function that modifies the input. The following listing (available as ch06/05_async-functional in the samples) shows how it’s done using async.map.

Listing 6.11. Transforming input into output with maps

var ids = [23, 33, 118];

async.map(ids, transform, done);

function transform (id, complete) {

http.get('/products/' + id, complete);

}

function done (err, results) {

// handle the error

// results[0] is the response for ids[0],

// results[1] is the response for ids[1],

// and so on

}

At the point done is called, it will either have an error argument as the first argument, which you should handle, or an array of results as the second argument, which will be in the same order as the list you provided when calling async.map. A few methods behave similarly to map inasync. They’ll take in an array and a function, apply the function to every item in the array, and then call done with the results.

For instance, async.sortBy allows you to sort an array in place (meaning it won’t create a copy), and all you need to do is pass in a value as the sort criteria for the done callback of the function. You could use it as shown in the following listing.

Listing 6.12. Sorting an array

async.sortBy([1, 23, 54], sort, done);

function sort (id, complete) {

http.get('/products/' + id, function (err, product) {

complete(err, product ? product.name : null);

});

}

function done (err, result) {

// handle the error

// result contains ids sorted by name

}

Both map and sortBy are based on each, which you can think of as parallel, or series if you use the eachSeries version. each merely loops through an array and applies a function to each element; then an optional done callback is invoked that has an error argument telling you if something went wrong. The following listing shows an example of using async.each.

Listing 6.13. Using async.each

async.each([2, 5, 6], iterator, done);

function iterator (item, done) {

setTimeout(function () {

if (item % 2 === 0) {

done();

} else {

done(new Error('expected divisible by 2'));

}

}, 1000 * item);

}

function done (err) {

// handle the error

}

More methods in the async library deal with functional situations, all of which revolve around asynchronously transforming an array into an alternative representation of its data. We won’t cover the rest of them, but I encourage you to look at the extensive documentation on GitHub.[2]

2 You can find the flow control library async on GitHub at https://github.com/caolan/async.

6.2.3. Asynchronous task queues

Moving on to the last method, async.queue, this method will create a queue object that can be used to run tasks in series or concurrently. It takes two arguments: the worker function, which will take a task object and a callback to signal that the work is complete, and the concurrency level, which determines how many tasks can run at any given moment.

If the concurrency level is 1, you’re effectively turning the queue into a series, executing tasks as the previous one ends. Let’s create a simple queue in the following listing (labeled ch06/06_async-queue in the samples).

Listing 6.14. Creating a simple queue

var q = async.queue(worker, 1);

function worker (id, done) {

http.get('/users/' + id, function gotResponse (err, user) {

if (err) { done(err); return; }

console.log('Fetched user ' + id);

done();

});

}

You can use the q object to put your queue to work. To add a new job to the queue, use q.push. You’ll need to pass a task object, which is what gets passed to the worker; in our case the task is a numeric literal, but it could be an object or even a function; and an optional callback, which gets called when this particular job is done. Let’s see how to do that in code:

var id = 24;

q.push(id, function (err) {

if (err) {

console.error('Error processing user 23', err);

}

});

That’s it. The nicety is that you can push more tasks at different points in time, and it’ll still work. In contrast, parallel or series are one-shot operations where you can’t add tasks to the list at a later time. That being said, our last topic regarding the async control flow library is about composing flows and creating task lists dynamically—both of which may bring further flexibility to your approach.

6.2.4. Flow composition and dynamic flows

At times, you’ll need to craft more advanced flows where

· Task b depends on task a

· While task c needs to be performed afterward

· And task d can be executed in parallel to all of that

When all of it is done, you’ll run a last task: task e.

Figure 6.4 shows what that flow might look like:

Figure 6.4. Dissection of a complex asynchronous flow. Hint: always group tasks, in your brain, according to their requirements.

· Tasks A (getting on a bus) and B (paying the bus fare) need to be executed in waterfall, as task B depends on the outcome of task A.

· Task C (getting to your workplace) needs to be executed in series, after tasks A and B have been resolved. It depends on both of them, but not directly.

· Task D (reading a book) doesn’t have any dependencies, so it can be executed in parallel to tasks A, B, and C.

· Task E (working) depends on both task C and task D, so it has to be executed once those tasks are finished.

This sounds, and looks, more complicated than it is. All you need to do, provided you’re using a control flow library such as async, is write a few functions on top of each other. That could look like the pseudo-code shown in the following example. Here I’m using async.apply, introduced in section 6.2.1, to make the code shorter. A fully documented sample can be found at ch06/07_async-composition in the samples:

async.parallel([

async.apply(async.series, [

async.apply(async.waterfall, [getOnBus, payFare]),

getToWork

]),

readBook

], doWork);

Composing flows in this way is most useful if you’re writing Node.js applications, which involve many async operations, such as querying a database, reading files, or connecting to an external API, all of which can often result in highly complex, asynchronous operation trees.

Composing flows dynamically

Creating flows dynamically, by adding tasks to an object, allows you to put together task lists that would’ve been much harder to organize without using a control flow library. This is a nod to the fact that you’re writing code in JavaScript, a dynamic language. You can exploit that by coding up dynamic functions, so do so! The following listing takes a list of items and maps each of them to a function, which then queries something with that item.

Listing 6.15. Mapping and querying a list of items

var tasks = {};

items.forEach(function queryItem (item) {

tasks[item.name] = function (done) {

item.query(function queried (res) {

done(null, res);

});

};

});

function done (err, results) {

// results will be organized by name

}

async.series(tasks, done);

A lightweight alternative to async

There’s something I’d like to mention about async regarding client-side usage. async was originally developed mostly for the Node.js community, and, as such, it isn’t as rigorously tested for the browser.

I built my own version, contra, which has an extensive suite of unit tests that get executed before every release. I kept the code in contra to a minimum; it’s 10 times smaller than async, making it ideal for the browser. It provides methods that can be found on async, as well as a simple way to implement event emitters, which are explained in section 6.4. You can find it on GitHub,[a] and it’s available on both npm and Bower.

a Get contra, my flow control library at https://github.com/bevacqua/contra, on GitHub.

Let’s move on to Promises, a way to deal with asynchronous programming by chaining functions together, and dealing in contracts. Have you used jQuery’s AJAX functionality? Then you’ve worked with a flavor of Promises called Deferred, which is slightly different than the official ES6 Promises implementation, but fundamentally similar.

6.3. Making Promises

Promises are an up-and-coming standard, and are in fact part of the official ECMAScript 6 draft specification. Currently you can use Promises by including a library, such as Q, RSVP.js, or when. You could also use Promises by adding the ES6 Promises polyfill.[3] A polyfill is a piece of code that enables technology that you’d expect the language runtime to provide natively. In this case, a polyfill for Promises would provide Promises, as they’re natively supposed to work in ES6, made available to previous implementations of the ES standard.

3 Find the ES6 Promises polyfill at http://bevacqua.io/bf/promises.

In this section, I’ll describe Promises per ES6, which you can use today, provided you include the polyfill. The syntax varies slightly if you’re using something other than the polyfill for Promises, but these variations are subtle enough, and the core concepts remain the same.

6.3.1. Promise fundamentals

Creating a Promise involves a callback function that takes fulfill and reject functions as its arguments. Calling fulfill will change the state of the Promise to fulfilled; you’ll see what that means in a minute. Calling reject will change the state to rejected. The following code is a brief and self-explaining Promise declaration where your Promise will be fulfilled half of the time and rejected the other half:

var promise = new Promise(function logic (fulfill, reject) {

if (Math.random() < 0.5) {

fulfill('Good enough.');

} else {

reject(new Error('Dice roll failed!'));

}

});

As you might’ve noticed, Promises don’t have any inherent properties that make them exclusively asynchronous, and you can also use them for synchronous operations. This comes in handy when mixing synchronous and asynchronous code, because Promises don’t care about that. Promises start out in pending, and once they fail or succeed, they’re resolved and can’t change state anymore. Promises can be in one of three mutually exclusive states:

· Pending: Hasn’t fulfilled or rejected yet.

· Fulfilled: The action relating to the Promise succeeded.

· Rejected: The action relating to the Promise failed.

Promise continuation

Once you create a Promise object, you can add callbacks to it via the then(success, failure) method. These callbacks will be executed accordingly when the Promise is resolved. When the Promise is fulfilled, or if it’s already fulfilled, the success callback will be called, and if it’s rejected or if it’s already rejected, failure will be invoked.

Figure 6.5 illustrates how Promises can be rejected or fulfilled and how Promise continuation works.

Figure 6.5. Promise continuation basics

There are a few takeaways from figure 6.5. First, remember that when creating a Promise object you’ll take both fulfill and reject callbacks, which you can then use to resolve the Promise. Calling p.then(success, fail) will execute success when and if the Promise is fulfilled, and fail when and if the Promise is rejected. Note that both callbacks are optional, and you could also use p.catch(fail) as syntactic sugar for p.then(null, fail).

The following expanded listing shows the then continuation calls added to our previous example. You can find it under ch06/08_promise-basics in the code samples.

Listing 6.16. Promise with continuation calls

You can invoke promise.then as many times as you please, and all the callbacks in the correct branch (either success or rejection) will be invoked when the Promise is resolved, in the same order in which they were added. If the code was asynchronous, maybe if a setTimeout or anXMLHttpRequest was involved, the callbacks that depend on the outcome of the Promise won’t be executed until the Promise resolves, as shown in the following listing. Once the Promise is resolved, callbacks passed to p.then(success, fail) or p.catch(fail) will be executed immediately, where appropriate: success callbacks will only be executed if the Promise was fulfilled, and fail callbacks will only be executed if the Promise was rejected.

Listing 6.17. Executing Promises

var promise = new Promise(function logic (fulfill, reject) {

console.log('Pending...');

setTimeout(function later () {

if (Math.random() < 0.5) {

fulfill('Good enough.');

} else {

reject(new Error('Dice roll failed!'));

}

}, 1000);

});

promise.then(function success (result) {

console.log('Succeeded', result);

}, function fail (reason) {

console.log('Rejected', reason);

});

Besides creating different branches by invoking .then multiple times on a Promise object, you could also chain those callbacks together, altering the result each time. Let’s look into Promise chaining.

Promise transformation chains

This is where things get harder to understand, but let’s go step by step. When you chain callbacks, they get whatever the previous one returned. Consider the following listing, where the first callback will parse the JSON value resolved by the Promise into an object, and the following callback prints whether buildfirst is true on that object.

Listing 6.18. Using a transformation chain

Chaining callbacks to transform previous values is useful, but it won’t do you any good if what you need is to chain asynchronous callbacks. How can you chain Promises that perform asynchronous tasks? We’ll look at that next.

6.3.2. Chaining Promises

Instead of returning values in your callbacks, you could also return other Promises. Returning a Promise has an interesting effect, where the next callback in the chain will wait until the returned Promise is completed. In preparation for your next example, where you’ll query the GitHub API for a list of users and then get the name of one of their repositories, let’s sketch out a Promise wrapper of the XMLHttpRequest object, the native browser API uses to make AJAX calls.

A bare AJAX call

The specifics of how the XMLHttpRequest works are outside of the scope of this book, but the code should be self-explanatory. The following listing shows how you could make an AJAX call using minimal code.

Listing 6.19. Making an AJAX call

var xhr = new XMLHttpRequest();

xhr.open('GET', endpoint);

xhr.onload = function loaded () {

if (xhr.status >= 200 && xhr.status < 300) {

// get me the response

} else {

// report error

}

};

xhr.onerror = function errored () {

// report error

};

xhr.send();

It’s a matter of passing in an endpoint, setting an HTTP method—GET, in this case—and doing something with the results asynchronously. That’s a perfect opportunity to turn AJAX into a Promise.

Promising AJAX data

You don’t have to change the code at all, other than appropriately wrapping the AJAX call in a Promise and calling resolve and reject as necessary. The following listing depicts a possible get implementation, which provides access to the XHR object through the use of Promises.

Listing 6.20. Promising AJAX

Once that’s out of the way, putting together the sequence of calls leading up to the name of a repository looks bafflingly easy. Notice how you’re mixing asynchronous calls thanks to Promises, and synchronous calls by using then transformations. Here’s what the code looks like, taking into account the get method you implemented:

get('https://api.github.com/users')

.catch(function errored () {

console.log('Too bad. That failed.');

})

.then(JSON.parse)

.then(function getRepos (res) {

var url = 'https://api.github.com/users/' + res[0].login + '/repos';

return get(url).catch(function errored () {

console.log('Oops! That one failed.');

});

})

.then(JSON.parse)

.then(function print (res) {

console.log(res[0].name);

});

You could’ve packed the JSON.parse method in the get method, but it felt like a good opportunity to display how you might mix and match asynchronous and synchronous operations using Promises.

This is great if you want to do operations similar to what you did with async.waterfall in section 6.2.1, where each task was fed the results from the previous one. What about using another flow control mechanism you got from async? Read on!

6.3.3. Controlling the flow

Flow control with Promises is arguably as easy as flow control using a library such as async. If you want to wait on a collection of Promises before doing another task, the way you did with async.parallel, you could wrap the Promises in a Promise.all call, as shown in the following listing.

Listing 6.21. Promising to pause

The delay(Math.min.apply(Math, results)) Promise will be run only after all the previous Promises have resolved successfully; also note how then(results) gets passed an array of results containing the result of each Promise. As you might’ve inferred from the .then call,Promise.all(array) returns a Promise which will be fulfilled when all the items in array are fulfilled.

Using Promise.all is particularly useful when executing long-running operations, such as a series of AJAX calls, because you wouldn’t want to make them in series if you could make them all at once. If you know all of the request endpoints, make the requests concurrently rather than serially. Then, once those requests are done, you can finally compute whatever depended on performing those asynchronous requests.

Functional programming using promises

To perform functional tasks such as the ones provided by methods such as async.map or async.filter, you’re better off using the native Array methods when using Promises. Rather than resorting to a Promise-specific implementation, you can use a .then call to transform the results into what you need. Consider the following listing, using the same delay function as above, which takes results above 400 and then sorts them.

Listing 6.22. Using the delay function to sort results

As you can see, mixing synchronous and asynchronous operations using Promises couldn’t be easier, even when functional operations or AJAX requests are involved. You’ve been looking at the happy path so far, where everything works fine, but how exactly should you approach sensible error handling when using Promises?

6.3.4. Handling rejected Promises

You can provide rejection handlers by passing a callback function as the second argument to a .then(success, failure) call, as you examined in section 6.3.1. Similarly, using .catch(failure) makes it easier to convey intent, and it’s an alias for .then(undefined, failure).

Until now we’ve talked in terms of explicit rejections, as in rejections when you explicitly call reject in the callback passed to the Promise constructor, but that’s not your only option.

Let’s examine the example below, which includes error throwing and handling. Note that I’m using throw in the Promise, although you should use the more semantic reject argument to display that you can throw exceptions from the original Promise as well as in then calls.

Listing 6.23. Catching and throwing

function delay (t) {

function wait (fulfill, reject) {

if (t < 1) {

throw new Error('Delay must be greater than zero.');

}

setTimeout(function later () {

console.log('Resolving after', t);

fulfill(t);

}, t);

}

return new Promise(wait);

}

Promise

.all([delay(0), delay(400)])

.then(function resolved (result) {

throw new Error('I dislike the result!');

})

.catch(function errored (err) {

console.log(err.message);

});

If you execute this example, you’ll notice how the error thrown by the delay(0) Promise will prevent the success branch from firing, therefore never showing the 'I dislike the result!' message. But if delay(0) wasn’t there, then the success branch would throw another error, which would prevent further progress in the success branch.

At this point, you’ve looked at callback hell and how to avert it. You’ve looked at asynchronous flow control using the async library, and you’ve also dealt with flow control using Promises, which is coming in ES6, but is already widely available through other libraries and polyfills.

Next up we’ll discuss events, which are a form of asynchronous JavaScript that I’m sure you’ve come across when dealing with JavaScript development at one point or another. Later, you’ll check out what else is coming in ES6 in terms of asynchronous flow. Namely, you’ll look at ES6 generators, which are a novel feature to deal with iterators lazily, similar to what you can find in languages such as C# in their enumerable implementation.

6.4. Understanding events

Events are also known as publish/subscribe or event emitters. An event emitter is a pattern where a component emits events of certain types and passes them arguments, and any interested parties can subscribe to events of interest and react to the event and the provided arguments. Many different ways exist to implement an event emitter, most of which involve prototypal inheritance in one way or another. But you could also attach the necessary methods to an existing object, as you’ll see in section 6.4.2.

Events are natively implemented in browsers, too. Native events might be an AJAX request getting a response, a human interacting with the DOM, or a WebSocket carefully listening for any action coming its way. Events are asynchronous by nature, and they’re sprinkled all over the browser, so it’s your job to manage them appropriately.

6.4.1. Events and the DOM

Events are one of the oldest asynchronous patterns of the web, and you can find them in the bindings that connect the browser DOM with your JavaScript code. The following example registers an event listener which will be triggered every time the document body gets clicked:

document.body.addEventListener('click', function handler () {

console.log('Click responsibly. Do not click and drive!');

});

DOM events are more often than not triggered by a human being who clicks, pans, touches, or pinches on their browser window. DOM events are hard to test for if they aren’t abstracted well enough. Even in the trivial case displayed below, consider the implications of having an anonymous function handling the click event:

document.body.addEventListener('click', function handler () {

console.log(this.innerHTML);

});

It’s hard to test functionality like this because you have no way to access the event handler independently from the event. For easier testing, and to avoid the hassle of simulating clicks to test the handler (which should still be done in integration testing, as you’ll see in chapter 8), it’s recommended that you either extract the handler into a named function, or you move the main body of the logic into a testable named function. This also favors reusability in case two events can be handled in the same way. The following piece of code shows how the click handler could be extracted:

function elementClick handler () {

console.log(this.innerHTML);

}

var element = document.body;

var handler = elementClick.bind(element);

document.body.addEventListener('click', handler);

Thanks to Function.prototype.bind you’re keeping the element as part of the context. Arguments exist both in favor of and against using this in this way. You should pick the strategy you’re most comfortable with and stick to it. Either always bind handlers to the relevant element or always bind handlers using a null context. Consistency is one of the most important traits of readable (and maintainable) code.

Next up you’ll implement your own event emitter, where you’ll attach the relevant methods to an object without using prototypes, making for an easy implementation. Let’s investigate what that might look like.

6.4.2. Creating your own event emitters

Event emitters usually support multiple types of events, rather than a single one. Let’s implement step by step your own function to create event emitters or improve existing objects as event emitters. In a first step, you’ll either return the object unchanged or create a new object if one wasn’t provided:

function emitter (thing) {

if (!thing) {

thing = {};

}

return thing;

}

Using multiple event types is powerful and only costs you an object to store the mapping of event types to event listeners. Similarly, you’ll use an array for each event type, so you can bind multiple event listeners to each event type. You’ll also add a simple function that registers event listeners. The following listing (found as ch06/11_event-emitter in the samples) displays how you could turn existing objects into event emitters.

Listing 6.24. Promoting objects to event emitter status

Now you can add event listeners once an emitter is created. This is how it works. Keep in mind that listeners can be provided with an arbitrary number of arguments when an event is fired; you’ll implement the method to fire events next:

var thing = emitter();

thing.on('change', function changed () {

console.log('thing changed!');

});

Naturally, that works like a DOM event listener. All you need to do now is implement the method that fires the events. Without it, there wouldn’t be an event emitter. You’ll implement an emit method that allows you to fire the event listeners for a particular event type, passing in an arbitrary number of arguments. The following listing shows how it looks.

Listing 6.25. Firing event listeners

thing.emit = function emit (type) {

var evt = events[type];

if (!evt) {

return;

}

var args = Array.prototype.slice.call(arguments, 1);

for (var i = 0; i < evt.length; i++) {

evt[i].apply(thing, args);

}

};

The Array.prototype.slice.call(arguments, 1) statement is an interesting one. Here you’ll apply Array.prototype.slice on the arguments object and tell it to start at index 1. This does two things. It casts the arguments object into a true array, and it gives a nice array with all of the arguments that were passed into emit, except for the event type, which you don’t need to invoke the event listeners.

Executing listeners asynchronously

There’s one last tweak to do, which is executing the listeners asynchronously so they don’t halt execution of the main loop if one of them blows up. You could also use a try/catch block here, but let’s not get involved with exceptions in event listeners; let the consumer handle that. To achieve this, use a setTimeout call, as shown in the following listing.

Listing 6.26. Event emission

thing.emit = function emit (type) {

var evt = events[type];

if (!evt) {

return;

}

var args = Array.prototype.slice.call(arguments, 1);

for (var i = 0; i < evt.length; i++) {

debounce(evt[i]);

}

function debounce (e) {

setTimeout(function tick () {

e.apply(thing, args);

}, 0);

}

};

You can now create emitter objects, or you can turn existing objects into event emitters. Note that, given that you’re wrapping the event listeners in a timeout, if a callback throws an error, the rest will still run to completion. This isn’t the case in synchronous implementations of event emitters, as an error will stop execution on the current code path.

As a fun experiment, I’ve prepared a listing using event emitters and thoroughly exploiting Function.prototype.bind in the following listing. Can you tell how it works and why?

Listing 6.27. Using event emitters

var beats = emitter();

var handleCalm = beats.emit.bind(beats, 'ripple', 10);

beats.on('ripple', function rippling (i) {

var cb = beats.emit.bind(beats, 'ripple', --i);

var timeout = Math.random() * 150 + 50;

if (i > 0) {

setTimeout(cb, timeout);

} else {

beats.emit('calm');

}

});

beats.on('calm', setTimeout.bind(null, handleCalm, 1000));

beats.on('calm', console.log.bind(console, 'Calm...'));

beats.on('ripple', console.log.bind(console, 'Rippley!'));

beats.emit('ripple', 15);

Obviously this is a contrived example that doesn’t do much, but it’s interesting how two of the listeners control the flow, while the others control the output, and a single emit fires an unstoppable chain of events. As usual, you’ll find a fully working copy of this snippet in the accompanying samples under ch06/11_event-emitter. While you’re at it, make sure to read the samples for all the previous examples!

The power of event emitters stems from their flexibility, and one possible way to use emitters is by inverting their meaning. Imagine you control a component with an event emitter, and you expose the emit functionality, rather than the “listen” functionality. Your component can now be passed by arbitrary messages, and process them, while at the same time it might also emit its own events and let others process them, resulting in effective communication across components.

I have one last topic for you in this chapter: ES6 generators. Generators are a special kind of function in ES6 that can be iterated over lazily, and provide amusing value. Let’s inspect them more closely.

6.5. Glimpse of the future: ES6 generators

JavaScript generators, heavily inspired by Python, are an interesting new perk coming our way, which allow you to represent sequences of values, such as the Fibonacci sequence, on which you can iterate. Although you’re already capable of iterating over arrays, generators are lazy. Lazy is good because it means it’s possible to create an infinite sequence generator and iterate over it without falling victim to an infinite loop or stack overflow exception. Generator functions are denoted with an asterisk, and items in the sequence must be returned using the yield keyword.

6.5.1. Creating your first generator

In the following listing you’ll see how to create a generator function that represents a never-ending Fibonacci sequence. By definition, the first two numbers in the series are 1 and 1, and each subsequent number is the sum of the previous two.

Listing 6.28. Using a Fibonacci sequence

function* fibonacci () {

var older = 0;

var old = 1;

yield 1;

while (true) {

yield old + older;

var next = older + old;

older = old;

old = next;

}

}

Once you have a generator, you may want to consume the values it produces, and to do that, you need to call the generator function, which will give you an iterator. The iterator can be used to get values from the generator, one at a time, by calling iterator.next(). That function call will result in an object such as { value: 1, done: false } for iterators using the generator in the previous listing. The done property will become true when the iterator’s done going through the generator function, but in this example it would never finish because of the infinitewhile(true) loop. The following example demonstrates how you could iterate over a few values using the never-ending fibonacci generator:

var iterator = fibonacci();

var i = 10;

var item;

while (i--) {

item = iterator.next();

console.log(item.value);

}

The easiest way to run the examples in this section is visiting http://es6fiddle.net, which will run ES6 code for you, including anything that uses generators. Alternatively, you could get Node v0.11.10 or later, which you can easily fetch from https://nodejs.org/dist. Then, doing node --harmony <file> when executing a script will enable ES6 features such as generators, including the function* () construct, the yield keyword, and the for..of construct, which comes next.

Iterate using for..of

The for..of syntax allows you to shortcut the process of iterating over a generator. Normally you’d call iterator.next(), store or use the provided result.value, and then check iterator.done to see if the iterator is exhausted. The for..of syntax handles that for you, trimming down your code. The following is a representation of iterating over a generator with a for..of loop. Note that you’re using a finite generator, because using a generator such as fibonacci would create an infinite loop, unless you use break to exit the loop:

function* keywords () {

yield 'buildfirst';

yield 'javascript';

yield 'design';

yield 'architecture';

}

for (keyword of keywords()) {

console.log(keyword);

}

At this point you might wonder how generators can help you deal with asynchronous flows, and we’re about to get to that. First, however, we need to go back to generator functions and explain what suspension means.

Execution suspension in generators

Let’s look at the first generator example again:

function* fibonacci () {

var older = 1;

var old = 0;

while (true) {

yield old + older;

older = old;

old += older;

}

}

How does that work? Why doesn’t it get stuck in an infinite loop? Whenever a yield statement is executed, execution in the generator gets suspended and relinquished back to the consumer, passing them the value which was yielded. That’s how iterator.next() gets a value. Let’s inspect this behavior more closely using a simple generator, which has side effects:

function* sentences () {

yield 'going places';

console.log('this can wait');

yield 'yay! done';

}

When you iterate over a generator sequence, execution in the generator will be suspended (pausing its execution until the next item in the sequence is requested) immediately after each yield call. This allows you to execute side effects such as the console.log statement in the previous example, as a consequence of calling iterator.next() for the second time. The following snippet shows how iterating over the previous generator would flow:

var iterator = sentences();

iterator.next();

// <- 'going places'

iterator.next();

// logged: 'this can wait'

// <- 'yay! done'

Armed with your newfound knowledge about generators, next you’ll try to figure out how to turn the tables and build an iterator that can consume generators in a way that makes asynchronous code easier to write.

6.5.2. Asynchronicity and generators

Let’s build an iterator that can exploit suspension well enough to combine synchronous and asynchronous flows in a seamless manner. How could you accomplish a flow method that would allow you to implement functionality such as that in the following listing (ch06/13_generator-flow)? In this listing, you use yield on a method that needs to be executed asynchronously, and then you invoke a next function that would be provided by the flow implementation once you’ve fetched all the needed food types. Note how you’re still using the callback convention where the first argument is either an error or a false value.

Listing 6.29. Building an iterator to exploit suspension

flow(function* iterator (next) {

console.log('fetching food types...');

var types = yield get;

console.log('waiting around...');

yield setTimeout(next, 2000);

console.log(types.join(', '));

});

function get (next) {

setTimeout(function later () {

next(null, ['bacon', 'lettuce', 'crispy bacon']);

}, 1000);

}

To make the previous listing work, you need to create the flow method in such a way that it allows yield statements to pause until next gets called. The flow method would take a generator as an argument, such as the one in the previous listing, and iterate through it. The generator should be passed a next callback so you can avoid anonymous functions, and you can, alternatively, yield functions that take a next callback and have the iterator pass the next callback to them as well. The consumer can let the iterator know that it’s time to move on by calling next(). Then execution would get unsuspended and pick up where it left off.

You can find how a flow function might be implemented in the following listing. It works much like the iterators you’ve seen so far, except that it also has the capability to let the generator function, which gets passed into flow, do the sequencing. The key aspect of this asynchronous generator pattern is the back-and-forth enabled by letting the generator suspend (by using yield) and unsuspend (by invoking next) the flow of the iteration.

Listing 6.30. Generator flow implementation

Using the flow function you can easily mix flows, and have the flow leap into (and out of) asynchronous mode easily. Going forward you’ll use a combination of plain old JavaScript callbacks and control flow using the contra library, which is a lightweight alternative to async.

6.6. Summary

That was a lot of ground to cover, so you might want to take a break for a minute and check out the source code samples and play around with them a little more.

· We established what callback hell is, and you learned how to stay away from it by naming your functions or composing your own flow control methods.

· You learned how to use async to meet different needs such as asynchronous series, mapping asynchronously, or creating asynchronous queues. You delved into the world of Promises. You understand how to create a Promise, how to compose multiple Promises, and how to mix and match asynchronous and synchronous flows.

· You took an implementation-agnostic look at events and learned how to implement your own event emitters.

· I gave you a glimpse of what’s coming in ES6 with Generators, and how you might use them to develop asynchronous flows.

In chapter 7 you’ll take a harder look at client-side programming practices. We’ll discuss the current state of interacting with the DOM, how to improve on that, and what the future holds in terms of componentized development. We’ll detail implications of using jQuery, how it might not be your one-size-fits-all library, and a few alternatives you can gravitate toward. You’ll also get your hands dirty with BackboneJS, an MVC framework.