Learning JavaScript (2016)
Chapter 12. Iterators and Generators
ES6 introduces two very important new concepts: iterators and generators. Generators depend on iterators, so we’ll start with iterators.
An iterator is roughly analogous to a bookmark: it helps you keep track of where you are. An array is an example of an iterable object: it contains multiple things (like pages in a book), and can give you an iterator (which is like a bookmark). Let’s make this analogy concrete: imagine you have an array called book where each element is a string that represents a page. To fit the format of this book, we’ll use Lewis Carroll’s “Twinkle, Twinkle, Little Bat” from Alice’s Adventures in Wonderland (you can imagine a children’s book version with one line per page):
const book = [
"Twinkle, twinkle, little bat!",
"How I wonder what you're at!",
"Up above the world you fly,",
"Like a tea tray in the sky.",
"Twinkle, twinkle, little bat!",
"How I wonder what you're at!",
];
Now that we have our book array, we can get an iterator with its values method:
const it = book.values();
To continue our analogy, the iterator (commonly abbreviated as it) is a bookmark, but it works only for this specific book. Furthermore, we haven’t put it anywhere yet; we haven’t started reading. To “start reading,” we call the iterator’s next method, which returns an object with two properties: value (which holds the “page” you’re now on) and done, which becomes false after you read the last page. Our book is only six pages long, so it’s easy to demonstrate how we can read it in its entirety:
it.next(); // { value: "Twinkle, twinkle, little bat!", done: false }
it.next(); // { value: "How I wonder what you're at!", done: false }
it.next(); // { value: "Up above the world you fly,", done: false }
it.next(); // { value: "Like a tea tray in the sky.", done: false }
it.next(); // { value: "Twinkle, twinkle, little bat!", done: false }
it.next(); // { value: "How I wonder what you're at!", done: false }
it.next(); // { value: undefined, done: true }
it.next(); // { value: undefined, done: true }
it.next(); // { value: undefined, done: true }
There are a couple of important things to note here. The first is that when next gives us the last page in the book, it tells us we’re not done. This is where the book analogy breaks down a little bit: when you read the last page of a book, you’re done, right? Iterators can be used for more than books, and knowing when you’re done is not always so simple. When you are done, note that value is undefined, and also note that you can keep calling next, and it’s going to keep telling you the same thing. Once an iterator is done, it’s done, and it shouldn’t ever go back to providing you data.1
While this example doesn’t illustrate it directly, it should be clear to you that we can do things between the calls to it.next(). In other words, it will save our place for us.
If we needed to enumerate over this array, we know we can use a for loop or a for...of loop. The mechanics of the for loop are simple: we know the elements in an array are numeric and sequential, so we can use an index variable to access each element in the array in turn. But what of the for...of loop? How does it accomplish its magic without an index? As it turns out, it uses an iterator: the for...of loop will work with anything that provides an iterator. We’ll soon see how to take advantage of that. First, let’s see how we can emulate a for...of loop with a whileloop with our newfound understanding of iterators:
const it = book.values();
let current = it.next();
while(!current.done) {
console.log(current.value);
current = it.next();
}
Note that iterators are distinct; that is, every time you create a new iterator, you’re starting at the beginning, and it’s possible to have multiple iterators that are at different places:
const it1 = book.values();
const it2 = book.values();
// neither iterator have started
// read two pages with it1:
it1.next(); // { value: "Twinkle, twinkle, little bat!", done: false }
it1.next(); // { value: "How I wonder what you're at!", done: false }
// read one page with it2:
it2.next(); // { value: "Twinkle, twinkle, little bat!", done: false }
// read another page with it1:
it1.next(); // { value: "Up above the world you fly,", done: false }
In this example, the two iterators are independent, and iterating through the array on their own individual schedules.
The Iteration Protocol
Iterators, by themselves, are not that interesting: they are plumbing that supports more interesting behavior. The iterator protocol enables any object to be iterable. Imagine you want to create a logging class that attaches timestamps to messages. Internally, you use an array to store the timestamped messages:
class Log {
constructor() {
this.messages = [];
}
add(message) {
this.messages.push({ message, timestamp: Date.now() });
}
}
So far, so good…but what if we want to then iterate over the entries in the log? We could, of course, access log.messages, but wouldn’t it be nicer if we could treat log as if it were directly iterable, just like an array? The iteration protocol allows us to make this work. The iteration protocol says that if your class provides a symbol method Symbol.iterator that returns an object with iterator behavior (i.e., it has a next method that returns an object with value and done properties), it is then iterable! Let’s modify our Log class to have a Symbol.iterator method:
class Log {
constructor() {
this.messages = [];
}
add(message) {
this.messages.push({ message, timestamp: Date.now() });
}
[Symbol.iterator]() {
return this.messages.values();
}
}
Now we can iterate over an instance of Log just as if it were an array:
const log = new Log();
log.add("first day at sea");
log.add("spotted whale");
log.add("spotted another vessel");
//...
// iterate over log as if it were an array!
for(let entry of log) {
console.log(`${entry.message} @ ${entry.timestamp}`);
}
In this example, we’re adhering to the iterator protocol by getting an iterator from the messages array, but we could have also written our own iterator:
class Log {
//...
[Symbol.iterator]() {
let i = 0;
const messages = this.messages;
return {
next() {
if(i >= messages.length)
return { value: undefined, done: true };
return { value: messages[i++], done: false };
}
}
}
}
The examples we’ve been considering thus far involve iterating over a predetermined number of elements: the pages in a book, or the messages to date in a log. However, iterators can also be used to represent object that never run out of values.
To demonstrate, we’ll consider a very simple example: the generation of Fibonacci numbers. Fibonacci numbers are not particularly hard to generate, but they do depend on what came before them. For the uninitiated, the Fibonacci sequence is the sum of the previous two numbers in the sequence. The sequence starts with 1 and 1: the next number is 1 + 1, which is 2. The next number is 1 + 2, which is 3. The fourth number is 2 + 3, which is 5, and so on. The sequence looks like this:
§ 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89, 144,...
The Fibonacci sequence goes on forever. And our application doesn’t know how many elements will be needed, which makes this an ideal application for iterators. The only difference between this and previous examples is that this iterator will never return true for done:
class FibonacciSequence {
[Symbol.iterator]() {
let a = 0, b = 1;
return {
next() {
let rval = { value: b, done: false };
b += a;
a = rval.value;
return rval;
}
};
}
}
If we used an instance of FibonacciSequence with a for...of loop, we’ll end up with an infinite loop…we’ll never run out of Fibonacci numbers! To prevent this, we’ll add a break statement after 10 elements:
const fib = new FibonacciSequence();
let i = 0;
for(let n of fib) {
console.log(n);
if(++i > 9) break;
}
Generators
Generators are functions that use an iterator to control their execution. A regular function takes arguments and returns a value, but otherwise the caller has no control of it. When you call a function, you relinquish control to the function until it returns. Not so with generators, which allow you to control the execution of the function.
Generators bring two things to the table: the first is the ability to control the execution of a function, having it execute in discrete steps. The second is the ability to communicate with the function as it executes.
A generator is like a regular function with two exceptions:
§ The function can yield control back to the caller at any point.
§ When you call a generator, it doesn’t run right away. Instead, you get back an iterator. The function runs as you call the iterator’s next method.
Generators are signified in JavaScript by the presence of an asterisk after the function keyword; otherwise, their syntax is identical to regular functions. If a function is a generator, you can use the yield keyword in addition to return.
Let’s look at a simple example—a generator that returns all the colors of the rainbow:
function* rainbow() { // the asterisk marks this as a generator
yield 'red';
yield 'orange';
yield 'yellow';
yield 'green';
yield 'blue';
yield 'indigo';
yield 'violet';
}
Now let’s see how we call this generator. Remember that when you call a generator, you get back an iterator. We’ll call the function, and then step through the iterator:
const it = rainbow();
it.next(); // { value: "red", done: false }
it.next(); // { value: "orange", done: false }
it.next(); // { value: "yellow", done: false }
it.next(); // { value: "green", done: false }
it.next(); // { value: "blue", done: false }
it.next(); // { value: "indigo", done: false }
it.next(); // { value: "violet", done: false }
it.next(); // { value: undefined, done: true }
Because the rainbow generator returns an iterator, we can also use it in a for...of loop:
for(let color of rainbow()) {
console.log(color):
}
This will log all the colors of the rainbow!
yield Expressions and Two-Way Communication
We mentioned earlier that generators allow two-way communication between a generator and its caller. This happens through the yield expression. Remember that expressions evaluate to a value, and because yield is an expression, it must evaluate to something. What it evaluates to are the arguments (if any) provided by the caller every time it calls next on the generator’s iterator. Consider a generator that can carry on a conversation:
function* interrogate() {
const name = yield "What is your name?";
const color = yield "What is your favorite color?";
return `${name}'s favorite color is ${color}.`;
}
When we call this generator, we get an iterator, and no part of the generator has been run yet. When we call next, it attempts to run the first line. However, because that line contains a yield expression, the generator must yield control back to the caller. The caller must call next again before the first line can resolve, and name can receive the value that was passed in by next. Here’s what it looks like when we run this generator through to completion:
const it = interrogate();
it.next(); // { value: "What is your name?", done: false }
it.next('Ethan'); // { value: "What is your favorite color?", done: false }
it.next('orange'); // { value: "Ethan's favorite color is orange.", done: true }
Figure 12-1 shows the sequence of events as this generator is run.
Figure 12-1. Generator example
This example demonstrates that generators are quite powerful, allowing the execution of functions to be controlled by the caller. Also, because the caller can pass information into the generator, the generator can even modify its own behavior based on what information is passed in.
NOTE
You can’t create a generator with arrow notation; you have to use function*.
Generators and return
The yield statement by itself doesn’t end a generator, even if it’s the last statement in the generator. Calling return from anywhere in the generator will result in done being true, with the value property being whatever you returned. For example:
function* abc() {
yield 'a';
yield 'b';
return 'c';
}
const it = count();
it.next(); // { value: 'a', done: false }
it.next(); // { value: 'b', done: false }
it.next(); // { value: 'c', done: true }
While this is correct behavior, keep in mind that things that use generators don’t always pay attention to the value property when done is true. For example, if we use this in a for...of loop, “c” won’t be printed out at all:
// will print out "a" and "b", but not "c"
for(let l of abc()) {
console.log(l);
}
WARNING
I recommend that you do not use return to provide a meaningful value in a generator. If you expect a useful value out of a generator, you should use yield; return should only be used to stop the generator early. For this reason, I generally recommend not providing a value at all when you call return from a generator.
Conclusion
Iterators provide a standard mechanism for collections or objects that can provide multiple values. While iterators don’t provide anything that wasn’t possible prior to ES6, they do standardize an important and common activity.
Generators allow functions that are much more controllable and customizable: no longer is the caller limited to providing data up front, waiting for the function to return, and then receiving the result of the function. Generators essentially allow computation to be deferred, and performed only as necessary. We will see in Chapter 14 how they provide powerful patterns for managing asynchronous execution.
1Because objects are responsible for providing their own iteration mechanism, as we’ll see shortly, it’s actually possible to create a “bad iterator” that can reverse the value of done; that would be considered a faulty iterator. In general, you should rely on correct iterator behavior.