D3.js in Action (2015)

Part 2. The pillars of information visualization

Chapter 8. Traditional DOM manipulation with D3

This chapter covers

· Making spreadsheets with data

· Drawing graphics with HTML5 canvas

· Building image galleries with data

· Populating drop-down lists with data-binding

Many introductions to D3 start with sizing <div> elements to create a bar chart. They figure you’re a web developer and that you won’t be as intimidated by a div as you’d be by an SVG rectangle. This book even begins by creating a set of <p> elements in chapter 1, the first time you saw data-binding in action. But then these tutorials (and this book) quickly transition into the creation of SVG elements, with an emphasis on the graphical display of information. This is at odds with traditional web development, which focuses on the presentation of blocks of text, images, buttons, lists, and so on. As a result, it seems like D3 is for data visualization, but somehow not for manipulating traditional DOM elements like paragraphs, divs, and lists (like those seen in figure 8.1). The benefit of using D3 to create these kinds of elements is that you can use D3 transitions, data-binding, and other functionality to make a more interactive and dynamic website.

Figure 8.1. The traditional DOM-based pieces that are created in this chapter are a spreadsheet (section 8.2) and an image gallery (section 8.4), with interactivity based on a data-driven drop-down list (section 8.4.2) and images drawn using HTML5 canvas (8.3).

The principles at work in D3 not only can be used for traditional DOM elements, but in many cases should be used that way. In this chapter, we’ll use D3 to create a spreadsheet as well as an image gallery. We’ll also explore how to use HTML5 canvas to draw and save images. This won’t include an exhaustive example of canvas, because that’s beyond the scope of the book, but it’ll give you the fundamentals to deploy it in tandem with D3 in your applications. In each case, we’ll use D3 data-binding, transitions, and selections the same way we did to make charts, but instead make more traditional HTML elements.

By using the same datasets and functions to deal with your DOM elements as you do with your SVG elements, you make it easier to tie them together and reduce the amount of syntax you need to learn to deploy rich sites. In later chapters, you’ll see these different methods of presenting information working in together in tandem.

8.1. Setup

As you may expect, we need to make a few changes to the files we’re working with, now that we’re going to do coding that resembles more traditional web development. In one case, this means simplifying, because our HTML page loses the <svg> element necessary for representing SVG graphics, but in another sense it means making things more complex. Although we used CSS primarily for graphical changes with SVG, we need to use it for more than that when working with traditional DOM elements.

8.1.1. CSS

You use more CSS when you work with traditional DOM elements, because if you want to manipulate them in the way you manipulate SVG elements, you typically need to set them up a bit differently; for instance, if you want to place HTML elements precisely like you do with SVG elements. Also, most of the graphical aspects of these elements aren’t set with attributes like in SVG, but with styles (we covered the difference between styles, attributes, and properties back in chapter 1). This shouldn’t be a surprise for anyone who’s had experience working with CSS, because it’s usually the case in the complex examples and under the hood when you use JavaScript libraries. For example, if you look at the CSS of various libraries that provide autocomplete or more sophisticated UI elements, you’ll see that they typically combine JavaScript with a variety of styles assigned to complex CSS selectors. In the following listing you’ll see the style sheet we’ll use for this chapter. Some of these elements, like <img.infinite>, you won’t see until the end of the chapter.

Listing 8.1. Style sheet for chapter 8

tr {

border: 1px gray solid;

}

td {

border: 2px black solid;

}

div.table {

position:relative;

}

div.data {

position: absolute;

width: 90px;

padding: 0 5px;

}

div.head {

position: absolute;

}

div.datarow {

position: absolute;

width: 100%;

border-top: 2px black solid;

background: white;

height: 35px;

overflow: hidden;

}

div.gallery {

position: relative;

}

img.infinite {

position: absolute;

background: rgba(255,255,255,0);

border-width: 1px;

border-style: solid;

border-color: rgba(0,0,0,0);

}

8.1.2. HTML

The HTML is pretty simple: a single <div> with the ID value of "traditional" in your <body> element, as shown in the following listing. You still need a reference to d3.js, but otherwise it’s a Spartan HTML page. You’ll either modify or add new elements to that div for every example.

Listing 8.2. chapter8.html

<!doctype html>

<html>

<body>

</div>

</body>

</html>

8.2. Spreadsheet

Let’s assume we want to take the tweets data that we’ve been working with throughout the book and present it as a spreadsheet. It may help to first think of spreadsheets as a kind of information visualization. They have an x-axis (columns) and a y-axis (rows) and visual channels to express information (not only color applied to text and cells but also position and font styling). This is especially true of large spreadsheets, because they also use aggregated functions to tally results.

8.2.1. Making a spreadsheet with table

The easiest way to make a spreadsheet is to use the HTML <table> element and data-binding to create rows and cells. As we’ve done previously, we create key values by using d3.keys on one of the entries in our dataset, which will be the venerable tweets.json. After we bind the dataset to the table, we need to create individual cells. We can accomplish this by taking each JSON object and applying d3.entries() to it, which turns an object into an array of key-value pairs perfectly suited for D3 data-binding.

Listing 8.3. Spreadsheet example

The result of listing 8.3 is a decent tabular presentation of our tweets data, as shown in figure 8.2. Notice that the arrays have been transformed into comma-delimited strings.

Figure 8.2. A tabular display of the data found in tweets.json using <table>, <tr>, and <td> elements

It’s a simple task to take data and bind it to create traditional DOM elements in the same way we bound data to create SVG elements. We could have created an <ol> element and appended <li> elements to it from our dataset just as easily. We can also use D3’s .on function to assign event listeners to highlight cells or rows by changing their background or font color. But rather than do that with a spreadsheet built using <table>, we’ll build another spreadsheet entirely out of <div> elements.

8.2.2. Making a spreadsheet with divs

Why use <div> elements? Because we’re going to start moving our cells and rows around however we want, and by the time we override all the styles that make a table and its constituent elements work, we’re better off starting fresh with a div. By setting the <div> position to absolute, we can use D3 transitions to move them around in the same way we moved SVG around in our earlier examples. We need to apply a bit more CSS to make the <div> elements take up the right amount of space, whereas <table> does that for us, but the added flexibility is worth it. A quick note for those of you who, like me, always forget the one crazy rule of positioning DOM elements: elements set to position:relative need to have a parent set to position:relative or position:absolute. We’ll create a parent <div> (div.table) with position:relative to hold the <div> elements that make up our table.

Listing 8.4. A spreadsheet made of divs

This code has some obvious oversimplifications. As shown in figure 8.3, it doesn’t make much sense to have each column the same width. Although we could create a method for measuring the maximum size of the text in that field, that’s not where we’ll go in this chapter. I want to show a general overview of manipulating elements like these rather than create the ultimate D3 spreadsheet.

Figure 8.3. Our improved spreadsheet built with <div> elements. You can see how each div is the same width. Because of our overflow settings, it displays as much of the text as it can.

8.2.3. Animating our spreadsheet

It’s time now to add interactivity to the static chart shown in figure 8.3. One traditional interaction technique applied to spreadsheets is the ability to sort them. We can do that with our spreadsheet by sorting the data and rebinding it to the cells, just like we did previously with SVG elements. By tying this to the same transition() behavior we used before, we can also animate that sorting.

Listing 8.5. Sorting functions

We have a spreadsheet with sortable rows that float over and under each other after we click the sorting button. Figure 8.4 shows that animation caught in an intermediate state. If we want to sort the columns, though, we need to do something slightly different.

Figure 8.4. The rows of your spreadsheet in the middle of the sort function.

Listing 8.6. Column sorting

d3.select("#traditional").insert("button", ".table")

.on("click", sortColumns).html("sort columns ");

d3.select("#traditional").insert("button", ".table")

.on("click", restoreColumns).html("restore columns");

function sortColumns() {

d3.selectAll("div.datarow")

.selectAll("div.data")

.transition()

.duration(2000)

.style("left", function(d,i,j) {

return (Math.abs(i - 4) * 100) + "px";

});

};

function restoreColumns() {

d3.selectAll("div.datarow")

.selectAll("div.data")

.transition()

.duration(2000)

.style("left", function(d,i,j) {

return (i * 100) + "px";

});

};

There you have it—a sortable animated spreadsheet that, if you catch it in midtransition as I have in figure 8.5, is rather messy. It’s animated, interactive, and data-driven with no SVG at all. Rather than adding more interactivity to our spreadsheet, we’ll switch gears and focus on a second kind of traditional component of a web page: image galleries. But before we get to that, we’ll need some images. Instead of loading them from external files, we’ll draw our own PNGs using HTML5 canvas, an API made for drawing static images. We’re not going to dive deep into canvas, but just use it to create circles with numbers on them to stand in for whatever images we might put in a gallery.

Figure 8.5. Sorting columns in our sheet. Because we didn’t define a background value for the divs, the text floats over itself. In this screenshot, you can see that I’ve added all the buttons for sorting and restoring columns and divs.

8.3. Canvas

We won’t use canvas too much here, but you should recognize that, although the canvas drawing syntax like that in listing 8.7 is different from SVG, it’s something you could easily tie to D3. You may do that because you want to create images like we’re doing here. Or you may use canvas because you can achieve greatly improved performance if you’re dealing with large datasets. A number of online examples use canvas instead of SVG for D3 (especially with maps like the one at http://bl.ocks.org/mbostock/3783604, but also the implementation of a Voronoi diagram in canvas at http://bl.ocks.org/mbostock/6675193). But for our purposes, we don’t need much code to create our image.

We’ll use canvas to draw circles with numbers in them. We’ll do this so we can have a set of images that we can use for our gallery. Your gallery probably has a set of images in a directory or called from an API, but because we don’t have that here, we’ll create them on the fly. At the same time, you’ll get a sense of the functionality of the canvas API in regard to how it can be used alongside D3.

8.3.1. Drawing with canvas

The first thing we’ll draw with canvas won’t use much D3 code. What little it does use, such as d3.select() and .node(), could easily be replaced with native JavaScript. Later, when we start drawing many different images, and pass those images on to other elements, you’ll see the kind of D3 functionality you’ve grown used to.

Listing 8.7. Canvas drawing code

d3.select("#traditional")

.append("canvas")

.attr("height", 500)

.attr("width", 500);

var context = d3.select("canvas").node().getContext("2d");

context.strokeStyle = "black";

context.lineWidth = 2;

context.fillStyle = "red";

context.beginPath();

context.arc(250,250,200,0,2*Math.PI);

context.fill();

context.stroke();

context.textAlign = "center";

context.font="200px Georgia";

context.fillStyle = "black";

context.textAligh = "center";

context.fillText("1",250,250);

The result is the circle in figure 8.6. You’ll notice a few important differences from the code we used earlier. First, we hardly use D3 in this example. We could easily have skipped it entirely by using the built-in selectors in core JavaScript. Second, we draw with canvas, not on an <svg>element, but on a <canvas> element that needs to be created in the DOM. Third, canvas has a syntax distinct from SVG.

Figure 8.6. A circle and text drawn using HTML5 canvas

But there’s one more major difference between the graphics created using canvas and the graphics created using SVG. You can see it if you inspect that circle, as shown in figure 8.7. Anything drawn in canvas is drawn to a bitmap, so you don’t have an individual text or circle element that you could assign an event listener to, or whose appearance or text content you can later modify. It’s also not vector-based, so if you try to zoom the image, you’ll see the pixilation you’re familiar with from zooming photos and other raster imagery. Because HTML5 canvas doesn’t create separate DOM elements, it benefits from higher performance when dealing with large amounts of those graphical elements. But you lose the flexibility of SVG.

Figure 8.7. Any graphics created in canvas are stored as a bitmap or raster image. Unlike in SVG, the individual shapes are no longer accessible or modifiable after being drawn.

8.3.2. Drawing and storing many images

We want images because our plan is to build an image gallery, but the canvas element in the DOM doesn’t act like the kind of image that you’re accustomed to dealing with in web development. You can’t right-click and save it, or open it in a new window in its current form. But the<canvas> element includes a .toDataURL() function that provides a string designed to be the src attribute of an <img> element. You can see in the following listing the results of .toDataURL() when applied to one of your drawn circles. This is only the first three lines—the actual value would go on like this for nine pages.

Listing 8.8. Sample toDataURL() output

data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAfQAAAH0CAYAAADL1t+KAAAgAElEQVR

4Xu2dC3xV1ZX/171B1JJggNoSsSY+QrWiQnB4dCoEH7Tgg4dVdNRCWg3SqQVm+i99TIfQmc7UPkbU

9sNDW0KVWluFYCl0FIdAW99AALWWUE2sCtWCgQQfkdz73+smV1NIyH3sc89ae//258MnKOf

In our new example in the following listing, we create 100 circles of varying colors with varying borders. We then use .toDataURL to create an array of values that can be bound to <img> elements to create our first gallery of one hundred images.

Listing 8.9. Drawing 100 circles with canvas

As shown in figure 8.8, each of our slightly different circles is turned into a PNG and assigned to an <img> element. We can also use toDataURL() to create JPEGs by specifying that format, but by default it creates PNGs. Because they’re <img> elements now, they resize automatically. Even though we only specified the height of the images, the <img> element by default proportionately scaled the width of the image so that it wouldn’t distort. Because of the float:left setting on those elements, they easily fill the div we created for them. And because it’s an <img>, we can do anything with these that we normally could with an image on a web page, including save them or open them in a new tab.

Figure 8.8. The final canvas-drawn circle (top) remains in our <canvas> element, and every variation according to the settings as an image in a div (bottom).

That’s not much of an image gallery, though. We’ll continue to expand on this code in the next section, and also make something a bit more interesting by taking advantage of the interaction and animation techniques we’ve already used.

8.4. Image gallery

You’ve spent time learning canvas so that you could make image elements for a gallery. When spec’ing out an image gallery, keep in mind a few features that everyone wants. First, you want more control over where you place images. Instead of using float, we’ll do the same thing we did with the spreadsheet divs in section 8.2.2 and use position:absolute along with top: and left: to place them like our div cells and rows or the SVG elements that we used in previous chapters. You also want images to cleanly fit the space you provide, and you want those images to grow or shrink if the user changes the size of the window.

For all these examples, we’ll use the same method described in listing 8.9 to create the imageArray dataset that we’ll use. The figures in this chapter will have slight variation from the results of running this code, because we randomly generate some of the visual elements. We can create our first gallery with surprisingly little code.

Listing 8.10. Resizing eight-image gallery

As shown in figure 8.9, this produces a scrollable div with eight images per line. The images not only scale to fit the div, but rescale as you adjust your browser window. The imgX and imgY functions create an object for each image that stores an x value. This should remind you of D3 layout accessor functions. We’ll build something more involved like this in chapter 9 and dive into writing layouts in chapter 10, but for this example we won’t try to create an image gallery layout.

Figure 8.9. Automatically scaled-to-fit images that pack eight images per row

8.4.1. Interactively highlighting DOM elements

From here, we can add interactivity, such as making an image expand on mouseover. The process is rather simple.

Listing 8.11. Expand image on mouseover

If you’re a savvy web developer, you’ve probably spotted an artifact from working with SVG in the code above. It’s the appendChild trick that we need to use to make SVG elements draw in front of each other. We’re using relative and absolute positioned DOM elements, so we don’t need this, because CSS has a z-index that allows elements to be drawn in front of each other. But I wanted to keep appendChild to remind you that working with traditional DOM elements has benefits that SVG elements don’t.

Another reason to use the DOM rather than a z-index for positioning is to highlight the array position value in the accessor functions in D3. You may think that the array position corresponds to the array position of a datapoint in the original JavaScript array that we bound to the selection, but it doesn’t. It corresponds to the array position of the DOM element in the selection. When you start to use appendChild to shift elements up and down the DOM, you change that array value. When we first created imageArray, we set the x value equal to the original array position, anddidn’t use array position to place the individual gallery images. This is why redrawGallery keeps drawing images in the right place, even after we start shifting images around in the DOM by mousing over them.

When you run the code in listing 8.11, D3’s transitions are smart enough to process the rgba string designating a transparent background, as shown in figure 8.10. In some cases, like the next example, you may have to use D3’s tweening capabilities to make sure that a DOM element interpolates properly. It probably doesn’t follow the rules that make shape and color transitions work so easily. Still, with color and simple size transitions, you can use exactly the same code for <div> elements that we used with <rect> and <circle> elements, unless you’re trying to transition to "height:auto" or some other nonnumerical value.

Figure 8.10. One of our gallery images in mid-transition. A border and background are added for UX purposes—the transparent regions of a PNG still register mouse2 events, and so the user should be reminded of the effective region for mouseover events.

8.4.2. Selecting

Our final example adds a drop-down list to select a particular image and scroll the gallery to the row that holds that image. To do so, we need to populate the <select> element with choices that correspond to our images, and write a function that scrolls the gallery to the correct line. If you know how a <select> element works (it has a bunch of <option> elements nested underneath it in the DOM), you should guess how to do this with D3 using imageArray as your data. But creating the scrolling function is a bit more involved because we need to write a custom tween to scroll the <div> element that contains our gallery.

Listing 8.12. Zoom to a specific image from a select input

This produces a gallery like that in figure 8.11. If we wanted to deploy this gallery, we’d need to do some cleaning up. But the purpose of this chapter is to demonstrate how you can use D3 functionality to work with bitmaps, divs, and other traditional materials of web design. Notice that we’ll need to adjust some of our workflows and syntax and also integrate CSS more, but buttons, images, and paragraphs can be data-driven and have the same kind of graphical and interactive sophistication as the geometric shapes you’ll work with more often.

Figure 8.11. Selecting an image from the list scrolls the div to the proper location and increases its size.

I tend to use D3 for my traditional DOM elements not only because of the flexibility, but because it uses the same syntax and abstractions. As a result, it’s easy to do things like create a view of data as a bulleted list to go along with a map.

8.5. Summary

In this chapter, you saw the potential of using D3 to create dynamic content with traditional DOM elements. You can embed your more traditional SVG-based data visualization in a web page that’s equally dynamic, or create dynamic web pages that don’t have any SVG at all. Specifically, this chapter focused on

· Using D3 to create and manipulate traditional DOM elements like <table>, <select>, <div>, and <img>

· Creating interactive and dynamic spreadsheets and galleries

· Getting a taste of the HTML5 canvas API

· Taking a closer look at tweening and transitions

In chapter 9 you’ll see how you can tie together multiple visualizations and traditional DOM elements with custom events to create your first interactive data-driven application that will examine tweets using multiple views into the data. Although we haven’t combined traditional DOM elements and SVG data visualization in this chapter, you’ll see that in the next chapter as we put sparklines in our spreadsheets and divs in our tree diagrams.