Tests: The key to confident code - Real-world recipes - Node.js in Practice (2015)

Node.js in Practice (2015)

Part 2. Real-world recipes

Chapter 10. Tests: The key to confident code

This chapter covers

· Assertions, custom assertions, and automated testing

· Ensuring things fail as expected

· Mocha and TAP

· Testing web applications

· Continuous integration

· Database fixtures

Imagine that you wanted to add a new currency to an online shop. First you’d add a test to define the expected calculations: subtotal, tax, and the total. Then you’d write code to make this test pass. This chapter will help you learn how to write tests by looking at Node’s built-in features for testing: the assert module and test scripts that you can set in your package.json file. We also introduce two major test frameworks: Mocha and node-tap.

Introduction to testing

This chapter assumes you have some experience at writing unit tests. Table 10.1 includes definitions of the terminology used; if you want to know what we mean by assertions, test cases, or test harnesses, you can refer to this table.

Table 10.1. Node testing concepts

Term

Description

Assertion

A logical statement that allows you to test expressions. Supported by the assert core module; for example: assert.equal(user.email, 'name@example.com');.

Test case

One or more assertions that test a particular concept. In Mocha, a test case looks like this: it('should calculate the square of a number', function() {
assert.equal(square(4), 16);
});

Test harness

A program that runs tests and collates output. The resulting reports help diagnose problems when tests fail. This builds on the previous example, so with Mocha a test harness looks like this: var assert = require('assert');
var square = require('./square');

describe('Squaring numbers', function() {
it('should calculate the square of a number', function() {
assert.equal(square(4), 16);
});

it('should return 0 for 0', function() {
assert.equal(square(0), 0);
});
});

Fixture

Test data that is usually prepared before tests are run. Let’s say you want to test a user accounts system. You could predefine users and their passwords, and then include the passwords in the tests to ensure users can sign in correctly. In Node, JSON is a popular file format for fixtures, but you could use a database, SQL dump, or CSV file. It depends on your application’s requirements.

Mock

An object that simulates another object. Mocks are often used to replace I/O operations that are either slow or difficult to run in unit tests; for example, downloading data from a remote web API, or accessing a database.

Stub

A method stub is used to replace functionality for the duration of tests. For example, methods used to communicate with an I/O source like a disk or remote API can be stubbed to return predefined data.

Continuous integration server

A CI server runs automated tests whenever a project is updated through a version control server.

For a more detailed introduction to testing, The Art of Unit Testing, Second Edition (Roy Osherove, Manning, 2013; http://manning.com/osherove2/) has step-by-step examples for writing maintainable and readable tests. Test Driven Development: By Example (Kent Beck, Addison-Wesley, 2002; http://mng.bz/UT12) is another wellknown foundational book on the topic.

One of the advantages of working with Node is that the community adopted testing early on, so there’s no shortage of modules to help you write fast and readable tests. You might be wondering what’s so great about tests and why we write them early on during development. Well, tests are important for exploring ideas before committing to them—you can think of them like small, flexible experiments. They also communicate your intent, which means they help document and expand on the ideas in the key parts of the project. Tests can also help reduce maintenance in mature projects by allowing you to check that changes haven’t broken existing working features.

The first thing to learn about is Node’s assert module. This module allows you to define an expectation that will throw an error when it isn’t met. Expressing and confirming expectations is the main purpose of tests, so you’ll see a lot of assertions in this chapter. Although you don’t have to use assert to write tests, it’s a built-in core module and similar to assertion libraries you might’ve used before in other languages. The first set of techniques in this chapter is all about assertions.

To get everyone up to speed, the next section includes a list of common terms used when working with tests.

10.1. Introduction to testing with Node

To make it easier for newcomers to automated testing, we’ve included table 10.1 that defines common terminology. This table also outlines what we mean by specific terms, because some programming communities use the same terms slightly differently.

The only feature from table 10.1 that Node directly supports is assertions. The other features are provided through third-party libraries—you’ll learn about CI servers in technique 86, and mocks and fixtures in technique 87. You don’t have to use all of these things to write tests, you can actually write tests with just the assertion module. The next section introduces the assert module so you can start writing basic tests.

10.2. Writing simple tests with assertions

So far we’ve briefly mentioned that assertions are used to test expressions. But what does this involve? Typically assertions are functions that cause an exception to be raised if a condition isn’t met. A failing assertion is like your credit card being declined in a store—your program will refuse to run no matter how many times you try. The idea of assertions has been around for a long time; even C has assertions.

In C, the standard library includes the assert() macro, which is used for verifying expressions. In Node, we have the assert core module. There are other assertion modules out there, but assert is built-in and easy to use and extend.

CommonJS unit testing

The assert module is based on the CommonJS Unit Testing 1.1 specification (http://wiki.commonjs.org/wiki/Unit_Testing/1.1). So even though it’s a built-in core module, you can use other assertion modules as well. The underlying principles are always the same.

This section introduces Node’s built-in assertions. By following the first technique, you’ll be able to write tests using the assert core module by using assert.equal to check for equality, and to automate the running of tests by using npm scripts.[1]

1 This is defined by the scripts property in a package.json file. See npm help scripts for details on this feature.

Technique 79 Writing tests with built-in modules

Have you ever tried to write a quick test for an important feature, but you found yourself lost in test library documentation? It can be hard to get started actually writing tests; it seems like there’s a lot to learn. If you just start using the assert module, though, you can write tests right now without any special libraries.

This is great when you’re writing a small module and don’t want to install any dependencies. This technique demonstrates how to write clean, expressive, single-file tests.

Problem

You have a clear idea of the acceptable input and output values for your module, class, or functions, and you want it to be clear when the output values don’t match the input.

Solution

Use the assert module and npm scripts.

Discussion

Node comes with an assertion module. You can think of this as a toolkit for checking expectations against outcomes. Internally this is done by comparing actual values against expected values. The assert.equal method demonstrates this perfectly: the arguments are actual, expected. There’s also a third optional argument: message. Passing a message makes it easier to understand what happened when tests fail.

Let’s say you’re writing an online shop that calculates order prices, and you’ve sold three items at $3.99 each. You could ensure the correct price gets calculated with this:

assert.equal(

order.subtotal, 11.97,

'The price of three items at $3.99 each'

);

In methods with only a single required argument, like assert(value), the expected value is true, so it uses the same pattern.

To see what happens when a test fails, try running the next listing.

Listing 10.1. The assert module

The first line you’ll see in most test files is one that loads the assert module . The assert variable is also a function aliased from assert.ok—which means you can use either assert() or assert.ok().

It’s easy to forget the order of the arguments for assert.equal, so you might find yourself checking Node’s documentation a lot. It doesn’t really matter how you order the arguments—some people might find it easier to list the expected value first so they can scan the code for values—but you should be consistent. That’s why this example is explicit about the naming of actual and expected .

This test has a function that has an intentional bug . You can run the test with node assertions.js, which should display an error with a stack trace:

These stack traces can be hard to read. But because we’ve included a message with the assertion that failed, we can see a description of what went wrong. We can also see that the assertion failed in the file assertions.js on line 7 .

The assert module has lots of other useful methods for testing values. The most significant is assert.deepEqual, which can check for equality between two objects. This is important because assert.equal can only compare shallow equality. Shallow equality is used for comparing primitive values like strings or numbers, whereas deepEqual can compare objects with nested objects and values.

You might find deepEqual useful when you’re writing tests that return complex objects. Think about the online shop example from earlier. Your shopping cart might look like this: { items: [ { name: "Coffee beans", price: 4.95 } ], subtotal: 4.95 }. It’s an object that contains an array of shopping cart items, and a subtotal that is calculated by another object. Now, to check this entire object against one that you’ve defined in your unit test, you’d use assert.deepEqual, because it’s able to compare objects rather than just primitive values.

The deepEqual method can be seen in the next listing.

Listing 10.2. Testing object equality

This example uses the assert module to test objects created by a constructor function, and an imaginary login system. The login system is accidentally loading normal users as if they were administrators .

The assert.deepEqual method will go over each property in the objects to see if any are different. When it runs into user.permissions.admin and finds the values differ, an AssertionError exception will be raised.

If you take a look at the assert module’s documentation, you’ll see many other useful methods. You can invert logic with notDeepEqual and notEqual, and even perform strict equality checks just like === with strictEqual and notStrictEqual.

There’s another aspect to testing, and that’s ensuring that things fail the way we expect. The next technique looks at testing for failures.

Technique 80 Testing for errors

Programs will eventually fail, but when they do, we want them to produce useful errors. This technique is about ensuring that expected errors are raised, and about how to cause exceptions to be raised during testing.

Problem

You want to test your error-handling code.

Solution

Use assert.throws and assert.ifError.

Discussion

One of the conventions we use as Node developers is that asynchronous methods should return an error as the first argument. When we design our own modules, we know there are places where errors are likely to occur. Ideally we should test these cases to make sure the correct errors are passed to callbacks.

The following listing shows how to ensure an error hasn’t been passed to an asynchronous function.

Listing 10.3. Handling errors from asynchronous APIs

Although assert.ifError works synchronously, it makes semantic sense to use it for testing asynchronous functions that pass errors to callbacks. Listing 10.3 uses an asynchronous function called readConfigFile to read a configuration file. In reality this might be the database configuration for a web application, or something similar. If the file isn’t found, then it returns default values . Any other error—and this is the important part—will be passed to the callback .

That means the assert.ifError test can easily detect whether an unexpected error has occurred. If something changes in the structure of the project that causes an unusual error to be raised, then this test will catch that and warn the developers before they release potentially dangerous code.

Now let’s look at raising exceptions during testing. Rather than using try and catch in our tests, we can use assert.throws.

To use assert.throws, you must supply the function to be run and an expected error constructor. Because a function is passed, this works well with asynchronous APIs, so you can use it to test things that depend on I/O operations.

The next listing shows how to use assert.throws with a fictitious user account system.

Listing 10.4. Ensuring that exceptions are raised

The assertion checks to ensure the expected exception is thrown. The first argument is a function to test, in this case loginAdmin, and the second is the expected error .

This highlights two things about assert.throws: it can be used with asynchronous APIs because you pass it a function, and it expects error objects of some kind. When developing projects with Node, it’s a good idea to use util.inherits to inherit from the built-in Error constructor. This allows people to easily catch your errors, and you can decorate them with extra properties that include useful additional information if required.

In this case we’ve created PermissionError , which is a clear name and therefore self-documenting—if someone sees a PermissionError in a stack trace, they’ll know what went wrong. A PermissionError is subsequently thrown in the login-Admin function .

This technique delved into error handling with the assert module. Combined with the previous techniques, you should have a good understanding of how to test a range of situations with assertions. With assert.equal you can quickly compare numbers and strings, and this covers a lot of problems like checking prices in invoices or email addresses in web application account-handling code. A lot of the time, assert.ok—which is aliased as assert()—is enough to get by, because it’s a quick and handy way for checking for truthy expressions. But there’s one last thing to master if you want to really take advantage of the assert module; read on to learn how to create custom assertions.

Technique 81 Creating custom assertions

Node’s built-in assertions can be extended to support application-specific expressions. Sometimes you find yourself repeatedly using the same code to test things, and it seems like there might be a better way. For example, suppose you’re checking for valid email addresses with a regular expression in assert.ok. Writing custom assertions can solve this problem, and is easier than you might think. Learning how to write custom assertions will also help you understand the assertion module from the inside out.

Problem

You’re repeating a lot of code in your tests that could be replaced if only you had the right assertion.

Solution

Extend the built-in assert module.

Discussion

The assert module is built around a single function: fail. assert.ok actually calls fail with the logic inverted, so it looks like this: if (!value) fail(value). If you look at how fail works, you’ll see that it just throws an assert.AssertionError:

function fail(actual, expected, message, operator, stackStartFunction) {

throw new assert.AssertionError({

message: message,

actual: actual,

expected: expected,

operator: operator,

stackStartFunction: stackStartFunction

});

}

The error object is decorated with properties that make it easier for test reporters to break down the location and cause of failures. The people who wrote this module knew that others would like to write their own assertions, so the fail function is exported, which means it can be reused.

Writing a custom assertion involves the following steps:

1. Define a method with a signature similar to the existing assertion library.

2. Call fail when an expectation isn’t matched.

3. Test to ensure failure results in an AssertionError.

Listing 10.5 puts these steps together to define a custom assertion that ensures a regular expression is matched.

Listing 10.5. A custom assertion

This example loads the assertion module and then defines a function called match that runs assert.fail to generate the right exception when the regular expression doesn’t match the actual value . The key detail to remember is to define the argument list to be consistent with other methods in the assertion module—the example here is based on assert.equal.

Listing 10.5 also includes some tests. In reality these would be in a separate file, but here they illustrate how the custom assertion works. First we check to see if it passes a simple test by matching a string against a regular expression , and then assert.throws is used to ensure the test really does fail when it’s meant to .

Your own domain-specific language

Using custom assertions is but one technique for creating your own testing DSL (domain-specific language). If you find you’re duplicating code between test cases, then by all means wrap that code in a function or class.

For example, setUpUserAccount({ email: 'user@example.com' }) is more readable than three or four lines of setup code, particularly if it’s repeated between test cases.

This example might seem simple, but understanding how to write custom assertions improves your knowledge of the underlying module. Custom assertions can help clean up tests where expectations have been made less expressive by squeezing concepts into built-in assertions. If you want to be able to say something like assert.httpStatusOK, now you can!

With assertions out of the way, it’s time to look at how to organize tests across multiple files. The next technique introduces test harnesses that can be used to organize groups of test files and run them more easily.

10.3. Test harnesses

A test harness, or automated test framework, generally refers to a program that sets up the runtime environment and runs tests, and then collects and compares the results. Since it’s automated, tests can be run by other systems including continuous integration (CI) servers, covered intechnique 86.

Test harnesses are used to execute groups of test files. That means you can easily run lots of tests with a single command. This not only makes it easier for you to run tests, but makes it easier for your collaborators as well. You may even decide to start all projects with a test harness before doing anything else. The next technique shows you how to make your own test harness, and how to save time by adding scripts to your package.json files.

Technique 82 Organizing tests with a test harness

Suppose you’re working on a project and it keeps on growing, and using a single test file is starting to feel messy. It’s hard to read and causes confusion that leads to mistakes. So you’d like to use separate files that are related in some way. Perhaps you’d even like to run tests one file at a time to help track down issues when things go wrong.

Test harnesses solve this problem.

Problem

You want to write tests organized into test cases and test suites.

Solution

Use a test harness.

Discussion

First, let’s consider what a test harness is. In Node, a test harness is a command-line script that you can run by typing the name of the script. At its most basic, it must run a group of test files and display any errors that occur. We don’t need anything particularly special to do that—a failed assertion will cause an exception to be thrown; otherwise the program will exit silently with a return code of 0.

That means a basic test harness is just node test/*.js, where test/ is a directory that contains a set of test files. We can go one better than that. All Node projects should have a package.json file. One of the properties in this file is scripts, and one of the default scripts is test. Any string you set here will be executed like a shell command.

The following listing shows an example package.json file with a test script.

Listing 10.6. A package.json with a test script

With node test-runner.js test.js test2.js set as the test script , other developers can now run your tests simply by typing npm test. This is much easier than having to remember a project-specific command.

Let’s expand this example by looking at how test harnesses work. A test harness is a Node program that runs groups of test files. Therefore, we should be able to give such a program a list of files to test. Whenever a test fails, it should display a stack trace so we can easily track down the source of the failure.

In addition, it should exit with a non-zero return code whenever a test fails. That allows tests to be run in an automated way—other software can easily see if a test failed without having to parse the textual output from the tests. This is how continuous integration (CI) servers work: they automatically run tests whenever code is committed to a version control system like Git.

The next listing shows what a test file for this system should look like.

Listing 10.7. An example test file

The it function looks strange, but it’s a global function that will be provided by our test framework. It gives each test case a name so it’s easier to understand the results when the tests are run. A failing test is included so we can see what happens when tests fail. The last test case should run even though the second one failed.

Now, the final piece of the puzzle: the next listing includes a program capable of executing this test.

Listing 10.8. Running tests in a prescribed manner

This example can be run by passing test files as arguments: node test-runner.js test.js test2.js test-n.js. The it function is defined as a global , and is called it so the tests and their output read logically. This makes sense when the results are printed .

Because it takes a test case name and a callback, the callback can be run under whatever conditions we desire. In this case we’re running it inside a try/catch statement , which means we can catch failed assertions and report errors to the user.

Tests are loaded by calling require on each of the files passed in as command-line arguments . In a more polished version of this program, the file handling would need to be more sophisticated. Wildcard expressions would need to be supported, for example.

A failed test case causes the exitCode variable to be set to a non-zero value. This is returned to the controlling process with process.exit in the exit handler .

Even though this is a minimal example, it can be run with npm test, gives test cases a little syntax sugar with it, improves the error reporting over a simple file full of assertions, and returns a non-zero exit code when something goes wrong. This is the basis for most popular Node test frameworks like Mocha, which we’ll look at in the next section.

10.4. Test frameworks

If you’re starting a new project, then you should install a test framework early on. Suppose that you’re building an online blogging system, or perhaps a simple content management system. You’d like to allow people to sign in, but only allow specific users to access the administration interface. By using a test framework like Mocha or node-tap, you can write tests that address these specific concerns: users signing up for accounts, and administrators signing in to the admin interface. You could create separate test files for these concerns, or bundle them up as groups of test cases under “user accounts tests.”

Test frameworks include scripts to run tests and other features that make it easier to write and maintain tests. This section features the Mocha test framework in technique 84 and the Test Anything Protocol (TAP; http://testanything.org/) in technique 85—two popular test frameworks favored by the Node community. Mocha is lightweight: it runs tests, provides three styles for structuring test cases,[2] and expects you to use either Node’s assert module or another third-party module. Conversely, node-tap, which implements TAP, uses an API that includes assertions.

2 Mocha supports API styles based on Behavior Driven Development (BDD), Test Driven Development (TDD), and Node’s module system (exports).

Technique 83 Writing tests with Mocha

There are many test frameworks for Node, so it’s difficult to choose the right one. Mocha is a popular choice because it’s well maintained and has the right balance of features and conventions.

In general, you use a test framework to organize tests for a project. You’d like to use a test framework that other people are familiar with so they can easily navigate and collaborate without learning a new module. Perhaps you’re just looking for a way to run tests the same way every time, or trigger them from an automated system.

Problem

You need to organize your tests in a way other developers will be familiar with, and run the tests with a single command.

Solution

Use one of the many open source test frameworks for Node, like Mocha.

Discussion

Mocha must be installed from npm before you can do anything else. The best way to install it is with npm install --save-dev mocha. The --save-dev option causes npm to install Mocha into node_modules/ and update your project’s package.json file with the latest version from npm. It will be saved as a development dependency.

Listing 10.9 shows an example of a simple test written with Mocha. It uses the assert core module to make assertions, and should be invoked using the mocha command-line binary. You should add "./node_modules/mocha/bin/mocha test/*.js" to the "test" property in package.json—see technique 82 for more details on how to do that.

Mocha versions

The version of Mocha we use for this chapter is 1.13.x. We prefer to run the tests by installing it locally to the project rather than as a systemwide Node module. That means that tests can be run using ./node_modules/mocha/bin/mocha test/*.js rather than just typing mocha. That allows different projects to have different versions of Mocha, just in case the API changes dramatically between major releases.

An alternative is to install Mocha globally with npm install --global mocha, and then run tests for a project by typing mocha. It will display an error if it can’t find any tests.

Listing 10.9. A simple Mocha test

The describe and it functions are provided by Mocha. The describe function can be used to group related test cases together, and it contains a collection of assertions that form a test case .

Special handling for asynchronous tests is required. This involves including a done argument in the callback for the test case , and then calling done() when the test has finished . In this example, a timeout will be triggered after a random interval, which means we need to call done in the index.randomTimeout method. The corresponding file under test is shown in the next listing.

Listing 10.10. A sample module to test

Controlling synchronous and asynchronous behavior

If done isn’t included as an argument to it, then Mocha will run the test synchronously. Internally, Mocha looks at the length property of the callback you pass to it to see if an argument has been included. This is how it switches between asynchronous and synchronous behavior. If you include an argument, then Mocha will wait around for done to be called until a timeout is reached.

This module defines two methods: one for squaring numbers and another that runs a callback after a random amount of time . It’s just enough to demonstrate Mocha’s main features in listing 10.9.

To set up a project for Mocha, the index.js file we’ve used in this example should be in its own directory, and at the same level should be a package.json file with a test subproperty of the scripts property set to "./node_modules/mocha/bin/mocha test/*.js". There should also be a test/directory that contains example_test.js.[3] With all that in place, you can run the tests with npm test.

3 The file can be called anything as long as it’s in the test/directory.

When the tests are run, you should notice some dots appearing. These mark a completed test case. When the tests take more than a preset amount of time, they’ll change color to denote they ran slower than is acceptable. Since index.randomTimeout prevents the second test from completing for a random amount of time, there will be times when Mocha thinks the tests are running too slowly. You can increase this threshold by passing --slow to Mocha, like this: ./node_modules/mocha/bin/mocha --slow 2000 test/*.js. Now you don’t need to feel guilty about seemingly slow tests!

Assertions per test

In listing 10.9, each test case has a single assertion. Some consider this best practice—and it can result in readable tests.

But we prefer the idea of a single concept per test. This style structures test cases around well-defined concepts, using the absolute necessary amount of assertions. This will typically be a small number, but occasionally more than one.

To see all of the command-line options, type node_modules/mocha/bin/mocha --help or visit http://mochajs.org/.

We’ve included the final package.json file in listing 10.11 in case you have trouble writing your own. You can install Mocha and its dependencies with npm install.

Listing 10.11. The Mocha sample project’s JSON file

{

"name": "mocha-example-1",

"version": "0.0.0",

"description": "A basic Mocha example",

"main": "index.js",

"dependencies": {},

"devDependencies": {

"mocha": "~1.13.0"

},

"scripts": {

"test": "./node_modules/mocha/bin/mocha --slow 2000 test/*.js"

},

"author": "Alex R. Young",

"license": "MIT"

}

In this technique the assert core module has been used, but you could swap it for another assertion library if you prefer. Others are available, like chai (https://npmjs.org/package/chai) and should.js (https://github.com/visionmedia/should.js).

Mocha is often used for testing web applications. In the next technique, you’ll see how to use Mocha for testing web applications written with Node.

Technique 84 Testing web applications with Mocha

Let’s suppose you’re building a web application with Node. You’d like to test it by running it in a way that allows you to send requests and receive responses—you want to make HTTP requests to test the web application works as expected.

Problem

You’re building a web application and would like to test it with Mocha.

Solution

Write tests with Mocha and the standard http module. Consider using an HTTP module designed for testing to simplify your code.

Discussion

The trick to understanding web application testing in Node is to learn to think in terms of HTTP. This technique starts off with a Mocha test and the http core module. Once you understand the principles at work and can write tests this way, we’ll introduce a third-party HTTP testing module to demonstrate how to simplify such tests. The built-in http module is demonstrated first because it’s useful to see what goes on behind the scenes and to get a handle on exactly how to construct such tests.

The following listing shows what the test looks like.

Listing 10.12. A Mocha test for a web application

This example is a test for a web service that can square numbers. It’s a simple web service that expects GET requests and responds with plain text. The goal of this test suite is to ensure that it returns the expected results and correctly raises errors when invalid data is sent. The tests aim to simulate browsers—or other HTTP clients, for that matter—and to do so, both the server and client are run in the same process.

To run a web service, all you need to do is create a web server with http.create-Server(). Exactly how this is done is shown in listing 10.13. Before discussing that, let’s finish looking at this test.

The test starts by creating a function for making HTTP requests . This is to reduce the amount of duplication that would otherwise be present in the test cases. This function could be its own module, which could be used in other test files. After a request has been sent, it listens for dataevents on the response object to store any data returned by the server . Then it runs the provided callback , which is passed in from the test cases.

Figure 10.1 shows how Node can run both servers and clients in the same process to make web application testing possible.

Figure 10.1. Node can run a web server and requests against it to support web application testing.

An example of this is the test for the /square method that ensures 4 * 4 === 16 . Once that’s done, we also make sure invalid HTTP query parameters cause the server to respond with a 500 error .

The standard assertion module is used throughout, and res.statusCode is used to test the expected status codes are returned.

The implementation of the corresponding web service is shown in the next listing.

Listing 10.13. A web application that can square numbers

Before doing anything else, http.createServer is used to create a server. Near the end of the file, .listen(8000) is used to make the server start up and listen for connections. Whenever a request with a URL matching/square comes in, the URL is parsed for a numerical parameter and then the number is squared and sent to the client . When the expected parameter isn’t present, a 500 is returned instead .

One part of listing 10.12 that can be improved on is the request method. Rather than defining a wrapper around http.request, we can use a library designed specifically for testing with web requests.

The module we’ve chosen is SuperTest (https://github.com/visionmedia/supertest) by TJ Holowaychuk, who also wrote Mocha. There are other similar libraries out there. The general idea is to simplify HTTP requests and allow assertions to be made about the request.

You can add SuperTest to the development dependencies for this example by running npm install --save-dev supertest.

The following listing shows how the test can be refactored using the SuperTest module.

Listing 10.14. The refactored Mocha test that uses SuperTest

Although functionally identical to listing 10.12, this example improves it by removing the boilerplate for making HTTP requests. The SuperTest module is easier to understand, and allows assertions to be expressed with less code while still being asynchronous. SuperTest expects an instance of an HTTP server , which in this case is the application that we want to test. Once the application has been passed to SuperTest’s main function, request, we can then make a GET request using request().get. Other HTTP verbs are also supported, and form parameters can be sent when using post() with the send method.

SuperTest’s methods are chainable, so once a request has been made, we can make an assertion by using expect. This method is polymorphic—it checks the type of the argument and acts accordingly. If you pass it a number , it’ll ensure that the HTTP status was that number. A regular expression will make it check the response body for a match . These expectations are perfect for the requirements of this test.

Any HTTP status can be checked, so when we actually expect a 500, we can test for it .

Though it’s useful to understand how to make simple web applications and test them using the built-in http module, we hope you can see how third-party modules like SuperTest can simplify your code and make your tests clearer.

Mocha captures the zeitgeist of the current state of testing in Node, but there are other approaches that are just as valid. The next technique introduces TAP and the Test Anything Protocol, due to its endorsement by Node’s maintainer and core contributors.

Technique 85 The Test Anything Protocol

Test harness output varies based on programming language and test framework. There are initiatives to unify these reports. One such effort that has been adopted by the Node community is the Test Anything Protocol (http://testanything.org). Tests that use TAP will produce lightweight streams of results that can be consumed by compatible tools.

Suppose you need a test harness that’s compatible with the Test Anything Protocol, either because you have other tools that use TAP, or because you’re already familiar with it from other languages. It could be that you don’t like Mocha’s API and want an alternative, or are interested in learning about other solutions to testing in Node.

Problem

You want to use a test framework that’s designed to interoperate with other systems.

Solution

Use Isaac Z. Schlueter’s tap module.

Discussion

TAP is unique because it aims to bridge test frameworks and tools by specifying a protocol that implementors can use. The protocol is stream-based, lightweight, and human-readable. In comparison to other, heavier XML-based standards, TAP is easy to implement and use.

It’s significant that the tap module (https://npmjs.org/package/tap) is written by Node’s former maintainer, Isaac Z. Schlueter. This is an important seal of approval by someone highly influential in the Node community.

The example in this technique uses the number squaring and random timeout module used in technique 83 so you can compare how tests look in TAP and Mocha.

The following listing shows what the test looks like. For the corresponding module, see listing 10.10.

Listing 10.15. Testing with TAP

This is different from the Mocha example because it doesn’t assume there are any global test-related methods like it and describe: a reference to tap.test has to be set up before doing anything else. Tests are then defined with the t.test() method , and can be nested if needed. Nesting allows related concerns to be grouped, so in this case we’ve created a test case for each method being tested.

The tap module has built-in assertions, and we’ve used these throughout the test file . Once a test case has finished, t.end() must be called . That’s because the tap module assumes tests are asynchronous, so t.end() could be called inside an asynchronous callback.

Another approach is to use t.plan . This method indicates that n assertions are expected. Once the last assertion has been called, the test case will finish running. Unlike the previous test case, the second one can leave off the call to t.end() .

This test can be run with ./node_modules/tap/bin/tap.js test/*_test.js. You can add this line to the test property of scripts in the package.json file to make it run with npm test.

If you run the test with the tap script, you’ll see some clean output that consolidates the results of each assertion. This is generated by one of tap’s submodules called tap-results. The purpose of the tap-results module is to collect lines from a TAP stream and count up skips, passes, and fails to generate a simplified report;

ok test/index_test.js ................................... 3/3

total ................................................... 3/3

ok

Due to the design of the tap module, you’re free to run the tests with node test/index_test.js. This will print out the TAP stream instead:

# Alex's handy mathematics module

# square

ok 1 should be equal

# randomTimeout

ok 2 (unnamed assert)

1..2

# tests 2

# pass 2

# ok

Tests written with the tap module will still return a non-zero exit code to the shell when tests fail—you can use echo $? to see the exit code. Try making the test in listing 10.15 fail on purpose and take a look at $?.

The fact that TAP is designed around producing and consuming streams fits in well with Node’s design. It’s also a fact of life that tests must interact with other automated systems in most projects, whether it’s a deployment system or a CI server. Working with this protocol is easier than heavyweight XML standards, so hopefully it will rise in popularity.

Figure 10.2 illustrates how some of node-tap’s submodules are used to test a program. Control is transferred from different modules, to your tests, back to your program, and then out through the reporter, which collects and analyzes results. The key thing to realize about this is that node-tap’s submodules can be reused and replaced—if you don’t like the way results are displayed with tap-results, it could be replaced with something else.

Figure 10.2. node-tap uses several reusable submodules to orchestrate tests.

Beyond test frameworks, real-world testing depends on several more important tools and techniques. The next section shows you how to use continuous integration servers and database fixtures, and how to mock I/O.

10.5. Tools for tests

When you’re working in a team, you want to quickly see when someone has committed changes that break the tests. This section will help you to set up a continuous integration server so you can do this. It also has techniques for other project-related issues like using databases with tests and mocking web services.

Technique 86 Continuous integration

Your tests are running, but what happens when someone makes a change that breaks the project? Continuous integration (CI) servers are used to automatically run tests. Because most test harnesses return a non-zero exit code on failure, they’re conceptually simple enough. Their real value comes becomes apparent when they can easily be hooked up to services like GitHub and send out emails or instant messages to team members when tests fail.

Problem

You want to see when members of a team commit broken code so you don’t accidentally release it.

Solution

Use a continuous integration server.

Discussion

You’re working in a team and want to see when tests start to fail. You’re already using a version control system like Git, and want to run tests whenever code is committed to a tracked repository. Or you’ve written an open source project and want to indicate on the GitHub or Bitbucket page that it’s well tested.

There are many popular open source and proprietary continuous integration services. In this technique we’ll look at Travis CI (https://travis-ci.org/), because it’s free for open source projects and popular in the Node community. If you want an open source CI server that you can install locally, take a look at Jenkins (http://jenkins-ci.org/).

Travis CI provides a link to an image that shows your project’s build status. To add a project, sign in with your GitHub account at travis-ci.org, and then go to the profile page at travis-ci.org/profile. You’ll see a list of your GitHub projects, and toggling a switch to On will cause the repository to be updated with a service hook that will notify Travis CI whenever you push an update to GitHub.

Once you’ve done that, you need to add a .travis.yml file to the repository to tell Travis CI about the environment your code depends on. All you need to do is set the Node version.

Let’s work through a full example and set up a project on Travis so you can see how it works. You’ll need three files: a package.json, a file to test, and the .travis.yml file. The following listing shows the file we’ll be testing.

Listing 10.16. A simple test to try with Travis CI

This is just a simple test that we can play with to see what Travis CI does. After running, it should result in an exit code of zero—type node test.js and then echo $? to see the exit code. Put this file in a new directory so you can set up a Git repository for it later. Before that we’ll need to create a package.json file. The next listing is a simple package.json that allows the tests to be run with npm test.

Listing 10.17. A basic package.json file

{

"name": "travis-example",

"version": "0.0.0",

"description": "A sample project for setting up Travis CI and Node.",

"main": "test.js",

"scripts": {

"test": "node test.js"

},

"author": "Alex R. Young",

"license": "MIT"

}

Finally, you’ll need a .travis.yml file. It doesn’t need to do much other than tell Travis CI that you’re using Node.

Listing 10.18. Travis CI configuration

language: node_js

node_js:

- "0.10"

Now go to GitHub.com and sign in; then click New Repository to create a public repository. We’ve called ours travis-example so people know it’s purely educational. Follow the instructions on how to commit and push the project to GitHub—you’ll need to run git init in the directory where you placed the preceding three code files, and then git add . and git commit -m 'Initial commit'. Then use git remote add <url> with the repository URL GitHub gives you, and push it with git push -u origin master.

Go to your profile at travis-ci.org/profile and toggle your new project to On. You might need to tell Travis CI to sync your project list—there’s a button near the top of the page.

There’s one last step before you can see any tests running on Travis CI. Make a single change in test.js—add another assertion if you like, and then commit and git push the change. This will cause GitHub to send an API request to Travis CI that will cause your tests to be run.

Travis CI knows how to run Node tests—it defaults to npm test. If you’re adapting this technique to an existing project and you use another command (perhaps make test), then you can change what Travis CI runs by setting the script value in the YML file. Documentation can be found under “Configuring your build” in the documentation (http://about.travis-ci.org/docs/user/build-configuration/#script).

If you go to the homepage at Travis CI, you should now see a console log with details on how the tests were run. Figure 10.3 shows what successful tests look like.

Figure 10.3. Travis CI running tests

Now that you have tests running successfully, you should edit test.js to make the tests fail to see what happens.

Travis can be configured to use most of the things you expect when running tests in real-world projects—databases and other services can be added (http://about.travis-ci.org/docs/user/database-setup/), and even virtual machines.

Getting a database configured with suitable fixtures for your projects is one of the most important parts of testing. The next technique shows how to set up databases for your tests.

Technique 87 Database fixtures

Most applications need to persist data in some way, and it’s important to test that data is stored correctly. This technique explores three solutions for handling database fixtures in Node: loading database dumps, creating data during tests, and using mocks.

Problem

You need to test code that stores data in a database, or performs some other kind of I/O like sending data over a network. You don’t want to access this I/O resource during testing, or you have test data that you want to preload before tests. Either way, your application is highly dependent on an I/O service, and you want to carefully test how your code interacts with it.

Solution

Preload data before the tests, or mock the I/O layer.

Discussion

The mark of well-written code is how testable it is. Code that performs I/O instinctively feels hard to test, but it shouldn’t be if the APIs are cleanly decoupled.

For example, if your code performs HTTP requests, then as you’ve seen in previous techniques, you can run a customized HTTP server within your tests to simulate a remote service. This is known as mocking. But sometimes you don’t want to mock I/O. You may wish to write tests that result in changes being made against a real database, albeit an instance of the database that tests can safely destroy and re-create. These types of tests are known as integration tests—they “integrate” disparate layers of software to deeply test behavior.

This technique presents two ways to handle database fixtures for integration tests; then we’ll broaden the scope by demonstrating how to use mocks. First up: preloading data using database dumps.

Database dumps

Using database dumps is the sledgehammer of database fixture techniques. All you need is to be able to run some code before all of your other tests so you can clear out a database and drop in a pristine copy. If this test data is dumped from a database, then you can use your existing database tools for preparing and exporting the data.

Listing 10.19 uses Mocha and MySQL, but you could adapt the same principles to work with other databases and test frameworks. See technique 83 for more on Mocha.

Listing 10.19. The assert module

The basic principle of this example is to run a database import before the other tests. If you use this approach with your own tests, make sure the import wipes the database first. Relational databases can do this with DROP TABLE IF EXISTS, for example.

To actually run this test, you need to pass the filename to mocha before the other tests, and make sure the test environment is used. For example, if listing 10.19 is called test/init.js, then you could run these commands in the shell: NODE_ENV=test ./node_modules/.bin/mocha test/init.js test/**/*_test.js. Or simply place the commands in your project’s package.json file under scripts, test.

The ran variable is used to ensure the importer is only run once . Mocha’s before function is used to run the importer once, but if test/init.js is accidentally loaded elsewhere (perhaps by running mocha test/**/*.js), then the import would happen twice.

To import the data, the loadFixture function is defined and run in the before callback . It accepts a filename and a callback, so it’s easy to use asynchronously. An additional check is performed to make sure the import is only run in the test environment . The reasoning here is that the database settings would be set by the rest of the application based on NODE_ENV, and you wouldn’t want to lose data by overwriting your development or production databases with the test fixtures.

Finally, the shell command to import the data is built up and run with child_process . This is database-dependent—we’ve used MySQL as an example, but a similar approach would work with MongoDB, PostgreSQL, or pretty much any database that has command-line tools.

Using dump files for fixtures has some benefits: you can author test data with your favorite database tool (we like Sequel Pro), and it’s easy to understand how it all works. If you change your database’s schema or the “model” classes that work with the data, then you’ll need to update your fixtures.

Creating test data with your ORM

An alternative approach is to create data programmatically. This style requires setup code—run in before callbacks or the equivalent in your test framework—which creates database records using your model classes.

The next listing shows how this works.

Listing 10.20. Preparing test data with an ORM

This example can be run with Mocha, and although it doesn’t use a real database layer, the User class fakes the kind of behavior you’re likely to see with a library for a relational database or even a NoSQL database. A save function is defined that has an asynchronous API so the tests look close to a real-world test.

In the describe block that groups together each test case, a variable called user is defined . This will be used by some of the following test cases. It’s defined above their scope so they can all access it, but also because we want to persist it asynchronously in the before block. This runs prior to the test cases .

Mocking the database

The final approach that will be discussed in this technique is mocking the database API. Although you should always write some integration tests, you can also write tests that never touch the database at all. Instead, the database API is abstracted away.

Should I use the ORM for test data?

Like the database dump example in listing 10.19, using an ORM to create test data is useful for integration tests where you really want to talk to a database server. It’s more programming effort than using database dumps, but it can be useful if you want to call methods defined above the database in the ORM layer. The downside of this technique is that a database schema change will potentially require changes in multiple test files.

JavaScript allows objects to be modified after they have been defined. That means you can override parts of the database module with your own methods to return test data. There are libraries designed to make this process easier and more expressive. One module that does this exceptionally well is Sinon.JS. The next example uses Sinon.JS along with Mocha to stub the database module.

Listing 10.21 presents an example that stubs a class that uses Redis for a user account database. The goal of the test is to check that password encryption works correctly.

Listing 10.21. Stubbing a database

This example is part of a large project that includes a package.json file and the User class being tested—it’s available in the code samples, under testing/mocha-sinon.

On the third line you’ll notice something new: sinon.mock wraps the whole database module . The database module is one we’ve defined that loads the node-redis module, and then connects to the database. In this test we don’t want to connect to a real database, so we callsinon.mock to wrap it instead. This approach can be applied to other projects that use MySQL, PostgreSQL, and so on. As long as you design the project to centralize the database configuration, you can easily swap it for a mock.

Next we set up some fields that we want to use for this user . In an integration test, these fields would be returned by the database. We don’t want to do that here, so in the before callback, we use a stub to redefine what Redis hmget does . The stubbing API is chainable, so we chain on the definition of what we want our version of hmget to do by using .callsArgWith .

The semantics of .callsArgWith can be confusing, so here’s a breakdown of how it works. In the User class, hmget is called like this:

this.db.hmget('user:' + this.id, 'fields', function(err, fields) {

this.fields = JSON.parse(fields);

cb(err, this);

}.bind(this));

As you can see, it takes three arguments: the record key, the hash value to fetch, and then a callback that receives an optional error object and the loaded values. When we stub this, we need to tell Sinon.JS how to respond. Therefore, the first argument to callsArgWith is the index of the callback, which is 2, and then the arguments that the callback should receive. We pass null for the error, and the user’s fields serialized as a strong. That gives us callsArgWith(2, null, JSON.stringify(fields)).

This test is useful because the intent of the test is to ensure users can sign in, but only with the correct password. The sign-in code doesn’t really require database access, so it’s better to pass in predefined values rather than going to the trouble of accessing the database. And, because the code serializes JSON to Redis, we don’t need a special library for serializing and decoding JSON—we can use the built-in JSON object.

Now you should know when and how to use integration tests, and mocks and stubs. All of these techniques will help you write better tests, but only if you use them in the correct circumstances. Table 10.2 provides a summary of these techniques and explains when to use each one.

Table 10.2. When to use integration tests, mocks, and stubs

Technique

When to use it

Integration testing

This means testing groups of modules. Here we’ve used the term to distinguish between tests that access a real database, and tests that somehow replace database access with a compatible API. You should use integration tests to ensure your database behaves the way you expect. Integration tests can help verify performance, but this is highly dependent on your test data. It may cause your tests to be more closely coupled to the database, which means that if you change the database or database API, you may need to change your test code as well.

Database dump

This is one way to preload data (before tests) into a test database. It requires a lot of work up front to prepare the data, and the data has to be maintained if you ever change the database schema. The added maintenance work is offset by the simplicity of the approach—you don’t need any special tools to create SQL, Mongo, or other data files. You should use this technique when you’re writing tests for a project that already has a database. Perhaps you’re moving to Node from another programming language or platform, and you’re using the existing database. You can take production data—being careful to remove or obscure any personal information, or other sensitive information—and then drop the resulting database export into your project’s repository.

ORM fixture

Rather than creating a file to import before the tests are run, you can use your ORM module to create and store data in your test code. This can make it hard to maintain over time—any schema changes mean tests have to be carefully updated. You should use this technique for tests where algorithms are closely tied to the underlying data. By keeping the data near the code that uses it, any relating issues can be easier to understand and fix.

Mocks and stubs

Mocks are objects that simulate other objects. In this chapter you saw Sinon.JS, a library for handling mocks and stubs for tests. You should use mocks when you don’t want to access an I/O resource. For example, if you’re writing tests for code that talks to a payment provider like WorldPay or Stripe, then you’d create objects that behave like Stripe’s API without actually communicating with Stripe. It’s generally safer to ensure tests never need to access the internet, so anything that hits the network should be mocked.

The next time you want to test code that connects to a remote web service, or you need to write tests that run against a database, you should know what to do. If you’ve found this section interesting and you want to find out more, continue reading for some ideas on what to learn next.

10.6. Further reading

Testing is a big topic, and although this chapter has been long, there are still important topics to consider. The Node community continues to explore ways to write better tests, and it has started to bring its ideas to client-side development. One such development is Browserify (http://browserify.org)—this allows Node’s module pattern and core modules like EventEmitter and stream.Readable to be used in the browser.

Some Node developers are taking advantage of Browserify to write better client-side tests. Not only can they take advantage of streams and Node’s module pattern for cleaner dependency management, but they can also write Mocha or TAP tests the way they do on the server. James Halliday, the author of Browserify, created Testling, which is a browser automation module for running client-side tests.

Along with continuous integration servers, another useful test-related tool is coverage reports. These analyze code to see how much of a project is hit when the tests are run. There may be functions, methods, or even clauses in if statements that never get executed, which means untested and potentially buggy code could be released to the production environment.

10.7. Summary

In this chapter you’ve learned how to write assertions and extend them, and how to use two popular test frameworks. When writing tests for your Node projects, you should always err on the side of readability—tests should be fast, but if they don’t communicate intent, they can cause maintenance issues in the future.

Here’s a recap of the main points we covered:

· Master the assert module by learning each method and how to ensure errors are correctly handled.

· Use test harnesses like Mocha and node-tap to help make tests readable and maintainable.

· Write tests for code that uses a database by loading data or using mocks and stubs.

· Improve mocks and stubs by using third-party modules like Sinon.JS.

· Develop your own domain-specific languages for tests—write functions and classes that help keep test cases lean and succinct.

One aspect of development that we haven’t covered yet is debugging Node programs. This can be an important part of writing software, depending on your development style and background. If you’re interested in learning the basics of the Node debugger, or want to learn more about it, then read on to dive into debugging with Node.