Pragmatic Unit Testing in Java 8 with JUnit (2015)

Part 1. Unit-Testing Foundations

Chapter 3. Digging Deeper into JUnit Assertions

In the last chapter we worked through a meaty example of writing unit tests against an existing bit of code. You learned how to use assertions to express expected outcomes.

In this chapter you’ll learn many additional ways to phrase asserts in JUnit by using a library known as Hamcrest. You’ll also learn how to write tests when you’re expecting exceptions.

Assertions in JUnit

Assertions (or asserts) in JUnit are static method calls that you drop into your tests. Each assertion is an opportunity to verify that some condition holds true. If an asserted condition does not hold true, the test stops right there, and JUnit reports a test failure.

(It’s also possible that when JUnit runs your test, an exception is thrown and not caught. In this case, JUnit reports a test error.)

JUnit supports two major assertion styles—classic-style assertions that shipped with the original version of JUnit, and a newer, more expressive style known as Hamcrest (an anagram of the word matchers).

Each of the two assertion styles provides a number of different forms for use in different circumstances. You can mix and match, but you’re usually better off sticking to one style or the other. We’ll briefly look at classic assertions but then will focus primarily on Hamcrest assertions.

assertTrue

The most basic assertion is:

Since assertions are so pervasive in JUnit tests, most programmers use a static import to reduce the clutter:

A couple of examples:

The preceding examples depend on the existence of an initialized Account instance. You can create an Account in an @Before method (see More on @Before and @After (Common Initialization and Cleanup) for more information) and store a reference to it as a field on the test class:

A test name such as depositIncreasesBalance is a general statement about the behavior you’re trying to verify. We can write assertions that are also generalizations; for example, we can assert that the balance after depositing is greater than zero. However, the code in our test provides a specific example, and as such you’re better off being explicit about the answer you expect.

assertThat Something Is Equal to Another Something

More often than not, we can compare an actual result returned against a result that we expect. Rather than simply verify that a balance is greater than zero, we can assert against a specific expected balance:

The assertThat() static method call is an example of a Hamcrest assertion. The first argument to a Hamcrest assertion is the actual expression—the value we want to verify (often a method call to the underlying system). The second argument is a matcher. A matcher is a static method call that allows comparing the results of an expression against an actual value. Matchers can impart greater readability to your tests. They read fairly well left-to-right as a sentence. For example, we can quickly paraphrase the preceding assertion as “assert that the account balance is equal to 100.”

To use the core Hamcrest matchers that JUnit provides, we need to introduce another static import:

We can pass any Java instance or primitive value to the equalTo matcher. As you might expect, equalTo uses the equals() method as the basis for comparison. Primitive types are autoboxed into instances, so we can compare any type.

Hamcrest assertions provide a more helpful message when they fail. The prior test expected account.getBalance() to return 100. If it returns 101 instead, you see this:

Not as much with assertTrue(). When it fails, we get the following stack trace:

That’s not a terribly useful stack trace; you’ll have to dig into the test and code to figure out what’s going on—maybe insert a few System.out.printlns or even hit the debugger.

The assertTrue() call is a classic assertion. You could try using a Hamcrest matcher for assertions against Boolean expressions, to see if you get better failure messages:

But it doesn’t provide any more useful information. Some folks find it a bit ridiculous with its extra, useless verbiage. We prefer a simple assertTrue() instead.

Let’s take a look at another Hamcrest assertion, one that uses a startsWith matcher (provided by the CoreMatchers class):

When the assertThat() call fails, we get the following stack trace:

The stack trace might be all the information we need to fix the problem!

Rounding Out the Important Hamcrest Matchers

The Hamcrest CoreMatchers class that ships with JUnit provides us with a solid starter set of matchers. Although you can survive using only a few matchers, your tests will gain expressiveness the more you reach deeper into the Hamcrest bag of matchers. This section presents a few key Hamcrest matchers.

You can use equalTo() to compare Java arrays or collection objects, and it compares them as you might expect. The following two assertions fail:

The assertions pass when the compared collections match:

You can make your matcher expressions more readable in some cases by adding the is decorator. It simply returns the matcher passed to it—in other words, it does nothing. Sometimes a little bit of nothing can make your code more readable:

You can also use the phrasing is("my big fat acct") to mean the same thing as equalTo("my big fat acct"). The use of these decorators is up to you. Our brains can fill in missing words like is for us automatically, so our preference is to omit the decorators and only specify equalTo.

If you must assert the opposite of something, use not:

(You could again choose to wrap the matcher expression with the is decorator: is(not(equalTo("plunderings"))).)

You can check for null values or not-null values, as the case may be:

Frequent not-null checking suggests a design issue, or maybe too much worrying. In many cases, not-null checks are extraneous and add little value:

You can eliminate the not-null assertion in the prior example. If account.getName() returns null, the second assertion (equalTo("...")) still prevents the test from passing. A minor distinction: the null reference exception that gets thrown generates a test error, not a test failure. JUnit reports an error for any exception thrown and not caught by the test.

If you’re hungry for more matchers, JUnit Hamcrest matchers let you:

· Verify the type of an object

· Verify that two object references represent the same instance

· Combine multiple matchers, requiring that either all or any of the matchers succeed

· Verify that a collection contains or matches an element

· Verify that a collection contains all of several items

· Verify that all elements in a collection conform to a matcher

…and much more! Refer to the Hamcrest API documentation^[11] for details, or better yet, try ‘em out in your IDE to get comfortable with how they work.

If those matchers still aren’t enough for your needs, you can create your own domain-specific custom matchers. The sky’s the limit! Google’s tutorial^[12] can show you how, and you’ll also learn how later in this book (see Creating a Custom Matcher to Verify an Invariant).

Comparing Two Floating-Point Numbers

Computers can’t represent every floating-point number.^[13] In Java, some of the numbers of the floating-point types (float and double) must be approximated. The implication for unit testing is that comparing two floating-point results doesn’t always produce the result we want:

That test looks like it should pass, but it doesn’t:

When comparing two float or double quantities, we want to specify a tolerance, or error margin, that the two numbers can diverge by. We could write such an assertion by hand using assertTrue():

Yuk. That assertion doesn’t read well, and when it fails, the failure message doesn’t read well either.

We can instead use a Hamcrest matcher named IsCloseTo, which provides a static method named closeTo(). (Note: The Hamcrest matchers shipped with JUnit are a subset of a larger set of matchers. If you want to use IsCloseTo, or one of dozens more potentially useful matchers, you’ll need to download the original Hamcrest matchers library separately and include it in your project. Visit the Hamcrest site^[14] for further details, and good luck!

The IsCloseTo matcher makes our floating-point comparison quite readable:

Explaining Asserts

All JUnit assert forms (classic, fail(), and assertThat()) support an optional first argument named message. The message allows us to supply a nice verbose explanation of the rationale behind the assertion:

That comment doesn’t even accurately describe the test. It’s a lie! The comment indicates an expected balance (100) that doesn’t match the real expectation in the test (50). Comments that explain implementation details are notorious for getting out of sync with the code.

If you prefer lots of explanatory comments, you might get some mileage out of assertion messages. However, the better route is to make your tests more descriptive. It’s easy to make dramatic improvements to your tests by renaming them, introducing meaningful constants, improving the names of variables, extracting complex setup to meaningfully named helper methods, and using more-literary Hamcrest assertions. We’ll step through an example of test cleanup in Chapter 11, Refactoring Tests.

Assert messages provide useful information slightly more quickly if a test does fail. But we’ll personally take the trade-off of having less-cluttered code.

Three Schools for Expecting Exceptions

In addition to ensuring that the happy path through our code works, we want to verify that exceptions get thrown when expected. Understanding the conditions that cause a class to throw exceptions can make life a lot easier for a client developer using the class.

JUnit supports at least three different ways of specifying that you expect an exception to be thrown. Let’s examine a simple case: ensure that Account code throws an Exception when a client attempts to withdraw more than the available balance.

Simple School: Using an Annotation

The JUnit @Test annotation supports passing an argument that specifies the type of an expected exception:

If an InsufficientFundsException gets thrown during execution of throwsWhenWithdrawingTooMuch, the test passes. Otherwise JUnit fails the test:

Demonstrate this exception by simply commenting out the withdrawal operation from throwsWhenWithdrawingTooMuch and rerunning the test.

Old School: Try and Fail-or-Catch

You can use a try/catch block that handles the expected exception getting thrown. If an exception doesn’t get thrown, explicitly fail the test by calling org.junit.Assert.fail():

If the account withdrawal generates an exception, control transfers to the catch block, then drops out of the test, meaning it passes. Otherwise, control drops to the fail statement. The try/catch idiom represents the rare case where it might be okay to have an empty catch block. Naming the exception variable expected helps reinforce to the reader that we expect an exception to be thrown and caught.

Purposely fail the test by commenting out the withdrawal operation.

The old-school technique is useful if you need to verify the state of things after the exception gets thrown. Perhaps you want to verify the exception message. For example:

New School: ExpectedException Rules

JUnit allows you to define custom rules, which can provide greater control over what happens during the flow of test execution. In a sense, rules provide us with a capability similar to aspect-oriented programming.^[15] They provide a way to automatically attach a cross-cutting concern—an interest in maintaining an invariant—to a set of tests.

JUnit provides a few useful rules out of the box (you don’t have to code them). Particularly, the ExpectedException rule lets you combine the best of the simple school and the old school when it comes to verifying exceptions.

Suppose we’re designing a test in which we withdraw funds from a new account—that is, one with no money. Withdrawing any money from the account should generate an exception.

To use the ExpectedException rule, declare a public instance of ExpectedException in the test class and mark it with @Rule (line 4 in the following test).

Our test setup requires telling the rule what we expect to happen at some point during execution of the rest of the test. We tell the thrown rule instance to expect that an InsufficientFundsException gets thrown (line 8).

We also want to verify that the exception object contains an appropriate message, so we set another expectation on the thrown rule (line 9). If we were interested, we could also tell the rule object to expect that the exception contains a cause object.

Finally, our act portion of the test withdraws money (line 11), which hopefully triggers the exception we expect. JUnit’s rule mechanism handles the rest, passing the test if all expectations on the rule were met and failing the test otherwise.

Three ways of asserting against expected exceptions—is that all? Not a chance. Searching the web reveals at least a couple more techniques, and Java 8 opens up new possibilities. For example, Stefan Birkner provides a small library named Fishbowl^[16] that helps you take advantage of the conciseness that lambda expressions can provide. Fishbowl lets you assign the result of an exception-throwing lambda expression to an exception object you can assert against.

Exceptions Schmexceptions

Most tests you write will be more carefree, happy-path tests where exceptions are highly unlikely to be thrown. But Java acts as a bit of a buzzkill, insisting that you acknowledge any checked exception types.

Don’t clutter your tests with try/catch blocks to deal with checked exceptions. Instead, rethrow any exceptions from the test itself:

Given that you’re designing these positive tests, you know they shouldn’t throw an exception except under truly exceptional conditions. You can stop worrying about those exceptional conditions: in the bizarre case that an unexpected exception surfaces, JUnit does the dirty work for you. It traps the exception and reports the test as an error instead of a failure.

After

You’ve learned myriad ways of expressing expectations in this chapter by using JUnit’s Hamcrest assertions. Next up, you’ll take a look at how to best structure and organize your JUnit tests.

Footnotes