Testing Some Tough Stuff - The Bigger Unit-Testing Picture - Pragmatic Unit Testing in Java 8 with JUnit (2015)

Pragmatic Unit Testing in Java 8 with JUnit (2015)

Part 4. The Bigger Unit-Testing Picture

Chapter 13. Testing Some Tough Stuff

Not everything is easy in unit testing. Some code will be downright tricky to test. In this chapter we’ll work through a couple of examples of how to test some of the more challenging situations. Specifically, we’ll write tests for code that involves threading and persistence.

In Chapter 10, Using Mock Objects, you learned to simplify testing by breaking difficult dependencies using stubs and mocks. In Chapter 5, FIRST Properties of Good Tests, you saw another example where you broke a dependency on the ever-changing current time (see FI[R]ST: Good Tests Should Be [R]epeatable). You also learned that the design of the code has a lot to do with how easy it is to test.

In this chapter our approach to testing threads and persistence will be based on these two themes: rework the design to better support testing, then break dependencies using stubs and mocks.

Testing Multithreaded Code

It’s hard enough to write code that works as expected. That’s one reason to write unit tests. It’s dramatically harder to write concurrent code that works.

In one sense, testing application code that requires concurrent processing is technically out of the realm of unit testing. It’s better classified as integration testing: you’re verifying that you can integrate the notion of your application-specific logic with the ability to execute portions of it concurrently.

Tests for threaded code tend to be slower—and we don’t like slower tests when unit testing—because you must expand the scope of execution time to ensure that you have no concurrency issues. Threading defects sometimes sneakily lie in wait, surfacing long after you thought you’d stomped them all out.

Don’t worry: even though this is a book on unit testing, not integration testing, you’ll still work through an example of testing multithreaded code.

Keeping It Simple, Smarty

Follow a couple of primary themes when testing threaded code:

· Minimize the overlap between threading controls and application code. Rework your design so that you can unit-test the bulk of application code in the absence of threads. Write thread-focused tests for the small remainder of the code.

· Trust the work of others. Java 5 incorporated Doug Lea’s wonderful set of concurrency utility classes (found in the java.util.concurrent package), which had already undergone years of hardening by the time Java 5 came out in 2004. Instead of coding producer/consumer (for example) yourself by hand—something too easy to get wrong—take advantage of the fact that a smart someone else already went through all the pain, and use the proven class BlockingQueue.

Java provides many, many alternatives for supporting concurrency. We’ll touch on just one here, and it won’t cover your specific case much of the time. But remember the two themes of this chapter: this example shows you how to redesign code to separate the concerns of threading and application logic.

Matchmaker, Matchmaker, Find Me All Matches

Let’s take a look at the ProfileMatcher class, a core piece of iloveyouboss. A ProfileMatcher collects all of the relevant profiles. Given a set of criteria from a client, the ProfileMatcher instance iterates the profiles and returns those matching the criteria, along with the MatchSet instance (which provides the ability to obtain the score of the match):

iloveyouboss/thread-1/src/iloveyouboss/ProfileMatcher.java

import java.util.*;

import java.util.concurrent.*;

import java.util.stream.*;

public class ProfileMatcher {

private Map<String, Profile> profiles = new HashMap<>();

private static final int DEFAULT_POOL_SIZE = 4;

public void add(Profile profile) {

profiles.put(profile.getId(), profile);

}

public void findMatchingProfiles(

Criteria criteria, MatchListener listener) {

ExecutorService executor =

Executors.newFixedThreadPool(DEFAULT_POOL_SIZE);

List<MatchSet> matchSets = profiles.values().stream()

.map(profile -> profile.getMatchSet(criteria))

.collect(Collectors.toList());

for (MatchSet set: matchSets) {

Runnable runnable = () -> {

if (set.matches())

listener.foundMatch(profiles.get(set.getProfileId()), set);

};

executor.execute(runnable);

}

executor.shutdown();

}

}

We need the application to be responsive, so we designed the findMatchingProfiles() method to calculate matches in the context of separate threads. Further, rather than block the client until all processing is complete, we instead designed findMatchingProfiles() to take a MatchListenerargument. Each matching profile gets returned via the MatchListener method foundMatch().

Paraphrasing the code: findMatchingProfiles() first collects a list of MatchSet instances for each profile. For each match set, it creates and spawns a thread that sends the profile and corresponding MatchSet object to the MatchListener if a matches request to the MatchSet returns true.

Extracting Application Logic

The findMatchingProfiles() method is pretty short but still manages to present a good testing challenge. The method intermingles application logic and threading logic. Our first goal is to separate the two.

Start by extracting the logic that gathers MatchSet instances into its own collectMatchSets() method:

iloveyouboss/thread-2/src/iloveyouboss/ProfileMatcher.java

public void findMatchingProfiles(

Criteria criteria, MatchListener listener) {

ExecutorService executor =

Executors.newFixedThreadPool(DEFAULT_POOL_SIZE);

*

for (MatchSet set: collectMatchSets(criteria)) {

Runnable runnable = () -> {

if (set.matches())

listener.foundMatch(profiles.get(set.getProfileId()), set);

};

executor.execute(runnable);

}

executor.shutdown();

}

*

List<MatchSet> collectMatchSets(Criteria criteria) {

*

List<MatchSet> matchSets = profiles.values().stream()

*

.map(profile -> profile.getMatchSet(criteria))

*

.collect(Collectors.toList());

*

return matchSets;

*

}

You know how to write tests for small bits of logic like collectMatchSets():

iloveyouboss/thread-2/test/iloveyouboss/ProfileMatcherTest.java

import static org.junit.Assert.*;

import static org.hamcrest.CoreMatchers.*;

import java.util.*;

import java.util.stream.*;

import org.junit.*;

public class ProfileMatcherTest {

private BooleanQuestion question;

private Criteria criteria;

private ProfileMatcher matcher;

private Profile matchingProfile;

private Profile nonMatchingProfile;

@Before

public void create() {

question = new BooleanQuestion(1, "");

criteria = new Criteria();

criteria.add(new Criterion(matchingAnswer(), Weight.MustMatch));

matchingProfile = createMatchingProfile("matching");

nonMatchingProfile = createNonMatchingProfile("nonMatching");

}

private Profile createMatchingProfile(String name) {

Profile profile = new Profile(name);

profile.add(matchingAnswer());

return profile;

}

private Profile createNonMatchingProfile(String name) {

Profile profile = new Profile(name);

profile.add(nonMatchingAnswer());

return profile;

}

@Before

public void createMatcher() {

matcher = new ProfileMatcher();

}

@Test

public void collectsMatchSets() {

matcher.add(matchingProfile);

matcher.add(nonMatchingProfile);

List<MatchSet> sets = matcher.collectMatchSets(criteria);

assertThat(sets.stream()

.map(set->set.getProfileId()).collect(Collectors.toSet()),

equalTo(new HashSet<>

(Arrays.asList(matchingProfile.getId(), nonMatchingProfile.getId()))));

}

private Answer matchingAnswer() {

return new Answer(question, Bool.TRUE);

}

private Answer nonMatchingAnswer() {

return new Answer(question, Bool.FALSE);

}

}

Similarly extract the application-specific logic that sends matching profile information to a listener:

iloveyouboss/thread-3/src/iloveyouboss/ProfileMatcher.java

public void findMatchingProfiles(

Criteria criteria, MatchListener listener) {

ExecutorService executor =

Executors.newFixedThreadPool(DEFAULT_POOL_SIZE);

for (MatchSet set: collectMatchSets(criteria)) {

Runnable runnable = () -> process(listener, set);

executor.execute(runnable);

}

executor.shutdown();

}

void process(MatchListener listener, MatchSet set) {

if (set.matches())

listener.foundMatch(profiles.get(set.getProfileId()), set);

}

Write a couple of fairly straightforward tests for the new process() method:

iloveyouboss/thread-3/test/iloveyouboss/ProfileMatcherTest.java

// ...

*

import static org.mockito.Mockito.*;

public class ProfileMatcherTest {

// ...

private MatchListener listener;

@Before

public void createMatchListener() {

listener = mock(MatchListener.class);

}

@Test

public void processNotifiesListenerOnMatch() {

matcher.add(matchingProfile);

MatchSet set = matchingProfile.getMatchSet(criteria);

matcher.process(listener, set);

verify(listener).foundMatch(matchingProfile, set);

}

@Test

public void processDoesNotNotifyListenerWhenNoMatch() {

matcher.add(nonMatchingProfile);

MatchSet set = nonMatchingProfile.getMatchSet(criteria);

matcher.process(listener, set);

verify(listener, never()).foundMatch(nonMatchingProfile, set);

}

// ...

}

The tests take advantage of Mockito’s ability to verify expectations—to verify that a method was called with the expected arguments. Refer to Simplifying Testing Using a Mock Tool for an overview of the Mockito mock tool and for another example of its use.

Steps in the first test, processNotifiesListenerOnMatch, are:

Use Mockito’s static mock() method to create a MatchListener mock instance. Verify expectations using this instance.

Add a matching profile (a profile that is expected to match the given criteria) to the matcher.

Ask for the MatchSet object for the matching profile given a set of criteria.

Ask the matcher to run the match processing, passing in the mock listener and the match set.

Ask Mockito to verify that the foundMatch method was called on the mock listener instance with the matching profile and match set as arguments. Mockito fails the test if that expectation isn’t met.

Redesigning to Support Testing the Threading Logic

The bulk of the code in findMatchingProfiles() that remains after we extract collectMatchSets() and process() is threading logic. (We could potentially go even one step further and create a generic method that spawns threads for each element in a collection, but let’s work with what we have now.) Here’s the current state of the method:

iloveyouboss/thread-3/src/iloveyouboss/ProfileMatcher.java

public void findMatchingProfiles(

Criteria criteria, MatchListener listener) {

ExecutorService executor =

Executors.newFixedThreadPool(DEFAULT_POOL_SIZE);

for (MatchSet set: collectMatchSets(criteria)) {

Runnable runnable = () -> process(listener, set);

executor.execute(runnable);

}

executor.shutdown();

}

Our idea for testing findMatchingProfiles() involves a little bit of redesign work. Here’s the reworked code:

iloveyouboss/thread-4/src/iloveyouboss/ProfileMatcher.java

Line 1

private ExecutorService executor =

-

Executors.newFixedThreadPool(DEFAULT_POOL_SIZE);

-

-

ExecutorService getExecutor() {

5

return executor;

-

}

-

-

public void findMatchingProfiles(

-

Criteria criteria,

10

MatchListener listener,

-

List<MatchSet> matchSets,

-

BiConsumer<MatchListener, MatchSet> processFunction) {

-

for (MatchSet set: matchSets) {

-

Runnable runnable = () -> processFunction.accept(listener, set);

15

executor.execute(runnable);

-

}

-

executor.shutdown();

-

}

-

20

public void findMatchingProfiles(

-

Criteria criteria, MatchListener listener) {

-

findMatchingProfiles(

-

criteria, listener, collectMatchSets(criteria), this::process);

-

}

25

-

void process(MatchListener listener, MatchSet set) {

-

if (set.matches())

-

listener.foundMatch(profiles.get(set.getProfileId()), set);

-

}

We need to access the ExecutorService instance from the test, so we extract its instantiation to the field level and provide a package-access-level getter method to return the ExecutorService reference.

Because we’ve already tested process, we can safely assume it’s correct and thus ignore its real logic when we test findMatchingProfiles. To support stubbing the behavior of process, overload findMatchingProfiles (see line 8). Change its existing implementation to take an additional argument, processFunction, that represents the function to execute in each thread. Use the processFunction function reference to call the appropriate logic to process each MatchSet (line 14).

Add an implementation of findMatchingProfiles with the original signature that delegates to the overloaded version (the one that takes a function argument, at line 20). For the function argument, pass this::process, which refers to the known-to-be-working implementation of process inProfileMatcher.

Writing a Test for the Threading Logic

The code should work exactly as it did before, but we’ve set things up to make it easier for us to write a test. Let’s give it a go:

iloveyouboss/thread-4/test/iloveyouboss/ProfileMatcherTest.java

// ...

*

import static org.mockito.Mockito.*;

public class ProfileMatcherTest {

// ...

@Test

public void gathersMatchingProfiles() {

Set<String> processedSets =

Collections.synchronizedSet(new HashSet<>());

BiConsumer<MatchListener, MatchSet> processFunction =

(listener, set) -> {

processedSets.add(set.getProfileId());

};

List<MatchSet> matchSets = createMatchSets(100);

matcher.findMatchingProfiles(

criteria, listener, matchSets, processFunction);

while (!matcher.getExecutor().isTerminated())

;

assertThat(processedSets, equalTo(matchSets.stream()

.map(MatchSet::getProfileId).collect(Collectors.toSet())));

}

private List<MatchSet> createMatchSets(int count) {

List<MatchSet> sets = new ArrayList<>();

for (int i = 0; i < count; i++)

sets.add(new MatchSet(String.valueOf(i), null, null));

return sets;

}

}

Create a set of strings to store profile IDs from MatchSet objects that the listener receives.

Define processFunction(), which will supplant the production version of process.

For each callback to the listener, add the MatchSet’s profile ID to processedSets.

Using a helper method, create a pile of MatchSet objects for testing.

Call the version of findMatchingProfiles that takes a function as an argument, and pass it the processFunction() implementation.

Grab the ExecutorService from the matcher, and loop until it indicates that all of its threads have completed execution.

Verify that the collection of processedSets (representing profile IDs captured in the listener) matches the profile IDs from all of the MatchSet objects created in the test.

By separating concerns between application logic and threading logic, we’ve been able to write a few tests in reasonably short order. The first tests take a little bit of effort and thought about how to best organize things. Each subsequent thread-related test gets easier, however, as we begin to build up a library of utility methods to help us get a handle on thread-focused testing.

Testing Databases

We first saw the StatCompiler code in [F]IRST: [F]ast!. We were able to refactor this class so that most of its code doesn’t directly interact with a QuestionController instance, which in turn let us write fast tests for the bulk of its logic. We were left with one method, questionText(), that interacts with a controller object, and we’d now like to test that method:

iloveyouboss/16-branch-persistence-redesign/src/iloveyouboss/domain/StatCompiler.java

public Map<Integer,String> questionText(List<BooleanAnswer> answers) {

Map<Integer,String> questions = new HashMap<>();

answers.stream().forEach(answer -> {

if (!questions.containsKey(answer.getQuestionId()))

questions.put(answer.getQuestionId(),

controller.find(answer.getQuestionId()).getText()); });

return questions;

}

The questionText() method takes a list of answer objects and returns a hash map of unique answer IDs to the corresponding question text. Paraphrasing the forEach loop: for each answer ID that’s not already represented in the responses map, find the corresponding question using the controller, and put the found question’s text into the responses map.

Thanks a Lot, Controller

The trouble with writing tests for questionText() is the controller, which talks to a Postgres database using the Java Persistence API (JPA). Our first question regards the QuestionController controller class: do we trust it and understand how it behaves? We’d like to make sure, by writing some tests for it. Here’s the code for the class:

iloveyouboss/16-branch-persistence-redesign/src/iloveyouboss/controller/QuestionController.java

import iloveyouboss.domain.*;

import java.time.*;

import java.util.*;

import java.util.function.*;

import javax.persistence.*;

public class QuestionController {

private Clock clock = Clock.systemUTC();

private static EntityManagerFactory getEntityManagerFactory() {

return Persistence.createEntityManagerFactory("postgres-ds");

}

public Question find(Integer id) {

return em().find(Question.class, id);

}

public List<Question> getAll() {

return em()

.createQuery("select q from Question q", Question.class)

.getResultList();

}

public List<Question> findWithMatchingText(String text) {

String query =

"select q from Question q where q.text like '%" + text + "%'";

return em().createQuery(query, Question.class) .getResultList();

}

public int addPercentileQuestion(String text, String[] answerChoices) {

return persist(new PercentileQuestion(text, answerChoices));

}

public int addBooleanQuestion(String text) {

return persist(new BooleanQuestion(text));

}

void setClock(Clock clock) {

this.clock = clock;

}

void deleteAll() {

executeInTransaction(

(em) -> em.createNativeQuery("delete from Question")

.executeUpdate());

}

private void executeInTransaction(Consumer<EntityManager> func) {

EntityManager em = em();

EntityTransaction transaction = em.getTransaction();

try {

transaction.begin();

func.accept(em);

transaction.commit();

} catch (Throwable t) {

t.printStackTrace();

transaction.rollback();

}

finally {

em.close();

}

}

private int persist(Persistable object) {

executeInTransaction((em) -> em.persist(object));

return object.getId();

}

private EntityManager em() {

return getEntityManagerFactory().createEntityManager();

}

}

Most of the logic in QuestionController is simple delegation to code that implements the JPA interfaces—there’s not much in the way of interesting logic. That’s good design—we’ve isolated our dependency on JPA. From the stance of testing, though, it raises a question: does it make sense to write a unit test against QuestionController? You could write unit tests in which you stub all of the relevant interfaces, but it would take a good amount of effort, the tests would be difficult, and in the end you wouldn’t have proven all that much.

You should instead write tests against QuestionController that demonstrate its ability to successfully interact with a real Postgres database. These slower tests will prove that everything is wired together correctly. Defects are fairly common in dealings with JPA, because three different pieces of detail must all work correctly in concert: the Java code, the mapping configuration (located in src/META-INF/persistence.xml in our codebase), and the database itself.

The Data Problem

You still want the vast majority of your JUnit tests to be fast. No worries—if you isolate all of your persistence interaction to one place in the system, you end up with a reasonably small amount of code that must be integration-tested.

(You might be tempted to consider using an in-memory database such as H2 to emulate your production database for the purpose of testing. You’ll get the speed you want, but good luck otherwise. Attempts we’ve encountered were fraught with problems due to sometimes subtle differences between the in-memory database and the production RDBMS.)

When you write integration tests for code that interacts with the real database, the data in the database and how it gets there become important considerations. To verify that database find operations return query results as expected, for example, you need to either put appropriate data into the database or assume it’s already there.

Assuming that data is already in the database is a long-term recipe for pain. Over time, the data will change without your knowledge, breaking tests. Divorcing the data from the test code makes it a lot harder to understand why a particular test passes or not. The meaning of the data with respect to the tests is lost by dumping it all into the database. Prefer to let the tests create and manage the data.

You must answer the question, whose database? If it’s your database on your own machine, the simplest route might be for each test to start with a clean database (or one prepopulated with necessary reference data). Each test then becomes responsible for adding and working with its own data. This minimizes intertest dependency issues, where one test breaks because of data that another test left lying around. (Those can be a headache to debug!)

If you can only interact with a shared database for your testing, then you’ll need a less intrusive solution. One option: if your database supports it, you can initiate a transaction in the context of each test, then roll it back. (The transaction handling is usually relegated to @Before and @Aftermethods.)

Ultimately, integration tests are harder to write and maintain. They tend to break more often, and when they do break, debugging the problem can take considerably longer. But they’re still an essential part of your testing strategy.

images/aside-icons/tip.png

Integration tests are essential but challenging to design and maintain. Minimize their number and complexity by maximizing the logic you verify in unit tests.

Clean-Room Database Tests

Our tests for the controller empty the database both before and after each test method’s execution:

iloveyouboss/16-branch-persistence-redesign/test/iloveyouboss/controller/QuestionControllerTest.java

public class QuestionControllerTest {

private QuestionController controller;

@Before

public void create() {

controller = new QuestionController();

controller.deleteAll();

}

@After

public void cleanup() {

controller.deleteAll();

}

@Test

public void findsPersistedQuestionById() {

int id = controller.addBooleanQuestion("question text");

Question question = controller.find(id);

assertThat(question.getText(), equalTo("question text"));

}

@Test

public void questionAnswersDateAdded() {

Instant now = new Date().toInstant();

controller.setClock(Clock.fixed(now, ZoneId.of("America/Denver")));

int id = controller.addBooleanQuestion("text");

Question question = controller.find(id);

assertThat(question.getCreateTimestamp(), equalTo(now));

}

@Test

public void answersMultiplePersistedQuestions() {

controller.addBooleanQuestion("q1");

controller.addBooleanQuestion("q2");

controller.addPercentileQuestion("q3", new String[] { "a1", "a2"});

List<Question> questions = controller.getAll();

assertThat(questions.stream()

.map(Question::getText)

.collect(Collectors.toList()),

equalTo(Arrays.asList("q1", "q2", "q3")));

}

@Test

public void findsMatchingEntries() {

controller.addBooleanQuestion("alpha 1");

controller.addBooleanQuestion("alpha 2");

controller.addBooleanQuestion("beta 1");

List<Question> questions = controller.findWithMatchingText("alpha");

assertThat(questions.stream()

.map(Question::getText)

.collect(Collectors.toList()),

equalTo(Arrays.asList("alpha 1", "alpha 2")));

}

}

The code calls the QuestionController method deleteAll() in both the @Before and @After methods. When trying to figure out a problem, you might need to comment out the deleteAll() call in the @After method so that you can take a look at the data after a test completes.

Our tests are simple and direct. We’re not testing end-to-end application functionality—we’re instead testing the query capabilities, which is most of what we care about. Our tests implicitly verify the ability of the controller to add items to the database.

Mocking the Controller

We’ve isolated all direct database interaction to QuestionController and tested it. Now we can move on to testing the questionText() method in StatCompiler. We now trust QuestionController, so we can safely stub out its find() method.

Think about mocking as making an assumption: you are assuming that what you’re mocking out works and that you know how it behaves—how it responds to inquiries and what side effects it creates. Without that knowledge, you might be making a bad assumption in your tests.

Here’s the questionText() method again:

iloveyouboss/16-branch-persistence-redesign/src/iloveyouboss/domain/StatCompiler.java

public Map<Integer,String> questionText(List<BooleanAnswer> answers) {

Map<Integer,String> questions = new HashMap<>();

answers.stream().forEach(answer -> {

if (!questions.containsKey(answer.getQuestionId()))

questions.put(answer.getQuestionId(),

controller.find(answer.getQuestionId()).getText()); });

return questions;

}

And here’s the test, which uses Mockito:

iloveyouboss/16-branch-persistence-3/test/iloveyouboss/domain/StatCompilerTest.java

@Mock private QuestionController controller;

@InjectMocks private StatCompiler stats;

@Before

public void initialize() {

stats = new StatCompiler();

MockitoAnnotations.initMocks(this);

}

@Test

public void questionTextDoesStuff() {

when(controller.find(1)).thenReturn(new BooleanQuestion("text1"));

when(controller.find(2)).thenReturn(new BooleanQuestion("text2"));

List<BooleanAnswer> answers = new ArrayList<>();

answers.add(new BooleanAnswer(1, true));

answers.add(new BooleanAnswer(2, true));

Map<Integer, String> questionText = stats.questionText(answers);

Map<Integer, String> expected = new HashMap<>();

expected.put(1, "text1");

expected.put(2, "text2");

assertThat(questionText, equalTo(expected));

}

Make sure you feel comfortable reading and paraphrasing what the test does. Mockito does a great job of keeping the mocking needs in our tests simple and declarative. Even without much knowledge of Mockito, you can read the test and quickly understand its intent. Refer to Simplifying Testing Using a Mock Tool if you need a refresher on Mockito.

After

Two common challenges—multithreading and database interaction—are tough-enough topics on their own. Often many of your defects will come from code in these areas.

In general, you want to adhere to the following strategy for testing these more-difficult scenarios:

· Separate concerns. Keep application logic apart from threading, database, or other dependencies causing you a problem. Isolate dependent code so it’s not rampant throughout your codebase.

· Use mocks to break dependencies of unit tests on slow or volatile code.

· Write integration tests where needed, but keep them simple and focused.

Next up: you’re almost ready to graduate. So far, you’ve focused on heads-down unit testing on your development machine. For the final chapter, you’ll learn about some topics relevant to unit testing as part of a development team.