Parameterized tests - CORE TESTING WITH SPOCK - Java Testing with Spock, Version 6 (2015)

Java Testing with Spock, Version 6 (2015)

PART 2: CORE TESTING WITH SPOCK

Chapter 5. Parameterized tests

In this chapter

· Definition of parameterized tests

· Using the where: block

· Understanding data tables and data pipes

· Writing custom data providers

In the previous chapter I presented all the Spock blocks that you can use in a single test with except for one. I left out on purpose the where: block because it deserves a chapter on its own.

The where: block is used for parameterized tests. Parameterized tests, are unit tests that share the same test logic (e.g. when temperature goes up, reactor must shut down), but need to run on different parameters (e.g. with low temperature and then with high temperature) in order to account for all cases.

In this chapter I will cover both some theory on when to use parameterized tests and also the facilities Spock offers for parameterized testing. You might have seen already parameterized tests with JUnit, so feel free to skip the first section and start reading at 5.2 for the specific Spock features if you already familiar with the concept of parameterized testing.

Spock is very flexible when it comes to parameterized tests. If offers a complete portfolio of techniques adaptable to your situation no matter the complexity of your test data. The most basic format of parameterized tests (data tables) were already demonstrated in chapter 3. I will also explain data pipes (the underlying mechanism of data tables) and finally show you how to write custom data providers with Spock, which is the most flexible solution (but needs more programming effort).

All these Spock techniques have their own advantages and disadvantages with regards to readability and flexibility of unit tests, therefore it is important to understand the tradeoffs between them. Understanding when to use each one, is one of the running themes of this chapter.

5.1 How to detect the need for parameterized tests

Experienced developers usually can understand the need for a parameterized test right away. But even if you are just starting with unit tests, there is a very easy rule of thumb that shows you the need for a parameterized test. Every time that you start a new unit test by copying-pasting an existing one, ask yourself: "is this test that much different from the previous one?". If you find yourself duplicating unit tests and then just changing one or two variables to create a similar scenario, take a step back and think if a parameterized test would be more useful in your case.

Duplicating unit test code is not a healthy habit

Actually I would like to generalize this rule. Any time you copy-paste a unit test, it means that you are creating code duplication because you haven't thought about reusable code segments. Like production code, test code should be treated with the same "respect". Refactoring unit tests to allow them to share code via composition instead of blind copy-paste should be one of your first priorities when adding new unit tests into an existing suite. More details will be presented in Chapter 7.

Assume for example that I have a single class that takes an image filename and returns true if the picture has an extension that is considered valid for the application:

public class ImageNameValidator { public boolean isValidImageExtension(String fileName) { [...redacted for brevity...] } }

A first naive approach would be to write a single Spock text for every image extension that needs to be examined. This approach is shown on listing 5.1 (and it clearly suffers from code duplication)

Listing 5.1 - Duplicate tests - DO NOT DO THIS

def "Valid images are JPG"() { given: "an image extension checker and a jpg file" ImageNameValidator validator = new ImageNameValidator() String pictureFile = "scenery.jpg" #A expect: "that the filename is valid" validator.isValidImageExtension(pictureFile) #B} def "Valid images are JPEG"() { given: "an image extension checker and a jpeg file" ImageNameValidator validator = new ImageNameValidator() String pictureFile = "house.jpeg" #A expect: "that the filename is valid" validator.isValidImageExtension(pictureFile) #B} def "Tiff are invalid"() { given: "an image extension checker and a tiff file" ImageNameValidator validator = new ImageNameValidator() String pictureFile = "sky.tiff" #A expect: "that the filename is invalid" !validator.isValidImageExtension(pictureFile) #B}

#A Each test differs in input data

#B Each test differs in output data

The original requirement was to accept JPG files only, and a single unit test was written. Then the customer reported that the application does not work on Linux because .jpeg files were used. Application code was updated and another test method was added (by copying the existing one).

Then a business analyst requested an explicit test for not supporting TIFF files. You can see where this is going. In large enterprise applications multiple developers might work on the same feature as time progresses. If each developer is adding blindly a new unit test by copy-paste (either because of lack of time or lack of experience) the result is a Spock test as shown in listing 5.1 that smells of code duplication from afar.

Notice that each test method by itself in listing 5.1 is well structured. It is documented, it tests one thing, the trigger action is small etc. The problem stems from the collection of those test methods that need further refactoring as they all have the exact same business logic.

5.1.1 What are Parameterized Tests?

An example of a parameterized test for this class in Spock, is shown in listing 5.2. With a single unit test, not only I have replaced all three tests of listing 5.1 but even added two more cases.

Listing 5.2 - Simple Spock parameterized test

def "Valid images are PNG and JPEG files"() { given: "an image extension checker" ImageNameValidator validator = new ImageNameValidator() #A expect: "that only valid filenames are accepted" validator.isValidImageExtension(pictureFile) == validPicture #E where: "sample image names are" #B pictureFile || validPicture #C "scenery.jpg" || true #D "house.jpeg" || true #D "car.png" || true #D "sky.tiff" || false #D "dance_bunny.gif" || false #D}

#A class under test

#B where: block contains parameters for multiple scenarios

#C First line of block is always the names of parameters

#D input and expected output for each scenario in each line

#E common test logic for all scenarios that use pictureFile and validPicture

The test method examines multiple scenarios where the test logic is always the same (i.e. validate a file name) and only the input (e.g. jpg) and output (i.e. valid/not valid) variables change each time. The test code is fixed, while the test input and output data come in the form of parameters (and thus we have a parameterized test).

The idea is better illustrated in figure 5.1

Figure 5.1 - Parameterized tests share the same test logic for different input/output data sets

The test code is shared among all parameters. Instead of duplicating this common code for each scenario we centralize it on a single test method. Then each scenario comes with its own scenario parameters that define what is the expected result for each input. Output 1 is expected when the test scenario is triggered with input 1, output 2 is expected if input 2 is used and so on.

5.2 The where: block

The where: block was introduced in chapter 3 and is responsible for holding all input and output parameters for a parameterized test. It can be combined with all other blocks shown in chapter 4 but it has to be the last block inside a Spock test. Only an and: block might follow a where: block (and doing that would be very rare)

The simpler given-expect-when structure was shown in listing 5.2. This works for trivial and relatively simple tests. The more usual way (and the recommended way for your larger parameterized tests) is the given-when-then-where structure as shown in listing 5.3:

Listing 5.3 The given-when-then-where structure

def "Valid images are PNG and JPEG files (enterprise version)"() { given: "an image extension checker" ImageNameValidator validator = new ImageNameValidator() when: "an image is checked" ImageExtensionCheck imageExtensionCheck = validator.examineImageExtension(pictureFile) #A then: "expect that only valid filenames are accepted" imageExtensionCheck.result == validPicture #B imageExtensionCheck.errorCode == error #B imageExtensionCheck.errorDescription == description #B where: "sample image names are" #C pictureFile || validPicture | error | description #D "scenery.jpg" || true | "" | "" "house.jpeg" || true | "" | "" "car.png" || true | "" | "" "sky.tiff" || false | "ERROR002" | "Tiff files are not supported" "dance_bunny.gif" || false | "ERROR999" | "Unsupported file type"}

#A input parameter (pictureFile) is used in the when: block

#B output parameters are checked in the then: block

#C where: block is last block in the test

#D input and output scenarios in each consequent line

Here I have modified the ImageNameValidator class to return a simple Java object named ImageExtensionCheck that groups the result of the check along with an error code and a human readable description. The when: block creates this result object and the then: block compares its contents against that parameterized variables in the where: block.

Notice again that the where: block is the last one in the Spock test. If you have other blocks after the where: block, Spock will refuse to run the test.

Now that you know the basic usage of the where: block it is time to focus on its contents. So far all the example you have seen have used data tables. This is one of the possible variations. Spock supports the following:

1. Data tables. This is the declarative style. Very easy to write but does not cope with complex tests. Readable by business analysts.

2. Data tables with programmatic expressions as values. A bit more flexible that data tables with some loss in readability.

3. Data pipes will fully dynamic input outputs. Very flexible but not as readable as data tables.

4. Custom data iterators. Your nuclear option when all else fails. They can be used for any extreme corner cases of data generation. Not readable at all by non-technical people.

We will examine the details of all these techniques in turn in the rest of the chapter.

5.2.1 Using data tables in the where: block

We have now established that the where: block must be the last block in a Spock test. In all examples you have seen so far, the where: block contains a data table. This data table holds multiple test cases where each line is a scenario and each column is an input or output variable to that scenario.

Listing 5.4 Using data tables in Spock

def "Trivial adder test"() { given: "an adder" Adder adder = new Adder() expect: "that it calculates the sum of two numbers" adder.add(first,second)==sum #A where: "some scenarios are" first |second || sum #B 1 | 1 || 2 #C 3 | 2 || 5 #C 82 | 16 || 98 #C 3 | -3 || 0 #C 0 | 0 || 0 #C}

#A relationship between output and input parameters. Sum is based on first and second

#B names of parameters, first and second are input and sum is output

#C scenarios that will be tested contain values for first and second and expected sum

The data table contains a header that names each parameter. You have to make sure that the names you give to parameters do not clash with existing variables in the source code (either in local scope or global scope).

You will notice that the data table is split with either single (|) or dual (|) pipe symbols. The single pipe denotes a column, and the double pipe shows where the input parameters stop and where the output parameters start. Normally only one column in a data table uses dual pipes.

In the simple example of listing 5.4 it is obvious what is the output parameter. In more complex examples such as listing 5.3 or the examples with the nuclear reactor in chapter 4, the dual pipe is much more helpful. Keep in mind that the dual pipe symbol is used strictly for readability purposes and it does not affect the way Spock uses the data table. You can omit it if you think that it is not needed (my personal recommendation is to include it always).

If you are a seasoned Java developer you should have noticed something strange in listing 5.4[42]. The types of the parameters are never declared. The data table contains the name and values of parameters but not their type!

Remember that Groovy (as explained in chapter 2) is an optionally typed language. In the case of data tables, Spock can understand the type of input and output parameters by the context of the unit test.

It is possible however to define explicitly the types of the parameters by using them as arguments in the test method as shown in listing 5.5

Listing 5.5 Using data tables in Spock with typed parameters

def "Trivial adder test (alt)"(int first, int second, int sum) { #A given: "an adder" Adder adder = new Adder() expect: "that it calculates the sum of two numbers" adder.add(first,second)==sum #B where: "some scenarios are" first |second || sum #C 1 | 1 || 2 3 | 2 || 5 82 | 16 || 98 3 | -3 || 0 0 | 0 || 0}

#A Declaring the types of parameters (all integers in this case)

#B Using the parameters as before

#C Declaring the values of parameters (all integers in this case)

Here I have included all parameters as arguments in the test method. This makes their type clear and can also help your IDE (i.e Eclipse) to understand the nature of the test parameters.

You should decide on your own if you need to declare the types of the parameters. For brevity reasons I do not declare them in any of the chapter examples. Just make sure that all developers on the team agree on the same decision.

5.2.2 Limitations of data tables

I have already stressed the fact that the where: block must be the last block in Spock test (and only an and: block can follow it as a rare exception). I have also shown you how to declare the types of parameters in listing 5.5 for the cases where it is not clear either to your IDE or even in Spock in some extreme cases.

Another corner case with Spock data tables is that they should have at least two columns. If you are writing a test that has only one parameter you must use a "filler" for a second column as shown in listing 5.6:

Listing 5.6 Data tables with one column

def "Tiff, gif, raw,mov and bmp are invalid extensions"() { given: "an image extension checker" ImageNameValidator validator = new ImageNameValidator() expect: "that only valid filenames are accepted" !validator.isValidImageExtension(pictureFile) #A where: "sample image names are" pictureFile || _ #B "screenshot.bmp" || _ #B "IMG3434.raw" || _ #B "christmas.mov" || _ #B "sky.tiff" || _ #B "dance_bunny.gif" || _ #B}

#A Output parameter is always false for this test. All images are invalid

#B Underscore acts as dummy filler for the boolean result of the test

Perhaps some of these limitations will be lifted in future versions of Spock, but for the time being you have to live with them. The advantages of Spock data tables still outperform these minor inconveniences.

5.2.3 Easy maintenance of data tables

The ultimate goal of a parameterized test is easy maintenance. Maintenance is affected by several factors, such as the size of the test, its readability, its comments and even its position inside the source tree.

The big advantage of Spock and they way it exploits data tables in parameterized tests is that it forces you to gather all input and output variables in a single place. Not only that, but unlike other solutions for parameterized tests (examples were shown with JUnit in chapter 3) data tables include both the names and the values of test parameters.

Adding a new scenario is literally a single line change. Adding a new output or input parameter is as easy as adding a new column. Here is a visual overview for listing 5.3

Figure 5.2 - Adding a new test scenario means adding a new line in the where: block. Adding a new parameters means adding a new column in the where: block

The ease of maintenance of Spock data tables is so addicting, that once you integrate data tables in your complex tests you will understand that the only reason parameterized tests are considered "hard" and boring is because of inefficient test tools.

The beauty of this format, is that data tables can be used for any parameterized test no matter the complexity involved. If you can isolate the input and output variables, the Spock test is just a simple process or writing down the requirements in the source code. I have worked in fact in enterprise projects where extracting the input/output parameters from the specifications is a more time consuming job than actually writing the unit test itself.

The extensibility of a Spock data table is best illustrated with a semi-real example as shown in listing 5.7

Listing 5.7 Capturing business needs in data tables

def "Discount estimation for the eshop"() { [...rest of code redacted for brevity..] where: "some of the possible scenarios are"price | isVip | points | order | discount | special || finalDiscount #A50 | false | 0 | 50 | 0 | false || 0 #B100 | false | 0 | 300 | 0 | false || 10 #B500 | false | 0 | 0 | 0 | true || 50 #B50 | true | 0 | 50 | 0 | false || 15 #B50 | true | 0 | 50 | 25 | false || 25 #B50 | true | 0 | 50 | 5 | false || 15 #B50 | true | 0 | 50 | 5 | true || 50 #B50 | false | 0 | 100 | 0 | false || 0 #B50 | false | 0 | 75 | 10 | false || 10 #B50 | false | 5000 | 50 | 0 | false || 75 #B50 | false | 3000 | 50 | 0 | false || 0 #B50 | true | 8000 | 50 | 3 | false || 75 #B}

#A There are six parameters that affect final discount

#B business scenarios one for each line. These are readable by business analysis

The actual unit test code is not really important. The data table contains the business requirements from the eshop example that was mentioned in chapter 1. A user selects multiple products by adding them in an electronic basket. The basket then calculates the final discount of each product which itself depends on:

· The price of the product

· The discount of the product

· Whether the customer has bonus/loyalty points

· The status of the customer (e.g. silver, gold, platinum)

· The price of the total order (i.e. the rest of the products)

· Any special deals that are active

The production code of the eshop may comprise of multiple Java classes with deep hierarchies and complex setups. With Spock you can directly map the business needs in a single data table.

Now imagine that you have just finished writing this Spock test and it passes correctly. You can just show that data table to your business analyst and ask him/her if all cases are covered. If another scenario is needed you can add it on the spot, run the test again and verify the correctness of the system.

In another situation your business analyst might not be sure about the current implementation status of the system[43] and he/she asks what happens in a specific scenario that is not yet covered by the unit test. To answer the question you don't even need to look at the production code. Again you simply add a new line/scenario in the Spock data table, run the unit test on the spot and if it passes you can answer that the requested feature is already implemented.

In less common situations, a new business requirement (or refactoring process) might add another input variable in the system. For example in the eshop scenario above, business has decided that coupon codes will be given away, that further impact that discount of a product. Rather than hunting down multiple unit test methods (as in the naive approach of listing 5.2) you can simply add a new column in the data table and have all test cases covered in one step.

Even though Spock offers several forms of the where: block that will be shown in the rest of the chapter, I tend to like the data table format for its readability and extensibility.

5.2.4 Lifecycle of the where: block

It is important to understand that the where: block in a parameterized test actually "spawns" multiple test runs (as many of its lines). A single test method that contains a where: block with ten scenarios will be run by Spock as ten individual methods. This means that all scenarios of the where: block are tested individually, so any change in state (either in the class under test or its collaborators) will simply reset in the next run.

To illustrate this individuality of data tables take a look at listing 5.8

Listing 5.8 Lifecycle of parameterized tests

class LifecycleDataSpec extends spock.lang.Specification{ def setup() { #A println "Setup prepares next run" } def "Trivial adder test"() { #B given: "an adder" #A Adder adder = new Adder() println "Given: block runs" when: "the add method is called for two numbers" #A int result = adder.add(first,second) println "When: block runs for first = $first and second = $second" then: "the result should be the sum of them" #A result == sum println "Then: block is evaluated for sum = $sum" where: "some scenarios are" #C first |second || sum 1 | 1 || 2 3 | 2 || 5 3 | -3 || 0 } def cleanup() { println "Cleanup releases resources of last run\n" } }

#A The when: and then: blocks will be executed once for each scenario

#B Single test method will run multiple times

#C Data table with three scenarios centralized input and output parameters

Because this unit test has three scenarios in the where: block the given-when-then blocks will be executed three times as well. Also, all lifecycle methods explained in chapter 4 are fully honored by parameterized tests. Bothsetup() and cleanup() will be run as many times as the scenarios of the where: block

If you run the unit test shown in listing 5.8 you will get the following output:

Setup prepares next runGiven: block runsWhen: block runs for first = 1 and second = 1Then: block is evaluated for sum = 2Cleanup releases resources of last run Setup prepares next runGiven: block runsWhen: block runs for first = 3 and second = 2Then: block is evaluated for sum = 5Cleanup releases resources of last run Setup prepares next runGiven: block runsWhen: block runs for first = 3 and second = -3Then: block is evaluated for sum = 0Cleanup releases resources of last run

It should be clear that each scenario of the where: block acts as if it was a test method on its own. This enforces the isolation of all test scenarios which is what you would expect in a well written unit test.

5.2.5 Using the @Unroll annotation

You have seen in the previous section the behavior of Spock in parameterized tests when the when: block contains multiple scenarios. Spock correctly treats each scenario as an independent run.

Unfortunately, for compatibility reasons[44], Spock still presents to the testing environment the collection of parameterized scenarios as a single test. For example, running in Eclipse the parameterized test of listing 5.8 will produce the following:

Figure 5.3 - By default, parameterized tests with multiple scenarios are shown as one test in Eclipse. The trivial adder test is shown only once even though the source code defines three scenarios

This behavior might not be a big issue when all your tests succeed. You still gain the advantage of using a full sentence as the name of the test in the same way as with non-parameterized Spock tests.

Now assume that out of the three scenarios of listing 5.8, the second scenario is a failure (while the other two scenarios pass correctly). For illustrations purposes I modify the data table as below:

where: "some scenarios are"first |second || sum1 | 1 || 23 | 2 || 73 | -3 || 0

The second scenario is obviously wrong because three plus two is not equal to seven. The other two scenarios are still correct. Running the modified unit test in Eclipse will show the following:

Figure 5.4 - When one scenario out of many fails, it is not clear which is failed one. You have to look at the failure trace, note down the parameters and go back to the source code to find the problematic line in the where: block.

Eclipse still shows the parameterized test in a single run. You can see that the test has failed but you don't know which of the scenarios is the problematic one. You have to look at the failure trace to understand what has gone wrong.

This is not very helpful when your unit test contains a lot of scenarios as the example in listing 5.8. It is crucial to be able to detect the failed scenario(s) as fast as possible.

To accommodate this issue, Spock offers the @Unroll annotation that makes multiple parameterized scenarios to "appear" as multiple test runs. The annotation can be added on the Groovy class (Spock specification) or on the test method itself as shown in listing 5.9. In the former case its effect will be applied to all test methods.

Listing 5.9 Unrolling parameterized scenarios

@Unroll #Adef "Trivial adder test"() { given: "an adder" Adder adder = new Adder() when: "the add method is called for two numbers" int result = adder.add(first,second) then: "the result should be the sum of them" result ==sum where: "some scenarios are" first |second || sum 1 | 1 || 2 #B 3 | 2 || 5 #B 3 | -3 || 0 #B}

#A Marking the test method so that multiple scenarios will appear as multiple runs

#B Scenarios that will appear as separate unit tests one for each line.

With the @Unroll annotation active, running this unit test in Eclipse will "unroll" the test scenarios and present them to the test runner as individual tests:

Figure 5.5 - By marking a parameterized test with @Unroll Eclipse now shows each run as an individual test

Of course the @Unroll annotation is even more useful when a test has failed, because you can see exactly which run was the problem. In really large enterprise projects where parameterized tests might contain a lot of scenarios, the @Unroll annotation becomes an essential tool if you want to quickly locate which scenarios have failed. Figure 5.6 shows again the same failure as before, but this time you can clearly see which scenario has failed:

Figure 5.6 - Locating failed scenarios with @Unroll is far easier than without. The failed scenario is shown instantly as a failed test.

Remember that you still get the individual failure state for each scenario if you click on it. Also note that the @Unroll annotation can be placed on the class level (i.e. the whole Spock specification) and it will apply in all test methods inside the class.

5.2.6 Documenting parameterized tests

As you have seen in the previous section, the @Unroll annotation is pretty handy when it comes to parameterized tests, because it forces all test scenarios in a single test method, to be reported as individual test runs. If you say that this feature is not really groundbreaking, and it should be the default I would agree with you[45].

But Spock has another trick! With a little more effort you can format the name shown for each scenario. And the most logical thing to include would be the test parameters as shown in listing 5.10

Listing 5.10 Printing parameters of each scenario

@Unroll("Adder test #first, #second and #sum (alt2)") #Adef "Trivial adder test (alt2)"() { given: "an adder" Adder adder = new Adder() when: "the add method is called for two numbers" int result = adder.add(first,second) then: "the result should be the sum of them" result ==sum where: "some scenarios are" first |second || sum #B 1 | 1 || 2 3 | 2 || 5 3 | -3 || 0}

#A Using parameter names with Unroll so that they are shown in the final run

#B Parameters that will be interpolated in the description of the test

The @Unroll annotation accepts a String argument where you can put any English sentence. Variables marked with # will be replaced[46] with their current values when each scenario runs. The final result of this evaluation will override the name of the unit test as shown in figure 5.7:

Figure 5.7 The parameter values for each scenario can be printed as part of the test name

I consider this feature one of the big highlights of Spock. I challenge you to find a test framework that accomplishes this visibility of test parameters with a simple technique. If you feel really lazy you can even embed the parameters directly on the test method name[47] as shown in listing 5.11.

Listing 5.11 Parameter rendering on the test method

@Unrolldef "Testing the Adder for #first + #second = #sum "{ #A given: "an adder" Adder adder = new Adder() [...rest of code is same as listing 5.10...]}

#A parameters inside the method name instead of using the unroll string

The result in Eclipse will be the same as with listing 5.10, so pick any approach you like (but as always, if you work in a team, agree beforehand on what is considered best practice).

5.2.7 Using expressions and statements in data tables

All the data tables I have shown you so far contained scalar values. There is nothing stopping you to use custom classes, collections, object factories or any other Groovy expression that results in something that can be used as an input or output parameter. Take a look at listing 5.12 below (created strictly for demonstration purposes):

Listing 5.12 Custom expressions in data tables

@Unrolldef "Testing the Adder for #first + #second = #sum "() { given: "an adder" Adder adder = new Adder() expect: "that it calculates the sum of two numbers" adder.add(first,second)==sum where: "some scenarios are" first |second || sum 2+3 | 10-2 || new Integer(13).intValue() #A MyInteger.FIVE.getNumber() #B | MyInteger.NINE.getNumber() || 14 IntegerFactory.createFrom("two") | (7-2)*2 || 12 [1,2,3].get(1) | 3 || IntegerFactory.createFrom("five") #C new Integer(5).intValue() | new String("cat").size() || MyInteger.EIGHT.getNumber() ["hello","world"].size() | 5 || IntegerFactory.createFrom("seven")}

#A full statement is used directly as a parameter in data table

#B Enumeration can be used as a parameter

#C An ObjectFactory that creates dynamically parameters

The class MyInteger is a simplistic enumeration that contains the first ten integers. The class IntegerFactory is a trivial factory that converts strings to integers. The details of the code are not really important, what you need to take away from this example is the flexibility of data tables. If you run this example, Spock will evaluate everything and present you with the final result as shown in figure 5.8

Figure 5.8 - Spock will evaluate all expressions and statements so that they can be used as normal parameters. All statements from listing 5.12 finally resolve to integers

I personally try to avoid this technique because I think it damages the readability of the test. I prefer to keep values in data tables really simple. Using too many expressions in your data tables is a sign that you need to convert the tabular data into data pipes as explained in the next section.

5.3 Data pipes

Data tables should be your bread and butter when writing Spock parameterized tests. They really shine, when all input and output parameters are known in advance and thus can be embedded directly in the source code.

Sometimes however the test parameters are computed on the spot or come from an external source (typically a file as you will see later in this chapter). For those cases, using data pipes is a better option. Data pipes are a lower level construct of Spock parameterized tests that can be used in cases where you want to dynamically create/read test parameters.[48]

As a first example let's rewrite the first data table code of listing 5.1, using data pipes this time. The result is shown in listing 5.13

Listing 5.13 Trivial example of data pipes

def "Valid images are PNG and JPEG files only"() { given: "an image extension checker" ImageNameValidator validator = new ImageNameValidator() expect: "that only valid filenames are accepted" validator.isValidImageExtension(pictureFile) == validPicture #A where: "sample image names are" pictureFile << ["scenery.jpg","house.jpeg", "car.png ","sky.tiff" dance_bunny.gif" ] #B validPicture << [ true, true, false, false, false] #C}

#A Relationship of input and output parameters

#B All values of input parameter are inside a collection

#C All values of output parameter are inside a collection

The code accomplishes the same thing as listing 5.1. This time however, the "tabular" format is "rotated" 90 degrees. Each line of the where: block contains a parameter and the scenarios of the test are the imaginary columns. The key point here is the usage of the left-shift operator symbol (<<). In the context of the where: block it means "for the first scenario pick the first value in the list for the input and output parameter, for the second scenario pick the second value in the list and so on".

In this example I pass both input and output parameters in a list. But the left-shift operator can work on several other things such as iterables, iterations, enumerations, other collections and even Strings. We will examine the most common cases in the next sections.

5.3.1 Dynamically generated parameters

Looking at listing 5.13 and comparing it to listing 5.2 you would be right to say that there is no real advantage for using data pipes. That is because in that particular scenario all parameters are known in advance and can be embedded directly in their full form. The power of data pipes becomes evident with computed data.

Let's see a different example where it would be impractical to use a data table due to the sheer size of input and output parameters:

Listing 5.14 Using Groovy ranges as data generators

def "Multiplying #first and #second is always a negative number"() { given: "a calculator" Calculator calc = new Calculator() expect: "that multiplying a positive and negative number is also negative" calc.multiply(first,second) < 0 #A where: "some scenarios are" first << (20..80) #B second << (-65..-5) #C}

#A No output parameter

#B This will expand to 60 positive numbers

#C This will expand to 60 negative numbers

The (M..N) notation is a Groovy range. It is similar to a list that will contain all values starting from M and ending in N. Thus the (20..80) notation means a range with all integers from 20 to 80. Groovy expands the ranges, and Spock picks each value in turn, resulting in a parameterized test with 60 scenarios. You can see the scenarios in detail if you run the unit test yourself:

Figure 5.9 - Using ranges to automatically generate 60 scenarios instead of creating a data table with 60 lines

Creating a data table with 120 values would make the unit test unreadable. By using data pipes and Groovy ranges we have created 60 scenarios on the spot while the source code contains just two statements (the ranges).

For a more realistic example, assume that you want to write an additional unit test for the ImageValidator class which ensures that all JPEG images are considered valid regardless of capitalization (anywhere in the name or the extension). Again embedding all possible combinations in a data table would be time consuming and error-prone.

You can calculate several possible variations with some Groovy magic as seen in listing 5.15

Listing 5.15 Using Groovy combinations

@Unroll("Checking image name #pictureFile")def "All kinds of JPEG file are accepted"() { given: "an image extension checker" ImageNameValidator validator = new ImageNameValidator() expect: "that all jpeg filenames are accepted regarless of case" validator.isValidImageExtension(pictureFile) where: "sample image names are" pictureFile << GroovyCollections.combinations([["sample.","Sample.","SAMPLE."], #A ['j', 'J'], ['p', 'P'],['e','E',''],['g','G']])*.join() #B}

#A combinations creates a collection of all variations

#B join() creates a string from each variation

The where: block here contains a single statement. If you run the unit test you will see that this statement creates 72 scenarios (from 3 x 2 x 2 x 3 x 2 Strings) as seen in figure 5.10

Figure 5.10 - Creating 72 unit test runs from a single Groovy statement

The code works as followings. The combinations method takes the variations of the words sample, the letters j,p,e,g and creates a new collection that contains all possible variations as collections themselves. The input parameters is a String. To convert each individual collection to a string we call the join() method that automatically creates a single string from a collection of strings. And because we want to do this with all collections we use the star-dot Groovy operator (*.) that applies the join() method to all of them.

If your head is spinning at this point, do not worry! It took me a while to write this statement, and as you gain more Groovy expertise you will be able to write Groovy liners as well. The example is supposed to impress you, but do not be distracted away by the core lesson here, which is the flexibility of Spock data pipes.

5.3.2 Parameters that stay constant

In all examples of parameterized tests I have shown you so far, all parameters are different for each scenario. There are times however where one or more parameters are constant. Spock allows you to use direct assignments if you want to indicate that a parameter is the same for each scenario. Instead of the left-shift operator you use the standard assignment operator as seen in listing 5.16

Listing 5.16 Constant parameters in Spock tests

def "Multipling #first and #second is always a negative number"() { given: "a calculator" Calculator calc = new Calculator() expect: "that multiplying a positive and negative number results in a negative number" calc.multiply(first,second) < 0 where: "some scenarios are" first << [20,34,44,67] #A second = -3 #B}

#A this parameter is different for each scenario

#B this parameter is always the same for each scenario.

The scenarios that will be used for listing 5.16 are [20, -3] then [34, -3] then [44, -3] and finally [67,-3]. I admit that the example is not very enticing. I need to show it to you as a stepping stone for the true use of the assignment operator in the where: block - derived variables.

5.3.3 Parameters that depend on other parameters

You have seen how the assignment operator is used for constant variables in listing 5.16. What is not evident from the listing is the fact that you can also refer to other variables in the definition of a variable.

In listing 5.17 the second parameter of the test is based on the first.

Listing 5.17 Derived parameters in Spock tests

def "Multipling #first and #second is always a negative number (alt)"() { given: "a calculator" Calculator calc = new Calculator() expect: "that multiplying a positive and negative number results in a number" calc.multiply(first,second) < 0 where: "some scenarios are" first << [20,34,44,67] #A second = first * -1 #B}

#A this parameter is explicitly defined, first is an integer

#B this parameter is depended to another one, second is the first with minus sign

Running this test will show you how the second parameter is recalculated for each scenario according to the value of the first as seen in figure 5.11

Figure 5.11 - Derived values are re-calculated for each test run. Here the second parameter is always the negative representation of the first one

This technique allows you to have variables that are dynamically generated based on the context of the current scenario.

5.4 Using dedicated Data Generators

In all the previous examples of data pipes I have used lists (Groovy ranges also act as lists) to hold the parameters for each test iteration. Grouping parameters in a list is the more readable option in my opinion but Spock can also iterate on:

· Strings (each iteration will fetch a character)

· Maps (each iteration will pick a key)

· Enumerations

· Arrays

· RegEx Matchers

· Iterators

· Iterables

The list is not exhaustive. Basically everything that Groovy can iterate on, can be used as a data generator. In Chapter 2 I have even included a Groovy Expando as an example of iterator. Iterables and Iterators are interfaces which means that you can implement your own classes for the greatest control on how Spock uses parameters. Even though custom implementations can handle complex transformations of test data when that is required, I consider them a last resort solution because they are not always as readable as simpler data tables.

Figure 5.12 - All solutions shown for parameterized Spock tests. Data tables are very limited but readable even by non-technical people. All other techniques sacrifice readability for more expressing power.

If you find the need to create a custom iterator for obtaining business data, you should always ask yourself if the transformation of the data actually belongs in the business class that you are trying to test or it is really part of the iterator.

Before trying custom iterators, you should spend some time to see if you can use existing classes in your application that already return data in the format that you expect. As an example of this, assume that I have a text file that holds image names that my program can accept. The contents of the file validImageNames.txt are:

hello.jpganother.jpegmodern0034.JPEGcity.Pngcity_004.PnGlandscape.JPG

In order to read this file, I do not need a custom iterator. The Groovy File class already contains a readLines() method that returns a list of all lines in a file. The respective Spock test is shown in listing 5.18

Listing 5.18 Using existing data generators

@Unroll("Checking image name #pictureFile")def "Valid images are PNG and JPEG files"() { given: "an image extension checker" ImageNameValidator validator = new ImageNameValidator() expect: "that all filenames are accepted" validator.isValidImageExtension(pictureFile) where: "sample image names are" pictureFile << new File("src/test/resources/validImageNames.txt") #A .readLines()}

#A Gets a list of all lines of the file

Here Groovy will open the file, read its lines in a list and pass them to Spock. Spock will fetch the lines one-by-one to create all the scenarios of the test. Running the test will produce the following:

Figure 5.13 - Reading test values from a file using Groovy code

Before resorting to custom iterators always see if you can obtain data with existing of your application or GDK/JDK facilities[49]. Always keep in mind the excellent facilities of Groovy for XML and JSON reading (these were shown in Chapter 2)

5.4.1 Writing a custom data generator

You show the unit test with the valid image names to your business analyst in order to explain what is supported by the system. He/she is impressed and as a new task you get the following file named invalidImageNames.txt:

#Found by QAstarsystem.tiffgalaxy.tif #Reported by client bunny04.gif looper.GIF dolly_take.mov afar.xpm

The file cannot be used as is in a unit test. It contains comments that start with the # sign, it has empty lines, and even tabs in front of some image names.

You want to write a Spock test that checks this file and confirms the rejection of the image names (they are all invalid). It is obvious that the Groovy File class cannot help you in this case, the file has to be processed before it is used in the Spock test[50].

To solve this new challenge first you should create a custom data iterator as shown in listing 5.19

Listing 5.19 Java iterator that processes invalidImageNames.txt

public class InvalidNamesGen implements Iterator<String>{ #A private List<String> invalidNames; private int counter =0; public InvalidNamesGen() { invalidNames = new ArrayList<>(); parse(); } private void parse() { [...code that reads the file and discards empty lines, tabs and comments not shown for brevity...] } @Override public boolean hasNext() { #B return counter < invalidNames.size(); } @Override public String next() { #C String result = invalidNames.get(counter); counter++; return result; } @Override public void remove() { #D }}

#A class will return Strings (lines)

#B generate values while lines are present in the file

#C get the next line from the file

#D No need to implement this for this example

There is nothing Spock specific about this class on its own. It is a standard Java iterator that reads the file and can be used to obtain String values. You can use this iterator directly in Spock as shown in listing 5.20

Listing 5.20 Using Java iterators in Spock

@Unroll("Checking image name #pictureFile")def "Valid images are PNG and JPEG files"() { given: "an image extension checker" ImageNameValidator validator = new ImageNameValidator() expect: "that all filenames are rejected" !validator.isValidImageExtension(pictureFile) #A where: "sample image names are" pictureFile << new InvalidNamesGen() #B}

#A This time we expect invalid images

#B instruct Spock to read Strings from the custom class

If you run this test, you will see that the file is correctly cleaned up and processed. Empty lines, comments and tabs are completely ignored and only the image names are used in each test scenario.

Figure 5.14 - Using a Java iterator in a Spock unit test, allows for more fine grained file reading.

Happy with the result you show the new test to your business analyst (thinking that you are finished). Apparently there is one last challenge that you must face...

Writing custom data generators in Groovy

In this section and the next, I used Java to implement a custom data generator because I assume that you are more familiar with Java. It is perfectly possible to write data generators in Groovy. In fact, this would be the preferred method once you know your way around Groovy, because you can include the generator inside the same source file as the Spock test (instead of having two separate files, one in Java for the iterator and one in Groovy for the Spock test).

5.4.2 Multi-valued data iterators

Your business analyst examines the two Spock tests (the one for valid images, and the one for invalid images) and decides that two files are not really needed. He/She combines the two files into one, called imageNames.txt with the following contents

#Found by QAstarsystem.tiff failgalaxy.tif fail desktop.png passeurope.jpg passmodern0034.JPEG passcity.Png passcity_004.PnG pass #Reported by client bunny04.gif fail looper.GIF fail dolly_take.mov fail afar.xpm fail

This file is similar to the other two with one important difference. It contains the word pass/fail in the same line as the image name[51]. At first glance it seems that you need to write a test similar to listing 5.13 but using two custom iterators like below:

where: "sample image names are" pictureFile << new CustomIterator1() validPicture << new CustomIterator2()

The first iterator is responsible for reading the image names as before, and there is a second iterator that reads the pass/fail flag and converts it to a boolean. This solution would certainly work, but it is not practical to have two custom iterators. They would both share similar code (i.e both of them need to ignore empty lines) and keeping them in sync if the file format changed would be a big challenge.

Hopefully with Spock tests you do not need extra custom iterators for each parameter. Spock supports multi-value iterators (powered by Groovy multi-valued assignments[52]) where you can obtain all your input/output parameters from a single iterator. In our example we will use for illustration purposes a custom iterator to fetch two variables but the same technique can work with any number of parameters. The iterator is shown in listing 5.21

Listing 5.21 Java multi-valued iterator

public class MultiVarReader implements Iterator< Object[]>{ #A private List<String> fileNames; private List<Boolean> validFlag; private int counter =0; public MultiVarReader() { fileNames = new ArrayList<>(); validFlag = new ArrayList<>(); parse(); } private void parse() { [...code that reads the file and discards empty lines, tabs and comments not shown for brevity...] } @Override public boolean hasNext() { #B return counter< fileNames.size(); } @Override public Object[] next() { #C Object[] result = new Object[2]; result[0] = fileNames.get(counter); result[1] = validFlag.get(counter); counter++; return result; } @Override public void remove() { #D }}

#A class will return multiple values

#B generate values while lines are present

#C first parameter is the file and second parameter is the pass/fail result

#D No need to implement this

Here the defined iterator returns two objects. We note down that the first object is the image name and the second object is a boolean that is true if the image should be considered valid or false if the image name should be rejected.

The Spock test that uses this multi-value iterator is shown in listing 5.22

Listing 5.22 Using multi-valued iterators in Spock

@Unroll("Checking image name #pictureFile with result=#result")def "Valid images are PNG and JPEG files only 2"() { given: "an image extension checker" ImageNameValidator validator = new ImageNameValidator() expect: "that all filenames are categorized correctly" validator.isValidImageExtension(pictureFile) == result #A where: "sample image names are" [pictureFile,result] << new MultiVarReader() #B}

#A result is now an output parameter

#B The iterator reads both parameters at once.

In the Spock test, the left-shift operator is used as before, but this time the left hand side is a list of parameters instead of a single parameter. Spock will read the respective values from the data generator and place them in the parameters in the order they are mentioned. The first value that comes out of the data generator will be placed in the first parameter (the image name in our case) while the second value from the generator (the Boolean flag in our case) is placed in the second parameter. Running the test will produce the following:

Figure 5.15 - Multi-valued iterators. For each test run Spocks reads both the input (image file name) and the output parameter (result of validity) from the data file.

Note that this capability of the left-shift operator to handle multi-values is not restricted to data generators (although that is where it is most useful). It is possible to perform multi-value parameter assignments using plain data pipes, as shown in listing 5.23

Listing 5.23 Using multi-valued assignments in Spock

@Unroll("Checking harcoded image name #pictureFile with #result")def "Valid images are PNG and JPEG files only"() { given: "an image extension checker" ImageNameValidator validator = new ImageNameValidator() expect: "that all filenames are categorized correctly" validator.isValidImageExtension(pictureFile) == result #A where: "sample image names are" [pictureFile,result] << [["sample.jpg",true],["bunny.gif",false]]}

#A Two parameters will be used in this Spock test pictureFile and result

#B multi-value assignment. The first parameter is pictureFile and the second is result

The right hand side of the assignment contains a list of all scenarios (which themselves are lists). For each scenario Spock again will pick the first element and place it in the first variable of the left-hand side list. The second element from the scenario will be placed in the second parameter and so on.

5.5 Third party data generators

With the current breadth of Java/Groovy input libraries that handle text, JSON, XML, CSV and other structure data it is very easy to write custom iterators that can handle your specific business case.

If you have already invested in JUnit tools that generate random data, or construct data according to your needs it should be very easy to adapt them for your Spock tests as well. If they already implement the iterator interface you can use them directly, or you can wrap them in your own data generator.

If your application is using a lot of data generators you might also find useful the Spock genesis library (https://github.com/Bijnagte/spock-genesis). It can be though as a meta-generator library because it allows you to:

· create lazily input data

· compose existing generators into new ones

· filter existing generators using predicates/closures

· randomize or order the output for other generators

Always remember that your data generators can be written in both Java and Groovy. Thus if you find a library in Java that suits you needs, you can integrate it directly in Spock.

A perfect example of this approach is the JFairy data generator library (https://github.com/Codearte/jfairy). The library itself is written in Java, but it can be used very easily in Spock. In fact its own unit tests are implemented in Spock!

5.6 Summary

· Parameterized tests are tests that share the same test logic but have multiple input and output parameters.

· Spock supports parameterized test via the where: block that defines what are the individual scenarios

· The simplest usage of the where: block is with data tables where you embed directly all parameters inside the source code.

· Data tables are very readable because the collect in a single place the names and values of all parameters.

· Each scenario on the where: block has its own lifecycle. A parameterized test with N scenarios will spawn N runs.

· The @Unroll annotation can be used to report individual runs of each scenario in its own test.

· In conjunction with the @Unroll annotation it is also possible to change the name of each test method to include the input and output parameters. This makes reporting very clear and pinpointing a failed test is very easy.

· It is possible to use statements and Groovy expressions in data tables. Care should be exercised to not harm the readability of the test

· A more advanced form of parameterized test is data pipes. These allow the automatic calculation of input/output parameters when it is not practical to embed them directly in the source code (either because of their size or their complexity)

· Data pipes can get data from a collection, Groovy ranges, Strings and even Regular Expression matchers. Anything that is iterable in Groovy can be used as data generator.

· It is possible to have test parameters depend on other test parameters. It is also very easy to define a test parameter as a constant

· Existing libraries/classes can be easily wrapped in a data generator or used directly as an iterator.

· Spock can assign multiple variables at once for each scenario. This is also possible in plain Groovy. Data generators are not limited to generating data for a single value. Multi-valued data generators can be used to handle all input/output parameters of a scenario in a single step

· Data generators can be implemented in both Java and/or Groovy.

[42] and also in listing 5.3 and 5.2 if you have been paying attention so far...

[43] A very common case in legacy projects

[44] with older IDEs and tools that are not very smart when it comes to JUnit runners

[45] After all, JUnit does this as well.

[46] The reasons that the # symbol is used instead of $ are purely technical and are not very relevant unless you are really interested in Groovy and Spock internals.

[47] Take that TestNG !

[48] Data tables can be seen as an abstraction over data pipes

[49] or even classes from Guava, Apache commons, CSV reading libraries etc.

[50] In this simple example, one could clear the file contents manually. You can imagine of course a larger file where this is not practical or even possible.

[51] In reality, this file would be a large XLS file with multiple columns that contained both important and unrelated data.

[52] http://www.groovy-lang.org/semantics.html#_multiple_assignment