Java Testing with Spock, Version 6 (2015)
PART 3: USING SPOCK FOR ALL KINDS OF TESTING NEEDED IN A BIG APPLICATION
Chapter 8. Spock features for Enterprise testing
In this chapter
· Spock annotations helpful to Enterprise testing
· Refactoring large Spock tests
· Testing legacy code with Spy Objects
One of the good qualities of a flexible software tool is the ability to adapt to any software situation and especially the corner cases that appear with large Enterprise projects. Enterprise projects often come in large code bases (think millions of code lines), have an endless stream of requirements and more often than not contain legacy modules that cannot be changed for political or technical reasons.
I said in chapter 1 that Spock is a holistic testing solution that will cover your needs regardless of the size of the application and whether you work solo or as part of a large team. A bigger codebase always amounts to extra complexity in all fronts (compilation, documentation, delivery) and it is good to know that Spock has you covered even when your needs are off the beaten path.
In this last chapter of the book I will show you some extra Spock features that are geared towards large Enterprise projects. These techniques are in no way "essential" for Spock testing as they solve very specific problems that you may not encounter in your current project. Therefore, before employing any of the advice shown in this chapter, make sure that you indeed suffer from the respective problem. More importantly, the last section explains Spy objects, a technique that I strongly advise you NOT to use, unless this is your last resort.
There are three distinct parts in this chapter and I have listed them roughly in the order I expect you to use them in your Spock tests:
1. Spock annotations for timeouts, exceptions, conditional test running, etc.
2. Refactoring of large then: blocks that contain assertions or interactions
3. Using Spies as partial mocks (continuing fake objects from chapter 6)
Spies are a very controversial feature (not just with Spock[110]) so make sure that you understand the implications of using them in your unit tests (and what that means for Java production code). Usage of spies implies that your Java code suffers from design problems as I will show you at the last section of the book.
8.1 Additional Spock features for Enterprise tests
In chapter 4 I covered all Spock blocks in detail and also talked about the general structure of a Spock test. Spock offers several complementary features in the form of annotations that further enhance the expressiveness of your tests.
The Spock tests demonstrated here are based on the eShop example introduced in chapter 6. They revolve around placing products in an electronic basket and paying via credit card.
8.1.1 Testing the (non)existence of exceptions : Thrown() and NotThrown()
In all Spock tests that I have shown you so far, the expected result is either a set of assertions or the verification of object interactions.
In some cases however the "expected" result is throwing an exception. If you are developing a library framework for example you have to decide what exceptions will be thrown to the calling code and verify this decision with a Spock test.
Listing 8.1 Expecting an exception in a Spock test
def "Error conditions for unknown products"() { given: "a warehouse" WarehouseInventory inventory = new WarehouseInventory() when: "warehouse is queried for the wrong product" inventory.isProductAvailable("productThatDoesNotExist",1) then: "an exception should be thrown" thrown(IllegalArgumentException) #A}
#A This test will pass only if an IllegalArgumentException is thrown in the when: block
Here I have designed my Warehouse class so that it throws an exception when it cannot find the name of a product. When this test runs, I explicitly tell Spock that the when: block will throw an exception. The test will fail if an exception is not thrown.
Figure 8.1 - The test will fail if an exception is not thrown in the when: block
Note that in this case I used an existing exception as offered by Java, but the same syntax would work with any kind of exception (I could have created a custom exception class called ProductNotFoundException instead).
It is also possible to "capture" the exception thrown and perform further assertions in order to make the test stricter.
Listing 8.2 Detailed examination of an expected exception
def "Error conditions for unknown products - better"() { given: "a warehouse" WarehouseInventory inventory = new WarehouseInventory() when: "warehouse is queried for the wrong product" inventory.isProductAvailable("productThatDoesNotExist",1) then: "an exception should be thrown" IllegalArgumentException e = thrown() #A e.getMessage() == "Uknown product productThatDoesNotExist" #B}
#A Keeps the exception thrown in the e variable
#B The test will only pass if exception contains a specific message
Listing 8.2 further enhances the code of listing 8.1 by checking both the type of the exception and its message. Here I examine the built-in message property that is present in all Java exceptions, but again I could examine any property of a custom made exception instead (i.e. the last statement in listing 8.2 is a standard Groovy assertion)
Finally, it is possible to define in a Spock test that you do NOT expect an exception for an operation in the when: block. I have to admit that the semantics of this syntax are very subtle but the capability is there if you need it.
Listing 8.3 Explicit declaration that an exception should not happen
def "Negative quantity is the same as 0"() { given: "a warehouse" WarehouseInventory inventory = new WarehouseInventory() and: "a product" Product tv = new Product(name:"bravia",price:1200,weight:18) when: "warehouse is loaded with a negative value" inventory.preload(tv,-5) then: "the stock is empty for that product" notThrown(IllegalArgumentException) #A !inventory.isProductAvailable(tv.getName(),1)}
#A Clarifies the intention of testing normal operation without exception.
I personally believe that the notThrown() syntax is intended as a hint to the human reader of the test and not so much to the test framework itself.
8.1.2 Mapping Spock tests to your issue tracking system : @Issue
In chapter 4 you saw the @Subject, @Title, and @Narrative annotations that serve as metadata for the Spock test. These annotations are particularly useful to non-technical readers of the tests (e.g. business analysts) and will really show their value when reporting tools use them for extra documentation.
Any non-trivial enterprise application has a product backlog or issue tracker that serves as a central database of current bugs and future features. Spock comes with an @Issue annotation that allows you to mark a test method that solves a specific issue with the code:
Listing 8.4 Marking a test method with the issue it solves
@Issue("JIRA-561") #Adef "Error conditions for unknown products"() { given: "a warehouse" WarehouseInventory inventory = new WarehouseInventory() when: "warehouse is queried for the wrong product" inventory.isProductAvailable("productThatDoesNotExist",1) then: "an exception should be thrown" thrown(IllegalArgumentException)}
#A This test method verifies the fix that happened for JIRA issue 561
Notice that the annotation has a strictly informational role. At least at the time of writing there is no automatic connection to any external system (i.e. to JIRA[111] in this example). In fact, the value inside the annotation is regarded as free text by Spock. Here is another example where I choose to use a full URL of a Redmine[112] tracker
Listing 8.5 Using the URL of an issue solved by a Spock test
@Issue("http://redmine.example.com/issues/2554") #Adef "Error conditions for unknown products - better"() { given: "a warehouse" WarehouseInventory inventory = new WarehouseInventory() when: "warehouse is queried for the wrong product" inventory.isProductAvailable("productThatDoesNotExist",1) then: "an exception should be thrown" IllegalArgumentException e = thrown() e.getMessage() == "Uknown product productThatDoesNotExist"}
#A This test method verifies the fix that happened for Redmine issue 2554
Finally, a common scenario is having multiple issue reports that stem from the same problem. Spock has you covered and you can use multiple issues as shown in listing 8.6
Listing 8.6 Marking a Spock test with multiple issues
@Issue(["JIRA-453","JIRA-678","JIRA-3485"]) #Adef "Negative quantity is the same as 0"() { given: "a warehouse" WarehouseInventory inventory = new WarehouseInventory() and: "a product" Product tv = new Product(name:"bravia",price:1200,weight:18) when: "warehouse is loaded with a negative value" inventory.preload(tv,-5) then: "the stock is empty for that product" notThrown(IllegalArgumentException) !inventory.isProductAvailable(tv.getName(),1)}
#A This test method verifies the fix for three duplicate bugs
The @Issue annotation is also very handy when you practice test driven development, as you can use it to mark Spock tests for product features before writing the production code itself.
8.1.3 Failing tests that do not finish on time : @Timeout
In chapter 7 we talked about integration and functional tests and how they differ from pure unit tests. A common characteristic of integration tests is their slow execution time because of real databases, web services and external systems that are often used as part of the test.
Getting quick feedback from a failed unit test should be one of your primary goals when writing integration tests. The external systems used in integration tests can affect the execution time in a non-deterministic way as their response time is affected by their current load or other environmental reasons.
Spock comes with an @Timeout annotation that unconditionally fails a test if its execution time passes the given threshold.
Listing 8.7 Declaring a test timeout
@Timeout(5) #Adef "credit card charge happy path"() { given: "a basket, a customer and a TV" Product tv = new Product(name:"bravia",price:1200,weight:18) BillableBasket basket = new BillableBasket() Customer customer = new Customer(name:"John",vip:false,creditCard:"testCard") and: "a credit card service" CreditCardProcessor creditCardSevice = new CreditCardProcessor() #B basket.setCreditCardProcessor(creditCardSevice) when: "user checks out the tv" basket.addProduct tv boolean success = basket.checkout(customer) #C then: "credit card is charged" success}
#A This test should finish within 5 seconds
#B The CreditCardProcessor class is an external service
#C This is a lengthy operation that contacts the credit card service
The reasoning behind the timeout annotation, is that it helps you isolate quickly environmental problems in your integration tests. If a service is down, then there is no point in waiting for the full timeout of your Java code (which could be 30 minutes for example) before moving to the next unit test.
Using the timeout annotation you can set your own bounds on the "expected" runtime of an integration test and have Spock automatically enforce it. The default unit is seconds as shown in the previous listing, but you can override it with your own setting:
Listing 8.8 Declaring a test timeout - custom unit
@Timeout(value = 5000, unit = TimeUnit.MILLISECONDS) #Adef "credit card charge happy path - alt "() { given: "a basket, a customer and a TV" Product tv = new Product(name:"bravia",price:1200,weight:18) BillableBasket basket = new BillableBasket() Customer customer = new Customer(name:"John",vip:false,creditCard:"testCard") and: "a credit card service" CreditCardProcessor creditCardSevice = new CreditCardProcessor() basket.setCreditCardProcessor(creditCardSevice) when: "user checks out the tv" basket.addProduct tv boolean success = basket.checkout(customer) then: "credit card is charged" success}
#A Treats the defined value as milliseconds
The importance of the Timeout annotation is evident in case of multiple long tests that take a long time to finish. I have seen build jobs that normally take some minutes and due to a misconfiguration can take hours if timeouts are not used correctly.
8.1.4 Ignoring certain Spock tests
A large enterprise application can have thousands of unit tests. In an ideal world all of them would be active at any given time. In real life, this is rarely the case.
Test environments that get migrated, features that wait to be implemented, business requirements that are not yet frozen are common reasons that force some tests to be skipped. Fortuantely, Spock offers several ways to skip one or more tests deliberately so your tests don't fail while these restructurings and developments are taking place.
Ignoring a Single Test : @Ignore
Spock allows you to skip knowingly one or more tests and even provides you with the ability to give a reason for skipping that test.
Listing 8.9 Ignoring a single test
@Ignore("Until credit card server is migrated") #Adef "credit card charge happy path"() { given: "a basket, a customer and a TV" Product tv = new Product(name:"bravia",price:1200,weight:18) BillableBasket basket = new BillableBasket() Customer customer = new Customer(name:"John",vip:false,creditCard:"testCard") and: "a credit card service" CreditCardProcessor creditCardSevice = new CreditCardProcessor() basket.setCreditCardProcessor(creditCardSevice) when: "user checks out the tv" basket.addProduct tv boolean success = basket.checkout(customer) then: "credit card is charged" success }
#A This test will be skipped when Spock runs it
The primary purpose of having a test skipped, is so that the rest of your test suite runs successfully by your build server. An ignored test should always be a temporary situation as you are vulnerable to code changes that would normally expose a bug verified by that test.
The human readable description inside the @Ignore annotation should give a hint on why this test is ignored (the value is free text as far as Spock is concerned). More often than not the original developer that marked a test as ignored does not always remove the @Ignore annotation, so it is essential to document inside the source code why the test was skipped in the first place.
You can place @Ignore either on a single test method or a whole class if you want all its tests methods to be skipped.
Ignoring All But One Test : @Ignorerest
If you are also lucky and instead of ignoring a single test method, you want to ignore all tests in a Spock specification apart from one, you can use the @IgnoreRest annotation.
Listing 8.10 Ignoring all tests apart from one
class KeepOneSpec extends spock.lang.Specification{ def "credit card charge - integration test"() { #A [...code redacted for brevity...] } @IgnoreRest #B def "credit card charge with mock"() { #C [...code redacted for brevity...] } def "credit card charge no charge - integration test"() { #D [...code redacted for brevity...] } }
#A This test uses the real credit card service - it will be skipped
#B Marking this test as the only one that will run
#C This test uses only mocks and thus can run normally
#D This test uses the real credit card service - it will be skipped
Running the Spock test shown in listing 8.10 will produce the following:
Figure 8.2 - Only the test marked with @IgnoreRest runs
Again I have to admit that this Spock annotation is very specialized and you might never need to use it.
Ignoring Spock tests according to the runtime environment : @IgnoreIf Part 1
The @Ignore annotations shown in the previous paragraph are completely static. A test is either skipped or not, and that decision is taken during compile time.
Spock offers a set of smarter @Ignore annotations that allow you to skip tests dynamically (i.e. by examining the runtime environment). As a first step, Spock allows a test to query
· the current environment variables
· the JVM system properties
· the operating system
and decide whether it will run or not depending on that result.
Listing 8.11 Skipping Spock tests according to the environment
class SimpleConditionalSpec extends spock.lang.Specification{ @IgnoreIf({ jvm.java9 }) #A def "credit card charge happy path"() { [...code redacted for brevity...] } @IgnoreIf({ os.windows }) #B def "credit card charge happy path - alt"() { [...code redacted for brevity...] } @IgnoreIf({ env.containsKey("SKIP_SPOCK_TESTS") }) #C def "credit card charge happy path - alt 2"() { [...code redacted for brevity...] } }
#A This test will be skipped on Java 9
#B This test will be skipped if run on Windows
#C This test will be skipped if the property SKIP_SPOCK_TESTS is defined
Running the listing above on my Windows system with JDK 7 and no extra JVM properties does the following:
Figure 8.3 A test is skipped because current OS is windows
I won't list all possible options supported by Spock. You can find the full details in its source code[113]. Ignoring tests depending on environment variables opens up interesting possibilities of allowing to split your tests in separate categories/groups which is a well known technique. An example would be to create "fast" and "slow" tests and setup your build server with two different jobs for a different feedback lifecycle.
Ignoring certain Spock tests with pre-conditions : @IgnoreIF Part 2
To obtain the maximum possible flexibility from @IgnoreIf annotations you need to define your own custom conditions.
This is very easily done in Spock, because the @IgnoreIf annotation actually accepts a full closure. The closure will be evaluated and the test will be skipped if the result is false. Here is a smarter Spock test that only runs if theCreditCardService is up and running.
Listing 8.12 Skipping a Spock test based on a dynamic precondition
@IgnoreIf({ !new CreditCardProcessor().online() }) #Adef "credit card charge happy path - alt"() { given: "a basket, a customer and a TV" Product tv = new Product(name:"bravia",price:1200,weight:18) BillableBasket basket = new BillableBasket() Customer customer = new Customer(name:"John",vip:false,creditCard:"testCard") and: "a credit card service" CreditCardProcessor creditCardSevice = new CreditCardProcessor() #B basket.setCreditCardProcessor(creditCardSevice) when: "user checks out the tv" basket.addProduct tv boolean success = basket.checkout(customer) #C then: "credit card is charged" success}
#A This test will only run it the method online() of the credit card service returns true
#B The CreditCardProcessor class is an external service
#C This is operation contacts the credit card service
In listing 8.12 I am assuming that the Java class which represents the external credit card system has a built-in method called online() that performs a "ping" on the remote host.
Spock will run this method and if it gets a negative result it will skip the test (there is no point in running it if the service is down).
Of course the contents of the closure passed as an argument in the @IgnoreIf annotation can be any custom code you write. If for example the built-in online() method wasn't present I could create my own Java (or Groovy) class that performed an HTTP request (or something appropriate) to the external system and have that inside the closure.
Reversing the Boolean condition of IgnoreIf : @Requires
If for some reason you find yourself always reverting the condition inside the @IgnoreIf annotation (as seen in listing 8.12 for example) you can instead use the @Requires annotation.
Listing 8.13 Requires is the opposite of IgnoreIf
@Requires({ new CreditCardProcessor().online() }) #Adef "credit card charge happy path"() { given: "a basket, a customer and a TV" Product tv = new Product(name:"bravia",price:1200,weight:18) BillableBasket basket = new BillableBasket() Customer customer = new Customer(name:"John",vip:false,creditCard:"testCard") and: "a credit card service" CreditCardProcessor creditCardSevice = new CreditCardProcessor() #B basket.setCreditCardProcessor(creditCardSevice) when: "user checks out the tv" basket.addProduct tv boolean success = basket.checkout(customer) #C then: "credit card is charged" success }
#A This test will only run it the method online() of the credit card service returns true
#B The CreditCardProcessor class is an external service
#C This is operation contacts the credit card service
The @Requires annotation has the same semantics as @IgnoreIf but with the reverse behavior. The test will be skipped by Spock if the code inside the closure does NOT evaluate to true. The option to use one or the other annotation comes as a personal preference.
8.1.5 Automatic cleaning of resources : @AutoCleanup
In chapter 4 I showed you the cleanup: block as a way to release resources (e.g. database connections) at the end of a Spock test regardless of its result.
An alternative way to achieve the same thing is by using the @AutoCleanup annotation as shown below:
Listing 8.14 Release resources with AutoCleanup
@AutoCleanup("shutdown") #Aprivate CreditCardProcessor creditCardSevice = new CreditCardProcessor() def "credit card connection is closed down in the end"() { given: "a basket, a customer and a TV" Product tv = new Product(name:"bravia",price:1200,weight:18) BillableBasket basket = new BillableBasket() Customer customer = new Customer(name:"John",vip:false,creditCard:"testCard") and: "a credit card service" basket.setCreditCardProcessor(creditCardSevice) when: "user checks out the tv" basket.addProduct tv boolean success = basket.checkout(customer) then: "credit card is charged" success}
#A The shutdown() method of the credit card service will be called at the end of the tests
If you mark a resource with the @AutoCleanup annotation, Spock will make sure that the close() method will be called on that resource at the end of the test (even if the test fails). You can use the annotation on anything you consider as a resource in your tests. Database connections, file handles and external services are good candidates for the @AutoCleanup annotation
You can override the method name that will be called by using it as argument in the annotation as I have done in listing 8.11. In my example the method shutdown() will be called instead (Spock will call close() by default)
I personally prefer to use the cleanup: block and cleanup()/cleanupSpec() methods as explained in chapter 4 (especially when multiple resources must be released), but if you are a big fan of annotations feel free to use@AutoCleanup instead[114].
As you might guess, @AutoCleanup works both with instance fields and objects marked with the @Shared annotation shown in chapter 4.
This concludes the additional Spock annotations[115], and we can now move to refactoring of big Spock tests.
8.2 Handling large Spock tests
The projects I have used in most examples so far, are trivial projects designed as a learning material instead of a production quality application. In the real world, Enterprise projects come with huge codebases that directly affect the size of unit tests as well.
Even in the case of pure unit tests (i.e. non-integration tests), preparing the class under test and its collaborators is often a lengthy process with many statements and boiler plate code that is essential for the correct functionality of the Java code tested, but otherwise unrelated to the business feature being tested.
I have provided some hints for making clear the intention of Spock tests using Groovy with() and Spock with() methods as seen in Chapter 4. In this section of the book, we will take this grouping of statements one step further by completely refactoring the respective statements in their own methods.
The running example here is a loan approval application as shown in figure 8.4:
Figure 8.4 A customer requests a loan from a bank. The bank approves or rejects the loan
The Java classes that take part in the system are
· Customer.java
· Loan.java
· CreditCard.java
· ContactDetails.java
· BankAccount.java
You can find the full source code in the Github repository of the book[116], but notice that most classes are just skeletons designed to demonstrate specific techniques in the Spock tests.
8.2.1 Using helper methods to improve code readability
In chapter 4 I stressed the importance of the when: block and how critical it is to keep its code short and understandable. In big enterprise projects however, long code segments can appear in any Spock block harming the readability of the test. As a starting example let's see a unit test that has a very long setup process:
Listing 8.15 A Spock test with very long setup - DO NOT DO THIS
def "a bank customer with 3 credit cards is never given a loan"() { given: "a customer that wants to get a loan" Customer customer = new Customer(name:"John Doe") and: "his credit cards" #A BankAccount account1 = new BankAccount() account1.with { setNumber("234234") setHolder("John doe") balance=30 } CreditCard card1 = new CreditCard("447978956666") card1.with{ setHolder("John Doe") assign(account1) } customer.owns(card1) BankAccount account2 = new BankAccount() account2.with{ setNumber("3435676") setHolder("John Doe") balance=30 } CreditCard card2 = new CreditCard("4443543354") card2.with{ setHolder("John Doe") assign(account2) } customer.owns(card2) BankAccount account3 = new BankAccount() account2.with{ setNumber("45465") setHolder("John Doe") balance=30 } CreditCard card3 = new CreditCard("444455556666") card3.with{ setHolder("John Doe") assign(account3) } customer.owns(card3) when:"a loan is requested" #B Loan loan = new Loan() customer.requests(loan) then: "loan should not be approved" !loan.approved #C}
#A A badly designed and: block. It contains too much code
#B A well designed when: block. Code is very short
#C A well designed then: block. Code is very short
At first glance this unit test follows correctly the best practices I outlined in chapter 4. All the blocks have human readable descriptions, the when: block clearly shows what is being tested (i.e. a loan request) and the final result is also very clear (either the loan approved or rejected).
The setup of the test however is a gigantic piece of code that is neither clear nor directly relevant to the business case tested. The description of the block talks about credit cards but it contains code that creates both credit cards and bank accounts (because apparently a credit card requires a valid bank account in place).
Even with the use of the with() method for grouping several statements that act on the same project, the setup code make the test very hard to read. It contains a lot of variables that are not immediately clear if they affect the test or not. For example, does it matter that the account balance is 30 dollars in each connected account? Does this affect the approval of the loan or not? One cannot answer that questions simply be reading the Spock test.
In such cases a refactoring must take place so that the intention of the test becomes clear and concise. Large amounts of code should be extracted to helper methods as shown in listing 8.16
Listing 8.16 Spock test with helper methods
def "a bank customer with 3 credit cards is never given a loan -alt"() { given: "a customer that wants to get a loan" Customer customer = new Customer(name:"John Doe") and: "his credit cards" customer.owns(createSampleCreditCard("447978956666","John Doe")) #A customer.owns(createSampleCreditCard("4443543354","John Doe")) #A customer.owns(createSampleCreditCard("444455556666","John Doe")) #A when:"a loan is requested" Loan loan = new Loan() customer.requests(loan) then: "loan should not be approved" !loan.approved} private CreditCard createSampleCreditCard(String number, String holder) #B{ BankAccount account = new BankAccount() #C account.with{ setNumber("45465") setHolder(holder) balance=30 } CreditCard card = new CreditCard(number) card.with{ setHolder(holder) assign(account) } return card #D}
#A setup code is now short and clear
#B A helper method that deals with Credit card
#C The fact that each credit card needs a bank account is hidden in the helper method
#D This helper method creates a credit card
Here I have extracted the common code into a helper method. The helper method has the following positive effects:
1. It obviously cuts down the amount of setup code
2. It clearly shows that the setup code is set of sample credit cards
3. It hides the fact that a bank account is needed for creating a credit card (as this is unrelated to the approval of a loan)
4. It shows by its arguments that the holder of the credit card must be the same as the customer that requests the loan.
The added advantage of helper methods is that you can share them across test methods or even across specifications (by creating an inheritance among Spock tests for example). You should therefore design them so they can be reused by multiple tests.
Depending on your business case you can further refine the helper methods you use to guide the reader of the test to what exactly is being tested. In a real world project I might modify the Spock test as below:
Listing 8.17 Using arguments that imply their importance in the test
def "a bank customer with 3 credit cards is never given a loan -alt 2"() { given: "a customer that wants to get a loan" String customerName ="doesNotMatter" #A Customer customer = new Customer(name:customerName) and: "his credit cards" customer.owns(createSampleCreditCard("anything",customerName)) #B customer.owns(createSampleCreditCard("whatever",customerName)) #B customer.owns(createSampleCreditCard("notImportant",customerName)) #B expect: "customer already has 3 cards" customer.getCards().size() == 3 #D when:"a loan is requested" Loan loan = new Loan() customer.requests(loan) then: "therefore loan is not approved" !loan.approved}
#A Enforces the same customer for the loan and credit cards
#B Makes it clear that credit card numbers are unused in loan approval
#C Explicitly verifies the result of the setup
In the improved listing 8.17 I have made some minor adjustments with the arguments of the helper method. First of all I use a single variable for the customer name. This guards against any spelling mistakes so that I can be sure that all credit cards are assigned to same customer (because as the description of the test says, the number of credit cards of the customer are indeed examined for loan approval).
Secondly I have replaced the credit card numbers with dummy strings. This helps the reader of the test understand that the number of each credit card is not really used in loan approval.
As a final test I have added an expect: block (as was demonstrated in chapter 4) that strengthens the readability of the setup code.
After all these changes compare listings 8.15 with 8.17. In the first case we have a huge setup code that is very hard to read, while in the second case, you can understand in seconds that the whole point of the setup code is to assign credit cards to the customer.
8.2.2 Reusing assertions in the then: block
Helper methods should be used in all Spock blocks, when you feel that the size of the code gets out of hand.
However, due to some technical limitations, the creation of helper methods for then: block requires some special handling. Again as a starting example of a questionable design let's start with a big then: block as shown in listing 8.18
Listing 8.18 Spock test with dubious then: block
def "Normal approval for a loan"() { given: "a bank customer" Customer customer = new Customer(name:"John Doe",city:"London",address:"10 Bakers",phone:"32434") and: "his/her need to buy a house " Loan loan = new Loan(years:5, amount:200.000) when:"a loan is requested" customer.requests(loan) then: "loan is approved as is" loan.approved #A loan.amount == 200.000 #A loan.years == 5 #A loan.instalments == 60#A loan.getContactDetails().getPhone() == "32434" #B loan.getContactDetails().getAddress() == "10 Bakers" #B loan.getContactDetails().getCity() == "London" #B loan.getContactDetails().getName() == "John Doe" #B customer.activeLoans == 1 #A}
#A These examine the loan approval
#B These checks are secondary
Here the then: block contains multiple statements with different significance. First there are some very important checks that the loan is indeed approved. Then there are other checks that examine the details of the approved loan (and especially the fact that they match the customer that request it).
Finally it is not clear if the numbers and strings that take part in the then: block are arbitrary or depend on something else[117].
As I first step to improve this test, I will split the then: block into two parts and also group similar statements as shown below:
Listing 8.19 Improved Spock test with clear separation of checks
def "Normal approval for a loan - alt"() { given: "a bank customer" Customer customer = new Customer(name:"John Doe",city:"London",address:"10 Bakers",phone:"32434") and: "his/her need to buy a house " int sampleTimeSpan=5 #A int sampleAmount = 200.000 #A Loan loan = new Loan(years:sampleTimeSpan, amount:sampleAmount) when:"a loan is requested" customer.requests(loan) then: "loan is approved as is" with(loan) #B { approved amount == sampleAmount years == sampleTimeSpan instalments == sampleTimeSpan * 12 #C } customer.activeLoans == 1 and: "contact details are kept or record" #D with(loan.contactDetails) { getPhone() == "32434" getAddress() == "10 Bakers" getCity() == "London" getName() == "John Doe" }}
#A Makes clear the connection between expected results
#B Grouping of primary loan checks
#C Makes clear the expected result
#D Different block for secondary checks
The improved version of the test clearly splits the checks according to the business case. I have also replaced the number 60 which was previously a magic number with the full logic that installments are years times 12 (for monthly installments).
The code that checks loan details has still hardcoded values. I can further improve the code by using helper methods:
Listing 8.20 Using helper methods for assertions
def "Normal approval for a loan - improved"() { given: "a bank customer" Customer customer = new Customer(name:"John Doe",city:"London",address:"10 Bakers",phone:"32434") and: "his/her need to buy a house " int sampleTimeSpan=5 int sampleAmount = 200.000 Loan loan = new Loan(years:sampleTimeSpan, amount:sampleAmount) when:"a loan is requested" customer.requests(loan) then: "loan is approved as is" loanApprovedAsRequested(customer,loan,sampleTimeSpan,sampleAmount) #A and: "contact details are kept or record" contactDetailsMatchCustomer(customer,loan) #A} private void loanApprovedAsRequested(Customer customer,Loan loan,int originalYears,int originalAmount){ with(loan) #B { approved amount == originalAmount loan.years == originalYears loan.instalments == originalYears * 12 } assert customer.activeLoans == 1 #C} private void contactDetailsMatchCustomer(Customer customer,Loan loan ){ with(loan.contactDetails) { phone == customer.phone #D address == customer.address #D city == customer.city #D name== customer.name #D }}
#A Helper methods with descriptive names
#B with() method works as expected in helper method
#C assert keyword is needed in helper method
#D Clear connection between loan and customer that requested it
In listing 8.20 I have refactored the two separate blocks into their own helper methods. The important thing to note is the format of each helper method.
Your first impulse might be to design each helper method to return a boolean if all its assertions pass, and have Spock check the result of that single boolean. This does not work as expected.
The recommended approach as shown in listing 8.20 is to have helper methods as void methods. Inside each helper method you can put one of the following
· A group of assertions with the Spock with() method
· a Groovy assert but WITH the assert keyword prepended.
Notice the line:
assert customer.activeLoans == 1
Because this statement exists in a helper method and not directly in a then: block, it needs the assert keyword so that Spock can understand that it is an assertion. If you miss the assert keyword the statement will pass the test regardless of the actually result (which is a very bad thing)
In listing 8.20 I have also refactored the second helper method to validate loan details against its arguments instead of hardcoded values. This makes the helper method reusable in other test methods where the customer could have other values.
Spend some time to compare listing 8.20 with the starting example at listing 8.18 to see the gradual improvement in the clarity of the unit test.
8.2.3 Reusing interactions in the then: block
As you saw in the previous section Spock needs some help to understand assertions in helper methods. A similar case happens with mocks and interactions.
Listings 8.21 shows an alternative Spock test, where the loan class is mocked instead of using the real class[118].
Listing 8.21 Spock tests with questionable then: block
def "Normal approval for a loan"() { given: "a bank customer" Customer customer = new Customer(name:"John Doe",city:"London",address:"10 Bakers",phone:"32434") and: "his/her need to buy a house " Loan loan = Mock(Loan) when:"a loan is requested" customer.requests(loan) then: "loan is approved as is" 1 * loan.setApproved(true) #A 0 * loan.setAmount(_) #A 0 * loan.setYears(_) #A _ * loan.getYears() >> 5 #B _ * loan.getAmount() >> 200.000 #B _ * loan.getContactDetails() >> new ContactDetails() #B }
#A These are the primary checks for the loan
#B These are stubbed methods needed for the correct functioning of the test
The test in listing 8.21 contains multiple interaction checks in the then: block that have a different business purpose. The Loan class is used in this case both as a mock and as a stub. This fact is implied by the cardinalities in the interaction checks.
We can improve this test by making clear the business need behind each interaction check as seen in listing 8.22:
Listing 8.22 Helper methods with interactions need to be declared explicitly
def "Normal approval for a loan - alt"() { given: "a bank customer" Customer customer = new Customer(name:"John Doe",city:"London",address:"10 Bakers",phone:"32434") and: "his/her need to buy a house " Loan loan = Mock(Loan) when:"a loan is requested" customer.requests(loan) then: "loan request was indeed evaluated" interaction { #A loanDetailsWereExamined(loan) #B } and: "loan was approved as is" interaction { #A loanWasApprovedWithNoChanges(loan) #B } } private void loanWasApprovedWithNoChanges(Loan loan){ 1 * loan.setApproved(true) 0 * loan.setAmount(_) 0 * loan.setYears(_)} private void loanDetailsWereExamined(Loan loan){ _ * loan.getYears() >> 5 _ * loan.getAmount() >> 200.000 _ * loan.getContactDetails() >> new ContactDetails()}
#A interaction blocks are needed for helper methods that contain mocks
#B Helper methods named after the business check
I have created two helper methods and also added a then: block. The first helper method hold the primary checks (i.e. the approval of the loan with its original values). The other helper method is secondary as it just contains the stubbed methods of the loan object (which are essential for the test but not really as important as the approval/rejection status of the loan)
The important thing to keep from listing 8.22 is the fact that I wrapped each helper method in an interaction block:
interaction { loanDetailsWereExamined(loan) }
This is needed so that Spock understands the special format of N * class.method(N) interaction check as was shown in chapter 6. Spock automatically understands this format in statements found directly under the then: block, but for helper methods you need to explicitly tell to Spock that statements inside the method are interaction checks.
Constructing a custom DSL for your testing needs
The Groovy language is perfect for creating your own Domain Specific Language that matches your business requirements. Rather than using simple helper methods, you can take your Spock tests to the next level, by creating a DSL that matches your business vocabulary. Creating a DSL with Groovy is outside the scope of this book, so feel free to consult chapter 19 of "Groovy in Action[119]" for more information on this topic.
8.3 Creating partial mocks with spies
In the last part of this chapter we will see how you can create partial mocks[120]. In chapter 6, I explained how Spock can create fake objects useful for testing, and showed you Mocks and Stubs. Spock supports a third type of "fake" object in the form of Spies.
Spies work as partial mocks. They take over a Java object and only mock some of its methods. Method calls can either by stubbed (like mocks) or can pass through to the real object.
Figure 8.5 A Spy is a real class where only a subset of methods are fake. The rest are the real methods.
I did not show you spies in chapter 6 on purpose, as they are a controversial technique that implies a problematic Java code. They can be useful in a very narrow set of cases. Their primary usage is creating unit tests for badly designed production code that cannot be refactored (a common scenario with legacy code).
8.3.1 A sample application with special requirements
Let's see an example that is well suited for writing a Spock test with Spies instead of mocks/stubs.
You are tasked with the development of unit tests for an existing Java application. The Java application in question is a security utility that gets video feed from an external camera and upon detecting intruders, deletes all files of the hard drive (to hide incriminating evidence)
The application code is implemented by two Java classes. The first class is responsible for deleting the hard drive, and the second class implements the face recognition algorithms that decide if the person in front of the camera is a friend or enemy:
Listing 8.23 Java code with questionable design
public class CameraFeed { #A [...code redacted for brevity...] public void setCurrentFrame(Image image) { [...code redacted for brevity...] }}public class HardDriveNuker { #B public void deleteHardDriveNow() #C { [...code redacted for brevity...] }}public class SmartHardDriveNuker extends HardDriveNuker{ #D public void activate(CameraFeed cameraFeed) #E { [...code redacted for brevity...] }}
#A Gets frames from video camera
#B Responsible for hard disk deletion
#C Immediately deletes the hard drive
#D Contains complex image recognition logic
#E calls deleteHardDriveNow() behind the scenes
You should instantly see the flawed design of this Java code. Here is an overview:
Figure 8.6 Hard drive deletion logic is hidden inside face recognition logic.
The applications does not use dependency injection. Instead of splitting responsibilities into separate entities, the application contains both the logic of deletion and the face recognition in a single "object".
You see this design flaw and start refactoring the application in order to write your unit tests. Unfortunately your boss says that the binary application is digitally signed and changing even the slightest thing in the source code will create an invalid signature[121]. Your boss says that even if you successfully refactored the code, your department does not have access to the digital certificate so you could not re-sign the binary after your change.
Therefore you need to write a unit test with the source code as is. You are asked to examine the effectiveness of the face recognition software using images of both kinds (those that have a threat and those that do not). This is one of the rare occasions that spies can be employed for unit testing.
8.3.2 Spies with Spock
You need to write a unit test the examines the activate() method of the SmartHardDriveNuker class. You know that behind the scenes it calls the deleteHardDriveNow() method. It would not be realistic to actually delete your hard drive each time you write a unit test that trigger the face recognition logic. You need to find a way where the dangerous method is mocked while the real method of the face recognition logic is kept as is.
Spock supports the creation of Spies. A spy is an fake object that automatically calls the real methods of a class unless they are explicitly mocked.
Listing 8.24 Creating a spy with Spock
def "automatic deletion of hard disk when agents are here"() { given: "a camera feed" CameraFeed cameraFeed = new CameraFeed() and: "the auto-nuker program" SmartHardDriveNuker nuker = Spy(SmartHardDriveNuker) #A nuker.deleteHardDriveNow() >> {println "Hard disk is cleared"} #B when:"agents are knocking the door" cameraFeed.setCurrentFrame(ImageIO.read(getClass().getResourceAsStream( "agents.jpg"))) nuker.activate(cameraFeed); #C then: "all files of hard drive should be deleted" 1 * nuker.deleteHardDriveNow() #D}
#A Creates a spy for the SmartHardDriveNuker class
#B mocks the dangerous method. All other methods are real
#C real face recognition code runs.
#D Examines the mocked method
Here I create a Spy of my class under test. By default, after creation all methods are real and pass-through to the real object[122]. Then I specifically mock the method that deletes the hard drive. However the method that employs the face recognition logic is still the real one.
When the method activate() is called it runs its real code (so that I can pass it different images and test the effectiveness of the face recognition code). In the case of an image that represent a "threat" and so triggers the hard deletion process, I know that the mocked method will be called (and thus my hard drive is safe)
I show only one test in listing 8.2 but in reality I would need to write a parameterized test with multiple images that examine the behavior of the face recognition code.
8.3.3 The need for spies shows a problematic code base
The reason that spies are used for legacy code, is mainly because of the bad quality of legacy code[123]. Well designed code does not ever need spies in the first place. Here is the flow diagram of using spies that you should keep in your head at all times. The diagram is not specific to Spock. It applies to all testing frameworks (i.e. Mockito as well)
Figure 8.7 - Spies can always be replaced with Mocks in well designed code
In the example with the security utility, a spy was essential because the Java code did not use dependency injection. This is just one of the code smells of badly designed code. Java code that comes as a big ball of mud[124], breaks the SOLID principles[125], contains God[126] objects, and generally suffers from big design flaws is not directly testable with mocks/stubs and spies are needed.
In those cases, you should resist the temptation of writing a Spock with spies and instead refactor the code first before writing your unit tests. You will find that in most cases (if not all) spies are not really needed after the refactoring is complete.
8.3.4 Replacing spies with mock
I used spies with the security utility because I could not refactor the Java code first, as this would invalidate the digital signature of the binary. If that constraint didn't hold, I would instead modify the Java code to use dependency injection properly.
Listing 8.25 Refactoring Java code to avoid spies
public class SmartHardDriveNuker{ #A private final HardDriveNuker hardDriveNuker; #B public SmartHardDriveNuker(final HardDriveNuker hardDriveNuker) { this.hardDriveNuker = hardDriveNuker; #C } public void activate(CameraFeed cameraFeed) { [...code redacted for brevity..] hardDriveNuker.deleteHardDriveNow(); #D [...code redacted for brevity..] } }
#A No inheritance is used
#B Code reuse via composition
#C Gets hard drive nuker via constructor injection
#D Calls the dangerous method of the external dependency
Here I have refactored my Java code to use composition instead of inheritance. I have also introduced the "dangerous" hard drive deletion code as an external dependency. After this refactoring I can write again my unit test using a normal mock:
Listing 8.26 Using a mock instead of a spy
def "automatic deletion of hard disk when agents are here"() { given: "a camera feed and a fake nuker" CameraFeed cameraFeed = new CameraFeed() HardDriveNuker nuker = Mock(HardDriveNuker) #A and: "the auto-nuker program" SmartHardDriveNuker smartNuker = new SmartHardDriveNuker(nuker) #B when:"agents are knocking the door" cameraFeed.setCurrentFrame(ImageIO.read(getClass().getResourceAsStream( "agents.jpg"))) smartNuker.activate(cameraFeed); #C then: "all files of hard drive should be deleted" 1 * nuker.deleteHardDriveNow() #D}
#A Using a mock instead of a spy
#B Mock is passed as a dependency
#C Calls the mocked nuker class behind the scenes
#D Examines the interaction of the mock
If you have read chapter 6, the code in listing 8 should be very easy to understand. Because I have refactored the Java code and hard drive deletion is now an external dependency, I can simply mock that class and pass it the face recognition code. This way my class under test - SmartHardDriveNuker - is a real one, and a mock is used for the collaborator class - HardDriveNuker.
The end result is that no spies are used. What you need to take away from this section of the book, is that that despite Spock support for spies, you should avoid using them, and instead spend time to improve the design of your code so that spies are not needed.
And with that knowledge about Spies, we have concluded this book! You can now put it down and go write your own Spock tests!
8.4 Summary
· You can create Spock tests that will pass if a certain exception is thrown
· You can define explicitly the type of the exception and perform assertions on it refine the conditions for passing a test when an exception is thrown
· The @Issue annotation can be used for documentation (and possibly reporting) purposes on a Spock test. Use it to show which issue is verified by a Spock test.
· Spock supports the @Timeout annotation that will forcibly fail a test if it takes too long.
· You can ignore specific Spock tests. They will be skipped by the Spock runner.
· You can skip automatically Spock tests according to the running environment. Tests can be skipped under specific operating systems, JVM configuration, System properties, environment variables and any other custom code you can implement yourself.
· The @AutoCleanup annotation automatically releases resources at the end of a Spock test, even if the test has failed. This is an alternative to the cleanup: block and/or the cleanup()/cleanupSpec() methods
· All Spock block (and the when: block in particular) should be short and concise. Large code segments should be extracted into helper methods
· Helper methods for assertions need to use the assert keyword or the with() method (otherwise Spock cannot understand that the code is Groovy assertions)
· Helper methods for interactions should be wrapped in an interaction block (otherwise Spock cannot understand the special syntax used for interaction verification)
· Spies in Spock offer the possibility of partial mocking. A spy can mock some methods of an object while leaving the rest of the methods in the original implementation.
· Spies are an advanced feature that should be used mostly in legacy code that cannot be refactored. Spy usage inside Spock tests should be minimal.
· The presence of Spock spies usually shows a badly designed production code. Refactoring the code should make spies redundant
[110] Mockito also has a huge warning on the official documentation against the usage of spies
[111] https://www.atlassian.com/software/jira
[112] http://www.redmine.org/
[113] https://github.com/spockframework/spock/tree/master/spock-core/src/main/java/spock/util/environment
[114] You call also ignore exceptions during cleanup if you use the annotation like @AutoCleanup(quiet = true). However, this is a practice I do not endorse unless you really know what you are doing.
[115] Yes, I know that expecting exceptions does not happen via annotations. Thanks for catching it!
[116] https://github.com/kkapelon/java-testing-with-spock/tree/master/chapter8/src/main/java/com/manning/spock/chapter8/loan
[117] In this simple example, it is obvious that the contact details of the loan are the same as the customer ones. In a real world unit test this is not usually the case
[118] In this particular example, mocking the loan class is a bit of an overkill. I mock it for illustration purposes only so that I can show you helper methods with mocks.
[119] http://www.manning.com/koenig2/
[120] and why you should NOT use them!
[121] My example is a bit extreme for illustration purposes. Usually the code cannot be changed for political reasons.
[122] Creating a spy without mocking any method is the same as using the object itself. Not very exciting...
[123] a universal fact. Legacy code is always badly designed code
[124] https://en.wikipedia.org/wiki/Big_ball_of_mud
[125] https://en.wikipedia.org/wiki/SOLID_(object-oriented_design)
[126] https://en.wikipedia.org/wiki/God_object