What is Spock, and why you should care - GETTING STARTED - Java Testing with Spock, Version 6 (2015)

Java Testing with Spock, Version 6 (2015)

PART 1: GETTING STARTED

Chapter 1. What is Spock, and why you should care

In this chapter

· Introducing Spock

· Bird’s eye view of the testing process

· Using Groovy to test Java

· Spock’s place in the testing world

We live in the computer revolution! We’ve reached the point where computers are so common-place most if us carry a pocket-sized one all the time: our mobile phone. Mobile phones now can perform real-time face recognition, something that used to require a mainframe or computer cluster. At the same time, access to cheap and “always-on” Internet services has created a communication layer that surrounds us.

As we enjoy the benefits of computerized services in our daily lives, our expectations are also changing. We expect information to be always available. Errors and unexpected behavior in a favorite software service, leaves us frustrated. Electronic e-commerce is on the rise and all brands fight for customer loyalty as we turn to the internet for our shopping needs. Once I ordered a single chair from a well-known furniture company[1] and my credit card was charged three times the amount shown on the product page because of a computer error. Naturally I never bought anything else from that online shop again.

These high expectations of error-free software create even more pressure on developers if the “user” of the software is an organization, another company, or even a government agency. Software errors can now result in loss of time/money/brand loyalty and more importantly loss of trust in the software.

If you are a software developer of any level, you know that writing programming code is only half of software creation. Testing the programming code is also essential in order to verify its correctness. Software problems (more commonly known as bugs) have a detrimental effect on the reliability of an application. A continuous goal of software development is the detection of bugs before the software is shipped or deployed to production.

The impact of a bug/issue that reaches production code can have a profound effect depending of the type of software. For example, if your software is a mobile application for tracking daily intake of calories you can sleep easily each night knowing that any issues found by users will simply inconvenience them and in the worst case they will delete your application from their mobile phone (if they really get angry with the problems)

If for example you are writing software that manages hotel reservations, things are more serious. Critical issues will result in customer anger, brand damage for the hotel and probably future financial losses.

On the extreme end of the spectrum, what would be the severity of issues with:

· Software that controls hospital equipment?

· Software that runs on a nuclear reactor?

· Software that tracks enemy ballistic missiles and retaliates with its own defensive missiles? (my favorite example)

How will you sleep at night if you’re not sure these applications are thoroughly tested before reaching production status?

This book is about Spock, a comprehensive testing framework for Java (and Groovy) code that can help you automate the boring, repetitive and manual process of testing a software application. Spock is comprehensive because it can be seen as a union of existing Java testing libraries as shown in figure 1.1

Figure 1.1. Spock among existing Java testing tools.

As the figure shows, Spock is a superset of the defacto testing framework for Java: JUnit Spock also comes with built-in capabilities for features that normally require additional libraries. At its core Spock is a testing framework capable of handling the full life cycle of a software application.

MOCKING AND STUBBING

The most basic unit tests (named as logic tests by some) are those that focus on the logic of a single Java class. In order to test a single class in a controlled environment and isolate it from the other classes it depends on (collaborators) Spock comes with built-in support for "faking" external object communication. This capability is not inside vanilla JUnit and one needs external libraries (e.g. Mockito, JMock) to achieve this isolation of a Java class.

BEHAVIOR-DRIVEN DEVELOPMENT (BDD)

Spock also embraces the paradigm of Behavior Driven Development (BDD), a development process that attempts to unify implementation, testing and business people inside a software organization by introducing a central way of documenting requirements and validating software functionality against those requirements in a clear and repeatable manner.

Spock combines these different facets into a single convenient package offering a holistic approach to software testing.

1.1 What is Spock?

Spock was initially created in 2008 by Peter Niederwieser, a software engineer with GradleWare.[2] It was inspired by existing test tools (such as JMock and RSpec) and is currently used by several libraries within the open source community (such as Apache Tapestry) and several commercial companies (e.g. Netflix). A second Spock committer is Luke Daley (also with Gradleware) who is also the creator of the popular Geb[3] functional testing framework (demonstrated in chapter 8). Spock is a new entry in the test framework arena and currently challenges the undisputed king - JUnit- armed with a bunch of fresh ideas against the legacy techniques of the past. Spock tests are written in Groovy, but they can test either Groovy or Java code.

Spock is:

· Enterprise ready

Spock can be easily integrated with the popular build systems (Maven and Gradle), runs as part of a build process, and produces reports of automated test runs. Spock can be used to test backend code, web pages, http services, and more.

· Comprehensive

Spock is a one-stop shop when it comes to testing. It has built-in mocking and stubbing capabilities allowing you to decide on the breadth of the testing context. Spock can test a single class, a code module, or a whole application context with ease. You can perform end-to-end testing with Spock or isolate one class/method for your testing needs without any external libraries.

· Familiar/Compatible

Spock runs on top of the JUnit runner which already enjoys mature support among tools and development environments. You run your Spock tests in the same way as your JUnit tests. You can even mix the two in the same project, and get reports on test failures, or code coverage in a similar way to JUnit. Run your tests in parallel or in a serial way, Spock does not care because it is fully compatible with existing JUnit tools.

· Inspired

Spock is relatively new and does not carry any legacy burden. It is designed from scratch but at the same time it takes the best features of existing testing libraries (and tries to avoid their disadvantages). For example Spock embraces the given-when-then structure of JBehave[4] but also discards the cumbersome record/replay code of older Mocking frameworks.

SPOCK’S CODING FEATURES

· Concise

Spock uses the Groovy syntax which is already concise on its own, and mixes its own simplified syntax on top. No more tests that hide the substance with boilerplate code!

· Readable

Spock follows a close-to-English flow of statements which can be readable even by non-technical people (for example business analysts). Collaboration between analysis, development and testing people can be greatly simplified with Spock tests. If you always wanted to name your test methods using full English sentences, now you can!

· Meticulous

When things goes wrong Spock gives as much details as possible on the inner workings of the code at the time of failure. In some cases this is more than enough for a developer to understand the problem without resorting to the time consuming debugging process.

· Extensible

Spock allows you to write your own extensions to cater for your specific needs. In fact several of its “core” features are actually extensions (or started as extensions).

Here is a sample test in Spock that illustrates several of these key coding features. The example shows a billing system that emails electronic invoices to customers only if they have provided an email address.

How to use the code listings

You can find almost all code listings of this book in https://github.com/kkapelon/java-testing-with-spock. For brevity reasons the book sometimes will point to you the source code (especially for long Java listings). I use the Eclipse IDE in my day to day work as shown in the screenshots throughout the book. You can find specific instructions for installing Spock and using it via Maven, Gradle, Ecipse and Intellij in Appendix A.

Don't be alarmed by the unknown keywords at this point. Even if you know absolutely no Groovy at all, you should be able to understand the scenario in question by the presence of full English text sentences. We’ll explain all details of the syntax in the coming chapters. The full Chapter 2 is devoted to Groovy and how it differs from Java.

Listing 1.1 Sample Spock test

class InvoiceMailingSpec extends spock.lang.Specification{ #A def "electronic invoices to active email addresses"() { #B given: "an invoice, a customer, a mail server and a printer" PrinterService printerService = Mock(PrinterService) #C EmailService emailService = Mock(EmailService) #C Customer customer = new Customer() FinalInvoiceStep finalInvoiceStep = new FinalInvoiceStep(printerService, emailService) Invoice invoice = new Invoice() when: "customer is normal and has an email inbox" #D customer.hasEmail("acme@example.com") finalInvoiceStep.handleInvoice(invoice, customer) then: "invoice should not be printed. Only an email should be sent" 0 * printerService.printInvoice(invoice) #E 1 * emailService.sendInvoice(invoice,"acme@example.com") #E }}

#A the Spock Specification can be executed by a JUnit runner

#B Full English sentences that describe what the test does

#C Integrated mocking of collaborator classes

#D given-when-then declarative style of BDD

#E Verifying interactions of mocked objects

As you can see the Spock test has a clear given-when-then flow denoted with labels (the BDD style of tests) and each label comes fully documented with an English sentence. Apart from the def keyword and the * symbol in the last two statements almost all code is Java-like. Note that the spock.lang.Specification class is runnable by JUnit, meaning that this class can act as a JUnit test as far as build tools are concerned. We will cover these and several other features of Spock in the coming chapters.

Testing is a highly conflicting subject among software developers and several times the discussion is focused on testing tools and how many tests are needed in an application. There are always heated discussions on what needs to be tested in a large application and whether tests help with deadlines or not. Several developers (hopefully a minority) even think that all tests are a waste of time, or that their code does not need unit tests. If you think that testing is hard, or you believe that you don’t have enough time to write tests, this book will show you that Spock uses a concise and self-documenting syntax for writing test cases.

If on the other hand you have already embraced sound testing practices in your development process, I will show you how the Spock paradigm compares to established tools such as JUnit and TestNG.

Before getting in the details of using Spock, let’s explore why you need a test framework in the first place. After all, you already test your code manually as part of every coding session, when you make sure what you just coded actually does what you intended.

1.2 The need for a testing framework

The first level of testing comes from you. When you implement a new feature, you make a small code change and then run the application to see if the required functionality is ready. Compiling and running your code is a daily task that happens many times a day, as you progress towards the required functionality

Some features such as “add a button here that sorts this table of the report” are trivial enough that they can be implemented and tested in one run. But more complex features such as “we need to change the policy of approving/rejecting a loan” will need several changes and runs of the application until the feature is marked as complete.

You can see this manual code-run-verify cycle in figure 1.1

Figure 1.2 Testing software manually becomes more cumbersome as the application code base grows

Manual testing is enough for very small software projects. A quick prototype, a side project, a weekend coding session can be indeed tested manually by a single person. In order for the cycle to work effectively, a single loop must be quick enough for the developer to see the results of the code change. In the ideal case a single code change should be verified in seconds. If running the whole application and reaching the point where the new feature is found requires several minutes, developer productivity suffers.

Writing software is a creative process that requires getting into the “zone.” Having constant interruptions with lengthy intervals between each code change is a guaranteed way to disrupt the developer’s thinking about the code structure (not to mention loss of time/money while waiting for the test to finish).

As the programming code grows past a certain point this manual cycle gets lengthier with more time spent running and testing the application than writing code. Soon the run-verify time dominates the “developing” time. Another problem is the time it takes to redeploy software with the new changes. Small software projects can be deployed in seconds, but larger codebases (think bank software) may need several minutes for a complete deployment further slowing down the manual testing cycle.

SPOCK AS THE ENTERPRISE-READY TEST FRAMEWORK

Spock is marketed as an “Enterprise-ready” test framework, so it’s best to explain the need for automated testing in the context of enterprise software: software designed to solve the problems of a large business enterprise. Let’s look at an example that will reveal why a test framework is essential for large enterprise applications.

Imagine you’ve been hired as a software developer for a multinational company that sells sports equipment on an online shop. Most processes of the company depend on a monolithic system that handles all daily operations.

You are one of several developers responsible for this central application that has all the characteristics of typical enterprise in-house software:

· The code base is large (more than 200.000 lines of code)

· The development team is 5-20 people

· No developer knows all code parts of the application

· The application already runs in production for several years

· New features are constantly requested by project stakeholders

· There is code written by developers who have left the software department.

The last point is the one that bothers you most. There are several areas of the application with non-existent documentation, and no one to ask for advice.

1.2.1 Dealing with new requirements in an Enterprise Application

You are told by your boss that because the snow season is approaching all ski-related materials will get a 25% discount for a limited time period that must also be configurable. The time period might be a day, a week, or any other arbitrary time period.

Your approach:

· You implement the feature, and then

· check the functionality by logging manually into the eshop and verifying the ski products do indeed have the additional discount during checkout. You also

· change the date of the system to simulate a day after the offer has ended, and

· log again into the eshop and verify that the discount no longer applies.

You might now be happy with your implementation and send the code change to the production environment thinking you’ve covered all possible cases as shown in figure 1.3

Figure 1.3 Scenarios tested after a simple code change

ENTERPRISE COMPLEXITY: OF MODULES AND MEN

The next morning your boss frantically tells you to revert the change, because the company is losing money! He explains that the eshop has several VIP customers that always get a 10% percent discount on all products. This VIP discount can never be applied with other existing discounts: a VIP now gets a total discount of 35%, far below the profit margin of the company. You revert the change and take a note that in any subsequent change you have to remember testing for VIP customers as well.

This is a direct result of a large code base with several modules affecting more than one user-visible feature, or a single user-visible feature being affected by more than one code module. In a large enterprise project there are even modules that affect all user visible features (typical examples are core modules for security and persistence). This asymmetric relationship is illustrated in figure 1.4

Figure 1.4 A single change in one place has an unwanted effect in another place

With the change reverted, you learn more about the business requirements of discounts. The final discount of a product is affected by:

· types of customers (first time, normal, silver, VIP),

· 3 coupon code types (personal, seasonal, special),

· adhoc limited time offers,

· standard seasonal discounts,

· time of products in the warehouse, and of course the

· 30+ categories of sport equipment of the company.

The next time you tamper with the discount code module, you’ll have to manually test more than 100 cases of all the possible combinations. Testing all of them manually would require at least 4 hours of boring, repetitive work as shown in figure 1.5

Figure 1.5 Some scenarios were missed by manual testing

This enterprise example should make it clear that complexity of software makes the manual testing cycle very slow. Adding a new feature becomes a time-consuming process because each code change must be examined for side effects in all other cases as well.

Another issue similar to module interaction is the human factor: In a big application, communication between domain experts, developers, testers, system administrators etc. is not always free of misunderstandings and conflicting requirements. Extensive documentation, clear communication channels and an open policy regarding information availability can mitigate the problems but not completely eliminate them.

As an example, a sales manager in the eshop decides that he wants to see all tables in the back office application sorted by the value of the order, while at the same time an inventory manager wants to sort the same tables by order size. It is possible that two separate developers are tasked with both cases without knowing that the requirement is conflicting as shown in figure 1.6.

Figure 1.6 Similar features that affect the same code module can cause conflicts

This enterprise example illustrates first-hand, the problems of every large software code base:

· Testing manually every possible combination of data input after a code change is very difficult and even impossible in some cases.

· It is hard to predict which parts of the application will be affected by a single code change. Developers are afraid to change existing code fearing they break existing functionality

· Code changes for a new feature can re-surface previous bugs that have already been fixed. (regressions)

· It is not easy to understand all system requirements from the existing code. Reading the code only provides knowledge on what happens but not why it happens.

· Redeploying the application just to see the effects of a code change is a lengthy process on its own the slows down development time even further.

Now you know the major problems faced by the software development team in a big enterprise project. Next, let’s look at different approaches to tackle these problems.

1.2.2 Common ways to handle Enterprise complexity

All software companies suffer from these problems and deal with them in one of these three ways or their variations (I have seen them all in real life!)

· Developers test manually everything after each code change

· Big code changes are avoided for fear of unforeseen bugs

· A layered testing approach is introduced that includes automated testing.

Let’s look at each of these solutions in turn.

MINDLESS MANUAL TESTING

In the first case (that is only possible with small-middle sized software projects) developers are not entirely sure what is broken and what is not after a code change. Therefore they test manually all parts of the application after they implement a new feature or fix an issue. This approach wastes a lot of time/money because developers suffer from the repetitive nature of testing (which is a natural candidate for automation).

In addition, as the project grows it becomes much more difficult to test everything by hand. Either the development progress comes into a crawl, as most developers deal with testing instead of adding new features, or (the most usual case) developers add features testing only parts of the application they think that might be affected. The result is that bugs enter production code and developers become firefighters where each passing day is a big crisis as the customer discovers missing functionality.

AVOIDING BIG CODE CHANGES

In the second case the “solution” for the problem is to never perform big code changes at all. This paradigm is often embraced by very big organizations with big chunks of legacy code (for example banks). Management realizes that new code changes may introduce bugs which are unacceptable. On the other hand manual testing of the code is next to impossible because of the depth and breadth of all user scenarios (for example you cannot possible test all systems of a bank in a logical time frame by hand).

The whole codebase is declared “sacred”. Changing or re-writing code is strictly forbidden by upper management. Developers are only allowed to add small features in the existing infrastructure without touching the existing code. There are local gurus that inspect each code change extensively before entering production status. Code re-use is not possible. A lot of code duplication is present because each new feature cannot modify existing code. Either you have already what you need to implement your feature or you are out of luck and need to implement it from scratch.

If you are a developer working in situations that belong to these first two cases (manual testing and the big codebase that nobody touches) I really feel for you! I have been there myself..

DELEGATING TO AN AUTOMATED TESTING FRAMEWORK

Of course there is a third possible approach and that is the one you should strive for.

In the third case an automated test framework is in place that runs after every code change. The framework is tireless, meticulous and precise. It runs in the background (or on demand) and checks several user features whenever a change takes place. In a well managed software creation process, the testing framework runs automatically after every developer commit as part of a build pipeline (e.g. with the Jenkins[5] build server). Results from this automatic run can influence further steps. For example, a common policy is that code modules with failed test results should never be deployed to a production environment.

The test framework acts as an early warning system against unwanted code effects. To illustrate the previous example, if you had a test framework in place you would get an automated report after any change as shown in figure 1.7

Figure 1.7 Detecting unwanted changes with a test framework

A test framework:

Reduces

· feedback time needed to verify effects of code changes

· boring repetitive tasks

Ensures

· confidence when a new feature is implemented, a bug is fixed or code is refactored

· detecting conflicting requirements

Provides

· documentation for code and explanation of reasons behind the current state

Code can be refactored, removed and updated with ease since the test framework continuously reports unwanted side effects. Developers are free to devote most of their time in coding new features and fixing existing (known) bugs. Features quickly come into production code and the customer receives a software package known to be stable and solid for all scenarios supported by the test framework. There is some time initial investment for the testing framework, but once it is in place the gains outperform the time it takes to write the test scripts. Catching code regressions and severe bugs before they enter the production environments is much cheaper than allowing them to reach the final users.

A test framework has also other benefits not instantly visible with regards to code quality. The process of making programming code testable enforces several constraints on encapsulation and extensibility that can be easily neglected if the code is not created with tests in mind. Techniques for making your code testable will be covered in chapter 7. But the most important benefit of a test framework is the high developer confidence when performing a deep code change.

Let’s dig in to how Spock as a testing framework specializing in enterprise applications can help you refactor code with such confidence.

1.3 Spock: the groovier testing framework

When I first came upon Spock, I thought that it would be the jUnit alternative for the Groovy programming language. After all, once a programming language reaches a critical mass, somebody ports the standard testing model (known as xUnit) to the respective runtime environment. There are already xUnit frameworks for all popular programming languages.

But Spock is not the xUnit of Groovy! It resembles higher-level testing frameworks (such as Rspec and Cucumber) that follow the concepts of Behavioral-Driven development, instead of the basic setup-stimulate-assert style of xUnit. Behavioral Driven Development attempts (among other things) to create a one-to-one mapping between business requirements and unit tests.

ASSERT VS ASSERTIONS

If you are familiar with JUnit, one of the first things you will notice with Spock is the complete lack of assert statements. Asserts are used in unit tests in order to verify the test. You define what is the expected result yourself and JUnit automatically fails the test if the expected output does not match the actual one. Assert statements are still there if you need them, but the preferred way is to use Spock “assertions” instead, a feature so powerful it has been in fact backported to Groovy itself. You will learn more about “Power asserts” in chapter 2 and how they can help you pinpoint the causes of a failing test.

AGNOSTIC TESTING OF JAVA AND GROOVY

Another unique advantage of Spock is the ability to test agnostically both Java and Groovy code as shown in figure 1.8

Figure 1.8 Spock can test both Java and Groovy code.

Groovy is a dynamic language that runs in the same Java Virtual Machine (JVM) as Java. Java supporters are very proud of the JVM and some believe that the value of the JVM as a runtime environment is even higher than Java the language. Spock is one example of the power the JVM has to accommodate code from different programming languages.

Spock can test any class that runs on the JVM, regardless of the original source code (Java or Groovy). It’s possible with Spock to test either a Java class or a Groovy class in the exact same way. Spock doesn’t care about the origin of the class, as long as it is JVM compatible. You can even verify both Java and Groovy code in the same Spock test if your project is a mix of the two

TAKING ADVANTAGE OF GROOVY TRICKS IN SPOCK TESTS

Finally, you need to know that Groovy is a dynamic language that behaves differently than Java in some important aspects (such as the declaration of variables), as you will learn in chapter 2. This means that several “tricks” you learn with Spock are in reality a mix of both Groovy and Spock magic, because Spock can extend Groovy syntax in ways that would be very difficult with Java (if not impossible). And yes, unlike Java, in Groovy it is possible for a library/framework to change the syntax of the code as well. Spock is one such library as you will learn in chapter 4.

As you become more familiar with Spock and Groovy the magic behind the curtain will start to appear and you might even be tempted to use Groovy outside of Spock tests as well!

AST transformations - changing the structure of Groovy language

Several tricks of Groovy magic come from the powerful meta-programming facilities offered during runtime that can change classes and methods in ways impossible with vanilla Java. At the same time Groovy also supports compile time macros (Abstract Syntax Tree transformations in Groovy parlance).

If you’re already familiar with macros in other programming languages you should already be aware of the power they bring in code transformations. Using AST transformations a programmer can add/change several syntactic features of Groovy code modifying the syntax in forms that were difficult/ impossible in Java.

Spock takes advantage of these compile and runtime code transformation features offered by Groovy in order to create a pseudo-DSL (domain specific language) specifically for unit tests. All the gory details on Spock syntax will be shown in chapter 4.

Before starting with the details of Spock code, let's take a bird's eye view on its major features.

1.3.1 Enterprise testing

A test framework geared towards big enterprise application has certain requirements in order to handle the complexity and possible configurations that come with enterprise software. Such a test framework must easily adapt to the existing ecosystem of build tools, coverage metrics, quality dashboard and other automation facilities.

Rather than re-inventing the wheel, Spock bases its tests on the existing JUnit runner. The runner is responsible for executing JUnit tests and presenting their results to the console or other tools (e.g. the IDE). Spock reuses the JUnit runner to get for free all the mature support of external tools already created by JUnit.

· Do you want to see code coverage reports with Spock?

· Do you want to run your tests in parallel?

· Do you want to split your tests in long running and short running?

The answer to all these questions is "yes you can, as you did before with JUnit".

1.3.2 Data-driven tests

A common target for unit tests is to handle input data for the system in development. It is impossible to know all possible uses for your application in advance, let alone the ways people are going to use and misuse your application.

Usually a number of unit tests is dedicated to possible inputs of the system in a gradual way. The test starts with the known set of allowed or disallowed input and as bugs are encountered, it is enriched with more cases. Common examples would be a test that checks if a username is valid or not, or what date formats are accepted in a web service.

These tests suffer from a lot of code duplication if code is handled carelessly. The test is always the same (is the username valid or not) and only the input changes. While JUnit has some facilities for this type of tests (parameterized tests), Spock takes a different turn allowing you to actually embed data tables in Groovy source code. Spock offers a special DSL (Domain Specific Language) that allows tabular data in the unit test source code. Data driven tests will be covered in Chapter 5

1.3.3 Mocking and Stubbing

For all its strengths Object Oriented Software suffers from an important flow. The fact that two objects work correctly individually does not imply that both objects will also work correctly when connected to each other. The reverse is also true, side effects from an object chain may hide or mask problems that happen in individual class.

A direct result from these flow is that testing OOP software usually needs to cover two level at once. The integration level where tests examine the system as a whole (integration tests) and the class level where tests examine each individual class (unit tests or logic tests).

In order to examine the microscopic level of a single class and isolate it out of the macroscopic level of the system a controlled running environment is needed. A developer has to focus on a single class and the rest of the system is to be assumed "correct". Attempting to test a single class inside the real system is very difficult because any bugs encountered are not immediately clear if they happen because of the class under test or the environment.

To this purpose a mocking framework is needed that "fakes" the rest of the system and leaves only the class under test to be "real". The class is then tested in isolation because even though it "thinks" that it is inside a real system, in reality all other collaborating classes (collaborators) are simple puppets with pre-programmed input and output[6].

In the JUnit world an external library is needed for mocking. There are numerous with both strengths and weaknesses at the same time (Mockito, jMock, Easymock, Powermock). Spock comes with its own mocking framework built-in. Combined with the power of Groovy meta-programming as will be shown in Chapter 2 Spock is a comprehensive Domain Specific Language that provides all puzzle pieces needed for testing.

Now that we covered the theory and you know the foundations of a solid testing process and how Spock can test classes written in Java it is time to delve into code!

1.4 A first look at Spock in action

The following examples should whet your appetite, so don't stress yourself on the strange syntax or any unknown keywords. I will cover all Spock syntax in the rest of the book.

1.4.1 A very simple test with JUnit

When introducing a new library/language/framework everybody expects a “hello world” example. In this section I will show what Spock looks like in a minimal, but fully functional example.

Here is the Java class we will test. For comparison purposes a possible JUnit test is first shown, as JUnit is the defacto testing framework for Java, still undisputed after more than a decade.

Listing 1.2 Java class under test and JUnit test

public class Adder { #A public int add(int a, int b) { return a+b; }}public class AdderTest { #B @Test public void simpleTest() { Adder adder = new Adder(); #C assertEquals("1 + 1 is 2", 2 ,adder.add(1, 1)); #D } @Test public void orderTest() #E { Adder adder = new Adder(); assertEquals("Order does not matter ",5,adder.add(2, 3)); #F assertEquals("Order does not matter ",5,adder.add(3, 2)); }}

#A A trivial class that will be tested (a.k.a. class under test)

#B Test case for the class in question

#C Initialization of class under test

#D JUnit assert statement that compares 2 and the result of add(1,1)

#E A second scenario for the test under class

#F Two assert statements that compare 5 with adding 2 and 3

We introduce two test methods, one that tests the core functionality of our Adder class, and one that tests the order of arguments in our “add” method.

Running this JUnit test in the Eclipse Development Environment (right click on the .java file and choose run as-> Junit test from the menu) gives the following:

Figure 1.9 Running a JUnit test in Eclipse

1.4.2 A very simple test with Spock

And here is the same test in Groovy/Spock. Again this test examines the correctness of the Java class Adder that creates the sum of two numbers.

Listing 1.3 Spock test for the Adder Java class

class AdderSpec extends spock.lang.Specification{ #A def "Adding two numbers to return the sum"() { #B when: "a new Adder class is created" #C def adder = new Adder(); #D then: "1 plus 1 is 2" #E adder.add(1, 1) == 2 #F } def "Order of numbers does not matter"() { #G when: "a new Adder class is created" def adder = new Adder(); then: "2 plus 3 is 5" adder.add(2, 3) == 5 #H and: "3 plus 2 is also 5" #I adder.add(3, 2) == 5 }}

#A All Spock tests extends the Specification class

#B A Groovy method with a human readable name that contains a test scenario

#C A "when" block that sets the scene

#D Initialization of Java class under test

#E A "then" block that will hold verification code

#F A Groovy assert statement

#G Another test scenario

#I A "and" block that accompanies the "then" block

If you have never seen Groovy code before, this Spock segment will indeed appear very strange to you. The code has mixed lines of things you know (like the first line with the extends keyword) and things completely alien to you like the def keyword. Details on Groovy syntax will be shown in Chapter 2.

On the other hand if you’ve already seen the concepts of Behavior-driven design (BDD) you’ll already be familiar with the “when/then” pattern of feature testing.

We will explain the Spock syntax in detail in the coming chapters. For example the def keyword (comes from "define") is how you declare things in Groovy without specifying explicitly their type (which is a strict requirement in Java). The Spock blocks (when:, then:, and:) will be covered in chapter 4.

So how do you run this test? You run it in the same way as a JUnit test! Again you right click on the Groovy class and select "Run as -> JUnit" test from the popup menu. The result in Eclipse looks like this:

Figure 1.10 Running a Spock test in Eclipse

Other than the most descriptive method names, there isn’t any big difference between JUnit and Spock results with this trivial example. While I use Eclipse here, Spock tests can run on all environments/tools that already support JUnit tests (e.g. Intellij)

TAKEAWAYS FROM THESE CODE EXAMPLES

What you need take away from this code sample is:

· The almost English-like flow of the code. It is very easy to see what is being tested even if you are a business analyst or don’t know Groovy.

· The lack of any assert statements. Spock has a declarative syntax, where you explain what you consider correct behavior.

· The fact that Spock tests can be run like JUnit tests.

Let's move on to one of the killer features of Spock (handling failed tests).

1.4.3 Inspecting failed tests with Spock

One of the big highlights of Spock code is the lack of assert statements compared to JUnit. In the previous section you saw what happens when all tests pass and the happy green bar is shown in Eclipse. But how does Spock cope with test failures?

In order to demonstrate the advantage over JUnit we will add another (trivial) Java class that we wish to test:

public class Multiplier { public int multiply(int a, int b) { return a * b; }}

For this class we will also write the respective JUnit test. But as an additional twist (for demonstration purposes) we wish to test this class not only by itself, but also in relation with the Adder class shown in the previous section.

Listing 1.4 A JUnit test for two Java classes

public class MultiplierTest { @Test public void simpleMultiplicationTest() { Multiplier multi = new Multiplier(); assertEquals("3 times 7 is 21",21,multi.multiply(3, 7)); } @Test public void combinedOperationsTest() #A { Adder adder = new Adder(); #B Multiplier multi = new Multiplier(); #C assertEquals("4 times (2 plus 3) is 20", #D 20,multi.multiply(4, adder.add(2, 3))); assertEquals("(2 plus 3) times 4 is also 20", 20,multi.multiply(adder.add(2, 3),4)); }}

#A A test scenario that will examine two Java classes at the same time

#B Creation of the first Java class

#C Creation of the second Java class

#D Verification of a mathematical result coming from both Java classes

Running this unit test will result in a green bar since both tests pass.

And now for the equivalent Spock test:

Listing 1.5 - Spock test for two Java classes

class MultiplierSpec extends spock.lang.Specification{ def "Multiply two numbers and return the result"() { when: "a new Multiplier class is created" def multi = new Multiplier(); then: "3 times 7 is 21" multi.multiply(3, 7) == 21 } def "Combine both multiplication and addition"() { #A when: "a new Multiplier and Adder classes are created" def adder = new Adder(); #B def multi = new Multiplier() #C then: "4 times (2 plus 3) is 20" #D multi.multiply(4, adder.add(2, 3)) == 20 and: "(2 plus 3) times 4 is also 20" multi.multiply(adder.add(2, 3),4) == 20 }}

#A A test scenario that will examine two Java classes at the same time

#B Creation of the first Java class

#C Creation of the second Java class

#D Verification of a mathematical result coming from both Java classes

Again running this test will pass with flying colors. You might start to believe that we gain nothing from using Spock instead of JUnit. But wait!

Let’s introduce an artificial bug in our code to see how both JUnit and Spock deal with failure. To mimic a real world bug, we’ll introduce it in the Multiplier class, but only for a special case.

Listing 1.6 Introducing an artificial bug in the Java class under test

public class Multiplier { public int multiply(int a, int b) { if(a == 4) #A { return 5 * b; //multiply an extra time. } return a *b; }}

#A A dummy bug that happens only if the first argument is 4

Now let’s run the all JUnit test and see what happens.

Figure 1.11 Failure of JUnit test in Eclipse

We have a test failure. Notice however anything strange here? Because the bug we introduced is very subtle, JUnit essentially says to us:

· Addition by itself works fine.

· Multiplication by itself works fine.

· When both of them run together we have problem.

But where is the problem? Is the bug on the addition code or the multiplication one? We cannot say just by looking at the test result (ok, ok the math here might give you a hint in this trivial example).

We need to insert a debugger in the unit test to find out what happened. This is an extra step that takes a lot of time because recreating the same context environment can be a lengthy process.

SPOCK KNOWS ALL THE DETAILS WHEN A TEST FAILS

Spock comes to the rescue! If we run the same bug against Spock we get the following:

Figure 1.12 Failure of a Spock test in Eclipse

Spock comes with a super-charged error message that not only says we have a failure, but also calculates intermediate results!

As you see it is clear by the test that the addition works correctly (2 + 3 is indeed 5) and the bug is on the multiplication code ( 4 times 5 does not equal 25)

Armed with this knowledge we can go directly to the Multiplier code and find the bug. This is one of the killer features of Spock, and may be enought to entice you to rewrite all your JUnit tests in Spock. But a complete rewrite is not necessary, as both Spock and JUnit tests can co-exist in the same codebase, which we’ll explore next.

1.5 Spock’s position in the Java ecosystem

The de-facto testing framework in a Java project is Junit, but TestNG is another testing framework for Java that is very similar to JUnit. At some point in time TestNG had several extra features that JUnit lacks, resulting in a lot of developers switching over to TestNG (especially for big enterprise projects). JUnit quickly closed the gap and TestNG failed to gain a majority in the mindset of Java developers. The throne of JUnit is still undisputed. I have personally seen junior Java developers who think that JUnit and unit testing is the exact same thing, but in reality JUnit is one of the many ways that unit tests can be implemented.

Both JUnit and TestNG are written in Java as well. Traditionally this has been seen as an advantage by Java developers, because they use the same programming language in both production code and testing code. Java is a verbose language (at least by today’s standards) with a lot of boilerplate code, several constraints (for example all code must be part of a class, even static methods) and a heavy syntax where everything has to be explicitly defined. Newer editions of Java (after version 7) attempt to rectify this issue with mixed success, never reaching the newer “convention-over-configuration” paradigm of other programming languages.

It doesn’t have to be this way though. There is no technical reason to constrain unit tests, so they are in the same programming language as the development code. In fact production and testing code have completely different requirements. The biggest difference is that testing code runs by definition before the application is deployed in production. A good engineer will use the best tool for the job. You can think of Spock as a special domain language created exclusively for testing purposes.

Compilation and running of unit tests is a common task for the developer or the build server inside a software company. This means that runtime and compile time errors in unit tests are detected at the same time. Java goes to great lengths to detect several errors during compile time instead of run time. This effort is wasted in unit tests because these two phases usually run one after the other during the software development lifecycle. The developer still pays the price for the verbosity of Java even for unit tests. There must be a better way.

Groovy comes to the rescue!

MAKING SPOCK GROOVY

Groovy is a dynamic programming language (similar to Python or Ruby) which means it gives the programmer power to defer several checks until runtime. This might seem like a disadvantage, but this feature is exactly what unit tests should exploit. Groovy also has a much nicer syntax than Java, because several programming aspects have carefully selected defaults if you don’t explicitly define them (convention over configuration).

As an example, if you omit the visibility modifier of a class in Java, the class is automatically package private, which ironically is the least-used modifier in Java code. Groovy does the logical thing, if you omit the visibility modifier of a class, the class is assumed to be public, which is what you want most times.

The times where I had to create JUnit tests with package private visibility in my programming career: zero! So all these years, I’ve “paid” the price of declaring all my unit tests (and I guess you have, as well) as public, without ever thinking “there must be a better way!” Groovy has embraced the convention over configuration concept and this paradigm is evident in Spock code as well.

Testing Groovy code with JUnit

The topic of this book is how to test Java code with the Spock framework (which is written in Groovy). The reverse is also possible with JUnit.

• You can write a normal JUnit test in Java, where the class under test is actually implemented in Groovy.

• You can also write the JUnit test in Groovy to test Groovy or Java code.

• Finally Groovy supports a GroovyTestCase class, which itself extends the standard TestCase from JUnit.

Because this is a book about Spock I will not cover these combinations here. See the book "Making Java Groovy" by Manning if you are interested on any of these cases.

With Spock you can gain the best of both worlds. You can keep the tried-and-true Java code in your core modules, and at the same time, you gain the developer productivity of Groovy in the testing code without sacrificing anything in return.

Production code is written with verbose and failsafe Java code, while unit tests are written in the friendlier and lighter Groovy syntax that cuts down on unneeded modifiers and provides a much more compact code footprint.

And the best part is that you keep your existing JUnit tests!

1.5.1 Adding Spock tests to existing projects that have JUnit tests

Every new technology faces a very big obstacle in its path for adoption: resistance to change. Tradition, inertia, and the projected cost of switching to another technology instead of the mature existing solution are always major factors that affect any proposal for improvement when a better solution comes along.

As an example, Gradle is a build system, also written in Groovy, which is in many ways more flexible than the defacto build system of Java (Maven). It is unrealistic to use two build systems in a big enterprise project. Gradle has to face the entrenched Maven supporters and to convince them that the switch offers compelling advantages.

Spock doesn’t suffer from this problem. You can integrate Spock today in your Java project without rewriting or removing a single line of code/configuration. This is a huge win for Spock because it allows a gradual adoption, where both old JUnit tests and newer Spock tests can co-exist peacefully. It is perfectly possible to implement a gradual Spock adoption strategy in your organization by implementing new tests in Spock during a trial period without losing anything if you decide to keep implementing JUnit tests as well.

The standard Maven directory structure already supports unit tests in Groovy in the src/test/groovy folder. All your Spock tests can go into this directory without affecting your existing JUnit tests located in src/test/java (or other directories), as shown in the next figure.

Figure 1.13 Spock tests in a Maven project

For more details on how to setup your individual development environment (IDE) for Spock testing see Appendix A.

With the Spock tests in place, the next question you might have is how to run them. You will be happy to know that Spock comes with a test runner called Sputnik (from Spock and JUnit) that runs on top of the existing JUnit runner thus keeping full backwards compatibility.

You can run any Spock test as you run any JUnit test:

· From your development environment

· From the command line

· From Maven/Gradle or any other build system that supports JUnit tests

· From an automated script or build server environment as explained in Chapter 8.

The Spock Web console

You can also run Spock tests without installing anything at all, with the Spock Web Console. If you visit https://meetspock.appspot.com/ , you can play with the Spock syntax and get a feel of how easy it is to write Spock tests using only your browser.

The Spock Web console is based on the excellent Groovy Web Console found at https://groovyconsole.appspot.com/ , that offers a Groovy playground on the web ready for you to explore from the comfort of the web browser.

1.5.2 Spock adoption paths in a Java project

Because Spock is compatible with JUnit runners, it can be introduced gradually in an existing Java codebase. Assuming you start with a 100% Java project as shown at the far left in the top row of figure 1.13, Spock can run alongside JUnit tests in the same codebase.

Of course, if you wish, it is perfectly possible to re-write all tests in Spock if that is what you want. Spock can work as a superset of JUnit as you will see in Chapter 3. The situation is shown in the third scenario, shown at the far right in the top row of figure 1.14

Figure 1.14 Spock used as a stepping stone into Groovy migration

As the figure shows, at this point if you really like Spock, the next possible evolution is to try replacing Maven (or even Ant) with Gradle now that you become more familiar with the Groovy syntax. Again this is a valid state for a Java project, commonly used for new or existing projects in organizations that find Maven too restrictive.

In the last two cases shown in the figure, after having seen how Groovy works in non-production code (tests and build scripts) you are ready to use Groovy either along Java as equal or as a scripting language inside the Java code.

New projects can even start with the full Groovy stack as shown in the last scenario. For this book I assume that you have no prior experience with Groovy, so most of the examples will be about situations 2 and 3 as shown in the figure.

1.6 Comparing Spock and JUnit

It is hard to compare JUnit and Spock in a single section, because both tools have a different philosophy when it comes to testing. JUnit is a Spartan library that provides the absolutely necessary thing you need to test and leaves additional functionality (such as mocking and stubbing) to external libraries.

Spock has a holistic approach, providing you a superset of the capabilities of JUnit, while at the same time reusing its mature integration with tools and developments environments. Spock can do everything that JUnit does and more, keeping backwards compatibility as far as test runners are concerned.

What follows is a brief tour of some Spock highlights. In the coming chapters several times I will compare similar functionality between Spock and JUnit (where applicable). If you are not familiar with JUnit, feel free to skip the comparisons and just follow the Spock examples.

1.6.1 Writing concise code with Groovy syntax

Spock is written in Groovy which is less verbose than Java. This means that Spock tests are more concise than the respective JUnit tests. Of course this advantage is not specific to Spock itself. Any other Groovy testing framework would probably share this trait. But at the moment only Spock exists in the Groovy world.

Here is the advantage in a visual way, shown in figure 1.15.

Figure 1.15 Amount of code in an application with JUnit and Spock tests

Less code is easier to read, easier to debug, and easier to maintain in the long run. I will go intor detail on how Groovy supports less verbose code than Java in Chapter 3.

1.6.2 Mocking and Stubbing with no external library

JUnit does not support Mocking and Stubbing on its own. There are several Java framework that fill this position. The main reason that I got interested in Spock in the first place is the fact that it comes full batteries included as mocking and stubbing are supported out of the box.

Figure 1.16 Spock is a superset of JUnit

I’ll let this example explain:

David goes into a software company and starts working on an existing Java code base. He’s already familiar with JUnit (defacto testing framework for Java). While working on the project, he needs to write some unit tests that need to run in specific order. JUnit does not support this. So David also includes TestNG in the project.

Later he realizes that he needs to use mocking for some very special features of the software (for example the credit card billing module). He spends some time to research all the available Java libraries (there are many). He chooses Mockito, and integrates it in the code base as well.

Months pass and David learns all about Behavior-Driven Development in his local Dev Meeting. He gets excited! Again he researches the tools and selects JBehave for his project in order to accomplish BDD.

Meanwhile Jane is a junior developer that knows only vanilla Java. She joins the same company and gets overwhelmed the first day because she has to learn 3-4 separate tools just to understand all the testing code.

In an alternate universe David starts working with Spock as soon as he joins the company. Spock has everything he needs for all testing aspects of the application. He never needs to add another library or spend time researching stuff as the project grows.

Jane joins the same company in this alternate universe. She asks David for hints on the testing code and he just replies “Learn Spock and you will understand all testing code”. Jane is happy because she has to focus on a single library instead of three.

You can learn more about stubbing/mocking/spying in chapter 6. A brief introduction of BDD is also offered in chapter 7. Even though Spock does not offer a full featured BDD workflow (as JBehave), it still offers the capability to write tests understandable by business analysts as shown in the next section.

1.6.3 Using English sentences in Spock tests and reports

Here is a bad JUnit test (I see these all the time). It contains cryptic method names that do not describe what is being tested.

Listing 1.7 - A JUnit test where method names are unrelated to business value

public class ClientTest { @Test public void scenario1() #A { CreditCardBilling billing = new CreditCardBilling(); Client client client = new Client(); billing.chargeClient(client,150); assertTrue("expect bonus",client.hasBonus()); #B } @Test public void scenario2() #A { CreditCardBilling billing = new CreditCardBilling(); Client client client = new Client(); billing.chargeClient(client,150); client.rejectsCharge(); assertFalse("expect no bonus",client.hasBonus()); }

#A A test method with a generic name

#B Non technical people cannot understand test

This code is only understandable by programmers. Also if the second test breaks, a project manager (PM) will see the report and know that “scenario2” is broken. This report has no value for the PM since he does not know what scenario2 does exactly without looking at the code.

Spock supports an English like flow. Compare the same example in Spock:

Listing 1.8 - A Spock test where methods explain the business requirements

class BetterSpec extends spock.lang.Specification{ def "Client should have a bonus if he spends more than 100 dollars"() { when: "a client buys something with value at least 100" #A def client = new Client(); def billing = new CreditCardBilling(); billing.chargeClient(client,150); then: "Client should have the bonus option active" #B client.hasBonus() == true } def "Client loses bonus if he does not accept the transaction"() { when: "a client buys something and later changes mind" #A def client = new Client(); def billing = new CreditCardBilling(); billing.chargeClient(client,150); client.rejectsCharge(); then: "Client should have the bonus option inactive" #B client.hasBonus() == false }}

#A Business description of test

#B human readable test result

Even if you are not a programmer. you can read just the English text in the code (sentences inside quotes) and get the following:

· Client should have a bonus if he spends more than 100 dollars

· when a client buys something with value at least 100

. then Client should have the bonus option active

· Client loses bonus if he does not accept the transaction

. when a client buys something and later changes mind

. then Client should have the bonus option inactive

This is very readable. A business analyst could read the test, and ask questions for other cases (what happens if the client spends 99.9? What happens if he changes his mind the next day and not immediately?)

Also if the second test breaks, the PM will see in the report a red bar with title “Client loses bonus if he does not accept the transaction.” He instantly knows the severity of the problem (perhaps he decides to ship this version if he considers it non-critical)

For more information on Spock reporting and how Spock can be used as part of an Enterprise delivery process see chapter 8

1.7 Summary

Spock is an alternative Test Framework written in the Groovy programming language

A test framework automates the boring and repetitive process of manual testing which is essential for any large application codebase

· Although Spock is written in Groovy, it can test both Java and Groovy code

· Spock has built-in support for Mocking and Stubbing without an external library

· Spock follows the given-when-then code flow commonly associated with the Behavioral Driven Development paradigm

· Both Groovy and Java build and run on the JVM. A large enterprise build can run both JUnit and Spock tests in the same time.

· Spock uses the JUnit runner infrastructure and therefore is compatible with all existing Java infrastructure. For example, code coverage with Spock is possible in the same way as JUnit.

· One for the killer features of Spock is the detail it gives when a test fails. JUnit only mentions the expected and actual value, where Spock records the surrounding running environment mentioning the intermediate results and allowing the developer to pinpoint the problem with greater ease than JUnit

· Spock can pave the way for full Groovy migration into a Java project if that is what you wish. Otherwise it is perfectly possible to keep your existing JUnit tests in place and only use Spock in new code

· Spock test have the ability to include full English sentences in their code structures allowing for easy documentation.

[1] the name of the furniture company starts with I and ends in A

[2] The same company behind Gradle - a build system in Groovy (replacement for Maven)

[3] http://www.gebish.org/

[4] http://jbehave.org/

[5] http://jenkins-ci.org/

[6] I always enjoyed this evil aspect of testing. My own puppet theater where the protagonist can't see behind the scenes.