Testing with F# (2015)
Chapter 3. Setting Up Your Test Environment
Before getting started with writing unit tests, we need to make sure we have an environment good enough for running them. There are some tools built into Visual Studio we will use for development: tools to download and tools to buy.
This chapter will be a step-to-step guide on how to set up your test environment by covering the following subjects:
· F# Interactive
· Testing in Visual Studio
· Testing outside Visual Studio
· Setting up a build script with FAKE (F# Make)
· Running tests in continuous integration
After reading this chapter, you will be ready to start writing tests.
F# Interactive
One of the coolest things about F# is that its development environment comes with an interactive test environment: F# Interactive. You can access this feature in Visual Studio's top menu by navigating to View | Other Windows | F# Interactive or pressing the default shortcut Ctrl + Alt + F, as shown in the following screenshot:
This is valuable when we need to do the following:
· Quickly validate computations
· Spike new concepts
· Validate the implemented solution
When working with F#, you can use the interactive window to quickly validate part of a solution before implementing it. It provides a kind of a workbench when developing a test environment.
Let's say you need to implement a function that checks whether the current year is a leap year. You can enter the following code into F# Interactive in order to understand how the .NET framework manages to create a date that is not valid:
> new System.DateTime(System.DateTime.Today.Year, 2, 29);;
System.ArgumentOutOfRangeException: Year, Month, and Day parameters describe an un-representable DateTime.
at System.DateTime.DateToTicks(Int32 year, Int32 month, Int32 day)
at <StartupCode$FSI_0004>.$FSI_0004.main@()
Stopped due to error
Working with F# Interactive, we end each statement with ;; in order to tell the interpreter that it should validate. That way we can enter several lines of code without having it run until we send ;; to F# Interactive.
This tells us that we can expect an ArgumentOutOfRangeException exception when we enter a date that is not valid. So, let's use this in our routine to find leap years:
open System
// is the year a leap year?
let isLeapYear year =
try
let date = DateTime(year, 2, 29)
true
with
| :? ArgumentOutOfRangeException -> false
Now, we can use F# Interactive to validate this function. We will do this by writing the code in new F# code file. Select the code and press Alt + Enter or right-click and select Execute in Interactive.
You will get the following output in the F# Interactive window:
val isLeapYear : year : int −> bool
Now, you can validate that the code is doing what you as a programmer expected it to do by interactively testing it from the interactive prompt:
> [2000..2014] |> List.filter isLeapYear ;;
val it : int list = [2000; 2004; 2008; 2012]
In most other languages, you can't test the written code before it has been compiled into a runnable state; however, in F#, you can grab parts of the implemented solution and test it interactively. This leads to you spending less time on debugging and features that only work the first time you run them.
Loading external references
Once your application starts to grow, you will need to perform more advanced actions in F# Interactive that will require you to load external libraries. The interactive shell cannot resolve references automatically, but rather, needs some help.
Let's say we include the Newtonsoft.Json library to be able to quickly turn our F# types into JSON, and we want to test this in F# Interactive:
open Newtonsoft.Json
let toJson = JsonConvert.SerializeObject
We will get the following error message when trying to execute the preceding lines in the interactive shell:
1232OS_05_02.fs(5,10): error FS0039: The namespace or module 'Newtonsoft' is not defined
In order to get the path to the library you want to reference, right-click on References in the Solution Explorer tab and copy the Full Path property value, as shown in the following screenshot:
The interactive shell doesn't have the solution directory as its current working directory, so we need to specify the absolute paths to the assemblies we want to reference:
> #I @"D:\code\packages\Newtonsoft.Json.6.0.5\lib\net45";;
--> Added 'D:\code\packages\Newtonsoft.Json.6.0.5\lib\net45' to library include path
> #r Newtonsoft.Json";;
--> Referenced 'D:\code\packages\Newtonsoft.Json.6.0.5\lib\net45\Newtonsoft.Json.dll' (file may be locked by F# Interactive process)
First, we'll add the installation path of the Newtonsoft library to the default assembly search path and then reference it by name.
Of course, when you have a large project, you don't want to include all your references manually. What you need to do then is create an include script in your F# application and reference that:
// add include paths
#I @".\packages\Newtonsoft.Json.6.0.5\lib\net45";;
// ... following more paths
// reference assemblies
#r @"Newtonsoft.Json";;
// ... following more references
In the include script, you can specify the paths that are relative to the script execution path, allowing the script to work with all developer machines and not just your own:
> #load @"D:\code\include.fsx";;
[Loading D:\code\include.fsx]
namespace FSI_0002
Now, you can use code that references types and modules within the external assemblies directly from F# Interactive.
Testing with Visual Studio
So far, we've been talking about interactive testing with F#. This is a basic concept in starting to write qualitative code, but what this book really is about is test automation.
Visual Studio Professional comes with some basic testing tools you can use to automate testing in your solution. They are, however, not made to work with F# and need some fiddling before you could begin using them, as explained in the following steps:
1. Create a new F# library project:
2. Add a reference to Microsoft.VisualStudio.QualityTools.UnitTestingFramework. This will later be referred to as MSTest:
3. Create a new F# source file in your project with the fs extension. Now, you can write your first test, as follows:
4. module MSTest
5. open Microsoft.VisualStudio.TestTools.UnitTesting
6. [<TestClass>] type CalculatorShould () =
[<TestMethod>] member this.``add 1 and 2 with result of 3`` () = Assert.AreEqual(3, 1 + 2)
7. After compiling the test, go to Test | Run All Tests in the top menu or press CTRL + R, A. This will bring up the Test Explorer window that will run the test. If the Test Explorer window is already open, you will have to switch your focus to it in order to see the test results:
Congratulations! You have written and run your first automated test within Visual Studio.
NUnit
MSTest is not the optimal test framework. Actually, I consider it just slightly more convenient than using no test framework at all. Instead, the test framework considered as standard today is NUnit. This is why we'll take a look at how to get NUnit running within Visual Studio with F#:
1. First, you need to add the NUnit Test Adapter extension. Go to Tools | Extensions and Updates in the top menu bar and search for nunit in Visual Studio Gallery:
This adapter will make it possible to use the built-in Visual Studio test runner to execute NUnit tests.
2. Create a new F# library project and add NUnit.Framework as a reference to it through the NuGet Package Manager:
3. Now, we can write a unit test featuring NUnit:
4. module NUnitTests
5. open NUnit.Framework
[<Test>] let ``calculator should add 1 and 2 with result of 3`` () = Assert.That(3, Is.EqualTo(1 + 2))
6. You can execute tests written with NUnit the same way you would execute with MSTest. After compiling the test, go to Test | Run All Tests in the top menu or press CTRL + R, A.
The main difference between tests written in MSTest and NUnit is that NUnit tests don't need a test class attribute. It is enough to mark out test methods that make the test procedure much simpler, as we can use the let statement for the tests, as well.
xUnit
While NUnit is a great replacement for MSTest, it still has a problem not being actively updated with new features, and the NUnit framework is very hard to extend. This is the major caveat of NUnit and the major issue xUnit is currently working to solve.
In order to run xUnit tests inside Visual Studio you need to add a NuGet package to your solution that contains the Visual Studio Test Adapter:
1. Right click on your solution in Solution Explorer and choose Manage NuGet Packages for Solution:
2. In the drop down box saying Stable Only you need to change it to Include Prerelease.
3. Search for xunit.runner.visualstudio and install the package to your solution.
4. Create a new F# library project and use the package manager to add xUnit.net to the current project:
5. Now, we can write a simple test to verify that the xUnit integration in Visual Studio works. The Test Explorer tab will pick up our unit test so we're able to execute it:
6. module 1232OS 05 06
7.
8. open Xunit
9.
[<Fact>] let ``calculator should add 1 and 2 with result of 3`` () = Assert.Equal(3, 1 + 2)
10. The xUnit framework doesn't require class fixtures; we're just happy to attribute simple let functions as our tests.
Comparing MSTest, NUnit, and xUnit
We have just taken a look at how to get started with the three different testing frameworks: MSTest, NUnit, and xUnit. How should we choose between the three?
None of these frameworks are particularly designed to work with F# or functional programming, but they apply pretty well. MSTest is a bit cumbersome, as it requires a class with the TestClass attribute in order to execute the test suite.
The following highlights the strengths of the frameworks:
· MSTest: This is a good choice if you're not allowed to run open source in your organization.
· NUnit: This provides great support for external test data. It is the most mature open source test framework.
· xUnit: This is the best test framework when it comes to extensibility.
I would choose MSTest for organizations that have extremely high requirements on what kind of software you bring into a project. These could be pharmaceutical companies or those in the financial sector, in situations where compliance is more important than efficiency.
NUnit is the most frequently used testing framework in .NET and most likely the best choice when it comes to stability and widespread support. If you're writing code that will be shared outside the company or you want as little friction as possible in your continuous integration setup, you should select NUnit.
NUnit is a challenge when it comes to extensibility, however, and if you need to do something with your test suite that is not completely standard, you should select xUnit instead. It comes with an extensibility story and is easier to modify to suit your specific needs.
Tools and frameworks
The built-in tools for unit testing in Visual Studio are not the greatest. To smoothen the process, you need to use some external frameworks and tools.
FsUnit
The tests in this book will mainly focus on FsUnit as the main tool to write unit tests. This is not a test framework like NUnit or xUnit, but more of a helper library to write tests in a more functionally idiomatic way.
Let's start with an example. This function will get the date and time for the Swedish Midsummer in a specified year:
// find date of midsummer eve in Sweden
let midsummerEve year =
// get a day in June
let dateInJune day = new System.DateTime(year, 6, day)
// is specified date a friday
let isFriday (date : System.DateTime) = date.DayOfWeek = System.DayOfWeek.Friday
// find first friday between 19-26 june
[19..26] |> List.map dateInJune |> List.find isFriday
Instead of writing a test like this:
open NUnit.Framework
// testing using NUnit
[<Test>]
let ``midsummer eve in 2014 should occur Jun 20`` () =
// test
let result = midsummerEve 2014
// assert
Assert.That(result.ToString("MMM d"), Is.EqualTo("Jun 20"))
You can instead write it with FsUnit, like this:
open NUnit.Framework
open FsUnit
// testing using fsunit
[<Test>]
let ``midsummer eve in 2014 should occur Jun 19`` () =
(midsummerEve 2015).ToString("MMM d") |> should equal "Jun 19"
The FsUnit way of writing the assertion is a bit more terse and functional. This is why it is preferred.
Start by adding FsUnit as a NuGet package to your project, as shown in the following screenshot:
The default testing framework for FsUnit is NUnit. If you want to use another testing framework, you need to find the FsUnit package for that framework. There are both FsUnit.xUnit and Fs30Unit.MsTest.
The first time you run this test, you will most probably get a MethodNotFoundException exception, which will look like this:
Test Name: midsummer eve in 2014 should occur Jun 19
Test FullName: chapter03.nunit._1232OS_03_07.midsummer eve in 2014 should occur Jun 19
Test Source: D:\code\nunit\1232OS_03_07.fs : line 27
Test Outcome: Failed
Test Duration: 0:00:00.001
Result Message: System.MissingMethodException : Method not found: 'Void FsUnit.TopLevelOperators.should(Microsoft.FSharp.Core.FSharpFunc'2<!!0,!!1>, !!0, System.Object)'.
Result StackTrace: at chapter05.nunit._1232OS_03_07.midsummer eve in 2014 should occur Jun 19()
The reason for this is that FsUnit was not compiled with the same version of Fsharp.Core that you're currently running. This can be amended by downloading the source for FsUnit and compiling it, or you can add the following configuration to your App.config file in the test project:
<?xml version="1.0" encoding="utf-8"?>
<configuration>
<runtime>
<assemblyBinding xmlns="urn:schemas-microsoft-com:asm.v1">
<dependentAssembly>
<assemblyIdentity name="FSharp.Core" publicKeyToken="b03f5f7f11d50a3a" culture="neutral" />
<bindingRedirect oldVersion="0.0.0.0-4.3.1.0" newVersion="4.3.1.0"/>
</dependentAssembly>
</assemblyBinding>
</runtime>
</configuration>
This will tell the test runner, when looking for an assembly version of FSharp.Core lesser than 4.3.1.0, to use 4.3.1.0 instead. Exchange the version number here with the current version of F# that you're running.
You are now ready to start writing tests with FsUnit.
ReSharper test runner
There is a Visual Studio extension called ReSharper, produced by JetBrains, that contains a lot of goodies to increase productivity within Visual Studio. This product comes with a license fee, which is reasonable for the production gains you will experience while writing C# code, but as for F#, the tool has no real application at all.
I'm mentioning it here because most F# developers come from a C# developer background and have heard of this tool or owned it in the past.
What you gain from this tool, in the context of this book, is a much better test runner than the one built into Visual Studio, for the following reasons:
· It provides a hierarchical view of test cases
· You can filter tests on their statuses
· You can create different test sessions
· You can see the complete output of the test
· You can export test results in different formats
· It executes faster than Visual Studio Test Explorer
All in all, it is a better test runner that will help you succeed when your test suite expands from a few hundred tests to a couple of thousands and you need to start organizing them:
In this example, I showed how a hierarchical test suite will look when writing tests for the GameOfLife project. The GameOfLife project itself has four distinct rules we use for the basis of the hierarchy, and from there, we break the problem into smaller parts.
This gives our tests the context and provides the needed structure when you have a large quantity of tests for your system.
xUnit.net
If you try to run xUnit.net tests from ReSharper, Visual Studio will tell you it can't find any in the project. ReSharper will need a plugin in order to recognize the XUnit.net FactAttribute attribute, as explained in the following steps:
1. Go to ReSharper | Extension Manager in the top menu navigation.
2. Search for xUnit and find the xUnit.net Test Support extension. Install it, and it will enable you to execute xUnit tests within your projects, as shown in the following screenshot:
3. Restart Visual Studio and the test runner should find and execute your tests when you right-click on a project and choose Run Tests.
Executing a test suite outside Visual Studio
After you've succeeded in running your test suite in the development environment inside Visual Studio, it becomes increasingly important to know how to execute the test suite outside Visual Studio. In the target environment, where this code will end up, you probably won't have Visual Studio installed and sometimes may need to run the test suite to diagnose errors. So, this becomes important, especially in terms of continuous integration and having a team working on the same code base.
MSTest
In order to run MSTest tests outside of Visual Studio, you need to open the Developer Command Prompt, which comes with every version of Visual Studio. For Visual Studio 2013, they've made it a bit harder to find by hiding it in a folder called Visual Studio Tools, as explained in the following steps:
1. Press the Windows key to open the start menu.
2. Type Visual Studio Tools and click on the corresponding folder.
3. Start the Developer Command Prompt for VS2013.
If you navigate to your source code folder, you will now be able to execute the MSTest executable on your test suite:
>mstest /testcontainer:mstest/bin/Debug/chapter03.mstest.dll
Microsoft (R) Test Execution Command Line Tool Version 12.0.21005.1
Copyright (c) Microsoft Corporation. All rights reserved.
Loading mstest/bin/Debug/chapter03.mstest.dll...
Starting execution...
Results Top Level Tests
------- ---------------
Passed mstest._1232OS_03_04+CalculatorShould.add 1 and 2 with result of 3
1/1 test(s) Passed
Summary
-------
Test Run Completed.
Passed 1
---------
Total 1
Results file: D:\code\TestResults\mikaellundin_MIKAELLUNDI35B5.trx
Test Settings: Default Test Settings
If you want to execute your test suite on a computer without Visual Studio, you have the possibility to just install a test agent on the machine. This is a little more lightweight than installing the whole Visual Studio development environment; it contains the MSTest executable with all its dependencies.
It is, however, not possible to execute the MSTest suite without installing any additional software.
NUnit
The NUnit test runner can be downloaded from nunit.org and installed on the machine where you want to run the tests, but personally, I prefer including the test runner as a NuGet package to the solution instead:
This will install everything you need in the solution directory path, \packages\NUnit.Runners.2.6.3\tools.
In order to execute the tests from a graphical user interface, open up nunit.exe and add the test assembly to it:
You can also execute the test suite from the Command Prompt:
>packages\NUnit.Runners.2.6.3\tools\nunit-console.exe nunit\bin\Debug\chapter03.nunit.dll
NUnit-Console version 2.6.3.13283
Copyright (C) 2002-2012 Charlie Poole.
Copyright (C) 2002-2004 James W. Newkirk, Michael C. Two, Alexei A. Vorontsov.
Copyright (C) 2000-2002 Philip Craig.
All Rights Reserved.
Runtime Environment -
OS Version: Microsoft Windows NT 6.2.9200.0
CLR Version: 2.0.50727.8009 ( Net 3.5 )
ProcessModel: Default DomainUsage: Single
Execution Runtime: net-3.5
......
Tests run: 6, Errors: 0, Failures: 0, Inconclusive: 0, Time: 0.5967672495353 seconds
Not run: 0, Invalid: 0, Ignored: 0, Skipped: 0
Because NUnit doesn't have any external dependencies, it is suitable to just copy NUnit.Runner to the machine where and when tests are needed. As for build servers, you can commit the test runner to a source control, as you have the same version of the test runner running on a test suite and using the same version of the NUnit framework.
xUnit
If you want to execute the xUnit test suite outside Visual Studio, you need to use NuGet in order to download the xUnit.net: Runners package from the official NuGet feed, as shown in the following screenshot:
This will install the binaries at packages/xunit.runners.1.9.2 /tools, based on your solution directory.
If you open up the XUnit.net test runner GUI, you can easily add the binary and run your tests. This is useful to quickly determine whether a test suite is green on a different computer than the developer machine:
We can also easily execute the tests from the Command Prompt, giving us some easy integration with build servers and other continuous integration tools, as follows:
>packages\xunit.runners.1.9.2\tools\xunit.console.clr4.exe xunit\bin\Debug\chapter03.xunit.dll
xUnit.net console test runner (64-bit .NET 4.0.30319.34014)
Copyright (C) 2013 Outercurve Foundation.
xunit.dll: Version 1.9.2.1705
Test assembly: D:\code\xunit\bin\Debug\chapter03.xunit.dll
1 total, 0 failed, 0 skipped, took 0.334 seconds
This concludes the section on how to execute your test suite outside of Visual Studio. As we've already seen, the support for this is quite weak for MSTest, but stronger for both NUnit and xUnit. This might be important in terms of the portability of your code when you want to execute the test suite on more machines than just the developer machines and affect the choice of your framework.
FAKE
The point of being able to run tests outside Visual Studio is not something you will have to do very often, but it is an absolute requirement to run your tests in a build script.
FAKE is a build automation system similar to the make of C and the rake of Ruby, featuring F# with a specific Domain Specific Language (DSL) to set up build scripts fast and easy.
To get started, install the NuGet package for FAKE in your solution. You don't have to add it to any particular project. This is shown in the following screenshot:
The package will install itself on the solution directory path, packages\FAKE.3.5.5\tools. Now, in the solution level, create a new F# script file called build.fsx:
// define includes
#I @"packages/FAKE.3.5.5/tools/"
#r @"FakeLib.dll"
open Fake
open Fake.FileUtils
// constants
let buildDir = "./build"
let deployDir = "./deploy"
// define targets
Target "Clean" (fun _ ->
CleanDirs [buildDir; deployDir]
)
// define dependencies
// execute
Run "Clean"
The only thing this build script does is remove the build directory if it exists and adds a new one. It is a shell of the beginning of a FAKE build script. When we run it, the output looks like this:
>fsi build.fsx
Building project with version: LocalBuild
Shortened DependencyGraph for Target Clean:
<== Clean
The resulting target order is:
- Clean
Starting Target: Clean
Deleting contents of ./build
Creating D:\code\deploy
Finished Target: Clean
---------------------------------------------------------------------
Build Time Report
---------------------------------------------------------------------
Target Duration
------ --------
Clean 00:00:00.0114626
Total: 00:00:00.0417724
Status: Ok
---------------------------------------------------------------------
This is very verbose and clearly shows that a build directory is removed and then created. Let's add some compiling to the mix:
Target "BuildTest" (fun _ ->
let projectFiles = !! (srcDir + "/**/*.fsproj")
MSBuildDebug testDir "Build" projectFiles
|> Log "Test build: "
)
This piece of code will debug and build the project files and output the result to the test directory. Now all we need to do is execute these tests in the output directory, as follows:
Target "Test" (fun _ ->
// mstest
[testDir + "/chapter03.mstest.dll"]
|> MSTest (fun p -> p)
// nunit
[testDir + "/chapter03.nunit.dll"]
|> NUnit (fun p ->
{ p with
DisableShadowCopy = true;
ToolPath = "./packages/NUnit.Runners.2.6.3/tools";
OutputFile = testDir + "/chapter03.nunit-TestResults.xml"}
)
// xunit
[testDir + "/chapter03.xunit.dll"]
|> xUnit (fun p ->
{ p with
ToolPath = "./packages/xunit.runners.1.9.2/tools/xunit.console.clr4.exe";
ShadowCopy = false;
HtmlOutput = true;
XmlOutput = true;
OutputDir = testDir
}
)
)
This target now runs all our test suites: MSTest, NUnit, and xUnit. It does so by using the command-line tools we tried out in the previous chapter. Sometimes, FAKE will be able to figure out where this tool is, for example, MSTest, where the binary exists in the system path. As for NUnit and xUnit, we merely point out the ToolPath path where we installed the NuGet packages for these runners.
After running the test suite, we get this report:
---------------------------------------------------------------------
Build Time Report
---------------------------------------------------------------------
Target Duration
------ --------
Clean 00:00:00.0194436
BuildTest 00:00:00.8076760
Test 00:00:04.0909364
Total: 00:00:04.9540847
Status: Ok
---------------------------------------------------------------------
We can now continue to expand this script by building the release version of the code, creating deployment packages, and sending them off to a continuous deployment server. The possibilities are endless, and the best thing is that your version will control the build script together with the rest of the code.
Continuous integration
Two of the largest risks of having tests is that they will not be run or even pass. This is what will happen if you have a test suite without experienced developers maintaining it. In this case, it will no longer bring value, but only baggage. A test suite that is not used and doesn't pass is a waste and should either be fixed or deleted.
I was once working on quite an advanced order flow. To maintain its quality by myself, I covered the solution with tests knowing that if my tests were green, the code would work as expected.
I handed over the code to a maintenance team and after six months, they called me up asking me to fix the code. They were going to make some changes, and when they did, the price calculation no longer matched up. "So, are the tests passing?" I asked them. It turned out the developers had inactivated the test suite when the first test turned red.
It is easy to avoid this by having continuous integration run the tests every time code is committed to source control, and send "you broke the build" e-mails when the tests fail. For some reason, there is a psychological barrier between the developer and the build server, making it a farther step to actually inactivate the tests in the build server.
If you inactivate the tests, you do it on your machine. If you break the build, you break it for the whole team.
Git Hooks
I'm a big fan of Git. My first large aha! moment was finding out I could perform version control without affecting anyone else on the team, as I could commit only to my local machine.
Git led me to commit code much more often and split tasks into units of work, where each unit could be committed with a single purpose message. This results in a very nice and readable version history.
The third was about finding out that I could branch everything, which led me to have the kind of flexibility in my workstream I never had before. Whenever a client rushed in to have a critical issue fixed right away, I didn't have to stash away the things I was currently working on, risking to mix them up with the fix. Instead, I just created a new fix branch from the master.
I'm not going to go into the details of how I use Git, but there is one aspect of it that I really like, concerning test automation.
This is the workflow when you work with a centralized repository:
1. Finish your code.
2. Execute your tests.
3. Commit the code to the Version Control System (VCS).
4. Continuous integration runs.
5. Report from the Continuous Integration (CI) build.
Every so often, developers, myself included, forget to perform step 2 before committing code to source control, leading to a failed build report from the continuous integration.
This would be no problem, except that the code is now committed to the central location, meaning it would affect the whole team. If the fix is particularly hard, it might hinder other developers to commit their code for several hours while you fix your failed build.
Even worse, other developers might not notice the failed build and commit code on top of the already failing build, making matters worse.
Git Hooks comes to the rescue!
Git lets us define scripts that should run before certain actions occur. These actions can be grouped into the server side and client side. Client-side hooks are those that will solve the problems I just stated.
Let's create a precommit hook that will run our tests before we commit code. Add the following line of code to the .git/hooks/pre-commit file:
#!/bin/sh
fsi build.fsx Test
By using the build script we created in the previous section, we will now execute the tests before committing the code. The build script will return a nonzero result if the test fails, causing Git to abort the commit. However, if the build is successful and returns 0, Git will go ahead and commit the changes.
This means we'll not be able to commit any changes to the Distributed Version Control System (DVCS) without making the tests pass. If the build fails in the build server, it will no longer be because we failed to run our tests locally.
In the same spirit, you can use a prepush hook to query the continuous integration server that the build has not failed before it is allowed to push more code. This way, we can avoid pushing more code to an already failing build.
TeamCity
TeamCity is one of the most prominent build servers for the .NET platform. It has taken over the continuous integration market, which you will understand unless you've completely invested in Team Foundation Server (TFS) and are forced to use TF Build.
TeamCity can be downloaded and installed from JetBrains and is free to use for open source projects. It will install itself as one of few Windows services that represent the build server controller and the agents that run the build.
We're now going to dive into how to set up a new build and run your tests through TeamCity.
1. Start by creating a new project, as shown in the following screenshot:
2. Enter the basic information about your project:
3. The next step is to create a build configuration. Normally, you have a compile step and then a test step, but for brevity, we'll skip immediately to the test step:
4. For code repository, I'm using Git and GitHub for this example. I will now enter the following details into the TeamCity VCS configuration:
5. Next up, it's time to configure the build steps. First, we need to restore the NuGet packages from the NuGet Package Feed because we're not committing the packages to source control, as shown in the following screenshot:
6. Next comes the command-line build step that will execute the FAKE build script, which will run the tests, as shown in the following screenshot:
7. Before running the build, we need to define some artifacts where the test results will be stored, as shown in the following screenshot:
8. Now, we can run the build. It will download all the required packages from the NuGet feed and then continue running our tests from the FAKE build script, as shown in the following screenshot:
The only thing left to do is set up a trigger so the build will run automatically every time a developer commits code, and then you'll be able to set up with continuous integration together with the F# project and tests.
Summary
In this chapter, we looked at how to get started and set up your environment for unit testing. We learned about three testing frameworks, along with their strengths and weaknesses, and saw that we can, without breaking a sweat, get started with all three in F#.
We also learned how to take testing with F# a step further, outside of Visual Studio and into our development process and continuous integration.
In the next chapter, we will dive deep into unit testing, all the techniques for writing good unit tests, and how to apply these with F#.