Testing and Debugging - THE RUBY WAY, Third Edition (2015)

THE RUBY WAY, Third Edition (2015)

Chapter 16. Testing and Debugging

I’ve just picked up a fault in the AE35 unit. It’s going to go 100% failure in 72 hours.

—HAL 9000, in 2001: A Space Odyssey

Testing is important. All competent programmers know this fact, even if it isn’t always at the forefront of their minds.

Of course, true exhaustive testing is usually impossible. A program or system of any significant size is always going to hold a few surprises as it goes through its life cycle. The best we can do is to test carefully and selectively, and get as wide a coverage as we can.

Historically, programmers don’t always test as adequately as they should. The typical reasons for this are tests that are difficult to set up and run, require manual intervention, or take too long to run.

In the 1990s, the computing community saw a “culture of testing” reach maturity. The term extreme programming is now old-fashioned, but the foundations are still relevant. Concepts such as test-driven development (TDD) and behavior-driven development (BDD) have gained firm mindshare among developers everywhere.

Whether you are a hard-core “test first” person is not the point. The point is that everyone can benefit from tests that are automated, easy to write, and easy to run.

Testing tools are in general easier to write in Ruby because Ruby is dynamic and flexible. In the same way, they are easy to use and (dare I say it?) sometimes even fun to use. There is something satisfying about making a software change and then watching every test run to completion and pass.

Besides these testing tools, Ruby also has various other tools and libraries for debugging, profiling, and code coverage. This chapter gives an overview of what is available.

We’ll start with the very popular RSpec, and also look at Minitest, which provides the classic style of testing. In addition, we’ll look at Cucumber for acceptance testing, and do a quick overview of other tools.

16.1 Testing with RSpec

In recent years, the idea of behavior-driven development (BDD) has become very widespread. This is the idea that a specification of a system’s behavior is a natural basis for testing it.

RSpec, the work of David Chelimsky and others, is probably the most common BDD framework for Ruby. It was originally based on JBehave for Java.

For the purposes of illustrating this testing framework as well as others, we’ll introduce a small piece of code here. (I hope you don’t mind a little flashback to 9th-grade math.)

The method quadratic (which we’ll store in the file quadratic.rb) solves a simple quadratic equation. It takes the three coefficients of a simple polynomial and returns an array of zero, one, or two real roots. For a little extra functionality, the fourth parameter is a Boolean value that tells whether to allow complex values as results:

require 'complex'

def quadratic(a, b, c, complex=false)
raise ArgumentError unless [a, b, c].all? {|x| Numeric === x }
discr = b*b - 4*a*c
if (discr > 0) || (complex && discr < 0)
val = Math.sqrt(discr)
r1 = (-b + val)/(2*a.to_f)
r2 = (-b - val)/(2*a.to_f)
return [r1, r2]
elsif discr.zero?
r1 = -b/(2*a.to_f)
return [r1]
else # complex is false here
return []
end

end

To test this, we’ll put together a fairly simple set of RSpec tests to be run using the rspec gem. These tests normally live under a spec directory and have a base name ending in _spec with an .rb extension.

In general, a single spec file tests a “chunk” of related functionality. This chunk usually corresponds to a single class, but may be smaller or larger than a class.

A typical spec file has the following form:

# spec/foobar_spec.rb
require 'foobar'

RSpec.describe Foobar do
describe "some_method" do
it "returns something when called" do
# Test code here...
end
end
end

After installing the rspec gem, you can run this spec file with the RSpec runner:

$ rspec spec/foobar_spec.rb

When rspec is run, it reads the .rspec file for command-line options. The default .rspec file contains —require spec_helper. This file, spec/spec_helper.rb, is a place to configure RSpec or place shared code that we need during testing. It is optional.

The spec file itself must require whatever class or other code is about to be tested. In our examples, we assume that the code being tested is located in lib. RSpec (or the spec helper) should add the lib directory to Ruby’s $LOAD_PATH so that files inside it can be required directly, as in the preceding example.

The describe method takes a parameter that is typically a class name but may be a descriptive string. The block contains all the test examples (and may also contain nested describe-blocks).

Now let’s look at a test of our quadratic formula method. See Listing 16.1 for the contents of the quadratic_spec.rb file.

Listing 16.1 Testing quadratic.rb with RSpec


require 'quadratic'

describe "Quadratic equation solver" do
it "can take integers as arguments" do
expect(quadratic(1, 2, 1)).to eq([-1.0])
end

it "can take floats as arguments" do
expect(quadratic(1.0, 2.0, 1.0)).to eq([-1.0])
end

it "returns an empty solution set when appropriate" do
expect(quadratic(2, 3, 9)).to eq([])
end

it "honors the 'complex' Boolean argument" do
solution = quadratic(1, -2, 2, true)
expect(solution).to eq([Complex(1.0,1.0), Complex(1.0,-1.0)])
end

it "raises ArgumentError when for non-numeric coefficients" do
expect { quadratic(3, 4, "foo") }.to raise_error(ArgumentError)
end
end


Each test example is a call of the it method; this method is named so that we have the illusion of an English-like pseudo-sentence with “it” as the subject. The string parameter completes that illusion, and the block contains code that actually performs the relevant test.

The test may contain arbitrary code, but it always must contain at least one statement that does the “real test” (technically called an expectation expression).

A typical expectation expression consists of the expect method (called on a subject), an expectation, and a matcher. The expectation methods are to and to_not.

In arguably its most common form, this expression simply checks to make sure that something equals something else: expect(result).to eq(value).

There are many matches, and you can define your own as well. Some of the most common ones are illustrated here (all testing true):

expect(5).to eq(5)
expect(5).to_not eq(6)
expect(5).to_not be_nil
expect(5).to be_truthy
expect(5).to_not eq(false)
expect(nil).to be_nil
expect("").to be_empty # "be_xxx" calls the "xxx?" method
expect([]).to be_empty # on the subject
expect([1,2,3]).to include(2)
expect([1,2]).to be_an(Array)

To test exceptions, the code must be placed inside a block that is passed to the expect method:

expect { x = 1 / 0 }.to raise_error

This raise_error method can take the class of the exception as a parameter as well as a string value. Both parameters are optional; usually the string comparison is considered overkill.

There is much more to RSpec. For advanced testing, you may need the before, after, and let methods (to handle setup, teardown, and so on). For these and other details, consult a complete reference.

Obviously RSpec does a lot of “black magic” under the hood. If it doesn’t suit your taste for this or any other reason, consider trying minitest, which is based on the more traditional “test unit” style of testing.

16.2 Testing with Minitest

The minitest gem, created by Ryan Davis, allows testing Ruby code in the “classic” test-unit style. It may be included with your copy of Ruby; if not, you can install it by running gem install minitest.

The Minitest::Test class uses reflection to analyze your test code. When you subclass it, any methods named starting with test are executed as test code:

require 'minitest/autorun'

class TestMyThing < Minitest::Test

def test_that_it_works
# ...
end

def test_it_doesnt_do_the_wrong_thing
# ...
end

# ...
end

It is highly inadvisable, and arguably incorrect, for the behavior of the tests to rely on the order in which they are run. Be careful to not write your tests in such a way that they only pass when run in the order they are written. Minitest will deliberately run them in a different random order each time.

It’s also not a bad idea to put a one-line comment describing the purpose and meaning of the test, especially if it’s complex or subtle. In general, each test should have one, and only one, purpose.

If we need to do some setup for each test, we can create setup and teardown methods for the class. It might seem counterintuitive, but these methods are called before and after every test.

What if we need to do some kind of setup that takes a long time? It’s not practical to do this for every single test, and we can’t put it inside the first test method (because the tests will be run in a random order). To do some setup only once, before any of the tests, you could put code in the body of the class, before the test methods (or even before the class itself).

But what if we want to do a corresponding teardown after all the tests? The “best” way is to override the class’s run method so as to “wrap” its functionality. While we’re at it, we add a class method for setup before all tests as well. Look at the example in Listing 16.2.

Listing 16.2 Setup and Teardown


require 'minitest/autorun'

class MyTest < Minitest::Test

def self.setup
# ...
end

def self.teardown
# ...
end

def self.run(*)
self.setup
super # run each test method as usual
self.teardown
end

def setup
# ...
end

def teardown
# ...
end

def test_it_works
# ...
end

def test_it_is_not_broken
# ...
end

# ...
end


You probably won’t find yourself doing this kind of thing often, but it can be extremely helpful when it’s needed.

Once the tests are running, what goes inside each test? We need to have some way of deciding whether a test passed or failed. We use assertions for that purpose.

The simplest assertion is just the assert method. It takes a parameter to be tested and an optional second parameter (which is a message); if the parameter tests true (that is, anything but false or nil), all is well. If it doesn’t test true, the test fails and the message (if any) is printed out.

Some other assertion methods follow (with comments indicating the meaning). Notice how the “expected” value always comes before the “actual” value; this is significant if you use the default error messages and don’t want the results to be stated backward:

assert_equal(expected, actual) # assert(expected == actual)
refute_equal(expected, actual) # assert(expected != actual)
assert_match(regex, string) # assert(regex =~ string)
refute_match(regex, string) # assert(regex !~ string)
assert_nil(object) # assert(object.nil?)
refute_nil(object) # assert(!object.nil?)

Some assertions have a more object-oriented flavor (and have paired refute versions as well):

assert_instance_of(klass, obj) # assert(obj.instance_of? klass)
assert_kind_of(klass, obj) # assert(obj.kind_of? klass)
assert_respond_to(obj, meth) # assert(obj.respond_to? meth)

Some deal specifically with exceptions and thrown symbols. They do not have refute counterparts, because if throw and raise are not called, the code will simply run as usual. Because calling them stops code from running as usual, they both have to take a block:

assert_throws(symbol) { ... } # checks that symbol was thrown
assert_raises(exception) { ... } # checks exception was raised

Collections and other Enumerable objects have a few matchers as well:

assert_empty(coll) # assert(coll.empty?)
assert_includes(coll, obj) # assert(coll.includes?(obj))

There are several others, but these form a basic complement that will cover most of what you need. For others, consult the Minitest documentation online.

There is also a flunk method, which always fails. This is more or less just a placeholder for tests that have not been written yet.

Armed with assertions, let’s test our quadratic formula. Test files go in the test directory. The naming convention is to mirror the lib directory (like RSpec), ending each filename in _test.rb (see Listing 16.3).

Listing 16.3 Testing quadratic.rb with Minitest::Test


require 'minitest/autorun'
require 'quadratic'

class QuadraticTest < MiniTest::Unit::TestCase
def test_integers
assert_equal [-1], quadratic(1, 2, 1)
end

def test_floats
assert_equal [-1.0], quadratic(1.0, 2.0, 1.0)
end

def test_no_real_solutions
assert_equal quadratic(2, 3, 9), []
end

def test_complex_solutions
actual = quadratic(1, -2, 2, true)
assert_equal actual, [Complex(1.0,1.0), Complex(1.0,-1.0)]
end

def test_bad_args
assert_raises(ArgumentError) { quadratic(3, 4, "foo") }
end
end


When you run a test file and do nothing special, the console test runner is invoked by default. This gives us feedback using good old-fashioned 1970s technology. In this case, Minitest produces the following output when running our tests:

Run options: —seed 7759

# Running tests:

.....

Finished tests in 0.000702s, 7122.5071 tests/s,
7122.5071 assertions/s.

5 tests, 5 assertions, 0 failures, 0 errors, 0 skips

If you are a fan of RSpec, Minitest can be used in a similar style. The primary difference is that RSpec uses the expect(subject).to something style, Minitest uses the more direct subject.must_something style. See Listing 16.4 for an example.

Listing 16.4 Testing quadratic.rb with Minitest::Spec


require 'minitest/autorun'
require 'quadratic'

describe "Quadratic equation solver" do
it "can take integers as arguments" do
quadratic(1, 2, 1).must_equal([-1.0])
end

it "can take floats as arguments" do
quadratic(1.0, 2.0, 1.0).must_equal([-1.0])
end

it "returns an empty solution set when appropriate" do
quadratic(2, 3, 9).must_equal([])
end

it "honors the 'complex' Boolean argument" do
actual = quadratic(1, -2, 2, true)
expected = [Complex(1.0,1.0), Complex(1.0,-1.0)]
actual.must_equal expected
end

it "raises ArgumentError when for non-numeric coefficients" do
lambda { quadratic(3, 4, "foo") }.must_raise ArgumentError
end
end


Running these spec-style tests produces the same output as running the tests did before.

In addition to what we’ve seen here, there are dozens of extensions for Minitest that add or change testing features. Some are listed in the Minitest readme, and you can check Rubygems.org and GitHub for more if you’re interested.

16.3 Testing with Cucumber

Another very powerful BDD test framework is Cucumber. This uses a very simple, flexible “language” or notation called Gherkin, which is close to plain English.

It is a matter of opinion when it might be appropriate to use Cucumber. It adds overhead and maintenance to test suites, but provides an explanation of what is being tested in language that nontechnical clients or managers can understand. An ideal case might be one where semi-technical or nontechnical stakeholders take part in writing specifications.

In other situations, especially those where everyone involved has a technical background, it can be dramatically less complex to write tests (up to and including integration and acceptance tests) directly in code.

The quadratic formula example is arguably less appropriate here, but let’s look at it anyhow. We’ll start by creating a structure with a directory called features and one under it called step_definitions. Under features there could be many files; in Listing 16.5, we’ll create just one, first.feature.

Listing 16.5 A Feature File for quadratic.rb


Feature: Check all behaviors of the quadratic equation solver

Scenario: Real roots
Given coefficients that should yield one real root
Then the solver returns an array of one Float r1
And r1 is a valid solution.

Given coefficients that should yield two real roots
Then the solver returns an array of two Floats, r1 and r2
And r1 and r2 are both valid solutions.

Scenario: No real roots, and complex flag is off
Given coefficients that should yield no real roots
And the complex flag is missing or false
Then the solver returns an empty array.

Scenario: No real roots, and complex flag is on
Given coefficients that should yield no real roots
And the complex flag is true
Then the solver returns an array of two complex roots, r1 and r2
And r1 and r2 are both valid solutions.


If you run Cucumber at this point, it does a clever and helpful thing—it creates a “skeleton” for you to use in writing step definitions, which are merely the Ruby implementations of the tests you described in the English-like spec. You can copy this into a file (in this case, I usedstep_definitions/first.rb) and modify it as you go along. Tests are already marked pending so that they will not “fail” when there is no code yet written.

Here is an example of that skeleton output:

Given(/^coefficients that should yield one real root$/) do
pending # express the regexp above with the code you wish you had
end

Then(/^the solver returns an array of one Float r(\d+)$/) do |arg1|
pending # express the regexp above with the code you wish you had
end

Then(/^r(\d+) is a valid solution\.$/) do |arg1|
pending # express the regexp above with the code you wish you had
end

Notice how the regular expression is used to associate the English (Gherkin) expression with the corresponding Ruby code. Here is how I filled out that section (in first.rb):

Given(/^coefficients that should yield one real root$/) do
@result = quadratic(1, 2, 1)
end

Then(/^the solver returns an array of one Float r(\d+)$/) do |arg1|
expect(@result.size).to eq(1)
expect(@result.first).to be_a(Float)
end

Then(/^r(\d+) is a valid solution\.$/) do |arg1|
expect(@result).to eq([-1.0])
end

These three small tests now pass. By filling out the rest of the step definitions in a similar way, it is possible to end up with a specification written in readable English that can still be executed to verify the results of your code.

Cucumber is very powerful and has many useful functions. For a fuller description of its features, check the Cucumber documentation online, or The Cucumber Book by Matt Wynne and Aslak Hellesøy.

16.4 Using the byebug Debugger

The most popular library for debugging in Ruby is the byebug gem. It is implemented using the TracePoint functionality that we investigated in Section 11.4.4, “Monitoring Program Execution,” of Chapter 11, “OOP and Dynamic Features in Ruby.” In versions of Ruby before 2.0, thedebugger gem provided very similar functionality.

Truthfully, debuggers don’t seem to get much standalone use in Ruby. My own use of Byebug is via the pry tool, which is examined in the next section. We’ll look at how to use the debugger first, and then move on to Pry and the additional features it provides beyond debugging.

To invoke Byebug by itself, simply invoke byebug instead of ruby on the command line:

byebug myfile.rb

At the prompt, which looks like (byebug), you can type such commands as step to step into a method call, list to print any or all of the program, and so on. Here are the most common of these commands:

continue (c)—Continue running up to a line.

break (b)—List or set a breakpoint.

delete—Delete some or all breakpoints.

catch—List or set a catchpoint.

step (s)—Step into a method.

next (n)—Got to the next line (step over a method).

where (w)—Print the current stacktrace.

help (h)—Get help (list all commands).

quit (q)—Quit the debugger.

Listing 16.6 presents the code of a simple program (too simple to need debugging, really).

Listing 16.6 A Simple Program for Debugging


STDOUT.sync = true

def palindrome?(word)
word == word.reverse
end

def signature(w)
w.split("").sort.join
end

def anagrams?(w1,w2)
signature(w1) == signature(w2)
end

print "Give me a word: "
w1 = gets.chomp

print "Give me another word: "
w2 = gets.chomp

verb = palindrome?(w1) ? "is" : "is not"
puts "'#{w1}' #{verb} a palindrome."

verb = palindrome?(w2) ? "is" : "is not"
puts "'#{w2}' #{verb} a palindrome."

verb = anagrams?(w1,w2) ? "are" : "are not"
puts "'#{w1}' and '#{w2}' #{verb} anagrams."


Listing 16.7 shows an entire debugging session, though some lines of the program that were printed have been removed for brevity. While reading it, keep in mind that the console is used for input and output as well as for debugging.

Listing 16.7 A Simple Debugging Session


$ byebug simple.rb

[1, 10] in simple.rb
=> 1: STDOUT.sync = true

(byebug) b Object#palindrome?
Created breakpoint 1 at Object::palindrome?
(byebug) b Object#anagrams?
Created breakpoint 2 at Object::anagrams?
(byebug) info b
Num Enb What
1 y at Object:palindrome?
2 y at Object:anagrams?
(byebug) c 16
Give me a word:
Stopped by breakpoint 3 at simple.rb:16

[11, 20] in simple.rb
15: print "Give me a word: "
=> 16: w1 = gets.chomp
17:

(byebug) c 19
live
Give me another word:
[14, 23] in simple.rb
18: print "Give me another word: "
=> 19: w2 = gets.chomp

(byebug) n
evil

[16, 25] in simple.rb
=> 21: verb = palindrome?(w1) ? "is" : "is not"
22: puts "'#{w1}' #{verb} a palindrome."

(byebug) c
Stopped by breakpoint 1 at Object:palindrome?

[1, 10] in simple.rb
=> 3: def palindrome?(word)
4: word == word.reverse

(byebug)

[1, 10] in simple.rb
3: def palindrome?(word)
=> 4: word == word.reverse
5: end

(byebug)
Next went up a frame because previous frame finished

[17, 26] in simple.rb
21: verb = palindrome?(w1) ? "is" : "is not"
=> 22: puts "'#{w1}' #{verb} a palindrome."

(byebug) c
'live' is not a palindrome.
'evil' is not a palindrome.

Stopped by breakpoint 2 at Object:anagrams?

[6, 15] in simple.rb
=> 11: def anagrams?(w1,w2)
12: signature(w1) == signature(w2)
13: end

(byebug) c
'live' and 'evil' are anagrams.


Note that you can repeat the last command given by simply pressing Return. While using a debugger, be aware that if you require other libraries, you may find yourself stepping over quite a few things at the beginning. I suggest you call continue with the line number of a line that interests you to execute to that point.

The debugger recognizes many other commands. You can examine the call stack and move up and down in it. You can “watch” expressions and break automatically when they change. You can add expressions to a “display” list. You can handle multiple threads and switch between them.

All these features are probably not well documented anywhere. If you use the debugger, I suggest you use its online help command and proceed by trial and error.

Some other debuggers are graphical. See Chapter 21, “Ruby Development Tools,” for a discussion of Ruby IDEs (integrated development environments) if you want this kind of tool.

16.5 Using pry for Debugging

Every Rubyist is familiar with irb. But in recent years, the pry tool has emerged as something that many people find superior. You can install Pry by adding it to your project’s Gemfile or running gem install pry.

Like irb, pry is a REPL (read-eval-print-loop) tool. However, it includes several more sophisticated features, as well as an extension system so anyone can add more. There are extensions that integrate pry with the byebug and debugger gems to provide all the features of a full debugger as well as the advanced REPL of pry.

Let’s look at how we can halt a running program and debug its internal behavior. We’ll take Listing 16.6 and make a couple of changes, adding require 'pry' at the beginning and then adding a call to binding.pry sometime later:

require 'pry'

# ...
w2 = gets.chomp

binding.pry

# ...

Here are some of the more important commands (excerpted from online help):

cd Move into a new context (object or scope).
find-method Recursively search for a method within a Class/
Module or the current namespace.
ls Show the list of vars and methods in the current
scope.
whereami Show code surrounding the current context.
wtf? Show the backtrace of the most recent exception.
show-doc Show the documentation for a method or class.
show-source Show the source for a method or class.
reload-code Reload the source file that contains the
specified code object.

So let’s run this. The following session shows how we enter the pry environment and look around:

$ ruby myprog.rb
Give me a word: parental
Give me another word: prenatal

From: /Users/Hal/rubyway/ch16/myprog.rb @ line 23 :

18: w1 = gets.chomp
19:
20: print "Give me another word: "
21: w2 = gets.chomp
22:
=> 23: binding.pry
24:
25: verb = palindrome?(w1) ? "is" : "is not"
26: puts "'#{w1}' #{verb} a palindrome."
27:
28: verb = palindrome?(w2) ? "is" : "is not"

[1] pry(main)> w1
=> "parental"
[2] pry(main)> w2
=> "prenatal"
[3] pry(main)> palindrome?(w1)
=> false
[4] pry(main)> anagrams?(w1, w2)
=> true
[5] pry(main)> exit
'parental' is not a palindrome.
'prenatal' is not a palindrome.
'parental' and 'prenatal' are anagrams.

Although pry by itself is not a “debugger” in the traditional sense, it can embed a debugger, and has many powerful features that plain debuggers do not have besides. Refer to Chapter 21 for more information on those features (and as always, the most recent documentation online).

16.6 Measuring Performance

I don’t like to place too much emphasis on optimizing for speed. The best way to do this in general is to simply select an algorithm that runs in a reasonable amount of time for your use case.

Certainly speed matters. In some cases, it matters greatly. However, it is usually a mistake to worry about speed too soon in the development cycle. The saying goes, “Premature optimization is the root of all evil.” (This originated with Hoare and was restated by Knuth.) Or as someone else put it: “Make it right, then make it fast.” At the application level, this is generally a good heuristic (though at the level of large systems, it may be of less value).

I would add to this precept: Don’t optimize until you measure.

That isn’t so much to ask. Refrain from refactoring for speed until you actually know two things: First, is it really too slow? And second, exactly which parts are causing the speed problem?

The second question is more important than it appears. Programmers tend to think they have a pretty good idea of where the execution time is going, but in reality, studies have shown conclusively that on the average they are extraordinarily bad at making these determinations. “Seat of the pants” optimization is not a viable option for most of us.

We need objective measurements. We need a profiler.

Ruby comes with a profiler called profile. Invoking it is as easy as requiring the library:

ruby -rprofile myprog.rb

Consider the program Listing 16.8. Its purpose is to open the /usr/share/dict/words file and look for anagrams. It then looks for which words have the most anagrams and prints them out.

Listing 16.8 Finding Anagrams in the Dictionary


words = File.readlines("/usr/share/dict/words")
words.map! {|x| x.chomp }

hash = {}
words.each do |word|
key = word.split("").sort.join
hash[key] ||= []
hash[key] << word
end

sizes = hash.values.map {|v| v.size }
most = sizes.max
list = hash.find_all {|k,v| v.size == most }

puts "No word has more than #{most-1} anagrams."
list.each do |key,val|
anagrams = val.sort
first = anagrams.shift
puts "The word #{first} has #{most-1} anagrams:"
anagrams.each {|a| puts " #{a}" }
end

num = 0
hash.keys.each do |key|
n = hash[key].size
num += n if n > 1
end

puts
puts "The dictionary has #{words.size} words,"
puts "of which #{num} have anagrams."


I know you are curious about the output. Here it is:

No word has more than 14 anagrams.
The word alerts has 14 anagrams:
alters
artels
estral
laster
lastre
rastle
ratels
relast
resalt
salter
slater
staler
stelar
talers

The dictionary has 483523 words,
of which 79537 have anagrams.

On my systems, that file has more than 483,000 words, and the program runs in a little more than 18 seconds. Where do you think the time is going? Let’s find out. The output from the profiler is more than 100 lines long, sorted by decreasing time. We’ll look at the first 20 lines:

% cumulative self self total
time seconds seconds calls ms/call ms/call name
42.78 190.93 190.93 15 12728.67 23647.33 Array#each
10.78 239.04 48.11 1404333 0.03 0.04 Hash#[]
7.04 270.48 31.44 2 15720.00 25575.00 Hash#each
5.66 295.73 25.25 483523 0.05 0.05 String#split
5.55 320.51 24.78 1311730 0.02 0.02 Array#size
3.64 336.76 16.25 1 16250.00 25710.00 Array#map
3.24 351.23 14.47 483524 0.03 0.03 Array#sort
3.12 365.14 13.91 437243 0.03 0.03 Fixnum#==
3.04 378.72 13.58 483526 0.03 0.03 Array#join
2.97 391.98 13.26 437244 0.03 0.03 Hash#default
2.59 403.53 11.55 437626 0.03 0.03 Hash#[]=
2.43 414.38 10.85 483568 0.02 0.02 Array#<<
2.29 424.59 10.21 1 10210.00 13430.00 Array#map!
1.94 433.23 8.64 437242 0.02 0.02 Fixnum#<=>
1.86 441.54 8.31 437244 0.02 0.02 Fixnum#>
0.72 444.76 3.22 483524 0.01 0.01 String#chomp!
0.11 445.26 0.50 4 125.00 125.00 Hash#keys
0.11 445.73 0.47 1 470.00 470.00 Hash#values
0.06 446.00 0.27 1 270.00 270.00 IO#readlines
0.05 446.22 0.22 33257 0.01 0.01 Fixnum#+

Here we see that the Array#each method is taking more time than anything else. That makes sense; we have a loop that runs for each word and does significant work with each iteration. The average is misleading because the first call of each takes almost all the time; the other 14 calls (see anagrams.each) take a short amount of time.

We also notice that Hash#[] is an expensive operation (largely because we do it so often); 1.4 million of these calls take not quite 11 seconds.

Notice where the readlines call fell—almost off our list of the top 20. This program is not I/O bound at all; it is compute bound. The read of the entire file took hardly more than a fourth of a second.

The real value of profiling is not seen in this example, however. This program doesn’t have any methods or any class structure. In real life, you would see your own methods listed among the core methods. You would then be able to tell which of your methods was among the top 20 “time wasters.”

One thing to realize about the Ruby profiler is that (perhaps ironically) the profiler itself is slow. It hooks into the program in many places and monitors execution at a low level (in pure Ruby). So don’t be surprised if your program runs an order of magnitude slower. This program when profiled took 7 minutes 40 seconds (or 460 seconds), slowed down by a factor of 25 or more.

Besides the profiler, there is a lower-level tool you should be aware of: The benchmark standard library is also useful in measuring performance.

One way to use this little tool is to invoke the Benchmark.measure method and pass it a block:

require 'benchmark'

file = "/usr/share/dict/words"
result = Benchmark.measure { File.readlines(file) }
puts result

# Output: 0.350000 0.070000 0.420000 ( 0.418825)

The output from this method is as follows:

• User CPU time (in seconds)

• System CPU time

• Sum of user and system times

• Elapsed real time

For comparing several different items, the Benchmark.bm method is convenient. Give it a block, and it will pass in a reporting object. Invoke that object with a label and a block, and it will output the label and time the block. Look at the following code:

require 'benchmark'

n = 200_000
s1 = ""
s2 = ""
s3 = ""

Benchmark.bm do |rep|
rep.report("str << ") { n.times { s1 << "x" } }
rep.report("str.insert ") { n.times { s3.insert(-1,"x") } }
rep.report("str += ") { n.times { s2 += "x" } }
end

Here, we compare three ways of getting a character into the end of a string; the result is the same in each case. We do each operation 200,000 times so as to measure the effect better. Here is the output:

user system total real
str << 0.180000 0.000000 0.180000 ( 0.174697)
str.insert 0.200000 0.000000 0.200000 ( 0.200479)
str += 15.250000 13.120000 28.370000 (28.375998)

Notice how time consuming the third case is? It’s a full two orders of magnitude slower. Why is this? What lesson can we learn?

You might conclude that the + operator is slow for some reason, but that isn’t the case. This is the only one of the three methods that doesn’t operate on the same object every time but creates a new object.

The lesson is that object creation is an expensive operation. There may be many such small lessons to be learned by using Benchmark, but I still recommend profiling at a higher level first.

Minitest also has benchmarking capabilities (see the minitest/benchmark library), and RSpec has the —profile option to print the slowest specs.

16.7 Pretty-Printing Objects

The purpose of the inspect method (and the p method that invokes it) is to show objects in human-readable form. In that sense, there is a connection with testing and debugging that justifies covering this here.

The only problem with p is that the object it prints out can be difficult to read. That is why we have the standard library pp, which adds a method of the same name.

Consider the following contrived object called my_obj:

class MyClass

attr_accessor :alpha, :beta, :gamma

def initialize(a,b,c)
@alpha, @beta, @gamma = a, b, c
end

end

x = MyClass.new(2, 3, 4)
y = MyClass.new(5, 6, 7)
z = MyClass.new(7, 8, 9)

my_obj = { x => y, z => [:p, :q] }
p my_obj

Running this code produces the following output:

{#<MyClass:0xb7eed86c @beta=3, @alpha=2,
@gamma=4>=>#<MyClass:0xb7eed72c @beta=6, @alpha=5, @gamma=7>,
#<MyClass:0xb7eed704 @beta=8, @alpha=7, @gamma=9>=>[:p, :q]}

It’s accurate, and it’s technically readable—but it isn’t pretty. Now let’s require the pp library and use pp instead:

require 'pp'

# ...

pp my_obj

In this case, we get the following output:

{#<MyClass:0xb7f7a050 @alpha=7, @beta=8, @gamma=9>=>[:p, :q],
#<MyClass:0xb7f7a1b8 @alpha=2, @beta=3, @gamma=4>=>
#<MyClass:0xb7f7a078 @alpha=5, @beta=6, @gamma=7>}

At least it adds some spacing and line breaks. It’s an improvement, but we can do better. Suppose we add the special pretty_print method to MyClass:

class MyClass

def pretty_print(printer)
printer.text "MyClass(#@alpha, #@beta, #@gamma)"
end

end

The printer argument is passed in by the caller (or ultimately by pp). It is a text accumulator of class PP; we call its text method and give it a textual representation of self. The result we get follows:

{MyClass(7, 8, 9)=>[:p, :q], MyClass(2, 3, 4)=>MyClass(5, 6, 7)}

Of course, we can customize this as much as we want. We could put the instance variables on separate lines and indent them, for instance.

In fact, the pp library does have a number of facilities for “pp-enabling” your own classes. Methods such as object_group, seplist, breakable, and others are available for controlling comma separation, line breaks, and similar formatting tasks. For more details, consult the documentation for pp.

16.8 Not Covered Here

This book can cover only a portion of the Ruby ecosystem. I’ll give passing mention to a few more items. Also see Chapter 21 because that material is related.

I am a great believer in code coverage tools. It is incredibly useful to know what parts of your code are not getting exercised. This helps you to know when you have areas that are lacking in unit tests.

In recent years, the most popular code coverage tool has been simple-cov; this is a successor to rcov and can be installed as a gem. On execution, it creates a directory called coverage, where you will find HTML files. The index.html has an overview of the results, and it links to all the sources with the lines highlighted according to whether they were executed.

There are several continuous integration tools for Ruby; the most popular self-hosted tool is Jenkins. The most popular hosted service for open-source projects is Travis CI. Hosted options for private projects include Travis CI, Circle CI, Semaphore, and Codeship. Search online for what might work best for you.

Also, many static code analysis tools are available for Ruby. I have used flog, which helps find “ugly” or complex code; flay helps find code that is structurally similar and can perhaps be refactored (in accordance with the DRY principle, “Don’t Repeat Yourself”). The tool heckle is particularly useful (and frustrating) because it helps discover weaknesses in your tests (by mutating the code and checking whether a failure results). The hosted service Code Climate automatically runs these quality tools, as well as code coverage and security-auditing tools, to track code quality and test coverage over time.

As far as consistent Ruby style goes, the rubocop tool is fairly useful. However, I specifically disagree with one or two of its default rules.

16.9 Conclusion

In this chapter, we’ve looked at some of the mechanics of testing (especially unit testing). We looked specifically at Minitest, RSpec, and Cucumber.

We’ve looked briefly at the Ruby debugger. We also saw how pry can be used to “jump” into an interactive REPL session. And, finally, we’ve looked at profiling and benchmarking to measure the speed of our Ruby code.

If you’re working on software for public consumption, what do you do after testing is complete? It’s time then to worry about packaging and distributing the software. This is the topic of the next chapter.