Build Awesome Command-Line Applications in Ruby 2: Control Your Computer, Simplify Your Life (2013)

Appendix 1. Common Command-Line Gems and Libraries

To keep things simple, we’ve used only a handful of tools to demonstrate the principles presented in this book. We chose OptionParser because it’s a standard library available with Ruby, and we used GLI because of the ease with which we can add the necessary features of a command suite.

Since Ruby allows such a wide variety in programming style, it should be no surprise that there are many different libraries for writing command-line apps, each with its own unique style. We’ll go over a few popular libraries that exhibit different styles of writing a command-line app. You might find some of them more suited to the way you prefer to work, but the goal is always the same: to make awesome command-line apps easily and quickly, using whatever tools work best for you. Also keep in mind that there are many more libraries available than we have time to look at, but these are tools you’re likely to see in the real world.

This appendix is split up into three major sections. In the first, we’ll show some alternative implementations of our simple command-line app db_backup.rb using three alternatives to OptionParser: trollop, methadone, and main. In the second section, we’ll do the same thing with our command suite todo, using thor and main. The final section will be a whirlwind tour of the tools we couldn’t cover and where you can find out more about them.

Since, by now, you should be well familiar with the problems that db_backup.rb and todo solve, we’ll show a lot more code at once. For each library, we’ll outline its general design and features, show the implementation using that library (possibly calling out interesting or confusing bits of code), and conclude with a brief discussion of its pros and cons.

A1.1 Alternatives for Simple Command-Line Apps

We saw in Chapter 2, Be Easy to Use and Chapter 3, Be Helpful that OptionParser is very powerful, particularly in the way in which its on method works. If we don’t need this power, however, OptionParser can feel a bit verbose. Often, all we want to do is have the existence/absence of flags and switches populate a Hash, like so:

This can get tedious and add unnecessary complexity to our code. Because of this, several libraries have been developed to make this simpler. We’ll look at three different libraries that try to solve this problem and make it easier for you to specify your command-line interface.

trollop

trollop[54] aims to allow parsing command-line options with as little code as possible. Instead of using the three lines per option that we’ve seen thus far, trollop allows you to specify options in one line. It doesn’t use OptionParser internally and supports only a subset of its features.

trollop has two modes of operation. In the simplest mode, you make a call to Trollop::options , which takes a block. That block defines your user interface (which we’ll see in a bit) and returns a Hash of the options the user provided on the command line. In the second mode, these two activities are split into two steps. You first declare your user interface by passing a block to Trollop::Parser.new , which returns a parser object. You then use that parser to parse the command line by calling parse , which returns a Hash of the command-line options.

For db_backup.rb, we have to use the second form, since we are reading defaults from an external config file. Unlike OptionParser, trollop does not provide direct access to the Hash of options beforehand. Instead, we’ll apply our defaults after parsing. There is a second advantage to this mode as well.

trollop provides a method Trollop::with_standard_exception_handling , which takes a block. Certain exceptions, if thrown inside this block, will kick off trollop’s default error handling. In addition to calling parse inside this block (since parse throws trollop-specific exceptions), we’ll also look for our default argument, the database name, and throw Trollop::HelpNeeded if we don’t find it. This replaces our explicit exit and call to to_s on OptionParser from our original implementation. Let’s see what the code looks like at this point:

The highlighted code is where we apply our defaults from the config file (it’s parsed the same as before). Only if the user didn’t specify something on the command line do we apply the default.

With this basic structure, how do we declare our UI? Trollop provides the method opt , which handles this for us. To declare our --username flag, which also responds to -u, we need just one line of code:

trollop will automatically assign a short-form option based on the first letter of the long-form option. Further, by specifying the :type => :string option, we indicate that this is a flag. If we used something like :int instead, trollop would convert the type for us.

In the case of our switch --end-of-iteration, we want the short form to be -i and not the -e that trollop would pick by default. We can change this using the :long option, like so:

There’s a slight bit of complication with the option parsing because trollop doesn’t support negatable options. In the OptionParser version, we used a string like "--[no-]gzip to accept both --gzip and --no-gzip. Although trollop doesn’t support this, we can simulate it using two options and the conflict method, like so:

If the user specifies both options, they will get an error, and we still have the fluent user interface that we want. The rest of our code is largely the same after option parsing; we have a Hash named options that contains the command-line options the user provided.

We’ve seen how trollop eliminates a lot of boilerplate needed in declaring the user interface. trollop is also designed to be distributed as a single file you can include in your application for distribution; this can be a huge advantage in environments where installing external dependencies is difficult or impossible.

So, if you are doing something idiomatic with OptionParser, trollop can save you quite a bit of code. The trade-off comes from a few missing features. trollop does not provide validation via a regexp, Array, or Hash, nor does it support OptionParser’s flexible type conversions. Still, trollop can be a time-saver for simpler apps that you want to implement quickly, where you might not need all of OptionParser’s power to get the job done.

methadone

methadone[55] was developed by the author while writing this book. Like trollop, its goal is to reduce the amount of code you have to write to parse the command line. Unlike trollop, however, methadone still provides access to all of OptionParser’s features. Further, methadone handles several other time-consuming tasks when writing a command-line application, such as the following:

· Bootstrap your application, providing a gemspec, Rakefile, README, a unit test, and an acceptance test with one easy command.

· Place main logic at the top of your file in a “main” method, instead of at the bottom where it’s hard to find.

· Automatically create the banner based on the UI you’ve declared.

Unlike trollop, methadone doesn’t provide its own command-line parsing; it’s simply a proxy to OptionParser, along with a few helper methods. Let’s talk about the helper methods first, since they define the structure of your executable’s source code. All of methadone’s helper methods and features can be brought into your app by including the Methadone::Main module. Immediately after that, make a call to the main method, which takes a block.

This block should be the logic of your application. It can access the parsed command-line options via the variable options, which is a simple Hash. Command-line arguments will be passed into the block, so define your block to accept whatever arguments you need. The block given to main is executed when you call another helper method, go! . go! parses the command line and then calls your block with the remaining unparsed command-line arguments. It handles exceptions thrown from your block, messaging the user and exiting accordingly, as well as effectively replacing the begin/rescue code we were using in the original db_backup.rb. The structure is as follows:

Between our call to main and our call to go! , we can declare our user interface. Because we’ve included Methadone::Main, we have access to the variable opts, which is an instance of OptionParser. We could then declare options as normal, like so:

methadone provides a helper method to make this idiomatic use of OptionParser more compact. The on method accepts all of the same arguments as the on method of OptionParser, but it takes care of extracting the value and placing it in the options hash. The following code is identical to the call to opts.on shown earlier:

Since opts is just an OptionParser, we could create our banner as normal; however, methadone takes care of this for us. We simply need to provide the missing bits of information, which can be done via the description and arg helper methods:

arg also does a bit of sanity checking for us. Since we’ve declared our argument as :required, methadone will raise an error if this is omitted on the command line.

As for using our external configuration for defaults, methadone doesn’t support this directly, so, as with trollop, we have to do things a bit more manually. Since methadone creates the options hash, we have to slightly change how we configure it with our defaults, and we then have to explicitly set the defaults from the config file, as shown here:

methadone removes quite a bit of boilerplate when using OptionParser in an idiomatic way but, since it’s just a proxy, still allows you complete access to everything that OptionParser can do. For many common command-line apps, this can cut down on a lot of code and let you get things up and running much more quickly. methadone also provides some additional Cucumber steps that can be used with Aruba to get better test coverage of your app.

Like trollop, however, we no longer have easy access to a Hash of options to manipulate, and methadone provides no assistance with our external configuration. Unlike trollop, methadone is available only as a gem, so any app that uses it will need the methadone gem installed.

main

main[56] describes itself as “a class factory and DSL for generating command-line programs real quick.” It is designed around unifying all input to your app, be it arguments, command-line options, or environment variables, and it provides an easy-to-understand, if verbose, syntax for describing your command-line interface.

When declaring your user interface with main, you indicate where each bit of input you require will come from, and main handles retrieving it from the right place. You can then access it all from the variable params, and your code doesn’t need to worry about where a particular bit of information came from. main will also generate documentation appropriate to these sources, which can be very handy if your app responds to environment variables. As we’ll see, this gives you exactly one way to access user input, which can serve to make your code more readable and extensible.

First let’s see the general outline of how db_backup.rb would like using main. Our entire app’s code goes into a block that’s given to the method Main . Inside this block, we can declare the options, arguments, and other input. Then, we call the method run , which takes a block containing the app’s main logic. Inside this block, params is available to provide access to any value the user provided as input, be it from options, the environment, or command-line arguments. Here’s the basic overview:

One thing to note is that params isn’t just a hash of the option values but a hash that maps the option names to instances of Main::Parameter. Instances of this class provide the method value , which provides the value provided by the user.

Next, we need to declare our user interface. main takes a different approach than methadone or trollop . Instead of trying to condense everything into one line of code, main provides a detailed, readable format. Inside the block given to Main , the methods option , argument , and environment allow you to declare an input to your program. These all take a block in which various methods can be called to describe the aspects of the input. Let’s look at one of our more complex options, --username:

Each line of code tells main one piece of information about the option. description is used to generate help text, argument allows us to specify that the flag’s argument is optional (or not), and default indicates the default value. Here, we’ve used the value option[:username], which we read from our external configuration, as the default value. main uses this in a manner similar to GLI, in that the default value will appear in documentation but will also be returned by params as the value if the user omits it from the command line.

Last in our block to option is the validate method, which takes a block. This block is given the value from the command line, and if the block evaluates to true, the option is accepted. Otherwise, it is rejected, and main will exit your app with an error. Since it takes a block, it’s a bit more powerful than OptionParser, although it’s slightly more verbose for simple cases.

You may have noticed that main provides only the method option , and not one for flags and another for switches. main treats these the same, so to accept a switch, we simply omit the argument call, as we’ve done here for our “end-of-iteration” switch:

You should also be aware that main does not support short-form options. Even if we declare the name of an option to be a single character, main will require it to be passed with two dashes (e.g., option(’i’) requires --i on the command line). Also, main doesn’t support negatable options, so we must declare both --gzip and --no-gzip separately, as we did with trollop.

As we mentioned, main allows you to specify arguments to your command-line app in the same fashion as with options. This turns out to be fairly handy, since main will check for required arguments, document them, and provide access to them via params. Here’s how we declare our single argument, database_name:

This looks similar in structure to our use of option and exemplifies main’s unified way of describing and accessing user input. When the user runs our app, params[’database_name’].value will provide the value the user specified on the command line.

Despite main’s limitations, the unified access is a useful design. It means we can change around where values come from, and it won’t affect our main logic. This can be quite handy when first developing your app, since you might not have a good sense of what should be a flag and what should be an argument. It’s also worth pointing out that main’s verbosity can be an advantage. Users unfamiliar with your app, or the tools used to create its UI, will more easily understand the code for a main-powered app than for one using trollop or methadone. If you happen to be the author of an internal command-line app that’s being used other developers or sysadmins, it can be very nice when they can enhance the app themselves, instead of bothering you for changes.

main, like trollop and methadone, is actively maintained. All three are fine alternatives to OptionParser, so use the one that feels right for you whenever you’re writing a simple command-line app.

In the next section, we’ll look at alternatives for making a command suite. We’ll see how main works well for these as well, and we’ll also look at thor, which is a popular tool for making command suites, as well as a library for making Ruby on Rails generators.

A1.2 Alternatives for Command Suites

We saw how GLI provides us with the tools to make an awesome command suite, but, like with OptionParser, you might want to do things differently. You may want to provide a much simpler application with less code, sacrificing some power for quicker delivery. In this case, thor is an excellent choice. You may, instead, not want to use a different tool for simple apps as you do for command suites. In this case, main is a good choice, especially if you are transitioning a simple app to a command suite. We’ll look at both of them here by reimplementing todo using each library.

main

Since we just looked at main in the previous section, let’s stay with it and see how it works for command suites. You may have noticed that the way in which GLI works is vastly different from how OptionParser works. main takes the complete opposite approach and works very similarly for both regular apps and command suites. main refers to commands as modes and provides the method mode to declare them. First let’s see how we declare our global options. It’s done just like with a simple app: inside the Main block:

Inside our mode block , we can declare options and arguments specific to that mode/command, like so:

The contents of our run block are pretty much the same as in the action block of the GLI version. The implementations of new and done are similar. One big difference in the main version of todo and the one we created using GLI is in how the user interface acts.

For example, there’s no formal differentiation between global and command-specific options. Global options can be specified anywhere on the command line and share the same namespace with command-specific options. If you’ll recall, todo used -f, when a global option, to indicate the to-do filename and used it as a command-specific option to new to indicate that the next task should be placed first in the tasklist. Since main doesn’t allow short-form options, this isn’t a problem in this case, but it’s a limitation to be aware of.

Although main provides a two-level help system, like the one we described in Chapter 3, Be Helpful, it works in a slightly different fashion than the help provided by most command suites. In a main-powered app, you get help for a command by issuing the -h or --help option or by giving it the argument help, like so:

main also doesn’t have direct support for external config files, like GLI does, so we’d need to read them in “by hand” as we’ve been doing in our simple command-line apps. Despite these limitations and nonstandard behavior, there’s a real advantage to using only one tool for all of your command-line apps. Aside from having to be knowledgeable in only a single tool, it’s also easy to transform a simple app into a command suite. Just add the necessary calls to mode .

thor

thor[57] is a library created by Yehuda Katz that, among other things, provides an easy way to make simple command-suite applications. thor also provides other features, such as support for Rails generators and the ability to install and manage tasks in your system, but it’s the command-suite support we’re interested in. thor is not as feature-filled as GLI but requires a lot less code to get a command suite up and running and can be very handy for getting something out quickly.

The thor gem provides a base class, Thor, that you extend to create your command suite. The methods of your subclass are the commands in your suite, with the arguments to those methods being their command-line arguments. Subclassing the Thor base class also provides access to several helper “macro”-style methods to let you describe your app and its commands, as well as define arguments. Thor provides the method start , which kicks off your app. Here’s the skeleton of a thor-powered app:

To declare global options, use the method class_option , which takes a string or symbol as the primary option name and then a hash of options. Here’s how we declare the global option for --filename, which declares where todo will find the task list:

The options do what they appear to do:

:aliases

This is a list of alternate forms for the option.

:default

This is the default value for the option, which is used both for documentation and to provide the default value in code when the user omits this option on the command line.

:desc

This is the description of the option, used to generate help text.

:banner

This is similar to GLI’s arg_name method; it is used to generate help text and name the argument accepted by this option.

To define a command, we simply define a method. The name of the method is the name of the command, and the name and number of arguments are the name and number of arguments to that command.

To describe the command and define command-specific options, the methods desc and method_option are called before we define our method. Here’s how we’d define the new command:

Note that, like main, thor makes no formal distinction between flags and switches. Here, we indicate that --first is a switch by using the :type option, with a value of :boolean. The values of our options will be available inside our methods via the options hash.

Note the format of the arguments to desc . The first argument is the invocation syntax (e.g., todo new task_names...), and the second argument is used for help text. Also, note that we’re using the “splat” style for our argument to the new method. This tells thor that new takes multiple arguments. If we’d omitted the asterisk and defined our method like def new(task_name), thor would notice that and accept only one argument.

As you can see, thor is incredibly simple. This simplicity comes at a trade-off, however. As with main, all options share a common namespace, meaning that we cannot have a global -f and an -f specific to new. This also affects the command-line invocation syntax. To invoke an app that uses thor, all switches and flags must come at the end of the command line. Consider this command-line invocation of todo that adds a task to a custom task list:

Our GLI version would add the new task to the file /tmp/foo.txt. The thor version prints out the help text and exits zero. If we move the option after the command, things get a bit more confusing:

thor views anything between the command name and the first option as the arguments. In this invocation, there is nothing between them, so task_names is empty in our new method, which kicks off the alternate flow to read tasks from the standard input. The correct invocation for our thor-powered app is as follows:

This may seem like a serious trade-off since thor apps don’t behave like common command suites, but don’t forget how simple thor is to use. If you need to whip up a basic command suite, thor makes it very easy, since you just need to make a class with a few methods in it.

A1.3 Other Relevant Libraries

As we mentioned, the tools we saw here are just a small part of the libraries available for writing command-line apps in Ruby. In this section, we’ll list some more that we don’t have space to delve into but that are still worth checking out.

Command-Line Parsers

We’ve seen a few command-line parsers already, but there are many more you can use. Many of these are similar to the tools we’ve seen, but they all have their own individual style and features, so if none of what we’ve seen really grabs you, please check these out:

choice ( https://github.com/defunkt/choice )

choice is a command-line parser for simple command-line apps that has a syntax similar to main. It provides typecasting, validation, automatic documentation, and all the other things you’d expect of a command-line parser.

commander ( https://github.com/visionmedia/commander )

commander can be used to create a command suite and has a very similar syntax to GLI.

cri ( https://github.com/ddfreyne/cri )

cri is a command-line parser that can create a simple command-line app or a command suite and uses a syntax that is a mix of main and GLI. Unlike these libraries, cri has support for nested commands, allowing for an invocation syntax like my_app command sub_command subsub_command args.

executable ( https://rubyworks.github.io/executable )

executable takes a novel and minimalist approach to command-line applications. To use it, simply mix in a module to a Ruby class, and that class becomes a command-line app. Its methods are used to determine the options, and the documentation comments for the methods are used for the help text.

Mixlib-CLI ( https://github.com/opscode/mixlib-cli )

Mixlib-CLI is maintained by Opscode, which maintains the popular chef system maintenance tool. Mixlib-CLI is for creating simple command-line apps and has a syntax similar to main, in that it is verbose yet readable.

optitron ( https://github.com/joshbuddy/optitron )

optitron is for making command suites and has a syntax similar to thor. optitron provides more validation of options and arguments and has a concise syntax for naming command-line arguments and documenting their defaults, via extracting the Ruby code for the methods that back the commands.

slop ( https://github.com/injekt/slop )

slop allows you to create simple command-line apps and has a syntax very similar to trollop. slop provides more validations and more features than you might find in OptionParser. slop also defaults switches to allow negative versions (e.g., --no-switch), which, as we know, makes for a more fluent user interface.

Libraries for Fancy User Interfaces

In Chapter 10, Add Color, Formatting, and Interactivity, we used terminal-table and rainbow to create a nonstandard user interface. Here are some additional tools that work differently or provide additional features, such as progress bars:

command_line_reporter ( https://github.com/wbailey/command_line_reporter )

command_line_reporter is designed around creating reports that output to the command line. It provides a comprehensive API around formatting data in tables (and is much more sophisticated than terminal-tables).

formatador ( https://github.com/geemus/formatador )

formatador is a general-purpose library for producing rich output to a terminal. It uses an HTML-like syntax to produce colored output but can also do some basic formatting such as indentation and overwriting (where a line of output is replaced without a linefeed moving it up one line). formatador can also display ASCII tables and progress bars.

highline ( https://github.com/JEG2/highline )

highline is a library for interacting with the user in a question-and-answer format (as opposed to readline, which is a full interactive prompt). You request input from the user by calling ask and can give the user output via the method say . There is support for type conversions and validation. This is an excellent library if your app must interact with the user to figure out what to do.

paint ( https://github.com/janlelis/paint )

paint allows you to produce colored output to the terminal, but, unlike rainbow, paint does not monkey-patch String. It provides a simple method, Paint , that handles everything. paint also allows you to specify your colors using RGB instead of simple names like “green.” This is handy for terminals that support 256 colors.

progress_bar ( https://github.com/paul/progress_bar )

progress_bar produces a progress bar in the terminal, complete with a percentage and an elapsed time.

term-ansicolor ( https://github.com/flori/term-ansicolor )

term-ansicolor is another library for producing colored output to the terminal. Unlike rainbow and paint, which color and style strings, term-ansicolor provides methods that turn colors and styles on and off, with a method clear that resets everything. This is closer to how the ANSI color codes actually work.

Testing Libraries Useful for Command-Line Apps

We’ve seen aruba and mocha for help in testing our command-line apps, but there are many other libraries for helping to test your application. Here are a few that are most useful when writing command-line apps:

construct ( https://github.com/devver/construct )

construct allows you to create temporary directory structures and files specifically for testing. This can be handy if your app is expecting to work within some sort of directory hierarchy. Instead of manipulating the actual system files, you can use construct to set things up for any situation you want to test, and everything gets cleaned up after your tests run.

FakeFS ( https://github.com/defunkt/fakefs )

FakeFS is similar to construct, in that it allows you to manipulate files during testing. FakeFS replaces Ruby’s internal file classes with its own and is somewhat more limited but can run faster since it doesn’t interact with the filesystem.

This concludes our whirlwind tour of alternatives for writing command-line apps. Our intention wasn’t to show you every library and tool there is but to demonstrate the wide variety of styles available in some of the more popular libraries. If you’d like to learn about more tools, try searching in http://github.com or http://rubygems.org to discover new libraries.

Footnotes