Build Awesome Command-Line Applications in Ruby 2: Control Your Computer, Simplify Your Life (2013)

Chapter 6. Make Configuration Easy

In the previous chapter, we learned how the design decisions we make provide direction to our users about how to use our apps. But, what about advanced users, who use our apps regularly but in unusual ways? Can we accommodate their requirements without sacrificing the usability we’ve worked so hard to build into our apps?

The answer is yes, and this chapter is about making that happen via external configuration, which is to say configuration that isn’t part of the app but that the app can access at runtime (often referred to as rc files by old-school UNIX hackers; see The .rc Suffix for the history of that name).

We’ll see how you can use a simplified text format called YAML to store and manage this configuration and how easy it is to access in your app via the Ruby standard library. We’ll then see how this applies to a command-suite application, finishing up with a discussion about how the use of configuration files affects your application design, along with some approaches you can take to keep your app easy to use and maintain.

6.1 Why External Configuration?

In Chapter 5, Delight Casual Users, we talked about making uncommon or dangerous things difficult to use on purpose. It might seem that the simplest solution to allowing power users to have easy access to these features is to make them easy to use. This is not the case and will result in a more complex “kitchen-sink” application that tries to do everything.

Our apps have a purpose, and the way we design them is our opinion on how they can best realize that purpose. This “opinion” is what allows new users to understand and use our app and what makes our app easy to use for the majority of users. We want to maintain this ease of use for everyday users while not complicating the work of power users.

The way to do this is to externalize aspects of our application’s behavior into a configuration file. This way, power users can adjust things how they would like. External configuration gives us the best of both worlds; typical users get the ease of use we’ve designed in, and power users, with a bit of up-front work, get to use the app just as simply but get their desired behavior.

6.2 Reading External Configuration from Files

External configuration allows a user to control the behavior of an app without using command-line options and arguments, in a more transparent and persistent way. The intent is to give users the ease of use provided by the default behavior but customized for their needs.

As an example, suppose Bob, our sysadmin, is using db_backup.rb to test the effect of changing the settings of a MySQL server. Bob wants to see how these settings affect the speed with which db_backup.rb can complete a backup. Bob is not particularly interested in the backup file that results, only the speed with which it’s produced. To do this, Bob will need to run db_backup.rb like so:

For Bob to type --force, along with his username and password every time he runs the application, is tedious. We want Bob to be able to create a configuration file and get the same behavior with just this invocation:

Currently, the default values for --username and --password are nil, and the default for --force is false. We want to allow Bob to override these defaults via an external file that our app will read at start-up. The canonical format for that in Ruby apps is YAML.

Using YAML as a Configuration File Format

YAML (which is a recursive acronym for YAML Ain’t Markup Language) is a text-based format that’s useful for storing structured data. Ruby includes a built-in module (naturally called YAML) that makes it very easy to translate YAML into Ruby objects and back again.

As an example, here’s what db_backup.rb’s default options look like in our code:

Why Not XML?

XML is a more widely used format than YAML for configuring software. Almost every piece of open source Java software is configured with XML. Although YAML is quite prolific in the Ruby community, there’s another reason we’re recommending it here: human-friendliness. Here’s Bob’s config file in XML:

As you can see, there’s more syntax/markup than there is data. The YAML version has less than half the amount of “noise” compared to the XML version. The result is a file that is very clear, easy to read, and easy to edit. This is the exact use case we want to enable with external configuration files.

We’re not saying that YAML is better than XML in every situation, but it’s far superior as a configuration file format, especially for simple applications like command-line apps.

Here’s how YAML would serialize them:

It looks almost like plain text; there’s not much syntax there at all. The file starts with three dashes (which is simply a requirement of the format) and is then made up of key-value pairs. A key’s name starts at the first column and ends at the first colon (obviously not counting the colon in the first column). When Ruby deserializes these keys, since they start with colons, it will turn them into Symbols instead of Strings. The values will get translated appropriately, as well. In this case, both values are booleans, but strings and numbers can be used and will be converted to the appropriate type. (YAML supports much richer encodings than just key-value pairs. The spec on http://yaml.org has a complete description, but, for our purposes, all we need to know are the basics.)

What we’d like to do is allow Bob to create a file like so:

When he runs db_backup.rb, we’d like it to read this file and use it for its default option values, replacing the built-in ones.

Reading Defaults from a YAML-Based Configuration File

To allow Bob to override db_backup.rb’s built-in defaults, we’ll need to replace them before we parse the command line. To do that, we’ll need to add a new step after the built-in defaults are set but before we start parsing. This step will read our configuration as a Hash and merge it with our defaults, overriding any defaults in our app with the user’s external configuration. Before doing this, we need to figure out where this file comes from.

As we discussed in Chapter 5, Delight Casual Users, configuration files should go in the user’s home directory and be “hidden” (i.e., their names should be preceded by a period). We’ll use the double-suffix rc.yaml to indicate that this file is configuration (rc) but that it’s structured as YAML (yaml).

Once we’ve checked that the file exists, we use the method YAML.load_file to deserialize the file into our default options Hash:

This is the only code we need to change; the defaults Bob configures override the built-in ones, but Bob can still override these on the command line.

As long as Bob places his configuration file in ~/.db_backup.rc.yaml, he no longer needs to specify anything on the command line. Although he’s using db_backup.rb in an unusual way, it’s still easy for him to get his work done.

YAML is very readable and editable, but it might be hard for the user to create this file correctly. The user might not know that the first line must have three dashes or that the options must be preceded by colons. Keeping with our goal of being helpful, we should create this file for the user if it doesn’t exist.

Generating a YAML Configuration File for Users

By creating a configuration file for users automatically, we keep them from having to get the YAML syntax just right and understanding what options are available. What we want to do is create a standard configuration file that has all of the available options, each set to their default value, even if that default is nil.

Since we’re already checking for the existence of the config file, we can add an else condition to create it. We’ll also need to be explicit about options whose default value is nil. Although our code won’t care if an option is omitted from the options Hash or mapped explicitly to nil, having each option mapped to a value will ensure each option shows up in the resulting YAML file.

Note that we let the user know we created this file, and we did so on the standard error stream, where such messages should go. Now, when the user runs db_backup.rb the first time, the following configuration file will be generated:

The user can easily see what configuration options are available and how to format them. It’s also worth pointing out that the ability to control the app via configuration makes it doubly important that our switches are negatable (i.e., have the [no-] form). Thinking about Bob and his unusual use of db_backup.rb, if he wanted to do a normal backup and there was no --no-force switch, he’d have to manually edit his config file.

We’ve seen how easy it is to generate this file, but should we? If your app has a lot of options (making it more likely to be configured externally), you should generate this file if it doesn’t exist. Still, you may not want your app writing files that the user didn’t request to be written. In this case, include a description and example of the format and options in the man page.

Configuring defaults for a command-line app is straightforward, and with just a few lines of code, we have everything we need. What about command suites?

6.3 Using Configuration Files with Command Suites

Suppose we’ve enhanced our todo app to sync our tasks with an external task management service like JIRA.[39] We created todo as a lightweight replacement for such systems, but we still might want to send tasks to and from this system to facilitate collaboration with others on our team. We’d need at least three new global options: a URL where JIRA is running, a username, and a password to access it.

Suppose further that the new command will require the user to provide a group name to allow JIRA to properly file the ticket (a group might be something like “corporate web site” or “analytics database”). Let’s assume we’ve added these features, as shown in this sample help output:

While this is a great feature that we’ll use frequently, it’s made our app a bit cumbersome to use:

We might not even provide this feature because of the complexity of the command-line syntax. External configuration solves this usability problem. Just as we did with Bob, if we can make these options configured in a file that todo will read, we don’t have to specify them every time. Our JIRA server location, name, password, and group are probably not going to change all that often, so these are perfect candidates for us to configure externally.

The flattened structure we’ve encoded in YAML thus far isn’t going to work. Bob configured db_backup.rb with a simple list of key-value pairs. For a command suite, different commands can use options of the same name, or a global option might be the same name as a command-specific one (as is the case with -f; it’s used as a global option as well as an option for new).

Fortunately, YAML can handle this situation easily. We’ve seen only the most basic YAML format; YAML can store almost any Ruby object, including a slightly deeper Hash. Suppose we think of our options as a hierarchy; at the top are all the global options. Each command creates its own level in the hierarchy containing options specific to that command. In YAML, we can store it like so:

YAML doesn’t ignore the indentation but rather uses it to re-create Ruby objects. The indented key-value list underneath commands is treated as a Hash mapped to by the string “commands.” It’s still easy to read and write by the user but rich enough to hold all of our options in code. Deserializing this into Ruby gives us the following Hash:

Using a Hash like this, we can easily tell which options are which. Applying the user’s external configuration is the same as we’ve already seen with db_backup.rb. We could then use these values with GLI’s default_value method to let the user change the defaults via external configuration, as such:

GLI actually can do this for us, with just one line of code. Instead of choosing a file format, adding the code to parse the file, and then applying it to each global and command-specific option, GLI bakes this concept in.

If you call the config_file method, giving it the path to the config file, GLI will do two things. First, it will look for that file in the nested YAML format we described earlier and apply its contents as defaults for all options. Second, it will create a new command for your command suite named initconfig that will generate the config file (if it doesn’t exist), initialized with the values of any options you specify on the command line when calling initconfig.

Let’s add this feature to todo and see how it works. First we call config_file :

Then we invoke todo using the new initconfig command:

We can then see the config file that was generated:

We can now use todo’s new feature without any of the complexity on the command line:

As an aside, we can see that our short-form-only options make things hard for the user. :p isn’t a very clear configuration option; the user will have to get help on the new command to figure out what it does. Compare that to our new global option :username, which states exactly what it is.

So far, we’ve used configuration to control the default values of command-line options. Should configuration control other aspects of our command-line apps? In short, no. We’ll explain why in the next section.

6.4 Design Considerations When Using Configuration

We stated that the configuration files for our command-line apps should not contain any configuration beyond default values for command-line options. This may sound limiting, but it’s a design constraint that will lead us to make our applications better.

Suppose we wanted to allow a power user to configure the level of compression that gzip uses. gzip takes an option of the form -# where # can be any number from 1 to 9, with 1 indicating a fast but weaker compression and with 9 being a slower but better compression. We might be tempted to put this in our configuration file:

Why should this option be limited to just users of the configuration file? Why not make it a full-fledged command-line option, available to anyone? We can easily signify it as an “advanced” option by using only a long-form name like --compression.

You should start to think of your command-line options as the only way to affect the behavior of your app. With sensible defaults and the ability to read in external configuration, it won’t ultimately matter how many options your app takes. Think of the configuration file as nothing more than a way to specify defaults for the command-line options. This leads to a very clear and easy-to-understand application design principle: give users the power to control and script the entire application at the command line, but provide a configuration file to allow anyone to override them.

6.5 Moving On

In this chapter, we learned how to make our applications configurable so that they can be easy to use for as many users as possible. We saw that driving our application design around command-line arguments and making those the vocabulary of our configuration can result in a clean, comprehensible application.

Everything we’ve learned up to this point has been to get our apps running well and ready for use by others. What we haven’t talked about is how to get our app into the hands of its users. This is what we’ll talk about next: making it painless for our users to install our apps.

Footnotes