Build Awesome Command-Line Applications in Ruby 2: Control Your Computer, Simplify Your Life (2013)

Chapter 1. Have a Clear and Concise Purpose

You need to solve a problem. It might be that you need two systems to talk to each other that weren’t designed for it. Or you may need to run some automated yet complex task periodically. Or, you may want to build simple productivity tools to help you work. This is where the command line shines, and these are the kinds of problems you’ll learn to solve in this book.

Although it may seem obvious that a focused, single-purpose app is more desirable than one with a “kitchen sink” full of features, it’s especially important for command-line apps. The way in which command-line apps get input, are configured, and produce output is incredibly simple and, in some ways, limiting. As such, a system of many single-purpose apps is better than a system of fewer (or one) complex apps. Simple, single-purpose apps are easier to understand, are easier to learn, are easier to maintain, and lead to more flexible systems.

Think of your command-line tasks as a set of layers: with the basic foundation of the standard UNIX tools, you can create more complex but still focused command-line apps. Those can be used for even more complex apps, each built on simpler tools below. The popular version control system git follows this design: many of git’s commands are “plumbing” and are not intended for regular use. These commands are then used to build “porcelain” commands, which are still simple and single-purpose but are built using the “plumbing.” This design comes in handy because, every once in a while, you need to use the “plumbing” directly. You can do this because git was designed around tools that each have a clear and concise purpose.

This chapter will set the stage for everything we’ll be learning in the book. We’ll look at two common problems and introduce two command-line apps to solve them. As a means of demonstrating more clearly what we mean by having a “clear and concise purpose,” each problem-solving app will get an iteration in this chapter. The first version of each app will be naive and then quickly revised to be more single-purpose, so we can see firsthand the level of function we want our apps to have.

1.1 Problem 1: Backing Up Data

Suppose our small development team is starting work on our company’s flagship web application. This application is heavily data-driven and highly complex, with many features and edge cases. To build it, we’re going to use an Agile methodology, where we work in two-week “sprints.” In each sprint, we’ll have a list of “user stories” representing the work we’re doing. To officially complete a user story, we’ll need to demonstrate that story functioning properly in a shared development environment.

To be able to demonstrate working features, we’ll have a set of databases with specially chosen data that can simulate all of our edge cases and user flows. Setting up this data is time-consuming because our app is complex, so even though this data is fake, we want to treat it like real production data and back it up. Since we’re constantly changing the data as we work, we want to save the state of each database every single day of the current iteration. We also want to keep a backup of the state of each database at the end of every iteration. So, if we’re on the fifth day of our third iteration, we want to be able to access a backup for iterations 1 and 2, as well as backups for the first four days of the third iteration.

Like with most teams, at our company, we can’t rely on a system administrator to back it up for us; we’re a fledgling start-up, and resources are limited. A command-line app to the rescue! We need an app that will do the following:

· Do a complete dump of any MySQL database

· Name the backup file based on the date of the backup

· Allow the creation of our “end-of-iteration” backup, using a different naming scheme

· Compress the backup files

· Delete backups from completed iterations

Let’s take a quick stab at it. We’ll set up a Hash that contains information about all the databases we want to back up, loop over it, and then use Ruby’s backtick operator to call mysqldump, followed by gzip[11]. We’ll also examine the first argument given to our app; if it’s present, we’ll take that to mean we want to do an “end-of-iteration” backup. Here’s what our initial implementation looks like:

If you’re wondering what’s going on the very first line, see Shebang: How the System Knows an App Is a Ruby Script. Notice how we use ARGV, which is an Array that Ruby sets with all the command-line arguments to detect whether this is an “end-of-iteration” backup. In that case, we assume that whatever the argument was should go into the filename, instead of the current date. We’d call it like so:

Shebang: How the System Knows an App Is a Ruby Script

Compiled programs include information in the executable file that tells that operating system how to start the program. Since programs written in a scripting language, like Ruby, don’t need to be compiled, the operating system must have some other way to know how to run these types of apps. On UNIX systems, this is done via the first line of code, commonly referred to as the shebang line .[12]

The shebang line starts with a number sign (#), followed by an exclamation point (!), followed by the path to an interpreter that will be used to execute the program. This path must be an absolute path, and this requirement can cause problems on some systems. Suppose we have a simple app like so:

For this app to work on any other system, there must be a Ruby interpreter located at /usr/bin/ruby. This might not be where Ruby is installed, and for systems that use RVM (an increasingly high number do so), Ruby will never be available in /usr/bin.

To solve this, the program /usr/bin/env, which is much more likely to be installed at that location, can be used to provide a level of indirection. env takes an argument, which is the name of a command to run. It searches the path for this command and runs it. So, we can change our program to use a shebang like so:

This way, as long as Ruby is in our path somewhere, the app will run fine. Further, since the number sign is the comment character for Ruby, the shebang is ignored if you execute your app with Ruby directly: ruby my_app.rb.

There are a lot of problems with this app and lots of room for improvement. The rest of the book will deal with these problems, but we’re going to solve the biggest one right now. This app doesn’t have a clear and concise purpose. It may appear to—after all, it is backing up and compressing our databases—but let’s imagine a likely scenario: adding a third database to back up.

To support this, we’d need to edit the code, modify the databases Hash, and redeploy the app to the database server. We need to make this app simpler. What if it backed up only one database? If it worked that way, we would call the app one time for each database, and when adding a third database for backup, we’d simply call it a third time. No source code changes or redistribution needed.

To make this change, we’ll get the database name, username, and password from the command line instead of an internal Hash, like this:

Now, to perform our backup, we call it like so:

It may seem like we’ve complicated things, but our app is a lot simpler now and therefore easier to maintain, enhance, and understand. To set up our backups, we’d likely use cron (which is a UNIX tool for regularly scheduling things to be run) and have it run our app three times, once for each database.

We’ll improve on db_backup.rb throughout the book, turning it into an awesome command-line app. Of course, automating specialized tasks is only one use of the command line. The command line can also be an excellent interface for simple productivity tools. As developers, we tend to be on the command line a lot, whether editing code, running a build, or testing new tools. Given that, it’s nice to be able to manage our work without leaving the command line.

1.2 Problem 2: Managing Tasks

Most software development organizations use some sort of task management or trouble-ticket system. Tools like JIRA, Bugzilla, and Pivotal Tracker provide a wealth of features for managing the most complex workflows and tasks, all from your web browser. A common technique when programming is to take a large task and break it down into smaller tasks, possibly even breaking those tasks down. Suppose we’re working on a new feature for our company’s flagship web application. We’re going to add a Terms of Service page and need to modify the account sign-up page to require that the user accept the new terms of service.

In our company-wide task management tool, we might see a task like “Add Terms of Service Checkbox to Signup Page.” That’s the perfect level of granularity to track the work by our bosses and other interested stakeholders, but it’s too coarse to drive our work. So, we’ll make a task list of what needs to be done:

· Add new field to database for “accepted terms on date.”

· Get DBA approval for new field.

· Add checkbox to HTML form.

· Add logic to make sure the box is checked before signing up is complete.

· Perform peer code review when all work is done.

Tracking such fine-grained and short-lived tasks in our web-based task manager is going to be too cumbersome. We could write this on a scrap of paper or a text file, but it would be better to have a simple tool to allow us to create, list, and complete tasks in order. That way, any time we come back to our computer, we can easily see how much progress we’ve made and what’s next to do.

To keep things single-purpose, we’ll create three command-line apps, each doing the one thing we need to manage tasks. todo-new.rb will let us add a new task, todo-list.rb will list our current tasks, and todo-done.rb will complete a task.

They will all work off a shared text file, named todo.txt in the current directory, and work like so:

We’ll start with todo-new.rb, which will read in the task from the command line and append it to todo.txt, along with a timestamp.

This is pretty straightforward; we’re using a comma-separated-values format for the file that stores our tasks. todo-list.rb will now read that file, printing out what it finds and generating the ID number.

Finally, for todo-done.rb, we’ll read the file in and write it back out, stopping when we get the task the user wants to complete and including a timestamp for the completed date as well:

As with db_backup_initial.rb, this set of command-line apps has some problems. The most important, however, is that we’ve gone too far making apps clear and concise. We have three apps that share a lot of logic. Suppose we want to add a new field to our tasks. We’ll have to make a similar change to all three apps to do it, and we’ll have to take extra care to keep them in sync.

Let’s turn this app into a command suite . A command suite is an app that provides a set of commands, each representing a different function of a related concept. In our case, we want an app named todo that has the clear and concise purpose of managing tasks but that does so through a command-style interface, like so:

The invocation syntax is almost identical, except that we can now keep all the code in one file. What we’ll do is grab the first element of ARGV and treat that as the command. Using a case statement, we’ll execute the proper code for the command. But, unlike the previous implementation, which used three files, because we’re in one file, we can share some code, namely, the way in which we read and write our tasks to the file.

Notice how the methods read_todo and write_todo encapsulate the format of tasks in our file? If we ever needed to change them, we can do it in just one place. We’ve also put the name of the file into a constant (TODO_FILE), so that can easily be changed as well.

1.3 What Makes an Awesome Command-Line App

Since the rest of this book is about what makes an awesome command-line app, it’s worth seeing a broad overview of what we’re talking about. In general, an awesome command-line app has the following characteristics:

Easy to use

The command-line can be an unforgiving place to be, so the easier an app is to use, the better.

Helpful

Being easy to use isn’t enough; the user will need clear direction on how to use an app and how to fix things they might’ve done wrong.

Plays well with others

The more an app can interoperate with other apps and systems, the more useful it will be, and the fewer special customizations that will be needed.

Has sensible defaults but is configurable

Users appreciate apps that have a clear goal and opinion on how to do something. Apps that try to be all things to all people are confusing and difficult to master. Awesome apps, however, allow advanced users to tinker under the hood and use the app in ways not imagined by the author. Striking this balance is important.

Installs painlessly

Apps that can be installed with one command, on any environment, are more likely to be used.

Fails gracefully

Users will misuse apps, trying to make them do things they weren’t designed to do, in environments where they were never designed to run. Awesome apps take this in stride and give useful error messages without being destructive. This is because they’re developed with a comprehensive test suite.

Gets new features and bug fixes easily

Awesome command-line apps aren’t awesome just to use; they are awesome to hack on. An awesome app’s internal structure is geared around quickly fixing bugs and easily adding new features.

Delights users

Not all command-line apps have to output monochrome text. Color, formatting, and interactive input all have their place and can greatly contribute to the user experience of an awesome command-line app.

1.4 Moving On

The example apps we saw in this chapter don’t have many aspects of an awesome command-line app. They’re downright awful, in fact, but we have to start somewhere, and these are simple enough and general enough that we can demonstrate everything we need to know about making an awesome command-line app by enhancing them.

In this chapter, we learned the absolute most important thing for a command-line app: have a clear, concise purpose that solves a problem we have. Next, we’ll learn how to make our app easier to use by implementing a more canonical command-line interface. As we work through the book, we’ll make refinement after refinement, starting our focus on the general users of our app, then focusing on power users, and then worrying about other developers helping us with our app, before finally finishing with tools and techniques to help us maintain the app.

Footnotes