Take Control of the Mac Command Line with Terminal (2015)

Install New Software

With just the software Mac OS X includes (and perhaps a few shell scripts you write on your own or find on the Web), you can do a tremendous number of useful activities on the command line. But sooner or later you’re likely to encounter a task that requires a command-line program you don’t already have, which means you’ll need to find and install it yourself. (Admittedly, this is not for everyone, and if the next few paragraphs give you a headache, skip ahead to Command-Line Recipes and forget I ever mentioned installing your own software!)

Fortunately, the vast majority of command-line software created for Unix and Unix-like operating systems (such as the various Linux distributions) can run on your Mac too! (Refer back to What’s Unix? for the differences between “Unix” and “Unix-like.”) Tens of thousands of command-line programs are at your disposal! Just a handful of examples:

· alpine: An email client

· FLAC: An audio format converter

· lynx: A command-line Web browser (yes, really)

· pdftohtml: A program that converts—you’ll never guess!—PDFs to HTML format

· postgresql: A relational database manager

· wget: A tool for downloading files from the Web

Except… on the command line, it’s almost never as simple as downloading an application and running it. Because each Unix and Unix-like operating system is a bit different, in most cases, a given program must be compiled for the specific platform in question—that is, the raw source code (in a language such as C) has to be run through a program called a compiler to produce a binary file that will run on the target system. (In fact, compiling can be vastly more complex than this description suggests, but that’s the basic idea.)

So, if you have an interest in adding third-party command-line software to your Mac, you’ll first need the tools that are required to compile and install them. You can get them easily (see Use Command Line Tools for Xcode, next), and in the process gain a bunch of extra programs that may be useful to you on their own.

Next, you have a choice:

· If you’re a glutton for punishment (or want to see how things work), you can Install Unix Software from Scratch. (do it at least once, just for the experience.)

· If you’d like to make life easier for yourself, however, you can often use a special program called a package manager to do the heavy lifting of finding, downloading, and (if necessary) compiling the software you want (see Use a Package Manager). Package managers are way faster and more convenient than compiling software from scratch, although not every program you may want to install is available in that form.

Use Command Line Tools for Xcode

Let’s start with something simple: a free software package from Apple called Command Line Tools for Xcode. This collection includes nearly 100 new command-line programs, mostly intended to perform functions useful to developers but not needed by the typical Mac user. However, since you now know your way around the command line, you’re not a typical Mac user! And in order to install new command-line software, you’ll almost certainly need tools such as make (to build a set of binary files from their source files), which in turn relies on a compiler such as gcc.

Both of these programs and dozens of other development tools are in this set, as well as such goodies as:

· CpMac and MvMac: Versions of cp (copy) and mv (move) that preserve Mac OS X-specific metadata and resource forks

· GetFileInfo: A command-line program that does something similar to the Finder’s Get Info window

· git: The git version control system

· svn: The Subversion version control system

You can obtain and use these command-line tools with or without Xcode, Apple’s software development system. Xcode is a free download from the Mac App Store, but it’s about 2.5 GB in size and takes up much more space than that after it’s installed. If you already have Xcode on your Mac, you can add the Command Line Tools with this command:

xcode-select --install

Follow the prompts to complete the installation.

If you don’t have Xcode and don’t want to bother with it, you can download the Command Line Tools separately (less than 200 MB). The catch is that you have to be registered as an Apple Developer—but even if you don’t want to pay $99 per year to join the Mac Developer Program, you can register for free to get access to Xcode and other tools.

Once you’ve done that, go to Downloads for Apple Developers, sign in with your Apple ID, and then download the version of Command Line Tools that corresponds to your version of Mac OS X. Double-click the installer and follow the instructions.

After you’ve installed the Command Line Tools for Xcode, you can immediately run any of the commands it includes (for a full list, enter ls /Library/Developer/CommandLineTools/usr/bin). You can also install software from other sources, as covered in the remainder of this chapter.

Install Unix Software from Scratch

Let’s suppose you’re looking for a command-line program that does X, and sure enough, you run across a Web page with what appears to be exactly the thing you want, a program I’ll call abc. But what the site offers is not a compiled binary for Mac OS X—it’s just a bunch of source files, so you have to build and install it yourself. How do you proceed? Although the procedure can vary greatly, I want to show you the basic steps involved in a typical installation.

But first, let me give you two key pieces of advice:

· Before you do anything else, check to see if the software is available via a package manager (such as Fink, Homebrew, or MacPorts, discussed ahead in Use a Package Manager)—this is often noted on Web pages where you can download Unix software. If so, installing the package manager, and then using that to install the abc program, is almost certainly the path of least resistance. I’d especially recommend using a package manager if you plan to install a different version of something that’s included with Mac OS X, such as PHP or Apache, because compiling your own and installing it manually could lead to unexpected conflicts.

· Look for installation instructions. In the vast majority of cases, the developer lists the exact steps to follow (sometimes, even including the download step), and if there are any variations for particular operating systems, they’re often included in these instructions. When in doubt, do exactly what the developer says.

Having read and followed many such instructions myself, I can tell you that they usually involve this sequence: download, configure, make, and make install. I explain those (and a couple of additional important steps) next.

Download

If you’re using a Mac OS X Web browser to locate the software you want to install, you can click a link to download it just as you would any other file. Once you’ve done that, you might want to move the downloaded file out of your Downloads folder to somewhere more convenient, but that’s up to you.

On the other hand, if you already have a Terminal window open, you can download software directly to your current directory, using the curl command and the URL. (If you don’t see the URL but just a link, you can right-click (Control-click) the link and choose Copy Link to put the URL on your clipboard.) To download the file, type curl -O (that’s an uppercase o, not a zero) followed by a space and the URL, as in:

curl -O http://some-web-site.com/something/abc-1.2.3.tgz

In this example (as very often occurs), the file that downloads includes the name of the program (abc) and a version number (1.2.3).

Tip: For more on using curl, see the recipe Download a File.

Decompress

Because command-line software often includes many source files that must be compiled to make the final product, they’re typically archived into a single file (often using a program called tar, for “tape archive”) and then compressed (often using a program called gzip). The resulting file usually ends in .tgz or .tar.gz. (I hasten to point out that there are many other ways to archive and compress files, and thus many other extensions in use; this is just an example.)

If you’ve downloaded the file using a Mac OS X Web browser such as Safari, it may be decompressed automatically, at which point you’ll end up with a folder (such as abc-1.2.3) in your Downloads folder.

If not, open a Terminal window, navigate to the folder containing the downloaded file, and enter (substitute the actual filename, of course):

tar -zxvf abc-1.2.3.tgz

If the file ends in .bz2, use this instead:

tar -jxvf abc-1.2.3.tar.bz2

At this point you’ll have a folder (such as abc-1.2.3) containing the files you need to work with.

Read “Read Me”

Now stop for a moment. Look through the files in that folder (either in the Finder or on the command line, using the tools you’ve already learned about in this book, such as cd). You will very likely find one or more files with names like README or INSTALL. These contain information about the program (README) and how to install it (INSTALL). They’re invariably plain text files that you can open in a text editor (TextEdit, BBEdit, nano, or whatever) or view using a program such as less or cat. In any case, read them. They’ll contain important instructions, and whatever they say takes precedence over what I tell you here!

One of the important things you might discover in a README file (or on the Web) is that the software you’re trying to install has certain dependencies—that it, it could rely on another program (or a library, which supplies features that any program can tap into) which must already be installed before the program will work. And that dependency might, in turn, have other dependencies. Working through those can sometimes be a long and frustrating process, which a reason to consider using a package manager when possible (see Use a Package Manager).

Configure

One of the instructions in the README or INSTALL file should tell you whether or not you need to perform a configuration step. This isn’t always necessary, and when it is, sometimes the preferred method is to edit a text file with information about your system. But more often than not, the step you take at this point is to run a script called configure. Assuming you’re in the same directory as the configure script, you do it like this:

./configure

The job of the configure script is to create a file called a makefile, which in turn contains all the instructions needed to compile the program for your particular computer. In most cases, configure doesn’t require any interaction; you just run it and move on to the next step.

Make

So, that makefile you just made in the last step with the configure command? Here’s where you use it. Assuming once again that you’re in the directory where the software resides, simply enter:

make

That’s it. The make command follows the instructions in the makefile to compile binary files for your Mac from the source files provided. This process may take anywhere from less than a second to many minutes or more, depending on the complexity of the software. You’ll probably see messages in Terminal as the build progresses. You’ll know the process is done when you see your command prompt again.

Make Install

Like Mac OS X apps, command line programs sometimes require lots of components to exist in specific places, beyond the executable file itself. Now that you have created all those components with the makecommand, it’s time to put them in the right locations and assign the proper permissions. To do so, enter:

sudo make install

Even for large, complex installations, the make install command is usually quite speedy. Once it has finished, you can run your newly installed software just as you would any other command line program.

Use a Package Manager

Now that you know the manual way to install command-line software, let’s look at a simpler approach: using a type of software known as a package manager. This whole rigamarole of figuring out what dependencies a given program has; downloading, configuring, making, and installing all of them; and then downloading, configuring, making, and installing the program you want, can all be automated into a single-step process. That’s what package managers do—they handle all the tedious details for you.

In most cases, package managers will download and install prebuilt binaries of the software you’re interested in (as well as any dependencies), which saves time, disk space, and hassle. If a binary isn’t available, if the latest available binary is out of date, or if there’s some complicated reason why it’s better to compile a particular program on your own Mac, the package manager can still do all that for you.

And, although not every command-line program you could want is available via a package manager, many thousands of them are, including all the most popular tools and programs.

Tip: As a reminder, you’ll need to have installed the Command Line Tools for Xcode before installing or using a package manager.

I’m aware of five reasonably full-featured package managers for Mac OS X, of which three (Fink, Homebrew, and MacPorts) are distinctly more popular than the other two (Pkgsrc and Rudix). And, of the three “cool kids,” almost anyone will tell you that the real contest these days is between the venerable MacPorts and newcomer Homebrew. I’ll say a bit about each package manager, but to some extent, you can’t make a bad choice; as long as the one you use offers the package that you’re interested in, it’ll be way easier than starting from scratch.

As you choose a package manager, keep these tips in mind:

· Pay attention to where on your disk the package manager stores binaries, and whether you have a choice in the matter. There are good reasons to choose any of several locations, but some of them are controversial (I’ll give an example when I talk about Homebrew).

· Whichever location your package manager uses for binaries, it must be included in your PATH for the software to operate correctly. That’s one advantage of Homebrew’s use of /usr/local/bin—that’s already in your PATH by default. To make sure the binary location is in your PATH, follow the steps in Modify Your PATH.

· Package managers differ in how they treat dependencies. Some try to supply all their own dependencies, while others rely as much as possible on programs and libraries included with Mac OS X. The former approach can take longer, use more space, and leave you with duplicates of certain programs. But the latter approach could break your programs when Apple updates OS X and in so doing removes a dependency (or supplies an incompatible version). There’s no right answer here, just different approaches to weigh.

· Under some circumstances, it might be possible to use more than one package manager at the same time, but I recommend against it. If you should happen to install the same software with each of two package managers, it’ll be hard to predict which one runs when you enter the program’s name (it’s the one whose path happens to be listed first in your PATH), and dependencies could get complicated.

With that background, here’s an overview of Mac package managers. (To download and install any of them, follow the instructions provided on their respective Web sites.)

Fink

Fink is the oldest package manager for the Mac, having first appeared in 2000. It’s based on a package manager for Debian Linux called apt-get, and as of March 2015 it had over 11,000 supported packagesplus another 11,000 or so that are outdated and no longer maintained. Fink tends to install its own dependencies rather than relying on software included with Mac OS X. It creates and uses the directory /sw by default.

Here are examples of how you might use Fink:

· Show all packages Fink can install:
fink list

· See if a particular program (lynx in this example) is available:
fink list lynx

· Update Fink’s listing of available packages:
sudo apt-get update

· Install the lynx package:
sudo apt-get install lynx

Homebrew

The new kid on the block, Homebrew, has made a big splash in just a few years because it’s modern, straightforward, and easy to use—it has a lot less baggage and clutter than Fink and MacPorts. On the other hand, because it’s relatively new, it also has fewer packages—just over 3,000 as of early 2015. Speaking of which, Homebrew doesn’t use the term “packages”; instead, it’s riddled with beer-brewing metaphors. A given program is offered either as a formula (instructions to download and compile the software) or as a bottle (a compiled binary).

Homebrew is written in Ruby, and specializes in tools of use to Ruby on Rails developers. It relies on existing Mac OS X software when possible, making it less complex than Fink or MacPorts, but with a greater danger of problems after upgrading Mac OS X. It does not use sudo for any of its work, making it less risky to use than other package managers.

However, unless you expressly specify a location, Homebrew takes over your /usr/local directory (and uses /usr/local/bin for the binaries it installs), which could be considered a misuse of that location’s intended purpose, and which might conflict with other software you’ve installed there by hand. Among other issues, that location makes it harder to remove Homebrew and all its installed binaries without also removing software that got in that directory in some other way.

In addition, it changes the ownership of that entire directory to you, the current user. That’s fine if you’re the only user of your Mac, but on a Mac with multiple users, other users may be unable to access that directory or run software in it—even if that software wasn’t installed by Homebrew. Conversely, if you had already created that directory and installed other software there that requires root ownership, Homebrew may display error messages because it really wants everything in that directory to have your username as the owner.

Some usage examples:

· Show all packages Homebrew can install:
brew search

· See if a particular program (lynx in this example) is available:
brew search lynx

· Install the lynx package:
brew install lynx

All things considered, Homebrew is probably the best package manager to try if you just want to dip your toes in, or install a few random programs, because the learning curve is so gentle. (And, as I said, it’s great for Ruby on Rails developers.) Otherwise, my top choice would be our next contender: MacPorts.

MacPorts

MacPorts started life in 2002 as DarwinPorts, and is based on the Ports system for BSD (which is appropriate since Mac OS X’s Unix layer is itself based on BSD). It now has the largest selection of packages (called ports) available—over 22,000. MacPorts uses the /opt/local directory by default. Unlike Fink, it relies as much as possible on programs and libraries already installed as part of Mac OS X.

The MacPorts syntax should look familiar by now:

· Show all packages MacPorts can install:
port list

· See if a particular program (lynx in this example) is available:
port search lynx

· Update MacPorts’ listing of available packages:
sudo port -d selfupdate

· Install the lynx package:
sudo port install lynx

If I had to pick just one package manager to recommend, it would be MacPorts. It’s not the easiest to use (the documentation goes on forever), but it’s solid and has a thorough library.

Pkgsrc

Unlike all the other package managers listed here, pkgsrc works on virtually every Unix and Unix-like operating system. As such, it might be a good choice if you also use it on another platform, but it’s less tailored to the specific needs and preferences of Mac users. Pkgsrc defaults to using either the /usr/pkg or the ~/pkg directory, depending on which installation mode you use. It currently offers over 12,000 binary packages.

Some syntax examples:

· Show all packages pkgsrc can install:
pkgin avail | wc -l

· See if a particular program (lynx in this example) is available:
pkgin search lynx

· Install the lynx package:
sudo pkgin -y install lynx

Rudix

Whereas Homebrew is written in Ruby, Rudix is written in Python, so it may be particularly attractive to Python developers. It has the smallest selection of packages by far—less than 300—but of course if that selection includes all the ones you care about, that’s not an issue. Rudix offers self-contained packages with all dependencies included, except those provided by Mac OS X libraries. It uses the /usr/localdirectory by default, just like Homebrew, but at least it doesn’t change the ownership of that directory. On the downside, that means you’ll have to use sudo to run the software Rudix installs.

Here are some example commands:

· Show all packages Rudix can install:
rudix search

· See if a particular program (lynx in this example) is available:
rudix search lynx

· Install the lynx package:
sudo rudix install lynx