The Shell - Using Fedora and Red Hat Enterprise Linux - A practical guide to Fedora and Red Hat Enterprise Linux, 7th Edition (2014)

A practical guide to Fedora and Red Hat Enterprise Linux, 7th Edition (2014)

Part II: Using Fedora and Red Hat Enterprise Linux

Chapter 5. The Shell

In This Chapter

The Working Directory

Your Home Directory

The Command Line

Standard Input and Standard Output

Redirection

Pipelines

Running a Command in the Background

kill: Aborting a Background Job

Filename Generation/Pathname Expansion

Builtins


Objectives

After reading this chapter you should be able to:

Image List special characters and methods of preventing the shell from interpreting these characters

Image Describe a simple command

Image Understand command-line syntax and run commands that include options and arguments

Image Explain how the shell interprets the command line

Image Redirect output of a command to a file, overwriting the file or appending to it

Image Redirect input for a command so it comes from a file

Image Connect commands using a pipeline

Image Run commands in the background

Image Use special characters as wildcards to generate filenames

Image Explain the difference between a stand-alone utility and a shell builtin

The introduction to the command line on page 119 described some of the advantages of using the command line over a GUI, how to use a terminal emulator, how to correct mistakes on the command line, and how to run some command-line utilities. This chapter takes a close look at the shell and explains how to use some of its features. It discusses command-line syntax and describes how the shell processes a command line and initiates execution of a program. This chapter also explains how to redirect input to and output from a command, construct pipelines and filters on the command line, and run a command in the background. The final section covers filename expansion and explains how you can use this feature in your everyday work.

The exact wording of the shell output differs from shell to shell: What the shell you are using displays might differ slightly from what appears in this book. Refer to Chapter 9 for more information on bash (the default shell under Fedora/ RHEL) and to Chapter 27 for information on writing and executing bash shell scripts.

Image Special Characters

Special characters, which have a special meaning to the shell, are discussed in “Filename Generation/Pathname Expansion” on page 165. These characters are mentioned here so you can avoid accidentally using them as regular characters until you understand how the shell interprets them. Avoid using any of the following characters in a filename (even though emacs and some other programs do) because they make the file harder to reference on the command line:

& ; | * ? ' " ' [ ] ( ) $ < > { } # / \ ! ~

Whitespace

Although not considered special characters, RETURN, SPACE, and TAB have special meanings to the shell. RETURN usually ends a command line and initiates execution of a command. The SPACE and TAB characters separate tokens (elements) on the command line and are collectively known aswhitespace or blanks.

Quoting special characters

If you need to use a character that has a special meaning to the shell as a regular character, you can quote (or escape) it. When you quote a special character, you prevent the shell from giving it special meaning. The shell treats a quoted special character as a regular character. However, a slash (/) is always a separator in a pathname, even when you quote it.

Backslash

To quote a character, precede it with a backslash (\). When two or more special characters appear together, you must precede each with a backslash (e.g., you would enter ** as \*\*). You can quote a backslash just as you would quote any other special character—by preceding it with a backslash (\\).

Single quotation marks

Another way of quoting special characters is to enclose them between single quotation marks: '**'. You can quote many special and regular characters between a pair of single quotation marks: 'This is a special character: >'. The regular characters are interpreted as usual, and the shell also interprets the special characters as regular characters.

The only way to quote the erase character (CONTROL-H), the line kill character (CONTROL-U), and other control characters (try CONTROL-M) is by preceding each with a CONTROL-V. Single quotation marks and backslashes do not work. Try the following:

$ echo 'xxxxxxCONTROL-U'
$ echo xxxxxxCONTROL-V CONTROL-U


Optional

Although you cannot see the CONTROL-U displayed by the second of the preceding pair of commands, it is there. The following command sends the output of echo (page 219) through a pipeline (page 158) to od (octal display; see the od man page) to display CONTROL-U as octal 25 (025):

$ echo xxxxxxCONTROL-V CONTROL-U | od -c
0000000 x x x x x x 025 \n
0000010

The \n is the NEWLINE character that echo sends at the end of its output.


Ordinary Files and Directory Files

Ordinary files, or simply files, are files that can hold documents, pictures, programs, and other kinds of data. Directory files, also referred to as directories or folders, can hold ordinary files and other directory files. For more information refer to “Ordinary Files and Directory Files” on page177.

The Working Directory

Image pwd

While you are logged in on a character-based interface to a Linux system, you are always associated with a directory. The directory you are associated with is called the working directory or current directory. Sometimes this association is referred to in a physical sense: “You are in (or working in) the zach directory.” The pwd (print working directory) builtin displays the pathname of the working directory.

login: max
Password:
Last login: Wed Oct 20 11:14:21 from 172.16.192.150
$ pwd
/home/max

Your Home Directory

When you first log in on a Linux system or start a terminal emulator window, the working directory is your home directory. To display the pathname of your home directory, use pwd just after you log in.

Image The Command Line

Command

This book uses the term command to refer to both the characters you type on the command line and the program that action invokes.

Command line

A command line comprises a simple command (below), a pipeline (page 158), or a list (page 162).

A Simple Command

The shell executes a program when you enter a command in response to its prompt. For example, when you give an ls command, the shell executes the utility program named ls. You can cause the shell to execute other types of programs—such as shell scripts, application programs, and programs you have written—in the same way. The line that contains the command, including any arguments, is called a simple command. The following sections discuss simple commands; see page 147 for a more technical and complete description of a simple command.

Syntax

Command-line syntax dictates the ordering and separation of the elements on a command line. When you press the RETURN key after entering a command, the shell scans the command line for proper syntax. The syntax for a simple command is

command [arg1] [arg2] ... [argn] RETURN

Whitespace (any combination of SPACEs and/or TABs) must separate elements on the command line. The command is the name of the command, arg1 through argn are arguments, and RETURN is the keystroke that terminates the command line. The brackets in the command-line syntax indicate that the arguments they enclose are optional. Not all commands require arguments: Some commands do not allow arguments; other commands allow a variable number of arguments; and still others require a specific number of arguments. Options, a special kind of argument, are usually preceded by one or two hyphens ().

Command Name

Usage message

Some useful Linux command lines consist of only the name of the command without any arguments. For example, ls by itself lists the contents of the working directory. Commands that require arguments typically give a short error message, called a usage message, when you use them without arguments, with incorrect arguments, or with the wrong number of arguments.

For example, the mkdir (make directory) utility requires an argument that specifies the name of the directory you want it to create. Without this argument, it displays a usage message (operand is another term for “argument”):

$ mkdir
mkdir: missing operand
Try 'mkdir --help' for more information.

Image Arguments

Token

On the command line each sequence of nonblank characters is called a token or word. An argument is a token that a command acts on (e.g., a filename, a string of characters, a number). For example, the argument to a vim or emacs command is the name of the file you want to edit.

The following command line uses cp to copy the file named temp to tempcopy:

$ cp temp tempcopy

Arguments are numbered starting with the command itself, which is argument zero. In this example, cp is argument zero, temp is argument one, and tempcopy is argument two. The cp utility requires at least two arguments on the command line. Argument one is the name of an existing file. In this case, argument two is the name of the file that cp is creating or overwriting. Here the arguments are not optional; both arguments must be present for the command to work. When you do not supply the right number or kind of arguments, cp displays a usage message. Try typing cp and then pressing RETURN.

Image Options

An option is an argument that modifies the effects of a command. These arguments are called options because they are usually optional. You can frequently specify more than one option, modifying the command in several ways. Options are specific to and interpreted by the program that the command line calls, not the shell.

By convention, options are separate arguments that follow the name of the command and usually precede other arguments, such as filenames. Many utilities require options to be prefixed with a single hyphen. However, this requirement is specific to the utility and not the shell. GNU long (multicharacter) program options are frequently prefixed with two hyphens. For example, ––help generates a (sometimes extensive) usage message.

The first of the following commands shows the output of an ls command without any options. By default, ls lists the contents of the working directory in alphabetical order, vertically sorted in columns. Next the –r (reverse order; because this is a GNU utility, you can also specify ––reverse) option causes the ls utility to display the list of files in reverse alphabetical order, still sorted in columns. The –x option causes ls to display the list of files in horizontally sorted rows.

$ ls
hold mark names oldstuff temp zach
house max office personal test
$ ls -r
zach temp oldstuff names mark hold
test personal office max house
$ ls -x
hold house mark max names office
oldstuff personal temp test zach

Combining options

When you need to use several options, you can usually group multiple single-letter options into one argument that starts with a single hyphen; do not put SPACEs between the options. You cannot combine options that are preceded by two hyphens in this way. Specific rules for combining options depend on the program you are running. The next example shows both the –r and –x options with the ls utility. Together these options generate a list of filenames in horizontally sorted rows in reverse alphabetical order.

$ ls -rx
zach test temp personal oldstuff office
names max mark house hold

Most utilities allow you to list options in any order; thus ls –xr produces the same results as ls –rx. The command ls –x –r also generates the same list.


Tip: The ––help option

Many utilities display a (sometimes extensive) help message when you call them with an argument of ––help. All utilities developed by the GNU Project (page 3) accept this option. Following is the help message displayed by the bzip2 compression utility (page 245).

$ bzip2 --help
bzip2, a block-sorting file compressor. Version 1.0.6, 6-Sept-2010.

usage: bunzip2 [flags and input files in any order]

-h --help print this message
-d --decompress force decompression
-z --compress force compression
-k --keep keep (don't delete) input files
-f --force overwrite existing output files
...
If invoked as 'bzip2', default action is to compress.
as 'bunzip2', default action is to decompress.
as 'bzcat', default action is to decompress to stdout.
...


Option arguments

Some utilities have options that require arguments. These arguments are not optional. For example, the gcc utility (C compiler) has a –o (output) option that must be followed by the name you want to give the executable file that gcc generates. Typically an argument to an option is separated from its option letter by a SPACE:

$ gcc -o prog prog.c

Some utilities sometimes require an equal sign between an option and its argument. For example, you can specify the width of output from diff in two ways:

$ diff -W 60 filea fileb

or

$ diff --width=60 filea fileb

Arguments that start with a hyphen

Another convention allows utilities to work with arguments, such as filenames, that start with a hyphen. If a file named –l is in the working directory, the following command is ambiguous:

$ ls -l

This command could be a request to display a long listing of all files in the working directory (–l option) or a request for a listing of the file named –l. The ls utility interprets it as the former. Avoid creating a file whose name begins with a hyphen. If you do create such a file, many utilities follow the convention that a –– argument (two consecutive hyphens) indicates the end of the options (and the beginning of the arguments). To disambiguate the preceding command, you can type

$ ls -- -l

Using two consecutive hyphens to indicate the end of the options is a convention, not a hard-and-fast rule, and a number of utilities do not follow it (e.g., find). Following this convention makes it easier for users to work with a program you write.


Tip: Displaying readable file sizes: the –h option

Most utilities that report on file sizes specify the size of a file in bytes. Bytes work well when you are dealing with smaller files, but the numbers can be difficult to read when you are working with file sizes that are measured in gigabytes or terabytes. Use the –h (or ––human-readable) option to display file sizes in kilobytes, megabytes, gigabytes, and terabytes. Experiment with the df –h (disk free) and ls –lh commands.


For utilities that do not follow this convention, there are other ways to specify a filename that begins with a hyphen. You can use a period to refer to the working directory and a slash to indicate the following filename refers to a file in the working directory:

$ ls ./-l

You can also specify the absolute pathname of the file:

$ ls /home/max/-l


Optional

Simple Commands

This section expands on the discussion of command-line syntax that starts on page 144.

A simple command comprises zero or more variable assignments followed by a command line. It is terminated by a control operator (e.g., &, ;, |, NEWLINE; page 341). A simple command has the following syntax:

[name=value ...] command-line

The shell assigns a value to each name and places it in the environment (page 1032) of the program that command-line calls so it is available to the called program and its children as a variable. The shell evaluates the name=value pairs from left to right, so if name appears more than once in this list, the rightmost value takes precedence. The command-line might include redirection operators such as > and < (page 153). The exit status (page 1029) of a simple command is its return value.

Placing a variable in the environment of a child

The following commands demonstrate how you can assign a value to a name (variable) and place that name in the environment of a child program; the variable is not available to the interactive shell you are running (the parent program). The script named echo_ee displays the value of the variable named ee. The first call to echo_ee shows ee is not set in the child shell running the script. When the call to echo_ee is preceded by assigning a value to ee, the script displays the value of ee in the child shell. The final command shows ee has not been set in the interactive shell.

$ cat echo_ee
echo "The value of the ee variable is: $ee"

$ ./echo_ee
The value of the ee variable is:
$ ee=88 ./echo_ee
The value of the ee variable is: 88
$ echo $ee

$


Processing the Command Line

As you enter a command line, the tty device driver (part of the Linux kernel) examines each character to see whether it must take immediate action. When you press CONTROL-H (to erase a character) or CONTROL-U (to kill a line), the device driver immediately adjusts the command line as required; the shell never sees the character(s) you erased or the line you killed. Often a similar adjustment occurs when you press CONTROL-W (to erase a word). When the character you entered does not require immediate action, the device driver stores the character in a buffer and waits for additional characters. When you press RETURN, the device driver passes the command line to the shell for processing.

Parsing the command line

When the shell processes a command line, it looks at the line as a whole and parses (breaks) it into its component parts (Figure 5-1). Next the shell looks for the name of the command. Usually the name of the command is the first item on the command line after the prompt (argument zero). The shell takes the first characters on the command line up to the first blank (TAB or SPACE) and then looks for a command with that name. The command name (the first token) can be specified on the command line either as a simple filename or as a pathname. For example, you can call the lscommand in either of the following ways:

$ ls

or

$ /bin/ls

Image

Figure 5-1 Processing the command line


Optional

The shell does not require the name of the program to appear first on the command line. Thus you can structure a command line as follows:

$ >bb <aa cat

This command runs cat with standard input coming from the file named aa and standard output going to the file named bb. When the shell recognizes the redirect symbols (page 153), it processes them and their arguments before finding the name of the program that the command line is calling. This is a properly structured—albeit rarely encountered and possibly confusing—command line.


Image Absolute versus relative pathnames

From the command line, there are three ways you can specify the name of a file you want the shell to execute: as an absolute pathname (starts with a slash [/]; page 181), as a relative pathname (includes a slash but does not start with a slash; page 182), or as a simple filename (no slash). When you specify the name of a file for the shell to execute in either of the first two ways (the pathname includes a slash), the shell looks in the specified directory for a file with the specified name that you have permission to execute. When you specify a simple filename (no slash), the shell searches through a list of directories for a filename that matches the specified name and for which you have execute permission. The shell does not look through all directories but only the ones specified by the variable named PATH. Refer to page 359 for more information on PATH. Also refer to the discussion of the which and whereis utilities on page 255.

When it cannot find the file, bash displays the following message:

$ abc
bash: abc: command not found...

Some systems are set up to suggest where you might be able to find the program you tried to run. One reason the shell might not be able to find the executable file is that it is not in a directory listed in the PATH variable. Under bash the following command temporarily adds the working directory (.) to PATH:

$ PATH=$PATH:.

For security reasons, it is poor practice to add the working directory to PATH permanently; see the following tip and the one on page 360.

When the shell finds the file but cannot execute it (i.e., because you do not have execute permission for the file), it displays a message similar to

$ def
bash: ./def: Permission denied

See “ls –l: Displays Permissions” on page 191 for information on displaying access permissions for a file and “chmod: Changes Access Permissions” on page 193 for instructions on how to change file access permissions.


Tip: Try giving a command as ./command

You can always execute an executable file in the working directory by prepending ./ to the name of the file. Because ./filename is a relative pathname, the shell does not consult PATH when looking for filename. For example, if myprog is an executable file in the working directory, you can execute it using the following command (regardless of how PATH is set):

$ ./myprog


Executing a Command

Image Process

If it finds an executable file with the name specified on the command line, the shell starts a new process. A process is the execution of a command by Linux (page 373). The shell makes each command-line argument, including options and the name of the command, available to the called program. While the command is executing, the shell waits for the process to finish. At this point the shell is in an inactive state named sleep. When the program finishes execution, it passes its exit status (page 1029) to the shell. The shell then returns to an active state (wakes up), issues a prompt, and waits for another command.

The shell does not process arguments

Because the shell does not process command-line arguments but merely passes them to the called program, the shell has no way of knowing whether a particular option or other argument is valid for a given program. Any error or usage messages about options or arguments come from the program itself. Some utilities ignore bad options.

Editing the Command Line

You can repeat and edit previous commands and edit the current command line. See pages 124 and 378 for more information.

Standard Input and Standard Output

Standard output is a place to which a program can send information (e.g., text). The program never “knows” where the information it sends to standard output is going (Figure 5-2). The information can go to a printer, an ordinary file, or the screen. The following sections show that by default the shell directs standard output from a command to the screen1 and describe how you can cause the shell to redirect this output to another file.

1. This book uses the term screen to refer to a screen, terminal emulator window, or workstation—in other words, to the device that the shell displays its prompt and messages on.

Image

Figure 5-2 The command does not know where standard input comes from or where standard output and standard error go

Standard input is a place a program gets information from; by default, the shell directs standard input from the keyboard. As with standard output the program never “knows” where the information comes from. The following sections explain how to redirect standard input to a command so it comes from an ordinary file instead of from the keyboard.

In addition to standard input and standard output, a running program has a place to send error messages: standard error. By default, the shell directs standard error to the screen. Refer to page 333 for more information on redirecting standard error.


Optional

By convention, a process expects that the program that called it (frequently the shell) has set up standard input, standard output, and standard error so the process can use them immediately. The called process does not have to know which files or devices are connected to standard input, standard output, or standard error.

However, a process can query the kernel to get information about the device that standard input, standard output, or standard error is connected to. For example, the ls utility displays output in multiple columns when the output goes to the screen, but generates a single column of output when the output is redirected to a file or another program. The ls utility uses the isatty() system call to determine whether output is going to the screen (a tty). In addition, ls can use another system call to determine the width of the screen it is sending output to; with this information it can modify its output to fit the screen. Compare the output of ls by itself and when you send it through a pipeline to less. See page 1020 for information on how you can determine whether standard input and standard output of shell scripts is going to/coming from the terminal.


The Screen as a File

Image Device file

Chapter 6 discusses ordinary files, directory files, and hard and soft links. Linux has an additional type of file: a device file. A device file resides in the file structure, usually in the /dev directory, and represents a peripheral device, such as a terminal, printer, or disk drive.

The device name the who utility displays following a username is the filename of the terminal that user is working on. For example, when who displays the device name pts/4, the pathname of the terminal is /dev/pts/4. When you work with multiple windows, each window has its own device name. You can also use the tty utility to display the name of the device that you give the command from. Although you would not normally have occasion to do so, you can read from and write to this file as though it were a text file. Reading from the device file that represents the terminal you are using reads what you enter on the keyboard; writing to it displays what you write on the screen.

The Keyboard and Screen as Standard Input and Standard Output

After you log in, the shell directs standard output of commands you enter to the device file that represents the terminal (Figure 5-3). Directing output in this manner causes it to appear on the screen. The shell also directs standard input to come from the same file, so commands receive as input anything you type on the keyboard.

Image

Figure 5-3 By default, standard input comes from the keyboard, and standard output goes to the screen

Image cat

The cat utility provides a good example of the way the keyboard and screen function as standard input and standard output, respectively. When you run cat, it copies a file to standard output. Because the shell directs standard output to the screen, cat displays the file on the screen.

Up to this point cat has taken its input from the filename (argument) you specify on the command line. When you do not give cat an argument (i.e., when you give the command cat followed immediately by RETURN), cat takes its input from standard input. Thus, when called without an argument, cat copies standard input to standard output, one line at a time.

To see how cat works, type cat and press RETURN in response to the shell prompt. Nothing happens. Enter a line of text and press RETURN. The same line appears just under the one you entered. The cat utility is working. Because the shell associates cat’s standard input with the keyboard andcat’s standard output with the screen, when you type a line of text cat copies the text from standard input (the keyboard) to standard output (the screen). The next example shows this exchange.

$ cat
This is a line of text.
This is a line of text.
Cat keeps copying lines of text
Cat keeps copying lines of text
until you press CONTROL-D at the beginning
until you press CONTROL-D at the beginning
of a line.
of a line.
CONTROL-D
$

CONTROL-D signals EOF

The cat utility keeps copying text until you enter CONTROL-D on a line by itself. Pressing CONTROL-D causes the tty device driver to send an EOF (end of file) signal to cat. This signal indicates to cat that it has reached the end of standard input and there is no more text for it to copy. The catutility then finishes execution and returns control to the shell, which displays a prompt.

Image Redirection

The term redirection encompasses the various ways you can cause the shell to alter where standard input of a command comes from and where standard output goes to. By default, the shell associates standard input and standard output of a command with the keyboard and the screen. You can cause the shell to redirect standard input or standard output of any command by associating the input or output with a command or file other than the device file representing the keyboard or the screen. This section demonstrates how to redirect input/output from/to text files and utilities.

Image Redirecting Standard Output

The redirect output symbol (>) instructs the shell to redirect the output of a command to the specified file instead of to the screen (Figure 5-4). The syntax of a command line that redirects output is

command [arguments] > filename

where command is any executable program (e.g., an application program or a utility), arguments are optional arguments, and filename is the name of the ordinary file the shell redirects the output to.

Image

Figure 5-4 Redirecting standard output

Using cat to create a file

The next example uses cat to demonstrate output redirection. This example contrasts with the example on page 153, where standard input and standard output are associated with the keyboard and screen. The input in the following example comes from the keyboard. The redirect output symbol on the command line causes the shell to associate cat’s standard output with the sample.txt file specified following this symbol.

$ cat > sample.txt
This text is being entered at the keyboard and
cat is copying it to a file.
Press CONTROL-D to indicate the
end of file.
CONTROL-D
$


Caution: Redirecting output can destroy a file I

Use caution when you redirect output to a file. If the file exists, the shell will overwrite it and destroy its contents. For more information see the tip “Redirecting output can destroy a file II” on page 156.


After giving the command and typing the text shown in the previous example, the sample.txt file contains the text you entered. You can use cat with an argument of sample.txt to display this file. The next section shows another way to use cat to display the file.

The previous example shows that redirecting standard output from cat is a handy way to create a file without using an editor. The drawback is that once you enter a line and press RETURN, you cannot edit the text until after you finish creating the file. While you are entering a line, the erase and kill keys work to delete text on that line. This procedure is useful for creating short, simple files.

The next example shows how to run cat and use the redirect output symbol to catenate (join one after the other—the derivation of the name of the cat utility) several files into one larger file. The first three commands display the contents of three files: stationery, tape, and pens. The next command shows cat with three filenames as arguments. When you call it with more than one filename, cat copies the files, one at a time, to standard output. This command redirects standard output to the file named supply_orders. The final cat command shows that supply_orders contains the contents of the three original files.

$ cat stationery
2,000 sheets letterhead ordered: October 7
$ cat tape
1 box masking tape ordered: October 14
5 boxes filament tape ordered: October 28
$ cat pens
12 doz. black pens ordered: October 4

$ cat stationery tape pens > supply_orders

$ cat supply_orders
2,000 sheets letterhead ordered: October 7
1 box masking tape ordered: October 14
5 boxes filament tape ordered: October 28
12 doz. black pens ordered: October 4

Image Redirecting Standard Input

Just as you can redirect standard output, so you can redirect standard input. The redirect input symbol (<) instructs the shell to redirect a command’s input to come from the specified file instead of from the keyboard (Figure 5-5). The syntax of a command line that redirects input is

command [arguments] < filename

Image

Figure 5-5 Redirecting standard input

where command is any executable program (such as an application program or a utility), arguments are optional arguments, and filename is the name of the ordinary file the shell redirects the input from.

The next example shows cat with its input redirected from the supply_orders file created in the previous example and standard output going to the screen. This setup causes cat to display the supply_orders file on the screen. The system automatically supplies an EOF signal at the end of an ordinary file.

$ cat < supply_orders
2,000 sheets letterhead ordered: October 7
1 box masking tape ordered: October 14
5 boxes filament tape ordered: October 28
12 doz. black pens ordered: October 4

Utilities that take input from a file or standard input

Giving a cat command with input redirected from a file yields the same result as giving a cat command with the filename as an argument. The cat utility is a member of a class of utilities that function in this manner. Other members of this class of utilities include lpr, sort, grep, and Perl. These utilities first examine the command line that called them. If the command line includes a filename as an argument, the utility takes its input from the specified file. If no filename argument is present, the utility takes its input from standard input. It is the utility or program—not the shell or operating system—that functions in this manner.


Caution: Redirecting output can destroy a file II

Depending on which shell you are using and how the environment is set up, a command such as the following can yield undesired results:

$ cat orange pear > orange
cat: orange: input file is output file

Although cat displays an error message, the shell destroys the contents of the existing orange file. The new orange file will have the same contents as pear because the first action the shell takes when it sees the redirection symbol (>) is to remove the contents of the original orange file. If you want to catenate two files into one, use cat to put the two files into a temporary file and then use mv to rename the temporary file:

$ cat orange pear > temp
$ mv temp orange

What happens in the next example can be even worse. The user giving the command wants to search through files a, b, and c for the word apple and redirect the output from grep (page 232) to the file a.output. Unfortunately the user enters the filename as a output, omitting the period and inserting a SPACE in its place:

$ grep apple a b c > a output
grep: output: No such file or directory

The shell obediently removes the contents of a and then calls grep. The error message could take a moment to appear, giving you a sense that the command is running correctly. Even after you see the error message, it might take a while to realize that you have destroyed the contents of a.


noclobber: Prevents Overwriting Files

The shell provides the noclobber feature, which prevents you from overwriting a file using redirection. Enable this feature by setting noclobber using the command set –o noclobber. The same command with +o unsets noclobber. With noclobber set, if you redirect output to an existing file, the shell displays an error message and does not execute the command. The following example creates a file using touch, sets noclobber, attempts to redirect the output from echo to the newly created file, unsets noclobber, and performs the same redirection:

$ touch tmp
$ set -o noclobber
$ echo "hi there" > tmp
-bash: tmp: cannot overwrite existing file
$ set +o noclobber
$ echo "hi there" > tmp

You can override noclobber by putting a pipeline symbol after the redirect symbol (>|). In the following example, the user creates a file by redirecting the output of date. Next the user sets the noclobber variable and redirects output to the same file again. The shell displays an error message. Then the user places a pipeline symbol after the redirect symbol, and the shell allows the user to overwrite the file.

$ date > tmp2
$ set -o noclobber
$ date > tmp2
-bash: tmp2: cannot overwrite existing file
$ date >| tmp2

Appending Standard Output to a File


Caution: Do not trust noclobber

Appending output is simpler than the two-step procedure described in the preceding caution box but you must be careful to include both greater than signs. If you accidentally use only one greater than sign and the noclobber feature is not set, the shell will overwrite theorange file. Even if you have the noclobber feature turned on, it is a good idea to keep backup copies of the files you are manipulating in case you make a mistake.

Although it protects you from overwriting a file using redirection, noclobber does not stop you from overwriting a file using cp or mv. These utilities include the –i (interactive) option that helps protect you from this type of mistake by verifying your intentions when you try to overwrite a file. For more information see the tip “cp can destroy a file” on page 224.


The append output symbol (>>) causes the shell to add new information to the end of a file, leaving existing information intact. This symbol provides a convenient way of catenating two files into one. The following commands demonstrate the action of the append output symbol. The second command accomplishes the catenation described in the preceding caution box:

$ cat orange
this is orange
$ cat pear >> orange
$ cat orange
this is orange
this is pear

The first command displays the contents of the orange file. The second command appends the contents of the pear file to the orange file. The final command displays the result.

The next example shows how to create a file that contains the date and time (the output from date), followed by a list of who is logged in (the output from who). The first command in the example redirects the output from date to the file named whoson. Then cat displays the file. The next command appends the output from who to the whoson file. Finally cat displays the file containing the output of both utilities.

$ date > whoson
$ cat whoson
Wed Mar 27 14:31:18 PST 2013
$ who >> whoson
$ cat whoson
Wed Mar 27 14:31:18 PST 2013
sam tty1 2013-03-27 05:00(:0)
max pts/4 2013-03-27 12:23(:0.0)
max pts/5 2013-03-27 12:33(:0.0)
zach pts/7 2013-03-26 08:45 (172.16.192.1)

/dev/null: Making Data Disappear

The /dev/null device is a data sink, commonly referred to as a bit bucket. You can redirect output you do not want to keep or see to /dev/null, and the output will disappear without a trace:

$ echo "hi there" > /dev/null
$

Reading from /dev/null yields a null string. The following command truncates the file named messages to zero length while preserving the ownership and permissions of the file:

$ ls -lh messages
-rw-rw-r--. 1 sam pubs 125K 03-16 14:30 messages
$ cat /dev/null > messages
$ ls -lh messages
-rw-rw-r--. 1 sam pubs 0 03-16 14:32 messages

See also page 503.

Image Pipelines

A pipeline consists of one or more commands separated by a pipeline symbol (|). The shell connects standard output (and optionally standard error) of the command preceding the pipeline symbol to standard input of the command following the pipeline symbol. A pipeline has the same effect as redirecting standard output of one command to a file and then using that file as standard input to another command. A pipeline does away with separate commands and the intermediate file. The syntax of a pipeline is

command_a [arguments] | command_b [arguments]

The preceding command line uses a pipeline to effect the same result as the following three commands:

command_a [arguments] > temp
command_b [arguments] < temp
rm temp

In the preceding sequence of commands, the first line redirects standard output from command_a to an intermediate file named temp. The second line redirects standard input for command_b to come from temp. The final line deletes temp. The pipeline syntax is not only easier to type but also is more efficient because it does not create a temporary file.


Optional

More precisely, a bash pipeline comprises one or more simple commands (page 147) separated by a | or |& control operator. A pipeline has the following syntax:

[time] [!] command1 [| | |& command2 ... ]

where time is an optional utility that summarizes the system resources used by the pipeline, ! logically negates the exit status returned by the pipeline, and the commands are simple commands (page 147) separated by | or |&. The | control operator sends standard output ofcommand1 to standard input of command2. The |& control operator is short for 2>&1 | (see “Sending errors through a pipeline” on page 335) and sends standard output and standard error of command1 to standard input of command2. The exit status of a pipeline is the exit status of the last simple command unless pipefail (page 403) is set, in which case the exit status is the rightmost simple command that failed (returned a nonzero exit status) or zero if all simple commands completed successfully.


Examples of Pipelines

Image tr

You can include in a pipeline any utility that accepts input either from a file specified on the command line or from standard input. You can also include utilities that accept input only from standard input. For example, the tr (translate) utility takes its input from standard input only. In its simplest usage tr has the following syntax:

tr string1 string2

The tr utility accepts input from standard input and looks for characters that match one of the characters in string1. Upon finding a match, it translates the matched character in string1 to the corresponding character in string2. That is, the first character in string1 translates into the first character in string2, and so forth. The tr utility sends its output to standard output. In both of the following tr commands, tr displays the contents of the abstract file with the letters a, b, and c translated into A, B, and C, respectively:

$ cat abstract
I took a cab today!

$ cat abstract | tr abc ABC
I took A CAB todAy!
$ tr abc ABC < abstract
I took A CAB todAy!

The tr utility does not change the contents of the original file; it cannot change the original file because it does not “know” the source of its input.

lpr

The lpr (line printer) utility accepts input from either a file or standard input. When you type the name of a file following lpr on the command line, it places that file in the print queue. When you do not specify a filename on the command line, lpr takes input from standard input. This feature enables you to use a pipeline to redirect input to lpr. The first set of the following commands shows how you can use ls and lpr with an intermediate file (temp) to send a list of the files in the working directory to the printer. If the temp file exists, the first command overwrites its contents. The second set of commands uses a pipeline to send the same list (with the exception of temp) to the printer.

$ ls > temp
$ lpr temp
$ rm temp

or

$ ls | lpr

sort

The commands in next example redirect the output from the who utility to temp and then display this file in sorted order. The sort utility (page 239) takes its input from the file specified on the command line or, when a file is not specified, from standard input; it sends its output to standard output. The sort command line takes its input from standard input, which is redirected (<) to come from temp. The output sort sends to the screen lists the users in sorted (alphabetical) order. Because sort can take its input from standard input or from a file named on the command line, omitting the < symbol from the command line yields the same result.

$ who > temp
$ sort < temp
max pts/4 2013-03-24 12:23
max pts/5 2013-03-24 12:33
sam tty1 2013-03-24 05:00
zach pts/7 2013-03-23 08:45
$ rm temp

The next example achieves the same result without creating the temp file. Using a pipeline, the shell redirects the output from who to the input of sort. The sort utility takes input from standard input because no filename follows it on the command line.

$ who | sort
max pts/4 2013-03-24 12:23
max pts/5 2013-03-24 12:33
sam tty1 2013-03-24 05:00
zach pts/7 2013-03-23 08:45

grep

When many people are using the system and you want information about only one of them, you can send the output from who to grep (page 232) using a pipeline. The grep utility displays the line containing the string you specify—sam in the following example:

$ who | grep sam
sam tty1 2013-03-24 05:00

less and more

Another way of handling output that is too long to fit on the screen, such as a list of files in a crowded directory, is to use a pipeline to send the output through less or more (both on page 220).

$ ls | less

The less utility displays text one screen at a time. To view another screen of text, press the SPACE bar. To view one more line, press RETURN. Press h for help and q to quit.


Optional

The pipeline symbol (|) implies continuation. Thus the following command line

$ who | grep 'sam'
sam tty1 2013-03-24 05:00

is the same as these command lines

$ who |
> grep 'sam'
sam tty1 2013-03-24 05:00

When the shell parses a line that ends with a pipeline symbol, it requires more input before it can execute the command line. In an interactive environment, it issues a secondary prompt (>; page 362) as shown above. Within a shell script, it processes the next line as a continuation of the line that ends with the pipeline symbol. See page 1063 for information about control operators and implicit command-line continuation.


Image Filters

A filter is a command that processes an input stream of data to produce an output stream of data. A command line that includes a filter uses a pipeline symbol to connect standard output of one command to standard input of the filter. Another pipeline symbol connects standard output of the filter to standard input of another command. Not all utilities can be used as filters.

In the following example, sort is a filter, taking standard input from standard output of who and using a pipeline symbol to redirect standard output to standard input of lpr. This command line sends the sorted output of who to the printer:

$ who | sort | lpr

The preceding example demonstrates the power of the shell combined with the versatility of Linux utilities. The three utilities who, sort, and lpr were not designed to work with one another, but they all use standard input and standard output in the conventional way. By using the shell to handle input and output, you can piece standard utilities together on the command line to achieve the results you want.

Image tee

The tee utility copies its standard input both to a file and to standard output. This utility is aptly named: It takes a single stream of input and sends the output in two directions. The next example sends the output of who via a pipeline to standard input of tee. The tee utility saves a copy of standard input in a file named who.out and also sends a copy to standard output. Standard output of tee goes via a pipeline to standard input of grep, which displays only those lines containing the string sam. Use tee with the –a (append) option to cause it to append to a file instead of overwriting it.

$ who | tee who.out | grep sam
sam tty1 2013-03-24 05:00
$ cat who.out
sam tty1 2013-03-24 05:00
max pts/4 2013-03-24 12:23
max pts/5 2013-03-24 12:33
zach pts/7 2013 -03-23 08:45


Optional

Image Lists

A list is one or more pipelines (including simple commands), each separated from the next by one of the following control operators: ;, &, &&, or ||. The && and || control operators have equal precedence; they are followed by ; and &, which have equal precedence. The ;control operator is covered on page 341 and & on page 342. See page 1063 for information about control operators and implicit command-line continuation.

An AND list has the following syntax:

pipeline1 && pipeline2

where pipeline2 is executed if and only if pipeline1 returns a true (zero) exit status. In the following example, the first command in the list fails (and displays an error message) so the shell does not execute the second command (cd /newdir; because it is not executed, it does not display an error message):

$ mkdir /newdir && cd /newdir
mkdir: cannot create directory '/newdir': Permission denied

The exit status of AND and OR lists is the exit status of the last command in the list that is executed. The exit status of the preceding list is false because mkdir was the last command executed and it failed.

An OR list has the following syntax:

pipeline1 || pipeline2

where pipeline2 is executed if and only if pipeline1 returns a false (nonzero) exit status. In the next example, the first command (ping tests the connection to a remote machine and sends standard output and standard error to /dev/null) in the list fails so the shell executes the second command (it displays a message). If the first command had completed successfully, the shell would not have executed the second command (and would not have displayed the message). The list returns an exit status of true.

$ ping -c1 station &>/dev/null || echo "station is down"
station is down

For more information refer to “&& and || Boolean Control Operators” on page 343.


Image Running a Command in the Background

Image Foreground

All commands up to this point have been run in the foreground. When you run a command in the foreground, the shell waits for it to finish before displaying another prompt and allowing you to continue. When you run a command in the background, you do not have to wait for the command to finish before running another command.

Jobs

A job is another name for a process running a pipeline (which can be a simple command). You can have only one foreground job in a window or on a screen, but you can have many background jobs. By running more than one job at a time, you are using one of Linux’s features: multitasking. Running a command in the background can be useful when the command will run for a long time and does not need supervision. It leaves the screen free so you can use it for other work. Alternately, when you are using a GUI, you can open another window to run another job.

Job number, PID number

To run a command in the background, type an ampersand (&; a control operator) just before the RETURN that ends the command line. The shell assigns a small number to the job and displays this job number between brackets. Following the job number, the shell displays the process identification (PID) number—a larger number assigned by the operating system. Each of these numbers identifies the command running in the background. The shell then displays another prompt, and you can enter another command. When the background job finishes, the shell displays a message giving both the job number and the command line used to run the command.

The following example runs in the background; it is a pipeline that sends the output of ls to lpr, which sends it to the printer.

$ ls -l | lpr &
[1] 22092
$

The [1] following the command line indicates that the shell has assigned job number 1 to this job. The 22092 is the PID number of the first command in the job. When this background job completes execution, you see the message

[1]+ Done ls -l | lpr

(In place of ls –l, the shell might display something similar to ls ––color=auto –l. This difference is due to the fact that ls is aliased [page 392] to ls ––color=auto.)

Image Moving a Job from the Foreground to the Background

CONTROL-Z and bg

You can suspend a foreground job (stop it from running) by pressing the suspend key, usually CONTROL-Z. The shell then stops the process and disconnects standard input from the keyboard. It does, however, still send standard output and standard error to the screen. You can put a suspended job in the background and restart it by using the bg command followed by the job number. You do not need to specify the job number when there is only one suspended job.

Redirect the output of a job you run in the background to keep it from interfering with whatever you are working on in the foreground (on the screen). Refer to “Control Operators: Separate and Group Commands” on page 341 for more detail about background tasks.

fg

Only the foreground job can take input from the keyboard. To connect the keyboard to a program running in the background, you must bring the program to the foreground. To do so, type fg without any arguments when only one job is in the background. When more than one job is in the background, type fg, or a percent sign (%), followed by the number of the job you want to bring to the foreground. The shell displays the command you used to start the job (promptme in the following example), and you can enter input the program requires to continue.

$ fg 1
promptme

Image kill: Aborting a Background Job

The interrupt key (usually CONTROL-C) cannot abort a background process because the keyboard is not attached to the job; you must use kill (page 465) for this purpose. Follow kill on the command line with either the PID number of the process you want to abort or a percent sign (%) followed by the job number.

Determining the PID of a process using ps

If you forget a PID number, you can use the ps (process status) utility (page 374) to display it. The following example runs a find command in the background, uses ps to display the PID number of the process, and aborts the job using kill:

$ find / -name memo55 > mem.out &
[1] 18228
$ ps | grep find
18228 pts/10 00:00:01 find
$ kill 18228
[1]+ Terminated find / -name memo55 > mem.out
$

Image Determining the number of a job using jobs

If you forget a job number, you can use the jobs command to display a list of jobs that includes job numbers. The next example is similar to the previous one except it uses the job number instead of the PID number to identify the job to be killed. Sometimes the message saying the job is terminated does not appear until you press RETURN after the RETURN that executes the kill command.

$ find / -name memo55 > mem.out &
[1] 18236

$ bigjob &
[2] 18237

$ jobs
[1]- Running find / -name memo55 > mem.out &
[2]+ Running bigjob &
$ kill %1
$ RETURN
[1]- Terminated find / -name memo55 > mem.out
$

Image Filename Generation/Pathname Expansion

Wildcards, globbing

When you specify an abbreviated filename that contains special characters, also called metacharacters, the shell can generate filenames that match the names of existing files. These special characters are also referred to as wildcards because they act much as the jokers do in a deck of cards. When one of these characters appears in an argument on the command line, the shell expands that argument in sorted order into a list of filenames and passes the list to the program called by the command line. Filenames that contain these special characters are called ambiguous file referencesbecause they do not refer to one specific file. The process the shell performs on these filenames is called pathname expansion or globbing.

Ambiguous file references can quickly refer to a group of files with similar names, saving the effort of typing the names individually. They can also help find a file whose name you do not remember in its entirety. If no filename matches the ambiguous file reference, the shell generally passes the unexpanded reference—special characters and all—to the command. See “Brace Expansion” on page 405 for a technique that generates strings that do not necessarily match filenames.

The ? Special Character

The question mark (?) is a special character that causes the shell to generate filenames. It matches any single character in the name of an existing file. The following command uses this special character in an argument to the lpr utility:

$ lpr memo?

The shell expands the memo? argument and generates a list of files in the working directory that have names composed of memo followed by any single character. The shell then passes this list to lpr. The lpr utility never “knows” the shell generated the filenames it was called with. If no filename matches the ambiguous file reference, the shell passes the string itself (memo?) to lpr or, if it is set up to do so, passes a null string (see nullglob on page 402).

The following example uses ls first to display the names of all files in the working directory and then to display the filenames that memo? matches:

$ ls
mem memo12 memo9 memomax newmemo5
memo memo5 memoa memos

$ ls memo?
memo5 memo9 memoa memos

The memo? ambiguous file reference does not match mem, memo, memo12, memomax, or newmemo5. You can also use a question mark in the middle of an ambiguous file reference:

$ ls
7may4report may4report mayqreport may_report
may14report may4report.79 mayreport may.report

$ ls may?report
may4report mayqreport may_report may.report

echo

You can use echo and ls to practice generating filenames. The echo builtin displays the arguments the shell passes to it:

$ echo may?report
may4report mayqreport may_report may.report

The shell first expands the ambiguous file reference into a list of files in the working directory that match the string may?report. It then passes this list to echo, as though you had entered the list of filenames as arguments to echo. The echo utility displays the list of filenames.

A question mark does not match a leading period (one that indicates a hidden filename; page 180). When you want to match filenames that begin with a period, you must explicitly include the period in the ambiguous file reference.

The * Special Character

The asterisk (*) performs a function similar to that of the question mark but matches any number of characters, including zero characters, in a filename. The following example first shows all files in the working directory and then shows commands that display all the filenames that begin with the string memo, end with the string mo, and contain the string alx:

$ ls
amemo memalx memo.0612 memoalx.0620 memorandum sallymemo
mem memo memoa memoalx.keep memosally user.memo

$ echo memo*
memo memo.0612 memoa memoalx.0620 memoalx.keep memorandum memosally

$ echo *mo
amemo memo sallymemo user.memo

$ echo *alx*
memalx memoalx.0620 memoalx.keep

The ambiguous file reference memo* does not match amemo, mem, sallymemo, or user.memo. Like the question mark, an asterisk does not match a leading period in a filename.

The –a option causes ls to display hidden filenames (page 180). The command echo * does not display . (the working directory), .. (the parent of the working directory), .aaa, or .profile. In contrast, the command echo .* displays only those four names:

$ ls
aaa memo.0612 memo.sally report sally.0612 saturday thurs

$ ls -a
. aaa memo.0612 .profile sally.0612 thurs
.. .aaa memo.sally report saturday

$ echo *
aaa memo.0612 memo.sally report sally.0612 saturday thurs

$ echo .*
. .. .aaa .profile

In the following example, .p* does not match memo.0612, private, reminder, or report. The ls .* command causes ls to list .private and .profile in addition to the contents of the . directory (the working directory) and the .. directory (the parent of the working directory). When called with the same argument, echo displays the names of files (including directories) in the working directory that begin with a dot (.) but not the contents of directories.

$ ls -a
. .. memo.0612 private .private .profile reminder report

$ echo .p*
.private .profile

$ ls .*
.private .profile
.:
memo.0612 private reminder report
..:
...

$ echo .*
. .. .private .profile

You can plan to take advantage of ambiguous file references when you establish conventions for naming files. For example, when you end the names of all text files with .txt, you can reference that group of files with *.txt. The next command uses this convention to send all text files in the working directory to the printer. The ampersand causes lpr to run in the background.

$ lpr *.txt &


Tip: The shell expands ambiguous file references

The shell does the expansion when it processes an ambiguous file reference, not the program that the shell runs. In the examples in this section, the utilities (ls, cat, echo, lpr) never see the ambiguous file references. The shell expands the ambiguous file references and passes a list of ordinary filenames to the utility. In the previous examples, echo demonstrates this fact because it simply displays its arguments; it never displays the ambiguous file reference.


The [ ] Special Characters

A pair of brackets surrounding one or more characters causes the shell to match filenames containing the individual characters within the brackets. Whereas memo? matches memo followed by any character, memo[17a] is more restrictive: It matches only memo1, memo7, and memoa. The brackets define a character class that includes all the characters within the brackets. (GNU calls this a character list; a GNU character class is something different.) The shell expands an argument that includes a character-class definition by substituting each member of the character class,one at a time, in place of the brackets and their contents. The shell then passes the list of matching filenames to the program it is calling.

Each character-class definition can replace only a single character within a filename. The brackets and their contents are like a question mark that substitutes only the members of the character class.

The first of the following commands lists the names of all files in the working directory that begin with a, e, i, o, or u. The second command displays the contents of the files named page2.txt, page4.txt, page6.txt, and page8.txt.

$ echo [aeiou]*
...

$ less page[2468].txt
...

A hyphen within brackets defines a range of characters within a character-class definition. For example, [6–9] represents [6789], [a–z] represents all lowercase letters in English, and [a–zA–Z] represents all letters, both uppercase and lowercase, in English.

The following command lines show three ways to print the files named part0, part1, part2, part3, and part5. Each of these command lines causes the shell to call lpr with five filenames:

$ lpr part0 part1 part2 part3 part5

$ lpr part[01235]

$ lpr part[0-35]

The first command line explicitly specifies the five filenames. The second and third command lines use ambiguous file references, incorporating character-class definitions. The shell expands the argument on the second command line to include all files that have names beginning with partand ending with any of the characters in the character class. The character class is explicitly defined as 0, 1, 2, 3, and 5. The third command line also uses a character-class definition but defines the character class to be all characters in the range 0–3 plus 5.

The following command line prints 39 files, part0 through part38:

$ lpr part[0-9] part[12][0-9] part3[0-8]

The first of the following commands lists the files in the working directory whose names start with a through m. The second lists files whose names end with x, y, or z.

$ echo [a-m]*
...

$ echo *[x-z]
...


Optional

When an exclamation point (!) or a caret (^) immediately follows the opening bracket ([) that starts a character-class definition, the character class matches any character not between the brackets. Thus [^tsq]* matches any filename that does not begin with t, s, or q.

The following examples show that *[^ab] matches filenames that do not end with the letter a or b and that [^b-d]* matches filenames that do not begin with b, c, or d.

$ ls
aa ab ac ad ba bb bc bd cc dd

$ ls *[^ab]
ac ad bc bd cc dd

$ ls [^b-d]*
aa ab ac ad

You can cause a character class to match a hyphen () or a closing bracket (]) by placing it immediately before the final (closing) bracket.


The next example demonstrates that the ls utility cannot interpret ambiguous file references. First ls is called with an argument of ?old. The shell expands ?old into a matching filename, hold, and passes that name to ls. The second command is the same as the first, except the ? is quoted (by preceding it with a backslash [\]; refer to “Special Characters” on page 142). Because the ? is quoted, the shell does not recognize it as a special character and passes it to ls. The ls utility generates an error message saying that it cannot find a file named ?old (because there is no file named ?old).

$ ls ?old
hold

$ ls \?old
ls: ?old: No such file or directory

Like most utilities and programs, ls cannot interpret ambiguous file references; that work is left to the shell.

Builtins

A builtin is a utility (also called a command) that is built into a shell. Each of the shells has its own set of builtins. When it runs a builtin, the shell does not fork a new process. Consequently builtins run more quickly and can affect the environment of the current shell. Because builtins are used in the same way as utilities, you will not typically be aware of whether a utility is built into the shell or is a stand-alone utility.

For example, echo is a shell builtin. It is also a stand-alone utility. The shell always executes a shell builtin before trying to find a command or utility with the same name. See page 1040 for an in-depth discussion of builtin commands and page 1055 for a list of bash builtins.

Listing bash builtins

To display a list of bash builtins, give the command info bash shell builtin. To display a page with information on each builtin, move the cursor to the Bash Builtins line and press RETURN. Alternately, you can view the builtins man page.

Getting help with bash builtins

You can use the bash help command to display information about bash builtins. See page 134 for more information.

Chapter Summary

The shell is the Linux command interpreter. It scans the command line for proper syntax, picking out the command name and arguments. The name of the command is argument zero. The first argument is argument one, the second is argument two, and so on. Many programs use options to modify the effects of a command. Most Linux utilities identify an option by its leading one or two hyphens.

When you give it a command, the shell tries to find an executable program with the same name as the command. When it does, the shell executes the program. When it does not, the shell tells you it cannot find or execute the program. If the command is a simple filename, the shell searches the directories listed in the PATH variable to locate the command.

When it executes a command, the shell assigns one file or device to the command’s standard input and another file to its standard output. By default, the shell causes a command’s standard input to come from the keyboard and its standard output to go to the screen. You can instruct the shell to redirect a command’s standard input from or standard output to any file or device. You can also connect standard output of one command to standard input of another command to form a pipeline. A filter is a command that reads its standard input from standard output of one command and writes its standard output to standard input of another command.

When a command runs in the foreground, the shell waits for the command to finish before it displays a prompt and allows you to continue. When you put an ampersand (&) at the end of a command line, the shell executes the command in the background and displays another prompt immediately. Run slow commands in the background when you want to enter other commands at the shell prompt. The jobs builtin displays a list of suspended jobs and jobs running in the background and includes the job number of each.

The shell interprets special characters on a command line to generate filenames. A reference that uses special characters (wildcards) to abbreviate a list of one or more filenames is called an ambiguous file reference. A question mark represents any single character, and an asterisk represents zero or more characters. A single character might also be represented by a character class: a list of characters within brackets.

A builtin is a utility that is built into a shell. Each shell has its own set of builtins. When it runs a builtin, the shell does not fork a new process. Consequently builtins run more quickly and can affect the environment of the current shell.

Utilities and Builtins Introduced in This Chapter

Table 5-1 lists the utilities introduced in this chapter.

Image

Table 5-1 New utilities

Exercises

1. What does the shell ordinarily do while a command is executing? What should you do if you do not want to wait for a command to finish before running another command?

2. Using sort as a filter, rewrite the following sequence of commands:

$ sort list > temp
$ lpr temp
$ rm temp

3. What is a PID number? Why are these numbers useful when you run processes in the background? Which utility displays the PID numbers of the commands you are running?

4. Assume the following files are in the working directory:

$ ls
intro notesb ref2 section1 section3 section4b
notesa ref1 ref3 section2 section4a sentrev

Give commands for each of the following, using wildcards to express filenames with as few characters as possible.

a. List all files that begin with section.

b. List the section1, section2, and section3 files only.

c. List the intro file only.

d. List the section1, section3, ref1, and ref3 files.

5. Refer to the info or man pages to determine which command will

a. Display the number of lines in its standard input that contain the word a or A.

b. Display only the names of the files in the working directory that contain the pattern $(.

c. List the files in the working directory in reverse alphabetical order.

d. Send a list of files in the working directory to the printer, sorted by size.

6. Give a command to

a. Redirect standard output from a sort command to a file named phone_list. Assume the input file is named numbers.

b. Translate all occurrences of the characters [ and { to the character (, and all occurrences of the characters ] and } to the character ), in the file permdemos.c. (Hint: Refer to the tr man page.)

c. Create a file named book that contains the contents of two other files: part1 and part2.

7. The lpr and sort utilities accept input either from a file named on the command line or from standard input.

a. Name two other utilities that function in a similar manner.

b. Name a utility that accepts its input only from standard input.

8. Give an example of a command that uses grep

a. With both input and output redirected.

b. With only input redirected.

c. With only output redirected.

d. Within a pipeline.

In which of the preceding cases is grep used as a filter?

9. Explain the following error message. Which filenames would a subsequent ls command display?

$ ls
abc abd abe abf abg abh
$ rm abc ab*
rm: cannot remove 'abc': No such file or directory

Advanced Exercises

10. When you use the redirect output symbol (>) on a command line, the shell creates the output file immediately, before the command is executed. Demonstrate that this is true.

11. In experimenting with variables, Max accidentally deletes his PATH variable. He decides he does not need the PATH variable. Discuss some of the problems he could soon encounter and explain the reasons for these problems. How could he easily return PATH to its original value?

12. Assume permissions on a file allow you to write to the file but not to delete it.

a. Give a command to empty the file without invoking an editor.

b. Explain how you might have permission to modify a file that you cannot delete.

13. If you accidentally create a filename that contains a nonprinting character, such as a CONTROL character, how can you remove the file?

14. Why does the noclobber variable not protect you from overwriting an existing file with cp or mv?

15. Why do command names and filenames usually not have embedded SPACEs? How would you create a filename containing a SPACE? How would you remove it? (This is a thought exercise, not recommended practice. If you want to experiment, create a file and work in a directory that contains only your experimental file.)

16. Create a file named answer and give the following command:

$ > answers.0102 < answer cat

Explain what the command does and why. What is a more conventional way of expressing this command?