The Command Shell - Basic - UNIX: The Complete Reference (2007)

UNIX: The Complete Reference (2007)

Part I: Basic

Chapter 4: The Command Shell

As you have seen, a large part of using UNIX is issuing commands. When you enter a command, you are dealing with the shell, the interface through which you control the resources of the UNIX operating system. The shell provides many of the features that make the UNIX System a uniquely powerful and flexible environment. It is a command interpreter, a programming language, and more.

This chapter describes the basic features that the shell provides, focusing on how it interprets your commands and how you can use its features to simplify your interactions with the UNIX System. It explains what the shell does for you, how it works, and how you use it to issue commands and to control how they are run. You will learn all of the basic shell commands and features that you need to understand in order to use the UNIX System effectively and confidently

The shell includes many features that allow you to customize your working environment. You can use shell variables and aliases to simplify common tasks, which will be covered in this chapter. In addition, you can use the shell’s command language as a high-level programming language to create programs called scripts. Shell scripting will be described later, in Chapter 20.

There are actually several different shell programs, but they all provide the same basic capabilities. This chapter covers the most common shells: sh (the original Bourne shell), csh and tcsh (the C shell and extended C shell), ksh (the Korn shell), and bash (the Bourne Again shell). It describes the features that you need to know to use any of these shells, as well as explaining the enhancements each of the shells has to offer, and how they differ from each other.

The Common Shells

The original UNIX System shell, sh, was written by Steve Bourne, and as a result it is known as the Bourne shell. Because it was the first, the Bourne shell lacks many enhancements common to the later shells. It can still be useful for certain tasks (such as scripting, described in Chapter 20), but almost all users will prefer one of the newer shells for entering commands.

The C shell, csh, was the first attempt to enhance the original Bourne shell. The syntax was strongly influenced by the C programming language. The C shell introduced the concepts of a command history list, job control, and aliases. However, like sh, it lacks some important features of later shells. A common complaint about csh is that the new syntax is not compatible with the Bourne shell, and so some scripts may not work properly in csh.

The extended C shell, tcsh, has replaced csh entirely on some versions of UNIX (including Linux). It retains all the features of csh and adds command-line editing (a very important shell feature) and history completion. It is one of the more popular shells, although like csh it has been criticized for not being compatible with the Bourne shell.

The Korn shell, ksh, was developed at AT&T Bell Laboratories by David Korn. Unlike the C shell, the Korn shell has a syntax compatible with sh. Like tcsh, ksh includes a command history list, job control, aliases, and command-line editing. The Korn shell was until recently proprietary to AT&T, although it can now be downloaded for free.

The Bourne Again Shell, bash, is part of the GNU project. It extends ksh further, while remaining compatible with the original Bourne shell syntax, and adds a few features from tcsh as well. bash is the default shell in Linux and may be the most popular shell today On some systems, the command sh will run bash instead.

There are other shells out there, including rc, which is a new shell similar to the original Bourne shell, pdksh, a public domain version of the Korn shell, and zsh, which adds even more features to bash. To see if your system includes a particular shell, check if it has a man page (e.g., try man ksh), or read the file /etc/shells, which is a list of the shells installed on the system. The section “Sources and Sites” at the end of this chapter lists the web sites where the various shells may be obtained, so that you can download and install your preferred shell if it didn’t come with your system.

Running the Shell

When you log in to the system, a shell program is automatically started for you. This is your login shell. The particular shell program that is run when you log in is determined by your entry in the file /etc/passwd.This file contains information the system needs to know about each user, including name, login ID, and so forth. The last field of this file contains the name of the program to run as your shell. A typical entry might look like

$ grep username /etc/passwd

username:x:3943:100:User Name:/home/username:/bin/bash

To view your own entry in /etc/passwd, just enter the command line shown, with your username. The command grep will search the file for the line with your username. (Chapter 19 describes the use of grep in detail.)

If you are using a graphical interface to UNIX, you may need to open a terminal window (such as xterm, gnome-terminal, or konsole) in order to use the shell. By default, your terminal program will run the same shell as your login shell.

Changing Your Login Shell

In general, you will probably want to keep the login shell that was assigned to you when your account was created. On a multiuser system, it can be a good idea to stick with the shell everyone else is using. Even if you are the only user on the system, the default shell has already been configured for you, which can be helpful.

In some cases, you may want to change your login shell. You may need a specific feature that your default shell lacks-for example, sh and csh lack some very important features of the newer shells, and tcsh is often criticized for not being compatible with Bourne shell scripts. Another reason would be if you are already very familiar with a particular shell and find it easier to continue using that shell.

As far as basic features are concerned, all of the common shells have now become very similar, although they did not start out that way For example, the availability of job control features in sh now removes one of the major differences between sh and csh. If you are trying to choose a shell to learn, you may want to consider bash. Not only is it the default choice on Linux systems, but it successfully combines most of the features of the Korn shell with some of the best features of the C shell, while remaining compatible with the original Bourne-shell syntax.

The command to change your login shell is chsh. In this example, the user rlf is changing her shell to bash:

$ chsh

Changing shell for rlf.

Password:

New shell [/bin/csh]: /bin/bash

Shell changed.

$

As you can see, rlf will need to enter her password before she can change her shell. Her current shell is /bin/csh. To change it, she needs to know the full pathname for the new shell.

To find the full pathname for a command or executable, type which followed by the name of the command. For example,

$ which bash

/bin/bash

You can also find the full pathnames for the shells installed on your system in the /etc/shells file.

You will not switch to the new shell until you log out and log back in to the system. On some systems, you may be restricted from changing your login shell. Contact your system administrator if the chshcommand is not available.

Logging Out

You log out from the UNIX System by terminating your login shell. There are two ways to do this. You can quit the shell by typing CTRL-D in response to the shell prompt, or you can use the exit command, which terminates your current shell:

$ exit

If you are using a graphical interface, you can also exit by closing the terminal window.

What the Shell Does

After you log in, much of your interaction with the UNIX System takes the form of a dialog with the shell. The dialog follows this simple sequence repeated over and over:

1. The shell prompts you when it is ready for input, and waits for you to enter a command.

2. You enter a command by typing in a command line.

3. The shell processes your command line to determine what actions to take and carries out the actions required.

4. After the program is finished, the shell prompts you for input, beginning the cycle again.

The part of this cycle where the real work takes place is the third step-when the shell reads and processes your command line and carries out the instructions it contains. For example, it replaces words in the command line that contain wildcards with the matching filenames. It determines where the input to the command is to come from and where its output goes. After carrying out these and similar operations, the shell runs the program you have indicated in your command, giving it the proper arguments (including options and filenames).

Entering Commands

In general, a command line contains the name of a command, optionally followed by a string of arguments. With a few exceptions (certain keywords like for and while), you end each command line with a newline, which is the UNIX System term for the character produced when you type the ENTER key The shell does not begin to process your command line until you end it with ENTER.

Command-line arguments include options that modify what a command does or how it does it, and information that the command needs, such as the name of a file from which to get data. Options are usually, but not always, indicated with a − sign. (As you saw in Chapter 3, the chmod command uses both + and − to indicate options, and the tar command described in Chapter 19 does not require a − in front of options.)

Your command line may also include arguments and symbols that are really instructions to the shell. For example, when you use the > symbol to direct the output of a command to a file, the shell will process this part of the command line without sending it as an argument to the command. Later in this chapter, you will learn how the shell processes this type of command-line instruction.

In this example of a command line,

$ ls -l Email > filelist

ls is the name of the command, -l is an option for the command, Email is a filename argument that gets sent to ls, and >filelist is an instruction for the shell.

Argument Expansion

The shell also interprets and replaces certain shortcuts before sending arguments on to commands. This is called expansion. For example, in the modern shells, ~ (tilde) is a shortcut for your home directory So if you enter a command like

$ mv dance ~

the shell will replace the ~ with your home directory (e.g., /home/raf) before sending the arguments to mv. Similarly, -username is a shortcut for any user’s home directory So if you enter

$ ln -s ~nate/ProjectFiles

the shell will expand the argument to /home/nate/ProjectFiles. You may have seen this use of ~ in URL’s (web addresses), as in home.webpages.net/~nate.

Tilde expansion is just one example of the shortcuts the shell recognizes. You already saw examples of filename expansion with wildcards in Chapter 3. Later in this chapter you will encounter variable and command expansion (such as $HOME to get the value of the variable HOME, or `ls` to get the value of the ls command).

Grouping Commands

Ordinarily you enter a single command (or a pipeline of two or more joined commands) on each line. If you want, though, you can enter several different commands at once on one line by separating them with a semicolon. For example, the following command line tells the shell to run date first, and then ls -l, just as if you had typed each command on a separate line:

$ date; ls -l

Using Wildcards

The shell gives you a way to abbreviate filenames through the use of special patterns called wildcards. You can use wildcards to specify a whole set of files at once, or to search for a file when you know only part of its name. The most commonly used wildcard is the *, which you encountered in Chapter 3. The asterisk matches a string of any length (including a string with zero characters). For example,

*html

Matches any filenames ending in html, including html and index.html.

note*

Matches any filenames beginning in note, such as note:8.28.

*kili*

Matches any filename containing the string “kili” anywhere in the name.

The standard UNIX System shell provides two other filename wildcards: ? and […]. The question mark matches any single character. For example,

email?

-Matches any filename consisting of “email” followed by exactly one character, including email1 but not email or email.1.

Brackets match any one of a set of characters that they enclose. For example,

[Jj]mf

Matches either of the filenames Jmf or jmf.

You can indicate a range or sequence of characters in brackets with a -. For example,

output[a-c]

Matches outputa, outputb, and outputc, but not outputd.

The range includes all characters in the ASCII character sequence from the first to the last. For example,

[A-N]

Matches any of the uppercase characters between A and N.

[a-z]

Matches any lowercase character.

[A-z]

Includes all upper- and lowercase characters.

[0–9]

Includes all digits.

You can use more than one of these wildcards at a time. For example,

[Rr]esearch*

Matches any filename that starts with Research or research.

The wildcard characters are the same for all the shells (sh, csh, bash, etc).

The shell’s use of wildcards to match filenames is somewhat similar to the regular expressions used by many UNIX System commands, including ed, grep, and awk. Wildcards are not the same as regular expressions, however. (The meaning of the * character in particular is different.) Regular expressions are discussed at several points later in this book, including the section on grep in Chapter 19.

Wildcards and Hidden Files

There is one important exception to the statement that * matches any sequence of characters. It does not match a . (dot) at the beginning of a filename. As discussed in Chapter 3, files with names beginning with . are treated as hidden files. They are used to hold information that is needed by the system or by particular commands, but that you are not usually interested in seeing.

To match a filename beginning with . (dot), you have to include a dot in the pattern. The following command will display the files profile and old_profile but will not print out your .profile:

$ cat *profile

If you want to view your .profile, the following command will work:

$ cat .pro*

How Wildcards Work

When the shell processes a command line, it replaces any word containing filename wildcards with the matching filenames, in sorted order. For example, suppose your current directory contains files named note1.tmp, note2.tmp, and note3.tmp. If you want to remove all of these, you could type the following short command:

$ rm *.tmp

Before rm is run, the shell replaces *.tmp with all of the matching filenames: note1.tmp, note2.tmp, and note3.tmp. The shell then passes these arguments to rm exactly as if you had typed them.

If no filename matches the pattern you specify, the shell makes no substitution. Instead, it passes the wildcard to the command as if it were part of a regular filename. So if you type

$ cat *.bk

and there is no file ending in .bk, the cat command will look for a (nonexistent) file named *.bk. When it doesn’t find one, it will give you an error message, as follows:

cat: cannot open *.bk

Whether you get an error message when this happens, and if so, what it says, depends on the command you are running. If you used the same wildcard with the command vi, the result would be the creation of a new file with the name *.bk. Although this doesn’t produce an error message, it is probably not what you wanted.

As noted in Chapter 3, the power of the * wildcard can cause problems if you are not careful in using it. If you try to enter the command

$ rm temp*

but accidentally type

$ rm temp *

you will remove all of the visible files in the current directory

Standard Input and Output

In Chapter 3, you saw that the output of a command can be sent to your screen, stored in a file, or used as the input to another command. Similarly, most commands accept input from your keyboard, from a stored file, or from the output of another command. This flexible approach to input and output is based on the UNIX System concepts of standard input and standard output, or standard I/O.

Figure 4–1 shows a command that uses standard I/O. The command gets its input through the channel labeled “standard input.” That input can come from your keyboard (the default), a file, or a command. Similarly, the command delivers its output through the channel labeled “standard output.” The output might go to your screen (the default), a file, or another command.

Image from book
Figure 4–1: A model for standard input and output

The command doesn’t need to know which of these sources the input comes from, or where the output goes. It is the shell that sets up these connections, according to the instructions in your command line. It does this through the I/O redirection mechanisms, which include pipes and file redirection.

A typical use of the pipe feature is the following command:

$ man find lp

This uses a pipe to send the output from the man command to the lp command, in order to print a hard copy of the manual page for find.

An example of file redirection is the following command:

$ man find > temp

This saves the output from the man command as the file temp.

Table 4–1 lists the symbols used to tell the shell where to get input and where to send output. These are called the shell redirection operators.

Table 4–1: Shell Redirection Operators

Symbol

Example

Function

|

cmd1 | cmd2

Run cmd1 and send output to cmd2

>

cmd > file

Send output of cmd to file

>>

cmd >> file

Append output of cmd to file

<

cmd < file

Take input for cmd from file

/dev/stdin

cmd /dev/stdin

Take input from keyboard

2>

cmd 2> errorfile

Send standard error to errorfile (ksh, bash)

2>&1

cmd > msgs 2>&1

Send both output and standard error to msgs (sh, ksh, and bash)

/dev/tty

(cmd > /dev/tty) >& error

Redirect output to screen, and error to error (csh, tcsh, and bash)

>&

cmd >& msgs

Send both output and errors to msgs (csh, tcsh, and bash)

Using Pipes

The pipe symbol (|) tells the shell to take the standard output of one command and use it as the standard input of another command. Using pipes to join individual commands together in pipelines is an easy way to use a sequence of simple commands to carry out a complex task.

For example, suppose you want to know if the user named sue is logged in. One way to find out would be to use the who command to list all of the users currently logged in, and to look for a line listing “sue” in the output. However, on a large system there could be many users-enough to make it difficult to find a specific name in the list.

A better solution is to use a pipe to redirect the output of who to grep. As explained in Chapter 19, the grep command searches through its input and prints the lines that match a target pattern.

$ who | grep sue

sue tty1 Mar 29 23:16

In this case, grep searches for the string “sue” in its standard input (which is the output of who) and prints any lines that contain it. The output shows that sue is currently logged in.

Piping the output from one command to another is like using a temporary, intermediate file to hold the output of the first command, and then using it as the input to the second, but the pipe mechanism is easier and more efficient. A typical use of pipes is to send the output of a command to a printer:

$ pr *.prog | lp

This pipeline uses lp to print a hard copy of all files with names ending in .prog. But it first uses the formatting command pr to attach a simple header to each file.

You can use pipes to create pipelines of several commands. For example, this pipeline of three commands prints the names of files in the current directory that have the string “jmf” somewhere in the listing:

$ ls -1 | grep jmf | lp

To make a long pipeline more readable, you can type the commands in it on separate lines. If the pipe symbol appears at the end of a line, the shell reads the command on the next line as the next element of the pipeline. The previous example could be entered as

$ ls -1 |

> grep jmf |

> 1P

Note that the > at the beginning of the second and third lines in this example is printed by the shell, not entered by the user. The > is not the file redirection operator. It is the shell’s default secondary prompt,PS2. The shell uses the secondary prompt to remind you that your command has not been completed and that the shell is waiting for more input. PS2 and other shell variables are discussed in the section called “Shell Variables” later in this chapter.

Output Redirection to a File

The > redirection operator sends the output of a command to a file. For example,

$ ls -l > filelist

causes the shell to save the output of ls -l as the file filelist. If a file with that name already exists in the current directory, it is overwritten-its contents are emptied and replaced by the output of the command. If the file filelist does not exist, the shell creates it before running the command.

As you can see, the > file redirection operator is very similar to the | command redirection operator. The only difference is that > is always followed by a filename, while a | is always followed by a command.

The >> operator appends data to a file without overwriting it. That means that if the file already exists, the new data will be added on to the end. In this example,

$ cat notes.jan >> research

the shell redirects the standard output from cat and appends it to the file named research. The result is that the contents of notes.jan are added to the end of research, without destroying any other information in the file. As before, if the file doesn’t exist, it will be created.

Input Redirection from a File

Just as you can use the greater than (or right arrow) symbol, >, to redirect standard output, you can use the less than (or left arrow) symbol, <, to redirect standard input. The < symbol tells the shell to interpret the filename that follows it as the standard input to a command.

For example, as you saw in Chapter 2, the command mail (or mailx) can be used to send a message that you type on standard input:

$ mail anita@bio.ca.edu

Subject: Congratulations!

I heard that you finished.

Nice work!

CTRL-D

You can use the < redirection operator to replace the keyboard with a file as the standard input. In this case, the mail command will send the contents of that file, instead of waiting for you to enter a message at the keyboard:

$ mail anita@bio.ca.edu < note

The < tells the shell to run mail with the contents of note as its standard input.

You can redirect input and output at the same time. The following example uses the sort command to take the information in source, alphabetize it, and put the output in dest:

$ sort < source > dest

The order in which you indicate the input and output files doesn’t matter, so the following example has the same result:

$ sort > dest < source

Many commands provide a way for you to specify an input file directly as a filename argument. For example, cat allows you to name one or more input files as arguments. Thus, the commands

$ cat < script.pl

and

$ cat script.pl

both display the same file on your screen. In the first case, the shell connects script.pl to the standard input that cat reads. In the second case, the cat command opens the file script.pl and reads its input from it.

Standard Input from the Keyboard

You can also force commands to accept input from the keyboard. The logical filename /dev/stdin refers to standard input. When used as an argument to a command, it causes the shell to send the input from the keyboard to the command, in place of a file.

For example, the command sort is used to alphabetize the lines in a file. If you give it the argument /dev/stdin, it will sort the lines you type in from the keyboard:

$ sort /dev/stdin > guestlist

Ben

Robin

Dan

Marissa

CTRL-D

This will sort the names alphabetically and save them in the file guestlist. It reads everything you type up to the CTRL-D, which indicates the end of input from your keyboard.

You can also mix input from the keyboard with input from a file. Suppose you want to personalize a form letter by combining typed information (such as the recipient’s name) with stored text:

$ cat /dev/stdin form_letter > output

Dear Sue,

CTRL-D

This concatenates input from your keyboard (standard input) with the contents of form_letter, and saves the result in output.

Some commands, including cat, interpret a minus sign (−) as a shortcut for /dev/stdin. The previous example could also be written as

$ cat - form_letter > output

Standard Error

In addition to standard input and standard output, the shells provide a third member of the standard I/O family, standard error. Standard error provides a second logical channel that a program can use to communicate with you, separate from standard output. As the name suggests, standard error is normally used for displaying error messages. For example, cat prints the following message to standard error when you try to read a nonexistent file:

$ cat snetmail

cat: snetmail: No such file or directory

Standard error is also used to display prompts, labels, help messages, and comments. If you try to delete a file for which you don’t have write permission, rm uses standard error to display the following message telling you that the file is protected:

$ rm save_file

rm: remove write-protected regular file 'save_file'?

Normally you don’t see any difference between standard output and standard error, but when you redirect standard output to a file, standard error remains connected to your screen. This makes sense, since even when you send the output of a command to a file, you probably still want to see any error messages that might occur. For example, if you try to cat a file that doesn’t exist into another file, the error message shows up on your screen, and you know the command didn’t work:

$ cat snetmail > mailcopy

cat: snetmail: No such file or directory

Redirecting Standard Error in sh, ksh, and bash

The Bourne-compatible shells (sh, ksh, and bash) allow you to redirect standard error. You use the same > and >> symbols as for redirecting standard output, but you precede them with a 2. The command

$ cat *.pl 2> errors

will send any error messages to the file errors. Similarly,

$ grep Robin guestlist > names 2>> errors

sends the output of the command to the file names, and appends any error messages to errors. Note that no space is allowed between the 2 and the >>.

Using the number 2 to redirect standard error may seem rather arbitrary The 2 is an example of a file descriptor, a number that a program uses to refer to a file. The default file descriptors are

File Descriptor

Name

Default Assignment

0

standard input

your keyboard

1

standard output

your screen

2

standard error

your screen

To redirect standard output and standard error to the same place, you can try to use the same filename twice, like this:

$ grep Robin guestlist > names 2> names

However, this will overwrite the output file names if there is both output and an error message. A better solution is to use the command

$ grep Robin guestlist > names 2>&1

In this example, the 2>&1 at the end says “send standard error to the same place as standard output.” So both standard output and standard error end up in the file names. Note that the 2>&1 must be written just like that, without any spaces, and it must go at the end of the line, or the shell will not parse it correctly

Redirecting Standard Error in csh and tcsh

The C shell and extended C shell also allow you to redirect standard error, but a little differently than the Bourne-compatible shells. To redirect both standard output and standard error to the same file, use the symbol >& (or >>&, to append to the file), as in the following example:

% cat *.pl >& perlscripts

This will also work in bash, but not in sh or ksh.

There isn’t really a way to redirect just standard error in the C shells. A work-around is to first redirect the standard output to a file, and then redirect the remaining standard error to a separate file, as shown:

$ (ls -1 > /dev/tty) >& errorfile

In this example, the logical filename /dev/tty refers to your current terminal (your screen, or the window in which you are running to shell). The command ls -1 > /dev/tty causes the shell to redirect the standard output from ls to the screen. This is put in parentheses to indicate that it should happen first. Once the standard output has been redirected to /dev/tty, the >& errorfile will cause the shell to redirect the standard error to errorfile.

Running Commands in the Background

Ordinarily, when the shell runs a command, it waits until the command is completed before it resumes its dialog with you. During this time, you cannot communicate with the shell-it does not prompt you for input, and you cannot issue another command. In some cases, you may want to start one command and then run another immediately, without waiting for the first to finish. This is especially true when you run a command that takes a long time to finish-while it’s working, you can be doing something else.

An & symbol at the end of a command line directs the shell to execute the command in the background. When a command is run in the background, the system will continue to process it while the shell prompts you for another command. As an example, formatting a long document using the troff text formatter (discussed on the companion web site) often takes a long time, so you ordinarily run troff in the background:

$ troff big_file &

[1] 1413

The shell acknowledges your background command with a message that contains two numbers. The first number, which is enclosed in brackets, is the job ID of this command. It is a small number that identifies which of your current jobs this is. The section “Job Control” later in this chapter explains how you can use this number with the shell job control features to manage your background jobs. The other number, “1413" in this example, is the process ID. It is a unique number that identifies this process among all of the processes in the system. Process IDs and their use are discussed in Chapter 11.

You can also run a pipeline of several commands in the background. The following command runs the troff formatter on big_file and sends the result to the printer:

$ troff big_file | lp &

All of the programs in a pipeline run in the background, not just the last command.

Running Windowed Commands

If you are using a graphical environment, such as GNOME or KDE, you will at times find yourself running commands that open in new windows. For example, you can open a new xterm window by typing

$ xterm

However, your previous shell will then wait for you to close that window, ending the xterm command, before accepting any new input. In general, it’s better to run the command in the background. So, to open a firefox browser window, you would enter

$ firefox &

This will allow you to switch back and forth between your various windows, using them all simultaneously

Standard 1/0 and Background Jobs

When you run a command in the background, the shell starts it, gives you a job identification message, and then prompts you for another command. It disconnects the standard input of the background command from your keyboard, but it does not automatically disconnect its standard output from your screen. So output from the command, whenever it occurs, shows up on your screen. Having the output of a background command suddenly dumped on your screen while you are entering another command, or using another program, can be very confusing. Thus when you run a command in the background, you also usually redirect its output to a file:

$ troff big_file > output &

When you run a command in the background, you should also consider whether you want to redirect standard error. You may sometimes want the standard error to appear on your screen so that you find out immediately if the command is successful or not, and why On the other hand, if you do not want error messages to show up on your screen, you should redirect standard error as well as standard output-either to the same file or to a different one.

The find command can be used to search through an entire directory structure for files with a particular name. This is a command that can take a lot of time, and you may want to run it in the background. In addition, find may generate messages through standard error if it encounters directories that you do not have permission to read. The following example uses the find command to search for files whose names end in .backup. It starts the search in the current directory, “.”, puts the filenames that it locates into the file called backupfiles, puts error messages in the file find.err, and runs the command in the background. This is how the command line would look in the Bourne-compatible shells (sh, ksh, and bash):

$ find . -name "*.backup" -print > backupfiles 2> find.err &

or in the C shells (csh and tcsh):

$ (find . -name "*.backup" -print > backupfiles) >& find.err &

To discard standard error entirely, redirect it to /deυ/null, which will cause it to vanish. (/deυ/null is a device that does nothing with information sent to it. It is like a black hole into which input vanishes. Sending output to /deυ/null is a handy way to get rid of it.) The command

$ troff big_file > output 2> /dev/null &

runs the troff command in the background on big_file, sends its output to output, and discards error messages. In csh or tcsh, this would look like

$ (troff big_file > output) >& /dev/null &

Logging Off with Active Jobs

If you run a command that takes a very long time, you may want to log out before it finishes. Ordinarily, if you log out while a background job is running it will be terminated. However, you can use the nohup (no hang up) command to run a job that will continue running even if you log out. For example,

$ nohup find / -name "lost_file" -print > lostfound 2>&1 &

in the Bourne-compatible shells, or

$ nohup find / -name "lost_file" -print >& lostfound &

in the C shells, allow find to continue even after you quit. This command starts looking in the root directory of the file system for any files named lost_file. Any pathnames that are found are put in the file named lostfound, along with any error messages. The whole thing is run in the background, to allow you to enter other commands or log out.

When you use nohup, you should be sure to redirect both standard output and standard error to files, so that when you log back in you can find out what happened. If you do not specify output files, nohupautomatically sends command output, including standard error, to the file nohup.out.

Job Control

Because the UNIX System provides the capability to run commands in the background, you sometimes have two or more commands running at once. There is always one job in the foreground. This may be the shell, when it is prompting you for input, or it may be any other command to which your keyboard input is connected, such as a text editor. In addition, there may be several jobs running in the background at any given time.

Job control is a crucial feature of the modern UNIX shells that was first introduced in csh and is also found in tcsh, ksh, and bash. The commands and syntax are for the most part identical in all four shells. The job control commands allow you to terminate a background job (kill it), suspend a background job temporarily (stop it), resume a suspended job in the background, move a background job to the foreground, and suspend a foreground job.

The jobs command displays a list of all your jobs, such as

$ jobs

[1] + Running find /home/jmf -print > files &

[2] Stopped vi filesplit.tcl

[3] - Stopped grep supv * | awk -f fixes > data &

The output shows your current foreground and background jobs, as well as jobs that are stopped or suspended. In this example, there are three jobs. The number at the beginning of each line is the job ID. Job 1 (the find command) is running in the background. Jobs 2 and 3 are both stopped. The plus sign (+) indicates the current job (the most recently started or restarted); minus (−) indicates the one before that.

You can suspend your current foreground job by typing CTRL-Z. This halts the program and returns you to your shell. For example, if you are running a command that is taking a long time, type CTRL-Z to suspend it so that you can do something else. The job will essentially be paused-that is, it will not do anything until you resume the job, but you can resume it at any time.

Suppose you ran the find command, and forgot to tell it to run in the background. You can use CTRL-Z to suspend it, but now you need to tell it to resume. The command bg will cause a job to run in the background. If the job is stopped, it will resume. By default, bg acts on the current job. To resume an older job, refer to it by the job ID:

$ j obs

[1]+Stopped find /home/jmf -print > files

$ bg %1

[1]+find /home/jmf -print > files &

You use the % sign to introduce the job identifier, so %1 refers to job 1.

Similarly, the command fg causes an existing job to run in the foreground. This can be used to resume a suspended job, or to move a background job to the foreground.

You can terminate any of your background or suspended jobs with the kill command. For example,

$ kill %2

terminates job number 2. Once a job is killed, it is gone-it can’t be resumed.

In addition to the job ID number, you can use the name of the command to tell the shell which job to kill. For instance,

$ kill %troff

kills a troff job running in the background. If you have two or more troff commands running, this will kill the most recent one.

The stop command halts execution of a background job but doesn’t terminate it, just like CTRL-Z for foreground jobs. The command sequence

$ stop %find

$ fg %find

stops the find command that is running in the background and then resumes executing it in the foreground. The stop command is supported by csh, tcsh, and ksh, but not by bash. In bash, you would use the following command line instead:

$ kill -s STOP %find

Table 4–2 summarizes the shell job control commands.

Table 4–2: Job Control Commands

Command

Effect

jobs

List all jobs

CTRL-Z

Suspend current (foreground) process

bg %n

Resume stopped job in background

fg %n

Resume job in foreground

stop %n

Suspend background job

In bash, use kill −s STOP %n

kill %n

Terminate job

Configuring the Shell

When your login shell starts up, it looks for certain files in your home directory. These files contain commands that can be used to configure your working environment. The particular files it looks for depend on which shell you are using:

§ sh runs the commands in a configuration file called .profile.

§ ksh also uses the .profile file. In addition, you can set a variable in your .profile to cause it to read the commands in a second file. The variable is called ENV. By convention, that file is often called .kshrc.

§ bash uses the file .bash_profile. If that file does not exist, it will look for the file .profile, instead. The .bash_profile often contains a line that causes bash to run the commands in a second file, bashrc.When you log out of bash, it will run the commands in .bash_logout.

§ csh looks for a file called .login. It will also run commands in .cshrc. When you log out of the C shell, it will run the commands in the file .logout.

§ tcsh uses the .login file as well. It also looks for .tshrc. If that file does not exist, it will look for .cshrc instead. Like the C shell, tcsh will run the .logout file when you log out.

This may all sound very confusing, but these configuration files all work in pretty much the same way Each of these files is actually an example of a shell script. They contain commands or instructions for the shell. The commands include settings that allow you to customize your environment.

The first file that the shell reads (.profile, .bash_profile, or .login) contains variables or settings that you want to be in effect throughout your login session. The section “Shell Variables” later in this chapter describes these variables. The file might also include commands you want to run at login, such as cal (to display a calendar for the current month) or who (to show the list of users who are currently logged in).

The newer shells support a second configuration file, which is used for defining command aliases,functions, and certain shell variables. This file is usually called .kshrc, .bashrc, .cshrc, or .tcshrc, depending on which shell you are using. (The rc stands for “read commands.” By convention, programs often look for initialization information in files ending in rc. Other examples are .exrc, which is used by vi, and .mailrc,used by mailx.)

Interactive Shells

You can start another shell after you log in by using the name of the shell as a command; for example, to start the Korn shell, you could type ksh at the command prompt. This type of shell is not a login shell, and you do not have to log in again to use it, but it is still an interactive shell, meaning that you interact with the shell by typing in commands (as opposed to using the shell to run a script, as discussed in Chapter 20). The instances of the shell that run in a terminal window when you are using a graphical interface are also interactive non-login shells. When you start a non-login shell, it does not read your .profile, .bash_profile, or .login file (or your .logout file), but it will still read the second shell configuration file (such as .bashrc). This means that you can test changes to your .bashrc by starting another instance of the shell, but if you are testing changes to your .profile or .login, you must log out and then back in to see the results.

Sample Configuration Files

If you define a variable by typing its new value on the command line, it will return to its previous value the next time you log in. In order to keep important variables such as PATH, PS1, and TERM defined every time you log in, they are usually included in one of your configuration files. Here are a few examples of those files. Don’t worry if the commands don’t make sense yet-you can come back to this later, after reading the sections “Shell Variables” and “Command Aliases.”

If you want to edit your shell configuration file, you will probably need to use a text editor, such as vi or emacs. These programs are explained in detail in Chapter 5.

The shell does not try to interpret lines that begin with #, or any text following a #. You can use this to include comments in your configuration files. A comment is just a note to yourself, to help you remember how the commands in your file work.

Sample .bash profile

Every time you log in, bash reads your .bash_profile. This file includes definitions of environment variables that will be shared with other programs and commands such as who that you want to run at the beginning of each login session. A typical .bash_profile might look some-thing like this:

# .bash_profile - example

# set environment variables

export TERM=vt100

export PATH=$PATH:/sbin:/usr/sbin:$HOME/bin

export MAILCHECK=30

# allow incoming messages from other users

mesg y

# make sure backspace works

stty erase "^H"

# show all users who are currently logged in

who

# load aliases and local variables

. -/.bashrc

The last line executes your .bashrc file. If you leave it out, bash will not read your aliases and variable definitions when you log in.

A .profile for ksh would look almost identical. The only important change would be to the lines at the end that execute the .bashrc file. These would be replaced by a line like

export ENV=$HOME/.kshrc

This will cause ksh to look for other configuration settings in the file .kshrc in your home directory Note that ENV may be set to any filename, although $HOME/.kshrc is a common choice.

Sample .bashrc File

When you start an interactive bash shell after logging in (e.g., by opening an xterm window), it reads the commands in your .bashrc file. This file includes commands and definitions that you want to have executed every time you run a shell-not just at login. The .bashrc file defines local shell variables (but not environment variables, which belong in .bash_profile), shell options, and command aliases. It might look something like this:

# .bashrc file-example

# set shell variables (such as the prompt)

PS1="\u \w> "

# set shell options

set −o noclobber

set −o emacs

set +o notify

# set default file permissions

umask 027

# define aliases

alias lg='ls -g -color=tty'

alias r='fc -s'

alias rm='rm -i'

alias cp='cp -r'

alias wg='who | grep'

alias hibernate='sudo apm -s'

A configuration file for ksh could look like this, as well. However, the line PS1=“\u \w>” in this example would have to be changed to PS1=‘$LOGNAME $(pwd)> ’, because ksh doesn’t support the bashshortcuts for defining the prompt.

Sample .login File

As the name suggests, tcsh reads the .login file only when you log in. Your .login file should contain commands and variable definitions that only need to be executed at the beginning of your session. Examples of things you would put in .login are commands for initializing your terminal settings, commands such as date that you want to run at the beginning of each login session, and definitions of environment variables.

The following is a short example of what you might put in a typical .login file:

# .login file-example

# show number of users on system

echo "There are" who wc -l "users on the system"

# set terminal options-- in particular,

# make sure that the Backspace key works

stty erase "^H"

# set environment variables

setenv term vt100

setenv mail ( 60 /var/spool/mail/$user)

These examples illustrate the use of setenv, the C shell command for defining environment variables. setenv and its use are discussed further later on, in the section “csh and tcsh Variables.”

Sample .tcshrc File

The difference between .tcshrc and .login is that tcsh reads .login only at login, but it reads .tcshrc both when it is being started up as a login shell and when it is invoked as an interactive non-shell. The .tcshrcfile includes commands and definitions that you want to have executed every time you run a shell-not just at login.

Your .tcshrc should include your alias definitions, and definitions of variables that are used by the shell but are not environment variables. Environment variables should be defined in .login.

# .tcshrc file-example

# set shell variables

set path = ($path /sbin /usr/sbin /usr/bin /$home/bin)

set prompt = "[%n@%m %c] tcsh % "

# turn on ignoreeof and noclobber

# turn off notify

set ignoreeof

set noclobber

unset notify

# define aliases

alias lsc ls -Ct

alias wg 'who | grep'

alias rm rm -i

alias cp cp -r

alias hibernate sudo apm -s

# set permissions for file creation

umask 027

This sample file includes C shell variable definitions and aliases, both of which are explained in the following sections.

Shell Variables

The shell provides a mechanism to define variables that can be used to hold pieces of information. Shell variables can be used to customize the way in which programs (including the shell itself) interact with you. This section will describe some of the standard variables used by the shell and other programs, and explain what they do for you.

Table 4–3 summarizes the commands for assigning variables and aliases in the various shells.

Table 4–3: Assigning Variables and Aliases

sh, ksh, or bash

csh or tcsh

Effect

VAR=value

set var=value

Assign a value to a variable

$VAR

$var

Get the value of a variable

set

set

List shell variables

unset VAR

unset var

Remove a variable

env

env

List all environment variables

VAR=value; export VAR export VAR=value

setenv var value

Create an environment variable

unset VAR

unsetenv var

Remove an environment variable

set -o

View shell options

set -o option

set option

Turn on a shell option

set +o option

unset option

Turn off an option

alias name=value

alias name value

Create a command alias

unalias name

unalias name

Remove an alias

Variables in sh, ksh, and bash

You define a shell variable by typing a name followed by an=sign and a value. For example, you could create a variable called PROJDIR to save the pathname for a directory you use often:

$ PROJDIR=/home/nate/Work/cs106x/Proj_3/lib/Source

To assign a value with a space in it, use quotes, like this:

$ FILELIST='graphics.c strings.c sorting.c'

In the Bourne-compatible shells, variable names are conventionally written in uppercase letters, although you can use lowercase names as well. As with filenames, variable names are case-sensitive, so the shell will treat PROJDIR and projdir as two different variables.

To get the value of a shell variable, precede the variable name with a dollar sign, $. You can print a variable with the echo command, which copies its standard input to its standard output.

$ echo $PROJDIR

/home/nate/Work/cs106x/Proj ect_3/lib/Source

You can also use variables in command lines. When the shell reads a command line, it interprets any word that begins with $ as a variable, and replaces that word with the value of the variable. For example,

$ cp $PROJDIR/graphics.c .

will copy the file /home/nate/Work/cs106x/Project_3/lib/Source/gmphics.c to the current directory

You can use the set command to view all of your current shell variables and their values. A typical output from set might look like

$ set

COLUMNS=80

HOME=/home/raf

HOSTNAME=localhost.localdomain

MAIL=/var/spool/mail/raf

MAILCHECK=30

PATH=/usr/local/bin:/bin:/usr/bin:/home/raf/bin

PROJDIR=/home/nate/Work/cs106x/Project_3/lib/Source

PS1='$ '

PS2='> '

SHELL=/bin/bash

TERM=vt100

Most of the variables on this list are standard shell variables that will be discussed later in this section. The exception is PROJDIR, which is a user-defined variable with no special meaning.

To remove a variable, use the command unset, as in

$ unset PROJDIR

$ echo $PROJDIR

$

Environment Variables in sh, ksh, and bash

When you run a command, the shell makes certain shell variables and their values available to the program. The program can then use this information to customize its actions. The collection of variables and values provided to programs is called the environment.

Your environment includes variables set by the system, such as HOME, LOGNAME, and PATH (described in the next section). You can display your environment variables with the command env:

$ env

HOSTNAME=localhost.localdomain

SHELL=/bin/bash

MAIL=/var/spool/mail/raf

PATH=/usr/local/bin:/bin:/usr/bin:/home/raf/bin

PWD=/home/raf/Proj ect

PS1='$ '

HOME=/home/raf

LOGNAME=raf

To make variables that you define yourself available to commands as part of the environment, they must be exported. For example, TERM is a common shell variable that is not always automatically part of the environment. To make it available to commands, you first define, then export it:

$ TERM=vt100

$ export TERM

A shortcut that does the same thing in ksh or bash is

$ export TERM=vt100

Common Shell Variables in sh, ksh, and bash

The following is a short summary of some of the most common shell variables, including those set automatically by the system.

§ HOME contains the absolute pathname of your login directory HOME is automatically defined and set to your login directory as part of the login process. The shell itself uses this information to determine the directory to change to when you type cd with no argument.

§ LOGNAME contains your login name. It is set automatically by the system.

§ PWD is a special variable that gets set automatically to your present working directory You can use this variable to include your current directory in the prompt or in a command line.

§ PATH lists the directories in which the shell searches to find the program to run when you type a command. A default PATH is set by the system, but many users modify it to add additional command directories.

A typical example of a customized PATH, in this case for user anita, is the following:

PATH=$PATH:/sbin:/usr/bin:/home/anita/bin

This setting for PATH indicates that when you enter a command, the shell first searches for the program in the default path (the previous value of the PATH variable), then in the directory /sbin; then in /usr/bin;and finally in the bin subdirectory of the user’s login directory. In these pathnames, bin stands for binaries, meaning executable programs. The directories /bin, /usr/bin, /sbin, and /usr/sbin are common locations for important commands and programs. If these directories are not in your default path, you may want to add them.

It is also common to create a subdirectory called bin in your home directory Instead of adding every directory that contains a command or executable to your PATH, you can create symbolic links to the commands in your bin directory (Remember to give yourself execute permission for the symbolic links. Creating links, and adjusting the permissions on them, is discussed in Chapter 3.)

§ CDPATH is similar to PATH. It lists, in order, the directories in which the shell searches to find a subdirectory to change to when you use the cd command. This means that you can “jump” from one directory to another without typing the full pathname. A good choice of CDPATH can make it much easier to move around in your file system.

§ ENV is a very important variable in the Korn shell. It tells ksh where to find the environment file that it reads at startup. If you are using ksh, the ENV variable should be set in your .profile. A common value is $HOME/.kshrc.

§ PS1 defines your prompt. The default value is $. Similarly, PS2 defines your secondary prompt, and has a default value of >. Most users like to customize the prompt by adding information such as the current working directory For example,

§ $ PS1='$LOGNAME $PWD> '

saul /home/saul/Email>

§ TMOUT tells the shell how many seconds to wait before timing out. If you don’t type a command within that period of time, the shell logs you off. This variable is not supported by sh; you can define it, but it won’t do anything. By default, TMOUT is set to 0, meaning that it will never time out.

§ MAIL contains the name of the file in which your newly arriving mail is placed. The shell uses this variable to notify you when new information is added to this file. This variable is set automatically when you log in.

§ MAILCHECK tells the system how frequently, in seconds, to check for new mail. By default, this is set to 60 in bash, 600 in ksh.

§ HISTSIZE tells the shell how many commands to save in your history file (see the section “Command History” later in this chapter). Not supported by sh. The default value for HISTSIZE in bash is 500; in ksh it is 128.

§ HISTFILE (which is also not supported by sh) specifies the location of your history file, such as .history (ksh) or .bash_history (bash).

§ TERM is used by vi and other screen-oriented programs to get information about the type of terminal you are using. This information is necessary to allow the programs to match their output to your terminal’s capabilities, and to interpret your terminal’s input correctly A common value is “vt100”.

§ SHELL contains the name of your shell program. This is used by some interactive commands to determine which shell program to run when you issue a shell escape command. (A shell escape temporarily interrupts the program and runs a shell for you.) This variable is typically set automatically at login.

§ VISUAL is a variable used only by ksh. It can be used to determine which command-line editor the shell uses, although this can also be done with an option setting. See the section “Command-Line Editing,” later in this chapter.

Shell Options in ksh and bash

The Korn shell and bash provide a number of options that turn on special features. To turn on an option, use the set command with -o (option) followed by the option name. To view your current option settings, use set -o by itself.

The noclobber option prevents you from overwriting an existing file when you redirect output from a command. This can save you from losing data that may be difficult or impossible to replace. You can turn on noclobber with set:

$ set -o noclobber

Suppose noclobber is set, and your current directory contains a file named temp. If you try to redirect the output of a command to temp, you get a warning:

$ ls -1 > temp

temp: file exists

You can tell the shell that you really do want to overwrite the file by putting a bar (pipe symbol) after the redirection symbol, like this:

$ ls -1 >| temp

To turn off an option, use set +o, as in

$ set +o noclobber

The ignoreeof feature prevents you from accidentally logging yourself off by typing CTRL-D. If you use this option, you must type exit to terminate the shell.

The notify option causes the shell to notify you as soon as your background jobs complete, without waiting for you to finish your current command. Some users find this useful, but others may not like getting interrupted by notifications.

You can also use set to turn on the screen editor option for command-line editing (discussed later in this chapter). The following line,

$ set -o emacs

tells the shell that you want to use the emacs-style command-line editor. You could use vi instead. (The text editors vi and emacs are covered in Chapter 5.)

Variables in csh and tcsh

As with the Bourne-compatible shells, the C shell and tcsh provide variables, including both standard system-defined variables and ones you define yourself. However, there are a number of differences in the way variables are defined, what they are named, and how they are used.

In csh and tcsh, you define a variable with the set command, as shown here:

% set projdir = /home/nate/Work/cs106x/Proj_3/lib/Source

If the value has a space in it, you must put quotes around it, like this:

% set filelist = 'graphics.c strings.c sorting.c'

C shell variables are generally lowercase. If you do create variables with names in uppercase, remember that variable names are case-sensitive, so the shell will treat filelist and FileList as two different variables.

To get the value of a shell variable, type a $ followed by the variable name. Since the echo command prints its standard input to its standard output, the command line echo $VARNAME will print the value of a variable:

% echo $projdir

/home/nate/Work/cs106x/Project_3/lib/Source

You can also use variables in command lines. For example,

% cp $projdir/graphics.c .

will copy the file /home/nate/Work/cs106x/Project_3/lib/Source/graphics.c to the current directory

As with the other shells, you can use the set command to view all of your current shell variables and their values, and the unset command to undefine a variable.

% unset projdir

% echo $projdir

projdir: Undefined variable.

%

Environment Variables in csh and tcsh

There are certain variables that the shell makes available to commands as part of the environment that the shell maintains. Commands can use these variables to get information such as your login name or the size of your screen. These variables are called environment variables.

To set an environment variable, use the command setenv:

% setenv term vt100

Note that, unlike defining a variable with set, you do not use an=sign when setting an environment variable with setenv.

You can view all of your environment variables with the command env. To remove a variable from the environment, use unsetenv.

Common Shell Variables in csh and tcsh

These are some of the most common C shell variables, including those set automatically by the system:

§ home is the full pathname of your login directory

§ user is your username.

§ cwd holds the full name of the directory you are currently in (the current working directory). It provides the information the pwd command uses to display your current directory.

§ path holds the list of directories the C shell searches to find a program when it executes your commands. It corresponds to the PATH variable in the Bourne-compatible shells.

By default, path is set to search first in your current directory, and then in /usr/bin. To add the directories /bin, /sbin, /usr/sbin, and your own bin directory to path, put a line like this in your .cshrc or .tcshrc file:

path = ($path /bin /sbin /usr/sbin $home/bin)

Because these directories are all common locations for commands and programs, you may want to add them to your path if they are not there already The directory $home/bin is often used to hold symbolic links to other commands. This way, you can run the commands without having to add a long list of directories to your path.

Unlike ksh or bash, which use a colon to separate items in the path, the C shell uses parentheses to group the different directories included in path. This use of parentheses to group multivalued variables is a general feature of the C shell. Other standard C shell variables with multiple values include cdpath and mail.

§ cdpath is the C shell equivalent of the CDPATH variable. It lists in order the directories in which csh or tcsh searches when you use the cd command. This allows you to move from one directory to another without typing the full pathname.

§ The prompt variable allows you to customize the prompt. You can set it to include information such as your username or working directory For example, you can set the tcsh prompt like this:

§ % set prompt="[%n@%m %c] tcsh % "

[liz@localhost ~/Email] tcsh %

The default C shell prompt is %. Note that unlike sh, the C shell does not allow you to redefine the secondary prompt.

§ The variable mail tells the shell how often to check for new mail, and where to look for it.

% set mail = ( 60 /var/spool/mail/liz )

This setting causes csh to check the file /var/spool/mail/liz every 60 seconds. If new mail has arrived in the directory specified in mail since the last time it checked, csh displays the message, “You have new mail.”

§ history is the number of commands the shell saves in your history file.

§ histfile is the name of the history file.

§ term identifies your terminal type. A common value is vt100.

Shell Options in csh and tcsh

The C shell uses special variables called toggles to turn certain shell features on or off. Toggle variables are variables that have only two settings: on and off. When you set a toggle variable, you turn the corresponding feature on. To turn it off, you use unset. Important toggle variables include noclobber, ignoreeof, and notify.

The noclobber toggle prevents you from overwriting an existing file when you redirect output from a command. To turn on the noclobber feature, use set as shown in this example:

% set noclobber

Suppose noclobber is set, and that a file named temp already exists in your current directory If you try to redirect the output of a command to temp, you get a warning like this:

% ls -1 > temp

temp: file exists

The preceding example tells you that a file named temp already exists and that your command will overwrite it. You can tell the shell that you really do want to overwrite a file by putting an exclamation mark after the redirection symbol:

% ls -1 >! temp

The ignoreeof toggle prevents you from accidentally logging yourself off by typing CTRL-D. This is a good command to add to your .cshrc or .tcshrc file:

% set ignoreeof

The notify toggle informs you when a background job finishes running. If notify is set, the shell will display a message when a background job is complete. This toggle is set by default, but if you do not want to get job completion messages while you are in the middle of something else, you can unset it, as shown here:

% unset notify

Command Aliases

Aliases are a very convenient feature introduced in csh and supported by tcsh, ksh, and bash. A command alias is a word linked to a block of text that is substituted by the shell whenever that word is used as a command. You can use aliases to give command names that are easier for you to remember or to type, and to automatically include particular options when you run a command.

The syntax for defining aliases varies slightly according to the shell you are using. In the C shell and extended C shell, the following alias lets you type lg as a substitute for the longer command ls -g:

alias lg ls -g # csh or tcsh

In the Korn shell or bash, the same alias would be

alias lg="ls -g" # bash or ksh

In either case, where you enter the command

% lg

the shell replaces the alias lg with the full text of the alias, so the effect is exactly the same as if you had entered this:

% ls -g

To see a list of the aliases you have defined, type the command alias by itself.

Aliases in ksh and bash

A valuable use of aliases is to automatically include options when you issue a command. For example, in Chapter 3 you saw that using the -i (interactive) option to the commands mv, cp, and rm can prevent you from accidentally deleting or overwriting files. By adding following lines to your .kshrc or .bashrc file, you can redefine those commands so that they always run with the -i option:

alias rm="rm -i"

alias mv="mv -i"

alias cp="cp -i"

Note that, just as when you assign a variable, you must put quotes around any values that include spaces, as in “rm -i”.

Should you decide to redefine a command name like this and later discover that you need to use the command without the aliased options, you have two choices: you can temporarily unalias the command

$ unalias rm

or you can use the full pathname of the command (found using the which command)

$ /bin/rm

Alternately, of course, you could choose an alias that isn’t a command name, such as

alias cpi="cp -i"

You can use aliases with command pipelines, as in

$ alias wg="who | grep"

which would allow you to type

$ wg dbp

instead of

$ who | grep dbp

Aliases in csh and tcsh

Aliases can be used to automatically include options when you issue a command. In Chapter 3 you saw that using the -i (interactive) option to the commands mv, cp, and rm can prevent you from accidentally deleting or overwriting files. You can redefine those commands by aliasing them so that they always run with the -i option:

alias rm rm -i

alias mv mv -i

alias cp cp -i

As with variables, aliases must be defined each time you start the shell. To save aliases that you want to use every time you log in, add them to your .cshrc or .tcshrc file.

If you redefine a command name with an alias and later discover that you need to use the command without the aliased options, you have two choices: you can temporarily unalias the command

$ unalias rm

or you can use the full pathname of the command

$ /bin/rm

An alternative would be to choose an alias that isn’t a command name, such as

alias cpi cp -i

There are many more uses for aliases. You could define

$ alias wg 'who grep'

which would allow you to type

$ wg dbp

instead of

$ who | grep dbp

Note that in this example you must include quotes (') around the alias. If you do not, the shell will assign the alias wg to the command who and then try to pipe the output to grep.

Command History

Most modern shells (including ksh, bash, csh, and tcsh) keep a list of all the commands you enter during a session. This history list can be used to review the commands you have recently entered or to repeat commands you have used.

You can display a list of previously entered commands with the history command. The following is a typical history list display:

$ history

113 cd Email

114 ls -l

115 find . -name "*old" -print

116 cd Save

117 vi draft-old

118 diff draft-old sent-old

119 rm draft-old

120 history

By default, the history command lists all the commands saved in your history file (in some versions of ksh, it might list only the 16 most recent commands). You can change the number of commands the shell saves by setting a variable-HISTSIZE in bash or ksh, and history in csh or tcsh. To display only the most recent commands, run history with an argument:

$ history 3

121 cp * Backups

122 rm *.old

123 history 3

In ksh, this would be history −3.

The lines in the history list are numbered sequentially as they are added to your history list. If you prefer, you can display your history without command numbers. This is useful if you want to save a series of command lines in a file that you will later use as a shell script. In csh and tcsh, the command to do this is history -h, as in

% history -h 7 > newscript # csh or tcsh

which saves the eight most recent commands in the file newscript. In bash or ksh, the equivalent command would be

$ fc -In -7 > newscript # ksh or bash

The command history list is preserved in a file across sessions, so you can use it to review or repeat commands from previous login sessions. The name of the file is specified by a shell variable-HISTFILE in bash and ksh, histfile in csh or tcsh. In addition to viewing commands from your history list, you can use your history list to redo previous commands. This is made possible by the history substitution feature. The syntax for history substitution is significantly different in the various shells. Table 4–4 shows the similarities and differences.

Table 4–4: History Substitution

ksh

bash

csh or tcsh

Effect

history

history

history

List commands in history

history-n

history n

history n

List n most recent commands

fc-ln

fc-ln

history -h

List history without line numbers

r

fc-s

!!

Repeat previous command

r n

fc -s n

!n

Redo command number n

r -n

fc -s -n

!-n

Redo nth most recent command

r cmd

fc -s cmd

!cmd

Redo most recent instance of cmd

History Substitution in csh and tcsh

History substitution is similar to the variable substitution discussed earlier in this chapter (and to command substitution, which will be discussed later). An exclamation mark at the beginning of a line tells csh or tcsh to substitute information from your history list.

Suppose you recently used the vi editor (discussed in Chapter 5) to edit a file named cs106xProject.c. If you want to do more editing on that file, you can use the history substitution feature to redo the command without having to retype it. For example,

% !vi

vi cs106xProject.c

repeats the last command beginning with vi. Note that the command automatically supplies the name of the file in this case. In general, it repeats all of the arguments to the command.

You can use command numbers from your history list to redo commands. The exclamation mark followed by a number repeats the history list command line with that number. For example, to repeat command number 114, you would type

% !114

ls -1

A number preceded by a minus sign tells the shell to go back that many commands in the list. If the last command you entered was number 119, the following command would take you back to command 116:

% !-3

cd Save

A very useful shorthand for repeating the previous command is two exclamation marks, as in the following:

% !!

This repeats the immediately preceding command.

In any of the previous examples, you can print the command without executing it by adding :p at the end, as in

% !!:p

History substitution can also be used to edit commands, and to copy commands or arguments from your history list into your command line. Although these features can be useful, they are difficult to remember and have to some extent been replaced by command-line editing, described in the next section. If you are determined to learn the full set of history substitution commands, see http://www.npa.uiuc.edu/docs/tcsh/History_substitution.html.

History Substitution in ksh and bash

In bash, the command fc -s is used to repeat commands. In the Korn shell, the alias r is used as a more memorable shortcut for fc -s. This alias is automatically defined by the shell. To repeat your most recent command in ksh, type

$ r

In bash, this would be

$ fc -s

To use the r command in bash, just add the line

$ alias r='fc -s'

to your .bashrc.

To repeat a specific command from your history list, type r followed by the number. For example, to repeat command 114, you would type

$ r 114 # fc -s 114

ls -1

A number preceded by a minus sign tells the shell to go back that many commands in the list. If the last command you entered was number 119, the following command would take you back to command 116:

% r -3 # fc -s -3

cd Save

You can also redo commands by specifying the command name. In this example,

$ vi cs106xProject.c

$ ls

cs106xProject.c ProjectBackup

$ r vi # fc -s vi

vi cs106xProject.c

r vi repeats the last command beginning with vi.

Command-Line Editing

Command-line editing is a very popular shell feature. It was introduced in tcsh (csh does not support command-line editing) and carried over to the Korn shell and bash. Command-line editing lets you use a special version of either the vi or emacs text editor to edit your current command line, or any of the commands in your history list. On most systems, command-line editing is enabled by default, although you may choose to switch editors. Chapter 5 compares vi and emacs and describes how to use each of them.

The command-line editor shows you a one-line “window” on your command history, starting with your current command. You can use the up/down arrow keys to move backward and forward in your history Once you edit a line, you can execute it by pressing ENTER.

The command-line editing features greatly enhance the value of the history list. You can use them to correct command-line errors and to modify previous commands. Command-line editing also makes it much easier to search through your command history list, because you can use the same search commands you use in vi or emacs.

Suppose you want to search your command history for your most recent use of the file project.backup. If your command-line editor is set to emacs, you can search by typing CTRL-R followed by the filename. As soon as you enter part of the string, emacs will begin to search your history

$ [CTRL] R

(reverse-i-search)'pr': lpr directions

To search further back in your history list, type CTRL-R again.

To perform the same search with the vi editor, you would type ESC followed by a / (slash) and the beginning of the filename, as shown.

$ ESC /proj

When you hit ENTER, the editor will search for the most recent command in your history list that contains the string “proj”. To find an earlier command containing that string, type “n” to repeat the search.

Table 4–5 shows the most useful commands for line editing. Note that the vi command-line editor begins in input mode. To use the vi commands, you must enter command mode by pressing ESC. You can use the emacs commands at any time. For this reason, some users who normally prefer vi use emacs as their line editor.

Table 4–5: Command-Line Editing Commands

Movement Commands

vi

emacs

One character left

h

CTRL-B

One character right

l

CTRL-F

One word left

b

ESC-B

One word right

w

ESC-F

Beginning of line

A

CTRL-A

End of line

$

CTRL-E

Back up one entry in history list

k

CTRL-P

Search for string xxx in history list

/xxx

CTRL-R XXX

Editing Commands

vi

emacs

Delete current character

X

CTRL-D

Delete current word

dw

ESC-D

Delete line

dd

(kill char)

Change word

cw

Append text

a

Insert text

i

Setting the Line Editor in bash and ksh

To enable command-line editing in bash or ksh, use

$ set -o vi

to turn on vi-style editing, or

$ set -o emacs

to enable the emacs line editor. In ksh, if you do not set either of these options, the shell will try to use the editor specified by the variable VISUAL.

Since command-line editing is such a useful feature, you may want to add this setting to your .kshrc or .bashrc file.

Setting the Line Editor in tcsh

The bindkey command in tcsh determines whether it uses emacs or vi for command-line editing, as shown:

bindkey -e # use emacs-style editing

bindkey -v # use vi-style editing

You may want to add one of these settings to your .tcshrc file.

Command Substitution

Earlier you saw how the shell substitutes the value of a variable into a command line. Command substitution is a similar feature that allows you to substitute the output of a command into your command line. To do this, you enclose the command in backquotes. Note that the backquote character (`) is different from the single quote character (’). On many keyboards, the backquote key is in the upper left, near the 1 key

Suppose the file names contains the e-mail addresses of the members of a working group:

$ cat names

rlf@library.edu nate@engineer.com liz@thebest.net

You can use command substitution to send mail to all of them by typing

$ mail 'cat names'

When this command line is processed, the backquotes tell the shell to run cat with the file names as input, and substitute the output of this command (which in this case is a list of e-mail addresses) into the command line. The result is exactly the same as if you had entered the command

$ mail rlf@library.edu nate@engineer.com liz@thebest.net

In the Korn shell and bash,

$ mail $ (cat names)

works exactly the same way. It even makes sense-as with variables, the $ causes the shell to replace the command with its value. Because of this, and because the backquote character is so easily confused with single quotes, you may find this syntax preferable.

Filename Completion

It can be difficult and time-consuming to type in long filenames. As you have seen, wildcards (such as *) can be used as shortcuts for filenames, but they can also cause mistakes-for example, if there are several files in the current directory that start with the same letters. Filename completion is a feature first introduced in csh that gives you a better shortcut for entering filenames.

Suppose the current directory contains the following files:

% ls

california newjersey newyork washington

If you type the letters cal in a command line and then press the TAB key, the shell will fill in the filename california for you. (In csh and some versions of ksh, you press the ESC key twice instead of using TAB. The public domain version of the Korn shell, pdksh, does support tab completion.) So the line

$ cat cal [TAB]

becomes

$ cat california

If more than one file in the directory starts with those letters, the shell will fill in as much as it can. So

$ rm n [TAB]

becomes

$ rm new

You can then add more letters, and press TAB again to complete the rest. In bash, you can type TAB twice in a row to see a list of all the files beginning with the same letters:

$ rm new [TAB] [TAB] # bash only

newjersey newyork

The newer shells, tcsh, ksh, and bash, have filename completion (also known as tab competition) turned on by default. In csh, you have to enable filename completion by setting the toggle variable filec. In ksh, the command-line editor you have selected may affect filename completion-see the FAQ on the Korn shell web site (listed at the end of this chapter) for more information.

Removing Special Meanings in Command Lines

As you have seen throughout this chapter, the shell command language uses a number of special symbols. These include the I/O redirection operators >, <, and |, the wildcard characters * and ?, and the $ symbol for variables substitution. When you type in a command line containing one of these special shell characters, it is interpreted by the shell as an instruction. Sometimes, however, you need to use one of these symbols as a normal character. A simple example is using grep to search for lines containing the pipe symbol. The logical command would be

$ grep | .kshrc

Usage: grep [OPTION]… PATTERN [FILE]…

ksh: .kshrc: command not found

but this doesn’t work. As the error messages indicate, the shell interprets | as an instruction to send the output of the grep command to a (nonexistent) command called ".kshrc”.

One way to get | into the command line as an ordinary character, rather than a special instruction to the shell, is to quote it. Enclosing any symbol or string in single quotes prevents the shell from treating it as a special character. For example,

$ grep '|' .kshrc

alias lc='ls -la | more'

There are two other ways to quote command-line input to protect it from shell interpretation-double quotes (“…”) and the backslash character (\):

$ grep " | " .kshrc

and

$ grep \ | .kshrc

Double quotes act like single quotes, except that they allow the shell to process the characters used for variable substitution and command substitution. The \ character quotes the character immediately following it. In the preceding example, this causes the shell to treat the | character as a normal character rather than as a command pipe. Compare the following three examples:

$ grep '$HOME' .profile

PATH=$PATH:$HOME/bin

$ grep \$HOME .profile

PATH=$PATH:$HOME/bin

$ grep "$HOME" .profile

$

In the first two examples, the single quotes and the backslash remove the special meaning of the $ character. This causes grep to search for the literal string $HOME. In the third example, the double quotes around $HOME allow the shell to substitute the value of the variable. So grep searches for a string like /home/raf, and does not find it in the .profile.

Quoting can also be used to prevent the shell from interpreting white space (blanks, tabs, and newlines) as command-line argument separators. For example, if you want to delete a file named Kili Photo.jpg, with a space in the middle of the filename, you need to group the two words as a single argument, either with quotes

$ rm "Kili Photo.jpg"

or with a backslash

$ rm Kili\ Photo.jpg

If you did not quote the filename, rm would attempt to delete two files, Kili and Photo.jpg.

Summary

UNIX and Linux systems give you a choice of shell, including sh, csh, tcsh, ksh, and bash. They all provide the essential features of a command interpreter and high-level programming language, but there are some important differences among them. Different systems provide one or another of these as the default shell, but most systems support the others and allow you to choose the shell you prefer. This chapter gave you some information that can help you to decide which shell to use as your own.

You now know how to use all the important features and functions that the shell provides. You can use shell filename matching, control standard input and output, construct command pipelines, run commands in the background, assign shell variables, use simple command aliases, and configure your chosen shell. By now, you have probably gotten a good sense of the combination of flexibility and features that makes UNIX such a powerful operating system. The shells described in this chapter are available from the sources listed in Table 4–6.

Table 4–6: The Common Shells

Command

Name

Source or Links

bash

Bourne Again Shell

Standard on Linux systems Also available from the GNU site, at http://www.gnu.org/software/bash/

csh

C shell

Included with most distributions

ksh

Korn Shell

http://www.kornshell.com/

pdksh

Public Domain ksh

http://www.math.mun.ca/~michael/pdksh/

sh

Bourne Shell

Included with UNIX System distributions

tcsh

Extended C Shell

http://www.tcsh.org/

zsh

Z shell

http://www.zsh.org/

How to Find Out More

The shell is a very complex and powerful tool, and has many features that couldn’t be covered in a single chapter. This article compares the various shells and includes a very detailed chart of the differences between the most popular shells. It also has some good advice on choosing which shell to use.

· NSCP Unix Shells: http://www.int.gu.edu.au/courses/2010int/nscp_shells.html

This is the UNIX FAQ. It goes into great detail in answering some common but tricky questions, such as how the configuration files for the various shells work, or how to get the C shell to display the working directory in the prompt.

· UNIX FAQ/faq Index: http://www.faqs.org/faqs/unix-faq/faq/

· UNIX FAQ/shell Index: http://www.faqs.org/faqs/unix-faq/shell/

Most introductory books on the UNIX operating system have a chapter on the shell. This book is a particularly good beginner’s introduction to Linux. It is not exclusively devoted to the shell, but if you’re looking for a great general reference, this is worth picking up.

· Sobell, Mark G. A Practical Guide to Linux Commands, Editors, and Shell Programming. 1st ed. Upper Saddle River, NJ: Prentice Hall, 2005.

· These two books are excellent guides to bash and ksh.

· Newham, Cameron and Bill Rosenblatt. Learning the bash Shell. 3rd ed. Sebastopol, CA: O’Reilly Media, 2005.

· Bolsky, Morris I. and David G.Korn. The New Korn Shell, Command and Programming Language. 2nd ed. Englewood Cliffs, NJ. Prentice Hall, 1995.

· One of the relatively few books devoted to the C shells. This reference does not cover shell scripting, only the interactive features of the shells.

· DuBois, Paul. Using csh & tcsh. 1st ed. Sebastopol, CA: O’Reilly Media, 1995.

· These two sources both go beyond the material in this chapter, with an emphasis on shell scripting.

· Johnson, Chris F.A. Shell Scripting Recipes. 1st ed. Berkeley, CA: Apress, 2005.

· Robbins, Arnold and Nelson H.F.Beebe. Classic Shell Scripting. 1st ed. Sebastopol, CA: O’Reilly Media, 2005.