UNIX: The Complete Reference (2007)

The UNIX file system provides a powerful and flexible way to organize and manage your information. This chapter introduces the basic concepts of the file system and explains the most important commands for manipulating files and directories. In particular, the commands that provide the basic file manipulation operations-viewing files, changing directories, deleting and moving files-are among the UNIX commands you will use most often.

By the end of this chapter, you will know how to display the contents of files and directories, and how to create, delete, and manage them. You will also be able to search for specific files, control user access to files by using permissions, and use the UNIX commands for printing files.

Files

A file is the basic structure that stores information on the UNIX System (and on Windows systems, as well). Conceptually, a computer file is similar to a paper document. Technically a file is a sequence of bytesthat is stored somewhere on a storage device, such as a hard drive. A file can contain any kind of information that can be represented as a sequence of bytes. Word processing documents, bitmap images, and computer programs are all examples of files.

Filenames

Every file has a title, called a filename. A filename can be almost any sequence of characters, and up to 255 characters long. (On some older versions of UNIX, two filenames are considered the same if the first 14 characters are identical, so be careful if you use long filenames on these systems.) You can use any ASCII character in a filename except for the null character (ASCII NUL) or the slash (/), which has a special meaning in the UNIX file system. The slash acts as a separator between directories and files.

Even though UNIX allows you to choose filenames with special characters, it is a good idea to stick with alphanumeric characters (letters and numbers) when naming files. You may encounter problems when you use or display the names of files containing nonalphanumeric characters. In particular, although the following characters can be used in filenames, it is better to avoid them. Many of these characters have special meanings in the command shell, which makes them difficult to work with in filenames.

Capitalization

Windows does not distinguish between uppercase and lowercase letters in filenames. You could save a file with the name Notes.DOC and find it by searching for notes.doc. The UNIX file system, however, is case-sensitive, meaning that uppercase and lowercase letters are distinct. In UNIX, NOTES, Notes, and notes would be three different files. If you save a file with the name Music, you will not find it by searching for music. This also applies to commands in UNIX. If you are trying to log out with the exit command, typing EXIT will not work. By the way, this explains why URLs (web addresses) can be case-sensitive, since the first web server was created on a UNIX-based platform, and many web servers still run UNIX.

Filename Extensions

In Windows, filenames typically consist of a basename, followed by a period and a short filename extension. Many Windows programs depend on the extension to determine how to use the file. For example, a file named solitaire.exe is considered to be a file named solitaire with the extension .exe, where the .exe extension tells Windows that it is an executable program. If the file extension is altered or deleted, it will be more difficult to work with the file in Windows.

In UNIX, file extensions are conveniences, rather than a necessary part of the filename. They can help you remember the content of files, or help you organize your files, but they are usually optional. In fact, many UNIX filenames do not have an extension. For example, an executable program would typically have a name like solitaire rather than solitaire.exe. In addition, filename extensions in UNIX can be longer than three characters. For example, some people use .backup to indicate a backup copy of a file, so notes.backup would be an extra copy of the file notes.

Some programs either produce or expect a file with a particular filename extension. For example, files that contain C source code have the extension .c, so sorting.c would be a C language file. Similarly, web browsers expect that HTML files will have the extension .html, such as index.html. Table 3–1 displays some of the most commonly used filename extensions.

UNIX files can have more than one extension. For example, the file book.tar.Z is a file that has first been archived using the tar command (which adds the extension .tar) and then compressed using the compress command (which adds the .Z). This enables a single script to both decompress the file and untar it, using the filename as input and parsing each of the extensions to perform the appropriate task.

The flexibility of filename conventions in UNIX allow for some variation in filenames. A program written in Perl could have the filename program.perl, the more frequently used program.pl, or even just the name program. You can even create your open file extensions. A text file containing research notes might be called res_nov, ResearchNovember, or research.notes.nov. In the last case, the extension .nov is just to remind you that the notes are from November. It will not change the way you work with the file.

Directories

Files contain the information you work with. Directories provide a way to organize your files. A directory is just a container for as many files as you care to put in it. If you think of a file as analogous to a document in your office, then a directory is like a file folder. In fact, directories in UNIX are exactly like folders in Windows.

For example, you may decide to create a directory to hold all of your notes. You could name it Notes and use it to hold only files that are your notes, keeping them separated from your e-mail, programs, and other files.

Subdirectories

A directory can also contain other directories. A directory inside another directory is called a subdirectory. You can create as many subdirectories inside a particular directory as you wish.

Choosing Directory Names

It is a good idea to adopt a convention for naming directories so that they can be easily distinguished from ordinary files. Some people give directories names that are all uppercase letters, some use directory names that begin with an uppercase letter, and others distinguish directories using the extension .d or .dir. For example, if you decide to use names beginning with an uppercase letter for directories and avoid naming ordinary files this way, you will know that Notes, Misc, Multimedia, and Programs are all directories, whereas note3, misc.note, mm_5, and progmmA are all ordinary files.

The Hierarchical File Structure

Because directories can contain other directories, which can in turn contain other directories, the UNIX file system is called a hierarchical file system. Within the UNIX System, there is no limit to the number of files and directories you can create in a directory that you own. File systems of this type are often called tree-structured file systems, because each directory allows you to branch off into other directories and files. Tree-structured file systems are usually depicted upside-down, with the root of the tree at the top of the drawing. Figure 3–1 depicts a typical hierarchical tree structure.

Figure 3–1: A sample directory structure

On every UNIX system, the root is a directory called /. In Figure 3–1, root contains a subdirectory called home. Inside home you have three subdirectories, each for a different user on the system. One of these subdirectories is for the user whose login name is raf; in that directory are two subdirectories (Email, Work) and a file (notes.august); and in those directories are other subdirectories or files (inbox, save, sent,final, save).

The directory in which you are placed when you log in is called your home directory. Generally, every user on a UNIX system has a unique home directory, often with the same name as their login name. In every login session, you start in your home directory and move up and down the directory tree. (Sometimes users have several home directories, each used for specific purposes, but beginning users do not need to worry about this.) There is a maximum number of directories a user can have; this number is set by the system administrator to prevent a user from ruining a file system.

Pathnames

Notice in Figure 3–1 that there are two files with the same name, but in different locations in the file system. There is a save file in the Email directory, and another file called save in the Work directory. In order to distinguish files with the same name, the UNIX System allows you to specify filenames by including the location of the file in the directory tree. This type of name is called a pathname, because it is a listing of the directories you travel through along the path you take to get to the file. The path through the file system starts at root (/), and the names of directories and files in a pathname are separated by slashes.

For example, the pathname for one of the save files is

/home/raf/Email/save

and the pathname for the other is

/home/raf/Work/save

Pathnames that trace the path from root to a file are called full (or absolute) pathnames. Specifying a full pathname provides a complete and unambiguous name for a file. In a full pathname, the first slash (/) refers to the root of the file system. All the other slashes separate the names of directories, until the last slash separates the filename from the name of the directory it’s in.

Using full pathnames can be awkward when there are many levels of directories, as in this filename:

/home/dkraut/Work/cs106x/Proj_1/lib/Source/strings.c

In cases like this, using the full pathname requires a good memory and a lot of typing. In Chapter 4 you will learn how to use shell variables as a shortcut to specify pathnames.

Relative Pathnames

You do not always have to specify the full pathnames when you refer to files. As a convenient shorthand, you can also specify a path to a file relative to your present directory Such a pathname is called a relative pathname. Instead of starting with a / for root, the relative pathname starts with the name of a subdirectory For example, suppose you are in your home directory, /home/raf. The relative path for the save file in the Email subdirectory is Email/save, and the relative path for the other save file is Work/save.

Specifying the Current Directory

A single dot (.) is used as a shortcut to refer to the directory you are currently in. This directory is known as the current directory.

Specifying the Parent Directory

Two dots (.., pronounced “dot-dot”) refer to the parent directory of the one you are currently in. The parent directory is the one at the next higher level in the directory tree. Because the file system is hierarchical, all directories have a parent directory The dot-dot references can be used many times to refer to things far up in the file system. The following sequence, for example,

../..

refers to the parent of the parent of the current directory. If you are in Work, then ../.. is the same thing as the home directory, since home is the parent of raf, which is the parent of Work.

Specifying a Home Directory

A tilde (~) can be used to refer to your home directory (Strictly speaking, this is a feature of the shell, which will be discussed in the next chapter. It will work on most modern UNIX systems.)

These shortcuts can be combined. For example, if your home directory is /home/raf, then

~/../liz

refers to the home directory for the user liz.

You can also use a tilde followed by a login name to refer to another user’s home directory For example, the shortcut ~nate refers to the user nate’s home directory

UNIX and Windows File Structure

The Windows file system was patterned after the UNIX hierarchical file system structure, but with some important differences. On a UNIX System, the root of the file system is depicted as / (slash). The root is the base of the entire system file structure, including files that may be on a different physical disk. In Windows, each drive or partition has a different root. On the main hard drive, the root is usually called C:\. A CD-ROM drive would have a different root than the hard drive (it might be D:\, for example). In addition, Windows uses a \ (backslash) instead of the / (forward slash) that separates directories in UNIX.

For example, this UNIX pathname

/home/raf

would look like this in Windows:

C:\home\raf

Note that the forward slashes in the UNIX pathname have become backslashes in the Windows pathname.

You may have noticed that a UNIX pathname looks a lot like part of a web address. That’s because the web inherited this style of pathname from the UNIX file system. The shortcuts for pathnames just described, such as .. for the parent directory, can be used in HTML code for web pages.

UNIX System File Types

The file is the basic unit of the UNIX System. Within UNIX, there are four different types of files: ordinary files, directories, symbolic links, and special files.

Ordinary Files

As a user, most of the information that you work with will be stored as an ordinary file. An ordinary file can contain data, such as text for documents or programs. Image files and binary executables are also examples of ordinary files.

Links

Sometimes it is useful to have a file that is accessible from several directories, without making separate copies of the file. For example, suppose you are working with someone else, and you need to share information contained in a single data file that each of you can update. It would be convenient for each of you to have a copy in your home directory However, you do not want to make separate copies of the file, because it will be hard to keep them in sync.

A link is not a kind of file but instead is a second name for a file. With a link, only one file exists on the disk, but it may appear in two places in the directory structure. This can allow two users to share the same file. Any changes that are made to the file will be seen by both users. This type of link is sometimes called a hard link, to distinguish it from a symbolic link.

Symbolic Links

Hard links can be used to assign more than one name to a file, but they have some important limitations. They cannot be used to give a directory more than one name, and they cannot be used to link files on different computers.

These limitations can be eliminated by using symbolic links (sometimes called symlinks). A symbolic link is a file that only contains the name (including the full pathname) of another file. When the operating system operates on a symbolic link, it is directed to the file that the symbolic link points to. Essentially, the symbolic link is a pointer to the other file. If you are familiar with the Windows operating system, you may be reminded of shortcuts, which are very similar to symbolic links.

Symbolic links can be used to assign more than one name to a file or directory, or to make it possible to access a file from several locations. (For example, you could use a symbolic link to give a short name, like ff, to a file with a long pathname, like /usr/bin/firefox/firefox.) Symbolic links can also be used for files or directories that reside on a different physical file system, such as files on different computers that are connected by a network. (File systems are discussed in detail in Chapter 14.)

Using Links: An Example

Suppose that Rebecca and Nathan are working on a project together, and they need to share the file project.index, which is in Nathan’s home directory Rebecca could make a copy of the file for herself, but then if she makes any changes to the file, Nathan won’t see them in his copy So instead, Rebecca makes a link (a hard link) to that file. Now she has a file called project.index in her home directory, too, even though there is only one copy of the information in that file saved on the disk. This means that if Rebecca makes any changes in the file, Nathan will see those changes, too. However, the file can be found in two different places in the file system-in Nathan’s home directory, and in Rebecca’s. If Nathan deletes the file from his home directory, Rebecca will still have the file in her directory If she deletes her file, too, then it will really be gone.

Now, suppose there is another file in Nathan’s directory they need to share, called project.data. This time, Rebecca makes a symbolic link to the file, and calls it project.data.symlink. Rebecca can still make changes to the file, and Nathan will be able to see them. However, if Nathan deletes the project.data, Rebecca won’t have the file anymore, either. If she tries to use project.data.symlink after the original file is deleted, she will get an error message.

Directories

A directory is actually a type of file too, a file that holds other files. For each directory, the UNIX file system stores a list of all the files and subdirectories that it contains, as well as their file types (whether they are ordinary files, symbolic links, directories, or special files) and other attributes.

Special Files

Special files are an unusual feature of the UNIX file system. A special file represents a physical device, such as a printer or a CD-ROM drive. From the user’s perspective, the file system treats special files just as it does ordinary files. This means that the commands that work on ordinary files also work on special files, so you can read or write to devices exactly the way you read and write to ordinary files. For example, you can use a command to take the characters typed at your keyboard and write them to a text file, or you could use the same command to send them to a printer. The UNIX System causes these read and write commands to activate the hardware connected to the device.

Common Commands for Files and Directories

This section discusses the basic UNIX file system commands. If you are working in a graphical environment, you will have to open a terminal window to enter these commands.

Listing the Contents of a Directory

Assume that you are in the directory called /home/raf from the example in Figure 3–1.

To see all the files in this directory, you enter the ls (list) command:

$ ls

Email notes.august Work

The ls command lists the contents of the current directory on your screen in multiple columns. (The precise behavior of the ls command varies in different releases of UNIX. For example, on most systems, it will list filenames in alphabetical order, but on some it will list all filenames that start with an uppercase character before listing filenames in lowercase. This is sometimes called ASCII order.) Notice that ls without arguments simply lists the contents by name. It does not tell you whether the names refer to files or directories.

If you want to view the contents of a subdirectory of your current directory, you can issue the ls command with an argument that is the name of the subdirectory For example,

$ ls Email

inbox save sent

If the object (file or directory) does not exist, ls gives you an error message, such as

$ ls Emial

Emial not found

You can see whether a file exists by supplying the pathname of the file, in relation to your current directory, as the argument to ls. If the file does exist, it will echo the name back to you.

$ ls Email/save

Email/save

Listing Directory Contents with Marks

When you use the ls command, you do not know whether a name refers to an ordinary file, a program that you can run, or a directory Running the ls command with the -F option produces a list in which the names are marked with symbols that indicate the kind of file that each name refers to.

Names of directories are listed with / (a slash) following their names. Executable files (those that can be run as programs) are listed with * (an asterisk) following their names. Symbolic links are listed with @ (an “at” sign) following their names. For instance, suppose that you run ls with the -F option to list the contents of a directory, producing the following result:

$ ls -F

Email/ notes Projects@

This example shows that the directory contains the ordinary file notes, the directory Email, and a symbolic link Projects. Another way to get information about file types and contents is with the file command, described later in this chapter.

Listing Files in the Current Directory Tree

You can add the -R (recursive) option to the ls command to list all the files in your current directory, along with all the files in each of its subdirectories, and so on. For example,

$ ls -R

Email notes.august Work

./Email

inbox save sent

./Work

final save

shows the contents of the current directory as well as the contents of its subdirectories Email and Work.

Viewing Files

The simplest and most basic way to view a file is with the cat command. cat (short for concatenate) takes any files you specify and displays them on the screen. For example, you could use cat to display on your screen the contents of the file review:

$ cat review

I recommend publication of this article. It provides

a good overview of the topic and is

appropriate for the lead article of this issue.

The cat command shows you everything in the file but nothing else: no header, title, file-name, size, or other information.

Viewing Files with Special Characters

The cat command recognizes eight-bit characters. In earlier versions of UNIX, it only recognized seven-bit characters. This enhancement permits cat to display characters from extended character sets, such as the kanji characters used to represent Japanese words.

If you try to display a binary file, such as an executable program, the output to your screen will usually be a mess. To better view files that contain nonprinting ASCII characters, you can use the -v option. For example, if the file output contains a line that includes the ASCII BEL character (CTRL-G), the command cat output will cause the computer to beep. Using the -v option, however, will replace the BEL character with the symbol ^G, as shown.

$ cat -v output

The ASCII control character ^G (007) will ring a

bell ^G^G^G^G on the user's terminal.

$

Directing the Output of cat

You can send the output of cat to a file as well as to the screen. For instance,

$ cat physics > physics.backup

copies the contents of physics to physics.backup, instead of displaying the contents on the screen. The > provides a general way to redirect the output of a command to a file. This is explained in detail in the section “Standard Input and Output” of Chapter 4.

In the preceding example, if there is no file named physics.backup in the current directory, the system creates one. If a file with that name already exists, the output of cat overwrites it-its original contents are replaced. (Note that this can be prevented by using the noclobber features of some shells.) Sometimes this is what you want, but sometimes you want to add information from one file to the end of another. In order to add information to the end of a file, do the following:

$ cat notes.august >> notes

The >> in the preceding example appends the contents of the file named notes.august to the end of the file named notes, without making any other changes to notes. It’s okay if notes does not exist; the system will create it if necessary The capability to append output to an existing file is another form of file redirection. Like simple redirection, it works with almost all commands, not just cat.

Combining Files and Using Wildcards

You can use cat to combine a number of files into one. For example, consider a directory that contains material being used in writing a chapter, as follows:

$ ls

Chapter1 macros section2

chapter.1 names section3

chapter.2 section1 sed_info

You can combine all of the sections into a chapter with cat:

$ cat section1 section2 section3 > chapter.3

This copies each of the files, section1, section2, and section3, in order into the new file chapter.3. This can be described as concatenating the files into one, hence the name cat.

To make commands like this easier, the shell provides a wildcard symbol, * (asterisk), that allows you to specify a number of files with similar names. An asterisk by itself matches all filenames in the current directory When the * is part of a word, it stands for or matches any string of characters. For example, the pattern section* matches any file whose name begins with section. So the command

$ cat section* > chapter.3

would have had the same effect as the command in the preceding example.

When you use the wildcard symbol * as part of a filename in a command, that pattern is replaced by the names of all files in the current directory that match the pattern, listed in alphabetical order. In the preceding example, section* matches section1, section2, and section3, and so would sect*. But se* would also match the file sed_info. A* (star, or asterisk) by itself matches all filenames in the current directory

When using wildcards, you can use ls to make sure that the wildcard pattern matches the files you want. For example,

$ ls se*

section1 section2 section3 sed_info

indicates that it would be a mistake to use se* unless you want to include sed_info.

You can also use * to simplify typing commands, even when you are not using it to match more than one file. The command

$ cat *1 *2 > temp

is a lot easier to type than

$ cat section1 section2 > temp

Chapter 4 describes the use of * and other wildcard characters in detail.

Note that there is an important difference between the UNIX System’s use of * and the similar use of it in Windows. In Windows, * does not match a . (dot), so section* would match section1 but not match section.txt. In Windows, you would have to use the pattern section*.* to match every file beginning with section.

Creating a File

So far, all the examples you have seen involved using cat to copy one or more normal files, either to another file or to your screen. But other possibilities exist. Just as your screen is the default output for cat and other commands, your keyboard is the default input. If you do not specify a file to use as input, cat will simply copy everything you type to its output. This provides a way to create simple files without using an editor. The command

$ cat > names

Nate nate@engineer.com

Rebecca rlf@library.edu

CTRL-D

sends everything you type to the file names. It sends the text one line at a time, after you hit ENTER. You can use BACKSPACE to correct your typing on the current line, but you cannot back up across lines. When you are finished typing, you must type CTRL-D (hold down the Ctrl key and press d) on a line by itself. This terminates cat and closes the file names. (CTRL-D is the end of file [EOF] mark in the UNIX System.)

Using cat in this way (cat > names) creates the file names if it does not already exist and overwrites (replaces) its contents if it does exist. You can use cat to add material to a file as well. For example,

$ cat >> names

Dan dkraut@bio.ca.edu

CTRL-D

will take everything you type at the keyboard and append it at the end of the file names. Again, you need to end by typing CTRL-D alone on a line.

Another command, touch, can also be used to create a file.

$ touch notes

will create an empty file called notes, if that file does not already exist. Unlike cat, touch can also be used to change the creation time or last accessed time of an existing file. This is discussed in further detail in Chapter 19.

Moving Around in Directories

Since many UNIX commands operate on the current directory it is useful to know what your current directory is. The command pwd (present working directory) tells you which directory you are currently in. For example,

$ pwd

/home/raf

tells you that the current directory is /home/raf.

You can move between directories by using the cd (change directory) command. If you are in your home directory, /home/raf, and wish to change to the subdirectory Work, type

$ cd Work

If you know where certain information is kept in a UNIX system, you can move directly there by specifying the full pathname of that directory:

$ cd /home/raf/Email

$ pwd

/home/raf/Email

$ ls

inbox save sent

You can also change to a directory by using its relative pathname. Since .. (dot-dot) refers to the parent directory (the one above the current directory in the tree),

$ cd . .

$ pwd

/home/raf

moves you to that directory To go a step further,

$ cd ../..

$ pwd

/

changes directories to the parent of the parent of the current directory, or in our example, to the / (root) directory

Moving to Your Home Directory

If you issue cd by itself, you will be moved to your home directory, the directory in which you are placed when you log in. This is an especially effective use of shorthand if you are somewhere deep in the file system. For instance, you can use the following sequence of commands to list the contents of your home directory when you are in the directory /home/dkmut/iuork/cs106x/proj1/lib/Source:

$ pwd

/home/dkraut/work/cs106x/proj1/lib/Source

$ cd

$ pwd

/home/raf

$ ls

Email notes.august Work

In the preceding example, the first pwd command shows that you are nested seven layers below the root directory The cd command moves you to your home directory as confirmed by the pwd command, which shows that the current working directory is /home/raf. The ls command shows the contents of that directory Since ~ is a shortcut that refers to your home directory,

$ cd ~

does exactly the same thing.

Returning to Your Previous Directory

The command cd-will return you to your previous directory For example, after moving to your home directory as shown above, you can return to the directory Source with:

$ cd -

$ pwd

/home/dkraut/work/cs106x/proj1/lib/Source

Like the ~ shortcut for your home directory, using cd-to return to the previous directory is a feature of the shell. Some shells also have the commands pushd and popd to allow you to save, and later return to, your current directory

Moving and Renaming Files and Directories

To keep your file system organized, you will need to move and rename files. You move a file from one directory to another with mv (from move). For example, the following moves the file names from the current directory to the directory /home/jmf/Info:

$ mv names /home/jmf/Info

If you use ls to check, it confirms that a file with that name is now in Info:

$ ls /home/jmf/Info/names

/home/jmf/Info/names

You can move several files at once to the same destination directory by first naming all of the files to be moved, and giving the name of the destination last. For example, the following command moves three files to the subdirectory called TermPaper:

$ mv section1 section2 section3 TermPaper

Of course you could make this easier by using the wildcard symbol, *. If the three files in the preceding example are the only files in the current directory with names beginning with sec, the following command has the same effect:

$ mv sec* TermPaper

UNIX has no separate command for renaming a file. Renaming is just part of moving. To rename a file in the current directory, you use mv, but with the new filename as the destination. For example, the following renames overview to intro:

$ mv overview intro

You can rename a file when you move it to a new directory by including the new filename as part of the destination. The following command puts notes in the directory Music and gives it the new name notes4:

$ mv notes Music/notes4

Compare with this, which moves notes to Research but keeps the old name, notes:

$ mv notes Music

To summarize, when you use mv, you first name the file or files to be moved, and then the destination. The destination can be a directory name, in which case the file is simply moved, or it can be a filename, in which case the file is renamed. The destination can be a full pathname, or a name relative to the current directory-for example, one of its subdirectories.

Moving files is very fast in UNIX. The actual contents of a file are not moved; you’re really only moving an entry in a table that tells the system what directory the data is in. So the size of the file being moved has no bearing on the time taken by the mv command.

Avoiding Mistakes with mv

When using mv, you should watch out for a few common mistakes. If you make a mistake in typing when you specify a destination directory, you may end up renaming the file in the current directory For example, suppose you meant to move a file to Info but made a mistake in typing.

$ mv names Ifno

In this case, you end up with a new file named Ifno in the current directory A similar mistake can happen if you try to move a file to a directory that does not exist. Again, the file will be renamed instead.

When you move a file to a new directory, it is a good idea to check first to make sure the directory does not already contain a file with that name. If it does, mv will simply overwrite it with the new file. The same thing will happen if you try to rename a file using a filename that already exists.

Newer versions of UNIX provide an option to the mv command that helps prevent accidentally overwriting files. The -i (interactive) option causes mv to inform you when a move would overwrite an existing file. It displays the filename followed by a question mark. If you want to continue the move, type y. Any other entry (including n) stops that move. The following shows what happens if you try to use mv -i to rename the file totals to data when the data file already exists:

$ mv -i totals data

mv: overwrite data?

In Chapter 4, you will see how to use an alias to change the mv command so that it always uses the -i option, if you choose. This can be very helpful for new users.

Moving Directories

Another feature not present in earlier versions of UNIX is the capability to use mv to move directories. You can use a single mv command to move a directory and all of its files and subdirectories just as you’d use it to move a single file. For example, if the directory Final contains all of your finished work on a document, you can move it to a directory in which you keep all of the versions of that document, Project, as shown here:

$ ls Project

Drafts

$ mv Final Project

$ ls Project

Drafts Final

Copying Files

The cp command is similar to mv, except that it copies files rather than moving or renaming them. cp follows the same model as mv: you name the files to be copied first and then give the destination. The destination can be a directory, a pathname for a file, or a new file in the current directory. The following command makes a backup copy of seattle and names the copy seattle.bk:

$ cp seattle seattle.bk

After you use this cp command, there are two separate copies of that file in the same directory The original is unchanged, and the contents of the copied file are identical to the original. The files are not linked in any way, so if you edit one of the files, the other will not change.

To create a copy of a file with the same name as the original but in a new directory, just use the directory name as the destination, as shown here:

$ cp seattle Backups

Note that if the destination directory already contains a file named seattle, the copy will overwrite it.

If you invoke cp with the -i (interactive) option, it will warn you before overwriting an existing file. For example, if there is already a file named data.2 in the current directory, cp warns you that it will be overwritten and asks if you want to go ahead:

$ cp -i data data.2

cp: overwrite data.2?

To go ahead and overwrite it, type y. Any other response, including n or ENTER, leaves the file as it was. Chapter 4 shows how to use an alias to replace cp with cp -i, if you choose.

Copying the Contents of a Directory

So far the discussion has assumed that you are copying an ordinary file to another file. If you try to copy a directory, you will get an error message. A feature of cp (found on most versions of UNIX) is the -r(recursive) option that lets you copy an entire directory structure. Suppose you have a directory called Project, and you wish to make a backup copy The following command creates a new directory, called Project.Backup, and copies all of the files and subdirectories in Project to the new directory:

$ cp -r Project Project.Backup

Linking Files

When you copy a file, you create another file with the same contents as the original. Each copy takes up additional space in the file system, and each can be modified independently of the others.

As you recall, a link is a way to share a file with another user without actually copying it on the disk. An example where this might be useful is a list of names and contact information that two or more people use, and that any of the users can add to or edit. Each user needs access to a common version of the file in a convenient place in each user’s own directory system.

Hard Links

The ln command creates a link between files, which enables you to make a single file accessible at two or more locations in the directory system. The following links the file project.main in dkraut’s home directory with a new file of the same name in the current directory:

$ ln /home/dkraut/project.main project.main

Using ln in this way creates a hard link to the file in /home/dkraut, but there is still only one file. Now if you add a new line of information to your linked copy of project.main, the line also appears in the file in dkraut’s directory, since this is really the same file.

Any changes to the contents of a linked file affect all the links. If you overwrite (or clobber) the information in your file, the information in dkraut’s copy is overwritten too. (For a description of a way to prevent clobbering of files like this, see the noclobber options to the C and Korn shells, which are described in Chapter 4.)

You can remove one of a set of hard-linked files with the rm command without affecting the others. For example, if you remove your hard-linked copy of project.main, dkraut’s copy is unchanged.

To see if two files are hard linked to each other, use the command ls -i. This will display the inode number of each file. If two files have the same inode number, then they are really the same file. For example,

$ ls -i /usr/bin/gcc

344135 /usr/bin/gcc

$ ls -i /usr/bin/cc

344135 /usr/bin/cc

shows that /usr/bin/gcc and /usr/bin/cc are hard linked. (Note: although you cannot hard link files across file systems, sometimes two files on different file systems will display the same inode number. This does not mean that they are linked. See Chapter 14 for a discussion of file systems.)

Symbolic Links

Symbolic links are created by using the ln command with the -s (symbolic) option. The following example shows how you could use ln to link a file in the /var file system to an entry in one of your directories within the /home file system:

$ ln -s /var/X/docs/readme temp/x.readme

This will create a symbolic link called x.readme in the temp directory

The second argument to ln is optional; if you do not specify the name of the new file, it will create a symbolic link with the same name as the target file. So, for example

$ ln -s /usr/bin/firefox/firefox

will create a file called firefox in the current directory that is a symbolic link to /usr/bin/firefox/firefox.

Symbolic links also enable you to link directories. The command

$ ln -s /home/dkraut/work/cs106x/proj1/lib/Source Project

will create a directory, called Project, that is a link to the directory /homedkraut/work/cs106x/ proj1/lib/Source. This is useful for directories with long pathnames that you need to access often.

Removing Files

To get rid of files you no longer want or need, use the rm (remove) command. rm deletes the named files from the file system, as shown in the following example:

$ ls

notes research temp

$ rm temp

$ ls

notes research

The ls command shows that after you use rm to delete temp, the file is no longer there.

Removing Multiple Files

The rm command accepts several arguments and takes several options. If you specify more than one filename, it removes all of the files you named. The following command removes the two files left in the directory:

$ rm notes research

$ ls

$

Remember that you can remove several files with similar names by using wildcard characters to specify them with a single pattern. The following will remove all files in the current directory:

$ rm *

Similarly, if you use a common suffix to help group files dealing with a single topic, for example .rlf to identify notes to user rlf, you can delete all of them at once with the following command:

$ rm *.rlf

Safely Removing Files

Almost every user has accidentally deleted files. In the preceding example, if you accidentally hit the SPACEBAR between the * and the extension and type

$ rm * .rlf

you will delete all of the files in the current directory As typed, this command says to remove all files (*), and then remove a file named .rlf.

To avoid accidentally removing files, use rm with the -i (interactive) option. When you use this option, rm prints the name of each file and waits for your response before deleting it. To go ahead and delete the file, type y. Responding n or hitting ENTER will keep the file rather than deleting it. For example, in a directory that contains the files notes, research, and temp, the interactive option to rm gives you the following:

$ rm -i *

notes: y

research: <ENTER>

temp: y

Your responses cause rm to delete both notes and temp, but not research.

New users may find it very helpful to change the rm command to always use the -i option. This can be done with an alias. Chapter 4 describes how to add this alias to your configuration files.

Restoring Files

When you remove a file using the rm command, it is gone. If you make a mistake, you can only hope that the file is available somewhere on a backup file system (on a tape or disk).

You can call your system administrator and ask to have the file you removed, say /home/you/Work/temp, restored from backup. If it has been saved, it can be restored for you. Systems differ widely in how, and how often, they are backed up. On a heavily supported system, all files are copied to a backup system every day and saved for some number of days, weeks, or months. On some systems, backups are done less frequently, perhaps weekly On personal workstations, backups occur when you get around to doing them. In any case, you will have lost all changes made since the last backup. (Backing up and restoring are discussed in Chapter 14.) You cannot, as a user, restore a file by attempting to recover pieces of the file left stored on disk.

Creating a Directory

You can create new directories in your file system with the mkdir (make directory) command. It is used as follows:

$ pwd

Work

$ ls

notes research temp

$ mkdir New

$ ls

notes New research temp

In this example, you are in the Work directory, which contains the files notes, research, and temp, and you use mkdir to create a new directory (called New) within Work.

Removing a Directory

There are two ways to remove or delete a directory If the directory is empty (it contains no files or subdirectories), you can use the rmdir (remove directory) command. If you try to use rmdir on a directory that is not empty, you’ll get an error message. The following removes the directory New added in the preceding example:

$ rmdir New

To remove a directory that is not empty, together with all of the files and subdirectories it contains, use rm with the -r (recursive) option, as shown here:

$ rm -r Work

The -r option instructs rm to delete all of the files it finds in Work and then go to each of the subdirectories and delete all of their files, and so forth, concluding by deleting Work itself.

Since rm -r removes all of the contents of a directory, be very careful in using it. You can add the -i option to step through all the files and directories, removing or leaving them one at a time.

$ rm -ir Work

rm: descend into directory 'Work'? y

rm: remove regular empty file 'Work/final'? y

rm: remove regular empty file 'Work/save'? <RETURN>

$ ls Work

save

In this example, the file final is deleted, but because save is not, the directory Work is not deleted, either.

Notice, by the way, that when a command is being run with two or more options (in this case, -r and -i), the options can be combined (in this case, -ir). This is optional; the command rm -i -r could have been used instead in the preceding example. The order of the options does not matter (rm -ir is the same as rm -ri). This applies to most UNIX commands, not just rm.

Getting Information About File Types

Sometimes you just want to know what kind of information a file contains. For example, you may decide to put all your shell scripts together in one directory You know that several scripts are scattered about in several directories, but you don’t know their names, or you aren’t sure you remember all of them. Or you may want to print all of the text files in the current directory, whatever their content.

You can use several of the commands already discussed to get limited information about file contents. For example, ls -l shows you if a file is executable-either a compiled program or a shell script (batch file). But the most complete and most useful command for getting information about the type of information contained in files is file.

file reports the type of information contained in each of the files you give it. The following shows typical output from using file on all of the files in the current directory:

$ file *

Backup: directory

cx: commands text

draft3: ascii text

fields: ascii text

linkfile: symbolic link to dirlink

mmxtest: [nt] roff, tbl, or eqn input text

pq: executable

send: English text

tag: data

You can use file to check on the type of information contained in a file before you print it. The preceding example tells you that you should use the troff formatter before printing mmxtest, and that you should not try to print pq, since it is an executable program, not a text file.

To determine the contents of a file, file reads information from the file header and compares it to entries in the file /etc/magic. This can be used to identify a number of basic file types-for example, whether the file is a compiled program. For text files, it also examines the first 512 bytes to try to make finer distinctions-for example, among formatter source files, C program source files, and shell scripts. Once in a while this detailed classification of text files can be incorrect, although basic distinctions between text and data are reliable.

Searching for Files

The command locate searches for a pattern in a database of filenames. For example,

$ locate pippin

searches the database for filenames containing the string “pippin”. The database contains the full pathname for each file, so this would find files in the directory pippin-photos as well as files such as 0915-pippin.jpg.

The locate command is very fast and easy to use. However, it will only work if the database of filenames is kept up to date. On many systems, the database is automatically updated once per day

The find command is a more powerful search tool, although it can be difficult to use. With find, you can search through any part of the file system, looking for all files with a particular name or with certain features. This section describes how to use find to do simple searches.

Using find

The find command searches through the contents of one or more directories, including all of their subdirectories. You have to tell find in which directory to start its search. The following example searches user jmf’s directory system for the file new_data and prints the full pathname of any file with that name that it finds:

$ pwd

/home/jmf

$ find . -name new_data -print

/home/jmf/Dir/Logs/new_data

/home/j mf/Cmds/new_data

Here, find shows two files named new_data, one in the directory Dir/Logs and one in the directory Cmds. This example illustrates the basic form of the find command. The first argument is the name of the directory in which the search starts. In this case it is the current directory (represented by the dot). The -name option is followed by the name of the file or files to search for. The final option, -print, tells find to print the full pathnames of any matching files.

Note that you have to include the -print option. If you don’t, find will carry out its search but will not notify you of any files it finds.

To search the entire file system, start in the system’s root directory, represented by the /:

$ find / -name new_data -print

This will find a file named new_data anywhere in the file system. Note that it can take a long time to complete a search of the entire file system; also keep in mind that find will skip any files or directories that it does not have permission to read.

You can tell find to look in several directories by giving each directory as an argument. The following command first searches the current directory and its subdirectories and then looks in /tmp/project and its subdirectories:

$ find . /tmp/project -name new_data -print

You can use wildcard symbols with find to search for files even if you don’t know their exact names. For example, if you are not sure whether the file you are looking for was called new_data, new.data, or mydata, but you know that it ended in data, you can use the pattern *data as the name to search for:

$ find -name "*data" -print

Note that when you use a wildcard with the -name argument, you have to quote it. If you don’t, the filename matching process would replace *data with the names of all of the files in the current directory that end in “data.” The way filename matching works, and the reason you have to quote an asterisk when it is used in this way, are explained in the discussion of wildcards in Chapter 4.

Running find in the Background

If necessary you can search through the entire system by telling find to start in the root directory, /. Remember, though, that it can take find a long time to search through a large directory and its subdirectories, and searching the whole file system, starting at /, can take a very long time on large systems. If you do need to run a command like this, you can use the multitasking feature of UNIX to run it as a background job, which allows you to continue doing other work while find carries out its search.

To run a command in the background, you end it with an ampersand (&). The following command line runs find in the background to search the whole file system and send its output to found:

$ find / -name new_data -print > found &

The advantage of running a command in the background is that you can go on to run other commands without waiting for the background job to finish.

Note that in the example just given, the output of find was directed to a file rather than displayed on the screen. If you don’t do this, output may appear on your screen while you are doing something else; for example, while you are editing a document. This is rarely what you want. Unfortunately, find will still display error messages (such as the names of directories it cannot search) on your screen. Chapter 4 gives more information about running commands in the background, including how to prevent these error messages from appearing.

Other Search Criteria

The examples so far have shown how to use find to search for a file having a given name. You can use many other criteria to search for files. The -mtime option lets you specify the number of days it has been since the file was modified. For example, to search for a file that was modified fewer than three days ago, use -mtime -3. The -user option restricts the search to files belonging to a particular user.

You can combine these and other find options. For example, the following command line tells find to look for a file called music belonging to user sue that was modified more than a week ago:

$ find . -name "music" -u sue -mtime +7 -print

The find command can do more than print the name of a file that it finds. For example, you can tell it to execute a command on every file that matches the search pattern. For this and other advanced uses, consult the UNIX man (manual) page for find (see Chapter 2 for an explanation of the UNIX man pages).

More About Listing Files

Many options can be used with the ls command. They are used either to obtain additional information about files or to control the format used to display this information. This section introduces the most important options that were not covered earlier. You can find a description of all options to the ls command and what they do by consulting the man page for ls.

Listing Hidden Files

Files with names beginning with a dot (.) are hidden in the sense that they are not normally displayed when you list the files in a directory These are typically configuration files that are used regularly by the system, but you will only rarely read or edit them. Suppose you see something like this when you list the files in your home directory:

$ ls

Email notes Work

This example shows that your home directory contains files named Email, Work, and notes. But there may also be hidden files that do not show up in this listing. Examples of common hidden files are your .profile, which sets up your work environment, and the .mailrc file, which is used by the mailx electronic mail command. To avoid clutter, ls does not list hidden files unless you explicitly ask to see them.

To see all files in this directory, use ls -a:

$ ls -a

. .. .mailrc .profile Email notes Work

The example shows two hidden files. In addition, it shows the current directory and its parent directory as . (dot) and .. (dot-dot), respectively.

Controlling the Way Is Displays Filenames

By default, in many flavors of UNIX, ls displays files in multiple columns, sorted down the columns, as shown here:

$ ls

1st b folders misc proposals

8.16letter BOOKS letters Names temp

abc drafts memos newletter x

You can use the -x option to have names of files displayed horizontally, in as many lines as necessary For example,

$ ls -x

1st 8.16letter abc b BOOKS drafts

folders letters memos misc Names newletter

proposals temp x

You also can use the -1 (one) option to have files displayed one line per row (as the old version of ls did), in alphabetical order:

$ ls -l

1st

8.16letter

abc

b

BOOKS

drafts

folders

letters

memos

misc

Names

newletter

proposals

temp

Showing Nonprinting Characters

Occasionally you will create a filename that contains nonprinting characters. This is usually an accident, and when it occurs it can be hard to find or operate on such a file.

Suppose you mean to create a file named Budget but accidentally type CTRL-B rather than SHIFT-B. When you try to run a command to read or edit Budget, you will get an error message, because no file of that name exists. If you use ls to check, you will see a file with the apparent name of udget, since the CTRL-B is not a printing character. If a filename contains only nonprinting characters, you won’t even see it in the normal ls listing. You can force ls to show nonprinting characters with the -b option. This replaces a nonprinting character with its octal code, as shown in this example:

$ ls

udget Expenses

$ ls -b

\002udget Expenses

An alternative is the -q option, which prints a question mark in place of a nonprinting character:

$ ls -q

?udget Expenses

Sorting Listings

Several options enable you to control the order in which ls sorts its output. Two of these options are particularly useful.

You can have ls sort according to when each file was created or last modified with the -t (time) option. With this option, the most recently changed files are listed first. This form of listing makes it easy to find a file you worked on recently

To reverse the order of a sort, use the -r (reverse) option. By itself, ls -r lists files in reverse alphabetical order. Combined with the -t option, it lists oldest files first and newest ones last.

Combining Options to Is

You can use more than one option to the ls command simultaneously For example, the following shows the result of using the ls command with the options -F and -a on a home directory:

$ ls -aF

./ ../ .mailrc* .profile* Letters/ memos notes@

You can combine any number of options. In the following example, three options are given to the ls command: -a to get the names of all files, -t to list files in temporal order (the most recently modified file first), and -F to mark the type of file.

$ ls -Fat

./ memos@ Letters/ notes .profile* .mailrc* ../

The Long Form of Is

The ls command and the options discussed so far provide limited information about files. For instance, with these options, you cannot determine the size of files or when they were last modified. To get more information about files, use the -l (long format) option of ls.

Here is an example of what the long format of ls might look like:

$ ls -1

total 8

drwxr-xr-x 3 jmf group1 4096 Nov 29 02:34 Letters

Irwxr-xr-x 1 jmf group1 13 Apr 1 21:17 memos -> Letters/memos

-rwxr-xr-x 2 jmf group1 682 Feb 2 08:08 notes

The first line (“total 8”) in the output gives the amount of disk space used in blocks. (A block is a unit of disk storage. On Linux systems, a block contains 1,024 bytes; on Solaris, a block is 512 bytes. The command df can be used to determine the block size-see Chapter 13 for details.) The rest of the lines in the listing show information about each of the files in the directory.

Each of these lines contains seven fields. The name of the file is in the seventh field, at the far right. (In the listing for memos, you can see that there is an arrow with another filename after it. The file memos is a symbolic link to that file, Letters/memos.) To the left of the filename, in the sixth field, is the date when the file was created or last modified. To the left of that, in the fifth field, is its size in bytes.

The third and fourth fields from the left show the owner of the file (in this case, the files are owned by the user jmf), and the group the file belongs to (group1). The concepts of file ownership and groups are discussed later in this chapter.

The second field from the left contains the link count. For a file, the link count is the number of linked copies of that file. For example, the “2” in the link count for notes shows that there is a linked copy of it somewhere. For a directory, the link count is the number of directories under it plus two (one for the directory itself, and one for its parent). So the directory Letters must have one subdirectory, since it has a link count of three.

The first character in each line tells you what kind of file this is.

This directory contains one ordinary file, one directory, and one symbolic link. (Notice that even though notes is a hard linked file, it does not have an l next to it, because it is not a symbolic link like memos.) Special character files and block files are covered as part of the discussion of system administration in Chapter 14.

The rest of the first field, that is, the next nine characters (in these examples, rwxr-xr-x), contains information about the file’s permissions. Permissions determine who can work with a file or directory and how it can be used. Permissions are an important and somewhat complicated part of the UNIX file system that will be covered next.

Permissions

The UNIX file system is designed to support multiple users. When many users are sharing one file system, it is important to be able to restrict access to certain files. The system administrator wants to prevent other users from changing important system files, for example, and many users have private files that they want to restrict others from viewing. File permissions are designed to address these needs.

Permissions for Files

There are three classes of file permissions, for the three classes of users: the owner (or user) of the file, the group the file belongs to, and all other users of the system. The first three letters of the permissions field, as seen in the output from ls -l, refer to the owner’s permissions; the second three letters refer to the permissions for members of the file’s group; and the last three to the permissions for any other users.

In the entry for the file named notes in the ls -l example shown in the preceding section, the first three letters, rwx, show that the owner of the file can read (r) it, write (w) to it, and execute (x) it. The second group of three characters, r-x, indicates that members of the group can read and execute the file but cannot write to it. The last three characters, r-x, show that all others can also read and execute the file but not write to it.

If you have read permission for a file, you can view its contents. Write permission means that you can alter its contents. Execute permission means that you can run the file as a program.

Special Permissions

There are a few other codes that occasionally appear in permission fields. For example, the letter s can appear in place of an x in the user’s or group’s permission field. This s refers to a special kind of execute permission that is relevant primarily for programmers and system administrators (discussed in Chapters 12 and 13). From a user’s point of view, the s is essentially the same as an x in that place. Also, the letter lmay appear in place of an r, w, or x. This means that the file will be locked when it is accessed, so that other users cannot access it while it is being used. This and other aspects of permissions and file security are discussed in Chapter 12.

Permissions for Directories

For directories, read permission allows users to list the contents of the directory. Write permission allows users to create or remove files or directories inside that directory, and execute permission allows users to change to this directory using the cd command or use it as part of a pathname.

In the ls -l example shown earlier, your permission settings on the Letters directory allow other users on the system to list its contents with ls (read permission), and to change to the directory (execute permission). The settings do not allow them to create or delete files in Letters (write permission).

The chmod Command

In the ls -l example, all of the files and directories have the same permissions set. Anyone on the system can read or execute any of them, but other users are not allowed to write, or alter, these files. Normally you don’t want all your files set up this way. You will often want to restrict other users from being able to view your files, for example. At times, you may want to allow members of your work group to edit certain files, or even make some files public to anyone on the system.

The UNIX System allows you to set the permissions of each file you own. Only the owner of a file or the superuser can alter the file permissions. You can independently manipulate each of the permissions to allow or prevent reading, writing, or executing by yourself, your group, or all users.

To alter a file’s permissions, you use the chmod (change mode) command. You specify the changes you want to make with a sort of code. First, show which set of permissions you are changing with u for user, gfor group, or o for other. Second, specify how they should be changed with + (to add permission) or − (to subtract permission). Third, list the permissions to alter: r for read, w for write, or x for execute. Finally, specify the file or files that the changes refer to.

The following example shows the permissions for the file quotations, changes the permissions using the chmod command, and shows the result:

$ ls -1 quotations

-rwxr-xr-x 1 nate group1 346 Apr 27 03:32 quotations

$ chmod go-rx quotations

$ ls -1 quotations

-rwx 1 nate group1 346 Apr 27 03:32 quotations

As you can see, the chmod command removed (−) both read and execute (rx) permissions for group and others (go). Essentially, you just said, “change mode for group and other by subtracting read and execute permissions on the quotations file.”

You can also add permissions with the chmod command:

$ chmod ugo+rwx quotations

$ ls -1 quotations

-rwxrwxrwx 1 nate group1 346 Apr 27 03:32 quotations

Here, chmod adds (+) read, write, and execute (rwx) permissions for user, group, and other (ugo) for the file quotations. When changing permissions for everyone like this, you can use a (all) as an abbreviation for ugo. Note that there cannot be any spaces between letters in the chmod options.

Setting Absolute Permissions

The form of the chmod command using the ugo+/-rwx notation enables you to change permissions relative to their current setting. As the owner of the file, you can add or take away permissions as you please. Another form of the chmod command lets you set the permissions directly by using a numeric code to specify them.

This code represents a file’s permissions by three digits: one for owner permissions, one for group permissions, and one for others. These three digits appear together as one three-digit number. For example, the following command sets read, write, and execute permissions for the owner only and allows no one else to do anything with the file:

$ chmod 700 quotations

$ ls -1 quotations

-rwxrwxrwx 1 nate group1 346 Apr 27 03:32 quotations

The following table shows how permissions are represented by this code:

Each digit in the “700” represents the permissions granted to quotations. Each column of the table refers to one of the users-owner, group, or other. If a user has read permission, you add 4; to set write permission, you add 2; and to set execute permission, you add 1. The sum of the numbers in each column is the code for that user’s permissions.

Let’s look at another example. The next table shows how the command

$ chmod 754 quotations

$ ls -1 quotations

-rwxr-xr--1 nate group1 346 Apr 27 03:32 quotations

sets read, write, and execute permissions for the owner, read and execute permissions for the group, and read-only permission for other users:

Setting Permissions for Groups of Files

You can use wildcards to set permissions for groups of files and directories. For example, the following command will remove read, write, and execute permissions for both group and others for all files, except hidden files, in the current directory:

$ chmod go-rwx *

To set the permissions for all files in the current directory so that the files can be read, written, and executed by the owner only, type

$ chmod 700 *

Another feature of chmod is the -R (recursive) option, which applies changes to all of the files and subdirectories in a directory For example, the following makes all of the files and subdirectories in Emailreadable by you:

$ chmod -R u+r Email

Using umask to Set Permissions

The chmod command allows you to alter permissions on a file-by-file basis. The umask command allows you to do this automatically when you create any file or directory Everyone has a default umask setting that is either set up either by the system administrator or included in a shell configuration file. (These configuration files are described in the next chapter.)

With the umask command, you specify the permissions that will be given to all files created after issuing the command. This means you will not have to worry about the file permissions for each individual file you create. Unfortunately, using umask to specify permissions is a little bit complicated. There are two rules to remember:

§ umask uses a numeric code for representing absolute permissions just as chmod does. For example, 777 means read, write, and execute permissions for user, group, and others (rwxrwxrwx).

§ You specify the permissions you want by telling umask what to subtract from the full permissions value, 777 (rwxrwxrwx).

For example, after you issue the following command, all new files in this session will be given permissions of rwxr-xr-x:

$ umask 022

In this example, we want the new files to have the permission value 755. When we subtract 755 from 777, we get 022. This is the “mask” we used for the command.

To make sure that no one other than yourself can read, write, or execute your files, you can run the umask command at the beginning of your login session by putting the following line in your .profile file (sometimes, this file will be called .login or .bash_profile; see Chapter 4 for details):

umask 077

This is similar to using chmod 700 or chmod go-rwx, but this will apply to all files you create after the umask command is issued.

Changing the Owner of a File

Every file has an owner. When you create a file, you are automatically its owner. The owner usually has broader permissions for manipulating the file than other users.

Sometimes you need to change the owner of a file; for example, if you take over responsibility for a file that previously belonged to another user. Even if someone else “gives” you a file by moving it to your directory that does not make you the owner. One way to become the owner of a file is to make a copy of it-when you make a copy, you are the owner of the new file.

However, changing ownership by copying only works when the new owner copies the file from the old owner, which requires the new owner to have read permission on the file. A simpler and more direct way to transfer ownership is to use the chown (change owner) command.

The chown command takes two arguments: the login name of the new owner and the name of the file. The following makes liz the new owner of the file contact_info:

$ chown liz contact_info

Only the owner of a file (or the superuser) can use chown to change its ownership.

Like chmod, newer versions of chown include a -R (recursive) option that you can use to change ownership of all of the files in a directory. If Project is one of your directories, you can make liz its owner (and owner of all of its files and subdirectories) with the following command:

$ chown -R liz Project

Changing the Group of a File

Groups are meant to help sets of users who need to share files more closely than other users on the system. For example, all the students taking a particular class may belong to the same group, so that they can more easily share files when they collaborate on projects. Groups are defined and edited by the system administrator (for details, see Chapter 13).

Every file belongs to a group. Sometimes, such as when new groups are set up on a system or when files are copied to a new system, you may want to change the group to which a particular file belongs. This can be done using the chgrp (change group) command. The chgrp command takes two arguments, the name of the new group and the name of the file. The following command changes data_file so that it belongs to the group students:

$ chgrp students data_file

$ ls -1 data_file

-rwxrwx 1 liz students 812 Jan 27 11:20 data_file

Note that only the owner of a file (or the superuser) can change the group to which this file belongs.

You can use the -R (recursive) option of chgrp to change the group to which all the files in a directory belong. It works just like the -R option of chown.

Viewing Long Files

You know how use cat to view files. But cat isn’t very satisfactory for viewing files that contain more lines than will fit on your screen. When you use cat to display a file, it prints the contents on your screen without pausing, so that long files quickly scroll past. A quick solution, when you only need to view a small part of the file, is to use cat and then hit BREAK when the part you want to read comes on the screen. This stops the program, but it leaves the output on the screen, so if your timing is good, you may get what you want.

A somewhat better solution is to use the sequence CTRL-S, to make the output pause whenever you get a screen you want to look at, and CTRL-Q to resume scrolling. This way of suspending output to the screen works for all UNIX commands, not just cat. This is still awkward, though. The best solution is to use a pager-a program that is designed specifically for viewing files.

UNIX gives you a choice of two pagers, pg and more, which are standard with all versions of UNIX, as well as an enhanced pager called less, available for many versions of UNIX, including Linux. less has more features than more and has pretty much replaced it. The following sections describe pg, mention some of the features of more, and then describe many (but not all) of the features of less.

Using pg

The pg command displays one screen of text at a time and prompts you for a command after each screen. You can use the various pg commands to move back and forth by one or more lines, by half screens, or by full screens. You can also search for and display the screen containing a particular string of text.

Moving Through a File with pg

The following command displays the file newyork one screen at a time:

$ pg newyork

To display the next screen of text, press ENTER. To move back one page, type the hyphen or minus sign (−). You can also move forward or backward several screens by typing a plus or minus sign followed by the number of screens and hitting ENTER. For example, +3 moves ahead three screens, and −3 moves back three.

You use 1 to move one or more lines forward or backward. For example, −5l moves back five lines. To move a half screen at a time, type d or press CTRL-D.

Searching for Text with pg

You can search for a particular string of text by enclosing the string between slashes. For example, the search command

/invoices/

tells pg to display the screen containing the next occurrence of the string “invoices” in the file.

You can also search backward by enclosing the target string between question marks, as in

?invoices?

which scrolls backward to the preceding occurrence of “invoices” in the file.

Other pg Commands and Features

You can tell pg to show you several files in succession. The following command,

$ pg doc.1 doc.2

shows doc.1 first; when you come to the end of it, pg shows you doc.2. You can skip from the current file to the next one by typing n at the pg prompt. And you can return to the preceding file by typing p.

The following command saves the currently displayed file with the name new_doc:

s new_doc

To quit pg, type q or Q, or press the BREAK or DELETE key

Using pg to View the Output of a Command

You also can use pg to view the output of a command that would otherwise overflow the screen. For example, if your home directory has too many files to allow you to list them on one screen, you can send the output of ls -l to pg with this command:

$ ls -1 pg

This is an example of the UNIX pipe feature. The pipe symbol (|) redirects the output of a command to the input of another command. It is like sending the output to a temporary file and then running the second command on that file, but it is much more flexible and convenient. Like the redirection operators, > and <, the pipe construct is a general feature of the UNIX System. Chapter 4 discusses pipes in greater detail.

Using more

UNIX has another pager, more. Like pg, more allows you to move through a file by lines, half screens, or full screens, and it lets you move backward or forward in a file and search for patterns.

To display the file newyork, just enter the command

$ more newyork

To tell more to move ahead by a screen, press the SPACEBAR. To move ahead one line, press ENTER. The commands for half-screen motions, d and CTRL-D, are the same as in pg. To move backward by a screen, use b or CTRL-B.

Using less

The less command, an enhanced version of the more command, is a feature-rich pager that can be used to interactively display portions of a file. It can be used to move either forward or backward in a file. Since less reads in portions of files, rather than entire files, it is very efficient for large files.

This will display the file newyork with less:

$ less newyork

Less Options

The less command has many useful options. For example, the -p option can be used to start less at the first occurrence of the pattern you specify (where the pattern is entered after the -p option) To display the file sanfrancisco, beginning with the first time the pattern “SFO" appears, you can use

$ less -p SFO sanfrancisco

Among the other options supported by less are -s, which squeezes consecutive blank lines into a single blank line; -S, which chops off lines longer than the screen (discarding them instead of folding them into the next line); and -U, which displays backspaces and carriage returns as control characters.

Less Commands

Many commands can be used with the less pager to display different parts of a file. For example, you can scroll forward one window by entering SPACEBAR or f, and you can scroll backward one window by entering b. You can scroll forward one line by entering e or pressing ENTER, and you can scroll backward one line by entering y. You can scroll to the next occurrence of the string “pattern” using the command /pattern, and you can scroll to the preceding occurrence of this string using the command ?pattern. You can find the next and preceding lines that do not contain the string “pattern” using the commands /!pattern and ?!pattern, respectively

You can use various commands to move the cursor when using less to display files. For example, you can move one space left using the LEFT ARROW key or ESC-H; you can move one space right using the RIGHT ARROW key or ESC-L. You can move one word to the left with ESC-B and one word to the right with ESC-W. You can also do some editing: BACKSPACE deletes the character to the left of the cursor, DELETE deletes the character under the cursor, CTRL-BACKSPACE deletes the word to the left of the cursor, and CTRL-DELETE deletes the word under the cursor.

Viewing the Beginning or End of a File

You can use cat, pg, more, and less to view whole files. But often what you really want is to look at the first few lines or the last few lines of a file. For example, if a database file is periodically updated with new account information, you may want to see whether the most recent updates have been done. You also might want to check the last few lines of a file to see if it has been sorted, or read the first few lines of each of several files to see which one contains the most recent version of a note. The head and tail commands are specifically designed for these jobs.

head shows you the beginning of a file, and tail shows you the end. For example, the command shown here displays the first ten lines of transactions.

$ head transactions

and the following command displays the last ten lines:

$ tail transactions

To display some other number, say the last three lines, you give head or tail a numerical argument. This command shows only the last three lines:

$ tail -3 transactions

A useful feature of tail is the -f (follow) option. This lets you use tail to check on the progress of a program that writes its output to a file. Suppose a file transfer program is getting information from a remote system and putting it in the file newdata. In this example, tail displays the last three lines of newdata, waits (sleeps) for a short time, looks to see if there has been any new input, displays any new lines, and so on:

$ tail -3 -f newdata

Printing Files

The UNIX System includes a collection of programs, called the lp system, for printing files and documents. You can use it to print everything from simple text files to large documents with complex formats. It provides a simple, uniform interface to a wide variety of printers.

The lp system is itself large and complex, but fortunately its complexity is well hidden from users. In fact, three basic commands, lp, lpstat, and cancel, are all you need to know to use this system. (On Linux, these three basic commands have different names; they are lpr, lpq, and lprm, respectively) This section describes how to use these commands to print your files. Administrators will also need to know how to set up and maintain the lp system. The administration of the lp system is discussed in Chapter 13.

Sending Output to the Printer

The basic command for printing a file is lp (line printer) (or on Linux, lpr). This command prints the file research.nov as follows:

$ lp research.nov

request id is lsr1–142 (1 file)

The confirmation message from lp returns a “request ID” that you can use to check on the status of the job or to cancel it if you want. As this example shows, the request ID is made up of two parts: the printer’s name (in this case, lsr1) and a number that identifies your particular request.

You can print several files at once by including all of them in the arguments to lp. For instance,

$ lp res*

request id is lsr1–154 (3 files)

prints all files whose names begin with res.

Specifying a Printer

The lp command does not ask you which printer to use. There may be several printers on the system, but one of them will be the system default. Unless you specify otherwise, this is the printer that lp uses. To find out which printers are available, you can ask your system administrator, or you can use the lpstat command, described in the next section.

Sometimes you want to use a printer that is not the default. To specify a particular printer, use the -d (destination) option, followed by the printer’s name. For example,

$ lp -d laser2 flightinfo

sends flightinfo to the printer named laser2.

You can change the default printer that lp uses for your jobs by setting the variable LPDEST. The next chapter explains how to set this type of variable, and how to make the change permanent by including it in your shell configuration file.

Print Spooling

When you print a file on the UNIX System, you do not have to wait until the file is printed (or until it is sent to the printer) before continuing with other work, and you do not have to wait until one print job is finished before sending another. lp spools its input to the UNIX print system, which means that it tells the print system what file to print and how to print it, and then leaves the work of getting the file through the printer to the system.

Your job is submitted and spooled, but it is not printed at the precise time you enter the lp command, and lp does not automatically tell you when your job is actually finished. If you want to be notified when it is printed, use the -m (mail) option. For example,

$ lp -m -d laser2 flightinfo

sends you mail when your file is successfully printed.

If you change the file between the time you issue the lp command and the time it actually goes to the printer, it is the changed file that will be printed. In particular, if you delete the file, or rename it, or move it to another directory the print system will not find it, and it will not be printed. To avoid this, use the -c (copy) option. The command

$ lp -c -d laser2 flightinfo

copies flightinfo to a temporary file in the print system and uses that copy as the input to the printer. Any changes you make to flightinfo after you issue this command will not appear in the printed output.

Printing Command Output

You use lp to print files. As Chapter 4 explains, the concept of a file in UNIX includes the output of a command (its standard output), so you can also use lp to print the output of a command directly To do this, use the UNIX pipe feature to send the output of the command to lp. For example, the following prints the long form of the listing of your current directory:

$ ls -1 lp

When you use a pipe to connect the output of a command to lp, the output does not appear on your screen.

Standard output and the pipe mechanism are general features of the UNIX system provided by the shell. They are used in many ways and with many different commands, not just with lp. Chapter 4 covers these in much greater detail.

Using lpstat to Monitor the Print System

Because your print jobs may not print immediately, and because they may be sent to printers located away from your desk, you sometimes need to check on their status. The lpstat command (on LINUX this is the lpq command) provides a way to get this and other useful information, such as which printers are currently available on the system and how many other print jobs are scheduled.

One of the most important uses of lpstat is to see if your print jobs are being taken care of or if there is some problem with the system. The following shows that a job is scheduled for printing but has not yet started printing:

$ lpstat

lsr1–142 jmf 1730 Apr 20 00:29

Suppose you send another file to be printed and then use lpstat to check again:

$ lp letter.draft

$ lpstat

lsr1–142 jmf 1730 Apr 20 00:29 on lsr1

sysptr-136 jmf 1930 Apr 20 00:32

This tells you that the first job is now printing on lsr1 and that the second job is scheduled for the default printer, sysptr.

If you need to get a file printed quickly, you may want to check on the status of a printer before you send the job to it. Use the -d (destination) option with the name of the printer. For example,

$ lpstat -d laser2

laser2 accepting requests since Wed Mar 6 10:23:12 2005

printer laser2 is idle. enabled since Tue Apr 15 10:22:04 2005. available.

shows the status of printer laser2. The -s (system) option shows the status of the system default printer.

To get an overview of the whole print system-what printers are available and how much work each has-use lpstat -t to print a brief summary of the status of the system.

Canceling Print Jobs

Sometimes you need to cancel a print job you have already submitted. You may have used the wrong file, or you may want to make more changes before it is printed. The cancel command (on Linux this command is called lprm) allows you to stop any of your print jobs, even one currently being printed. For example, if lp gave the ID inkjet-133 to one of your print jobs, and you want to stop it, you can use the following command to delete it from the printer system:

$ cancel inkjet-133

request"inkjet-133" canceled

If you did not save the number of the job when you submitted it, you can get it with lpstat.

Formatting

lp prints exactly what you give it. It does not add anything to the file-no headers, no page numbers, and no formatting. The UNIX System leaves responsibility for all of these sorts of things to formatting programs.The programs pr and fmt can be used for simple formatting, such as adding a header or line numbers to a file before printing it. These commands are described in Chapter 19, in the section “Editing and Formatting Files.”

Summary

In this chapter, you were introduced to the UNIX System concepts of files and directories, and to the basic commands you can use to manage them. You learned how to create, name, and move files and directories. You learned how you can list the files in a directory, view the contents of a file, search for files, and print files.

In the next chapter, you will learn more about entering commands and customizing the UNIX System environment. Chapter 5 explains how to use text editors to create and modify files. More advanced commands for working with files are explained in Chapter 19, “Filters and Utilities.”

Table 3–2 summarizes the commands for using files and directories that were introduced in this chapter.

How to Find Out More

Many of the commands introduced in this chapter have more options than were listed. The find command, for example, has many other options for specifying the type of file you are searching for. The UNIX man (manual) pages describe the commands in more detail, including all of these options. Chapter 2 explains how to use the UNIX man pages. You can also find man pages on the web. One useful site, which includes many UNIX variants, is

· FreeBSD Hypertext Man Pages: http://www.freebsd.org/cgi/man.cgi

This book includes a command reference similar in style to the man pages:

· Robbins, Arnold. UNIX in a Nutshell. 4th ed. Sebastopol, CA: O’Reilly Media, 2006.

Most books on UNIX have information about the topics covered in this chapter. Two good introductions for beginners are

· Ray, Deborah, and Eric Ray UNIX: Visual QuickStart Guide. 3rd ed. Berkeley, CA: Peachpit Press, 2006.

· Peek, Jerry, et al. Learning the UNIX Operating System. 5th ed. Sebastopol, CA: O’Reilly Media, 2001.