Advanced bash techniques - Linux Nitty Gritty: Working at the Ubuntu Command-Line Prompt (2011)

Linux Nitty Gritty: Working at the Ubuntu Command-Line Prompt (2011)

Advanced bash techniques

Although they might be classed as advanced command-line skills, redirection, piping and brace expansion are not hard to understand. However, the fact they will only find use by more advanced users puts them outside the scope of this book, and they’re explained here in a concise manner.

Redirection

Sometimes it’s useful to send the output of a command into a file, rather than displaying it on screen. This is the principle of output redirection.

Let’s say you wanted to make a permanent record of a file listing. To do this using redirection, type the following:

ls > listing.txt

The right angle symbol (>) tells bash to redirect output of the ls command into the file listing.txt, rather than send it to the screen. Think of the angle-bracket as a funnel—the output of the ls command is “poured” into the listing.txtfile.

If listing.txt doesn’t exist, it will be created. If a file of that name already exists, its contents will be overwritten. Using two angle brackets (>>) instead of one will cause the output to be added to the bottom of an existing file.

You can view your new file in a text editor, or by using the less text viewer:

less listing.txt

To redirect the contents of a file into a particular command, the reverse angle bracket is used (<). Not all commands are designed to take redirected input, of course, but the sort command is one of them. It can be used to alphabetically sort a list. If we had a file called shoppinglist.txt, that contained a list of products we wanted to buy from the store, and we wanted to sort the list alphabetically, we could redirect the contents of the file into the command, like so:

sort < shoppinglist.txt

Piping

Piping is similar to redirection except that it’s used to pass the output of one command to another—rather like connecting a pipe between the two commands, in fact!

The pipe symbol (|) is used for this purpose. On most keyboards this symbol can be found by hitting Shift and backslash.

Taking the previous example, if we wanted to pipe the output of the ls command into sort, to alphabeticize it, we would type the following:

ls|sort

A common use of piping is to pass the output of a command to the grep command, which searches for a word or phrase. For example, if you have a huge file listing containing hundreds of files/folders, and want to find if report.docis amongst them, you could type the following:

ls|grep report.doc

Incidentally, if you see no output, the search phrase wasn’t found.

Brace expansion

Brace expansion is a labor-saving trick. It’s best explained by example.

Let’s say you’ve come back from a European tour with a digital camera full of photos, and want to create four folders on your PC, named photos_germany, photos_france, photos_england, and photos_spain. You could create each manually using the mkdir command, but brace expansion lets you to create all four in one fell swoop, as follows:

mkdir photos_{germany,france,england,spain}

If we view a file listing, we’ll find we have four new folders:

ls

photos_england photos_france photos_germany photos_spain

In other words, anything within the braces ({}) is individually added to photos_ and a new folder is created using that name. Each item within the braces should be separated by a comma.

If you subsequently wanted to delete the photos_spain and photos_france folders, rather than type two separate rm commands, you could use brace expansion again:

rm -r photos_{france,spain}

Brace expansion works with all commands, not just mkdir and rm, and is packed with features. For example, rather than specifying individual expansions separated by commas, a range of letters or numbers can be specified. If you wanted to create 10 folders named photos (i.e. photos1, photos2, etc.), the following will do the trick:

mkdir photos{1..10}

Letters can be specified instead of letters. The following will create photosA, photosB etc., all the way to Z:

mkdir photos{A..Z}

Wildcards

Worth mentioning alongside brace expansion is the concept of wildcards. The term is taken from Poker, where a card that’s “wild” can represent any card. This might help you understand wildcards at the command-line: they’re symbols that specify any character(s).

The most commonly-used wildcard is the asterisk (*), used to indicate any combination (or number) of characters.

If you wanted to delete all files in a folder, you could type the following:

rm *

In this example, the asterisk wildcard represents any filename that contains any number of characters. That’s all of them!

To return to the vacation folder example used above, the photos_spain, photos_germany etc. folders could be deleted like this:

rm -r photos_*

This will delete any file whose name begins with photos_, but has any number of characters following (should there be a folder called photos_, with nothing after it, that too will be deleted).

A question mark (?) can be used to indicate any single character. The following will delete photos1, photosA, photos!, and so on:

rm -r photos?

The following will delete photos1, photos3 and photos4, but leave in place any other folder named photos and followed by any character:

rm -r photos[134]

In other words, each character within the square brackets is individually substituted within the filename, in a similar way to brace expansion. However, wildcards are designed to match patterns of characters in existing files (or data), so won’t work when creating files.