Ubuntu Unleashed 2017 Edition (2017)

---

There is no GUI for Amanda. Configuration is done in the time-honored UNIX tradition of editing text configuration files located in /etc/amanda. The default installation in Ubuntu includes a sample cron file because it is expected that you will be using cron to run Amanda regularly. The client utilities are installed with the package amanda-client; the Amanda server is called amanda-server. Install both. As far as backup schemes are concerned, Amanda calculates an optimal scheme on-the-fly and schedules it accordingly. It can be forced to adhere to a traditional scheme, but other tools are possibly better suited for that job.

The man page for Amanda (the client is amdump) is well written and useful, explaining both the configuration of Amanda and detailing the several programs that actually make up Amanda. The configuration files found in /etc/amanda are well commented; they provide a number of examples to assist you in configuration.

The program’s home page is www.amanda.org. There you can find information about subscribing to the mail list and links to Amanda-related projects and a FAQ.

Alternative Backup Software

Commercial and other freeware backup products do exist; BRU and Veritas are good examples of effective commercial backup products. Here are some useful free software backup tools that are not installed with Ubuntu:

flexbackup—This backup tool is a large file of Perl scripts that makes dump and restore easier to use. flexbackup’s command syntax can be found by using the command with the -help argument. It also can use afio, cpio, and tar to create and restore archives locally or over a network using rsh or ssh if security is a concern. Its home page is www.edwinh.org/flexbackup/. Note that it has not received any updates or changes in a very long time.

afio—This tool creates cpio formatted archives but handles input data corruption better than cpio (which does not handle data input corruption very well at all). It supports multi-volume archives during interactive operation and can make compressed archives. If you feel the need to use cpio, you might want to check out afio at http://freshmeat.net/projects/afio/.

Many other alternative backup tools exist, but covering all of them is beyond the scope of this book. Two good places to look for free backup software are Freshmeat (www.freshmeat.net) and Google (www.google.com/linux).

Copying Files

Often, when you have only a few files that you need to protect from loss or corruption, it might make better sense to simply copy the individual files to another storage medium rather than to create an archive of them. You can use the tar, cp, rsync, or even the cpio commands to do this; you can also use a handy file management tool known as mc. Using tar is the traditional choice because older versions of cp did not handle symbolic links and permissions well at times, causing those attributes (characteristics of the file) to be lost; tar handled those file attributes in a better manner. cp has been improved to fix those problems, but tar is still more widely used. rsync has recently been added to Ubuntu and is an excellent choice for mirroring sets of files, especially when done over a network.

To illustrate how to use file copying as a backup technique, the examples here show how to copy (not archive) a directory tree. This tree includes symbolic links and files that have special file permissions we need to keep intact.

Copying Files Using tar

One choice for copying files into another location is to use the tar command; you just create a tar file that is piped to tar to be uncompressed in the new location. To accomplish this, first change to the source directory. Then the entire command resembles this:

Click here to view code image

matthew@seymour:~$ tar -cvf files | (cd target_directory ; tar -xpf)

In this command, files are the filenames you want to include; use * to include the entire current directory.

Here is how this command works: You have already changed to the source directory and executed tar with the cvf arguments that tell tar to

c—Create an archive.

v—Verbose; lists the files processed so you can see that it is working.

f—The filename of the archive will be what follows. (In this case, it is -.)

The following tar commands can be useful for creating file copies for backup purposes:

l—Stay in the local file system (so that you do not include remote volumes).

atime-preserve—Do not change access times on files, even though you are accessing them now (to preserve the old access information for archival purposes).

The contents of the tar file (held for us temporarily in the buffer, which is named -) are then piped to the second expression, which extracts the files to the target directory. In shell programming (refer to Chapter 14, “Automating Tasks and Shell Scripting”), enclosing an expression in parentheses causes it to operate in a subshell and be executed first.

First you change to the target directory, and then

x—Extract files from a tar archive.

p—Preserve permissions.

f—The filename will be -, the temporary buffer that holds the files archived with tar.

Compressing, Encrypting, and Sending tar Streams

The file copy techniques using the tar command in the previous section can also be used to quickly and securely copy a directory structure across a LAN or the Internet (using the ssh command). One way to make use of these techniques is to use the following command line to first compress the contents of a designated directory and then decompress the compressed and encrypted archive stream into a designated directory on a remote host:

Click here to view code image

matthew@seymour:~$ tar -cvzf data_folder | ssh remote_host '( cd ~/mybackup_dir; tar -xvzf )'

The tar command is used to create, list, and compress the files in the directory named data_folder. The output is piped through the ssh (Secure Shell) command and sent to the remote computer named remote_host. On the remote computer, the stream is then extracted and saved in the directory named /mybackup_dir. You are prompted for a password to send the stream.

Copying Files Using cp

To copy files, we could use the cp command. The general format of the command when used for simple copying is as follows:

Click here to view code image

matthew@seymour:~$ cp -a source_directory target_directory

The -a argument is the same as giving -dpR, which would be

-d—Preserves symbolic links (by not dereferencing them) and copies the files that they point to instead of copying the links.

-p—Preserves all file attributes if possible. (File ownership might interfere.)

-R—Copies directories recursively.

The cp command can also be used to quickly replicate directories and retain permissions by using the -avR command-line options. Using these options preserves file and directory permissions, gives verbose output, and recursively copies and re-creates subdirectories. You can also create a log of the backup during the backup by redirecting the standard output like this:

Click here to view code image

matthew@seymour:~$ sudo cp -avR directory_to_backup destination_vol_or_dir 1 > /root/backup_log.txt

or

Click here to view code image

matthew@seymour:~$ sudo cp -avR ubuntu /test2 1 > /root/backup_log.txt

This example makes an exact copy of the directory named /ubuntu on the volume named /test2 and saves a backup report named backup_log.txt under /root.

Copying Files Using mc

The Midnight Commander (available in the Universe repository, under the package mc; see Chapter 9, “Managing Software,” for how to enable the Universe and Multiverse repositories) is a command-line file manager that is useful for copying, moving, and archiving files and directories. The Midnight Commander has a look and feel similar to the Norton Commander of DOS fame. By executing mc at a shell prompt, a dual-pane view of the files is displayed. It contains drop-down menu choices and function keys to manipulate files. It also uses its own virtual file system, enabling it to mount FTP directories and display the contents of tar files, gzip tar files (.tar.gz or .tgz), bzipfiles, DEB files, and RPM files, as well as extract individual files from them. As if that is not enough, mc contains a File Undelete virtual file system for ext2/3 partitions. By using cd to “change directories” to an FTP server’s URL, you can transfer files using FTP. The default font chosen for Ubuntu makes the display of mc ugly when used in a tty console (as opposed to an xterm), but does not affect its performance.

In the interface, pressing the F9 key drops down the menu, and pressing F1 displays the Help file. A “feature” in the default GNOME terminal intercepts the F10 key used to exit mc, so use F9 instead to access the menu item to quit, or just click the menu bar at the bottom with your mouse. The configuration files are well documented, and it would appear easy to extend the functionality of mc for your system if you understand shell scripting and regular expressions. It is an excellent choice for file management on servers not running X.

Using rsync

An old favorite for backing up is rsync. One big reason for this is because rsync enables you to copy only those files that have changed since the last backup. So although the initial backup might take a long time, subsequent backups are much faster. It is also highly configurable and can be used with removable media such as USB hard drives or over a network. Here is one way to use rsync.

First, create an empty file and call it backup.sh:

Click here to view code image

matthew@seymour:~$ sudo touch backup.sh

Then, using your favorite text editor, enter the following command into the file and save it:rsync --force --ignore-errors --delete --delete-excluded --exclude-

Click here to view code image

from=/home/matthew-exclude.txt --backup --backup-dir='date +%Y-%m-%d' -av /
/media/externaldrive/backup/Seymour

Make the file executable:

Click here to view code image

matthew@seymour:~$ sudo chmod +x backup.sh

This command uses several options with rsync and puts them in a script that is quick and easy to remember and run. You can run the script at the command line using sudo sh ./backup.sh or as an automated cron job.

Here is a rundown of what is going on in the command. Basically, rsync is told to copy all new and changed files (what to back up) and delete from any existing backup any files that have been deleted on the source (and back them up in a special directory, just to be safe). It is told where to place the backup copy and is given details on how to deal with specific issues in the process. Read the rsync man page for more options and to customize to your needs.

Following are the options used here:

--force—Forces deletion of directories in the target location that are deleted in the source, even if the directories in the destination are not empty.

--ignore-errors—Tells --delete to go ahead and delete files even when there are I/O errors.

--delete—Deletes extraneous files from destination directories.

--delete-excluded—Also deletes excluded files from destination directories.

--exclude-from=/home/matt-exclude.txt—Prevents backing up files or directories listed in this file. (It is a simple list with each excluded directory on its own line.)

--backup—Creates backups of files before deleting them from a currently existing backup.

--backup-dir='date +%Y-%m-%d'—Creates a backup directory for the previously mentioned files that looks like this: 2013-07-08. Why this format for the date? Because it is standard, as outlined in ISO 8601 (see: http://www.iso.org/iso/home/standards/iso8601.htm). It is clear, works with scripts, and sorts beautifully, making your files easy to find.

-av—Tells rsync to use archive mode and verbose mode.

/—Denotes the directory to back up. In this case, it is the root directory of the source, so everything in the file system is being backed up. You could put /home here to back up all user directories or make a nice list of directories to exclude in the file system.

/media/externaldrive/backup/seymour—Sets the destination for the backup as the /backup/seymour directory on an external hard drive mounted at /mount/externaldrive.

To restore from this backup to the same original location, you reverse some of the details and may omit others. Something like this works nicely:

Click here to view code image

matthew@seymour:~$ rsync --force --ignore-errors --delete --delete-excluded
/media/externaldrive/backup/seymour /

This becomes even more useful when you think of ways to script its use. You could create an entry in crontab, as described in Chapter 14, “Automating Tasks and Shell Scripting.” Even better, you could set two computers to allow for remote SSH connections using private keys created with ssh-keygen, as described in Chapter 19, “Remote Access with SSH, Telnet, and VNC,” so that one could back up the files from one computer to the other computer without needing to login manually. Then you could place that in an automated script.

Version Control for Configuration Files

For safety and ease of recovery when configuration files are corrupted or incorrectly edited, the use of a version control system is recommended. In fact, this is considered an industry best practice. Many top-quality version control systems are available, such as Git, Subversion, Mercurial, and Bazaar. If you already have a favorite, perhaps one that you use for code projects, you can do what we describe in this section using that version control system. The suggestions here are to get people thinking about the idea of using version control for configuration files and to introduce a few well-used and documented options for those who are unfamiliar with version control. First, some background.

Version control systems are designed to make it easy to revert changes made to a file, even after the file has been saved. This is done a little bit differently by each system, but the basic idea is that not only is the current version of the file saved, but each and every version that existed previously is also saved. Some version control systems do this by saving the entire file every time. Some use metadata to describe just the differences between each version. In any case, it is possible to roll back to a previous version of the file, to restore a file to a state before changes were made. Developers who write software are well aware of the power and benefit to being able to do this quickly and easily; it is no longer required that the file editor remember the technical details of where, what, or even how a file has been edited. When a problem occurs, the file is simply restored to its previous state. The version control system is also able to inform the user where and how each file has changed at each save.

Using a version control system for configuration files means that every time a configuration is changed, those changes are recorded and tracked. This enables easy discovery of intruders (if a configuration has been changed by an unauthorized person trying to reset, say, the settings for Apache so that the intruder can allow a rogue web service or site to run on your server), easy recovery from errors and glitches, and easy discovery of new features or settings that have been enabled or included in the configuration by software upgrades.

Many older and well-known tools do this task, such as changetrack, which is quite a good example. All of them seek to make the job of tracking changes to configuration files easier and faster, but with the advances in version control systems, most provide very little extra benefit. Instead of suggesting any of these tools, you are probably better off learning a modern and good version control system. One exception is worth a bit of discussion because of its ability to work with your software package manager, which saves you the task of remembering to commit changes to your version control system each time the package manager runs. This exception is etckeeeper.

etckeeper takes all of your /etc directory and stores the configuration files from it in a version control system repository. You can configure the program by editing the etckeeper.conf file to store data in a Git, Mercurial, Bazaar, or Subversion repository. In addition, etckeeper connects automatically to the APT package management tool used by Ubuntu and automatically commits changes made to /etc and the files in it during normal software package upgrades. Other package managers, such as Yum, can also be tracked when using other Linux distributions such as Fedora. It even tracks file metadata that is often not easily tracked by version control systems, like the permissions in /etc/shadow.