Ubuntu 15.04 Server with systemd: Administration and Reference (2015)

Part II. Services

Chapter 5. Managing Services with systemd

A single Linux system can provide several different kinds of services, ranging from security to administration, and including more obvious Internet services like web and FTP sites, e-mail, and printing. Security tools, such as the Secure Shell (SSH) and Kerberos run as services, along with administrative network tools, such as Dynamic Host Control Protocol (DHCP) and Lightweight Directory Access Protocol (LDAP). The network connection interface is, itself, a service that you can restart at will. Each service operates as a continually running daemon looking for requests for its particular services. In the case of a web service, the requests come from remote users. You can turn services on or off by starting or shutting down their daemons.

System start up is managed by the systemd service. The original System V init system for starting individual services has been phased out. The Upstart service used in previous Ubuntu releases has been deprecated. For information about changing from Upstart to systemd see:

https://wiki.ubuntu.com/SystemdForUpstartUsers

systemd

Linux systems traditionally used the Unix System V init daemon to manage services by setting up runlevels at which they could be started or shutdown. Linux has since replaced the SystemV init daemon with the systemd init daemon. Whereas the System V init daemon would start certain services when the entire system started up or shut down using shell scripts run in sequence, systemd is uses sockets for all system tasks and services. systemd sets up sockets for daemons and coordinates between them as they start up. This allows systemd to start daemons at the same time (in parallel). Should one daemon require support from another, systemd coordinates the data from their sockets (buffering), so that one daemon receives the information from another daemon that it needs to continue. This parallel start up compatibility allows for very fast boot times.

In effect, you can think of systemd as a combination of System V init scripts and the inetd daemon (xinetd), using socket activation applied to all system start up tasks and to network servers. The socket activation design was originally inspired by the inetd service that used sockets (AT_INET) to start internet daemons when requested. The socket activation design was used by in Apple's OS X system to apply to all sockets (AF_UNIX). This allowed all start up processes to start at the same time in parallel, making for very fast boot times. Sockets are set up and managed by systemd. When D-BUS needs to write to journald (logging), it writes to the systemd-journald socket set up and managed by systemd. It does not have to communicate directly with journald. This means that services no longer have to be started and shutdown in a particular sequence as they were under System V. They can all start and stop at the same time. Also, as systemd controls the socket, if a service fails, it socket remains in place. The service can be restarted using the same socket with no loss of information. systemd manages all types of sockets including UNIX (system), INET (network), NETLINK (devices), FIFO (pipes), and POSIX (messages). See the following for more details:

http://fedoraproject.org/wiki/Systemd
https://wiki.ubuntu.com/systemd
http://www.freedesktop.org/wiki/Software/systemd/

systemd sets up sockets for all system tasks and services. Configuration for systemd tasks are defined in unit files in /lib/systemd/system directory. In this respect, systemd files replace the entries that used to be in the Sys V init's /etc/inittab file. systemd also has its own versions ofshutdown, reboot, halt, init, and telinit, each with their own man page.

systemd is entirely compatible with both System V scripts in the /etc/init.d directory and the /etc/fstab file. The SystemV scripts and /etc/fstab are treated as additional configuration files for systemd to work with. If System V scripts are present in the /etc/init.d directory, it will use them to generated a corresponding unit configuration file, if there are no corresponding systemd unit configuration files already. systemd configuration always takes precedence. systemd will also, if needed, use the start and stop priority files in the System V init /etc/rc.d directories to determine dependencies. Entries in /etc/fstab are used to generate corresponding systemd unit files that are then used to manage file systems. systemd also supports snapshots that allow restoring services to a previous state.

The systemd configuration (.service) files are located in the /lib/systemd/system directory and are considered system files that you should not modify. It is possible to copy them to the /etc/systemd/system directory and make changes to the copies. File in the /etc/systemd/system file take precedence. The configuration file for the logind daemon, logind.conf, is located in the /etc/systemd directory.

Ubuntu also uses systemd services for time (timedated), location (localed), login (logind), and hostname (hostnamed) (systemd-services package). They can be managed with corresponding systemd control applications: timedatectl, localectl, and hostnamectl. Ubuntu uses the shim daemon (systemd-shim) to interface systemd daemons through Dbus. Configuration files located at /etc/dbus-1/system.d.

systemd basic configuration files

You can configure systemd for system, login manager, users, and, journal service using the configuration files located in the /etc/systemd directory. When run for a system service, systemd uses the system.conf file, otherwise it uses the user.conf file. You can set options such as the log level (LogLevel) and resource size limits. See the man page for systemd.conf (system.conf and user.conf), logind.conf, and journald.conf for details on the options available.

units

systemd organizes tasks into units, each with a unit configuration file. There are several types of units (see Table 5-1 ). A unit file will have an extension to its name that specifies the type of unit it is. Service types have the extension .service, and mount types have the extension .mount. The service type performs much the same function as System V init scripts. Services can be stopped, started and restarted. The systemctl command will list all units, including the ones it generates.

Units can also be used for socket, device, and mount tasks. The socket type implements the kind of connection used for inetd and xinetd, allowing you to start a service on demand. The device type references devices as detected by udev. The mount type manages a file system's mount point, and automount activates that mount point should it be automounted. An automount unit file has a corresponding mount unit file, which it uses to mount a file system. Similarly, a socket type usually has a corresponding service type used to perform a task for that socket.

Within each unit are directives that control a service, socket, device, or mount point. Some directives are unique to the type of unit. These are listed in the man page for that service, such as systemd.service for the service unit, or systemd.mount for a mount unit (see Table 5-1 ).

man systemd.service

Options common to units are listed in the systemd.exec and systemd.unit pages. The systemd.unit page lists directives common to all units such as Wants, Conflicts, Before, SourcePath, and Also. The systemd.exec pages list options for the execution of a program for a unit, such as the starting of a server daemon. These include options such as User, group, WorkingDirectory, Nice, Umask, and Environment. The systemd.exec page covers options for service, socket, mount, and swap units. The systemd.directives man page provides a listing of all systemd unit options and the man page for each option.

Table 5-1: systemd unit types and man pages

The target unit is used to group units. For example, targets are used to emulate runlevels. A multi-user target groups units (services) together that, in System V, would run on runlevel 3. In effect, targets group those services that run on a certain runlevel for a certain task. The printer target activates the CUPS service, and the graphical target emulates runlevel 5. A target can also be used to reference other targets. A default target designated the default runlevel. Some unit files are automatically generated by systemd. For example, operations specified in the /etc/fstab are performed by mount units, which are automatically generated from the fstab entries.

Units can be dependent on one another, where one unit may require the activation of other units. This dependency is specified using directories with the .wants extension. For example, the poweroff.target is dependent on the plymouth-poweroff service. The directorypoweroff.target.wants has a symbolic links to this service. Should you want a service dependent on your graphical desktop, you can add symbolic links to it in the graphical.target.wants directory.

It is important to distinguish between the wants directories in the /etc/systemd/system directory and those in the /lib/systemd/system directory. Those in the /lib/systemd directory are set up by your system and should be left alone. To manage your own dependencies, you can set up corresponding wants directories in the /etc/systemd/system directory. The /etc/systemd directory always takes priority. For example, in the /etc/systemd/system/multi-user.target.wants directory you can place links to services that you want started up for the multi-user.target (runlevel 3). Your system automatically installs links for services you enable, such as ufw.service and vsftpd.service. These are all links to the actual service files in the /lib/systemd/system directory. The multi-user.target.wants directory holds a link to ufw.sevice, starting up the firewall. Theprinter.target.wants directory has a link to the cups service.

Disabling a service removes its link from its wants directory in /etc/systemd/system. For example, disabling the vsftpd service removes its link from the /etc/systemd/system/multi-user.target.wants directory. The original service file, in this case vsftpd.service, remains in the/lib/systemd/system directory. If you enable the service again, a link for it is added to the /etc/systemd/system/multi-user.target.wants directory.

The /etc/systemd/system directory also hold links to services. The /etc/systemd/system/syslog.service file is a link to /lib/systemd/system/rsyslog.service. The /etc/systemd/system/dbus-org.freedesktop.Avahi.service link references /lib/systemd/system/avahi-daemon.service.

To manage systemd you can use systemctl. The older service management tool, service, has been modified to use systemctl to perform actions on services such as starting and stopping.

unit file syntax

A unit file is organized into sections designated by keywords enclosed in brackets. All units have a unit section, [Unit], and an install section, [Install]. The options for these sections are descript in the systemd.unit Man page. Comments can be written by beginning a line with # or ;character. See Table 5-2 for a listing of commonly used Unit and Install section options.

The Unit section of a unit file holds generic information about a unit. The Description option provides information about the task the unit manages, such as the Vsftpd server as shown here.

Description=vsftpd FTP server

The Documentation option lists URIs for the application's documentation.

Documentation=man:dhcpd(8)

Note: The syntax for unit file is base on .desktop files, which in turn are inspired by Windows ini files. The .desktop type of file conforms to the XDB Desktop Entry Specification.

Types of dependencies can be specified using the Before, After, Requires, Wants, and Conflicts options in the unit section. After and Before configure the ordering of a unit. In the following After option in the vsftpd.service file, the vsft[d service is started after networking.

After= network.target

Table 5-2: systemd Unit and Install section options (common to all units, systemd.unit)

The Requires option sets up a dependency between units. This is a strong dependency. If one fails so does the other. In the following Requires option from the graphical.target unit file, the graphical target can only be started if the multi-user and rescue targets are activated.

After=multi-user.target display-manager.service rescue.service rescue.target

The Wants option sets up a weaker dependency, requiring activation, but not triggering failure should it occur. This is the case with the graphical target and the display manager service like GDM.

Wants=display-manager.service

Several condition options are available such as ConditionACPower which, if true, checks to see if a system is using AC power. In the following example, ConditionPathExists checks for the existence of a file with runtime options for the dhcp server in the /etc/default directory.

ConditionPathExists=/etc/default/isc-dhcp-server

Some unit files are automatically generated by systemd, allowing you to use older configuration methods. For example, the unit file used to manage the mounting of your file systems is generated from the configuration information in the /etc/fstab file. The SourcePath option specifies the configuration file used to generate the unit file. The SourcePath option for the boot.mount unit file is shown here.

SourcePath=/etc/fstab

The Install section provides installation information for the unit. The WantedBy and RequiredBy options specify units that this unit wants or requires. For service units managing servers like Vsftpd and the DCHP servers, the install section has a WantedBy option for the multi-user.target. This has the effect of running the server at runlevels 2, 3, 4, and 5. When the multi-user target becomes activated so does that unit.

WantedBy=multi-user.target

The WantedBy option is implemented by setting up a link to the unit in a wants subdirectory for the wanted by unit. For the multi-user.target unit, a subdirectory called multi-user.target.wants has symbolic links in to all the units that want it, such as vsftpd.service for the vsfptd FTP service. These wants symbolic links are set up in the /etc/systemd/system directory, which can be changed as you enable and disable a service. Disabling a service removes the link. RequiredBy is a much stronger dependency.

The Alias option lists other unit names that could reference this unit. In the ssh.service file you will find an Alias option for sshd.service.

Alias=sshd.service

The Also option lists other unit that should be activated when this unit is started. The CUPS service has an Also option to start the CUPS socket and path.

Also=cups.socket cups.path

Different types of units have their own options. Service, socket, target, and path units all have options appropriate for their tasks.

special targets

A target file groups units for services, mounts, sockets, and devices. systemd has a set of special target files designed for specific purposes (see Table 5-3 ). Some are used for on demand services such as bluetooth and printer. When a bluetooth device is connected the bluetooth target becomes active. When you connect a printer, the printer target is activated which, in turn, activates the CUPS print server. The sound target is activated when the system starts and runs all sound-related units. See the special.target Man page for more details.

There are several special target files that are designed to fulfill the function of runlevels in System V (see Table 5-4 ). These include the rescue, multi-user, and graphical targets. On boot, systemd activates the default target, which is a link to a special target, such as multi-user.targetand graphical.target. You can override the default target with a systemd.unit kernel command line option in the GRUB.

The following will start up the rescue target.

systemd.unit=rescue.target

On the GRUB startup menu, you could edit the kernel boot line and add the following option to boot to the command line instead of the desktop.

systemd.unit=multi-user.target

The following will start up the rescue target.

systemd.unit=rescue.target

Table 5-3: special units

Table 5-4: special runlevel targets (boot)

On the GRUB startup menu, you could edit the kernel boot line and add the following option to boot to the command line instead of the desktop.

systemd.unit=multi-user.target

You could also simply add a 3 as in previous releases, as runlevel links also reference the special targets in systemd. The 3 would reference the runlevel 3 target, which links to the multi-user target. A copy of the multi-user.target file follows. The multi-user target requires the basic target which load the basic system (Requires). It conflicts with the rescue target (Conflicts), and it is run after the basic target. It can be isolated allowing you to switch special targets (AllowIsolate).

multi-user.target

[Unit]
Description=Multi-User System
Documentation=man:systemd.special(7)
Requires=basic.target
Conflicts=rescue.service rescue.target
After=basic.target rescue.service rescue.target
AllowIsolate=yes

The graphical.target depends on the multi-user.target. A copy of the graphical.target unit file follows. It requires that the multi-user.target be activated (Requires). Anything run for the multi-user target, including servers, is also run for the graphical target, the desktop. The desktop target is run after the multi-user.target (After). It is not run for the rescue.target (Conflicts). It also wants the display-manager service to run (GDM or KDM) (Wants). You can isolate it to switch to another target (AllowIsolate). .

graphical.target

[Unit]
Description=Graphical Interface
Documentation=man:systemd.special(7)
Requires=multi-user.target
Wants=display-manager.service
Conflicts=rescue.service rescue.target
After=multi-user.target rescue.service rescue.target display-manager.service
AllowIsolate=yes

Modifying unit files: /etc/systemd/system

systemd uses unit files to manage devices, mounts, and services. These are located in the /lib/systemd/system directory and are considered system files that you should not modify. Instead, to modify a unit file, you should copy it to the /etc/systemd/system directory. Unit files in this directory take precedence over those in the /lib/systemd/system directory. You can then modify the unit file version in /etc/systemd/system. Use a cp command to copy the file. The following command copies the Samba service unit file.

cp /lib/systemd/system/vsftpd.service /etc/systemd/system/vsftpd.service

If you just want to add unit options to a unit file, not changing the original options in the /lib/systemd/system version, you do not have to copy the original unit file. Instead, you can set up a corresponding new unit file in /etc/systemd/system that has an include options that reads in the original unit file from /lib/systemd/system. Then, in the new /etc/systemd/system file, you can add the new systemd options.

Keep in mind that most runtime options for a service application, such as the Vsftpd server, are still held in the appropriate /etc/default file, such as /etc/default/vsftpd. This file is read by the service when it is activated.

The actual enabling or disabling of services such as vsftpd, is handled through symbolic links set up or removed from the multi-user.target.wants directory in the /etc/systemd/system directory. This is considered a user based modification appropriate for /etc/systemd/system.

Execution Environment Options

The unit files of type service, sockets, mount, and swap share the same options for the execution environment of the unit (see Table 5-5 ). These are found in the unit section for that type such as [Service] for service units or [Socket] for socket unit. With these options you can set features such as the working directory (WorkingDirectory), the file mode creation mask (UMask), and the system logging level (SysLogLevel). Nice sets the default scheduling priority level. User specifies the user id for the processes the unit runs.

User=mysql

Table 5-5: systemd exec options (Service, Socket, Mount, Swap) (systemd.exec)

service unit files

A service unit file is used to run applications and commands such as the Samba (smb) and Web (httpd) servers. They have a [Service] section with options specified in the systemd.service Man page. See Table 5-6 for a listing of several common service options. A service unit file has the extension .service and the prefix is the name of the server program, such as isc-dhcp-server.service for the DHCP server and vsftpd.service for the Very Secure FTP server. Table 5-7 lists several popular servers.

A copy of the Vsftpd service unit file, vsftpd.service, follows. The Vsfptd FTP service is started after the network has started. The server program to run is specified, /usr/sbin/vsftpd (ExecStart). The service is installed by the multi-user.target (WantedBy), when the system starts up.

vsftpd.service

[Unit]
Description=vsftpd FTP server
After=network.target

[Service]
Type=simple
ExecStart=/usr/sbin/vsftpd /etc/vsftpd.conf
ExecReload=/bin/kill -HUP $MAINPID
ExecStartPre=-/bin/mkdir -p /var/run/vsftpd/empty

[Install]
WantedBy=multi-user.target

Table 5-6: systemd service options [Service] (systemd.service)

The bind9.service file is even simpler, incorporating configuration references into the program command. It is run after the network service, and is started by the multi-user.target.

bind9.service

[Unit]
Description=BIND Domain Name Server
Documentation=man:named(8)
After=network.target
[Service]
ExecStart=/usr/sbin/named -f -u bind
ExecReload=/usr/sbin/rndc reload
ExecStop=/usr/sbin/rndc stop
[Install]
WantedBy=multi-user.target

Table 5-7: Collection of Service unit files in /lib/systemd/system

System V Scripts and generated systemd service files: /etc/init.d and /run/systemd/generator.late

Some services are not yet configured natively for use by systemd. They are installed without a systemd service file. Instead a SysV script is installed in the /etc/init.d directory. The systemd-sysv-generator tool automatically reads this script and generates a corresponding systemd unit file in the /var/run/systemd/generator.late directory, which, in turn, will use the script to manage the service. For Ubuntu 15.04, the Apache Web server is such a service, with an apache2 script in /etc/init.d and a corresponding systemd service file in the /var/run/systemd/generator.latedirectory. A copy of the apache2.service unit service file follows. It is started before the runlevel targets, which are links to the multi-user and graphical targets (Before), which is the equivalent of runlevels 2,3,4, and 5. It is run after network, remote file system mounts, and name service lookup. The server program is run using the /etc/init.d/apache2 script (ExecStart). The same script is used to reload and stop the server but with different options (ExecReload and ExecStop).

There are .target.wants directories for different runlevels in the /run/systemd/generator.late directory, which hold links for the services active for a given runlevel. These runlevel target wants directories reference the runlevel targets, which, in turn, are simply links to the systemd multi-user and graphical targets.

apache2.service

# Automatically generated by systemd-sysv-generator

[Unit]
Documentation=man:systemd-sysv-generator(8)
SourcePath=/etc/init.d/apache2
Description=LSB: Apache2 web server
Before=runlevel2.target runlevel3.target runlevel4.target runlevel5.target shutdown.target
After=local-fs.target remote-fs.target network-online.target systemd-journald-dev-log.socket nss-lookup.target
Wants=network-online.target
Conflicts=shutdown.target

[Service]
Type=forking
Restart=no
TimeoutSec=5min
IgnoreSIGPIPE=no
KillMode=process
GuessMainPID=no
RemainAfterExit=yes
ExecStart=/etc/init.d/apache2 start
ExecStop=/etc/init.d/apache2 stop
ExecReload=/etc/init.d/apache2 reload

On Demand and Standalone Services (socket)

The On Demand activation of services, formerly implemented by inetd, is the default in systemd. Should you want a standalone service, you can specify that it is wanted by a special target so that it will be started up at boot time, instead of when it is first activated. In the Install section of a service unit file, a WantedBy option specifying the multi-user target will start the service at boot, making it a standalone service. In the following, the service is wanted by the multi-user.target. To put it another way, the service starts at runlevels 2, 3, 4, and 5. Note that the graphical target (5) is dependent on (Requires) the multi-user target (2, 3, and 4), so by specifying the multi-user target, the service is also started with the graphical target.

[Install]
WantedBy=multi-user.target

The Bluetooth service only wants the bluetooth.target which is only activated if a bluetooth device is present. It is not started at boot.

[Install]
WantedBy=bluetooth.target

Should you want the service started at all runlevels.

[Install]
WantedBy=basic.target

To emulate an on-demand server service, as inetd used to do, you would use a .socket file to compliment a .service file. This is the case with CUPS, which has a cups service file and corresponding cups socket file. The WantedBy option for sockets.target ties the socket to the special target sockets.target, which makes the unit socket-activated.

The Socket section lists options for the socket, usually what socket to listen on (ListenStream). The systemd.socket Man page lists socket options. Table 5-8 lists common options.

Table 5-8: systemd socket file options [Socket] (systemd.socket)

cups.socket

[Unit]
Description=CUPS Scheduler

[Socket]
ListenStream=/var/run/cups/cups.sock

[Install]
WantedBy=sockets.target

cups.service

[Unit]
Description=CUPS Scheduler
Documentation=man:cupsd(8)

[Service]
ExecStart=/usr/sbin/cupsd -l
Type=simple

[Install]
Also=cups.socket cups.path
WantedBy=printer.target

Path units

systemd uses path units to monitor a path. Sometimes a service unit has a corresponding path unit to monitor directories, as is the case with cups.path and cups.service. Options for the Path section are listed in Table 5-9 and on the systemd.path Man page. The cups.path unit file is shown here. The PathExists option checks if the printer spool files exist.

cups.path

[Unit]
Description=CUPS Scheduler

[Path]
PathExists=/var/cache/cups/org.cups.cupsd

[Install]
WantedBy=multi-user.target

Table 5-9: path option (systemd.path)

Template unit files

There is a special type of unit file called a template file, which allow for the generation of several unit files at runtime using one template file. Templates are used for services that generate instances of a service such as a getty terminal, an OpenVPN connection, and an rsync connection. A template file name ends with an @ sign. If a corresponding unit file is not found for a service, systemd will check to see if there is template file that can be applied to it. systemd matches the service name with the template name. It then generates an instance unit file for that particular service.

For example, a terminal uses the getty service (get TTY). As you do not know how many terminals you may use, they are generated automatically using the getty@.service unit file.

In the configuration file, the %I specifier is used to substitute for the service name. Given the service name getty@tty3, the %I specifier substitutes for tty3.

ExecStart=-/sbin/agetty --noclear %I $TERM

The getty@.service template file is shown here.

getty@.service

[Unit]
Description=Getty on %I
Documentation=man:agetty(8) man:systemd-getty-generator(8)
Documentation=http://0pointer.de/blog/projects/serial-console.html
After=systemd-user-sessions.service plymouth-quit-wait.service
After=rc-local.service

# If additional gettys are spawned during boot then we should make
# sure that this is synchronized before getty.target, even though
# getty.target didn't actually pull it in.
Before=getty.target
IgnoreOnIsolate=yes

# On systems without virtual consoles, don't start any getty. Note
# that serial gettys are covered by serial-getty@.service, not this
# unit.
ConditionPathExists=/dev/tty0

[Service]
# the VT is cleared by TTYVTDisallocate
ExecStart=-/sbin/agetty --noclear %I $TERM
Type=idle
Restart=always
RestartSec=0
UtmpIdentifier=%I
TTYPath=/dev/%I
TTYReset=yes
TTYVHangup=yes
TTYVTDisallocate=yes
KillMode=process
IgnoreSIGPIPE=no
SendSIGHUP=yes

# Unset locale for the console getty since the console has problems
# displaying some internationalized messages.
Environment=LANG= LANGUAGE= LC_CTYPE= LC_NUMERIC= LC_TIME= LC_COLLATE= LC_MONETARY= LC_MESSAGES= LC_PAPER= LC_NAME= LC_ADDRESS= LC_TELEPHONE= LC_MEASUREMENT= LC_IDENTIFICATION=

[Install]
WantedBy=getty.target
DefaultInstance=tty1

Runlevels and Special Targets

Under the old System V, a Linux system could run in different levels, called runlevels, depending on the capabilities you want to give it. Under System V, Linux had several runlevels, numbered from 0 to 6. When you power up your system, you enter the default runlevel. Runlevels 0, 1, and 6 are special runlevels that perform specific functions. Runlevel 0 was the power-down state. Runlevel 6 was the reboot state—it shuts down the system and reboots. Runlevel 1 was the single-user state, which allowed access only to the superuser and does not run any network services.

systemd uses special targets instead of runlevels create the same effect as runlevels, grouping services to run for specified targets. Runlevels are no longer directly implemented. There are two major special targets: multi-user and graphical. The multi-user target is similar to runlevel 3, providing you with a command line login. The graphical target is similar to runlevel 5, providing you with a graphical login and interface.

You set the default target (runlevel) by linking a target's systemd service file to the systemd default target file. This operation replaces the way inittab was used to specify a default runlevel in previous releases. The inittab file is no longer used. The following makes the graphical interface the default (runlevel 5).

ln -s /lib/systemd/system/graphicl.target /etc/systemd/system/default.target

systemd does provide compatibility support for runlevels. Runlevel compatibility is implemented using symbolic links in /lib/system/systemd directory to systemd targets. The runlevel0.target link references the systemd poweroff.target. Runlevel 2, 3, and 4 targets all link to the same multi-user.target (command line interface). The runlevel6.target links to the reboot target, and runlevel5.target links to graphical.target (desktop interface). The runlevels and their targets are listed in Table 5-10 .

Table 5-10: System Runlevels (States)

You can still use the runlevel command to see what state you are currently running in. It lists the previous state followed by the current one. If you have not changed states, the previous state will be listed as N, indicating no previous state. This is the case for the state you boot up in. In the next example, the system is running in state 3, with no previous state change:

# runlevel
N 3

Changing runlevels can be helpful if you have problems at a particular runlevel. For example, if your video card is not installed properly, then any attempt to start up in runlevel 5 (graphical.target) will likely fail, as this level immediately starts your graphical interface. Instead, you should use the command line interface, runlevel 3 (multi-user.target), to fix your video card installation.

No matter what runlevel you start in, you can change from one runlevel to another with the telinit command. If your default runlevel is 3, you power up in runlevel 3, but you can change to, say, runlevel 5 with telinit5. The command telinit0 shuts down your system. In the next example, the telinit command changes to runlevel 1, the administrative state:

telinit 1

Before systemd was implemented, you could also use init to also change runlevels. With systemd, both telinit and init are now systemd emulation versions of the original Unix commands. The telinit command is always used to change runlevels. If you use init with a runlevel number, it now merely invokes telinit to make the change.

Alternatively you can use the systemctl command directly to change runlevels (targets). The systemctl command with the isolate option and the name of the target file changes to that target (runlevel). The following command changes to the multi-user target.

sudo systemctl isolate multi-user.target

You could also use the runlevel link instead.

sudo systemctl isolate runlevel3.target

This is what the telinit command actually does.

systemd and automatically mounting file systems: /etc/fstab

The systemd unit files with the extension .mount can be used to mount file systems automatically. Normally systemd will read the /etc/fstab for mount information. If a mount unit file exists in the /etc/systemd directory, it takes precedence, but /etc/fstab takes precedence over any unit mount files in the /lib/systemd directory. The /etc/fstab file is used for mount configuration information. Most of the options for a mount unit file correspond to those of the /etc/fstab file, specifying the device path name, the mount point, file system type, and mount options (see Table 5-11). In fact, the entries in the /etc/fstab file are converted to mount unit files at boot, which are then used by systemd to perform the actual mount operations. These mount unit files are created by the systemd-fstab-generator and can be found in the /run/systemd/generator directory.

The following fstab file entries have corresponding mount files created in the /run/systemd/generator directory: boot.mount for the boot file system (boot-efi.mount for a EUFI boot system), home.mount for the home file system, and -.mount for the root file system. For the swap file system a swap unit file is generated.

UUID=1059a-4a86-4072-982e-000717229b9f / ext4 errors=remount-ro 0 1
UUID-=5537-AF41 /boot/efi vfat umask=0077 0 1
UUID=147b-4a86-4072-982e-000717229b6g /home ext4 default 0 1
UUID=cba958e4-4a86-4072-982e-000717228355 none swap sw 0 0

For this example, the -.mount file used for the root file system will have the following mount options. The root directory is represented in the mount file name as a dash, -, instead of a slash, /. The mount options are listed in a [Mount] section.

[Mount]
What=/dev/disk/by-uuid/1059a-4a86-4072-982e-000717229b9f
Where=/
Type=ext4
Options=errors=remount-ro

The home.mount file references partition for the home file system and mounts it to the /home directory.

[Mount]
What=/dev/disk/by-uuid/147b-4a86-4072-982e-000717229b6g
Where=/home
Type=ext4
FsckPassNo=2

The boot.mount file mounts the ext4 file system that holds the kernel in the /boot directory.

[Mount]
What=/dev/disk/by-uuid/e759aa59-4a86-4072-982e-000717229b4a
Where=/boot
Type=ext4
FsckPassNo=2

The boot-efi.mount file mounts the vfat efi file system that holds the boot information.

[Mount]
What=/dev/disk/by-uuid/5537-AF41
Where=/boot/efi
Type=vfat
Options=umask=0077

Table 5-11: systemd mount and automount file options [Mount] [Automount]

All the unit files will designate the /etc/fstab file as the SourcePath, the file from which the configuration was generated from.

SourcePath=/etc/fstab

All are mounted before any local file systems.

Before=local-fs.target

Local and remote file systems are distinguished by Wants options in their unit files for local-fs.target or remote-fs.target.

A mount unit file has to be named for the mount point it references. The path name slashes are replaced by dashes in the unit name. For example, the proc-fs-nfsd.mount file references the mount point /proc/fs/nfsd. The root path name, /, becomes simple a dash, -.

For file systems to be automatically mounted when accessed you can use the automount unit type. An automount unit must have a corresponding mount unit of the same name.

The systemd-fsck@service file provides a file system check with fsck, using the disk name as an argument.

RequiresOverridable=system-fsck@dev-disk-by\x2duuid-5537\x2dAF41.service
After=system-fsck@dev-disk-by\x2duuid-5537\x2dAF41.service

systemd slice and scope units

The slice and scope units are designed to group units to more easily control their processes and resources. The scope units are generated by systemd to manage a process and its sub processes. An example of a scope unit is a user session scope that groups the processes for a user session together. A slice is used to manage resources for processes, such as the machine slice for virtual machines, the system slice for system services, and the user slice for usr sessions.

System V: /etc/init.d

The SysVinit support for services is no longer implemented. There are no rc.d scripts for starting services. systemd manages all services directly. Check the README file in the /etc/init.d directory. For a very few system tasks, you may find System V scripts in the /etc/init.d directory. systemd will read these scripts as configuration information for a service, generating a corresponding unit configuration file for it. The unit file, in turn, may use the init.d script to start, stop, and restart the service. Should there be a unit file already in existence, that unit file is used and the System V script is ignored.

For Ubuntu 15.04, some servers use unit files generated from System V scripts in the /etc/init.d directory. These include Apache (apache2), Samba (smbd and nmbd), Postfix (postfix), NIS (nis), and Squid (squid3). The unit files are generated by systemd-sysv-generator and located in the /var/run/systemd/generator.late directory.

An rc-local.service unit file in the /lib/systemd/system directory will run a /etc/rc.local file, if present. This is to maintain compatibility with older System V configuration.

Shutdown and Poweroff

You can use the shutdown and poweroff commands to power down the system. The shutdown command provides more options. Keep in mind that the shutdown command used is the systemd version, which will use poweroff.target to actually shutdown the system. The poweroffcommand with the -f option forces a shutdown, with the --reboot option, it reboots the system.

poweroff

The shutdown command has a time argument that gives users on the system a warning before you power down. You can specify an exact time to shut down, or a period of minutes from the current time. The exact time is specified by hh:mm for the hour and minutes. The period of time is indicated by a + and the number of minutes.

The shutdown command takes several options with which you can specify how you want your system shut down. The -h option, which stands for halt, simply shuts down the system, whereas the -r option shuts down the system and then reboots it. In the next example, the system is shut down after ten minutes:

shutdown -h +10

To shut down the system immediately, you can use +0 or the word now. The shutdown options are listed in Table 5-12 . The following example shuts down the system immediately and then reboots:

shutdown -r now

With the shutdown command, you can include a warning message to be sent to all users currently logged in, giving them time to finish what they are doing before you shut them down.

shutdown -h +5 "System needs a rest"

If you do not specify either the -h or the -r options, the shutdown command shuts down the multi-user mode and shifts you to an administrative single-user mode. In effect, your system state changes from 3 (multi-user state) to 1 (administrative single-user state). Only the root user is active, allowing the root user to perform any necessary system administrative operations with which other users might interfere.

Note: the halt command now merely halts the system, it does not turn it off.

Table 5-12: System Shutdown Options

Managing Services

You can select certain services to run and the special target (runlevel) at which to run them. Most services are servers like a Web server or FTP server. Other services provide security, such as SSH or Kerberos. You can decide which services to use with the systemctl or service tools.

Enabling services: starting a service automatically at boot

Services such as the Apache Web server, Samba server, and the FTP server are now handled by the systemd daemon. You can manage services using the systemctl command. The older service command is simply a front end to the systemctl command.

To have a service start up at boot, you need to first enable it using the systemctl tool as the root user. Use the enable command to enable the service. The following command enables the vsftpd server and the Samba server (smb). The systemctl command uses the service's service configuration file located at the /lib/systemd/system or /etc/systemd/system directory.

sudo systemctl enable vsftpd.service
sudo systemctl enable smb

Managing services manually

Use the start, stop, and restart command with systemctl to manually start, stop, and restart a service. The enable command only starts up a service automatically. You could choose to start it manually using the start command. You can stop and restart a service any time using the stopand restart commands. The condrestart command only starts the server if it is already stopped. Use the status command to check the current status of service.

sudo systemctl start vsftpd
sudo systemctl restart vsftpd
sudo systemctl condrestart vsftpd
sudo systemctl stop vsftpd
sudo systemctl status vsftpd

You can also use service to start, stop, or restart a service. It is simply a front end for the systemctl command which performs the actual operation using systemd.

service start vsftpd

The service Command

The service command is now simply a front end for the systemctl command which performs the actual operation using systemd. The service command cannot enable or disable services. It only performs management operations such as stop, restart, and status. To start and stop services manually, you can use the service command. With the service command, you enter the service name with the stop argument to stop it, the start argument to start it, and the restart argument to restart it. The service command is run from a Terminal window. You will have to first login as the rootuser, using the su command, or use the sudo command, if configured. The following will start the vsftod FTP service.

sudo service vsftpd start

The systemd version of the service command actually invokes the systemctl command to run the service's systemd.service unit file in /lib/systemd/system. If a service is not enabled, systemd will enable it. You can perform the same operations as the service command, using thesystemctl command. The following is the equivalent of the previous command.

sudo systemctl vsftpd vsftpd

Note: Upstart had been deprecated.

/etc/default

The /etc/default directory holds scripts for setting runtime options when a service starts up. For example, the /etc/default/apache2 script sets cache cleaning options. The /etc/default/ufw scripts sets firewall default policies. You can edit these scripts and change the values assigned to options, changing the behavior of a service.

Service Startup Management with rcconf

Ubuntu provides the rcconf (Debian) toos which you can use to start or stop services when you boot up your system (see Figure 5-1 ). The rcconf tool was developed by Debian and is used on Debian, Ubuntu, and similar distributions. The rcconf works both for System V init service scripts located in the /etc/init.d directory and for systemd service files, changing appropriate links the the /etc/systemd/multi-user.target.wants directory.

Figure 5-1: rcconf service management

The rcconf is run from a terminal window or from the command line, and provides an easy cursor-based interface for using arrow keys and the spacebar to turn services on or off.

sudo rcconf

Network Time Protocol, NTP

For servers to run correctly, they need to always have the correct time. Internet time servers worldwide provide the time in the form of the Universal Time Coordinated (UTC). Local time is then calculated using the local system's local time zone. The time is obtained from Internet time servers from an Internet connection. You have the option of using a local hardware clock instead, though this may be much less accurate.

Normally, the time on a host machine is kept in a Time of Year chip (TOY) that maintains the time when the machine is off. Its time is used when the machine is rebooted. A host using the Network Time Protocol then adjusts the time, using the time obtained from an Internet time server. If there is a discrepancy of more than 1000 seconds (about 15 minutes), the system administrator is required to manually set the time. Time servers in the public network are organized in stratum levels, the highest being 1. Time servers from a lower stratum obtain the time from those in the next higher level.

For servers on your local network, you may want to set up your own time server, insuring that all your servers are using a synchronized time. If all your servers are running on a single host system that is directly connected to the Internet and accessing an Internet time server, you will not need to set up a separate time server. You can use the ntpdate command to update directly from an Internet time server.

sudo ntpdate ntp.ubuntu.com

If the servers are on different host systems, then you may want a time server to insure their times are synchronized. Alternatively, you could just use the ntpdate command to update those hosts directly at given intervals. You could set up a cron job to perform the ntpdate operation automatically.

There are packages on the Ubuntu repository for both the NTP server and its documentation. You can install them with apt-get, aptitude, or (from the desktop) the Synaptic Package Manager.

ntp
ntp-doc

The documentation will be located in the /usr/share/doc/ntp-doc directory in Web page format.

/usr/share/doc/ntp-doc/html/index.html

The ntp server

The NTP server name is ntpd and is managed by the /etc/init.d/ntp script. Use the start, stop, and restart options to manage the server. A corresponding systemd service file, ntp.service, is generated in the /run/systemd/generator.late directory. s

sudo service ntp start

Your host systems can then be configured to use NTP and access your NTP time server.

To check the status of your time server, you can use the ntpq command. With the -p option is displays the current status.

ntpq -p

The ntp.conf configuration file

The NTP server configuration file is /etc/ntp.conf. This file lists the Internet time servers that your own time server used to deterring the time. Check the ntp.conf Man page for a complete listing of the NTP server configuration directives.

In the ntp.conf file, the server directive specifies the Internet time server's Internet address that your NTP server uses to access the time. There is a default entry for the Ubuntu time server, but you can add more server entries for other time servers.

server ntp.ubuntu.com

NTP access controls

Access control to the NTP server is determined by the restrict directives. An NTP server is accessible from the Internet, anyone can access it. You can specify access options and the addresses of hosts allowed access. The default option lets you specify the set of default options. Thenoquery, notrust, nopeer, and nomodify option deny all access. The notrust option will not trust hosts unless specifically allowed access. The nomodify option prevents any modification of the time server. The noquery option will not even allow queries from other hosts, unless specifically allowed.

restrict -4 default kod notrap nomodify nopeer noquery

The default /etc/ntp.conf file is shown here.

#/etc/ntp.conf, configuration for ntpd: see ntp.conf(5) for help
driftfile /ver/lib/ntp/ntp.drift

#Enable this if you want statistics to be logged.
#statsdir /var/log/ntpstats/
statistics loopstats peerstats clockstats
filegen loopstats file loopstats type day enable
filegen peerstats file peerstats type day enable
filegen clockstats file clockstats type day enable
# Use Servers from the NTP Pool Project. Approved by Ubuntu Technical Board
# on 2011-02-08 (LP: #104525). See http://www.pool.ntp.org/join.html for
# more information
server 0.ubuntu.pool.ntp.org
server 1.ubuntu.pool.ntp.org
server 2.ubuntu.pool.ntp.org
server 3.ubuntu.pool.ntp.org

# Use Ubuntu's ntp server as fallback
server ntp.ubuntu.com

# Access control configuration; see
# /usr/share/doc/ntp-doc/html/accopt.html for
# details. The web page <http://support.ntp.org/bin/view/Support/AccessRestrictions>
# might also be helpful.
#
# Note that "restrict" applies to both servers and clients, so a configuration
# that might be intended to block requests from certain clients could also end
# up blocking replies from your own upstream servers.

# By default, exchange time with everybody, but don't allow configuration.
restrict -4 default kod notrap nomodify nopeer noquery
restrict -6 default kod notrap nomodify nopeer noquery

# Local users may interrogate the ntp server more closely.
restrict 127.0.0.1
restrict ::1

# Clients from this (example!) subnet have unlimited access, but only if
# cryptographically authenticated.
#restrict 192.168.123.0 mask 255.255.255.0 notrust

# If you want to provide time to your local subnet, change the next
# line. (Again, the address is an example only.)
#broadcast 192.168.123.255

# If you want to listen to time broadcasts on your local subnet,
# de-comment the next lines. Please do this only if you trust everybody
# on the network!
#disable auth
#broadcastclient

Then local users, users on the same host that is running the NTP server, are allowed to access the NTP server. Addresses are specified for both IPv4 and IPv6 local host, 127.0.0.1 and ::1.

restrict 127.0.0.1
restrict ::1

To allow access from hosts on a private local network, you can use the restrict directive to specify the local network address and mask. The following allows access to a local network, 192.168.123, with a network mask of 255.255.255.0 to determine the range of allowable host addresses.

restrict 192.168.123.0 mask 255.255.255.0

If you want to require the use of encrypted keys for access, add the notrust option. Use ntp-keygen to generate the required public/private keys.

You can also run the time server in broadcast mode where the time is broadcasted to your network clients (this can involve security risks). Use the broadcast directive and your network's broadcast address. Your host systems need to have the broadcastclient setting set, which will listen for time broadcasts.

broadcast 192.168.123.255

NTP clock support

You can also list a reference to the local hardware clock, and have that clock be used if your connection to the Internet time server should fail. The hardware clock is references by the IP address that has the prefix 127.127 followed by the clock type and instance, as in 127.127.1.1. The type for the local clock is 1.

server 127.127.1.1

The fudge directive is used to specify the time for a hardware clock, passing time parameters for that clock's driver.

AppArmor security

Ubuntu installs AppArmor as its default security system. AppArmor (Application Armor) is designed as an alternative to SELinux (Security-Enhanced Linux, http://www.nsa.gov/research/selinux/ and http://selinuxproject.org/page/Main_Page). It is much less complicated, but makes use of the same kernel support provided for SELinux. AppArmor is a simple method for implementing mandatory access controls (MAC) for specified Linux applications. It is used primarily for servers like Samba, the CUPS print servers, and the time server. In this respect it is much more limited in scope than SELinux, which tries to cover every object. Instead of labeling each object, which SELinux does, AppArmor identifies an object by its path name. The object does not have to be touched. Originally developed by Immunix and later supported for a time by Novell (OpenSUSE), AppArmor is available under the GNU Public License. You can find out more about AppArmor at http://en.opensuse.org/Apparmor.

AppArmor works by setting up a profile for supported applications. Essentially, this is a security policy similar to SELinux policies. A profile defines what an application can access and use on the system. Ubuntu will install the apparmor and apparmor-utils packages (Ubuntu main repository). Also available are the apparmor-profiles (Universe repository) and apparmor-doc packages.

AppArmor is still installed with the /etc/init.d/apparmor script. You can use the service or systemctl commands to start, stop, and restart AppArmor.

sudo service apparmor start

Currently, the generated systemd service file, apparmor.service (/run/systemd/generator.late), uses the /etc/init.d/apparmor script to manage Apparmor.

AppArmor utilities

The appArmor-utils packages installs several AppArmor tools, including enforce, which enables AppArmor and complain, which instructs AppArmor to just issue warning messages (see Table 5-13 ). The unconfined tool will list applications that have no AppArmor profiles. Theaudit tool will turn on AppArmor message logging for an application (uses enforce mode).

The apparmor_status tool will display current profile information. The --complaining options lists only those in complain mode, and --enforced for those in enforcing mode.

sudo apparmor_status

Table 5-13: AppArmor Utilities

The aa-logprof tool will analyze AppArmor logs to determine if any changes are needed in any of the application profiles. Suggested changes will be presented and the user can allow (A) or deny them (D). In complain mode, allow is the default, and in enforce mode, deny is the default. You can also make your own changes with the new (N) option. Should you want the change applied to all files and directories in a suggested path, you can select the glob option (G), essentially replacing the last directory or file in a path with the * global file matching symbol.

The aa-autodep tool will generate a basic AppArmor profile for a new or unconfined application. If you want a more effective profile, you can use aa-genprof to analyze the application's use and generate profile controls accordingly.

The aa-genprof tool will update or generate a detailed profile for a specified application. aa-genprof will first set the profile to complain mode. You then start up the application and use it, generating complain mode log messages on that use. Then, vgenprof prompts you to either scan the complain messages to further refine the profile (S), or to finish (F). When scanned, different violations are detected and the user is prompted to allow or deny recommended controls. You can then repeat the scan operation until you feel the profile is acceptable. Select finish (F) to finalize the profile and quit.

AppArmor configuration

AppArmor configuration is located in the /etc/apparmor directory. Configurations for different profiles are located in the /etc/apparmor.d directory. Loaded profile configuration file have the name of their path, using periods instead of slashes to separate directory names. The profile file for the smbd (Samba) application is usr.sbin.smbd. For CUPS (cupsd) it is usr.sbin.cupsd. For the time server it is user.sbin.ntpd. Additional profiles like the Samba and Apache profiles are installed with the apparmor-profiles package (not installed by default).

sudo apt-get install apparmor-profiles

Configuration rules for AppArmor profiles consist of a path and permissions allowable on that path. A detailed explanation of AppArmor rules and permissions can be found in the apparmor.d Man page, including a profile example. A path ending in a * matching symbol will select all the files in that directory. The ** symbol selects all files and subdirectories. All file matching operations are supported (*[]?). Permissions include r (read), w (write), x (execute), and l (link). The u permission allows unconstrained access. The following entry allows all the files and subdirectories in the /var/log/samba/cores/smdb directory to be written to.

/var/log/samba/cores/smbd/** rw,

The /etc/apparmor.d/abstractions directory has files with profile rules that are common to different profiles. Rules from these files are read into actual profiles using the include directive. There are abstractions for applications like audio, samba, and video. Some abstractions will include yet other more general abstractions, like those for the X server (X) or GNOME (gnome). For example, the profile for the Samba smbd server, usr.sbin.smbd, will have a include directive for the samba abstraction. This abstraction holds rules common to both the smbd and nmbdservers, both used by the Samba service. The <> used in an include directive indicates the /etc/apparmor.d directory. A list of abstraction files can be found in the apparmor.d Man page. The include directive begins with a # character.

#include <abstractions/samba>

In some cases, a profile may need access to some files in a directory that it normally should not have access to. In this case it may need to use a sub-profile to allow access. In effect, the application changes hats, taking on permissions it does not have in the original profile.

The armor-profiles package will activate several commonly used profiles, setting up profile files for them in the /etc/apparmor.d directory, like those for samba (usr.sbin.nmbd and usr.sbin.smbd), the Dovecot mail pop and imap server (usr.sbin.dovecot), and Avahi (usr.sbin.avahi-daemon).

The package also will provide profile default files for numerous applications in the /usr/share/doc/apparmor-profiles/extras directory, such as the vsftpd FTP server (usr.sbin.vsftpd), the ClamAV virus scanner (usr.bin.freshclam), and the Squid proxy server (usr.sbin.squid). Some service applications are located in the /usr/lib directory, and will have usr.lib prefix such as those for the Postfix server, which uses several profiles, beginning with usr.lib.postfix. To use these extra profiles, copy them to the /etc/apparmor.d directory. The following example copies the profile for the vsftpd FTP server.

sudo cp /usr/share/doc/apparmor-profiles/extras/usr.sbin.vsftpd /etc/apparmor.d

Remote Administration

For remote administration you can use the OpenSSH and Puppet. OpenSSH lets you remotely control and transfer files securely over your network. Puppet lets you manage remote configuration of services.

Puppet

Puppet allows you to configure remote systems automatically, even though they may be running different linux distributions with varying configuration files. Instead of configuring each system on a network manually, you can use Puppet to configure them automatically. Puppet abstracts administration tasks as resources in a resource abstraction layer (RAL). You then specify basic values or operations for a particular resource using a Puppet configuration language. Administration types include services, files, users, and groups. For example, you could use puppet to perform an update for a service (server) on systems using different package managers such as APT or Synaptic.

Puppet configuration can become very complex. Once set up though is fully automates configuration changes across all your networked systems. For detailed documentation and guides see the following.

http://docs.puppetlabs.com/

Puppet configuration is located in the /etc/puppet directory. Puppet operations on services are specified in modules programmed in the init.pp file located in a manifests directory. In the Ubuntu example for Apache (Ubuntu Server Guide, Puppet), the apache module is created in the init.pp file in:

/etc/puppet/modules/apache2/manifests/init.pp

On Ubuntu, clients use the puppet client (puppet package) and the server uses the puppetmaster daemon (puppetmaster package).

On puppet clients, use the service command to manually start and stop the puppet service. For the server, enable the puppetmaster service.

On the firewall add access for the Puppet port, 8140.

For the client, the puppet runtime configuration is set in the /etc/default/puppet file. Here you set the daemon to start by setting the START variable to yes

If your network is running a DNS server, you can set up a CNAME puppet entry for the puppet server. The puppet clients can then use the CNAME to locate the puppet server.

puppet IN CNAME turtle.mytrek.com

You could also add a host entry for the puppet server in each client's /etc/host file.

On the server, the puppetmaster configuration in /etc/default/puppetmaster file lets you set port entries and the log service. Default entries are commented out. Remove the comment character, #, to enable.

When you first set up a client server puppet connection, the client and server have to sign the client's SSL certificate. First run puppet on the client. On the server run the puppet cert --list command to see the clients certificate request. Then use puppet cert --sign to sign the certificate.

The Secure Shell: OpenSSH

Although a firewall can protect a network from attempts to break into it from the outside, the problem of securing legitimate communications to the network from outside sources still exists. A particular problem is one of users who want to connect to your network remotely. Such connections could be monitored, and information such as passwords and user IDs used when the user logs in to your network could be copied and used later to break in. One solution is to use SSH for remote logins and other kinds of remote connections such as FTP transfers. SSH encrypts any communications between the remote user and a system on your network.

The SSH protocol has become an official Internet Engineering Task Force (IETF) standard. A free and open source version is developed and maintained by the OpenSSH project, currently supported by the OpenBSD project. OpenSSH is the version supplied with most Linux distributions, including Ubuntu. You can find out more about OpenSSH at www.openssh.org, where you can download the most recent version, though your distribution will provide current RPM versions.

SSH secures connections by both authenticating users and encrypting their transmissions. The authentication process is handled with public key encryption. Once authenticated, transmissions are encrypted by a cipher agreed upon by the SSH server and client for use in a particular session. SSH supports multiple ciphers. Authentication is applied to both hosts and users. SSH first authenticates a particular host, verifying that it is a valid SSH host that can be securely communicated with. Then the user is authenticated, verifying that the user is who they say they are.

Encryption

The public key encryption used in SSH authentication makes use of two keys: a public key and a private key. The public key is used to encrypt data, while the private key decrypts it. Each host or user has its own public and private keys. The public key is distributed to other hosts, who can then use it to encrypt authenticated data that only the host’s private key can decrypt. For example, when a host sends data to a user on another system, the host encrypts the authentication data with a public key, which it previously received from that user. The data can be decrypted only by the user’s corresponding private key. The public key can safely be sent in the open from one host to another, allowing it to be installed safely on different hosts. You can think of the process as taking place between a client and a server. When the client sends data to the server, it first encrypts the data using the server’s public key. The server can then decrypt the data using its own private key.

It is recommended that SSH transmissions be authenticated with public-private keys controlled by passphrases. Unlike PGP, SSH uses public-key encryption for the authentication process only. Once authenticated, participants agree on a common cipher to use to encrypt transmissions. Authentication will verify the identity of the participants. Each user who intends to use SSH to access a remote account first needs to create the public and private keys along with a passphrase to use for the authentication process. A user then sends their public key to the remote account they want to access and installs the public key on that account. When the user attempts to access the remote account, that account can then use the user’s public key to authenticate that the user is who they claim to be. The process assumes that the remote account has set up its own SSH private and public key. For the user to access the remote account, they will have to know the remote account’s SSH passphrase. SSH is often used in situations where a user has two or more accounts located on different systems and wants to be able to securely access them from each other. In that case, the user already has access to each account and can install SSH on each, giving each its own private and public keys along with their passphrases.

Authentication

The mechanics of authentication in SSH version 1 and version 2 differ slightly. However, the procedure on the part of users is the same. Essentially, a user creates both public and private keys. For this you use the ssh-keygen command. The user’s public key then has to be distributed to those users that the original user wants access to. Often this is an account a user has on another host. A passphrase further protects access. The original user will need to know the other user’s passphrase to access it.

SSH version 1 uses RSA authentication. When a remote user tries to log in to an account, that account is checked to see if it has the remote user’s public key. That public key is then used to encrypt a challenge (usually a random number) that can be decrypted only by the remote user’s private key. When the remote user receives the encrypted challenge, that user decrypts the challenge with its private key. SSH version 2 can use either RSA or DSA authentication. The remote user will first encrypt a session identifier using its private key, signing it. The encrypted session identifier is then decrypted by the account using the remote user’s public key. The session identifier has been previously set up by SSH for that session.

SSH authentication is first carried out with the host, and then with users. Each host has its own host keys, public and private keys used for authentication. Once the host is authenticated, the user is queried. Each user has their own public and private keys. Users on an SSH server who want to receive connections from remote users will have to keep a list of those remote user’s public keys. Similarly, an SSH host will maintain a list of public keys for other SSH hosts.

SSH Packages, Tools, and Server

SSH is implemented on Linux systems with OpenSSH. The full set of OpenSSH packages includes the OpenSSH meta-package (ssh), the OpenSSH server (openssh-server), and the OpenSSH client (openssh-clients). These packages also require OpenSSL (openssl), which installs the cryptographic libraries that SSH uses.

The SSH tools are listed in Table 5-7 . They include several client programs such as scp, ssh, as well as the ssh server. The ssh server (sshd) provides secure connections to anyone from the outside using the ssh client to connect. Several configuration utilities are also included, such asssh-add, which adds valid hosts to the authentication agent, and ssh-keygen, which generates the keys used for encryption.

For version 2, names of the actual tools have a 2 suffix. Version 1 tools have a 1 as their suffix. During installation, however, links are set for each tool to use only the name with the suffix. For example, if you have installed version 2, there is a link called scp to the scp2 application. You can then use the link to invoke the tool. Using scp starts scp2. Table 5-14 specifies only the link names, as these are the same for each version. Some applications, such as sftp, are available only with version 2.

You can start, stop, and restart the server manually with the service or systemctl commands.

sudo service sshd restart

Table 5-14: SSH Tools

You have to configure your firewall to allow access to the ssh service. The services is set up to operate using the TCP protocol on port 22, tcp/22. If you are managing your IPTables firewall directly, you could manage access directly by adding the following IPtables rule. This accepts input on port 22 for TCP/IP protocol packages.

iptables -A INPUT -p tcp --dport 22 -j ACCEPT

SSH Setup

Using SSH involves creating your own public and private keys and then distributing your public key to other users you want to access. These can be different users or simply user accounts of your own that you have on remote systems. Often people remotely log in from a local client to an account on a remote server, perhaps from a home computer to a company computer. Your home computer would be your client account, and the account on your company computer would be your server account. On your client account, you need to generate your public and private keys and then place a copy of your public key in the server account. You can do this by simply e-mailing the key file or copying the file from a floppy disk. Once the account on your server has a copy of your client user’s public key, you can access the server account from your client account. You will be also prompted for the server account’s passphrase. You will have to know this to access that account. Figure 5-2 illustrates the SSH setup that allows a user george to access the account cecelia.

Figure 5-2: SSH setup and access

To allow you to use SSH to access other accounts:

You must create public and private keys on your account along with a passphrase. You will need to use this passphrase to access your account from another account.

You must distribute your public key to other accounts you want to access, placing them in the .ssh/authorized_keys file.

Other accounts also have to set up public and private keys along with a passphrase.

You must know the other account’s passphrase to access it.

Creating SSH Keys with ssh-keygen

You create your public and private keys using the ssh-keygen command. You need to specify the kind of encryption you want to use. You can use either DSA or RSA encryption. Specify the type using the -t option and the encryption name in lowercase (dsa or rsa). In the following example, the user creates a key with the RSA encryption:

ssh-keygen -t rsa

The ssh-keygen command prompts you for a passphrase, which it will use as a kind of password to protect your private key. The passphrase should be several words long. You are also prompted to enter a filename for the keys. If you do not enter one, SSH will use its defaults. The public key will be given the extension .pub. The ssh-keygen command generates the public key and places it in your .ssh/id_dsa.pub or .ssh/id_rsa.pub file, depending on the type of key you specified; it places the private key in the corresponding .ssh/id_dsa or .ssh/id_rsa file.

Table 5-15: SSH Configuration Files

If you need to change your passphrase, you can do so with the ssh-keygen command and the -p option. Each user will have their own SSH configuration directory, called .ssh, located in their own home directory. The public and private keys, as well as SSH configuration files, are placed here. If you build from the source code, the make install operation will automatically run ssh-keygen. Table 5-15 lists the SSH configuration files.

Authorized Keys

A public key is used to authenticate a user and its host. You use the public key on a remote system to allow that user access. The public key is placed in the remote user account’s .ssh/authorized_keys file. Recall that the public key is held in the .ssh/id_dsa.pub file. If a user wants to log in remotely from a local account to an account on a remote system, they would first place their public key in the .ssh/authorized_keys file in the account on the remote system they want to access. If the user larisa on turtle.mytrek.com wants to access the aleina account onrabbit.mytrek.com, larisa’s public key from /home/larisa/.ssh/id_dsa.pub first must be placed in aleina’s authorized_keys file, /home/aleina/.ssh/authorized_keys. User larisa can send the key or have it copied over. A simple cat operation can append a key to the authorized key file. In the next example, the user adds the public key for aleina in the larisa.pub file to the authorized key file. The larisa.pub file is a copy of the /home/larisa/.ssh/id_dsa.pub file that the user received earlier.

cat larisa.pub >> .ssh/authorized_keys

Note: You can also use seahorse to create and manage SSH keys.

Note: The .ssh/identity filename is used in SSH version 1; it may be installed by default on older distribution versions. SSH version 2 uses a different filename, .ssh/id_dsa or .ssh/id_rsa, depending on whether RSA or DSA authentication is used.

Loading Keys

If you regularly make connections to a variety of remote hosts, you can use the ssh-agent command to place private keys in memory where they can be accessed quickly to decrypt received transmissions. The ssh-agent command is intended for use at the beginning of a login session. For GNOME, you can use the openssh-askpass-gnome utility, invoked by ssh-add, which allows you to enter a password when you log in to GNOME. GNOME will automatically supply that password whenever you use an SSH client.

Although the ssh-agent command enables you to use private keys in memory, you also must specifically load your private keys into memory using the ssh-add command. ssh-add with no arguments loads your private key from your .ssh/id_dsa or .ssh/id_rsa.pub file. You are prompted for your passphrase for this private key. To remove the key from memory, use ssh-add with the -d option. If you have several private keys, you can load them all into memory. ssh-add with the -l option lists those currently loaded.

SSH Clients

SSH was originally designed to replace remote access operations, such as rlogin, rcp, and Telnet, which perform no encryption and introduce security risks. You can also use SSH to encode X server sessions as well as FTP transmissions (sftp). The ssh-clients package contains corresponding SSH clients to replace these applications. With slogin or ssh, you can log in from a remote host to execute commands and run applications, much as you can with rlogin and rsh. With scp, you can copy files between the remote host and a network host, just as with rcp. Withsftp, you can transfer FTP files secured by encryption.

ssh

With ssh you can remotely log in from a local client to a remote system on your network operating as the SSH server. The term local client here refers to one outside the network, such as your home computer, and the term remote refers to a host system on the network to which you are connecting. In effect, you connect from your local system to the remote network host. It is designed to replace rlogin, which performs remote logins, and rsh, which executes remote commands. With ssh, you can log in from a local site to a remote host on your network and then send commands to be executed on that host. The ssh command is also capable of supporting X Window System connections. This feature is automatically enabled if you make an ssh connection from an X Window System environment, such as GNOME or KDE. A connection is set up for you between the local X server and the remote X server. The remote host sets up a dummy X server and sends any X Window System data through it to your local system to be processed by your own local X server.

The ssh login operation function is much like the rlogin command. You enter the ssh command with the address of the remote host, followed by a -l option and the login name (username) of the remote account you are logging in to. The following example logs in to the aleina user account on the rabbit.mytrek.com host:

ssh rabbit.mytrek.com -l aleina

You can also use the username in an address format with ssh, as in

ssh aleian@rabbit.mytrek.com

The following listing shows how the user george accesses the cecelia account on turtle.mytrek.com:

[george@turtle george]$ ssh turtle.mytrek.com -l cecelia
cecelia@turtle.mytrek.com's password:
[cecelia@turtle cecelia]$

A variety of options are available to enable you to configure your connection. Most have corresponding configuration options that can be set in the configuration file. For example, with the -c option, you can designate which encryption method you want to use, for instance, idea, des,blowfish, or arcfour. With the -i option, you can select a particular private key to use. The -C option enables you to have transmissions compressed at specified levels (see the ssh Man page for a complete list of options).

scp

You use scp to copy files from one host to another on a network. Designed to replace rcp, scp uses ssh to transfer data and employs the same authentication and encryption methods. If authentication requires it, scp requests a password or passphrase. The scp program operates much like rcp. Directories and files on remote hosts are specified using the username and the host address before the filename or directory. The username specifies the remote user account that scp is accessing, and the host is the remote system where that account is located. You separate the user from the host address with an @, and you separate the host address from the file or directory name with a colon. The following example copies the file party from a user’s current directory to the user aleina’s birthday directory, located on the rabbit.mytrek.com host:

scp party aleina@rabbit.mytrek.com:/birthday/party

Of particular interest is the -r option (recursive) option, which enables you to copy whole directories. See the scp Man page for a complete list of options. In the next example, the user copies the entire reports directory to the user justin’s projects directory:

scp -r reports justin@rabbit.mytrek.com:/projects

In the next example, the user george copies the mydoc1 file from the user cecelia’s home directory:

[george@turtle george]$ scp cecelia@turtle.mytrek.com:mydoc1 .
cecelia@turtle.mytrek.com's password:
mydoc1 0% | | 0 --:--
ETA
mydoc1 100% |*****************************| 17 00:00
[george@turtle george]$

From a Windows system, you can also use scp clients such as winscp, which will interact with Linux scp-enabled systems.

sftp and sftp-server

With sftp, you can transfer FTP files secured by encryption. The sftp program uses the same commands as ftp. This client, which works only with ssh version 2, operates much like ftp, with many of the same commands. Use sftp instead of ftp to invoke the sftp client.

sftp download.ubuntu.com

To use the sftp client to connect to an FTP server, that server needs to be operating the sftp-server application. The ssh server invokes sftp-server to provide encrypted FTP transmissions to those using the sftp client. The sftp server and client use the SSH File Transfer Protocol (SFTP) to perform FTP operations securely.

Port Forwarding (Tunneling)

If, for some reason, you can connect to a secure host only by going through an insecure host, ssh provides a feature called port forwarding. With port forwarding, you can secure the insecure segment of your connection. This involves simply specifying the port at which the insecure host is to connect to the secure one. This sets up a direct connection between the local host and the remote host, through the intermediary insecure host. Encrypted data is passed through directly. This process is referred to as tunneling, creating a secure tunnel of encrypted data through connected servers.

You can set up port forwarding to a port on the remote system or to one on your local system. To forward a port on the remote system to a port on your local system, use ssh with the -R option, followed by an argument holding the local port, the remote host address, and the remote port to be forwarded, each separated from the next by a colon. This works by allocating a socket to listen to the port on the remote side. Whenever a connection is made to this port, the connection is forwarded over the secure channel, and a connection is made to a remote port from the local machine. In the following example, port 22 on the local system is connected to port 23 on the rabbit.mytrek.com remote system:

ssh -R 22:rabbit.mytrek.com:23

To forward a port on your local system to a port on a remote system, use the ssh -L option, followed by an argument holding the local port, the remote host address, and the remote port to be forwarded, each two arguments separated by a colon. A socket is allocated to listen to the port on the local side. Whenever a connection is made to this port, the connection is forwarded over the secure channel and a connection is made to the remote port on the remote machine. In the following example, port 22 on the local system is connected to port 23 on the rabbit.mytrek.com remote system:

ssh -L 22:rabbit.mytrek.com:23

You can use the LocalForward and RemoteForward options in your .ssh/ssh_config file to set up port forwarding for particular hosts or to specify a default for all hosts you connect to.

SSH Configuration

The SSH configuration file for each user is in their .ssh/ssh_config file. The /etc/ssh/ssh_config file is used to set site-wide defaults. In the configuration file, you can set various options, as listed in the ssh_config Man document. The configuration file is designed to specify options for different remote hosts to which you might connect. It is organized into segments, where each segment begins with the keyword HOST, followed by the IP address of the host. The following lines hold the options you have set for that host. A segment ends at the next HOST entry. Of particular interest are the User and Cipher options. Use the User option to specify the names of users on the remote system who are allowed access. With the Cipher option, you can select which encryption method to use for a particular host. Encryption methods include triple-DES (3DES) , Arcfour (RSA’s RC4), and Advanced Encryption Standard (AES). The following example allows access from larisa at turtle.mytrek.com and uses 3des encryption for transmissions:

Host turtle.mytrek.com
User larisa
Compression no
Cipher 3des

Most standard options, including ciphers are already listed as commented entries. Remove the # to activate.

# Ciphers aes128-ctr,aes192-ctr,aes256-ctr,arcfour256,arcfour128,aes128-cbc,3des-cbc

To specify global options that apply to any host you connect to, create a HOST entry with the asterisk as its host, HOST *. This entry must be placed at the end of the configuration file because an option is changed only the first time it is set. Any subsequent entries for an option are ignored. Because a host matches on both its own entry and the global one, its specific entry should come before the global entry. The asterisk (*) and the question mark (?) are both wildcard matching operators that enable you to specify a group of hosts with the same suffix or prefix.

Host *
PasswordAuthentication yes
ConnectTimeout 0
Cipher 3des

The protocol option lets you specify what version of SSH to use, 1, 2, or both. By default, both protocols are acceptable, for backward compatibility. You can restrict use to just the more advanced and secure SSH2 protocol by changing the Protocol option to just 2.

Protocol 2

You use the /etc/ssh/sshd_config file to configure an SSH server. Here you will find server options like the port to use, password requirement, and PAM usage.