CompTIA Linux+ / LPIC-1 Cert Guide (Exams LX0-103 & LX0-104/101-400 & 102-400) (2016)

The peers command gives details about each of the servers defined with the server keyword in the configuration. The association command gives more details on each server, including how well the remote server is performing.

In Example 18-5 there are four servers. The first column shows the name of the remote server but is prefixed by a special character. If that character is a *, that server is the currently active peer. If the character is a +, the server is a good time source but not the main server. Anything else and the server is not considered to be good for time synchronization and will be monitored.

The second column shows the name of the remote server from where your machine is getting its time. The third column shows which stratum that server is, and the remaining columns are statistics used by ntpd.

The associations command gives even greater detail about each server in the same order that they appear in the peers command. The most interesting column is the condition column, which explains the state of the server. The peers command showed that the second server was not being used because the name of the server was prefixed with a minus sign (-), and the condition of that server is outlyer. This means that the time on that server was too different from the others to be trusted.

Other than the condition, you can see that the servers are all reachable.

System Logging

Logs tell you what was happening while you weren’t looking. The kernel and running applications emit logs when events happen. These logs might be informational, such as a web request, or they might be errors such as the ntpd daemon exiting because the time difference was too large.

Most applications in Linux log in some form or another. Linux provides a centralized logging facility that applications can use. This means that an application can use a standard interface to log and doesn’t have to worry about configuring file locations. This also means that the administrator can split up logs according to function, or to ignore unimportant logs and email errors directly to themselves.

Some applications, such as web servers, choose to do their own logging. Usually this is needed to get the performance or customization that the application needs, or sometimes it’s because the application was designed to work on many different systems, some that don’t have syslog. As a systems administrator you will have to deal with all kinds of logs.

systemd and syslog

Linux has two different logging systems: syslog and systemd’s journal. syslog is a standardized system that can receive messages from local applications or remote servers and write them to disk or send to another server. It is ubiquitous such that many network enabled appliances generate syslog messages. The systemd journal is a newer form of logging for distributions that have moved to systemd. Fortunately, it is backward compatible so that applications that can log to syslog can also log to systemd, and it can also forward messages to a syslog server for centralized logging.

Applications that use systemd’s version of logging can also log the additional metadata tracked by the journal. For example, the name of the method and the line number are logged alongside the message, which allows for additional statistics to be gathered.

syslog

syslog is a simple protocol, which has led to its adoption across many different systems and applications.

A message can be logged in several ways:

From within an application using the syslog library call

From the command line using the logger command

Remotely by connecting to a syslog server over the network

A message is logged with a severity and a facility, and the syslog daemon takes care of processing the message. The severities are listed in Table 18-3.

Table 18-3 Syslog Severities

When you log at a particular level, you’re saying that you’re only capturing logs at that level or with a lower priority level and throwing away anything else. For example, most systems log at the informational level (6). This would ignore debug logs (7) and capture anything else. You may decide to log at the warning level (4) to ignore reports of normal operation at levels 5, 6, and 7 and only concern yourself with level 4 warnings and more severe items at levels 1, 2, and 3.

For the exam, it is important to remember the seven log levels and their order.

Each log is also tied to a facility that describes what generated the log. The combination of facility and level gives you control over what is stored. You may choose to log the email system at the informational level, which gives you reports of each email that goes through the system, but logs cron at warning, so you only hear about things that go wrong.

Table 18-4 lists the available facilities.

Table 18-4 Log Facilities

The syslog daemon is responsible for the collection and distribution of log messages. You can think of it as a pipeline that takes messages from various sources and directs them to the appropriate destination. Figure 18-1 shows a typical syslog configuration in graphical form.

Figure 18-1 The flow of logging information

In Figure 18-1 the logs come in from one of two places: remotely, over the network, or from the local system. Locally, the logger command acts as an application and takes messages from the command and turns them into local syslog messages.

The middle box analyzes the messages and, based on the severity and facility, decides where to send them. Starting from the left, anything with an emergency severity is displayed to the system console. This is partially to capture the attention of anyone who happens to be watching the console, but also to ensure a trace is left. If the log is truly about an emergency, it’s possible that it’s related to the disks or the system is in the process of crashing. Next, all mail messages logged with a severity of at least info are stored in /var/log/maillog. All messages except for mail messages go to /var/log/messages. Finally, all messages are sent to a remote syslog server.

A single message may be sent to multiple outputs, or none. An emergency mail message would go to the console, /var/log/maillog, and the remote syslog server. A mail related debugging message would only go to the remote server.

Figure 18-1 only shows a sample. Your needs may be different and call for a different configuration.

The logger Command

The logger command is helpful for both testing your logging configuration and for logging within scripts. The easiest way to use it is just to pass the log message on the command line:

$ logger Starting script processing

Looking in the logs, such as /var/log/messages, you see something like

Apr 20 19:55:02 bob sean: Starting script processing

The log entry contains both the hostname (bob) and the user (sean).

For additional information you can pass the process ID with the -i flag.

$ logger -i Starting script processing
Apr 20 19:55:57 bob sean[8969]: Starting script processing

The default destination of the message is the user facility at the notice level, which can be overridden with the -p flag:

$ logger -i -p mail.info Checking user quotas
Apr 20 19:57:28 bob sean[9054]: Checking user quotas

Configuring syslogd

Linux has several syslog daemons that can be used, but the simplest is part of the sysklogd package. The name is an amalgamation of syslog and klogd. syslogd is the component that handles the logging of messages as described in Figure 18-1. klogd listens specifically for kernel logs and forwards them to syslogd. klogd can also do processing on the message before sending it along, such as to translate memory addresses into names that would be helpful to kernel developers.

klogd has very little to configure, so the interesting parts are found in syslogd.

syslogd’s configuration file is in /etc/syslog.conf. This file defines a set of rules, one per line. Each rule is evaluated, and if the log matches that rule, the log is sent to the destination specified in the rule.

Example 18-6 shows a simple syslog.conf.

Example 18-6 A Sample /etc/syslog.conf