UNIX: The Complete Reference (2007)

Part I: Basic

Chapter List

Chapter 1: Background

Chapter 2: Getting Started

Chapter 3: Working with Files and Directories

Chapter 4: The Command Shell

Chapter 5: Text Editing

Chapter 6: The GNOME Desktop

Chapter 7: The CDE and KDE Desktops

Chapter 1: Background

Overview

The UNIX computer operating system has had a fascinating history and evolution. Starting as a research project begun by a handful of people, it has become an important product used extensively in business, academia, and government. Today, people use operating systems with many different names that are variants of UNIX. Many of the commands and utilities in these different variants are identical and others are extremely similar. The differences between these variants often lie in the inner workings of the operating system, not seen by the user, as well as in special added capabilities for advanced users or system administrations.

This chapter provides a foundation for understanding what UNIX is and how it has evolved. It describes the structure of UNIX and introduces its major components, including the shell, the file system, and the kernel. You will see how the applications and commands you use relate to this structure. Understanding the relationships among these components will help you read the rest of this book and use any version of UNIX effectively.

To gain an insight into how the relationships between the different components of UNIX evolved, you should learn something about the history of UNIX from its birth at Bell Laboratories to the early twenty-first century Understanding this history will also help you understand the origins of different UNIX variants and help you see how they are related. The chapter also describes the standards that have been developed and are now used as the yardstick for determining whether an operating system can be called “UNIX.”

This chapter also includes a description of the most widely used UNIX variants. In particular, you will read about the history and philosophy of Linux, an open-source version of UNIX that has become exceedingly popular. You will also learn about the most widely used UNIX variants, including Solaris, AIX, and HP-UX, as well as FreeBSD, NetBSD, OpenBSD, Mac OS X, UnixWare, and IRIX. You will also find an extensive timeline that displays how the important variants of UNIX have evolved. Important contributors to the development of UNIX are also noted.

Because UNIX variants often compete with versions of Windows NT, this chapter compares these two operating systems. The chapter concludes with a discussion of the future of UNIX and some words of advice about which UNIX version you might want to choose.

What Is UNIX?

UNIX once referred to a specific operating system. However, today it is not a single operating system, but rather a large family of closely related operating systems. These different operating systems are sometimes known as UNIX variants, or UNIX-like operating systems. All these operating systems are built using a collection of enabling technologies that were originally developed in the 1970s at AT&T Bell Laboratories and at the University of California, Berkeley They have much in common and share a set of utilities and programs. However, each variant has its own peculiarities and differs from other variants particularly in its kernel, or inner code, and in specialized features.

Why Is UNIX Important?

During the past 35 years, the operating system known as UNIX has evolved into a powerful, flexible, and versatile operating system. The different variants of UNIX conform to a variety of standards and are closely related. To understand how to use any or all of them, you need to only understand the basic conceptual model upon which UNIX is built. Once this conceptual model is understood, it is straightforward to learn the peculiarities of a variant of UNIX or to learn how to use a new variant of UNIX if you already know how to use another.

UNIX, as it is implemented in its many variants, serves as the operating system for all types of computers, including personal computers and engineering workstations, multiuser microcomputers, minicomputers, mainframes, and supercomputers, as well as special-purpose devices. The number of computers running a variant of UNIX has grown explosively with more than 40 million computers now running a variant of UNIX and more than 300 million people using these systems. This rapid growth, especially for computers running Linux, is expected to continue, according to most computer industry experts. The success of UNIX is due to many factors, including its portability to a wide range of machines, its adaptability and simplicity, the wide range of tasks that it can perform, its multiuser and multitasking nature, and its suitability for networking, which has become increasingly important as the Internet has blossomed. What follows is a description of the features that have made UNIX so popular.

Open Source Code

The source code for key variants of UNIX, and not just the executable code, has been made available to users and programmers. Because of this, many people have been able to adapt UNIX in different ways. This openness has led to the introduction of a wide range of new features and versions customized to meet special needs. It has been easy for developers to adapt to UNIX, because the computer code for UNIX is straightforward, modular, and compact. This has fostered the evolution of UNIX. New features are constantly being developed for various versions of UNIX, with most of these features compatible with earlier versions.

Cooperative Tools and Utilities

The UNIX System provides users with many different tools and utilities that can be leveraged to perform an amazing variety of jobs. Some of these tools are simple commands that you can use to carry out specific tasks. Other tools and utilities are really small programming languages that you can use to build scripts to solve your own problems.

Most important, the tools are intended to work together, like machine parts or building blocks. Not only are many tools and utilities included with UNIX, but many others are available as add-ons, including many that are available free of charge from archives on the Internet.

Multiuser and Multitasking Abilities

The UNIX operating system can be used for computers with many users or a single user, because it is a multiuser system. It is also a multitasking operating system, because a single user can carry out more than one task at once. For instance, you can run a program that checks the spelling of words in a text file while you simultaneously read your electronic mail.

Excellent Networking Environment

The UNIX operating system provides an excellent environment for networking. It offers programs and utilities that provide the services needed to build networked applications-the basis for distributed, networked computing. With networked computing, information and processing is shared among different computers in a network. The UNIX system has proved to be useful in client/server computing where machines on a network can be both clients and servers at the same time. UNIX also has been the base system for the development of Internet services and for the growth of the Internet. UNIX provides an excellent platform for web servers. Consequently, with the growing importance of distributed computing and the Internet, the popularity of UNIX has grown.

Portability

It is far easier to port UNIX to new machines than other operating systems-that is, far less work is needed to adapt it to run on a new hardware platform. The portability of UNIX results from its being written almost entirely in the C programming language. The portability to a wide range of computers makes it possible to move applications from one system to another.

The preceding brief description shows some of the important attributes of UNIX that have led to its explosive growth. More and more people are using UNIX variants, especially Linux, as they realize that it provides a computing environment that supports their needs. Also, many people use UNIX without even knowing it, such as people using the desktop environment of Mac OS X without knowing that it is built on UNIX, and people who use devices running a UNIX variant designed to support embedded systems. Moreover, many people now use computers running a variety of operating systems, with clients, servers, and special-purpose computers running different operating systems. UNIX plays an important role in this mix of operating systems. Many people run both a variety of Windows and one of UNIX on the same personal computer; some of these machines even ask the user which operating system to boot when the machine is turned on.

The Structure of the UNIX Operating System

To understand how UNIX works, you need to understand its structure. The UNIX operating system is made up of several major components. These components include the kernel, the shell, the file system, and the commands (or user programs). The relationship among the user, the shell, the kernel, and the underlying hardware is displayed in Figure 1–1.

Figure 1–1: The structure of the UNIX System

The Kernel

The kernel is the part of the operating system that interacts directly with the hardware of a computer, through device drivers that are built into the kernel. It provides sets of services that can be used by programs, insulating these programs from the underlying hardware. The major functions of the kernel are to manage computer memory, to control access to the computer, to maintain the file system, to handle interrupts (signals to terminate execution), to handle errors, to perform input and output services (which allow computers to interact with terminals, storage devices, and printers), and to allocate the resources of the computer (such as the CPU or input/output devices) among users.

Programs interact with the kernel through system calls. System calls tell the kernel to carry out various tasks for the program, such as opening a file, writing to a file, obtaining information about a file, executing a program, terminating a process, changing the priority of a process, and getting the time of day Different implementations of a variant of the UNIX system may have compatible system calls, with each call having the same functionality However, the internals, programs that perform the functions of system calls (usually written in the C language), and the system architecture in two different UNIX variants or even two different implementations of a particular UNIX variant may bear little resemblance to one another.

Utilities

The UNIX System contains several hundred utilities or user programs. Commands are also known as tools, because they can be used separately or put together in various ways to carry out useful tasks. You execute these utilities by invoking them by name through the shell; this is why they are called commands.

A critical difference between UNIX and earlier operating systems is the ease with which new programs can be installed-the shell need only be told where to look for commands, and this is user-definable.

You can perform many tasks using the standard utilities supplied with UNIX. There are utilities for text editing and processing, for managing information, for electronic communications and networking, for performing calculations, for developing computer programs, for system administration, and for many other purposes. Much of this book is devoted to a discussion of utilities. In particular, Chapters 3, 4, and 19cover a variety of tools of interest to users. Specialized tools, including both those included with UNIX and others available as add-ons, and are introduced throughout the book. One of the nice, but not unique, features of UNIX is the availability of a wide variety of add-on utilities, either free of charge or by purchase from software vendors.

The File System

The basic unit used to organize information in UNIX is called a file. The UNIX file system provides a logical method for organizing, storing, retrieving, manipulating, and managing information. Files are organized into a hierarchical file system, with files grouped together into directories. An important simplifying feature of UNIX is the general way it treats files. For example, physical devices are treated as files; this permits the same commands to work for ordinary files and for physical devices; for instance, printing a file (on a printer) is treated similarly to displaying it on the terminal screen.

The Shell

The shell reads your commands and interprets them as requests to execute a program or programs, which it then arranges to have carried out. Because the shell plays this role, it is called a command interpreter. Besides being a command interpreter, the shell is also a programming language. As a programming language, it permits you to control how and when commands are carried out. The shell (and its major variants) is discussed in Chapters 4 and 20.

Applications

You can use applications built using UNIX commands, tools, and programs. Application programs carry out many different types of tasks. Some perform general functions that can be used by a variety of users in government, industry, and education. These are known as horizontal applications and include such programs as word processors, compilers, database management systems, spreadsheets, statistical analysis programs, and communications programs. Others are industry-specific and are known as vertical applications. Examples include software packages used for managing a hotel, running a bank, and operating point-of-sale terminals. UNIX application software is discussed in Chapter 26. UNIX text processing software packages are covered on the companion web site (http://books.mcgraw-hill.com/getbook.php?isbn=0072263369&template=computing).

Several classes of applications have experienced explosive growth in the past few years. The first of these involves network applications, including those that let people make use of the wide range of services available on the Internet. Chief among these are web browsers and web server applications. Another important class of applications deals with multimedia. Such applications let users create and view multimedia files, including audio, images, and video.

The UNIX Philosophy

As it has evolved, UNIX has developed a characteristic, consistent approach that is sometimes referred to as the UNIX philosophy. This philosophy has deeply influenced the structure of the system and the way it works. Keeping this philosophy in mind helps you understand the way UNIX treats files and programs, the kinds of commands and programs it provides, and the way you use it to accomplish a task.

The UNIX philosophy is based on the idea that a powerful and complex computer system should still be simple, general, and extensible, and that making it so provides important benefits for both users and program developers. Another way to express the basic goals of the UNIX philosophy is to note that, for all its size and complexity, UNIX still reflects the idea that “small is beautiful.” This approach is especially reflected in the way UNIX treats files and in its focus on software tools.

UNIX views files in an extremely simple and general way within a single model. It views directories, ordinary files, devices such as printers and disk drives, and your keyboard and terminal screen all in the same way The file system hides details of the underlying hardware from you; for example, you do not need to know which drive a file is on. This simplicity allows you to concentrate on what you are really interested in-the data and information the file contains. In a local area network, the concept of a remote file system even saves you from needing to know which machine your files are on.

The fact that your screen and keyboard are treated as files enables you to use the same programs or commands that deal with ordinary stored files for taking input from your terminal or displaying information on it.

A unique characteristic of UNIX is the large collection of commands or software tools that it provides. This is another expression of the basic philosophy These tools are small programs, each designed to perform a specific function, and all designed to work together. Instead of a few large programs, each trying to do many things, UNIX provides many simple tools that can be combined to do a wide range of things. Some tools carry out one basic task and have mnemonic names. Others are programming languages in their own right with their own complicated syntaxes.

A good example of the tools approach is the sort command, which takes a file, sorts it according to one of several possible rules, and outputs the result. It can be used with any text file. It is often used together with other programs to sort their output.

A separate program for sorting means that other programs do not have to include their own sorting operations. This has obvious benefits for developers, but it also helps you. By using a single, generic, sorting program, you avoid the need to learn the different commands, options, and conventions that would be necessary if each program had to provide its own sorting.

The emphasis on modular tools is supported by one of the most characteristic features of UNIX-the pipe. This feature, important for both users and programmers, is a general mechanism that enables you to use the output of one command as the input of another. It is the “glue” used to join tools together to perform the tasks you need. UNIX treats input and output in a simple and consistent way, using standard input and standard output. For instance, input to a command can be taken either from a terminal or from the output of another command without using a different version of the command.

The Birth of the UNIX System

The history of the UNIX System dates back to the late 1960s when MIT, AT&T Bell Labs, and then-computer manufacturer GE (General Electric) worked on an experimental operating system called Multics. Multics, from Multiplexed Information and Computing System, was designed to be an interactive operating system for the GE 645 mainframe computer, allowing information sharing while providing security Development met with many delays, and production versions turned out to be slow and required extensive memory For a variety of reasons, Bell Labs dropped out of the project. However, the Multics system implemented many innovative features and produced an excellent computing environment.

In 1969, Ken Thompson, one of the Bell Labs researchers involved in the Multics project, wrote a game for the GE computer called Space Travel. This game simulated the solar system and a space ship. Thompson found that the game ran jerkily on the GE machine and was costly-approximately $75 per run! With help from Dennis Ritchie, Thompson rewrote the game to run on a spare DEC PDP-7. This initial experience gave him the opportunity to write a new operating system on the PDP-7, using the structure of a file system Thompson, Ritchie, and Rudd Canaday had designed. Thompson, Ritchie, and their colleagues created a multitasking operating system, including a file system, a command interpreter, and some utilities for the PDP-7. Later, after the new operating system was running, Space Travel was revised to run under it. Many things in the UNIX System can be traced back to this simple operating system.

Because the new multitasking operating system for the PDP-7 could support two simultaneous users, it was humorously called UNICS for the Uniplexed Information and Computing System; the first use of this name is attributed to Brian Kernighan. The name was changed slightly to UNIX in 1970, and that has stuck ever since. The Computer Science Research Group wanted to continue to use the UNIX System, but on a larger machine. Ken Thompson and Dennis Ritchie managed to get a DEC PDP-11/20 in exchange for a promise of adding text processing capabilities to the UNIX System; this led to a modest degree of financial support from Bell Laboratories for the development of the UNIX System project. The UNIX operating system, with the text formatting program runoff, both written in assembly language, was ported to the PDP-11/20 in 1970. This initial text processing system, consisting of the UNIX operating system, an editor, and runoff, was adopted by the Bell Laboratories Patent Department for text processing. runoffevolved into troff, the first electronic publishing program with typesetting capability.

In 1972, the second edition of the UNIX Programmer’s Manual mentioned that there were exactly ten computers using the UNIX System, but that more were expected. In 1973, Ritchie and Thompson rewrote the kernel in the C programming language, a high-level language unlike most systems for small machines, which were generally written in assembly language. Writing the UNIX operating system in C made it much easier to maintain and to port to other machines. The UNIX System’s popularity grew because it was innovative and was written compactly in a high-level language with code that could be modified to individual preferences. AT&T did not offer the UNIX System commercially because, at that time, AT&T was not in the computer business. However, AT&T did make the UNIX System available to universities, commercial firms, and the government for a nominal cost.

UNIX System concepts continued to grow. Pipes, originally suggested by Doug McIlroy, were developed by Ken Thompson in the early 1970s. The introduction of pipes made possible the development of the UNIX philosophy, including the concept of a toolbox of utilities. Using pipes, tools can be connected, with one taking input from another utility and passing output to a third.

By 1974, the fourth edition of the UNIX System had become widely used inside Bell Laboratories. (Releases of the UNIX System produced by research groups at Bell Laboratories have traditionally been known as editions.) By 1977, the fifth and sixth editions had been released; these contained many new tools and utilities. The number of machines running the UNIX System, primarily at Bell Laboratories and universities, increased to more than 600 by 1978. The seventh edition, the direct ancestor of the UNIX operating system available today, was released in 1979.

UNIX System III, based on the seventh edition, became AT&T’s first commercial release of the UNIX System in 1982. However, after System III was released, AT&T, through its Western Electric manufacturing subsidiary, continued to sell versions of the UNIX System. UNIX System III, the various research editions, and experimental versions were distributed to colleagues at universities and other research laboratories. It was often impossible for a computer scientist or developer to know whether a particular feature was part of the mainstream UNIX System or just part of one of the variants that might fade away. To foster the success of UNIX, AT&T needed to clarify what constituted mainstream UNIX, which they did when they released UNIX System V, discussed in the next subsection.

UNIX System V

To eliminate the confusion over varieties of the UNIX System, AT&T introduced UNIX System V Release 1 in 1983. (UNIX System IV existed only as an internal AT&T release.) With UNIX System V Release 1, for the first time, AT&T promised to maintain upward compatibility in its future releases of the UNIX System. This meant that programs built on Release 1 would continue to work with future releases of System V.

Release 1 incorporated some features from the version of the UNIX System developed at the University of California, Berkeley, including the screen editor vi and the screenhandling library curses. AT&T offered UNIX System V Release 2 in 1985. Release 2 introduced protection of files during power outages and crashes, locking of files and records for exclusive use by a program, job control features, and enhanced system administration. Release 2.1 introduced two additional features of interest to programmers: demand paging, which allows processes to run that require more memory than is physically available, and enhanced file and record locking.

In 1987, AT&T introduced UNIX System V Release 3.0; it included a simple, consistent approach to networking. These capabilities include STREAMS, used to build networking software, the Remote File System, used for file sharing across networks, and the Transport Level Interface (TLI), used to build applications that use networking. Release 3.1 made UNIX System V adaptable internationally, by supporting wider character sets and date and time formats. It also provided for several important performance enhancements for memory use and for backup and recovery of files. Release 3.2 provided enhanced system security, including displaying a user’s last login time, recording unsuccessful login attempts, and a shadow password file that prevents users from reading encrypted passwords. Release 3.2 also introduced the Framed Access Command Environment (FACE), which provides a menu-oriented user interface.

Release 4 unified various versions of UNIX that were developed inside and outside AT&T, including the BSD System, the SunOS, and XENIX, each discussed later in this chapter. These variants were all merged into UNIX System V Release 4. UNIX System V Release 4 met its goal of providing a single UNIX System environment, meeting the needs of a broad array of computer users. Because of this, SVR4 has served as the basis for much of the further evolution of UNIX.

After releasing UNIX System V Release 4, AT&T split off its UNIX System Laboratories (USL) as a separate subsidiary AT&T held a majority stake in USL, selling off portions of USL to other companies. USL developed UNIX System V Release 4.2, also known as Destiny, to address the market for running UNIX on the desktop. Release 4.2 included a graphical user interface that helps users manage their desktop environment and simplifies many administrative tasks.

In July 1993, AT&T sold its UNIX System Laboratories to Novell. Companies competing with Novell in the UNIX market, including the Santa Cruz Operation (SCO) and Sun Microsystems, objected to Novell’s control of UNIX System V; they felt that this control would give Novell an advantage over competing products in the UNIX marketplace.

To counter this perception, in October 1993, Novell transferred trademark rights to the UNIX operating system to X/Open (which is now part of the Open Group-discussed later in this chapter). Under this agreement, any company could use the name UNIX for an operating system, as long as the operating system complied with X/Open’s specifications, with a royalty fee going to X/Open. Novell continued to license System V Release 4 source code to other companies, either taking royalty payments or making a lumpsum sale. Novell also developed its own version of System V Release 4, called UnixWare.

In 1995, Novell sold its ownership of UNIX System V Release 4 and its version of UNIX System V Release 4, UnixWare, to the Santa Cruz Operation. SCO became the owner of the UNIX System V Release 4 source code and continued the development of UNIX System V Release 4 (and in 1997 introduced UNIX System V Release 5 under the name SCO UnixWare 7-see later in this chapter for more information). Unlike UNIX System V Release 4, which was licensed by many computer companies, this newer release of UNIX System V was not licensed by other computer companies. In 2000 SCO sold the rights to its UnixWare operating system, including its ownership of the source code for UNIX System V, as well as parts of its company, to Caldera Systems, a company whose original product was a distribution of Linux (see later in this chapter of a discussion of Linux). Caldera later changed its name to the SCO Group. (The old SCO company had changed the name to the part of its company not sold to Caldera to Tarantella.) The SCO Group (the company formerly called Caldera) has instituted some extremely controversial legal actions asserting its intellectual property rights from its ownership of the UNIX System V source code. These legal actions (discussed later in this chapter) have caused uproar in the UNIX/Linux communities, and the ultimate disposition of these legal actions is still up in the air.

The Berkeley Software Distribution (BSD)

Many important innovations to UNIX have been made at the University of California, Berkeley. Some of these enhancements had been made part of UNIX System V in earlier releases, and many more were introduced in UNIX System V Release 4. Furthermore, several important UNIX variants are primarily based on earlier versions of UNIX developed at the University of California, Berkeley.

U.C. Berkeley became involved with UNIX in 1974, starting with the fourth edition. The development of Berkeley’s version of UNIX was fostered by Ken Thompson’s 1975 sabbatical at the Department of Computer Science. While at Berkeley, Thompson ported the sixth edition to a PDP-11/70, making UNIX available to a large number of users. Graduate students Bill Joy and Chuck Haley did much of the work on the Berkeley version. They put together an editor called ex and produced a Pascal compiler. Joy put together a package that he called the “Berkeley Software Distribution.” He also made many other valuable innovations, including the C shell and the screen-oriented editor vi-an expansion of ex. In 1978, the Second Berkeley Software Distribution was made; this was abbreviated as 2BSD. In 1979, 3BSD was distributed; it was based on 2BSD and the seventh edition, providing virtual memory features that allowed programs larger than available memory to run. 3BSD was developed to run on the DEC VAX-11/780.

In the late 1970s, the United States Department of Defense’s Advanced Research Projects Agency (DARPA) decided to base their universal computing environment on UNIX. DARPA decided that the development of their version of UNIX should be carried out at Berkeley Consequently, DARPA provided funding for 4BSD. In 1983, 4.1BSD was released; it contained performance enhancements. The 4.2BSD operating systems, also released in 1983, introduced networking features, including TCP/IP networking, which can be used for file transfer and remote login, and a new file system that sped access to files. Release 4.3BSD came out in 1987, with minor changes to 4.2BSD.

Many computer vendors have used the BSD System as a foundation for the development of their variants of UNIX. One of the most important of these variants is the Sun Operating System (SunOS, which has evolved into Solaris, discussed later in this chapter), developed by Sun Microsystems, a company cofounded by Joy SunOS added many features to 4.2BSD, including networking features such as the Network File System (NFS). The SunOS was one of the UNIX variants that were merged to create UNIX System V Release 4.

Although the BSD System played an important role in the creation of UNIX System V Release 4, it continued to evolve independently The latest version of BSD was 4.4 BSD, which included a wide variety of enhancements, many involving networking capabilities. Furthermore, both the source code and the binary code for a variant of 4.4BSD, known as 4.4 BSD-Lite, were freely distributed, encumbered by licenses for earlier AT&T developed versions of UNIX.

Many UNIX variants are based on BSD releases, including 386BSD, a free version of BSD developed in the early 1990s for the Intel 80836 processor. FreeBSD, a widely used free UNIX variant, is based on 386BSD and 4.4 BSD-Lite. FreeBSD, and several other important UNIX variants based on BSD, including NetBSD and OpenBSD, are discussed later in this chapter.

XENIX

In 1980, Microsoft introduced XENIX, a variant of UNIX designed to run on microcomputers. The introduction of XENIX brought UNIX capabilities to desktop machines; previously these capabilities were available only on larger computers. XENIX was originally based on the Seventh Edition, with some utilities borrowed from 4.1BSD. In Release 3.0 of XENIX, Microsoft incorporated new features from AT&T’s UNIX System III, and in 1985 XENIX was moved to a UNIX System V base.

In 1987 XENIX was ported to 80386-based machines by the Santa Cruz Operation, a company that had worked with Microsoft on XENIX development. In 1987, Microsoft and AT&T began joint development efforts to merge XENIX with UNIX System V, and they accomplished this in UNIX System V Release 3.2. This effort provided a unified version of the UNIX System that runs on systems ranging from desktop personal computers to supercomputers. Of all the early variants of UNIX, the XENIX System achieved the largest installed base of machines. This position was only surpassed by Linux in 2000, the widely used free variant of UNIX which we next discuss.

GNU and Linux

In 1984 Richard Stallman began work on a free operating system called GNU (a reverse acronym, that is, an acronym that refers to itself, for GNU is Not UNIX). Stallman founded the Free Software Foundation, a nonprofit organization supporting the creation and sharing of free software, and in particular, the GNU project. The goal of the GNU project was to make GNU like UNIX, without using any UNIX source code. Stallman wanted to develop an operating system that could evolve through the work of a community, with users free to study the source code and to modify and publish enhancements to it. Because constructing an entire operating system, including the kernel and user utilities, is a daunting task, GNU was designed to be modular so that different people could develop different parts of the operating system and so that it could easily incorporate already existing free software. By 1990 GNU had its own versions of the almost all the utilities, tools, and core libraries of UNIX, as well as the emacs text editor and a C compiler, GCC. However, it lacked a kernel, and initial efforts to develop one were not entirely successful.

Meanwhile, in 1991 Linus Torvalds, then a student at the University of Helsinki, Finland, decided to build a kernel for a new UNIX-like operating system for PCs. Torvalds had been working with the Minix operating system built by Andrew Tannenbaum to illustrate features of UNIX. Torvalds wanted a UNIX version for PCs that captured the features of Minix. He considered his work on this new kernel, which was eventually named Linux, to be a hobby and thought his new operating system would never become anything remotely like a professional-quality operating system. Torvalds invited other people to download a copy of his new kernel over the Internet and to improve and add to it. Many people decided to take up Torvalds’s offer, relating to his goals and the inherent technical challenges. They worked alone, and in teams, to improve Linux. All this work was, and continues to be, done under the direction of Linus Torvalds, with communication and collaboration done over the Internet.

A key goal of the developers of Linux kernel is not to use any proprietary code. The kernel is legally protected by the GNU Public License; it is packaged with many executables making up a fully functional version of UNIX. Combined with GNU software, containing UNIX-like utilities, tools, core libraries, compilers, text editors, desktop environments, and other components, Linux (sometimes called GNU/Linux) provides a complete UNIX environment.

The latest major release of the Linux kernel is Linux 2.6, which was introduced in 2003; minor releases are frequently made. The Linux 2.6 kernel supports 64-bit computing (that is, computing that supports addressing up to 2⁶⁴ bytes of virtual memory, which far exceeds the amount needed to support even 4 GB of RAM) and hyperthreading (which allows multiple threads of computer code to run at the same time on Intel Pentium IV processors, providing improved performance and allowing more users to be supported on a server). It provides performance improvements for database applications and for networking and offers increased levels of security.

Linux has become increasingly popular and receives wide attention in the computer industry Linux has become popular because, among other reasons, it is free, a large community of developers is constantly adding new features and capabilities to Linux, and many people relate to the philosophy behind Linux. This philosophy, which endorses the notion that software should be open and free, runs counter to the way Microsoft has done most of its business. (For example, Microsoft has long kept the code for its Windows operating system closed.) To use Linux, you need to obtain a Linux distribution. We will discuss Linux distributions later in this chapter when we discuss widely used UNIX variants.

The SCO Lawsuit

Although Linux has been designed to be free of commercial code, in 2003 the SCO Group filed a lawsuit against IBM. The SCO Group claimed that IBM had contributed some code that was protected by an SCO Group copyright to Linux, violating the license IBM holds to use UNIX. Also, the SCO Group filed suits against other companies. This controversy has not yet been settled, although most experts feel that the SCO Group is incorrect in their assertions of copyright infringement. These experts feel that these SCO Group lawsuits will ultimately be dismissed.

UNIX Standards

Standards also steer the evolution of the UNIX System. First, features are developed for a particular variety of UNIX, and then sometimes these features become part of a standards process. Once a feature is standardized, different versions of UNIX include a compliant version of this feature.

The use of different versions of UNIX led to problems for applications developers who wanted to build programs for a range of computers running UNIX. To solve these problems, various standards have been developed. These standards define the characteristics a system should have so that applications can be built to work on any system conforming to the standard.

The System V Interface Definition (SVID)

For UNIX System V to become an industry standard, other vendors needed to be able to test their versions of the UNIX System for conformance to System V functionality In 1983, AT&T published the System V Interface Definition (SVID). The SVID specifies how an operating system should behave for it to comply with the standard. Developers could build programs that are guaranteed to work on any machine running a SVID-compliant version of the UNIX System. Furthermore, the SVID specifies features of the UNIX System that were guaranteed not to change in future releases, so that applications were guaranteed to run on all releases of UNIX System V. Vendors could check whether their versions of UNIX were SVID-compliant by running the System V Verification Suite developed by AT&T. The SVID evolved with new releases of UNIX System V. A newer version of the SVID was prepared in conjunction with UNIX System V Release 4.

POSIX

An independent effort to define a standard operating system environment was begun in 1981 by/usr/group, an organization made up of UNIX System users who wanted to ensure the portability of applications. They published a standard in 1984. Because of the magnitude of the job, in 1985 the committee working on standards merged with the Institute for Electrical and Electronics Engineers (IEEE) Project 1003 (P1003). The goal of P1003 was to establish a set of American National Standards Institute (ANSI) standards. The standards that the various working groups in P1003 are establishing are called the Portable Operating System Interface for Computer Environments (POSIX). POSIX is a family of standards that defines the way applications interact with an operating system. Among the areas covered by POSIX standards are system calls, libraries, tools, interfaces, verification and testing, real-time features, and security The POSIX standard that has received the most attention is P1003.1 (also known as POSIX.1), which defines the system interface. Another important POSIX standard is P1003.2 (also known as POSIX.2), which deals with shells and utilities. POSIX 1003.3 covers testing methods for POSIX compliance; POSIX 1003.4 covers real-time extensions. Many additional POSIX standards have been developed besides these four standards.

POSIX has been endorsed by the National Institute of Standards and Technology (NIST), previously known as the National Bureau of Standards (NBS), as part of the Federal Information-Processing Standard (FIPS). The FIPS must be met by computers purchased by the U.S. federal government.

The Open Software Foundation (OSF)

In 1988 a group of computer vendors, including IBM, DEC, and Hewlett-Packard, formed a consortium called the Open Software Foundation (OSF) to develop a version of the UNIX System to compete with UNIX System V Release 4. Their version of the UNIX System, called OSF/1, never really has played much of a role in the UNIX marketplace. Of all the major vendors in this consortium, only the Digital Equipment Corporation (DEC, later bought by Compaq, which in turn was bought by HP) based its core strategy on an OSF version of UNIX. OSF also sponsored its own graphical user interface, called MOTIF, which was created as a composite of graphical user interfaces from several vendors in OSF. Unlike OSF/1, MOTIF saw wide marketplace acceptance. After 1990, the OSF changed direction; instead of developing new technology, it acted as a clearinghouse for open systems technology. In 1996 OSF merged with the X/OPEN Consortium to form the Open Group. (See later in this chapter for discussions of X/OPEN and the Open Group.)

The X/OPEN Consortium

Another way that vendors addressed the problem posed by competing versions of UNIX was to set standards that an operating system could meet to be “UNIX.” One such set standard was provided by X/Open, an international consortium of computer vendors established in 1984. X/Open adopted existing standards and interfaces, without developing its own standards. X/Open was begun by European computer vendors and grew to include most U.S. computer companies.

The goal of X/Open was to standardize software interfaces. They did this by publishing their Common Applications Environment (CAE). The CAE was based on the SVID and contained the POSIX standards. UNIX System V Release 4 conformed to XPG3, the third edition of the X/Open Portability Guide. In 1992 X/Open announced XPG4, the fourth edition of their portability guide. XPG4 includes updates to specifications in XPG3 and many new interface specifications, with a strong emphasis on interoperability between systems. In 1996 X/Open merged with the OSF to form the Open Group (discussed later in this chapter).

The X/OPEN API

One of the major problems in the UNIX (and open systems) industry is that a software vendor must devote a great deal of effort to porting a particular software product to different UNIX systems. In 1993, to help mitigate this problem, X/Open assumed the responsibility for managing the evolution of a common application programming interface (API) specification. This specification allowed a vendor of UNIX System software to develop applications that will work on all UNIX platforms supporting this specification. The original name for this specification was Spec 1170, for the 1,170 different application programming interfaces originally in it. These 1,170 APIs came from X/Open’s XPG4, from the System V Interface Definition, from the OSF’s Application Environment Specification (AES) Full Use Interface, and from user-based routines derived from a source code analysis of leading UNIX System application programs. When X/Open took over responsibility for this specification, it made some additions and changes, defining what is now called the Single UNIX Specification. Systems demonstrating conformance to the Single UNIX Specification received the mark UNIX 95. Among the vendors that registered UNIX 95 systems with X/Open were HP, DEC (which was purchased by Compaq, which was later purchased by HP), IBM, NCR, SCO, SGI, and Sun.

The Common Open Software Environment (COSE)

In 1993, some of the major UNIX vendors created the Common Open Software Environment consortium. Among these vendors were Hewlett-Packard, IBM, SunSoft, SCO, Novell, and the UNIX System Laboratories. The goal of this consortium was to define industry standards for UNIX systems in six areas: graphical user interface, multimedia, networking, object technology, graphics, and system management. The first of these areas to be implemented was the graphical user interface. COSE began work on the Common Desktop Environment (CDE), which was designed to be the industry-standard graphical user interface for UNIX systems. Later COSE went out of existence; work on the CDE was taken over by the OSF (which later merged into the Open Group-see the text that follows). Implementations of the CDE first appeared in 1994; it is now included in all major UNIX variants.

The Open Group and the Single UNIX Specification

The Open Group was formed in 1996 when the Open Software Foundation, which had outlived its original charter, and X/Open merged. The Open Group is a consortium whose members include computer vendors, software companies, and end-user organizations. Their vision, considerably expanded from that of X/Open and the OSF, is to foster “boundary-less information flow” that “will enable access to integrated information, within and among enterprises, based on open standards and global interoperability” Their specification of UNIX is only part of this broad mission. For more information on the Open Group, go to http://www.opengroup.org/.

Version 2 of the Single UNIX Specification

In 1997 the Open Group developed an enhanced version of the Single UNIX Specification, called Version 2. The Open Group stated that this specification was developed to ensure that UNIX remains the best platform for enterprise mission-critical systems and for high-performance graphical applications. Version 2 builds upon the original Single UNIX Specification, updating it with new standards and industry advances. Version 2 includes the following:

§ Large file extensions, permitting UNIX systems to support files of arbitrary size, of particular relevance for database applications

§ Dynamic linking extensions that permit applications to share common code across applications, yielding simplified software maintenance and performance enhancements for applications

§ Changes known as the N-bit cleanup, removing hardware data-length dependencies and restrictions, enabling the move to 64-bit processors

§ Changes known as Year 2000 Alignment, designed to minimize the impact of the millennium rollover

§ Extended threads functions, allowing significant performance gains on multiprocessor hardware and increased application throughput

§ Alignment with the latest POSIX standards, including real-time features

The Single UNIX Specification Version 2 contains 1,434 programming interfaces, while the original Single UNIX Specification had 1,170.

UNIX 98

The Open Group has specified UNIX 98 as the mark for systems that conform to Version 2 of the Single UNIX Specification. UNIX 98 is a family of standards for different types of computers, such as basic systems, workstations, and servers:

§ UNIX 98, the base product standard

§ UNIX 98 Workstation, the base product standard together with the Common Desktop Environment

§ UNIX 98 Server, the base product standard together with the Internet Protocol Suite, Java support, and Internet capabilities that support network computing

The UNIX 98 Server is designed to meet the needs for highly reliable Internet applications. HP, IBM, Sun Microsystems, and Fujitsu all had UNIX 98 registered products.

Version 3 of the Single UNIX Specification

Version 3 of the Single UNIX specification was released in 2003. It was developed by the Austin Group, a joint working group of members of the IEEE Portable Applications Standards Committee, the Open Group, and the ISO/IEC Joint Technical Committee 1. The Austin Group created the Single UNIX Specification Version 3 by revising, combining, and updating a collection of diverse UNIX standards, including ISO/IEC 9945–1 and 9945–2, IEEE Standards 1003.1 and 1003.2, and the Base Specifications of The Open Group Single UNIX Specification.

The revision of the Base Specifications were made with the goal of minimizing the number of changes needed to existing implementations conforming to the earlier versions of the standards to bring them into conformance with the new standard.

Besides the Base Specifications, the Single UNIX Specification, Version 3 includes an updated X/Open Curses specification.

UNIX 03

The UNIX 98 Product Standard has been enhanced to produce the UNIX 03 Product Standard. The most important enhancement is alignment with the Single UNIX Specification, Version 3, including new issue of The Open Group Base Specifications (identical to IEEE Std 1003.1– 2001 and ISO/IEC 9945:2002).

The mandatory enhancements beyond the UNIX 98 Product Standard include the alignment of interfaces with ISO/IEC 9899:1999 (relating to the C language) and the addition of new functionality for this alignment, the addition of new networking functionality with the latest issue of XNS and IEEE Standard 1003.1g-2000, and the incorporation of additions and corrections to the core POSIX system interfaces and utilities. These additions and corrections are derived from the P1003.1a and P1003.2b standards.

Optional enhancements included in the UNIX 03 product standard are networking functionality with optional support for Internet Protocol Version 6 (IPv6), additional sets of APIs for real-time support, and the Batch Utilities extension, derived from IEEE Standard 1003.2d-1994.

IBM and Sun both have products that have been certified to meet the UNIX 03 product standard.

Widely Used UNIX Variants

As mentioned before, there is no single operating UNIX operating system. Instead, there is a large collection of UNIX variants. All these variants share a large number of features. Furthermore, porting software between the widely used variants is relatively straightforward. Many of the differences between variants are hidden to the user, and other difference result from the way these variants have evolved. The most significant differences are in the areas of add-ons that help make particular UNIX variants well suited for particular purposes and tasks.

We will briefly describe some of the most important variants here. Subsequent chapters will address the common features shared by different variants of UNIX as well as some of their differences. They will also address some of the specific aspects of some of the most widely used UNIX variants, including Linux, Solaris, HP-UX, Mac OS X, and AIX.

Linux

Linux is an extremely popular variant of the UNIX System. Among the reasons for this popularity is that it can be used free of charge, as well as the depth and breadth of its capabilities and the large amount of software that runs on Linux, made possible by the expertise and dedication of the large Linux development community. This popularity has also been helped by the support for Linux by many commercial computer companies, including IBM, HP, Sun, and Novell. Linux is covered by a copyright under the terms of the GNU General Public License, which prevents people from selling it without allowing the buyer to freely copy and distribute it. The Linux kernel is available on the Internet at hundreds of FTP sites. Linux is now available for many different processors, including the Intel x86 family, Motorola 68k, Sparc, and Power PC.

Today, Linux is widely used both on the desktop and on servers, in the homes, small businesses, and enterprises. Although Linux is not compliant with the POSIX.1 standard, it exhibits a high degree of POSIX compliance. The goal of its developers is to make it as compliant as possible with standards from the Open Group. Linux shares many features of UNIX System V and has many enhancements. It has become a widely popular version of UNIX for use on personal computers. Several desktop environments, including GNOME and KDE, run on Linux, making it easy for users to use Linux if they are used to Windows PCs. Also, application software is readily available for Linux. In the past five years it has become widely used for server applications and is now extensively run on web servers, mail servers, file servers, and firewalls.

To begin using Linux, you will need to obtain a Linux distribution. (You can also buy a PC that is preconfigured with a Linux distribution.) A Linux distribution contains the Linux kernel, a collection of programs and applications that run on Linux, and an installation program. Linux distributions are available from commercial vendors, from nonprofit organizations, from teams of people, and from individuals. Linux distributions can be sold as long as they do not limit the redistribution of their software. There are many different Linux distributions (one count lists more than 450-see http://en.wikipedia.org/wiki/List_of_Linux_distributions).Some Linux distributions are general purpose for desktops or for servers, and some are designed for specific purposes ranging from embedded systems to real-time computing. Although most people speak of Linux as one single operating system, each distribution is really a separate operating system. That is, the many different Linux distributions are really distinct variants of UNIX.

Among the more popular general-purpose Linux distributions (available either via Internet downloading or on CD-ROM) are those from Red Hat (http://www.redhat.com/), Caldera (owned by the SCO Group) (http://www.caldera.com/), Debian (http://www.debian.org/), SuSE (now owned by Novell) (http://www.novell.com/linux/suse/), Mandriva (http://www.mandriva.com/), TurboLinux (http://www.turbolinux.org/), and Slackware (http://www.slackware.org/). Sometimes, vendors of Linux distributions offer a free version of their distribution via Internet download and also sell their distribution on media and with support. Linux distributions may vary in many ways, including the version of the Linux kernel, the programs they include along with the kernel, and their installation programs. Among the applications included with many Linux distributions are web browsers, the Apache web server, security tools, and office applications such as Open Office, a complete office suite. Because of the differences between Linux distributions, applications that run on one distribution may not run on a different distribution. To remedy potential incompatibilities, an effort is underway called the Linux Standard Base (LSB) to develop and promote standards that will increase compatibility among Linux distributions with the goal that applications will be able to run on any compliant Linux system. To learn more about the LSB, go to http://www.linuxbase.org/.

Most of the material in this book is relevant for Linux users, and special attention has been taken to explain some of the most important variations found in Linux. A good starting place for more information about Linux is http://www.linuxresources.com/.

BSD Variants: FreeBSD, NetBSD, and OpenBSD

The Berkeley Software Distribution has been used a base for the development for several widely used UNIX variants, including FreeBSD, NetBSD, and OpenBSD. Surveys show that among users of variants of BSD, FreeBSD is by far the most commonly used, followed by OpenBSD, and then NetBSD.

FreeBSD

FreeBSD 1.0 was introduced in 1993. It was originally based on 4.3BSD-Lite and 386BSD, with many GNU components. Because of legal concerns regarding 386BSD source code, FreeBSD 2.0, released in 1994, was based on 4.4BSD-Lite. FreeBSD runs on a wide range of processors, including Intel x86 processors.

The developers of FreeBSD maintain two branches of simultaneous development, the STABLE branch and the CURRENT branch, which offers an aggressive new kernel and features for users. FreeBSD 5.0 was released in early 2003. It introduced support for application threads, advanced multiprocessors, and new platforms, including the IA-64 platform. FreeBSD 5 introduced improved symmetric multiprocessor (SMP) support. FreeBSD 5 also includes new security features that were developed as part of the TrustedBSD project, a project whose purpose is to add trusted operating system functionality to FreeBSD. This functionality includes an extensible access control framework, access control lists, and a new file system. FreeBSD 6.0 was released in late 2005, and 7.0-CURRENT is under development. These versions continue the work on SMP and threading optimization, as well as additional work in the area of advanced 802.11 functionality and added security functionality.

FreeBSD provides an easy way to install software that has been ported to FreeBSD, called the Ports Collection. Using the Ports Collection, software can be installed using the make command, with little extra work. In particular, most applications are automatically downloaded from the Internet, patched and configured if necessary, compiled, installed, and registered in the package database. Over 14,000 pieces of software are currently available in the Ports Collection. FreeBSD also provides binary compatibility with Linux so that FreeBSD users can run applications developed for Linux that are distributed only in binary form.

To learn more about FreeBSD go to http://www.freebsd.org/.

NetBSD

NetBSD was born in 1993; its first multiplatform release, NetBSD 1.0, was introduced in 1994. NetBSD replaced source code based on 4.3BSD NET/2 with source code based on 4.4BSD-lite, making it freely redistributable without restriction. NetBSD is noted for the quality of its design and implementation. It was developed using 4.3BSD NET/2 and 386BSD as the base. The name NetBSD comes from the importance of the Internet in the distributed way it was developed.

NetBSD is noted for its portability, as well as for the ease of this portability. To emphasize this, the motto for NetBSD is “Of course it runs NetBSD.” Currently, NetBSD runs on more than 50 different hardware platforms, ranging from 64-bit machines, to desktop systems, to handheld devices and embedded systems.

In 1998, NetBSD 1.3 introduced a Package Collection (pkgsrc), which provides the changes needed so that a large collection of freely available software can be run on NetBSD. NetBSD 2.0 was released in 2004. With this release NetBSD introduced support for symmetric multiprocessing (SMP) for several CPU architectures, as well as a native threads implementation. With release 2.0, approximately 50 different platforms were supported.

The current release, NetBSD 3.0, was released at the end of 2005. NetBSD 3.0 supports the Xen Virtual Machine Monitor, which allows NetBSD to support execution of multiple guest operating systems at high level of performance and with resource isolation. Because of the portability of NetBSD, it was long said that NetBSD is portable to every type of machine except perhaps a kitchen toaster. However, with this new release, NetBSD now can control a kitchen toaster through the porting of the operating system to an embedded-system single-board computer that can be housed in the empty space of a toaster. Over 5,700 third-party packages are now supported in pkgrc.

NetBSD also provides system-call level binary compatibility with Linux, FreeBSD, Darwin, Solaris, HP-UX, Solaris, and UnixWare. This allows NetBSD users to run many applications that are distributed only in binary form for other UNIX variants.

For more information about NetBSD, go to http://www.NetBSD.org/.

OpenBSD

OpenBSD was split off from NetBSD by its founder Theo de Raadt in 1994. The initial release of OpenBSD was made in 1996. The project introduces a new release every six months, maintained and supported for one year. It is noted for strong support of security, offering security features and capabilities not found in most other UNIX variants. OpenBSD is based entirely on open-source code that can be licensed free of restrictions. The developers of OpenBSD make extra efforts to audit the source code for bugs and for security problems. As in other BSD-based variants of UNIX, the kernel and user programs of NetBSD are developed together in a single source repository.

The latest release is OpenBSD 3.8, which was released in late 2005. OpenBSD currently runs on 16 different hardware platforms. Third-party software is available as binary packages or may be built from source using its ports collection.

For more information about OpenBSD, go to http://www.openbsd.org/.

Solaris

The original operating system of Sun Microsystems was called the SunOS. It was based on UNIX System V Release 2 and 4.3BSD. In 1991, Sun Microsystems set up SunSoft as a separate subsidiary for the development and marketing of software, including operating systems. At its inception, SunSoft began the task of migrating from the SunOS to a new version of UNIX based on UNIX SVR4. SunSoft’s first version of UNIX, Solaris 1.0, was an enhanced version of the SunOS.

With Solaris 2.0, SunSoft moved to an operating system based on SVR4. Although Solaris 2.0 was the first “official” version of Solaris, it was not widely used due to the limited number of workstations it supported. The first version of Solaris to run on all Sun SPARC-based workstations and Intel x86-based workstations was Solaris 2.1, released in late 1992.

The next significant version was Solaris 2.3, released in November 1993, which introduced many changes to the Solaris environment, included the latest version of the X Window System and began using Display PostScript for some of its graphics subsystems. Solaris 2.3 was also POSIX compliant. Solaris 2.4 was released in 1994; it included support for Motif. Solaris 2.5 was released in 1995 and included many new features such as the Common Desktop Environment (CDE), POSIX Threads, and NFS over TCP.

Solaris 2.6, the first version of Solaris to add support for Java, was released in late 1997. Solaris 2.6 also conformed to the UNIX 95 standard from X/OPEN and contained Y2K fixes.

Solaris 7 (the designation 2.7 was dropped in favor of simply 7) was released in 1998; it included many new features for improved usability and reliability. Some of the improvements are support for 64-bit applications and web-based administration and configuration.

Solaris 8 was introduced in 2000. It includes many performance and administration enhancements, including a network cache accelerator for serving web pages, support for clustering of processors, automatic dynamic reconfiguration for reallocating system resources, hot-patching capabilities, live updating for the operating system, and centralized management capabilities. It also provides security enhancements, including a built-in firewall, support for Kerberos, role-based access control, and support for IPsec (IP security) which enables secure, authenticated connections over the Internet. It introduced support for IP version 6, integrated into its NFS/RPC (Network File System/Remote Procedure Call) and NIS/NIS+ (Network Information Services). Solaris 8 also includes the StarOffice 5.1 Productivity Suite, which provides a word processor, a spreadsheet program, a presentation program, a database program, and so on. With this release, Sun began offering its Solaris software free of charge.

Solaris 9 was released in 2002; it includes enhancements to system administration that gave administrators the ability to allocate resources on a system, to monitor usage of resources, and to generate accounting information about system usage. It introduced a new graphical user interface, called Web Start, for system administrations. Solaris 9 also supports the Secure Shell, used for secure connections. Solaris 9 also supports a new fixed-priority scheduling class for processes and a directory server for enterprise-wide users and resources.

In 2005 Sun released its most recent version of Solaris, Solaris 10. Solaris 10 on Sparc-based systems has been registered as a certified UNIX 03 product by the Open Group. Furthermore, to counter the popularity of Linux, Sun has engineered Solaris 10 for use on Intel x86 and AMD 64-based systems. Sun has introduced performance enhancements for these lower-end platforms. For server and for enterprise use, Solaris 10 provides enhancements to resource management. Limits can now be placed on resource use by applications so that systems are not overwhelmed by out-of-control applications. It allows systems to be logically partitioned into zones, each with its own specific functionality using NI containers. Solaris 10 also supports authentication using smart cards. Binary compatibility between Solaris 10 and Linux has been introduced. In 2005, Sun also released OpenSolaris, an open-source version of Solaris, so that outside contributions could help Solaris evolve.

Consult the Sun Microsystems web site, http://www.sun.com/solaris/, for more information about Solaris. For information on obtaining Solaris free of charge, go to http://www.sun.com/software/solaris/get.jsp.Additional information about OpenSolaris can be found at http://www.opensolaris.org.

MAC OS X

The Mac OS, the operating system developed by Apple Computer for its Macintosh computers, was first developed in 1984. The original versions of this operating system were very different from other operating systems, including UNIX. In particular, the Mac OS had an entirely graphical user interface with no command-line interface. However, the original Mac OS hindered the development of more modern versions of the Mac OS. The original architecture of the Mac OS was used up until Mac OS 9. Apple Computer decided to build new versions of the Mac OS, beginning with Mac OS X, on a UNIX-like operating system. To accomplish this, they developed Darwin, first released in 2000, which is a free, open-source variant of UNIX and the core upon which Mac OS X is built. The kernel of Darwin, called XNU, is based on the kernels of FreeBSD 5 and Mach 3, developed at Carnegie Mellon University. As an aside, Apple Computer’s first variant of UNIX was A/UX (from Apple’s UNIX). A/UX 3 merged the functionality of the UNIX System with the Macintosh System 7 operating system. A/UX 3 was based on UNIX System V Release 2.2 but included many extensions from System V Releases 3 and 4 and from 4.2BSD and 4.3BSD.

The initial release of Mac OS X, Version 10.0, called Cheetah, was introduced in early 2001. (Versions of Mac OS X are named after big cats.) This release was incomplete and slow, and few applications ran on it. However, it was a release upon which future versions could be built. Version 10.1, called Puma, was released in late 2001, which improved system performance and provided missing features. Mac OS X version 10.2, called Jaguar, was introduced in 2002. Jaguar was considered to be the first solid release of Mac OS X; it provides performance enhancements, an improved user interface, and over 150 separate enhancements. The next release, Mac OS X version 10.3, called Panther, was introduced in 2003. Panther provided further performance enhancements, an extensive update to the user interface, and greater interoperability with Microsoft Windows. Mac OS version 10.4, called Tiger, was introduced in 2005. Among the new features introduced in Jaguar are Spotlight, a fast content and metadata-based file search tool, and support for 64-bit platforms and Intel x86 platforms. Mac OS Version 10.5, named Leopard, will be released in early 2007.

Although Mac OS X is not open source, Darwin, the operating system upon which it is built, is open source. Furthermore, in 2002, Apple and the Internet Systems Corporation founded OpenDarwin, a community set up to enable the cooperative development of new versions of Darwin. OpenDarwin develops new releases of the Darwin operating system. This group also offers DarwinPorts, which provides an easy way to install various open-source software on versions of Darwin and Mac OS X systems. For more about OpenDarwin, go to http://www.opendarwin.org/, and for more on DarwinPorts, go to http://darwinports.opendarwin.org/.

AIX

IBM’s version of UNIX is called AIX (short for Advanced Interactive eXchangeI) and is primarily developed for use on IBM workstations. IBM has invested billions of dollars in the development of its UNIX servers, both for hardware development and development of AIX. The fruits of this investment can be seen in the increasing power and added capabilities of AIX that make AIX an extremely competitive version of UNIX for servers.

AIX Version 1 was released in 1986 and was based on UNIX System V Release 3. In subsequent releases, source code from BSD 4.2 and BSD 4.3 was introduced into AIX. Version 2 was released in 1987. AIX Version 3 was released in 1990 as a developer release licensed only to OSF. Release 1 of AIX Version 3 introduced the Journaled File System (JFS).

Version 4 of AIX, denoted by AIX 4, was introduced in 1994. In 1995, the CDE desktop environment replaced the Motif X Window Manager in AIX 4. Support for 64-bit architectures was introduced in AIX 4.3 in 1997. AIX 4.3 was registered with the UNIX 98 mark by the Open Group and conforms with the POSIX 1 and POSIX 2 standards.

The latest version of AIX is AIX 5L, released in 2001. The letter L in AIX 5L indicates a strong affinity of this operating system to Linux; AIX 5L incorporates libraries of Linux routines and application programming interfaces that enable almost all Linux applications to run on AIX 5L. The current release, AIX 5L Version 5.3, supports as many as 64 central processing units and a total of two terabytes of RAM. The JFS2 file system has been introduced to AIX 5L. It supports files and partitions as large as 16 terabytes. Many other enhancements have been made to AIX in AIX 4 and AIX 5L, especially in the areas of scalability, security, performance, server capabilities, networking, and administration. Some versions of AIX 5L are UNIX 03-registered products. For more information about AIX, including features, consult the following page on the IBM web site: http://www.austin.ibm.com/software/aix_os.html.

HP-UX

The variant of the UNIX operating system developed and sold by Hewlett-Packard for use on its computers and workstations is called HP-UX. The first version of HP-UX was introduced in 1986. HP-UX was originally based on UNIX System V Release 2.0, but many enhancements have been introduced through the years. Significant advances were made with the introduction of HP-UX 9.0 in 1992, which provided support for workstations. HP-UX 9.0 met many standards, including POSIX 1003.1 and 1003.2, XPG4, and the SVID 2 and 3. It incorporated many features of 4.3BSD and a graphical user interface, called the Visual User Environment (VUE). In 1995 HP-UX 10.0 was introduced, providing enhancements in networking, system management, security, and many other areas. It incorporated the SVR4 File System Directory Layout structure. HP-UX 10.0 added conformance to the Single UNIX Specification and POSIX 1003.1b (Real Time Standard). Furthermore, HP-UX 10.0 included support for the Common Desktop Environment (CDE). It also met the C2 level of security (controlled access protection) specified by the National Computer Security Center.

HP-UX 11.0, released in 1997, provides a 64-bit operating environment and includes many features needed for servers running mission-critical applications, as well as many new features for workstations, including increasing networking and 3-D graphics support. Of the many subsequent substantial releases, HP-UX 11.11, released in 2000, is the most noteworthy. This release, also known as HP 11 i, introduced the notion of operating environments, which are bundled groups of layered applications designed for specific types of use. Available types include Foundation (designed for use by web servers and content servers), Enterprise (designed for use by database servers), Mission Critical (designed for use for back-end application servers and transaction processing), Minimal Technical (designed for use on general-purpose workstations), and Technical Computing (designed for use on compute-intensive workstations).

You can obtain more information about HP-UX at the HP web site; start with the page at http://www.hp.com/unixwork/.

UNIXWARE

The Santa Cruz Operation originally based its operating systems on UNIX System V/386 Release 3.2, a version of UNIX System V Release 3 designed for use on Intel 80386 processors. SCO has evolved this original version of UNIX into a family of operating system in its OpenServer product line. The Santa Cruz Operation also offered UnixWare, a UNIX variant jointly developed by the AT&T UNIX Systems Laboratory and Novell, following the sale of all UnixWare products by Novell to SCO. UnixWare 2, based on an integration of UNIX System V Release 4.2 and Novell NetWare, which supports client/server computing, was released in 1995.

The Santa Cruz Operation, as the owner of UNIX System V, developed System V Release 5, concentrating on further developing the technology of the UNIX kernel. The SVR5 kernel was optimized for large-scale server applications. Among the areas of improvement in SVR5 were system performance, system capacity and scalability, and reliability and availability. Performance gains resulted from improved process synchronization, scheduling, and memory management. System capacity and enhanced scalability result from support of up to 64 GB of main memory, up to 1TB file and file systems, and 512 logical disks. The higher availability and reliability result from support for server clustering and built-in device fail-over capabilities. SVR5 also provides support for 64-bit file systems and implements 64-bit commands, libraries, and APIs.

The Santa Cruz Operation based its subsequent UnixWare products on the System V Release 5 kernel. Their latest release of UnixWare was UnixWare 7. Because it is based on the SVR5 kernel, UnixWare 7 supports 64-bit files systems and operations and includes development tools that support 64-bit integer operations. UnixWare 7 includes the Common Desktop Environment (CDE). It also includes an integrated Netscape browser and web server. It provides Java-based administration and support with a web interface and access and management of applications over a network. UnixWare 7 also includes support for Java. The Santa Cruz Operation also evolved the original version of UNIX into a family of operating systems in its OpenServer product line.

In 2000 the Santa Cruz Operation sold the rights to UnixWare to Caldera Systems. Caldera later changed its name to the SCO Group. (The Santa Cruz Operation was originally known as SCO; they changed their name to Tarantella when they sold UnixWare and the part of their company dealing with operating systems to Caldera.) The SCO Group has continued to develop further releases of UnixWare; the latest release is UnixWare 7.1.4. New features in this release include added security functionality, including support for IPsec and support for OpenSSH and OpenSSL, and advanced hyperthreading capabilities. Also, the SCO Group has continued to evolve the OpenServer product line; in 2005, the SCO Group released SCO OpenServer 6, which is bundled with many open-source applications, including Apache, Samba, MySQL, OpenSSH, Firefox, and KDE. OpenServer 6 provides many improvements over OpenServer 5, including vastly improved SMP support, with support for as many as 32 x86-family processors on a single server and support for files larger than one terabyte on a partition.

Go to http://www.sco.com/products/unix/ for more information about OpenServer and UnixWare operating systems.

Tru64 UNIX

For many years the Digital Equipment Corporation (DEC) sold computers running their version of UNIX, which was called ULTRIX and was based on 4.2BSD. Later, with the advent of their Alpha processor-based computers, they focused on a different UNIX variant, DEC OSF/1, based on the OSF/1 operating system developed by the Open Systems Foundation. DEC OSF/1 included extensive enhancements beyond what is included in OSF/1. In particular, it provided 64-bit support, real-time support, enhanced memory management, symmetric multiprocessing, and a fast-recovery file system. DEC OSF/1 integrated OSF/1, System V, and BSD components, ran under a Mach kernel, and provided backward compatibility for ULTRIX applications. DEC OSF/1 was compliant with Spec 1170 (except for curses support) and with POSIX 1003.1, POSIX 1003.2, and X/Open XPG4. DEC OSF/1 was renamed Digital UNIX.

In January 1998, Compaq Computer Corporation purchased DEC and continued the development of Digital UNIX. They have renamed Digital UNIX, giving it the new name Tru64 UNIX, highlighting that it is a 64-bit operating system that can take advantage of 64-bit hardware. This UNIX variant includes a wide range of features designed to support highly reliable networked applications running on servers. In 2002, HP purchased Compaq. HP announced its intention to migrate many of Tru64 UNIX’s more unique features to HP-UX. The current release is Tru64 UNIX 5.1; HP has committed to support this operating system until at least 2011.

For more information about Tru64 UNIX, consult the HP Tru64 web pages at http://h30097.www3.hp.com/.

IRIX

IRIX is a proprietary version of UNIX System V Release 4 provided by Silicon Graphics for use on its MIPS-based workstations. IRIX is a 64-bit operating system, which is one of its features that optimize its performance for graphics applications requiring intensive CPU processing. The current release of IRIX, IRIX 6.5, offers scalability, large-scale data management, real-time 3-D visualization capability, and middleware platforms. IRIX has been designed so that it provides functionality in many areas, including server support, applications launching, and digital media support. IRIX is compliant with many UNIX standards. Consult the Silicon Graphics web site, http://www.sgi.com/, for more information about IRIX.

A UNIX System Timeline

The following timeline summarizes the development of UNIX from its beginning to 2006. For an incredibly detailed timeline of releases of different UNIX variants, go to http://www.levenez.com/unix/.

UNIX Contributors

The following table summarizes important contributors to evolution of UNIX:

The UNIX System and Microsoft Windows NT Versions

Microsoft’s Windows NT operating system has been positioned as an alternative to UNIX, particularly in the server and network operating system arenas. However, it fails to equal UNIX in many areas, including adaptability, efficient use of resources, and reliability Also, as a proprietary system (unlike open-source versions of UNIX such as Linux and FreeBSD) it lacks the flexibility and readiness to incorporate new features that UNIX offers, as you will learn in this section.

Windows NT

Windows NT is a multitasking operating system designed by Microsoft to have many of the features of UNIX and other advanced capabilities not found in Microsoft Windows. Microsoft began work on NT in 1988, when it hired one of the leaders in the development of the Digital VMS operating system, David Cutler, to head this project. Windows NT was designed to compete with UNIX as the operating system for servers. Early versions of Windows NT had many problems, including a large number of bugs, poor performance, problems with memory, and a lack of application software. Release 3.5 of Windows NT eliminated many of the problems of earlier releases. Since then, many different releases of Windows NT have been introduced. In particular, Windows XP marketed by Microsoft as a desktop operating system, is really just a version of Windows NT, and was referred internally at Microsoft as Windows NT 5.1. Similarly, Windows Server 2003 is referred to internally at Microsoft as Windows NT 5.2.

Windows NT accomplished POSIX compliance using what Microsoft calls an environment subsystem. An environment subsystem is a protected subsystem of NT running in a nonprivileged processor mode that provides an application programming interface specific to an operating system. Besides the POSIX environment subsystem, Windows NT has Win32, 16-bit Windows, MS-DOS, and OS/2 environment subsystems that allow Windows, DOS, and OS/2 programs to run under Windows NT. Reviewers of NT have found many deficiencies in the Windows NT POSIX environment subsystem.

Differences Between Windows NT and UNIX

Windows NT was designed to share many of the features of UNIX, but there are many substantial differences. UNIX is a case-sensitive operating system, whereas NT often ignores case. This can cause problems, since a user may really want to have two files in the same directory that differ only by the cases of their names (such as DRA and Dra). Both Windows NT and UNIX System V are multitasking operating systems. However, Windows NT supports only one user at a time (although applications on servers may allow concurrent use by multiple users even though the operating system only deals with one user at a time), whereas the UNIX System can support many simultaneous users. There is only one Windows NT, controlled by Microsoft, but there are many versions of UNIX, but with standardization efforts, different versions of UNIX share features and interfaces. For example, both Windows NT and UNIX support a user-friendly graphical user interface. With the standardization and adoption of the CDE by essentially all UNIX vendors, the graphical user interface for UNIX is compatible across different UNIX variants.

One of the major advantages of UNIX is its capability to be adapted to new hardware. For example, Windows NT is a 32-bit or 64-bit operating system, whereas most versions of UNIX are now 64-bit operating systems, with 128-bit versions now available. This allows UNIX to support complex computing applications that require a large address space, such as applications that arise in DNA research, and to take advantage of the performance gains produced when 128-bit processors are used. There is a fundamental difference in the system design of UNIX and NT. Windows is an event-driven operating system, whereas UNIX is a process-driven operating system.

You can run Windows programs using either Windows NT or a version of UNIX with a Windows emulation package. Windows NT is only partially compliant with POSIX standards, by contrast with the most widely used variants of UNIX. Windows NT complies with the POSIX 1003.1 specification, but only within its POSIX environment subsystem. Windows applications are not POSIX compliant. On the other hand, many widely used UNIX variants are POSIX 1003.1 compliant. Unlike many versions of UNIX, Windows NT is not compliant with the POSIX.2 specification that defines command processor and command interfaces for standard applications. NT also does not comply to the POSIX.4 specification for a threads interface.

Windows NT runs on a limited set of processors. UNIX, on the other hand, runs on just about every processor in use today. Windows NT requires 12 MB of memory to run on a computer, whereas UNIX requires much less memory, with some versions requiring as little as 2 MB. One reason for this is that variants of UNIX can be run without a graphical user interface (GUI), unlike Windows.

Comparing UNIX and NT for Servers

The Microsoft Corporation has been developing Windows NT to compete head to head with UNIX for use on servers. The vast marketing effort undertaken by Microsoft has made inroads in this market, and Windows NT has become suitable for some, but not all, server applications. However, UNIX is continuing to evolve more quickly than Windows, primarily because of its openness and the large community of developers working on UNIX.

Many differences distinguish Windows NT and UNIX in the server area. UNIX is considered much more scalable than Windows NT for large applications, such as those that use extremely large databases, with systems that use as many as 128 processors. Windows NT is limited to 32 processors and two gigabytes of addressable memory on all the architectures it supports; the same is not true of UNIX. UNIX-based systems have run more than 100,000 transactions per minute.

Reliability is another important area where UNIX outshines Windows NT. Several UNIX vendors have developed sophisticated clustering capabilities that permit a large number of UNIX systems to run as a single unit. Windows NT does not support this capability for more than two systems. Load balancing across machines in a cluster is another area in which UNIX has outpaced Microsoft’s NT operating system.

Considerable cost savings can also result from the use of open-source variants of UNIX instead of Windows NT, especially when it is possible to clone servers. With Windows NT, additional costs are incurred for each server closed. Furthermore, you can obtain open-source application software for variants of UNIX, such as web server software, mail server software, database software, and integrated office software. Although analogous open-source software is also available for Windows, it often does not run as well on Windows platforms. To obtain application software of similar quality for Windows NT, you would need to spend a considerable amount of money.

How the Evolution of UNIX Differs from That of NT

Unlike UNIX, NT is not an open operating system. You cannot gain access to the source code for NT as you can for many important variants of UNIX. Source code for these variants of UNIX is readily available, either free of charge or for a fee from a vendor. NT is also a proprietary operating system, so that Microsoft controls its evolution. Some versions of UNIX are proprietary, but others are not, and no one can control the evolution of UNIX (although such people as Linus Torvalds can influence it). The openness and lack of central control makes it possible for UNIX to evolve as people develop new features, which may find their way into future versions. The only way for NT to evolve is for Microsoft to develop enhancements-and this is a severe limitation, even with the large technical staff employed by Microsoft.

The Future of UNIX

The UNIX System continues to evolve. An abiding virtue of UNIX is its capability to grow and incorporate new features as technology progresses. Undoubtedly, many new features, tools, utilities, and networking capabilities will be developed in the next few years. New capabilities are continually being developed as communities of developers add features and capabilities to Linux and other UNIX variants, including FreeBSD, NetBSD, and OpenBSD, Darwin, and OpenSolaris. Many developers will continue to volunteer their efforts to create enhancements to UNIX that can be used free of charge. Concurrently, vendors who want to offer the most robust version of UNIX for particular types of applications will continue to develop new features for their proprietary versions of UNIX, including IBM, HP, Sun, Apple, and the SCO Group, with emphasis from IBM, HP, and Sun on increasing the capabilities of UNIX for server and enterprise applications, and Sun and Apple furthering the utility of UNIX on the desktop.

The unification of UNIX that began with the development of UNIX SVR4 has been furthered by the Single UNIX Specification from the Open Group. After wide testing and use, some of the features introduced in different UNIX variants will find their way into later versions of the Single UNIX Specification.

The vast number of creative people working on new capabilities for UNIX assures that it has an interesting and exciting future. There will also probably be many different variants of UNIX, especially those with community of developers and offered free of charge. However, the number of different variants offered by large computer companies will probably decrease as these vendors either work together to unify versions of UNIX or adopt an open-source variant. More UNIX variants will develop to meet specific application and platform needs. Although these different versions of UNIX will generally conform to some base set of standards, such as the Single UNIX Specification, each will contain its own unique set of enhancements. More and more applications will run on an ever-wider range of UNIX platforms through porting collections, binary compatibility and the use of the APIs described in the Single UNIX Specification.

Some people believe that UNIX variants will be increasingly used for desktop computing, as well as portable computing. The pace at which this happens depends on the development of a robust collection of easy-to-use applications that run on UNIX variants, analogous or identical to those running on Windows. UNIX, as implemented in the many variants of UNIX, including those called Linux, will thrive as the operating system of choice for demanding applications on servers, especially for networked environments. It will also be adapted for new hardware platforms of all types. In both of these areas, it will most likely outpace proprietary offerings, including those from Microsoft. The future development of the UNIX System will also be furthered by collaboration over the Internet, and the Internet itself will benefit from new features of UNIX that have been developed to enable networked applications. Finally, UNIX will continue to be used for enterprise and transaction-intensive applications, as vendors ensure that their UNIX platforms meet the needs of these computing intensive applications.

Choosing a UNIX Variant

As you have seen, there are a multitude of UNIX variants. Picking a variant that meets your needs depends on how you plan to use UNIX. For example, you may want to run UNIX on your desktop or your laptop computer. If you want to use UNIX this way, you have many choices. You can buy a computer with a UNIX variant already installed, such as a Macintosh computer with Mac OS X or a computer with Linux, FreeBSD, or some other UNIX variant already preinstalled. If you are a more experienced user and would like to configure your own machine, you can select a free or low-cost UNIX variant from the many available choices, including a large collection of different Linux distributions, FreeBSD, OpenBSD, NetBSD, Solaris, and many other UNIX variants. Although you can get many of these variants free via Internet download, you may prefer to purchase a supported version of one of these variants. Instead of downloading such a variant, you will be provided with media containing the operating system. Before selecting a UNIX variant, you should examine how well each of these variants meets your particular computing needs. You should also research how easy it is to install each variant on the hardware platform you have. Many people relate their experiences and problems installing and running different UNIX variants on web sites that you can find using a search engine. You also need to consider the software you want to run on your machine on top of your UNIX variant. For each UNIX variant, there are thousands of software programs that have been ported to run on it. However, you should make sure that the particular software programs you would like to run have been ported to the variant of UNIX you are considering, or that they are already included in your distribution.

Choosing a UNIX variant to run a low-end server, such as a web server, involves some of the same considerations as choosing a UNIX variant to run on the desktop. If you want to run Linux on your server, you should select a Linux variant that includes a full suite of system administration capabilities and strong support for security, such as Red Hat, Slackware, or Debian. Among the most common BSD variants, many people consider FreeBSD to be an excellent choice for running a variety of server applications, including a web server, a file server, or a mail server. OpenBSD is considered to be the best choice for security applications, including running a firewall or an authentication server. NetBSD is considered the best choice for running servers on unusual machines, such as computers salvaged from other uses.

Different considerations apply when choosing a UNIX variant for running a high-end server, an enterprise or mission-critical application, or computing-intensive applications. For such uses, you could examine supported UNIX variants that include additional capabilities to ensure reliability and availability, excellent performance, scalability, supportability, interoperability, adequate security, and other features important for this type of computing. You should examine the proprietary UNIX variants that major computer companies such as HP, IBM, and Sun Microsystems offer, as well as their supported Linux distributions, which include add-ons needed for enterprise applications.

Summary

You have learned about the structure and components of UNIX. You will find this background information useful as you move on to Chapters 2, 3, and 4, where you will learn how to use the basic features and capabilities of UNIX, such as files and directories, basic commands, and the shell. This chapter has described the birth, history, and evolution of UNIX. In particular, you learned about the evolution of UNIX System V, the Berkeley Software Distribution, GNU, and Linux. Then you became acquainted with the modern history of UNIX, including descriptions of the important standards in the UNIX world. This chapter then covered the origins and features of many important UNIX variants, including Linux, Solaris, FreeBSD, AIX, Mac OS X, HP-UX, and others. The chapter also compared and contrasted the UNIX System and Windows NT. (Chapter 18 will tell you how to use UNIX and Windows together.) Finally, this chapter briefly explored the possible future of UNIX and proffered advice on selecting a UNIX variant that can meet your needs.

How to Find Out More

You can learn more about the history and evolution of UNIX by consulting these books:

Dunphy, Ed. The UNIX Industry and Open Systems in Transition. 2nd ed. New York: Wiley, 1994.

Libes, Don, and Sandy Ressler. Life with UNIX. Englewood Cliffs, NJ: Prentice Hall, 1989.

Lucent Technologies. “The Creation of the UNIX Operating System.” http://www.bell-labs.com/history/unix/

Ritchie, D.M. “The Evolution of the UNIX Time-Sharing System.” AT&T Bell Laboratories Technical Journal, vol. 63, no. 8, part 2, October 1984.

Salus, Peter. A Quarter Century of UNIX. Reading, Massachusetts, 1994.

The UNIX System Oral History Project. Edited and transcribed by Michael S. Mahoney. AT&T Bell Laboratories.