Linux Administration (2016)

Shell Scripting

A script is a command line program that contains a series of commands. The commands contained in the script are executed by an interpreter. In the case of shell scripts, the shell acts as the interpreter and executes the commands listed in the script one after the other.

Anything you can execute at the command line, you can put into a shell script. Shell scripts are great at automating tasks. If you find yourself running a series of commands to accomplish a given task, and know you will need to perform that task again in the future, you can—and probably should—create a shell script for that task.

Let's look at a simple shell script. The name of this script is script1.sh.

#!/bin/bash

echo "Scripting is fun!"

Before you try to execute the script, make sure that it is executable.

$ chmod 755 script1.sh

Here is what happens when you execute the script.

$ ./script1.sh

Scripting is fun!

The Shebang

You'll notice that the first line of the script starts with #! followed by the path to the bash shell program, /bin/bash. The number sign is very similar to the sharp sign used in music notation. Also, some people refer to the exclamation mark as a "bang." So, #! can be spoken as "sharp bang." The term Shebang is an inexact contraction of "sharp bang."

When a script's first line starts with a shebang, what follows is used as the interpreter for the commands listed in the script. Here are three examples of shell scripts, each using a different shell program as the interpreter.

#!/bin/csh

echo "This script uses csh as the interpreter."

#!/bin/ksh

echo "This script uses ksh as the interpreter."

#!/bin/zsh

echo "This script uses zsh as the interpreter."

When you execute a script that contains a shebang, what actually happens is that the interpreter is executed and the path used to call the script is passed as an argument to the interpreter. You can prove this by examinging the process table.

Let's start this script, sleepy.sh, in the background and look at the process table.

The contents of sleepy.sh:

#!/bin/bash

sleep 90

Let's execute it in the background and take a look at the processes.

$ ./sleepy.sh &

[1] 16796

$ ps -fp 16796

UID PID PPID C STIME TTY TIME CMD

jason 16796 16725 0 22:50 pts/0 00:00:00 /bin/bash ./sleepy.sh

You can see that what is actually running is /bin/bash ./sleepy.sh. Let's use a full path to the script.

$ /tmp/sleepy.sh &

[1] 16804

$ ps -fp 16804

UID PID PPID C STIME TTY TIME CMD

jason 16804 16725 0 22:51 pts/0 00:00:00 /bin/bash /tmp/sleepy.sh

Sure enough, /bin/bash /tmp/sleepy.sh is being executed. Also, you can see that /bin/bash is executing the sleep command, which is the first and only command command in the shell script.

$ ps -ef| grep 16804 | grep -v grep

jason 16804 16725 0 22:51 pts/0 00:00:00 /bin/bash /tmp/sleepy.sh

jason 16805 16804 0 22:51 pts/0 00:00:00 sleep 90

$ pstree –p 16804

sleepy.sh(16804)───sleep(16805)

If you do not supply a shebang and specify an interpreter on the first line of the script, the commands in the script will be executed using your current shell. Even though this can work just fine under many circumstances, it's best to be explicit and specify the exact interpreter to be used with the script. For example, there are features and syntax that work just fine with the bash shell that will not work with the csh shell.

Also, you don't have to use a shell as the interpreter for your scripts. Here is an example of a Python script named hi.py.

#!/usr/bin/python

print "This is a Python script."

Let's make it executable and run it.

$ chmod 755 hi.py

$ ./hi.py

This is a Python script.

For more information on python programming and scripting, see my book Python Programming for Beginners at http://www.linuxtrainingacademy.com/python-book.

Let's get back to shell scripting.

Variables

You can use variables in your shell scripts. Variables are simply storage locations that have a name. You can think of variables as name-value pairs. To assign a value to a variable, use the syntax VARIABLE_NAME="Value". Do not use spaces before or after the equals sign. Also, variables are case sensitive, and, by convention, variable names are in all uppercase.

#!/bin/bash

MY_SHELL="bash"

To use a variable, preceed the variable name with a dollar sign.

#!/bin/bash

MY_SHELL="bash"

echo "I like the $MY_SHELL shell."

You can also enclose the variable name in curly braces and preceed the opening brace with a dollar sign. Syntax: ${VARIABLE_NAME}.

#!/bin/bash

MY_SHELL="bash"

echo "I like the ${MY_SHELL} shell."

Here is the output of the script:

I like the bash shell.

The curly brace syntax is optional unless you need to immediately precede or follow the variable with additional data.

#!/bin/bash

MY_SHELL="bash"

echo "I am ${MY_SHELL}ing on my keyboard."

Output:

I am bashing on my keyboard.

If you do not encapsulate the variable name in curly braces, the shell will treat the additional text as part of the variable name. Since a variable with that name does not exist, nothing is put in it's place.

#!/bin/bash

MY_SHELL="bash"

echo "I am $MY_SHELLing on my keyboard."

Output:

I am on my keyboard.

You can also assign the output of a command to a variable. To do this, enclose the command in parentheses and precede it with a dollar sign.

#!/bin/bash

SERVER_NAME=$(hostname)

echo "You are running this script on ${SERVER_NAME}."

The output of the command hostname is stored in the variable SERVER_NAME. In this sample output, the server name is linuxsvr.

You are running this script on linuxsvr.

You can also enclose the command in back ticks. This is an older syntax that is being replaced by the $() syntax. However, you may see this in older scripts.

#!/bin/bash

SERVER_NAME=`hostname`

echo "You are running this script on ${SERVER_NAME}."

Valid variable names

Variable names can contain letters, digits, and underscores. They can start with letters or underscores, but cannot start with a digit. Here are examples of valid variable names.

FIRST3LETTERS="ABC"

FIRST_THREE_LETTERS="ABC"

firstThreeLetters="ABC"

Here are some examples of invalid variable names.

3LETTERS="ABC"

first-three-letters="ABC"

first@Three@Letters="ABC"

Tests

Scripts are designed to replace the need for a person to physically sit at a keyboard and type in a series of commands. What if you have a task you want to automate, but it requires different actions based on different circumstances? Since a person may not be around to make decisions when the script needs to run, we'll need to test for those conditions and have the script act accordingly.

To create a test, place a conditional expression between brackets. The syntax is: [ condition-to-test-for ]. You can test for several types of situations. For example, you can compare if strings are equal, if a number is greater than another one, or if a file exists. This test checks to see if/etc/passwd exists. If it does, it returns true. In other words, the command exits with a status of 0. If the file doesn't exist, it returns false. I.e., the command exits with a status of 1.

[ -e /etc/passwd ]

If you are using the bash shell, you can run the command help test to see the various types of tests you can perform. You can also read the man page for tests: man test. Here are of some of the more common tests you can perform.

File operators:

-d FILE True if file is a directory.

-e FILE True if file exists.

-f FILE True if file exists and is a regular file.

-r FILE True if file is readable by you.

-s FILE True if file exists and is not empty.

-w FILE True if the file is writable by you.

-x FILE True if the file is executable by you.

String operators:

-z STRING True if string is empty.

-n STRING True if string is not empty.

STRING True if string is not empty.

STRING1 = STRING2

True if the strings are equal.

STRING1 != STRING2

True if the strings are not equal.

Arithmetic operators:

arg1 –eq arg2 True if arg1 is equal to arg2.

arg1 –ne arg2 True if arg1 is not equal to arg2.

arg1 –lt arg2 True if arg1 is less than arg2.

arg1 –le arg2 True if arg1 is less than or equal to arg2.

arg1 –gt arg2 True if arg1 is greater than arg2.

arg1 –ge arg2 True if arg1 is greater than or equal to arg2.

The if Statement

Now that you know how to determine if a certain condition is true or not, you can combine that with the if statement to make decisions in your scripts.

The if statement starts with the word if and is then followed by a test. The following line contains the word then. Next is a series of commands that will be executed if the tested condition is true. Finally, the if statement ends with fi, which is if spelled backwards. Here is the syntax.

if [ condition-true ]

then

command 1

command 2

...

Here is an example:

#!/bin/bash

MY_SHELL="bash"

if [ "$MY_SHELL" = "bash" ]

then

echo "You seem to like the bash shell."

It is a best practice to enclose variables in quotes to prevent unexpected side effects when performing conditional tests. Here is the output of running the script:

You seem to like the bash shell.

You can also perform an action if the condition is not true by using an if/else statement. Here is what an if/else statement looks like.

if [ condition-true ]

then

command 1

command 2

...

else #

command 3

command 4

...

Let's update the script to perform an action if the statement is not true.

#!/bin/bash

MY_SHELL="csh"

if [ "$MY_SHELL" = "bash" ]

then

echo "You seem to like the bash shell."

else

echo "You don't seem to like the bash shell."

Here is the output. Because [ "$MY_SHELL" = "bash" ] evaluated as false, the statements following else were executed.

You don't seem to like the bash shell.

You can also test for multiple conditions using elif. The word elif is a contraction for "else if." Like if, follow elif with a condition to test for. On the following line, use the word then. Finally, provide a series of commands to execute if the condition evaluates as true.

if [ condition-true ]

then

command 1

command 2

...

elif [ condition-true ]

then

command 3

command 4

...

else #

command 5

command 6

...

Here is an updated script using elif:

#!/bin/bash

MY_SHELL="csh"

if [ "$MY_SHELL" = "bash" ]

then

echo "You seem to like the bash shell."

elif [ "$MY_SHELL" = "csh" ]

then

echo "You seem to like the csh shell."

else

echo "You don't seem to like the bash or csh shells."

Output:

You seem to like the csh shell.

The for Loop

If you want to perform an action on a list of items, use a for loop. The first line of a for loop starts with the word for followed by a variable name, followed by the word in and then a list of items. The next line contains the word do. Place the statements you want to execute on the following lines; then end the for loop with the word done on a single line.

for VARIABLE_NAME in ITEM_1 ITEM_2 ITEM_N

command 1

command 2

...

done

Esentially, what happens is that the first item in the list is assigned to the variable and the code block is executed. The next item in the list is then assigned to the variable and the commands are executed. This happens for each item in the list.

Here is a simple script that shows how a for loop works:

#!/bin/bash

for COLOR in red green blue

echo "COLOR: $COLOR"

done

Output:

COLOR: red

COLOR: green

COLOR: blue

It's common practice for the list of items to be stored in a variable as in this example.

#!/bin/bash

COLORS="red green blue"

for COLOR in $COLORS

echo "COLOR: $COLOR"

done

Output:

COLOR: red

COLOR: green

COLOR: blue

This shell script, rename-pics.sh, renames all of the files that end in jpg by prepending today’s date to the original file name.

#!/bin/bash

PICTURES=$(ls *jpg)

DATE=$(date +%F)

for PICTURE in $PICTURES

echo "Renaming ${PICTURE} to ${DATE}-${PICTURE}"

mv ${PICTURE} ${DATE}-${PICTURE}

done

Here's what happens when you run this script:

$ ls

bear.jpg man.jpg pig.jpg rename-pics.sh

$ ./rename-pics.sh

Renaming bear.jpg to 2015-03-06-bear.jpg

Renaming man.jpg to 2015-03-06-man.jpg

Renaming pig.jpg to 2015-03-06-pig.jpg

$ ls

2015-03-06-bear.jpg 2015-03-06-man.jpg 2015-03-06-pig.jpg rename-pics.sh

Positional Parameters

Positional parameters are variables that contain the contents of the command line. These variables are $0 through $9. The script itself is stored in $0, the first parameter in $1, the second in $2, and so on. Take this command line as an example:

$ script.sh parameter1 parameter2 parameter3

The contents of $0 is "script.sh", $1 is "parameter1", $2 is "parameter2", and $3 is "parameter3".

This script, archive_user.sh, accepts a parameter which is a username:

#!/bin/bash

echo "Executing script: $0"

echo "Archiving user: $1"

# Lock the account

passwd –l $1

# Create an archive of the home directory.

tar cf /archives/${1}.tar.gz /home/${1}

Comments

Anything that follows the pound sign is a comment. The only exception to this is the shebang on the first line. Everywhere else in the script, when a pound sign is encountered it marks the beginning of a comment. Comments are dutifully ignored by the interpreter as they are for the benefit of us humans.

Anything that follows the pound sign is ignored. If a pound sign starts at the beginning of a line the entire line is ignored. If a pound sign is encountered in the middle of a line, the information to the right of the pound sign is ignored.

Here is what the output looks like when we execute the archive_user.sh script:

$ ./archive_user.sh elvis

Executing script: ./archive_user.sh

Archiving user: elvis

passwd: password expiry information changed.

tar: Removing leading `/' from member names

Instead of referring to $1 throughout the script, let's assign its value to a more meaningful variable name.

#!/bin/bash

USER=$1 # The first parameter is the user.

echo "Executing script: $0"

echo "Archiving user: $USER"

# Lock the account

passwd –l $USER

# Create an archive of the home directory.

tar cf /archives/${USER}.tar.gz /home/${USER}

The output remains the same.

$ ./archive_user.sh elvis

Executing script: ./archive_user.sh

Archiving user: elvis

passwd: password expiry information changed.

tar: Removing leading `/' from member names

You can access all the positional parameters starting at $1 to the very last one on the command line by using the special variable $@. Here is how to update the archive_user.sh script to accept one or more parameters.

#!/bin/bash

echo "Executing script: $0"

for USER in $@

echo "Archiving user: $USER"

# Lock the account

passwd –l $USER

# Create an archive of the home directory.

tar cf /archives/${USER}.tar.gz /home/${USER}

done

Let's pass multiple users into the script.

$ ./archive_user.sh chet joe

Executing script: ./archive_user.sh

Archiving user: chet

passwd: password expiry information changed.

tar: Removing leading `/' from member names

Archiving user: joe

passwd: password expiry information changed.

tar: Removing leading `/' from member names

Getting User Input

If you want to accept standard input, use the read command. Remember that standard input typically comes from a person typing at the keyboard, but it can also come from other sources, like the output of a command in a command pipeline. The format for the read command is read -p "PROMPT" VARIABLE_NAME. This version of the archive_user.sh script asks for the user account.

#!/bin/bash

read –p "Enter a user name: " USER

echo "Archiving user: $USER"

# Lock the account

passwd –l $USER

# Create an archive of the home directory.

tar cf /archives/${USER}.tar.gz /home/${USER}

Let's run this script and archive the mitch account.

$ ./archive_user.sh

Enter a user name: mitch

Archiving user: mitch

passwd: password expiry information changed.

tar: Removing leading `/' from member names

Summary

The first line in a shell script should start with a shebang followed by the path to the interpreter that should be used to execute the script.

To assign a value to a variable, start with the variable name, followed by an equals sign, followed by the value. Do not use a space before or after the equals sign.

You can access the value stored in a variable by using $VARIABLE_NAME or ${VARIABLE_NAME}. The latter form is required if you want to precede or follow the variable with additional data.

To assign the output of a command to a variable, enclose the command in parentheses and precede it with a dollar sign. VARIABLE_NAME=$(command)

Perform tests by placing an expression in brackets. Tests are typically combined with if statements.

Use if, if/else, or if/elif/else statements to make decisions in your scripts.

To perform an action or series of actions on multiple items, use a for loop.

To access items on the command line, use positions parameters. The name of the program is represented by $0, the first parameter is represented by $1, and so on. To access all the items on the command line starting at the first parameter ($1), use the special variable $@.

You can place comments in your scripts by using the pound sign.

Accept user input by using the read command.

Quiz

1. The first line of a shell script typically starts with a shebang followed by the path to an interpreter that will be used to execute the commands in the script.

1. True

2. False

2. Which of the following variables is valid?

1. 3LETTERS="ABC"

2. first-three-letters="ABC"

3. first@Three@Letters="ABC"

4. FIRST3LETTERS="ABC"

3. What is the value of "$1" given the following command line:

$ ./add-user.sh tom richard harry

1. ./add-user.sh

2. tom

3. richard

4. harry

4. Which is the proper way to assign a value to a variable?

1. VAR="VALUE"

2. VAR = "VALUE"

5. Which of the following will assign the output of the hostname command to the variable HOSTNAME?

1. HOSTNAME=$(hostname)

2. HOSTNAME=`hostname`

3. All of the above.

Quiz Answers

1. A

2. D

3. B

4. A

5. C