Command Line Kung Fu (2014)
Shell Scripting
Use a for Loop at the Command Line
$ for VAR in LIST
> do
> # use $VAR
> done
When you need to perform the same action for a list of items, you can use a for loop right from your shell.
$ for USER in bob jill fred
> do
> sudo passwd -l $USER
> logger -t naughty-user $USER
> done
Locking password for user bob.
passwd: Success
Locking password for user jill.
passwd: Success
Locking password for user fred.
passwd: Success
$ sudo tail -3 /var/log/messages
Apr 8 19:29:03 linuxserver naughty-user: bob
Apr 8 19:29:03 linuxserver naughty-user: jill
Apr 8 19:29:03 linuxserver naughty-user: fred
You can also type entire loop on one command line
$ for USER in bob jill fred; do sudo passwd -l $USER; logger -t naughty-user $USER; done
...
Command Substitution
$ VAR=`command`
$ VAR=$(command)
There are two forms of command substitution. The first form uses backticks (`) to surround a command while the second form uses a dollar sign followed by parenthesis that surround a command. They are functionally equivalent with the backtick form being the older style. The output of the command can be used as an argument to another command, to set a variable, or for generating the argument list for a for loop.
$ EXT_FILESYSTEMS=$(grep ext fstab | awk '{print $2}')
$ echo $EXT_FILESYSTEMS
/ /boot
$ cp file.txt file.txt.`date +%F`
$ ls file.txt*
file.txt file.txt.2014-04-08
$ ps -fp $(cat /var/run/ntpd.pid)
UID PID PPID C STIME TTY TIME CMD
ntp 1210 1 0 Apr06 ? 00:00:05 ntpd -u ntp:ntp -p /var/run/ntpd
$ sudo kill -9 $(cat /var/run/ntpd.pid)
$ for x in $(cut -d: -f1 /etc/passwd); do groups $x; done
jason : jason sales
bobdjr : sales
jim : jim
Store Command Line Output as a Variable to Use Later
$ for VAR in LIST
> do
> VAR2=$(command)
> VAR3=$(command)
> echo "$VAR2 VAR3"
> done
Command substitution can be used to assign values to variables. If you need to reuse the output of a command multiple times, assign it to a variable once and reuse the variable. This example shows how the output of the id command is used multiple times in one script.
$ for USER in $(cut -f1 -d: /etc/passwd)
> do
> UID_MIN=$(grep ^UID_MIN /etc/login.defs | awk '{print $NF}')
> USERID=$(id -u $USER)
> [ $USERID -lt $UID_MIN ] || {
> echo "Forcing password expiration for $USER with UID of $USERID."
> sudo passwd -e $USER
> }
> done
Forcing password expiration for bob with UID of 1000.
Forcing password expiration for bobdjr with UID of 1001.
Forcing password expiration for bobh with UID of 1002.
Read in Input One Line at a Time
$ while read LINE
> do
> # Do something with $LINE
> done < file.txt
$ command | while read LINE
> do
> # Do something with $LINE
> done
If you want to iterate over a list of words, use a for loop. If you want to iterate over a line, use a while loop in combination with a read statement and redirection.
Let's look for file systems that are over 90% utilized. If we try to use an if statement it will break up the output into word chunks like this.
$ df | head -1
Filesystem 1K-blocks Used Available Use% Mounted on
$ for x in $(df)
> do
> echo $x
> done
Filesystem
1K-blocks
Used
Available
Use%
Mounted
on
...
We need to read in entire lines at a time like this.
$ df | while read LINE
> do
> echo $LINE
> done
Filesystem 1K-blocks Used Available Use% Mounted on
...
Here is one way to find file systems that are over 90% utilized.
$ df
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/sda2 28891260 3270340 25327536 12% /
tmpfs 515320 72 515248 1% /dev/shm
/dev/sda1 495844 453683 16561 97% /boot
$ df | grep [0-9]% | while read LINE
> do
> use=$(echo $LINE | awk '{print $5}' | tr -d '%')
> mountpoint=$(echo $LINE | awk '{print $6}')
> [ $use -gt 90 ] && echo "$mountpoint is over 90% utilized."
> done
/boot is over 90% utilized.
$
Instead of assigning variables within the while loop, you can assign them with the read statement. Here is how this method looks.
$ df | grep [0-9]% | while read fs blocks used available use mountpoint
> do
> use=$(echo $use | tr -d '%')
> [ $use -gt 90 ] && echo "$mountpoint is over 90% utilized."
> done
/boot is over 90% utilized.
Accept User Input and Store It in a Variable
$ read VAR
$ read -n 1 VAR
$ read -p "Prompt text" VAR
To accept user input from a user, use the read command. Read will accept an entire line of input and store it into a variable. You can force read to only read a limited number of characters by using the -n option. Instead of using echo statements before a read command, you can supply a prompt by using the -p option. Here is a sample script that uses these techniques.
The contents of backup.sh:
#!/bin/bash
while true
do
read -p "What server would you like to backup? " SERVER
echo "Backing up $SERVER"
/usr/local/bin/backup $SERVER
read -p "Backup another server? (y/n) " -n 1 BACKUP_AGAIN
echo
[ "$BACKUP_AGAIN" = "y" ] || break
done
$ ./backup.sh
What server would you like to backup? thor
Backing up thor
Backup another server? (y/n) y
What server would you like to backup? loki
Backing up loki
Backup another server? (y/n) n
$
Sum All the Numbers in a given Column of a Text
$ awk '{ sum += $1 } END { print sum }' file
$ cat file | awk '{ sum += $1 } END { print sum }
Awk can be used to tally up a column of values. You can use this trick to add up all the disk space used across all the file systems on a given system, for example.
$ df -mt ext4
Filesystem 1M-blocks Used Available Use% Mounted on
/dev/mapper/vg_root-lv_root 28215 3285 24644 12% /
/dev/sda1 485 55 406 12% /boot
$ df -mt ext4 | awk '{ sum += $3 } END {print sum}'
3340
$ sudo dmidecode --type memory
Size: No Module Installed
Size: 4096 MB
Size: No Module Installed
Size: 4096 MB
$ sudo dmidecode --type memory | grep 'Size:' | awk '{sum+=$2} END {print sum}'
8192
$
Automatically Answer Yes to Any Command
$ yes | command
$ yes "string" | command
If you are trying to automate a process that requires user input, check out the yes command. By default yes simply prints out "y" until it is killed. You can make yes repeat any string. If you wanted to automatically answer "no" you could run "yes no."
$ ./install-my-app.sh
Are you sure you want to install my-app? (y/n) y
Ok, my-app installed.
$ yes | ./install-my-app.sh
Ok, my-app installed.
$