CS330 Shell Scripts


Highlights of this lab:


Different Shells and Script Intro

There are several different shells, they offer their own advantages and disadvantages. For instance, some allow for auto completion using the tab key; others don't.

A few common shells are the following:

For more on shells, click here.

To see what shells exist on your current Unix system, try the following command:

$ cat /etc/shells

From the command line, you can type various commands which have the same responses in each of these shells. For instance chmod, ls, cd, and touch.

Let's say that you want to type the same commands over and over again. If that it the case, it is good to create a shell script.

For example, you could create a basic shell script by:

  1. Typing the following lines into a file called people.sh:
    ls -l
    mkdir tom
    cd tom
    touch greta monica george
  2. Making the file executable:
    chmod +x people.sh
  3. Executing the file at the command line:
    % people.sh
GOOD HABIT- To better identify your shell scripts from other files you can add .sh as an extension (however, it is not required)

Executing people.sh will do the following: first do a long listing of the current directory, then make a directory called tom, and create files: greta, monica, and george under the tom directory.

We can also add some comments to this file. Any line preceded with # is a comment (or anything after the # is ignored):

#list the files in the current directory
ls -l
 
#make a directory called tom
mkdir tom
 
#change into the tom directory and create three files.
cd tom
touch greta monica george

I've led you to believe that all shells are created equal. That is not the case. There is a major split between Bourne-compatible shells (such as sh and bash) and csh-compatible shells (such as csh and tcsh).

These differences show up in such things as conditional statements and loops. The following table shows the basic syntax for an if and while statement in both bash and tcsh

bash tcsh
if [expression]
then
    commands
fi
if (expression) then
    commands
endif 
while [expression]
do
   commands 
done
while (expression)
   commands
end

The difference is very subtle, but it is enough to create problems.

For the remainder of this lab, we will be focusing on bash shell scripts.


Starting off with a She-bang

In the first section, we stated that there are several shells and corresponding shell scripts. The question is: how do you indicate which shell environment you are using to execute the commands?

The answer is: there is a special notation used on the first line of the shell script. The line begins with a number sign (#), then an exclamation point (!), followed by the full path name of the shell on the computer's file system.

For instance, to state that you will interpret the above commands with the bash shell (on our Linux system), you would add the following to the beginning of the file:

#!/bin/bash
ls -l
mkdir tom
cd tom
touch greta monica george

The location of this bash shell is dependent on the system. You can verify the location by using:

$ cat /etc/shells

For instance, on hercules, you will use:

#!/usr/local/bin/bash 
(This also works for Linux)

Where did she-bang come from?

There are a couple of different stories:


Bash Variables

What would a program be without variables? Bash shells have their own way of handling variables.

Some rules about variables are the following:

There is a difference in shell scripts between the name of a variable and its value:

Assignment can be made in three major ways:

  1. with an = (myvar=2)
  2. with a read statement (read myvar)
  3. in a loop (for myvar in 1 2 3)

Note 1: When setting variables in bash shells, make sure that there are no spaces on either side of the equal sign.

Note2: $myvar is actually a simplified form of ${myvar}. Some situations may require the longer usage.

An example of assigning and using variables is the following code (echo allows us to print to the screen):

#!/usr/local/bin/bash
#file: variables.sh


var1="My string with spaces"
echo "var1 is: " $var1
echo


echo "Enter a number: "
read var2
echo "var2 is: " $var2
echo

for var3 in 1 2 3
do
        echo "var3 is: " $var3
done

A sample run would yield the following results:

% variables.sh
var1 is: My string with spaces
Enter a number:
30
var2 is: 30
var3 is: 1
var3 is: 2
var3 is: 3

Quotes

In the topic of quotes, we will discuss four special characters:

Double quotes do not interfere with variable substitution. They are sometimes referred to as weak quotes.

For instance:

var1=20
echo "This is $var1"

Will print: This is 20

Single quotes cause the variable name to be used literally. They prevent the shell from interpreting special characters. This is sometimes known as full quoting or strong quoting.

For instance:

var2=30
echo 'This is $var2'

Will print: This is $var2

The backslash is another way of preventing the shell from interpreting special characters. It only works on the character immediately following the backslash. We can also say that the backslash escapes the special meaning of the succeeding character.

For instance:

#Escape the normal meaning of the space character
var3=The\ price\ is--
 
#Escape the normal meaning of a $
var4=\$20.00
 
echo $var3 $var4

Will print : The price is-- $20.00

Note that without the escape from the space, the shell will try and interpret "price" and "is--" as commands.

Back quotes enable an expression to be evaluated as a command, and replaced with whatever the expression prints to its standard output. Back quotes are sometimes also known as back ticks.

For instance:

var5=`date`
echo "The date is $var5"

Will print: The date is Wed Nov 24 20:38:09 CST 2004

Note that on the keyboard, the (`) key is usually on the same key as the (~)

Conditionals

There are two conditional structures:

Before we talk about the if statement, we need to talk about the test command that is used to evaluate conditional expressions.

The basic format is:

test expression

or

[ expression ]

Where expression has built-in operators. The operators are classified into four different groups:

The following chart summarizes these operators:

Integer Comparisons
int1 -gt int2 Greater-than
int1 -lt int2 Less-than
int1 -ge int2 Greater-than-or-equal-to
int1 -le int2 Less-than-or-equal-to
int1 -eq int2 Equal
int1 -ne int2 Not-equal
String Comparisons
-z str Returns true if length of str is equal to zero
-n str Returns true if str length is greater than zero
str1 = str2 Equal strings
str1 != str2 Not-equal strings
str Returns true if str is not null
Logical Operations
expr1 && expr2 Logical AND (true if expr1 and expr2 are true)
expr1 || expr2 Logical OR (true if expr1 or expr2 are true)
! expr Logical NOT
File Tests
-f file File exists and is a regular file
-s file File is not empty
-r file File is readable
-w file File can be written to, modified
-x file File is executable
-d file Filename is a directory name
-h file Filename is a symbolic link
-c file Filename references a character device
-b file Filename references a block file

The If Statement. The basic format for the if statement is the following:

if condition ; then
	commands
[elif condition ; then
	commands] ...
[else
	commands]

fi

The statements shown in square brackets are optional. Notice that you end the if with a fi

The Case Statement. The basic format for the case statement is the following:

case expression in
	pattern1)
		commands;;
	pattern2)
		commands;;
	*)		
		commands;;
esac

Some sample code making use of if and case statements is the following

#!/usr/local/bin/bash
#file: conditions.sh

if [ -f ./conditions.sh ]; then
        echo "file exists"
fi

#notice this read statement includes the prompt
read -p "Enter a letter " var1

if test $var1 = 'a' ;  then
        echo "a selected"
fi

read -p "Enter a string " var2

case $var2 in
        hello)
        echo "hello found";;
        *bye)
        echo "bye found";;
        *)
        echo "nothing found";;
esac

read -p "Enter a number " var3

if [ $var3 -gt 0 ] && [ $var3 -lt 10 ] ; then
        echo "1 to 9"

elif [ $var3 -gt 9 ] && [ $var3 -lt 20 ] ; then
        echo "10 to 19"
		
elif [ $var3 -gt 19 ] || [ $var3 -eq 0 ] ; then
        echo "greater than 19 or equal to 0"

else
        echo "other range"

fi

A "condition" can also be any command. If a command returns a zero exit status, the condition is true; otherwise, the condition is false. You can write things like the following:

#!/usr/local/bin/bash
#file: conditions2.sh

user=temp0
if grep $user /etc/passwd; then
        echo "$user has an account"
else
        echo "$user doesn't have an account"
fi

Note: When using the square brackets, ensure that there is a space before and after the bracket.


Loops

We will go over the syntax of two looping structures:

The While Loop. The basic format for the while loop is the following:

while condition; do
	commands
done

As you would expect, the while loop executes the commands as long as the condition is true.

The For Loop. The basic format for the for loop is the following:

for var in list; do
	commands
done

The for loop iterates over all of the elements in a list. As it is iterating, var will be assigned the value of each item in the list in turn.

The following is an example of these two looping structures:

#!/usr/local/bin/bash
#file: loops.sh


#A while loop with user input
read -p "Enter a letter or x to quit " var1
while [ $var1 != "x" ] ; do
        read -p "Enter a letter or x to quit " var1
done


#A for loop echoing the contents of file
if [ ! -f /net/share/aliases/CS330-001.200430 ] ; then
        exit 1

else
        for var2 in `cat /net/share/aliases/CS330-001.200430`; do
                echo $var2
        done
fi 

Truncating Variables

Sometimes it is useful to strip off parts of a file path or name that match a pattern. You have the following at your disposal:

The following example changes all *.JPG files to *.jpg and truncates the home path

#!/usr/local/bin/bash
# file: truncate.sh


#first create some bogus *.JPG files
touch me.JPG you.JPG harry.JPG


#view them
ls -l *.JPG


#rename them
for i in `ls *.JPG`
do
 newname=${i%.*}
 echo $newname
 newer=$newname.jpg
 mv $i $newer
done


#display your current working directory
var1=$PWD
echo $var1
newpath=${var1#/home/hercules}
echo $newpath
A sample run does the following:
% truncate.sh
-rw------- 1 temp0 temp 0 Nov 25 01:37 harry.JPG
-rw------- 1 temp0 temp 0 Nov 25 01:37 me.JPG
-rw------- 1 temp0 temp 0 Nov 25 01:37 you.JPG
harry
me
you
/home/hercules/t/temp0/330
/t/temp0/330 

Pattern Replacement

The above code could also be replaced by a substitution pattern. You have a choice of two pattern replacements:

In both of these situations, if replacement is omitted, then patt is replaced by nothing, that is, deleted.

#!/usr/local/bin/bash
# file: substitute.sh


#first create some bogus *.JPG files
touch me.JPG you.JPG harry.JPG


#view them
ls -l *.JPG


#rename them
for i in `ls *.JPG`
do
 newname=${i/JPG/jpg}
 echo $newname
 mv $i $newname
done


#display your current working directory
var1=$PWD
echo $var1
newpath=${var1/\/home\/hercules/}
echo $newpath

Arithmetic Evaluations

Sometimes you might perform arithmetic operations on variables. To do this, you use the let command in combination with an arithmetic expression.

You have two kinds of operators to use with the let:

The following table summarizes some of these operators:

Arithmetic Operators
* multiplication
/ division
+ addition
- subtraction
% modulo-results in the remainder of a division
++ increment operator
-- decrement operator
Relational Operators
> greater-than
< less-than
>= greater-than-or-equal-to
<= less-than-or-equal-to
= equal in expr
== equal in let
!= not-equal
& logical AND
| logical OR
! logical NOT

An example of several of the uses of let are in the following sample code:

#!/usr/local/bin/bash
# file: arithmetic.sh

num1=3
num2=10

let res=$num1+$num2             #note no spaces
echo "num1 + num2 = $res"

let "res2= $num1 + 5"           #note how to make it work with spaces
echo "num1 + 5 = $res2"

#Decrement or increment
#let res2++                     #increment operator
let res2--                      #decrement operator

echo $res2


#Logical assignment
#let "mybool=$res2<$res"        #needs quotes to escape the pipe meaning of <
#let mybool=$res2\<$res         #or backslash

let mybool=$res2\>$res
echo $mybool

A sample run of the program would look like this:

% arithmetic.sh
num1 + num2 = 13
num1 + 5 = 8
7
0

Note that you cannot just say: res=$num1+$num2, because it interprets it as a string "3+10". You must use the let keyword.


Command Line Arguments

When you run a shell program that has command line options, each of the options are stored in a positional parameter. The idea is much the same as the argv[] in C programming. For instance, the command name (the name of the shell program) is stored in a variable called 0, the first command line argument is stored in a variable called 1, the second argument is stored in a variable called 2, and so on.

The following table contains a list of built in variables that are related to the command line. This table was taken from: http://docs.rinet.ru/LiSys/lsg26.htm

Variable Use
$# Stores the number of command line arguments that were passed to the shell program
$? Stores the exit value of the last executed command
$0 Stores the first word of the entered command, which is the name of the shell program
$* Stores all the arguments that were entered on the command line ("$1 $2 ...")
"$@" Stores all arguments that were entered on the command line, individually quoted ("$1" "$2" ...)

A sample program using the positional parameters is the following:

#!/usr/local/bin/bash
#command.sh

if [ $# -lt 3 ] ; then
        echo "Please include three command line arguments"
else
        echo "First one: $1"
        echo "Second one: $2"
        echo "Third one: $3"
fi

A sample run might do the following:

% command.sh testing this command
First one: testing
Second one: this
Third one: command

Pipes and Redirections

In bash shell scripts, you can use pipes and I/O redirections as you would on the command line. For instance, you can combine commands (such as awk and grep) with the pipe (|), and you can redirect standard output or standard error to files.

You have use of three default file descriptors:

The following chart summarizes some redirectors (taken from Linux in a Nutshell). Commands in blue are csh only, whereas commands in green are bash only.

  Redirector   Function
> file Direct standard output to file.
< file Take standard input from file.
cmd1 | cmd2

Pipe; take standard output of cmd1 as standard input to cmd2.

>> file

Direct standard output to file; append to file if it already exists.

>| file Force standard output to file even if noclobber is set.
n>| file

Force output from the file descriptor n to file even if noclobber is set.

<> file Use file as both standard input and standard output.
<< text

Read standard input up to a line identical to text (text can be stored in a shell variable). Input is usually typed on the screen or in the shell program. Commands that typically use this syntax include cat, echo, ex, and sed. If text is enclosed in quotes, standard input will not undergo variable substitution, command substitution, etc.

n> file Direct file descriptor n to file.
n< file Set file as file descriptor n.
>&n Duplicate standard output to file descriptor n.
<&n Duplicate standard input from file descriptor n.
&>file Direct standard output and standard error to file.
<&- Close the standard input.
>&- Close the standard output.
n>&- Close the output from file descriptor n.
n<&- Close the input from file descriptor n.

The following program provides a sample of three of these uses:

#!/usr/local/bin/bash
# file: pipes.sh


#To redirect standard output to standard error:
#only for this one command
        echo
        echo "Redirecting stdout to stderr"
        echo "Usage error: see administrator" 1 >&2

#send the files found (stdout) to a file "filelist"
#send standard error to a file "no_access"
#try the find without the redirection to see what the output looks like
        echo
        echo "Using find..."
        echo "Redirecting stdout to \"filelist\" and stderr to \"no_access\""
        find / -name "linux" -print >filelist 2>no_access

#a pipeline with two filters
#prints out temp0's home directory
        echo
        echo "Using a pipe to find temp0's home directory"
        grep temp0 /etc/passwd | awk -F: '{print $7}'


References