Chris Tyler's Wiki

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
====== Bash Scripting 1 =====

===== What is a script? =====

A shell script is a computer **program** which is **interpreted** by an operating system **shell**.

Scipts are used to automate procedures that could be manually performed from the command line. They can potentially save a huge amount of time by eliminating repetitive commands. For example, if you're going to compile and test a program 100x, and each compilation and test cycle requires 25 steps (commands), you're looking at performing 2500 steps. It's much more efficient to create a script containing those 25 steps and run it as needed -- in fact, you can even set things up so those commands execute automatically as soon as you save a new version of your program.

===== Basic Requirements for Shell Scripts =====

1. **Create a text file containing shell commands.** Use any text editor (nano, vi, VS Code, gnome-text-editor, eclipse, ...) to create the file.

2. **Tell the operating system which shell to use.** Add a "shbang" line to the very top of the file, containing the text:

  #!/usr/bin/bash
  
The first two chacters, the **sh**arp (#) and **bang** (!) give this line its name. They are recognized by the operating system kernel as identifying a script. The remainder of this line is interpreted by the kernel as the name of the shell which is to be used to interpret the script. In this case, ''/usr/bin/bash'' is the absolute path of the bash shell. You can substitute other interpreters to write scripts in other shell dialects (such as the Z-shell, ''/usr/bin/zsh'') or languages (such as python, ''/usr/bin/python'').

Note that there must be nothing in font of the #! characters -- no space and no blank lines.

3. **Ensure that the script has appropriate permissions.** The kernel requires execute [x] permission, and the shell requires read [r] permission. Set this up with the chmod command (for example, ''chmod u+rx //scriptname//'').

Here is a simple example script using two commands, ''echo'' and ''date'':

  #!/usr/bin/bash
  echo "The current date and time is:"
  date

Notice the presence of the shbang line.

If this is save into the file named "now", the permission could be set with this command:

  $ chmod u+rx now

The script can then be executed. Normally, the current working directory is not searched, so to run the a script in the current directory, you will need to explicitly specify the directory name like this:

  $ ./now
  The current date and time is:
  Sat Mar  6 12:03:32 EST 2038

===== Comments =====

A comment in a bash script starts with a sharp symbol (#) and is ignored by the shell interpreter:

<code># test script #1
# written by Jason Bourne</code>

A comment may also be just one portion of a line:

<code>cd - # change to the user's previous working directory</code>

Note that a shbang line is a comment from the point of view of the shell interpreter -- it's there for the kernel to use, not the shell!

===== Variables =====

==== Setting a Variable ====

To set a variable, simply type the variable name, an equal sign, and the variable value:

  A=5
  B=World
  TheNameOfTheUser=Jason

If the variable does not exist, it will be created. If it does exist, the previous value will be discarded.

Variable names may contain letters, digits, or underscores, but must not start with a digit.

Unlike some computer languages such as C, variables do not need to be declared. Variables are not typed -- they may be used as strings, integers, or decimal values.

Note that there must not be a space on either side of the equal sign.
==== Accessing a Variable ====

To access a variable, place a dollar sign [$] in front of it, and use it in a command as an argument (or as a command name):

  $ B=World
  $ echo $B
  World
  $ echo Hello $B
  Hello World

==== Quoting ====
=== Word Splitting and Quoting ===

Spaces and tabs are used to split a bash command line into individual "words". This means that if you use an argument that contains a space, it will be treated as two separate arguments:

  $ mkdir test
  $ cd test           # this is new so it will be empty
  $ touch new file
  $ ls -l
  total 0                                                                                                                 
  -rw------- 1 chris.tyler users 0 Mar  6 12:12 file                                                                      
  -rw------- 1 chris.tyler users 0 Mar  6 12:12 new    

Notice that ''touch new file'' created two files, because ''new'' and ''file'' were treated as separate arguments.

To prevent word splitting, quote the text.

  $ mkdir test
  $ cd test          # this directory is new so it will be empty
  $ touch "new file"
  $ ls -l 
  total 0                                                                                                                 
  -rw------- 1 chris.tyler users 0 Mar  6 12:15 new file 

Notice that only one file was created by the command ''touch "new file"'' because the quotes prevented word splitting, and the single argument ''new file'' was used as a single filename.

You'll also need to use quoting when assigning a string variable value containing a space:

  $ A="Seneca Polytechnic"

You can quote text with single-quotes ['] or double-quotes ["]. Variable expansion takes place inside double quotes (this is called //interpolation//), but not inside single quotes:

  $ B=World
  $ echo "Hello $B"    # notice that $B is replaced with the value of B
  Hello World
  $ echo 'Hello $B'    # notice that $B used as-in
  Hello $B

You should always double-quote variables that may contain a space in their value when using them as command arguments. This is especially true for filenames -- you never know when a user is going to put a space in a filename! Many scripts work find with opaque filenames (those containing no whitespace) but fail with non-opaque names.

Here is an example -- the difference between the first ''ls'' statement (which does not double-quote the variable) and the second ''ls'' statement (where the variable expansion is double-quoted):

  $ touch "red maple"
  
  $ FILE="red maple"
  
  $ ls $FILE
  ls: cannot access 'red': No such file or directory
  ls: cannot access 'maple': No such file or directory
  
  $ ls "$FILE"
  'red maple'

=== Backslashes ===

A backslash [\] character outside of quotes or inside double quotes instructs the shell to ignore any special meaning that the following character may have. Examples:

  $ touch "new file"
  $ ls -l new\ file     # The space loses its special meaning as an argument separator
  -rw-r--r--. 1 chris chris 0 Jun 18 22:49 'new file'
  
  $ echo "This string contains a \"quoted\" string"
  This string contains a "quoted" string
  
  $ A=Testing
  $ echo "  \$A"   # The dollar sign loses its special meaning
    $A

===== Environment Variables =====

By default, a variable is local to the shell in which it is running.

You can //export// variables to make them //environment variables//. That means that they are passed to programs executed by the shell.

Environment variables are used by all proccesses, not just the shell, and are commonly used to pass configuration information to programs.

To create an environment variable named X with a value of 999:

  $ X=999
  $ export X

Or in a single step:

  $ export X=999

You can view all of the current variables (and shell functions) with the ''set'' command, or just the environment variables with the ''printenv'' command (you'll probably want to pipe the output through ''less'').

The term //environment variable// is often shortened to the abbreviation //envar//.

==== Common Environment Variables ====

^ Environment Variable ^ Purpose ^ Examples ^
| PS1 | Normal (first-level) shell prompt | ''PS1="Enter command: "\\ PS1="[\u@\h \W]\$ "'' |
| EDITOR | Path to the default text editor (typically /usr/bin/nano) | ''EDITOR=/usr/bin/vi'' |
| PATH | A colon-separated list of directories that will be searched when looking for a command | ''PATH="$PATH:."''\\ ''PATH="/usr/bin:/usr/sbin:."'' |
| LANG | The default language -- used to select message translations as well as number, currency, date, and calendar formats. | ''LANG=en_CA.UTF-8''\\ ''LANG=fr_CA.UTF-8'' |
| HOME | The user's home directory - used for relative-to-home pathnames. | ''HOME=/var/tmp'' |
| RANDOM | A random integer (0-32767) | |

===== Reading Variable Values from Stdin: read =====

The ''read'' command reads a line of text from stdin and assigns it to the specified variable. For example, ''read A'' reads a line of text and assigns it to the variable ''A''.

The ''read'' command can also send a prompt to stdout using the -p option:

  $ read -p "Enter your name: " NAME
  Enter your name: Chris
  $ echo $NAME
  Chris

Here is a script which uses a couple of ''read'' statements:

  #!/usr/bin/bash
  read -p "Please enter your name: " NAME
  echo "Pleased to meet you, $NAME"
  read -p "Please enter a filename: " FILE
  echo "Saving your name into the file..."
  echo "NAME=$NAME" >>$FILE
  echo "Done."

===== Command Capture =====

You can capture the output (stdout) of a command as a string using the notation ''$( )'' and then use that string in a variable assignment or as a command argument:

  $ echo "The current date and time is: $(date)"
  The current date and time is: Mon 19 Jun 2034 12:02:11 AM EDT

  $ FILES="$(ls|wc -l)"
  $ echo "There are $FILES files in the current directory $(pwd)"
  There are 2938 files in the current directory /bin

**Avoid Backticks!** You may see old scripts that use backticks (reverse single quotes) for command capture, like this:

  $ A=`ls`

This is an archaic syntax which is depricated -- avoid doing this. Some fonts make it hard to distiguish between backticks and single-quotes, and nesting backticks is difficult.

===== Arithmetic =====

Bash can perform integer arithmetic.

To evaluate an arithmetic expression and return a value, use ''<nowiki>$(( ))</nowiki>'':

  $ A=100
  $ B=12
  $ echo $((A*B))
  1200
  $ echo $((B++))
  12
  $ echo $B
  13

Note that inside the double-parenthesis, spaces don't matter, and it is not necessary to use a dollar-sign [$] in front of variables being accessed.

To evaluate an arithmetic expression without returning a value, use ''<nowiki>(( ))</nowiki>'':

  $ A=100
  $ B=13
  $ ((A++))
  $ echo $A
  101
  $ ((C=A*B*2))
  $ echo "The answer is $C"
  The answer is 2626

===== Exit Status Codes =====

When any process finishes executing, it exits with a numeric value. This can be called the exit code, status code, exit status code, or error code.

Usually, an exit status code of zero (0) means that no errors were encountered, and a non-zero code means that something went wrong. Therefore, it may be easiest to think of this as the error code, with 0 meaning no errors.

Be aware that program authors can use this value as they see fit, so the status code may indicate something else, such as the number of data items processed.

The special variable ''$?'' is set to the exit status code of the last command executed by the shell. 

For example:

  $ ls -d /etc
  /etc
  $ echo $?
  0
  
  $ ls -d /this/does/not/exist
  ls: cannot access '/this/does/not/exist': No such file or directory
  $ echo $?
  2

Why is this important? Because exit status codes are the key to conditional logic (if...) and looping (for/while/until/...) in bash scripting.

===== Conditional Logic: if / then / elif / else / fi =====

Bash provides an ''if'' command to support conditional logic:

  if LIST1
  then
    LIST2
  fi

If the command or commands in LIST1 execute successfully and return an exit status code of 0, then the commands in LIST2 are executed.

For example, you could use a ''grep'' command as LIST1 and an ''echo'' command as LIST2:

  if grep -q "OPS102" courses.txt
  then
    echo "The course code 'OPS102' was found in the file."
  fi

The ''if'' command also supports the ''else'' keyword:

  if grep -q "OPS102" courses.txt
  then
    echo "The course code 'OPS102' was found in the file."
  else
    echo "The course code 'OPS102' was NOT found in the file."
  fi

It also supports the ''elif'' (else-if) keyword:

  if grep -q "OPS102" courses.txt
  then
    echo "The course code 'OPS102' was found in the file."
  elif grep -q "ULI101" courses.txt
  then
    echo "The course code 'ULI101' was found in the file.
  else
    echo "Neither 'OPS102' nor 'ULI101' was found in the file."
  fi

Putting this all together, you could have a script like this:

  #!/usr/bin/bash
  
  read -p "Enter a filename: " FILE
  
  if grep -q "OPS102" "$FILE"
  then
    echo "The course code 'OPS102' was found in the file $FILE."
  elif grep -q "ULI101" courses.txt
  then
    echo "The course code 'ULI101' was found in the file $FILE.
  else
    echo "Neither 'OPS102' nor 'ULI101' was found in the file $FILE."
  fi

==== The test Command ====

To perform tests, such as comparisons, bash provides the ''test'' command.

Test accepts parameters that comprise a test, such as a test for string equality, and returns a status code of 0 if the test succeeds or non-0 if it fails:

  test "$NAME" == "Chris"

This is used with the ''if'' statement like this:

  if test "$NAME" == "Chris"
  then
    SUPERPOWERS="Yes"
  fi

However, this syntax is a bit ugly! So bash provides a synonym for ''test'' which is ''<nowiki>[[ ]]</nowiki>'':

  if [[ "$NAME" == "Chris" ]]
  then
    SUPERPOWERS="Yes"
  fi

Remember that the double square-brackets are really just the ''test'' command in disguise. This means that the arguments are command arguments, and need to be separated by spaces, just as they would with any other command such as ''ls'' or ''cp''.

**Note:** The original test command in the original Unix shell (the Bourne shell) was aliased to the single square bracket ''[ ]''. For compatibility, this is still available in the bash shell. However, the double square-bracket version of the test command has some improved capabilities, and is recommended (except when you need a script that is compatible with a wide range of different shells).

=== Available Tests ===

There are four main types of tests available:

== Tests Group 1: Filesystem Entries ==

These tests check a filename to see if it is a regular file, a directory, or a symbolic link:

  [[ -f filename ]]   # True if filename is a regular file
  [[ -d filename ]]   # True if filename is a directory
  [[ -L filename ]]   # True if filename is a symbolic link

**Note:** If a symbolic link points to a file, then both the ''-f'' and ''-l'' tests will succeed. If a symbolic link points to a directory, then both the ''-d'' and ''-l'' tests will succeed.

== Tests Group 2: File Permissions ==

These tests check a filename to see if the person running the script can read, write, or execute the file (or access the directory):

  [[ -r filename ]]   # True if filename is readable
  [[ -w filename ]]   # True if filename is writable
  [[ -x filename ]]   # True if filename is executable (accessible if a directory)

== Tests Group 3:  Strings ==

These tests accept two string arguments, which are compared:

  [[ string1 == string2 ]]   # True if the strings are equal
  [[ string1 != string2 ]]   # True if the strings are not equal
  [[ string1 >  string2 ]]   # True if string1 sorts after string2 lexicographically
  [[ string1 <  string2 ]]   # True if string1 sorts before string2 lexicographically

**A note on the term //lexicographically//:** Sorting //lexicographically// means sorting according to character code. This is like sorting alphabetically, but it applies to non-alphabetic characters as well, such as digits and punctuation marks. See the manpage for "ascii" to see the sequence of the first 128 character codes (or refer to a Unicode table for all of the character codes).

When sorting lexicographically, a comes before aa, which comes before b:

  a
  aa
  b

In a similar way, 1 comes before 11, which comes before 2:

  1
  11
  2

Note that this is different from numeric order, where 2 would preceed 11:

  1
  2
  11

== Test Group 4: Integers ==

These tests accept to integer arguments, which are compared:

  [[ integer1 -eq integer2 ]]  # Integers are equal
  [[ integer1 -ne integer2 ]]  # Integers are not equal
  [[ integer1 -gt integer2 ]]  # Integer1 is greater than integer2
  [[ integer1 -ge integer2 ]]  # Integer1 is greater than or equal to integer2
  [[ integer1 -lt integer2 ]]  # Integer1 is less than integer2
  [[ integer1 -le integer2 ]]  # Integer1 is less than or equal to integer2

== Other Tests ==

The four groups of tests above will cover the vast majority of situations. There are additional tests available to test other conditions, such as whether a variable is defined, or a file refers to a device. See the man page for bash(1) for more information if you're interested in other tests: ''man bash''
 
== Negating and Combining Tests ==

The ''!'' operator can be used to negate a test:

  [[ ! -f "$FILE" ]]   # True if "$FILE" is not a file (doesn't exist, or is a directory)

The AND operator ''&&'' combines tests, with the result being True if the tests on the left and right are both true:

  [[ $A -lt $B && $B -lt $C ]]  # True if A is less than B, and also B is less than C.

The OR operator ''||'' combines tests, with the result being True if either of the tests are true:

  [[ $A -gt $B || $A -lt 0 ]] # True if either: A is greater than B, or if A is less than 0.

You can use multiple ''!'', ''&&'', and ''||'' operators toegether:

  [[ -f "$FILE" && -r "$FILE" && -w "$FILE" ]] # True if "$FILE" is a readable, writable regular file.

== Tips on Using Tests ==

  * Remember to quote any arguments which include whitespace, or which may be null (empty).
  * Be careful with the ''<'' and ''>'' comparison operators: if you have a syntax error, you may accidentally redirect data (which in the case of ''>'' might overwrite a file!)

===== Parameters =====

Arguments to a script are called parameters. You can access the parameters using the special variables ''$0'', ''$1'', ''$2'', and so forth. ''$0'' contains the name of the script, ''$1'' contains the first parameter, ''$2'' contains the second parameter, and so forth.

The special variable ''$#'' contains the total number of parameters.

Here is a simple script which shows you what parameters have been received:

  #!/usr/bin/bash
  echo "Number of parameters:  $#"
  echo "Parameter 0:           $0"
  echo "Parameter 1:           $1"
  echo "Parameter 2:           $2"
  echo "Parameter 3:           $3"
  echo "Parameter 4:           $4"

When you run this script with three parameters (red, green, and blue), you get this output:

  $ ./params red green blue
  Number of parameters:  3
  Parameter 0:           ./params
  Parameter 1:           red
  Parameter 2:           green
  Parameter 3:           blue
  Parameter 4:

The ''shift'' command discards parameter ''$1'' and moves each of the remaining parameters to the previous position (so the value in parameter ''$2'' is moved to ''$1'', and ''$3'' is moved to ''$2''). We can modify the previous script to demonstrate this:

  $ cat params2
  #!/usr/bin/bash
  echo "Number of parameters:  $#"
  
  echo "Parameter 0:           $0"
  echo "Parameter 1:           $1"
  echo "Parameter 2:           $2"
  echo "Parameter 3:           $3"
  echo "Parameter 4:           $4"
  
  echo "---- Performing shift ----"
  shift
  
  echo "Parameter 0:           $0"
  echo "Parameter 1:           $1"
  echo "Parameter 2:           $2"
  echo "Parameter 3:           $3"
  echo "Parameter 4:           $4"
  
  $ ./params2 red green blue
  Number of parameters:  3
  Parameter 0:           ./params2
  Parameter 1:           red
  Parameter 2:           green
  Parameter 3:           blue
  Parameter 4:           
  ---- Performing shift ----
  Parameter 0:           ./params2
  Parameter 1:           green
  Parameter 2:           blue
  Parameter 3:           
  Parameter 4:  

The ''shift'' command is useful for looping through parameters, and for accessing parameters higher than number 9.

===== Example Scripts =====

=== Computer Architecture ===

This script displays a message based on the architecture of the computer:

  #!/usr/bin/bash
  architecture="$(uname -m)" # uname gets system information
  
  if [[ "$architecture" == "x86_64" ]]
  then
     echo "Your computer architecture is Intel/AMD x86_64."
  elif [[ "$architecture" == "aarch64" ]]
  then
     echo "Your computer uses the 64-bit Arm architecture."
  else
    echo "Your computer uses an unrecognized architecture."
  fi

=== Age Check ===

This script checks whether a customer is of legal drinking age in Ontario:

  #!/usr/bin/bash
  read -p "Enter the customer's date of birth: " B
  
  # Calculate the time in seconds that the customer turns/tuned 19
  D="$(date -d "$B + 19 years" +%s)"
   
  # Get the current time in seconds
  NOW="$(date +%s)"
  
  # Tell the user if the customer is old enough to be served alcohol
  # This tests checks to see if the customer's 19th birthday is
  # less than (before) the current date.
  if [[ "$D" -lt "$NOW" ]]
  then
      echo "The customer is of legal drinking age in Ontario."
  else
      echo "The customer is too young to legally drink in Ontario."
  fi

=== Coinflip ===

This script flips a virtual coin:

  #!/usr/bin/bash
  
  COINFLIP=$((RANDOM % 2))  # % is the modulus operator
  if [[ "$COINFLIP" == 0 ]]
  then
      echo "Heads! 🙂"
  else
      echo "Tails 😦"
  fi

The COINFLIP variable is set to the remainder of the division of ''$RANDOM'' by 2. Therefore, it will have a random value of 0 or 1.

Note that this script uses extended Unicode characters -- however, for these to display properly, your terminal and your terminal font must both support the extended characters. Besides emoji, extended characters may be used to display accented characters, symbols, and characters from other languages.

=== Cautious File Delete ===

This script checks a file, provided as a positional argument (parameter), to ensure that it is a regular file and is writable, and then asks the user if they want to delete it. Note that this script uses the ''-f'' (file) test, ''-w'' (writeable) test, and combines a number of string tests with the ''||'' (OR) operator:

  #!/usr/bin/bash
  if [[ "$#" -ne 1 ]]
  then
      echo "$(basename $0): Error: one filename argument must be provided." >&2
      exit 1
  fi
  F="$1"  # Put the first (and only!) argument value into the variable F
  if [ ! -f "$F" ]
  then
      echo "The filename '$F' does not refer to a regular file - skipping."
  elif [ ! -w "$F" ]
  then
      echo "The file '$F' is not writeable (by you) - skipping."
  else
      read -p "Delete the regular file '$F'? (Y/N): " YESNO
      if [[ "$YESNO" == "Y" || "$YESNO" == "y" || "$YESNO" == "Yes" 
             || "$YESNO" == "yes" || "$YESNO" == "YES" ]]
      then
          echo "Deleting the file '$F'..."
          rm "$F"    # We should add some code to check if the rm succeeds or fails
          echo "...done."
      else
          echo "Skipping the file '$F' as requested."
      fi
  fi