Categories : Linux

 

1. Introduction
1.1. The Basics

2. Script Fundamentals
2.0.1. What it Means
2.1. Variables
2.2. Special Syntactic Elements
2.2.1. Comments
2.2.2. Assignment
2.2.3. Expression Evaluation
2.2.4. Condition Evaluation
2.2.5. Mathematical Operations
2.3. Control Statements
2.3.1. if/else/elif Switches
2.3.2. While Loop
2.3.3. For Loop

3. Useful Programs
3.1. Basename
3.2. Cat
3.2.1. Combining Several Files
3.2.2. Appending a File to Another File
3.3. Cut
3.4. Date
3.5. Echo
3.6. Grep
3.7. Sort

4. Putting it All Together
4.1. Make Life Easier
4.2. Make it Useful
4.3. Have Fun

1. Introduction

One of the best parts about Linux is that is is easily customizable. You don’t have to be affluent in the arcane craft of writing programs in c, c++, or Java, nor do you have to learn the features of a more advanced scripting language like Perl. All you really have to know is how to execute a few commands.

Yes, it is THAT simple. If you can execute a few commands, then you can write a shell script. Of course, if you learn a few simple tools, you can do so much more. Whether you want to construct a simple mnemonic, or link programs together in complex ways, shell scripting is a simple, easy, and efficient to make Linux your own.

Top

1.1. The Basics

For this tutorial, I will presume that you are using the Bash shell, which is by far the most popular, and usually the default. I will also presume that you have some flavor of sed and awk installed on your system, again, these are usually installed as part of the default. Finally, I am going to presume that you are competent in creating a text file and executing simple commands on your system. If not, do see the New2Unix tutorial, and/or LINUX IN A NUTSHELL by O’Reilly Press. That being said, let’s begin.

Top

2. Script Fundamentals

The first thing we need to do is to define exactly what we mean by shell script. Breaking this apart, the shell is the interface between the user and the kernel; it is the place where you enter all of your commands. A script is a list of instructions (commands) that the shell will execute automatically. You could type everything in by hand and get the same result, but the point of a shell script is that it saves time by saving on typing.

For example, beginning programming students will often compile a c program, and then try and execute it. Since the program is often times not correct, either the compilation or the execution fails, and so they edit the file and try again. The commands to compile the file and then (hopefully) execute the binary can get in the way, slowing down the programmer, and even be a source of frustration (a typo causes the file to be not found, and since it’s 4 AM, you don’t realize it’s just a typo). The following shell script could make life a LOT easier:

 

Example 2.0 A: compile.sh
=======================

#!/bin/bash

gcc $1

./a.out

rm a.out

=======================

This shell script takes a single command line parameter (the name of the source file) and tries to compile it. It then tries executing the binary before removing the binary.

If the source file doesn’t compile, you still get the error message, and the script simply won’t find the binary to execute. Likewise, it won’t be able to remove the binary, however, it is essential to remove the binary if it is created successfully. This is because a binary can be successfully created, the source changed (so that it is incorrect) and recompiled, followed by running the script again. You would get both a compiler error and an executed binary. This way, you may get a few more error messages, but that’s a trade off. There are, of course, ways of suppressing these error messages, but that will come later.

The next thing to do is to make the script executable, which we do by specifying the command “chmod +x “, in this case “chmod +x compile.sh”. Then, all we have to do is run the script by entering “./compile.sh “, and it runs all of the commands in the script.

Top

2.0.1. What it Means

The above example is relatively simple, but for the novice, I’ve already used a bunch of thing that are new, so I’m going to take a minute to explain them.

“#!/bin/bash” is a special kind of comment that begins most scripts. This tells the script what interpreter to use to actually run the script. Putting this in here is the equivalent of typing “bash compile.sh” on the command line.

“gcc $1″ tells the script to execute the gcc program on the first command line argument. Variables in shell scripts have a $ attached to them, but variables that consist of a $ followed by a number are reserved for command line arguments.

“./a.out” and “rm a.out” are just simple commands that you would enter on the command line to achieve the same result. Remember when I told you a shell script was really just a bunch of simple commands? Well there you go.

Top

2.1. Variables

Shell scripting would be damned inefficient if it didn’t allow us to use variable substitution. As you saw in the previous example, using variables makes your shell script a lot more flexible. However, shell script variables are a little bit different then their C and Java counterparts. For instance, to initialize a variable “num” to the value “1″, the syntax is: ” num=”1″ “. You could give the value num a string too; the shell isn’t specific about what kind of variable it is working with.

Getting the variable out of the script is a little bit different. To print the value of num to the screen, we would type “echo $num”. This tells the shell to evaluate num to a value. Lacking the preceding $, the word “num” would be echoed to the command line, but if you don’t believe me, you can try it out yourself…

 

Example 2.1 A: printnum.sh
=============================================

#!/bin/bash

num="1"

echo $num #prints the number 1 to the screen

echo num  #prints the word num to the screen

=============================================

Top

2.2. Special Syntactic Elements

The shell has a syntax that is in many ways quite similar to the syntax of C or Java. However, there are several distinct features that are worth noting.

Top

2.2.1. Comments

You comment out code in Bash (and many other scripting languages) by using a hash mark (aka a pound sign); #. Everything after the hash mark is ignored by Bash.

Top

2.2.2. Assignment

In Bash, spacing is critical when assigning variables. If you try giving i the value of 1 by saying “i = 1″, it’s not going to work. Instead, Bash will try executing “i” as if it were a command. However, “i=1″ will yield the result you’re looking for.

Top

2.2.3. Expression Evaluation

One thing that trips up a lot of bash newbies is expression evaluation. You can’t just assign a variable a value like you would in C. For example, if you were trying to get the current working directory stored into a variable, and you tried something like “current=pwd”, and then tried echoing $current, you would get pwd printed out. To get the shell to execute the command, enclose it in back-quotes (`). So, in the previous example, we would say “current=`pwd`” to store the current directory in a variable.

Top

2.2.4. Condition Evaluation

Condition evaluation refers to testing various conditional operators. This is another this that you are going to use a lot, so pay attention. Condition evaluation occurs by way of the “test” command; the syntax of which is “test condition”, where condition is a test of some sort that returns either true or false. However, the shell provides a nicer way of writing this, so it resembles more closely the syntax of C: “[ condition ]“. This is the same thing as writing “test condition”, but looks better, and is more clear to programmers.

Note well the spaces that exist between “condition” and its enclosing brackets. That is part of the syntax, and if you do not include the space, then you’ll get an error.

There are several different types of condition tests; too many to list here, so if you want to know all of them, take a look at “man test”. However, here is a short list of numerical comparison tests. Note, the dash preceding the letters is PART of the syntax.

 

-eq The first value is equal to the second value.
-ge The first value is greater than or equal to the second value.
-gt The first value is greater than the second value.
-le The first value is less than or equal to the second value.
-lt The first value is less than the second value.
-ne The first value is not equal to the second value.

These tests all take the form of “value1 test value2″, so to see if the value of i is less than 10, you would use “[ $i -lt 10 ]“. Simple, right? See the man page to find out what else you can do.

Top

2.2.5. Mathematical Operations

Mathematical operations are essential to writing an effective program in any language. Being able to increment (or decrement) a number is one of the most basic and important things that a language can do. That being said, Bash does not do this automatically. You can not enter i=1, $i++, and get 2 as a result.

The easiest way (in my opinion, anyhow) to do math in Bash is with the “let” command. Using let is pretty simple. For example, you can use “let i=i+1″ for auto-incrementing. Plus, if you enclose the expression (in this case “i=i+1″) in quotations marks, you can even space it out to “i = i + 1″, so it’s a bit easier to read.

Top

2.3. Control Statements

Again, it would be really nasty if we could only execute a sequence of commands in a specific order, and then have to stop. Sometimes, you want to do something until a condition changes. At other times, you want to do one thing or another, depending on some condition. Fortunately, bash offers us if/else/elif (else if), for loops, and while loops. Again, the syntax isn’t the same as legacy languages… in fact it is dramatically different, so pay close attention here, you’ll be glad that you did.

Top

2.3.1. if/else/elif Switches

This is probably the most basic type of control statement there is. If condition 1 is true, do this, else do that. Only slightly more complicated is the If condition 1 is true, do this, else if condition 2 is true, do that, else do something or other. These types of statements are the easiest way to add some dynamic functionality to you scripts.

The basic syntax is…

if [ condition1 ] then # statements go here elif [ condition2 ] then # more statements go here else # still more statements fi

I should note that just because you have an if, doesn’t mean you need an elif, or an else. You can use an if alone, with an elif, with an else, or with any combination of the above, however you must remember the “fi” that follows the block. See copy.sh or cdrip.sh for examples.

Also, note well that the spacing between the conditions and the brackets is important. If there is not one space between the beginning bracket and the beginning of the condition, and one space between the end of the expression and the closing bracket, you will get an error message.

Top

2.3.2. While Loop

A while loop is another essential control structure. It is less complicated than its do-while and for loop cousins, and provides enough functionality to get you through just about any scenario in which you would need a loop.

The basic syntax for the while loop is…

while [ condition ] do # code goes here done

Note well that, just as in the case of the if switches, the space between the brackets and the condition is NECESSARY.

Top

2.3.3. For Loop

The for loop operates differently in Bash than it does in C. In Bash, you use a for loop to execute a command on each element of a list. The syntax for this is…

for i in LIST do # various commands done

I can tell already that you’re confused, so let me explain.

LIST is a list of values that can come from a file, or another command. i is a variable to which a member of the LIST is assigned, and then some command is performed on i. The next element in the LIST is taken, and so on, until we are done. Here’s an example that looks at my home directory, and checks every file to see whether or not it is a directory. Directories are echoed back to the command line while other files are ignored.

 

Example 2.3.3 A: fortest.sh
===========================

#!/bin/bash

for foo in `ls $HOME`
do

  if [ -d $foo ]
  then

   echo "$foo"

  fi

done

===========================

This script applies a lot of what you’ve learned so far, so if you understand it, reward yourself with a candy bar. If you don’t, reward yourself with a Pepsi, and look over the fuzzy parts again.

Please note that “foo” is just a variable name, and that I used it here instead of a variable that made sense to illustrate that it is JUST a variable name. However, “in” is a required part of the syntax.

Top

3. Useful Programs

Like I said, shell scripting is a way of automatically executing a bunch of programs so you don’t have to do it manually every time. The usefulness of this is far easier to see when you actually have an idea as to what some of these programs are, so I am here providing a very short list of the types of programs you might want to use, and how you would use them, in your shell scripts.

I am going to presume that, by now, you are familiar with simple commands like “ls” and “cd”. If you are not, you should probably study the basics before you get any further into shell scripting. Again, this is a short list. To find more, you’ll need to do some research.

Top

3.1. Basename

Basename removes the directory path, returning the value of the file. For example, “basename /usr/bin/calc” would return “calc”. This isn’t as useful as some other programs, but script long enough, and you’ll find a situation where you’ll want it.

Top

3.2. Cat

Cat is one of the most useful programs you’ll come across in Linux. Using cat you can display the contents of a file to standard output, combine several files into one file, append one file to another file, and create a file from your terminal. Here’s a look at how some of this works.

Top

3.2.1. Combining Several Files

To combine several files, the files must first exist on your system. Then, the command is “cat file1 file2 > completefile”. Note that you can do this with as many files as you want.

Top

3.2.2. Appending a File to Another File

This is fairly simple, “cat file1 >> file2″ appends the text of the first file to the end of the second file. Be careful, if you only use one angle bracket (>) you will overwrite the contents of file2.

Top

3.3. Cut

Cut is very handy. As its name implies, you use cut to copy certain columns or fields out of a file and send them somewhere else (standard output, or another file. The syntax for cut is too complex to cover here in any great detail, so you’ll need to look at the man page to learn more, but there are a few options that are particularly useful that I’ll discuss. First, an example:

cut -d: -f1,3 /etc/passwd

This example prints a list of each username and userid found in the /etc/passwd file. The first option, “-d:” sets the field delimiter to a colon(:). If we wanted to set the delimiter to anything else, we would use whatever it is we want to change the field separator to. For instance, if you want each field to end with a semi-colon (as is typical for email lists), you would use “-d;”. Special characters, such as a space, must be quoted; ‘-d” ” ‘ would make a single space the field-separating character (note that the single quotes are purely for the purpose of isolating the statement).

The second argument to the above command, “-f1,3″, tells cut which fields to display. In this case, we grab the first and the third fields (which correspond to the username and userid in /etc/passwd), but more fields can be added using further colons. For example, to print the username, userid, and login shell, you would use “-f1,3,7″ as a field specifier.

As I said before, cut has more features, so see the man page for more info.

Top

3.4. Date

This is pretty simple, but has a lot of options. The date command, in it’s simplest form, returns the current date and time. However there are several options that allow you to specify the format of how you want the date displayed. Take a look at the man page if you want to know more.

Top

3.5. Echo

This is the simplest way of getting output onto the screen. If you want to tell the user anything that is not the product of some other command, you can use echo to make them see it. The syntax is “echo string”. There are several characters with special meanings that you can escape (see the man page for more info), but the standard Red Hat installation (and others I imagine) turned these off by default, so you might want to specify the “-e” option. For example, to print a message and ring a bell, the command would be:

echo -e “Installation Complete \a”

Top

3.6. Grep

While grep isn’t essential, it is very useful. Grep searches the files you tell it to search for a pattern you tell it to search, and prints any successful results to the screen. This is, of course, grep in its simplest form, as there are several options that can customize how data is displayed, where patterns come from, and how far down the directory tree to search. If you thought I was going to tell you to see the man page, give yourself a cookie for being right. But here’s a simple example of how to use grep. This example searches all the files in a directory for the pattern foobar: “grep foobar *”.

Another point worthy of note is that grep can handle regular expressions… but regular expressions are a discussion for another time.

Top

3.7. Sort

I hope you’re sitting down for this one… sort SORTS stuff. Specifically, it sorts files, but it will also sort input from a pipe. There are many different ways of sorting. The “-d” option tells the program to sort in dictionary order (A-Z), while the “-n” options will sort things in numerical order. If you want to get really fancy, “-r” sorts things in reverse order. See the man page to find out more. Here’s a quick example shell script that gets the username and userids from /etc/passwd, and sorts them numerically based on the userid.

 

Example 3.7 A sorttest.sh
=========================

#!/bin/bash

cut -d: -f1,3 /etc/passwd | sort +1n -n -t:

=========================

You’re already familiar with cut, so lets talk about sort. The “+1n” command tells sort to skip that many fields before it starts sorting. Since we’ve already cut /etc/passwd, and have preserved the username and the userid, we want to sort the second field, so we skip the first. “-n” I’ve already talked about; it tells sort to sort things in numerical order. “-t:” is like “-d:” in cut, it specifies a field delimiter. Since cut has separated the fields with a colon, we can use the colon for the delimiter when sorting it too.

Top

4. Putting it All Together

Congratulations, you now know what you need to know to start writing some scripts. But before I say goodbye, I would like to leave you with a little bit of advice regarding shell scripting.

Top

4.1. Make Life Easier

The point of writing a shell script is to make life easier in some way. There’s no reason why something has to be very complicated and have lots of options if the only reason you wrote it was to provide you with a mnemonic. If your shell script is there solely to constantly enable some option on ls, you might want an option to disable it, but if you make it too fancy (options for options), you will live to regret it. Not only will it take you longer to write and debug the script, it will also take longer for the script to run.

Top

4.2. Make it Useful

Go ahead and write any script you want, but before you install it as a permanent resident in you executable path, make sure that it is something that is both USEFUL and UNOBTRUSIVE. Writing a shell script that emulates the “make” utility is a nice exercise, but make works just fine, and if you call your script make, you are going to run into a few problems. Re-inventing the wheel is okay, as long as you add a feature that makes it work better, or more to your liking.

Top

4.3. Have Fun

Go ahead and try new things, play around, and see what you can do. Some aspiring individuals write very complex programs solely in bash, while a lot of sys admins just use it to make life easier (automatically executing common commands and what-not). Good luck!

 Posted on : August 22, 2014

You might also likeclose