Revising some Basic Commands

There are some basic commands that we are going to look at. The idea is to get you into the process of understanding how commands are structured and build an understanding of what the commands do.

From hereon out, I'm going to assume that you can find out lots of things about commands primarily by looking at info and man pages.

Almost every Linux command can be run from the command line using various switches (or arguments / options) which allow one to change the output of this command in a number of different ways.

The who command

The who command is designed to tell you who's logged on to the system.

If we run the who command without any switches, the left hand column shows the user id. This the user currently logged on to the system. In your case, you might be logged on as root, or perhaps as your user. The second column indicates where you are logged in.

riaan@debian:~> who
riaan    :0           Apr 30 11:13 (console)
riaan    pts/0        Apr 30 11:13
riaan    pts/3        Apr 30 11:14
riaan    pts/4        Apr 30 11:30
riaan    pts/5        Apr 30 13:19
riaan    pts/6        Apr 30 12:07
riaan    pts/7        Apr 30 12:09
riaan@debian:~>
                

So if you look at the who command output, my user riaan is logged in from :0 which is the X console. He's also logged on to

pts/0 and 
pts/1
                

These are pseudo terminals, indicating he's also logged into two pseudo terminals.

The final, third column indicates what time the user logged on.

The who command tells us about users using our system. That's great!

What are the other switches that we can use with who.

who --help 
                

This will show you the various switches that we can use with the who; command. So if we use a:

who -H
                

it prints a heading line for us. The output should look as follows:

$ who -H
NAME             LINE     TIME         FROM
heidi            ttyp1    Nov 27 17:29 (168.210.56.177:S)
mwest            ttyp2    Nov 10 15:04 (apotheosis)
heidi            ttyp4    Nov 11 13:18 (168.210.56.177:S)
                

To view a short listing which is the default listing for the who command:

who -s 
                

Using the -u switch:

who -u
                

will show the users and their process id''s.

In scripts, one can use the same commands as on the command line, including all the switches those commands use. One can run any command and produce standard text output, which one can use. We'll talk about how you can use the output later.

Run the command

$ who -u
root     tty2         Aug  4 10:41   .          2839
riaan    :0           Aug  4 07:53  old         2836 (console)
                

to identify which users are logged into your system and from which processes they are logged on.

This will show how long a terminal has been idle. It will show not only which users are logged on and what process ids they are but also how long that user has been idle. Idle users might have gone out for lunch or they might have left for the day. In default mode, most of these systems don't log you out if you're idle for longer than 10 or 15 minutes. In the old days, most systems were configured to automatically log you out after 15 minutes.

[Note] Note

On Debian, the -i switch does not add any extra output, it simply prints a message suggesting that you not use -i as it will be removed in future releases. Use -u as above. However the -i switch may work with other brands of Linux.

Okay, so that's the who command. We're going to use these commands later to build a system to monitor our system automatically, because we want to be spending our time doing things we enjoy.

who command exercises:

  1. How many users are currently logged onto your system?

  2. Is any user logged onto your terminals tty1 -> tty6?

  3. Who (which user and or terminal) has been idle for the longest time?

  4. Ensure that all output is displayed with a heading line.

  5. What run-level are you currently running in and what was your previous run-level? What other command will show you this information?

  6. What would you imagine a users message status would be? (Take a guess or read the man page for write)

The w Command

What does the w command do? You could run:

riaan@debian:~> whatis w
w (1)                - Show who is logged on and what they are doing.
riaan@debian:~>
                
riaan@linux:~> w
 21:40:17 up 11:03,  6 users,  load average: 0.30, 0.34, 0.30
USER     TTY        LOGIN@   IDLE   JCPU   PCPU WHAT
root     tty2    	21:40    8.00s    0.06s  0.06s -bash
riaan    :0        	10:38    ?xdm?  58:31   0.04s -:0
riaan    pts/0     10:38   11:01m  0.00s  2.08s kdeinit: kwrited
riaan    pts/3     11:18   10:22m 14:37   2.63s /usr/lib/java/bin/java -mx32m -jar /home/riaan/jedit/4.2pre9/jedit.j
riaan    pts/4     11:28     1:07m  0.21s   0.21s /bin/bash
riaan    pts/5     11:28     0.00s   0.17s   0.03s w
                

which should print some information about the w command.

The w command not only tells us who are logged in, but also what they're doing. Are these users running applications? What actual processes are they running at this time? Perhaps someone's running an application like OpenOffice. w will tell us this.

If you look at the output of this command, it's got a list of headings that are fairly similar to the format of the who command.

Later we'll have a look at modifying the report columns, to get the output into a different format that may be more useful.

The "date" command

One of the reasons for taking you through these commands is that we're going to start writing our first shell scripts using these commands, so it is as well that we understand them now.

The date command is a useful command that can do all sorts of nifty things for us (apart from printing out the date).

It can convert between Unix time, (which is the number of seconds since 1/1/1970 - commonly known as the epoch) and a human readable (normal) date and time.

Conversely, it can also convert back from date time today to the number of seconds that have elapsed since the 1/1/1970. It can format output in a whole variety of ways. Let's look at some examples of the date command.

For that I'm going to do:

info date
                

If you scroll down, you will see a section with examples. Looking at the example:

date +"     "
                

We may now include a string describing the format inside these quotation marks.

In the shell there's a big distinction between double quotes, single quotes (which is another lesson altogether, see Chapter 5), and back quotes - let's not get them confused for now.

Within this double quoted string we can include any number of arguments. What arguments can we include? Each argument starts with a percentage sign.

To display the time, we could use:

%H - -will give us the hours in 24 hour format (0-23).

%M - -will give us the minutes (0-59) of the day
                

If we had the following string:

date +"%H:%M"
                

we will end up with hours and minutes of the day on our system. The result of the above command should be similar to:

15:04
                

But let's say that we want the hours in 12-hour format rather than 24-hour format. We could then replace the %H with a %l. The result would then be:

3:04
                

There's a host of other things that we could do. For example if we are in 12-hour format, 3:04 doesn't indicate whether it is morning or afternoon. Thus we could include %p:

date +"%l:%M %p"
                

This would show us that the time is actually:

3:04 PM
                

rather than 3:04 AM.

That's for time, but what about for the date? What happens if we want to show the date, which is:

24-Nov-2003
                

then, we should in theory be able to create a date string to reflect this format.

A way we can do is this is using the following:

date +"%d-%b-%Y"
                

where %b is a short format for month to produce Nov instead of November.

If we want to combine the date and time:

date +"%d-%b-%Y %l:%M %p"
                

This would give us the full date and time report:

24-Nov-2003  3:04 PM
                

There are a lot of other parameters that you can use within the date command. You can view these by looking at the relevant info page with :

info date
                

We're going to use this command in our script, because in almost every script that you will write you are going to want to know what time the script started, what time the script ended, when it did a particular job within the script, etc.

date Exercises

  1. Using the info command for assistance, format the output of date, so that it resembles the following: Today's date is Tuesday, 27 January 2004. The time is currently 11h 32m and 49s.

  2. Show the date in Julian format.

  3. What day of the year is today?

  4. Include in the output of the date, the time zone and the AM/PM indicator

  5. Given the number 1075200287, determine what date, day and time this represents.

The 'echo' command

The final command I want to describe is a command used to send output to the screen: echo.

We've seen so far that we were able to run commands but, as yet, we don't know how to simply output some text to the screen. We may want to print a string to the screen, prior to printing the date.

Something such as:

Today's date is:
24-Nov-2003  3:04 PM
                

We would need some way of echoing that to the screen, wouldn't we?

In order to do this, there is the echo command. echo can be a bit of a nasty gremlin because there are at least two echo commands on your system. One is a shell built-in, and one is an external command and therefore it can be a bit tricky.

We're going to start off with a vanilla case. Later on we will see how we can choose which echo command we want to use.

So by way of an example, we'll use it to format the date command.

echo "Today's date is: "
date +"%d-%b-%Y %l:%M %p"
                

This would be a good time to show you how to create your first shell script. We're going to edit a file and for this you can use any editor[1]

Open your editor (whatever editor you prefer) and put the following commands in the first lines of the file:

echo "Today's date is: "
date +"%d-%b-%Y %l:%M %p"
                

Save that file as myfirstscript and exit your editor.

You've just created your first shell script. Great! How easy was that? How do you run it to make it actually do its job?

Running a Shell Script

Linux has three sets of permissions that set the permission mode of the file . One for the owner, one for the group and one for everyone else (i.e. Everyone that is not the owner, and is not part of the group to which the file belongs). You would have covered this in an earlier course (Linux Fundamentals). The mode of the file will also determine whether the file is executable or not.

Thus, to make the file executable, the mode of the file must have the execute (x) permissions set.

Note that this differs from Microsoft Windows which looks at the extension to decide the type of the file. Windows assumes that .com and .exe are executable files. Notice too, that myfirstscript does not have an extension. Is that valid? Sure it is. This is Linux after all!!

In sum, to make our script executable we must change the mode of the file. How? Using chmod (change mode) command as follows:

chmod +x myfirstscript
                    

This script is now executable. Does this mean that we've executed it? No, not yet. We execute it with:

./myfirstscript
                    

and that should output:

Today's date is: 
24-Nov-2003  3:04 PM
                    

Finally, you will notice that in order to execute this command, I preceded it with a "./".

Try running the script without the "./". What happens?

What the "./" does is that the shell FIRST looks in they current directory (.) for the script before trying to look in the PATH. We will cover this is a little more detail later.

Of course, you could add the script to a new directory ~/bin (in my case /home/hamish/bin). Since this directory is in my PATH environment, the script will be "found" and will execute even without using "./"

Creating Scripts Exercises
  1. Write a simple script to print "Hello <YOUR USER NAME>" on the screen.

  2. Modify your scripts to additionally output the directory you are currently in.

  3. Write a simple menu system, with the menu options as follows(note: this menu does not have to respond to the user's input at this stage):

    • 0. Exit

    • 1. Output a text version

    • 2. Output an HTML version

    • 3. Print help

    • 4. Provide a shell prompt

    Select an option [1-4] or 0 to exit:

  4. Include the current date and time in the top right hand corner of your menu

  5. Ensure that your menu system (I'm assuming you called it menu.sh) can execute. Execute it.

Challenge sequence:

Consult the appendixAppendix B and teach yourself the dialog package.

There are many examples in /usr/share/doc/dialog-xxx.yyy.

Once you understand this package, modify your menu system in 3 above to use the ncurses library (dialog) to spruce it up.

File Commands

This section is going to cover file commands. File commands are commands such as ls (list).

Notice again, how the laziness of the Unix people comes to the fore. They could have had a command called list, but that would have required two more characters (and two more carpals - fingers!) and clearly that was a lot more effort, so we just have the ls command. The ls command shows us a listing of files in a particular directory.

This is an appropriate place to take a detour on our tour de scripting and have a look at file matching and wildcard matching. It may be something that you're familiar with, but let's have a look at it and come back to ls in a moment.

Wildcards

Wildcard matching allows us to match a number of files using a combination of characters. What are those characters?

Table 1.1. Wildcards

Symbol Name Purpose
* Splat matches 0 or more of any character
? question mark matches 0 or 1 character
[ ] square brackets matches one from a range of characters
! bang invert the pattern
Wildcard characters with the asterisk (*)

So if we typed

ls *
                        

we end up listing 0 or more of any characters. So it would match any filename of any length because any filename would have a minimum of a single character. The splat matches 0 or more characters following each other.

the question mark ( ? ) wildcard character

The question mark will match a single instance of any character. Later, when we study regular expressions, the full stop ( . ) matches any single character. Given the expression:

ls -la ab?a
                        

this would match the files:

abba
ab1a
ab_a
ab9a
abca
...	
                        
The square brackets ( [] )

What range of characters do [] include? Well we may say something like:

[abc] 	
                        

which would match any (single character) within the following range: a or b or c but it does not match abc since this is 3 characters long. Similarly

[ab]cde	
                        

would match acde or bcde but it would not match abcde. Square brackets indicate a range of alternative characters. How would we match both:

ham01 	
                        

and

ham10
                        

Simple, with:

ham[0-9][0-9]
                        

as [0-9] implies that any character in the range 0 to 9 will match.

Would it also match ham9?

No, for the reason that our pattern search is expecting 2 numeric characters after the pattern ham. (two sets of square brackets are displayed)

We could also combine letters and numbers, for example:

ls [abc123] 
                        

which would match the files:

a
b
c
1
2
3
                        

(Yes Linux and UNIX can have files with all of the above names!)

Notice when we used the square brackets [0-9], we use the range specifier, which was the dash character in the middle. This dash has nothing to do with the minus sign and means match anything in that range from 0 through to 9.

Thus, typing:

[a-z] 
                        

matches any character in the range from a to z.

the bang ( ! )

The final pattern matching wildcard is the bang command. The exclamation mark is the inverse of what you're trying to match. If we were to look at our pattern matching, we could say:-

ls [!a-c]*
                        

which, would match everything NOT starting with an "a" or a "b" or a "c", followed by anything else.

So would it match abc ?

No, because it starts with an a.

Would it match bcde?

No, because it starts with a b.

Finally would it match erst?

Yes, because q is not in the range a to c. So it would match q followed by any set of zero or more characters.

Let's do some more examples using the pattern matching wildcards.

Prior to doing the following command ensure that you create a new temporary directory, and change directory to this new directory [ by doing so, it will be easier to clean up after the exercise is over. ]

I'd like you run the following command (I'm not going to explain it now, it will be covered later).

	
touch {planes,trains,boats,bikes}_{10,1,101,1001}.{bak,bat,zip,tar}
                        

This command creates a number of files for us in one go. Typing:

ls p* 
                        

will show us all files that start with a 'p', followed by 0 or more characters (planes_ in our case). Similarly

ls b* 
                        

will show us the files boats_ and bikes_ since only these files begin with a 'b'. If we typed:

ls bik*
                        

it will produce all the bikes files, while

ls b*_??.* 
                        

will indicate all the files that start with a 'b', followed by 0 or more characters, followed by an underscore followed by exactly 2 characters, followed by a full stop, followed by 0 or more characters. Thus only

boats_10.bak
boats_10.bat
boats_10.zip
boats_10.tar
bikes_10.bak
bikes_10.bat
bikes_10.zip
bikes_10.tar
                        

I've given you some additional examples that you can try out in your own time. It would be worth your time running through these examples and ensuring that you understand everything.

Exercises:

Run the following command in a new subdirectory

touch {fred,mary,joe,frank,penny}_{williams,wells,ferreira,gammon}.{1,2,3,4,5}
                        

Look at the following tu understand the different fields

touch {fred,mary,joe,frank,penny}_{williams,wells,ferreira,gammon}.{1,2,3,4,5}
  ------Name----------    ----------Surname-------    -Category-
                        
  1. list all the people who fall into category 5

  2. List all the people whose surnames begin with a w.

  3. List only people in category 2, whom's surnames begin with a w, and whom's first name begins with an "f"

  4. Move all "fred's" into a directory on their own, but exclude the "gammons"

  5. Ensure that joe and penny wells in category 3 and 5 are deleted.

  6. List only those people whose names have 5 characters in them.

Returning to file commands

Returning from our detour, there's a whole bunch of other file commands that we can look at. We've touched on the ls command, which gives us a listing of files and from the previous set of examples on pattern matching and wildcarding, you will have gotten an idea of how ls works.

ls , like every other Linux command can take switches. Below is a quick summary to some of the switch options.

-l show a long listing (include file name, file size, date last modified, the permissions, the mode of the file, the owner, the group of the file)

-a shows all files including {hidden} files (. and ..)

Two special hidden files are:

. 	this file is our current directory
.. 	this file is our previous or parent directory. 
                    

Often in your home directory, you will have full control over your . directory (i.e. your current directory) but you will most probably have absolutely no control of your .. directory (i.e. your parent directory).

We have other commands like:

cp copy files or directories

mv move files or directories

wc shows the number of lines, number of words and number of characters in a file.

wc -l show us the number of lines in our file.

nl numbers the lines in our file.

System Commands

There are many system commands that we can use. We're going to start using these in our shell scripts.

Remember, a shell script is nothing more than a group of Linux commands working together to produce a new command.

In order to build our system that is going to manage our Linux machine, we're going to need to know a little bit about system commands. System commands such as:

df shows the disk free space

du shows the disk usage

fdisk shows the partitioning on our disk

iostat shows the input output status

vmstat shows the virtual memory status

free shows the amount of free memory

We will use these commands, but they are a subset of the commands available to us for monitoring our system.

The df and du commands

For example, mostly we want to run a command to check whether our file systems are filling up. It makes no sense to have a full filesystem! For that we might use the df command

df would produce a listing of our partitions on our Linux machine and will show us which partitions are 100% full, which partitions are almost empty, which partitions are mounted, etc..

What we're going to be doing, is working our way towards building a system that will automatically show us when a partition becomes 100% full.

Or perhaps we want to build a system that shows us when it's greater than a particular threshold. So we might set a threshold of 95% full, it's no good telling us when the system has crashed that it's crashed; we want to know before the system crashes.

Several switches can be used with df such as:

df -h
                    

This produces output in human readable format. If you run this command now, you should see at least one partition and that's the root partition. It should show you how much space is being used, how much is available, what the size of the partition is and what particular filesystem it's mounted on.

The df command is what we'll start working on for our practical, because it most certainly is one of the most useful commands that you're going to need to run to make sure your system is not becoming overfull.

Alternately, you could use the du (disk usage) command to show you which files are consuming the largest volume of disk space:

du - s 
                    

will show a summary of our current filesystems' capacity. Again, how do you get information about these commands? Using the info or man command will inform you about switches pertinent to these commands.

For example a further switch for the df command might be:

df - hT
                    

which will tell us what type of filesystem we're running. It might be an extended 2, 3, or a vfat filesystem.

The "du" command, like the "df" command, has a complete set of switches unique to it and shortly we'll start looking at those in a bit more detail.

the fdisk command

The fdisk command is used primarily to show us what partitions we have on a particular disk.

BlackBeauty:/install # fdisk -l

Disk /dev/hda: 10.0 GB, 10056130560 bytes
240 heads, 63 sectors/track, 1299 cylinders
Units = cylinders of 15120 * 512 = 7741440 bytes

   Device 	Boot      Start         End      Blocks   		Id  System
/dev/hda1   *           1          760     	5745568+ 	83 	Linux
/dev/hda2             761        1299     4074840    	5  	Extended
/dev/hda5             761         827      506488+  	82  	Linux swap
/dev/hda6             828        1299     3568288+  	83  	Linux

                    

It should show you what your partitions are.

If you execute the above command as a user (in other words, your prompt is a dollar) you're not going to be able to see what your partitions are set to. Why? Because only the superuser (root) has permission to look at the disk. In order to run this command, you need to be root.

The free command

baloo:/home/riaan# free
	     total       used       free     shared    buffers     cached
Mem:        498572     493308       5264          0      48700     230492
-/+ buffers/cache:     214116     284456
Swap:       706852       8196     698656
                    

This command shows the total memory, used memory, free memory, swap space, how much of our swap space has been used and how much of our swap space is still available for use.

the vmstat command

baloo:/home/riaan# vmstat
procs -----------memory---------- ---swap-- -----io---- --system-- ----cpu----
 r  b   swpd   free   buff  cache   si   so    bi    bo   in    cs us sy id wa
 0  0   3792   3508  14208  59348    0    0    32    72 1065   818 11  4 84  2

                    

The vmstat command shows us how busy our system is, how many processes are running and how many are blocked. It also shows memory information: how much memory is being used by the swap daemon and what our buffers and caches are. Additionally, it shows us how many processes are being swapped into and out of memory. Finally, it shows users, system, idle and waiting time. We're going to use it later to monitor our system

the iostat command

Finally, the iostat command.

iostat
Linux 2.6.4-52-default (debian)    09/02/04

avg-cpu:  %user   %nice    %sys %iowait   %idle
           2.51    0.03    1.99    0.82   94.64

Device:            tps   Blk_read/s   Blk_wrtn/s   Blk_read   Blk_wrtn
fd0               0.00         0.00         0.00          4          0
hda               3.93       304.75        23.83    3868650     302512
hdd               0.01         0.59         0.00       7524          0


                    

This command gives you information about input and output on your system, and how well (or otherwise) it is performing.

We'll take a closer look at the system performance commands in more detail later.

[Note] Note

In order to provide you with further information on the performance of your system, you should install the sysstat package. rpm -ivh sysstat.x.y.z-r1.rpm (RedHat system) (see the appendix for other distributions) The iostat command is part of the sysstat package, so if you don't install sysstat, then skip the iostat stuff)

System Commands Exercises

  1. Write a simple script to display the free/used disk space on your machine

  2. Additionally, show the status of the memory and allow the vmstat commands to produce 3 iterations.

  3. Using the iostat command, write a script that will illustrate how busy you machine is

Challenge Sequence

Using the dialog package, ensure that you produce the output formatted in a presentable manner.



[1] If you haven't been taught vi, an excellent place to start is using vimtutor. Type vimtutor on the command line to begin the lessons. It only takes 45 minutes, but is well worth the effort. vi is the best editor in the world, in my humble opinion Almost everyone in the UNIX/Linux world has some relationship with it - a love-hate relationship: some people love it, some people hate it. But if you're going to learn any editor, learn vi. Since it is the de-facto editor on almost any UNIX/Linux variant in the market today, learning the basics at least should stand you in good stead in you years as a system administrator.