The UNIX Shell

(update Dickson Chiu, Spring 2001)

Learning Objectives:
1.

2.

3.

4.

To give detailed description for using commands in Unix Shell To introduce more advanced techniques for handling complicate commands in Unix Shell To understand the the usage of background jobs & shell switching To learn more about pattern matching

COMP111 Lecture 5 / Slide 2

The UNIX Shell

Table of Content
Basic Shell Syntax Command options How Does the Shell find a Command Alias Standard Input Tee Background Jobs Jobs Switching Shells Combining Commands More Pattern Matching More Pattern Matching (grep)

COMP111 Lecture 5 / Slide 3

Basic Shell Syntax

command [-[options]] [arg] [arg] The name of the command is first Options are normally single letters that turn an option on or off. They can be combined or given separately.
$ ls -dil $ ls -l -d -i

Options sometimes also take a value. The value can usually be either given right after the option or separately:
$ ypcat -d ug.cs.ust.hk passwd

COMP111 Lecture 5 / Slide 4

Command Options (1)

Most commands require you to give all options before filename arguments
$ cat names -n Bill Clinton Bill Gates Bill Gates Bill Clinton Monica Lewinski cat: cannot open -n

Spaces separate options. To turn something with spaces into a single argument, use quotes:
$ grep some pizza letter1 grep: cant open pizza
letter1:It is getting late. Please order some pizza and stop

$ grep some pizza letter1 It is getting late. Please order some pizza and stop

COMP111 Lecture 5 / Slide 5

Command Options (2)

Double quotes and single quotes are a bit different (say, when shell variables are involved), but you may use them interchangeably here.
$ grep some pizza letter1 It is getting late. Please order some pizza and stop

To escape a single character (prevent it from being treated specially) proceed it with a backslash:
$ grep We\ll letter1 by my office. We'll tidy up a few more things before $ echo * * $ echo * * $ echo \* * $ echo * letter1 names

secret/

COMP111 Lecture 5 / Slide 6

How Does the Shell Find a Command? (1)

The shell searches a list of directories for an executable file with the same name. The list of directories is stored in the PATH variable for Bourne shells and in the path array for csh/tcsh
$ PATH=/usr/local/bin:$PATH % set path=(/usr/local/bin $path) sh csh, tcsh

If there is a match in more than one directory, the shell uses the first one it finds. If you want to run a command that is not in one of these directories, you can give a pathname (relative or absolute) instead.
~horner/bin/csound

COMP111 Lecture 5 / Slide 7

How Does the Shell Find a Command? (2)

A few commands are built into the shell. This varies from shell to shell. The echo command, for example, is often builtin, for efficiency. You can find out where the shell is getting a particular command using the which command in any shell:
$ which echo echo: shell built-in command. $ which cat /usr/bin/cat $ which grep /usr/bin/grep $ which ls ls: aliased to ls -F

Gives / after directories, * after executables, and @ after soft links

COMP111 Lecture 5 / Slide 8

Alias
The C Shell has the alias command, which allows you to create command shortcuts.
$ alias dir "ls -F" $ alias rm "rm -i" $ alias + "chmod u+x *" $ alias - "chmod u-x *" $ alias 111 "cd ~horner/111" $ pwd /bin $ 111 $ pwd /homes/horner/111

If you put the alias commands in your .cshrc file, you can use them every time you login.

COMP111 Lecture 5 / Slide 9

Standard Input (1)

Every time you login, or run a shell, you are connected to the computer on a particular terminal.
$ who horner horner pts/3 pts/18 Feb 11 10:23 Feb 11 11:57 (csnt1.cs.ust.hk) (csz096.cs.ust.hk)

These devices (pts/3) are actually files in the directory /dev. So, if you are logged in on pts/3, this works just fine (otherwise Permission denied):
$ date > /dev/pts/3 Thu Feb 11 17:08:21 HKT 1999

COMP111 Lecture 5 / Slide 10

Standard Input (2)

In fact, you can redirect stdout to a different device (e.g., pts/18), if you have permission. (The write command works this way.)

You can find out which terminal a particular shell is connected to using the tty command:
$ tty /dev/pts/3 $ echo $
"Hi

Andrew!" > /dev/pts/18

COMP111 Lecture 5 / Slide 11

Tee
A special command called tee acts like a T-joint in plumbing:
$ who horner pts/3 Feb 11 10:23 (csnt1.cs.ust.hk) horner pts/0 Feb 11 11:57 (csz096.cs.ust.hk) $ who | sort | tee sortedwho | wc -l

2
$ cat sortedwho horner pts/0 Feb 11 11:57 horner pts/3 Feb 11 10:23 $ (csz096.cs.ust.hk) (csnt1.cs.ust.hk)

In this example, the output of sort is placed in a file sortedwho and piped to wc -l, which counts the number of lines.

COMP111 Lecture 5 / Slide 12

Background Jobs (1)

A simple command or pipeline can be put into the background by following it with the & character:
$ sort names > names.sort & [1] 3236 $

The shell will print the process ID (PID), and a job number (1, in this case). In some shells, you will be notified when the job is done (you may have to hit return again):
$ sort names > names.sort & [1] 3236 $ [1] Done sort names > names.sort

COMP111 Lecture 5 / Slide 13

Background Jobs (2)

Put a job in the background by typing CTRL-Z
$ ypcat passwd | sort >passwd.sort ^Z Suspended $

The job is suspended - not running - until you either place it in the background using bg:
$ bg [1] $ ypcat passwd | sort > passwd.sort &

or back to the foreground using fg:

$ fg ypcat passwd | sort >passwd.sort

COMP111 Lecture 5 / Slide 14

Jobs (1)
The jobs command tells you what jobs are running:
$ ypcat passwd | sort > passwd.sort ^Z Suspended $ jobs [1] + Suspended ypcat passwd | sort > passwd.sort $

You can stop a job with the kill command:

$ ypcat passwd | sort > passwd.sort & [1] 3414 3415 $ kill %1 $ [1] Terminated ypcat passwd | Exit 2 sort > passwd.sort $

The %1 means job #1. You can also use the PID.

COMP111 Lecture 5 / Slide 15

Jobs (2)
The ps command is the main way to find out about jobs:
$ ps PID TTY TIME CMD 1401 pts/0 0:01 csh $ $ ypcat passwd | sort > passwd.sort & [1] 3476 3477 $ ps PID TTY TIME CMD 3477 pts/0 0:00 sort 1401 pts/0 0:01 csh 3476 pts/0 0:01 ypcat $

COMP111 Lecture 5 / Slide 16

Jobs (3)
Note that if you put something into the background, you better redirect stdout, or the output will appear on your screen anyway!
$ ypcat passwd &
ma_wmkaa:uq2jXK0sFQ8Jg:36747:5000:Woo Man Kei,,=EXP.2001.05.30=nc99S:/homes/ma_h ma_chyaa:CS9wq.1zOxnhI:35435:5000:Chu How Yin Agnes,,=EXP.2001.05.30=nc98F:/homs ee_tkcaa:9LtI7Tipk2Ca6:35651:5000:Tsang Kong Chau,,=EXP.2001.05.30=nc99S:/homesh eg_cck:yi7XtKxxP5KaQ:43555:10010:Cheung Chi Keung,,ce98_yr1:/homes/eg_cck:/bin/h cs_wks:dtjvwifI2G7v2:24514:10001:Wong Kin Shing,,cs98_yr1:/homes/cs_wks:/bin/tch cs_lwk:OWiGoJRXSjn.s:24032:10001:Leung Wai Kei Ricky,,cs98_yr2:/homes/cs_lwk:/bh ph_chyac:CSJUo9e2KGqKg:35955:5000:Chan Hoi Yan,,=EXP.2001.05.30=nc98F:/homes/phs ee_wkkab:dfbi3GqWjvf5U:35644:5000:Wong Ka Keung,,=EXP.2001.05.30=nc99S:/homes/eh ph_lcy:CSwGgr5IeIvqc:36689:5000:Lam Chi Yin,,=EXP.2001.05.30=nc99S:/homes/ph_lcx ^C

$ ypcat passwd >file & $

COMP111 Lecture 5 / Slide 17

Switching Shells
You can switch shells by just typing its name:
csl3su1.cs.ust.hk> ps PID TTY TIME CMD 3496 pts/0 0:01 csh csl3su1.cs.ust.hk> tcsh csl3su1.cs.ust.hk> ps PID TTY TIME CMD 3650 pts/0 0:00 tcsh 3496 pts/0 0:01 csh csl3su1.cs.ust.hk> sh $ ps PID TTY TIME CMD 3650 pts/0 0:00 tcsh 3496 pts/0 0:01 csh 3659 pts/0 0:00 sh $ ^D csl3su1.cs.ust.hk> ps PID TTY TIME CMD 3650 pts/0 0:00 tcsh 3496 pts/0 0:01 csh

COMP111 Lecture 5 / Slide 18

Combining Commands (1)

Multiple pipelines can be input on one command line by separating them with semicolons. When entering a long command, use a backslash (\) to continue the command on the next line.
$ date; sort names; \ who Thu Feb 11 19:40:28 HKT 1999 Bill Clinton Bill Clinton Bill Gates Bill Gates Monica Lewinski horner pts/3 Feb 11 10:23 horner pts/0 Feb 11 19:11

(csnt1.cs.ust.hk) (csz096.cs.ust.hk)

COMP111 Lecture 5 / Slide 19

Combining Commands (2)

Commands can be grouped together using parentheses There are two main reasons to group commands:

To create a single command out of a group of commands (especially useful before a pipe):
$ (cat letter1; head -2 names) | sort >list

To run a set of commands in their own subshell (especially when trying to limit the effect of a cd command):
$ (cd secret; ls | wc -l); ls | wc -l 3 25

This line has the effect of counting the files in secret, and then counting the files in the current directory.

COMP111 Lecture 5 / Slide 20

More Pattern Matching (1)

The notation [abcd] matches any single one of the enclosed characters.
$ ls [il]* it it1 $
ith its@ letter1 letter4

The notation [a-z] matches any lowercase letter. The notation [0-9] matches any digit character. The notation [0-59] matches any the digit characters 0, 1, 2, 3, 4, 5, and 9.
$ ls letter* letter1 letter4 $ ls letter[0-35] letter1 $ ls letter[0-24] letter1 letter4

COMP111 Lecture 5 / Slide 21

More Pattern Matching (2)

Most shells allow you to give a list of strings in curly brackets, comma separated:
$ ls *{1,.sort} NAMES1 it1 f1 letter1 secret1: secret1: Permission denied $ names.sort passwd.sort s1@

COMP111 Lecture 5 / Slide 22

More Pattern Matching grep (1)

Note the patterns for other utilities such as grep, etc., are different. We use grep as an example. The following symbols are treated specially:
^ . * ? start of line any character match zero or more match zero or one $ \ + end of line quote next character match one or more

[aeiou0-9] [^aeiou0-9]

match a,e,i,o,u, and 0 thru 9 match anything but a,e,i,o,u, and 0 thru 9

COMP111 Lecture 5 / Slide 23

More Pattern Matching grep (2)

$ cat names Bill Gates Jacky Gs James Bill J. Jones $ grep '^Bill names Bill Gates $ grep 'Bill$ names James Bill $ grep J...s names James Bill J. Jones $ grep 'G.*s' names Bill Gates Some versions of grep Jacky Gs cannot process ?+ $ grep 'G.+s' names $ egrep 'G.+s' names Bill Gates $ egrep 'G.s' names $ egrep 'G.?s' names J. Jones

$ grep 'J\.' names J. Jones

COMP111 Lecture 5 / Slide 24

More Pattern Matching grep (3)

$ cat names Bill Gates Jacky Gs James Bill J. Jones $ grep 'Ja[cm]' names Jacky Gs James Bill $ grep 'J[^a]' names J. Jones $ grep [t-y]' names Bill Gates Jacky Gs $ grep -in bill names 1:Bill Gates -i ignores case 3:James Bill -n shows line number $ grep -v Bill names Jacky Gs -v means line *not* J. Jones containing pattern $ grep -l Bill n* names -l shows *filename*
containing pattern

$ grep -c Bill names 2 -c shows matching count