CS2028 - Unix Internals

Chapter 1
 Agenda

 General Overview of the Unix

 History

 System structure

 User Perspective

 Operating System Services

 Assumptions about hardware

 What is Unix?

What is an operating system?

The low­level (first layer loaded) software which 
handles the interface to system hardware (input/output 
devices, memory, file system, etc), schedules tasks, 
and provides common core services such as a basic 
user interface.
all of the software that comes with a system before 
applications are installed.

3
 What Is Unix?

Examples of Operating Systems:
Unix and Unix­like:  A/UX, AIX, *BSD (Free, Micro, 
Net, Open, etc), Darwin (Mac OS X), HP­UX, Hurd, 
IRIX, Linux, LynxOS, Minix, QNX, SE­Linux, Solaris, 
System V, Triance, TRUSIX, Tru64, UnixWare, VSTa, 
z/OS, etc.
Embedded:  BeOS, Chorus OS, eCos, FreeRTOS, 
IOS, JUNOS, LynxOS, QNX, VRTX, VxWorks, 
Windows CE, RTLinux, RTAI, Symbian, etc.
Others:  AOS, JavaOS, MorphOS, Primos, Windows 
3.1/95/98/NT/XP/2000/2003, etc.

4
 What is Unix?
A portable, multi-tasking and multi-user operating system
 Portable: runs on many different hardware
architectures (Intel x86 and IA-64, Alpha, MIPS, HP
PA-RISC, PowerPC, IBM S/390, SPARC, Motorola
680x0, etc.).
 Preemptive multi-tasking: several programs can run
at the same time (time slices, interrupts, and task
switching).
 Multi-user: many users can share the computer
system at the same time.

5
 What is Unix? Other Features
 Uses a simple, uniform file model which includes devices
and access to other services in a flexible, hierarchical file
system.
 Written in a high-level language (“ C” ) making it easy to
read, understand, change and port.
 The command prompt is a simple user process, the Unix
shell, which is also a convenient job programming
language.
 Includes support for regular expressions which are
convenient for complex searching.

6
 Why Do I Need to Know This?
 Ubiquitous: Most big computers and much of the Internet
infrastructure runs on some variant of Unix (SUN, SGI, HP, etc.).
 Linux is the fastest growing operating system in the market.
 Source code availability provides the ability to “ get under the
hood” of operating system design and function (and TCP/IP).
 By understanding the Unix/Linux community and its culture, it is
easy to collect critical information.

7
 HISTORY
 Multics: 1965 Bell with General Electronic Company and
Massachusetts Institute of Technology.GE645. Multiuser, Large
data storage & power, data sharing.
 UNIX: 1969 Thompson and Ritchie at AT&T Bell Labs – Unix File
system – Thompson (Space travel program) – PDP 7
 UNIX moved to PDP – 11 in 1971
 UNIX in C (Ritchie)
 1977 – UNIX ported into non PDP machine , Interdata 8/32
 BSD: 1978 Berkeley Software Distribution- UNIX system III –
V(1983)
 Commercial Vendors: Sun, HP, IBM, SGI, DEC
 GNU: 1984 Richard Stallman, FSF
 POSIX: 1986 IEEE Portable Operating System unIX
 Minix: 1987 Andy Tannenbaum
 SVR4: 1989 AT&T and Sun
 Linux: 1991 Linus Torvalds Intel 386 (i386)
 Open Source: GPL, LGPL, Cathedral and the Bazaar
 Unix History
 1965 joint project between AT&T Bell Labs,
GE(General Electric company), and
MIT(Massachusetts Institute of Technology) to
develop a new OS.

 Goal : develop an OS that could provide

computational power, data storage and the ability to
share data among multiple users.

 Result: Multiplexed Information & Computer Service

- MULTICS.

9
 Unix History

 1969 Bell Labs withdraws from group.

 Two Bell Lab scientists, Ken Thompson and Dennis

Ritchie, continue research. They were still left
without a “ Convenient interactive computing
service” *.

* Ritchie, D.M. “ The Evolution of the Unix Time-sharing System” , AT&T Bell Laboratories Technical Journal,
Oct. 1984, Vol 63, No.8, Part 2, pp. 1577-1594.

10
 Unix History
 At the same time Ken Thompson
wrote a game “ space travel” in
Fortran to run on GECOS OS
(Honeywell 635).

 The spaceship was hard to control

and it was expensive to run. He
was told to get the game off his
work computer.

11
 Unix History
 Thompson ported the game to a little used PDP-7
computer.

 Unics (later Unix) was born as a pun on Multics.

12
Unix History
PDP-7

13
 Unix History

 Dennis Ritchie developed

“ B” . Then wrote “ C” a
compiled language.

 In 1973 entire OS ported to

“ C” .

14
 Unix History
➲ Because of a 1956 Consent Decree AT&T 
could not market Unix so it provided it to 
academia.
➲ Late 70s : Thompson took a sabbatical to 
teach Unix at UC Berkley – 

        Birth of BSD Unix. Introduced many new 
features.
➲ AT&T Bell Labs realized the commercial 
potential and began distributing System V.
➲ Commercialization of Unix (70s / 80s)

    AT&T, Sun, SGI, HP, DEC, NCR, IBM.
15
 Linus Torvalds

 1991 Linux 0.02 is first released to the public.
 1994 Linux 1.0 is released.

16
 Three Definitions of Linux
Linux Kernel: The very low-level software that manages
your computer hardware and provides a library (POSIX)
interface for user-level software. The Linux kernel runs on
many platforms (Intel x86 and IA-64, Alpha, MIPS, HP PA-
RISC, PowerPC, IBM S/390, SPARC, Motorola 680x0, etc.).

GNU/Linux OS: The Linux kernel plus utility software to

provide a useful working environment.

Linux Distributions: The packaging of the Linux Kernel,

the GNU/Linux OS and lots of other software to make Linux
easy to install, configure, and use (at least for the target
audience).
Tux, the Linux Mascot
 What is UNIX?

 Unix is a Operating System that popular since 1969.

 System divided into two parts
 First parts-programs and services such as shell, mail,
text processing packages and source code control
systems
 Second parts-system calls and hardware for supports to
first parts
 Example is UNIX system V produced by AT & T.
 Now Unix is maintained by Open Group.
 Unix Features

 High level language ‘ C’

 Simple user interface
 Complex programs built from simpler programs
 Hierarchical file system
 Consistent format for files-byte streams
 Consistent interface to peripheral devices.
 Multi user, multi process system
 Hides machine architecture – portable
 Even though Unix is in ‘ C’ we can run other languages like cobol,
fortran, ada, lisp, prolog, basic, pascal, java with their Compiler or
interpreter.
 Linux Features

 “ UNIX-like” operating system

 “ aims at” standards compliance
 “ all the features you would expect in a modern UNIX”
 preemptive multitasking
 virtual memory (protected memory, paging)
 shared libraries
 demand loading, dynamic kernel modules
 shared copy-on-write executables
 TCP/IP networking
 other features:
 SMP support, large memory, large files
 advanced networking, advanced filesystems
 efficient, stable, highly portable, supports most device hardware
 active development community, support, documentation, open source
 GUIs, applications
 Architecture of Unix System

emacs
• OS interacts directly with 
sh who   the hardware 
kernel date • Such OS is called 
cpp   system kernel
ed
cc as hardware
ld wc
grep
nroff
Other apps
 Unix: Structure

• Operating System: a system that manages the resources
   of a computer.
• Resources: CPUs, Memory, I/O devices, Network
• Kernel: the memory resident portion of Unix system
• File system and process control system are two major
  components of Unix Kernel.
 Unix System Kernel

• Three major tasks of kernel:
 Process Management
 Device Management
 File Management
• Three additional Services for Kernel:
✯ Virtual Memory
✯ Networking
✯ Network File Systems
• Experimental Kernel Features:
✱ Multiprocessor support
✱ Lightweight process (thread) support
 Block Diagram of System Kernel

User Programs
Libraries
User Level
Kernel   System Call Interface
Level
Inter­process
File Subsystem Process communication

control Scheduler
Device drivers Memory
subsystem
management
hardware control

hardware 
Hardware Level
 Process Control Subsystem

• Process Synchronization
• Interprocess communication
• Memory management: 
• Scheduler: process scheduling 
(allocate CPU to Processes)
 File subsystem

• A file system is a collection of files and directories on
  a disk or tape in standard UNIX file system format. 
• Kernel’s file sybsystem regulates data flow between
  the kernel and secondary storage devices. 
 Hardware Control

• Hardware control is responsible for handling interrupts
   and for communicating with the machine.
• Devices such as disks or terminals may interrupt the 
   CPU while a process is executing.
• The kernel may resume execution of the interrupted 
   process after servicing the interrupt.
 System Call

• A process accesses system resources through system call.
• System call for
❂ Process Control: 
     fork: create a new process
     wait: allow a parent process to synchronize its 
execution with the exit of a child process.
     exec: invoke a new program.
     exit: terminate process execution
❂ File system:
    File: open, read, write, lseek, close
    inode: chdir, chown chmod, stat fstat
    others: pipe dup, mount, unmount, link, unlink 
User Perspective-File system
 Architecture of Unix
Unix file system

root (/)

bin usr
dev etc
var
home tmp bin
spool log X11R6
(users’ accounts) mail

(users’ mail) 31
 Typical Directories

➲  / :  Root of the tree. Where it starts.

➲ bin, sbin, usr/bin: software for the shells and 
most common Unix commands. 

➲ dev: short for devices, holds the files necessary 
to operate peripherals such as printers and 
terminals. 

➲ home: contains the home directories of users 
(/export/home on sun computers).
32
 Typical Directories

tmp: holds temporary files.

var: contains files that vary in size; (Mail

directories, printer spool files, logs, etc.)

etc: administrative files such as lists of user names

and passwords.

33
 Typical Directories

 usr: Contains application programs

 lib: Contains libraries for programs

 proc: a pseudo-filesystem used as an interface to

kernel data structures.

34
 Unix File System

 File system is organised as a heirarchy of directories

 It starts from a single directory called root(represented
by a /).
Types

 Ordinary files
 Directories
 Special files
 Pipes
 Directory

 Directory is file containing list of files and subdirectories

 It has fixed size records of 16 bytes each which contains
- a filename(14 bytes)
- an inode number (2 bytes) which acts as a pointer
to where the system can find info about the file.
 Special Files

 Special files are contained in the directory /dev.

 They are used to represent a real physical device such
as a printer, tape device etc.
 Ex: Special device - /dev/null(unwanted output can be
redirected).
Pipes
 UNIX allows us to link commands together using a pipe.
 The pipe acts as a temporary file which only exists to
hold data from one command until its read by another.

 ex: command1 | command2 | command3....

 Processes

• A program is an executable file.
• A process is an instance of the program in execution.
• For example: create two active processes
$ emacs &
$ emacs & 
$ ps
PID   TTY   TIME CMD
12893  pts/4 0:00 tcsh
12581  pts/4 0:01 emacs
12582  pts/4 0:01 emacs
$
 Processes

• A program is an executable file.
• A process is an instance of the program in execution.
• For example: create two active processes
$ emacs &
$ emacs & 
$ ps
PID   TTY   TIME CMD
12893  pts/4 0:00 tcsh
12581  pts/4 0:01 emacs
12582  pts/4 0:01 emacs
$
 Processes

• A process has 
 text: machine instructions  
(may be shared by other processes)
 data
 stack
• Process may execute either in user mode and in kernel
   mode.
• Process information are stored in two places:
❋ Process table 
❋ User table
 System call: fork()

• fork: the only way for a user to create a process in Unix
   operating system.
• The process that invokes fork is called parent process
   and the newly created process is called child process.
• The syntax of fork system call:
newpid = fork();
• On return from fork system call, the two processes have 
   identical copies of their user­level context except for the
   return value pid. 
• In parent process, newpid = child process id
• In child process, newpid = 0;
 $ cc forkEx1.c ­o forkEx1
$ forkEx1
/* forkEx1.c */ Before forking ...
#include <stdio.h> Child Process fpid=0
After forking fpid=0
main() Parent Process fpid=14707
{ After forking fpid=14707
   int fpid;  $
   printf("Before forking ...\n");
   fpid = fork();
   if (fpid == 0) {
      printf("Child Process fpid=%d\n", fpid);
   } else {
      printf("Parent Process fpid=%d\n", fpid); 
   }
   printf("After forking fpid=%d\n", fpid);   
}
 $ cc forkEx1.c ­o forkEx1
$ forkEx1
/* forkEx1.c */ Before forking ...
#include <stdio.h> Child Process fpid=0
After forking fpid=0
main() Parent Process fpid=14707
{ After forking fpid=14707
   int fpid;  $
   printf("Before forking ...\n");
   fpid = fork();
   if (fpid == 0) {
      printf("Child Process fpid=%d\n", fpid);
   } else {
      printf("Parent Process fpid=%d\n", fpid); 
   }
   printf("After forking fpid=%d\n", fpid);   
}
 $ forkEx2
/* forkEx2.c */
Before forking ...
#include <stdio.h>
   PID TTY      TIME CMD
 14759 pts/9    0:00 tcsh
main()
 14778 pts/9    0:00 sh
{
 14777 pts/9    0:00 forkEx2
   int fpid; 
   PID TTY      TIME CMD
   printf("Before forking ...\n");
 14781 pts/9    0:00 sh
   system("ps");
 14759 pts/9    0:00 tcsh
   fpid = fork();
 14782 pts/9    0:00 sh
   system("ps");
 14780 pts/9    0:00 forkEx2
   printf("After forking 
 14777 pts/9    0:00 forkEx2
fpid=%d\n", fpid);
After forking fpid=14780
}
$    PID TTY      TIME CMD
$  ps  14781 pts/9    0:00 sh
   PID TTY      TIME CMD  14759 pts/9    0:00 tcsh
 14759 pts/9    0:00 tcsh  14780 pts/9    0:00 forkEx2
$ After forking fpid=0 
 System Call: getpid() getppid()

• Each process has a unique process id (PID). 
• PID is an integer, typically in the range 0 through 30000.
• Kernel assigns the PID when a new process is created.
• Processes can obtain their PID by calling getpid().
• Each process has a parent process and a corresponding 
   parent process ID.
• Processes can obtain their parent’s PID by calling 
   getppid().
/* pid.c */
#include <stdio.h>
#include <sys/types.h>
#include <unistd.h>

main()
{
   printf("pid=%d ppid=%d\n",getpid(), getppid());
}
$ cc pid.c ­o pid
$ pid
pid=14935 ppid=14759
$
/* forkEx3.c */
#include <stdio.h>
#include <sys/types.h>
#include <unistd.h>
main()
{
   int fpid; 
   printf("Before forking ...\n");
   fpid = fork();
   if (fpid == 0) {
      printf("Child Process fpid=%d pid=%d ppid=%d\n", 
       fpid, getpid(), getppid());
   } else {
      printf("Parent Process fpid=%d pid=%d ppid=%d\n", 
       fpid, getpid(), getppid());
   }
   printf("After forking fpid=%d pid=%d ppid=%d\n", 
       fpid, getpid(), getppid());
}
$ cc forkEx3.c ­o forkEx3
$ forkEx3
Before forking ...
Parent Process fpid=14942 pid=14941 ppid=14759
After forking fpid=14942 pid=14941 ppid=14759
$ Child Process fpid=0 pid=14942 ppid=1
After forking fpid=0 pid=14942 ppid=1

$ ps 
   PID TTY      TIME CMD
 14759 pts/9    0:00 tcsh
 System Call: wait()

• wait system call allows a parent process to wait
   for the demise of a child process. 
#include <stdio.h>
#include <sys/types.h>
#include <unistd.h>
main()
{
   int fpid, status; 
   printf("Before forking ...\n");
   fpid = fork();
   if (fpid == 0) {
      printf("Child Process fpid=%d pid=%d ppid=%d\n", 
fpid, getpid(), getppid());
   } else {
      printf("Parent Process fpid=%d pid=%d ppid=%d\n", 
fpid, getpid(), getppid());
   }
   wait(&status);
   printf("After forking fpid=%d pid=%d ppid=%d\n", 
fpid, getpid(), getppid());
}
$ cc forkEx4.c ­o forkEx4
$ forkEx4
Before forking ...
Parent Process fpid=14980 pid=14979 ppid=14759
Child Process fpid=0 pid=14980 ppid=14979
After forking fpid=0 pid=14980 ppid=14979
After forking fpid=14980 pid=14979 ppid=14759
$ 
 System Call: exec()

• exec() system call invokes another program by replacing
  the current process
• No new process table entry is created for exec() program.
   Thus, the total number of processes in the system isn’t 
   changed.
• Six different exec functions:
   execlp, execvp, execl, execv, execle, execve,
   (see man page for more detail.)
• exec system call allows a process to choose its successor.
/* execEx1.c */
#include <stdio.h>
#include <unistd.h>

main()
{
   printf("Before execing ...\n");
   execl("/bin/date", "date", 0);
   printf("After exec\n");   
}

$ execEx1
Before execing ...
Sun May  9 16:39:17 CST 1999
$
/* execEx1.c */
#include <stdio.h>
#include <unistd.h>

main()
{
   printf("Before execing ...\n");
   execl("/bin/date", "date", 0);
   printf("After exec\n");   
}

$ execEx1
Before execing ...
Sun May  9 16:39:17 CST 1999
$
/* execEx2.c */
#include <sys/types.h>
#include <unistd.h> 
#include <stdio.h> $ execEx2
Before execing ...
main() After exec and fpid=14903
{ $ Sun May  9 16:47:08 CST 1999
   int fpid; $
   printf("Before execing ...\n");
   fpid = fork();
   if (fpid == 0) {
     execl("/bin/date", "date", 0);
   }
   printf("After exec and fpid=%d\n",fpid);   
}
/* execEx2.c */
#include <sys/types.h>
#include <unistd.h> 
#include <stdio.h> $ execEx2
Before execing ...
main() After exec and fpid=14903
{ $ Sun May  9 16:47:08 CST 1999
   int fpid; $
   printf("Before execing ...\n");
   fpid = fork();
   if (fpid == 0) {
     execl("/bin/date", "date", 0);
   }
   printf("After exec and fpid=%d\n",fpid);   
}
 Building Block Primitives

 User to create complex program effectively.

 Users capability to
 (1)redirect I/O.
 User much access three files
 Std input file

 Std output file

 Std error file

 Example: ls, ls > ssn.txt, mail x < letter, 2>(err)

 (2)Pipe: mechanisms allows a stream of data to be

passed between reader and writer processes.
 Example: grep ssn file1.c file2.c file3.c file4.c | wc -l
 < file - means open a file for reading and associate with STDIN.
 << token - Means use the current input stream as STDIN for
the program until token is seen.
 > file - means open a file for writing and truncate it and
associate it with STDOUT.
 >> file - means open a file for writing and seek to the end and
associate it with STDOUT. This is how we append to a file using
a redirect.
 n>&m means redirect FD n to the same places as FD m. Eg,
2>&1 means send STDERR to the same place that STDOUT is
going to.
 FD 0 – STDIN , FD 1 – STDOUT, FD 2 -STDERR
 Operating System Services

 Controlling
 Scheduling
 Memory Management
 Allocating to main memory
 Allocating to secondary memory
 Process management – controlled access to devices
 Transparent Service – Kernel hides the type of file from
user.
 ASSUMPTIONS ABOUT H/W:User mode and Kernel
mode

• At any given instant a computer running the Unix system
   is either executing a process or the kernel itself is running
• The computer is in user mode when it is executing 
   instructions in a user process and it is in kernel mode
   when it is executing instructions in the kernel.
• Executing System call ==> User mode to Kernel mode
   perform I/O operations
   system clock interrupt
   
 Difference between two modes

 Process in user mode can access their own instruction

not kernal instructions and data of other processes but in
kernal mode can access both.
 Example: virtual address space access only in kernal mode.
 Some machine instructions are privileged.
 Example: the process status register is not capable to access
from the user mode.

 Kernel runs on behalf of the user processes. Part of

each user process.
 Interrupts and Exceptions

 Interrupts: for multitasking CPU is shared by many

processes
 Blocks low priority interrupts but services high priority
interrupts.
 Exception refers to unexpected events caused by
process.
 Example: addressing illegal memory, executing privileged
instructions and dividing by zero.
 Restart the instruction after handling exception.
 Exceptions in middle of an instruction whereas interrupts
in between 2 instructions.
 Processor Execution Levels

 Kernal prevent the occurrence of interrupts during critical

activity.
 Manipulating linked list pointers
 High priority interrupts
 Machine Errors
 Clock
 Disk
 Lower Priority interrupts
 Network Devices
 Terminals
 Software Interrupts
 Masking a level will block below and allow above.
 Memory Management

 Kernal resides in main memory permanently

 Compiler generates virtual address space
 Kernal Mapping virtual address to physical address