DESIGN AND

ANALYSIS O
OF
ALGORITHMS
HM

s
es
y Pr
sit
er

Associate
sociate
ciate Prof
Professor
P
iv

Department of Information
formatio Science and Technology
Un

Collegee off Enginee

Engineering, Guindy Campus
Engi
Anna UnUniversity, Chennai
d
or
xf
O

KhW
 
Oxford University Press is a department of the University of Oxford.
It furthers the Universitys objective of excellence in research, scholarship,
and education by publishing worldwide. Oxford is a registered trade mark of
Oxford University Press in the UK and in certain other countries.

Published in India by
Oxford University Press
YMCA Library Building, 1 Jai Singh Road, New Delhi 110001, India

Oxford University Press 2014

The moral rights of the author/s have been asserted.

First published in 2014

All rights reserved. No part of this publication may be reproduced,

ed, stored
store in
a retrieval system, or transmitted, in any form or by any means,
ns, without
withou the

ss
prior permission in writing of Oxford University Press, or as expressly
pressly permitted
pe
by law, by licence, or under terms agreed with the appropriate
priate reprographics
iate repro

re
rights organization. Enquiries concerning reproduction outside the scope of the
above should be sent to the Rights Department, Oxfordo d University
Universi Press, at the

yP
address above.

You must not circulate this work

k in any oth
sit other form
and you must impose this same condition
ndition on any
an acquirer.

ISBN-13: 978-0-19-809369-5
-0-19-809369-5
19-80936
er
ISBN-10:: 0-19-809369-1
-19-80936
iv

Typesetet in Times New Roman

by Ideal
all Publishing
Publishin Solutions,
S Delhi
Un

Printed in India
ndia by Radha
Rad Press, New Delhi 110031

Third-party website
ite addresses
ebsite a dresses
dres mentioned in this book are provided
by Oxford University
versity Press
P ss in good faith and for information only.
d

Oxford University Press disclaims

disclaim any responsibility for the material contained therein.
or
xf
O

KhW
 Features of the Book
C H A P T E R

Topical Coverage
Introducon to
Algorithms
1 C H A P T E R
The book provides
extensive coverage of

18 Basics of Computaonal
Complexity C H A P T E R



Data StructuresI 5 C H A P T E R

11 Greedy Algorithms
design techniques, Bad programmers worry about the code. Good programmers worry about data structures

followed by discussions


s
WZD C Hby Athe way...
Greed is all right, P T I think
E Rgreed is healthy. You can be greedy

13

es
d till f l d b t lf

Dynamic Programming

y Pr
sit
Treatment of Concepts
er
11.8.1 M S T
The history of an MST is as interesting as its concept. In 1926, Otaker
er Boruvka formulated
formula
form ^
the MST problem. A Polish mathematician, Vojtech Jarnik, described bed
d the probl
problem in 1929
iv

in a letter to Otaker Boruvka. The same problem was conceived ed independently

ndependen by Kruskal

in 1956. +ence, Kruskal rediscovered the problem. /ater it was as dened independently
iindep by are provided using
Un

Robert Prim in 1957 and by Edsger Dijkstra in 1958 

Example 11.12 Consider the graph G shown sh wn in Fig.
Fig 11.11. Construct an MST for the
given graph G using Kruskals algorithm. 
lgorithm is to sort all the edges and form an edge list,
6ROXWLRQ The rst step in Kruskals algorithm 
d

say E. The edges of graph G shown in Fig. 11.2

11.21 are sorted and shown in Table 11.13.

or

<

Since the disjoint set data structure is used, ed, initiali
initialization takes at most (|V|) time. The time


xf

complexity of the algorithm depends ends on the numnumber of edges. As there are |E| edges, O(| E
|log| E |) time is required to sortt these edge
edges. The disjoint set takes at most 2| E | nd opera- 
O

refore, th
tions and |V |1 operations. Therefore, the total complexity of Kruskals algorithm is at most
O(| E |log| E |) time. 

Algoritm Presentaon

Step 1: Create a node x by allocating memory for it.
in two ways, that is, step- Step 2: Assign the required value to the item part of node x.
wise approach item(x) = value
Step 3: Set in the pointer to null.
 next(x) = null
pseudocode approach Step 4: ReturnAlgorithm
the node x. create(L, x, value)
 %% Input: List L and element x with 'value'
 %% Output: Node x
Begin
understanding of the allot(x) %% Allot memory for node x with two eldsitem and next
item(x) = value
logic behind solving a next(x) = null





KhW
 Historical Notes Box 1.1 Origin of the word algorithm
The word algorithm is derived from the name of a Persian Europe. He also introduced the simple step-b
Historical notes are mathematician, Abu Jafer Mohammed Ibn Musa al for addition, subtraction, multiplication, and
provided throughout the Khowarizmi, who lived sometime around 780850 AD. He his book. The word algebra has also been d
was from the town of Khowarazm, now in Uzbekistan. He the title of this book. When his book was t
 Box 17.2 inGeorge
was a teacher of mathematics Bernard
Baghdad. Dantzig
He wrote a book Latin, his name was quoted as Algorismus
d *eorge Bernard Dantzig was born in 1914 at Portland, duality theory. He worked with Fulkerson an
Oregon, United States. His father became a professor of in formulating the travelling salesperson pro
 mathematics at University of Maryland after World War linear programming and solved the TSP proble
II. Dantzigs biggest contribution is that he designed the 49 cities at that time. In 197, he was award
 simplex method for solving LPPs. Apart from the simplex National Medal of Science, the highest honou

Glossary and Summary

Q GLOSSARY Q 
Agent A performer of an algorithm $OJRULWKPYHULFDWLRQ The process of provid 

Algorist A person who is skilled in algorithm development cal proof that the algorithm works correctly f



ss
Algorithm A step-by-step procedure for solving a given Algorithm validation The process of chec
problem ness of an algorithm, this is done by givin given at the end of each
giv
Algorithm gap The difference between lower and upper it and checking its results with expected
SUMMARY

re
Q Q
chapter to help readers
An algorithm is a step-by-step procedure for solving a Algorithm verication is a process
es of provv 

yP
given problem. ematical proof that the givenn algorithm
algorith
orit wo o 
A computational problem is characterized by two for all instances of data.
factorsspecication of valid input and output param- A proof of an algorithm
sit m is said to exist
exis if t
er
iv

Revie Yuesons Q REVIEW QUESTIONS Q

Eercises and
Un

1.1 Dene an n algorithm. 1.8 What is the difference between algo

1.2 at are the characteristics
What char of an algorithm" and algorithm validation"
Addional Prolems 1.3 S
Surveyy the Inte
Internet and list out at least ve algorithms 1.9 How is an algorithm validated and
h
that have huge impact on our daily lives. with an example.
E 4
1.4 a the stages of problem solving"
What are 1.10 State algorithm classications. Wha
EXERCISES Q
d

Q

1.1 Assume that there are two algorithms A and B for a complexities of A, B, and C are 3n
or

 given problem P. The time complexity functions of algo- respectively. Assume that the input i
 rithms A and B are, respectively, 5n and log2n. Which Assume that the machine executes
algorithm should be selected assuming that all other per second. How much time will algo
xf

pter
the end of every chapter conditions remain the same for both the problems" C take" Which algorithm will be the
h ADDITIONAL diPROBLEMh
Q Q
12 L f l h
er
to test the readers
O

conceptual knowledge ge
and also enhance their

1.1 John MacCormick had written a book titled Nine recognition, data compression, datab
Algorithms That Changed the Future: The Ingenious signatures, that changed the world.
Ideas that drive Todays Computers, Princeton University (a) What are these algorithms" Se
Press, Princeton, that had listed the nine wonderful and nd what these algorithms
algorithms, namely, search engine indexing, page rank, (b) Identify one more algorithm that

Q CROSSWORD Q
1 Crossword Puzzles
2
3
Crossword puzzles,
4 5 


exercise at the end of each
7
8 
9 10 readers to self-check their

11

KhW
 W
Computers have become an integral part of our daily life in recent times. They have enormously impacted
our personal, professional, as well as social lives. Computers help us in tasks such as document editing,
Internet browsing, sending emails, making presentations, performing complex scientic computations,
social networking, and playing games. Industries and government ofces use computers effectively in
production, e-governance, and e-commerce. Considering the increasing demand of computers in society,
schools, colleges, and universities have included computer education in their curriculum, to help students
become skilled in programming and developing applications which can be used to solve various business,
scientic, and social problems.
Programming is a process of converting a given problem into an executable utable
able co
code for the computer. It

ss
involves understanding, analysis, and solving problems to create an algorithm. rithm.
hm. Veri
Verication of the algorithm,
coding of the algorithm in a specic programming language, testing, ing, debu
debug
debugging, and maintaining the

re
source code are also part of the programming process. Therefore, in n order to construct efcient programs,

yP
a ne understanding of algorithms is essential. An algorithm is a sset et of
o llogical instructions for solving a
given problem. It is expected to give correct results for valid alid inputs
input aand should be efcient, consuming
less computer resources. An algorithm is implemented ass a program using a programming language. A
sit
well-designed algorithm runs faster and consumes lesser sser com
compu
computer resources, namely time and space.
Therefore, expertise in programming is more related atedd to efc
efcie
efciency in problem-solving and effective de-
er
signing of algorithms, rather than developing codes with th the help of programming languages. Though
programming languages are important, theirr role ole is ju
just li
limited to the implementation of a well-designed
iv

algorithm. For this reason, algorithms aree a central th theme of computer study.
Un

In fact, history of algorithms is much ch older tha

than that of computers, dating back to 3000 BC. The
vilizat
lizat on w
ancient people of the Sumerian civilization wer
were aware of basic numeric computation like addition. A
uphrates
rates riv
Sumerian tablet found in the Euphrates river showed how to partition a given quantity of wheat in a way
d

that each person receives thehe specie

specied quantity. Such tablets were also used by ancient Babylonians
or

ematicians
aticians oof this period such as Euclid, Al-Khwarizmi, Leonardo Pisano (also
(2000 to 1650 BC). Mathematicians
known as Fibonacci), and nd others produced
p procedures that provided the foundations of the concepts of
xf

velopment
elopmen in the eld of algorithms were due to the contributions of Gottfried
algorithms. Later developments
O

rt, and Alan Turing. The history of modern computers, however, only starts from
Leibnitz, David Hilbert,
the 1940s. Thus, algorithms have played a very important role in the development of modern computing.
The study of algorithms, called algorithmics, includes three aspectsalgorithm design, analysis, and
computational complexity of problems. Algorithm design is a creative activity. It includes various techniques
(such as divide-and-conquer, greedy approach, dynamic programming, backtracking) that help in producing
outputs at a faster pace by consuming lesser computer resources. Algorithm analysis is the estimation of
how much resource is required by the algorithm. Computational complexity deals with the analysis and
solvability of problems itself. A compulsory course on algorithms design and analysis is generally offered
to computer science and information technology students in most universities. The course aims to help
students create efcient algorithms in common engineering design situations and analyse the asymptotic
performance of algorithms by using the important algorithmic design paradigms and methods of analysis.

ABOUT THE BOOK

Design and Analysis of Algorithms is designed to serve as a textbook for the rst level course in algorithms
that discusses all the fundamental and necessary information related to the three important aspects of

KhW
 Preface vii

algorithm study using minimal mathematics and lucid language. This book is suitable for undergradu-
ate students of computer science and engineering (CSE) and information technology (IT), as well as
for postgraduate students of computer applications. It is also useful for diploma courses, competitive
examinations (like GATE), and recruitment interviews for this subject.
The book begins with an introduction to algorithms and problem-solving concepts followed by an
introduction to algorithm writing and analysis of iterative and recursive algorithms. In-depth explana-
tions and designing techniques of various types of algorithms used for problem-solving such as the
brute force technique, divide-and conquer-technique, decrease-and-conquer strategy, greedy approach,
transform-and-conquer strategy, dynamic programming, branch-and-bound approach, and backtracking,
are provided in the book. Subsequent chapters of the book delve into the discussion of string algorithms,
iterative improvement, linear programming, computability theory, NP-hard problems, NP- completeness,
probability analysis, randomized algorithms, approximation algorithms, and parallel algorithms, with
the appendices throwing light on basic mathematics and proof techniques.
The various design techniques have been elucidated with the help off numer numerous problems, solved
numero

ss
examples, and illustrations (including schematics, tables, and cartoons).. The algor
algorithms are presented in
plain English (informal algorithm presentation) and pseudocode approach proach (formal algorithm presenta-
oach (for

re
tion) to make the book programming language-independent and easy-to-comprehend.
sy-to-com The book includes
a variety of chapter-end pedagogical features such as point-wise isee ssummary,
umm
umma glossary, review questions,

yP
exercises, and additional problems to help readers assimilate
tee and im implement
imp l the concepts learnt.
sit
KEY FEATURES
er
Provides simple and coherent explanations without ithout
out using
usin eexcessive theorems, proofs, and lemmas
Detailed coverage for topics such as greedy dynamic programming, transform-and-conquer
y approach, dy
iv

technique, decrease-and-conquer technique, que, linea programming, and randomized and approximation
linear pr
Un

algorithms
Dedicated chapters on backtracking ng and
ing branch-and-bound techniques, string matching algorithms,
nd bra
branc
and parallel algorithms
d

Simple and judicious presentationation of algorithms throughout the text in both informal and formal
entation
or

forms, followed by the discussion

scussion oof their complexity analysis
Numerous review questions,
estions, exercises, and additional problems given at the end of each chapter to
ns, exe
xf

help readers apply and practis

practise the concepts learnt
Includes glossary and poi point-wise summary at the end of each chapter to help readers quickly
O

recapitulate the important

rtan concepts
Provides historical notes on various topics and crossword puzzles at the end of each chapter to elicit
learning interest in students

ORGANIZATION OF THE BOOK

The book consists of twenty chapters. A chapter-wise scheme of the book is presented here.
Chapter 1 provides an overview of algorithms. It introduces all the basic concepts of algorithms and
the fundamental stages of problem-solving. The chapter ends with the classication of algorithms.
Chapter 2 starts with the basic tools used for problem-solving. All the guidelines required for present-
ing the pseudocode and ow charts are provided along with many examples. The focus of this chapter
is to provide some practice on writing algorithms. The basics of recursion and algorithm correctness are
also covered in this chapter.
Chapter 3 covers the basics of algorithm complexity and analysis of iterative algorithms. Step count
and operation count methods used for analysing iterative algorithms are discussed in detail. Asymptotic

KhW
viii Preface

analysis is also discussed in detail. Finally, the chapter ends with the concept of analysing the efciency
of algorithms.
Chapter 4 discusses the analysis of recursive algorithms. It explains the basics of recurrence equations
along with the methods for solving them. Generating functions are also briey discussed as part of this chapter.
Data structures are an important component of algorithms. Chapter 5 deals with the fundamentals
concepts related to data structures such as stacks, queues, linear lists, and linked lists. Trees and graphs
are also explained in this chapter. Chapter 6 covers advanced data structures used for organizing
large amounts of dynamic data. Binary search trees and AVL trees used for organizing dictionaries
are discussed in this chapter, in addition to priority queues and heaps. Finally, a discussion on disjoint
sets and amortized analysis is provided.
Chapter 7 deals with the brute force techniques, which use no special logic but instead follow an
intuitive way of solving problems using the problem statement. Various problems such as sequential
search, bubble sort, and selection sort that can be solved using this approachh are a explained. Some basic
computational geometry problems such as closest-pair and convex hull aree also covered. The chapter

ss
ends with the discussions on exhaustive searching problems such as 15-puzzle 5-puzzle
puzzle pproblem, 8-queen prob-
lem, magic square problem, knapsack problem, container loading problem, and assignment problem.
blem, an

re
Chapter 8 discusses the divide-and-conquer design paradigm. Important problems such as quicksort,
merge sort, nding maximum and minimum, multiplication off lon long integers, Strassen matrix multiplica-
ong int
integ

yP
tion, tiling problem, closest-pair problem, and convex hull are solv solved
solved using this technique. The chapter
ends with a discussion of the Fourier transform problem. m.
sit
Chapter 9 explains the decrease-and-conquer technique, hnique, which is also known as the incremental or
ique, w
inductive approach. Examples problems such as insertion nsertion sor ttopological sort, generating permutations
rtion sort,
er
and subsets, binary search, fake coin detection, andd Russian
Russi peasant multiplication problem are used to
illustrate the decrease by constant and constant methods. Finally, discussions on interpolation search,
nt factor m
metho
iv

selection, and nding the median problems ms using ththe ddecrease by variable factor method are provided.
Un

Chapter 10 deals with timespace tradeoffs. The selection of one type of efciency over the other
radeoffs. T
Th
and problems related to linear sorting ngg aand Hashing are discussed. The chapter ends with a discussion on
d Has
Hashin
B-trees and their operations.
d

Chapter 11 explains the greedy

eedy approach concept. This chapter discusses important problems such as
dy approa
or

coin change problem, scheduling,

duling, knapsack problem, optimal storage of tapes, Huffman code, minimum
uling, kna
spanning tree algorithms, Dijkstras shortest path algorithm to illustrate the greedy approach.
ms, and Dijk
xf

Chapter 12 discusses
sses transform-and-conquer approach and its three basic techniquesinstance
ses the tran
tra
entatio change, and problem reduction. Problems such as Gaussian elimination,
entation
simplication, representation
O

decomposition methods, nding determinant and matrix inverses are discussed. Heap sort, Horners
method, binary exponentiation algorithm, and reduction problems are also covered in this chapter.
Chapter 13 deals with dynamic programming. Important example problems such as Fibonacci prob-
lem, binomial coefcient multistage graph, Graph algorithms, Floyd-Warshall Algorithm, BellmanFord
algorithms, travelling salesman problem, chain matrix multiplication, knapsack problem, and optimal
binary search tree problem are discussed to illustrate the dynamic programming concept. Finally, the
chapter concludes with the ow-shop scheduling algorithms.
Chapter 14 discusses backtracking algorithms. This chapter covers important problems such as N-queen
problem, Hamiltonian circuit problem, sum of subsets, vertex colouring problem, graph colouring prob-
lems, Graham scan, and generating permutations.
Chapter 15 explains branch-and-bound techniques. Search techniques using this concept are discussed.
Important problems such as assignment problem and 15-puzzle are covered and the chapter ends with a
discussion on traveling salesperson and knapsack problems.
Chapter 16 deals with the string algorithms. Some basic string algorithms such as nding the length
of strings, nding substrings, concatenation of two strings, longest common sequence, and pattern

KhW
 Preface ix

recognition algorithms such as RabinKarp, Harspool, KnuthMorrisPratt and BoyerMoore algorithms

are discussed in this chapter. Finally, approximate string matching algorithm is discussed.
Chapter 17 discusses the iterative approach and basics of linear programming. The linear programming
formulation of a problem and simplex method is discussed in this chapter. Minimization problem, prin-
ciple of duality, and max-ow problems are also explained. Finally, matching algorithms are considered
for better understanding of computational complexity.
Chapter 18 explains the basics of computational complexity and the upper and lower bound theory.
Decision problems, complexity classes, and reduction concepts are also discussed. This chapter also
covers theory of NP-complete problems and examples for proving NP-completeness.
Chapter 19 covers the basic concepts and types of both randomized and approximation algorithms.
Randomized algorithms are illustrated through examples such as hiring problem, primality testing, com-
parison of strings, and randomized quicksort. Approximation algorithms are illustrated through examples
based on heuristic, greedy, linear, and dynamic programming approaches.
Chapter 20 begins with an introduction to parallel processing and classication
classic
ssic of parallel sys-

ss
tems. It then discusses the fundamentals of parallel algorithms and parallel rallel ran
random access machine
(PRAM) model. The concept of parallelism is illustrated through examples xamples related to parallel search-
mples re

re
ing, parallel sorting, and graph and matrix multiplication problems. lems.
There are two appendices in this book. Appendix A explains thee ba
ns th basics of mathematics such as sets,
basic

yP
series and sequences, relations, functions, matrix algebra, and probability
d probab
probabil i that are necessary for algorithm
study. Appendix B deals with mathematical logic and proof techniques.
ooff techniqu
technique
sit
ONLINE RESOURCES
er

To aid teachers and students, the book is accompanied

ompanied
mpanied by online resources that are available at http://
iv

nalysis-Alg
nalysis-Algori
oupinheonline.com/book/sridhar-Design-Analysis-Algorithms/9780198093695. The content for the online
resources are as follows:
Un

For Instructors
d

PowerPoint slides
or

Solutions manual
xf

For Students
Answers to the crossword
ssword
sword ppuzzles
O

ACKNOWLEDGEMENTS
This book would not have been possible without the help and encouragement of many friends, colleagues,
and well-wishers. I thank all my students who motivated me by asking interesting questions related to
the subject. I acknowledge the assistance of my friend, P. Kanniappan, in typing the manuscripts and for
all his advice. I also express my gratitude to all my colleagues at the departments of Computer Science
and Engineering and Information Science and Technology, Anna 8niversity and National Institute of
Technology, Tiruchirapalli for reviewing my manuscripts and providing constructive suggestions for
improvement. I acknowledge all my reviewers whose feedback has helped making this book better. I
am also very thankful to the editorial team at Oxford 8niversity Press, India for providing valuable as-
sistance. My sincere thanks are also due to my family members, especially my wife, Dr N. Vasanthy, my
mother, Mrs Parameswari, my mother-in-law, Mrs Renuga, and my children, Shobika and Shreevarshika,
for their constant support and encouragement during the development of this book.
S. Sridhar

KhW
 Brief Contents
Features of the Book iv
Preface vi
Detailed Contents xi

1 Introduction to Algorithms 1
2 Basics of Algorithm Writing 22
3 Basics of Algorithm Analysis 58
4 Mathematical Analysis of Recursive Algorithms 98

ss
5 Data StructuresI 141

re
6 Data StructuresII 194

yP
7 Brute Force Approaches 231
8 Divide-and-conquer Approach 262
sit
9 Decrease-and-conquer Approach 309
er
10 TimeSpace Tradeoffs 342
iv

11 Greedy Algorithms 372

Un

12 Transform-and-conquer Approach 417

13 Dynamic Programming 455
d

14 Backtracking 517
or

15 Branch-and-bound Technique
chniqu 543
xf

16 String Algorithms
mss 569
O

17 Iterative Improvement and Linear Programming 601

18 Basics of Computational Complexity 638
19 Randomized and Approximation Algorithms 663
20 Parallel Algorithms 705

Appendix AMathematical Basics 735

Appendix BProof Techniques 752
Bibliography 763
Index 766

KhW
 Detailed Contents
Features of the Book iv
Preface vi
Brief Contents x

1 Introducon to Algorithms 1 2.3 Non-recursive Algorithms 31

1.1 Introduction 1 2.3.1 Flowcharts 32
1.2 Need for Algorithmic Thinking 1 2.4 Basics of Recursion 42
1.3 Overview of Algorithms 3 2.5 Recursive Algorithms 43
1.3.1 Computational Problems, rre
2.6 Algorithm Correctness 52
Instance, and Size 5

ss
1.4 Need for Algorithm Efciency 7 3 Basics of Algorithm
gorithm
rithm AAn
Analysis 58
1.5 Fundamental Stages of Problem 3.1 Basics
cs of Algor
Algorithm Complexity 58

re
Solving 9 3.2 Introduction
roductio to Time complexity
roduction 60

yP
1.5.1 Understanding the Problem 9 3.2.
3.2.1
2 Ran
R
Random Access Machine 60
1.5.2 Planning an Algorithm 9 3.33 Anal
Analys
Analysis of Iterative Algorithms 62
1.5.3 Designing an Algorithm 11 3.3.1 Measuring Input Size 63
sit
1.5.4 Validating and Verifying an 3.3.2 Measuring Running Time
3.3 63
3.3.3 Best-, Worst-, and
er
Algorithm 11
1.5.5 Analysing an Algorithm 12 Average-case Complexity 71
iv

1.5.6 Implementing an Algorithm 3.4 Rate of Growth 73

Un

and Performing Empiricall 3.4.1 Measuring Larger Inputs 73

Analysis 14 3.4.2 Comparison Framework 74
1.5.7 Post (or Postmortem)
em) 3.5 Asymptotic Analysis 76
d

Analysis 14 3.5.1 Asymptotic Notations 78

or

1.6 Classication of Algorithms

gorithms 15 3.5.2 Asymptotic Rules 85
1.6.1 Based onn Implement
Implemen
Implementation 15 3.5.3 Asymptotic Complexity Classes 87
xf

1.6.2 Based onn Design 16 3.6 Space Complexity Analysis 88

O

1.6.3 Based on Area of 3.7 Empirical Analysis and Algorithm

Specialization 17 Visualization 88
1.6.4 Based on Tractability 18 3.7.1 Experimental Purpose 89
3.7.2 Statistical Tests 90
2 Basics of Algorithm Wring 22
2.1 Tools for Problem-solving 22 Mathemacal Analysis of Recursive
2.1.1 Stepwise 5enement or Algorithms 98
Top-down design 23 4.1 Introduction to Recurrence
2.1.2 Bottom-up Approach 24 Equations 98
2.1.3 Structured Programming 25 4.1.1 Linear Recurrences 99
2.2 Algorithm Specications 26 4.1.2 Non-linear Recurrences 101
2.2.1 Guidelines for Writing 4.2 Formulation of Recurrence
Algorithms 27 Equations 103

KhW
xii Detailed Contents

4.3 Techniques for Solving 6 Data StructuresII 194

Recurrence Equations 106 6.1 Introduction to Dictionary 194
4.3.1 Guess-and-verify Method 106 6.1.1 Introduction to Binary
4.3.2 Substitution Method 109 Search Tree 194
4.3.3 Recurrence-tree Method 112 6.1.2 AVL Trees 200
4.3.4 Difference Method 117 6.2 Priority Queues and Heaps 206
4.4 Solving Recurrence Equations 6.2.1 Binary Heaps 207
8sing Polynomial Reduction 118 6.2.2 Binomial Heaps 212
4.4.1 Solving Homogeneous 6.2.3 Fibonacci heap 218
Equations 118 6.3 Disjoint Sets 221
4.4.2 Solving Non-homogeneous 6.3.1 Representation and
Equations 122 Operations
ion 221
4.5 Generating Functions 123 6.4 Amortized d Analys
Analysis 224

ss
4.5.1 Properties of Generating 6.4.1 Aggregate
gregate MMethod 225
Functions 124 2 Accounti
6.4.2 Account
Accounting Method 226

re
4.6 Divide-and-conquer Recurrences 127 4.3 Poten
6.4.3 Pote
Potential Method 226

yP
4.6.1 Master Theorem 127
4.6.2 Transformations 133 7 Brute Fo
Forc
Force Approaches 231
4.6.3 Conditional Asymptotics 135 7.1 Intro
Introduction 231
sit
77.1.1 Advantages and
er
5 Data StructuresI 141
1 Disadvantages of Brute
5.1 Data Structures and Algorithms 141 Force Method 232
iv

5.2 Lists 142 7.2 Sequential Search 232

Un

5.2.1 Linear Lists and Arrays 14

143 7.2.1 Analysis of Recursion
5.2.2 Linked Lists 146
14 Programs 234
5.3 Stacks 152 7.2.2 Recursive Form of Linear
5.3.1 Representationn of and Search Algorithm 235
Operationss onn Stacks 152 7.3 Sorting Problem 236
5.4 Queues 154 7.3.1 Classication of Sorting
5.4.1 Queuee Represe
Represen
Representation 154 Algorithms 237
5.5 Trees 157 7.3.2 Properties of Sorting
5.5.1 Tree Terminologies 157 Algorithms 237
5.5.2 Classication of Trees 159 7.3.3 Bubble Sort 238
5.5.3 Binary Tree Representation 161 7.3.4 Selection Sort 242
5.5.4 Binary Tree Operations 163 7.4 Computational Geometry Problems 245
5.6 Graphs 167 7.4.1 Closest-pair Problem 245
5.6.1 Terminologies and 7.4.2 Convex Hull Problem 247
Types of Graphs 168 7.5 Exhaustive Searching 249
5.6.2 Graph Representation 172 7.5.1 15-puzzle Problem 250
5.6.3 Graph Traversal 174 7.5.2 8-queen Problem 251
5.6.4 Elementary Graph Algorithms 178 7.5.3 Magic Squares 252
5.6.5 Spanning Tree and Minimum-cost 7.5.4 Container Loading Problem 253
Spanning Tree 186 7.5.6 Assignment Problem 256

KhW
 Detailed Contents xiii

8 Divide-and-conquer Approach 262 9.4.1 Interpolation Search 333

8.1 Introduction 262 9.4.2 Selection and Ordered
8.1.1 Recurrence Equation for Statistics 334
Divide and Conquer 263 9.4.3 Finding Median 336
8.1.2 Advantages and Disadvantages
of Divide-and-conquer 10 TimeSpace Tradeos 342
Paradigm 264 10.1 Introduction to TimeSpace
8.2 Merge Sort 264 Tradeoffs 342
8.3 Quicksort 269 10.2 Linear Sorting 342
8.3.1 Partitioning Algorithms 270 10.2.1 Counting Sort 342
8.3.2 Variants of Quicksort 275 10.2.2 Bucket Sort 347
8.4 Finding Maximum and Minimum 10.2.3 Radix So
Sort 349
Elements 277 10.3 Hashing andd Hash Tables 351

ss
8.5 Multiplication of Long Integers 280 10.3.1 Properties
roperties of Hash
8.6 Strassen Matrix Multiplication 284 Functions
Functi
Functio 353

re
8.7 Tiling Problem 288 10.3.2
0.3.2 Hash
Ha Table Operations 354

yP
8.8 Closest-pair Problem 291 10.3.3
0 3 Collision
10.3 C 356
8.8.1 Using Divide-and-conquer 10.4 B-tree
B-tr
B-trees 359
Method 291 10.4.1
10
10.4
4 B-tree Balancing
sit
8.9 Convex Hull 293 Operations 361
er
8.9.1 Quickhull 2933 10.4.2 B-tree Operations 363
8.9.2 Merge Hull 295
95
iv

8.10 Fourier Transform 296 11 Greedy Algorithms

1 372
Un

8.10.1 Polynomial Multiplication

ion
on 296
29 11.1 Introduction to Greedy Approach 372
8.10.2 Application of Fourier
rier
er 11.1.1 Components of Greedy
Transform 2988 Algorithms 373
d

8.10.3 Fast Fourierr Transfor

Transform 302 11.2 Suitability of Greedy Approach 374
or

11.3 Coin Change Problem 375

11.4 Scheduling Problems 376
xf

9 Decrease-and-conquer
nquer App
Approach 309
9.1 Introduction 309 11.4.1 Scheduling without
O

9.2 Decrease by Constant

onst Method 311 Deadline 377
9.2.1 Insertion Sort 311 11.4.2 Scheduling with Deadline 379
9.2.2 Topological Sort 315 11.4.3 Activity Selection Problem 382
9.2.3 Generating Permutations 321 11.5 Knapsack Problem 384
9.2.4 Generating Subsets 323 11.6 Optimal Storage of Tapes 388
9.3 Decrease by Constant Factor 11.7 Optimal Tree Problems 390
Method 325 11.7.1 Optimal Merge 390
9.3.1 Binary Search 326 11.7.2 Huffman Coding 393
9.3.2 Fake Coin Detection 330 11.7.3 Tree Vertex Splitting Problem 398
9.3.3 Russian Peasant 11.8 Optimal Graph Problems 401
Multiplication Problem 331 11.8.1 Minimum Spanning Trees 401
9.4 Decrease by Variable 11.8.2 Single-source Shortest-path
Factor Method 332 Problems 407

KhW
xiv Detailed Contents

12 Transform-and-conquer Approach 417 13.6 FloydWarshall All Pairs

12.1 Introduction to Transform and Shortest-path Algorithm 478
Conquer 417 13.6.1 Shortest-path
12.2 Introduction to Instance Reconstruction 481
Simplication 418 13.7 BellmanFord Algorithm 481
12.3 Matrix Operations 419 13.8 Travelling Salesperson Problem 486
12.3.1 Gaussian Elimination 13.9 Chain Matrix Multiplication 488
Method 420 13.9.1 Dynamic Programming
12.3.2 LU Decomposition 427 Approach for Solving Chain
12.3.3 Crouts Method of Matrix Multiplication
Decomposition 434 Problem 490
12.3.4 Finding Matrix Inverse 436 13.10 Knapsack Problem
Pr 496
12.3.5 Finding Matrix 13.11 Optimall Binary Search Trees 500

ss
Determinant 439 13.11.1
1 Brute FForce Approach
12.4 Change of Representation 441 for Constructing
C Optimal

re
12.4.1 Heap Sort 441 B
BSTs 500

yP
12.4.2 Polynomial Evaluation 13.11.2
3.11 Dynamic Programming
13.11.2
Using Horners Method 445 Approach for Constructing
12.4.3 Binary Exponentiation 447 Optimal BSTs 502
sit
12.5 Problem Reduction 450 13.1 Flow-shop Scheduling Problem 507
13.12
13.12.1 Single-machine
er

13 Dynamic Programming 455 Sequencing Problem 508

iv

13.1 Basics of Dynamic Programmingg 455 13.12.2 Two-machine Sequencing

13.1.1 Components of Dynamicc
Un

Problem 509
Programming 456
13.1.2 Characteristics of Dynamic
Dynam
ynam 14 Backtracking 517
14.1 Introduction 517
d

Programmingg 458
14.2 Basics of Backtracking 518
or

13.2 Fibonacci Problemem

m 460
13.3 Computing Binomial Coefcients 463
omial Co 14.3 N-queen Problem 522
xf

13.4 Multistage Graph Problem

Pro 466 14.3.1 State Space of 4-queen
Problem 523
O

13.4.1 Forward
ard Computation
Co
Procedure 467 14.4 Sum of Subsets 525
13.4.2 Backward Computation 14.5 Vertex Colouring Problem 527
Procedure 471 14.6 Hamiltonian Circuit Problem 531
13.5 Transitive Closure and Warshall 14.6.1 Promising (or Bounding
Algorithm 472 Function) for Hamiltonian
13.5.1 Finding Transitive Closure Using Problem 531
Brute Force Approach 473 14.7 Generating Permutation 534
13.5.2 Finding Transitive Closure Using 14.8 Graham Scan 536
Warshall Algorithm 473 15 Branch-and-ound Technique 543
13.5.3 Alternative Method to 15.1 Introduction 543
Warshall Algorithm for 15.2 Search Techniques for Branch-
Finding Transitive Closure 476 and-bound Technique 545

KhW
 Detailed Contents xv

15.2.1 BFS using Branch-and- 17.8 Max-ow Problem 623

bound algorithm 17.9 Bipartite Matching Problem 628
FIFOBB 545 17.10 Stable Marriage Problem 630
15.2.2 LIFO with Branch and
Bound 547
18 Basics of Computaonal
15.2.3 Least Cost with Branch
Complexity 638
and Bound 547
18.1 Introduction to Computational
15.3 15-puzzle Game 548
Complexity 638
15.4 Assignment Problem 552
18.2 Algorithm Complexity,
15.5 Traveling Salesperson Problem 559
8pper and Lower Bound Theory 639
15.6 Knapsack Problem 562
18.2.1 Upper and Lower Bounds 640
18.3 Decision Problems
oble
ble and Turing
16 String Algorithms 569 Machine 644

ss
16.1 Introduction to String 18.4 Complexityxity Classes
plexity Cla 647
Processing 569

re
18.4.1
4.1 Class P 648
16.2 Basic String Algorithms 571 118.4.2
.4.2 NP Class 648

yP
16.2.1 Length of Strings 571 18.5
.5 Th
Theo
Theoryry of NP-complete
16.2.2 Concatenation of Problems
Prob
Proble 650
Two Strings 571
sit
18.6 Reductions
Red 651
16.2.3 Finding Substrings 572 18.6.1 Turing Reduction 651
16.3 Longest Common Subsequences 573
er
18.6.2 Karp Reduction 652
16.4 nave String Matching 18.7 Satisability Problem and
iv

Algorithm 576 Cooks Theorem 654

Un

16.5 Pattern Matching 8sing Finitee 18.8 Example Problems for Proving
Automata 578
5 NP-completeness 655
16.6 RabinKarp Algorithm m 580
5 18.8.1 SAT is NP-complete 655
d

16.7 KnuthMorrisPratt tt Algorithm 584 18.8.2 Problem 3-CNF-SAT is

or

16.8 Harspool Algorithm

thm
m 588 NP-complete 656
16.9 BoyerMooree String Matching
Ma
M
xf

18.8.3 Clique Decision Problem

Algorithm 590 is NP-complete 657
O

16.10 Approximatee String

Stri Matching 594 18.8.4 Sum of Subsets
(from 3-CNF-SAT) 658
17 Iterave Improvement and
Linear Programming 601 19 Randomized and Approximaon
17.1 Introduction to Iterative Algorithms 663
Improvement 601 19.1 Dealing with NP-hard
17.2 Linear Programming 601 Problems 663
17.3 Formulation of LPPs 603 19.2 Introduction to Randomized
17.4 Graphical Method for Algorithms 664
Solving LPPs 606 19.2.1 Generation of Random
17.5 Simplex Method 611 Numbers 666
17.6 Minimization Problems 615 19.2.2 Types of Randomized
17.7 Principle of Duality 619 Algorithms 669

KhW
xvi Detailed Contents

19.3 Examples of Randomized 20.2 Classication of Parallel Systems 705

Algorithms 669 20.2.1 Flynn Classication 706
19.3.1 Hiring Problem 669 20.2.2 Address-space (or Memory
19.3.2 Primality Testing Algorithm 672 Mechanism-based)
19.3.3 Comparing Strings Using Classication 708
Randomization Algorithm 674 20.2.3 Classication Based on
19.3.4 Randomized Quicksort 675 Interconnection Networks 710
19.4 Introduction to Approximation 20.3 Introduction to PRAM Model 711
Algorithms 678 20.4 Parallel Algorithm Specications
19.5 Types of Approximation and Analysis 712
Algorithms 679 20.4.1 Parallel Algorithm
19.6 Examples of Approximation Analysis
si 714
Algorithms 680 20.5 Simple Parallel
arallel
allel A
Algorithms 716

ss
19.6.1 Heuristic-based 20.5.1 Prex
rex Computation
Com 716
Approximation Algorithms 681 20.5.2
5.2 List Ranking
Ra 718

re
19.6.2 Greedy Approximation 0.5.3 Euler
20.5.3 Eu Tour
Eule 719

yP
Algorithms 688 20.6
6 Pa
P
Parallel
rall Searching and Parallel
rallel
19.6.3 Approximation Algorithm Sorting
Sor
Sorti 720
Design Using Linear 20.6.1
20
20.66 Parallel Searching 720
sit
Programming 693 220.6.2 OddEven Swap Sort 722
19.6.4 Designing Approximation 20.6.3 Parallel MergeSplit
er

Algorithms Using Dynamic Algorithm 724

iv

Programming 696 20.7 Additional Parallel Algorithms 726

Un

20.7.1 Parallel Matrix

20 Parallel Algorithms 705 Multiplication 726
20.1 Introduction to Parallel
el Process
Processing
ng 705 20.7.2 Parallel Graph Algorithms 729
d
or

Appendix AMathematical
ical Basics 735
xf

Appendix BProof Techniques

echniques 752
Bibliography 763
O

Index 766

KhW
 C H A P T E R

Introducon to
Algorithms
1
Ideas are the beginning point of all fortunes.
Napolean Hill

1.1 INTRODUCTION

ss
Learning Oecves
Computers are powerful tools of computing. ing.
g. One ca
cannot ignore the impact of

re
This chapter introduces
the basics of algorithms.
computers on our modern life. We usee computecomputers for personal needs such as
computer
typing documents, browsing the Internet,
ntern ssending emails, playing computer
ernet, sen

yP
All important denitions
and concepts related to games, performing numeric calculations,
alculation and so on. Industries and govern-
alculations,
the study of algorithms are ments use computers much effectively to perform complicated tasks
h more effec
effecti
sit
the focus of this chapter. to improve productivity and efciency. Applications of computer systems in
d efcie
efciency
The reader would be
airline reservation, video
deo surveillance, biometric recognition, e-governance, and
o surveil
surveillanc
er
familiar with the following
concepts by the end of e-commerce are all lll examples of their usefulness in improving efciency and
iv

this chapter: productivity. The increasing iimportance of computers in our lives has prompted
he increasin
schools and universities tto introduce computer science as an integral part of
d universitie
Un

Basic terminologies of
algorithm study our modernern education.
dern educat Informally, everyone is expected to handle computers to
Need for algorithms accomplish
plish certain
mplish ce ain basic tasks. This knowledge of using computers to perform
d

Characteristics of our
ur day-to-day
day-to-da activities is often called computational thinking. Computational
or

algorithms
thinking isi a necessity to survive in this modern world. However, computer
Stages of a problem-
science
scien professionals are expected to accomplish much more than acquiring
xf

solving process
Need for efcient this
th basic skill of computer usage. They are required to write specic computer
O

algorithms programs to provide computer-automated solutions for problems. Writing a

Classication of program is slightly more complicated than merely learning to use computers,
algorithms as it requires algorithmic thinking. Algorithmic thinking is an important
analytical skill that is required for writing effective programs in order to solve
given problems. Algorithmic thinking is not conned to computer science only,
but it spreads through all disciplines of study. Computer science is a domain
where the skills of algorithmic thinking are taught to the aspiring computer
professionals for solving computational problems.

1.2 NEED FOR ALGORITHMIC THINKING

Norman E. Gibbs and Allen B. Tucker1 proposed a denition for computer
science that captures the core truth of computer science study. According to
1
Gibbs, Norman E., Allen B. Tucker, A Model Curriculum for a Liberal Arts Degree
in Computer Science, Communications of ACM, Vol. 29, Issue 3, 1986.

KhW
2 Design and Analysis of Algorithms

them, Computer science is the systematic study of algorithms and data structures, specically
their formal properties, their mechanical and linguistic realizations, and their applications.
Hence, there cannot be any dispute regarding the fact that study of algorithms is the central
theme of computer science.
Let us elaborate this denition further. The formal and mathematical properties of
algorithms include the study of algorithm correctness, algorithm design, and algorithm
analysis for understanding the behaviour of algorithms. Hardware realizations include
the study of computer hardware, which is necessary to run the algorithms in the form of
programs. Linguistic realizations include the study of programming languages and their
design, translators such as interpreters and compilers, and system software tools such as
linkers and loaders, so that the algorithms can be executed by hardware in the form of
programs. Applications of algorithms include the study of design es and development of
efcient software packages and software tools so that these se algor
algorithms can be used to

ss
solve specic problems.
Thus, computer scientists consider the study of algorithms
rithms the core theme of computer
hms as th

re
science. The art of designing, implementing, and analysing algorithms is called algorithmics.
ysing alg
algo

yP
Algorithmics is a general word that comprises alll aspe
aspects
p cts of the study of algorithms. Let us
now attempt to dene algorithms formally. An algorithm is i a set of unambiguous instructions
or procedures used for solving a given problem em to provide
blem provi correct and expected outputs for
prov
sit
all valid and legal input data (refer to Box
ox 1.1).
An algorithm can also be referred d by othe terminologies such as recipe, prescription,
other te
er

process, or computational procedure. re. Box 1.2

dure. 1 provides a brief history of algorithms.
iv

Algorithms thus serve as prescriptions

rescriptions of
rescription o how a computer should carry out instructions
Un

to solve problems. Hence, algorithms ca can be visualized as strategies for solving a problem.
One cannot write a programgram given problem without the necessary analytical skills or a
ram for a giv
strategy for solving the proble
problem.
m TThus, the knowledge of how to solve a problem is called
d

algorithmic thinking.
king
g.
or

One can compare

mpare program
pare a pr
pro construction with the construction of a house. The blueprint
of the housesee incorporating
incorpora
incorpor all the planning necessary for the construction of the house can
xf

be visualized
zed as an algorithm. A computer program is only an algorithm expressed using a
ized
O

programming language such as C or C++. In addition to possessing the essential skills of

ng la
using a programming language, the ability to conceive a strategy or apply analytical skills
are important for solving a problem or constructing a program.

Box 1.1 Origin of the word algorithm

The word algorithm is derived from the name of a Persian
mathematician, Abu Jafer Mohammed Ibn Musa al
Khowarizmi, who lived sometime around 780850 AD. He
was from the town of Khowarazm, now in Uzbekistan. He
was a teacher of mathematics in Baghdad. He wrote a book
titled Kitab al Jabr wal Muqabala (Rules of Restoration
and Reduction) and Algoritmi Numero Indorum where
he introduced the old ArabicIndian number systems to
Europe. He also introduced the simple step-by-step rules
for addition, subtraction, multiplication, and division in
his book. The word algebra has also been derived from
the title of this book. When his book was translated to
Latin, his name was quoted as Algorismus from which
the word algorithm emerged. Algorithm as a word thus
became famous for referring to procedures that are used
by computers for solving problems.

KhW
 Introduction to Algorithms 3

Box 1.2 Short history of compung and algorithms

The history of algorithms dates back to ancient civilizations Alan Turings Turing machines developed in 193 play
that used numbers. Ancient civilizations such as Egyptian, a very important role in computational complexity theory.
Indian, Greek, and Chinese were known to have used This Turing machine and Alonzo Churchs lambda calcu-
algorithms or procedures to carry out tasks like simple lus formed the basis for formalization of the complexity
calculations. This is in contrast to the history of modern theory. Thus, the history of algorithms is much richer than
computers that were designed in the 1940s only. Thus, that of computers.
the history of algorithms is much older and more exciting. The history of computing is another interesting story.
Euclid, who lived in ancient Greece around 400300 BC, Abacus and other mechanical devices used for computing
is credited with writing the rst algorithm in history for com- have been utilized at various stages of human history. A real
puting the greatest common divisor (GCD). Archimedes attempt was made in the 17th century by Leibniz, a German
created an algorithm for the approximation of the number mathematician and philosopher. He invented innitesimal
pi. Eratosthenes introduced to the world the algorithm for calculus, generating the idea of computers.
co In 1822, Charles
nding prime numbers. Averroes (1121198) also used Babbage introduced the difference
ference engine, which required
algorithmic methods for calculations. changing of gears manually ally to perform
nually perf calculations. These
A number of inuences led to the formalization of the ideas nally led to the invention of computers in the 1940s.
theory of algorithms. Some of the noteworthy developments In 1942, the US S government
governmen designed a machine called
include the introduction of Boolean algebra by George ENIAC (electronic
ctro ic numerical
numer integrator and computer). This
Boole (1847) and set theory by Gottlob Frege (1879). Set was succeeded
ceeded
eded b byy EDVAC
E (electronic discrete variable
theory has become a foundation of modern mathemat- automatic
atic computer)
matic compute in 1951. In 1952, IBM introduced
compu
ics. The concept of recursion formulated by Kurt Gdel, its mainframe
ainframe computer.
com Developments in the hardware
and contributions of Giuseppe Peano (1888), Alfred North domain
main and de declining costs have shifted computer usage
Whitehead, and Bertrand Russell in mathematical logic from big research
res and military environments to homes and
led to the formalization of algorithm as an independent dent educational
educ
educatio institutions. In 1981, IBM personal computers
eld of study. were
w introduced for home and ofce use.
Un

1.3 OVERVIEW OF ALGORITHMS

MSS
d

Algorithms are generic

neric and nnot limited to computers alone (refer to Box 1.2). We perform
or

many algorithms
ms in our daddaily life unknowingly. Consider a few of our daily activities, some
of which are listed
ted as follows:
fo
xf

1. Searching
ng for a specic book
ing
O

2. Arranging bo
bbooks based on titles
3. Using a recipe to cook a new dish
4. Packing items in a suitcase
5. Scheduling daily activities
6. Finding the shortest path to a friends house
7. Searching for a document on the Internet
8. Preparing a CD of compressed personal data
9. Sending messages via email or SMS
We perform many tasks without being aware of their inherent algorithms. For example,
consider the activity of searching a word in a dictionary. How do we search? It can be noted
that indexing the words in a dictionary reduces the effort of searching signicantly. We just
open the book, compare the word with the index given, and accordingly decide on the por-
tions of the books to be searched for nding the meaning of that particular word. This kind

KhW
4 Design and Analysis of Algorithms

of a plan or an idea is called a strategy, which is more

Agent
formally known as an algorithm.
The environment of a typical algorithm is shown in
Valid Fig. 1.1. Here, agent, which can be a human or a computer,
Algorithm Output
input is the performer.
Fig. 1.1 Typical algorithm To illustrate further, let us consider the example of tea
preparation. The hardware used here includes cooking
utensils a heater, and a person. To make tea, one needs to have the following ingredients
water, tea powder, and sugar. These ingredients are analogous to the inputs of an algorithm.
Here, preparing a cup of tea is called a process. The output of this process is, in this case, tea.
This corresponds to the output of an algorithm. The procedure for preparing tea is as follows:
1. Put tea powder in a cup. 4. Pour milk.
2. Boil the water and pour it into the cup. 5. Add sugar if necessary.
ecessary.
essary

ss
3. Filter it. 6. Pour the teaa into cup.
nto a cup

re
This kind of a procedure can be called an algorithm. hm.. It can be noted that an algorithm
hm
consists of step-by-step instructions that are required accomplish a certain task.
ed to accom

yP
Humans often perform such procedures intuitively vely or eeven mechanically without spending
ively
much conscious thought, and hence, they label sit actions as habitual activity. Many of our
ell such acti
action
day-to-day activities are not very efcient.. However,
However al algorithms that are meant for computers
represent a different case altogether. Computer procedures should be efcient as computer
mputer ppro
er
resources are scarce. Hence, much ought is ggiven for writing computer procedures that
h thought
can solve problems. Problems can classied into two types: computational problems and
an be clas
classie
iv

non-computational problems.
Un

Computational problemss can be solvedssolv by a computer system. A computational problem

is characterized by twoo factors:
actor : (a)
factor (a the
t formalization of all legal inputs and expected outputs
of a given problem m andd (b) the characterization of the relationship between problem output
d

and input. Thus, algorithm is expected to give the expected output for all legal inputs.
s, an algori
algorit
or

If an algorithm
hm yields tthe correct output for a legal input, then it is called an algorithmic
xf

solution.
Non-computational
omputati
mputati problems cannot be solved by a computer system. This classication
O

shows the fundamental

un differences between computers and humans in solving problems.
Computers are more effective than humans in performing calculations and can crunch num-
bers in a fraction of seconds with more precision and efciency. However, humans outscore
computers in recognition. The ability of recognizing an object by humans is much better than
that by machines. Recognition of an object by computer systems requires lots of programming
involving images and concepts of image processing. In addition, some tasks are plainly not
possible to be carried out by computer systems. Can a computer offer its opinion or show
emotion like humans? To put it simply, problems involving more intellectual complexity are
much more difcult to solve by computers as computer systems lack intelligence.
Thus, developing algorithms to make computers more intelligent and make them perform
tasks like humans becomes much more crucial and challenging. Developing algorithms for
computers is both an art and a science. Algorithm design is an art that involves a lot of creative
ideas, novelty, and even adventurous strategies and knowledge. Algorithm design is also a
science because its construction usually involves the application of some set of principles. The

KhW
 Introduction to Algorithms 5

review and knowledge of these principles can facilitate the development of better algorithms,
based on sound mathematical and scientic principles.

1.3.1 Computaonal Prolems Instance and Size

One encounters many types of computational problems in the domain of computer science.
Some of the problems are as follows:
Structuring problems In structuring problems, the input is restructured based on certain
conditions or properties. For example, sorting a list in an ascending or a descending order is
a structuring problem.
Search problems A search problem involves searching for a target in a list of all possibili-
ties. All potential solutions may be generated and represented in ththe form of a list or graph.
n problem
Based on a property or condition, the best solution for the given roblem is searched. Puzzles

ss
are good examples of search problems where the target or goal oal is sea
searched in a huge list of
possible solutions.

re
Construction problems These problems involvee th
he const
the construction of solutions based on

yP
the constraints associated with the problem.
Decision problems Decision problems are re yes/no typ
type of problems where the algorithm
sit
output is restricted to answering yes or no. Let us assu
aassume that a road network map of a city
ny road conne
is given. The problem, say, Is there any con
connectivity between two cities, say Hyderabad
er

ecision
cision pro
and Chennai?, can be called as a decision proble
problem as the output of this algorithm is restricted
iv

to either yes or no.

Un

ptimizati problems constitute a very important set of problems

Optimization problems Optimization
that are often encounteredred in compu science domain. The decision problem about road con-
mpute
computer
nectivity between Chennai nnai and Hyderabad can also be posed as an optimization problem as
follows: What is the shortes
shortest distance between Chennai and Hyderabad? This is an optimiza-
tion problem, as thehe prob
probl
problem involves nding the shortest path. Thus, optimization problems
ertain
rtain objec
involve a certain object
objective function that is typically of the following form: maximize (say prot)
or minimizezee (say eeffort) based on a set of constraints that are associated with the problem.
ro
Once the problem is recognized, the input and output of an algorithm should be identied.
Consider, for example, a problem of nding the factorial of a number. The factorial of a positive
number N can be written as follows: N! = N (N 1) (N 2) . . . 1. A valid input can be
called an instance of a problem. For example, factorial of a negative number is not possible.
Therefore, all valid positive integers {0, 1, 2, . . .} can serve as inputs and every legal input is
called an instance. All possible inputs of a problem are often called a domain of the input data.
The input should be encoded in a suitable form so that computers can process it. The number
of binary bits used to represent the given input, say N, is called the input size. Input size is
important, as a larger input size consumes more computer time and space.
The core question still remains to be answered: how to solve a given problem? To illustrate
problem solving, let us consider a simple problem of counting the number of students in a
tuition centre who have passed or failed a test, assuming that the pass mark is 50. Details of
the students such as their registration number, name, and more importantly, marks obtained,
which are necessary for the given problem, are shown in Table 1.1.

KhW
6 Design and Analysis of Algorithms

Table 1.1 Students course marks

Registration number Student name Course marks

1 Abraham 80
2 Beena 30
3 Chander 83
4 David 23
5 Elizabeth 90
 Fauzia 78

The class strength of the tuition centre is 6. The pass mark of the course is given as 50. How do
we manually solve this problem? First, we will read the student marks marks. Thus, the inputs for this
problem are a set of student marks. The goal of this problem iss too print tthe pass and fail counts

ss
of the students, which is also the output of this algorithm. The process of reading a students
he proc
proce
mark is done manually. Compare the student mark with the pass m mark, that is, 50. If the student
ma

re
hould be ad
mark is greater than or equal to 50, then pass count should added by one. Otherwise, the fail

yP
count should be added by one. This process is repeatedpeated forr aall the students.
ated fo
The procedure that is done manually can bee given as an algorithm. Therefore, informally,
sit
the algorithm for this problem can be given follows:
en as follows
follow
Step 1: Let counter = 1, number of students
dents = 6
tudents
er

Step 2: While (counter number of students)

student
iv

2.1: Read the marks off thehe stude

students
2.2: Compare the marks the sstudent with 50
arks of th
Un

2.3: If student ma
mark greater than or equal to 50
rk is grea
Then increment
crement the ppass count
ment th
Else increment
rement tthe fail count
2.4: Increment th counter
crement the
Step 3: Print
nt pass count and fail count
ss coun
Step 4: Exit
xit
Thus, algorithmic
gorith solving can be observed to be much similar to how we solve problems
manually.

Some of the important characteristics are listed as follows:
Input An algorithm can have zero or more inputs.
Output An algorithm should produce at least one or more outputs.
'eniteness An algorithm is characterized by deniteness. Its instructions should be clear and
unambiguous without any confusion. All operations should be well dened. For example, opera-
tions involving the division of zero or taking a square root of a negative number are unacceptable.
8niTueness An algorithm should be a well-dened and ordered procedure that consists of
a set of instructions in a specic order. The order of the instructions is important as a change
in the order of execution leads to a wrong result or uncertainty.

KhW
 Introduction to Algorithms 7

&orrectness The algorithm should be correct.

(IIectiYeness An algorithm should be effective, which implies that it should be traceable
manually.
Finiteness An algorithm should have a nite number of steps and should terminate after
executing the nite set of instructions. Therefore, niteness is an important characteristic of
an algorithm.
Some of the additional characteristics that an algorithm is supposed to possess are the
following:
6iPpOicit\ Ease of implementation is another important characteristic of an algorithm.
*eneraOit\ An algorithm should be generic, independent of any y pprogramming language
or operating systems, and able to handle all ranges of inputs;; itt shoul
should not be written for a

ss
specic instance.

re
An algorithm that is denite and effective is also called a computational
co
com procedure. In
addition, an algorithm should be correct and efcient,ent that is, should work correctly for all

yP
m that ex
valid inputs and yield correct outputs. An algorithm xec
executes fast but gives a wrong result
is useless. Thus, efciency of an algorithm iss secondary ccompared to program correctness.
sit
ailable
able for a ggiven problem, then one has to select
However, if many correct solutions are available
on)
n) based on factors such as speed, memory usage,
the best solution (or the optimal solution)
er
and ease of implementation.
iv

W
Un

Algorithms can be contrasted ted with p pro

programs. Algorithms, like blueprints that are used
for constructing a house, help
e,, he solve a given problem logically. A program is an expres-
lp sol
sion of that idea, consisting
ting of a se
nsisting set of instructions written in any programming language.
d

The development of algorith

algorithms can also be contrasted with software development. At the
or

industrial level, softwar development is usually undertaken by a team of programmers

l, software
who developp software,
software often for third-party customers, on a commercial basis. Software
xf

engineering domain that deals with large-scale software development. Project manage-
ngg is a dom
do
O

ment issues such as team management, extensive planning, cost estimation, and project
scheduling are the main focus of software engineering. On the other hand, algorithm
design and analysis as a eld of study take a micro view of program development. Its
focus is to develop efcient algorithms for basic tasks that can serve as a building block
for various applications. Often project management issues of software development are
not relevant to algorithms.

1.4 NEED FOR ALGORITHM EFFICIENCY

Computer resources are limited. Hence, many problems that require a large amount of resources
cannot be solved. One good example is the travelling salesperson problem (TSP). Its brief
history is given in Box 1.3.
A TSP is illustrated in Fig. 1.2. It can be modelled as a graph. A graph consists of a set of
nodes and edges that interconnect the nodes. In this case, cities are the nodes and the paths
that connect the cities are the edges. Edges are undirected in this case. Alternatively, it is

KhW
8 Design and Analysis of Algorithms

Box 1.3 History of travelling salesperson prolem

A travelling salesperson problem (TSP) is one of the most
important optimization problems studied in history. It is
very popular among computer scientists. It was studied
and developed in the 19th century by Sir William Rowan
Hamilton and Thomas Pennington Kirkman. In 1857,
Hamilton created an Icosian game, a pegboard with 20
holes called vertices. The objective is to nd a tour starting
from a city, visiting all other cities only once, and nally
returning to the city where the tour has started. This is
called a Hamiltonian cycle. The TSP is to nd a Hamiltonian
cycle in a graph. The problem was then studied extensively
by Karl Menger in Vienna and by Harvard later in 1920.

A A B possible to move from a particular city

(a) (b) to any other city. The complete details
A B
of graphshs are pprovided in Chapter 5. A
TSP P involves
nvolves a travelling person who

ss
A B
starts
rts from a city, visits all other cities

re
only onc
once, and returns to the city from
wh
wher
where he started.

yP
C C D A brute force technique can be used
(c) (d) ffor solving this problem. Let us enu-
sit
Fig. 1.2 Travelling salesperson problem (a) One cityno yno
no path merate all the possible routes. For a
TSP involving only one city, there is
er
(b) Two cities (c) Three cities (d) Four cities
itiess
no path. For two cities, there is only
iv

one path (AB). For three cities, s, there are twotw paths. In Fig. 1.2, these two paths are ABC
and ACB, assuming that A is the o orig
origin from where the travelling salesperson started.
Un

For four cities, the paths hss aare

re {ADBCA,
{AD
{A ADCBA, ABCDA, ABDCA,
ACDBA, and ACDB CDB A}
ACDBA}.
d

Thus, every addition ion of a city can be noted to increase the path exponentially. Table 1.2
ddition
or

shows the numbermber possible routes.

er of poss
Therefore,e, it can beb observed that, for N cities, the number of routes would be (N 1)!
xf

for N 2.. The avail

availability of an algorithm for a problem does not mean that the problem is
O

solvable by y a computer.
com
co There are many problems that cannot be solved by a computer and
for many problems algorithms require a huge amount of resources that cannot be provided
practically. For example, a TSP cannot be solved in reality. Why? Let us assume that there
are 100 cities. As N = 100, the possible routes are then
(100 1)! = 99!.
Table 1.2 Complexity of TSP
The value of the number 50! is 3041409320171337804
Number of cities Number of routes 3612608166064768844377641568960512000000000000.
1 0 (as there is no route) Therefore, 99! is a very large number, and even if a computer
2 1 takes one second for exploring a route, the algorithm will
3 2
run for years, which is plainly not acceptable. Just imagine
how much time this algorithm will require, if the TSP is
4 
tried out for all cities of India or USA. This shows that the
5 24
development of efcient algorithms is very important as
 120
computer resources are often limited.

KhW
 Introduction to Algorithms 9

1.5 FUNDAMENTAL STAGES OF PROBLEM SOLVING

Problem solving is both an art and a science. The problem-solving process starts with the
understanding of a given problem and ends with the programming code of the given problem.
The stages of problem solving are shown in Fig. 1.3.
The following are the stages of problem solving:
1. Understanding the problem 5. Analysing an algorithm
2. Planning an algorithm 6. Implementing an algorithm
3. Designing an algorithm 7. Performing empirical analysis (if necessary)
4. Validating and verifying an algorithm
1.5.1 Understanding the Prolem
Problem
The study of an algorithm starts with h the computability theory.
Theoretically, the primary question off compu computability theory is the

ss
following: Is the given problem solvable?olvable?
vable? TTherefore, a guideline for

re
problem solving is necessary to understan
understand the problem fully. Hence,
Planning
a proper problem statementt is rrequired.
equi
equire Any confusion regarding or

yP
misunderstanding of a problem sta statement will ultimately lead to a
wrong result. Often, solving
olving numerical instances of a given problem
ving the nu
num
sit
Algorithm design can give an insightt to problem. Similarly, algorithmic solutions of
o the pro
proble
a related problem em
m will pprov
provide more knowledge for solving a given
er

algorithm.
iv

Algorithm correctness ally,

Generally,y, compu
computer systems cannot solve a problem if it is not properly
dened.
ed. often fall under this category, which needs supreme level
d. Puzzles ofte
Un

off int
intelligence.
lligence. T
lligenc These problems illustrate the limitations of computing
Algorithm analysis
power.
ower. H Humans
uma are better than computers in solving these problems.
rd

1.5. Planning an Algorithm

1.5.2
o

T second important stage of problem solving is planning. Some

The
xf

Implementationn and
alysis
empirical analysis of the important decisions are detailed in the following subsections.
O

D
A computation model or computational model is an abstraction of a
Post analysis
real-world computer. A model of computation is a theoretical math-
ematical model and does not exist in physical sense. It is thus a virtual
machine, and all programs can be executed on this virtual machine
Is theoretically. What is the need for a computing model? It is meaning-
OK less to talk about the speed of an algorithm as it varies from machine
No to machine. An algorithm may run faster in machine A compared to in
Yes machine B. This kind of an analysis is not correct as algorithm analysis
should be independent of machines. A computing model thus facilitates
Exit
such machine-independent analysis.
Fig. 1.3 Stages of First, all the valid operations of the model of computation should be
problem solving specied. These valid operations help specify the input, process, and

KhW
10 Design and Analysis of Algorithms

Box 1.4 Alan Turing

Alan Turing (19121954) is well known for his contribution a theoretical machine, called the Turing machine that is used
towards computing. He is considered by many as the father widely in computability theory and complexity analysis. Alan
of modern computing. His contribution towards computability Turing is also credited with the designing of the Turing Test
theory and articial intelligence is monumental. He designed as a measure of testing the intelligence of a machine.

output. In addition, the computing models provide a notion of the necessary steps to compute
time and space complexity of a given algorithm.
For algorithm study, a computation model such as a random access machine (RAM) or a
Turing machine is used to perform complexity analysis of algorithms. RAM is discussed in
Chapter 3 and Turing machines are discussed in Chapter 17. The contribution of Alan Turing
he con
cont
ls (refer
is vital for the study of computability and computing models refer to Box 1.4).

ss
O

re
Data structure concerns the way data and its relationships hips are sstored. Algorithms require data

yP
for solving a given problem. The nature of data and nd thei organization can have impacts on the
their org
efciency of algorithms. Therefore, algorithms ms and data sstructures together often constitute
sit
an important aspect of problem solving.
Data organization is something we are familiar with
re famili w in our daily life. Figure 1.4 shows
er
an example of data organization called ed a queu
qqueue. A gas station with one servicing point
should have a queue, as shown in n the gure,
gure to avoid chaos. A queue (or rst come rst
g
iv

serveFCFS) is an organization tion where

ation wher the
th processing (lling of gas) is done in one end
Un

and a vehicle is added at the end. All vehicles have same priority in this case. Often
he other eend
a problem dictates thee choi
choicee of st
structures. However, this structure may not be valid in
cases of, for example,
ple,, handling
handl medical emergencies, where highest priority should be
d

given to urgent cases.

or

Thus, data organization

anization is a very important issue that inuences the effectiveness of algorithms.
xf

At some point problem solving, careful consideration should be given to storing the data
oint of prob
pro
effectively. popular statement in computer science is algorithm + data structure = program.
y. A popu
O

ecti
ti of a data structure often proves fatal in the problem-solving process.
A wrong selection

Fuel station

Queue of trucks

Fig. 1.4 Example of a queue

KhW
 Introduction to Algorithms 11

1.5.3 Designing an Algorithm

Algorithm design is the next stage of problem solving. The following are the two primary
issues of this stage:
1. How to design an algorithm? 2. How to express an algorithm?
Algorithm design is a way of developing algorithmic solutions using an appropriate design
strategy. Design strategies differ from problem to problem. Just imagine searching a name in a
telephone directory. If anyone starts searching from page 1, it is termed as a brute force strategy.
Since a telephone directory is indexed, it makes sense to open the book and use the index at the
top to locate the name. This is a better strategy. One may come across many design paradigms
in the algorithm study. Some of the important design paradigms are divide and conquer, and
dynamic programming. Divide and conquer, for example, divides thee pr pproblem into sub-problems
and combines the results of the sub-problems to get the nal solution lution oof the given problem.

ss
Dynamic programming visualizes the problem as a sequence decisions. Then it combines
ce of decis
the optimal solutions of the sub-problems to get an optimalmal global ssolution. Many design vari-

re
ants such as greedy approach, backtracking, and branch bound techniques have been dealt
ch and bou

yP
mportan
porta t iin developing efcient algorithms.
with in this book. The role of algorithm design is important
Thus, one important skill required for problem solving is th
sit the selection and application of suit-
able design paradigms. A skilled algorithm designer
signer is ca
called an algorist.
er
^
After the algorithm is designed, it should be communicated to a programmer so that the
iv

algorithm can be coded as a program. This sstage is called algorithm specication. Only three
ogram. Th
Un

possibilities exist for communicating

unicating the idea of algorithms. One is to use natural languages
such as English to communicate
munic th aalgorithm. This is preferable. However, the natural lan-
unic te the
guage has some disadvantages
antages such as ambiguity and lack of precision. Hence, an algorithm
dvantages
d

is often written and communicated through pseudocode. Pseudocode is a mix of the natural
nd commu
commun
or

language and mathematics

mathematics.
hematic Another way is to use a programming language. The problem with
xf

programming language notation is that readers often get bogged down by the programming
ng languag
code details.
ils. Therefore, the pseudocode approach is preferable. Even though there are no
s. There
O

specic ruless for

f algorithm writing, certain guidelines can be followed to make algorithms
fo
more understandable. Some of these guidelines are presented in Chapter 2.

1.5.4 Validang and Verifying an Algorithm

Algorithm validation and verication are the methods of checking algorithm correctness.
An algorithm is expected to give correct outputs for all valid inputs. Sometimes, algorithms
may not give correct outputs due to logical errors. Hence, validation of algorithms becomes
a necessity. Algorithm validation is a process of checking whether the given algorithm gives
correct outputs for valid inputs or not. This is done by comparing the output of the algorithm
with the expected output.
Once validation is over, algorithm verication or algorithm proving begins. Algorithm veri-
cation is a process of providing a mathematical proof that the given algorithm works correctly
for all instances of data. One way of doing it is by creating a set of assertions that are expressed
using mathematical logic. Assertions are statements that indicate the conditions of algorithm

KhW
12 Design and Analysis of Algorithms

variables at various points of the algorithm. Preconditions indicate the conditions or variables
before the execution of an algorithm, and postconditions indicate the status of the variables at
the end of the execution of an algorithm. Remember, still the algorithm has not been translated
or converted into a code of any programming language. Hence, all these executions are rst
carried out theoretically on a paper and proof for the algorithm is determined. A proof of an
algorithm is said to exist if the preconditions can be shown to imply the postconditions logically.
A complete proof is to write each statement clearly for proving that the algorithm is right. In
addition to assertions, mathematical proof techniques such as mathematical induction can be
used for proving that algorithms do work correctly. Mathematical induction is one such useful
proof technique that is often used to prove that the algorithm works for all possible instances.
Mathematical proofs are rigorous and always better than algorithm validation. Program cor-
rectness itself is a major study and extensive research is done in this
h eld.

1.5.5 Analysing an Algorithm

ss
In algorithm study, complexity analysis is important as we aree mainly iinterested in nding optimal

re
algorithms that use fewer computer resources. Complexity exity theory
theor is a eld of algorithms that
the

yP
mputat
uta ional
deals with the analysis of a solution in terms of computational ion resources and their optimization.
Humans are more complex than, say, amoeba. So o what dodoes the word complexity refer to here?
Complexity is the degree of difculty associatedted
d with a problem
pro
prob and the algorithm. The complex-
sit
ity of an algorithm can be observed to be related
ated to iits iinput size. For example, an algorithm for
sorting an array of 10 numbers is easy, y, but the ssam
same algorithm becomes difcult for 1 million
er

inputs. This shows the connection between

etween cocomplexity and the size of the input.
comp
iv

Thus, complexity analysis iss useful for

fo the
th following two purposes:
Un

y of the giv
1. To decide the efciency given algorithms
hmss fo
2. To compare algorithms for decidi
dec
deciding the effective solutions for a given problem
d

Consider the following

ollowing scenario: for a problem =, two algorithms A and B exist. Find the
owing sce
or

optimal algorithm. m. To solve this, there must be some measures based on which comparisons can
hm.
be made. In general, two types of measures are available for comparing algorithmssubjective
neral, tw
xf

and objective.
ive. Subjective
ctive. Subje measures are factors such as ease of implementation, algorithm style,
O

and readability
ility oof the algorithm. However, the problem with these subjective measures is that
these factors cannot be quantied. In addition, a measure such as the ease of implementation,
style of the algorithm, or understandability of algorithms is a subjective measure that varies
from person to person. Therefore, in algorithm study, comparisons are limited to some objective
measures. Objective measures, unlike subjective measures, can be quantied. The advantages
of objective measures are that they are measurable and quantiable, and can be used for predic-
tions. Often time and space are used as objective measures for analysing algorithms.
Time complexity means the time taken by an algorithm to execute for different increasing
inputs (i.e., differently-scaled inputs). In algorithm study, two time factors are considered
execution time and run time. Execution time (or compile time) does not depend on problem
instances. Additionally, a program may be compiled many times. Therefore, time in an algo-
rithm context always refers to the run time as only this is characterized by instances. Another
complexity is space complexity, which is the measurement of memory space requirements
of an algorithm. Technically, an algorithm is efcient if lesser resources are used. Chapter 3
focuses on the analysis of algorithms.

KhW
 Introduction to Algorithms 13

In algorithm study, time complexity is not measured in absolute terms. For example, one
cannot say the algorithm takes 3.67 seconds. It is wrong as time complexity is always de-
noted as a complexity function t(n) or T(N), where t(n) is a function of input size n and is
not an absolute value. The variables n and N are used interchangeably and always reserved
to represent the input size of the algorithm. Recollect that input size is the number of binary
bits used to represent the input instance. The logic here is that for larger inputs the algorithm
would take more time. For example, sorting a list of 10 elements is easy, but sorting a list
of 1 billion elements is difcult. Generally, algorithms whose time complexity function is
a polynomial, for example, say N or log N, can be solved easily, and but problems whose
algorithms have exponential functions, say 2N, are difcult to solve.

Example 1.1 Assume that there are two algorithms A and B for a given problem P. The
time complexity functions of algorithms A and B are, respectively,
ly,, 3n and 2n. Which algorithm
3n an

s
n the
should be selected assuming that all other conditions remain he same for both algorithms?

es
6oOution Assuming that all conditions remain the same for both algorithms, the best

Pr
algorithm takes less time when the input is changedged
d to a la
larger value. Results obtained
ble 1.3
employing different values of n are shown in Table 1.3.

y
sit
Table 1.3 Time complexities
plexities
exities of algorithms
alg A and B
er
Input size (n) hm
m A T(n)
Algorithm T(n) = 3n
3n T n) = 2n
T(
Algorithm B T(n)

1 3 2
iv
Un

5 15 32

10 30 1024
d

100 300 2100

or
xf

It can bee observed that algorithm A performs better as n is increased; time complexity
nearly aand
increases linearly an gives lesser values for larger values of n. The second algorithm instead
O

enti
grows exponentially, and 2100 is a very large number.
Therefore, algorithm A is better than algorithm B.

Example 1.2 Let us assume that, for a problem P, two algorithms are possiblealgorithm
A and algorithm B. Let us also assume that the time complexities of algorithms A and B are,
respectively, 3n and 10n2 instructions. In other words, the instructions or steps of algorithms A
and B are 3n and 10n2 respectively. Here, n is the input size of the problem. Let the input size
n be 105 instructions. If the computer system executes 109 instructions per second, how much
time do algorithms A and B take?
6oOution Here, n = 105, and the computer can execute 109 instructions per second.
5
Therefore, algorithm A (time complexity 3n) would take 3 10 9
3
= 4 = 0.0003 seconds.
10 10
5 2
10 (10 ) 10 1010
Algorithm B (time complexity 10n2) would take = = 100 seconds.
109 109

KhW
14 Design and Analysis of Algorithms

It can be observed that algorithm A would take very less time compared to algorithm B.
Hence, it is natural that algorithm A is the preferred one.
One may wonder whether this has got anything to do efciency. Imagine that we are now
executing algorithm A on a slower machinea machine that executes only 105instructions.
What would be the scenario in this case?
5
Algorithm A would take 3 10 = 3 seconds.
105 5 2
We can see that algorithm A is still better compared to algorithm B that takes 10 (10
5
) =
10
106 seconds. Therefore, the important point to be noted here is that speed of the machine
does not affect selection of algorithm A as a better algorithm. Hence, while computer speed
is crucial, the role of a better designed algorithm is still signicant.

1.5.6 Implemenng an Algorithm and Performing Empirical

call Anal
Analy
Analysis

ss
After the algorithm is designed, it is expressed as a program and d a suitab
suitable programming language

re
is chosen to implement the algorithm as a code. After the he program is written, it must be tested.
Even if the algorithm is correct, sometimes the program ogram may not give expected results. This

yP
may be due to syntax errors in coding the algorithm thm in a programming language or hardware
ithm
fault. The error due to syntax problems of a languageanguage called a logical error. The process of
nguage is ca
sit
removing logical errors is called debugging. g. Iff the program
progra leads to an error even in the absence
pro
of syntax errors, then one has to go back ck to the de design stage to correct the algorithmic errors.
desig
er
Now, complexity analysis can be performperformed for the developed program. The analysis
gram
that is carried out after the program am is ddeve
developed is called empirical analysis (or a priori
iv

sis).
analysis or theoretical analysis). s). This ana
aanalysis is popular for larger programs; a new area,
Un

hmics,, whe
hmics
called experimental algorithmics, w
where analysis is performed for larger programs using a
dataset, is emerging. A dataset
datas
atas t is
i a huge collection of valid input data of an algorithm; the
hat iss often used for testing of algorithms is called a benchmark dataset.
standard dataset that
d

Interpretation off the resul

result
results of a program on a benchmark dataset provides vital statistical
or

information about ut the bbehaviour of the algorithm. This kind of analysis is called empirical
xf

analysis (also
also called a posteriori analysis). In addition, the term proling is often used to
denote thee proce
proces
process of running a program on a dataset and measuring the time/space require-
O

ment of the pro

program empirically.

1.5.7 Post or Postmortem Analysis

A problem-solving process ends with postmortem analysis. Any analysis should end with a valu-
able insight. The following are the questions that are often raised in this problem-solving process:
1. Is the problem solvable?
2. Are there any limits for this algorithm? Is there any theoretical limit for the efciency of
this problem?
3. Is the algorithm efcient, and are there any other algorithms that are faster than the current
algorithm?
Answers to these questions give a lot of insight into the algorithms that are necessary for
rening the design of algorithms to make them more efcient. The best possible or optimal
solution that is theoretically possible for a given problem species a lower bound of a given
problem. The worst-case estimate of resources required by the algorithm is called an upper

KhW
 Introduction to Algorithms 15

bound. Theoretically, the best solution of a problem should be closer to its lower bound. The
difference between the lower and upper bounds should be minimal for a better solution. The
difference between the upper and the lower bounds is called an algorithmic gap. Technically,
this should be zero. However, in practice, there may be vast difference between an upper and
a lower bound. A problem-solving process tries to reduce this difference by focusing on better
planning, design, and implementation on a continuous basis.

1.6 CLASSIFICATION OF ALGORITHMS

As algorithms are important for us, let us classify them for further elaboration. No single
criterion exists for classifying algorithms. Figure 1.5 shows some of the criteria used for
classication of algorithms.

ss
Algorithms

re
yP
Classication Classication Classication
Cla Classication
based on based on sit based on area of based on
implementation design specialization
spec tractability

Fig. 1.5 Classication

ssicatio oof algorithms
er
iv

1.6.1 Based on Implementaon

on
Un

Based on implementation, one

ne can cl
classi
classify the algorithms as follows
ZE/
E
E
d

Problem reduction is a scient

scientic
scienti principle for problem solving. Take a problem, reduce it,
or

eduction
duction pr
and repeat the reduction pro
process till the problem is reduced to a level where it can be solved
ng recursion
directly. Using recursion, the problem is reduced to another problem with a decrease in input
xf

instance. The
he transfo
transformed problem is the same as the original one but with a different input,
O

ss than that of the original problem. The problem reduction process is continued
which is less
till the given problem is reduced to a smaller problem that can be solved directly. Then the
results of the sub-problems are combined to get the result of the given problem. This strategy
of problem solving is called recursion.
Recursive algorithms use recursive functions for creating repetitions required for solving
a given problem.
A good example of recursion is computing the factorial of a number n. Factorial of a number
can be computed using the recursive function as follows:
0 if n = 0
n! =
n (n 1)! for n 1
Thus, a recursive function has a base case and an inductive case. A base case is the simplest
problem that can be solved directly. In this factorial example, the base case is nding 0!, as 0!
can be solved directly. An inductive case is a recursive denition of the problem that captures
the essence of a problem reduction process. It can be observed from Fig. 1.6, which shows the
computation of 4!, that recursion works by the principle of work postponement or delaying the

KhW
16 Design and Analysis of Algorithms

Reduce work. For a given factorial program, the facto-

4! rial function calls itself by reducing its input by
Reduce 1 till the program reaches 0! This is the base
3! case and the algorithm stops here; results of the
4321
Reduce
sub-problems are collected for computing the
2! nal answer of the given problem.
321
Reduce The correctness of recursive algorithms is
1! due to its close relationship with the concept
21 0! of mathematical induction. In other words, re-
cursion is a mirror of mathematical induction.
1 Non-recursive (or iterative) algorithms, on
Fig. 1.6 Computation of 4! using recursion the other hand, aree deductive in nature. Non-
recursive algorithmsms do not use the recursion
ithms

ss
concept but, instead, rely on looping constructs, such as for or while statement, to create repetition
hile statem
of tasks. Non-recursive (or iterative) algorithms and theirr ana lyses aare discussed in Chapter 2,
analyses

re
and recursive algorithms and their analyses in Chapter 3.

yP
EEW
An algorithm that is designed for a single processor
rocessor
cessor is cca
called a sequential algorithm.
sit
ystems
tems that
A parallel algorithm is designed for systems tha use
u a set of processors. The concept of
ocessing
essing are
parallel processing and distributed processing a interrelated. Parallel systems have mul-
er
osely. Distr
tiple processors that are located closely. Distribu
Distributed systems, on the contrary, have multiple
ifferent
ferent pl
processors that are located at different place
places separated by a vast distance geographically.
iv

Hence, distributed systems aree called lloo

loosely coupled systems, while parallel systems are
Un

ms. Distribu
called tightly coupled systems. Distri
Distributed algorithms are implemented in a distributed system
environment. Parallel algorith
algorithms are discussed in Chapter 20.
orithms ar
d



or

An exact algorithm
orithm
thm nds
nd the exact solutions for a given problem. Some problems are so
complex that
at nding ttheir exact solutions is difcult. Approximation algorithms (discussed
hat
xf

in Chapter
er 19 of this
th book) nd equivalent or approximate solutions for a given problem.
O

E

Deterministic algorithms always provide xed predictable results for a given input. In con-
trast, non-deterministic algorithms or randomized algorithms take a different approach. For
deterministic algorithms, the output should always be true. On the other hand, randomized
algorithms relax this condition. It is argued that outputs based on random decisions may not
often result in correct answers or the algorithm may not terminate at all. Thus, the accuracy
of an output is associated with a probability. In daily life, we often use random decisions,
for example, in games such as dice. Industries conduct random quality checks on products.
Randomized samples are used to predict poll results and so on. One can be very sceptical
about randomized algorithms. However, randomized algorithms have been proved to be very
effective. Randomized algorithms are discussed in Chapter 19.

1.6.2 Based on Design

Based on design techniques, algorithms can be classied into various categories. As discussed
earlier, every design technique uses a strategy for solving a problem. Algorithms can be
KhW
 Introduction to Algorithms 17

classied based on design as brute force, divide and conquer, dynamic programming, greedy
approach, backtracking, and branch and bound algorithms. This textbook is organized based
on this classication of algorithms only.

1.6.3 Based on Area of Specializaon

Algorithms can be classied based on the area of specialization. General algorithms such as
searching and sorting, and order statistics such as nding mean, median, rank, and so on are
useful for all elds of specialization. These algorithms can be considered as building blocks
of any area of specialization. Other than these basic algorithms, some algorithms are special-
ized for a particular domain. String algorithms, graph algorithms, combinatorial algorithms,
and geometric problems are some examples of specialized algorithms that are discussed in
this book. Let us discuss some of these algorithms now.
'

ss
Sorting algorithm is a general algorithm that is useful in all areas oof specialization. Sorting

re
problem is a structural problem. It involves structuring or rearrang
rearranging the sequence in a spe-
cic order. The importance of this problem arises fromom fact that all modern applications
m the fac

yP
require sorting. For example, an organization may y require
equi so ssorting of employee records based
on employee identication number (called primary
sitmary key). A library system may want books to
be ordered based on titles or ISBN numbers. Sorting
orting prob
pr
problems for small instances are relatively
easy. When sorting is required for a larger number, ssay a billion elements, designing sorting
gerr number
er
algorithms becomes more challenging. ng.
iv

^


Let us discuss some of these domain-spec
domain-sp
domain-specic algorithms now:
Un

String algorithms String ring algorithms

g al orith are used frequently in document editing, web searches,
orithms
and pattern matching. g.
A sequence off character
characters is called a string. The sequence can be text, bits, numbers, or
gene sequences es (A,
A, C, G, or T). One problem that is of considerable interest is string match-
ing, which takes a pattern,
patt
pat searches the text string, and reports whether a pattern is present
or not. String
ring algorithms,
ng algo
algor which are discussed in Chapter 16, constitute a very important
eld of study.
y
Graph algorithms A graph is used for modelling complex systems by representing the rela-
tionships among the subsystems. A graph represents a set of nodes and edge. The nodes are also
called vertices. The nodes are connected by edges, which are also known as arcs. Consider the
graph shown in Fig. 1.7. Here, the nodes are given as {A, B, C, D, E} and the edges as {(A,B),
(A,C), (A,D), (B,A), (B,E), (B,C), (C,A), (C,B), (C,D), (C,E), (D,A), (D,C), (D,E), (E,B), (E,C),
and (E,D)}.
A
Many interesting problems can be formulated using this graph structure. The
following are some of the interesting graph algorithms:
B C D
7raYeOOing saOesperson proEOeP As stated earlier, a salesperson starts from a
node, visits all other nodes only once, and gets back to the starting node. This
E
problem is reduced to a Hamiltonian cycle if the graph is undirected. In the
case of a weighted graph (where edges carry some weights), the TSP is about
Fig. 1.7 Sample graph nding a tour of minimum cost.
KhW
18 Design and Analysis of Algorithms

Chinese postman problem This problem is the same as a TSP, but instead of vertices, an
edge should be visited only once. There is no restriction on vertices in this problem.
Graph colouring problem This problem is about how to colour all the vertices distinctly using
only a small number of colours such that no two neighbouring vertices share the same colour.
Combinatorial algorithms The focus of combinatorial algorithms is to nd a combinatorial
object inherent in the problem such as permutations, combinations, or a subset that satises
some constraints and objective functions. These problems are difcult to solve, and many
problems do not have algorithmic solutions. TSPs and graph colouring problems are examples
of combinatorial problems.
Geometric algorithms Geometric algorithms deal with geometrical objects such as points,
lines, and polygons. The following are some of the algorithms discussed in this textbook:
Closest pair problem This problem deals with nding the distance tance bbetween points in a 2D

ss
space and nding a pair of points that are closest to each other
other.
her..

re
Convex hull problem This problem deals with nding smallest convex polygon that
ding the sm
includes all points in a 2D space.

yP
1.6.4 Based on Tractability sit
Tractability means solvability of a given n problem wiwithin a reasonable amount of time and
cultt to solv
space. An intractable problem is difcult solve w
within the reasonable amount of computer
er

hee following
resources. Based on tractability, the followin categories are possible:
iv

Easily solvable problems These hese prob

problem
problems have polynomial time complexity or their upper
Un

bounds are characterized by a polynomia

polynomial. These problems are solvable. For example, sorting
polynom
is an example of solvable polynomial problems.
blee or polynom
polyn
ems Prob
Unsolvable problems Problems such as halting problems cannot be solved at all. These
d

vable or no
are called unsolvable non
non-computable problems. Knowledge about the non-computability
or

lems
ms helps in project management.
of these problems
xf

ble
Intractable problem These problems are of two categories. One category comprises a
le problems
blems
lems that
set of problems tth have algorithmic solutions but require more computer resources. Hence,
O

these solutions are practically non-implementable. In addition, these have been proved to be
computationally hard. The other category consists of a set of problems that have been proved
to be computationally hard. For example, a TSP is a proven computationally hard problem;
it had already been discussed that time complexity of the algorithm increases exponentially
with an increase in the number of cities.

KhW
 Introduction to Algorithms 19

Thus, algorithm study can be concluded as the central theme of computer science. An as-
piring computer professional should be a better problem solver. The problem-solving process
starts with the understanding of a given problem and ends with nding an efcient program-
ming code for the given problem. Problem solving poses many challenges as it is a creative
process. Chapter 2 introduces the basics of a problem-solving process and also introduces
basic guidelines for writing algorithms.

Q SUMMARY Q

An algorithm is a step-by-step procedure for solving a Algorithm verication is a process of providing a math-
given problem. ematical proof that the given algorithm works correctly
A computational problem is characterized by two for all instances of data.
factorsspecication of valid input and output param- A proof of an algorithm m is said to exist if the precondi-
eters of the algorithm, and specication of the relation- tions can be shown to implymply postconditions
po logically.
ship between inputs and outputs. An estimation off thee time andaan space complexities of
An algorithm that yields the correct output for a legal an algorithm for varying input
in sizes is called algorithm
input is called an algorithmic solution. analysis.
The art of designing, analysing, and implementing Time complexity
omplex
le ty re
ref
refers to the measurement of run time
algorithms is called algorithmics. of an algorithm in terms of its input size, and space
Algorithms can be contrasted with programs. A program mplexity is the
complexity tth measurement of space required for
is an expression of algorithm in a programming language. a given a algorithm.
algo
A valid input is called an instance. The number of binary A data
datase
dataset is a huge collection of valid input data
bits necessary to encode inputs of an algorithm is called ed of an
a algorithm. The standard dataset that is often
the input size. used for testing of algorithms is called a benchmark
use
An algorithm should have characteristics such uch as a dataset.
well-dened order, inputs, outputs, niteness, ess, denite-
ss, denite The theoretically best possible or optimal solution for
ness, efciency, and generality. a given problem species a lower bound of a given
Problem solving starts with the understanding
nderstan
erstan iing of
o a problem. The worst-case estimate of resources that can
problem statement without any confusion. fusion. be required by an algorithm is called an upper bound.
A computation or computational nal model is an abstraction The difference between the upper and the lower bound
of a real-world computer. is called an algorithmic gap.
Algorithm design is a wayy of developing
deve
dev algorithmic Problem reduction is a scientic principle for problem
solutions using a suitable
itable course
uitable cour of action called a
cours solving. Take a problem, reduce it, and repeat the re-
design strategy. Algorithm
rithm specication
sp is about com- duction process till the problem is reduced to a level
municating the design strategy
tra to a programmer often where it can be solved directly.
in the form of a pseudocode. Algorithms can be categorized based on their imple-
Algorithmic validation means checking whether the mentation methods, design techniques, eld of study,
algorithm gives a correct result or not. and tractability.

Q GLOSSARY Q
Agent A performer of an algorithm $OJRULWKPYHULFDWLRQ The process of providing a mathemati-
Algorist A person who is skilled in algorithm development cal proof that the algorithm works correctly for all valid inputs
Algorithm A step-by-step procedure for solving a given Algorithm validation The process of checking the correct-
problem ness of an algorithm, this is done by giving valid inputs to
Algorithm gap The difference between lower and upper it and checking its results with expected values
bounds Approximation algorithms Algorithms that provide ap-
$OJRULWKPVSHFLFDWLRQ Formalization of an algorithm in proximate solutions for problems whose exact solutions
a suitable form that can be conveyed to a programmer are difcult to obtain

KhW
20 Design and Analysis of Algorithms

Complexity theory $ HOG RI DOJRULWKP VWXG\ WKDW GHDOV
ZLWK WKH DQDO\VLV RI DOJRULWKPV
Computability theory $ HOG RI VWXG\ WKDW GHDOV ZLWK WKH
WKHRU\ RI DOJRULWKPV DQG VROYDELOLW\ RI WKH DOJRULWKPV
Computational model $Q DEVWUDFWLRQ RI D UHDO-ZRUOG FRP-
SXWHU V\VWHP
Computational problem $ SUREOHP WKDW FDQ EH VROYHG E\
FRPSXWHU V\VWHPV
Deterministic algorithm $OJRULWKPV WKDW DOZD\V JLYH WKH
VDPH UHVXOW IRU D [HG LQSXW
Exact algorithm $Q DOJRULWKP WKDW LV GHVLJQHG WR JLYH DQ
H[DFW VROXWLRQ IRU D JLYHQ SUREOHP
Experimental algorithmics $ HOG RI VWXG\ WKDW GHDOV ZLWK
WKH DQDO\VLV RI DOJRULWKPV H[SHULPHQWDOO\ XVLQJ D ODUJH
GDWDVHW
Intractable problems 3UREOHPV WKDW FDQQRW EH VROYHG ZLWKLQ
Non-recursive algorithm (iterative algorithm) $Q DOJRULWKP
WKDW LV GHYHORSHG XVLQJ LWHUDWLYH FRQVWUXFWV IRU FUHDWLQJ
UHSHWLWLRQ RI WDVNV LQVWHDG RI XVLQJ UHFXUVLYH IXQFWLRQV
Parallel algorithm $Q DOJRULWKP GHVLJQHG IRU V\VWHPV WKDW
XVH PXOWLSOH SURFHVVRUV
3UROLQJ 7KH SURFHVV RI PHDVXULQJ WLPH DQG VSDFH FRP-
SOH[LW\ E\ UXQQLQJ D SURJUDP RQ D ODUJHU GDWDVHW
Proof 7KH ORJLFDO GHULYDWLRQ VKRZLQJ WKDW WKH SUHFRQGLWLRQV
RI DOJRULWKPV LPSO\ WKHLU SRVWFRQGLWLRQV
Recursive algorithm $Q DOJRULWKP WKDW XVHV D UHFXUVLYH
IXQFWLRQ
Sequential algorithm $Q DOJRULWKP WKDW LV GHVLJQHG IRU D
VLQJOH-SURFHVVRU V\VWHP
Space complexity 0HDVXUHPHQW RI VSDFH UHTXLUHG IRU
DQ DOJRULWKP
Strategy $ ZD\ RI GHVLJQLQJLJQLQJ DOJRULWKPV
D

ss
D UHDVRQDEOH DPRXQW RI FRPSXWHU UHVRXUFHV Time complexity 5XQ WLPH PHDVXUHPHQW RI WKH DOJRULWKP
Lower bound $ WKHRUHWLFDO EHVW VROXWLRQ IRU D JLYHQ SUREOHP ZKHQ WKH LQSXWS LV VFDOHG
SXW VFDOH WR D ODUJHU YDOXH

re
Non-deterministic algorithm $Q DOJRULWKPV WKDW XVHV UDQ- oblems 3UREOHPV
Tractable problems 3 IRU ZKLFK DOJRULWKPV WKDW

yP
GRPL]HG FKRLFHV WKDW GHWHUPLQH WKH FRXUVH RI H[HFXWLRQ QG VROXWLRQV
WLR V ZLWKLQ
ROXWLR Z
ZLWK D UHDVRQDEOH DPRXQW RI FRPSXWHU
RI LQVWUXFWLRQV KHQFH WKH RXWSXW RI DQ DOJRULWKP YDULHV VRXUFHV
RXUFHV H[LVW
UHVRXUFHV H
H[L
HYHQ IRU D [HG LQSXW pper bound 7
er boun
Upper 7KH ZRUVW-FDVH FRPSOH[LW\ RI DQ DOJRULWKP
sit
REVIEW QUESTIONS Q
W QUESTION
QUEST
er
Q

 'HQHDQDOJRULWKP  .

 :KDWLVDQDOJRULWKPVSHFLFDWLRQ":KDWLVDSVHXGR-
iv

 :KDWDUHWKHFKDUDFWHULVWLFVRIDQDOJRULWKP"P" FRGHDQGKRZGRHVLWKHOSLQDOJRULWKPVSHFLFDWLRQ"

Un

YH DOJRULWKPV
DOJRULWK
 6XUYH\WKH,QWHUQHWDQGOLVWRXWDWOHDVWYHDOJRULWKPV   :KDWLVWKHGLIIHUHQFHEHWZHHQDOJRULWKPYHULFDWLRQ
LYHV
YHV
WKDWKDYHKXJHLPSDFWRQRXUGDLO\OLYHV DQGDOJRULWKPYDOLGDWLRQ"
VROYLQJ"
YLQJ"
 :KDWDUHWKHVWDJHVRISUREOHPVROYLQJ"   +RZLVDQDOJRULWKPYDOLGDWHGDQGYHULHG"([SODLQ
d

 :KDWLVPHDQWE\WLPHDQGVSDFHFRPSOH[LWLHV"
SDFH FRPSOH[ ZLWKDQH[DPSOH
or

DUDGLJP.
UDGLJP.
 'HQHWKHWHUPGHVLJQSDUDGLJP  :KDWDUHWKHFULWHULDXVHGIRUFODVVLFDWLRQRIDOJRULWKPV"
xf

Q EXERCISES Q
O

 $VVXPHWKDWWKHUHDUHWZRDOJRULWKPV$DQG%IRUD
WZ
JLYHQSUREOHP37KHWLPHFRPSOH[LW\IXQFWLRQVRIDOJR-
ULWKPV$DQG%DUHUHVSHFWLYHO\nDQGORJ2n:KLFK
DOJRULWKPVKRXOGEHVHOHFWHGDVVXPLQJWKDWDOORWKHU
FRQGLWLRQVUHPDLQWKHVDPHIRUERWKWKHSUREOHPV"
2 /HWXVDVVXPHWKDWIRUDWHOHSKRQHGLUHFWRU\VHDUFK
SUREOHP3WKUHHDOJRULWKPVH[LVW$%DQG&7LPH
FRPSOH[LWLHV RI$ % DQG & DUH n n DQG ORJ n
UHVSHFWLYHO\$VVXPHWKDWWKHLQSXWLQVWDQFHnLV3.
$VVXPH WKDW WKH PDFKLQH H[HFXWHV 9 LQVWUXFWLRQV
SHU VHFRQG. +RZ PXFK WLPH ZLOO DOJRULWKPV $ % DQG
& WDNH" :KLFK DOJRULWKP ZLOO EH WKH EHVW"

Q ADDITIONAL PROBLEM Q
. -RKQ 0DF&RUPLFN KDG ZULWWHQ D ERRN WLWOHG Nine UHFRJQLWLRQ GDWD FRPSUHVVLRQ GDWDEDVHV DQG GLJLWDO
Algorithms That Changed the Future: The Ingenious VLJQDWXUHV WKDW FKDQJHG WKH ZRUOG.
Ideas that drive Todays Computers, 3ULQFHWRQ 8QLYHUVLW\ D :KDW DUH WKHVH DOJRULWKPV" 6HDUFK WKH ,QWHUQHW
3UHVV 3ULQFHWRQ WKDW KDG OLVWHG WKH QLQH ZRQGHUIXO DQG QG ZKDW WKHVH DOJRULWKPV DUH IRU.
DOJRULWKPV QDPHO\ VHDUFK HQJLQH LQGH[LQJ SDJH UDQN E ,GHQWLI\ RQH PRUH DOJRULWKP WKDW \RX IHHO FKDQJHV
SXEOLF NH\ FU\SWRJUDSK\ HUURU FRUUHFWLQJ FRGHV SDWWHUQ WKH ZRUOG ZH OLYH LQ.

KhW
 Introduction to Algorithms 21

Q CROSSWORD Q
1
2
3
4 5


7
8
9 10

s
11

es
y Pr
12 sit
13
er
iv

Across Down
D
Dow
ble
1. Algorithms that are solved within reasonable e amount 1.
1 Measurement of time
Un

of computable resources 2. Study of analysis of algorithms

ons
4. Algorithms that use recursive functions 3. A step-by-step procedure
. Who coined the word algorithm analysisysis 5. A person skilled in algorithms
d

7. An abstraction of a real world problem 9. Process of removing syntax errors

or

8. Statistical process of measuring

ing resources
suring resourc on a larger
resou
dataset
xf

10. Process of checking algorithms

11. Theory of algorithms
O

12. Analytical skills

13. Attributed as Father of Modern Computing by many

Answers to the Crossword are available as online resources

KhW