You are on page 1of 35

# Greedy Programming

Deliverables

## Coin Change Problem

Interval Scheduling

## Optimal Merge Sort

Huffman compression

## 0/1 and partial knapsack

Minimizing Lateness

## 6/7/2012 8:57 PM Copyright @ gdeepak.Com 2

Primarily used in Optimization problems to find
the maximum or minimum value satisfying
given constraints.

## It helps in constructing solution for a problem by

a sequence of steps (piecewise) where each step
is considered a partial solution, which is
extended progressively to get complete solution.

## 6/7/2012 8:57 PM Copyright @ gdeepak.Com 3

Choice of steps
Choice of each step in greedy is based on following

It must be feasible

## It must be locally optimal

It must be irrevocable

## 6/7/2012 8:57 PM Copyright @ gdeepak.Com 4

5 Pillars of Greedy Technique
Proof of correctness is necessary and important.
1. A candidate set, from which a solution is created
2. A selection function, which chooses the best candidate to be
3. A feasibility function that is used to determine if a candidate
can be used to contribute to a solution
4. An objective function, which assigns a value to a solution, or
a partial solution, and
5. A solution function, which will indicate when we have
discovered a complete solution

## 6/7/2012 8:57 PM Copyright @ gdeepak.Com 5

General Greedy Algorithm
a[1:n]; sol ;
for i 1 to n do
x SELECT(a); //as per some optimization criteria
if FEASIBLE(sol, x) then solution UNION(sol, x);
return sol

## Feasibility checks whether the selected element meets the

feasibility criteria and Union integrates the element x in the
solution

## 6/7/2012 8:57 PM Copyright @ gdeepak.Com 6

Coin Change Problem
An amount and collection of coins to reach to that amount Objective is to
minimize the number of coins
Greedy theory says always return the largest coin
if coins are of denomination 32, 8, 1 it has the greedy choice property because
no amount over 32 can be made without omitting 32.

## if coins are of denomination 30,20,5,1 then it doesnt have greedy choice

property because 40 can be made with two 20 coins but it returns 30,5,5 coins

Similarly if denominations are 1, 10, 21, 34, 70, 100, 350 140 Greedy: 100, 34,
1, 1, 1, 1, 1, 1 Optimal: 70, 70

## 6/7/2012 8:57 PM Copyright @ gdeepak.Com 7

Interval Scheduling Problem
Given set S = {a1, , an} of activities and activity start and
finish times, e.g.
i 1 2 3 4 5 6 7 8 9 10 11
si 1 3 0 5 3 5 6 8 8 2 12
fi 4 5 6 7 8 9 10 11 12 13 14
Activities are compatible if their time intervals do not
overlap, i.e., ai is compatible with aj is si fj or sj fi.
To find Subset of non overlapping intervals so that number
of intervals are maximum.
6/7/2012 8:57 PM Copyright @ gdeepak.Com 8
Earliest Start Time

## 6/7/2012 8:57 PM Copyright @ gdeepak.Com 9

Shortest interval First

## 6/7/2012 8:57 PM Copyright @ gdeepak.Com 10

Fewest Conflicts First

## 6/7/2012 8:57 PM Copyright @ gdeepak.Com 11

Interval Scheduling Algorithm
Interval that ends first and remove the intervals that
overlap with these and recurse.
Sort jobs by finish times so that f1 f2 ... fn.
A
for j = 1 to n
{
if (job j compatible with A)
A A {j}
}
return A

## 6/7/2012 8:57 PM Copyright @ gdeepak.Com 12

Efficient greedy algorithm
Once youve identified a reasonable greedy heuristic:
Prove that it always gives the correct answer
Develop an efficient solution

## 6/7/2012 8:57 PM Copyright @ gdeepak.Com 13

Is our greedy approach correct?
Stays ahead argument: show that no matter what other
solution someone provides you, the solution provided by
choice could do better

## 6/7/2012 8:57 PM Copyright @ gdeepak.Com 14

Is our greedy approach correct?
Let r1, r2, r3, , rk be the solution found by our approach

r1 r2 r3
rk
Let o1, o2, o3, , ok of another optimal solution

o1 o2 o3
ok
Show our approach stays ahead of any other solution

r1 r2 r3
rk

o1 o2 o3
ok

## 6/7/2012 8:57 PM Copyright @ gdeepak.Com 16

r1 r2 r3
rk

o1 o2 o3
ok

finish(r1) finish(o1)

r2 r3
rk

o2 o3
ok

## 6/7/2012 8:57 PM Copyright @ gdeepak.Com 18

Optimal Merge Sort
Suppose there are 3 sorted lists L1, L2, and L3, of sizes 30, 20, and 10,
respectively, which need to be merged into a combined sorted list but
we can merge only two at a time.
We intend to find an optimal merge pattern which minimizes the total
number of comparisons.

Merge L1 & L2,: 30 + 20 = 50 comparisons , then merge the list & L3:
50 + 10 = 60 comparisons total number of comparisons: 50 + 60 = 110.

## Alternatively, merge L2 & L3: 20 + 10 = 30 comparisons, the resulting

list (size 30) then merge the list with L1: 30 + 30 = 60 total number of
comparisons: 30 + 60 = 90.

## 6/7/2012 8:57 PM Copyright @ gdeepak.Com 19

Merge sort example

## 6/7/2012 8:57 PM Copyright @ gdeepak.Com 20

Huffman Compression
File f has one million characters. If only a, b, c, d, e, f chars come in the file then the
total size of the file is 8 million bits

If we represent these with 000 001 010 011 101 111 then only 3 million bits
0 1 00 11 10 11 will give us less then 2 million bits

## Assume frequency of occurrence of a, b, c, d, e, f is 45%,13%,12%,16%,9%,5%

leaves of the binary tree will represent the bit strings

It will give us the prefix free code. use greedy for making the binary tree and then
assign 0 to every left node and 1 to every right node. The nodes with larger frequency
will be nearer to the root and with lesser frequency will be towards the leaves.

Huffman Example

## 6/7/2012 8:57 PM Copyright @ gdeepak.Com 22

Knapsack
Given n objects and a knapsack. Object i has weight wi and the
knapsack has a capacity M.

## if a fraction xi, 0<= xi <= 1 of object i is placed into the knapsack

then a profit of pixi is earned. The objective is to obtain a filling of
the knapsack that maximizes the total profit earned.

## Since knapsack capacity is M, total weight of all chosen objects

should be M. The profits and weights are positive

## 6/7/2012 8:57 PM Copyright @ gdeepak.Com 23

Partial Knapsack-Greedy

## 6/7/2012 8:57 PM Copyright @ gdeepak.Com 24

0/1 knapsack
1st is optimal but greedy will give 2nd solution

## 6/7/2012 8:57 PM Copyright @ gdeepak.Com 25

Algorithm for partial knapsack
Input Set S of items with weight wi , benefit bi and total weight W
Output amount xi of each item i to maximize benefit with weight
W for each item i in S
xi=0
vi=bi/wi
w=0
while w<W
remove item i with highest vi
xi=min(wi, W-w)
w=w+min(wi, W-w)

## 6/7/2012 8:57 PM Copyright @ gdeepak.Com 26

Single resource processes one job at a time.
Job j requires tj units of processing time and is due at time dj.
If j starts at time sj, it finishes at time fj = sj + tj.
Lateness: lj = max { 0, fj - dj }.
Goal: Schedule all jobs to minimize maximum lateness
L=max lj.
1 2 3 4 5 6
tj 3 2 1 4 3 2
dj 6 8 9 9 14 15
6/7/2012 8:57 PM Copyright @ gdeepak.Com 27
Minimizing lateness
Consider jobs in ascending order of processing time tj

1 2
tj 1 10
dj 100 10
Consider jobs in ascending order of slack dj - tj
1 2
tj 1 10
dj 2 10

## 6/7/2012 8:57 PM Copyright @ gdeepak.Com 28

Minimizing Lateness

## Consider Jobs in the sequence of earliest deadline first

T 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14
J 1 1 1 2 2 3 4 4 4 4 5 5 5 6 6
D 6 6 6 8 8 9 9 9 9 9L 14 14 14 15 15

## 6/7/2012 8:57 PM Copyright @ gdeepak.Com 29

Single Job-Single resource
There is no preemption and these have to be allocated
processor. Only one processor is available. Objective is to
minimize the sum of the finishing times
f1=t1
f2=t1+t2
fi=t1+t2++ti
:
fn=nt1+(n-1)t2++tn
So t1 should be smallest and so on

Question 1

## Considering the following graph as a multi stage graph and using

Greedy Approach the shortest path is
A) 9
B) 23
C) 5
D) 17

## 6/7/2012 8:57 PM Copyright @ gdeepak.Com 32

Question 2
Greedy Technique is related to the theory of
A) Calculus
B) Probability
C) Matroid
D) Algebra

## 6/7/2012 8:57 PM Copyright @ gdeepak.Com 33

Question 3
For those problems where Greedy gives the optimal
solution, the complexity is better as compared to the
implementation of the same problem with Dynamic
Programming.
A) True
B) False

## 6/7/2012 8:57 PM Copyright @ gdeepak.Com 34

??
Given a question, Can you find whether greedy strategy
work in general is NP-Complete.