Chapter 8

Dynamic Programming
[Anany Levitin. 2012. Introduction to the Design and Analysis of Algorithms. 3
rd
Ed. Addison
Wesley. ISBN: 0132316811.]

Chapter Contents
8.1 Basic Examples
8.2 The Knapsack Problem
8.3 Floyds Algorithm

Dynamic programming (DP) is an algorithm design technique
DP is a general method for optimizing multistage decision processes.
The word programming in the name of this technique stands for planning.
DP is a technique for solving problems with overlapping subproblems. These subproblems
arise from a recurrence relating a given problems solution to solutions of its smaller
subproblems.
Rather than solving overlapping subproblems again and again, dynamic programming suggests
solving each of the smaller subproblems only once and recording the results in an array (1D or
2D array) from which a solution to the original problem can then be obtained.
Two types of DP approaches:
- The classic bottom-up dynamic programming approach solves all smaller subproblems of a
given problem.
- The top-down dynamic programming approach avoids solving unnecessary subproblems.
The crucial step in designing such a DP algorithm is deriving a recurrence relating a solution to
the problem to solutions to its smaller subproblems.
A majority of dynamic programming algorithms deal with optimization problems.
A general principle that underlines DP algorithms is the principle of optimality.
The principle of optimality says that an optimal solution to any instance of an optimization
problem is composed of optimal solutions to its subinstances.
The principal difficulty in developing a dynamic programming algorithm is defining a
recurrence relation.

8.1 Basic Examples

The coin-row problem: There is a row of n coins whose values are some positive integers c
1
,
c
2
, , c
n
, not necessarily distinct. The goal is to pick up the maximum amount of money subject
to the constraint that no two coins adjacent in the initial row can be picked up.

Let F(n) be the maximum amount that can be picked up from the row of n coins.
To derive a recurrence for F(n), we partition all the allowed coin selections into two groups:
- Group 1: includes the last coin.
- The largest amount we can get from the first group is equal to c
n
+ F(n - 2) - the value
of the nth coin plus the maximum amount we can pick up from the first n - 2 coins.
- Group 2: does not include the last coin.
1
- The maximum amount we can get from the second group is equal to F(n - 1) by the
definition of F(n).
Thus, we have the following recurrence:

F(n) = max{c
n
+ F(n - 2), F(n - 1)} for n > 1,
F(0) = 0, F(1) = c
1
. // subject to the initial conditions

We can compute F(n) by filling the one-row table left to right.

Here is pseudocode of the CoinRow algorithm

Algorithm CoinRow(C[1..n])
//Applies bottom-up DP to find the maximum amount of money
//that can be picked up from a coin row without picking two adjacent coins
//Input: Array C[1..n] of positive integers indicating the coin values
//Output: The maximum amount of money that can be picked up
F[0] 0; F[1] C[1]
for i 2 to n do
F[i] max(C[i] + F[i - 2], F[i - 1])
return F[n]

Example 1: Solve the coin-row problem by DP for the coin row 5, 1, 2, 10, 6, 2.
Answer: The application of the DP algorithm to the coin row of denominations 5, 1, 2, 10, 6, 2 is
shown in Figure 8.1. It yields the maximum amount of 17.


FIGURE 8.1 Solving the coin-row problem by dynamic programming for the coin row
2
5, 1, 2, 10, 6, 2.

To find the coins with the maximum total value found, we need to backtrack the computations
to see which of the two possibilities - c
n
+ F(n - 2) or F(n - 1) - produced the maxima in formula
(8.3).
- In the last application of the formula, it was the sum c
6
+ F(4), which means that the coin c
6

= 2 is a part of an optimal solution.
- Moving to computing F(4), the maximum was produced by the sum c
4
+ F(2), which means
that the coin c
4
= 10 is a part of an optimal solution as well.
- Finally, the maximum in computing F(2) was produced by F(1), implying that the coin c
2
is
not the part of an optimal solution and the coin c
1
= 5 is.
- Thus, the optimal solution is {c
1
, c
4
, c
6
}.
Using the CoinRow algorithm to find F(n), the largest amount of money that can be picked up,
as well as the coins composing an optimal set, clearly takes (n) time and (n) space.

The change-making problem: Give change for the amount n using the minimum number of
coins of denominations d
1
< d
2
< < d
m
, where d
1
= 1. Assume that the quantities of coins for
each of the m denominations are unlimited.

Let F(n) be the minimum number of coins whose values add up to n. Of course, we have F(0) =
0.
The amount n can only be obtained by adding one coin of denomination d
j
to the amount n - d
j

for j = 1, 2, , m such that d
j
n.
Therefore, we can consider all such denominations and select the one minimizing F(n - d
j
) + 1.
We have the following recurrence for F(n):

F(n) = min{F(n - d
j
)} + 1 for n > 0,
j : d
j
n
F(0) = 0. // initial condition

We can compute F(n) by filling a one-row table left to right.
Here is pseudocode of the ChangeMaking algorithm

Algorithm ChangeMaking(D[1..m], n)
//Applies dynamic programming to find the minimum number of coins
//of denominations d
1
< d
2
< < d
m
where d
1
= 1 that add up to a
//given amount n
//Input: Positive integer n and array D[1..m] of increasing positive
// integers indicating the coin denominations where D[1] = 1
//Output: The minimum number of coins that add up to n
F[0] 0
for i 1 to n do // amount 1, 2, , n
temp ; j 1
while j m and D[j] i do
temp min(F[i - D[j]], temp) // find smallest F(n - d
j
)
j j + 1 // next denominations
F[i] temp + 1
3
return F[n]

Example 1: Solve the change-making problem by DP for the amount n = 6 and the
denominations 1, 3, and 4.
Answer: The application of the algorithm to amount n = 6 and denominations 1, 3, and 4 is
shown in Figure 8.2.


FIGURE 8.2 Application of Algorithm ChangeMaking to amount n = 6 and coin
denominations 1, 3, and 4.

To find the coins of an optimal solution, we need to backtrack the computations to see which of
the denominations produced the solution.
The last application of the algorithm was produced by d
2
= 3. The second minimum (for n = 6 -
3) was also produced for a coin of denomination 3. Thus, the minimum-coin set for n = 6 is two
d
2
s (i.e., two 3s).
The time and space efficiencies of the change-making algorithm are O(nm) and (n),
respectively.

8.2 The Knapsack Problem

The knapsack problem: Given n items of known weights w
1
, , w
n
and values v
1
, , v
n
and a
knapsack of capacity W, find the most valuable subset of the items that fit into the knapsack.
We assume that all the weights and the knapsack capacity are positive integers.
The item values do not have to be integers.
To design a dynamic programming algorithm, we need to derive a recurrence relation that
expresses a solution to an instance of the knapsack problem in terms of solutions to its smaller
subinstances.
4
Let us consider an instance defined by the first i items, 1 i n, with weights w
1
, , w
i
, values
v
1
, , v
i
, and knapsack capacity j, 1 j W.
Let F(i, j) be the value of an optimal solution to the instance of the first i items, i.e., the value of
the most valuable subset of the first i items that fit into the knapsack of capacity j.
We can divide all the subsets of the first i items that fit the knapsack of capacity j into two
categories:
- Category 1: does not include the ith item.
- The value of an optimal subset is, by definition, F(i - 1, j).
- Category 2: includes the ith item (hence, w
i
j).
- An optimal subset is made up of the ith item and an optimal subset of the first i - 1 items
that fits into the knapsack of capacity j - w
i
. The value of an optimal subset is v
i
+ F(i -
1, j - w
i
).
Thus, the value of an optimal solution among all feasible subsets of the first i items is the
maximum of these two values.
Of course, if the ith item does not fit into the knapsack, the value of an optimal subset selected
from the first i items is the same as the value of an optimal subset selected from the first i - 1
items.
We have the following recurrence relation:

max{F(i - 1, j), v
i
+ F(i - 1, j - w
i
)} if j w
i
,
F(i, j) =
F(i - 1, j) if j < w
i
.

F(0, j) = 0 for j 0 and F(i, 0) = 0 for i 0. // initial conditions

Our goal is to find F(n, W), the maximal value of a subset of the n given items that fit into the
knapsack of capacity W, and an optimal subset itself.
Here is pseudocode of the bottom-up DP algorithm for the knapsack problem

Algorithm DPKnapsack(w[1..n], v[1..n], W)
//Solves the knapsack problem by dynamic programming (bottom up)
//Input: Arrays w[1..n] and v[1..n] of weights and values of n items,
// knapsack capacity W
//Output: The table F[0..n, 0..W] that contains the value of an optimal
// subset in F[n, W] and from which the items of an optimal
// subset can be found
for i 0 to n do F[i, 0] 0
for j 0 to W do F[0, j] 0
for i 1 to n do
for j 1 to W do
if j w[i]
F[i, j] max{ F[i - 1, j], v[i] + F[i - 1, j - w[i]]}
else // j < w[i]
F[i, j] F[i - 1, j]
return F[0..n, 0..W]

5
The values related to F(i, j) are illustrated in Figure 8.4.

0 j - w
i
j W
0 0 0 0 0
0
i - 1 0 F(i - 1, j - w
i
) F(i - 1, j)
w
i
, v
i
i 0 F(i, j)
0
n 0 goal

FIGURE 8.4 Table for solving the knapsack problem by dynamic programming.

For i, j > 0, to compute the entry in the ith row and the jth column, F(i, j), we compute the
maximum of the entry in the previous row and the same column and the sum of v
i
and the entry
in the previous row and w
i
columns to the left.
The table can be filled either row by row or column by column.

Example 1: Consider the instance of the knapsack problem given by the following data.

item weight value
1 2 $12
2 1 $10 W = 5
3 3 $20
4 2 $15

The DP table is shown in Figure 8.5.


FIGURE 8.5 Example of solving an instance of the knapsack problem by the dynamic
programming algorithm.

Thus, the maximal value is F(4, 5) = $37.
We can find the composition of an optimal subset by backtracking the computations of this
entry in the table.
- Since F(4, 5) > F(3, 5), item 4 has to be included in an optimal solution along with an
optimal subset for filling 5 - 2 = 3 remaining units of the knapsack capacity. The value of
the latter is F(3, 3).
- Since F(3, 3) = F(2, 3), item 3 need not be in an optimal subset.
6
- Since F(2, 3) > F(1, 3), item 2 is a part of an optimal selection, which leaves element F(1, 3
- 1) to specify its remaining composition.
- Since F(1, 2) > F(0, 2), item 1 is the final part of the optimal solution {item 1, item 2, item
4}.
The time efficiency and space efficiency of this algorithm are both in (nW).
The time needed to find the composition of an optimal solution is in O(n).

8.3 Floyds Algorithm

Floyds Algorithm for the All-Pairs Shortest-Paths Problem

The All-Pairs Shortest-Paths Problem: Given a weighted connected graph (undirected or
directed) that does not contain a cycle of a negative length, the all-pairs shortest-paths problem
asks to find the lengths of the shortest paths from each vertex to all other vertices.
The lengths of shortest paths are recorded in an n n matrix D called the distance matrix: the
element d
ij
in the ith row and the jth column of the distance matrix indicates the length of the
shortest path from the ith vertex to the jth vertex.
Example 1: The weighted connected graph, its weight and distance matrices are shown in
Figure 8.14.


FIGURE 8.14 (a) Digraph. (b) Its weight matrix. (c) Its distance matrix.

Floyds algorithm computes the distance matrix of a weighted graph with n vertices through a
series of n n matrices:

D
(0)
, , D
(k-1)
, D
(k)
, , D
(n)
.

The element in the ith row and the jth column of matrix D
(k)
(i, j = 1, 2, , n, k = 0, 1, ,
n) is equal to the length of the shortest path among all paths from the ith vertex to the jth vertex
with each intermediate vertex, if any, numbered not higher than k.
) (k
ij
d
The first matrix D
(0)
does not allow any intermediate vertices in its paths. Hence, D
(0)
is the
weight matrix of the graph.
The last matrix D
(n)
contains the lengths of the shortest paths among all paths that can use all n
vertices as intermediate. Thus, D
(n)
is the distance matrix being sought.
We can compute all the elements of each matrix D
(k)
from its immediate predecessor D
(k-1)
.
Let be the element in the ith row and the jth column of matrix D
(k)
. This means that is
equal to the length of the shortest path among all paths from the ith vertex v
i
to the jth vertex v
j

with their intermediate vertices numbered not higher than k:
) (k
ij
d
) (k
ij
d
7

v
i
, a list of intermediate vertices each numbered not higher than k, v
j
.

We can partition all such paths into two disjoint subsets:
- Subset 1: does not use the kth vertex v
k
as intermediate vertex.
- Since the paths of the first subset have their intermediate vertices numbered not higher
than k - 1, the shortest of them is, by definition of our matrices, of length .
) 1 ( k
ij
d
- Subset 2: use the kth vertex v
k
as intermediate vertex.
- What is the length of the shortest path in the second subset?
- Assume that the paths in the second subset use vertex v
k
as their intermediate vertex
exactly once (because visiting v
k
more than once increases the paths length).
- All the paths in the second subset have the following form:

v
i
, vertices numbered k - 1, v
k
, vertices numbered k - 1, v
j
.

- In other words, each of the paths is made up of a path from v
i
to v
k
with each
intermediate vertex numbered not higher than k - 1 and a path from v
k
to v
j
with each
intermediate vertex numbered not higher than k - 1. The situation is demonstrated in
Figure 8.15.


FIGURE 8.15 Underlying idea of Floyds algorithm.

Since the length of the shortest path from v
i
to v
k
among the paths that use intermediate vertices
numbered not higher than k - 1 is equal to and the length of the shortest path from v
k
to v
j

among the paths that use intermediate vertices numbered not higher than k - 1 is equal to ,
the length of the shortest path among the paths that use the kth vertex is equal to + .
) 1 ( k
ik
d
) 1 ( k
kj
d
) 1 ( k
kj
) 1 ( k
ik
d d
Taking into account the lengths of the shortest paths in both subsets, we have the following
recurrence:

) (k
ij
d = min{ , + } for k 1,
) 1 ( k
ij
d
) 1 ( k
ik
d
) 1 ( k
kj
d
) 0 (
ij
d = w
ij
. // initial condition

8
The element in row i and column j of the current distance matrix D
(k)
is replaced by the sum of
the elements in the same row i and the column k and the row k and in the same column j if and
only if the sum is smaller than its current value.
Here is pseudocode of Floyds algorithm.

Algorithm Floyd(W[1..n, 1..n])
//Implements Floyds algorithm for the all-pairs shortest-paths problem
//Input: The weight matrix W of a graph with no negative-length cycle
//Output: The distance matrix of the shortest paths lengths
D W //is not necessary if W can be overwritten
for k 1 to n do
for i 1 to n do
for j 1 to n do
D[i, j] min{D[i, j], D[i, k] + D[k, j]}
return D

Obviously, the time efficiency of Floyds algorithm is cubic (i.e., (n
3
)).
The application of Floyds algorithm to the graph in Figure 8.14 is illustrated in Figure 8.16.


FIGURE 8.16 Application of Floyds algorithm to the digraph shown. Updated elements
9
10
are shown in bold.