Dynamic Programming

Kun-Mao Chao ( ) Department of Computer Science and Information Engineering National Taiwan University, Taiwan
E-mail: kmchao@csie.ntu.edu.tw WWW: http://www.csie.ntu.edu.tw/~kmchao

Dynamic Programming
Dynamic programming is a class of solution methods for solving sequential decision problems with a compositional cost structure. Richard Bellman was one of the principal founders of this approach.

Two key ingredients

Two key ingredients for an optimization problem to be suitable for a dynamicprogramming solution:
1. optimal substructures

2. overlapping subproblems

Subproblems are dependent. Each substructure is optimal. (otherwise, a divide-andconquer approach is the (Principle of optimality) 3 choice.)

Three basic components

The development of a dynamicprogramming algorithm has three basic components:
The recurrence relation (for defining the value of an optimal solution); The tabular computation (for computing the value of an optimal solution); The traceback (for delivering an optimal solution).
4

Fibonacci numbers
The Fibonacci numbers are defined by the following recurrence:
F !0 0 F !1 1 F ! F   F  for i>1. i i 1 i 2
5

How to compute F10 F9 F10 F8 F8 F7 F7 F6

6

Tabular computation
The tabular computation can avoid recompuation.
F0 F1 F2 F3 F4 F5 F6 F7 F8 F9 F10 0 1 1 2 3 5 8 13 21 34 55

Longest increasing subsequence(LIS)

The longest increasing subsequence is to find a longest increasing subsequence of a given sequence of distinct integers a1a2an . e.g. 9 2 5 3 7 11 8 10 13 6 2 3 7 5 7 10 13 9 7 11
are not increasing subsequences. are increasing subsequences. We want to find a longest one.

3 5 11 13
8

A naive approach for LIS

Let L[i] be the length of a longest increasing subsequence ending at position i.
L[i] = 1 + max j = 0..i-1{L[j] | aj < ai} (use a dummy a0 = minimum, and L[0]=0) 9 2 5 3 7 11 8 10 13 6
L[i] 1 1 2 2 3 4 ?

A naive approach for LIS

L[i] = 1 + max j = 0..i-1 {L[j] | aj < ai} 9 2 5 3 7 11 8 10 13 6
L[i] 1 1 2 2 3 4 4 5 6 3
The maximum length

The subsequence 2, 3, 7, 8, 10, 13 is a longest increasing subsequence. This method runs in O(n2) time.
10

Binary search
Given an ordered sequence x1x2 ... xn, where x1<x2< ... <xn, and a number y, a binary search finds the largest xi such that xi< y in O(log n) time.
n/2 n/4

...
11

Binary search
How many steps would a binary search reduce the problem size to 1? n n/2 n/4 n/8 n/16 ... 1
How many steps? O(log n) steps.

n/2 !1

s ! log 2 n
12

An O(n log n) method for LIS

Define BestEnd[k] to be the smallest number of an increasing subsequence of length k. 9 2 5 3 7 11 8 10 13 6
9 2 2 2 5 3 2 3 7 2 3 7 11 2 3 7 8 2 3 7 2 3 7
BestEnd[1] BestEnd[2] BestEnd[3] BestEnd[4] BestEnd[5] BestEnd[6]
13

8 8 10 10 13

An O(n log n) method for LIS

Define BestEnd[k] to be the smallest number of an increasing subsequence of length k. 9 2 5 3 7 11 8 10 13 6
9 2 2 2 5 3 2 3 7 2 3 7 11 2 3 7 8 2 3 7 2 3 7 2 3 6
BestEnd[1] BestEnd[2] BestEnd[3] BestEnd[4] BestEnd[5] BestEnd[6]
14

For each position, we perform a binary search to update BestEnd. Therefore, the running time is O(n log n).

8 8 8 10 10 10 13 13

Longest Common Subsequence (LCS)

A subsequence of a sequence S is obtained by deleting zero or more symbols from S. For example, the following are all subsequences of president: pred, sdn, predent. The longest common subsequence problem is to find a maximum-length common subsequence between two sequences.
15

LCS
For instance, Sequence 1: president Sequence 2: providence Its LCS is priden.
president

providence
16

LCS
Another example: Sequence 1: algorithm Sequence 2: alignment One of its LCS is algm.
a l g o r i t h m

a l i g n m e n t
17

How to compute LCS?

Let A=a1a2am and B=b1b2bn . len(i, j): the length of an LCS between a1a2ai and b1b2bj With proper initializations, len(i, j)can be computed as follows.
0 if i ! 0 or j ! 0, len(i, j ) ! len(i  1, j  1)  1 if i , j " 0 and ai ! b j , max(len(i, j  1), len(i  1, j )) if i , j " 0 and ai { b j .
18

procedure LCS-Length(A, B) 1. for i 0 to m dolen(i,0) 0 2. for j 1 to n dolen(0,j) 0 3. for i 4. 5. 6. 7. 8. 9. return len and prev for j 1 to m do 1 to n do ( len i, j) ! len(i 1, j 1) 1 if ai ! bj then ( previ, j) !" " ( ( else if len i 1, j) u len i, j 1)
len(i, j) ! len(i 1, j) then ( previ, j) !" " len(i, j) ! len(i, j 1) else ( previ, j) !" "

19

0 0

1 p
0 1 1 1 1 1 1 1 1 1

2 r
0 1 2 2 2 2 2 2 2 2

3 o
0 1 2 2 2 2 2 2 2 2

4 v
0 1 2 2 2 2 2 2 2 2

5 i
0 1 2 2 2 3 3 3 3 3

6 d
0 1 2 2 2 3 4 4 4 4

7 e
0 1 2 3 3 3 4 5 5 5

8 n
0 1 2 3 3 3 4 5 6 6

9 c
0 1 2 3 3 3 4 5 6 6

10 e
0 1 2 3 3 3 4 5 6 6

0 1 2 3 4 5 6 7 8 9

p r e s i d e n t

0 0 0 0 0 0 0 0 0

20

proce 1 2 3 4

re Output-LCS( , prev, i, j)

if i = 0 or j = 0 t e retur Output  LCS ( A, prev, i  1, j  1) if prev(i, j)= t e print ai else if prev(i, j)= t e Output-LCS( , prev, i-1, j) else Output-LCS( , prev, i, j-1)

21

0 0

1 p
0 1 1 1 1 1 1 1 1 1

2 r
0 1 2 2 2 2 2 2 2 2

3 o
0 1 2 2 2 2 2 2 2 2

4 v
0 1 2 2 2 2 2 2 2 2

5 i
0 1 2 2 2 3 3 3 3 3

6 d
0 1 2 2 2 3 4 4 4 4

7 e
0 1 2 3 3 3 4 5 5 5

8 n
0 1 2 3 3 3 4 5 6 6

9 c
0 1 2 3 3 3 4 5 6 6

10 e
0 1 2 3 3 3 4 5 6 6

0 1 2 3 4 5 6 7 8 9

p r e s i d e n t

0 0 0 0 0 0 0 0 0

Output: priden

22

Longest Common Increasing Subsequence

Proposed by Yang, Huang and Chao
IPL 2005

Improvement for some special case:

Katriel and Kutz (March 2005) Chan, Zhang, Fung, Ye and Zhu (ISAAC 2005)

23