chapter 6 dynamic programming -...

C.K. Liang al-06 dynamic programming 1

Chapter 6

Dynamic Programming


Introduction

• Motivation:

– Top-down algorithm design is natural and powerful.

– Solve a complex problem by breaking it down to subproblems.

– Divide-and-Conquer approach.

– Recursion has its limitations.

• Example: Fibonacci numbers

– 0, 1, 1, 2, 3, 5, 8, 13, 21, ...

– F(n)=F(n-1) + F(n-2) for n > 1, F(0)=0 and F(1)=1.

– Recursive computation of F(5): Total 15 calls.


An example

5

4 3

3 2 2 1

2 1 1 0 1 0

1 0

The amount of work

done by recursive

computation of F(n)

is more than (1.6)n.

Note:


General concept

• Properties:

– Main Idea of Dynamic Programming: (Do not calculate the same thing

twice).

– Dynamic programming is a bottom-up technique.

– A dynamic programming algorithm stores the results, or solutions, for

small sub-problems and looks them up, rather than re-computing them,

when it needs them later to solve larger sub-problems.

– Dynamic programming is especially well suited to problems in which a

recursive algorithm would solve many of the sub-problems repeatedly.

– Usually keeps a table of known results, which we fill up sub-instances are

solved.


DP vs. DAC

• Dynamic programming VS D-A-C:

– Dynamic programming, like the divide-and-conquer method, solves

problems by combining the solutions to sub-problems.

– Divide-and-conquer algorithms partition the problem into independent sub-

problems, solve the sub-problems recursively, and then combine their

solutions to solve the original problem.

– Dynamic programming is applicable when the sub-problems are not

independent, that is, when sub-problems share subsubproblems.

– Programming refers to a tabular method.

– A dynamic programming algorithm solves every sub-problem just once

and then saves its answer in a table.


Matrix-chain multiplication

• Problem:

– Given a sequence (chain) <A1,A2,A3,...,An> of n matrices to be multiplied,

we wish to compute the product

A1*A2*A3*...*An.

– Matrix multiplication is associative, and so all parenthesizations yield the

same product.

– We want a way of parenthesization that minimizes the number of scalar

multiplications.

A1*A2*A3*A4

(A1 (A2 (A3 A4) ) )

(A1 ( (A2 A3) A4) )

( (A1 A2) (A3 A4) )

( (A1 (A2 A3) ) A4)

( ( (A1 A2) A3) A4)

A1:13*5, A2:5*89, A3:89*3, A4:3*34

(A1 (A2 (A3 A4) ) ) = 10582

(A1 ( (A2 A3) A4) ) = 4055

( (A1 A2) (A3 A4) ) = 54021

( (A1 (A2 A3) ) A4) = 2856

( ( (A1 A2) A3) A4) = 26418



• Properties:

– P(n): the number of different ways to parenthesize a sequence of n

matrices.

– P(n) = 1 if n=1;

– P(n) = SP(k)P(n-k), if n>=2

– P(n) = C(n-1)

– C(n) : Catalan number

– C(n) = W(4n/n3/2).

n 1 2 3 4 5 10 15

P(n) 1 1 2 5 14 4862 2674440

Note:



• How to solve?

– M(i,j) = the minimum number of multiplications needed to compute A1*...*Aj

for 1<=i<=j<=n.

– An optimal parenthesization of the product A1*...*An splits the product

between Ak and Ak+1, 1<=k<n.

– We want to compute M(i,n).

– Recursively compute M(i,j).

– Ai : di-1 * di

– M(i,j) = min{M(i,k)+M(k+1,j)+di-1*dk*dj} if i<j.(i<=k<j)

– M(i,i)=0



• Example:

– A1*A2*A3*A4, d=(13,5,89,3,34)

– M(1,2)=13*5*89=5785

– M(2,3)=5*89*3=1335

– M(3,4)=89*3*34=9078

– M(1,3)=min(M(1,1)+M(2,3)+13*5*3,M(1,2)+M(3,3)+13*89*3)

=min(1530,9256) = 1530

1 2 3 4

1 0 5785 1530 2856 2 0 1335 1845 3 0 9078 4 0

O(n3)


Shortest-path problem

• Problem:

– Give a directed graph G with starting and ending nodes S and T

respectively.

– We are asked to find the shortest path from S to T.

• Example:

– Greedy: S->A->D->T = 1+4+18=23

– optimal: S->C->F->T = 5+2+2 = 9

S

A

B

C

D

E

F

T

1

2

5

4

11 9

5

16

2

18

13

2



• How to solve?

– Use dynamic programming approach.

• Example:

– d(X,T) : the length of the shortest path from X to T.

– d(S,T) = min{1+d(A,T),2+d(B,T),5+d(C,T)}

S

A

B

C

T

1

2

5

d(A,T)

d(B,T)

d(C,T)



A D

E T

4

11 d(D,T)

d(E,T)

d(A,T) = min{4+d(D,T),11+d(E,T)}

d(B,T)=min{9+d(D,T),5+d(E,T),16+d(F,T)}

B

D

E

F

T

9

5

16

d(D,T)

d(E,T)

d(F,T)


Elements of dynamic programming

• When should we look for a dynamic-programming solution to a

problem ?

– optimal substructure

– overlapping sub-problems

– multi-stage problems


發展一 DP 演算法的步驟

1. Characterize the structure of an optimal solution.

2. Derive a recursive formula for computing the values of

optimal solutions.

3. Compute the value of an optimal solution in a bottom-up

fashion (top-down is also applicable).

4. Construct an optimal solution in a top-down fashion.


Optimal substructure

• Optimal substructure:

– We say that a problem exhibits optimal substructure if an optimal solution

to the problem contains within it optimal solutions to sub-problems.

• Examples:

– an optimal parenthesization of A1 A2 A3 ... An that splits the product

between Ak and Ak+1 contains within it optimal solutions to the problems of

parenthesizing A1 A2 ... Ak and Ak+1 Ak+2 ... An.

– for the shortest path problem, if X is a node on the shortest path from S to

T, then that part of the path from S to X, and that from X to T, must also be

optimal

• Principle of optimality:

– in an optimal sequence of decisions or choices, each subsequence must

also be optimal to sub-problem.


Overlapping subproblems

• Overlapping subproblems:

– When a recursive algorithm revisits the same problem over and over

again, we say that the optimization problem has overlapping sub-

problems.

– Typically, the total number of distinct sub-problems is a polynomial in the

input size.


Multistage graphs

• Multistage graphs:

– G(V,E) : directed graph in which the vertices are partitioned into k>= 2

disjoint sets Vi, 1 i k.

– If (u,v) E, then u Vi and v Vi+1 for some i.

– V1 and Vk are such that |V1|=|Vk|=1.

– S : source node.

– T : sink node

S T

0,1

1,1

2,1

3,1

0,3

1,3

2,3

3,3

0

2

8

9

0

5 6 7

0 5

6

0

5

0

5

4

2

0


Resource allocation problem

• Problem:

– n units of resource are to be allocated to r projects.

– if j units of resource are allocated to project i then the resulting net profit is

N(i,j).

– Goal: maximize the total net profit.

1 2 8 9 2 5 6 7 3 4 4 4 4 2 4 5

project

resource

1 2 3



• How to solve?

– Formulate the resource allocation problem as an r+1 stages graph

problem.

– stage i : represents project i.

– stage i contains n+1 vertices v(i,j), 0 j n.

– stages 1 and r+1 each have one vertex v(1,0)=S and v(r+1,n)=T

respectively.

– v(i,j) : total j units of resource have been allocated to projects 1,2,...,i-1.

– P(i,j) : minimum cost path from vertex j in vi to vertex T.

– cost(i,j): the cost of this path.

– cost(i,j)=min{c(j,x)+cost(i+1,x)}, where x in vi+1.

– multi-stage graph.

– Finding the longest path from S to T.



S T

0,1

1,1

2,1

3,1

0,2

1,2

2,2

3,2

0,3

1,3

2,3

3,3

0

2

8

9

0

5 6 7

0

5

6

0

5

0

0

4

4 4

0

4

4

0

4

0

5

4

2

0


Optimal binary search trees

• Problem:

– how to best arrange a set of keys in a binary search tree to minimize the

average search time if we know that some keys are looked up more often

than others.

– The key at each node is greater than all the keys in its left subtree and

less than all keys in its right subtree.

and come king pig said the time wind

call of talk wall

has

ring

thing



• Problem:

– There are many different binary trees.

• Example:

– 3, 7, 9, 12

3

7

9

12

(a)

3

7

9

12

(b)

3

7

9

12

(c)

12

9

3

7

(d)



• Problem:

– We want to find a binary search tree for the keys K1, K2, ..., Kn with search

prob. p1,p2, ..., pn that has minimum average search time.

• Example: average search time = 3.25.

Key # of searches Prob.(Pi) Comparisons(Ci) Pi*Ci and 30 0.15 4 0.6 call 5 0.025 3 0.075 come 10 0.05 4 0.2 has 5 0.025 2 0.05 king 10 0.05 4 0.375 of 25 0.125 3 0.375 pig 5 0.025 4 0.1 ring 15 0.075 1 0.075 said 15 0.075 4 0.3 talk 10 0.05 3 0.15 the 30 0.15 4 0.6 thing 15 0.075 2 0.15 time 10 0.05 2 0.1 wall 5 0.025 3 0.075 wind 10 0.05 4 0.2



• How to solve?

– Let di denote the depth of key Ki in the tree.

– For a given tree the average number of comparisons needed is C = Spi(di+1).

– We want to minimize the above function.

– Consider the sequence of successive keys Ki, Ki+1, ..., Kj, j>=i.

– Suppose that in an optimal tree containing all the n keys this sequence of

j-i+1 keys occupies the nodes of a substree.

– The average number of comparisons carried out in this subtree is C*= *

pk(dk*+1).

– The principle of optimality holds: in an optimal tree all subtrees must also

be optimal with respect to the keys they contain.



• How to solve?

– If we choose Kk as the root for the tree, then K1, K2, ..., Kk-1 must go in the

left subtree and Kk+1, ..., Kn must go in the right substree, and we now

need optimal arrangements for the two subtrees.

– How k should be chosen?

– We minimize over all choices.

– A[i,j] = minimum average search time for a binary search tree with keys Ki

< ... < Kj, where i<=i<=j<=n.



Kk

K1,...Kk-1 Kk+1,...,Kn

Ave. search time: A[1,k-1]

Ave. search time: A[k+1,n]

For each key searched, one additional

comparison is done at the root.

A[i,j]= min{ A[i,k-1] + S pq + A[k+1,j] + S pq + pk }

= min{ A[i,k-1] + A[k+1,j] } + S pq , i<=k<=j.

A[i,i] = pi.

Time: O(n3).


補充內容


鐵條切割問題 (Rod cutting

problem) 給一段長度為N(整數)單位的鐵條, 令i為任一正整數,

假設p[i]表示長度為i的鐵條可以賣出的價格.試問應

如何切割該鐵條使得其總賣價最高?

長度 i 1 2 3 4 5 6 7 8 9 10

價格p[i] 1 5 8 9 10 17 17 20 24 30

例：

N=7, 7=3+4可賣得17 7=1+6=2+2+3 可賣得18.


General concept

• Top-Down求算方式

• 子問題重覆 (Overlapping Subproblem) 是Divided-and-Conquer主要的問題所在

• 求算F8和F7時，F6會被用到2次，但我們用表格記錄已算過的部份!!

• Bottom-Up求算方式

• 動態規劃 (Dynamic Programming)是一種表格式的演算法設計原則。

– 其精神是將一個較大的問題定義為較小的子問題組合，先處理較小的問題，並將其

用表格儲存起來，再進一步地以較小問題的解逐步建構出較大問題的解。

– Programming: 用表格存起來，有 “以空間換取時間”之涵意

n 0 1 2 3 4 5 6 7 8

Fn 0 1 1 2 3 5 8 13 21


DP vs. DAC

• Divide-and-Conquer 和 Dynamic Programming都是將問題切割再

採用遞迴方式處理子問題，但是:

– Divide-and-Conquer可能會對相同子問題進行重覆計算

– Dynamic Programming會使用表格將子問題的計算結果加以儲存，在後面

階段如果需要這個計算結果，再直接由表格中取出使用，因此可以避免許多

重覆的計算，以提高效率。

• Divide-and-Conquer v.s. Dynamic Programming

Divide-and-Conquer Dynamic Programming

額外記憶體空間不需額外記憶體空間需要額外記憶體空間

解題方式 Top-Down Bottom-Up

適用時機適用non-overlap子問題適用overlap子問題


DP vs. Greedy

• 對於具有限制的最佳化問題，可以採用 “貪婪法則” 或 “動態規劃” 來設計

演算法則。

– Greedy Approach:

• 是一種階段性 (Stage) 的方法

• 具有一選擇程序 (Selection Procedure)，自某起始點(值) 開始，在每一個階段逐一檢查

每一個輸入是否適合加入答案中，重複經過多個階段後，即可順利獲得最佳解

• 較為簡單 (若遇最佳化問題，先思考可否用Greedy Approach解，若不行再考慮用

Dynamic Programming)

• 如果所要處理的最佳化問題無法找到一個選擇程序來逐一檢查，則需要以一次考慮所有的

可能情況的方式來處理，就是屬於Dynamic Programming

– Dynamic Programming

• 先把所有的情況都看過一遍，才去挑出最佳的結果

• 考慮問題所有可能的情況，將最佳化問題的目標函數表示成一個遞迴關係式，結合Table

的使用以找出最佳解


Dynamic Programming的解題方式

• 使用Dynamic Programming解決問題的過程如下：

– 先找出問題的遞迴式 (遞迴關係式; Recurrence Relations)

– 接著，透過Forward Approach或是Backward Approach，利用迴圈與表格

來處理遞迴式，進而解決問題。

• 假設有n個數值 (X1, X2, …, Xi-2, Xi-1, Xi, Xi+1, Xi+2, …, Xn-1, Xn)

• Forward Approach (前向法；順向法)

– 如果要計算出Xi的值，需要透過前方(Xi+1, …, Xn-1, Xn)等數值資料

– 由Xn向後求解問題的解答(Solved Backward)

• Backward Approach (後向法；逆向法)

– 如果要計算出Xi的值，需要透過後方(X1, X2, …, Xi-1)等數值資料

– 由X1向前求解問題的解答(Solved Forwards)



• 假設我們要將一個 2 × 3 的矩陣乘上一個 3 × 4 的矩陣：

– 所產生的結果為一個 2 × 4的矩陣

– 結果矩陣中的每一個元素都必須經過3次乘法的運算

– 因為結果矩陣中有 2 × 4 = 8個元素，因此總共需要的乘法次數為：

• 一般來說，一個 i × j matrix 乘上一個 j × k matrix ，總共需要的乘

法次數為：



• Chained Matrix Multiplication:

– Def: 給一Matrix Chain: A1, A2, …, An，求此Chain所需之乘法次數為最少之

括號方式 (即: 最佳的矩陣乘法組合方式)。

• 若Ai, Ai+1, …, Aj 在某組合方式所需的乘法次數為最小 (最佳)，則

必存在一個k，使得Ai, Ai+1, …, Ak 和Ak+1, Ak+2, …, Aj皆為最佳。

((Ai Ai+1 … Ak )(Ak+1 Ak+2 …, Aj))

最佳組合

最佳子組合最佳子組合



• Matrix Chain的遞迴式

• 此遞迴式涵蓋以下兩個規則：

– M[i][j] = 矩陣Ai乘到Aj所需的最少乘法數 (其中 i < j)

– M[i][i] = 0

ji if }dddj]1,M[kk]{M[i,min

ji if 0 j]M[i,

jk1i1jki



• Chained Matrix Multiplication 問題的演算法需有兩個表格和一個

主要變數:

– M[i, j]

• 記錄多個矩陣相乘 (e.g., Ai … Aj)時，所需的 “最少” 乘法次數

– P[i, j]

• 記錄多個矩陣相乘 (e.g., Ai … Aj) 所需最少乘法次數之 “最佳乘法順序” 是

由哪一個矩陣開始分割

– diagonal

• 主要指出在Matrix Chain中，每一次有多少個矩陣要相乘

– diagonal = 1 只有1個矩陣，∴不會執行乘法動作

– diagonal = 2 每一次有2個矩陣要相乘



– …



• 六個矩陣相乘的最佳乘法順序可以分解成以下的其中一種型式：

• 第k個分解型式所需的乘法總數，為前後兩部份 (一為A1, A2, …, Ak

和Ak+1, …, A6) 各自所需乘法數目的最小值相加，再加上相乘這前

後兩部份矩陣所需的乘法數目。



• Matrix Chain的遞迴式

• Example: A133, A

237, A

372, A

429, A

594, 求此五矩陣的最小乘法

次數。

Sol:

建立兩陣列 M[1…5, 1…5]及P[1…4, 2…5]

ji if }dddj]1,M[kk]{M[i,min

ji if 0 j]M[i,

jk1i1jki

M 1 2 3 4 5

1

2

3

4

5

P 2 3 4 5

1

2

3

4



A1, A2, A3, A4, A5

A1 A2, A2A3, A3A4, A4A5

A1 A2A3, A2A3A4, A3A4A5

A1 A2A3A4, A2A3A4A5

A1 A2A3A4A5

diagonal = 1

diagonal = 2

diagonal = 3

diagonal = 4

diagonal = 5


Dynamic Programming and

Optimization Problems • Dynamic Programming 和 Greedy Approach看似可以處理所有

的最佳化問題，然而它們所能處理的最佳化問題需滿足下列原則

– 最佳化原則(Principle of Optimality): 當一個問題存在著最佳解，則表示

其所有子問題也必存在著最佳解

• 以最短路徑問題來看，如果vk 是 vi 到 vj 間最短路徑上的一個頂點，則 vi 到

vk 以及從 vk 到 vj 這兩個子路徑也必定是最短路徑.


Dynamic Programming and

Optimization Problems • 每一個輸入的計算都必須是根據先前輸入的最佳結果再進一步計算，如此才能

夠得到最佳解，同時可以將無法獲得最佳解的情況去除，以避免需要將每一種

可能情況都加以考慮。

– 對於n個輸入的最佳化問題 (X1, X2, …, Xn):

• 有些被歸類為 “部份集合” (Subset) 問題，會有2n種可能的情況

• 有些被歸類為 “排列組合” (Permutation) 問題，會有n!種可能的情況

– 因此皆屬指數複雜度的問題，若可採用最佳化原則通常可以將這一些問題的

複雜度由指數複雜度降為多項式複雜度。

• 然而，並不是所有求最佳化的問題都合乎最佳化原則，此時就只

能用其它的方法求解了。


The Longest Common

Subsequence (LCS) Problem • 假設有一條字串(String) S:

– S = “a t g a t g c a a t”

– Substrings of S: “g a t g c”, “t g c a a t”.

– Subsequences of S: “a g g t”, “a a a a”.

• String

– a segment of consecutive characters.

– usually called sequence in Biology.

• Sequence

– need not be consecutive.

• In Biology, String = Sequence.


The Longest Common

Subsequence (LCS) Problem • Common subsequences of two string X = < B, C, B, A, C > and Y

= < B, D, A, B, C > :

– BC, BA, BB, BAC, BBC, ...

– The longest common subsequence (LCS) of X and Y = BAC or BBC

• [Note]

– LCS可能不唯一

– 若用暴力法來解，必須先產生字串X的所有子序列，再逐一檢查這些子序列

是否存在於字串Y中。如果字串X的長度為m，則X會產生2m個子序列。因此

，採用暴力法來解共同子序列問題的時間複雜度會是指數等級，不適用於長

的序列。


The Longest Common

Subsequence (LCS) Problem • 求LCS 問題：

– 給定兩個Sequence X=<x1, x2, …, xm>和Y=<y1, y2, …, yn>，求X和Y所構成

的最長共同子序列LCS(X, Y) = Z = < z1, z2, …, zk > 為何。

• LCS問題的遞迴式

– 令c[i, j]表示為兩個序列 X = <x1, x2, …, xi> 和 Y = <y1, y2, …, yj> 所構成之最長共同

子序列的長度，則：

ji

ji

yx0ji, if j]) 1,-c[i 1],-j Max(c[i,

yx0ji, if 11]-j 1,-c[i

0jor 0i if 0

j] c[i,

且

且


The Longest Common

Subsequence (LCS) Problem • LCS問題的遞迴式設計概念：

– 若 i 或 j 為 0，表示 X 或 Y 這兩條序列的其中一條為空序列。

– 若 xi = yj，則 c[i, j]所表示的序列長度，是由<x1, x2, …, xi-1>和<y1, y2, …, yj-

1>兩序列所構成之最長共同子序列的長度(即: c[i-1, j-1])再加上 1。

– 若 xi ≠ yj，則 c[i, j]的序列長度，是由下列兩個不同的最長共同子序列長度

當中之最大值所構成：

• <x1, x2, …, xi-1> 和 <y1, y2, …, yj> 兩序列所構成之最長共同子序列的長度c[i-1, j]

• <x1, x2, …, xi> 和 <y1, y2, …, yj-1> 兩序列所構成之最長共同子序列的長度 c[i, j-

1]


The Longest Common

Subsequence (LCS) Problem • Ex. 1 (當 xi = yj):

– X = <A, B, C, D> Y = <A, C, D>

– 共同子序列：<A>, <C>, <D>, <A, C>, <A, D>, <C, D>, <A, C, D>

– 最長共同子序列 Z = <A, C, D>

• 長度為3，此序列長度是由<A, B, C>和<A, C>所構成之最長共同子序列的長度再

加上1

• Ex. 2 (當 xi ≠ yj):

– X = <A, B, C, D> Y = <D, B, C>

– 共同子序列：<D>, <B>, <C>, <B, C>

– 最長共同子序列 Z = <B, C>

• 長度為2，序列長度是由下列兩個不同的最長共同子序列長度當中之最大值所構

成：

– <A, B, C> 和 <D, B, C> 兩序列所構成之最長共同子序列的長度: 2

– <A, B, C, D> 和 <D, B> 兩序列所構成之最長共同子序列的長度: 1


The Longest Common

Subsequence (LCS) Problem • Give two sequences X = <A, B, D, B, C> and Y = <B, A, D, C>.

Find the longest common subsequence of X and Y.

Ans:

– 最長共同子序列的長度 = 3

– 最長共同子序列：

• <A, D, C>

• <B, D, C>

Y0 Y1 Y2 Y3 Y4

B A D C

X0 0 0 0 0 0

X1 A 0 0 1 1 1

X2 B 0 1 1 1 1

X3 D 0 1 1 2 2

X4 B 0 1 1 2 2

X5 C 0 1 1 2 3


Longest Increasing Subsequence

(LIS) • 給一數字序列 X = <x1, …, xm>，找出X之最長遞增子序列。

– Ex: X = <5, 1, 4, 2, 3>，則 LIS(X) = <1, 2, 3>

• Algorithm:

Y ← Sort(X); //Y具有遞增性

Z ← LCS(X, Y); //找出X與Y之間的最長共同子序列，且具有遞增性。

Return Z


Longest Increasing Subsequence

(LIS) • X = <5, 1, 4, 2, 3>，找出 LIS(X)。

<Ans>:

Y ← Sort(X)。∴Y = <1, 2, 3, 4, 5>

Z ← LCS(X, Y);

Z = <1, 2, 3>，LIS長度為3

Return Z

Y0 Y1 Y2 Y3 Y4 Y5

1 2 3 4 5

X0 0 0 0 0 0 0

X1 5 0 0 0 0 0 1

X2 1 0 1 1 1 1 1

X3 4 0 1 1 1 2 2

X4 2 0 1 2 2 2 2

X5 3 0 1 2 3 3 3


The traveling salesperson (TSP)

problem • e.g. a directed graph :

• Cost matrix:

12

3

2

44

2

56

7

104

8

39

1 2 3 4

1 2 10 5

2 2 9

3 4 3 4

4 6 8 7


The traveling salesperson (TSP)

problem

(1) (1,3)

(1,2)

(1,4)

2

5

10

(1,2,3)

(1,2,4)

(1,3,2)

(1,3,4)

(1,4,2)

(1,4,3)

9

3

4

8

7

¡Û

(1,2,3,4)

(1,2,4,3)

(1,3,2,4)

(1,3,4,2)

(1,4,2,3)

(1,4,3,2)

¡Û

4

7

8

9

3

1

4

6

6

2

4

2


0/1 Knapsack Problem

• How to solve?

– P(i,w) = the optimal profit obtained when chosing items only from first I

items under the restriction that the total weight cannot exceed w.

– P(i,w) = max{P(i-1,w), P(i-1,w-wi)+pi}, if wi <= w

– P(I,w) = P(i-1,w), if wi>w

• Example:

– Wi = (5,10,20), Pi = (50,60,140), W = 30

chapter 6 dynamic programming -...

Documents