Divide-and-Conquer & Dynamic Programming

Divide-and-Conquer: Divide a problem to independent subproblems, find the solutions of the subproblems, and then merge the solutions of the subproblems to the solution of the original problem.

Dynamic Programming: Solve the subproblems (they may overlap with each other) and save their solutions in a table, and then use the table to solve the original problem.

Example 1: 　 Compute Fibonacci number f(0)=0, f(1 ） =1 ， f(n)=f(n-1)+f(n-2)

• Using Divide-and-Conquer:

F(n) = F(n-1) + F(n-2)

F(n-2) + F(n-3) + F(n-3) + F(n-4)

　　　　　　　　　　　　　　　　 　 F(n-3)+F(n-4) + F(n-4)+F(n-5) + F(n-4)+F(n-5) + F(n-5) + F(n-6)

　　　　　　　…………………… .

Computing time: 　 T(n) = T(n-1) + T(n-2), T(1) = 0 　 T(n) ＝ O(2　 ）

• Using Dynamic Programmin: Computing time=O(n)

n

Chapter 8 Dynamic Programming (Planning)

F(0) F(1) F(2) F(3) F(4) …… F(n)

75005)100(105)5100())A(A(A computing for tionsmultiplica ofNumber

52505)5(105)10010())AA((A computing for tionsmultiplica ofNumber

ely.resepectiv 5,5 5,100 100,10 columns and rows ofnumber with thematrices thebe A,A,Alet

example,For tions.multiplicascalar ofnumber thechanges zingparenthesi of way The

321

321

321

tions.multiplicascalar ofnumber theminimizes way that ain

product thezeparenthesifully ,demension has matrix 21for where

matrices, of ,,,,chain agiven :problemchain -Matrixtion Multiplica

321

1

321

n

ii-i

n

AAAA

ppA,...,n,i

nAAAA

Example 2 The matrix-train mutiplication problem

rqpBA rqqp for tionsmultiplica ofnumber :Note

j if ippp,j]m[km[i,k]

j,i if m[i,j]

.ppp,j]m[km[i,k]m[i,j]

AA

A

kA

Am[i,j]

jki

jki

jkki

i..j

i..j

ji

}1{Min

0 Therefore,

1then

), ofzation parenthesi optimal (the) ofzation parenthesi optimal (the

) ofzation parenthesi optimal (the

i.e., ,at zedparenthezi is ofzation parenthesi optimalan If

. compute toneeded tionsmultiplicascalar ofnumber minimum the Let

11-jki

..1..

..

Structure of an optimal parenthesization

. be of size theand Let 111.. ii-ijiiiji ppAAAAAA

Matrix Size

A1 30×35

A2 　　　　 35×15

A3 　　　　 15×5

A4 　　　　 5×10

A5 　　　　 10×20

A6 　　　　 20×25

Input

3 s[2,5]

7125

1137520103504375]5,5[]4,2[

71252053510002625]5,4[]3,2[

11300020153525000]5,3[]2,2[

min]5,2[

541

531

521

pppmm

pppmm

pppmm

m

1

3 3

3 3 3

3 3 3 5

1

2

3

4

5

i

j

1 2 3 4 5

s[i,j]

6

5

3

2

4

6

5

3

2

4

i

jm[i,j]

15125 10500 5375 3500 5000 0

11875 7125 2500 1000 0

9375 4375 750 0

7875 2625 0

15750 0

0

1 2 　 3 4 5 　 6

１

. computingin cost optimal theachived ofindex :

compute toneeded ionsmutiplicatscalar ofnumber minimum the:

Output

m[i,j]ks[i,j]

Am[i,j] i..j

Matrix-Chain-Order(p)

１　 n := length[p] -1;

2 for i 　 = 1 　 to n

3 do m[i,i] := 0;

4 for l =2 to n

5 do {for i=1 to n-l +1

6 do { j := i+ l-1;

7 m[i,j] := ;

8 for k = i to j-1

9 do {q := m[i,k]+m[k+1,j] +p p p ;

10 if q < m[i,j]

11 then {m[i,j] :=q;

12 s[i,j] := k}; }; };

13 return m, s;

Input of algorithm: 　 p , p , … , p 　　　　（ The size of A = ｐ * ｐ ）

Computing time O(n )

8

i-1 k j

0 1 n i i+1i

3

),...,,( 10 npppp

Example 3 Longest common subsequence (LCS)A problem from Bioinformatics: the DNA of one organism may be

S1 = ACCGGTCGAGTGCGCGGAAGCCGGCCGAAA, while the DNA of another organism may be S2 = GTCGTTCTTAATGCCGTTGCTCTGTAAA. One goal of comparing two strands of DNA is to determine how “similar” the two strands are, as some measure of how closely related the two organisms are.

Problem Formulization

Given a sequence X = ( ), another sequence Z = ( ) is a subsequence of X if there exists a strictly increasing sequence ( ) of indices of X such that for all j = 1, 2, …k, we have .ji zx

j

kzzz ,...,, 21

kiii ,...,, 21

mxxx ,...,, 21

mxxx ,...,, 21

Theorem

Let X = ( ) and Y = ( ) be sequence, and Z = ( ) be any LCS of X and Y.

nyyy ,...,, 21 kzzz ,...,, 21

. and X of LCSan is Z that implies then, If 3.

. and X of LCSan is Z that implies then, If 2.

. and X of LCSan is Z and then, If 1.

1

1-m

11-m1-k

nmknm

mknm

nnmknm

Yxzyx

Yxzyx

Yyxzyx

nn

mm

yyyyY

xxxxX

...

...

121

121

Find a match

nn

mm

yyyyY

xxxxX

...

...

121

121

nn

mm

yyyyY

xxxxX

...

...

121

121

Didn’t find a match

Let c[i,j] be the length of an LCS of the sequences . and ji YX

. and 0ji, if ]),1[],1,[max(

, and 0, if 1]1,1[

,0or 0 if 0

],[

ji

ji

yxjicjic

yxjijic

ji

jic

Procedure Print-LCS(c,X,i,j) 1 if i = 0 or j = 0 then return2 if c[i,j] = c[i-1,j-1] + 1 then { Print-LCS(c,X,i-1,j-1) print X[i] } else if c[i-1, j] > c[i,j-1] then Print-LCS(c,X,i-1,j) else Pring-LCS(c,X,i,j-1)

Procedure LCS-Length(X,Y) m := length[X];n := length[Y];for i := 0 to m do c[i, 0] := 0for j := 0 to n do c[0,j] := 0for i := 1 to m do for j := 1 to n do if X[i] = Y[j] then c[i,j] := c[i-1, j-1] +1 else if c[i-1,j] > c[i,j-1] then c[i,j] := c[i-1,j] else c[i,j] := c[i,j-1]return c

jjj

iii

yyyyY

xxxxX

...

...

121

121

Find a match

jjj

iii

yyyyY

xxxxX

...

...

121

121

jjj

iii

yyyyY

xxxxX

...

...

121

121

Didn’t find a match

Chapter 9 Greedy TechniqueGreedy Technique: It makes a locally optimal choice in the hope that this choice will lead to a globally optimal solution.

Example 1 Activity-selection problem

1 resource: such as classroom 　

n activities ：　 S={1,2,…n}

Activity i has a start time and a finish time 　

i.e., activity i takes place during time interval 　

The activity-selection problem is to select a maximum-size of mutually compatibal activities.

ifsi

),[ ifsi

.in activities selected previously all with compatible is if into activity Put

far. so selected activities theofset thebe Let

:repeatly following theof what do 32each For (2)

.1activity select (1)

others. of an thoseearlier th is efinish tim its ifearlier activity select the :hniqueGreedy tec

.... that Suppose 21

AiAi

A

,...,n, i

fff n

Greedy-Activity-Selector(S,f)

1 n := length[S];

2 A = {1};

3 j = 1;

4 for i = 2 to n

5 do if

6 then { A = AU{i}

7 j = i;};

8 return A;

ji fs

Example

ifsi i　 　　 　

1 1 4

2 3 5

3 0 6

4 5 7

5 3 8

6 5 9

7 6 10

8 8 11

9 8 12

10 2 13

11 12 14

Computing time = O(n)

Example 2 Huffman Code

Considering the problem of designing a binary character code wherein each character is represented by a unique binary code

Fixed length code: each character is coded with same length. For example: to code 6 characters a, b, c, d, e, f, 3 bits are needed, where a=000, b=001, c=010, d=011, e=100, f=101.

Variable-length code: 　 giving frequent characters short codewords and infrequent characters long codewords.

　　　　 　　　 a b c d e f 　　　　

Frequency (in thousands) 45 13 12 16 9 5 　

Fixed-length codeword 000 001 010 011 100 10

Variable-length codeword 0 101 100 111 1101 1100

A data file of 100,000 characters contains only the characters a – f with the frequencies indicated. 300,000 bits are needed for fixed length code. Using variable-length code only (45x1+ 13x3 ＋ 12x3+ 16x3 + 9x4 + 5x4 )x1000 = 224000 bits are needed.

thusis file a encode torequired bits ofnumber The

treein the leaf sc' ofdepth the:)(

file in the character offrequency the:)(

cd

ccf

T

Cc

T cdcfTB )()()(

Prefix codes No codeword is also a prefix of some other codeword

　　　　　　　　　　　　　　　 100

　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　 ａ :45 55

25 30

　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　 ｃ :12 b:13 14 d:16

f:5 e:9

0 1

0 1

0 1 0 1

0 1

Constructing a Huffman code

Greedy technique: the character with higher frequency has smaller depth. （ a) f:5 e:9 c:12 b:13 d:16 a:45 (b) c:12 b:13 14 d:16 a:45

f:5 e:9

(c) 14 d:16 25 a:45 (d) 25 30 a:45

f:5 e:9 c:12 b:13 c:12 b:13 14 d:16

f:5 e:9

(e) a:45 55 (f) 100

25 30 a:45 55

c:12 b:3 14 d:16 25 30

f:5 e:9 c:12 e:13 14 d:16

f:5 e:9

0 1

0 1 0 1 0 1 0 1

0 1

0 1

0 1 0 1

0 1

0 1

0 1

0 1 0 1

0 1C: the set of characters. f(c): c’s frequency

Huffman(C)

１　 n:= |C|; Q := C;

2 for i = 1 to n-1

3 do {z := New-Node();

4 x := left[z] :=Extract-Min(Q ); y := right[z] :=Extract-Min(Q)

5 f[z] := f[x] + f[y]

6 Insert(Q, z)};

7 Return Q

Computing time

• Q is a priority queue.

• Initialization of Q: O(n)

• for statement: n － 1 times of each using O(logn) time.

Totoally, O(nlogn) time.

Example 3 Single-source shortest-paths problem

For a given vertex called the source in a weighted connected graph, find shortest paths to all other vertices.

s

y

v

x

u

7

２

５

3

2

9

10

4

1

6s

y

v

x

u

7

２

５

3

2

9

10

4

1

6

Find shortest paths from source s

Input:

Graph G=(V,E)

Weight w(u,v) for each edge (u,v)

Adjacent list Adj[v] for each vertex v

Output:

p(v): parent of v on shortest path from s to v

d[v]: weight of the shortest path from s to v

V={s,u,v,x,y}, E={(s,u),(s,x),(x,y),(u,x),(x,u),(u,v),(x,v),(v,y),(y,v),(y,s)} w(s,u)=10,w(s,x)=5,w(x,y)=2,… Adj[s]={x,u}, Adj[x]={y,u,v},…

p(x)=s, p(u)=x, p(y)=x, p(v)=u

d[s]=0,d[x]=5,d[y]=7,d[u]=8,d[v]=9

0d[s] 4

NIL[v] 3

d[v] do 2

V[G]xeach verte 1

),(

for

sGSourceSingleInitialize

}u;[v] 3

v];w[u,d[u]d[v] { then 2

v)w(u,d[u]d[v] if 1

),,(Relax

p

wvu

u vsw(u,v)d[u]

d[v]

• Initilization

s

y

v

x

u

7

２

５

3

2

9

10

4

1

6

G

0

7

２

５

3

2

9

10

4

1

6

∞u v

yx

s∞

∞∞

• Relaxiation

Repeatly selecting an edge to improve the shortest paths found so far.

Supposing that edge (u,v) is selected and d[v] is the weight of the shortest path from s to v found so far, if d[v] < d[u]+w(u,v) then the shortest path from s to v is improved as follows: p(v)=u and d[v]=d[u]+w(u,v)

0

7

２

５

3

2

9

10

4

1

6

∞u v

yx

s∞

∞∞

0

7

２

５

3

2

9

10

4

1

6

8u v

yx

s14

75

0

7

２

５

3

2

9

10

4

1

6

10u v

yx

s∞

∞5

0

7

２

５

3

2

9

10

4

1

6

8u v

yx

s13

75

0

7

２

５

3

2

9

10

4

1

6

8u v

yx

s9

75

0

7

２

５

3

2

9

10

4

1

6

8u v

yx

s9

75

Red vertices: processed

Green vertices: unprocessed

Dijkstra’s Algorithm

S: Set of the processed vertices

Q: Set of the unprocessed vertices. Each vertex v of Q has a value d[v]. Q is constructed to be a priority queue.　

Adjacent list Adj[v] for each vertex v in Graph G.

Greedy-technique: repeatly select a vertex from Q who is closest to s so far. 　

}w);v,Relax(u, do 8

Adj[u] for v 7

;{u}S S 6

Min(Q);-Extract u { do 5

0 Q while4

V[G]; Q 3

0;S 2

s);Source(G,-Single-Initialize 1

),,(

swGDijkstra

S:

d:d[s]=0,d[u]=d[v]=d[x]=d[y]=∞

Q: s,u,v,x,y

Q: u,x,v,y

S: s

d: d[s]=0,d[u]=3, d[v]=∞ ， d[x]=5,d[y]=∞

Q: x,v,y

S: s,u

d: d[s]=0,d[u]=3,d[v]=9,d[x]=5,d[y]=∞

Q: v,y

S: s,u,x

d: d[s]=0,d[u]=3,d[v]=9,d[x]=5,d[y]=9

Q: y

S: s,u,x,v

d: d[s]=0,d[u]=3,d[v]=9,d[x]=5,d[y]=9

Q:

S: s,u,x,v,y

d: d[s]=0,d[u]=3,d[v]=9,d[x]=5,d[y]=9

初期化

0

7

２

５

3

2

9

10

4

1

6

∞u v

yx

s∞

∞∞

}w);v,Relax(u, do 8

Adj[u] for v 7

;{u}S S 6

Min(Q);-Extract u { do 5

0 Q while4

V[G]; Q 3

0;S 2

s);Source(G,-Single-Initialize 1

),,(

swGDijkstra

S:

d:d[s]=0,d[u]=d[v]=d[x]=d[y]=∞

Q: s,u,v,x,y

Q: u,x,v,y

S: s

d: d[s]=0,d[u]=10, d[v]=∞ ， d[x]=5,d[y]=∞

Q: u,v,y

S: s,x

d: d[s]=0,d[u]=8,d[v]=14,d[x]=5,d[y]=7

Q: u,v

S: s,x,y

d: d[s]=0,d[u]=8,d[v]=13,d[x]=5,d[y]=7

Q: v

S: s,u,x,y

d: d[s]=0,d[u]=8,d[v]=9,d[x]=5,d[y]=7

Q:

S: s,u,x,v,y

d: d[s]=0,d[u]=8,d[v]=9,d[x]=5,d[y]=7

Initiliation

0

7

２

５

3

2

9

10

4

1

6

∞u v

yx

s∞

∞∞

0

7

２

５

3

2

9

10

4

1

6

8u v

yx

s14

75

0

7

２

５

3

2

9

10

4

1

6

10u v

yx

s∞

∞5

0

7

２

５

3

2

9

10

4

1

6

8u v

yx

s13

75

0

7

２

５

3

2

9

10

4

1

6

8u v

yx

s9

75

0

7

２

５

3

2

9

10

4

1

6

8u v

yx

s9

75

}w);v,Relax(u, do 8

Adj[u] for v 7

;{u}S S 6

Min(Q);-Extract {u do 5

0 Q while4

V[G]; Q 3

;0S 2

s);Source(G,-Single-Initialize 1

),,(

swGDijkstra Q is a priority queue. One Extract-Min(Q) operation uses O(log |V|) time. One Relax(u,v,w) uses constant time and revise d[v] in Q uses O(log|V|) time. Totally, the algorithm runs in O((|V|+|E|)log|V|) = O((n+m)logn) time. 　　

Computing Time of Dijkstra’s Algorithm

G=G(V,E), where |V|=n, |E|=m

S: Set of the processed vertices

Q: Set of the unprocessed vertices. Each vertex v of Q has a value d[v]. Q is constructed to be a priority queue.　

Adjacent list Adj[v] for each vertex v in Graph G.