Divide-and-Conquer & Dynamic Programming
Divide-and-Conquer: Divide a problem to independent subproblems, find the solutions of the subproblems, and then merge the solutions of the subproblems to the solution of the original problem.
Dynamic Programming: Solve the subproblems (they may overlap with each other) and save their solutions in a table, and then use the table to solve the original problem.
Example 1: Compute Fibonacci number f(0)=0, f(1 ) =1 , f(n)=f(n-1)+f(n-2)
• Using Divide-and-Conquer:
F(n) = F(n-1) + F(n-2)
F(n-2) + F(n-3) + F(n-3) + F(n-4)
F(n-3)+F(n-4) + F(n-4)+F(n-5) + F(n-4)+F(n-5) + F(n-5) + F(n-6)
…………………… .
Computing time: T(n) = T(n-1) + T(n-2), T(1) = 0 T(n) = O(2 )
• Using Dynamic Programmin: Computing time=O(n)
n
Chapter 8 Dynamic Programming (Planning)
F(0) F(1) F(2) F(3) F(4) …… F(n)
75005)100(105)5100())A(A(A computing for tionsmultiplica ofNumber
52505)5(105)10010())AA((A computing for tionsmultiplica ofNumber
ely.resepectiv 5,5 5,100 100,10 columns and rows ofnumber with thematrices thebe A,A,Alet
example,For tions.multiplicascalar ofnumber thechanges zingparenthesi of way The
321
321
321
tions.multiplicascalar ofnumber theminimizes way that ain
product thezeparenthesifully ,demension has matrix 21for where
matrices, of ,,,,chain agiven :problemchain -Matrixtion Multiplica
321
1
321
n
ii-i
n
AAAA
ppA,...,n,i
nAAAA
Example 2 The matrix-train mutiplication problem
rqpBA rqqp for tionsmultiplica ofnumber :Note
j if ippp,j]m[km[i,k]
j,i if m[i,j]
.ppp,j]m[km[i,k]m[i,j]
AA
A
kA
Am[i,j]
jki
jki
jkki
i..j
i..j
ji
}1{Min
0 Therefore,
1then
), ofzation parenthesi optimal (the) ofzation parenthesi optimal (the
) ofzation parenthesi optimal (the
i.e., ,at zedparenthezi is ofzation parenthesi optimalan If
. compute toneeded tionsmultiplicascalar ofnumber minimum the Let
11-jki
..1..
..
Structure of an optimal parenthesization
. be of size theand Let 111.. ii-ijiiiji ppAAAAAA
Matrix Size
A1 30×35
A2 35×15
A3 15×5
A4 5×10
A5 10×20
A6 20×25
Input
3 s[2,5]
7125
1137520103504375]5,5[]4,2[
71252053510002625]5,4[]3,2[
11300020153525000]5,3[]2,2[
min]5,2[
541
531
521
pppmm
pppmm
pppmm
m
1
3 3
3 3 3
3 3 3 5
1
2
3
4
5
i
j
1 2 3 4 5
s[i,j]
6
5
3
2
4
6
5
3
2
4
i
jm[i,j]
15125 10500 5375 3500 5000 0
11875 7125 2500 1000 0
9375 4375 750 0
7875 2625 0
15750 0
0
1 2 3 4 5 6
1
. computingin cost optimal theachived ofindex :
compute toneeded ionsmutiplicatscalar ofnumber minimum the:
Output
m[i,j]ks[i,j]
Am[i,j] i..j
Matrix-Chain-Order(p)
1 n := length[p] -1;
2 for i = 1 to n
3 do m[i,i] := 0;
4 for l =2 to n
5 do {for i=1 to n-l +1
6 do { j := i+ l-1;
7 m[i,j] := ;
8 for k = i to j-1
9 do {q := m[i,k]+m[k+1,j] +p p p ;
10 if q < m[i,j]
11 then {m[i,j] :=q;
12 s[i,j] := k}; }; };
13 return m, s;
Input of algorithm: p , p , … , p ( The size of A = p * p )
Computing time O(n )
8
i-1 k j
0 1 n i i+1i
3
),...,,( 10 npppp
Example 3 Longest common subsequence (LCS)A problem from Bioinformatics: the DNA of one organism may be
S1 = ACCGGTCGAGTGCGCGGAAGCCGGCCGAAA, while the DNA of another organism may be S2 = GTCGTTCTTAATGCCGTTGCTCTGTAAA. One goal of comparing two strands of DNA is to determine how “similar” the two strands are, as some measure of how closely related the two organisms are.
Problem Formulization
Given a sequence X = ( ), another sequence Z = ( ) is a subsequence of X if there exists a strictly increasing sequence ( ) of indices of X such that for all j = 1, 2, …k, we have .ji zx
j
kzzz ,...,, 21
kiii ,...,, 21
mxxx ,...,, 21
mxxx ,...,, 21
Theorem
Let X = ( ) and Y = ( ) be sequence, and Z = ( ) be any LCS of X and Y.
nyyy ,...,, 21 kzzz ,...,, 21
. and X of LCSan is Z that implies then, If 3.
. and X of LCSan is Z that implies then, If 2.
. and X of LCSan is Z and then, If 1.
1
1-m
11-m1-k
nmknm
mknm
nnmknm
Yxzyx
Yxzyx
Yyxzyx
nn
mm
yyyyY
xxxxX
...
...
121
121
Find a match
nn
mm
yyyyY
xxxxX
...
...
121
121
nn
mm
yyyyY
xxxxX
...
...
121
121
Didn’t find a match
Let c[i,j] be the length of an LCS of the sequences . and ji YX
. and 0ji, if ]),1[],1,[max(
, and 0, if 1]1,1[
,0or 0 if 0
],[
ji
ji
yxjicjic
yxjijic
ji
jic
Procedure Print-LCS(c,X,i,j) 1 if i = 0 or j = 0 then return2 if c[i,j] = c[i-1,j-1] + 1 then { Print-LCS(c,X,i-1,j-1) print X[i] } else if c[i-1, j] > c[i,j-1] then Print-LCS(c,X,i-1,j) else Pring-LCS(c,X,i,j-1)
Procedure LCS-Length(X,Y) m := length[X];n := length[Y];for i := 0 to m do c[i, 0] := 0for j := 0 to n do c[0,j] := 0for i := 1 to m do for j := 1 to n do if X[i] = Y[j] then c[i,j] := c[i-1, j-1] +1 else if c[i-1,j] > c[i,j-1] then c[i,j] := c[i-1,j] else c[i,j] := c[i,j-1]return c
jjj
iii
yyyyY
xxxxX
...
...
121
121
Find a match
jjj
iii
yyyyY
xxxxX
...
...
121
121
jjj
iii
yyyyY
xxxxX
...
...
121
121
Didn’t find a match
Chapter 9 Greedy TechniqueGreedy Technique: It makes a locally optimal choice in the hope that this choice will lead to a globally optimal solution.
Example 1 Activity-selection problem
1 resource: such as classroom
n activities : S={1,2,…n}
Activity i has a start time and a finish time
i.e., activity i takes place during time interval
The activity-selection problem is to select a maximum-size of mutually compatibal activities.
ifsi
),[ ifsi
.in activities selected previously all with compatible is if into activity Put
far. so selected activities theofset thebe Let
:repeatly following theof what do 32each For (2)
.1activity select (1)
others. of an thoseearlier th is efinish tim its ifearlier activity select the :hniqueGreedy tec
.... that Suppose 21
AiAi
A
,...,n, i
fff n
Greedy-Activity-Selector(S,f)
1 n := length[S];
2 A = {1};
3 j = 1;
4 for i = 2 to n
5 do if
6 then { A = AU{i}
7 j = i;};
8 return A;
ji fs
Example
ifsi i
1 1 4
2 3 5
3 0 6
4 5 7
5 3 8
6 5 9
7 6 10
8 8 11
9 8 12
10 2 13
11 12 14
Computing time = O(n)
Example 2 Huffman Code
Considering the problem of designing a binary character code wherein each character is represented by a unique binary code
Fixed length code: each character is coded with same length. For example: to code 6 characters a, b, c, d, e, f, 3 bits are needed, where a=000, b=001, c=010, d=011, e=100, f=101.
Variable-length code: giving frequent characters short codewords and infrequent characters long codewords.
a b c d e f
Frequency (in thousands) 45 13 12 16 9 5
Fixed-length codeword 000 001 010 011 100 10
Variable-length codeword 0 101 100 111 1101 1100
A data file of 100,000 characters contains only the characters a – f with the frequencies indicated. 300,000 bits are needed for fixed length code. Using variable-length code only (45x1+ 13x3 + 12x3+ 16x3 + 9x4 + 5x4 )x1000 = 224000 bits are needed.
thusis file a encode torequired bits ofnumber The
treein the leaf sc' ofdepth the:)(
file in the character offrequency the:)(
cd
ccf
T
Cc
T cdcfTB )()()(
Prefix codes No codeword is also a prefix of some other codeword
100
a :45 55
25 30
c :12 b:13 14 d:16
f:5 e:9
0 1
0 1
0 1 0 1
0 1
Constructing a Huffman code
Greedy technique: the character with higher frequency has smaller depth. ( a) f:5 e:9 c:12 b:13 d:16 a:45 (b) c:12 b:13 14 d:16 a:45
f:5 e:9
(c) 14 d:16 25 a:45 (d) 25 30 a:45
f:5 e:9 c:12 b:13 c:12 b:13 14 d:16
f:5 e:9
(e) a:45 55 (f) 100
25 30 a:45 55
c:12 b:3 14 d:16 25 30
f:5 e:9 c:12 e:13 14 d:16
f:5 e:9
0 1
0 1 0 1 0 1 0 1
0 1
0 1
0 1 0 1
0 1
0 1
0 1
0 1 0 1
0 1C: the set of characters. f(c): c’s frequency
Huffman(C)
1 n:= |C|; Q := C;
2 for i = 1 to n-1
3 do {z := New-Node();
4 x := left[z] :=Extract-Min(Q ); y := right[z] :=Extract-Min(Q)
5 f[z] := f[x] + f[y]
6 Insert(Q, z)};
7 Return Q
Computing time
• Q is a priority queue.
• Initialization of Q: O(n)
• for statement: n - 1 times of each using O(logn) time.
Totoally, O(nlogn) time.
Example 3 Single-source shortest-paths problem
For a given vertex called the source in a weighted connected graph, find shortest paths to all other vertices.
s
y
v
x
u
7
2
5
3
2
9
10
4
1
6s
y
v
x
u
7
2
5
3
2
9
10
4
1
6
Find shortest paths from source s
Input:
Graph G=(V,E)
Weight w(u,v) for each edge (u,v)
Adjacent list Adj[v] for each vertex v
Output:
p(v): parent of v on shortest path from s to v
d[v]: weight of the shortest path from s to v
V={s,u,v,x,y}, E={(s,u),(s,x),(x,y),(u,x),(x,u),(u,v),(x,v),(v,y),(y,v),(y,s)} w(s,u)=10,w(s,x)=5,w(x,y)=2,… Adj[s]={x,u}, Adj[x]={y,u,v},…
p(x)=s, p(u)=x, p(y)=x, p(v)=u
d[s]=0,d[x]=5,d[y]=7,d[u]=8,d[v]=9
0d[s] 4
NIL[v] 3
d[v] do 2
V[G]xeach verte 1
),(
for
sGSourceSingleInitialize
}u;[v] 3
v];w[u,d[u]d[v] { then 2
v)w(u,d[u]d[v] if 1
),,(Relax
p
wvu
u vsw(u,v)d[u]
d[v]
• Initilization
s
y
v
x
u
7
2
5
3
2
9
10
4
1
6
G
0
7
2
5
3
2
9
10
4
1
6
∞u v
yx
s∞
∞∞
• Relaxiation
Repeatly selecting an edge to improve the shortest paths found so far.
Supposing that edge (u,v) is selected and d[v] is the weight of the shortest path from s to v found so far, if d[v] < d[u]+w(u,v) then the shortest path from s to v is improved as follows: p(v)=u and d[v]=d[u]+w(u,v)
0
7
2
5
3
2
9
10
4
1
6
∞u v
yx
s∞
∞∞
0
7
2
5
3
2
9
10
4
1
6
8u v
yx
s14
75
0
7
2
5
3
2
9
10
4
1
6
10u v
yx
s∞
∞5
0
7
2
5
3
2
9
10
4
1
6
8u v
yx
s13
75
0
7
2
5
3
2
9
10
4
1
6
8u v
yx
s9
75
0
7
2
5
3
2
9
10
4
1
6
8u v
yx
s9
75
Red vertices: processed
Green vertices: unprocessed
Dijkstra’s Algorithm
S: Set of the processed vertices
Q: Set of the unprocessed vertices. Each vertex v of Q has a value d[v]. Q is constructed to be a priority queue.
Adjacent list Adj[v] for each vertex v in Graph G.
Greedy-technique: repeatly select a vertex from Q who is closest to s so far.
}w);v,Relax(u, do 8
Adj[u] for v 7
;{u}S S 6
Min(Q);-Extract u { do 5
0 Q while4
V[G]; Q 3
0;S 2
s);Source(G,-Single-Initialize 1
),,(
swGDijkstra
S:
d:d[s]=0,d[u]=d[v]=d[x]=d[y]=∞
Q: s,u,v,x,y
Q: u,x,v,y
S: s
d: d[s]=0,d[u]=3, d[v]=∞ , d[x]=5,d[y]=∞
Q: x,v,y
S: s,u
d: d[s]=0,d[u]=3,d[v]=9,d[x]=5,d[y]=∞
Q: v,y
S: s,u,x
d: d[s]=0,d[u]=3,d[v]=9,d[x]=5,d[y]=9
Q: y
S: s,u,x,v
d: d[s]=0,d[u]=3,d[v]=9,d[x]=5,d[y]=9
Q:
S: s,u,x,v,y
d: d[s]=0,d[u]=3,d[v]=9,d[x]=5,d[y]=9
初期化
0
7
2
5
3
2
9
10
4
1
6
∞u v
yx
s∞
∞∞
}w);v,Relax(u, do 8
Adj[u] for v 7
;{u}S S 6
Min(Q);-Extract u { do 5
0 Q while4
V[G]; Q 3
0;S 2
s);Source(G,-Single-Initialize 1
),,(
swGDijkstra
S:
d:d[s]=0,d[u]=d[v]=d[x]=d[y]=∞
Q: s,u,v,x,y
Q: u,x,v,y
S: s
d: d[s]=0,d[u]=10, d[v]=∞ , d[x]=5,d[y]=∞
Q: u,v,y
S: s,x
d: d[s]=0,d[u]=8,d[v]=14,d[x]=5,d[y]=7
Q: u,v
S: s,x,y
d: d[s]=0,d[u]=8,d[v]=13,d[x]=5,d[y]=7
Q: v
S: s,u,x,y
d: d[s]=0,d[u]=8,d[v]=9,d[x]=5,d[y]=7
Q:
S: s,u,x,v,y
d: d[s]=0,d[u]=8,d[v]=9,d[x]=5,d[y]=7
Initiliation
0
7
2
5
3
2
9
10
4
1
6
∞u v
yx
s∞
∞∞
0
7
2
5
3
2
9
10
4
1
6
8u v
yx
s14
75
0
7
2
5
3
2
9
10
4
1
6
10u v
yx
s∞
∞5
0
7
2
5
3
2
9
10
4
1
6
8u v
yx
s13
75
0
7
2
5
3
2
9
10
4
1
6
8u v
yx
s9
75
0
7
2
5
3
2
9
10
4
1
6
8u v
yx
s9
75
}w);v,Relax(u, do 8
Adj[u] for v 7
;{u}S S 6
Min(Q);-Extract {u do 5
0 Q while4
V[G]; Q 3
;0S 2
s);Source(G,-Single-Initialize 1
),,(
swGDijkstra Q is a priority queue. One Extract-Min(Q) operation uses O(log |V|) time. One Relax(u,v,w) uses constant time and revise d[v] in Q uses O(log|V|) time. Totally, the algorithm runs in O((|V|+|E|)log|V|) = O((n+m)logn) time.
Computing Time of Dijkstra’s Algorithm
G=G(V,E), where |V|=n, |E|=m
S: Set of the processed vertices
Q: Set of the unprocessed vertices. Each vertex v of Q has a value d[v]. Q is constructed to be a priority queue.
Adjacent list Adj[v] for each vertex v in Graph G.