cs 253: algorithms chapter 4 divide-and-conquer recurrences master theorem credit: dr. george bebis
Post on 21-Dec-2015
245 views
TRANSCRIPT
CS 253: Algorithms
Chapter 4
Divide-and-Conquer
Recurrences
Master Theorem
Credit: Dr. George Bebis
Recurrences and Running Time
Recurrences arise when an algorithm contains recursive calls
to itself
Running time is represented by an equation or inequality that
describes a function in terms of its value on smaller inputs.
T(n) = T(n-1) + n
What is the actual running time of the algorithm? i.e. T(n)
= ?
Need to solve the recurrence
◦ Find an explicit formula of the expression
◦ Bound the recurrence by an expression that involves n
Example Recurrences
T(n) = T(n-1) + n Θ(n2)Recursive algorithm that loops through the input to eliminate one item
T(n) = T(n/2) + c Θ(lgn)Recursive algorithm that halves the input in one step
T(n) = T(n/2) + n Θ(n)Recursive algorithm that halves the input but must examine every item in the input
T(n) = 2T(n/2) + 1 Θ(n)Recursive algorithm that splits the input into 2 halves and does a constant amount of other work
BINARY-SEARCH
Finds if x is in the sorted array A[lo…hi]
Alg.: BINARY-SEARCH (A, lo, hi, x)
if (lo > hi)return FALSE
mid (lo+hi)/2if x = A[mid]
return TRUEif ( x < A[mid] )
BINARY-SEARCH (A, lo, mid-1, x)if ( x > A[mid] )
BINARY-SEARCH (A, mid+1, hi, x)
12
11
10
97532
1 2 3 4 5 6 7 8
midlo hi
Example 1
A[8] = {1, 2, 3, 4, 5, 7, 9, 11} lo = 1 hi = 8 x = 7
mid = 4, lo = 5, hi = 8
mid = 6, A[mid] = x Found!
119754321
119754321
1 2 3 4 5 6 7 8
8765
A[8] = {1, 2, 3, 4, 5, 7, 9, 11}lo = 1 hi = 8 x = 6
lo = 1, hi = 8, mid = 4. A[4]=4
lo = 5, hi = 8, mid = 6, A[6]=7
11
9754321
11
9754321
1 2 3 4 5 6 7 8
11
9754321 lo = 5, hi = 5, mid = 5, A[5]=5
11
9754321
low high
low
lowhigh
high
lo = 6, hi = 5 NOT FOUND!
Example 1I
Analysis of BINARY-SEARCH
Alg.: BINARY-SEARCH (A, lo, hi, x)if (lo > hi)
return FALSEmid (lo+hi)/2if x = A[mid]
return TRUEif ( x < A[mid] )
BINARY-SEARCH (A, lo, mid-1, x)if ( x > A[mid] )
BINARY-SEARCH (A, mid+1, hi, x)
T(n) = c + T(n/2)
constant time: c2
same problem of size n/2
same problem of size n/2
constant time: c1
constant time: c3
8
Methods for Solving Recurrences
Iteration method
Recursion-tree method
Master method
The Iteration Method
Convert the recurrence into a summation and solve it using a
known series
Example: T(n) = c + T(n/2)
T(n) = c + T(n/2)
= c + c + T(n/4)
= c + c + c + T(n/8)
= c + c + c + c + T(n/24)
Assume n=2k then k = lg n and
T(n) = c + c + c + c + c + … + T(n/2k)
(k times)
T(n) = k * c + T(1)
T(n) = clg n
Iteration Method – Example 2
T(n) = n + 2T(n/2) Assume n=2k k = lg
n
T(n) = n + 2T(n/2)
= n + 2(n/2 + 2T(n/4))
= n + n + 4T(n/4)
= n + n + 4(n/4 + 2T(n/8))
= n + n + n + 8T(n/8)
T(n) = 3n + 23T(n/23)
= kn + 2kT(n/2k)
= nlgn + nT(1)
T(n) = O(nlgn)
11
Methods for Solving Recurrences
Iteration method
Recursion-tree method
Master method
The recursion-tree method
Convert the recurrence into a tree:
◦ Each node represents the cost incurred at various
levels of recursion
◦ Sum up the costs of all levels
Used to “guess” a solution for the recurrence
Example 1 W(n) = 2W(n/2) + n2
Subproblem size at level i = n/2i
At level i: Cost of each node = (n/2i)2 # of nodes = 2i Total cost = (n2/2i)
h = Height of the tree n/2h=1 h = lgn Total cost at all levels:
W(n) = O(n2)
22
0
2lg
0
2lg
0
2
2
21
1
1
2
1
2
1
2)( nnnn
nnW
i
in
i
in
ii
Example 2 T(n) = 3T(n/4) + cn2
Subproblem size at level i = n/4i
At level i: Cost of each node= c(n/4i)2 # of nodes= 3i Total cost =cn2(3/16)i
h = Height of the tree n/4h=1 h = log4n
Total cost at all levels: (last level has 3log4n = nlog
43 nodes)
T(n) = O(n2)
)(
163
1
1
16
3
16
3)( 23log23log2
0
3log21log
0
444
4
nOncnncnncnnTi
iin
i
15
Example 3
• The longest path from the root to a
leaf:
n (2/3)n (2/3)2 n … 1
(2/3)in =1 i = log3/2n
• Cost of the problem at level i = n
• Total cost:
via further analysis W(n) =Θ(nlgn)
W(n) = W(n/3) + W(2n/3) + n
3/2
lg( ) (log ) ( lg )
lg(3 / 2)
nW n n n n O n n
16
Methods for Solving Recurrences
Iteration method
Recursion-tree method
Master method
Master Theorem
“Cookbook” for solving recurrences of the form:
Idea: compare f(n) with nlogba
f(n) is asymptotically smaller or larger than
nlogba by a polynomial factor n
OR
f(n) is asymptotically equal with nlogba
0 and 1, b 1, a where )()(
f(n)nfb
naTnT
18
Master Theorem
Case 1: if f(n) = O(nlogba -) for some > 0, then: T(n) =
(nlogba)
Case 2: if f(n) = (nlogba), then: T(n) = (nlogba lgn)
Case 3: if f(n) = (nlogba +) for some > 0, and if
af(n/b) ≤ cf(n) for some c < 1 and all sufficiently large n,
then:
T(n) = (f(n))
regularity condition
0 and 1, b 1, a where )()(
f(n)nfb
naTnT
19
Example 1
T(n) = 2T(n/2) + n
a = 2, b = 2, log22 = 1
Compare nlogba=n1 with f(n) = n
f(n) = (nlogba=n1) Case 2
T(n) = (nlogba lgn) = (nlgn)
Case 1: if f(n) = O(nlogba -) for some > 0, then: T(n) = (nlogba)
Case 2: if f(n) = (nlogba), then: T(n) = (nlogba lgn)
Case 3: if f(n) = (nlogba +) for some > 0, and if
af(n/b) ≤ cf(n) for some c < 1 and all sufficiently large n, then: T(n)
=(f(n))
T(n) = 2T(n/2) + n2
a = 2, b = 2, log22 = 1
Compare nlog22=n1 with f(n) = n2
f(n) = (nlog22+) Case 3 (* need to verify regularity
cond.)
a f(n/b) ≤ c f(n) 2 n2/4 ≤ c n2 (½ ≤ c <1)
T(n) = (f(n)) = (n2)
Example 2Case 1: if f(n) = O(nlogba -) for some > 0, then: T(n) = (nlogba)
Case 2: if f(n) = (nlogba), then: T(n) = (nlogba lgn)
Case 3: if f(n) = (nlogba +) for some > 0, and if
af(n/b) ≤ cf(n) for some c < 1 and all sufficiently large n, then: T(n)
=(f(n))
21
T(n) = 2T(n/2) + √n
a = 2, b = 2, log22 = 1
Compare n with f(n) = n1/2
f(n) = O(n1-) Case 1
T(n) = (nlogba) = (n)
Example 3Case 1: if f(n) = O(nlogba -) for some > 0, then: T(n) = (nlogba)
Case 2: if f(n) = (nlogba), then: T(n) = (nlogba lgn)
Case 3: if f(n) = (nlogba +) for some > 0, and if
af(n/b) ≤ cf(n) for some c < 1 and all sufficiently large n, then: T(n)
=(f(n))
T(n) = 3T(n/4) + nlgn
a = 3, b = 4, log43 = 0.793
Compare n0.793 with f(n) = nlgn
f(n) = (nlog43+) Case 3
Check regularity condition:
3(n/4)lg(n/4) ≤ (3/4)nlgn = c f(n), (3/4 ≤ c < 1)
T(n) = (nlgn)
Example 4Case 1: if f(n) = O(nlogba -) for some > 0, then: T(n) = (nlogba)
Case 2: if f(n) = (nlogba), then: T(n) = (nlogba lgn)
Case 3: if f(n) = (nlogba +) for some > 0, and if
af(n/b) ≤ cf(n) for some c < 1 and all sufficiently large n, then: T(n)
=(f(n))
T(n) = 2T(n/2) + nlgn
a = 2, b = 2, log22 = 1
Compare n with f(n) = nlgn
Is this a candidate for Case 3 ?
Case 3: if f(n) = (nlogba +) for some > 0
In other words, If nlgn ≥ n1.n for some > 0
lgn is asymptotically less than n for any > 0 !!
Therefore, this in not Case 3! (somewhere between Case 2 & 3)
Example 5Case 1: if f(n) = O(nlogba -) for some > 0, then: T(n) = (nlogba)
Case 2: if f(n) = (nlogba), then: T(n) = (nlogba lgn)
Case 3: if f(n) = (nlogba +) for some > 0, and if
af(n/b) ≤ cf(n) for some c < 1 and all sufficiently large n, then: T(n)
=(f(n))
T(n) = 2T(n/2) + nlgn
Use Iteration Method to show that
T(n) = nlg2n
T(n) = nlgn + 2T(n/2)
= nlgn + 2(n/2*lg(n/2) + 2T(n/4))
= nlgn + nlg(n/2) + 22T(n/22)
= ….
Exercise:
25
Changing variables
Try to transform the recurrence to one that you have seen before
◦Rename: m = lgn n = 2m
T (2m) = 2T(2m/2) + m
◦Rename: k=m and S(k) = T(2k)
S(k) = 2S(k/2) + k S(k) = (k*lgk)
T(n) = T(2m) = S(m) = (mlgm)= (lgn*lglgn)
**See Page 86 of the Textbook.
T(n) = 2T( ) + lgn n
26
Use Iteration Method to show that
T(n) = (lgn*lglgn)
T(n) = lgn + 2T(n1/2)
= lgn + 2(lg(n1/2) + 2T(n1/4))
= lgn + lgn + 22T(n1/4)
= lgn + lgn + … + 2kT(2) k=lglgn
T(n) = 2T( ) + lgn n
Exercise:
Appendix A
Summations
2
)1(321
1
nnnk
n
k
6
)12)(1(321
1
22222
nnnnk
n
k
4
)1(321
22
1
33333
nnnk
n
k
xxx
x
xxxxx
n
k
k
n
nn
k
nk
1
1lim then 1|| If
1
11
0
1
0
2
)1(ln1
3
1
2
11
1 Series Harmonic
1
Onnk
Hn
kn