algorithm design and analysisudel.edu/~amotong/teaching/algorithm design...randomized algorithms...
TRANSCRIPT
Algorithm Design and Analysis
Summer 2018 Amo G. Tong 1
Lecture 4Randomized Algorithms• Randomized Quick Sort
• Randomized Selection
Summer 2018 Amo G. Tong 2
• Quick Sort:• Take an element 𝑥 = 𝐴[𝑞] in 𝐴[1,… , 𝑛].
• Partition into two subarrays 𝐴[1,… , 𝑞 − 1] and 𝐴[𝑞 + 1,… , 𝑛] such that each element of 𝐴[1,… , 𝑞 − 1] is less than or equal to 𝑥 which is less than or equal to each element of 𝐴[𝑞 + 1,… , 𝑛].
• Sort the subarrays recursively.
• No need to combine.
Randomized Algorithms
Summer 2018 Amo G. Tong 3
Input: an array 𝐴[1,… , 𝑛] of numbers
• Quick Sort:
Randomized Algorithms
Summer 2018 Amo G. Tong 4
Input: an array 𝐴[1,… , 𝑛] of numbers
PARTITION (𝐴, 𝑝, 𝑟)1 select an element 𝑥 = 𝐴[𝑘] in 𝐴[𝑝. . 𝑟]2 exchange A[k] with A[r];3 𝑖 = 𝑝 − 14 for 𝑗 = 𝑝 to 𝑟 − 15 if 𝐴 𝑗 ≤ 𝑥6 𝑖 = 𝑖 + 17 exchange 𝐴[𝑖] with 𝐴[𝑗]8 end9 end10 exchange 𝐴[𝑖 + 1] with 𝐴[𝑟]11 return 𝑖 + 1;
• Quick Sort:
Randomized Algorithms
Summer 2018 Amo G. Tong 5
Input: an array 𝐴[1,… , 𝑛] of numbers
0 9 5 2 1 𝑖 = 0, exchange 5 with 1
0 1 2 5 9
PARTITION (𝐴, 𝑝, 𝑟)1 select an element 𝑥 = 𝐴[𝑘] in 𝐴[𝑝. . 𝑟]2 exchange A[k] with A[r];3 𝑖 = 𝑝 − 14 for 𝑗 = 𝑝 to 𝑟 − 15 if 𝐴 𝑗 ≤ 𝑥6 𝑖 = 𝑖 + 17 exchange 𝐴[𝑖] with 𝐴[𝑗]8 end9 end10 exchange 𝐴[𝑖 + 1] with 𝐴[𝑟]11 return 𝑖 + 1;
• Quick Sort:
Randomized Algorithms
Summer 2018 Amo G. Tong 6
Input: an array 𝐴[1,… , 𝑛] of numbers
0 9 5 2 10 9 1 2 5
𝑖 = 0, exchange 5 with 1
𝑖 = 1, exchange 0 with 0
0 1 2 5 9
PARTITION (𝐴, 𝑝, 𝑟)1 select an element 𝑥 = 𝐴[𝑘] in 𝐴[𝑝. . 𝑟]2 exchange A[k] with A[r];3 𝑖 = 𝑝 − 14 for 𝑗 = 𝑝 to 𝑟 − 15 if 𝐴 𝑗 ≤ 𝑥6 𝑖 = 𝑖 + 17 exchange 𝐴[𝑖] with 𝐴[𝑗]8 end9 end10 exchange 𝐴[𝑖 + 1] with 𝐴[𝑟]11 return 𝑖 + 1;
• Quick Sort:
Randomized Algorithms
Summer 2018 Amo G. Tong 7
Input: an array 𝐴[1,… , 𝑛] of numbers
0 9 5 2 10 9 1 2 50 9 1 2 5
𝑖 = 0, exchange 5 with 1
𝑖 = 1, exchange 0 with 0
𝑖 = 1
0 1 2 5 9
PARTITION (𝐴, 𝑝, 𝑟)1 select an element 𝑥 = 𝐴[𝑘] in 𝐴[𝑝. . 𝑟]2 exchange A[k] with A[r];3 𝑖 = 𝑝 − 14 for 𝑗 = 𝑝 to 𝑟 − 15 if 𝐴 𝑗 ≤ 𝑥6 𝑖 = 𝑖 + 17 exchange 𝐴[𝑖] with 𝐴[𝑗]8 end9 end10 exchange 𝐴[𝑖 + 1] with 𝐴[𝑟]11 return 𝑖 + 1;
• Quick Sort:
Randomized Algorithms
Summer 2018 Amo G. Tong 8
Input: an array 𝐴[1,… , 𝑛] of numbers
0 9 5 2 10 9 1 2 50 9 1 2 50 9 1 2 5
𝑖 = 0, exchange 5 with 1
𝑖 = 1, exchange 0 with 0
𝑖 = 1
𝑖 = 2, exchange 9 with 1
0 1 2 5 9
PARTITION (𝐴, 𝑝, 𝑟)1 select an element 𝑥 = 𝐴[𝑘] in 𝐴[𝑝. . 𝑟]2 exchange A[k] with A[r];3 𝑖 = 𝑝 − 14 for 𝑗 = 𝑝 to 𝑟 − 15 if 𝐴 𝑗 ≤ 𝑥6 𝑖 = 𝑖 + 17 exchange 𝐴[𝑖] with 𝐴[𝑗]8 end9 end10 exchange 𝐴[𝑖 + 1] with 𝐴[𝑟]11 return 𝑖 + 1;
• Quick Sort:
Randomized Algorithms
Summer 2018 Amo G. Tong 9
Input: an array 𝐴[1,… , 𝑛] of numbers
0 9 5 2 10 9 1 2 50 9 1 2 50 9 1 2 50 1 9 2 5
𝑖 = 0, exchange 5 with 1
𝑖 = 1, exchange 0 with 0
𝑖 = 1
𝑖 = 2, exchange 9 with 1
𝑖 = 3, exchange 9 with 2
0 1 2 5 9
PARTITION (𝐴, 𝑝, 𝑟)1 select an element 𝑥 = 𝐴[𝑘] in 𝐴[𝑝. . 𝑟]2 exchange A[k] with A[r];3 𝑖 = 𝑝 − 14 for 𝑗 = 𝑝 to 𝑟 − 15 if 𝐴 𝑗 ≤ 𝑥6 𝑖 = 𝑖 + 17 exchange 𝐴[𝑖] with 𝐴[𝑗]8 end9 end10 exchange 𝐴[𝑖 + 1] with 𝐴[𝑟]11 return 𝑖 + 1;
• Quick Sort:
Randomized Algorithms
Summer 2018 Amo G. Tong 10
Input: an array 𝐴[1,… , 𝑛] of numbers
0 9 5 2 10 9 1 2 50 9 1 2 50 9 1 2 50 1 9 2 50 1 2 9 5
𝑖 = 0, exchange 5 with 1
𝑖 = 1, exchange 0 with 0
𝑖 = 1
𝑖 = 2, exchange 9 with 1
𝑖 = 3, exchange 9 with 2
exchange 9 with 5
0 1 2 5 9
PARTITION (𝐴, 𝑝, 𝑟)1 select an element 𝑥 = 𝐴[𝑘] in 𝐴[𝑝. . 𝑟]2 exchange A[k] with A[r];3 𝑖 = 𝑝 − 14 for 𝑗 = 𝑝 to 𝑟 − 15 if 𝐴 𝑗 ≤ 𝑥6 𝑖 = 𝑖 + 17 exchange 𝐴[𝑖] with 𝐴[𝑗]8 end9 end10 exchange 𝐴[𝑖 + 1] with 𝐴[𝑟]11 return 𝑖 + 1;
• Quick Sort:
Randomized Algorithms
Summer 2018 Amo G. Tong 11
Input: an array 𝐴[1,… , 𝑛] of numbers
0 9 5 2 10 9 1 2 50 9 1 2 50 9 1 2 50 1 9 2 50 1 2 9 50 1 2 5 9
𝑖 = 0, exchange 5 with 1
𝑖 = 1, exchange 0 with 0
𝑖 = 1
𝑖 = 2, exchange 9 with 1
𝑖 = 3, exchange 9 with 2
exchange 9 with 5
Return 4
0 1 2 5 9
PARTITION (𝐴, 𝑝, 𝑟)1 select an element 𝑥 = 𝐴[𝑘] in 𝐴[𝑝. . 𝑟]2 exchange A[k] with A[r];3 𝑖 = 𝑝 − 14 for 𝑗 = 𝑝 to 𝑟 − 15 if 𝐴 𝑗 ≤ 𝑥6 𝑖 = 𝑖 + 17 exchange 𝐴[𝑖] with 𝐴[𝑗]8 end9 end10 exchange 𝐴[𝑖 + 1] with 𝐴[𝑟]11 return 𝑖 + 1;
• Quick Sort:
Randomized Algorithms
Summer 2018 Amo G. Tong 12
Input: an array 𝐴[1,… , 𝑛] of numbers
QUICKSORT (𝐴, 𝑝, 𝑟)1 if 𝑝 < 𝑟2 𝑞 =PARTITION(A, p, r)3 QUICKSORT (𝐴, 𝑝, 𝑞 − 1);4 QUICKSORT (𝐴, 𝑞 + 1, 𝑟);5 end
PARTITION (𝐴, 𝑝, 𝑟)1 select an element 𝑥 = 𝐴[𝑘] in 𝐴[𝑝. . 𝑟]2 exchange A[k] with A[r];3 𝑖 = 𝑝 − 14 for 𝑗 = 𝑝 to 𝑟 − 15 if 𝐴 𝑗 ≤ 𝑥6 𝑖 = 𝑖 + 17 exchange 𝐴[𝑖] with 𝐴[𝑗]8 end9 end10 exchange 𝐴[𝑖 + 1] with 𝐴[𝑟]11 return 𝑖 + 1;
0 9 5 2 1
0 1 2 5 9
• Quick Sort:
Randomized Algorithms
Summer 2018 Amo G. Tong 13
Input: an array 𝐴[1,… , 𝑛] of numbers
QUICKSORT (𝐴, 𝑝, 𝑟)1 if 𝑝 < 𝑟2 𝑞 =PARTITION(A, p, r)3 QUICKSORT (𝐴, 𝑝, 𝑞 − 1);4 QUICKSORT (𝐴, 𝑞 + 1, 𝑟);5 end
PARTITION (𝐴, 𝑝, 𝑟)1 select an element 𝑥 = 𝐴[𝑘] in 𝐴[𝑝. . 𝑟]2 exchange A[k] with A[r];3 𝑖 = 𝑝 − 14 for 𝑗 = 𝑝 to 𝑟 − 15 if 𝐴 𝑗 ≤ 𝑥6 𝑖 = 𝑖 + 17 exchange 𝐴[𝑖] with 𝐴[𝑗]8 end9 end10 exchange 𝐴[𝑖 + 1] with 𝐴[𝑟]11 return 𝑖 + 1;
Which element should we take?
• Quick Sort:• Take an element 𝑥 in 𝐴[1,… , 𝑛].
• Partition into two subarrays 𝐴1[1, … , 𝑎] and 𝐴2[1, … , 𝑏] such that each element of 𝐴1 is less than or equal to x which is less than or equal to each element of 𝐴2. (𝑎 + 𝑏 = 𝑛 − 1)
• Sort the subarrays recursively.
• No need to combine.
• 𝑥 decides the size of the two subarrays.• 𝑇(𝑛) = 𝑇(𝑎) + 𝑇(𝑏) + Θ(𝑛);
Randomized Algorithms
Summer 2018 Amo G. Tong 14
• 𝑥 decides the size of the two subarrays.• 𝑇 𝑛 = 𝑇 𝑎 + 𝑇 𝑏 + Θ 𝑛 .
• If the array is always evenly partitioned:• 𝑥=median
Randomized Algorithms
Summer 2018 Amo G. Tong 15
• 𝑥 decides the size of the two subarrays.• 𝑇 𝑛 = 𝑇 𝑎 + 𝑇 𝑏 + Θ 𝑛 .
• If the array is always evenly partitioned:• 𝑥=median
• 𝑇 𝑛 = 𝑇 𝑛/2 + 𝑇 𝑛/2 + Θ 𝑛 .
• 𝑇(𝑛) = Θ(𝑛 log 𝑛)
Randomized Algorithms
Summer 2018 Amo G. Tong 16
• 𝑥 decides the size of the two subarrays.• 𝑇 𝑛 = 𝑇 𝑎 + 𝑇 𝑏 + Θ 𝑛 .
• If the array is always evenly partitioned:• 𝑥=median
• 𝑇 𝑛 = 𝑇 𝑛/2 + 𝑇 𝑛/2 + Θ 𝑛 .
• 𝑇(𝑛) = Θ(𝑛 log 𝑛)
• If the array is always partitioned with a fixed proportion 𝒂:• x= the 𝑎𝑛 -th smallest element
• 𝑇(𝑛) = 𝑇(𝑎𝑛) + 𝑇((1 − 𝑎)𝑛) + Θ(𝑛) where 0 < 𝑎 < 1;
• 𝑇(𝑛) = Θ(𝑛 log 𝑛)
Randomized Algorithms
Summer 2018 Amo G. Tong 17
• 𝑥 decides the size of the two subarrays.• 𝑇 𝑛 = 𝑇 𝑎 + 𝑇 𝑏 + Θ 𝑛 .
• If one of the subarray is always empty:• 𝑥= the maximum or the minimum
• 𝑇(𝑛) = 𝑇(𝑛 − 1) + 𝑇(0) + Θ(𝑛)
• 𝑇(𝑛) = Θ(𝑛2)
Randomized Algorithms
Summer 2018 Amo G. Tong 18
• 𝑥 decides the size of the two subarrays.• 𝑇 𝑛 = 𝑇 𝑎 + 𝑇 𝑏 + Θ 𝑛 .
• If one of the subarray is always empty:• 𝑥= the maximum or the minimum
• 𝑇(𝑛) = 𝑇(𝑛 − 1) + 𝑇(0) + Θ(𝑛)
• 𝑇(𝑛) = Θ(𝑛2)
• If one of the subarray has a constant length 𝒂:• x= the a-th smallest element
• 𝑇(𝑛) = 𝑇(𝑛 − 1 − 𝑎) + 𝑇(𝑎) + Θ(𝑛) where 0 < 𝑎 < 𝑛;
• 𝑇(𝑛) =? ? (try it yourself)
Randomized Algorithms
Summer 2018 Amo G. Tong 19
• If we always select the last element as the pivot:
Randomized Algorithms
Summer 2018 Amo G. Tong 20
• If we always select the last element as the pivot:• Worst-case running time:
• 𝑇(𝑛) = 𝑇(𝑛 − 1) + 𝑇(0) + Θ(𝑛)
• 𝑇(𝑛) = 𝑂(𝑛2)
Randomized Algorithms
Summer 2018 Amo G. Tong 21
• If we always select the last element as the pivot:• Worst-case running time:
• 𝑇(𝑛) = 𝑇(𝑛 − 1) + 𝑇(0) + Θ(𝑛)
• 𝑇(𝑛) = 𝑂(𝑛2)
• Expected running time:• Assume each of the possible permutations appears with the same
probability.
Randomized Algorithms
Summer 2018 Amo G. Tong 22
• If we always select the last element as the pivot:• Worst-case running time:
• 𝑇(𝑛) = 𝑇(𝑛 − 1) + 𝑇(0) + Θ(𝑛)
• 𝑇(𝑛) = 𝑂(𝑛2)
• Expected running time:• Assume each of the possible permutations appears with the same
probability.
• With probability 1/n, the last element is the 𝒊-th smallest.
Randomized Algorithms
Summer 2018 Amo G. Tong 23
• If we always select the last element as the pivot:• Worst-case running time:
• 𝑇(𝑛) = 𝑇(𝑛 − 1) + 𝑇(0) + Θ(𝑛)
• 𝑇(𝑛) = 𝑂(𝑛2)
• Expected running time:• Assume each of the possible permutations appears with the same
probability.
• With probability 1/n, the last element is the 𝑖-th smallest.
• If the last element is the 𝑖-th smallest, 𝑇 𝑛 = T(𝑖 − 1) + 𝑇(𝑛 − 𝑖) + 𝑐𝑛
Randomized Algorithms
Summer 2018 Amo G. Tong 24
• If we always select the last element as the pivot:• Worst-case running time:
• 𝑇(𝑛) = 𝑇(𝑛 − 1) + 𝑇(0) + Θ(𝑛)
• 𝑇(𝑛) = 𝑂(𝑛2)
• Expected running time:• Assume each of the possible permutations appears with the same
probability.
• With probability 1/n, the last element is the 𝑖-th smallest.
• If the last element is the 𝑖-th smallest, 𝑇 𝑛 = T(𝑖 − 1) + 𝑇(𝑛 − 𝑖) + 𝑐𝑛
• The expected value is
𝐸 𝑇 𝑛 = σ𝑖=1𝑛 1
𝑛(𝐸[𝑇 𝑖 − 1 ] + 𝐸[𝑇 𝑛 − 𝑖 ] + 𝑐𝑛)
Randomized Algorithms
Summer 2018 Amo G. Tong 25
Randomized Algorithms
Summer 2018 Amo G. Tong 26
𝐸 𝑇 𝑛 = σ𝑖=1𝑛 1
𝑛(𝑇(𝑖 − 1) + 𝑇(𝑛 − 𝑖) + 𝑐𝑛)⇒ 𝐸 𝑇 𝑛 = O(n log n)
Proof: Let 𝑇 𝑘 ≤ 𝑎 for 𝑘 < 2.
Randomized Algorithms
Summer 2018 Amo G. Tong 27
𝐸 𝑇 𝑛 = σ𝑖=1𝑛 1
𝑛(𝑇(𝑖 − 1) + 𝑇(𝑛 − 𝑖) + 𝑐𝑛)⇒ 𝐸 𝑇 𝑛 = O(n log n)
Proof: Let 𝑇 𝑘 ≤ 𝑎 for 𝑘 < 2.Inductive hypothesis: 𝐸 𝑇 𝑛 ≤ 𝑏 𝑛 log𝑛
Randomized Algorithms
Summer 2018 Amo G. Tong 28
𝐸 𝑇 𝑛 = σ𝑖=1𝑛 1
𝑛(𝑇(𝑖 − 1) + 𝑇(𝑛 − 𝑖) + 𝑐𝑛)⇒ 𝐸 𝑇 𝑛 = O(n log n)
Proof: Let 𝑇 𝑘 ≤ 𝑎 for 𝑘 < 2.Inductive hypothesis: 𝐸 𝑇 𝑛 ≤ 𝑏 𝑛 log𝑛
Basic: 𝐸[𝑇(2)] =1
2(𝑇(0) + 𝑇(1) + 𝑐 ⋅ 2) +
1
2(𝑇(1) + 𝑇(0) + 𝑐 ⋅ 2) = 2𝑎 + 2𝑐.
We need 2𝑎 + 2𝑐 < 2𝑏.
Randomized Algorithms
Summer 2018 Amo G. Tong 29
𝐸 𝑇 𝑛 = σ𝑖=1𝑛 1
𝑛(𝑇(𝑖 − 1) + 𝑇(𝑛 − 𝑖) + 𝑐𝑛)⇒ 𝐸 𝑇 𝑛 = O(n log n)
Proof: Let 𝑇 𝑘 ≤ 𝑎 for 𝑘 < 2.Inductive hypothesis: 𝐸 𝑇 𝑛 ≤ 𝑏 𝑛 log𝑛
Basic: 𝐸[𝑇(2)] =1
2(𝑇(0) + 𝑇(1) + 𝑐 ⋅ 2) +
1
2(𝑇(1) + 𝑇(0) + 𝑐 ⋅ 2) = 2𝑎 + 2𝑐.
We need 2𝑎 + 2𝑐 < 2𝑏.Induction: Suppose 𝐸 𝑇 𝑛 ≤ 𝑏 𝑛 log 𝑛 for all 2 < 𝑛 < 𝑘.
Randomized Algorithms
Summer 2018 Amo G. Tong 30
𝐸 𝑇 𝑛 = σ𝑖=1𝑛 1
𝑛(𝑇(𝑖 − 1) + 𝑇(𝑛 − 𝑖) + 𝑐𝑛)⇒ 𝐸 𝑇 𝑛 = O(n log n)
Proof: Let 𝑇 𝑘 ≤ 𝑎 for 𝑘 < 2.Inductive hypothesis: 𝐸 𝑇 𝑛 ≤ 𝑏 𝑛 log𝑛
Basic: 𝐸[𝑇(2)] =1
2(𝑇(0) + 𝑇(1) + 𝑐 ⋅ 2) +
1
2(𝑇(1) + 𝑇(0) + 𝑐 ⋅ 2) = 2𝑎 + 2𝑐.
We need 2𝑎 + 2𝑐 < 2𝑏.Induction: Suppose 𝐸 𝑇 𝑛 ≤ 𝑏 𝑛 log 𝑛 for all 2 < 𝑛 < 𝑘.
𝑬[𝑻(𝒌)] = σ𝒊=𝟏𝒌 𝟏
𝒌(𝑬[𝑻 𝒊 − 𝟏 ] + 𝑬[𝑻 𝒌 − 𝒊 ] + 𝒄𝒌)
Randomized Algorithms
Summer 2018 Amo G. Tong 31
𝐸 𝑇 𝑛 = σ𝑖=1𝑛 1
𝑛(𝑇(𝑖 − 1) + 𝑇(𝑛 − 𝑖) + 𝑐𝑛)⇒ 𝐸 𝑇 𝑛 = O(n log n)
Proof: Let 𝑇 𝑘 ≤ 𝑎 for 𝑘 < 2.Inductive hypothesis: 𝐸 𝑇 𝑛 ≤ 𝑏 𝑛 log𝑛
Basic: 𝐸[𝑇(2)] =1
2(𝑇(0) + 𝑇(1) + 𝑐 ⋅ 2) +
1
2(𝑇(1) + 𝑇(0) + 𝑐 ⋅ 2) = 2𝑎 + 2𝑐.
We need 2𝑎 + 2𝑐 < 2𝑏.Induction: Suppose 𝐸 𝑇 𝑛 ≤ 𝑏 𝑛 log 𝑛 for all 2 < 𝑛 < 𝑘.
𝐸[𝑇(𝑘)] = σ𝑖=1𝑘 1
𝑘(𝐸[𝑇 𝑖 − 1 ] + 𝐸[𝑇 𝑘 − 𝑖 ] + 𝑐𝑘)
=𝟏
𝒌(𝟐σ𝒊=𝟎
𝒌−𝟏𝑬[𝑻 𝒊 ] + 𝒄𝒌𝟐)
Randomized Algorithms
Summer 2018 Amo G. Tong 32
𝐸 𝑇 𝑛 = σ𝑖=1𝑛 1
𝑛(𝑇(𝑖 − 1) + 𝑇(𝑛 − 𝑖) + 𝑐𝑛)⇒ 𝐸 𝑇 𝑛 = O(n log n)
Proof: Let 𝑇 𝑘 ≤ 𝑎 for 𝑘 < 2.Inductive hypothesis: 𝐸 𝑇 𝑛 ≤ 𝑏 𝑛 log𝑛
Basic: 𝐸[𝑇(2)] =1
2(𝑇(0) + 𝑇(1) + 𝑐 ⋅ 2) +
1
2(𝑇(1) + 𝑇(0) + 𝑐 ⋅ 2) = 2𝑎 + 2𝑐.
We need 2𝑎 + 2𝑐 < 2𝑏.Induction: Suppose 𝐸 𝑇 𝑛 ≤ 𝑏 𝑛 log 𝑛 for all 2 < 𝑛 < 𝑘.
𝐸[𝑇(𝑘)] = σ𝑖=1𝑘 1
𝑘(𝐸[𝑇 𝑖 − 1 ] + 𝐸[𝑇 𝑘 − 𝑖 ] + 𝑐𝑘)
=1
𝑘(2σ𝑖=0
𝑘−1𝐸[𝑇 𝑖 ] + 𝑐𝑘2)
=𝟏
𝒌(𝟐σ𝒊=𝟐
𝒌−𝟏𝑬[𝑻 𝒊 ] + 𝟐𝑻(𝟎) + 𝟐𝑻(𝟏) + 𝒄𝒌𝟐)
Randomized Algorithms
Summer 2018 Amo G. Tong 33
𝐸 𝑇 𝑛 = σ𝑖=1𝑛 1
𝑛(𝑇(𝑖 − 1) + 𝑇(𝑛 − 𝑖) + 𝑐𝑛)⇒ 𝐸 𝑇 𝑛 = O(n log n)
Proof: Let 𝑇 𝑘 ≤ 𝑎 for 𝑘 < 2.Inductive hypothesis: 𝐸 𝑇 𝑛 ≤ 𝑏 𝑛 log𝑛
Basic: 𝐸[𝑇(2)] =1
2(𝑇(0) + 𝑇(1) + 𝑐 ⋅ 2) +
1
2(𝑇(1) + 𝑇(0) + 𝑐 ⋅ 2) = 2𝑎 + 2𝑐.
We need 2𝑎 + 2𝑐 < 2𝑏.Induction: Suppose 𝐸 𝑇 𝑛 ≤ 𝑏 𝑛 log 𝑛 for all 2 < 𝑛 < 𝑘.
𝐸[𝑇(𝑘)] = σ𝑖=1𝑘 1
𝑘(𝐸[𝑇 𝑖 − 1 ] + 𝐸[𝑇 𝑘 − 𝑖 ] + 𝑐𝑘)
=1
𝑘(2σ𝑖=0
𝑘−1𝐸[𝑇 𝑖 ] + 𝑐𝑘2)
=1
𝑘(2σ𝑖=2
𝑘−1𝐸[𝑇 𝑖 ] + 2𝑇(0) + 2𝑇(1) + 𝑐𝑘2)
≤𝟏
𝒌(𝟐σ𝒊=𝟐
𝒌−𝟏𝒃 𝒊 𝐥𝐨𝐠 𝒊) + (𝟒𝒂 + 𝒄𝒌𝟐)/𝒌
Randomized Algorithms
Summer 2018 Amo G. Tong 34
𝐸 𝑇 𝑛 = σ𝑖=1𝑛 1
𝑛(𝑇(𝑖 − 1) + 𝑇(𝑛 − 𝑖) + 𝑐𝑛)⇒ 𝐸 𝑇 𝑛 = O(n log n)
Proof: Let 𝑇 𝑘 ≤ 𝑎 for 𝑘 < 2.Inductive hypothesis: 𝐸 𝑇 𝑛 ≤ 𝑏 𝑛 log𝑛
Basic: 𝐸[𝑇(2)] =1
2(𝑇(0) + 𝑇(1) + 𝑐 ⋅ 2) +
1
2(𝑇(1) + 𝑇(0) + 𝑐 ⋅ 2) = 2𝑎 + 2𝑐.
We need 2𝑎 + 2𝑐 < 2𝑏.Induction: Suppose 𝐸 𝑇 𝑛 ≤ 𝑏 𝑛 log 𝑛 for all 2 < 𝑛 < 𝑘.
𝐸[𝑇(𝑘)] = σ𝑖=1𝑘 1
𝑘(𝐸[𝑇 𝑖 − 1 ] + 𝐸[𝑇 𝑘 − 𝑖 ] + 𝑐𝑘)
=1
𝑘(2σ𝑖=0
𝑘−1𝐸[𝑇 𝑖 ] + 𝑐𝑘2)
=1
𝑘(2σ𝑖=2
𝑘−1𝐸[𝑇 𝑖 ] + 2𝑇(0) + 2𝑇(1) + 𝑐𝑘2)
≤2𝑏
𝑘(σ𝑖=2
𝑘−1 𝑖 log 𝑖) + (4𝑎 + 𝑐𝑘2)/𝑘
(see additional reading material)
≤2b
𝑘(
𝑘2
2log 𝑘 −
𝑘2
4+
1
4) + (4𝑎 + 𝑐𝑘2)/𝑘
Randomized Algorithms
Summer 2018 Amo G. Tong 35
We need this to be ≤ 𝑏 𝑘 log 𝑘.
𝐸 𝑇 𝑛 = σ𝑖=1𝑛 1
𝑛(𝑇(𝑖 − 1) + 𝑇(𝑛 − 𝑖) + 𝑐𝑛)⇒ 𝐸 𝑇 𝑛 = O(n log n)
Proof: Let 𝑇 𝑘 ≤ 𝑎 for 𝑘 < 2.Inductive hypothesis: 𝐸 𝑇 𝑛 ≤ 𝑏 𝑛 log𝑛
Basic: 𝐸[𝑇(2)] =1
2(𝑇(0) + 𝑇(1) + 𝑐 ⋅ 2) +
1
2(𝑇(1) + 𝑇(0) + 𝑐 ⋅ 2) = 2𝑎 + 2𝑐.
We need 2𝑎 + 2𝑐 < 2𝑏.Induction: Suppose 𝐸 𝑇 𝑛 ≤ 𝑏 𝑛 log 𝑛 for all 2 < 𝑛 < 𝑘.
𝐸[𝑇(𝑘)] = σ𝑖=1𝑘 1
𝑘(𝐸[𝑇 𝑖 − 1 ] + 𝐸[𝑇 𝑘 − 𝑖 ] + 𝑐𝑘)
=1
𝑘(2σ𝑖=0
𝑘−1𝐸[𝑇 𝑖 ] + 𝑐𝑘2)
=1
𝑘(2σ𝑖=2
𝑘−1𝐸[𝑇 𝑖 ] + 2𝑇(0) + 2𝑇(1) + 𝑐𝑘2)
≤2𝑏
𝑘(σ𝑖=2
𝑘−1 𝑖 log 𝑖) + (4𝑎 + 𝑐𝑘2)/𝑘
(see additional reading material)
≤2b
𝑘(
𝑘2
2log 𝑘 −
𝑘2
4+
1
4) + (4𝑎 + 𝑐𝑘2)/𝑘
• If we always select the last element as the pivot:• Worst-case running time:
• 𝑇(𝑛) = 𝑇(𝑛 − 1) + 𝑇(0) + Θ(𝑛)
• 𝑇(𝑛) = 𝑂(𝑛2)
• Expected running time:• Assume each of the possible permutations appears with the same
probability.
• The expected running time is
𝐸 𝑇 𝑛 = Θ(𝑛 log 𝑛)
Randomized Algorithms
Summer 2018 Amo G. Tong 36
• If we always select the last element as the pivot:• Worst-case running time:
• 𝑇(𝑛) = 𝑇(𝑛 − 1) + 𝑇(0) + Θ(𝑛)
• 𝑇(𝑛) = 𝑂(𝑛2)
• Expected running time:• Assume each of the possible permutations appears with the same
probability.
• The expected running time is
𝐸 𝑇 𝑛 = Θ(𝑛 log 𝑛)
Randomized Algorithms
Summer 2018 Amo G. Tong 37
This is not always true.We want a Θ(𝑛 log 𝑛)algorithm which does not rely on this assumption
• Randomized Quick Sort;• Select the pivot uniformly at random from the given array.
Randomized Algorithms
Summer 2018 Amo G. Tong 38
• Randomized Quick Sort;• Select the pivot uniformly at random from the given array.
• With probability 1/𝑛, 𝐴[𝑖] is selected as the pivot.
• With probability 1/𝑛, the 𝑖-th smallest element is selected.
Randomized Algorithms
Summer 2018 Amo G. Tong 39
• Randomized Quick Sort;• Select the pivot uniformly at random from the given array.
• With probability 1/𝑛, 𝐴[𝑖] is selected as the pivot.
• With probability 1/𝑛, the 𝑖-th smallest element is selected.
Randomized Algorithms
Summer 2018 Amo G. Tong 40
RANDOM-PARTITION (𝐴, 𝑝, 𝑟)1 randomly select an element x = 𝐴[𝑘] in 𝐴[𝑝. . 𝑟],
where each element has the same probability to be selected.2 exchange A[k] with A[r];3 𝑖 = 𝑝 − 14 for 𝑗 = 𝑝 to 𝑟 − 15 if 𝐴 𝑗 ≤ 𝑥6 𝑖 = 𝑖 + 17 exchange 𝐴[𝑖] with 𝐴[𝑗]8 end9 end10 exchange 𝐴[𝑖 + 1] with 𝐴[𝑟]11 return 𝑖 + 1;
• Randomized Quick Sort;• Select the pivot uniformly at random from the given array.
• With probability 1/𝑛, 𝐴[𝑖] is selected as the pivot.
• With probability 1/𝑛, the 𝑖-th smallest element is selected.
Randomized Algorithms
Summer 2018 Amo G. Tong 41
QUICKSORT (𝐴, 𝑝, 𝑟)1 if 𝑝 < 𝑟2 𝑞 =PARTITION(A, p, r)3 QUICKSORT (𝐴, 𝑝, 𝑞 − 1);4 QUICKSORT (𝐴, 𝑞 + 1, 𝑟);5 end
RANDOM-QUICKSORT (𝐴, 𝑝, 𝑟)1 if 𝑝 < 𝑟2 𝑞 =RANDOM-PARTITION(A, p, r)3 RANDOM-QUICKSORT (𝐴, 𝑝, 𝑞 − 1);4 RANDOM-QUICKSORT (𝐴, 𝑞 + 1, 𝑟);5 end
• Randomized Quick Sort;• Select the pivot uniformly at random from the given array.
• With probability 1/𝑛, 𝐴[𝑖] is selected as the pivot.
• With probability 1/𝑛, the 𝑖-th smallest element is selected.
• Expected running time:• No assumption on the input.
• The expected running time is
𝐸 𝑇 𝑛 = σ𝑖=1𝑛 1
𝑛(𝐸[𝑇 𝑖 − 1 ] + 𝐸[𝑇 𝑛 − 𝑖 ] + 𝑐𝑛)
Randomized Algorithms
Summer 2018 Amo G. Tong 42
The same as that in page 24.
• Selection:
Randomized Algorithms
Summer 2018 Amo G. Tong 43
Input: an array 𝐴[1,… , 𝑛] of distinct numbers and an integer 𝑖, 1 ≤ 𝑖 ≤ 𝑛.
Output: the 𝑖-th smallest element in 𝐴[1,… , 𝑛] .
• Selection:
Randomized Algorithms
Summer 2018 Amo G. Tong 44
Pseudocode:RANDOMIZED-SELECT(𝐴, 𝑝, 𝑟, 𝑖)1 if 𝑝 == 𝑟2 return A[p]3 end4 q=RANDOM-PARTITION(𝐴, 𝑝, 𝑟)5 𝑘 = 𝑝 − 𝑞 + 1;6 if 𝑖 == 𝑘7 return 𝐴[𝑞]8 end9 if 𝑖 < 𝑘10 return RANDOM-SELECT(𝐴, 𝑝, 𝑞 − 1, 𝑖)11 else12 return RANDOM-SELECT(𝐴, 𝑞 + 1, 𝑟, 𝑖 − 𝑘)13 end
• Selection:
Randomized Algorithms
Summer 2018 Amo G. Tong 45
Pseudocode:RANDOMIZED-SELECT(𝐴, 𝑝, 𝑟, 𝑖)1 if 𝑝 == 𝑟2 return A[p]3 end4 𝑞 =RANDOM-PARTITION(𝐴, 𝑝, 𝑟)5 𝑘 = 𝑝 − 𝑞 + 1;6 if 𝑖 == 𝑘7 return 𝐴[𝑞]8 end9 if 𝑖 < 𝑘10 return RANDOMIZED-SELECT(𝐴, 𝑝, 𝑞 − 1, 𝑖)11 else12 return RANDOMIZED-SELECT(𝐴, 𝑞 + 1, 𝑟, 𝑖 − 𝑘)13 end
Θ(𝑛)
𝑇(max(𝑞 − 𝑝, 𝑟 − 𝑞))
T(𝑛)
• Randomized selection;• Select the pivot uniformly at random from the given array.
• With probability1/𝑛, 𝐴[𝑖] is selected as the pivot.
• With probability1/𝑛, the 𝑖-th smallest element is selected.
• Running time:• If the 𝑖-th smallest element is selected as the pivot, the running time is
𝑇 𝑛 = 𝑇 max 𝑖 − 1, 𝑛 − 𝑖 + Θ(𝑛)
• So the expected running time is:
𝐸 𝑇 𝑛 =
𝑖=0
𝑛−11
𝑛(𝐸[𝑇 max 𝑖 − 1, 𝑛 − 𝑖 ] + Θ(𝑛) )
Randomized Algorithms
Summer 2018 Amo G. Tong 46
Randomized Algorithms
Summer 2018 Amo G. Tong 47
𝐸 𝑇 𝑛
=
𝑖=1
𝑛1
𝑛(𝐸[𝑇 max 𝑖 − 1, 𝑛 − 𝑖 ] + Θ(𝑛) )
=
𝑖=1
𝑛/2 −11
𝑛(𝐸[𝑇 max 𝑖 − 1, 𝑛 − 𝑖 ] + Θ(𝑛) ) +
𝑖= 𝑛/2 −1
𝑛1
𝑛(𝐸[𝑇 max 𝑖 − 1, 𝑛 − 𝑖 ] + Θ(𝑛) )
=
𝑖=1
𝑛/2 −11
𝑛(𝐸[𝑇 𝑛 − 𝑖 ] + Θ(𝑛) ) +
𝑖= 𝑛/2
𝑛1
𝑛(𝐸[𝑇 𝑖 − 1 ] + Θ(𝑛) )
=
𝑖= 𝑛/2
𝑛2
𝑛(𝐸[𝑇 𝑖 − 1 ] + Θ(𝑛) )
𝐸 𝑇 𝑛 =
𝑖= 𝑛/2
𝑛2
𝑛(𝐸[𝑇 𝑖 − 1 ] + Θ(𝑛) )
Prove 𝐸[𝑇(𝑛)] = 𝑂(𝑛)
• Expected running time:• The algorithm is not randomized. The expectation is taken over the
distribution of the inputs.
Randomized Algorithms
Summer 2018 Amo G. Tong 48
• Expected running time:• The algorithm is not randomized. The expectation is taken over the
distribution of the inputs.
• The algorithm is randomized. The expected running time is 𝑶(𝒇(𝒏)) for any given input.
Randomized Algorithms
Summer 2018 Amo G. Tong 49
• Expected running time:• The algorithm is not randomized. The expectation is taken over the
distribution of the inputs.
• The algorithm is randomized. The expected running time is 𝑂(𝑓(𝑛)) for any given input.
• The algorithm is randomized and the input is also sampled.
Randomized Algorithms
Summer 2018 Amo G. Tong 50
• Expected running time:• The algorithm is not randomized. The expectation is taken over the
distribution of the inputs.
• The algorithm is randomized. The expected running time is 𝑂(𝑓(𝑛)) for any given input.
• The algorithm is randomized and the input is also sampled.• Consider the Randomized Quick Sort with an input uniformly sampled
from {all the permutations over {1, … , 𝑛}}.
• What is the running time? (exercise)
Randomized Algorithms
Summer 2018 Amo G. Tong 51