lecture 3: probabilistic analysis, randomized quicksort
TRANSCRIPT
![Page 1: Lecture 3: Probabilistic Analysis, Randomized Quicksort](https://reader031.vdocuments.net/reader031/viewer/2022012422/617690aa98e1aa7d9a5e5923/html5/thumbnails/1.jpg)
Lecture 3: Probabilistic Analysis, Randomized Quicksort
Michael Dinitz
September 7, 2021601.433/633 Introduction to Algorithms
Michael Dinitz Lecture 3: Probability, Randomized Quicksort September 7, 2021 1 / 16
![Page 2: Lecture 3: Probabilistic Analysis, Randomized Quicksort](https://reader031.vdocuments.net/reader031/viewer/2022012422/617690aa98e1aa7d9a5e5923/html5/thumbnails/2.jpg)
Introduction: Sorting
� Sorting: given array of comparable elements, put them in sorted order
� Popular topic to cover in Algorithms courses� This course:
� I assume you know the basics (mergesort, quicksort, insertion sort, selection sort, bubble sort,etc.) from Data Structures
� Today: more advanced sorting (randomized quicksort)� Next week: Sorting lower bound and ways around it.
Michael Dinitz Lecture 3: Probability, Randomized Quicksort September 7, 2021 2 / 16
![Page 3: Lecture 3: Probabilistic Analysis, Randomized Quicksort](https://reader031.vdocuments.net/reader031/viewer/2022012422/617690aa98e1aa7d9a5e5923/html5/thumbnails/3.jpg)
Randomized Algorithms and Probabilistic Analysis
First lecture: “Average-case” problematic.
� What is the “average case”?
� Want to design algorithms that work in all applications.
Instead of assuming random distribution over inputs (average-case analysis, machine learning),add randomization inside algorithm!
� Still assume worst-case inputs, give bound on worst-case expected running time.
Many Fall semesters: 601.434/634 Randomized and Big Data Algorithms. Great class!
Today: adding randomness into quicksort.
Michael Dinitz Lecture 3: Probability, Randomized Quicksort September 7, 2021 3 / 16
![Page 4: Lecture 3: Probabilistic Analysis, Randomized Quicksort](https://reader031.vdocuments.net/reader031/viewer/2022012422/617690aa98e1aa7d9a5e5923/html5/thumbnails/4.jpg)
Randomized Algorithms and Probabilistic Analysis
First lecture: “Average-case” problematic.
� What is the “average case”?
� Want to design algorithms that work in all applications.
Instead of assuming random distribution over inputs (average-case analysis, machine learning),add randomization inside algorithm!
� Still assume worst-case inputs, give bound on worst-case expected running time.
Many Fall semesters: 601.434/634 Randomized and Big Data Algorithms. Great class!
Today: adding randomness into quicksort.
Michael Dinitz Lecture 3: Probability, Randomized Quicksort September 7, 2021 3 / 16
![Page 5: Lecture 3: Probabilistic Analysis, Randomized Quicksort](https://reader031.vdocuments.net/reader031/viewer/2022012422/617690aa98e1aa7d9a5e5923/html5/thumbnails/5.jpg)
Randomized Algorithms and Probabilistic Analysis
First lecture: “Average-case” problematic.
� What is the “average case”?
� Want to design algorithms that work in all applications.
Instead of assuming random distribution over inputs (average-case analysis, machine learning),add randomization inside algorithm!
� Still assume worst-case inputs, give bound on worst-case expected running time.
Many Fall semesters: 601.434/634 Randomized and Big Data Algorithms. Great class!
Today: adding randomness into quicksort.
Michael Dinitz Lecture 3: Probability, Randomized Quicksort September 7, 2021 3 / 16
![Page 6: Lecture 3: Probabilistic Analysis, Randomized Quicksort](https://reader031.vdocuments.net/reader031/viewer/2022012422/617690aa98e1aa7d9a5e5923/html5/thumbnails/6.jpg)
Randomized Algorithms and Probabilistic Analysis
First lecture: “Average-case” problematic.
� What is the “average case”?
� Want to design algorithms that work in all applications.
Instead of assuming random distribution over inputs (average-case analysis, machine learning),add randomization inside algorithm!
� Still assume worst-case inputs, give bound on worst-case expected running time.
Many Fall semesters: 601.434/634 Randomized and Big Data Algorithms. Great class!
Today: adding randomness into quicksort.
Michael Dinitz Lecture 3: Probability, Randomized Quicksort September 7, 2021 3 / 16
![Page 7: Lecture 3: Probabilistic Analysis, Randomized Quicksort](https://reader031.vdocuments.net/reader031/viewer/2022012422/617690aa98e1aa7d9a5e5923/html5/thumbnails/7.jpg)
Quicksort Basics (Review)
Input: array A of length n.
Algorithm:
1. If n = 0 or 1, return A (already sorted)
2. Pick some element p as the pivot
3. Compare every element of A to p. Let L be the elements less than p, let G be theelements larger than p. Create array [L,p,G]
4. Recursively sort L and G.
Not fully specified: how to choose p?
� Traditionally: some simple deterministic choice (first element, last element, etc.)
� Next lecture: better deterministic choice (not very practical)
� Now: first element
Michael Dinitz Lecture 3: Probability, Randomized Quicksort September 7, 2021 4 / 16
Dd A
to
![Page 8: Lecture 3: Probabilistic Analysis, Randomized Quicksort](https://reader031.vdocuments.net/reader031/viewer/2022012422/617690aa98e1aa7d9a5e5923/html5/thumbnails/8.jpg)
Quicksort Basics (Review)
Input: array A of length n.
Algorithm:
1. If n = 0 or 1, return A (already sorted)
2. Pick some element p as the pivot
3. Compare every element of A to p. Let L be the elements less than p, let G be theelements larger than p. Create array [L,p,G]
4. Recursively sort L and G.
Not fully specified: how to choose p?
� Traditionally: some simple deterministic choice (first element, last element, etc.)
� Next lecture: better deterministic choice (not very practical)
� Now: first element
Michael Dinitz Lecture 3: Probability, Randomized Quicksort September 7, 2021 4 / 16
![Page 9: Lecture 3: Probabilistic Analysis, Randomized Quicksort](https://reader031.vdocuments.net/reader031/viewer/2022012422/617690aa98e1aa7d9a5e5923/html5/thumbnails/9.jpg)
Quicksort Analysis
Upper bound:If p picked as pivot in step 2, then in correct place after step 3
�⇒ step 2 and 3 executed at most n times.
Step 3 takes time O(n) (compare every element to pivot)�⇒ total time at most O(n2)Lower Bound:Suppose A already sorted.�⇒ p = A[0] is smallest element �⇒ L = � and G = A[1..n − 1]�⇒ in one call to quicksort, do ⌦(n) work to compare everything to p, then recurse on arrayof size n − 1�⇒ running time is T(n) = T(n − 1) + cn �⇒ T(n) =⇥(n2)
Michael Dinitz Lecture 3: Probability, Randomized Quicksort September 7, 2021 5 / 16
![Page 10: Lecture 3: Probabilistic Analysis, Randomized Quicksort](https://reader031.vdocuments.net/reader031/viewer/2022012422/617690aa98e1aa7d9a5e5923/html5/thumbnails/10.jpg)
Quicksort Analysis
Upper bound:If p picked as pivot in step 2, then in correct place after step 3�⇒ step 2 and 3 executed at most n times.
Step 3 takes time O(n) (compare every element to pivot)�⇒ total time at most O(n2)Lower Bound:Suppose A already sorted.�⇒ p = A[0] is smallest element �⇒ L = � and G = A[1..n − 1]�⇒ in one call to quicksort, do ⌦(n) work to compare everything to p, then recurse on arrayof size n − 1�⇒ running time is T(n) = T(n − 1) + cn �⇒ T(n) =⇥(n2)
Michael Dinitz Lecture 3: Probability, Randomized Quicksort September 7, 2021 5 / 16
![Page 11: Lecture 3: Probabilistic Analysis, Randomized Quicksort](https://reader031.vdocuments.net/reader031/viewer/2022012422/617690aa98e1aa7d9a5e5923/html5/thumbnails/11.jpg)
Quicksort Analysis
Upper bound:If p picked as pivot in step 2, then in correct place after step 3�⇒ step 2 and 3 executed at most n times.
Step 3 takes time O(n) (compare every element to pivot)
�⇒ total time at most O(n2)Lower Bound:Suppose A already sorted.�⇒ p = A[0] is smallest element �⇒ L = � and G = A[1..n − 1]�⇒ in one call to quicksort, do ⌦(n) work to compare everything to p, then recurse on arrayof size n − 1�⇒ running time is T(n) = T(n − 1) + cn �⇒ T(n) =⇥(n2)
Michael Dinitz Lecture 3: Probability, Randomized Quicksort September 7, 2021 5 / 16
![Page 12: Lecture 3: Probabilistic Analysis, Randomized Quicksort](https://reader031.vdocuments.net/reader031/viewer/2022012422/617690aa98e1aa7d9a5e5923/html5/thumbnails/12.jpg)
Quicksort Analysis
Upper bound:If p picked as pivot in step 2, then in correct place after step 3�⇒ step 2 and 3 executed at most n times.
Step 3 takes time O(n) (compare every element to pivot)�⇒ total time at most O(n2)
Lower Bound:Suppose A already sorted.�⇒ p = A[0] is smallest element �⇒ L = � and G = A[1..n − 1]�⇒ in one call to quicksort, do ⌦(n) work to compare everything to p, then recurse on arrayof size n − 1�⇒ running time is T(n) = T(n − 1) + cn �⇒ T(n) =⇥(n2)
Michael Dinitz Lecture 3: Probability, Randomized Quicksort September 7, 2021 5 / 16
![Page 13: Lecture 3: Probabilistic Analysis, Randomized Quicksort](https://reader031.vdocuments.net/reader031/viewer/2022012422/617690aa98e1aa7d9a5e5923/html5/thumbnails/13.jpg)
Quicksort Analysis
Upper bound:If p picked as pivot in step 2, then in correct place after step 3�⇒ step 2 and 3 executed at most n times.
Step 3 takes time O(n) (compare every element to pivot)�⇒ total time at most O(n2)Lower Bound:Suppose A already sorted.
�⇒ p = A[0] is smallest element �⇒ L = � and G = A[1..n − 1]�⇒ in one call to quicksort, do ⌦(n) work to compare everything to p, then recurse on arrayof size n − 1�⇒ running time is T(n) = T(n − 1) + cn �⇒ T(n) =⇥(n2)
Michael Dinitz Lecture 3: Probability, Randomized Quicksort September 7, 2021 5 / 16
![Page 14: Lecture 3: Probabilistic Analysis, Randomized Quicksort](https://reader031.vdocuments.net/reader031/viewer/2022012422/617690aa98e1aa7d9a5e5923/html5/thumbnails/14.jpg)
Quicksort Analysis
Upper bound:If p picked as pivot in step 2, then in correct place after step 3�⇒ step 2 and 3 executed at most n times.
Step 3 takes time O(n) (compare every element to pivot)�⇒ total time at most O(n2)Lower Bound:Suppose A already sorted.�⇒ p = A[0] is smallest element
�⇒ L = � and G = A[1..n − 1]�⇒ in one call to quicksort, do ⌦(n) work to compare everything to p, then recurse on arrayof size n − 1�⇒ running time is T(n) = T(n − 1) + cn �⇒ T(n) =⇥(n2)
Michael Dinitz Lecture 3: Probability, Randomized Quicksort September 7, 2021 5 / 16
![Page 15: Lecture 3: Probabilistic Analysis, Randomized Quicksort](https://reader031.vdocuments.net/reader031/viewer/2022012422/617690aa98e1aa7d9a5e5923/html5/thumbnails/15.jpg)
Quicksort Analysis
Upper bound:If p picked as pivot in step 2, then in correct place after step 3�⇒ step 2 and 3 executed at most n times.
Step 3 takes time O(n) (compare every element to pivot)�⇒ total time at most O(n2)Lower Bound:Suppose A already sorted.�⇒ p = A[0] is smallest element �⇒ L = � and G = A[1..n − 1]
�⇒ in one call to quicksort, do ⌦(n) work to compare everything to p, then recurse on arrayof size n − 1�⇒ running time is T(n) = T(n − 1) + cn �⇒ T(n) =⇥(n2)
Michael Dinitz Lecture 3: Probability, Randomized Quicksort September 7, 2021 5 / 16
![Page 16: Lecture 3: Probabilistic Analysis, Randomized Quicksort](https://reader031.vdocuments.net/reader031/viewer/2022012422/617690aa98e1aa7d9a5e5923/html5/thumbnails/16.jpg)
Quicksort Analysis
Upper bound:If p picked as pivot in step 2, then in correct place after step 3�⇒ step 2 and 3 executed at most n times.
Step 3 takes time O(n) (compare every element to pivot)�⇒ total time at most O(n2)Lower Bound:Suppose A already sorted.�⇒ p = A[0] is smallest element �⇒ L = � and G = A[1..n − 1]�⇒ in one call to quicksort, do ⌦(n) work to compare everything to p, then recurse on arrayof size n − 1
�⇒ running time is T(n) = T(n − 1) + cn �⇒ T(n) =⇥(n2)
Michael Dinitz Lecture 3: Probability, Randomized Quicksort September 7, 2021 5 / 16
![Page 17: Lecture 3: Probabilistic Analysis, Randomized Quicksort](https://reader031.vdocuments.net/reader031/viewer/2022012422/617690aa98e1aa7d9a5e5923/html5/thumbnails/17.jpg)
Quicksort Analysis
Upper bound:If p picked as pivot in step 2, then in correct place after step 3�⇒ step 2 and 3 executed at most n times.
Step 3 takes time O(n) (compare every element to pivot)�⇒ total time at most O(n2)Lower Bound:Suppose A already sorted.�⇒ p = A[0] is smallest element �⇒ L = � and G = A[1..n − 1]�⇒ in one call to quicksort, do ⌦(n) work to compare everything to p, then recurse on arrayof size n − 1�⇒ running time is T(n) = T(n − 1) + cn
�⇒ T(n) =⇥(n2)
Michael Dinitz Lecture 3: Probability, Randomized Quicksort September 7, 2021 5 / 16
![Page 18: Lecture 3: Probabilistic Analysis, Randomized Quicksort](https://reader031.vdocuments.net/reader031/viewer/2022012422/617690aa98e1aa7d9a5e5923/html5/thumbnails/18.jpg)
Quicksort Analysis
Upper bound:If p picked as pivot in step 2, then in correct place after step 3�⇒ step 2 and 3 executed at most n times.
Step 3 takes time O(n) (compare every element to pivot)�⇒ total time at most O(n2)Lower Bound:Suppose A already sorted.�⇒ p = A[0] is smallest element �⇒ L = � and G = A[1..n − 1]�⇒ in one call to quicksort, do ⌦(n) work to compare everything to p, then recurse on arrayof size n − 1�⇒ running time is T(n) = T(n − 1) + cn �⇒ T(n) =⇥(n2)
Michael Dinitz Lecture 3: Probability, Randomized Quicksort September 7, 2021 5 / 16
![Page 19: Lecture 3: Probabilistic Analysis, Randomized Quicksort](https://reader031.vdocuments.net/reader031/viewer/2022012422/617690aa98e1aa7d9a5e5923/html5/thumbnails/19.jpg)
Randomized Quicksort
Randomized Quicksort: pick p uniformly at random from A.
Today: prove that expected running time at most O(n log n) for every input A.
� Better than an average-case bound: holds for every single input!
� Maybe in one application inputs tend to be pretty well-sorted: original deterministicquicksort bad, this still good!
� Today only expectation. Can be more clever to get high probability bounds.
Before doing analysis, quick review of basic probability theory.
Michael Dinitz Lecture 3: Probability, Randomized Quicksort September 7, 2021 6 / 16
![Page 20: Lecture 3: Probabilistic Analysis, Randomized Quicksort](https://reader031.vdocuments.net/reader031/viewer/2022012422/617690aa98e1aa7d9a5e5923/html5/thumbnails/20.jpg)
Randomized Quicksort
Randomized Quicksort: pick p uniformly at random from A.
Today: prove that expected running time at most O(n log n) for every input A.
� Better than an average-case bound: holds for every single input!
� Maybe in one application inputs tend to be pretty well-sorted: original deterministicquicksort bad, this still good!
� Today only expectation. Can be more clever to get high probability bounds.
Before doing analysis, quick review of basic probability theory.
Michael Dinitz Lecture 3: Probability, Randomized Quicksort September 7, 2021 6 / 16
![Page 21: Lecture 3: Probabilistic Analysis, Randomized Quicksort](https://reader031.vdocuments.net/reader031/viewer/2022012422/617690aa98e1aa7d9a5e5923/html5/thumbnails/21.jpg)
Randomized Quicksort
Randomized Quicksort: pick p uniformly at random from A.
Today: prove that expected running time at most O(n log n) for every input A.
� Better than an average-case bound: holds for every single input!
� Maybe in one application inputs tend to be pretty well-sorted: original deterministicquicksort bad, this still good!
� Today only expectation. Can be more clever to get high probability bounds.
Before doing analysis, quick review of basic probability theory.
Michael Dinitz Lecture 3: Probability, Randomized Quicksort September 7, 2021 6 / 16
![Page 22: Lecture 3: Probabilistic Analysis, Randomized Quicksort](https://reader031.vdocuments.net/reader031/viewer/2022012422/617690aa98e1aa7d9a5e5923/html5/thumbnails/22.jpg)
Randomized Quicksort
Randomized Quicksort: pick p uniformly at random from A.
Today: prove that expected running time at most O(n log n) for every input A.
� Better than an average-case bound: holds for every single input!
� Maybe in one application inputs tend to be pretty well-sorted: original deterministicquicksort bad, this still good!
� Today only expectation. Can be more clever to get high probability bounds.
Before doing analysis, quick review of basic probability theory.
Michael Dinitz Lecture 3: Probability, Randomized Quicksort September 7, 2021 6 / 16
![Page 23: Lecture 3: Probabilistic Analysis, Randomized Quicksort](https://reader031.vdocuments.net/reader031/viewer/2022012422/617690aa98e1aa7d9a5e5923/html5/thumbnails/23.jpg)
Probability Basics I
Only semi-formal here. Look at CLRS Chapter 5, take Introduction to Probability
⌦: Sample space. Set of all possible outcomes.
� Roll two dice. ⌦ = {1,2, . . . ,6} × {1,2, . . . ,6}. Not {2,3, . . . ,12}Event: subset of ⌦
� “Event that first die is 3”: {(3,x) ∶ x ∈ {1,2, . . . ,6}}� “Event that dice add up to 7 or 11”: {(x,y) ∈ ⌦ ∶ (x + y = 7) or (x + y = 11)}
Random Variable: X ∶ ⌦→ R� X1: value of first die. X1(x,y) = x� X2: value of second die. X2(x,y) = y� X = X1 +X2: sum of the dice. X(x,y) = x + y = X1(x,y) +X2(x,y)
Random variables super important! Running time of randomized quicksort is a randomvariable.
Michael Dinitz Lecture 3: Probability, Randomized Quicksort September 7, 2021 7 / 16
![Page 24: Lecture 3: Probabilistic Analysis, Randomized Quicksort](https://reader031.vdocuments.net/reader031/viewer/2022012422/617690aa98e1aa7d9a5e5923/html5/thumbnails/24.jpg)
Probability Basics I
Only semi-formal here. Look at CLRS Chapter 5, take Introduction to Probability
⌦: Sample space. Set of all possible outcomes.
� Roll two dice. ⌦ = {1,2, . . . ,6} × {1,2, . . . ,6}. Not {2,3, . . . ,12}Event: subset of ⌦
� “Event that first die is 3”: {(3,x) ∶ x ∈ {1,2, . . . ,6}}� “Event that dice add up to 7 or 11”: {(x,y) ∈ ⌦ ∶ (x + y = 7) or (x + y = 11)}
Random Variable: X ∶ ⌦→ R� X1: value of first die. X1(x,y) = x� X2: value of second die. X2(x,y) = y� X = X1 +X2: sum of the dice. X(x,y) = x + y = X1(x,y) +X2(x,y)
Random variables super important! Running time of randomized quicksort is a randomvariable.
Michael Dinitz Lecture 3: Probability, Randomized Quicksort September 7, 2021 7 / 16
![Page 25: Lecture 3: Probabilistic Analysis, Randomized Quicksort](https://reader031.vdocuments.net/reader031/viewer/2022012422/617690aa98e1aa7d9a5e5923/html5/thumbnails/25.jpg)
Probability Basics I
Only semi-formal here. Look at CLRS Chapter 5, take Introduction to Probability
⌦: Sample space. Set of all possible outcomes.
� Roll two dice. ⌦ =
{1,2, . . . ,6} × {1,2, . . . ,6}. Not {2,3, . . . ,12}Event: subset of ⌦
� “Event that first die is 3”: {(3,x) ∶ x ∈ {1,2, . . . ,6}}� “Event that dice add up to 7 or 11”: {(x,y) ∈ ⌦ ∶ (x + y = 7) or (x + y = 11)}
Random Variable: X ∶ ⌦→ R� X1: value of first die. X1(x,y) = x� X2: value of second die. X2(x,y) = y� X = X1 +X2: sum of the dice. X(x,y) = x + y = X1(x,y) +X2(x,y)
Random variables super important! Running time of randomized quicksort is a randomvariable.
Michael Dinitz Lecture 3: Probability, Randomized Quicksort September 7, 2021 7 / 16
![Page 26: Lecture 3: Probabilistic Analysis, Randomized Quicksort](https://reader031.vdocuments.net/reader031/viewer/2022012422/617690aa98e1aa7d9a5e5923/html5/thumbnails/26.jpg)
Probability Basics I
Only semi-formal here. Look at CLRS Chapter 5, take Introduction to Probability
⌦: Sample space. Set of all possible outcomes.
� Roll two dice. ⌦ = {1,2, . . . ,6} × {1,2, . . . ,6}.
Not {2,3, . . . ,12}Event: subset of ⌦
� “Event that first die is 3”: {(3,x) ∶ x ∈ {1,2, . . . ,6}}� “Event that dice add up to 7 or 11”: {(x,y) ∈ ⌦ ∶ (x + y = 7) or (x + y = 11)}
Random Variable: X ∶ ⌦→ R� X1: value of first die. X1(x,y) = x� X2: value of second die. X2(x,y) = y� X = X1 +X2: sum of the dice. X(x,y) = x + y = X1(x,y) +X2(x,y)
Random variables super important! Running time of randomized quicksort is a randomvariable.
Michael Dinitz Lecture 3: Probability, Randomized Quicksort September 7, 2021 7 / 16
G x G
![Page 27: Lecture 3: Probabilistic Analysis, Randomized Quicksort](https://reader031.vdocuments.net/reader031/viewer/2022012422/617690aa98e1aa7d9a5e5923/html5/thumbnails/27.jpg)
Probability Basics I
Only semi-formal here. Look at CLRS Chapter 5, take Introduction to Probability
⌦: Sample space. Set of all possible outcomes.
� Roll two dice. ⌦ = {1,2, . . . ,6} × {1,2, . . . ,6}. Not {2,3, . . . ,12}
Event: subset of ⌦
� “Event that first die is 3”: {(3,x) ∶ x ∈ {1,2, . . . ,6}}� “Event that dice add up to 7 or 11”: {(x,y) ∈ ⌦ ∶ (x + y = 7) or (x + y = 11)}
Random Variable: X ∶ ⌦→ R� X1: value of first die. X1(x,y) = x� X2: value of second die. X2(x,y) = y� X = X1 +X2: sum of the dice. X(x,y) = x + y = X1(x,y) +X2(x,y)
Random variables super important! Running time of randomized quicksort is a randomvariable.
Michael Dinitz Lecture 3: Probability, Randomized Quicksort September 7, 2021 7 / 16
![Page 28: Lecture 3: Probabilistic Analysis, Randomized Quicksort](https://reader031.vdocuments.net/reader031/viewer/2022012422/617690aa98e1aa7d9a5e5923/html5/thumbnails/28.jpg)
Probability Basics I
Only semi-formal here. Look at CLRS Chapter 5, take Introduction to Probability
⌦: Sample space. Set of all possible outcomes.
� Roll two dice. ⌦ = {1,2, . . . ,6} × {1,2, . . . ,6}. Not {2,3, . . . ,12}Event: subset of ⌦
� “Event that first die is 3”: {(3,x) ∶ x ∈ {1,2, . . . ,6}}� “Event that dice add up to 7 or 11”: {(x,y) ∈ ⌦ ∶ (x + y = 7) or (x + y = 11)}
Random Variable: X ∶ ⌦→ R� X1: value of first die. X1(x,y) = x� X2: value of second die. X2(x,y) = y� X = X1 +X2: sum of the dice. X(x,y) = x + y = X1(x,y) +X2(x,y)
Random variables super important! Running time of randomized quicksort is a randomvariable.
Michael Dinitz Lecture 3: Probability, Randomized Quicksort September 7, 2021 7 / 16
Fa
![Page 29: Lecture 3: Probabilistic Analysis, Randomized Quicksort](https://reader031.vdocuments.net/reader031/viewer/2022012422/617690aa98e1aa7d9a5e5923/html5/thumbnails/29.jpg)
Probability Basics I
Only semi-formal here. Look at CLRS Chapter 5, take Introduction to Probability
⌦: Sample space. Set of all possible outcomes.
� Roll two dice. ⌦ = {1,2, . . . ,6} × {1,2, . . . ,6}. Not {2,3, . . . ,12}Event: subset of ⌦
� “Event that first die is 3”: {(3,x) ∶ x ∈ {1,2, . . . ,6}}� “Event that dice add up to 7 or 11”: {(x,y) ∈ ⌦ ∶ (x + y = 7) or (x + y = 11)}
Random Variable: X ∶ ⌦→ R� X1: value of first die. X1(x,y) = x� X2: value of second die. X2(x,y) = y� X = X1 +X2: sum of the dice. X(x,y) = x + y = X1(x,y) +X2(x,y)
Random variables super important! Running time of randomized quicksort is a randomvariable.
Michael Dinitz Lecture 3: Probability, Randomized Quicksort September 7, 2021 7 / 16
![Page 30: Lecture 3: Probabilistic Analysis, Randomized Quicksort](https://reader031.vdocuments.net/reader031/viewer/2022012422/617690aa98e1aa7d9a5e5923/html5/thumbnails/30.jpg)
Probability Basics I
Only semi-formal here. Look at CLRS Chapter 5, take Introduction to Probability
⌦: Sample space. Set of all possible outcomes.
� Roll two dice. ⌦ = {1,2, . . . ,6} × {1,2, . . . ,6}. Not {2,3, . . . ,12}Event: subset of ⌦
� “Event that first die is 3”: {(3,x) ∶ x ∈ {1,2, . . . ,6}}� “Event that dice add up to 7 or 11”: {(x,y) ∈ ⌦ ∶ (x + y = 7) or (x + y = 11)}
Random Variable: X ∶ ⌦→ R� X1: value of first die. X1(x,y) = x� X2: value of second die. X2(x,y) = y� X = X1 +X2: sum of the dice. X(x,y) = x + y = X1(x,y) +X2(x,y)
Random variables super important! Running time of randomized quicksort is a randomvariable.
Michael Dinitz Lecture 3: Probability, Randomized Quicksort September 7, 2021 7 / 16
O O
![Page 31: Lecture 3: Probabilistic Analysis, Randomized Quicksort](https://reader031.vdocuments.net/reader031/viewer/2022012422/617690aa98e1aa7d9a5e5923/html5/thumbnails/31.jpg)
Probability Basics I
Only semi-formal here. Look at CLRS Chapter 5, take Introduction to Probability
⌦: Sample space. Set of all possible outcomes.
� Roll two dice. ⌦ = {1,2, . . . ,6} × {1,2, . . . ,6}. Not {2,3, . . . ,12}Event: subset of ⌦
� “Event that first die is 3”: {(3,x) ∶ x ∈ {1,2, . . . ,6}}� “Event that dice add up to 7 or 11”: {(x,y) ∈ ⌦ ∶ (x + y = 7) or (x + y = 11)}
Random Variable: X ∶ ⌦→ R� X1: value of first die. X1(x,y) = x� X2: value of second die. X2(x,y) = y� X = X1 +X2: sum of the dice. X(x,y) = x + y = X1(x,y) +X2(x,y)
Random variables super important! Running time of randomized quicksort is a randomvariable.
Michael Dinitz Lecture 3: Probability, Randomized Quicksort September 7, 2021 7 / 16
![Page 32: Lecture 3: Probabilistic Analysis, Randomized Quicksort](https://reader031.vdocuments.net/reader031/viewer/2022012422/617690aa98e1aa7d9a5e5923/html5/thumbnails/32.jpg)
Probability Basics II
Want to define probabilities. Should use measure theory. Won’t.
For each e ∈ ⌦ let Pr[e] be probability of e (probability distribution)
� Pr[e] ≥ 0 for all e ∈ ⌦, and ∑e∈⌦Pr[e] = 1� Probability of an event A is Pr[A] = ∑e∈A Pr[e]
Conditional probability: if A and B are events:
Pr[B�A] = Pr[A�B]Pr[A] = ∑e∈A∩B Pr[e]
∑e∈A Pr[e]
Michael Dinitz Lecture 3: Probability, Randomized Quicksort September 7, 2021 8 / 16
![Page 33: Lecture 3: Probabilistic Analysis, Randomized Quicksort](https://reader031.vdocuments.net/reader031/viewer/2022012422/617690aa98e1aa7d9a5e5923/html5/thumbnails/33.jpg)
Probability Basics II
Want to define probabilities. Should use measure theory. Won’t.
For each e ∈ ⌦ let Pr[e] be probability of e (probability distribution)
� Pr[e] ≥ 0 for all e ∈ ⌦, and ∑e∈⌦Pr[e] = 1� Probability of an event A is Pr[A] = ∑e∈A Pr[e]
Conditional probability: if A and B are events:
Pr[B�A] = Pr[A�B]Pr[A] = ∑e∈A∩B Pr[e]
∑e∈A Pr[e]
Michael Dinitz Lecture 3: Probability, Randomized Quicksort September 7, 2021 8 / 16
I
![Page 34: Lecture 3: Probabilistic Analysis, Randomized Quicksort](https://reader031.vdocuments.net/reader031/viewer/2022012422/617690aa98e1aa7d9a5e5923/html5/thumbnails/34.jpg)
Probability Basics II
Want to define probabilities. Should use measure theory. Won’t.
For each e ∈ ⌦ let Pr[e] be probability of e (probability distribution)
� Pr[e] ≥ 0 for all e ∈ ⌦, and ∑e∈⌦Pr[e] = 1� Probability of an event A is Pr[A] = ∑e∈A Pr[e]
Conditional probability: if A and B are events:
Pr[B�A] = Pr[A�B]Pr[A] = ∑e∈A∩B Pr[e]
∑e∈A Pr[e]
Michael Dinitz Lecture 3: Probability, Randomized Quicksort September 7, 2021 8 / 16
Do
![Page 35: Lecture 3: Probabilistic Analysis, Randomized Quicksort](https://reader031.vdocuments.net/reader031/viewer/2022012422/617690aa98e1aa7d9a5e5923/html5/thumbnails/35.jpg)
Probability Basics III: Expectations
Expectation of a random variable:
E[X] = �e∈⌦
X(e)Pr[e]“Average” of the random variable according to probability distribution
Can be useful to rearrange terms to get di↵erent equation:
E[X] = �e∈⌦
X(e)Pr[e] = �y∈R �
e∈⌦∶X(e)=yy ⋅Pr[e] = �
y∈Ry ⋅Pr[X = y]
Conditional Expectation: A an event, X a random variable.
E[X�A] = 1
Pr[A] �e∈AX(e)Pr[e]
Michael Dinitz Lecture 3: Probability, Randomized Quicksort September 7, 2021 9 / 16
![Page 36: Lecture 3: Probabilistic Analysis, Randomized Quicksort](https://reader031.vdocuments.net/reader031/viewer/2022012422/617690aa98e1aa7d9a5e5923/html5/thumbnails/36.jpg)
Probability Basics III: Expectations
Expectation of a random variable:
E[X] = �e∈⌦
X(e)Pr[e]“Average” of the random variable according to probability distribution
Can be useful to rearrange terms to get di↵erent equation:
E[X] = �e∈⌦
X(e)Pr[e] = �y∈R �
e∈⌦∶X(e)=yy ⋅Pr[e] = �
y∈Ry ⋅Pr[X = y]
Conditional Expectation: A an event, X a random variable.
E[X�A] = 1
Pr[A] �e∈AX(e)Pr[e]
Michael Dinitz Lecture 3: Probability, Randomized Quicksort September 7, 2021 9 / 16
![Page 37: Lecture 3: Probabilistic Analysis, Randomized Quicksort](https://reader031.vdocuments.net/reader031/viewer/2022012422/617690aa98e1aa7d9a5e5923/html5/thumbnails/37.jpg)
Probability Basics III: Expectations
Expectation of a random variable:
E[X] = �e∈⌦
X(e)Pr[e]“Average” of the random variable according to probability distribution
Can be useful to rearrange terms to get di↵erent equation:
E[X] = �e∈⌦
X(e)Pr[e] = �y∈R �
e∈⌦∶X(e)=yy ⋅Pr[e] = �
y∈Ry ⋅Pr[X = y]
Conditional Expectation: A an event, X a random variable.
E[X�A] = 1
Pr[A] �e∈AX(e)Pr[e]
Michael Dinitz Lecture 3: Probability, Randomized Quicksort September 7, 2021 9 / 16
![Page 38: Lecture 3: Probabilistic Analysis, Randomized Quicksort](https://reader031.vdocuments.net/reader031/viewer/2022012422/617690aa98e1aa7d9a5e5923/html5/thumbnails/38.jpg)
Linearity of ExpectationsAmazing feature of expectations: linearity!
TheoremFor any two random variables X and Y, and any constants ↵ and �:E[↵X +�Y] = ↵E[X] +�E[Y]
Consider rolling two dice. Let X be sum. What is E[X]?� E[X] = ∑e∈⌦X(e)Pr[e]. 36 term sum!� E[X] = ∑y∈R y ⋅Pr[X = y]. What is Pr[X = 2], Pr[X = 3], . . . ?
Instead: X = X1 +X2. So E[X] = E[X1 +X2] = E[X1] + E[X2]E[X1] = E[X2] = 6�
y=11
6y = 21
6= 3.5
�⇒ E[X] = 3.5 + 3.5 = 7
Michael Dinitz Lecture 3: Probability, Randomized Quicksort September 7, 2021 10 / 16
![Page 39: Lecture 3: Probabilistic Analysis, Randomized Quicksort](https://reader031.vdocuments.net/reader031/viewer/2022012422/617690aa98e1aa7d9a5e5923/html5/thumbnails/39.jpg)
Linearity of ExpectationsAmazing feature of expectations: linearity!
TheoremFor any two random variables X and Y, and any constants ↵ and �:E[↵X +�Y] = ↵E[X] +�E[Y]Consider rolling two dice. Let X be sum. What is E[X]?� E[X] = ∑e∈⌦X(e)Pr[e]. 36 term sum!� E[X] = ∑y∈R y ⋅Pr[X = y]. What is Pr[X = 2], Pr[X = 3], . . . ?
Instead: X = X1 +X2. So E[X] = E[X1 +X2] = E[X1] + E[X2]E[X1] = E[X2] = 6�
y=11
6y = 21
6= 3.5
�⇒ E[X] = 3.5 + 3.5 = 7
Michael Dinitz Lecture 3: Probability, Randomized Quicksort September 7, 2021 10 / 16
![Page 40: Lecture 3: Probabilistic Analysis, Randomized Quicksort](https://reader031.vdocuments.net/reader031/viewer/2022012422/617690aa98e1aa7d9a5e5923/html5/thumbnails/40.jpg)
Linearity of ExpectationsAmazing feature of expectations: linearity!
TheoremFor any two random variables X and Y, and any constants ↵ and �:E[↵X +�Y] = ↵E[X] +�E[Y]Consider rolling two dice. Let X be sum. What is E[X]?� E[X] = ∑e∈⌦X(e)Pr[e]. 36 term sum!� E[X] = ∑y∈R y ⋅Pr[X = y]. What is Pr[X = 2], Pr[X = 3], . . . ?
Instead: X = X1 +X2. So E[X] = E[X1 +X2] = E[X1] + E[X2]
E[X1] = E[X2] = 6�y=1
1
6y = 21
6= 3.5
�⇒ E[X] = 3.5 + 3.5 = 7
Michael Dinitz Lecture 3: Probability, Randomized Quicksort September 7, 2021 10 / 16
Xcel X e face Veer
![Page 41: Lecture 3: Probabilistic Analysis, Randomized Quicksort](https://reader031.vdocuments.net/reader031/viewer/2022012422/617690aa98e1aa7d9a5e5923/html5/thumbnails/41.jpg)
Linearity of ExpectationsAmazing feature of expectations: linearity!
TheoremFor any two random variables X and Y, and any constants ↵ and �:E[↵X +�Y] = ↵E[X] +�E[Y]Consider rolling two dice. Let X be sum. What is E[X]?� E[X] = ∑e∈⌦X(e)Pr[e]. 36 term sum!� E[X] = ∑y∈R y ⋅Pr[X = y]. What is Pr[X = 2], Pr[X = 3], . . . ?
Instead: X = X1 +X2. So E[X] = E[X1 +X2] = E[X1] + E[X2]E[X1] = E[X2] = 6�
y=11
6y = 21
6= 3.5
�⇒ E[X] = 3.5 + 3.5 = 7
Michael Dinitz Lecture 3: Probability, Randomized Quicksort September 7, 2021 10 / 16
![Page 42: Lecture 3: Probabilistic Analysis, Randomized Quicksort](https://reader031.vdocuments.net/reader031/viewer/2022012422/617690aa98e1aa7d9a5e5923/html5/thumbnails/42.jpg)
Linearity of ExpectationsAmazing feature of expectations: linearity!
TheoremFor any two random variables X and Y, and any constants ↵ and �:E[↵X +�Y] = ↵E[X] +�E[Y]Consider rolling two dice. Let X be sum. What is E[X]?� E[X] = ∑e∈⌦X(e)Pr[e]. 36 term sum!� E[X] = ∑y∈R y ⋅Pr[X = y]. What is Pr[X = 2], Pr[X = 3], . . . ?
Instead: X = X1 +X2. So E[X] = E[X1 +X2] = E[X1] + E[X2]E[X1] = E[X2] = 6�
y=11
6y = 21
6= 3.5
�⇒ E[X] = 3.5 + 3.5 = 7Michael Dinitz Lecture 3: Probability, Randomized Quicksort September 7, 2021 10 / 16
![Page 43: Lecture 3: Probabilistic Analysis, Randomized Quicksort](https://reader031.vdocuments.net/reader031/viewer/2022012422/617690aa98e1aa7d9a5e5923/html5/thumbnails/43.jpg)
Linearity of Expectations: Proof
TheoremFor any two random variables X and Y, and any constants ↵ and �:E[↵X +�Y] = ↵E[X] +�E[Y]Proof.
E[↵X +�Y] = �e∈⌦
Pr[e] (↵X(e) +�Y(e))
= ↵�e∈⌦
Pr[e]X(e) +��e∈⌦
Pr[e]X(e)= ↵E[X] +�E[Y]
Holds no matter how correlated X and Y are!
Michael Dinitz Lecture 3: Probability, Randomized Quicksort September 7, 2021 11 / 16
![Page 44: Lecture 3: Probabilistic Analysis, Randomized Quicksort](https://reader031.vdocuments.net/reader031/viewer/2022012422/617690aa98e1aa7d9a5e5923/html5/thumbnails/44.jpg)
Linearity of Expectations: Proof
TheoremFor any two random variables X and Y, and any constants ↵ and �:E[↵X +�Y] = ↵E[X] +�E[Y]Proof.
E[↵X +�Y] = �e∈⌦
Pr[e] (↵X(e) +�Y(e))= ↵�
e∈⌦Pr[e]X(e) +��
e∈⌦Pr[e]X(e)
= ↵E[X] +�E[Y]Holds no matter how correlated X and Y are!
Michael Dinitz Lecture 3: Probability, Randomized Quicksort September 7, 2021 11 / 16
![Page 45: Lecture 3: Probabilistic Analysis, Randomized Quicksort](https://reader031.vdocuments.net/reader031/viewer/2022012422/617690aa98e1aa7d9a5e5923/html5/thumbnails/45.jpg)
Linearity of Expectations: Proof
TheoremFor any two random variables X and Y, and any constants ↵ and �:E[↵X +�Y] = ↵E[X] +�E[Y]Proof.
E[↵X +�Y] = �e∈⌦
Pr[e] (↵X(e) +�Y(e))= ↵�
e∈⌦Pr[e]X(e) +��
e∈⌦Pr[e]X(e)
= ↵E[X] +�E[Y]
Holds no matter how correlated X and Y are!
Michael Dinitz Lecture 3: Probability, Randomized Quicksort September 7, 2021 11 / 16
![Page 46: Lecture 3: Probabilistic Analysis, Randomized Quicksort](https://reader031.vdocuments.net/reader031/viewer/2022012422/617690aa98e1aa7d9a5e5923/html5/thumbnails/46.jpg)
Linearity of Expectations: Proof
TheoremFor any two random variables X and Y, and any constants ↵ and �:E[↵X +�Y] = ↵E[X] +�E[Y]Proof.
E[↵X +�Y] = �e∈⌦
Pr[e] (↵X(e) +�Y(e))= ↵�
e∈⌦Pr[e]X(e) +��
e∈⌦Pr[e]X(e)
= ↵E[X] +�E[Y]Holds no matter how correlated X and Y are!
Michael Dinitz Lecture 3: Probability, Randomized Quicksort September 7, 2021 11 / 16
![Page 47: Lecture 3: Probabilistic Analysis, Randomized Quicksort](https://reader031.vdocuments.net/reader031/viewer/2022012422/617690aa98e1aa7d9a5e5923/html5/thumbnails/47.jpg)
Randomized Quicksort I
Theorem
The expected running time of randomized quicksort is at most O(n log n).
Assume for simplicity all elements distinct. Running time = ⇥(# of comparisons)Definitions:
� X =# of comparisons (random variable)
� ei = i’th smallest element (for i ∈ {1, . . . ,n})� Xij random variable for all i, j ∈ {1, . . . ,n} with i < j:
Xij =�������1 if algorithm compares ei and ej at any point in time
0 otherwise
Michael Dinitz Lecture 3: Probability, Randomized Quicksort September 7, 2021 12 / 16
![Page 48: Lecture 3: Probabilistic Analysis, Randomized Quicksort](https://reader031.vdocuments.net/reader031/viewer/2022012422/617690aa98e1aa7d9a5e5923/html5/thumbnails/48.jpg)
Randomized Quicksort I
Theorem
The expected running time of randomized quicksort is at most O(n log n).Assume for simplicity all elements distinct. Running time = ⇥(# of comparisons)
Definitions:
� X =# of comparisons (random variable)
� ei = i’th smallest element (for i ∈ {1, . . . ,n})� Xij random variable for all i, j ∈ {1, . . . ,n} with i < j:
Xij =�������1 if algorithm compares ei and ej at any point in time
0 otherwise
Michael Dinitz Lecture 3: Probability, Randomized Quicksort September 7, 2021 12 / 16
![Page 49: Lecture 3: Probabilistic Analysis, Randomized Quicksort](https://reader031.vdocuments.net/reader031/viewer/2022012422/617690aa98e1aa7d9a5e5923/html5/thumbnails/49.jpg)
Randomized Quicksort I
Theorem
The expected running time of randomized quicksort is at most O(n log n).Assume for simplicity all elements distinct. Running time = ⇥(# of comparisons)Definitions:
� X =# of comparisons (random variable)
� ei = i’th smallest element (for i ∈ {1, . . . ,n})� Xij random variable for all i, j ∈ {1, . . . ,n} with i < j:
Xij =�������1 if algorithm compares ei and ej at any point in time
0 otherwise
Michael Dinitz Lecture 3: Probability, Randomized Quicksort September 7, 2021 12 / 16
![Page 50: Lecture 3: Probabilistic Analysis, Randomized Quicksort](https://reader031.vdocuments.net/reader031/viewer/2022012422/617690aa98e1aa7d9a5e5923/html5/thumbnails/50.jpg)
Randomized Quicksort II
Algorithm never compares the same two elements more than once �⇒ X = ∑n−1i=1 ∑n
j=i+1 Xij
E[X] = E������n−1�i=1
n�j=i+1
Xij
������ =n−1�i=1
n�j=i+1
E[Xij]So just need to understand E[Xij]Simple cases:
� j = i + 1: Xij = 1 no matter what, so E[Xij] = 1� i = 1, j = n: e1 and en compared if and only if first pivot chosen is e1 or en�⇒ E[X1n] = 2
n
Michael Dinitz Lecture 3: Probability, Randomized Quicksort September 7, 2021 13 / 16
![Page 51: Lecture 3: Probabilistic Analysis, Randomized Quicksort](https://reader031.vdocuments.net/reader031/viewer/2022012422/617690aa98e1aa7d9a5e5923/html5/thumbnails/51.jpg)
Randomized Quicksort II
Algorithm never compares the same two elements more than once �⇒ X = ∑n−1i=1 ∑n
j=i+1 Xij
E[X] = E������n−1�i=1
n�j=i+1
Xij
������ =n−1�i=1
n�j=i+1
E[Xij]
So just need to understand E[Xij]Simple cases:
� j = i + 1: Xij = 1 no matter what, so E[Xij] = 1� i = 1, j = n: e1 and en compared if and only if first pivot chosen is e1 or en�⇒ E[X1n] = 2
n
Michael Dinitz Lecture 3: Probability, Randomized Quicksort September 7, 2021 13 / 16
![Page 52: Lecture 3: Probabilistic Analysis, Randomized Quicksort](https://reader031.vdocuments.net/reader031/viewer/2022012422/617690aa98e1aa7d9a5e5923/html5/thumbnails/52.jpg)
Randomized Quicksort II
Algorithm never compares the same two elements more than once �⇒ X = ∑n−1i=1 ∑n
j=i+1 Xij
E[X] = E������n−1�i=1
n�j=i+1
Xij
������ =n−1�i=1
n�j=i+1
E[Xij]So just need to understand E[Xij]
Simple cases:
� j = i + 1: Xij = 1 no matter what, so E[Xij] = 1� i = 1, j = n: e1 and en compared if and only if first pivot chosen is e1 or en�⇒ E[X1n] = 2
n
Michael Dinitz Lecture 3: Probability, Randomized Quicksort September 7, 2021 13 / 16
![Page 53: Lecture 3: Probabilistic Analysis, Randomized Quicksort](https://reader031.vdocuments.net/reader031/viewer/2022012422/617690aa98e1aa7d9a5e5923/html5/thumbnails/53.jpg)
Randomized Quicksort II
Algorithm never compares the same two elements more than once �⇒ X = ∑n−1i=1 ∑n
j=i+1 Xij
E[X] = E������n−1�i=1
n�j=i+1
Xij
������ =n−1�i=1
n�j=i+1
E[Xij]So just need to understand E[Xij]Simple cases:
� j = i + 1: Xij = 1 no matter what, so E[Xij] = 1� i = 1, j = n: e1 and en compared if and only if first pivot chosen is e1 or en�⇒ E[X1n] = 2
n
Michael Dinitz Lecture 3: Probability, Randomized Quicksort September 7, 2021 13 / 16
![Page 54: Lecture 3: Probabilistic Analysis, Randomized Quicksort](https://reader031.vdocuments.net/reader031/viewer/2022012422/617690aa98e1aa7d9a5e5923/html5/thumbnails/54.jpg)
Randomized Quicksort II
Algorithm never compares the same two elements more than once �⇒ X = ∑n−1i=1 ∑n
j=i+1 Xij
E[X] = E������n−1�i=1
n�j=i+1
Xij
������ =n−1�i=1
n�j=i+1
E[Xij]So just need to understand E[Xij]Simple cases:
� j = i + 1:
Xij = 1 no matter what, so E[Xij] = 1� i = 1, j = n: e1 and en compared if and only if first pivot chosen is e1 or en�⇒ E[X1n] = 2
n
Michael Dinitz Lecture 3: Probability, Randomized Quicksort September 7, 2021 13 / 16
Fft A
Fifita
![Page 55: Lecture 3: Probabilistic Analysis, Randomized Quicksort](https://reader031.vdocuments.net/reader031/viewer/2022012422/617690aa98e1aa7d9a5e5923/html5/thumbnails/55.jpg)
Randomized Quicksort II
Algorithm never compares the same two elements more than once �⇒ X = ∑n−1i=1 ∑n
j=i+1 Xij
E[X] = E������n−1�i=1
n�j=i+1
Xij
������ =n−1�i=1
n�j=i+1
E[Xij]So just need to understand E[Xij]Simple cases:
� j = i + 1: Xij = 1 no matter what, so E[Xij] = 1
� i = 1, j = n: e1 and en compared if and only if first pivot chosen is e1 or en�⇒ E[X1n] = 2n
Michael Dinitz Lecture 3: Probability, Randomized Quicksort September 7, 2021 13 / 16
![Page 56: Lecture 3: Probabilistic Analysis, Randomized Quicksort](https://reader031.vdocuments.net/reader031/viewer/2022012422/617690aa98e1aa7d9a5e5923/html5/thumbnails/56.jpg)
Randomized Quicksort II
Algorithm never compares the same two elements more than once �⇒ X = ∑n−1i=1 ∑n
j=i+1 Xij
E[X] = E������n−1�i=1
n�j=i+1
Xij
������ =n−1�i=1
n�j=i+1
E[Xij]So just need to understand E[Xij]Simple cases:
� j = i + 1: Xij = 1 no matter what, so E[Xij] = 1� i = 1, j = n:
e1 and en compared if and only if first pivot chosen is e1 or en�⇒ E[X1n] = 2n
Michael Dinitz Lecture 3: Probability, Randomized Quicksort September 7, 2021 13 / 16
![Page 57: Lecture 3: Probabilistic Analysis, Randomized Quicksort](https://reader031.vdocuments.net/reader031/viewer/2022012422/617690aa98e1aa7d9a5e5923/html5/thumbnails/57.jpg)
Randomized Quicksort II
Algorithm never compares the same two elements more than once �⇒ X = ∑n−1i=1 ∑n
j=i+1 Xij
E[X] = E������n−1�i=1
n�j=i+1
Xij
������ =n−1�i=1
n�j=i+1
E[Xij]So just need to understand E[Xij]Simple cases:
� j = i + 1: Xij = 1 no matter what, so E[Xij] = 1� i = 1, j = n: e1 and en compared if and only if first pivot chosen is e1 or en�⇒ E[X1n] = 2
n
Michael Dinitz Lecture 3: Probability, Randomized Quicksort September 7, 2021 13 / 16
I 1 In O CI Ey
![Page 58: Lecture 3: Probabilistic Analysis, Randomized Quicksort](https://reader031.vdocuments.net/reader031/viewer/2022012422/617690aa98e1aa7d9a5e5923/html5/thumbnails/58.jpg)
E[Xij]: General Case (i < j)If p = ei or p = ej:
Xij = 1If ei < p < ej: Xij = 0If p < ei or p > ej: ? Both ei, ej in same recursive call.
� Condition on ei ≤ p ≤ ej: E[Xij � ei ≤ p ≤ ej] = 2j−i+1
� Condition on p �∈ [ei,ej]: still undetermined
So Xij not determined until ei ≤ p ≤ ej, and when it is determined has E[Xij] = 2j−i+1�⇒ E[Xij] = 2
j−i+1
Michael Dinitz Lecture 3: Probability, Randomized Quicksort September 7, 2021 14 / 16
Éte
sorted A
tisfiffeiore chosen as
pivot before any ee i Kc
![Page 59: Lecture 3: Probabilistic Analysis, Randomized Quicksort](https://reader031.vdocuments.net/reader031/viewer/2022012422/617690aa98e1aa7d9a5e5923/html5/thumbnails/59.jpg)
E[Xij]: General Case (i < j)If p = ei or p = ej: Xij = 1
If ei < p < ej: Xij = 0If p < ei or p > ej: ? Both ei, ej in same recursive call.
� Condition on ei ≤ p ≤ ej: E[Xij � ei ≤ p ≤ ej] = 2j−i+1
� Condition on p �∈ [ei,ej]: still undetermined
So Xij not determined until ei ≤ p ≤ ej, and when it is determined has E[Xij] = 2j−i+1�⇒ E[Xij] = 2
j−i+1
Michael Dinitz Lecture 3: Probability, Randomized Quicksort September 7, 2021 14 / 16
![Page 60: Lecture 3: Probabilistic Analysis, Randomized Quicksort](https://reader031.vdocuments.net/reader031/viewer/2022012422/617690aa98e1aa7d9a5e5923/html5/thumbnails/60.jpg)
E[Xij]: General Case (i < j)If p = ei or p = ej: Xij = 1If ei < p < ej:
Xij = 0If p < ei or p > ej: ? Both ei, ej in same recursive call.
� Condition on ei ≤ p ≤ ej: E[Xij � ei ≤ p ≤ ej] = 2j−i+1
� Condition on p �∈ [ei,ej]: still undetermined
So Xij not determined until ei ≤ p ≤ ej, and when it is determined has E[Xij] = 2j−i+1�⇒ E[Xij] = 2
j−i+1
Michael Dinitz Lecture 3: Probability, Randomized Quicksort September 7, 2021 14 / 16
![Page 61: Lecture 3: Probabilistic Analysis, Randomized Quicksort](https://reader031.vdocuments.net/reader031/viewer/2022012422/617690aa98e1aa7d9a5e5923/html5/thumbnails/61.jpg)
E[Xij]: General Case (i < j)If p = ei or p = ej: Xij = 1If ei < p < ej: Xij = 0
If p < ei or p > ej: ? Both ei, ej in same recursive call.
� Condition on ei ≤ p ≤ ej: E[Xij � ei ≤ p ≤ ej] = 2j−i+1
� Condition on p �∈ [ei,ej]: still undetermined
So Xij not determined until ei ≤ p ≤ ej, and when it is determined has E[Xij] = 2j−i+1�⇒ E[Xij] = 2
j−i+1
Michael Dinitz Lecture 3: Probability, Randomized Quicksort September 7, 2021 14 / 16
![Page 62: Lecture 3: Probabilistic Analysis, Randomized Quicksort](https://reader031.vdocuments.net/reader031/viewer/2022012422/617690aa98e1aa7d9a5e5923/html5/thumbnails/62.jpg)
E[Xij]: General Case (i < j)If p = ei or p = ej: Xij = 1If ei < p < ej: Xij = 0If p < ei or p > ej:
? Both ei, ej in same recursive call.
� Condition on ei ≤ p ≤ ej: E[Xij � ei ≤ p ≤ ej] = 2j−i+1
� Condition on p �∈ [ei,ej]: still undetermined
So Xij not determined until ei ≤ p ≤ ej, and when it is determined has E[Xij] = 2j−i+1�⇒ E[Xij] = 2
j−i+1
Michael Dinitz Lecture 3: Probability, Randomized Quicksort September 7, 2021 14 / 16
![Page 63: Lecture 3: Probabilistic Analysis, Randomized Quicksort](https://reader031.vdocuments.net/reader031/viewer/2022012422/617690aa98e1aa7d9a5e5923/html5/thumbnails/63.jpg)
E[Xij]: General Case (i < j)If p = ei or p = ej: Xij = 1If ei < p < ej: Xij = 0If p < ei or p > ej: ? Both ei, ej in same recursive call.
� Condition on ei ≤ p ≤ ej: E[Xij � ei ≤ p ≤ ej] = 2j−i+1
� Condition on p �∈ [ei,ej]: still undetermined
So Xij not determined until ei ≤ p ≤ ej, and when it is determined has E[Xij] = 2j−i+1�⇒ E[Xij] = 2
j−i+1
Michael Dinitz Lecture 3: Probability, Randomized Quicksort September 7, 2021 14 / 16
![Page 64: Lecture 3: Probabilistic Analysis, Randomized Quicksort](https://reader031.vdocuments.net/reader031/viewer/2022012422/617690aa98e1aa7d9a5e5923/html5/thumbnails/64.jpg)
E[Xij]: General Case (i < j)If p = ei or p = ej: Xij = 1If ei < p < ej: Xij = 0If p < ei or p > ej: ? Both ei, ej in same recursive call.
� Condition on ei ≤ p ≤ ej:
E[Xij � ei ≤ p ≤ ej] = 2j−i+1
� Condition on p �∈ [ei,ej]: still undetermined
So Xij not determined until ei ≤ p ≤ ej, and when it is determined has E[Xij] = 2j−i+1�⇒ E[Xij] = 2
j−i+1
Michael Dinitz Lecture 3: Probability, Randomized Quicksort September 7, 2021 14 / 16
![Page 65: Lecture 3: Probabilistic Analysis, Randomized Quicksort](https://reader031.vdocuments.net/reader031/viewer/2022012422/617690aa98e1aa7d9a5e5923/html5/thumbnails/65.jpg)
E[Xij]: General Case (i < j)If p = ei or p = ej: Xij = 1If ei < p < ej: Xij = 0If p < ei or p > ej: ? Both ei, ej in same recursive call.
� Condition on ei ≤ p ≤ ej: E[Xij � ei ≤ p ≤ ej] = 2j−i+1
� Condition on p �∈ [ei,ej]: still undetermined
So Xij not determined until ei ≤ p ≤ ej, and when it is determined has E[Xij] = 2j−i+1�⇒ E[Xij] = 2
j−i+1
Michael Dinitz Lecture 3: Probability, Randomized Quicksort September 7, 2021 14 / 16
![Page 66: Lecture 3: Probabilistic Analysis, Randomized Quicksort](https://reader031.vdocuments.net/reader031/viewer/2022012422/617690aa98e1aa7d9a5e5923/html5/thumbnails/66.jpg)
E[Xij]: General Case (i < j)If p = ei or p = ej: Xij = 1If ei < p < ej: Xij = 0If p < ei or p > ej: ? Both ei, ej in same recursive call.
� Condition on ei ≤ p ≤ ej: E[Xij � ei ≤ p ≤ ej] = 2j−i+1
� Condition on p �∈ [ei,ej]:
still undetermined
So Xij not determined until ei ≤ p ≤ ej, and when it is determined has E[Xij] = 2j−i+1�⇒ E[Xij] = 2
j−i+1
Michael Dinitz Lecture 3: Probability, Randomized Quicksort September 7, 2021 14 / 16
![Page 67: Lecture 3: Probabilistic Analysis, Randomized Quicksort](https://reader031.vdocuments.net/reader031/viewer/2022012422/617690aa98e1aa7d9a5e5923/html5/thumbnails/67.jpg)
E[Xij]: General Case (i < j)If p = ei or p = ej: Xij = 1If ei < p < ej: Xij = 0If p < ei or p > ej: ? Both ei, ej in same recursive call.
� Condition on ei ≤ p ≤ ej: E[Xij � ei ≤ p ≤ ej] = 2j−i+1
� Condition on p �∈ [ei,ej]: still undetermined
So Xij not determined until ei ≤ p ≤ ej, and when it is determined has E[Xij] = 2j−i+1�⇒ E[Xij] = 2
j−i+1
Michael Dinitz Lecture 3: Probability, Randomized Quicksort September 7, 2021 14 / 16
![Page 68: Lecture 3: Probabilistic Analysis, Randomized Quicksort](https://reader031.vdocuments.net/reader031/viewer/2022012422/617690aa98e1aa7d9a5e5923/html5/thumbnails/68.jpg)
E[Xij]: General Case (i < j)If p = ei or p = ej: Xij = 1If ei < p < ej: Xij = 0If p < ei or p > ej: ? Both ei, ej in same recursive call.
� Condition on ei ≤ p ≤ ej: E[Xij � ei ≤ p ≤ ej] = 2j−i+1
� Condition on p �∈ [ei,ej]: still undetermined
So Xij not determined until ei ≤ p ≤ ej, and when it is determined has E[Xij] = 2j−i+1�⇒ E[Xij] = 2
j−i+1
Michael Dinitz Lecture 3: Probability, Randomized Quicksort September 7, 2021 14 / 16
![Page 69: Lecture 3: Probabilistic Analysis, Randomized Quicksort](https://reader031.vdocuments.net/reader031/viewer/2022012422/617690aa98e1aa7d9a5e5923/html5/thumbnails/69.jpg)
E[Xij]: General Case (formally)
Let Yk be event that the k’th pivot is in [ei,ej] and all previous pivots not in [ei,ej]
�⇒ by definition, the Yk events are disjoint and partition sample space
Showed that E[Xij�Yk] = 2j−i+1 for all k.
E[Xij] = n�k=1
E[Xij�Yk]Pr[Yk] (Yk disjoint and partition ⌦)
= 2
j − i + 1n�
k=1Pr[Yk]
= 2
j − i + 1
Michael Dinitz Lecture 3: Probability, Randomized Quicksort September 7, 2021 15 / 16
![Page 70: Lecture 3: Probabilistic Analysis, Randomized Quicksort](https://reader031.vdocuments.net/reader031/viewer/2022012422/617690aa98e1aa7d9a5e5923/html5/thumbnails/70.jpg)
E[Xij]: General Case (formally)
Let Yk be event that the k’th pivot is in [ei,ej] and all previous pivots not in [ei,ej]�⇒ by definition, the Yk events are disjoint and partition sample space
Showed that E[Xij�Yk] = 2j−i+1 for all k.
E[Xij] = n�k=1
E[Xij�Yk]Pr[Yk] (Yk disjoint and partition ⌦)
= 2
j − i + 1n�
k=1Pr[Yk]
= 2
j − i + 1
Michael Dinitz Lecture 3: Probability, Randomized Quicksort September 7, 2021 15 / 16
![Page 71: Lecture 3: Probabilistic Analysis, Randomized Quicksort](https://reader031.vdocuments.net/reader031/viewer/2022012422/617690aa98e1aa7d9a5e5923/html5/thumbnails/71.jpg)
E[Xij]: General Case (formally)
Let Yk be event that the k’th pivot is in [ei,ej] and all previous pivots not in [ei,ej]�⇒ by definition, the Yk events are disjoint and partition sample space
Showed that E[Xij�Yk] = 2j−i+1 for all k.
E[Xij] = n�k=1
E[Xij�Yk]Pr[Yk] (Yk disjoint and partition ⌦)
= 2
j − i + 1n�
k=1Pr[Yk]
= 2
j − i + 1
Michael Dinitz Lecture 3: Probability, Randomized Quicksort September 7, 2021 15 / 16
![Page 72: Lecture 3: Probabilistic Analysis, Randomized Quicksort](https://reader031.vdocuments.net/reader031/viewer/2022012422/617690aa98e1aa7d9a5e5923/html5/thumbnails/72.jpg)
E[Xij]: General Case (formally)
Let Yk be event that the k’th pivot is in [ei,ej] and all previous pivots not in [ei,ej]�⇒ by definition, the Yk events are disjoint and partition sample space
Showed that E[Xij�Yk] = 2j−i+1 for all k.
E[Xij] = n�k=1
E[Xij�Yk]Pr[Yk] (Yk disjoint and partition ⌦)
= 2
j − i + 1n�
k=1Pr[Yk]
= 2
j − i + 1
Michael Dinitz Lecture 3: Probability, Randomized Quicksort September 7, 2021 15 / 16
![Page 73: Lecture 3: Probabilistic Analysis, Randomized Quicksort](https://reader031.vdocuments.net/reader031/viewer/2022012422/617690aa98e1aa7d9a5e5923/html5/thumbnails/73.jpg)
Randomized Quicksort: Final AnalysisExpected running time of randomized quicksort:
E[X] = n−1�i=1
n�j=i+1
E[Xij] (linearity of expectations)
= n−1�i=1
n�j=i+1
2
j − i + 1= 2 n−1�
i=1�12+ 13+ ⋅ ⋅ ⋅ + 1
n − i + 1�≤ 2 n−1�
i=1Hn
��Hn = n�
j=11
j
��
≤ 2nHn≤ O(n log n)Michael Dinitz Lecture 3: Probability, Randomized Quicksort September 7, 2021 16 / 16
th legal