CS 312: Algorithm Design & Analysis

Lecture #12: Average Case Analysis of Quicksort

This work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License.

Slides by: Eric Ringger, with contributions from Mike Jones, Eric Mercer, Sean Warnick

Announcements

Due now: HW #8

Project #7: TSP Early: Wednesday Due: next Friday Sent: link for Leaderboard Coming up: Google survey for One-time

Competition Set

Objectives

Sketch the proof of the average case analysis of Quicksort

Apply ideas from probability theory

Average Case Analysis

Given an array of unique items, how does Quicksort perform on average?

Strategy: Specify sample space: set of all permutations of the list Assume that each permutation is equally likely. i.e., the probability

distribution over possible permutations is uniform Specify Random Variable to be the run-time of Quicksort as a

function of . Derive the probability mass function over legal values of . Compute the expected value of according to the distribution

Let’s sketch the proof now …

Quicksort: Average Case Analysis What is the Sample Space?

1 2 3

1 3 2

2 1 3

2 3 1

3 1 2

3 2 1

Quicksort: Average Case Analysis What is the probability distribution?

Probability

1 2 3

1 3 2

2 1 3

2 3 1

3 1 2

3 2 1

1/n!

1/n!

1/n!

1/n!

1/n!

1/n!

Uniform

Quicksort: Average Case Analysis Without loss of generality, Choose first element as pivot

Probability PivotValue

1 2 3

1 3 2

2 1 3

2 3 1

3 1 2

3 2 1

1/n!

1/n!

1/n!

1/n!

1/n!

1/n!

1

1

2

2

3

3

Why does this choice not bias the proof?

Quicksort: Average Case Analysis Do the pivot

Probability PivotLocation

1 2 3

1 3 2

1 2 3

1 2 3

1 2 3

2 1 3

1/n!

1/n!

1/n!

1/n!

1/n!

1/n!

1st

1st

2nd

2nd

3rd

3rd

Quicksort: Average Case Analysis

Probability PivotLocation

1/n!

1/n!

1/n!

1/n!

1/n!

1/n!

t(0)+t(2) + 2

t(0)+t(2) + 2

t(1)+t(1) + 2

t(1)+t(1) + 2

t(2)+t(0) + 2

t(2)+t(0) + 2

1st

1st

2nd

2nd

3rd

3rd

Where is theaverage time to sortan array of size

1 2 3

1 3 2

1 2 3

1 2 3

1 2 3

2 1 3

Value ofRandomVariable

What is the Random Variable? Runtime on given input

Quicksort: Average Case Analysis How many of each unique value of the RV are there?

Probability PivotLocation

1/n!

1/n!

1/n!

1/n!

1/n!

1/n!

t(0)+t(2) + 2

t(0)+t(2) + 2

t(1)+t(1) + 2

t(1)+t(1) + 2

t(2)+t(0) + 2

t(2)+t(0) + 2

1st

1st

2nd

2nd

3rd

3rd

1 2 3

1 3 2

1 2 3

1 2 3

1 2 3

2 1 3

(n-1)!

(n-1)!

(n-1)!

Value ofRandomVariable

Quicksort: Average Case Analysis Group similar samples into events.

Probability

(n-1)!/n! = 1/n

(n-1)!/n! = 1/n

(n-1)!/n! = 1/n

t(0)+t(2) + 2

t(1)+t(1) + 2

t(2)+t(0) + 2

1 2 3

1 2 3

1 2 3

Value ofRandomVariable

Exemplar

Our probability mass function for random variable is:

In general, for pivot position :

𝑡 (⌊𝑛2⌋−1)+𝑡 (⌊

𝑛2⌋ )+𝑔(𝑛)

Probability Mass Function

𝑡 (0)+𝑡 (𝑛−1)+𝑔(𝑛) 𝑡 (𝑛−1)+𝑡(0)+𝑔(𝑛)𝑥

1/n𝑝 (𝑥)

… …

(a worst case) (best case) (a worst case)

Quicksort: Average Case Analysis What is the Expected, or Average, value of , the

running time of Quicksort?

1

1( 1) ( 1) ( )

n

p

n t p t n pn

Probability that will be chosen as the pivotRunning time of each possible pivot selectionRunning time of sorting the smallest elements (ahead of the pivot)

Cost of sorting the remaining elements (after the pivot)

1

1( 1) ( ) ( 1)

n

p

t p t n p nn

( )( ) [ ] ( )·p xx

t n E X p x x

1

1

1 1

1( ) [ ] ( 1) ( 1) ( )

1( 1) ( 1) ( )

1( 1) ( 1) ( )

n

p

n

p

n n

p p

t n E X n t p t n pn

n t p t n pn

n t p t n pn

Quicksort: Average Case Analysis

Next goal: Simplify

Quicksort: Average Case Analysis

Probability

1/n

1/n

1/n

t(0)+t(2) + 2

t(1)+t(1) + 2

t(2)+t(0) + 2

1 2 3

1 2 3

1 2 3

Value ofRandomVariable

1 1

( 1) ( )n n

p p

t p t n p

1

1

1 1

1 1

0 0

1

0

1

0

1( ) [ ] ( 1) ( 1) ( )

1( 1) ( 1) ( )

1( 1) ( 1) ( )

1( 1) ( ) ( )

2( 1) ( )

2Thus, ( ) ( ) ( 1)

n

p

n

p

n n

p p

n n

k j

n

k

n

k

t n E X n t p t n pn

n t p t n pn

n t p t n pn

n t k t jn

n t kn

t n t k nn

Quicksort: Average Case Analysis

Next goal: Eliminate some of the t() terms

Quicksort: Average Case Analysis1

0

2( ) ( ) ( 1)

n

k

t n t k nn

2( ) (0) ... ( 2) ( 1) ( 1)t n t t n t n n

n

2( 1) (0) ... ( 2) ( 2)

1t n t t n n

n

1 2 ( 1)( 2)( 1) (0) ... ( 2)

n n nt n t t n

n n n

1 2 ( 1)( 2)( ) ( 1) ( 1) ( 1)

n n nt n t n t n n

n n n

1 2 ( 1) ( 1)( 2)( ) ( 1)

n n n n nt n t n

n n n

Next goal: Simplify

Quicksort: Average Case Analysis

1 2( 1)( ) ( 1)

n nt n t n

n n

1 2 2( 1)( ) ( 1)

n nt n t n

n n n

1 2 ( 1) ( 1)( 2)( ) ( 1)

n n n n nt n t n

n n n

Next goal: Solve the recurrence

Quicksort: Average Case Analysis1 2( 1)

( ) ( 1)n n

t n t nn n

( ) ( 1) 2( 1)

1 ( 1)

t n t n n

n n n n

( )

Let ( )1

t ny n

n

2( 1)

( ) ( 1)( 1)

ny n y n

n n

1

2( 1)( )

( 1)

n

i

iy n

i i

1

1as , ( ) 2 2ln

( 1)

n

i

n y n ni

( ) ( 1) ( )t n n y n

( ) 2( 1) ln

2 ln ( log )2ln ( ln )

t n n n

n n n nO n nn O

(0) 0y

Summary: Average Case Analysis

We have walked through the process for conducting an average case analysis:

1. Characterize the input Sample space Probability distribution

2. Expected value Define your random variable: running time of algorithm Derive the probability mass function over possible running times Compute the expected value of your random variable

3. Algebra & Limit Requires cleverness with algebra, incl. recurrence relations! Non-trivial limit

4. Conclusion: Asymptotic Order of Growth

Probability theory

I expect you to know and be able to apply the definitions for our terms from probability theory: Sample space Sample Event Probability distribution Probability space Random variable Value of a random variable Probability mass function over values of a random variable Expected value of a random variable (with respect to the

probability mass function)

Assignment

Homework: HW #8.5 A simple average case analysis!