CS 312: Algorithm Analysis

Lecture #12: Average Case Analysis of Quicksort

This work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License.

Slides by: Eric Ringger, with contributions from Mike Jones, Eric Mercer, Sean Warnick

Announcements

Homework #8 Due now

Project #3 You sent me your chosen theorem, right? Help session: Tuesday at 5pm in 1066 TMCB Scope: 1-2 pages

Objectives

To sketch the proof of the average case analysis of Quicksort

Average Case Analysis Given an array of unique items. How does Quicksort perform on average?

Specify sample space: set of all permutations of the list Assume that each permutation is equally likely. i.e., the

probability distribution ( ) over possible permutations 𝑝 𝑥is uniform

Specify Random Variable to be the run-time of Quicksort as a function of .

Compute the expected value of according to the distribution

Let’s sketch the proof now …

Quicksort: Average Case Analysis What is the Sample Space?

1 2 3

1 3 2

2 1 3

2 3 1

3 1 2

3 2 1

Quicksort: Average Case Analysis What is the Random Variable?

Values ofRandomVariable

1 2 3

1 3 2

2 1 3

2 3 1

3 1 2

3 2 1

Runtime on given input

t(left of pivot)+t(right of pivot) + linear pivot step

t(left of pivot)+t(right of pivot) + linear pivot step

t(left of pivot)+t(right of pivot) + linear pivot step

t(left of pivot)+t(right of pivot) + linear pivot step

t(left of pivot)+t(right of pivot) + linear pivot step

t(left of pivot)+t(right of pivot) + linear pivot step

Quicksort: Average Case Analysis What is the probability distribution?

Probability

1 2 3

1 3 2

2 1 3

2 3 1

3 1 2

3 2 1

1/n!

1/n!

1/n!

1/n!

1/n!

1/n!

t(left of pivot)+t(right of pivot) + linear pivot step

t(left of pivot)+t(right of pivot) + linear pivot step

t(left of pivot)+t(right of pivot) + linear pivot step

t(left of pivot)+t(right of pivot) + linear pivot step

t(left of pivot)+t(right of pivot) + linear pivot step

t(left of pivot)+t(right of pivot) + linear pivot step

Values ofRandomVariable

Uniform

Quicksort: Average Case Analysis Without loss of generality, Choose first element as pivot

Probability PivotValue

1 2 3

1 3 2

2 1 3

2 3 1

3 1 2

3 2 1

1/n!

1/n!

1/n!

1/n!

1/n!

1/n!

t(left of pivot)+t(right of pivot) + linear pivot step

t(left of pivot)+t(right of pivot) + linear pivot step

t(left of pivot)+t(right of pivot) + linear pivot step

t(left of pivot)+t(right of pivot) + linear pivot step

t(left of pivot)+t(right of pivot) + linear pivot step

t(left of pivot)+t(right of pivot) + linear pivot step

1

1

2

2

3

3

Value ofRandomVariable

Quicksort: Average Case Analysis Do the pivot

Probability PivotLocation

1 2 3

1 3 2

1 2 3

1 2 3

1 2 3

2 1 3

1/n!

1/n!

1/n!

1/n!

1/n!

1/n!

t(left of pivot)+t(right of pivot) + linear pivot step

t(left of pivot)+t(right of pivot) + linear pivot step

t(left of pivot)+t(right of pivot) + linear pivot step

t(left of pivot)+t(right of pivot) + linear pivot step

t(left of pivot)+t(right of pivot) + linear pivot step

t(left of pivot)+t(right of pivot) + linear pivot step

1st

1st

2nd

2nd

3rd

3rd

Value ofRandomVariable

Quicksort: Average Case Analysis Update the values of the RV to reflect the location of

the pivotProbability Pivot

Location

1/n!

1/n!

1/n!

1/n!

1/n!

1/n!

t(0)+t(2) + g(n)

t(0)+t(2) + g(n)

t(1)+t(1) + g(n)

t(1)+t(1) + g(n)

t(2)+t(0) + g(n)

t(2)+t(0) + g(n)

1st

1st

2nd

2nd

3rd

3rd

Where is theaverage time to sortan array of size m

And

is thelinear time to pivot

1 2 3

1 3 2

1 2 3

1 2 3

1 2 3

2 1 3

Value ofRandomVariable

Quicksort: Average Case Analysis How many of each unique value of the RV are there?

Probability PivotLocation

1/n!

1/n!

1/n!

1/n!

1/n!

1/n!

t(0)+t(2) + g(n)

t(0)+t(2) + g(n)

t(1)+t(1) + g(n)

t(1)+t(1) + g(n)

t(2)+t(0) + g(n)

t(2)+t(0) + g(n)

1st

1st

2nd

2nd

3rd

3rd

1 2 3

1 3 2

1 2 3

1 2 3

1 2 3

2 1 3

(n-1)!

(n-1)!

(n-1)!

Value ofRandomVariable

Quicksort: Average Case Analysis Group similar samples into events.

Probability

(n-1)!/n! = 1/n

(n-1)!/n! = 1/n

(n-1)!/n! = 1/n

t(0)+t(2) + g(n)

t(1)+t(1) + g(n)

t(2)+t(0) + g(n)

1 2 3

1 2 3

1 2 3

Value ofRandomVariable

Probability Mass Function The probability mass function of random variable is just :

In general, for pivot position :

𝑥𝑡 (0)+𝑡 (𝑛−1)+𝑔(𝑛) 𝑡 (𝑛−1)+𝑡(0)+𝑔(𝑛)𝑡 (𝑛/2)+𝑡 (𝑛 /2)+𝑔(𝑛)

1/n𝑝 (𝑥)

… …

Quicksort: Average Case Analysis What is the Expected, or Average, value of , the

running time of Quicksort?

1

1( ) ( 1) ( )n

p

g n t p t n pn

Probability that will be chosen as the pivot

Cost of each possible pivot selection

Cost of sorting the smallest elements (before the pivot)

Cost of sorting the remaining elements (after the pivot)

1

1 ( 1) ( ) ( )n

p

t p t n p g nn

( ) [ ] ( )x

t n E X p x x

1

1

1 1

1( ) [ ] ( ) ( 1) ( )

1( ) ( 1) ( )

1( ) ( 1) ( )

n

p

n

p

n n

p p

t n E X g n t p t n pn

g n t p t n pn

g n t p t n pn

Quicksort: Average Case Analysis

Quicksort: Average Case Analysis

Probability

(n-1)!/n! = 1/n

(n-1)!/n! = 1/n

(n-1)!/n! = 1/n

t(0)+t(2) + g(n)

t(1)+t(1) + g(n)

t(2)+t(0) + g(n)

1 2 3

1 2 3

1 2 3

Value ofRandomVariable

1

1

1 1

1 1

0 0

1 1

0 0

1

0

1( ) [ ] ( ) ( 1) ( )

1( ) ( 1) ( )

1( ) ( 1) ( )

1( ) ( ) ( )

2 2( ) ( ) ( 1) ( )

2Thus, ( ) ( ) (

n

p

n

p

n n

p p

n n

k k

n n

k k

n

k

t n E X g n t p t n pn

g n t p t n pn

g n t p t n pn

g n t k t kn

g n t k n t kn n

t n t kn

1)n

Quicksort: Average Case Analysis

Quicksort: Average Case Analysis1

0

2( ) ( ) ( 1)n

k

t n t k nn

2( ) (0) ... ( 2) ( 1) ( 1)t n t t n t n nn

2( 1) (0) ... ( 2) ( 2)1

t n t t n nn

1 2 ( 1)( 2)( 1) (0) ... ( 2)n n nt n t t nn n n

1 2 ( 1)( 2)( ) ( 1) ( 1) ( 1)n n nt n t n t n nn n n

1 2 ( 1) ( 1)( 2)( ) ( 1)n n n n nt n t nn n n

Quicksort: Average Case Analysis

1 2( 1)( ) ( 1)n nt n t nn n

1 2 2( 1)( ) ( 1)n nt n t nn n n

1 2 ( 1) ( 1)( 2)( ) ( 1)n n n n nt n t nn n n

Quicksort: Average Case Analysis1 2( 1)( ) ( 1)n nt n t nn n

( ) ( 1) 2( 1)1 ( 1)

t n t n nn n n n

( )Let ( )

1t ny nn

2( 1)( ) ( 1)( 1)ny n y n

n n

1

2( 1)( )( 1)

n

i

iy ni i

1

1as , ( ) 2 2ln( 1)

n

i

n y n ni

( ) ( 1) ( ) 2( 1) ln ( log )t n n y n nn O nn

Summary: Average Case Analysis

We have walked through the process for conducting an average case analysis:

Characterize your input Sample space Probability Distribution

Measure Define your random variable: average running time of algorithm Compute the expected value of this RV

Algebra Requires cleverness with recurrence relations!

Asymptotic Order of Growth

Fair Game?

You could pick this proof for the proof project and fill in the blanks If you do so, include a section that lays out

the high-level method for doing average case analysis in general!

Make clear which parts of the proof accomplish these steps.

Probability theory I expect you to know the definitions for our

terms from probability theory: Sample space Sample Event probability distribution random variable value of a random variable expected value of a random variable (with respect

to a given probability distribution)

Assignment

Homework: HW #8.5 Some more probability exercises