MA 252: Data Structures and Algorithms
Lecture 9http://www.iitg.ernet.in/psm/indexing_ma252/y12/index.htmlhttp://www.iitg.ernet.in/psm/indexing_ma252/y12/index.html
Partha Sarathi Mandal
Dept. of Mathematics, IIT Guwahati
• We are lucky if the partitioning is balance i.e.,
(n/2, n/2).
• We are unlucky if the partitioning is imbalance
i.e., (1, n-1) or (n-1, 1).
More about quicksort
i.e., (1, n-1) or (n-1, 1).
P. S. Mandal, IITG
More intuition
• Suppose we alternate lucky, unlucky, lucky, unlucky, lucky, ….
• L(n)= 2U(n/2) + Θ(n) lucky
• U(n)= L(n –1) + Θ(n) unlucky• U(n)= L(n –1) + Θ(n) unlucky
Solving:
L(n) = 2(L(n/2 –1) + Θ(n/2)) + Θ(n)
= 2L(n/2 –1) + Θ(n)
= Θ(nlg n)
How can we make sure we are usually lucky ?
P. S. Mandal, IITG
Randomized quicksort
• IDEA: Partition around a random element.
• Running time is independent of the input order.
• No assumptions need to be made about the • No assumptions need to be made about the input distribution.
• No specific input elicits the worst-case behavior.
• The worst case is determined only by the output of a random-number generator.
P. S. Mandal, IITG
Pseudocode for
Randomized quicksort
P. S. Mandal, IITG
Randomized quicksort analysis
• Let T(n) =the random variable for the running time of randomized quicksort on an input of size n, assuming random numbers are independent.
• For k= 0, 1, …, n–1, define the indicator random
variable:variable:
• E[Xk] = 0.Pr{Xk=0} + 1.Pr{Xk=1} = Pr{Xk = 1} = 1/n, since all splits are equally likely, assuming elements are distinct.
P. S. Mandal, IITG
Analysis (continued)
P. S. Mandal, IITG
Calculating expectation
P. S. Mandal, IITG
Calculating expectation
P. S. Mandal, IITG
Calculating expectation
P. S. Mandal, IITG
Calculating expectation
P. S. Mandal, IITG
Calculating expectation
P. S. Mandal, IITG
Exercise
(The k = 0, 1 terms can be absorbed in the Θ(n) )
Exercise: Exercise:
Prove: E[T(n)] ≤ anlg n for constant a > 0.
– Choose a large enough so that anlgn dominates
E[T(n)] for sufficiently small n ≥ 2.
Use fact:
P. S. Mandal, IITG
Substitution method
P. S. Mandal, IITG
Substitution method
P. S. Mandal, IITG
Substitution method
P. S. Mandal, IITG
Substitution method
P. S. Mandal, IITG
Quicksort in practice
• Quicksort is a great general-purpose sorting
algorithm.
• Quicksort is typically over twice as fast as
merge sort.merge sort.
• Quicksort can benefit substantially from code
tuning.
• Quicksort behaves well even with caching and
virtual memory.
P. S. Mandal, IITG
How fast can we sort?
• All the sorting algorithms we have seen so far are comparison sorts: only use comparisons to determine the relative order of elements.
• E.g., insertion sort, merge sort, quicksort, heapsort.heapsort.
• The best worst-case running time that we’ve seen for comparison sorting is O(nlg n).
Is O(nlg n) the best we can do ?
• Decision tree scan help us answer this question.
P. S. Mandal, IITG
Decision-tree example
• Each internal node is labeled i:j for i, j ∈ {1, 2, …, n}.
• The left subtree shows subsequent comparisons if ai ≤ aj.
• The right subtree shows subsequent comparisons if ai ≥ aj.
P. S. Mandal, IITG
Decision-tree example
• Each internal node is labeled i:j for i, j ∈ {1, 2, …, n}.
• The left subtree shows subsequent comparisons if ai ≤ aj.
• The right subtree shows subsequent comparisons if ai ≥ aj.
P. S. Mandal, IITG
Decision-tree example
• Each internal node is labeled i:j for i, j ∈ {1, 2, …, n}.
• The left subtree shows subsequent comparisons if ai ≤ aj.
• The right subtree shows subsequent comparisons if ai ≥ aj.
P. S. Mandal, IITG
Decision-tree example
• Each internal node is labeled i:j for i, j ∈ {1, 2, …, n}.
• The left subtree shows subsequent comparisons if ai ≤ aj.
• The right subtree shows subsequent comparisons if ai ≥ aj.
P. S. Mandal, IITG
Decision-tree example
• Each leaf contains a permutation ⟨π(1), π(2),…, π(n)⟩
to indicate that the ordering aπ(1) ≤ aπ(2) ≤ … ≤ aπ(n) has
been established.
P. S. Mandal, IITG
Decision-tree model
A decision tree can model the execution of any comparison sort:
• One tree for each input size n.
• View the algorithm as splitting whenever it compares two elements.compares two elements.
• The tree contains the comparisons along all possible instruction traces.
• The running time of the algorithm = the length of the path taken.
• Worst-case running time = height of tree.
P. S. Mandal, IITG
Lower bound for decision-tree sorting
Theorem: Any decision tree that can sort nelements must have height Ω(nlg n).
Proof: The tree must contain ≥ n! leaves, since there are n! possible permutations. A height-h binary tree has ≤ 2h leaves. Thus, n! ≤ 2h.
∴
tree has ≤ 2h leaves. Thus, n! ≤ 2h.
∴ h ≥ lg (n!) (lg is monotonically increasing)
≥ lg ((n/e)n) (Stirling’s formula)
= nlg n – nlg e
= Ω(nlg n)
P. S. Mandal, IITG
Lower bound for comparison sorting
Corollary: Heapsort and merge sort are
asymptotically optimal comparison sorting
algorithms.
P. S. Mandal, IITG