![Page 1: Algorithms and Data Structures - STAR - Homestargroup.uwaterloo.ca/~ece250/materials/notes/Lecture18-Heaps.pdf · Selection Sort v A takes Θ(n) and B takes Θ(1): Θ(n2) in total](https://reader034.vdocuments.net/reader034/viewer/2022042416/5f323f4f717b2d591844411e/html5/thumbnails/1.jpg)
ECE250: Algorithms and Data Structures
Heaps
Materials from CLRS: Chapter 6.1, 6.2, 6.3
Ladan Tahvildari, PEng, SMIEEE Professor
Software Technologies Applied Research (STAR) Group
Dept. of Elect. & Comp. Eng.
University of Waterloo
![Page 2: Algorithms and Data Structures - STAR - Homestargroup.uwaterloo.ca/~ece250/materials/notes/Lecture18-Heaps.pdf · Selection Sort v A takes Θ(n) and B takes Θ(1): Θ(n2) in total](https://reader034.vdocuments.net/reader034/viewer/2022042416/5f323f4f717b2d591844411e/html5/thumbnails/2.jpg)
Acknowledgements
v The following resources have been used to prepare materials for this course: Ø MIT OpenCourseWare Ø Introduction To Algorithms (CLRS Book) Ø Data Structures and Algorithm Analysis in C++ (M. Wiess) Ø Data Structures and Algorithms in C++ (M. Goodrich)
v Thanks to many people for pointing out mistakes, providing suggestions, or helping to improve the quality of this course over the last ten years: Ø http://www.stargroup.uwaterloo.ca/~ece250/acknowledgment/
Lecture 18 ECE250 2
![Page 3: Algorithms and Data Structures - STAR - Homestargroup.uwaterloo.ca/~ece250/materials/notes/Lecture18-Heaps.pdf · Selection Sort v A takes Θ(n) and B takes Θ(1): Θ(n2) in total](https://reader034.vdocuments.net/reader034/viewer/2022042416/5f323f4f717b2d591844411e/html5/thumbnails/3.jpg)
Lecture 18 ECE250 3
Sorting Algorithms
v Insertion Sort Ø Worst-case running time Θ(n2)
v Merge Sort Ø Worst-case running time Θ(n log n), but requires
additional memory Θ(n);
![Page 4: Algorithms and Data Structures - STAR - Homestargroup.uwaterloo.ca/~ece250/materials/notes/Lecture18-Heaps.pdf · Selection Sort v A takes Θ(n) and B takes Θ(1): Θ(n2) in total](https://reader034.vdocuments.net/reader034/viewer/2022042416/5f323f4f717b2d591844411e/html5/thumbnails/4.jpg)
Lecture 18 ECE250 4
Selection Sort
v A takes Θ(n) and B takes Θ(1): Θ(n2) in total
v Idea for improvement: use a data structure, to do both A and B in O(lg n) time, balancing the work, achieving a better trade-off, and a total running time O(n log n)
Selection-Sort(A[1..n]): For i = n downto 2 A: Find the largest element among A[1..i] B: Exchange it with A[i]
![Page 5: Algorithms and Data Structures - STAR - Homestargroup.uwaterloo.ca/~ece250/materials/notes/Lecture18-Heaps.pdf · Selection Sort v A takes Θ(n) and B takes Θ(1): Θ(n2) in total](https://reader034.vdocuments.net/reader034/viewer/2022042416/5f323f4f717b2d591844411e/html5/thumbnails/5.jpg)
Lecture 18 ECE250 5
Heaps
v Binary heap data structure A Ø array Ø Can be viewed as a nearly complete binary tree
§ All levels, except the lowest one are completely filled Ø The key in root is greater or equal than all its children, and
the left and right subtrees are again binary heaps
v Two attributes Ø length[A] Ø heap-size[A]
![Page 6: Algorithms and Data Structures - STAR - Homestargroup.uwaterloo.ca/~ece250/materials/notes/Lecture18-Heaps.pdf · Selection Sort v A takes Θ(n) and B takes Θ(1): Θ(n2) in total](https://reader034.vdocuments.net/reader034/viewer/2022042416/5f323f4f717b2d591844411e/html5/thumbnails/6.jpg)
Lecture 18 ECE250 6
A Max Heap
1 2 3 4 5 6 7 8 9 10 16 15 10 8 7 9 3 2 4 1
Parent (i) return ⎣i/2⎦
Left (i) return 2i
Right (i) return 2i+1
Heap property:
A[Parent(i)] ≥ A[i]
![Page 7: Algorithms and Data Structures - STAR - Homestargroup.uwaterloo.ca/~ece250/materials/notes/Lecture18-Heaps.pdf · Selection Sort v A takes Θ(n) and B takes Θ(1): Θ(n2) in total](https://reader034.vdocuments.net/reader034/viewer/2022042416/5f323f4f717b2d591844411e/html5/thumbnails/7.jpg)
Lecture 18 ECE250 7
Heaps
v Notice the implicit tree links; children of node i are 2i and 2i+1
v Why is this useful? Ø In a binary representation, a multiplication/division
by two is left/right shift Ø Adding 1 can be done by adding the lowest bit
![Page 8: Algorithms and Data Structures - STAR - Homestargroup.uwaterloo.ca/~ece250/materials/notes/Lecture18-Heaps.pdf · Selection Sort v A takes Θ(n) and B takes Θ(1): Θ(n2) in total](https://reader034.vdocuments.net/reader034/viewer/2022042416/5f323f4f717b2d591844411e/html5/thumbnails/8.jpg)
Lecture 18 ECE250 8
Heapify
v i is index into the array A
v Binary trees rooted at Left(i) and Right(i) are heaps
v But, A[i] might be smaller than its children, thus violating the heap property
v The method Heapify makes A a heap once more by moving A[i] down the heap until the heap property is satisfied again
![Page 9: Algorithms and Data Structures - STAR - Homestargroup.uwaterloo.ca/~ece250/materials/notes/Lecture18-Heaps.pdf · Selection Sort v A takes Θ(n) and B takes Θ(1): Θ(n2) in total](https://reader034.vdocuments.net/reader034/viewer/2022042416/5f323f4f717b2d591844411e/html5/thumbnails/9.jpg)
Lecture 18 ECE250 9
Heapify
![Page 10: Algorithms and Data Structures - STAR - Homestargroup.uwaterloo.ca/~ece250/materials/notes/Lecture18-Heaps.pdf · Selection Sort v A takes Θ(n) and B takes Θ(1): Θ(n2) in total](https://reader034.vdocuments.net/reader034/viewer/2022042416/5f323f4f717b2d591844411e/html5/thumbnails/10.jpg)
Lecture 18 ECE250 10
Heapify Example
![Page 11: Algorithms and Data Structures - STAR - Homestargroup.uwaterloo.ca/~ece250/materials/notes/Lecture18-Heaps.pdf · Selection Sort v A takes Θ(n) and B takes Θ(1): Θ(n2) in total](https://reader034.vdocuments.net/reader034/viewer/2022042416/5f323f4f717b2d591844411e/html5/thumbnails/11.jpg)
Lecture 18 ECE250 11
Heapify: Running Time
v The running time of Heapify on a subtree of size n rooted at node i is Ø determining the relationship between elements: Θ(1)
Ø plus the time to run Heapify on a subtree rooted at one of the children of i, where 2n/3 is the worst-case size of this subtree.
Ø Alternatively § Running time on a node of height h: O(h)
( ) (2 /3) (1) ( ) (log )T n T n T n O n≤ +Θ ⇒ =
![Page 12: Algorithms and Data Structures - STAR - Homestargroup.uwaterloo.ca/~ece250/materials/notes/Lecture18-Heaps.pdf · Selection Sort v A takes Θ(n) and B takes Θ(1): Θ(n2) in total](https://reader034.vdocuments.net/reader034/viewer/2022042416/5f323f4f717b2d591844411e/html5/thumbnails/12.jpg)
Lecture 18 ECE250 12
Building a Heap
v Convert an array A[1...n], where n = length[A], into a heap
v Notice that the elements in the subarray A[(⎣n/2⎦ + 1)...n] are already 1-element heaps to begin with!
[they are leaves, each is a 1-element heap already]
![Page 13: Algorithms and Data Structures - STAR - Homestargroup.uwaterloo.ca/~ece250/materials/notes/Lecture18-Heaps.pdf · Selection Sort v A takes Θ(n) and B takes Θ(1): Θ(n2) in total](https://reader034.vdocuments.net/reader034/viewer/2022042416/5f323f4f717b2d591844411e/html5/thumbnails/13.jpg)
Building a
Heap
(4,1,3,2,16,9,10,14,8,7)
![Page 14: Algorithms and Data Structures - STAR - Homestargroup.uwaterloo.ca/~ece250/materials/notes/Lecture18-Heaps.pdf · Selection Sort v A takes Θ(n) and B takes Θ(1): Θ(n2) in total](https://reader034.vdocuments.net/reader034/viewer/2022042416/5f323f4f717b2d591844411e/html5/thumbnails/14.jpg)
Lecture 18 ECE250 14
Building a Heap: Analysis (1)
v Correctness: induction on i, all trees rooted at m > i are heaps
v Running time: n calls to Heapify = n O(lg n) = O(n lg n)
v Good enough for an O(n lg n) bound on Heapsort, but sometimes we build heaps for other reasons, would be nice to have a tight bound Ø Intuition: for most of the time Heapify works on smaller than n
element heaps
![Page 15: Algorithms and Data Structures - STAR - Homestargroup.uwaterloo.ca/~ece250/materials/notes/Lecture18-Heaps.pdf · Selection Sort v A takes Θ(n) and B takes Θ(1): Θ(n2) in total](https://reader034.vdocuments.net/reader034/viewer/2022042416/5f323f4f717b2d591844411e/html5/thumbnails/15.jpg)
Lecture 18 ECE250 15
Building a Heap: Analysis (2)
v Definitions Ø height of node: longest path from node to leaf Ø height of tree: height of root
Ø time to Heapify = O(height (k) of subtree rooted at i) Ø assume n = 2k – 1 (a complete binary tree height = ⎣lg n⎦)
( )( )
lg lg
21 1
1 1 1( ) 2 3 ... 12 4 8
1/ 21 since 2 2 2 1 1/ 2
( )
n n
i ii i
n n nT n O k
i iO n
O n
⎢ ⎥ ⎢ ⎥⎣ ⎦ ⎣ ⎦
= =
+ + +⎛ ⎞= + ⋅ + ⋅ + + ⋅⎜ ⎟⎝ ⎠
⎛ ⎞= + ⋅ = =⎜ ⎟⎜ ⎟ −⎝ ⎠=
∑ ∑
![Page 16: Algorithms and Data Structures - STAR - Homestargroup.uwaterloo.ca/~ece250/materials/notes/Lecture18-Heaps.pdf · Selection Sort v A takes Θ(n) and B takes Θ(1): Θ(n2) in total](https://reader034.vdocuments.net/reader034/viewer/2022042416/5f323f4f717b2d591844411e/html5/thumbnails/16.jpg)
Lecture 18 ECE250 16
Building a Heap: Analysis (3)
v How? By using the following "trick"
v Threfore Build-Heap time is O(n)
( )
( )
0
12
1
21
1
1 if 1 //differentiate1
1 //multiply by 1
1 //plug in 21
1/ 2 22 1/ 4
i
i
i
i
i
i
ii
x xx
i x xx
xi x xx
i
∞
=
∞−
=
∞
=
∞
=
= <−
⋅ =−
⋅ = =−
= =
∑
∑
∑
∑