04/21/2304/21/23 11

Chapter 12 - HeapsChapter 12 - Heaps

04/21/23 2

IntroductionIntroduction

►Heaps are largely about priority queues.Heaps are largely about priority queues.►They are an alternative data structure They are an alternative data structure

to implementing priority queues (we to implementing priority queues (we had arrays, linked lists…) had arrays, linked lists…)

►Recall the advantages and Recall the advantages and disadvantages of queues implemented disadvantages of queues implemented as arraysas arrays Insertions / deletions? O(n) … O(1)!Insertions / deletions? O(n) … O(1)!

►Priority queues are critical to many real-Priority queues are critical to many real-world applications.world applications.

04/21/23 3

IntroductionIntroduction

► First of all, a heap is a First of all, a heap is a kind ofkind of treetree that that offers both insertion and deletion in O(logoffers both insertion and deletion in O(log22n) n) time.time.

► Fast for insertions; not so fast for deletions.Fast for insertions; not so fast for deletions.

04/21/23 4

Introduction to HeapsIntroduction to Heaps

► Characteristics:Characteristics: 1. A heap is ‘complete’ (save for the last row – going 1. A heap is ‘complete’ (save for the last row – going

left to right – see figure 12.1, p. 580)left to right – see figure 12.1, p. 580) 2. 2. usuallyusually implementedimplemented as an as an arrayarray 3. Each node in a heap satisfies the ‘heap 3. Each node in a heap satisfies the ‘heap

condition,’ which states that condition,’ which states that every node’s key is every node’s key is larger than or equal to the keys of its childrenlarger than or equal to the keys of its children ..

The heap is thus an The heap is thus an abstractionabstraction; we can draw it ; we can draw it to to looklook like a tree, but recognize that it is the like a tree, but recognize that it is the array array that that implementsimplements this abstractionthis abstraction and that it is and that it is stored and processed in stored and processed in primaryprimary memory (RAM)memory (RAM)..

No ‘holes’ in the array.No ‘holes’ in the array.

04/21/23 5

Priority Queues, Heaps, and Priority Queues, Heaps, and ADTsADTs►Heaps are mostly used to implement Heaps are mostly used to implement

priority queues.priority queues.►Again, a heap is usually Again, a heap is usually implementedimplemented

as an as an arrayarray..

04/21/23 6

““Weakly Ordered”Weakly Ordered”► We know how a We know how a binarybinary searchsearch treetree is developed – with lesser is developed – with lesser

keys to the left; greater keys to the right as we descend.keys to the left; greater keys to the right as we descend. Because of this, we have nice, neat algorithms for binary search Because of this, we have nice, neat algorithms for binary search

trees.trees.

► Here: No Strong Ordering: Here: No Strong Ordering: But for nodes in a heap, But for nodes in a heap, we don’t have this strong orderingwe don’t have this strong ordering - and this - and this

can cause us some difficulty. can cause us some difficulty.

► Cannot assert much: Cannot assert much: We can We can only assert only assert as one descends in the heap, nodes will be in as one descends in the heap, nodes will be in

descending order (see rule 3)descending order (see rule 3)

Equivalently, nodes Equivalently, nodes belowbelow a node are <= the parent node. a node are <= the parent node.

► Weakly-ordered:Weakly-ordered: Heaps are thus said to be Heaps are thus said to be weakly orderedweakly ordered……

04/21/23 7

Weakly Ordered – MoreWeakly Ordered – More► No Convenient Search Mechanism: No Convenient Search Mechanism:

Because of weak ordering, there Because of weak ordering, there is no convenient searching is no convenient searching for a specified for a specified key as we have in binary search trees. key as we have in binary search trees.

Don’t have enough info at a node to decide whether to descend left or Don’t have enough info at a node to decide whether to descend left or right.right.

► Delete:Delete: So to delete a node with a So to delete a node with a specific key specific key there are issuesthere are issues No real slick way to find it.No real slick way to find it.

► Randomness: Randomness: So, a heap’s organization approaches ‘randomness.’ So, a heap’s organization approaches ‘randomness.’

► Sufficient Ordering: Sufficient Ordering: Yet, there is ‘sufficient ordering’ to allowYet, there is ‘sufficient ordering’ to allow quickquick removalremoval (yes, a delete) of the (yes, a delete) of the maximummaximum node and node and fastfast insertioninsertion of new nodes. of new nodes.

► As it turns out, these are the As it turns out, these are the only operations only operations one needs in using one needs in using a heap as a priority queue. a heap as a priority queue.

► We will discuss the algorithms later…We will discuss the algorithms later…

04/21/23 8

RemovalRemoval

►Removal is easy:Removal is easy: When we remove from a heap, we always When we remove from a heap, we always

remove the node with the remove the node with the largestlargest key. key. Hence, removal is quite easy and has index 0 Hence, removal is quite easy and has index 0

of the heap array.of the heap array. maxNode = heapArray[0];maxNode = heapArray[0];

►But tree is then not complete:But tree is then not complete: But once root is gone, tree is not complete and But once root is gone, tree is not complete and

we must fill this cell.we must fill this cell.

►Now this becomes interesting…Now this becomes interesting…

04/21/23 9

Removal of “maxNode”Removal of “maxNode”

►Move ‘last node’ to rootMove ‘last node’ to root.. Start by Start by movingmoving the ‘last node’ into the root. the ‘last node’ into the root. The ‘last’ node is the rightmost node in the The ‘last’ node is the rightmost node in the

lowest occupied level of the tree.lowest occupied level of the tree. This also corresponds to the This also corresponds to the last filled cell last filled cell

in the array (ahead).in the array (ahead).

►Trickle-downTrickle-down:: Then Then trickletrickle this last node this last node downdown until it is until it is

below a larger node and above a smaller one.below a larger node and above a smaller one.

04/21/23 10

100

90

30

80

50 70

45 5

60

20 10 40 55 Last node added

Last node added is 5.Delete Algorithm: heapArray[0] = heapArray [n-1] n--; // size of array is decreased by one.

Heap is represented logically as a tree. (Tree is organized to represent a priority queue. Easy to visualize.)

Heap is physically implemented (here) using an array.

Removal (Delete)

Note: NOT binary search tree!

04/21/23 11

95

82

63

51

37 10

30

70

43 27 53 34Last node added

Trickle down swapping node w/larger child until node >= children

As we trickle down, the nodes swap out always swapping the larger node with the node we are trickling down (so as to maintain the larger nodes above…).

We trickle down selecting the largest child with which to swap.We must compare, but always swap with the larger of the two.Note: we very well may NOT trickle down to a leaf node

Example(Different tree)

Delete (Removal)Trickle Down

04/21/23 12

82

70

63

51

37 1053

43 27 30 34

The new arrangement via a delete would be as above.

Go through this…Node 95 (the largest) is deleted.Tree remains balanced after delete and the rules for the heap are preserved.

(95 removed; compare 30 with 82; 82 selected and moved to root; Compare 30 with 70. 70 moves up. Compare 30 with 53; 53 moves up. 30 is a leaf node. )

04/21/23 13

Insertion – Trickle Insertion – Trickle UpUp► Pretty easy too. Easier than Deletion.Pretty easy too. Easier than Deletion.► Insertions usually Insertions usually trickle uptrickle up..

► Start at the Start at the bottombottom (first open position) via code: (first open position) via code:heapArray[n] = newNode;heapArray[n] = newNode;n++;n++;

InsertingInserting atat bottombottom will likely destroy the heap will likely destroy the heap condition.condition.

This will happen when the new node is larger than its This will happen when the new node is larger than its parent.parent.

TrickleTrickle upwardsupwards until node is until node is belowbelow a node larger a node larger than it and it is than it and it is aboveabove a node smaller (or equal to) it. a node smaller (or equal to) it.

Consider the following slides:Consider the following slides:

04/21/23 14

82

70

63

51

37 1055

43 27 30 34 95 Node to be added…

Clearly 95 must trickle up (and appears it will end up at the root!)

95

70

63

82

51 1055

43 27 30 34 37

Resultant heap:Easy to see swaps…

Trickle Up

Can easily see the swaps.

04/21/23 15

Insertion - moreInsertion - more

► Note that this is easier because we don’t have to Note that this is easier because we don’t have to comparecompare which node to swap (as we do in going which node to swap (as we do in going down). down).

► We only have We only have ONEONE parent, and it is equal to or larger! parent, and it is equal to or larger!► Progress until parent is larger, at whatever level that Progress until parent is larger, at whatever level that

might occur.might occur.

► A side point: Clearly, the same nodes may constitute A side point: Clearly, the same nodes may constitute different heaps depending upon their different heaps depending upon their arrivalarrival..

04/21/23 16

Insert / Delete – a ‘little bit’ of Insert / Delete – a ‘little bit’ of ImplementationImplementation

►While some implementations actually do the swap, While some implementations actually do the swap, there’s a there’s a lot of overheadlot of overhead here for a large heap. here for a large heap.

► A gain in efficiency is acquired if we substitute a A gain in efficiency is acquired if we substitute a ‘‘copycopy’ for a ‘’ for a ‘swapswap.’ (like the selection sort…).’ (like the selection sort…)

►Here, we Here, we copycopy the node to be trickled to a the node to be trickled to a temptemp areaarea and just do compares and copies (moves) until and just do compares and copies (moves) until the right spot is found. Then we can the right spot is found. Then we can copycopy (move) (move) the node (temp) (to be trickled down) into that node the node (temp) (to be trickled down) into that node location.location.

►More efficient than doing More efficient than doing swaps swaps repeatedly. repeatedly.

04/21/23 17

Heap AppletHeap Applet

►Google: Lafore appletsGoogle: Lafore applets

04/21/23 18

Java Code for HeapsJava Code for Heaps

► Code is reasonably straightforward.Code is reasonably straightforward.► We do, however, need to ensure that the We do, however, need to ensure that the

array is not full.array is not full.► If full, what do we do?If full, what do we do?► Likely if the array is an Likely if the array is an ArrayListArrayList or a or a VectorVector

(one dimensional array), we are in good (one dimensional array), we are in good shape and can (for the first step) merely add shape and can (for the first step) merely add the new entry to the end (bottom) of the the new entry to the end (bottom) of the Vector.Vector.

► No problems. No problems.

04/21/23 19

Heap ImplementationHeap Implementation

►Easy to discuss heaps Easy to discuss heaps as if as if heaps were heaps were implemented as trees even though the implemented as trees even though the implementationimplementation has been based on an has been based on an array or vector. array or vector.

►As implemented with an array, we can use As implemented with an array, we can use a a binarybinary treetree, but , but notnot a a binarybinary searchsearch treetree, because the ordering is not strong., because the ordering is not strong.

►Such a tree, complete at all times – with Such a tree, complete at all times – with no missing nodes – is called a no missing nodes – is called a tree-heaptree-heap..

04/21/23 20

Class ExerciseClass Exercise

►Draw an appropriate Draw an appropriate arrayarray (or vector) (or vector) to implement a tree-based heap whose to implement a tree-based heap whose ‘entries’ are sufficient to support ‘entries’ are sufficient to support Inserting into the heapInserting into the heap Removal from the heapRemoval from the heap Making the structure dynamicMaking the structure dynamic

►Pseudo-code your algorithms.Pseudo-code your algorithms.

Class ExerciseClass Exercise

► Implement a heap with a linked list.Implement a heap with a linked list.►Draw the structure you design.Draw the structure you design.►Present pseudocode to Present pseudocode to

Insert intoInsert into Delete fromDelete from

►the physical structure.the physical structure.

04/21/23 21

04/21/23 22

Sorting with a Heap:Sorting with a Heap:HeapHeapSortSort: insert() and : insert() and

remove()remove()► Very easy to implement.Very easy to implement.► We simply We simply insert()insert() all unordered items into the heap all unordered items into the heap

trickling trickling downdown using an insert() routine…. using an insert() routine….► Then, repeatedly use the Then, repeatedly use the removeremove() items in sorted () items in sorted

order….order….► Both insert() and remove() operate in O(logBoth insert() and remove() operate in O(log2 2 n) time n) time

and this must be applied n times, thus rendering an and this must be applied n times, thus rendering an O(nlogO(nlog22 n) time. n) time.

► This is the same as a QuickSortThis is the same as a QuickSort► Not as fast because there are more operations in the Not as fast because there are more operations in the

trickledown than in the inner loop of a quicksort.trickledown than in the inner loop of a quicksort.► Can be done both Can be done both iterativelyiteratively (using stacks) and (using stacks) and

recursively.recursively.► Code is in your book.Code is in your book.

04/21/23 23

Uses of HeapsUses of Heaps

► Use of Use of heap treesheap trees can be used to obtain improved can be used to obtain improved running times for several network optimization running times for several network optimization algorithms.algorithms.

► Can be used to assist in dynamically-allocating Can be used to assist in dynamically-allocating memory partitions.memory partitions.

► Lots of variants of heaps (see Google)Lots of variants of heaps (see Google)

► A heapsort is considered to be one of the best A heapsort is considered to be one of the best sorting methods being in-place with no quadratic sorting methods being in-place with no quadratic worst-case scenarios.worst-case scenarios.

► Finding the min, max, both the min and max, Finding the min, max, both the min and max, median, or even the k-th largest element can be median, or even the k-th largest element can be done in linear time using heaps.done in linear time using heaps.

► And more….And more….