Download - HEAPS
HEAPS
Amihood Amir
Bar Ilan University2014
Sorting
Bubblesort:Until no exchanges:
For i=1 to n-1
if A[i]>A[i+1] then exchange their values
end
end
Time: O(n2)
"Bubbling Up" the Largest Element
Traverse a collection of elements– Move from the front to the end– “Bubble” the largest value to the end
using pair-wise comparisons and swapping
512354277 101
1 2 3 4 5 6
Swap42 77
"Bubbling Up" the Largest Element
Traverse a collection of elements– Move from the front to the end– “Bubble” the largest value to the end
using pair-wise comparisons and swapping
512357742 101
1 2 3 4 5 6
Swap35 77
"Bubbling Up" the Largest Element
Traverse a collection of elements– Move from the front to the end– “Bubble” the largest value to the end
using pair-wise comparisons and swapping
512773542 101
1 2 3 4 5 6
Swap12 77
"Bubbling Up" the Largest Element
Traverse a collection of elements– Move from the front to the end– “Bubble” the largest value to the end
using pair-wise comparisons and swapping
577123542 101
1 2 3 4 5 6
No need to swap
"Bubbling Up" the Largest Element
Traverse a collection of elements– Move from the front to the end– “Bubble” the largest value to the end
using pair-wise comparisons and swapping
577123542 101
1 2 3 4 5 6
Swap5 101
"Bubbling Up" the Largest Element
Traverse a collection of elements– Move from the front to the end– “Bubble” the largest value to the end
using pair-wise comparisons and swapping
77123542 5
1 2 3 4 5 6
101
Largest value correctly placed
Sorting
Selection sort:For i=1 to n do
For j=i+1 to n do
If A[i]>A[j] then exchange them
end
end
Time: = O(n2)
n
i
i1
Selecting the Smallest Element
Traverse a collection of elements– Move from the front to the end– Select the smallest value using pair-wise
comparisons and swapping
512354277 101
1 2 3 4 5 6
Swap42 77
Selecting the Smallest Element
Traverse a collection of elements– Move from the front to the end– Select the smallest value using pair-wise
comparisons and swapping
512357742 101
1 2 3 4 5 6
Swap
35 42
Selecting the Smallest Element
Traverse a collection of elements– Move from the front to the end– Select the smallest value using pair-wise
comparisons and swapping
51277 101
1 2 3 4 5 6
Swap
35 4212 35
Selecting the Smallest Element
Traverse a collection of elements– Move from the front to the end– Select the smallest value using pair-wise
comparisons and swapping
577 101
1 2 3 4 5 6
4212 35
No need to swap
Selecting the Smallest Element
Traverse a collection of elements– Move from the front to the end– Select the smallest value using pair-wise
comparisons and swapping
577 101
1 2 3 4 5 6
4212 35
Swap
5 12
Selecting the Smallest Element
Traverse a collection of elements– Move from the front to the end– Select the smallest value using pair-wise
comparisons and swapping
77 101
1 2 3 4 5 6
42 355 12
Smallest value correctly placed
Sorting
Insertion sort:
For i=1 to n do
Insert A[i+1] into appropriate (sorted)
position in A[1],…,A[i]
end
Time: = O(n2)
n
i
i1
Keeping Prefix Sorted
Traverse a collection of elements– Move from the front to the end– Keep the prefix sorted throughout
512354277 101
1 2 3 4 5 6
Keeping Prefix Sorted
Traverse a collection of elements– Move from the front to the end– Keep the prefix sorted throughout
512357742 101
1 2 3 4 5 6
Keeping Prefix Sorted
Traverse a collection of elements– Move from the front to the end– Keep the prefix sorted throughout
512357742 101
1 2 3 4 5 6
Keeping Prefix Sorted
Traverse a collection of elements– Move from the front to the end– Keep the prefix sorted throughout
512357742 101
1 2 3 4 5 6
Keeping Prefix Sorted
Traverse a collection of elements– Move from the front to the end– Keep the prefix sorted throughout
512773542 101
1 2 3 4 5 6
Keeping Prefix Sorted
Traverse a collection of elements– Move from the front to the end– Keep the prefix sorted throughout
512774235 101
1 2 3 4 5 6
Keeping Prefix Sorted
Traverse a collection of elements– Move from the front to the end– Keep the prefix sorted throughout
512774235 101
1 2 3 4 5 6
Keeping Prefix Sorted
Traverse a collection of elements– Move from the front to the end– Keep the prefix sorted throughout
577124235 101
1 2 3 4 5 6
Keeping Prefix Sorted
Traverse a collection of elements– Move from the front to the end– Keep the prefix sorted throughout
577421235 101
1 2 3 4 5 6
Keeping Prefix Sorted
Traverse a collection of elements– Move from the front to the end– Keep the prefix sorted throughout
577423512 101
1 2 3 4 5 6
Keeping Prefix Sorted
Traverse a collection of elements– Move from the front to the end– Keep the prefix sorted throughout
577423512 101
1 2 3 4 5 6
Keeping Prefix Sorted
Traverse a collection of elements– Move from the front to the end– Keep the prefix sorted throughout
577423512 101
1 2 3 4 5 6
Keeping Prefix Sorted
Traverse a collection of elements– Move from the front to the end– Keep the prefix sorted throughout
577423512 101
1 2 3 4 5 6
Keeping Prefix Sorted
Traverse a collection of elements– Move from the front to the end– Keep the prefix sorted throughout
577423512 101
1 2 3 4 5 6
Keeping Prefix Sorted
Traverse a collection of elements– Move from the front to the end– Keep the prefix sorted throughout
10177423512 5
1 2 3 4 5 6
Keeping Prefix Sorted
Traverse a collection of elements– Move from the front to the end– Keep the prefix sorted throughout
101 5423512 77
1 2 3 4 5 6
Keeping Prefix Sorted
Traverse a collection of elements– Move from the front to the end– Keep the prefix sorted throughout
101 42 53512 77
1 2 3 4 5 6
Keeping Prefix Sorted
Traverse a collection of elements– Move from the front to the end– Keep the prefix sorted throughout
101 42 35 512 77
1 2 3 4 5 6
Keeping Prefix Sorted
Traverse a collection of elements– Move from the front to the end– Keep the prefix sorted throughout
101 42 35 12 5 77
1 2 3 4 5 6
Keeping Prefix Sorted
Traverse a collection of elements– Move from the front to the end– Keep the prefix sorted throughout
101 42 35 12 5 77
1 2 3 4 5 6
Complexity
The Time of all algorithms we saw:
O(n2)
Can we do better?
Merge Sort
Based on the Merging operation.
Given two sorted arrays: A[1],…,A[n] and B[1],…,B[m].
merge them into one sorted array: C[1],…,C[n+m]
MergingPa, Pb, Pc <- 1
While (Pa < n+1 and Pb < m+1)
If A[Pa] ≤ B[Pb] then C[Pc] <- A[Pa]
Pa <- Pa+1
Pc <- Pc+1
else C[Pc] <- B[Pb]
Pb <- Pb+1
Pc <- Pc+1
end
MergingIf Pa=n+1 and Pb < m+1
then for i=Pb to m do
C[Pc] <- B[i]Pc <- Pc +1
endIf Pb=n+1 and Pa < m+1
then for i=Pa to n doC[Pc] <- A[i]Pc <- Pc +1
endend Algorithm
Merging
3102354 152575A: B:
C:
Merging
3102354 52575
1
A: B:
C:
Merging
102354 52575
13
A: B:
C:
Merging
102354 2575
135
A: B:
C:
Merging
2354 2575
13510
A: B:
C:
Merging
54 2575
1351023
A: B:
C:
MergingMerging
54 75
135102325
A: B:
C:
Merging
75
13510232554
A: B:
C:
Merging
1351023255475
A: B:
C:
Time: O(n+m) Linear !!!
Merge SortA recursive sorting algorithm:Mergesort(A)If n=1 then Return(A) else
Split A[1],…,A[n] to two length n/2 arrays:
A[1],…,A[n/2] and A[n/2+1],…,A[n]
Mergesort(A[1],…,A[n/2])
Mergesort(A[n/2+1],…,A[n])
Merge(A[1],…,A[n/2], A[n/2+1],…,A[n], B)
A <- B
Return(A)
Merge Sort Example996861558358640
Merge Sort Example996861558358640
9968615 58358640
Merge Sort Example996861558358640
9968615 58358640
8615996 5835 8640
Merge Sort Example996861558358640
9968615 58358640
8615996 5835 8640
99 6 86 15 58 35 86 40
Merge Sort Example996861558358640
9968615 58358640
8615996 5835 8640
99 6 86 15 58 35 86 40
4 0
Merge Sort Example
99 6 86 15 58 35 86 04
4 0Merge
Merge Sort Example
1586699 5835 0486
99 6 86 15 58 35 86 04
Merge
Merge Sort Example
6158699 04355886
1586699 5835 0486
Merge
Merge Sort Example046153558868699
6158699 04355886
Merge
Merge Sort Example046153558868699
Merge Sort TimeThe recurrence:
TM(n)=2TM(n/2)+n
TM(1)=1
Closed Form of TM(n):
O(n log n)
A Data Structures AlgorithmSort array A using AVL Trees:
For i=1 to n do:
Insert A[i] into AVL tree
For i=1 to n do:
Extract smallest element from AVL tree and put in A[i]
Sorting Algorithm using AVLCan we extract the smallest element
from AVL tree?
-- Yes. Go all the way to the left.
Time: O(log n)Therefore:
Sorting Algorithm Time: O(n log n)
Priority QueueWhat property of the AVL Tree did we
use?
Only that we can find the minimum fast.
A Priority Queue is a data structure that supports the following operations:
INSERT (Q,k)
EXTRACTMIN(Q,m)
Priority Queue Implementation
Heap:A binary tree where every vertex has a
key.For every vertex v, key(v) ≤ key(w),
where w is a child of v.
Difference between heap and binary search tree:10 10
5 3040 30
Heap Binary search tree
HeapAdditional Heap Property:All levels of the heap are full, except
possibly the last level, which is left-filled. 4
6
207
811
5
9
1214
15
2516
Example:
Heap PropertiesThe keys on every path from the root to
a leaf are nondecreasing.
4
6
207
811
5
9
1214
15
2516
Example:
Heap Properties
The height of a heap with n elements is O(log n).
)1(log n
4
6
207
811
5
9
1214
15
2516
Example:
1
0 0
12222h
i
h
i
hihi n h < log n ≤ h+1
Heap Insertion
Insert 6
3
35232822
8201021
74
25 6
Heap Insertion
Insert 6
3
35232822
8201021
74
25 620
6
Heap Insertion
Insert 6
3
35232822
81021
74
25 20
6
6
7
OK
Heap Deletion
Delete Min 3
3
35232822
81021
4
25 20
6
7
20
Now fix heap
Heap Deletion
Delete Min 3
35232822
81021
4
25
6
7
20
20
4
Heap Deletion
Delete Min 3
35232822
81021
25
6
720
20
4
10
OK
Heap Operations Time
O(log n)
Conclude: Heap of n elements can be constructed in time:
O(n log n)
But: We can do a lot better!!!
Build Heap
Put all elements in a tree where all levels are full and the last level is left-filled.
Construct heaps from leaves up
Time Analysis
For trees in level h : 1For trees in level h-1 : 2
For trees in level h-i : i
For trees in level 0 : log n
Time Analysis
Number of trees in level h: n/2Number of trees in level h-1: n/22
……Number of trees in level h-i:n/2i+1
…Number of trees in level 0 : 1
Total Time
Calculate it: Note that n/2 trees do constant time work, and another quarter need to go to two levels. Only one tree needs to fix log n levels.
Formally:
Need to calculate this
n
i
n
iiin
ini
nn
nnnn log
1 1log22 22
log2
...32
22
12
Total Time
We know: for x<1.
Differentiate both sides:
Multiply both sides by x:
1 1
1
i
i
xx
12
1
)1(
1
i
i
xix
12)1(i
i
x
xix
Total Time
In our case: x = 1/2.
So total time: O(n) .
2)1( 4
1
21
12
21
21
21
n
i
ii
In our case:
Look Ma No Pointers
Our tree is almost complete. All levels are full and the last one is left-filled.
So write down all keys sequentially from top to bottom and left to right.
Getting rid of Pointers
Example:
3,4,6,21,10,7,8,22,28,23,35,25,20
3
35232822
871021
64
25 20
Calculate Locationnode i in level j = location 2j-1+i in array.
Location i in array = node in level .
1logmod ii
3
35232822
871021
64
25 20
ilog
Node j in row i is the 2i-1+j’s element.Its left son is the 2i+1-1+2(j-1)+1’s
element.
But 2i+1-1+2(j-1)+1 = 2i+1+2(j-1) = 2i+1+2j-2 = 2(2i-1+j).
Location of children
row i j
Conclude:
Location of left son of i is 2i.Location of right son of i is 2i+1.Location of father of i is: i/2, if i is even (i-1)/2, if i is odd
Location of Children
How?Given array A of n elements:Construct a heap HA in-place.For i=n downto 1 do
EXTRACTMIN(HA,v)Put v in location A[i]
Endfor
Time: O(n) for heap construction and O(nlog n) for loop.
Heap Sort
Note:Array A is now sorted in non-increasing
order.What do we do if we want to sort in
non-decreasing order?Options:• Reverse order of A in linear time in-
place.• Use MAX rather than MIN priority
Queue.
Heap Sort