priority queues - university of haifaoren/ads/lecture 3 - binomial and fibonacci heaps.pdf · n $ b...

2

Priority Queues

Supports the following operations. !  Insert element x. !  Return min element. !  Return and delete minimum element. !  Decrease key of element x to k.

Applications. !  Dijkstra's shortest path algorithm. !  Prim's MST algorithm. !  Event-driven simulation. !  Huffman encoding. !  Heapsort. !  . . .

Thursday, December 13, 12

4

Dijkstra/Prim 1 make-heap |V| insert |V| delete-min |E| decrease-key

Priority Queues

make-heap

Operation

insert

find-min

delete-min

union

decrease-key

delete

1

Binary

log N 1

log N

N log N log N

1

Binomial

log N log N log N

log N log N log N

1

Fibonacci *

1 1

log N

1 1

log N

1

Relaxed

1 1

log N

1 1

log N

1

Linked List

1 N N

1 1 N

is-empty 1 1 1 1 1

Heaps

O(|E| + |V| log |V|) O(|E| log |V|) O(|V|2)


20

Priority Queues

make-heap

Operation

insert

find-min

delete-min

union

decrease-key

delete

1

Binary

log N 1

log N

N log N log N

1

Binomial

log N log N log N

log N log N log N

1

Fibonacci *

1 1

log N

1 1

log N

1

Relaxed

1 1

log N

1 1

log N

1

Linked List

1 N N

1 1 N

is-empty 1 1 1 1 1

Heaps


21

Binomial Tree

Binomial tree. !  Recursive definition:

Bk-1

Bk-1

B0 Bk

B0 B1 B2 B3 B4


22

Binomial Tree

Useful properties of order k binomial tree Bk. !  Number of nodes = 2k. !  Height = k. !  Degree of root = k. !  Deleting root yields binomial

trees Bk-1, … , B0.

Proof. !  By induction on k.

B0 B1 B2 B3 B4

B1

Bk

Bk+1

B2 B0


24

Binomial Heap

Binomial heap. Vuillemin, 1978. !  Sequence of binomial trees that satisfy binomial heap property.

–  each tree is min-heap ordered –  0 or 1 binomial tree of order k

B4 B0 B1

55

45 32

30

24

23 22

50

48 31 17

44 8 29 10

6

37

3 18


26

Binomial Heap: Properties

Properties of N-node binomial heap. !  Min key contained in root of B0, B1, . . . , Bk. !  Contains binomial tree Bi iff bi = 1 where bn$ b2b1b0 is binary

representation of N. !  At most !log2 N" + 1 binomial trees. !  Height % !log2 N".

B4 B0 B1

55

45 32

30

24

23 22

50

48 31 17

44 8 29 10

6

37

3 18

N = 19 # trees = 3 height = 4 binary = 10011


27

Binomial Heap: Union

Create heap H that is union of heaps H' and H''. !  "Mergeable heaps." !  Easy if H' and H'' are each order k binomial trees.

–  connect roots of H' and H'' –  choose smaller key to be root of H

H'' 55

45 32

30

24

23 22

50

48 31 17

44 8 29 10

6

H'


28


0 0 1 1

1 0 0 1 +

0 1 1 1

1 1

1 1 0

1

19 + 7 = 26

55

45 32

30

24

23 22

50

48 31 17

44 8 29 10

6

37

3 18

41

33 28

15

25

7 12

+


29


55

45 32

30

24

23 22

50

48 31 17

44 8 29 10

6

37

3 18

41

33 28

15

25

7 12

+


30


55

45 32

30

24

23 22

50

48 31 17

44 8 29 10

6

37

3

41

33 28

15

25

7

+

12

18

18

12


31

55

45 32

30

24

23 22

50

48 31 17

44 8 29 10

6

37

3

41

33 28

15

25

7

+

12

18

25

37 7

3

18

12

18

12


32

55

45 32

30

24

23 22

50

48 31 17

44 8 29 10

6

37

3

41

33 28

15

25

7

12

+

18

25

37 7

3

41

28 33 25

37 15 7

3

18

12

18

12


33

55

45 32

30

24

23 22

50

48 31 17

44 8 29 10

6

37

3

41

33 28

15

25

7

+

18

12

41

28 33 25

37 15 7

3

12

18

25

37 7

3

41

28 33 25

37 15 7

3

18

12


34

55

45 32

30

24

23 22

50

48 31 17

44 8 29 10

6

37

3

41

33 28

15

25

7

+

18

12

41

28 33 25

37 15 7

3

12

18

25

37 7

3

41

28 33 25

37 15 7

3

55

45 32

30

24

23 22

50

48 31 17

44 8 29 10

6

18

12


35


Create heap H that is union of heaps H' and H''. !  Analogous to binary addition.

Running time. O(log N) !  Proportional to number of trees in root lists % 2( !log2 N" + 1).

0 0 1 1

1 0 0 1 +

0 1 1 1

1 1

1 1 0

1

19 + 7 = 26


36

3

37

6 18

55

45 32

30

24

23 22

50

48 31 17

44 8 29 10

H

Binomial Heap: Delete Min

Delete node with minimum key in binomial heap H. !  Find root x with min key in root list of H, and delete !  H' & broken binomial trees !  H & Union(H', H)

Running time. O(log N)


37

Binomial Heap: Delete Min

Delete node with minimum key in binomial heap H. !  Find root x with min key in root list of H, and delete !  H' & broken binomial trees !  H & Union(H', H)


55

45 32

30

24

23 22

50

48 31 17

37

6 18

44 8 29 10

H

H'


38

3

37

6 18

55

x 32

30

24

23 22

50

48 31 17

44 8 29 10

H

Binomial Heap: Decrease Key

Decrease key of node x in binomial heap H. !  Suppose x is in binomial tree Bk. !  Bubble node x up the tree if x is too small.

Running time. O(log N) !  Proportional to depth of node x % !log2 N" .

depth = 3


39

Binomial Heap: Delete

Delete node x in binomial heap H. !  Decrease key of x to -'. !  Delete min.



40

Binomial Heap: Insert

Insert a new node x into binomial heap H. !  H' & MakeHeap(x) !  H & Union(H', H)


3

37

6 18

55

45 32

30

24

23 22

50

48 31 17

44 8 29 10

H

x

H'


41

Binomial Heap: Sequence of Inserts

Insert a new node x into binomial heap H. !  If N = .......0, then only 1 steps. !  If N = ......01, then only 2 steps. !  If N = .....011, then only 3 steps. !  If N = ....0111, then only 4 steps.

Inserting 1 item can take #(log N) time. !  If N = 11...111, then log2 N steps.

But, inserting sequence of N items takes O(N) time! !  (N/2)(1) + (N/4)(2) + (N/8)(3) + . . . % 2N !  Amortized analysis. !  Basis for getting most operations

down to constant time.

50

48 31 17

44 29 10

3

37

6 x


1

Priority Queues

make-heap

Operation

insert

find-min

delete-min

union

decrease-key

delete

1 Binary

log N 1

log N

N log N log N

1 Binomial

log N log N log N

log N log N log N

1 Fibonacci †

1 1

log N

1 1

log N

1 Relaxed

1 1

log N

1 1

log N

1 Linked List

1 N N

1 1 N

is-empty 1 1 1 1 1

Heaps

this time † amortized

3

Fibonacci Heaps: Structure Fibonacci heap. � Set of min-heap ordered trees.

7 23

30

17

35

26 46

24

H 39

41 18 52

3

44

min

marked

4

Fibonacci Heaps: Implementation Implementation. � Represent trees using left-child, right sibling pointers and circular,

doubly linked list. – can quickly splice off subtrees

� Roots of trees connected with circular doubly linked list. – fast union

� Pointer to root of tree with min element. – fast find-min

7 23

30

17

35

26 46

24

H 39

41 18 52

3

44

min

5

Fibonacci Heaps: Potential Function Key quantities. � Degree[x] = degree of node x. � Mark[x] = mark of node x (black or gray). � t(H) = # trees. � m(H) = # marked nodes. � )(H) = t(H) + 2m(H) = potential function.

7 23

30

17

35

26 46

24

H

t(H) = 5, m(H) = 3 )(H) = 11

39

41 18 52

3

44

min degree = 3

6

Fibonacci Heaps: Insert Insert. � Create a new singleton tree. � Add to left of min pointer. � Update min pointer.

7 23

30

17

35

26 46

24

H 39

41 18 52

3

44

min 21

Insert 21

7


39

41

7 23

18 52

3

30

17

35

26 46

24

44

min

H

21

Insert 21

8


Running time. O(1) amortized � Actual cost = O(1). � Change in potential = +1. � Amortized cost = O(1).

39

41

7

18 52

3

30

17

35

26 46

24

44

min

H

21 23

Insert 21

9

Fibonacci Heaps: Union Union. � Concatenate two Fibonacci heaps. � Root lists are circular, doubly linked lists.

39

41

7 17

18 52

3

30

23

35

26 46

24

44

min

H' H''

21

min

10

Fibonacci Heaps: Union Union. � Concatenate two Fibonacci heaps. � Root lists are circular, doubly linked lists.

Running time. O(1) amortized � Actual cost = O(1). � Change in potential = 0. � Amortized cost = O(1).

39

41

7 17

18 52

3

30

23

35

26 46

24

44

min

H' H''

21

11

Fibonacci Heaps: Delete Min Delete min. � Delete min and concatenate its children into root list. � Consolidate trees so that no two roots have same degree.

39

41 18 52

3

44

min

17 23

30

7

35

26 46

24

12


39

41 17 23 18 52

30

7

35

26 46

24

44

current min

13


39

41 17 23 18 52

30

7

35

26 46

24

44

current

0 1 2 3

min

14


39

41 17 23 18 52

30

7

35

26 46

24

44

current

0 1 2 3

min

15


39

41 17 23 18 52

30

7

35

26 46

24

44 current

0 1 2 3

min

16


39

41 17 23 18 52

30

7

35

26 46

24

44 current

0 1 2 3

Merge 17 and 23 trees.

min

17


39

41 17

23

18 52

30

7

35

26 46

24

44

current

0 1 2 3


min

18


39

41 7

30

18 52

17

35

26 46

24

44

current

0 1 2 3

23 Merge 7 and 24 trees.

min

19


39

41 7

30

18 52

23

17

35

26 46

24 44

current

0 1 2 3

min

Oren Weimann

20


39

41 7

30

18 52

23

17

35

26 46

24 44

current

0 1 2 3

min

Oren Weimann

21


39

41 7

30

18 52

23

17

35

26 46

24 44

current

0 1 2 3

min

Oren Weimann

22


39

41 7

30

18 52

23

17

35

26 46

24 44

current

0 1 2 3


min

Oren Weimann

23


39 41

7

30

18 52

23

17

35

26 46

24

44

current

0 1 2 3

min

Oren Weimann

24


39 41

7

30

18 52

23

17

35

26 46

24

44

current

0 1 2 3

min

Oren Weimann

25


39 41

7

30

18 52

23

17

35

26 46

24

44

min

Stop.

Oren Weimann

26

Fibonacci Heaps: Delete Min Analysis Notation. � D(n) = max degree of any node in Fibonacci heap with n nodes. � t(H) = # trees in heap H. � )(H) = t(H) + 2m(H).

Actual cost. O(D(n) + t(H)) � O(D(n)) work adding min's children into root list and updating min.

– at most D(n) children of min node � O(D(n) + t(H)) work consolidating trees.

– work is proportional to size of root list since number of roots decreases by one after each merging

– d D(n) + t(H) - 1 root nodes at beginning of consolidation

Amortized cost. O(D(n)) � t(H') d D(n) + 1 since no two trees have same degree. � ')(H) d D(n) + 1 - t(H).

27

Fibonacci Heaps: Delete Min Analysis Is amortized cost of O(D(n)) good?

� Yes, if only Insert, Delete-min, and Union operations supported.

– in this case, Fibonacci heap contains only binomial trees since we only merge trees of equal root degree

– this implies D(n) d ¬log2 N¼

� Yes, if we support Decrease-key in clever way. – we'll show that D(n) d ¬logI N¼, where I is golden ratio – I2 = 1 + I – I = (1 + �5) / 2 = 1.618… – limiting ratio between successive Fibonacci numbers!

28

Decrease key of element x to k. � Case 0: min-heap property not violated.

– decrease key of x to k – change heap min pointer if necessary

24

46

17

30

23

7

88

26

21

52

39

18

41

38

Decrease 46 to 45. 72

Fibonacci Heaps: Decrease Key

45

35

min

29

Decrease key of element x to k. � Case 1: parent of x is unmarked.

– decrease key of x to k – cut off link between x and its parent – mark parent – add tree rooted at x to root list, updating heap min pointer

24

45

17

30

23

7

88

26

21

52

39

18

41

38



15

35

min

30



24

15

17

30

23

7

88

26

21

52

39

18

41

38


24


35

min

31



24 17

30

23

7

88

26

21

52

39

18

41

38

Decrease 45 to 15.

24


35

min 15

72

32

35

Decrease key of element x to k. � Case 2: parent of x is marked.

– decrease key of x to k – cut off link between x and its parent p[x], and add x to root list – cut off link between p[x] and p[p[x]], add p[x] to root list � If p[p[x]] unmarked, then mark it. � If p[p[x]] marked, cut off p[p[x]], unmark, and repeat.

24

15

17

30

23

7

88

26

21

52

39

18

41

38

Decrease 35 to 5.

72 24


5

min

33



24 17

30

23

7

26

21

52

39

18

41

38

Decrease 35 to 5.

24

5


88

parent marked

15

72

min

34



24

26

17

30

23

7

21

52

39

18

41

38

Decrease 35 to 5.

88 24

5


15

72

parent marked

min

35



26

17

30

23

7

21

52

39

18

41

38

Decrease 35 to 5.

88

5


15 24

72

min

36

Notation. � t(H) = # trees in heap H. � m(H) = # marked nodes in heap H. � )(H) = t(H) + 2m(H).

Actual cost. O(c) � O(1) time for decrease key. � O(1) time for each of c cascading cuts, plus reinserting in root list.

Amortized cost. O(1) � t(H') = t(H) + c � m(H') d m(H) - c + 2

– each cascading cut unmarks a node – last cascading cut could potentially mark a node

� ') d c + 2(-c + 2) = 4 - c.

Fibonacci Heaps: Decrease Key Analysis

37

Delete node x. � Decrease key of x to -f. � Delete min element in heap.

Amortized cost. O(D(n)) � O(1) for decrease-key. � O(D(n)) for delete-min. � D(n) = max degree of any node in Fibonacci heap.

Fibonacci Heaps: Delete

Oren Weimann

(not necessarily Min)

Oren Weimann

38

Fibonacci Heaps: Bounding Max Degree Definition. D(N) = max degree in Fibonacci heap with N nodes. Key lemma. D(N) d logI N, where I = (1 + �5) / 2. Corollary. Delete and Delete-min take O(log N) amortized time.

Lemma. Let x be a node with degree k, and let y1, . . . , yk denote the children of x in the order in which they were linked to x. Then:

Proof. � When yi is linked to x, y1, . . . , yi-1 already linked to x, � degree(x) = i - 1 � degree(yi) = i - 1 since we only link nodes of equal degree

� Since then, yi has lost at most one child – otherwise it would have been cut from x

� Thus, degree(yi) = i - 1 or i - 2

¯®

t�

t1if21if0

)(degreeiii

yi

39

Fibonacci Heaps: Bounding Max Degree Key lemma. In a Fibonacci heap with N nodes, the maximum degree of any node is at most logI N, where I = (1 + �5) / 2. Proof of key lemma. � For any node x, we show that size(x) t Idegree(x) .

– size(x) = # node in subtree rooted at x – taking base I logs, degree(x) d logI (size(x)) d logI N.

� Let sk be min size of tree rooted at any degree k node. – trivial to see that s0 = 1, s1 = 2 – sk monotonically increases with k

� Let x* be a degree k node of size sk, and let y1, . . . , yk be children in order that they were linked to x*.

Assume k t 2

¦�

¦�t

¦�t

¦�

�

�

2

0

22

2]deg[

2

2

2

2

)(2

*)(size

k

ii

k

ii

k

iy

k

ii

k

s

s

s

ysize

xs

i

≥

40

Fibonacci Facts Definition. The Fibonacci sequence is: � 1, 2, 3, 5, 8, 13, 21, . . . • Slightly nonstandard definition.

Fact F1. Fk t Ik, where I = (1 + �5) / 2 = 1.618… Fact F2. Consequence. sk t Fk t Ik. � This implies that size(x) t Idegree(x)

for all nodes x.

°̄

°®

t�

2ifFF1if20if1

F

2-k1-k

k

kkk

¦� t�

2

02,2For

k

iik F Fk

¦�

¦�t

¦�t

¦�

�

�

2

0

22

2]deg[

2

2

2

2

)(2

*)(size

k

ii

k

ii

k

iy

k

ii

k

s

s

s

ysize

xs

i

≥

41

Golden Ratio Definition. The Fibonacci sequence is: 1, 2, 3, 5, 8, 13, 21, . . . Definition. The golden ratio I = (1 + �5) / 2 = 1.618… � Divide a rectangle into a square and smaller rectangle such that the

smaller rectangle has the same ratio as original one.

Parthenon, Athens Greece

43

Fibonacci Numbers and Nature

Pinecone

Cauliflower

44

Fibonacci Proofs Fact F1. Fk t Ik. Proof. (by induction on k) � Base cases:

– F0 = 1, F1 = 2 t I. � Inductive hypotheses:

– Fk t Ik and Fk+1 t Ik+1 Fact F2. Proof. (by induction on k) � Base cases:

– F2 = 3, F3 = 5 � Inductive hypotheses:

¦� t�

2

02,2For

k

iik F Fk

2

2

112

)()1(

�

��

� �t�

k

k

k

kkkkk FFF

MMM

MMMM

I2 = I + 1

¦� �

2

02

k

iik FF ¦�

¦ ��

�

�

�

��

k

ik

k

iki

kkk

F

FF

FFF

0

2

01

12

2

2

45

On Complicated Algorithms "Once you succeed in writing the programs for [these] complicated algorithms, they usually run extremely fast. The computer doesn't need to understand the algorithm, its task is only to run the programs."

R. E. Tarjan

priority queues - university of haifaoren/ads/lecture 3 - binomial and fibonacci heaps.pdf · n $ b...

Documents