1 greedy algorithms and mst dr. ying lu [email protected] raik 283 data structures & algorithms

1

Greedy Algorithms and MST

Dr. Ying [email protected]

RAIK 283Data Structures & Algorithms

2

RAIK 283Data Structures & Algorithms

Giving credit where credit is due:» Most of slides for this lecture are based on slides

created by Dr. David Luebke, University of Virginia.

» Some examples and slides are based on lecture notes created by Dr. Chuck Cusack, Hope College, Dr. Jim Cohoon, University of Virginia, and Dr. Ben Choi, Louisiana Technical University.

» I have modified them and added new slides

3

Greedy Algorithms

Main Concept: Make the best or greedy choice at any given step.

Choices are made in sequence such that» Each individual choice is best according to some limited “short-

term” criterion, that is not too expensive to evaluate

» Once a choice is made, it cannot be undone! Even if it becomes evident later that it was a poor choice Sometimes life is like that

The goal is to » take a sequence of locally optimal actions, hoping to yield a

globally optimal solution.

4

When to be Greedy

Greedy algorithms apply to problems with» Optimal substructures: optimal solutions contain optimal sub-

solutions.

» The greedy choice property: an optimal solution can be obtained by making the greedy choice at each step.

» Many optimal solutions, but we only need one such solution.

5

Minimum Spanning Tree (MST)

A spanning tree for a connected, undirected graph, G=(V, E) is

1. a connected subgraph of G that forms an

2. undirected tree incident with each vertex.

In a weighted graph G=(V, E, W), » the weight of a subgraph is the sum of the weights of

the edges in the subgraph.

A minimum spanning tree (MST) for a weighted graph is

» a spanning tree with the minimum weight.

6

Minimum Spanning Tree

Problem: given a connected, undirected, weighted graph:

1410

3

6 4

5

2

9

15

8

find a spanning tree using edges that minimize the total weight

7


Which edges form the minimum spanning tree (MST) of the graph?

H B C

G E D

F

A

1410

3

6 4

5

2

9

15

8

8


Answer:

H B C

G E D

F

A

1410

3

6 4

5

2

9

15

8

9

Another Example

Given a weighted graph G=(V, E,W), find a MST of G

3

64

5

6

3

3

5

7

2

10

Finding a MST

MSTs satisfy the optimal substructure property: an optimal tree is composed of optimal subtrees

Principal greedy methods: algorithms by Prim and Kruskal Prim

» Grow a single tree by repeatedly adding the least cost edge that connects a vertex in the existing tree to a vertex not in the existing tree

Intermediary solution is a subtree

Kruskal» Grow a tree by repeatedly adding the least cost edge that does not

introduce a cycle among the edges included so far Intermediary solution is a spanning forest

11

Prime’s Algorithm (High-Level Pseudocode)

Prime(G)

//Input: A weighted connected graph G = <V, E>

//Output: ET --- the set of edges composing MST of G

VT = {v0}

ET =

for i = 1 to |V| - 1 do

find a minimum-weight edge e* = (u*, v*) among all the edges (u, v) such that u is in VT and v is in V-VT

VT = VT {v*}

ET = ET {e*}

return ET

12


Prime(G)



VT = {v0}

ET =

for i = 1 to |V| - 1 do


VT = VT {v*}

ET = ET {e*}

return ET

H B C

G E D

F

A

1410

3

6 45

2

9

15

8

13


Prime(G)



VT = {v0}

ET =

for i = 1 to |V| - 1 do


VT = VT {v*}

ET = ET {e*}

return ET

H B C

G E D

F

A

1410

3

6 45

2

9

15

8

14

Prim’s Algorithm

MST-Prim(G, w, r) Q = V[G]; for each u Q key[u] = ; key[r] = 0; p[r] = NULL; while (Q not empty) u = ExtractMin(Q); for each v Adj[u] if (v Q and w(u,v) < key[v]) p[v] = u; key[v] = w(u,v);

Grow a single tree by repeatedly adding the least cost edge that connects a vertex in the existing tree to a vertex not in the existing tree


15

Prim’s Algorithm


1410

3

6 45

2

9

15

8

Run on example graph

16

Prim’s Algorithm


1410

3

6 45

2

9

15

8

Run on example graph

17

Prim’s Algorithm


0

1410

3

6 45

2

9

15

8

Pick a start vertex r

r

18

Prim’s Algorithm


0

1410

3

6 45

2

9

15

8

Red vertices have been removed from Q

u

19

Prim’s Algorithm


0

3

1410

3

6 45

2

9

15

8

Red arrows indicate parent pointers

u

20

Prim’s Algorithm


14

0

3

1410

3

6 45

2

9

15

8

u

21

Prim’s Algorithm


14

0

3

1410

3

6 45

2

9

15

8u

22

Prim’s Algorithm


14

0 8

3

1410

3

6 45

2

9

15

8u

23

Prim’s Algorithm


10

0 8

3

1410

3

6 45

2

9

15

8u

24

Prim’s Algorithm


10

0 8

3

1410

3

6 45

2

9

15

8u

25

Prim’s Algorithm


10 2

0 8

3

1410

3

6 45

2

9

15

8u

26

Prim’s Algorithm


10 2

0 8 15

3

1410

3

6 45

2

9

15

8u

27

Prim’s Algorithm


10 2

0 8 15

3

1410

3

6 45

2

9

15

8

u

28

Prim’s Algorithm


10 2 9

0 8 15

3

1410

3

6 45

2

9

15

8

u

29

Prim’s Algorithm


10 2 9

0 8 15

3

4

1410

3

6 45

2

9

15

8

u

30

Prim’s Algorithm


5 2 9

0 8 15

3

4

1410

3

6 45

2

9

15

8

u

31

Prim’s Algorithm


5 2 9

0 8 15

3

4

1410

3

6 45

2

9

15

8

u

32

Prim’s Algorithm


5 2 9

0 8 15

3

4

1410

3

6 45

2

9

15

8

u

33

Prim’s Algorithm


5 2 9

0 8 15

3

4

1410

3

6 45

2

9

15

8

u

34

Prim’s Algorithm


5 2 9

0 8 15

3

4

1410

3

6 45

2

9

15

8

u

35

Review: Prim’s Algorithm


What is the hidden cost in this code?

36


MST-Prim(G, w, r) Q = V[G]; for each u Q key[u] = ; key[r] = 0; p[r] = NULL; while (Q not empty) u = ExtractMin(Q); for each v Adj[u] if (v Q and w(u,v) < key[v]) p[v] = u; DecreaseKey(v, w(u,v));

delete the smallest element from the min-heap

decrease an element’s value in the min-heap

(outline an efficient algorithm for it)

37


MST-Prim(G, w, r) Q = V[G]; for each u Q key[u] = ; key[r] = 0; p[r] = NULL; while (Q not empty) u = ExtractMin(Q); for each v Adj[u] if (v Q and w(u,v) < key[v]) p[v] = u; DecreaseKey(v, w(u,v));

How often is ExtractMin() called?How often is DecreaseKey() called?

38



What will be the running time?

There are n=|V| ExtractMin calls and m=|E| DecreaseKey calls.

The priority Q implementation has a large impact on

performance.

E.g., O((n+m)lg n) = O(m lg n) using min-heap for Q(for connected graph, n – 1 m)

39

Finding a MST

Principal greedy methods: algorithms by Prim and Kruskal

Prim» Grow a single tree by repeatedly adding the least cost edge that

connects a vertex in the existing tree to a vertex not in the existing tree


Kruskal» Grow a tree by repeatedly adding the least cost edge that does not

introduce a cycle among the edges included so far Intermediary solution is a spanning forest

40

MST Applications?

H B C

G E D

F

A

1410

3

6 4

5

2

9

15

8

41

MST Applications

Network design» telephone, electrical power, hydraulic, TV cable, computer,

road

MST gives a minimum-cost network connecting all sites

MST: the most economical construction of a network

42

Kruskal’s Algorithm (High-Level Pseudocode)

Kruskal(G)//Input: A weighted connected graph G = <V, E>

//Output: ET --- the set of edges composing MST of GSort E in nondecreasing order of the edge weight

ET = ; encounter = 0 //initialize the set of tree edges and its size

k = 0 //initialize the number of processed edgeswhile encounter < |V| - 1

k = k+1

if ET {ek} is acyclic

ET = ET {ek}; encounter = encounter + 1

return ET

43

Kruskal’s Algorithm (High-Level Pseudocode)

Kruskal(G)//Input: A weighted connected graph G = <V, E>

//Output: ET --- the set of edges composing MST of GSort E in nondecreasing order of the edge weight

ET = ; encounter = 0 //initialize the set of tree edges and its size

k = 0 //initialize the number of processed edgeswhile encounter < |V| - 1

k = k+1

if ET {ek} is acyclic

ET = ET {ek}; encounter = encounter + 1

return ET

H B C

G E D

F

A

1410

3

6 45

2

9

15

8

44

Kruskal’s Algorithm

Kruskal()

{

T = ; for each v V MakeSet(v);

sort E by increasing edge weight w

for each (u,v) E (in sorted order) if FindSet(u) FindSet(v) T = T U {{u,v}}; Union(FindSet(u), FindSet(v));

}

Grow a tree by repeatedly adding the least cost edge that does not introduce a cycle among the edges included so far

Intermediary solution is a spanning forest

45


Kruskal()

{




}

2 19

9

1

5

13

1725

148

21

Run the algorithm:

46


Kruskal()

{




}

2 19

9

1

5

13

1725

148

21

Run the algorithm:

47


Kruskal()

{




}

2 19

9

1

5

13

1725

148

21

Run the algorithm:

48


Kruskal()

{




}

2 19

9

1?

5

13

1725

148

21

Run the algorithm:

49


Kruskal()

{




}

2 19

9

1

5

13

1725

148

21

Run the algorithm:

50


Kruskal()

{




}

2? 19

9

1

5

13

1725

148

21

Run the algorithm:

51


Kruskal()

{




}

2 19

9

1

5

13

1725

148

21

Run the algorithm:

52


Kruskal()

{




}

2 19

9

1

5?

13

1725

148

21

Run the algorithm:

53


Kruskal()

{




}

2 19

9

1

5

13

1725

148

21

Run the algorithm:

54


Kruskal()

{




}

2 19

9

1

5

13

1725

148?

21

Run the algorithm:

55


Kruskal()

{




}

2 19

9

1

5

13

1725

148

21

Run the algorithm:

56


Kruskal()

{




}

2 19

9?

1

5

13

1725

148

21

Run the algorithm:

57


Kruskal()

{




}

2 19

9

1

5

13

1725

148

21

Run the algorithm:

58


Kruskal()

{




}

2 19

9

1

5

13?

1725

148

21

Run the algorithm:

59


Kruskal()

{




}

2 19

9

1

5

13

1725

148

21

Run the algorithm:

60


Kruskal()

{




}

2 19

9

1

5

13

1725

14?8

21

Run the algorithm:

61


Kruskal()

{




}

2 19

9

1

5

13

1725

148

21

Run the algorithm:

62


Kruskal()

{




}

2 19

9

1

5

13

17?25

148

21

Run the algorithm:

63


Kruskal()

{




}

2 19?

9

1

5

13

1725

148

21

Run the algorithm:

64


Kruskal()

{




}

2 19

9

1

5

13

1725

148

21?

Run the algorithm:

65


Kruskal()

{




}

2 19

9

1

5

13

1725?

148

21

Run the algorithm:

66


Kruskal()

{




}

2 19

9

1

5

13

1725

148

21

Run the algorithm:

67


Kruskal()

{




}

2 19

9

1

5

13

1725

148

21

Run the algorithm:

68


Kruskal()

{



for each (u,v) E (in sorted order) if FindSet(u) FindSet(v) T = T U {{u,v}};

Union(FindSet(u), FindSet(v));

}

What will affect the running time?Let n=|V| and m=|E|

O(n) MakeSet() calls1 Sort

O(m) FindSet() callsO(n) Union() calls

69

Kruskal’s Algorithm: Running Time

To summarize: » Sort edges: O(m lg m) » O(n) MakeSet()’s» O(m) FindSet()’s» O(n) Union()’s

Upshot: » Best disjoint subsets union-find algorithm makes above 3

operations only slightly worse than linear» Refer to the textbook for a discussion on the algorithms» Overall thus O(m lg m)

70

In-Class Exercise (9.1.7)

71

In-Class Exercise

Applying Kruskal’s algorithm to 9.2.1 (b)

1 greedy algorithms and mst dr. ying lu [email protected] raik 283 data structures & algorithms

Documents