introduction to algorithmsdspace.mit.edu/bitstream/handle/1721.1/37150/6-046... · introduction to...

Post on 01-Jun-2020

23 Views

Category:

Documents

1 Downloads

Preview:

Click to see full reader

TRANSCRIPT

Introduction to Algorithms6.046J/18.401J

Prof. Charles E. Leiserson

LECTURE 13Graph algorithms• Graph representation• Minimum spanning treesGreedy algorithms• Optimal substructure• Greedy choice• Prim’s greedy MST

algorithm

Introduction to Algorithms October 27, 2004 L13.2© 2001–4 by Charles E. Leiserson

Graphs (review)Definition. A directed graph (digraph)G = (V, E) is an ordered pair consisting of• a set V of vertices (singular: vertex),• a set E ⊆ V × V of edges.In an undirected graph G = (V, E), the edge set E consists of unordered pairs of vertices.In either case, we have |E | = O(V 2). Moreover, if G is connected, then |E | ≥ |V | – 1, which implies that lg |E | = Θ(lgV). (Review CLRS, Appendix B.)

Introduction to Algorithms October 27, 2004 L13.3© 2001–4 by Charles E. Leiserson

Adjacency-matrix representation

The adjacency matrix of a graph G = (V, E), where V = {1, 2, …, n}, is the matrix A[1 . . n, 1 . . n]given by

A[i, j] = 1 if (i, j) ∈ E,0 if (i, j) ∉ E.

Introduction to Algorithms October 27, 2004 L13.4© 2001–4 by Charles E. Leiserson

Adjacency-matrix representation

The adjacency matrix of a graph G = (V, E), where V = {1, 2, …, n}, is the matrix A[1 . . n, 1 . . n]given by

A[i, j] = 1 if (i, j) ∈ E,0 if (i, j) ∉ E.

22 11

33 44

A 1 2 3 41234

0 1 1 00 0 1 00 0 0 00 0 1 0

Θ(V 2) storage ⇒ denserepresentation.

Introduction to Algorithms October 27, 2004 L13.5© 2001–4 by Charles E. Leiserson

Adjacency-list representationAn adjacency list of a vertex v ∈ V is the list Adj[v]of vertices adjacent to v.

22 11

33 44

Adj[1] = {2, 3}Adj[2] = {3}Adj[3] = {}Adj[4] = {3}

Introduction to Algorithms October 27, 2004 L13.6© 2001–4 by Charles E. Leiserson

Adjacency-list representationAn adjacency list of a vertex v ∈ V is the list Adj[v]of vertices adjacent to v.

22 11

33 44

Adj[1] = {2, 3}Adj[2] = {3}Adj[3] = {}Adj[4] = {3}

For undirected graphs, |Adj[v] | = degree(v).For digraphs, |Adj[v] | = out-degree(v).

Introduction to Algorithms October 27, 2004 L13.7© 2001–4 by Charles E. Leiserson

Adjacency-list representationAn adjacency list of a vertex v ∈ V is the list Adj[v]of vertices adjacent to v.

22 11

33 44

Adj[1] = {2, 3}Adj[2] = {3}Adj[3] = {}Adj[4] = {3}

For undirected graphs, |Adj[v] | = degree(v).For digraphs, |Adj[v] | = out-degree(v).Handshaking Lemma: ∑v∈V = 2 |E | for undirected graphs ⇒ adjacency lists use Θ(V + E) storage —a sparse representation (for either type of graph).

Introduction to Algorithms October 27, 2004 L13.8© 2001–4 by Charles E. Leiserson

Minimum spanning trees

Input: A connected, undirected graph G = (V, E)with weight function w : E → R.• For simplicity, assume that all edge weights are

distinct. (CLRS covers the general case.)

Introduction to Algorithms October 27, 2004 L13.9© 2001–4 by Charles E. Leiserson

Minimum spanning trees

Input: A connected, undirected graph G = (V, E)with weight function w : E → R.• For simplicity, assume that all edge weights are

distinct. (CLRS covers the general case.)

∑∈

=Tvu

vuwTw),(

),()( .

Output: A spanning tree T — a tree that connects all vertices — of minimum weight:

Introduction to Algorithms October 27, 2004 L13.10© 2001–4 by Charles E. Leiserson

Example of MST

6 125

14

3

8

10

15

9

7

Introduction to Algorithms October 27, 2004 L13.11© 2001–4 by Charles E. Leiserson

Example of MST

6 125

14

3

8

10

15

9

7

Introduction to Algorithms October 27, 2004 L13.12© 2001–4 by Charles E. Leiserson

Optimal substructureMST T:

(Other edges of Gare not shown.)

Introduction to Algorithms October 27, 2004 L13.13© 2001–4 by Charles E. Leiserson

u

vRemove any edge (u, v) ∈ T.

Optimal substructureMST T:

(Other edges of Gare not shown.)

Introduction to Algorithms October 27, 2004 L13.14© 2001–4 by Charles E. Leiserson

u

vRemove any edge (u, v) ∈ T.

Optimal substructureMST T:

(Other edges of Gare not shown.)

Introduction to Algorithms October 27, 2004 L13.15© 2001–4 by Charles E. Leiserson

u

vRemove any edge (u, v) ∈ T. Remove any edge (u, v) ∈ T. Then, T is partitioned into two subtrees T1 and T2.

T1

T2u

v

Optimal substructureMST T:

(Other edges of Gare not shown.)

Introduction to Algorithms October 27, 2004 L13.16© 2001–4 by Charles E. Leiserson

u

vRemove any edge (u, v) ∈ T. Remove any edge (u, v) ∈ T. Then, T is partitioned into two subtrees T1 and T2.

T1

T2u

v

Optimal substructureMST T:

(Other edges of Gare not shown.)

Theorem. The subtree T1 is an MST of G1 = (V1, E1), the subgraph of G induced by the vertices of T1:

V1 = vertices of T1,E1 = { (x, y) ∈ E : x, y ∈ V1 }.

Similarly for T2.

Introduction to Algorithms October 27, 2004 L13.17© 2001–4 by Charles E. Leiserson

Proof of optimal substructure

w(T) = w(u, v) + w(T1) + w(T2).Proof. Cut and paste:

If T1′were a lower-weight spanning tree than T1 for G1, then T ′ = {(u, v)} ∪ T1′ ∪ T2 would be a lower-weight spanning tree than T for G.

Introduction to Algorithms October 27, 2004 L13.18© 2001–4 by Charles E. Leiserson

Proof of optimal substructure

w(T) = w(u, v) + w(T1) + w(T2).Proof. Cut and paste:

If T1′were a lower-weight spanning tree than T1 for G1, then T ′ = {(u, v)} ∪ T1′ ∪ T2 would be a lower-weight spanning tree than T for G.

Do we also have overlapping subproblems?•Yes.

Introduction to Algorithms October 27, 2004 L13.19© 2001–4 by Charles E. Leiserson

Proof of optimal substructure

w(T) = w(u, v) + w(T1) + w(T2).Proof. Cut and paste:

If T1′were a lower-weight spanning tree than T1 for G1, then T ′ = {(u, v)} ∪ T1′ ∪ T2 would be a lower-weight spanning tree than T for G.

Great, then dynamic programming may work!•Yes, but MST exhibits another powerful property which leads to an even more efficient algorithm.

Do we also have overlapping subproblems?•Yes.

Introduction to Algorithms October 27, 2004 L13.20© 2001–4 by Charles E. Leiserson

Hallmark for “greedy” algorithms

Greedy-choice propertyA locally optimal choice

is globally optimal.

Introduction to Algorithms October 27, 2004 L13.21© 2001–4 by Charles E. Leiserson

Hallmark for “greedy” algorithms

Greedy-choice propertyA locally optimal choice

is globally optimal.

Theorem. Let T be the MST of G = (V, E), and let A ⊆ V. Suppose that (u, v) ∈ E is the least-weight edge connecting A to V – A. Then, (u, v) ∈ T.

Introduction to Algorithms October 27, 2004 L13.22© 2001–4 by Charles E. Leiserson

Proof of theoremProof. Suppose (u, v) ∉ T. Cut and paste.

∈ A∈ V – A

T:

u

v

(u, v) = least-weight edge connecting A to V – A

Introduction to Algorithms October 27, 2004 L13.23© 2001–4 by Charles E. Leiserson

Proof of theoremProof. Suppose (u, v) ∉ T. Cut and paste.

∈ A∈ V – A

T:

u

Consider the unique simple path from u to v in T.

(u, v) = least-weight edge connecting A to V – A

v

Introduction to Algorithms October 27, 2004 L13.24© 2001–4 by Charles E. Leiserson

Proof of theoremProof. Suppose (u, v) ∉ T. Cut and paste.

∈ A∈ V – A

T:

u(u, v) = least-weight edge connecting A to V – A

v

Consider the unique simple path from u to v in T. Swap (u, v) with the first edge on this path that connects a vertex in A to a vertex in V – A.

Introduction to Algorithms October 27, 2004 L13.25© 2001–4 by Charles E. Leiserson

Proof of theoremProof. Suppose (u, v) ∉ T. Cut and paste.

∈ A∈ V – A

T ′:

u(u, v) = least-weight edge connecting A to V – A

v

Consider the unique simple path from u to v in T. Swap (u, v) with the first edge on this path that connects a vertex in A to a vertex in V – A.A lighter-weight spanning tree than T results.

Introduction to Algorithms October 27, 2004 L13.26© 2001–4 by Charles E. Leiserson

Prim’s algorithmIDEA: Maintain V – A as a priority queue Q. Key each vertex in Q with the weight of the least-weight edge connecting it to a vertex in A.Q ← Vkey[v] ←∞ for all v ∈ Vkey[s] ← 0 for some arbitrary s ∈ Vwhile Q ≠ ∅

do u ← EXTRACT-MIN(Q)for each v ∈ Adj[u]

do if v ∈ Q and w(u, v) < key[v]then key[v] ← w(u, v) ⊳ DECREASE-KEY

π[v] ← u

At the end, {(v, π[v])} forms the MST.

Introduction to Algorithms October 27, 2004 L13.27© 2001–4 by Charles E. Leiserson

Example of Prim’s algorithm

∈ A∈ V – A

∞∞

∞∞ ∞∞

∞∞ 00

∞∞

∞∞

∞∞

6 125

14

3

8

10

15

9

7

Introduction to Algorithms October 27, 2004 L13.28© 2001–4 by Charles E. Leiserson

Example of Prim’s algorithm

∈ A∈ V – A

∞∞

∞∞ ∞∞

∞∞ 00

∞∞

∞∞

∞∞

6 125

14

3

8

10

15

9

7

Introduction to Algorithms October 27, 2004 L13.29© 2001–4 by Charles E. Leiserson

Example of Prim’s algorithm

∈ A∈ V – A

∞∞

∞∞ 77

∞∞ 00

1010

∞∞

1515

6 125

14

3

8

10

15

9

7

Introduction to Algorithms October 27, 2004 L13.30© 2001–4 by Charles E. Leiserson

Example of Prim’s algorithm

∈ A∈ V – A

∞∞

∞∞ 77

∞∞ 00

1010

∞∞

1515

6 125

14

3

8

10

15

9

7

Introduction to Algorithms October 27, 2004 L13.31© 2001–4 by Charles E. Leiserson

Example of Prim’s algorithm

∈ A∈ V – A

1212

55 77

∞∞ 00

1010

99

1515

6 125

14

3

8

10

15

9

7

Introduction to Algorithms October 27, 2004 L13.32© 2001–4 by Charles E. Leiserson

Example of Prim’s algorithm

∈ A∈ V – A

1212

55 77

∞∞ 00

1010

99

1515

6 125

14

3

8

10

15

9

7

Introduction to Algorithms October 27, 2004 L13.33© 2001–4 by Charles E. Leiserson

Example of Prim’s algorithm

∈ A∈ V – A

66

55 77

1414 00

88

99

1515

6 125

14

3

8

10

15

9

7

Introduction to Algorithms October 27, 2004 L13.34© 2001–4 by Charles E. Leiserson

Example of Prim’s algorithm

∈ A∈ V – A

66

55 77

1414 00

88

99

1515

6 125

14

3

8

10

15

9

7

Introduction to Algorithms October 27, 2004 L13.35© 2001–4 by Charles E. Leiserson

Example of Prim’s algorithm

∈ A∈ V – A

66

55 77

1414 00

88

99

1515

6 125

14

3

8

10

15

9

7

Introduction to Algorithms October 27, 2004 L13.36© 2001–4 by Charles E. Leiserson

Example of Prim’s algorithm

∈ A∈ V – A

66

55 77

33 00

88

99

1515

6 125

14

3

8

10

15

9

7

Introduction to Algorithms October 27, 2004 L13.37© 2001–4 by Charles E. Leiserson

Example of Prim’s algorithm

∈ A∈ V – A

66

55 77

33 00

88

99

1515

6 125

14

3

8

10

15

9

7

Introduction to Algorithms October 27, 2004 L13.38© 2001–4 by Charles E. Leiserson

Example of Prim’s algorithm

∈ A∈ V – A

66

55 77

33 00

88

99

1515

6 125

14

3

8

10

15

9

7

Introduction to Algorithms October 27, 2004 L13.39© 2001–4 by Charles E. Leiserson

Example of Prim’s algorithm

∈ A∈ V – A

66

55 77

33 00

88

99

1515

6 125

14

3

8

10

15

9

7

Introduction to Algorithms October 27, 2004 L13.40© 2001–4 by Charles E. Leiserson

Q ← Vkey[v] ←∞ for all v ∈ Vkey[s] ← 0 for some arbitrary s ∈ Vwhile Q ≠ ∅

do u ← EXTRACT-MIN(Q)for each v ∈ Adj[u]

do if v ∈ Q and w(u, v) < key[v]then key[v] ← w(u, v)

π[v] ← u

Analysis of Prim

Introduction to Algorithms October 27, 2004 L13.41© 2001–4 by Charles E. Leiserson

Q ← Vkey[v] ←∞ for all v ∈ Vkey[s] ← 0 for some arbitrary s ∈ Vwhile Q ≠ ∅

do u ← EXTRACT-MIN(Q)for each v ∈ Adj[u]

do if v ∈ Q and w(u, v) < key[v]then key[v] ← w(u, v)

π[v] ← u

Analysis of Prim

Θ(V)total

Introduction to Algorithms October 27, 2004 L13.42© 2001–4 by Charles E. Leiserson

Q ← Vkey[v] ←∞ for all v ∈ Vkey[s] ← 0 for some arbitrary s ∈ Vwhile Q ≠ ∅

do u ← EXTRACT-MIN(Q)for each v ∈ Adj[u]

do if v ∈ Q and w(u, v) < key[v]then key[v] ← w(u, v)

π[v] ← u

Analysis of Prim

|V |times

Θ(V)total

Introduction to Algorithms October 27, 2004 L13.43© 2001–4 by Charles E. Leiserson

Q ← Vkey[v] ←∞ for all v ∈ Vkey[s] ← 0 for some arbitrary s ∈ Vwhile Q ≠ ∅

do u ← EXTRACT-MIN(Q)for each v ∈ Adj[u]

do if v ∈ Q and w(u, v) < key[v]then key[v] ← w(u, v)

π[v] ← u

Analysis of Prim

degree(u)times

|V |times

Θ(V)total

Introduction to Algorithms October 27, 2004 L13.44© 2001–4 by Charles E. Leiserson

Handshaking Lemma ⇒Θ(E) implicit DECREASE-KEY’s.

Q ← Vkey[v] ←∞ for all v ∈ Vkey[s] ← 0 for some arbitrary s ∈ Vwhile Q ≠ ∅

do u ← EXTRACT-MIN(Q)for each v ∈ Adj[u]

do if v ∈ Q and w(u, v) < key[v]then key[v] ← w(u, v)

π[v] ← u

Analysis of Prim

degree(u)times

|V |times

Θ(V)total

Introduction to Algorithms October 27, 2004 L13.45© 2001–4 by Charles E. Leiserson

Handshaking Lemma ⇒Θ(E) implicit DECREASE-KEY’s.

Q ← Vkey[v] ←∞ for all v ∈ Vkey[s] ← 0 for some arbitrary s ∈ Vwhile Q ≠ ∅

do u ← EXTRACT-MIN(Q)for each v ∈ Adj[u]

do if v ∈ Q and w(u, v) < key[v]then key[v] ← w(u, v)

π[v] ← u

Analysis of Prim

degree(u)times

|V |times

Θ(V)total

Time = Θ(V)·TEXTRACT-MIN + Θ(E)·TDECREASE-KEY

Introduction to Algorithms October 27, 2004 L13.46© 2001–4 by Charles E. Leiserson

Analysis of Prim (continued)

Time = Θ(V)·TEXTRACT-MIN + Θ(E)·TDECREASE-KEY

Introduction to Algorithms October 27, 2004 L13.47© 2001–4 by Charles E. Leiserson

Analysis of Prim (continued)

Time = Θ(V)·TEXTRACT-MIN + Θ(E)·TDECREASE-KEY

Q TEXTRACT-MIN TDECREASE-KEY Total

Introduction to Algorithms October 27, 2004 L13.48© 2001–4 by Charles E. Leiserson

Analysis of Prim (continued)

Time = Θ(V)·TEXTRACT-MIN + Θ(E)·TDECREASE-KEY

Q TEXTRACT-MIN TDECREASE-KEY Total

array O(V) O(1) O(V2)

Introduction to Algorithms October 27, 2004 L13.49© 2001–4 by Charles E. Leiserson

Analysis of Prim (continued)

Time = Θ(V)·TEXTRACT-MIN + Θ(E)·TDECREASE-KEY

Q TEXTRACT-MIN TDECREASE-KEY Total

array O(V) O(1) O(V2)binary heap O(lg V) O(lg V) O(E lg V)

Introduction to Algorithms October 27, 2004 L13.50© 2001–4 by Charles E. Leiserson

Analysis of Prim (continued)

Time = Θ(V)·TEXTRACT-MIN + Θ(E)·TDECREASE-KEY

Q TEXTRACT-MIN TDECREASE-KEY Total

array O(V) O(1) O(V2)binary heap O(lg V) O(lg V) O(E lg V)

Fibonacci heap

O(lg V)amortized

O(1)amortized

O(E + V lg V)worst case

Introduction to Algorithms October 27, 2004 L13.51© 2001–4 by Charles E. Leiserson

MST algorithms

Kruskal’s algorithm (see CLRS):• Uses the disjoint-set data structure (Lecture 10).• Running time = O(E lg V).

Introduction to Algorithms October 27, 2004 L13.52© 2001–4 by Charles E. Leiserson

MST algorithms

Kruskal’s algorithm (see CLRS):• Uses the disjoint-set data structure (Lecture 10).• Running time = O(E lg V).

Best to date:• Karger, Klein, and Tarjan [1993].• Randomized algorithm.• O(V + E) expected time.

top related