umass lowell computer science 91.503 analysis of algorithms prof. karen daniels spring, 2005

UMass Lowell Computer Science 91.503

Analysis of Algorithms Prof. Karen Daniels

Spring, 2005

UMass Lowell Computer Science 91.503

Analysis of Algorithms Prof. Karen Daniels

Spring, 2005

Lecture 4Lecture 4Tuesday, 2/15/05Tuesday, 2/15/05

Graph Algorithms: Part 1Graph Algorithms: Part 1Shortest PathsShortest PathsChapters 24-25Chapters 24-25

91.404 Graph Review 91.404 Graph Review

Elementary Graph AlgorithmsElementary Graph Algorithms

Minimum Spanning TreesMinimum Spanning Trees

Single-Source Shortest PathsSingle-Source Shortest Paths

Introductory Graph ConceptsIntroductory Graph Concepts

G= (V,E)G= (V,E) Vertex DegreeVertex Degree Self-LoopsSelf-Loops

B

E

C

FD

A B

E

C

FD

A

Directed Graph (digraph)Directed Graph (digraph) Degree: in/outDegree: in/out Self-Loops allowedSelf-Loops allowed

Undirected GraphUndirected Graph No Self-LoopsNo Self-Loops Adjacency is symmetricAdjacency is symmetric

This treatment follows 91.503 textbook Cormen et al. Some definitions differ slightly from other graph literature.This treatment follows 91.503 textbook Cormen et al. Some definitions differ slightly from other graph literature.

Introductory Graph Concepts:RepresentationsIntroductory Graph Concepts:Representations

B

E

C

FD

A B

E

C

FD

A

Undirected GraphUndirected Graph Directed Graph (digraph)Directed Graph (digraph)

0 1 1 0 0 0

0 0 1 0 1 1

0 0 0 0 0 0

0 0 0 1 0 0

0 1 0 1 0 0

0 0 0 0 1 0

A B C D E FA B C D E FA A B B C C D D E E FF

0 1 1 0 0 0

1 0 1 0 1 1

1 1 0 0 0 0

0 0 0 0 1 0

0 1 0 1 0 1

0 1 0 0 1 0

A B C D E FA B C D E FA A B B C C D D E E FF

A BC A BC B ACEFB ACEFC ABC ABD ED EE BDFE BDFF BEF BE

A BC A BC B CEFB CEFC C D DD DE BDE BDF EF E

Adjacency Adjacency MatrixMatrix

Adjacency ListAdjacency List Adjacency Adjacency MatrixMatrix

Adjacency ListAdjacency ListThis treatment follows 91.503 textbook Cormen et al. Some definitions differ slightly from other graph literature.This treatment follows 91.503 textbook Cormen et al. Some definitions differ slightly from other graph literature.

Introductory Graph Concepts:Paths, CyclesIntroductory Graph Concepts:Paths, Cycles

Path:Path: length: number of edgeslength: number of edges simple: all vertices distinctsimple: all vertices distinct

Cycle: Cycle: Directed Graph: Directed Graph:

<v<v00,v,v11,...,v,...,vkk > forms cycle if v > forms cycle if v00=v=vkk and and k>=1k>=1

simple cycle: vsimple cycle: v11,v,v22..,v..,vk k also distinctalso distinct self-loop is cycle of length 1self-loop is cycle of length 1

Undirected Graph: Undirected Graph: <v<v00,v,v11,...,v,...,vkk > forms (simple) cycle if > forms (simple) cycle if

vv00=v=vkk and k>=3 and k>=3 simple cycle: vsimple cycle: v11,v,v22..,v..,vk k also distinctalso distinct

B

E

C

FD

A path <A,B,F>path <A,B,F>

B

E

C

FD

A simple cycle simple cycle <E,B,F,E><E,B,F,E>


B

E

C

FD

Asimple cycle <A,B,C,A>= <B,C,A,B>simple cycle <A,B,C,A>= <B,C,A,B>

most of most of our cycle our cycle work will work will be for be for directed directed graphsgraphs

Introductory Graph Concepts:ConnectivityIntroductory Graph Concepts:Connectivity

Undirected GraphUndirected Graph: : connectedconnected every pair of vertices is connected by a pathevery pair of vertices is connected by a path one one connected componentconnected component connected components:connected components:

equivalence classes under “is reachable from” equivalence classes under “is reachable from” relation relation

Directed GraphDirected Graph: : strongly connectedstrongly connected every pair of vertices is reachable from each every pair of vertices is reachable from each

otherother one one stronglystrongly connected componentconnected component strongly connected components:strongly connected components:

equivalence classes under “mutually reachable” equivalence classes under “mutually reachable” relation relation

B

E

C

FD

A

B

E

C

FD

A

connectedconnected

2 connected 2 connected componentscomponents

not not strongly strongly connectedconnected

strongly strongly connected connected componentcomponent

B

E

C

FD

A

B

E

C

FD

A


Elementary Graph Algorithms:SEARCHINGSEARCHING: DFS, BFSElementary Graph Algorithms:SEARCHINGSEARCHING: DFS, BFS

Breadth-First-Search (BFS):Breadth-First-Search (BFS): Shortest Path DistanceShortest Path Distance

From From sourcesource to each reachable vertex to each reachable vertex Record during traversalRecord during traversal

Foundation of many “Foundation of many “shortest pathshortest path” ” algorithmsalgorithms

See DFS, BFS Handout for PseudoCodeSee DFS, BFS Handout for PseudoCode

Depth-First-Search (DFS):Depth-First-Search (DFS): Encountering, finishing times Encountering, finishing times

“ “well-formed” nested (( )( ) ) structure well-formed” nested (( )( ) ) structure Every edge of undirected G is either a tree edge Every edge of undirected G is either a tree edge

or a back edgeor a back edge EdgeColor of vertex when first tested EdgeColor of vertex when first tested

determines edge type determines edge type

for for unweighted unweighted directeddirected or or undirectedundirected graph G=(V,E) graph G=(V,E)TimeTime: O(|V| + |E|) adj list: O(|V| + |E|) adj list O(|V|O(|V|22) adj matrix) adj matrix

predecessor subgraph = forest of spanning treespredecessor subgraph = forest of spanning trees

Vertex color shows status:Vertex color shows status:not yet encounterednot yet encountered

encountered, but not yet finishedencountered, but not yet finished

finishedfinished See 91.404 DFS/BFS slide showSee 91.404 DFS/BFS slide show

Elementary Graph Algorithms:DFS, BFSElementary Graph Algorithms:DFS, BFS

Review problem: TRUE or FALSE?Review problem: TRUE or FALSE? The tree shown below on the right can be a DFS tree for some The tree shown below on the right can be a DFS tree for some

adjacency list representation of the graph shown below on the adjacency list representation of the graph shown below on the

left.left. B

E

C

F

D

A

A

C

B

E

D

F

Tree Edge

Tree Edge

Tree Edge

Tree Edge

Tree Edge

Cross Edge

Back Edge

Elementary Graph Algorithms:Topological SortElementary Graph Algorithms:Topological Sort

source: 91.503 textbook Cormen et al.source: 91.503 textbook Cormen et al.

TOPOLOGICAL-SORT(G)TOPOLOGICAL-SORT(G)1 DFS(G) computes “finishing times” for each vertex1 DFS(G) computes “finishing times” for each vertex2 as each vertex is finished, insert it onto front of list2 as each vertex is finished, insert it onto front of list3 return list3 return list

for Directed, Acyclic Graph (DAG) G=(V,E)

Produces linear ordering of vertices.Produces linear ordering of vertices.For edge (u,v), u is ordered before v.For edge (u,v), u is ordered before v.

See also 91.404 DFS/BFS slide showSee also 91.404 DFS/BFS slide show

Minimum Spanning Tree:Greedy Algorithms Minimum Spanning Tree:Greedy Algorithms


for Undirected, Connected, Weighted Graph

G=(V,E)

A B

CD

E

F

G

2

2

1 1

3

4

4

5 6

68

7

Produces minimum weight tree of Produces minimum weight tree of edges that includes every vertex.edges that includes every vertex.

Invariant: Minimum weight spanning forest

Becomes single tree at end

Invariant: Minimum weight tree

Spans all vertices at end

TimeTime: O(|E|: O(|E|lg|E|) given lg|E|) given fast FIND-fast FIND-SET, SET, UNIONUNION

TimeTime: O(|E|: O(|E|lg|V|) = O(|lg|V|) = O(|E|lg|E|) E|lg|E|) slightly slightly faster with faster with fast priority fast priority queuequeue

Minimum Spanning TreesMinimum Spanning Trees

Review problem:Review problem: For the undirected, weighted graph below, show 2 different For the undirected, weighted graph below, show 2 different

Minimum Spanning Trees. Draw each using one of the 2 graph Minimum Spanning Trees. Draw each using one of the 2 graph copies below. Thicken an edge to make it part of a spanning copies below. Thicken an edge to make it part of a spanning tree. What is the sum of the edge weights for each of your tree. What is the sum of the edge weights for each of your Minimum Spanning Trees?Minimum Spanning Trees?

A B

CD

E

F

G

2

2

1 1

3

4

4

5 6

68

7

Shortest Paths Shortest Paths

Chapters 24 & 25Chapters 24 & 25

BFS as a Basis for Some Shortest Path AlgorithmsBFS as a Basis for Some Shortest Path Algorithms

Source/Sink Source/Sink Shortest Path Shortest Path

ProblemProblem: Given 2 vertices : Given 2 vertices u, v, find the shortest path u, v, find the shortest path in G from u to v.in G from u to v.

SolutionSolution: BFS : BFS starting at u. starting at u. Stop at v.Stop at v.

Single-Source Single-Source Shortest Paths Shortest Paths

ProblemProblem: Given a vertex : Given a vertex u, find the shortest path in u, find the shortest path in G from u to each vertex.G from u to each vertex.

SolutionSolution: BFS : BFS starting at u. Full starting at u. Full BFS tree.BFS tree.

source: based on Sedgewick, Graph Algorithmssource: based on Sedgewick, Graph Algorithms

BFSBFSfor for unweighted,unweighted, undirected undirected graph G=(V,E)graph G=(V,E)

TimeTime: O(|V| + |E|) adj list: O(|V| + |E|) adj list

O(|V|O(|V|22) adj matrix) adj matrix

TimeTime: O(|V|(|V| + |E|)) adj list: O(|V|(|V| + |E|)) adj list

O(|V|O(|V|33) adj matrix) adj matrix

All-Pairs All-Pairs Shortest Paths Shortest Paths

ProblemProblem: Find the shortest : Find the shortest path in G from each vertex u path in G from each vertex u to each vertex v.to each vertex v.

SolutionSolution: : For each uFor each u: : BFS starting at u; full BFS starting at u; full BFS tree.BFS tree.

but for weighted, directed graphs…but for weighted, directed graphs…

Shortest Path ApplicationsShortest Path Applications

Road mapsRoad maps Airline routesAirline routes Telecommunications network routingTelecommunications network routing VLSI design routingVLSI design routing

Weight ~ Cost ~ DistanceWeight ~ Cost ~ Distance

source: based on Sedgewick, Graph Algorithmssource: based on Sedgewick, Graph Algorithms

for for weighted,weighted, directed directed graph G=(V,E)graph G=(V,E)

Transitive Closure (Matrix):Unweighted, Directed GraphTransitive Closure (Matrix):Unweighted, Directed Graph

source: Sedgewick, Graph Algorithmssource: Sedgewick, Graph Algorithms

Transitive Closure GraphTransitive Closure Graph contains edge (u,v) if there contains edge (u,v) if there exists a directed path in G exists a directed path in G from u to v.from u to v.

G

Transitive Closure Transitive Closure concepts will be useful for concepts will be useful for All-Pairs Shortest Path All-Pairs Shortest Path calculation in directed, calculation in directed, weighted graphsweighted graphs

“self-loops” added for algorithmic purposes

Transitive Closure (Matrix)Transitive Closure (Matrix)


Boolean Matrix ProductBoolean Matrix Product: : and, orand, or replace replace *,+*,+

G G2

2

“self-loops” added for algorithmic purposes



2

3

4

Algorithm 1Algorithm 1: Find : Find , , , , ,..., ,..., V-1V-1

Algorithm 2Algorithm 2: Find : Find , , , , ,..., ,..., VV

Algorithm 3Algorithm 3: [Warshall]: [Warshall]for i 0 to |V|-1for i 0 to |V|-1 for s 0 to |V|-1for s 0 to |V|-1 for t 0 to |V|-1for t 0 to |V|-1

if if [s][i] and [s][i] and [i][t][i][t]then then [s][t] 1[s][t] 1

TimeTime: O(|V|: O(|V|33))


TimeTime: O(|V|: O(|V|33lg|V|)lg|V|)

why this upper limit?why this upper limit?



Warshall

good for good for dense dense graphsgraphs

Transitive Closure (Matrix)Transitive Closure (Matrix)Warshall

Correctness by Induction on i:Inductive Hypothesis: ith iteration of loop sets [s][t] to 1 iff there’s a directed path from s to t with (internal) indices at most i.Inductive Step for i+1 (sketch): 2 cases for path <s…t>-internal indices at most i - covered by inductive hypothesis in prior iteration so

[s][t] already set to 1-an internal index exceeds i (= i+1) - [s][i+1], [i+1][t] set in a prior iteration so

[s][t] set to 1 in current iterationsource: Sedgewick, Graph Algorithmssource: Sedgewick, Graph Algorithms

Shortest Path Trees


Shortest Path Tree gives shortest path from root to each other vertexShortest Path Tree gives shortest path from root to each other vertex

.99.99

.51.51

.5.5

.36.36

.38.38

.45.45

.83.83

.21.21 .1.1

.41.41

.1.1

.51.51

.51.51

.38.38

.45.45

.45.45.45.45.83.83 .83.83

.83.83

.1.1 .1.1 .1.1

.41.41 .41.41

.1.1.1.1

.1.1

shortest path need not be uniqueshortest path need not be unique

All Shortest Paths


Shortest Path Trees


predecessor vertex in treepredecessor vertex in tree

Shortest Path Tree is a spanning tree.Shortest Path Tree is a spanning tree.

3 as root for 3 as root for reversereverse graph graph

All Shortest Paths (Compact)


Entry s,t gives Entry s,t gives nextnext vertex on shortest vertex on shortest path from s to t.path from s to t.

Total distance of shortest pathTotal distance of shortest path Shortest PathsShortest Paths

All Shortest Paths In a Network


Shortest Path Trees for reverse graphShortest Path Trees for reverse graph

Shortest Path Principles: RelaxationShortest Path Principles: Relaxation

““Relax” a constraint to try to improve solutionRelax” a constraint to try to improve solution Relaxation of an Edge (u,v):Relaxation of an Edge (u,v):

test if shortest path to v test if shortest path to v [found so far][found so far] can be improved can be improved by going through uby going through u

A B

C D

E

F

G

2

2

1 1

3

4

4

5 6

68

7

Single Source Shortest Paths Bellman-FordSingle Source Shortest Paths Bellman-Ford


for for (negative)(negative) weighted,weighted, directed directed graph G=(V,E) graph G=(V,E) with no negative-weight cycleswith no negative-weight cycles

Time is in O(|V||E|)Time is in O(|V||E|)

detect negative-detect negative-weight cycleweight cycle

weightsweights

sourcesource

why this upper bound?why this upper bound?

update d(v) if d(u)+w(u,v) < d(v)update d(v) if d(u)+w(u,v) < d(v)

Bellman-FordBellman-Ford


for for (negative)(negative) weighted,weighted, directed directed graph G=(V,E)graph G=(V,E)with no negative-weight cycleswith no negative-weight cycles

Single Source Shortest Paths: Dijkstra’s Algorithm

1

2 3 4

6 5

101

5

4

3

31

2

61

1

8

Dijkstra’s algorithm maintains a set S of vertices whose final shortest path weights have already been determined.

It also maintains, for each vertex v not in S, an upper bound d[v] on the weight of a shortest path from source s to v.

Dijkstra’s algorithm solves problem efficiently for the case in which all weights are nonnegative (as in the example graph).

The algorithm repeatedly selects the vertex u V – S with minimum bound d[u], inserts u into S, and relaxes all edges leaving u (determines if passing through u makes it “faster” to get to a vertex adjacent to u).

Single Source Shortest Paths: Dijkstra’s AlgorithmSingle Source Shortest Paths: Dijkstra’s Algorithm


for for (nonnegative)(nonnegative) weighted,weighted, directed directed graph G=(V,E)graph G=(V,E)

implicit DECREASE-KEYimplicit DECREASE-KEY

Single Source Shortest Paths Dijkstra’s AlgorithmSingle Source Shortest Paths Dijkstra’s Algorithm

A B

C D

E

F

G

2

2

1 1

3

4

4

5 6

68

7



shortest path weightshortest path weight

shortest path weight estimateshortest path weight estimate


Review problem:Review problem: For the directed, weighted graph below, find the shortest

path that begins at vertex A and ends at vertex F. List the vertices in the order that they appear on that path. What is the sum of the edge weights of that path?

A B

C D

E

F

G

2

2

1 1

3

4

4

5 6

68

7

Why can’t Dijkstra’s algorithm handle negative-weight edges?Why can’t Dijkstra’s algorithm handle negative-weight edges?

All-Pairs Shortest PathsAll-Pairs Shortest Paths


for for (negative)(negative) weighted,weighted, directed directed graph G=(V,E)graph G=(V,E)with no negative-weight cycleswith no negative-weight cycles

similar to Transitive Closure Algorithm 1similar to Transitive Closure Algorithm 1


Note: D here is replaced by L in new editionNote: D here is replaced by L in new edition



similar to Transitive Closure Algorithm 2similar to Transitive Closure Algorithm 2

TimeTime: O(|V|: O(|V|33lg|V|)lg|V|)




How can output be How can output be used to detect a used to detect a negative-weight negative-weight

cycle? cycle?

Can have negative-Can have negative-weight edges weight edges

similar to Transitive similar to Transitive Closure Algorithm 3 Closure Algorithm 3 [Warshall][Warshall]



Food for thought…Food for thought…

What does the following matrix (the nxn form of it) What does the following matrix (the nxn form of it) used in shortest-path algorithms correspond to in used in shortest-path algorithms correspond to in regular matrix multiplication?regular matrix multiplication?

0

0

0)0(D


Shortest Path AlgorithmsShortest Path Algorithms


umass lowell computer science 91.503 analysis of algorithms prof. karen daniels spring, 2005

Documents

graph literature

connectivityundirected

undirected graph g

weighted graph g

acyclic graph dag g

forms cycle

forms simple cycle

textbook cormen et