1 cse 326: data structures: graphs lecture 22: monday, march 3 rd, 2003

1

CSE 326: Data Structures: Graphs

Lecture 22: Monday, March 3rd, 2003

2

Today

• Example of a formal correctness proof:– Dijkstra’s Algorithm

• All pairs shortest paths

• NP completeness– Will finish on Wednesday

3

Dijkstra’s Algorithm for Single Source Shortest Path

• Classic algorithm for solving shortest path in weighted graphs (with only positive edge weights)

• Similar to breadth-first search, but uses a priority queue instead of a FIFO queue:– Always select (expand) the vertex that has a lowest-

cost path to the start vertex – a kind of “greedy” algorithm

• Correctly handles the case where the lowest-cost (shortest) path to a vertex is not the one with fewest edges

4

void shortestPath(Node startNode) {Heap s = new Heap;for v in Nodes do

{ v.dist = ; s.insert(v); }

startNode.dist = 0; s.decreaseKey(startNode);startNode.previous = null;

while (!s.empty()) {x = s.deleteMin();for y in x.children() do

if (x.dist+c(x,y) < y.dist) { y.dist = x.dist+c(x,y); s.decreaseKey(y); y.previous = x;

}}

}

void shortestPath(Node startNode) {Heap s = new Heap;for v in Nodes do

{ v.dist = ; s.insert(v); }

startNode.dist = 0; s.decreaseKey(startNode);startNode.previous = null;

while (!s.empty()) {x = s.deleteMin();for y in x.children() do

if (x.dist+c(x,y) < y.dist) { y.dist = x.dist+c(x,y); s.decreaseKey(y); y.previous = x;

}}

}

Dijkstra’s Algorithm

5

Dijkstra’s Algorithm:Correctness Proof

Partition the set of all nodes, V, into two sets:• The heap, s• The rest of the nodes, known

V = s known

We prove the following Invariant:1. v known, v.dist = cost of shortest path startNode v 2. v known, v.dist = cost of shortest path startNode v

going only through nodes in Knownexcept for v itself

6

Claim 1

startNode

Known

v

Claim 1:if v known, then v.dist = cost of shortest path startNode v

s

7

Claim 2

startNode

Known

Claim 2:if v known, v.dist = cost of shortest path startNode v

going only through nodes in Knownexcept for v itself

v

s

8

Proof by Induction: Base Case

startNode

Known =

s

Need to checkClaim 2:

0

When v = startNode, then the only such paths is (startNode): has cost 0

When v startNode, then no such path exists: minimum cost is

9

void shortestPath(Node startNode) {Heap s = new Heap;for v in Nodes do

{ v.dist = ; s.insert(v); }

startNode.dist = 0; s.decreaseKey(startNode);startNode.previous = null;

while (!s.empty()) {x = s.deleteMin();for y in x.children() do

if (x.dist+c(x,y) < y.dist) { y.dist = x.dist+c(x,y); s.decreaseKey(y); y.previous = x;

}}

}

void shortestPath(Node startNode) {Heap s = new Heap;for v in Nodes do

{ v.dist = ; s.insert(v); }

startNode.dist = 0; s.decreaseKey(startNode);startNode.previous = null;

while (!s.empty()) {x = s.deleteMin();for y in x.children() do

if (x.dist+c(x,y) < y.dist) { y.dist = x.dist+c(x,y); s.decreaseKey(y); y.previous = x;

}}

}

Proof by Induction: Induction Step

Assume theinvariant holds

here

Prove theinvariant holds

here

10

Proof by Induction: Induction Step

startNode

Known

Need to check Claim 1 and/or Claim 2 in each of the following case:

v known v = xv known

x = deleteMin(s)

s

11

Case v = x

startNode

Known

Here we need to check Claim 2 (why ?)

Let startNode = x1, x2, ..., xk-1, xk=x be the shortest paths to x

Let xi be the first node s.t. xi known

Then x.dist xi.dist (why ?) cost(x1, x2, ..., xk-1, xk) (why ?)

By induction hypothesis there exists a path startNode xof cost = x.dist,and going only through known

It follows that that paths is also a shortestpath, and has cost = x.dist

s

x = deleteMin(s)

xi

12

Case v known

startNode

Known

x = deleteMin(s)

s

vHere we need to check Claim 1Follows trivially from the induction hypothesis

13

Case v known

startNode

Known

Here we need to check Claim 2 (why ?)

Let startNode = x1, x2, ..., xk-1, xk=v be the shortest paths to vthat goes only through known

Look at the last node, xk-1, on this path

Case 1: xk-1 x

Then v.dist = cost(x1, x2, ..., xk-1, xk)by induction hypothesis(why ?)

s

x = deleteMin(s)

v

14

Case v known

startNode

Known

Here we need to check Claim 2 (why ?)

Let startNode = x1, x2, ..., xk-1, xk=v be the shortest paths to vthat goes only through known

Look at the last node, xk-1, on this path

Case 2: xk-1 = x. Then the followinginstruction:

ensures that v.dist = cost(x1, x2, ..., xk-1, xk)

s

x = deleteMin(s)

v

if (x.dist+c(x,y) < y.dist) {y.dist = x.dist+c(x,y);}

if (x.dist+c(x,y) < y.dist) {y.dist = x.dist+c(x,y);}

15

End of Induction

startNode

Known

v

Use Claim 1:For every v, v known (because s = )

hence v.dist = cost of shortest path startNode v

s =

16

All Pairs Shortest Path

• Suppose you want to compute the length of the shortest paths between all pairs of vertices in a graph…– Run Dijkstra’s algorithm (with priority queue)

repeatedly, starting with each node in the graph:

– Complexity in terms of V when graph is dense:

17

Dynamic Programming Approach

,

, ,

1, 2

, , 1, , 1, ,

,

1, ,

Note that path for

distance from to that u

either does not use ,

or merges the paths

ses

only ,..., as intermediates

min{ , }

and

k i j

k i j i j

k

k i j

k

i

k i j k i k k k j

k k j

D v v

v v v

D D D D

D v

v v v v

Notice that Dk-1, i, k = Dk, i, k and Dk-1, k, j = Dk, k, j;hence we can use a single matrix, Di, j !

18

Floyd-Warshall Algorithm

// C – adjacency matrix representation of graph

// C[i][j] = weighted edge i->j or if none// D – computed distancesfor (i = 0; i < N; i++){ for (j = 0; j < N; j++) D[i][j] = C[i][j]; D[i][i] = 0.0; }for (k = 0; k < N; k++) for (i = 0; i < N; i++) for (j = 0; j < N; j++) if (D[i][k] + D[k][j] < D[i][j]) D[i][j] = D[i][k] + D[k][j];

// C – adjacency matrix representation of graph

// C[i][j] = weighted edge i->j or if none// D – computed distancesfor (i = 0; i < N; i++){ for (j = 0; j < N; j++) D[i][j] = C[i][j]; D[i][i] = 0.0; }for (k = 0; k < N; k++) for (i = 0; i < N; i++) for (j = 0; j < N; j++) if (D[i][k] + D[k][j] < D[i][j]) D[i][j] = D[i][k] + D[k][j];

Run time =

How could we compute the paths?

19

NP-Completeness

• Really hard problems

20

Today’s Agenda• Solving pencil-on-paper puzzles

– A “deep” algorithm for Euler Circuits

• Euler with a twist: Hamiltonian circuits

• Hamiltonian circuits and NP complete problems

• The NP =? P problem– Your chance to win a Turing award!– Any takers?

• Weiss Chapter 9.7

1ie

W. R. Hamilton

(1805-1865)

L. Euler(1707-1783)

21

It’s Puzzle Time!

Which of these can you draw without lifting your pencil, drawing each line only once?Can you start and end at the same point?

22

Historical Puzzle: Seven Bridges of Königsberg

KNEIPHOFF

PREGEL

Want to cross all bridges but…Can cross each bridge only once (High toll to cross twice?!)

23

A “Multigraph” for the Bridges of Königsberg

Find a path thattraverses every edgeexactly once

24

Euler Circuits and Tours• Euler tour: a path through a graph that visits each edge

exactly once• Euler circuit: an Euler tour that starts and ends at the same

vertex• Named after Leonhard Euler (1707-1783), who cracked this

problem and founded graph theory in 1736• Some observations for undirected graphs:

– An Euler circuit is only possible if the graph is connected and each vertex has even degree (= # of edges on the vertex) [Why?]

– An Euler tour is only possible if the graph is connected and either all vertices have even degree or exactly two have odd degree [Why?]

25

Euler Circuit Problem

• Problem: Given an undirected graph G = (V,E), find an Euler circuit in G

• Note: Can check if one exists in linear time (how?)

• Given that an Euler circuit exists, how do we construct an Euler circuit for G?

• Hint: Think deep! We’ve discussed the answer in depth before…

26

Finding Euler Circuits: DFS and then Splice

• Given a graph G = (V,E), find an Euler circuit in G– Can check if one exists in O(|V|) time

(check degrees)• Basic Euler Circuit Algorithm:

1. Do a depth-first search (DFS) from a vertex until you are back at this vertex

2. Pick a vertex on this path with an unused edge and repeat 1.

3. Splice all these paths into an Euler circuit

• Running time = O(|V| + |E|)

27

Euler Circuit ExampleA

B C

D E

F

B C

D E

G G

D E

G

DFS(A) :A B D F E C A

DFS(B) :B G C B

DFS(G) :G D E G

A B G C B D F E C A A B G D E G C B D F E C ASplice at B

Splice at G

28

Euler with a Twist: Hamiltonian Circuits

• Euler circuit: A cycle that goes through each edge exactly once

• Hamiltonian circuit: A cycle that goes through each vertex exactly once

• Does graph I have:– An Euler circuit?– A Hamiltonian circuit?

• Does graph II have:– An Euler circuit?– A Hamiltonian circuit?

B C

D E

G

B C

D E

G I

II

29

Finding Hamiltonian Circuits in Graphs

• Problem: Find a Hamiltonian circuit in a graph G = (V,E)– Sub-problem: Does G contain a Hamiltonian circuit?

– No known easy algorithm for checking this…

• One solution: Search through all paths to find one that visits each vertex exactly once– Can use your favorite graph search algorithm (DFS!) to find

various paths

• This is an exhaustive search (“brute force”) algorithm

• Worst case need to search all paths

– How many paths??

30

Analysis of our Exhaustive Search Algorithm

• Worst case need to search all paths

– How many paths?

• Can depict these paths as a search tree

• Let the average branching factor of each node in this tree be B

• |V| vertices, each with B branches

• Total number of paths B·B·B … ·B

= O(B|V|)

• Worst case Exponential time!

B C

D E

G

B

D G C

G E D E C G E

Etc.

Search tree of paths from B

31

How bad is exponential time?N log

NN log N N2 2N

1 0 0 1 2

2 1 2 4 4

4 2 8 16 16

10 3 30 100 1024

100 7 700 10,000 1,000,000,000,000,00,000,000

,000,000,000

1000 10 10,000 1,000,000 Fo’gettaboutit!

1,000,000 20 20,000,000 1,000,000,000,000 ditto

32

Review: Polynomial versus Exponential Time

• Most of our algorithms so far have been O(log N), O(N), O(N log N) or O(N2) running time for inputs of size N– These are all polynomial time algorithms– Their running time is O(Nk) for some k > 0

• Exponential time BN is asymptotically worse than any polynomial function Nk for any k– For any k, Nk is (BN) for any constant B > 1

33

The Complexity Class P• The set P is defined as the set of all problems

that can be solved in polynomial worse case time– Also known as the polynomial time

complexity class

– All problems that have some algorithm whose running time is O(Nk) for some k

• Examples of problems in P: tree search, sorting, shortest path, Euler circuit, etc.

34

The Complexity Class NP• Definition: NP is the set of all problems for

which a given candidate solution can be tested in polynomial time

• Example of a problem in NP:– Hamiltonian circuit problem: Why is it in

NP?

35

The Complexity Class NP• Definition: NP is the set of all problems for

which a given candidate solution can be tested in polynomial time

• Example of a problem in NP:– Hamiltonian circuit problem: Why is it in

NP?• Given a candidate path, can test in linear time if it

is a Hamiltonian circuit – just check if all vertices are visited exactly once in the candidate path (except start/finish vertex)

36

Why NP?• NP stands for Nondeterministic Polynomial time

– Why “nondeterministic”? Corresponds to algorithms that can guess a solution (if it exists) the solution is then verified to be correct in polynomial time

– Nondeterministic algorithms don’t exist – purely theoretical idea invented to understand how hard a problem could be

• Examples of problems in NP:– Hamiltonian circuit: Given a candidate path, can test in linear

time if it is a Hamiltonian circuit

– Sorting: Can test in linear time if a candidate ordering is sorted

– Are any other problems in P also in NP?

37

More Revelations About NP

• Are any other problems in P also in NP?

– YES! All problems in P are also in NP • Notation: P NP• If you can solve a problem in polynomial

time, can definitely verify a solution in polynomial time

• Question: Are all problems in NP also in P?

– Is NP P?

38

Your Chance to Win a Turing Award: P = NP?

• Nobody knows whether NP P– Proving or disproving this will bring you

instant fame!• It is generally believed that P NP, i.e.

there are problems in NP that are not in P– But no one has been able to show even one

such problem!– Practically all of modern complexity theory

is premised on the assumption that P NP

• A very large number of useful problems are in NP

Alan Turing(1912-1954)

39

NP-Complete Problems• The “hardest” problems in NP are called NP-complete

problems (NPC)

• Why “hardest”? A problem X is NP-complete iff:1. X is in NP and

2. Any problem Y in NP can be converted to an instance of X in polynomial time, such that solving X also provides a solution for Y

In other words: Can use algorithm for X as a subroutine to solve Y

• Thus, if you find a poly time algorithm for just one NPC problem, all problems in NP can be solved in poly time – E.g: The Hamiltonian circuit problem can be shown to be NP-

complete

40

Another NP-Complete Problem

• SAT: Given a formula in Boolean logic, e.g.

determine if there is an assignment of values to the variables that makes the formula true (=1).

• Why is it in NP?

a b a c b c

41

Why SAT is NP-Complete

• Cook (1971) showed that SAT could be used to simulate any non-deterministic Turing machine!

• Idea: consider the tree of possible execution states of the Turing machine– A Boolean logic formula can

represent this tree – the “state transition” function

– Formula also asserts the final state is one where a solution has been found

– “Guessed” variables determine which branch to take

42

P, NP, and Exponential Time Problems

• All currently known algorithms for NP-complete problems run in exponential worst case time– Finding a polynomial time

algorithm for any NPC problem would mean:

• Diagram depicts relationship between P, NP, and EXPTIME (class of problems that provably require exponential time to solve)

It is believed that P NP EXPTIME

EXPTIME

NP

P

NPC

43

The Graph of NP-Completeness• Stephen Cook first showed

(1971) that satisfiability of Boolean formulas (SAT) is NP-complete

• Hundreds of other problems (from scheduling and databases to optimization theory) have since been shown to be NPC

• How? By showing an algorithm that converts a known NPC problem to your pet problem in poly time then, your problem is also NPC!

44

Showing NP-completeness: An example

• Consider the Traveling Salesperson (TSP) Problem: Given a fully connected, weighted graph G = (V,E), is there a cycle that visits all vertices exactly once and has total cost K?

• TSP is in NP (why?)

• Can we show TSP is NP-complete?– Hamiltonian Circuit (HC) is NPC

– Can show TSP is also NPC if we can convert any input for HC to an input for TSP in poly time

B C

D E

G

Input for HC

B C

D E

Convertto inputfor TSP

Cyclewith cost 8 BDCEB

3

3

1

12

4

45

TSP is NP-complete!• We can show TSP is also NPC if we can convert any input for

HC to an input for TSP in polynomial time. Here’s one way:

B C

D E

G

B C

D E

G

This graph has a Hamiltonian circuit iff this fully-connected graph has a TSP cycle of total cost K, where K = |V| (here, K = 5)

HC TSP

2 2

1 1

1

1

1

1

1

1

46

Longest Path• Decision problem version: Is there a simple path in

G (between two given vertices s and t) of length at least k?

• Clearly in NP. Why?• To prove the longest path problem is NP-

complete,we can again reduce Hamiltonian circuit problem1. Input: a HC problem G with n vertices2. Duplicate some vertex s in G, call it t3. Add an edge of weight 0 between s and t4. Ask longest path: is there a path of at least weight n

between s and t?

47

Coping with NP-Completeness

1. Settle for algorithms that are fast on average: Worst case still takes exponential time, but doesn’t occur very often. But some NP-Complete problems are also average-time NP-Complete!

2. Settle for fast algorithms that give near-optimal solutions: In TSP, may not give the cheapest tour, but maybe good enough. But finding even approximate solutions to some NP-Complete problems

is NP-Complete!

3. Just get the exponent as low as possible! Much work on exponential algorithms for Boolean satisfiability: in practice can often solve problems with 1,000+ variablesBut even 2n/100 will eventual hit the exponential curve!

48

A Great Book You Should Own!

• Computers and Intractability: A Guide to the Theory of NP-Completeness, by Michael S. Garey and David S. Johnson