242-535 ADA: 12. Max Flow 1

• Objectiveo describe the maxflow problem, explain and

analyse the Ford-Fulkerson, Edmonds-Karp, and capacity scaling algorithms, and look at two application areas (bipartite matching and edge disjoint paths)

Algorithm Design and Analysis

(ADA)242-535, Semester 1 2014-2015

12. Max Flow Networks

Overview1. A Flow Network2. The Maxflow Problem3. The Mincut Problem4. Ford-Fulkerson (FF)

Algorithm5. Residual Network6. Ford-Fulkerson (FF)

Again

7. Edmonds-Karp Algorithm

8. Capacity Scaling9. Two Flow Problems

Again10.Bipartite Matching11.Edge Disjoint Paths12.Making Maxflow

Faster

242-535 ADA: 12. Max Flow 3

1. A Flow Networkpostive flowcapacities

source sink

242-535 ADA: 12. Max Flow 4

• A flow network is a directed graph G = (V, E) with two special vertices: a source s and a sink t

• Each edge (u, v) ∈ E has a non-negative capacity c(u, v)

• If(u, v) ∉ E, then c(u, v) = 0

242-535 ADA: 12. Max Flow 5

• The maxflow problemo find a flow of maximum value

• The mincut problemo find a cut of minimum capacity

• Two problems that appear very different but are actually two versions of the same question.

• Most of this part is about the theory and implementation of maxflow.

Two Flow Problems

242-535 ADA: 12. Max Flow 6

• A flow is an assignment of values to edges such that:o capacity constraint: 0 ≤ edge's flow ≤ edge's

capacityo flow conservation: inflow = outflow at every vertex

(except s and t).

2. The Maxflow Problem

flow: f(u,v)capacity: c(u,v)

inflow at v = 5 + 5 + 0 = 10

outflow at v = 10 + 0 = 10

[ / is not division]

242-535 ADA: 12. Max Flow 7

• The flow value f is the inflow at the sink to or the outflow at the source s

• Maxflow problem: find a flow of maximum value

flow =8 + 10 + 10= 28

242-535 ADA: 12. Max Flow 8

A flow f in G with value 19.

Another Example

A flow network G labelled with its edge capacities c(u,v)

242-535 ADA: 12. Max Flow 9

• A flow on G is a function f(u,v) satisfying the following conditions:

o capacity constraint: u, v ∈ V, f(u, v) ≤ c(u, v)

o flow conservation: u ∈ V - {s, t},

o skew symmetry: u, v ∈ V, f(u, v) = -f(v, u)

• The flow value f = f(V, t) ( or f(s, V) )

Flow Notation

242-535 ADA: 12. Max Flow 10

• Bipartite matching• Data mining• Network reliability• Image processing• Network connectivity• Distributed computing• many more ...

Many Applications of Max Flow

blood flow analysis

242-535 ADA: 12. Max Flow 11

• A cut is a partition of a flow network's vertices into two disjoint sets, with the source (s) in one set A and the sink (t) in the other set B.

• A cut's capacity is the sum of the capacities of the edges from A to B.

• The minimum cut (mincut) problem: find a cut of minimum capacity.

3. The Mincut Problem

242-535 ADA: 12. Max Flow 12

Examples

capacity = 10 + 8 + 16 = 34

don't countedges from B to A

capacity = 10 + 5 + 15 = 30

The minimum cut (mincut) problem:

find a cut of minimum capacity.

242-535 ADA: 12. Max Flow 13

• Initialization. Start with a 0 flow on every edge.• Find an augmenting path from s to t such that:

o we can increase flow on forward edges (while they are not full)

o we can decrease flow on backward edge (while they are not empty)

4. Ford-Fulkerson (FF) Algorithm

initiallyflow value =0

242-535 ADA: 12. Max Flow 14

First Augmenting Path

242-535 ADA: 12. Max Flow 15

Second Augmenting Path

242-535 ADA: 12. Max Flow 16

Third Augmenting Path

242-535 ADA: 12. Max Flow 17

Fourth Augmenting Path

242-535 ADA: 12. Max Flow 18

• No more augmenting paths• All paths from s to t are blocked by either:

o a full forward edge, oro a empty backward edge

Termination

maxflow = 28

mincut = 28

242-535 ADA: 12. Max Flow 19

• Let f be a flow on G = (V, E). The residual network Gf(V, Ef) is a graph showing residual capacitieso cf(u, v) = c(u, v) - f(u, v) > 0

• The idea is to draw the flow network G as graph Gf to make it easier to find augmented paths, and to determine the flow increase (and decrease) the chosen path causes.

5. Residual Network

u v6 / 17

flow f() capacity c()Graph G

u v

11

6

two residualcapacities cf()Graph Gf

for increasing flow in (u,v)

for decreasing flow in (u,v)

242-535 ADA: 12. Max Flow 20

• Residual networks also show how flow along an edge can be reducedo this is shown by having extra edges in Gf which are not

in G

• To represent a possible decrease of a flow f(u,v) on an edge in G, the edge (v,u) is added to Gf. Its residual capacity is:

cf(v, u) = f(u,v)

Gf Edges for Reducing

242-535 ADA: 12. Max Flow 21

• The complete definition of cf() is:

Residual Capacity cf() for increasing flow in (u,v)

for decreasing flow in (u,v)

u v6 / 17

flow f() capacity c()Graph G

u v

11

6

two residualcapacities cf()Graph Gf

242-535 ADA: 12. Max Flow 22

• A path from s to t in Gf is an aug menting path in G which carries flow f.

• The flow amount can be increased along an augmenting path by the minimum amountcf( p) = min {cf(u,v)}

• cf(p) is the residual or bottleneck capacity – it is the most that the flow can be increased due to one or more of p's edges being used at full capacity.

Augmenting Paths

(u,v) ∈ p

242-535 ADA: 12. Max Flow 23

Example

The flow network Gfrom an earlier example.

The residual network Gf with a possible augmenting path p shaded; its bottleneck capacity is

cf(p) = cf(v2, v3) = 4.

Edges with residual capacity equal to 0, such as (v1,v3) are not shown.

242-535 ADA: 12. Max Flow 24

The new flow in G that results from augmenting along path p by its bottleneck capacity 4.

Edges carrying no flow, such as (v3, v2), are labeled only by their capacity.

The residual network Gf version of this G graph.

No more augmented paths can be added, so f is the maxflow (f = 23).

242-535 ADA: 12. Max Flow 25

• Start with 0 flow.• While there exists an augmenting path:

o find an augmenting patho compute bottleneck capacityo increase flow on that path by bottleneck capacity

6. Ford-Fulkerson (FF) Again

242-535 ADA: 12. Max Flow 26

ford-Fulkerson(G) { foreach e ∈ E in G f(e) = 0 Gf = residual graph of G

while (there exists augmenting path P) { cf(p) = min{ cf(u,v) : (u,v) ∈ P } // bottleneck capacity foreach e ∈ P { // augment the flows with cf(p) if (e ∈ E in G) f(e) = f(e) + cf(p) // forward edge: e = (u,v) else f(e') = f(e') - cf(p) // reverse edge: e' = (v,u) } update Gf }}

FF in More Detail

242-535 ADA: 12. Max Flow 27

• The figures on the next 2 slides show successive iterations of the FF while-loop.

• The left side shows the residual network Gf with the chosen augmenting path p drawn as a shaded thick line.

• The right side shows the new flow f in G that results from adding the bottleneck capacity of p (cf(p)).

Executing of FF with a Residual Network

242-535 ADA: 12. Max Flow 28

Gf G

242-535 ADA: 12. Max Flow 29

The last residual network has no augmenting paths, and so the flow f shown in (e) above is a maximum flow (f = 23).

242-535 ADA: 12. Max Flow 30

• In the worst case, the running time of FF is O(E · |f|), where f is the maxflowo in the worst case, the FF while loop will iterate f times

since it's possible for the flow to increase by only I flow unit at a time (see example on the next slides)

o finding a path can have varying execution times, but if DFS or BFS is used to find a path, the running time is O(V + E), or O(E) is a dense graph

FF Running Time

242-535 ADA: 12. Max Flow 31

• The following example is designed so that each augmented path only increases the flow by 1 unit (i.e. the cf(p) is always 1).

Worst Running Time

242-535 ADA: 12. Max Flow 32

The Augmenting Paths

First Second

242-535 ADA: 12. Max Flow 33

Third Fourth

many,manymore

242-535 ADA: 12. Max Flow 34

199th 200th (= maxflow)

242-535 ADA: 12. Max Flow 35

• This case is easily avoided by a choosing a better series of augmenting paths.

Choosing Paths is Important

Only twoiterationsare needed.

242-535 ADA: 12. Max Flow 36

• Use care when selecting augmenting pathso Some choices lead to exponential algorithmso Clever choices lead to polynomial algorithmso If capacities are not integers, some algorithm are not

guaranteed to terminate

• Goal: choose augmenting paths so that:o Can find augmenting paths efficientlyo Few iterations

• Choose augmenting paths with: o Max bottleneck capacityo Sufficiently large bottleneck capacityo Fewest number of edges

Choosing Good Augmenting Paths

242-535 ADA: 12. Max Flow 37

• Edmonds and Karp's implementation of Ford-Fulkerson finds an augmenting path by using a breadth-first search.o the algorithm is slightly changed: a weight of 1 is

assigned to every edge, not the edge's capacity

• It runs in O(V· E2) timeo this can be proved by counting the number of

augmenting paths needed to find the maxflow (which is O(V· E) )

o the proof requires the monotonicity lemma

7. Edmonds-Karp Algorithm

242-535 ADA: 12. Max Flow 38

• Let d(v) = df(s, v) be the breadth-first distance from s (the source) to a vertex v in Gf o each edge has a weight of 1

• Lemma. During the execution of Edmonds-Karp, d(v) increases monotonically (only gets bigger).

Monotonicity Lemma

242-535 ADA: 12. Max Flow 39

• Suppose that augmenting a flow f on G produces a new flow f'. Let d′(v) = df'(s, v).

• We’ll show d′(v) ≥ d(v) by induction on d′(v)o we are proving that increased flow makes the shortest

path distance bigger for every vertex v

• For the base case, d′(v) = 0 implies v = s (the source), and since d(s) = 0, we have d′(v) ≥ d(v).

• So the base case is true.

Proof

242-535 ADA: 12. Max Flow 40

• For the inductive case, consider a breadth-first path s → …→ u → v in Gf'

• We must have d′(v) = d′(u) + 1, since subpaths of shortest paths are shortest paths (optimality).

• So, we have d′(u) ≥ d(u) by induction, because d′(v) > d′(u).

• (u, v) ∈ Ef' (because we assumed it; see above).• But we have to consider two cases depending on

whether (u, v) ∈ Ef or not.

242-535 ADA: 12. Max Flow 41

• We have d(v) ≤ d(u) + 1 (triangle inequality) ≤ d′(u) + 1 (induction) = d′(v) (breadth-first path)

and thus monotonicity of d(v) is established.

Case 1: (u, v) ∈ Ef

242-535 ADA: 12. Max Flow 42

• Since (u, v) ∈ Ef' , the augmenting path p that produced f' from f must have included (v, u).

• Moreover, p is a breadth-first path in Gf : o p = s → … → v → u → … → t

• Thus, we haved(v) = d(u) - 1 (breadth-first path) ≤ d′(u) - 1 (induction) = d′(v) - 2 (breadth-first path) < d′(v)

thereby establishing monotonicity for this case, too.

Case 2: (u, v) ∉ Ef

242-535 ADA: 12. Max Flow 43

• Theorem. The number of flow augmentations in the Edmonds-Karp algorithm (Ford-Fulkerson with breadth-first augmenting paths) is O(V· E).

• Proof. Let p be an augmenting path, and suppose that we have cf(u, v) = cf(p) for edge (u, v) ∈ po i.e. the bottleneck edge is (u, v)

• We say that (u, v) is critical, and it disappears from the residual graph after flow augmentation.

Counting Flow Augmentations

242-535 ADA: 12. Max Flow 44

• The first time an edge (u, v) is critical, we have d(v) = d(u) + 1, since p is a breadth-first path.

• We must wait until (v, u) is on an augmenting path before (u, v) can be critical again.

• Let d′ be the distance function when (v, u) is on an augmenting path. Then, we have d′(u) = d′(v) + 1 (breadth-first path) ≥ d(v) + 1 (monotonicity) = d(u) + 2 (breadth-first path)

242-535 ADA: 12. Max Flow 45

• Example

Why "+2"?

assume(u,v) iscritical

242-535 ADA: 12. Max Flow 46

was 5, now is 7

was 6, now is 8

assume(v,u) iscritical

242-535 ADA: 12. Max Flow 47

• Path distances start out non-negative, never decrease, and are at most |V| - 1 long until the vertex becomes unreachable.

• Thus, (u, v) occurs as a critical edge at most O(V/2) times, because d(v) increases by at least 2 between occurrenceso simplify O(V/2) to be O(V)

• Since the residual graph contains O(E) edges, the number of flow augmentations is O(V· E).

Running time of Edmonds-Karp

242-535 ADA: 12. Max Flow 48

• Corollary. The Edmonds-Karp maximum-flow algorithm runs in O(V· E2) time.

• Proof. Breadth-first search runs in O(E) time (actually O(V + E) but ignore the V), so the total running time is:

• O(V· E · E) = O(V· E2)

242-535 ADA: 12. Max Flow 49

boolean[] marked; // true if s->v path is in residual network

FlowEdge[] edgeTo; // last edge on s->v path

Queue<Integer> q = new Queue<Integer>(); // use when finding an augmenting path

Edmonds-Karp Code

242-535 ADA: 12. Max Flow 50

double edmondsKarp(FlowNetwork graph, int s, int t){ double value = 0; // flow value while (hasAugmentingPath(graph, s, t)) { double bottle = Double.POSITIVE_INFINITY; // initial bottleneck capacity for (int v = t; v != s; v = edgeTo[v].other(v)) bottle = Math.min(bottle, edgeTo[v].residualCapacityTo(v));

for (int v = t; v != s; v = edgeTo[v].other(v)) edgeTo[v].addResidualFlowTo(v, bottle);

value += bottle; // augment flow } return value; // will be the maxflow }

242-535 ADA: 12. Max Flow 51

boolean hasAugmentingPath(FlowNetwork graph, int s, int t){ marked.clear(); // set all marks to false q.reset(s); marked[s] = true; while (!q.isEmpty()) { int v = q.remove(); for (FlowEdge e : graph.adj(v)) { int w = e.other(v); // found path from s to w? if(e.residualCapacityTo(w) > 0 && !marked[w]) { edgeTo[w] = e; // save last edge on path to w; mark w; marked[w] = true; q.add(w); } } } return marked[t]; // is t reachable from s in residual network?}

242-535 ADA: 12. Max Flow 52

• Choosing an augmenting path with the highest bottleneck capacity will increase the flow by the biggest possible amount.o Don't try to find an exact highest bottleneck path (too

slow)o Use a scaling parameter Δ insteado Let Gf (Δ) be the subgraph of the residual graph consisting

of only arcs with capacity of at least Δ

8. Capacity Scaling

242-535 ADA: 12. Max Flow 53

scaling-Max-Flow(G, C) { foreach e ∈ E in G f(e) = 0 Δ = smallest power of 2 greater than or equal to C Gf = residual graph of G while (Δ ≥ 1) { Gf(Δ) = Δ-residual graph while (there exists augmenting path P in Gf(Δ)) { cf(p) = min{ cf(u,v) : (u,v) ∈ P } // bottleneck capacity foreach e ∈ P { // augment flows with P if (e ∈ E) f(e) = f(e) + b // forward edge e=(u,v) else f(e') = f(e') - b // reverse edge e'=(v,u) } update Gf(Δ) } Δ = Δ / 2 }}

Capacity Scaling Code

242-535 ADA: 12. Max Flow 54

• The number of augmenting paths this algorithm calculates is O(E log C) o the log component is due to the outer while loop

repeating O(log2 C) times since initially C ≤ Δ < 2C, and Δ decreases by a factor of 2 on each iteration

• Total running time is O(E2 log C)o since each augmentating path will be calculated in O(E)

time using BFS (as before)

Running Time

242-535 ADA: 12. Max Flow 55

• The maxflow problemo find a flow of maximum value

• The mincut problemo find a cut of minimum capacity

• Two problems that appear very different but are actually two versions of the same question.

• Maxflow-mincut theorem: maxflow value = mincut capacity

9. Two Flow Problems Again

242-535 ADA: 12. Max Flow 56

• The net flow across a cut (A, B) is the sum of the flows on its edges from A to B minus the sum of the flows on its edges from from B to A.

• If f is the current flow and let (A, B) be any cut. Then, the net flow across (A, B) equals the value of f.

Relationship between Flows and

Cuts

242-535 ADA: 12. Max Flow 57

Example

flow= 25

net flow across cut =10 + 5 + 10 = 25

242-535 ADA: 12. Max Flow 58

Example

maxflow = 28

mincut = 28

242-535 ADA: 12. Max Flow 59

• It is easy to calculate the mincut (A, B) if the maxflow is known (f).

• The A set of the mincut = the set of vertices connected to s by an undirected path with no full forward or empty backward edges.

Computing a mincut from a

maxflow

242-535 ADA: 12. Max Flow 60

• N students apply for N jobs.

• Each gets several offers.

• Is there a way to match every student to a job?

10. Bipartite Matching

242-535 ADA: 12. Max Flow 61

• Input: undirected, bipartite graph G = (L R, E). • M E is a matching if each node appears in at

most edge in M. • Max matching: find a maximum cardinality

matching.

More Formally

Matching:1- 2', 3 – 1', 4 – 5'

242-535 ADA: 12. Max Flow 62

• Direct all edges in E from L to R, and assign capacity 1 to each edge.

• Add source s, and capacity 1 edges from s to each node in L.

• Add sink t, and capacity 1 edges from each node in R to t. • The resulting digraph G' = (L R {s, t}, E' ).

Changing Bipartite to Max Flow

242-535 ADA: 12. Max Flow 63

Conversion to Max Flow

AliceBob

Carol DaveEliza

Frank

source

Adobe

Amazon

Facebook Google

IBM

Yahoo

sink

242-535 ADA: 12. Max Flow 64

• Given a digraph G = (V, E) and two nodes s and t, find the max number of edge-disjoint s-t paths.o two paths are edge-disjoint if they have no edge in

common

• Example: communication networks

11. Edge Disjoint Paths

242-535 ADA: 12. Max Flow 65

• A solution using 2 edge-disjoint paths:

242-535 ADA: 12. Max Flow 66

• Assign unit capacity to every edge. • The maximum number of edge-disjoint source-to-

sink paths equals the max flow value.

Max Flow Formulation

242-535 ADA: 12. Max Flow 67

• Given a digraph G = (V, E) and two nodes s and t, find the minimum number of edges whose removal disconnects t from s.

• A set of edges F E disconnects t from s if each s-t paths uses at least one edge in Fo removing F would make t unreachable from s

Dual Problem: Network

Connectivity

Removingtwo edgesbreaks thes-t link

242-535 ADA: 12. Max Flow 68

• Theorem [Menger 1927]. The max number of edge-disjoint s-t paths is equal to the min number of edges whose removal disconnects t from s.

utilizes the mincut

242-535 ADA: 12. Max Flow 69

12. Making MaxFlow Faster

year method worst caserunning time

discovered by

1955 augmenting path O(E · |f|) Ford-Fulkerson

1970 shortestaugmented path

O(V · E2) Edmonds-Karp

1983 dynamic trees O(E2 ·log E) Sleator-Tarjan

1985 capacity scaling O(E2 · log C) Gabow

1997 length function O(E3/2 · log E · log C) Goldberg-Rao

? ? E ?

C is the maximum capacity of any edge

242-535 ADA: 12. Max Flow 70

• Worst-case big-Oh is generally not useful for predicting or comparing real-world max flow algorithm performance.

• Best in practice: push-relabel method with gap relabeling: O(E3/2) (can be written as O(V3)o very close to linear in E, but still an open question

Analysis is Tricky