AHG14 Report

Optimization algorithms for solving computer vision problemsOlgierd StankiewiczKrzysztof Wegner

Chair of Multimedia Telecommunications and MicroelectronicsPozna University of Technology

Pozna, April 2015

1

Computer Vision ProblemsSegmentationAssigning each pixel of the image to a certain segmentDepth estimationAssigning a depth value to each pixel of the image

2

Computer Vision ProblemsImage stitchingAssigning each pixelof the output image to a certain source image (transformed)Image restorationAssigning to each pixel of the output image a colour from the source image

3

Computer Vision GeneralizationCan be seen as labeling problemAssigning to each pixel of the output imagea label defined in a certain wayLabel is an index from all possible answersSegment indexDisparityStitched image indexColour

4

1255621435344183540472967224536821400343

Energy minimizationThere are many ways to label pixels in an imageWhich one is better?What it the goal?Energy minimalization problem

5

Simple?Not simple!Multivariable, e.g. 1920 x 1080 2M varables Energy function can be very complexNon-monotonicNon-linearImplicit, with inter-label referencesClassic Stepest DesentNot too efficientWould probably not find the solution anyway

6

Efficient minimalizationSpecial class of energy functions can be minimalized more efficientlyEnergy function decomposed into sum of:Unary terms

Pairwise terms

Unary and pairwise terms7

Efficient minimalizationEven more efficient whenBinary labeling problemFunction argument can be 0 or 1Energy function is convex (submodular)Triangle inequality

E.g.MonotoneLinear (Planar etc.)

8

Example 19

}

Unary termsPairwise terms

Example 210

}

Unary termsPairwise terms

Optimization algorithmsViterbiState transitionsWell knowmBelief PropagationMessage passingPresented before

Graph Cuts11

Graph CutsGraph Cuts can be used for efficient unary and pairwise energy minimizationMin Cut == Max Flow theoremSolving of Minimal Cut problem in a graphis equal to solving of Maximal Flow problem in the same graphEfficient generic algorithmsExpression of energy minimization problemas MinCut12

GraphsNodesEdgesCapacityFlow (in a particular solution)ConstraintsFlow CapacityFlow conservationE.g. communication network13

Minimum s-t cutsSpecial nodesS - SourceT - Sink (Terminal)AlgorithmsAugmenting paths [Ford & Fulkerson, 1962]Push-relabel [Goldberg-Tarjan, 1986]14

Augmenting PathsFind a path from S to T along non-saturated edgesIncrease flow along this path until some edge saturates

15

Augmenting PathsFind next pathIncrease flow

16

Augmenting PathsIterate until all paths from S to T have at least one saturated edge

17

ExampleLets assume a graphNodes: s,o,p,q,r,tFlow=0

18

ExamplePath 1, Free Capacity:219

ExamplePath 1, Add Flow:2

20

ExamplePath 2, Free Capacity:1

21

ExamplePath 2, Add Flow:1

22

ExamplePath 3, Free Capacity:0

23

ExamplePath 4, Free Capacity:2

24

ExamplePath 4, Add Flow:2

25

Example - flowFlow from sink: 5 = Flow to terminal: 5Maximal flow = 526

Example - cutAll possible cuts27

Example minimal cutMinimal Cut = 5 Two equi-optimal cuts

28

Complexity29

Graph construction30

Each cut throught the graph must represent energy (some potential solution)The graph is a sum of elementary graphs for each energy term

Graph construction31

s

t

s

t

Graph constructionx32

s

t

Graph construction33=+++

Graph construction

34

s

t

Multilabel energy35

Ishikawa graphRoy&Cox 98 and Ishikawa 1998, 2000, 2003

36

Ishikawa graph

37

Ishikawa graph38Many nodes required at onceMany edgesVery slowRestricted only to linear, pairwise terms

a-expansion 39

a-expansion

Start with any* initial solutionFor each label a in any (e.g. random) orderCompute optimal a-expansion move (binary problem)Reject the move if there is no energy decreaseStop when no expansion move would decrease energy40

a-expansionTypically two cycles throught all labels are required*Depends on the initial solution

At given iteration some solution is knownIn Ishikawa only after solving the whole graph41

Thank you for attentionQuestions?42