map estimation in binary mrfs using bipartite multi-cuts
DESCRIPTION
MAP Estimation in Binary MRFs using Bipartite Multi-Cuts. Sashank J. Reddi Sunita Sarawagi Sundar Vishwanathan Indian Institute of Technology, Bombay. TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: A A A A A A A A A. MAP Estimation. θ i (x i ). - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: MAP Estimation in Binary MRFs using Bipartite Multi-Cuts](https://reader035.vdocuments.net/reader035/viewer/2022062301/56813a63550346895da25957/html5/thumbnails/1.jpg)
MAP Estimation in Binary MRFs using Bipartite Multi-Cuts
Sashank J. Reddi Sunita Sarawagi Sundar Vishwanathan
Indian Institute of Technology, Bombay
![Page 2: MAP Estimation in Binary MRFs using Bipartite Multi-Cuts](https://reader035.vdocuments.net/reader035/viewer/2022062301/56813a63550346895da25957/html5/thumbnails/2.jpg)
MAP Estimation• Energy Function
• MAP Estimation: Find the labeling which minimizes the energy function– NP Hard in general
xi
xjθij(xi,xj)
θi(xi)
![Page 3: MAP Estimation in Binary MRFs using Bipartite Multi-Cuts](https://reader035.vdocuments.net/reader035/viewer/2022062301/56813a63550346895da25957/html5/thumbnails/3.jpg)
Popular Approximation• Based on this LP relaxation
• Can be solved efficiently using Message Passing algorithms (TRW-S) and Graph cut based algorithms (QPBO)
![Page 4: MAP Estimation in Binary MRFs using Bipartite Multi-Cuts](https://reader035.vdocuments.net/reader035/viewer/2022062301/56813a63550346895da25957/html5/thumbnails/4.jpg)
Tightening Pairwise LP Relaxation• MPLP (David Sontag et.al 2008), Cycle Repairing Algorithm
(Nikos Komodakis et.al 2008)• MPLP uses higher order relaxation
![Page 5: MAP Estimation in Binary MRFs using Bipartite Multi-Cuts](https://reader035.vdocuments.net/reader035/viewer/2022062301/56813a63550346895da25957/html5/thumbnails/5.jpg)
Tightening Pairwise LP Relaxation• MPLP (David Sontag et.al 2008), Cycle Repairing Algorithm
(Nikos Komodakis et.al 2008)• MPLP uses higher order relaxation
![Page 6: MAP Estimation in Binary MRFs using Bipartite Multi-Cuts](https://reader035.vdocuments.net/reader035/viewer/2022062301/56813a63550346895da25957/html5/thumbnails/6.jpg)
MAP Estimation via Graph Cuts• Construct a specialized graph for the particular energy
function• Minimum cut on this graph minimizes the energy
function• Exact MAP in polynomial time when the energy
function is sub-modular–
![Page 7: MAP Estimation in Binary MRFs using Bipartite Multi-Cuts](https://reader035.vdocuments.net/reader035/viewer/2022062301/56813a63550346895da25957/html5/thumbnails/7.jpg)
Assumption on Parameters• Symmetric, that is and
• Zero-normalized, that is and
• Any Energy function can be transformed in to an equivalent energy function of this form
![Page 8: MAP Estimation in Binary MRFs using Bipartite Multi-Cuts](https://reader035.vdocuments.net/reader035/viewer/2022062301/56813a63550346895da25957/html5/thumbnails/8.jpg)
Graph Construction
00
01
![Page 9: MAP Estimation in Binary MRFs using Bipartite Multi-Cuts](https://reader035.vdocuments.net/reader035/viewer/2022062301/56813a63550346895da25957/html5/thumbnails/9.jpg)
Graph Construction
00
01
i0
i1
![Page 10: MAP Estimation in Binary MRFs using Bipartite Multi-Cuts](https://reader035.vdocuments.net/reader035/viewer/2022062301/56813a63550346895da25957/html5/thumbnails/10.jpg)
Graph Construction
00
01
i0i1
θi(1)
θi(0)
![Page 11: MAP Estimation in Binary MRFs using Bipartite Multi-Cuts](https://reader035.vdocuments.net/reader035/viewer/2022062301/56813a63550346895da25957/html5/thumbnails/11.jpg)
Graph Construction
00
01
i0 j0
θi(1)
θi(0)
θj(1)
θj(0)
θij/2
θij= θij(0,1)+ θij(1,0)- θij(0,0)- θij(1,1) > 0
i1 j1
![Page 12: MAP Estimation in Binary MRFs using Bipartite Multi-Cuts](https://reader035.vdocuments.net/reader035/viewer/2022062301/56813a63550346895da25957/html5/thumbnails/12.jpg)
Graph Construction
00
01
i0 j1
θi(1)
θi(0)
-θij/2
θij= θij(0,1)+ θij(1,0)- θij(0,0)- θij(1,1) < 0
i1 j0
θj(1)
θj(0)
![Page 13: MAP Estimation in Binary MRFs using Bipartite Multi-Cuts](https://reader035.vdocuments.net/reader035/viewer/2022062301/56813a63550346895da25957/html5/thumbnails/13.jpg)
Bipartite Multi-Cut problem• Given an undirected graph with non-negative
edge weights and k ST pairs– Objective: Find the minimum cut with divides the graph into 2 regions
and separates the ST pairs
• LP and SDP Relaxations (Sreyash Kenkre et.al 2006)– LP Relaxation gives approximation– SDP Relaxation gives approximation
• Bipartite Multi-Cut vs Multi-Cut– Additional constraint on the number of regions the graph is cut
![Page 14: MAP Estimation in Binary MRFs using Bipartite Multi-Cuts](https://reader035.vdocuments.net/reader035/viewer/2022062301/56813a63550346895da25957/html5/thumbnails/14.jpg)
BMC LP• are the ST pairs in the Bipartite Multi-Cut problem • Terminals =
![Page 15: MAP Estimation in Binary MRFs using Bipartite Multi-Cuts](https://reader035.vdocuments.net/reader035/viewer/2022062301/56813a63550346895da25957/html5/thumbnails/15.jpg)
BMC LP• are the ST pairs in the Bipartite Multi-Cut problem • Terminals = is jt
00 01
de
D00(is)
D00(jt)
k0k1
Dk0(is) Dk0
(jt)
![Page 16: MAP Estimation in Binary MRFs using Bipartite Multi-Cuts](https://reader035.vdocuments.net/reader035/viewer/2022062301/56813a63550346895da25957/html5/thumbnails/16.jpg)
BMC LP• are the ST pairs in the Bipartite Multi-Cut problem • Terminals =
i0 i1
j0j1
![Page 17: MAP Estimation in Binary MRFs using Bipartite Multi-Cuts](https://reader035.vdocuments.net/reader035/viewer/2022062301/56813a63550346895da25957/html5/thumbnails/17.jpg)
BMC LP• are the ST pairs in the Bipartite Multi-Cut problem • Terminals =
is jt
00 01
de
D00(is)D00(jt)
![Page 18: MAP Estimation in Binary MRFs using Bipartite Multi-Cuts](https://reader035.vdocuments.net/reader035/viewer/2022062301/56813a63550346895da25957/html5/thumbnails/18.jpg)
BMC LP• Infeasible to solve it using LP solvers– Large number of constraints
• Combinatorial Algorithm– Solving LP for general graphs may not be easy– Flexibility in construction of the graph– Combinatorial algorithm for a special kind of
construction
![Page 19: MAP Estimation in Binary MRFs using Bipartite Multi-Cuts](https://reader035.vdocuments.net/reader035/viewer/2022062301/56813a63550346895da25957/html5/thumbnails/19.jpg)
Symmetric Graph Construction
ee ww 00
01
i0 j0
θj(1)/2
θj(0)/2θi(0)/2
θij/4
θij= θij(0,1)+ θij(1,0)- θij(0,0)- θij(1,1)
00
01
i1 j1
θj(0)/2
θj(1)/2
θi(0)/2
θi(1)/2
θij/4
θi(1)/2
![Page 20: MAP Estimation in Binary MRFs using Bipartite Multi-Cuts](https://reader035.vdocuments.net/reader035/viewer/2022062301/56813a63550346895da25957/html5/thumbnails/20.jpg)
Symmetric Graph Construction• More ST pairs are added• Trade off is combinatorial algorithm
![Page 21: MAP Estimation in Binary MRFs using Bipartite Multi-Cuts](https://reader035.vdocuments.net/reader035/viewer/2022062301/56813a63550346895da25957/html5/thumbnails/21.jpg)
Approach of the Algorithm
BMC LP
Multicommodity flow LP
MultiCut LP
![Page 22: MAP Estimation in Binary MRFs using Bipartite Multi-Cuts](https://reader035.vdocuments.net/reader035/viewer/2022062301/56813a63550346895da25957/html5/thumbnails/22.jpg)
Multi-Cut LP• denotes all paths between pair of vertices in ST• denotes the set of paths in which contain edge e
• Can be solved approximately i.e ε-approximation for any error parameter ε
Multi – Cut LP Multi -Commodity Flow LP
![Page 23: MAP Estimation in Binary MRFs using Bipartite Multi-Cuts](https://reader035.vdocuments.net/reader035/viewer/2022062301/56813a63550346895da25957/html5/thumbnails/23.jpg)
Symmetric BMC LP (BMC-Sym LP)
• Very similar to Multi-Cut LP except for the symmetric constraints
![Page 24: MAP Estimation in Binary MRFs using Bipartite Multi-Cuts](https://reader035.vdocuments.net/reader035/viewer/2022062301/56813a63550346895da25957/html5/thumbnails/24.jpg)
Equivalence of LPs
Theorem – When the constructed graph is symmetric, BMC LP, BMC-Sym LP and Multi-Cut LP are equivalent
Proof Outline• Any feasible solution of each of the LP can be transformed into
a feasible solution of other LPs without changing the objective value
![Page 25: MAP Estimation in Binary MRFs using Bipartite Multi-Cuts](https://reader035.vdocuments.net/reader035/viewer/2022062301/56813a63550346895da25957/html5/thumbnails/25.jpg)
Combinatorial AlgorithmPrimal Step
• Find shortest path P between
• Update
• Update Complementary path
Dual Step
• Let• Update the flow in the path
by• Update flow in
complementary path
• Converges in• Can be improved to (Fleischer 1999)
![Page 26: MAP Estimation in Binary MRFs using Bipartite Multi-Cuts](https://reader035.vdocuments.net/reader035/viewer/2022062301/56813a63550346895da25957/html5/thumbnails/26.jpg)
Relationship with Cycle Inequalities
• BMC LP is closely related to cycle inequalities– BMC LP with slight modification is equivalent to cycle inequalities
• Relates our work to many recent works solving cycle inequalities
![Page 27: MAP Estimation in Binary MRFs using Bipartite Multi-Cuts](https://reader035.vdocuments.net/reader035/viewer/2022062301/56813a63550346895da25957/html5/thumbnails/27.jpg)
Empirical Results• Data Sets
– Synthetic Problems• Clique Graphical models
– Benchmark Data Set
• Algorithms– TRW-S– BMC
• ε = 0.02
– MPLP• 1000 iterations or up to convergence
![Page 28: MAP Estimation in Binary MRFs using Bipartite Multi-Cuts](https://reader035.vdocuments.net/reader035/viewer/2022062301/56813a63550346895da25957/html5/thumbnails/28.jpg)
Empirical Results
![Page 29: MAP Estimation in Binary MRFs using Bipartite Multi-Cuts](https://reader035.vdocuments.net/reader035/viewer/2022062301/56813a63550346895da25957/html5/thumbnails/29.jpg)
Empirical Results
Convergence Comparison of BMC and MPLP
![Page 30: MAP Estimation in Binary MRFs using Bipartite Multi-Cuts](https://reader035.vdocuments.net/reader035/viewer/2022062301/56813a63550346895da25957/html5/thumbnails/30.jpg)
Empirical Results
![Page 31: MAP Estimation in Binary MRFs using Bipartite Multi-Cuts](https://reader035.vdocuments.net/reader035/viewer/2022062301/56813a63550346895da25957/html5/thumbnails/31.jpg)
Conclusion & Future Work• Conclusion– MAP estimation can be reduced to Bipartite Multi-Cut
problem– Algorithms for multi commodity flow can be used for MAP
estimation– BMC LP is closely related to cycle inequalities
• Future Work– Extensions to multi-label graphical models– Combinatorial algorithm for solving SDP relaxation