hierarchical graph cuts for semi-metric labeling m. pawan kumar joint work with daphne koller
Post on 21-Dec-2015
220 views
TRANSCRIPT
![Page 1: Hierarchical Graph Cuts for Semi-Metric Labeling M. Pawan Kumar Joint work with Daphne Koller](https://reader030.vdocuments.net/reader030/viewer/2022032522/56649d6a5503460f94a486e8/html5/thumbnails/1.jpg)
Hierarchical Graph Cutsfor Semi-Metric Labeling
M. Pawan Kumar
Joint work with
Daphne Koller
![Page 2: Hierarchical Graph Cuts for Semi-Metric Labeling M. Pawan Kumar Joint work with Daphne Koller](https://reader030.vdocuments.net/reader030/viewer/2022032522/56649d6a5503460f94a486e8/html5/thumbnails/2.jpg)
AimTo obtain accurate MAP estimate for Semi-Metric MRFs efficiently
V1 V2 … … …
… … … … …
… … … … …
… … … … Vn
Random Variables V = { V1, V2, …, Vn}
![Page 3: Hierarchical Graph Cuts for Semi-Metric Labeling M. Pawan Kumar Joint work with Daphne Koller](https://reader030.vdocuments.net/reader030/viewer/2022032522/56649d6a5503460f94a486e8/html5/thumbnails/3.jpg)
Aim
Va Vb
li
ab(i,j) a(i) : arbitrary
ab(i,j) = sab d(i,j)
sab ≥ 0a(i) b(j)
lj
d( i , i ) = 0 for all i d( i , j ) = d( j , i ) > 0 for all i≠j
Semi-metric Distance Function
d( i , j ) - d( j , k ) ≤ d( i , k )
Metric Distance Function
To obtain accurate MAP estimate for Semi-Metric MRFs efficiently
![Page 4: Hierarchical Graph Cuts for Semi-Metric Labeling M. Pawan Kumar Joint work with Daphne Koller](https://reader030.vdocuments.net/reader030/viewer/2022032522/56649d6a5503460f94a486e8/html5/thumbnails/4.jpg)
Aim
Va Vb
li
ab(i,j) a(i) : arbitrary
a(i) b(j)
lj
f* = arg minf a(f(a)) + ab(f(a),f(b))
ab(i,j) = sab d(i,j)
sab ≥ 0
To obtain accurate MAP estimate for Semi-Metric MRFs efficiently
![Page 5: Hierarchical Graph Cuts for Semi-Metric Labeling M. Pawan Kumar Joint work with Daphne Koller](https://reader030.vdocuments.net/reader030/viewer/2022032522/56649d6a5503460f94a486e8/html5/thumbnails/5.jpg)
Visualizing Metrics
l5
l1l2
l4l3
w1w2
w3
w4
w5
w6
w7 w9w8
d( i , j ) : shortest path defined by the graph
![Page 6: Hierarchical Graph Cuts for Semi-Metric Labeling M. Pawan Kumar Joint work with Daphne Koller](https://reader030.vdocuments.net/reader030/viewer/2022032522/56649d6a5503460f94a486e8/html5/thumbnails/6.jpg)
Overview
+
f1 f2f
![Page 7: Hierarchical Graph Cuts for Semi-Metric Labeling M. Pawan Kumar Joint work with Daphne Koller](https://reader030.vdocuments.net/reader030/viewer/2022032522/56649d6a5503460f94a486e8/html5/thumbnails/7.jpg)
Outline
• Simpler Metrics
• Labeling for Simpler Metrics
• Approximating General Metrics/Semi-Metrics
• Combining Labelings
• Results
![Page 8: Hierarchical Graph Cuts for Semi-Metric Labeling M. Pawan Kumar Joint work with Daphne Koller](https://reader030.vdocuments.net/reader030/viewer/2022032522/56649d6a5503460f94a486e8/html5/thumbnails/8.jpg)
r-HST Metrics
Edge lengths for all children are the same
l1 l2 l3 l4 l5 l6
w1 w1
w2 w2w2 w3 w3
w3
Graph is a Tree. Labels are leaves
![Page 9: Hierarchical Graph Cuts for Semi-Metric Labeling M. Pawan Kumar Joint work with Daphne Koller](https://reader030.vdocuments.net/reader030/viewer/2022032522/56649d6a5503460f94a486e8/html5/thumbnails/9.jpg)
r-HST Metrics
Edge lengths decrease by factor r ≥ 2
w2 ≤ w1/r w3 ≤ w1/r
l1 l2 l3 l4 l5 l6
w1 w1
w2 w2w2 w3 w3
w3
![Page 10: Hierarchical Graph Cuts for Semi-Metric Labeling M. Pawan Kumar Joint work with Daphne Koller](https://reader030.vdocuments.net/reader030/viewer/2022032522/56649d6a5503460f94a486e8/html5/thumbnails/10.jpg)
Outline
• Simpler Metrics
• Labeling for Simpler Metrics
• Approximating General Metrics/Semi-Metrics
• Combining Labelings
• Results
![Page 11: Hierarchical Graph Cuts for Semi-Metric Labeling M. Pawan Kumar Joint work with Daphne Koller](https://reader030.vdocuments.net/reader030/viewer/2022032522/56649d6a5503460f94a486e8/html5/thumbnails/11.jpg)
r-HST Metric Labeling
r-HST Metrics admit Divide-and-Conquer
Divide original problem into subproblems
l1 l2 l3 l4 l5 l6
w1 w1
w2 w2w2 w3 w3
w3
![Page 12: Hierarchical Graph Cuts for Semi-Metric Labeling M. Pawan Kumar Joint work with Daphne Koller](https://reader030.vdocuments.net/reader030/viewer/2022032522/56649d6a5503460f94a486e8/html5/thumbnails/12.jpg)
r-HST Metric Labeling
Subproblem defined at vertex ‘m’
l1 l2 l3 l4 l5 l6
w1 w1
w2 w2w2 w3 w3
w3
f* = arg minf a(f(a)) + ab(f(a),f(b))
such that f(a) m
![Page 13: Hierarchical Graph Cuts for Semi-Metric Labeling M. Pawan Kumar Joint work with Daphne Koller](https://reader030.vdocuments.net/reader030/viewer/2022032522/56649d6a5503460f94a486e8/html5/thumbnails/13.jpg)
r-HST Metric Labeling
Trivial problem
l1 l2 l3 l4 l5 l6
w1 w1
w2 w2w2 w3 w3
w3
f* = arg minf a(f(a)) + ab(f(a),f(b))
such that f(a) { l4 }
![Page 14: Hierarchical Graph Cuts for Semi-Metric Labeling M. Pawan Kumar Joint work with Daphne Koller](https://reader030.vdocuments.net/reader030/viewer/2022032522/56649d6a5503460f94a486e8/html5/thumbnails/14.jpg)
r-HST Metric Labeling
Original problem
l1 l2 l3 l4 l5 l6
w1 w1
w2 w2w2 w3 w3
w3
f* = arg minf a(f(a)) + ab(f(a),f(b))
such that f(a) { l1, …, l6 }
![Page 15: Hierarchical Graph Cuts for Semi-Metric Labeling M. Pawan Kumar Joint work with Daphne Koller](https://reader030.vdocuments.net/reader030/viewer/2022032522/56649d6a5503460f94a486e8/html5/thumbnails/15.jpg)
r-HST Metric Labeling
Problems get tougher as we move up
Solve the simple subproblems(starting with trivial subproblems)
Use their solutions to solve difficult subproblems
![Page 16: Hierarchical Graph Cuts for Semi-Metric Labeling M. Pawan Kumar Joint work with Daphne Koller](https://reader030.vdocuments.net/reader030/viewer/2022032522/56649d6a5503460f94a486e8/html5/thumbnails/16.jpg)
r-HST Metric Labeling
w ww
f1 f2 f3
Find new labeling using -Expansion
![Page 17: Hierarchical Graph Cuts for Semi-Metric Labeling M. Pawan Kumar Joint work with Daphne Koller](https://reader030.vdocuments.net/reader030/viewer/2022032522/56649d6a5503460f94a486e8/html5/thumbnails/17.jpg)
r-HST Metric Labeling
w ww
f1 f2 f3
Continue till we reach the root
![Page 18: Hierarchical Graph Cuts for Semi-Metric Labeling M. Pawan Kumar Joint work with Daphne Koller](https://reader030.vdocuments.net/reader030/viewer/2022032522/56649d6a5503460f94a486e8/html5/thumbnails/18.jpg)
Analysis
w ww
Mathematical Induction
All variables Va such that f*(a) m
m
1 bound on the unary potentials
2r/(r-1) bound on the pairwise potentials
![Page 19: Hierarchical Graph Cuts for Semi-Metric Labeling M. Pawan Kumar Joint work with Daphne Koller](https://reader030.vdocuments.net/reader030/viewer/2022032522/56649d6a5503460f94a486e8/html5/thumbnails/19.jpg)
Analysis
w ww
Mathematical Induction
m
Initial step of M.I. trivial (for leaf nodes)
Given children, prove for parent
![Page 20: Hierarchical Graph Cuts for Semi-Metric Labeling M. Pawan Kumar Joint work with Daphne Koller](https://reader030.vdocuments.net/reader030/viewer/2022032522/56649d6a5503460f94a486e8/html5/thumbnails/20.jpg)
Analysis
w ww
a(f(a)) +
i ab(fi(a),fi(b)) +
i≠j ab(fi(a),fj(b))
f(a) = fi(a)f(b) = fi(b)
f(a) = fi(a)f(b) = fj(b)
![Page 21: Hierarchical Graph Cuts for Semi-Metric Labeling M. Pawan Kumar Joint work with Daphne Koller](https://reader030.vdocuments.net/reader030/viewer/2022032522/56649d6a5503460f94a486e8/html5/thumbnails/21.jpg)
Analysis
w ww
a(f*(a)) +
i ab(fi(a),fi(b)) +
i≠j ab(fi(a),fj(b))
![Page 22: Hierarchical Graph Cuts for Semi-Metric Labeling M. Pawan Kumar Joint work with Daphne Koller](https://reader030.vdocuments.net/reader030/viewer/2022032522/56649d6a5503460f94a486e8/html5/thumbnails/22.jpg)
Analysis
w ww
a(f*(a)) +
i ab(f*(a),f*(b)) +
i≠j ab(fi(a),fj(b))
2rr-1
![Page 23: Hierarchical Graph Cuts for Semi-Metric Labeling M. Pawan Kumar Joint work with Daphne Koller](https://reader030.vdocuments.net/reader030/viewer/2022032522/56649d6a5503460f94a486e8/html5/thumbnails/23.jpg)
Analysis
w ww
a(f*(a)) +
i ab(f*(a),f*(b)) +
i≠j ab(f*(a),f*(b)) dmax
dmin
2
2rr-1
![Page 24: Hierarchical Graph Cuts for Semi-Metric Labeling M. Pawan Kumar Joint work with Daphne Koller](https://reader030.vdocuments.net/reader030/viewer/2022032522/56649d6a5503460f94a486e8/html5/thumbnails/24.jpg)
Analysis
w ww
dmax = 2w(1+1/r+1/r2+….)
dmin = 2w
![Page 25: Hierarchical Graph Cuts for Semi-Metric Labeling M. Pawan Kumar Joint work with Daphne Koller](https://reader030.vdocuments.net/reader030/viewer/2022032522/56649d6a5503460f94a486e8/html5/thumbnails/25.jpg)
Analysis
w ww
i≠j ab(f*(a),f*(b)) 2rr-1
a(f*(a)) +
i ab(f*(a),f*(b)) + 2rr-1
![Page 26: Hierarchical Graph Cuts for Semi-Metric Labeling M. Pawan Kumar Joint work with Daphne Koller](https://reader030.vdocuments.net/reader030/viewer/2022032522/56649d6a5503460f94a486e8/html5/thumbnails/26.jpg)
Analysis
Overall approximation bound 2r/(r-1)
Previous best bound 2r/(r-2)
Not Tight ?
![Page 27: Hierarchical Graph Cuts for Semi-Metric Labeling M. Pawan Kumar Joint work with Daphne Koller](https://reader030.vdocuments.net/reader030/viewer/2022032522/56649d6a5503460f94a486e8/html5/thumbnails/27.jpg)
Overview
+
f1 f2f
![Page 28: Hierarchical Graph Cuts for Semi-Metric Labeling M. Pawan Kumar Joint work with Daphne Koller](https://reader030.vdocuments.net/reader030/viewer/2022032522/56649d6a5503460f94a486e8/html5/thumbnails/28.jpg)
Outline
• Simpler Metrics
• Labeling for Simpler Metrics
• Approximating General Metrics/Semi-Metrics
• Combining Labelings
• Results
![Page 29: Hierarchical Graph Cuts for Semi-Metric Labeling M. Pawan Kumar Joint work with Daphne Koller](https://reader030.vdocuments.net/reader030/viewer/2022032522/56649d6a5503460f94a486e8/html5/thumbnails/29.jpg)
Approximating Metrics
D = {dt(.,.), t = 1,2,… T}, dt(i,j) ≥ d(i,j)
Pr(.) over the elements of D
Given distance d(.,.)
minD,Pr(.) maxi≠j ∑tPr(t) dt(i,j)
d(i,j)
![Page 30: Hierarchical Graph Cuts for Semi-Metric Labeling M. Pawan Kumar Joint work with Daphne Koller](https://reader030.vdocuments.net/reader030/viewer/2022032522/56649d6a5503460f94a486e8/html5/thumbnails/30.jpg)
Approximating Metrics
l1 l2 l3 l4 l5 l6
w1 w1
w2 w2w2 w3 w3
w3
r-HST : hierarchical clustering of labels
Use a clustering algorithm
![Page 31: Hierarchical Graph Cuts for Semi-Metric Labeling M. Pawan Kumar Joint work with Daphne Koller](https://reader030.vdocuments.net/reader030/viewer/2022032522/56649d6a5503460f94a486e8/html5/thumbnails/31.jpg)
Approximating MetricsFakcharoenphol, Rao and Talwar, 2003
max d(i,j) = 2M mini≠j d(i,j) > 1
Level ‘1’
Level ‘2’
Clustering at level 2??
Sample [1,2]
Choose a permutation π of labels
= { l1,…, lh }
![Page 32: Hierarchical Graph Cuts for Semi-Metric Labeling M. Pawan Kumar Joint work with Daphne Koller](https://reader030.vdocuments.net/reader030/viewer/2022032522/56649d6a5503460f94a486e8/html5/thumbnails/32.jpg)
Approximating Metrics
max d(i,j) = 2M mini≠j d(i,j) > 1
Level ‘m-2’
Level ‘m-1’
Clustering at level m??
Choose a permutation π of labels
Fakcharoenphol, Rao and Talwar, 2003
Sample [1,2]
![Page 33: Hierarchical Graph Cuts for Semi-Metric Labeling M. Pawan Kumar Joint work with Daphne Koller](https://reader030.vdocuments.net/reader030/viewer/2022032522/56649d6a5503460f94a486e8/html5/thumbnails/33.jpg)
Approximating Metrics
l1 l2l3l4 l5l6
l1 l2 l3
π
d(1,4) ≤ 2M-m ?
Fakcharoenphol, Rao and Talwar, 2003
![Page 34: Hierarchical Graph Cuts for Semi-Metric Labeling M. Pawan Kumar Joint work with Daphne Koller](https://reader030.vdocuments.net/reader030/viewer/2022032522/56649d6a5503460f94a486e8/html5/thumbnails/34.jpg)
Approximating Metrics
l1 l2l3l4 l5l6
l1
l2 l3
π
d(2,4) ≤ 2M-m ?
Fakcharoenphol, Rao and Talwar, 2003
![Page 35: Hierarchical Graph Cuts for Semi-Metric Labeling M. Pawan Kumar Joint work with Daphne Koller](https://reader030.vdocuments.net/reader030/viewer/2022032522/56649d6a5503460f94a486e8/html5/thumbnails/35.jpg)
Approximating Metrics
l1 l2l3l4 l5l6
l1
l2 l3
π
d(2,1) ≤ 2M-m ?
Fakcharoenphol, Rao and Talwar, 2003
![Page 36: Hierarchical Graph Cuts for Semi-Metric Labeling M. Pawan Kumar Joint work with Daphne Koller](https://reader030.vdocuments.net/reader030/viewer/2022032522/56649d6a5503460f94a486e8/html5/thumbnails/36.jpg)
Approximating Metrics
l1 l2l3l4 l5l6
l1 l2
l3
π
d(3,4) ≤ 2M-m ?
Fakcharoenphol, Rao and Talwar, 2003
![Page 37: Hierarchical Graph Cuts for Semi-Metric Labeling M. Pawan Kumar Joint work with Daphne Koller](https://reader030.vdocuments.net/reader030/viewer/2022032522/56649d6a5503460f94a486e8/html5/thumbnails/37.jpg)
Approximating Metrics
l1 l2l3l4 l5l6
l1 l2
l3
π
Edge length = Diameter of cluster / 2
Fakcharoenphol, Rao and Talwar, 2003
![Page 38: Hierarchical Graph Cuts for Semi-Metric Labeling M. Pawan Kumar Joint work with Daphne Koller](https://reader030.vdocuments.net/reader030/viewer/2022032522/56649d6a5503460f94a486e8/html5/thumbnails/38.jpg)
Approximating Metrics
Choose . Choose π
Initialize root node as trivial cluster (all labels)
Choose a cluster at level m-1
Run procedure to get clusters at level m
Repeat for all clusters at level m-1
Stop when all clusters are singletons
Repeat to get a set of r-HST metrics
Fakcharoenphol, Rao and Talwar, 2003
![Page 39: Hierarchical Graph Cuts for Semi-Metric Labeling M. Pawan Kumar Joint work with Daphne Koller](https://reader030.vdocuments.net/reader030/viewer/2022032522/56649d6a5503460f94a486e8/html5/thumbnails/39.jpg)
Analysis
d(i,j) ≤ ∑Pr(t) dt(i,j) ≤ O(log h) d(i,j)
How many r-HST metrics ??
O(h log h)
Charikar, Chekuri, Goel, Guha and Plotkin, 1998
Fakcharoenphol, Rao and Talwar, 2003
![Page 40: Hierarchical Graph Cuts for Semi-Metric Labeling M. Pawan Kumar Joint work with Daphne Koller](https://reader030.vdocuments.net/reader030/viewer/2022032522/56649d6a5503460f94a486e8/html5/thumbnails/40.jpg)
Approximating Semi-Metrics
d(i,j) ≤ ∑Pr(t) dt(i,j) ≤ O(( log h)2) d(i,j)
How many r-HST metrics ??
O(h log h)
d(i,j) - d(j,k) ≤ d(i,k)
![Page 41: Hierarchical Graph Cuts for Semi-Metric Labeling M. Pawan Kumar Joint work with Daphne Koller](https://reader030.vdocuments.net/reader030/viewer/2022032522/56649d6a5503460f94a486e8/html5/thumbnails/41.jpg)
Overview
+
f1 f2f
![Page 42: Hierarchical Graph Cuts for Semi-Metric Labeling M. Pawan Kumar Joint work with Daphne Koller](https://reader030.vdocuments.net/reader030/viewer/2022032522/56649d6a5503460f94a486e8/html5/thumbnails/42.jpg)
Outline
• Simpler Metrics
• Labeling for Simpler Metrics
• Approximating General Metrics/Semi-Metrics
• Combining Labelings
• Results
![Page 43: Hierarchical Graph Cuts for Semi-Metric Labeling M. Pawan Kumar Joint work with Daphne Koller](https://reader030.vdocuments.net/reader030/viewer/2022032522/56649d6a5503460f94a486e8/html5/thumbnails/43.jpg)
Combining Labelings
Use -Expansion !!
![Page 44: Hierarchical Graph Cuts for Semi-Metric Labeling M. Pawan Kumar Joint work with Daphne Koller](https://reader030.vdocuments.net/reader030/viewer/2022032522/56649d6a5503460f94a486e8/html5/thumbnails/44.jpg)
Analysis
Bound for r-HST Labeling = O(1)
Distortion for Metrics = O(log h)
Bound for Metric Labeling = O(log h)
Distortion for Semi-Metrics = O(( log h)2)
Bound for Semi-Metric Labeling = O(( log h)2)
![Page 45: Hierarchical Graph Cuts for Semi-Metric Labeling M. Pawan Kumar Joint work with Daphne Koller](https://reader030.vdocuments.net/reader030/viewer/2022032522/56649d6a5503460f94a486e8/html5/thumbnails/45.jpg)
Analysis
When h < n, all known LP boundscan be obtained using move making algorithms.
![Page 46: Hierarchical Graph Cuts for Semi-Metric Labeling M. Pawan Kumar Joint work with Daphne Koller](https://reader030.vdocuments.net/reader030/viewer/2022032522/56649d6a5503460f94a486e8/html5/thumbnails/46.jpg)
Refining the Labeling
Current energy Q(f; d) = Q(f; dt)
Q(f’; d) ≤ Q(f’; dt), f’ ≠ f
Find best ft according to dt(.,.)
Fakcharoenphol, Rao and Talwar, 2003
r-HST Metric Labeling
f = ft. Repeat till convergence.
![Page 47: Hierarchical Graph Cuts for Semi-Metric Labeling M. Pawan Kumar Joint work with Daphne Koller](https://reader030.vdocuments.net/reader030/viewer/2022032522/56649d6a5503460f94a486e8/html5/thumbnails/47.jpg)
Outline
• Simpler Metrics
• Labeling for Simpler Metrics
• Approximating General Metrics/Semi-Metrics
• Combining Labelings
• Results
![Page 48: Hierarchical Graph Cuts for Semi-Metric Labeling M. Pawan Kumar Joint work with Daphne Koller](https://reader030.vdocuments.net/reader030/viewer/2022032522/56649d6a5503460f94a486e8/html5/thumbnails/48.jpg)
Synthetic Data
T. Lin. T. Quad. r-HST Met S-Met
Exp 48645 52094 50221 48112 47613
Swap 48721 51938 51055 48487 47579
TRW-S 47506 51318 48132 47355 46612
BP-S 50942 60269 52841 48136 47402
R-Swap 48045 51842 - - -
R-Exp 47998 51641 - - -
Our 47850 51587 48146 47538 46651
Our+EM 47823 51413 48146 47382 46638
![Page 49: Hierarchical Graph Cuts for Semi-Metric Labeling M. Pawan Kumar Joint work with Daphne Koller](https://reader030.vdocuments.net/reader030/viewer/2022032522/56649d6a5503460f94a486e8/html5/thumbnails/49.jpg)
Synthetic Data
T. Lin. T. Quad. r-HST Met S-Met
Exp 0.44 0.36 0.29 0.30 0.36
Swap 0.65 0.86 0.52 0.51 0.47
TRW-S 104.29 178.97 713.70 703.82 709.36
BP-S 15.78 45.63 150.36 129.68 141.79
R-Swap 1.97 10.73 - - -
R-Exp 5.78 30.73 - - -
Our 10.22 12.84 1.86 10.58 12.25
Our+EM 25.66 64.08 5.02 32.75 57.50
![Page 50: Hierarchical Graph Cuts for Semi-Metric Labeling M. Pawan Kumar Joint work with Daphne Koller](https://reader030.vdocuments.net/reader030/viewer/2022032522/56649d6a5503460f94a486e8/html5/thumbnails/50.jpg)
Image Denoising
![Page 51: Hierarchical Graph Cuts for Semi-Metric Labeling M. Pawan Kumar Joint work with Daphne Koller](https://reader030.vdocuments.net/reader030/viewer/2022032522/56649d6a5503460f94a486e8/html5/thumbnails/51.jpg)
Image DenoisingExp Swap TRW-S
BP-S Our Our+EM
75641,5.09 74426,25.22 68226,174.33
105845,32.94 72828,70.55 72332,204.55
![Page 52: Hierarchical Graph Cuts for Semi-Metric Labeling M. Pawan Kumar Joint work with Daphne Koller](https://reader030.vdocuments.net/reader030/viewer/2022032522/56649d6a5503460f94a486e8/html5/thumbnails/52.jpg)
Image Denoising
![Page 53: Hierarchical Graph Cuts for Semi-Metric Labeling M. Pawan Kumar Joint work with Daphne Koller](https://reader030.vdocuments.net/reader030/viewer/2022032522/56649d6a5503460f94a486e8/html5/thumbnails/53.jpg)
Image DenoisingExp Swap TRW-S
BP-S Our Our+EM
86163,26.13 89264,90.74 73383,529.60
526969,115.84 81820,294.72 81820,465.57
![Page 54: Hierarchical Graph Cuts for Semi-Metric Labeling M. Pawan Kumar Joint work with Daphne Koller](https://reader030.vdocuments.net/reader030/viewer/2022032522/56649d6a5503460f94a486e8/html5/thumbnails/54.jpg)
Stereo Reconstruction
![Page 55: Hierarchical Graph Cuts for Semi-Metric Labeling M. Pawan Kumar Joint work with Daphne Koller](https://reader030.vdocuments.net/reader030/viewer/2022032522/56649d6a5503460f94a486e8/html5/thumbnails/55.jpg)
Stereo ReconstructionExp Swap TRW-S
BP-S Our Our+EM
78776,12.07 97999,34.59 62777,263.28
126824,50.38 65116,152.74 65008,361.81
![Page 56: Hierarchical Graph Cuts for Semi-Metric Labeling M. Pawan Kumar Joint work with Daphne Koller](https://reader030.vdocuments.net/reader030/viewer/2022032522/56649d6a5503460f94a486e8/html5/thumbnails/56.jpg)
Scene Registration
![Page 57: Hierarchical Graph Cuts for Semi-Metric Labeling M. Pawan Kumar Joint work with Daphne Koller](https://reader030.vdocuments.net/reader030/viewer/2022032522/56649d6a5503460f94a486e8/html5/thumbnails/57.jpg)
Scene RegistrationExp Swap TRW-S
BP-S Our Our+EM
82036,1.66 83023,8.15 81118,1371.11
84396,218.04 81315,104.89 81258,373.60
![Page 58: Hierarchical Graph Cuts for Semi-Metric Labeling M. Pawan Kumar Joint work with Daphne Koller](https://reader030.vdocuments.net/reader030/viewer/2022032522/56649d6a5503460f94a486e8/html5/thumbnails/58.jpg)
Scene Registration
![Page 59: Hierarchical Graph Cuts for Semi-Metric Labeling M. Pawan Kumar Joint work with Daphne Koller](https://reader030.vdocuments.net/reader030/viewer/2022032522/56649d6a5503460f94a486e8/html5/thumbnails/59.jpg)
Scene RegistrationExp Swap TRW-S
BP-S Our Our+EM
68572,1.27 69767,2.78 67616,1058.25
70239,159.98 67682,73.61 67676,240.49
![Page 60: Hierarchical Graph Cuts for Semi-Metric Labeling M. Pawan Kumar Joint work with Daphne Koller](https://reader030.vdocuments.net/reader030/viewer/2022032522/56649d6a5503460f94a486e8/html5/thumbnails/60.jpg)
Scene Segmentation
Energy Accuracy Timing
Exp 302272 60.62 3.18
Swap 302389 60.60 3.73
TRW-S 302211 60.68 451.02
BP-S 310825 60.44 102.14
Our 302265 60.64 157.03
![Page 61: Hierarchical Graph Cuts for Semi-Metric Labeling M. Pawan Kumar Joint work with Daphne Koller](https://reader030.vdocuments.net/reader030/viewer/2022032522/56649d6a5503460f94a486e8/html5/thumbnails/61.jpg)
Future Work
• Tighter approximations for semi-metrics
• Higher-order potentials?
• Learning the parameters?
![Page 62: Hierarchical Graph Cuts for Semi-Metric Labeling M. Pawan Kumar Joint work with Daphne Koller](https://reader030.vdocuments.net/reader030/viewer/2022032522/56649d6a5503460f94a486e8/html5/thumbnails/62.jpg)
A Diffusion Algorithm for Upper Envelope Potentials
M. Pawan Kumar
Joint work with
Pushmeet Kohli
![Page 63: Hierarchical Graph Cuts for Semi-Metric Labeling M. Pawan Kumar Joint work with Daphne Koller](https://reader030.vdocuments.net/reader030/viewer/2022032522/56649d6a5503460f94a486e8/html5/thumbnails/63.jpg)
AimEfficient MAP estimation of sparse higher order potentials
V1 V2 … … …
… … … … …
… … … … …
… … … … Vn
![Page 64: Hierarchical Graph Cuts for Semi-Metric Labeling M. Pawan Kumar Joint work with Daphne Koller](https://reader030.vdocuments.net/reader030/viewer/2022032522/56649d6a5503460f94a486e8/html5/thumbnails/64.jpg)
AimEfficient MAP estimation of sparse higher order potentials
Z
In general, f(z) Lc
Some special cases computationally feasible
![Page 65: Hierarchical Graph Cuts for Semi-Metric Labeling M. Pawan Kumar Joint work with Daphne Koller](https://reader030.vdocuments.net/reader030/viewer/2022032522/56649d6a5503460f94a486e8/html5/thumbnails/65.jpg)
Lower Envelope Potentials
Z
mini z(i) + ∑aC za(i,f(a))
f(z) L’ Lc
![Page 66: Hierarchical Graph Cuts for Semi-Metric Labeling M. Pawan Kumar Joint work with Daphne Koller](https://reader030.vdocuments.net/reader030/viewer/2022032522/56649d6a5503460f94a486e8/html5/thumbnails/66.jpg)
Lower Envelope Potentials
ENERGY
![Page 67: Hierarchical Graph Cuts for Semi-Metric Labeling M. Pawan Kumar Joint work with Daphne Koller](https://reader030.vdocuments.net/reader030/viewer/2022032522/56649d6a5503460f94a486e8/html5/thumbnails/67.jpg)
Lower Envelope Potentials
ENERGY
![Page 68: Hierarchical Graph Cuts for Semi-Metric Labeling M. Pawan Kumar Joint work with Daphne Koller](https://reader030.vdocuments.net/reader030/viewer/2022032522/56649d6a5503460f94a486e8/html5/thumbnails/68.jpg)
Lower Envelope Potentials
ENERGY
![Page 69: Hierarchical Graph Cuts for Semi-Metric Labeling M. Pawan Kumar Joint work with Daphne Koller](https://reader030.vdocuments.net/reader030/viewer/2022032522/56649d6a5503460f94a486e8/html5/thumbnails/69.jpg)
Lower Envelope Potentials
mini z(i) + ∑aC za(i,f(a))
f(z) {0,1}
ENERGY
Robust Pn Model
![Page 70: Hierarchical Graph Cuts for Semi-Metric Labeling M. Pawan Kumar Joint work with Daphne Koller](https://reader030.vdocuments.net/reader030/viewer/2022032522/56649d6a5503460f94a486e8/html5/thumbnails/70.jpg)
Lower Envelope Potentials
+ ∑z (mini z(i) + ∑aC za(i,f(a))) f* = arg minf a(f(a)) + ab(f(a),f(b))
![Page 71: Hierarchical Graph Cuts for Semi-Metric Labeling M. Pawan Kumar Joint work with Daphne Koller](https://reader030.vdocuments.net/reader030/viewer/2022032522/56649d6a5503460f94a486e8/html5/thumbnails/71.jpg)
Lower Envelope Potentials
f* = arg minf a(f(a)) + ab(f(a),f(b))
f(z) L’
+ ∑z z(f(z)) + ∑aC za(f(z),f(a))
Use your favorite pairwise MRF algorithm
![Page 72: Hierarchical Graph Cuts for Semi-Metric Labeling M. Pawan Kumar Joint work with Daphne Koller](https://reader030.vdocuments.net/reader030/viewer/2022032522/56649d6a5503460f94a486e8/html5/thumbnails/72.jpg)
Upper Envelope Potentials
Z
maxi z(i) + ∑aC za(i,f(a))
f(z) L’ Lc
![Page 73: Hierarchical Graph Cuts for Semi-Metric Labeling M. Pawan Kumar Joint work with Daphne Koller](https://reader030.vdocuments.net/reader030/viewer/2022032522/56649d6a5503460f94a486e8/html5/thumbnails/73.jpg)
Upper Envelope Potentials
SilhouetteObject
Ray
Ray
Cameracenter
At least one voxel on the ray labeled ‘object’
![Page 74: Hierarchical Graph Cuts for Semi-Metric Labeling M. Pawan Kumar Joint work with Daphne Koller](https://reader030.vdocuments.net/reader030/viewer/2022032522/56649d6a5503460f94a486e8/html5/thumbnails/74.jpg)
Upper Envelope Potentials
SilhouetteObject
Ray
Ray
Cameracenter
maxi z(i) + ∑aC za(i,f(a))
f(z) {0,1}
![Page 75: Hierarchical Graph Cuts for Semi-Metric Labeling M. Pawan Kumar Joint work with Daphne Koller](https://reader030.vdocuments.net/reader030/viewer/2022032522/56649d6a5503460f94a486e8/html5/thumbnails/75.jpg)
Upper Envelope Potentials
+ ∑z (maxi z(i) + ∑aC za(i,f(a))) f* = arg minf a(f(a)) + ab(f(a),f(b))
![Page 76: Hierarchical Graph Cuts for Semi-Metric Labeling M. Pawan Kumar Joint work with Daphne Koller](https://reader030.vdocuments.net/reader030/viewer/2022032522/56649d6a5503460f94a486e8/html5/thumbnails/76.jpg)
Upper Envelope Potentials
+ ∑z tz
f* = arg minf a(f(a)) + ab(f(a),f(b))
tz ≥ z(i) + ∑aC tza(i)
tza(i) ≥ za(i,f(a))
LP Relaxation
![Page 77: Hierarchical Graph Cuts for Semi-Metric Labeling M. Pawan Kumar Joint work with Daphne Koller](https://reader030.vdocuments.net/reader030/viewer/2022032522/56649d6a5503460f94a486e8/html5/thumbnails/77.jpg)
Dual
max a mini a(i) + (a,b) mini,j ab(i,j)
+ z mini z(i) + (z,a) mini,j za(i,j)
∑iz(i) = 1∑jza(i,j) = z(i) za(i,j)≥ 0
a(i) = a(i)
ab(i,j) = ab(i,j)
z(i) = z(i)a(i)
za(i,j) = za(i,j)ab(i,j)
![Page 78: Hierarchical Graph Cuts for Semi-Metric Labeling M. Pawan Kumar Joint work with Daphne Koller](https://reader030.vdocuments.net/reader030/viewer/2022032522/56649d6a5503460f94a486e8/html5/thumbnails/78.jpg)
Dual Without Z
max a mini a(i) + (a,b) mini,j ab(i,j)
![Page 79: Hierarchical Graph Cuts for Semi-Metric Labeling M. Pawan Kumar Joint work with Daphne Koller](https://reader030.vdocuments.net/reader030/viewer/2022032522/56649d6a5503460f94a486e8/html5/thumbnails/79.jpg)
Diffusion
Va
3
1 0
2
Va
5
10 12
3
Va
4
2
0 2 3
![Page 80: Hierarchical Graph Cuts for Semi-Metric Labeling M. Pawan Kumar Joint work with Daphne Koller](https://reader030.vdocuments.net/reader030/viewer/2022032522/56649d6a5503460f94a486e8/html5/thumbnails/80.jpg)
Diffusion
Va
3
0 0
1
Va
0
5 9
0
Va
4
2
0
1
5
3
![Page 81: Hierarchical Graph Cuts for Semi-Metric Labeling M. Pawan Kumar Joint work with Daphne Koller](https://reader030.vdocuments.net/reader030/viewer/2022032522/56649d6a5503460f94a486e8/html5/thumbnails/81.jpg)
Diffusion
Va
3
0 0
1
Va
0
5 9
0
Va
3
2
3
2
3
2
![Page 82: Hierarchical Graph Cuts for Semi-Metric Labeling M. Pawan Kumar Joint work with Daphne Koller](https://reader030.vdocuments.net/reader030/viewer/2022032522/56649d6a5503460f94a486e8/html5/thumbnails/82.jpg)
Diffusion
Va
6
2 3
3
Va
3
8 11
2
Va
3
2
2 2 2
![Page 83: Hierarchical Graph Cuts for Semi-Metric Labeling M. Pawan Kumar Joint work with Daphne Koller](https://reader030.vdocuments.net/reader030/viewer/2022032522/56649d6a5503460f94a486e8/html5/thumbnails/83.jpg)
Diffusion for Auxiliary Variable
z
3
1 0
2
z
5
10 12
3
z
4
2
z(i) = ’z(i) + (z(i) - ’z(i))a(i)
za(i,j) = ’za(i,j) + (za(i,j) - ’za(i,j))za(i,j)
![Page 84: Hierarchical Graph Cuts for Semi-Metric Labeling M. Pawan Kumar Joint work with Daphne Koller](https://reader030.vdocuments.net/reader030/viewer/2022032522/56649d6a5503460f94a486e8/html5/thumbnails/84.jpg)
Diffusion for Auxiliary Variable
max ( mini z(i) + ∑a mini,j za(i,j) )
∑I z(i) = 1
z(i)≥ 0
∑j za(i,j) = z(i)
za(i,j)≥ 0
Solve for Expensive
![Page 85: Hierarchical Graph Cuts for Semi-Metric Labeling M. Pawan Kumar Joint work with Daphne Koller](https://reader030.vdocuments.net/reader030/viewer/2022032522/56649d6a5503460f94a486e8/html5/thumbnails/85.jpg)
Diffusion for Auxiliary Variable
max ( mini z(i) ) max ( mini,j za(i,j) )
∑I z(i) = 1
z(i)≥ 0
∑j za(i,j) = z(i) ?
∑ij za(i,j) = 1
za(i,j)≥ 0
![Page 86: Hierarchical Graph Cuts for Semi-Metric Labeling M. Pawan Kumar Joint work with Daphne Koller](https://reader030.vdocuments.net/reader030/viewer/2022032522/56649d6a5503460f94a486e8/html5/thumbnails/86.jpg)
Diffusion for Auxiliary Variable
max ( mini z(i) ) max ( mini,j za(i,j) )
∑I z(i) = 1
z(i)≥ 0
∑ij za(i,j) = 1
za(i,j)≥ 0
+ ∑z;iz(i) + ∑za;iza(i,j)
z;i + ∑aza;i = 0Fractional Packing Problem
![Page 87: Hierarchical Graph Cuts for Semi-Metric Labeling M. Pawan Kumar Joint work with Daphne Koller](https://reader030.vdocuments.net/reader030/viewer/2022032522/56649d6a5503460f94a486e8/html5/thumbnails/87.jpg)
Diffusion for Auxiliary Variable
max ( mini z(i) ) max ( mini,j za(i,j) )
∑I z(i) = 1
z(i)≥ 0
∑ij za(i,j) = 1
za(i,j)≥ 0
+ ∑z;iz(i) + ∑za;iza(i,j)
z;i + ∑aza;i = 0Plotkin, Shmoys and Tardos, 1995
![Page 88: Hierarchical Graph Cuts for Semi-Metric Labeling M. Pawan Kumar Joint work with Daphne Koller](https://reader030.vdocuments.net/reader030/viewer/2022032522/56649d6a5503460f94a486e8/html5/thumbnails/88.jpg)
Diffusion for Auxiliary Variable
z
3
1 0
2
z
5
10 12
3
z
4
2
Run Standard Diffusion on
![Page 89: Hierarchical Graph Cuts for Semi-Metric Labeling M. Pawan Kumar Joint work with Daphne Koller](https://reader030.vdocuments.net/reader030/viewer/2022032522/56649d6a5503460f94a486e8/html5/thumbnails/89.jpg)
The Algorithm
Choose a variable (random or auxiliary)
If random variable, run standard diffusion
If auxiliary variable, obtain and then run standard diffusion
Repeat till convergence
![Page 90: Hierarchical Graph Cuts for Semi-Metric Labeling M. Pawan Kumar Joint work with Daphne Koller](https://reader030.vdocuments.net/reader030/viewer/2022032522/56649d6a5503460f94a486e8/html5/thumbnails/90.jpg)
Future Work
• Write the code
• Do the experiments
• A better way to get ??