![Page 1: CS 468, spring 2012 · EMD is a metric when d(x,y) satisfies the triangle inequality. The Earth Mover's Distance as a Metric for Image Retrieval Rubner, Tomasi, and Guibas International](https://reader033.vdocuments.net/reader033/viewer/2022042108/5e87c14a624b44684d16343e/html5/thumbnails/1.jpg)
TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAAAAAAAAAAAAA
Justin Solomon Stanford University
![Page 2: CS 468, spring 2012 · EMD is a metric when d(x,y) satisfies the triangle inequality. The Earth Mover's Distance as a Metric for Image Retrieval Rubner, Tomasi, and Guibas International](https://reader033.vdocuments.net/reader033/viewer/2022042108/5e87c14a624b44684d16343e/html5/thumbnails/2.jpg)
NOT
COMPREHENSIVE
![Page 3: CS 468, spring 2012 · EMD is a metric when d(x,y) satisfies the triangle inequality. The Earth Mover's Distance as a Metric for Image Retrieval Rubner, Tomasi, and Guibas International](https://reader033.vdocuments.net/reader033/viewer/2022042108/5e87c14a624b44684d16343e/html5/thumbnails/3.jpg)
NOT
COMPREHENSIVE
Biased toward computational applications (and things I know about)
![Page 4: CS 468, spring 2012 · EMD is a metric when d(x,y) satisfies the triangle inequality. The Earth Mover's Distance as a Metric for Image Retrieval Rubner, Tomasi, and Guibas International](https://reader033.vdocuments.net/reader033/viewer/2022042108/5e87c14a624b44684d16343e/html5/thumbnails/4.jpg)
Which is closer, 1 or 2?
Query 1 2
![Page 5: CS 468, spring 2012 · EMD is a metric when d(x,y) satisfies the triangle inequality. The Earth Mover's Distance as a Metric for Image Retrieval Rubner, Tomasi, and Guibas International](https://reader033.vdocuments.net/reader033/viewer/2022042108/5e87c14a624b44684d16343e/html5/thumbnails/5.jpg)
Which is closer, 1 or 2?
Query 1 2
![Page 6: CS 468, spring 2012 · EMD is a metric when d(x,y) satisfies the triangle inequality. The Earth Mover's Distance as a Metric for Image Retrieval Rubner, Tomasi, and Guibas International](https://reader033.vdocuments.net/reader033/viewer/2022042108/5e87c14a624b44684d16343e/html5/thumbnails/6.jpg)
Which is closer, 1 or 2?
Query 1 2
p(x; y)
p1(x; y) p2(x; y)
![Page 7: CS 468, spring 2012 · EMD is a metric when d(x,y) satisfies the triangle inequality. The Earth Mover's Distance as a Metric for Image Retrieval Rubner, Tomasi, and Guibas International](https://reader033.vdocuments.net/reader033/viewer/2022042108/5e87c14a624b44684d16343e/html5/thumbnails/7.jpg)
¡
k ¢ k
p1(x) p2(x)
p1(x)¡ p2(x)
kp1 ¡ p2k
Lp norm KL divergence
![Page 8: CS 468, spring 2012 · EMD is a metric when d(x,y) satisfies the triangle inequality. The Earth Mover's Distance as a Metric for Image Retrieval Rubner, Tomasi, and Guibas International](https://reader033.vdocuments.net/reader033/viewer/2022042108/5e87c14a624b44684d16343e/html5/thumbnails/8.jpg)
Which is closer, 1 or 2?
Query 1 2
p(x; y)
p1(x; y) p2(x; y)
![Page 9: CS 468, spring 2012 · EMD is a metric when d(x,y) satisfies the triangle inequality. The Earth Mover's Distance as a Metric for Image Retrieval Rubner, Tomasi, and Guibas International](https://reader033.vdocuments.net/reader033/viewer/2022042108/5e87c14a624b44684d16343e/html5/thumbnails/9.jpg)
Neither!
Query 1 2
p(x; y)
p1(x; y) p2(x; y)
![Page 10: CS 468, spring 2012 · EMD is a metric when d(x,y) satisfies the triangle inequality. The Earth Mover's Distance as a Metric for Image Retrieval Rubner, Tomasi, and Guibas International](https://reader033.vdocuments.net/reader033/viewer/2022042108/5e87c14a624b44684d16343e/html5/thumbnails/10.jpg)
Measured overlap, not displacement.
![Page 11: CS 468, spring 2012 · EMD is a metric when d(x,y) satisfies the triangle inequality. The Earth Mover's Distance as a Metric for Image Retrieval Rubner, Tomasi, and Guibas International](https://reader033.vdocuments.net/reader033/viewer/2022042108/5e87c14a624b44684d16343e/html5/thumbnails/11.jpg)
Smaller bins worsen histogram distances
![Page 12: CS 468, spring 2012 · EMD is a metric when d(x,y) satisfies the triangle inequality. The Earth Mover's Distance as a Metric for Image Retrieval Rubner, Tomasi, and Guibas International](https://reader033.vdocuments.net/reader033/viewer/2022042108/5e87c14a624b44684d16343e/html5/thumbnails/12.jpg)
Permuting histogram bins has
no effect on these distances.
![Page 13: CS 468, spring 2012 · EMD is a metric when d(x,y) satisfies the triangle inequality. The Earth Mover's Distance as a Metric for Image Retrieval Rubner, Tomasi, and Guibas International](https://reader033.vdocuments.net/reader033/viewer/2022042108/5e87c14a624b44684d16343e/html5/thumbnails/13.jpg)
Compare in this direction
Not in this direction
![Page 14: CS 468, spring 2012 · EMD is a metric when d(x,y) satisfies the triangle inequality. The Earth Mover's Distance as a Metric for Image Retrieval Rubner, Tomasi, and Guibas International](https://reader033.vdocuments.net/reader033/viewer/2022042108/5e87c14a624b44684d16343e/html5/thumbnails/14.jpg)
Match mass from the distributions
![Page 15: CS 468, spring 2012 · EMD is a metric when d(x,y) satisfies the triangle inequality. The Earth Mover's Distance as a Metric for Image Retrieval Rubner, Tomasi, and Guibas International](https://reader033.vdocuments.net/reader033/viewer/2022042108/5e87c14a624b44684d16343e/html5/thumbnails/15.jpg)
Match mass from the distributions
Cost to move mass 𝒎 from 𝒙 to 𝒚:
𝒎 ⋅ 𝒅(𝒙, 𝒚) x
y
![Page 16: CS 468, spring 2012 · EMD is a metric when d(x,y) satisfies the triangle inequality. The Earth Mover's Distance as a Metric for Image Retrieval Rubner, Tomasi, and Guibas International](https://reader033.vdocuments.net/reader033/viewer/2022042108/5e87c14a624b44684d16343e/html5/thumbnails/16.jpg)
p q
𝒎 ⋅ 𝒅(𝒙, 𝒚)
Starts at 𝒑
Ends at 𝒒
Positive mass
![Page 17: CS 468, spring 2012 · EMD is a metric when d(x,y) satisfies the triangle inequality. The Earth Mover's Distance as a Metric for Image Retrieval Rubner, Tomasi, and Guibas International](https://reader033.vdocuments.net/reader033/viewer/2022042108/5e87c14a624b44684d16343e/html5/thumbnails/17.jpg)
EMD is a metric when d(x,y) satisfies the triangle inequality.
“The Earth Mover's Distance as a Metric for Image Retrieval” Rubner, Tomasi, and Guibas
International Journal of Computer Vision 40.2 (2000): 99—121.
![Page 18: CS 468, spring 2012 · EMD is a metric when d(x,y) satisfies the triangle inequality. The Earth Mover's Distance as a Metric for Image Retrieval Rubner, Tomasi, and Guibas International](https://reader033.vdocuments.net/reader033/viewer/2022042108/5e87c14a624b44684d16343e/html5/thumbnails/18.jpg)
Comparing histogram descriptorshttp://web.mit.edu/vondrick/ihog/
![Page 19: CS 468, spring 2012 · EMD is a metric when d(x,y) satisfies the triangle inequality. The Earth Mover's Distance as a Metric for Image Retrieval Rubner, Tomasi, and Guibas International](https://reader033.vdocuments.net/reader033/viewer/2022042108/5e87c14a624b44684d16343e/html5/thumbnails/19.jpg)
Quadratically-scaling LP
![Page 20: CS 468, spring 2012 · EMD is a metric when d(x,y) satisfies the triangle inequality. The Earth Mover's Distance as a Metric for Image Retrieval Rubner, Tomasi, and Guibas International](https://reader033.vdocuments.net/reader033/viewer/2022042108/5e87c14a624b44684d16343e/html5/thumbnails/20.jpg)
Multi-Commodity Flow
![Page 21: CS 468, spring 2012 · EMD is a metric when d(x,y) satisfies the triangle inequality. The Earth Mover's Distance as a Metric for Image Retrieval Rubner, Tomasi, and Guibas International](https://reader033.vdocuments.net/reader033/viewer/2022042108/5e87c14a624b44684d16343e/html5/thumbnails/21.jpg)
Multi-Commodity Flow
Useful conclusions:
1. Practical
Can do better than generic solvers.
![Page 22: CS 468, spring 2012 · EMD is a metric when d(x,y) satisfies the triangle inequality. The Earth Mover's Distance as a Metric for Image Retrieval Rubner, Tomasi, and Guibas International](https://reader033.vdocuments.net/reader033/viewer/2022042108/5e87c14a624b44684d16343e/html5/thumbnails/22.jpg)
Multi-Commodity Flow
Useful conclusions:
1. Practical
2. Theoretical
Can do better than generic solvers.
𝑻 ∈ 𝟎, 𝟏 𝒏×𝒏 usually contains 𝑶(𝒏) nonzeros.
![Page 23: CS 468, spring 2012 · EMD is a metric when d(x,y) satisfies the triangle inequality. The Earth Mover's Distance as a Metric for Image Retrieval Rubner, Tomasi, and Guibas International](https://reader033.vdocuments.net/reader033/viewer/2022042108/5e87c14a624b44684d16343e/html5/thumbnails/23.jpg)
Underlying map!
![Page 24: CS 468, spring 2012 · EMD is a metric when d(x,y) satisfies the triangle inequality. The Earth Mover's Distance as a Metric for Image Retrieval Rubner, Tomasi, and Guibas International](https://reader033.vdocuments.net/reader033/viewer/2022042108/5e87c14a624b44684d16343e/html5/thumbnails/24.jpg)
![Page 25: CS 468, spring 2012 · EMD is a metric when d(x,y) satisfies the triangle inequality. The Earth Mover's Distance as a Metric for Image Retrieval Rubner, Tomasi, and Guibas International](https://reader033.vdocuments.net/reader033/viewer/2022042108/5e87c14a624b44684d16343e/html5/thumbnails/25.jpg)
![Page 26: CS 468, spring 2012 · EMD is a metric when d(x,y) satisfies the triangle inequality. The Earth Mover's Distance as a Metric for Image Retrieval Rubner, Tomasi, and Guibas International](https://reader033.vdocuments.net/reader033/viewer/2022042108/5e87c14a624b44684d16343e/html5/thumbnails/26.jpg)
Thresholded ground distance Pele and Werman 2009
Linear/cyclic/grid domains Assorted theory papers
![Page 27: CS 468, spring 2012 · EMD is a metric when d(x,y) satisfies the triangle inequality. The Earth Mover's Distance as a Metric for Image Retrieval Rubner, Tomasi, and Guibas International](https://reader033.vdocuments.net/reader033/viewer/2022042108/5e87c14a624b44684d16343e/html5/thumbnails/27.jpg)
Monge-Kantorovich Problem
<pause>
![Page 28: CS 468, spring 2012 · EMD is a metric when d(x,y) satisfies the triangle inequality. The Earth Mover's Distance as a Metric for Image Retrieval Rubner, Tomasi, and Guibas International](https://reader033.vdocuments.net/reader033/viewer/2022042108/5e87c14a624b44684d16343e/html5/thumbnails/28.jpg)
Beware: Confusing notation!
![Page 29: CS 468, spring 2012 · EMD is a metric when d(x,y) satisfies the triangle inequality. The Earth Mover's Distance as a Metric for Image Retrieval Rubner, Tomasi, and Guibas International](https://reader033.vdocuments.net/reader033/viewer/2022042108/5e87c14a624b44684d16343e/html5/thumbnails/29.jpg)
Monge-Kantorovich Problem
min¼2¦(¹;º)
ZZ
X£Xc(x; y)d¼(x; y)
![Page 30: CS 468, spring 2012 · EMD is a metric when d(x,y) satisfies the triangle inequality. The Earth Mover's Distance as a Metric for Image Retrieval Rubner, Tomasi, and Guibas International](https://reader033.vdocuments.net/reader033/viewer/2022042108/5e87c14a624b44684d16343e/html5/thumbnails/30.jpg)
Monge-Kantorovich Problem
¹(U) = ¼(U £X)
º(V ) = ¼(X £ V )8U;V µX
Measure coupling
min¼2¦(¹;º)
ZZ
X£Xc(x; y)d¼(x; y)
![Page 31: CS 468, spring 2012 · EMD is a metric when d(x,y) satisfies the triangle inequality. The Earth Mover's Distance as a Metric for Image Retrieval Rubner, Tomasi, and Guibas International](https://reader033.vdocuments.net/reader033/viewer/2022042108/5e87c14a624b44684d16343e/html5/thumbnails/31.jpg)
Wp(¹; º) ´ min¼2¦(¹;º)
µZZ
X£Xd(x; y)p d¼(x; y)
¶1=p
Ground distance from shortest path
Shortest path distance
Expectation
![Page 32: CS 468, spring 2012 · EMD is a metric when d(x,y) satisfies the triangle inequality. The Earth Mover's Distance as a Metric for Image Retrieval Rubner, Tomasi, and Guibas International](https://reader033.vdocuments.net/reader033/viewer/2022042108/5e87c14a624b44684d16343e/html5/thumbnails/32.jpg)
http://realgl.blogspot.com/2013/01/pdf-cdf-inv-cdf.html
PDF [CDF] CDF-1
W1(¹; º) = kCDF(¹)¡CDF(º)k1W2(¹; º) = kCDF¡1(¹)¡CDF¡1(º)k2
![Page 33: CS 468, spring 2012 · EMD is a metric when d(x,y) satisfies the triangle inequality. The Earth Mover's Distance as a Metric for Image Retrieval Rubner, Tomasi, and Guibas International](https://reader033.vdocuments.net/reader033/viewer/2022042108/5e87c14a624b44684d16343e/html5/thumbnails/33.jpg)
Advect distributions using minimal work.
Benamou & Brenier
![Page 34: CS 468, spring 2012 · EMD is a metric when d(x,y) satisfies the triangle inequality. The Earth Mover's Distance as a Metric for Image Retrieval Rubner, Tomasi, and Guibas International](https://reader033.vdocuments.net/reader033/viewer/2022042108/5e87c14a624b44684d16343e/html5/thumbnails/34.jpg)
Similar to graph problem
Total work
Advects from 𝝆𝟎 to 𝝆𝟏
“Beckmann problem”
![Page 35: CS 468, spring 2012 · EMD is a metric when d(x,y) satisfies the triangle inequality. The Earth Mover's Distance as a Metric for Image Retrieval Rubner, Tomasi, and Guibas International](https://reader033.vdocuments.net/reader033/viewer/2022042108/5e87c14a624b44684d16343e/html5/thumbnails/35.jpg)
J(x) =rf(x) +R¢rg(x)
Curl-free Div-free
SIGGRAPH 2014
![Page 36: CS 468, spring 2012 · EMD is a metric when d(x,y) satisfies the triangle inequality. The Earth Mover's Distance as a Metric for Image Retrieval Rubner, Tomasi, and Guibas International](https://reader033.vdocuments.net/reader033/viewer/2022042108/5e87c14a624b44684d16343e/html5/thumbnails/36.jpg)
Mass moves along shortest paths
W2
“Explains” shortest path.
Image from “Optimal Transport with Proximal Splitting” (Papadakis, Peyré, and Oudet)
![Page 37: CS 468, spring 2012 · EMD is a metric when d(x,y) satisfies the triangle inequality. The Earth Mover's Distance as a Metric for Image Retrieval Rubner, Tomasi, and Guibas International](https://reader033.vdocuments.net/reader033/viewer/2022042108/5e87c14a624b44684d16343e/html5/thumbnails/37.jpg)
Consider set of distributions as a manifold
Tangent spaces from advection
Geodesics from displacement interpolation
![Page 38: CS 468, spring 2012 · EMD is a metric when d(x,y) satisfies the triangle inequality. The Earth Mover's Distance as a Metric for Image Retrieval Rubner, Tomasi, and Guibas International](https://reader033.vdocuments.net/reader033/viewer/2022042108/5e87c14a624b44684d16343e/html5/thumbnails/38.jpg)
Learning
𝑣 ∈ 𝑉0
𝑣 ∉ 𝑉0 “Wasserstein Propagation for Semi-Supervised Learning” (Solomon et al.)
“Fast Computation of Wasserstein Barycenters” (Cuturi and Doucet)
![Page 39: CS 468, spring 2012 · EMD is a metric when d(x,y) satisfies the triangle inequality. The Earth Mover's Distance as a Metric for Image Retrieval Rubner, Tomasi, and Guibas International](https://reader033.vdocuments.net/reader033/viewer/2022042108/5e87c14a624b44684d16343e/html5/thumbnails/39.jpg)
Morphing and registration
“Displacement Interpolation Using Lagrangian Mass Transport” (Bonneel et al.)
“An Optimal Transport Approach to Robust Reconstruction and Simplification of 2D Shapes” (de Goes et al.)
![Page 40: CS 468, spring 2012 · EMD is a metric when d(x,y) satisfies the triangle inequality. The Earth Mover's Distance as a Metric for Image Retrieval Rubner, Tomasi, and Guibas International](https://reader033.vdocuments.net/reader033/viewer/2022042108/5e87c14a624b44684d16343e/html5/thumbnails/40.jpg)
Graphics
“Earth Mover’s Distances on Discrete Surfaces” (Solomon et al.)
“Blue Noise Through Optimal Transport” (de Goes et al.)
![Page 41: CS 468, spring 2012 · EMD is a metric when d(x,y) satisfies the triangle inequality. The Earth Mover's Distance as a Metric for Image Retrieval Rubner, Tomasi, and Guibas International](https://reader033.vdocuments.net/reader033/viewer/2022042108/5e87c14a624b44684d16343e/html5/thumbnails/41.jpg)
Vision and image processing
“Geodesic Shape Retrieval via Optimal Mass Transport” (Rabin, Peyré, and Cohen)
“Adaptive Color Transfer with Relaxed Optimal Transport” (Rabin, Ferradans, and Papadakis)
![Page 42: CS 468, spring 2012 · EMD is a metric when d(x,y) satisfies the triangle inequality. The Earth Mover's Distance as a Metric for Image Retrieval Rubner, Tomasi, and Guibas International](https://reader033.vdocuments.net/reader033/viewer/2022042108/5e87c14a624b44684d16343e/html5/thumbnails/42.jpg)
Learning applications Variational methods, metric learning, …
Efficient computation in L2 case
![Page 43: CS 468, spring 2012 · EMD is a metric when d(x,y) satisfies the triangle inequality. The Earth Mover's Distance as a Metric for Image Retrieval Rubner, Tomasi, and Guibas International](https://reader033.vdocuments.net/reader033/viewer/2022042108/5e87c14a624b44684d16343e/html5/thumbnails/43.jpg)
TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAAAAAAAAAAAAA
Questions?