some recent methods in non-rigid shape matching, with and … · some recent methods in non-rigid...
TRANSCRIPT
![Page 1: Some recent methods in non-rigid shape matching, with and … · Some recent methods in non-rigid shape matching, with and without learning GAMES 2019 webinar Maks Ovsjanikov Based](https://reader035.vdocuments.net/reader035/viewer/2022081617/60476439d2e1202bf01d2cb8/html5/thumbnails/1.jpg)
Some recent methods in non-rigid shape matching, with and without learning
GAMES 2019 webinar
Maks Ovsjanikov
Based on joint work with: E. Corman, Michael Bronstein, Emanuele Rodolà, Justin Solomon, Adrien Butscher, Mirela Ben-Chen, Leonidas Guibas, Frederic Chazal ….
![Page 2: Some recent methods in non-rigid shape matching, with and … · Some recent methods in non-rigid shape matching, with and without learning GAMES 2019 webinar Maks Ovsjanikov Based](https://reader035.vdocuments.net/reader035/viewer/2022081617/60476439d2e1202bf01d2cb8/html5/thumbnails/2.jpg)
My Background
• 2005 – 2010: PhD from Stanford University (advisor Leonidas Guibas).
• 2011: engineer at Google Inc. • Since 2012 Professor in the Computer Science Department at
Ecole Polytechnique in France.
![Page 3: Some recent methods in non-rigid shape matching, with and … · Some recent methods in non-rigid shape matching, with and without learning GAMES 2019 webinar Maks Ovsjanikov Based](https://reader035.vdocuments.net/reader035/viewer/2022081617/60476439d2e1202bf01d2cb8/html5/thumbnails/3.jpg)
University
• Ecole Polytechnique: located very near Paris. • Ranked 2nd best small university in the world in 2019. • Est. 1794. Students and professors such as Ampère, Cauchy,
Fourier, Hermite, Lagrange, Monge, Poincaré, Poisson…• Very international campus.
![Page 4: Some recent methods in non-rigid shape matching, with and … · Some recent methods in non-rigid shape matching, with and without learning GAMES 2019 webinar Maks Ovsjanikov Based](https://reader035.vdocuments.net/reader035/viewer/2022081617/60476439d2e1202bf01d2cb8/html5/thumbnails/4.jpg)
Our group
• Currently 5 PhD students and 2 PostDocs.• Part of the larger STREAM team dedicated to visual
computing with 3 other professors.• Many international collaborations: Stanford, MIT,
UCL, KAUST, Univ. Toronto, Univ. Rome, etc.• Funding for PhD students and postdocs !
![Page 5: Some recent methods in non-rigid shape matching, with and … · Some recent methods in non-rigid shape matching, with and without learning GAMES 2019 webinar Maks Ovsjanikov Based](https://reader035.vdocuments.net/reader035/viewer/2022081617/60476439d2e1202bf01d2cb8/html5/thumbnails/5.jpg)
General Overview
Overall Objective: Create tools for computing and analyzing mappingsbetween geometric objects.
![Page 6: Some recent methods in non-rigid shape matching, with and … · Some recent methods in non-rigid shape matching, with and without learning GAMES 2019 webinar Maks Ovsjanikov Based](https://reader035.vdocuments.net/reader035/viewer/2022081617/60476439d2e1202bf01d2cb8/html5/thumbnails/6.jpg)
Talk Overview
Course Notes:
[Related] Course Website:http://www.lix.polytechnique.fr/~maks/fmaps_SIG17_course/
Linked from the website. Or useAttention: (significantly) more material than in the lectures
Sample Code:See Sample Code link on the website.
or http://bit.do/fmaps2017
http://bit.do/fmaps2017_notes
Demo code for basic operations.6
![Page 7: Some recent methods in non-rigid shape matching, with and … · Some recent methods in non-rigid shape matching, with and without learning GAMES 2019 webinar Maks Ovsjanikov Based](https://reader035.vdocuments.net/reader035/viewer/2022081617/60476439d2e1202bf01d2cb8/html5/thumbnails/7.jpg)
Talk Overview
Motivation and Problem Taxonomy
Rigid Matching: ICP
Functional Map representation, properties
Open problems, Q&A
Basic pipeline for non-rigid matching
Recent extensions, improvements
![Page 8: Some recent methods in non-rigid shape matching, with and … · Some recent methods in non-rigid shape matching, with and without learning GAMES 2019 webinar Maks Ovsjanikov Based](https://reader035.vdocuments.net/reader035/viewer/2022081617/60476439d2e1202bf01d2cb8/html5/thumbnails/8.jpg)
What is a Shape?
Discrete: a graph embedded in 3D (triangle mesh).
Continuous: a surface embedded in 3D.
• Connected.• Manifold.• Without Boundary.
Common assumptions:
![Page 9: Some recent methods in non-rigid shape matching, with and … · Some recent methods in non-rigid shape matching, with and without learning GAMES 2019 webinar Maks Ovsjanikov Based](https://reader035.vdocuments.net/reader035/viewer/2022081617/60476439d2e1202bf01d2cb8/html5/thumbnails/9.jpg)
What is a Shape?
5k – 200k triangles
Shapes from the FAUST, SCAPE, and TOSCA datasets
Discrete: a graph embedded in 3D (triangle mesh).
Continuous: a surface embedded in 3D.
![Page 10: Some recent methods in non-rigid shape matching, with and … · Some recent methods in non-rigid shape matching, with and without learning GAMES 2019 webinar Maks Ovsjanikov Based](https://reader035.vdocuments.net/reader035/viewer/2022081617/60476439d2e1202bf01d2cb8/html5/thumbnails/10.jpg)
Overall Goal
Given two shapes, find correspondences between them.
![Page 11: Some recent methods in non-rigid shape matching, with and … · Some recent methods in non-rigid shape matching, with and without learning GAMES 2019 webinar Maks Ovsjanikov Based](https://reader035.vdocuments.net/reader035/viewer/2022081617/60476439d2e1202bf01d2cb8/html5/thumbnails/11.jpg)
Overall Goal
Given two shapes, find correspondences between them.
Finding the best map between a pair of shapes.
![Page 12: Some recent methods in non-rigid shape matching, with and … · Some recent methods in non-rigid shape matching, with and without learning GAMES 2019 webinar Maks Ovsjanikov Based](https://reader035.vdocuments.net/reader035/viewer/2022081617/60476439d2e1202bf01d2cb8/html5/thumbnails/12.jpg)
Problem Taxonomy
Local vs. Globalrefinement (e.g. ICP) | alignment (search) . .
Rigid vs. Deformablerotation, translation | general deformation.
Semi vs. Fully Automaticgiven landmarks | a priori model
Learning-Based vs. Directknown examples | unseen data
![Page 13: Some recent methods in non-rigid shape matching, with and … · Some recent methods in non-rigid shape matching, with and without learning GAMES 2019 webinar Maks Ovsjanikov Based](https://reader035.vdocuments.net/reader035/viewer/2022081617/60476439d2e1202bf01d2cb8/html5/thumbnails/13.jpg)
Problem Taxonomy
Local vs. Globalrefinement (e.g. ICP) | alignment (search) . .
Rigid vs. Deformablerotation, translation | general deformation.
Semi vs. Fully Automaticgiven landmarks | a priori model
Learning-Based vs. Directknown examples | unseen data
![Page 14: Some recent methods in non-rigid shape matching, with and … · Some recent methods in non-rigid shape matching, with and without learning GAMES 2019 webinar Maks Ovsjanikov Based](https://reader035.vdocuments.net/reader035/viewer/2022081617/60476439d2e1202bf01d2cb8/html5/thumbnails/14.jpg)
Why Shape Matching?
Given a correspondence, we can transfer:
texture and parametrization
segmentation and labels
deformation
Other applications: shape interpolation, reconstruction ... Sumner et al. ‘04.
14
![Page 15: Some recent methods in non-rigid shape matching, with and … · Some recent methods in non-rigid shape matching, with and without learning GAMES 2019 webinar Maks Ovsjanikov Based](https://reader035.vdocuments.net/reader035/viewer/2022081617/60476439d2e1202bf01d2cb8/html5/thumbnails/15.jpg)
Rigid Shape Matching
• The unknowns are the rotation/translation parameters of the source onto the target shape.
• Given a pair of shapes, find the optimal Rigid Alignment between them.
![Page 16: Some recent methods in non-rigid shape matching, with and … · Some recent methods in non-rigid shape matching, with and without learning GAMES 2019 webinar Maks Ovsjanikov Based](https://reader035.vdocuments.net/reader035/viewer/2022081617/60476439d2e1202bf01d2cb8/html5/thumbnails/16.jpg)
Iterative Closest Point (ICP)
• Classical approach: iterate between finding correspondences and finding the transformation:
example in 2DM N
Given a pair of shapes, and , iterate:1. For each find nearest neighbor .2. Find optimal transformation minimizing:
argminR,t
!
i
∥Rxi + t− yi∥2
2
M N
xi ∈ M yi ∈ N
Besl, McKay (1992). "A Method for Registration of 3-D Shapes".
![Page 17: Some recent methods in non-rigid shape matching, with and … · Some recent methods in non-rigid shape matching, with and without learning GAMES 2019 webinar Maks Ovsjanikov Based](https://reader035.vdocuments.net/reader035/viewer/2022081617/60476439d2e1202bf01d2cb8/html5/thumbnails/17.jpg)
Iterative Closest Point
• Classical approach: iterate between finding correspondences and finding the transformation:
M N
Given a pair of shapes, and , iterate:1. For each find nearest neighbor .2. Find optimal transformation minimizing:
argminR,t
!
i
∥Rxi + t− yi∥2
2
M N
xi ∈ M yi ∈ N
![Page 18: Some recent methods in non-rigid shape matching, with and … · Some recent methods in non-rigid shape matching, with and without learning GAMES 2019 webinar Maks Ovsjanikov Based](https://reader035.vdocuments.net/reader035/viewer/2022081617/60476439d2e1202bf01d2cb8/html5/thumbnails/18.jpg)
Iterative Closest Point
• Classical approach: iterate between finding correspondences and finding the transformation:
M N
Given a pair of shapes, and , iterate:1. For each find nearest neighbor .2. Find optimal transformation minimizing:
argminR,t
!
i
∥Rxi + t− yi∥2
2
M N
xi ∈ M yi ∈ N
![Page 19: Some recent methods in non-rigid shape matching, with and … · Some recent methods in non-rigid shape matching, with and without learning GAMES 2019 webinar Maks Ovsjanikov Based](https://reader035.vdocuments.net/reader035/viewer/2022081617/60476439d2e1202bf01d2cb8/html5/thumbnails/19.jpg)
Iterative Closest Point
• Classical approach: iterate between finding correspondences and finding the transformation:
Given a pair of shapes, and , iterate:1. For each find nearest neighbor .2. Find optimal transformation minimizing:
argminR,t
!
i
∥Rxi + t− yi∥2
2
M N
xi ∈ M yi ∈ N
M N
![Page 20: Some recent methods in non-rigid shape matching, with and … · Some recent methods in non-rigid shape matching, with and without learning GAMES 2019 webinar Maks Ovsjanikov Based](https://reader035.vdocuments.net/reader035/viewer/2022081617/60476439d2e1202bf01d2cb8/html5/thumbnails/20.jpg)
Iterative Closest Point
• Classical approach: iterate between finding correspondences and finding the transformation:
Given a pair of shapes, and , iterate:1. For each find nearest neighbor .2. Find optimal transformation minimizing:
argminR,t
!
i
∥Rxi + t− yi∥2
2
M N
xi ∈ M yi ∈ N
M N
![Page 21: Some recent methods in non-rigid shape matching, with and … · Some recent methods in non-rigid shape matching, with and without learning GAMES 2019 webinar Maks Ovsjanikov Based](https://reader035.vdocuments.net/reader035/viewer/2022081617/60476439d2e1202bf01d2cb8/html5/thumbnails/21.jpg)
1. Finding nearest neighbors: can be done with space-partitioning data structures (e.g., KD-tree).
2. Finding the optimal transformation minimizing:
Iterative Closest Point
• Classical approach: iterate between finding correspondences and finding the transformation:
Can be done efficiently via SVD decomposition.
argminR,t
!
i
∥Rxi + t− yi∥2
2
M
N
Arun et al., Least-Squares Fitting of Two 3-D Point Sets
![Page 22: Some recent methods in non-rigid shape matching, with and … · Some recent methods in non-rigid shape matching, with and without learning GAMES 2019 webinar Maks Ovsjanikov Based](https://reader035.vdocuments.net/reader035/viewer/2022081617/60476439d2e1202bf01d2cb8/html5/thumbnails/22.jpg)
Non-Rigid Shape Matching
Unlike rigid matching with rotation/translation, there is no compact representation to optimize for in non-rigid matching.
22
![Page 23: Some recent methods in non-rigid shape matching, with and … · Some recent methods in non-rigid shape matching, with and without learning GAMES 2019 webinar Maks Ovsjanikov Based](https://reader035.vdocuments.net/reader035/viewer/2022081617/60476439d2e1202bf01d2cb8/html5/thumbnails/23.jpg)
Non-Rigid Shape Matching
What does it mean for a correspondence to be “good”?How to compute it efficiently in practice?
Main Questions:
23
![Page 24: Some recent methods in non-rigid shape matching, with and … · Some recent methods in non-rigid shape matching, with and without learning GAMES 2019 webinar Maks Ovsjanikov Based](https://reader035.vdocuments.net/reader035/viewer/2022081617/60476439d2e1202bf01d2cb8/html5/thumbnails/24.jpg)
Isometric Shape Matching
Good maps must preserve geodesic distances.Deformation Model:
Geodesic: length of shortest path lying entirely on the surface.
dM(x, y)
dN (T (x), T (y))
M
N
24
![Page 25: Some recent methods in non-rigid shape matching, with and … · Some recent methods in non-rigid shape matching, with and without learning GAMES 2019 webinar Maks Ovsjanikov Based](https://reader035.vdocuments.net/reader035/viewer/2022081617/60476439d2e1202bf01d2cb8/html5/thumbnails/25.jpg)
Isometric Shape Matching
Approach:
Find the point mapping by minimizing the distance distortion:
The unknowns are point correspondences.
Topt = argminT
!
x,y
∥dM(x, y)− dN (T (x), T (y))∥
dM(x, y)
dN (T (x), T (y))
M
N
![Page 26: Some recent methods in non-rigid shape matching, with and … · Some recent methods in non-rigid shape matching, with and without learning GAMES 2019 webinar Maks Ovsjanikov Based](https://reader035.vdocuments.net/reader035/viewer/2022081617/60476439d2e1202bf01d2cb8/html5/thumbnails/26.jpg)
Isometric Shape Matching
Approach:
The space of possible solutions is highly non-linear, non-convex.
Problem:
Find the point mapping by minimizing the distance distortion:
Topt = argminT
!
x,y
∥dM(x, y)− dN (T (x), T (y))∥
dM(x, y)
dN (T (x), T (y))
M
N
![Page 27: Some recent methods in non-rigid shape matching, with and … · Some recent methods in non-rigid shape matching, with and without learning GAMES 2019 webinar Maks Ovsjanikov Based](https://reader035.vdocuments.net/reader035/viewer/2022081617/60476439d2e1202bf01d2cb8/html5/thumbnails/27.jpg)
Functional Map Representation
We would like to define a representation of shape maps that is more amenable to direct optimization.
1. A compact representation for “natural” maps.
2. Inherently global and multi-scale.
3. Handles uncertainty and ambiguity gracefully.
4. Allows efficient manipulations (averaging, composition).
5. Leads to simple (linear) optimization problems.
27
![Page 28: Some recent methods in non-rigid shape matching, with and … · Some recent methods in non-rigid shape matching, with and without learning GAMES 2019 webinar Maks Ovsjanikov Based](https://reader035.vdocuments.net/reader035/viewer/2022081617/60476439d2e1202bf01d2cb8/html5/thumbnails/28.jpg)
Functional Approach to Mappings
Given two shapes and a pointwise map
The map induces a functional correspondence:TF (f) = g, where g = f ◦ T
T : N → M
M
NT
T
28Functional maps: a flexible representation of maps between shapes, O., Ben-Chen, Solomon, Butscher, Guibas, SIGGRAPH 2012
![Page 29: Some recent methods in non-rigid shape matching, with and … · Some recent methods in non-rigid shape matching, with and without learning GAMES 2019 webinar Maks Ovsjanikov Based](https://reader035.vdocuments.net/reader035/viewer/2022081617/60476439d2e1202bf01d2cb8/html5/thumbnails/29.jpg)
Functional Approach to Mappings
f : M → RTF
TF (f) = g : N → R
The map induces a functional correspondence:T
29
TF (f) = g, where g = f ◦ TFunctional maps: a flexible representation of maps between shapes, O., Ben-Chen, Solomon, Butscher, Guibas, SIGGRAPH 2012
Given two shapes and a pointwise map T : N → M
![Page 30: Some recent methods in non-rigid shape matching, with and … · Some recent methods in non-rigid shape matching, with and without learning GAMES 2019 webinar Maks Ovsjanikov Based](https://reader035.vdocuments.net/reader035/viewer/2022081617/60476439d2e1202bf01d2cb8/html5/thumbnails/30.jpg)
Functional Approach to Mappings
f : M → RTF
TF (f) = g : N → R
The map induces a functional correspondence:T
30
TF (f) = g, where g = f ◦ TFunctional maps: a flexible representation of maps between shapes, O., Ben-Chen, Solomon, Butscher, Guibas, SIGGRAPH 2012
Given two shapes and a pointwise map T : N → M
![Page 31: Some recent methods in non-rigid shape matching, with and … · Some recent methods in non-rigid shape matching, with and without learning GAMES 2019 webinar Maks Ovsjanikov Based](https://reader035.vdocuments.net/reader035/viewer/2022081617/60476439d2e1202bf01d2cb8/html5/thumbnails/31.jpg)
Functional Approach to Mappings
The induced functional correspondence is linear:
f : M → RTF
TF (f) = g : N → R
TF (α1f1 + α2f2) = α1TF (f1) + α2TF (f2)31
Given two shapes and a pointwise map T : N → M
![Page 32: Some recent methods in non-rigid shape matching, with and … · Some recent methods in non-rigid shape matching, with and without learning GAMES 2019 webinar Maks Ovsjanikov Based](https://reader035.vdocuments.net/reader035/viewer/2022081617/60476439d2e1202bf01d2cb8/html5/thumbnails/32.jpg)
Functional Map Representation
The induced functional correspondence is complete.
f : M → RTF
TF (f) = g : N → R
32
Given two shapes and a pointwise map T : N → M
![Page 33: Some recent methods in non-rigid shape matching, with and … · Some recent methods in non-rigid shape matching, with and without learning GAMES 2019 webinar Maks Ovsjanikov Based](https://reader035.vdocuments.net/reader035/viewer/2022081617/60476439d2e1202bf01d2cb8/html5/thumbnails/33.jpg)
Observation
Express both and in terms of basis functions:f TF (f)
Since is linear, there is a linear transformation from to . TF {ai} {bj}
M
f : M → R
g : N → RTF
N
f =
!
i
aiφMi
Assume that both: f ∈ L2(M), g ∈ L2(N )
g = TF (f) =!
j
bjφNj
![Page 34: Some recent methods in non-rigid shape matching, with and … · Some recent methods in non-rigid shape matching, with and without learning GAMES 2019 webinar Maks Ovsjanikov Based](https://reader035.vdocuments.net/reader035/viewer/2022081617/60476439d2e1202bf01d2cb8/html5/thumbnails/34.jpg)
Functional Map Representation
Eigenfunctions of the Laplace-Beltrami operator:
Generalization of Fourier bases to surfaces.
Ordered by eigenvalues and provide a natural notion of scale.
λ0 = 0 λ1 = 2.6 λ2 = 3.4 λ3 = 5.1 λ4 = 7.6
∆φi = λiφi
Choice of Basis:
34
∆(f) = −div∇(f)
![Page 35: Some recent methods in non-rigid shape matching, with and … · Some recent methods in non-rigid shape matching, with and without learning GAMES 2019 webinar Maks Ovsjanikov Based](https://reader035.vdocuments.net/reader035/viewer/2022081617/60476439d2e1202bf01d2cb8/html5/thumbnails/35.jpg)
Functional Map Representation
Eigenfunctions of the Laplace-Beltrami operator:
Generalization of Fourier bases to surfaces. Form an orthonormal basis for .
Ordered by eigenvalues and provide a natural notion of scale.
∆φi = λiφi
Choice of Basis:
Can be computed efficiently, with a sparse matrix eigensolver.
35
L2(M)
![Page 36: Some recent methods in non-rigid shape matching, with and … · Some recent methods in non-rigid shape matching, with and without learning GAMES 2019 webinar Maks Ovsjanikov Based](https://reader035.vdocuments.net/reader035/viewer/2022081617/60476439d2e1202bf01d2cb8/html5/thumbnails/36.jpg)
Observation
Express both and in terms of basis functions:f TF (f)
Since is linear, there is a linear transformation from to . TF {ai} {bj}
M
f : M → R
g : N → RTF
N
f =
!
i
aiφMi
g = TF (f) =!
i
biφNi
![Page 37: Some recent methods in non-rigid shape matching, with and … · Some recent methods in non-rigid shape matching, with and without learning GAMES 2019 webinar Maks Ovsjanikov Based](https://reader035.vdocuments.net/reader035/viewer/2022081617/60476439d2e1202bf01d2cb8/html5/thumbnails/37.jpg)
Functional Map Representation
Since the functional mapping TF is linear:
TF can be represented as a matrix C, given a choice of basis for function spaces.
TF (α1f1 + α2f2) = α1TF (f1) + α2TF (f2)
37Functional maps: a flexible representation of maps between shapes, O., Ben-Chen, Solomon, Butscher, Guibas, SIGGRAPH 2012
TF
=
![Page 38: Some recent methods in non-rigid shape matching, with and … · Some recent methods in non-rigid shape matching, with and without learning GAMES 2019 webinar Maks Ovsjanikov Based](https://reader035.vdocuments.net/reader035/viewer/2022081617/60476439d2e1202bf01d2cb8/html5/thumbnails/38.jpg)
Functional Map Definition
Functional map: matrix C that translates coefficients from to . ΦM ΦN
38
![Page 39: Some recent methods in non-rigid shape matching, with and … · Some recent methods in non-rigid shape matching, with and without learning GAMES 2019 webinar Maks Ovsjanikov Based](https://reader035.vdocuments.net/reader035/viewer/2022081617/60476439d2e1202bf01d2cb8/html5/thumbnails/39.jpg)
Example Maps in a Reduced Basis
Triangle meshes with pre-computed pointwise maps
“Good” maps are close to being diagonal
39
(a) source (b) ground-truth map (c) left-to-right map (d) head-to-tail map
Try fmap_computation_demo on the course website
![Page 40: Some recent methods in non-rigid shape matching, with and … · Some recent methods in non-rigid shape matching, with and without learning GAMES 2019 webinar Maks Ovsjanikov Based](https://reader035.vdocuments.net/reader035/viewer/2022081617/60476439d2e1202bf01d2cb8/html5/thumbnails/40.jpg)
Reconstructing from LB basis
Map reconstruction error using a fixed size matrix.
40
0.5
1
1.5
2
2.5
3
3.5
4
4.5
reco
nstr
uctio
n er
ror
Number of basis (eigen)-functions
27.9k vertices
Tryfmap_reconstruction_demo on the course website
![Page 41: Some recent methods in non-rigid shape matching, with and … · Some recent methods in non-rigid shape matching, with and without learning GAMES 2019 webinar Maks Ovsjanikov Based](https://reader035.vdocuments.net/reader035/viewer/2022081617/60476439d2e1202bf01d2cb8/html5/thumbnails/41.jpg)
Functional Map algebra
1. Map composition becomes matrix multiplication.
2. Map inversion is matrix inversion (in fact, transpose).
3. Algebraic operations on functional maps are possible.
E.g. interpolating between two maps with
C = αC1 +(1−α)C2.
41
![Page 42: Some recent methods in non-rigid shape matching, with and … · Some recent methods in non-rigid shape matching, with and without learning GAMES 2019 webinar Maks Ovsjanikov Based](https://reader035.vdocuments.net/reader035/viewer/2022081617/60476439d2e1202bf01d2cb8/html5/thumbnails/42.jpg)
Talk Overview
Motivation and Problem Taxonomy
Rigid Matching: ICP
Functional Map Representation, properties
Open Problems, Q&A
Basic pipeline for non-rigid matching
Extensions, Improvements
![Page 43: Some recent methods in non-rigid shape matching, with and … · Some recent methods in non-rigid shape matching, with and without learning GAMES 2019 webinar Maks Ovsjanikov Based](https://reader035.vdocuments.net/reader035/viewer/2022081617/60476439d2e1202bf01d2cb8/html5/thumbnails/43.jpg)
In practice we do not know C. Given two objects our goal is to find the correspondence.
How can the functional representation help to compute the map in practice?
Shape Matching
?
![Page 44: Some recent methods in non-rigid shape matching, with and … · Some recent methods in non-rigid shape matching, with and without learning GAMES 2019 webinar Maks Ovsjanikov Based](https://reader035.vdocuments.net/reader035/viewer/2022081617/60476439d2e1202bf01d2cb8/html5/thumbnails/44.jpg)
Matching via Function Preservation
where
Given enough pairs, we can recover C through a linear least squares system.
f =
!iaiφ
Mi, g =
!ibiφ
Ni.
{a,b}
Suppose we don’t know C. However, we expect a pair of functions and to correspond. Then, C must be s.t.
Ca ≈ b
f : M → R g : N → R
![Page 45: Some recent methods in non-rigid shape matching, with and … · Some recent methods in non-rigid shape matching, with and without learning GAMES 2019 webinar Maks Ovsjanikov Based](https://reader035.vdocuments.net/reader035/viewer/2022081617/60476439d2e1202bf01d2cb8/html5/thumbnails/45.jpg)
Function preservation constraint is general and includes:
• Attribute (e.g., color) preservation.
• Descriptor preservation (e.g. Gauss curvature).
• Landmark correspondences (e.g. distance to the point).
• Part correspondences (e.g. indicator function).
Map Constraints
Suppose we don’t know C. However, we expect a pair of functions and to correspond. Then, Cmust be s.t.
Ca ≈ b
f : M → R g : N → R
![Page 46: Some recent methods in non-rigid shape matching, with and … · Some recent methods in non-rigid shape matching, with and without learning GAMES 2019 webinar Maks Ovsjanikov Based](https://reader035.vdocuments.net/reader035/viewer/2022081617/60476439d2e1202bf01d2cb8/html5/thumbnails/46.jpg)
Commutativity Constraints
Regularizations:Commutativity with other operators:
C
Note that the energy: is quadratic in C.
SM SN
CSM = SNC
∥CSM − SNC∥2F
![Page 47: Some recent methods in non-rigid shape matching, with and … · Some recent methods in non-rigid shape matching, with and without learning GAMES 2019 webinar Maks Ovsjanikov Based](https://reader035.vdocuments.net/reader035/viewer/2022081617/60476439d2e1202bf01d2cb8/html5/thumbnails/47.jpg)
Regularization
Lemma 1:
The mapping is isometric, if and only if the functional map matrix commutes with the Laplacian:
Implies that exact isometries result in diagonal functional maps.
C∆M = ∆NC
Functional maps: a flexible representation of maps between shapes, O., Ben-Chen, Solomon, Butscher, Guibas, SIGGRAPH 2012
Linking functional and point-to-point maps
![Page 48: Some recent methods in non-rigid shape matching, with and … · Some recent methods in non-rigid shape matching, with and without learning GAMES 2019 webinar Maks Ovsjanikov Based](https://reader035.vdocuments.net/reader035/viewer/2022081617/60476439d2e1202bf01d2cb8/html5/thumbnails/48.jpg)
Basic Pipeline
Given a pair of shapes :
1. Compute the first k (~80-100) eigenfunctions of the Laplace-Beltrami operator. Store them in matrices:
2. Compute descriptor functions (e.g., Wave Kernel Signature) on . Express them in , as columns of :
3. Solve
4. Convert the functional mapto a point to point map T.
Copt =
diagonal matrices of eigenvalues of LB operator
M,N
ΦM,ΦN
ΦM,ΦN
∆M,∆N :
Copt = argminC
∥CA−B∥2 + ∥C∆M −∆NC∥2
A,B
M,N
![Page 49: Some recent methods in non-rigid shape matching, with and … · Some recent methods in non-rigid shape matching, with and without learning GAMES 2019 webinar Maks Ovsjanikov Based](https://reader035.vdocuments.net/reader035/viewer/2022081617/60476439d2e1202bf01d2cb8/html5/thumbnails/49.jpg)
Recent Implementation
Recent implementation incorporating efficient spatial and spectral constraints.
https://github.com/llorz/SGA18_orientation_BCICP_code
Continuous and Orientation-preserving Correspondences via Functional Maps Jing Ren, Adrien Poulenard, Peter Wonka, Maks Ovsjanikov, SIGGRAPH Asia 2018
![Page 50: Some recent methods in non-rigid shape matching, with and … · Some recent methods in non-rigid shape matching, with and without learning GAMES 2019 webinar Maks Ovsjanikov Based](https://reader035.vdocuments.net/reader035/viewer/2022081617/60476439d2e1202bf01d2cb8/html5/thumbnails/50.jpg)
Results
A very simple method that puts together many constraints and uses 100 basis functions gives reasonable results:
50Functional maps: a flexible representation of maps between shapes, O., Ben-Chen, Solomon, Butscher, Guibas, SIGGRAPH 2012
![Page 51: Some recent methods in non-rigid shape matching, with and … · Some recent methods in non-rigid shape matching, with and without learning GAMES 2019 webinar Maks Ovsjanikov Based](https://reader035.vdocuments.net/reader035/viewer/2022081617/60476439d2e1202bf01d2cb8/html5/thumbnails/51.jpg)
Results
radius 0.025
radius 0.05
Functional maps: a flexible representation of maps between shapes, O., Ben-Chen, Solomon, Butscher, Guibas, SIGGRAPH 2012
A very simple method that puts together many constraints and uses 100 basis functions gives reasonable results:
51
![Page 52: Some recent methods in non-rigid shape matching, with and … · Some recent methods in non-rigid shape matching, with and without learning GAMES 2019 webinar Maks Ovsjanikov Based](https://reader035.vdocuments.net/reader035/viewer/2022081617/60476439d2e1202bf01d2cb8/html5/thumbnails/52.jpg)
Segmentation Transfer without P2P
To transfer functions we do not need to convert functional to pointwise maps.
E.g. we can also transfer segmentations: for each segment, transfer its indicator function, and for each point pick the segment that gave the highest value.
![Page 53: Some recent methods in non-rigid shape matching, with and … · Some recent methods in non-rigid shape matching, with and without learning GAMES 2019 webinar Maks Ovsjanikov Based](https://reader035.vdocuments.net/reader035/viewer/2022081617/60476439d2e1202bf01d2cb8/html5/thumbnails/53.jpg)
Talk Overview
Motivation and Problem Taxonomy
Rigid Matching: ICP
Functional Map Representation, properties
Open Problems, Q&A
Basic pipeline for non-rigid matching
Extensions, Improvements
![Page 54: Some recent methods in non-rigid shape matching, with and … · Some recent methods in non-rigid shape matching, with and without learning GAMES 2019 webinar Maks Ovsjanikov Based](https://reader035.vdocuments.net/reader035/viewer/2022081617/60476439d2e1202bf01d2cb8/html5/thumbnails/54.jpg)
Some Recent Extensions
Efficient Refinement
Unsupervised Learning
ZoomOut: Spectral Upsampling for Efficient Shape CorrespondenceMelzi, Ren, Rodolà, Sharma, Wonka, Ovsjanikov, SIGGRAPH Asia 2019
Unsupervised Deep Learning for Structured Shape MatchingRoufosse, Sharma, Ovsjanikov, ICCV, 2019 (oral).
![Page 55: Some recent methods in non-rigid shape matching, with and … · Some recent methods in non-rigid shape matching, with and without learning GAMES 2019 webinar Maks Ovsjanikov Based](https://reader035.vdocuments.net/reader035/viewer/2022081617/60476439d2e1202bf01d2cb8/html5/thumbnails/55.jpg)
Main Question
55
What happens if the descriptors are bad?
![Page 56: Some recent methods in non-rigid shape matching, with and … · Some recent methods in non-rigid shape matching, with and without learning GAMES 2019 webinar Maks Ovsjanikov Based](https://reader035.vdocuments.net/reader035/viewer/2022081617/60476439d2e1202bf01d2cb8/html5/thumbnails/56.jpg)
56
Learning approach to computing descriptors.
O. Litany, T. Remez, E. Rodolà, A. Bronstein, M. Bronstein: Deep functional maps: Structured prediction for dense shape correspondence. In Proc. ICCV (2017).
D1<latexit sha1_base64="/NqoDpuSU5l6AFS0+IbhLzW/c/k=">AAACE3icbVDLSsNAFJ34rPFVdekmWAQRKYkIuizahRuxon1AE8pkepMOnUzCzEQooZ/gUv0Yd+LWD/Bb3Dhps9DWAwOHc+69c+/xE0alsu0vY2FxaXlltbRmrm9sbm2Xd3ZbMk4FgSaJWSw6PpbA KIemoopBJxGAI59B2x9e5X77EYSkMX9QowS8CIecBpRgpaX7es/plSt21Z7AmidOQSqoQKNX/nb7MUkj4IowLGXXsRPlZVgoShiMTTeVkGAyxCF0NeU4Aullk1XH1qFW+lYQC/24sibq744MR1KOIl9XRlgN5KyXi/953VQFF15GeZIq4GT6UZAyS8VWfrfVpwKIYiNNMBFU72qRARaYKJ2O6dZB3yLgRs+9TUBgFYvjzMUijCgf69tC9yRnpk7Lmc1mnrROq45dde7OKrXLIrcS2kcH6Ag56BzV0DVqoCYiKERP6AW9Gs/Gm/FufExLF4yiZw/9gfH5AyFcnkg=</latexit><latexit sha1_base64="/NqoDpuSU5l6AFS0+IbhLzW/c/k=">AAACE3icbVDLSsNAFJ34rPFVdekmWAQRKYkIuizahRuxon1AE8pkepMOnUzCzEQooZ/gUv0Yd+LWD/Bb3Dhps9DWAwOHc+69c+/xE0alsu0vY2FxaXlltbRmrm9sbm2Xd3ZbMk4FgSaJWSw6PpbA KIemoopBJxGAI59B2x9e5X77EYSkMX9QowS8CIecBpRgpaX7es/plSt21Z7AmidOQSqoQKNX/nb7MUkj4IowLGXXsRPlZVgoShiMTTeVkGAyxCF0NeU4Aullk1XH1qFW+lYQC/24sibq744MR1KOIl9XRlgN5KyXi/953VQFF15GeZIq4GT6UZAyS8VWfrfVpwKIYiNNMBFU72qRARaYKJ2O6dZB3yLgRs+9TUBgFYvjzMUijCgf69tC9yRnpk7Lmc1mnrROq45dde7OKrXLIrcS2kcH6Ag56BzV0DVqoCYiKERP6AW9Gs/Gm/FufExLF4yiZw/9gfH5AyFcnkg=</latexit><latexit sha1_base64="/NqoDpuSU5l6AFS0+IbhLzW/c/k=">AAACE3icbVDLSsNAFJ34rPFVdekmWAQRKYkIuizahRuxon1AE8pkepMOnUzCzEQooZ/gUv0Yd+LWD/Bb3Dhps9DWAwOHc+69c+/xE0alsu0vY2FxaXlltbRmrm9sbm2Xd3ZbMk4FgSaJWSw6PpbA KIemoopBJxGAI59B2x9e5X77EYSkMX9QowS8CIecBpRgpaX7es/plSt21Z7AmidOQSqoQKNX/nb7MUkj4IowLGXXsRPlZVgoShiMTTeVkGAyxCF0NeU4Aullk1XH1qFW+lYQC/24sibq744MR1KOIl9XRlgN5KyXi/953VQFF15GeZIq4GT6UZAyS8VWfrfVpwKIYiNNMBFU72qRARaYKJ2O6dZB3yLgRs+9TUBgFYvjzMUijCgf69tC9yRnpk7Lmc1mnrROq45dde7OKrXLIrcS2kcH6Ag56BzV0DVqoCYiKERP6AW9Gs/Gm/FufExLF4yiZw/9gfH5AyFcnkg=</latexit><latexit sha1_base64="/NqoDpuSU5l6AFS0+IbhLzW/c/k=">AAACE3icbVDLSsNAFJ34rPFVdekmWAQRKYkIuizahRuxon1AE8pkepMOnUzCzEQooZ/gUv0Yd+LWD/Bb3Dhps9DWAwOHc+69c+/xE0alsu0vY2FxaXlltbRmrm9sbm2Xd3ZbMk4FgSaJWSw6PpbA KIemoopBJxGAI59B2x9e5X77EYSkMX9QowS8CIecBpRgpaX7es/plSt21Z7AmidOQSqoQKNX/nb7MUkj4IowLGXXsRPlZVgoShiMTTeVkGAyxCF0NeU4Aullk1XH1qFW+lYQC/24sibq744MR1KOIl9XRlgN5KyXi/953VQFF15GeZIq4GT6UZAyS8VWfrfVpwKIYiNNMBFU72qRARaYKJ2O6dZB3yLgRs+9TUBgFYvjzMUijCgf69tC9yRnpk7Lmc1mnrROq45dde7OKrXLIrcS2kcH6Ag56BzV0DVqoCYiKERP6AW9Gs/Gm/FufExLF4yiZw/9gfH5AyFcnkg=</latexit>
D2<latexit sha1_base64="4fQo4Jpdl8nw3AmI++XyHse7Qqg=">AAACE3icbVDLSsNAFJ3UV42vqks3wSKISEmKoMuiXbgRK9oHNKFMprfp0MkkzEyEEvoJLtWPcSdu/QC/xY2TNgttPTBwOOfeO/ceP2ZUKtv+MgpLyyura8V1c2Nza3untLvXklEiCDRJxCLR8bEE Rjk0FVUMOrEAHPoM2v7oKvPbjyAkjfiDGsfghTjgdEAJVlq6r/eqvVLZrthTWIvEyUkZ5Wj0St9uPyJJCFwRhqXsOnasvBQLRQmDiekmEmJMRjiArqYchyC9dLrqxDrSSt8aREI/rqyp+rsjxaGU49DXlSFWQznvZeJ/XjdRgwsvpTxOFHAy+2iQMEtFVna31acCiGJjTTARVO9qkSEWmCidjunWQd8i4EbPvY1BYBWJk9TFIggpn+jbAvc0Y6ZOy5nPZpG0qhXHrjh3Z+XaZZ5bER2gQ3SMHHSOaugaNVATERSgJ/SCXo1n4814Nz5mpQUj79lHf2B8/gAjCZ5J</latexit><latexit sha1_base64="4fQo4Jpdl8nw3AmI++XyHse7Qqg=">AAACE3icbVDLSsNAFJ3UV42vqks3wSKISEmKoMuiXbgRK9oHNKFMprfp0MkkzEyEEvoJLtWPcSdu/QC/xY2TNgttPTBwOOfeO/ceP2ZUKtv+MgpLyyura8V1c2Nza3untLvXklEiCDRJxCLR8bEE Rjk0FVUMOrEAHPoM2v7oKvPbjyAkjfiDGsfghTjgdEAJVlq6r/eqvVLZrthTWIvEyUkZ5Wj0St9uPyJJCFwRhqXsOnasvBQLRQmDiekmEmJMRjiArqYchyC9dLrqxDrSSt8aREI/rqyp+rsjxaGU49DXlSFWQznvZeJ/XjdRgwsvpTxOFHAy+2iQMEtFVna31acCiGJjTTARVO9qkSEWmCidjunWQd8i4EbPvY1BYBWJk9TFIggpn+jbAvc0Y6ZOy5nPZpG0qhXHrjh3Z+XaZZ5bER2gQ3SMHHSOaugaNVATERSgJ/SCXo1n4814Nz5mpQUj79lHf2B8/gAjCZ5J</latexit><latexit sha1_base64="4fQo4Jpdl8nw3AmI++XyHse7Qqg=">AAACE3icbVDLSsNAFJ3UV42vqks3wSKISEmKoMuiXbgRK9oHNKFMprfp0MkkzEyEEvoJLtWPcSdu/QC/xY2TNgttPTBwOOfeO/ceP2ZUKtv+MgpLyyura8V1c2Nza3untLvXklEiCDRJxCLR8bEE Rjk0FVUMOrEAHPoM2v7oKvPbjyAkjfiDGsfghTjgdEAJVlq6r/eqvVLZrthTWIvEyUkZ5Wj0St9uPyJJCFwRhqXsOnasvBQLRQmDiekmEmJMRjiArqYchyC9dLrqxDrSSt8aREI/rqyp+rsjxaGU49DXlSFWQznvZeJ/XjdRgwsvpTxOFHAy+2iQMEtFVna31acCiGJjTTARVO9qkSEWmCidjunWQd8i4EbPvY1BYBWJk9TFIggpn+jbAvc0Y6ZOy5nPZpG0qhXHrjh3Z+XaZZ5bER2gQ3SMHHSOaugaNVATERSgJ/SCXo1n4814Nz5mpQUj79lHf2B8/gAjCZ5J</latexit><latexit sha1_base64="4fQo4Jpdl8nw3AmI++XyHse7Qqg=">AAACE3icbVDLSsNAFJ3UV42vqks3wSKISEmKoMuiXbgRK9oHNKFMprfp0MkkzEyEEvoJLtWPcSdu/QC/xY2TNgttPTBwOOfeO/ceP2ZUKtv+MgpLyyura8V1c2Nza3untLvXklEiCDRJxCLR8bEE Rjk0FVUMOrEAHPoM2v7oKvPbjyAkjfiDGsfghTjgdEAJVlq6r/eqvVLZrthTWIvEyUkZ5Wj0St9uPyJJCFwRhqXsOnasvBQLRQmDiekmEmJMRjiArqYchyC9dLrqxDrSSt8aREI/rqyp+rsjxaGU49DXlSFWQznvZeJ/XjdRgwsvpTxOFHAy+2iQMEtFVna31acCiGJjTTARVO9qkSEWmCidjunWQd8i4EbPvY1BYBWJk9TFIggpn+jbAvc0Y6ZOy5nPZpG0qhXHrjh3Z+XaZZ5bER2gQ3SMHHSOaugaNVATERSgJ/SCXo1n4814Nz5mpQUj79lHf2B8/gAjCZ5J</latexit> Neural net
Func. Map
T : S1 → S2<latexit sha1_base64="jsg/YwaKstOMgPlQ2pkgRWaCgy8=">AAACKHicbVBNTxsxEPVSPtMCoUhcerGIKlUIRbsICcQJAYdeKlJBACkbRbPOZGPhtVf2LCha8md6LPyY3qpc+RtccEIOLfAkS8/vzdgzL8mVdBSGo2Dmw+zc/MLiUuXjp+WV1era5wtnCiuwKYwy 9ioBh0pqbJIkhVe5RcgShZfJ9fHYv7xB66TR5zTIsZ1BqmVPCiAvdaob5wf8rBPx2Mq0T2CtufX3nU61FtbDCfhbEk1JjU3R6FSf4q4RRYaahALnWlGYU7sES1IoHFbiwmEO4hpSbHmqIUPXLifzD/lXr3R5z1h/NPGJ+m9HCZlzgyzxlRlQ3732xuJ7Xqug3n67lDov CLV4+ahXKE6Gj8PgXWlRkBp4AsJKPysXfbAgyEdWiU/Q72Lxh3/3NEcLZOxWGYNNM6mHfrc03h6zik8rep3NW3KxU4/CevRzt3Z4NM1tkX1hm+wbi9geO2TfWYM1mWB37Be7Zw/B7+BP8DcYvZTOBNOedfYfgsdnmGGlvQ==</latexit><latexit sha1_base64="jsg/YwaKstOMgPlQ2pkgRWaCgy8=">AAACKHicbVBNTxsxEPVSPtMCoUhcerGIKlUIRbsICcQJAYdeKlJBACkbRbPOZGPhtVf2LCha8md6LPyY3qpc+RtccEIOLfAkS8/vzdgzL8mVdBSGo2Dmw+zc/MLiUuXjp+WV1era5wtnCiuwKYwy 9ioBh0pqbJIkhVe5RcgShZfJ9fHYv7xB66TR5zTIsZ1BqmVPCiAvdaob5wf8rBPx2Mq0T2CtufX3nU61FtbDCfhbEk1JjU3R6FSf4q4RRYaahALnWlGYU7sES1IoHFbiwmEO4hpSbHmqIUPXLifzD/lXr3R5z1h/NPGJ+m9HCZlzgyzxlRlQ3732xuJ7Xqug3n67lDov CLV4+ahXKE6Gj8PgXWlRkBp4AsJKPysXfbAgyEdWiU/Q72Lxh3/3NEcLZOxWGYNNM6mHfrc03h6zik8rep3NW3KxU4/CevRzt3Z4NM1tkX1hm+wbi9geO2TfWYM1mWB37Be7Zw/B7+BP8DcYvZTOBNOedfYfgsdnmGGlvQ==</latexit><latexit sha1_base64="jsg/YwaKstOMgPlQ2pkgRWaCgy8=">AAACKHicbVBNTxsxEPVSPtMCoUhcerGIKlUIRbsICcQJAYdeKlJBACkbRbPOZGPhtVf2LCha8md6LPyY3qpc+RtccEIOLfAkS8/vzdgzL8mVdBSGo2Dmw+zc/MLiUuXjp+WV1era5wtnCiuwKYwy 9ioBh0pqbJIkhVe5RcgShZfJ9fHYv7xB66TR5zTIsZ1BqmVPCiAvdaob5wf8rBPx2Mq0T2CtufX3nU61FtbDCfhbEk1JjU3R6FSf4q4RRYaahALnWlGYU7sES1IoHFbiwmEO4hpSbHmqIUPXLifzD/lXr3R5z1h/NPGJ+m9HCZlzgyzxlRlQ3732xuJ7Xqug3n67lDov CLV4+ahXKE6Gj8PgXWlRkBp4AsJKPysXfbAgyEdWiU/Q72Lxh3/3NEcLZOxWGYNNM6mHfrc03h6zik8rep3NW3KxU4/CevRzt3Z4NM1tkX1hm+wbi9geO2TfWYM1mWB37Be7Zw/B7+BP8DcYvZTOBNOedfYfgsdnmGGlvQ==</latexit><latexit sha1_base64="jsg/YwaKstOMgPlQ2pkgRWaCgy8=">AAACKHicbVBNTxsxEPVSPtMCoUhcerGIKlUIRbsICcQJAYdeKlJBACkbRbPOZGPhtVf2LCha8md6LPyY3qpc+RtccEIOLfAkS8/vzdgzL8mVdBSGo2Dmw+zc/MLiUuXjp+WV1era5wtnCiuwKYwy 9ioBh0pqbJIkhVe5RcgShZfJ9fHYv7xB66TR5zTIsZ1BqmVPCiAvdaob5wf8rBPx2Mq0T2CtufX3nU61FtbDCfhbEk1JjU3R6FSf4q4RRYaahALnWlGYU7sES1IoHFbiwmEO4hpSbHmqIUPXLifzD/lXr3R5z1h/NPGJ+m9HCZlzgyzxlRlQ3732xuJ7Xqug3n67lDov CLV4+ahXKE6Gj8PgXWlRkBp4AsJKPysXfbAgyEdWiU/Q72Lxh3/3NEcLZOxWGYNNM6mHfrc03h6zik8rep3NW3KxU4/CevRzt3Z4NM1tkX1hm+wbi9geO2TfWYM1mWB37Be7Zw/B7+BP8DcYvZTOBNOedfYfgsdnmGGlvQ==</latexit>
FMNet
Neural net
![Page 57: Some recent methods in non-rigid shape matching, with and … · Some recent methods in non-rigid shape matching, with and without learning GAMES 2019 webinar Maks Ovsjanikov Based](https://reader035.vdocuments.net/reader035/viewer/2022081617/60476439d2e1202bf01d2cb8/html5/thumbnails/57.jpg)
FMNet
57
Training loss:
Learning approach to computing descriptors.
O. Litany, T. Remez, E. Rodolà, A. Bronstein, M. Bronstein: Deep functional maps: Structured prediction for dense shape correspondence. In Proc. ICCV (2017).
![Page 58: Some recent methods in non-rigid shape matching, with and … · Some recent methods in non-rigid shape matching, with and without learning GAMES 2019 webinar Maks Ovsjanikov Based](https://reader035.vdocuments.net/reader035/viewer/2022081617/60476439d2e1202bf01d2cb8/html5/thumbnails/58.jpg)
Our Goals
58
1. Avoid using ground truth correspondences• Replace supervised loss with unsupervised one
2. Avoid using geodesic distances• Perform all computations in the spectral domain
Main question: how to measure the quality of a map?
Note: related concurrent paper by Halimi et al. Unsupervised learning of dense shape correspondence. In CVPR, 2019
![Page 59: Some recent methods in non-rigid shape matching, with and … · Some recent methods in non-rigid shape matching, with and without learning GAMES 2019 webinar Maks Ovsjanikov Based](https://reader035.vdocuments.net/reader035/viewer/2022081617/60476439d2e1202bf01d2cb8/html5/thumbnails/59.jpg)
59
Replace supervised loss with unsupervised one
Unsupervised Deep Learning for Structured Shape Matching, J.-M. Rouffosse, A. Sharma, M. O., ICCV 2019
Our approach
FMNet
FMNet
Ereg(C12,C21)
D1
D2
C12 =
C21 =
argminC
∥CAT (D1) −AT (D2)∥2,
argminC
∥CAT (D2) −AT (D1)∥2
T (D1)
T (D2)
![Page 60: Some recent methods in non-rigid shape matching, with and … · Some recent methods in non-rigid shape matching, with and without learning GAMES 2019 webinar Maks Ovsjanikov Based](https://reader035.vdocuments.net/reader035/viewer/2022081617/60476439d2e1202bf01d2cb8/html5/thumbnails/60.jpg)
Our approach
Bijectivity
Area-preservation
Functional map close to pointwise one.
All penalties are in the reduced basis. 50x faster than FMNet
Near-isometry
!
E1(C12, C21) = ∥C12C21 − Id∥2
E1(C12, C21) = ∥C21C12 − Id∥2<latexit sha1_base64="WgkVSqF5wLBrfFa0rp5a+7DQM5k=">AAACpHicfVFta9RAEN6kvtT4dtaPggwe6in1SI5C/SIUW0FBsaLXFm6vYbOZ5JZuNmF3Ixwxf8J/5z8p+MXNXUBtxYFlH56ZZ2bn2aSSwtgw/OH5G1euXru+eSO4eev2nbuDe1tHpqw1xykvZalP EmZQCoVTK6zEk0ojKxKJx8nZfpc//oraiFJ9scsK5wXLlcgEZ9ZR8eA7lZhZ2gBNMBeqYVqzZdvI9k0cjfbjJpq02+DuSdQ+g1dAv8Ga7Dl4Ae9Sx55OACiF4H+qrrwX/1ZRVGk/FKgW+cKO48EwHIergMsg6sGQ9HEYD37StOR1gcpyyYyZRWFl566rFVxiG9DaYMX4 Gctx5qBiBZp5s/KuhceOSSErtTvKwor9U9GwwphlkbjKgtmFuZjryH/lZrXNXs4boaraouLrQVktwZbQfQSkQiO3cukA41q4twJfMM24dd8V0AN0u2j84Pp+rFAzW+rnDWU6L4Rq3W453e5Q4NyKLnpzGRxNxlE4jj7tDPde975tkgfkERmRiOySPfKWHJIp4eTce+g9 9Ub+E/+9/9mfrkt9r9fcJ3+Ff/oLK7PJCw==</latexit><latexit sha1_base64="WgkVSqF5wLBrfFa0rp5a+7DQM5k=">AAACpHicfVFta9RAEN6kvtT4dtaPggwe6in1SI5C/SIUW0FBsaLXFm6vYbOZ5JZuNmF3Ixwxf8J/5z8p+MXNXUBtxYFlH56ZZ2bn2aSSwtgw/OH5G1euXru+eSO4eev2nbuDe1tHpqw1xykvZalP EmZQCoVTK6zEk0ojKxKJx8nZfpc//oraiFJ9scsK5wXLlcgEZ9ZR8eA7lZhZ2gBNMBeqYVqzZdvI9k0cjfbjJpq02+DuSdQ+g1dAv8Ga7Dl4Ae9Sx55OACiF4H+qrrwX/1ZRVGk/FKgW+cKO48EwHIergMsg6sGQ9HEYD37StOR1gcpyyYyZRWFl566rFVxiG9DaYMX4 Gctx5qBiBZp5s/KuhceOSSErtTvKwor9U9GwwphlkbjKgtmFuZjryH/lZrXNXs4boaraouLrQVktwZbQfQSkQiO3cukA41q4twJfMM24dd8V0AN0u2j84Pp+rFAzW+rnDWU6L4Rq3W453e5Q4NyKLnpzGRxNxlE4jj7tDPde975tkgfkERmRiOySPfKWHJIp4eTce+g9 9Ub+E/+9/9mfrkt9r9fcJ3+Ff/oLK7PJCw==</latexit><latexit sha1_base64="WgkVSqF5wLBrfFa0rp5a+7DQM5k=">AAACpHicfVFta9RAEN6kvtT4dtaPggwe6in1SI5C/SIUW0FBsaLXFm6vYbOZ5JZuNmF3Ixwxf8J/5z8p+MXNXUBtxYFlH56ZZ2bn2aSSwtgw/OH5G1euXru+eSO4eev2nbuDe1tHpqw1xykvZalP EmZQCoVTK6zEk0ojKxKJx8nZfpc//oraiFJ9scsK5wXLlcgEZ9ZR8eA7lZhZ2gBNMBeqYVqzZdvI9k0cjfbjJpq02+DuSdQ+g1dAv8Ga7Dl4Ae9Sx55OACiF4H+qrrwX/1ZRVGk/FKgW+cKO48EwHIergMsg6sGQ9HEYD37StOR1gcpyyYyZRWFl566rFVxiG9DaYMX4 Gctx5qBiBZp5s/KuhceOSSErtTvKwor9U9GwwphlkbjKgtmFuZjryH/lZrXNXs4boaraouLrQVktwZbQfQSkQiO3cukA41q4twJfMM24dd8V0AN0u2j84Pp+rFAzW+rnDWU6L4Rq3W453e5Q4NyKLnpzGRxNxlE4jj7tDPde975tkgfkERmRiOySPfKWHJIp4eTce+g9 9Ub+E/+9/9mfrkt9r9fcJ3+Ff/oLK7PJCw==</latexit><latexit sha1_base64="WgkVSqF5wLBrfFa0rp5a+7DQM5k=">AAACpHicfVFta9RAEN6kvtT4dtaPggwe6in1SI5C/SIUW0FBsaLXFm6vYbOZ5JZuNmF3Ixwxf8J/5z8p+MXNXUBtxYFlH56ZZ2bn2aSSwtgw/OH5G1euXru+eSO4eev2nbuDe1tHpqw1xykvZalP EmZQCoVTK6zEk0ojKxKJx8nZfpc//oraiFJ9scsK5wXLlcgEZ9ZR8eA7lZhZ2gBNMBeqYVqzZdvI9k0cjfbjJpq02+DuSdQ+g1dAv8Ga7Dl4Ae9Sx55OACiF4H+qrrwX/1ZRVGk/FKgW+cKO48EwHIergMsg6sGQ9HEYD37StOR1gcpyyYyZRWFl566rFVxiG9DaYMX4 Gctx5qBiBZp5s/KuhceOSSErtTvKwor9U9GwwphlkbjKgtmFuZjryH/lZrXNXs4boaraouLrQVktwZbQfQSkQiO3cukA41q4twJfMM24dd8V0AN0u2j84Pp+rFAzW+rnDWU6L4Rq3W453e5Q4NyKLnpzGRxNxlE4jj7tDPde975tkgfkERmRiOySPfKWHJIp4eTce+g9 9Ub+E/+9/9mfrkt9r9fcJ3+Ff/oLK7PJCw==</latexit>
E4(C) =!
i
∥CXfi − YgiC∥2
<latexit sha1_base64="hKJXk5HeOm7C6uQBrmfQ19aPFMk=">AAACQXicbZBBT9swGIadskHXDejYcRdrFVI3QUmqSuyCVNFN4jIBEqWdmhI57pfUqu1EtoNUhf6M/RqObD9iP2E3tGsvOKWHUfZJlh6/rz/b3xumnGnjur+d0tqLl+sb5VeV1282t7arb3cudZIp Cl2a8ET1Q6KBMwldwwyHfqqAiJBDL5x0Cr93DUqzRF6YaQpDQWLJIkaJsVJQPfgatOqdj/gI+zoTQc5m2L/BHdwP8iiwm338PcjjgjrWuGoG1ZrbcBeFn4O3hBpa1llQnfujhGYCpKGcaD3w3NQMc6IMoxxmFT/TkBI6ITEMLEoiQA/zxWAzvGuVEY4SZZc0eKH+25ETofVUhPakIGasV71C/J83yEz0eZgzmWYGJH18KMo4NgkuUsIjpoAaPrVAqGL2r5iOiSLU2Cwr/hewsyj4Zu89TUERk6hPuU9ULJic2dlif6+gik3LW83mOVw2G57b8M5btfbxMrcyeo8+oDry0CFqoxN0hrqIoh/oFv1Ev5w7549z7/x9PFpylj3v0JNy5g/uta5G</la texit><latexit sha1_base64="hKJXk5HeOm7C6uQBrmfQ19aPFMk=">AAACQXicbZBBT9swGIadskHXDejYcRdrFVI3QUmqSuyCVNFN4jIBEqWdmhI57pfUqu1EtoNUhf6M/RqObD9iP2E3tGsvOKWHUfZJlh6/rz/b3xumnGnjur+d0tqLl+sb5VeV1282t7arb3cudZIp Cl2a8ET1Q6KBMwldwwyHfqqAiJBDL5x0Cr93DUqzRF6YaQpDQWLJIkaJsVJQPfgatOqdj/gI+zoTQc5m2L/BHdwP8iiwm338PcjjgjrWuGoG1ZrbcBeFn4O3hBpa1llQnfujhGYCpKGcaD3w3NQMc6IMoxxmFT/TkBI6ITEMLEoiQA/zxWAzvGuVEY4SZZc0eKH+25ETofVUhPakIGasV71C/J83yEz0eZgzmWYGJH18KMo4NgkuUsIjpoAaPrVAqGL2r5iOiSLU2Cwr/hewsyj4Zu89TUERk6hPuU9ULJic2dlif6+gik3LW83mOVw2G57b8M5btfbxMrcyeo8+oDry0CFqoxN0hrqIoh/oFv1Ev5w7549z7/x9PFpylj3v0JNy5g/uta5G</la texit><latexit sha1_base64="hKJXk5HeOm7C6uQBrmfQ19aPFMk=">AAACQXicbZBBT9swGIadskHXDejYcRdrFVI3QUmqSuyCVNFN4jIBEqWdmhI57pfUqu1EtoNUhf6M/RqObD9iP2E3tGsvOKWHUfZJlh6/rz/b3xumnGnjur+d0tqLl+sb5VeV1282t7arb3cudZIp Cl2a8ET1Q6KBMwldwwyHfqqAiJBDL5x0Cr93DUqzRF6YaQpDQWLJIkaJsVJQPfgatOqdj/gI+zoTQc5m2L/BHdwP8iiwm338PcjjgjrWuGoG1ZrbcBeFn4O3hBpa1llQnfujhGYCpKGcaD3w3NQMc6IMoxxmFT/TkBI6ITEMLEoiQA/zxWAzvGuVEY4SZZc0eKH+25ETofVUhPakIGasV71C/J83yEz0eZgzmWYGJH18KMo4NgkuUsIjpoAaPrVAqGL2r5iOiSLU2Cwr/hewsyj4Zu89TUERk6hPuU9ULJic2dlif6+gik3LW83mOVw2G57b8M5btfbxMrcyeo8+oDry0CFqoxN0hrqIoh/oFv1Ev5w7549z7/x9PFpylj3v0JNy5g/uta5G</la texit><latexit sha1_base64="hKJXk5HeOm7C6uQBrmfQ19aPFMk=">AAACQXicbZBBT9swGIadskHXDejYcRdrFVI3QUmqSuyCVNFN4jIBEqWdmhI57pfUqu1EtoNUhf6M/RqObD9iP2E3tGsvOKWHUfZJlh6/rz/b3xumnGnjur+d0tqLl+sb5VeV1282t7arb3cudZIp Cl2a8ET1Q6KBMwldwwyHfqqAiJBDL5x0Cr93DUqzRF6YaQpDQWLJIkaJsVJQPfgatOqdj/gI+zoTQc5m2L/BHdwP8iiwm338PcjjgjrWuGoG1ZrbcBeFn4O3hBpa1llQnfujhGYCpKGcaD3w3NQMc6IMoxxmFT/TkBI6ITEMLEoiQA/zxWAzvGuVEY4SZZc0eKH+25ETofVUhPakIGasV71C/J83yEz0eZgzmWYGJH18KMo4NgkuUsIjpoAaPrVAqGL2r5iOiSLU2Cwr/hewsyj4Zu89TUERk6hPuU9ULJic2dlif6+gik3LW83mOVw2G57b8M5btfbxMrcyeo8+oDry0CFqoxN0hrqIoh/oFv1Ev5w7549z7/x9PFpylj3v0JNy5g/uta5G</la texit>
E3(C) = ∥Λ2C − CΛ1∥2
<latexit sha1_base64="CzTN9/B8DDCe+3FqU2comsGDPO0=">AAACQnicbVBdaxNBFJ2NtcZU22gffRkaCqlo2E0FfRFC04IPlkYwH5CNy93JTTpkdnaZmRXCun/DX+Nj9T/0L/hW+mgfOvmg2KQXLhzOPffOnBMmgmvjupdO4dHG480nxaelrWfPt3fKL152dJwq hm0Wi1j1QtAouMS24UZgL1EIUSiwG06as3n3OyrNY/nVTBMcRDCWfMQZGEsFZfckOKw2D+hH6v+g/me7OIQgq+e0Sd/avmO83Aq+1SkNyhW35s6LrgNvCSpkWa2g/M8fxiyNUBomQOu+5yZmkIEynAnMS36qMQE2gTH2LZQQoR5kc2c53bfMkI5iZVsaOmf/38gg0noahVYZgTnXq7MZ+dCsn5rRh0HGZZIalGzx0CgV1MR0FhMdcoXMiKkFwBS3f6XsHBQwY8Ms+cdovSg8tXfPElRgYvU680GNIy5z623sv5mhkk3LW81mHXTqNc+teV/eVRpHy9yK5BXZI1XikfekQT6RFmkTRn6SX+Q3+eNcOH+dK+d6IS04y51dcq+cm1vE0a2N</latexi t><latexit sha1_base64="CzTN9/B8DDCe+3FqU2comsGDPO0=">AAACQnicbVBdaxNBFJ2NtcZU22gffRkaCqlo2E0FfRFC04IPlkYwH5CNy93JTTpkdnaZmRXCun/DX+Nj9T/0L/hW+mgfOvmg2KQXLhzOPffOnBMmgmvjupdO4dHG480nxaelrWfPt3fKL152dJwq hm0Wi1j1QtAouMS24UZgL1EIUSiwG06as3n3OyrNY/nVTBMcRDCWfMQZGEsFZfckOKw2D+hH6v+g/me7OIQgq+e0Sd/avmO83Aq+1SkNyhW35s6LrgNvCSpkWa2g/M8fxiyNUBomQOu+5yZmkIEynAnMS36qMQE2gTH2LZQQoR5kc2c53bfMkI5iZVsaOmf/38gg0noahVYZgTnXq7MZ+dCsn5rRh0HGZZIalGzx0CgV1MR0FhMdcoXMiKkFwBS3f6XsHBQwY8Ms+cdovSg8tXfPElRgYvU680GNIy5z623sv5mhkk3LW81mHXTqNc+teV/eVRpHy9yK5BXZI1XikfekQT6RFmkTRn6SX+Q3+eNcOH+dK+d6IS04y51dcq+cm1vE0a2N</latexi t><latexit sha1_base64="CzTN9/B8DDCe+3FqU2comsGDPO0=">AAACQnicbVBdaxNBFJ2NtcZU22gffRkaCqlo2E0FfRFC04IPlkYwH5CNy93JTTpkdnaZmRXCun/DX+Nj9T/0L/hW+mgfOvmg2KQXLhzOPffOnBMmgmvjupdO4dHG480nxaelrWfPt3fKL152dJwq hm0Wi1j1QtAouMS24UZgL1EIUSiwG06as3n3OyrNY/nVTBMcRDCWfMQZGEsFZfckOKw2D+hH6v+g/me7OIQgq+e0Sd/avmO83Aq+1SkNyhW35s6LrgNvCSpkWa2g/M8fxiyNUBomQOu+5yZmkIEynAnMS36qMQE2gTH2LZQQoR5kc2c53bfMkI5iZVsaOmf/38gg0noahVYZgTnXq7MZ+dCsn5rRh0HGZZIalGzx0CgV1MR0FhMdcoXMiKkFwBS3f6XsHBQwY8Ms+cdovSg8tXfPElRgYvU680GNIy5z623sv5mhkk3LW81mHXTqNc+teV/eVRpHy9yK5BXZI1XikfekQT6RFmkTRn6SX+Q3+eNcOH+dK+d6IS04y51dcq+cm1vE0a2N</latexi t><latexit sha1_base64="CzTN9/B8DDCe+3FqU2comsGDPO0=">AAACQnicbVBdaxNBFJ2NtcZU22gffRkaCqlo2E0FfRFC04IPlkYwH5CNy93JTTpkdnaZmRXCun/DX+Nj9T/0L/hW+mgfOvmg2KQXLhzOPffOnBMmgmvjupdO4dHG480nxaelrWfPt3fKL152dJwq hm0Wi1j1QtAouMS24UZgL1EIUSiwG06as3n3OyrNY/nVTBMcRDCWfMQZGEsFZfckOKw2D+hH6v+g/me7OIQgq+e0Sd/avmO83Aq+1SkNyhW35s6LrgNvCSpkWa2g/M8fxiyNUBomQOu+5yZmkIEynAnMS36qMQE2gTH2LZQQoR5kc2c53bfMkI5iZVsaOmf/38gg0noahVYZgTnXq7MZ+dCsn5rRh0HGZZIalGzx0CgV1MR0FhMdcoXMiKkFwBS3f6XsHBQwY8Ms+cdovSg8tXfPElRgYvU680GNIy5z623sv5mhkk3LW81mHXTqNc+teV/eVRpHy9yK5BXZI1XikfekQT6RFmkTRn6SX+Q3+eNcOH+dK+d6IS04y51dcq+cm1vE0a2N</latexi t>
E2(C) = ∥CTC − Id∥2
<latexit sha1_base64="SY1mjogmSReAQo1WPj+rMid7iWw=">AAACL3icbVBJSwMxGM3Urdat6lGEYBGqaJkpgl6EYhX0IFawC3Takkm/1mAmMyQZoYw9+Ws8qj9GvIhXf4MX0+Xg9iDw8r4lec8LOVPatl+txMTk1PRMcjY1N7+wuJReXqmoIJIUyjTggax5RAFn AsqaaQ61UALxPQ5V76Y4qFdvQSoWiCvdC6Hhk65gHUaJNlIrvX7SymeLW/gQu3e42LzCRbyLz9rm1sxj3Epn7Jw9BP5LnDHJoDFKrfSn2w5o5IPQlBOl6o4d6kZMpGaUQz/lRgpCQm9IF+qGCuKDasRDG328aZQ27gTSHKHxUP0+ERNfqZ7vmU6f6Gv1uzYQ/6vVI905aMRMhJEGQUcPdSKOdYAHmeA2k0A17xlCqGTmr5heE0moNsml3GMwXiScm70XIUiiA7kdu0R2fSb6xlvX3RmwlEnL+Z3NX1LJ5xw751zuZQpH49ySaA1toCxy0D4qoFNUQmVE0T16QE/o2Xq0Xqw3633UmrDGM6voB6yPL0p4pcI=</latexit><latexit sha1_base64="SY1mjogmSReAQo1WPj+rMid7iWw=">AAACL3icbVBJSwMxGM3Urdat6lGEYBGqaJkpgl6EYhX0IFawC3Takkm/1mAmMyQZoYw9+Ws8qj9GvIhXf4MX0+Xg9iDw8r4lec8LOVPatl+txMTk1PRMcjY1N7+wuJReXqmoIJIUyjTggax5RAFn AsqaaQ61UALxPQ5V76Y4qFdvQSoWiCvdC6Hhk65gHUaJNlIrvX7SymeLW/gQu3e42LzCRbyLz9rm1sxj3Epn7Jw9BP5LnDHJoDFKrfSn2w5o5IPQlBOl6o4d6kZMpGaUQz/lRgpCQm9IF+qGCuKDasRDG328aZQ27gTSHKHxUP0+ERNfqZ7vmU6f6Gv1uzYQ/6vVI905aMRMhJEGQUcPdSKOdYAHmeA2k0A17xlCqGTmr5heE0moNsml3GMwXiScm70XIUiiA7kdu0R2fSb6xlvX3RmwlEnL+Z3NX1LJ5xw751zuZQpH49ySaA1toCxy0D4qoFNUQmVE0T16QE/o2Xq0Xqw3633UmrDGM6voB6yPL0p4pcI=</latexit><latexit sha1_base64="SY1mjogmSReAQo1WPj+rMid7iWw=">AAACL3icbVBJSwMxGM3Urdat6lGEYBGqaJkpgl6EYhX0IFawC3Takkm/1mAmMyQZoYw9+Ws8qj9GvIhXf4MX0+Xg9iDw8r4lec8LOVPatl+txMTk1PRMcjY1N7+wuJReXqmoIJIUyjTggax5RAFn AsqaaQ61UALxPQ5V76Y4qFdvQSoWiCvdC6Hhk65gHUaJNlIrvX7SymeLW/gQu3e42LzCRbyLz9rm1sxj3Epn7Jw9BP5LnDHJoDFKrfSn2w5o5IPQlBOl6o4d6kZMpGaUQz/lRgpCQm9IF+qGCuKDasRDG328aZQ27gTSHKHxUP0+ERNfqZ7vmU6f6Gv1uzYQ/6vVI905aMRMhJEGQUcPdSKOdYAHmeA2k0A17xlCqGTmr5heE0moNsml3GMwXiScm70XIUiiA7kdu0R2fSb6xlvX3RmwlEnL+Z3NX1LJ5xw751zuZQpH49ySaA1toCxy0D4qoFNUQmVE0T16QE/o2Xq0Xqw3633UmrDGM6voB6yPL0p4pcI=</latexit><latexit sha1_base64="SY1mjogmSReAQo1WPj+rMid7iWw=">AAACL3icbVBJSwMxGM3Urdat6lGEYBGqaJkpgl6EYhX0IFawC3Takkm/1mAmMyQZoYw9+Ws8qj9GvIhXf4MX0+Xg9iDw8r4lec8LOVPatl+txMTk1PRMcjY1N7+wuJReXqmoIJIUyjTggax5RAFn AsqaaQ61UALxPQ5V76Y4qFdvQSoWiCvdC6Hhk65gHUaJNlIrvX7SymeLW/gQu3e42LzCRbyLz9rm1sxj3Epn7Jw9BP5LnDHJoDFKrfSn2w5o5IPQlBOl6o4d6kZMpGaUQz/lRgpCQm9IF+qGCuKDasRDG328aZQ27gTSHKHxUP0+ERNfqZ7vmU6f6Gv1uzYQ/6vVI905aMRMhJEGQUcPdSKOdYAHmeA2k0A17xlCqGTmr5heE0moNsml3GMwXiScm70XIUiiA7kdu0R2fSb6xlvX3RmwlEnL+Z3NX1LJ5xw751zuZQpH49ySaA1toCxy0D4qoFNUQmVE0T16QE/o2Xq0Xqw3633UmrDGM6voB6yPL0p4pcI=</latexit>
Replace supervised loss with unsupervised one
![Page 61: Some recent methods in non-rigid shape matching, with and … · Some recent methods in non-rigid shape matching, with and without learning GAMES 2019 webinar Maks Ovsjanikov Based](https://reader035.vdocuments.net/reader035/viewer/2022081617/60476439d2e1202bf01d2cb8/html5/thumbnails/61.jpg)
Datasets
61
Datasets released as part of: Continuous and Orientation-preserving Correspondences via Functional Maps, J. Ren, A. Poulenard, P. Wonka, M. O, SIGGRAPH Asia 2018
![Page 62: Some recent methods in non-rigid shape matching, with and … · Some recent methods in non-rigid shape matching, with and without learning GAMES 2019 webinar Maks Ovsjanikov Based](https://reader035.vdocuments.net/reader035/viewer/2022081617/60476439d2e1202bf01d2cb8/html5/thumbnails/62.jpg)
62
Comparison to unsupervised methods
Results
Unsupervised Deep Learning for Structured Shape Matching, J.-M. Rouffosse, A. Sharma, M. O., ICCV 2019
State-of-the art among unsupervised methods.
![Page 63: Some recent methods in non-rigid shape matching, with and … · Some recent methods in non-rigid shape matching, with and without learning GAMES 2019 webinar Maks Ovsjanikov Based](https://reader035.vdocuments.net/reader035/viewer/2022081617/60476439d2e1202bf01d2cb8/html5/thumbnails/63.jpg)
63
State-of-the art among unsupervised methods.
Results
Unsupervised Deep Learning for Structured Shape Matching, J.-M. Rouffosse, A. Sharma, M. O., ICCV 2019
![Page 64: Some recent methods in non-rigid shape matching, with and … · Some recent methods in non-rigid shape matching, with and without learning GAMES 2019 webinar Maks Ovsjanikov Based](https://reader035.vdocuments.net/reader035/viewer/2022081617/60476439d2e1202bf01d2cb8/html5/thumbnails/64.jpg)
64
Comparison to supervised methods
Comparable results even to supervised methods
Results
Unsupervised Deep Learning for Structured Shape Matching, J.-M. Rouffosse, A. Sharma, M. O., ICCV 2019
![Page 65: Some recent methods in non-rigid shape matching, with and … · Some recent methods in non-rigid shape matching, with and without learning GAMES 2019 webinar Maks Ovsjanikov Based](https://reader035.vdocuments.net/reader035/viewer/2022081617/60476439d2e1202bf01d2cb8/html5/thumbnails/65.jpg)
65
Comparable results even to supervised methods
Results
Unsupervised Deep Learning for Structured Shape Matching, J.-M. Rouffosse, A. Sharma, M. O., ICCV 2019
![Page 66: Some recent methods in non-rigid shape matching, with and … · Some recent methods in non-rigid shape matching, with and without learning GAMES 2019 webinar Maks Ovsjanikov Based](https://reader035.vdocuments.net/reader035/viewer/2022081617/60476439d2e1202bf01d2cb8/html5/thumbnails/66.jpg)
66
Results
Original vs. learned descriptors.
Unsupervised Deep Learning for Structured Shape Matching, J.-M. Rouffosse, A. Sharma, M. O., ICCV 2019
![Page 67: Some recent methods in non-rigid shape matching, with and … · Some recent methods in non-rigid shape matching, with and without learning GAMES 2019 webinar Maks Ovsjanikov Based](https://reader035.vdocuments.net/reader035/viewer/2022081617/60476439d2e1202bf01d2cb8/html5/thumbnails/67.jpg)
67
1. How can we build up a functional map progressively?
2. Given a small functional map, can we use it to transfer high frequency functions?
3. Simplify and speed-up functional map refinement?
Several related questions
![Page 68: Some recent methods in non-rigid shape matching, with and … · Some recent methods in non-rigid shape matching, with and without learning GAMES 2019 webinar Maks Ovsjanikov Based](https://reader035.vdocuments.net/reader035/viewer/2022081617/60476439d2e1202bf01d2cb8/html5/thumbnails/68.jpg)
ZoomOut
68
A two-lines-of-code algorithm:1) Given a functional map C1 of size k x k convert it to a p2p map T.2) Convert T to C2 of size (k+1) x (k+1)
Repeat for progressively larger k
ZoomOut: Spectral Upsampling for Efficient Shape Correspondence, S. Melzi, J. Ren, A. Sharma, E. Rodolà, P. Wonka, M. O., SIGGRAPH Asia 2019
![Page 69: Some recent methods in non-rigid shape matching, with and … · Some recent methods in non-rigid shape matching, with and without learning GAMES 2019 webinar Maks Ovsjanikov Based](https://reader035.vdocuments.net/reader035/viewer/2022081617/60476439d2e1202bf01d2cb8/html5/thumbnails/69.jpg)
69
Upsampling vs. computing directly:
ZoomOut: Spectral Upsampling for Efficient Shape Correspondence, S. Melzi, J. Ren, A. Sharma, E. Rodolà, P. Wonka, M. O., SIGGRAPH Asia 2019
ZoomOut
![Page 70: Some recent methods in non-rigid shape matching, with and … · Some recent methods in non-rigid shape matching, with and without learning GAMES 2019 webinar Maks Ovsjanikov Based](https://reader035.vdocuments.net/reader035/viewer/2022081617/60476439d2e1202bf01d2cb8/html5/thumbnails/70.jpg)
70
Extreme case, from 2x2 to 100x100
Dataset provided by the Natural History Museum in Paris.
ZoomOut – Results
![Page 71: Some recent methods in non-rigid shape matching, with and … · Some recent methods in non-rigid shape matching, with and without learning GAMES 2019 webinar Maks Ovsjanikov Based](https://reader035.vdocuments.net/reader035/viewer/2022081617/60476439d2e1202bf01d2cb8/html5/thumbnails/71.jpg)
ZoomOut – Results
71
From 5x5 to 50x50
ZoomOut: Spectral Upsampling for Efficient Shape Correspondence, S. Melzi, J. Ren, A. Sharma, E. Rodolà, P. Wonka, M. O., SIGGRAPH Asia 2019
![Page 72: Some recent methods in non-rigid shape matching, with and … · Some recent methods in non-rigid shape matching, with and without learning GAMES 2019 webinar Maks Ovsjanikov Based](https://reader035.vdocuments.net/reader035/viewer/2022081617/60476439d2e1202bf01d2cb8/html5/thumbnails/72.jpg)
72
From 20x20 to 120x120
ZoomOut – Results
![Page 73: Some recent methods in non-rigid shape matching, with and … · Some recent methods in non-rigid shape matching, with and without learning GAMES 2019 webinar Maks Ovsjanikov Based](https://reader035.vdocuments.net/reader035/viewer/2022081617/60476439d2e1202bf01d2cb8/html5/thumbnails/73.jpg)
73
Ours is 50-300x faster than state-of-the-art with higher accuracy
Evaluated on:• Intrinsic symmetry detection• Complete matching• Partial matching• Function transfer…
Compared against 14 baselines
ZoomOut – Results
![Page 74: Some recent methods in non-rigid shape matching, with and … · Some recent methods in non-rigid shape matching, with and without learning GAMES 2019 webinar Maks Ovsjanikov Based](https://reader035.vdocuments.net/reader035/viewer/2022081617/60476439d2e1202bf01d2cb8/html5/thumbnails/74.jpg)
74
Consider the optimization problem:
Theorem:
if and only if the point-to-point map is an isometry.
ZoomOut can be derived as a iterative method for solving this optimization problem.
: functional map arising from some pointwise map.: leading principal submatrix of .
E(C) = 0<latexit sha1_base64="K2LMUGTHjne5kmz0VeSMUsRlsWY=">AAACI3icbVDLSsNAFJ34tr5SXboZLIKKlEQE3QjiA9yICtYWmlIm05t26GQSZiZKCfkUl+rHuBM3LvwSN07aLLR6YOBw7mPOPX7MmdKO82FNTE5Nz8zOzZcWFpeWV+zy6p2KEkmhRiMeyYZPFHAm oKaZ5tCIJZDQ51D3+6d5vX4PUrFI3OpBDK2QdAULGCXaSG27fL7lhUT3/CA9zbbxEXbadsWpOkPgv8QtSAUVuG7bX14nokkIQlNOlGq6TqxbKZGaUQ5ZyUsUxIT2SReahgoSgmqlQ+sZ3jRKBweRNE9oPFR/TqQkVGoQ+qYzt6nGa7n4X62Z6OCwlTIRJxoEHX0UJBzrCOc54A6TQDUfGEKoZMYrpj0iCdUmrZJ3BuYWCZdm71UMkuhI7qQekd2Qiczc1vV2c1Yyabnj2fwld3tV16m6N/uV45Mitzm0jjbQFnLRATpGF+ga1RBFD+gRPaMX68l6td6s91HrhFXMrKFfsD6/AVEio30=</latexit><latexit sha1_base64="K2LMUGTHjne5kmz0VeSMUsRlsWY=">AAACI3icbVDLSsNAFJ34tr5SXboZLIKKlEQE3QjiA9yICtYWmlIm05t26GQSZiZKCfkUl+rHuBM3LvwSN07aLLR6YOBw7mPOPX7MmdKO82FNTE5Nz8zOzZcWFpeWV+zy6p2KEkmhRiMeyYZPFHAm oKaZ5tCIJZDQ51D3+6d5vX4PUrFI3OpBDK2QdAULGCXaSG27fL7lhUT3/CA9zbbxEXbadsWpOkPgv8QtSAUVuG7bX14nokkIQlNOlGq6TqxbKZGaUQ5ZyUsUxIT2SReahgoSgmqlQ+sZ3jRKBweRNE9oPFR/TqQkVGoQ+qYzt6nGa7n4X62Z6OCwlTIRJxoEHX0UJBzrCOc54A6TQDUfGEKoZMYrpj0iCdUmrZJ3BuYWCZdm71UMkuhI7qQekd2Qiczc1vV2c1Yyabnj2fwld3tV16m6N/uV45Mitzm0jjbQFnLRATpGF+ga1RBFD+gRPaMX68l6td6s91HrhFXMrKFfsD6/AVEio30=</latexit><latexit sha1_base64="K2LMUGTHjne5kmz0VeSMUsRlsWY=">AAACI3icbVDLSsNAFJ34tr5SXboZLIKKlEQE3QjiA9yICtYWmlIm05t26GQSZiZKCfkUl+rHuBM3LvwSN07aLLR6YOBw7mPOPX7MmdKO82FNTE5Nz8zOzZcWFpeWV+zy6p2KEkmhRiMeyYZPFHAm oKaZ5tCIJZDQ51D3+6d5vX4PUrFI3OpBDK2QdAULGCXaSG27fL7lhUT3/CA9zbbxEXbadsWpOkPgv8QtSAUVuG7bX14nokkIQlNOlGq6TqxbKZGaUQ5ZyUsUxIT2SReahgoSgmqlQ+sZ3jRKBweRNE9oPFR/TqQkVGoQ+qYzt6nGa7n4X62Z6OCwlTIRJxoEHX0UJBzrCOc54A6TQDUfGEKoZMYrpj0iCdUmrZJ3BuYWCZdm71UMkuhI7qQekd2Qiczc1vV2c1Yyabnj2fwld3tV16m6N/uV45Mitzm0jjbQFnLRATpGF+ga1RBFD+gRPaMX68l6td6s91HrhFXMrKFfsD6/AVEio30=</latexit><latexit sha1_base64="K2LMUGTHjne5kmz0VeSMUsRlsWY=">AAACI3icbVDLSsNAFJ34tr5SXboZLIKKlEQE3QjiA9yICtYWmlIm05t26GQSZiZKCfkUl+rHuBM3LvwSN07aLLR6YOBw7mPOPX7MmdKO82FNTE5Nz8zOzZcWFpeWV+zy6p2KEkmhRiMeyYZPFHAm oKaZ5tCIJZDQ51D3+6d5vX4PUrFI3OpBDK2QdAULGCXaSG27fL7lhUT3/CA9zbbxEXbadsWpOkPgv8QtSAUVuG7bX14nokkIQlNOlGq6TqxbKZGaUQ5ZyUsUxIT2SReahgoSgmqlQ+sZ3jRKBweRNE9oPFR/TqQkVGoQ+qYzt6nGa7n4X62Z6OCwlTIRJxoEHX0UJBzrCOc54A6TQDUfGEKoZMYrpj0iCdUmrZJ3BuYWCZdm71UMkuhI7qQekd2Qiczc1vV2c1Yyabnj2fwld3tV16m6N/uV45Mitzm0jjbQFnLRATpGF+ga1RBFD+gRPaMX68l6td6s91HrhFXMrKFfsD6/AVEio30=</latexit>
ZoomOut – Rationale
ZoomOut: Spectral Upsampling for Efficient Shape Correspondence, S. Melzi, J. Ren, A. Sharma, E. Rodolà, P. Wonka, M. O., SIGGRAPH Asia 2019
![Page 75: Some recent methods in non-rigid shape matching, with and … · Some recent methods in non-rigid shape matching, with and without learning GAMES 2019 webinar Maks Ovsjanikov Based](https://reader035.vdocuments.net/reader035/viewer/2022081617/60476439d2e1202bf01d2cb8/html5/thumbnails/75.jpg)
75
In some cases also works for non-isometric shapes
ZoomOut – Non-isometric
![Page 76: Some recent methods in non-rigid shape matching, with and … · Some recent methods in non-rigid shape matching, with and without learning GAMES 2019 webinar Maks Ovsjanikov Based](https://reader035.vdocuments.net/reader035/viewer/2022081617/60476439d2e1202bf01d2cb8/html5/thumbnails/76.jpg)
Other Extensions
Maps in Collections
Promoting Pointwise MapsInformative Descriptor Preservation via Commutativity for Shape Matching, Nogneng, O., Eurographics 2017
Functional map networks for analyzing and exploring large shape collections Huang, Wang, Guibas, SIGGRAPH 2014
Manifold OptimizationMADMM: A generic algorithm for non-smooth optimization on manifolds. Kovnatsky, Glashoff, M. Bronstein, ECCV, 2016.
Huang et al., SIGGRAPH 2014
![Page 77: Some recent methods in non-rigid shape matching, with and … · Some recent methods in non-rigid shape matching, with and without learning GAMES 2019 webinar Maks Ovsjanikov Based](https://reader035.vdocuments.net/reader035/viewer/2022081617/60476439d2e1202bf01d2cb8/html5/thumbnails/77.jpg)
Consistency via Latent Space Optimization
Application to Co-segmentation:
Image Co-Segmentation via Consistent Functional MapsWang, Huang, Guibas, CVPR 2013
![Page 78: Some recent methods in non-rigid shape matching, with and … · Some recent methods in non-rigid shape matching, with and without learning GAMES 2019 webinar Maks Ovsjanikov Based](https://reader035.vdocuments.net/reader035/viewer/2022081617/60476439d2e1202bf01d2cb8/html5/thumbnails/78.jpg)
Other Extensions
Measuring Differences between shapes
Tangent Vector Field processing
Maps Between Partial shapesPartial Functional Correspondence, Rodolà, Cosmo, A. Bronstein, Torsello, Cremers, CGF 2017
Map-Based Exploration of Intrinsic Shape Differences and Variability Rustamov, Ovsjanikov, Azencot, Ben-Chen, Chazal, Guibas, SIGGRAPH 2014
An Operator Approach to Tangent Vector Field Processing Azencot, Ben-Chen, Chazal, Ovsjanikov, SGP, 2013.
Azencot et al., SGP 2014
![Page 79: Some recent methods in non-rigid shape matching, with and … · Some recent methods in non-rigid shape matching, with and without learning GAMES 2019 webinar Maks Ovsjanikov Based](https://reader035.vdocuments.net/reader035/viewer/2022081617/60476439d2e1202bf01d2cb8/html5/thumbnails/79.jpg)
Some Open Problems
What is the optimal choice of basis?
How to guarantee a continuous pointwise map?
What are better deformation models?
Shape interpolation without converting to p2p?
![Page 80: Some recent methods in non-rigid shape matching, with and … · Some recent methods in non-rigid shape matching, with and without learning GAMES 2019 webinar Maks Ovsjanikov Based](https://reader035.vdocuments.net/reader035/viewer/2022081617/60476439d2e1202bf01d2cb8/html5/thumbnails/80.jpg)
Conclusions
Functional maps provide an efficient way to encode “generalized” mappings.
Can be computed in practice with simple (least squares) optimization.
Many different constraints can be incorporated: pointwise maps, consistency in collections, etc.
Recent work incorporating learning of descriptors.
![Page 81: Some recent methods in non-rigid shape matching, with and … · Some recent methods in non-rigid shape matching, with and without learning GAMES 2019 webinar Maks Ovsjanikov Based](https://reader035.vdocuments.net/reader035/viewer/2022081617/60476439d2e1202bf01d2cb8/html5/thumbnails/81.jpg)
Questions?
Acknowledgements:A. Poulenard, M.-J. Rakotosaona, Y. Ponty, J.-M. Rouffosse, A. Sharma, S. Melzi, E. Rodolà, J. Ren, P. Wonka …. Work supported by KAUST OSR Award No. CRG-2017-3426, a gift from Nvidia and the ERC Starting Grant StG-2017-758800 (EXPROTEA)
Thank You