1 michael bronstein heat diffusion descriptors deformable michael bronstein weizmann institute of...
TRANSCRIPT
1Michael Bronstein Heat diffusion descriptors
deformable
Michael Bronstein
Weizmann Institute of Science, 4 November 2010
Institute of Computational ScienceUniversita della Svizzera Italiana
Lugano, Switzerland
shapesfor
Heat diffusion descriptors
2Michael Bronstein Heat diffusion descriptors
Dan Raviv
Technion
Ron Kimmel
Technion
Maks Ovsjanikov
Stanford
Leo Guibas
Stanford
Iasonas Kokkinos
ECP Paris
Alex Bronstein
TAU
5Michael Bronstein Heat diffusion descriptors
Bags of words
Notre Dame de Paris is a Gothic cathedral in the fourth quarter of Paris, France. It was the first Gothic architecture cathedral, and its construction spanned the Gothic period.
con
stru
ctio
nar
chit
ectu
reIt
aly
Fra
nce
cath
edra
lch
urc
hb
asili
caP
aris
Ro
me
Go
thic
Ro
man
St. Peter’s basilica is the largest church in world, located in Rome, Italy. As a work of architecture, it is regarded as the best building of its age in Italy.
Notre Dame de Paris is a Gothic cathedral in the fourth quarter of Paris, France. It was the first Gothic architecture cathedral, and its construction spanned the Gothic period.
St. Peter’s basilica is the largest church in world, located in Rome, Italy. As a work of architecture, it is regarded as the best building of its age in Italy.
6Michael Bronstein Heat diffusion descriptors
Outline
Feature descriptor
Geometric words
Bag of geometric words
Geometric expressions
Spatially-sensitive bag of words
“ ”
“ ”
Volumetric descriptors
Scale invariance
7Michael Bronstein Heat diffusion descriptors
Curvature
Rigid Bending Topology
Integralvolume1
Scale
Spin image2
Shape context3
Representation
HKS4
SI-HKS5
vHKS6
1 Gelfand et al. 2005; 2 Johnson, Hebert 1999; 3 Belongie et al. 2002; 4 Sun et al. 2009
Shape descriptors
Any
Volume/Mesh
Any
Any
Any
Any
Volume/Mesh
5 B, Kokkinos 2010; 6 Raviv, BBK 2010
8Michael Bronstein Heat diffusion descriptors
Diffusion geometry
Amount of heat transferred from point x to point y in time t
Heat equation
where
- positive semidefinite Laplace-Beltrami
operator
- heat distribution
Fundamental solution (heat kernel, ) – heat equation solution
for initial conditions
Spectral expression
9Michael Bronstein Heat diffusion descriptors
Sun, Ovsjanikov, Guibas, 2009
Heat kernel interpretation
Geometric interpretation: “multiscale Gaussian curvature”
Probabilistic interpretation: the probability of a random walk to remain
at point x after time t.
10Michael Bronstein Heat diffusion descriptors
Sun, Ovsjanikov, Guibas, 2009
Heat kernel signature
Multiscale descriptor
Time (scale)
■ Intrinsic, hence deformation-invariant
■ Provably informative
■ Efficiently computable on different shape representations
■Multiscale
11Michael Bronstein Heat diffusion descriptors
Ovsjanikov, BB, Guibas, 2009BB. Ovsjanikov, Guibas 2010
Shape
Geometric vocabulary
Bag of geometric words
12Michael Bronstein Heat diffusion descriptors
Index in geometric vocabulary1 64
Ovsjanikov, BB, Guibas, 2009BB. Ovsjanikov, Guibas 2010
Bags of geometric words
13Michael Bronstein Heat diffusion descriptors
B et al. 2010
SHREC 2010: Robust shape retrieval benchmark
Transformation
Query set
Database (>1K shapes)
15Michael Bronstein Heat diffusion descriptors
Bags of words using HKS descriptor,vocabulary of size 48
Shape
Bags of words using spin imagedescriptor
Performance results
Toldo et al. 2009
Toldo et al. 2009B et al. 2010
Performance criterion: mean average precision (mAP) in %
16Michael Bronstein Heat diffusion descriptors
Scale invariance
Original shape Scaled by
Not scale invariant!
17Michael Bronstein Heat diffusion descriptors
Scale-invariant HKS
B, Kokkinos CVPR 2010
Log scale-space
Scaling = shift and multiplicative constant
log + d/d
Undo scaling
Fourier transformmagnitude
Undo shift
0 100 200 300-15
-10
-5
0
t0 100 200 300
-0.04
-0.03
-0.02
-0.01
0
t0 2 4 6 8 10 12 14 16 18 20
0
1
2
3
4
=2k/T
20Michael Bronstein Heat diffusion descriptors
B, Kokkinos 2010
HKS, vocabulary of size 48 SI-HKS, vocabulary of size 48
HKS vs SI-HKS
Performance criterion: mean average precision (mAP) in %
21Michael Bronstein Heat diffusion descriptors
Expressions
In math science, matrix decomposition is a factorization of a matrix into some canonical form. Each type of decomposition is used in a particular problem.
In biological science, decomposition is the process of organisms to break down into simpler form of matter. Usually, decomposition occurs after death.
Matrix is a science fiction movie released in 1999. Matrix refers to a simulated reality created by machines in order to subdue the human population.
mat
rix
dec
om
po
siti
on
mat
rix
fact
ori
zati
on
scie
nce
fic
tio
nca
no
nic
al f
orm
In math science, matrix decomposition is a factorization of a matrix into some canonical form. Each type of decomposition is used in a particular problem.
In biological science, decomposition is the process of organisms to break down into simpler form of matter. Usually, decomposition occurs after death.
Matrix is a science fiction movie released in 1999. Matrix refers to a simulated reality created by machines in order to subdue the human population.
mat
rix
dec
om
po
siti
on is a
the of in to by
scie
nce
form
In math science, matrix decomposition is a factorization of a matrix into some canonical form. Each type of decomposition is used in a particular problem.
Matrix is a science fiction movie released in 1999. Matrix refers to a simulated reality created by machines in order to subdue the human population.
Ovsjanikov, BB & Guibas 2009
22Michael Bronstein Heat diffusion descriptors
Expressions
In math science, matrix decomposition is a factorization of a matrix into some canonical form. Each type of decomposition is used in a particular problem.
mat
rix
dec
om
po
siti
on is a
the of in to by
scie
nce
form
In particular matrix used type a some science, decomposition form a factorization of is canonical. matrix math decomposition is in a Each problem. into of
mat
rix
dec
om
po
siti
on
mat
rix
fact
ori
zati
on
scie
nce
fic
tio
nca
no
nic
al f
orm
Ovsjanikov, BB & Guibas 2009
23Michael Bronstein Heat diffusion descriptors
Geometric expressions
Ovsjanikov, BB & Guibas 2009
“Yellow Yellow”Yellow
No total order between points (only “far” and “near”)
Geometric expression = a pair of spatially close geometric words
24Michael Bronstein Heat diffusion descriptors
Ovsjanikov, BB & Guibas 2009
Spatially-sensitive bags of words
25Michael Bronstein Heat diffusion descriptors
B et al. 2010
HKS, vocabulary of size 48 Spatially-sensitive HKS, vocabulary of size 8x8
HKS vs SI-HKS
Performance criterion: mean average precision (mAP) in %
26Michael Bronstein Heat diffusion descriptors
Is our shape model good?
Raviv, BBK 2010
Boundary ∂X
Interior
X
27Michael Bronstein Heat diffusion descriptors
Is our shape model good?
Camel illustration from Sumner et al.Raviv, BBK 2010
Volume isometryBoundary isometry
Preserves geodesic distances on
the boundary surface
Preserves geodesic distances
inside the volume
28Michael Bronstein Heat diffusion descriptors
where
Diffusion equation
Raviv, BBK 2010
Volumetric diffusionBoundary diffusion
- Laplace-Beltrami operator - Euclidean Laplacian
- normal to boundary surface
29Michael Bronstein Heat diffusion descriptors
where
Heat kernels
Raviv, BBK 2010
Volumetric heat kernelBoundary heat kernel
Geometric interpretation
“Multiscale Gaussian curvature”
30Michael Bronstein Heat diffusion descriptors
Heat kernel signatures
Raviv, BBK 2010
vHKSHKS
Boundary+volume isometry Boundary+volume isometry
Boundary isometry Boundary isometry
32Michael Bronstein Heat diffusion descriptors
HKS, vocabulary of size 48 vHKS, vocabulary of size 48
HKS vs vHKS
Raviv, BBK 2010
Performance criterion: mean average precision (mAP) in %
33Michael Bronstein Heat diffusion descriptors
Summary
Feature descriptor
Geometric words
Bag of geometric words
Geometric expressions
Spatially-sensitive bag of words
“ ”
“ ”
Volumetric descriptors
Scale invariance