Diffusion-geometricmaximally stable component detection in

deformable shapes

Roee Litman, Alexander Bronstein, Michael Bronstein

Diffusion-geometricmaximally stable component detection in

deformable shapes

In a nutshell…

MSERMaximally Stable Extremal Region

Diffusion Geometry

ShapeMSER

The Feature Approachfor Images

Deformable Shape

Analysis

Feature Approach in Images

Feature based methods are the infrastructure laid in the base of many computer vision algorithms:

– Content-based image retrieval– Video tracking– Panorama alignment– 3D reconstruction form stereo

3

Problem formulation

• Find a semi-local feature detector– High repeatability– Invariance to isometric deformation– Robustness to noise, sampling, etc.

• Add discriminative descriptor

RESULTSThe “what”

Visual Example

6

Visual Example

7

More Results

8

(Taken from the TOSCA dataset)

Horse regions + Human regions

Region Matching

Query 1st, 2nd, 4th, 10th, and 15th matches9

3D Human Scans

10Taken from the SCAPE dataset

Scanned Region Matching

Query 1st, 2nd, 4th, 10th, and 15th matches11

Volume vs. Surface

12

Volume & surface isometry Boundary isometryOriginal

Volumetric Shapes

• Usually shapes are modeled as 2D boundary of a 3D shape.

• Volumetric shape model better captures "natural" behavior of non-rigid deformations.(Raviv et-al)

• Diffusion geometry terms can easily be applied to volumes

• 2D Meshes can be voxelized

13

Volumetric Regions

14Taken from the SCAPE dataset

METHODOLOGY(The “how)”

Original MSER (Matas et-al)

MSER

• Popular image blob detector• Near-linear complexity:• High repeatability [Mikolajczyk et al. 05]• Robust to affine transformation and

illumination changes

nnO loglog

17

MSER – In a nutshell

1. Threshold image at consecutive gray-levels2. Search regions whose area stay nearly the

same through a wide range of thresholds

• Efficient detection of maximally stable regions requires construction of a component tree

18

MSER – In a nutshell

19

Algorithm overview

Algorithm overview

Represent as weighted

graph

Component tree

Stable component detection

Represent as

weighted graph

Component tree

Stable component detection

Algorithm overview

Represent as weighted

graph

Component tree

Stable component detection

Represent as

weighted graph

Image as weighted graph

• An undirected graph can be created from an image, where:– Vertices are pixels– Edges by adjacency rule, e.g. 4-neiborhood

23

Weighting the graph

In images• Gray-scale as vertex-weight• Color as edge-weight [Forssen]

In Shapes• Curvature (not deformation invariant)• Diffusion Geometry

Weighting Option

• For every point on the shape:• Calculate the prob. of a random walk to return

to the same point.– Similar to Gaussian curvature– Intrinsic, i.e. – deformation invariant

Weight example

Color-mapped Level-set animation

26

Diffusion Geometry

• Analysis of diffusion (random walk) processes• Governed by the heat equation

• Solution is heat distributionat point at time

27

txf ,

tx

Heat-Kernel

• Given– Initial condition – Boundary condition, if these’s a boundary

• Solve using:

• i.e. - find the “heat-kernel”

28

0,0 xfxf

X t ydayfyxhtxf 0,,

yxht ,

The probability density for a transitionby random walk of length ,from to

Probabilistic Interpretation

yxht ,

x

t

y

x y29

Spectral Interpretation

• How to calculate ?• Heat kernel can be calculated directly from

eigen-decomposition of the Laplacain

• By spectral decomposition theorem:

xx iiiX

i

iit

t yxeyxh i ,

30

yxht ,

Laplace-Beltrami Eigenfunctions

Deformation Invariance

Computational aspects

• Shapes are discretized as triangular meshes– Can be expressed as undirected graph– Heat kernel & eigenfunctions are vectors

• Discrete Laplace-Beltrami operator

• Several weight schemes for • is usually discrete area elements

j

jiiji

iX ffwa

f1

ijw

ia

33

Computational aspects

• In matrix notation

• Solve eigendecomposition problem

iii AW

WfAfX1

j

jiiji

iX ffwa

f1

34

Auto-diffusivity

• Special case - • The chance of returning to after time • Related to Gaussian curvature by

• Now we can attach scalar value to shapes!

x t

xxht ,

2

3

11

4

1, xOxK

txxht

xK

36

Weight example

Color-mapped Level-set animation

37

Algorithm overview

Represent as weighted

graph

Component tree

Stable component detection

Represent as

weighted graph

Component tree

Stable component detection

The Component Tree

• Tree construction is a pre-process of stable region detection

• Contains level-set hierarchy,i.e. nesting relations.

• Constructed based on a weighted graph (vertex- or edge-weight)

• Tree’s nodes are level-sets(of the graph’s cross-sections)

39

Tree Example

“Graphic” Example

• A graph• Edge-weighted• 7 Cross-Section• 5 Cross-section• Two 5 level-sets

(with altitude 4)• Every level-set has

– Size (area)– Altitude (maximal weight)

1

4

78

9

8

4

144

1

4

78

9

8

4

1

Tree Construction

46

1

4

78

9

8

4

1

1 4

7

8

4

Algorithm overview

Represent as weighted

graph

Component tree

Stable component detection

Represent as

weighted graph

Component tree

Stable component detection

Detection Process

• For every leaf component in the tree:– “Climb” the tree to its root, creating the sequence:– Calculate component stability

– Local maxima of the sequenceare “Maximally stable components”

11

11

ii

iii

i

iii CACA

CwCwCA

CA

CwCACs

KCCC ...21

132 ,...,, KCsCsCs

48

PERFORMANCEThe Detils

Benchmarking The Method

• Method was tested on SHREC 2010 data-set:– 3 basic shapes (human, dog & horse)– 9 transformations, applied in 5 different strengths– 138 shapes in total

50

Original Deformation

Noise

Scale

Holes

Results

51

Quantitative Results

• Vertex-wise correspondences were given• Regions were projected onto another shape,

and overlap ratio was measured• Overlap ratio between a region and its

projected counterpart is

• Repeatability is the percent of regions with overlap above a threshold

R

'R

'

'',

RRA

RRARRO

52

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

10

20

30

40

50

60

70

80

90

100

X: 0.7586Y: 64.06

overlap

repe

atab

ility

(%

)Repeatability

53

65% at 0.75

Conclusion

• Stable region detector for deformable shapes• Generic detection framework:

– Vertex- and edge-weighted graph representation– Works on surface and/or volume data

• Partial matching & retrieval potential• Tested quantitatively (on SHREC10)

Thank You

Any Questions?