1 numerical geometry of non-rigid shapes invariant shape similarity invariant shape similarity ©...

55
1 al Geometry of Non-Rigid Shapes Invariant shape similarity Invariant shape similarity © Alexander & Michael Bronstein, 2006-2009 © Michael Bronstein, 2010 tosca.cs.technion.ac.il/book 048921 Advanced topics in vision Processing and Analysis of Geometric Shapes EE Technion, Spring 2010

Post on 24-Jan-2016

227 views

Category:

Documents


1 download

TRANSCRIPT

Page 1: 1 Numerical Geometry of Non-Rigid Shapes Invariant shape similarity Invariant shape similarity © Alexander & Michael Bronstein, 2006-2009 © Michael Bronstein,

1Numerical Geometry of Non-Rigid Shapes Invariant shape similarity

Invariant shape similarity

© Alexander & Michael Bronstein, 2006-2009© Michael Bronstein, 2010tosca.cs.technion.ac.il/book

048921 Advanced topics in visionProcessing and Analysis of Geometric Shapes

EE Technion, Spring 2010

Page 2: 1 Numerical Geometry of Non-Rigid Shapes Invariant shape similarity Invariant shape similarity © Alexander & Michael Bronstein, 2006-2009 © Michael Bronstein,

2Numerical Geometry of Non-Rigid Shapes Invariant shape similarity

Invariant similarity

SIMILARITY

TRANSFORMATION

Page 3: 1 Numerical Geometry of Non-Rigid Shapes Invariant shape similarity Invariant shape similarity © Alexander & Michael Bronstein, 2006-2009 © Michael Bronstein,

3Numerical Geometry of Non-Rigid Shapes Invariant shape similarity

Equivalence

Equal

Congruent Isometric

Page 4: 1 Numerical Geometry of Non-Rigid Shapes Invariant shape similarity Invariant shape similarity © Alexander & Michael Bronstein, 2006-2009 © Michael Bronstein,

4Numerical Geometry of Non-Rigid Shapes Invariant shape similarity

Equivalence

Equivalence is a binary relation on the space of shapes

which

for all satisfies

Reflexivity:

Symmetry:

Transitivity: Can be expressed as a binary function

if and only if

Quotient space is the space of equivalence classes

Page 5: 1 Numerical Geometry of Non-Rigid Shapes Invariant shape similarity Invariant shape similarity © Alexander & Michael Bronstein, 2006-2009 © Michael Bronstein,

5Numerical Geometry of Non-Rigid Shapes Invariant shape similarity

Equivalence

Page 6: 1 Numerical Geometry of Non-Rigid Shapes Invariant shape similarity Invariant shape similarity © Alexander & Michael Bronstein, 2006-2009 © Michael Bronstein,

6Numerical Geometry of Non-Rigid Shapes Invariant shape similarity

Equivalence

All deformations of the

human shape are “the same”

Page 7: 1 Numerical Geometry of Non-Rigid Shapes Invariant shape similarity Invariant shape similarity © Alexander & Michael Bronstein, 2006-2009 © Michael Bronstein,

7Numerical Geometry of Non-Rigid Shapes Invariant shape similarity

Similarity

Shapes are rarely truly equivalent (e.g., due to acquisition noise or

since

most shapes are rigid)

We want to account for “almost equivalence” or similarity

-similar = -isometric (w.r.t. some metric)

Define a distance on the shape space quantifying the degree of

dissimilarity of shapes

Page 8: 1 Numerical Geometry of Non-Rigid Shapes Invariant shape similarity Invariant shape similarity © Alexander & Michael Bronstein, 2006-2009 © Michael Bronstein,

8Numerical Geometry of Non-Rigid Shapes Invariant shape similarity

Similarity

A monkey shape is

more similar to a

deformation of a

monkey shape…

…than to a

human shape

Page 9: 1 Numerical Geometry of Non-Rigid Shapes Invariant shape similarity Invariant shape similarity © Alexander & Michael Bronstein, 2006-2009 © Michael Bronstein,

9Numerical Geometry of Non-Rigid Shapes Invariant shape similarity

Isometry-invariant distance

Non-negative function satisfying for all

Corollary: is a metric on the quotient space

Similarity: and are -isometric;

and are -isometric

(In particular, if and only if )

Symmetry:

Triangle inequality:

Given discretized shapes and

sampled with radius

Consistency to sampling:

Page 10: 1 Numerical Geometry of Non-Rigid Shapes Invariant shape similarity Invariant shape similarity © Alexander & Michael Bronstein, 2006-2009 © Michael Bronstein,

10Numerical Geometry of Non-Rigid Shapes Invariant shape similarity

Canonical forms distance

Compute Hausdorff distance over all isometries in

Minimum-distortion

embedding

Minimum-distortion

embedding

No fixed embedding space will give distortion-less canonical forms

Page 11: 1 Numerical Geometry of Non-Rigid Shapes Invariant shape similarity Invariant shape similarity © Alexander & Michael Bronstein, 2006-2009 © Michael Bronstein,

11Numerical Geometry of Non-Rigid Shapes Invariant shape similarity

Gromov-Hausdorff distance

Isometric

embedding

Isometric

embedding

Mikhail Gromov

Gromov-Hausdorff distance: include

into minimization

Page 12: 1 Numerical Geometry of Non-Rigid Shapes Invariant shape similarity Invariant shape similarity © Alexander & Michael Bronstein, 2006-2009 © Michael Bronstein,

12Numerical Geometry of Non-Rigid Shapes Invariant shape similarity

Properties of Gromov-Hausdorff distance

Metric on the quotient space of isometries of shapes

Similarity: and are -isometric;

and are -isometric

Consistent to sampling: given discretized shapes

and sampled with radius

Generalization of Hausdorff distance:

Hausdorff distance between subsets of a metric space

Gromov-Hausdorff distance between metric spacesGromov, 1981

Page 13: 1 Numerical Geometry of Non-Rigid Shapes Invariant shape similarity Invariant shape similarity © Alexander & Michael Bronstein, 2006-2009 © Michael Bronstein,

13Numerical Geometry of Non-Rigid Shapes Invariant shape similarity

Alternative definition I (metric coupling)

is the disjoint union of and

the (semi-) metric satisfies and

where

Mémoli, 2008

Page 14: 1 Numerical Geometry of Non-Rigid Shapes Invariant shape similarity Invariant shape similarity © Alexander & Michael Bronstein, 2006-2009 © Michael Bronstein,

14Numerical Geometry of Non-Rigid Shapes Invariant shape similarity

Optimization over translates into finding the values of

Mémoli, 2008

A lot of constraints!

Alternative definition I (metric coupling)

Page 15: 1 Numerical Geometry of Non-Rigid Shapes Invariant shape similarity Invariant shape similarity © Alexander & Michael Bronstein, 2006-2009 © Michael Bronstein,

15Numerical Geometry of Non-Rigid Shapes Invariant shape similarity

Correspondence

A subset is called a correspondence between and

if for every there exists at least one such that

and similarly for every there exists such that

Particular case: given and

Page 16: 1 Numerical Geometry of Non-Rigid Shapes Invariant shape similarity Invariant shape similarity © Alexander & Michael Bronstein, 2006-2009 © Michael Bronstein,

16Numerical Geometry of Non-Rigid Shapes Invariant shape similarity

Correspondence distortion

The distortion of correspondence is defined as

Page 17: 1 Numerical Geometry of Non-Rigid Shapes Invariant shape similarity Invariant shape similarity © Alexander & Michael Bronstein, 2006-2009 © Michael Bronstein,

17Numerical Geometry of Non-Rigid Shapes Invariant shape similarity

Alternative definition II (correspondence distortion)

1. Show that for any there exists with

Proof sketch

Since , by definition of , and are subspaces of

some such that

Let

By triangle inequality, for

Page 18: 1 Numerical Geometry of Non-Rigid Shapes Invariant shape similarity Invariant shape similarity © Alexander & Michael Bronstein, 2006-2009 © Michael Bronstein,

18Numerical Geometry of Non-Rigid Shapes Invariant shape similarity

Alternative definition II (correspondence distortion)

2. Show that for any

Let

It is sufficient to show that there is a (semi-)metric on the disjoint union

such that , , and

Construct the metric as follows

(in particular, for ).

Page 19: 1 Numerical Geometry of Non-Rigid Shapes Invariant shape similarity Invariant shape similarity © Alexander & Michael Bronstein, 2006-2009 © Michael Bronstein,

19Numerical Geometry of Non-Rigid Shapes Invariant shape similarity

Alternative definition II (correspondence distortion)

First,

For each

Since for ,

Second, we need to show that is a (semi-)metric on

On and , it is straightforward

We only need to show metric properties hold on

Page 20: 1 Numerical Geometry of Non-Rigid Shapes Invariant shape similarity Invariant shape similarity © Alexander & Michael Bronstein, 2006-2009 © Michael Bronstein,

20Numerical Geometry of Non-Rigid Shapes Invariant shape similarity

Alternative definition III

measures how much is distorted by when embedded into

Page 21: 1 Numerical Geometry of Non-Rigid Shapes Invariant shape similarity Invariant shape similarity © Alexander & Michael Bronstein, 2006-2009 © Michael Bronstein,

21Numerical Geometry of Non-Rigid Shapes Invariant shape similarity

measures how much is distorted by when embedded into

Alternative definition III

Page 22: 1 Numerical Geometry of Non-Rigid Shapes Invariant shape similarity Invariant shape similarity © Alexander & Michael Bronstein, 2006-2009 © Michael Bronstein,

22Numerical Geometry of Non-Rigid Shapes Invariant shape similarity

measures how far is from being the inverse of

Alternative definition III

Page 23: 1 Numerical Geometry of Non-Rigid Shapes Invariant shape similarity Invariant shape similarity © Alexander & Michael Bronstein, 2006-2009 © Michael Bronstein,

23Numerical Geometry of Non-Rigid Shapes Invariant shape similarity

Generalized MDS

A. Bronstein, M. Bronstein & R. Kimmel, 2006

Page 24: 1 Numerical Geometry of Non-Rigid Shapes Invariant shape similarity Invariant shape similarity © Alexander & Michael Bronstein, 2006-2009 © Michael Bronstein,

24Numerical Geometry of Non-Rigid Shapes Invariant shape similarity

Difficulties

How to represent points on ?

Global parametrization is not always available.

Some local representation is required in general case.

No more closed-form expression for .

Metric needs to be approximated.

Minimization algorithm.

Page 25: 1 Numerical Geometry of Non-Rigid Shapes Invariant shape similarity Invariant shape similarity © Alexander & Michael Bronstein, 2006-2009 © Michael Bronstein,

25Numerical Geometry of Non-Rigid Shapes Invariant shape similarity

Local representation

is sampled at and represented as a

triangular mesh .

Any point falls into one of the triangles .

Within the triangle, it can be represented as convex combination

of triangle vertices ,

Barycentric coordinates .

We will need to handle discrete indices in minimization algorithm.

Page 26: 1 Numerical Geometry of Non-Rigid Shapes Invariant shape similarity Invariant shape similarity © Alexander & Michael Bronstein, 2006-2009 © Michael Bronstein,

26Numerical Geometry of Non-Rigid Shapes Invariant shape similarity

Geodesic distances

Distance terms can be precomputed, since are

fixed.

How to compute distance terms ?

No more closed-form expression.

Cannot be precomputed, since are minimization variables.

can fall anywhere on the mesh.

Precompute for all .

Approximate

for any .

Page 27: 1 Numerical Geometry of Non-Rigid Shapes Invariant shape similarity Invariant shape similarity © Alexander & Michael Bronstein, 2006-2009 © Michael Bronstein,

27Numerical Geometry of Non-Rigid Shapes Invariant shape similarity

Geodesic distance approximation

Approximation from .

First order accurate:

Consistent with data:

Symmetric:

Smoothness: is and a closed-form expression

for its derivatives is available to minimization algorithm.

Might be only at some points or along some lines.

Efficiently computed: constant complexity independent of .

Page 28: 1 Numerical Geometry of Non-Rigid Shapes Invariant shape similarity Invariant shape similarity © Alexander & Michael Bronstein, 2006-2009 © Michael Bronstein,

28Numerical Geometry of Non-Rigid Shapes Invariant shape similarity

Geodesic distance approximation

Compute for .

falls into triangle and is represented as

Particular case:

Hence, we can precompute distances

How to compute from ?

Page 29: 1 Numerical Geometry of Non-Rigid Shapes Invariant shape similarity Invariant shape similarity © Alexander & Michael Bronstein, 2006-2009 © Michael Bronstein,

29Numerical Geometry of Non-Rigid Shapes Invariant shape similarity

Geodesic distance approximation

We have already encountered this problem in fast marching.

Wavefront arrives at triangle vertex at time .

When does it arrive to ?

Adopt planar wavefront model.

Distance map is linear in the triangle (hence, linear in )

Solve for coefficients and obtain a linear interpolant

Page 30: 1 Numerical Geometry of Non-Rigid Shapes Invariant shape similarity Invariant shape similarity © Alexander & Michael Bronstein, 2006-2009 © Michael Bronstein,

30Numerical Geometry of Non-Rigid Shapes Invariant shape similarity

Geodesic distance approximation

General case: falls into triangle and is

represented

as

Apply previous steps in triangle to obtain

Apply once again in triangle to obtain

Page 31: 1 Numerical Geometry of Non-Rigid Shapes Invariant shape similarity Invariant shape similarity © Alexander & Michael Bronstein, 2006-2009 © Michael Bronstein,

31Numerical Geometry of Non-Rigid Shapes Invariant shape similarity

A four-step dance

Page 32: 1 Numerical Geometry of Non-Rigid Shapes Invariant shape similarity Invariant shape similarity © Alexander & Michael Bronstein, 2006-2009 © Michael Bronstein,

32Numerical Geometry of Non-Rigid Shapes Invariant shape similarity

Minimization algorithm

How to minimize the generalized stress?

Particular case: L2 stress

Fix all and all except for some .

Stress as a function of only becomes quadratic

Page 33: 1 Numerical Geometry of Non-Rigid Shapes Invariant shape similarity Invariant shape similarity © Alexander & Michael Bronstein, 2006-2009 © Michael Bronstein,

33Numerical Geometry of Non-Rigid Shapes Invariant shape similarity

Quadratic stress

Quadratic stress

is positive semi-definite.

is convex in (but not necessarily in

together).

Page 34: 1 Numerical Geometry of Non-Rigid Shapes Invariant shape similarity Invariant shape similarity © Alexander & Michael Bronstein, 2006-2009 © Michael Bronstein,

34Numerical Geometry of Non-Rigid Shapes Invariant shape similarity

Quadratic stress

Closed-form solution for minimizer of

Problem: solution might be outside the

triangle.

Solution: find constrained minimizer

Closed-form solution still exists.

Page 35: 1 Numerical Geometry of Non-Rigid Shapes Invariant shape similarity Invariant shape similarity © Alexander & Michael Bronstein, 2006-2009 © Michael Bronstein,

35Numerical Geometry of Non-Rigid Shapes Invariant shape similarity

Minimization algorithm

Initialize

For each

Fix and compute gradient

Select corresponding to maximum .

Compute minimizer

If constraints are active

translate to adjacent triangle.

Iterate until convergence…

Page 36: 1 Numerical Geometry of Non-Rigid Shapes Invariant shape similarity Invariant shape similarity © Alexander & Michael Bronstein, 2006-2009 © Michael Bronstein,

36Numerical Geometry of Non-Rigid Shapes Invariant shape similarity

How to move to adjacent triangles?

Three cases

All : inside triangle.

: on edge opposite to .

: on vertex .

inside on edge on vertex

Page 37: 1 Numerical Geometry of Non-Rigid Shapes Invariant shape similarity Invariant shape similarity © Alexander & Michael Bronstein, 2006-2009 © Michael Bronstein,

37Numerical Geometry of Non-Rigid Shapes Invariant shape similarity

Point on edge

on edge opposite to .

If edge is not shared by any other triangle

we are on the boundary – no translation.

Otherwise, express the point as in triangle .

contains same values as .

May be permuted due to different vertex

ordering in .

Complication: is not on the edge.

Evaluate gradient in .

If points inside triangle, update to .

Page 38: 1 Numerical Geometry of Non-Rigid Shapes Invariant shape similarity Invariant shape similarity © Alexander & Michael Bronstein, 2006-2009 © Michael Bronstein,

38Numerical Geometry of Non-Rigid Shapes Invariant shape similarity

Point on vertex

on vertex .

For each triangle sharing vertex

Express point as in .

Evaluate gradient in .

Reject triangles with pointing

outside.

Select triangle with maximum

.

Update to .

Page 39: 1 Numerical Geometry of Non-Rigid Shapes Invariant shape similarity Invariant shape similarity © Alexander & Michael Bronstein, 2006-2009 © Michael Bronstein,

39Numerical Geometry of Non-Rigid Shapes Invariant shape similarity

MDS vs GMDS

Stress

Generalized MDSMDS

Generalized stress

Analytic expression for

Nonconvex problem

Variables: Euclidean

coordinates

of the points

must be interpolated

Nonconvex problem

Variables: points on in

barycentric coordinates

Page 40: 1 Numerical Geometry of Non-Rigid Shapes Invariant shape similarity Invariant shape similarity © Alexander & Michael Bronstein, 2006-2009 © Michael Bronstein,

40Numerical Geometry of Non-Rigid Shapes Invariant shape similarity

Multiresolution

Stress is non convex – many small local minima.

Straightforward minimization gives poor results.

How to initialize GMDS?

Multiresolution:

Create a hierarchy of grids in ,

Each grid comprises

Sampling:

Geodesic distance matrix:

Page 41: 1 Numerical Geometry of Non-Rigid Shapes Invariant shape similarity Invariant shape similarity © Alexander & Michael Bronstein, 2006-2009 © Michael Bronstein,

41Numerical Geometry of Non-Rigid Shapes Invariant shape similarity

Multiresolution

Initialize at the coarsest resolution in .

For

Starting at initialization , solve the GMDS problem

Interpolate solution to next resolution level

Return .

Page 42: 1 Numerical Geometry of Non-Rigid Shapes Invariant shape similarity Invariant shape similarity © Alexander & Michael Bronstein, 2006-2009 © Michael Bronstein,

42Numerical Geometry of Non-Rigid Shapes Invariant shape similarity

GMDS

Interpolation

GMDS

Page 43: 1 Numerical Geometry of Non-Rigid Shapes Invariant shape similarity Invariant shape similarity © Alexander & Michael Bronstein, 2006-2009 © Michael Bronstein,

43Numerical Geometry of Non-Rigid Shapes Invariant shape similarity

Multiresolution encore

So far, we created a hierarchy of embedded spaces .

One step further: create a hierarchy of embedding spaces

.

Page 44: 1 Numerical Geometry of Non-Rigid Shapes Invariant shape similarity Invariant shape similarity © Alexander & Michael Bronstein, 2006-2009 © Michael Bronstein,

44Numerical Geometry of Non-Rigid Shapes Invariant shape similarity

Labeling problem

Torresani, Kolmogorov, Rother 2008Wang, B 2010

Build a graph with vertices and edges

Label each vertex

Minimum distortion correspondence = graph labeling problem

Efficient solvers with good global convergence properties

Complexity:

Hierarchical solution complexity can be lowered to

Page 45: 1 Numerical Geometry of Non-Rigid Shapes Invariant shape similarity Invariant shape similarity © Alexander & Michael Bronstein, 2006-2009 © Michael Bronstein,

45Numerical Geometry of Non-Rigid Shapes Invariant shape similarity

MATLAB® intermezzoGMDS

Page 46: 1 Numerical Geometry of Non-Rigid Shapes Invariant shape similarity Invariant shape similarity © Alexander & Michael Bronstein, 2006-2009 © Michael Bronstein,

46Numerical Geometry of Non-Rigid Shapes Invariant shape similarity

Discrete Gromov-Hausdorff distance

Two coupled GMDS problems

Can be cast as a constrained problem

Bronstein, Bronstein & Kimmel, 2006

Page 47: 1 Numerical Geometry of Non-Rigid Shapes Invariant shape similarity Invariant shape similarity © Alexander & Michael Bronstein, 2006-2009 © Michael Bronstein,

47Numerical Geometry of Non-Rigid Shapes Invariant shape similarity

Numerical example

Canonical forms (MDS based on 500 points)

Gromov-Hausdorff distance(GMDS based on 50 points)

Bronstein, Bronstein & Kimmel, 2006

Page 48: 1 Numerical Geometry of Non-Rigid Shapes Invariant shape similarity Invariant shape similarity © Alexander & Michael Bronstein, 2006-2009 © Michael Bronstein,

48Numerical Geometry of Non-Rigid Shapes Invariant shape similarity

Extrinsic similarity using Gromov-Hausdorff distance

Mémoli (2008)

Connection between Euclidean GH and ICP distances:

Congruence Euclidean isometry

EXTRINSIC SIMILARITY

ICP distance: GH distance with Euclidean

metric:

Mémoli, 2008

Page 49: 1 Numerical Geometry of Non-Rigid Shapes Invariant shape similarity Invariant shape similarity © Alexander & Michael Bronstein, 2006-2009 © Michael Bronstein,

49Numerical Geometry of Non-Rigid Shapes Invariant shape similarity

Connection to canonical form distance

Page 50: 1 Numerical Geometry of Non-Rigid Shapes Invariant shape similarity Invariant shape similarity © Alexander & Michael Bronstein, 2006-2009 © Michael Bronstein,

50Numerical Geometry of Non-Rigid Shapes Invariant shape similarity

Correspondence again

A different representation for correspondence using indicator functions

defines a valid correspondence if

Page 51: 1 Numerical Geometry of Non-Rigid Shapes Invariant shape similarity Invariant shape similarity © Alexander & Michael Bronstein, 2006-2009 © Michael Bronstein,

51Numerical Geometry of Non-Rigid Shapes Invariant shape similarity

Lp Gromov-Hausdorff distance

We can give an alternative formulation of the Gromov-Hausdorff distance

Can we define an Lp version of the Gromov-Hausdorff distance by

relaxing the above definition?

Page 52: 1 Numerical Geometry of Non-Rigid Shapes Invariant shape similarity Invariant shape similarity © Alexander & Michael Bronstein, 2006-2009 © Michael Bronstein,

52Numerical Geometry of Non-Rigid Shapes Invariant shape similarity

Measure coupling

Let be probability measures defined on and

The measure can be considered as a relaxed version of the indicator

function or as fuzzy correspondence

A measure on is a coupling of and if

for all measurable sets

Mémoli, 2007

(a metric space with measure is called a metric measure or mm space)

Page 53: 1 Numerical Geometry of Non-Rigid Shapes Invariant shape similarity Invariant shape similarity © Alexander & Michael Bronstein, 2006-2009 © Michael Bronstein,

53Numerical Geometry of Non-Rigid Shapes Invariant shape similarity

Gromov-Wasserstein distance

The relaxed version of the Gromov-Hausdorff distance is given by

and is referred to as Gromov-Wasserstein distance [Memoli 2007]

Mémoli, 2007

Page 54: 1 Numerical Geometry of Non-Rigid Shapes Invariant shape similarity Invariant shape similarity © Alexander & Michael Bronstein, 2006-2009 © Michael Bronstein,

54Numerical Geometry of Non-Rigid Shapes Invariant shape similarity

Earth Mover’s distance

Let be a metric space, and measures supported

on

EMD as optimal mass transport:

mass transported from to

distance traveled

Mémoli, 2007

The Wasserstein or Earth Mover’s distance (EMD) is given by

Define the coupling of

Page 55: 1 Numerical Geometry of Non-Rigid Shapes Invariant shape similarity Invariant shape similarity © Alexander & Michael Bronstein, 2006-2009 © Michael Bronstein,

55Numerical Geometry of Non-Rigid Shapes Invariant shape similarity

The analogy

Hausdorff

Mémoli, 2007

Distance between subsets

of a metric space .

Gromov-Hausdorff

Distance between metric spaces

Wasserstein

Distance between subsets

of a metric measure

space .

Gromov-Wasserstein

Distance between metric

measure spaces