1Numerical geometry of non-rigid shapes Multidimensional scaling

Numerical geometry of non-rigid shapes

Multidimensional scaling

Alexander Bronstein, Michael Bronstein, Ron Kimmel© 2007 All rights reserved

2Numerical geometry of non-rigid shapes Multidimensional scaling

How to measure shape similarity?

Extrinsic similarity Intrinsic similarity

Given two shapes and represented as discrete samples

and , compute their similarity

3Numerical geometry of non-rigid shapes Multidimensional scaling

Rigid shape similarity: congruence

Degrees of freedom: Euclidean transformations (rotation+translation)

Classical solution: iterative closest point (ICP) algorithm

Rigid (extrinsic) similarity

Hausdorff distance as a measure of congruence between point clouds

(other distances are usually preferred)

ROTATION MATRIX

TRANSLATION VECTOR

Y. Chen, G. Medioni, 1991P. J. Besl, N. D. McKay, 1992

4Numerical geometry of non-rigid shapes Multidimensional scaling

ICP in fairy tales

Cinderella measuring the glass slipper

Image: Disney

5Numerical geometry of non-rigid shapes Multidimensional scaling

Congruence is not a good criterion for similarity of non-rigid shapes

Geodesic distances are invariant to isometric deformations and can be

easily computed using FMM

Naïve approach: directly compare matrices of geodesic distances

Non-rigid (intrinsic) similarity

Problem: arbitrary ordering of points (permutation of rows and columns)

degrees of freedom

6Numerical geometry of non-rigid shapes Multidimensional scaling

Isometric embedding

Represent the intrinsic geometry in a Euclidean space by isometrically

embedding it into

Treat the resulting images (canonical forms) as rigid shapes, using ICP

or other rigid similarity algorithms

Isometric embedding “undoes” the degrees of freedom

Shape Canonical form

A. Elad, R. Kimmel, CVPR, 2001

7Numerical geometry of non-rigid shapes Multidimensional scaling

Mapmaking problem

Earth (sphere) Planar map

A

BB

A

8Numerical geometry of non-rigid shapes Multidimensional scaling

Embedding error

Theorema Egregium: a sphere has positive

Gaussian curvature, the plane has zero

Gaussian curvature, therefore, they are not

isometric.

Every cartographer knows: impossible to create a distance-preserving

planar map of the Earth!

Does isometric embedding into higher-dimensional spaces exist?

9Numerical geometry of non-rigid shapes Multidimensional scaling

Linial’s example

A

C D

1 1

2

AC D

1 1

1 1

2C D

B

A

B

C

D

B

A

C D

1 1

B

1

Conclusion: generally, isometric embedding does not exist!

10Numerical geometry of non-rigid shapes Multidimensional scaling

Find an embedding that distorts the distances the least

Stress function is a measure of distortion

Minimum distortion embedding

Multidimensional scaling (MDS) problem

where

11Numerical geometry of non-rigid shapes Multidimensional scaling

Examples of canonical forms

Canonical forms

Near-isometric deformations of a shape

A. Elad, R. Kimmel, CVPR, 2001

12Numerical geometry of non-rigid shapes Multidimensional scaling

- an matrix of coordinates in the embedding space

- an constant matrix with values

Matrix expression of the L2-stress

- an matrix-valued function

13Numerical geometry of non-rigid shapes Multidimensional scaling

variables

Non-convex non-linear optimization problem

Using convex optimization techniques is liable to local convergence

Optimum defined up to Euclidean transformation

MDS problem

14Numerical geometry of non-rigid shapes Multidimensional scaling

Instead of , minimize a convex majorizing function

satisfying

Iterative majorization

Start with some and iteratively update

15Numerical geometry of non-rigid shapes Multidimensional scaling

SMACOF algorithm

Majorize the stress by a convex quadratic function

Analytic expression for the minimum of :

SMACOF (Scaling by Minimizing a COnvex Function)

16Numerical geometry of non-rigid shapes Multidimensional scaling

Equivalent to constant-step gradient descent

SMACOF algorithm (cont)

Guarantees monotonically decreasing sequence of stress values

No guarantee of global convergence

Iteration cost:

17Numerical geometry of non-rigid shapes Multidimensional scaling

Application: face recognition

Facial expressions are approximate isometries of the facial surface

Identity = intrinsic geometry

Expression = extrinsic geometry

A. M. Bronstein et al., IJCV, 2005

18Numerical geometry of non-rigid shapes Multidimensional scaling

How to canonize a person?

3D surface acquisition

Smoothing CanonizationCropping

A. M. Bronstein et al., IJCV, 2005

19Numerical geometry of non-rigid shapes Multidimensional scaling

Application: face recognition

Canonical forms

Facial expressions

A. M. Bronstein et al., IJCV, 2005

20Numerical geometry of non-rigid shapes Multidimensional scaling

-1000 -800 -600 -400 -200 0 200 400 600 800 1000-1000

-800

-600

-400

-200

0

200

400

600

800

1000

-1000 -800 -600 -400 -200 0 200 400 600 800 1000-1000

-800

-600

-400

-200

0

200

400

600

800

1000

Rigid similarity Non-rigid similarity(canonical forms)

MichaelAlex

Application: face recognition

A. M. Bronstein et al., IJCV, 2005

21Numerical geometry of non-rigid shapes Multidimensional scaling

Multiresolution MDS: motivation

Coarse MDS problem (N=200)

Fine MDS problem (N~1000)

22Numerical geometry of non-rigid shapes Multidimensional scaling

Multiresolution MDS

Interpolate

Coarse grid solution

Fine grid solution

Fine grid initialization

Solve coarse grid problem

Coarse grid initialization

Solve fine grid problem

Bottom-up: solve coarse grid MDS problem to initialize fine grid

problem

Can be performed on multiple resolution levels

Reduce complexity (less fine-grid iterations)

Reduce the risk of local convergence (good initialization)

23Numerical geometry of non-rigid shapes Multidimensional scaling

Multigrid MDS

Interpolateresidual

Coarse grid solution

Fine grid initialization

Fine grid residual

Solve coarse grid problem

Coarse grid initialization

Top-down: start with a fine grid initialization

Improve the fine grid initialization by solving a coarse grid problem and

propagating the error

Decimate

Coarse grid residual

Improved solution

24Numerical geometry of non-rigid shapes Multidimensional scaling

Problem: the minima of fine and coarse grid problems do not coincide!

Correction

CORRECTION

Add correction term to coarse grid problem to compensate for

inconsistency

Choosing

guarantees that is a coarse grid solution

M. M. Bronstein et al., NLAA, 2006

25Numerical geometry of non-rigid shapes Multidimensional scaling

Another problem: the new coarse grid problem is unbounded!

Modified stress

Modified stress: add a quadratic penalty to the stress

thus resolving translation ambiguity (forcing to be centered at the

origin)

( can be made arbitrarily large by adding a constant to

without changing the stress)

M. M. Bronstein et al., NLAA, 2006

26Numerical geometry of non-rigid shapes Multidimensional scaling

Plugging everything together

Hierarchy of data

Interpolation and decimation operators to transfer variables and

residuals from one resolution level to another

Hierarchy of optimization problems

Relaxation: optimization algorithm used to improve solution

M. M. Bronstein et al., NLAA, 2006

27Numerical geometry of non-rigid shapes Multidimensional scaling

Decimate

Relax1X

Decimate

Solve coarsest grid problem

Interpolate and correct

RelaxRelax

Interpolate and correct

V-cycle

M. M. Bronstein et al., NLAA, 2006

28Numerical geometry of non-rigid shapes Multidimensional scaling

Convergence example

M. M. Bronstein et al., NLAA, 2006

Order of magnitude speedup, especially pronounced for large

Time (sec)

Str

ess

Convergence of SMACOF and MG MDS (N=2145)

29Numerical geometry of non-rigid shapes Multidimensional scaling

How to choose the embedding space?

A generic, non-Euclidean embedding space

Must result in small embedding error (good representation)

Convenient representation of points in (local or preferably global

parametrization)

Simple (preferably analytic) expression for distances

The isometry group is simple (few degrees of freedom)

30Numerical geometry of non-rigid shapes Multidimensional scaling

Possible choices

Schwartz et al. 1989:

Elad & Kimmel 2001:

Elad & Kimmel 2002:

BBK 2005:

Walter & Ritter 2002:

Euclidean

Spherical

Hyperbolic

Problem: embedding error can be reduced, but not made zero!

31Numerical geometry of non-rigid shapes Multidimensional scaling

Generalized MDS

Embedding space = triangular mesh

Generalized stress

Generalized MDS (GMDS) problem

where is the image on triangular mesh

A. M. Bronstein et al., PNAS, 2006

32Numerical geometry of non-rigid shapes Multidimensional scaling

Difference 1: the distances have no analytic expression

Consequence 1: geodesic distance interpolation

Main differences

A. M. Bronstein et al., SIAM, 2006

Difference 2: points represented in local barycentric

coordinates

Consequence 2: optimization with a modified line search (unfolding)

33Numerical geometry of non-rigid shapes Multidimensional scaling

Distance interpolation

How to approximate the distances between

points ?

Precompute the pair-wise distances between all mesh

vertices using FMM

Find triangles and enclosing

Interpolate from known

distances

Interpolate from

A. M. Bronstein et al., SIAM, 2006

34Numerical geometry of non-rigid shapes Multidimensional scaling

Modified line search: unfolding

Optimization on triangular mesh requires displacing a point along a

ray

(line search)

Line search in barycentric coordinates requires unfolding

Result: polylinear path

A. M. Bronstein et al., SIAM, 2006

35Numerical geometry of non-rigid shapes Multidimensional scaling

Geodesic distances are intrinsic descriptors of non-rigid shapes

invariant to isometric deformations

MDS is an efficient method for representing and comparing intrinsic

invariants

Multiresolution and multigrid methods can yield a significant

convergence speedup and reduce the risk on local convergence

Generalized MDS allows avoiding the embedding error by embedding

one surface into another

Conclusions so far