1Numerical geometry of non-rigid shapes Lecture I – Introduction

Numerical geometryof shapes

Lecture I – Introduction

non-rigid

Michael Bronstein

2Numerical geometry of non-rigid shapes Lecture I – Introduction

Welcome to non-rigid world

3Numerical geometry of non-rigid shapes Lecture I – Introduction

Non-rigid shapes everywhere

Articulatedshapes

Volumetricmedical data

Computer graphics models

Two-dimensional shapes

4Numerical geometry of non-rigid shapes Lecture I – Introduction

Auguste Rodin

Non-rigid shapes in art

5Numerical geometry of non-rigid shapes Lecture I – Introduction

Rock

Paper

Scissors

じゃんけんぽん

Jan-ken-pon (Rock-paper-scissors )

6Numerical geometry of non-rigid shapes Lecture I – Introduction

Hands

Rock

Paper

Scissors

じゃんけんぽん

7Numerical geometry of non-rigid shapes Lecture I – Introduction

Invariant similarity

SIMILARITY

TRANSFORMATION

8Numerical geometry of non-rigid shapes Lecture I – Introduction

Deformation-invariant similarity

Define a class of deformations

Find properties of the shape which are invariant under the class of

deformations and discriminative (uniquely describe the shape)

Define a shape distance based on these properties

9Numerical geometry of non-rigid shapes Lecture I – Introduction

Rigid Elastic

TopologicalInelastic

Invariance

10Numerical geometry of non-rigid shapes Lecture I – Introduction

Invariant correspondence

CORRESPONDENCE

TRANSFORMATION

11Numerical geometry of non-rigid shapes Lecture I – Introduction

Analysis and synthesis

Elephant image: courtesy M. Kilian and H. Pottmann

SYNTHESISANALYSIS

12Numerical geometry of non-rigid shapes Lecture I – Introduction

Landscape

“HORSE”

Image processing Geometry processing

Pattern recognition

Computervision

Computergraphics2D world 3D world

13Numerical geometry of non-rigid shapes Lecture I – Introduction

In a nutshell

Analysis and synthesis of non-rigid shapes

Archetype problems: shape similarity and correspondence

Metric geometry as a common denominator

Tools from geometry, algebra, optimization, numerical analysis,

statistics,

and multidimensional data analysis

Practical numerical methods

Applications in computer vision, pattern recognition, computer

graphics,

and geometry processing

14Numerical geometry of non-rigid shapes Lecture I – Introduction

Additional reading

Excerpts from the book

On paperOnline

tosca.cs.technion.ac.il/book

ProblemsSolutions

Lecture slides

Software

Links

Tutorials

Data

Springer, October 2008

15Numerical geometry of non-rigid shapes Lecture I – Introduction

Raffaello Santi, School of Athens, Vatican

16Numerical geometry of non-rigid shapes Lecture I – Introduction

Metric model

Shape = metric space , where is a metric

Shape similarity = similarity of metric spaces

17Numerical geometry of non-rigid shapes Lecture I – Introduction

Isometries

Two metric spaces and are equivalent if there exists a

distance-preserving map (isometry) satisfying

Self-isometries of form an isometry group

Such and are called isometric, denoted

18Numerical geometry of non-rigid shapes Lecture I – Introduction

Euclidean metric

Shape is a subset of the Euclidean embedding space

Restricted Euclidean metric

for all

19Numerical geometry of non-rigid shapes Lecture I – Introduction

Euclidean isometries

Isometry group in the Euclidean space consists of rigid

motions

Two shapes differing by a Euclidean isometry are congruent

Rotation Translation Reflection

20Numerical geometry of non-rigid shapes Lecture I – Introduction

Geodesic metric

Given a path on , define its length

The length can be induced by the Euclidean metric

Geodesic (intrinsic) metric

Geodesic = minimum-length path

Technical condition: is a smooth submanifold of

21Numerical geometry of non-rigid shapes Lecture I – Introduction

Riemannian view

Define a Euclidean tangent space at every point

Define an inner product (Riemannian metric) on the tangent space

Measure the length of a curve using the Riemannian metric

Bernhard Riemann(1826-1866)

22Numerical geometry of non-rigid shapes Lecture I – Introduction

Nash embedding theorem

John Forbes Nash

Embedding theorem (1956): Any smooth

Riemannian manifold can be realized as

an embedded surface in Euclidean space

of sufficiently high yet finite dimension

Technical conditions:

Manifold is

For -dimensional manifold,

embedding

space dimension isPractically: intrinsic and extrinsic views are

equivalent!Nash, 1956

23Numerical geometry of non-rigid shapes Lecture I – Introduction

Uniqueness of the embedding

Nash theorem guarantees existence but not uniqueness of

embedding

Embedding is clearly defined up to a congruence (Euclidean

isometry)

IN OTHER WORDS:

Do isometric yet incongruent shapes exist?

Are there cases of non-trivial non-uniqueness?

Riemannian

manifold

Embedded surface

24Numerical geometry of non-rigid shapes Lecture I – Introduction

Bending

Shapes with incongruent isometries are called bendable

Plane is the simplest example of a bendable surface

Shapes that do not have incongruent isometries are called rigid

Extrinsic geometry of a rigid shape is fully determined by the

intrinsic one

25Numerical geometry of non-rigid shapes Lecture I – Introduction

Rigidity conjecture

Leonhard Euler(1707-1783)

In practical applications shapes

are represented as polyhedra

(triangular meshes), so…

If the faces of a polyhedron were made of

metal plates and the polyhedron edges

were replaced by hinges, the polyhedron

would be rigid.

Do non-rigid shapes really exist?

26Numerical geometry of non-rigid shapes Lecture I – Introduction

Rigidity conjecture timeline

Euler’s Rigidity Conjecture: every polyhedron is rigid1766

1813

1927

1974

1977

Cauchy: every convex polyhedron is rigid

Connelly finally disproves Euler’s conjecture

Cohn-Vossen: all surfaces with positive Gaussian

curvature are rigid

Gluck: almost all simply connected surfaces are rigid

27Numerical geometry of non-rigid shapes Lecture I – Introduction

Connelly sphere

Isocahedron

Rigid polyhedron

Connelly sphere

Non-rigid polyhedron

Connelly, 1978

28Numerical geometry of non-rigid shapes Lecture I – Introduction

“Almost rigidity”

Most of the shapes (especially, polyhedra) are rigid

This may give the impression that the world is more rigid than non-rigid

This is true if isometry is considered in the strict sense:

if exists such that

Many objects have some elasticity and therefore can bend almost

isometrically

No known results about “almost rigidity” of shapes

29Numerical geometry of non-rigid shapes Lecture I – Introduction

Rock-paper-scissors again

INTRINSICALLY

SIMILAR

EXTRINSICALLY

SIMILAR

Invariant to

inelastic deformations

Invariant to

rigid motions

30Numerical geometry of non-rigid shapes Lecture I – Introduction

Extrinsic vs. intrinsic similarity

INTRINSIC SIMILARITY

isometry w.r.t.

geodesic metric

EXTRINSIC SIMILARITY

isometry w.r.t.

Euclidean metric

31Numerical geometry of non-rigid shapes Lecture I – Introduction

Extrinsic vs. intrinsic similarity

RIGID

MOTION

EXTRINSIC SIMILARITY

= CONGRUENCE

For rigid shapes, intrinsic similarity = extrinsic similarity

(since all the isometries are congruences)

32Numerical geometry of non-rigid shapes Lecture I – Introduction

Extrinsic similarity

Given two shapes and , find the degree of their incongruence

Compare and as subsets of the Euclidean space

Invariance to Euclidean isometry where

Euclidean isometries = rotation, translation, (reflection):

is a rotation matrix,

is a translation vector

33Numerical geometry of non-rigid shapes Lecture I – Introduction

Given two shapes and , find the best rigid motion

bringing as close as possible to :

is some shape-to-shape distance

Minimum = extrinsic dissimilarity of and

Minimizer = best rigid alignment between and

ICP is a family of algorithms differing in

The choice of the shape-to-shape distance

The choice of the numerical minimization algorithm

Iterative closest point (ICP) algorithms

34Numerical geometry of non-rigid shapes Lecture I – Introduction

Shape-to-shape distance

Hausdorff distance: distance between subsets of a metric space

where ,

Non-symmetric version of Hausdorff distance

where is closest-point correspondence

35Numerical geometry of non-rigid shapes Lecture I – Introduction

Iterative closest point algorithm

Initialize

Find the closest point correspondence

Minimize the misalignment between corresponding points

Update

Iterate until convergence…Chen & Medioni, 1991; Besl & McKay, 1992

36Numerical geometry of non-rigid shapes Lecture I – Introduction

Iterative closest point algorithm

Closest point correspondenceOptimal alignment

37Numerical geometry of non-rigid shapes Lecture I – Introduction

And now, intrinsic similarity…

INTRINSIC SIMILARITYEXTRINSIC SIMILARITY

Part of the same metric space Two different metric spaces

SOLUTION: Find a representation of and

in a common metric space

38Numerical geometry of non-rigid shapes Lecture I – Introduction

Canonical forms

Isometric embedding

Elad & Kimmel, 2003

39Numerical geometry of non-rigid shapes Lecture I – Introduction

Canonical form distance

Compute canonical formsEXTRINSIC SIMILARITY OF CANONICAL FORMS

INTRINSIC SIMILARITY

= INTRINSIC SIMILARITY

Elad & Kimmel, 2003

40Numerical geometry of non-rigid shapes Lecture I – Introduction

Examples of canonical forms

Elad & Kimmel, 2003

41Numerical geometry of non-rigid shapes Lecture I – Introduction

Expression-invariant face recognition

Images: Leonid Larionov

42Numerical geometry of non-rigid shapes Lecture I – Introduction

Is geometry sensitive to expressions?

x

x’

y

y’

Euclidean distances

43Numerical geometry of non-rigid shapes Lecture I – Introduction

Is geometry sensitive to expressions?

x

x’

y

y’

Geodesic distances

44Numerical geometry of non-rigid shapes Lecture I – Introduction

Extrinsic vs. intrinsic

Distance distortion distribution

Extrinsic geometry sensitive to expressions

Intrinsic geometry insensitive to expressionsBronstein, Bronstein & Kimmel, 2003

45Numerical geometry of non-rigid shapes Lecture I – Introduction

Isometric model of expressions

Expressions are approximately inelastic deformations of the facial

surface

Identity = intrinsic geometry

Expression = extrinsic geometryBronstein, Bronstein & Kimmel, 2003

46Numerical geometry of non-rigid shapes Lecture I – Introduction

Canonical forms of faces

Bronstein, Bronstein & Kimmel, 2005

47Numerical geometry of non-rigid shapes Lecture I – Introduction

Telling identical twins apart

Extrinsic similarity Intrinsic similarity

MichaelAlexBronstein, Bronstein & Kimmel, 2005

48Numerical geometry of non-rigid shapes Lecture I – Introduction

Telling identical twins apart

MichaelAlex

49Numerical geometry of non-rigid shapes Lecture I – Introduction

50Numerical geometry of non-rigid shapes Lecture I – Introduction

Summary

Shape = metric space

Shape similarity = distance between metric spaces

Invariance = isometry

Definition of the metric determines the class of transformations to

which the similarity is invariant

Extrinsic similarity = congruence (Euclidean metric) computed using

ICP

Intrinsic similarity = congruence of canonical forms obtained by

isometric embedding

51Numerical geometry of non-rigid shapes Lecture I – Introduction

References

Metric geometry

Burago, Burago, Ivanov, A course on metric geometry, AMS (2001)

Rigidity

S. E. Cohn-Vossen, Nonrigid closed surfaces, Annals of Math. (1929)

R. Connelly, The rigidity of polyhedral surfaces, Math. Magazine (1979)

Iterative closest point algorithms

Y. Chen and G. Medioni, Object modeling by registration of multiple range images, Proc. Robotics and Automation (1991)

P. J. Besl and N. D. McKay, A method for registration of 3D shapes, Trans. PAMI(1992)

52Numerical geometry of non-rigid shapes Lecture I – Introduction

References

S. Rusinkiewicz and M. Levoy, Efficient variants of the ICP algorithm, Proc. 3DDigital Imaging and Modeling (2001)

N. Gelfand, N. J. Mitra, L. Guibas, and H. Pottmann, Robust global registration,Proc. SGP (2005)

H. Li and R. Hartley, The 3D-3D registration problem revisited, Proc. ICCV (2007)

N. J. Mitra, N. Gelfand, H. Pottmann, and L. Guibas, Registration of point clouddata from a geometric optimization perspective, Proc. SGP (2004)

Canonical forms

A. Elad and R. Kimmel, On bending invariant signatures for surfaces, Trans. PAMI (2003)

53Numerical geometry of non-rigid shapes Lecture I – Introduction

References

Face recognition

A. M. Bronstein, M. M. Bronstein, R. Kimmel, Expression-invariant 3D face recognition, Proc. AVBPA (2003)

A. M. Bronstein, M. M. Bronstein, R. Kimmel, Three-dimensional face recognition, IJCV (2005)

A. M. Bronstein, M. M. Bronstein, R. Kimmel, Expression-invariant representationof faces, Trans. Image Processing (2007)