rock, paper, and scissors: extrinsic vs. intrinsic ...ron/papers/brobrokimiccv07.pdfrock, paper, and...

6
Rock, Paper, and Scissors: extrinsic vs. intrinsic similarity of non-rigid shapes Alexander M. Bronstein Dept. of Computer Science Technion, Haifa 32000, Israel [email protected] Michael M. Bronstein Dept. of Computer Science Technion, Haifa 32000, Israel [email protected] Ron Kimmel Dept. of Computer Science Technion, Haifa 32000, Israel [email protected] Abstract This paper explores similarity criteria between non-rigid shapes. Broadly speaking, such criteria are divided into in- trinsic and extrinsic, the first referring to the metric struc- ture of the objects and the latter to the geometry of the shapes in the Euclidean space. Both criteria have their ad- vantages and disadvantages; extrinsic similarity is sensitive to non-rigid deformations of the shapes, while intrinsic sim- ilarity is sensitive to topological noise. Here, we present an approach unifying both criteria in a single distance. Nu- merical results demonstrate the robustness of our approach in cases where using only extrinsic or intrinsic criteria fail. 1. Introduction Most of us are familiar with the childhood Rock, Paper, Scissors game, in which the players bend their fingers in different ways to make the hand resemble one of the three objects (rock, represented by a closed fist; paper – by an open palm; or scissors – by the extended index and middle fingers). In the context of pattern recognition, this example demonstrates the difficulty of defining the similarity of non- rigid shapes: one may recognize the hand shapes as objects they intend to imitate, while others may say that all these “objects” are actually just deformations of the same hand. Using geometric terminology, the first similarity criterion is extrinsic, i.e., relates to the way the shape is embedded in the ambient space. The second criterion, on the other hand, is related to the intrinsic properties of the shape, described by its metric structure. The problem of shape similarity is traditionally con- sidered in computer vision, pattern recognition, and com- putational geometry literature, either implicitly or explic- itly, from the extrinsic point of view (see, for example, [1, 11, 9]). A classical result for rigid object matching is the iterative closest point (ICP) method, introduced by Chen and Medioni [6] and Besl and McKay [2]. ICP methods try to find a Euclidean transformation between two shapes, minimizing an extrinsic distance between them, usually a variant of the Hausdorff distance. In some sense, one can think of ICP as finding the best possible rigid alignment be- tween the shapes. Intrinsic similarity was explored in the paper of Elad and Kimmel [7], who proposed a non-rigid shape recognition method based on Euclidean embeddings as a generaliza- tion of the previous work of Schwartz et al. [15]. The key idea is to map the metric structure of the shapes to a low- dimensional Euclidean space and compare the resulting im- ages (called canonical forms) in this space. The canonical forms are computed using multidimensional scaling (MDS) [3]. One of the main disadvantages of this method is that the canonical forms can represent the intrinsic geometry of the shapes only approximately, as it is generally impos- sible to find an isometry between a non-flat surface and a flat Euclidean space. emoli and Sapiro [13] showed how the representation error can be theoretically avoided by using the Gromov-Hausdorff distance [8]. The Gromov- Hausdorff distance is invariant to isometries, which makes it a useful tool for intrinsic shape comparison. M´ emoli and Sapiro showed an approximation of the Gromov-Hausdorff distance, related to the latter by a probabilistic bound. In a follow-up paper, Bronstein et al. [4] proposed a method for the computation of the Gromov-Hausdorff distance based on a numerical core similar to MDS, referred to as general- ized MDS (GMDS). The choice of whether to use intrinsic or extrinsic sim- ilarity depends significantly on the application. The draw- back of extrinsic similarity is its sensitivity to non-rigid de- formations. Using our example, a gesture of the hand can be extrinsically more similar to a rock or scissors rather than another hand. This make extrinsic criteria unsuitable for the analysis of non-rigid objects with a high degree of flexibil- ity. The intrinsic criterion, on the other hand, is insensi- tive to deformations which can be approximated by isome- tries, since isometries preserve the metric structure of the shape. However, intrinsic similarity is sensitive to topologi- cal noise and can be problematic in cases where the objects are partially missing (for example, connecting the fingers of

Upload: others

Post on 22-Jun-2020

16 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Rock, Paper, and Scissors: extrinsic vs. intrinsic ...ron/PAPERS/BroBroKimICCV07.pdfRock, Paper, and Scissors: extrinsic vs. intrinsic similarity of non-rigid shapes Alexander M. Bronstein

Rock, Paper, and Scissors: extrinsic vs. intrinsic similarity of non-rigid shapes

Alexander M. BronsteinDept. of Computer Science

Technion, Haifa 32000, [email protected]

Michael M. BronsteinDept. of Computer Science

Technion, Haifa 32000, [email protected]

Ron KimmelDept. of Computer Science

Technion, Haifa 32000, [email protected]

Abstract

This paper explores similarity criteria between non-rigidshapes. Broadly speaking, such criteria are divided into in-trinsic and extrinsic, the first referring to the metric struc-ture of the objects and the latter to the geometry of theshapes in the Euclidean space. Both criteria have their ad-vantages and disadvantages; extrinsic similarity is sensitiveto non-rigid deformations of the shapes, while intrinsic sim-ilarity is sensitive to topological noise. Here, we present anapproach unifying both criteria in a single distance. Nu-merical results demonstrate the robustness of our approachin cases where using only extrinsic or intrinsic criteria fail.

1. IntroductionMost of us are familiar with the childhood Rock, Paper,

Scissors game, in which the players bend their fingers indifferent ways to make the hand resemble one of the threeobjects (rock, represented by a closed fist; paper – by anopen palm; or scissors – by the extended index and middlefingers). In the context of pattern recognition, this exampledemonstrates the difficulty of defining the similarity of non-rigid shapes: one may recognize the hand shapes as objectsthey intend to imitate, while others may say that all these“objects” are actually just deformations of the same hand.Using geometric terminology, the first similarity criterion isextrinsic, i.e., relates to the way the shape is embedded inthe ambient space. The second criterion, on the other hand,is related to the intrinsic properties of the shape, describedby its metric structure.

The problem of shape similarity is traditionally con-sidered in computer vision, pattern recognition, and com-putational geometry literature, either implicitly or explic-itly, from the extrinsic point of view (see, for example,[1, 11, 9]). A classical result for rigid object matching is theiterative closest point (ICP) method, introduced by Chenand Medioni [6] and Besl and McKay [2]. ICP methodstry to find a Euclidean transformation between two shapes,

minimizing an extrinsic distance between them, usually avariant of the Hausdorff distance. In some sense, one canthink of ICP as finding the best possible rigid alignment be-tween the shapes.

Intrinsic similarity was explored in the paper of Elad andKimmel [7], who proposed a non-rigid shape recognitionmethod based on Euclidean embeddings as a generaliza-tion of the previous work of Schwartz et al. [15]. The keyidea is to map the metric structure of the shapes to a low-dimensional Euclidean space and compare the resulting im-ages (called canonical forms) in this space. The canonicalforms are computed using multidimensional scaling (MDS)[3]. One of the main disadvantages of this method is thatthe canonical forms can represent the intrinsic geometry ofthe shapes only approximately, as it is generally impos-sible to find an isometry between a non-flat surface anda flat Euclidean space. Memoli and Sapiro [13] showedhow the representation error can be theoretically avoidedby using the Gromov-Hausdorff distance [8]. The Gromov-Hausdorff distance is invariant to isometries, which makesit a useful tool for intrinsic shape comparison. Memoli andSapiro showed an approximation of the Gromov-Hausdorffdistance, related to the latter by a probabilistic bound. In afollow-up paper, Bronstein et al. [4] proposed a method forthe computation of the Gromov-Hausdorff distance basedon a numerical core similar to MDS, referred to as general-ized MDS (GMDS).

The choice of whether to use intrinsic or extrinsic sim-ilarity depends significantly on the application. The draw-back of extrinsic similarity is its sensitivity to non-rigid de-formations. Using our example, a gesture of the hand canbe extrinsically more similar to a rock or scissors rather thananother hand. This make extrinsic criteria unsuitable for theanalysis of non-rigid objects with a high degree of flexibil-ity. The intrinsic criterion, on the other hand, is insensi-tive to deformations which can be approximated by isome-tries, since isometries preserve the metric structure of theshape. However, intrinsic similarity is sensitive to topologi-cal noise and can be problematic in cases where the objectsare partially missing (for example, connecting the fingers of

Page 2: Rock, Paper, and Scissors: extrinsic vs. intrinsic ...ron/PAPERS/BroBroKimICCV07.pdfRock, Paper, and Scissors: extrinsic vs. intrinsic similarity of non-rigid shapes Alexander M. Bronstein

Figure 1. The difference between intrinsic and extrinsic similarity:right and middle shapes are extrinsically similar while being in-trinsically dissimilar (the two shapes have different topology, sincetwo fingers touch). Left and middle shapes are intrinsically similar(one can be obtained from the other by a near-isometric deforma-tion), but extrinsically dissimilar.

the hand results in a significantly different intrinsic geome-try, see Figure 1). Consequently, it appears that two seman-tically similar shapes can be substantially different both intheir extrinsic and intrinsic geometries.

In this paper, we propose an approach for computing thesimilarity between non-rigid shapes trying to take the ad-vantages of both intrinsic and extrinsic similarity criteria,while avoiding their shortcomings. The proposed similaritycriterion is essentially a tradeoff between the extrinsic andthe intrinsic criteria. As an illustration, one can think of fit-ting a rubber glove onto a hand. The extent to which therubber surface is stretched represents the intrinsic geometrydistortion. The fit quality, or in other words, how close theglove is to the hand surface, represents the extrinsic distancebetween the two objects.

The rest of this paper is organized as follows. In Sec-tion 2, we introduce the mathematical background and for-mulate the standard approaches to measuring intrinsic andextrinsic similarity. In Section 3, we present our approachfor computing the joint intrinsic-extrinsic similarity. In Sec-tion 4, we show the numerical framework for computingthis distance. Section 5 is dedicated to experimental results.Section 6 concludes the paper. Due to space limitations,derivations and technical details are omitted and will appearwith additional experimental validation in the extended ver-sion of this paper.

2. Mathematical foundations

We model a non-rigid shape as a pair (X, dX), where Xis a two-dimensional smooth compact connected and com-plete Riemannian surface (possibly with boundary) embed-ded into R3, and dX : X ×X → R is the geodesic metricmeasuring the lengths of the shortest paths on the manifold,induced by the Euclidean length structure. For brevity ofnotation, we will write simply X , implying (X, dX).

We broadly refer to properties described in terms of themetric dX as to the intrinsic geometry of X , and to proper-ties associated with the restriction of the Euclidean metricdR3 |X as the extrinsic geometry. From the intrinsic pointof view, two objects X and Y are similar if the metrics dX

and dY are sufficiently close to each other. Formally, we saythat X and Y are ε-isometric if there exists an ε-surjectivemap ϕ : X → Y (i.e., dY (y, ϕ(X)) ≤ ε for all y ∈ Y ),which has a distortion

dis ϕ = supx,x′∈X

|dX(x, x′)− dY (ϕ(x), ϕ(x′))| = ε.

Such a ϕ is called an ε-isometry. A 0-isometry is calledsimply an isometry, and two objects related by such a mapare termed isometric. Isometric shapes are indistinguish-able in terms of intrinsic geometry. An isometry from X toitself is called a self-isometry. The collection of all the self-isometries forms a group with the function composition op-erator and is referred to as the isometry group, denoted byIso(X). Self-isometries express the intrinsic symmetries ofan object.

Isometries are very different from ε-isometries. Partic-ularly, an isometry is always bi-Lipschitz continuous [5],which is not necessarily true for an ε-isometry. If we re-lax the requirement of ε-surjectivity by demanding that ϕhas only disϕ ≤ ε, we refer to such ϕ as an ε-isometricembedding.

2.1. Extrinsic similarity

The basic tool in extrinsic shape comparison is the Haus-dorff distance, measuring the distance between two sets ofpoints X and Y in R3,

dR3

H (X, Y ) = max{

supx∈X

dR3(x, Y ), supy∈Y

dR3(y, X)}

,

where dR3(y,X) = infx∈X ‖y − x‖2 denotes the distancebetween the set X and the point y. A non-symmetric ver-sion of the Hausdorff distance,

dR3

NH(X, Y ) = supx∈X

dR3(x, Y ), (1)

is often preferred since it allows for partial comparisonof shapes. The Hausdorff distance (both its symmetricand non-symmetric versions) regards the objects as sets ofpoints in R3, is completely extrinsic, and consequently, issensitive to non-rigid deformations of the shapes. More-over, it also depends on rigid transformations of shapes;for example, translating one shape with respect to anotherchanges the Hausdorff distance.

ICP-type methods try to get rid of this alignment ambi-guity by minimizing the Hausdorff distance between X and

Page 3: Rock, Paper, and Scissors: extrinsic vs. intrinsic ...ron/PAPERS/BroBroKimICCV07.pdfRock, Paper, and Scissors: extrinsic vs. intrinsic similarity of non-rigid shapes Alexander M. Bronstein

Y over all the isometries in R3 (i.e., rigid transformations,including rotations, translations and reflections),

dICP(X,Y ) = infi∈Iso(R3)

dR3

H (i(X), Y ).

Though resolving the rigid transformations ambiguity, ICPmethods are still sensitive to non-rigid deformations. Nu-merically, dICP is computed using local optimization meth-ods, liable to converge to a suboptimal solution (local min-imum).

2.2. Intrinsic similarity

Canonical form-type methods compute the intrinsic sim-ilarity by posing the problem of intrinsic geometries com-parison as the problem of extrinsic geometry comparison,which is relatively easy to compute. The approach consistsof two stages. First, an extrinsic representation of the intrin-sic geometry of the shapes (near-isometric embedding) insome common metric space (Z, dZ) is constructed by find-ing two maps ϕ : X → Z and ψ : Y → Z with minimumdistortions dis ϕ and dis ψ. The embedding, in a sense, al-lows to “undo” all the isometric deformations of the objects(though, some degree of ambiguity stemming from isome-tries in Z still remains). Typically, Z is selected as the Eu-clidean space. The second stage is extrinsic comparisonof the canonical forms ϕ(X) and ψ(Y ), using, for exam-ple, the ICP distance dICP(ϕ(X), ψ(Y )). Since achievinga true isometric embedding is usually impossible [12], thecanonical forms are only an approximate representation ofthe intrinsic geometry of the shapes.

The problem of inaccuracy introduced by the embeddinginto Z can be resolved if we do not assume a given embed-ding space, but instead, include Z as a variable into the op-timization. We can always find a sufficiently complicatedmetric space into which both X and Y can be embeddedisometrically, and compare the images using the Hausdorffdistance,

dGH(X, Y ) = infZ

ϕ:X→Zψ:Y→Z

dZH(ϕ(X), ψ(Y )),

(here ϕ and ψ are assumed to be isometric embeddings).Such an approach is referred to as Gromov-Hausdorff dis-tance [8]. For compact surfaces, dGH can be expressed interms of the distortion obtained by embedding one surfaceinto another,

dGH(X,Y ) =12

infϕ:X→Y

ψ:X→Y

max{disϕ, dis ψ, dis (ϕ,ψ)},

where

dis (ϕ,ψ) = supx∈X,y∈Y

|dX(x, ψ(y))− dY (y, ϕ(x))|.

The computation of the distortions can be performed us-ing GMDS, a procedure similar in its spirit to MDS, butnot limited to spaces with analytically expressed geodesicdistances. The Gromov-Hausdorff distance is a metric onthe quotient space of non-rigid shapes under the isometryrelation. Particularly, this implies that dGH(X,Y ) = 0if and only if X and Y are isometric. More generally, ifdGH(X,Y ) ≤ ε, then X and Y are 2ε-isometric and con-versely, if X and Y are ε-isometric, then dGH(X, Y ) ≤ 2ε[5].

3. Joint similarityLet us now return to our example of glove fitting. As-

sume that Y is the hand surface, and X is the glove we wishto fit. Our goal is to find such a deformation of X , denotedhereinafter by Z = f(X), that is the most similar to Y inthe extrinsic sense, yet, at the same time preserves its intrin-sic geometry as much as possible (i.e., intrinsically similarto X). In a generic setting, we assume that Y (probe) is ob-tained from X (model) by means of some deformation (notnecessarily isometric). We allow for the possibility that thetopology of Y is different from that of X due to the pres-ence of noise or as the result of the deformation (e.g, in theglove fitting example, the fingers of the hand can touch eachother).

In order to quantify the intrinsic and extrinsic similarity,we define generic extrinsic and intrinsic distances dE anddI. We require that dE(X,Y ) = 0 if X and Y are extrinsi-cally similar (i.e., X = Y ) and dI(X,Y ) = 0 if X and Yare intrinsically similar (i.e., dX = dY ). Since the deforma-tion f gives a one-to-one correspondence between X and Z,we can set ϕ = f and ψ = f−1 in the Gromov-Hausdorffdistance, obtaining

dI(X, Z) =12dis ϕ

=12

supx,x′∈X

|dX(x, x′)− dZ(f(x), f(x′))| (2)

as the intrinsic distance. Note that the correspondence be-tween the surfaces is now fixed and does not participateanymore in the minimization. dI(X, Z) defined this waymeasures the distortion in the intrinsic geometry of X in-troduced by the extrinsic deformation f . However, thegeodesic distances in Z have to be re-computed every timethe deformation changes.

Due to its sensitivty to noise, the L∞ formulation of dI

can be less practical when working with real shapes. Forthat reason, we prefer to use its L2 version,

dI(x,Z) =∫

X×X

(dX(x, x′)− dZ(f(x), f(x′)))2da(x)da(x′), (3)

Page 4: Rock, Paper, and Scissors: extrinsic vs. intrinsic ...ron/PAPERS/BroBroKimICCV07.pdfRock, Paper, and Scissors: extrinsic vs. intrinsic similarity of non-rigid shapes Alexander M. Bronstein

where da denotes the area element on X .As the extrinsic distance, we use an L2 version of the

non-symmetric Hausdorff distance,

dE(Y,Z) =∫

Z

‖z − y∗(z)‖2 da(z), (4)

where y∗(z) = arg miny∈Y ‖z − y‖2 is the closest point toz on Y (we write y∗, since in practice we work with com-pact objects, on which this minimum is always achieved).The set of closest points y∗(Z) has to be recomputed ev-ery time the deformation changes. The lack of symmetry ofdE(Y,Z) allows for partial matching.

Since it is usually impossible to say which of the two cri-teria is more important, we judge the similarity as a tradeoffbetween them,

dJ(X,Y ) = minZ

dI(X, Z) + λ dE(Z, Y ), (5)

which we refer to as the joint similarity criterion. The pa-rameter λ controls the relative significance of each crite-rion. This approach generalizes the similarity criteria basedpurely on extrinsic or intrinsic geometry. Setting λ ¿ 1penalizes the distortions in the intrinsic geometry, yieldinga generalization of ICP, since we now allow for non-rigidisometries and not only for the rigid ones. On the otherhand, setting λ À 1, the probe is forced to be attached tothe model surface, which boils down to a problem similarto GMDS.

4. Numerical frameworkFor practical computations, we work with discretized

shapes. The surface X is sampled at N points X ={x1, ..., xN} ⊆ X , constituting an r-covering (i.e., X =⋃N

n=1 BX(xn, r), where BX(x, r) is a metric ball of radiusr centered at x). The extrinsic coordinates of X are repre-sented as an N × 3 matrix X, whose ith row corresponds toa point xi ∈ R3. The discrete shape is represented as a tri-angular mesh; each triangle is a triplet of indices of verticesbelonging to it. Vertices connected by an edge (belonging tothe same triangle) are said to be adjacent; we denote by Ethe set of all adjacent pairs of vertices in X . The geodesicdistances on X are approximated using the Dijkstra algo-rithm, forming an N × N matrix D(X), whose elementsare dij(X) ≈ dX(xi, xj).

Assuming the deformed surface Z = f(X) maintainsthe connectivity of X , we can formulate the following min-imization problem with respect to the N×3 matrix Z of theextrinsic coordinates of Z:

dJ(X, Y ) = minZ

1N2

N∑

i,j=1

(dij(X)− dij(Z))2

N

N∑

i=1

d2R3(zi, Y ) (6)

where d(zi, Y ) denotes the Euclidean distance from thepoint zi to the discretized surface Y . We denote the abovecost function by σ(Z). The first term in it is the discretiza-tion of dI, whereas the second term is the discretization ofdE. In the following, we show how to compute these twoterms and their derivatives with respect to Z, required forthe minimization of (6).

4.1. Intrinsic distance computation

The main challenge in optimization involving the com-putation of the intrinsic distance term dI is the necessityto evaluate the geodesic distances on Z = f(X) andtheir derivatives with respect to the extrinsic geometry ofZ changing at each iteration of the minimization algorithm.We first define the matrix D(Z) of local distances, whoseelements are

dij(Z) ={ ‖zi − zj‖ : (i, j) ∈ E

0 : (i, j) /∈ E.(7)

Using the Dijkstra algorithm, we compute the set of theshortest paths between all pairs of points (i, j). For exam-ple, let Pij = {(i, i1), (i1, i2), ..., (in−1, in), (in, j)} ⊂ Ebe the shortest path between the points i and j. Its length isgiven by L(Pij) = di,i1 + di1,i2 + ... + din,j , which is alinear combination of the elements of D(Z). We can there-fore “complete” the missing entries in the matrix D(Z) bydefining the matrix of approximate global distances

D(Z) = I(D(Z))D(Z), (8)

where I is a sparse fourth order tensor, with the elementsIijkl = 1 if the edge (k, l) is contained in the shortest pathPij , and 0 otherwise. Note that I depends on the connectiv-ity E, which is assumed to be fixed, and the matrix of localdistances D, which, in turn, depends on Z.

In order to compute the derivative of D(Z) with respectto Z, we assume that a small perturbation dZ of Z does notchange the connectivity of the points on Z, and as the re-sults, the trajectory of the shortest paths between the pointson Z traverses the same edges. Thus, we may write

D(Z + dZ) = I(D(Z + dZ))D(Z + dZ)= I(D(Z))D(Z + dZ),

and compute the derivative of D(Z) as the derivative of thelinear form I(D(Z)). If I(D(Z)) 6= I(D(Z + dZ)), theassumption does not hold and the derivative of D usuallydoes not exist. Yet, the derivative of I(D(Z)) belongs to thesub-gradient set of D(Z) at the point Z. This is sufficientfor many minimization algorithms to work correctly [14].

The intrinsic distance can be written in terms of D(Z) as

Page 5: Rock, Paper, and Scissors: extrinsic vs. intrinsic ...ron/PAPERS/BroBroKimICCV07.pdfRock, Paper, and Scissors: extrinsic vs. intrinsic similarity of non-rigid shapes Alexander M. Bronstein

the Frobenius norm

dI(Z) =1

N2‖D(Z)− D(X)‖2F (9)

=1

N2tr((D(Z)− D(X))T(D(Z)− D(X))).

Its derivative with respect to X is given by

∂dI(Z)∂Z

=2

N2(D(Z)− D(X))T

∂D(Z)∂Z

, (10)

where

∂dij(Z)∂Z

=∑

k,l

Iijkl∂dkl(Z)

∂Z, (11)

and the elements of ∂dkl(Z)∂Z are given by

∂dkl

∂zmn

=1

dkl

zmk − zm

l : n = kzml − zm

k : n = l0 : n 6= k, l

(12)

for m = 1, 2, 3.

4.2. Extrinsic distance computation

The computation of the extrinsic distance is similar to theone used in ICP algorithms, where the main difficulty arisesfrom the need to re-compute the closest points each timethe extrinsic geometry of Z changes. The extrinsic distanceterm can be written as

dE(Z) =1N

tr ((Z−Y∗(Z))(Z−Y∗(Z))T) (13)

where Y∗(Z) denotes the N × 3 matrix, whose i-th rowis the closest point on Y corresponding to xi. The closestpoints are computed as a weighted average of the points onY closest to xi. The weights are selected in inverse propor-tion to the distance from xi.

In ICP algorithms, it is common to assume Y∗(Z +dZ) ≈ Y∗(Z). By fixing Y∗, dE(Z) becomes a simplequadratic function, and its derivative can be written as

∂dE(Z)∂Z

=2N

(Z−Y∗(Z))T. (14)

4.3. Iterative minimization algorithm

For Z in the neighborhood of some X, the cost functionthat needs to be minimized can be approximated as

σ(Z) ≈1

N2tr (D(Z)TD(Z)− 2D(X)TD(Z) + D(X)TD(X)) +

λ

Ntr (ZZT − 2Y∗(X)ZT + Y∗(X)TY∗(X)), (15)

where D(Z) ≈ I(D(X))D(Z). Like in ICP, after find-ing a new Z which decreases σ(Z), the closest points Y∗

and the operator I are updated. The iterative minimizationalgorithm can be summarized as follows:

Compute the closest points Y∗(Z).1

Compute the shortest paths between all pairs of points2

on Z and assemble I.Update Z by ∆Z such that Z + ∆Z sufficiently3

decreasing the cost function (15).If the change in Z is small, stop. Else, go to Step 1.4

The update of Z in Step 3 can be safeguarded by evaluatingthe true cost function (with I(Z + ∆Z) and Y∗(Z + ∆Z)instead of I(Z) and Y∗(Z)). In our implementation, nosafeguard was used, and the minimization in Step 3 wasdone using conjugate gradients.

The initialization of the algorithm can be done in severalways, the simplest of which is Z = X. This choice workswell when the extrinsic dissimilarity between X and Y isnot too large; for large dE(X, Y ), the algorithm will sufferfrom poor convergence similar to most ICP methods. An-other choice is to initialize Z by the corresponding pointson Y resulting from the solution of the GMDS problem.This choice is suitable for objects having sufficiently similarintrinsic geometries, making the intrinsic correspondencecomputed by GMDS meaningful.

5. Results

In order to demonstrate the proposed approach, we useda data set of four objects with four non-rigid deformations(Figure 2). While the first three deformations of each objectwere nearly isometric, the fourth one introduced topologi-cal changes modeled by welding points on the mesh. Threedistances were computed: a non-symmetric L2 approxima-tion of the Gromov-Hausdorff distance, a non-symmetricL2 approximation of the Hausdorff distance, and the jointdistance dJ. The Gromov-Hausdorff distance was com-puted using GMDS with 100 points embedded into a surfacerepresented with 1000 points. The Hausdorff distance wascomputed using ICP with meshes sampled at 1000 points.The joint distance was implemented in MATLAB and com-puted on meshes sampled at 100 points; its computation re-quired approximately one minute per pair of shapes.

Figure 2 visualizes the computed similarities. It appearsthat while the intrinsic similarity is nearly invariant to iso-metric deformations, it is sensitive to topological changes,which introduce significant distortions in the intrinsic ge-ometry. At the other end, the extrinsic geometry is insensi-tive to topological changes, yet sensitive to intrinsic defor-mation. The joint similarity criterion combines both advan-tages, and is insensitive to both isometries and topologicalchanges.

Page 6: Rock, Paper, and Scissors: extrinsic vs. intrinsic ...ron/PAPERS/BroBroKimICCV07.pdfRock, Paper, and Scissors: extrinsic vs. intrinsic similarity of non-rigid shapes Alexander M. Bronstein

Intrinsic similarity (dGH).

Extrinsic similarity (dICP).

Joint similarity (dJ).

Figure 2. Visualization of different types of similarity (distances in the plane represent the degree of similarity).

6. Conclusions

We presented a new approach for the computation ofnon-rigid shape similarity as a tradeoff between extrinsicand intrinsic similarity criteria, which can be thought of asa hybrid of ICP and GMDS. Our approach can be illustra-tively presented as deforming one shape in order to makeit the most similar to another from the extrinsic point ofview, while trying to preserve as much as possible its in-trinsic geometry. The joint intrinsic and extrinsic similarityappears to be advantageous over traditional purely extrinsicor intrinsic similarity criteria. While extrinsic similarity issensitive to strong non-rigid deformations of the shapes andintrinsic similarity is sensitive to topology changes resultingfrom noise or non-rigid deformations, our joint similaritycriterion allows to gracefully handle such problems. Exper-imental results demonstrate that it can be useful in situationswhere intrinsic and extrinsic similarities fail. In addition toshape analysis, our approach can be used for shape syn-thesis (e.g., morphing and animation problems), in a waysimilar to [10].

References[1] R. Basri, L. Costa, D. Geiger, and D. Jacobs. Determin-

ing the similarity of deformable shapes. Vision Research,38:2365–2385, 1998. 1

[2] P. J. Besl and N. D. McKay. A method for registration of 3Dshapes. IEEE Trans. PAMI, 14:239–256, 1992. 1

[3] I. Borg and P. Groenen. Modern multidimensional scaling -theory and applications. Springer-Verlag, Berlin HeidelbergNew York, 1997. 1

[4] A. M. Bronstein, M. M. Bronstein, and R. Kimmel. Gener-alized multidimensional scaling: a framework for isometry-

invariant partial surface matching. Proc. National Academyof Sciences, 103(5):1168–1172, January 2006. 1

[5] D. Burago, Y. Burago, and S. Ivanov. A course in metricgeometry, volume 33 of Graduate studies in mathematics.American Mathematical Society, 2001. 2, 3

[6] Y. Chen and G. Medioni. Object modeling by registrationof multiple range images. In Proc. IEEE Conference onRobotics and Automation, 1991. 1

[7] A. Elad and R. Kimmel. On bending invariant signatures forsurfaces. IEEE Trans. PAMI, 25(10):1285–1295, 2003. 1

[8] M. Gromov. Structures metriques pour les varietes rieman-niennes. Number 1 in Textes Mathematiques. 1981. 1, 3

[9] D. Jacobs, D. Weinshall, and Y. Gdalyahu. Class represen-tation and image retrieval with non-metric distances. IEEETrans. PAMI, 22:583–600, 2000. 1

[10] M. Kilian, N. J. Mitra, and H. Pottmann. Geometric model-ing in shape space. In Proc. SIGGRAPH, volume 26, 2007.6

[11] L. J. Latecki and R. Lakamper. Shape similarity measurebased on correspondence of visual parts. IEEE Trans. PAMI,22(10):1185–1190, 2000. 1

[12] N. Linial, E. London, and Y. Rabinovich. The geometry ofgraphs and some its algorithmic applications. Combinator-ica, 15:333–344, 1995. 3

[13] F. Memoli and G. Sapiro. A theoretical and computationalframework for isometry invariant recognition of point clouddata. Foundations of Computational Mathematics, 5:313–346, 2005. 1

[14] B. T. Polyak. Minimization of non-smooth function-als. USSR Computational Mathematics and MathematicalPhysics, 9:14–29, 1969. 4

[15] E. L. Schwartz, A. Shaw, and E. Wolfson. A numerical solu-tion to the generalized mapmaker’s problem: flattening non-convex polyhedral surfaces. IEEE Trans. PAMI, 11:1005–1008, 1989. 1