harmonic shape analysis: from fourier to wavelets

Harmonic Shape Analysis: From Fourier toWavelets

Ming Zhong

Department of Computer ScienceState University of New York at Stony Brook

Advisor: Professor Hong Qin

September 2012

Abstract

Harmonic analysis studies the representation of functions as the linear combinationof basic wave-like functions. It plays a fundamental role in the processing of time-seriessignals and images. Recent years have witnessed many efforts to adapt the Fourier andwavelet analysis to the domain of 3D shapes.

The manifold Fourier analysis relies on the eigenfunctions of the Laplace-Beltramioperator to analyze geometric shapes. Analogous to classic Fourier basis, the eigenfunc-tions of the Laplace-Beltrami operator form an orthonormal basis of a Hilbert space onthe manifold, and can be used to decompose functions defined on the manifold as gen-eralized Fourier series. The manifold Fourier analysis is essential in solving the heatequation and heat kernel on manifold. Since the eigenvalues and eigenfunctions of theBeltrami-Laplace operator are intrinsic and globally shape-aware, they are well suited forconstructing isometry-invariant shape descriptors and distance metrics, facilitating higher-level analysis tasks such as matching, registration and retrieval.

Wavelet transform allow signals to be decomposed into elementary forms at differentpositions and scales. It is advantageous to Fourier transform in that wavelet functions canbe simultaneously localized in both time/space and frequency domain. Various types ofmanifold wavelets have been proposed, based on subdivision, diffusion operator or spectraldecomposition.

In this report, we first explain the theoretical background of the manifold Fourier anal-ysis, focusing on its connection with signal processing and heat diffusion. We also givean overview of shape analysis applications based on spectral methods. We then provide asurvey of various types of manifold wavelet transform, particularly the spectral manifoldwavelet transform. Finally we present some of our on-going research on spectral shapeanalysis, including isometric shape matching, registration and retrieval.

i

Contents

1 Introduction 1

2 Theoretical Background 32.1 Classical Fourier Analysis . . . . . . . . . . . . . . . . . . . . . . . . . 32.2 Manifold Fourier Analysis . . . . . . . . . . . . . . . . . . . . . . . . . 42.3 Heat Diffusion on Manifold . . . . . . . . . . . . . . . . . . . . . . . . . 72.4 Discrete Settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102.5 Computation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

3 Fourier Shape Analysis and Processing 153.1 Spectral Geometry Processing . . . . . . . . . . . . . . . . . . . . . . . 153.2 Laplacian-based Shape Analysis . . . . . . . . . . . . . . . . . . . . . . 183.3 Diffusion-based Shape Analysis . . . . . . . . . . . . . . . . . . . . . . 20

3.3.1 Diffusion Kernel . . . . . . . . . . . . . . . . . . . . . . . . . . 203.3.2 Heat Kernel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 223.3.3 Learning Diffusion Kernel . . . . . . . . . . . . . . . . . . . . . 253.3.4 Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

4 Manifold Wavelets 314.1 Classis Wavelet Transform . . . . . . . . . . . . . . . . . . . . . . . . . 324.2 Subdivision Wavelets . . . . . . . . . . . . . . . . . . . . . . . . . . . . 334.3 Diffusion Wavelets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 354.4 Spectral Manifold Wavelet . . . . . . . . . . . . . . . . . . . . . . . . . 35

4.4.1 Spectral Manifold Wavelet Transform . . . . . . . . . . . . . . . 364.4.2 Scaling Functions . . . . . . . . . . . . . . . . . . . . . . . . . . 414.4.3 Reconstruction . . . . . . . . . . . . . . . . . . . . . . . . . . . 41

iii

5 Preliminary Work 435.1 High Order Shape Matching . . . . . . . . . . . . . . . . . . . . . . . . 43

5.1.1 Heat kernel tensor . . . . . . . . . . . . . . . . . . . . . . . . . 435.1.2 Matching hierarchy . . . . . . . . . . . . . . . . . . . . . . . . . 455.1.3 Experimental results . . . . . . . . . . . . . . . . . . . . . . . . 46

5.2 Hierarchical Shape Registration . . . . . . . . . . . . . . . . . . . . . . 485.2.1 Hierarchical registration . . . . . . . . . . . . . . . . . . . . . . 505.2.2 Experimental Results . . . . . . . . . . . . . . . . . . . . . . . . 53

5.3 Bag-of-Feature-Graphs Shape Retrieval . . . . . . . . . . . . . . . . . . 535.3.1 Shape-Google revisit . . . . . . . . . . . . . . . . . . . . . . . . 555.3.2 Bag of feature graphs . . . . . . . . . . . . . . . . . . . . . . . . 575.3.3 Shape retrieval . . . . . . . . . . . . . . . . . . . . . . . . . . . 585.3.4 Experimental results . . . . . . . . . . . . . . . . . . . . . . . . 59

6 Conclusion 61

iv

Chapter 1

Introduction

In real world, most things that we observe directly are represented as functions or signalsin space or time [53]. For example, a piece of music can be seen as signals over a period oftime; a color image is defined as vector-valued functions in a discrete 2d space. In contrastto space or time, frequency is not the natural domain that we perceive or describe a signal.But in many occasions, representing a function in the frequency domain may immenselybenefit certain types of analysis and processing that are difficult or cumbersome in spaceor time.

The area studying the frequency-domain representation of functions is often called theharmonic analysis. The name “harmonic” comes from the fact that functions or signalsare typically decomposed as the linear combinations of basic wave-like, periodic functions,whose frequencies are the integer multiple of a fundamental frequency, analogous to theharmonics in music theory. Originated from Fourier analysis, the concept of harmonicanalysis has evolved to include analysis employing non-periodic basic functions whosefrequencies are not fixed, such as wavelets analysis. Harmonic analysis is extensivelyused in various scientific and engineering fields, e.g. signal processing, image processing,quantum mechanics, seismic wave analysis, etc. Recent years have witnessed many effortsto adapt the Fourier and wavelet analysis to the domain of 3D shapes.

On a manifold, the set of eigenfunctions of the Laplace-Beltrami operator [67] formsa complete and orthonormal basis of the space of square-integrable functions defined onthe manifold. In another word, such functions can be decomposed as the linear combi-nations of these eigenfunctions. The eigenbasis of the manifold, also called the manifoldharmonic basis (MHB), bears significant similarity with the classical Fourier basis in Eu-clidean space, and are leveraged to define the generalized Fourier transform (also calledmanifold harmonic transform) and Fourier series on the manifold. Seeing the coordinatesof vertices as functions defined on a surface, we can conduct diverse signal processing op-

1

2

erations on the mesh geometry with the help of MHB, such as mesh filtering, compression,watermarking, etc.

The eigenfunctions and eigenvalues of the Laplace-Beltrami not only support mani-fold Fourier analysis on functions, but also carry significant information of the underly-ing shape. They are globally shape-aware, and by definition isometry-invariant, i.e., theeigenfunctions and eigenvalues keep unchanged when the shape undergoes nonelastic de-formations, such as rotation, translation, reflection and bending. Many shape analysisapplications, e.g., shape classification, vertex sequencing, symmetry detection, etc, relyon the eigenfunctions and eigenvalues of the Laplacian to reveal the intrinsic properties ofthe shape.

The manifold Fourier analysis is essential in solving the heat equation and heat ker-nel on manifold. The heat kernel is the fundamental solution to the heat equation. Ona manifold it is completely determined by the underlying shape and can be representedby the manifold harmonic basis. The heat kernel and its induced characteristics, suchas heat kernel signatures and heat kernel coordinates, are isometry-invariant, stable andsupport multi-scale analysis. They are well suited to be the building blocks of more com-plex shape descriptors and distance metrics, facilitating higher-level analysis tasks such asshape matching, registration and retrieval. The formulation of heat kernel can be extendedto general diffusion kernels, all can be expressed by the manifold harmonics.

In recent years, wavelet analysis has gained increasingly prominent role in harmonicanalysis. Wavelet transform allow signals to be decomposed into elementary forms atdifferent positions and scales. It is advantageous to Fourier transform in that wavelet func-tions can be simultaneously localized in both time/space and frequency domain, whileFourier functions are always globally defined in time/space. Extending wavelets to mani-fold domain is not a trivial task. Various types of manifold wavelets have been proposed,basing either on subdivision, diffusion operator or spectral decomposition.

In this report, we first explain the theoretical background of the manifold Fourier anal-ysis, concentrating on its connection with signal processing and heat diffusion, as well asits discrete setting on triangular meshes. We then give an overview of shape analysis andgeometry processing applications based on the spectrum of the Laplace-Beltrami opera-tor and diffusion geometry. Next, we provide a brief survey of various types of manifoldwavelet transforms, particularly the spectral manifold wavelet transform (SMWT). Finallywe present some of our on-going research on spectral shape analysis, including isometricshape matching, registration and retrieval.

Chapter 2

Theoretical Background

The most prominent tool for the so-called time-frequency (or space-frequency) analysis isthe Fourier analysis, the key idea of which is to represent functions as linear combinationsof a set of simple oscillating functions (e.g., sines and cosines), known as the Fourier basis.The collection of Fourier basis functions {φn;n ∈ Z} form a complete orthonormal set,or orthonormal basis [29], of the underlying function space F . F must be a Hilbert spacewith an inner product. The decomposition of the original signal is given by

f =∞∑

k=−∞

〈f, φk〉φk (2.0.1)

Normally, we refer to the process of Fourier transform as analysis, and the process ofinverse transform/reconstruction as synthesis. The term Fourier analysis covers the studyof both the transform and the inverse transform.

2.1 Classical Fourier AnalysisWe first consider the classical Fourier analysis where functions are defined on the realline and can be decomposed into trigonometric series. We denote by L2

T (R) the space ofperiodic square-integrable functions with period T > 0. Given f ∈ L2

T (R), the Fourierseries representation of f is

f(x) =∞∑

n=−∞

x(n)ei2πnxT =

∞∑n=−∞

f(n)eiωnx, (2.1.1)

where ωn = 2πn/T . The family of Fourier basis functions {φn(x) = eiωnx, n ∈ Z}form an orthonormal basis of L2

T (R). f(n) is called the Fourier coefficient (or Fourier

3

4 2.2. MANIFOLD FOURIER ANALYSIS

transform). It measures the amplitude of the frequency component corresponding to ωn ofthe function f , and can be computed as

f(n) = 〈f, φn〉 =1

T

∫ T

0

f(x)φn(x)dx =1

T

∫ T

0

f(x)e−iωnxdx (2.1.2)

Suppose that A(x) and B(x) are two square-integrable periodic functions of period Twith Fourier series representations

A(x) =∞∑

n=−∞

aneiωnx

and

B(x) =∞∑

n=−∞

bneiωnx

respectively. Then we have the following relation known as the Parseval’s Theorem∞∑

n=−∞

anbn = 〈A,B〉 =1

T

∫ T

0

A(x)B(x)dx (2.1.3)

From the perspective of partial differential equations, the k-th Fourier basis functionφk(x) satisfies the following Helmholtz equation in the real line:

− ∂2φk(x)

∂x2= ω2

kφk(x) (2.1.4)

In another word, φk(x) is the k-th eigenfunction of the differential operator −∂2/∂x2and is associated with the eigenvalue ω2

k. Here −∂2/∂x2 is the 1D Laplace operator andcan be easily extended to higher-dimensional Euclidean space. The Helmholtz equation isalso known as the Laplacian eigenvalue problem.

2.2 Manifold Fourier AnalysisWe want to generalize the Fourier transform and Fourier series to manifold domain. Givena manifoldMwith Riemannian metric g. The goal is to find a family of functions {φk(x)}that form a orthonormal and complete basis of the Hilbert space Ł2(M, }) : M → R.Then we can decompose any square-integrable real-valued function f defined onM intothe linear combination of {φk(x)}, just like the construction of traditional Fourier series:

f(x) =∞∑k=0

〈f(x), φk(x)〉φk(x), (2.2.1)

CHAPTER 2. THEORETICAL BACKGROUND 5

where the inner product is given by 〈f, g〉 =∫M f(x)g(x) dvol(x). Here dvol represents

the volume form of (M, g) [67]:

dvol =√

det gdx1 ∧ . . . ∧ dxn. (2.2.2)

In Sec. 2.1 we already know that, in 1D Euclidean space, the Fourier bases are theeigenfunctions of the Laplace operator ∂2

∂x2. In manifold space, the Laplace operator can

be generalized to the Laplace-Beltrami operator which acts on scalar functions definedon Riemannian manifold. We denote by ∆M the Laplace-Beltrami operator ofM. Justlike the Euclidean Laplacian, the Laplace-Beltrami operator is a second-order differentialoperator defined as the divergence of the gradient

∆f := div(gradf). (2.2.3)

In local coordinates, the formula for ∆M applied to a scalar function f is

∆Mf = − 1√|g|∂i(√|g|gij∂if), (2.2.4)

where gij are the components of the inverse of the metric tensor g, and |g| = |det(g)| isthe absolute value of the determinant of gij . See [67] for the proof.

The Laplace-Beltrami operator is a self-adjoint and semi-positive definite operator[67]. By self-adjoint we mean that 〈δu, v〉 = 〈u, δv〉 whenever u and v are sufficientlysmooth and vanish along the boundary ofM [83]. It follows that ∆M admits an orthonor-mal eigensystem. By solving the Dirichlet problem for the Laplacian:{

∆Mf(x) = −λf(x), x ∈Mf |∂M = 0

(2.2.5)

we obtain the eigenvalues {λk}∞k=0 and eigenfunctions {φk(x)}∞k=0 of the Laplace-Beltramioperator. According to the Spectral Theorem, the eigenvalues constitute a real divergingsequence

0 ≤ λ0 ≤ λ1 ≤ · · · ≤ +∞

and the eigenfunctions {φk}∞k=0 form a complete and orthonormal basis of the Hilbertspace L2(M) [42].

The eigenvalues {λk}∞k=0 are sometimes called the Laplace-Beltrami spectra [66]. Notethat whenM is a closed manifold, λ0 is always equal to zero, corresponding to a constanteigenfunction.

The eigenvalues are analogous to {ω2n} in Eq. (2.1.4) in classical Fourier transform

and their square roots can be deemed as the shape frequencies. The eigenfunctions, also

6 2.2. MANIFOLD FOURIER ANALYSIS

known as the manifold harmonics or shape harmonics [80], have periodic oscillations onmanifolds, behaving similarly to sine and cosine functions over the real line.

A function f(x) ∈ L2(M) can be expanded on the manifold harmonics as a series

f(x) =∞∑k=0

f(k)φk(x), (2.2.6)

wheref(k) = 〈f(x), φk(x)〉 =

∫M

f(x)φk(x) dvol(x). (2.2.7)

We may call f(k) the manifold harmonics transform, Fourier transform, or Fouriercoefficient of function f(x) [33]. Eq. (2.2.6) reconstructs function f(x) from its Fouriercoefficient and can be seen as the manifold Fourier series.

Similar to Eq. (2.1.3), the Parseval’s Theorem also holds for manifold Fourier trans-form. For any functions f(x), g(x) ∈ L2(M), the following relations holds

〈f(x), g(x)〉 =∞∑k=0

f(k)g(k). (2.2.8)

See [33] for the proof. This theorem implies the preservation of energy when transformingbetween the space and frequency domain.

We can extend the concept of Fourier transform and expansion to bivariate case. Sup-pose we have a bivariate kernel function θ : M ×M → R which corresponds to aself-adjoint operator Θ. The bivariate kernel can be expanded on the manifold Fourierbases

θ(x, y) =∞∑k=0

θ(k)φk(x)φk(y) (2.2.9)

whereθ(k) = 〈〈θ(x, y), φk(x)〉, φk(y)〉. (2.2.10)

θ(k) can be regarded as the Fourier transform of the bivariate kernel with a slight abuse oflanguage.

For example, the Laplace-Beltrami operator can be expanded as

∆M(x, y) =∞∑i=0

λkφk(x)φk(y). (2.2.11)

Hence, its Fourier transform is ∆M(k) = λk.


Aside from being employed for Fourier analysis on a shape, the eigenvalues and eigen-functions of the Laplace-Beltrami operator, together called the eigenstructure, are ex-tremely useful for constructing shape analysis tools that are both global and intrinsic.Because the Laplace-Beltrami operator is globally defined and is completely determinedby the metric tensor, which is itself an isometry invariant, the eigenvalues and eigenfunc-tions encode meaningful global information about the shape and they are invariant underisometric deformations up to a change in sign [70][77].

Two geometric objects are isometric if there exists a homeomorphism from one to theother that preserves geodesic distances, i.e. the arc length of a curve on one shape doesnot change when it is mapped to another shape. Such homeomorphism is called isometry.If a deformation on a shape preserves its isometry, we call it an isometric deformation.Intuitively, isometric deformation allow us to “bend” the shape, but stretching the shapewill break the isometry. Many real-life deformations can be deemed as approximatelyisometric, such as facial expressions and articulated human movements, hence designingshape representations that are isometry-invariant is of great importance for various shapeanalysis tasks.

Eigenfunctions and eigenvalues of the Laplace-Beltrami operator are very suitable forisometric shape analysis. On the one hand, geometric characteristics based on the eigen-structure are isometry-invariant. On the other hand, such characteristics carry consider-able global information and tends to be very stable comparing with local ones, such as thegeodesic distance. In another word, they are less sensitive to small perturbations or noiseson the shape.

2.3 Heat Diffusion on ManifoldIt is well known that classic Fourier series was first proposed by French engineer JosephFourier for the purpose of solving the heat equation on metal plate. Unsurprisingly, theLaplace-Beltrami operator and manifold harmonics play similar roles in solving the heatequation on manifolds.

The heat equation governs the heat diffusion process. It describes the distributionof heat (or temperature) in a given region over time. Consider a compact RiemannianmanifoldM possibly with boundary and function u(x, t) :M×R+ → R which representthe heat at point x and time t. The heat equation is written as

(∂

∂t−∆M)u(x, t) = 0, (2.3.1)

where ∆M denotes the Laplace-Beltrami operator of M. Usually, the heat equation isaccompanied by initial conditions and boundary conditions. The initial condition specifies

8 2.3. HEAT DIFFUSION ON MANIFOLD

the initial heat distribution u(x, 0) = u0(x). The boundary conditions restrains valuesand/or derivatives at the manifold boundary ∂M. Here we use the Dirichlet boundarycondition: u(x, t) = f(x), x ∈ M. To solve the heat equation, we first need to find thefundamental solution K(t, x, y) to the Dirichlet problem for heat equation:

∂K(t,x,y)∂t

= ∆MK(t, x, y)limt→0K(t, x, y) = δy(x)K(t, x, y) = 0, x ∈ ∂M or y ∈ ∂M

(2.3.2)

The fundamental solution is also called the heat kernel. Physically speaking,K(t, x, y)represents the heat distribution at time t and point x given that an initial unit of heat energyis placed at point y at time t = 0 and the boundary ofM has a fixed temperature 0. Theunit heat at y is represented by the Dirichlet distribution δy(·). Intuitively, the heat kernelmeasures the tendency of heat transfer from point y to x in time t. Another physicalinterpretation is that the heat kernel is the transition density function of Brownian motionon the manifold [34].

In Euclidean space Rd, the heat kernel has the form

K(t, x, y) =1

(4πt)d/2e− 4t|x−y|2 . (2.3.3)

On manifold, the heat kernel does not have an analytical form. Its definition is dependenton the underlying shape. Suppose the Laplace-Beltrami operator ∆M has eigendecompo-sition {λk, φk(x)}∞k=0. Set K(t, x, y) =

∑k ck(t)φk(x), we have

∂K(t, x, y)

∂t= ∆MK(t, x, y),

⇒∑k

dck(t)

dtφk(x) =

∑k

ck(t)∆Mφk(x) =∑k

−ck(t)λkφk(x).

Because of {φk(x)} form an orthonormal basis, the coefficients must be identical:

dck(t)

dt= −λkck(t),

⇒ck(t) = ck(0)e−λkt.

Incorporating the initial condition

K(0, x, y) = δy(x),


⇒∑k

ck(0)φk(x) =∑k

φk(x)φk(y),

⇒ck(0) = φk(y).

Hence the formal expression for the heat kernel is

K(t, x, y) =∞∑k=0

e−λktφk(x)φk(y). (2.3.4)

If we see K(t, x, y) as Kt,x(y), a function of y, its generalized Fourier coefficient isKt,x(k) = e−λktφk(x).

Suppose the initial heat distribution is u0(x), the heat distribution at time t can becomputed from the convolution of the heat kernel and u0(X)

u(t, x) = u0(y) ∗K(t, x, y) =

∫MK(x, y, t)u0(y) dvol(y). (2.3.5)

Applying the Parseval’s Theorem (Eq. 2.2.8), we obtain

u(t, x) =∞∑k=0

u0(k)Kt,x(k) (2.3.6)

=∞∑k=0

u0(k)e−λktφk(x) (2.3.7)

=∞∑k=0

〈u0, φk〉e−λktφk(x). (2.3.8)

As articulated in [77], the heat kernel has many nice properties:

• Symmetric. K(t, x, y) = K(t, y, x), x, y ∈M.

• Intrinsic. The heat kernel is completely determined by Riemannian structure ofthe manifold, so it only involves intrinsic properties of the manifold, i.e., the heatkernel is an isometric invariant. This invariance can also be deduced based on therepresentation of heat kernel. From Eq. 2.3.4) it is easy to see that the heat kernel isconstructed from the eigenstructure of the Laplace-Beltrami operator, which itself anisometric-invariant. This property makes the heat kernel a possible tool for isometricshape analysis.

10 2.4. DISCRETE SETTINGS

• Informative. The heat kernel contains all the information about the intrinsic geome-try of the shape and thus can determine shapes up to isometry.

• Multi-Scale. The introduction of time variable t makes the heat kernel a much moreversatile tool for shape analysis. When t is small, the function K(t, x, ·) is mainlydetermined by small neighborhood of x. With t becoming larger, K(t, x, ·) reflectslarger scale information of the shapeM from the point of view of x.

• Stable. Inherited from the eigenstructure of the Laplace-Beltrami operator, heatkernel shows great stability under perturbations. From the Brownian motion pointof view, the heat kernel is a weighted average over all paths between two points.Consequently, it is less sensitive to local perturbations.

Of all the above properties of the heat kernel, the multi-scale property is perhaps themost important one. By setting the time variable at a series of values, the heat kernelenables flexible representation and analysis of shapes.

2.4 Discrete SettingsIn practical applications, manifold shapes are usually represented as triangular meshes.For the purpose of shape analysis and processing based on mesh representations, we needdiscrete versions of the Laplace-Beltrami operator, manifold Fourier analysis and heatkernel, etc.

Consider a manifold M approximated by triangular mesh M , with vertex set V :={pi, i = 1, . . . , N}, edge set E, and face set F . |V | = N is the size of M . In addition, wedefine N(i) = {j|(pi, pj) ∈ E} and di = |N(i)|. N(i) denotes the index set of the 1-ringneighborhood of the vertex pi, and di is the valence of pi.

The discrete Laplace-Beltrami operator on mesh M is represented as a N ×N matrix∆ = (δij), defined by its linear action on vertex-based functions defined on V :

∆f(pi) =N−1∑j=0

δijf(pj). (2.4.1)

One simple approximation to the smooth Laplace-Beltrami operator is to define ∆f(pi)as the average difference between the function value at pi and its 1-ring neighborhood:

∆f(pi) =1

di

∑j∈N(i)

[f(pi)− f(pj)]. (2.4.2)


Figure 2.1: Angles in cotangent weights. [74]

The corresponding Laplacian matrix is

∆(i, j) =

di i = j−1 (pi, pj) ∈ E0 otherwise

(2.4.3)

∆ and its simple variations are called the combinatorial Laplacian or graph Laplacian,since they only considers the connectivity of the mesh graph. Graph Laplacian can wellapproximate the smooth Laplacian only when the mesh is uniformly distributed.

To faithfully approximate smooth Laplacian on arbitrary meshes, we need to take intoaccount geometric information such as the distances between neighboring vertices andthe angles between contiguous edges. Such defined Laplacian is called the geometricLaplacian.

Constructing discrete Laplace-Beltrami operator on general meshes is not a trivial task.In fact, it is impossible to make discrete Laplacian to simultaneously converge to smoothLaplacian and be symmetric on general meshes [70]. Many different versions of geometricdiscrete Laplacian have been proposed [62, 23, 84, 42, 80, 5]. One of the most popularscheme is developed by Meyer et al. [52]. It uses the cotangents of the two angles oppositeto an edge to weight the edge, and the area of the Voronoi cell size surrounding a vertex toweight the vertex. Its action on vertex-based function f on mesh M is

∆f(pi) =1

ai

∑j∈N(i)

wijf(pi)− f(pj)

di. (2.4.4)

12 2.4. DISCRETE SETTINGS

Here ai is the area of the Voronoi cells around vertex pi, and weights

wij :=cotαij + cot βij

2, (2.4.5)

where αij and βij denote the two angles opposite to the edge (pi, pj) (See Fig. 2.4).Let us define the area matrix A = diag(ai) and weight matrix W as

W (i, j) =

∑

k∈N(i)wik i = j

−wij (pi, pj) ∈ E0 otherwise

The geometric Laplacian matrix is then L = A−1W .Generally, such defined L is not symmetric. We rewrite the Laplace equation Lv = λv

as the generalized eigenvalue problem

Wv = λAv. (2.4.6)

Since W is symmetric and A is symmetric positive-definite, the generalized eigenvectorsv corresponding to different generalized eigenvalues λ are orthogonal, and all of the gen-eralized eigenvalues/eigenvectors are real. The orthogonality is with respect to theA-innerproduct

〈vi,vj〉 = vTi Avj = 0, i 6= j.

If the mesh vertices are evenly distributed, i.e., each vertex has the same Voronoi cellsize, we can make A = I by proper normalization. In this case, the A-inner productbecomes the standard dot product (I-inner product). This is unfortunately not valid formeshes whose vertices are not distributed uniformly over the surface area. To obtain sym-metric mesh Laplacian, we may use the system A−1/2WA−1/2y = λy which will givethe same eigenvalues [80]. The original eigenvectors can be retrieved by v = W−1/2y.However, this symmetrization is not preferable since solving the generalized eigenvalueproblem is more stable than inverting W [64].

Let {λi}N−1i=0 be the set of generalized eigenvalues of ∆M = A−1W , and {φi ∈ RN}their corresponding eigenvectors. A scalar function f defined on V can be expanded as thelinear combination of the eigenvectors:

f(p) =N−1∑k=0

〈f , φk〉φk(p), (2.4.7)

where the inner-product is the A-based scalar product

〈f ,g〉 = fTAg =N−1∑i=0

ai(i)fg(i). (2.4.8)


Correspondingly, the discrete heat kernel of M at time t > 0, denoted as KM(t, x, y),can be defined as the matrix:

KM(t, x, y) =N−1∑i=0

e−λktφk(x)φk(y). (2.4.9)

In practice, we usually only solve the first m eigenvectors of the Laplacian matrixinstead of a full decomposition.

2.5 ComputationSolving the eigenvalues λk and eigenvectors φk of the Laplacian matrix ∆ is a prerequisitefor real life applications of Fourier shape analysis. Ordinary eigendecomposition methodsrequireO(n3) time, which is prohibitive when the mesh has thousands of vertices or more.This exorbitant computation cost had traditionally limit the feasibility of spectral methods.

One way to reduce the time complexity is to partition the mesh into submeshes andindividually compute the eigendecomposition of their respective Laplacian matrices [38].This however break the global property of global Laplacian and bring difficulties in han-dling boundaries of submeshes.

In [42], Levy et al. articulated a band-by-band decomposition of Laplacian matrixbased on spectral shift. This method makes it possible to find the eigenvalues and eigen-vectors of large models with hundreds of thousands vertices.

Voxman et al. [81] proposed a multi-resolution approach to accelerate the computa-tion of heat kernel on very large meshes. The central idea is to construct a multi-resolutionstructure of the original mesh through mesh simplification and compute the heat kernel ofthe mesh in the coarsest level. Given the mapping between the vertices of meshes in neigh-boring levels, the heat kernel of the original mesh can be approximated by interpolatingthe heat kernel of the coarsest level.

14 2.5. COMPUTATION

Chapter 3

Fourier Shape Analysis and Processing

3.1 Spectral Geometry ProcessingAs shown in the previous chapter, the classic Fourier basis functions and the manifoldharmonic basis, which are the eigenfunctions of the Laplace-Beltrami operator, have manyproperties in common:

• They both form complete orthonormal bases of their respective function spaces. i.e.,they both can be used to decompose scalar functions defined on their domains.

• They both are globally defined and oscillate over their respective domains.

• Both the Fourier basis and the manifold harmonic basis are related to a discrete setof frequencies, representing the periods of oscillations.

Many applications take advantages of these striking similarities and employ the mani-fold harmonic transform (MHT) and its inverse for geometry processing in the frequencydomain, similar to the way Fourier transform is utilized in signal processing. The typicalpipeline for spectral geometry processing is

1. Compute the manifold harmonic basis (MHB) of the input triangular mesh by solv-ing the decomposition of the discrete mesh Laplacian.

2. Transform the geometry of the shape, i.e., the coordinates of the vertices, into fre-quency space by computing the MHT.

3. Apply certain frequency space filter to the Fourier coefficients and obtain trans-formed geometry.

15

16 3.1. SPECTRAL GEOMETRY PROCESSING

Figure 3.1: Effect of applying different MHT filters [80]. From left to right: original mesh;low-pass; low-enhance; band-exaggeration.

4. Transform back to the geometric space by computing the inverse MHT.

Assume mesh M has vertex set V = {pi = (xi, yi, zi), i = 1, . . . , N}. The meshLaplacian ∆M has the eigendecomposition {λk, φk}N−1k=0 :

∆Mφk = λkφk. (3.1.1)

The vertex coordinates, seen as functions defined over the mesh, can be represented withthe manifold harmonic basis {φk}:

pi =N−1∑k=0

pi(k)φk, (3.1.2)

where pi is the MHT of the coordinates vector pi:

pi(k) = 〈pi, φk〉. (3.1.3)

Mesh filtering. As we know, the manifold harmonics corresponding to small eigenval-ues represent low-frequency, large-scale information, i.e., the overall pose of the shape.In correspondence, the manifold harmonics associated with large eigenvalues carry high-frequency, small-scale details. Generally, we can enhance or smoothen (denoise) a partic-ular frequency component of the shape simply by applying appropriate frequency spacefilter F (λk):

pi =N−1∑k=0

F (λk)pi(k)φk. (3.1.4)

CHAPTER 3. FOURIER SHAPE ANALYSIS AND PROCESSING 17

Figure 3.2: Pose transfer: (C) is generated by copying the 5 first coefficients of (A) to (B)[42]

Fig. 3.1 demonstrates the effect of applying different isotropic frequency space filter toa mesh. In [15], Chuang developed an interactive mesh editing system that supportscurvature-guided anisotropic mesh filters, which affords more flexible and precise con-trol in geometry processing.

This mesh filtering technique can also be leveraged for detail-preserving editing anddetail transferring [75]. By replacing the first few Fourier coefficients of one model tothose of the other while keeping the rest coefficients intact, the first model will mimic theoverall shape of the second model but its details are kept largely unchanged ( see Fig. 3.2for an example). Alternatively, we may transfer the local geometry of one shape to anothershape, known as detail coating, by replacing the high-frequency coefficients.

Mesh compression. Karni and Gotsman [38] propose an approach to compress the ge-ometry of triangle meshes. The mesh vertex coordinates are projected into the spectralspace spanned by the Laplacian eigenvectors. Then the coefficients associated with thelarger eigenvalues are removed, which would correspond to high-frequency details. Thisis analogous to the JPEG image compression in which high-frequency Fourier componentsare truncated to save space.

Another approach to compress mesh geometry is through high-pass quantization [76].Directly performing quantization on the Cartesian coordinates of the vertices will producehigh-frequency errors that may significantly alter the surface normals, causing very un-natural appearances. Instead, we can quantize the differential coordinates, the Laplacian

18 3.2. LAPLACIAN-BASED SHAPE ANALYSIS

Fourier coefficients of the coordinates. The quantization errors are then mainly distributedin the low-frequency parts, which are much less conspicuous to human eyes.

Mesh watermarking. Ohbuchi et al. [56, 55] adopt the eigen-projection approach toinsert watermarks into triangular meshes by specially modifying at the last bits of thedifferential coordinates. Similar to high-frequency quantization, the added watermark doesnot produce noticeable discrepancies since the errors are well-spread over the entire modelafter reconstruction.

3.2 Laplacian-based Shape Analysis

As noted in the previous chapter, the eigenfunctions and eigenvectors of the Laplace-Beltrami operator are isometry-invariant and carry global information of the shape, andhence are good candidates for constructing tools for isometric shape analysis.

Shape classification. Reuter et al. directly use the eigenvalues of the Laplace-Beltramioperator, denoted as Laplace-spectra, as the signature of a shape [65, 66]. It can be usedto as a quality measure to ensure isometry, but its power to discriminate/classify differentshapes is relatively weak.

In [70], Rustamov defined the Global Point Signature (GPS) of a point p on surface asthe vector

GPS(p) = (1√λ1φ1(p),

1√λ2φ2(p),

1√λ3φ3(p), . . .).

In practice, we only use the first m eigenvalues and eigenfunctions to constitute the GPS.By definition, the dot product of the GPS of two points x, x′ is the Green’s function of

the Laplacian equation:

G(x, x′) =∞∑i=1

φi(x)φi(x′)

λi= GPS(x) ·GPS(x′). (3.2.1)

The GPS embedding of a shape is inherently isometry-invariant. For the purpose ofshape classification, we compute the histogram of pairwise distances in the GPS embed-ding space between the points that are uniformly sampled from the original surface. Theisometric similarity between shapes are then measured in terms of their GPS histograms.Fig. 3.3 shows how the histogram changes in the event of a small topological change.


Figure 3.3: Histogram of pair-wise distances distribution in GPS embedding space. [70]

Symmetry detection. In [60], Ovsjanikov et al. employ the aforementioned Global Position Signatureto detect and quantify the intrinsic symmetry of a shape rather than extrinsic, pose-dependentsymmetry. By computing the embedding using the isometry-invariant GPS, intrinsicsymmetries of the original shape are mapped into Euclidean extrinsic (rotational or re-flectional) symmetries in the signature space, in which point-to-point correspondences ofsymmetric points can be efficiently detected.

Mesh sequencing. The eigenvector corresponding to the first non-zero eigenvalue of themesh Laplacian, also known as the Fiedler vector [16], has a shape frequency correspond-ing to the whole size of the mesh. In another word, the values of the Fiedler vector oscillateonly one period over the entire shape (see Fig. 3.4). This inspired the concept of stream-ing meshes [36]. The idea is to progressively process very large meshes by streaming itscomponents, i.e., transferring blocks of vertices and faces, preferably neighboring to eachother, in an incremental manner. Naturally, it is highly desirable that the order in whichthe vertices and faces are transferred minimizes vertex separation, and the Fiedler vectorprovides a good heuristic for ordering vertices on the mesh.

20 3.3. DIFFUSION-BASED SHAPE ANALYSIS

Figure 3.4: The values of Fielder vector visualized.

3.3 Diffusion-based Shape Analysis

3.3.1 Diffusion KernelDiffusion geometry studies the the geometry of a manifold or a graph by considering adiffusion process on the manifold/graph. This idea is populated by Coifman et al. [20, 18,19] and has been adopted for various patter recognition applications, including deformableshape analysis [4, 10, 51, 50, 49].

The foundation of diffusion geometry is the kernel function k(x, y). The kernel rep-resents a sense of affinity between points in the domain as it describes the relationshipbetween each pair of points. The choice of how to construct the kernel should be guidedby the demand of specific applications. On domain X with a probability measure µ,k : X ×X → R should satisfies [18]:

• Symmetry: k(x, y) = k(y, x),

• Positive preserving: for all x, y ∈ X, k(x, y) ≥ 0,

• Positive-semidefiniteness: for all real-valued bounded functions f defined on X ,∫X

∫X

k(x, y)f(x)f(y)dµ(x)dµ(y) ≥ 0. (3.3.1)

If we additionally require


• Square integrability:∫X

∫Xk2(x, y)dµ(x)dµ(y) <∞,

• Conservation:∫Xk(x, y)dµ(y) = 1,

then the value of k(x, y) can be interpreted as a transition probability from x to y by stepof random walk on X [13]. k(x, y) defines the diffusion operator K

Kf =

∫X

k(x, y)f(y)dµ(y), (3.3.2)

which is both self-adjoint and compact. Hence, K admits a well-defined eigensystem{ai, ψi}∞i=0. According to the spectral theorem, the diffusion kernel can be expanded as

k(x, y) =∞∑i=0

aiφi(x)φi(y). (3.3.3)

We already know that on manifold M, the Laplace-Beltrami operator ∆M has theeigendecomposition {λi, φi}∞i=0, where the eigenfunctions {φi}∞i=0 form an complete andorthonormal basis of L2(M). Through a change of basis, we can represent any diffusionkernel K defined onM by the eigenbasis of the Laplace-Beltrami operator

k(x, y) =∞∑i=0

K(λi)φi(x)φi(y). (3.3.4)

From the perspective of signal processing, K(λ) is the transfer function of the lowpassfilter λ(i) = λi. k(x, y) can be interpreted as the point spread function originated from thepoint y.

Multi-scale

The diffusion kernel can be easily adapted to support multi-scale analysis. One approachis to use discrete powers of the diffusion operator {Kt}, with the scale parameter t. Thisdefines a scale space of diffusion kernels {kt(x, y)}t∈T ′ . Another approach is to define thetransfer function as K(tλi). The diffusion kernel then has the addition scale parameter

k(t, x, y) =∞∑i=0

K(tλi)φi(x)φi(y). (3.3.5)

In both formulations, the scale parameter can be interpreted as the diffusion time orthe length of random walk on the manifold.


Diffusion Distance

In [20] Coifman et al. proposed an isometric embedding Φ(x) = K(λi)φi(x)∞i=0, calleddiffusion map. Based on the diffusion map, we can define a metric between points on X

d2(x, y) = ‖Φ(x)− Φ(y)‖L2 =∞∑i=0

K2(λi)(φi(x)− φi(y))2. (3.3.6)

d2(x, y) is called the diffusion distance. Intuitively speaking, d2(x, y) measures the “con-nectivity” of point x and point y in by random walk of unit length. The connected x and yare, the smaller the distance is.

For example, the biharmonic distance [43] has the transfer function 1λi

. The associateddiffusion distance is

d2B(x, y) =∞∑i=0

1

λ2iφi(x)φi(y). (3.3.7)

Diffusion Kernel Signature

The diagonal of diffusion kernel k(x, x) represents the probability of a random walk thatstarts at point x ends at the same point in a unit length. Employing the multi-scale formu-lation k(t, x, y), we may define a signature for each point on the manifold as the vector

KS(x) = (k(t1, x, x), k(t2, x, x), . . . , k(tm, x, x)). (3.3.8)

This can be seen as sampling the t-based function k(t, x, x) at m discrete values. Bychoosing a representative set of time scales, the diffusion kernel signature captures bothlocal and global information in reference to point x.

3.3.2 Heat Kernel

The heat kernel is a special case of the diffusion kernel articulated in the previous section.We already know that on manifold the expression of the heat kernel is

HK(t, x, y) =∞∑i=0

e−λitφi(x)φi(y). (3.3.9)

Comparing with using eigenfunctions of the Laplacian directly, heat kernel has severaladvantages for isometric shape analysis


• The numerical solution to eigenfunctions are not stable. Neighboring eigenfunctionsmay flip, resulting in a change of sign. The values of heat kernel are aggregated fromhundreds of eigenfunctions and hence extremely stable.

• The heat kernel support multi-scale analysis.

• The heat kernel has a clear physics interpretation.

• The heat kernel are local objects, which ensures the local parameterization will notbe affected by faraway changes.

• When a manifold is approximated by a mesh, the manifold heat kernels convergerather nicely to the heat kernel [3]

In the language of diffusion kernel, the transfer function of the heat kernel is K(λi) =exp(−λit). Plugging in this transfer function, we obtain the heat kernel signature, firstproposed by Sun et al. [77], which is defined by the HKS function

HKS(t, x) =∞∑i=0

e−λitφi(x)2, (3.3.10)

and the heat diffusion distance

d2H =∞∑=0

e−2λit[φi(x)− φi(y)]2. (3.3.11)

The heat kernel can also be used for manifolds parameterization [37]. One parametriza-tion approach, known as the Heat Kernel Map [59], is by selecting an anchor point p0 onthe manifoldM and a set of time scales ti, i = 1, . . . ,m and mapping a point x ∈M as

x→ (k(t1, p0, x), k(t2, p0, x), . . . , k(tm, p0, x)). (3.3.12)

An alternative parameterization is by selecting a set of anchor points pi ∈M, i = 1, . . . , land a proper time scale t and mapping x as

x→ (k(t, p1, x), k(t, p2, x), . . . , k(t, pl, x)). (3.3.13)

These two approaches can be flexibly integrated to construct the heat kernel coordi-nates (HKC) [31] with multiple anchor points and heat scales.

In [77], Sun et al. proved that heat kernel signature contain almost as much infor-mation as the entire heat kernel and summarized the nice properties of the heat kernel


Figure 3.5: The scaled HKS corresponding to the four marked points from a syntheticmodel. [77]


signature: intrinsic; informative; multi-scale; stable. The article also proposed the scaledHKS scheme, which scales each k(t, x, x) by

∫M k(t, x, x) dvol(x) =

∑i exp(−λit) to

ensure the signature function values at different time scales contribute to the signaturevector equally. Fig. 3.5 demonstrates the strong discriminative power of HKS.

In [14], the heat kernel signature is extended to be scale-invariant. This facilitatesnon-rigid shape recognition of shapes with different scale.

Raviv et al. [63] generalized the idea of surface heat kernel signature to volumetricmodels. The Volumetric HKS is defined analogous to the normal HKS by considering theheat equation over the entire volume as a 3-submanifold and defining a Neumann bound-ary condition over the 2-manifold boundary of the shape. Volumetric HKS characterizestransformations up to a volume isometry, which represent the transformation for real 3Dobjects more faithfully than boundary isometry.

One limitation of heat kernel is its high computational cost. With the observationthat detailed geometry of the surface only influences the heat kernel of small time scales,and that the heat kernel provides a coarser view of the surface geometry as t increases,Vaxman et al. [81] devised a multi-resolution approach that approximates the heat kernelat large time t with a sparse matrix related to a lower-resolution version of the originalsurface. Their method achieved good approximation in a fraction time of the traditional,eigen-decomposition based algorithms. For small time t, the heat kernel can be reliablycomputed by the exponential of the Laplace operator: HKt = exp(−tL). In [73], Shi etal. proposed an efficient and robust algorithm for computing matrix exponentials.

3.3.3 Learning Diffusion KernelObserve the expression of HKS

HKS(t, x) =∞∑i=0

e−λitφi(x)2. (3.3.14)

It is easy to see that the transfer function e−λit is a lowpass filter. The larger a eigenvalue λiis, the smaller the corresponding eigenfunction φi contributes to the the HKS function. Inanother word, the HKS is highly dominated by low-frequency, i.e. large scale, informationof the shape, which is disadvantageous for applications like high-precision matching wherecapturing small scale information is important.

Aubry et al. [2] proposed a new shape descriptor, dubbed wave kernel signature(WKS), to address the drawback of HKS. The basic idea of WKS is to characterize apoint x ∈ M by the average probabilities of quantum particles of different energy levelsto be measured at x.


Consider a quantum particle with unknown location on surface M. The initial en-ergy probability distribution is f 2

E with expectation value E. By solving the Schrodingerequation

(i∆M +∂

∂tψ(x, t) = 0, (3.3.15)

we obtain the wave function

ψE(x, t) =∞∑k=0

eiλktfE(λk)φk(x). (3.3.16)

Assume |ψE(x, t)|2 is the probability density that at time t the particle is at point x.By integrating the time from 0 to infinity, we obtain the wave kernel signature functionWKS(x,E), which measure the average probability that a particle of energy E is at x:

WKS(x,E) = limT→∞

1

T

∫ T

0

|ψE(x, t)|2dt =∞∑k=0

f 2E(λk)φk(x)2. (3.3.17)

The transfer function of WKS is parameterized by the particle energy E rather thantime scale t as in HKS. The energy distribution is selected as

f 2E(λk) = (

∑i

e−(logE−log λi)

2

2σ2 )−1e−(logE−log λi)

2

2σ2 (3.3.18)

= CEe− (logE−log λi)

2

2σ2 . (3.3.19)

Given a discrete sequence of energy levels E = {e1, . . . , en}, the wave kernel signatureat point x is defined as the vector (WKS(x, e1), . . . ,WKS(x, en)).

Obviously, the transfer function f 2E function is a band-pass filter centered around the

frequency λc ≈ E. When E is small, the contribution of low-energy (low-frequency, largescale) eigenfunctions are enhanced. Whereas when E is large, the high-energy (high-frequency, small scale) eigenfunctions influence more to the WKS function.

By properly selecting the energy sequences E, the WKS vector can be discriminativefor both high-frequency and low-frequency signals.

Fig. 3.6 compares the discriminative power of HKS and WKS. In this example, theperformance of WKS is clearly better.

Inspired by the formulation of WKS, Bronstein [11] and Aflalo [1] suggest that, in-stead of purposely select or design a analytical transfer function K(λi), we may use thespecific data and problems to train the optimal transfer functions. Fig. 3.7 shows the ex-amples of transfer functions of different diffusion kernel signatures.


Figure 3.6: Similarity plot in the signature space. From blue to red, the similarity with thegiven reference point decreases (the distance in the signature space increases). [2]


Figure 3.7: The families of transfer functions of different diffusion kernel signatures. top:HKS; middle: WKS; bottom: trained signatures.


3.3.4 ApplicationsThe diffusion-based geometric characteristics have several advantages over the Laplacianeigenvalues and eigenfunctions in terms of deformable shape anlaysis

• Diffusion-based descriptors and distance metrics are often represented as the aggre-gate of eigenfunctions, which are much more stable and basically insusceptible tothe problem of eigenfunction switching which is common in numerical computation.

• The scaling parameter, which can be interpreted as the diffusion time, affords flexi-ble analysis of the shape in different levels of details.

• The diffusion-based characteristics are localized at vertices, contrary to the Lapla-cian eigenfunctions which are globally defined. This means, by keeping the diffu-sion time small, the diffusion-based descriptors inside the shape may change verylittle when the boundary changes. This allows wide application in analyzing partialdeformable shapes.

Shape Correspondence

Shape correspondence is an important problem in shape analysis and computer vision. Thegoal is to establish optimal mapping between points on two different shapes. The shapecorrespondence is often an important immediate step in applications such as shape interpo-lation, attribute transfer, surface completion, shape recognition and retrieval, etc. The wordcorrespondence sometimes can be used interchangeably with matching/registration/alignment.In this report, we refer to shape matching as sparse points correspondences, usually con-cerned about matching a small number of pre-selected feature points, and by shape regis-tration we mean dense correspondence across two shapes.

Comparing with rigid shapes, corresponding points on deformable shapes are morechallenging, typically requiring a special embedding of the geometry that is invariant un-der different deformations. [68][71] employ the isometry-invariant eigenvalues and eigen-functions of the Laplace-Beltrami operators for 3D shape matching. Diffusion-based char-acteristics, such as heat kernel, are invariant under isometric shape deformations just likegeodesics, and tend to be more stable in the presence of holes and noises, avoiding theneed for surface surgery or repair. Hence, these characteristics are often adopted in iso-metric correspondence in place of geodesics. In [59], Ovsjanikov et al. defined heat kernelmap (HKM) which parameterizes a surface by computing multi-scale heat kernels from afixed reference point; given two shapes whose reference points are matched beforehand,the full correspondence can be recovered via a greedy global Nearest-Neighbor search.[31] proposed the heat kernel coordinates, which utilize multiple feature points as anchors


and globally parameterize the surface via heat kernel. It started with shape feature pointsas the sources of heat kernels via feature detection and matching. Following the priorityorder determined by the magnitude of HKCs, the dense registration is progressively prop-agated from feature sources to all points. Instead of searching for optimal correspondencein the entire domain as in [59], the search space for each point is restricted to the vicinity ofalready-registered points, generating results of great geometric compatibility and adaptingwell to partial shapes with changing boundaries.

Shape Recognition and Retrieval

Non-rigid 3D shape recognition and retrieval are always challenging problems in computervision. For deformable shapes, diffusion based characteristics provide a stable, intrinsicdescriptions which are very suitable for non-rigid shape comparison.

In [13], Bronstein et al. present a general framework for non-rigid shape recognition.For each of the compared shapes, a diffusion kernel is used to compute a family of pair-wise distances between points in each shape, possibly in different diffusion scales. Thedissimilarity between shapes are measured by the dissimilarity of the distributions of thespectral distances.

One of the most-recent techniques is based on an informative shape representationin graphics: the heat kernel signature (HKS) [77]. Unlike conventional mesh and point-cloud representations, the HKS characterizes the shape up to isometry, making it idealfor non-rigid shape comparison. It has been rapidly applied to the state-of-the-art shapegoogle [12, 14, 58]. Given a vocabulary of geometric words, the shape google computesfrequencies of words over the entire shape, which costs a lot of computation.

Recently, bag-of-words (BoW) methods prevail in shape retrieval, coincident with thetrend in image retrieval. It can be traced back to the previous work of shape topics [45]. In[78], a part-based representation is utilized by partitioning the model into subparts. In [39],uniform sampling and local spectral descriptor are adopted for partial shape retrieval. TheShape Google method, originally proposed by Ovsjanikov et al. [12, 14, 58], employs thescale-invariant HKS as the feature descriptor. The features are used to construct geometricwords by taking into account their spatial relations, from which shapes can be constructed,analogous to using features as words and shapes as sentences. Shapes themselves arerepresented using compact binary codes to form an indexed collection. Given a queryshape, similar shapes in the index with possibly isometric transformations can be retrievedby using the Hamming distance of the code as the nearness-measure.

Chapter 4

Manifold Wavelets

Intuitively speaking, a wavelet is a wave-like pulse in time (or space), i.e. a small wave.By scaling and translating a single mother wavelets, we may obtain a family of waveletfunctions that are orthogonal, complete and of varying durations. Just like that in Fourieranalysis a function f can be decomposed into a series of component harmonics, in waveletanalysis f is represented by a family of component wavelets. But there are some funda-mental differences between the Fourier and wavelet analysis:

• In Fourier analysis, each component harmonic is globally defined in space/time. Inwavelet analysis, the component wavelet are all locally defined at different locations.

• In Fourier analysis, each component harmonic has an exclusive frequency. In waveletanalysis, we use multiple wavelets localized at different locations to represent theinformation of a single frequency. Generally, for large scale information (low fre-quency), we use fewer wavelets; whereas for small scale information (high fre-quency), we use more wavelets.

• The component harmonics in Fourier analysis are all orthogonal to each other. Infact, the Fourier basis functions form an orthonormal basis of the space of square-integrable functions. This is not necessarily true for wavelet functions.

In short, wavelet functions can be simultaneously localized in both time/space and fre-quency domain, in contrast to the Fourier transform in which the basis harmonic functionsare all globally defined in time/space. For signals whose primary information lies in lo-calized singularities, such as edges in images or step discontinuities in time series signals,wavelet transform affords a more compact representations than a transform with globalbasis such as the Fourier transform.

31

32 4.1. CLASSIS WAVELET TRANSFORM

Figure 4.1: Comparison of the families of Fourier and wavelet basic functions

There have been many efforts to introduce wavelet methods to the field of visual com-puting. Representative applications include image segmentation [26], image-based ren-dering [57], volume rendering [44], scientific visualization [22], spectral rendering [17],animation compression [61], etc.

4.1 Classis Wavelet Transform

We first give a brief overview of classical wavelet transform for a function f ∈ L2(R), be-ginning with continuous wavelet transform (CWT). Usually, wavelets are constructed froma single mother wavelet ψ(x) ∈ L2(R). By scaling and translating the mother wavelets,we obtain wavelets at different locations and scales. The wavelet function of scale a > 0and localized at location b can be written as

ψa,b(x) =1

aψ(x− ba

) (4.1.1)

For small scales a, ψa,b has high frequencies, and vice versa. Given function f , itswavelet transform (or wavelet coefficient) at scale a and location b is given by the inner

CHAPTER 4. MANIFOLD WAVELETS 33

product of f with ψa,b

Wf (a, b) =

∫ ∞−∞

ψa,b(x)f(x)dx = 〈ψa,b, f〉 (4.1.2)

To reconstruct f from its wavelet transform, the wavelet ψ must satisfy the admissibil-ity condition

Cψ =

∫ ∞0

|ψ(ω)|2

ωdω <∞ (4.1.3)

where ψ(ω) is the Fourier transform of ψ(x).This condition implies, for continuously ψ, that ψ(0) = 0. Hence, ψ(x) must be zero

mean, i.e. ∫ ∞−∞

ψ(x)dx = 0 (4.1.4)

The inverse CWT is given by the following relation

f(x) =1

Cψ

∫ ∞0

∫ ∞−∞

Wf (a, b)ψa,b(x)dadb

a(4.1.5)

In another perspective, Eq.4.1.5 means that f(x) can be decomposed as a superpositionof the time-frequency atoms ψa,b(x).

Analyzing a signal using all wavelet coefficients is computationally impossible. Thisgives rise to the Discrete Wavelet Transform (DWT), which is able to reconstruct a signalfrom a discrete subset of wavelet coefficients. Given real parameter m > 1, n > 0,typically chosen as m = 2 and n = 1, we obtain a series of baby wavelets with integersm,n ∈ Z

ψm,n(x) = s−m/2ψ(s−mx− nt) (4.1.6)

If the functions {ψm,n : m,n ∈ Z} form a tight frame of L2(R), f(x) can be recon-structed by the inverse DWT equation

f(x) =∑m∈Z

∑n∈Z

〈f, ψm,n〉ψm,n(x) (4.1.7)

4.2 Subdivision WaveletsTo construct wavelets on non-Euclidean domains such as graph, mesh and manifold, themost important thing is to imitate the concept of scaling (or dilating). One popular schemeon meshed surfaces is achieved via explicit subdivision, which iteratively refines the mesh

34 4.2. SUBDIVISION WAVELETS

geometry, and at the same time, also refines the functions defined on the mesh. The con-structed wavelets are biorthogonal and locally supported.

The subdivision wavelets rely on the subdivision connectivity of the mesh, which re-stricts the application scope to data compression and level-of-detail rendering. The ideaof subdivision wavelets was first proposed by Schroder and Sweldens [72], in which thelifting scheme was used to construct wavelets on sphere. Lounsbery et al. [46] studiedMRA of wavelets constructed on surfaces of arbitrary topology type. In [6], Bertram et al.utilized bicubic B-spline subdivision to construct wavelet transform that affords boundarycurves and sharp features. In [7], B-spline wavelets were combined with the lifting schemefor biorthogonal wavelet construction.

As a drawback, the subdivision wavelet requires the meshes to have subdivision con-nectivity, where remeshing process is frequently needed. To avoid remeshing, Valette andProst [79] extended the subdivision wavelet for triangular meshes using irregular subdi-vision scheme that can be directly computed on irregular meshes. On spherical domains,Haar wavelets [54, 8] were constructed over nested triangular grids generated by subdi-vision. Recently, the spherical Haar wavelet basis was improved to the SOHO waveletbasis [41] that is both orthogonal and symmetric. In subdivision wavelets, the dilation ofscaling functions strictly follows the subdivision scheme, which depends on the meshing.In [82], a biorthogonal wavelet analysis based on the 3-subdivision was proposed. It is awell orchestrated solution on triangular meshes since the 3-subdivision is of the slowesttopological refinement among all the traditional triangular subdivisions.

The essential motivation of subdivision wavelets is to apply the MRA to arbitrary sur-faces. The subdivision seeks to model a smooth surface via a recursive process of refiningpolygonal faces from a coarse base mesh, which is essentially a model-driven, top-downmethodology towards wavelets definition and construction. The subdivision wavelets havebeen frequently used for geometry compression and level-of-detail data visualization. Itrequires to construct the subdivision hierarchy before defining wavelets, which may limitits application scope. The regularly-refined hierarchy is computationally expensive andperhaps even harder to build. Consequently, it gives rise to strong demand in flexiblyadapted wavelet tools without building the subdivision explicitly, which can be used forfast space frequency processing. Applications span traditional geometry processing to vi-sual analysis, feature extraction, feature-driven data mapping, etc., many of which requirelocal operations on fine details at different frequencies.


4.3 Diffusion WaveletsAnother method to construct wavelets is diffusion-driven, first proposed in [21] on graphsand manifolds, has been proposed. In sharp contrast to the aforementioned subdivisionwavelets, the diffusion wavelets adopt a fundamentally different, bottom-up philosophythat starts from the fine input data. They use a diffusion operator and its powers to ex-pand the nested subspaces, where scaling functions and wavelet functions are obtainedby orthogonalization and rank-revealing compression. This diffusion-driven methodologynaturally dilates the functions associated with the underlying heat diffusion process, whichsolely depends on manifold geometry. It allows flexible construction directly from data.

However, the constructed scaling and wavelet functions are not locally-supported,which limits the functionality of space localization. In fact, it is impossible to constructwavelets that are simultaneously fully orthogonal, locally supported, and symmetric [46].As an improvement, the biorthogonal diffusion wavelets (BDW) [47] were introduced,relieving the excessively-strict orthogonality property of scaling functions. In [48], dif-fusion wavelets were adopted to approximate scalar-valued functions based on analyzingthe structure and topology of the state space. Rustamov [69] studies the relation betweenmesh editing and diffusion wavelets by introducing the generalized linear editing (GLE).However, neither the DW nor the BDW have achieved localization in both manifold andfrequency domain. Intuitively, they are not both attenuated and oscillating on the man-ifold. Moreover, their construction requires expensive operations of QR decompositionand matrix inverse. The rank-revealing QR decomposition downsamples the subspaces tokeep the scaling functions with full rank. Consequently, the reconstruction, i.e., inversetransform, is carried out by either orthogonal or biorthogonal basis at each frequency. Itis, however, inconvenient for spectral processing in multiple frequencies.

In [32] an admissible diffusion wavelets (ADW) on meshed surfaces and point cloudsis proposed. The ADW are constructed in a bottom-up manner that starts from a local oper-ator in a high frequency, and dilates by its dyadic powers to low frequencies. By relievingthe orthogonality and enforcing normalization, the wavelets are locally-supported and ad-missible, hence facilitating data analysis and geometry processing. We define the novelrapid reconstruction, which recovers the signal from multiple bands of high frequenciesand a low-frequency base in full resolution. It enables operations localized in both spaceand frequency by manipulating wavelet coefficients through space-frequency filters.

4.4 Spectral Manifold WaveletThe primary reason that classical wavelet transforms cannot be directly adapted to graphor manifold is that for a mother function ψ(x) defined on a manifold, there is no obvi-

36 4.4. SPECTRAL MANIFOLD WAVELET

ous definition for ψ(sx). One approach to solve this problem is appealing to the Fourierdomain, with the help of aforementioned manifold harmonics. Although scaling cannotbe explicitly expressed on manifold domain, it can be easily defined on the frequency do-main. The idea of spectral wavelet transform was introduced in [30] on the graph domain,denoted as the Spectral Graph Wavelet Transform (SGWT). Here we extend the conceptto general manifold, denoted as the Spectral Manifold Wavelet Transform (SMWT)

Given manifoldMwith appropriate boundary condition. Assume its Laplace-Beltramioperator ∆M has the eigen-decomposition {λk, φk}. The eigenvectors {φk} form a com-plete and orthonormal basis of L2(M), commonly known as the manifold harmonics. Thecorresponding eigenvalues {λk} satisfy

0 = λ0 < λ1 ≤ λ2 ≤ · · · (4.4.1)

For any function f defined onM, its generalized Fourier transform f is defined as

f(k) = 〈φk, f〉 =∞∑k=0

φk(x)f(x) (4.4.2)

And the inverse Fourier transform is

f(x) =∞∑k=0

f(k)φk(x) (4.4.3)

The Parseval relations holds for the manifold harmonics transform

〈f, g〉 = 〈f , g〉 (4.4.4)

4.4.1 Spectral Manifold Wavelet TransformWe generate SMWT from a special wavelet operator that acts on functions defined on themanifold. Given a real-valued transfer function g, the wavelet operator Tg is defined byhow it modulate on f :M→ R on Fourier domain

Tgf(k) = g(λk)f(k) (4.4.5)

Employing the inverse Fourier transform, we obtain the spectral representation of Tgf

(Tgf)(·) =∞∑k=0

g(λk)f(k)φk(·) (4.4.6)


To obtain spectral manifold wavelets, we need localized and scaled versions of Tgf .The scaling is defined by dilating the transfer function as g(tλk). The localization at pointx ∈ M is realized by applying wavelet operators to unit impulse at x, represented by theDirac delta function δx(·)

Since

δx(·) =∞∑k=0

φk(x)φk(·) (4.4.7)

We have the Fourier transform of δx

δx(k) = φk(x) (4.4.8)

Set f = δx in (4.4.6), we have the spectral manifold wavelet at scale t and localized atpoint x

ψt,x(·) = (T tgδx)(·) =∞∑k=0

g(tλk)φk(x)φk(·) (4.4.9)

The spectral manifold wavelet can also be represented as a bivariate kernel

Ψt(x, y) = ψt,x(y) =∞∑k=0

g(tλk)φk(x)φk(y) (4.4.10)

For a real-valued function f defined onM, the spectral manifold wavelet transform is

Wψf (x, t) = 〈ψx,t, f〉 (4.4.11)

Applying the Parseval relation (4.4.4), we obtain the spectral representation of contin-uous SMWT

Wψf (x, t) = 〈ψx,t, f〉 =

∞∑k=0

ψx,t(k)f(k) =∞∑k=0

g(tλk)f(k)φk(x) (4.4.12)

If seen as a function of x, the Fourier transform of the above SMWT is

Wψf (k) = g(tλk)f(k) (4.4.13)

Similar to classical wavelet transform, the spectral manifold wavelet transform is in-vertible only if the transfer function g satisfies the admissibility condition

Cψ =

∫ ∞0

g2(a)

ada <∞ (4.4.14)


and the zero-mean condition g(0) = 0. Then we consider the summation of waveletsψt,x multiplied by corresponding wavelet coefficient Wψ

f (t, x), subject to a non-constantweight dt/t

∑x∈M

∫ ∞0

Wψf (x, t)ψx,t(y)

tdt

=

∫ ∞0

dt

t

∑x∈M

Wψf (x, t)ψy,t(x)

=

∫ ∞0

dt

t

∞∑k=0

Wψf (k)ψy,t(k)

=

∫ ∞0

dt

t

∞∑k=0

(g(λkt)f(k))(g(λkt)φk(y))

=∞∑k=0

(

∫ ∞0

|g(λkt)|2dtt

)f(k)φk(y)

=(

∫ ∞0

|g(a)|2daa

)∞∑k=1

f(k)φk(y)

=Cψ(f(y)− f(0)φ0(y))

This yields the formula of inverse continuous SMWT

f(y) =1

Cψ

∫ ∞0

Wψf (x, t)ψx,t(y)

tdt+ f(0)φ0(y) (4.4.15)

In classical wavelets defined on real line, the space localization is apparent. If themother wavelet ψ(x) is localized in the interval [−ε, ε], then the wavelet ψa,b(x) will belocalized with [b− aε, b+ aε]. in the limit as b→ 0, ψa,b(x)→ 0 for x 6= b.

For spectral manifold wavelets, the localization property is less straightforward sincethe scaling is defined implicitly in the Fourier domain. For g sufficiently regular, thenormalized spectral manifold wavelet ψt,x/‖ψt,x‖ will vanish on vertices sufficiently farfrom x in the limit of fine scales, i.e. as t → 0. We should expect ψt,x(y) to be small if xand y are separated and t is small.

If two transfer functions g and g′ are close to each other, then the derived spectralwavelets should be close to each other in the manifold domain.

As an example, we consider the Mexican hat wavelet. In 1D Euclidean space, the


Figure 4.2: 1D Mexican-hat Wavelet

Figure 4.3: Color plots of the spectral Mexican-hat wavelet with scales of 10 and 30. Thereference point is denoted by the orange ball. We can easily spot the function values oscil-late around the reference point. When t is larger, the period of oscillation also increases.


50 100 150 200 250 300k

0.05

0.10

0.15

0.20

0.25

0.30

0.35

f Ht ΛkL

t = 30

t = 10

Figure 4.4: The Mexican-hat wavelet transfer function in the frequency domain.

Mexican hat wavelet is defined as

ψ(t) =2

√3σπ

14

(1− t2

σ2)e−t22σ2 . (4.4.16)

Its graph is shown in Fig. 4.2, which exhibits clear localization in space. In manifoldspace, we may analogously define the spectral Mexican hat wavelet as

ψt,x(·) =∞∑k=0

t2λ2ke−t2λ2kφk(x)φk(·), (4.4.17)

with the transfer function g(tλ) = t2λ2e−t2λ2 .

Fig. 4.3 visualizes the value of the wavelet functions over the surface, with the scalet = 10 and t = 30. Fig. 4.4 shows the Fourier transform of the wavelet functions infrequency domain. It is easy to see that

• The spectral wavelet function is localized both in manifold and frequency domain.

• On manifold, the values of the spectral wavelet functions attenuate and oscillate asthe distance from the reference point increases. This very similar to how the 1DEuclidean Mexican-hat wavelet behaves over the real line.


• For a larger scale, the Mexican hat wavelet has a wider windows in space, but anarrower window in frequency.

4.4.2 Scaling Functions

By construction, the spectral manifold wavelets ψt,x are all orthogonal to the the nulleigenvector φ0, and nearly orthogonal to φl for λl near zero [30].

Spectral manifold scaling functions are determined by a single real valued functionh : R+ → R, which acts as a low-pass filter and satisfies h(0) > 0 and limx→∞ h(x)→ 0.Introducing the scaling functions helps ensure stable recovery of the original signal f fromthe wavelet coefficients when the scale parameter t is sampled at discrete values tj . Stablerecovery will be assured if G(λ) = h(λ)2 +

∑Jj=1 g(tjλ)2 is bounded away from zero.

The scaling functions defined in this way are merely to smoothly represent the lowfrequency content on the graph. The design of the scaling function generator h is thusuncoupled from the choice of wavelet kernel g.

4.4.3 Reconstruction

The spectral manifold wavelets depend on the continuous scale parameter t. For practicalcomputation, t must be sampled at a finite number of scales {tj}Jj=1, which will engenderNJ wavelets ψtj ,n along with N scaling functions sn. And it is important to be able torecover a signal from the set of discrete wavelet coefficients. It can be proven that [30] theset Γ = {φn, n = 0, . . . , N − 1} ∪ {ψtj ,n, j = 1, . . . , J, n = 1, . . . , N} form a frame withbounds

A = minλ∈[0,λN−1]

G(λ)

andB = max

λ∈[0,λN−1]G(λ),

where G(λ) = h2(λ) +∑

j g(tjλ)2. That is to say, for all f defined on the manifold, thefollowing inequality holds

A‖f‖2 ≤∑k

|〈f,Γk〉|2 ≤ B‖f‖2, (4.4.18)

where Γ = {Γk}.The SMWT is an overcomplete transform. Including the scaling functions φn in the

wavelet frame, the SMWT maps an input vector f of size N to N(J + 1) coefficients c =


Wf . W has an infinite number of let-inverse. A natural choice is to use the pseudoinverseL = (W ∗W )−1W ∗.

Given a set of coefficients c, the synthesis/reconstruction of f can be given by solvingthe matrix equation

(W ∗W )f = W ∗c. (4.4.19)

Chapter 5

Preliminary Work

In this chapter, we present some of our preliminary research on deformable shape analysis.As described in Chapter 2 and 3, the heat kernel on manifold are globally shape-aware,isometry-invariant and resilient to noises. It is particularly suitable for constructing basicshape analysis tools such as point descriptors and feature distances. All of our methodsmake heavy use of heat kernel representations.

5.1 High Order Shape Matching

The heat kernel induced signatures and distances are well suited for corresponding pointson deformable shapes. For better feature points correspondence, other than consideringthe similarity of point heat kernel signatures and pair-wise heat kernel distances, we ad-ditionally take into account the compatibility of the heat kernels across more than twopoints by conducting high-order graph matching on the manifold. The heat kernel tensor(HKT) is a high-order potential of geometric compatibility of feature tuples on manifolds.To facilitate the matching process, we further build up a two-level hierarchy via featureclustering. This simple hierarchy greatly reduces the search space of HKT, and thereforethe computation time.

5.1.1 Heat kernel tensor

Geometric relations among features are extremely important on deformable shapes, andcollectively they are much more reliable than single feature point in shape matching.Therefore, we adopt the advanced tensor matching [25], and transplant it to manifolds

43

44 5.1. HIGH ORDER SHAPE MATCHING

i1

j1

k1

i2

j2

k2

dt(i1, j1) dt(i2, j2)

dt(i1, k1)

dt(j1, k1)

dt(i2, k2)

dt(j2, k2)

OutliersOutlier

Figure 5.1: HKT for shape matching. Three candidate matches (i, j, k) form two “trian-gles”. Some outlier features are circled.

via a diffusion-driven relation measure, given by

dt(x, y) =1

4(−t log ht(x, y))1/2. (5.1.1)

Here, ht(x, y) denotes the heat kernel from point x to y at time t

ht(x, y) =∞∑l=0

e−λltφl(x)φl(y), (5.1.2)

where λl and φl are the l-th eigenvalue and eigenfunction of the Laplace-Beltrami operator.When t→ 0, dt(x, y) is indeed a metric and converges to the geodesic between x and y.

We consider two partial shapes M1 and M2 with overlaps and boundary changes. LetN1 be the number of features extracted onM1, andN2 be the one onM2. A pair i = (i1, i2)denotes a candidate match with a point i1 from M1 and i2 from M2. The problem ofmatching point sets is equivalent to finding an assignment matrix XN1×N2 , such that

Xi1,i2 =

{1 i1 matches i20 otherwise

, with∑i2

Xi1,i2 ≤ 1. (5.1.3)

Note that there may be outliers in the feature set. As shown in Fig. 5.1, some out-liers are circled. For an outlier i1, there is no match in the second feature set, i.e.,∑

i2Xi1,i2 = 0. We adopt the tensor formulation [24] for high-order graph matching

on manifold. Specifically, we consider a tuple of three candidate matches (i, j, k) without

CHAPTER 5. PRELIMINARY WORK 45

conflicts, i.e., i1 6= j1 6= k1 and i2 6= j2 6= k2. They may form two “triangles” by connect-ing them with dt, as shown in Fig. 5.1. Since small heat kernels are error-prone, we selectlarge heat kernels with a threshold εh(t) = 10−6. In the case when the three points do notform a triangle, we simply drop this tuple.

The tuple of candidate matches is then embedded into a 3D space by three angles ofthis triangle. The distance in the embedded space is given by

dθ(i, j, k) = ‖θi1,j1,k1 − θi2,j2,k2‖2, (5.1.4)

where θi1,j1,k1 is a vector comprising three angles of the triangle formed by points i1, j1, k1,and ‖.‖2 denotes the l2-norm. The affinity of the tuple (i, j, k) without conflicts is definedas

τi,j,k = e−dθ(i,j,k)2/σ, (5.1.5)

where σ is a parameter, which can be set as σ = mean(dθ). For tuples with conflicts, welet their affinities equal to zero. The high-order score of assignment X is defined as

score(X) =∑i,j,k

τi,j,kXi1,i2Xj1,j2Xk1,k2 . (5.1.6)

We rewrite the score using tensor notation, given by

score(X) = T ⊗1 X ⊗2 X ⊗3 X, (5.1.7)

where ⊗d denotes the tensor product in d dimension. We call T the heat kernel tensor, asit utilizes heat kernels to form the tensor. The HKT can be fused with different order of po-tentials. Here, the HKT is a 3rd-order tensor with entries τi,j,k defined in Eq. (5.1.5). Thefinal results are obtained according to their matching scores subject to conflict constraintsin Eq. (5.1.3).

5.1.2 Matching hierarchyFor articulated shapes, we design a two-level hierarchy to improve the time performanceby reducing the searching space. Articulated shapes with long branches can be easily seg-mented using some low frequency eigenfunctions of the Laplace-Beltrami operator [64].In the upper level, we find centers of clusters as the local extrema of the first two non-trivial Laplace-Beltrami eigenfunctions, and remove redundant ones that are very closeto selected centers. In the lower level, extracted shape features are then clustered intosub-graphs based on their heat kernels to the cluster centers. The goal of the upper-levelmatching is to reduce the searching space, and it can be skipped whenever necessary.

46 5.1. HIGH ORDER SHAPE MATCHING

Figure 5.2: Matching hierarchy. Extracted features are clustered into sub-graphs.

The cluster centers comprise a hyper-graph in the upper level of the hierarchy, as shownin Fig. 5.2. In the hyper-graphs with hyper-nodes (cluster centers), we compute theirHKT. We release conflicting constraints by allowing candidate matches that have matchingscores greater than 80% of the maximal one. This will prune diverse sub-graphs, andreduce the search space of HKT. At the lower level, we run HKT in each cluster. For thehigh-order optimization in Eq. (5.1.7), we use the tensor power iteration with l1-norms ofcolumns. The complexity of one power iteration is O(m), where m is the number of non-zero elements in the tensor. We restrict the number of triangles (i.e., non-zero elements) to64N1 by randomly selecting tuples. As a result, the computation of HKT is very efficient.

5.1.3 Experimental results

To evaluate the proposed method, we conduct various experiments, including scale change(Fig. 5.3), noise/topology change (Fig. 5.4)), large deformation (Fig. 5.5), and semanticmatching (Fig. 5.6). The only parameter of our method needed to be tuned is time t inHKT. We set t = 20 for two-level matching and t = 60 for one-level matching.

In Fig. 5.4, we add Gaussian noise (σ = 10% of average edge length) to vertex coor-dinates, and also introduce topology noise by punching holes on models. For large defor-mation, we test our method through a sequence of deforming shapes of a dancing woman,with selected frames shown in Fig. 5.5. The matching results are stable under large de-


Figure 5.3: Experiment of deforming shapes with scale changes by our method.

Figure 5.4: Matching with noise (Left) and topology changes (Right).

48 5.2. HIERARCHICAL SHAPE REGISTRATION

Figure 5.5: Selected frames of matching largely deforming objects.

formations across these frames. Our method can also be applied to match partial shapesfrom similar objects, resulting in semantic correspondences shown in Fig. 5.6. We skipthe upper-level matching, since we may need geometric constraints from far-away points.The fists of a man are matched to the wrists of a woman. This is because the woman’sarms are thinner than the man’s arms, thus, the heat diffuses faster on the woman’s arms.And the first hand is a closed surface, while the second one is open.

For quantitative evaluation, we compute differences of geodesics between matchedpairs. For a match pair i = (i1, i2) in the correspondence set S, we find j = minj(dµ(i1, j1))with geodesic dµ, and compute

error(i) =|dµ(i1, j1)− dµ(i2, j2)|

e, (5.1.8)

where e denotes the average edge length. The mean errors of all correspondences aredocumented in Tab. 5.1.3. We also detail the time performance running on a laptop. Weadopt the HKS for feature detection, which shares the computation of eigen-decompositionwith the HKT. It may be noted that, the computation of eigen-decomposition is not listedin Tab. 5.1.3. It costs about 45 seconds for a pair of meshes by our implementation.

5.2 Hierarchical Shape RegistrationWe improve the registration algorithm in [31] by implementing a hierarchical correspond-ing process. The central idea is to generate correspondences in multiple levels in a coarse-to-fine manner, with additional features incrementally inserted in each level. The regis-tration starts from the coarsest resolution. Registration results obtained in one level serveas references for the registration in the next level. We adopt the heat kernel coordinatesfor local shape parameterization, giving rise to a complete solution capable of registeringpartial shapes undergoing isometric deformation with higher accuracy.


Figure 5.6: Matching similar objects.

Data #V1, #V2Time (sec)

Mean errorHKS HKT

man1 10.0k, 5.7k 4.99 1.30 0.788∗

man2 10.0k, 5.9k 3.14 0.5 0.703∗

woman 10.0k, 5.8k 5.59 0.81 0.822hand 10.0k, 9.0k 1.45 0.16 N/A

∗ evaluated before scaling or noise

Table 5.1: Time performance and quantitative evaluation of our method.


Initial features

Construct multi-res structure

Coarse registration

Additional features

Refined registration

Figure 5.7: Pipeline overview of our hierarchical registration framework.

A common and effective approach to dense correspondence is first matching a smallnumber of pre-selected feature points, and then using the matched features as referencesfor dense correspondences. Features, encoding important information of shapes, can beused to parameterize the shapes and serve as anchors to bootstrap the matching of the restpoints. In general, it is necessary to have a fairly large number of matched features to ob-tain dense correspondence of good quality. Otherwise, the feature-based parameterizationmay have difficulty in discriminating nearby elements, especially in parts of shapes thatare far away from any features. However, automatically finding and matching a large num-ber of features is very difficult and error-prone. Even in the case of user-assisted featurematching, one would prefer a small set of matched features, since manually correspondingmany features is burdensome and time-consuming.

As illustrated in Figure 5.7, the main steps of our method are: (1) Detect and matchfeatures to get a small initial set of feature matches; (2) Construct hierarchical structures ofinput shapes; (3) Perform registration at the coarsest level using the initial feature set; (4)Select some newly registered points as additional features; (5) Perform registration at thenext level using results from the previous level and the expanded set of feature references;(6) Repeat step (4) and (5) until all valid points are registered.

The rationale of our approach is that distinguishing elements that are distant from eachother on the surface is much more accurate than nearby elements. Even with a smallnumber of features, we can achieve very good registration on a heavily downsampledversion of the original shapes. The registration result of a coarse resolution can serve asseed correspondences when performing registration in a finer level. The large number ofavailable seeds significantly reduce the chances of correspondences being trapped in anincorrect location. Moreover, the multi-resolution process enables us to pick additionalfeatures from already registered points. This greatly enhances the discriminative strengthas the meshes become more refined.

5.2.1 Hierarchical registrationGiven a source shape S and a target shape T , both represented as triangular meshes, andlet V S = {si} and V T = {ti} be their respective vertex sets, the objective of dense


(a) (b) (c) (d)

Figure 5.8: Major steps of our hierarchical registration algorithm. The blue shape is thesource and the red one is the target. We use a three level hierarchy in this example. (a)Initial feature correspondences; (b) Coarse registration result (Third level); (c) Expandedfeature correspondences (Third level); (d) Final registration result.

registration is to find an optimal mapping τ : V S → V T . In practice, we represent theregistration results as a set of correspondencesR = {vSi , vTi }. When the shapes in questionare not complete, some vertices in V S may not have correspondences in R.

Initial feature detection and matching

The goal of this step is to obtain a small feature correspondence set C∗. One can employany good method to find and match features as long as the matched features are stableand representative. In this work, we adopt the heat kernel signature (HKS) [77] to extractmulti-scale features and spectral graph matching method [40] to match them.

Multiresolution Structure

Once we obtained features correspondence set C∗, we can use it as reference to propagatethe correspondences by searching in the vicinity of already matched vertices, until everysource vertex is mapped to a vertex in the target shape [31, 35]. However, when the size ofC∗ is small, simple propagation approaches often cannot produce satisfactory registration.On one hand, with insufficient features as anchors, it is difficult to distinguish nearby ver-tices no matter what kind of parameterization scheme we employ. On the other hand, sincethe sources for propagation are few, wrong correspondences are more likely to accumulatefollowing a mismatch.

To address this issue, instead of computing registration in a single run, we perform ithierarchically in a coarse-to-fine manner. We construct a multi-resolution structure of theoriginal shapes, and in each level we only register vertices that belong to the current resolu-tion. Given a triangular mesh M0 = (V0, F0) and constants d,m ∈ Z, we downsample M


and obtain the mesh hierarchy {M0,M1, . . . ,Mm}. Assume Mi = (Vi, Fi) and ni = |Vi|,we enforce that ni+1 = ni/d. We adopt the method in [28] for mesh downsampling. Inour implementation, we select d = 4.

Correspondence Propagation and Feature Expansion

Let both the initial correspondence set Rm+1 and initial feature set Cm+1 be C∗. In levell, we input the previous level’s registration result Rl+1 and feature set Cl+1. The goal is tofind the l-th level correspondence setRl that registers meshes Sl and Tl, with an augmentedfeature set Cl.

For each vertex x in Sl and Tl, we compute its heat kernel coordinates

HKC(x) = (ht(x, c1), . . . , ht(x, cz)), ci ∈ Cl+1. (5.2.1)

Inheriting Rl+1 as the initial correspondence set, we propagate correspondence tomatch the rest vertices in Sl and Tl. We use a heap to determine the order by whichthe vertices in Sl are processed, prioritizing on the magnitude of HKC. For an alreadymatched pair (sj, tj) and one of sj’s immediate neighbor si, we search for si’s best corre-spondence ti ∈ V T

k in the neighborhood of tj , and add (si, ti) into the correspondence set.ti is selected using the following criterion

ti = arg mint∈n(tj ,Tk)

‖HKCS(si)− HKCT (t)‖2 (5.2.2)

where n(tj, Tk) represent the set of tj’s neighboring vertices in Tk, and HKCS and HKCT

denote the heat kernel coordinates of points on S and T .The correspondence propagation continues until all vertices in Sk have been matched

and we get the correspondence set {(si, ti)} ⊂ V Sm−1 × V T

m−1. Note that for each corre-spondence (si, ti), the endpoint ti actually represent a set of vertices K(ti) in the originalmesh T0. To find the precise correspondence of si in the original target mesh, we searchK(ti) and replace ti with tj ∈ K(ti) if tj is closer to si in the embedding space. The resultis the l-th level correspondence set Rk that relates points si ∈ Sk to ti ∈ T0. For eachcorrespondence (si, tj), we assign a matching score

score(si, tj) = exp(−‖HKCS(si)− HKCT (tj)‖2). (5.2.3)

We then select from Rl some vertex pairs as new features and insert them into thefeature set. These new added feature pairs should be both reliable (having great matchingscore) and not in the δ-neighborhood of any existing feature points. The expanded featureset Cl enables a more discriminative HKC in the next level. We carry on this process fromthe coarsest level to the finest level until we obtain the final registration set R0 between theoriginal meshes S0 and T0. Figure 5.8 shows the major steps of our algorithm.


data |V S|, |V T | #initialfeatures

#final fea-tures

#levels error(multi-level)

error (sin-gle level)

Face 10.5K, 9.8K 11 68 2 0.058 0.108Horse 8.4K, 8.4K 18 37 3 0.083 0.101Cat 7.2K, 7.2K 14 65 3 0.123 0.167Man 10.0K, 10.0K 13 90 3 0.077 0.117

Woman 10.0K, 10.0K 25 106 3 0.048 0.101

Table 5.2: Evaluation result. Our method has lower errors than the single-level method.

5.2.2 Experimental Results

To assess the registration results represented by map τ : V S → V T , we randomlysample N = 300 pairs of source vertices {(s11, s12), (s21, s22), . . . , (sN1 , sN2 )}. We measurethe quality of τ in terms of the mean relative error of geodesics:

error(τ) =1

N

N∑i=1

|dSG(si1, si2)− dTG(τ(si1), τ(si2))|dSG(si1, s

i2)

(5.2.4)

where dSG and dTG are the respective geodesic distance functions on surface S and T .We evaluate our algorithm on various models and compare it with the single-resolution

method [31]. Table 5.2.2 documents the evaluation results. Starting with the same ini-tial feature set, our hierarchical method consistently achieves a better registration thanthe single-resolution method. In average, the sampled error is only 64% of the single-resolution approach. Figure 5.9 shows a few registration results in our experiments.

5.3 Bag-of-Feature-Graphs Shape RetrievalWe present a new paradigm, called bag-of-feature-graphs (BoFG), for non-rigid shape re-trieval. The basic idea is to represent a shape by constructing graphs among its features,which significantly reduces the number of points involved in computation. Given a vo-cabulary of geometric words, for each word the BoFG builds a graph that records spatialinformation of features, weighted by their similarities to this word. This eliminates un-likely points in a word category, during shape comparison. Feature graphs are governedby their affinity matrices of weighted heat kernels, whose eigenvalues form a concise shape

54 5.3. BAG-OF-FEATURE-GRAPHS SHAPE RETRIEVAL

Figure 5.9: Some registration results by our multi-resolution method (Left) and the single-resolution method [31] (Right). Large colored dots represent matched features.


descriptor. Given a vocabulary of geometric words, corresponding to each word we builda graph that records spatial information between features, weighted by their similarities tothis word. Specific characteristics of the BoFG include:

• It is concise by significantly reducing the number of points involved in representa-tion, and thus, is fast to compute.

• It explicitly records spatial information among features.

• It is representative, since features are salient points containing important informationof the shape.

• Graphs have different dominating features associated with corresponding words.This greatly improves the accuracy of shape comparison by eliminating unlikelyword-distributions.

5.3.1 Shape-Google revisitWe first revisit the shape google originally introduced by [58]. It utilize a HKS-basedBoW. The HKS descriptor K(x) is a vector of HKS sampled at different values of t. LetW = {W1, . . . ,WV } be a vocabulary of geometric words with size V . The words {Wi} arerepresentative HKS vectors in the descriptor space clustered by the k-means algorithm. Foreach point x, the shape google computes its word distribution Θ(x) = [θ1(x), . . . , θV (x)]T .The similarity of x and word Wi is given by

θi(x) = c(x)e−‖K(x)−Wi‖

2

2σ2 , (5.3.1)

where σ is a parameter, and c(x) is a constant for normalization. The BoW descriptor of asurface M is computed by integrating word similarities over the entire shape

f(M) =

∫M

Θ(x)dµ(x), (5.3.2)

where µ(x) denotes the surface area of x. As shown in Fig. 5.10, the BoW descriptor isa V×1 vector that measures the frequencies of words appearing on the shape. The shapegoogle also introduced a SS-BoW descriptor, given by

F (M) =

∫M×M

Θ(x)ΘT (y)ht(x, y)dµ(x)dµ(y). (5.3.3)

As shown in Fig. 5.10, it is a V×V matrix that measures frequencies of word pairs.Assume the time complexity for computing a HKS descriptor is O(D). For a shape with N


BoW vector (V×1)

Shape SS-BoW matrix (V×V)

. . .BoFG matrices (|F|×|F|)

Figure 5.10: Representations of a give shape.


points, the time complexity of BoW is O(ND), and SS-BoW is O(N2D) which is quadraticto N.

According to the Informative Theorem in [77], the HKS contains all the information ofheat kernels. Thus, the SS-BoW has no more geometry information than the BoW beforeintegration. For the ease of comparison, the shape google highly suppresses the geom-etry information by computing the frequencies of words or word-pairs on the shape. Itends up with concise descriptors for comparison, yet completely loses spatial information.Besides, the shape-google algorithms are time-consuming, since they are working on allthe points in the data. The BoW needs computing HKS values of all points, while theSS-BoW needs computing all point-to-point heat kernels. Assume the time complexity forcomputing a HKS descriptor is O(D). For a shape with N points, the time complexity ofBoW is O(ND), and SS-BoW is O(N2D) which is quadratic to N.

5.3.2 Bag of feature graphsTo reduce the complexity of the shape google, one needs to reduce the number of pointsinvolved in representing the shape. A straightforward solution is to select feature points,which keep most information of the shape geometry. Because of the multi-scale property,HKS features contain geometry information ranging from points in small scales to theentire shape in large scales. However, one concern is that a reduced number of pointsmay not be sufficient to faithfully represent the shape. Therefore, instead of countingword frequencies, we construct graphs on detected features, giving rise to a bag-of-feature-graphs (BoFG) paradigm. The graphs encode spatial relations between features, whichcontain much more geometry information in representing the shape.

We adopt weighted heat kernel matrices to capture global structures of graphs. Specifi-cally, for a shape M with feature set F , only points x ∈ F are involved in computing worddistributions Θ(x), which reduces much computation. Features are vector-quantized by afuzzy classification, which assigns θi(x) portion of similarity to word Wi in the distribu-tion of feature x. The distribution Θ(x) is computed by Eq. (5.3.1) with σ set as a quarterof the average distance of words in the vocabulary. This fuzzy classification reduces ambi-guities in graph comparison, and also avoids misclassification in a hard quantization. Fora geometric word Wi, we construct a matrix Gi, whose entry Gi(x, y) with (x, y) ∈ F ×Fis computed by,

Gi(x, y) = θi(x)θi(y)ht(x, y). (5.3.4)

It is the heat kernel between x and y weighted by their similarities to the geometric wordWi.

The matrix set G(M) = {G1, . . . , GV } comprises a BoFG representation of the shapeM . As shown in the bottom row of Fig. 5.10, matrices characterize spatial information of


0 5 10 15 20 250

0.002

0.004

0.006

0.008

0.01

0.012

Figure 5.11: Some non-rigid shapes and their BoFG descriptors.

features assigned to different word categories. The near-zero entries in a matrix indicatethey are hardly classified to this category, and therefore, not considered in this graph. Itcontains all the geometric information of features in a multi-scale way, which faithfullycharacterizes the shape. The computation complexity for this matrix representation isO(|F|2D), as the computed heat kernels can be shared by all matrices. Considering the sizeof feature set is always much less than the total number of points on the shape, the BoFGis much faster than the shape google.

5.3.3 Shape retrieval

The mechanism of shape retrieval is to build concise BoFG descriptors of shape models ina database in an off-line process, and retrieve related shapes for a query by the approximatenearest neighbor (ANN) search. The BoFG descriptor consists of significant eigenvaluesof BoFG matrices. Each Gi is a real symmetric matrix, whose eigenvalues are all realand eigenvectors are perpendicular to each other. We choose its six largest eigenvaluesdenoted as Si(M), which contributes to a 6V×1 vector [S1(M), . . . , SV (M)]T as a concisedescriptor. This reduces the dimension of the matrix by multi-dimensional scaling (MDS)[9]. Fig. 5.11 shows some non-rigid shapes and their BoFG descriptors. The deformed cat-models have very similar BoFG descriptors, while the horse-model has a quite differentone. It projects the matrix to its main directions with coordinates leaving in Si(M), whichare stable to a small amount of outliers. Then, we define the similarity distance between


0.69 2.14

136.36

0

20

40

60

80

100

120

140

160

BoFG BoW SS-BoW

1.89 18.83

520.24

0

100

200

300

400

500

600

BoFG BoW SS-BoW

Figure 5.12: Time performance (in seconds) of three descriptors on two shapes.

two shapes M1 and M2 as

d(M1,M2) =V∑i=1

‖Si(M1)− Si(M2)‖2. (5.3.5)

The above distance is based on one-scale heat kernels, which can be easily extendedto multi-scale by averaging distances of heat kernels at different values of t.

5.3.4 Experimental resultsWe conduct various experiments of shape retrieval to evaluate the proposed method. Thetest database includes non-rigid shapes of the TOSCA 1 dataset as positives, and shapesfrom the Princeton Shape Benchmark 2 as negatives. The TOSCA database contains 12classes of a total 148 non-rigid shapes. The TOSCA database contains 12 classes (cat,centaur, dog, wolf, horse, lion, gorilla, seahorse, shark, woman, and two men) of a total148 non-rigid shapes. Each category has shapes with different poses (i.e., isometric de-formations). For negatives, we select 40 classes of a total 400 shapes from the PrincetonShape Benchmark, which are different from the positives.

1http://tosca.cs.technion.ac.il/book/shrec.html2http://shape.cs.princeton.edu/benchmark/

http://tosca.cs.technion.ac.il/book/shrec.html

http://shape.cs.princeton.edu/benchmark/


0 0.2 0.4 0.6 0.8 10.6

0.7

0.8

0.9

1

BoWFSS−BoWSI−HKSBoFG

0 0.2 0.4 0.6 0.8 10.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 10.5

0.6

0.7

0.8

0.9

1

Figure 5.13: Precision-recall curves of evaluated methods, with categories of (from Left toRight) null, scale change, and hole.

First, we compare the time performance of computing a descriptor of BoW, SS-BoW,and BoFG. For a query shape, one needs to compute its descriptor first to initiate the re-trieval. Fig. 5.12 shows the time performances of three descriptors on a shape with 3kvertices (Left) and another one with 30k vertices (Right). The feature numbers involvedin BoFG for two shapes are 42 and 98, respectively. The time for computing Laplace-Beltrami eigenfunctions are excluded, since it is shared by all three methods. By reducingthe number of points used in computation, the BoFG significantly improves the time per-formance of computing shape descriptors. The improvement is more significant when theratio of points to features is greater.

The query shapes are obtained from the positives of the database. To test the methodsunder some challenging cases, we apply transformations to the query shapes. This leadsto categorized experiments, including null (no transformation), scale change (scaling ver-tex coordinates), and hole (topological change and missing information). For comparisonpurpose, we also evaluate some state-of-the-art methods that are similar to ours, includingthe BoW shape google, the SS-BoW shape google, and the SI-HKS. Since the SS-BoWruns extremely slow, we use over 100 features in its implementation, denoted as FSS-BoW. The three methods share a vocabulary with 48 words. For the BoFG, the vocabularysize depends on the diversity of models in the database, and the number of features usu-ally identified on a shape. Here, we use about 30 to 50 features for one shape, and thevocabulary size is 4.

The methods are quantitatively evaluated by the precision-recall (PR) curve that isoften adopted for evaluating retrieval performance [27]. It plots the trade-off betweenprecision (ratio of the number of relevant shapes retrieved and the total number of shapesretrieved) and recall (ratio of the number of relevant shapes retrieved and the total numberof existing relevant shapes that could be ideally retrieved). Fig. 5.13 plots the PR curvesof evaluated methods, with categories of null, scale change, and hole.

Chapter 6

Conclusion

In this report, we have briefly reviewed the theories and applications of harmonic analysison deformable shapes. The classical Fourier analysis in Euclidean space can be conve-niently generalized to the manifold space, and later to discrete meshes, through the man-ifold harmonics (eigenfunctions of the Laplace-Beltrami operator). We articulated theconnections between the classical Fourier analysis and the manifold Fourier analysis, thelatter formed upon the manifold harmonics. We introduced the three major applications ofthe manifold harmonics:

• Spectral geometry processing, such as smoothing, enhancing, compression and wa-termarking the mesh geometry.

• Deformable shape analysis, such as shape classification and symmetry detection.

• Representing diffusion-based geometric characteristics, such as heat kernel and dif-fusion distance, which are widely used in shape matching and recognition.

The wavelet analysis have many nice properties that are absent in Fourier analysis, andhave replaced Fourier analysis in many areas. We discussed the construction of differenttypes of manifold wavelets, with a focus on the spectral manifold wavelet, which shares acommon formulation with diffusion kernels on manifold.

Finally, we presented three of our preliminary researches on diffusion-based shapeanalysis: high-order shape matching, hierarchical shape registration and Bag-of-Feature-Graphs (BoFG) shape retrieval.

For future work, we plan to extend of our current work in the following areas:

• Make improvements on our shape correspondence algorithm by incorporating thesimilarity of point descriptors when evaluating candidates for best matches, and

61

62

employing alternative diffusion kernels for local shape parameterization. We alsoplan to explore related applications for shape correspondence, such as space-timemodel completion and surface reconstruction.

• Continue the study on spectral manifold wavelet, including the efficient computationof inverse transform, design of generator function, and possible applications such aslocalized space-frequency mesh filtering and multi-scale feature detection.

Bibliography

[1] Yonathan Aflalo, Alexander Bronstein, Michael Bronstein, and Ron Kimmel. De-formable shape retrieval by learning diffusion kernels. In Scale Space and Vari-ational Methods in Computer Vision, volume 6667 of Lecture Notes in ComputerScience, pages 689–700. Springer Berlin / Heidelberg, 2012.

[2] M. Aubry, U. Schlickewei, and D. Cremers. The wave kernel signature: A quan-tum mechanical approach to shape analysis. In Computer Vision Workshops (ICCVWorkshops), 2011 IEEE International Conference on, pages 1626 –1633, nov. 2011.

[3] M. Belkin and P. Niyogi. Using manifold structure for partially labelled classifi-caiton. Adv NIPS, 15, 2003.

[4] Mikhail Belkin and Partha Niyogi. Laplacian eigenmaps for dimensionality reduc-tion and data representation. Neural Computation, 13:1373–1396, 2003.

[5] Mikhail Belkin, Jian Sun, and Yusu Wang. Discrete laplace operator on meshedsurfaces. In Proceedings of the twenty-fourth annual symposium on Computationalgeometry, SCG ’08, pages 278–287, New York, NY, USA, 2008. ACM.

[6] Martin Bertram, Mark A. Duchaineau, Bernd Hamann, and Kenneth I. Joy. Bicubicsubdivision-surface wavelets for large-scale isosurface representation and visualiza-tion. In Proceedings of the 11th IEEE Visualization 2000 Conference (VIS 2000),VISUALIZATION ’00, Washington, DC, USA, 2000. IEEE Computer Society.

[7] Martin Bertram, Mark A. Duchaineau, Bernd Hamann, and Kenneth I. Joy. General-ized b-spline subdivision-surface wavelets for geometry compression. IEEE Trans-actions on Visualization and Computer Graphics, 10(3):326–338, May 2004.

[8] Georges-Pierre Bonneau. Optimal triangular haar bases for spherical data. In Pro-ceedings of the conference on Visualization ’99: celebrating ten years, VIS ’99,pages 279–284, Los Alamitos, CA, USA, 1999. IEEE Computer Society Press.

63

64 BIBLIOGRAPHY

[9] A. M. Bronstein, M. M. Bronstein, and R. Kimmel. Generalized multidimen-sional scaling: a framework for isometry-invariant partial surface matching. PNAS,103(5):1168–1172, 2006.

[10] Alexander Bronstein, Michael Bronstein, Ron Kimmel, Mona Mahmoudi, andGuillermo Sapiro. A gromov-hausdorff framework with diffusion geometryfortopologically-robust non-rigid shape matching. International Journal of Com-puter Vision, 89:266–286, 2010. 10.1007/s11263-009-0301-6.

[11] Alexander M. Bronstein. Spectral descriptors for deformable shapes. CoRR,abs/1110.5015, 2011.

[12] Alexander M. Bronstein, Michael M. Bronstein, Leonidas J. Guibas, and Maks Ovs-janikov. Shape google: Geometric words and expressions for invariant shape re-trieval. ACM Trans. Graph., 30(1):1:1–1:20, February 2011.

[13] M.M. Bronstein and A.M. Bronstein. Shape recognition with spectral distances. Pat-tern Analysis and Machine Intelligence, IEEE Transactions on, 33(5):1065 –1071,may 2011.

[14] M.M. Bronstein and I. Kokkinos. Scale-invariant heat kernel signatures for non-rigidshape recognition. In Computer Vision and Pattern Recognition (CVPR), 2010 IEEEConference on, pages 1704 –1711, june 2010.

[15] Ming Chuang and Michael Kazhdan. Interactive and anisotropic geometry process-ing using the screened poisson equation. ACM Trans. Graph., 30(4):57:1–57:10, July2011.

[16] F. R. K. CHUNG. Spectral Graph Theory. AMS, Providence, RI, USA, 1997.

[17] Jean Claude Iehl and Bernard Peoche. An adaptive spectral rendering with a percep-tual control. Computer Graphics Forum, 19(3):291–300, 2000.

[18] R. R. Coifman, S. Lafon, A. B. Lee, M. Maggioni, B. Nadler, F. Warner, and S. W.Zucker. Geometric diffusions as a tool for harmonic analysis and structure definitionof data: Diffusion maps. PNAS, 102(21):7426–7431, 2005.

[19] R. R. Coifman, S. Lafon, A. B. Lee, M. Maggioni, B. Nadler, F. Warner, and S. W.Zucker. Geometric diffusions as a tool for harmonic analysis and structure definitionof data: Multiscale methods. PNAS, 102(21):7432–7437, 2005.

BIBLIOGRAPHY 65

[20] Ronald R. Coifman and Stphane Lafon. Diffusion maps. Applied and ComputationalHarmonic Analysis, 21(1):5 – 30, 2006. Special Issue: Diffusion Maps and Wavelets.

[21] Ronald R. Coifman and Mauro Maggioni. Diffusion wavelets. Applied and Compu-tational Harmonic Analysis, 21(1):53 – 94, 2006.

[22] Gheorghe Craciun, Ming Jiang, David Thompson, and Raghu Machiraju. Spatialdomain wavelet design for feature preservation in computational data sets. IEEETransactions on Visualization and Computer Graphics, 11:149–159, 2005.

[23] Mathieu Desbrun, Mark Meyer, Peter Schroder, and Alan H. Barr. Implicit fairing ofirregular meshes using diffusion and curvature flow. In Proceedings of the 26th an-nual conference on Computer graphics and interactive techniques, SIGGRAPH ’99,pages 317–324, New York, NY, USA, 1999. ACM Press/Addison-Wesley PublishingCo.

[24] O. Duchenne, F. Bach, I. Kweon, and J. Ponce. A tensor-based algorithm for high-order graph matching. In Computer Vision and Pattern Recognition, 2009. CVPR2009. IEEE Conference on, pages 1980 –1987, june 2009.

[25] O. Duchenne, F. Bach, In-So Kweon, and J. Ponce. A tensor-based algorithm forhigh-order graph matching. Pattern Analysis and Machine Intelligence, IEEE Trans-actions on, 33(12):2383 –2395, dec. 2011.

[26] M.A.T. Figueiredo. Bayesian image segmentation using wavelet-based priors. InComputer Vision and Pattern Recognition, 2005. CVPR 2005. IEEE Computer Soci-ety Conference on, volume 1, pages 437 – 443 vol. 1, june 2005.

[27] Thomas Funkhouser, Patrick Min, Michael Kazhdan, Joyce Chen, Alex Halderman,David Dobkin, and David Jacobs. A search engine for 3d models. ACM Trans.Graph., 22(1):83–105, January 2003.

[28] Michael Garland. Quadric-Based Polygonal Surface Simplification. PhD thesis,Carnegie Mellon University, 1999.

[29] Jonas Gomes and Luiz Velho. From fourier analysis to wavelets. Course Notes -SIGGRAPH, 1999.

[30] David K. Hammond, Pierre Vandergheynst, and Rmi Gribonval. Wavelets on graphsvia spectral graph theory. Applied and Computational Harmonic Analysis, 30(2):129– 150, 2011.

66 BIBLIOGRAPHY

[31] Tingbo Hou and Hong Qin. Robust dense registration of partial nonrigid shapes.TVCG, 99, 2011.

[32] Tingbo Hou and Hong Qin. Admissible diffusion wavelets and their applicationsin space-frequency processing. IEEE Transactions on Visualization and ComputerGraphics, 99(PrePrints), 2012.

[33] Tingbo Hou and Hong Qin. Continuous and discrete mexican hat wavelet transformson manifolds. Graphical Models, 74(4):221 – 232, 2012.

[34] E.P. HSU. Stochastic Analysis on Manifolds. American Mathematical Society, 2002.

[35] Qi-Xing Huang, Bart Adams, Martin Wicke, and Leonidas J. Guibas. Non-rigidregistration under isometric deformations. Computer Graphics Forum, 27(5):1449–1457, 2008.

[36] M. Isenburg and P. Lindstrom. Streaming meshes. In Visualization, 2005. VIS 05.IEEE, pages 231 – 238, oct. 2005.

[37] Peter W. Jones, Mauro Maggioni, and Raanan Schul. Manifold parametrizations byeigenfunctions of the laplacian and heat kernels. PNAS, 105(6):1803–1808, 2008.

[38] Zachi Karni and Craig Gotsman. Spectral compression of mesh geometry. InProceedings of the 27th annual conference on Computer graphics and interactivetechniques, SIGGRAPH ’00, pages 279–286, New York, NY, USA, 2000. ACMPress/Addison-Wesley Publishing Co.

[39] Guillaume Lavoue. Bag of words and local spectral descriptor for 3d partial shaperetrieval. In 3DOR, pages 41–48, 2011.

[40] M. Leordeanu and M. Hebert. A spectral technique for correspondence problemsusing pairwise constraints. In ICCV’05, 2005.

[41] Christian Lessig and Eugene Fiume. Soho: Orthogonal and symmetric haar waveletson the sphere. ACM Trans. Graph., 27(1):4:1–4:11, March 2008.

[42] B. Levy. Laplace-beltrami eigenfunctions towards an algorithm that ”understands”geometry. In Shape Modeling and Applications, 2006. SMI 2006. IEEE InternationalConference on, page 13, june 2006.

[43] Yaron Lipman, Raif M. Rustamov, and Thomas A. Funkhouser. Biharmonic distance.ACM Trans. Graph., 29(3):27:1–27:11, July 2010.

BIBLIOGRAPHY 67

[44] L. Lippert, Technische Hochschule, Politecnico Federale Di Zurigo, M. H. Gross,and M. H. Gross. Fast wavelet based volume rendering by accumulation of transpar-ent texture maps, 1995.

[45] Yi Liu, Hongbin Zha, and Hong Qin. Shape topics: A compact representation andnew algorithms for 3d partial shape retrieval. In In Proceedings of IEEE Conferenceon Computer Vision and Pattern Recognition (CVPR 2006, 2006.

[46] Michael Lounsbery, Tony D. DeRose, and Joe Warren. Multiresolution analysis forsurfaces of arbitrary topological type. ACM Trans. Graph., 16(1):34–73, January1997.

[47] Mauro Maggioni, James C. Bremer, Ronald R. Coifman, and Arthur D. Szlam.Biorthogonal diffusion wavelets for multiscale representations on manifolds andgraphs. In Manos Papadakis, Andrew F. Laine, and Michael A. Unser, editors, Pro-ceedings of the SPIE, volume 5914, 2005.

[48] Sridhar Mahadevan and Mauro Maggioni. Value function approximation with dif-fusion wavelets and laplacian eigenfunctions. Technical Report 38, Department ofComputer Science, University of Massachusetts, 2005.

[49] Mona Mahmoudi and Guillermo Sapiro. Three-dimensional point cloud recognitionvia distributions of geometric distances. Graphical Models, 71(1):22 – 31, 2009.

[50] D. Mateus, R. Horaud, D. Knossow, F. Cuzzolin, and E. Boyer. Articulated shapematching using laplacian eigenfunctions and unsupervised point registration. InComputer Vision and Pattern Recognition, 2008. CVPR 2008. IEEE Conference on,pages 1 –8, june 2008.

[51] F. Memoli. Spectral gromov-wasserstein distances for shape matching. In ComputerVision Workshops (ICCV Workshops), 2009 IEEE 12th International Conference on,pages 256 –263, 27 2009-oct. 4 2009.

[52] Mark Meyer, Mathieu Desbrun, Peter Schroder, and Alan H. Barr. Discretedifferential-geometry operators for triangulated 2-manifolds. In Hans-ChristianHege and Konrad Polthier, editors, Visualization and Mathematics III, pages 35–57.Springer-Verlag, Heidelberg, 2003.

[53] Martin J. Mohlenkamp and Mara Cristina Pereyra. Wavelets, Their Friends, andWhat They Can Do for You. European Mathematical Society, 2008.

68 BIBLIOGRAPHY

[54] Gregory M. Nielson, Il-Hong Jung, and Junwon Sung. Haar wavelets over triangu-lar domains with applications to multiresolution models for flow over a sphere. InProceedings of the 8th conference on Visualization ’97, VIS ’97, pages 143–ff., LosAlamitos, CA, USA, 1997. IEEE Computer Society Press.

[55] R. Ohbuchi, A. Mukaiyama, and S. Takahashi. A frequency-domain approach towatermarking 3d shapes. Computer Graphics Forum, 21(3):373–382, 2002.

[56] Ryutarou Ohbuchi, Shigeo Takahashi, Takahiko Miyazawa, and Akio Mukaiyama.Watermarking 3d polygonal meshes in the mesh spectral domain. In No descriptionon Graphics interface 2001, GRIN’01, pages 9–17, Toronto, Ont., Canada, Canada,2001. Canadian Information Processing Society.

[57] Ryan S. Overbeck, Craig Donner, and Ravi Ramamoorthi. Adaptive wavelet render-ing. ACM Trans. Graph., 28(5):140:1–140:12, December 2009.

[58] M. Ovsjanikov, A.M. Bronstein, M.M. Bronstein, and L.J. Guibas. Shape google: acomputer vision approach to isometry invariant shape retrieval. In Computer VisionWorkshops (ICCV Workshops), 2009 IEEE 12th International Conference on, pages320 –327, 27 2009-oct. 4 2009.

[59] Maks Ovsjanikov, Quentin Mrigot, Facundo Mmoli, and Leonidas Guibas. One pointisometric matching with the heat kernel. Computer Graphics Forum, 29(5):1555–1564, 2010.

[60] Maks Ovsjanikov, Jian Sun, and Leonidas Guibas. Global intrinsic symmetries ofshapes. In Proceedings of the Symposium on Geometry Processing, SGP ’08, pages1341–1348, Aire-la-Ville, Switzerland, Switzerland, 2008. Eurographics Associa-tion.

[61] Frederic Payan and Marc Antonini. Temporal wavelet-based compression for 3danimated models. Computers & Graphics, 31(1):77–88, January 2007.

[62] Ulrich Pinkall and Konrad Polthier. Computing discrete minimal surfaces and theirconjugates. Experimental Mathematics, 1(2), 1993.

[63] Dan Raviv, Michael M. Bronstein, Alexander M. Bronstein, and Ron Kimmel. Vol-umetric heat kernel signatures. In Proceedings of the ACM workshop on 3D objectretrieval, 3DOR ’10, pages 39–44, New York, NY, USA, 2010. ACM.

BIBLIOGRAPHY 69

[64] Martin Reuter, Silvia Biasotti, Daniela Giorgi, Giuseppe Patane, and Michela Spagn-uolo. Discrete laplacebeltrami operators for shape analysis and segmentation. Com-puters and Graphics, 33(3):381 – 390, 2009.

[65] Martin Reuter, Franz-Erich Wolter, and Niklas Peinecke. Laplace-spectra as finger-prints for shape matching. In Proceedings of the 2005 ACM symposium on Solid andphysical modeling, SPM ’05, pages 101–106, New York, NY, USA, 2005. ACM.

[66] Martin Reuter, Franz-Erich Wolter, and Niklas Peinecke. Laplace-beltrami spectra as’shape-dna’ of surfaces and solids. Computer-Aided Design, 27(4):342 – 366, 2008.Symposium on Solid and Physical Modeling 2005.

[67] Steve Rosenberg. The Laplacian on a Riemannian Manifold: An Introduction toAnalysis on Manifold. Cambridge University Press, 1997.

[68] Mauro R. Ruggeri, Giuseppe Patane, Michela Spagnuolo, and Dietmar Saupe.Spectral-driven isometry-invariant matching of 3d shapes. Int. J. Comput. Vision,89(2-3):248–265, September 2010.

[69] R. M. Rustamov. On mesh editing, manifold learning, and diffusion wavelets. InProceedings of the 13th IMA International Conference on Mathematics of SurfacesXIII, pages 307–321, Berlin, Heidelberg, 2009. Springer-Verlag.

[70] Raif M. Rustamov. Laplace-beltrami eigenfunctions for deformation invariant shaperepresentation. In Fifth Eurographics Symposium on Geometry Processing, pages225–234, July 2007.

[71] Y. Sahillioglu and Y. Yemez. Coarse-to-fine combinatorial matching for dense iso-metric shape correspondence. Computer Graphics Forum, 30(5):1461–1470, 2011.

[72] Peter Schroder and Wim Sweldens. Spherical wavelets: efficiently representing func-tions on the sphere. In Proceedings of the 22nd annual conference on Computergraphics and interactive techniques, SIGGRAPH ’95, pages 161–172, New York,NY, USA, 1995. ACM.

[73] Zhefu Shi, Cory Beard, and Ken Mitchell. Competition, cooperation, and optimiza-tion in multi-hop csma networks. In Proceedings of the 8th ACM Symposium onPerformance evaluation of wireless ad hoc, sensor, and ubiquitous networks, PE-WASUN ’11, pages 117–120, New York, NY, USA, 2011. ACM.

[74] O Sorkine. Laplacian mesh processing. In Eurographics State-of-the Art Report,2005.

70 BIBLIOGRAPHY

[75] O. Sorkine, D. Cohen-Or, Y. Lipman, M. Alexa, C. Rossl, and H.-P. Seidel. Laplaciansurface editing. In Proceedings of the 2004 Eurographics/ACM SIGGRAPH sympo-sium on Geometry processing, SGP ’04, pages 175–184, New York, NY, USA, 2004.ACM.

[76] Olga Sorkine, Daniel Cohen-Or, and Sivan Toledo. High-pass quantization for meshencoding. In Proceedings of the 2003 Eurographics/ACM SIGGRAPH symposiumon Geometry processing, SGP ’03, pages 42–51, Aire-la-Ville, Switzerland, Switzer-land, 2003. Eurographics Association.

[77] Jian Sun, Maks Ovsjanikov, and Leonidas Guibas. A concise and provably informa-tive multi-scale signature based on heat diffusion. Computer Graphics Forum, 28(5),2009.

[78] Roberto Toldo, Umberto Castellani, and Andrea Fusiello. The bag of words ap-proach for retrieval and categorization of 3d objects. Vis. Com-put., 26(10):1257–1268, October 2010.

[79] S. Valette and P. Prost. Wavelet-based multiresolution analysis of irregular surfacemeshes. Visualization and Computer Graphics, IEEE Transactions on, 10(2):113–122, march-april 2004.

[80] B. Vallet and B. Levy. Spectral geometry processing with manifold harmonics. Com-puter Graphics Forum, 27(2):251–260, 2008.

[81] Amir Vaxman, Mirela Ben-Chen, and Craig Gotsman. A multi-resolution approachto heat kernels on discrete surfaces. ACM Trans. Graph., 29:121:1–121:10, July2010.

[82] Huawei Wang, Kaihuai Qin, and Hanqiu Sun.√

3-subdivision-based biorthogonalwavelets. Visualization and Computer Graphics, IEEE Transactions on, 13(5):914–925, sept.-oct. 2007.

[83] Max Wardetzky, Saurabh Mathur, Felix Kalberer, and Eitan Grinspun. Discretelaplace operators: no free lunch. In Proceedings of the fifth Eurographics sympo-sium on Geometry processing, SGP ’07, pages 33–37, Aire-la-Ville, Switzerland,Switzerland, 2007. Eurographics Association.

[84] Guoliang Xu. Convergent discrete laplace-beltrami operators over triangular sur-faces. In Geometric Modeling and Processing, 2004. Proceedings, pages 195 – 204,2004.

harmonic shape analysis: from fourier to wavelets

Documents