![Page 1: STATISTICAL ANALYSIS OF SHAPES OF 2D CURVES, 3D …STATISTICAL ANALYSIS OF SHAPES OF 2D CURVES, 3D CURVES & FACIAL SURFACES Anuj Srivastava Department of Statistics Florida State University](https://reader034.vdocuments.net/reader034/viewer/2022043000/5f779d2ef8354e60cc0aa2eb/html5/thumbnails/1.jpg)
STATISTICAL ANALYSIS OF SHAPES OF 2D CURVES, 3DCURVES & FACIAL SURFACES
Anuj SrivastavaDepartment of Statistics
Florida State University
This research supported in part by ARO and NSF.
Presented at IMA Workshop on Shape Spaces, April 4th, 2006.
![Page 2: STATISTICAL ANALYSIS OF SHAPES OF 2D CURVES, 3D …STATISTICAL ANALYSIS OF SHAPES OF 2D CURVES, 3D CURVES & FACIAL SURFACES Anuj Srivastava Department of Statistics Florida State University](https://reader034.vdocuments.net/reader034/viewer/2022043000/5f779d2ef8354e60cc0aa2eb/html5/thumbnails/2.jpg)
COLLABORATORS
• Mathematics: Eric Klassen, Washington Mio, Dave Wilson (Univ of
Florida)
• Computer Science: Mohamed Daoudi (ENIC, France), Chafik Samir
(grad, ENIC, France), Aastha Jain (undergrad, IIT Delhi)
• Statistics: Dave Kaziska (AFIT), Sanjay Saini (grad)
• Electrical Engineering: Shantanu Joshi (grad)
• Neuroscience: Zhaohua Ding and Chunming Li (Vanderbilt Inst. of
Imag. Sc.)
![Page 3: STATISTICAL ANALYSIS OF SHAPES OF 2D CURVES, 3D …STATISTICAL ANALYSIS OF SHAPES OF 2D CURVES, 3D CURVES & FACIAL SURFACES Anuj Srivastava Department of Statistics Florida State University](https://reader034.vdocuments.net/reader034/viewer/2022043000/5f779d2ef8354e60cc0aa2eb/html5/thumbnails/3.jpg)
TENETS OF GRENANDER’S PATTERN THEORY
General Pattern Theory, Oxford University Press, 1993, In Introduction:
1. Create representations in terms of algebraic systems with probabilistic
superstructures.
What probabilistic superstructures should we have on our shape spaces?
2. Analyze structures from the perspective of ....statistical inference.
How can we generate inferences on such nonlinear, high-dimensional
space?
3. Apply ... to particular applications.
Are we there yet?
![Page 4: STATISTICAL ANALYSIS OF SHAPES OF 2D CURVES, 3D …STATISTICAL ANALYSIS OF SHAPES OF 2D CURVES, 3D CURVES & FACIAL SURFACES Anuj Srivastava Department of Statistics Florida State University](https://reader034.vdocuments.net/reader034/viewer/2022043000/5f779d2ef8354e60cc0aa2eb/html5/thumbnails/4.jpg)
TOPICS COVERED
• Closed Curves in R2
– Geodesics Using Shooting Method
– Statistics, Bayesian Analysis
– Stochastic Processes on Shape Spaces
• Closed Curves in R3
– Geodesics Using Path-Straightening Method
• Shapes of Facial Surfaces
– Representations Using Facial Curves
– Face Recognition
• Summary
![Page 5: STATISTICAL ANALYSIS OF SHAPES OF 2D CURVES, 3D …STATISTICAL ANALYSIS OF SHAPES OF 2D CURVES, 3D CURVES & FACIAL SURFACES Anuj Srivastava Department of Statistics Florida State University](https://reader034.vdocuments.net/reader034/viewer/2022043000/5f779d2ef8354e60cc0aa2eb/html5/thumbnails/5.jpg)
PAST WORK: PLANAR CURVES
1. Representations of curves: Assuming arc-length parametrization.
0 1 2 3 4 5 6 7−2
−1
0
1
2
3
4
5
6
7
Original Shape Angle Function θ
0 1 2 3 4 5 6 7−30
−20
−10
0
10
20
30
0 1 2 3 4 5 6 7−1.5
−1
−0.5
0
0.5
1
1.5
2
Curvature Function Coordinate Functions
C = {θ| 1
2π
Z 2π
0
θ(s)ds = π,
Z 2π
0
cos(θ(s))ds = 0,
Z 2π
0
sin(θ(s))ds = 0}
![Page 6: STATISTICAL ANALYSIS OF SHAPES OF 2D CURVES, 3D …STATISTICAL ANALYSIS OF SHAPES OF 2D CURVES, 3D CURVES & FACIAL SURFACES Anuj Srivastava Department of Statistics Florida State University](https://reader034.vdocuments.net/reader034/viewer/2022043000/5f779d2ef8354e60cc0aa2eb/html5/thumbnails/6.jpg)
2. Shape-preserving transformations:
Automatically: Rotation, translation, and scaling.
Manually: Placement of origin, i.e. registration group is S1.
0 10 20 30 40 50 60 70 80 90 100−2
−1
0
1
2
3
4
5
6
7
8
0 10 20 30 40 50 60 70 80 90 100−1
0
1
2
3
4
5
6
7
Shape space S = C/S1.
3. Metric: L2 metric on angle functions
For g, h ∈ Tθ(S), we define:
〈g, h〉 =∫ 2π
0
g(s)h(s)ds .
![Page 7: STATISTICAL ANALYSIS OF SHAPES OF 2D CURVES, 3D …STATISTICAL ANALYSIS OF SHAPES OF 2D CURVES, 3D CURVES & FACIAL SURFACES Anuj Srivastava Department of Statistics Florida State University](https://reader034.vdocuments.net/reader034/viewer/2022043000/5f779d2ef8354e60cc0aa2eb/html5/thumbnails/7.jpg)
4. Geodesics: Constructed numerically using shooting method. Paths
correspond to minimum bending energy between shapes.
0 2 4 6 8 10 12
0
1
2
0 2 4 6 8 10 12
0
1
2
5. References : Klassen et al., (Asilomar Conference (2002), EMMCVPR
(2003), and IEEE PAMI (March 2004)).
![Page 8: STATISTICAL ANALYSIS OF SHAPES OF 2D CURVES, 3D …STATISTICAL ANALYSIS OF SHAPES OF 2D CURVES, 3D CURVES & FACIAL SURFACES Anuj Srivastava Department of Statistics Florida State University](https://reader034.vdocuments.net/reader034/viewer/2022043000/5f779d2ef8354e60cc0aa2eb/html5/thumbnails/8.jpg)
PAST WORK: ELASTIC PLANAR CURVES
1. Representations: Shapes are now represented by a pair of functions:
log-speed (φ) and angle (θ). eφ(s) provides speed of traversal along the
curve.
∫exp(φ)
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
C = {(φ, θ)| 1
2π
Z 2π
0
θ(s)eφ(s)ds = π,
Z 2π
0
cos(θ(s))eφ(s)ds = 0,
Z 2π
0
sin(θ(s))eφ(s)ds = 0}
![Page 9: STATISTICAL ANALYSIS OF SHAPES OF 2D CURVES, 3D …STATISTICAL ANALYSIS OF SHAPES OF 2D CURVES, 3D CURVES & FACIAL SURFACES Anuj Srivastava Department of Statistics Florida State University](https://reader034.vdocuments.net/reader034/viewer/2022043000/5f779d2ef8354e60cc0aa2eb/html5/thumbnails/9.jpg)
2. Shape-preserving transformations:
Automatic: Rotation, translation, and scaling.
Manually: (i) Placement of origin. (ii) Variable-speed parametrization, i.e.
re-parametrization group D: for γ ∈ D,
(φ, θ) ◦ γ = (φ ◦ γ + log(γ′), θ ◦ γ) .Shape space S = C/(S1 ×D).
Dynamic programming is used to match shapes.
3. Metric: L2 metric on angle and log-speed functions
For (g1, h1), (g2, h2) ∈ T(φ,θ)(S), we define:
〈(g1, h1, (g2, h2)〉 = a
∫ 2π
0
g1(s)g2(s)eφ(s)ds+b∫ 2π
0
h1(s)h2(s)eφ(s)ds .
![Page 10: STATISTICAL ANALYSIS OF SHAPES OF 2D CURVES, 3D …STATISTICAL ANALYSIS OF SHAPES OF 2D CURVES, 3D CURVES & FACIAL SURFACES Anuj Srivastava Department of Statistics Florida State University](https://reader034.vdocuments.net/reader034/viewer/2022043000/5f779d2ef8354e60cc0aa2eb/html5/thumbnails/10.jpg)
4. Geodesics: Constructed numerically using shooting method. Paths
correspond to optimal bending and stretching/compression between
shapes.
Top row: non-elastic representation. Bottom row: elastic representation.
5. References : Mio et al., (CVPR 2004, IJCV in review).
![Page 11: STATISTICAL ANALYSIS OF SHAPES OF 2D CURVES, 3D …STATISTICAL ANALYSIS OF SHAPES OF 2D CURVES, 3D CURVES & FACIAL SURFACES Anuj Srivastava Department of Statistics Florida State University](https://reader034.vdocuments.net/reader034/viewer/2022043000/5f779d2ef8354e60cc0aa2eb/html5/thumbnails/11.jpg)
SHAPE CLUSTERING
613 21
2344
7 14 24 25 36 39
918
2242
10 16 19 30 3345 47 48
1117
28 31
34 37 41 49
8 1526
3240
434612 20 27 29 35 38
50
1 2 3 4 5
Minimization of cumulative dispersion within classes using Metropolis-based
simulated annealing
References : Joshi et al., (SSP 2003, ECCV 2004, IEEE PAMI April 2005).
![Page 12: STATISTICAL ANALYSIS OF SHAPES OF 2D CURVES, 3D …STATISTICAL ANALYSIS OF SHAPES OF 2D CURVES, 3D CURVES & FACIAL SURFACES Anuj Srivastava Department of Statistics Florida State University](https://reader034.vdocuments.net/reader034/viewer/2022043000/5f779d2ef8354e60cc0aa2eb/html5/thumbnails/12.jpg)
STATISTICS OF ELASTIC SHAPES
Examples of observed shapes
Intrinsic definition of mean shape: d(·, ·) denotes geodesic lengths.
µ = argminθ∈S
∫Sd(θ, φ)2f(φ)dφ .
![Page 13: STATISTICAL ANALYSIS OF SHAPES OF 2D CURVES, 3D …STATISTICAL ANALYSIS OF SHAPES OF 2D CURVES, 3D CURVES & FACIAL SURFACES Anuj Srivastava Department of Statistics Florida State University](https://reader034.vdocuments.net/reader034/viewer/2022043000/5f779d2ef8354e60cc0aa2eb/html5/thumbnails/13.jpg)
TANGENT PRINCIPAL COMPONENTS (TPCA)
• Let Tµ(S) be the space of vectors tangent to S at µ ∈ S . Geodesics
are used to map shapes from S to Tµ(S).
• S is a nonlinear manifold but Tµ(S) is a vector space. Derive probability
models in this space. For example, use PCA to reduce dimension and
impose models on coefficients.
![Page 14: STATISTICAL ANALYSIS OF SHAPES OF 2D CURVES, 3D …STATISTICAL ANALYSIS OF SHAPES OF 2D CURVES, 3D CURVES & FACIAL SURFACES Anuj Srivastava Department of Statistics Florida State University](https://reader034.vdocuments.net/reader034/viewer/2022043000/5f779d2ef8354e60cc0aa2eb/html5/thumbnails/14.jpg)
PROBABILITY MODELS ON TPCs
1. Non-parametric models:
• Let f(1)j , j = 1, . . . , d be the density estimate of aj , the jth TPC of
the shape α.
• Assuming independence of TPCs, we obtain:
f (1)(α) =∏dj=1 fj(aj)
2. Gaussian Models:
• Let Σ ∈ Rd×d be the diagonal matrix in SVD of K .
• The Gaussian model is given as, f (2)(α) =∏dj=1 h(aj ; 0,Σjj),
where h(y; z, σ2) ≡ 1√2πσ2 exp(−(y − z)2/(2σ2))
3. Gaussian Mixture models:
Define the mixture density
f (3)j(α) =
∏dj=1
(∑Kk=1 pkh(aj ; zk, σ
2k)
), and
∑k pk = 1.
Estimate f (3) using EM algorithm
![Page 15: STATISTICAL ANALYSIS OF SHAPES OF 2D CURVES, 3D …STATISTICAL ANALYSIS OF SHAPES OF 2D CURVES, 3D CURVES & FACIAL SURFACES Anuj Srivastava Department of Statistics Florida State University](https://reader034.vdocuments.net/reader034/viewer/2022043000/5f779d2ef8354e60cc0aa2eb/html5/thumbnails/15.jpg)
• Density Estimates:
−1 −0.8 −0.6 −0.4 −0.2 0 0.2 0.4 0.6 0.80
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
−0.5 −0.4 −0.3 −0.2 −0.1 0 0.1 0.2 0.3 0.40
0.5
1
1.5
2
2.5
−0.8 −0.6 −0.4 −0.2 0 0.2 0.4 0.6 0.80
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
1.8
−0.5 −0.4 −0.3 −0.2 −0.1 0 0.1 0.2 0.3 0.40
0.5
1
1.5
2
2.5
−0.6 −0.5 −0.4 −0.3 −0.2 −0.1 0 0.1 0.2 0.30
1
2
3
4
5
6
−0.5 −0.4 −0.3 −0.2 −0.1 0 0.1 0.20
1
2
3
4
5
6
−0.5 −0.4 −0.3 −0.2 −0.1 0 0.1 0.2 0.3 0.4 0.50
0.5
1
1.5
2
2.5
3
3.5
4
4.5
−0.4 −0.3 −0.2 −0.1 0 0.1 0.2 0.3 0.40
0.5
1
1.5
2
2.5
3
3.5
4
4.5
−0.4 −0.3 −0.2 −0.1 0 0.1 0.2 0.3 0.4 0.50
0.5
1
1.5
2
2.5
3
3.5
4
−0.2 −0.1 0 0.1 0.2 0.3 0.4 0.50
1
2
3
4
5
6
−0.4 −0.3 −0.2 −0.1 0 0.1 0.2 0.3 0.40
0.5
1
1.5
2
2.5
3
3.5
4
−0.2 −0.15 −0.1 −0.05 0 0.05 0.1 0.15 0.2 0.250
1
2
3
4
5
6
Dogs Pears Mugs
• Likelihood Ratio Tests: Gaussian mixture outperforms others.
f(1)(α)f(3)(α)
5 10 15 20 25 30 35 400
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
5 10 15 20 25 30 35 400
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
5 10 15 20 25 30 35 400
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
f(3)(α)f(2)(α)
5 10 15 20 25 30 35 400
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
5 10 15 20 25 30 35 400
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
5 10 15 20 25 30 35 400
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
• References : Srivastava et al. (PAMI, April 2005, ACCV 2006).
![Page 16: STATISTICAL ANALYSIS OF SHAPES OF 2D CURVES, 3D …STATISTICAL ANALYSIS OF SHAPES OF 2D CURVES, 3D CURVES & FACIAL SURFACES Anuj Srivastava Department of Statistics Florida State University](https://reader034.vdocuments.net/reader034/viewer/2022043000/5f779d2ef8354e60cc0aa2eb/html5/thumbnails/16.jpg)
BAYESIAN SHAPE ESTIMATION
MAP estimate of a closed curve, under a given prior model, by minimizing:
Etotal(α, I) = Eimage(α, I) + Esmooth(α) + Eprior(θ) .
Image I GVF of I Prior Mean
Evolution of curve under a gradient flow.
Reference: Joshi et al. (ECCV Workshop, 2006)
![Page 17: STATISTICAL ANALYSIS OF SHAPES OF 2D CURVES, 3D …STATISTICAL ANALYSIS OF SHAPES OF 2D CURVES, 3D CURVES & FACIAL SURFACES Anuj Srivastava Department of Statistics Florida State University](https://reader034.vdocuments.net/reader034/viewer/2022043000/5f779d2ef8354e60cc0aa2eb/html5/thumbnails/17.jpg)
BAYESIAN SHAPE COMPLETION
Top row: Given images; Middle Row: Estimated shape using our approach;
Bottom row: Human drawing
![Page 18: STATISTICAL ANALYSIS OF SHAPES OF 2D CURVES, 3D …STATISTICAL ANALYSIS OF SHAPES OF 2D CURVES, 3D CURVES & FACIAL SURFACES Anuj Srivastava Department of Statistics Florida State University](https://reader034.vdocuments.net/reader034/viewer/2022043000/5f779d2ef8354e60cc0aa2eb/html5/thumbnails/18.jpg)
EXTRACTION OF CARDIAC CURVES IN ULTRASOUND
(With Dave Wilson, UF)
ECG images of heart with expert generated contours for cardiac layers
overlaid.
![Page 19: STATISTICAL ANALYSIS OF SHAPES OF 2D CURVES, 3D …STATISTICAL ANALYSIS OF SHAPES OF 2D CURVES, 3D CURVES & FACIAL SURFACES Anuj Srivastava Department of Statistics Florida State University](https://reader034.vdocuments.net/reader034/viewer/2022043000/5f779d2ef8354e60cc0aa2eb/html5/thumbnails/19.jpg)
Simply compute a geodesic path between the given shapes and overlay
them on the intermediate images.
References : Joshi et al. (IPMI, 2005).
![Page 20: STATISTICAL ANALYSIS OF SHAPES OF 2D CURVES, 3D …STATISTICAL ANALYSIS OF SHAPES OF 2D CURVES, 3D CURVES & FACIAL SURFACES Anuj Srivastava Department of Statistics Florida State University](https://reader034.vdocuments.net/reader034/viewer/2022043000/5f779d2ef8354e60cc0aa2eb/html5/thumbnails/20.jpg)
Use the Bayesian approach to improve upon the estimation.
References : Joshi et al. (ECCV Workshop, 2006).
![Page 21: STATISTICAL ANALYSIS OF SHAPES OF 2D CURVES, 3D …STATISTICAL ANALYSIS OF SHAPES OF 2D CURVES, 3D CURVES & FACIAL SURFACES Anuj Srivastava Department of Statistics Florida State University](https://reader034.vdocuments.net/reader034/viewer/2022043000/5f779d2ef8354e60cc0aa2eb/html5/thumbnails/21.jpg)
BAYESIAN SHAPE MODEL SELECTION
Approach: Shape is considered as nuisance variable in recognition
(Grenander et al. IEEE IT 2001):
....
...
0
5
10
15
20
25
30
35
40
models
E total
q1 q
3 q
5 q
7 q
9 q
4r q
6r q
8r q
2r
![Page 22: STATISTICAL ANALYSIS OF SHAPES OF 2D CURVES, 3D …STATISTICAL ANALYSIS OF SHAPES OF 2D CURVES, 3D CURVES & FACIAL SURFACES Anuj Srivastava Department of Statistics Florida State University](https://reader034.vdocuments.net/reader034/viewer/2022043000/5f779d2ef8354e60cc0aa2eb/html5/thumbnails/22.jpg)
CYCLOSTATIONARY PROCESSES ON SHAPE SPACES
• Representation of human gait as a stochastic process X(t) on C.
• Cyclo-stationary process: statistics are periodic
• Metric for comparing gaits:
dp(X,Y ) = argminκ∈[0,τy ],ψ
(∫ τx
0
d(E[X(t)], E[Y (κ+ ψ(t))])2dt),
![Page 23: STATISTICAL ANALYSIS OF SHAPES OF 2D CURVES, 3D …STATISTICAL ANALYSIS OF SHAPES OF 2D CURVES, 3D CURVES & FACIAL SURFACES Anuj Srivastava Department of Statistics Florida State University](https://reader034.vdocuments.net/reader034/viewer/2022043000/5f779d2ef8354e60cc0aa2eb/html5/thumbnails/23.jpg)
– E[·] is the expectation operator.
– κ, ψ are used to register the two gait cycles. ψ is solved using dynamic
programming.
– d(·, ·)2 is the elastic geodesic distance.
– Detection of gait cycles is automatic.
• Experimental Results: 26 subjects, different test and training data
i nearest neighbor 1 2 3 4 5 6 7 8 9 10
Elastic Shape Model 17 20 21 21 21 22 22 23 24 24
Mean-Shape Approach 13 14 16 19 19 20 20 20 20 22
Landmark-Based Approach 10 11 14 17 19 19 20 20 21 22
• References: Kaziska et al (PhD Thesis 2005, ECCV 2006).
![Page 24: STATISTICAL ANALYSIS OF SHAPES OF 2D CURVES, 3D …STATISTICAL ANALYSIS OF SHAPES OF 2D CURVES, 3D CURVES & FACIAL SURFACES Anuj Srivastava Department of Statistics Florida State University](https://reader034.vdocuments.net/reader034/viewer/2022043000/5f779d2ef8354e60cc0aa2eb/html5/thumbnails/24.jpg)
3D FACE RECOGNITION
• We want to statistically analyze shapes of facial surfaces, i.e. define
geodesics, means, covariances, etc.
• Not as simple as curves, as there is no natural ordering of points.
• We chose a specific parametrization of facial surfaces.
![Page 25: STATISTICAL ANALYSIS OF SHAPES OF 2D CURVES, 3D …STATISTICAL ANALYSIS OF SHAPES OF 2D CURVES, 3D CURVES & FACIAL SURFACES Anuj Srivastava Department of Statistics Florida State University](https://reader034.vdocuments.net/reader034/viewer/2022043000/5f779d2ef8354e60cc0aa2eb/html5/thumbnails/25.jpg)
3D FACE RECOGNITION USING FACIAL CURVES
• Assume that gaze direction is aligned with the z-direction. Let F be the
depth function on a facial surface.
• Let cλ be the level-curve of F for value λ. It has a shape component
and other variables (scale, rotation, etc).
• A surface S is represented by a path, parameterized by λ, on C(ignoring level curves that are not simple, closed curves).
![Page 26: STATISTICAL ANALYSIS OF SHAPES OF 2D CURVES, 3D …STATISTICAL ANALYSIS OF SHAPES OF 2D CURVES, 3D CURVES & FACIAL SURFACES Anuj Srivastava Department of Statistics Florida State University](https://reader034.vdocuments.net/reader034/viewer/2022043000/5f779d2ef8354e60cc0aa2eb/html5/thumbnails/26.jpg)
• Examples:
• Metric: d(S1, S2) =∫λd(c1λ, c
2λ)dλ.
• Recognition rates: ∼ 60 people, 360 scans
1 1.5 2 2.5 3 3.5 4 4.5 530
40
50
60
70
80
90
Facial expressions per class used in training data sets
Rec
ogni
tion
Rat
e %
C5EucC4EucC53ucC2EucC1Euc
1 1.5 2 2.5 3 3.5 4 4.5 530
40
50
60
70
80
90
100
Facial expressions per class used in training data sets
Rec
ogni
tion
Rat
e %
C5GeoC4GeoC3GeoC2GeoC1Geo
1 1.5 2 2.5 3 3.5 4 4.5 560
65
70
75
80
85
90
Number of facial curves
Rec
ogni
tion
Rat
e %
C42Geo
C42Euc
• References: Samir et al (ICASSP 2006, PAMI in review).
![Page 27: STATISTICAL ANALYSIS OF SHAPES OF 2D CURVES, 3D …STATISTICAL ANALYSIS OF SHAPES OF 2D CURVES, 3D CURVES & FACIAL SURFACES Anuj Srivastava Department of Statistics Florida State University](https://reader034.vdocuments.net/reader034/viewer/2022043000/5f779d2ef8354e60cc0aa2eb/html5/thumbnails/27.jpg)
INVARIANT FACIAL CURVES
• Define a function F (x) = d(x, p) where p is the tip of nose, and d(·, ·)is the geodesic distance on a facial surface. (Bronstein et al.IJCV 2005).
• Define facial curves to be the level curves of F .
• Compare surfaces by comparing their corresponding facial curves.
Facial curves are now closed curves in R3.
![Page 28: STATISTICAL ANALYSIS OF SHAPES OF 2D CURVES, 3D …STATISTICAL ANALYSIS OF SHAPES OF 2D CURVES, 3D CURVES & FACIAL SURFACES Anuj Srivastava Department of Statistics Florida State University](https://reader034.vdocuments.net/reader034/viewer/2022043000/5f779d2ef8354e60cc0aa2eb/html5/thumbnails/28.jpg)
REPRESENTATION OF CLOSED CURVES in R3
• Let p : [0, 2π) �→ R3 be a C1-curve of length 2π,
parameterized by the arc length (can be relaxed as earlier).
• For v(s) ≡ p(s) ∈ R3, we have ‖v(s)‖ = 1 for all s ∈ [0, 2π).
• The function v is called the direction function of p and itself can be
viewed as a curve on the unit sphere S2, i.e. v : [0, 2π) �→ S
2.
![Page 29: STATISTICAL ANALYSIS OF SHAPES OF 2D CURVES, 3D …STATISTICAL ANALYSIS OF SHAPES OF 2D CURVES, 3D CURVES & FACIAL SURFACES Anuj Srivastava Department of Statistics Florida State University](https://reader034.vdocuments.net/reader034/viewer/2022043000/5f779d2ef8354e60cc0aa2eb/html5/thumbnails/29.jpg)
REPRESENTATION OF CLOSED CURVES
• P be the set of all continuous maps from [0, 2π) → S2.
• Since we are interested in the shapes of closed curves, and we establish
that subset as follows. Define a map φ : P �→ R3 by
φ(v) =∫ 2π
0v(s)ds, and define
C = φ−1(0) ≡ {v ∈ P|φ(v) = 0} ⊂ P .
• Elements of C denote closed curves in R3.
• Inner product on Tv(C): for f, g ∈ Tv(C),
〈f, g〉 =∫ 2π
0
(f(s) · g(s))ds .
![Page 30: STATISTICAL ANALYSIS OF SHAPES OF 2D CURVES, 3D …STATISTICAL ANALYSIS OF SHAPES OF 2D CURVES, 3D CURVES & FACIAL SURFACES Anuj Srivastava Department of Statistics Florida State University](https://reader034.vdocuments.net/reader034/viewer/2022043000/5f779d2ef8354e60cc0aa2eb/html5/thumbnails/30.jpg)
GEODESICS: A PATH STRAIGHTENING APPROACH
• Define Space: Let C be the space of all closed curves in R3. Let H be
the set of all paths in C, parameterized by t ∈ [0, 1].
• Initialize Path: For any two curves, denoted by v0 and v1, let α be a path
connecting them in C. That is, α : [0, 1] �→ C, such that α(0) = v0
and α(1) = v1.
• Path Energy: Let H0 be all such paths between v0 and v1. Define an
energy E on H0 whose critical point (gradient is zero) is a geodesic
path.
• Path-Straightening: Update α according to (negative) gradient of E until
convergence.
• Shape Space: Define a shape space S = C/T , and construct
geodesics in S .
![Page 31: STATISTICAL ANALYSIS OF SHAPES OF 2D CURVES, 3D …STATISTICAL ANALYSIS OF SHAPES OF 2D CURVES, 3D CURVES & FACIAL SURFACES Anuj Srivastava Department of Statistics Florida State University](https://reader034.vdocuments.net/reader034/viewer/2022043000/5f779d2ef8354e60cc0aa2eb/html5/thumbnails/31.jpg)
PATH ENERGY AND ITS GRADIENT
• For a path α on H, it path energy is given by:
E(α) ≡ 12
∫ 1
0
⟨dα
dt,dα
dt
⟩dt .
(A critical point of E is a geodesic on C.)
• Goal: Our goal is to find the minimizer:
α = argminα∈H0
E(α) .
We will use a gradient approach to find this minimizer.
• To make H a Riemannian manifold, we use the Palais metric: for w1,
w2 ∈ Tα(H),
〈〈w1, w2〉〉 = 〈w1(0), w2(0)〉 +∫ 1
0
⟨Dw1
dt(t),
Dw2
dt(t)
⟩dt ,
![Page 32: STATISTICAL ANALYSIS OF SHAPES OF 2D CURVES, 3D …STATISTICAL ANALYSIS OF SHAPES OF 2D CURVES, 3D CURVES & FACIAL SURFACES Anuj Srivastava Department of Statistics Florida State University](https://reader034.vdocuments.net/reader034/viewer/2022043000/5f779d2ef8354e60cc0aa2eb/html5/thumbnails/32.jpg)
Theorem 1 Let α : [0, 1] �→ C be a path such that α ∈ H0. Then, with
respect to the Palais metric:
1. The gradient of the energy function E on H is the vector field q along α
satisfying q(0) = 0 and Dqdt = dα
dt . That is, q is a covariant integral ofdαdt .
2. The gradient of the energy function E restricted to H0 is
w(t) = q(t) − tq(t), where q is the vector field defined in the previous
item, and q is the vector field obtained by parallel translating q(1)backwards along α.
At the critical point, we have q(t) = tq(t). Then, q(t) is covariantly
constant, q(t) = tq(t) is covariantly linear, and hence, dαdt is covariantly
constant.
References: Klassen et al (ECCV 2006, SIAM J. Computation, in review).
![Page 33: STATISTICAL ANALYSIS OF SHAPES OF 2D CURVES, 3D …STATISTICAL ANALYSIS OF SHAPES OF 2D CURVES, 3D CURVES & FACIAL SURFACES Anuj Srivastava Department of Statistics Florida State University](https://reader034.vdocuments.net/reader034/viewer/2022043000/5f779d2ef8354e60cc0aa2eb/html5/thumbnails/33.jpg)
EXPERIMENTAL RESULTS
Experiment 1: Let the two curves of interest be:
p1(t) = (a cos(t), b sin(t), c√b2 − a2 sin2(t))
p2(t) = (a(1 + cos(t)), sin(t), 2 sin(t/2))
−1
−0.5
0
−1
−0.5
0
0.5
1
−0.15−0.1
−0.050
−1.5
−1
−0.5
0
−0.5
0
0.5
0
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
1 1.5 2 2.5 3 3.5 4 4.5 5
3.45
3.5
3.55
3.6
![Page 34: STATISTICAL ANALYSIS OF SHAPES OF 2D CURVES, 3D …STATISTICAL ANALYSIS OF SHAPES OF 2D CURVES, 3D CURVES & FACIAL SURFACES Anuj Srivastava Department of Statistics Florida State University](https://reader034.vdocuments.net/reader034/viewer/2022043000/5f779d2ef8354e60cc0aa2eb/html5/thumbnails/34.jpg)
Geodesic Path:
0
2
4
6
8
10
12
14
16
−2
0
2
−1
0
1
02
46
810
1214
16
−2
−1
0
1
2
−101
![Page 35: STATISTICAL ANALYSIS OF SHAPES OF 2D CURVES, 3D …STATISTICAL ANALYSIS OF SHAPES OF 2D CURVES, 3D CURVES & FACIAL SURFACES Anuj Srivastava Department of Statistics Florida State University](https://reader034.vdocuments.net/reader034/viewer/2022043000/5f779d2ef8354e60cc0aa2eb/html5/thumbnails/35.jpg)
Experiment 2
0
0.2
0.4
0.6
0.8
1
1.2−0.2
0
0.2
0.4
0.6
0.8
1
−1
−0.5
0
−0.2
0
0.2
0.4
0.6
0.8
1
1.2 0
0.2
0.4
0.6
0.8
1
1.2
−0.8−0.6−0.4−0.2
00.2
0 2 4 6 8 10 12 1411
12
13
14
15
16
17
18
19
20
21
0 2 4 6 8 10 12 14 16
−2
−1
0
0 2 4 6 8 10 12 14 16
−1
0
1
2
![Page 36: STATISTICAL ANALYSIS OF SHAPES OF 2D CURVES, 3D …STATISTICAL ANALYSIS OF SHAPES OF 2D CURVES, 3D CURVES & FACIAL SURFACES Anuj Srivastava Department of Statistics Florida State University](https://reader034.vdocuments.net/reader034/viewer/2022043000/5f779d2ef8354e60cc0aa2eb/html5/thumbnails/36.jpg)
FACIAL CURVES
Very preliminary:
−0.3
−0.2
−0.1
0
0.1
0.2
0.3−0.4
−0.3
−0.2
−0.1
0
0.1
0.2
0.3
0.40.05
0.1
0.15
0.2
0.25
0.3
0.35
−0.4−0.2
00.2
0.4 −0.4
−0.2
0
0.2
0.4
0.1
0.15
0.2
0.25
0.3
0.35
0
2
4
6
8
10
12
−3
−2
−1
0
1
−1
0
1
![Page 37: STATISTICAL ANALYSIS OF SHAPES OF 2D CURVES, 3D …STATISTICAL ANALYSIS OF SHAPES OF 2D CURVES, 3D CURVES & FACIAL SURFACES Anuj Srivastava Department of Statistics Florida State University](https://reader034.vdocuments.net/reader034/viewer/2022043000/5f779d2ef8354e60cc0aa2eb/html5/thumbnails/37.jpg)
SUMMARY
• Motivated of analysis of shapes of curves.
• Presented a representation of planar shapes that is invariant to rotation,
translation, and scaling. Showed examples of computing geodesic paths
in shape spaces.
• Presented examples of statistical analysis of shapes in R2, and imposed
probability models on shapes using TPCA.
• Demonstrated an application of probability models in Bayesian shape
extraction.
• Shapes of curves in R3: mentioned a path-straightening mechanism for
constructing geodesics.
• Used analysis of 3D curves to propose a method for studying shapes of
facial surfaces.