recovering articulated object models from 3d range data dragomir anguelov daphne koller hoi-cheung...
Post on 22-Dec-2015
221 views
TRANSCRIPT
Recovering Articulated Object Models from 3D Range Data
Dragomir Anguelov Daphne Koller Hoi-Cheung Pang Praveen Srinivasan
Sebastian Thrun
Computer Science Dept. Stanford University
Problem overview Data:
Articulated object observed in different configurations
3D range surfaces (meshes) obtained by a scanner
Task: Recover the articulated object model1. Register all surfaces2. Cluster the object surface
into rigid parts
3. Recover joints linking the parts
Applications of articulated models
Animation, object detection, scene segmentation
Surface registration Algorithm input: meshes Z0, Z1, … ZN
Pick template mesh X = Z0 Need to recover point-to-point correspondences
between X and instance meshes Zi , i={1, …, N} Any registration algorithm providing can be used
Correlated Correspondences algorithm [Anguelov et al ‘04]
Unsupervised nonrigid registration algorithm Handles large deformations, occlusion (but not clutter)
Algorithm Overview Computes embedding of mesh Z into mesh X
defined as a joint assignment to correspondences Encodes correspondence correlations (minimizing
deformation, preserving geodesic distances) as pairwise potentials in a Markov network over
Uses Loopy-BP to obtain a good joint assignment to all variables
21
3
Clustering into rigid parts:Generative model
Template X
TransformedModel
Instance Z
Transformations
z1
b1
a1
Tx1
y1
aM
xM
yM
zk
bk
…
…
…
…
Part labels
Points
Point labels
Points
j = Part (xj) k = ModelPoint (zk)
Assume values provided by anonrigid registration algorithm
1. Rigid part points share the same rigid transformaition in each instance
rotation translation
2. Each part is a connected region of the template mesh surface
a) Soft contiguity constraints:
If and are connected in X
forb) Hard contiguity constraints:
Each part consists of a single connected component of the surface of X
Clustering into rigid parts: Optimization
Model Summary: Ignoring hard contiguity constraints, generative model
defines a Markov Network over the part labels :
Objective
Iterative Optimization E-step: Given transformations , compute the part labels
maximizing Eq. (1) and satisfying the hard contiguity constraints
M-step: Given part labels , compute transformations
Point in mesh Zi , generated
by template point xj
Clustering into rigid parts: E-Step
The problem of finding a MAP assignment to given which maximizes Eq. (1) can be expressed as an integer program (IP). A relaxation of the IP produces a linear program (LP) [Kleinberg et al ’99].
For 2-part models, the optimal LP solution is also the optimal IP solution.
For n-part models, n>2: In case of a fractional LP solution, a rounding procedure produces an IP
solution within a factor of 2 of the optimal. The LP can be solved very efficiently by using multiway min-
cuts [Kolmogorov et al ’04]
The LP solution may not satisfy hard contiguity constraints. A part consisting of m connected components is split into m
separate parts which doesn’t change the objective in Eq. (1).
Clustering into rigid parts: LP formulation
Introduce variables s.t. Log-cost associated with potential becomes
where
Define separation cost of an edge as where
Putting everything together:
Relax binary constraints to obtain LP
Clustering into rigid parts: M-step and Annealing
M-step: find rigid part transformations given part labels
Optimization function decomposes into a separate equation for
each :
Eq. (2) is isomorphic to the ICP problem [Besl’ 92]. Close form estimate of is obtained by solving a simple LP.
Annealing Number of parts in solution depends on the tradeoff
where large values of result in fewer parts When estimate of is poor and is large, solving E-step LP is
hard, because multiple plausible solutions with similar scores exist
Starting with low values of and gradually increasing performs best in practice
Clustering into Rigid Parts: Initialization
Need good initial values for the transformations
Template X
Method 1: Cluster local rigid transformations [Cheung et al ’03] • Associate a vector of transformations with each point (estimate using the known correspondences for all instance meshes)• Cluster points into k parts using adaptive PCA [Archer et al ’02]• Slow: >10 mins for 2000 point models
Method 2: Break surface into k arbitrary fragments of approximately equal size• Fast: <1s for 2000 point models • Same final results!
a) Rigid parts after initializationb) Rigid parts after EM
a) b)
Optimizing the Joint Positions
A joint is a point belonging to two parts simultaneously: [Cheung et al ’03]
To solve for the joint, must optimize
Sometimes, the solution can be an entire space of points
Example: a joint with one degree of freedom Introduce an additional regularization term,
enforcing that the joint is close to the boundary between two parts in X
a) b)
a) Rigid parts after EMb) Estimated joints
Joint between parts p and q
Results: Initialization
Testing EM Initialization Initialize with a different number of parts k If k > 15, the algorithm consistently finds the correct
number of parts (15)
Results: Arm Dataset
Correctly identifies 4 arm parts and the skeleton in the presence of non-rigid deformation
Results: Annealing
Results of annealing with increasing value of Increasing decreases the number of parts (A – D) Increasing too much results in a partition which tries to
reduce the number of links between parts (E)
Conclusions and Future Work
Algorithm for recovering articulate object models from 3D data
Clusters the object surface into rigid parts Recovers the articulated object skeleton
Algorithm properties Probabilistic formulation Efficient global inference
Algorithm recovers complex models consisting of a large number of parts in real world datasets
Future work Extend the algorithm to allow and explicitly model (small)
deformations of the articulated parts
Robustness to local maxima