selecting distinctive 3d shape descriptors for similarity retrieval
DESCRIPTION
Selecting Distinctive 3D Shape Descriptors for Similarity Retrieval. Philip Shilane and Thomas Funkhouser. Computer Graphics (Princeton Shape Benchmark). Mechanical CAD (National Design Repository). Molecular Biology (Protein Databank). Large Databases of 3D Shapes. Shape Retrieval. - PowerPoint PPT PresentationTRANSCRIPT
Selecting Distinctive 3D Shape Descriptors for Similarity
Retrieval
Philip Shilane and Thomas Funkhouser
Large Databases of 3D Shapes
Mechanical CAD(National Design Repository)
Molecular Biology(Protein Databank)
Computer Graphics(Princeton Shape Benchmark)
Shape Retrieval
3D Model Model
Database
BestMatche
s
Local Matches for Retrieval
3D Model Model
Database
BestMatche
s
Local Matches for Retrieval
3D Model Model
Database
BestMatche
s
i
i YXC ),(
Cost Function
Local Matches for Retrieval
3D Model Model
Database
BestMatche
s
i
i YXC ),(
Cost Function
Using many local descriptors is slow.
Local Matches for Retrieval
3D Model Model
Database
BestMatche
s
i
i YXC ),(
Cost Function
Using many local descriptors is slow.Many descriptors do
not represent distinguishing parts.
Local Matches for Retrieval
3D Model Model
Database
BestMatche
s
i
i YXC ),(
Cost Function
Focusing on the distinctive regions improves retrieval time and accuracy.
Related Work
Selecting Local Descriptors• Random
Mori 2001Frome 2004
Related Work
Selecting Local Descriptors• Random• Salient
Gal 2005Lee 2005Frintrop 2004
Related Work
Selecting Local Descriptors• Random• Salient• Likelihood
Johnson 2000Shan 2004
Distinction = Retrieval Performance
QueryDescriptors
The distinction of each local descriptor is based on how well it retrieves shapes of the correct class.
Retrieval Results
Distinction = Retrieval Performance
QueryDescriptors
The distinct descriptors that distinguish between classes are classification dependent.
Retrieval Results
Approach
Descriptors
Distinction
We want a predicted distinction score for each descriptor on the model.
ApproachWe map descriptors into a 1D space where we learn distinction from a training set.
Dis
tinc
tion
1D Parameterization
Descriptors
Distinction
Approach
Descriptors
Distinction
Likelihood of shape descriptors is a 1D function that groups descriptors with similar distinction.
Likelihood Parameterization
System Overview
Likelihood
RetrievalEvaluation
Training
Query
ShapeDB
LocalDescriptors
DescriptorDB
Likelihood EvaluateDistinction
LocalDescriptors
Classification
Shape
DistinctionFunction
Match
RetrievalList
SelectDescriptors
System Overview
Likelihood
RetrievalEvaluation
Training
Query
ShapeDB
LocalDescriptors
DescriptorDB
Likelihood EvaluateDistinction
LocalDescriptors
Classification
Shape
DistinctionFunction
Match
RetrievalList
SelectDescriptors
System Overview
Likelihood
RetrievalEvaluation
Training
Query
ShapeDB
LocalDescriptors
DescriptorDB
Likelihood EvaluateDistinction
LocalDescriptors
Classification
Shape
DistinctionFunction
Match
RetrievalList
SelectDescriptors
System Overview
Likelihood
RetrievalEvaluation
Training
Query
ShapeDB
LocalDescriptors
DescriptorDB
Likelihood EvaluateDistinction
LocalDescriptors
Classification
Shape
DistinctionFunction
Match
RetrievalList
SelectDescriptors
)(21exp
2)( 1
2
21
xxxdensityt
d
Multi-dimensional normal density [Johnson 2000]
matrix covariance d x d vectorfeaturemean
vectorfeature ldimensiona d
x
Likelihood of Descriptors
Likelihood of Descriptors
)(21)(
))(densityln()(
1
xxxp
xxpt
The likelihood function is proportional to the descriptor density.
matrix covariance d x d vectorfeaturemean
vectorfeature ldimensiona d
x
Map from Descriptors to LikelihoodFlat regions are the most common while wing tips
and the cockpit area are rarer.
Less Likely
More Likely
System Overview
Likelihood
RetrievalEvaluation
Training
Query
ShapeDB
LocalDescriptors
DescriptorDB
Likelihood EvaluateDistinction
LocalDescriptors
Classification
Shape
DistinctionFunction
Match
RetrievalList
SelectDescriptors
Measuring Distinction
0.33
QueryDescriptors
Evaluation Metric
Evaluate the retrieval performance of every query descriptor.
Retrieval Results
Measuring Distinction
0.33
1.0
QueryDescriptors
Evaluation Metric
Some descriptors are better for retrieval than others.
Retrieval Results
System Overview
Likelihood
RetrievalEvaluation
Training
Query
ShapeDB
LocalDescriptors
DescriptorDB
Likelihood EvaluateDistinction
LocalDescriptors
Classification
Shape
DistinctionFunction
Match
RetrievalList
SelectDescriptors
Build Distinction FunctionMeasure likelihood and retrieval performance of each descriptor.
Build Distinction FunctionMeasure likelihood and retrieval performance of each descriptor.
Build Distinction FunctionMeasure likelihood and retrieval performance of each descriptor.
Build Distinction FunctionRetrieval performance is averaged within each likelihood bin.
Descriptor DistinctionA likelihood mapping separates descriptors with different retrieval performance.
Less Likely
More Likely
Less Likely
More Likely
Descriptor DistinctionThe most common features are the worst for retrieval.
Predicting Distinction
Distinction Function
Descriptors
Distinction
The likelihood mapping predicts descriptor distinction.
System Overview
Likelihood
RetrievalEvaluation
Training
Query
ShapeDB
LocalDescriptors
DescriptorDB
Likelihood EvaluateDistinction
LocalDescriptors
Classification
Shape
DistinctionFunction
Match
RetrievalList
SelectDescriptors
Selecting Distinctive DescriptorsThe k most distinctive descriptors with a minimum distance constraint are selected.
Mesh Descriptors DistinctionScores
3 SelectedDescriptors
Matching with Selected Descriptors
k
i
ki
k YXCYX ),(
3D Model Model
Database
BestMatche
s
Results
• Examples of Distinctive Descriptors• Evaluation for Retrieval
Distinctive Descriptor ExamplesDescriptors on the head and neck represent
consistent regions of the models.
Distinctive Descriptor ExamplesDescriptors on the front of the jet are consistent as
opposed to on the wings.
ChallengeThe wheels are consistent features for cars.
Shape Database
• 100 Models in 10 Classes from the Princeton Shape Benchmark
• Models come from different branchesof the hierarchical classification
Shape Descriptors• Mass per Shell Shape Histogram
(SHELLS)Ankerst 1999
• Spherical Harmonics of the Gaussian Euclidean Distance Transform (SHD)
Kazhdan 2003
0.25 0.5 1.0 2.0
Radius of Descriptors Considered
Local vs. Global DescriptorsUsing local descriptors improves retrieval relative to global descriptors.
Global vs Local
0
0.2
0.4
0.6
0.8
1
0 0.2 0.4 0.6 0.8 1
Recall
Prec
isio
n
GlobalAll Local
Focus on Distinctive DescriptorsUsing a small number of distinct descriptors maintains retrieval performance while improving retrieval time.
Global vs Local
0
0.2
0.4
0.6
0.8
1
0 0.2 0.4 0.6 0.8 1
Recall
Prec
isio
n
GlobalAll Local10 Distinct3 Distinct
Alternative Selection Techniques
Selection Techniques
-5%
0%
5%
10%
15%
20%
10% 30% 50% 70% 90%
Recall
% Im
prov
emen
t Pre
cisi
on
Johnson 2000 (DB)
Random
Alternative Selection Techniques
Selection Techniques
-5%
0%
5%
10%
15%
20%
10% 30% 50% 70% 90%
Recall
% Im
prov
emen
t Pre
cisi
on
Johnson 2000(Model)Johnson 2000 (DB)
Random
Alternative Selection Techniques
Selection Techniques
-5%
0%
5%
10%
15%
20%
10% 30% 50% 70% 90%
Recall
% Im
prov
emen
t Pre
cisi
on
Distinctive
Johnson 2000(Model)Johnson 2000 (DB)
Random
Distinction improves retrieval more than other techniques.
Conclusion
• Method to select distinctive descriptors
• Distinctive descriptors can improve retrieval
• Mapping descriptors through likelihood and learned retrieval performance to distinction is better than other alternatives
• Distinction is independent of type of descriptor
Future Work
• Explore other definitions of likelihood including mixture models
Future Work
• Explore other definitions of likelihood including mixture models
• Consider non-likelihood parameterizations
Future Work
• Explore other definitions of likelihood including mixture models
• Consider non-likelihood parameterizations
• Combine descriptors while accounting for deformation [Funkhouser and Shilane, SGP]
Acknowledgements
Szymon RusinkiewiczJoshua PodolakPrinceton Graphics Group
Funding Sources:National Science Foundation Grant CCR-0093343
and Grant 11S-0121446Air Force Research Laboratory Grant FA8650-04-1-
1718
The End