recognition by probabilistic hypothesis construction
DESCRIPTION
Recognition by Probabilistic Hypothesis Construction. P. Moreels, M. Maire, P. Perona California Institute of Technology. Background. Rich features. Probabilistic constellations, categories. Efficient matching. • Fischler & Elschlager, 1973 • v.d. Malsburg et al. ‘93. • Burl et al. ‘96 - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Recognition by Probabilistic Hypothesis Construction](https://reader035.vdocuments.net/reader035/viewer/2022062314/56813d54550346895da70e2f/html5/thumbnails/1.jpg)
Recognition by Probabilistic Hypothesis Construction
P. Moreels, M. Maire, P. Perona
California Institute of Technology
![Page 2: Recognition by Probabilistic Hypothesis Construction](https://reader035.vdocuments.net/reader035/viewer/2022062314/56813d54550346895da70e2f/html5/thumbnails/2.jpg)
Rich features,probabilistic,fast learning,
efficient matching
Background
• Huttenlocher & Ullman, 1990
Efficientmatching
Rich features
• Fischler & Elschlager, 1973• v.d. Malsburg et al. ‘93
• Burl et al. ‘96• Weber et al. ‘00• Fergus et al. ‘03
• Lowe ‘99, ‘04
Probabilistic constellations,
categories
![Page 3: Recognition by Probabilistic Hypothesis Construction](https://reader035.vdocuments.net/reader035/viewer/2022062314/56813d54550346895da70e2f/html5/thumbnails/3.jpg)
Objective: Individual object recognition
• D.Lowe, constellation model.
• Hypothesis and score.
• Scheduling of matches.
• Experiments: compare with D.Lowe.
Outline
![Page 4: Recognition by Probabilistic Hypothesis Construction](https://reader035.vdocuments.net/reader035/viewer/2022062314/56813d54550346895da70e2f/html5/thumbnails/4.jpg)
Lowe’s recognition system
Lowe’99,’04
…
Test image
Models
![Page 5: Recognition by Probabilistic Hypothesis Construction](https://reader035.vdocuments.net/reader035/viewer/2022062314/56813d54550346895da70e2f/html5/thumbnails/5.jpg)
Constellation model
Burl’96, Weber’00, Fergus’03
![Page 6: Recognition by Probabilistic Hypothesis Construction](https://reader035.vdocuments.net/reader035/viewer/2022062314/56813d54550346895da70e2f/html5/thumbnails/6.jpg)
• Principled
detection/recognition
• Learn parameters from data
• Model clutter, occlusion,
distortions
+
-
Lowe’s recognition system Constellation model
• High number of parameters (O(n2))• 5-7 parts per model
• many training examples needed
• learning expensive
• Many parts redundancy
• Learn from 1 image
• Fast
Pros and Cons
• Manual tuning of parameters
• Rigid planar objects
• Sensitive to clutter
![Page 7: Recognition by Probabilistic Hypothesis Construction](https://reader035.vdocuments.net/reader035/viewer/2022062314/56813d54550346895da70e2f/html5/thumbnails/7.jpg)
How to adapt the constellation model to our needs ?
![Page 8: Recognition by Probabilistic Hypothesis Construction](https://reader035.vdocuments.net/reader035/viewer/2022062314/56813d54550346895da70e2f/html5/thumbnails/8.jpg)
Reducing degrees of freedom
1. Common reference frame ([Lowe’99],[Huttenlocher’90])
2. Share parameters ([Schmid’97])
3. Use prior information learned on foreground and background ([FeiFei’03])
model m
position of model m
![Page 9: Recognition by Probabilistic Hypothesis Construction](https://reader035.vdocuments.net/reader035/viewer/2022062314/56813d54550346895da70e2f/html5/thumbnails/9.jpg)
Foreground
Gaussian
shape pdf
Gaussian part
appearance pdf
Gaussian
relative scale pdf
log(scale)
Prob. of detection
0.8
Based on [Fergus’03][Burl’98]
Parameters and priors
0.8 0.75 0.9
Gaussian background
appearance pdf
ClutterConstellation model
Gaussian part
appearance pdfGaussian
relative scale pdf
log(scale)
Prob. of detection
0.80.2 0.2 0.2
Gaussian background
appearance pdf
Gaussian conditional
shape pdf
Foreground ClutterSharing parameters
![Page 10: Recognition by Probabilistic Hypothesis Construction](https://reader035.vdocuments.net/reader035/viewer/2022062314/56813d54550346895da70e2f/html5/thumbnails/10.jpg)
Hypotheses – features assignments
= models from database
New scene (test image)
. . .
Interpretation
. . .
![Page 11: Recognition by Probabilistic Hypothesis Construction](https://reader035.vdocuments.net/reader035/viewer/2022062314/56813d54550346895da70e2f/html5/thumbnails/11.jpg)
Models fromdatabaseNew scene (test image)
Hypotheses – model position
1
2
3
Θ = affine transformation
![Page 12: Recognition by Probabilistic Hypothesis Construction](https://reader035.vdocuments.net/reader035/viewer/2022062314/56813d54550346895da70e2f/html5/thumbnails/12.jpg)
Score of a hypothesis
Hypothesis:model + position + assignments
observed featuresgeometry + appearance
database of models
(Bayes rule)
constantConsistency Hypothesis probability
![Page 13: Recognition by Probabilistic Hypothesis Construction](https://reader035.vdocuments.net/reader035/viewer/2022062314/56813d54550346895da70e2f/html5/thumbnails/13.jpg)
Score of a hypothesis
- Consistency between observations and hypothesis
- Probability of number of clutter detections
- Probability of detecting the indicated model features
- Prior on the pose of the given model
foreground features ‘null’ assignments
geometry geometryappearance appearance
![Page 14: Recognition by Probabilistic Hypothesis Construction](https://reader035.vdocuments.net/reader035/viewer/2022062314/56813d54550346895da70e2f/html5/thumbnails/14.jpg)
Efficient matching process
![Page 15: Recognition by Probabilistic Hypothesis Construction](https://reader035.vdocuments.net/reader035/viewer/2022062314/56813d54550346895da70e2f/html5/thumbnails/15.jpg)
Scheduling – inspired from A*
empty hypothesis
1 assignment …
scene features, no assignment done
PP P Pperfect completion
(admissible heuristic, used as a guide for the search)
Increase computational efficiency:
- at each node, searches only a fixed number of sub-branches
- forces terminationPearl’84,Grimson’87
‘null’ assignment
… …2 assignments
P P P P
P Pcan be compared
explore most promising branches first
P P
Score
![Page 16: Recognition by Probabilistic Hypothesis Construction](https://reader035.vdocuments.net/reader035/viewer/2022062314/56813d54550346895da70e2f/html5/thumbnails/16.jpg)
….models
fromdatabase
New scene
Recognition: the first match No clue regarding geometry first match based on appearance
best match
second best match
features Initialization ofhypotheses queue
….
P P P P P
P P P P P
P P P P P
![Page 17: Recognition by Probabilistic Hypothesis Construction](https://reader035.vdocuments.net/reader035/viewer/2022062314/56813d54550346895da70e2f/html5/thumbnails/17.jpg)
….models
fromdatabase
New scene
Scheduling – promising branches first
features Updated hypothesesqueue
….
P P P P
P P P
P P P?
![Page 18: Recognition by Probabilistic Hypothesis Construction](https://reader035.vdocuments.net/reader035/viewer/2022062314/56813d54550346895da70e2f/html5/thumbnails/18.jpg)
Experiments
![Page 19: Recognition by Probabilistic Hypothesis Construction](https://reader035.vdocuments.net/reader035/viewer/2022062314/56813d54550346895da70e2f/html5/thumbnails/19.jpg)
Toys database – models
153 model images
![Page 20: Recognition by Probabilistic Hypothesis Construction](https://reader035.vdocuments.net/reader035/viewer/2022062314/56813d54550346895da70e2f/html5/thumbnails/20.jpg)
Toys database – test images (scenes)
- 90 test images- multiple objects or different view of model
![Page 21: Recognition by Probabilistic Hypothesis Construction](https://reader035.vdocuments.net/reader035/viewer/2022062314/56813d54550346895da70e2f/html5/thumbnails/21.jpg)
100 model images
Kitchen database – models
![Page 22: Recognition by Probabilistic Hypothesis Construction](https://reader035.vdocuments.net/reader035/viewer/2022062314/56813d54550346895da70e2f/html5/thumbnails/22.jpg)
Kitchen database – test images
- 80 test images- 0-9 models / test image
![Page 23: Recognition by Probabilistic Hypothesis Construction](https://reader035.vdocuments.net/reader035/viewer/2022062314/56813d54550346895da70e2f/html5/thumbnails/23.jpg)
Test image Identified modelTest image Identified model
Examples
Lowe’s model implemented using [Lowe’97,’99,’01,’03]
Lo
we’
s m
eth
od
Ou
r sy
stem
![Page 24: Recognition by Probabilistic Hypothesis Construction](https://reader035.vdocuments.net/reader035/viewer/2022062314/56813d54550346895da70e2f/html5/thumbnails/24.jpg)
a. Object found, correct pose Detection
b. Object found, incorrect pose False alarm
c. Wrong object found False alarm
d. Object not found Non detection
Performance evaluation
Test image hand-labeledbefore the experiments
![Page 25: Recognition by Probabilistic Hypothesis Construction](https://reader035.vdocuments.net/reader035/viewer/2022062314/56813d54550346895da70e2f/html5/thumbnails/25.jpg)
Results – Toys imagesS
cene
s (t
est
imag
es)
Mod
els
(dat
abas
e)
- 80% recognition with false alarms / test set = 0.2- Lower false alarm rate than Lowe’s system.
- 153 model images- 90 test images- 0-5 models / test image
![Page 26: Recognition by Probabilistic Hypothesis Construction](https://reader035.vdocuments.net/reader035/viewer/2022062314/56813d54550346895da70e2f/html5/thumbnails/26.jpg)
Results – Kitchen images
- Achieves 77% recognition rate with 0 false alarms
- 100 training images- 80 test images- 0-9 models / test image- 254 objects to be detected
![Page 27: Recognition by Probabilistic Hypothesis Construction](https://reader035.vdocuments.net/reader035/viewer/2022062314/56813d54550346895da70e2f/html5/thumbnails/27.jpg)
• Unified treatment• Best of both worlds
• Probabilistic interpretation of Lowe [‘99,’04].• Extension of [Burl,Weber,Fergus ‘96-’03] to many-features,
many-models, one-shot learning.
• Higher performance• Comparison with Lowe [‘99,’04].
• Future work: categories
Conclusions