autonomous learning of object models on mobile robots xiang li ph.d. student supervised by dr. mohan...
TRANSCRIPT
Autonomous Learning of Object Models on Mobile Robots
Xiang Li
Ph.D. student supervised by Dr. Mohan SridharanStochastic Estimation and Autonomous Robotics Lab (SEARL)
Department of Computer ScienceTexas Tech University
October 23, 2012
1
Object Recognition
2
Learn Object Modelcolor, texture or shape
Test
match
Challenges:• How to identify ROI in the image (Region Of Interest) ?
• What features to extract in ROI (Object model) ?
• Efficient Implementation.
ROI
Related Work
• Object model O. Bjorn, PAMI12; R. Fergus, CVPR03; P. Felzenszwalb, IJCV04; N.E.K. Roman,
AR10.
• Mobile robot
C. Guo, ICRA11; M. Sridharan, IROS07; D. Parikh, PAMI12; J. Hoey, CVIU10.
• Local image feature
D. Lowe, IJCV05; J. Matas, BMVC02; S. Park, IROS09; M. Calonder, ECCV10.
3
Motivation
4
• Goal
Learn and recognize objects autonomously in dynamic environments.
• Our Work Identify ROIs autonomously based on a limited number of images with moving objects. Build probabilistic object models using the complementary properties of different visual cues. Fuse the information by generative model and energy minimization algorithm.
Autonomously Learning• Supervised Learning
Images with the labeled regions
• Unsupervised Learning Images without any labeled regions
Track and cluster local image gradient features [128D vector] A short sequence of images (motion cue)
5
ROI
t t+1
ROI
Object Model• Given ROI (autonomous or manual)• Use the complementary properties of different visual cues
6
Motivation of SCV and Undirected Graph
• The difference between correct and incorrect match The spatial arrangements of local features The connection between the local features
7
• Match by Nearest Neighbor algorithm(shortest Euclidean distance).
correct
Incorrect
SCV from gradient features
• The individual gradient features may not be unique.• The spatial arrangement of local gradient features corresponding to a
specific object is difficult to duplicate.
8
Connection Potentials
• Connection potential is computed as the color distribution of pixels between gradient features in the image ROI.
9
• Build an undirected graph of connection potentials to model the neighborhood relationships between gradient features.
Parts from image segments
• Considers the arrangement of object parts made up of image segments.
• Pixels within a part have similar values, while pixels in neighboring parts have dissimilar values.
10
Color-based Representation• Computes the distance between every pair of pdfs and models the
distribution of distances as a Gaussian.
11
Second order color distribution statistics Color histogram (pdf)
… …
Local context from image segments
• Probabilistic mixture models
• Relative positions (on, under, beside)
12
Information Fusion
13
• Generative model considers the relationship between the components
• Energy minimization algorithm Identifies the ROI for recognizing the stationary objects
Robot Platforms: Erratic
• 1.6GHz Core2 Duo CPU
• 2 cameras (monocular & stereo)
• Laser range finder
• 640 × 480 Resolution
• Wi-Fi
• On board computation
14
Experimental Trial
15
Test Image Match Probabilities Net Match
Experimental Trial (Cont)
16
Test Image Match Probabilities Net Match
Object Categories
17
The Classification Accuracy
18
Conclusions• Mobile robot can identify interesting objects based on motion
cues, and autonomously and efficiently learn object models that exploit the complementary properties of appearance-based and contextual visual cues.
• Exploiting the complementary properties of these visual cues enables the robot to use generative model and energy minimization algorithms to reliably and efficiently recognize the learned.
19
Future Work
• Consider image sequences with multiple moving objects.
• Add Shape representation into object model.
• Extend to a team of robots collaborating in dynamic environments.
20
Q & A
21
Convex Hull
• The convex hull of a set of points is the smallest convex set that contains the points.
• Quickhull algorithm1 computes the convex hull.
1. Barber, C.B., Dobkin, D.P., and Huhdanpaa, H.T., “The Quickhull algorithm for convex hulls”, ACM Trans. on Mathematical Software, 22(4):469-483, Dec 1996, http://www.qhull.org
Gamma distribution
• Context Probability
• Context Probability Using Gamma
Matched Not Matched
context 0.28±0.15 0.03±0.03
Other components 0.7 0.3
Matched Not Matched
context 0.5±0.25 0.09±0.10
Other components 0.7 0.3
Experiment Example• Learned robot model (Corridor)
• Testing
Corridor(1)Prob = 0.89
Corridor(2)Prob = 0.77
OfficeProb = 0.12
Generative Model
• Randomly generating observable data• Typically given some hidden parameters• Specifies a joint probability distribution over
observation and label sequences.
25
Generative model (Cont)
26