foreground modeling the shape of things that came nathan jacobs advisor: robert pless computer...
Post on 17-Dec-2015
215 Views
Preview:
TRANSCRIPT
Foreground ModelingThe Shape of Things that
Came
Nathan JacobsAdvisor: Robert Pless
Computer ScienceWashington University in St. Louis
Nathan Jacobs 2
Visual Surveillance
• Observe people and vehicles– Where are they?– Where have they been?– Where are they going?
• Answering these questions requires object tracking.
Nathan Jacobs 3
Probabilistic Tracking
• Tracking is commonly cast in a Bayesian framework to estimate object shape and location– Initial estimate = combination of image data likelihood and
initialization prior– Updated estimate = combination of image data likelihood and
state prediction prior
• Likelihood functions are the focus of most tracking work– Color histograms, templates
• Our focus is on the prior terms
Nathan Jacobs 4
Quotes from yesterday
• “Initialization of tracking is important but not addressed here.”
• “Our object model assumes a well calibrated camera and a flat-ground plane.”
• “The prior term is a tricky thing to design.”
Nathan Jacobs 5
Passive Vision : The Big Picture
• Learn strong scene-specific priors by watching the same scene for a long time– Made easier because the cameras are static– Should be learned online
• Priors can be used to improve anomaly detection and tracking algorithms
Nathan Jacobs 6
Scene-specific Motion Priors
Nathan Jacobs 7
Unusual Traffic Motion
Video segment with anomalous motion (an ambulance using the median to pass stopped cars).
False color sequence highlighting anomalous motions.
Nathan Jacobs 8
Online Prior Learning for Tracking
Online learning and use of motion priors:
• reduces the number of particles needed
• increases the number of objects that can be tracked. Frames
Obj
ects
Nathan Jacobs 9
What else can we model?
• Watching for a long time allows us to build models of– Pixel intensity– Image derivatives– Image motion patterns
• We now transition to features based on the shape of foreground objects
Nathan Jacobs 10
An Example Video
Nathan Jacobs 11
Generating Examples Shapes
Current FrameForeground Mask Shape Descriptor
Background Image
For a long time:
1. Detect foreground objects
2. Generate a shape descriptor for each object
3. Add shape descriptor to training set
Nathan Jacobs 12
Shape Descriptor
• Currently using a simple shape model– A 20-dimensional feature vector– Each dimension is the distance from center to edge
of object
• Other shape models are possible
Nathan Jacobs 13
Two Shape Model Types
• Both models are PCA subspaces
• Global model– Subspace is location independent – Distribution estimate is location dependent
• Local model– Subspace and distribution are both location
dependent
Nathan Jacobs 14
A location-independent Shape Basis
Generate shape subspace using PCA on all shapes in training set.
First Principle Component
(~size)
Second Principle Component
(~orientation)
Nathan Jacobs 15
Location-dependent Coefficients
First Principle Component
(~size)
Second Principle Component
(~orientation)
Nathan Jacobs 16
Location-specific Shape Subspaces
Generate a shape subspace using shapes found in a small region of the image.
Location-specific mean shapes
Nathan Jacobs 17
Location-specific Shape Subspaces
Shape subspaces are location dependent.
Much smaller variations in some regions.
First PC Variations Second PC Variations
Nathan Jacobs 18
Shapes in (Shape) Space
Nathan Jacobs 19
An Example from PETS
First Principle Component
(~size)
Second Principle Component(~aspect)
Nathan Jacobs 20
Location-dependent Mean Shapes
Mean Shapes
Nathan Jacobs 21
Location-dependent Subspaces
First PC Variations Second PC Variations
Nathan Jacobs 22
Object Initialization for Tracking
• Object initialization is a crucial step of any tracking algorithm
• Use shape priors to determine object boundaries– Combines image information and shape prior– Penalize unlikely shapes– More accurate than image information alone
• Major point: strong priors make simple methods work
Nathan Jacobs 23
Object Boundary Detection
• Goal is to determine object boundaries to improve tracking initialization
• Algorithm– Find candidates using background subtraction– Initialize each candidate with a location-
specific mean shape– Optimize shape by gradient descent in PCA
shape subspace (penalize object overlap)• Image data term: sum of per-pixel foreground
probability inside shape• Shape prior term: sum of absolute value of PCA
coefficients
Nathan Jacobs 24
Segmentation Results
Subspace only Subspace and Prior
Global shape model
Local shape model
Nathan Jacobs 25
Summary
1. Static cameras give strong priors.
2. Unsupervised training of a localized shape prior is possible.
3. Localized shape priors can be used to improve object initialization for tracking.
Background
Nathan Jacobs 27
Efficient Segmentation
• Use gradient descent in low-dimensional shape subspace
• Gradient estimation– For each underlying
shape parameter• Sum along two edges
of polygon– For PC components and
object location• Weighted combination
of polygon edge scores
Nathan Jacobs 28
Choice of Support Region
Nathan Jacobs 29
Choosing Constants for Updating Prior Models
Threshold
.99999
0.9999
0.999
0.99
0.9
The best learning rate depends on scene,
application, time-of-day, weather, image location.
Slow update Fast updateCurrent Frame
VSSN 2006
Nathan Jacobs 30
Segmentation Energy Function
Minimize
Penalty on size
Per-pixel foreground likelihood
Shape penalty based on prior (sum of PCA coefficients)
top related