Tracking

Tracking: General idea• Initialize model in the first frame• Given model estimate for frame t-1:

• Predict for frame t– Use dynamics model of how the model changes

• Correct for frame t– Use observations from the image

predictpredict correctcorrect

Tracking issues• Deciding on the structure of the model

points curvespoints curves

pictorial structures

Tracking issues• Deciding on the structure of the model• Initialization• Specifying the dynamics model• Specifying the observation model

• Data association problem: which measurements tell us about the object(s) being tracked?

Data association• Simple strategy: only pay attention to the

measurement that is “closest” to the di tiprediction

Data association• Simple strategy: only pay attention to the

measurement that is “closest” to the di tiprediction

Doesn’t always workDoesn t always work…

Data association• Simple strategy: only pay attention to the

measurement that is “closest” to the di tiprediction

• More sophisticated strategy: keep track of multiple state/observation hypothesesmultiple state/observation hypotheses

• This is a general problem in computer vision, there is no easy solutionthere is no easy solution

Tracking issues• Deciding on the structure of the model• Initialization• Specifying the dynamics model• Specifying the observation model

• Data association problem• Prediction vs. correction

• If the dynamics model is too strong, will end up ignoring the data If the observation model is too strong tracking is• If the observation model is too strong, tracking is reduced to repeated detection

• DriftDrift

Drift

D. Ramanan, D. Forsyth, and A. Zisserman. Tracking People by Learning their Appearance. PAMI 2007.

Tracking with person-specific appearance models

pictorial structure

Tracker

D. Ramanan, D. Forsyth, and A. Zisserman. Tracking People by Learning their Appearance. PAMI 2007.

Tracking with person-specific appearance models• Structure and dynamics are generic,

appearance is person-specific• Trying to acquire an appearance model “on• Trying to acquire an appearance model on

the fly” can lead to drift• Instead, can use the whole sequence toInstead, can use the whole sequence to

initialize the appearance model and then keep it fixed while tracking

• Given strong structure and appearance models, tracking can essentially be done by repeated detection (with some smoothing)repeated detection (with some smoothing)

D. Ramanan, D. Forsyth, and A. Zisserman. Tracking People by Learning their Appearance. PAMI 2007.

Bottom-up initialization: Clustering

D. Ramanan, D. Forsyth, and A. Zisserman. Tracking People by Learning their Appearance. PAMI 2007.

Top-down initialization: Exploit “easy” poses

D. Ramanan, D. Forsyth, and A. Zisserman. Tracking People by Learning their Appearance. PAMI 2007.

Tracking by model detection

D. Ramanan, D. Forsyth, and A. Zisserman. Tracking People by Learning their Appearance. PAMI 2007.

Example results

http://www.ics.uci.edu/~dramanan/papers/pose/index.html