Automatic Facial Landmark Tracking in Video Sequences using Kalman Filter Assisted Active Shape Models

Utsav Prabhu, Keshav Seshadri, Marios Savvides

報告人 : 李治衡 2011/06/08

Outline• Background

- ASM

- Kalman Filter • Tracking Methods

- Purely ASM

- Kalman Filter Assisted ASM• Experiments and Results• Conclusions and Future Work

Background – ASM Active Shape Model

1) Generate facial model using training images

2) Detect face in test image

3) Deform model to fit face in test image

Background – ASM • Active Shape Model

- Any Facial Shape

- Mean Shape

P - Eigenvector matrix

b - Projection coefficients

Background – Kalman FilterEstimate optimal state at time t

with a measurement given by

Prediction Stage

Correction Stage

Tracking MethodsPurely ASM Based Approaches• ASM on individual frames• ASM on individual frames with correction• ASM with initialization using previous frame

Kalman Filter Assisted ASM• Tracking landmark coordinates across frames• Tracking parameters that affect landmark positions

Tracking Methods – Kalman Filter Assisted

Experiments and Results

(a) (b) (c)

(a) Initialization provided by face detection(b) Initialization provided by using ASM results of previous frame(c) Initialization provided by prediction step of Kalman filter

Experiments and Results

(a) (b) (c) (d) (e)

(a) ASM on individual frames(b) ASM on individual frames with correction(c) ASM initialized using results of previous frame(d) ASM with Kalman filtering of landmark coordinates(e) ASM with Kalman filtering of parameters affecting landmark locations

Methods Result Comparison

Experiments and Results

Conclusion & Future Work • Experiments on 3 videos confirm our Kalman based

approaches enable better ASM initialization and lower

fitting errors• Background subtraction and re‐initialization of ASM to deal

with scene changes, zooming in of subject etc.• Speed optimizations for our ASM and Kalman tracking

implementations• Benchmark our approach on publicly available datasets/more

challenging datasets

Thank you for your attention!