Belief Networks in Computer Vision Applications

Alex YakushevCMPS 290C final project

Winter 2006

Kalman Filters

Predicted position =(Previous Position)*

(Motion model )+ Noise

True position =observed position

+ Noise

Extrapolate object position by taking a weighted average

of the two.

Only depends on current observationand previous state

Kalman Filter as a Belief Network

xk xk+1

zk zk+1

P x k∣x k−1 ~N x k ; x k−1 , P k−1TQ

P z k∣x k ~N Hx k ; Z k , R

Motion model

Prediction covariance

Model Noise

State model

Observation noise

Kalman Filters

Becomes a simple update rule

x k1= xkPk1= Pk

TQK k= Pk H

T H P k HTR−1

xk= x kK k zk−H k xk P k= I−K k H P k

Something completely different● Are old fashioned Bayesian networks of any

use?● What if the object we are trying to track does

not have a constant velocity or acceleration?● Sherrah and Gong have designed a BN to

track hand and face movements.

Something completely different

(Sherrah, Gong)

Results (Sherrah, Gong)● For their setup they had 70% of frames classified

correctly without any contextual knowledge● This improved to 78% with a Kalman filter● 87% with their method...

...but Assumptions are unreasonable:1. the subject is oriented roughly towards the camera for most of the time2. the subject is wearing long sleeves3. reasonably good color segmentation of the head and hands is possible, and4. the head and hands are the larges moving skin color clusters in the image

“Dynamic Bayesian Network”

(Pavlovic, Rehg, Cham, Murphy)

State variables st belong to a set of S discrete symbols.

Dynamic Bayesian Network

● Two hidden variables (xt, st)● If x0 is Gaussian then x1 is a mixture of S

Gaussians, x2 is a mixture of S2 Gaussians...● No “simple” update rule● Exact inference is expensive

(Pavlovic, Rehg, Cham, Murphy)

Conclusions● As usual, increase in expressive power costs

computation time● People create custom networks, and either

give a solution for the specific structure, or use approximate inference

● No silver bullet, good results possible with both simple and complex network structures

● Using a simpler structure (Markov Network) may work for many applications

References● Jamie Sherrah, Shaogang Gong, Tracking Discontinuous Motion

Using Bayesian Inference, Lecture Notes in Computer Science, Volume 1843, Jan 2000, Pages 150 – 166

● V. Pavlović, J. M. Rehg, T. J. Cham, and K. P. Murphy, A dynamic Bayesian network approach to figure tracking using learned dynamic models, in International Conference on Computer Vision, Corfu, Greece, September 1999.

● Murphy, K., Dynamic Bayesian Networks: Representation, Inference and Learning. PhD thesis, UC Berkeley, Computer Science Division (2002)

● Hai Tao, Object Tracking and Kalman Filtering, CMPE264 Lecture notes, http://www.soe.ucsc.edu/classes/cmpe264/Winter06/Lec15.pdf