principal axis-based correspondence between multiple cameras for people tracking

Principal Axis-Based Correspon-dence between Multiple Cameras

for People Tracking

Dongwook Seoseodonguk@islab.ulsan.ac.kr

2012.04.07

2Intelligent Systems

Lab.

Overview

3Intelligent Systems

Lab.

Detection of principal axes in a single cameraMotion segmentation and object classification

Using the vertical projection histogram to distinguish people from vehicles

1, , 1,height

yh x I x y x width

- I(x,y): binary image- height, width: the height and width of motion region

The spread of a vertical projection histogram

1

1

1

1width

xwidth

x

h x h xSpread

h x

4Intelligent Systems

Lab.

Detection of Principal Axes

2argmin ,i ilL median D X l

Principal axis of an isolated personUsing the Least Median of Squares to determine the princi-pal axis of an isolated person

- : the perpendicular distance between the ith foreground pixel and axis

5Intelligent Systems

Lab.

Detection of Principal Axes(Cont.)Principal axes of people in group

(a) input image

(b) Detected foreground region

(c) Vertical projection histogram

(d) segmented individuals

(e) Principal axes

6Intelligent Systems

Lab.

Detection of Principal Axes(Cont.)Principal axes of people under occlusion

Using the color template-based method to segment people

, , 1 1

, 1

, 1 1

i i

ii

i

M X t M X t I X if X

P M X t if XP M X

P M X t if X

F

FF

- : color model of object i consist of a color variable - : the rgb color of each pixel X of object i- : the likelihood of object i being observed at pixel X

7Intelligent Systems

Lab.

TrackingThe construction of correspondence relationships between “tracked objects” in previous frames and “detected objects” in the current frame

To track people using Kalman filter: the state of a person

: the position of a person in the image plane: the velocity of a person

Using “ground-point” on the image plane for the position of individual

8Intelligent Systems

Lab.

Correspondence between multiple camerasHomography recovery

A homography is a 3 by 3 matrix H.

Consider a point in one image and in another image

11 12 13

21 22 23

31 32 1

h h hh h hh h

H

'11 12 13

'21 22 23

31 321 1 1

i i

i i

x h h h xy h h h y

h h

9Intelligent Systems

Lab.

Correspondence between multiple cameras(Cont.)

Geometrical relationship and correspondence likeli-hood

10Intelligent Systems

Lab.

Correspondence between multiple cameras(Cont.)

The function of correspondence likelihood

1/2 1

1/2 1

, | |

1| 2 exp21| 2 exp2

i j i ji i ijs k s ks k sk

Ti ji i i ji i i jis ks s s ks s s ks

Ti ij j j ij j j ijk sk k k sk k k sk

L L p X Q p X Q

p X Q X Q X Q

p X Q X Q X Q

- : covariance matrixes (diagonal matrix-)- : covariance matrixes (diagonal matrix-)

The correspondence distance () for principal axis pairs

1

1

,

i Tij i ji i jisk s ks s ks

j Tj ij j ijk sk k sk

i i j js k

D X Q X Q

X Q X Q

where

11Intelligent Systems

Lab.

Correspondence between multiple cameras(Cont.)

Correspondence between multiple camerasStep1. A list() of all possible correspondence pairs of princi-pal axes is created.Step2. For each pair in the pair list , it is checked whether pair satisfies the constraint

: Threshold to classify true or false correspondence pairsStep3. To find all possible pairing modes

, k: index of a paring mode Step4. The minimum sum of correspondence distance

All principal axis pairs in pair mode are the matched one.Step5. The pairs in pair set are labeled.

12Intelligent Systems

Lab.

ExperimentsResults on NLPR Database

Tracking and correspondence of multiple people with two cameras

# 3286

# 3297

# 3380

13Intelligent Systems

Lab.

Experiments(Cont.)Results on PETS2001 Database

Tracking and correspondence of multiple people with three cameras

14Intelligent Systems

Lab.

Experiments(Cont.)Tracking and correspondence

15Intelligent Systems

Lab.

Experiments(Cont.)Comparison

(a) Trajectory acquired using this paper and true data. E=3.2(b) Centroid trajectory and true data. E=5.8

16Intelligent Systems

Lab.

Experiments(Cont.)Comparison

- The white ones are acquired using this paper, and the black ones are centroid trajectories.(a) Trajectories in view 1.(b) Trajectories in view 2.

17Intelligent Systems

Lab.

ConclusionsFor matching people across multiple cameras

Using principal axis-based methodCamera calibration is not needed and there is less sensitivity to errors in motion detection.

Future workApplying this algorithms for non-planar ground surfaces

18Intelligent Systems

Lab.

Thank you!!!