unified crowd segmentation

P. TU, T. SEBASTIAN, G . DORETO, N. KRAHNSTOEVER , J . R ITTSCHER , T. YU

ECCV 2008

Unified Crowd Segmentation

Presenter: Ramin Mehran

2

Goal: Segmentation

10/2/2008CV Lab @ UCF - Presenter: Ramin Mehran

Crowd Analysis Conventional Approach

Segmentation Object Detection Tracking Analysis

3

Ideas: a Robust Algorithm

Partial Occlusion

Dynamic Background

Foreground Clutter

10/2/2008CV Lab @ UCF - Presenter: Ramin Mehran

CV Lab @ UCF - Presenter: Ramin Mehran

4

Approaches

10/2/2008

Bottom up Low level feature grouping Whole body classifiers

Top down Background segmentation

BOTH!

CV Lab @ UCF - Presenter: Ramin Mehran

5

Algorithm Overview

Head-Shoulder Detection

Assigning Patches to

Heads

Grouping Patches

E-Step (assignment

update)

M-Step (consistency

check)

10/2/2008

CV Lab @ UCF - Presenter: Ramin Mehran

6

Affinity of Patches to Hypotheses

10/2/2008

Ck

zj

zi

gk(zi)

gk(zi, zj)•Top Down Head/Shoulder

Detection•Bottom up Patch Affinities

Ck Hypothesis

Head/Shoulder

Background

Head-Shoulder Detection

Assigning Patches to

Heads

Grouping Patches

CV Lab @ UCF - Presenter: Ramin Mehran

7

Maximum Likelihood

10/2/2008

How likely is the assignment a patch to a head

E-M Steps

Is it likely near the head? Is it like neighboring patches?

Consistency

Assignment

indicatorAffinit

y

CV Lab @ UCF - Presenter: Ramin Mehran

8

E-Step

10/2/2008

V

XZVLVp );|()(

)Hypotheses ,(Patch kiP

V

k

iVVIncrease the expectations

CV Lab @ UCF - Presenter: Ramin Mehran

9

E-Step affect

10/2/2008

CV Lab @ UCF - Presenter: Ramin Mehran

10

M-Step

10/2/2008

Enforce Consistency Where is the head? Contiguous patches

Path from Patches to the Head Dynamic Programming Shortest low cost path

Ck

Cm

Inconsistent patches

No Contribution to V (assignment)

CV Lab @ UCF - Presenter: Ramin Mehran

11

Head/Shoulder Body Detection

10/2/2008

Strong Classifier out of Weak Classifiers

Strong Classifier (SC(s,z))Orientatio

nOrientatio

nOrientatio

nOrientatio

n

Aggregation of weak Classifiers

CV Lab @ UCF - Presenter: Ramin Mehran

12

Weak Classifier

10/2/2008

Weak Classifier Ratio of the intersection of

patch and the region Sobel Orientation

Ck

zi

R

Needs Camera

Calibration

CV Lab @ UCF - Presenter: Ramin Mehran

13

What’s the Affinity?

10/2/2008

Patch to Head/Shoulder Affinity

Patch to Patch Affinity Histogram matching of

colors Motion matching of

flow

),()( ikik zssczg

Head and body bounding box

patch

),(),( jijik zzsimzzg

CV Lab @ UCF - Presenter: Ramin Mehran

14

Patch Size

10/2/2008

Camera CalibrationWidth of the average pedestrianLarge Patches

Occlusion HandlingSmall Patches

Discrimination of correct and incorrect assigments

CV Lab @ UCF - Presenter: Ramin Mehran

15

Sample Results

10/2/2008

CV Lab @ UCF - Presenter: Ramin Mehran

16

Results

10/2/2008

Compared to Histogram of Oriented Gradient method

Superior mostly because of handling occlusions

CV Lab @ UCF - Presenter: Ramin Mehran

17

Thank you!

10/2/2008