domenico bloisi, luca iocchi, dorothy monekosso, paolo remagnino [email protected]

A Novel Segmentation A Novel Segmentation Method for Crowded Method for Crowded

ScenesScenes

Domenico Bloisi, Luca Iocchi,

Dorothy Monekosso, Paolo Remagnino

[email protected]

04/18/23A Novel Segmentation Method for Crowded Scenes – VISAPP 2009

Video surveillance tasks

A video surveillance system may accomplish a series of well-defined tasks:• To detect objects of interest

(we may want to detect all the moving cars in a street)[Yoneama et al. 2005, <long list>]

• To track objects of interest(we may want to know the exact number of people standing in a room) [Khan and Shah 2006, <long list>]

• To react to particular events(we may want to send an alarm if an unauthorized person enters a restricted area) [Leo et al. 2005, <long list>]

• …


PB-KU

Visual modeling of people behaviors and interactions for professional training (PB-KU)

• The method is applied to the training of nurses in the School of Nursing in the Faculty of CISM at Kingston University [ http://www.healthcare.ac.uk ].

• The aim is to detect and track people in order to analyze their behavior


Summary

• Project Overview

• Segmentation

• Height Image Algorithm

• Examples

• Results


Features

• Background:– Dynamic background (indoor, crowded)

• Number of objects to track:– Up to 15 people in the scene

• Camera:– Two stereo cameras

• Evaluation method:– Evaluation on a on-site built data-set


General architecture


Hardware

Videre DesignSTH-MDCS

Intel Core 2 Duo2,0 GHz CPU

Mac mini

Videre DesignSTH-MDCS

Intel Core 2 Duo2,0 GHz CPU

Mac mini

Wireless connection

Firewireconnection

Firewireconnection


Segmentation(Detecting objects of interest)

BackgroundEstimate

CurrentFrame

BackgroundModel

ForegroundExtraction

List ofDetected Objects


Background Modeling

Background Imagecomputed from S

(the image displays only the higher

Gaussian values)

Set S of n images from a camera

Raw images

Artificial image


Foreground Extraction(Background Subtraction Technique)

THRESHOLD T(based on illumination conditions)

blobs (Binary Large OBjectS)

>

T

current frame

foreground image

background image


Background Subtraction Problems

Background subtraction is a fast and effective technique, but it presents a series of problems:

How to compute a correct background? [Heikkilä and Silven 1999] [Stauffer and Grimson 1999]

How to manage gradual and sudden illumination changes? [Bloisi et al. 2007]

How to manage high-frequencies background objects (such as artificial light flickering, windows) [Bloisi and Iocchi 2008]


Proposed Solution

Background Subtraction +Stereo Vision +

Edge Detection +Height Image Algorithm

Advantages:robust and efficient foreground extraction, shadow suppression, 3D information, non-moving object filtering, accurate multiple object segmentation.


System architecture PB-KU


Height ImageAlgorithm

t: minimum area for a blob to be considered of interest

A: set of found activity blobs

F: final set of the segmented objects we are searching for

H: the set of height images.


Height Image

a) Active blobsb) Height imagec) Segmented

image


Example

Segmented image Ground plane view


Example

Crowd flowAnalysis


Evaluation and Metrics

• Evaluation On-site dataset

• Metrics Scene accuracy

A is the average accuracy

n

nnai

ˆ

1

Number of detected people

Number of people in the scene


Segmentation Results

Segmentation Accuracyon 100 randomly chosen images


Algorithm Speed


Conclusions

• Summary of results Accurate segmentation even in

case of 15 people in the scene Real-time computation Ground plane view projection for

crowd flow analysis


Future Work (1)

Add Radio Frequency Identifiers (RFID) to stereo for helping segmentation and dealing with occlusions

RFID

Identity

LocalizationStereo


Future Work (2)

Crowd flow analysis based on ground plane projection

• ExampleHow many people are near a bed in event of an emergency?


References

- A. Yoneama C.H. Yeh, C.C.J. Kuo. Robust Vehicle and Traffic Information Extraction for Highway Surveillance, JASP(2005), No. 14, pp. 2305-2321, 2005. - M. Leo, T. D’Orazio, A. Caroppo, T. Martiriggiano P. Spagnolo. Automatic Monitoring of Forbidden Areas to Prevent Illegal Accesses ICAPR (2), pp. 635-643, 2005. - S. Khan and M. Shah. A multiview approach to tracking people in crowded scenes using a planar homography constraint. In ECCV (4), pp. 133–146, 2006. - Y.T. Tsai, H.C. Shih and C.-L. Huang. Multiple human objects tracking in crowded scenes. In ICPR ’06, pp. 51–54, 2006.- J. Heikkilä, O. Silven. A real-time system for monitoring of cyclists and pedestrians. Proc. 2° IEEE International Workshop on Visual Surveillance, pp. 74-81, 1999.-C. Stauffer, W. Grimson. Adaptive background mixture models for real-time tracking. (CVPR'99), pp.246-252, 1999.- D. Bloisi, L. Iocchi, G.R. Leone, R. Pigliacampo, L. Tombolini, L. Novelli. A Distributed Vision System for Boat Traffic Monitoring in the Venice Grand Canal (VISAPP), pp. 549-556, 2007.- D. Bloisi and L. Iocchi. ARGOS - A Video Surveillance System for Boat Traffic Monitoring in Venice. IJPRAI, 2008.

A Novel Segmentation A Novel Segmentation Method for Crowded Method for Crowded

ScenesScenes

Domenico Bloisi, Luca Iocchi,

Dorothy Monekosso, Paolo Remagnino

[email protected]

domenico bloisi, luca iocchi, dorothy monekosso, paolo remagnino [email protected]

Documents

novel segmentation method

crowded scenes visapp

features background

correct background

crowded number of objects

behavior slide

dynamic background indoor

general architecture