![Page 1: Real Time, Online Detection of Abandoned Objects in Public Areas](https://reader036.vdocuments.net/reader036/viewer/2022062423/56814ebe550346895dbc5e8c/html5/thumbnails/1.jpg)
1
Real Time, Online Detection of Abandoned Objects in Public Areas
Proceedings of the 2006 IEEE International Conference on Robotics and Automation
Authors : Nathaniel Bird, Stefan Atev, Nicolas Caramelli, Robert Martin, Osama Masoud, Nikolaos Papanikolopoulos
![Page 2: Real Time, Online Detection of Abandoned Objects in Public Areas](https://reader036.vdocuments.net/reader036/viewer/2022062423/56814ebe550346895dbc5e8c/html5/thumbnails/2.jpg)
2
Outline
Introduction Method Description
A. Low-Level Processing B. Short-Term Logic C. Long-Term Logic D. Image Similarity
Results Conclusions and Future Work
![Page 3: Real Time, Online Detection of Abandoned Objects in Public Areas](https://reader036.vdocuments.net/reader036/viewer/2022062423/56814ebe550346895dbc5e8c/html5/thumbnails/3.jpg)
3
I. Introduction
This paper addresses this issue by presenting an algorithm for automated detection of abandoned objects.
Abandoned object : a stationary object has not been touching a person for some time threshold.
The method must Online in real time Stay active around the clock Not detect still people as abandoned objects Detect abandoned objects even if they are
occluded by moving crowds of people for periods of time
![Page 4: Real Time, Online Detection of Abandoned Objects in Public Areas](https://reader036.vdocuments.net/reader036/viewer/2022062423/56814ebe550346895dbc5e8c/html5/thumbnails/4.jpg)
4
II. Method DescriptionA. Low-Level Processing
Background segmentation is performed using [6].
Processing only every tenth frame is still a high enough rate considering the temporal scale at which the events of interest occur.
[6] C. Stauffer and W. E. L. Grimson, “Adaptive background mixture models for real-time tracking,” Proceedings of the IEEE Computer Vision and Pattern Recognition, vol. 2, pp. 2246-2252, June 1999
![Page 5: Real Time, Online Detection of Abandoned Objects in Public Areas](https://reader036.vdocuments.net/reader036/viewer/2022062423/56814ebe550346895dbc5e8c/html5/thumbnails/5.jpg)
5
II. Method DescriptionA. Low-Level Processing
The background segmentation method is restricted to user-specified regions of interest in the image. block out areas of the image where any background
changes detected can be considered as noise (walls) Remove areas that are too far from the camera for
accurate abandoned object identification
![Page 6: Real Time, Online Detection of Abandoned Objects in Public Areas](https://reader036.vdocuments.net/reader036/viewer/2022062423/56814ebe550346895dbc5e8c/html5/thumbnails/6.jpg)
6
II. Method DescriptionA. Low-Level Processing
Binary foreground mask Blob extraction is then performed
on the binary foreground mask. Correlating the blobs detected in
the last frame with the blobs detected in the current frame.
![Page 7: Real Time, Online Detection of Abandoned Objects in Public Areas](https://reader036.vdocuments.net/reader036/viewer/2022062423/56814ebe550346895dbc5e8c/html5/thumbnails/7.jpg)
7
II. Method DescriptionB. Short-Term Logic
Blob types: Abandoned Object (A) Person (P) Still Person (SP) Unknown (U)
Blob behaviors: Creation Splits Merges Blob centroid velocity
![Page 8: Real Time, Online Detection of Abandoned Objects in Public Areas](https://reader036.vdocuments.net/reader036/viewer/2022062423/56814ebe550346895dbc5e8c/html5/thumbnails/8.jpg)
8
II. Method DescriptionB. Short-Term Logic
Person Group (PG) A PG is created containing the new blobs that
split from a P or an SP. All blobs contained within a PG are classified as U until one of them becomes a P, at which all other blobs within it can be classified normally.
This is to stop a sitting person from being incorrectly classified as an abandoned object if they place a bag beside them that splits from their blob.
: threshold velocity above which an abandoned object, still person, or unknown becomes a person.
![Page 9: Real Time, Online Detection of Abandoned Objects in Public Areas](https://reader036.vdocuments.net/reader036/viewer/2022062423/56814ebe550346895dbc5e8c/html5/thumbnails/9.jpg)
9
II. Method DescriptionB. Short-Term Logic
![Page 10: Real Time, Online Detection of Abandoned Objects in Public Areas](https://reader036.vdocuments.net/reader036/viewer/2022062423/56814ebe550346895dbc5e8c/html5/thumbnails/10.jpg)
10
II. Method DescriptionB. Short-Term Logic
![Page 11: Real Time, Online Detection of Abandoned Objects in Public Areas](https://reader036.vdocuments.net/reader036/viewer/2022062423/56814ebe550346895dbc5e8c/html5/thumbnails/11.jpg)
11
II. Method DescriptionC. Long-Term Logic
The long term logic maintains a set of potential abandoned objects and a set of still people. Contour in the image plane Timestamp of when they were first detected.
When an A type blob is found, it first checked if it does not overlap with any item in the potential abandoned object or still person sets. Yes => copied into potential abandoned object
set No => ignore
So did SP type blob.
![Page 12: Real Time, Online Detection of Abandoned Objects in Public Areas](https://reader036.vdocuments.net/reader036/viewer/2022062423/56814ebe550346895dbc5e8c/html5/thumbnails/12.jpg)
12
II. Method DescriptionC. Long-Term Logic
All items in the potential abandoned object set and the still person set are checked every time a frame is processed. If their corresponding area in the binary
foreground mask is filled less than some percentage, p, then the item is dropped.
P were empirically found to be between 75% and 80%
![Page 13: Real Time, Online Detection of Abandoned Objects in Public Areas](https://reader036.vdocuments.net/reader036/viewer/2022062423/56814ebe550346895dbc5e8c/html5/thumbnails/13.jpg)
13
II. Method DescriptionC. Long-Term Logic
time threshold t If during a check it is discovered that t
time has elapsed since an item was added to the potential abandoned object set, an alarm is triggered for that object.
After the alarm is triggered, the long-term logic adjusts a mask used by the background segmentation module so that it will not improperly learn the abandoned object into the background.
![Page 14: Real Time, Online Detection of Abandoned Objects in Public Areas](https://reader036.vdocuments.net/reader036/viewer/2022062423/56814ebe550346895dbc5e8c/html5/thumbnails/14.jpg)
14
II. Method DescriptionD. Image Similarity
When the potential abandoned object is first detected, a copy of the image at its location is saved.
At every time step, the area surrounding the object ( a “halo” excluding the object) is checked for significant foreground activity.
If there is no noticeable foreground activity in the halo, the current image is compared pixel-by-pixel to the stored image for the potential abandoned object, and an average per-pixel difference is calculated.
![Page 15: Real Time, Online Detection of Abandoned Objects in Public Areas](https://reader036.vdocuments.net/reader036/viewer/2022062423/56814ebe550346895dbc5e8c/html5/thumbnails/15.jpg)
15
II. Method DescriptionD. Image Similarity
An exponential running average of this difference is then updated.
If the value of the exponential running average exceeds an empirically determined threshold, the potential abandoned object is deemed to be moving too much to be a stationary object. => reclassified to a still person.
sumavgicountssumavg )1(][
![Page 16: Real Time, Online Detection of Abandoned Objects in Public Areas](https://reader036.vdocuments.net/reader036/viewer/2022062423/56814ebe550346895dbc5e8c/html5/thumbnails/16.jpg)
16
III. ResultsA. Alarm Description
The following information is what is recorded for every alarm : 1. Identification Number 2. Start Time 3. Trigger Time 4. End Time 5. Image-Plane Location
![Page 17: Real Time, Online Detection of Abandoned Objects in Public Areas](https://reader036.vdocuments.net/reader036/viewer/2022062423/56814ebe550346895dbc5e8c/html5/thumbnails/17.jpg)
17
III. ResultsB. Ground Truth
The ground truth for a given video sequence is determined manually for every sequence by a human operator.
![Page 18: Real Time, Online Detection of Abandoned Objects in Public Areas](https://reader036.vdocuments.net/reader036/viewer/2022062423/56814ebe550346895dbc5e8c/html5/thumbnails/18.jpg)
18
III. ResultsC. PED/PAT Score Description
A candidate match is declared if there is sufficient spatial proximity and/or overlap between the two alarms as well as a temporal distance below a specific tolerance.
The candidate matches will usually result in a many-to-many relationship Bipartite graph
We therefore give each edge a weight equal to the timestamp difference between the two alarms and then find the minimum-weighted maximum-cardinality matching.
![Page 19: Real Time, Online Detection of Abandoned Objects in Public Areas](https://reader036.vdocuments.net/reader036/viewer/2022062423/56814ebe550346895dbc5e8c/html5/thumbnails/19.jpg)
19
III. ResultsD. Overall Score Description
We define an overall score as follows:
x is relative importance we wish to give to the PED score over the PAT score. x=0 => PAT plot X=0.5 => weight PED and PAT equally X=1 => PED plot
We use a value of x=0.75 because we consider finding true alarms more important than some false positives.
![Page 20: Real Time, Online Detection of Abandoned Objects in Public Areas](https://reader036.vdocuments.net/reader036/viewer/2022062423/56814ebe550346895dbc5e8c/html5/thumbnails/20.jpg)
20
III. ResultsE. Test Sequences
![Page 21: Real Time, Online Detection of Abandoned Objects in Public Areas](https://reader036.vdocuments.net/reader036/viewer/2022062423/56814ebe550346895dbc5e8c/html5/thumbnails/21.jpg)
21
III. ResultsF. Results
![Page 22: Real Time, Online Detection of Abandoned Objects in Public Areas](https://reader036.vdocuments.net/reader036/viewer/2022062423/56814ebe550346895dbc5e8c/html5/thumbnails/22.jpg)
22
III. ResultsF. Results
![Page 23: Real Time, Online Detection of Abandoned Objects in Public Areas](https://reader036.vdocuments.net/reader036/viewer/2022062423/56814ebe550346895dbc5e8c/html5/thumbnails/23.jpg)
23
IV. Conclusions and Future Work
We have presented a method to detect abandoned objects that works online in real time, uses color data, can adapt to scene changes around the clock, does not detect still people as abandoned objects and detects abandoned objects even if they are occluded by moving crowds of people for periods of time.
The results for densely populated scenes are not as good, indicating that future research should look into defining a short-term logic that characterizes the behavior of blobs corresponding to crowds.