An Efficient Smoke Detection Algorithm for Video Surveillance

Systems Based on Optical Flow

R. Bogush, N. Brovko

Polotsk State University, Blokhin str., 29, Novopolotsk, Belarus, 211440

bogushr@mail.ru, dyanad@yandex.ru

Abstract: In this paper we propose an algorithm for

smoke detection on the color video sequences obtained

from the stationary camera. Our algorithm composed of

three basic steps: moving areas (blobs) segmentation in a

current input frame; received blobs classification; merge

of the moving blobs obtained at the previous steps. We

use blocks matching approach for optical flow

calculation. It considers a primary direction of smoke

propagation. Moving blobs classification based of the

contrast analysis as a whole current input frame, as the

blocks, obtained after its splitting. In the final step of

algorithm moving blobs merging the connected

component analysis is used. Real video surveillance

sequences were used for smoke detection with utilization

our algorithm. A set of experimental results are presented

in the paper.

Keywords: smoke detection, video sequences, blocks

matching, contrast analysis.

1. Introduction

Early fire detection of on open spaces, in buildings, in

territories of the industrial enterprises etc. is a key critical

task for fire alarm systems. For today fire alarm systems

are based on infrared sensors, optical sensors, or ion

sensors that depend on certain characteristics of fire, such

as smoke, heat, or radiation. Traditional fire alarm

systems are not alerted until the particles actually reach

the sensors, and usually are unable to provide any

additional information, such as the location and size of the

fire and the degree of burning. It essentially increases

time fire detection. Application of sensors is not reliable

for the objects located on open air, or on objects with

application of natural or artificial ventilation, for example,

in tunnels.

Video surveillance systems and fire alarm systems

combination in the uniform decision of the visual control

of space allows reducing final cost of the equipment

considerably. The basic requirements to smoke detection

in video sequences are low probability of false operations

on moving objects, movement of clouds, a fog, and

processing of video sequences to a real time. Smoke

detection methods often use color and motion information

to detect smoke from digital images.

The algorithm of background subtraction is

traditionally applied to movement definition in video

sequence [1-4]. Common technique is using adaptive

Gaussian Mixture Model to approximate the background

modeling process [1, 2]. However a number of elements

of the image belonging to a smoke, is characterized by

small moving, therefore eventually they will be included

in background model. It will lead to impossibility smoke

detection in the given areas of an input frame.

In [5] optical flow calculation is applied to detection

of movement of a smoke. Lacks of the given approach are

high sensitivity to noise and high computational cost.

Algorithms based on color and dynamic

characteristics of a smoke are applied for classification of

the given moving blobs. In [6] the algorithm comparative

evaluation of the histogram-based pixel level

classification is considered. In this algorithm the training

set of video sequences on which there is a smoke is

applied to the analysis. However, methods based on

preliminary training are dependence of quality of

classification on a training set. It demands much of

qualitative characteristics of processed video images. The

area of decreased high frequency energy component is

identified as smoke using wavelet transforms [1, 2].

However change of scene illumination can be contours

degradation reason. Therefore such approach requires

additional estimations. Color information is also used for identifying smoke in video. Smoke color at different

stages of ignition and depending on a burning material is

distributed in a range from almost transparent white to

saturated gray and black. In [1] decrease in value of

chromatic components U and V of color space YUV is

estimated. In [7] the color model in space RGB and

estimation component of saturation S from space HSV is

proposed.

In this paper we propose a method for smoke

detection on the color video sequences obtained from the

stationary camera. Our algorithm composed of three basic

steps: moving blobs segmentation in a current input

frame; received blobs classification; merge of the moving

blobs obtained at the previous steps. At a stage block

matching detection method of calculation of optical flow

is applied. It considers a primary direction of smoke

propagation. In [8] it is shown, that global direction of

smoke is 0-45°. This statement allows to simplify

procedure blocks matching detection and, hence,

considerably to reduce number of calculations. Moving

blobs classification based of the contrast analysis as a

whole current input frame, as the blocks, obtained after its

splitting. In the final step of algorithm moving blobs

merging the connected component analysis is used.

This paper is organized as follows. Section 2 presents

our smoke detection algorithm. Section 3 presents

experimental results, and section 4 contains the

conclusion and feature work.

2. Algorithm description

The proposed algorithm is a group of different

modules as showed in Fig.1.

Fig. 1 – Flow chart of our proposed algorithm

2.1 Block matching algorithm

Blocks matching approach for optical flow calculation

assumes that the frame is divided into small regions called

blocks. Blocks are typically square and contain some

number of pixels. These blocks are not overlap. In our

realization frames in the size 320×240 pixels divided into

blocks 4×4 pixel and fames in the size 640×480 and more

divided into blocks 8×8 pixels. Block matching algorithm

attempt to divide both the previous and current frames

into such blocks and then compute the motion of these

blocks. Our implementation uses a search in three

directions of the original block ,

prev

x yb (in the previous

frame) and compares the candidate new blocks 1, 1

curr

x yb ,

, 1

curr

x yb and 1, 1

curr

x yb (in the current frame) with the original

(Fig. 2).

Fig.2 – Search for moving vectors

This comparison is calculated as follows:

, ,{ 1;0;1}

, , 1 , [2; ]

, ,

min( , )( , )

max( , )

prev curr

i j i jprev curr k

x y x k y x y N prev curr

i j i j

I IF b b

I I

, (1)

where ,

prev

i jI is the intensity value of pixel on the

previous frame, belonging to the block ,

prev

x yb , ,

curr

i jI is the

intensity value of pixel on the current frame, belonging to

the block ,

curr

x yb , N is count of blocks into which divided

the previous and current frame.

The result of this step is a binary mask of motion

detection MM on the current frame, where a value 1

corresponding the maximum value F .

2.2 Contrast analysis

The following step of algorithm is classification the

moving blobs based on the contrast analysis. For a current

frame the contrast mask CM is obtained as follows:

,

, , [1; ]

,

min( )( )

max( )

curr

i jcurr

x y x y N curr

i j

ICM b

I

, (2)

where ,

curr

i jI is the intensity value of pixel on the current

frame, belonging to the block ,

curr

x yb ; N is count of blocks

into which divided the current frame.

Then average value of contrast on all blocks is

calculated:

1x x 1x

1y

y

x

Current frame

Previous frame

No

Yes

No Yes

Masks combination by means of

logical operation AND

if Time = Acc_Time?

Update MHI Accumulate

MHI

Binary mask M

Connected component

analysis

MHI

Find Smoke?

Connected BLOBs

Alarm

Current frame

Previous frame

Optical flow

calculation

Contrast

analysis

Contrast mask

for previous

frame

Contrast mask

for current

frame

Calculate the

absolute

difference

Binary mask of

moving

Binary mask of absolute

difference

1

N

i

i

CM

TN

. (3)

Average value of contrast is used as a threshold to

obtain a binary contrast mask BCM of a current frame:

1, ,

0, ,

currcurr ii

if CM TBCM

othewise

(4)

where [1; ]i N .

To find moving areas with the lowered contrast the

absolute difference between binary masks of the previous

and current frames is calculated as follows:

curr prev

i i iBCM BCM BCM , (5)

where [1; ]i N .

Figure 3 shows the previous and current frames and its

corresponding masks prevBCM , currBCM and BCM .

(a)

(b)

(c)

Fig.3 – Previous (a) and current (b) frames with moving

vectors, the absolute difference between binary contrast

masks (c)

2.3 Change detection and accumulate motion

history

The following step of algorithm is combining of

masks MM and BCM by means of logic operation

AND as follows:

&i i iM MM BCM , (6)

where [1; ]i N .

For obtaining of more filled areas there is a record the

history of previous movement and thus are referred to as

the motion history image MHI . In our realization time of

accumulation MHI makes 2 seconds.

Figure 4 shows some examples for this step of

algorithm.

(a) (b)

(c) (d)

Fig. 4 - The binary mask of moving (a), the absolute

difference between binary contrast masks (b) of the previous

and current frames and their combining (c) by means of

logical operation AND, motion history image MHI (d)

2.4 Connected component analysis

On last step of algorithm for clearing of noise and

connection of moving blobs the connected components

analysis is used [9]. This form of analysis takes in a noisy

input mask MHI . It then uses the morphological

operation open to shrink areas of small noise close to 0

followed by the morphological operation close to rebuild

the area of surviving components that was lost in opening.

Then search of all contours is carried out. Next it tosses

the contours that are too small and approximate the rest

with polygons. The figure 5 shows the results of

connected components analysis to clean up the noisy

input mask MHI .

(a) (b)

Fig.5 – The noisy input mask MHI (a) is completely clean

up (b) by the connected components analysis

3. Experimental results

All experiments are implemented on personal

computer (Pentium(R) DualCore CPU T4300, 2,1 GHz,

ОЗУ 1,96GB). Our program is implemented using Visual

C++ and an open source computer vision library

OpenCV. The proposed algorithm has been evaluated

using data set publicly available at the web address

http://signal.ee.bilkent.edu.tr/VisiFire/Demo/SampleClips

.html. Figure 6 shows some examples of smoke detection.

Fig. 6 – Smoke detection in real video sequences

4. Conclusion and feature work

Results of researches show, that the algorithm

provides stable smoke detection on a dynamic

background. Herewith practically there are no false

alarms on movement of transport, people, a dither of leaf

etc. Smoke detection is achieved in real time. The

processing time per frame is about 31 ms. for frames with

sizes of 320 by 240 pixels.

However when the smoke is strongly rarefied also its

distribution occurs on low contrast background qualitative

characteristics of algorithm decrease. In the feature work

the algorithm needs improvements of qualitative

characteristics and the analysis in frequency domain.

References

[1] P. Piccinini, S. Calderara, and R. Cucchiara,

“Reliable smoke detection system in the domains of

image energy and color”, in 15th

International Conference

on Image Processing (ICIP 2008), San Diego, California,

USA, 12-15 October, 2008, pp. 1376-1379.

[2] B.Ugur Toreyin et al, "Wavelet based real-time

smoke detection in video," Signal Processing: Image

Communication, EURASIP, Elsevier, vol. 20, 2005, pp.

255-26.

[3] DongKeun Kim and Yuan-Fang Wang “Smoke

Detection in Video”, World Congress on Computer

Science and Information Engineering (CSIE 2009), Los

Angeles, USA, March 31 - April 2, 2009, pp. 759-763.

[4] B. Ugur Toreyin, Yigithan Dedeoglu, and A. Enis

Cetin “Contour based smoke detection in video using

wavelets”, In European Signal Processing Conference

(EUSIPCO 2006), Florence, Italy, September 4-8, 2006,

5p.

[5] F. Comez-Rodriuez et al, “Smoke Monitoring and

measurement Using Image Processing. Application to

Forest Fires”, Automatic Target Recognation XIII,

Proceedings of SPIE Vol. 5094, pp. 404-411, 2003.

[6] D.Krstinić, T.Jakovčević, D.Stipaničev, Histogram-

Based Smoke Segmentation in Forest Fire Detection

System, Information Technology and Control, Vol.38,

No.3, 2009, pp. 237-244.

[7] Turgay Çelik, Hüseyin Özkaramanli, and Hasan

Demirel, Fire and smoke detection without sensors: Image

processing based approach, 15th European Signal

Processing Conference (EUSIPCO 2007), Poznan,

Poland, September 3-7, 2007, pp. 1794 – 1798.

[8] Rubaiyat Yasmin Detection of Smoke Propagation

Direction Using Color Video Sequences, International

Journal of Soft Computing 4 (1), 2009, pp.45-48.

[9] Bradski, G. Learning OpenCV / G. Bradski, A.

Kaehler, – O'Reilly Media, 2008, 576p.