1-s2.0-016786559500081x-main.pdf

ELSEVIER Pattern Recognition Letters 16 (1995) 1321-1330

Pattern Recognition Letters

An image detection technique based on morphological edge detection and background differencing for real-time

traffic analysis

M. Fathy a, M.Y. Siyal b,, a Dept. of Computer Engineering, Iran University ofSc. & Technology, Tehran, Iran

b School of Electrical and Electronic Engineering, Nanyang Technological University, Nanyang Avenue, Singapore 2263, Singapore

Received 24 March 1995; revised 20 June 1995

Abstract

The real-time vehicle detection from a traffic scene is the major process in image processing based traffic data collection and analysis techniques. The most common algorithm used for real-time vehicle detection is based on background differencing and thresholding operations. The efficiency of this method of image detection is heavily dependent on the background updating and threshold selection techniques. In this paper, a new background updating and a dynamic threshold selection technique is presented. An alternative image detection technique used in image processing is based on edge detection techniques. However, an edge detector extracts the edges of the objects of a scene irrespective of whether it belongs to the background details or the objects. Therefore, to separate these two, extra information is required. We have developed a new image detection method based on background differencing and edge detection techniques, which separates the objects from their backgrounds and works well under various lighting and weather conditions. This image detection technique together with other techniques for calculating traffic parameters e.g. counting number of vehicles, works in real-time on an 80386-based microcomputer operating at a clock speed of 33 MHz.

1. Introduction

In recent years, extensive research and develop- ment efforts have been devoted to image processing techniques applied to traffic data collection and analysis (Hoose, 1991, 1992). For real-time traffic image processing applications, the sequence of images have to be processed at a rate of 25 frames/second, which produces a large amount of data. Therefore, the image processing algorithms have to be simple but effective so that they can be executed in real-time.

* Corresponding author. Email: [email protected]

The first image processing algorithm required to extract traffic parameters, is an image detection technique. A common and simple image detection approach used in traffic application is the background differencing technique. This approach is based on pixel-by-pixel comparison of a background picture and the current frame of the scene and has been used by various researchers in traffic applications (Dickin- son and Waterfall, 1984a,b). In practice the effectiveness of this method depends mostly on the accuracy of the background updating technique and the selection of a suitable threshold value. In this paper we introduce an effective and low-cost background

0167-8655/95/$09.50 © 1995 Elsevier Science B.V. All rights reserved SSDI 0 1 6 7 - 8 6 5 5 ( 9 5 ) 0 0 0 8 1 - X

1322 M. Fathy, M.Y. Siyal / Pattern Recognition Letters 16 (1995) 1321-1330

updating technique and a dynamic threshold selection technique.

An alternative image detection technique used in image processing is based on the edge detection technique. Edge-based image detection is generally more effective than background differencing and has been used by few researchers in traffic applications (Hoose, 1991). In fact, the edge information of the objects remains still significant despite the variation of the ambient lighting. The conventional gradient- based edge detection operations have found wide acceptance in image processing applications. How- ever, morphological edge detectors have shown better performance than conventional edge detectors while having a lower computational cost (Lee et al., 1987; Fathy et al., 1994). In this paper, a simple and effective morphological edge detector using the decomposition property of the morphological operators and a separable median filtering is presented.

An edge detector extracts the edges of the objects of a scene irrespective of whether it belongs to the background details or the objects. Therefore, to separate these two, extra information is required. This extra information can be obtained from the edges of the background picture. We have developed a new image detection technique based on background differencing and edge detection techniques, which separates the objects from their backgrounds. This image detection technique together with other techniques for calculating traffic parameters e.g. counting the number of vehicles, works in real-time on an 80386- based microcomputer operating at a clock speed of 33 MHz.

2. Background updating

One of the main problems associated with the background differencing technique, is the change in lighting conditions, which makes the current background frame invalid. These temporal changes occur even on overcast days and the problem becomes further complicated by spatial changes caused by emergence and disappearance of shadows on bright days. Consequently, background updating has to be performed whenever the illumination level changes. Interrupting a real-world situation, whenever a background frame has to be updated is not always possi-

ble. Therefore, the updating of the background frame has to be automatic and concurrent with the processing if real-world traffic scenes are to be analyzed.

In the background differencing approach, the current picture (Cpt) is subtracted from the background picture (Bpt), resulting in a difference image (Dpl). The differencing operation is normally an absolute differencing operation, to detect vehicles having brighter or darker grey-value than the background. Each pixel of Dpl is compared with a threshold value (T) and a binary picture is generated (Bpt). If the result of subtraction of Dpl and T is greater than zero, the corresponding pixel of Bpt becomes 1, otherwise 0. These operations can be summarized as follows.

For each pixel of Dpl, Cpt , Bpt do Opl = I Cpt - Bpt I If Dpl ~> T Then

Bpt = i (object) Else

Bpt = 0 (no object)

To make the background differencing technique more effective, the changes in ambient lighting must be compensated by an effective background updating technique. Currently several operational traffic surveillance systems have been constructed by using various background updating techniques (Seed and Houghton, 1988). The most commonly used background updating techniques are based on averaging (Siyal et al., 1994; Seed and Houghton, 1988) or selective techniques (TRIP Project Report, 1986; Rourke and Bell, 1988).

2.1. Frame averaging technique

In this method the average of a number of input images in a period of time is referred to as an updated background image. The general form of averaging is not easily implementable in real-time. Alternatively, if the weighting factors of all pictures used for averaging are identical, then the averaging of N + 1 frames can be performed using the following equation (Hoose, 1992; Takaba et al., 1984):

Bpt = K B p t _ 1 + (1 - K ) Cpt_l,

0 < K < 1 and K = N / ( N + 1) (1)

M. Fathy, M.Y. Siyal / Pattern Recognition Letters 16 (1995) 1321-1330 1323

where Bpt is the updated background picture, Bpt_ 1 is the previous background picture and Cpt_ 1 is the previous frame of the scene.

The above equation is also known as the exponen- tial updating technique and is practically implementable in real-time. The value of K determines the rate of updating. When K approaches to 1, the new background becomes closer to the previous one and the effect of the current picture is reduced. The appropriate value of K depends on the ambient lighting conditions and is often adjusted manually.

2.2. Selective updating technique

Ambient changes are usually smaller than the changes due to the objects. This implies that those sections of the scene which are not covered by moving objects should only be updated. The updating process is performed simply by replacing the background pixel values by the current pixel values at selected pixel points. This selective updating technique can be expressed as follows (Hoose, 1992; Seed and Houghton, 1988):

I f Dpl > T Then

Bpt = Bpt- 1 (don't update) Else

Bpt = Cpt 1 (update)

2.3. A new background updating technique

The drawback of the averaging technique is the quick response to the regions where there is a significant difference between the background and the current picture, while this difference is mostly due to the objects. Also, the proper setting of the K value is not an easy task. The effectiveness of the selective updating technique depends on the accuracy of the

threshold value. If the threshold value is not selected properly, the object pixels are miss-classified as the background pixels, and the background picture becomes quickly unusable.

The proposed updating technique uses the intensity changes between two consecutive frames as a measure of the ambient lighting variations. If the difference of the corresponding pixels of two consecutive frames is less than the maximum expected variation of intensity between the period of two consecutive frames, the updating is performed, pro- vided that no object is detected. The values of T 1 and T 2 are selected automatically by analyzing the histograms of Dpl and Dp2 for a number of frames (see Section 3). This updating operation can be expressed as follows.

For each pixel of Cpt and Bpt do I f [Dpl = [ C p t - Bpt 1] < T 1 Then If [ Dp2 = [ Cpt - Cpt + 1 [ ] < T2 Then

Bpt +1 = (Bpt + Cpt + 1 ) / 2 (update) Else

Bpt + 1 = Bpt (no update)

The operations of the above equation under various traffic conditions are shown in Table 1.

This updating technique is based on replacing the background of the selected points (see column 2 of Table 1) by the average of the current and background picture points, instead of directly replacing the background points by the current picture points. Thus, the problem of miss-classification of the objects as background in selective updating is significantly reduced, as the averaging of the Cpt and Bpt is used as a new Bpt and also updating is performed when no object on the current and the previous frame is detected. The following points should be noted regarding the selective-averaging technique.

Table 1

Dpl >/T 1 No No Yes Yes Dp2 >/T 2 No Yes No Yes Updating background Yes No No No Object detected No No Yes Yes Status of the traffic at Empty road Motion (vehicle Stopped Moving the pixel to be tested leaving the scene) objects objects

1324 M. Fathy, M.Y. Siyal/Pattern Recognition Letters 16 (1995) 1321-1330

(a) The computation of Dpl and Dp2 imposes no computational overhead as these two image differ- ences are used for object detection and motion detection respectively.

(b) Instead of linear averaging of the Bpt and Cpt frames, a weighting factor can be used. However, linear averaging has a lower computational cost and in most lighting conditions gives better results, as both Cpt and Bpt are accounted equally.

(c) This updating method enjoys the advantages of both averaging and selective techniques, and it is less sensitive to the threshold selection technique. However, we use a dynamic threshold selection technique to improve the effectiveness of this updating technique.

3. A dynamic threshold selection technique

The selection of the threshold value is important as a low threshold value will classify many objects as the background and a high threshold value might not be able to detect some pixels of the objects. The method proposed here is based on analyzing the histograms of windows of the difference images (Dpl or Dp2) for a period of time. The window is located across the lane, to detect vehicles passing through the road. This method is expandable to the local thresholding technique, by locating a window at each local area of the scene.

Following the computation of Do1 and Dp2, the histograms of the windows of these images are computed and the left-limit grey-value of each histogram is extracted. The left-limit value is a grey-value of the histogram, where there are approximately zero number of pixels having higher grey-value. When the window contains an object, the left limit of the histogram shifts toward the maximum grey-value, otherwise it shifts towards the origin. The minimum of these left limits for a period of 10 minutes is selected for the next period of processing. This minimum value is used to evaluate the number of pixels in Dpl or Do2 whose brightness is significant enough and in most cases can be set arbitrarily between 0-30 in a 256 grey-level system to speed up the operation.

For threshold selection, the number of difference points greater than the left-limit grey-value of each

200 20O

17S 175

150 ~.50

(a) (b)

Fig. 1. The histogram of the left-limit of histogram of 200 frames (a) before applying median filtering, (b) after applying median filtering.

window is extracted for a large number of frames (200 frames). These numbers are used to create an array (Fig. l(a)), where its horizontal axis corresponds to the number of points and its vertical axis corresponds to the number of frames. The histogram of Fig. l(a) is smoothed by using a median filter and a point in the valley of the smoothed histogram is selected as the threshold value. The smoothed histogram is shown in Fig. l(b).

The histogram of Fig. l(b) indicates that if we select the threshold value between 30 and 40, there is a possibility of detecting vehicles of which only a small part passes the window. The program selects the threshold value between 40 and 60 to detect vehicles of which the most part passes the window. The above operations are performed on both Dpl and Dp2 and T 1 and T 2 are extracted, respectively.

This threshold selection operation consumes very low computational power due to the small size of the window required for this purpose. Also, by choosing the same window for both traffic analysis and threshold selection, the time for generating the array of Fig. l(a) is significantly reduced to O(N), where N is the number of frames (200) in a 10 minutes interval. In other cases, O(N) operation is required to generate the histogram, where N is the number of points in the window.

4. A separable morphological edge detector

The background differencing technique discussed earlier is effective for many full-frame or window- based image analysis techniques. However, to reduce the effect of background updating for detecting ob-


jects, an edge-based segmentation technique which is less sensitive to the variation of ambient lighting can be used.

Few morphological edge detectors have been developed for both intensity (Krishnapuram and Gupta, 1992) and range images (Lee et al. 1987; Fathy, 1990). In the case of intensity images, a fiat-top structuring element (Fathy et al., 1994) is considered to analyze the operations of different edge detectors. However, for range images, special shapes of structuring elements are used to detect different types of edges. In this paper, we concentrate on intensity images.

A suitable noise removal filter which has proven to preserve edges, is median filtering. A version of this filter named as "Separable Median Filtering" has shown comparable performance to the median filter but has faster computer implementation (Fathy et al., 1994; Fathy, 1990). Indeed, a separable median filtering can be accomplished by running a one-dimensional median filtering along any horizontal line and then along any vertical line in the image. By applying a decomposition technique, the manner in which a separable median filter is accomplished is identical to dilation and erosion, except on selecting median values instead of Max or Min values. Com- pared with other edge detection operators, the SMED operator has the lowest computational requirement and thus is more suitable for real-time image processing applications. This operator can be defined as

SMED = [O(S(f)) -g (s ( f ) ) ] (2)

where S(f) is the result of applying a separable median filtering to the horizontal image f. D 0 and E0 are dilation and erosion operators, respectively.

4.1. Evaluation of SMED operator

To compare the performance of the SMED operator with other edge detectors, a test pattern employed by other researchers to evaluate the performance of similar detectors has been used. The test pattern is a checker board comprising of dark and light squares with grey-values of 50 and 100 respectively. To measure the performance of different operators, noise with gaussian distribution of zero mean and different standard deviations was added to the test pattern. The criterion used to compare the operators was

" /ZI !I11171 0.8 . . . . . .

0 .5 - - - i

0.4 1.'66 2 2:5 3.'33 4_ 5.'54 5_ 5.55 6/25 6.'66 sC,Cr~ ~ noCse rat¢o

-~t-- open-dose -+-- ATM -:~ - blv, -rrlln ~ srned ede

F i g . 2. P e r f o r m a n c e c o m p a r i s o n (3 x 3 ) .

conditional probability of the true edge over the detected edge, P(T/D), and the detected edge over the true edge, P(D/T). In this method, the threshold values are selected in such a way that P(D/T) = P ( T / D ) .

The plots of probability versus signal to noise ratio (SNR) for the edge detectors are shown in Figs. 2 and 3. Fig. 2 shows that the Open-close and SMED operators have relatively better performance than the ATM and Blur-min operators. When SNR is large (SNR > 4), the performance of the SMED operator is closer to the Open-close operator. Fig. 3 shows that in the case of 5 x 5 neighborhood size, the performance of SMED is better than of ATM and Blur-min and is much closer to the 3 X 3 Open-close operator (dotted line). It should be noted that the

095 ........... ~ . . . . . . . . . . . . .

o~54 - ; , / ' - ~ . . . . . . . . . . . . . . . . . . . .

0 7 5

OB5

~stnal t~n~atla open-close -4 - - ATM - ~ - blur-rain - ~ - smed-ede

Fig. 3. Performance comparison (5 × 5).


Fig. 4. Applying the SMED operator to a section of a traffic scene.

computational cost of a 3 X 3 Open-close operator is about 8 times of a 3 X 3 SMED operator. The result of application of a 3 X 3 SMED operator on a real- world traffic scene is shown in Fig. 4.

4.2. Reasons for choosing SMED

The reasons for adopting the SMED operator in our application are

(a) SMED is based on summation of erosion-residue (Er) and dilation-residue (Dr) operators (Ede), which can detect edges at different angles, while

@

= l - e a ~

Fig. 5. Block diagram of the combined image detection technique.

other morphological operators (except Open-close) use Er, Dr or the minimum of these which are unable to detect some kind of edges.

(b) The strength of the edges detected by SMED is twice that of either Dr or Er, which has been used in other edge detectors. ATM and Blur-min use Er and are unable to detect thin lines and corners and the strength of the edges detected is reduced.

(c) SMED uses separable median filtering for noise removal. Median filtering has proven to preserve edges while removing impulse noises. Separa- ble median filtering has shown to have performance comparable to the true median filtering but requires less computer power.

In brief, SMED uses compatible and easily implementable operators and has a lower computational requirement, compared to the other morphological edge-detection operators. Open-close has better performance than the SMED operator, but it has about 8 times computational power requirement, so it is not very suitable for real-time image processing applications.

5. A combined edge detection and background differencing technique

The application of the SMED operator on an image, can be used to extract the edges of the image.


However, to distinguish between the edges of the desired objects and the edges of the background details, extra information is required. This extra information can be obtained by using a background picture containing only the stationary objects. Thus, to extract the edges of the desired objects, an image detection technique based on combining the SMED edge detector and the selective-averaging background differencing technique is used.

In this method the edges of the current scene and the background scene are obtained by applying a SMED operator, and then the edges of the desired objects are obtained by subtracting the edges of the current and the background scenes. This process is shown in Fig. 4. The main features of this image detection technique are as follows.

(a) As the segmentation is based on edge detection, the effect of the ambient lighting is reduced.

(b) Despite the insensitivity of the algorithm to the background updating technique, we use a low-cost and effective background updating and threshold selection technique to eliminate the effect of the intensity changes of the scene entirely.

The low computational cost and effectiveness of the SMED operator make the proposed algorithm

suitable for real-time applications. The proposed algorithm has been applied on full-frame and window-based image detection techniques using a DT2867 (Data Translation Manual, 1993) frame grabber board and an 80386 microcomputer system. The DT2867 board allows the capture of more than 4 frames/second which is suitable for our applications. It has got special hardware for convolution-type operations, which were not used in our application.

As stated earlier, the difference of two consecutive frames is used for motion detection and updating operations. As shown in Table 1, if Dp2 is greater than T2, a motion is detected for that point. Now, if the number of moving points in an area is significant, a motion condition is detected, otherwise a stop condition is detected in the area. Therefore, it can be said that the proposed algorithm can be used for a wide range of full-frame or window-based image-detection and motion-detection techniques.

6. Counting number of vehicles

The vehicle-detection operation discussed above is used to measure other traffic parameters. To count the number of vehicles passing through a window, a

Fig. 6. Vehicle detection using selective background updating technique.


Fig. 7. Vehicle detection using selective-averaging background updating technique.

status vector is created for the window. In this manner, for each frame, if a vehicle is detected in the window, a ' 1 ' is stored at the status vector of the

window, otherwise, a ' 0 ' is stored at the status vector of the window. A group of ls (ones) corresponds to a vehicle and a group of 0s (zeros) corre-

(a) (c)

(b) (d) Fig. 8. Operations of the proposed image detection technique.

M. Fathy, M.Y. Siyal / Pattern Recognition Letters 16 (1995) 1321-1330

I III Manual ra Corfiputer I

1329

Rai~

Sno'

loo

Fig. 9. Output of the vehicle count for a period of 90 minutes.

500

sponds to the distance between two vehicles. A one alone could be considered as a motorcycle or a fast moving car.

7. Results

The algorithms were tested under various traffic and lighting conditions using an 80386-based per- sonal computer system operating at a clock speed of 33 MHz. The algorithm has been implemented in a user-friendly package written in C language for MS Windows 3.1 environment. The package allows the user to define regions and the result of the operations are reported graphically on-line by user-defined peri- ods.

In the photographs of Figs. 6 and 7, there are four pictures illustrating the effects of the various processing stages. The first, in the top left comer is the current frame, while the updated background frame is given at the bottom left corner. The thresholded binary picture is shown in the top right comer, while the final binary picture (without/with reduced noise) is positioned at the bottom right quadrant.

In Fig. 6, the selective background updating technique was used, while in Fig. 7, the new background updating technique (selective-averaging) was employed. As can be seen, the selective background updating technique was unable to remove the shadows, which became part of the vehicle and will

complicate the task of vehicle tracking. On the other hand, the selective-averaging background updating technique was able to remove the shadows and detect the vehicles as well.

The operation of the proposed image detection algorithm on a traffic image is shown in Fig. 8. Fig. 8(a) shows the current picture of the scene. The thresholded edges of the background are shown in Fig. 8(b). The binary picture resulting from the subtraction of the edges of the background and the current picture is shown in Fig. 8(c). The final binary image (Fig. 8(d)) is obtained by applying noise removal operations.

The result of testing shows that the image detection algorithm can detect vehicles passing through windows nearly free of errors, and if the vehicles move in their own lane, the counting program also counts the vehicles with about 100% accuracy. How- ever, in this particular situation, drivers didn't move in their own lanes, therefore, up to 5% error is produced due to changing lane of vehicles. Fig. 9 shows the graphical output of the result of counting of vehicles for a period of 90 minutes under various weather conditions.

8. Conclusion

The problems associated with the background- based image detection techniques are mainly due to

1330 M. Fathy, M.Y. Siyal/Pattern Recognition Letters 16 (1995) 1321-1330

the variations of ambient lighting and threshold selection operations. In this paper an effective background updating technique based on automatic esti- mation of the ambient lighting changes and automatic selection of the pixels which should be updated is introduced. This selective-averaging updating technique has the advantages of the two most commonly used updating techniques, i.e., averaging and selective, while it eliminates the disadvantages of these two algorithms. We also implemented a dynamic threshold selection technique, which can be used for both local and global thresholding schemes. The proposed image detection technique has been tested under various lighting conditions and satisfac- tory results have been achieved.

In some situations a more robust image detection technique, which is not sensitive to the variations of ambient lighting, is preferable to a low-cost image detection technique. In those applications, a combined edge detection and background differencing technique can be used. A low-cost approach to this method using the morphological edge detection in conjunction with the proposed background updating technique is implemented. This algorithm has been used in a wide range for full-frame and window-based applications in real-time. The information computed during the operation of the proposed algorithm is also used for vehicle detection and motion detection, without imposing significant extra computational cost. This combination was used to eliminate the edges of unimportant parts of roads, like shadows of the trees, stopped cars, white road markings etc.

At present work is underway to extend the proposed vehicle-detection algorithm to calculate other traffic parameters such as speed of vehicles, type of

vehicles and traffic movements at traffic junctions e.g. length of the queue, occurrence of the queue etc.

References

Data Translation Manual (1993). Data Translation Inc., USA. Dickinson, K.W. and R.C. Waterfall (1984a). Image processing

applied to traffic: a general review. Traffic Engineering Con- trolJ., Jan., 6-13.

Dickinson, K.W. and R.C. Waterfall (1984b). Image processing applied to traffic: practical experiences. Traffic Engineering Control J., Feb., 60-67.

Fathy, M. (1990). A RISC-type programmable morphologic image processor. Ph.D. Thesis, UMIST, England.

Fathy, M., M.Y. Siyal and C.G. Darkin (1994). A low-cost approach to real-time morphological edge detection. Proc. IEEE TENCON Conf., Singapore.

Hoose, N. (1991). Computer Image Processing in Traffic Engi- neering. Research Studies Press, Taunton.

Hoose, N. (1992). IMPACT: an Image analysis tool for motorway analysis and surveillance. Traffic Engineering ControlJ., Mar., 140-147.

Krishnapuram, R. and S. Gupta (1992). Morphological methods and classification of edges in range images. J. Mathematical Imaging Vision 2, 351-375.

Lee, J. et al. (1987). Morphologic edge detection. IEEE J. Robot. Automat. 3 (2).

Rourke, A. and M.G.H. Bell (1988). Traffic analysis using low cost image processing. Proc. Seminar on Transportation Plan- ning Methods, PTRC, Bath, UK.

Seed, N.L. and A.D. Houghton (1988). Background updating for real-time processing at TV rates. SPIE 901, 73-81.

Siyal, M.Y., M. Fathy and C.G. Darkin (1994). Image processing algorithms for detecting moving objects. ICARCV'94, Vol. 3, 1719-1723.

Takaba, S. et al. (1984). A traffic flow measuring system using a solid state sensor. Proc. IEE Conf. on Road Traffic Data Collection, London, UK.

TRIP Project Report (1986). Investigation into video image processing applied to traffic data collection. Reports, May-March 1983, UMIST, England.

1-s2.0-016786559500081x-main.pdf

Documents