89

VIDEO-BASED TRAFFIC MONITORING

M Kilger

Siemens AG, FRG

ABSTRACT A video-based system for traffic monitoring (e.g. on highways or near crossroads) is presented. The objective of the system is to set up a high-level description of the traffic scene, comprising the position, speed and class of vehicles in a range of view between 0 and about 100 meters. The algorithms run in real-time on a low-cost system (386-PC with a DSP plug-in board).

INTRODUCTION Today’s road traffic demands more efficient and intelligent management and control strategies. As a prerequisite, on- line traffic data acquisition is necessary. Typical requirements a re [81: - -

-

wide range ofview of a t least lOOm should be monitored, vehicles should be counted, tracked and classified, the speed and position of each vehicle should be reported rather exactly.

Conventional techniques (e.g. inductive loop) cannot provide these data, if costs have to be kept within reasonable limits. In this paper a video nsed traffic monitoring system IS presented. Several other authors ([41,[51,~61.[71.[81 in Europe and some others in Japan) have presented similar work, but have not fully met the above requirements. No system is able to track and classify multiple vehicles in real-time with high resolution and without using special dedicated hardware. This work shows that real-time traffic monitoring is possible even with general-purpose hardware. A series of problems has to be addressed: -

- - difficult illumination conditions, - real-time processing, - cost aspects. On the other hand, the envisioned application allows to take advantage of a couple of assumptions: - only objects, which are moving or have been moving in the

past have to be registered, the movement of the vehicles can be modelled using simple physical laws, the shape of the vehicles can be modelled using simple geometric models.

- only part of the image (the traffic region) is of interest. Information accumulated over a sequence of frames may be used. In Fig.1 the main processing steps a re shown. For some steps we provide several algorithms, which diiTer in memory and

great variability of the possible scenes, partial occlusion of vehicles in heavy traffic conditions,

-

-

timing requirements and are tailored to different situations. According to the situation we select the most appropriate of the algorithms.

ALGORITHMS

image sequence Q I detection of moving objects I

cluster resolution 25 object list

ri tracking

track list * classification

Fig.1: Flow chart

Detection of moving objects Moving objects are detected very reliably by comparing the current image with a background image. A robust background extraction routine is used ([11,[21,[31). The whole detection algorithm proceeds in several steps:

. Subtraction ofcurrent image from background image. A threshold decision which separates regions with fast motion from regions with no or slow motion, is made and delivers a binary object mask. Of course the properties of the object mask depend critically on the chosen threshold value. If the threshold is too high, the mask will often be fragmented (Fig.3). In the opposite case, lots of uninteresting movements will be detected, like the rustle of the leaves. This problem can be overcome by continously adapting the threshold.

- Segmentation ofthe object mask.

90

A segment list is built up. For each connected component the bounding box, the area in the image and an estimation of the corresponding area in the real world a re computed. The latter is derived using the inverse perspective mapping under the assumption that vehicles move on the road at height zero. Removal of segments that are unlikely to be related to vehicles. We remove segments which are off the traffic regions and segments, which are too small. Update of the background image. Outside a mask, which corresponds to moving objects in the image, an adaptation routine is applied. It is based on recursive filtering with variable coefficients. The mask consists of the bounding boxes of individual vehicles (see below). They represent the objects occluding the background more precisely than the object mask originally created in step 1.

-

-

Fig.2: Current image

Cluster resolution A special problem is the resolution of clusters into single vehicles in heavy traffic situations where vehicles may partially occlude each other. Under the following assumptions a solution is found: - daylight conditions, diffuse illumination, - the camera is positioned to the side of the road, e.g. on a

pole for the traffic light or on a bridge. The first step is the detection ofclusters within the segments. If the bounding box is greater than a defined limit, the segment probably is a cluster. Next the clusters have to be resolved into single vehicles. Under our assumptions parts of the leading edges can be seen as prominent feature ofthe vehicles. These parts are detected

Fig.3: Object mask

Fig.4Object mask plus the bounding box

and then matched with a geometric model. As geometric models for vehicles in the image, bounding boxes turn out to be sufficient.

Furthermore the knowledge about the past is utilized. The vehicles moving off the camera usually have been seen isolated in the past and only later join into clusters. Hence, this information can be used. On the other hand. vehicles which move towards the camera are first seen within clusters, which then gradually resolve. In this case, the resolution algorithm, which takes its main information from one image frame, is effectively used. Plausibility values for the result of the resolution are built up. Segments which belong to the same object have to be joined. Here the same information as for resolving clusters is used.

91

Fig.5: Updated background image

Q Fig.6: model arrangement

The results ofthis processing step are stored in an object list.

'ig.8: Object mask plus the segments aRer the resolution

Tracking The main task of the tracking algorithm is to track the detected objects and to predict their respective positions in the next image frame. In the track list, the whole description of the scene is stored. Each track has the following attributes:

- current position, -

- current speed, - - width, - plausibility. The tracking algorithm comprises the following subtasks: - Assign detected moving objects to existing tracks:

The position of an object is defined as the centre of the leading edge (see above!). The object that is closest to the predicted position of a track is assigned to tha t track.

A new track is built, if the detected object cannot be assigned to a n existing track.

If, in the current cycle, an object cannot be assigned to an existing track, the track is extrapolated via a Kalman- filter prediction. Ifthe extrapolated position is outside the considered scene, the track is removed. Otherwise the plausibility of the track is reduced. If the plausibility is lower than a defined limit, the track is also removed. Predict the position and the speed for the next cycle: The position and speed of the tracks are filtered in world coordinates using a Kalman-filter. The use of Kalman- filters implies a smoothing of the measurements and provides a n estimation of position and speed. Experiments have shown that a constant Kalman gain is sufficient.

predicted position for the next image frame,

predicted speed for the nest image frame,

- Create new tracks:

- Remove tracks:

-

Classification The vehicles have to be classified according to: - trucks, - cars, - motorcyles. bicycles. The selection of the features best suited for the classification depends on the situation. If the vehicle is moving towards the camera, the first feature to evaluate is the width of the bounding box. This width is tracked over the sequence and an estimation over several frames is made. If a rotation of the vehicle is detected by the tracking algorithm, the leading edge is used as suitable feature. This leading edge again is tracked over several frames and an estimation of the width is made. This predicted position is used for feature tracking. Another useful feature to evaluate is the height of the bounding box. These measurements are filtered with a Kalman-filter and the length and height of the vehicle are calculated.

IMPLEMENTATION The following input format turns out to be appropriate: -

- grey levels: 8 bit -

The algorithms were first implemented and tested on a SUN Sparcstation 1. Real-time processing is now done on a personal computer (PC) with an Intel i386 processor a s host, a digital signal processor (Motorola DSP96002) plug-in board and a frame grabber. The

resolution: 256 x 256 pixel

frame rate: 8 1i3 Hz

92

low-level image processing (background extraction and segmentation) is done by the DSP. The host operates on the segment list and carries out the high-level algorithms. In addition i t supervises the communication between the subsystems.

RESULTS AND CONCLUSION The algorithms were tested for various traffic and illumination conditions. For low to medium traffic the clusters can be successfully resolved. The errors in estimating position and speed were smaller than 3% Current work follows several lines: - Automatic adaptation of parameters (e.g. camera

parameters), Learning of the scene geometry,

routines (e.g. background adaptation and segmentation),

illumination conditions.

- - Feedback from high-level to low and medium level

- Switching between various algorithms depending on

REFERENCES 111 W.Feiten. A.v.Brandt, G.Lawitzky, LLeuthausser . Ein

videobasiertes System zur Erfassung von Verkehrsdaten. Proc. 13th DAGM 1991, Munich, pp. 507-514 K.P.Karmann. A.v.Brandt. Moving object segmentation based on adaptive reference images. Proc. of EUSIPCO 1990, Barcelona

[31 R.Ger1. Detektion bewegter Objekte in digitalen Bildfolgen natiirlicher Szenen mit unbewegtem Hintergrund. 1990, Diplomarbeit TU Munich A. Bielik. T. Abramczuk. Real-time wide-area traffic monitoring: information reduction and model-based approach. Proc. 6th Scandinavian Conference on Image Analysis, 1989, Oulu, Finland, pp.1223-1230

[51 S.Beucher, J.M.Blosseville, F.Lenoir. Traffic Spatial Measurements Using Video Image Processing. SPIE Vo1.848 Intelligent Robots and Computer Vision: Sixth in a Series (1987). pp. 648-655

[61 N.Hoose, L.G.Willumsen. Automatically extracting traffic data from videotape using the CLIP4 parallel image processor. Patt. Recog. Letters, 6. 1987. pp. 199- 213

(71 A.D.Houghton, GSHobson, N.L.Seed. R.C.Tozer. Automatic vehicle recognition. 1989, University of Sheffield, U.K.

[81 W.Leutzbach, H.P.Bahr, U.Becker, T.Vogtle. Entwicklung eines Systems zur Erfassung von Verkehrsdaten mittels photogrammetrischer Aufnahmeverfahren und Verfahren zur automatischen Bildauswertung. 1987, Technischer Schldbericht, University Karlsruhe

121

I41