Download - TraVis CTTHES3

Introduction

TraVis: Web-Based Vehicle Counter with Traffic Congestion Estimation Using Computer Vision

Presented by: Aguirre, Byron Franco Alcantara, Jan Andre Trinidad, John Ferdinand

Thesis adviser: Dr. Joel P. Ilao

OutlineIntroduction

Problem Statement

General Objectives

Specific Objectives

Project Scope and Limitation

System Overview

System Implementation

Demo

Experiments

Conclusion

Problem Statement

Problem Statement

In the Philippines, many traffic cameras are being installed where recorded traffic videos are monitored by traffic personnel. However, these cameras might have been recording videos 24/7 but the people who monitor them might not be watching. The “unmonitored” videos then become unused since a traffic situation has already occurred in an area. With this, a traffic monitoring system that makes use of all recorded videos can provide information to the public regarding the traffic situation in an area.

Objectives

General Objective• The aim of this study is to develop a

vision-based system for counting vehicles and estimating traffic congestion levels in road sections installed with traffic surveillance cameras, accessible through a web interface.

Specific Objectives

● gather traffic videos taken by roadside traffic surveillance cameras;

● develop machine vision algorithms for processing traffic videos that can:o classify vehicles according to type;o track and count the number of vehicles

seen;

Specific Objectives (cont.)

● generate graphs of traffic congestion levels based on vehicle count statistics estimated from traffic videos;

● design a suitable database for efficiently storing traffic surveillance videos and corresponding traffic statistics; and

● develop an interactive web interface for accessing relevant data and information from traffic surveillance videos;

Project Scope and LimitationThis study will count and classify vehicles

seen in a traffic surveillance camera’s Field Of View via application of computer vision techniques. The system that will be developed can:● identify and classify vehicles

● distinguish adjacent vehicles

Project Scope and Limitation (cont.)

● estimate traffic congestion levels based on vehicle counts

● allow users to view results using an intuitive web interface


● no hardware implementation of video acquisition

● use traffic videos from Archer’s Eye


● Factors affecting the quality of counting:

o low quality videos

o slow frame rates

o variations in lighting

o occlusion


● Factors that are not considered

o Traffic at night

o Swerving vehicles


● IP Cameras were used● Videos obtained through ITS● Video file: .mp4 at 6 FPS


● Can detect:

Small Vehicles

Medium Vehicles

Large Vehicles

Car SUV Truck

Sedan Jeep Bus

Van


● Performance assessed by:

o Accuracy

● Against:

o Occlusion

o Number of vehicles present

System Overview

System Implementation

• Video Input• Invocation of Matlab Instance

Video Acquisition

Module

• Object Detection• Vehicle Tracking• Vehicle Classification• Vehicle Counting• Congestion Estimation

Vehicle Detection Module

Statistics Generation

Module

Architectural Design

• Video Acquisition

Video Frames / Image

Sequence

• Vehicle Detection

Vehicle Counts• Statistics

Generation


● Video Acquisition

● Vehicle Detection

● Statistics Generation

Video Acquisition

• Video Input

Traffic Video

• Frame Extraction

RGB and Gray scale

Frames • Video Frames

Video Acquisition Flowchart



o Video Input

o Frame Extraction



Video Input

The videos are stored locally and accessed by the server via the URL passed when the used has chosen a video.

Invoke Matlab process

After choosing a video to process, a java servlet invokes a Matlab instance to process the video. With the URL as the input, the servlet passes in the value of the video directory.



o Video Input

o Frame Extraction



Frame ExtractionThe input video is converted into frames to prepare them

for processing in the Vehicle Detection module.

Architectural Design● Video Acquisition● Vehicle Detection

o Object Detectiono Trackingo Classificationo Countingo Congestion Estimate


Vehicle Detection Module

Vehicle Detection Flowchart

Background Modeling and Object Detection Flowchart

Byron Aguirre

jan

• Initial background estimate(left)• Road mask from selected camera location(right)

Gray-scale frames

Foreground objects after subtraction

Foreground mask via threshold

Foreground filtered with road mask

Foreground mask applied with morphological operations

Foreground applied with edge detection

Application of edge detection

Applying a threshold to the image with threshold of 300 pixels in size.

Vehicle TrackingVehicle tracking is

performed by using the Kalman Filter to estimate the next locations of the vehicles. Once a vehicle is tracked, classification and counting follows.

Green box = detectedRed box = predicted

• Why Kalman filter?– Because of its consistent performance in the

previous researches referenced in this project.

– The future states of the vehicles, represented by their locations in the image are estimated using the Kalman filter via their centroids.

“An object at rest will remain at rest unless an external force acts upon it. An object in motion will not change its velocity unless an external

force acts upon it.”

• Why Kalman filter?– Because it makes use of Newton’s Law of

Motion, which is applicable to the project’s objectives and scope.

VehicleTracking

Flowchart

Vehicle Classification And Counting Flowchart

Vehicle ClassificationOnce potential vehicles

are tracked, they will be classified into different types such as small, medium and large vehicles.

*The basis of the sizes (in pixels) of the vehicle types were obtained by getting the area properties of the binary mask of a vehicle via the Matlab function, regionprops().

Types SizescarAreas.smallAreaMin 300

carAreas.smallAreaMax 800

carAreas.mediumAreaMin 801

carAreas.mediumAreaMax 5000

carAreas.largeAreaMin 5001

carAreas.largeAreaMax 45000

Vehicle CountingWhen the detected vehicles are classified into their types,

the system will now count their number in preparation for the traffic congestion estimation.

*A tracked vehicle’s count was done on its first detection in the scene.

Counts: Manual vs System

August 26, 2014 Manual Count System Count Small Medium Large Small Medium Larg

e0700-0800 130 191 14 267 354 210800-0900 167 210 17 280 372 190900-1000 365 287 22 252 289 231000-1100 309 241 21 268 227 181100-1200 330 288 17 277 271 15

August 25, 2014 Manual Count System Count Small Medium Large Small Medium Larg

e0700-0800 147 248 11 190 130 50800-0900 179 273 18 143 290 60900-1000 239 304 15 312 258 141000-1100 209 366 14 408 669 291100-1200 320 379 12 552 638 22

Counts: Manual vs System (cont.)

One Time BG Model

Frequent BG Model

Manual Count

0600-0630 308 153 1900630-0700 387 223 1980700-0730 73 173 2140730-0800 206 152 1920800-0830 67 167 2220830-0900 101 272 2480900-0930 30 349 2940930-1000 276 235 264

May 23, 2014, North Gate applying multiplying factor values Manual Count Applying Mult Factor Smal

lMedium

Large

Small

Medium

Large

0600-0700 71 70 24 109 108 460700-0800 131 142 18 202 219 350800-0900 131 121 35 202 189 670900-1000 133 134 45 205 206 861200-1300 142 145 47 219 223 90

Congestion Estimation● Traffic congestion will be estimated by making use of

the number of vehicles per type that are present in the scene and the number of vehicles that have passed in the area in 5 minutes.

● Traffic congestion varies with time so the vehicle counts and types per 5 minutes in the video will be used to estimate the congestion.

Congestion Estimation (cont.)Summarized statistics such as the counts of the vehicles types, number of vehicles in the scene in five minutes and the traffic congestion estimate are inserted into the database after five minutes in the video has passed.

Congestion Estimation (cont.)

Statistics Generation Module● Traffic data is summarized

● Insert into the database

● Access through the web application

Statistics Generation Flowchart

DatabaseRelational

Model

Data InsertionInsertion of data happens inside the Matlab process. ● ODBC/JDBC Connection must be ensured between

Matlab and database in order for them to communicate. ● After the connection, the data is immediately inserted

into the database every 5 minutes or (1800 frames).

Database Contents for Statistics Generation

Data Fetching and Displaying of Results

● Using TraVis’ web document, data is produced by the system and then is stored into the database which is shown into the web user interface.

● The data is fetched from the MySQL database through the use of query statements which is then provided to generate the graph using FusionCharts.

Experiments● Actual program implementation

● Bounding Box Aspect Ratio Checking

● Vehicle Detection on Background Modeling Scheme

● Accuracy of Vehicle Counting

Bounding Box Aspect Ratio Checking

System Output without Aspect Ratio Checking

Bounding Box Aspect Ratio Checking

System Output with Aspect Ratio Checking

Vehicle Detection on Background Modeling Scheme

5th minute (frame 1800)

Background Models

1st minute (frame 1)

Vehicle Detection on Background Modeling Scheme

6-6:30

am

6:30-7

am

7-7:30

am

7:30-8

am

8-8:30

am

8:30-9

am

9-9:30

am

9:30-1

0am

050

100150200250300350400450

One Time BG ModelFrequent BG ModelManual Count

Pearson’s One Time BG Model to

Manual Count

Pearson’s Frequent BG

Model to Manual Count

-0.51302 0.87499

Accuracy of Vehicle Counting• Comparing of system and manual counts

• Use of multiplying factor

• Use of Pearson’s Correlation Coefficient

• Percentage Relative Error

Manual Counts 25-Aug-14 Manual

Count26-Aug-14

Manual Count 23-May-14 Manual Count

Small Medium Large Total Small Medium Large Total Small Medium Large

0700-0800 147 248 11 406 0700-0800 130 191 14 335 0600-0700 71 70 24

0800-0900 179 273 18 470 0800-0900 167 210 17 394 0700-0800 131 142 18

0900-1000 239 304 15 558 0900-1000 365 287 22 674 0800-0900 131 121 35

1000-1100 209 366 14 589 1000-1100 309 241 21 571 0900-1000 133 134 45

1100-1200 320 379 12 711 1100-1200 330 288 17 635

1200-1300 297 420 14 731 1200-1300 320 352 21 693 1200-1300 142 145 47

1300-1400 293 488 12 793 1300-1400 302 276 24 602

1400-1500 268 462 16 746 1400-1500 362 292 23 677

1500-1600 274 413 19 706 1500-1600 333 280 18 631

1600-1700 303 429 14 746 1600-1700 272 249 21 542

1700-1800 326 461 18 805 1700-1800 258 245 14 517

System Counts25-Aug-14 System Count 26-Aug-14 System Count 23-May-14 System Count

Small Medium Large Small Medium Large Small Medium Large

0700-0800 190 130 5 0700-0800 267 354 21 0600-0700 127 101 35

0800-0900 143 290 6 0800-0900 280 372 19 0700-0800 210 204 24

0900-1000 312 258 14 0900-1000 252 289 23 0800-0900 171 167 45

1000-1100 408 669 29 1000-1100 268 227 18 0900-1000 221 230 28

1100-1200 552 638 22 1100-1200 277 271 15

1200-1300 269 299 138 1200-1300 261 107 5 1200-1300 258 284 35

1300-1400 9 14 0 1300-1400 571 1122 191

1400-1500 172 135 131 1400-1500 856 985 156

1500-1600 868 864 316 1500-1600 587 733 228

1600-1700 841 662 119 1600-1700 1269 1285 271

1700-1800 358 505 332 1700-1800 503 718 299

Multiplying Factors25-Aug-14 Multiplying

Factors 26-Aug-14 Multiplying Factors 23-May-14 Multiplying

factors


0700-0800 1.292517007 0.524194 0.454545455 0700-0800 2.053846 1.853403 1.5 0600-0700 1.788732 1.442857 1.458333

0800-0900 0.798882682 1.062271 0.333333333 0800-0900 1.676647 1.771429 1.117647059 0700-0800 1.603053 1.43662 1.333333

0900-1000 1.305439331 0.848684 0.933333333 0900-1000 0.690411 1.006969 1.045454545 0800-0900 1.305344 1.380165 1.285714

1000-1100 1.95215311 1.827869 2.071428571 1000-1100 0.867314 0.941909 0.857142857 0900-1000 1.661654 1.716418 0.622222

1100-1200 1.725 1.683377 1.833333333 1100-1200 0.839394 0.940972 0.882352941 1200-1300 1.816901 1.958621 0.744681

Average 1.414798426 1.189279 1.125194805 Average 1.225522 1.302936 1.080519481

1200-1300 0.905723906 0.711905 9.857142857 1200-1300 0.815625 0.303977 0.238095238 Average 1.635137 1.586936 1.088857

1300-1400 0.030716724 0.028689 0 1300-1400 1.890728 4.065217 7.958333333

1400-1500 0.641791045 0.292208 8.1875 1400-1500 2.364641 3.373288 6.782608696

1500-1600 3.167883212 2.09201 16.63157895 1500-1600 1.762763 2.617857 12.66666667

1600-1700 2.775577558 1.543124 8.5 1600-1700 4.665441 5.160643 12.9047619

1700-1800 1.098159509 1.095445 18.44444444 1700-1800 1.949612 2.930612 21.35714286

Average 1.436641992 0.960563 10.27011104 Average 2.241468 3.075266 10.31793478

System Counts with applied multiplying factors

25-Aug-14System Counts

with applied multiplying

factors26-Aug-14

System Counts with applied multiplying

factors23-May-14

System Counts with applied multiplying

factors


0700-0800 134.2947 109.3099268 4.443675 0700-0800 217.8662864 271.694 19.4351 0600-0700 77.669334 63.64465 32.14380425

0800-0900 101.0745 243.8452212 5.33241 0800-0900 228.4740082 285.509 17.58413 0700-0800 128.4296074 128.5496 22.04146577

0900-1000 220.5261 216.9381623 12.44229 0900-1000 205.6266074 221.8067 21.28606 0800-0900 104.5783946 105.2342 41.32774832

1000-1100 288.3803 562.5257 25.77331 1000-1100 218.682265 174.2219 16.65865 0900-1000 135.1568726 144.9334 25.7150434

1100-1200 390.1616 536.4594867 19.55217 1100-1200 226.0260724 207.9918 13.88221

1200-1300 187.2422 311.2757295 13.43705 1200-1300 116.4415229 34.79374 0.484593 1200-1300 157.7849462 178.9612 32.14380425

1300-1400 6.264609 14.57478332 0 1300-1400 254.7437149 364.8465 18.51146

1400-1500 119.7236 140.5425534 12.75546 1400-1500 381.8925044 320.2975 15.1193

1500-1600 604.1867 899.472342 30.7689 1500-1600 261.8818926 238.3534 22.09745

1600-1700 585.3929 689.1790398 11.58702 1600-1700 566.146715 417.8501 26.26495

1700-1800 249.1922 525.7332554 32.32682 1700-1800 224.4064599 233.4758 28.97867

0700-0800 0800-0900 0900-1000 1000-1100 1100-1200 1200-1300 1300-1400 1400-1500 1500-1600 1600-1700 1700-18000

100

200

300

400

500

600

700

800

900

1000 August 25 Small Count

Manual Count

System Count

System count with applied mul-tiplying fac-tors

0700-0800 0800-0900 0900-1000 1000-1100 1100-1200 1200-1300 1300-1400 1400-1500 1500-1600 1600-1700 1700-18000

100

200

300

400

500

600

700

800

900

1000 August 25 Medium Count

Manual Count

System Count

System Counts with applied mul-tiplying factors

0700-0800 0800-0900 0900-1000 1000-1100 1100-1200 1200-1300 1300-1400 1400-1500 1500-1600 1600-1700 1700-18000

5

10

15

20

25

30

35 August 25 Large Count

Manual Count

System Count with multiplying factors

0700-0800 0800-0900 0900-1000 1000-1100 1100-1200 1200-1300 1300-1400 1400-1500 1500-1600 1600-1700 1700-18000

100

200

300

400

500

600 August 26 Small Count

Manual Count

System Counts with applied mul-tiplying fac-tors

0700-0800

0800-0900

0900-1000

1000-1100

1100-1200

1200-1300

1300-1400

1400-1500

1500-1600

1600-1700

1700-1800

0

50

100

150

200

250

300

350

400

450 August 26 Medium Count

Manual Count

System Count


0700-0800 0800-0900 0900-1000 1000-1100 1100-1200 1200-1300 1300-1400 1400-1500 1500-1600 1600-1700 1700-18000

5

10

15

20

25

30

35 August 26 Large Count

Manual Count


0600-0700

0700-0800

0800-0900

0900-1000

1200-1300

0

20

40

60

80

100

120

140

160

180

May 23 Small Count

Manual CountSystem Count after ap-plying multiplying factors

Cou

nts

0600-0700 0700-0800 0800-0900 0900-1000 1200-13000

20

40

60

80

100

120

140

160

180

200

May 23 Medium Count

Manual CountSystem Count after applying multiplying fac-tors

Coun

ts

0600-0700 0700-0800 0800-0900 0900-1000 1200-13000

5

10

15

20

25

30

35

40

45

50

May 23 Large Count

Manual CountSystem CountSystem Count after applying multiplying fac-torsAxis Title

Pearson’s Correlation Coefficient

It is a measure of how well related the sets of data are.

● High correlation: .5 to 1.0 ● Medium correlation: .3 to .5 ● Low correlation: .1 to .3

Pearson's Correlation Coefficient

System Count to Manual Count

System Count with Multiplying Factors to Manual Count

Aug 25 2014 0.532330547 0.665698685

Aug 26 2014 0.532337468 0.712235061

May 23 2014 0.962154466 0.955987229

Average 0.675607493 0.777973658

Percentage Relative Error• Relative error is the measure of mean

absolute error to the mean value of the measured dataset

Percentage Relative ErrorDate Time MCount SCount RelError %RelError

25-Aug 0700-0800 406 248.0483511 0.389043 38.90435

0800-0900 470 350.2521005 0.254783 25.47828

0900-1000 558 449.906567 0.193716 19.37158

1000-1100 589 876.6793188 -0.48842 -48.842

1100-1200 711 946.1732442 -0.33076 -33.0764

1200-1300 731 511.9549749 0.299651 29.96512

1300-1400 793 20.83939207 0.973721 97.37208

1400-1500 746 273.0216483 0.634019 63.40192

1500-1600 706 1534.427951 -1.17341 -117.341

1600-1700 746 1286.158946 -0.72407 -72.4074

1700-1800 805 807.2522873 -0.0028 -0.27979

Average 660.0909 664.0649801 0.002315 0.231519


26-Aug 0700-0800 335 508.9954 -0.51939 -51.9389

0800-0900 394 531.5671 -0.34916 -34.9155

0900-1000 674 448.7194 0.334244 33.42443

1000-1100 571 409.5628 0.282727 28.27272

1100-1200 635 447.9 0.294646 29.46456

1200-1300 693 151.7199 0.781068 78.1068

1300-1400 602 638.1017 -0.05997 -5.99696

1400-1500 677 717.3093 -0.05954 -5.95411

1500-1600 631 522.3327 0.172214 17.22144

1600-1700 542 1010.262 -0.86395 -86.3952

1700-1800 517 486.8609 0.058296 5.829616

Average 570.0909 533.9392 0.006472 0.647172


23-May 0600-0700 165 173.4578 -0.05126 -5.12593

0700-0800 291 279.0207 0.041166 4.116609

0800-0900 287 251.1404 0.124946 12.49464

0900-1000 312 305.8053 0.019855 1.985486

1200-1300 334 368.89 -0.10446 -10.4461

Average 277.8 275.6628 0.006049 0.604942

Percentage Relative Error

Date RelError %RelError

25-Aug 0.002315195 0.231519

26-Aug 0.00647172 0.647172

23-May 0.006049424 0.604942

Average 0.004945446 0.494545

Conclusion• We developed a vision-based system for

counting vehicles and estimating traffic congestion levels in road sections installed with traffic surveillance cameras, accessible through a web interface.

Conclusion• Performance have been affected by the

following:

– Quality of video

– Background model

– Position of camera towards the road

– Detection of unwanted blobs/segmented blobs

Conclusion• System Counts has a .78 Correlation

Coefficient value

• It also has a 0.49 Percentage Relative Error

Conclusion• Gathered traffic videos from the Archer’s

Eye

• Developed Algorithms for Classification of vehicles and track and count the number of vehicles

• Generated Historical Graphs of Traffic Congestion levels

Conclusion• Designed a Database for storage of videos

and Traffic Statistics

• Developed an interactive Web interface

TraVis: Web-Based Vehicle Counter with Traffic Congestion Estimation Using

Computer Vision

Presented by:Aguirre, Byron FrancoAlcantara, Jan Andre

Trinidad, John Ferdinand

Adviser:Dr. Joel P. Ilao

Download - TraVis CTTHES3

Top Related