![Page 1: Computer Vision for Automatic Traffic Monitoring · Computing: low cost/energy computers, Ps, GPUs, cloud computing, ... M6 M7 M8 E1 E2 E3 E4 E5 E6 E7 E8 E9. Streamed Information](https://reader034.vdocuments.net/reader034/viewer/2022050605/5fac4e0a795b824a2252da6c/html5/thumbnails/1.jpg)
Computer Vision for Automatic Traffic Monitoring
March 2018
Group of Prof. Yehoshua Y. Zeevi
![Page 2: Computer Vision for Automatic Traffic Monitoring · Computing: low cost/energy computers, Ps, GPUs, cloud computing, ... M6 M7 M8 E1 E2 E3 E4 E5 E6 E7 E8 E9. Streamed Information](https://reader034.vdocuments.net/reader034/viewer/2022050605/5fac4e0a795b824a2252da6c/html5/thumbnails/2.jpg)
2
Agenda
• Context
• Goals / Vision
• Technologies / Infrastructure
• Proposed solution
• Challenges
• Results
• Conclusions and Recommendations
• Further work
![Page 3: Computer Vision for Automatic Traffic Monitoring · Computing: low cost/energy computers, Ps, GPUs, cloud computing, ... M6 M7 M8 E1 E2 E3 E4 E5 E6 E7 E8 E9. Streamed Information](https://reader034.vdocuments.net/reader034/viewer/2022050605/5fac4e0a795b824a2252da6c/html5/thumbnails/3.jpg)
Context - Smart City
![Page 4: Computer Vision for Automatic Traffic Monitoring · Computing: low cost/energy computers, Ps, GPUs, cloud computing, ... M6 M7 M8 E1 E2 E3 E4 E5 E6 E7 E8 E9. Streamed Information](https://reader034.vdocuments.net/reader034/viewer/2022050605/5fac4e0a795b824a2252da6c/html5/thumbnails/4.jpg)
Smart City
Smart Infrastructure
Traffic Control
Safety Efficiency Modularity Security
Context - Smart City
![Page 5: Computer Vision for Automatic Traffic Monitoring · Computing: low cost/energy computers, Ps, GPUs, cloud computing, ... M6 M7 M8 E1 E2 E3 E4 E5 E6 E7 E8 E9. Streamed Information](https://reader034.vdocuments.net/reader034/viewer/2022050605/5fac4e0a795b824a2252da6c/html5/thumbnails/5.jpg)
5
Focus – Pedestrian Monitoring
![Page 6: Computer Vision for Automatic Traffic Monitoring · Computing: low cost/energy computers, Ps, GPUs, cloud computing, ... M6 M7 M8 E1 E2 E3 E4 E5 E6 E7 E8 E9. Streamed Information](https://reader034.vdocuments.net/reader034/viewer/2022050605/5fac4e0a795b824a2252da6c/html5/thumbnails/6.jpg)
Demo
![Page 7: Computer Vision for Automatic Traffic Monitoring · Computing: low cost/energy computers, Ps, GPUs, cloud computing, ... M6 M7 M8 E1 E2 E3 E4 E5 E6 E7 E8 E9. Streamed Information](https://reader034.vdocuments.net/reader034/viewer/2022050605/5fac4e0a795b824a2252da6c/html5/thumbnails/7.jpg)
• Traffic monitoring / Junction overview
• Vehicle monitoring
• Pedestrian monitoring
• Pedestrian safety
• …
7
Goals
![Page 8: Computer Vision for Automatic Traffic Monitoring · Computing: low cost/energy computers, Ps, GPUs, cloud computing, ... M6 M7 M8 E1 E2 E3 E4 E5 E6 E7 E8 E9. Streamed Information](https://reader034.vdocuments.net/reader034/viewer/2022050605/5fac4e0a795b824a2252da6c/html5/thumbnails/8.jpg)
8
Possible Technologies
Sensors: motion sensors, magnetic sensors, cameras, BT, LIDARs…
Connectivity: wired / wireless, PTP, IoT, cloud services, …
Computing: low cost/energy computers, PCs, GPUs, cloud computing, …
Computer Vision: simple algorithms, advanced algorithms, deep learning
![Page 9: Computer Vision for Automatic Traffic Monitoring · Computing: low cost/energy computers, Ps, GPUs, cloud computing, ... M6 M7 M8 E1 E2 E3 E4 E5 E6 E7 E8 E9. Streamed Information](https://reader034.vdocuments.net/reader034/viewer/2022050605/5fac4e0a795b824a2252da6c/html5/thumbnails/9.jpg)
Haifa Municipal Traffic Control Center Cameras
Raspberry Pi board with HD camera (~100$)
ODROID (~200$)
NVIDIA Jetson GPU (~600$)
INTEL NUC (100 – 500 $)
9
Infra Structure
![Page 10: Computer Vision for Automatic Traffic Monitoring · Computing: low cost/energy computers, Ps, GPUs, cloud computing, ... M6 M7 M8 E1 E2 E3 E4 E5 E6 E7 E8 E9. Streamed Information](https://reader034.vdocuments.net/reader034/viewer/2022050605/5fac4e0a795b824a2252da6c/html5/thumbnails/10.jpg)
Simple• Spatial Operators• Time OperatorsAdvanced• Machine Learning
• Object Detection• Object Recognition• Object Tracking• Scene Understanding
10
Computer Vision Tools
![Page 11: Computer Vision for Automatic Traffic Monitoring · Computing: low cost/energy computers, Ps, GPUs, cloud computing, ... M6 M7 M8 E1 E2 E3 E4 E5 E6 E7 E8 E9. Streamed Information](https://reader034.vdocuments.net/reader034/viewer/2022050605/5fac4e0a795b824a2252da6c/html5/thumbnails/11.jpg)
11
Control center
Proposed Solution
• IoT (Internet of Things) Methodology
להחליף לרמזור הולכי רגל
![Page 12: Computer Vision for Automatic Traffic Monitoring · Computing: low cost/energy computers, Ps, GPUs, cloud computing, ... M6 M7 M8 E1 E2 E3 E4 E5 E6 E7 E8 E9. Streamed Information](https://reader034.vdocuments.net/reader034/viewer/2022050605/5fac4e0a795b824a2252da6c/html5/thumbnails/12.jpg)
12
Golda junction – prior data
Lane recognition
Vehicle counting per lane
Starting PositionVehicle traffic monitoring
![Page 13: Computer Vision for Automatic Traffic Monitoring · Computing: low cost/energy computers, Ps, GPUs, cloud computing, ... M6 M7 M8 E1 E2 E3 E4 E5 E6 E7 E8 E9. Streamed Information](https://reader034.vdocuments.net/reader034/viewer/2022050605/5fac4e0a795b824a2252da6c/html5/thumbnails/13.jpg)
13
Chorev junction
Vehicle traffic monitoring
![Page 14: Computer Vision for Automatic Traffic Monitoring · Computing: low cost/energy computers, Ps, GPUs, cloud computing, ... M6 M7 M8 E1 E2 E3 E4 E5 E6 E7 E8 E9. Streamed Information](https://reader034.vdocuments.net/reader034/viewer/2022050605/5fac4e0a795b824a2252da6c/html5/thumbnails/14.jpg)
Main FocusPedestrian Monitoring
• Pedestrian counting
• Average waiting time
• (abnormal behavior detection)
Spatial & Time Operators
Person No Person
Machine learning
* Videos from Haifa Traffic Control Center
![Page 15: Computer Vision for Automatic Traffic Monitoring · Computing: low cost/energy computers, Ps, GPUs, cloud computing, ... M6 M7 M8 E1 E2 E3 E4 E5 E6 E7 E8 E9. Streamed Information](https://reader034.vdocuments.net/reader034/viewer/2022050605/5fac4e0a795b824a2252da6c/html5/thumbnails/15.jpg)
Pedestrian counting
Pedestrian monitoring
* Videos from Haifa Traffic Control Center
![Page 16: Computer Vision for Automatic Traffic Monitoring · Computing: low cost/energy computers, Ps, GPUs, cloud computing, ... M6 M7 M8 E1 E2 E3 E4 E5 E6 E7 E8 E9. Streamed Information](https://reader034.vdocuments.net/reader034/viewer/2022050605/5fac4e0a795b824a2252da6c/html5/thumbnails/16.jpg)
Pedestrian monitoring algorithm
* Videos from Haifa Traffic Control Center
![Page 17: Computer Vision for Automatic Traffic Monitoring · Computing: low cost/energy computers, Ps, GPUs, cloud computing, ... M6 M7 M8 E1 E2 E3 E4 E5 E6 E7 E8 E9. Streamed Information](https://reader034.vdocuments.net/reader034/viewer/2022050605/5fac4e0a795b824a2252da6c/html5/thumbnails/17.jpg)
Pedestrian Counting
• New video data• Taken with Raspberry Pi camera
at Haifa Ziv junction
![Page 18: Computer Vision for Automatic Traffic Monitoring · Computing: low cost/energy computers, Ps, GPUs, cloud computing, ... M6 M7 M8 E1 E2 E3 E4 E5 E6 E7 E8 E9. Streamed Information](https://reader034.vdocuments.net/reader034/viewer/2022050605/5fac4e0a795b824a2252da6c/html5/thumbnails/18.jpg)
Some statistics
Person People Empty
TPR(%) PPV(%) ACC(%) TPR(%) PPV(%) ACC(%) TPR(%) PPV(%) ACC(%)
Nesher1b 83 85 96 87 94 97 98 97 97
Nesher4a 98 92 99 100 94 99 98 99 98
Ziv1a 98 93 97 98 95 98 95 97 96
TPR(%) PPV(%) ACC(%)
Average Std Average Std Average Std
Person 93 9 90 4 97 2
People 95 7 94 1 98 1
Empty 97 2 98 1 97 1
![Page 19: Computer Vision for Automatic Traffic Monitoring · Computing: low cost/energy computers, Ps, GPUs, cloud computing, ... M6 M7 M8 E1 E2 E3 E4 E5 E6 E7 E8 E9. Streamed Information](https://reader034.vdocuments.net/reader034/viewer/2022050605/5fac4e0a795b824a2252da6c/html5/thumbnails/19.jpg)
Challenges
• Sensitivity to viewing angles
![Page 20: Computer Vision for Automatic Traffic Monitoring · Computing: low cost/energy computers, Ps, GPUs, cloud computing, ... M6 M7 M8 E1 E2 E3 E4 E5 E6 E7 E8 E9. Streamed Information](https://reader034.vdocuments.net/reader034/viewer/2022050605/5fac4e0a795b824a2252da6c/html5/thumbnails/20.jpg)
20
Control center
IoT Cloud Services
• IoT (Internet of Things) Methodology
![Page 21: Computer Vision for Automatic Traffic Monitoring · Computing: low cost/energy computers, Ps, GPUs, cloud computing, ... M6 M7 M8 E1 E2 E3 E4 E5 E6 E7 E8 E9. Streamed Information](https://reader034.vdocuments.net/reader034/viewer/2022050605/5fac4e0a795b824a2252da6c/html5/thumbnails/21.jpg)
Chorev Junction Mapping 1
cam01
cam02
cam03
cam04
cam05
cam06
cam02 cam04
cam06
![Page 22: Computer Vision for Automatic Traffic Monitoring · Computing: low cost/energy computers, Ps, GPUs, cloud computing, ... M6 M7 M8 E1 E2 E3 E4 E5 E6 E7 E8 E9. Streamed Information](https://reader034.vdocuments.net/reader034/viewer/2022050605/5fac4e0a795b824a2252da6c/html5/thumbnails/22.jpg)
cam07
cam08
cam09
cam10
cam07
cam08cam10
cam09
Chorev Junction Mapping 2
![Page 23: Computer Vision for Automatic Traffic Monitoring · Computing: low cost/energy computers, Ps, GPUs, cloud computing, ... M6 M7 M8 E1 E2 E3 E4 E5 E6 E7 E8 E9. Streamed Information](https://reader034.vdocuments.net/reader034/viewer/2022050605/5fac4e0a795b824a2252da6c/html5/thumbnails/23.jpg)
Junction Diagram
M1
M2
M3M4
M5
M6
M7
M8
E1
E2
E3
E4
E5
E6
E7
E8
E9
![Page 24: Computer Vision for Automatic Traffic Monitoring · Computing: low cost/energy computers, Ps, GPUs, cloud computing, ... M6 M7 M8 E1 E2 E3 E4 E5 E6 E7 E8 E9. Streamed Information](https://reader034.vdocuments.net/reader034/viewer/2022050605/5fac4e0a795b824a2252da6c/html5/thumbnails/24.jpg)
Streamed Information
Control center
No images
No infringement of privacy
![Page 25: Computer Vision for Automatic Traffic Monitoring · Computing: low cost/energy computers, Ps, GPUs, cloud computing, ... M6 M7 M8 E1 E2 E3 E4 E5 E6 E7 E8 E9. Streamed Information](https://reader034.vdocuments.net/reader034/viewer/2022050605/5fac4e0a795b824a2252da6c/html5/thumbnails/25.jpg)
Results
![Page 26: Computer Vision for Automatic Traffic Monitoring · Computing: low cost/energy computers, Ps, GPUs, cloud computing, ... M6 M7 M8 E1 E2 E3 E4 E5 E6 E7 E8 E9. Streamed Information](https://reader034.vdocuments.net/reader034/viewer/2022050605/5fac4e0a795b824a2252da6c/html5/thumbnails/26.jpg)
26
Control center
Into the Future
• IoT (Internet of Things) Methodology
Smart Traffic Lights Standardization
Use of IoT
![Page 27: Computer Vision for Automatic Traffic Monitoring · Computing: low cost/energy computers, Ps, GPUs, cloud computing, ... M6 M7 M8 E1 E2 E3 E4 E5 E6 E7 E8 E9. Streamed Information](https://reader034.vdocuments.net/reader034/viewer/2022050605/5fac4e0a795b824a2252da6c/html5/thumbnails/27.jpg)
Conclusions
• Standardization: Traffic lights with built-in cameras
• Distributed computation• Low cost smart sensors• Connectivity through cloud services• Availability to nearby vehicles• Identification of behaviour with reference to age
![Page 28: Computer Vision for Automatic Traffic Monitoring · Computing: low cost/energy computers, Ps, GPUs, cloud computing, ... M6 M7 M8 E1 E2 E3 E4 E5 E6 E7 E8 E9. Streamed Information](https://reader034.vdocuments.net/reader034/viewer/2022050605/5fac4e0a795b824a2252da6c/html5/thumbnails/28.jpg)
Current Activity
Tel Aviv
Traffic Light Control - Habima
Disabled Pedestrian Monitoring
![Page 29: Computer Vision for Automatic Traffic Monitoring · Computing: low cost/energy computers, Ps, GPUs, cloud computing, ... M6 M7 M8 E1 E2 E3 E4 E5 E6 E7 E8 E9. Streamed Information](https://reader034.vdocuments.net/reader034/viewer/2022050605/5fac4e0a795b824a2252da6c/html5/thumbnails/29.jpg)
The Team
Principal Investigator:
Prof. Y. Zeevi
Researchers :
Dr. Eli Appleboim, Dr. Israel Berger, Johanan Erez, Dr. Rami Cohen, Roy Miterani, Aviad Levis
Vision and Image Sciences Lab :
Daniel Yagodin, Ina Talmon, Aviel Avraham, Alon David,
Zvi Lederer, Ben Ajami
Networked Software Systems Lab:
Roy Miterani, Hovav Gazit
~15 students:
Tom Shitrit, Yan Yampolsky, Lior Haimovich, Roi Sinoff, Tali Srebo, Ohad Spitzer, Alon Mamistavlov, Yonatan Shlain, Yonatan Shahor, Amit Enoch, Amit Gacket, Emanual Alkobi, Nofar Mann, Alon Zabatani, Morag Tohamy
Technion Transportation Institute
Dr. Ayellet Gal-Zur, Oded Komar, Iliah Finkelberg
Haifa Traffic Control Center
Anat Gilad, Sharona Cohen
![Page 30: Computer Vision for Automatic Traffic Monitoring · Computing: low cost/energy computers, Ps, GPUs, cloud computing, ... M6 M7 M8 E1 E2 E3 E4 E5 E6 E7 E8 E9. Streamed Information](https://reader034.vdocuments.net/reader034/viewer/2022050605/5fac4e0a795b824a2252da6c/html5/thumbnails/30.jpg)
30
Field Tests
![Page 31: Computer Vision for Automatic Traffic Monitoring · Computing: low cost/energy computers, Ps, GPUs, cloud computing, ... M6 M7 M8 E1 E2 E3 E4 E5 E6 E7 E8 E9. Streamed Information](https://reader034.vdocuments.net/reader034/viewer/2022050605/5fac4e0a795b824a2252da6c/html5/thumbnails/31.jpg)
31
Field Tests