autonomous driving: overview - iit delhi
TRANSCRIPT
![Page 1: Autonomous Driving: Overview - IIT Delhi](https://reader034.vdocuments.net/reader034/viewer/2022042122/62583196b800c0225d2831b5/html5/thumbnails/1.jpg)
Autonomous Driving: Overview
Jan 6th , 2018
Sahil Narang
University of North Carolina, Chapel Hill
1
![Page 2: Autonomous Driving: Overview - IIT Delhi](https://reader034.vdocuments.net/reader034/viewer/2022042122/62583196b800c0225d2831b5/html5/thumbnails/2.jpg)
University of North Carolina at Chapel Hill
Autonomous vehicle: a motor vehicle that uses artificial intelligence,
sensors and global positioning system coordinates to drive itself without the
active intervention of a human operator
Focus of enormous investment [80+ bn USD as of 2017]
2
Tesla
Waymo
Nutonomy
Autonomous Driving
![Page 3: Autonomous Driving: Overview - IIT Delhi](https://reader034.vdocuments.net/reader034/viewer/2022042122/62583196b800c0225d2831b5/html5/thumbnails/3.jpg)
Autonomous Driving: Motivation
Cars are ubiquitous
~ 1 bn vehicles for a global population of ~7 bn [est. 2010]
Car accidents can result in catastrophic costs [NHTSA study 2010]
94% serious crashes are due to human error
594 bn USD due to loss/decrease in life quality
242 bn USD in economic activity
Health costs
33k fatalities, 2 million+ injuries in 5.4 million crashes in U.S. [2010]
Premature deaths due to pollution inhalation
3
![Page 4: Autonomous Driving: Overview - IIT Delhi](https://reader034.vdocuments.net/reader034/viewer/2022042122/62583196b800c0225d2831b5/html5/thumbnails/4.jpg)
Autonomous Driving: Levels of Autonomy
0: Standard Car
1: Assist in some part of driving
Cruise control
2: Perform some part of driving
Adaptive CC + lane keeping
3: Self-driving under ideal conditions
Human must remain fully aware
4: Self-driving under near-ideal conditions
Human need not remain constantly aware
5: Outperforms human in all circumstances
4
![Page 5: Autonomous Driving: Overview - IIT Delhi](https://reader034.vdocuments.net/reader034/viewer/2022042122/62583196b800c0225d2831b5/html5/thumbnails/5.jpg)
Autonomous Driving
Urban driving is particularly challenging
5
![Page 6: Autonomous Driving: Overview - IIT Delhi](https://reader034.vdocuments.net/reader034/viewer/2022042122/62583196b800c0225d2831b5/html5/thumbnails/6.jpg)
Structure
History of Autonomous Driving
Main Components
Other Approaches
Other Issues
6
![Page 7: Autonomous Driving: Overview - IIT Delhi](https://reader034.vdocuments.net/reader034/viewer/2022042122/62583196b800c0225d2831b5/html5/thumbnails/7.jpg)
Autonomous Driving: 1980’s - 2010
1980’s
Ernst Dickmann’s VaMoRs
CMU NavLab
1990’s
PROMETHEUS project: VaMP car
2000’s
DARPA Grand Challenge 2004: 150 mile offroad coarse
DARPA Grand Challenge 2005: 132 mile offroad coarse
DARPA Grand Challenge 2007: 60 miles urban driving
7
![Page 8: Autonomous Driving: Overview - IIT Delhi](https://reader034.vdocuments.net/reader034/viewer/2022042122/62583196b800c0225d2831b5/html5/thumbnails/8.jpg)
Autonomous Driving: State of the Art Today
Mercedes Benz – historic Bertha route in Germany
Tesla Autopilot System
Google’s self-driving car (WayMo)
8
![Page 9: Autonomous Driving: Overview - IIT Delhi](https://reader034.vdocuments.net/reader034/viewer/2022042122/62583196b800c0225d2831b5/html5/thumbnails/9.jpg)
Structure
History of Autonomous Driving
Main Components
Perception
Planning
Control
Other Approaches
Other Issues
9
![Page 10: Autonomous Driving: Overview - IIT Delhi](https://reader034.vdocuments.net/reader034/viewer/2022042122/62583196b800c0225d2831b5/html5/thumbnails/10.jpg)
Autonomous Driving: Main Components
10
![Page 11: Autonomous Driving: Overview - IIT Delhi](https://reader034.vdocuments.net/reader034/viewer/2022042122/62583196b800c0225d2831b5/html5/thumbnails/11.jpg)
Autonomous Driving: Main Components
11
Perception
collect information and extract relevant knowledge from the
environment.
![Page 12: Autonomous Driving: Overview - IIT Delhi](https://reader034.vdocuments.net/reader034/viewer/2022042122/62583196b800c0225d2831b5/html5/thumbnails/12.jpg)
Autonomous Driving: Main Components
12
Planning
Making purposeful decisions in order to achieve the robot’s higher order
goals
![Page 13: Autonomous Driving: Overview - IIT Delhi](https://reader034.vdocuments.net/reader034/viewer/2022042122/62583196b800c0225d2831b5/html5/thumbnails/13.jpg)
Autonomous Driving: Main Components
13
Control
Executing planned actions
![Page 14: Autonomous Driving: Overview - IIT Delhi](https://reader034.vdocuments.net/reader034/viewer/2022042122/62583196b800c0225d2831b5/html5/thumbnails/14.jpg)
Structure
History of Autonomous Driving
Main Components
Perception
Planning
Control
Other Approaches
Other Issues
14
![Page 15: Autonomous Driving: Overview - IIT Delhi](https://reader034.vdocuments.net/reader034/viewer/2022042122/62583196b800c0225d2831b5/html5/thumbnails/15.jpg)
Autonomous Driving: Perception
Sensing Challenges
Sensor Uncertainty
Sensor Configuration
Weather / Environment
15
![Page 16: Autonomous Driving: Overview - IIT Delhi](https://reader034.vdocuments.net/reader034/viewer/2022042122/62583196b800c0225d2831b5/html5/thumbnails/16.jpg)
Autonomous Driving: Challenges in Perception
Sensor Misclassification
“When is a cyclist not a cyclist?”
When is a sign a stop sign?
Whether a semi or a cloud?
16
![Page 17: Autonomous Driving: Overview - IIT Delhi](https://reader034.vdocuments.net/reader034/viewer/2022042122/62583196b800c0225d2831b5/html5/thumbnails/17.jpg)
Autonomous Driving: Perception
Environmental Perception
LIDAR
Cameras
Other approaches
RADAR, Ultrasonic sensors
Fusion
17
![Page 18: Autonomous Driving: Overview - IIT Delhi](https://reader034.vdocuments.net/reader034/viewer/2022042122/62583196b800c0225d2831b5/html5/thumbnails/18.jpg)
Autonomous Driving: Perception using LIDAR
Light Detection and Ranging
Illuminate target using pulsed laser lights, and measure reflected pulses
using a sensor
18
![Page 19: Autonomous Driving: Overview - IIT Delhi](https://reader034.vdocuments.net/reader034/viewer/2022042122/62583196b800c0225d2831b5/html5/thumbnails/19.jpg)
Autonomous Driving: Perception using LIDAR
LIDAR Challenges
Scanning sparsity
Missing points
Unorganized patterns
Knowledge gathering can be
difficult
19
![Page 20: Autonomous Driving: Overview - IIT Delhi](https://reader034.vdocuments.net/reader034/viewer/2022042122/62583196b800c0225d2831b5/html5/thumbnails/20.jpg)
Autonomous Driving: Perception using LIDAR
Data Representation
Point clouds
Features: lines, surfaces etc
Grid based approaches
20
![Page 21: Autonomous Driving: Overview - IIT Delhi](https://reader034.vdocuments.net/reader034/viewer/2022042122/62583196b800c0225d2831b5/html5/thumbnails/21.jpg)
Autonomous Driving: Perception using LIDAR
Knowledge Extraction
3D point cloud segmentation
Edge based
Region based
Model based
Attribute based
Graph based
Classification
21
![Page 22: Autonomous Driving: Overview - IIT Delhi](https://reader034.vdocuments.net/reader034/viewer/2022042122/62583196b800c0225d2831b5/html5/thumbnails/22.jpg)
Autonomous Driving: Perception using LIDAR
LIDAR in practice
Velodyne 64HD lidar
22
![Page 23: Autonomous Driving: Overview - IIT Delhi](https://reader034.vdocuments.net/reader034/viewer/2022042122/62583196b800c0225d2831b5/html5/thumbnails/23.jpg)
Autonomous Driving: Perception
Environmental Perception
LIDAR
Cameras
Fusion
Other approaches
RADAR, Ultrasonic sensors
23
![Page 24: Autonomous Driving: Overview - IIT Delhi](https://reader034.vdocuments.net/reader034/viewer/2022042122/62583196b800c0225d2831b5/html5/thumbnails/24.jpg)
Autonomous Driving: Perception using Cameras
Camera based vision
Road detection
Lane marking detection
Road surface detection
On-road object detection
24
![Page 25: Autonomous Driving: Overview - IIT Delhi](https://reader034.vdocuments.net/reader034/viewer/2022042122/62583196b800c0225d2831b5/html5/thumbnails/25.jpg)
Autonomous Driving: Perception using Cameras
25
Challenges in Lane Detection
![Page 26: Autonomous Driving: Overview - IIT Delhi](https://reader034.vdocuments.net/reader034/viewer/2022042122/62583196b800c0225d2831b5/html5/thumbnails/26.jpg)
Autonomous Driving: Perception using Cameras
Camera based vision
Road detection
Lane marking detection
Road surface detection
On-road object detection
26
![Page 27: Autonomous Driving: Overview - IIT Delhi](https://reader034.vdocuments.net/reader034/viewer/2022042122/62583196b800c0225d2831b5/html5/thumbnails/27.jpg)
Autonomous Driving: Perception using Cameras
Approaches to road surface detection
Feature-based
Deep learning
Direct pixel/block labelling
High memory and computation
requirements
Requires annotated data
Black box
27
![Page 28: Autonomous Driving: Overview - IIT Delhi](https://reader034.vdocuments.net/reader034/viewer/2022042122/62583196b800c0225d2831b5/html5/thumbnails/28.jpg)
Autonomous Driving: Perception using Cameras
Camera based vision
Road detection
Lane marking detection
Road surface detection
On-road object detection
28
![Page 29: Autonomous Driving: Overview - IIT Delhi](https://reader034.vdocuments.net/reader034/viewer/2022042122/62583196b800c0225d2831b5/html5/thumbnails/29.jpg)
Autonomous Driving: Perception using Cameras
On-road object detection
Pedestrian, cyclists, other cars
Challenging due to the various types,
appearances, shapes, and sizes of the
objects
29
![Page 30: Autonomous Driving: Overview - IIT Delhi](https://reader034.vdocuments.net/reader034/viewer/2022042122/62583196b800c0225d2831b5/html5/thumbnails/30.jpg)
Autonomous Driving: Perception using Cameras
On-road object detection
Pedestrian, cyclists, other cars
Challenging due to the various types,
appearances, shapes, and sizes of the
objects
Deep learning methods are far superior
30
![Page 31: Autonomous Driving: Overview - IIT Delhi](https://reader034.vdocuments.net/reader034/viewer/2022042122/62583196b800c0225d2831b5/html5/thumbnails/31.jpg)
Autonomous Driving: Perception using Cameras
Mobileye
31
![Page 32: Autonomous Driving: Overview - IIT Delhi](https://reader034.vdocuments.net/reader034/viewer/2022042122/62583196b800c0225d2831b5/html5/thumbnails/32.jpg)
Autonomous Driving: Perception using Sensor Fusion
LIDAR
3D measurements
Impervious to illumination changes
Prone to noise
Hard to extract knowledge
Cameras
Provide rich appearance details in 2D
Affected by illumination/ weather
32
![Page 33: Autonomous Driving: Overview - IIT Delhi](https://reader034.vdocuments.net/reader034/viewer/2022042122/62583196b800c0225d2831b5/html5/thumbnails/33.jpg)
Autonomous Driving: Vehicle Localization
Determining the pose of the ego vehicle and measuring its own motion
Fusing data
Satellite-based navigation system
Inertial navigation system
Map aided localization
SLAM
33
![Page 34: Autonomous Driving: Overview - IIT Delhi](https://reader034.vdocuments.net/reader034/viewer/2022042122/62583196b800c0225d2831b5/html5/thumbnails/34.jpg)
Structure
History of Autonomous Driving
Main Components
Perception
Planning
Control
Other Approaches
Other Issues
34
![Page 35: Autonomous Driving: Overview - IIT Delhi](https://reader034.vdocuments.net/reader034/viewer/2022042122/62583196b800c0225d2831b5/html5/thumbnails/35.jpg)
Autonomous Driving: Main Components
35
Planning
Making purposeful decisions in order to achieve the robot’s higher order
goals
![Page 36: Autonomous Driving: Overview - IIT Delhi](https://reader034.vdocuments.net/reader034/viewer/2022042122/62583196b800c0225d2831b5/html5/thumbnails/36.jpg)
Autonomous Driving: Planning
Compare to Pedestrian Techniques:
Route Planning: road selection (global)
Path Planning: preferred lanes (global)
Maneuver-search: high level maneuvers
(local)
Trajectory planning: Lowest level of
planning (local)
36
![Page 37: Autonomous Driving: Overview - IIT Delhi](https://reader034.vdocuments.net/reader034/viewer/2022042122/62583196b800c0225d2831b5/html5/thumbnails/37.jpg)
Autonomous Driving: Planning
37
![Page 38: Autonomous Driving: Overview - IIT Delhi](https://reader034.vdocuments.net/reader034/viewer/2022042122/62583196b800c0225d2831b5/html5/thumbnails/38.jpg)
Structure
History of Autonomous Driving
Main Components
Perception
Planning
Control
Other Approaches
Other Issues
38
![Page 39: Autonomous Driving: Overview - IIT Delhi](https://reader034.vdocuments.net/reader034/viewer/2022042122/62583196b800c0225d2831b5/html5/thumbnails/39.jpg)
Autonomous Driving: Control Planning
Convert plans into actions
Provide inputs to the hardware level to generate the desired motion
Common Approaches
Proportional-Integral-Derivative (PID) controller
Model Predictive Control (MPC)
39
![Page 40: Autonomous Driving: Overview - IIT Delhi](https://reader034.vdocuments.net/reader034/viewer/2022042122/62583196b800c0225d2831b5/html5/thumbnails/40.jpg)
Structure
History of Autonomous Driving
Main Components
Other Approaches
Other Issues
40
![Page 41: Autonomous Driving: Overview - IIT Delhi](https://reader034.vdocuments.net/reader034/viewer/2022042122/62583196b800c0225d2831b5/html5/thumbnails/41.jpg)
Autonomous Driving: End-End Approaches
Nvidia PilotNet
Deep learning to directly map video frames to control
41
![Page 42: Autonomous Driving: Overview - IIT Delhi](https://reader034.vdocuments.net/reader034/viewer/2022042122/62583196b800c0225d2831b5/html5/thumbnails/42.jpg)
Structure
History of Autonomous Driving
Main Components
Other Approaches
Other Issues
42
![Page 43: Autonomous Driving: Overview - IIT Delhi](https://reader034.vdocuments.net/reader034/viewer/2022042122/62583196b800c0225d2831b5/html5/thumbnails/43.jpg)
Autonomous Driving: Other Issues
Other challenges:
Communication
Coordination
Ethical Issues
Trolley Problem
43
![Page 44: Autonomous Driving: Overview - IIT Delhi](https://reader034.vdocuments.net/reader034/viewer/2022042122/62583196b800c0225d2831b5/html5/thumbnails/44.jpg)
Autonomous Driving: Other Issues
Other challenges:
MIT “Moral Machine” [https://goo.gl/RL4pr5]
44
![Page 45: Autonomous Driving: Overview - IIT Delhi](https://reader034.vdocuments.net/reader034/viewer/2022042122/62583196b800c0225d2831b5/html5/thumbnails/45.jpg)
Autonomous Driving: Other Issues
Civil Engineering / Ethics
Traffic impacts?
Pro: Vehicles should respond appropriately to traffic reducing jams
Con: Many more vehicles per person possible
People may not own cars?
Pro: Less emission? Less Traffic?
Con: Less access?
45
![Page 46: Autonomous Driving: Overview - IIT Delhi](https://reader034.vdocuments.net/reader034/viewer/2022042122/62583196b800c0225d2831b5/html5/thumbnails/46.jpg)
Next Lecture
Modeling a car – kinematics & dynamics
Motion Planning
Control
Modeling interactions with other vehicles
AutonoVi: simulation platform for autonomous driving
46
![Page 47: Autonomous Driving: Overview - IIT Delhi](https://reader034.vdocuments.net/reader034/viewer/2022042122/62583196b800c0225d2831b5/html5/thumbnails/47.jpg)
References
Katrakazas, C., Quddus, M., Chen, W. H., & Deka, L. (2015). Real-time motion planning methods for autonomous
on-road driving: State-of-the-art and future research directions. Transportation Research Part C: Emerging
Technologies, 60, 416-442.
Pendleton, S. D., Andersen, H., Du, X., Shen, X., Meghjani, M., Eng, Y. H., ... & Ang, M. H. (2017). Perception,
Planning, Control, and Coordination for Autonomous Vehicles. Machines, 5(1), 6.
Paden, B., Čáp, M., Yong, S. Z., Yershov, D., & Frazzoli, E. (2016). A survey of motion planning and control
techniques for self-driving urban vehicles. IEEE Transactions on Intelligent Vehicles, 1(1), 33-55.
Urmson, C., Baker, C., Dolan, J., Rybski, P., Salesky, B., Whittaker, W., ... & Darms, M. (2009). Autonomous
driving in traffic: Boss and the urban challenge. AI magazine, 30(2), 17.
Bojarski, M., Yeres, P., Choromanska, A., Choromanski, K., Firner, B., Jackel, L., & Muller, U. (2017). Explaining
How a Deep Neural Network Trained with End-to-End Learning Steers a Car. arXiv preprint arXiv:1704.07911.
MIT moral machine: http://moralmachine.mit.edu/
U.S. National Highway Transfprotation Safety Administration: https://www.nhtsa.gov/technology-
innovation/automated-vehicles-safety
47