the road to autonomous vehicles - … · advantages of redundant sensor fusion references: 1 hall,...
TRANSCRIPT
![Page 1: The Road to Autonomous Vehicles - … · Advantages of Redundant Sensor Fusion References: 1 Hall, David L., “Mathematical Techniques in Multisensor Data Fusion", Artech House Information](https://reader031.vdocuments.net/reader031/viewer/2022022613/5b9ea65009d3f26e288bc551/html5/thumbnails/1.jpg)
The Road to Autonomous Vehicles
Bob Feldmaier Director of the Center for Advanced
Automotive Technology Macomb Community College
![Page 2: The Road to Autonomous Vehicles - … · Advantages of Redundant Sensor Fusion References: 1 Hall, David L., “Mathematical Techniques in Multisensor Data Fusion", Artech House Information](https://reader031.vdocuments.net/reader031/viewer/2022022613/5b9ea65009d3f26e288bc551/html5/thumbnails/2.jpg)
Main Questions
• Why the interest in autonomous vehicles?
• How does the technology work?
• What are the remaining challenges?
![Page 3: The Road to Autonomous Vehicles - … · Advantages of Redundant Sensor Fusion References: 1 Hall, David L., “Mathematical Techniques in Multisensor Data Fusion", Artech House Information](https://reader031.vdocuments.net/reader031/viewer/2022022613/5b9ea65009d3f26e288bc551/html5/thumbnails/3.jpg)
GM’s Road to Automated Driving
![Page 4: The Road to Autonomous Vehicles - … · Advantages of Redundant Sensor Fusion References: 1 Hall, David L., “Mathematical Techniques in Multisensor Data Fusion", Artech House Information](https://reader031.vdocuments.net/reader031/viewer/2022022613/5b9ea65009d3f26e288bc551/html5/thumbnails/4.jpg)
Some of Today’s Advanced Driver Assistance Technologies
![Page 5: The Road to Autonomous Vehicles - … · Advantages of Redundant Sensor Fusion References: 1 Hall, David L., “Mathematical Techniques in Multisensor Data Fusion", Artech House Information](https://reader031.vdocuments.net/reader031/viewer/2022022613/5b9ea65009d3f26e288bc551/html5/thumbnails/5.jpg)
NHTSA Automated Driving Levels (0-4)
Today 2020 2025+
Driver only Assisted Partial Conditional Full
Active high beam
Traffic jam assist
Collision avoidance
Valet self-parking
Collision imminent
braking
Adaptive cruise & lane keeping
Automated highway
Highway point-to-point
Cruise control Self-parking (with driver)
Automated urban
Urban point-to-point
Forward Radar Forward Radar Lidar & 360⁰ Radar
Lidar & 360⁰ Radar
Forward Vision Forward Vision High accuracy GPS
High accuracy GPS
Multi-domain controller
Multi-domain controller
Multi-domain controller
Driver State Sensor (DSS)
Forward Vision cameras, DSS
Forward Vision cameras, DSS
V2X V2X V2X
IMU IMU
Feat
ure
Te
chn
olo
gy
Level 0 Level 1 Level 2 Level 3 Level 4
![Page 6: The Road to Autonomous Vehicles - … · Advantages of Redundant Sensor Fusion References: 1 Hall, David L., “Mathematical Techniques in Multisensor Data Fusion", Artech House Information](https://reader031.vdocuments.net/reader031/viewer/2022022613/5b9ea65009d3f26e288bc551/html5/thumbnails/6.jpg)
Potential Benefits of Vehicle Automation
![Page 7: The Road to Autonomous Vehicles - … · Advantages of Redundant Sensor Fusion References: 1 Hall, David L., “Mathematical Techniques in Multisensor Data Fusion", Artech House Information](https://reader031.vdocuments.net/reader031/viewer/2022022613/5b9ea65009d3f26e288bc551/html5/thumbnails/7.jpg)
The Impact of Car Crashes on the Economy beyond 34,000 Deaths per Year in the US Alone
![Page 8: The Road to Autonomous Vehicles - … · Advantages of Redundant Sensor Fusion References: 1 Hall, David L., “Mathematical Techniques in Multisensor Data Fusion", Artech House Information](https://reader031.vdocuments.net/reader031/viewer/2022022613/5b9ea65009d3f26e288bc551/html5/thumbnails/8.jpg)
The Past & Present: Automotive Safety…
30,000
35,000
40,000
45,000
50,000
55,000
1975 1980 1985 1990 1995 2000 2005 2010 2015
Automotive fatalities: USA
Deaths
Seat Belt Mandates
Air Bag Sensors/ Mandates
ESC and Roll Over Sensors
Distracted Driving
Source: http://www.iihs.org/iihs/topics/t/general-statistics/fatalityfacts/overview-of-fatality-facts
National Highway Traffic Safety Administration (NHTSA)
![Page 9: The Road to Autonomous Vehicles - … · Advantages of Redundant Sensor Fusion References: 1 Hall, David L., “Mathematical Techniques in Multisensor Data Fusion", Artech House Information](https://reader031.vdocuments.net/reader031/viewer/2022022613/5b9ea65009d3f26e288bc551/html5/thumbnails/9.jpg)
US Consumers Rate Safety and Advanced Driver Assistance Technologies Most Important
![Page 10: The Road to Autonomous Vehicles - … · Advantages of Redundant Sensor Fusion References: 1 Hall, David L., “Mathematical Techniques in Multisensor Data Fusion", Artech House Information](https://reader031.vdocuments.net/reader031/viewer/2022022613/5b9ea65009d3f26e288bc551/html5/thumbnails/10.jpg)
Dr. Lawrence D. Burns, 2015
![Page 11: The Road to Autonomous Vehicles - … · Advantages of Redundant Sensor Fusion References: 1 Hall, David L., “Mathematical Techniques in Multisensor Data Fusion", Artech House Information](https://reader031.vdocuments.net/reader031/viewer/2022022613/5b9ea65009d3f26e288bc551/html5/thumbnails/11.jpg)
Integrated Systems Approach to Vehicle Automation
Coming application: 2017 Cadillac “Super Cruise”
![Page 12: The Road to Autonomous Vehicles - … · Advantages of Redundant Sensor Fusion References: 1 Hall, David L., “Mathematical Techniques in Multisensor Data Fusion", Artech House Information](https://reader031.vdocuments.net/reader031/viewer/2022022613/5b9ea65009d3f26e288bc551/html5/thumbnails/12.jpg)
ADAS and Automated Vehicle Sensors
Multi Domain
Controller
Side Radars
Side Radars
V2V/V2X
Rear View Cam
Side View Cam
Side View Cam
Driver State
Sensing Side Radars
Front View Cam
Front Radars
Front Cameras
Lidar
Side Radars
![Page 13: The Road to Autonomous Vehicles - … · Advantages of Redundant Sensor Fusion References: 1 Hall, David L., “Mathematical Techniques in Multisensor Data Fusion", Artech House Information](https://reader031.vdocuments.net/reader031/viewer/2022022613/5b9ea65009d3f26e288bc551/html5/thumbnails/13.jpg)
Vision/Radar/Lidar Operation and Fusion
![Page 14: The Road to Autonomous Vehicles - … · Advantages of Redundant Sensor Fusion References: 1 Hall, David L., “Mathematical Techniques in Multisensor Data Fusion", Artech House Information](https://reader031.vdocuments.net/reader031/viewer/2022022613/5b9ea65009d3f26e288bc551/html5/thumbnails/14.jpg)
![Page 15: The Road to Autonomous Vehicles - … · Advantages of Redundant Sensor Fusion References: 1 Hall, David L., “Mathematical Techniques in Multisensor Data Fusion", Artech House Information](https://reader031.vdocuments.net/reader031/viewer/2022022613/5b9ea65009d3f26e288bc551/html5/thumbnails/15.jpg)
Sensor Fusion Improves Performance
Vision Lane Information
Vision Vehicle Information Radar Object Information
Radar Barrier Detection
Left Right
Adjacent Left
Adjacent Right
![Page 16: The Road to Autonomous Vehicles - … · Advantages of Redundant Sensor Fusion References: 1 Hall, David L., “Mathematical Techniques in Multisensor Data Fusion", Artech House Information](https://reader031.vdocuments.net/reader031/viewer/2022022613/5b9ea65009d3f26e288bc551/html5/thumbnails/16.jpg)
• Probability of correct detection and classification1
– Increases with additional sensors and redundancy
– Utilize sensors with highest signal to noise ratio (S/N) under the ambient conditions
– Disregard sensors that have low S/N under the ambient conditions
– Marginal gains decrease for more than 5 sensors
• Reliability of systems2
– Adding more sensors increases the reliability of the overall system
– Mean time to failure of a system with more sensors is increased
Advantages of Redundant Sensor Fusion
References: 1Hall, David L., “Mathematical Techniques in Multisensor Data Fusion", Artech House Information Warfare Library, February 26, 2004 2Deyst, John, “Real Time Systems for Aerospace Vehicles”, MIT 16.840 Aeronautics & Astronautics Course Notes, Spring 1999
![Page 17: The Road to Autonomous Vehicles - … · Advantages of Redundant Sensor Fusion References: 1 Hall, David L., “Mathematical Techniques in Multisensor Data Fusion", Artech House Information](https://reader031.vdocuments.net/reader031/viewer/2022022613/5b9ea65009d3f26e288bc551/html5/thumbnails/17.jpg)
Multi-domain Controller
17
Enables future system optimization/upgradability
Multi-domain controller
Front Radar
controller Camera
controller Airbag Controller
Rear and Side Radar controller
ESC controlle
r
Scalable software platform
Reduced architecture complexity
Faster communication/interconnection
Multi-processor configuration
Production launch in 2017
Centralized Sensor Fusion/Control Active Safety Multi-domain controller
![Page 18: The Road to Autonomous Vehicles - … · Advantages of Redundant Sensor Fusion References: 1 Hall, David L., “Mathematical Techniques in Multisensor Data Fusion", Artech House Information](https://reader031.vdocuments.net/reader031/viewer/2022022613/5b9ea65009d3f26e288bc551/html5/thumbnails/18.jpg)
Typical Software Applications: Lines of Code 24 million lines of code
F-35 fighter jet
61 million lines of code Facebook
12 million lines of code Android Operating System
Premium vehicle
44 million lines of code Microsoft Office 2013
50+ computers To deliver a world-class user
experience, active safety and high performance drivability
*Software lines of code information courtesy of informationisbeautiful.net
Premium vehicles today operate with over 100 million+ lines of code
![Page 19: The Road to Autonomous Vehicles - … · Advantages of Redundant Sensor Fusion References: 1 Hall, David L., “Mathematical Techniques in Multisensor Data Fusion", Artech House Information](https://reader031.vdocuments.net/reader031/viewer/2022022613/5b9ea65009d3f26e288bc551/html5/thumbnails/19.jpg)
Automated Driving: Enabling and Supporting Technology
19
Source: Texas Instruments ADAS Solutions Guide
HIGH DEFINITION MAPS V2X COMMUNICATIONS
![Page 20: The Road to Autonomous Vehicles - … · Advantages of Redundant Sensor Fusion References: 1 Hall, David L., “Mathematical Techniques in Multisensor Data Fusion", Artech House Information](https://reader031.vdocuments.net/reader031/viewer/2022022613/5b9ea65009d3f26e288bc551/html5/thumbnails/20.jpg)
Introducing the Concept of “Connected” Vehicles
![Page 21: The Road to Autonomous Vehicles - … · Advantages of Redundant Sensor Fusion References: 1 Hall, David L., “Mathematical Techniques in Multisensor Data Fusion", Artech House Information](https://reader031.vdocuments.net/reader031/viewer/2022022613/5b9ea65009d3f26e288bc551/html5/thumbnails/21.jpg)
26B connected devices 250M connected vehicles
by 2020…
![Page 22: The Road to Autonomous Vehicles - … · Advantages of Redundant Sensor Fusion References: 1 Hall, David L., “Mathematical Techniques in Multisensor Data Fusion", Artech House Information](https://reader031.vdocuments.net/reader031/viewer/2022022613/5b9ea65009d3f26e288bc551/html5/thumbnails/22.jpg)
The Connected Car Evolves…
![Page 23: The Road to Autonomous Vehicles - … · Advantages of Redundant Sensor Fusion References: 1 Hall, David L., “Mathematical Techniques in Multisensor Data Fusion", Artech House Information](https://reader031.vdocuments.net/reader031/viewer/2022022613/5b9ea65009d3f26e288bc551/html5/thumbnails/23.jpg)
Cars talking with surrounding infrastructure…
![Page 24: The Road to Autonomous Vehicles - … · Advantages of Redundant Sensor Fusion References: 1 Hall, David L., “Mathematical Techniques in Multisensor Data Fusion", Artech House Information](https://reader031.vdocuments.net/reader031/viewer/2022022613/5b9ea65009d3f26e288bc551/html5/thumbnails/24.jpg)
![Page 25: The Road to Autonomous Vehicles - … · Advantages of Redundant Sensor Fusion References: 1 Hall, David L., “Mathematical Techniques in Multisensor Data Fusion", Artech House Information](https://reader031.vdocuments.net/reader031/viewer/2022022613/5b9ea65009d3f26e288bc551/html5/thumbnails/25.jpg)
The Process of Delivering Real-Time Maps
![Page 26: The Road to Autonomous Vehicles - … · Advantages of Redundant Sensor Fusion References: 1 Hall, David L., “Mathematical Techniques in Multisensor Data Fusion", Artech House Information](https://reader031.vdocuments.net/reader031/viewer/2022022613/5b9ea65009d3f26e288bc551/html5/thumbnails/26.jpg)
Once a Vehicle is Connected, Many More Features Become Available
![Page 27: The Road to Autonomous Vehicles - … · Advantages of Redundant Sensor Fusion References: 1 Hall, David L., “Mathematical Techniques in Multisensor Data Fusion", Artech House Information](https://reader031.vdocuments.net/reader031/viewer/2022022613/5b9ea65009d3f26e288bc551/html5/thumbnails/27.jpg)
The Vehicle Becomes Integrated with the Web of Everything
![Page 28: The Road to Autonomous Vehicles - … · Advantages of Redundant Sensor Fusion References: 1 Hall, David L., “Mathematical Techniques in Multisensor Data Fusion", Artech House Information](https://reader031.vdocuments.net/reader031/viewer/2022022613/5b9ea65009d3f26e288bc551/html5/thumbnails/28.jpg)
![Page 29: The Road to Autonomous Vehicles - … · Advantages of Redundant Sensor Fusion References: 1 Hall, David L., “Mathematical Techniques in Multisensor Data Fusion", Artech House Information](https://reader031.vdocuments.net/reader031/viewer/2022022613/5b9ea65009d3f26e288bc551/html5/thumbnails/29.jpg)
![Page 30: The Road to Autonomous Vehicles - … · Advantages of Redundant Sensor Fusion References: 1 Hall, David L., “Mathematical Techniques in Multisensor Data Fusion", Artech House Information](https://reader031.vdocuments.net/reader031/viewer/2022022613/5b9ea65009d3f26e288bc551/html5/thumbnails/30.jpg)
With Connectivity, Data Becomes “Bigger”
![Page 31: The Road to Autonomous Vehicles - … · Advantages of Redundant Sensor Fusion References: 1 Hall, David L., “Mathematical Techniques in Multisensor Data Fusion", Artech House Information](https://reader031.vdocuments.net/reader031/viewer/2022022613/5b9ea65009d3f26e288bc551/html5/thumbnails/31.jpg)
With More Data and Connectivity Comes More Vulnerability of Cybersecurity
![Page 32: The Road to Autonomous Vehicles - … · Advantages of Redundant Sensor Fusion References: 1 Hall, David L., “Mathematical Techniques in Multisensor Data Fusion", Artech House Information](https://reader031.vdocuments.net/reader031/viewer/2022022613/5b9ea65009d3f26e288bc551/html5/thumbnails/32.jpg)
Toyota’s Assessment of Automated Vehicle Technology
![Page 33: The Road to Autonomous Vehicles - … · Advantages of Redundant Sensor Fusion References: 1 Hall, David L., “Mathematical Techniques in Multisensor Data Fusion", Artech House Information](https://reader031.vdocuments.net/reader031/viewer/2022022613/5b9ea65009d3f26e288bc551/html5/thumbnails/33.jpg)
Toyota’s Assessment (Continued)
![Page 34: The Road to Autonomous Vehicles - … · Advantages of Redundant Sensor Fusion References: 1 Hall, David L., “Mathematical Techniques in Multisensor Data Fusion", Artech House Information](https://reader031.vdocuments.net/reader031/viewer/2022022613/5b9ea65009d3f26e288bc551/html5/thumbnails/34.jpg)
Mcity at U of M: the First Extensive Testing Facility Built for Automated Vehicles
• https://www.youtube.com/watch?v=gfSNlIQ5KN8
• https://www.youtube.com/watch?v=aKduQC8wNu4
![Page 35: The Road to Autonomous Vehicles - … · Advantages of Redundant Sensor Fusion References: 1 Hall, David L., “Mathematical Techniques in Multisensor Data Fusion", Artech House Information](https://reader031.vdocuments.net/reader031/viewer/2022022613/5b9ea65009d3f26e288bc551/html5/thumbnails/35.jpg)
Summary of Major Advantages
• Fewer traffic collisions
• Increased roadway capacity and reduced congestion
• Relief for occupants from driving and navigation
• Removal of constraints on occupants’ state or handicaps
• Lighter more fuel efficient vehicles
• Reduced insurance costs
• Higher speed limits
• Increased productivity
![Page 36: The Road to Autonomous Vehicles - … · Advantages of Redundant Sensor Fusion References: 1 Hall, David L., “Mathematical Techniques in Multisensor Data Fusion", Artech House Information](https://reader031.vdocuments.net/reader031/viewer/2022022613/5b9ea65009d3f26e288bc551/html5/thumbnails/36.jpg)
Summary of Major Concerns
• Assignment of liability for errors
• Resistance to loss of vehicle control
• Hardware function in bad weather
• Software decision protocols
• Software reliability
• Cybersecurity
• Loss of privacy
• Managing the transition from automated control to driver control
![Page 37: The Road to Autonomous Vehicles - … · Advantages of Redundant Sensor Fusion References: 1 Hall, David L., “Mathematical Techniques in Multisensor Data Fusion", Artech House Information](https://reader031.vdocuments.net/reader031/viewer/2022022613/5b9ea65009d3f26e288bc551/html5/thumbnails/37.jpg)
Typical Technician Skills Required in the Field of Automated Vehicles
• Basic automotive and prototype shop knowledge (teardown vehicles, build harnesses, basic fabrication skills, troubleshoot auto systems without manuals)
• Electronics skills (ECMs, sensors and sensor fusion, antennas, CAN and cable protocols, displays, soldering, shielding, troubleshooting)
• Software Skills (embedded systems, basic programming, networks, security systems, user interfaces)
• Understanding of Communication protocols (Satellite, LTE/cellular, WiFi, DSRC, Bluetooth)
• Lab testing, data acquisition and analysis
37
![Page 38: The Road to Autonomous Vehicles - … · Advantages of Redundant Sensor Fusion References: 1 Hall, David L., “Mathematical Techniques in Multisensor Data Fusion", Artech House Information](https://reader031.vdocuments.net/reader031/viewer/2022022613/5b9ea65009d3f26e288bc551/html5/thumbnails/38.jpg)
Stay Connected with the CAAT
• Visit our website at www.autocaat.org
• Sign up for our monthly newsletter
• Follow us on social media
• Contact us with your seed funding project ideas!
– Connected/Automated Vehicles
– Lightweighting
– Testing
– http://autocaat.org/Educators/Seed_Funding/
38
![Page 39: The Road to Autonomous Vehicles - … · Advantages of Redundant Sensor Fusion References: 1 Hall, David L., “Mathematical Techniques in Multisensor Data Fusion", Artech House Information](https://reader031.vdocuments.net/reader031/viewer/2022022613/5b9ea65009d3f26e288bc551/html5/thumbnails/39.jpg)
Thank You!
Questions?