autonomous driving intelligence system for safer automobiles...2.0 2.5 3.0 0 20 40 60 v elocity at...
TRANSCRIPT
1
Autonomous Driving Intelligence System
for Safer Automobiles
Hideo InoueKanagawa Institute of Technology (KAIT) , TUAT
Pongsathorn Raksincharoensak Yuichi Saito
Tokyo University of Agriculture and Technology (TUAT)
2018.September 11-12_IPG-Automotive Apply & Innovate 2018
2
1. Motivation and objectives
2. Risk predictive control ; outline of physical model
3. Safety cushion ;
Context-sensitive hazard anticipation based on
driver behavior analysis and cause-and-effect chain study
4. Summary
Table of contents
Academia
Industry
Government
CommunityCitizen
cooperative project Realizing lively and active lifestyle
for elderly people
3
Deterioration in driving ability reduces self-confidence in driving.
However, elderly drivers are highly motivated to improve QOL.
Intelligent driving systems can help to compensate for this deterioration in
physical ability and overcome the fear of driving.
Recognized the warning but did not brake.
Did not recognize the warning and did not brake
The older the driver, the higher the ratio
that could not recognize the warning.
Drivers over age 60 recognized the
warning but did not brake.
0% 20% 40% 60% 100%80%
Ag
e
20 to 39
40 to 59
60 to 69
Recognized the warning and operated brake.
Fig. 1 Reaction of drivers to active safety system
(Experimental study using Toyota Driving Simulator)
Fig. 2 Vehicle necessity
0% 10% 20% 30% 40% 50% 60% 70% 80% 90%
1
Can be used anytime, anywhere
Trains and buses cannot be used
Trains and buses limit time
Trains and buses take too much time
Love to drive for fun
Elderly drivers have high motivation to drive.
Motivation and objectives: Overcoming fear of driving
4
Our challenge to enhance safety technologies by foresighted driving
Establish driving intelligence system
to realize risk predictive driving.
Normal
driving
Risk
predictive
driving
Emergency
driving
Accident !!
10s
5s
1s
10ms
Adaptive Cruise Control
Lane Keeping Assist
Autonomous Emergency Brake
Electronic Stability Control
Avoid collision
Put to practical use
Predict and avoid risk
Enhance comfort
Not established
Put to practical use
Research target
5
Autonomous Driving Intelligence System
Motion P
lannin
g
Refe
rence M
aneu
ver
Course
Distance
to obstacle
Blind spotHorizon
camera
Map
LiDAR
radar
IMU
+
Haptic S
ha
red C
ontr
ol
Elderly Drivers
Vehic
le+
Haptic
Controlwith HMI
Intervention
CAN
Emergency avoidance model
Vped V f
D ped
歩行者飛び出し 速度、タイ ミング
走行速度
Potential
Driver
Maneuver
Dynamics
Environment
10 20 30 40 -2 0 20
2000
4000
6000
Y[m]X[m]
Sensor
Fusio
nS
ce
ne
de
tectio
n ・
Ris
k d
efin
itio
n
A:Driving Environment
Recognition System(Lean SLAM)
B:Risk Predictive System(Expert driver model)
C:Shared Control System(Haptic control torque)
Data-driven
AI
6
1. Motivation and objectives
2. Risk predictive control ; outline of physical model
3. Safety cushion ;
Context-sensitive hazard anticipation based on
driver behavior analysis and cause-and-effect chain study
4. Summary
Table of contents
7
Risk prediction model: Defensive driving, object motion prediction, hazard anticipation
Contour of risk potential generated from surroundings and on-road objects
Driving intelligence must predict the possibility of the appearance of pedestrians from
behind an object and determine the optimum safe speed.
Near-miss data High risk
Low risk
Risk prediction model
8
YXUYXUYXU obsroadrisk ,,,
+
YXU road , YXUobs ,
pyJt
p
min*
Risk predictionRepulsive potential form
Motion planningOptimization problem for optimal yaw rate calculation
n
i
ipipipriskpy qYXUJ1
2
,,, )(),(
Optimal yaw rate:
Cost function:
* Risk is determined by environmental states
(road and obstacles)
Risk potential Steering input
Planned vehicle motion
Risks are predicted by environmental states.
Trajectories of expert drivers can be simulated.
Movie④
Movie③
Expert Drivers Standard Deviation Simulation
Risk predicted motion planning
9
Select the deceleration which results in minimum risk path.
Candidate
Candidate
Risk Potential of Collision with pedestrian
Cost Function Balancing Collision Risk and Vehicle Deceleration (Speed-down)
Longitudinal Control : avoid crash with (virtual) pedestrian
10
Risk Potential Based Motion Planning Method
2
2
2
2 )()(exp,
oY
o
oX
oobs
YYXXYXU
X
Y
+ (Xo, Yo)
Ys (way-point)
Xo Yo
2
2
2exp1,
r
sroad
YYYXU
Ys
Y
X X
Y
`Risk Potential of obstacle
Overtaking a parked car scene
(Including a rush-out pedestrian)
Risk Potential of road boundary
11
X
Y
2
2
2
2 )()(exp,
oY
o
oX
oobs
YYXXYXU
X
Y
+ (Xo, Yo)
Ys (way-point)
Xo Yo
2
2
2exp1,
r
sroad
YYYXU
Ys
Y
X
`
X
Y
YXUwYXUwYXU obsoroadrrisk ,,,
Risk Potential of obstacle
Overtaking a parked car scene
(Including a rush-out pedestrian)
Risk Potential of road boundary
Risk Potential Based Motion Planning Method
12
X
Y
Low risk
+ (Xo, Yo)
Ys (way-point)
X
Y
Medium
X
Y
Risk
high
+ (Xo, Yo)
Ys (way-point)
+ (Xo, Yo)
Ys (way-point)
Low Risk Potential High
Near-miss incident DB
YXUwYXUwYXU obsoroadrrisk ,,, YXUwYXUwYXU obsoroadrrisk , ,, YXUwYXUwYXU obsoroadrrisk ,,, wowo
Cyber-Physical System : Data-driven A.I. for intelligent vehicles
13
Hierarchical structure of risk predictive control
Waypoints
Normal
Driving
Course
Intersection
Ref. driving
course
Parked vehicle Pedestrian
Occluded object
Risk Predictive
Driving
Invisible risks potential
Basic automated driving potential
Data-driven AI
Hazard weighting factor
(Pedestrian, bicycle, etc. )
+
Map
information
Recognition
information
① Basic driving potential to reproduce course collecting of the model driver.
② Obvious risk potential linked with Data-driven AI
to optimal control brock
Virtual objects
Near-miss incident DB
Visible risks potential
14
1. Motivation and objectives
2. Risk predictive control ; outline of physical model
3. Safety cushion ;
Context-sensitive hazard anticipation based on
driver behavior analysis and cause-and-effect chain study
4. Summary
Table of contents
15
Near-miss Incident
16
Traffic Big Data: TUAT Near-miss Incident Database
A
Annotation
Road type (straight/curved)
Intersection type
Traffic participants density
Number of bars
Pedestrian age
….
17
Pedestrian
Conflict point×
Obstacle
Stopping distance
Pedestrian
Conflict point×
Obstacle
Stopping distance Safety cushion
𝑉𝑐𝑎𝑟′𝑉𝑐𝑎𝑟
𝑉𝑐𝑎𝑟
Unsafe driver
Careful driver
𝑉𝑐𝑎𝑟 > 𝑉𝑐𝑎𝑟′
Dcar DpedOccluded area
Occluded area
Adequate adjustments of safety cushion decreases the number of necessary emergency brakes.
Safety Cushion
18
Conflict: too fast driving: high risk
Our motivation is to create adequate safety cushion through machine learning
Techniques by taking advantage of past knowledge/experience of near-miss incident.
Motivation: Adequate Safety Cushion
too slow driving: no driver acceptance
19
Y
X
Pedestrian
carVCollision point
×
Blind area
Parked vehicle
Ygap
Velocity data
න𝑡∗
𝑡2
𝑉𝑑𝑡Dcar (t*) +Dped (t*) =
t* t2
Dcar Dped
Vehicle
Condition for avoiding a crash
Distance to collision point Require distance to stop
τ: System delay
→Time margin indicator to show near-miss incident level
Safety cushion time
Levels of Near-Miss Incident
20
Pedestrian behavior
(Dped, Vped)
Driver behavior (Vcar)
Vehicle dynamics
constraints
t1 t2 t3No road crossing
Caution/ careless driver
Good anticipation/ No-good anticipation driver
Stopping distance [m]
Multiple nonstationary
t3 t2 t1
μ, amax, τ
Vcar, ax
Factors that Reduce Safety Cushion Time: Conflict Pattern
No road crossing Road crossing !!
Safety cushion time represents the result of conflict of these factors.
21
• The Behavior Models were created for each traffic opponent group (vehicle, pedestrian, bicycle).
• The Behavior Models use the quantified context parameter values (fixed, cannot be influenced) and
the quantified driving behavior (can be actively adapted) as input and correlate them with the
incident level (criticality of the incident)
• The Behavior Models describe the relationship between the input parameters and are used to predict
the resulting incident level.
Quantified context parameter
⇒ Risk values
Behavior Model:
Influences of each input parameter and interactions
Prediction of incident level
Quantified driving behavior
⇒ LHP (Long-term Hazard Potential)
Behavior Model Generation
22
Near-miss
incident database
Test scenario
Annotations
Vehicle data
Feature value
Extraction
Preprocessing
LHP
Risk
values
Level identification
Machine learning
Statically processing
SCT
High lev. : 0 < SCT <= 1 sec
Mid. lev. : 1 < SCT <= 2 sec
Low lev. : 2 sec < SCT
Training data
Preprocessing
⇒ Cause and Effect Chain Studies
⇒ Human Err. Analysis
Important Aspects
Context parameter quantification
Driving behavior quantification
23
Proposed index to quantify the driving behavior
Long-term Hazard Potential (LHP)
The index focusing on the speed behaviour from [-10 sec to -4 sec] before the incident.
・ Maximum velocity increases LHP with large portion.
・ High median of the velocity increases LHP
・ Accelerating situation increases LHP.
・ Decelerating situation decreases LHP.
・ Large variance of velocity decreases LHP.[ -10 sec
to -4 sec ]
24
1.0
1.5
2.0
2.5
3.0
0 20 40 60
Velocity at time when the pedestr ian initiated the road crossing [km/h]
LH
P
0
10
20
30
1.0 1.5 2.0 2.5 3.0
LHP
Fre
qu
en
cy
Careful driver Aggressive driver1.85
R =0.81
Speed profile of careful and aggressive drivers is an acceptable trade-off
between desired vehicle velocity and driving safety.
Long-term Hazard Potential
25
Near-miss
incident database
Test scenario
Annotations
Vehicle data
Feature value
Extraction
Preprocessing
LHP
Risk
values
Level identification
Machine learning
Statically processing
SCT
High lev. : 0 < SCT <= 1 sec
Mid. lev. : 1 < SCT <= 2 sec
Low lev. : 2 sec < SCT
Training data
Preprocessing
⇒ Cause and Effect Chain Studies
⇒ Human Err. Analysis
Important Aspects
Context parameter quantification
Driving behavior quantification
26
Context properties type Definition
Static properties Area type Residential area/Urban and business area/Rural area/Other
Road type Other/One way/Both way
Sidewalk type
Intersection type Other/ T and Y types/ 4 type/ More/ Straight
Road width Lanes: other/ 1/ 2/ 3/ 4/ 5 over
Crosswalk Without/ With
Dynamic properties Parked vehicle 0~2/3~5/More
Pedestrian 0~2/3~9/More
Traffic 0~2/3~9/More
Leading vehicle Without/ With
Other properties Time 6:00~10:00/10:00~16:00/16:00~20:00/20:00~6:00
Weather Sunny and cloudy/Rain and Snow
Age of pedestrian Unknown/Elderly/Mature/Young/Child
Qualitative Context Properties
27
sidewalk type
cond.1 cond.2 cond.3 cond.40
20
40
60
Nu
mb
er o
f in
cid
ent
[%]
High
Mid.
Low
sidewalk type
cond.1 cond.2 cond.3 cond.40
50
100
150
Ris
k v
alue
ID: 355291
ID: 389293
ID: 388181
ID: 388072
1 2
3 4
whigh : High level weight
wmid. : Mid level weight
wlow : Low level weight
fi : frequency, Order value∈ [1, 10]
Quantifying the risk level from driving context properties (Annotations)
28
0
1
2
3
4
5
0 1 2 3 4 5
Predicted safety cushion time
Mea
su
red
sa
fety
cush
ion
tim
e
N=109
SCT prediction model by linear regression analysis
29
• Area type: urban area
• Road type: two-way
• Intersection type: T type
• Parked vehicle: high density
• Leading vehicle: without
uncontrollablecontrollable
0.63
sec
4.02
sec
-2.26
sec
-1.16
sec
ID 355961
① Driver is required to decrease the velocity.
② Ref. vel. can be given from the linear regression.
Reference velocity
30
System Framework
TUAT
Near-Miss
Incident
Database
Camera
GPS, map
Motion sensor
Driving
Context
&
Driving
Behavior
Sensing
Driving
Context
Risk Level
&
Driving
Behavior
Evaluation(Speed,
Acceleration,
Yaw rate, etc.)
Desired safe speed
Real Time
Measurement
・・・・
desVSafe Speed Computation
(Worst Case Scenario)Safe Speed
Scene
classification by
traffic opponent
Vehicle
Pedestrian
Bicycle
Context
parameter
quantification
Risk Level value
for each context
parameter value
Driving behavior
quantification
Long-term Hazard
Potential (LHP)
Quantification of
the influence of
context parameters
and driving
behavior on the
incident level
Behavior Models
Motion
planning
(Risk potential)
31
Normal driving Risk predictive drivingEmergency
driving
10ms100ms1s2s5s10s
Car following, Course tracking AEB, PCS
Accident !!
Lane Keeping Control
Cooperative-ACC
Risk Predictive
Driving Intelligence System
Lane Departure Prevention
Autonomous Emergency Braking/
Pre-Crash Safety
Source : Toyota Source : Toyota
LEVEL 1, 2
LEVEL 1
Summary1: Advanced Safety Vehicle with Driving Intelligence
32
FeedbackTravel conditions database Human factor database
Fun to drive Optimum
eco-driving
Accident
avoidance
• Local driving intelligenceADAS with autonomous driving intelligence
• ICT: Info. & communication technology
V2X
Vehicle
• Global driving intelligenceNear-miss data analysis/accident analysis
Enhanced map for dynamic usage
Traffic congestion prediction
etc.
Big data
Summary2: Cyber-physical system for intelligent driving systems
33
Acknowledgement
This study has been conducted as a part of the research project “Autonomous Driving
System to Enhance Safe and Secured Traffic Society for Elderly Drivers” granted by
Japan Science and Technology Agency (JST). Special thanks to the agency providing
the financial support to conduct the research.
In addition, EDI has supported “Context sensitive hazard
anticipation study” through collaboration with TUAT and KAIT.