an adaptive driver alert system beyer bertolotti cristiani dehni

7/31/2019 An Adaptive Driver Alert System Beyer Bertolotti Cristiani Dehni

1/40

Gilbert Beyer, Gian Mario Bertolotti, Andrea Cristiani, Shadi Al Dehni

AN ADAPTIVE DRIVER ALERT

SYSTEM MAKING USE OFIMPLICIT SENSING AND

NOTIFICATION TECHNIQUES

2ND WORKSHOP ON USER CENTRIC PERVASIVEADAPTIVE SYSTEMS | 06.12.2010


2/40

AN ADAPTIVE DRIVER ALERT SYSTEM | UCPA2010| 06.12.2010

Page 2 of 41

Agenda

1 Adaptation Strategy

2 Sensor-Actuator Support3 Hardware and Sotware

4 Perormance Tests

5 Discussion


3/40


Page 3 of 41

Driver Alertness


4/40


Page 4 of 41

Causes o Driver Distraction

The driver himsel (atigue, drunkeness)

The vehicle (type, condition)The environment (trac, weather)


5/40


Page 5 of 41

Long-Term Driver Distraction

Psycho-physiological state (atigue)

Secondary tasks (operating car stereo)Social activities (chatting with co-driver)


6/40


adapted from Riener and Ferscha (2009)

Page 6 of 41

Feedback Loops in Vehicles


7/40


Page 7 of 41

Sensing Driver Distraction

Hands-o-wheel Time

Eyes-o-road TimeConversational Level

Emotional expressions

Signs o drowsiness


8/40


Page 8 of 41

Notication Modalities

1 Vision, Hearing, Touch (each

having its own timely behavior)2 Multimodal inormation


9/40


Page 9 of 41

Adaptation Strategy

Actuator: Visual, implicit

Sensor 1: DrowsinessSensor 2: Head-position

Sensor 3: Can Bus data


10/40


Page 10 of 41

Combing Car and User Data

Eyes-o-road while...

1 driving ast, is dangerous2 standing, is sae

3 turning, is required


11/40


http://www.bmw.com/ (2010)

Page 11 of 41

Head-up Displays


12/40


Karvonen et al. (2006) Dienelt, M. (2005)

Page 12 of 41

Adaptive Head-up Displays


13/40


Blanco et al., US Patent No. 7764247 AWAKE Project, IST-2000-28062

Page 13 of 41

Adaptive Head-up Displays


14/40


Page 14 of 41

Sensing Drowsiness

1 Intrusive techniques:

Electromyogram, Respiration,

GSR, Electro-oculography

2 Non-intrusive techniques:

Vision-based


15/40


Page 16 of 41

Automotive Demonstrator


16/40


Page 17 of 41

Sensor-Actuator Support


17/40


Page 18 of 41

Simulator Setup


18/40


Page 19 of 41

Implicit Sensing Sotware

1 Drowsiness Detection

2 Head Position Detection


19/40


Page 20 of 41

Drowsiness Detection

Johns, M.W. (2003)


20/40


Page 21 of 41

Drowsiness Detection

Johns, M.W. (2003)


21/40


Page 22 of 41

Main Constraints


22/40


Page 23 of 41

Main Constraints

1 Reduce computational load

2 Reduce error rate


23/40


Page 24 of 41

Eye Detection Algorithm

1 Search eye using Viola-Jones

2 I eye is ound, save copy as template

3 I eye is not ound: template matching

4 Successive ailures: re-initialization


24/40


Page 25 of 41

Eye Detection Algorithm

Largest rectangle: Viola Jones, Template Matching Adjusted search window


25/40


Page 26 of 41

Blink Test Algorithm

1 Eye-image processed to binary picture

2 Eliminating pupil refections and noise

3 Obtain vertical projection histogram

4 Calculate eye openness level


26/40


Greyscale image Binary image

Page 27 of 41



27/40


Open eye Closed eye

Page 28 of 41



28/40


Page 29 of 41

Head-Position Detection

Eye within rest area Eye outside rest area


29/40


Page 30 of 41

Experiments and User Analysis

1 Perormance Tests

2 Driver Observation in the Lab

3 Road Tests


30/40


Page 31 of 41

Passive Sensing Perormance Tests

Collected sets o data:

1 Recognition o slow/ast blinks

2 Tracking o slow/ast eye movements

3 Slow/ast head movements

4 Blink ail errors


31/40


Page 32 of 41


Kinds o error:

1 tracking miss (in successive rames)

2 detection miss (shadows or occlusions)

3 blink miss (e.g. due to rapid blinks)

4 blink ail (erroneously detected blink)


32/40


Page 33 of 41


AN ADAPTIVE DRIVER ALERT SYSTEM | UCPA 2010 | 06 12 2010


33/40


Page 34 of 41




34/40


Page 35 of 41

Night Vision Camera



35/40


Page 36 of 41

Lab Study



36/40


Page 37 of 41

Lab Study

1 Eyes-o-road and Car State Symbols

2 15 Participants (students, colleagues)

3 Co-Driver was investigator

4 Semi-structured Interviews and

Camera Observations



37/40


Page 38 of 41

Lab Study

1 80 % understood that displayed

icons related to their driving behavior

2 66 % ound notications helpul

to adapt their driving style

AN ADAPTIVE DRIVER ALERT SYSTEM | UCPA 2010 | 06.12.2010


38/40


Page 39 of 41

Onboard Testing Session

1 Displaying Car State Inormation

2 Thinking Aloud Method

3 Renement o the hardware

and sotware setup



39/40

Page 40 of 41

Onboard Testing Session



40/40

Page 41 of 41

Discussion

Algorithm Improvements

Sensing Techniques

Future Tests and Studies

HUD Development

System Design

an adaptive driver alert system beyer bertolotti cristiani dehni

Documents