pegasus method for assessment of highly automated ......pegasus method for assessment of highly...

PEGASUS Method for Assessment of Highly Automated Driving Function

Introduction to the Research Project

2

January 1st, 2016 – June 30th, 201942 Months Term

OEM: Audi, BMW, Daimler, Opel, Volkswagen

Tier 1: Bosch, Continental

Test Lab: TÜV SÜD

SMB: fka, iMAR, IPG, QTronic, TraceTronic, VIRES

Scientific institutes: DLR, TU Darmstadt

17 German Partners

i.a. IFR, ika, OFFIS12 Subcontracts

approx. 34,5 Mio. EUR

Subsidies: 16,3 Mio. EUR

Project Volume

approx. 1.791 man-month or 149 man-yearsPersonnel Deployment

Project for the establishment of generally accepted quality criteria, tools and methods as well as scenarios and situations for the release of HAD functions.

© PEGASUS

High standards regarding quality and performance of the automated vehicle

Measures that product needs to meet

Basic functionality is technologically given

Has been demonstrated in various projects

Together with electric driving, automated driving is tomorrow’s subject matter.


Resulting Starting Position – Automated Driving

3

Existing measures for testing and release are insufficient, too cost-intensive and too complex

Consequently, the introduction of highly automated driving features today can only be achieved with great expenditure.

© PEGASUS

How can complete-ness of relevant test runs be ensured?

What do the criteria and measures for these test runs look like?

What can be tested in labs or in simulation? What must be tested on proving grounds, what must be tested on the road?

Which tools, methods and processes are necessary?

What human capacity does the application require?

What about technical capacity?

Is it sufficiently accepted?

Which criteria and measures can be deducted from it?


Major Questions of the Project

4© PEGASUS

Scenario Analysis & Quality Measures

ImplementationProcess Testing

Is the concept sustainable?

How does the process of embedding work?

Reflection of Results & Embedding

What level of performance is expected of an automated vehicle?How can we verify that it achieves the desired performance consistently?

Zentrale Fragestellungen in PEGASUS

How safe is safe enough andhow can we verify that it achieves the

desired performance consistently?

is:

Method for Assessment ofHighly Automated Driving Function

PEGASUS approach

to answer the question

5© PEGASUS

Database

Data processing Database

Assessment of Highly Automated Driving Function

Requirementsdefintion

Test HAD-F:

SimulationProving Ground

Real World Drive

Evidence

Co

ntrib

utio

n to

Safe

tyState

me

nt

Data inPEGASUS-

Format

Application of Metrics + Mapping to Logical

Scenarios

Logical Scenarios +

Parameter Space

Preparation for Test Concept

IntegrationPass Criteria

Application of Test Concept incl. Variation

Method

PEGASUS Method for Assessment of Highly Automated Driving Function (HAD-F)

Risk Assessment

Release Test Evaluation Test Execution Test Case Derivation

Source of Information Evaluation & Conversion Scenarios

Argumentation

Contribution to Safety

Argumentation

Data / Content

Procedure

Workflow

Process Instruction

Processing

Preprocessing / Reconstruction

Systematic Identification of Scenarios

21

10 11

875

4

Daten:

Test DriveSimulationSimulatorFOT/NDSAccident

2

Use Case,Knowledge,

Data

V1.4 Status21.09.2018

Process Guidelines + Metrics for HAD

Assessment

6

cen

tral

dec

entr

al

20 19 15 13

Space of Logical Test Cases

12Test C

ases

14

Test D

ata

16

Test R

esu

lts

18

Evaluation and

Classification

17

9Knowledge:

Laws,Standards,

Guidelines, ...

1Requirements Analysis

3

6© PEGASUS

Database




Test HAD-F:


Real World Drive

Evidence

Co

ntrib

utio

n to

Safe

tyState

me

nt

Data inPEGASUS-

Format


Scenarios

Logical Scenarios +

Parameter Space




Method


Risk Assessment



Argumentation


Argumentation

Data / Content

Procedure

Workflow

Process Instruction

Processing



21

10 11

875

4

Daten:


2

Use Case,Knowledge,

Data

V1.4 Status21.09.2018


Assessment

6

cen

tral

dec

entr

al

20 19 15 13


12Test C

ases

14

Test D

ata

16

Test R

esu

lts

18

Evaluation and

Classification

17

9Knowledge:

Laws,Standards,

Guidelines, ...


3

7

Start:Use-Case

Goal:Safety

argument

© PEGASUS

Database




Evidence


Argumentation

V1.4 Status21.09.2018

8

Start:Use-Case

Goal:Safety

argument

© PEGASUS

Database




Evidence


Argumentation

Use Case,Knowledge,

Data

V1.4 Status21.09.2018

9© PEGASUS

Use Case

© PEGASUS 10

Safeguarding of Level 3 (Highly Automated Driving) function

Based on an application-oriented example, highway chauffeur Basic function: Highways or highway-like roads incl.

road markings Speed 0 - 130 km/h Automated following in stop & go traffic

jams Automated lane changing Automated emergency braking and

collision avoidance

Construction sites Automated exiting off the highway Extreme weather conditions

source: VW

Database




Evidence


Source of Information Evaluation & Conversion

Argumentation

Data / Content

Procedure

Workflow

Process Instruction

2

Use Case,Knowledge,

Data

V1.4 Status21.09.2018


Assessment

6

cen

tral

dec

entr

al

Knowledge:Laws,

Standards,

Guidelines, ...


3

11© PEGASUS

How good is good enough?

Which functional performance does a highly automated druving function need to reach a social acceptance?

© PEGASUS 12

Driving skills

%

Ø ?

Autopilot ?

less good good realy good

Human driver

Social acceptance

Database




Evidence


Source of Information

Argumentation

Data / Content

Procedure

Workflow

Process Instruction

Daten:


2

Use Case,Knowledge,

Data

V1.4 Status21.09.2018


Assessment

6

cen

tral

dec

entr

al

Knowledge:Laws,

Standards,

Guidelines, ...


3

13© PEGASUS

Input data

© PEGASUS 14

NDS / FOT

Simulator

crash

Real world

Simulation

source: UDRIVE, IPG, Audi, DLR

http://www.google.de/url?sa=i&rct=j&q=&esrc=s&source=images&cd=&cad=rja&uact=8&ved=2ahUKEwjk6cKot6PaAhVC_qQKHZ7KAaIQjRx6BAgAEAU&url=http://www.zeit.de/mobilitaet/2015-04/autonomes-fahren-audi&psig=AOvVaw3EWli0Blw1yZP6DiUp-U3i&ust=1523027909795447

http://www.google.de/url?sa=i&rct=j&q=&esrc=s&source=images&cd=&cad=rja&uact=8&ved=2ahUKEwjt4YmBtqPaAhUxsaQKHY8PDOQQjRx6BAgAEAU&url=http://www.traffictechnologytoday.com/news.php?NewsID%3D72233&psig=AOvVaw0YpXlPuN3Uk7Ql5PgodotM&ust=1523027564065435

https://www.google.de/url?sa=i&rct=j&q=&esrc=s&source=images&cd=&cad=rja&uact=8&ved=2ahUKEwiQzpLjtqPaAhVB_aQKHXttB2YQjRx6BAgAEAU&url=https://mhh-unfallforschung.de/de_DE/publikationen/&psig=AOvVaw3hag7IG3pfLGU3MxmaC62O&ust=1523027784576979

Database




Evidence

Data inPEGASUS-

Format


Scenarios

Logical Scenarios +

Parameter Space



Argumentation

Data / Content

Procedure

Workflow

Process Instruction



875

4

Daten:


2

Use Case,Knowledge,

Data

V1.4 Status21.09.2018


Assessment

6

cen

tral

dec

entr

al

9Knowledge:

Laws,Standards,

Guidelines, ...


3

15© PEGASUS

Scenarios and possibilities for description

© PEGASUS 16

Functional scenariosBase road network:Three-lane motorway in a curve, 100 km/h speed limitindicated by traffic signsStationäre Objekte:

-

Bewegliche Objekte:

Ego vehicle, Traffic jam;Interaction: Ego in maneuver„approaching“ on the middlelane, traffic jam moves slowly

Umwelt:

Summer, rain

Logical scernarios

Basisstrecke:Lane width [2..4] mCurve radius [0,6..0,9] kmPosition traffic sign [0..200] m

Stationäre Objekte:

-

Bewegliche Objekte:

End of traffic jam [10..200] mTraffic jam speed [0..30] km/hEgo distance [50..300] mEgo speed [80..130] km/h

Umwelt:

Temperature [10..40] °CDroplet size [20..100] µmrainfall [0,1..10] mm/h

Level of abstraction

Concrete scenariosBasisstrecke:

Lane width 3Curve radius 0,7 kmPosition traffic sign150 m

Stationäre Objekte:

-

Bewegliche Objekte:

Stauende_Pos 40 mStau_Geschw. 30 km/hEgo_Abstand 200 mEgo_Geschw. 100 km/h

Umwelt:

Temperature 20 °CDroplet size 30 µmrainfall 2 mm/h

Number of scenarios

Layer model Level of abstraction

1

2

3

4

5

sourcee: fka, TU BS

Database




Evidence

Data inPEGASUS-

Format


Scenarios

Logical Scenarios +

Parameter Space





Argumentation

Data / Content

Procedure

Workflow

Process Instruction

Processing



10 11

875

4

Daten:


2

Use Case,Knowledge,

Data

V1.4 Status21.09.2018


Assessment

6

cen

tral

dec

entr

al


12

9Knowledge:

Laws,Standards,

Guidelines, ...


3

17© PEGASUS

Database




Evidence

Data inPEGASUS-

Format


Scenarios

Logical Scenarios +

Parameter Space




Method


Test Case Derivation


Argumentation

Data / Content

Procedure

Workflow

Process Instruction

Processing



10 11

875

4

Daten:


2

Use Case,Knowledge,

Data

V1.4 Status21.09.2018


Assessment

6

cen

tral

dec

entr

al

13


12Test C

ases

14

9Knowledge:

Laws,Standards,

Guidelines, ...


3

18© PEGASUS

Test concept

© PEGASUS 19

Space of logical test

cases

Allocation of test

cases to test

platform

Automatized variation of parameter

Manual selection of scenarios/ parameter(e.g. tests

as per ECE R79, rating

tests)

Specific scenarios

Test execution looking for

critical cases(e.g.

accidents)

Evaluated concrete scenarios(Pass/Fail)

Pass-/Fail-Criteria

Test object: Functional implementation of highway chauffeurTest level: Functional TestTest platform : Simulation

Test object: Overall system highway chaffeurincluding vehicle behaviorTest level: Vehicle testTest platform : Test ground/ field

Test execution Driving on

TG + validation simulation

Test execution of real world drive with

guidelines:• Route• Weather• Time• Specific

features

“all logical scenarios”

„selected scenarios“

no direct link to space of logical test cases

Comparison

Test platform

Simulation

Feedback

Replay2Sim

Logical scenario+ parameter

space+ exposure of

parameter

Test platform

Test ground

Test platform

Field

Concretescenarios

„Surprises“

Measured data for

database

Measured data for Replay2Sim

Critical cases

Database




Test HAD-F:


Real World Drive

Evidence

Data inPEGASUS-

Format


Scenarios

Logical Scenarios +

Parameter Space




Method


Test Evaluation Test Execution Test Case Derivation


Argumentation

Data / Content

Procedure

Workflow

Process Instruction

Processing



10 11

875

4

Daten:


2

Use Case,Knowledge,

Data

V1.4 Status21.09.2018


Assessment

6

cen

tral

dec

entr

al

15 13


12Test C

ases

14

Test D

ata

16

Test R

esu

lts

18

Evaluation and

Classification

17

9Knowledge:

Laws,Standards,

Guidelines, ...


3

20© PEGASUS

Database




Test HAD-F:


Real World Drive

Evidence

Co

ntrib

utio

n to

Safe

tyState

me

nt

Data inPEGASUS-

Format


Scenarios

Logical Scenarios +

Parameter Space




Method


Risk Assessment



Argumentation


Argumentation

Data / Content

Procedure

Workflow

Process Instruction

Processing



21

10 11

875

4

Daten:


2

Use Case,Knowledge,

Data

V1.4 Status21.09.2018


Assessment

6

cen

tral

dec

entr

al

20 19 15 13


12Test C

ases

14

Test D

ata

16

Test R

esu

lts

18

Evaluation and

Classification

17

9Knowledge:

Laws,Standards,

Guidelines, ...


3

22© PEGASUS

Contact:

Prof. Dr..Ing. Thomas FormHead of Vehicle Technology and Mobility Experience,Group ResearchVolkswagen [email protected]

www.pegasusprojekt.de

23© PEGASUS

pegasus method for assessment of highly automated ......pegasus method for assessment of highly...

Documents