1

Energy Research Institute @ NTU (ERI@N)1 CleanTech Loop, #06-04 CleanTech One, Singapore 637141Phone: (65) 6592 1786 / 2468 Fax: (65) 6694 6217

Singapore TR 68 : 2019 - Technical Reference for Autonomous Vehicles

Presentation for IDVS4, 28 Oct 2019

Niels de Boer

2 CONFIDENTIAL

Singapore Urban Mobility Challenge

66%increase in domestic travel in last 10

years

55%of bus drivers are foreigner

12%of land used for roads

14%of land is used to house 5.5M people

versus

There is not enough space to increase the number of vehicles at the same rate as the population growth

1/3of the work force is over 50

+ 4%of land used for car parks

3 CONFIDENTIAL

Car Lite Estates

3

4

5 CONFIDENTIAL

CETRAN Centre of Excellence for Testing & Research of AVs –NTU

Centre of Excellence to support Singapore AV community Standards development Developing AV testing procedures Perform AV testing on behalf of LTA to support issue

of AV Authorization Technical lead in development of AV Technical

Reference Operator of CETRAN AV Test Centre Linking with other countries to align on standards

and testing Supporting skills development for Industry AV Developer support

6 CONFIDENTIAL

AV Trial Testbed @ one north

Test bed at One North as a future proof concept for AV testing Infrastructure to support trials Closed Circuit CCTV 5.9 GHz Dedicated Short

Range Communication (DSRC) beacons

Backend system

7 CONFIDENTIAL

Milestones are used to assess AVs before they are allowed to proceed to their next phase of trial Each milestone test will produce a test report with recommendation which is used by the Land Transport

Authority as one of the decision criteria to “pass” the AVs

Milestone 1 Ability to safely conduct

testing of autonomous vehicles with safety driver in a small scale testbed.

Milestone 2 Ability to safely conduct testing of autonomous vehicles with safety driver in a complex

environment.

Milestone 3 Ability to safely conduct testing of autonomous vehicles without or with a safety driver (with limited

control) in a complex environment. This implies high technical maturity.

Milestones for AV Trials

8 CONFIDENTIAL

Circuit Based Test

Milestone 1 Test Framework

Assessment of basic safe construction of vehicle

Assessment of the ability of the safety driver to conduct on road tests safely Not a test of the driver, but controls available to the driver

Basic AV capabilities Lane keeping / basic driving capability Controls available to safety driver Static obstacle Dynamic obstacle

9 CONFIDENTIAL

Document Review Simulation Circuit

Based Test Road Test

Milestone 2 Test Framework

Assessment of basic capabilities of vehicle

Assessment of the process capabilities of the developer Do they have processes in place to ensure that updates to the AV hardware/software is

carried out in a safe manner?

Assessment of simulation capabilities Do they have appropriate/adequate simulation capabilities to test out their software

before deployment?

10 CONFIDENTIAL

M3 assessment flow

The overall M3 test scope tailored to the operational environment and capabilities of the vehicle

AV developers request additional clarification on the relationships between milestones and test areas in order to develop true driverless capabilities

TR-68 AUTONOMOUS VEHICLES

12 CONFIDENTIAL

Technical Reference - 68

Technical Reference (TR) is a guideline published by the Singapore standards body Enterprise Singapore (formerly named SPRING) Managed in a same way as a standard Technical Committee to oversee TR Working Groups to write/revise content Industry participation at Technical Committee and Work Groups

TR seeks industry adoption as trial for a period of time, and comments received will be reviewed after the trial period

TR can become a Singapore Standards, continue as TR for further industry trials or be withdrawn

12

13 CONFIDENTIAL

TR Development Guideline

Form TR Development

Workgroup

Establish TR

Revise content if required

Sep/Oct 2017 ~ 202x

Establish Singapore Standards

Releases• First release TR-68: 2019 Release 31 Jan 2019• Second release TR-68: 2020 Target Release Q4 2020• Third release TR-68: 2021 Considered for release Q4 2021

14 CONFIDENTIAL

Technical Reference Scope

AV Technical Reference

Basic Behavioural Guidelines Safety Guidelines Cybersecurity Guidelines Vehicular Data Types and

Formats

Drive like a safe driver

&clarify rules in Theory of

Driving

Ensure the system is

safe

Ensure the system is

secure

Talk a common language

PART 1 – BASIC BEHAVIOURAL GUIDELINES

16 CONFIDENTIAL

Basic Behavioural Guidelines

What are they? Rules and clarification to the driving license requirements as described in the Basic Theory of

Driving and the Final Theory of Driving

Why are they needed? The driving rules are written for human drivers, not engineers Because desired driving behaviour is context sensitive, the driving rules are not black or white. “The driver shall indicate in time when changing lanes” (what is “in time”?) “The driver shall pass a stationary object on the road with a gap of minimum 1.5m” (So if the 1.5 m gap

is not feasible, do you stop and just wait till something makes it feasible?) With the current state of technology, some driving rules are no longer relevant or are no longer

able to be complied with. Waiting till the technology is matured might not be a realistic option. “The driver shall follow the hand signals of an authorised officer” (if AI can recognise all hand signals,

can it differentiate between an authorised officer and a random person? And what if this random person is a citizen taking necessary action at an accident site until the police has arrived?)

16

17 CONFIDENTIAL

Challenges: what is the desired behaviour?

Stop DistanceV

Y

A

BC

DBraking/detection zone

X

A B C D

speed

Milestone 1 test

X = 2 m

time

18 CONFIDENTIAL

Challenges: what is the desired behaviour?

Stop DistanceV

Y

A

BC

DBraking/detection zone

X

A B C D

speed

Milestone 2 test

X = 2 mtime

Gentle slowdown equivalent to lifting throttle

More gentle braking due to slow down

(risk reduction) earlier

Increased response time

PART 2 - SAFETY GUIDELINES

20 CONFIDENTIAL

Safety Guidelines

What are they? Guidelines for recommended safety processes to assure the safety of systems

deploying SAE level 4 and level 5 autonomous vehicles

Why are they defined? Capture industry best practices Contains elements of railway practices used for assurance of driverless trains Contains elements of aviation practices used for assurance of drones and UAVs

Especially help new entrants (start-ups and companies from non-automotive backgrounds) to understand expectations and needs for ensuring safety of driverless vehicles

21 CONFIDENTIAL

Overall approach in Goal Structured Notation format

Autonomous Vehicles operations are acceptably safe

by design

Argument design

suits ODD

Argument HARA is

comprehen-sive and complete

Argument of acceptable design

and production quality

Argument that acceptable

evidence has been provided for initial safety compliance

Argument that acceptable

evidence has been provided for

continuing safety compliance

ODD adequately described for

designers of each class of road environment

Goal

Solution

Argument

PART 3 - CYBERSECURITY

23 CONFIDENTIAL

Cybersecurity Assessment - Process

System Assessment

Penetration Testing

Report

Evaluations Recommendations

VulnerabilityAnalysis Fuzz Testing Attack Simulation

Document Review

Results

Threat riskassessment

Threat Management

System Architecture

OverviewIntended Use

Test ThreatRisk AnalysisSystems Inspection Threat Risk

Analysis Process

PART 4 – DATA FORMATS

24

25 CONFIDENTIAL

AV cosystem

25

26 CONFIDENTIAL

Message format: Abstract Syntax Notation One (ASN.1)

27 CONFIDENTIAL

Feedback on TRDevelop feedback on TR

1. Use TR content as input to milestone plan2. Get feedback from AV developers 3. Work with AV developers to find solutions and improvements4. Feed back recommendation to TR Working Group for inclusion in revision

Existing standards, data and best practices

Input to Technical Reference

Supporting R&D

Milestone Test Development

Industry Feedback

TR Working Groups

Energy Research Institute @ NTU (ERI@N)1 CleanTech Loop, #06-04 CleanTech One, Singapore 637141Phone: (65) 6592 1786 / 2468 Fax: (65) 6694 6217

Thank you