replacing the driver - self driving track days€¦ · replacing the driver. public 2 autonomous...

PUBLIC

NXP AUTO BU, ADAS PL

MARK O’DONNELL

REPLACING THE DRIVER

JULY 2017

PUBLIC 1

Autonomous driving

Driver replacement

Accelerations for awareness and for

knowledgeable decisions

NXP’s approach

Replacing the Driver

PUBLIC 1

PUBLIC 2PUBLIC 2

Autonomous Driving

01

PUBLIC 3PUBLIC 3

Safe & Secure Mobility 90% innovation through electronics

Enjoying Life

One hour per

day in the vehicle

Seamlessly Connected

Mobility Experience

US mandates 163 grams / mile

and 54.5 MPG by 2025

Reducing CO2

Energy

Efficiency

Saving Lives

1.3M global road fatalities

every year

ADAS Towards

Self-Driving

PUBLIC 4PUBLIC 4

Safe & Secure Mobility 90% innovation through electronics

Enjoying Life

One hour per

day in the vehicle

Seamlessly Connected

Mobility Experience

US mandates 163 grams / mile

and 54.5 MPG by 2025

Reducing CO2

Energy

Efficiency

Saving Lives

1.3M global road fatalities

every year

ADAS Towards

Self-Driving

PUBLIC 5

Automated & Autonomous – A Revolution

Level 3 Conditional

Level 4 High Automation

Level 5 Full Automation

Driver

Vehicle

Driver

Vehicle

Driver

Vehicle

360º Sense

Behavior

Safe

Redundant

Fail Operations

Recoverable

Highest coverage

Full redundant

Dependable

Not a question of IF, WHEN, or HOW

But HOW FAST and will it be ACCEPTED?

PUBLIC 6

Comfort Safety Efficiency Easier Mobility

We Seek…

PUBLIC 7

Legislation &

Harmonization

New

Technology &

Lifestyle

Trust &

Dependability

Challenges Are Not Only Technical

PUBLIC 8

Situational

Assessment

Behavioral Policy

& Actuation

At a High Level…

Sensors & Actuators mix

Fusion architectures &

levels of complexity

Interoperability and

heterogeneous solutions

Decision frameworks

and algorithms

Safe and secure actuation

Scalability of application

and deployment

PUBLIC 9PUBLIC 9PUBLIC 9

DRIVER REPLACEMENT

Driver

Repla

cem

ent

Radar

Camera

Lidar

Fusion

SENSE THINK ACT

02


Situational Assessment – 360º Sensing

Radar LidarCamera

Robust performance

under all conditions

Miniaturization /

cost effectiveness

Improved Perception: height,

imaging, classification

Interference sensitive

Semantic aware

See colors

Most advanced ‘human’ sense –

friendly representation

Strong AI know-how attached

Worsen at night and in bad weather

Highly precise

Used for global mapping

Fast developing technology

Classification and motion

detection

PUBLIC 11PUBLIC 11

V2X Map & Localization

“Out of Sight” Sensing

Sensing + Communication

5G and DSRC both

coexisting

GLOBAL and LOCAL

Mapping both required,

reciprocally integrated

Data Management

(maintenance and OTA

update)

PUBLIC 11

Additional Key Dimensions

PUBLIC 12PUBLIC 12

Probabilistic• Analyze Scenario

• Make Contextual Decision

• Strong AI centered

Deterministic• Initiate Safe Measure

• Fail Safe / Operational

• Traditional Automotive

Automated

Drive

Co-pilot Collision

Avoidance

Self Parking Lane Keeping Collision Warning Sign View

Super Computing Safe Computing

Modeling and behavioral policy

processing

Safe Processing

x xx x x xx x

$

x xx x

$

Massive Growth in Computing and Safety Requirements

PUBLIC 13PUBLIC 13

Topologies for Fusion (DoL Classification)

13

Most likely: flexible

and adaptable• Allows both Smart Sensing and

central fusion

• Manages costs of Data distribution

vs. precision for fusion

• Distributes effort and helps for

interoperability

Likely: complexity vs. cost• Requires industry level interoperability

• Data observation to be aligned and

associated all at once

• Most accurate way for fusion

• Extra large BW for raw data to be

TRX-ed

• Sensors are simplified and cheaper

Not likely: redundant ECUs• Each ECU does fusion at object level

• Each ECU knows all others data

• Lower BW for ‘fused’ data to be

exchanged

• Simultaneity required

• Increase sensors costs and complexity

Centralized Central server and naïve sensors

Distributed No central fusion unit

HybridCentral fusion & smart sensors

Central

Fusion

Domain

Fusion

Domain

Fusion

LOCAL

FUSION

Central

Fusion

PUBLIC 14

Accelerators for

Awareness & for

Knowledgeable

Decisions

03

PUBLIC 14

PUBLIC 15PUBLIC 15

AI

Machine Learning

Big Data

Computational

Acceleration

PUBLIC 15

PUBLIC 16PUBLIC 16

Dedicated

Accelerators

Von Neumann

Architecture

seems insufficient

Data Flow Architecture preferred:

Which one?

Extra

Large

Data

Execution

ParallelismPower

Efficiency

PUBLIC 16

PUBLIC 17

Always at Least One CPU to Over Control...and Then...

High parallel

DNN’s king

High power

Flexible

Future proof

Expensive

Efficient

Ecosystem

Complex

Very efficient

Non standard

Tailored

GPU FPGA DSP ML SPCPU

PUBLIC 18PUBLIC 18

Which Other Variables to Consider

When Choosing

Programming ease

Computational performance

Power

Safety & security

Throughput & latency

Data bandwidth

Very rare

Battle field

Largest missed promise

Possible but…

Often achieved

Very frequent “gap”

PUBLIC 18

PUBLIC 19

Research

• Efficient Nets (Kernels + NN)

• NXPNET (High detrate) – Fixed/FP

Gen Purp DSP DN

• >1TFlops/unit

• >20TeraFlops

• High Perf/power

• AI SDK

ML ACC – S32

• Fixed Point Net

• Power optimized

• Ease to Use – SDK

• OpenCL/VX

i.MX

• GPU Solutions

• Stereo and surround vision

• Efficient poring –OpenCL/VX

NXP Ecosystem Capabilities

PUBLIC 20

NXP’s approach

04

PUBLIC 20

PUBLIC 21PUBLIC 21

More than a brain on 4 wheels.

The core of safe and secure mobility.

LEVERAGINGmassive eco-system capabilities from

leadership businesses inside and

outside traditional automotive:

processing, security and mobile.

SENSE THINK ACT

ARCHITECTING OUR

FUTURE

21PUBLIC


SENSE THINK ACT

Bo

dy &

Com

fort

Driver

Experie

nce

Con

ne

ctivity

Dri

ve

r

Rep

lace

me

nt

Po

we

rtra

in &

Ve

hic

le

Dyn

am

ics

V2X

Broadcast Radio

Cellular

NFC, WiFi, BT, GPS

Smart Car Access

Radar

Camera

Lidar

Motion

Speed

Ultrasonic

Infotainment

Fusion

Smart Light

Access, Door Ctrl

eCockpit

Amplifiers

Powertrain &

Vehicle DynamicsEngine

Transmission

Brake

Steering

Airbag

Suspension

NXP Leads Domain Based

Vehicle Architectures:

Connectivity

Driver Replacement

Powertrain & Vehicle

Dynamics

Body & Comfort

Driver Experience

COMPLETE

SOLUTIONS

➢ FASTER TTM

➢ FULL SCALABILITY

Gate

way


SENSE THINK ACT

Bo

dy &

Com

fort

Driver

Experie

nce

Con

ne

ctivity

Dri

ve

r

Rep

lace

me

nt

Po

we

rtra

in &

Ve

hic

le

Dyn

am

ics

V2X

Broadcast Radio

Cellular

NFC, WiFi, BT, GPS

Smart Car Access

Radar

Camera

Lidar

Motion

Speed

Ultrasonic

Infotainment

Fusion

Smart Light

Access, Door Ctrl

eCockpit

Amplifiers

Powertrain &


Transmission

Brake

Steering

Airbag

Suspension

NXP Leads Domain Based

Vehicle Architectures:

Connectivity

Driver Replacement

Powertrain & Vehicle

Dynamics

Body & Comfort

Driver Experience

COMPLETE

SOLUTIONS

➢ FASTER TTM


Gate

way

PUBLIC 24PUBLIC 24

BlueBox 2.0

• A prototyping platform for Automated Drive and Sensor Fusion applications

• Covers scaled software support and processing needs

− High performance compute with 16GB DDR4 and 250GB SSD

− ASIL-B compute, automotive interfaces, with vision acceleration

− ASIL-D subsystem, with dedicated interfaces

• Numerous interfaces

− Ethernet 100M/1G/10G, SFP+, 8*100BASE-T1, CAN[-FD], FlexRay, 8*Camera Coax, others

• Ships with near-desktop style embedded Yocto Linux for ease of use

− User compatible to BlueBox 1.0

PUBLIC 25PUBLIC 25

Synchronizing and Processing Real-Time Data Streams

• RTMaps from Intempora• RTMaps stands for “Real-Time

Multisensor Applications”

• A modular and component-based toolkit for

ADAS and highly automated driving

applications

• Capture, record, playback, develop, test,

validate, execute and deploy

• RTMaps Embedded is available for NXP

BlueBox

• RTMaps is distributed by dSPACE


SENSE THINK ACT

Bo

dy &

Com

fort

Driver

Experie

nce

Con

ne

ctivity

Dri

ve

r

Rep

lace

me

nt

Po

we

rtra

in &

Ve

hic

le

Dyn

am

ics

V2X

Broadcast Radio

Cellular

NFC, WiFi, BT, GPS

Smart Car Access

Radar

Camera

Lidar

Motion

Speed

Ultrasonic

Smart Light

Access, Door Ctrl

eCockpit

Amplifiers

Powertrain &


Transmission

Brake

Steering

Airbag

Suspension

Infotainment

Fusion

+

QCOM Complements NXP

• Cellular

• WiFi / BT

• Fusion

• Infotainment

COMPLETE SOLUTIONS

➢ FASTER TTM


Gate

way

PUBLIC 27PUBLIC 27

Final

Conclusions

05

PUBLIC 28

Everybody Talks About It…

It is at the Top of its Hype Now

From largest OEMs to the smallest startup

Everyone is throwing out ideas, resources and claims of

success

Most needed is Level 2+/Level 3 but only Level 4 or above

are debated

PUBLIC 29

AD is Valued and Needs Investments

Semiconductor values double because of it

Large investment from traditional and new

entrants

There will be many self-proclaimed winners and

for sure many, many losers

Still today, no OEM has a L4/5 solution

achieved in a reasonable performance/cost

envelope

The investment is very demanding and needs

to be sustained for a prolonged time

replacing the driver - self driving track days€¦ · replacing the driver. public 2 autonomous...

Documents