smart autonomous machine platforms > iot in motion … · 2018-08-20 · 9 confidential and...

CONFIDENTIAL AND PROPRIETARY

DR. KWOK WU

NXP SEMICONDUCTOR

“ SMART AUTONOMOUS MACHINE PLATFORMS > IOT IN MOTION “

- SMART CONNECTED WORLD

IOT on Wings

IOT on Wheels

Industrial AGV (Automated Guided Vehicles) Service Robots

https://en.wikipedia.org/wiki/File:Pylon.detail.arp.750pix.jpg

CONFIDENTIAL AND PROPRIETARY1

Smart

Transportation

:

(Electrified,

Autonomous,

Connected)

3S- Safe,

Secure,

(Reduce

Accident)

- Sustainable

(Green, Zero

emission,

reduce

pollution,

efficiency –

energy,

congestion)

1

Mobile Device

Integration Cloud connectivity

ADAS (Camera,Radar)

Semi-Autonomous

Managed

Connectivity

Services

Ev, Ev Charging

Car/Ride Sharing

V2X Communication

Telematics

- 5G, LTE-V

(C-V2X)

- Natural Language

- In-Vehicle

Services

- Level 3, 4

Autonomy

- Environmental

Sensing

- Secured

- Advanced

HMI – AR

- Real-time analytic (Predict)

- Connected

- IOT

- Smart Autonomous

Machines

(Cars, Drones,

Rovers, Robots)

Smart Autonomous Machine Platforms - Smart Connected World


SMART CONNECTED WORLD (IOT)

BUSINESS EFFICIENCY ↑

QUALITY OF LIFE ↑


EXPLOSIVE GROWTH

In Connected Devices

World Population

2x

6x

3x

# Connected Devices/

Person

7.2B6.8B

6.5B6.3B

7.6B

2003 2010 2015 20202008

50 Billion connected devices

For 8 Billion people

By 2020

Collaborative IoT – Smart IOT Platform for Smart Business and Smart Life

Business Efficiency ↑

Quality of Life ↑


4


Quality of Life ↑



5

Edge Nodes PAN/LAN

Connectivity

Gateway WAN

Connectivity

MCUs Energy

Sensors Connectivity

Cloud

Application/

Action

BIG

Data

INFRACTURCUTRE FOR THE INTERNET OF THINGS

3G, 4G LTE, LTE-A,

5G, Fibre, PLC etc.

NXP DISTRIBUTED CONNECTED EDGE INTELLIGENCE

(Smart, Secured)

Sensing Embedded Processing CommunicationsCPS


6

Fiber,

PLC

TV White Space,

Sub-Gig, etc.

Cloud

Storage, Analytics

Cellular

Local Gateway

3G, LTE, LTE-A,

4G, 5G, etc.

Edge Nodes – Sensors, Measurements,

Monitoring

Sensing is enable for Smart Things: Sensor Fusion

Signals and DataTera Bytes

/day/person

KnowledgeMega Bytes

/day/person

WISDOM

InformationGiga Bytes

/day/person


NXP DISTRIBUTED CONNECTED EDGE INTELLIGENCE

(Smart, Secured)

TM

External Use 7

SMART CONNECTED WORLD (IOT)EDGE INTELLIGENCE

(CONNECTED INTELLIGENCE: CONTEXT-AWARE)

(AI/ML DATA ANALYSIS + IOT SENSOR FUSION)


Quality of Life ↑


8

ADAS and Intelligent Transportation System (ITS)Sensors & Sensor Fusion

On-Board Unit (OBU)

& Roadside units (RSU)


Isolated Sensor Technology

Partial Solution Limitation

Radar

(include Lidar, IR,

Ultrasound)

Object Detection:

multiple targets

location, distance

and speed

a. No object distinction between

cars, pedestrians, bikes.

b. Limited Range and zone of

coverage

c. Lidar has Limited operation in

bad weather (Radar is better

for bad weather)

Video Camera

(with Image

Recognition SW)

Object Recognition:

Distinguish

between cars and

pedestrians, bikes …

a. Limit to Line of sight (LOS)

target coverage,

b. Limited Detection of distance

and speed of objects

c. Limited operation in bad

weather

V2X 802.11p DSRC

Detect objects

around corners

(non Line of Sights

non-LOS)

Limitation on instantaneous

tracking distance and speed of

objects (can be part of V2X

messages)

ADAS Sensors Comparisons (Pros/Cons of each sensor)

Need Integrated Sensor Fusion of multiple Sensors


••

Sensor

Fusion

Autonomous Machine Platform (IOT Sensor Fusion)

Edge Computing (Sense, Analyze, Act), V2X (V2V, V2I, V2P)


ADAS – Advanced Driver Assistance System


Autonomous Machine Platform (IOT Sensor Fusion)

Edge Computing (Sense, Analyze, Act)Sense

(Context-Aware)Think，(Analyze)

ACT，(Control &

Communicate)

NXP Radar App.

NXP Camera App.Obje

ct

De

tectio

n

NXP Motion

Sensors.Vehic

le

Sta

te

NXP Sensor

Fusion, Edge

Analytic, Cloud

Analytic

Collision Avoidance

Actions

(Stop, Re-route…)

Vehicle Control

message

(V2X- V2V, V2I, V2P)

Cloud Analytic

Local Analytics,

Market-Specific Apps

(Services – target tracking

..)

ROS – Robotic Operating Stack

SLAM - Simultaneous Localization and Mapping

NXP MK5 V2X App.


AI – Artificial Intelligence,

ML – Machine Learning

Context-Aware, Cognitive:

- Sense

- Think

- ACT

- Communicate


Intelligent Transportation System (ITS)

Direct V2V Pedestrian

(VRU)

detection

CLOUD

Back haul to the cloud

Broadband hotspot

DSRC relay

VRU

warning,

If the other

car

has no V2X

DSRC

radio

5

GDS

RC

5

GDS

RC

5

G

DS

RC

DSR

CObstruction

Legacy

vehicle

detection

Pedestrian

(VRU)

detection

RSUCloud

optimize

traffic data to

send to RSU

control traffic

lights

(across

intersections)

RSU accurate

Detection of Cars

and Pedestrians

(include NLOS),

Send queueing

traffic data

to Cloud

Smart Road-Side Unit (RSU)

Reduce Traffic accidents &

Congestions

Integrated Sensor Fusion,Distributed Edge Intelligence (Collaborative IOT)

(Radar + Camera image recognition + V2X (DSRC) wireless communication)

ADAS Onboard Unit (OBU), Smart Roadside Unit (RSU)


V2X extends sensing beyond own vehicle & beyond line of sight (LOS)

V2X IEEE 802.11p DSRC

(Next 5G: LTE-V (C-V2X)

required for Safety-

critical V2X features:

Low Latency, Secure

&

Beyond-line-of-sight (LOS)

Providing additional safety

data earlier than any other

sensor can „see“

Platooning / cooperative driving5m gap @ 80km/h based on 802.11p low latency

Roadworks beyond line-of-sight Emergency vehicle around corner

Motorcycle approaching / „do not pass!“

#NXPFTF

8P

U

B

L

I

C

U

S

E


Autonomous Machines, Cars, Drones (IOT on Wheels, Wings),

Industrial Rovers (AGVs – Automated Guided Vehicles);

Service Robots …..Industry IOT 4.0, Agriculture 4.0, ITS 4.0 ……

IOT on Wings

Industrial AGV (Automated Guided Vehicles) Service Robots

“ Smart Autonomous Machine Platforms > IoT in Motion “

- Smart Connected World


Autonomous Drone Taxi and Delivery Drones, Rovers (AGVs)

17

Delivery Rovers (AGVs)

Automated Guided Vehicles


- Smart Connected World


MARKETS FORSMART CONNECTED WORLD SERVICES

(MONETIZED ON SENSING DATA)


Quality of Life ↑


Smart Autonomous machines Applications for Smart City’s 4 Pillars19

Smart City

Smart Industry4.0

Industrial IOT (IIOT)

Transportation

(Mobility &

Logistics)

Energy

(Resources)

Health

(Agriculture,

Food,

Environment,

water, air, noise)

Safety, Security

(Buildings,

Infratstructures,

Public Services

..Lighting, parking,

Retail..etc)

FactoryAsset monitoring,

Inspection,

Mapping,

Surveillance,

Environment

IOT on WingsIOT on Wheels

Smart Doctor

Assistant

Service

Robots


Quality of Life ↑

IOE (The Internet of EveryThings)

Bridge Inspection

Deep Data Analytic-Predict

Video-what is one wearing?

Audio – e.g. sound types

for gun-shots ….?

Deeper Data Analytic (Predict)

Now: Driver Attention Detection

Next: Attention Mitiigation?


Collaborative Internet of Service (IoS) - Success Criteria (KPI): 4S, 4A

Kwok Wu, [email protected] (1-512-971-5364)

Simple

StandardScalable

Sustainable

Smart

Secure

Aware (Context)

Anticipate(Predict)

Analyze(Take

Action)

Automate (Autonomous)

(Who, When, Where …)

mailto:[email protected]


Thank You!Dr. Kwok Wu

NXP Semiconductor

[email protected], 512-971-5364

RSURSU

RSU

V2X


- Smart Connected World (Edge Intelligence)

mailto:[email protected]


Backup

22


23Intelligent Transportation System (ITS)

ADAS Onboard Unit (OBU), Smart Roadside Unit (RSU)

Integrated Sensor Fusion,Distributed Edge Intelligence (Collaborative IOT)

(Radar + Camera image recognition + V2X (DSRC) wireless communication)

smart autonomous machine platforms > iot in motion … · 2018-08-20 · 9 confidential and...

Documents