nxp dronecode platform for developing low cost small
TRANSCRIPT
Company Public – NXP, the NXP logo, and NXP secure connections for a smarter world are trademarks of NXP
B.V. All other product or service names are the property of their respective owners. © 2018 NXP B.V.
Drone Program Lead,
Systems Innovation Automotive
NXP Semiconductors
Iain Galloway
NXP Dronecode Platform for Developing
Low Cost Small Autonomous Vehicles
Leveraging Automotive, IoT and Other Technologies for Drones and Rovers
Dronecode Program Manager
The Linux Foundation
Ramon Roche
June 2018 | AMF-AUT-T3177
COMPANY PUBLIC 1
Industry Trends: Commercial Application Taking Off and
Industry Becoming Horizontally Integrated
2009 2012 Today 2021
$21B
DIY
Cargo,
industrial &
‘Flying Cars’Commercial
Consumer
Service
Pilot
ODM
Compon
ents
Marketplace / search
App App App
Mission planning / UTM
ODM ODM
Components Comp.
Fleet Fleet Fleet
Vertical Horizontal
Flight stack
COMPANY PUBLIC 2
What do typical “drone” devices
look like?Think “Small Autonomous Vehicles” Instead
COMPANY PUBLIC 3
Octocopter -Flat
Quadcopter
Octocopter - Coax Multicopter
Tricopter Hexacopter
COMPANY PUBLIC 4
AirshipVTOL transitioning Wing
Flying WingPlane VTOL
Pusher prop
Longer flight Duration
COMPANY PUBLIC 5
Submarine
Boat Security Lawnmower or Agricultural?
Delivery rover
COMPANY PUBLIC 6
Sidewalk
Delivery rover
Rover Delivery trials in cities today for food and package delivery
(DoorDash, Dominos, Tesco Grocery store, EAT24)
Fewer regulatory hurdles?
Cooperation with Drones?
Street Delivery
Rover
COMPANY PUBLIC 7
What are typical uses of drones?
COMPANY PUBLIC 8
Hobby, Racing
COMPANY PUBLIC 9
Commercial Photography,
Cinematography &
TV News
COMPANY PUBLIC 10
Police, Fire, and
Rescue (Medical)
• Search and rescue
• Situational awareness
• Safety – egress
• Medical delivery
COMPANY PUBLIC 11
Construction and Mining
Photogrammetry
COMPANY PUBLIC 12
Inspection
COMPANY PUBLIC 13
Inventory Management
• NXP RAIN UHF RFID Tags
• Indoor or outdoor logistics or inventory
management
• Augment existing fixed RAIN
installations
• Find open storage locations in
warehouses
• Outdoor - Inventory City property?
COMPANY PUBLIC 14
Anatomy of a Drone/UAV
COMPANY PUBLIC 15
Base Drone/UAV functions
One motor controller (ESC) per
motor
One flight controller (FMU) with
motion control (Accel/Gyro/Mag) and
at least one radio interface
(MCUs: NXP’s Kinetis, LPC, i.MX RT)
1
COMPANY PUBLIC 16
Advanced Drone/UAV function
Companion computer and communications for
Autonomy, Data Analysis and Vision systems
RADAR, LiDAR, V2X, GPS, UWB
Sense and avoid technology
(Linux/SafetyOS + ROS)
Advanced battery management,
wireless charging, security, power
distribution, payload, actuators…
2
COMPANY PUBLIC 17
Some Architectures
1. Basic hobby drone
2. Advanced hobby drone
3. Industrial small autonomous vehicle• Drone/UAV
• Delivery Rover
• Other (Commercial Lawnmower, agricultural product)
COMPANY PUBLIC 18
Hobby Drone (Basic)
FMU Flight
Management
UnitIMU
Sensors
LightingGimbal:
ESC/encoder/sensors
Battery
Power Management
ESCKinetis V
Series
ESCKinetis V
Series
ESCKinetis V
SeriesESC
RC Radio
GPS
(advanced option)
• Lots of wires
• Lots of point-to-point
connections
COMPANY PUBLIC 19
WIFI/BT
Hobby Drone (Advanced with Vision)
FMU Flight
Management
Unit
UART
SDIO/USB
MIPI
CSI
BLE/Thread
(KW41)
IMU
Sensors
PCIe
Lighting
Gimbal:
ESC/encoder/sensors
(Single chip with CAN)
Aux
IMU
sensors
SSD/M.2
HDD
Storage
(Video)
Companion Computer
Data Analytics
Video
Camera
Sensor(s)
FPV
Camera
Bolometer
(NXP Fab)
Battery
Power Management
Battery Cell Monitoring
ESCKinetis V
Series
ESCKinetis V
Series
ESCKinetis V
Series
ESCKinetis V
Series
RC Radio
GPS
• Companion computer and FMU may be combined
Vision
Subsystem
COMPANY PUBLIC 20
FMU Flight
Management
UnitKinetis K, V or L
WIFI/BT
Industrial-Grade Small AV – Modular with CAN and Ethernet
NFC NCF3320
(522)
I2S
UART
SDIO/USB2x CAN BUS (1 primary 1 aux)
MIPI
CSI
D2X
(radio only)
BLE/Thread
(KW41)
RADAR
S32R
IMU
Sensors
PCIe
Lighting
ADS-B
(Automatic Dependent
Surveillance –
Broadcast
(SDR chip)
Payload/manipulator/
medical devices
Gimbal:
ESC/encoder/sensors
(Single chip with CAN)
Aux IMU
sensors
CA
N-F
D?
USB
CAN
Digital
SERV
O
GPS
Cellular
Radio
SSD/M.2
HDD
Storage
(Video)
Audio Codec
100BaseT1 2-Wire
Automotive ABV
hub/Switch
POE/PODL LIDARSmart optical
flow Camera?
Smart Ethernet
Sensor/CameraSmart Ethernet
Sensor/CameraCamera,
Sensors
Smart
Companion Computer, Data Analytics,
Vision
S32V244 (A53),
i.MX 6Q MPU (Cortex-A9)
Layerscape MPU (Cortex-A53/72)
or
i.MX 8 series MPU (future)
(ARM Cortex-A53 + A72)
Secure
Element
Camera
Sensor(s)
FPV
Camera
Bolometer
(NXP Fab)
USB- C RMII
Mantra
Sub-GHz
Battery
Battery Authentication SecureID
Power Management
Wireless Charging System
RX
Battery cell monitoring
Authentication
Authentication
ESCKinetis V
Series
ESCKinetis V
Series
ESCKinetis V
Series
ESCKinetis V
Series
Authentication
RC Radio
GPS
D2X-IDU
FMU Flight
Management UnitKinetis K, V
or i.MX RTSecure
Element
Ethernet
COMPANY PUBLIC 21
Tomorrow:
Self-Driving
Robots with
End-to-End
Services
SENSE THINK ACT
COMPANY PUBLIC 22
WIFI/BT
Industrial-Grade UAV – Functional Areas
FMU Flight Management
UnitKinetis K, V or L
D2X (radio only)
BLE/Thread(KW41)
Secure Element
RADARS32R
LightingPayload/manipulator
/medical devices
Gimbal:ESC/encoder/sensors(Single chip with CAN)
Aux IMU sensors
CAN Digital SERVO
GPS
Cellular Radio
SSD/M.2 HDD Storage
(Video)
Audio Codec
Companion Computer, Data Analytics,
visionS32V,i.MX
Secure Element
MantraSub-GHz RC Radio
GPS
Connectivity
Vision, Sense and Avoid Powertrain and Vehicle Dynamics, Motor Control
Battery and Power Management
Authentication
Authentication
Authentication Authentication
In Vehicle networking
COMPANY PUBLIC 23
Hands-on Learning in Small VehiclesIntroducing a flexible program for new vehicle concepts
HoverGames DroneI NXPhlite Flight Management unit
I DroneCode PX4 Flight Stack
I Complete starter kit
Modular and flexible development platform
I New technologies - UAVCAN and 100BaseT1 capable
I Suitable for Rovers and other Robotic vehicle systems
I System test bed for adjacent technologies and products
HoverGames
COMPANY PUBLIC 24
WIFI/BT
Industrial Reference UAV
NFC NCF3320
(522)
I2S
FMU Flight
Management
UnitKinetis K, V or L
UART
SDIO/USB
2x CAN (1 primary 1 aux)
MIPI
CSI
D2X
(radio only)
BLE/Thread
(KW41)
Secure
Element
RADAR
S32R
IMU
Sensors
PCIe
Lighting
ADS-B
(Automatic Dependent
Surveillance –
Broadcast
(SDR chip)
Payload/manipulator/
medical devices
Gimbal:
ESC/encoder/sensors
(Single chip with CAN)
Aux IMU
sensors
CA
N-F
D?
USB
CAN
Digital
SERV
O
GPS
Cellular
Radio
SSD/M.2
HDD
Storage
(Video)
Audio Codec
100BaseT1 2-Wire
Automotive ABV
hub/Switch
POE/PODL LIDARSmart optical
flow Camera?
Smart Ethernet
Sensor/CameraSmart Ethernet
Sensor/CameraSmart Ethernet
Sensor/Camera
Companion Computer, Data Analytics,
Vision
S32V244 (A53),
i.MX 6Q MPU (Cortex-A9)
Layerscape MPU (Cortex-A53/72)
or
i.MX 8 series MPU (future)
(ARM Cortex-A53 + A72)
Secure
Element
Camera
Sensor(s)
FPV
Camera
Bolometer
(NXP Fab)
USB- C RMII
Mantra
Sub-GHz
Battery
Battery Authentication SecureID
Power Management
Wireless Charging System
RX
Battery cell monitoring
Authentication
Authentication
ESCKinetis V
Series
ESCKinetis V
Series
ESCKinetis V
Series
ESCKinetis V
Series
Authentication
RC Radio
GPS
D2X-IDU
COMPANY PUBLIC 25
Hovergames Platform – Starting Point
NFC
FMU Flight
Management
UnitKinetis K, V or L
UART
2x CAN (1 primary 1 aux)
D2X
BLE/Thread
(KW41)
IMU
Sensors
GPS
ESCKinetis V
Series
ESCKinetis V
Series
ESCKinetis V
Series
ESCKinetis V
Series
RC Radio
2wire Ethernet
COMPANY PUBLIC 26
NXPhliteV3RC15 FMU ComponentsVehicle Management Unit with Automotive-Grade Components
Onboard components for
• Legacy RC PWM
• USB debug terminal for
QGroundControl config.
• Automotive IMU
components
(accel/gyro/mag)
• Barometric pressure
• Dual CAN bus
• 2 wire 100BaseT1
Ethernet
Kinetis K66, Arm M4
2M Flash @ 180Mhz
FXAS21002CQ
Gyroscope
FXOS8700CQ
Accelerometer Compass
MPL3115A2 Barometer
Pressure Sensor
RC PWM
TJA1100 2-Wire Ethernet
USB
DUAL TJA1042 CAN Transceiver
TJA1100 2-wire Ethernet Phy
COMPANY PUBLIC 27
NXPhliteV3RC15 FMU ComponentsVehicle Management Unit with Automotive grade components
Onboard components for
• Legacy RC PWM
• USB debug terminal for
QGroundControl config.
• Automotive IMU
components
(accel/gyro/mag)
• Barometric pressure
• Dual CAN bus
• 2 wire 100BaseT1
Ethernet
Kinetis K66, Arm M4
2M Flash @ 180Mhz
FXAS21002CQ
Gyroscope
FXOS8700CQ
Accelerometer Compass
MPL3115A2 Barometer
Pressure Sensor
RC PWM
TJA1100 2-Wire Ethernet
USB
DUAL TJA1042 CAN Transceiver
TJA1100 2-wire Ethernet Phy
COMPANY PUBLIC 28
NXPhliteV3RC15 FMU ComponentsVehicle Management Unit with Automotive grade components
CAN0
USONIC
CAN1
PPM input
Arming
Switch
ADC6V
2 Wire ETH
POWER
GPS
UART
Telemetry
DCD-LZ
Debug
SBUS
NFC/I2C
G
A
E
P
Ports for external
• NTAG or other NFC
• Secure Authentication
• External radios such 802.11p D2X and
Kinetis KW41 BLE/Thread mesh
• USB/I2C/SPI/UARTS
• Ultrasonic transducer
• IrDA Infrared
• Arming switch+Buzzer
ADC3V
Buzzer
IRDA
SPI
28
COMPANY PUBLIC 29
HoverGames DroneSmaller than an autonomous car!
Reference Design
• Complete low cost ‘hobby’ drone
platform, but really an open design
robot.
• 500mm size big enough for easy
experimentation
• Complete system to test new
components (e.g. FOC ESC with
UAVCAN or secure battery)
• Reuse of components for Rover
29
COMPANY PUBLIC 30
COMPANY PUBLIC 31
Authentication
Secure authentication of components
Can you trust the drone if you can’t trust the
components?
• GPS spoofing
• Unapproved battery packs
• Modified components
• UL/FCC/TUV Certification
• Brand protection
Can certifying agencies protect their certifications?
Can you insure it without knowing it’s components?
COMPANY PUBLIC 32
HoverGames DroneMobile IoT
Integration with
Rapid IOT Platform
• DroneCode UAVCAN
connectors
• Onboard KW42 Radio -
IPV6, 6lowpan, Thread
radio. Bluetooth,
802.15.4 Zigbee
• A71CH Secure Element
• Onboard sensors
• 250+ Pluggable “Click”
sensor/actuator modules
32
COMPANY PUBLIC 3333
COMPANY PUBLIC 34
THE OPEN SOURCE UAV PLATFORM
A neutral place where industry and
community developers can work together
to build the world’s leading open UAV
software platform
COMPANY PUBLIC 35
Leading products and platforms
based on the Dronecode Platform
35 35
COMPANY PUBLIC 36
The Dronecode Platform
36
▪ 651 Contributors
▪ 6,140 Closed issues
▪ 10,094 Closed pull requests
Key platform stats (feb 18):
▪ 43,240 commits
▪ 118 Releases
CommunicationVehicle Ground Control Station
SDKMiddlewareAutonomy
Vision based
localization and
avoidance
Qualcomm
Snapdragon
Flight
Intel® Aero
Hardware
36
COMPANY PUBLIC 37
Proprietary
differentiation:
- Capabilities
- Customization
- Features
- Certification
- UX
How do companies use the platform?
2008 Today
Pla
tform
COMPANY PUBLIC 38
PX4 Pro Autopilot
PX4 is an open-source, open-hardware platform
aiming to become the safest and most reliable flight
stack to the commercial, industrial, and academic
markets.
• The core of a complete end-to-end UAV platform
• One code base for all vehicle types
• IP-friendly license (BSD License)
• World Class Devs
• The best from academia, industry, and agile
startups
• An active community supporting users and
developers across the flight stack.
38
COMPANY PUBLIC 39
QGroundControl: The opensource Ground Control Station
• State of the art software runs on Android, iOS, Windows, Linux and Mac OS.
• User friendly for flying and mission planning
• Survey, mapping support
• Digital video streaming support
• Enables firmware customization
• Simple airframe setup, including firmware, joysticks, sensors and flight modes.
39 39
COMPANY PUBLIC 40
QGroundControl
flight planner and
configuration tool
COMPANY PUBLIC 41
Demo in Techlab
DroneCode.org QGroundControl
optimized by Qt Company to run on a
Technexion TEP-15 industrial panel computer.
− NXP i.MX6 Quad processor
− Low power, rugged, fan-less, cost effective reference solution
− QGroundControl is an intuitive and powerful Ground Control Station for PX4 and ArduPilot UAVs
− Technexion TEP-15 industrial panel computer running Ubuntu or Yocto Linux
− The Qt Company optimized HMI & app
Technexion offers both SBCs SOMs and Panel computers using NXP i.MX family processors
Qt Company provides optimized solutions and consulting services for Qt framework
Linux Foundation is the Dronecode Open source Community supporting QGC, MAVLINK, and PX4
Auterion offers commercial go to market support for vehicles using Dronecode software stack
COMPANY PUBLIC 42
What’s Next
COMPANY PUBLIC 43
Maximum quality of components is
mandatory for high-value system
Looks at
intentional hazardsUnpredictable and irregular
Component Reliability
& Robustness
Looks at
unintentional hazardsPredictable and regular
Automotive Functional Safety and Security Is Critical
COMPANY PUBLIC 44
Industry Demands a Common MPU Architecture PlatformMaximizes hardware and software reuse across product and applications
Safety Radar Gateway Vision
Braking
Island
Arm Cortex
Interconnect
Memory
Radar
Island
Gateway
Island
Vision
Island
General
Purpose
Island
General Purpose Vision/ Autonomous
Fusion
Island
DebugSecurity
Safety
Common Architecture
Peripherals
Common Chassis
Arm Cortex
Interconnect
Memory
Common Chassis
Arm Cortex
Interconnect
Memory
Common Chassis
Arm Cortex
Interconnect
Memory
Common Chassis
Arm Cortex
Interconnect
Memory
Common Chassis
Arm Cortex
Interconnect
Memory
Common Chassis
One Common Architecture BaseFixed set of Cores and Interconnect
Consistent Safety Concept & Security
Identical Base Peripheral set
Consistent Boot and Debug
COMPANY PUBLIC 46
Complimented with Automotive-grade MCUs
Common Architecture
One Common Architecture BaseLIN, CAN and Ethernet end nodes
Sensors, interfaces and actuators
COMPANY PUBLIC 47
Medium Range
RADARBlind Side Detection
Short range/
Medium range
RADARPark Assist
Cross-Traffic Alert
Junction Assist
Long Range
RADARAdaptive Cruise Control
Automatic Emergency Braking
Forward Collision Warning
TOMORROW
Higher
Resolution
77GHz CMOS
RadarEvolves to 360° view with high performance integration
COMPANY PUBLIC 48
V2X IEEE 802.11p extends sensing
beyond your vehicle and line of sight (LOS)
Platooning / cooperative driving5m gap @ 80km/h based on 802.11p low latency
Roadworks beyond line-of-sight Emergency vehicle around corner
Motorcycle approaching / „do not pass!“
Low Latency, Secure
and
Beyond-line-of-sight
(LOS)
Providing additional safety
data earlier than any other
sensor can „see“
COMPANY PUBLIC 49
D2X CommunicationIEEE 802.11p
• Drone suitability performance evaluation
• Reliable secure medium range (2Km+?) digital communications for
• Command and control
• Digital Video
• Mesh networking, Drone to Drone
▪ Integration with DSRC/ITS-G5?
COMPANY PUBLIC 50
Next steps in the Drone program
• We are working with partners to enable:
• i.MX RT consumer market fit FMU
• S32K Industrial FMU
• Modular components− BLDC ESC motor controller (FOC,UAVCAN)
− Functional safety capable smart BMS
− Functional safety capable Power module (FS6500/4500)
− UAVCAN sensors and actuator
COMPANY PUBLIC 51
Tools and
Resources
• Local NXP Sales office and Distributors
• FTF-Connects and Tech Days
• Website: http://www.nxp.com/UAV
• HoverGames: www.nxp.com/hovergames
• Video: https://youtu.be/Sifn1p6fhVg
• Contact: [email protected] or
We want to hear from you!
If interested in NXP Drone reference design, send an email
and we will keep you informed.
Many additional topic areas and parts, lets talk!
NXP, the NXP logo, and NXP secure connections for a smarter world are trademarks of NXP B.V. All other product or service names are the property of their respective owners. © 2018 NXP B.V.
www.nxp.com