embedded aiarmtechforum.com.cn/attached/article/sh-a6_bj-a3...thunderworld ai market size 4 source:...
TRANSCRIPT
Embedded AIChallenges and Practices
Edge Computing & Embedded AI Enabler
1
ThunderWorld
AI Trend and Public Media Awareness
3
Source: CB Insights
Google Trends: AI VS Big Data
ThunderWorld
AI Market Size
4
Source: Tractica
2017, the AI revenue will be about $1.4 billion
0
10
20
30
40
2016 2017 2018 2019 2020 2021 2022 2023 2024 2025
($ B
illio
n)
AI Revenue, World Markets: 2016~2025
2025, the AI revenue will be about $36.8 billion
ThunderWorld
AI As a Service
5
AssS: AI as a Service
SW HWAI
Service Carrier
AI
Source: CB Insights
• Cover multi-scenes, make AI happen
• Close to end users, easy to get data
• Closed AI service experience loop
• Horizontal Machine Learning APIs(Speech, Vision, NLP…)
• Vertical Machine Learning APIs(Healthcare, Industrial,
Financial, Agriculture…)
ThunderWorld
Vertical Collaboration – Embedded (+ Edge) + Cloud
6
Source: Open AI Lab
Embedded AI
• AI Inference Downwarding: Running AI on IoT Devices
• IoT Trend: Connection, Intelligent, Self-Control
• Benefits: Faster; Low Consumption; Secure
Cloud AI
• AI Training on Cloud: Data Center, GPUs, TPUs…
• Flexible DDNNs: Distributed Deep Neuron Networks
• Benefits: Powerful Computing & Big Data
Running on
Cloud
Based on
Big Data
Cluster Intelligence
Fast
Running
on
Device
Secure & Intelligent
ThunderWorld
Why Embedded AI ?
7
About 5GB data will be generated every second by a Boeing
787, but the bandwidth …
1GB data will be generated by the autonomous car every second
and it requires real time processing…
Smart Scene Detection in Smart Phones Camera requires on device
AI in image pipeline processing in order to have good UX
Home appliances are trying to add vision algorithms (like gesture
control or human behavior analytics with Camera Sensors), while
consumer may worry about privacy.
ThunderWorld
Challenges of Embedded AI
8
630 GFlops 10 TFlops
Snapdrag
on 835
Tesla P100
Limited computing power Power constraint Thermal constraint
ThunderWorld
Challenges of Embedded AI
9
• Balance of adding new dedicated neural processing units <with cost up>
and algorithm accuracy on existing computing architecture <DSP, GPU>.
• IoT with even lower computing power, like MCU, needs to upgrade chipset
or adding SoC to benefit some advanced AI algorithms.
• Delta UX = New UX - Old UX, need to justify the cost up
• Delta Productivity = New Productivity – Old Productivity, needs to justify
the cost
ThunderWorld
AI Development Kit
10
Powerful ChipsetRich Interfaces
AI
Depth Camera
• Qualcomm® Snapdragon™ 820
• Dual Camera with Sync
• TOF Support (Optional)
• WiFi
• USB
• GPIO
• Algorithm
• AI Applications
ThunderWorld
Fragmented Running Environment
11
• SoC specific optimization,
porting, benchmarking, etc.
• Heterogeneous Optimization
Embedded Device
FastCV Compute Lib* NPE**
Algorithms
NEON OpenCL HVX
** Qualcomm® Snapdragon™
Neural Processing Engine
* ARM® Computer Vision and
Machine Learning Library
ThunderWorld
A Case Study
12
Segment Your Food
Recognize The Food
• Up to 700 diverse dishes
• Up to 80% Top-1 accuracy
• Up to 6 FPS with only model optimization
• 5x performance boost using Deep Learning Engine
• Depth camera
• Relative precise bounding contours
• Calorie counting
Segment The Food
ThunderWorld
How it works
13
Dataset
Training with Framework
Framework
Model
SNPE
Convert
DLC Model
Test
Image
SNPE Recognize the
building
Training Phase
(on PC)
Deploy Phase
(on Device)
ThunderWorld
Acceleration by GPU / DSP
14
Inference Speed (ms) on Inception V3
iPhone 7
DSP
GPU
CPU
ThunderWorld
Acceleration by GPU / DSP
15
DeviceFloat-point
RepresentationNeed Quantization
CPU 32bit Float NO
GPU 16bit Float NO
DSP 8bit Fixed-Point YES
ThunderWorld
Architecture on Android
16
ThunderWorld
Squeeze the hardware capabilities
17
Target
Chipset
Estimated
Performance
Benchmark
Requested FPS
Complexity
Upper Bound
Network Trim based on
Constraints
Actual
Performance
Performance
meet REQ?
Benchmark
No
Train and Test
Network
Yes
Accuracy
Iteration
Performance iteration / adjust upper bound
Accuracy
meet REQ?
DONE
Yes
No
ThunderWorld
From Computer Vision to Intelligent Vision
18
ThunderWorld
Events
19
Company Profile
• Positioning:
World Leading Intelligent Devices Platform Technology Provider
• We Provide:
OS / Middleware / AI Technology and Service
• Target Market:
Smart Phone, Smart IoT, Smart Vehicle
20
Footprint
BeijingHeadquarters
Shenyang
Dalian
TokyoSeoul
Taipei
Hong Kong
Shenzhen
Chengdu
Chongqing
Xian
Wuhan
Shanghai
Nanjing
Silicon Valley
Helsinki
San Diego
18 Offices
• 15 R&D & Support Centers
• 10 in China
• And in USA, Japan, Korea, Finland, India
• 9 Sales Offices
Global Coverage
• China, USA, Japan, Korea, Europe, India
Hyderabad
21
Value Proposition
OS / Middleware /
AI Technology
Smart Phone
Chipset OS & AI Platform
Smart VehicleSmart IoT
22
Products & Technologies
Core Technologies
Products and Solutions
• TurboX Series SoM Products
• Customized Android / Linux OS
• Smart Algorithm
• Cloud Platform
• Customization
• APP & UI/UE development
• Global Carrier Certification
• Automatic Test Solution
• Secure Phone Solution
• Linux & Android based IVI
• Cockpit
• KANZI® KANZI® Connect®
• Test Automation Tool
• Sensor-Fusion InfoADAS
Smart Phone
Smart Vehicle
Smart IoT
• Small RAM
• Fast Boot
• Power Saving
• System Tailoring
System
Optimization
• System Container
• Secure APPs
• Secure Call
• Device Management
Security
• Camera Tuning
• Camera Turnkey Solution
• Multi Camera
• Embedded AI
• Algorithms
Intelligent Vision
• UI Engine
• AR/VR Middleware
• Multi-screen
• Multi-display
Graphics /
User Experience
Services
OS Customization
Carrier Certification
Customer Technical Support
Component Verification and
Driver Development
BSP&APP Maintenance
Android OS Upgrade
23
Experts in EDGE Computing and Embedded
AI
2008 2009 2014 2017
Trial
Smartphone Boom
Automotive & IoT
Edge Computing & Embedded AI
24
Copyright 2008-2017 Thunder Software Co., Ltd.
Company Confidential