"processors for embedded vision: technology and market trends," a presentation from the...
Post on 16-Aug-2015
20 Views
Preview:
TRANSCRIPT
EVA - September 10, 2014 © The Linley Group 2014 1
Processors for Embedded Vision Technology and Market Trends
Linley Gwennap, Principal Analyst, The Linley Group
Embedded Vision Alliance, Sept 2014
EVA - September 10, 2014 © The Linley Group 2014 2
About Linley Gwennap
• Founder, principal analyst, The Linley Group
• Leading vendor of technical reports on mobile and
communications semiconductors
• Editor-in-chief of Microprocessor Report
• Author of recent articles on ARM, Broadcom,
Cavium, Intel, Marvell, Nvidia, Qualcomm, et al
• Author of “A Guide to Mobile Processors” and
“Mobile Semi Market Share Forecast”
• Former CPU designer at Hewlett-Packard
EVA - September 10, 2014 © The Linley Group 2014 3
Agenda
• Mobile devices
• Automotive
• Cloud servers
• Internet of Things
EVA - September 10, 2014 © The Linley Group 2014 4
Mobile Devices Need Vision Processing
• Billions of people already use smartphones and tablets to
take pictures and get information on their surroundings
• Face recognition can provide security access
to unlock phone (Android)
• Object recognition for shopping services
(Amazon Fire phone)
• Augmented reality to provide information on
surroundings such as in museum, shopping,
or tourism (e.g. AcrossAir, SkyMap)
• Computational photography for photo editing,
panoramic views, etc
“Firefly” on Amazon Fire Phone
EVA - September 10, 2014 © The Linley Group 2014 5
Mobile Device Market—It’s the Big One!
0
500
1000
1500
2000
2500
2013 2014 2015 2016 2017 2018
Mill
ion
s o
f U
nit
s
Tablet
Smartphone
• Mobile devices require small size, low power and cost
• e.g. less than $3 to add a new feature
252M tablets in 2013 rising to 404M in 2018
1.04B phones in 2013 rising to 1.8B in 2018
(Source: The Linley Group)
EVA - September 10, 2014 © The Linley Group 2014 6
A Few Companies Control the Platform
• Qualcomm, MediaTek,
Spreadtrum provide
complete chip sets
• Processor, baseband, RF,
Wi-Fi combo, NFC, power
management, etc.
• These vendors control
75% of all smartphones
• For vision processing to
succeed in smartphones,
it must come from these
vendors
(Source: The Linley Group)
Smartphone
Processor Share,
2014
Qualcomm
MediaTek
Apple
Spreadtrum
Samsung* Other
*Exynos processors only
EVA - September 10, 2014 © The Linley Group 2014 7
Nvidia Tegra Image Processing Architecture
• Tegra K1 uses
two image
processors (ISP)
• Each ISP can
process up to
600Mpixels/s
• ISP handles basic
functions such as
YUV conversion,
AWB, gamma,
enhance edges,
noise reduction
ISP-A
Memory
Back
Camera
HW Accelerators HW Accelerators
CPU CPU
CPU CPU
AP
Memory Interface
Mu
x
GigaThread Engine
Raster Engine
SMX
ROP
L2 Cache
Memory Interface
ROP ROP ROP
Front
Camera
CSI x4
Kepler GPU Main CPUs
ISP-B
HW Accelerators HW Accelerators AP
Memory Interface
CSI x4
CSI x1
EVA - September 10, 2014 © The Linley Group 2014 8
Heterogeneous Image Processing on Tegra
• Two crossbar buses enable sharing data among CPUs,
GPU, and ISPs
• Each unit provides different degrees of flexibility, power
efficiency, and performance
HW-ISP Kernels
GPU
K0
State Bus
K1
S0 …
S1 Sn
…
Kn
VI-
Mu
x
CS
I
Camera
Sensor
Kernels
CPU
K0
K1
…
Kn
LS BP AWB
DM EE CCM
ᵞ
Kernels
CPU
K0
K1
…
Kn
Kernels
GPU
K0
K1
…
Kn
Frame / Image Bus F0 … F1 Fn
Images
State
YUV
EVA - September 10, 2014 © The Linley Group 2014 9
Tegra K1—Built for Vision
• Programmable engines can implement vision algorithms
while ISPs efficiently offload common imaging tasks
• GPU can handle multiple kernels (threads) with high FP
performance
• CPU cluster can handle four threads with general-purpose
programming
• Nvidia has demonstrated object tracking on Tegra
• Objects with as many as 4,096 focus points arranged in
matrices of 64x64 points
• Variable location and spacing
EVA - September 10, 2014 © The Linley Group 2014 10
Tegra K1—Fast and Cheap
• Four Cortex-A15 CPUs at 2.3GHz
or two “Denver” CPUs at 2.5GHz
• Kepler GPU with 192 shaders (365 peak GFLOPS)
• Dual ISPs at 1.2 Gpixels/s total
• 2MB integrated cache memory
• DRAM controller with 17GB/s of peak bandwidth
• Complete SoC connects to camera, display, USB, HDMI,
serial ports, etc
• 15mm x 15mm package with PoP memory
• Operating power of about 2W to 4W
• Sells for ~$20 in high volume
EVA - September 10, 2014 © The Linley Group 2014 11
Qualcomm Targets Smartphones
• Four Krait CPUs at 2.45GHz or
four Cortex-A53 CPUs at ~2.2GHz
• High-performance Adreno GPU
• Dual ISPs at >1.0 Gpixels/s
• Dual Hexagon DSPs at >600MHz
• DRAM controller with 25.6GB/s of peak bandwidth
• Complete SoC connects to camera, display, USB, HDMI,
serial ports, etc
• Integrated LTE baseband
• 14mm x 14mm package with PoP memory
• Sells for $25–$35 in high volume
EVA - September 10, 2014 © The Linley Group 2014 12
Coprocessor Chips Available
• Irish startup Movidius offers Myriad 2
• Chosen as part of Google’s Project Tango
• Myriad 2 combines custom programmable cores with
hardware accelerators
• Two SPARC cores to run RTOS, scheduling
• 12 SHAVE cores at 600MHz total 86 peak GFLOPS
• Rated at 600 Mpixels/s
• Tiny 5mm x 5mm package
• Sells for $5-$10 in high volume
• Less performance than Tegra or Snapdragon
EVA - September 10, 2014 © The Linley Group 2014 13
Coprocessor IP
• Cognivue offers vision accelerator with software
• APEX offers 32+ computational units
• Videantis targets vision with its IP
• Startup Adapteva offers FP accelerator
• Epiphany engine cranks out 71 GFLOPS in just 2mm2 (28nm LP)
• Some GPUs approach this level of FLOPS/mm2
• Ceva licenses industry-leading DSP cores
• Clock speeds greater than 1GHz in 28nm LP
• XC4410 performs 32 MACs/clock plus hardware accelerators
• IP will be successful only if major suppliers license it
EVA - September 10, 2014 © The Linley Group 2014 14
Embedded Vision in Mobile—Summary
• Smartphone makers are desperately seeking innovation
• Samsung Galaxy S5 offers few new features
• Big innovation in iPhone 6 is copying Samsung’s bigger screens
• Vision processor enables cool new capabilities
• High-end mobile processors already include powerful
hardware that can be used for vision processing
• This level of performance will become ubiquitous in 3-5 years
• Need to demonstrate powerful use cases
• Need to integrate into OS for ease of use
• Major platform makers (Apple, Samsung, Qualcomm)
must lead innovation
EVA - September 10, 2014 © The Linley Group 2014 15
Embedding Vision in Automobiles
• Many current vehicles offer advanced
driver assistance (ADAS) features
such as:
• Lane departure warning/keeping
• Automatic parking
• Collision avoidance
• Drowsy driver detection
• 2015 Mercedes C-class offers
autonomous “stop-and-go” driving
• Fully autonomous commercial
vehicles expected by 2020
Lane detection
Google self-driving car
EVA - September 10, 2014 © The Linley Group 2014 16
Automobiles Use More Processors
0
20
40
60
80
100
120
140
2013 2014 2015 2016 2017 2018
Mill
ion
s o
f U
nit
s
Light Vehicles
With Nav Sys
• Cars require small size, high reliability for processors
• ADAS features at high end, moving into mainstream
• May have several processors in vehicle for different functions
About 100M cars and small trucks ship every year
~25% have nav system, rising to 45% in 2018
(Source: The Linley Group)
EVA - September 10, 2014 © The Linley Group 2014 17
Auto Makers Turn to Mobile Processors
• Nvidia has shipped millions of Tegra chips into vehicles
(e.g. Audi, BMW, VW)
• Other mobile vendors starting to compete
• Traditional auto supplier Freescale has repositioned
mobile i.MX processor for automakers
• TI is bringing its DSPs (high FLOPS) to automakers
• Difficult to beat FLOPS per dollar of mobile processors
EVA - September 10, 2014 © The Linley Group 2014 18
Embedding Vision in the Cloud
• Mobile devices have limits in performance, storage
• Some vision tasks can be offloaded to cloud data center
for additional processing
• Amazon Firefly service uses cloud server to recognize
object from library of possibilities
• This partitioning is also used for voice services such as Apple Siri
• Many augmented reality apps work the same way
• Impossible to store all possible scenes on the phone
• High-speed LTE network improves response time
EVA - September 10, 2014 © The Linley Group 2014 19
Servers Have Small Units, Big Dollars
0
2
4
6
8
10
12
14
16
18
20
2013 2014 2015 2016 2017 2018
Mill
ion
s o
f U
nit
s
All Servers
Cloud Servers
• Servers average 1.8 processor chips at $375 ASP each
• Coprocessors can replace server processors at similar price
• But only a small portion of servers implement cloud vision
Server units rise from 10M to 20M in seven years
Cloud rises from 20% to 35% of all servers by 2018
(Source: The Linley Group)
EVA - September 10, 2014 © The Linley Group 2014 20
What’s in Your Cloud?
• Most servers use standard Intel Xeon processors
• Data-center operators prefer this approach because all servers are
interchangeable to meet rapidly evolving market needs
• Easy to program and deploy new services
• But coprocessors can improve operating costs
• Doubling throughput of algorithm cuts number of servers in half
• Microsoft testing coprocessors (for Bing)
• Little deployment as yet
• Too few vision-based services to make it worth developing custom
hardware
EVA - September 10, 2014 © The Linley Group 2014 21
Sorting Through the Options
• Many options for accelerating vision processing
• Discrete GPU chips/cards (e.g. GeForce, Radeon)
• High-performance DSP chips (e.g. TI C62xx)
• Design custom FPGA to accelerate algorithms
• Various levels of difficulty in building, programming
• GPUs can be programmed through APIs (e.g. OpenCL, CUDA)
• DSPs are typically programmed by hand for best performance
• FPGAs typically use a hardware design language (e.g. VHDL)
• Tools emerging to program FPGAs using C-like language
• Unlike true hardware (ASIC), FPGAs can be reprogrammed
• FPGAs are more flexible for a variety of (non-vision) applications
• Not clear what will emerge as winning approach
EVA - September 10, 2014 © The Linley Group 2014 22
What Do “Things” Need to See?
• Internet of Things spans a broad set of applications
• Some of which are not yet defined or even known
• Smart electric meters, parking lots, vending machines, washer/dryer,
thermostat (Nest), security cameras, lighting, door locks…
• Most IoT devices don’t need vision
• Smart parking needs to sense if parking space is occupied
• Other industrial-control applications
• Security cameras could use vision processing
• Automatically detect change in scene
and notify owner via Internet
• Automatically adjust lighting, temperature
if no activity detected
Nest thermostat
EVA - September 10, 2014 © The Linley Group 2014 23
Internet of Things Forecast Remains Vague
0
200
400
600
800
1000
1200
1400
1600
1800
2000
2013 2014 2015 2016 2017 2018 2019 2020
Mill
ion
s o
f U
nit
s
"Things"
Consumer
Commercial
• Commercial systems such as smart parking can cost hundreds
of dollars per unit (space) and still be profitable to install
• Consumer systems (e.g., cameras) must be low cost
Lights, doors, windows, etc
Washer, dryer, other appliances
Smart meters, parking, vending, industrial
(Source: The Linley Group)
EVA - September 10, 2014 © The Linley Group 2014 24
Conclusions
• Mobile is the largest volume market for vision processing
• Vision hardware already integrated into leading hardware platforms
• Near-term opportunity to provide IP or coprocessors to other mobile-
platform suppliers (e.g. Apple, MediaTek, Samsung, Spreadtrum)
• Long-term opportunity to provide software for these platforms
• Some mobile vision processing will be in the cloud
• Most processing will be on standard Xeon platforms
• As volume grows, opportunity to provide hardware acceleration
• Low volumes but high ASP
• Automotive vision processing is growing rapidly
• ADAS requires several powerful processors per car
• Internet of Things is hot, but few opportunity for vision
top related