mipi devcon 2016: mobile user interface aggregation for heterogeneous compute intensive solutions

17
Mobile User Interface Aggregation for Heterogeneous Compute Intensive Solutions Abdullah Raouf Lattice Semiconductor

Upload: mipi-alliance

Post on 13-Jan-2017

48 views

Category:

Mobile


1 download

TRANSCRIPT

Mobile User Interface

Aggregation for Heterogeneous

Compute Intensive Solutions

Abdullah Raouf Lattice Semiconductor

Agenda •  Data Capture

•  Sensors available •  Use case mapping

•  Data Transfer •  MIPI interface options •  Value of using FPGA to transfer data via MIPI interfaces

•  Data Analysis •  Analytics trends •  MHC architectures •  MHC solutions used every day

•  FPGA Architecture & Solutions

2

Always On, Always Aware

3

Capturing Data

Audio Sensors: Voice recognition, phrase detection, location sensing

4

Image Sensors: Gesture, eye tracking, proximity, depth, movement

Powerefficientheterogeneouscompu2ngneededforsensor

s2tchingandreal-2mefeedback

Health Sensors: EKG, EEG, EMG, temp

Other Sensors: Gyro, compass, accelerometer, magnetometer, light

Real tim

e sensor feedback

Always aware and worki

ng

Iris ▪ Fingerprint ▪ Motion ▪ Image ▪ EKG

▪ Gyro ▪ Audio ▪ Accelerometer ▪ EEG ▪

Mag

neto

met

er ▪

Ges

tur

e ▪ EMG ▪

A Variety of Interface Needs Exist

5

analog

DSI

I2SYUVSLVS

I2CSPI

I3C

D-PHY

C-PHY

SLVS-ECHiSPi

CSI-2

PCMsLVDS

PDM

SLIMbusVirtualChannels

RFFE UART

RAW

proprietary

analog PDM SoundWire

SLIMbusI2S

D-PHY YUV RGB SPI C-PHY DSI

CSI-2 sLVDS SLVS I3C

PCM

Bridge/Aggregate Easily with FPGAs

6

SOC/AP

SoundWire iCEFamily

PDM I2S

UART

I2C/SPI I3C

CSI-DSI

SLVS LVDS

121balls36balls 16Balls

80balls

1.4mmx1.4mm6mmx6mm

MIPI Transfer Optimization

7

… Microphones

Speakers

Displays

Cameras

SOC/AP Sensors

UseCase MIPIStandard

Display MIPIDSI

Imagesensor MIPICSI-2

Genericsensoraggrega2on SPI,I2C,UART,GPIO!MIPIRIO,MIPII3C

Audio MIPISoundWire,MIPISLIMbus

RF MIPIRFFE

BaWery MIPIBIF

Contextuallyawarecompu2ngAnalyzingData

Facerecogni2on

Voicerecogni2on

Backgroundremoval

MHC

Gesturerecogni2on

ExamplesofRepeHHveComputaHon

TradiHonalAlgorithmsvsNeuralNetworks

TradiHonalAlgorithms

•  Basedonpre-conceivedevents•  Usedtofindexactorapproximatesol’n•  Domainexper2serequired•  Architecture&dataflowexplicitlydefined•  Func2onallycomprehensible•  ImplementedinSW

NeuralNetwork

•  Basedonexperienceprobabili2es•  UsedtofindpaWernsindata•  Structureanddataflowevolveswithinthe

networkbasedontraining•  CanbeimplementedinSW(GPUs)orHW

(DSPsandFPGAs)

BasicNeuralNetworkEach “neuron” is multiply-accumulate

Mul2ply Add/Accumulate

BasicNeuralNetworkGPUs use serial approach

Mul2ply Add/Accumulate

GPU

WeightMemory

BasicNeuralNetworkFPGAs are parallel using local memory

FPGA

LUTS

DSPEBR

Mul2ply Add/Accumulate

Enabledby:" Moore’sLaw" Voltagescaling

Constrainedby:×  Power×  Complexity

Enabledby:" Moore’sLaw" Voltagescaling

Constrainedby:×  Power×  ParallelSW×  Scalability

Enabledby:" Abundantdataparallelism" Powerefficiency" Repe22vecomputa2on

Constrainedby:×  Programmingtools/models×  Highcostofentry

Programmability

Repe22veComputa2onalQuickness

Enabledby:" Timetomarket" FlexibleI/Os" Accelerators" Parallelprocessing" Powerefficiency" Repe22vecomputa2on

Constrainedby:×  Programmingtools

MobileHeterogeneousCompuHngArchitectures

Micro-ProcessorAdvancement

Single-CoreProcessor

Mul2-CoreProcessor

HeterogeneousProcessor

HETEROGENEOUSCOMPUTING

FPGA

MHC

HOMOGENEOUSCOMPUTING

CNNHardwareEnergyEfficiencyFPGAs win in published results

Complex Functions Always-onsensordataAcous2cbeamforming

FeatureRich>5KLUTs>6xDSPs>1MbRAM

FPGARangeofMobileSoluHons

Simple Functions MIPIRFFEantennatuning,RGBLED,Levelshieing,etc.

Mid Range FunctionsPedometer,FFTs,co-processing,etc.

MidRangeFeatureSet>3KLUTs>2xDSPs

>100KbSRAM

SimpleFeatureSetSub2KLUTs

Sub80KbRAM

1.4x1.4mm 2.08x2.08mm

>21IOs10IOs

2.15x2.55mm

>21IOs

2.5x2.5mm

>21IOs

Machine Vision 4Kvideotransferwith

MIPID-PHY

HighSpeedI/O>5KLUTs>6xDSPs

HardD-PHY

Summary

FPGAsarechangingthewayyouinteractwithyourdevices

•  Energy-efficientparallelprocessingforrepe22venumbercrunchingneededforMHC

•  Embeddedmemoryenablesimplementa2onofalways-onsensordatabuffers

•  Supportsavarietyofbridging,bufferinganddisplayapplica2ons(FlexibleI/Os)

•  AcceleratekeyinnovaHonsinnext-genera2onmobileandindustrialproducts

•  CarpeDatumakaseizethedata•  Datahasnovalueifitisnotcaptured,transferredandanalyzedcorrectly•  UseMIPII/OandLajcepre-processingsolu2onstotrulyseizevalueoutofdata!