energy-proportional image sensing for robert likamwa bodhi priyantha matthai philipose victor bahl...
TRANSCRIPT
ENERGY-PROPORTIONAL IMAGE SENSING
FOR
Robert LiKamWa
Bodhi PriyanthaMatthai
PhiliposeVictor BahlLin Zhong
CONTINUOUS MOBILE VISION
http://roblkw.comhttp://research.microsoft.com
The Devil Wears Prada
CONTINUOUS MOBILE VISION
CONTINUOUS
MOBILE
VISION
2 of 33
CONTINUOUS MOBILE VISION
CONTINUOUS
MOBILE
VISION
Gestures
Object Memory
3 of 33
Face Recognition
Victor
Fine-grained Localization
BATTERY LIFEBATTERY DEATH
Google Glass
2-3 hours
LooxCie
2-3 hours
GoPro Hero
2-3 hours
4 of 33
2000 mWh
10 h200 mW
5 of 33
Image Sensor
Goal < 25 mW
Sensors
~ 5 mW
Processor
~150 mW
Network Stack
~20 mW
6 of 33
Image Sensor
Goal < 25 mW
7 of 33
Image Sensor
Reality > 250 mW
KEY IDEA: ENERGY α QUALITYPo
wer
Frame rate
Pow
er
Resolution
ENERGY PROFILE OF AN IMAGE SENSOR
1 MP, 5 fps250 mW
0.3 MP, 15 fps245 mW
1 MP, 15 fps295 mW
< 25 mWGoal:
Reality: > 230 mW
0.3 MP, 5 fps232 mW
9 of 38
ENERGY-EFFICIENTIMAGE SENSING
Image Sensor Characterization
Energy Reduction Techniques
Energy vs. Vision Performance
10 of 38
IMAGE SENSOR MEASUREMENT
Camera Module
Programmable Clock (I2C)
NI DAQ Device
Power RailResistors
Power
VDD CLK
* Profiled 5 state-of-the-artimage sensors from 2 manufacturers
*
11 of 38
Camera Module
Programmable Clock (I2C)
NI DAQ Device
Power RailResistors
IMAGE SENSOR MEASUREMENT
12 of 38
IMAGE SENSOR WAVEFORMSActive Period
Idle Period
Analog
Digital
PLL
13 of 38
IMAGE SENSOR WAVEFORMSActive Period
Idle Period
14 of 38
IMAGE SENSOR WAVEFORMSActive Period
Idle Period
Pixel Count divided by
Clock Frequency
Frame Timeminus
Active Time
IMAGE SENSOR PIXEL COUNT (N)
Region-of-Interest(Windowing)
Active Active
Pow
er
Time
Pow
er
Time 16 of 38
Scaled Resolution(Pixel Skipping)
IMAGE SENSOR PIXEL COUNT (N)
Active Active
Pow
er
Time
Pow
er
Time
Video (30 FPS) Power vs. Resolution
17 of 38
ActiveFrame
Readout
Active
Pow
er
Time
Active
IMAGE SENSOR FRAME RATE (R)
Video (0.1 MP) Power vs. FPS
ActiveFrame
ReadoutActive
Pow
er
Time
Active ActiveActive Active
1.0s
18 of 38
CHARACTERIZATION CONCLUSION:
NO ENERGY PROPORTIONALITY
Video (0.1 MP) Power vs. FPS
Video (30 FPS) Power vs. Resolution
19 of 38
ENERGY-EFFICIENTIMAGE SENSING
Image Sensor Characterization
Energy Reduction Techniques
Energy vs. Vision Performance
20 of 38
TECHNIQUE #1: AGGRESSIVE STANDBY
Active Active
Active Active
21 of 38
Active Active
Active Active
CAVEAT TO AGGRESSIVE STANDBY
Active Active
Not enough exposure time
This won’t work for long active periods,i.e., high resolution, high frame rate.
22 of 38
TECHNIQUE #2: CLOCK SCALING (f)
One pixel per clock period
23 of 38
TECHNIQUE #2: CLOCK SCALING (f)
Faster clock
Lower Active Time
Higher Active Power
Higher Idle Power
Slower clock
Higher Active Time
Lower Active Power
Lower Idle Power
24 of 38
TECHNIQUE #2: CLOCK SCALING (f)
Active
Active
Low Pixel CountLow Frame Rate
Slowed Clock
Optimal clock frequency depends onPixel Count & Frame Rate
25 of 38
AGGRESSIVE STANDBY + CLOCK OPTIMIZATION
Sped-upClock
Aggressive Standby
ActiveReadout
Active
Readout
Optimal clock frequency depends onPixel Count & Frame Rate
Exposure Time26 of 38
ENERGY α QUALITYFr
am
e r
ate
(FP
S)
Resolution (MP)
30
25
20
15
10
5
021 3 4 5
Fra
me r
ate
(FP
S)
Resolution (MP)
30
25
20
15
10
5
021 3 4 5
Pow
er
(mW
)
350
300
250
200
150
100
50
0
Aggressive Standby &Clock Optimization
Unoptimized
Pow
er
(mW
)
350
300
250
200
150
100
50
0
27 of 38
CPU
Driv
er
28 of 37
29 of 38
CURRENT IMAGE SENSOR DESIGN
Image Processor
Pixel ArrayColumn Output
Gain, ADC
Analog Signal Chain
70-85% Power Consumption
30 of 38
Image Processor
Pixel ArrayColumn Output
High-Speed ADC
Mid-Speed ADC
Low-Speed ADC
Gain
HETEROGENEOUS SENSOR DESIGN
Analog Signal ChainHeterogeneous
31 of 38
ENERGY α QUALITY
Default
Clock select
Standby
HW Fix
Power vs. Framerate (at 0.1 MP)
Power vs. Resolution(at 5 FPS)
Default
Clock select
Standby
HW Fix
32 of 36
ENERGY-EFFICIENTIMAGE SENSING
Image Sensor Characterization
Energy Reduction Techniques
Energy vs. Vision Performance
33 of 38
ENERGY vs. VISION: VISION TASK
IMAGE REGISTRATION
34 of 36
ENERGY vs. VISION: PERFORMANCE Image
Registration Success
Power Reduction with software
assist
Estimated Power Reduction with hardware assist
Full VGA Resolution0.1 MP, 30 FPS 99.9% 51% 84%
Frame Rate Reduction0.1 MP, 3 FPS 95.7% 95% 98%
30% Window0.06 MP, 30 FPS 96.5% 63% 91%
Subsampled by 20.3 MP, 30 FPS 91.8% 71% 94%
185 mW
TypicalAverage Power
10 mW
With aggr. standby& optimal clock
3 mW
With heterogeneousanalog signal chain
Scalable
Computer
Vision
Algorithms
IntegratedSystems
Design
Developer
SupportEnergy
Proportional
Image
Sensing
36 of 36
Continuous Mobile Vision
ENERGY-PROPORTIONAL IMAGE SENSINGFOR
CONTINUOUS MOBILE VISION
http://roblkw.comhttp://research.microsoft.com
Image sensors are not energy-proportional…
Fra
me r
ate
(FP
S)
Resolution (MP)
30
20
10
01 3 5
…but we can make them energy-proportional…
Aggressive Standby
Clock Optimization
Sensor Modifications
… and this is just the beginning.
CMV