cmos image sensors and prospects for high …...cmos image sensor + george smith ccd pinned...
TRANSCRIPT
![Page 1: CMOS Image Sensors and Prospects for High …...CMOS image sensor + George Smith CCD Pinned photodiode CCD image sensor Buckingham Palace Reception December 2017 Eric Fossum Nobukazu](https://reader034.vdocuments.net/reader034/viewer/2022042802/5f3df407a0cf417e862d5508/html5/thumbnails/1.jpg)
THAYER SCHOOL OF ENGINEERING AT DARTMOUTH
CMOS Image Sensors and Prospects for High-Speed Applications
Eric R. Fossum
September 14, 2018
ULITIMA 2018
Argonne National Laboratory
![Page 2: CMOS Image Sensors and Prospects for High …...CMOS image sensor + George Smith CCD Pinned photodiode CCD image sensor Buckingham Palace Reception December 2017 Eric Fossum Nobukazu](https://reader034.vdocuments.net/reader034/viewer/2022042802/5f3df407a0cf417e862d5508/html5/thumbnails/2.jpg)
THAYER SCHOOL OF ENGINEERING AT DARTMOUTH
Where is Dartmouth College?
Pasadena & Los Angeles,California
Hanover,New Hampshire
![Page 3: CMOS Image Sensors and Prospects for High …...CMOS image sensor + George Smith CCD Pinned photodiode CCD image sensor Buckingham Palace Reception December 2017 Eric Fossum Nobukazu](https://reader034.vdocuments.net/reader034/viewer/2022042802/5f3df407a0cf417e862d5508/html5/thumbnails/3.jpg)
THAYER SCHOOL OF ENGINEERING AT DARTMOUTH
CMOS IMAGE SENSORS
![Page 4: CMOS Image Sensors and Prospects for High …...CMOS image sensor + George Smith CCD Pinned photodiode CCD image sensor Buckingham Palace Reception December 2017 Eric Fossum Nobukazu](https://reader034.vdocuments.net/reader034/viewer/2022042802/5f3df407a0cf417e862d5508/html5/thumbnails/4.jpg)
THAYER SCHOOL OF ENGINEERING AT DARTMOUTH
https://www.nytimes.com/2018/04/27/arts/design/mona-lisa-instagram-art.html
CMOS Image Sensors Enable Billions of Cameras Each Year
![Page 5: CMOS Image Sensors and Prospects for High …...CMOS image sensor + George Smith CCD Pinned photodiode CCD image sensor Buckingham Palace Reception December 2017 Eric Fossum Nobukazu](https://reader034.vdocuments.net/reader034/viewer/2022042802/5f3df407a0cf417e862d5508/html5/thumbnails/5.jpg)
THAYER SCHOOL OF ENGINEERING AT DARTMOUTH
About 5 Billion Cameras Made Each Year(More than 150 per second)
5.5
5.5B units/year => 174.4 cameras/secAt 1 sensor per camera (100% yield)
http://image-sensors-world.blogspot.com/2018/05/cmos-sensor-sales-grow-at-record.html
![Page 6: CMOS Image Sensors and Prospects for High …...CMOS image sensor + George Smith CCD Pinned photodiode CCD image sensor Buckingham Palace Reception December 2017 Eric Fossum Nobukazu](https://reader034.vdocuments.net/reader034/viewer/2022042802/5f3df407a0cf417e862d5508/html5/thumbnails/6.jpg)
THAYER SCHOOL OF ENGINEERING AT DARTMOUTH
![Page 7: CMOS Image Sensors and Prospects for High …...CMOS image sensor + George Smith CCD Pinned photodiode CCD image sensor Buckingham Palace Reception December 2017 Eric Fossum Nobukazu](https://reader034.vdocuments.net/reader034/viewer/2022042802/5f3df407a0cf417e862d5508/html5/thumbnails/7.jpg)
THAYER SCHOOL OF ENGINEERING AT DARTMOUTH
Many Kinds Of Digital Cameras
![Page 8: CMOS Image Sensors and Prospects for High …...CMOS image sensor + George Smith CCD Pinned photodiode CCD image sensor Buckingham Palace Reception December 2017 Eric Fossum Nobukazu](https://reader034.vdocuments.net/reader034/viewer/2022042802/5f3df407a0cf417e862d5508/html5/thumbnails/8.jpg)
THAYER SCHOOL OF ENGINEERING AT DARTMOUTH
MOS “Photomatrices”0th Generation Image Sensor
~June 1966
First self-scanned →Sensor 10x10 1966/67
Peter JW Noble
Mid-late 1960’sMOS arrays at Plesseywith startup Integrated PhotomatrixLtd. (IPL)
And Fairchild with startup ReticonGene Weckler
![Page 9: CMOS Image Sensors and Prospects for High …...CMOS image sensor + George Smith CCD Pinned photodiode CCD image sensor Buckingham Palace Reception December 2017 Eric Fossum Nobukazu](https://reader034.vdocuments.net/reader034/viewer/2022042802/5f3df407a0cf417e862d5508/html5/thumbnails/9.jpg)
THAYER SCHOOL OF ENGINEERING AT DARTMOUTH
V1 V2 V3 V1
Parallel shift registers
Fast s
hift re
gis
ter
Amp
Charge-Coupled Device (CCD)1st Generation Image Sensor
• CCD invented at Bell Labs 1969, then CCD image sensor in 1970.
• Perfected with mass production in Japan.
• Mainstay of digital cameras and camcorders in 1980’s and 1990’s.
V1 V2 V3
PD
![Page 10: CMOS Image Sensors and Prospects for High …...CMOS image sensor + George Smith CCD Pinned photodiode CCD image sensor Buckingham Palace Reception December 2017 Eric Fossum Nobukazu](https://reader034.vdocuments.net/reader034/viewer/2022042802/5f3df407a0cf417e862d5508/html5/thumbnails/10.jpg)
THAYER SCHOOL OF ENGINEERING AT DARTMOUTH
CCD Cameras1970’s - 1990’s
Early 70’s Bell Labs CCD camera by Mike Tompsett et al.
Steve Sasson with first Kodak self-contained digital camera (1975)
RCA CamcorderDALSA industrial CCD
camera late ’80’sSony Camcorder
early 90’s
NASA Galileo Spacecraft CCD camera(with optics) early ’80s (800x800)
![Page 11: CMOS Image Sensors and Prospects for High …...CMOS image sensor + George Smith CCD Pinned photodiode CCD image sensor Buckingham Palace Reception December 2017 Eric Fossum Nobukazu](https://reader034.vdocuments.net/reader034/viewer/2022042802/5f3df407a0cf417e862d5508/html5/thumbnails/11.jpg)
THAYER SCHOOL OF ENGINEERING AT DARTMOUTH
2009 Nobel Prize in Physics
"for the invention of an imaging semiconductor circuit –
the CCD sensor"
CCD image sensor inventor:Michael F. TompsettUS patent no. 4,085,456
![Page 12: CMOS Image Sensors and Prospects for High …...CMOS image sensor + George Smith CCD Pinned photodiode CCD image sensor Buckingham Palace Reception December 2017 Eric Fossum Nobukazu](https://reader034.vdocuments.net/reader034/viewer/2022042802/5f3df407a0cf417e862d5508/html5/thumbnails/12.jpg)
THAYER SCHOOL OF ENGINEERING AT DARTMOUTH
Camera
1990’s Need: Smaller cameras for smaller spacecraft at JPL/Caltech
![Page 13: CMOS Image Sensors and Prospects for High …...CMOS image sensor + George Smith CCD Pinned photodiode CCD image sensor Buckingham Palace Reception December 2017 Eric Fossum Nobukazu](https://reader034.vdocuments.net/reader034/viewer/2022042802/5f3df407a0cf417e862d5508/html5/thumbnails/13.jpg)
THAYER SCHOOL OF ENGINEERING AT DARTMOUTH
Some Problems with CCDs
• Charge must be perfectly transferred thousands of times to get to output amplifier.
• Requires high voltages
• Requires special device structures
• Very susceptible to radiation damage and traps
• Requires power to drive huge whole-chip capacitance
• Requires many support chips
• Difficult to make it work right
• Serial readout gives slow frame rate
• High bandwidth (noisy) output amplifier
![Page 14: CMOS Image Sensors and Prospects for High …...CMOS image sensor + George Smith CCD Pinned photodiode CCD image sensor Buckingham Palace Reception December 2017 Eric Fossum Nobukazu](https://reader034.vdocuments.net/reader034/viewer/2022042802/5f3df407a0cf417e862d5508/html5/thumbnails/14.jpg)
THAYER SCHOOL OF ENGINEERING AT DARTMOUTH
Active Pixels with Intra-Pixel Charge Transfer
One pixel• Complete charge transfer to suppress lag• Correlated double-sampling to suppress kTC noise• Double-delta sampling to suppress fixed pattern noise• On-chip ADC, timing and control, etc.
light
electronsin silicon
amplifier
correlateddoublesampling (CDS)
![Page 15: CMOS Image Sensors and Prospects for High …...CMOS image sensor + George Smith CCD Pinned photodiode CCD image sensor Buckingham Palace Reception December 2017 Eric Fossum Nobukazu](https://reader034.vdocuments.net/reader034/viewer/2022042802/5f3df407a0cf417e862d5508/html5/thumbnails/15.jpg)
THAYER SCHOOL OF ENGINEERING AT DARTMOUTH
“Camera-on-a-Chip”Enables Much Smaller Cameras
Camera-Phone
![Page 16: CMOS Image Sensors and Prospects for High …...CMOS image sensor + George Smith CCD Pinned photodiode CCD image sensor Buckingham Palace Reception December 2017 Eric Fossum Nobukazu](https://reader034.vdocuments.net/reader034/viewer/2022042802/5f3df407a0cf417e862d5508/html5/thumbnails/16.jpg)
THAYER SCHOOL OF ENGINEERING AT DARTMOUTH
Most of the JPL Team circa 1995
Missing: Sabrina Kemeny, Junichi Nakamura, Sunetra Mendis
![Page 17: CMOS Image Sensors and Prospects for High …...CMOS image sensor + George Smith CCD Pinned photodiode CCD image sensor Buckingham Palace Reception December 2017 Eric Fossum Nobukazu](https://reader034.vdocuments.net/reader034/viewer/2022042802/5f3df407a0cf417e862d5508/html5/thumbnails/17.jpg)
THAYER SCHOOL OF ENGINEERING AT DARTMOUTH
Entrenched industry moves slowly in adopting new technologies so in February 1995 we founded Photobit Corporation to commercialize the CMOS image sensor technology ourselves
Technology Transfer
S.Kemeny, N. Doudoumopoulos, E. Fossum, R. Nixon
![Page 18: CMOS Image Sensors and Prospects for High …...CMOS image sensor + George Smith CCD Pinned photodiode CCD image sensor Buckingham Palace Reception December 2017 Eric Fossum Nobukazu](https://reader034.vdocuments.net/reader034/viewer/2022042802/5f3df407a0cf417e862d5508/html5/thumbnails/18.jpg)
THAYER SCHOOL OF ENGINEERING AT DARTMOUTH
The Photobit Corporation Team(early 2000)
![Page 19: CMOS Image Sensors and Prospects for High …...CMOS image sensor + George Smith CCD Pinned photodiode CCD image sensor Buckingham Palace Reception December 2017 Eric Fossum Nobukazu](https://reader034.vdocuments.net/reader034/viewer/2022042802/5f3df407a0cf417e862d5508/html5/thumbnails/19.jpg)
THAYER SCHOOL OF ENGINEERING AT DARTMOUTH
Perspiration Phase
1995-2001 Photobit grows to about 135 persons• Self funded with custom-design contracts from private industry• Important support from SBIR programs (NASA/DoD)• Later, investment from strategic business partners to develop
catalog products• Over 100 new patent applications filed• Nov 2001 Photobit acquired by Micron Technology
![Page 20: CMOS Image Sensors and Prospects for High …...CMOS image sensor + George Smith CCD Pinned photodiode CCD image sensor Buckingham Palace Reception December 2017 Eric Fossum Nobukazu](https://reader034.vdocuments.net/reader034/viewer/2022042802/5f3df407a0cf417e862d5508/html5/thumbnails/20.jpg)
THAYER SCHOOL OF ENGINEERING AT DARTMOUTH
The Technology Developsa Life of its Own
• Today, over 5 billion cameras are manufactured each year that use the CMOS image sensor technology we invented at JPL, or more than 150 cameras per second, 24/365.
• Semiconductor sales of CMOS image sensors are over $13B/yr in 2018. • Thousands of engineers working on this around the globe.• Caltech has successfully enforced its patents against all the major players.• NASA is now just adopting the technology for use in space (e.g. Mars 2020).
16Mpix camera modulesFrom Sony ~2012
Endoscopy Camera From Awaiba ~2012
![Page 21: CMOS Image Sensors and Prospects for High …...CMOS image sensor + George Smith CCD Pinned photodiode CCD image sensor Buckingham Palace Reception December 2017 Eric Fossum Nobukazu](https://reader034.vdocuments.net/reader034/viewer/2022042802/5f3df407a0cf417e862d5508/html5/thumbnails/21.jpg)
THAYER SCHOOL OF ENGINEERING AT DARTMOUTH
~$14B Semiconductor Sales in 2018~5 Billion Cameras in 2018
![Page 22: CMOS Image Sensors and Prospects for High …...CMOS image sensor + George Smith CCD Pinned photodiode CCD image sensor Buckingham Palace Reception December 2017 Eric Fossum Nobukazu](https://reader034.vdocuments.net/reader034/viewer/2022042802/5f3df407a0cf417e862d5508/html5/thumbnails/22.jpg)
THAYER SCHOOL OF ENGINEERING AT DARTMOUTH
2017 Queen Elizabeth Prize for Engineering
CMOSimage sensor
+ George Smith CCD
Pinnedphotodiode
CCDimage sensor
Buckingham Palace ReceptionDecember 2017
EricFossum
NobukazuTeranishi
MikeTompsett
For the creation of digital imaging sensors
![Page 23: CMOS Image Sensors and Prospects for High …...CMOS image sensor + George Smith CCD Pinned photodiode CCD image sensor Buckingham Palace Reception December 2017 Eric Fossum Nobukazu](https://reader034.vdocuments.net/reader034/viewer/2022042802/5f3df407a0cf417e862d5508/html5/thumbnails/23.jpg)
THAYER SCHOOL OF ENGINEERING AT DARTMOUTH
QUANTA IMAGE SENSOR
![Page 24: CMOS Image Sensors and Prospects for High …...CMOS image sensor + George Smith CCD Pinned photodiode CCD image sensor Buckingham Palace Reception December 2017 Eric Fossum Nobukazu](https://reader034.vdocuments.net/reader034/viewer/2022042802/5f3df407a0cf417e862d5508/html5/thumbnails/24.jpg)
THAYER SCHOOL OF ENGINEERING AT DARTMOUTH
Group at Dartmouth
L-R: Song Chen, Saleh Masoodian, Rachel Zizza, Zhaoyang Yin, Donald Hondongwa, Wei Deng, Dakota Starkey, Eric Fossum, Jiaju Ma, Leo Anzagira
24
![Page 25: CMOS Image Sensors and Prospects for High …...CMOS image sensor + George Smith CCD Pinned photodiode CCD image sensor Buckingham Palace Reception December 2017 Eric Fossum Nobukazu](https://reader034.vdocuments.net/reader034/viewer/2022042802/5f3df407a0cf417e862d5508/html5/thumbnails/25.jpg)
THAYER SCHOOL OF ENGINEERING AT DARTMOUTH
QUANTA IMAGE SENSORPhoton-Counting Image Sensor Concept
Cubicle
Image reconstruction
X-Y-t Bit Density ➔ Gray Scale
25
Vision: A billion jots readout at 1000 fps with single photon-counting capability (1Tb/s) and consuming less than a watt.
![Page 26: CMOS Image Sensors and Prospects for High …...CMOS image sensor + George Smith CCD Pinned photodiode CCD image sensor Buckingham Palace Reception December 2017 Eric Fossum Nobukazu](https://reader034.vdocuments.net/reader034/viewer/2022042802/5f3df407a0cf417e862d5508/html5/thumbnails/26.jpg)
THAYER SCHOOL OF ENGINEERING AT DARTMOUTH
Implementation Challenges in 2011
1. How to make a tiny sub-diffraction-limit (SDL) pixel (< 500nm) with deep sub-electron read noise in a mainstream process?
2. How to readout a very large array of binary pixels or jots at 1000 fps with less than 1Watt power?
3. How do you process the jot data to create pixels?
26
![Page 27: CMOS Image Sensors and Prospects for High …...CMOS image sensor + George Smith CCD Pinned photodiode CCD image sensor Buckingham Palace Reception December 2017 Eric Fossum Nobukazu](https://reader034.vdocuments.net/reader034/viewer/2022042802/5f3df407a0cf417e862d5508/html5/thumbnails/27.jpg)
THAYER SCHOOL OF ENGINEERING AT DARTMOUTH
Photoresponse as Bit Density
𝐵𝑖𝑡 𝐷𝑒𝑛𝑠𝑖𝑡𝑦 𝐷 ≜𝑀1
𝑀= 1 − 𝑒−𝐻
QIS Log D – Log H
27
![Page 28: CMOS Image Sensors and Prospects for High …...CMOS image sensor + George Smith CCD Pinned photodiode CCD image sensor Buckingham Palace Reception December 2017 Eric Fossum Nobukazu](https://reader034.vdocuments.net/reader034/viewer/2022042802/5f3df407a0cf417e862d5508/html5/thumbnails/28.jpg)
THAYER SCHOOL OF ENGINEERING AT DARTMOUTH
Issues with Single Photon Avalanche Detectors (SPADs) for QIS Application
SPADs use avalanche multiplication for gain
• High internal electric fields
• Higher operating voltages (15-20V)
• Larger pixels (8-25um)
• High dark count rates (100-1000Hz)
• Dead time
• Low fill factor (low PDE <50%)
• Low manufacturing yield
• Small array sizes (below 0.1M jots)
![Page 29: CMOS Image Sensors and Prospects for High …...CMOS image sensor + George Smith CCD Pinned photodiode CCD image sensor Buckingham Palace Reception December 2017 Eric Fossum Nobukazu](https://reader034.vdocuments.net/reader034/viewer/2022042802/5f3df407a0cf417e862d5508/html5/thumbnails/29.jpg)
THAYER SCHOOL OF ENGINEERING AT DARTMOUTH
Our Approach
Use very low capacitance sense node
DV = DQ / C
1mV = 1.6e-19 / 0.16fF
One pixel
29
light
electronsin silicon
amplifier
correlateddoublesampling (CDS)
![Page 30: CMOS Image Sensors and Prospects for High …...CMOS image sensor + George Smith CCD Pinned photodiode CCD image sensor Buckingham Palace Reception December 2017 Eric Fossum Nobukazu](https://reader034.vdocuments.net/reader034/viewer/2022042802/5f3df407a0cf417e862d5508/html5/thumbnails/30.jpg)
THAYER SCHOOL OF ENGINEERING AT DARTMOUTH
Voltage Output with No Electronics Noise
𝑃 𝑘 =𝑒−𝐻𝐻𝑘
𝑘!, 𝑘 = 0, 1, 2, 3…
H=2Probability mass function =0.27
Probability mass function =0.18
Probability mass function =0.09
Poisson probability mass function
CG = conversion gain = q/C [V/e-]
30
![Page 31: CMOS Image Sensors and Prospects for High …...CMOS image sensor + George Smith CCD Pinned photodiode CCD image sensor Buckingham Palace Reception December 2017 Eric Fossum Nobukazu](https://reader034.vdocuments.net/reader034/viewer/2022042802/5f3df407a0cf417e862d5508/html5/thumbnails/31.jpg)
THAYER SCHOOL OF ENGINEERING AT DARTMOUTH
Broadened by 0.12e- rms read noise
Un = Vn / CG [e- rms]
31
![Page 32: CMOS Image Sensors and Prospects for High …...CMOS image sensor + George Smith CCD Pinned photodiode CCD image sensor Buckingham Palace Reception December 2017 Eric Fossum Nobukazu](https://reader034.vdocuments.net/reader034/viewer/2022042802/5f3df407a0cf417e862d5508/html5/thumbnails/32.jpg)
THAYER SCHOOL OF ENGINEERING AT DARTMOUTH
Broadened by 0.25e- rms read noise
Model
32
![Page 33: CMOS Image Sensors and Prospects for High …...CMOS image sensor + George Smith CCD Pinned photodiode CCD image sensor Buckingham Palace Reception December 2017 Eric Fossum Nobukazu](https://reader034.vdocuments.net/reader034/viewer/2022042802/5f3df407a0cf417e862d5508/html5/thumbnails/33.jpg)
THAYER SCHOOL OF ENGINEERING AT DARTMOUTH
Quantized Values Broadened by Readout Noise
“0” “1”Single-bit QIS
33
![Page 34: CMOS Image Sensors and Prospects for High …...CMOS image sensor + George Smith CCD Pinned photodiode CCD image sensor Buckingham Palace Reception December 2017 Eric Fossum Nobukazu](https://reader034.vdocuments.net/reader034/viewer/2022042802/5f3df407a0cf417e862d5508/html5/thumbnails/34.jpg)
THAYER SCHOOL OF ENGINEERING AT DARTMOUTH
Pump-Gate Jot: Minimize TG-FD Overlap Capacitance
34
Highest possible CG(Lowest possible cap.)
BSI
TGFD
SW
BSI
vertical lateral
![Page 35: CMOS Image Sensors and Prospects for High …...CMOS image sensor + George Smith CCD Pinned photodiode CCD image sensor Buckingham Palace Reception December 2017 Eric Fossum Nobukazu](https://reader034.vdocuments.net/reader034/viewer/2022042802/5f3df407a0cf417e862d5508/html5/thumbnails/35.jpg)
THAYER SCHOOL OF ENGINEERING AT DARTMOUTH
Experimental DataPhoton Counting Histograms
20k reads of same jot, 0.175e- rms read noise ~21DN/e- (61.2uV rms 350uV/e- or 0.45fF)Room temperature, no avalanche, 20 CMS cycles, jot:TPG PTR BC
35
![Page 36: CMOS Image Sensors and Prospects for High …...CMOS image sensor + George Smith CCD Pinned photodiode CCD image sensor Buckingham Palace Reception December 2017 Eric Fossum Nobukazu](https://reader034.vdocuments.net/reader034/viewer/2022042802/5f3df407a0cf417e862d5508/html5/thumbnails/36.jpg)
THAYER SCHOOL OF ENGINEERING AT DARTMOUTH
Experimental DataPhoton Counting Histograms
20k reads of same jot, 0.2e- rms read noise ~21DN/e-Room temperature, no avalanche, 20 CMS cycles, jot:TPG PTR BC
Ma, Masoodian, Wang, Fossum 2017
H=8.25
36
![Page 37: CMOS Image Sensors and Prospects for High …...CMOS image sensor + George Smith CCD Pinned photodiode CCD image sensor Buckingham Palace Reception December 2017 Eric Fossum Nobukazu](https://reader034.vdocuments.net/reader034/viewer/2022042802/5f3df407a0cf417e862d5508/html5/thumbnails/37.jpg)
THAYER SCHOOL OF ENGINEERING AT DARTMOUTH
Read Noise and Photon-Counting Error
37
![Page 38: CMOS Image Sensors and Prospects for High …...CMOS image sensor + George Smith CCD Pinned photodiode CCD image sensor Buckingham Palace Reception December 2017 Eric Fossum Nobukazu](https://reader034.vdocuments.net/reader034/viewer/2022042802/5f3df407a0cf417e862d5508/html5/thumbnails/38.jpg)
THAYER SCHOOL OF ENGINEERING AT DARTMOUTH
Dark Current
Room Temp: ~0.16e-/s avg. (~2pA/cm2)Previously measured ~2x every 10C
Ma, Masoodian, Wang, Fossum 2017
38
Storage well isolated from surface
![Page 39: CMOS Image Sensors and Prospects for High …...CMOS image sensor + George Smith CCD Pinned photodiode CCD image sensor Buckingham Palace Reception December 2017 Eric Fossum Nobukazu](https://reader034.vdocuments.net/reader034/viewer/2022042802/5f3df407a0cf417e862d5508/html5/thumbnails/39.jpg)
THAYER SCHOOL OF ENGINEERING AT DARTMOUTH
Lag
Ma, Anzagira and Fossum IEEE JEDS 4(2) 2016
39
![Page 40: CMOS Image Sensors and Prospects for High …...CMOS image sensor + George Smith CCD Pinned photodiode CCD image sensor Buckingham Palace Reception December 2017 Eric Fossum Nobukazu](https://reader034.vdocuments.net/reader034/viewer/2022042802/5f3df407a0cf417e862d5508/html5/thumbnails/40.jpg)
THAYER SCHOOL OF ENGINEERING AT DARTMOUTH
Quantum Efficiency
40
QE data courtesy of Gigajot
![Page 41: CMOS Image Sensors and Prospects for High …...CMOS image sensor + George Smith CCD Pinned photodiode CCD image sensor Buckingham Palace Reception December 2017 Eric Fossum Nobukazu](https://reader034.vdocuments.net/reader034/viewer/2022042802/5f3df407a0cf417e862d5508/html5/thumbnails/41.jpg)
THAYER SCHOOL OF ENGINEERING AT DARTMOUTH
Stacked BSI CIS Using Wafer Bonding
Sony IMX260 dual pixel AF sensor from Samsung S7 teardown
DetectorLayer
CircuitLayer
WaferBonding
Connection
Sony 2017 ISSCC
41
![Page 42: CMOS Image Sensors and Prospects for High …...CMOS image sensor + George Smith CCD Pinned photodiode CCD image sensor Buckingham Palace Reception December 2017 Eric Fossum Nobukazu](https://reader034.vdocuments.net/reader034/viewer/2022042802/5f3df407a0cf417e862d5508/html5/thumbnails/42.jpg)
THAYER SCHOOL OF ENGINEERING AT DARTMOUTH
3D Stacked Cluster-Parallel Readout to Increase Frame Rate and Reduce Power
42
• Two or more stacked layers
• A group of jots form a cluster
• Readout circuits of a cluster of jots are located underneath cluster
• Clusters function in parallel
• Column line length is reduced, parasitics are reduced
![Page 43: CMOS Image Sensors and Prospects for High …...CMOS image sensor + George Smith CCD Pinned photodiode CCD image sensor Buckingham Palace Reception December 2017 Eric Fossum Nobukazu](https://reader034.vdocuments.net/reader034/viewer/2022042802/5f3df407a0cf417e862d5508/html5/thumbnails/43.jpg)
THAYER SCHOOL OF ENGINEERING AT DARTMOUTH
Prototype 1Mjot 1040fps QIS(1b Digital Output)
43
• Process technology: CMOS BSI 45nm/65nm 2-layer Stacking
• Cluster-Parallel Architecture
• Readout Variation: ➢ Analog➢ Single-bit Digital
• Resolution: 1024x1024
• Jot pitch size: 1.1µm
• Jot types: ➢Tapered-reset Pump-Gate
(TPG)➢Punch-Through Reset (PTR)➢JFET SF
Detector Substrate
1126.4um
1126
.4um
ASIC Substrate
Addre
ssin
g
Hig
h-Spee
d
Dig
ital P
ADs
16x16=256 clusters
4096 jots in each
cluster
16x16=256 readout
clusters
8 CDS units and a
1b-ADC in each
readout cluster
![Page 44: CMOS Image Sensors and Prospects for High …...CMOS image sensor + George Smith CCD Pinned photodiode CCD image sensor Buckingham Palace Reception December 2017 Eric Fossum Nobukazu](https://reader034.vdocuments.net/reader034/viewer/2022042802/5f3df407a0cf417e862d5508/html5/thumbnails/44.jpg)
THAYER SCHOOL OF ENGINEERING AT DARTMOUTH
Digital and Analog Readout Organization
44
DIGITALHigh speed
ANALOGLow speed
![Page 45: CMOS Image Sensors and Prospects for High …...CMOS image sensor + George Smith CCD Pinned photodiode CCD image sensor Buckingham Palace Reception December 2017 Eric Fossum Nobukazu](https://reader034.vdocuments.net/reader034/viewer/2022042802/5f3df407a0cf417e862d5508/html5/thumbnails/45.jpg)
THAYER SCHOOL OF ENGINEERING AT DARTMOUTH
1Mjot Prototype QIS Experimental Results
45
1Mpixel QIS photon-counting binary image sensor operating at 1040fps
Target scene
Purdue denoising
![Page 46: CMOS Image Sensors and Prospects for High …...CMOS image sensor + George Smith CCD Pinned photodiode CCD image sensor Buckingham Palace Reception December 2017 Eric Fossum Nobukazu](https://reader034.vdocuments.net/reader034/viewer/2022042802/5f3df407a0cf417e862d5508/html5/thumbnails/46.jpg)
THAYER SCHOOL OF ENGINEERING AT DARTMOUTH
Summary of Measured Results
46
Equiv. PD Dead Time <0.1%Array 1024 (H) x 1024 (V)
Field rate 1040fpsADC sampling rate 4MSa/s
ADC resolution 1 bit
Output data rate32 (output pins) x
34Mb/s= 1090Mb/s
Package PGA with 224 pins
Power
Array 2.3mW256 ADCs 7.5mW
Addressing 4.1mWI/O pads 3.7mW
Total 17.6mWFOM ADC 6.9pJ/b
Process45nm (jot layer),
65nm (ASIC layer)
VDD
1.8V & 2.5V (Analog, digital and
array), 3V & 2.2V (I/O pads)
Jot typeBSI Tapered Pump
Gate2-Way Shared RO
Jot pitch 1.1µmBSI Fill Factor ~100%
Quantum Efficiency 79% @ 550nmConversion gain on
column345µV/e-
Input Referred Noise 0.22e- r.m.s.Corresponding BER ~1%
Avg. Dark current (RT) 0.16e-/sEquiv. Dark Count Rate
(RT)0.16Hz/jot
𝐹𝑂𝑀 =𝑃𝑜𝑤𝑒𝑟 𝐶𝑜𝑛𝑠𝑢𝑚𝑝𝑡𝑖𝑜𝑛
# 𝑜𝑓 𝑝𝑖𝑥𝑒𝑙𝑠 × 𝑓𝑟𝑎𝑚𝑒 𝑟𝑎𝑡𝑒[𝑝𝐽
𝑏]
![Page 47: CMOS Image Sensors and Prospects for High …...CMOS image sensor + George Smith CCD Pinned photodiode CCD image sensor Buckingham Palace Reception December 2017 Eric Fossum Nobukazu](https://reader034.vdocuments.net/reader034/viewer/2022042802/5f3df407a0cf417e862d5508/html5/thumbnails/47.jpg)
THAYER SCHOOL OF ENGINEERING AT DARTMOUTH
Dr. Eric R. Fossum
Dr. Saleh Masoodian
Dr. Jiaju Ma
Gigajot spinoff (2017)
47
![Page 48: CMOS Image Sensors and Prospects for High …...CMOS image sensor + George Smith CCD Pinned photodiode CCD image sensor Buckingham Palace Reception December 2017 Eric Fossum Nobukazu](https://reader034.vdocuments.net/reader034/viewer/2022042802/5f3df407a0cf417e862d5508/html5/thumbnails/48.jpg)
THAYER SCHOOL OF ENGINEERING AT DARTMOUTH
1 2 3 4 5 6 70
1Mpixel 3b QIS ImageExposure of 0.87e-/pixel average
Raw image and Histogram
2x2x2 cubicle sum only
2x2x2 cubicle denoise
48
![Page 49: CMOS Image Sensors and Prospects for High …...CMOS image sensor + George Smith CCD Pinned photodiode CCD image sensor Buckingham Palace Reception December 2017 Eric Fossum Nobukazu](https://reader034.vdocuments.net/reader034/viewer/2022042802/5f3df407a0cf417e862d5508/html5/thumbnails/49.jpg)
THAYER SCHOOL OF ENGINEERING AT DARTMOUTH
Comments onVERY HIGH SPEED IMAGE SENSORS
49
![Page 50: CMOS Image Sensors and Prospects for High …...CMOS image sensor + George Smith CCD Pinned photodiode CCD image sensor Buckingham Palace Reception December 2017 Eric Fossum Nobukazu](https://reader034.vdocuments.net/reader034/viewer/2022042802/5f3df407a0cf417e862d5508/html5/thumbnails/50.jpg)
THAYER SCHOOL OF ENGINEERING AT DARTMOUTH
What is High Speed?
• Old Target circa 1995
• 1Mpix @ 1Kfps
• Continuous Readout
• 1Gpix/s @10b
• New Target 2019 (?)
• 1Mpix @ 100K+fps
• Continuous Readout
• 100Gpix/s @ 10+b
50
![Page 51: CMOS Image Sensors and Prospects for High …...CMOS image sensor + George Smith CCD Pinned photodiode CCD image sensor Buckingham Palace Reception December 2017 Eric Fossum Nobukazu](https://reader034.vdocuments.net/reader034/viewer/2022042802/5f3df407a0cf417e862d5508/html5/thumbnails/51.jpg)
THAYER SCHOOL OF ENGINEERING AT DARTMOUTH
Issues
• Pixel Size, QE, and Charge Transport• Higher frame rate, fewer photons per frame
• Thicker material, better QE, worse charge transport
• Larger pixel, larger aperture, more photons
• Larger pixel, longer charge transport distance• T ~ L^2 or at best L
• Global Shutter vs. Rolling Shutter
51
![Page 52: CMOS Image Sensors and Prospects for High …...CMOS image sensor + George Smith CCD Pinned photodiode CCD image sensor Buckingham Palace Reception December 2017 Eric Fossum Nobukazu](https://reader034.vdocuments.net/reader034/viewer/2022042802/5f3df407a0cf417e862d5508/html5/thumbnails/52.jpg)
THAYER SCHOOL OF ENGINEERING AT DARTMOUTH
Issues
• On-chip Analog-to-Digital Conversion (ADC)• Chip area per ADC v. pixel pitch
• 3D Stacking for Pixel-Parallel or Cluster-Parallel
• How many bits? 1,2,3….16b
• Conversion cycles – SA if resolution <= 6b
• Power dissipation limits • 1Mpixel @ 1uW/pix = 1W
• Energy/conversion related to ADC resolution
52
![Page 53: CMOS Image Sensors and Prospects for High …...CMOS image sensor + George Smith CCD Pinned photodiode CCD image sensor Buckingham Palace Reception December 2017 Eric Fossum Nobukazu](https://reader034.vdocuments.net/reader034/viewer/2022042802/5f3df407a0cf417e862d5508/html5/thumbnails/53.jpg)
THAYER SCHOOL OF ENGINEERING AT DARTMOUTH
Issues
• On-chip data reduction and off-chip readout• Mostly pertains to continuous mode readout
• Power dissipation is critical, pad count is limited.
• 1Mpixel @ 100Kframe/s = 0.1 Tpixel/sec = 1 Tb/sec for 10b ADC
• For sparse illumination can reduce number of pixels read out.
• Compressive sensing might help but not if data spans full space of values.
• Image data must be received, and stored at same data rate – also a problem.
53
![Page 54: CMOS Image Sensors and Prospects for High …...CMOS image sensor + George Smith CCD Pinned photodiode CCD image sensor Buckingham Palace Reception December 2017 Eric Fossum Nobukazu](https://reader034.vdocuments.net/reader034/viewer/2022042802/5f3df407a0cf417e862d5508/html5/thumbnails/54.jpg)
THAYER SCHOOL OF ENGINEERING AT DARTMOUTH
Issues
• Architecture• Continuous mode
• 1Mpixel @ 100Kframe/s = 0.1Tpixel/s data rate (!)
• Burst mode• 1Mpixel @ 1Gframes/s x m frames on-chip storage (ok)
54
![Page 55: CMOS Image Sensors and Prospects for High …...CMOS image sensor + George Smith CCD Pinned photodiode CCD image sensor Buckingham Palace Reception December 2017 Eric Fossum Nobukazu](https://reader034.vdocuments.net/reader034/viewer/2022042802/5f3df407a0cf417e862d5508/html5/thumbnails/55.jpg)
THAYER SCHOOL OF ENGINEERING AT DARTMOUTH
1996 ISSCC
In-pixel transport
~18um
55
![Page 56: CMOS Image Sensors and Prospects for High …...CMOS image sensor + George Smith CCD Pinned photodiode CCD image sensor Buckingham Palace Reception December 2017 Eric Fossum Nobukazu](https://reader034.vdocuments.net/reader034/viewer/2022042802/5f3df407a0cf417e862d5508/html5/thumbnails/56.jpg)
THAYER SCHOOL OF ENGINEERING AT DARTMOUTH
1996 ISSCC
Pixel data buffer storage Off-chip readout architecture
100um
56
![Page 57: CMOS Image Sensors and Prospects for High …...CMOS image sensor + George Smith CCD Pinned photodiode CCD image sensor Buckingham Palace Reception December 2017 Eric Fossum Nobukazu](https://reader034.vdocuments.net/reader034/viewer/2022042802/5f3df407a0cf417e862d5508/html5/thumbnails/57.jpg)
THAYER SCHOOL OF ENGINEERING AT DARTMOUTH
Burst ModeWork led by Etoh at Kinki Univ
57
![Page 58: CMOS Image Sensors and Prospects for High …...CMOS image sensor + George Smith CCD Pinned photodiode CCD image sensor Buckingham Palace Reception December 2017 Eric Fossum Nobukazu](https://reader034.vdocuments.net/reader034/viewer/2022042802/5f3df407a0cf417e862d5508/html5/thumbnails/58.jpg)
THAYER SCHOOL OF ENGINEERING AT DARTMOUTH
Continuous Readout circa 1998
600Mpix/s
>1000Mpix/s
58
![Page 59: CMOS Image Sensors and Prospects for High …...CMOS image sensor + George Smith CCD Pinned photodiode CCD image sensor Buckingham Palace Reception December 2017 Eric Fossum Nobukazu](https://reader034.vdocuments.net/reader034/viewer/2022042802/5f3df407a0cf417e862d5508/html5/thumbnails/59.jpg)
THAYER SCHOOL OF ENGINEERING AT DARTMOUTH
Continuous Readout circa 2000
820Mpix/s
59
![Page 60: CMOS Image Sensors and Prospects for High …...CMOS image sensor + George Smith CCD Pinned photodiode CCD image sensor Buckingham Palace Reception December 2017 Eric Fossum Nobukazu](https://reader034.vdocuments.net/reader034/viewer/2022042802/5f3df407a0cf417e862d5508/html5/thumbnails/60.jpg)
THAYER SCHOOL OF ENGINEERING AT DARTMOUTH
Luxima 2018
Alex Krymski, Lin Ping Ang
See also, CMOSIS (AMS)
2000Mpix/s
3600Mpix/s
60
![Page 61: CMOS Image Sensors and Prospects for High …...CMOS image sensor + George Smith CCD Pinned photodiode CCD image sensor Buckingham Palace Reception December 2017 Eric Fossum Nobukazu](https://reader034.vdocuments.net/reader034/viewer/2022042802/5f3df407a0cf417e862d5508/html5/thumbnails/61.jpg)
THAYER SCHOOL OF ENGINEERING AT DARTMOUTH
5000Mpix/s
http://www.imagesensors.org/Past%20Workshops/2013%20Workshop/2013%20Papers/11-5_076-cremers.pdf
ON Semiconductor, Belgium
YEAR PWR?
61
![Page 62: CMOS Image Sensors and Prospects for High …...CMOS image sensor + George Smith CCD Pinned photodiode CCD image sensor Buckingham Palace Reception December 2017 Eric Fossum Nobukazu](https://reader034.vdocuments.net/reader034/viewer/2022042802/5f3df407a0cf417e862d5508/html5/thumbnails/62.jpg)
THAYER SCHOOL OF ENGINEERING AT DARTMOUTH
Vision ResearchPhantom Camera
https://www.phantomhighspeed.com/products/cameras/ultrahighspeed/v2512
~26,000Mpix/s 12b according to website
62
![Page 63: CMOS Image Sensors and Prospects for High …...CMOS image sensor + George Smith CCD Pinned photodiode CCD image sensor Buckingham Palace Reception December 2017 Eric Fossum Nobukazu](https://reader034.vdocuments.net/reader034/viewer/2022042802/5f3df407a0cf417e862d5508/html5/thumbnails/63.jpg)
THAYER SCHOOL OF ENGINEERING AT DARTMOUTH
Sony
~1000 Mpix/s, ~750mW, 14b
63
![Page 64: CMOS Image Sensors and Prospects for High …...CMOS image sensor + George Smith CCD Pinned photodiode CCD image sensor Buckingham Palace Reception December 2017 Eric Fossum Nobukazu](https://reader034.vdocuments.net/reader034/viewer/2022042802/5f3df407a0cf417e862d5508/html5/thumbnails/64.jpg)
THAYER SCHOOL OF ENGINEERING AT DARTMOUTH
The END
64
• Dartmouth graduate students• Ma, Masoodian, Starkey, Deng,
Zizza, Anzagira, Hondongwa, Song• Faculty colleagues
• Odame, Liu, Chan• Rambus
• Endsley, Stark, Guidash• TSMC
• Wei, Yamashita, Wang• DARPA DETECT (a little bit)
Acknowledgments for QIS Part