![Page 1: SIGGRAPH 2014 Course on Computational Cameras and Displays (part 3)](https://reader033.vdocuments.net/reader033/viewer/2022052623/55983a6e1a28ab28628b4774/html5/thumbnails/1.jpg)
Compressive Display Systems
Gordon Wetzstein
MIT / Stanford University
media.mit.edu/~gordonw
displayblocks.org
![Page 2: SIGGRAPH 2014 Course on Computational Cameras and Displays (part 3)](https://reader033.vdocuments.net/reader033/viewer/2022052623/55983a6e1a28ab28628b4774/html5/thumbnails/2.jpg)
![Page 3: SIGGRAPH 2014 Course on Computational Cameras and Displays (part 3)](https://reader033.vdocuments.net/reader033/viewer/2022052623/55983a6e1a28ab28628b4774/html5/thumbnails/3.jpg)
This slide has a 16:9 media windowEvolution?
1928 2014
![Page 4: SIGGRAPH 2014 Course on Computational Cameras and Displays (part 3)](https://reader033.vdocuments.net/reader033/viewer/2022052623/55983a6e1a28ab28628b4774/html5/thumbnails/4.jpg)
Nature
Image: National Geographic
Evolution!
![Page 5: SIGGRAPH 2014 Course on Computational Cameras and Displays (part 3)](https://reader033.vdocuments.net/reader033/viewer/2022052623/55983a6e1a28ab28628b4774/html5/thumbnails/5.jpg)
uberpixel
![Page 6: SIGGRAPH 2014 Course on Computational Cameras and Displays (part 3)](https://reader033.vdocuments.net/reader033/viewer/2022052623/55983a6e1a28ab28628b4774/html5/thumbnails/6.jpg)
Next-generation Devices
computation optics & electronics human visual
system
interaction
optics (compressive) computationsensing
Computational &
Compressive Displays
Computational
Imaging
![Page 7: SIGGRAPH 2014 Course on Computational Cameras and Displays (part 3)](https://reader033.vdocuments.net/reader033/viewer/2022052623/55983a6e1a28ab28628b4774/html5/thumbnails/7.jpg)
displayblocks.org
![Page 8: SIGGRAPH 2014 Course on Computational Cameras and Displays (part 3)](https://reader033.vdocuments.net/reader033/viewer/2022052623/55983a6e1a28ab28628b4774/html5/thumbnails/8.jpg)
Images: Wikipedia, Shinya Yoshioka
![Page 9: SIGGRAPH 2014 Course on Computational Cameras and Displays (part 3)](https://reader033.vdocuments.net/reader033/viewer/2022052623/55983a6e1a28ab28628b4774/html5/thumbnails/9.jpg)
This slide has a 16:9 media windowNature
Image: Desafio Monteverde and Arenal Volcano Tours Costa Rica
![Page 10: SIGGRAPH 2014 Course on Computational Cameras and Displays (part 3)](https://reader033.vdocuments.net/reader033/viewer/2022052623/55983a6e1a28ab28628b4774/html5/thumbnails/10.jpg)
Three-layer Tensor Display
![Page 11: SIGGRAPH 2014 Course on Computational Cameras and Displays (part 3)](https://reader033.vdocuments.net/reader033/viewer/2022052623/55983a6e1a28ab28628b4774/html5/thumbnails/11.jpg)
This slide has a 16:9 media windowCompressive Light
Field Displays
![Page 12: SIGGRAPH 2014 Course on Computational Cameras and Displays (part 3)](https://reader033.vdocuments.net/reader033/viewer/2022052623/55983a6e1a28ab28628b4774/html5/thumbnails/12.jpg)
This slide has a 16:9 media window
![Page 13: SIGGRAPH 2014 Course on Computational Cameras and Displays (part 3)](https://reader033.vdocuments.net/reader033/viewer/2022052623/55983a6e1a28ab28628b4774/html5/thumbnails/13.jpg)
This slide has a 16:9 media window
viewer moves right
vie
wer
moves d
ow
n
4D Light Field
![Page 14: SIGGRAPH 2014 Course on Computational Cameras and Displays (part 3)](https://reader033.vdocuments.net/reader033/viewer/2022052623/55983a6e1a28ab28628b4774/html5/thumbnails/14.jpg)
Display OpticsComputational
Processing
Compressive Displays
![Page 15: SIGGRAPH 2014 Course on Computational Cameras and Displays (part 3)](https://reader033.vdocuments.net/reader033/viewer/2022052623/55983a6e1a28ab28628b4774/html5/thumbnails/15.jpg)
![Page 16: SIGGRAPH 2014 Course on Computational Cameras and Displays (part 3)](https://reader033.vdocuments.net/reader033/viewer/2022052623/55983a6e1a28ab28628b4774/html5/thumbnails/16.jpg)
2D display
barrier
Parallax Barriers – Ives 1903
• low resolution & very dim
![Page 17: SIGGRAPH 2014 Course on Computational Cameras and Displays (part 3)](https://reader033.vdocuments.net/reader033/viewer/2022052623/55983a6e1a28ab28628b4774/html5/thumbnails/17.jpg)
Integral Imaging – Lippmann 1908
• low-res, but brighter than parallax barriers
2D display
len
sle
ts
![Page 18: SIGGRAPH 2014 Course on Computational Cameras and Displays (part 3)](https://reader033.vdocuments.net/reader033/viewer/2022052623/55983a6e1a28ab28628b4774/html5/thumbnails/18.jpg)
patent drawings - early 20th century
![Page 19: SIGGRAPH 2014 Course on Computational Cameras and Displays (part 3)](https://reader033.vdocuments.net/reader033/viewer/2022052623/55983a6e1a28ab28628b4774/html5/thumbnails/19.jpg)
multiplexing
nonlinear
compression
optimization
user experience
content
community
next
MN
O
C
U C
C
N
Structural Formula for Compressive Displays
![Page 20: SIGGRAPH 2014 Course on Computational Cameras and Displays (part 3)](https://reader033.vdocuments.net/reader033/viewer/2022052623/55983a6e1a28ab28628b4774/html5/thumbnails/20.jpg)
multiplexing
nonlinear
compression
optimization
user experience
content
community
next
MN
O
C
U C
C
N
Nonlinear Pixel Interaction & Coupling
![Page 21: SIGGRAPH 2014 Course on Computational Cameras and Displays (part 3)](https://reader033.vdocuments.net/reader033/viewer/2022052623/55983a6e1a28ab28628b4774/html5/thumbnails/21.jpg)
layer 2
layer 1
Conventional Parallax Barriers
![Page 22: SIGGRAPH 2014 Course on Computational Cameras and Displays (part 3)](https://reader033.vdocuments.net/reader033/viewer/2022052623/55983a6e1a28ab28628b4774/html5/thumbnails/22.jpg)
layer 2
layer 1
Conventional Parallax Barriers
blocked!
p1
p2
l = p1*p2
nonlinear
![Page 23: SIGGRAPH 2014 Course on Computational Cameras and Displays (part 3)](https://reader033.vdocuments.net/reader033/viewer/2022052623/55983a6e1a28ab28628b4774/html5/thumbnails/23.jpg)
layer 2
layer 1
Most Volumetric Displays / Additive Layers
e.g., LEDs or transparent OLEDs
not blocked!
p1
p2
l = p1+p2
linear
![Page 24: SIGGRAPH 2014 Course on Computational Cameras and Displays (part 3)](https://reader033.vdocuments.net/reader033/viewer/2022052623/55983a6e1a28ab28628b4774/html5/thumbnails/24.jpg)
Holography – Nonlinear Interaction
plane wave
hologram
emitted wavefront:
screen or retina
received intensity:
I(x) = T U(x){ }2
= Re T U(x){ }{ }2
+ Im T U(x){ }{ }2
U(x)
![Page 25: SIGGRAPH 2014 Course on Computational Cameras and Displays (part 3)](https://reader033.vdocuments.net/reader033/viewer/2022052623/55983a6e1a28ab28628b4774/html5/thumbnails/25.jpg)
Holography – Coupling
plane wave
hologram
W (x,u = l sin(q )) = t x+x '
2
æ
èç
ö
ø÷t x -
x '
2
æ
èç
ö
ø÷e
2pix 'u dx 'ò
Fourier transform of all points interacting
with each other!
nonlinear interaction & pixel coupling!
![Page 26: SIGGRAPH 2014 Course on Computational Cameras and Displays (part 3)](https://reader033.vdocuments.net/reader033/viewer/2022052623/55983a6e1a28ab28628b4774/html5/thumbnails/26.jpg)
attenuation layers with
spacers
backlight
Layered 3D – SIGGRAPH 2011
![Page 27: SIGGRAPH 2014 Course on Computational Cameras and Displays (part 3)](https://reader033.vdocuments.net/reader033/viewer/2022052623/55983a6e1a28ab28628b4774/html5/thumbnails/27.jpg)
Layered 3D – SIGGRAPH 2011
layer 2
layer K
layer 1
…
![Page 28: SIGGRAPH 2014 Course on Computational Cameras and Displays (part 3)](https://reader033.vdocuments.net/reader033/viewer/2022052623/55983a6e1a28ab28628b4774/html5/thumbnails/28.jpg)
Layered 3D – SIGGRAPH 2011
layer 2
layer K
layer 1
…
p1
p2
pK
l = p1*p2*…*pK
![Page 29: SIGGRAPH 2014 Course on Computational Cameras and Displays (part 3)](https://reader033.vdocuments.net/reader033/viewer/2022052623/55983a6e1a28ab28628b4774/html5/thumbnails/29.jpg)
Layered 3D – SIGGRAPH 2011
…
layer 2
layer K
layer 1
![Page 30: SIGGRAPH 2014 Course on Computational Cameras and Displays (part 3)](https://reader033.vdocuments.net/reader033/viewer/2022052623/55983a6e1a28ab28628b4774/html5/thumbnails/30.jpg)
Tensor Displays – SIGGRAPH 2012
directional backlight three layer
Reconfigurable
![Page 31: SIGGRAPH 2014 Course on Computational Cameras and Displays (part 3)](https://reader033.vdocuments.net/reader033/viewer/2022052623/55983a6e1a28ab28628b4774/html5/thumbnails/31.jpg)
![Page 32: SIGGRAPH 2014 Course on Computational Cameras and Displays (part 3)](https://reader033.vdocuments.net/reader033/viewer/2022052623/55983a6e1a28ab28628b4774/html5/thumbnails/32.jpg)
time
Tensor Displays – Directional Backlight
Perceptual Integration
layer 2
microlens array
layer 1
![Page 33: SIGGRAPH 2014 Course on Computational Cameras and Displays (part 3)](https://reader033.vdocuments.net/reader033/viewer/2022052623/55983a6e1a28ab28628b4774/html5/thumbnails/33.jpg)
LCD with directional backlight, rank 6
LCD + directional BL
conventional lenslets
view from above
![Page 34: SIGGRAPH 2014 Course on Computational Cameras and Displays (part 3)](https://reader033.vdocuments.net/reader033/viewer/2022052623/55983a6e1a28ab28628b4774/html5/thumbnails/34.jpg)
LCD with directional backlight, rank 6 (as seen by observer)
![Page 35: SIGGRAPH 2014 Course on Computational Cameras and Displays (part 3)](https://reader033.vdocuments.net/reader033/viewer/2022052623/55983a6e1a28ab28628b4774/html5/thumbnails/35.jpg)
front LCD
directional backlight
Filmed with High-speed Camera
![Page 36: SIGGRAPH 2014 Course on Computational Cameras and Displays (part 3)](https://reader033.vdocuments.net/reader033/viewer/2022052623/55983a6e1a28ab28628b4774/html5/thumbnails/36.jpg)
four stacked liquid crystal panels
two crossed polarizers
![Page 37: SIGGRAPH 2014 Course on Computational Cameras and Displays (part 3)](https://reader033.vdocuments.net/reader033/viewer/2022052623/55983a6e1a28ab28628b4774/html5/thumbnails/37.jpg)
LCD 1
LCD 2
LCD 3
backlight
Polarization Fields – SIGGRAPH ASIA 2011
polarizer
polarizercolor filter array
polarizercolor filter array
polarizercolor filter array
![Page 38: SIGGRAPH 2014 Course on Computational Cameras and Displays (part 3)](https://reader033.vdocuments.net/reader033/viewer/2022052623/55983a6e1a28ab28628b4774/html5/thumbnails/38.jpg)
backlight
Polarization Fields – SIGGRAPH ASIA 2011
LCD 1
LCD 2
LCD 3
polarizer
polarizer
![Page 39: SIGGRAPH 2014 Course on Computational Cameras and Displays (part 3)](https://reader033.vdocuments.net/reader033/viewer/2022052623/55983a6e1a28ab28628b4774/html5/thumbnails/39.jpg)
image formation
f1
f2
f3
L(x,q) = sin2 Q(x,q)( )
backlight
Polarization Fields – SIGGRAPH ASIA 2011
LCD 1
LCD 2
LCD 3
![Page 40: SIGGRAPH 2014 Course on Computational Cameras and Displays (part 3)](https://reader033.vdocuments.net/reader033/viewer/2022052623/55983a6e1a28ab28628b4774/html5/thumbnails/40.jpg)
multiplexing
nonlinear
compression
optimization
user experience
content
community
next
MN
O
C
U C
C
N
Compression
![Page 41: SIGGRAPH 2014 Course on Computational Cameras and Displays (part 3)](https://reader033.vdocuments.net/reader033/viewer/2022052623/55983a6e1a28ab28628b4774/html5/thumbnails/41.jpg)
Which Light Field is More Compressible?
a b
![Page 42: SIGGRAPH 2014 Course on Computational Cameras and Displays (part 3)](https://reader033.vdocuments.net/reader033/viewer/2022052623/55983a6e1a28ab28628b4774/html5/thumbnails/42.jpg)
Which Light Field is More Compressible?
a b
![Page 43: SIGGRAPH 2014 Course on Computational Cameras and Displays (part 3)](https://reader033.vdocuments.net/reader033/viewer/2022052623/55983a6e1a28ab28628b4774/html5/thumbnails/43.jpg)
This slide has a 16:9 media window
4D Light Field
Uniform or
Directional BacklightStacked Layers
Nonnegative Tensor
Factorization
Display-adaptive
Compression
Compressive
Computed Tomography
(LCDs or Transparencies)
Optics
Observer = Decoder
![Page 44: SIGGRAPH 2014 Course on Computational Cameras and Displays (part 3)](https://reader033.vdocuments.net/reader033/viewer/2022052623/55983a6e1a28ab28628b4774/html5/thumbnails/44.jpg)
Give those Pixels a Break!
Applied Mathematics
• sparse optimization
• low-rank factorization
• computed tomography
• …
Benefits for Optics & Electronics
• fewer pixels
• relaxation on refresh rate
• thinner form factors
• …
![Page 45: SIGGRAPH 2014 Course on Computational Cameras and Displays (part 3)](https://reader033.vdocuments.net/reader033/viewer/2022052623/55983a6e1a28ab28628b4774/html5/thumbnails/45.jpg)
Parallax Barriers
1903
Time-Shifted
Parallax Barriers 2007
t
HR3D
SIG Asia 2010
t tLayered 3D
SIGGRAPH 2011
Tensor Displays
SIGGRAPH 2012
Conventional Parallax Barriers
layer 2
layer 1
From Conventional to Compressive Displays
![Page 46: SIGGRAPH 2014 Course on Computational Cameras and Displays (part 3)](https://reader033.vdocuments.net/reader033/viewer/2022052623/55983a6e1a28ab28628b4774/html5/thumbnails/46.jpg)
…
…
Parallax Barriers
1903
Time-Shifted
Parallax Barriers 2007
t
HR3D
SIG Asia 2010
t tLayered 3D
SIGGRAPH 2011
Tensor Displays
SIGGRAPH 2012
…
time
Perceptual Integration
Tensor Displays – Multilayer & Directional Backlighting
From Conventional to Compressive Displays
![Page 47: SIGGRAPH 2014 Course on Computational Cameras and Displays (part 3)](https://reader033.vdocuments.net/reader033/viewer/2022052623/55983a6e1a28ab28628b4774/html5/thumbnails/47.jpg)
Parallax Barriers
1903
Time-Shifted
Parallax Barriers 2007
t
HR3D
SIG Asia 2010
t tLayered 3D
SIGGRAPH 2011
Tensor Displays
SIGGRAPH 2012
Perceptual Integration
Tensor Displays – Directional Backlighting
time
thin
!
From Conventional to Compressive Displays
![Page 48: SIGGRAPH 2014 Course on Computational Cameras and Displays (part 3)](https://reader033.vdocuments.net/reader033/viewer/2022052623/55983a6e1a28ab28628b4774/html5/thumbnails/48.jpg)
multiplexing
nonlinear
compression
optimization
user experience
content
community
next
MN
O
C
U C
C
N
Real-time Optimization
![Page 49: SIGGRAPH 2014 Course on Computational Cameras and Displays (part 3)](https://reader033.vdocuments.net/reader033/viewer/2022052623/55983a6e1a28ab28628b4774/html5/thumbnails/49.jpg)
This slide has a 16:9 media window
4D Light Field
Display-adaptive
Compression
CompressiveOptics
via optimization
![Page 50: SIGGRAPH 2014 Course on Computational Cameras and Displays (part 3)](https://reader033.vdocuments.net/reader033/viewer/2022052623/55983a6e1a28ab28628b4774/html5/thumbnails/50.jpg)
attenuation layers with
spacers
backlight
Layered 3D – SIGGRAPH 2011
![Page 51: SIGGRAPH 2014 Course on Computational Cameras and Displays (part 3)](https://reader033.vdocuments.net/reader033/viewer/2022052623/55983a6e1a28ab28628b4774/html5/thumbnails/51.jpg)
?
• limited baseline tomography
• use algebraic approaches!
Layered 3D – SIGGRAPH 2011
![Page 52: SIGGRAPH 2014 Course on Computational Cameras and Displays (part 3)](https://reader033.vdocuments.net/reader033/viewer/2022052623/55983a6e1a28ab28628b4774/html5/thumbnails/52.jpg)
This slide has a 16:9 media windowComputed Tomography (CT)
Image: W
ikip
edia
x-ray source
x-ray sensor
3D Reconstruction
Reconstructed 2D Slices
![Page 53: SIGGRAPH 2014 Course on Computational Cameras and Displays (part 3)](https://reader033.vdocuments.net/reader033/viewer/2022052623/55983a6e1a28ab28628b4774/html5/thumbnails/53.jpg)
Tomographic Light Field Synthesis
q
2D light field
x
xq
backlight
attenuation volume
virtual planesimage formation
L(x,q ) = e- m (r )dr
cò
log L x,q( )( ) = - m(r)drc
ò
2
20
P)log(argmin
L
tomographic synthesis
![Page 54: SIGGRAPH 2014 Course on Computational Cameras and Displays (part 3)](https://reader033.vdocuments.net/reader033/viewer/2022052623/55983a6e1a28ab28628b4774/html5/thumbnails/54.jpg)
Limits of Tomographic Light Field Synthesis
image formation
L(x,q ) = e- m (r )dr
cò
log L x,q( )( ) = - m(r)drc
ò
2
20
P)log(argmin
L
tomographic synthesis
log space
…
p1
p2
pK
…
p1
p2
pK
time
l = p1*…*pK
…
p1
p2
pK
log(l) = log(p1)+…+log(pK)
l = (p1*p2*…*pK)1+…+(p1*p2*…*pK)N
log
lin
???
![Page 55: SIGGRAPH 2014 Course on Computational Cameras and Displays (part 3)](https://reader033.vdocuments.net/reader033/viewer/2022052623/55983a6e1a28ab28628b4774/html5/thumbnails/55.jpg)
backlight
rear layer
front layer
two-layer light field display
Low-rank Light Field Factorization
light field
fm(1)(x1)
fm(2)(x2)
x1
x2
L(x1, x2)
![Page 56: SIGGRAPH 2014 Course on Computational Cameras and Displays (part 3)](https://reader033.vdocuments.net/reader033/viewer/2022052623/55983a6e1a28ab28628b4774/html5/thumbnails/56.jpg)
backlight
rear layer
front layer
two-layer light field display
fm(1)(x1)
fm(2)(x2)
x1
x2
L(x1, x2)
`
front layer
rear
laye
r
rank-1
Lanman et al. – SIGGRAPH Asia 2010
Low-rank Light Field Factorization
![Page 57: SIGGRAPH 2014 Course on Computational Cameras and Displays (part 3)](https://reader033.vdocuments.net/reader033/viewer/2022052623/55983a6e1a28ab28628b4774/html5/thumbnails/57.jpg)
backlight
rear layer
front layer
two-layer light field display
fm(1)(x1)
fm(2)(x2)
x1
x2
L(x1, x2)
` rank-4
Lanman et al. – SIGGRAPH Asia 2010
high-speed LCDs = perceptual average
Low-rank Light Field Factorization
![Page 58: SIGGRAPH 2014 Course on Computational Cameras and Displays (part 3)](https://reader033.vdocuments.net/reader033/viewer/2022052623/55983a6e1a28ab28628b4774/html5/thumbnails/58.jpg)
` rank-4F
G
L~
Lanman et al. – SIGGRAPH Asia 2010
Low-rank Light Field Factorization
high-speed LCDs = perceptual average
arg minF,G
L-FGW
2, for F,G ³ 0
objective function:
![Page 59: SIGGRAPH 2014 Course on Computational Cameras and Displays (part 3)](https://reader033.vdocuments.net/reader033/viewer/2022052623/55983a6e1a28ab28628b4774/html5/thumbnails/59.jpg)
fm(1)(x1)
fm(2)(x2)
fm(3)(x3)
x1
x2
x3
L(x2, x3)
multi-layer light field display
backlight
rear layer
middle layer
front layer
light field
Light Field Slice Representation
![Page 60: SIGGRAPH 2014 Course on Computational Cameras and Displays (part 3)](https://reader033.vdocuments.net/reader033/viewer/2022052623/55983a6e1a28ab28628b4774/html5/thumbnails/60.jpg)
fm(1)(x1)
fm(2)(x2)
fm(3)(x3)
x1
x2
x3
L(x1,x2,x3)
Rear
Layer
x3
x2
x1
L(x1,x2,x3)
light field tensor
backlight
rear layer
middle layer
front layer
light field
multi-layer light field display
Light Field Tensor Representation
![Page 61: SIGGRAPH 2014 Course on Computational Cameras and Displays (part 3)](https://reader033.vdocuments.net/reader033/viewer/2022052623/55983a6e1a28ab28628b4774/html5/thumbnails/61.jpg)
fm(1)(x1)
fm(2)(x2)
fm(3)(x3)
x1
x2
x3
Rear
Layer
x3
x2
x1
L(x1,x2,x3)
L(x1,x2,x3)
light field tensor
backlight
rear layer
middle layer
front layer
light field
multi-layer light field display
Light Field Tensor Representation
![Page 62: SIGGRAPH 2014 Course on Computational Cameras and Displays (part 3)](https://reader033.vdocuments.net/reader033/viewer/2022052623/55983a6e1a28ab28628b4774/html5/thumbnails/62.jpg)
fm(1)(x1)
fm(2)(x2)
fm(3)(x3)
Rear
Layer
light field tensor
backlight
rear layer
middle layer
front layer
light field
multi-layer light field display
Light Field Tensor Representation
![Page 63: SIGGRAPH 2014 Course on Computational Cameras and Displays (part 3)](https://reader033.vdocuments.net/reader033/viewer/2022052623/55983a6e1a28ab28628b4774/html5/thumbnails/63.jpg)
fm(1)(x1)
fm(2)(x2)
fm(3)(x3)
Rear
Layer
light field tensor
backlight
rear layer
middle layer
front layer
light field
multi-layer light field display
Light Field Tensor Representation
![Page 64: SIGGRAPH 2014 Course on Computational Cameras and Displays (part 3)](https://reader033.vdocuments.net/reader033/viewer/2022052623/55983a6e1a28ab28628b4774/html5/thumbnails/64.jpg)
fm(1)(x1)
fm(2)(x2)
fm(3)(x3)
Rear
Layer
light field tensor
backlight
rear layer
middle layer
front layer
light field
multi-layer light field display
Light Field Tensor Representation
![Page 65: SIGGRAPH 2014 Course on Computational Cameras and Displays (part 3)](https://reader033.vdocuments.net/reader033/viewer/2022052623/55983a6e1a28ab28628b4774/html5/thumbnails/65.jpg)
frame M
+ ... +
Target Light Field Tensor
frame 1
nonnegative tensorfactorization (NTF)
Rank-M Approximation
perceptualintegration
frame 2
+
Low-rank Tensor Factorization
![Page 66: SIGGRAPH 2014 Course on Computational Cameras and Displays (part 3)](https://reader033.vdocuments.net/reader033/viewer/2022052623/55983a6e1a28ab28628b4774/html5/thumbnails/66.jpg)
iterative update rules
nonlinear (multilinear)
optimization problem
Low-rank Tensor Factorization
• standard form – Tensor Compendium
• multiplicative update keep factors positive
• basically steepest descent with fixed step length
![Page 67: SIGGRAPH 2014 Course on Computational Cameras and Displays (part 3)](https://reader033.vdocuments.net/reader033/viewer/2022052623/55983a6e1a28ab28628b4774/html5/thumbnails/67.jpg)
“standard” formulationalternate formulation
Low-rank Tensor Factorization
![Page 68: SIGGRAPH 2014 Course on Computational Cameras and Displays (part 3)](https://reader033.vdocuments.net/reader033/viewer/2022052623/55983a6e1a28ab28628b4774/html5/thumbnails/68.jpg)
alternate formulation
forward projection (multiview rendering) back projection (projective texture mapping)
Efficient GPU Implementation
Tensor Factorization - Implementation
![Page 69: SIGGRAPH 2014 Course on Computational Cameras and Displays (part 3)](https://reader033.vdocuments.net/reader033/viewer/2022052623/55983a6e1a28ab28628b4774/html5/thumbnails/69.jpg)
![Page 70: SIGGRAPH 2014 Course on Computational Cameras and Displays (part 3)](https://reader033.vdocuments.net/reader033/viewer/2022052623/55983a6e1a28ab28628b4774/html5/thumbnails/70.jpg)
multiplexing
nonlinear
compression
optimization
user experience
content
community
next
MN
O
C
U C
C
N
Who cares?
![Page 71: SIGGRAPH 2014 Course on Computational Cameras and Displays (part 3)](https://reader033.vdocuments.net/reader033/viewer/2022052623/55983a6e1a28ab28628b4774/html5/thumbnails/71.jpg)
Vision-correcting Displayperceived image
displayed image
SIGGRAPH 2014 - Display Session, Tue morning
![Page 72: SIGGRAPH 2014 Course on Computational Cameras and Displays (part 3)](https://reader033.vdocuments.net/reader033/viewer/2022052623/55983a6e1a28ab28628b4774/html5/thumbnails/72.jpg)
Vision-correcting Display
iPod Touch prototype printed transparency
SIGGRAPH 2014 - Display Session, Tue morning
![Page 73: SIGGRAPH 2014 Course on Computational Cameras and Displays (part 3)](https://reader033.vdocuments.net/reader033/viewer/2022052623/55983a6e1a28ab28628b4774/html5/thumbnails/73.jpg)
prototype construction
300 dpi or higher
![Page 74: SIGGRAPH 2014 Course on Computational Cameras and Displays (part 3)](https://reader033.vdocuments.net/reader033/viewer/2022052623/55983a6e1a28ab28628b4774/html5/thumbnails/74.jpg)
vision-correcting displayconventional display
![Page 75: SIGGRAPH 2014 Course on Computational Cameras and Displays (part 3)](https://reader033.vdocuments.net/reader033/viewer/2022052623/55983a6e1a28ab28628b4774/html5/thumbnails/75.jpg)
multiplexing
nonlinear
compression
optimization
user experience
content
community
next
MN
O
C
U C
C
N
What’s next?
![Page 76: SIGGRAPH 2014 Course on Computational Cameras and Displays (part 3)](https://reader033.vdocuments.net/reader033/viewer/2022052623/55983a6e1a28ab28628b4774/html5/thumbnails/76.jpg)
resolution
contrast
3D display capabilities
![Page 77: SIGGRAPH 2014 Course on Computational Cameras and Displays (part 3)](https://reader033.vdocuments.net/reader033/viewer/2022052623/55983a6e1a28ab28628b4774/html5/thumbnails/77.jpg)
Compressive Multi-mode DisplayOptics Express 2014
![Page 78: SIGGRAPH 2014 Course on Computational Cameras and Displays (part 3)](https://reader033.vdocuments.net/reader033/viewer/2022052623/55983a6e1a28ab28628b4774/html5/thumbnails/78.jpg)
This slide has a 16:9 media window
High-speed LCD
High-speed LCD
Electronically-switchable Diffuser
![Page 79: SIGGRAPH 2014 Course on Computational Cameras and Displays (part 3)](https://reader033.vdocuments.net/reader033/viewer/2022052623/55983a6e1a28ab28628b4774/html5/thumbnails/79.jpg)
High-speed LCD
High-speed LCD
Electronically-switchable Diffuser OFF
3D Display Mode
![Page 80: SIGGRAPH 2014 Course on Computational Cameras and Displays (part 3)](https://reader033.vdocuments.net/reader033/viewer/2022052623/55983a6e1a28ab28628b4774/html5/thumbnails/80.jpg)
Light Field Factorization – LCD Patterns
Front Layer Rear Layer
![Page 81: SIGGRAPH 2014 Course on Computational Cameras and Displays (part 3)](https://reader033.vdocuments.net/reader033/viewer/2022052623/55983a6e1a28ab28628b4774/html5/thumbnails/81.jpg)
Light Field Factorization – Results
![Page 82: SIGGRAPH 2014 Course on Computational Cameras and Displays (part 3)](https://reader033.vdocuments.net/reader033/viewer/2022052623/55983a6e1a28ab28628b4774/html5/thumbnails/82.jpg)
High Dynamic Range Display Mode
High-speed LCD
High-speed LCD
Electronically-switchable Diffuser OFF
![Page 83: SIGGRAPH 2014 Course on Computational Cameras and Displays (part 3)](https://reader033.vdocuments.net/reader033/viewer/2022052623/55983a6e1a28ab28628b4774/html5/thumbnails/83.jpg)
Target HDR Image
![Page 84: SIGGRAPH 2014 Course on Computational Cameras and Displays (part 3)](https://reader033.vdocuments.net/reader033/viewer/2022052623/55983a6e1a28ab28628b4774/html5/thumbnails/84.jpg)
Solver = Light Field without Parallax
Front Layer Rear Layer
![Page 85: SIGGRAPH 2014 Course on Computational Cameras and Displays (part 3)](https://reader033.vdocuments.net/reader033/viewer/2022052623/55983a6e1a28ab28628b4774/html5/thumbnails/85.jpg)
ConventionalHigh Dynamic Range
![Page 86: SIGGRAPH 2014 Course on Computational Cameras and Displays (part 3)](https://reader033.vdocuments.net/reader033/viewer/2022052623/55983a6e1a28ab28628b4774/html5/thumbnails/86.jpg)
Superresolution Display Mode
High-speed LCD
High-speed LCD
Electronically-switchable Diffuser ON
![Page 87: SIGGRAPH 2014 Course on Computational Cameras and Displays (part 3)](https://reader033.vdocuments.net/reader033/viewer/2022052623/55983a6e1a28ab28628b4774/html5/thumbnails/87.jpg)
Results from Prototype
![Page 88: SIGGRAPH 2014 Course on Computational Cameras and Displays (part 3)](https://reader033.vdocuments.net/reader033/viewer/2022052623/55983a6e1a28ab28628b4774/html5/thumbnails/88.jpg)
Results from Prototype
![Page 89: SIGGRAPH 2014 Course on Computational Cameras and Displays (part 3)](https://reader033.vdocuments.net/reader033/viewer/2022052623/55983a6e1a28ab28628b4774/html5/thumbnails/89.jpg)
Light Field Projection SIGGRAPH 2014 – ETech & Paper
![Page 90: SIGGRAPH 2014 Course on Computational Cameras and Displays (part 3)](https://reader033.vdocuments.net/reader033/viewer/2022052623/55983a6e1a28ab28628b4774/html5/thumbnails/90.jpg)
Compressive Light Field Projection
![Page 91: SIGGRAPH 2014 Course on Computational Cameras and Displays (part 3)](https://reader033.vdocuments.net/reader033/viewer/2022052623/55983a6e1a28ab28628b4774/html5/thumbnails/91.jpg)
![Page 92: SIGGRAPH 2014 Course on Computational Cameras and Displays (part 3)](https://reader033.vdocuments.net/reader033/viewer/2022052623/55983a6e1a28ab28628b4774/html5/thumbnails/92.jpg)
![Page 93: SIGGRAPH 2014 Course on Computational Cameras and Displays (part 3)](https://reader033.vdocuments.net/reader033/viewer/2022052623/55983a6e1a28ab28628b4774/html5/thumbnails/93.jpg)
multiplexing
nonlinear
compression
optimization
user experience
content
community
next
N
O
C
U C
C
N
Structural Formula for Compressive Displays
M
Mobile Displays Optics Express 2014
Projection Displays SIGGRAPH 2014
Coded Illumination for
Microscopy & LithographyHead Mounted Displays
CGF 2010, SIGGRAPH 2014
Monitors / TVs SIG2011,2012,2013, SIGAsia 2009,2011
![Page 94: SIGGRAPH 2014 Course on Computational Cameras and Displays (part 3)](https://reader033.vdocuments.net/reader033/viewer/2022052623/55983a6e1a28ab28628b4774/html5/thumbnails/94.jpg)
Cool Displays at SIGGRAPH 2014
Technical Papers Sessions
Emerging Technologies
• Displays, Tuesday 10:45-12:15, Hall A
• Computational Sensing and Display, Tue 3:45-5:15, Hall A
• AR & VR Displays
• Light Field Projection, HDR Projection
• much more!
![Page 95: SIGGRAPH 2014 Course on Computational Cameras and Displays (part 3)](https://reader033.vdocuments.net/reader033/viewer/2022052623/55983a6e1a28ab28628b4774/html5/thumbnails/95.jpg)
Gordon Wetzstein
Massachusetts Institute of Technology
media.mit.edu/~gordonw
displayblocks.org
Matt Hirsch (MIT)
Doug Lanman (MIT/NVIDIA/Oculus VR)
Andrew Maimone (UNC)
Felix Heide (UBC)
Fu-Chung Huang (UC Berkeley)
Belen Masia (University of Zaragoza)
collaborators
sources of funding & hardware
Wolfgang Heidrich (UBC/KAUST)
Ramesh Raskar (MIT)
Diego Gutierrez (University of Zaragoza)
Brian Barsky (UC Berkeley)
Henry Fuchs (UNC)
![Page 96: SIGGRAPH 2014 Course on Computational Cameras and Displays (part 3)](https://reader033.vdocuments.net/reader033/viewer/2022052623/55983a6e1a28ab28628b4774/html5/thumbnails/96.jpg)
Gordon Wetzstein
Massachusetts Institute of Technology
media.mit.edu/~gordonw
displayblocks.org
Stanford looking for:
• students, postdocs, interns
• (industry) collaborators