see through walls with...
TRANSCRIPT
![Page 1: See Through Walls with Wi-Figraphics.cs.cmu.edu/courses/15-463/2019_fall/lectures/lecture23.pdftransient camera single-pixel l out l in VD VS • What assumption are we making here?](https://reader035.vdocuments.net/reader035/viewer/2022070916/5fb6afdb4ee1681fbc611544/html5/thumbnails/1.jpg)
Non-line-of-sight imaging
15-463, 15-663, 15-862Computational Photography
Fall 2018, Lecture 23http://graphics.cs.cmu.edu/courses/15-463
![Page 2: See Through Walls with Wi-Figraphics.cs.cmu.edu/courses/15-463/2019_fall/lectures/lecture23.pdftransient camera single-pixel l out l in VD VS • What assumption are we making here?](https://reader035.vdocuments.net/reader035/viewer/2022070916/5fb6afdb4ee1681fbc611544/html5/thumbnails/2.jpg)
• Homework 6 posted, due November 30th.- Three-week homework.- Do not leave for last minute, you won’t have time to complete it.- Homework 7 will be appropriately shorter so that it can be done in the last week.
• Doodle for scheduling final project presentations was posted on Piazza.- https://doodle.com/poll/zf4ur49692m772eg- Please make sure to vote!
• Project checkpoint meetings next week.- I will post a spreadsheet so that you can sign up for a meeting slot.
Course announcements
![Page 3: See Through Walls with Wi-Figraphics.cs.cmu.edu/courses/15-463/2019_fall/lectures/lecture23.pdftransient camera single-pixel l out l in VD VS • What assumption are we making here?](https://reader035.vdocuments.net/reader035/viewer/2022070916/5fb6afdb4ee1681fbc611544/html5/thumbnails/3.jpg)
Overview of today’s lecture
• The non-line-of-sight (NLOS) imaging problem.
• Active NLOS imaging using time-of-flight imaging.
• Active NLOS imaging using WiFi.
• Passive NLOS imaging using accidental pinholes.
• Passive NLOS imaging using accidental reflectors.
• Passive NLOS imaging using corners.
![Page 4: See Through Walls with Wi-Figraphics.cs.cmu.edu/courses/15-463/2019_fall/lectures/lecture23.pdftransient camera single-pixel l out l in VD VS • What assumption are we making here?](https://reader035.vdocuments.net/reader035/viewer/2022070916/5fb6afdb4ee1681fbc611544/html5/thumbnails/4.jpg)
Slide credits
Many of these slides were directly adapted from:
• Matthew O’Toole (CMU).• Shree Nayar (Columbia).• Fadel Adib (MIT).• Katie Bouman (MIT).• Adithya Pediredla (Rice).
![Page 5: See Through Walls with Wi-Figraphics.cs.cmu.edu/courses/15-463/2019_fall/lectures/lecture23.pdftransient camera single-pixel l out l in VD VS • What assumption are we making here?](https://reader035.vdocuments.net/reader035/viewer/2022070916/5fb6afdb4ee1681fbc611544/html5/thumbnails/5.jpg)
The non-line-of-sight (NLOS) imaging problem
![Page 6: See Through Walls with Wi-Figraphics.cs.cmu.edu/courses/15-463/2019_fall/lectures/lecture23.pdftransient camera single-pixel l out l in VD VS • What assumption are we making here?](https://reader035.vdocuments.net/reader035/viewer/2022070916/5fb6afdb4ee1681fbc611544/html5/thumbnails/6.jpg)
Time-of-flight (ToF) imaging
![Page 7: See Through Walls with Wi-Figraphics.cs.cmu.edu/courses/15-463/2019_fall/lectures/lecture23.pdftransient camera single-pixel l out l in VD VS • What assumption are we making here?](https://reader035.vdocuments.net/reader035/viewer/2022070916/5fb6afdb4ee1681fbc611544/html5/thumbnails/7.jpg)
Looking around the corner
![Page 8: See Through Walls with Wi-Figraphics.cs.cmu.edu/courses/15-463/2019_fall/lectures/lecture23.pdftransient camera single-pixel l out l in VD VS • What assumption are we making here?](https://reader035.vdocuments.net/reader035/viewer/2022070916/5fb6afdb4ee1681fbc611544/html5/thumbnails/8.jpg)
Looking around the corner
![Page 9: See Through Walls with Wi-Figraphics.cs.cmu.edu/courses/15-463/2019_fall/lectures/lecture23.pdftransient camera single-pixel l out l in VD VS • What assumption are we making here?](https://reader035.vdocuments.net/reader035/viewer/2022070916/5fb6afdb4ee1681fbc611544/html5/thumbnails/9.jpg)
Active NLOS imaging using time-of-flight imaging
![Page 10: See Through Walls with Wi-Figraphics.cs.cmu.edu/courses/15-463/2019_fall/lectures/lecture23.pdftransient camera single-pixel l out l in VD VS • What assumption are we making here?](https://reader035.vdocuments.net/reader035/viewer/2022070916/5fb6afdb4ee1681fbc611544/html5/thumbnails/10.jpg)
Processing a single-pixel transient
light
source
transient camera
single-pixel
time
intensity
VSVD
![Page 11: See Through Walls with Wi-Figraphics.cs.cmu.edu/courses/15-463/2019_fall/lectures/lecture23.pdftransient camera single-pixel l out l in VD VS • What assumption are we making here?](https://reader035.vdocuments.net/reader035/viewer/2022070916/5fb6afdb4ee1681fbc611544/html5/thumbnails/11.jpg)
Processing a single-pixel transient
light
source
transient camera
single-pixel
time
intensity
VSVD
t1 = 10 ps
![Page 12: See Through Walls with Wi-Figraphics.cs.cmu.edu/courses/15-463/2019_fall/lectures/lecture23.pdftransient camera single-pixel l out l in VD VS • What assumption are we making here?](https://reader035.vdocuments.net/reader035/viewer/2022070916/5fb6afdb4ee1681fbc611544/html5/thumbnails/12.jpg)
Processing a single-pixel transient
light
source
transient camera
single-pixel
time
intensity
t1 = 10 ps
lout
lin
lnlos = t1 ∙ c - lin - lout
Where in the world can we find
points creating this pathlength?
VSVD
![Page 13: See Through Walls with Wi-Figraphics.cs.cmu.edu/courses/15-463/2019_fall/lectures/lecture23.pdftransient camera single-pixel l out l in VD VS • What assumption are we making here?](https://reader035.vdocuments.net/reader035/viewer/2022070916/5fb6afdb4ee1681fbc611544/html5/thumbnails/13.jpg)
Processing a single-pixel transient
light
source
transient camera
single-pixel
time
intensity
t1 = 10 ps
lout
lin
lnlos = t1 ∙ c - lin - lout
Ellipse with foci on the virtual source
and detector and pathlength lnlos
VSVD
![Page 14: See Through Walls with Wi-Figraphics.cs.cmu.edu/courses/15-463/2019_fall/lectures/lecture23.pdftransient camera single-pixel l out l in VD VS • What assumption are we making here?](https://reader035.vdocuments.net/reader035/viewer/2022070916/5fb6afdb4ee1681fbc611544/html5/thumbnails/14.jpg)
Processing a single-pixel transient
light
source
transient camera
single-pixel
time
intensity
t2 = 20 ps
lout
lin
lnlos = t2 ∙ c - lin - lout
Ellipse with foci on the virtual source
and detector and pathlength lnlos
VSVD
![Page 15: See Through Walls with Wi-Figraphics.cs.cmu.edu/courses/15-463/2019_fall/lectures/lecture23.pdftransient camera single-pixel l out l in VD VS • What assumption are we making here?](https://reader035.vdocuments.net/reader035/viewer/2022070916/5fb6afdb4ee1681fbc611544/html5/thumbnails/15.jpg)
Processing a single-pixel transient
light
source
transient camera
single-pixel
time
intensity
t2 = 20 ps
lout
lin
VSVD
• What assumption are we making here?
• Is there any other information we are not using?
![Page 16: See Through Walls with Wi-Figraphics.cs.cmu.edu/courses/15-463/2019_fall/lectures/lecture23.pdftransient camera single-pixel l out l in VD VS • What assumption are we making here?](https://reader035.vdocuments.net/reader035/viewer/2022070916/5fb6afdb4ee1681fbc611544/html5/thumbnails/16.jpg)
Processing another single-pixel transient
light
source
transient camera
single-pixel
lout
lin
VSVD
• What assumption are we making here?
• Is there any other information we are not using?
Photons only interact
with the NLOS scene
once (3-bounce paths)
time
intensity
t2 = 20 ps
![Page 17: See Through Walls with Wi-Figraphics.cs.cmu.edu/courses/15-463/2019_fall/lectures/lecture23.pdftransient camera single-pixel l out l in VD VS • What assumption are we making here?](https://reader035.vdocuments.net/reader035/viewer/2022070916/5fb6afdb4ee1681fbc611544/html5/thumbnails/17.jpg)
Processing another single-pixel transient
light
source
transient camera
single-pixel
lout
lin
VSVD
• What assumption are we making here?
• Is there any other information we are not using?
Photons only interact
with the NLOS scene
once (3-bounce paths)
We need to also
account for intensity
time
intensity
t2 = 20 ps
![Page 18: See Through Walls with Wi-Figraphics.cs.cmu.edu/courses/15-463/2019_fall/lectures/lecture23.pdftransient camera single-pixel l out l in VD VS • What assumption are we making here?](https://reader035.vdocuments.net/reader035/viewer/2022070916/5fb6afdb4ee1681fbc611544/html5/thumbnails/18.jpg)
Elliptic backprojection
light
source
transient camera
single-pixel
lout
lin
VSVD
![Page 19: See Through Walls with Wi-Figraphics.cs.cmu.edu/courses/15-463/2019_fall/lectures/lecture23.pdftransient camera single-pixel l out l in VD VS • What assumption are we making here?](https://reader035.vdocuments.net/reader035/viewer/2022070916/5fb6afdb4ee1681fbc611544/html5/thumbnails/19.jpg)
Elliptic backprojection
![Page 20: See Through Walls with Wi-Figraphics.cs.cmu.edu/courses/15-463/2019_fall/lectures/lecture23.pdftransient camera single-pixel l out l in VD VS • What assumption are we making here?](https://reader035.vdocuments.net/reader035/viewer/2022070916/5fb6afdb4ee1681fbc611544/html5/thumbnails/20.jpg)
Elliptic backprojection
![Page 21: See Through Walls with Wi-Figraphics.cs.cmu.edu/courses/15-463/2019_fall/lectures/lecture23.pdftransient camera single-pixel l out l in VD VS • What assumption are we making here?](https://reader035.vdocuments.net/reader035/viewer/2022070916/5fb6afdb4ee1681fbc611544/html5/thumbnails/21.jpg)
Problem
matrix is extremely large in practice, and hard to store and invert directly!
3D measurements
Our approach
express image formation model as a 3D convolution, by:
1. confocalizing measurements2. performing a change of variables
(set , )
light cone transform
(confocal NLOS image formation model)(reparametrized confocal NLOS image formation model)
hidden 3D vol.NLOS system
3D measurements hidden 3D volumegeometric termradiometric term
![Page 22: See Through Walls with Wi-Figraphics.cs.cmu.edu/courses/15-463/2019_fall/lectures/lecture23.pdftransient camera single-pixel l out l in VD VS • What assumption are we making here?](https://reader035.vdocuments.net/reader035/viewer/2022070916/5fb6afdb4ee1681fbc611544/html5/thumbnails/22.jpg)
Step 2: 3D convolution
*
inverse filter
measurements
Step 1: resampleand attenuatealong -axis
convolution kernel recovered volume
Step 3: resampleand attenuatealong -axis
Alg. Complexity:
compute: memory:
Current Implementation:
volume: 32 (x) x 32 (y) x 512 (t)reconstruction rate: ~60 Hzscan rate: ~2 Hz
Assumptions:
confocal scanisotropic or retro. reflectorplanar wall
light cone transform
![Page 23: See Through Walls with Wi-Figraphics.cs.cmu.edu/courses/15-463/2019_fall/lectures/lecture23.pdftransient camera single-pixel l out l in VD VS • What assumption are we making here?](https://reader035.vdocuments.net/reader035/viewer/2022070916/5fb6afdb4ee1681fbc611544/html5/thumbnails/23.jpg)
concave spherical shell(Lambertian BRDF)
Fermat paths: reconstruction based on geometric optics
reconstructions shown in pink and yellow
points at object boundary
points forming specular paths
transienttime
ph
oto
ns
discontinuity from boundary
path
discontinuity from specular
path
100cm
100cm100cm
setup
SPAD + laser
NLOS
We can uniquely reconstruct the NLOS surface point and normal as
xNLOS = xLOS – td(xLOS) ⋅ ∇td(xLOS)
![Page 24: See Through Walls with Wi-Figraphics.cs.cmu.edu/courses/15-463/2019_fall/lectures/lecture23.pdftransient camera single-pixel l out l in VD VS • What assumption are we making here?](https://reader035.vdocuments.net/reader035/viewer/2022070916/5fb6afdb4ee1681fbc611544/html5/thumbnails/24.jpg)
diffuser
custom time-domain OCT
system
hidden object2 cm
wall
confocal measurements
hidden object
5 cm x 5 cm, 1920 x 1080 spots
1 cm
custom time-domain OCT system
wall
confocal measurements
hidden object
80 cm x 80 cm, 64 x 64 spots
80 cm
SPAD and picosecond laser
shape around the corner
with picosecond
setup
shape around the corner with
femtosecond setup
shape through diffuser with femtosecond
setup
Surface reconstruction examples
![Page 25: See Through Walls with Wi-Figraphics.cs.cmu.edu/courses/15-463/2019_fall/lectures/lecture23.pdftransient camera single-pixel l out l in VD VS • What assumption are we making here?](https://reader035.vdocuments.net/reader035/viewer/2022070916/5fb6afdb4ee1681fbc611544/html5/thumbnails/25.jpg)
Reconstructing Hidden Rooms
Visible wall
Invisible walls
![Page 26: See Through Walls with Wi-Figraphics.cs.cmu.edu/courses/15-463/2019_fall/lectures/lecture23.pdftransient camera single-pixel l out l in VD VS • What assumption are we making here?](https://reader035.vdocuments.net/reader035/viewer/2022070916/5fb6afdb4ee1681fbc611544/html5/thumbnails/26.jpg)
Reconstructing Hidden Rooms: One plane at a time
Visible wall
Invisible walls
![Page 27: See Through Walls with Wi-Figraphics.cs.cmu.edu/courses/15-463/2019_fall/lectures/lecture23.pdftransient camera single-pixel l out l in VD VS • What assumption are we making here?](https://reader035.vdocuments.net/reader035/viewer/2022070916/5fb6afdb4ee1681fbc611544/html5/thumbnails/27.jpg)
Reconstructing Hidden Rooms: One plane at a time
Visible wall
Invisible walls
![Page 28: See Through Walls with Wi-Figraphics.cs.cmu.edu/courses/15-463/2019_fall/lectures/lecture23.pdftransient camera single-pixel l out l in VD VS • What assumption are we making here?](https://reader035.vdocuments.net/reader035/viewer/2022070916/5fb6afdb4ee1681fbc611544/html5/thumbnails/28.jpg)
Reconstructing Hidden Rooms: One plane at a time
Visible wall
Invisible walls
![Page 29: See Through Walls with Wi-Figraphics.cs.cmu.edu/courses/15-463/2019_fall/lectures/lecture23.pdftransient camera single-pixel l out l in VD VS • What assumption are we making here?](https://reader035.vdocuments.net/reader035/viewer/2022070916/5fb6afdb4ee1681fbc611544/html5/thumbnails/29.jpg)
Reconstructing Hidden Rooms: One plane at a time
Visible wall
Invisible walls
![Page 30: See Through Walls with Wi-Figraphics.cs.cmu.edu/courses/15-463/2019_fall/lectures/lecture23.pdftransient camera single-pixel l out l in VD VS • What assumption are we making here?](https://reader035.vdocuments.net/reader035/viewer/2022070916/5fb6afdb4ee1681fbc611544/html5/thumbnails/30.jpg)
Reconstructing Hidden Rooms: One plane at a time
Visible wall
Invisible walls
![Page 31: See Through Walls with Wi-Figraphics.cs.cmu.edu/courses/15-463/2019_fall/lectures/lecture23.pdftransient camera single-pixel l out l in VD VS • What assumption are we making here?](https://reader035.vdocuments.net/reader035/viewer/2022070916/5fb6afdb4ee1681fbc611544/html5/thumbnails/31.jpg)
Reconstructing Hidden Rooms: One plane at a time
Visible wall
Invisible walls
![Page 32: See Through Walls with Wi-Figraphics.cs.cmu.edu/courses/15-463/2019_fall/lectures/lecture23.pdftransient camera single-pixel l out l in VD VS • What assumption are we making here?](https://reader035.vdocuments.net/reader035/viewer/2022070916/5fb6afdb4ee1681fbc611544/html5/thumbnails/32.jpg)
Reconstructing Rectangular Rooms
Original room Reconstructed room
• Average AICP error for all the walls is TBD mm (TBD %). – Normalized with average room
length of 1.1m
![Page 33: See Through Walls with Wi-Figraphics.cs.cmu.edu/courses/15-463/2019_fall/lectures/lecture23.pdftransient camera single-pixel l out l in VD VS • What assumption are we making here?](https://reader035.vdocuments.net/reader035/viewer/2022070916/5fb6afdb4ee1681fbc611544/html5/thumbnails/33.jpg)
Active NLOS imaging using WiFi
![Page 34: See Through Walls with Wi-Figraphics.cs.cmu.edu/courses/15-463/2019_fall/lectures/lecture23.pdftransient camera single-pixel l out l in VD VS • What assumption are we making here?](https://reader035.vdocuments.net/reader035/viewer/2022070916/5fb6afdb4ee1681fbc611544/html5/thumbnails/34.jpg)
Imaging through occlusions
![Page 35: See Through Walls with Wi-Figraphics.cs.cmu.edu/courses/15-463/2019_fall/lectures/lecture23.pdftransient camera single-pixel l out l in VD VS • What assumption are we making here?](https://reader035.vdocuments.net/reader035/viewer/2022070916/5fb6afdb4ee1681fbc611544/html5/thumbnails/35.jpg)
Imaging through occlusionsusing radio frequencies
![Page 36: See Through Walls with Wi-Figraphics.cs.cmu.edu/courses/15-463/2019_fall/lectures/lecture23.pdftransient camera single-pixel l out l in VD VS • What assumption are we making here?](https://reader035.vdocuments.net/reader035/viewer/2022070916/5fb6afdb4ee1681fbc611544/html5/thumbnails/36.jpg)
Key Idea
![Page 37: See Through Walls with Wi-Figraphics.cs.cmu.edu/courses/15-463/2019_fall/lectures/lecture23.pdftransient camera single-pixel l out l in VD VS • What assumption are we making here?](https://reader035.vdocuments.net/reader035/viewer/2022070916/5fb6afdb4ee1681fbc611544/html5/thumbnails/37.jpg)
Challenges
Wall refection is 10,000x stronger than reflections coming from behind the wall
![Page 38: See Through Walls with Wi-Figraphics.cs.cmu.edu/courses/15-463/2019_fall/lectures/lecture23.pdftransient camera single-pixel l out l in VD VS • What assumption are we making here?](https://reader035.vdocuments.net/reader035/viewer/2022070916/5fb6afdb4ee1681fbc611544/html5/thumbnails/38.jpg)
Wi-Vi: Small, Low-Power, Wi-Fi
• Eliminate the wall’s reflection
• Track people from reflections
• Gesture-based interface
• Implemented on software radios
![Page 39: See Through Walls with Wi-Figraphics.cs.cmu.edu/courses/15-463/2019_fall/lectures/lecture23.pdftransient camera single-pixel l out l in VD VS • What assumption are we making here?](https://reader035.vdocuments.net/reader035/viewer/2022070916/5fb6afdb4ee1681fbc611544/html5/thumbnails/39.jpg)
How Can We Eliminate the Wall’s Reflection?
![Page 40: See Through Walls with Wi-Figraphics.cs.cmu.edu/courses/15-463/2019_fall/lectures/lecture23.pdftransient camera single-pixel l out l in VD VS • What assumption are we making here?](https://reader035.vdocuments.net/reader035/viewer/2022070916/5fb6afdb4ee1681fbc611544/html5/thumbnails/40.jpg)
Idea: transmit two waves that cancel each other when they reflect off static objects but not moving objects
Wall is static
People tend to move
disappears
detectable
![Page 41: See Through Walls with Wi-Figraphics.cs.cmu.edu/courses/15-463/2019_fall/lectures/lecture23.pdftransient camera single-pixel l out l in VD VS • What assumption are we making here?](https://reader035.vdocuments.net/reader035/viewer/2022070916/5fb6afdb4ee1681fbc611544/html5/thumbnails/41.jpg)
Eliminating the Wall’s Reflection
Transmit Antennas
Receive Antenna:
αx
x
![Page 42: See Through Walls with Wi-Figraphics.cs.cmu.edu/courses/15-463/2019_fall/lectures/lecture23.pdftransient camera single-pixel l out l in VD VS • What assumption are we making here?](https://reader035.vdocuments.net/reader035/viewer/2022070916/5fb6afdb4ee1681fbc611544/html5/thumbnails/42.jpg)
Eliminating the Wall’s Reflection
h2
h1
y = h1 x + h2αx0
α = -h1 / h2
Receive Antenna:
αx
x
![Page 43: See Through Walls with Wi-Figraphics.cs.cmu.edu/courses/15-463/2019_fall/lectures/lecture23.pdftransient camera single-pixel l out l in VD VS • What assumption are we making here?](https://reader035.vdocuments.net/reader035/viewer/2022070916/5fb6afdb4ee1681fbc611544/html5/thumbnails/43.jpg)
Eliminating All Static Reflections
![Page 44: See Through Walls with Wi-Figraphics.cs.cmu.edu/courses/15-463/2019_fall/lectures/lecture23.pdftransient camera single-pixel l out l in VD VS • What assumption are we making here?](https://reader035.vdocuments.net/reader035/viewer/2022070916/5fb6afdb4ee1681fbc611544/html5/thumbnails/44.jpg)
Eliminating All Static Reflections
y = h1 x + h2αx
Static objects (wall, furniture, etc.) have constant channels
y = h1 x + h2(- h1/ h2)x0
People move, therefore their channels change
y = h1’ x + h2
’(- h1/ h2)x
Not Zero
h2
h1
αx
x
![Page 45: See Through Walls with Wi-Figraphics.cs.cmu.edu/courses/15-463/2019_fall/lectures/lecture23.pdftransient camera single-pixel l out l in VD VS • What assumption are we making here?](https://reader035.vdocuments.net/reader035/viewer/2022070916/5fb6afdb4ee1681fbc611544/html5/thumbnails/45.jpg)
How Can We Track Using Reflections?
![Page 46: See Through Walls with Wi-Figraphics.cs.cmu.edu/courses/15-463/2019_fall/lectures/lecture23.pdftransient camera single-pixel l out l in VD VS • What assumption are we making here?](https://reader035.vdocuments.net/reader035/viewer/2022070916/5fb6afdb4ee1681fbc611544/html5/thumbnails/46.jpg)
Direction of reflection
θ
Antenna Array
Tracking Motion
RF source
![Page 47: See Through Walls with Wi-Figraphics.cs.cmu.edu/courses/15-463/2019_fall/lectures/lecture23.pdftransient camera single-pixel l out l in VD VS • What assumption are we making here?](https://reader035.vdocuments.net/reader035/viewer/2022070916/5fb6afdb4ee1681fbc611544/html5/thumbnails/47.jpg)
θ
Antenna Array
Tracking MotionDirection of motion
At any point in time, we have a single measurement
![Page 48: See Through Walls with Wi-Figraphics.cs.cmu.edu/courses/15-463/2019_fall/lectures/lecture23.pdftransient camera single-pixel l out l in VD VS • What assumption are we making here?](https://reader035.vdocuments.net/reader035/viewer/2022070916/5fb6afdb4ee1681fbc611544/html5/thumbnails/48.jpg)
θ
Antenna Array
Tracking MotionDirection of motion
θ
Direction
of motion
![Page 49: See Through Walls with Wi-Figraphics.cs.cmu.edu/courses/15-463/2019_fall/lectures/lecture23.pdftransient camera single-pixel l out l in VD VS • What assumption are we making here?](https://reader035.vdocuments.net/reader035/viewer/2022070916/5fb6afdb4ee1681fbc611544/html5/thumbnails/49.jpg)
θ
Antenna Array
Tracking MotionDirection of motion
θ
Direction
of motionHuman motion emulates antenna array
![Page 50: See Through Walls with Wi-Figraphics.cs.cmu.edu/courses/15-463/2019_fall/lectures/lecture23.pdftransient camera single-pixel l out l in VD VS • What assumption are we making here?](https://reader035.vdocuments.net/reader035/viewer/2022070916/5fb6afdb4ee1681fbc611544/html5/thumbnails/50.jpg)
![Page 51: See Through Walls with Wi-Figraphics.cs.cmu.edu/courses/15-463/2019_fall/lectures/lecture23.pdftransient camera single-pixel l out l in VD VS • What assumption are we making here?](https://reader035.vdocuments.net/reader035/viewer/2022070916/5fb6afdb4ee1681fbc611544/html5/thumbnails/51.jpg)
A Through-Wall Gesture Interface
• Sending Commands with Gestures
• Two simple gestures to represent bit ‘0’ and bit ‘1’
• Can combine sequence of gestures to convey longer message
![Page 52: See Through Walls with Wi-Figraphics.cs.cmu.edu/courses/15-463/2019_fall/lectures/lecture23.pdftransient camera single-pixel l out l in VD VS • What assumption are we making here?](https://reader035.vdocuments.net/reader035/viewer/2022070916/5fb6afdb4ee1681fbc611544/html5/thumbnails/52.jpg)
Gesture Encoding
Bit ‘0’: step forward followed by step backward
Bit ‘1’: step backward followed by step forward
Step Backward
Step Forward
![Page 53: See Through Walls with Wi-Figraphics.cs.cmu.edu/courses/15-463/2019_fall/lectures/lecture23.pdftransient camera single-pixel l out l in VD VS • What assumption are we making here?](https://reader035.vdocuments.net/reader035/viewer/2022070916/5fb6afdb4ee1681fbc611544/html5/thumbnails/53.jpg)
Bit ‘1’Bit ‘0’
Gesture Decoding
Matched Filter
Peak Detector
Gesture interface that works through walls and none-line-of-sight
![Page 54: See Through Walls with Wi-Figraphics.cs.cmu.edu/courses/15-463/2019_fall/lectures/lecture23.pdftransient camera single-pixel l out l in VD VS • What assumption are we making here?](https://reader035.vdocuments.net/reader035/viewer/2022070916/5fb6afdb4ee1681fbc611544/html5/thumbnails/54.jpg)
Imaging through occlusionsusing radio frequencies
Head
Right Arm
Chest
Left Arm
LowerLeft Arm
LowerRight Arm
Feet
Our output
Camera image
Our RF sensor
![Page 55: See Through Walls with Wi-Figraphics.cs.cmu.edu/courses/15-463/2019_fall/lectures/lecture23.pdftransient camera single-pixel l out l in VD VS • What assumption are we making here?](https://reader035.vdocuments.net/reader035/viewer/2022070916/5fb6afdb4ee1681fbc611544/html5/thumbnails/55.jpg)
Output of 3D RF Scan
Blobs of reflection power
Challenge: Don’t get reflections from most points in RF
![Page 56: See Through Walls with Wi-Figraphics.cs.cmu.edu/courses/15-463/2019_fall/lectures/lecture23.pdftransient camera single-pixel l out l in VD VS • What assumption are we making here?](https://reader035.vdocuments.net/reader035/viewer/2022070916/5fb6afdb4ee1681fbc611544/html5/thumbnails/56.jpg)
At frequencies that traverse walls, human body parts are specular (pure mirror)
Challenge: Don’t get reflections from most points in RF
![Page 57: See Through Walls with Wi-Figraphics.cs.cmu.edu/courses/15-463/2019_fall/lectures/lecture23.pdftransient camera single-pixel l out l in VD VS • What assumption are we making here?](https://reader035.vdocuments.net/reader035/viewer/2022070916/5fb6afdb4ee1681fbc611544/html5/thumbnails/57.jpg)
An
ten
nas
Cannot Capture Reflection
At every point in time, get reflections from only a subset of body parts
At frequencies that traverse walls, human body parts are specular (pure mirror)
Challenge: Don’t get reflections from most points in RF
![Page 58: See Through Walls with Wi-Figraphics.cs.cmu.edu/courses/15-463/2019_fall/lectures/lecture23.pdftransient camera single-pixel l out l in VD VS • What assumption are we making here?](https://reader035.vdocuments.net/reader035/viewer/2022070916/5fb6afdb4ee1681fbc611544/html5/thumbnails/58.jpg)
Solution Idea: Exploit Human Motion and Aggregate over Time
RF Imaging Array
![Page 59: See Through Walls with Wi-Figraphics.cs.cmu.edu/courses/15-463/2019_fall/lectures/lecture23.pdftransient camera single-pixel l out l in VD VS • What assumption are we making here?](https://reader035.vdocuments.net/reader035/viewer/2022070916/5fb6afdb4ee1681fbc611544/html5/thumbnails/59.jpg)
RF Imaging Array
Solution Idea: Exploit Human Motion and Aggregate over Time
![Page 60: See Through Walls with Wi-Figraphics.cs.cmu.edu/courses/15-463/2019_fall/lectures/lecture23.pdftransient camera single-pixel l out l in VD VS • What assumption are we making here?](https://reader035.vdocuments.net/reader035/viewer/2022070916/5fb6afdb4ee1681fbc611544/html5/thumbnails/60.jpg)
3m 2.5m 2m
Chest (Largest Convex Reflector)
Use it as a pivot: for motion compensation and segmentation
Human Walks toward Sensor
![Page 61: See Through Walls with Wi-Figraphics.cs.cmu.edu/courses/15-463/2019_fall/lectures/lecture23.pdftransient camera single-pixel l out l in VD VS • What assumption are we making here?](https://reader035.vdocuments.net/reader035/viewer/2022070916/5fb6afdb4ee1681fbc611544/html5/thumbnails/61.jpg)
3m 2.5m 2m
Human Walks toward Sensor
Chest (Largest Convex Reflector)
Use it as a pivot: for motion compensation and segmentationCombine the various snapshots
Right arm
Left arm
Head
Lower TorsoFeet
![Page 62: See Through Walls with Wi-Figraphics.cs.cmu.edu/courses/15-463/2019_fall/lectures/lecture23.pdftransient camera single-pixel l out l in VD VS • What assumption are we making here?](https://reader035.vdocuments.net/reader035/viewer/2022070916/5fb6afdb4ee1681fbc611544/html5/thumbnails/62.jpg)
Human Walks toward Sensor
![Page 63: See Through Walls with Wi-Figraphics.cs.cmu.edu/courses/15-463/2019_fall/lectures/lecture23.pdftransient camera single-pixel l out l in VD VS • What assumption are we making here?](https://reader035.vdocuments.net/reader035/viewer/2022070916/5fb6afdb4ee1681fbc611544/html5/thumbnails/63.jpg)
Implementation
• Hardware– 2D Antenna Array
– Built RF circuit• 1/1,000 power of WiFi
• USB connection to PC
• Software– Coarse-to-fine algorithm implemented in GPU to
generate reflection snapshots in real-time
Rx
Tx
![Page 64: See Through Walls with Wi-Figraphics.cs.cmu.edu/courses/15-463/2019_fall/lectures/lecture23.pdftransient camera single-pixel l out l in VD VS • What assumption are we making here?](https://reader035.vdocuments.net/reader035/viewer/2022070916/5fb6afdb4ee1681fbc611544/html5/thumbnails/64.jpg)
Evaluation
• RF-Capture sensor placed behind the wall
• 15 participants
• Use Kinect as baseline when needed
SnapshotsareCollectedacrossTimeSensorisPlacedBehindaWall CapturedHumanFigure
SENSORSETUP CAPTUREALGORITHM
Sensor
Head
RightArm
Chest
Le Arm
LowerLe Arm
LowerRightArm
Feet
![Page 65: See Through Walls with Wi-Figraphics.cs.cmu.edu/courses/15-463/2019_fall/lectures/lecture23.pdftransient camera single-pixel l out l in VD VS • What assumption are we making here?](https://reader035.vdocuments.net/reader035/viewer/2022070916/5fb6afdb4ee1681fbc611544/html5/thumbnails/65.jpg)
Sample Captured Figures through Walls
![Page 66: See Through Walls with Wi-Figraphics.cs.cmu.edu/courses/15-463/2019_fall/lectures/lecture23.pdftransient camera single-pixel l out l in VD VS • What assumption are we making here?](https://reader035.vdocuments.net/reader035/viewer/2022070916/5fb6afdb4ee1681fbc611544/html5/thumbnails/66.jpg)
Sample Captured Figures through Walls
-0.2 0 0.2 0.4 0.6 0.8x-axis (meters)
0
0.5
1
1.5
2
y-a
xis
(m
ete
rs)
-0.2 0 0.2 0.4 0.6 0.8x-axis (meters)
0
0.5
1
1.5
2y-a
xis
(m
ete
rs)
-0.2 0 0.2 0.4 0.6 0.8x-axis (meters)
0
0.5
1
1.5
2
y-a
xis
(m
ete
rs)
-0.2 0 0.2 0.4 0.6 0.8x-axis (meters)
0
0.5
1
1.5
2
y-a
xis
(m
ete
rs)
![Page 67: See Through Walls with Wi-Figraphics.cs.cmu.edu/courses/15-463/2019_fall/lectures/lecture23.pdftransient camera single-pixel l out l in VD VS • What assumption are we making here?](https://reader035.vdocuments.net/reader035/viewer/2022070916/5fb6afdb4ee1681fbc611544/html5/thumbnails/67.jpg)
Tracking result
![Page 68: See Through Walls with Wi-Figraphics.cs.cmu.edu/courses/15-463/2019_fall/lectures/lecture23.pdftransient camera single-pixel l out l in VD VS • What assumption are we making here?](https://reader035.vdocuments.net/reader035/viewer/2022070916/5fb6afdb4ee1681fbc611544/html5/thumbnails/68.jpg)
Writing in the air
Median Accuracy is 2cm
Kinect (in red)
![Page 69: See Through Walls with Wi-Figraphics.cs.cmu.edu/courses/15-463/2019_fall/lectures/lecture23.pdftransient camera single-pixel l out l in VD VS • What assumption are we making here?](https://reader035.vdocuments.net/reader035/viewer/2022070916/5fb6afdb4ee1681fbc611544/html5/thumbnails/69.jpg)
Passive NLOS imaging using accidental pinholes
![Page 70: See Through Walls with Wi-Figraphics.cs.cmu.edu/courses/15-463/2019_fall/lectures/lecture23.pdftransient camera single-pixel l out l in VD VS • What assumption are we making here?](https://reader035.vdocuments.net/reader035/viewer/2022070916/5fb6afdb4ee1681fbc611544/html5/thumbnails/70.jpg)
What does this image say about the world outside?
![Page 71: See Through Walls with Wi-Figraphics.cs.cmu.edu/courses/15-463/2019_fall/lectures/lecture23.pdftransient camera single-pixel l out l in VD VS • What assumption are we making here?](https://reader035.vdocuments.net/reader035/viewer/2022070916/5fb6afdb4ee1681fbc611544/html5/thumbnails/71.jpg)
Accidental pinhole camera
![Page 72: See Through Walls with Wi-Figraphics.cs.cmu.edu/courses/15-463/2019_fall/lectures/lecture23.pdftransient camera single-pixel l out l in VD VS • What assumption are we making here?](https://reader035.vdocuments.net/reader035/viewer/2022070916/5fb6afdb4ee1681fbc611544/html5/thumbnails/72.jpg)
Accidental pinhole camera
window is an aperture
projected pattern on the wall window with smaller gap
upside down view outside window
![Page 73: See Through Walls with Wi-Figraphics.cs.cmu.edu/courses/15-463/2019_fall/lectures/lecture23.pdftransient camera single-pixel l out l in VD VS • What assumption are we making here?](https://reader035.vdocuments.net/reader035/viewer/2022070916/5fb6afdb4ee1681fbc611544/html5/thumbnails/73.jpg)
Accidental pinspeck camera
![Page 74: See Through Walls with Wi-Figraphics.cs.cmu.edu/courses/15-463/2019_fall/lectures/lecture23.pdftransient camera single-pixel l out l in VD VS • What assumption are we making here?](https://reader035.vdocuments.net/reader035/viewer/2022070916/5fb6afdb4ee1681fbc611544/html5/thumbnails/74.jpg)
Passive NLOS imaging using accidental reflectors
![Page 75: See Through Walls with Wi-Figraphics.cs.cmu.edu/courses/15-463/2019_fall/lectures/lecture23.pdftransient camera single-pixel l out l in VD VS • What assumption are we making here?](https://reader035.vdocuments.net/reader035/viewer/2022070916/5fb6afdb4ee1681fbc611544/html5/thumbnails/75.jpg)
Ophthalmology
• Anatomy, physiology
[Helmholtz 09;…]
Computer Vision
•Iris recognition
[Daugman 93;…]
HCI
• Gaze as an interface
[Bolt 82;…]
Computer Graphics
• Vision-realistic Rendering
[Barsky et al. 02;…]
![Page 76: See Through Walls with Wi-Figraphics.cs.cmu.edu/courses/15-463/2019_fall/lectures/lecture23.pdftransient camera single-pixel l out l in VD VS • What assumption are we making here?](https://reader035.vdocuments.net/reader035/viewer/2022070916/5fb6afdb4ee1681fbc611544/html5/thumbnails/76.jpg)
Corneal Imaging System
![Page 77: See Through Walls with Wi-Figraphics.cs.cmu.edu/courses/15-463/2019_fall/lectures/lecture23.pdftransient camera single-pixel l out l in VD VS • What assumption are we making here?](https://reader035.vdocuments.net/reader035/viewer/2022070916/5fb6afdb4ee1681fbc611544/html5/thumbnails/77.jpg)
Geometric Model of the Cornea
mm5.5mm18.2 ==Lb
rt
R=7.6mm
eccentricity = 0.5
Cornea
ScleraIris Pupil
![Page 78: See Through Walls with Wi-Figraphics.cs.cmu.edu/courses/15-463/2019_fall/lectures/lecture23.pdftransient camera single-pixel l out l in VD VS • What assumption are we making here?](https://reader035.vdocuments.net/reader035/viewer/2022070916/5fb6afdb4ee1681fbc611544/html5/thumbnails/78.jpg)
Self-calibration: 3D Coordinates, 3D Orientation
![Page 79: See Through Walls with Wi-Figraphics.cs.cmu.edu/courses/15-463/2019_fall/lectures/lecture23.pdftransient camera single-pixel l out l in VD VS • What assumption are we making here?](https://reader035.vdocuments.net/reader035/viewer/2022070916/5fb6afdb4ee1681fbc611544/html5/thumbnails/79.jpg)
Viewpoint Loci
camera pupil
Z
X
cornea
![Page 80: See Through Walls with Wi-Figraphics.cs.cmu.edu/courses/15-463/2019_fall/lectures/lecture23.pdftransient camera single-pixel l out l in VD VS • What assumption are we making here?](https://reader035.vdocuments.net/reader035/viewer/2022070916/5fb6afdb4ee1681fbc611544/html5/thumbnails/80.jpg)
Viewpoint Loci
Z
X
cornea
camera pupil
![Page 81: See Through Walls with Wi-Figraphics.cs.cmu.edu/courses/15-463/2019_fall/lectures/lecture23.pdftransient camera single-pixel l out l in VD VS • What assumption are we making here?](https://reader035.vdocuments.net/reader035/viewer/2022070916/5fb6afdb4ee1681fbc611544/html5/thumbnails/81.jpg)
Viewpoint Loci
cornea
camera pupil
![Page 82: See Through Walls with Wi-Figraphics.cs.cmu.edu/courses/15-463/2019_fall/lectures/lecture23.pdftransient camera single-pixel l out l in VD VS • What assumption are we making here?](https://reader035.vdocuments.net/reader035/viewer/2022070916/5fb6afdb4ee1681fbc611544/html5/thumbnails/82.jpg)
Resolution and Field of View
X
Z
gaze direction
cornea
![Page 83: See Through Walls with Wi-Figraphics.cs.cmu.edu/courses/15-463/2019_fall/lectures/lecture23.pdftransient camera single-pixel l out l in VD VS • What assumption are we making here?](https://reader035.vdocuments.net/reader035/viewer/2022070916/5fb6afdb4ee1681fbc611544/html5/thumbnails/83.jpg)
Resolution and Field of View
X
Z
FOV
gaze direction
cornea
resolution
![Page 84: See Through Walls with Wi-Figraphics.cs.cmu.edu/courses/15-463/2019_fall/lectures/lecture23.pdftransient camera single-pixel l out l in VD VS • What assumption are we making here?](https://reader035.vdocuments.net/reader035/viewer/2022070916/5fb6afdb4ee1681fbc611544/html5/thumbnails/84.jpg)
Resolution and Field of View
X
Z
FOV
gaze direction
cornea
27 45person’s FOV
resolution
![Page 85: See Through Walls with Wi-Figraphics.cs.cmu.edu/courses/15-463/2019_fall/lectures/lecture23.pdftransient camera single-pixel l out l in VD VS • What assumption are we making here?](https://reader035.vdocuments.net/reader035/viewer/2022070916/5fb6afdb4ee1681fbc611544/html5/thumbnails/85.jpg)
![Page 86: See Through Walls with Wi-Figraphics.cs.cmu.edu/courses/15-463/2019_fall/lectures/lecture23.pdftransient camera single-pixel l out l in VD VS • What assumption are we making here?](https://reader035.vdocuments.net/reader035/viewer/2022070916/5fb6afdb4ee1681fbc611544/html5/thumbnails/86.jpg)
Environment Map from an Eye
![Page 87: See Through Walls with Wi-Figraphics.cs.cmu.edu/courses/15-463/2019_fall/lectures/lecture23.pdftransient camera single-pixel l out l in VD VS • What assumption are we making here?](https://reader035.vdocuments.net/reader035/viewer/2022070916/5fb6afdb4ee1681fbc611544/html5/thumbnails/87.jpg)
What Exactly You are Looking AtEye Image:
Computed Retinal Image:
![Page 89: See Through Walls with Wi-Figraphics.cs.cmu.edu/courses/15-463/2019_fall/lectures/lecture23.pdftransient camera single-pixel l out l in VD VS • What assumption are we making here?](https://reader035.vdocuments.net/reader035/viewer/2022070916/5fb6afdb4ee1681fbc611544/html5/thumbnails/89.jpg)
![Page 90: See Through Walls with Wi-Figraphics.cs.cmu.edu/courses/15-463/2019_fall/lectures/lecture23.pdftransient camera single-pixel l out l in VD VS • What assumption are we making here?](https://reader035.vdocuments.net/reader035/viewer/2022070916/5fb6afdb4ee1681fbc611544/html5/thumbnails/90.jpg)
Corneal Stereo System
right cornea left cornea
camera pupil
epipolar curve
epipolar plane
![Page 91: See Through Walls with Wi-Figraphics.cs.cmu.edu/courses/15-463/2019_fall/lectures/lecture23.pdftransient camera single-pixel l out l in VD VS • What assumption are we making here?](https://reader035.vdocuments.net/reader035/viewer/2022070916/5fb6afdb4ee1681fbc611544/html5/thumbnails/91.jpg)
From Two Eyes in an Image …
Epipolar CurvesReconstructed Structure (frontal and side view)
Y
X
X
Y
Z
![Page 92: See Through Walls with Wi-Figraphics.cs.cmu.edu/courses/15-463/2019_fall/lectures/lecture23.pdftransient camera single-pixel l out l in VD VS • What assumption are we making here?](https://reader035.vdocuments.net/reader035/viewer/2022070916/5fb6afdb4ee1681fbc611544/html5/thumbnails/92.jpg)
Eyes Reveal …
• Where the person is
• What the person is looking at
• The structure of objects
![Page 93: See Through Walls with Wi-Figraphics.cs.cmu.edu/courses/15-463/2019_fall/lectures/lecture23.pdftransient camera single-pixel l out l in VD VS • What assumption are we making here?](https://reader035.vdocuments.net/reader035/viewer/2022070916/5fb6afdb4ee1681fbc611544/html5/thumbnails/93.jpg)
Dynamic Illumination in a Video
![Page 94: See Through Walls with Wi-Figraphics.cs.cmu.edu/courses/15-463/2019_fall/lectures/lecture23.pdftransient camera single-pixel l out l in VD VS • What assumption are we making here?](https://reader035.vdocuments.net/reader035/viewer/2022070916/5fb6afdb4ee1681fbc611544/html5/thumbnails/94.jpg)
Point Source Direction from the Eye
intensity
![Page 95: See Through Walls with Wi-Figraphics.cs.cmu.edu/courses/15-463/2019_fall/lectures/lecture23.pdftransient camera single-pixel l out l in VD VS • What assumption are we making here?](https://reader035.vdocuments.net/reader035/viewer/2022070916/5fb6afdb4ee1681fbc611544/html5/thumbnails/95.jpg)
Inserting a Virtual Object
![Page 96: See Through Walls with Wi-Figraphics.cs.cmu.edu/courses/15-463/2019_fall/lectures/lecture23.pdftransient camera single-pixel l out l in VD VS • What assumption are we making here?](https://reader035.vdocuments.net/reader035/viewer/2022070916/5fb6afdb4ee1681fbc611544/html5/thumbnails/96.jpg)
Sampling Appearance using Eyes
![Page 97: See Through Walls with Wi-Figraphics.cs.cmu.edu/courses/15-463/2019_fall/lectures/lecture23.pdftransient camera single-pixel l out l in VD VS • What assumption are we making here?](https://reader035.vdocuments.net/reader035/viewer/2022070916/5fb6afdb4ee1681fbc611544/html5/thumbnails/97.jpg)
Sampling Appearance using Eyes
![Page 98: See Through Walls with Wi-Figraphics.cs.cmu.edu/courses/15-463/2019_fall/lectures/lecture23.pdftransient camera single-pixel l out l in VD VS • What assumption are we making here?](https://reader035.vdocuments.net/reader035/viewer/2022070916/5fb6afdb4ee1681fbc611544/html5/thumbnails/98.jpg)
Computed Point Source Trajectory
intensity
![Page 99: See Through Walls with Wi-Figraphics.cs.cmu.edu/courses/15-463/2019_fall/lectures/lecture23.pdftransient camera single-pixel l out l in VD VS • What assumption are we making here?](https://reader035.vdocuments.net/reader035/viewer/2022070916/5fb6afdb4ee1681fbc611544/html5/thumbnails/99.jpg)
Fitting a Reflectance Model
![Page 100: See Through Walls with Wi-Figraphics.cs.cmu.edu/courses/15-463/2019_fall/lectures/lecture23.pdftransient camera single-pixel l out l in VD VS • What assumption are we making here?](https://reader035.vdocuments.net/reader035/viewer/2022070916/5fb6afdb4ee1681fbc611544/html5/thumbnails/100.jpg)
3D Model Reconstruction
![Page 101: See Through Walls with Wi-Figraphics.cs.cmu.edu/courses/15-463/2019_fall/lectures/lecture23.pdftransient camera single-pixel l out l in VD VS • What assumption are we making here?](https://reader035.vdocuments.net/reader035/viewer/2022070916/5fb6afdb4ee1681fbc611544/html5/thumbnails/101.jpg)
Relighting under Novel Illumination
![Page 102: See Through Walls with Wi-Figraphics.cs.cmu.edu/courses/15-463/2019_fall/lectures/lecture23.pdftransient camera single-pixel l out l in VD VS • What assumption are we making here?](https://reader035.vdocuments.net/reader035/viewer/2022070916/5fb6afdb4ee1681fbc611544/html5/thumbnails/102.jpg)
VisualEyesTM
http://www.cs.columbia.edu/CAVE/
with Akira Yanagawa
![Page 103: See Through Walls with Wi-Figraphics.cs.cmu.edu/courses/15-463/2019_fall/lectures/lecture23.pdftransient camera single-pixel l out l in VD VS • What assumption are we making here?](https://reader035.vdocuments.net/reader035/viewer/2022070916/5fb6afdb4ee1681fbc611544/html5/thumbnails/103.jpg)
Passive NLOS imaging using corners
![Page 104: See Through Walls with Wi-Figraphics.cs.cmu.edu/courses/15-463/2019_fall/lectures/lecture23.pdftransient camera single-pixel l out l in VD VS • What assumption are we making here?](https://reader035.vdocuments.net/reader035/viewer/2022070916/5fb6afdb4ee1681fbc611544/html5/thumbnails/104.jpg)
![Page 105: See Through Walls with Wi-Figraphics.cs.cmu.edu/courses/15-463/2019_fall/lectures/lecture23.pdftransient camera single-pixel l out l in VD VS • What assumption are we making here?](https://reader035.vdocuments.net/reader035/viewer/2022070916/5fb6afdb4ee1681fbc611544/html5/thumbnails/105.jpg)
![Page 106: See Through Walls with Wi-Figraphics.cs.cmu.edu/courses/15-463/2019_fall/lectures/lecture23.pdftransient camera single-pixel l out l in VD VS • What assumption are we making here?](https://reader035.vdocuments.net/reader035/viewer/2022070916/5fb6afdb4ee1681fbc611544/html5/thumbnails/106.jpg)
![Page 107: See Through Walls with Wi-Figraphics.cs.cmu.edu/courses/15-463/2019_fall/lectures/lecture23.pdftransient camera single-pixel l out l in VD VS • What assumption are we making here?](https://reader035.vdocuments.net/reader035/viewer/2022070916/5fb6afdb4ee1681fbc611544/html5/thumbnails/107.jpg)
![Page 108: See Through Walls with Wi-Figraphics.cs.cmu.edu/courses/15-463/2019_fall/lectures/lecture23.pdftransient camera single-pixel l out l in VD VS • What assumption are we making here?](https://reader035.vdocuments.net/reader035/viewer/2022070916/5fb6afdb4ee1681fbc611544/html5/thumbnails/108.jpg)
![Page 109: See Through Walls with Wi-Figraphics.cs.cmu.edu/courses/15-463/2019_fall/lectures/lecture23.pdftransient camera single-pixel l out l in VD VS • What assumption are we making here?](https://reader035.vdocuments.net/reader035/viewer/2022070916/5fb6afdb4ee1681fbc611544/html5/thumbnails/109.jpg)
![Page 110: See Through Walls with Wi-Figraphics.cs.cmu.edu/courses/15-463/2019_fall/lectures/lecture23.pdftransient camera single-pixel l out l in VD VS • What assumption are we making here?](https://reader035.vdocuments.net/reader035/viewer/2022070916/5fb6afdb4ee1681fbc611544/html5/thumbnails/110.jpg)
![Page 111: See Through Walls with Wi-Figraphics.cs.cmu.edu/courses/15-463/2019_fall/lectures/lecture23.pdftransient camera single-pixel l out l in VD VS • What assumption are we making here?](https://reader035.vdocuments.net/reader035/viewer/2022070916/5fb6afdb4ee1681fbc611544/html5/thumbnails/111.jpg)
![Page 112: See Through Walls with Wi-Figraphics.cs.cmu.edu/courses/15-463/2019_fall/lectures/lecture23.pdftransient camera single-pixel l out l in VD VS • What assumption are we making here?](https://reader035.vdocuments.net/reader035/viewer/2022070916/5fb6afdb4ee1681fbc611544/html5/thumbnails/112.jpg)
![Page 113: See Through Walls with Wi-Figraphics.cs.cmu.edu/courses/15-463/2019_fall/lectures/lecture23.pdftransient camera single-pixel l out l in VD VS • What assumption are we making here?](https://reader035.vdocuments.net/reader035/viewer/2022070916/5fb6afdb4ee1681fbc611544/html5/thumbnails/113.jpg)
![Page 114: See Through Walls with Wi-Figraphics.cs.cmu.edu/courses/15-463/2019_fall/lectures/lecture23.pdftransient camera single-pixel l out l in VD VS • What assumption are we making here?](https://reader035.vdocuments.net/reader035/viewer/2022070916/5fb6afdb4ee1681fbc611544/html5/thumbnails/114.jpg)
![Page 115: See Through Walls with Wi-Figraphics.cs.cmu.edu/courses/15-463/2019_fall/lectures/lecture23.pdftransient camera single-pixel l out l in VD VS • What assumption are we making here?](https://reader035.vdocuments.net/reader035/viewer/2022070916/5fb6afdb4ee1681fbc611544/html5/thumbnails/115.jpg)
![Page 116: See Through Walls with Wi-Figraphics.cs.cmu.edu/courses/15-463/2019_fall/lectures/lecture23.pdftransient camera single-pixel l out l in VD VS • What assumption are we making here?](https://reader035.vdocuments.net/reader035/viewer/2022070916/5fb6afdb4ee1681fbc611544/html5/thumbnails/116.jpg)
![Page 117: See Through Walls with Wi-Figraphics.cs.cmu.edu/courses/15-463/2019_fall/lectures/lecture23.pdftransient camera single-pixel l out l in VD VS • What assumption are we making here?](https://reader035.vdocuments.net/reader035/viewer/2022070916/5fb6afdb4ee1681fbc611544/html5/thumbnails/117.jpg)
![Page 118: See Through Walls with Wi-Figraphics.cs.cmu.edu/courses/15-463/2019_fall/lectures/lecture23.pdftransient camera single-pixel l out l in VD VS • What assumption are we making here?](https://reader035.vdocuments.net/reader035/viewer/2022070916/5fb6afdb4ee1681fbc611544/html5/thumbnails/118.jpg)
![Page 119: See Through Walls with Wi-Figraphics.cs.cmu.edu/courses/15-463/2019_fall/lectures/lecture23.pdftransient camera single-pixel l out l in VD VS • What assumption are we making here?](https://reader035.vdocuments.net/reader035/viewer/2022070916/5fb6afdb4ee1681fbc611544/html5/thumbnails/119.jpg)
![Page 120: See Through Walls with Wi-Figraphics.cs.cmu.edu/courses/15-463/2019_fall/lectures/lecture23.pdftransient camera single-pixel l out l in VD VS • What assumption are we making here?](https://reader035.vdocuments.net/reader035/viewer/2022070916/5fb6afdb4ee1681fbc611544/html5/thumbnails/120.jpg)
![Page 121: See Through Walls with Wi-Figraphics.cs.cmu.edu/courses/15-463/2019_fall/lectures/lecture23.pdftransient camera single-pixel l out l in VD VS • What assumption are we making here?](https://reader035.vdocuments.net/reader035/viewer/2022070916/5fb6afdb4ee1681fbc611544/html5/thumbnails/121.jpg)
![Page 122: See Through Walls with Wi-Figraphics.cs.cmu.edu/courses/15-463/2019_fall/lectures/lecture23.pdftransient camera single-pixel l out l in VD VS • What assumption are we making here?](https://reader035.vdocuments.net/reader035/viewer/2022070916/5fb6afdb4ee1681fbc611544/html5/thumbnails/122.jpg)
![Page 123: See Through Walls with Wi-Figraphics.cs.cmu.edu/courses/15-463/2019_fall/lectures/lecture23.pdftransient camera single-pixel l out l in VD VS • What assumption are we making here?](https://reader035.vdocuments.net/reader035/viewer/2022070916/5fb6afdb4ee1681fbc611544/html5/thumbnails/123.jpg)
![Page 124: See Through Walls with Wi-Figraphics.cs.cmu.edu/courses/15-463/2019_fall/lectures/lecture23.pdftransient camera single-pixel l out l in VD VS • What assumption are we making here?](https://reader035.vdocuments.net/reader035/viewer/2022070916/5fb6afdb4ee1681fbc611544/html5/thumbnails/124.jpg)
References
Basic reading:• Kirmani et al., “Looking around the corner using ultrafast transient imaging,” ICCV 2009 and IJCV 2011.• Velten et al., “Recovering three-dimensional shape around a corner using ultrafast time-of-flight imaging,”
Nature Communications 2012.• O’Toole et al., “Confocal non-line-of-sight imaging,” Nature 2018.
The first two papers introduced the idea of using ToF imaging for looking around the corner. The third paper discusses the advantages of the confocal scanning case.
• Abib and Katabi, “See Through Walls with Wi-Fi!,” SIGCOMM 2013.• Abib et al., “Capturing the Human Figure Through a Wall,” SIGGRAPH Asia 2015.
The two papers showing that WiFi can be used to see through walls.• Torralba and Freeman, “Accidental Pinhole and Pinspeck Cameras,” CVPR 2012.
The paper discussing passive NLOS imaging using accidental pinholes.• Nishimo and Nayar, “Corneal Imaging System: Environment from Eyes,” IJCV 2006.
The paper discussing passive NLOS imaging using accidental reflectors.• Bouman et al., “Turning corners into cameras: Principles and Methods,” ICCV 2017.
The paper discussing passive NLOS imaging using corners.
Additional reading:• Pediredla et al., “Reconstructing rooms using photon echoes: A plane based model and reconstruction
algorithm for looking around the corner,” ICCP 2017.The paper on NLOS room reconstruction using ToF imaging.
• Nishimo and Nayar, “Eyes for relighting,” SIGGRAPH 2004.A follow-up paper to the paper on corneal imaging, show how similar ideas can be used for relighting and other image-based rendering tasks.