![Page 1: Interactive Deformation of Light Fields Billy Chen Eyal Ofek Heung-Yeung Shum Marc Levoy](https://reader036.vdocuments.net/reader036/viewer/2022081602/551476e5550346b2598b45c4/html5/thumbnails/1.jpg)
Interactive Deformation Interactive Deformation of Light Fieldsof Light Fields
Billy Chen
Eyal Ofek
Heung-Yeung Shum
Marc Levoy
![Page 2: Interactive Deformation of Light Fields Billy Chen Eyal Ofek Heung-Yeung Shum Marc Levoy](https://reader036.vdocuments.net/reader036/viewer/2022081602/551476e5550346b2598b45c4/html5/thumbnails/2.jpg)
Image-based renderingImage-based rendering
Levoy et al., 1996 Shum et al., 1999
Zitnick et al., 2004 Uyttendaele et al., 2004
![Page 3: Interactive Deformation of Light Fields Billy Chen Eyal Ofek Heung-Yeung Shum Marc Levoy](https://reader036.vdocuments.net/reader036/viewer/2022081602/551476e5550346b2598b45c4/html5/thumbnails/3.jpg)
Previous workPrevious work
Seitz, S., Kutulakos, K. N. “Plenoptic image editing,” In Proc. ICCV 1998.
Zhang, Z., Wang, L., Guo, B., Shum, H.-Y, “Feature-based light field morphing,” In Proc. SIGGRAPH 2002.
Weyrich, T., Pfister, H., Gross, M., “Rendering deformable surface reflectance fields,” In IEEE Transactions on Computer Graphics and Visualization.
![Page 4: Interactive Deformation of Light Fields Billy Chen Eyal Ofek Heung-Yeung Shum Marc Levoy](https://reader036.vdocuments.net/reader036/viewer/2022081602/551476e5550346b2598b45c4/html5/thumbnails/4.jpg)
Light field deformationLight field deformation
![Page 5: Interactive Deformation of Light Fields Billy Chen Eyal Ofek Heung-Yeung Shum Marc Levoy](https://reader036.vdocuments.net/reader036/viewer/2022081602/551476e5550346b2598b45c4/html5/thumbnails/5.jpg)
OutlineOutline
Coaxial light fields Framework for deformation
Splitting Deforming Rendering
Results
![Page 6: Interactive Deformation of Light Fields Billy Chen Eyal Ofek Heung-Yeung Shum Marc Levoy](https://reader036.vdocuments.net/reader036/viewer/2022081602/551476e5550346b2598b45c4/html5/thumbnails/6.jpg)
OutlineOutline
Coaxial light fields Framework for deformation
Splitting Deforming Rendering
Results
![Page 7: Interactive Deformation of Light Fields Billy Chen Eyal Ofek Heung-Yeung Shum Marc Levoy](https://reader036.vdocuments.net/reader036/viewer/2022081602/551476e5550346b2598b45c4/html5/thumbnails/7.jpg)
Coaxial light fieldsCoaxial light fields
… …
![Page 8: Interactive Deformation of Light Fields Billy Chen Eyal Ofek Heung-Yeung Shum Marc Levoy](https://reader036.vdocuments.net/reader036/viewer/2022081602/551476e5550346b2598b45c4/html5/thumbnails/8.jpg)
![Page 9: Interactive Deformation of Light Fields Billy Chen Eyal Ofek Heung-Yeung Shum Marc Levoy](https://reader036.vdocuments.net/reader036/viewer/2022081602/551476e5550346b2598b45c4/html5/thumbnails/9.jpg)
![Page 10: Interactive Deformation of Light Fields Billy Chen Eyal Ofek Heung-Yeung Shum Marc Levoy](https://reader036.vdocuments.net/reader036/viewer/2022081602/551476e5550346b2598b45c4/html5/thumbnails/10.jpg)
OutlineOutline
Coaxial light fields Framework for deformation
Splitting Deforming Rendering
Results
![Page 11: Interactive Deformation of Light Fields Billy Chen Eyal Ofek Heung-Yeung Shum Marc Levoy](https://reader036.vdocuments.net/reader036/viewer/2022081602/551476e5550346b2598b45c4/html5/thumbnails/11.jpg)
Splitting a light fieldSplitting a light field
A sub-light field consists of a bundle of rays a deformation box
u
s
![Page 12: Interactive Deformation of Light Fields Billy Chen Eyal Ofek Heung-Yeung Shum Marc Levoy](https://reader036.vdocuments.net/reader036/viewer/2022081602/551476e5550346b2598b45c4/html5/thumbnails/12.jpg)
Sub-light field exampleSub-light field example
![Page 13: Interactive Deformation of Light Fields Billy Chen Eyal Ofek Heung-Yeung Shum Marc Levoy](https://reader036.vdocuments.net/reader036/viewer/2022081602/551476e5550346b2598b45c4/html5/thumbnails/13.jpg)
Projector-based light field Projector-based light field segmentationsegmentation
![Page 14: Interactive Deformation of Light Fields Billy Chen Eyal Ofek Heung-Yeung Shum Marc Levoy](https://reader036.vdocuments.net/reader036/viewer/2022081602/551476e5550346b2598b45c4/html5/thumbnails/14.jpg)
…
![Page 15: Interactive Deformation of Light Fields Billy Chen Eyal Ofek Heung-Yeung Shum Marc Levoy](https://reader036.vdocuments.net/reader036/viewer/2022081602/551476e5550346b2598b45c4/html5/thumbnails/15.jpg)
![Page 16: Interactive Deformation of Light Fields Billy Chen Eyal Ofek Heung-Yeung Shum Marc Levoy](https://reader036.vdocuments.net/reader036/viewer/2022081602/551476e5550346b2598b45c4/html5/thumbnails/16.jpg)
OutlineOutline
Coaxial light fields Framework for deformation
Splitting Deforming Rendering
Results
![Page 17: Interactive Deformation of Light Fields Billy Chen Eyal Ofek Heung-Yeung Shum Marc Levoy](https://reader036.vdocuments.net/reader036/viewer/2022081602/551476e5550346b2598b45c4/html5/thumbnails/17.jpg)
DeformationDeformation
![Page 18: Interactive Deformation of Light Fields Billy Chen Eyal Ofek Heung-Yeung Shum Marc Levoy](https://reader036.vdocuments.net/reader036/viewer/2022081602/551476e5550346b2598b45c4/html5/thumbnails/18.jpg)
Ray warpingRay warping
• preserves mapping specified by the animator• preserves C0
continuity of points• maps lines to lines
![Page 19: Interactive Deformation of Light Fields Billy Chen Eyal Ofek Heung-Yeung Shum Marc Levoy](https://reader036.vdocuments.net/reader036/viewer/2022081602/551476e5550346b2598b45c4/html5/thumbnails/19.jpg)
Inverse view ray warpingInverse view ray warping
![Page 20: Interactive Deformation of Light Fields Billy Chen Eyal Ofek Heung-Yeung Shum Marc Levoy](https://reader036.vdocuments.net/reader036/viewer/2022081602/551476e5550346b2598b45c4/html5/thumbnails/20.jpg)
Inverse view ray warpingInverse view ray warping
![Page 21: Interactive Deformation of Light Fields Billy Chen Eyal Ofek Heung-Yeung Shum Marc Levoy](https://reader036.vdocuments.net/reader036/viewer/2022081602/551476e5550346b2598b45c4/html5/thumbnails/21.jpg)
Hardware accelerated warpingHardware accelerated warping
front id map back id map deformation
• 256 x 256 resolution texture map• anisoptric filter for mip-mapped texture• 64 x 64 triangles• real-time with a NVIDIA GeForce 3
![Page 22: Interactive Deformation of Light Fields Billy Chen Eyal Ofek Heung-Yeung Shum Marc Levoy](https://reader036.vdocuments.net/reader036/viewer/2022081602/551476e5550346b2598b45c4/html5/thumbnails/22.jpg)
OutlineOutline
Coaxial light fields Framework for deformation
Splitting Deforming Rendering
Results
![Page 23: Interactive Deformation of Light Fields Billy Chen Eyal Ofek Heung-Yeung Shum Marc Levoy](https://reader036.vdocuments.net/reader036/viewer/2022081602/551476e5550346b2598b45c4/html5/thumbnails/23.jpg)
For each camera pixel1. Segment ray2. Warp ray3. Render ray4. Mask ray
Rendering and visibilityRendering and visibility
![Page 24: Interactive Deformation of Light Fields Billy Chen Eyal Ofek Heung-Yeung Shum Marc Levoy](https://reader036.vdocuments.net/reader036/viewer/2022081602/551476e5550346b2598b45c4/html5/thumbnails/24.jpg)
ResultsResults
![Page 25: Interactive Deformation of Light Fields Billy Chen Eyal Ofek Heung-Yeung Shum Marc Levoy](https://reader036.vdocuments.net/reader036/viewer/2022081602/551476e5550346b2598b45c4/html5/thumbnails/25.jpg)
ConclusionConclusion
A pipeline for deforming light fields Light field segmentation Ray warping Rendering multiple light fields
Coaxial light fields Enables interactive deformation of photo-
realistic objects Applications: photo-realistic rapid prototyping,
games
![Page 26: Interactive Deformation of Light Fields Billy Chen Eyal Ofek Heung-Yeung Shum Marc Levoy](https://reader036.vdocuments.net/reader036/viewer/2022081602/551476e5550346b2598b45c4/html5/thumbnails/26.jpg)
Thank youThank you
Reviewers NSF (IIS-0219856-001) DARPA (NBCH-1030009) Elizabeth Chen, Vaibhav Vaish, Gaurav Garg
![Page 27: Interactive Deformation of Light Fields Billy Chen Eyal Ofek Heung-Yeung Shum Marc Levoy](https://reader036.vdocuments.net/reader036/viewer/2022081602/551476e5550346b2598b45c4/html5/thumbnails/27.jpg)
Failure casesFailure cases
![Page 28: Interactive Deformation of Light Fields Billy Chen Eyal Ofek Heung-Yeung Shum Marc Levoy](https://reader036.vdocuments.net/reader036/viewer/2022081602/551476e5550346b2598b45c4/html5/thumbnails/28.jpg)
Video of acquisition processVideo of acquisition process
![Page 29: Interactive Deformation of Light Fields Billy Chen Eyal Ofek Heung-Yeung Shum Marc Levoy](https://reader036.vdocuments.net/reader036/viewer/2022081602/551476e5550346b2598b45c4/html5/thumbnails/29.jpg)
DeformationDeformationand compositingand compositing