light field
DESCRIPTION
Light Field. Modeling a desktop. Image Based Rendering. Fast Realistic Rendering without 3D models. Start from Ray Tracing. Rendering is about computing color along each ray. Sampling Rays. Sampling Rays by Taking Pictures. Rendering as Ray Resampling. Ray space. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Light Field](https://reader036.vdocuments.net/reader036/viewer/2022062408/56813863550346895da011d1/html5/thumbnails/1.jpg)
Light FieldLight Field
![Page 2: Light Field](https://reader036.vdocuments.net/reader036/viewer/2022062408/56813863550346895da011d1/html5/thumbnails/2.jpg)
Modeling a desktopModeling a desktop
![Page 3: Light Field](https://reader036.vdocuments.net/reader036/viewer/2022062408/56813863550346895da011d1/html5/thumbnails/3.jpg)
![Page 4: Light Field](https://reader036.vdocuments.net/reader036/viewer/2022062408/56813863550346895da011d1/html5/thumbnails/4.jpg)
Image Based RenderingImage Based Rendering
Fast Realistic Rendering without 3D models
![Page 5: Light Field](https://reader036.vdocuments.net/reader036/viewer/2022062408/56813863550346895da011d1/html5/thumbnails/5.jpg)
Start from Ray TracingStart from Ray Tracing
Rendering is about computing color along each ray
![Page 6: Light Field](https://reader036.vdocuments.net/reader036/viewer/2022062408/56813863550346895da011d1/html5/thumbnails/6.jpg)
Sampling RaysSampling Rays
![Page 7: Light Field](https://reader036.vdocuments.net/reader036/viewer/2022062408/56813863550346895da011d1/html5/thumbnails/7.jpg)
Sampling Rays by Taking PicturesSampling Rays by Taking Pictures
![Page 8: Light Field](https://reader036.vdocuments.net/reader036/viewer/2022062408/56813863550346895da011d1/html5/thumbnails/8.jpg)
Rendering as Ray ResamplingRendering as Ray Resampling
![Page 9: Light Field](https://reader036.vdocuments.net/reader036/viewer/2022062408/56813863550346895da011d1/html5/thumbnails/9.jpg)
Ray spaceRay space
How to parameterize the ray space
How to sample and resample rays
![Page 10: Light Field](https://reader036.vdocuments.net/reader036/viewer/2022062408/56813863550346895da011d1/html5/thumbnails/10.jpg)
Two Plane ParameterizationTwo Plane Parameterization
![Page 11: Light Field](https://reader036.vdocuments.net/reader036/viewer/2022062408/56813863550346895da011d1/html5/thumbnails/11.jpg)
Stanford Camera ArrayStanford Camera Array
![Page 12: Light Field](https://reader036.vdocuments.net/reader036/viewer/2022062408/56813863550346895da011d1/html5/thumbnails/12.jpg)
Light Field RenderingLight Field Rendering
Very Fast
![Page 13: Light Field](https://reader036.vdocuments.net/reader036/viewer/2022062408/56813863550346895da011d1/html5/thumbnails/13.jpg)
Light Field RenderingLight Field Rendering
4D interpolation
![Page 14: Light Field](https://reader036.vdocuments.net/reader036/viewer/2022062408/56813863550346895da011d1/html5/thumbnails/14.jpg)
Dynamic Reparameterized Light Fields
![Page 15: Light Field](https://reader036.vdocuments.net/reader036/viewer/2022062408/56813863550346895da011d1/html5/thumbnails/15.jpg)
Dynamic Reparameterized Light Fields
• Move to desired new focal surface• Create a new 4D space with new focal surface• Recove ray with Reparameterization
• (u, v, s, t) => (u, v, f, g)F
![Page 16: Light Field](https://reader036.vdocuments.net/reader036/viewer/2022062408/56813863550346895da011d1/html5/thumbnails/16.jpg)
Dynamic Reparameterized Light Fields
• Recover ray r• Resample from ray (s’, t’, f, g) and (s’’, t’’, f, g)• Interpolation, reconstruction with filter, … , etc
![Page 17: Light Field](https://reader036.vdocuments.net/reader036/viewer/2022062408/56813863550346895da011d1/html5/thumbnails/17.jpg)
Dynamic Reparameterized Light Fields
• Change the shape of focal surface • Gives focus on 3D object rather than planes
![Page 18: Light Field](https://reader036.vdocuments.net/reader036/viewer/2022062408/56813863550346895da011d1/html5/thumbnails/18.jpg)
Dynamic Reparameterized Light Fields
![Page 19: Light Field](https://reader036.vdocuments.net/reader036/viewer/2022062408/56813863550346895da011d1/html5/thumbnails/19.jpg)
Dynamic Reparameterized Light Fields
![Page 20: Light Field](https://reader036.vdocuments.net/reader036/viewer/2022062408/56813863550346895da011d1/html5/thumbnails/20.jpg)
Variable Apertures
• Also can generate variable aperture• Aperture
– Control amount of light– Control depth of fields
• Aperture Filter: – Control how many cameras are used to resample a required ray– Larger apertures produce images with narrow range of focus
![Page 21: Light Field](https://reader036.vdocuments.net/reader036/viewer/2022062408/56813863550346895da011d1/html5/thumbnails/21.jpg)
Aperture Filters
![Page 22: Light Field](https://reader036.vdocuments.net/reader036/viewer/2022062408/56813863550346895da011d1/html5/thumbnails/22.jpg)
Variable Apertures
![Page 23: Light Field](https://reader036.vdocuments.net/reader036/viewer/2022062408/56813863550346895da011d1/html5/thumbnails/23.jpg)
Variable Apertures
![Page 24: Light Field](https://reader036.vdocuments.net/reader036/viewer/2022062408/56813863550346895da011d1/html5/thumbnails/24.jpg)
Marc Levoy
Stanford multi-camera array
• 640 × 480 pixels ×30 fps × 128 cameras
• synchronized timing
• continuous streaming
• flexible arrangement
![Page 25: Light Field](https://reader036.vdocuments.net/reader036/viewer/2022062408/56813863550346895da011d1/html5/thumbnails/25.jpg)
Marc Levoy
Ways to use large camera arrays
• widely spaced light field capture
• tightly packed high-performance imaging
• intermediate spacing synthetic aperture photography
![Page 26: Light Field](https://reader036.vdocuments.net/reader036/viewer/2022062408/56813863550346895da011d1/html5/thumbnails/26.jpg)
Marc Levoy
Intermediate camera spacing:synthetic aperture photography
![Page 27: Light Field](https://reader036.vdocuments.net/reader036/viewer/2022062408/56813863550346895da011d1/html5/thumbnails/27.jpg)
Marc Levoy
Example using 45 cameras[Vaish CVPR 2004]
![Page 28: Light Field](https://reader036.vdocuments.net/reader036/viewer/2022062408/56813863550346895da011d1/html5/thumbnails/28.jpg)
![Page 29: Light Field](https://reader036.vdocuments.net/reader036/viewer/2022062408/56813863550346895da011d1/html5/thumbnails/29.jpg)
Tiled camera array
• world’s largest video camera
• no parallax for distant objects
• poor lenses limit image quality
• seamless mosaicing isn’t hard
Can we match the image quality of a cinema camera?
![Page 30: Light Field](https://reader036.vdocuments.net/reader036/viewer/2022062408/56813863550346895da011d1/html5/thumbnails/30.jpg)
Tiled panoramic image(before geometric or color calibration)
![Page 31: Light Field](https://reader036.vdocuments.net/reader036/viewer/2022062408/56813863550346895da011d1/html5/thumbnails/31.jpg)
Tiled panoramic image(after calibration and blending)
![Page 32: Light Field](https://reader036.vdocuments.net/reader036/viewer/2022062408/56813863550346895da011d1/html5/thumbnails/32.jpg)
Tiled camera array
• world’s largest video camera
• no parallax for distant objects
• poor lenses limit image quality
• seamless mosaicing isn’t hard
• per-camera exposure metering
• HDR within and between tiles
Can we match the image quality of a cinema camera?
![Page 33: Light Field](https://reader036.vdocuments.net/reader036/viewer/2022062408/56813863550346895da011d1/html5/thumbnails/33.jpg)
same exposure in all cameras
individually metered
checkerboardof exposures
![Page 34: Light Field](https://reader036.vdocuments.net/reader036/viewer/2022062408/56813863550346895da011d1/html5/thumbnails/34.jpg)
Marc Levoy
High-performance photography as multi-dimensional sampling
• spatial resolution
• field of view
• frame rate
• dynamic range
• bits of precision
• depth of field
• focus setting
• color sensitivity
![Page 35: Light Field](https://reader036.vdocuments.net/reader036/viewer/2022062408/56813863550346895da011d1/html5/thumbnails/35.jpg)
Light field photography using a handheld plenoptic camera
Ren Ng, Marc Levoy, Mathieu Brédif,Gene Duval, Mark Horowitz and Pat Hanrahan
Stanford University
![Page 36: Light Field](https://reader036.vdocuments.net/reader036/viewer/2022062408/56813863550346895da011d1/html5/thumbnails/36.jpg)
Marc Levoy
What’s wrong with conventional cameras?
Aperture
![Page 37: Light Field](https://reader036.vdocuments.net/reader036/viewer/2022062408/56813863550346895da011d1/html5/thumbnails/37.jpg)
Marc Levoy
Capture the light field inside a camera
125*125 μm
500 microns
Aperture
![Page 38: Light Field](https://reader036.vdocuments.net/reader036/viewer/2022062408/56813863550346895da011d1/html5/thumbnails/38.jpg)
Marc Levoy
Conventional versus light field camera
uv-plane st-plane
![Page 39: Light Field](https://reader036.vdocuments.net/reader036/viewer/2022062408/56813863550346895da011d1/html5/thumbnails/39.jpg)
Marc Levoy
Conventional versus light field camera
uv-planest-plane
![Page 40: Light Field](https://reader036.vdocuments.net/reader036/viewer/2022062408/56813863550346895da011d1/html5/thumbnails/40.jpg)
Prototype camera
4000 × 4000 pixels ÷ 292 × 292 lenses = 14 × 14 pixels per lens
Contax medium format camera Kodak 16-megapixel sensor
Adaptive Optics microlens array 125μ square-sided microlenses
![Page 41: Light Field](https://reader036.vdocuments.net/reader036/viewer/2022062408/56813863550346895da011d1/html5/thumbnails/41.jpg)
Marc Levoy
Light Field in a Single Exposure
![Page 42: Light Field](https://reader036.vdocuments.net/reader036/viewer/2022062408/56813863550346895da011d1/html5/thumbnails/42.jpg)
Marc Levoy
Light Field in a Single Exposure
![Page 43: Light Field](https://reader036.vdocuments.net/reader036/viewer/2022062408/56813863550346895da011d1/html5/thumbnails/43.jpg)
Marc Levoy
Light field inside a camera body
![Page 44: Light Field](https://reader036.vdocuments.net/reader036/viewer/2022062408/56813863550346895da011d1/html5/thumbnails/44.jpg)
Marc Levoy
Digitally stopping-down
• stopping down = summing only the central portion of each microlens
Σ
Σ
![Page 45: Light Field](https://reader036.vdocuments.net/reader036/viewer/2022062408/56813863550346895da011d1/html5/thumbnails/45.jpg)
Marc Levoy
Digital refocusing
• refocusing = summing windows extracted from several microlenses
Σ
Σ
![Page 46: Light Field](https://reader036.vdocuments.net/reader036/viewer/2022062408/56813863550346895da011d1/html5/thumbnails/46.jpg)
Marc Levoy
Example of digital refocusing
![Page 47: Light Field](https://reader036.vdocuments.net/reader036/viewer/2022062408/56813863550346895da011d1/html5/thumbnails/47.jpg)
Marc Levoy
Refocusing portraits
![Page 48: Light Field](https://reader036.vdocuments.net/reader036/viewer/2022062408/56813863550346895da011d1/html5/thumbnails/48.jpg)
Marc Levoy
Action photography
![Page 49: Light Field](https://reader036.vdocuments.net/reader036/viewer/2022062408/56813863550346895da011d1/html5/thumbnails/49.jpg)
Extending the depth of field
conventional photograph,main lens at f / 22
conventional photograph,main lens at f / 4
![Page 50: Light Field](https://reader036.vdocuments.net/reader036/viewer/2022062408/56813863550346895da011d1/html5/thumbnails/50.jpg)
Marc Levoy
Scene-dependent focal plane
Σ
Depth from focus problem
Interactive solution [Agarwala 2004]
![Page 51: Light Field](https://reader036.vdocuments.net/reader036/viewer/2022062408/56813863550346895da011d1/html5/thumbnails/51.jpg)
Marc Levoy
Extending the depth of field
conventional photograph,main lens at f / 22
conventional photograph,main lens at f / 4
light field, main lens at f / 4,after all-focus algorithm
[Agarwala 2004]
![Page 52: Light Field](https://reader036.vdocuments.net/reader036/viewer/2022062408/56813863550346895da011d1/html5/thumbnails/52.jpg)
Marc Levoy
A digital refocusing theorem
• an f / N light field camera, with P × P pixels under each microlens, can produce views as sharp as an f / (N × P) conventional camera
• these views can be focused anywhere within the depth of field of the f / (N × P) camera
![Page 53: Light Field](https://reader036.vdocuments.net/reader036/viewer/2022062408/56813863550346895da011d1/html5/thumbnails/53.jpg)
Marc Levoy
Prior work
• integral photography– microlens array + film
– application is autostereoscopic effect
• [Adelson 1992]– proposed this camera
– built an optical bench prototype using relay lenses
– application was stereo vision, not photography
![Page 54: Light Field](https://reader036.vdocuments.net/reader036/viewer/2022062408/56813863550346895da011d1/html5/thumbnails/54.jpg)
Marc Levoy
Digitally moving the observer
• moving the observer = moving the window we extract from the microlenses
Σ
Σ
![Page 55: Light Field](https://reader036.vdocuments.net/reader036/viewer/2022062408/56813863550346895da011d1/html5/thumbnails/55.jpg)
Marc Levoy
Example of moving the observer
![Page 56: Light Field](https://reader036.vdocuments.net/reader036/viewer/2022062408/56813863550346895da011d1/html5/thumbnails/56.jpg)
Marc Levoy
Moving backward and forward
![Page 57: Light Field](https://reader036.vdocuments.net/reader036/viewer/2022062408/56813863550346895da011d1/html5/thumbnails/57.jpg)
Marc Levoy
Implications
• cuts the unwanted link between exposure(due to the aperture) and depth of field
• trades off (excess) spatial resolution for ability to refocus and adjust the perspective
• sensor pixels should be made even smaller, subject to the diffraction limit
36mm × 24mm ÷ 2.5μ pixels = 266 megapixels
20K × 13K pixels
4000 × 2666 pixels × 20 × 20 rays per pixel
• Application in microscope