thank you for the introduction
DESCRIPTION
Shading Direct reflection under distant illumination with no visibility and global illumination effects Rendering an image is computationally expensive. from Wikipedia This paper is about shading, that is computing the amount of illumination arriving at each visible surface point in a virtual scene. That involves computing an integral over all the incident directions for every image pixel. Rendering an image is therefore computationally expensive. Illumination BRDFTRANSCRIPT
![Page 1: Thank you for the introduction](https://reader036.vdocuments.net/reader036/viewer/2022062311/5a4d1b0f7f8b9ab05998dbd2/html5/thumbnails/1.jpg)
Mahdi M. Bagher / Cyril Soler / Kartic Subr / Laurent Belcour / Nicolas HolzschuchMahdi M. Bagher / Cyril Soler / Kartic Subr / Laurent Belcour / Nicolas Holzschuch
Interactive rendering of Interactive rendering of acquired materials on dynamic geometryacquired materials on dynamic geometry
using bandwidth predictionusing bandwidth prediction
![Page 2: Thank you for the introduction](https://reader036.vdocuments.net/reader036/viewer/2022062311/5a4d1b0f7f8b9ab05998dbd2/html5/thumbnails/2.jpg)
• Direct reflection under distant illumination with no visibility and global illumination effects
• Rendering an image is computationally expensive.
Shading
from
Wik
iped
ia
Illumination BRDF
![Page 3: Thank you for the introduction](https://reader036.vdocuments.net/reader036/viewer/2022062311/5a4d1b0f7f8b9ab05998dbd2/html5/thumbnails/3.jpg)
Acquired Materials
• Using measured materials• photo-realistic rendering• large memory footprint
(33MB for a single isotropic BRDF)
• Not suitable for real-time rendering
Phong shading Measured “Color-Changing-Paint-3”BRDF measurement Gantry [Matusik2002]
![Page 4: Thank you for the introduction](https://reader036.vdocuments.net/reader036/viewer/2022062311/5a4d1b0f7f8b9ab05998dbd2/html5/thumbnails/4.jpg)
Monte Carlo Sampling
• Monte Carlo approximation to the shading integral
• Importance sampling• less noisy
• Data driven reflectance• no analytical importance function• pre-computed Importance samples
Monte Carlo sampling
BRDF importance sampling
![Page 5: Thank you for the introduction](https://reader036.vdocuments.net/reader036/viewer/2022062311/5a4d1b0f7f8b9ab05998dbd2/html5/thumbnails/5.jpg)
Sampling
• Dealing with two types of sampling• Reconstruction• sampling in image space• to render an image
• Integration• sampling in the space of incident directions • to shade each individual pixel
Illumination BRDF Cosine
![Page 6: Thank you for the introduction](https://reader036.vdocuments.net/reader036/viewer/2022062311/5a4d1b0f7f8b9ab05998dbd2/html5/thumbnails/6.jpg)
The Problem
• Shading is slow…
• How to speed-up shading?
![Page 7: Thank you for the introduction](https://reader036.vdocuments.net/reader036/viewer/2022062311/5a4d1b0f7f8b9ab05998dbd2/html5/thumbnails/7.jpg)
Case Study
• What difference various materials make in rendering an image?
Measured reflectance data from [Matusik2002]
![Page 8: Thank you for the introduction](https://reader036.vdocuments.net/reader036/viewer/2022062311/5a4d1b0f7f8b9ab05998dbd2/html5/thumbnails/8.jpg)
Diffuse Materials
• Diffuse materials• shading varies slowly across the image (low frequencies)• many integration samples for each pixel (wide BRDF lobe)
Measured Teflon [Matusik2002] BRDF Lobe [BRDFLab]
![Page 9: Thank you for the introduction](https://reader036.vdocuments.net/reader036/viewer/2022062311/5a4d1b0f7f8b9ab05998dbd2/html5/thumbnails/9.jpg)
Specular Materials
• Specular materials• shading varies quickly across the image (high frequencies) • few integration samples for each pixel (narrow BRDF lobe)
Measured color-Changing-Paint3 [Matusik2002] BRDF Lobe [BRDFLab]
![Page 10: Thank you for the introduction](https://reader036.vdocuments.net/reader036/viewer/2022062311/5a4d1b0f7f8b9ab05998dbd2/html5/thumbnails/10.jpg)
The Problem
• Rendering is slow…• How to speed-up shading?
• Can we exploit the coherency between • Reconstruction samples…• Integration samples…
• …for adaptive sampling?• How to measure the coherency
between samples?
![Page 11: Thank you for the introduction](https://reader036.vdocuments.net/reader036/viewer/2022062311/5a4d1b0f7f8b9ab05998dbd2/html5/thumbnails/11.jpg)
Intuition
• Study the frequencies in the image• As a function of what happens to the light• Traveling from the light source to the camera
![Page 12: Thank you for the introduction](https://reader036.vdocuments.net/reader036/viewer/2022062311/5a4d1b0f7f8b9ab05998dbd2/html5/thumbnails/12.jpg)
The Traveling Light
• Light when traveling is affected by• Travel through free space• Object’s curvature• Reflectance• BRDF• Texture
• Occlusion
![Page 13: Thank you for the introduction](https://reader036.vdocuments.net/reader036/viewer/2022062311/5a4d1b0f7f8b9ab05998dbd2/html5/thumbnails/13.jpg)
Our Contribution
• How to speed-up shading?By sparse sampling the image• How to combine the sparse samples?• Multi-resolution rendering and bi-lateral up-sampling
By adaptive sampling for integration
![Page 14: Thank you for the introduction](https://reader036.vdocuments.net/reader036/viewer/2022062311/5a4d1b0f7f8b9ab05998dbd2/html5/thumbnails/14.jpg)
Our Contribution (Illustrated)
Sparse reconstruction samples Adaptive Integration samples
,
Final shaded image
Multi-resolution shading and bi-lateral up-sampling
∝ the average variance of the shading integrand∝ the local maximum variations in the image
![Page 15: Thank you for the introduction](https://reader036.vdocuments.net/reader036/viewer/2022062311/5a4d1b0f7f8b9ab05998dbd2/html5/thumbnails/15.jpg)
Assumptions
• Direct reflection under distant illumination
• No visibility and global illumination effects
• Allow dynamic editable geometry
• Screen-space rendering in the context of deferred-shading
• Any shading model is possible
• Save more if shading cost is higher e.g. measured materials
• BRDF importance sampling
Normal
![Page 16: Thank you for the introduction](https://reader036.vdocuments.net/reader036/viewer/2022062311/5a4d1b0f7f8b9ab05998dbd2/html5/thumbnails/16.jpg)
Dynamic Geometry Animation
QuickTime™ and a decompressor
are needed to see this picture.
![Page 17: Thank you for the introduction](https://reader036.vdocuments.net/reader036/viewer/2022062311/5a4d1b0f7f8b9ab05998dbd2/html5/thumbnails/17.jpg)
Roadmap
• Related work
• Our technique
• The goal
• Sparse sampling for reconstruction
• Adaptive sampling for integration
• Validation
• The rendering algorithm
• Results and comparisons
• Extension: MSAA
• Conclusion / Future work
![Page 18: Thank you for the introduction](https://reader036.vdocuments.net/reader036/viewer/2022062311/5a4d1b0f7f8b9ab05998dbd2/html5/thumbnails/18.jpg)
Related Work GPU Rendering of Complex Materials
[Heidrich&Seidel1999]
[Ramamoorthi2002]
[Kautz2002]
[Claustres2007]
[Kautz&McCool1999]
[Latta&Kolb2002][McCool2001]
[Wang2009]
![Page 19: Thank you for the introduction](https://reader036.vdocuments.net/reader036/viewer/2022062311/5a4d1b0f7f8b9ab05998dbd2/html5/thumbnails/19.jpg)
Related WorkMulti-Resolution Screen-Space Algorithms
[Nichols&Wyman2009] [Shopf2009][Nichols&Wyman2010]
[Soler2010]
[Nichols2010][Segovia2006]
![Page 20: Thank you for the introduction](https://reader036.vdocuments.net/reader036/viewer/2022062311/5a4d1b0f7f8b9ab05998dbd2/html5/thumbnails/20.jpg)
Related WorkPre-Computed Transport
[Sloan2002]
[Ramamoorthi2009]
[Sun2007]
[Liu2004]
[Loos2011]
[Loos2012]
![Page 21: Thank you for the introduction](https://reader036.vdocuments.net/reader036/viewer/2022062311/5a4d1b0f7f8b9ab05998dbd2/html5/thumbnails/21.jpg)
Roadmap
• Related work
• Our technique
• The goal
• Sparse sampling for reconstruction
• Adaptive sampling for integration
• Validation
• The rendering algorithm
• Results and comparisons
• Extension: MSAA
• Conclusion / Future work
![Page 22: Thank you for the introduction](https://reader036.vdocuments.net/reader036/viewer/2022062311/5a4d1b0f7f8b9ab05998dbd2/html5/thumbnails/22.jpg)
Goal
• Sparse sampling for reconstruction• ∝ maximum local variations (local bandwidth)
• Adaptive sampling for integration• ∝ variance of the shading integrand
• Multi-resolution rendering and bilateral up-sampling.• a single coarse-to-fine rendering pass
![Page 23: Thank you for the introduction](https://reader036.vdocuments.net/reader036/viewer/2022062311/5a4d1b0f7f8b9ab05998dbd2/html5/thumbnails/23.jpg)
Roadmap
• Related work
• Our technique
• The goal
• Sparse sampling for reconstruction
• Local light-field definition
• Spectral Operations
• 2D local bandwidth
• Image-space local bandwidth
• Adaptive sampling for integration
• Validation
• …
![Page 24: Thank you for the introduction](https://reader036.vdocuments.net/reader036/viewer/2022062311/5a4d1b0f7f8b9ab05998dbd2/html5/thumbnails/24.jpg)
Local Light-Field
• 4D local light-field illustrated in 2D
Central ray
Spac
e
Angle
![Page 25: Thank you for the introduction](https://reader036.vdocuments.net/reader036/viewer/2022062311/5a4d1b0f7f8b9ab05998dbd2/html5/thumbnails/25.jpg)
Roadmap
• Related work
• Our technique
• The goal
• Sparse sampling for reconstruction
• Local light-field definition
• Spectral Operations
• 2D local bandwidth
• Image-space local bandwidth
• Adaptive sampling for integration
• Validation
• …
![Page 26: Thank you for the introduction](https://reader036.vdocuments.net/reader036/viewer/2022062311/5a4d1b0f7f8b9ab05998dbd2/html5/thumbnails/26.jpg)
Spectral Operations
•Spectral operations [Durand et al. 2005][D
urand et al. 2005]
![Page 27: Thank you for the introduction](https://reader036.vdocuments.net/reader036/viewer/2022062311/5a4d1b0f7f8b9ab05998dbd2/html5/thumbnails/27.jpg)
Roadmap
• Related work
• Our technique
• The goal
• Sparse sampling for reconstruction
• Local light-field definition
• Spectral Operations
• 2D local bandwidth
• Image-space local bandwidth
• Adaptive sampling for integration
• Validation
• …
![Page 28: Thank you for the introduction](https://reader036.vdocuments.net/reader036/viewer/2022062311/5a4d1b0f7f8b9ab05998dbd2/html5/thumbnails/28.jpg)
2D Local Bandwidth
• 2D Local Bandwidth• maximum local variations about a light path both in angle and space.
spatial angular
2D local bandwidth
![Page 29: Thank you for the introduction](https://reader036.vdocuments.net/reader036/viewer/2022062311/5a4d1b0f7f8b9ab05998dbd2/html5/thumbnails/29.jpg)
Propagating Local Bandwidth
![Page 30: Thank you for the introduction](https://reader036.vdocuments.net/reader036/viewer/2022062311/5a4d1b0f7f8b9ab05998dbd2/html5/thumbnails/30.jpg)
Propagating Local Bandwidth
• One-bounce 2D bandwidth propagation• Transport (from light to reflector)• Reflection (BRDF & Texture)• Transport (from reflector to pixel)
• Matrix operators on 2D bandwidth
where
Bandwidthat pixel p
Reflectionoperator
Illumination bandwidth
Scale Curvature Mirror BRDF and texture
Curvature ScaleTransport to image plane
Transport to the surface
![Page 31: Thank you for the introduction](https://reader036.vdocuments.net/reader036/viewer/2022062311/5a4d1b0f7f8b9ab05998dbd2/html5/thumbnails/31.jpg)
Input Signals
• Illumination• environment map
• Reflectance• Acquired isotropic BRDFs
• TexturesGold-paint[ Matusik2003]
Grace cathedral [Debevec2001]
![Page 32: Thank you for the introduction](https://reader036.vdocuments.net/reader036/viewer/2022062311/5a4d1b0f7f8b9ab05998dbd2/html5/thumbnails/32.jpg)
Input Signals’ Bandwidth
• Illumination• environment map• Purely angular bandwidth
• Reflectance• Acquired isotropic BRDFs• Purely angular bandwidth
• Textures• Purely spatial bandwidth
Gold-paint[ Matusik2003]
Grace cathedral [Debevec2001]
![Page 33: Thank you for the introduction](https://reader036.vdocuments.net/reader036/viewer/2022062311/5a4d1b0f7f8b9ab05998dbd2/html5/thumbnails/33.jpg)
Computing Local Bandwidth
• Discrete Wavelet Transform to compute bandwidth• environment map and reflectance (BRDF & Texture)
• Discrete Wavelet Transform instead of WFFT • wavelets are well localized both in space and frequency.
Space Frequency
Daubechies 4 tap wavelet
FFT
![Page 34: Thank you for the introduction](https://reader036.vdocuments.net/reader036/viewer/2022062311/5a4d1b0f7f8b9ab05998dbd2/html5/thumbnails/34.jpg)
Local Bandwidth Examples
Gold-paint[ Matusik2003]
Angular bandwidth
Angular bandwidthGrace cathedral [Debevec2001]
![Page 35: Thank you for the introduction](https://reader036.vdocuments.net/reader036/viewer/2022062311/5a4d1b0f7f8b9ab05998dbd2/html5/thumbnails/35.jpg)
Roadmap
• Related work
• Our technique
• The goal
• Sparse sampling for reconstruction
• Local light-field definition
• Spectral Operations
• 2D local bandwidth
• Image-space local bandwidth
• Adaptive sampling for integration
• Validation
• …
![Page 36: Thank you for the introduction](https://reader036.vdocuments.net/reader036/viewer/2022062311/5a4d1b0f7f8b9ab05998dbd2/html5/thumbnails/36.jpg)
Image-Space Bandwidth
• We combine bandwidth from various incident directions to get the final bandwidth at a given pixel.
• Taking a max is too conservative, we take the weighted average instead.
• Bandwidth at each pixel
Bandwidth estimate based on max Bandwidth estimate Rendered image
![Page 37: Thank you for the introduction](https://reader036.vdocuments.net/reader036/viewer/2022062311/5a4d1b0f7f8b9ab05998dbd2/html5/thumbnails/37.jpg)
Roadmap
• Related work
• Our technique
• The goal
• Sparse sampling for reconstruction
• Adaptive sampling for integration
• Validation
• The rendering algorithm
• Results and comparisons
• Extension: MSAA
• Conclusion / Future work
![Page 38: Thank you for the introduction](https://reader036.vdocuments.net/reader036/viewer/2022062311/5a4d1b0f7f8b9ab05998dbd2/html5/thumbnails/38.jpg)
Variance of the Shading Integrand
• # shading samples ∝ sum of bandwidths, weighted by the illumination times reflectance squared.
• The sum is a conservative approximation of the actual variance of the shading integrand.
Angular bandwidth arriving at the surface
Angular bandwidth of the reflectance
![Page 39: Thank you for the introduction](https://reader036.vdocuments.net/reader036/viewer/2022062311/5a4d1b0f7f8b9ab05998dbd2/html5/thumbnails/39.jpg)
Roadmap
• Related work
• Our technique
• The goal
• Sparse sampling for reconstruction
• Adaptive sampling for integration
• Validation
• The rendering algorithm
• Results and comparisons
• Extension: MSAA
• Conclusion / Future work
![Page 40: Thank you for the introduction](https://reader036.vdocuments.net/reader036/viewer/2022062311/5a4d1b0f7f8b9ab05998dbd2/html5/thumbnails/40.jpg)
Validation of the Bandwidth and Variance
Estimated variance Measured varianceEstimated bandwidth Measured bandwidth(WFFT)
![Page 41: Thank you for the introduction](https://reader036.vdocuments.net/reader036/viewer/2022062311/5a4d1b0f7f8b9ab05998dbd2/html5/thumbnails/41.jpg)
Roadmap
• Related work
• Our technique
• The goal
• Sparse sampling for reconstruction
• Adaptive sampling for integration
• Validation
• The rendering algorithm
• Results and comparisons
• Extension: MSAA
• Conclusion / Future work
![Page 42: Thank you for the introduction](https://reader036.vdocuments.net/reader036/viewer/2022062311/5a4d1b0f7f8b9ab05998dbd2/html5/thumbnails/42.jpg)
The Algorithm
• Step 1• Compute g-buffers• Load Illumination and reflectance bandwidth
• Step 2• Estimate bandwidth and variance
• Step 3• Mip-map bandwidth and variance
• Step 4• Shade and up-sample
![Page 43: Thank you for the introduction](https://reader036.vdocuments.net/reader036/viewer/2022062311/5a4d1b0f7f8b9ab05998dbd2/html5/thumbnails/43.jpg)
Video: algorithm
QuickTime™ and a decompressor
are needed to see this picture.
![Page 44: Thank you for the introduction](https://reader036.vdocuments.net/reader036/viewer/2022062311/5a4d1b0f7f8b9ab05998dbd2/html5/thumbnails/44.jpg)
Roadmap
• Related work
• Our technique
• The goal
• Sparse sampling for reconstruction
• Adaptive sampling for integration
• Validation
• The rendering algorithm
• Results and comparisons
• Extension: MSAA
• Conclusion / Future work
![Page 45: Thank you for the introduction](https://reader036.vdocuments.net/reader036/viewer/2022062311/5a4d1b0f7f8b9ab05998dbd2/html5/thumbnails/45.jpg)
Result (Final Image)
![Page 46: Thank you for the introduction](https://reader036.vdocuments.net/reader036/viewer/2022062311/5a4d1b0f7f8b9ab05998dbd2/html5/thumbnails/46.jpg)
Result (Bandwidth)
![Page 47: Thank you for the introduction](https://reader036.vdocuments.net/reader036/viewer/2022062311/5a4d1b0f7f8b9ab05998dbd2/html5/thumbnails/47.jpg)
Result (Variance)
![Page 48: Thank you for the introduction](https://reader036.vdocuments.net/reader036/viewer/2022062311/5a4d1b0f7f8b9ab05998dbd2/html5/thumbnails/48.jpg)
Timings
• Fast bandwidth/variance estimation• (8 ms)
(ms)(ms)
![Page 49: Thank you for the introduction](https://reader036.vdocuments.net/reader036/viewer/2022062311/5a4d1b0f7f8b9ab05998dbd2/html5/thumbnails/49.jpg)
Timings
• Computation time scales linearly with the total number of shading samples
![Page 50: Thank you for the introduction](https://reader036.vdocuments.net/reader036/viewer/2022062311/5a4d1b0f7f8b9ab05998dbd2/html5/thumbnails/50.jpg)
Comparison with ReferenceOur algorithm
(1015 ms)Similar time
(906 ms)Similar quality
(2639 ms)
![Page 51: Thank you for the introduction](https://reader036.vdocuments.net/reader036/viewer/2022062311/5a4d1b0f7f8b9ab05998dbd2/html5/thumbnails/51.jpg)
Comparison with Spherical Gaussian Approximation (Wang et al. 2009)
Fast but different from ground-truth (notice that color changing effects are missing)
Path-traced reference
Interactive but more accurate
![Page 52: Thank you for the introduction](https://reader036.vdocuments.net/reader036/viewer/2022062311/5a4d1b0f7f8b9ab05998dbd2/html5/thumbnails/52.jpg)
Roadmap
• Related work
• Our technique
• The goal
• Sparse sampling for reconstruction
• Adaptive sampling for integration
• Validation
• The rendering algorithm
• Results and comparisons
• Extension: MSAA
• Conclusion / Future work
![Page 53: Thank you for the introduction](https://reader036.vdocuments.net/reader036/viewer/2022062311/5a4d1b0f7f8b9ab05998dbd2/html5/thumbnails/53.jpg)
ExtensionAdaptive Multi-Sample Anti-Aliasing
![Page 54: Thank you for the introduction](https://reader036.vdocuments.net/reader036/viewer/2022062311/5a4d1b0f7f8b9ab05998dbd2/html5/thumbnails/54.jpg)
Video: adaptive MSAA
QuickTime™ and a decompressor
are needed to see this picture.
![Page 55: Thank you for the introduction](https://reader036.vdocuments.net/reader036/viewer/2022062311/5a4d1b0f7f8b9ab05998dbd2/html5/thumbnails/55.jpg)
Timings
• Sub-linear performance for MSAA
![Page 56: Thank you for the introduction](https://reader036.vdocuments.net/reader036/viewer/2022062311/5a4d1b0f7f8b9ab05998dbd2/html5/thumbnails/56.jpg)
Roadmap
• Related work
• Our technique
• The goal
• Sparse sampling for reconstruction
• Adaptive sampling for integration
• Validation
• The rendering algorithm
• Results and comparisons
• Extension: MSAA
• Conclusion / Future work
![Page 57: Thank you for the introduction](https://reader036.vdocuments.net/reader036/viewer/2022062311/5a4d1b0f7f8b9ab05998dbd2/html5/thumbnails/57.jpg)
Conclusion
• Interactively shading dynamic geometry with acquired materials• Shade a fraction of pixels using bandwidth• Adaptively sample shading integrals• Combine shaded pixels using up-sampling
• Exploiting bandwidth to sub-linearly scale MSAA with deferred shading
![Page 58: Thank you for the introduction](https://reader036.vdocuments.net/reader036/viewer/2022062311/5a4d1b0f7f8b9ab05998dbd2/html5/thumbnails/58.jpg)
Future Work
• Local light sources
• Better up-sampling algorithm
• Depth-of-field
• 6D bandwidth computation for SVBRDFs
• Visibility and indirect illumination
![Page 59: Thank you for the introduction](https://reader036.vdocuments.net/reader036/viewer/2022062311/5a4d1b0f7f8b9ab05998dbd2/html5/thumbnails/59.jpg)
Thank you
Interactive rendering of Interactive rendering of acquired materials on dynamic geometryacquired materials on dynamic geometry
using bandwidth predictionusing bandwidth prediction