computational fundamentals of reflection coms 6998-3, lecture 7 0 01 2
Post on 19-Dec-2015
216 views
TRANSCRIPT
Computational Fundamentals of Computational Fundamentals of ReflectionReflection
COMS 6998-3, Lecture 7
lA
2 / 3
/ 4
0l0 1 2
MotivationMotivation
Understand intrinsic computational structure of reflection and illumination
Necessary for many applications in computer graphics (cannot solve by brute force!!)• Real-time forward rendering• IBR sampling rates, dimensionality explosion• Inverse rendering and inverse problems in general• Computer vision: complex lighting, materials
Real-Time Rendering DemoReal-Time Rendering Demo
Motivation: Interactive rendering with complex natural illumination and realistic, measured BRDFs
QuestionsQuestions
• Images are view-dependent (4D quantity)
• Can we find low-dimensional structure to capture view-dependence?
Space of images as lighting variesSpace of images as lighting varies
Illuminate subject from many incident directions
Example ImagesExample Images
Images from Debevec et al. 00
Principal Component AnalysisPrincipal Component Analysis• Try to approximate with low dimensional subspace
• Linear combination of few principal components
= .5 + .5 + …
= .7 + .3 + …
Principal component imagesPrincipal component images
Lighting VariabilityLighting Variability
TheoryInfinite number of light directions, one coefficient/directionSpace of images infinite dimensional [Belhumeur 98]
Empirical [Hallinan 94, Epstein 95]Diffuse objects: 5D subspace suffices
No satisfactory theoretical explanation of observations
Complex Light TransportComplex Light Transport
• Shadows high frequency
• Analysis possible?
• Low-dim. structure?
• Real-time complex lights?
Agrawala, Ramamoorthi, Heirich, Moll SIGGRAPH 00
ChallengesChallenges
• Illumination complexity
• Material (BRDF)/view complexity
• Transport complexity (shadows, interreflection)
Fundamental questions• Theoretical analysis of intrinsic complexity• Sampling rates and resolutions• Efficient practical algorithms
OutlineOutline
• Lighting variability in appearance [PAMI Oct, 2002]
• View variability real-time rendering [SIGGRAPH 02]
• Visibility/shadows [In Progress]
Lighting variability analysisLighting variability analysis
1. Frequency space analytic PCA construction
2. Mathematical derivation of principal components
3. Explain empirical results quantitatively• Dimensionality of approximating subspace• Forms of principal components• Relative importance of principal components
AssumptionsAssumptions
• Single view of single object
• Lambertian
• Distant illumination
• Discount texture
• Discount concavities: interreflection, cast shadows
Consider attached shadows (backfacing normals)
DefinitionsDefinitions
max( ,0) max(cos ,0)E L N Irradiance(image)
Radiance(light)
Normal Lambertian = half-cosine
Previous Theoretical WorkPrevious Theoretical Work
• Discount attached shadows [Shashua 97, …]
Resulting 3D subspace does not fully explain data
• Analytic PCA (without shadows) [Zhao & Yang 99]
cosE L N
max( ,0) max(cos ,0)E L N Irradiance(image)
Radiance(light)
Normal Lambertian = half-cosine
Spherical HarmonicsSpherical Harmonics
-1-2 0 1 2
0
1
2
.
.
.
( , )lmY
xy z
xy yz 23 1z zx 2 2x y
l
m
1
Spherical Harmonic ExpansionSpherical Harmonic Expansion
Expand lighting (L), irradiance (E) in basis functions
0
( , ) ( , )l
lm lml m l
L L Y
0
( , ) ( , )l
lm lml m l
E E Y
= .67 + .36 + …
Lambertian BRDF Expansion Lambertian BRDF Expansion
10
0
max(cos ,0) ( )l l ll
AY
-1-2 0 1 2
0
1
2
( , )lmY
xy z
xy yz 23 1z zx 2 2x y
l
m
1
Lambertian coefficients
0.89 1.02 0.50z 23 1z 1
Analytic Irradiance FormulaAnalytic Irradiance Formula
Lambertian surface acts like low-pass filter
lm l lmE A LlA
2 / 3
/ 4
0
2 1
2
2
( 1) !2
( 2)( 1) 2 !
l
l l l
lA l even
l l
l0 1 2
Basri & Jacobs 01Ramamoorthi & Hanrahan 01
9 Parameter Approximation9 Parameter Approximation
Exact imageOrder 29 terms
RMS Error = 1%
For any illumination, average error < 2% [Basri Jacobs 01]
-1-2 0 1 2
0
1
2
( , )lmY
xy z
xy yz 23 1z zx 2 2x y
l
m
1
Open QuestionsOpen Questions
• Relationship between spherical harmonics, PCA
• 9D approximation > 5D empirical subspace
Key insight: Consider approximations over visible normals (upper hemisphere), not entire sphere
Intuition: Backwards Half-CosineIntuition: Backwards Half-Cosine
00 10 20max( cos ,0) 0.89 1.02 0.50Y Y Y
-1-2 0 1 2
0
1
2
( , )lmY
xy z
xy yz 23 1z zx 2 2x y
l
m
1
Front 0
1 parameter irrelevant8 or fewer params
enough
Back cos
Intuition: dimensionality reductionIntuition: dimensionality reduction
Start with 9D space, remove dimensions• Mean (constant term) subtracted • Backwards half-cosine• x, xz very similar • y, yz very similar
Left with 5D subspace
11 21( , )Y Y
1 1 2 1( , )Y Y
-1-2 0 1 2
0
1
2
( , )lmY
xy z
xy yz 23 1z zx 2 2x y
l
m
1
Results: Image of a SphereResults: Image of a Sphere
• Principal components (eigenvectors) mix (linear combinations of) spherical harmonics
• Results agree with experiment [Epstein 95]• We predict: 3 eigenvectors = 91% variance, 5 give 96%• Empirical : 3 eigenvectors = 94% variance, 5 give 98%
% VAF(eigenvalue)
43% 24% 24% 2% 2%
Results: Human FaceResults: Human Face
• Numerically compute orthogonality matrix
• Specific distribution of surface normals important• Symmetries in sphere broken (faces are elongated)• Principal components somewhat different from sphere
% VAF 42% 33% 16% 4% 2%
Results: Human FaceResults: Human Face
• Prediction: Principal components have specific forms
• Empirical : [Hallinan 94]Frontal lighting, side, above/below, extreme side, corner
Frontal% VAF 42% 33% 16% 4% 2%
Side Above/Below Extreme side Corner
Results: Human FaceResults: Human Face
• Prediction: Space is close to 5D3 principal components = 91% variance, 5 components =
97%
• Empirical : [Epstein 95]3 principal components = 90% variance, 5 components =
94%
Frontal% VAF 42% 33% 16% 4% 2%
Side Above/Below Extreme side Corner
Results: Human FaceResults: Human Face
• Prediction: groups of principal components• Group 1: first two (frontal and side)• Group 2: next three [with above/below always 3rd]
• Empirical: [Hallinan 94]• Two groups [first two (frontal,side) and next three]• Within group, %VAF close, may exchange places
Frontal% VAF 42% 33% 16% 4% 2%
Side Above/Below Extreme side Corner
Summary: Lighting AnalysisSummary: Lighting Analysis
• Analytic PCA construction with attached shadowsSpherical harmonic analysis: Orthogonality matrix
• Mathematically derive principal components
• Qualitative, quantitative agreement with experiment
• Extend 9D Lambertian model to single view case
Implications Lighting AnalysisImplications Lighting Analysis
• Attached shadows nearly free: 5D subspace enough
• Mathematical derivation of principal components• Basis functions for subspace methods for recognition,…• Graphics applications: Image-Based, inverse rendering
• Complex illumination in computer vision
OutlineOutline
• Lighting variability in appearance [PAMI Oct, 2002]
• View variability real-time rendering [SIGGRAPH 02]
• Visibility/shadows [In Progress]
Reflection EquationReflection Equation
( )L R N l
2D Environment Map
Reflection EquationReflection Equation
( )L R N l
2D Environment Map
NL
,l V
BRDF
Reflection EquationReflection Equation
4D Orientation Light Field
2D Environment Map
Previous Work: Blinn & Newell 76, Miller & Hoffman 84,
Greene 86, Kautz & McCool 99, Cabral et al. 99, …
( , ) ( ) ,B N V L R N l l V dl
BRDF
NL
GoalsGoals
• Efficiently precompute and represent OLF
• Real-time rendering with OLF
QuestionsQuestions
• Parameterization and structure of OLF
• Structure leads to representation
• Computation and rendering of OLF
OLF ParameterizationOLF Parameterization
N LN
V
( , )B N V
OLF ParameterizationOLF Parameterization
N
V
( , )B N V
N
V
( , )B R V
RReparameterize
by reflection vector
OLF ParameterizationOLF Parameterization
• Captures structure of BRDF (and hence OLF) better
• Reflective BRDFs become low-dimensional
N
V
( , )B N V
N
V
( , )B R V
RReparameterize
by reflection vector
OLF StructureOLF Structure
( , )B R V
( )VB R
( )RB V
2D view array of reflection maps
2D image arrayof view maps
OLF Structure: PhongOLF Structure: Phong
( , ) ( )B R V B R
( )VB R
( )RB V
2D view array of reflection maps
2D image arrayof view maps
Environment Map Phong Reflection Map(blurred environment map)
Same reflection map for all views
OLF Structure: PhongOLF Structure: Phong
( , ) ( )B R V B R
( )VB R
( )RB V
Same reflection map for all views
Viewx
Vie
wy
OLF Structure: PhongOLF Structure: Phong
( , ) ( )B R V B R
( )VB R
( )RB V
Same reflection map for all views View maps constant for each R
Viewx
Vie
wy
OLF Structure: PhongOLF Structure: Phong
( , ) ( )B R V B R
( )VB R
( )RB V
Same reflection map for all views View maps constant for each R
Viewx
Vie
wy
Reflectionx
Ref
lect
ion
y
OLF Structure: LafortuneOLF Structure: Lafortune
( )VB R
Vie
wy
• Single 2D reflection map no longer sufficient
• But variation with viewing direction is slow
Viewx
OLF Structure: LafortuneOLF Structure: Lafortune
( )VB R
( )RB V
View maps vary slowly
Reflectionx
Ref
lect
ion
y
Vie
wy
Viewx
A Simple FactorizationA Simple Factorization
( )VB R
( )RB V
Viewx
Vie
wy
Ref
lect
ion
yReflectionx
( , ) ( ) * ( )B R V f R g V
*
Spherical Harmonic Reflection MapSpherical Harmonic Reflection Map
• View-dependent reflection (cube)map
• Encode view maps with low-order spherical harmonics
( )RB V
PrefilteringPrefiltering
• Directly compute SHRM from Lighting, BRDF
• Convolution easier to compute in frequency domain
,,() iijijiiji LLBLBR
Input Lighting and BRDF
Spherical Harmonic coeffs.
Convolution SHRM
PrefilteringPrefiltering
• 3 to 4 orders of magnitude faster (< 1 s compared to minutes or hours)
• Detailed analysis, algorithms, experiments in paper
,,() iijijiiji LLBLBR
Input Lighting and BRDF
Spherical Harmonic coeffs.
Convolution SHRM
Number of terms: CURETNumber of terms: CURET
Analysis for all 61 samples [full bar chart in paper]• For essentially all materials, 9-16 terms in SHRM suffice
DemoDemo
Summary view variabilitySummary view variability
• Theoretical, empirical analysis of sampling rates and resolutions• Frequency space analysis directly on lighting, BRDF• Low order expansion suffices for essentially all BRDFs
• Spherical Harmonic Reflection Maps• Hybrid angular-frequency space• Compact, efficient, accurate• Easy to analyze errors, determine number of terms
• Fast computation using convolution
ImplicationsImplications
• Frequency space methods for rendering • Global illumination• Fast computation of surface light fields
• Compression for optimal factored representations• PCA on SHRMs
• Theoretical analysis of sampling rates, resolutions• General framework for sampling in image-based rendering
OutlineOutline
• Lighting variability in appearance [PAMI Oct, 2002]
• View variability real-time rendering [SIGGRAPH 02]
• Visibility/shadows [In Progress]
Visibility complexity (high freq)Visibility complexity (high freq)
But Sparse (< 4%)But Sparse (< 4%)
Questions on VisibilityQuestions on Visibility
• Theory• Locally low-dimensional subspaces?• Intrinsic complexity of binary function?
• Practical• Real-time rendering with complex soft shadows, changing
illumination for lighting design, simulation• Efficient encoding/decoding (wavelets, PCA, dictionaries,
hierarchical?)
• In progress….
Overall SummaryOverall Summary
Many applications in graphics cannot be solved by brute force• Real-time rendering• IBR sampling rates, dimensionality explosion• Inverse rendering, inverse problems• Computer vision: complex lighting, materials
Need fundamental understanding of nature of reflection/lighting• Illumination complexity• Material (view) complexity• Transport complexity
Overall SummaryOverall Summary
Theoretical analysis tools• Signal processing, sampling theory• Low-dimensional subspaces• Information theory, information-based complexity?
Practical algorithms• Real-time rendering with complex lights,view, transport?• Lighting, Material design?• Exploit theoretical analysis (sampling rates,
forward/inverse duality, angular/frequency/sparsity duality, subspace results, differential analysis, perception)