1

All-Frequency Rendering of Dynamic, Spatially-Varying Reflectance

Jiaping Wang, Peiran Ren, Minmin Gong,John Snyder, Baining Guo

SIGGRAPH Asia 2009

Presenter: KevinApril 14, 2010

2

Authors

Jiaping Wang [1]

Peiran Ren [2]

Minmin Gong [1]

John Snyder [3]

Baining Guo [2]

Microsoft Research Asia [1]

Tsinghua University [2]

Microsoft Research [3]

3

Introduction

4

Introduction• Final goal of real-time realistic rendering– Dynamic lighting– Changeable viewpoint– All-frequency effects– Dynamic, editable, and spatially-varying materials– Dynamic, deformable scenes

Ng et al. SIGGRAPH ‘04 Wang et al. SIGGRAPH Asia ‘09 Sloan et al. SIGGRAPH ‘05

5

Introduction-2

• Challenges– BRDF complexity

• Modeling the complex reflectance of real world materials

– Light integration complexity• Integration over the whole hemisphere (cannot afford especially

when environment maps are used)

– Light transport complexity• Interreflection, shadows, …etc

6

Introduction-2

Off-lineHigh Quality Real-time

Real-timeRay-tracing

ProgrammableGraphics

Hardware

PrecomputedRadiance Transfer

Wang et al. SIGGRAPH ‘09 Dachsbacher et al. SIGGRAPH ‘07Ritschel et al. SIGGRAPH Asia ‘08

7

Introduction-3

• Precomputed Radiance Transfer (PRT)– The term comes from [Sloan ’02]– Precompute “light transport function”

– Compress by basis (SH, Wavelet, SRBF…)– The computation of rendering equation reduces to

inner/vector products in the run time

iioiiio dxnxVLxL

)]0),(max(),()[,()(),(4

8

Introduction-4

• PRT timeline (02~05)

02 03 04 05

Sloan et al. SIGGRAPH ‘02

Ng et al. SIGGRAPH ‘03

Sloan et al. SIGGRAPH ‘03

Ng et al. SIGGRAPH ‘04

Wang et al. EGSR ‘04

Lui et al. EGSR ‘04

Sloan et al. SIGGRAPH ‘05

Zhou et al. SIGGRAPH ‘05

Pioneer, SHCPCA

Wavelet

Triple Product

In-OutFac.

DynamicScenes

9

Introduction-5

• PRT timeline (06~09)

07 08 09

Wang et al. ACMTOG ‘06

Green et al. I3D ‘06 Green et al. EGSR ‘07

Sun et al. SIGGRAPH ‘07

Ben-Artzi et al. ACMTOG ‘08

Ramamoorthi CG&V ‘09

Wang et al. SIGGRAPH Asia ‘09

06

Xu et al. TVCG ‘08

Ben-Artzi et al. SIGGRAPH ‘06

Tsai et al. SIGGRAPH ‘06

SRBF

BRDF-Editing

CTA

SVBRDF, BRDF-Editing

SurveyBRDF-EditingWith G.I.

BRDF-EditingWith G.I.

10

Introduction-6

• Summary– Compression• PCA, Clustered PCA (CPCA), Clustered Tensor

Approximation (CTA) …

– Basis• Spherical harmonics (SH)• Wavelets• Zonal harmonics (ZH)• Spherical Radial Basis Function (SRBF)

11

Introduction-7

• Summary (cont.)– How to choose good basis for representation?• Can model all-frequency effects• Rotational invariant• Accuracy• Compact

Fit clamped cosine term to basisWang et al. SIGGRAPH Asia ‘09 – supplement materials

12

Introduction-8

• Spherical Gaussian (SG)– A type of SRBF, symmetric around a specific lobe

axis

– All advantages in the previous page– Inner product & cross product can be efficiently

computed

)1(),,;( pvepvG lobe axis

lobe sharpnesslobe amplitude

13

Introduction-9

• SG mixtures

n

iiiipvGvF

1

* ),,;()(

n

iiiipvGvF

1

* ),,;()( RRRotated version

Single Lobe Two Lobes Multiple Lobes

14

Overview

15

This Paper• Real-time• Dynamic (editable, change with time),

spatially-varying BRDFs• All-frequency effects from both environmental

and local point lights

• Static scenes• No interreflection

17

Contributions• Propose two new representations for BRDFs

and Visibility to compact the size of data– SG mixtures for microfacet-based reflectance• Accurate and compact• Parametric BRDFs can be fit on-the-fly• Fast rotation, warping, and products in run time

– SSDFs for visibility• Ghost-free, per-pixel interpolation

• Dynamic local point lights

18

• Decoupling BRDF from visibility

Approach Overview

6D SVBRDF

Represent

4D NDF(Microfacet)

Visibility(binary)

SSDF(Spherical Signed Distance Functions)

Fit into SGs

Map PCA

Mixture of SGs(Spherical Gaussians)

Eigen-Vectors

19

SVBRDF Representation

20

SVBRDF Representation• Microfacet Model

• Why use microfacet model?– General– Compact

oi

hioihoi

FGD

coscos4

),(),()(),(

Fresnel Term

Geometry Term

NormalDistribution

Function

21

SVBRDF Representation-2

• Reflectance representation using SGs– Parametric BRDFs (on-the-fly fitting)

– Example: Cook-Torrance Model

)()(),( hDiMio oremaining factor

(shadowing+Fresnel)NDF

Very Smooth High-Frequency Fit into SGs

))((),(),()(

oninioSioFiM CTCT

o

)1,/2,;()( 2)/)(arccos( 2

mnhGehD mnh

22

SVBRDF Representation-3

ground truth single-lobe SG(this paper)

256-term 64-term 16-termBRDF factorization

Cook-Torrancem=0.1

Cook-Torrancem=0.045

23

SVBRDF Representation-4

ground truth 7-lobe SG(this paper)

256-term 64-termBRDF factorization

Ashikhmin-Shirleynu=8,nv=128

Ashikhmin-Shirleynu=25,nv=400

Ashikhmin-Shirleynu=75,nv=1200

24

SVBRDF Representation-5

• Accuracy– Parametric isotropic models

25

SVBRDF Representation-6

• Accuracy (cont.)– Parametric anisotropic models

26

SVBRDF Representation-7

• Reflectance representation using SGs– Measured BRDFs (preprocessing)• Using tabulated NDF [Wang et al. SIGGRAPH ‘08] and

shadowing factor S at each surface point• Compress shadowing function by PCA (8D)• Fit NDF with a small number of SGs

27

SVBRDF Representation-8fabricgreen

delrin steelphenolicyellow

albmbronze

violetacrylic

groundtruth

1SG

2SG

3SG

256-TermFac.

64-TermFac.

28

SVBRDF Representation-9

• Accuracy– Measured BRDFs

29

Visibility Representation

30

Visibility Representation• Spatially-varying visibility is represented with

Spherical signed distance function (SSDF)– Directly interpolate binary visibilities will produce

ghost effects

– SSDF maps binary visibility to continuous function

31

Visibility Representation-2

• SSDF mapping– Positive: visible; negative: occluded– Value: the angular distance to the nearest direction

t on the shadow boundary

)(iV;1)( if ),arccos( min

0)(

iVit

tV

.0)( if ),arccos( min 1)(

iVittV

32

Visibility Representation-3

)(' iV

Reconstructed Visibility

d-2

(i) if ,1

otherwise ,0δ: elevation angle

Inner product / vector product of SGs and V’(i) can be efficiently evaluated in the run time!

33

Visibility Representation-3

• Compression– Using PCA

• PCA coefficients are stored as vertex attributes and Interpolated to each pixel during rasterization• Eigenvectors are encoded in multiple textures

Nv

j

Vjxjx wiViV

1,)()(

PCA coefficientsPCA eigenvectors

34

Visibility Representation-4

• Compression results

Ray-Traced Uncompressed SSDF SSDF/PCA 384 Terms

SSDF/PCA 144 Terms SSDF/PCA 48 Terms SSDF/PCA 16 Terms

35

Lighting Representation

36

Lighting Representation• Local point lights– Approximated with a single-lobe SG– Yielding a spatially-varying radiance field

• Infinitely-distant light from direction I

)||||

),||||

2(,||||

;()( 22

21*

xls

xlrf

xlxliGiL ax

l: 3D light position

s: intensity

)),2(,;()( 21* srfIiGiL a

37

Lighting Representation-2

• Distant environmental lighting– Apply to diffuse component• Fit the environment map with a SG mixture • [Tsai. et al. SIGGRAPH 2006]

– Apply to specular component• Preconvolve environmental radiance with SG kernels• The run-time inner product is reduced to a MIPMAP

texture fetch• [Kautz et al. EGWR 2000], [McAllister et al. GH 2002],

[Green et al. I3D 2006]

38

Run-Time Rendering

39

Run-Time Rendering• Per-vertex attributes– Position, texture coordinates, local coordinate

frame, PCA coefficient for SSDF

• BRDF parameters, tabulated SG lobes (for measured BRDFs), and PCA-compressed shadow factors are stored in textures

interploated across triangle mesh when passing from vertex shader to pixel shader

40

Run-Time Rendering-2

)()( oRkRkoR ssdd

2

),0max()()(Sd diniiViLR

2),0max()(),()()(

S ss diniiVioiLoR

170.1 ,133.2 ,),,;();(* ccccxx niGniC CosineApproximation

LightingApproximation

)(* iL

BRDFApproximation

VisibilityApproximation

)())();(( ** iViLniCR dxxd )())(),();(()( **

,* iLiVoiniCoR d

xxsxs

SGs for NDFSpherical Warp

)()( ** iWhD Multiplied remaining factor

)()();( ** iWiMoi os

PCA coefficients of SSDFs

Uncompressedon GPU

41

Experimental Results

42

Experimental Results• Data size and precomputation time

43

Experimental Results-2

• Per-vertex vs. per-pixel

wireframe per-vertex shading per-pixel shading

44

Experimental Results-3

• Distant environmental light + nearby point light

45

Experimental Results-4

• Results for isotropic BRDFs

46

Experimental Results-5

• Results for anisotropic BRDFs

47

Experimental Results-6

• Video

48

Conclusion• Solution for highly-specular, spatially varying,

dynamic materials, natural lighting, changeable viewpoints realistic rendering– SG mixtures for microfacet-based reflectance• Accurate and compact• Fast rotation, warping, and products in run time

– SSDFs for visibility• Ghost-free, per-pixel interpolation• Allow sparse set of per-vertex visibility samples

49

End

Thanks for yourAttention

50

[Sloan et al. 02]• Precomputed Radiance Transfer for Real-Time

Rendering in Dynamic, Low-Frequency Lighting Environment– Sloan, Kautz, and Snyder– SIGGRAPH 2002

51

[Sloan et al. 02]-2

• Algorithm (diffuse)– Express lighting as SH

– Reflection Equation becomes

– Define transfer function and project to SH

– Rendering reduces to

iiiijj jo dnxVYxL

)0,max(),()()(4

)()(2)1*(

1 il

j jji YL

iiiijj dnxVYxT

)0,max(),()()(4

)()(1 ij jjo TxL

Transfer vector (diffuse)Transfer Matrix (glossy)

52

[Ng et al. 03]• All-Frequency Shadows Using Non-linear

Wavelet Lighting Approximation– Ng, Ramamoorthi, Hanrahan– SIGGRAPH 2003

53

[Ng et al. 03]-2

• Relighting: Matrix-Vector Multiplication

)0),(max())(,(),(),( xnxxVxT io

)(),()( ii io LxTxL

TLLo

=Lo Li

Column:Lighting direction

Row:Pixel or Vertex

54

[Ng et al. 03]-3

• The dimension of T is too large– 300000 x 25000 for a 512x512 image and 6 x 64 x

64 environment mapApproximate by basis fitting(How to preserve high-frequency shadow?)

Approach:Approximate lighting dynamically at run-time using wavelet

100~200 basis coefficients are enough to generate high-quality results (compared to 20000 with SH)

55

[Ng et al. 04]• Triple Product Wavelet Integrals for All-

Frequency Relighting– Ng, Ramamoorthi, Hanrahan– SIGGRAPH 2004

56

[Ng et al. 04]-2

• Changeable view makes a 6D function• Factor the BRDF and visibility, reduce 6D

function into two 4D functions– For each spatial location and outgoing direction,

store 3 functions in cubemaps (Light, Visibility, BRDF)

– Calculate triple-product in run time

j k l lkjjkl

iilikijj k l lkj

il ilolk ikkj ijjo

VLC

dVL

dxnxVLxB

)()()(

)()),(()()()(),(

4

4

57

[Ng et al. 04]-3

58

[Wang et al. 04]• All-Frequency Relighting of Non-Diffuse

Objects using Separable BRDF Approximation– Rui Wang, John Tran, and David Luebke– EGSR 2004

59

[Wang et al. 04]-2

• Factor the BRDF into terms which depend only on incident / outgoing angles

• We can then define and precompute a view-independent transport function

• Thus, rendering is reduced to

K

k ikokkoi gh1

)()(),(

)0),(max()(),(),( xngxVxT iikiik

iK

k ikiokko dxTLhxB

14

),()()(),(

60

[Wang et al. 04]-3

• Comparison– Triple product• Pros: support true all-frequency effects• Cons: performance

– BRDF in-out factorization• Pros: speed and simple• Cons: k can not be large, make it only suit for glossy or

broad specular lobes

61

[Tsai et al. 06]• All-Frequency Precomputed Radiance Transfer

using Spherical Radial Basis Functions and Clustered Tensor Approximation– Yu-Ting Tsai and Zen-Chung Shih– SIGGRAPH 2006

62

[Tsai et al. 06]-2

63

[Ben Artzi et al. 06]• Real-Time BRDF Editing in Complex Lighting– Ben-Artzi, Overbeck, Ramamoorthi– SIGGRAPH 2006

64

[Ben Artzi et al. 06]-2

• Fixed the scene, lighting, and viewpoint• Write the BRDF as an expansion in terms of

basis functions, instead of lighting

Encapsulated in a single 1D function or curve

J

j jjoiqoioiqoi bcf1

)(),()),((),(),( Fixed part Editable part

65

[Ben Artzi et al. 08]• A Precomputed Polynomial Representation for

Interactive BRDF Editing with Global Illumination– Ben-Artzi, Egan, and Ramamorthi– ACMTOG 2008