Advances in Real-Time Rendering in 3D Graphics and GamesNew Orleans, LA (August 2009)

Light Propagation Volumes in Light Propagation Volumes in CryEngineCryEngine®® 3 3

Anton KaplanyanAnton Kaplanyan

Advances in Real-Time Rendering in 3D Graphics and Games

AntonK@Crytek.deAntonK@Crytek.de

Advances in Real-Time Rendering in 3D Graphics and GamesNew Orleans, LA (August 2009)

Agenda

IntroductionIntroduction

CryEngineCryEngine® ® 3 lighting pipeline overview3 lighting pipeline overview

Core ideaCore idea

Applications (with video)Applications (with video)

ImprovementsImprovements

Combination with other technologies (with video)Combination with other technologies (with video)

Optimizations for consolesOptimizations for consoles

Conclusion and future workConclusion and future work

Live demoLive demo

Advances in Real-Time Rendering in 3D Graphics and GamesNew Orleans, LA (August 2009)

Introduction into real-time graphics

Strictly fixed budget per frame

Many techniques are not physically-based

Consistent performance

Game production is complicated

This talk is mostly about massive and indirect lighting

This is a high level talk

– More implementation details in the paper

Advances in Real-Time Rendering in 3D Graphics and GamesNew Orleans, LA (August 2009)

CryEngine®® 3 renderer overview (1 / 5)

Advances in Real-Time Rendering in 3D Graphics and GamesNew Orleans, LA (August 2009)

CryEngine®® 3 renderer overview (2 / 5)

Advances in Real-Time Rendering in 3D Graphics and GamesNew Orleans, LA (August 2009)

CryEngine®® 3 renderer overview (3 / 5)

Advances in Real-Time Rendering in 3D Graphics and GamesNew Orleans, LA (August 2009)

CryEngine®® 3 renderer overview (4 / 5)

Advances in Real-Time Rendering in 3D Graphics and GamesNew Orleans, LA (August 2009)

CryEngine®® 3 renderer overview (5 / 5)

Lighting accumulation pipeline:

– Apply global / local hemispherical ambient

– Optionally: Replace it with Deferred Light Probes locally

– [Global illumination solution should take place here]

– Multiply indirect term by SSAO to apply ambient occlusion

– Apply Direct Lighting on top of Indirect Lighting

AmbientDeferred

light probesDirect

lightingSSAO

Indirect lighting term

Global Illumination

Optional

Advances in Real-Time Rendering in 3D Graphics and GamesNew Orleans, LA (August 2009)

Real-time rendering development trends

Rendering is a multi-dimensional query [Mittring09]

– R = R(View, Geometry, Material, Lighting)

Divide-and-conquer strategy, some examples:

– Shadow maps (decouple visibility queries)

– Deferred techniques (decouple lighting / shading)

– Screen-space techniques (SSAO, SSGI, etc.)

– Reprojection techniques (partially decouples view)

Why?

– Less interdependencies => more consistent performance

– Future trends: parallel and distributed computations friendly

Advances in Real-Time Rendering in 3D Graphics and GamesNew Orleans, LA (August 2009)

Paper reference icon

This icon means that details are in the paper

TM

Advances in Real-Time Rendering in 3D Graphics and GamesNew Orleans, LA (August 2009)

Light Propagation Volumes

Advances in Real-Time Rendering in 3D Graphics and GamesNew Orleans, LA (August 2009)

Light Propagation Volumes: Goals

Decouples lighting complexity from screen coverage (resolution×overdraw)

– Radiance caching and storing technique

Massive lighting with point light sources

Global illumination

Participating media rendering (still work in progress…)

Consoles friendly (Xbox 360, PlayStation 3)

Advances in Real-Time Rendering in 3D Graphics and GamesNew Orleans, LA (August 2009)

Related work

Irradiance Volumes [GSHG97], [Tatarchuk04], [Oat05]Irradiance Volumes [GSHG97], [Tatarchuk04], [Oat05]

+ Signed Distance Fields [Evans06]

Lightcuts: A Scalable Approach to Illumination [WFABDG05]

Multiresolution Splatting for Indirect Illumination [NW09]

Hierarchical Image-Space Radiosity for Interactive Global Illumination [NSW09]

Non-interleaved Deferred Shading of Interleaved Sample Patterns [SIMP06]

Advances in Real-Time Rendering in 3D Graphics and GamesNew Orleans, LA (August 2009)

SH Irradiance volumes

A grid of irradiance samples is taken throughout the scene

Each irradiance samplestored in SH form

At render time, the volume is queried and near-by irradiance samples are interpolated to estimate the global illumination at a point in the scene

From [GSHG97], [Tatarchuk04]

Advances in Real-Time Rendering in 3D Graphics and GamesNew Orleans, LA (August 2009)

Low-frequency radiance volumes

Similar to SH Irradiance Volumes [Tatarchuk04]

Stores radiance distribution instead

Low resolution 3D texture on GPU (up to 323 texels)

SH approximation is low order (up to linear band)

Radiance is not smooth [GSHG97]

– But what is the error introduced by approximating it?

From [GSHG97]

Advances in Real-Time Rendering in 3D Graphics and GamesNew Orleans, LA (August 2009)

Radiance approximation

Advances in Real-Time Rendering in 3D Graphics and GamesNew Orleans, LA (August 2009)

Light propagation in radiance volume

Start with given initial radiance distribution from emitters

Iterative process of radiance propagation

6-points axial stencil for adjacent cells

– Gathering, more efficient for GPUs

– Energy conserving

Each iteration adds to result, then propagates further

Advances in Real-Time Rendering in 3D Graphics and GamesNew Orleans, LA (August 2009)

Light propagation in radiance volume

Advances in Real-Time Rendering in 3D Graphics and GamesNew Orleans, LA (August 2009)

Rendering with Light Propagation Volume

Regular shading, similar to SH Irradiance Volumes

– Simple 3D texture look-up using world-space position

– Integrate with normal’s cosine lobe to get irradiance

• Simple computation in the shader for 2nd order SH

– Lighting for transparent objects and participating media

Deferred shading / lighting

– Draw volume’s shape into accumulation buffer

– Supports almost all deferred optimizations

Advances in Real-Time Rendering in 3D Graphics and GamesNew Orleans, LA (August 2009)

Massive Lighting with point light sources

Advances in Real-Time Rendering in 3D Graphics and GamesNew Orleans, LA (August 2009)

Massive lighting

Option 1: Inject initial energy, then propagate radiance

– A bit faster for crazy amount of lights

Option 2: Add pre-propagated radiance into each cell

– Simple analytical equation in the shader for point lights

– Higher quality, no propagation error

Error depends on the ratio (light source radius / cell size)

– Radius threshold for lighting with radiance volume

Advances in Real-Time Rendering in 3D Graphics and GamesNew Orleans, LA (August 2009)

Glossy reflections with Light Propagation Volumes

Advances in Real-Time Rendering in 3D Graphics and GamesNew Orleans, LA (August 2009)

Glossy reflections example

Advances in Real-Time Rendering in 3D Graphics and GamesNew Orleans, LA (August 2009)

Massive lighting: Results

NVIDIA GeForce GTX 280 GPU, Intel Core 2 Quad CPU @ 2.66 GHz, DirectX 9.0c API, HDR rendering @ 1280x720, no MSAA, Volume size: 323

Advances in Real-Time Rendering in 3D Graphics and GamesNew Orleans, LA (August 2009)

Massive lighting video

Advances in Real-Time Rendering in 3D Graphics and GamesNew Orleans, LA (August 2009)

Global Illumination with Light Propagation Volumes

Advances in Real-Time Rendering in 3D Graphics and GamesNew Orleans, LA (August 2009)

Global Illumination with Light Propagation Volumes

Instant Radiosity [Keller97]

– The main idea is to represent light bouncing as a set of secondary light sources: Virtual Point Lights (VPL)

Splatting Indirect Illumination [DS07]

– Based on Instant Radiosity

– Reflective Shadow Maps (RSM) are used to generate initial set of VPLs on GPU

– Importance sampling of VPLs from RSM

Advances in Real-Time Rendering in 3D Graphics and GamesNew Orleans, LA (August 2009)

Reflective Shadow Maps

Reflective Shadow Map – efficient VPL generator

Shadow map with MRT layout: depth, normal and color

Advances in Real-Time Rendering in 3D Graphics and GamesNew Orleans, LA (August 2009)

Inject the initial radiance from VPLs into radiance volume

– Point rendering

– Place each point into appropriate cell

• Using vertex texture fetch / R2VB

– Approximate initial radiance of each VPL with SH

• Simple analytical expression in shader

Propagate the radiance

Render scene with propagated radiance

Global Illumination with Light Propagation Volumes

Advances in Real-Time Rendering in 3D Graphics and GamesNew Orleans, LA (August 2009)

Implementation details

Light Propagation Volume moves with camera

3D cell-size snapping for volume movement

2D texel-size snapping for RSM movement

RSM is higher in resolution than radiance volume

Smart down-sampling of RSM

Advances in Real-Time Rendering in 3D Graphics and GamesNew Orleans, LA (August 2009)

Global Illumination with Light Propagation Volumes

Advances in Real-Time Rendering in 3D Graphics and GamesNew Orleans, LA (August 2009)

Issue: Cell-alignment of VPLs

Injection of VPLs involvesposition shifting

– Position of injected VLP becomes grid-aligned

– Consequence of spatial radiance approximation

Unwanted radiance bleeding

– Lighting of double-sided and thin geometry

Advances in Real-Time Rendering in 3D Graphics and GamesNew Orleans, LA (August 2009)

Cell-alignment of VPLs: Bleeding example

Advances in Real-Time Rendering in 3D Graphics and GamesNew Orleans, LA (August 2009)

Cell-alignment of VPLs: Solution

VPL half-cell shifting

– towards normal

– towards light direction

Coupled by anisotropic bilateral filtering

– During final rendering pass

– Sample radiance with offset by surface normal

– Compute radiance gradient

– Compare radiance with radiance gradient

Advances in Real-Time Rendering in 3D Graphics and GamesNew Orleans, LA (August 2009)

Cascaded Light Propagation Volumes for GI

One grid is limited in dimensions and low resolution

Multiresolution approach for radiance volumes

– Similar to Cascaded Shadow Maps technique [SD02]

– Preserves surrounding radiance outside of the view

Each cascade is independent

– With separate RSM for each cascade

– Transmit radiance across adjacent edges

– Filter objects by size for particular RSM

Efficient hierarchical representation of radiance emitters

Advances in Real-Time Rendering in 3D Graphics and GamesNew Orleans, LA (August 2009)

Global Illumination Video

Advances in Real-Time Rendering in 3D Graphics and GamesNew Orleans, LA (August 2009)

Global Illumination: Combination with SSAO

No secondary occlusion for light propagation volumes

Can be approximated by Ambient Occlusion term

SSAO on, GI off SSAO off, GI on GI + SSAO

Advances in Real-Time Rendering in 3D Graphics and GamesNew Orleans, LA (August 2009)

Global Illumination: Combination with SSGI

Screen-Space Global Illumination [RGS09]

Limitations of SSGI

– Only screen-space information

– Huge kernel radius for close objects

Limitations of Light Propagation Volumes

– Local solution

– Low resolution spatial approximation

Supplementing each other

– Custom blending

Advances in Real-Time Rendering in 3D Graphics and GamesNew Orleans, LA (August 2009)

Global Illumination: Combination with SSGI

SSGI off SSGI on

Advances in Real-Time Rendering in 3D Graphics and GamesNew Orleans, LA (August 2009)

Optimizations for consoles: Xbox 360 / PS3

3D texture look-up with trilinear filtering

Radiance volume is 32 bpp for all three SH textures

Xbox 360, ~3,5 ms per frame

– Vertex texture fetching for RSM injection

– Work-around to resolve into particular slice of 3D texture

PlayStation 3, ~3,4 ms per frame

– Emulate signed blending in the shader

– R2VB for RSM injection (using memory remapping)

– Render to unwrapped 2D RT then remap as 3D texture

Advances in Real-Time Rendering in 3D Graphics and GamesNew Orleans, LA (August 2009)

Future work

Better radiance approximation…

Participating media rendering

Occlusion for indirect lighting

Multiple bounces

Improve quality

– Improved propagation scheme

– Better angular approximation

– Adaptive grids

Support for arbitrary types of light sources

Advances in Real-Time Rendering in 3D Graphics and GamesNew Orleans, LA (August 2009)

References

Advances in Real-Time Rendering in 3D Graphics and GamesNew Orleans, LA (August 2009)

Acknowledgment

Michael Endres, Felix Dodd, Marco Siegel, Frank Meinl, Alexandra Cicorschi, Helder Pinto, Efgeni Bischoff and other artists and designers at Crytek for created scenes

Martin Mittring, Vladimir Kajalin, Tiago Sousa, Ury Zhilinsky, Mark Atkinson, Evgeny Adamenkov and the whole Crytek R&D team

Special thanks to Carsten Dachsbacher and Natalia Tatarchuk

Advances in Real-Time Rendering in 3D Graphics and GamesNew Orleans, LA (August 2009)

AntonK@Crytek.deAntonK@Crytek.de