high-quality pre-integrated volume rendering using hardware accelerated pixel shading

High-Quality Pre-Integrated Volume Rendering Using Hardware Accelerated Pixel Shading

Klaus Engel, Martin Kraus, Thomas ErtlVisualization and Interactive Systems GroupUniversity of Stuttgart, Germany

Siggraph/Eurographics Workshop on Graphics Hardware 2001

Visualization and Interactive Systems Group,University of Stuttgart

Motivation

• Problems of texture-based volume graphics on PCs:– scientific visualization

• limited quality• 3D textures not commonly available• pre-classification shading

– computer games• high rasterization requirements

=> low performance• high texture memory requirements

Visualization and Interactive Systems Group,University of Stuttgart

Texture-based Volume Rendering

2D textures(axis-aligned

slices)

3D textures(view-aligned

Slices)

texturing(trilinear

interpolation)compositing

(blending)

texturing(bilinear

interpolation)compositing

(blending)

Visualization and Interactive Systems Group,University of Stuttgart

Volume Rendering Quality Improvement• Higher sampling rates: additional tri-linearly interpolated

slices– 3D textures:

• render additional slice polygons– 2D textures:

• render intermediate slices using multi-texture approach(Rezk-Salama,Engel et al., GH2000)

• But:– decreases rendering speed due to additional rasterization– many additional slices required even for low-resolution volumes – non-linear transfer functions:

not sufficient to sample volume with the Nyquist frequency of scalar data

Visualization and Interactive Systems Group,University of Stuttgart

Volume Classification

voxels

Post-classification

interpolation

interpolation

Pre-classification

classification

transfer functions

classification

Visualization and Interactive Systems Group,University of Stuttgart

Pre-Integrated Volume Rendering• Our approach:

– texture-based (2D/3D)– ray-segments computed in a pre-processing step– pre-computed ray-segment lookup (dependent texture)– small number of slices => fast

• Especially suited for:– low resolution volume data– non-linear transfer functions

• Capable of: – high-quality rendering– direct volume rendering, volume shading, isosurfaces

Visualization and Interactive Systems Group,University of Stuttgart

Pre-integrated Volume Rendering - Ideaslice-by-slice slab-by-slab

sb

sfsf

sb

fetch integral fromdependent texture

sbsf

pre-integrate all possible combinations

hardware-accelerated implementation

on NVidia GeForce3 chip

sf sb

front slice

back slice

project slice

texture polygon

Visualization and Interactive Systems Group,University of Stuttgart

Texture Shaders

• Texture Shaders (when enabled) replace the standard OpenGL texture fetch mechanism

• With texture shaders, results from previous stages can affect the lookup of a subsequent stage

ARB_multitexture with NV_texture_shader

ARB_multitexture only

(s,t,r,q)0

(s,t,r,q)1

(s,t,r,q)2

(s,t,r,q)0

(s,t,r,q)1

(s,t,r,q)2

stage 0math & fetch

stage 1math & fetch

stage 2math & fetch

unit 0fetch

unit 1fetch

unit 2fetch

Visualization and Interactive Systems Group,University of Stuttgart

Pre-integrated Volume Rendering - Texel Fetch

RGB1

(s2,t2,r2) (1,0,0)

stage 2DOT_PRODUCT_NV(1,0,0) • RGB1=sb

RGB0

(s3,t3,r3) (1,0,0)

stage 3DOT_PRODUCT_TEXTURE2D_NV

(1,0,0) • RGB0=sf

RGBA result

on to register combiners

front slice

back slice

sb

stage 1TEXTURE_2D

(s1,t1)RGBA result

sf

stage 0TEXTURE_2D

(s0,t0)RGBA result

sb

sf

Visualization and Interactive Systems Group,University of Stuttgart

Isosurfaces

Isosurfaces:particular dependent texture

32 96 sf

sb

32

96

front slice

back slice

1.

1

front slice

back slice

4.

4front slice

back slice

2.

2

front slice

back slice

3.3

3a

3a

Visualization and Interactive Systems Group,University of Stuttgart

Isosurfaces – Gradient Interpolation

• g = IP gb + (1-IP) gf

interpolation weight: IP = (siso – sf)/(sb – sf)

• store gradient with scalar data in RGBA texture• store IP in dependent texture• implement interpolation in reg. combiners

sb

front slice

back slice

gfgbg

Visualization and Interactive Systems Group,University of Stuttgart

Pre-integrated Volume Rendering – Register Combiners

• Combiner 0+1: rebuild gradients (front + back)

gy gz gx

R BG A R BG A

• Combiner 2: interpolate gradients

gx gy gz

I = Ia + Id ( n . l ) + Is ( n . h )p

• Combiner 3-7: shading (isosurface or volume)

pmax = 256

IP = (siso – sf)/(sb – sf) g = IP gb + (1-IP) gf

s s

Visualization and Interactive Systems Group,University of Stuttgart

Interactive Transfer Function Update

• 8 bit data: 256 x 256 = 65536 ray segments=> ~20 seconds on Athlon 650 (for constant ray segment length)

• 3 optimizations:– assume constant ray segment length– local modification of TF requires local update of DT– neglect self-attenuation within ray-segments and use

integral functions (as in Max et al., VolVis’90)

Visualization and Interactive Systems Group,University of Stuttgart

Results – Direct Volume Rendering

128 slicespre-

classification

128 slicespre-integrated

284 slicespost-

classification

128 slicespost-

classification

Visualization and Interactive Systems Group,University of Stuttgart

Results – Direct Volume Rendering, Random TF

Visualization and Interactive Systems Group,University of Stuttgart

Results - Isosurfaces

Visualization and Interactive Systems Group,University of Stuttgart

Results Performance

• PCNvidia GeForce3 64 MB DDR SDRAM (3.8 ns), 200 MHz internal clock, 460 MHz memory clock, 512 x 512 viewport

• 25 % slower than approach that used 1 or 2 texture fetches=> loopback in texture_shaders for 3rd and 4th fetch (2 clock cycles)

• Interactive transfer function update when neglecting self-attenuation

Visualization and Interactive Systems Group,University of Stuttgart

Problems

• artifacts in isosurfaces

• unequal ray segment lengths• only linear interpolation for each ray segment

• artifacts on volume boundaries• clipping planes front

sliceback slice

Visualization and Interactive Systems Group,University of Stuttgart

Conclusions• Pre-Integrated Volume Rendering

– High quality• even for small volumes• with small number of slices• with non-linear transfer functions• reproduces high frequencies in the transfer function

– Low rasterization requirements– Semi-transparent Volume Rendering– Volume Shading– Isosurfaces

• arbitrary number• independently colored• per-pixel lighting

Visualization and Interactive Systems Group,University of Stuttgart

Demo

Demo and Data Download:http://wwwvis.informatik.uni-stuttgart.de/~engel/pre-integrated/

Visualization and Interactive Systems Group,University of Stuttgart

Appendix: Volume Shading and Isosurfaces

• Per-pixel shading requires per-voxel gradient or dot product– dynamic lighting

– static lighting (fixed light sources)

voxel gradientx y z

voxel intensity

R BG A

dot product voxel intensity

L A

Visualization and Interactive Systems Group,University of Stuttgart

Appendix: Gradient Rebuild

• dot product affects only the RGB components voxel intensity stored in R, gradient in GBA

voxel gradient

y z x

voxel intensity

R BG A

• Reroute x-component of gradient to R channel in the register combiners for lighting

Visualization and Interactive Systems Group,University of Stuttgart

Appendix: Register Combiners

Visualization and Interactive Systems Group,University of Stuttgart

Appendix: Register Combiners

input registersRGB A

const. color1

G

h

l spare 1

Id

Ia

prim. color

sec. color

Isconst. color0

const. 0

lighting

ABCD

A • B

dotproduct

A

B

C

D

AB

A

B

C

D

AB

GC3 GC5 GC6

I = Ia + Id ( n . l ) + Is ( n . h )p

GC7

ABCD

AB+CD

sum

Id ( n . l ) + Is ( n . h )p

C • D

A

B

C

D

AB

GC4 FC

sumA10D

AB+(1-A)C

+D

Visualization and Interactive Systems Group,University of Stuttgart

Appendix: Dependent Texture Shaders

• DEPENDENT_AR_-TEXTURE_2D_NV:The previous texture shader’s alpha and red colors are used as the (s,t) for a 2D texture lookup

• not suitable for us, because we need (s,t) from two different textures

(s0,t0)

RGBA result

on to texture environment or

register combiners

Texture coordinates unused …

RGBA

stage 0TEXTURE_2D

GL_RGBA8internal format

stage 1DEPENDENT_AR_-TEXTURE_2D_NV

nonprojective_lookup_2D( A, R )

Visualization and Interactive Systems Group,University of Stuttgart

Appendix: Ray-segment Integration

)()(1

))((exp

))(())(())((exp'

density opticalscalar )),,((ty vectorchromatici )),,((

2

2

0

00

fbfl

Dt

l

Dt

lll

sssxs

Idtts

RGB

dttstsduusI

zyxfzyxf

lx

l

tl

Visualization and Interactive Systems Group,University of Stuttgart

Appendix: Results - Quality

• Comparison:– pre-classification– post-classification– pre-integrated classification

pre-integrated dependent texture post-classification dependent texturefor tri-linear ip. intermediate slices

sf

sb

asb

(1-a)sf

Visualization and Interactive Systems Group,University of Stuttgart

Appendix: Performance Results

volume size 162 322 642 1282 2562

direct 90 50 25 13 4isosurface 70 40 21 11 1

TF update (seconds) global localself-attenuation 20 < 10w/o self-attenuation 0.3 < 0.15