advanced real-time shader techniques natalya tatarchuk 3d application research group ati research

Advanced Real-Time Shader Techniques

Natalya Tatarchuk3D Application Research GroupATI Research

Advanced Real-Time Shader Techniques – Siggraph 2003, San Diego, CA

Overview

• Problems that RenderMonkey solves for you

• Real-time shader creation using RenderMonkey development environment

– Building an effect from scratch

• Efficient HLSL shaders: tips and tricks

• Advanced Shader Examples


RenderMonkey Addresses the Needs of Real-Time Effect Developers

• Shaders are more than just assembly code

• Encapsulating shaders can be complex– Cannot deliver shader-based effects using a standard

mechanism

• Solve problems for shader development – Designing software on emerging hardware is difficult– Lack of currently existing tools for full shader development– Need a collaboration tool for artists and programmers for

shader creation


Designed for Extensibility

• Flexible framework design

• Evolves with the graphics API

• Allows easy incorporation of existing APIs

• Full DirectX® 9.0 support, including Microsoft HLSL

• Extensible framework to support emerging HL standards

• OpenGL 2.0 Shading Language Prototype


Making Your Shader Development More Effective

• Simplifies shader creation

• Fast prototyping and debugging of new graphics algorithms

• Helps you share graphics effects with other developers and artists

• Easy integration of effects into existing applications

• Quickly create new components using our flexible framework

• A communication tool between artists and developers


Program Your Shaders Using Our Intuitive, Convenient IDE

Workspace View

Output Window

Preview Window

Editor Windows

Artist Editor


Creating a Shader-based Effect Using RenderMonkey

1. Setup all your effect parameters 1. Variables

2. Stream mapping

3. Models and textures

4. Rendering states

2. Create your vertex and pixel shaders

3. Link RenderMonkey parameter nodes to your shaders as necessary

4. Compile and explore!


Phong Illumination Example

Classic lighting equation example:

speculardiffuseambient IIII

where each of the lighting contribution components can be computed as follows:

aaambient IkI

)( LNIkI dddiffuse

snssspecular RVIkI )(


RenderMonkey Run-Time Database Overview• Encapsulate all effect data in a single text file• Each Effect Workspace consists of:

– Effect Group(s) • Effect(s)

– Pass(es)• Render State• Pixel Shader• Vertex Shader• Geometry• Textures

– Variables, notes and stream mapping nodes


RenderMonkey Database Uses Standard XML File Format• Allows easy data representation

– Industry standard – User-extensible– Parsers are readily available– User-readable file format– Easily add Perl scripts to post-process your files– Published DTD for RenderMonkey XML allows robust file validation

• Describes all effect-related information– Shader code– Render states– Models / texture information– Rendering states

• Import and export easily from your native formats– Use our parser and run-time format– Write an exporter / importer plug-in (Example: Our Microsoft FX

Exporter)


Workspace View: Your Window into Our Database• All effect data organized in

a hierarchical Workspacetree view

• Node rules enforce correct datarelationships

• Full Cut / Copy / Paste / Undo

• Easily identify node data typesby their icons

• Quickly identify invalid data

• Easy perusal of data values via automatic tooltips


Effect Group Nodes

• Group related effects in one container

• Provides mechanism for dealing with a lot of effects

• Facilitate fallback versions of same effect

• Group shaders by type, pick the first one that validates

• How you group effects is entirely up to you


Effect Nodes

• Encompass all information needed to implement a real time visual effect

• Composed of one or more passes

• Inherit data and states from a Default Effect

– Used to set a known starting state for all effects


Data Scope and Traversal

• Store common effect data in the default effect:– All other effects inherit data from it– Easy to share data from a common point

• Common rendering states• Common vertex and pixel Shaders

• Effects don’t inherit data from other effects in the workspace

– Common data that should be global to the effects:• Stream mapping• Texture variables• Model variables

• Variable scope is similar to C-style variable scope

• Data validation occurs upward through passes in the effect then upward through passes in the default effect


Pass Nodes

• Every pass is a draw call

• Passes inherit data from previous pass within the effect– First pass inherits from default effect

• A typical pass contains:– A vertex and pixel shader pair (either HLSL or ASM) (required)

– Render state block• Render states are inherited from pass to pass

– Texture objects• Must reference a valid texture variable• Each texture object stores associated texture states

– Geometry model reference (required)– Stream mapping reference (required)– May also contain nodes of other types (variables, etc)

• Different geometry can be used in each pass


Variable Nodes

• Parameters to your shaders

• More intuitive way of dealing with constant store registers

• Give shader constants meaningful names and types

• Manipulate shader constants using convenient GUI widgets

• Supported variable types:

- Matrices - Vectors- Scalars - Colors- Textures - Strings (Notes)


Pre-defined Variables Help You Set Up Your Shaders Quickly

• RenderMonkey IDE calculates their values at run-time

• Provide a set of commonly used parameters:

– View projection matrix – View matrix – Inverse view matrix– Projection matrix– View direction vector– View position vector– Time, time cycle period– cos_time, sin_time, tan_time

• and more…


Edit Shader Parameters Using GUI Widgets via Editor plug-ins

• Utilize knowledge of the variable type for editing:

– Color Editor– Vector Editor– Matrix Editor– Scalar Editor

• Edit variables with custom widgets: easy to create your own

• Only accept changes you are happy with


Phong Illumination Parameters

• Ambient, diffuse, and specular light contributions:– Coefficients – Color intensities – Can be expressed as shader constants

• Normal, view and reflection vectors– Normal vector is one of vertex shader inputs– View and reflection vectors are computed per-vertex

• Specular reflection parameter– Also can be expressed as a shader constant parameter


Simplified Stream Mapping Setup

• Setup each stream using theStream Mapping Editor

• A stream mapping node can be created at any point in the workspace

• Stream mapping nodes can be shared by multiple effects

• Multiple references to a stream mapping node can be created throughout the workspace

• Each pass must have its own stream mapping reference


Full Support for DirectX® 9.0 Shader Models

• Assembly shaders

– Vertex shader versions 1.0/1.1 – 2.0

– Pixel shader versions 1.0/1.1/1.3/1.4 – 2.0

• Integrated HLSL support– Supported compilation targets:

vs_1_0, vs_2_0;ps_1_x and ps_2_0

• Future support for shader versions 2_0_sw, 2_0_x, and 3_0


Edit Shaders Using Specialized Editors

• Tabbed view allows editing multiple passesin the effect

– Easy switching between vertex and pixel shaders in the pass

– Full support for standard Windows editor functionality• Undo / Cut / Copy / Paste / Font settings

• Automatic syntax coloring of shader code– Separate syntax rules for HLSL and ASM shaders

• Type your code, compile and see instant results!


Assembly Shader Editor Plug-in

• Easily link RenderMonkey variable nodes to constant store registers

• Syntax colored for shader assembly language

• Automatically maintains count of ALU and texture ops in your shader as you type!

• Allows saving shader code to text files


HLSL Shader Editor Plug-in

• Allows incredible ease of shader creation

• Simple interface links RenderMonkey nodes to HLSL variables andsamplers

– Link vectors, colors, scalars and matricesto variable parameters

– Link texture objects to samplers

• Control target and entry points for your shaders


struct VS_OUTPUT{ float4 Pos : POSITION; float3 Normal: TEXCOORD0; float3 Light : TEXCOORD1; float3 View : TEXCOORD2; };VS_OUTPUT main( float4 Pos: POSITION, float3 Norm: NORMAL ){ VS_OUTPUT Out = (VS_OUTPUT) 0;

Out.Pos = mul( view_proj_matrix, Pos ); // Transformed position

Out.Normal = normalize( mul(view_matrix, Norm) ); // Normal Out.Light = normalize(-lightDir); // Light vector

float3 Pview = mul( view_matrix, Pos ); // Compute view position

Out.View = -normalize( Pview ); // Compute view vector

return Out;}

Specular Lighting: Vertex Shader

Compute view vectorTransform and output vertex position

Compute normaland light vectors


Specular Lighting: Pixel Shader

float4 ambient;float4 diffuse;float4 specular;float Ka;float Ks;float Kd;float N;

float4 main( float4 Diff : COLOR0, float3 Normal: TEXCOORD0, float3 Light : TEXCOORD1, float3 View : TEXCOORD2 ) : COLOR{ // Compute the reflection vector: float3vReflect = normalize(2*dot(Normal, Light)*Normal - Light);

// Final color is composed of ambient, diffuse and specular // contributions: float4 FinalColor = Ka * ambient + Kd * diffuse * dot( Normal, Light ) + Ks * specular * pow( max( dot( vReflect,

View), 0), N ) ; return FinalColor;}

Compute reflectionvectorSpecular contribution

Compute ambientand diffuse contributions


Read All Application Messages in a Single Place - The Output Module

• Output the results of shader compilation and application messages

• Linked with the shader editor for compilation error highlighting

• View messages from the renderer of your effect: – Notifies you whenever resources are created (textures

loaded, shaders compiled)


Integrated Compile Time Error Reporting Simplifies Shader Creation

• Compilation errors displayedin the output module window

• Double-clicking on the errorhighlights the line containingerroneous code in the editor

• Optional compilation of a single shader, all shaders in active effect or all shaders in the workspace


Interactively Preview Your Effects in the Viewer Plug-in

• All changes to the shaderor its parameters modify the rendered image in real time

• DirectX® 9.0 preview– HAL / REF

• Customize the preview:– Standard trackball navigation – Customizable settings for camera and clear colors– A set of preset views (Front / Back / Side / etc)


Setup All Render States in the Render State Editor Plug-in

• Modify any render state within a particular pass

• Render states are inherited

– From previous passes in the effect

– From the default effect

• Great for exploring the results of changing a render state – a useful learning tool


Using Textures in a RenderMonkey

Effect

• Texture variables can beshared between differenteffects

• Support the following texturetypes:

– 1D, 2D texture maps (JPEG,TGA, BMP formats)

– Cube maps (DDS)– Volume textures (DDS)– Dynamic renderable textures

• Texture objects belong to apass node

– Reference a texture variable to sample from– Store all related texture and sampler states– Maps directly to HLSL samplers


Texture and Sampler State Editing

• Specify texture and sampler state values (filtering, clamping, etc) for each texture node within a pass

• Texture and samplerstates are bundled together in one editor

• State changes modify rendered output in real time


Dynamic Texture Rendering Example

• Direct output of one or more passes to a texture map

• Sample from that texture to create interesting effects

– Scene post-processing • Depth of Field• Gaussian Blur• Tone Mapping• HDR Rendering• Other image processing techniques


Rendering to a Texture Using RenderMonkey

• Simple to setup

• Special texture variable type:– Renderable Texture

• Direct pass output to a texture – Use Render Target node to link to a Renderable

Texture

• Modify parameters using appropriate editors– Render Target Editor– Renderable Texture Editor


Customize Renderable Texture

• Specify desired texture format– Select from a variety of formats

• Control texture dimensions

– Width and Heightor

– Tie the texture size to viewport dimensions

• Specify whether mip maps will be generated automatically


Control Render Target Parameters

• Modify parameters to suit your needs

• Specify whether the textureshould be cleared

• Specify clear color

• Toggle whether the depth buffer should be cleared

• Specify the depth clear value


Develop Shaders With Your Artists

• Use the Artist Editor Interface to explore shaders:

– Expose the power of programmable shaders to artists and designers

– Programmers and Artists living in harmony!

• View workspace using Art tab

– Only view data relevant to the artists

– Programmers can select which data is artist-editable

• Provides look and feel of GUI widgets artists are familiar with

• See changes in real time


Edit All Artist Parameters Using a Single Interface • Programmer has control of what parameters can be

modified– Flag variables as ‘artist-editable’ as needed

• Artist modifies only specified variables:– Use the Artist Editor interface

• Data organization follows the effect structure:– Variables are grouped in tab sheets by passes/effects they

belong to

• Artists can tweak parameters and instantly see changes:– Modify vectors, scalars, colors using convenient controls


Artist Editor Interface


Overview

• Writing optimal HLSL code– Compiling issues

– Optimization strategies

– Code structure pointers

• HLSL Shader Examples– Multi-layer car paint effect

– Translucent Iridescent Shader

– Überlight Shader


Why use HLSL?

• Faster, easier effect development– Instant readability of your shader code – Better code re-use and maintainability– Optimization

• Added benefit of HLSL compiler optimizations• Still helps to know what’s under the hood

• Industry standard which will run on cards from any vendor

– Current and future industry direction

• Increase your ability to iterate on a given shader design, resulting in better looking games

• Conveniently manage shader permutations


Compile Targets

• Legal HLSL is still independent of compile target chosen

• But having an HLSL shader doesn’t mean it will always run on any hardware!

• Currently supported compile targets:– vs_1_1, vs_2_0, vs_2_sw– ps_1_1, ps_1_2, ps_1_3, ps_1_4, ps_2_0, ps_2_sw

• Compilation is vendor-independent and is done by a D3DX component that Microsoft can update independent of the runtime release schedule


Compilation Failure

• The obvious: program errors (bad syntax, etc)

• Compile target specific reasons – your shader is too complex for the selected target– Not enough resources in the selected target

• Uses too many registers (temporaries, for example)• Too many resulting asm instructions for the compile target

– Lack of capability in the target• Such as trying to sample a texture in vs_1_1• Using dynamic branching when unsupported in the target• Sampling texture too many times for the target (Example: more

than 6 for ps_1_4)

• Compiler provides useful messages


Use Disassembly for Hints

• Very helpful for understanding relationship between compile targets and code generation

• Disassembly output provides valuable hints when “compiling down” to an older compile target

• If successfully compiled for a more recent target (eg. ps_2_0), look at the disassembly output for hints when failing to compile to an older target (eg. ps_1_4)

– Check out instruction count for ALU and tex ops– Figure out how HLSL instructions get mapped to assembly


Getting Disassembly Output for Your Shaders• Directly use FXC

– Compile for any target desired– Compile both individual shader files and

full effects– Various input arguments

• Allow to turn shader optimizations on / off• Specify different entry points• Enable / disable generating debug information


Easier Path to Disassembly

• Use RenderMonkey while developing shaders

– See your changes in real-time

• Disassembly output is updated every time a shader is compiled

– Displays count for ALUand texture ops, as well as the limits forthe selected target

– Can save resulting assembly code into text file


Optimizing HLSL Shaders

• Don’t forget you are running on a vector processor

• Do your computations at the most efficient frequency

– Don’t do something per-pixel that you can do per-vertex

– Don’t perform computation in a shader that you can precompute in the app

• Use HLSL intrinsic functions– Helps hardware to optimize your shaders– Know your intrinsics and how they map to asm,

especially asm modifiers


HLSL Syntax Not Limited• The HLSL code you write is not limited by the compile

target you choose

• You can always use loops, subroutines, if-else statements etc

• If not natively supported in the selected compile target, the compiler will still try to generate code:– Loops will be unrolled– Subroutines will be inlined– If – else statements will execute both branches, selecting

appropriate output as the result

• Code generation is dependent upon compile target• Use appropriate data types to improve instruction count

– Store your data in a vector when needed– However, using appropriate data types helps compiler do

better job at optimizing your code


Using If Statement in HLSL

• Can have large performance implications– Lack of branching support in most asm

models

– Both sides of an ‘if’ statement will be executed

– The output is chosen based on which side of the ‘if’ would have been taken

• Optimization is different than in the CPU programming world


Example of Using If in Vs_1_1

If ( Threshold > 0.0 ) Out.Position = Value1;else Out.Position = Value2;

// calculate lerp value based on Value > 0mov r1.w, c2.xslt r0.w, c3.x, r1.w

// lerp between Value1 and Value2mov r7, -c1add r2, r7, c0mad oPos, r0.w, r2, c1

generates following assembly output:


Example of Function Inlining// Bias and double a value to take it from 0..1 range to -1..1 range

float4 bx2(float x){ return 2.0f * x - 1.0f;}

float4 main( float4 tc0 : TEXCOORD0, float4 tc1 : TEXCOORD1, float4 tc2 : TEXCOORD2, float4 tc3 : TEXCOORD3) : COLOR{ // Sample noise map three times with different // texture coordinates float4 noise0 = tex2D(fire_distortion, tc1); float4 noise1 = tex2D(fire_distortion, tc2); float4 noise2 = tex2D(fire_distortion, tc3);

// Weighted sum of signed noise

float4 noiseSum = bx2(noise0) * distortion_amount0 + bx2(noise1) * distortion_amount1 + bx2(noise2) * distortion_amount2;

// Perturb base coordinates in direction of noiseSum as function of height (y) float4 perturbedBaseCoords = tc0 + noiseSum * (tc0.y * height_attenuation.x + height_attenuation.y);

// Sample base and opacity maps with perturbed coordinates float4 base = tex2D(fire_base, perturbedBaseCoords); float4 opacity = tex2D(fire_opacity, perturbedBaseCoords);

return base * opacity;}


Code Permutations By Compiling Out Portions of the Codestatic const bool bAnimate = false;VS_OUTPUT vs_main( float4 Pos: POSITION, float2 Tex: TEXCOORD0 )

{

VS_OUTPUT Out = (VS_OUTPUT) 0; Out.Pos = mul( view_proj_matrix, Pos );

if ( bAnimate )

{

Out.Tex.x = Tex.x + time / 2;

Out.Tex.y = Tex.y - time / 2;

}

else

Out.Tex = Tex;

return Out;

}

vs_1_1

dcl_position v0

dcl_texcoord v1

mul r0, v0.y, c1

mad r0, c0, v0.x, r0

mad r0, c2, v0.z, r0

mad oPos, c3, v0.w, r0

mov oT0.xy, v1

5 instructions

bool bAnimate = false;VS_OUTPUT vs_main( float4 Pos: POSITION, float2 Tex: TEXCOORD0 )

{

VS_OUTPUT Out = (VS_OUTPUT) 0;

Out.Pos = mul( view_proj_matrix, Pos );

if ( bAnimate )

{

Out.Tex.x = Tex.x + time / 2;

Out.Tex.y = Tex.y - time / 2;

}

else

Out.Tex = Tex;

return Out;

}

vs_1_1

def c6, 0.5, 0, 0, 0

dcl_position v0 8 instructionsdcl_texcoord v1

mul r0, v0.y, c1

mad r0, c0, v0.x, r0

mov r1.w, c4.x

mul r1.x, r1.w, c6.x

mad r0, c2, v0.z, r0

mov r1.y, -r1.x

mad oPos, c3, v0.w, r0

mad oT0.xy, c5.x, r1, v1

=>

=>


Scalar and Vector Data Types Optimization Notes• Scalar data types are not all natively

supported in hardware– i.e. integers are emulated on float hardware

• Not all targets have native half and none currently have double

• Can apply swizzles to vector types float2 vec = pos.xy– But!

• Not all targets have fully flexible swizzles• Acquaint yourself with the swizzles native to the

relevant compile targets (particularly ps_2_0 and lower)


Integer Data Type

• Added to make relative addressing more efficient

• Using floats for addressing purposes without defined truncation rules can result in incorrect access to arrays.

• All inputs used as ints should be defined as ints in your shader


Example of Integer Data Type Usage

• Matrix palette indices for skinning– Declaring variable as an int is a ‘free’

operation => no truncation occurs

– Using a float and casting it to an int or using directly => truncation will happen

Out.Position = mul( inPos, World[Index]);

// Index declared as float

frc r0.w, r1.w

add r2.w, -r0.w, r1.w

mul r9.w, r2.w, c61.x

mova a0.x, r9.w

m4x4 oPos, v0, c0[a0.x]

// Index declared as int

mul r0.w, c60.x, r1.w

mova a0.x, r0.w

m4x4 oPos, v0, c0[a0.x]

Code generated with float index vs integer index


Real-World Shader Examples

• Will present several case studies of developing shaders used in ATI’s demos

– Multi-tone car paint effect

– Translucent iridescent effect

– Classic überlight example

• Examples are presented as RenderMonkeyTM workspaces

– Distributed publicly with version 1.0 release


Multi-Tone Car Paint


Multi-Tone Car Paint Effect

• Multi-tone base color layer

• Microflake layer simulation

• Clear gloss coat

• Dynamically Blurred Reflections


Car Paint Layers Build Up

Multi-Tone Base Color Microflake Layer

Clear gloss coat Final Color Composite


Multi-Tone Base Paint Layer

• View-dependent lerpingbetween three paintcolors

• Normal from appearancepreserving simplificationprocess, N

• Uses subtractive tone to control overall color accumulation


Multi-Tone Base Coat Vertex ShaderVS_OUTPUT main( float4 Pos : POSITION, float3 Normal : NORMAL, float2 Tex : TEXCOORD0, float3 Tangent : TANGENT, float3 Binormal: BINORMAL ){ VS_OUTPUT Out = (VS_OUTPUT) 0;

// Propagate transformed position out: Out.Pos = mul( view_proj_matrix, Pos );

// Compute view vector: Out.View = normalize( mul(inv_view_matrix, float4( 0, 0, 0, 1)) - Pos );

// Propagate texture coordinates: Out.Tex = Tex; // Propagate tangent, binormal, and normal vectors to pixel shader: Out.Normal = Normal; Out.Tangent = Tangent; Out.Binormal = Binormal; return Out;}


Multi-Tone Base Coat Pixel Shaderfloat4 main( float4 Diff: COLOR0, float2 Tex: TEXCOORD0, float3 Tangent: TEXCOORD1, float3 Binormal: TEXCOORD2, float3 Normal: TEXCOORD3, float3 View: TEXCOORD4 ) : COLOR{ float3 vNormal = tex2D( normalMap, Tex ); vNormal = 2 * vNormal - 1.0;

float3 vView = normalize( View );

float3x3 mTangentToWorld = transpose( float3x3( Tangent, Binormal, Normal )); float3 vNormalWorld = normalize( mul(mTangentToWorld,vNormal)); float fNdotV = saturate( dot( vNormalWorld, vView ) );

float fNdotVSq = fNdotV * fNdotV; float4 paintColor = fNdotV * paintColor0 + fNdotVSq * paintColorMid + fNdotVSq * fNdotVSq * paintColor2;

return float4( paintColor.rgb, 1.0 );}

Fetch normal froma normal map andscale and bias it

to move into [-1; 1]

Normalize the viewvector to ensure

higher quality results

Compute Nw • V using world-space

normal vector

Compute the result color by lerping

three input tones using computed

fresnel term.


Microflake Layer


Microflake Deposit Layer

• Simulating light interaction resulting from metallic flakes suspended in the enamel coat of the paint

• Uses high frequency normalized vector noise map (Nn) which is repeated across the surface of the car


Computing Microflake Layer Normals

• Start out by using normal vector fetched from the normal map, N

• Using the high frequency noise map, compute perturbed normal Np

• Simulate two layers of microflake deposits by computing perturbed normals Np1 and Np2

bNaN

bNaNN

n

np

1

where a << b

dNcN

dNcNN

n

np

2

where c = b


Microflake Layer Pixel Shaderfloat4 main(float4 Diff: COLOR0, float2 Tex : TEXCOORD0,

float3 Tangent: TEXCOORD1, float3 Binormal: TEXCOORD2, float3 Normal: TEXCOORD3, float3 View: TEXCOORD4, float3 SparkleTex : TEXCOORD5 ) : COLOR

{ … fetch and signed scale the normal fetched from the normal map

float3 vFlakesNormal = 2 * tex2D( microflakeNMap, SparkleTex ) - 1;float3 vNp1 = microflakePerturbationA * vFlakesNormal + normalPerturbation * vNormal ; float3 vNp2 = microflakePerturbation * ( vFlakesNormal + vNormal ) ;

float3 vView = normalize( View );float3x3 mTangentToWorld = transpose( float3x3( Tangent, Binormal, Normal ));

float3 vNp1World = normalize( mul( mTangentToWorld, vNp1) );float fFresnel1 = saturate( dot( vNp1World, vView ));

float3 vNp2World = normalize( mul( mTangentToWorld, vNp2 ));float fFresnel2 = saturate( dot( vNp2World, vView ));

float fFresnel1Sq = fFresnel1 * fFresnel1;float4 paintColor = fFresnel1 * flakeColor + fFresnel1Sq * flakeColor + fFresnel1Sq * fFresnel1Sq * flakeColor + pow( fFresnel2, 16 ) * flakeColor;

return float4( paintColor, 1.0 );}

Fetch initial perturbed normal vector from the noise mapCompute normal vectors for

both microflake layers

Compute dot productsof the normalized viewvector with the two

microflaker layer normals

Compose the microflakelayer color


Clear Gloss Coat


Dynamically Blurred Reflections

Blurred Reflections


Dynamic Blurring of Environment Map Reflections• A gloss map can be supplied to specify the

regions where reflections can be blurred

• Use bias when sampling the environment map to vary blurriness of the resulting reflections

• Use texCUBEbias for to access the cubic environment map

• For rough specular, the bias is high, causing a blurring effect

• Can also convert color fetched from environment map to luminance in rough trim areas


Clear Gloss Coat Pixel Shaderfloat4 ps_main( ... /* same inputs as in the previous shader */ ){ // ... use normal in world space (see Multi-tone pixel shader)

// Compute reflection vector:float fFresnel = saturate(dot( vNormalWorld, vView));float3 vReflection = 2 * vNormalWorld * fFresnel - vView;

float fEnvBias = glossLevel;

// Sample environment map using this reflection vector and bias:float4 envMap = texCUBEbias( showroomMap, float4( vReflection, fEnvBias ) );

// Premultiply by alpha: envMap.rgb = envMap.rgb * envMap.a;

// Brighten the environment map sampling result:envMap.rgb *= brightnessFactor;

// Combine result of environment map reflection with the paint // color:float fEnvContribution = 1.0 - 0.5 * fFresnel;

return float4( envMap.rgb * fEnvContribution, 1.0 );}

Compute the reflection vectorto fetch from the environment map

Shader parameter is used to dynamicallyblur the reflections by biasing

the texture fetch from the environment map

Premultiply by alpha channelof the environment map to avoid clamping highlights and brighten

the reflections

Resulting reflective highlights


Compositing Multi-Tone Base Layer and Microflake Layer

• Base color and flake effect are derived from Np1 and Np2 using the following polynomial:

color0(Np1·V) + color1(Np1·V)2 + color2(Np1·V)4 + color3(Np2·V)16

Base Color Flake


Compositing Final Look{

...

// Compute final paint color: combines all layers of paint as well// as two layers of microflakes:

float fFresnel1Sq = fFresnel1 * fFresnel1;

float4 paintColor = fFresnel1 * paintColor0 + fFresnel1Sq * paintColorMid + fFresnel1Sq * fFresnel1Sq * paintColor2 + pow( fFresnel2, 16 ) * flakeLayerColor;

// Combine result of environment map reflection with the paint // color:

float fEnvContribution = 1.0 - 0.5 * fNdotV;

// Assemble the final look:

float4 finalColor;

finalColor.a = 1.0;finalColor.rgb = envMap * fEnvContribution + paintColor;

return finalColor;

}


Original Hand-Tuned Assemblyps.2.0

def c0, 0.0, 0.5, 1.0, 2.0

def c1, 0.0, 0.0, 1.0, 0.0

dcl_2d s0

dcl_2d s1

dcl_cube s2

dcl_2d s3

dcl t0

dcl t1

dcl t2

dcl t3

dcl t4

dcl t5

texld r0, t0, s1

texld r8, t5, s3

mad r3, r8, c0.w, -c0.z

mad r6, r3, c4.r, r0

mad r7, r3, c4.g, r0

dp3 r4.a, t4, t4

rsq r4.a, r4.a

mul r4, t4, r4.a

mul r2.rgb, r0.x, t1

mad r2.rgb, r0.y, t2, r2

mad r2.rgb, r0.z, t3, r2

dp3 r2.a, r2, r2

rsq r2.a, r2.a

mul r2.rgb, r2, r2.a

dp3_sat r2.a, r2, r4

mul r3, r2, c0.w

mad r1.rgb, r2.a, r3, -r4

mov r1.a, c10.a

texldb r0, r1, s2




dp3 r10.a, r10, r10

rsq r10.a, r10.a






dp3 r10.a, r10, r10

rsq r10.a, r10.a




mul r0.rgb, r0, c2.r

mov r4.a, r6.a

mul r4.rgb, r4.a, c5

mul r4.a, r4.a, r4.a

mad r4.rgb, r4.a, c6, r4

mul r4.a, r4.a, r4.a


pow r4.a, r7.a, c4.b


mad r1.a, r2.a, c2.z, c2.w

mad r6.rgb, r0, r1.a, r4

mov oC0, r6

40 ALU ops3 Tex Fetches

43 Total


Car Paint Shader HLSL Compiler Disassembly Outputps_2_0

def c9, 0.5, 1, 0, 0

def c10, 2, -1, 16, 1

dcl t0.xy

dcl t1.xyz

dcl t2.xyz

dcl t3.xyz

dcl t4.xyz

dcl t5.xy

dcl_2d s0

dcl_2d s1

dcl_cube s2

texld r0, t0, s1

mad r5.xyz, c10.x, r0, c10.y

mul r0.xyz, r5.y, t2

dp3 r1.x, t4, t4

mad r0.xyz, t1, r5.x, r0

rsq r0.w, r1.x

mad r1.xyz, t3, r5.z, r0

mul r3.xyz, r0.w, t4

nrm r0.xyz, r1

dp3_sat r6.x, r0, r3

mul r0.xyz, r0, r6.x

add r0.xyz, r0, r0

mad r0.xyz, t4, -r0.w, r0

mov r0.w, c8.x

texld r1, t5, s0

texldb r0, r0, s2


mul r1.xyz, r5, c2.x

mad r1.xyz, c3.x, r2, r1



add r2.xyz, r5, r2


nrm r1.xyz, r4




mul r1.w, r5.x, r5.x


mul r1.xyz, r1.w, c6


mul r1.w, r1.w, r1.w

nrm r2.xyz, r4

mad r1.xyz, r5.x, c4, r1


mad r1.xyz, r1.w, c5, r1

pow r1.w, r2.x, c10.z

mad r1.xyz, r1.w, c1, r1

mul r0.xyz, r0.w, r0

mad r0.w, r6.x, -c9.x, c9.y


mad r0.xyz, r0, r0.w, r1

mov r0.w, c10.w

mov oC0, r0

38 ALU ops3 Tex Fetches

41 Total !


Full Result of the Application of Multi-Layer Paint to Car Body


Translucent Iridescent Shader: Butterfly Wings


Translucent Iridescent Shader: Butterfly Wings

• Simulates translucency of delicate butterfly wings– Wings glow from scattered reflected light– Similar to the effect of softly backlit rice paper

• Displays subtle iridescent lighting – Similar to rainbow pattern on the surface of soap bubbles– Caused by the interference of light waves resulting from

multiple reflections of light off of surfaces of varying thickness

• Combines gloss, opacity and normal maps for a multi-layered final look

– Gloss map contributes to satiny highlights– Opacity map allows portions of wings to be transparent– Normal map is used to give wings a bump-mapped look


RenderMonkey Butterfly Wings Shader Example

• Parameters that contribute to the translucency and iridescence look:

– Light position and scene ambient color

– Translucency coefficient

– Gloss scale and bias

– Scale and bias for speed of iridescence change

• Workspace:HLSL_IridescentButterly.xml


Translucent Iridescent Shader: Vertex Shader

..// Propagate input texture coordinates:

Out.Tex = Tex;

// Define tangent space matrix: float3x3 mTangentSpace; mTangentSpace[0] = Tangent; mTangentSpace[1] = Binormal; mTangentSpace[2] = Normal;

// Compute the light vector (object space): float3 vLight = normalize( mul( inv_view_matrix, lightPos ) - Pos ); // Output light vector in tangent space: Out.Light = mul( mTangentSpace, vLight ); // Compute the view vector (object space): float3 vView = normalize( mul( inv_view_matrix, float4(0,0,0,1)) - Pos );

// Output view vector in tangent space: Out.View = mul( mTangentSpace, vView ); // Compute the half angle vector (in tangent space): Out.Half = mul( mTangentSpace, normalize( vView + vLight ) );

return Out;

Define tangent space matrixCompute light vector intangent spaceCompute view vectorin tangent space

Compute Halfway vectorH = V + L

in tangent space


Translucent Iridescent Shader: Loading Information

float3 vNormal, baseColor;float fGloss, fTranslucency;

// Load normal and gloss map:float4( vNormal, fGloss ) = tex2D( bump_glossMap, Tex );

// Load base and opacity map:float4 (baseColor, fTranslucency) = tex2D( base_opacityMap, Tex );

Load normal from a normal map and gloss valuefrom a gloss map (combined in one texture map)

Load base texture color and alpha value from combined base and opacity texture map


Diffuse Illumination For Translucency

float3 scatteredIllumination = saturate(dot(-vNormal, Light)) * fTranslucency * translucencyCoeff;

float3 diffuseContribution = saturate(dot(vNormal,Light)) + ambient;

baseColor *= scatteredIllumination + diffuseContribution;

Light scattered on the butterfly wings iscomputed based on the negative normal(for scattering off the surface), light vectorand translucency coefficient and value forthe given pixel.

Compute diffusely reflected light usingthe bump-mapped normal and ambient

contribution

Combine diffuse and scattered light with base texture

*( +) =


Adding Opacity to ButterlyWings

Resulted color is modulated by the opacity value to add

transparency to the wings:

// Premultiply alpha blend to avoid clamping the highlights:baseColor *= fOpacity;

*=


Making Butterfly Wings Iridescent

// Compute index into the iridescence gradient map, which// consists of N*V coefficient float fGradientIndex = dot( vNormal, View) *

iridescence_speed_scale + iridescence_speed_bias;

// Load the iridescence value from the gradient map:float4 iridescence = tex1D( gradientMap, fGradientIndex );

Iridescence is a view-dependent effectScale and bias gradient map index to make

iridescence change quicker across the wingsSample gradient map based on the computed index

Resulting iridescence image:


Assembling Final Color

// Compute glossy highlights using values from gloss map:float fGlossValue = fGloss * ( saturate( dot( vNormal, Half )) *

gloss_scale + gloss_bias );

// Assemble the final color for the wingsbaseColor += fGlossValue * iridescence;

Assemble final wings colorCompute gloss value based on the original

gloss map input and < N, H> dot product


HLSL Disassembly Comparison to the Hand-Tuned Assembly

ps.2.0def c0, 0, .5, 1, 2

def c1, 4, 0, 0, 0

...

texld r1, t0, s1

mad r1.xyz, r1, c0.w, -c0.z

dp3_sat r4.y, r1, t2

dp3_sat r4.w, r1, -t2

texld r0, t0, s0

mul r4.w, r4.w, r0.a

mad r5.w, r4.w, c1.x, r4.y

add r5.rgb, r5.w, c3

mul r0.rgb, r0, r5

sub_sat r0.a, c0.z, r0.a

dp3 r6.xy, r1, t1

dp3_sat r6.y, r1, t3

mad r6.y, r6.y, c4.x, c4.y

mul r6.z, r6.y, r1.w

mad r6.x, r6.x, c4.z, c4.w

texld r2, r6, s2


mad r0.rgb, r6.z, r2, r0

mov oC0, r0

ps_2_0def c6, 2, -1, 1, 0

texld r0, t0, s1


dp3_sat r0.x, r2, t3

mov r1.w, c5.x

mad r1.w, r0.x, r1.w, c3.x

dp3 r0.x, r2, t1

mul r2.w, r0.w, r1.w

mov r0.w, c2.x

mad r0.xy, r0.x, r0.w, c0.x

texld r1, r0, s2

texld r0, t0, s0

dp3_sat r4.x, r2, t2

dp3_sat r3.x, -r2, t2

add r2.xyz, r4.x, c4

mul r1.w, r0.w, r3.x

mul r1.xyz, r2.w, r1

mad r2.xyz, r1.w, c1.x, r2

mul r0.xyz, r0, r2

add r0.w, -r0.w, c6.z

mad r0.xyz, r0, r0.w, r1

mov oC0, r0

Hand-Tuned Assembly Code HLSL Compiler-Generated Disassembly Code

12 ALU3 Texture15 Total

15 ALU3 Texture18 Total


Example of Translucent Iridescent Shader


Optimization Study: Überlight

• Flexible light described in JGT article “Lighting Controls for Computer Cinematography” by Ronen Barzel of Pixar

• Überlight is procedural and has many controls:– light type, intensity, light color, cuton, cutoff, near edge,

far edge, falloff, falloff distance, max intensity, parallel rays, shearx, sheary, width, height, width edge, height edge, roundness and beam distribution

• Code here is based upon the public domain RenderMan® implementation by Larry Gritz


Überlight Spotlight Mode

• Spotlight mode defines a procedural volume with smooth boundaries

• Shape of spotlight is made up of two nested superellipses which are swept along direction of light

• Also has smooth cuton and cutoff planes

• Can tune parameters to get all sorts of looks


Überlight Spotlight Volume

Roundness = ½


Überlight Spotlight Volume

Outer swept Outer swept superellipsesuperellipse

Inner swept Inner swept superellipsesuperellipse

aa

bb

Roundness = 1

BBAA


Original clipSuperellipse() routine

float clipSuperellipse ( float3 Q, // Test point on the x-y plane float a, // Inner superellipse float b, float A, // Outer superellipse float B, float roundness) // Same roundness for both ellipses{ float x = abs(Q.x), y = abs(Q.y);

float re = 2/roundness; // roundness exponent

float q = a * b * pow (pow(b*x, re) + pow(a*y, re), -1/re); float r = A * B * pow (pow(B*x, re) + pow(A*y, re), -1/re);

return smoothstep (q, r, 1);}

• Computes attenuation as a function of a point’s position in the swept superellipse.

• Directly ported from original RenderMan source• Compiles to 42 cycles in ps_2_0, 40 cycles on R3x0

Separate calculations of absolute value

Computes ellipse roundness exponent for every point


Vectorized Version

float clipSuperellipse ( float2 Q, // Test point on the x-y plane float4 aABb, // Dimensions of superellipses float2 r) // Two precomputed functions of roundness{ float2 qr, Qabs = abs(Q);

float2 bx_Bx = Qabs.x * aABb.wzyx; // Swizzle to unpack bB float2 ay_Ay = Qabs.y * aABb; qr.x = pow (pow(bx_Bx.x, r.x) + pow(ay_Ay.x, r.x), r.y); qr.y = pow (pow(bx_Bx.y, r.x) + pow(ay_Ay.y, r.x), r.y);

qr *= aABb * aABb.wzyx;

return smoothstep (qr.x, qr.y, 1);}

• Precompute functions of roundness in app• Vectorize abs() and all of the multiplications• Compiles to 33 cycles in ps_2_0, 28 cycles on R3x0

Vectorized computation of the absolute value

Contains precomputed 2/roundness and

–roundness / 2 parameters

Compute b * x and B * x in a single instruction

and a * y and A * y in another instruction

Final result computation that feeds into smoothstep() function


smoothstep() function

• Standard function in procedural shading• Intrinsics built into RenderMan and

DirectX HLSL:

0

1

edge0 edge1


C implementation

float smoothstep (float edge0, float edge1, float x){ if (x < edge0) return 0;

if (x >= edge1) return 1;

// Scale/bias into [0..1] range x = (x - edge0) / (edge1 - edge0);

return x * x * (3 - 2 * x);}


HLSL implementation

float smoothstep (float edge0, float edge1, float x){ // Scale, bias and saturate x to 0..1 range x = saturate((x - edge0) / (edge1 – edge0));

// Evaluate polynomial return x * x * (3 – 2 * x);}

• The free saturate handles x outside of [edge0..edge1] range


float3 smoothstep3 (float3 edge0, float3 edge1, float3 OneOverWidth, float3 x){ // Scale, bias and saturate x to [0..1] range x = saturate( (x - edge0) * OneOverWidth );

// Evaluate polynomial return x * x * (3 – 2 * x); }

Vectorized HLSL Implementation

float3 smoothstep3 (float3 edge0, float3 edge1, float3 OneOverWidth, float3 x){ // Scale, bias and saturate x to [0..1] range x = saturate( (x - edge0) * OneOverWidth );

// Evaluate polynomial return x * x * (3 – 2 * x); }

• With these optimizations, the entire spotlight volume computation of überlight compiles to 47 cycles in ps_2_0, 41 cycles on R3x0

Precompute 1 / (edge 1- edge 0)

Operations are performed on float3 to compute 3 different smoothstep operations

in parallel


Conclusion

• RenderMonkey IDE is a powerful, intuitive environment for developing shaders

• Prototype your shaders quickly

• Explore all parameters for your shaders using convenient GUI widgets

• Let your artists tweak shader parameters to find the desired look

• Develop shaders with your artists!


Presentation Summary

• Building effects in real-time with RenderMonkey

• Writing optimal HLSL code

• Shader Examples– Shipped with RenderMonkey version 1.0

see www.ati.com/developer

HLSL Car Paint.xml HLSL Iridescent Butterfly.xmll

advanced real-time shader techniques natalya tatarchuk 3d application research group ati research

Documents

realtime shader creation

san diego

shaderbased effects

ca rendermonkey database

ca program

rendermonkey xml

effect data

effect workspace