Download - Abstract Shade Trees
![Page 1: Abstract Shade Trees](https://reader036.vdocuments.net/reader036/viewer/2022062304/56812f4c550346895d94ddd5/html5/thumbnails/1.jpg)
Abstract Shade TreesAbstract Shade Trees
Morgan McGuire1 George Stathis2
Hanspeter Pfister2,3 Shriram Krisnamurthi1
1Brown University 2Harvard Extension School 3MERL
![Page 2: Abstract Shade Trees](https://reader036.vdocuments.net/reader036/viewer/2022062304/56812f4c550346895d94ddd5/html5/thumbnails/2.jpg)
Shaders
• Programs that color (“shade”) a pixel
• Often execute on a graphics card
• Pervasive in film and game rendering
[Proudfoot01]
![Page 3: Abstract Shade Trees](https://reader036.vdocuments.net/reader036/viewer/2022062304/56812f4c550346895d94ddd5/html5/thumbnails/3.jpg)
Shaders
• Programs that color (“shade”) a pixel
• Often execute on a graphics card
• Pervasive in film and game rendering
[Proudfoot01]
![Page 4: Abstract Shade Trees](https://reader036.vdocuments.net/reader036/viewer/2022062304/56812f4c550346895d94ddd5/html5/thumbnails/4.jpg)
Let’s author a pixel shader
![Page 5: Abstract Shade Trees](https://reader036.vdocuments.net/reader036/viewer/2022062304/56812f4c550346895d94ddd5/html5/thumbnails/5.jpg)
Shading Language
(Renderman, RTSL, HLSL, GLSL, Cg)// glass_ball.glsluniform float etaRatio;uniform float fresnelPower;uniform samplerCube environmentCubeMap;varying vec3 objectSpaceIncomingVector;varying vec3 n, b,t;uniform sampler2D normalMap;varying vec3 tangentSpaceViewVector;varying vec2 textureCoordinates;
void main(void) { float bump = ((texture2D(heightFieldTexture, textureCoordinates).a) + bumpBias) * bumpScale; vec2 bumpCoords = bump * vec2(tangentSpaceViewVector.x, tangentSpaceViewVector.y) + textureCoordinates;
vec3 tangentSpaceNormalVector = texture2D(normalMap, vec2(bumpCoords.x,bumpCoords.y)).xyz * 2.0 - 1.0;
mat3 tangentObjectSpaceMat = mat3(t,b,n); vec3 objectSpaceNormalVector = tangentObjectSpaceMat*tangentSpaceNormalVector; vec3 objectSpaceReflectedVector = normalize(reflect(objectSpaceIncomingVector,objectSpaceNormalVector)); vec4 color1 = textureCube(environmentCubeMap, objectSpaceReflectedVector);
float fresnelRatio = max(0.0,min(1.0, fresnelBias + fresnelScale* pow(1.0 + dot(normalize(objectSpaceIncomingVector), normalize(objectSpaceNormalVector)),fresnelPower))); vec3 objectSpaceRefractedVector = normalize(refract(objectSpaceIncomingVector,objectSpaceNormalVector,etaRatio)); vec4 color2 = textureCube(environmentCubeMap, objectSpaceRefractedVector); vec4 mixedColorOutput = mix(color1,color2,fresnelRatio);
gl_FragColor = mixedColorOutput;}
![Page 6: Abstract Shade Trees](https://reader036.vdocuments.net/reader036/viewer/2022062304/56812f4c550346895d94ddd5/html5/thumbnails/6.jpg)
Shade Tree for Visual Programming[Cook84; Abram+Whitted90; Borau04; Unreal06; Hargreaves04]
Parameter
Function call
![Page 7: Abstract Shade Trees](https://reader036.vdocuments.net/reader036/viewer/2022062304/56812f4c550346895d94ddd5/html5/thumbnails/7.jpg)
Authoring in a production context…
![Page 8: Abstract Shade Trees](https://reader036.vdocuments.net/reader036/viewer/2022062304/56812f4c550346895d94ddd5/html5/thumbnails/8.jpg)
Shader Authoring Problems1. Dependent workflow
– Artists define and use
– Programmers implement
– Frequently block on each other
2. Shaders are rarely modular– Overlapping features [Dijkstra76]
– Exacerbated by restrictive execution environments
– Programmers frequently rewrite from scratch
Artist Programmer
![Page 9: Abstract Shade Trees](https://reader036.vdocuments.net/reader036/viewer/2022062304/56812f4c550346895d94ddd5/html5/thumbnails/9.jpg)
Shader Authoring Problems1. Dependent workflow
– Artists define and use
– Programmers implement
– Frequently block on each other
2. Shaders are rarely modular– Overlapping features [Dijkstra76]
– Exacerbated by restrictive execution environments
– To combine features, programmers frequently rewrite both from scratch
Artist Programmer
void main() {
![Page 10: Abstract Shade Trees](https://reader036.vdocuments.net/reader036/viewer/2022062304/56812f4c550346895d94ddd5/html5/thumbnails/10.jpg)
New authoring idea…
![Page 11: Abstract Shade Trees](https://reader036.vdocuments.net/reader036/viewer/2022062304/56812f4c550346895d94ddd5/html5/thumbnails/11.jpg)
Shade Tree
Abstract…
Abstract Shade TreeSuperposition
Reflection Refraction
Parallax Bump Mapping
![Page 12: Abstract Shade Trees](https://reader036.vdocuments.net/reader036/viewer/2022062304/56812f4c550346895d94ddd5/html5/thumbnails/12.jpg)
Abstract Shade Tree
Superposition
Reflection Refraction
Parallax Bump Mapping
• Node = Code atom– Like functions with funny scopes
(See the paper for details)
• Arrow = Dependency– Not parameters!
• Outline = Feature– Note the overlap at Fresnel
• Requires a weaver to convert into executable code
![Page 13: Abstract Shade Trees](https://reader036.vdocuments.net/reader036/viewer/2022062304/56812f4c550346895d94ddd5/html5/thumbnails/13.jpg)
New Workflow
1. Programmer writes atoms2. Both can visually combine atoms into features3. Artist visually combines features into trees4. Weaver synthesizes:
– Shader entry and exit points– Program structure– Parameter linkage– Type coercions– “Glue” code
5. Rendered in IDE for feedback
![Page 14: Abstract Shade Trees](https://reader036.vdocuments.net/reader036/viewer/2022062304/56812f4c550346895d94ddd5/html5/thumbnails/14.jpg)
Problems Addressed
1. Workflow dependency reduced– Programmer writes atoms once– Artist can independently generate many shaders– Abstract Tree is independent of atom signatures
2. Modularity improved– Visualized feature interaction– Weaver resolves overlapping features– Abstract Tree is independent of atom signatures
![Page 15: Abstract Shade Trees](https://reader036.vdocuments.net/reader036/viewer/2022062304/56812f4c550346895d94ddd5/html5/thumbnails/15.jpg)
Related Feature Work• Features as abstractions
[Parnas72; Dijkstra76; Turner99]
• Feature/Aspect-oriented programming
[Batory92; Kiczales97; Batory04]
• Software product lines
[Clements02]
• Sh Language, Combining shaders
[McCool et al. 02, McCool et al. 04]
• Über-Shader alternative
[Barzel97/Gritz98; Pellacini+Vidimce04; McGuire05]
![Page 16: Abstract Shade Trees](https://reader036.vdocuments.net/reader036/viewer/2022062304/56812f4c550346895d94ddd5/html5/thumbnails/16.jpg)
The Weaver
![Page 17: Abstract Shade Trees](https://reader036.vdocuments.net/reader036/viewer/2022062304/56812f4c550346895d94ddd5/html5/thumbnails/17.jpg)
Weaving Algorithm
• Input: Abstract shade tree• Output: GLSL shader
1. -Rename parameters2. Topologically sort nodes3. Match inputs to outputs4. Generate code5. Generate boilerplate
![Page 18: Abstract Shade Trees](https://reader036.vdocuments.net/reader036/viewer/2022062304/56812f4c550346895d94ddd5/html5/thumbnails/18.jpg)
Match Inputs to Outputs
Lambertian Shading
Normal Map
Some node in the middle of an Abstract Shade Tree
![Page 19: Abstract Shade Trees](https://reader036.vdocuments.net/reader036/viewer/2022062304/56812f4c550346895d94ddd5/html5/thumbnails/19.jpg)
Match Inputs to Outputs
OWT
Lambertian Shading
Transform
Normalize
ON
Normal Map
WN̂
WL̂
WN
12
1ON
Unpack
The abstracted parameters the Weaver must reconstruct
![Page 20: Abstract Shade Trees](https://reader036.vdocuments.net/reader036/viewer/2022062304/56812f4c550346895d94ddd5/html5/thumbnails/20.jpg)
Lambertian Shading
Transform
Normalize
ON
Normal Map
WN̂
WL̂
WN
12
1ON
Unpack
Match Inputs to Outputs
Globals
Basis Change
Upstream Output
Storage Class Change
OWT
The abstracted parameters the Weaver must reconstruct
![Page 21: Abstract Shade Trees](https://reader036.vdocuments.net/reader036/viewer/2022062304/56812f4c550346895d94ddd5/html5/thumbnails/21.jpg)
Matching Algorithm
• Bind each input parameter to the “closest” matching output parameter of some ancestor node
• “Closest” = shortest/fastest type coercion
• Typical GLSL types are just storage classes– e.g., float3 vector color point …– float LambertianShading(float3 normal, float3 light)
– Not much information in “float3”
• Need a richer type system…
![Page 22: Abstract Shade Trees](https://reader036.vdocuments.net/reader036/viewer/2022062304/56812f4c550346895d94ddd5/html5/thumbnails/22.jpg)
Semantic Types
• Encode information like “world-space vector” in types
• Two variables with precisely the same semantic type are likely semantically interchangeable
Semantic Types
1
TLK JavaC/GLSL
Too general Useful Too specificNumber of types
![Page 23: Abstract Shade Trees](https://reader036.vdocuments.net/reader036/viewer/2022062304/56812f4c550346895d94ddd5/html5/thumbnails/23.jpg)
Some Semantic Types for Shading
Float1 LambertianShading(UnitWorldNormalFloat3 normal, UnitWorldVectorFloat3 light)
Dimension: {2, 3, 4, any }
Length: {Unit, any }
Basis: {Tangent, Object, World, Screen, any }
Interpretation: {Color, Texcoord, Normal, Point, Vector, any}
Precision: {float32, int32, Boolean, any }
• Type coercion now includes normalization, basis change, unpacking, storage change, etc.
• “any” allows polymorphism• Could add “meters,” “non-negative,” etc.
![Page 24: Abstract Shade Trees](https://reader036.vdocuments.net/reader036/viewer/2022062304/56812f4c550346895d94ddd5/html5/thumbnails/24.jpg)
Parameter Matching is Semantic Coercion Tree Search
VEC3___O_NOR
NormalWorldToEyeSpace
VEC3___W_NOR
NormalTangentToObjectSpace
VEC3___T_NOR
VEC3___O_NOR
VEC3___E_NOR
VEC3___T_NOR
VEC3___O_NOR
NormalObjectToTangentSpace NormalObjectToWorldSpace
NormalWorldToObjectSpace NormalTangentToObjectSpace
![Page 25: Abstract Shade Trees](https://reader036.vdocuments.net/reader036/viewer/2022062304/56812f4c550346895d94ddd5/html5/thumbnails/25.jpg)
Weaver Properties
• Always terminates• Bounded (and small) memory requirements• Always generates a legal program
• …but sometimes it is the wrong program• Most common error: Parameter aliasing
– Artist missed a dependency– Programmer assigned an overly generic type– Anecdotally infrequent
![Page 26: Abstract Shade Trees](https://reader036.vdocuments.net/reader036/viewer/2022062304/56812f4c550346895d94ddd5/html5/thumbnails/26.jpg)
Results
![Page 27: Abstract Shade Trees](https://reader036.vdocuments.net/reader036/viewer/2022062304/56812f4c550346895d94ddd5/html5/thumbnails/27.jpg)
Bumpy Glass
![Page 28: Abstract Shade Trees](https://reader036.vdocuments.net/reader036/viewer/2022062304/56812f4c550346895d94ddd5/html5/thumbnails/28.jpg)
Bumpy GlassManual
(GLSL)
Automatic
(AST)
Final Source Length 58 lines 188 lines
Assembled 41 instr. 46 instr.
Radeon 9700 @ 512 x 512
245 fps 240 fps
1024 x 768 50 fps 50 fps
User experience?
![Page 29: Abstract Shade Trees](https://reader036.vdocuments.net/reader036/viewer/2022062304/56812f4c550346895d94ddd5/html5/thumbnails/29.jpg)
IDE
![Page 30: Abstract Shade Trees](https://reader036.vdocuments.net/reader036/viewer/2022062304/56812f4c550346895d94ddd5/html5/thumbnails/30.jpg)
Phong Illum. + Parallax Map + Texture Map
![Page 31: Abstract Shade Trees](https://reader036.vdocuments.net/reader036/viewer/2022062304/56812f4c550346895d94ddd5/html5/thumbnails/31.jpg)
Anisotropic + Diffuse Lighting
![Page 32: Abstract Shade Trees](https://reader036.vdocuments.net/reader036/viewer/2022062304/56812f4c550346895d94ddd5/html5/thumbnails/32.jpg)
Phong Illum. + Projective Light + Shadow Map
![Page 33: Abstract Shade Trees](https://reader036.vdocuments.net/reader036/viewer/2022062304/56812f4c550346895d94ddd5/html5/thumbnails/33.jpg)
Summary of Contributions
• Abstract shade trees
• Semantic type system
• Weaver for shader synthesis
• Visual editor
![Page 34: Abstract Shade Trees](https://reader036.vdocuments.net/reader036/viewer/2022062304/56812f4c550346895d94ddd5/html5/thumbnails/34.jpg)
Future Directions
• Consider performance model during parameter matching
• Beyond pixel shaders• Other interesting semantic types
• How do you debug abstract trees?
• Applications to general scripting
![Page 35: Abstract Shade Trees](https://reader036.vdocuments.net/reader036/viewer/2022062304/56812f4c550346895d94ddd5/html5/thumbnails/35.jpg)
Thanks
• NVIDIA
• MERL
• NSF
• Iron Lore Entertainment
![Page 36: Abstract Shade Trees](https://reader036.vdocuments.net/reader036/viewer/2022062304/56812f4c550346895d94ddd5/html5/thumbnails/36.jpg)