developing a fractal experience at 90 fps conemarching in vr...any gpu stall will kill the...
TRANSCRIPT
![Page 1: Developing a Fractal experience at 90 FPS Conemarching in VR...Any GPU stall will kill the experience If we can’t hit the target framerate, time warping kicks in, halving fps. We](https://reader036.vdocuments.net/reader036/viewer/2022090611/60784b1243ac66238a311ac9/html5/thumbnails/1.jpg)
/ Framestore
Conemarching in VR Developing a Fractal experience at 90 FPS
Johannes SaamMariano Merchante
FRAMESTORE
![Page 2: Developing a Fractal experience at 90 FPS Conemarching in VR...Any GPU stall will kill the experience If we can’t hit the target framerate, time warping kicks in, halving fps. We](https://reader036.vdocuments.net/reader036/viewer/2022090611/60784b1243ac66238a311ac9/html5/thumbnails/2.jpg)
THE CONCEPT
![Page 3: Developing a Fractal experience at 90 FPS Conemarching in VR...Any GPU stall will kill the experience If we can’t hit the target framerate, time warping kicks in, halving fps. We](https://reader036.vdocuments.net/reader036/viewer/2022090611/60784b1243ac66238a311ac9/html5/thumbnails/3.jpg)
THE CONCEPT FRACTALS ANDCOLLISIONS
![Page 4: Developing a Fractal experience at 90 FPS Conemarching in VR...Any GPU stall will kill the experience If we can’t hit the target framerate, time warping kicks in, halving fps. We](https://reader036.vdocuments.net/reader036/viewer/2022090611/60784b1243ac66238a311ac9/html5/thumbnails/4.jpg)
THE CONCEPT FRACTALS ANDCOLLISIONS
RAYMARCHING AND VR
![Page 5: Developing a Fractal experience at 90 FPS Conemarching in VR...Any GPU stall will kill the experience If we can’t hit the target framerate, time warping kicks in, halving fps. We](https://reader036.vdocuments.net/reader036/viewer/2022090611/60784b1243ac66238a311ac9/html5/thumbnails/5.jpg)
THE CONCEPT FRACTALS ANDCOLLISIONS
RAYMARCHING AND VR THE SHADING CHALLENGE
![Page 6: Developing a Fractal experience at 90 FPS Conemarching in VR...Any GPU stall will kill the experience If we can’t hit the target framerate, time warping kicks in, halving fps. We](https://reader036.vdocuments.net/reader036/viewer/2022090611/60784b1243ac66238a311ac9/html5/thumbnails/6.jpg)
THE CONCEPT
![Page 7: Developing a Fractal experience at 90 FPS Conemarching in VR...Any GPU stall will kill the experience If we can’t hit the target framerate, time warping kicks in, halving fps. We](https://reader036.vdocuments.net/reader036/viewer/2022090611/60784b1243ac66238a311ac9/html5/thumbnails/7.jpg)
● Explore classic fractals in real-time VR
● Simple interaction model, sandbox feel
● Break a few rules!
THE CONCEPT
![Page 8: Developing a Fractal experience at 90 FPS Conemarching in VR...Any GPU stall will kill the experience If we can’t hit the target framerate, time warping kicks in, halving fps. We](https://reader036.vdocuments.net/reader036/viewer/2022090611/60784b1243ac66238a311ac9/html5/thumbnails/8.jpg)
HOW DO WE TRANSMIT SENSE OF SCALE?
Concept ○ Fractals ○ Collisions ○ Raymarching and VR ○ Shading tricks
![Page 9: Developing a Fractal experience at 90 FPS Conemarching in VR...Any GPU stall will kill the experience If we can’t hit the target framerate, time warping kicks in, halving fps. We](https://reader036.vdocuments.net/reader036/viewer/2022090611/60784b1243ac66238a311ac9/html5/thumbnails/9.jpg)
HOW DO WE TRANSMIT SENSE OF SCALE? THE LOCOMOTION CHALLENGE
Concept ○ Fractals ○ Collisions ○ Raymarching and VR ○ Shading tricks
![Page 10: Developing a Fractal experience at 90 FPS Conemarching in VR...Any GPU stall will kill the experience If we can’t hit the target framerate, time warping kicks in, halving fps. We](https://reader036.vdocuments.net/reader036/viewer/2022090611/60784b1243ac66238a311ac9/html5/thumbnails/10.jpg)
● Scale the player’s eyes as they move around
HOW DO WE TRANSMIT SENSE OF SCALE?
Concept ○ Fractals ○ Collisions ○ Raymarching and VR ○ Shading tricks
![Page 11: Developing a Fractal experience at 90 FPS Conemarching in VR...Any GPU stall will kill the experience If we can’t hit the target framerate, time warping kicks in, halving fps. We](https://reader036.vdocuments.net/reader036/viewer/2022090611/60784b1243ac66238a311ac9/html5/thumbnails/11.jpg)
● Scale the player’s eyes as they move around
● Make sure we smooth scale to prevent discontinuities
HOW DO WE TRANSMIT SENSE OF SCALE?
Concept ○ Fractals ○ Collisions ○ Raymarching and VR ○ Shading tricks
![Page 12: Developing a Fractal experience at 90 FPS Conemarching in VR...Any GPU stall will kill the experience If we can’t hit the target framerate, time warping kicks in, halving fps. We](https://reader036.vdocuments.net/reader036/viewer/2022090611/60784b1243ac66238a311ac9/html5/thumbnails/12.jpg)
● Scale the player’s eyes as they move around
● Make sure we smooth scale to prevent discontinuities
● Design shading such that there is enough size contrast
Concept ○ Fractals ○ Collisions ○ Raymarching and VR ○ Shading tricks
![Page 13: Developing a Fractal experience at 90 FPS Conemarching in VR...Any GPU stall will kill the experience If we can’t hit the target framerate, time warping kicks in, halving fps. We](https://reader036.vdocuments.net/reader036/viewer/2022090611/60784b1243ac66238a311ac9/html5/thumbnails/13.jpg)
● Scale the player’s eyes as they move around
● Make sure we smooth scale to prevent discontinuities
● Design shading such that there is enough size contrast
● Loop each fractal to emphasize the sense of infinity
Concept ○ Fractals ○ Collisions ○ Raymarching and VR ○ Shading tricks
![Page 14: Developing a Fractal experience at 90 FPS Conemarching in VR...Any GPU stall will kill the experience If we can’t hit the target framerate, time warping kicks in, halving fps. We](https://reader036.vdocuments.net/reader036/viewer/2022090611/60784b1243ac66238a311ac9/html5/thumbnails/14.jpg)
● Always move in the direction of your head…
THE LOCOMOTION CHALLENGE
![Page 15: Developing a Fractal experience at 90 FPS Conemarching in VR...Any GPU stall will kill the experience If we can’t hit the target framerate, time warping kicks in, halving fps. We](https://reader036.vdocuments.net/reader036/viewer/2022090611/60784b1243ac66238a311ac9/html5/thumbnails/15.jpg)
● Always move in the direction of your head…
● … unless you prefer to strafe
THE LOCOMOTION CHALLENGE
![Page 16: Developing a Fractal experience at 90 FPS Conemarching in VR...Any GPU stall will kill the experience If we can’t hit the target framerate, time warping kicks in, halving fps. We](https://reader036.vdocuments.net/reader036/viewer/2022090611/60784b1243ac66238a311ac9/html5/thumbnails/16.jpg)
● Always move in the direction of your head…
● … unless you prefer to strafe
● Velocity tied to scale and closest distance THE LOCOMOTION CHALLENGE
![Page 17: Developing a Fractal experience at 90 FPS Conemarching in VR...Any GPU stall will kill the experience If we can’t hit the target framerate, time warping kicks in, halving fps. We](https://reader036.vdocuments.net/reader036/viewer/2022090611/60784b1243ac66238a311ac9/html5/thumbnails/17.jpg)
● Always move in the direction of your head…
● … unless you prefer to strafe
● Velocity tied to scale and closest distance
● Option to rotate 90 degrees with fade
THE LOCOMOTION CHALLENGE
![Page 18: Developing a Fractal experience at 90 FPS Conemarching in VR...Any GPU stall will kill the experience If we can’t hit the target framerate, time warping kicks in, halving fps. We](https://reader036.vdocuments.net/reader036/viewer/2022090611/60784b1243ac66238a311ac9/html5/thumbnails/18.jpg)
● Always move in the direction of your head…
● … unless you prefer to strafe
● Velocity tied to scale and closest distance
● Option to rotate 90 degrees with fade
● Vignette intensity directly proportional to velocity and angular velocity.
THE LOCOMOTION CHALLENGE
![Page 19: Developing a Fractal experience at 90 FPS Conemarching in VR...Any GPU stall will kill the experience If we can’t hit the target framerate, time warping kicks in, halving fps. We](https://reader036.vdocuments.net/reader036/viewer/2022090611/60784b1243ac66238a311ac9/html5/thumbnails/19.jpg)
FRACTALS AND COLLISIONS
![Page 20: Developing a Fractal experience at 90 FPS Conemarching in VR...Any GPU stall will kill the experience If we can’t hit the target framerate, time warping kicks in, halving fps. We](https://reader036.vdocuments.net/reader036/viewer/2022090611/60784b1243ac66238a311ac9/html5/thumbnails/20.jpg)
FRACTALS
● Require a distance field to be marched
● Generally very expensive!○ Usually have multiple iterations for a single
estimation
Concept ○ Fractals ○ Collisions ○ Raymarching and VR ○ Shading tricks
![Page 21: Developing a Fractal experience at 90 FPS Conemarching in VR...Any GPU stall will kill the experience If we can’t hit the target framerate, time warping kicks in, halving fps. We](https://reader036.vdocuments.net/reader036/viewer/2022090611/60784b1243ac66238a311ac9/html5/thumbnails/21.jpg)
FRACTALS
● Require a distance field to be marched
● Generally very expensive!○ Usually have multiple iterations for a single
estimation
● Scale detail as player approaches fractal○ Rendering more expensive as we get smaller!
Concept ○ Fractals ○ Collisions ○ Raymarching and VR ○ Shading tricks
![Page 22: Developing a Fractal experience at 90 FPS Conemarching in VR...Any GPU stall will kill the experience If we can’t hit the target framerate, time warping kicks in, halving fps. We](https://reader036.vdocuments.net/reader036/viewer/2022090611/60784b1243ac66238a311ac9/html5/thumbnails/22.jpg)
FRACTALS
● Require a distance field to be marched
● Generally very expensive!○ Usually have multiple iterations for a single
estimation
● Scale detail as player approaches fractal○ Rendering more expensive as we get smaller!
● Hard to predict and high frequency shapes
● Scaling leads to floating point precision problems!○ Theoretically infinite detail
Concept ○ Fractals ○ Collisions ○ Raymarching and VR ○ Shading tricks
![Page 23: Developing a Fractal experience at 90 FPS Conemarching in VR...Any GPU stall will kill the experience If we can’t hit the target framerate, time warping kicks in, halving fps. We](https://reader036.vdocuments.net/reader036/viewer/2022090611/60784b1243ac66238a311ac9/html5/thumbnails/23.jpg)
COLLISIONS
● We already use the fractal SDF for rendering
● We can use the same SDF for collisions!○ R: distance, GBA: normal for bounce
● Render a 1x1 texture and read it asynchronously (Thanks Unity!)
○ Prevent GPU stall due to VR context
● Why not a compute shader?○ Why not CPU?
● Predict a bit due to latency
Concept ○ Fractals ○ Collisions ○ Raymarching and VR ○ Shading tricks
![Page 24: Developing a Fractal experience at 90 FPS Conemarching in VR...Any GPU stall will kill the experience If we can’t hit the target framerate, time warping kicks in, halving fps. We](https://reader036.vdocuments.net/reader036/viewer/2022090611/60784b1243ac66238a311ac9/html5/thumbnails/24.jpg)
COLLISIONS
● We already use the fractal SDF for rendering
● We can use the same SDF for collisions!○ R: distance, GBA: normal for bounce
● Render a 1x1 texture and read it asynchronously (Thanks Unity!)
○ Prevent GPU stall due to VR context
● Why not a compute shader?○ Why not CPU?
● Predict a bit due to latency
Concept ○ Fractals ○ Collisions ○ Raymarching and VR ○ Shading tricks
![Page 25: Developing a Fractal experience at 90 FPS Conemarching in VR...Any GPU stall will kill the experience If we can’t hit the target framerate, time warping kicks in, halving fps. We](https://reader036.vdocuments.net/reader036/viewer/2022090611/60784b1243ac66238a311ac9/html5/thumbnails/25.jpg)
COLLISIONS
● We already use the fractal SDF for rendering
● We can use the same SDF for collisions!○ R: distance, GBA: normal for bounce
● Render a 1x1 texture and read it asynchronously (Thanks Unity!)
○ Prevent GPU stall due to VR context
● Why not a compute shader?○ Why not CPU?
● Predict a bit due to latency
Concept ○ Fractals ○ Collisions ○ Raymarching and VR ○ Shading tricks
![Page 26: Developing a Fractal experience at 90 FPS Conemarching in VR...Any GPU stall will kill the experience If we can’t hit the target framerate, time warping kicks in, halving fps. We](https://reader036.vdocuments.net/reader036/viewer/2022090611/60784b1243ac66238a311ac9/html5/thumbnails/26.jpg)
COLLISIONS
● We already use the fractal SDF for rendering
● We can use the same SDF for collisions!○ R: distance, GBA: normal for bounce
● Render a 1x1 texture and read it asynchronously (Thanks Unity!)
○ Prevent GPU stall due to VR context
● Why not a compute shader?○ Why not CPU?
● Predict a bit due to latency
Concept ○ Fractals ○ Collisions ○ Raymarching and VR ○ Shading tricks
![Page 27: Developing a Fractal experience at 90 FPS Conemarching in VR...Any GPU stall will kill the experience If we can’t hit the target framerate, time warping kicks in, halving fps. We](https://reader036.vdocuments.net/reader036/viewer/2022090611/60784b1243ac66238a311ac9/html5/thumbnails/27.jpg)
COLLISIONS
● We already use the fractal SDF for rendering
● We can use the same SDF for collisions!○ R: distance, GBA: normal for bounce
● Render a 1x1 texture and read it asynchronously (Thanks Unity!)
○ Prevent GPU stall due to VR context
● Why not a compute shader?○ Why not CPU?
● Predict a bit due to latency
Concept ○ Fractals ○ Collisions ○ Raymarching and VR ○ Shading tricks
![Page 28: Developing a Fractal experience at 90 FPS Conemarching in VR...Any GPU stall will kill the experience If we can’t hit the target framerate, time warping kicks in, halving fps. We](https://reader036.vdocuments.net/reader036/viewer/2022090611/60784b1243ac66238a311ac9/html5/thumbnails/28.jpg)
FIRST STEPS: MANDELBULB
Concept ○ Fractals ○ Collisions ○ Raymarching and VR ○ Shading tricks
![Page 29: Developing a Fractal experience at 90 FPS Conemarching in VR...Any GPU stall will kill the experience If we can’t hit the target framerate, time warping kicks in, halving fps. We](https://reader036.vdocuments.net/reader036/viewer/2022090611/60784b1243ac66238a311ac9/html5/thumbnails/29.jpg)
FIRST STEPS: MANDELBULB
Concept ○ Fractals ○ Collisions ○ Raymarching and VR ○ Shading tricks
● Famous 3D fractal developed by Daniel White and Paul Nylander
○ The fractalforums thread is amazing!
![Page 30: Developing a Fractal experience at 90 FPS Conemarching in VR...Any GPU stall will kill the experience If we can’t hit the target framerate, time warping kicks in, halving fps. We](https://reader036.vdocuments.net/reader036/viewer/2022090611/60784b1243ac66238a311ac9/html5/thumbnails/30.jpg)
FIRST STEPS: MANDELBULB
Concept ○ Fractals ○ Collisions ○ Raymarching and VR ○ Shading tricks
● Famous 3D fractal developed by Daniel White and Paul Nylander
○ The fractalforums thread is amazing!
● Decided to use Íñigo Quílez’s formula, but found bubbles of overestimation for Julia offsets
○ We built a tool for finding these bubbles through stochastic brute force
![Page 31: Developing a Fractal experience at 90 FPS Conemarching in VR...Any GPU stall will kill the experience If we can’t hit the target framerate, time warping kicks in, halving fps. We](https://reader036.vdocuments.net/reader036/viewer/2022090611/60784b1243ac66238a311ac9/html5/thumbnails/31.jpg)
Concept ○ Fractals ○ Collisions ○ Raymarching and VR ○ Shading tricks
● Famous 3D fractal developed by Daniel White and Paul Nylander
○ The fractalforums thread is amazing!
● Decided to use Íñigo Quílez’s formula, but found bubbles of overestimation for Julia offsets
● Tweaked the formula to prevent these bubbles○ Kept the maximum derivative as we iterate
![Page 32: Developing a Fractal experience at 90 FPS Conemarching in VR...Any GPU stall will kill the experience If we can’t hit the target framerate, time warping kicks in, halving fps. We](https://reader036.vdocuments.net/reader036/viewer/2022090611/60784b1243ac66238a311ac9/html5/thumbnails/32.jpg)
Concept ○ Fractals ○ Collisions ○ Raymarching and VR ○ Shading tricks
● Famous 3D fractal developed by Daniel White and Paul Nylander
○ The fractalforums thread is amazing!
● Decided to use Íñigo Quílez’s formula, but found bubbles of overestimation for Julia offsets
● Tweaked the formula to prevent these bubbles○ Kept the maximum derivative as we iterate
● More iterations needed to render, but overall better than reducing the epsilon
![Page 33: Developing a Fractal experience at 90 FPS Conemarching in VR...Any GPU stall will kill the experience If we can’t hit the target framerate, time warping kicks in, halving fps. We](https://reader036.vdocuments.net/reader036/viewer/2022090611/60784b1243ac66238a311ac9/html5/thumbnails/33.jpg)
Concept ○ Fractals ○ Collisions ○ Raymarching and VR ○ Shading tricks
https://www.shadertoy.com/view/MdSBDR
![Page 34: Developing a Fractal experience at 90 FPS Conemarching in VR...Any GPU stall will kill the experience If we can’t hit the target framerate, time warping kicks in, halving fps. We](https://reader036.vdocuments.net/reader036/viewer/2022090611/60784b1243ac66238a311ac9/html5/thumbnails/34.jpg)
RAYMARCHING AND VR
![Page 35: Developing a Fractal experience at 90 FPS Conemarching in VR...Any GPU stall will kill the experience If we can’t hit the target framerate, time warping kicks in, halving fps. We](https://reader036.vdocuments.net/reader036/viewer/2022090611/60784b1243ac66238a311ac9/html5/thumbnails/35.jpg)
CLASSIC RAYMARCHING
● Define a function that gives us the distance to a surface
Concept ○ Fractals ○ Collisions ○ Raymarching and VR ○ Shading tricks
![Page 36: Developing a Fractal experience at 90 FPS Conemarching in VR...Any GPU stall will kill the experience If we can’t hit the target framerate, time warping kicks in, halving fps. We](https://reader036.vdocuments.net/reader036/viewer/2022090611/60784b1243ac66238a311ac9/html5/thumbnails/36.jpg)
● Define a function that gives us the distance to a surface
Concept ○ Fractals ○ Collisions ○ Raymarching and VR ○ Shading tricks
![Page 37: Developing a Fractal experience at 90 FPS Conemarching in VR...Any GPU stall will kill the experience If we can’t hit the target framerate, time warping kicks in, halving fps. We](https://reader036.vdocuments.net/reader036/viewer/2022090611/60784b1243ac66238a311ac9/html5/thumbnails/37.jpg)
● Define a function that gives us the distance to a surface
● Iterate through a ray with a fixed step
Concept ○ Fractals ○ Collisions ○ Raymarching and VR ○ Shading tricks
![Page 38: Developing a Fractal experience at 90 FPS Conemarching in VR...Any GPU stall will kill the experience If we can’t hit the target framerate, time warping kicks in, halving fps. We](https://reader036.vdocuments.net/reader036/viewer/2022090611/60784b1243ac66238a311ac9/html5/thumbnails/38.jpg)
● Define a function that gives us the distance to a surface
● Iterate through a ray with a fixed step
● If the distance to the surface is less than a specified amount, we hit the surface
Concept ○ Fractals ○ Collisions ○ Raymarching and VR ○ Shading tricks
![Page 39: Developing a Fractal experience at 90 FPS Conemarching in VR...Any GPU stall will kill the experience If we can’t hit the target framerate, time warping kicks in, halving fps. We](https://reader036.vdocuments.net/reader036/viewer/2022090611/60784b1243ac66238a311ac9/html5/thumbnails/39.jpg)
● Define a function that gives us the distance to a surface
● Iterate through a ray with a fixed step
● If the distance to the surface is less than a specified amount, we hit the surface
● Use finite differences for normal estimation and shading
● Shadows, AO, SSS, godrays very straightforward
● Very expensive!
Concept ○ Fractals ○ Collisions ○ Raymarching and VR ○ Shading tricks
![Page 40: Developing a Fractal experience at 90 FPS Conemarching in VR...Any GPU stall will kill the experience If we can’t hit the target framerate, time warping kicks in, halving fps. We](https://reader036.vdocuments.net/reader036/viewer/2022090611/60784b1243ac66238a311ac9/html5/thumbnails/40.jpg)
SPHERE TRACING
● Each step jump is proportional to the distance of the surface
Concept ○ Fractals ○ Collisions ○ Raymarching and VR ○ Shading tricks
![Page 41: Developing a Fractal experience at 90 FPS Conemarching in VR...Any GPU stall will kill the experience If we can’t hit the target framerate, time warping kicks in, halving fps. We](https://reader036.vdocuments.net/reader036/viewer/2022090611/60784b1243ac66238a311ac9/html5/thumbnails/41.jpg)
● Each step jump is proportional to the distance of the surface
Concept ○ Fractals ○ Collisions ○ Raymarching and VR ○ Shading tricks
![Page 42: Developing a Fractal experience at 90 FPS Conemarching in VR...Any GPU stall will kill the experience If we can’t hit the target framerate, time warping kicks in, halving fps. We](https://reader036.vdocuments.net/reader036/viewer/2022090611/60784b1243ac66238a311ac9/html5/thumbnails/42.jpg)
● Each step jump is proportional to the distance of the surface
Concept ○ Fractals ○ Collisions ○ Raymarching and VR ○ Shading tricks
![Page 43: Developing a Fractal experience at 90 FPS Conemarching in VR...Any GPU stall will kill the experience If we can’t hit the target framerate, time warping kicks in, halving fps. We](https://reader036.vdocuments.net/reader036/viewer/2022090611/60784b1243ac66238a311ac9/html5/thumbnails/43.jpg)
● Each step jump is proportional to the distance of the surface
Concept ○ Fractals ○ Collisions ○ Raymarching and VR ○ Shading tricks
![Page 44: Developing a Fractal experience at 90 FPS Conemarching in VR...Any GPU stall will kill the experience If we can’t hit the target framerate, time warping kicks in, halving fps. We](https://reader036.vdocuments.net/reader036/viewer/2022090611/60784b1243ac66238a311ac9/html5/thumbnails/44.jpg)
● Each step jump is proportional to the distance of the surface
Concept ○ Fractals ○ Collisions ○ Raymarching and VR ○ Shading tricks
![Page 45: Developing a Fractal experience at 90 FPS Conemarching in VR...Any GPU stall will kill the experience If we can’t hit the target framerate, time warping kicks in, halving fps. We](https://reader036.vdocuments.net/reader036/viewer/2022090611/60784b1243ac66238a311ac9/html5/thumbnails/45.jpg)
● Each step jump is proportional to the distance of the surface
Concept ○ Fractals ○ Collisions ○ Raymarching and VR ○ Shading tricks
![Page 46: Developing a Fractal experience at 90 FPS Conemarching in VR...Any GPU stall will kill the experience If we can’t hit the target framerate, time warping kicks in, halving fps. We](https://reader036.vdocuments.net/reader036/viewer/2022090611/60784b1243ac66238a311ac9/html5/thumbnails/46.jpg)
● Each step jump is proportional to the distance of the surface
● Can lead to overestimation if the function is not well defined
○ Fractals are not ideal!○ We bias each fractal SDF empirically
Concept ○ Fractals ○ Collisions ○ Raymarching and VR ○ Shading tricks
SPHERE TRACING
![Page 47: Developing a Fractal experience at 90 FPS Conemarching in VR...Any GPU stall will kill the experience If we can’t hit the target framerate, time warping kicks in, halving fps. We](https://reader036.vdocuments.net/reader036/viewer/2022090611/60784b1243ac66238a311ac9/html5/thumbnails/47.jpg)
● Each step jump is proportional to the distance of the surface
● Can lead to overestimation if the function is not well defined
○ Fractals are not ideal!○ We bias each fractal SDF empirically
● Preferred way to raymarch in modern approaches
Concept ○ Fractals ○ Collisions ○ Raymarching and VR ○ Shading tricks
SPHERE TRACING
![Page 48: Developing a Fractal experience at 90 FPS Conemarching in VR...Any GPU stall will kill the experience If we can’t hit the target framerate, time warping kicks in, halving fps. We](https://reader036.vdocuments.net/reader036/viewer/2022090611/60784b1243ac66238a311ac9/html5/thumbnails/48.jpg)
VIRTUAL REALITY CONSTRAINTS
● Our target is 90fps, so that gives us ~10ms
● Any GPU stall will kill the experience
● If we can’t hit the target framerate, dynamic time warping kicks in, halving fps.
● We have to render the same thing twice! :(
● Pipeline is not designed for raymarching, we have to hack Unity a bit
○ Big triangle where everything happens
Concept ○ Fractals ○ Collisions ○ Raymarching and VR ○ Shading tricks
![Page 49: Developing a Fractal experience at 90 FPS Conemarching in VR...Any GPU stall will kill the experience If we can’t hit the target framerate, time warping kicks in, halving fps. We](https://reader036.vdocuments.net/reader036/viewer/2022090611/60784b1243ac66238a311ac9/html5/thumbnails/49.jpg)
VIRTUAL REALITY CONSTRAINTS
● Our target is 90fps, so that gives us ~10ms
● Any GPU stall will kill the experience
● If we can’t hit the target framerate, dynamic time warping kicks in, halving fps.
● We have to render the same thing twice! :(
● Pipeline is not designed for raymarching, we have to hack Unity a bit
○ Big triangle where everything happens
Concept ○ Fractals ○ Collisions ○ Raymarching and VR ○ Shading tricks
![Page 50: Developing a Fractal experience at 90 FPS Conemarching in VR...Any GPU stall will kill the experience If we can’t hit the target framerate, time warping kicks in, halving fps. We](https://reader036.vdocuments.net/reader036/viewer/2022090611/60784b1243ac66238a311ac9/html5/thumbnails/50.jpg)
VIRTUAL REALITY CONSTRAINTS
● Our target is 90fps, so that gives us ~10ms
● Any GPU stall will kill the experience
● If we can’t hit the target framerate, time warping kicks in, halving fps.
● We have to render the same thing twice! :(
● Pipeline is not designed for raymarching, we have to hack Unity a bit
○ Big triangle where everything happens
Concept ○ Fractals ○ Collisions ○ Raymarching and VR ○ Shading tricks
![Page 51: Developing a Fractal experience at 90 FPS Conemarching in VR...Any GPU stall will kill the experience If we can’t hit the target framerate, time warping kicks in, halving fps. We](https://reader036.vdocuments.net/reader036/viewer/2022090611/60784b1243ac66238a311ac9/html5/thumbnails/51.jpg)
VIRTUAL REALITY CONSTRAINTS
● Our target is 90fps, so that gives us ~10ms
● Any GPU stall will kill the experience
● If we can’t hit the target framerate, time warping kicks in, halving fps.
● We have to render the same thing twice! :(
● Pipeline is not designed for raymarching, we have to hack Unity a bit
○ Big triangle where everything happens
Concept ○ Fractals ○ Collisions ○ Raymarching and VR ○ Shading tricks
![Page 52: Developing a Fractal experience at 90 FPS Conemarching in VR...Any GPU stall will kill the experience If we can’t hit the target framerate, time warping kicks in, halving fps. We](https://reader036.vdocuments.net/reader036/viewer/2022090611/60784b1243ac66238a311ac9/html5/thumbnails/52.jpg)
VIRTUAL REALITY CONSTRAINTS
● Our target is 90fps, so that gives us ~10ms
● Any GPU stall will kill the experience
● If we can’t hit the target framerate, time warping kicks in, halving fps.
● We have to render the same thing twice! :(
● Pipeline is not designed for raymarching, we have to hack Unity a bit
○ Big triangle where everything happens
Concept ○ Fractals ○ Collisions ○ Raymarching and VR ○ Shading tricks
![Page 53: Developing a Fractal experience at 90 FPS Conemarching in VR...Any GPU stall will kill the experience If we can’t hit the target framerate, time warping kicks in, halving fps. We](https://reader036.vdocuments.net/reader036/viewer/2022090611/60784b1243ac66238a311ac9/html5/thumbnails/53.jpg)
CONEMARCHING
● Originally from the demoscene○ Fractus by Fulcrum, Revision 2012
Concept ○ Fractals ○ Collisions ○ Raymarching and VR ○ Shading tricks
![Page 54: Developing a Fractal experience at 90 FPS Conemarching in VR...Any GPU stall will kill the experience If we can’t hit the target framerate, time warping kicks in, halving fps. We](https://reader036.vdocuments.net/reader036/viewer/2022090611/60784b1243ac66238a311ac9/html5/thumbnails/54.jpg)
● Originally from the demoscene○ Fractus by Fulcrum, Revision 2012
● Idea is to progressively render the same scene at different resolutions
Concept ○ Fractals ○ Collisions ○ Raymarching and VR ○ Shading tricks
![Page 55: Developing a Fractal experience at 90 FPS Conemarching in VR...Any GPU stall will kill the experience If we can’t hit the target framerate, time warping kicks in, halving fps. We](https://reader036.vdocuments.net/reader036/viewer/2022090611/60784b1243ac66238a311ac9/html5/thumbnails/55.jpg)
Concept ○ Fractals ○ Collisions ○ Raymarching and VR ○ Shading tricks
● Originally from the demoscene○ Fractus by Fulcrum, Revision 2012
● Idea is to progressively render the same scene at different resolutions
● On each pass, sphere trace until we can’t guarantee there isn’t an intersection
![Page 56: Developing a Fractal experience at 90 FPS Conemarching in VR...Any GPU stall will kill the experience If we can’t hit the target framerate, time warping kicks in, halving fps. We](https://reader036.vdocuments.net/reader036/viewer/2022090611/60784b1243ac66238a311ac9/html5/thumbnails/56.jpg)
Concept ○ Fractals ○ Collisions ○ Raymarching and VR ○ Shading tricks
● Originally from the demoscene○ Fractus by Fulcrum, Revision 2012
● Idea is to progressively render the same scene at different resolutions
● On each pass, sphere trace until we can’t guarantee there isn’t an intersection
![Page 57: Developing a Fractal experience at 90 FPS Conemarching in VR...Any GPU stall will kill the experience If we can’t hit the target framerate, time warping kicks in, halving fps. We](https://reader036.vdocuments.net/reader036/viewer/2022090611/60784b1243ac66238a311ac9/html5/thumbnails/57.jpg)
Concept ○ Fractals ○ Collisions ○ Raymarching and VR ○ Shading tricks
● Originally from the demoscene○ Fractus by Fulcrum, Revision 2012
● Idea is to progressively render the same scene at different resolutions
● On each pass, sphere trace until we can’t guarantee there isn’t an intersection
● Double the resolution and reuse the distance○ Some bias is usually needed
![Page 58: Developing a Fractal experience at 90 FPS Conemarching in VR...Any GPU stall will kill the experience If we can’t hit the target framerate, time warping kicks in, halving fps. We](https://reader036.vdocuments.net/reader036/viewer/2022090611/60784b1243ac66238a311ac9/html5/thumbnails/58.jpg)
Concept ○ Fractals ○ Collisions ○ Raymarching and VR ○ Shading tricks
● Originally from the demoscene○ Fractus by Fulcrum, Revision 2012
● Idea is to progressively render the same scene at different resolutions
● On each pass, sphere trace until we can’t guarantee there isn’t an intersection
● Double the resolution and reuse the distance○ Some bias is usually needed
![Page 59: Developing a Fractal experience at 90 FPS Conemarching in VR...Any GPU stall will kill the experience If we can’t hit the target framerate, time warping kicks in, halving fps. We](https://reader036.vdocuments.net/reader036/viewer/2022090611/60784b1243ac66238a311ac9/html5/thumbnails/59.jpg)
Concept ○ Fractals ○ Collisions ○ Raymarching and VR ○ Shading tricks
● Originally from the demoscene○ Fractus by Fulcrum, Revision 2012
● Idea is to progressively render the same scene at different resolutions
● On each pass, sphere trace until we can’t guarantee there isn’t an intersection
● Double the resolution and reuse the distance○ Some bias is usually needed
![Page 60: Developing a Fractal experience at 90 FPS Conemarching in VR...Any GPU stall will kill the experience If we can’t hit the target framerate, time warping kicks in, halving fps. We](https://reader036.vdocuments.net/reader036/viewer/2022090611/60784b1243ac66238a311ac9/html5/thumbnails/60.jpg)
Concept ○ Fractals ○ Collisions ○ Raymarching and VR ○ Shading tricks
● Originally from the demoscene○ Fractus by Fulcrum, Revision 2012
● Idea is to progressively render the same scene at different resolutions
● On each pass, sphere trace until we can’t guarantee there isn’t an intersection
● Double the resolution and reuse the distance○ Some bias is usually needed
![Page 61: Developing a Fractal experience at 90 FPS Conemarching in VR...Any GPU stall will kill the experience If we can’t hit the target framerate, time warping kicks in, halving fps. We](https://reader036.vdocuments.net/reader036/viewer/2022090611/60784b1243ac66238a311ac9/html5/thumbnails/61.jpg)
Concept ○ Fractals ○ Collisions ○ Raymarching and VR ○ Shading tricks
● Originally from the demoscene○ Fractus by Fulcrum, Revision 2012
● Idea is to progressively render the same scene at different resolutions
● On each pass, sphere trace until we can’t guarantee there isn’t an intersection
● Double the resolution and reuse the distance○ Some bias is usually needed
![Page 62: Developing a Fractal experience at 90 FPS Conemarching in VR...Any GPU stall will kill the experience If we can’t hit the target framerate, time warping kicks in, halving fps. We](https://reader036.vdocuments.net/reader036/viewer/2022090611/60784b1243ac66238a311ac9/html5/thumbnails/62.jpg)
CONEMARCHING
● Originally from the demoscene○ Fractus by Fulcrum, Revision 2012
● Idea is to progressively render the same scene at different resolutions
● On each pass, sphere trace until we can’t guarantee there isn’t an intersection
● Double the resolution and reuse the distance○ Some bias is usually needed
● Optimizing the number of iterations and passes is not trivial. Some emergent behaviour can appear!
Concept ○ Fractals ○ Collisions ○ Raymarching and VR ○ Shading tricks
![Page 63: Developing a Fractal experience at 90 FPS Conemarching in VR...Any GPU stall will kill the experience If we can’t hit the target framerate, time warping kicks in, halving fps. We](https://reader036.vdocuments.net/reader036/viewer/2022090611/60784b1243ac66238a311ac9/html5/thumbnails/63.jpg)
Concept ○ Fractals ○ Collisions ○ Raymarching and VR ○ Shading tricks
● 55-150 steps per pass● 8 steps at highres
![Page 64: Developing a Fractal experience at 90 FPS Conemarching in VR...Any GPU stall will kill the experience If we can’t hit the target framerate, time warping kicks in, halving fps. We](https://reader036.vdocuments.net/reader036/viewer/2022090611/60784b1243ac66238a311ac9/html5/thumbnails/64.jpg)
Concept ○ Fractals ○ Collisions ○ Raymarching and VR ○ Shading tricks
● 55-150 steps per pass● 8 steps at highres
![Page 65: Developing a Fractal experience at 90 FPS Conemarching in VR...Any GPU stall will kill the experience If we can’t hit the target framerate, time warping kicks in, halving fps. We](https://reader036.vdocuments.net/reader036/viewer/2022090611/60784b1243ac66238a311ac9/html5/thumbnails/65.jpg)
Concept ○ Fractals ○ Collisions ○ Raymarching and VR ○ Shading tricks
● 55-150 steps per pass● 8 steps at highres
![Page 66: Developing a Fractal experience at 90 FPS Conemarching in VR...Any GPU stall will kill the experience If we can’t hit the target framerate, time warping kicks in, halving fps. We](https://reader036.vdocuments.net/reader036/viewer/2022090611/60784b1243ac66238a311ac9/html5/thumbnails/66.jpg)
Concept ○ Fractals ○ Collisions ○ Raymarching and VR ○ Shading tricks
● 55-150 steps per pass● 8 steps at highres
![Page 67: Developing a Fractal experience at 90 FPS Conemarching in VR...Any GPU stall will kill the experience If we can’t hit the target framerate, time warping kicks in, halving fps. We](https://reader036.vdocuments.net/reader036/viewer/2022090611/60784b1243ac66238a311ac9/html5/thumbnails/67.jpg)
Concept ○ Fractals ○ Collisions ○ Raymarching and VR ○ Shading tricks
● 55-150 steps per pass● 8 steps at highres
![Page 68: Developing a Fractal experience at 90 FPS Conemarching in VR...Any GPU stall will kill the experience If we can’t hit the target framerate, time warping kicks in, halving fps. We](https://reader036.vdocuments.net/reader036/viewer/2022090611/60784b1243ac66238a311ac9/html5/thumbnails/68.jpg)
Concept ○ Fractals ○ Collisions ○ Raymarching and VR ○ Shading tricks
● 55-150 steps per pass● 8 steps at highres
![Page 69: Developing a Fractal experience at 90 FPS Conemarching in VR...Any GPU stall will kill the experience If we can’t hit the target framerate, time warping kicks in, halving fps. We](https://reader036.vdocuments.net/reader036/viewer/2022090611/60784b1243ac66238a311ac9/html5/thumbnails/69.jpg)
Concept ○ Fractals ○ Collisions ○ Raymarching and VR ○ Shading tricks
● 55-150 steps per pass● 8 steps at highres
![Page 70: Developing a Fractal experience at 90 FPS Conemarching in VR...Any GPU stall will kill the experience If we can’t hit the target framerate, time warping kicks in, halving fps. We](https://reader036.vdocuments.net/reader036/viewer/2022090611/60784b1243ac66238a311ac9/html5/thumbnails/70.jpg)
Concept ○ Fractals ○ Collisions ○ Raymarching and VR ○ Shading tricks
● 55-150 steps per pass● 8 steps at highres
![Page 71: Developing a Fractal experience at 90 FPS Conemarching in VR...Any GPU stall will kill the experience If we can’t hit the target framerate, time warping kicks in, halving fps. We](https://reader036.vdocuments.net/reader036/viewer/2022090611/60784b1243ac66238a311ac9/html5/thumbnails/71.jpg)
BUT...
● We’re still rendering everything twice!
Concept ○ Fractals ○ Collisions ○ Raymarching and VR ○ Shading tricks
![Page 72: Developing a Fractal experience at 90 FPS Conemarching in VR...Any GPU stall will kill the experience If we can’t hit the target framerate, time warping kicks in, halving fps. We](https://reader036.vdocuments.net/reader036/viewer/2022090611/60784b1243ac66238a311ac9/html5/thumbnails/72.jpg)
REPROJECTION
● We’re still rendering everything twice!
● Two problems: depth estimation and shading○ Maybe we can we reproject depth? Yes!
Concept ○ Fractals ○ Collisions ○ Raymarching and VR ○ Shading tricks
![Page 73: Developing a Fractal experience at 90 FPS Conemarching in VR...Any GPU stall will kill the experience If we can’t hit the target framerate, time warping kicks in, halving fps. We](https://reader036.vdocuments.net/reader036/viewer/2022090611/60784b1243ac66238a311ac9/html5/thumbnails/73.jpg)
● We’re still rendering everything twice!
● Two problems: depth estimation and shading○ Maybe we can we reproject depth? Yes!
● Render the center eye with the conemarcher
Concept ○ Fractals ○ Collisions ○ Raymarching and VR ○ Shading tricks
![Page 74: Developing a Fractal experience at 90 FPS Conemarching in VR...Any GPU stall will kill the experience If we can’t hit the target framerate, time warping kicks in, halving fps. We](https://reader036.vdocuments.net/reader036/viewer/2022090611/60784b1243ac66238a311ac9/html5/thumbnails/74.jpg)
● We’re still rendering everything twice!
● Two problems: depth estimation and shading○ Maybe we can we reproject depth? Yes!
● Render the center eye with the conemarcher
Concept ○ Fractals ○ Collisions ○ Raymarching and VR ○ Shading tricks
![Page 75: Developing a Fractal experience at 90 FPS Conemarching in VR...Any GPU stall will kill the experience If we can’t hit the target framerate, time warping kicks in, halving fps. We](https://reader036.vdocuments.net/reader036/viewer/2022090611/60784b1243ac66238a311ac9/html5/thumbnails/75.jpg)
● We’re still rendering everything twice!
● Two problems: depth estimation and shading○ Maybe we can we reproject depth? Yes!
● Render the center eye with the conemarcher
● Reproject to left and right eye
Concept ○ Fractals ○ Collisions ○ Raymarching and VR ○ Shading tricks
![Page 76: Developing a Fractal experience at 90 FPS Conemarching in VR...Any GPU stall will kill the experience If we can’t hit the target framerate, time warping kicks in, halving fps. We](https://reader036.vdocuments.net/reader036/viewer/2022090611/60784b1243ac66238a311ac9/html5/thumbnails/76.jpg)
● We’re still rendering everything twice!
● Two problems: depth estimation and shading○ Maybe we can we reproject depth? Yes!
● Render the center eye with the conemarcher
● Reproject to left and right eye
● Screen space ray marching with horizontal offset
Concept ○ Fractals ○ Collisions ○ Raymarching and VR ○ Shading tricks
![Page 77: Developing a Fractal experience at 90 FPS Conemarching in VR...Any GPU stall will kill the experience If we can’t hit the target framerate, time warping kicks in, halving fps. We](https://reader036.vdocuments.net/reader036/viewer/2022090611/60784b1243ac66238a311ac9/html5/thumbnails/77.jpg)
● We’re still rendering everything twice!
● Two problems: depth estimation and shading○ Maybe we can we reproject depth? Yes!
● Render the center eye with the conemarcher
● Reproject to left and right eye
● Screen space ray marching with horizontal offset
Concept ○ Fractals ○ Collisions ○ Raymarching and VR ○ Shading tricks
![Page 78: Developing a Fractal experience at 90 FPS Conemarching in VR...Any GPU stall will kill the experience If we can’t hit the target framerate, time warping kicks in, halving fps. We](https://reader036.vdocuments.net/reader036/viewer/2022090611/60784b1243ac66238a311ac9/html5/thumbnails/78.jpg)
● We’re still rendering everything twice!
● Two problems: depth estimation and shading○ Maybe we can we reproject depth? Yes!
● Render the center eye with the conemarcher
● Reproject to left and right eye
● Screen space ray marching with horizontal offset
Concept ○ Fractals ○ Collisions ○ Raymarching and VR ○ Shading tricks
![Page 79: Developing a Fractal experience at 90 FPS Conemarching in VR...Any GPU stall will kill the experience If we can’t hit the target framerate, time warping kicks in, halving fps. We](https://reader036.vdocuments.net/reader036/viewer/2022090611/60784b1243ac66238a311ac9/html5/thumbnails/79.jpg)
● We’re still rendering everything twice!
● Two problems: depth estimation and shading○ Maybe we can we reproject depth? Yes!
● Render the center eye with the conemarcher
● Reproject to left and right eye
● Screen space ray marching with horizontal offset
Concept ○ Fractals ○ Collisions ○ Raymarching and VR ○ Shading tricks
![Page 80: Developing a Fractal experience at 90 FPS Conemarching in VR...Any GPU stall will kill the experience If we can’t hit the target framerate, time warping kicks in, halving fps. We](https://reader036.vdocuments.net/reader036/viewer/2022090611/60784b1243ac66238a311ac9/html5/thumbnails/80.jpg)
● We’re still rendering everything twice!
● Two problems: depth estimation and shading○ Maybe we can we reproject depth? Yes!
● Render the center eye with the conemarcher
● Reproject to left and right eye
● Screen space ray marching with horizontal offset
Concept ○ Fractals ○ Collisions ○ Raymarching and VR ○ Shading tricks
![Page 81: Developing a Fractal experience at 90 FPS Conemarching in VR...Any GPU stall will kill the experience If we can’t hit the target framerate, time warping kicks in, halving fps. We](https://reader036.vdocuments.net/reader036/viewer/2022090611/60784b1243ac66238a311ac9/html5/thumbnails/81.jpg)
● We’re still rendering everything twice!
● Two problems: depth estimation and shading○ Maybe we can we reproject depth? Yes!
● Render the center eye with the conemarcher
● Reproject to left and right eye
● Screen space ray marching with horizontal offset
● To get a better converging distance, we use the conemarching passes at lower res
Concept ○ Fractals ○ Collisions ○ Raymarching and VR ○ Shading tricks
![Page 82: Developing a Fractal experience at 90 FPS Conemarching in VR...Any GPU stall will kill the experience If we can’t hit the target framerate, time warping kicks in, halving fps. We](https://reader036.vdocuments.net/reader036/viewer/2022090611/60784b1243ac66238a311ac9/html5/thumbnails/82.jpg)
● Original paper deals with rasterization and couldn’t fix the discontinuities
Concept ○ Fractals ○ Collisions ○ Raymarching and VR ○ Shading tricks
● Fast Gather-based Construction of Stereoscopic Images Using Reprojection [van de Hoef, Zalmstra]
![Page 83: Developing a Fractal experience at 90 FPS Conemarching in VR...Any GPU stall will kill the experience If we can’t hit the target framerate, time warping kicks in, halving fps. We](https://reader036.vdocuments.net/reader036/viewer/2022090611/60784b1243ac66238a311ac9/html5/thumbnails/83.jpg)
Concept ○ Fractals ○ Collisions ○ Raymarching and VR ○ Shading tricks
● Original paper deals with rasterization and couldn’t fix the discontinuities
● We can!○ Just keep marching where there is a lot of
stereo disparity
![Page 84: Developing a Fractal experience at 90 FPS Conemarching in VR...Any GPU stall will kill the experience If we can’t hit the target framerate, time warping kicks in, halving fps. We](https://reader036.vdocuments.net/reader036/viewer/2022090611/60784b1243ac66238a311ac9/html5/thumbnails/84.jpg)
Concept ○ Fractals ○ Collisions ○ Raymarching and VR ○ Shading tricks
● Original paper deals with rasterization and couldn’t fix the discontinuities
● We can!○ Just keep marching where there is a lot of
stereo disparity
○ Also, shift samples horizontally
![Page 85: Developing a Fractal experience at 90 FPS Conemarching in VR...Any GPU stall will kill the experience If we can’t hit the target framerate, time warping kicks in, halving fps. We](https://reader036.vdocuments.net/reader036/viewer/2022090611/60784b1243ac66238a311ac9/html5/thumbnails/85.jpg)
Concept ○ Fractals ○ Collisions ○ Raymarching and VR ○ Shading tricks
● Original paper deals with rasterization and couldn’t fix the discontinuities
● We can!○ Just keep marching where there is a lot of
stereo disparity
○ Also, shift samples horizontally
● Small artifacts, but can be masked
![Page 86: Developing a Fractal experience at 90 FPS Conemarching in VR...Any GPU stall will kill the experience If we can’t hit the target framerate, time warping kicks in, halving fps. We](https://reader036.vdocuments.net/reader036/viewer/2022090611/60784b1243ac66238a311ac9/html5/thumbnails/86.jpg)
Concept ○ Fractals ○ Collisions ○ Raymarching and VR ○ Shading tricks
● Original paper deals with rasterization and couldn’t fix the discontinuities
● We can!○ Just keep marching where there is a lot of
stereo disparity
○ Also, shift samples horizontally
● Small artifacts, but can be masked
● Decreases quality with stereo separation○ Fortunately, we already scale the fractal all
the time, so that you’re always one unit away from the closest intersection
![Page 87: Developing a Fractal experience at 90 FPS Conemarching in VR...Any GPU stall will kill the experience If we can’t hit the target framerate, time warping kicks in, halving fps. We](https://reader036.vdocuments.net/reader036/viewer/2022090611/60784b1243ac66238a311ac9/html5/thumbnails/87.jpg)
Concept ○ Fractals ○ Collisions ○ Raymarching and VR ○ Shading tricks
● With the conemarcher, we had to estimate depth twice
![Page 88: Developing a Fractal experience at 90 FPS Conemarching in VR...Any GPU stall will kill the experience If we can’t hit the target framerate, time warping kicks in, halving fps. We](https://reader036.vdocuments.net/reader036/viewer/2022090611/60784b1243ac66238a311ac9/html5/thumbnails/88.jpg)
Concept ○ Fractals ○ Collisions ○ Raymarching and VR ○ Shading tricks
● With the conemarcher, we had to estimate depth twice
● Now we can directly cut a huge part of the pipeline
![Page 89: Developing a Fractal experience at 90 FPS Conemarching in VR...Any GPU stall will kill the experience If we can’t hit the target framerate, time warping kicks in, halving fps. We](https://reader036.vdocuments.net/reader036/viewer/2022090611/60784b1243ac66238a311ac9/html5/thumbnails/89.jpg)
Concept ○ Fractals ○ Collisions ○ Raymarching and VR ○ Shading tricks
● With the conemarcher, we had to estimate depth twice
● Now we can directly cut a huge part of the pipeline
● Reprojection pass is usually fast○ Performance improves proportionally to the conemarching pass
![Page 90: Developing a Fractal experience at 90 FPS Conemarching in VR...Any GPU stall will kill the experience If we can’t hit the target framerate, time warping kicks in, halving fps. We](https://reader036.vdocuments.net/reader036/viewer/2022090611/60784b1243ac66238a311ac9/html5/thumbnails/90.jpg)
Concept ○ Fractals ○ Collisions ○ Raymarching and VR ○ Shading tricks
● With the conemarcher, we had to estimate depth twice
● Now we can directly cut a huge part of the pipeline
● Reprojection pass is usually fast○ Performance improves proportionally to the conemarching pass
● Shading pass requires some tweaking, because some of the reprojection estimates are not perfect
![Page 91: Developing a Fractal experience at 90 FPS Conemarching in VR...Any GPU stall will kill the experience If we can’t hit the target framerate, time warping kicks in, halving fps. We](https://reader036.vdocuments.net/reader036/viewer/2022090611/60784b1243ac66238a311ac9/html5/thumbnails/91.jpg)
FIXED FOVEATED RENDERING
Concept ○ Fractals ○ Collisions ○ Raymarching and VR ○ Shading tricks
● Still expensive to render the shading
![Page 92: Developing a Fractal experience at 90 FPS Conemarching in VR...Any GPU stall will kill the experience If we can’t hit the target framerate, time warping kicks in, halving fps. We](https://reader036.vdocuments.net/reader036/viewer/2022090611/60784b1243ac66238a311ac9/html5/thumbnails/92.jpg)
FIXED FOVEATED RENDERING
Concept ○ Fractals ○ Collisions ○ Raymarching and VR ○ Shading tricks
● Still expensive to render the shading
● We don’t need that much detail on the periphery
![Page 93: Developing a Fractal experience at 90 FPS Conemarching in VR...Any GPU stall will kill the experience If we can’t hit the target framerate, time warping kicks in, halving fps. We](https://reader036.vdocuments.net/reader036/viewer/2022090611/60784b1243ac66238a311ac9/html5/thumbnails/93.jpg)
FIXED FOVEATED RENDERING
Concept ○ Fractals ○ Collisions ○ Raymarching and VR ○ Shading tricks
● Still expensive to render the shading
● We don’t need that much detail on the periphery
● We want something dynamic that can be scaled per fractal, and depending hardware
![Page 94: Developing a Fractal experience at 90 FPS Conemarching in VR...Any GPU stall will kill the experience If we can’t hit the target framerate, time warping kicks in, halving fps. We](https://reader036.vdocuments.net/reader036/viewer/2022090611/60784b1243ac66238a311ac9/html5/thumbnails/94.jpg)
● Still expensive to render the shading
● We don’t need that much detail on the periphery
● We want something dynamic that can be scaled per fractal, and depending hardware
● Render at half res on the periphery, full resolution at the center and blend the edges
Concept ○ Fractals ○ Collisions ○ Raymarching and VR ○ Shading tricks
![Page 95: Developing a Fractal experience at 90 FPS Conemarching in VR...Any GPU stall will kill the experience If we can’t hit the target framerate, time warping kicks in, halving fps. We](https://reader036.vdocuments.net/reader036/viewer/2022090611/60784b1243ac66238a311ac9/html5/thumbnails/95.jpg)
● Still expensive to render the shading
● We don’t need that much detail on the periphery
● We want something dynamic that can be scaled per fractal, and depending hardware
● Render at half res on the periphery, full resolution at the center and blend the edges
Concept ○ Fractals ○ Collisions ○ Raymarching and VR ○ Shading tricks
![Page 96: Developing a Fractal experience at 90 FPS Conemarching in VR...Any GPU stall will kill the experience If we can’t hit the target framerate, time warping kicks in, halving fps. We](https://reader036.vdocuments.net/reader036/viewer/2022090611/60784b1243ac66238a311ac9/html5/thumbnails/96.jpg)
● Still expensive to render the shading
● We don’t need that much detail on the periphery
● We want something dynamic that can be scaled per fractal, and depending hardware
● Render at half res on the periphery, full resolution at the center and blend the edges
● Compose the result○ Strong vignette saves us some computation
time
Concept ○ Fractals ○ Collisions ○ Raymarching and VR ○ Shading tricks
![Page 97: Developing a Fractal experience at 90 FPS Conemarching in VR...Any GPU stall will kill the experience If we can’t hit the target framerate, time warping kicks in, halving fps. We](https://reader036.vdocuments.net/reader036/viewer/2022090611/60784b1243ac66238a311ac9/html5/thumbnails/97.jpg)
THE SHADING CHALLENGE
![Page 98: Developing a Fractal experience at 90 FPS Conemarching in VR...Any GPU stall will kill the experience If we can’t hit the target framerate, time warping kicks in, halving fps. We](https://reader036.vdocuments.net/reader036/viewer/2022090611/60784b1243ac66238a311ac9/html5/thumbnails/98.jpg)
HOW DO PEOPLE RENDER FRACTALS?
● How can we texture them?○ We only have position and normal
● People generally use simple shading○ Heavy use of shadows, occlusion and
scattering to improve look○ Heavy use of iteration glow○ Orbit traps are not used cleverly
● We wanted to do something different
Concept ○ Fractals ○ Collisions ○ xRaymarching and VR ○ Shading tricks
![Page 99: Developing a Fractal experience at 90 FPS Conemarching in VR...Any GPU stall will kill the experience If we can’t hit the target framerate, time warping kicks in, halving fps. We](https://reader036.vdocuments.net/reader036/viewer/2022090611/60784b1243ac66238a311ac9/html5/thumbnails/99.jpg)
MORE CONSTRAINTS!
● Shadows are expensive!● Normals are expensive!● Occlusion is expensive!● SSS is expensive!● Our iteration glow is broken due to conemarching!
Concept ○ Fractals ○ Collisions ○ Raymarching and VR ○ Shading tricks
![Page 100: Developing a Fractal experience at 90 FPS Conemarching in VR...Any GPU stall will kill the experience If we can’t hit the target framerate, time warping kicks in, halving fps. We](https://reader036.vdocuments.net/reader036/viewer/2022090611/60784b1243ac66238a311ac9/html5/thumbnails/100.jpg)
SHADING
● Experimented a lot with different ideas○ Step the orbit traps to guide glowing
elements○ Warp the traps into themselves○ Sample the fractal as the texture
■ Warp the fractal too!○ Sample the derivative instead of the SDF○ Use less iterations to fake SSS and AO
● Decided to design each fractal with a two color palette and heavy use of glowing sections.
Concept ○ Fractals ○ Collisions ○ Raymarching and VR ○ Shading tricks
![Page 101: Developing a Fractal experience at 90 FPS Conemarching in VR...Any GPU stall will kill the experience If we can’t hit the target framerate, time warping kicks in, halving fps. We](https://reader036.vdocuments.net/reader036/viewer/2022090611/60784b1243ac66238a311ac9/html5/thumbnails/101.jpg)
● Normal estimation generally uses finite differences
Concept ○ Fractals ○ Collisions ○ Raymarching and VR ○ Shading tricks
![Page 102: Developing a Fractal experience at 90 FPS Conemarching in VR...Any GPU stall will kill the experience If we can’t hit the target framerate, time warping kicks in, halving fps. We](https://reader036.vdocuments.net/reader036/viewer/2022090611/60784b1243ac66238a311ac9/html5/thumbnails/102.jpg)
● Normal estimation generally uses finite differences
● Requires 6 samples of the sdf○ Can be reduced to 3 by losing some quality
Concept ○ Fractals ○ Collisions ○ Raymarching and VR ○ Shading tricks
![Page 103: Developing a Fractal experience at 90 FPS Conemarching in VR...Any GPU stall will kill the experience If we can’t hit the target framerate, time warping kicks in, halving fps. We](https://reader036.vdocuments.net/reader036/viewer/2022090611/60784b1243ac66238a311ac9/html5/thumbnails/103.jpg)
● Normal estimation generally uses finite differences
● Requires 6 samples of the sdf○ Can be reduced to 3 by losing some quality
● Iq proposed on pouet.net a way to fake diffuse lighting with only one sample
Concept ○ Fractals ○ Collisions ○ Raymarching and VR ○ Shading tricks
![Page 104: Developing a Fractal experience at 90 FPS Conemarching in VR...Any GPU stall will kill the experience If we can’t hit the target framerate, time warping kicks in, halving fps. We](https://reader036.vdocuments.net/reader036/viewer/2022090611/60784b1243ac66238a311ac9/html5/thumbnails/104.jpg)
● Normal estimation generally uses finite differences
● Requires 6 samples of the sdf○ Can be reduced to 3 by losing some quality
● Iq proposed on pouet.net a way to fake diffuse lighting with only one sample
● Apply this trick to everything!○ Reasonable fake AO!○ Reasonable fake SSS!○ Rim lighting?
Concept ○ Fractals ○ Collisions ○ Raymarching and VR ○ Shading tricks
![Page 105: Developing a Fractal experience at 90 FPS Conemarching in VR...Any GPU stall will kill the experience If we can’t hit the target framerate, time warping kicks in, halving fps. We](https://reader036.vdocuments.net/reader036/viewer/2022090611/60784b1243ac66238a311ac9/html5/thumbnails/105.jpg)
● Normal estimation generally uses finite differences
● Requires 6 samples of the sdf○ Can be reduced to 3 by losing some quality
● Iq proposed on pouet.net a way to fake diffuse lighting with only one sample
● Apply this trick to everything!○ Reasonable fake AO!○ Reasonable fake SSS!○ Rim lighting?
Concept ○ Fractals ○ Collisions ○ Raymarching and VR ○ Shading tricks
![Page 106: Developing a Fractal experience at 90 FPS Conemarching in VR...Any GPU stall will kill the experience If we can’t hit the target framerate, time warping kicks in, halving fps. We](https://reader036.vdocuments.net/reader036/viewer/2022090611/60784b1243ac66238a311ac9/html5/thumbnails/106.jpg)
● Normal estimation generally uses finite differences
● Requires 6 samples of the sdf○ Can be reduced to 3 by losing some quality
● Iq proposed on pouet.net a way to fake diffuse lighting with only one sample
● Apply this trick to everything!○ Reasonable fake AO!○ Reasonable fake SSS!○ Rim lighting?
● Has some banding problems due to fractal epsilon○ We still needed a proper normal for some
fractals
Concept ○ Fractals ○ Collisions ○ Raymarching and VR ○ Shading tricks
![Page 107: Developing a Fractal experience at 90 FPS Conemarching in VR...Any GPU stall will kill the experience If we can’t hit the target framerate, time warping kicks in, halving fps. We](https://reader036.vdocuments.net/reader036/viewer/2022090611/60784b1243ac66238a311ac9/html5/thumbnails/107.jpg)
[OUR MANDELBULB]
● Moving a little back to the camera instead generates a very interesting pattern
● Use an animated cosine palette that scrolls based on this function
● Heavy use of fake SSS
● Use of lower iteration Mandelbulb code for AO
Concept ○ Fractals ○ Collisions ○ Raymarching and VR ○ Shading tricks
![Page 108: Developing a Fractal experience at 90 FPS Conemarching in VR...Any GPU stall will kill the experience If we can’t hit the target framerate, time warping kicks in, halving fps. We](https://reader036.vdocuments.net/reader036/viewer/2022090611/60784b1243ac66238a311ac9/html5/thumbnails/108.jpg)
Concept ○ Fractals ○ Collisions ○ Raymarching and VR ○ Shading tricks
● Don’t render very far
● Use homogeneous scattering to hide it○ This helps immersion, fortunately!
![Page 109: Developing a Fractal experience at 90 FPS Conemarching in VR...Any GPU stall will kill the experience If we can’t hit the target framerate, time warping kicks in, halving fps. We](https://reader036.vdocuments.net/reader036/viewer/2022090611/60784b1243ac66238a311ac9/html5/thumbnails/109.jpg)
Concept ○ Fractals ○ Collisions ○ Raymarching and VR ○ Shading tricks
● Don’t render very far
● Use homogeneous scattering to hide it○ This helps immersion, fortunately!
Acquiring Scattering Properties of Participating Media by Dilution https://cseweb.ucsd.edu/~ravir/dilution.pdf
![Page 110: Developing a Fractal experience at 90 FPS Conemarching in VR...Any GPU stall will kill the experience If we can’t hit the target framerate, time warping kicks in, halving fps. We](https://reader036.vdocuments.net/reader036/viewer/2022090611/60784b1243ac66238a311ac9/html5/thumbnails/110.jpg)
FUTURE WORK (SHORT TERM)
● Temporal reprojection of depth estimation
● Reprojection of low frequency effects
● Use screen space normals to reduce shading complexity
○ Use lower quality shading on periphery
● Implement TAA + FXAA with the outlines provided by the stereo reprojection disparity
![Page 111: Developing a Fractal experience at 90 FPS Conemarching in VR...Any GPU stall will kill the experience If we can’t hit the target framerate, time warping kicks in, halving fps. We](https://reader036.vdocuments.net/reader036/viewer/2022090611/60784b1243ac66238a311ac9/html5/thumbnails/111.jpg)
● Temporal reprojection of depth estimation
● Reprojection of low frequency effects
● Use screen space normals to reduce shading complexity
○ Use lower quality shading on periphery
● Implement TAA + FXAA with the outlines provided by the stereo reprojection disparity
![Page 112: Developing a Fractal experience at 90 FPS Conemarching in VR...Any GPU stall will kill the experience If we can’t hit the target framerate, time warping kicks in, halving fps. We](https://reader036.vdocuments.net/reader036/viewer/2022090611/60784b1243ac66238a311ac9/html5/thumbnails/112.jpg)
FUTURE WORK (LONG TERM)
● Implement stereo reprojection for shading (Oculus’ approach)
● Optimize ray scheduling to maximize occupancy○ GPU Unchained (Timothy Lottes) from
NVScene 2015
● Experiment with low resolution voxel grid to accelerate depth estimation and other effects
● Alleviate strong aliasing artifacts
![Page 113: Developing a Fractal experience at 90 FPS Conemarching in VR...Any GPU stall will kill the experience If we can’t hit the target framerate, time warping kicks in, halving fps. We](https://reader036.vdocuments.net/reader036/viewer/2022090611/60784b1243ac66238a311ac9/html5/thumbnails/113.jpg)
© Framestore 2018 / Framestore
Thank you
Johannes [email protected] | @johannessaam
Mariano [email protected] | @mmerchante
coral-vr.com
![Page 114: Developing a Fractal experience at 90 FPS Conemarching in VR...Any GPU stall will kill the experience If we can’t hit the target framerate, time warping kicks in, halving fps. We](https://reader036.vdocuments.net/reader036/viewer/2022090611/60784b1243ac66238a311ac9/html5/thumbnails/114.jpg)
© Framestore 2018 / Framestore
![Page 115: Developing a Fractal experience at 90 FPS Conemarching in VR...Any GPU stall will kill the experience If we can’t hit the target framerate, time warping kicks in, halving fps. We](https://reader036.vdocuments.net/reader036/viewer/2022090611/60784b1243ac66238a311ac9/html5/thumbnails/115.jpg)
© Framestore 2018 / Framestore
![Page 116: Developing a Fractal experience at 90 FPS Conemarching in VR...Any GPU stall will kill the experience If we can’t hit the target framerate, time warping kicks in, halving fps. We](https://reader036.vdocuments.net/reader036/viewer/2022090611/60784b1243ac66238a311ac9/html5/thumbnails/116.jpg)
© Framestore 2018 / Framestore
BONUS!
![Page 117: Developing a Fractal experience at 90 FPS Conemarching in VR...Any GPU stall will kill the experience If we can’t hit the target framerate, time warping kicks in, halving fps. We](https://reader036.vdocuments.net/reader036/viewer/2022090611/60784b1243ac66238a311ac9/html5/thumbnails/117.jpg)
REPROJECT ALL THE THINGS![COMIC HERE]
● We liked the reprojection results
● Can we reproject low frequency image effects?○ Glow○ Volumetric scattering (godrays)○ Global Illumination?○ Ambient Occlusion?
● Still working on it!
Concept ○ Fractals ○ Collisions ○ Raymarching and VR ○ Shading tricks
![Page 118: Developing a Fractal experience at 90 FPS Conemarching in VR...Any GPU stall will kill the experience If we can’t hit the target framerate, time warping kicks in, halving fps. We](https://reader036.vdocuments.net/reader036/viewer/2022090611/60784b1243ac66238a311ac9/html5/thumbnails/118.jpg)
TEMPORAL REPROJECTION
● Can we reuse the previous frame’s distance estimation?
○ TAA deals with similar ideas, but only for shading
Concept ○ Fractals ○ Collisions ○ Raymarching and VR ○ Shading tricks
![Page 119: Developing a Fractal experience at 90 FPS Conemarching in VR...Any GPU stall will kill the experience If we can’t hit the target framerate, time warping kicks in, halving fps. We](https://reader036.vdocuments.net/reader036/viewer/2022090611/60784b1243ac66238a311ac9/html5/thumbnails/119.jpg)
TEMPORAL REPROJECTION
● Can we reuse the previous frame’s distance estimation?
○ TAA deals with similar ideas, but only for shading
● Use motion vectors to predict positions
Concept ○ Fractals ○ Collisions ○ Raymarching and VR ○ Shading tricks
![Page 120: Developing a Fractal experience at 90 FPS Conemarching in VR...Any GPU stall will kill the experience If we can’t hit the target framerate, time warping kicks in, halving fps. We](https://reader036.vdocuments.net/reader036/viewer/2022090611/60784b1243ac66238a311ac9/html5/thumbnails/120.jpg)
TEMPORAL REPROJECTION
● Can we reuse the previous frame’s distance estimation?
○ TAA deals with similar ideas, but only for shading
● Use motion vectors to predict positions
● Can’t use simple eye reprojection technique because delta is both positional and rotational
○ Scatter vs gather
Concept ○ Fractals ○ Collisions ○ Raymarching and VR ○ Shading tricks
![Page 121: Developing a Fractal experience at 90 FPS Conemarching in VR...Any GPU stall will kill the experience If we can’t hit the target framerate, time warping kicks in, halving fps. We](https://reader036.vdocuments.net/reader036/viewer/2022090611/60784b1243ac66238a311ac9/html5/thumbnails/121.jpg)
TEMPORAL REPROJECTION
● Can we reuse the previous frame’s distance estimation?
○ TAA deals with similar ideas, but only for shading
● Use motion vectors to predict positions
● Can’t use simple eye reprojection technique because delta is both positional and rotational
○ Scatter vs gather
● The conemarching passes serve as lower bounds○ So we can grab (dis)appearing objects!
Concept ○ Fractals ○ Collisions ○ Raymarching and VR ○ Shading tricks
![Page 122: Developing a Fractal experience at 90 FPS Conemarching in VR...Any GPU stall will kill the experience If we can’t hit the target framerate, time warping kicks in, halving fps. We](https://reader036.vdocuments.net/reader036/viewer/2022090611/60784b1243ac66238a311ac9/html5/thumbnails/122.jpg)
TEMPORAL REPROJECTION
● Can we reuse the previous frame’s distance estimation?
○ TAA deals with similar ideas, but only for shading
● Use motion vectors to predict positions
● Can’t use simple eye reprojection technique because delta is both positional and rotational
○ Scatter vs gather
● The conemarching passes serve as lower bounds○ So we can grab (dis)appearing objects!
● We’re still working on this!
Concept ○ Fractals ○ Collisions ○ Raymarching and VR ○ Shading tricks