dominic eskofier (nvidia) every millisecond counts: how to render faster for virtual reality
TRANSCRIPT
EVERY MILLISECOND COUNTSHow to render faster for Virtual RealityAWE Europe, Berlin 2016
Dominic Eskofier
Virtual Reality Manager [email protected]
Gaming
Movies
Sports
Concerts
Travel Retail
PERSONAL ENTERTAINMENT
VR
NVIDIA CONFIDENTIAL. DO NOT DISTRIBUTE.
QUADRO PROFESSIONAL VR/AR
MEDICAL ARCHITECTURE
DESIGN MANUFACTURING COLLABORATION
DIGITAL CONTENT CREATION
COMPUTING CHALLENGES IN SIMULATING REALITY
GRAPHICS PHYSICAL SIMULATIONAUDIO TOUCH
5
TRADITIONAL = 60 MP/S
(1920 X 1080 @ 30 FPS)
VIRTUAL REALITY = 450 MP/S(3024 X 1680* @ 90 FPS)
1920
1080
1512
1680
*VR render resolution
1512
VR PERFORMANCE DEMANDSUltra-High Resolution and Frame Rate
Motion to Photon: ≤ 20 ms
VR PERFORMANCE DEMANDSUltra-Low Latency
GRAPHICS 101
CREATE 3D GEOMETRY
PROJECT TO 2D VIEW
CALCULATE PIXELS
Warped ImageRendered Image
VR RENDERING 101
Rendered Image Warped Image
VR RENDERING 101GPU RENDERS MANY PIXELS THAT NEVER MAKE IT TO THE
SCREEN
NVIDIA Confidential – Do Not Distribute
NVIDIA MULTI-RES SHADINGIntroduced in Maxwell
LENS MATCHED SHADINGRenders to a Lens Corrected Surface
LENS MATCHED SHADINGRenders to a Lens Corrected Surface
Original Image LMS Image
TRADITIONAL STEREO RENDERING
Requires 2 Geometry Passes
Left Eye (Pass 1) Right Eye (Pass 2)
SINGLE PASS STEREORenders Left & Right Eye in One Geometry Pass
Left Eye
Right Eye
‘MOTION TO PHOTON’ LATENCY KEY
TO COMFORTABLE VR
VR LATENCY WITHOUT TIMEWARPCPU
GPU
Flip
Flash backlight
Scanout
Sample head pose
Submit to GPU
Latency
Timewarpbased on latest
head poseCPU
GPU
Latency
Scanout
Flip
Submit to GPU
Flash backlight
Sample head pose
VR LATENCY WITH TIMEWARP
PREEMPTION IN VRAsynchronous Time Warp
11 ms (90 Hz)
GRAPHICS OR COMPUTE
ConservativePreemption
Request
ATW
DisplayRefresh
DisplayRefresh
PREEMPTION IN VR - PASCALImproved Latency & Speed
11 ms (90 Hz)
GRAPHICS OR COMPUTE
PascalPreemption
Request
ATW
DisplayRefresh
DisplayRefresh
VRWORKS GRAPHICSBringing Modern Visuals to VR
GREATER GEOMETRIC DETAIL W/ SINGLE PASS STEREO
FASTER PIXEL SHADING W/ LENS MATCHED SHADING
FASTER TIMEWARP WITH PREEMPTION
DIRECTION PROPAGATION
SIMULATING AUDIO IN VR
SYNTHESIS
Creation of Source Sounds
Location of Incoming Sound
How Sound Moves in Space
DIRECTIONAL AUDIO‘HRTF’ MODELS THE AUDIO CUES THAT YOUR BRAIN USES TO DETERMINE THE POSITION
OF SOUND
Level LLevel
R
Interaural Level Difference Interaural Time Difference
Time LTime R
Spectral Cues
Models Direction and Propagation Using Ray Tracing
Models Direction and Propagation Using Ray Tracing
VR TOUCHRealistic Collision Detection for Haptics
FORCE FRICTION
PhysX API
PhysX Constraint Solver
Haptics Layer
RESTITUTION
PHYSX FLEX
HAIRWORKS FLOW
DESTRUCTION
CLOTH
NVIDIA PHYSXSimulating behavior in VR
NVIDIA VR FUNHOUSEFull Source Code Available on Github
Comprehensive SDK for VR DevelopersGRAPHICS HEADSET AUDIOTOUCH & PHYSICS
NVIDIA VRWORKS
PROFESSIONALVIDEO
“ Forward thinking features in PASCAL VRWorks enable us to push the limits of photoreal graphics. ”
— Kjartan Pierre Emilsson, CEO, Sólfar Studios
DEVELOPERS EMBRACE THE NEW VRWORKS
“ The new VRWorks Lens Matched Shading and Single Pass Stereo features are smart, novel approaches to the performance challenges VR applications are faced with. ”
— Brandon Laatsch, Co-founder, Stress Level Zero
“ We took the performance headroom we gained by adding VRWorks technologies to Mars 2030 and we used it to bring new levels of immersion and realism to experience. ”
— Julian Reyes, lead VR producer for FUSION
“ We are looking forward to bringing NVIDIA's new PASCAL VRWorks features to Valkyrie to take the game’s visuals and performance to another level. ”
— Hilmar Veigar Pétursson, CEO, CCP Games
GROWING VRWORKS ADOPTION
APPLICATIONS
HEADSETS ENGINES
VRWORKS GAME ENGINE INTEGRATION
VRWorks Graphics Available Now on Github!
VRWorks is Coming to Unity Soon!
FOR MORE INFO ON VRWebsite
www.nvidia.com/virtualreality
VRWorks SDKhttps://developer.nvidia.com/vrworks
GameWorks SDKhttps://developer.nvidia.com/gameworks
Thank you and have a great AWE Europe!Questions?
Dominic Eskofier
Virtual Reality Manager [email protected]