enabling mobile virtual reality gaming | nizar romdan
TRANSCRIPT
![Page 1: Enabling Mobile Virtual Reality Gaming | Nizar Romdan](https://reader031.vdocuments.net/reader031/viewer/2022021815/5a6888017f8b9a177a8b57ad/html5/thumbnails/1.jpg)
1
Enabling Mobile Virtual Reality Gaming
Know Your Hardware!
Nizar Romdan
Director of Developer Ecosystem
16th February 2016
![Page 2: Enabling Mobile Virtual Reality Gaming | Nizar Romdan](https://reader031.vdocuments.net/reader031/viewer/2022021815/5a6888017f8b9a177a8b57ad/html5/thumbnails/2.jpg)
2
About ARMApps Processor
Cortex-A5
Cortex-A9Cortex-
A15
Cortex-A7
Mali™-400
Mali-T760
Mali-450
Mali-T604
Cortex-
A17Mali-T720
Mali-V500Mali-
DP500
Power
MgmtCortex-M3
Cortex-M0BluetoothCortex-M3
Cortex-M0
Cellular
ModemCortex®-R4
Cortex-R5
Cortex-R7
WiFiCortex-M4
SIMSecurCore®
SC300™
GPS
Cortex-M3
Cortex-M0
Flash
ControllerCortex-M3
Touchscreen
& Sensor
HubCortex-M0
Cortex-M3
Cortex-M4
Sensor HubCortex-
M0+
Camera
Cortex-M3
![Page 3: Enabling Mobile Virtual Reality Gaming | Nizar Romdan](https://reader031.vdocuments.net/reader031/viewer/2022021815/5a6888017f8b9a177a8b57ad/html5/thumbnails/3.jpg)
3
The ARM Business Model
2-3 yearsARM research
and development
Cost
incurred
2-3
yearsPartner chip
development
License
revenu
e
$
20+ yearsMultiple applications
development and
sales
Royalty
revenu
e
$
![Page 4: Enabling Mobile Virtual Reality Gaming | Nizar Romdan](https://reader031.vdocuments.net/reader031/viewer/2022021815/5a6888017f8b9a177a8b57ad/html5/thumbnails/4.jpg)
4
Real presence scales emotions 10x
The feeling of actually being there
![Page 5: Enabling Mobile Virtual Reality Gaming | Nizar Romdan](https://reader031.vdocuments.net/reader031/viewer/2022021815/5a6888017f8b9a177a8b57ad/html5/thumbnails/5.jpg)
5
VR is not new – People were discussing VR the 90’s
Expensive and bulky hardware
Inadequate technology offered limited graphics capabilities
Virtuality – Sega VR – Nintendo Virtual Boy
VR is now accessible
Emergence of low-cost hardware based on existing platforms
Mobile platforms offering console-like performance offering
more realism
VR is about immersing users into Virtual worlds
All the virtual world is graphically rendered
Introducing VRVirtuality
Sega VR
![Page 6: Enabling Mobile Virtual Reality Gaming | Nizar Romdan](https://reader031.vdocuments.net/reader031/viewer/2022021815/5a6888017f8b9a177a8b57ad/html5/thumbnails/6.jpg)
6
VR Use Cases
Games Experiences Training
Education TV/Film
Advertising
![Page 7: Enabling Mobile Virtual Reality Gaming | Nizar Romdan](https://reader031.vdocuments.net/reader031/viewer/2022021815/5a6888017f8b9a177a8b57ad/html5/thumbnails/7.jpg)
7
ARM Ice Cave VR Demo
Frozen ice cave demo developed in Unity 5.0
High-end graphical showcase in Virtual Reality
Geomerics Enlighten
Reflections, refractions, soft shadows, …
OpenGL ES 3.0
Artwork from AAA art studio (RealtimeUK)
Optimised for Mali-T760 MP8 and above
Implemented camera streaming from GearVRdevice to another device (other people can see what the player is doing)
Alpha version shown at APM 2015
Final at Unite Boston 21st September 2015 Screenshots from Samsung Galaxy S6
![Page 8: Enabling Mobile Virtual Reality Gaming | Nizar Romdan](https://reader031.vdocuments.net/reader031/viewer/2022021815/5a6888017f8b9a177a8b57ad/html5/thumbnails/8.jpg)
8
![Page 9: Enabling Mobile Virtual Reality Gaming | Nizar Romdan](https://reader031.vdocuments.net/reader031/viewer/2022021815/5a6888017f8b9a177a8b57ad/html5/thumbnails/9.jpg)
9
![Page 10: Enabling Mobile Virtual Reality Gaming | Nizar Romdan](https://reader031.vdocuments.net/reader031/viewer/2022021815/5a6888017f8b9a177a8b57ad/html5/thumbnails/10.jpg)
10
![Page 11: Enabling Mobile Virtual Reality Gaming | Nizar Romdan](https://reader031.vdocuments.net/reader031/viewer/2022021815/5a6888017f8b9a177a8b57ad/html5/thumbnails/11.jpg)
11
VR Opportunities Could Hit $30B by 2020 – Likely to be mainly from Gaming & Hardware
![Page 12: Enabling Mobile Virtual Reality Gaming | Nizar Romdan](https://reader031.vdocuments.net/reader031/viewer/2022021815/5a6888017f8b9a177a8b57ad/html5/thumbnails/12.jpg)
12
VR Hardware Landscape Worldwide
![Page 13: Enabling Mobile Virtual Reality Gaming | Nizar Romdan](https://reader031.vdocuments.net/reader031/viewer/2022021815/5a6888017f8b9a177a8b57ad/html5/thumbnails/13.jpg)
13
Virtual Reality Segmentation
Samsung Gear VR Zeiss VR One Google Cardboard
Oculus Rift Sony Morpheus HTC Vive (Steam VR)
Mobile
Desktop/Console
![Page 14: Enabling Mobile Virtual Reality Gaming | Nizar Romdan](https://reader031.vdocuments.net/reader031/viewer/2022021815/5a6888017f8b9a177a8b57ad/html5/thumbnails/14.jpg)
14
Headset based on Smartphone
Partnership between Samsung Mobile & Oculus
First version for Galaxy Note 4 (Mali-T760)
Second version for Galaxy S6 (Mali-T760)
Resolution 1080p – 60 Hz
First VR appstore
Introducing GearVR
![Page 15: Enabling Mobile Virtual Reality Gaming | Nizar Romdan](https://reader031.vdocuments.net/reader031/viewer/2022021815/5a6888017f8b9a177a8b57ad/html5/thumbnails/15.jpg)
15
China VR Headsets – DeePoon Case Study
![Page 16: Enabling Mobile Virtual Reality Gaming | Nizar Romdan](https://reader031.vdocuments.net/reader031/viewer/2022021815/5a6888017f8b9a177a8b57ad/html5/thumbnails/16.jpg)
16
2020 Opportunity in Mobile Computing
Mid-Range
Smartphone
Entry-Level
Smartphones
Premium
Smartphone
650mdevices in 2020
450mdevices in 2020
1,000mdevices in 2020
Laptops
and 2-in-1s
250mdevices in 2020
Tablets
450mdevices in 2020
Source: Gartner and ARM
estimates
![Page 17: Enabling Mobile Virtual Reality Gaming | Nizar Romdan](https://reader031.vdocuments.net/reader031/viewer/2022021815/5a6888017f8b9a177a8b57ad/html5/thumbnails/17.jpg)
17
How Long Until Desktop GPU Compute Is In Mobile?
0
1,000
2,000
3,000
4,000
5,000
6,000
2006 2008 2010 2012 2014 2016 2018
GF
LO
PS
/Se
c
PS3
Xbox 360
Current mobile compute capability very near console
Mobile Today
PS4
XboxOne
![Page 18: Enabling Mobile Virtual Reality Gaming | Nizar Romdan](https://reader031.vdocuments.net/reader031/viewer/2022021815/5a6888017f8b9a177a8b57ad/html5/thumbnails/18.jpg)
18
Stereoscopic Display (via headset) – Providing wearer with a sense of depth
Render left eye and right eye independently
Barrel distortion effect applied to rendered output via post processing
Sensor input for head-tracking – To add realism to the output image
Sensor input for interaction (hands-tracking) – To provide an immersive
experience
How Virtual Reality Works
Left eye
Right eye
Eye gap (IPD)
Left eye Right eye
Lens
Lens
2xCPU
Processing
2xVertex
processing
IncreasedFragment
Processing
![Page 19: Enabling Mobile Virtual Reality Gaming | Nizar Romdan](https://reader031.vdocuments.net/reader031/viewer/2022021815/5a6888017f8b9a177a8b57ad/html5/thumbnails/19.jpg)
19
Virtual Reality Technical Challenges
![Page 20: Enabling Mobile Virtual Reality Gaming | Nizar Romdan](https://reader031.vdocuments.net/reader031/viewer/2022021815/5a6888017f8b9a177a8b57ad/html5/thumbnails/20.jpg)
20
ARM Graphics Developer Centre –
malideveloper.arm.com
Supporting Mali
developers with a
full range of
resources through
one, easy-access
portal
![Page 21: Enabling Mobile Virtual Reality Gaming | Nizar Romdan](https://reader031.vdocuments.net/reader031/viewer/2022021815/5a6888017f8b9a177a8b57ad/html5/thumbnails/21.jpg)
21
• API Trace & Debug
• OpenGL® ES, OpenCL™
• Debug and improve performance at frame level
Mali Graphics Debugger
Mali GPU
- Timeline
- HW Counters
- OpenCL visualizer
ARM DS-5 Streamline
• Analyze shader performance
• Generates binary shaders
• Command line tool
Mali Offline Compiler
• Emulate OpenGL ES 2.0 and 3.0
• Windows and Linux
• Khronos Conformant
OpenGL ES Emulator
• Command line and GUI
• ETC, ETC2, ASTC
• 3D textures
Texture Compression Tool
Integration with partners’ tools
Third Party Tools
Mali GPU Software ToolsPerformance Analysis, Debug, and Software Development
![Page 22: Enabling Mobile Virtual Reality Gaming | Nizar Romdan](https://reader031.vdocuments.net/reader031/viewer/2022021815/5a6888017f8b9a177a8b57ad/html5/thumbnails/22.jpg)
22
ARM Guide to The Unity Developers – Released
Available under http://malideveloper.arm.com/armunityguide
Released at Unite Seattle 2014, and regularly updated and expanded at each key
Unite event
ARM-specific and general advice
Optimization process
Profiling and Debugging
Advanced Graphics Techniques
![Page 23: Enabling Mobile Virtual Reality Gaming | Nizar Romdan](https://reader031.vdocuments.net/reader031/viewer/2022021815/5a6888017f8b9a177a8b57ad/html5/thumbnails/23.jpg)
23
ARM Mali VR SDK v0.1 available now under http://malideveloper.arm.com/resources/sdks/ Tutorial – Introduces the concepts of stereoscopic vision
Fundamentals of stereoscopic rendering Effects of lenses and methods for correcting lens distortion in software Calibration
Sample code Eye separation – Viewing distance – Size of screen – Lens separation
ARM Mali VR SDK v0.2 to be released mid Dec 2015 Support for using OVR_multiview extension Foveated rendering
ARM Mali VR SDK v1.0 planned for Mar16 @ GDC’16 Expose functionality as a user-friendly framework Add support for priority contexts
ARM Mali VR SDK under Development
![Page 24: Enabling Mobile Virtual Reality Gaming | Nizar Romdan](https://reader031.vdocuments.net/reader031/viewer/2022021815/5a6888017f8b9a177a8b57ad/html5/thumbnails/24.jpg)
24
Thank You!
malideveloper.arm.com