mobile day - webvr

VR/AR/Mixed RealityRabimba Karanjai@rabimba

VR/AR/MR in a NutshellRabimba Karanjai, Mozilla Research Mixed Reality Mozilla Tech Speaker@rabimba

What the Heck Doodle?

ARKit ARCore

MRVR

XRAR

OculusSonyVive

Meta

HoloLens

What You’ll Learn Today● What is VR / AR / MR?● Why is this all happening now?● What do I really need to know?

What is VR/AR/MR?

VR: Virtual Reality AR: Augmented Reality

MR: Mixed Reality XR: We don’t know Reality

Virtual Reality● Completely immersive● Wide field of view● Fast refresh rate (90+fps)● Fast sensor refresh

Augmented Reality● Overlaid on real world● Anchored to real world● Non-immersive

Partial AR

Why Now?

The Long History of VR/AR/MR

Ivan Sutherland, MIT

Sword of Damocles (1968)

3D wireframe rooms

1980s: VPL

● Data Glove● Eye Phone● Data Suit

Mattel’s Nintendo Power Glove

1990s: Virtuality

Virtual Reality Markup Language

● Scene based● In the browser● Slow to download● Slow to render

Smartphone Era

Smartphone+ Era

● Dual cameras● Depth sensing cameras● Machine Learning in

hardware● Even more powerful

GPU & CPU

What do I need to know?

Tech Stack for VR & AR

Input: Sensors & Camera Processing: Image & Sensor Fusion

Drawing: 3D Graphics API Application: Framework

Input: Sensors & Camera● Accelerometer: relative to gravity● Magnetometer: relative to earth’s magnetic field● Gyroscope: relative angle movement

○ 9DOF sensor breakout board for 15$ @ Adafruit!

● Optical Camera: standard phone camera● Stereo Camera: dual camera setup● Depth Sensing Camera: Kinect, Face ID

Processing the 3D Environment

● Markers: Symbols or Beacons● SLAM: Simultaneous Localization and Mapping● MonoSLAM: SLAM in real-time with just one camera

HTC Vive Sensor Beacontwo beacons + headset sensors = highly accurate

Processing: VR/AR APIs● Vuforia● ARToolkit● WebVR● ARKit● ARCore● WebXR

Vuforia

ARToolkit

WebVR

● Mobile: used by web apps for Google Cardboard on any phone.

● Desktop: works with major HMDs

● Shipping: Firefox for Windows

● Beta: Chrome, FF for other platforms

ARKit

Tango ARCore

WebXR: Work In Progress!● Proposed Future Web Standard● Extend WebVR with new features● Work with ARKit & ARCore● Polyfill for old browsers using WebRTC + sensors

https://github.com/mozilla/webxr-apihttps://github.com/mozilla/webxr-polyfill

Drawing: Graphics APIs

● Medium Level○ OpenGL (ES)○ DirectX○ WebGL

● Low Level○ Metal○ Vulkan○ Direct X 12

● High Level○ Unity○ Unreal○ Three.js○ Babylon.js

Application: Frameworks● Scene Kit● Cardboard & Daydream● Hololens Academy● React VR● ArgonJS● A-Frame

What do you *really* need to know?

● Start with web stack● Follow Mozilla and Google and Microsoft● Play with A-Frame● Get a Cardboard viewer.

What can I do?● Experiment! We are still in the Model T days!● No one knows what the killer app will be.● First new medium in decades● Go play!

State of WebVR and AframeThe web framework for building VR experiences

@rabimba | Mozilla VR | aframe.io

Friction of VR Ecosystems

Gatekeepers Installs Closed

WebVRAn open virtual reality platform with the advantages of

the Web

Open Connected Instant

Browser APIs that enable WebGL rendering to headsetsand access to VR sensors

https://w3c.github.io/webvr/

FirefoxNightly

Chromium(Experimental)

SamsungInternet

MobilePolyfill

A-FrameA declarative framework for building virtual reality

experiences on the Web

Hello World<a-scene>

<a-box color="#4CC3D9" position="-1 0.5 -3" rotation="0 45 0"></a-box>

<a-cylinder color="#FFC65D" position="1 0.75 -3" radius="0.5" height="1.5

<a-sphere color="#EF2D5E" position="0 1.25 -5" radius="1.25"></a-sphere>

<a-plane color="#7BC8A4" position="0 0 -4" rotation="-90 0 0" width="4" h

<a-sky color="#ECECEC"></a-sky>

</a-scene>

Hello World

<a-animation> <a-box> <a-camera> <a-circle> <a-collada-model>

<a-cone> <a-cursor> <a-curvedimage> <a-cylinder>

<a-dodecahedron> <a-isocahedron> <a-image> <a-light>

<a-obj-model> <a-plane> <a-ring> <a-tetrahedron> <a-torus>

<a-torus-knot> <a-sky> <a-sound> <a-sphere> <a-videosphere>

RegistryCurated collection of A-Frame components

RegistryCurated collection of A-Frame components

Motion CaptureRecord then develop room scale VR on the go

Remix Glitch

Communityhttps://aframe.io/blog/

Powerful and Extensible

Art - A-Painter@mozillavr

Journalism - Fear of the SkyAmnesty International UK

Journalism - Journey toMars

The Washington Post

Guri VRBuild your own world in VR

Sandbox - Aincraft@andgokevin

AincraftLets play

Data Visualization - Adit@datatitian

Gaming - A-Blast@mozillavr

AR - AR.js + A-Frame@jerome_etienne

Come build WebVR with us!

aframe.io/community

Thank you

@rabimba | rabimba@mozilla.com| aframe.io