building blocks of webvr › webvr-meetup › mirek - vr + webvr introduction.pdfapis useful for...

www.swing3d.io

Building Blocks of WebVRIntroduction to VR and WebVRwith Mirek Ciastek, SwingDev’s Front-end Developer

What’s virtual reality

It’s a computer-generated scenario that simulates a realistic experience. The immersive environment can be similar to the real world in order to create a lifelike experience grounded in reality or sci-fi.

Back to the past

90’s and recently

Today

Designing for VR

Key concpets of VR

Head-mounted display (HMD) Position / Orientation / Velocity / Acceleration

Field of ViewStereoscopic vision

UX of VR

• Comfortable using

• In-world interface

• Immersive sounds

• Uninterrupted movement

• Interaction with world

Problems of VR

Hardware

Health issues

Best experience = High price

+

VR in browser

From VR to WebVR

WebVR is an open specification that makes it possible to experience VR in your browser. The goal is to make it easier for everyone to get into VR experiences, no matter what device you have.

All this thanks to browser environment!

APIs useful for WebVR

• WebVR API - manages displays

• WebGL - provides a way to run 3D in the browser

• Gamepad API - helps with using controllers

• Web Audio API - if you want to add some sound

• Web Video API - for 360 videos

• Web Workers - might give good performance boost

Libraries

Let’s make a VR in the browser

Make a VR scene

setScene () { this.camera = new THREE.PerspectiveCamera( 60, ASPECT_RATIO, 0.01, 10000 )

this.scene = new THREE.Scene() this.scene.add(this.camera) this.scene.background = new THREE.Color().setHSL(0.6, 0, 1) this.scene.fog = new THREE.Fog( this.scene.background, 1, 5000 )}

setRenderer () { const canvasEl = document.getElementById('scene')

this.renderer = new THREE.WebGLRenderer({ canvas: canvasEl }) this.renderer.setPixelRatio(window.devicePixelRatio) this.renderer.setSize(window.innerWidth, window.innerHeight) this.renderer.gammaInput = true this.renderer.gammaOutput = true this.renderer.setClearColor(0x000000)

this.renderer.shadowMap.enabled = true}

setComponents () { const sky = Sky() const ground = Ground() this.wall = new Wall(this.renderer) this.turret = new Turret(this.renderer) this.cursor = new Cursor()

this.turret.init() .then(this.handleModelLoad)

this.wall.init() .then(this.handleModelLoad)

this.scene.add(this.cursor.mesh) this.scene.add(sky) this.scene.add(ground)}

setLights () { const { hemiLight, dirLight } = makeLights() this.scene.add(hemiLight) this.scene.add(dirLight)}

Allow to move camera

MousePanControlshandleMouseMove = (e) => { if (!this.tracking) { return }

const width = this.target.innerWidth || this.target.clientWidth const height = this.target.innerHeight || this.target.clientHeight

const deltaX = (typeof this.lastX === 'number') ? e.screenX - this.lastX : 0 const deltaY = (typeof this.lastY === 'number') ? e.screenY - this.lastY : 0 this.lastX = e.screenX this.lastY = e.screenY

this.yaw += THREE.Math.degToRad( deltaX / width * this.camera.fov * this.camera.aspect )

this.pitch += THREE.Math.degToRad(deltaY / height * this.camera.fov) this.pitch = Math.max(-HALF_PI, Math.min(HALF_PI, this.pitch))};

DeviceOrientationControlsupdate () { if (!this.enabled) { return } const alpha = this.deviceOrientation.alpha || 0 const beta = this.deviceOrientation.beta || 0 const gamma = this.deviceOrientation.gamma || 0 const orientation = this.screenOrientation

// Update the camera rotation quaternion const quaternion = this.camera.quaternion euler.set(beta, alpha, -gamma, 'YXZ') quaternion.setFromEuler(euler)

if (this._initialAlpha !== null) { rotation.setFromAxisAngle(Y_UNIT, -this._initialAlpha) quaternion.premultiply(rotation) }

quaternion.multiply(SCREEN_ROTATION) rotation.setFromAxisAngle(Z_UNIT, -orientation) quaternion.multiply(rotation)}

VRControlsclass VRControls { constructor (camera, vrDisplay) { this.camera = camera this._vrDisplay = vrDisplay }

update (options) { const pose = options.frameData ? options.frameData.pose : null if (pose) { if (pose.position) { this.camera.position.fromArray(pose.position) } if (pose.orientation) { this.camera.quaternion.fromArray(pose.orientation) } } }}

Allow to interact with world

GamepadControlslistenGamepadEvents () { const gamepads = getGamepads() // navigator.getGamepads

for (let i = 0; i < gamepads.length; i += 1) { if (gamepads[i]) { const buttons = gamepads[i].buttons

for (let j = 0; j < buttons.length; j += 1) { if (isPressed(buttons[j])) { this.batchedEvents.push({ type: 'GamepadEvent', eventType: 'keydown', button: buttons[j], gamepad: i }) } } } }}

Check for box hit

hit (origin, direction) { raycaster.set(origin, direction)

const intersects = raycaster.intersectObjects(this.root.children)

for (let i = 0; i < intersects.length; i += 1) { const { object } = intersects[i] this.root.remove(object) }

if (this.root.children.length === 0) { this.build() }}

Add some extra features

Crosshairconst texture = new THREE.TextureLoader().load('images/crosshair.png')const geometry = new THREE.PlaneGeometry( DEFAULT_CURSOR_WIDTH, DEFAULT_CURSOR_WIDTH)

const material = new THREE.MeshBasicMaterial({ transparent: true, side: THREE.DoubleSide, depthTest: false, depthWrite: false, map: texture})

this.mesh = new THREE.Mesh(geometry, material)this.mesh.raycast = () => nullthis.mesh.renderOrder = 1

Explosionsconst particleSettings = { texture: { value: textureLoader.load(smokeImage) }, depthTest: true, depthWrite: false, blending: THREE.NormalBlending, maxParticleCount: 1000}

const emitters = [ { particleCount: 600, type: SPE.distributions.SPHERE, position: { radius: 0.1 }, maxAge: { value: 0.5 }, activeMultiplier: 20, velocity: { value: new THREE.Vector3(1.2) }, size: { value: 1.5 }, opacity: { value: [0.5, 0] } }]

// init particlesinitParticles () { this.particleGroup = new SPE.Group( particleSettings )

this.particleGroup.addPool(1, emitters, false) this.headGroup.add(this.particleGroup.mesh)}

// play sound and show smoketriggerSmoke () { player.play('shot') this.particleGroup.triggerPoolEmitter( 1, smokePosition )}

I used Shader Particle Engine

Soundsclass SoundPlayer { constructor () { this.context = new (window.AudioContext || window.webkitAudioContext)() this.bufferLoader = new BufferLoader( this.context, sounds, this.handleLoad )

this.loaded = false this.bufferLoader.load() }

handleLoad = () => { this.loaded = true }

play (soundName) { if (this.loaded) { const source = this.context.createBufferSource() source.buffer = this.bufferLoader.bufferList[soundName]

source.connect(this.context.destination) source.start(0) } }}

…and allow to enter VR mode

VRDisplay API provides all needed methods and properties to serve your scene in VR compatible format

• eyes position - you need to slice your scene in half

• isPresenting flag - check if you can serve VR frames

• frameData - includes device position, etc.

• submitFrame - remember to call it when your frame is prepared

Check three.js VREffect interface for more details

Result

See the demo

Check out the code

How you can extend it

• Use any WebVR framework (A-frame or ReactVR) - the farther you go the harder it gets to maintain all the things

• Add any UI to allow user to track their progress

• Add some physics - make the wall collapse

• Add support for 3DoF and 6DoF controller - yes, you can do it in your browser!

Further reading 1/2• The User Experience of Virtual Reality

• Virtual Reality Basics

• VR for UX Designers: What I Learned During My First Project

• History Of Virtual Reality

• The very real health dangers of virtual reality

• The Story of VR - A Look at the History Behind Virtual Reality

• Design Practices in Virtual Reality

• The Future of Virtual Reality

• Designing Screen Interfaces for VR (video)

Further reading 2/2• WebVR concepts

• Introduction to VR Web

• Building Virtual Reality on the Web with WebVR (video)

• Using the WebVR API

• Using the Gamepad API

• Using VR controllers with WebVR

• Getting started with three.js

• Web Audio API

Thank you