AR/VR in perspective

Tech , Effect and ValueCarl Henrikson & Ahmed Assal

Building the right things

Building thingsright

Running thingsefficiently

Carl HenriksonSenior Consultant Business Advisory

Virtual reality - an old dream which is finally becoming reality

Stanford University Libraries

1966

AUGMENTED REALITY

The Ultimate Display – a.k.a. Damocles Sword

Head mounted augmented reality display

Ivan Sutherland 1968

80-ies and 90-ies...

But finally...

2014: Rirtualenaissance

V

VRAR

Learning and training

Decision making

Communication

Sales

Marketing

Immersive design

Location based

Entertainmentand information

Emerging applications

Learning and training

Virtual manuals

SimulationAR and Industrial

maintenance

Assistive AR/VR tools

Decision making

Real time data and analytics visualization

What if scenarios

Remote systems control and analysis

Communication

Augmented vision

Remote experts Remote collaboration

Telepresence

Remote Collaboration

Remote Expert

Telepresence

25

Sales

Showcases and demos

Configurators & customizers

Trials

Marketing

Experential campaigns

Brand recognition Virtual characters

Store attraction

Immersive design

Conceptual design Design review

Location based

Virtual tours On location gaming

Entertainment& information

Games

Movies

Concerts

Emerging applications

Crime scene reconstruction

Therapy, Pain relief Virtual workspaces

Immersive journalism

MemoAR

Technology Effect Value

The TECHNOLOGY The EFFECT that can be / need to be achived The business/customer MOTIVE or DRIVER

Interactive VR

Consuming VR

VR 360

Cameras

Empathy

engine

360 experience

Virtual

presence

Real time

virtual

presence

Near real

experience /

simulation

Capture your

experience

Anyone can

create a VR

environment

Be able to build/interact with a world around you

Cheap, Easy

access

Time & place

travel

Manipulate the

brain

Experience something

which is not built

Different for each customer and use case

The business value and customer

motive

Begin with...

The effect

which is

needed

then...

The technology needed to

achieve the effect

and finally...

But, how? ...

Outside-in metodology to understand users

gain

gain

gain

gain

Charity organization working with homeless people

Main objective: communicate what it’s like to be homeless so that people get emphatic with their cause and donate money and time

”Empathy engine”

Outside-in metodology to understand users

To get the most from this technology we must...

Become imagineers!

* Stolen with pride from Walt Disney Inc.

*

KEEP CALMAR + VR

IS CURRENTLY AT 1%

AR / VR / MR / 3D RealtimeGraphicsA Technical Perspective

VR / AR / MR / Realtime 3D

Current - Future Future+

Software

Hardware

Content

Procedural

Content

Generation

Automatic and Semi –automatic

Content Generation

Automatic

Animation

Creation Automatic

Mesh Creation

Automated, Mixed-authoring and Procedural Content Generation

44

Automatic Animation Generation

45

Natural Environements

47

Spectacular Hyper-realism

48

Mapbox & Unity 3D

49

Mapbox Styling

50

L-systems and Plants – SpeedTree

51

L-systems and Plants – Xfrog

52

Cities, Streets, Buildings & CityEngine

53

Cities, Streets, Buildings & CityEngine

54

Crowd Creation & Animation – Anima 3

55

Today’s journey

56

AR Works - Occlusion

57

“AR systems never “work” or “don’t work”. It’s always a question of do things

work well enough in a wide enough range of situations.”

58

Google search volumes (AR) over time (trends)

PokemonGo Distribution: July 2016

22-sept-2017: initial release Iphone 819-sept-2017: Initial release of iOS 11 iOS

NEW API: ARKit

April 2012: Google unveiled Project Glass

“Similarly, in your business, be sure that before you focus on theresults or outcome, that you make it crystal clear what problemsyou solve, or why people might need what you offer” - Ian Altmanat Forbes

Jan 2015

AR technologies compared

61

Microsoft Hololens Meta 2 ODG R-7

Magic Leap, Google Glass 2.0, Vuzix, ODG R-9, Daqri Smart glasses, ...

Daqri Moverio BT-300

+ universal use+ poweful, open platform+ standalone device+ strong go-to-market push

- limited battery life

+ universal use+ poweful platform

- connected device

+ universal use+ lightweight

- limited FoV- limited battery life

+ industry use tailored+ powerful platform

- industry use only- limited battery life

+ industry use tailored+ powerful platform

- industry use only- limited FoV

AR SDKsRegular Handheld Devices

62

Vuforia

ARCore

ARKit

Dedicated HWHMD Tango Tablet

Vuforia

ARCore

ARKit

Hololens

Tango

AR SDKs on Handheld and HMD Devices63

Augmented Reality

SLAM

Sensor Fusion

Computer VisionPose Deblurring

Visual-inertial Odometry

Optimal CorrectionSurface Estimation

Camera Calibration

Nonlinear OptimizationScene Understanding

Inertial Measurement Unit

Triangulation

AR SDKs on Handheld and HMD Devices64

It enables developers to position and orient virtual objects, such as

3D models and other media, in relation to real world images when these are viewed through the camera of a mobile or HMD device.

Motion TrackingVisual Inertial Odometry (VIO) – to track your position in space (your 6dof pose) in real-timeYour pose is recalculated in-between every frame refresh on your display by matching a point in the real world to a pixel on the camera sensor each frame

AR SDKs on Handheld and HMD Devices65

It enables developers to position and orient virtual objects, such as

3D models and other media, in relation to real world images when these are viewed through the camera of a mobile or HMD device.

3D Reconstruction (point cloud)

66

ARKit vs Tango vs Hololens

HW Design

SW Stack

"RGB-IR" camera for color image and infrared depth detectionFisheye motion cameraAccelerometer, gyroscope & magnetometerAmbient light sensorsMicrosoft Holographic Processing Unit

VIO (motion tracking) Sparse Mapping (area learning)Dense 3D reconstruction (depth perception).

Regular phone cameraAccelerometer & gyroscope

Highly accurate Dead Reckoning Guess to

compensate lack of stereoscopic images

ARKit – Augmented Reality for iOS

67

• API that lets third-party developers to build Augmented Reality apps for iOS• Available only for processors Apple A9 and later (6S, 6S Plus and later) due to performance• Available only operatig system iOS 11 and later (officially released on 19-sept-2017)

Optimal performance from model iPhone 8

• Depth sensing camera system calibrated for AR• Lo-light and 60 fps• New gyro and accelerometer• Accurate motion tracking

Microsoft Hololens

68

• Accelerometer, gyroscope, magnetometer

• Spatial Mapping

• Gesture recognition

• 3D display (FOV: 30°×17.5°)

• Video (2.4 megapixel)

• Sound

• Bluetooth

• Wi-Fi

VR everywhere at MWC!

VR was the big attraction! ..but the real use cases were few!

VR devices in all ranges!

March $599April $799

2nd gen $99

May $200

Oct $499

Multiple version $5 Co

nsu

min

gIn

teractin

g

But Google DayDream and ARM next Gen processors will drive mobile VR from consuming to interactive

71

THANKS FOR LISTENING!