Introduction to mixed realities

Course 01

Content● Augmented Reality (AR)

● Virtual Reality (VR)

● Mixed Reality (MR)

● Extended Reality (XR)

2

Augmented Reality (AR) - Definitions● Augmented reality is a field of computer research which deals with the

combination of real-world and computer-generated data (El Kaim, 2016)● AR is the art of superimposing computer generated content over a live

view of the world● AR is the integration of digital information with live video or the user's

environment in real time● AR is a combination of a real scene viewed by a user and a virtual scene

generated by a computer that augments the scene with additional information (Ganatra, 2015)

3

AR - Dictionary.com

augmented realitynoun

Digital Technology. Abbreviation: AR

1. an enhanced image or environment as viewed on a screen or other display, produced by overlaying computer-generated images, sounds, or other data on a real-world environment.

2. a system or technology used to produce such an enhanced environment: With augmented reality, you can get relevant extra information about what you are looking at.

4

AR SystemsAn AR system adds virtual computer generated objects, audio and other sense enhancements to a real-world environment which is interactive in real time

Aim: To enhance a person’s performance and perception of the world

5

AR Market

6

Display● Head-mounted Display (HMD)● Eye Glasses● Contact Lenses● Virtual Retina Display● Handheld● Spatial

7

How AR works?● Pick A Real World Scene● Add your Virtual Objects in it (using a tracker or not)● Delete Real World Objects● Not Virtual Reality since Environment Real

8

Tracking system

9

Tracking Technologieshttps://www.slideshare.net/marknb00/comp-4010-lecture10-ar-tracking

10

Marker Tracking

11

Tracking...

12

Markerless TrackingImages

Location based tracking

13

AR PlatformsAugment https://www.augment.com/

Eon Reality http://www.eonreality.com/eon-ar/

Hoppala http://www.hoppala-agency.com/

OpenHybrid http://openhybrid.org/

Vuforia https://www.ptc.com/en/products/vuforia

14

Current and Future Revenue of AR

15

16

AR ApplicationsPokemon Go https://pokemongolive.com/ en/

Ink Hunter http://www.inkhunter.tattoo/

Star Chart https://play.google.com/store/apps/details?id=com.escapistgames.starchart.ar.free&hl=en_US

Ingres https://play.google.com/store/apps/details?id=com.nianticproject.ingress&hl=en

IKEA Catalogue https://www.ikea.com/gb/en/customer-service/mobile-apps/ 17

What is Virtual Reality (VR)?Virtual Reality (VR) (or ”immersive multimedia”) is a computer simulated environment that can simulate physical presence in places in the non-physical world or imagined world. Virtual reality can recreate sensory experiences, which include virtual taste, sight, smell, sound, and touch.

VR refers to a high-end user interface that involves real-time simulation and interactions through multiple sensorial channels

VR is able to immerse you in a computer generated world of your own making a room, a city, the interior of human body. With VR, you can explore any uncharted territory of the human imagination

VR is a computer generated world with which the user can interact 18

History of VR - SensoramaIn 1962 Morton Heilig built an user console called Sensorama (that included a stereoscopic display, fans, or emitters, stereo speakers and a moving chair). This enabled the user watch television in three dimensional ways

19

History of VR - HMDTelesphere Mask (patented 1960 by Morton Heilig) was the first example of a head-mounted display (HMD). The headset provided stereoscopic 3D and wide vision with stereo sound

In 1961, two Philco Corporation engineers (Comeau & Bryan) developed the first precursor to the HMD as we know it today – the Headsight

20

History of VR - Ultimate DisplayIn 1965, a computer scientist named Ivan Sutherland envisioned what he called the “Ultimate Display”. After using this display a person imagines the virtual world very similar to the real world

21

Types of VRWindow on World Systems (WoW)

Video Mapping

Immersive Systems

Telepresence

Distributed VR

22

VR - Window on World Systems (WoW)Some systems use a conventional computer monitor to display the visual world. This sometimes called Desktop VR or a Window on a World (WoW)

“The challenge to computer graphics is to make the picture in the window look real, sound real and the objects act real." [Computer Graphics V26#3]

23

VR - Video MappingMerges a video input of the user's silhouette with a 2D computer graphic.

The user watches a monitor that shows his body's interaction with the world

24

VR - Immersive SystemsImmerse the user's personal viewpoint inside the virtual world

These "immersive" VR systems are often equipped with a HMD

The user has no visual contact with the physical world

25

VR - TelepresenceRepresents a complete computer generated worlds

Links remote sensors in the real world with the senses of a human operator. The remote sensors might be located on a robot. Useful for performing operations in dangerous environments

26

VR - Distributed VRA simulated world runs on several computers which are connected over network and the people are able to interact in real time, sharing the same virtual world

27

28

VR - HardwareHMD

Data gloves, control devices

(BOOM) Binocular Omni-Orientation Monitor

The CAVE (Cave Automatic Virtual Environment)

29

VR - SoftwareThree Dimensional Modelling Software (VRML- virtual reality modelling language, X3D, Java 3D, AC3D, Alice)

Two-Dimensional Graphics Software (Adobe Photoshop, Adobe Illustrator, Fractal Design Painter)

Digital Sound Editing Software (allows to cut, splice, mix, and loop the sounds)

Simulation Software (Sense-8, Division, Superscape, Cosmo, VRML)

30

Augmented Reality (AR) vs. Virtual Reality (VR)

● System augments the real world scene

● User maintains a sense of presence in real world

● Needs a mechanism to combine virtual and real worlds

● Totally immersive environment● While immersed, the user

cannot see the real world around him

● Visual senses are under control of system (sometimes aural and proprioceptive senses too)

31

Reality vs. Augmented Reality vs. Virtual Reality

Reality Augmented Reality Virtual Reality

32

Mixed Reality (MR)Mixed reality is "anywhere between the extrema of the virtuality continuum” (Milgram and Kishino, 1994)

33

34

Mixed Reality (MR)The seamless merging of real space and virtual space

Integrate the computer-generated virtual objects into the physical world which become in a sense an equal part of our natural environment

MR is everything between augmented reality (AR) and augmented virtuality (AV) (Rokhsaritalemi et al., 2020) (see below figure (Milgram and Kishino, 1994))

35

MR - GoalThe main goal of MR is the creation of a big space by merging real and virtual environments wherein real and virtual objects coexist and interact in real-time for user scenarios (Rokhsaritalemi et al., 2020)

Mixed Reality is a class of simulators that combines both virtual and real objects to create a hybrid of the virtual and real worlds

36

MR Characters (Rokhsaritalemi et al., 2020)MR characters may be defined using three terms: immersion, information, and interaction

Immersion refers to the real-time processing and interpretation of the user’s environment

User interaction with MR space is performed without any controller, using natural communication modes such as gestures, voice, and gaze

Information refers to virtual objects being registered in time and space in the user environment. This allows the user to interact with real and virtual objects in the user environment

37

38

39

General Architecture of a MR System

(Rokhsaritalemi et al., 2020) 40

The Mixed Reality Spectrumhttps://www.youtube.com/watch?v=_xpI0JosYUk

41

Extended Reality (XR)XR is an emerging umbrella term for all the immersive technologies (AR, VR and MR)

42

XR Applications for BusinessRetail: XR gives customers the ability to try before they buy (Rolex, IKEA)

Training: XR can provide training tools that are hyper-realistic that will help soldiers, healthcare professionals, pilots/astronauts, chemists, etc.

Remote work: Workers can connect to the home office

Marketing: The possibilities to engage with prospective customers and consumers through XR

Real estate: individuals can “walk through” spaces to decide if they want it

Entertainment: this industry will continue to use immersive technologies43

ConclusionsVisualization of complicated, large data is helpful for understanding and analysis

XR offers us a new way to interact with computer and with applications

XR enables us to experience the virtual world that is impossible in real world

XR is changing our life, eventually XR will increasingly become a part of our life

44

QuestionsWhat are the advantages of using AR/VR/MR technologies?

What are the disadvantages of using AR/VR/MR technologies?

45

Bibliography● El Kaim, W. (2016) Introduction to Augmented Reality. SlideShare

https://www.slideshare.net/welkaim/introduction-to-augmented-reality-66641831● Ganatra, K. (2015) Augmented Reality. SlideShare

https://www.slideshare.net/Khyati14Ganatra/augmented-reality-ppt-47315337● Milgram, P., Kishino, F. (1994) A taxonomy of mixed reality visual displays. IEICE (Institute of

Electronics, Information and Communication Engineers) Transactions on Information and Systems, Special issue on Networked Reality.

● Patel, M. (2015) Virtual Reality https://www.slideshare.net/MohitPatel34/virtual-reality-53825283

● Patel, V. (2016) Virtual Reality https://www.slideshare.net/viral0004/virtual-reality-61247706 ● Rokhsaritalemi, S., Sadeghi-Niaraki, A., Choi, S. M. (2020) A Review on Mixed Reality: Current

Trends, Challenges and Prospects. Appl. Sci. 2020, 10, 636; doi:10.3390/app10020636 ● Silva, R., Oliveira, J. C., Giraldi, G. A. (2003) Introduction to Augmented Reality. LNCC Research

Report #25/2003, National Laboratory for Scientific Computation.46

Links● Augmented Reality Market:

https://www.marketsandmarkets.com/Market-Reports/augmented-reality-market-82758548.html● Augmented Reality Headsets Market Size, Share & Trends Analysis Report By Type (Standalone,

Tethered, Smartphone-enabled), By Application (Enterprise, Consumer), By Region, And Segment Forecasts, 2019 - 2025 https://www.grandviewresearch.com/industry-analysis/augmented-reality-ar-headsets-market

● Augmented reality for architectural visualisation https://pt.slideshare.net/Sochipeshkom/augmented-reality-for-architectural-visualisation/5?smtNoRedir=1

● Augmented Reality: https://pt.slideshare.net/jishnupradeep/augmented-reality-33005861/6● History of VR: https://www.vrs.org.uk/virtual-reality/history.html ● What Is Extended Reality Technology? A Simple Explanation For Anyone:

https://www.forbes.com/sites/bernardmarr/2019/08/12/what-is-extended-reality-technology-a-simple-explanation-for-anyone/

47