augmented reality and virtual reality: research advances in creative industry

Researcher: Zi Siang See

Augmented Reality and Virtual Reality:

Research Advances in Creative Industries

Zi Siang See

Research: [email protected]

Industrial collaboration: [email protected]

Research Workshop Seminar

Faculty of Arts, Sunway University

Research Workshop Seminar: Zi Siang See

Centre for Research Creation in Digital Media

22 June 2016 – University of Reading

Research &

collaboration


Workshop

1) Introduction

2) Demonstration

3) Work-in-progress

4) Discussion

Research &

collaboration


Professional Background:

Industrial involvements since 2003/04

Researcher: Zi Siang SeeVirtual Reality 360 (VR360) | Presenter: Zi Siang See

Research Interest

• Virtual Reality 360 (VR360)

• Augmented Reality

• Interactive Multimedia

• Human Computer Interaction

• Computational Photography

• Imography Research

Research and Specialization


Work: McDonald’s Malaysia

Creative: NagaDDB

Operation: Quality Ingredient Story

Creative Imaging: Reina Imaging

Researcher: Zi Siang SeeDigital Imaging (DI) image manipulation for advertising usage, high technical finishing | Presenter: Zi Siang See

Researcher: Zi Siang SeeDigital Imaging (D() for Advertising Visuals| Presenter: Zi Siang See

Researcher: Zi Siang SeeAdvertising visual | Presenter: Zi Siang See

Researcher: Zi Siang SeeDigital Imaging (DI) for Advertising Visuals| Presenter: Zi Siang See

Researcher: Zi Siang SeeObject Virtual Reality (Object VR) | Presenter: Zi Siang See


Discover MINIEnter at your own risk of losing control


www.vsmm2016.org


“Creativity is not an option, it’s an absolute necessity”

Sir Ken Robinson 2014


VISIONThe Centre seeks to be a cornerstone for transformative digital media research, scholarship and innovation to advance Sunway University onto the global new media-scape.

MISSIONTo facilitate synergy between the arts and sciences, cultivating an innovative media research-creation environment.

To create significant collaborations with key local and international centresand labs to catalyze new partnerships.

To motivate a new generation of students and researchers in the domain of creative digital media (CDM) merging core attributes of the arts, sciences, humanities, media, and technology.

Vision & Mission


Centre for Research-Creation in Digital Media


UC Berkeley

Collaboration


The Project

Founding Projects


Figure: VSMM2016 also features the system by Prof Sarah Kenderdine


Introduction

Multimedia (Combination of Elements)

• Text

• Image

• Sound

• Video

• Animation

Industries

• Creative and advertising

• Corporate communications visuals

• Architectural imaging

• Cultural heritage preservation

• Scientific achieve


Multimedia (Combination of Elements)

• Text

• Image

• Sound

• Video

• Animation

• Touch

• Scent

• Taste

• Sight

Introduction

Trend in Industries

• Mixed and Augmented Reality

• Human Computer Interaction

Industries

• Creative and advertising

• Corporate communications visuals

• Architectural imaging

• Cultural heritage preservation

• Scientific achieve


Figure: a vision about 2000 imagined in 1910, through 24 illustrations by Villemard.


Figure: Sensorama (Morton L Heiling, 1962)


Major changes in computing platforms

Desk /Table Pocket Palm /Hand Head /Eye Ubiquitous


Figure: augmented reality, key examples from search engine

Mixed, Augmented & Virtual Reality


Virtual Reality

Figure: NASA, 1988 Virtual Environment System


Development

Figure: Super cockpit project, Tom Furness.


Near-eye-display and fully augmented interactive display experience

How far can we go?


Augmented Reality

Figure: augmented reality experience via google glass project


Augmented Reality

• Simulation of interface at www.glasssim.com

• Exploration in modern wearable interface (fashionable, UX practical)

http://www.glasssim.com/


Google Glass

• Google Glass, developer version, 2013

• Can be configured on user’s customized or personal spectacles.


Recon Jet

• Available 2014?

• Sporty design

• Cost “friendly”

• Android compatible

• Smartphone connectivity


Figure: potential areas, google glass project


Augmented Reality

• Augmented reality (AR) permits computer aided visualization that works with

human interaction (Billinghurst, 2012).

• Billinghurst, M. (2011). The Future of Augmented Reality in Our Everyday Life.

• Damian, I., Kistler, F., Obaid, M., Bueling, R., Billinghurst, M. and Andre, E.

(2013) Motion Capturing Empowered Interaction with a Virtual Agent in an

Augmented Reality Environment.

Figure: Human Interface Technology Laboratory (HITLab), University of Canterbury, NZ.


Mixed Reality

• Human Pacman: a mobile wide-area entertainment system based on physical,

social, and ubiquitous computing. (Cheok et al., 2004)

• Digital Taste Interface project, Electronic taste stimulation (Ranasinghe et al.,

2011)

• Mediating intimacy in long-distance relationships using kiss messaging

(Saadatian et al., 2013)

Figure: Mixed Reality Lab, the future of internet.


Introduction: Mixed & Augmented Reality

• Displays, tracers, and AR systems in general need to become more accurate,

lighter, cheaper, and less power consuming (Azuma, 2001).

• Augmented reality (AR) is related to the concept of virtual reality (VR); VR

attempts to create an artificial world that a person can experience and explore

interactively, predominantly through his or her sense of vision, but also via

audio, tactile, and other forms of feedback (Höllerer and Feiner, 2004)

Figure: wearable computing, Steve Mann.


Possibilities

• Human Pacman: a mobbile wide-area entertainment system based on physical,

social, and ubquitos computing. (Cheok et al., 2004)

• Digital Taste Interface project, Digital taste and smell communication

(Ranasinghe et al., 2011)

• Digital Smell and Taste Communication, City University London.


ExperimentAugmented reality medical laboratory

(example)

http://code.wikitude.com/medicalmurmurs

Hearable AR


Augmented Reality: Various Approaches

• Various marker-based or markerless AR tracking approaches has

been discussed (Billinghurst, 2012; Cheok et al., 2004; Lieberknecht

et al., 2009)

• Image-based AR tracking (Lieberknecht et al., 2009)

• CAD /3D object feature tracking (Metaio, 2014)

• Face tracking (Metaio, 2014)


Augmented Reality using High Fidelity Spherical Panorama with HDRI(Authoring process for AR environment recognition)

Spherical Panorama facilitated with HDRI(Cubic projection)

Cropped /selective cubic facadeworking with image-based AR

Zi Siang See, Mark Billinghurst, Adrian David Cheok (2015) Augmented Reality using High Fidelity Spherical Panorama with HDRI. SIGGRAPH Asia 2015 Symposium On Mobile Graphics And Interactive Applications.http://dx.doi.org/10.1145/2818427.2818445http://www.zisiangsee.com/research/zisiangsee_research_p009.pdf

http://dx.doi.org/10.1145/2818427.2818445

http://www.zisiangsee.com/research/zisiangsee_research_p009.pdf


AR for Mobile and Wearable

Figure: handheld experience of markerless AR.



Figure: handheld experience of markerless AR (video).



Figure: stereo image of AR – adaptable for HMD such as Oculus or Google Cardboard.


Google Cardboard

• Carl Zeiss just announced VR One, a virtual reality headset for use with a

smartphone. It is a viewer designed to work with phones between 4.7 and 5.2

inches (Zeiss, 2014).


Google Cardboard (v1)

• Carl Zeiss just announced VR One, a virtual reality headset for use with a

smartphone. It is a viewer designed to work with phones between 4.7 and 5.2

inches (Zeiss, 2014).

Programmable

NFC Tag

Low cost resin

/plastic lens

Mobile device as

computing module

Endless configurable size

and shapes (and materials)

Magnetometer


Google Cardboard (v2)

more reading at http://www.gizmag.com/google-cardboard-2-review-initial/37777/

Presses the

screen

Figure: google cardboard v2 supports larger phone.


What do I see?


Samsung Gear VR

• Samsung has launched Gear VR in October 2014, co-developed with Oculus.

Gear VR can house specific mobile device (Note 4) for providing virtual reality

and augmented user experience (Samsung, 2014)

• Control interface included



Samsung Gear VR

• VR /AR for social media

• UX prototyping


Figure: able to produce 360 images or videos, for HMD and other interface usage.

Gear 360


Oculus RiftFigure: Oculus, acquired by Facebook


Android VRFigure: Android VR collaborating with other developers

Google has announced eight hardware partners that will make Daydream-ready phones, including Samsung, HTC, LG, Xiaomi, Huawei, ZTE, Asus, and Alcatel.


MetaioFigure: Metaio, acquired by Apple

Metaio was one of the most technologically-ambitious AR organization, acquired by Apple in 2015


Media Companies’ Interest

Corporate media investment into AR/VR statup - infographic.



User Experience


Figure: European researchers used 3D-modeling and Instagram to recreate Mosul Museum destroyed in civil war.


User Experience


User Experience (Concept)

• Concept video produced by Infinity AR, 2014.

• Interface design, UX, daily applications.

Hyper-reality by Keiichi Matsuda


Mixed, Virtual, Augmented Reality

• Ikea AR



• McDonald’s McMission AR



• Penguins NAVI



• Octagon 4D (Animal series)



• Octagon 4D (Octaland series)



• Concept video produced by Infinity AR, 2014.

• Interface design, UX, daily applications.


Archiving our world in 3D

• LIDAR 3D Scanning

• Characteristics: trained users, costly, precision CAD content.



• Quiver

• Physical involvement during creation process

• Visual prototyping /conceptualizing


Gartner’s Hype Cycle


Definitions

Virtual Reality

• Virtual Reality (VR) which can be referred to as immersive

multimedia or computer-simulated life, replicates an environment

that simulates physical presence in places in the real world or

imagined worlds and lets the user interact in that world. Virtual

reality artificially creates sensory experiences, which can even

include sight, hearing, touch, and smell.

Augmented Reality

• Augmented Reality (AR) mixes a live real-world view with virtual

interactive content on a mobile or wearable device. One of the key

enablers for this is tracking technology, such as computer vision

techniques for tracking off pre-defined markers or markerless

images.


VR AR

Source and reinterpreted from http://www.augment.com/blog/which-headset-is-right-for-you/


Research Possibility

Explore into

• Method and apparatus for augmented reality

• Method and apparatus for virtual reality

• Method and apparatus for mixed reality

• Ubiquitous computing

• Obstacles and issues

• Technology management

• Education technology

• Engineering and concept design

• Collaboration /Multi-disciplinary

• User interface (UI) design

• User experience (UX)


Everything is achievable

through technology


Research Collaboration & Publications

• Lee Xia Sheng, Chung Pui Yan and Zi Siang See (2016) Irregular Shaped Building Design Optimization with Building Information Modeling.Proceeding in 4th International Building Control Conference 2016, Universiti Malaya. [indexed in SCOPUS]

• Zi Siang See, Mark Billinghurst, Adrian David Cheok (2015) Augmented Reality Using High Fidelity Spherical Panorama with HDRI. SIGGRAPH Asia2015 Symposium On Mobile Graphics And Interactive Applications, Kobe, Japan.

• Zi Siang See, Mark Billinghurst, Adrian David Cheok (2015) Augmented Reality Using High Fidelity Spherical Panorama with HDRI – Demonstration.SIGGRAPH Asia 2015 Symposium On Mobile Graphics And Interactive Applications, Kobe, Japan.

• Zi Siang See, Adrian David Cheok. (2015) Virtual Reality 360 Interactive Panorama Reproduction Obstacle and Issue. Virtual Reality, Volume 19, Issue2, pp 71-81, Springer. [indexed by ISI, SCOPUS]

• Lee Xia Sheng, Lee Ching Hua, Cheah Shyn Yi and Zi Siang See (2014) Architectural Transformation of a Night Market Stall. International Conferenceon Innovation and Technology for Sustainable Built Environment 2 2014, Universiti Teknologi MARA (UiTM) Perak.

• Zi Siang See, Khairul Hazrin Hashim, Harold Thwaites, Lee Xia Sheng, Ooi Wooi Har. (2012). Retrieving Extended High Dynamic Range From DigitalNegative Image - An Experiment On Architectural Photo Imaging. World Academy of Science, Engineering and Technology, Vol 6(8) August 2012,pp. 1833-1841. [indexed by EBSCO]

• Zi Siang See, Lee Xia Sheng. (2011). Non-bias Architectural Image Archive using High Dynamic Range Approach. 2nd International Conference onResearch and Innovation in Information Systems 2011 (ICRIIS’11) [CD Proceedings], Kuala Lumpur, Malaysia, 23-24 November. [indexed by SCOPUS,IEEE Xplorer]

• Zi Siang See. (2011). Non-bias Scientific Imaging Archive for Urban Morphology Research. Exchanging Views on How to Sustain Urban and RuralEnvironments in Aspects of Technology, Social and Ecology in Green Innovation Research Group (GIRG) [CD-ROM], Universiti Teknologi Malaysia(UTM), Malaysia.


Thank you

Research Discussion

research collaboration

www.zisiangsee.com

http://www.zisiangsee.com/


Facebook: Zi Siang See

research collaboration

www.zisiangsee.com

http://www.zisiangsee.com/