—eee 27 development of hybrid mobile application oral... · development of hybrid mobile...
TRANSCRIPT
Development of Hybrid Mobile Application—EEE 27
Li Jiaqi Raffles Institution
Koh Rei Min AshleyVictoria Junior College
Prof. Perry Ping ShumNanyang Technological University
Introduction
What are hybrid mobile applications?Hybrid mobile applications
1. Not exclusive to any operating server.2. Built with a combination of web
technologies like Hypertext Markup Language (HTML), Cascading Style Sheets (CSS), and JavaScript as well as native applications.
3. Utilises a WebView of a mobile platform enabling access to the device’s capabilities and facilities such as the accelerometer, camera, contacts, and more.
Native mobile applications
1. Built to work on a specific mobile operating system with the necessary tools for developers to use to create an app.
E.g. Apple IOS app has to be written in Xcode using Swift or Objective C languages and it only works on Apple products
Advantages of hybrid mobile applications
1. For developers, the origination cost is of hybrid mobile apps is lower and initial speed is faster.
What is Augmented Reality(AR)?
Augmented reality (AR) is a technology that superimposes a computer-generated image on a user's view of the real world, thus providing a composite view. Less than 25% of the AR world is consisted of digital images.
Applications of AR—Education
Blippbuilder — AR learning platform
Applications of AR— Entertainment
Popular AR game - Pokemon Go Movie scenes showing AR technology in the future
Applications of AR— Medication
Use AR in surgery
Therefore, ...while a considerable amount of resources and effort has been put into research of augmented reality and its application in various domains, such as aforementioned entertainment, education and medication,
the shopping environment has yet to be well explored. Thus this gap in the current market spurred us to explore in this direction.
Overview of our research
This research will explore the potential use and development of MAR in a shopping environment by developing a hybrid AR shopping assistant mobile application — ARShop.
Aims and Objectives
“ARShop” mobile application
Function 1: providing additional computer-generated perceptual information that is difficult to be presented on normal 2D packaging, such as videos, audios, to shoppers.
Function 2: enabling sellers to put various animated tags on commodities that can attract shoppers attention when they scan through shelves.
Built with Placenote SDKBuilt with Unity-Arkit-Plugin
Methodology
UNITY:
DEVELOPMENT SOFTWARE USED
ARKIT 2.0: PLACENOTE:- Cross-platform game engine- Can be used to create both 3D and 2D
games as well as simulations for its many platforms.
- easily accessible- Most widely used AR and VR
development platform- Has a highly optimized rendering
pipeline and rapid iteration capabilities.
- Combines device motion tracking, camera scene capture, advanced scene processing, and display conveniences to simplify the task of building an AR experience.
- Offers full 2D image tracking
- Allows for quickly build cloud-based AR apps that permanently save AR content in physical locations.
- Lets users scan any space and turn it into a smart canvas for positioning AR objects.
- Integrates with ARKit on iOS devices
PROVIDING ADDITIONAL PERCEPTUAL INFORMATION
FUNCTION 1 — “Scanning”
BUILDING PROCESS
For 3D animation, build animations in Unity with animation player function
Duplicate plane of image target. Insert trailer/animation onto
the duplicated plane. This will be called upon when the
image target is captured on camera
Import reference image (Covers of CDs or Games) This will be the image target.
Create new scene in Unity
End Result
Trailers of songs and movies will be played when the camera in the app recognises the image target.
3D animation of how the board game is played will show up when the camera in the app recognises the image target (board game packaging)
Video of trailer of the album
3D animation demonstrating the gaming process
1) ALLOW SELLERS TO SCAN SHELVES AND ADD TAGS
2) ALLOWS CUSTOMERS TO LOAD MAPS WITH TAGS ADDED TO COMMODITIES
FUNCTION 2— “Trial”
INTRODUCTION TO PLACENOTEPlacenote allows users to scan any space and turn it into a smart canvas for positioning AR objects.
space
AR objects
Shelves within a certain space in supermarkets or stores
Animated tags to attract consumers’ attention
DEVELOPMENT OF INTERFACE FOR SELLERS
Start a new map button for sellers
Feature points
1) Sellers start a new map
FEATURE POINTS
● Embedded function of Placenote SDK.
● Created by scanning through the space.
● Allow the space to be relocated when the map is reloaded by API key.
When a hit test was activated on
the screen, a new tag would be
created at the nearest plane
detected and a new info list of this
tag would also be created.
Position Rotation type
2) Adding tags to commodities
● 5 types of tags sellers can choose to put on commodities.
● Users can click on the bottom at the button of the screen to switch type of tags added.
TAGS
E.g. “Recommend” Tag
E.g. “Discount” Tag
● Built with embedded function of Placenote SDK.
● Sellers can assign a name for the map to help consumers find the correct map when they reload it.
3) Save Map with Name
End Result—Sellers
DEVELOPMENT OF INTERFACE FOR CONSUMERS
➔ We made reference to the scripts in the example. These included loading a map list from API key online and loading
the chosen map by On Load Map Clicked function.
ENABLING CUSTOMERS TO VIEW MAPS
End Result
Animated tags attached to board games with sparkles emerging from them to attract consumers’ attention
➔ We created a Home page scene in Unity which appears first when consumers launch the app
INTEGRATING THE 2 FUNCTIONS
Subsequently, we built this Unity project in Xcode and Android Studio, enabling the app to be installed onto the selected
iOS and Android devices.
Function 1
Function 2
Switch between functions
End Result—Consumers
Results1) AR hybrid app as a means to help consumers
make a better decision
2) AR hybrid app as a means to achieve more
enjoyable shopping experience
Conclusion1) App facilitates information transfer and provides
customers with an interesting shopping experience
2) Other potential developments: AR trying on of
clothes
Vector Icons by Matthew Skiles
Wei Zhu (2006, November). Recent advances in augmented reality. Retrieved on December 25,
2018, from https://apps.dtic.mil/dtic/tr/fulltext/u2/a606245.pdf
Bristowe, J. (2015, March 2519). What is a Hybrid Mobile App?. Retrieved September 1, 2018, from
https://developer.telerik.com/featured/what-is-a-hybrid-mobile-app/
REFERENCES
Connolly, P., Chambers, C., Eagleson, E., Matthews, D., & Rogers, T. (2010, October).
Augmented reality effectiveness in advertising. In proc. 65th Midyear Conference on Engineering
Design Graphics Division of ASEE.
Acknowledgement
We would like to take this opportunity to thank our mentor, Professor Shum Ping, for his constant support throughout the whole of this project. We would also like to thank our teacher-in-charge Dr. Lena Lui Wai Yi and Mr Chan Soo Heong Julius for their guidance and support.
Thank YouAny questions?