cloud based augmented reality

Cloud Based Augmented RealityDAV CLOUD MOBILE SYSTEM ARCHITECTUREDAV CONTENT GUIDELINE FOR DEVELOPER

By Gia MuhammadWIR R&D

Outline Presentation

Problems Proposed Solution Overall design architecture between control center and devices Launcher for APK Update Image Tracking based Cloud News / Image Slide of Advertise based Cloud Content based Cloud Development Guideline Test Result Conclusion and Future Works

Problems

DAV Devices have been assembled at convenience store such as Alfamart, Alfamidi and Lawson. Each stores have different shelf categories so it’s have different contents category which has been decided by control center but the devices limited by some factors such as internet connection, memory, storage, category boundary etc. The category boundary is one of the most critical problem to limit products from AR scanner.

News or Image slide of advertise always changes in close time range. Updating APK is disturbing devices operation which not get news update.

Sometime the main application has changed that affected all contents in devices so the main application should be updated.

A little change could make greedy bandwidth (not efficient)

Proposed Solution

All depended file which will use must stored at cloud Files which will use are controlled by Server in case of different

shelf and store/convenience store There must be main application that handled cloud mechanism in

each devices, the main application also should have update so that must have launcher application that will update the main application.

Overall System Architecture

Launcher for APK Update

Launcher or “DAV Launcher” is a first executor application which will handling some task before a main DAV application launches.

There are several tasks which will execute in DAV Launcher Registering new device serial number to server Latest update check Download and installing the updated latest apk Store and Shelf assignment QR Code generator for QC DAV (Main Apps) crash backup or substitute

Launcher for APK UpdateWorkflow

The Launcher always try to launch DAV apps (loop) After DAV apps launch, DAV Launcher will standby

and wait until DAV apps exit or crash, if exit,Launcher will recall to the first state of procedure

Launcher for APK UpdateHow to Upload

Launcher for APK UpdateHow to Upload

Image Tracking based Cloud

Image Tracking is a media which will intermediate between real world and virtual environment (apps)

The Image tracking should be recognized before it deployed into the apps

image trackings are depending on many image of products but each products are stored in different place or shelf so not all image trackings are deployed in the same devices.

Consequently the image tracking should deploy dynamically, the image tracking had applied with cloud-based mechanism that have been handled by DAV Control Center.

Image Tracking based CloudWorkflow

Image Tracking based CloudHow to Upload

Image Tracking based CloudHow to Upload

News / Image Slide of Advertise based Cloud

When the main application launches, the application will show idle mode that there image shows a captured image from camera in real-time.

This idle mode used for showing some advertisement such as image slide or video.

These image slides and videos are deployed by store and shelf in other words called dynamic.

Consequently, these advertises also should run with cloud-based mechanism that have been handled by DAV Control Center.

OGV and MP3 as DataType

News / Image Slide of Advertise based CloudHow to Upload

Content based Cloud

Same as Image Tracking, content deployed into device and must be handled by DAV Control Center.

These contents are deployed by store and shelf in other words called dynamic.

Sometime many image tracking can load only one content. Unity3D format & dll as Data Type

News and Content CloudWorkflow

Content Workflow in Device

Image Recognized

Find Asset by Web Service

Info.

Load Asset by Scene

Name

Preload Content

From Cloud Storage

Asset Loaded into

Memory

Is Asset Available?

Load Scene and Run

Asset

DAV Asset Script Loader

DAV Asset Script LoaderWorkflow Asset

Loaded

Script (dll) Load into Memory

Checklist script will load into

asset

Find scene path at each

scripts

Load script into scene

by path info. one-by-one

Script run on each loaded

scriptsReady to

Play

Content Development GuideUnity

Always use mirror object because DAV device using front camera. Every material such as shader, mat, etc should be gathered in

one directory with format same as namespace, store in assets/resources/{content name} path directory and build in asset bundle (unity3d) as additional file

The content that built can’t attach script in object scene, it must develope separately and not recommended to place script into deep hierarchy.

Content Development GuideVisual C#

Every script in content must give a namespace with format [ContentName]_[Year&Month]. Ex : Pocari_201506

In middle of content scene always give times at least 10 second to prepare back to main/idle scene.

If any ‘PAUSE’ scenario, the content game supposes to run timer that will back to main/idle scene if the timer time out.

Main/idle scene located at index 1 so the content must be loaded to index 1 if the content want to back to idle/main scene (including exit scenario). Don‘t use scene name.

Content Development GuideCreate Main Script to define script will load

Content Development GuideDetermine Object Scene Path

Content Development GuideDetermine Object at Variable

Since object attached into script and the script loaded cloudly, object should re-defined

Common Development Cloud Development

Content Development GuideArray Case

Since object attached into script and the script loaded cloudly, object should re-defined

Common Development Cloud Development

Content Development GuideButton Function Case

Since object attached into script and the script loaded cloudly, object should re-defined

Common Development

Cloud Development

It’s recommended to put the script in the same file script as button function

Content Development GuideGetting Image Tracking Status

SimpleJSON library already compiled within DAV Script Loader DLL file

Content Development GuideRun a script when not in startup condition

WhereParameter 1 : NamespaceParameter 2 : Class NameParameter 3 : Object Scene Path

Content Development GuideLoad File from Additional Asset

Since object attached into script and the script loaded cloudly, object should re-defined Call DAV.Asset before load

particular file in Additional asset

Loaded from png as Texture2DLoaded from wav/mp3 as AudioClip

Loaded from prefab as GameObject

Content Development GuidePlugin Reference

LeanTween and Mobile Movie Texture have found the solution but No solution for Object Pool Plugin

Content Development GuideObject Pool Substitute

Content Development GuideLeanTween Utilization

No change, same as common development

Content Development GuideMobile Movie Texture (MMT) Utilization

MMT using dll that have been compiled and ready to use

Lets path empty, path would filled by script

Content Development GuideMobile Movie Texture (MMT) Utilization

Content ID, determined by DAV Control Center

Generated File Name, determined by DAV Control Center

Test Result

Build a script and update to control center Script Loader in DAV Cloud (Editor) Build Asset and Additional Asset

Conclusion and Future Works

Image Tracking / Augmented Reality SDK switch to EasyAR Image match function for Image that upload to DAV Control

Center as image tracking to avoid same image. If the part of data content updates, old file of data content won’t

be deleted. The apps should have sync procedure after content update.