tendencies in industrial ar
Post on 11-Jan-2017
232 Views
Preview:
TRANSCRIPT
Timo Engelke
AR Community Meeting
Munich, 09/13/14
Tendencies in Industrial AR
§ Tendencies § Web-Trends § ARVIDA Project § Streaming Services
§ Web Standards Tendencies and their use
§ WebRTC § WebComponents
Tutorial Outline
Related Technology Trends
3D Data & Documents
3D Printer
3D Scanner
GEO Daten
3D PLM
Web-based Engineering
Cloud based Rendering
Mixed Reality
Convergence of Application-
Platform
HTML5
Titanium
Peper NaCI
Windows8
W3C WebApps
Chrome OS PhoneGap
Divergence of System- Platform
Table
Android
SmartPhones
Desktop
Cloud-Bases
WindowsPhone
Mac OS
Amazon Kindel
Windows
iOS
Web-Service Architecture
Current State of 3D on the Web plugin-based extension to user agent
Client Flash 11 Silverlight 5 Java, Java3D, JOGL and JavaFX (SUN) O3D (Google): Javascript based scene-graph API X3D (ISO, web3d consortium): plugins with SAI interface MPEG-4 & MPEG-4 Part 11 (ISO, Moving Picture Experts Group)
Server ParaViewWeb, OnLive, Gaikai, …
General Issues:
=> Installation/Security => Bound to single Technology => Data/Event model is separated from HTML-page => plugin-specific scripting interface (e.g. SAI for X3D)
§ JavaScript API to OpenGL ES 2.0 § Khronos (not W3C) specification § First Version in 2010 § Solid support in 2014 § Chrome, Firefox, Opera, IE11 § Available on Desktop and Mobile
§ General Issues: § => Low level Graphics API § => Most application build on unstandardized middleware § => Most middleware solutions (e.g. three.js) build for entertainment
apps § => Security § => Application limited to W3C technology stack (e.g. data/image
loader)
WebGL the rising star
Scenario: Web-Application with interactive 3D Canvas
Declarative Scenegraph Part of HTML-document DOM Integration CSS/ Events
<canvas>
Imperative Procedural API Drawing context Flexible
2D (Final HTML5 spec)
3D (No W3C spec yet)
Declarative (X)3D in HTML Completes todays graphics technologies
Declarative (X)3D in HTML Embed a live scenegraph in the DOM
<!DOCTYPE html > <html >
<body> <h1>Hello X3DOM World</h1> <x3d xmlns=‘…’ profile=‘HTML’ > <scene> <shape> <box></box> </shape> </scene> </x3d> </body>
</html>
DOM / HTML Document Binary asset resources
Fraunhofer IGD / Autor / Abteilung
Explicit Binary Container - Directly loaded to TypedArrays - Data assignment in JS - Multiple arrays per file - Multiple files per scene
Images and Videos - Encodes int/float arrays (e.g.
coordinate, normal, texCoords, generic-attributes) in RGBA-images
- Multiple images per array - Multiple images per scene
Separate structure and data + priority controlled rendering
X3DOM: Makes “Declarative 3D” available OS Polyfill build on HTML5, JS and Plugin/WebGL/Flash
Fraunhofer IGD / Autor / Abteilung
Application examples www.x3dom.org
§ Platform/operation system independent development § Scalability § Server-based Rendering § Server-based Tracking § Hybrid Rendering (Culling on the server-site, Streaming of the
Culling Geometry) § Thin Clients & Interfaces to PDM-Environments § Exchangeability of Web-Services § Use of different rendering/tracking/simulation services § Dynamic Processing of Augmented Reality Resources
Possibilities of Web-Technologies in Augmented Reality
http://www.arvida.de/
13
ARVIDA Consortium
§ To establish an Augmented/Virtual Reality Reference architecture § that makes uses Web-technologies § supports exchangeability of technological components § to evaluate the performance of web-based Augmented Reality
§ Applications / Use Cases § the project is driven by industrial use cases § the requirements and KPIs for the evaluation are defiend by
industrial ene users § Applications in the field of (Automtive, Ship Building, Aviation, § Variance Analysis § Assistence Systems § Motion Analysis and ergonomics § Product Validation
Goals of ARVIDA
Efficient processing of large data sets
Real applications tend to be huge HTML-files Unpleasant non-interactive user experience Browser are not build to hold GByte of DOM attribute data (e.g. multiple data copies)
Reference external sub-trees X3D “Inline” node black-/white-box interface?
Binary XML decompression x3z: (ISO) Decoding on JS-Level x3db: (ISO) Fast Infoset: No UA or JS-lib EXI: (W3C) Even worse
Transcoder Service Async data preparation and caching infrastructure § RESTful service to translate CAD-data § Optimization for specific classes
(e.g. mobile GPU) § Asynchronous JOB-model returns target resource immediately § Http update: 200, 202 (with time estimation), 404
Transcoder Service Async data preparation and caching infrastructure
§ 2250 JT-Parts
§ 32 Threads
§ ~ 60 Sec
§ WebSocket notification § Efficient Cache infrastructure minimizes data processing § Most operations are massively parallel and only IO bound § Can use http/https or DB/PDM resource handler (e.g. TeamCenter)
Transcoder Service Example Siemens JT-Engine
Progressive Data Loading
§ Client-Based Rendering § Web-GL based Rendering
on the Client § e.g. X3DOM
§ Server-based Rendering § Bi-direction Streaming: § Streaming of the captured
camera image to the server § Re-streaming of the
rendered images to the client
§ Hybrid Approaches § Culling of the Geometry
on the server § Transfer of the culled
Geometry to the client
Client-Server Scalability
Server
Mobile Device 1
request
Touch-Events, Camera
Port
Display
Port Assignment
Start Application instance
Start Mobile Application
Mobile Application
running Application instance running
Visualization Service Smart combination of server/client based rendering
Pro/Con Latency
HW-Req. Net trafic Secuity Simplicty Scalabiltiy
client ++ -- + -- ++ --
hybrid ++ + ++ + + +
server -- ++ -- ++ -- ++
Performance Client / Data in DOM
Client / Data in BG/SIG
Hybrid Server
Million Tri. 0.2 15 150 500
Hybrid Rendering 80. Mil. Tri. http://www.youtube.com/watch?v=h0SUWqJfQsE
instant3DHub - WebVIS Secure, efficient and open CAD-Data visualization
CAxWebApp
PLMXMLViewer
IssueViewer
ColorCoder
DBPDMAPI
HTTP/HTTPSREST
TeamCenter
ProvidesResources
OpenJTCATIA (V4/V5)
ACISIGES
InventorPro/E
SolidWorksX3D/VRML
Collada
WebVISLibrary (JS)
PartView
CRUD
API
ToolBoxAPI
Instant Reality Plugin
OpenGL VGRRRT
X3DOM
WebGL IPPFlash
Remote Buffer Viewer
WebRTC WSMJPG
TranscoderTranslation and
Optimisation Cache Service
PIXIVisualisation Service
CAx Data
GB
Images
CAx Data
CAx DataOpen
GL VGRRRT
WebVIS Secure, efficient and open CAD-Data visualization
WebVis Resource Service (Win7 Box)
WebDesktop (HTML5 Browser + WebGL)
PLM andCAD-Daten
Transcoder
WebPartCache
(Apache)
3DWebVis3DWebVis
3D WebVis(IFrame)
InteraktiveProdukt-struktur
User Auth.
CUD CADPart
Part Picking
DocumentDocument
WebPart
CUD Part
R Part
Request CADPart+ User Auth.
Request WebPart+ User Auth
WebPart Data
CRUD Part Meta-Data
WebVis Architektur V03, JB, 2013-10-07
CADPart Data
WebVIS Efficient CAx visualization in WebApps
Single JavaScript (JS) library for WebApp developer and integrator Minimalistic JS-interface hides service infrastructure and communication
createVis ( URI, ID, { optional Data } ); URI: CAx-Resource (e.g https://foo/bar.jt ) or “local://group” Can transport authentication tokens updateVis ( ID, color/highLight/visibility, value ); Update the element color, hightLight or visibility property addCallback ( ID, onClick/touch/over, callBack-Function ); allows the application to react on element specific user input
removeVis ( ID ); Remove a specific element
Allows WebApps to visualize CAx-Date (almost) directly Hidden and automated client and server based caching infrastructure
WebVIS – PLMXML Frontend Efficient CAx visualization in WebApps
Example: Review WebApp “PQM Review/Web"
WebRTC = Web Real-Time Communication
AR and WebRTC – What is WebRTC?
WebRTC is an API definition drafted by the World Wide Web Consortium (W3C) that supports browser-to-browser applications for voice calling, video chat, and P2P file sharing without plugins.
§ 2010 Google acquired Global IP Solutions (GIPS) and therewith the basis for the technologies.
§ 2011: § W3C – Standardization Group § Internet Engineering Task Force - IETF § Release of free reference famework as source code
§ 2012: § First integrated in Chrome
§ 2013: § In Firefox and Opera
§ 2014: § Microsoft? – Maybe not?
WebRTC - History
§ Main Page: www.webrtc.org § io13webrtc.appspot.com – Nice intro video and slides § www.html5rocks.com/en/tutorials/webrtc/basics/ § http://dev.w3.org/2011/webrtc/editor/webrtc.html
Low level development: www.webrtc.org
WebRTC - Resources
Basic Platforms: § Chrome § Chrome for Android § Firefox § Opera § Native Bindings avaliable for:
§ Android (Java) § OSX/iOS (OBJ-C) § Linux § Windows
WebRTC – on different Platforms and Services
Services:
WebRTC - Technologies
3 Core - Components
Acquiring audio and video
MediaStream (aka getUserMedia)
Communicating audio and video RTCPeerConnection
Communicating arbitrary data RTCDataChannel
WebRTC – Typical Scenarios for Network based driven AR
§ Local Tracking and remote Assistance
Remote Expert Repository, Signaling
Client, Tracks
WebRTC
WebRTC – Typical Scenarios for Network based driven AR
§ Remote Tracking and remote Assistance
Remote Expert Repository, Signaling
Thin Client
WebRTC
Dedicated tracking
WebRTC – Typical Scenarios for Network based driven AR
§ Typical Scenarios § Remote Expert
§ Local Tracking and remote assistance § Remote Tracking and remote assistance
§ Remote Tracking § Send Image and get Pose back § Send Image and get Augmented Image Back
§ Recording of Sessions § Local Tracking and remote Rendering § Remote Rendering
WebRTC – Typical Scenarios for Network based driven AR
§ Hands on Demo § Early Approach using local tracking
and remote assistance
WebComponents - Future
§ Resource: webcomponents.org § As soon as available, might change the web
radically – including AR in the web
)
Example: Mobile VE-Applications UI elements to explore data-elements as part of the temp.
top related