jass `04 benjamin fingerle, christian wachinger1 2nd joined advanced student school ubiquitous...
TRANSCRIPT
JASS `04JASS `04
Benjamin Fingerle, Christian Wachinger 1
2nd Joined Advanced Student School
Ubiquitous Tracking based
AR-SetupsBenjamin Fingerle
Christian Wachinger
JASS `04JASS `04
Benjamin Fingerle, Christian Wachinger 2
Procedure
• We will present a scenario in which a user - Gerhard - gets into the benefits of an AR-enabled Environment
• First the scenario will be presented from the user’s perspective
• In a 2nd stage we’ll look behind the surface and see how the AR-environment can be modeled using the DWARF Service concept
• Finally we’ll go even deeper into detail and present how spatial relationships of objects can be represented using the Ubiquitous Tracking Framework
JASS `04JASS `04
Benjamin Fingerle, Christian Wachinger 3
The User’s Equipment
• Gerhard wears an Optical See Through Head Mounted Display (OST-HMD)
• Up on this HMD two 6DOF markers - one optical, one magnetic - are rigidly mounted
• Additionally a stereo-vision camera is installed on top of his head
• Gerhard wears special gloves with attached optical 6DOF markers
JASS `04JASS `04
Benjamin Fingerle, Christian Wachinger 4
A Day in Gerhard’s Life …
• Gerhard strolls down the TUM hallway greeting colleagues of his through closed doors
• Eventually he reaches his own office and steps in• Sitting down in front of the desk he reads a virtual
message of his Russian friend Vladimir from St. Petersburg asking for a chess game
• He starts the RemoteChess application whereupon a virtual chess board appears aligned on the desk and the virtual counterpart of Vladimir takes a seat across from him
JASS `04JASS `04
Benjamin Fingerle, Christian Wachinger 5
… A Day in Gerhard’s Life
• Suddenly an emergency request for instant help by his befriended Spanish colleague José shows up
• José asks Gerhard - a renowned surgeon - for advises regarding a difficult surgery José is currently conducting
• Just where a minute ago the chess board was visible a 3D - Model showing the patient crops up
• Gerhard studies the patient while having a look from different sides and getting computer tomography images registered on the patient on demand
• In vivid discussion with José a life is saved
JASS `04JASS `04
Benjamin Fingerle, Christian Wachinger 6
The Scenario Can Be Modeled With Services
• Services– Have needs
• Refined by predicates
– Offer abilities• Refined by attributes
• Connectors– Offer interfaces for data
exchange
• Service Managers– One for each network
node– Detect mutually
satisfying services– Provide services with
connectors
<service name=“OpticalTracker”><need name=“video” type=“VideoStream”>
<connector protocol=“sharedMemory”></need>
<need name=“marker” type=“MarkerData”><connector
protocol=“ObjectReference”></need>
<ability name=“markerPose” type=“PoseData”><attribute name=“location”
value=“$(marker.location)”><attribute name=“identity”
value=“$(marker.identity)”><connector
protocol=“NotificationPush”></ability>
</service>
Context = {location, identity, activity, time} can be modeled using Predicates and Attributes
JASS `04JASS `04
Benjamin Fingerle, Christian Wachinger 7
Services are Specified Using XML
<service name=“HMDOpticalTracker”><need name=“
...</need>
<ability name=“HeadPose” type=“PoseData”><attribute name=“location”
value=“$(landmark.location)”><attribute name=“identity”
value=“Gerhard”><connector protocol=“NotificationPush”>
</ability>
<ability name=“markerPose” type=“PoseData”>...
</ability></service>
JASS `04JASS `04
Benjamin Fingerle, Christian Wachinger 8
Hallway Services
Service Name Needs Abilities
GerhardConfigData -- landmarkDescription
HallwayConfigData -- landmarkDescription
HMDCamera -- videostream
HMDOpticalTracker videostream markerPose, HMDPose
HMDVideoShow videostream --
WhatsBehind pose videostream
ContextEstimator landmark context
JASS `04JASS `04
Benjamin Fingerle, Christian Wachinger 9
Office Services
Service Name Needs Abilities
GerhardConfigData -- landmarkDescription
HMDCamera -- videostream
HMDOpticalTracker videostream markerPose, HMDPose
HMDVideoShow videostream --
ContextEstimator landmark context
RoomConfigData -- landmarkDescription
RoomCamera -- videostream
roomMagneticTracker landmark markerPose
roomOpticalTracker videostream markerPose
Table 3D-content --
VirtualChess chessPartner 3D-content
VirtualSurgery handPose 3D-content
VirtualCommunication communicationPartner 3D-content
JASS `04JASS `04
Benjamin Fingerle, Christian Wachinger 10
Mutually Satisfying Services are Found by ServiceManager
• For each network node (e.g. Room, Hall, …) one ServiceManager exists
• This ServiceManager observes needs and abilities of those services belonging to his particular network node
• When a match is found the ServiceManager provides all involved parties with connector-objects
• These connectors offer push-, pull-, or shared memory access to abilities via various interfaces
JASS `04JASS `04
Benjamin Fingerle, Christian Wachinger 11
Matching of Mutually Satisfying Services
JASS `04JASS `04
Benjamin Fingerle, Christian Wachinger 12
Ubiquitous Tracking - The Formal Layer
• The crucial problem of Augmented Reality applications is the correct tracking of objects
• It is common to integrate the tracking procedure into the application
• Different tracking technologies are combined to get better results
• To release the AR application of tracking and to enable a seamless integration of new tracking devices a formal layer is introduced
• The formal framework, called UbiquitousTracking, forms the formal layer
JASS `04JASS `04
Benjamin Fingerle, Christian Wachinger 13
Ubiquitous Tracking – The Formal Framework
• Requests to the framework about the spatial relationship of objects can be send
• The answer delivers the optimal relationship available
Graph-model:• nodes == objects• edges == spatial relationships
JASS `04JASS `04
Benjamin Fingerle, Christian Wachinger 14
Ubiquitous Tracking - The Underlying Graph Model
• Properties of a spatial relationship– Represents the transformation and translation of the source
coordinate system to the target coordinate system– Attributes characterizing the quality of the relationship
• Three different types of graphs– Real relationship graph– Measured relationship graph– Inferred Relationship
JASS `04JASS `04
Benjamin Fingerle, Christian Wachinger 15
Real Relationship Graph
Each pair of objects has at every point of time a geometric relationship
JASS `04JASS `04
Benjamin Fingerle, Christian Wachinger 16
Measured Relationship Graph
• Estimates of relationships are just available for certain objects for discrete points of time
• Additionally Attributes characterizing the quality of the measurement exist
• Error function describing
the quality of a relationship
JASS `04JASS `04
Benjamin Fingerle, Christian Wachinger 17
Inferred Relationship Graph
• Knowledge about spatial relationships not just for discrete points of time is necessary
• Knowledge has to be inferred about the relationship of objects• Error functions help to find optimal inferences
JASS `04JASS `04
Benjamin Fingerle, Christian Wachinger 19
What Is a Good Error Function?
Aim: Finding optimal paths in the graph!
= { Latency,
Update frequency
Confidence value
Pose accuray
Time to live }
JASS `04JASS `04
Benjamin Fingerle, Christian Wachinger 20
Optimisation of Graph Algorithms
• Precomputing the Data Flow Graphs– Infrequently changing structure of the spatial relationships– Infrequently changing Attributes– No dependency on the pose measurements
• Grouping Nodes– Several nodes can be represented by a single supernode– Enabling level of detail hierarchies Faster graph search
What is the coordinate system of the new supernode?
JASS `04JASS `04
Benjamin Fingerle, Christian Wachinger 21
Security & Safety Aspects
• What if the “WhatsBehind” Service can be used in front of every door?
• What if your boss knows what you have done this weekend?
• What if other users know about every place where you have been this week?
• What if Indra the intern uses the AR setup of Gerhard?• What if Eve corrupts the data showing you the false
distance to the oncoming bus?• What if you can’t see the car because the latest sports
news occluded it?
Privacy
Restrictions
Authenticity
Privacy
Restrictions
Authenticity
JASS `04JASS `04
Benjamin Fingerle, Christian Wachinger 22
Finding Matches Forms Crucial Performance Issue
Matching problem of high computational complexity• One Service Manager for each Network Node• Number of potential matchings grows exponentially with
the number of services available inhibiting scalability
Different strategies for coping with this issue could include• Heuristics, based on context information, realized as
graph searches on the spatial relationship graph• Interpretation of matching problem as predicate logic
formula, applying specialized algorithms known from model checking
• …
JASS `04JASS `04
Benjamin Fingerle, Christian Wachinger 23
Issues Concerning Graph Representation
Possible ways of storing the spatial relationship graph include
• One graph-service holding the complete Graph
+ all graph algorithms applicable- contradicts the distributed computing paradigm
• Each Service knows its adjacency
+ complies with distributed computing paradigm
- not all graph algorithms applicable
JASS `04JASS `04
Benjamin Fingerle, Christian Wachinger 24
Issues Concerning Access to the Graph Information
Possible ways of accessing spatial relationships include• Request formulated as need for spatialRelationship with predicates
source and target, abilities offered by – Certain GraphInformationServices
+ Graph can be split up between different such services
- different graph search strategies for the same relation cannot coexist
– One service for each relationship instantiated by special relationShipGenerators
+ relationships instantly available
- different graph search strategies for the same relation cannot coexist
- number of services raises dramatically
• Requesting a graphInformerObject providing an interface getSpatialRelationShip(source, target)+ Objects implementing different graph searches can coexist
- centralization
JASS `04JASS `04
Benjamin Fingerle, Christian Wachinger 25
Open Questions
• How much information about the world do we have to put in the error function?
• Which graph algorithms have to be applicable?• Which graph representation allows theses algorithms?• How to perform context changes?• How to enable new users to enter AR environments?• What has then to be calibrated?
JASS `04JASS `04
Benjamin Fingerle, Christian Wachinger 26
Critical success factors
• Number of AR ready buildings• Development and acceptance of standards• Prices and convenience• Legal issues• Products and applications urging the user to buy