an architecture to enable spontaneous mobile spatial interaction with pervasive services

13.05.07

RelateGateways

An Architecture to Enable SpontaneousMobile Spatial Interaction with Pervasive ServicesMaster Thesis, Dominique Guinard, Fribourg 2007Supervisors: Hans Gellersen, Denis Lalanne, Rolf Ingold.

RelateGateways: Dominique Guinard

2 / 35

Context

Lancaster, Northern UK:^ Pop: 30’000 ^ (100’000 when

including the sheeps).

Infolab 21:^ 250 IT researchers.^ Ubicomp Group.


3 / 35

Today’s Menu…

Introduction, Aim E2ESD Model

^Spatial Discovery^Video^Network and Service Discovery^ Invocation and Interoperability

• Cross-device Interactions

Evaluation Conclusion


4 / 35

Introduction: Bob’s Adventure

Bob, researcher in biology at Unifr talk at Lancs.

Bob needs to print his presentation.

Where is the printer?

?T - 30 minutes


5 / 35


Bob finds someone who knows someone who met someone who knows where the printer is located!

Hem, Bob feels less cool.

?T - 10 minutes


6 / 35


Bob now needs:^ To find the printer’s

drivers.^ Install the printer.^ Get the right to access it.^ Find out about the

printer’s properties and accepted formats.

^ Etc…^ … print the document!

Arg, Bob doesn’t feel cool anymore!T - 30 seconds


7 / 35

Introduction: Summary

Mobile users can benefit from access to pervasive services.

Network and service discovery technologies facilitate spontaneous connections.

However, these approaches are not user centric:^ Difficult for users to identify services;^ Lack of simple and natural interaction techniques:^ => Lack of spontaneity in interaction…

Introduction


8 / 35

Aim

Using the mobile device to:^ Identify services available in the user’s

immediate environment.^Consume the services in a natural and

standard manner.

Need to:^ Include users in the discovery process.^Address identification, discovery and

invocation.

Introduction


9 / 35

Today’s Menu…






10 / 35

End-to-End Service Discovery Model (E2ESD)

Spatial Discovery

Invocation & Interoperability

Network & Service Discovery

Interaction Life-cycle

Mobile Code

Layer 1 Layer 2 Layer 3Mobile User Pervasive Services

Mo

del

Imp

lem

en

tati

on

$

E2ESD Model


11 / 35

Layer 1: Spatial Discovery

Spatial Discovery




Mobile Code


Mo

del

Imp

lem

en

tati

on

$


12 / 35

Gateways User Interface

Widgets as access points to the services:^Gateways.

Users visually discover the services:^Mapping the user’s

view of his environment on the mobile desktop.

^UI as a compass.

Keyboard

Beamer and Display

Printer

Spatial Discovery


13 / 35

Implementation

Small Java Swing windows representing the service providers.

At the screen periphery, integrated to the desktop.

Two interaction modes:^ Drag-and-Drop.^ Click.

Spatial Discovery


14 / 35

Spatial Context

Spatial context initialy delivered to the mobile client by a Wizard of Oz interface.

Introducing Relate:^ EU-founded project.^ Ad-hoc sensor network.^ Providing relative

positioning. Achieved: first extend to

provide real-time positioning data to the compass UI.

Spatial Discovery


15 / 35

Spatial Context: Deployement

To provide the user interface with spatial context we need:^A USB Dongle/Brick on

the mobile device.^An autonomous Dot on

each service provider.

Spatial Discovery


16 / 35

Video Demonstration


17 / 35

Layer 2: Network & Service Discovery

Spatial Discovery




Mobile Code


Mo

del

Imp

lem

en

tati

on

$


18 / 35

Extending Relate: BeforeNetwork Discovery

USB DONGLE

USB DONGLE

USB DONGLEUSB Connection

USB Connection

USB Connection

RF + Ultrasound

USB / Ethernet


19 / 35

Extending Relate: After

DOT

DOT

DOT

DOTPublic

Display Server

Ethernet Connection

DOT

DOT

USB DONGLE

USB Connection

Position + IP to Service

Network Discovery


20 / 35

Layer 3: Invocation and Interoperability

Spatial Discovery




Mobile Code


Mo

del

Imp

lem

en

tati

on

$

Interoperability


21 / 35

Modeling the Services

A service is composed of:^ A ServiceProvider, enclosing the service logic.^ A ServiceRequester containing enough information to invoke the

service. 2 types of services:

^ Push service (can be invoked using a Universal Requester).^ Pull-and-Push service.

Interoperability


22 / 35

« Plug and Play » Invocation

3: Load Mobile Code1: Discover Service

using Relate

Mobile Code

2: Get Mobile Code

4: Invoke Service

The semantics of Pull-and-Push Services is unknown to the mobile client before discovery.

Packets of Mobile Code (ServiceRequesters, descriptions, icon, etc.) are downloaded and dynamically loaded on the mobile device.

Interoperability


23 / 35

Cross-Device Interactions

Using the computing power as a service.

Use-cases:^ Collaborative tasks^ Cross-device

interactions for single user.

Extending the EBL toolkit:^ Cooperation with UCL

(Université Catholique de Louvain la Neuve, HCI Lab).

Interoperability


24 / 35

Today’s Menu…






25 / 35

Settings

Test run in Lancaster, formative user study in Munich.

Total of 20 users. 3 “service enabled”

devices within a large office.

Keyboard

Tablet PC

Handheld PCMobile PhoneUser

OR

OR

OR

Beamer and Display

Printer

Wireless/Wired Router

Evaluation


26 / 35

Qualitative Results

Most cited benefits:1. No installation, no configuration: saves time.

2. Ease of interaction with the services: drag and drop.

3. Dynamic spatial arrangement of the gateways: making the UI more natural, especially useful in unknown places.

Suggested a number of UI improvements. And services to implement.

Evaluation


27 / 35

Today’s Menu…






28 / 35Bootstraping Spontaneous Mobile Spatial Interactions

Implementation of the E2ESD bundled into a single, runnable application offering:Â spatial user interface « toolkit » (MVC based

+ contextual rules-engine).Â Network and Service Discovery system.Ân Invocation and Interoperability system.Ân architecture for prototyping pervasive

services.^Various simulation and debugging tools.

A framework (SOA) supporting the rapid prototyping of mobile spatial interactions.

Conclusion


29 / 35

Quantitative Outputs

Dissemination^ 3 accepted workshop papers:

• MSI @ CHI 07, Permid @ Pervasive 07, SensorNet 07

^ 2 conference papers submited• Ubicomp 07 (Demo Paper), LoCA 07

Prototyping Framework:• ~16’000 lines of code.• ~160 classes (to consolidate!).• To few hours of sleep…

Conclusion


30 / 35

Open Questions

Scalability of the application:^ In terms of user interface.^ In terms of prototyping framework.

User study is formative:^Need for a comparative study as well.^Need to evaluate the framework and its use for

the prototyping of mobile spatial applications.

Security concerns:^Spontaneity is nice but it leaves a number of

doors open for attackers.

Conclusion


31 / 35

Questions ?

Thanks for your attention…. Project’s homepage:

http://ubicomp.lancs.ac.uk/relategateways Contact: [email protected]

Conclusion

an architecture to enable spontaneous mobile spatial interaction with pervasive services

Technology