Johan Mattsson

Post Desktop user-interfaces

iWand, evaluation, iStuff,

iLounge, iROS & USE

iWand-explicit augmentation of users behaviour

• Selective augmentation of gestures as a way of interaction and introducing a peripheral knowledge of others actions

• Laser pointing device with a wireless connection to the room network, a iWand

• iWand-targets that reacts when a iWand-beam sweeps over them and sends a message back to the iWand using IR

• The received message is used as a key to services available in the room

• This magic wand, uses a laser-beam to trace targets of a spell, it could be any object “magic item” associated with an attached and configured iWand-target

Related work

• UbiControl, uses a laser-pointer attached to a 802.11 enabled PDA to interact with custom devices– Modulated id or the IP of the pointer where modulated on the

laser beam– Solar panel targets connected to a server and associated with

custom devices receives the IP– The server opens a communication to the PDA– The server transmits a UI for the associated custom device to

the PDA

• Video tracked laser-pointing interaction have been implemented and evaluated in a number of settings

Implementation

• iWand targets– Freestanding active targets

activated by the modulated beam

– Beaconing an IR id, e.g. an URL/IP while pointed on

• Enables non-pre-configuration of the iWands

• Location configuration of needed local targets

– Extended IR range and angels

• iWand– Modulated laser

distinguishable from 0 or 50/60 Hz daylight and lamps

– IR receiver– Wireless network /

communicationCurrently based on an IPAQ

with 802.11-lan and an attached laser-pointer. Lunches the received beaconed URL in a browser when targets are hit

Usage concepts, spells

• Service Launcher Spell – Runs an service related to the targeted item e.g. a iCrafter or

any other remote control application • Information Spell

– Provides information on the targeted item, e.g. help or museum guiding systems

• Diapering Computers -Link Spell– Links targeted devices, e.g. select a targeted input device for a

particular screen, could be a selection tool for point-write or multi-browse

• Interaction Spell– Interact using the iWand by capturing specific gestures

My expectation on interactive post-desktop user interfaces

• Support for many users to interact simultaneously and switch among tasks and resources, e.g. enhance both public and private work

• Support for creative computer aided user-collaboration rather than machine-machine communication

• Intuitive interaction tools to share computer resources and room equipment in a easily overviewed manner

Evaluation ideas for iWand

Elaborating my post-desktop expectations turned out as this three dimensions for comparing of collaborative interaction techniques

• Intuitive interaction tools to share computer resources and room equipment in a easily overviewed manner as Collaborative performance

• Support for many users to interact simultaneously and switch among tasks and resources as Transparency between personal and shared interaction

• Support for creative computer aided user-collaboration rather than machine-machine communication as Peripheral awareness and interaction

Collaborative performance

• The users involvement

• The groups common performance

• Turn taking or the ease of switching chair-user

How well a group of users collaborates using specific artifacts in terms of time and accuracy

Transparency between personal and shared interaction

• Fluent target switch-over

• Easy task recovery

• Target knowledge

How to easily perform interaction with several devices/targets in the means of flow and minimal confusion

Peripheral awareness and interaction

• Intrusions of personal work by surrounding interactions

• Knowledge of why and by who interaction or feedback are initiated

• Ability to interfere or give an opinion on co-workers showing a intention on performing an discussable tasks

Important issues while in regular face-to-face setting, how are they supported and ported into a interactive co-located collaborative workspace, also considered as the inter-personal or social protocol

Interactive Fizz Buzz

• Highly collaborative game• Simple well-defined

comparable goal/task• Personal/shared switch-

over by letting participants take notes on their sessions

• Peripheral awareness and interference by proposing users to interrupt when predicting a fail choice

Data analysis

• Comparisons of the results for interactive fizz buzz setups with the iWand, traditional desktop-interaction and other iStuff– Collaborative performance maps to the number of

correct target-selections over a specified time– Interaction transparency maps to how well the notes

are made in proportion to the collaborative performance

– Peripheral awareness could be mapped to how many of the proposed interferences that are taken into account

Future iWand directions

Beyond the touch screen a smoother and more slimed physical design could be made

• Accelerometers could provide a more precise notion of gestures as a main input

• An embedded RFID reader could made a nearby tagged display to share their display to the iWand, e.g. physical multi-browse selection

• The RFID reader could also be used to make the iWand a tool for linking a close mouse to a big shared display, configuring point-write

Ubiquitous Service Environments

Questions considering a more dynamic service oriented view on the environment:

• What happens if a group of users brings some personal & sharable services ?

• How does services adapt to new groups and environment constraints ?

• How do we model the users, environments services and devices ?

USE-iROS: supporting adhoc configurations of iroom groups anywhere

Sensors and implicit interaction patterns

• Applying sensors & sensor data into iLounge

• Modelling pattern of events & sensor data, e.g. as a extension of the patch-panel

• Interactive events deduced from context models

How to implement and apply such services in iLounge and partially as a dynamic service environment