towards user-defined cross-device interaction
TRANSCRIPT
Towards User-defined Cross-
Device InteractionAUDREY SANCTORUM & BEAT SIGNER
VRIJE UNIVERSITEIT BRUSSEL
0
1
2
3
4
5 or more
0% 5% 10% 15% 20% 25% 30%
How many connected devices do people use?
Japan Spain USA Switzerland BelgiumPercentage of users
Num
ber o
f dev
ices
The Connected Consumer Survey 2014/2015Google Inc.
65%
Conductor (2014)
HuddleLamp (2014)
Connichiwa (2015)
Panelrama (2014)
MultiSpace (2006)
ReticularSpaces (2012)
Existing Classifications
"The 4C Reference Model for Distributed User Interfaces" by Demeure et al.• computation, configuration, communication and coordination
"Distributed User Interfaces: State of the Art"by Niklas Elmqvist• input, output, platform, space and time
UI + data + UI elements
UI + data
Table / Camera Room Network connection
to the server Anywhere
HuddleLamp
MultiSpace
ReticularSpaces
Panelrama
Conductor
Connichiwa
Granularity of distribution
Location constraint
UI + data + UI elements
UI + data
Table / Camera Room Network connection
to the server Anywhere
HuddleLamp
Airlift
MultiSpace
ReticularSpaces
ARIS
GroupTogether
iLand
iRoom
Panelrama
Conductor
IMPROMPTU
THAW
Deep Shot
Weave
Connichiwa
XDStudio
WebSplitter
Melchior et al. (2009)
CAMELEON-RT
Granularity of distribution
Location constraint
Robertson et al. (1996)
Frosini et al. (2013)MultiMasher
Pick-and-Drop
End Users
How to allow end users to define customised cross-device interactions?
How much control will end users have in terms of the granularity of the UI components to be distributed?
Will end users be limited by a specific location, space or office setting?
Will end users be able to share their configuration of customised DUIs?
Can end users reuse parts of other configurations?
What will end users be able to modify?
…
<?xml version="1.0"encoding="UTF-8"?> <activeComponents><activeComponent> <identifier>Sound< /identifier> <logic>…</logic></activeComponent></activeComponents>
Active Component
Architecture
ActiveComponents
Resource/Selector Plug-ins
Developer RegistryCore
RSL
User Profile
End-User RegistryConfiguration
Pool
User Developer
Button 1
Scenarios
Swipe AC
Double Swipe AC
Triple Swipe AC
Direction AC
Swipe Area
TABLET
Sound AC
TV Runtime environment
Runtime environment
...Gesture AC
Play AC
Data Transfer AC
Data Transfer AC
Proposed Approach
[28] Signer and Norrie As We May Link: A General Metamodel for Hypermedia Systems.(2007)
Authoring Rather Than Programming
Swipe GestureDevice: Tablet
AreaDevice: Tablet
Data TransferSDevice:
TabletTDevice: TV
PlayDevice: TV
SoundDevice: TVVolume: 80%
_ X Components
Data Transfer +Play +Sound +Swipe +
Area +
Double Swipe +Triple Swipe +Direction
+... +
Conclusion•Classification of DUI systems•User-defined cross-device interactions• Linking UI components and application logic• RSL hypermedia metamodel• Arbitrary level of granularity • Sharing of user-defined interactions
Takeaway Message
Hypermedia based DUI development
Authoring rather than programming approach
ReferenceSanctorum A. and Signer B. :Towards User-defined Cross-Device Interaction. In the Workshop on Distributed User Interfaces. Lugano, Switzerland (2016).