User Interfaces Composition with UsiXML

Sophie LEPREUX (1), Jean VANDERDONCKT (2), Christophe KOLSKI (1)

(1) LAMIH“Automated Reasoning and Human-Computer Interaction” Research Group

University of Valenciennes and Hainaut-Cambrésis, France

(2) BCHIUniversity of Louvain-La-Neuve, Belgium

UsiXML-EICS 2010 - Berlin, Germany - June 20, 2010

- To take the user caracteristics, the platform used, the environment into account, to allow an universal access to information, knowledge, services… in numerous different contexts

Introduction

New UI design problems, focus on UI Composition

Brain-Computer Interface

- Tree-algebra based UI Composition

-UsiXML framework

-Composition/decomposition Operators

- Compose to support the 7 Dimensions

- Conclusion

Summary

Tree-algebra based UI Composition

Window (id=window, name=window, width=« 500 » height=« 350 »)

Box (type=« vertical »)

Button (DefaultContent = Save)

Button(DefaultContent=Close)

Output (DefaultContent =« Fiche Client »)

Box (type = horizontal)

Box (type = horizontal)

Output(…)

Input(…)

Box (type = horizontal)

Output(…)

Input(…)

tag content

<cuiModel id="Client_registration-cui_31" name="Client_registration-cui"> <window id="window_component_0" name="window_component_0"

defaultContent=« Nouveau client" width="500" height="350">

<box id="box_0" name="box_0" type="vertical"><outputText id="output_text_component_2"

name="output_text_component_2" defaultContent=« Fiche Client"

isVisible="true" isEnabled="true" isBold="true" textColor="#000000"/>

<box id="box_1" name="box_1" type="horizontal"><outputText id="output_text_component_2"

name="output_text_component_2«  defaultContent=« Fiche Client"

isVisible="true" isEnabled="true" isBold="true" textColor="#000000"/>

… </window> </cuiModel>

- Tree-algebra based UI Composition

-UsiXML framework

-Composition/decomposition Operators

- Compose to support the 7 Dimensions

- Conclusion

Summary

UsiXML framework

Task & Concepts

Abstract UI

Concrete UI

Final UI

Source = < User, Environnement, Platform >

Target = < User, Environnement, Platform >

Task & Concepts

Abstract UI

Concrete UI

Final UI

Summary

- Tree-algebra based UI Composition

-UsiXML framework

-Composition/decomposition Operators

- Compose to support the 7 Dimensions

- Conclusion

Composition/decomposition Operators

Cut( T1, )

T1 T2Unary operators Binary operators

Similarity

Equivalence

Left difference

Right difference

Intersection

Unique union

Normal union

Fusion

∩Complementary( T1, )

Projection( T1, )

Selection( T1, )

T1

Composition/decomposition OperatorsExample: Normal Union operator applied on two user interfaces

Uhorizontal

=

Composition/decomposition OperatorsIllustrations of operators on CUI models

T1 = tourist application UI T2 = event management application UI

(a) Selection (T1, set of elements)

(b) Projection (T1, {outputText, Button})

(c) Intersection (T1, T2)

- Tree-algebra based UI Composition

-UsiXML framework

-Composition/decomposition Operators

- Compose to support the 7 Dimensions

- Conclusion

Summary

Compose to support the 7 Dimensions

- Use of (de)composition operators to adapt the UI according to the 7 dimensions:

(1) Multi-device Usage and/or(2) Multi-User interface,(3) Multi-linguality,(4) Multi- organization,(5) Multi-context usage,(6) Multi-Modality usage,(7) Multi-platform Usage

Compose to support the 7 Dimensions(1) Multi-device usage

Task & Concepts

Abstract UI

Concrete UI

Final UI

Source = < User, Environnement, Platform = 1 device >

Task & Concepts

Abstract UI

CUI’1

FUI’

1

Target = < User, Environnement, Platform = n devices >

FUI’n

CUI’n

…

…

CUI’1 = Selection (set elements of Music controller task, Concrete UI)CUI’2 = Selection (set elements of Sounder player task, Concrete UI)CUI’3 = Selection (set elements of volume controller , Concrete UI)

Adaptation 1: decomposition of the CUI Source in several

CUIs

Adaptation 2: Fusion of several decompositions

of the CUI source

CUI’1 = Selection (set elements of Music controller task, Concrete UI)CUI’2 = Selection (set elements of Sounder player task, Concrete UI)Union Selection (set elements of volume controller , Concrete UI)CUI’3 = Selection (set elements of volume controller , Concrete UI)

- Illustration: control of a music player

Compose to support the 7 Dimensions(2) Multi-user interface

- Two interesting cases: Case #1: a shared UI is co-localized, the users can simultaneously interact on one UI

Ex.: tactile multi-touch interface or tangible objects used

Case #2: a user arrives in a collective work situation (or leaves it): the user needs a work space, with personal data and/or functions

- Other interesting problems to consider :

Distant shared UI

Consideration of the user rights or roles (ex: moderator UI in a brainwriting session at a distance: composed differently as the UI of the other participants)

Compose to support the 7 Dimensions(3) Multi-linguality

- if two similar UI are modeled at the CUI level (or higher) with specifications given for one (or more) language and another UI modeled with others languages, it can be possible to compose these UI to obtain one UI modeled for all these languages

- Importance to consider translation considerations leading to necessities about composition/decomposition due to socio-cultural aspects

Ex: source language read from left to right, and downwards (english) target language read from right to left, and downwards (arabic), or upwards (for instance: Chinese)

Consequence: different zones of the UI have to be composed/decomposed differently

Compose to support the 7 Dimensions(4) Multi-organization

T1 & C

T’& C

T2 & C

T’ = T1 U T2

Source = < User, Environnement, Platform >

Target = < User, Environnement, Platform >

Source = < User, Environnement, Platform >

- Importance to consider how the tasks have to be performed in different organizations or groups

- Importance to consider the organization models described in the literature (ex. in social sciences or multi-agent system domain)

Compose to support the 7 Dimensions(5) Multi-context usage

- Several composition/decompositions strategies are possible:

Ex.: development methodology based on patterns, business component and machine learning (plastic UI approach, [PhD Hariri 08])

Ex: integration of the task model in the business component in order to compose complete application and to facilitate the co-evolution of the system [Bourguin G., Lewandowski A. & Tarby J.C., TAMODIA’2007] - A context modification can generate evolutions at each abstraction level

Ex: a lighting change acts on the luminosity of the device (FUI) whereas the background noise change acts (or not) on the choice of the vocal modality.

Ex: a modification of the user work brings modification of tasks and/or on the tasks planning Interesting composition/decomposition problems to consider

Compose to support the 7 Dimensions(6) Multi-Modality usage

- The AUI layer allows specifying the UI independently of the modality According to needs brought by context change, a UI composition at a high level allows merging UI which will be concretized by different modalities in the lower levels

- Ex : Food order realized with several modalities

AUI1

AUI’

AUI

T’ = T1 U T2

Order Chinese food

AUI03: Choose meal

Chooseappetizer

Choosestarter

Choosesoup

Chooserice

AUI02: Give delivering address

Givename

Givephone

Giveaddress

AUI’ =AUI1 U AUI2

Normal Union (AUI (order a pizza), AUI (Order Chinese food))

Compose to support the 7 Dimensions(7) Multi-platform Usage

- Analogy: example of multi-platform language is: Java Execution on several platforms without changing model or implemention

- Our objective: to do the same from the UI point of view

to generate code from CUI model to FUI model

For example, graphical editor GraphiXML allows generating java or XHTML code

So the graphical UI which is modeled with GraphiXML is multi-platform

The composition operators: not needed into this process

- Tree-algebra based UI Composition

-UsiXML framework

-Composition/decomposition Operators

- Compose to support the 7 Dimensions

- Conclusion

Summary

Conclusion- Presentation of operators developed initially to compose at design time and at the CUI level

- Currently: operators used to each abstraction level of UI

- Operators: could be used at the runtime

- These operators: a support to UI adaptation

- Illustrations in the paper for the 7 dimensions: Multi-device Usage, Multi-User interface, Multi-linguality, Multi- organization, Multi-context usage, Multi-Modality usage, Multi-platform Usage

But: adaptations to criterion (dimension) can be done to the detriment of the others

As a result, the adaptation must be global i.e. the adaptation has to take into account all the criteria simultaneously and not considered one by one

- Research questions are open concerning: (1) the one by one or simultaneous consideration of these dimensions, (2) the order in taking them into consideration …

User Interfaces Composition with UsiXML

Sophie LEPREUX (1), Jean VANDERDONCKT (2), Christophe KOLSKI (1)

(1) LAMIH“Automated Reasoning and Human-Computer Interaction” Research Group

University of Valenciennes and Hainaut-Cambrésis, France

(2) BCHIUniversity of Louvain-La-Neuve, Belgium

UsiXML-EICS 2010 - Berlin, Germany - June 20, 2010

Ajout logo1 JVD

Ajout logo2 JVD

THANK YOU FOR YOUR ATTENTION