![Page 1: Mir Farooq Ali Computer Science, Virginia Tech May 9, 2003 Building Multi-platform User Interfaces using UIML](https://reader035.vdocuments.net/reader035/viewer/2022070403/56649f325503460f94c4e6a6/html5/thumbnails/1.jpg)
Mir Farooq AliComputer Science, Virginia
TechMay 9, 2003
Building Multi-platform User Interfaces using UIML
![Page 2: Mir Farooq Ali Computer Science, Virginia Tech May 9, 2003 Building Multi-platform User Interfaces using UIML](https://reader035.vdocuments.net/reader035/viewer/2022070403/56649f325503460f94c4e6a6/html5/thumbnails/2.jpg)
Outline
• Multi-platform User Interface (UI) problem
• User Interface Markup Language (UIML)
• Task Modeling• Future Research
![Page 3: Mir Farooq Ali Computer Science, Virginia Tech May 9, 2003 Building Multi-platform User Interfaces using UIML](https://reader035.vdocuments.net/reader035/viewer/2022070403/56649f325503460f94c4e6a6/html5/thumbnails/3.jpg)
Introduction
• Variety of Hardware
• Variety of target platforms
Netscape 6.1 for HTML
Internet Explorer 6.0 for HTMLJava Swing
![Page 4: Mir Farooq Ali Computer Science, Virginia Tech May 9, 2003 Building Multi-platform User Interfaces using UIML](https://reader035.vdocuments.net/reader035/viewer/2022070403/56649f325503460f94c4e6a6/html5/thumbnails/4.jpg)
Introduction
• Users want access to the same applications and data
• Traditional approaches are geared towards a single platform
• New approach is needed to build multi-platform UIs
![Page 5: Mir Farooq Ali Computer Science, Virginia Tech May 9, 2003 Building Multi-platform User Interfaces using UIML](https://reader035.vdocuments.net/reader035/viewer/2022070403/56649f325503460f94c4e6a6/html5/thumbnails/5.jpg)
Goals for UIML
• Able to describe implementations of UIs that are– Multi-platform– Multi-lingual
• Usable for non-traditional UIs: voice, etc
• Same expressive power as target languages (Java, HTML, WML, …)
![Page 6: Mir Farooq Ali Computer Science, Virginia Tech May 9, 2003 Building Multi-platform User Interfaces using UIML](https://reader035.vdocuments.net/reader035/viewer/2022070403/56649f325503460f94c4e6a6/html5/thumbnails/6.jpg)
UIML
• UIML is an XML-based markup language• Meta-language for UIs• Can be used currently in conjunction
with platform-specific vocabularies to create Uis– A vocabulary is defined to be the set of UI
widgets, their properties and events for one specific platform
![Page 7: Mir Farooq Ali Computer Science, Virginia Tech May 9, 2003 Building Multi-platform User Interfaces using UIML](https://reader035.vdocuments.net/reader035/viewer/2022070403/56649f325503460f94c4e6a6/html5/thumbnails/7.jpg)
UIML (Cont’d…)
• Provides complete description of a UI• Renderers are needed to take UIML
file and render it to UI• Rendering can be done through
– Interpretation– Translation
• Renderers are available for Java, HTML, WML, VoiceXML and PalmOS
![Page 8: Mir Farooq Ali Computer Science, Virginia Tech May 9, 2003 Building Multi-platform User Interfaces using UIML](https://reader035.vdocuments.net/reader035/viewer/2022070403/56649f325503460f94c4e6a6/html5/thumbnails/8.jpg)
Example UI
Widget
s
Events associated with widgets
Look and feel
![Page 9: Mir Farooq Ali Computer Science, Virginia Tech May 9, 2003 Building Multi-platform User Interfaces using UIML](https://reader035.vdocuments.net/reader035/viewer/2022070403/56649f325503460f94c4e6a6/html5/thumbnails/9.jpg)
Separation of Concerns
• UIML considers an UI to be composed of 4 sections– structure– style– behavior– content
![Page 10: Mir Farooq Ali Computer Science, Virginia Tech May 9, 2003 Building Multi-platform User Interfaces using UIML](https://reader035.vdocuments.net/reader035/viewer/2022070403/56649f325503460f94c4e6a6/html5/thumbnails/10.jpg)
<?xml version="1.0" encoding="ISO-8859-1" ?><!DOCTYPE uiml PUBLIC "-//UIT//DTD UIML 2.0 Draft//EN" "UIML2_0g.dtd">
<uiml> <head>...</head> <interface> <structure>...</structure> <style>...</style> <behavior>...</behavior> <content>...</content> </interface> <peers>...</peers> <template>...</template></uiml>
UIML (Cont’d…)
• Skeleton of UIML document
![Page 11: Mir Farooq Ali Computer Science, Virginia Tech May 9, 2003 Building Multi-platform User Interfaces using UIML](https://reader035.vdocuments.net/reader035/viewer/2022070403/56649f325503460f94c4e6a6/html5/thumbnails/11.jpg)
UIML <structure>
• Structure section indicates the hierarchical relationship between various ‘parts’
<structure>
<part id="BigFrame" class="JFrame">
<part id="label1" class="JLabel"/>
</part>
</structure>
Widget
![Page 12: Mir Farooq Ali Computer Science, Virginia Tech May 9, 2003 Building Multi-platform User Interfaces using UIML](https://reader035.vdocuments.net/reader035/viewer/2022070403/56649f325503460f94c4e6a6/html5/thumbnails/12.jpg)
UIML <style>
• Style section describes “look and feel” of UI using properties for individual parts
<style>
<property part-name="BigFrame" name="title">UIML Example</property>
<property part-name="label1" name="text">Hello World!</property>
</style>
![Page 13: Mir Farooq Ali Computer Science, Virginia Tech May 9, 2003 Building Multi-platform User Interfaces using UIML](https://reader035.vdocuments.net/reader035/viewer/2022070403/56649f325503460f94c4e6a6/html5/thumbnails/13.jpg)
UIML <behavior>
• Behavior section defines what happens when you perform any action on the parts
<behavior> <rule> <condition> <event class="actionPerformed" part-name="topButton"/> </condition> <action> <property part-name="label1" name="text">Top button
pressed.</property> </action> </rule></behavior>
![Page 14: Mir Farooq Ali Computer Science, Virginia Tech May 9, 2003 Building Multi-platform User Interfaces using UIML](https://reader035.vdocuments.net/reader035/viewer/2022070403/56649f325503460f94c4e6a6/html5/thumbnails/14.jpg)
UIML Examples
1. Hello World example• Java Swing• WML• HTML
2. UI with events• UIML with Java Swing vocabulary• Actual Java Swing program
![Page 15: Mir Farooq Ali Computer Science, Virginia Tech May 9, 2003 Building Multi-platform User Interfaces using UIML](https://reader035.vdocuments.net/reader035/viewer/2022070403/56649f325503460f94c4e6a6/html5/thumbnails/15.jpg)
Example: UI for 3 Platforms
Date book:
Java
Palm OS WML
![Page 16: Mir Farooq Ali Computer Science, Virginia Tech May 9, 2003 Building Multi-platform User Interfaces using UIML](https://reader035.vdocuments.net/reader035/viewer/2022070403/56649f325503460f94c4e6a6/html5/thumbnails/16.jpg)
What UIML Is and Is Not
• UIML: is a language for implementing UIs
• UIML: is not a language for UI design
(We assume UIML is generated by a model, specification language, design tool, or even manually)
![Page 17: Mir Farooq Ali Computer Science, Virginia Tech May 9, 2003 Building Multi-platform User Interfaces using UIML](https://reader035.vdocuments.net/reader035/viewer/2022070403/56649f325503460f94c4e6a6/html5/thumbnails/17.jpg)
Example: Vocabularies & Mappings
• Recall:<part class=“Button” name=“submit”> <style><property name=“title”>OK!</property></style></part>
• Vocabulary: class, property, event names (e.g., “Button”)
• Mapping:<d-class name=“Button" … maps-to="javax.swing.JButton">…</d-class>versus<d-class name="Button" … maps-to=“html:input">…</d-class>
![Page 18: Mir Farooq Ali Computer Science, Virginia Tech May 9, 2003 Building Multi-platform User Interfaces using UIML](https://reader035.vdocuments.net/reader035/viewer/2022070403/56649f325503460f94c4e6a6/html5/thumbnails/18.jpg)
How UIML Addresses Multi-Platform UIs
1. Tree structure (cascading):describe a family of UIs as in tree form
2. Generic vocabulary + transformation3. Building block of overall
methodology
![Page 19: Mir Farooq Ali Computer Science, Virginia Tech May 9, 2003 Building Multi-platform User Interfaces using UIML](https://reader035.vdocuments.net/reader035/viewer/2022070403/56649f325503460f94c4e6a6/html5/thumbnails/19.jpg)
1. Tree Structure
Factor out common elements to UIs on all platforms in form of an XML tree– Structure: changes across families– Style: changes across families– Content: may change across families– Behavior: may largely remain intact– Logic: API to logic unchanged– Presentation: differs by family
![Page 20: Mir Farooq Ali Computer Science, Virginia Tech May 9, 2003 Building Multi-platform User Interfaces using UIML](https://reader035.vdocuments.net/reader035/viewer/2022070403/56649f325503460f94c4e6a6/html5/thumbnails/20.jpg)
2. Generic Vocabulary + Transformation
• Vocabulary is defined outside UIML
• Vocabulary represents set of abstractions. Abstractions could be…1. identical to UI toolkit (“Platform-Specific
vocabulary”)2. cross UI toolkits (e.g., generic across Java/HTML,
VoiceXML, WML)3. domain-specific (e.g., abstractions to build UIs for
news articles -- title, abstract, body, navigation)
![Page 21: Mir Farooq Ali Computer Science, Virginia Tech May 9, 2003 Building Multi-platform User Interfaces using UIML](https://reader035.vdocuments.net/reader035/viewer/2022070403/56649f325503460f94c4e6a6/html5/thumbnails/21.jpg)
Sample Generic Vocabulary (#2 in Previous List)
Generic Part UIML Class NameGeneric Top Container
G:TopContainer
Generic Area G:Area
Generic List G:List
Generic Label G:Label
Generic Button G:Button
Generic Icon G:Icon
Generic Radiobutton G:RadioButton
![Page 22: Mir Farooq Ali Computer Science, Virginia Tech May 9, 2003 Building Multi-platform User Interfaces using UIML](https://reader035.vdocuments.net/reader035/viewer/2022070403/56649f325503460f94c4e6a6/html5/thumbnails/22.jpg)
Transformations
• Transformation maps UIML document with generic vocabulary to UIML document with platform-specific vocabulary
<part class=H:html> <part class=H:head>... <part class=H:title>......<part class=body>...
<part class=G:TopContainer>
![Page 23: Mir Farooq Ali Computer Science, Virginia Tech May 9, 2003 Building Multi-platform User Interfaces using UIML](https://reader035.vdocuments.net/reader035/viewer/2022070403/56649f325503460f94c4e6a6/html5/thumbnails/23.jpg)
Transform Engine
HTML Renderer
Java for Swing toolkit
Example of How UIML Is Used
HTML for Web browser
UIML/HTML vocabulary
UIML/Java vocabulary
Java Renderer
UIML describing application with generic vocabulary
![Page 24: Mir Farooq Ali Computer Science, Virginia Tech May 9, 2003 Building Multi-platform User Interfaces using UIML](https://reader035.vdocuments.net/reader035/viewer/2022070403/56649f325503460f94c4e6a6/html5/thumbnails/24.jpg)
3. UIML as Building Block
Task Model
UIML Family Model for desktop platform
UIML Family Model for WML
platform
UIML for Java Swing
UIML for HTML
UIML for VoiceXML
UIML for WML
Model->UIMLTransformation 1
UIML->UIMLTransformation
UIML Family Model for Voice platform
Generic UIML
Platform-Specific UIML
![Page 25: Mir Farooq Ali Computer Science, Virginia Tech May 9, 2003 Building Multi-platform User Interfaces using UIML](https://reader035.vdocuments.net/reader035/viewer/2022070403/56649f325503460f94c4e6a6/html5/thumbnails/25.jpg)
Multi-platform UIs
• Family model is in UIML using generic vocabulary– Generic vocabulary contains UI widgets
like G:TopContainer, G:Label, etc.
![Page 26: Mir Farooq Ali Computer Science, Virginia Tech May 9, 2003 Building Multi-platform User Interfaces using UIML](https://reader035.vdocuments.net/reader035/viewer/2022070403/56649f325503460f94c4e6a6/html5/thumbnails/26.jpg)
Task Model
• Using ConcurTaskTree notation– Developed by Fabio Paterno, CNUCE-
C.N.R., Pisa, Italy
• Hierarchical tree-like structure– Each node in tree indicates a task– Temporal operator exists between
adjacent sibling tasks
![Page 27: Mir Farooq Ali Computer Science, Virginia Tech May 9, 2003 Building Multi-platform User Interfaces using UIML](https://reader035.vdocuments.net/reader035/viewer/2022070403/56649f325503460f94c4e6a6/html5/thumbnails/27.jpg)
Task Types
• Four different task types– User– Interaction– Abstract– Application
• Eight different kinds of temporal operators
![Page 28: Mir Farooq Ali Computer Science, Virginia Tech May 9, 2003 Building Multi-platform User Interfaces using UIML](https://reader035.vdocuments.net/reader035/viewer/2022070403/56649f325503460f94c4e6a6/html5/thumbnails/28.jpg)
Use of Task Model
• Developer builds task model specifying task types and temporal operators
• Transformation done by system to create generic UIML, mapping – tasks to UIML ‘parts’ or widgets– temporal operators to UIML ‘behavior’
![Page 29: Mir Farooq Ali Computer Science, Virginia Tech May 9, 2003 Building Multi-platform User Interfaces using UIML](https://reader035.vdocuments.net/reader035/viewer/2022070403/56649f325503460f94c4e6a6/html5/thumbnails/29.jpg)
Sample Task Model
![Page 30: Mir Farooq Ali Computer Science, Virginia Tech May 9, 2003 Building Multi-platform User Interfaces using UIML](https://reader035.vdocuments.net/reader035/viewer/2022070403/56649f325503460f94c4e6a6/html5/thumbnails/30.jpg)
Interaction design
Task Model-centric Interaction Design
Task Model
Family Model for desktop platform
Family Model for phone platform
UIML for Java Swing
UIML for HTML
UIML for VoiceXML
UIML for WML
Family Model for Voice platform
Task/Domain analysis
Interaction s/w design
Interaction s/w impl.
Traditional Usability Engineering
![Page 31: Mir Farooq Ali Computer Science, Virginia Tech May 9, 2003 Building Multi-platform User Interfaces using UIML](https://reader035.vdocuments.net/reader035/viewer/2022070403/56649f325503460f94c4e6a6/html5/thumbnails/31.jpg)
Future Research
• Accessibility– Treat accessible toolkits as a separate
platform
• TIDE– Enhance the current version– Introduce more platforms
![Page 32: Mir Farooq Ali Computer Science, Virginia Tech May 9, 2003 Building Multi-platform User Interfaces using UIML](https://reader035.vdocuments.net/reader035/viewer/2022070403/56649f325503460f94c4e6a6/html5/thumbnails/32.jpg)
TIDE
![Page 33: Mir Farooq Ali Computer Science, Virginia Tech May 9, 2003 Building Multi-platform User Interfaces using UIML](https://reader035.vdocuments.net/reader035/viewer/2022070403/56649f325503460f94c4e6a6/html5/thumbnails/33.jpg)
Publications
• CADUI’2002 Full Paper• CHI’2002 Short Paper• Book chapter in edited book “Multi-
Device User Interfaces: Engineering and Application Frameworks”
![Page 34: Mir Farooq Ali Computer Science, Virginia Tech May 9, 2003 Building Multi-platform User Interfaces using UIML](https://reader035.vdocuments.net/reader035/viewer/2022070403/56649f325503460f94c4e6a6/html5/thumbnails/34.jpg)
Questions?