1 early usability evaluation in model-driven architecture environments 6 th international conference...

1

Early Usability Evaluation in Model-Driven Architecture

Environments

6th INTERNATIONAL CONFERENCE ON QUALITY SOFTWARE (QSIC

2OO6)Beijing, China - October 27-28, 2006

Silvia Abrahão and Emilio Insfrán

Departmento de Sistemas Informáticos y Computación (DSIC)Universidad Politécnica de Valencia{sabrahao, einsfran}@dsic.upv.es

mailto:[email protected]

mailto:[email protected]

mailto:einsfran%[email protected]

2QSIC 2006, Beijing, China - October 27-28

Contents

Introduction A Framework for Usability Evaluation

Usability at the Platform Independent Model (PIM) Usability at the Platform Specific Model (PSM)

Usability Model Instantiating the Usability Framework in a MDA

tool Results Conclusions and Further Work


Introduction Software development companies recognize the usability

as a key factor for the success of their products. However, software engineering practices don’t consider

usability aspects properly. Several studies show that 80% of total maintenance costs

are related to problems of users with the system. Among these costs, 64% are related to usability problems! (Landauer95)

Many approaches to evaluate the software usability have been proposed in the last years. Most of them focus on defining a set of attributes that explains usability and on developing guidelines and heuristics for testing it. They mainly focus on artifacts used in the late stages of

software development without connecting usability evaluation results with changes in software design.


Introduction Model-Driven Software Development (MDSD) appears with

its main characteristics: Separation of business functionality from technology

implementation The use of models along the complete software development

process by means of transformations (until the final source code)

This is a new challenge for the usability evaluation!

There are many tools and approaches for MDA (including User Interface specification) but none of these incorporate the usability explicitly There are no approaches which systematically evaluate the

usability of the applications obtained as a result of a process of successive transformations.


Contents


Usability at the Platform Independent Model Usability at the Platform Specific Model




A Framework for Usability EvaluationGoal A generic framework that incorporates usability in MDA

processes, where: The usability of a software application is evaluated and

improved at the platform independent model (PIM) using a Usability Model.

As outcome, a list of usability problems is produced. Instead of relate a given usability problem with changes in the

interface tier (code model level), we relate it to the elements of the PIM that are affected by it.

The evaluation also gives a feedback to the transformation rules itself (PSM).


A Framework for Usability Evaluation

An iterative process that combines model development and usability evaluation. It is applied until the PIM has the required level of usability.

The evaluation is made without users.

This allows very early usability evaluation by verifying the PIM properties that are related to usability.

MODEL COMPILATION

SYSTEM ANALYSIS

APPLICATION MODEL

EVALUATION WITHOUT USERS

Usability Model

PLATFORM INDEPENDENT

MODEL

PLATFORM SPECIFIC MODEL

CODE MODEL

1 A

REQUIREMENTSMODEL

REQUIREMENTS ELICITATION

COMPUTATION INDEPENDENT

MODEL

1 B

PLATFORM INDEPENDENT

USABILITY REPORT

CONCEPTUAL MODELS

OBJECT MODEL

TASKS MODEL

DIALOG MODEL

PRESENTATION MODEL

SOURCE CODE

INTERFACE TIER

APPLICATION TIER

PERSISTENCE TIER

MDA proposalMDA-based

development processEarly Usability evaluation

process

...


Contents






The development of the model followed several activities.

According to the Goal-Question-Metric (GQM), the goal of our usability model is:

Analyze PIM (task, dialog, presentation models, etc.) and/or the CM (final UI) For the purpose of Evaluate with respect to its Usability from the viewpoint of a set of evaluators In the context of Software applications obtained with a

MDA development process

Usability ModelGoal


Attractiveness

Compliance

This activity is concerned with the decomposition of Usability into a set of sub-characteristics and measurable attributes.

The ISO/IEC 9126-1 quality model is used as the basis for building the proposed usability model:

Usability ModelSpecify Sub-Characteristics and Attributes

Operability

ISO 9126

FunctionalityReliabilityUsabilityEfficiencyMaintainabilityPortability

Understandability

Learnability

Concepts with high level of abstractionConcepts with high level of abstraction



The decomposition is done to support usability inspection methods and to identify and fix usability problems.

1-Learnability refers to the attributes of a software product that facilitate learning. In our model it is defined in terms of: Help facilities (pre-defined keywords, wizards, on-line help, and documentation) Predictability which refers to the ease with which a user can determine the result of his/her future actions, Informative feedback in response to user actions Memorability as a measure of how quickly and accurately users can remember how to use an application that

they have used before.



2-Understandability refers to the attributes that facilitate understanding : We differentiate the own optical legibility of texts and images from

readability that refers to the information grouping cohesiveness and density.

Familiarity is the degree to which a user recognizes UI components and views their interaction as natural; UI can be familiar by imitating the visual appearance of real-world objects, by relying on standardized commands, or by following other common metaphors.

Brevity is related to the reduction of user cognitive effort (i.e., limiting the reading and input workload and the number of action steps).

User guidance is related to the message quality and navigability to guide the user by providing mechanisms such as path and current position.



3-Operability refers to the attributes that facilitate user control and operation: Capacity of the system to provide instalability facilities (e.g., user’s

help, documentation and installation procedures) Data validation (data validity), the user’s degree of control to services

execution (controlability), Capability of adaptation of the UI or services. It refers to the system

capacity to behave contextually and according to the users’ needs and preferences. We distinguish between

Adaptability, which is the system capacity to be adapted by the user, and Adaptivity, which is the system capacity to adapt to the users’ needs.

Operational consistency in the execution of services and controls (consistency), the prevention and error management, and the capacity to monitor the state of the system (state system monitoring).



The other two sub-characteristics are related to the perception of the end-user (4-Attractiveness) or evaluator (5-Compliance) using the system. It can be mainly measured using subjective measures (e.g., surveys).

Attractiveness: some aspects of attractiveness related to aesthetic design can also be quantified by measuring the UI uniformity in terms of font color, font style, font size and elements position. Example: Some studies suggest that designs should not use

more than 2-3 fully saturated intense colors.



Compliance: can be measured by assessing the agreement of the proposed usability model with respect to the following standards: ISO/IEC 9126-1 (from SE field) ISO 9241-11 (from HCI field) Microsoft and Java style guides …

The first two assessments can be performed at PIM or CM levels, whereas the last one can only be done at the CM, as it analyses the fulfillment of the UI widgets to the prescriptions of the style guides.


Usability Model

Learnability Help Facilities

Documentation Completeness

Multi-user Documentation Capability Profiled Role Profiled

Predictability Icon Significance Icon/Link Title Significance Action Determination

Informative Feedback Memorabitity

Time to Remember Accuracy

Understandability Legibility

Font Size Contrasting Text Disposition

Readability Information Grouping

Cohesiveness Location Grouping Format Grouping

Information Density Familiarity

Labeling Significance Internationalization Metaphor

Workload Reduction Brevity

Values Inizialization Initial Values Completion Initial Values Modifiability

Actions Minimization Self-descriptiveness

User Guidance Message Quality Navigability


Usability Model Operability

Installability Ease of Installation Multiplicity of Installation Updateability Update Transparency

Data Validity Controlability

Edition Deferral Cancel Support Explicit Execution Interruption Support Undo Support Redo Support

Capability of Adaptation Adaptability Adaptivity

Consistency Steady Behavior of Controls Permanence of Controls Stability of Controls Order Consistency Label Consistency

Error Management Error Prevention Error Recovery

State System Monitoring Attractiveness

Background Color Uniformity Font Color Uniformity Font Style Uniformity Font Size Uniformity UI Position Uniformity Subjective Appealing

Compliance Degree of Fulfillment with

the ISO 9241-10 / ISO 9241-11

Degree of Fulfillment with the Microsoft style guide

Degree of Fulfillment with the Java style guide


Contents



Usability Model Operationalizing the Usability Model into a MDA



Operationalizing the Usability Model into a MDA tool The Usability framwork was

instantiate in a specific MDA tool: OlivaNova Model Execution (ONME)

This tool is based on:

Conceptual Model (PIM): Object Model, Dynamic Model, Functional Model and Presentation Model

Model Compilers that establish the specific to a given platform (PSM) and generates the code (CM) applying transformation rules

Hierarchical Action Tree

Service IU

Instance IU

Population IU

Master/Detail IU

Introduction

Defined Selection

Argument Grouping

Supplementary Information

Order Criterium

Display Set

Filter

Dependency

Actions

Navigation

A uses B

A B

Key


Operationalizing the Usability Model into a MDA tool

Association of one or more elements of the PIM / CM models to each identified attribute.

The attributes in our Usability Model are inherent to UI modeling and make a product usable.

One or more metrics are associated to each model attribute.

ISO/IEC 9126

Usability

Learnability Compliance

AttractivenessOperability Understandaility

Platform Independent Model (PIM)

UI de maestro/detalle UI de poblaciónUI de instanciaInstance IU UI de servicioService IU AJAHAT

Display set

N A ActionNavegationOrder criteria

Filter

Usability definitions

Abstract interface patterns

Ergonomic criteria

Concrete interface

components

Explicit ControlInformation Significance

Platform Specific Model (PSM)

Interface standards

- IBM CUA - Microsoft

Poblation IU

Explore Bar

Control Filter

Generic component INSTANCE

Radio Button

CIO (Concrete Interface Object) VB

Prompting

Menu

Grid

Choose of platform

Legibility

ONME Presentation patterns

F

. . .

Computing Platforms

Visual Basic

Java/Swing ColdFusion

JSP

ASPPocketPC

Error Management

Consistency

Master-Detail IU

explicit control

Workload

A/Z

AIO (Abstract Interface Objects)


Contents






Results Examples of sub-characteristics, attributes, PIM elements

affected and potential metrics.


The Usability Model was applied across several usability inspections carried out in ONME interface models (PIM) and generated applications (CM).

This allowed us to validate the model experimentally. Some identified Usability Problems (UPs):

Lack of descriptive labels Difficult navigation Inflexible/invalid search capabilities Inconsistent menu item and window labels No feedback for user selection Meaningless error messages

Results Obtained and their Effects on the PIM and PSM


Once identified the usability problems, its source must be determined (PIM or PSM) to define a Recommendations Plan.

e.g. the problem Meaningless error messages affects the quality of error messages

Although the message informs how to solve the problem, they must be meaningful for users!

In this case, the source of the problem is in the PIM (Presentation Model) and relates to the property “Validation message” of the Introduction Pattern of a Service Interaction Unit (Service IU).

Usability Problems that affect the PIM


Recommending a change in the PIM

Identifier: 17

Usability Problem: Lack of descriptive labels

Context: Instance Interaction Unit (IIU) and Supplementary Information pattern

Related ergonomic criteria:

Guidance

Usability attribute: Labeling significance

Example: Note that the display set of the IIU (indicated by the row) has no label.

Recommendation: Identify the patterns in the Presentation Model without alias and define them.Apply the proposed metric (proportion of elements with meaningful names).


Information grouping cohesiveness and information density attributes were poor evaluated because of the lack of grouping mechanisms. This problem and recommendation is as follows:

Usability Problems that affect the PSM

Identifier 10

Usability Problem The users’ understanding of a display screen depends on the grouping and density of the objects (images, texts, etc.) that are presented. Currently, in ONME, all the attributes are shown in an abstract interaction unit without any grouping mechanism.

Context Instance Interaction Units, Master-Detail Interaction Units

Related ergonomic criteria:

User explicit control, Predictability

Usability attribute: Information group cohesiveness and Information density

Recommendation: Change the transformation rules (PSM) to incorporate different representations to the grouping mechanisms such as boxes, tabs, eye fishing, etc.


In a MDA environment, some usability attributes are garantee by the transformation process User guidance attributes

Visual feedback in menus and dialogs boxes where choices can be selected or when cursor is pointing (tooltips)

Visual feedback when elements (objects) are selected …

Legibility attributes: Prompts and error messages always appear in the same place(s) Error messages displayed in pop-up windows with the same

structure: what-where-howToSolveIt …

These give us empirical evidence of the intrinsic quality of UIs generated by a MDA tool.

MDA Benefits


Contents






We have presented a framework for early usability evaluation that fits in MDA development processes.

Unlike existing proposals, the usability is evaluated and improved at the PIM level instead of in its final user interface (CM).

This framework has been operationalized by means of a Usability Model which represents the relationships among the ISO/IEC 9126 and the usability patterns and criteria from the HCI field.

This model was applied in the inspection of several PIMs (presentation models) specified using a MDA tool.

Conclusions


The framework was found to be very useful not only to evaluate PIMs but also to discover deficiencies and/or limitations of the PIM expressiveness and the transformation rules (giving feedback for their improvement!).

The contribution of the work is to provide a set of attributes (independent of any method or technology) that can be verified in the PIM/PSM/CM defining the corresponding relationships and metrics.

Conclusions


Further Work The application of the usability framework to other MDA

tools specifically oriented to Web development (i.e., VisualWade).

We are concerned, in particular, to explore and evaluate to which extend our usability model can improve the development of Web applications.

The operationalization of the Usability Model using a set of usability metrics from the Web Quality Model (Calero et al., 2005). The first project will be the evaluation of the usability of a Spanish museum (Thyssen-Bornemisza Museum) from its code model.

This will allow to measure the level of usability of a model or a implemented UI.


Thanks for your attention!

1 early usability evaluation in model-driven architecture environments 6 th international conference...

Documents

software usability

usability framework

usability evaluationusability

usability aspects

usability evaluation

early usability evaluation

list of usability problems

required level of usability