M i S d b U biliMapping Study about Usability Requirements ElicitationRequirements Elicitation

Yeshica Isela Ormeño, Jose Ignacio Panach

Index

1. Introduction2. Mapping Study Designpp g y g3. Results4 Conclusions4. Conclusions

1. Introduction

How to capture functional requirements?

1. Introduction

How to capture functional requirements?

1. Introduction

How to capture interaction requirements?

Create suscriberCreate suscriber

Process suscriberCategory

[ ]>>

g y

1. Introduction

How to capture usability requirements?

1. Introduction• Usability requirements aim to improve the system

usability.• The goal of developing usable systems has been dealt

with the Human Computer Interaction (HCI) and Software Engineering (SE) fieldsSoftware Engineering (SE) fields.– HCI has proposed several usability guidelines, but they are too

abstract. – SE has defined sound methods to capture functional

requirements, but usability is relegated to last stages.

I thi i t d i t id tif th i ti• In this mapping study we aim to identify the existing methods for capturing usability requirements.

1. Introduction• MS provides an objective procedure for identifying the

t d t t f th h th t i il bl tnature and extent of the research that is available to answer a particular question.We aim to identify gaps in current research and to• We aim to identify gaps in current research and to suggest areas for further investigation.

• In particular we focus on finding works to capture• In particular, we focus on finding works to capture usability requirements in the MDD methods.

Index

1. Introduction2. Mapping Study Designpp g y g3. Results

C4. Conclusions

2. Mapping Study Design• The MP has been defined according to Kitchenham’s

kworks.• There are previous MP related to usability but none of

them focuses on usability requirementsthem focuses on usability requirements.

2.1 Research Question• Research question: “What are the proposals to elicit

bilit i t th h t th ftusability requirements throughout the software development process?”

1 Methods to elicit usability requirements– 1.Methods to elicit usability requirements– 2.Methods to elicit interaction requirements– 3.Usability guidelines to elicit usability requirementsy g y q

2.1 Research Question• Research question: “What are the proposals to elicit

bilit i t th h t th ftusability requirements throughout the software development process?”

1 Methods to elicit usability requirements– 1.Methods to elicit usability requirements– 2.Methods to elicit interaction requirements– 3.Usability guidelines to elicit usability requirementsy g y q– 4 Tools to support usability requirements elicitation– 5 Notations to elicit usability requirements– 6 Empirical validation environment

2.2 Search Strategy• Search strategy: We have used the following search

IEEE l ACM Di it l Lib S i Li ksources: IEEExplore, ACM Digital Library, Springer Link, and Science Direct.

• We have used Scopus as search tool• We have used Scopus as search tool.• The sources explored were the proceedings of

conferences journals books and workshops (from 2000conferences, journals, books, and workshops (from 2000 to 2011).

RESEARCH SEARCH SELECTIONRESEARCH QUESTION

SEARCH STRATEGY

SELECTION CRITERIA

PUBLICATIONS SELECTION

QUALITY ASSESSMENT

DATA EXTRACTION STRATEGY

2.2 Search Strategy• Search strategy: The search string is a set of terms to

obtain the publications that answer the research questionobtain the publications that answer the research question.• Search string = (Usability Requirement) AND (Software

Engineering)Engineering)– Usability Requirement = (usability requirement OR user

requirement OR usability elicitation OR interaction requirement OR f ti l OR bilit id li )non-functional OR usability guidelines).

– Software Engineering = (MDD OR model-driven OR MDA OR notation OR tool OR interface OR engineering OR test). g g )

2.3 Selection Criteria• Selection criteria: Inclusion and exclusion criteria are

d t i ddetermined– Inclusion criteria:

• Publications that define how to extract usability requirementsPublications that define how to extract usability requirements.• Approaches applied to an environment based on MDD conceptual

models.P blications that define ho to represent the sabilit req irements• Publications that define how to represent the usability requirements.

RESEARCH SEARCH SELECTIONRESEARCH QUESTION

SEARCH STRATEGY

SELECTION CRITERIA

PUBLICATIONS SELECTION

QUALITY ASSESSMENT

DATA EXTRACTION STRATEGY

2.3 Selection Criteria• Selection criteria: Inclusion and exclusion criteria are

d t i ddetermined:– Inclusion criteria:

• Publications that define how to extract usability requirementsPublications that define how to extract usability requirements.• Approaches applied to an environment based on MDD conceptual

models.P blications that define ho to represent the sabilit req irements• Publications that define how to represent the usability requirements.

– Exclusion criteria:• Publications focused on guidelines, notations, and tools where g

usability has not been considered or has not been included.• Publications that consider only functional requirements.• Publications written in a language that is not EnglishPublications written in a language that is not English.

2.4 Publication Selection• Publication Selection with a systematic process:

– Reading the title and the abstract. From 150 papers, we chose 65 (potential publications).

– Reading the whole publication. From 65 papers, we chose 27Reading the whole publication. From 65 papers, we chose 27 (initial selected publications).

– Searching in references. We added two new publications, res lting in a total of 29 papers (selected p blications)resulting in a total of 29 papers (selected publications).

RESEARCH SEARCH SELECTIONRESEARCH QUESTION

SEARCH STRATEGY

SELECTION CRITERIA

PUBLICATIONS SELECTION

QUALITY ASSESSMENT

DATA EXTRACTION STRATEGY

2.5 Quality Assessment• Quality assessment: to specify the quality of the

l t d bli tiselected publications.• We use a Likert-questionnaire to specify the quality.

RESEARCH QUESTION

SEARCH STRATEGY

SELECTION CRITERIA

PUBLICATIONS SELECTION

QUALITY ASSESSMENT

DATA EXTRACTION SELECTIONASSESSMENTSTRATEGY

2.5 Quality AssessmentSubjective Questions 1=Yes 0=Partially -1=No

1. Is the method to elicit the usability requirements clear?2. Are the guidelines to elicit requirements comprehensible?3. Are the guidelines to elicit requirements useful in other contexts?4. Are the publications tools downloadable?5. Is there a clear case study or example illustrating the proposal?6. Is the whole proposal empirically validated?7. Are the results clearly explained?8. Is the notation to elicit requirements easy to learn?

Objective Questions1 Has the publication been published in journal or conference1. Has the publication been published in journal or conference

proceedings?1=Very important 0=Important -1=Not important2 Has the publication been cited by other authors?2. Has the publication been cited by other authors?1= More than 4 0=Between 2 and 4 -1=Less than 2

2.6 Data Extraction Strategy• Data extraction strategy: it consists of classifying the

ibl f h h b tipossible answers for each research subquestion.

RESEARCH QUESTION

SEARCH STRATEGY

SELECTION CRITERIA

PUBLICATIONS QUALITY DATA EXTRACTION SELECTIONASSESSMENTEXTRACTION STRATEGY

Index

1. Introduction2 Mapping Study Design2. Mapping Study Design3. Results4. Conclusions

3. Results

• SQ1 Methods to elicit usability requirements.– Existing methods deal with usability requirements together with

other NFR.– NFR are elicited at the same time the system functionality andNFR are elicited at the same time the system functionality and

architecture are defined.– A holistic quality model that fits every context does not exist, and

th d t li it NFR l id b i i tmethods to elicit NFR only provide basic requirements management by means of extensions.

– A few approaches deal with usability requirements at early pp y q ystages.

3. Results

• SQ2 Methods to elicit interaction requirements.– Most methods are based on patterns or models.– Some methods improve usability by means of applying formal

modellingmodelling. – However, further work is needed to deal with dynamic

specifications that depend on system functionality.

3. Results

• SQ3 Guidelines to elicit usability requirements.– There is a huge variety of guidelines.– The application of these guidelines is not easy.

A few works propose a definition of new guidelines– A few works propose a definition of new guidelines. – Some representations are based on patterns, templates, or

models and they can be adapted according to which usability requirements are being captured.

– Nowadays, the guidelines do not provide precise support to address usability requirements elicitation at the early stagesaddress usability requirements elicitation at the early stages.

3. Results

• SQ4 Tools to support usability requirements elicitation– Selected publications present tools to support: frameworks,

structured styles, scenarios, notations and methods.– Some tools incorporate a mechanism of transformation (forSome tools incorporate a mechanism of transformation, (for

example, from task flow diagrams to formal representations).– A few tools are model-based and they can resolve traceability

b t i t d t tbetween requirements and next stages.

3. Results

• SQ5 Notations to elicit usability requirements:– More than one notation is usually applied to the development

method. – The user requirements specifications are usually presented toThe user requirements specifications are usually presented to

end-users in normal text, even though the analyst works with languages based on models. S l i t i t t f ti l i t d– Some proposals aim to integrate functional requirements and NFR in the same elicitation process.

3. Results

• SQ6 Empirical validation environment:– There is not explicit metrics to evaluate the usability

requirements elicitation. – Existing validations are focused on quantitative and qualitativeExisting validations are focused on quantitative and qualitative

usability requirements.– Studied publications are focused on evaluating a few usability

f tfeatures. – Patterns, scenario management, checklists, work sessions, and

templates are the most common artefacts used to evaluate pusability.

3. ResultsStudied Publications Quality of Publications

13

15

POTENTIAL PUBLICATIONS

SELECTED PUBLICATIONS12

16

14

16

18

8

6 6

87 7

8

5

10 12

6

8

10

12

2 2 23

12 1 1

5

3 3 3 3

0

1

0

2

4

Very good Good Bad

3.1 Discussion• Usability requirements are elicited once all functional

requirements have been capturedrequirements have been captured.• Existing approaches do not propose a precise and

unambiguous notation to represent usabilityunambiguous notation to represent usability requirements.

• Guidelines are not defined such a way end-users canGuidelines are not defined such a way end users can understand them.

• Guidelines are not a holistic support to develop usable pp psystems.

• The methods are inflexible and require considerable effort to be applied in contexts that are different from the contexts where they have been defined.

Index

1. Introduction2 Mapping Study Design2. Mapping Study Design3. Results4. Conclusions

4. Conclusions• We have explored the development methods that

consider usability as a requirement.y q• A total of 29 publications were selected from an initial set

of 150 publications returned by the search string.• 97% of selected publications have a good quality.• If we focus on MDD, there are few works in this field• We plan to develop a framework to elicit usability

requirements in such a way that it could be used in any MDD method.

• We plan to develop transformation rules from the bilit i t t t l i d d iusability requirements to generate analysis and design

models

Th k !Thanks!Questions?Questions?