designing search user interfaces - department of … · designing search user interfaces anders...

Designing Search UserInterfaces

Anders Salander

March 15, 2011Master’s Thesis in Computing Science, 30 credits

Supervisor at CS-UmU: Lars-Erik JanlertExaminer: Jerry Eriksson

UmeaUniversityDepartment of Computing Science

SE-901 87 UMEASWEDEN

Abstract

Searching for information has become a natural task in today’s society. Nowadays more than2 billion people are connected to the Internet and using some kind of search functionality forfinding information is a central activity. The amount of digital information that is producedputs high constraints on the design of search interfaces.

This master thesis report presents a proof-of-concept design and implementation of asearch user interface. The main focus of the project is usability, especially in the aspectsof navigation and content presentation. There are many features that can support theuser during the search activity and this interface has been developed on the basis of thesefeatures.

The report describes an iterative user-centered design process that served as the foun-dation during the proof-of-concept development. The target user group has been involvedthroughout the project and the design has been evaluated on several stages in the designprocess and the proposed interface was well received by the target users. An interactivehigh-fidelity prototype was implemented in Adobe Flex 3 for the purpose of evaluation anddemonstration.

Contents

1 Introduction 1

1.1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

1.2 The eX-IFS application . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

1.3 Target group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

1.4 Assignment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

1.5 Method and Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

1.6 Limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

1.7 Report outline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

2 Designing usable Search user interfaces 5

2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

2.2 History and recent development of Search user interfaces . . . . . . . . . . . . 6

2.3 Usability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

2.4 Designing a search interface . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

2.4.1 Understanding the search process . . . . . . . . . . . . . . . . . . . . . 9

2.4.2 Design guidelines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

2.4.3 Interface features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

2.5 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

3 Method 19

3.1 Design process overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

3.2 Research . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

3.3 Gathering Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

3.3.1 Contextual inquiry . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

3.3.2 Interviews . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

3.3.3 Brainstorming . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

3.3.4 Focus groups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

3.4 Communicating with prototypes . . . . . . . . . . . . . . . . . . . . . . . . . 24

3.4.1 Low-fidelity prototyping . . . . . . . . . . . . . . . . . . . . . . . . . . 25

3.4.2 Hi-fidelity prototyping . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

3.5 Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

3.5.1 Think-aloud protocol . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

3.5.2 Quick and dirty evaluation: Asking users . . . . . . . . . . . . . . . . 27

3.5.3 Long-table approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

3.5.4 Questionnaires . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

iii

iv CONTENTS

4 Accomplishments 294.1 Research phase . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

4.1.1 Observing users . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 304.1.2 User interviews . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 304.1.3 Focus group sessions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

4.2 Design phase . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 314.2.1 Low-fidelity prototypes . . . . . . . . . . . . . . . . . . . . . . . . . . 314.2.2 High-fidelity prototype . . . . . . . . . . . . . . . . . . . . . . . . . . . 35

4.3 Final implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37

5 Results 395.1 The eX-IFS GUI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39

5.1.1 Layout . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 395.1.2 Start page . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 405.1.3 Search results page . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 405.1.4 Component details . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40

5.2 Search interface characteristics . . . . . . . . . . . . . . . . . . . . . . . . . . 425.2.1 Search results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 425.2.2 Search history . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 435.2.3 Search path . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 435.2.4 Search term suggestions . . . . . . . . . . . . . . . . . . . . . . . . . . 455.2.5 Limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45

6 Discussion and conclusion 476.1 Assignment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 476.2 Research and design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 476.3 Proof-of-concept implementation . . . . . . . . . . . . . . . . . . . . . . . . . 486.4 Future work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48

7 Acknowledgements 51

References 53

A Summary of user observations and interviews 57

B Focus group 59

C Focus group analysis 63

D Workflow prototypes (low-fidelity) 65

E Workflow prototype user tasks 69

F Workflow prototype final survey 71

G In-depth paper prototype (low-fidelity) 75

H High-fidelity prototype user tasks 77

I Summarized prototype evaluation 79

CONTENTS v

J GUI evaluation questionnaire 83

vi CONTENTS

List of Figures

2.1 The usability framework. Adapted from [2] . . . . . . . . . . . . . . . . . . . 72.2 The information seeking task. Adapted from [25] . . . . . . . . . . . . . . . . 102.3 The figure shows Holscher and Strube’s model and process with probabilities

calculated over both research groups. (a) Overview of the information seekingprocess (b) Close-up of direct interaction with a search engine. Adapted from[14] . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

2.4 A categorized result list (left) and a common result list (right) [6]. . . . . . . 15

3.1 The design process [34] . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

4.1 Vertical workflow layout . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 324.2 Horizontal workflow layout . . . . . . . . . . . . . . . . . . . . . . . . . . . . 334.3 Pop-up workflow layout . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 334.4 Wireframe overview of the in-depth paper-prototype layout . . . . . . . . . . 344.5 First version of High-fidelity prototype . . . . . . . . . . . . . . . . . . . . . . 36

5.1 Application overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 405.2 Component details overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . 415.3 Search results presentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 425.4 Non-empty result set . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 435.5 Historical view over the 10 most recent searches . . . . . . . . . . . . . . . . . 445.6 Path Breadcrumbs (within a search for: mp4) . . . . . . . . . . . . . . . . . . 445.7 Search term suggestions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45

vii

viii LIST OF FIGURES

List of Tables

I.1 The scores for each evaluated part for respective prototype version. Themaximum score is 8,00 for each column . . . . . . . . . . . . . . . . . . . . . 80

ix

x LIST OF TABLES

Chapter 1

Introduction

This chapter gives an introduction to the master thesis project through a short backgrounddescription, followed by a presentation of the task and its limitations.

1.1 Background

Searching is a central part of many computer systems today. Typically, systems need tohave the ability to find and display different kinds of information. A major challenge isthat a search via a web-search engine, such as Google, can result in a span of zero tomillions result hits. However, many systems are used to find specific information througha company database instead of information over the Internet. Typically, this limits theamount of information to some degree but there are still challenges and difficulties thatmust be overcome. A user needs to know what kind of information one can search for, beable to formulate a proper query, interpret the result list to find the expected informationas well as being able to reformulate the query if no proper result was found. There are waysto simplify the search process to better support the users in their searching task.

Saab Security and Defense Solutions is a high-technology company that develops complexmilitary products and systems. One of their commitments to their customers is to providethe products with support, such as spare-parts. Some of their products have a lifecycle ofmore than 30 years; hence, it is important for them to know exactly what components thata product consists of as well as which products an individual component is a part of. Thecomponent hierarchy tree looks like this:

– Project. This is the root of the component tree. A project can consist of systems,products, and components.

– System. This is a collection of a large set of compound parts. The parts can consistof other systems, products, and components.

– Product. This is a collection of a reasonable amount of compound parts. The parts ina product may be other products and individual components.

– Component. Typically, this is the leaf nodes in the component tree. However, somecomponents may consist of other components as well.

1

2 Chapter 1. Introduction

The information need is the same for every node type, that is, every node is handled likean individual component no matter if it is a project, system, product or component. Hence-forth the term component will be used for these node types, however, to avoid confusionsome of the above node labels may be used as a clarification when necessary.

The organization has an existing computer-based desktop system for this. However, thelack of usability makes the system inefficient and results in user frustration.

1.2 The eX-IFS application

The initiator to this thesis project is the Through Life Support (TLS) department at SaabSecurity and Defense Solutions in Sweden. This is a company within the Saab Group andthe TLS department’s main responsibility is after-sales errands on naval military systems.They have a need to develop a new web-based system for managing their customers’ navalproducts and systems.

A system may consist of 50.000+ components at different hierarchical levels within thesystem; the department must be able to get information regarding every significant partin the system in case of a failing component. Their existing system does not match theusability needs of the users, hence the initiative for this project. The users’ everyday workwith the existing and desired system is mainly based on searching and finding informationregarding the components.

1.3 Target group

The target group of the eX-IFS application is primarily personnel working at the ThroughLife Support (TLS) department at Saab Security and Defense Solutions, Sweden. The usersshare a common set of characteristics, these are:

– They use this kind of application frequently in their everyday work.

– Their knowledge of the problem domain is very large; hence they are regarded asexpert users.

– Searching is a central part of their work assignments.

– Typically, they know what to search for, that is, they use known-item search.

The target group wants a more usable application for support in their everyday work,in comparison to their existing system.

1.4 Assignment

The primary focus in this thesis project is to design and develop a Graphical User Interface(GUI) for the eX-IFS application prototype. The GUI should provide the ability to easilyperform searches over a collection of delivered systems in order to get information regardingthe current status of a system and its embedded parts.

The entire project is divided into two separate master thesis projects; one regardingdatabase design and searching and one regarding the development of a usable graphical userinterface (GUI) of a proof-of-concept (POC) application. The latter is the basis for thismaster thesis project.

1.5. Method and Results 3

The vision is to develop a functional application with real data that can be used in theTLS departments’ everyday work. The goal is to provide a partly functional prototype forthe application to serve as a POC for future development and implementation.

The selected platform for the prototype was an Adobe Flex front-end with a .NETWebService back-end; the latter is out of scope of this thesis project. That is, the authorof this thesis report was responsible for the usability and design of the front-end as well asthe POC implementation of the GUI.

1.5 Method and Results

In order to accomplish the specified goals of this thesis project, three primary areas whereidentified and explored. These are:

– An in-depth research and exploration study regarding the research area of how todesign usable Search User Interfaces.

– An iterative and user-centered approach to develop the design.

– A proof-of-concept implementation based on the research.

The in-depth study of Search user interfaces (SUI) was conducted with the purpose toget a deeper understanding of the problem domain. The result of this research study is achapter regarding certain important aspects and characteristics of SUI’s.

In order to develop a design proposal that is accepted by the target-group, a user-centeredapproach was adopted. Real users from the target-group were available throughout the entirethesis work. This resulted in a design that was developed through continuous collaborationwith the target-group.

The implementation of the proof-of-concept (POC) application design was based on theresults of the previous iterations of the design process. The application should be a web-based solution; hence a Rich Internet Application (RIA) prototype was developed. Thechosen platform was Adobe Flex, a common RIA-platform. The main goal was to developa POC interface that was intuitive and supported the users’ everyday tasks.

1.6 Limitations

The assigned task has a limited timeframe of 20 weeks for one person. This implies thatseveral important areas and aspects regarding design, evaluation and implementation ofthe eX-IFS POC have been omitted during the thesis project. It is important to mentionthat the implemented POC application is not a fully functional application. This should beregarded as a concept proposal for further development. Due to the time constraints, noaccount was taken of graphical design, underlying algorithms or system architecture.

1.7 Report outline

– Chapter 2. Designing usable Search user Interfaces. This section is an in-depth liter-ature study of Search user interface usability. The chapter touches different aspectsof search user interfaces such as how users perform searches as well as what kind offunctionality to incorporate to support the search process.

4 Chapter 1. Introduction

– Chapter 3. Method. This chapter introduces the scientific techniques and methodsthat were used throughout this thesis project. This includes a brief description as wellas pros and cons of the methods.

– Chapter 4. Accomplishments This chapter presents a chronological overview of thedifferent phases performed during the development of the eX-IFS GUI prototype.Each section and subsection touches important areas and aspects of the development.

– Chapter 5. Results. This chapter gives a more detailed description of the resultsfor each section. The individual results formed a basis for the final conceptual GUIprototype.

– Chapter 6. Discussion and conclusion. This chapter focuses on personal reflectionand discussion by the author regarding the thesis work. Some suggestions of futuredevelopments will conclude the chapter.

Chapter 2

Designing usable Search userinterfaces

This chapter is a literature study and aims to explain the concept of Search User Interfaces(SUI) with the focus on how to successfully design the user-interface for such applications.

2.1 Introduction

The fast paced development of the Internet has had a great impact on today’s society. Inthe beginning the Internet was primarily used by computer enthusiasts but now ordinarypeople use the Internet as an integral part of their everyday life, at home, at work or atschool [40]. People tend to turn to the Internet in order to find information and the use ofsearch-engines is now the second most frequent activity on the Internet [12], only beaten bye-mail.

A major problem today is that information management makes the everyday life morecomplicated, this is due to the vast amount of information we encounter and process eachday. The information management problem entails a high burden on our physical andcognitive resources. There are more choices to make, more information to search throughand information is deliberately intended to influence our behavior. This complicates ourinformation retrieval process. Electronic systems for information seeking must improve andhave a high usability in order to better support users during information seeking tasks [25].

A research survey performed in December 2009 by PEW [5] showed that 74% of theAmerican adults use the Internet. The survey also revealed that 88% of these users have themain goal of finding information through a search engine. One indication of the importanceof a successful and usable Search Interface is shown in a study by Forrester Research, where76% of firms considered search as “extremely important” while only 24% of them consideredtheir own web site’s search functionality to be “extremely useful” [13].

According to [9] the total amount of searches performed worldwide on the Internet wasover 131 billion during December 2009. This indicates an increase of 46% on the amountof searches compared to the corresponding period in 2008. During this period there wasapproximately 0.2 billion new Internet users registered, resulting in a total of about 2 billionInternet users worldwide [16]. How can one design a user interface that can handle theamount of information we use and produce today, and still maintain a high usability level?

The aim of this literature study is to present different aspects of design and development

5

6 Chapter 2. Designing usable Search user interfaces

of search user interfaces. The study will not cover all details or aspects; instead it shouldbe seen as an overview and a starting point for further research for those interested.

The study is organized as follows: First a section regarding the history and recentdevelopments in the concept of Search-user interfaces is explained, followed by a sectionabout general usability. Then a section regarding the search process, design guidelines andimportant characteristics of SUI’s is presented. Finally, the chapter concludes with a sectionthat contains a discussion of SUI characteristics.

2.2 History and recent development of Search user in-terfaces

The usage of Search User Interfaces has drastically changed since the advent of the Internetand World Wide Web. Previously searching was mainly performed by trained professionalssuch as librarians, researchers and journalists. The wide spread use of the Internet led tothe development of search engines such as AltaVista, Yahoo and Google [12].

The development of search interfaces is not limited to only the target group. The user-interfaces, the searchable information as well as the nature of the queries have all changedover the last years. Early systems were often closed, that is, non-public and content-monopolized. Unfortunately this discouraged the development of usable interfaces sincethere was a lack of competition amongst content-providers. Usually the interfaces werecommand-line based and the searcher had to learn and remember complex search-operatorsin order to perform successful searches [12].

It was uncommon to be able to search over full text articles; the searchable informationusually consisted of abstracts, metadata and values such as titles. The main goal of thesekinds of searches was often to find the name and location of the physical full text version[12]. However, since the general public got access to the amount of information availableon the Internet, the importance of functional search interfaces has become crucial. Theinterfaces today are quite simple and new features for supporting the user in the searchprocess are constantly being developed.

Full-text searches are now common since the web often provides full text documents.Modern search user interfaces commonly have very advanced algorithms underneath thegraphical surface. One powerful advantage of modern systems is that users have the abilityto use both keyword based searches and in some cases even natural language in their queries[12].

2.3 Usability

Usability is about understanding how to design interactive systems to best support theusers. However, it is not as simple as that. One must understand what the users want andneed as well as how to best match these requirements in order to be able to design a UIwith high usability. Beside this knowledge a usability expert or interaction designer mustalso have knowledge of the users’ work situation and the environment in which the systemwill operate [34].

There are many aspects of usability and the concept is somewhat subjective. Hearst [12]and Nielsen [30] refers to usability as how easy a user-interface is to use [12] while Faulkner[11] has a slightly different description and refers to usability as how satisfying the interfaceis to the users and how comfortable they are using it. The International Organization forStandardization (ISO) defines Usability as follows in ISO DIS 9241-11 [2]:

2.3. Usability 7

Figure 2.1: The usability framework. Adapted from [2]

“The extent to which a product can be used by specified users to achieve specified goalswith effectiveness, efficiency and satisfaction in a specified context of use”

In their definition, ‘effectiveness’ means whether or not the user successfully can carryout the intended tasks. The time of accomplishment is not regarded in this definition incontrast to an early definition by Shackles who included both speed and performance in‘effectiveness’. That is, a system was considered effective only if the users were able toperform the intended tasks within specified time limits [11].

However, in ISO’s definition one dimension of time is included in ‘efficiency’. If one shouldcompare two systems and one of them is faster than the other in performing the task, thenthe faster system is more efficient. Shackle make no distinction between ‘effectiveness’ and‘efficiency’, instead he defines usability in terms of ‘effectiveness’, ‘learnability’, ‘flexibility’and ‘attitude’ [11].

Bevan [2] points out that a strength of ISO’s definition is the flexibility in that it alsoaccounts for the context of use. The context of use consists of the users, the tasks, and theequipment in use as well as the physical and organizational environment. Figure 2.1 showsthat all these factors may influence the usability of a product.

More recently Nielsen [30] identified five core components that defines usability, theseare:

– Learnability. How fast a user can understand the GUI and accomplish the intendedtasks the first time a user encounters the design

– Efficiency. The improvement in time to accomplish tasks when a user has learned theGUI


– Memorability. How fast a user can reestablish proficiency when the user returns tothe design after some time of not using it

– Errors. The amount and severity of the errors a user encounters as well as how fastthe user can recover from them

– Satisfaction. How pleasant it is to use the design

Nielsen disregards ISO’s broad description of context of use. Instead he focuses moreon the system and the interface itself. Although he accounts for some context of use in thesense that he argues that in order to improve usability, one should evaluate new designswith a number of actual users.

Preece et al. [34] complements Nielsen’s list with utility, meaning how well the systemprovides the right functionality according to what the users need or want to do.

2.4 Designing a search interface

It is hard to define the characteristics of a good UI for an information seeking system. Itis highly dependent of the kind of answers users are looking for. A SUI must in somesense try to adapt parts of the UI, such as the presentation of search results, based on thequestion at hand. That is, different types of questions might benefit by formatting anddisplaying the search results with an appropriate representation. An example of this is aquestion regarding the current weather; this kind of question could benefit from a graphicalrepresentation instead of a text-based representation.

Typically, web based search engines must often be able to handle a wide range of querytypes. They must support both known-item search and searches that aim to answer diffuseand complex problems [13]. Some example queries could be:

– How is the weather in Stockholm right now?

– What are some good examples of graphical design?

– What are some promising untried treatments of breast cancer?

Lee at al. [23] highlights the complexity of the definition for known-item search. Anissue that complicates the concept of known-item search is that one could discuss the amountof knowledge the user must have regarding the known-item. Typically, researchers tend toarticulate their own definition of the concept as well, to suit their particular research context.The definition used in this thesis is formulated by Kim and Allen [18]:

“...to find a piece of information that is known to exist and to give a specific answer tothe question...”

However, it is not enough to just support these different types of questions, the systemmust also be able to produce and present a result set that is relevant to the users’ queryand information needs [13]. Moreover, users with varying levels of computer skills will mostcertainly use the SUI differently; hence the system UI must support both expert-users aswell as novices [14].

A SUI should be a tool that supports the users during the entire search process. Itshould be able to help users to concretize the information need, formulating the queries,help users understand the search results as well as showing the status and progress of theinformation seeking effort [12].

2.4. Designing a search interface 9

When discussing information retrieval systems, two terms are frequently used to char-acterize the quality of the search results, these are: precision and recall. The definition ofthese terms is [37]:

– Precision. The ratio of relevant documents that “should” have been retrieved inrelation to the total number of retrieved documents.

– Recall. The ratio of relevant documents retrieved in relation to all relevant documentsin the database.

Typically, using phrased queries increases the precision at the expense of recall. That is, asearch for “air pollution” is likely to generate a result set with less irrelevant documents thana search for “air” and “pollution”. However, the phrased query might overlook importantresults such as documents containing information regarding air quality or other topics relatedto air pollution [37].

One difficult decision a SUI designer has to make is how much automation the interfaceshould provide. An interface might have multiple levels of automation at the same timedepending on the complexity of the interface [1]. Bates [1] highlights that much researchis focused on how to develop “optimal” information retrieval systems with completely au-tomated search functionality. However, she argues that not all users would appreciate thiskind of system since some users want to do and direct their own search. Users tend to wantthe speed and power of an automated system, but they want to feel in control of the system.

This is supported by a study by Koenemann and Belkin [19] on how users experienced aninformation retrieval system with different levels of relevance feedback on novice searchers.The results showed that the users achieved better results when they had more control overthe queries than the automated system. Bates [1] stresses that a designer should thoroughlythink about the question of what capabilities the user should be able to exercise as well aswhat the system should be designed to do.

2.4.1 Understanding the search process

Marchionini [25] describes the information seeking process as a kind of problem solvingactivity. The basis for this claim is that either or both of the information sought (problem) orthe search process (solution path) may be complex. The author presents a model consistingof five basic components. Even though the information seeking task is an iterative processthe model is visualized in a non-linear fashion since there are several possible paths betweenthe definition of a problem and the solution. A rough overview is illustrated in figure 2.2and the five main components from [25] are described briefly below.

– Define problem. This is the initial step in the information seeking task. The definitionof a problem can vary much in complexity, a known-item search is often much simplerthan for example trying to find information that supports an argument. Moreover theproblem definition evolves during the whole problem solving activity as the problemmay branch into different sub-problems.

– Select Source. Regardless if the problem definition is simple or complex, a user mustalways select a source to begin the search process. It is shown that end users primarilytend to start with sources that have proved to be successful for them in the past.

– Articulate problem. In order to perform a successful search the defined problem mustbe articulated. This often means that an end user must formulate a query or determinea starting point for the search in the system.


Figure 2.2: The information seeking task. Adapted from [25]

– Examine results. A system typically responds with a set of hits as a result to a userquery. This list might be represented either graphically or in textual form as for ex-ample a ranked list. Based on the result set the user must make a decision regardingwhich function to call next. This is typically to view more detailed information re-garding one (or more) of the result hits. It is crucial to the examination of results thatfull text information or images can be displayed since primary information is of highimportance for the users.

– Extract information. After the examination of results a user must extract and managerelevant information in order to be able to verify that it can be applied to the definedproblem. Some examples of managed tasks are copy, cut and paste as well as save orprint the information.

The information seeking task presented above is described from a user perspective as itincludes cognitive processes such as problem definition. However, this can be matched toother studies with a more systematic perspective.

Holscher and Stube [14] performed a set of research studies aiming at finding a commonmodel for information retrieval via search engines. The study included two research groups,one with computer experts and one with novice computer users. Their findings showed thattheir model (illustrated in figure 2.3) could be applied to both user research groups.

According to their study there was a probability of 81% that a user in any of the researchgroups would use a search engine to try to find the answer to a question. It also indicatesthat search engine interaction is a somewhat iterative procedure during information seeking(figure 2.3a); this is supported by a 42% probability of query reformulation when a userexamines the page results. The study also highlights some important differences in searchengine usage between the expert and novice users, this is however out of the scope of thisthesis and readers are directed to [14] for further details.

Nielsen [28] advises not to provide users with advanced search functionality since usersoften use it incorrectly. However, Holscher and Strube’s study shows that expert users tendto use advanced search techniques continually [14].

2.4.2 Design guidelines

In 1997, Schniderman et al. [37] found a problem with SUI’s. The UI’s was unnecessarilycomplex and there was no consistency in how different SUI’s were designed. This resulted in


Figure 2.3: The figure shows Holscher and Strube’s model and process with probabilitiescalculated over both research groups. (a) Overview of the information seeking process (b)Close-up of direct interaction with a search engine. Adapted from [14]

a paper with a presentation of eight guidelines that targets SUI usability. These guidelinesalong with a brief explanation are presented below:

– Strive for consistency. Designers should pay careful attention to details such as layout,fonts, terms used in the UI and colors [37]. Hearst [12] stresses that SUI’s often showrich information of complex nature and that changing small details in the design canresult in severe effects on the user-experience.

– Provide shortcuts for skilled users. The design of a SUI should allow skilled usersto work efficiently by implementing shortcuts to relevant functionality [37]. Someexamples of such shortcuts could be to support immediate answers to directed andfocused questions or to provide deeper level site links in the result-list (see section2.4.3 for more details) [12].

– Offer informative feedback. A user should be informed of all aspects regarding thecurrent search, such as displaying what is being searched for and showing the sources[37]. Hearst [12] presents a number of suggestions of valuable features that can beused to increase the user-experience by providing efficient and informative feedback.This includes showing search results immediately, highlight query terms, show queryterms suggestions, allow sorting and support rapid response.

– Design for closure. It should be possible for user to see and understand things such aswhen all results in the list have been viewed or whether or not the search was madeover the entire database [37]. Findings in [12] show how important the graphicaldesign of the UI is. Good designs that appeal to the users seem to raise the overallacceptance of a site as this correlates to their perceptions of the UI quality and theuser satisfaction. Users also tend to persist longer in search process in an appealinginterface [12]. However, a good graphical design is not enough the interaction design isalso very important [34] and the UI should support the users’ actions and give valuablefeedback [12].


– Offer simple error handling. A user should be able to easily update their search termsand parameters. If an error occurs it is important to present a concise and easyto understand error message with as little technical details as possible [37]. A fewsuggestions in order to improve the error handling are to not display empty result setsand to address the vocabulary problem. A more detailed description of these can befound in [12].

– Permit easy reversal of actions. A user should be able to go back to a previous actionor state; this means that every action should be reversible. An easy way to accomplishthis is to offer some kind of history mechanism in the UI that allows a user to re-issuea previously issued query [37].

– Support user control. A good design UI allows the user to always feel in control.Constraints in the UI like forcing a user to enter search parameters in a specific ordershould be avoided. Instead the user should be given the control over the UI and itis the user that should initiate actions [37]. However, Hearst [12] argues that theremust be a balance between user control and system automated action. She mentionstwo important functions that should be considered, even though they might imply atradeoff between opaque system control and transparent user control, when designinga SUI. These are rank ordering and query transformation.

– Reduce short-term memory load. The human short-term memory (STM) is activationbased, that is, information of this kind is easily lost [10]. In order to avoid unnecessaryload of the STM a SUI should support functionality such as recent search history andcompact presentation [37]. Two other examples of functionality that might help reducethe STM load is to suggest search action in an entry form and to integrate navigationand search. This can be accomplished by using for example a watermarked textboxand faceted navigation [12].

Although several researchers have proposed guidelines for search interfaces, Hearst [12]argues that even if guidelines might be helpful in interface design, guidelines alone arenot enough. The main argument for this claim is that guidelines can be hard to follow.Guidelines often overlap and conflict with each other and they seldom provide concretesolutions on how to achieve the guidelines’ goals.

2.4.3 Interface features

This section is not supposed to cover all aspects of Search-user Interface features; instead itaims to highlight some important characteristics that might improve the usability of a SUI.

History management

Research has shown that people tend to have a need to revisit previously viewed information.A study by Cockburn and McKenzie [8] shows that approximately 81% of the URL’s thata user visit is in fact revisits. This behavior is also common in SUI’s where users tend tore-send search queries [12]. History is a common feature in web-browsers, SUI’s can benefitfrom this functionality as well by for example providing a list of the current user’s recentsearch-terms [12].

Komlodi [20] claims that improved history functionality in UI’s can be beneficial forusers, since it would enable users to:


– get an overview of the search process

– better ability to manage tasks

– compare and relate result sets

– collect documents and information

A number of researchers [21] present suggestions of important features in search historyrepresentation. One suggestion is that the history feature should be able to handle paralleltasks as users tend to switch task continuously. Results by Cockburn and Jones [7] showthat the common functionality of history navigational aids in web browsers, that is the backand forward button, mismatch the mental model of the users regarding the session history.A browser’s history is typically stack-based, that is, it does not reflect the user’s actualhistory. If a user hits the back button a number of times and then follows a link from thecurrent page, the browser overwrites the previous forward history with the new target page.

Furthermore the screen real estate is important; a basic idea is that the history function-ality should support the users’ task. Although the history is likely to grow over the session,its screen real estate should not be dominant. It should be constrained properly in relationto other parts of the UI [21].

Providing Suggestions

Hearst [12] discusses several possible solutions for providing users with suggestions to en-hance the SUI experience. Two variants, with different purposes in the search process, arebriefly presented below:

– Query specification in entry forms. There are several ways to provide suggestions tothe users via an entry form. One common approach is to use watermarked text blocks,that is, the text block shows a short description of the type of query or query termsa user is expected to use. When the text block gets focus the watermark disappearsand the user can type instead [12]. Another approach is to give users alternatives ofpossible and popular query refinements based on the typed query. The refinementsuggestions may be presented as a list somewhere in the UI. This kind of feature hasproved to be useful and appreciated by users [41].

– Query specification with dynamic term suggestions. Dynamic term suggestions meanthat users get some kind of real-time alternatives during their query specification.White and Manchionini developed an Interface where the system generates a list ofsuggestions of possible relevant alternatives for the next query term. The suggestionswere based on the users previous query terms and the list updated and displayed whena user pressed the space-key. The results showed that some users thought it was ahuge time saver [42].

Another possible way of displaying suggestions is to use auto-complete functionality,like the SUI Live Search from Microsoft. Auto-complete is a way to show possiblequery term suggestions on a letter-by-letter basis. According to Hearst it is likelythat this kind of functionality will become increasingly useful as the development ofmodern and more responsive term-suggestion features progress [12].


Result presentation

The presentation of search results is very important in a SUI. A few aspects of resultpresentation are discussed below.

– Results per page and search result ordering. Typically, search engines display aboutten hits on the search results per page [12]. In a usability study from Google [26]they found that speed was of great importance to the user. They experimented withdisplaying a list of thirty search hits instead of ten. Analysis showed that there was anaverage of 0.5 second delay when displaying thirty hits. This rather small delay had ahuge impact on the user experience as Google detected a 25% reduction in traffic onthe site over a six-week period.

Studies have shown that the ordering of the search results is crucial. A study byJoachims et al. [17] shows that users tends to primarily explore the first and secondsearch result; beyond that there is a significant drop in interest for the user. Nielsen[28] found that people almost never look at the second page of search results. Aconsequence of a user not finding relevant information on the first result page couldresult in the user giving up or having to reformulate the query [12]. According to [28]the latter has proved to be difficult since a query reformulation often leads to evenworse search results.

– Search result presentation. Some of the challenges that search engines faces is thatwords used as search terms can have several diverse meanings [39] and that documentsand web pages often discuss several different topics interchangeably [12].

Paek et al. [32] studied what kind of impact different summary lengths of result hitshave on user performance. Three interface variations were tested, these were:

• Normal view. A simple listing of search results, where a mouse click opens upthe full text document.

• Instant view. A mouse click expands the summary information with additionalsentences, containing query terms, from the full-text document.

• Dynamic view. As long as the mouse hovers over a particular result hit thesummary expands a few words at a time to display more information.

Their study showed that users performed faster and produced more accurate resultsusing the instant view than with the other two.

A common strategy for presenting search results is a common ranked list [39], that is, alist of ranked results. The ranking is based on relevance of the search hit calculated bysome underlying algorithm. However, several studies [39] [6] have shown that one canimprove the user experience by either using clustering or categorization techniques.

Categorization is a technique for ordering search results in a logical and systematicmanner. The basic idea is that one uses pre-defined categories to organize the informa-tion into different groups or topics [6]. A drawback with categorization is that it oftenrequires a high amount of manual work with category assignment in order to success-fully implement categorization [12]. However, Chen and Dumais [6] present a moreautomate solution using an on-the-fly web directory as source for the categorization.Figure 2.4 shows an example of a UI using a categorized search result list.

A problem with assigning documents to a single category is that they often discussmany different topics that may fall under different categories. One way to solve this


Figure 2.4: A categorized result list (left) and a common result list (right) [6].

problem is to use faceted navigation, that is, one can use metadata such as tags inorder to categorize documents into different topics. This would allow a user to findthe same document under several categories relevant to its content [12].

Clustering is a technique that orders results based on internal similarities, such aswords or phrases [12], that is, clustering is a way to organize information based onkeywords in content itself [39]. An advantage with clustering is that it is highlyautomatable and requires little manual work. However, the advantage also has adownside since clustered search results do not have as high accuracy as categorizedresults [12]. Wang and Zhai [39] present a solution where they use query logs from pastsearches in order to cluster the information. Their findings show that their techniqueperforms more accurately on average than ordinary clustering techniques.

Both categorization and clustering outperforms an ordinary ranked list, both in speedand user experience [12].

Other interface features

Hearst [12] presents several features that may increase the user experience of a SUI. A fewof these are shortly presented below:

– Query-term highlighting. A feature that has proved to be very useful for increasingthe user experience in search-result presentation is to highlight the query-terms in theresult hits. The highlighting can be done in several ways, such as reverse video, boldtext or with a colored background. It can also be implemented so it is shown both inthe short summary below the search hit as well as in the complete document that isdisplayed when a user navigates to a specific search hit.

– Sitelinks. A feature that has increased in popularity is to present links to popularsubpages of a search-hit. The subpage-links are often indented directly below the main


search-hit link. This kind of implementation might speed up the users’ navigation tothe subpage presenting the desired information.

– Shortcuts. Some search-engines try not only to find results where the answer to thequery might be, they also try to answer directed and focused questions such as “what’sthe weather in X” directly by displaying for example weather information in the result-list.

– Blended results and media types. Modern search engines find different kind of mediacontent related to the search query. It is common to display diverse content types,such as images, in the search-result list.

– Previews of document content. Recent development in some search engine’s search-result lists include the ability for the user to preview a thumbnail image of the resulttarget, without having to leave the search-result page.

Navigation

Navigation is an important part of a SUI. Nielsen [27] claims that even though interfaces,in his example a website, implement search functionality the UI still needs navigation and astrong sense of structure in order to be usable. During his research he found some importantcharacteristics in order to enhance the user-experience and the use of search; some of theseare:

– Search should be a box. Usability studies [28] have shown that users often look for“the little box where they can type”. When Nielsen updated the search functionalityon his website by exchanging a link with a search-box the search engine use increasedby 91%.

– Search should be easily available on every page. If a user feels lost on a webpage theytend to use the search functionality in order to get on the right track again. Since itis impossible to predict where and when a user will be lost the search functionalityshould be visible and available on every page in the structure so they don’t have tosearch for the search [27].

A common navigational aid in UI’s is breadcrumbs. These are often thought of as asecondary navigational feature, although its popularity increases. Critics argue that bread-crumbs have the disadvantage of taking up screen real-estate; however, they bring severaladvantages to a UI. Some advantages are that they show the current location, they allowusers to navigate to higher navigational levels with one click and they take up very smallamount of screen real-estate compared to many other navigational aids [31].

A study by Rogers and Chapparo [35] indicates that users tend to get a better under-standing of a website’s structure if the site in question implements breadcrumbs. Instone[15] identifies three different types of breadcrumbs:

– Location. This type of breadcrumb reflects a page’s relative position in the overallnavigational structure, that is, where the user is.

– Path. This type of breadcrumb reflects the users’ actual path to the current location.This can be useful in dynamically generated pages where a single page can be reachedby different routes.

2.5. Discussion 17

– Attribute. These types of breadcrumb do not reflect either the path or location of apage. However, this is commonly used by e-commerce sites for describing characteris-tics of a product by using meta-data.

According to Nielsen [31] the proper way to implement breadcrumbs is by implementingthe Location-based type. The argument for this is that breadcrumbs should reflect the site’snavigational structure instead of the users’ history.

2.5 Discussion

There is no doubt that search user interface design and development is a very complex task.Many factors of highly different nature must be taken into account in order to succeed. Thetask gets even more complex by the fact that there are large uncertainties, like key factorssuch as the user’s computer skill-level and the different types of queries needed. Thisimplicates that several tough design decisions often must be made based on assumptions.Section 2.4.2 presented a set of guidelines in SUI development; these guidelines will not bediscussed further. Readers interested in details regarding those guidelines are referred tothe sources; these can be found in the reference section.

This section aims at highlighting some important aspects of SUI’s based upon the re-search literature presented in this chapter. All aspects will not be discussed here since thisis a rather large area with many different approaches. This section focuses on importantcharacteristics of SUI’s and should serve as a guide for those looking beyond the generalguidelines presented earlier.

Design and optimize for efficiency. A responsive interface is very important for a gooduser experience, SUI’s is no exception as speed seems to be an important factor for usersduring their information seeking task. A designer should also be aware that small UI changescan result in severe consequences for the business. As previously mentioned, Google [26]became aware of this during one of their user interface experiments, they lost a significantamount of traffic over a short period of time because of a small change in the UI.

Another angle of efficiency is that the design of the SUI should support and guide users tofind the answer to the query at hand. A good design should therefore try to use visualizationtechniques such as categorization or clustering since they often increase the efficiency forthe users. A carefully designed SUI should also have an appealing graphical design; this isnot only a way to increase the acceptance among users [12], one can also use graphics toguide the users during the search process.

Get to know your user task and their domain of interest. Far from all SUI’s have tobe as flexible as web search engines such as Google or Yahoo. The characteristics of theusers’ information need within the domain could highly influence the SUI design. Known-item search queries are enough in many websites and applications and the UI should beadapted to such prerequisites. The presentation of search results for directed queries could(and should) probably be designed much better if a designer adapted the design to theinformation at hand, the user needs and the domain instead of blindly follow guidelines.Many company SUI’s seem to lack this understanding since there is a large gap between theperceptions of the importance of searching compared to their own website’s usability [13].

Always think about the users, not the cool-factor. The basic idea with a SUI is to supportthe users during a very complex task. Even though there are many features that are cool andinnovative from an interactive perspective, designers should always be aware of the users and


think twice regarding the functionality. Features that distract the users and dissipate theirattention should be disregarded while features such as query term highlighting or instantview that clearly benefit the users should be considered in the design process.

Chapter 3

Method

The project was performed using an iterative user-centered approach and this chapter pro-vides a detailed description regarding the methods and technique used during the thesis.First is a section presenting an overview of the user-centered design processes, followed bya few sections that discuss the methods and techniques used in more detail. These are pre-sented in chronological order with respect to the design process for this thesis, starting withtechniques for gathering information, followed by sections for prototyping and evaluation.The chapter concludes with a section regarding details of the implementation.

3.1 Design process overview

Interaction design is about investigating the usage of a product and the problem domain;this is typically done by utilizing a user-centered approach on development. A user-centeredapproach is goal-driven in the sense that the users’ needs form the basis for the developmentprocess [34]. However, interaction design is also about trade-offs. That is, to balance theconflicting requirements and constraints [34]. This is supported by Lowgren and Stolter-man [24] who argue that every design process is unique and somewhat unpredictable. Theauthors’ main argument is that there are three variables that differ from project to project,these are: those responsible for the job, the conditions and constraints regarding time andresources, and the current situation.

Furthermore, they stress that a designer’s primary assignment is to develop a “good-enough” design given the constraints for the project. Designers continually switch betweenoverview and detailed features; they often have an understanding of what should be done andhave a set of creative solutions for the problem at hand. However, they often face a situationthat is both chaotic and concrete. This implies that thinking on various levels simultaneouslyis necessary and a designer cannot rely entirely on a rational model or framework [24].

A general lifecycle model for the interaction design process is presented by Preece et al.[34]. This model identifies four basic activities in user-centered design (see figure 3.1), theseare:

– Identifying needs and establishing requirements. In order to design a system thatsupports the target users, one must know who the users are and how an interactiveproduct could support them best. The users’ needs form a basis for the product’srequirements and the following design and development activities.

19

20 Chapter 3. Method

Figure 3.1: The design process [34]

– Developing alternative designs. During this stage several alternative suggestions andideas that meet the target-users’ requirements are presented. This is the core activityof designing and it can be divided into two separate sub-activities; conceptual designand physical design. A conceptual design includes a conceptual model that focuseson what the product should do, what it should look like as well as how it shouldbehave. The physical design goes more into details of the product and describesdesign attributes such as colors, sounds, menu design and what images to use.

– Building interactive versions of the design. A natural part of user-centered design isto let the users evaluate a set of design alternatives. The development of interactiveprototypes facilitates the collection of feedback since many UI problems can be foundby watching users interact with the prototypes. There are different levels of interactiveprototypes; paper-based prototypes are fast and easy to develop while software-basedprototypes are more complex and time-consuming to develop. However, each type ofprototype serves its own purpose in the design process.

– Evaluating designs. The process of interaction design requires a high level of userinvolvement throughout the entire process. In order to ensure that the users’ needsand requirements are met, the product must be evaluated in various stages during thedevelopment process. The evaluation focus on the users and their tasks, one typicallystudies criteria like the amount of errors, how appealing it is and how well it matchesthe specified user requirements.

Preece et al. [34] present three important key characteristics of the interaction designprocess. These are: focus on users, specific usability criteria, and iteration. The user focusis crucial in user-centered design since their needs form the basis for all stages throughoutthe entire design and development phases.

A set of specific usability criteria should be identified at the beginning of the project.These criteria guide the designer in the process of choosing between ideas, solutions, anddifferent design alternatives. It is important to incorporate an iterative process. This is thekey to the possibility of evaluations on different levels as well as refinements to the designbased on user feedback. A greater understanding of the users and their environment is

3.2. Research 21

gained throughout the entire user-centered process; this is a powerful tool for developing asystem that has high acceptability among the target-users [34].

3.2 Research

A usability expert or interaction designer (henceforth referred to as a designer) often findthemselves in a situation where they need to design a product or a system that is to beused in an area of which they have no prior knowledge. However, a prerequisite for asuccessful design is that the product or system is tailored to fit the intended usage area.Hence, designers need to research the problem domain. This is usually done by collectinginformation regarding the intended users and the operational environment, a necessity forthe development of a system that efficiently helps the users achieve their goals and fulfilltheir needs [11].

According to Preece et al. [34] it is crucial to understand the problem domain or elseusability goals may be overlooked. This can be avoided by clarifying the usability and theuser experience goals. Lowgren and Stolterman [24] argue that it is not unusual that adesigner has to answer questions regarding what can be done and what should be done. Inorder to answer that question the designer must rely on the knowledge gained through theresearch phase. The authors present two approaches of research:

– Exploration. The search for several different alternatives and ideas in order to solvethe problem.

– Investigation. The search for one solution by focusing on the current situation.

Typically, both of these methods are used somewhere in the design process since theyserve two different purposes in the design process. Exploration allows a designer to ex-amine several different aspects and solutions the problem. However, almost every designprocess ends with investigation, since the ultimate goal is to develop a product, system orspecification that focuses on a specific problem [24].

The design process described in Section 3.1 is a basis for the research phase in thisthesis project. Section 3.3 present methods for identifying users’ needs and establishingrequirements. Techniques used to develop alternative and interactive versions of the designare presented in Section 3.4. During the research for this thesis project, the investigationpart consists of observations, user interviews and focus groups sessions. The exploration partof the thesis is a literature study regarding search user interfaces (Chapter 2 - Designingusable Search user interfaces).

3.3 Gathering Data

The foundation of user-centered design is to know the users, their environment and whatthey are trying to do. Usability is not about providing the easiest solution; it is aboutproviding the most appropriate solution for the users and their task at hand [11]. Thefollowing sections present a set of methods that can be used to collect valuable informationfor a forthcoming design.

3.3.1 Contextual inquiry

Contextual inquiry is an ethnographic approach used for design. The method is based oncollaboration between a user and a designer [34]. Beyer and Holtzenblatt [3] introduced


contextual inquiry with the goal to strengthen the relationship between the user and thedesigner. Their model builds on an apprenticeship relation model, that is, the user servesas the master and the designer serves as the apprentice. In order to collect as valuableinformation as possible the apprentice observes the master while performing their naturalworking activities. However, the observation is not passive, instead the master explainswhat they are doing and why, in parallel with performing the activity. The apprentice canat any time interrupt the activity by prompting questions regarding some aspect of the task.

There are many benefits with contextual inquiry [3]; some of these are presented below:

– Doesn’t require teaching ability. Many users lack a teaching background. However,they are often experts on their natural working activities. The master-apprenticerelation allows a designer to get a better understanding of the natural activities in theproblem domain.

– Reveals what matters. Often users are not aware of everything they do. How theyperform an activity can be based on years of training and some sub-tasks are obviousto them due to their experience. Since the designer is allowed to put questions, theuser can be forced to reflect over why they do these automated tasks. This may leadto more efficient workflows in the new design.

– Reveals details. Important characteristics of the users’ activities can be revealed bycontextual inquiry. This is due to the fact that the users perform and explain thetask simultaneously. This prevents the user from generalizing the task to other similartasks.

– Reveals structure. Since users communicate basic underlying strategies and structuresfor the activities by performing them, a designer can get a better understanding forthe different aspects of the activities by observations. If a designer has the opportunityto observe multiple users and instances of the task, the designer can easier form anidea of how one could perform the activities in a different and more efficient way.

– Improves learning. Every event that happens while the designer is present is a startingpoint for discussions of past events. A designer often has a limited time to get anoverview of the problem at hand; by using contextual inquiry a designer can takeadvantage of the user’s previous experience.

The typical form of contextual inquiry is by doing a contextual interview. This is acombination of observation, discussion and reconstruction of past events [34]. Snyder [38]highlights the importance of watching the users in their natural work environment. Sheargues that contextual interviews are a very helpful method to get valuable information forthe design task at hand.

3.3.2 Interviews

Preece et al. [34] describes interviews as a ”conversation with a purpose”. The arrangementof interviews can differ significantly. Which kind of approach and structure an interview ses-sion should have is highly dependent on what kind of questions to be answered, the adoptedparadigm, and the session’s evaluation goals. There are four main types of interviews [34],these are:

– Unstructured. An unstructured interview can generate a lot of useful data since thearrangement is more like a conversation on a specific topic. That is, both parties have

3.3. Gathering Data 23

the opportunity to control the interview session. There is no explicit manuscript forthe topic at hand, instead the questions are open-ended and the interviewee is free toanswer the questions as thoroughly as he or she wishes. If one use an unstructuredinterview it is important to be organized and have a plan for the topics to be covered.

– Structured. The questions in a structured interview are predetermined and closed,like those in a questionnaire. This kind of interview works best if the goals are clearlyunderstood and in order to work well the questions should be specific, short and clearlyworded. All participants get the same questions and the questions often require aprecise answer.

– Semi-structured. A semi-structured interview is a combination of features from bothstructured and unstructured interviews. The interviewer has a script with basic ques-tions for guidance in order to cover the same topic among the participants. There isa mixture of both closed and open-ended questions, and the typical approach is thatthe interviewer starts with preplanned questions and then probe follow-up questionsuntil no new relevant information is presented.

– Group. This kind of interview is based on a group of participants with representativeusers. The arrangement is that the whole group discusses the questions together. Onepopular variant of group interviews is called focus groups, and is discussed in moredetail in section 3.3.4.

3.3.3 Brainstorming

Brainstorming is a technique for idea generation. A recommended group size of a brain-storming session is between five to fifteen people. The basic principle of brainstorming isthat the participants collaboratively generate ideas on a specific topic or problem. One cangenerate ideas either by presenting a new idea or by further developing other participants’ideas. It can be beneficial to have participants with diverse backgrounds or area of expertisein a brainstorming session; one idea can trigger new innovative ideas from participants witha different perspective on the current topic [33].

A prerequisite for a successful session is that one must create a relaxed environmentwhere the participants feel comfortable. There are basically three rules that must be followedduring a session [24], these are:

– No one is allowed to criticize someone else’s ideas

– One should not hold back or hide any thoughts; the goal for the group is to generateas many ideas as possible

– Participants should be encouraged to combine or improve other ideas

At the end of the session there is a need to systematize the results. One way to do thisis to categorize the results collaboratively into suitable categories. During this process oneshould also clear repeated ideas and clarify under-specified notes into more concrete ones.This results in a structured set of ideas [24].

A disadvantage with brainstorming is that the problem is often too complex and cannotbe solved by idea generations alone [33]. Furthermore, there is no evidence that brainstorm-ing generates more or better ideas or solutions than if each participant spent equal timeindividually on the problem [24]. However, the technique can be used to generate ideas ei-ther for sub-parts or to the entire problem as well as give directional hints toward a solution[33].


3.3.4 Focus groups

The purpose of a focus group is to take advantage of the possibility that a group has thecapacity to become more than the sum of its individual parts [22]. A focus group consistsof a selection of people that share certain characteristics, relevant for the topic of interest.The size of a focus group can vary between three to ten people [34], however, Casey andKrueger [22] recommends a group size of six to eight people for best result.

Discussions among the participants form the basis for the session; hence it is crucialwith careful planning of relevant questions before the focus group session is conducted.Some important characteristics of a good questioning route are that the questions are open-ended, that they are sequenced and that they move from general to specific questions. Thatis, one question at a time is discussed and the questions should become more in-depth anddetailed as the session progresses.

A skilled interviewer is in charge during the session, the main responsibility of the inter-viewer is to lead the group to get the answers to the questions of interest. It is importantthat the participants focus on the topic at hand and not discuss irrelevant topics; this canbe constrained by the interviewer. There are primarily three stages during a developmentprocess where one can use focus groups successfully [22], these are:

– Early in the development. The main purpose is to gain a greater understanding re-garding the problem domain. This is achieved by learning how the participants feel,think and talk about the area of interest. Designers can then use this information asa basis for the creation of prototypes of the program or product.

– In the middle of the development. Focus groups in this stage can give designers earlyfeedback regarding if the design is on the right track. Participants are asked to evaluatethe designs as well as to compare and contrast each option, that is, what they like ordislike between the prototypes.

– After release. The primary goal with a focus group in this stage of the developmentprocess is to evaluate the program. Questions regarding how one can improve theproduct and if it achieves the expected results is typically in focus during these sessions.

Several focus group studies can be conducted on a specific topic. This allows for morethan one target group to be studied. The results can then be compared and analyzed amongthe groups in order to get an even better understanding of different aspects of the problemdomain [22].

A disadvantage with focus groups is that it might be hard to find suitable time and placefor the participants to meet. It is also crucial that the interviewer is skillful so that the timeis used efficiently and not wasted on irrelevant topics or issues [34].

3.4 Communicating with prototypes

Prototypes are a good approach to test new ideas for a design as well as to solve and com-municate problems and opportunities within the development team and other stakeholders.The technique has proven successful since it allows a designer to evaluate multiple ideas inan early stage in the design process [38]. A benefit of prototyping is that it is fast and cheapin relation to the development of a fully functional system.

The characteristics of a prototype can differ substantially; a developer may develop aprototype in a programming language while a designer can use sketches instead. Typically,

3.4. Communicating with prototypes 25

prototypes are divided into two different levels depending on how advanced it is and whichtechnique used in the development; low-fidelity and high-fidelity prototypes [34]. Which kindof prototyping method to use depends on what part of the design that is to be evaluated;each method fills its own purpose [38]. Preece et al [34] differentiates design into two broadcategories, these are:

– Conceptual. This kind of design highlights a conceptual model of what the productwill do and how it will behave.

– Physical. The primary focus is on details of the design, such as menu structures, iconsand other graphics.

A design goes through both categories as the iterative design process consists of design,evaluation and redesign phases. Typically, the design evolves on a more detailed level thefurther into the design process one comes [34].

3.4.1 Low-fidelity prototyping

The characteristics of low-fidelity prototypes are that they are clearly a mock-up version ofan interface, that is, they don’t look and behave as the final product. This kind of prototypeis often very limited in scope and they are used early in the design process as conceptualproposals [34]. There are several approaches for this kind of prototype, such as storyboards,interface sketches or interactive paper prototypes [24]. These are described briefly below:

– Interface sketches. Sketching is a technique to accomplish fast results of an idea orconcept for an interface. The sketch illustrates a proposal of how the interface issupposed to look. Despite the simple technique of sketching, it is a powerful tool tocommunicate ideas among team members and to develop visions for the interface [24].An interface sketch can be as simple as a wireframe representation of a system and ascomplex as a screenshot of a real system [38].

– Storyboards. This is a combination of interface sketches and scenarios. That is, aseries of interface sketches form a walkthrough of how the system can be used to solvea specific pre-determined task [34]. An advantage of storyboards is the possibility toshow dynamic properties of the interface and the ability to better communicate howthe interface might be used. However, a disadvantage with storyboards is that it isvery time consuming to revise if one has to make changes to the interface [24].

– Dynamic paper prototypes. This is a somewhat interactive version of a user interface.Paper prototypes are often drawn by hand and a prototype can have different levelsof complexity. They can be as simple as a set of predefined state sketches that theuser could end up in, but they can also incorporate more advanced features such as“real time” scrolling, drop down menus, pop up windows as well as drag and dropfunctionality. Interaction is simulated by the supervisor who acts as a computer andreacts to the user’s action during user tests. A dynamic paper prototype is fast andcheap to develop, however it can still produce valuable information regarding theinterface [38].

3.4.2 Hi-fidelity prototyping

This kind of prototype is digital interactive versions of the interface, in contrast to low-fidelity prototypes that often are paper based. The prototype has the “look and feel” and the


responsiveness of a final implementation [24]. Hi-fidelity prototypes often involve some kindof programming in order to create the interactive features. This makes the prototypes moretime consuming and expensive to develop than their low-fidelity counterparts. However, theaccuracy of user testing increases since the system responds to the user interaction [36].

A hi-fidelity prototype can become very large and complex; sometimes it can be usefulto develop a somewhat limited variant of the prototype. There are primarily two basicapproaches for this [36], these are:

– Vertical. A vertical variant only contains a subset of the prototype’s features andfunctionality; however, this subset is developed and presented with a high level ofdetail.

– Horizontal. A horizontal variant contains a wide range of features and functionality;however, while the prototype offers many features the tradeoff is that the prototypehas a limited level of detail on the information.

Both of these approaches have a limited scope, however, they often provide enoughfunctionality or details to be useful during usability testing and evaluation [36]. Other usesof hi-fidelity prototypes, besides testing technical and interactive features, is selling andcommunicating ideas to other stakeholders [34].

3.5 Evaluation

It is possible to gather rich and detailed information regarding a system by conductingstudies of the actual use in the natural operation environment. This data can then beused in later stages in the design process, such as evaluation and design reiterations [4].Typically users want a system to be easy to learn as well as efficient, effective, safe to useand satisfying.

Evaluation is a crucial part in order to be able to develop a user-friendly system andit aims at finding out whether or not the users like it and understand how to use it [34].Evaluation is a native part of the iterative design process and it should be performed ineach stage of the design [11]. There are commonly two types of evaluation [11], these are:

– Formative evaluation. The main purpose of this type of evaluation is to support thedesign process, that is, to formulate and refine a design. In order to accomplish thisone must work closely with the users to gather their feedback of the system. Typically,this method is a qualitative evaluation which is used during early stages of the designprocess.

– Summative evaluation. This kind of evaluation type is mainly used after the design orparts of the design are complete. Summative evaluation aims at answering questionsregarding the impact, usability and effectiveness of the system. That is, it resultsin measurements regarding the performance improvements (if any) that the designentails.

Both formative and summative evaluations give important measurements of differentaspects of the design. Hence, one should not choose one or the other but rather performboth types of evaluation [11].

Preece et al. [34] introduces the DECIDE framework for evaluation. The basis for theframework is a checklist with the purpose of guiding novice evaluators through the evaluationprocess. The framework includes the following activities:

3.5. Evaluation 27

– Determine the overall goals that the evaluation addresses

– Explore the specific questions to be answered

– Choose the evaluation paradigm and techniques to answer the questions

– Identify the practical issues that must be addressed, such as selecting participants

– Decide how to deal with the ethical issues

– Evaluate, interpret, and present the data

The following subsections gives a short introduction to some methods and techniquesused for evaluation of ideas, concepts and data during this thesis project.

3.5.1 Think-aloud protocol

A designer can get valuable feedback by observing users while they perform a set of tasks inthe system. However, observation alone may result in a loss of crucial information sinceit does not give any insight regarding the users’ thoughts and expectations [11]. Thethink-aloud protocol aims to bridge this gap by encouraging the user to communicate theirthoughts and expectations continuously during the observation. A challenge with the think-aloud protocol is to avoid the silent periods that may occur if a user is unaccustomed oruncomfortable with communicating their thoughts verbally [34]. One way to avoid theproblem is to allow the evaluator to prompt questions to the user during these periods [11].

3.5.2 Quick and dirty evaluation: Asking users

A fast approach to collect early feedback is to use a “quick and dirty” evaluation technique.The basic idea of this method is to meet users under a less formal setting than ordinaryusability tests. Typically, the evaluator can get immediate feedback from the users bypresenting the prototype to them; by watching and talking to users the evaluator can usethe prototype as a basis for further discussion. A strength with this approach is that it cantake place in a casual form anywhere and anytime, while still producing quick and valuablefeedback [34].

3.5.3 Long-table approach

Typically, a large amount of ideas and comments emerge during a focus group session.The long-table approach is a method for systemizing the results into manageable chunks.The basic idea of this method is to collaboratively sort the comments and ideas into thepredefined categories as well as create new categories where needed. The primary reason forthe creation of new categories is that comments and ideas often overlap and regard two ormore different categories. Instead of duplicating the comment, one can create new categorieswhere they belong. After finishing the categorization each category is analyzed individuallybased on frequency, specificity, emotion and extensiveness [22].

3.5.4 Questionnaires

One common method for collecting information from users is to work with questionnaires.The results from a questionnaire are a good source for measuring the users’ attitude andopinion regarding a system [11]. The questions can range from simple binary (yes/no)


questions to Likert-scales and open-ended questions. Each type of question fulfill its ownpurpose; one should carefully decide when to use each question type based on the informationneed [34]. Some disadvantages with questionnaires are the complexity of designing a goodquestionnaire as well as the risk of a time-consuming analysis. However, the strength is thatit can produce a vast amount of useful data [11].

Chapter 4

Accomplishments

This chapter provides a description of how the work was conducted throughout this thesisproject. The different phases of the development are explained in the following subsections.

4.1 Research phase

The primary goal with the research phase was to get a deeper understanding regarding thearea of use and the user needs in order to have a foundation for the conceptual GUI design tothe eX-IFS application. In order to accomplish these goals the research phase was plannedand performed using user-centered techniques and methods. More specifically the researchwas planned to give more insight into the following aspects:

– The target users

– The usage context for the eX-IFS application

– Problem areas in the existing system

The basic idea was that the research phase results would be used in later stages through-out the thesis project. Since the result of the thesis project was meant to serve as a proof-of-concept for a possible future development of a real application, the research was conductedwith the aim that the project initiator could use the research results as a basis for thatdevelopment.

In order to collect the necessary information to the above listed aspects the followingdata-gathering methods was used:

– Naturalistic observations

– Contextual inquiry

– Focus group sessions

The gathered information was summarized and documented for use in later stages ofthe thesis project. The following sections describe the use of the data-gathering methods inmore detail.

29

30 Chapter 4. Accomplishments

4.1.1 Observing users

The goal with this phase was to get a deeper understanding regarding the users and theirwork tasks as well as discussing problems with the existing systems GUI. Another motivationfor this phase was to investigate which kind of information that was relevant to most users.This was considered highly important since the employees at the department have differentuser-roles and a responsibility for different stages of the support process.

A total of 10 observation and interview sessions were conducted with a mean-lengthof approximately 1 hour per user. The observations took place in the users’ natural workenvironment and each observation session was combined with a user interview (see section3.3.2 for details).

The users were encouraged to first explain their work tasks and then to show how theyinteracted with the existing system in order to find the information they needed. Duringthe observation notes and questions were continuously documented for later use.

4.1.2 User interviews

As described in the previous section, the user interview was conducted in conjunction withthe user observations. The selected interview approach was contextual inquiry (see section3.3.1 for a more detailed description). The primary reason for the use of this method wasthat the user observations were performed simultaneously. By watching users in their naturalenvironment “silent information” that would have been lost outside of their workplace couldbe found and documented.

When the users were finished with the interaction demo using the existing system (asdescribed in section 4.1.1) the interview started. Since the users had so different userroles, no pre-determined questions regarding their work-tasks were included in the interview.Instead information collected through the user observations served as a basis for that typeof questions. The questions asked throughout the interview mainly regarded issues with theGUI of the existing system, the lack of functionality to solve their work-tasks, the desiresfor functionality in the eX-IFS prototype as well as general usability questions.

Since the interview took place in the users’ natural environment, the users had thepossibility to show and discuss issues and questions on a deeper level since they couldinteract with the existing system during the interview. During the interviews continuousnotes were taken and they proved to be highly useful during the design phase. A summaryof the interview results can be found in Appendix A.

4.1.3 Focus group sessions

During the research phase two focus group sessions were conducted, one with employee’sat Saab’s after-sales department and one with students from the M.Sc. Programme inInteraction Technology and Design Engineering at Umea University.

The sessions were carefully planned in advance and a Moderator Guide (Appendix B)was developed and used as a structure throughout the sessions. A conference room witha large table and a Whiteboard was booked on each location. Each session was dividedinto two parts with a total duration of approximately 3.5 hours (2x1.5 hour active time)including a coffee break.

Before the actual session started the participants were asked to read a short introductorydocument that briefly described the thesis background, the session structure and the problemat hand. The motivation for this was to give the participants some insight regarding thetopics to be discussed. The focus group session was audio recorded for later analysis.

4.2. Design phase 31

The first part was a brainstorming activity based on usability aspects identified duringthe research phase. This part was the same for both groups and aimed at finding newideas of relevant functionality for the eX-IFS prototype. This part was further divided intosix sub-parts where ideas should be generated around a specific topic, one at a time. TheWhiteboard areas were divided likewise. Before the session started each participant got apencil and a set of Post-It notes. When a participant came up with an idea he or she wroteit down on a Post-It and then placed the note in the column for the topic at hand on theWhiteboard. When the first part was finished all participants discussed and classified theresults into more detailed categories. This served as documentation for later analysis.

The second part differed among the focus groups and consisted of open-ended questionsto allow more in-depth discussions among the participants’ area of specialty. The employeesat Saab discussed questions regarding the existing system, desired functionality and futuredevelopment of the eX-IFS prototype while the students discussed visualization, interactionand usability issues. During this part the Whiteboard area could be used to communicateand discuss ideas among the participants by using text, sketches or other graphics.

The focus group sessions generated many interesting functions and the discussions high-lighted many important issues and aspects for the problem at hand, including how to visu-alize large amount of data and how to design navigation-interaction for this kind of system.

4.2 Design phase

The main goal with the design phase was to generate creative solutions and ideas for a user-friendly GUI that matched the needs of the target group. Several user-centered methodspresented in the research section of this thesis were used in order to accomplish this. Thesemethods are discussed in the following subsections.

4.2.1 Low-fidelity prototypes

During the thesis project two iterations with dynamic paper prototypes and evaluation wereperformed. The motivation for this was to find a suitable overall layout as well as to findmore specific functionality for the final prototype implementation.

Workflow paper-prototypes

The first iteration with low-fidelity prototypes aimed at finding a suitable design thatmatched the workflow of the users’ everyday task. Based on the user observations andinterviews three different design proposals were created. Since the primary goal was to eval-uate how well each design proposal matched the actual workflow instead of evaluating theactual design, little or no consideration was given to properties like color, font and graphicaldesign.

The prototypes were based on paper and drawn by hand, techniques for creating andevaluating paper prototypes were based on [38], some basic interactivity could be simu-lated during user evaluations. A short description of each individual workflow proposal ispresented below (for a more detailed description of each proposal see Appendix D).

– Vertical. The layout was structured in a vertical manner. The search-block andreport-block were found in a Tab-control at the top of the page and the search-resultsor component details were presented directly below the tab-control.


The basic idea with this layout was to separate actions and content. That is, groupingfunctionality where a user has to perform a conscious act such as searching or gener-ating reports from pure component information. An overview of the vertical layoutcan be seen in figure 4.1.

Figure 4.1: Vertical workflow layout

– Horizontal. The layout was divided into three main parts (search, content, report)placed in a horizontal manner. On the left-hand side of the layout the search-blockwas located, the content/search results were located in the middle and the ’generatereport’-block was located on the right-hand side of the screen.

The basic idea with this layout was to support a workflow where a user searches for acomponent, gets the search-results or component details presented in the content areaand then has the possibility to generate a report relevant to the current component.Figure 4.2 shows an overview of the horizontal layout.

– Pop-up. The layout was structured in a way where much of the functionality andinformation was hidden until needed. The component details are always visible in thecenter of the screen. The search-block (located to the left) and generate report-block(located to the right) is hidden until a user clicks the Search or Report “button”, thena drop-down block appears with respective functionality. Detailed information abouta component is shown when a user hovers over the component header.

The basic idea with this layout was to have the possibility to use the screen real-estatemore efficiently and still allow a user to easily access detailed information for thecurrent presented content without requiring a mouse-click. An overview of the pop-uplayout can be seen in figure 4.3.


Figure 4.2: Horizontal workflow layout

Figure 4.3: Pop-up workflow layout


In-depth paper-prototype

Before the development of the in-depth low-fidelity prototype started, the results from theworkflow prototype were summarized and discussed with the project initiator. A decisionregarding which concept to further develop into an in-depth prototype was taken in consentwith the project initiator based on the results of the workflow-prototype evaluation.

The main goal of this iteration was to develop a prototype that had enough details tobe able to find usability problems as well as evaluate if the design suited the users and theirtasks. The design was primarily based on the horizontal workflow concept although someparts were extended and re-designed based on the results from the evaluation of the previousiteration. Figure 4.4 shows an overview of the in-depth paper prototype structure.

Figure 4.4: Wireframe overview of the in-depth paper-prototype layout

Evaluation of paper-prototypes

All evaluation session were voice recorded for later analysis and each session concluded witha short discussion with the respective user to collect additional feedback.

The low-fidelity prototypes were evaluated using scenarios and the think-aloud protocol(see section 3.5.1 for more details about the think-aloud protocol). The scenarios weretask-based, that is, the users got a description of what tasks to perform. In order toaccomplish this they interacted with the prototypes at the same time as they were toldto communicate their thoughts by talking aloud. The evaluator was allowed to probe theuser with questions during the session in order to get a deeper understanding of the user’sthoughts and interaction.

The evaluation of the workflow prototypes aimed at gathering feedback and user opinionsregarding the suggested designs. The primary goal was to investigate which of the prototypes


that the users liked the most and eliminate the other two. In order to accomplish this, usersfilled out a questionnaire after all prototypes were evaluated; some example questions were:

– Which prototype best supported your natural workflow?

– Which prototype offered the best information display?

– Which prototype had the most intuitive interface?

All workflow prototypes were evaluated by three users; more specifically, the same threeusers evaluated all three workflow prototypes. The scenarios for this iteration were developedin a way so the user had to navigate through the interface structure several times in orderto find a suitable workflow.

The evaluation sessions were performed using a variation of the Latin Square design. ALatin Square is an NxN matrix. This implied a 3x3 matrix in this thesis project since therewere three users and three concept prototypes to be evaluated. The main purpose for thisevaluation technique was to minimize the biased effect of evaluation order.

If all participants evaluate the prototypes in the same order, they gain a deeper un-derstanding of the problem domain and knowledge of similar GUI features of previouslyevaluated prototypes. This could result in unfair results. However, evaluating the proto-types in different order for each participant can minimize the biased results and create asolid foundation for future work. A more detailed description of the Latin Square designcan be found in [12].

The evaluation of the in-depth paper prototype was performed with five new users, thatis, the three users from the workflow evaluation session were excluded from this evalua-tion phase. During this iteration another set of scenarios was used. These scenarios weredeveloped to let the users focus on functionality instead of workflow.

The number of users was chosen based on Nielsen’s [29] theory that an evaluation byfive users is enough to find about 85 percent of all usability problems in a prototype. Thesetting was similar to the evaluation of the workflow prototypes, that is, using the talk-aloudprotocol and questionnaires as primary data collection methods. However, the questionnairewas re-designed to better match the prototype scope, that is, the limitations in both func-tionality and scope of the managed information. Most questions were more detailed andfocused on specific part of the interface, typical questions were:

– How intuitive and usable is the interface?

– Was it easy to find and understand the search functionality?

– Was it easy to find the relevant information for a component?

A summary of the evaluation analysis can be found in Appendix I.

4.2.2 High-fidelity prototype

After the two previous iterations an interactive prototype was developed and evaluated. Thegoal with this prototype was to get more accurate feedback regarding the interaction andGUI. The users could actually interact with the prototype for “real” during this iterationand get a feeling of how well it responds to their actions and fulfill their needs. This feedbackwas considered crucial for the future thesis work.


Figure 4.5: First version of High-fidelity prototype

Interactive Adobe Flex prototype

The high-fidelity prototype was primarily based on the results from the evaluation of the in-depth prototype. The interactive prototype was developed in Adobe Flex and ActionScript3. The main motivation for this technique was that the final prototype would be a browser-independent web-solution. The choice of a technique that supported that functionality fromthe start allowed for the possibility to speed up the implementation a certain amount ofcode could be reused and extended for the final prototype.

The interactive prototype had most of the desired functionality except for a PDF-generator; however, the prototype was limited in the sense that it used a small amountof dummy information stored in a XML-file instead of using real data from the database.Figure 4.5 shows the first version of the POC GUI, this version was used during the evalu-ation sessions.

Evaluation of interactive prototype

The evaluation of the interactive prototype was similar to those performed during the low-fidelity evaluations, except that this iteration only regarded the final prototype variant. Theusers used the think-aloud protocol (see section 3.5.1 for further description) to communicatetheir expected result before, during and after interaction with the GUI.

This evaluation was performed on five new users. That is, none of the users had previ-ously evaluated the GUI. The users used a laptop running a webpage with the interactive

4.3. Final implementation 37

prototype embedded. The scenarios for the interactive prototype were a combination of thescenarios used in the workflow prototype and the in-depth prototype. By combining thesescenarios the user had to explore both navigation and functionality to solve tasks in thescenarios.

The session ended with a short discussion and a questionnaire just like in the previousiterations. The questionnaire for the in-depth prototype was re-used in order to be ableto compare the results between the prototypes. The user-feedback was documented bothduring and after the session. When all sessions were finished the results and user-feedbackwere evaluated before the design of the final GUI prototype started.

4.3 Final implementation

The last iteration during the GUI development was a final implementation of the proof-of-concept. This implementation was based on some code from the high-fidelity prototype andhence developed using Adobe Flex 3 and ActionScript 3.

The final GUI was revised after the evaluation of the high-fidelity prototype and somefunctionality was added to resolve some issues found during the previous iteration. Thechanges regarded both navigation and tools to support the search process.

An example of navigational issues was that some users wanted to use the browser’s backand forward buttons for navigation. Since this kind of application lives in the browser’s“Sandbox” this kind of behavior was non-standard for this version of Flex. However in thefinal prototype a variant of the Back-/Forward functionality was added using deep-linkingin Flex.

As for the search-process issues, one desired function was some kind of search history ofthe last searches. This functionality conforms to the theory in chapter 2 (Designing usableSearch-user interfaces) and hence an area showing the user’s last 10 searches was added inthe final proof-of-concept implementation.

Chapter 5

Results

This chapter presents the achieved results during the thesis work as well as the final GUI de-sign. The chapter presents some functionality and solutions in more detail with a descriptionof how they conform to the theory of search interfaces in Chapter 2.

5.1 The eX-IFS GUI

The eX-IFS application targets expert users that need an application that supports themin their everyday activities. This has been the focus throughout this thesis project. TheeX-IFS application work with dummy data and the GUI is based on the users’ needs andwork activities. This implies that it might not be possible to implement a final applicationthat includes every aspect of this POC without performing more research of how to retrievethe information from the corporate database.

The main purpose of the POC is to develop and design an intuitive GUI that supportsthe users’ workflow as well as their search process. Searching is a central part of theireveryday work activities, hence it is an important aspect of this thesis.

The workflow, structure, composition and interactions are the result of the iterative user-centered design process performed during this master’s thesis project. The POC includes:

– Navigation within the component hierarchy tree

– Search for components by reference number or project name

– Report generation with Alive PDF (limitations presented in section 5.2.5)

Worth mentioning is that the eX-IFS RIA application should be embedded in an ordinaryweb page for easy access within the company network. The pictures in this report disregardthe enclosing web page and only focus at the eX-IFS application GUI.

5.1.1 Layout

The users’ main activities are to search for a product, review its detailed information andgenerate a report for future follow-up. The layout consists of three main regions, these are:search, content, and report generation (see figure 5.1). Since the application is web basedsome guidelines targeting web design has been followed for the POC design, one of theseis that the clickable parts are shown as hyperlinks. Note that some parts of the UI have

39

40 Chapter 5. Results

Figure 5.1: Application overview

been colored for clarity only, otherwise no effort has been given to the graphical design asmentioned in chapter 1.

5.1.2 Start page

The start page is presented to the users immediately when starting the application, that is,when the users type in the URL for the eX-IFS application. The users’ natural entry pointin the application is an overview of all delivered projects, that is, the root of the componenthierarchy tree. From the root node a user should be able to traverse the component tree ina top-down approach all the way to the leaf nodes. Figure 5.1 shows the eX-IFS applicationentry point.

The users’ have the possibility to go into a detailed view of each project or component byclicking the object id, that is, the hyperlink (see figure 5.3). This functionality is explainedin more detail in section 5.1.3.

5.1.3 Search results page

The search result page shows the hits for the search query. Each search hit is presentedwith the reference number and a descriptive name in the header, where the reference numberserves as a link to the components detail view. However, in some cases the reference numberand the name is not enough to distinguish two or more components from each other. Tosolve this issue a set of hidden information is available for each component; this informationcan be displayed when needed. Another aspect of this functionality is presented in section5.2.1.

5.1.4 Component details

The users’ need to be able to see certain details of every part in the system, hence, thedetailed information is rendered in the same way for every part regardless of node type.

5.1. The eX-IFS GUI 41

Figure 5.2: Component details overview

That is, no matter if the desired part is a project or an individual component, each parthas the same basic set of detailed information. Henceforth, each node-type is referred to ascomponent. Figure 5.2 shows an overview of the component details information views. Thedetailed information consists of:

– Overview. This is an overview of important basic information regarding the compo-nent. Typically, this information is of “good to know”-character, such as productstatus, End-of-life (EoL) status, and creation date.

– Part of. If an error occurs in a component (or manufactured series of components) itis important to know where this component is located. From this view a user can getinformation regarding which projects as well as which components one level up thetree that includes the current component.

– Consists of. The contrary to the “part of ” is the “consist of ”. This view provides theusers with information regarding the sub-components that the component contains.The list of components is limited to one level down the components tree.

– EoL-components. A very important user task is to investigate if a component or sub-component is EoL. That is, if the estimated life-time of the component has passed.This view displays a list of all sub-components (one level down the component tree)


that is marked as EoL. This view also contains a bar chart over the amount of EoLcomponents over time.

– Documents. This view lists the documents related to the current component. Thiscould for example be documents regarding tests or hardware composition.

– Service/History. Users need to be able to get a quick overview of previous and activeservice errands. This view presents all service orders for the component as well as thecurrent status for each service order. This view also contains some statistical informa-tion regarding the Mean-time between failures (MTBF). The statistics are displayedas a line chart.

Worth noting is that the POC application GUI is based on the user needs, this impliesthat it might not be possible to extract all this data in a final application without moreinformation regarding the database.

5.2 Search interface characteristics

The central activity in eX-IFS application is to search and find information regarding specificcomponents in Saab’s naval products. This section presents some functionality that aims tosupport users during the search process.

5.2.1 Search results

The presentation of search results is important in this kind of system. The users must beable to find the desired component as fast and easy as possible. Some of the features ofthe search result presentation in the POC application that aims to support the users searchprocess is presented below:

– Result set. Each hit in the result list shows the reference number and the name ofthe respective component in the header. This information is the most significantinformation for the users. A user can click on the reference number to navigate to thedetailed view of the component; this can be seen in figure 5.3.

Figure 5.3: Search results presentation

5.2. Search interface characteristics 43

If a performed search results in zero hits, the application will automatically try torefine the query by removing the last character in the query term until the result setis non-empty (see figure 5.4). It then displays the hits of the refined query along witha text that indicates that the query was reformulated.

Figure 5.4: Non-empty result set

– Instant view. If a user performs a search that results in several similar hits, one canget more information regarding each hit by expanding hidden information. The basicidea of this functionality is to support the user in the decision making regarding thechoice of component. Information regarding which information to show in the hiddenpart was found during user interviews and user testing. The instant view (discussedin section 2.4.3) approach simplifies the users’ decision without having to navigatethrough every component before finding the desired one.

– Advanced search. Sometimes a user must use multiple search criteria; hence the GUIis prepared for advanced searching if needed. The advanced search allows the usersto specify criteria such as operating country, technology platform, and project. Theadvanced search facility is hidden by default but can be made visible by the user whennecessary.

Some limitations of these features in the POC application is presented in section 5.2.5.

5.2.2 Search history

The GUI was supplemented with a search history function after the high-fidelity prototypeuser testing. Several users emphasized that it is a common procedure to return to a recentsearch and expressed a desire for this kind of functionality. This resulted in a list of the 10most recent searches (see figure 5.5). Each item in the list is a link to the actual search, thatis, clicking an item in the list will trigger the search functionality with the given parameters.

5.2.3 Search path

Since users navigate up and down the components hierarchy tree it should be possible to seewhere you have been, that is, how you got to the current component. The solution was touse path breadcrumbs. Each time a user clicks a component link in “consist of ”, “part of ”or “EoL”, the current component is reloaded. The clicked item will serve as the new currentcomponent and its detail view will show. Figure 5.6 show the search path breadcrumbs.


Figure 5.5: Historical view over the 10 most recent searches

Figure 5.6: Path Breadcrumbs (within a search for: mp4)

Whenever a user switches component, the search path is updated with the new compo-nent (if it does not already exist in the path), this implies that a user can trace the pathtaken to the current view. One can return to a previous component in the path by clickingan item in the search path. If the user performs a new search through the search box thesearch path will be reset to be able to display the future path within the latest search.

During user testing several users tried to use the browser’s back and forward buttonsto navigate through the previous searches. In the prototype that was evaluated this func-tionality was not implemented. However, after the user testing session an easy version ofdeep-linking was added to the prototype to allow user to navigate with the browser’s buttonsas well.

5.2. Search interface characteristics 45

5.2.4 Search term suggestions

To support the users in their search process, a functionality that supported dynamic searchterm suggestions was implemented. The basis for this functionality was an auto-completesearch box, that is, once a user starts typing in the search box a set of existing results thatmatches the user’s query string is presented. The list is filtered according the query stringand the suggested search terms become more specific as the user formulates the query. Auser can choose to type the whole query string or select an appropriate item from the listof suggested search terms. Figure 5.7 shows the search term suggestion feature in the POCGUI.

Figure 5.7: Search term suggestions

5.2.5 Limitations

Some of the limitations in the POC GUI are presented below:

– Searching. A search in the POC GUI is limited to search terms consisting of a projectname, reference number, or part of a reference number. This limitation is due tothe lack of database connection in the POC GUI; the main reason for this is that thesearch logic should be implemented in the database and not the client application. Theresult of this limitation is that the advanced search functionality is only conceptualin the GUI and all possible scenarios that can produce empty result sets might nothave been covered in the POC GUI. Another limitation is that there is no paginationor search result ranking algorithm in the POC GUI; however, the result sets are oftenquite limited so this limitation was disregarded in the project initialization.

– Search history. The search history is a simple implementation, in the sense that ituses the previous query term to perform a new search. This functionality could beextended to save the state of the previous search, that is, to save the search pathwithin each search in the history list.

– Report generation. Since the POC GUI only deals with a limited set of XML data thegenerated reports use hardcoded dummy data, that is, no real data is used. However,the functionality shows that it is possible to generate real reports by implementingthis functionality properly when one can use real data.


– Functionality. Some features of the POC GUI might be limited in functionality. Thisis due to the fact that it is a POC implementation. One example of a feature withlimited functionality is the autocomplete search box. A user cannot select an itemfrom the search term suggestion list using the keyboard; this is one example of afunctionality that should be implemented in a final application.

As mentioned earlier, the POC is based on the user needs and activities. It might notbe possible to extract all data and information needed to use this kind of functionality in areal setting.

Chapter 6

Discussion and conclusion

This chapter is a summary of the work and results of this thesis project. Important aspectsof the different stages in the process will be discussed in further detail followed by a sectionof suggestions for future work.

6.1 Assignment

The assignment included both usability aspects as well as an implementation of a GUI for theeX-IFS proof-of-concept (POC) application. The main focus throughout this thesis projectwas to create a user-friendly GUI where searching was a central part of the user-interaction.The result of this thesis project (design and implementation) is not a complete application.A more thorough investigation regarding the problem domain and opportunities with thiskind of application should be conducted.

The assignment was very interesting and stimulating. However, there were several chal-lenging parts throughout this thesis and a lot of obstacles were found during the project.Most of them regarded the possibility of completing the implementation phase of the finalPOC application. This is a direct consequence of Saab being a high-technology companywith very high security policies on their computer environment. Several tools, such as We-bORB, would have been highly useful during the implementation but had to be disregardedduring the project because of these security limitations.

Furthermore, other limitations that affected the thesis project was the lack of a pre-specified requirement specification, lack of an entity-relationship diagram, and no pre-studyregarding privileges and setup of the development environment as well as what kind of datathat actually could be retrieved from the existing database. These obstacles had severedirect or indirect implications for the POC implementation and the time-frame for thisthesis project.

6.2 Research and design

The in-depth study, “Designing usable Search User Interfaces” fulfilled its purpose to in-crease the understanding and knowledge of the problem domain. There were several possibleapproaches to the problem domain, one direction could have been to focus on search algo-rithms and the impact they have on search interfaces and user experience. This kind ofinformation could probably be useful both for SUI designers as well as developers.

47

48 Chapter 6. Discussion and conclusion

Both data gathering and data analysis was made by a single individual. This impliesthat the collected data is mostly qualitative. The qualitative research is more sensitive forbiased results than a quantitative method. However, the careful planning of each iterationin the interaction process should limit the possibility for misinterpretation of the results.The easy access and continuous discussions with users from the actual target group shouldalso minimize the risk for misinterpretation of the user experience and usability issues foundduring this thesis project.

More user testing and observation could have been performed; however this was notpossible due to the limited time constraints for this thesis. The user testing sessions duringthis project was done in a research environment, that is, a controlled environment. However,a final implementation of the application should also be evaluated in the users’ natural workenvironment. This could increase the understanding about how users interact with thesystem in the context of use, as well as give valuable information of possible improvementsin the GUI.

6.3 Proof-of-concept implementation

The POC application was planned to give a demonstration regarding how one can design auser interface for this kind of system. There was a desire for a fully functional applicationthat could be used in the everyday operations on the TLS department. However, due toseveral different factors such as delayed development platforms, limited time constraints,complex it-architecture and restricted permissions to the company networks this could notbe achieved.

A limitation of the POC is that it is not integrated to the company database; insteadXML-files serve as data source for the application. In order for the POC to communicatewith the actual data source, that is, the company database, a WebService was required.However, the developed service was not fully compatible with the “Sandboxed” Adobe Fleximplementation; hence a connection could not be established. It should be mentioned thoughthat the POC is prepared for this kind of connection but it requires some changes in theunderlying POC code.

At the time of performing this thesis project Adobe Flex was the most mature RIA-platform. However, the development of other emerging techniques such as Microsoft’s Sil-verlight has significantly improved lately. One should perform a research study to evaluatewhich platform that is most suitable and beneficial for the company as well as the ease ofintegration with their IT-environment.

6.4 Future work

During the research for this thesis project, several possible extensions to this applicationwere identified. It is likely that a structured coordination of these extensions could have avery positive effect on the everyday operation efficiency. An example of such an extension isintegration with other existing systems as well as the ability to gather the information in oneplace. This could be beneficial since other user roles or departments could access the sameinformation simultaneously. A more thorough pre-study that collects different departments’needs with this kind of application should be performed and measured based on businessvalue.

If one wishes to use this kind of application it is recommended that the final imple-mentation is designed and developed in a more object-oriented manner. The focus of this

6.4. Future work 49

thesis project was on usability and GUI design; this implies that no effort was spent onthe underlying architecture, modularity or information flow. Furthermore, the developmentproject should also have the required resources and permissions needed to develop this kindof system.

50 Chapter 6. Discussion and conclusion

Chapter 7

Acknowledgements

I would like to take the opportunity to thank all people that supported me and in some waycontributed to my work throughout this thesis project.

Thanks to all people at Saab Systems for taking time for questions, discussions and user-testing. I would like to direct a special thanks to the project initiator and my supervisorTobias Lindell.

Thanks to my project colleague Mattias “MJ” Johansson, I enjoyed working with youand I appreciate all interesting discussions we had both on-site as well as off-site. Thanksto the people at Umea University, especially those of you that participated during the focusgroup session. A special thanks to my supervisor Lars-Erik Janlert in the Department ofComputer Science at Umea University for supporting me during the thesis work and pointingme in the right direction.

I would also like to thank my family and friends who supported and encouraged meduring the thesis project. I couldn’t have done this without you. Some people that deservespecial thanks are Anders Lundgren and Christian Olander that gave me valuable feedbackon the report. And finally, thanks to Matilda who challenged me to finish this thesis, atlast!

51

52 Chapter 7. Acknowledgements

References

[1] Marcia J. Bates. Where should the person stop and the information search interfacestart? Information Processing and Management, 26(5):575–591, 1990.

[2] Nigel Bevan. Human-computer interaction standards. Proceedings of the 6th Interna-tional Conference on Human Computer Interaction, July 1995.

[3] Hugh R. Beyer and Karen Holtzblatt. Apprenticing with the customer. Communica-tions of the ACM, 38(5):45–52, May 1995.

[4] John M. Carroll. HCI Models, Theories and Frameworks: Toward a multidisciplinaryscience. Morgan Kaufmann Publishers, 2003.

[5] Pew Research Center. Pew internet. http://www.pewinternet.org/Static-Pages/

Trend-Data/Online-Activites-Total.aspx, [2010-03-30].

[6] Hao Chen and Susan Dumais. Bringing order to the web: automatically categorizingsearch results. CHI ’00 Proceedings of the SIGCHI conference on Human factors incomputing systems, pages 145–152, 2000.

[7] Andy Cockburn and Steve Jones. Which way now? analysing and easing inadequaciesin www navigation. International Journal of Human Computer Studies, 45(1):105–129,December 1996.

[8] Andy Cockburn and Bruce McKenzie. What do web users do? an empirical analysisof web use. International Journal of Human-Computer Studies, 54:903–922, 2001.

[9] comScore. Press release: Global search market grows 46 percent in2009. http://www.comscore.com/Press_Events/Press_Releases/2010/1/Global_

Search_Market_Grows_46_Percent_in_2009, [2010-04-01].

[10] Michael W. Eysenck and Mark Keane. Cognitive Psychology: A Student’s Handbook.Psychology Press Ltd., 5 edition, 2005.

[11] Xristine Faulkner. Usability Engineering. Palgrave, 2000.

[12] Marti A. Hearst. Search User Interfaces. Cambridge University Press, Blacksburg,Virginia, 2009.

[13] Marti A. Hearst, Ame Elliott, Jennifer English, Rashmi Sinha, Kirsten Swearingen,and Ka-Ping Yee. Finding the flow in web site search. Communications of ACM,45(9):42–49, September 2002.

53

54 REFERENCES

[14] Christoph Holscher and Gerhard Strube. Web search behavior of internet experts andnewbies. Proceedings of WWW9, 2000.

[15] Keith Instone. Location, path and attribute breadcrumbs. 3rd Annual InformationArchitecture Summit, March 15-17 2002.

[16] ITU. The world in 2010: Ict facts and figures. http://www.itu.int/ITU-D/ict/

material/FactsFigures2010.pdf, [2011-01-27].

[17] Thorsten Joachims, Laura Granka, Bing Pan, Helene Hembrooke, and Geri Gay. Ac-curately interpreting clickthrough data as implicit feedback. SIGIR ’05 Proceedings ofthe 28th annual international ACM SIGIR conference on Research and development ininformation retrieval, pages 154–161, 2005.

[18] Kyung-Sun Kim and Bryce Allen. Cognitive and task influences on web searchingbehavior. Journal of the American Society for Information Science and Technology,53(2):109–119, 2002.

[19] Jurgen Koenemann and Nicholas J. Belkin. A case for interaction: a study of interactiveinformation retrieval behavior and effectiveness. Proceedings of the SIGCHI conferenceon Human factors in computing systems (CHI ’96), pages 205–212, April 13-18 1996.

[20] Anita Komlodi. Search history for user support in information-seeking interfaces. InCHI ’00 extended abstracts on Human factors in computing systems, CHI EA ’00, pages75–76, The Hague, The Netherlands, 2000. ACM.

[21] Anita Komlodi, Dagobert Soergel, and Gary Marchionini. Search histories for usersupport in user interfaces. Journal of the American Society for Information Scienceand Technology, 57(6):803–807, April 2006.

[22] Richard A. Krueger and Mary Anne Casey. Focus Groups: A Practical Guide forApplied Research. Sage Publications, 3 edition, 2000.

[23] Jin Ha Lee, Allen Renear, and Linda Smith. Known-item search: Variations on aconcept. Proceedings of the American Society for Information Science and Technology,43(1):1–17, 2006.

[24] Jonas Lowgren and Erik Stolterman. Design av informationsteknik: Materialet utanegenskaper. Studentlitteratur, 1998.

[25] Gary Marchionini. Interfaces for end-user information seeking. Journal of the AmericanSociety for Information Science, 43(2):156–163, March 1999.

[26] Marissa Mayer. In search of...a better, faster, stronger web. http://

velocityconference.blip.tv/file/2290442/, [2011-01-12].

[27] Jacob Nielsen. Alertbox - search usability: Search and you may find. http://www.

useit.com/alertbox/9707b.html, [2010-04-03].

[28] Jacob Nielsen. Alertbox - search: Visible and simple. http://www.useit.com/

alertbox/20010513.html, [2010-04-03].

[29] Jacob Nielsen. Alertbox - usability testing with 5 users. http://www.useit.com/

alertbox/20000319.html, [2010-04-04].

REFERENCES 55

[30] Jacob Nielsen. Alertbox - usability: 101. http://www.useit.com/alertbox/

20030825.html, [2010-04-07].

[31] Jacob Nielsen. Alertbox - breadcrumb navigation increasingly useful. http://www.

useit.com/alertbox/breadcrumbs.html, [2010-04-12].

[32] Tim Paek, Susan Dumais, and Ron Logan. Wavelens: a new view onto internet searchresults. CHI ’04 Proceedings of the SIGCHI conference on Human factors in computingsystems, pages 727–734, 2004.

[33] Gerhard Pahl and Wolfgang Beitz. Engineering Design: A systematic approach.Springer-Verlag, 2 edition, 1999.

[34] Jennifer Preece, Yvonne Rogers, and Helen Sharp. Interaction Design: Beyond Human-Computer Interaction. John Wiley and Sons, Inc., 2002.

[35] Bonnie Lida Rogers and Barbara Chaparro. Breadcrumb navigation: Further investi-gation of usage. Usability News, 5(2), August 2003.

[36] Jim Rudd, Ken Stern, and Scott Isensee. Low vs. high-fidelity prototyping debate.Interactions, 3(1):76–85, January 1996.

[37] Ben Shneiderman, Don Byrd, and W. Bruce Croft. Clarifying search: a user interfaceframework for text searches. DLib Magazine, Januari 1997.

[38] Carolyn Snyder. Paper Prototyping: The fast and easy way to design and refine userinterfaces. Elsevier Science USA, 2003.

[39] Xuanhui Wang and ChengXiang Zhai. Learn from web search logs to organize search re-sults. Proceedings of the 30th annual international ACM SIGIR conference on Researchand development in information retrieval, pages 87–94, 2007.

[40] Barry Wellman and Caroline A. Haythornthwaite. The Internet in everyday life. Wiley-Blackwell, 2002.

[41] Ryen W. White, Mikhail Bilenko, and Silviu Cucerzan. Studying the use of populardestinations to enhance web search interaction. Proceedings of the 30th annual interna-tional ACM SIGIR conference on Research and development in information retrieval,pages 159–166, 2007.

[42] Ryen W. White and Gary Marchionini. Examining the effectiveness of real-time queryexpansion. Information Processing and Management, 2006.

56 REFERENCES

Appendix A

Summary of user observationsand interviews

A total of ten user observations and interviews were conducted during the thesis project. Themain purpose with these observations was to investigate the users’ needs and requirements,get an understanding of different user roles and their everyday work tasks, as well as toidentify pros and cons with the existing application. However, since the final result of thisthesis should be a proof-of-concept interface design, the main focus was to identify themost common activities among the user roles. The observations were conducted with usersfrom the target group and they represented seven different user roles, among them projectmanager, configuration manager, system engineer, and hardware analyst. A summary ofthe most important results from these observations are presented below:

Common activities:

– Search for components

– View information regarding the component and its sub-components. Important infor-mation is:

• End-of-life

• Component version

• Part of/Consists of

• Service orders

– Conduct investigations of the component and its sub-components

– Manual summarization of information regarding information, changes, and errors ina component (this is done manually based on a set of printed information of thecomponent)

Problems with the existing application

– The interface contains a tab-control with many tabs without fixed positions for eachrespective tab (the order changes depending on the currently active tab)

57

58 Chapter A. Summary of user observations and interviews

– Some tabs are empty and there is no indication whether or not a particular tab containsany information

– It is not possible to navigate the component-tree upwards; one has to copy-paste thecomponent-ID and perform a new search to be able to go to higher levels in the tree

– The application feedback is useless, for example a search for a product that doesnot exist opens up a new window with bland information instead of an informationmessage

– Searching in some parts of the application is case-sensitive while searching in otherparts of the application is not

– The application lacks consistency regarding the use of right-and-left mouse clicks

– The search functionality is limited since one can only use component-ID for mostsearches

Desired functionality:

– A more flexible search functionality that supports more than searches for component-ID

– Functionality that supports easy traversing of the component-tree

– Better overview of each component

– Integration with other applications or databases (for example FRACAS)

– Easy accessible information regarding End-of-life components (and sub-components)as well as some statistics of the components’ End-of-life status

– The possibility to generate reports regarding the component (the desired reports differamong the different user roles)

Appendix B

Focus group

The purpose of this document is to serve as a guide for the moderator during the focusgroup sessions. The session is divided into two parts, one general and one targeting theparticipants’ area of expertise. There should be a break for about 20-30 minutes betweenthe different parts and some kind of refreshments should be available during this break.

1. Introduction (5-10 minutes)

– Briefly explain the task and the expectations with the focus group session andrun a short demo-session to give the participants insight in the process.

– Handle out a Post-It block, a pen and a paper with background information andfurther instructions of the session structure to each participant. Give a shortintroduction about yourself and introduce the participants to each other.

– The discussions should be kept as focused as possible on the topic at hand. Tryto involve all participants in the discussions. The session will be audio recorded.The recording and the Post-It notes will act as documentations for later analysis.

2. Part 1 (75 minutes) Part one is a Brainstorming session. The goal is to generatenew ideas for functionality and interaction for the eX-IFS prototype.

(a) Searching (10 minutes)

The only possibility for a user to find a system/product/component in the existingsystem is to search on article number. This means you have to use the articlenumber that the desired system/product/component has in order to retrieve itsinformation.

How can the functionality be extended to offer the users a smartersearch-function for large information sets?

(b) Graphic design (10 minutes)

The prototype might be able to present a large amount of data at the same time.However, many users only need certain information and functions depending oftheir user-role.

How can the Graphical User Interface (GUI) be designed to be asusable as possible at the same time that it targets each user/user-roleas precise as possible?

59

60 Chapter B. Focus group

(c) Reporting (10 minutes)

Different user-roles have certain needs regarding reporting. Sometimes the usersneed to continue process the information in for example Microsoft Excel andsometimes they just have to create a report for the archives. The prototypeshould have a set of pre-defined report templates but new and different reporttemplates might be needed in the future both within and outside of the after-salesdepartment.

One idea is to implement some kind of report generator where the users bythemselves can define and specify SQL-queries through a GUI to retrieve theinformation they need. They might also have the possibility to define the graphicsand layout of the report.

What kind of functionality should such a report generator have?

(d) Communication (10 minutes) Users of the eX-IFS prototype may have aneed to communicate with other users based on certain information in a sys-tem/product/component (e.g. contact the person responsible for the electronics).

How can this kind of communication functionality be solved?

(e) Other functionality (10 minutes)

Support-systems that allow the ability to manage systems/products/componentsare very important for many companies today. These kinds of systems often offergreat possibilities for improving service and support errands both internally andexternally.

What functionality would be usable in this kind of systems?

(f) Categorization and discussion (25 minutes) Discuss the generated ideasand re-categorize the results into more specific categories if needed.

3. Part 2 at Saab (75 minutes) The goal with this part is to encourage the participantsto open discussions regarding their area-of-specialty. This group is specialized in thepros and cons with the existing system, what they need and desire/expect from a newsystem.

(a) What are the main problems/issues with today’s system? The goal withthis question is to identify the main problems with the existing system to be ableto avoid them in the future. The main focus should be on information regardingusability issues, functionality issues, informational issues and integration issues.

(b) What kind of functionality would be desired in the prototype? Thegoal with this question is to identify possible features and functionality that havebeen overlooked during user-observations/interviews. The participants should beallowed to discuss freely but the discussions should preferably focus on thingssomewhat relevant to the scope of the prototype.

(c) What level of administration would be preferred in the prototype ora future system? The primary goal with this question is to get a sense ofhow much administration the users are willing to do in order to have an updatedsystem. This includes administering of users and user-roles.

(d) What kind of future developments could be possible or desired for thiskind of system? The goal with this question is to get the participants to think

61

about the future developments of this kind of system. This could for exampleinclude possible integration with other systems, future functionality and so onin order to get a deeper understanding of the possible development needs anddirections. The participants should be encouraged to discuss freely with little orno boundaries although discussions of realistic functionality is preferred.

4. Part 2 at Umea University (75 minutes) The goal with this part is to encouragethe participants to open discussions regarding their area of specialty. This group isspecialized in usability, visualization and development issues.

(a) Information visualization The primary goal with this discussion is to getideas regarding how to visualize the information for the users. The prototypemight have to be able to process and give an overview of large system/product-structures with over 50.000 components. The participants should be able todiscuss ideas freely as long as there is some kind of realistic solution.

(b) Component versioning A component might exist in several different versionslocated in different systems/products. A user might have the ability to keeptrack of both the product version, the version history and where each version islocated. The focus here should be to get ideas regarding how to simplify andvisualize this for the users in a usable manner.

(c) Modality and Information overload If the prototype is proved to be usefuland able to successfully support the users at the after-sales department otherdepartments might be interested of extending the prototype with different func-tionality and information. The goal with this question is to find inspirationregarding how to structure the GUI in order to support different user-roles anddepartments.

62 Chapter B. Focus group

Appendix C

Focus group analysis

During the thesis project two separate focus group sessions were conducted, one at Saab andone at Umea University. The main purpose of the sessions was to benefit from each groupof participants’ area of expertise. Both sessions were successful and gave a lot of valuableinformation for the thesis project. However, the focus group session at Umea Universitywas somewhat limited; this was primarily a consequence of Saab’s restrictions regardingclassified information and the project overall.

The participants in that session couldn’t get enough information regarding the project toget a sense of the details of the project. A result of this was that many innovative ideas thatcould be highly successful in other projects but not in this particular one were presented.One example of this was the photo-match functionality, that is, one takes a picture of acomponent and then uploads it to a server that automatically tries to match the photo toall components stored in the database. Another example is the related search functionalitysuch as ’other also searched for these components’. Both of these ideas could be useful inseveral other projects, however, the scope, restrictions and users’ work activities for thisproject make them more or less useless for the proof-of-concept.

The participants in both sessions were very active and every participant got the chanceto talk. Many interesting discussions occurred and the mediator only interrupted the dis-cussions a few times when they got too much off topic. The most relevant ideas from thefocus group sessions are presented below:

Part 1 (Saab and Umea University):

– Searching for more than component-ID

– Filtering search results and other lists (such as End-of-life components)

– Real-time term suggestions in searching

– Show a short overview of the component to increase the “hit rate” for the correctcomponent. That is, show information that enables the user to distinguish similarcomponents from each other.

– Provide several different report templates that enables users to generate different kindsof reports

– A carefully designed detail-view of the component

63

64 Chapter C. Focus group analysis

– Make it possible to navigate to related information from within a component

– Integrate actions with for example MS Outlook, that is, one could choose to receivealarms or e-mails when a specific component is updated or changed

Part 2 - Shortcomings of the current application as well as how to avoidthem and better support the work activities in the POC-interface (Saab):

It is hard to find information in today’s application, one often has to open up severalinstances of the application and perform several searches in order to find the desired in-formation. This makes it very easy to lose the perception of exactly where you are in theapplication and the work process. The POC implementation should be easy to use and itshould be easy to get an overview of the current application state (where you are in theapplication), where you came from, where you can go and to see the details of each com-ponent. One of the most important functions is that it should be possible to traverse thecomponent-tree, upwards as well as downwards.

The interface used today has low usability, and both users and decision makers arereluctant to use the application since it is so complicated and hard to find the relevantinformation. The everyday work activities at the TLS department are not supported in theapplication and it requires a significant amount of manual operations in order to compileand analyze information into reports. The POC interface should contain functionality thatsupports the user in their work activities. Some examples of this kind of functionalitycould be smart search functionality that does not generate empty result sets as well as thepossibility to generate digital reports directly from the application. This would probablyincrease the service level to the customers as well as significant improvements in department’sefficiency.

Part 2 - Usability, interaction and Visualization (Umea University):

The application interface should be intuitive and easy to use. The interface shouldalso be easy to learn and understand so users that only use the application sporadicallycan remember how to use it. Functionality that supports the users’ work activities shouldbe provided; examples of such functionality are real-time search suggestions and filteringfunctionality.

One could use buttons or hyperlinks to enable the possibility to traverse the component-tree, that is, one can navigate to a sub-/parent component directly from the detailed-viewof the current component. One could also improve the user experience by implementingfiltering functions that allow users to filter out relevant components in a list by providingsignificant parameters for the desired components.

The application should follow general usability guidelines such as providing the user withunderstandable and relevant error messages if an error occurs.

Appendix D

Workflow prototypes(low-fidelity)

This Appendix presents the workflow of each low-fidelity prototype.

The following pages shows how to use each interface in order to find requested informa-tion. A summary of the scenario in the following pictures is∗:

– Search for: “Radarenhet”

– Select the correct unit from the list

– Find information regarding the unit

– Navigate to a related component, that is, a product in the component tree

– Find information regarding the related component

∗ Note that the scenario in the following pictures is not the same scenario as during theevaluation sessions. The primary reason for this is that the following pictures are used onlyto get an overview of the prototype interfaces. The evaluation sessions were more in-depthand complex. The scenarios used during workflow prototype evaluations can be found inAppendix E

65

66 Chapter D. Workflow prototypes (low-fidelity)

Horizontal layout

67

Vertical layout

68 Chapter D. Workflow prototypes (low-fidelity)

Pop-up layout

Appendix E

Workflow prototype user tasks

Moderator guide

The purpose of this moderator guide is to give the users an introduction to the evaluationsession. The information presented below should be read out loud to the user before theevaluation session begins.

The evaluation is based on:

– Think-aloud protocol

– Probing

– Questionnaires (for each prototype version)

– Final survey (comparing the different prototype versions)

– Closing discussions (after each evaluation session)

Important information:

– It is the prototype that is evaluated, not you!

– The evaluation sessions will be voice recorded for future evaluation

– The main objective is to present a conceptual design proposal to get valuable feedbackregarding improvements for future development, not to present a final design

Basic prototype information:

– Based on previous investigation during the thesis project

– The prototype is limited, both regarding information and design

– The prototype is not in scale 1:1

– The interaction is limited and in some cases disregarded

– The response time is slow (due to the manual computer simulation)

Evaluation tasks

69

70 Chapter E. Workflow prototype user tasks

– Task 1 - Main focus: Find article information

Task 1.1: Find the article with id ’RP667700-06’ (main article)

Task 1.1.1: Find the following information: Country of export

Task 1.2: Investigate if there exists EoL-components in the main article

Task 1.2.1: Confirm the EoL-components (if any)

Task 1.2.2: Investigate if the main article contains a EoL-component that exists inother systems as well

Task 1.3: Investigate the MTBF of the main article

Task 1.4: Find the information regarding export restrictions for the main article

Task 1.5: Find a component/system that contains the main article

Task 1.6: Investigate how many ’Servo units’ that the main article contain as well asthe respective POS

– Task 2 - Main focus: Navigate the interface structure

Task 2.1: Investigate how many failures the ’Radar unit’ (RP658900-06) in the mainarticle had during 1999

Task 2.2: Investigate if the current article has any documents

Task 2.3: Find a test regulation protocol for the current article

Task 2.4: Return to the main articles overview page

Task 2.5: Find a system where the main article is included

Task 2.6: Investigate the degree of freedom for:

• Case 1: Prototype == #1: 9LV435 FCS ASMD

• Case 2: Prototype != #1: 9LV435 FCS MK3

– Task 3 - Main focus: Service and reporting

Task 3.1: Find all work orders for 9LV435 FCS MK3

Task 3.2: Investigate if work order 2046 is closed

Task 3.3: Find a ’radar unit’ that is located in Malaysia

Task 3.4: Investigate the content of a product owner report

Task 3.5: Generate a product owner report

Appendix F

Workflow prototype final survey

1. Rank the importance of the following interface features according to youropinion (1=Most important, 6= Least important)

Easy to navigate structure (layout)

Easy access to search field

Easy to generate reports

The possibility to use free-text search

Good, clear and transparent information display

A clear workflow

For the following questions:

Please observe that the order of the prototype list in this document might not reflectthe order in which you evaluated the prototypes.

2. Which of the prototype versions do you perceive as most useful? (1=Most,3=Least)

“Horizontal”

“Vertical”

“Pop-up”

List three good feature of the prototype you ranked as most useful:

1.

2.

3.

List three bad features of the prototype you ranked as most useful:

1.

2.

3.

71

72 Chapter F. Workflow prototype final survey

3. Which of the prototype versions offered the best navigation features? (1=Most,3=Least)

“Horizontal”

“Vertical”

“Pop-up”

Motivate your choice:

4. Which of the prototype versions offered the most transparent informationdisplay? (1=Most, 3=Least)

“Horizontal”

“Vertical”

“Pop-up”


5. Which of the prototype versions offered the best functionality? (1=Most,3=Least)

“Horizontal”

“Vertical”

“Pop-up”


6. Which of the prototype versions offered the best workflow? (1=Most,3=Least)

“Horizontal”

“Vertical”

“Pop-up”


73

7. Which of the prototype versions offered the best layout structure? (1=Most,3=Least)

“Horizontal”

“Vertical”

“Pop-up”


74 Chapter F. Workflow prototype final survey

Appendix G

In-depth paper prototype(low-fidelity)

The in-depth low-fidelity prototype is highly similar in design as the horizontal layout inappendix D. The biggest difference of the two prototypes is that the in-depth versionhandles more information and it goes somewhat deeper into the design and navigationalissues comparing to the workflow version. Because of the similarities in design this appendixonly highlights a few of the design changes made between the evaluation iterations.

75

76 Chapter G. In-depth paper prototype (low-fidelity)

Appendix H

High-fidelity prototype usertasks

Moderator guide

The purpose of this moderator guide is to give the users an introduction to the evaluationsession. The information presented below should be read out loud to the user before theevaluation session begins.

The evaluation is based on:

– Think-aloud protocol

– Probing

– Questionnaires

– Closing discussions (after each evaluation session)

Important information:

– It is the prototype that is evaluated, not you!

– The evaluation sessions will be voice recorded for future evaluation

– The main objective is to present a conceptual design proposal to get valuable feedbackregarding improvements for future development, not to present a final design

Basic prototype information:

– Based on previous investigation during the thesis project

– The prototype is limited, both regarding information and graphical design

– Some interaction features might not be fully implemented

– The response time might not reflect the performance of a final version (due to theproof-of-concept implementation)

77

78 Chapter H. High-fidelity prototype user tasks

Evaluation tasks

1. You heard that some sub-components to a specific article has become End-of-life. Nowyou want to investigate whether or not this is correct as well as which other systemsthat are affected if this is true. You know that the article has a component-ID thatstarts with RP667700 and that it is located in the project 9LV453 FCS MK3E. Usethe interface to perform the task/find the requested information.

2. You choose to return to the Start page and decide to search for the ’Radar unit’ withcomponent-ID: RP141918-03. The interesting information is its sub-components aswell as which systems that contains this component. Use the interface to perform thetask/find the requested information.

3. You discover that that ’Radar unit’ is a part of the article with component-ID:RP667700-06. Suddenly you remember that there have been some problems withthe ’Laser sensor’ in that component. You would like to investigate whether or notthis article has some work orders and if so, if any of them has been marked as closed.Use the interface to perform the task/find the requested information.

4. You receive an e-mail with the information that something is wrong with the ’Servounit’ in the article with component-ID: RP667700-11. This is the second time in ashort period and you would like to find information regarding how many errors thiscomponent has had during the last 3 years and how long time it seems to be betweenthe errors. You feel concerned over the frequent error reports for this component andyou decide to create a report that contains information regarding the amount of errorsand the relation between error and time. Use the interface to perform the task/findthe requested information.

5. One of your colleagues calls you and complains that he can’t find the POS-informationfor the IR-camera in the Director with component-ID: RP667700-06. Find the re-quested information and send it in an e-mail to your colleague. Use the interface toperform the task/find the requested information.

6. Shortly afterwards, you receive another phone call (once again from the same col-league). This time he doesn’t find any test instruction documents for the ’Radar unit’in the same Director as earlier (RP667700-06). You feel a little bit annoyed and in-vestigate whether or not there is some test instruction documents for the componentin question. Use the interface to perform the task/find the requested information.

Appendix I

Summarized prototypeevaluation

The POC interface has been evaluated three times on different iterations of the designprocess, two evaluations using paper and finally one computerized version of the interface.A total of thirteen unique participants participated in the evaluation sessions. During theevaluations the participants ranked the importance of some interface characteristics (seequestion 1 in Appendix F and Appendix J). The results are presented below (starting withmost important):

1. Easy to navigate structure (layout)

2. Good, clear and transparent information display

3. Easy access to search field

4. The possibility to use free-text search

5. A clear workflow

6. Generate reports

In the first evaluation iteration, three different conceptual interface designs were eval-uated with the goal to investigate which interface that best supported the users’ workactivities. The second evaluation was done on an in-depth prototype based on the resultsfrom the previous iteration. Finally, an evaluation of an interactive high-fidelity prototypewas conducted. The interactive interface was based on the results from in-depth paperprototype. A summarized presentation of the results of each evaluation is found below:

Workflow prototypes (3 paper based designs on 3 participants):The evaluation of the workflow prototypes showed that the most appropriate design was

the horizontal layout. Each prototype was individually evaluated using the questionnaire inAppendix J, the evaluation session closed with a final survey that compared the interfaces(Appendix F). Every participant rated the horizontal layout as most useful and the pop-uplayout as least useful. The summarized results for each evaluated part of the interfaces arepresented in table I.1.

Some comments of each prototype interface are presented below:

79

80 Chapter I. Summarized prototype evaluation

Prototype Layout Component Structure End of Life Search Reports Functionality TotalHorizontal 7,27 7,25 7,50 7,58 6,83 7,17 7,27

Vertical 6,80 6,92 7,00 6,92 7,17 7,00 6,97Pop-up 5,87 6,17 6,17 6,67 5,50 6,00 6,06

Table I.1: The scores for each evaluated part for respective prototype version. The maximumscore is 8,00 for each column

– Horizontal. The participants seemed to like that the information was divided andstructured with a simple tab-control. There were several comments for this struc-ture, among them that it allowed easy navigation and that it reduced the amount ofinformation displayed simultaneously on the screen.

One negative comment regarding the interface was that the search and report fieldsclaimed screen real-estate all the time. Another was that there was some confusionregarding which parts that could be expanded and how to do it.

– Vertical. The participants seemed to like that the component information could usemore screen real-estate for presentation and they liked that the information was easilyaccessible; one participant commented that this structure provided a good overviewof the current component since all information is presented at the same page and itwas appreciated that one could directly expand more relevant information regardingthe component.

One interface feature that the participants lacked was a search result amount counterthat represented the number of search hits for a search. One participant highlightedthat it is important to see the number of search hits in order to decide whether or notto refine the search.

– Pop-up. This layout was rated lowest by all participants. They perceived it to be hardto use and that the hidden information was somewhat confusing. One appreciatedfunction was the search path breadcrumbs, two participants stated that it was reallynice to see ’where I came from’. A functionality that got both good and bad reviewswas that the search box and the report box could be made visible on-demand. Oneparticipant liked the idea of pop-up but stated that it could be used to display clarifyingtooltip information instead of the component information. One possible explanationfor the low rating could be that it was difficult to simulate the pop-up functionalityin a fair manner.

One interesting aspect of the workflow prototype evaluation is that the results of thefinal survey differed from the questionnaire results for each prototype interface. Accordingto the final survey the vertical layout was slightly better than the horizontal layout (thepop-up layout still scored significantly worse than the other two prototypes).

One possible explanation for this effect might be that there was some confusion regardingthe naming of the prototypes, more specifically, during the evaluations the prototypes werecalled one, two and three while the survey names them horizontal, vertical and pop-up.Another possible explanation could be a form of bias: the department uses an old system thatshares similarities with the vertical prototype, so it might be possible that some participantconsidered the vertical interface as the most useful since it was more familiar than thehorizontal.

The results from the individual prototype evaluation were considered as more importantthan the final survey. This decision was based on the fact that the evaluation questionnaire

81

for each design was a more in-depth evaluation with higher accuracy than the final survey.After a short discussion with the project initiator it was decided that the future interfacedesigns should be based on the horizontal interface.

In-depth low-fidelity prototype (1 paper based design on 5 participants):The structure of the in-depth prototype was based on results of the previous evalua-

tions, that is, on the horizontal layout proposal. The participants gave positive commentsregarding the layout and structure of the interface. Several participants emphasized thatthe information was tastefully divided into suitable parts due to the tab control. Two func-tions that were well received by the participants were the real-time term suggestions andthe search path breadcrumbs.

Some negative comments regarding the interface were some confusing headings and thedisplay of irrelevant component details in the search result listings. There was also someconfusion regarding the report generation: in order to generate a report the users weresupposed to click either on the report name in the header or on a small thumbnail in thecontent detail view. However, the participant didn’t find this intuitive and suggested to addan ordinary button for report generation instead of clicking the thumbnail.

Interactive high-fidelity prototype v.0 (1 computerized design on 5 partici-pants):

The participants’ positive comments regarding the interactive interface included that ithad good presentation of the information, that the interface was well structured as well aseasy to use. The overall rating score of the prototype interface was 6,80 out of 8,00.

Most of the negative comments regarded graphical design issues, such as small font, lackof colors, unnecessary margins (the evaluated interface had a fixed width of 1024 pixelsand was displayed on a widescreen laptop with high-resolution, hence the margins), andsomewhat unclear marking of clickable links. Other small issues were misplaced information(such as the EoL bar chart that was originally placed under the Service and History tab) aswell as vague formulations of headings and other texts.

However, some bigger issues were identified as well; since the prototype is embedded in aweb browser some of the participants found it confusing that they couldn’t use the browser’sback- and forward button. Two of the participants commented that they missed smartsearch functionality (that is, real-time term suggestions. This functionality was disregardedin version 0 of the interactive prototype, however, this functionality was present duringearlier evaluations) as well as a search history list that presents a set of previously performedsearches.

Interactive high-fidelity prototype v.1 (1 computerized design on 0 partici-pants):

Based on the feedback collected during the high-fidelity evaluation session, some changesto the prototype were made. Unfortunately, this new version of the prototype was neverevaluated due to time constraints. However, most changes were small and tightly coupledto the participants’ feedback. Some changes were:

– Indicate clickable part of the GUI as web hyperlinks (blue and underlined)

– Increased the font size, the size of GUI components as well as the resolution

– Moved information such as the EoL bar chart (from the Service/History tab to theEnd of Life tab)

82 Chapter I. Summarized prototype evaluation

– Added a search history component that displays the 10 most recent searches for eachindividual user

– Added real-time term suggestion functionality to the search box

A more detailed description of the final interface can be found in chapter 5.

Appendix J

GUI evaluation questionnaire

1. Rank the importance of the following interface features according to youropinion (1=Most important, 6= Least important)

Easy to navigate structure (layout)

Easy access to search field

Easy to generate reports

The possibility to use free-text search

Good, clear and transparent information display

A clear workflow

How well did you perceive (according to your opinion) that the prototypefulfilled the following statements?

Instructions: Mark the most appropriate box in each respective scale with an ’X’.

2. General (Layout)

2.1. The interface was well organized

2.2. It was easy to distinguish the different parts of the interface

83

84 Chapter J. GUI evaluation questionnaire

2.3. The interface was easy to understand

2.4. The interface was easy to work with

2.5. It was easy to understand where to find the desired information

2.6. Suggested improvements regarding layout for the prototype

3. Component structure

3.1. It was easy to get an overview of: Part of

3.2. It was easy to get an overview of: Consists of

85

3.3. It was easy to understand how to navigate the component structure

3.4. It was easy to understand which components one had previously visited

3.5. Suggested improvements regarding component structure for the prototype

4. End-of-life (EoL)

4.1. It was easy to get an overview of the EoL sub-components in a specific component

4.2. It was easy to get an overview of other projects that is affected of the EoLcomponent

4.3. Suggested improvements regarding the handling of EoL in the prototype


5. Searchability

5.1. The search field(s) was easy to find

5.2. It was easy to understand how to use the search functionality

5.3. The search results was presented clearly

5.4. The “search help” was useful ∗

5.5. It was easy to pick the correct component from the search results

5.6. Suggested improvements regarding searching in the prototype

87

6. Reporting

6.1. It was easy to get an overview of the content of each respective report type

6.2. It was easy to understand how to generate a report

6.3. Suggested improvements regarding the handling of reports in the prototype

7. Functionality

7.1. The prototype implemented the most necessary functionality in order for me todo my everyday work

7.2. It was easy to understand how to use the different functionality

7.3. Suggested improvements regarding the functionality of the prototype


8. Other

8.1. List three good things of the prototype

1.

2.

3.

8.2. List three bad things with the prototype

1.

2.

3.

8.3. Other comments/improvement suggestions for future work of the prototype

∗ The question regarding “search help” (Question 5.4) was disregarded in the high-fidelityevaluation. The main goal with the question was to get a rating for the term-suggestionfeature. However, due to time-constraints this feature was not implemented before theevaluation sessions.