1

SE 204, IES 506 – Human Computer Interaction

Lecture 6: Evaluating Interface Designs

Lecturer: Gazihan Alankuş

20.02.2012

Please look at the end of the presentation for assignments (marked with TODO)

© 2010 Pearson Addison-Wesley. All rights reserved.

Addison Wesley is an imprint of

Designing the User Interface:Strategies for Effective Human-Computer Interaction

Fifth Edition

Ben Shneiderman & Catherine Plaisantin collaboration with

Maxine S. Cohen and Steven M. Jacobs

CHAPTER 4:Evaluating interface Designs

Outline

• Weekly show and tell about interfaces that you use

• Talking about your experience with the homework

• Introduction to evaluating designs

Show and Tell

• What good/bad interfaces have you seen lately?

TCDD

TCDD

Can you guess what kind of beds we have available? It wouldn’t be fun if we just told you now, would it? >:)

Others?

Your experience with the homework

• How did it go? • What were the difficulties? • Have you learned anything? • What would you change next time?

Some hints about future ethnographic observations

• Don’t make up your mind before the user sessions!

• Make sure that most of the requirements come from users’ needs.

• Test your hypotheses with users!

What have we done so far?

• Identify a problem• Identify requirements• Create a prototype

What was left? >Evaluation

IDEAS

CODEDATA

BUILDLEARN

MEASURE

What to do after creating a prototype

• After creating a prototype, we need to learn from it– Evaluate the prototype

• See if it does what we thought it would • If not, see how we can improve it

– Use the prototype as a tool to make more effective ethnographic observation

A sample iteration to start collecting requirements

IDEAS

CODEDATA

BUILDLEARN

MEASURE

There is a problemthat needs to be solved with software

These must be the usersThese seem to be the tasks

We don’t know much, yet.Maybe we should observe them while they are in the tasks

Create an environmentto put users in this situation

Use the situation to learn what is going on

ObserveAsk questions

We saw that this and thathappens during the task

If our software did this, it would help them!

Ethnographic Observation:

Let’s create a paper prototype

What are we going to measure?How are we going to measure it?

What do we need to do next?

• Some questions to answer through measurement– Is what we created any good? – Is it reflecting our intentions? Are we surprised? – Is it usable at all? – Can users use it the way that we assumed? – Does it handle the task that we built it for?

Some options

• You can evaluate it yourself• You can get an expert to evaluate it• You can evaluate it through user tests

Some options

• You can evaluate it yourself• You can get an expert to evaluate it• You can evaluate it through user tests

Evaluating the design by yourself

• You like what you created• You really need to have a critical look• You will probably fail to evaluate it adequately• It’s difficult to let go

– Need wisdom and humility

1-18

© 2010 Pearson Addison-Wesley. All rights reserved.

Expert Reviews• While informal demos to colleagues or customers can

provide some useful feedback, more formal expert reviews have proven to be effective

• Expert reviews entail one-half day to one week effort, although a lengthy training period may sometimes be required to explain the task domain or operational procedures

• There are a variety of expert review methods to chose from: – Heuristic evaluation – Guidelines review – Consistency inspection – Cognitive walkthrough– Metaphors of human thinking – Formal usability inspection

4-18

1-19

© 2010 Pearson Addison-Wesley. All rights reserved.

Expert Reviews (cont.)

• Expert reviews can be scheduled at several points in the development process when experts are available and when the design team is ready for feedback.  

• Different experts tend to find different problems in an interface, so 3-5 expert reviewers can be highly productive, as can complementary usability testing.  

• The dangers with expert reviews are that the experts may not have an adequate understanding of the task domain or user communities.    

• Even experienced expert reviewers have great difficulty knowing how typical users, especially first-time users will really behave.

4-19

Some options

• You can evaluate it yourself• You can get an expert to evaluate it• You can evaluate it through user tests ^^

Usability Testing

• Observing users using the software• Benefits

– Confirms the progress made– Results in recommendations for changes

The emergence of usability testing

• Initially was just an interesting idea• In time, companies understood how important

it is– Evaluative feedback helped designers guide their

work– Managers saw fewer disasters near deadlines– Sped up many projects– Reduced costs

1-23

© 2010 Pearson Addison-Wesley. All rights reserved.

Step-by-Step Usability Guide from http://usability.gov/

4-23

Usability testing with paper prototypes

• You don’t have to develop the actual software and still get an idea of its usage

• You have great flexibility to change things at will, while doing the user tests. It’s more difficult with software prototypes.

• Let’s watch an example!– Usability testing with paper prototypes

• http://www.youtube.com/watch?v=9wQkLthhHKA– Usability testing for the Spore game

• http://www.youtube.com/watch?v=bz4C9XLdYzg

4-24

1-25

© 2010 Pearson Addison-Wesley. All rights reserved.

Usability Testing and Laboratories

4-25

1-26

© 2010 Pearson Addison-Wesley. All rights reserved.

Usability Testing and Laboratories (cont.)• The emergence of usability testing and laboratories since the early

1980s • Usability testing not only sped up many projects but that it produced

dramatic cost savings.

• The movement towards usability testing stimulated the construction of usability laboratories.

• A typical modest usability lab would have two 10 by 10 foot areas, one for the participants to do their work and another, separated by a half-silvered mirror, for the testers and observers

• Participants should be chosen to represent the intended user communities, with attention to – background in computing, experience with the task, motivation,

education, and ability with the natural language used in the interface.

4-26

1-27

© 2010 Pearson Addison-Wesley. All rights reserved.

Usability Testing and Laboratories (cont.)• Participation should always be voluntary, and informed consent

should be obtained. • Professional practice is to ask all subjects to read and sign a

statement like this one: – I have freely volunteered to participate in this experiment. – I have been informed in advance what my task(s) will be and

what procedures will be followed. – I have been given the opportunity to ask questions, and have had

my questions answered to my satisfaction. – I am aware that I have the right to withdraw consent and to

discontinue participation at any time, without prejudice to my future treatment.

– My signature below may be taken as affirmation of all the above statements; it was given prior to my participation in this study.

• Institutional Review Boards (IRB) often governs human subject test process

4-27

IRBs

• Ensures that what you are doing does not harm your users

• Protects your institution from lawsuits• Originated from medical studies

– What could go wrong in a usability study? • (let’s watch an example)

4-28

1-29

© 2010 Pearson Addison-Wesley. All rights reserved.

Usability Testing and Laboratories (cont.)• Videotaping participants performing tasks is often valuable for later

review and for showing designers or managers the problems that users encounter. – Use caution in order to not interfere with participants– Invite users to think aloud (sometimes referred to as concurrent

think aloud) about what they are doing as they are performing the task.

• Many variant forms of usability testing have been tried:– Paper mockups– Discount usability testing– Competitive usability testing– Universal usability testing– Field test and portable labs– Remote usability testing– Can-you-break-this tests

4-29

Steps of a usability test

• Preparation• Introduction• The test• Collecting data through interviews and surveys

• We will dive deeper into these next week

30

TODO NO HOMEWORK!

TODO For IES 506

• Keep sending the reports every week before class

• Give information about the undergraduate students that are helping you– If none, say so.

TODO Preparing for next week

• A future homework will be about using the paper prototypes you already created for usability testing.

• You may want to think about it in the meantime and get prepared mentally.