chapter 12/13: evaluation/decide framework. why evaluate? why: to check that users can use the...
TRANSCRIPT
![Page 1: Chapter 12/13: Evaluation/Decide Framework. Why Evaluate? Why: to check that users can use the product and that they like it. Designers need to check](https://reader036.vdocuments.net/reader036/viewer/2022082816/56649f4e5503460f94c6f2b8/html5/thumbnails/1.jpg)
Chapter 12/13: Evaluation/Decide
Framework
![Page 2: Chapter 12/13: Evaluation/Decide Framework. Why Evaluate? Why: to check that users can use the product and that they like it. Designers need to check](https://reader036.vdocuments.net/reader036/viewer/2022082816/56649f4e5503460f94c6f2b8/html5/thumbnails/2.jpg)
Why Evaluate?
• Why: to check that users can use the product and that they like it.
• Designers need to check that they understand users’ requirements.
• Avoid wasting money
![Page 3: Chapter 12/13: Evaluation/Decide Framework. Why Evaluate? Why: to check that users can use the product and that they like it. Designers need to check](https://reader036.vdocuments.net/reader036/viewer/2022082816/56649f4e5503460f94c6f2b8/html5/thumbnails/3.jpg)
What to Evaluate
• What: a conceptual model, early prototypes of a new system and later, more complete prototypes.
![Page 4: Chapter 12/13: Evaluation/Decide Framework. Why Evaluate? Why: to check that users can use the product and that they like it. Designers need to check](https://reader036.vdocuments.net/reader036/viewer/2022082816/56649f4e5503460f94c6f2b8/html5/thumbnails/4.jpg)
Where to Evaluate
• Where: in natural and laboratory settings.
![Page 5: Chapter 12/13: Evaluation/Decide Framework. Why Evaluate? Why: to check that users can use the product and that they like it. Designers need to check](https://reader036.vdocuments.net/reader036/viewer/2022082816/56649f4e5503460f94c6f2b8/html5/thumbnails/5.jpg)
When to Evaluate
• When: throughout design; finished products can be evaluated to collect information to inform new products.
![Page 6: Chapter 12/13: Evaluation/Decide Framework. Why Evaluate? Why: to check that users can use the product and that they like it. Designers need to check](https://reader036.vdocuments.net/reader036/viewer/2022082816/56649f4e5503460f94c6f2b8/html5/thumbnails/6.jpg)
The language of evaluation
• Analytical evaluation• Controlled experiment• Formative evaluation• Heuristic evaluation• Predictive evaluation
• Summative evaluation• Usability laboratory• User testing
![Page 7: Chapter 12/13: Evaluation/Decide Framework. Why Evaluate? Why: to check that users can use the product and that they like it. Designers need to check](https://reader036.vdocuments.net/reader036/viewer/2022082816/56649f4e5503460f94c6f2b8/html5/thumbnails/7.jpg)
Evaluation approaches
• Usability testing• Field studies• Analytical evaluation
• Combining approaches• Opportunistic evaluations
![Page 8: Chapter 12/13: Evaluation/Decide Framework. Why Evaluate? Why: to check that users can use the product and that they like it. Designers need to check](https://reader036.vdocuments.net/reader036/viewer/2022082816/56649f4e5503460f94c6f2b8/html5/thumbnails/8.jpg)
Characteristics of approachesUsability testing
Field studies
Analytical
Users do task natural not involved
Location controlled natural anywhere
When prototype early prototype
Data quantitative
qualitative
problems
Feed back measures & errors
descriptions
problems
![Page 9: Chapter 12/13: Evaluation/Decide Framework. Why Evaluate? Why: to check that users can use the product and that they like it. Designers need to check](https://reader036.vdocuments.net/reader036/viewer/2022082816/56649f4e5503460f94c6f2b8/html5/thumbnails/9.jpg)
Evaluation approaches and methods
Method Usability testing
Field studies
Analytical
Observing x x
Asking users
x x
Asking experts
x x
Testing x
Modeling x
![Page 10: Chapter 12/13: Evaluation/Decide Framework. Why Evaluate? Why: to check that users can use the product and that they like it. Designers need to check](https://reader036.vdocuments.net/reader036/viewer/2022082816/56649f4e5503460f94c6f2b8/html5/thumbnails/10.jpg)
Six evaluation case studies
• Evaluating early design ideas for a mobile device for rural nurses in India.
• Evaluating cell phones for different markets.• Evaluating affective issues: challenge and engagement
in a collaborative immersive game.• Improving a design: the Hutch World patient support
system.• Multiple methods help ensure good usability: the
Olympic messaging system (OMS).• Evaluating a new kind of interaction: an ambient
system.
![Page 11: Chapter 12/13: Evaluation/Decide Framework. Why Evaluate? Why: to check that users can use the product and that they like it. Designers need to check](https://reader036.vdocuments.net/reader036/viewer/2022082816/56649f4e5503460f94c6f2b8/html5/thumbnails/11.jpg)
Team Extra Credit
Book Exercise in Chapter 12(Interaction Design
Book). Due with Milestone 5.
![Page 12: Chapter 12/13: Evaluation/Decide Framework. Why Evaluate? Why: to check that users can use the product and that they like it. Designers need to check](https://reader036.vdocuments.net/reader036/viewer/2022082816/56649f4e5503460f94c6f2b8/html5/thumbnails/12.jpg)
DECIDE: a framework to guide evaluation
• Determine the goals.• Explore the questions.• Choose the evaluation approach and methods.• Identify the practical issues.• Decide how to deal with the ethical issues.• Evaluate, analyze, interpret and present the
data.
![Page 13: Chapter 12/13: Evaluation/Decide Framework. Why Evaluate? Why: to check that users can use the product and that they like it. Designers need to check](https://reader036.vdocuments.net/reader036/viewer/2022082816/56649f4e5503460f94c6f2b8/html5/thumbnails/13.jpg)
Determine the goals
• What are the high-level goals of the evaluation? • Who wants it and why?• Some examples of goals:
Check to ensure that the final interface is consistent. Investigate how technology affects working practices. Improve the usability of an existing product .
![Page 14: Chapter 12/13: Evaluation/Decide Framework. Why Evaluate? Why: to check that users can use the product and that they like it. Designers need to check](https://reader036.vdocuments.net/reader036/viewer/2022082816/56649f4e5503460f94c6f2b8/html5/thumbnails/14.jpg)
Explore the questions• All evaluations need goals & questions to guide them.• What questions might you ask about the design of a cell
phone?
![Page 15: Chapter 12/13: Evaluation/Decide Framework. Why Evaluate? Why: to check that users can use the product and that they like it. Designers need to check](https://reader036.vdocuments.net/reader036/viewer/2022082816/56649f4e5503460f94c6f2b8/html5/thumbnails/15.jpg)
Choose the evaluation approach & methods
• The evaluation approach influences the methods used, and in turn, how data is collected, analyzed and presented.
• E.g. field studies typically:Involve observation and interviews.Do not involve controlled tests in a laboratory.
Produce qualitative data.
![Page 16: Chapter 12/13: Evaluation/Decide Framework. Why Evaluate? Why: to check that users can use the product and that they like it. Designers need to check](https://reader036.vdocuments.net/reader036/viewer/2022082816/56649f4e5503460f94c6f2b8/html5/thumbnails/16.jpg)
Identify practical issues
For example, how to:
• Select users• Stay on budget• Stay on schedule• Find evaluators• Select equipment
![Page 17: Chapter 12/13: Evaluation/Decide Framework. Why Evaluate? Why: to check that users can use the product and that they like it. Designers need to check](https://reader036.vdocuments.net/reader036/viewer/2022082816/56649f4e5503460f94c6f2b8/html5/thumbnails/17.jpg)
Decide about ethical issues
• Develop an informed consent form
• Participants have a right to:- Know the goals of the study;- Know what will happen to the findings;- Privacy of personal information;- Leave when they wish; - Be treated politely.
![Page 18: Chapter 12/13: Evaluation/Decide Framework. Why Evaluate? Why: to check that users can use the product and that they like it. Designers need to check](https://reader036.vdocuments.net/reader036/viewer/2022082816/56649f4e5503460f94c6f2b8/html5/thumbnails/18.jpg)
Evaluate, interpret & present data
• The approach and methods used influence how data is evaluated, interpreted and presented.
• The following need to be considered:- Reliability: can the study be replicated?- Validity: is it measuring what you expected?- Biases: is the process creating biases?- Scope: can the findings be generalized?- Ecological validity: is the environment influencing the findings? - i.e. Hawthorn effect.