1 d.j.schianochi2004 tutorial usability…and beyond! understanding usefulness, usability & use...

1 D.J.SchianoCHI2004 Tutorial

Usability…and Beyond!

Understanding Usefulness, Usability & Use

CHI 2004 TutorialApril 2004

Diane J. Schiano<Abbreviated for PARC>


PageIntroduction 15

A Primer on User Exp Research Design 30---Core UE Research Design Principles 37---Key Pragmatic Issues 56

Methods, Measures & More 67---Key Research Design Decisions

68---Methods: Self-Report & Observational 69---Measures & More: Quantitative, Qualitative 128

More On Communicating Results

148

Appendix 155

Table of Contents


This tutorial will provide a general understanding of: Usefulness, usability and use studies: what they are, how and why they are done,

and the kinds of information they yield, with extended examples and resources available in the tutorial notes.

User research design principles, and the pragmatic challenges of attaining validity, reliability, generalizability and robustness in user studies.

The major self-report and observational methods available for user studies, and how to choose and apply them appropriately.

Approaches to dealing with qualitative, quantitative and hybrid data, and to summarizing results and making design recommendations from them

Pragmatic considerations in applying procedures, discussed from logistical, organizational, and personal/privacy perspectives

Also provided: • Guidelines, readings, other resources for conducting studies and evaluating

findings, both in class and in the tutorial notes

• Opportunities for expert and peer feedback on practical problems participants are invited to bring in for consideration during the User Research Design Clinic.

Course Learning Objectives


AbstractDigital products are growing increasingly complex, encompassing interactions among humans as well as between humans and technology. Careful methods are required to understand user experience of these products, and to use this understanding to inform iterative design. The goal of this tutorial is to provide a practical understanding of the principles and procedures used to assess product usefulness, usability and use. Participants are given guidance and grounding in general user research design principles and procedures to aid them in choosing methods, conducting studies, evaluating results, and making recommendations effectively, even under constrained conditions. A principled yet pragmatic approach is advocated, consolidating the best of classic usability engineering and ethnographic methods—and applied appropriately for the research question and context at hand. Useful exercises, extended examples and extensive references and other resources are provided. Finally, attendees are invited to bring actual problems to discuss in the ‘User Research Design Clinic’ for peer and expert feedback.



Understanding…Usefulness, Usability &

Use



Understanding…User Experience!

Focusing on the PrinciplesUnderlying the Procedures


Introduction


• Depends critically on effective User Research.

• “If we build it they will come… NOT!!!”

• What’s the use… of designing products that aren’t…

Useful,

Usable,

Used???!!!

Creating Useful, Usable & Used Products…


Core User Experience Research Concepts

• Usefulness• Why--and how--could the product be useful to

people? Design (& marketing) implications from current practice?

• Usability• How easily--and well--can the product be learned

and used? Implications for re-design?

• Use• How do people actually use the product?

Implications for re-design?


User-Centered Product Research & Design

Usefulness --> Use <----------------------> Usability

From UsabilityNet


Usefulness, Usability, Use in Product Cycle

Usefulness

Usability

Use

Planning +

Feasibility

Requirements

Design

Implementation

Test + Measure

Post-Release


Usefulness, Usability, Use in Product Cycle

Usefulness

Usability Use

From UsabilityNet


We’ll Focus on Research Design Principles

• Primarily the WHYs behind choosing and implementing user research methods.

• For more on HOW-TOs, see…• UsabilityNet (http://www.usabilitynet.org)

• Appendix references & recommendations


UsabilityNet: An Excellent Resource!

• High Quality, Free• How-Tos, Mini-Tutorials• http://www.usabilitynet.org


Core User Experience Research Concepts

• Usefulness• Why--and how--could the product be useful to

people? Design (& marketing) implications from current practice?

• Usability• How easily--and well--can the product be learned

and used? Implications for re-design?

• Use• How do people actually use the product?

Implications for re-design?


• Ethnography (Usefulness & Use)

• Human Factors Engineering (Usability)

Two Classic Approaches to UE Research


The Ethnographic Approach

Traditional Emphases:

• Usefulness & Use (motivations, practice)

• Self-report w/ contextualized observation

• Naturalistic context, no (or low) control

• “Why? How?” questions • Qualitative data & deliverables


The Human Factors Engineering Approach

Traditional Emphases:

• Usability

• Observation (task performance)

• Lab context, high control

• “How often/fast/much?” questions • Quantitative data & deliverables


These Approaches are Now Converging…

• Self-report & observation are complementary

• “Naturalistic” observations are becoming increasingly common (esp. on the Internet)

• Using converging methods is more informative and cost-effective.


These Approaches are Now Converging…

• So it is becoming increasingly important to understand the core principles underlying ALL user experience research.

• And that’s why we’re here!


A Primer on User Research Design


A Primer on User Research Design

‘Science is the elucidation of common sense.’

Francis Bacon (attrib)

‘There are the hard sciences, and then there are the difficult sciences.’

Gregory Bateson


The Art & Science of UE Research Design

The creative use of ….

research principles & pragmatics

…to construct, conduct & communicate research to effectively inform product design.


• Prioritize. Focus on what you want to learn.

• Design your research project using appropriate methods based on:

- Research principles & pragmatic considerations taken together.

• Conduct the research appropriately.

• Analyze & interpret findings responsibly.

- Use caution; qualify as needed.

• Communicate your findings effectively.

Overview of the Research Process


• Methods (What you can do):

Ask < ----------- > Observe (Self-Report) (Behavior)

• Context (How--& where--you do it):

Naturalistic < ----------- > Controlled

• Data analyses & deliverables:

Qualitative < ----------- > Quantitative

Your Key Research Design Decisions


My Advice…

KISS: Keep it simple, s’il vous plait!

>> Focus on what you really need to learn.• Your questions, goals, deliverables

>> Prioritize!• Design principles & pragmatic considerations

>> What evidence would convince YOU?• Use common sense• Be your own best critic. Challenge yourself!


Core Research Design Principles

• Validity

• Reliability

• Generalizability



• Validity (Am I really studying what I think I am?) • aka “internal validity”

• Reliability (Will my findings be repeatable?)• aka “statistical significance”

• Generalizability (Do my findings apply appropriately?)• aka “external validity”


Validity & Reliability



High Reliability, High ValidityConsistent & ON-Target



High Reliability, Low ValidityConsistent but OFF-Target



High Reliability, Low ValidityNOT Consistent & OFF-Target


Generalizability


Generalizability, aka “External Validity”

External Applicability“Throughput” to Related Target(s)



• Validity (Am I really studying what I think I am?) • Confounds & controls, errors & biases• Look for disconfirming evidence

• Reliability (Will my findings be repeatable?)• Statistical significance v chance• Sample size (# participants, observations)

• Generalizability (Will my findings apply appropriately?)• Representativeness (of sample, context)



• Validity

• Reliability

• Generalizability

• These 3 principles may seem simple at first glance, but they are profound…and the issues involved can become quite complex.

• They provide a foundation for evaluating ALL research—qualitative or quantitative, naturalistic or controlled, market research or usability testing.


Example: IRC’s LambdaMOO Project


Goals: Assess “Hype-otheses”, Characterize Community

• Ask (Self-Report) < ----------- > Observe Behavior

Survey, Interviews < ----------- > Logfile Analysis

• Naturalistic < ----------- > Controlled Context

LambdaMOO < ------> Lab?

• Qualitative < ----------- > Quantitative Data

E.g., Transcripts < ----------- > # Hours Logged On

Convergent Methods Approach

LambdaMOO Project Overview


LambdaMOO Project Methods

• Survey (Self-Report)• 1 Week Call upon Login; 581 Respondents

• ~ 30 Questions, Various Formats, Online

• Interviews (Self-Report)• 12+ Real-Life, Long-Term Participants (Many IVR,etc)

• 1.5-2 hr In-Depth, Semi-Structured Interviews & Maps & Follow-ups

• Logging Studies (Naturalistic Observed Behavior)• Who/Where/When @ ~ 1min Intervals, 24 hr/day, ~2wks

• Privacy Respected

• Data on ~ 4,000 Users Obtained (Twice, ~ 6 Mo. Interval)

• And Much More...• Participant observations, attending BayMOO mtgs, comparison

studies, etc.


Q: Addiction to LambdaMOO?

• Self-Report Use Estimates Very High

• Previous research papers, popular books & press:

• 80 hrs/wk “not uncommon”

• Our interview findings not inconsistent

• Our Logfile Observations Differed Greatly

• Mean ~ 8 hrs/wk (with multi-tasking & idle time!)

• Less than 5% users on for 20 or more hrs/wk


LambdaMOO Use Data from Logfiles

Experience Quartile

0.00

0.20

0.40

0.60

0.80

1.00

1.20

1.40

1.60

1.80

HighMedHighMedLowLow

females

males

others

Modal Presenting Gender

Mean=1.13 hrs/day; ~ 8 hrs/wk


Q: Are you convinced by the logfile data? Why or

why not? Can you explain divergent results?

>>Validity(Am I really studying what I think I am?)

>>Reliability (Will my findings be repeatable?)

>>Generalizability (Will my findings apply appropriately?)



Key Pragmatic Issues

• Robustness (How strong is this effect?)

• Impact (How important is this finding?)

• Convergence (More is better!)


Key Pragmatic Issues

• Robustness (How strong is this effect?) • Large in magnitude (effect size)

• Impact (How important is this finding?)• High in priority, severity

• Convergence (More is better!)• Multiple modest methods can be most informative and cost effective.



• Robustness (How strong is this effect?) • Huge effect size (difference): 80 v 8 hrs/wk

• Impact (How important is this finding?)• High potential social import, impact

• Convergence (More is better!)• Self Report & Observation; Qualitative & Quantitative

• Triangulating “why”, “how”, “how often/fast/much” questions from various perspectives.


Other Pragmatic Considerations…

• Complexity of Product/Design/System

• Your Deliverables• Design recommendations? Presentation? Paper?

• Time Frame • Product deadlines, readiness, design cycle

• Cost • Time, money, personnel

• Other Resources & Constraints• Availability of participants, prototypes, tools• Your skills, expertise & interests• Organizational & political priorities• Etc.


Example: LambdaMOO Project

• Extremely Complex System, User Community

• Our Deliverables• Business & design recommendations

• Time-Frame • System complete but evolving; cohort effects

• Cost • High

• Other Resources & Constraints• Research ideal: Many resources, few constraints


Pragmatic Suggestions from UsabilityNet….


1) Given Limited Time & Resources…


2) Given No Direct Access to Users…


3) Given Limited Skill/Expertise…


Methods, Measures & More…








Key Research Design Decisions


Methods


Methods Overview

Self-Report (e.g., Surveys, Interviews)

• Explanations: Meaning, salience, satisfaction• Feelings, opinions, preferences, priorities• Other otherwise unobservables (w/ caution!)

Observations (e.g., Logfile & Clickstream Analyses, Task Performance)

• “Naturalistic” behavior • Task performance


Taxonomy of Common Methods (after Nielsen, 1993)

• Heuristic Evaluation Self-Report (of Experts)

• Performance Measures Observed Behavior

• Verbal Protocols Self-Report

• Observation Observed Behavior

• Questionnaires Self-Report

• Interviews Self-Report

• Focus groups Self-Report (in Groups)

• Logging actual use Observed Behavior

• User feedback Self-Report








Key Research Design Decisions


Self-Report Methods


Self-Report Methods

• Surveys, Questionnaires

• Interviews

• And More…


Surveys, Questionnaires


• What you can do: Ask (for Self-Report) < ----------- > Observe Behavior

• How (& where) you do it:

Naturalistic < ----------- > Controlled Context



Surveys







Surveys


Surveys

What’s wrong with this picture? - - >


01. How often do you use <Feature X>?

02. How good a user of MS Word do you consider yourself?

03. Which version of Word do you usually use?

04. When you need to find out how to do something using < Feature X >, what do you usually do first?

a) Run through the menus c) Read the manual

b) Run through the toolbars d) Access the easy-to-use help index

…

10. How do you feel about < Feature X >?

a) It's one of the best things Microsoft ever did

b) I like it d) I don't like it

c) Indifferent e) I hate it so much I turned it off

11. Why?

Survey


Surveys

• + Broad demographics, bkgnd information,

• + Specific questions, often quantitative

• + Fairly cheap & easy to conduct & analyze

• + Useful pre- & post- task comparisons

• - Limited, not natural context

• - Subject to biases, memory effects


Surveys: General Issues & Guidelines

• Be Brief, Clear, Specific, Consistent & Easy

• Question Design:• Specific > General• Forced-Choice > Agree/Disagree• Offer No Opinon or N/A & Comment options• Use rating scales for measuring intensity

• Ease of Using Surveys Comes At a Price: • Measurement errors & biases. Sampling issues.• Open-ended Qs harder to analyze, but more valid?

• ALWAYS Pilot Test & Iterate on Questions!


Example: LambdaMOO Survey

• Survey

• 1 Week Call upon Login; 581 Respondents

• ~ 30 Questions, Various Formats, Online

• A relatively quick and easy way to ask for a fairly large amount of self-report information from a large population. The survey was structured, and response formats for were designed for ease of coding, analysis and comparison.

• Note: Quantitative survey results are often more valid, useful, and interesting when considered comparatively rather than absolutely


k (Self-Report) < ----------- >

• Ask (Self-Report) < ----------- > Observe Behavior



Example: LambdaMOO Survey


Example: Sociality in LambdaMOO?

• Survey Responses


User Surveys for Design I (From UsablityNet)

Usefulness


User Surveys for Design II


User Surveys for Design III


Subjective Assessment Surveys I

Usability & Use


Subjective Assessment Surveys II


Subjective Assessment Surveys III


Interviews


Interviews

• + Less limited, more naturalistic

• + Greater depth of understanding

• - Difficult to collect & analyze results

• - Subject to biases, memory effects


Interviews: General Issues & Guidelines

• Context choices:• Conversational style or structured interview? • At your facility or theirs?

• ALWAYS use a guide sheet.

• Be prepared! • Consider logistics, how to record data & take notes, AND how to analyze & present results.


Example: Sociality in LambdaMOO?

• Previous reports emphasized the importance of sociality in LambdaMOO .

• “Great good place” >>> Club/pub analogy

• Our survey and logfile data was mixed:

• Survey suggested most time spent socializing.

• Logfile analysis showed most time spent alone!

• Modal # characters in rooms together ~ 1.


Q: Sociality in LambdaMOO?

• Interviews explained how BOTH were correct:

• Most people discussed spending most time:

• SOCIALIZING ALONE!

Using remote messaging

– MOOmail, paging (presaging IM)

Typically (& increasingly) from “home”

– For security & multi- tasking


Interviews I (From UsablityNet)

Usefulness, Usability, Use


Interviews II


More Self-Report Methods…

• See UsabilityNet for a variety of other self-report—and hybrid—methods.

• If you’re interested—and if there’s time--we can discuss some of these methods later in the tutorial.

Observational Methods


Observational Methods

‘You can learn a lot from looking.’

Yogi Bera (attrib.)







Observations


Naturalistic Observations

• Direct • Outside observer “in context”, or• Participant Observer

• Indirect (Recorded) • Video, audio recordings• Clickstream and logfile data

---> “Notes & Quotes”


Ethnographic Naturalistic Observations

not h

yp

oth

esis

driv

en

Fernando’s filing system(from Nardi)


Ethnographic Observations

• Typically, detailed and extended observation of behavior and artifacts in context. Often in conjunction with interviews & participant observation.

• Ethnographic core concepts:• Holism• Natives’ point(s) of view• Natural context• History


Classic Example: Ethnographic Research

• Schiano, Nardi, Gumbrecht & Swartz. (CHI2004 Short paper submission). “Blogging by the Rest of Us”.


• Ask (for Self-Report) < ----------- > Observe Behavior



Example: LambdaMOO Log Observations


LambdaMOO Logfile Studies

• Logged information• State of each object (who/where/when) in system

• Recorded @ ~ 1min intervals, 24 hr/day, ~2wks

• Data on ~ 4,000 users obtained

• Twice, with ~ 6 month interval btwn studies

• Huge, rich database of “objective” information on use patterns in naturalistic context.

• Extensive data re-coding and analysis was required.

• The findings are very compelling.



• Self-Report Use Estimates Very High

• Previous research papers, popular books & press:

• 80 hrs/wk “not uncommon”

• Our interview findings not inconsistent

• Our Logfile Observations Differed Greatly

• Mean ~ 8 hrs/wk (with multi-tasking & idle time!)

• Less than 5% users on for 20 or more hrs/wk

100

D.J.SchianoCHI2004 Tutorial

Experience Quartile

0.00

0.20

0.40

0.60

0.80

1.00

1.20

1.40

1.60

1.80

HighMedHighMedLowLow

females

males

others

Modal Presenting Gender


Mean=1.13 hrs/day; ~ 8 hrs/wk

101


• Convergent methods were very effective; we felt we gained a good understanding of the community.

• Our company decided NOT to invest in MUDs or other online communities…But by interpreting the findings narrowly, missed a major opportunity: IM & chat!

• Schiano (1998). “ Lessons from LambdaMOO”. Presence.

LambdaMOO Project Epilog

102


User Observations (From UsablityNet)

Usefulness, Use

103


User Observations II

104


Usability Tests as Controlled Observations

• Task performance is observed behavior

• Exploratory, assessment, validation & comparison tests (Rubin)

• Highly controlled task, lab context

• Aids direct comparisons, ease of analysis, reliability

• May hinder validity, generalizability

105


Classic Example: Human Factors Research

• Schiano, Ehrlich, Raharja & Sheridan (2000). Face to Interface: Facial Affect in (Hu)Man and Machine”. CHI2000.

106


Brief Example: Online Web Usability Tests

Data: Speed, accuracy…& clickstreams!

107


Performance Testing I (From UsablityNet)

Usability

108


Performance Testing II

Observational Methods: Issues & Guidelines

• Naturalistic v Controlled Context Issues

• Major Data Coding & Analysis Issues• Responsible data reduction required

• - Validity, Generalizability Issues:• Inferring users’ attention, intentions

• Remember, correlation <> causation!

• Never Underestimate:• Privacy/Permissions Issues • Preparedness & Pragmatics

110


Observation Special Topic: Clickstreams

111


• Who is Visiting Your Site• How Many, How Long, How Often

• Paths Taken Through Your Pages

• Page Analyses• Frequency of Use, Time Spent on Each Page

• Entries & Exits• Where Are Users Coming From? Leaving?

• Success?• Transactions, Downloads, Info Viewed

Clickstream Information

112


Example: Netraker Clickstream Data

113


Example: Netraker Clickstream Data

Pogo-stickingwith

Back-button

Start page

Target page

Backtracking from target page - everyone resorted to search

Majority of participants traveled this

path

Page color: Red=long time

Orange=mediumYellow=short

Dead-end

Participants used many navigation

strategies

And search terms

Multiple paths to the same page

Preferred strategy:browsing

FeaturesExample Observations

Link color: Black=link Red=back-

button

114


• Who is Visiting Your Site• How Many, How Long, How Often

• Paths Taken Through Your Pages

• Page Analyses• Frequency of Use, Time Spent on Page

• Entries & Exits• Where Are Users Coming From? Leaving?

• Success?• Transactions, Downloads, Info Viewed

Web “Clickstream” Data: Useful Information

Rosenbloom, 2000?

115


Caveats Re. Clickstream Observations

• Much missing or potentially misleading data…• Same cookie, diff user? Diff address, same user?• Cached content (& back button) issues• Different behaviors of diff browsers, portals, ISPs

• Much is inferred…• Especially users’ intention, attention

• Validity & generalizability issues• Incredibly rich, readily available data • Best in convergence with other methods

• Netraker, Vividence, Enviz, etc.

116


Measures & More…

• Quantitative & “Quantified” Data

117


Quantitative Summary Statistics

• Data-->”How often/fast/much”

• Measures of Central Tendency• Mean (Average value)• Median (Middle value)• Mode (Most common value)

• Measures of Variability • Range (Interval btwn lowest & highest values)• Variance (Sum of squared deviations from mean)• Standard Deviation (Square root of variance)

118


“Quantifying” Qualitative Data

• Recoding “why”, “how” data to assess “How often/fast/much”

• Often compelling, but can be difficult to do, and easily subject to bias.

• Use with caution!

119


Quantitative Analyses

Match measures & analyses to your questions… and what you want to communicate.

• Collapsing, transforming, summarizing data• Graphical representations• Significance tests• T-tests, ANOVAs• Correlations, etc.• Etc.

120


Brief Example: IM Logfile Study

[2000-1208-13:23:30] IDLE: Donna[2000-1208-14:52:59] ACTIVE: Wally[2000-1208-14:53:45] CLE: [Wally @ [Home, 650 123 4567]] moved focusinto TIM with Donna (uid: 2)[2000-1208-14:54:05] CLE: [Donna @ [972 999-1234]] moved focus into TIMwith Wally (uid: 1)[2000-1208-14:54:05] TEXT: [Wally @ [Home, 650 123 4567] => Donna] [Wow,Edith left me a message saying that the ABX thing went well and thatthey want "to go to the next step".][2000-1208-14:54:06] CLE: [Donna @ [972 999-1234]] moved focus out ofTIM with Wally (uid: 1)[2000-1208-14:54:13] TEXT: [Wally @ [Home, 650 123 4567] => Donna][Interesting...][2000-1208-14:54:15] CLE: [Wally @ [Home, 650 123 4567]] moved focus outof TIM with Donna (uid: 2)[2000-1208-14:59:44] IDLE: Wally[2000-1208-15:17:48] CLE: [Donna @ [972 999-1234]] moved focus into TIMwith Wally (uid: 1)[2000-1208-15:17:55] CLE: [Donna @ [972 999-1234]] moved focus out ofTIM with Wally (uid: 1)[2000-1208-15:18:00] ACTIVE: Donna[2000-1208-15:19:13] CLE: [Donna @ [972 999-1234]] moved focus into TIM with Wally (uid: 1)

121


IM Data Coding & Analysis Issues

• Units of Analysis?

• “Chunk”: Chat activity separated by > 5 min idle time• Pairs: Hi v Lo IM Familiarity; Pair Interaction

• Measures • “Objective” Data: Who, Where, When,

How Often/ Fast/ Much?• Interpretive Coding: How? Why?

• Inter-Coder Reliability Issues

• Analyses• Primarily Quantitative

122


CHUNKSTATS: Donna,Wally,2000-1208-2,22,1208-15:19:24,2,1,1,1.00,N/A,22.00,0,idle,84,N/A,1180

******

Opening Negotiation: N / YTrigger: I / O / S / RReference to Previous Conversation in Day: N / YExplict Reference To Prior Communication In Other Medium: N / YSwitch Media: A / E / NInteraction Through Location Field: N / YThird Party Involvement: N / YMisunderstanding: N / YExplicit Interrupt: N / YClosing: N / YContent: W, S, RThreads: N / YTopics: (list them)

Interpretive Coding:Human Judgment

“Objective” Measures

123


Initial Results

124


Initial Results -->Further Coding, Analyses

125


For Further Information…

•Schiano, Ehrlich, Raharja & Sheridan (2000). “Face to Interface: Facial Affect in (Hu)Man and Machine”. CHI2000.

126


Match Your Measures to Your Questions

• Qualitative (“Why?” “How?”)

• Quantitative (“How often/fast/much?”)

• Convergent (Bits of both)

Trying to translate general hypotheses into specific quantitative questions can be very useful. Not all issues lend themselves to quantification. But forthose that do, the precision and ability to make direct comparisons that quantification provides can prove highly useful.

127


But Let’s be Clear…

• Quantification is not a “magic bullet”…even with so-called “objective” measures. Human judgment—your judgment--is always required in conducting research, analyzing & interpreting data…and communicating your findings!

• Numbers can be impressive…or intimidating, depending on your audience, and they are easily misused…even unintentionally. Think carefully about what you’re trying to find out in your research, why, and for whom. Remember that the apparent precision of numerical results may be misleading. In short, be responsible!

128


Measures & More…

• Qualitative Data

129


Qualitative Measures & More…

• Notes & Quotes• Photos, Video clips• Narratives, Summaries• Structured Products to Inform Design

• Personas/User Profiles• Use Cases / Scenarios of Use

• User Advocacy in Design

130


Personas/ User Profiles

131


Example: Personas/User Profiles I

http://ccm.redhat.com/user-centered/personas.html

132


Personas/User Profiles II

133


Personas/User Profiles III

134


Use Cases/ Scenarios of Use

135


Example: Use Cases/ Scenarios of Use

136



137



‘There are lies, damn lies and statistics.’

Mark Twain

138


Medium as Message…

Your products are communicative, persuasive.• Presentations• Papers/reports• Structured products for design (e.g., personas, use

cases)• User advocacy in design meetings

You are making a case with every selection,summary or presentation of results…• Notes & Quotes• Photos, Video clips• Narratives, summaries• Statistical summaries, graphs & tables

139


Bottom Line: Be Responsible, Credible…Useful!

• In Designing & Conducting Research• As we’ve discussed at length

• In Reporting Results

• Refer to research principles in summarizing and presenting results

• Provide access to methods & raw data

• In Making Recommendations• Be judicious and pragmatic• Prioritize by robustness and impact.

140


User-Centered Product Research & Design

Usefulness --> Use <----------------------> Usability

From UsabilityNet

141


Special Discussion Topics

142


User Research Design Clinic

143


Appendix

1 d.j.schianochi2004 tutorial usability…and beyond! understanding usefulness, usability & use...

Documents