the secret sauce for effective usability testing

Usability Testing

How a Green, Bewildered UX Researcher Uncovered Loads of Valuable Data

Diane Bowen

User Experience Practitioner

@UXDiane

1

Usability Testing

Testing with one user is 100% better than testing with none.

Krug’s first law of usability

@UXDiane

2

Usability Testing

The interview is a powerful method to understand human behavior.

@UXDiane

3

Usability Testing

The interview is a powerful method to understand human behavior.

I had no idea what I was doing.

@UXDiane

4

Usability Testing vs User Testing

Usability testing: “Is this product or service easy to learn and use?”

Participants are users.

User Testing: Finds bugs before the product or service is released.

Participants are QA or Developers

@UXDiane

5

Usability Testing

Usability testing: “Is this product or service easy to learn and use?”

Participants are users.

Discovers the goals, motivations, and behaviors of users.

@UXDiane

6

Interviewing Users by Portigal

•Leave your worldview at thedoor.

•The user is the expert.

•Build Rapport.

@UXDiane

7

Nielsen Norman Group

The best results come

from testing no more

than 5 users and

running as many

small tests as you

can afford.

@UXDiane

8

My First Study

• Interviewed Participants

• Wrote up observations

• Scheduled a meeting

• Made recommendations

@UXDiane

9

My First Study

• Interviewed Participants

• Wrote up observations

• Scheduled a meeting

• Made recommendations

This wasn’t very effective

@UXDiane

10

A Better Way

Ask questions

• Who wants this research?

• Why are we testing this?

• Why is this a business goal?

• What answers would inform the development team?

@UXDiane

11

A Better Way

Ask questions

• What are the key parts to test?

• What does the perfect participant look like?

• When does the Product Owner need results?

@UXDiane

12

A Better Way

• Enlist participants

• Write jargon-free and non-leading interview questions

• Minimize interview questions that have high cognitive load

@UXDiane

13

Observing the User Experience by Kuniavsky

Six phases of an interview:

• Introduction

• Warm-up

• General Issues

• Deep Focus

• Retrospective

• Wrap-up

@UXDiane

14

A Better Way

• Pilot test before holding interviews.

• Schedule interviews and invite the Product Team.

• Schedule debriefing meetings directly following the interview.

• Email the findings to the Product Team.

@UXDiane

15

A Better Way

Webster’s definition of Synthesis:

“The composition or combination of parts or elements to as to form a whole.”

Synthesis is insight from analysis

@UXDiane

16

A Recent Example

@UXDiane

• Take notes during the

interview

• Debrief with the team

immediately after the

interview

• Transcribe notes, one

thought per sticky note.

17

Usability and Synthesis Workshop

@UXDiane

• Place the sticky notes

into a task analysis.

• Identify trouble spots.

18

Sources

• Portigal, Steve. Interviewing Users. Brooklyn. Rosenfeld Media. 2013. Print

• Nielsen, Jakob. “Why You Only Need to Test With Five Users.” Nielsen Norman Group. http://www.nngroup.com/articles/why-you-only-need-to-test-with-5-users/, Accessed 8-26-2015

• Kuniavsky, Mike. Observing the User Experience: A practitioner’s guide to user research. San Francisco. Elsevier. 2003. Print.

@UXDiane

19

Uncover User Insights and Deliver Value

@UXDiane

• Talk with users.

• Invite the Product Team to observe

the interviews.

• Debrief with the Team. Include

everyone’s thoughts.

• Send a synthesized findings doc to

the team to remind them of

their discoveries. Diane BowenUX Practitioner, Covenant Eyes@UXDiane 20