rapid research - gotomedia inc

Rapid ResearchFor 2.0 Development

Presented by Kelly Goto kelly@gotomedia.com

"Successful user experiences deliver a firm's value proposition

-- the brand promise -- to customers in the most effective and

appropriate way.”

Karen Donoghue, Built For Use

research methods

TALK ASK WATCH OBSERVE

focus groups interviews usability testing ethnography

Typically 4-9 people 1-on-1 1-on-1 1-on-1 or group

Discussion about critical aspects

Questionnaire

Brainstorming

Users complete representative tasks

Field observation of sociocultural phenomena

Moderator Conducted by Interviewer

Observer/Moderator Observing in natural environment

1-2 hours 1-2 hours 1- 1.5 hours 1 hour- many years

WHAT PEOPLE THINK HOW PEOPLE LIVE

concepttesting

customer interviews

cardsorting

focus groups

expertanalysis

usabilitytesting

contextualinquiry

verification testing

concepttesting

customer interviews

cardsorting

focus groups

expertanalysis

usabilitytesting

Contextualinquiry

verification testing

http://www.slis.indiana.edu/technology/usability_lab.html

• Formal facility• 8-20 carefully screened applicants• Formal test plan• Complex video monitoring (picture in picture) practices, including high-light videos

• Participant's work environment• 3-10 recruited participants meeting criteria• Impartial monitor• Informal test plan• (Simple screen capturing, with voice, using programs like Camtasia or Morae)

http://www.adaptivetechnologies.com

formal informalsemi-formal

• In-house (or remote)• 3-5 Friends, co-workers or family members• Impartial monitor• Informal test plan• No video taping, only notes and observations

rapid research

traditional

alpha release(get it up quick!)

product beta(trial users)

live product(free trial, sign up today)

focus groups(how are we doing?)

iterative

proof of concept(what do they want?)

beta release(do they like it?)

beta2 release(can we simplify it?)

live product(does it meet their needs?)

Rapid usability testing is characterized by rapid cycles of testing between iterations of

prototyping

Characteristics:

+ Informal testing of 5 participants and one floater per day (one or two days of testing scheduled)+ Modifications to prototype (paper or HTML) between testing days (or sometimes during testing, but not in this case)+ Team present for testing in order to observe first hand what is being found+ Raw notes/ rough observations delivered immediately following testing + Informal report (word format) delivered approx. 48-hours following testing

toolkit: essentials

Windows Media Encoder (download.com) – FREE easy to use full-

motion screen capture tool.

CamStudio & Camtasia Player – FREE full-

motion screen capture & player (NOT the usability software)

Participant ID:Date: Start time: End time:

Task Time total

Video highlight at

Notes/Main Observations Quotes Events

1 2:33 1:30 (time

stamp)

User struggles to sign up due to unclearinstructions;

“I don’t understand why I can’t sign in!”

2 3:20 User can not find account information to edit the password; looks in options instead

3 7:45 User can not find support and FAQ section to change his order preferences; gives up after looking everywhere, including the search function

“This is really frustrating! I would stop trying now!”

4 … No problem to find shopping cart “I like the shopping cart”

Legend: Task failed Prompt by moderator Struggle Task solved Insight

Notetaking Format (with macros)

Notetaking Format (with macros)

A classic overview of the testing process – surprisingly straightforward although there is very little mention of the web!

A classic – humorous and practical approach to usability. Steve has created the true ‘must read’ book for teams and stakeholders.

contextual inquiry

idea!

proof-of-concept

alpha-release

usability testing

When to test?

beta release

beta release 2

launch

launch 1

alpha-release

alpha-release

alpha-release

alpha-release

Contextual Inquiry

1 determineaudience 2 develop

interview script

3 observe

64 photo diary 5 follow up compile & report

Sample Project: Technorati

1 determineaudience 2 develop

interview script

3 observe

64 photo diary 5 follow up compile & report

16 participants in 2 citiesthe more specific profiles, the better

2 hours to a full day with notes and photos in context

Ask participants to take photos of daily life over next 3 days to 1 week

Interview 2-hours and review photo diaries

Sample Project: Technorati

Online Survey in 5 cities

Photo Diaries & Contextual Interviews in L.A. & Minneapolis

Interview Summary

Audience Segmentation Grid

Audience Segmentation Grid

Contextual Findings Report

rapid usability testing

Rapid Usability Testing

1 determineaudience 2 develop

test plan 3 recruit

64 conduct testing 5 high-level findings

informal report

6

1 determineaudience 2 develop

test plan 3 recruit

4 conduct testing 5 high-level findings

informal report

Sample Project: Technorati Testing

5 target audience users

5 one-hour testing sessions

Extract high-level findings, 2-4 pages

Observations and recommendations with screenshots, 8-12 pages

develop test plan including task list

define recruiting criteria with client

Recruiting

Confirmation Email

Informal Usability Testing

Participant ID:Date: Start time: End time:

Task Time total

Video highlight at

Notes/Main Observations Quotes Events

1 2:33 1:30 (time

stamp)

User struggles to sign up due to unclearinstructions;

“I don’t understand why I can’t sign in!”

2 3:20 User can not find account information to edit the password; looks in options instead

3 7:45 User can not find support and FAQ section to change his order preferences; gives up after looking everywhere, including the search function

“This is really frustrating! I would stop trying now!”

4 … No problem to find shopping cart “I like the shopping cart”

Legend: Task failed Prompt by moderator Struggle Task solved Insight

Note Taking

Pre- and Post Test Surveys:Paper vs. Online

Recording Testing Sessions

rapid, iterative cycles during development process

“sprints” are approx. 3-4 weeksrounds of testing should take place

during this timeframe

rele

aseco

nce

pt

rele

ase

Concept/Prototype

focusgroups

contextualinquiry

con

cep

t

Beta Release

assessmentusabilitytesting

expertanalysis

rele

aseco

nce

pt

AlphaRelease

expertanalysis

verification usabilitytesting

rele

aseco

nce

pt

ProductLaunch

site stats

post-launchusabilitytesting

rele

aseco

nce

pt

recommended reading

The first guide to linking business strategy with the art and science of the online user experience

resources

usability resourceshttp://www.gotomedia.com/resources.html

presentation postedhttp://www.gotomedia.com/goto/usability/

this presentation can be found at:www.gotomedia.com/goto/lifestyle

sign up for our quarterly newsletter on user experience and usability: www.gotomedia.com/gotoreport

mobile user experience:www.gotomobile.comthank you!