jonathan rubin - the benefits of do it yourself usability testing
DESCRIPTION
Presented by Jonathan Rubin, MS, on September 7, 2012 at the third annual Center for Health Literacy Conference: Plain Talk in Complex Times.TRANSCRIPT
The benefits of “Do It
Yourself” Usability Testing
By Jonathan Rubin
First Fridays Program Manager
General Services Administration
2
Let me tell you a story
3
Hanging chads
4
inspections
5
butteflu
They didn’t listen to customer’s
wants, needs and expectations.
TAKEWAY MESSAGE: Testing
with customers gets you
valuable feedback (duh)!
Why did problems occur?
7
In this hour…
1. What is usability
2. Why you need usability
3. Idea: Do your own testing!
1. What is usability?
Usability =
How people interact with a
product, system or service,
and how they expect it to
work.
Usability ≠ Quality Control
10
Good products are usable
11
12
What is it? How do you use it?
13
What is it? How do you use it?
14
Don’t feel stupid!
1. Observe users (customers) using your
product
2. (Re)design with users in mind
3. Get a usable product
4. Repeat
4 Steps to Working with Users
• Education • Demonstration Tests • Awareness
We’ve helped these folks
• DOT
• Interior
• IRS
• U.S. Army
• NASA
• Labor
• OMB
• GSA
17
• State
• FCC
• USAJobs
• Census
• NSF
• NIH
• And 35
others
Free @ Howto.gov/firstfridays
• Usability Testing Scripts • Email Templates • Best Practices • Final Report Examples • Project Management • Before and After Screenshots • Videos • Etc.
2. Why you need usability
Top 5 Government Usability
Problems
• Too many words
Top 5 Government Usability
Problems
• Too many words
• Gov speak / jargon
Top 5 Government Usability
Problems
• Too many words
• Gov speak / jargon
• Top tasks hard to find
Top 5 Government Usability
Problems
• Too many words
• Gov speak / jargon
• Top tasks hard to find
• Ineffective navigation
Top 5 Government Usability
Problems
• Too many words
• Gov speak / jargon
• Top tasks hard to find
• Ineffective navigation
• Ineffective search results
Example 2: GSA Intranet Example 2: GSA Intranet
?
9/11/2012 27
Before After
• Reduced developer time
Usability’s Return on Investment (ROI)
• Reduced developer time
• Reduced training time
Usability’s Return on Investment (ROI)
• Reduced developer time
• Reduced training time
• Reduced help desk calls / emails
Usability’s Return on Investment (ROI)
• Reduced developer time
• Reduced training time
• Reduced help desk calls / emails
• Higher task completion rate
Usability’s Return on Investment (ROI)
• Reduced developer time
• Reduced training time
• Reduced help desk calls / emails
• Higher task completion rate
• PR boost
Usability’s Return on Investment (ROI)
Know your users, for you are not them
- Usability Yoda
3. Idea: Do your own testing!
38
Must read: ‘Rocket Surgery Made Easy’
• Commit to monthly
product testing
• Same time each
month
• More observers +
more testing = more
buy-in
Needed for “discount” usability testing
• Executive champion!
Needed for UT
• Executive champion!
• One morning a month
Needed for UT
Needed for “discount” usability testing
• Executive champion!
• One morning a month
• 1-2 rooms
Needed for “discount” usability testing
• Executive champion!
• One morning a month
• 1-2 rooms
• Laptops
Needed for “discount” usability testing
• Executive champion!
• One morning a month
• 1-2 rooms
• Laptops
• Screen sharing
software ($1000/yr)
Needed for “discount” usability testing
• Executive champion!
• One morning a month
• 1-2 rooms
• Laptops
• Screen sharing
software ($1000/yr)
• Volunteers
Needed for “discount” usability testing
• Executive champion!
• One morning a month
• 1-2 rooms
• Laptops
• Screen sharing
software ($1000/yr)
• Volunteers
• Food ($10/person)
Needed for “discount” usability testing
• Executive champion!
• One morning a month
• 1-2 rooms
• Laptops
• Screen sharing
software ($1000/yr)
• Volunteers
• Food ($10/person)
• Incentives ($0- $80)
Needed for “discount” usability testing
• Executive champion!
• One morning a month
• 1-2 rooms
• Laptops
• Screen sharing
software ($1000/yr)
• Volunteers
• Food ($10/person)
• Incentives ($0- $80)
• Training (FF)
Needed for “discount” usability testing
48
How many people do I need?
Spending on usability
49
When usability is present, 10% of
development budget spent on
usability
- Usability Return on Investment (2008, Nielsen
Norman Group Report)
Staffing options (all are good)
1. Usability Contractor ($17,000 per test)
2. Dedicated Usability Staff ($60,000+)
3. Get training and do it yourself (Free)
50
Matrixed Approach to Staffing
51
• Part-time volunteers! • Borrow from other departments • You are “providing skill building opportunities” • Flexible needs: 1 -20 hours a month
Types of First Fridays Tests
• 1. Formal, talk-aloud tests
Stakeholder
Meeting
Develop
Scenarios
Recruit
Participants
+ Observers
Conduct
Test
Write
Report
Follow-up
Meeting
Formal Testing Process
Testing Room Observation Room
It takes two (rooms) to test
55
Formal Testing Schedule
• 8:20 – 8:30 Welcome
• 8:30 - 9:30 Tester #1
• 9:30 – 10:30 Tester #2
• 10:30 – 11:30 Tester #3
• 11:30 – Noon Lunch
• Noon – 1:30 Debrief / Critique
Absolute Minimum Usability Staffing
Testing Room
T
F
T = Tester,
F = Facilitator
Minimum First Fridays Staffing
Observation Room Testing Room
T
O
F M
O O
O O
O = Observer
F = Facilitator
M = Room Manager
Better Staffing
Observation Room Testing Room
T
F M
O
O
G = Greeter
DG = Data Gatherer
BU = Back up
G
F O O O O O
DG
BU
BU
BU
O = Observer
F = Facilitator
M = Room Manager
Types of First Fridays Tests
1. Formal, talk-aloud tests
2. Hallway Tests
Types of First Fridays Tests
1. Formal, talk-aloud tests
2. Hallway Tests
3. Navigation Improvement / Card sort
3. Navigation Improvement
Types of First Fridays Tests
1. Formal, talk-aloud tests
2. Hallway Tests
3. Navigation Improvement / Card sort
4. Quick Win Evaluation (Heuristics Analysis)
64
4. Quick Win Evaluation (Heuristics)
• Not for newbies
• Group or single person feedback
– Checklist
– Task run-through
• PRO: Fast! <1 week
• CON: No users
Teach em to fish Teach ‘em to fish
• Executive champion!
• One morning a month
• 1-2 rooms
• Laptops
• Screen sharing
software
• Volunteers
• Food
• Incentives for testers
• Training (First Fridays!)
Review: What you need to fish
Howto.gov/firstfridays
Free usability resources!
• Observe a test
More info on First Fridays
• Observe a test
• Recommend a website
More info on First Fridays
• Observe a test
• Recommend a website
• Volunteer as a tester
(remote or in person)
More info on First Fridays
• Observe a test
• Recommend a website
• Volunteer as a tester
(remote or in person)
• Learn to fish (Facilitator
in Training program)
More info on First Fridays
Once a month
