2012-11-26: usability testing as validation

Usability Testing as Validation

Amy Rubino @Amy_Rubino

November 26, 2012

Lean Startup Baltimore

arubino@systemsalliance.com

linkedin.com/pub/amy-rubino/13/837/31b

@amy_rubino

Amy Rubino User Experience Consultant, Systems Alliance •  15 years of design experience as a

print and web designer, information architect and user experience advocate.

•  Worked with universities, corporations and non-profits.

•  Master’s degree in Interaction Design and Information Architecture.

Usability Testing as Validation

About Me

2

What We’ll Cover Usability

 Users   Learn About Users   Learn From Users

Usability Testing  How to Do It  What You Can Test  Why You Should Do It

Usability Testing and Lean Startup Demo Resources Photo credit: Amy Rubino

3 Usability Testing as Validation

Photo credit: http://www.flickr.com/photos/reticulating

Usability

4

User Experience Honeycomb

Photo credit: http://semanticstudios.com/

5 Usability Testing as Validation

  Creating products that are easy to use  Require less mental effort to use  Decrease frequency of mistakes and mistakes are less disastrous  Can be used to do more or do it faster  Can be figured out quicker

Usability

6 Usability Testing as Validation

Usability

Photo credit: http://impossibleobjects.com

7 Usability Testing as Validation

  Based on human behavior   Focus on people who will use

it or users

Usability

Photo credit: sanjaykumarux.blogspot.com

8 Usability Testing as Validation

Photo credit: http://www.leveltendesign.com/

 Use your website or app  Part of your target

audience  Not directly affiliated with

your company

Users = Real people

9

You Are NOT Like Your Users

You Know a lot about

your product and company

Are passionate about what you offer

Have ideas on how your

website or app will be used based on being an insider

Your Users Know little or nothing about your product or company Just want to get things done May use your website or app differently than was intended

10 Usability Testing as Validation

 Surveys   Focus groups  Web analytics

Learn About Users

Photo credit: http://www.flickr.com/photos/scottandsarah 11

Photo credit: http://www.flickr.com/photos/blesseurope

  Talk to them  Observe them in their own

environment  Usability testing

Learn From Users

12

Photo credit: http://www.openroad.ca

Usability Testing

13

Steve Krug Author, “Don’t Make Me Think” and “Rocket Surgery Made Easy”

“ … If you want to know whether your software… is easy enough to use, watch some people while they try to use it and note where they run into trouble. Then fix it, and test it again.”

14 Usability Testing as Validation

Photo credit: http://www.flickr.com/photos/doos/3050195096

  Develop a test plan   Choose a testing environment   Find and select participants   Prepare test materials   Conduct the sessions   Debrief with participants and

observers   Analyze data and observations   Create findings and

recommendations

Traditional Process

15

Photo credit: http://vator.tv/news/2011-05-23-guidelines-for-product-usability-testing

16

17 17 Photo credit: http://vator.tv/news/2011-05-23-guidelines-for-product-usability-testing

17

18 18 Photo credit: http://vator.tv/news/2011-05-23-guidelines-for-product-usability-testing

18

19 19 Photo credit: http://vator.tv/news/2011-05-23-guidelines-for-product-usability-testing

19

  Invite people to see a beta version and watch them complete tasks

  Pros  See and hear participants’ reactions  Can record conversation and click path   Your stakeholders and team can

participate

  Cons   You or participants have to travel to meet

up  May not have access to participants who

live outside of this region

In-Person Usability Testing

Photo credit: http://www.flickr.com/photos/curiouslee/

20 Usability Testing as Validation

  Use screen sharing software to watch participants complete tasks in another location

  Pros  Don’t have to travel to participants  Have access to participants who live

outside of this region  Can record conversation and click

path   Your stakeholders and team can

participate

  Cons  Don’t see participants’ reactions

Remote Usability Testing

Photo credit: http://www.flickr.com/photos/rosenfeldmedia/

21 Usability Testing as Validation

  Use a service that allows participants to complete the tasks on their one time without you moderating

  Pros  Participants can do it when it’s convenient for them  Have access to participants who live outside of this region

  Cons  Can’t ask follow up questions  Don’t hear participants think out loud  Don’t see participants’ reactions

Unmoderated Usability Testing

22 Usability Testing as Validation

Photo credit: http://www.jamesjpierce.com/energy_challenge_08.html

 Real products  Digital prototypes  Paper prototypes

What You Can Test

23

  Do test the site NOT the participants   Do note participant’s success in

completing each task   Do ask open-ended follow up

questions   Do compensate participants

DO

“I would open

Goggle and search

for the website.” Quote from usability testing participant

24 Usability Testing as Validation

  Don’t give participants tasks that can not be completed

  Don’t guide participants   Don’t answer questions until the

end   Don’t take it personally

DON’T

“Do you really

think people will

use this?” Quote from usability testing participant

25 Usability Testing as Validation

  Reduce costs Once a piece of software makes it into the field, the cost of fixing an error can be 100 times as high as it would have been during the development stage. Source: IEEE (Institute of Electrical and Electronics Engineers) http://spectrum.ieee.org/computing/software/why-software-fails/

Why You Should Do It

$0

$20

$40

$60

$80

$100

$120

During Development

After Completion

Fix Errors

26 Usability Testing as Validation

  Increase revenue   Increase sales, applications, memberships or donations   Increase retention  Remain competitive

Why You Should Do It

27 Usability Testing as Validation

  Improve effectiveness  Reduce user error   Increase efficiency   Increase user satisfaction

Why You Should Do It

Photo credit: http://www.flickr.com/photos/terryfreedman

28 Usability Testing as Validation

Photo credit: http://thebln.com

Usability Testing and Lean Startup

29

Lean Startup

Photo credit: http://lean.st/

30 Usability Testing as Validation

  Develop a test plan   Choose a testing environment   Find and select participants   Prepare test materials   Conduct the sessions   Debrief with participants and

observers   Analyze data and observations   Create findings and

recommendations

Traditional Process

Photo credit: http://www.flickr.com/photos/doos/3050195096

31

Photo credit: http://www.flickr.com/photos/adactio/

  Develop a test plan   Choose a testing environment   Prepare test materials   Find and select participants   Conduct the sessions   Debrief with participants and

observers   Analyze data and observations   Create findings and

recommendations

Lean Process

32

  Go to a public place and ask customers to complete tasks

  Pros  See and hear participants’ reactions  See participants use product on different

devices

  Cons  Might need to get manager’s approval  Customers may not be your target

audience   Your stakeholders and team can’t

participate

Guerilla Usability Testing

Photo credit: http://www.flickr.com/photos/_dchri

33 Usability Testing as Validation

Jakob Nielsen principal, Nielsen Norman Group

“ … The best results come from testing no more than 5 users and running as many small tests as you can afford.”

34 Usability Testing as Validation

Demo

Photo credit: http://www.flickr.com/photos/vaniercollege 35

Try it Yourself!   Join Baltimore Lean Startup on Facebook:

https://www.facebook.com/groups/BaltimoreLeanStartup   Do some guerrilla testing   Post your results

36 Usability Testing as Validation

Resources

Photo credit: http://www.flickr.com/photos/rodrigogalindez/ 37

Books   “Don’t Make Me Think: A Common

Sense Approach to Web Usability” by Steve Krug

  “Handbook of Usability Testing: How to Plan, Design, and Conduct Effective Tests” by Jeffrey Rubin and Dana Chisnell

  “Remote Research: Real Users,

Real Time, Real Research” by Nate Bolt and Tony Tulathimutte

Photo credit: amazon.com

38 Usability Testing as Validation

Websites   sensible.com   useit.com   usability.gov   alistapart.com   uie.com

Photo credit: http://usability.gov/

39 Usability Testing as Validation

  Morae   Optimal Workshop   UserTesting.com   Loop11   GoToMeeting   Skype

Testing Tools

Photo credit: http://beta.skype.com/en/

40 Usability Testing as Validation

Questions?

Photo credit: http://en.wikipedia.org/wiki/On_the_Internet,_nobody_knows_you%27re_a_dog

41 Usability Testing as Validation

Stay in touch: www.systemsalliance.com

@systemsalliance

info@systemsalliance.com

Amy Rubino arubino@systemsalliance.com

Contact

42 Usability Testing as Validation