How to Conduct Usability How to Conduct Usability Testing:Testing:

In 9 Easy StepsIn 9 Easy Steps

By Jennifer L. BowieBy Jennifer L. Bowie

What is Usability?What is Usability?

““A function of particular users A function of particular users performing particular tasks in a performing particular tasks in a particular environment” (Smith et al. particular environment” (Smith et al. 68)68)

The “The “people who use the productpeople who use the product can do so quickly and easily to can do so quickly and easily to accomplish accomplish their own taskstheir own tasks” (Dumas ” (Dumas and Redish 4)and Redish 4)

User-centered design, not “user-User-centered design, not “user-friendly”friendly”

What is Usability Testing?What is Usability Testing?

An empirical study of a product’s An empirical study of a product’s usability by observing actual users usability by observing actual users do real tasks with the productdo real tasks with the product

Involves:Involves: Real usersReal users Real tasksReal tasks Specific usability goals/concernsSpecific usability goals/concerns Observing and recording the testingObserving and recording the testing Data analysisData analysis

Step 1: Step 1: Analyze your Actual Analyze your Actual Users & Choose ProfilesUsers & Choose Profiles

Who are your actual users? You may need Who are your actual users? You may need to break your users into typical user to break your users into typical user categories. Consider:categories. Consider: Demographics of your users (age, sex, race, Demographics of your users (age, sex, race,

education level, cultural background, education level, cultural background, socioeconomic status,…)? Will these socioeconomic status,…)? Will these differences impact use?differences impact use?

Experience level (with the product, with products Experience level (with the product, with products of the same genre, with required technology,...)?of the same genre, with required technology,...)?

Other things: motivation, learning style, subject Other things: motivation, learning style, subject matter knowledge, location of use, physical matter knowledge, location of use, physical characteristics, people with disabilities or characteristics, people with disabilities or impairments (from color blindness and learning impairments (from color blindness and learning disabilities to more severe disabilities), …disabilities to more severe disabilities), …

Step 1: Step 1: Analyze your Actual Analyze your Actual Users & Choose ProfilesUsers & Choose Profiles con.con.

Create user profiles:Create user profiles: Break users into clear subgroupsBreak users into clear subgroups Profile/Define the characteristics of each Profile/Define the characteristics of each

subgroupsubgroup Choose user profiles to test:Choose user profiles to test:

Ideally users from all major profiles will Ideally users from all major profiles will be testedbe tested

If limited testing: Choose profiles based If limited testing: Choose profiles based on highest number of users in that on highest number of users in that profile or profiles that you think may profile or profiles that you think may have the greatest usability issues have the greatest usability issues

Step 2: Step 2: Select Test PointsSelect Test Points

What is your objective? What test What is your objective? What test points “get you there”?points “get you there”? Test Task/ProcedureTest Task/Procedure- look for tasks - look for tasks

with a high chance or high cost of user with a high chance or high cost of user failurefailure

Test TerminologyTest Terminology- consider level of - consider level of useruser

Test your document design Test your document design strategiesstrategies- consider cueing patterns, - consider cueing patterns, heading/layout, navigation, heading/layout, navigation, extraordinary features, how text and extraordinary features, how text and visuals relate...visuals relate...

Step 3: Step 3: Choose Type of TestChoose Type of Test

Task (can-they-do-it):Task (can-they-do-it): Requires Requires users to perform a procedureusers to perform a procedure

Terminology (can-they-Terminology (can-they-understand-it):understand-it): Requires users to Requires users to provide a summary of what they provide a summary of what they have read or definitions of key termshave read or definitions of key terms

Document Design (can-they-find-Document Design (can-they-find-it):it): Requires users to use mocked- Requires users to use mocked-up portions of the manual to find up portions of the manual to find information on key topicsinformation on key topics

Step 4: Step 4: Set Performance Set Performance ObjectivesObjectives Consider how you will come up Consider how you will come up

with your datawith your data Time related:Time related: time taken to time taken to

perform procedure or find a topicperform procedure or find a topic Error related:Error related: number of errors number of errors

made during a procedure, made during a procedure, number of time passage re-read, number of time passage re-read, number of triesnumber of tries

Objectives should be task Objectives should be task individualizedindividualized

Step 5: Step 5: Select your UsersSelect your Users

Actual users:Actual users: always best always best Similar users:Similar users: not as best, may not as best, may

have to find ways to make them have to find ways to make them more even with a actual testers (give more even with a actual testers (give them more information, have them them more information, have them pretend they know less) pretend they know less)

Non-Similar users:Non-Similar users: very different very different and the worse types. May have to do and the worse types. May have to do a lot of work to make them more a lot of work to make them more appropriateappropriate

Step 6: Step 6: Create Test PlanCreate Test Plan

Create a written plan for the testing:Create a written plan for the testing: Choose order of tasks: start easyChoose order of tasks: start easy Create written test materials: Create written test materials:

Task list for usersTask list for users Written welcome speech/Intro to be read to userWritten welcome speech/Intro to be read to user Consent forms Consent forms Observation formsObservation forms Pre-task and post task questionnaires & interview questionsPre-task and post task questionnaires & interview questions Other materialsOther materials

Define team member’s roles:Define team member’s roles: Facilitator/Briefer (necessary): Often only team member to Facilitator/Briefer (necessary): Often only team member to

interact with usersinteract with users Observation recorder/note taker (necessary)Observation recorder/note taker (necessary) Camera operator (optional)Camera operator (optional) Help desk operator (optional)Help desk operator (optional) Test administer (optional)Test administer (optional)

Create a script/plan for the actual testCreate a script/plan for the actual test

Steps 7-9: Work your DataSteps 7-9: Work your Data

7.7. Record Information Accurately:Record Information Accurately: timing, copious notes, possible timing, copious notes, possible recordings…recordings…

8.8. Interpret the Data:Interpret the Data: Calculate data Calculate data Analyze the dataAnalyze the data Determine cause of problemsDetermine cause of problems Determine scope/severity of problemsDetermine scope/severity of problems Determine what needs to be Determine what needs to be

fixed/changedfixed/changed

9.9. Incorporate the feedback into the Incorporate the feedback into the DesignDesign

Good Luck & Have Fun!Good Luck & Have Fun!

Where to find out more:Barnum Usability Testing and ResearchBarker Writing Software Documentation, Chapter 6 “Conducting Usability Tests”Hom “General Concepts of Usability Testing” http://jthom.best.vwh.net/usability/general.htm