practical approaches to usability testing julie nowicki president optavia corporation web2000

Practical Approaches toUsability Testing

Julie NowickiPresident

Optavia Corporation

WEB2000

Practical Approaches to Usability Testing WEB2000

2

Goals

Put usability testing in perspective• In the bigger picture of usability• Compared to other techniques

Understand the breadth of usability testing methods available

Be able to focus and choose the right method• Depending on the web site’s development

stage• Balancing resource costs against benefits of

various methods


3

Topics

The role of usability testing Definition and comparison to other

techniques Designing the usability test

• Planning• Choosing specific methods

Tuxedos or jeans? Summary: Getting what you want

• And what your customers want!


4

E-commerce and Usability

Three E-commerce segments• B2C (Business-to-Consumer)• B2B (Business-to-Business)• B2E (Business-to-Employee)

All focus on the customer Ability of customer to

effectively complete a task, withhigh satisfaction levelsis key to the bottom line

Usability


5

Qualities of a usable Web site

Learnability• Can customers easily grasp how to use the site the

very first time? Memorability

• Do customers remember how the next time? Ease-of-use

• Can customers navigate easily?• Is the design consistent and predictable?

Efficiency• Can customers complete tasks and transactions

effectively? Appeal

• More than visual: do customers feel positive about the experience?


6

How can usability testing help?

Learnability Measure time-on-taskMeasure success rate

Ease-of-use Measure errors/false startsTrack navigation patterns (and why)Record comprehension of words and graphics

Efficiency Measure time-on-task

Appeal Record facial expressions and commentsAdminister post-test survey


7

Characteristics of a usability test

Uses real customers (users) Customers perform actual tasks with

the site or a prototype Data is recorded

• May be qualitative or quantitative• Includes customer comments

Provides objective information Facilitated by a skilled specialist Observed by a skilled specialist


8

Limitations of usability testing

Based on a few customers Guinea pig effect

• Test environment cannot duplicate the real world

• Users know they are being watched Not thorough by nature

• Testing a few tasks, not every aspect of site usage

• Some issues are not readily apparent• Use of color and fonts• Accessibility


9

Usability testing compared…

Web site log statistics Automated use testing Focus groups QA and system testing User acceptance testingProven usability techniques: Heuristic evaluation and guidelines

reviews Walkthroughs


10

Tip

Usability encompasses customer-centered design and evaluation.

However…Usability testing is an evaluation technique.

Usability testing is not a design technique.


11

Customer-centered design process

AnalysisAnalysis

EvaluationEvaluation

Understand objectivesUnderstand customersUnderstand their tasks

Develop design conceptSpecify the details

Test with real customers (usability testing)Review with design specialists

DesignDesign

Iterative!

Actual TimelineMonth 1 2 3 4 5-7

VisionField StudyCustomer analysisTask analysis

WorkflowReview Analysis Doc

Conceptual ModelPrototype SketchesStart Interact Spec Doc

1st Online Prototype

Refine Interact Spec Doc

2nd Online Prototype(real code)

Finish Interact Spec Doc

DevelopUserAssistance

WalkthroughUsability Testing with Sketches

Usability Testing with Online Prototype

Guidelines ReviewUsability Testing

Ongoing mini-reviews


13

Anatomy of a usability test

Plan and Design

Report and Recommend

Conduct

Plan: Answer the big questionsSelect the test methodsGet participants


14

Overview of usability testing methods

To prepare for the test, you must1. Plan

Answer some big-picture questions Understand objectives, customers, and tasks

2. Choose the test methods Considering a number of parameters, Pick the best usability testing methods for

your situation


15

Planning:Big decisions to make first

What are you trying to accomplish? When will you do the test?

• Based on product lifecycle

What are the customer profiles? What tasks will customers do? How will you present the results? How will you make recommendations?

• Explore design alternatives


16

What are you trying to accomplish?

Find out how your site compares to the competition

Find out if your new design works for• Existing customers• New customers

Get statistically sound data


17

Tips for setting objectives

Prioritize the four usability components• Learnability• Ease-of-use• Efficiency• Appeal

Be specific• NO “Registration is easy to use.”• YES “New customers should be able to

complete the registration process in 5 minutes without help.”


18

Tip

Usability testing done early in the design cyclereduces risk.


19

The Cost to Make a Change

1

10

100

Adapted from Rapid Application Development by James Martin

Usability

U-test a must

Usability


20

Don’t get fooled by online development tools!

Sketches are highly advantageous• They are easy to throw away• People don’t assume the system is done• People are more likely to suggest changes• Usability testing can be done with sketches

Tip


21

Tip

Usability testing done late in the production processhelps you plan for the next iteration.


22

Who are the customers?

Marketing information such as• Age• Income• Education• Career

Experience with the Internet Experience with online shopping Anticipated frequency of use (your site) Environment they work in

• Home, office…


23

What are the tasks?

Creating task scenarios is a must! Don’t use language that leads the user

• NO “Go to My Profile and change your address to…”

• YES “You have moved. Change your address to…”


24

Presenting the results

Consider allowing observers during the test• Make sure they do not interfere

Highlight tape• Compelling and efficient

Results of data collection Summary of issues, prioritized by

• Usability criticality• Implementation effort

Recommendations


25

The next step

Once you have• considered what you want to accomplish,

and when• identified your customer profiles• determined the tasks you want customers

to perform• decided how to communicate the results

you are ready to design the test methods


26

Test methods:Parameters that you must select

Equipment and facilities What to measure

• Quantitative and qualitative

How to collect measurements Use of participants (protocol) How many participants Level of facilitation Level of participant verbalization


27

Equipment and facilities

Conference roomPaper notes

One cameraPaper notes

Portable labLogging software

Lab facilityLogging software


28

Example Usability Labs

From www.usabilitysystems.com

From www.microsoft.com/usability


29

Advantages of video tape

Records very valuable, compelling qualitative data• Page navigation• Expressions• Comments in context

Provides a real snippet of an actual customer experience

Watch out• Someone needs to understand the equipment• Time to setup, review, create highlight tape


30

What to measure: Quantitative

Time on task Number of errors Successful completion of task (yes/no)

How you collect measurements is determined by facility and equipment


31

What to watch for: Qualitative

Navigation patterns• Getting lost?

Predictability• Understand graphics?• Understand words?• Looking for it to work like something else?

Flexibility• Looking for a quicker way?• Looking for help?


32

Protocols (Use of participants)

Individual Paired Comparison

• Good for competitive analysis• Between-subjects: each customer uses one

site• Within-subjects: each customer uses both

sites


33

How many participants?

5-8 participants will find 80% of usability problems

4 is good minimum number

Qualitative dataEarly prototype

Qualitative and quantitativeLater in development

Paired or comparison protocols

3 4 5 6 7 8 9 10 11 12

Statistically sound results


34

Level of facilitation

Facilitator in another room

Facilitator in same roomDiscussion after each task

Facilitator and participant interact during task


35

Level of participant verbalization

Think-aloud Heads-down work


36

Interaction

XQualitative dataDiscussion-orientedInformal environmentThink-aloud

Quantitative dataHeads-down taskFormalLab

Mostly qualitativeUser works aloneFormalLab

High

LowLow High Participant verbalization

Faci

litati

on

Putting it all together

LoFi Prototype(paper)

HiFi Prototype(online)

Pre-production Production site

Qualitative data

Quantitative Data

Conference rm Conf rm or lab Lab Lab

Hi facilitation Medium Medium Lo facilitation

Hi participant verbalization

Lo participant verbalization

3-4 users 4-8 users 4-8 users 6-12 users

Participants <1 hour

<1-2 hours 1-2 hours Participants 2+ hours

Individuals Individuals Indiv’s or pairs Indiv’s or pairs

Days Weeks

Informal Formal


38

Now that we’ve said that…

Can you do informal testing on a production site?• Sure!

Can you do quick, but formal testing in a lab on a prototype?• Sure!


39

What if…

You’re in the fast-paced e-commerce environment

You want customer data but you need it• Now (yesterday)• Whenever you make significant changes

You want guidance in making design decisions or changes

Variation

LoFi Prototype(paper)

HiFi Prototype(online)

Pre-production Production site

Qualitative data

Quantitative Data

Conference rm Conf rm or lab Lab Lab

Hi facilitation Medium Medium Lo facilitation

Hi participant verbalization

Lo participant verbalization

3-4 users 4-8 users 4-8 users 6-12 users

Participants <1 hour

<1-2 hours 1-2 hours Participants 2+ hours

Individuals Individuals Indiv’s or pairs Indiv’s or pairs

Days Weeks

Informal Formal


41

Resource costs will be determined by your choices

Cost of facilities and equipment Time for each participant Your time resources

• Planning test• Conducting test• Reporting and decision-making

Staff resources and skill levels Generally all go up as you move from

informal to formal


42

Results

“When [we] introduced an electronic expense payment system, users declared the application incredibly efficient – as early as the testing phase. That never would have happened if we hadn’t had the usability group to help us meet our business objectives. No company should ever move forward on a project in today’s business environment without some usability in their back pocket.”

From Supervisor of Integrated Business Practices

Usability is more than just usability testing…

It is a process

Julie NowickiPresidentOptavia [email protected] 260 9000www.optavia.com

practical approaches to usability testing julie nowicki president optavia corporation web2000

Documents

usability testing web2000how

usability testing help

howeverusability testing

linepractical approaches

memorabilitydo customers

usecan customers

design technique

design consistent