the why and how of usability and user experience (ux) testing

BT12 Concurrent Session 11/8/2012 3:45 PM 

"The Why and How of Usability and User Experience (UX) Testing"

Presented by:

Philip Lew XBOSoft

Brought to you by:  

340 Corporate Way, Suite 300, Orange Park, FL 32073 888‐268‐8770 ∙ 904‐278‐0524 ∙ sqeinfo@sqe.com ∙ www.sqe.com

Philip Lew XBOSoft, Inc.

After working for twenty years in various management and technical positions in product development and management, CEOPhilip Lew now leads the direction and strategy of XBOSoft (xbosoft.com). He has helped hundreds of organizations assess the quality of their software, examine software quality processes, and set forth measurement plans so they can consistently improve software quality using systematic methods. Phil has authored articles in IEEE and ACM journal publications and trade journals; presented at several conferences on web application usability, user experience, and quality evaluation; and is conducting post-doctorate research focused on user experience measurement frameworks and software learnability.

1

The Why and How of Usability and User Experience (UX) Testing

Philip Lew

Agenda

• What is usability and UX and why i t t?important?

• Usability and UX – What are they• How to set up models and measurements• Case study and examples in testing and

evaluating applicationsevaluating applications

2

What is Usability-UX and Why is it Important

The Year of Mobile?

• Last year was the year of mobile in terms f th th i b th bil d thof the growth in both mobile usage and the

availability of mobile sites and mobileapps. • Time to redesign your mobile site,

because your existing version is probably far below users' growing expectations forfar below users growing expectations for user experience quality.

• What do you think users expect?

4

3

History Repeats Itself• In the beginning, the Web was experimental —it was

acceptable to have a somewhat shaky, experimentalacceptable to have a somewhat shaky, experimental website.

• Many sites were crippled by misguided design advice• Many didn't know any better (because they didn't do

usability studies). Now, people simply expect websites to work.For mobile last year it might have been cool simply to• For mobile, last year, it might have been cool simply to have an app.

• Now, that app better be good. Expectations have gone up.

5

Web and Mobile User Expectations

• Business models have changed– Instead of paying

upfront and ‘owning’ the software

– Pay as you go, pay by subscriptionp

• Cloud and mobile converge

• Behavior and expectations have changed

4

Basic Usability Concepts

User Experience

Usability Effect

Usability Design

7

Design-Test and Evaluate

Design• What will the mobileapp do?

Test and Evaluate

Test and Evaluate

mobileapp do?• Is it a conversion of existing app?

• What functions will a user really 

Release

8

access?

5

Usability - Design Perspective

• Understandability• Learn-ability• Operability• Attractiveness• Navigation• Responsiveness-performance

What else can you think of?

9

Usability-Effect

D t hi h ifi dDegree to which specified users can achieve specified goals with effectiveness, efficiency and satisfaction in a

Source: ISO 25010

efficiency and satisfaction in a specified context of use.

10

6

Usability-Effect“Context” and “Specified”

• User role specified users

• Objective• Task• Environment

specified goals

Environment• Domain• …

specified context of use

What else can you think of?

11

Usability-EffectUser Experience

• Satisfaction– The degree to which users are g

satisfied in a specified context of use. Satisfaction is further subdivided into sub-characteristics:

• Likability (cognitive satisfaction)

Source: ISO 25010

• Pleasure (emotional satisfaction)• Comfort (physical satisfaction)• Trust

• Including many other factors experienced over time and other channels

12

7

Task Ease(effectiveness and efficiency)

• Buttons are a key tool in the user experience designer’s box of tricks.box of tricks.

• Prioritize tasks - Some tasks more important than others. • Understand the objectives of the application • Understand which tasks are really important. • Majority of your app’s value is provided by a small

number of tasks.• Remove any friction that inhibits the user’s progress. • Choose your words carefully to make it clear what the

buttons or functions do

13

What could add ‘friction’ ?

Design and Test For the Task• Highly specific tasks.

– You are in an electronics store looking at the Minolta GS234 camera It costs $120 25 in the store CheckGS234 camera. It costs $120.25 in the store. Check costco.com to see if you can get a better price online.

• Directed, but less specific. – Find a moisturizer with SPF 45 or above that is

suitable for your skin." (Use Walmart app.)• Open-ended, but restricted to a predetermined site-app.

– "See if you can find any interesting pictures related to today's news." (While using the USA Today app.)

• Web-wide tasks that let users go anywhere. – "Find out which is the tallest building in the world."

14

8

Usability

Basic Design Issues and ConceptsConcepts

Design Testing and Evaluation

15

Design Concepts  Test and Evaluation Criteria

• Context• Context• Positioning• Color usage• Size usage• Size usage

9

User Context is King

• Mis-Targeted user b k dbackground can cause loss of effectiveness

• Know your users.

17

Who are your users?What are their special needs/characteristics?

Positioning• Effectiveness of

application buttons ppis affected by position.

• Emphasis is gained by – removing clutter– placing the button

where the user’s eyes will mostly likely be.

18

10

Using Color

• Color creates emphasis. • Dependent on the importance of the

application’s function.• Make easy to for user to see the most

important stuff.Color can affect usefulness• Color can affect usefulness.

19

LinkedIn - ColorCreate an account!!!

• The ‘View Full Profile’ button– LinkedIn wants you to click that.– You’ll be prompted to create an account!

• Button has a unique color that isn’t shared with any other part of the design.

• Page has tons of information, this button still stands out.20

11

Don’t Overuse a Color

21

Size Matters• Time taken to point at an object is directly influenced

by the size of that object. • Big is beautiful depending on function-what you want

the user to do.• Size can dictate the button’s importance over

everything else on the page. • Take the Firefox page for example, Mozilla don’t mess

around with subtlety here.y• Good design communicates priority.• With one massive button on the page, that priority is

obvious.

22

12

Using Size AND Color

23

Multivariate Testing• Small changes can make a big

differencedifference.• Laura Ashley arrived at this design

following multi-variate testing of 5 different options

• Differences in the testing included:g– Link colors and locations

• The "Go to checkout" button was dark gray instead of green

• Achieved 11% increase in checkouts 24

13

Let’s Go Mobile

Usability and UX For Mobile

25

Usability - Desktop to Mobile• Just loading websites on the phone is not

enoughenough • What matters for users

– Time to load the website– User interface– Accessibility of various functions available

• Usability design needs to change, otherwise usability effect and UX will suffer

26

14

Different Platforms Need Different Design

• Horizontal swiping now used to "flip" through deck-of-cards or carousel features. – Include a visible cue when people can swipe

• Avoid swipe ambiguity: don't use the same swipe gesture to mean different things on different areas of the same screen. – Use the same meanings for mobile phones

and tablets – Mobile-device users typically expect to horizontally

swipe while desktop websites avoid horizontal scrolling

• Mobile sites should perform better than full sites when used on a mobile device.

Nov 2012 Better Software East 2012 27

Let’s Step Through a Few Examples

Usability and UX For Mobile

28

15

Laura Ashley Mobile Site

• Same as web versionG ti k t• Green sticks out

29

Laura Ashley

• What is wrong here?

30

16

Laura Ashley – 9/2012

31

LA MS

• Consistent greenOb i l t t• Obviously want you to click and add to the bag

32

17

LAShopping bag

• Clear green signals… BUYBUY

33

LA MobileApp

• Main ScreenD t i b tt• Decent size buttons

• No priority or importance weighting

34

18

LA Mobile App

• Big buttonsG d f bi fi• Good for big fingers

• Semi-meaningful icons on the bottom– Note contextual

location

35

Buy or Share?

• No placement of importance or priorityimportance or priority

36

19

Select Size

• Good navigationG d i b tt• Good size buttons

• No priority

37

In my Basket

• What should I do now?now?

• Buy or share?

38

20

Shopping Basket

• Empty or Checkout?

39

Bloomberg

• Simple Information AppApp

• Simple colors• Meaningful Icons• Big enough to see

40

21

Fidelity

• Easy to use scroll– Thumb Friendly

• Simple colors• Very limited task

based functionality

41

Fidelity

• Size and ImportanceP tt t• Pretty easy to see what they want you to do

• Consistent icons on bottom

Si l– Simple– Not everything

42

22

Yelp

• Easy on the eyesS i i f l• Semi-meaningful icons

43

Yelp

• ConsistentFi f i dl• Finger friendly

• Consistent location of icons on bottom

Similar to Fidelity

44

What does this mean?

23

Making Assumptions

• Web sites automatically sense the platform and

it h t ‘ bil it ’switch to a ‘mobile site’ (for instance, United Airlines, Google, and others) – Provide mobile users with

more efficient web experience. e pe e ce

• Has the potential of making too many assumptions regarding the users’ expectations.

45

Web-Mobile Integration

• Overall integration of web-apps needs to be thought out carefully with mobile scenarios in mind.

• Friend wanted to show me a product at www.costco.com.

• He sent me a link from his iPhone. I was at my desktop, and opened the link and got the mobile version.

Nov 2012 Better Software East 2012 46

24

Give a Choice to Avoid Making the Wrong Assumption

• Managing user expectations of how the applicationof how the application should behave needs to be thought out carefully.

• Context of the user and their expectations is key

• LinkedIn gives you a choice when accessing their site;when accessing their site; an optimized mobile experience via an mobile-app, or just their ordinary website.

Nov 2012 Better Software East 2012 47

Determine the Needs at Hand

• What need is the mobile application attempting to mobilize?

• How can the workflow on a mobile device be designed more efficiently to accomplish the task?

• How can the specific features and characteristics of a mobile device improve and complement the experience in contrast to the normal web-based papplication or other mobile platforms?

Nov 2012 Better Software  East 2012 48

25

Determine the Needs at Hand• Fidelity implements top

scenarios their users want and optimizing efficiency for p g ythose user scenarios ONLY.

• Filling an order was a user scenario optimized just for the mobile platform.

• Note that it only has 4 data items to fill in with 2 beingitems to fill in, with 2 being scroll buttons, while the biggest button indicates precisely what the purpose of the task is.

Nov 2012 Better Software  East 2012 49

Mobile Usability DesignBest Practices Summary

• Quick - Can you read the text without zooming or scrolling?scrolling?

• Simple Navigation-Task Oriented• Thumb Friendly• Visibility• Easy to Convert/Complete the task• Contextual

Design for the small screen. Don’t make users struggle to click tiny areas that are much smaller than their fingers.

• Seamless with main webapp• When you have a smaller screen, you must limit the

number of features to those that matter the most for the mobile use case.

50

26

What Is Designed, We Test

Understanding Design Leads to B tt U d t di E l tiBetter Understanding, Evaluation,

and Testing

What does want to achieve in the Usability UX

Standpoint?

52

27

Introduction to• iMedTablet - patient care coordination for

all types of long term care providersall types of long-term care providers. • Allows the user to securely access and

document patient care information in real-time utilizing Cloud technology, GPS technology and intra-facility gy ycommunications.

• Designed to seamlessly integrate with many pharmacy and long-term care software systems.

53

- Login Screen

54

28

-Med Admin Record

55

• Users are _________.• Heavily context oriented based on user expectations and habits.–Best design is not equal to the best usability.

29

Setting Up Usability Measurements

Measure and Improve

57

Big Picture on Usability and UX

User Experience

Usability Effect

Usability Design

58

30

Design-Test and Evaluate

Design

Test and Evaluate

Test and Evaluate

Release

59

Let’s Define UsabilityProduct (Design) Viewpoint

UsabilityUsability

Characteristic 1Characteristic 1 Characteristic 2Characteristic 2 Characteristic nCharacteristic n

Subcharacteristic 1Subcharacteristic 1 Subcharacteristic 2Subcharacteristic 2 Subcharacteristic  nSubcharacteristic  n

SimplenessSimplenessNavigationNavigation

Control StabilityControl Stability Button VisabilityButton Visability

# Buttons# ButtonsAttribute 1Attribute 1 Attribute 2Attribute 2PositionPosition Color UsageColor Usage

• Attributes expressed hierarchically• Any number of sub-levels is OK• Achieving the sub-attributes=achieving the high level attribute -measurable

31

Defining Usability from an Effect-Real usage Point of View

QualityQualityUsabilityyy

EffectivenessEffectiveness EfficiencyEfficiency Characteristic nCharacteristic n

AccuracyAccuracy CompletenessCompleteness Subcharacteristic  nSubcharacteristic  n

y

SatisfactionSatisfaction

ErrorsErrors Attribute 2Attribute 2 Attribute 3Attribute 3 Attribute 1Attribute 1

Potential Attributes Measurements for ‘Effect’ Usability

• Effectiveness– Completion rates– Completion rates– Error rate– Help usage

• Efficiency– Task time/Speed

• Learnability– Learning rate– Task time deviation– Change in task times over time period

32

Measurable Attributes

• Attribute name• Description and purpose• How to measure• What is measured• Measurement/Calculation

Need a model before you can start measuring…

• Range (min, max)• Objective• Current

Usability Measurement MethodsFocus groups

WalkThroughsS i f i

Logging

Heuristic Evaluation

Labs

ThroughsSatisfaction

Surveys

Let’s get started

Logging

33

Usability MeasurementAttribute Scale How Measure or

CalculationObjective Current

Help Access Percent  ofUsers

Server Log files

% 30% 40%Users AccessingHelp

files

Task Simplicity

Keystrokes tofind/use a feature/function/information

Heuristic measurement

% 90% < 3 50%

65

ormation

Task Times Time finish‐time start

Server log files

% 90% < 3 50%

Accuracy Numbererrors

Server Log files

Integer or %

<5 10

Sample Heuristic EvaluationUsability Measure Elementary Partial1. Understandability 75%1.1  Label/icon recognizable 2 50%1.2  Function‐task recognizable 4 100%2. Navigation 63%2.1  Back‐Forth 3 75%2.2  Full‐mobile 2 50%3. Swiping 63%3.1  Consistency 3 75%3.2  User Awareness 2 50%4  Learnability 33%

Nov 2012 66

y4.1  Error prevention 1 33%4.2  Error detection 1 33%Total 58%

Through defining the model, then benchmarking with heuristic evaluation, we were able to determine weak areas and make recommendations for significant improvements

34

Usability LoggingMeasurement and Data Collection• Identify users by using session ID

to identify a unique user.to identify a unique user. • Iteratively insert code into the

application• Collect data• Analyze the data for each attribute

in different dimensions and aggregations

• Determine the need for further• Determine the need for further calculations and what attributes to measure further

• Revise the data we are collecting, adding or decreasing granularity

67

Satisfaction SurveysThe process:1 Calculate the usability score (satisfaction) of1. Calculate the usability score (satisfaction) of version X

2. Do the survey3. Change to version X.1 – make changes to the software directly correlated to the usability factors to either increase or decrease thefactors to either increase or decrease the usability score

4. Do the survey again 5. See if differences made change impact the survey results 68

35

Putting It All Together

Usability Design User ExperienceUsability in 

Use

User Satisfaction

69

Getting Started with Measurement

• Produce an action planProduce an action plan– What usability attributes are important to your

organization?• Develop a model

– What data can you collect/Which technique can you usecan you use

• Maybe some elements of the model drop out-can’t be measured that easily

– Start collecting and developing benchmarks

70

36

Conclusion-Usability and UX • Paramount for today’s applications with users’ short

attention spans.• Designs should be as simple as possible for the user• Designs should be as simple as possible for the user. • UX is heavily influenced by expectations. • Make navigation easy on a small screen, thumb friendly,

and as intuitive as possible. • If the user cannot figure it out in 30 seconds, they are

gone. • Mobile-app versus your web-app - differentiate and

discern access from each platform. • Determine the features you really need and optimize

screens for certain workflows rather than trying to do it all.

71

Conclusion-Measurement• Defining UX is different for each organization• Need a model for your organizationy g

– What is most important to you depends on your users and their behavior and expectations

• The model is the foundation of what to measure, evaluate and improve

72

37

Thanks

Questions and Answers

www.xbosoft.com408-350-0508

Philip Lewphilip.lew@xbosoft.com

73