So much UX data! Now what?

Jennifer Romano BergstromApril 12, 2014

UXPA-DC Workshop| Arlington, VA@romanocog

2

Measuring the UX

• How does it work for the end user?

• What does the user expect?

• How does it make the user feel?

• What is the user’s story and habits?

• What are the user’s needs?

Usability = “the extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency, and satisfaction in a specified context of use.” ISO 9241-11

+ emotions and perceptions = UX

@romanocog @uxpadc

3

Usability vs. User Experience (UX)?

The 5 Es to Understanding Users (W. Quesenbery): http://www.wqusability.com/articles/getting-started.htmlUser Experience Design (P. Morville): http://semanticstudios.com/publications/semantics/000029.php

Whitney’s 5 Es of Usability

Peter’s User Experience Honeycomb

@romanocog @uxpadc

4Krug, S. Don’t Make Me Think

What People do on the Web @romanocog @uxpadc

5

UX data

OBSERVATIONAL+ Ethnography

+ Time to complete task+ Reaction time

+ Selection/click behavior+ Ability to complete

tasks+ Accuracy

IMPLICIT+ Facial expression analysis

+ Eye tracking+ Electrodermal activity (EDA)

+ Behavioral analysis+ Linguistic analysis of

verbalizations+ Implicit associations

+ Pupil dilation

EXPLICIT+ Post-task satisfaction

questionnaires+ In-session difficulty

ratings+ Verbal responses

+ Moderator follow up+ Real-time +/- dial

@romanocog @uxpadc

6

Explicit data

*Satisfaction Questionnaire Q7: Please rate how difficult it was to log in on this device. 1= not difficult at all to 5= extremely difficult.

Participant Ratings0

3

6

9

How likely would you be to recommend this site to a friend?

Not likely at all or Slightly likely

Moderately Likely

Very Likely

Participant Ratings0

3

6

9

How likely would you be to use this site in the future?

Not likely at all or Slightly likely

Moderately Likely

Very Likely

“Love the picture in the middle of it.”“It looks very clean and very simple.”

“It looks pretty organized, it's a nice design.”

When asked how they would save information, four of six participants said they would bookmark the page or take a screenshot of the information. Only two mentioned that they would use the site functionality to save for later use.

83%

9%9%

Percentage of Difficulty Rat-ings*

1 & 2

3

>=4

@romanocog @uxpadc

7

Observational data

Details page about Military Physician Assistant career, only accessible through Military section.

Details page about Civilian Physician Assistant career.

Evidence: Six participants clicked on Salary & Wages when looking for states with the most jobs before looking under Job Growth. • Five participants thought they had

completed the task once they found the Salary & Wages map on the wrong page.

Time on Page0

5

10

15Average time spent on landing page

Seco

nds

@romanocog @uxpadc

88

Implicit data

8

00:00.00 00:03.88 00:07.75 00:11.63 00:15.50 00:19.38 00:23.25 00:27.13 00:31.00 00:34.88 00:38.75 00:42.63 00:46.50 00:50.38 00:54.256

6.57

7.58

8.59

P29 iPhone

Begin date and time selection.

End date and time selection.

Trouble with scrolling.

More trouble with scrolling.

@romanocog @uxpadc

99

Implicit data

9

00:00.00 00:03.88 00:07.75 00:11.63 00:15.50 00:19.38 00:23.25 00:27.13 00:31.00 00:34.88 00:38.75 00:42.63 00:46.50 00:50.38 00:54.256

6.57

7.58

8.59

P29 iPhone

Begin date and time selection.

End date and time selection.

Trouble with scrolling.

More trouble with scrolling.

@romanocog @uxpadc

10

Eye tracking @romanocog @uxpadc

11

Heat maps @romanocog @uxpadc

12

@romanocog @uxpadcGaze plots

13

@romanocog @uxpadcGaze plots

14

@romanocog @uxpadcGaze plots

15

Gaze plots and commentsM

“Man, this is a long paragraph.”

“There's a lot of information, it'd be a lot better in list form. Ideally, you want to get your information quick without reading through all this.”

@romanocog @uxpadc

16

Usability test of a low-fi prototype

@forsmarshgroup @romanocog

@romanocog @uxpadc

• video

17

Usability test of a high-fi prototype

@forsmarshgroup @romanocog

@romanocog @uxpadc

• video

18

More data

Fixation count Heat map of the Round 2 home page from 7 participants during the “scam” task

Left navigation

Center icons

Mean time elapsed before AOI is fixated

79.55 (18.54). N = 8

29.36 (9.2). N = 6

10

3

6

9 “How clear is the information on this page?”**

# o

f Part

icip

ants

Not at all to Slightly Clear

Moderately Clear

Very to Extremely Clear

@romanocog @uxpadc

19

• “Where and how you click is a bit counter-intuitive. [It’s] not super obvious which button to click to get to next sections.”

• “I feel like the ‘Next’ should be at the bottom and not the top.”

Intuitive ‘Next’ button location

Non-intuitive ‘Next’ button location

Usability test across devices @romanocog @uxpadc

20

Combining data

• “I’m not expecting them to email or call me. I don’t expect any person to notify me.”

• “I’m not sure when I’d get an answer. It’s not like Yahoo! Answers where it’s immediate.”

Participants had different expectations about what would happen next. • Five expected to hear

back via email• One expected an

immediate response• One expected to see the

answer posted somewhere on the site

• Two said they would call for assistance.

@romanocog @uxpadc

video

So how much data do I need?

22

Considerations

• Qualitative• Quantitative• Iterative testing• User groups

@romanocog @uxpadc

Questions

Jennifer Romano Bergstrom@romanocog

jbergstrom@forsmarshgroup.com

UXPA-DC Workshop| Arlington, VA