@pnoreault #LavaCon

Improve the UX of Your Content and Prove It

Pam Noreault – ACI WorldwideTara Knapp – ACI Worldwide

Contact usTara Knapp

Manager, Information DevelopmentACI Worldwide

tara.knapp@aciworldwide.comtaraknapp@gmail.com@tara_knapp (Twitter)Tara Knapp (LinkedIn)

Pam Noreault

SR Manager, Information DevelopmentACI Worldwide

pam.noreault@aciworldwide.compamnoreault@gmail.com

@pnoreault (Twitter)Pam Noreault (LinkedIn)

Journey• Beginning• Imagine - What could be• Our road• Content quality• (Re)evolution• What’s stopping you?

Beginning

Imagine!Customers are your biggest advocates.

Where is your content on a scale of 1 to 3?

1 = Hot Mess 2 = Partial Mess 3 = Utopia

Methodology before

Methodology now – inching upwardsPersonas

(product level)

User Research and Analysis

(release level)*usage patterns

Information Model(release level)

User Stories

User-Centered ContentInformation Model

(deliverable)Concise writing

Topic-based writingTask-based writing

Writing for translationWriting for accessibility

Validation Testing

How we select content to fix1. Select deliverable2. Select content to uplift

Before model After models

Before overview

After overview

Before topic

After topic

Road led us here – BUT prove the changes made a difference

Contextual overviewsConcise/clear contentReduced

content/eliminated clicksTopic-based (text scanning)Accessibility – checklist of

fixesTranslation – checklist of

fixes

How many of you are doing content validation with customers?

Validation methodologyModel for PDF Documents• Uplifted four documents

Methodology• Teams of 4-7 writers per document• Tested each document with at least 2 users from 2 customers• Tested 1 hour via WebEx• 4 tasks tested on each doc• Presented 2 docs – old + new• Order of docs alternated

Validation protocolDirections

• Think aloud• Tell us when you have completed the task or you give up

Test Protocol• Each task was timed. • Each task was completed successfully/unsuccessfully. Testers could

give up.• After all tasks were completed on one doc, testers rated the content.

Scale: from 1 to 5. • After all tasks were completed for both docs, testers rated their overall

experience. Scale: from 1 (poor) to 7 (great).• Data recorded in a Google form• Sessions recorded with permission

Sample taskYou are a operations manager put in charge of monitoring the system.  Task: Use the user guide to determine your two areas of responsibility in terms of configuration.

?

Percentage of tasks completed correctly

Document 1 Document 2 Document 3 Document 40%10%20%30%40%50%60%70%80%90%

100%

Old Doc New Doc

Results – Document 1 (Average before & after)

Simple 1 1.5 2 2.5 3 3.5 4 4.5 5 Complex

Effective

1 1.5 2 2.5 3 3.5 4 4.5 5 Ineffective

Clean 1 1.5 2 2.5 3 3.5 4 4.5 5 Confusing

Clear 1 1.5 2 2.5 3 3.5 4 4.5 5 Cluttered

Valuable

1 1.5 2 2.5 3 3.5 4 4.5 5 Not Valuable

Simple 1 1.5 2 2.5 3 3.5 4 4.5 5 Complex

Effective

1 1.5 2 2.5 3 3.5 4 4.5 5 Ineffective

Clean 1 1.5 2 2.5 3 3.5 4 4.5 5 Confusing

Clear 1 1.5 2 2.5 3 3.5 4 4.5 5 Cluttered

Valuable

1 1.5 2 2.5 3 3.5 4 4.5 5 Not Valuable

Results – Document 2 (Average before & after)

Simple 1 1.5 2 2.5 3 3.5 4 4.5 5 Complex

Effective

1 1.5 2 2.5 3 3.5 4 4.5 5 Ineffective

Clean 1 1.5 2 2.5 3 3.5 4 4.5 5 Confusing

Clear 1 1.5 2 2.5 3 3.5 4 4.5 5 Cluttered

Valuable

1 1.5 2 2.5 3 3.5 4 4.5 5 Not Valuable

Simple 1 1.5 2 2.5 3 3.5 4 4.5 5 Complex

Effective

1 1.5 2 2.5 3 3.5 4 4.5 5 Ineffective

Clean 1 1.5 2 2.5 3 3.5 4 4.5 5 Confusing

Clear 1 1.5 2 2.5 3 3.5 4 4.5 5 Cluttered

Valuable

1 1.5 2 2.5 3 3.5 4 4.5 5 Not Valuable

Results – Document 3 (Average before & after)

Simple 1 1.5 2 2.5 3 3.5 4 4.5 5 Complex

Effective

1 1.5 2 2.5 3 3.5 4 4.5 5 Ineffective

Clean 1 1.5 2 2.5 3 3.5 4 4.5 5 Confusing

Clear 1 1.5 2 2.5 3 3.5 4 4.5 5 Cluttered

Valuable

1 1.5 2 2.5 3 3.5 4 4.5 5 Not ValuableSimple 1 1.5 2 2.5 3 3.5 4 4.5 5 Complex

Effective

1 1.5 2 2.5 3 3.5 4 4.5 5 Ineffective

Clean 1 1.5 2 2.5 3 3.5 4 4.5 5 Confusing

Clear 1 1.5 2 2.5 3 3.5 4 4.5 5 Cluttered

Valuable

1 1.5 2 2.5 3 3.5 4 4.5 5 Not Valuable

Results – Document 4 (Average before & after)

Simple 1 1.5 2 2.5 3 3.5 4 4.5 5 Complex

Effective

1 1.5 2 2.5 3 3.5 4 4.5 5 Ineffective

Clean 1 1.5 2 2.5 3 3.5 4 4.5 5 Confusing

Clear 1 1.5 2 2.5 3 3.5 4 4.5 5 Cluttered

Valuable

1 1.5 2 2.5 3 3.5 4 4.5 5 Not Valuable

Simple 1 1.5 2 2.5 3 3.5 4 4.5 5 Complex

Effective

1 1.5 2 2.5 3 3.5 4 4.5 5 Ineffective

Clean 1 1.5 2 2.5 3 3.5 4 4.5 5 Confusing

Clear 1 1.5 2 2.5 3 3.5 4 4.5 5 Cluttered

Valuable

1 1.5 2 2.5 3 3.5 4 4.5 5 Not Valuable

Average overall ranking – 1 poor to 7 great

Document 1 Document 2 Document 3 Document 40

1

2

3

4

5

6

7

Old Doc New Doc

• Content located faster in 3 of 4 new models. • Validation tasks completed with increased

success in 2 of 4 new models. • Content rating higher in 3 of 4 new models.• Overall content ranking higher in 2 of 4 new

models.

Laugh and cry moments• Surfing & browsing• Clueless & perfectionist• Change haters• Fear of failure• When is done really done• Aha moments• Technical snafus• Testing heavy-duty reference

content was a bust

Customers are their biggest advocates. What the writers learned

Lessons the writers came up with• Do a dry run

• Observing the users’ choices can be as useful as the data

• Cannot predict how users will do the tasks

• Define what “done” means

• Reference-based content should not be tested with the same methodology as task-based content

• Rebooting your computer prior to testing has its benefits

• Repeated contact with customers removes the fear factor over time

1. We require user research and content validation, where appropriate.

2. We get creative…..• Collaborate with people who work with customers• Monitor and mine data from social networking sites• Join LinkedIn groups - ask questions & post surveys• Seek input from people who represent the same personas as our users• Participate in customer-focus groups (Design Partner Programs)

Re(evolution)

What are you waiting for?

Pam Noreault• pam.noreault@aciworldwide.com• pamnoreault@gmail.com• @pnoreault (Twitter)• Pam Noreault (LinkedIn)

Tara Knapp• tara.knapp@aciworldwide.com• taraknapp@gmail.com• @tara_knapp (Twitter)• Tara Knapp (LinkedIn)