testing content: early, often, & well
DESCRIPTION
You've heard about testing design. What about testing content? This presentation walks through a case study of testing content with people and offers key testing takeaways.TRANSCRIPT
Testing Content
Early, Often, & Well
Colleen Jones [email protected]
Kevin O’Connor [email protected]
Testing design.
You’ve probably heard of it.
It’s good. But, it’s not enough.
We’ve talked about testing content for years, but as an industry...
Too many usability tests focus ONLY on finding information—not on how the information [content] itself works for people.
－ Ginny Redish
“ “
Content includes the text, graphics, video, & audio that make up an interactive experience.
－ Kristina Halvorson
“ “
What we mean by “content.”
THE SITUATION
CDC Travelers’ Health wanted to assess their content.
TREND: Content is critical to the health industry.
…providers of online health resources can increase the relevance & influence of their Web sites if they • focus on site usability, • bolster the credibility of their content, & • bridge the gap between Web & point of care.
－ Who Cares About Online Health Content? Forrester Research
“ “
The site is a big opportunity to improve health decisions through content.
• Top 5 most popular CDC websites
• In 2009 • 9.5 million visits • 31 million page views
With such rich content on our site, I was concerned about the number of people who were calling because they couldn’t find vaccine information.
－ Kelly Holton Communication & Education Team Lead Travelers’ Health, CDC
“ “
I knew it was time to do something different to help drive our content strategy. I felt we needed to find out what users really think about & do with our content.
－ Kelly Holton Communication & Education Team Lead Travelers’ Health, CDC
“ “
CDC had clear objectives.
• Gather feedback about different user groups’…
• Content needs & preferences.
• Perspectives on content quality, presentation, & format.
• Create a content and usability testing model.
OUR APPROACH
Testing content helps you choose the right content direction.
We tested content iteratively.
Baseline Test Concepts Validation Test
BASELINE TEST
We focused on a critical content sample:
DESTINATIONS
We tested with these user groups.
Healthcare Professionals Doctors, nurses, & pharmacists who are in a posi3on to provide travel health-‐related informa3on & care (N=12)
Travelers
Consumers who do or do not research health-‐related informa3on before traveling to foreign countries (N=12)
Travel Professionals
Travel & tourism industry representa3ves (e.g., travel agents, tour organizers, etc.) (N=6)
Travel Medicine Professionals Doctors, clinic admins, nurses, & specialists provide travel health informa3on & care (N=6)
We observed & interviewed people one-on-one.
We needed a deep understanding of WHY.
Our protocol focused on key content questions…
Can users… • Find & read the content they need?
• Understand the content?
• Act on the content?
We learned a lot.
Can they find & read it? NO • Users couldn’t find vaccinations.
Do they understand it? PARTLY • Users were not sure whether the content was specific to the
destination.
• Users were distracted by & misunderstood "Travel Notices.”
Can & will they act on it? NO • Users trusted CDC but…
• Felt overwhelmed by the content.
• Were confused about which vaccinations were required, necessary, or most relevant to their level of risk.
• Were not clear on the next step they should take.
CONFUSING DATES RISK LEVELS?
AMBIGUOUS LANGUAGE
HARD-TO-READ FORMATS
TMI
PORTABLE FORMATS?
Lost in Content
We recommended focusing on travelers.
Travelers
Consumers who do or do not research health-‐related informa3on before traveling to foreign countries (N=12)
• Other sections, such as Yellow Book, addressed travel medicine professionals better.
• 80 / 20 rule applies.
• Health professionals & travel professionals act as catalysts or referrals.
THE CONCEPTS
Travelers need to decide… • Whether they are at
risk.
• What shots to get.
• How to get shots.
• How to avoid common diseases.
• What health items to bring.
• More
Design from the content out.
－ Jeffrey Zeldman
“ “ That’s exactly what we did.
We explored content…
• Organization & priority
• Layering & layout
• Writing style & tone
• Calls to action
5 organization schemes
Priority on vaccinations
Tone & language
Rhetorical / psychological framing
Layering / progressive disclosure
Decision table
Call to action
Then, we tried variations.
Concept 1 Concept 2
Layering & layout
• Less intro instruction • Tabs • More explanation in
vaccination summary
• More intro instruction • Reveal • Less explanation in
vaccination summary
Calls to action • Bold buttons • Contextual text
Style & tone • More familiar • Less familiar
Does this content work better for travelers?
TESTING THE CONCEPTS
We tested 2 concepts + the original.
…by presenting users with alternative…solutions, subjective ratings are less prone to inflation & give rise to more & stronger criticisms.
−Getting the Right Design & the Design Right CHI 2006
“
“
The protocol was similar to the baseline.
• Can they find and read it?
• Do they understand it?
• Will they act on it?
We didn’t expect a “winner,” just better feedback.
We followed best testing techniques.
• Tested on a small scale.
• Avoided order bias.
• Followed a discussion guide.
• Took time stamped notes & looked at patterns of behaviors.
The concepts tested better.
Can they find it? YES
Do they understand it? MOSTLY
• Users understood the vaccination content but….
• Asked for more explanation of the ratings.
• Were still distracted by & misunderstood "Travel Notices.”
Can and will they act on it? YES
• Users preferred more explanation in the decision table.
• Users knew the next step they should take.
• Despite tone & framing, users still trusted the content.
Preference for Concept
What’s Next
• Testing • Analysis • Synthesis
1 Plan
• Concept • Testing
2 Create
• Refinement • Testing
3 Deliver
• Evaluation • Quality Control
4 Govern
Process Inspired by Content Strategy for the Web
We’re not done, but we know we’re on the right track.
Top Takeaways for Testing Content
1. Test the content early in a project, then iteratively.
2. Test for whether people can use, understand, and act on the content.
3. Test using a mix of concrete tasks & exploratory questions.
4. Test with the right people.
5. Test more than one concept for the best feedback.
Find out whether your content works for people.
And learn all you can from the journey.
Acknowledgments
• Centers for Disease Control & Prevention (CDC)
• Oak Ridge Associated Universities
• Kim Ware, Content Science
• Karen Williams, User Insight