merging methodologies: individual and group card sorting
DESCRIPTION
This presentation is a case study describing how we combined individual card sorts with focus groups and group card sorting to improve the content hierarchy and organization of www.libertymutual.com, the personal insurance website of Liberty Mutual, which customers can visit to get an insurance quote, service their insurance policies, or find insurance-related information. We analyzed quantitative and qualitative data from 26 participants, on which we based our recommendations for a new hierarchy and site structure. Our paper will show how the results from the individual and group sorts differed, how the individual exercise informed the group exercise, and how the group exercise informed the recommendations. We believe this combination of individual sorting, group sorting, and focus group discussion makes this methodology unique. The paper on which this presentation is based is available here: http://bit.ly/14jWa7GTRANSCRIPT
Merging Methodologies: Combining Individual and Group Card SortingRobert L. Thomas and Ian JohnsonLiberty Mutual, Personal Insurance, Boston, MA[robertl.thomas, ian.johnson]@liberty.mutual.com
Personal Insurance
Liberty Mutual Insurance 2
Our old-style navigation was no longer useful for a major insurance company’s website
Historyo www.liberymutual.com
had moved over the years to
a large site with multiple
lines of business
Issueso Display of a total of only
nine options in the top right
corner of each page
o Lack of a traditional
navigation bar
Proprietary - Trade Secret (Competitively Sensitive Information)
Liberty Mutual Insurance 3
We needed to make content discoverable and easy to find
Scopeo Create a new navigational
hierarchy to replace the current one
High-Level Objectiveso Focus on the navigation and
organization of the website
o Ensure the navigation labels are understandable
o Make the site attractive to customers
Proprietary - Trade Secret (Competitively Sensitive Information)
Liberty Mutual Insurance 4
To meet the needs of the project, we needed to run a card sort, a categorization exercise where users group physical or virtual cards together
• “Card sorting is excellent for situations where you want the users’ mental model to drive the information architecture of the product.”- Courage & Baxter (2005)
• “Open sorts are used for discovery. Closed sorts are used for validation.”- Morville & Rosenfeld (2006)
Proprietary - Trade Secret (Competitively Sensitive Information)
Liberty Mutual Insurance 5
We reviewed research on card sorting methodologies and then developed a new methodology
Proprietary - Trade Secret (Competitively Sensitive Information)
• Repeated card sorting“Repeated sorting of a set of items to understand underlying dimensions or characteristics of a product or service” - Curran, Rugg, Corr (2005)
• Delphi and modified Delphi card sorting“Participants build on the results from previous card sorts” - Paul (2008)
• Focus group card sorting“Participants… complete the card sort individually. Then, [you] lead a discussion with [them] regarding the… strategies they used….” - Hawley (2008)
We developed a new methodology, one that had not been done before.
Simple to run
Requires only 10-20 people
Useful for understanding emotional dimensions of a product
Not useful for grouping information into different categories
Reduces time
Requires only 8-10 people, but…
Subsequent participants are modifying a card sort others started
UX research experts recommend 15-30 participants for a card sort
Collect quantitative card sort data
Collect qualitative insights
Focus group discussion may be biased by group think
Be careful about running a card sort with only 1 group of 8-15 participants
Liberty Mutual Insurance 6
Our methodology combined online card sorting, physical card sorting, and focus group discussion with 26 participants split between 2 sessions
Proprietary - Trade Secret (Competitively Sensitive Information)
Source: http://www.flickr.com/photos/cflc/2098395894 (far left); Schlesinger Associates, New Jersey (far right)
• 13 laptops & 13 participants• 79 card sort labels• WebSort online card sort tool• 45-minute session• 15-minute break
• 3 groups of 4-5 participants• 79 labels on Post-It’s• 1 moderator & 1 note-taker• Group spokesperson
presented• 40-minute session
• 1 moderator & 5 note-takers• Discussion topics• 20-minute session
Individual, Online, Un-moderated, Open Card Sort
Team-based, Physical, Moderated, Open Card Sort
Focus Group Discussion
Liberty Mutual Insurance
Breaking into teams, we followed a four-step analysis process to arrive at our final navigation categories
Proprietary - Trade Secret (Competitively Sensitive Information) 7
Group Card Sort Team
Individual Card Sort Team
1Standardized
Labels & Combined Categories
2Compared
Grouping of Labels
3Created &
Compared Site Maps
4Finalized
Navigation Categories
Source: Illustrations by Jeffery Callender; Template by Livia Labate
Liberty Mutual Insurance
Standardized labels and combined categories
Proprietary - Trade Secret (Competitively Sensitive Information) 8
Sorter Original category Standardized categorySort20 About About the CompanySort7 About Our Company About the CompanySort17 About the Company About the CompanySort23 About the Company About the CompanySort19 About Us About the CompanySort21 About Us About the CompanySort8 ABOUT US About the CompanySort9 about us About the CompanySort22 company About the CompanySort1 Company and website info About the CompanySort10 Company Info About the CompanySort2 Company Info About the CompanySort24 company info About the CompanySort25 Company info/history About the Company
1 2 3 4
Table 1. Standardized Categories for Individual Card Sorts
Liberty Mutual Insurance
Table 2. Final list of 11 in-person, group standardized categories
Proprietary - Trade Secret (Competitively Sensitive Information) 9
Sorters Who Total Cards in Unique Standardized Category Used This This Category Cards Agreement
Standardized labels and combined categories
1 2 3 4
Average of 13
cards (40÷3) in this category
Liberty Mutual Insurance
Group Card Sort Results Dendrogram
Proprietary - Trade Secret (Competitively Sensitive Information) 10
Compared grouping of labels 1 2 3 4
Information
Tasks
Liberty Mutual Insurance
Home
Our Company
Policy, Billing & Advice
Types of Insurance
Individual Card Sort Site Map
Created and compared site maps
1 2 3 4
Liberty Mutual Insurance
Home
About Us Auto Insurance
Home Insurance
Insurance Basics
Insurance Products
Claims Center
Group Card Sort Site Map
Created and compared site maps
1 2 3 4
Liberty Mutual Insurance
Finalized navigation categories
Proprietary - Trade Secret (Competitively Sensitive Information) 13
1 2 3 4
Liberty Mutual Insurance 14
In summary, the combination of individual card sorts, group card sorts, and focus groups was successful
Our new methodology allowed us to:o Obtain qualitative and quantitative data
o Recruit a large sample size
o Save money and time
The benefit of this unique method is that the individual card sort activity enriched the group card sort activity and focus group discussion
Proprietary - Trade Secret (Competitively Sensitive Information)
Liberty Mutual Insurance 15
In conclusion, two years later, the site navigation has had a positive effect on the business
Key performance indicators and web analytics demonstrate that customers are actively using the new navigation
In multiple usability tests since the navigation update, users have commented positively on the navigation, rating it as one of the best attributes of the site
J.D. Power and Associates stated that “Liberty Mutual provide[s] menus that [allow] shoppers to easily find relevant shopping tools on any page”
Proprietary - Trade Secret (Competitively Sensitive Information)
Proprietary - Trade Secret (Competitively Sensitive Information) 16