web analytics and usability testing
DESCRIPTION
Triangulate your way to better design recommendationsTRANSCRIPT
Web Analytics and Usability Testing:
Triangulate your way to better design
recommendations by Eva Kaniasty
What’s your favorite method?
Web Analytics Explosion
Who wins the battle?
Usability Testing Sucks
Usability Testing
Small SampleResults not Generalizable
Artificial SituationTask Design Bias
Recruiting BiasUnrepresentative Participants
Moderator Required
Moderator Bias
Usability Testing
3 users experienced X…Consider changing X to Y.
Usability Testing is Disconnected
Web Analytics are Awesome!
Web Analytics
Representative/ Large Sample Size
Real GoalsReal Tasks
Real Users“In the Wild”
No ModeratorAutomatic Data Collection/Analysis
Numbers and graphs: universally beloved
Analytics rundown
Visitor/Session Tracking
Visitor/Session Tracking
Data Gathered Uses•Some visitor characteristics: locations, browsers, OS•Visit & page view trends•Engagement measures: time on page•Search data: keyword referrals•Custom tracking: detailed information on paths/goal conversion
•Confirm Usability Test findings•Inform usability testing tasks
Clickmaps/Heatmaps
Clickmaps/Heatmaps
Data Gathered Uses
•Info not on just which pages are being accessed, but from where•Full picture of users’ interactions with the page•Measure of user interest/engagement •Some segmentation by browser/screen size/time to click
•IA•Visual hierarchy•Interface refinement
Email Data Mining
Email Data Mining
Data Gathered Uses
•Demographics•Geography•User Profiles•Site usage data: accounts on amazon, facebook, linkedin, etc.
•Utest Recruiting Profiles•Personas
Session Recording
Session Recording
Data Gathered Uses•View all interactions with page, including mouse movements •Scroll maps•Form analytics
•Identify potential usability problems•QA/Browser Testing
A/B or Multivariate Testing
A/B or Multivariate Testing
Data Gathered Uses
•Relative success of design alternatives: impact on conversion and other engagement measures
•IA•Visual hierarchy•Interface refinement
Unmoderated Usability Testing
Unmoderated Usability Testing
Data Gathered Uses
•Immediate feedback from real users as they engage with the website
•Alternative to usability testing, when large samples are required•Compensate on weaknesses of survey methods
Custom Metrics
Custom Metrics
Data Gathered Uses
•Functionality usage•Form errors•Patterns of use•Conversion measurement
•Sky is the limit
The what.
But why?
Analytics Suck
Web Analytics
Limited information about context No info about user intent
One-dimensional view (i.e. Page-level data only from Clickmaps)
Overwhelming amounts of data to analyze
Data on current state only
Usability Testing is Awesome
Usability Testing
Rich data/ holistic view
Info about goals
Moderator can probe to clarify intent
Ability to test new interfaces/ prototypes
Triangulate!
Triangulate
• Focus• Deep Data• Goals and Intent
• Confirm Problems• Extent• Severity
Usability Testing
Analytics
Case Study
• Form errors• No abandonment• Confusion about
messaging
• Detail on types/extent of errors
• True abandonment rates
• Visual hierarchy problems
Usability Testing
Analytics
Triangulate
Surveys
AnalyticsUsability Testing
Questions first
CSI or Law & Order?
Resources
• Google Analytics• Crazyegg• Clicktales • Keynote Systems