usability assessment, round two: re-examining the uw libraries information gateway jennifer ward...
DESCRIPTION
Looking Back Move to Web interface for most services – early 1998 Rapid design, testing, and prototyping of website Only formal usability testing was by students as a class assignmentTRANSCRIPT
Usability Assessment, Round Two:Re-Examining the UW Libraries
Information Gateway
Jennifer WardUniversity of [email protected]
Presentation Overview
Background and History Current environment Hardware, personnel Efforts to date Lessons learned and futures
Looking Back Move to Web interface for most
services – early 1998 Rapid design, testing, and prototyping
of website Only formal usability testing was by
students as a class assignment
Current Environment
Web Steering Committee (WSC) established August, 2000
Decentralized website management Approximately 170 staff in 40 units with
write permissions No testing on current interface
Lab and Support Usability lab
– Other labs on campus– Currently in a shared conference room; will
have dedicated space soon– PC with Camtasia, video camera
Half-time graduate student dedicated to usability efforts– Brought a fresh perspective to the process
Work with WSC to determine priorities System-wide support for assessment
Summer, 2001 Online Survey
July 23 – August 13, 2001 49 unique responses; almost 1/3 each
faculty, students, staff Goal was to get a sense of what users
thought of the Gateway Based on a previous survey Used WebQ survey software, linked
from home page
Summer, 2001 Survey Results Asked a lot of open-ended questions
about navigation and features Nearly 80% successfully found what
they wanted– “Once one becomes familiar with the
site, it’s easy to navigate.” Library Catalog most important resource
– Most requested new features focused on catalog functionality
Points to Ponder
Timing is everything!– Low response rate
Data quality– Many open-ended questions went
unanswered– Too focused on navigation and features;
need to broaden it
Autumn, 2001 Online Survey September 21 – October 20, 2001 Same goal as before, but do a better job
this time Plugged in online campus newsletter Incentive was provided, but not
advertised 131 unique responses
– 10% faculty, 28% staff, 27% grad students, 27% undergrads, 5% alumni, and 2% non-affiliates
Autumn, 2001 Survey Structure Restructured summer survey so we’d
get more answers Ran past a couple of local experts for
feedback 15 questions, multiple choice or open-
ended Users could remain anonymous
– Didn’t get affiliation, which would have been nice for data analysis
Autumn, 2001 Survey Results
Participants for follow up studies Part of site that drew the most
comments was our catalog (!!) Overwhelming response – it’s not
“broken”– Almost 80% very satisfied or satisfied– Some areas need work and we’re
addressing them– “Micro” level changes
Autumn, 2001 Survey Results
Asked what new feature users wanted:– Some things were already on our site;
looked at making them more prominent– Database selection wizard/helper– Online tutorial for Gateway and catalog– Better integration with campus
authentication– $100 put into their bank account
Autumn, 2001 Survey Results Data clumping Wording and Navigation Databases and Journals
– Show vendors on database list– Need an “all-purpose” database instead of
having so many– No clear winner on hierarchical vs
alphabetical listing of databases and e-journals
Card Sorting November 13 – December 6, 2001
– Developed and started recruiting while we were waiting for online survey to conclude
To determine the significance of features of the Gateway and how users would organize and name these features
Standardized procedures and documentation– Consent forms, scripts, task lists
Participants– one undergrad, one grad, one faculty, and
two staff members (not what we wanted)
Card Sorting – Tasks Pilot Test Think aloud protocol, videotape, index
cards Two sets of tasks with 46 cards Rank according to perceived
importance– Students more flexible than faculty/staff
Group according to their own navigational structure– Change wording if they wanted
Card Sorting – Results Statistical analysis (dendrogram)
– More significant with more participants Items in upper right are more important Confusing wording (i.e., Cascade,
Connecting, Starting Points) Didn’t understand News and Current
Events Have not used data in redesign
“By Subject” Pages One of the “broken” items from the survey
and WSC’s perspective Politically hot topic How to help beginning researcher get started,
also try to address the layout Hierarchical vs alphabetical presentation Recruitment
– Targeted email, undergrads from library units
http://www.lib.washington.edu/subjects/
“By Subject” Pages – Tasks Think aloud protocol, captured screen
activity with software Write answers to questions as they go
along Mockups on a test server Undergrad tasks varied – did they use
“Core Resources”? Graduate student and faculty tasks were
targeted to their discipline
“By Subject” Pages – Results Undergraduates
1. Wanted listing by class name, searched Yahoo! for everything (more comfortable)– Wanted ranking system (4 stars)
2. Expert researcher who never left the test pages; completed tasks in 2-3 minutes– Understood and liked “Core Resources”
3. Stayed with test pages except when he felt lost, then returned to his favorite search tool – Google– Drawn to resources with the folder icon
“By Subject” Pages – Results
Graduate Students1.Went there once, wasn’t satisfied and
never came back2.Not inclined to leave the pages; weakest
researcher Faculty
1.Resources were already bookmarked
“By Subject” – Other Efforts
We’re working on adding a survey to the bottom of these pages
Search box on each page Simplify the design by removing
explanatory blurbs at top
Lessons Learned
However long you think something will take, multiply the time by 3 and you might be close
Standardize the process as much as possible; use templates and set guidelines
Refine tests before the involving users – run pilot tests or talk with experts
Timing, timing, timing Try not to put something online unless you
know it works. If users don’t find what they’re looking for the first time, they don’t come back.
Questions?
The website:http://www.lib.washington.edu/usability/
Contact us:[email protected] or [email protected]