september 29, 2011 case study: usability testing for it training website redesign project jae kim,...
TRANSCRIPT
September 29, 2011
Case Study:Usability Testing for IT Training
Website Redesign Project
Jae Kim, Jenny Hertel, Greg Hanek, Tom Mason, Chris Payne
The User eXperience Group (UXG) assists IT teams improve the user experience of their products and services through a variety of user-centered design and evaluation methods, including:
• Expert Reviews• Information Architecture Reviews• Design Critiques• Prototypes and Mockups• Focus Groups• User Testing
User TestingWHAT
UXG Consultants observe people using a system as they attempt to complete a set of representative tasks
WHY
1. Through user testing, we identify problems and issues; we then provide a report outlining suggested improvements and changes.
2. User testing is the best way to determine how well people interact with your system. It can provide valuable insights into the problems that real users might have, and offers the greatest potential for improving the usability of your site.
Instructor-led workshops
• Nearly 1,000 per year• 10,000+ participants per
year• 95% Very Good/Good rating
Self-paced training
downloadableworkshopmaterials
Why Usability Testing?
• Usability testing is essential to ensuring good customer experience.
• IT Training first tested its site in 2001 when introducing its first dynamic web site.
• The tests we’ll talk about today were conducted in 2009-2011.
• Goals were to make it easier for users to find training and to make the site more engaging.
Results
What Was Working
• TASK: Find a specific workshop• OBSERVATION 1: Specific workshops
were easy to find using the search on the IT Training website
• OBSERVATION 2: Specific workshops were easy to find using Google search
Image source: prlog.org
Search:
What Was Working
Registration:
• TASK: Register for a workshop• OBSERVATION: 90% of testers
and ~80% of all ‘real’ users were able to register for a class in 2009
2009 2011
EASY HARD FAIL
What Was Working
User Profile
• TASK 1: Update your profile
• TASK 2: Find a list of previously attended workshops
• OBSERVATION: Both tasks were easily completed.
‘09 ’11 ‘09 ‘11
EASY HARD FAIL
User Testing Video Clips
1b. Confusing Navigation -
Lack of clear path to main content1a. Confusing Navigation - Unintuitive modality-centered navigation
2. Workshop Pricing
3. The Shopping Cart
4a. Certificates and Certifications -
Unclear certification options
4b. Certificates and Certifications -
Certification comparison table
VIDEO CLIP
What was NOT working?
• TASK: Find a specific workshop
• OBSERVATION: We expected users to use the online listing of workshops to complete this task, but many instead downloaded the PDF catalog
1a. Confusing Navigation – Lack of clear path to main content:
2009 2011
EASY HARD FAIL
User Testing Video Clips
1a. Confusing Navigation -
Lack of clear path to main content
1b. Confusing Navigation - Unintuitive modality-centered navigation2. Workshop Pricing
3. The Shopping Cart
4a. Certificates and Certifications -
Unclear certification options
4b. Certificates and Certifications -
Certification comparison table
VIDEO CLIP
What was NOT working?
• TASK: Find ALL self-paced training options
• OBSERVATION: Users either couldn’t find any options, or they found only one and assumed there were no more
1b. Confusing Navigation –Unintuitive modality-centered navigation:
2009 2011
EASY HARD FAIL
Old IT Training Home Page
Old IT Training Site – Navigation Expanded
Old IT Training Site – Instructor Led by Topic
User Testing Video Clips
1a. Confusing Navigation -
Lack of clear path to main content
1b. Confusing Navigation -Unintuitive modality-centered navigation
2. Workshop Pricing3. The Shopping Cart
4a. Certificates and Certifications -
Unclear certification options
4b. Certificates and Certifications -
Certification comparison table
VIDEO CLIP
What was NOT working?
• TASK: Find the cost of a workshop• OBSERVATION:
o Users expected the cost to be near the workshop description
o Link for the pricing opened the FAQ page, not the specific article
2. Workshop Pricing:
2009 2011
EASY HARD FAIL
User Testing Video Clips
1a. Confusing Navigation -
Lack of clear path to main content
1b. Confusing Navigation -Unintuitive modality-centered navigation
2. Workshop Pricing
3. The Shopping Cart4a. Certificates and Certifications -
Unclear certification options
4b. Certificates and Certifications -
Certification comparison table
VIDEO CLIP
What was NOT working?
• TASK: Register for a workshop
• OBSERVATION: The fix for the navigation problem found in the 2009 test resulted in a screen layout change that called for better indication that an item was added to the cart
Image source: modmyi.com
3. The Shopping Cart:
User Testing Video Clips
1a. Confusing Navigation -
Lack of clear path to main content
1b. Confusing Navigation -Unintuitive modality-centered navigation
2. Workshop Pricing
3. The Shopping Cart
4a. Certificates and Certifications -
Unclear certification options4b. Certificates and Certifications –
Certification comparison table
VIDEO CLIP
• TASK: Find all certification and certificate options.
• OBSERVATION: o Users couldn’t find all
accreditation options, “settling” on one or two
o Were not confident if what they found satisfied the task
What was NOT working?
4a. Certificates and Certifications – Unclear certification options
2009 2011
EASY HARD FAIL
User Testing Video Clips
1a. Confusing Navigation -
Lack of clear path to main content
1b. Confusing Navigation -Unintuitive modality-centered navigation
2. Workshop Pricing
3. The Shopping Cart
4a. Certificates and Certifications –
Unclear certification options
4b. Certificates and Certifications -
Certification comparison table
VIDEO CLIP
• TASK: Find all certification and certificate options.
• OBSERVATION: Comparison table was useful, but was difficult to find
What was NOT working?
4b. Certificates and Certifications - Certification comparison table
2009 2011
EASY HARD FAIL
Task Success Rates
’09 ’11 ’09 ’11 ’09 ’11 ’09 ’11 ’09 ’11 ’09 ’11 ’09 ’11 ’09 ’11 ’09 ’11
EASY HARD FAIL
Next Steps
• Get user profile information from central systems.
• Users indicated they didn’t know why they need to have an IT Training profile
• Will align with IU systems and user expectations
• Addressed in near term through FAQ
• Make it easier to find old versions of materials.
• Current site tells users to contact us.
• Need to make self-service.
Next Steps
• Events calendar
Other new features to come
• More filtering
Other new features to come
• Deep linking into 3rd party elearning where possible.
Other new features to come
• We will use usability testing to make sure we build these features the right way!
Where do we go from here?
• Usability testing is a very valuable tool• It helps us see what is going on with our
changing audience, instructional delivery and business model
• It has already helped us greatly improve our customer’s experience and we know it will continue to do so
• We believe in it and we teach it!
Conclusions
QUESTIONS?
User eXperience Group
IT Training