Hui-Hsien TsaiJiazhen Wang

University of Missouri

Using e-learning to deliver training material becomes a trend in business (Neidj & Wolpers,2004).Higher dropout rates and users easily felt frustrated.Most Influential factor: poor usability design of e-learning applications

2

To what extent does the information structure and site functionality meet heuristic standards as evaluated by usability experts?

How efficient and effective are learners in finding and retrieving needed training materials as requested in the usability testing protocol?

How successfully do they complete the task?

How much time does it take to complete the task?

How satisfied are learners with the web site as a training and support site?

Content: self-paced Microsoft Excel 2003 e-learning lessonsDemographic data:

Participants 2 experienced usability testing users

4 staff membersGender 2 males

4 femalesAge 25 to 55 years old

Microsoft Excel Experiences

2 advanced users 2 novice users

Research Questions Data Collection Analysis

1. To what extent does the information structure and site functionality meet heuristic standards as evaluated by usability experts?

• Heuristic Evaluation • Descriptive Analysis

2. How efficient and effective are learners in finding and retrieving needed training materials as requested in the usability testing protocol?

• Exploratory Test • Descriptive Analysis

3. How satisfied are learners with the web site as a training and support site?

• Interviews • Qualitative analysis

Adapted from Jacob Nielsen (2005), Deniese Pierotti (2003), and align with the five perspectives of usability in e-learning from Mehlenbacher et al. in 2005.

12 categories with 84 criteria.

Overall, the Microsoft Excel e-learning lessons only met 57.3% of the heuristic evaluation criteria.Categories with lowest percentage:

User Control and Freedom (33.3%); Error Prevention (33.3%); Help and Documentation (33.3%); andPleasurable and Respectful Interaction with the user (37.5%).

Purpose: efficiency, effectiveness, and the satisfaction of the Microsoft Excel e-learning lessons from users’ perspectives

6 major tasks with 14 subtasks

User Exploratory Test Results

General Rating: Easiest: Setting up a Workbook;Hardest: Performing Calculations on Data;

Task Completion Rate: Hardest task: Finding the location of the animated instruction in formatting a chart (25%)

User Exploratory Test Results(Cont.)

Time on Tasks: The most time-consuming task: creating a chart in a new worksheet; The second time-consuming task: Finding the instruction of errors in calculation

Navigation Patterns: Different navigation patterns;Participants’ navigation patterns influence their performance and their navigation patterns are based on their understanding of the website functionality.

Interviews

Overall Impression: In general, all of these participants did not know this website;Half participants think this website is helpful;Experienced users think overall of the MELL website is good;Suggestions from participations: e.g. Increase font size, especially for headings

RecommendationsBased on User Exploratory Test and Interviews:

Avoiding the terms that novices are not familiar with, such as smart tags; Ordering the sequence of lessons, according to novices’ mental models;Using correct colors, such as headings;Reducing the white spaces in the main page;Increasing the font size