U.C. Berkeley Calendar NetworkUsability Evaluation

Nadine Fiebrich & Myra LiuIS214

May 4, 2004

Introduction

Background

Description of BCN toolhttp://dream.sims.berkeley.edu/EventCalendar/Documents/ThirdInteractivePrototype/login.php

Usability Methods1. Interviews2. Usability Tests3. Survey

Method #1: Interviews

Conducted in January as part of needs assessment processGoalsTo gather information about:

Current calendar/event management processes Technical expertise and development capabilities Likes and dislikes about current system/methods Communicating events within the UCB community Event data Recommendations

Participants 12 participants 3 technical sophistication categories (high, medium,

low)Procedure

1.5 hours per interview 2 group members - one conducting interview, one

taking notes

Method #1: Interviews

Summary of results Current process Last minute changes No security concerns Private vs. public Archived events Push vs. pull Sufficient data model Personal calendars Maintain ‘look and feel’

Method #2: Usability Tests

Conducted several rounds of usability testing from early March to late AprilGoals

Ensure that participants can find and use the various functions in the interface

Determine whether the functionality provided is valuable

Determine whether the interaction flow is natural and intuitive and matches current user workflow

Determine whether the application will meet the needs of these calendar administrator

Participants 3 participants - 2 calendar administrators, 1

calendar content manager All participants are familiar with web-based

applications and other calendaring toolsProcedure

3 task scenarios

Method #2: Usability Tests

Summary of results Task Time Completion

Participant Scenario 1 Scenario 2 Scenario 3

1 3m48s 4m16s 6m05s

2 3m17s 4m28s 6m09s

3 3m55s 4m45s 5m14s

Avg. Time/Task

3m40s 4m30s 5m49s

Method #2: Usability Tests

Summary of results Event Manager

Users had trouble finding some functions in the sub-nav

Users would like to be able to search all tabs Users did not notice confirmation messages

Format Calendar Need more explanatory text Clarify terminology

Event Details The flow from this page doesn’t make sense if

the user doesn’t want to take any action Create Event form

Need more explanatory text

Method #3: Survey

Goals Capture participants overall satisfaction Gather opinions on specific areas of functionality Gather standard set of responses that can be

compared across participants

Participants 3 participants - 2 calendar administrators, 1

calendar content manager

Procedure After our usability tests, we emailed our test

participants with a follow-up survey in a Word document

Method #3: Survey

Summary of results Users found the system relatively easy to use,

though more explanatory text and help is needed in many areas

Users thought the system would be valuable to their department as well as other campus departments

Users thought the functionality provided was valuable, but some functions more than others depending on their needs

Conclusions

Specifically the usability methods have been useful for:Interviews

Used interview data to design initial workflow and supported tasks of our application

Determine how our tool can fit into the UCB community

Gathered ideas for functionality

Usability Tests Conducted usability testing for each iteration of our

design Used results to improve specific parts of the

interface

Survey Get standardized user ratings of the system Use results to prioritize development of

functionality

Lessons Learned Craft the language used in task scenarios for

user testing carefully, as unclear language can cause unnecessary user confusion

You can iterate a lot longer than you might think and continue to improve your design

You continue to learn from usability testing even with just a few users

Interviewing users after you have a prototype can yield even more useful and specific information

Questions?