Avaliação da Usabilidade e da Acessibilidade do Ambiente Virtual de

Aprendizagem TelEduc: levando o “aprender” a todos

André C. da Silva Ariel Vargas

Caroline C. Letizio Ricardo E. Caceffo

Prof. Dr. Heloísa V. da RochaProfessor

Institute of Computing - NIEDUNICAMP

Evaluation Competition October, 28th 2011

Agenda

• Ideas• Materials• Methods• Examples• Results• Conclusion

IdeasVirtual Learning Environments (VLE) characteristics

Funcionalities

Security

Performance

Scalability

Availability

UsabilityAccessibility

Reliability

Maintainability... Is the VLE for everyone?

Ideas

• TelEduc Environmento First Brazilian VLE open sourceo More than 4,000 installationso Analyzed version: 4.2o It has been used in our university

http://www.teleduc.org.br/

Materials

• Communication among all participants → Mail/Correio and Discussion Forums/Fórum de Discussão

• Administration of several contents → Support

Materials/Material de Apoio

• Development, delivery and monitoring of activities → Portfolio/Portfólio

Methods

Usability

Log Interaction Analysis

Inspection

User Tests

Nielsen

Questionnaire

Guidelines

Heuristic

WebHint

Methods - WebHint

Users

• Students with some VLE experience: 17

• One user with tool to change font size

Methods

Accessibility

WCAG (W3C)

Inspection

User Tests

IBM

Tanaka (2009)

Guidelines

Heuristic

Hera

Methods – Tanaka(2009)

• Similarly Nielsen usability evaluation

• 5 Heuristics:

6)Support for different types of input and output7)Contents for all kind of users8)Avoid independence of use9)Respect the user preferences10)Efficiency of alternative interaction

Examples

There is not showed a confirmation message

Examples of problems found

Usability: Add comment to a portfolio item

- Nielsen Heuristics: Visibility and status of system

- Log Interaction Analysis: Half of the users had executed the task distinct than expected

- Questionnaire: Half of the user agreed partially

"The information showed by the system after I performed some action request are enough to make me understand the

results of my action"

It is not possible to check a checkbox using only the keyboard

Accessibility: It is not possible use the system only with keyboard input.

- Accessibility Heuristcs: Support for different types of input and output

- Guidelines: Checkpoint 6.4 - For scripts and applets, ensure that event handlers are input device-independent

Examples of problems found

Results

Tool Errors x Severity Total

0 1 2 3 4

Mail/Correio 0 3 1 1 0 5

Portfolio/Portfólio 0 4 5 12 1 22

Discussion Forums/Fórum de Discussão

0 2 3 2 0 7

Suport Material/ Material de Apoio

1 7 2 4 0 14

General/Geral 0 3 2 2 2 9

Total 1 19 13 21 3 58

• Usability Heuristic Evaluation

Results

• Questionnaire

• WebHint Method:

o 5 usability's problems found

o A few number of users

Results

• Accessibility Heuristic Evaluation

ResultsTool Errors x Severity Total

0 1 2 3 4

Mail/Correio 0 0 0 0 13 13

Portfolio/Portfólio 0 0 0 0 9 9

Discussion Forums/Fórum de Discussão

0 0 0 2 12 14

Suport Material/ Material de Apoio

0 0 0 0 16 16

General/Geral 0 0 2 5 11 18

Total 0 0 2 7 61 70

• Guidelines Evaluations

ResultsPriority Mail/

CorreioForum Portfolio Support

Material / Material de Apoio

Total

1 2 1 1 2 6

2 8 8 8 8 32

3 1 3 3 3 10

Total 11 12 12 13 48

Conclusions

• Methods are complementary: • The usability results show that the TelEduc has good

usability, also pointing specific problems that can be solved to improve VLE's usability.

• The accessibility results show critical accessibility problems in TelEduc – Is NOT for everyone!!

o Accessibility 2 inspection methods

o Usability 1 inspection method 2 user tests methods

Thanks

• Instituto de Computação - Unicamp• Capes• CNPq• FAPESP

More information about this workhttp://www.students.ic.unicamp.br/~ra059664/ihc2011/

Extra: Accessibility and Usability

• In some situations were found problems that affect both usability and accessibility of TelEduc

• For example:• User starts the upload of some content in his/her

portfolio• User opens another tab and continues browsing

TelEduc, for example to read mail• When the user comes back to the portfolio tab, s/he

is not able to know if the upload was successfully or not

Extra: Accessibility and Usability

• It happens because the success or not success message is showed for only a few seconds (usability issue) and can't be “read” by the screen reader (accessibility issue)