are massive open online courses (moocs) really open to

Are Massive Open Online Courses (MOOCs) Really Open to Everyone?: A Study of Accessibility Evaluation from the

Perspective of Universal Design for Learning

Kyudong Park Dept. of Creative IT

Engineering, POSTECH Pohang, South Korea [email protected]

Hyo Jin Kim Dept. of Creative IT

Engineering, POSTECH Pohang, South Korea

[email protected]

Hyo-Jeong So Dept. of Creative IT

Engineering, POSTECH Pohang, South Korea

[email protected]

ABSTRACT Recently, Massive Open Online Courses (MOOCs) have emerged as an open learning space with the development of Information and Communication Technology (ICT) and the movement of knowledge sharing. MOOCs are an open online learning space based on unconstrained interaction among learners unlike traditional e-learning systems. Despite the promise of openness in learning, there is lack of research studies to elucidate the accessibility of MOOCs for learners with disabilities. The main goals of this paper are 1) to evaluate the accessibility of MOOCs from the perspective of Universal Design for Learning (UDL) and 2) to identify critical barriers diminishing the accessibility issues. First, we performed a manual testing to evaluate three MOOC platforms (i.e., Coursera, edX, and Khan Academy) on the computer and the mobile platform with the UDL principles. Second, we conducted a user study with four participants with visual impairment. From the walkthrough experience in the interview sessions, we identified critical obstacles in accessibility of MOOCs. The results indicate that the three MOOCs platforms met only 52% of the 40 UDL categories. Overall, the computer-based MOOC platform received higher accessibility scores than that of the mobile-based MOOC platform though the difference was marginal. Finally, we present four main obstacles related to the low accessibility issues in MOOCs.

Author Keywords MOOC; Visually Impaired Person; Universal Design; Accessibility Evaluation; Mobile Web;

ACM Classification Keywords H.5.m. Information interfaces and presentation (e.g., HCI): Miscellaneous; K.4.2. Computers and Society: Social Issues – Assistive technologies for persons with disabilities

INTRODUCTION Leveraging the affordances of the emerging technologies such as mobiles and ubiquitous computing, learning is becoming increasingly more open, accessible and universal to learners with diverse backgrounds and disabilities. The emergence of Massive Open Online Courses (MOOCs) in

education scenes indicates the growing interests and efforts towards creating learning opportunities to reach out to a wide range of learners. MOOCs platforms with the openness movement require a higher level of accessibility to users with and without disabilities. While there has been a growing body of literature regarding the emergence and impact of MOOCs in higher education contexts, research on the accessibility of MOOCs is still in its early stage. To date, little research studies have attempted to address the issues of accessibility in MOOCs [2, 3, 10].

In this paper, we adopt Universal Design for Learning (UDL) principles as an accessibility guideline, given the fact that MOOCs are positioned as an open learning platform reaching out to massive users. Elias has proposed the framework of UDL with 8 principles and 40 categories covering both accessibility and universal design issues [7]. In this study, two experts who have knowledge about accessibility evaluated the representative MOOC platforms (i.e., Coursera, edX, and Khan Academy) with UDL categories. We also discovered concrete barriers leading to low accessibility from a user study with four participants with visual impairment. In conclusion, we discuss main obstacles in accessibility issues of MOOCs and lessons learned from the study.

THEORETICAL BACKGROUND & RELATED WORK

Massive Open Online Courses (MOOCs) In 2008, George Siemen and Stephen Downes first introduced the notion of Massive Open Online Courses (MOOCs). They opened the online course “Connectivism and Connective Knowledge (CCK08)”, which is widely considered as the first MOOC [15]. The original conception of this innovative method is for open learning where learners can improve ideas each other and generate new knowledge through continual improvement. Along with the explosive development of the Internet, communication, and mobile technologies, MOOCs have emerged as a platform for life-

Proceedings of HCI Korea 2016

- 29 - © 2016 Hanbit Media & HCI Korea

long learning for a wide range of learners. This new educational stream has transformed conventional classroom-based education to become an open environment without any limitation in time and places.

Universal Design for Learning (UDL) Historically, UDL is rooted in the philosophy and notion of universal design in the field of architecture. Universal design is defined as “the design of product and environments to be usable by all people, to the greatest extent possible, without the need for adaptation or specialized design” [12]. The Center for Universal Design [17] suggested seven principles for universal design: 1) Equitable use, 2) Flexibility in use, 3) Simple and intuitive use, 4) Perceptible information, 5) Tolerance for error, 6) Low physical effort, and 7) Size and space for approach and use. When the idea of universal design is integrated with learning, the focus has shifted from product / environment design to curriculum design that allows multiple means of presentation, expression, and engagement [13]. Since then, some adaptions to the original UDL framework have been made to better reflect the nature of inclusion and accessibility in educational contexts [4, 7, 8, 16]. Scott, McGuire, and Shaw [16] added two categories to the universal design principles regarding an instructional environment; 7) Community of learners and supports, and 8) Instructional climates. With these eight principles, Elias [7] provided 40 sub-categories of online course accessibility and specific recommendations (Table 1). In this paper, we test MOOCs accessibility using these categories and principles proposed by Elias. Web Accessibility Evaluation We apply the web accessibility evaluation method to evaluate webpages in MOOC platforms that support responsive webs to provide optimal viewing across multiple devices with different resolutions. According to the study about the web accessibility evaluation [1], in general, there are three types of webpage accessibility evaluation techniques: automated testing, manual testing, and user testing. In this study, we performed the manual testing and user testing to evaluate accessibility issues in MOOCs.

1) Automated testing is a technique to evaluate accessibility using automatic programs or tools. This method usually does not need the help of human-evaluators. For example, Iniesto et al. [10] evaluated MOOCs using eXaminator, which is an automatic online testing tool. We did not perform this automated testing since there was no reliable tool available for evaluating mobile platforms.

2) Manual testing is a technique to discover and test the accessibility issues by human-evaluators while they sometimes can be accompanied with software tools. In this testing technique, the evaluator can be an important factor. The evaluator types include non-technical checks, technical checks, and expert checks. In our study, experts who have good knowledge of screen readers and assistive technology performed the manual testing.

Principle 1. Equitable Use (1) All content online (2) Anywhere Anytime (3) Translator

Principle 2. Flexible Use (4) Mind maps / diagram displays (5) Conferencing tools (6) Video / audio presentation tools (7) Slide presentation tools (8) Video / audio assignment tools (9) Links to additional information (10) Choice of study topics / assignments

Principle 3. Simple and Intuitive Use (11) Resume course (12) Simple interface (13) Direct link to new posts (14) Easy-to-navigate menus (15) Books (16) Searchable forums (17) Searchable content (18) Mobile interface (19) Text-only interface (20) Offline resources

Principle 4. Perceptible Information (21) Screen preferences, font size, masking, colors (22) Screen / document readers (23) Text-to-speech (24) Screen / cursor magnifiers (25) Transcription (26) Captions

Principle 5. Tolerance for Error (27) Ability to edit after posting (28) Confirmation before sending assignments (29) Warnings when leaving course site

Principle 6. Technical and Physical Effort (30) Voice recognition (31) Word prediction (32) Built-in assistive technologies (33) Limited use of external links (34) Embedded multimedia (35) Browser capability checker

Principle 7. Learner Community and Support (36) Study group (37) Links to support services

Principle 8. Instructional Climate (38) Involvement in discussion forums (39) Regular email contact with students (40) Availability for one-on-one consultation

Table 1. Universal Design for Learning (UDL) principles and categories of online course accessibility [7]



3) User testing is a technique to test a target web site with real end-users. In this research, we recruited four people with visual impairment to test webpages by performing pre-defined tasks in MOOCs.

Accessibility of MOOCs MOOCs can provide alternative learning opportunities for people with disabilities who could not attend schools through traditional means. However, it is still questionable whether MOOCs are really open and universal to learners with disabilities. For instance, Bohnsack and Puhl [3] examined the accessibility of five MOOCs sites (i.e., Coursera, Udacity, edX, OpenCourseWorld, and Iversity) and found that there were significant accessibility problems in these platforms. Iniesto et al. [10] tested Spanish and Portuguese MOOCs using the automated tool and simulator. They demonstrated that both platforms have serious accessibility problems. Al-Mouh et al. [2] evaluated Coursera through the heuristic evaluation method with the Web Content Accessibility Guidelines (WCAG) 2.0 by the World Wide Web Consor-tium (W3C) on the personal computer (PC). Their study found that the Coursera platform failed to comply with the WCAG 2.0.

In this study, we attempt to advance the existing knowledge about the accessibility of MOOCs by evaluating both PC and mobile-based platforms from the perspectives of universal design for learning (UDL). Our ultimate goal is to take a closer look at barriers that limit the accessibility of MOOCs to learners with disabilities.

METHODOLOGY In this paper, we focus on uncovering and evaluating the accessibility of MOOCs for users with visual impairment. We first used a manual testing [1] by two experts and then performed a user study with four participants with visual impairment.

Research Questions This research is guided by the following research questions:

1) How well do the current MOOCs support the accessible web on the PC and mobile (iPhone) platforms?

2) What are the differences in the accessibility of MOOCs between PC and mobile platforms?

3) What are the main barriers that limit the accessibility of MOOCs to users with visual impairment?

4) What accessibility functions should be added in MOOCs for visually impaired users?

5) How are MOOCs useful for learning to visually impaired users?

Manual Testing

Apparatus The evaluators used JAWS (screen reader program, version 16.0) with a computer running the Windows 8.1 operating system. iPhone 5S with VoiceOver (screen reader function for iPhone) running the iOS 9.1 operating system was

employed to evaluate accessibility in the mobile platform. In this testing, three major MOOC platforms and five courses for each MOOC were selected for evaluation. We selected courses from various topics. When information about the number of users is available, we selected courses with a high number of users. The course titles are:

1. Coursera [5] 1) Art & Activity 2) Jazz Improvisation 3) International Marketing in Asia 4) Chinese for Beginners 5) Child Nutrition and Cooking

2. edX [6] 1) The Science of Everyday Thinking 2) The Rise of Superheroes and Their Impact on Pop

Culture 3) Developing International Software 4) English Grammar and Style 5) Introduction to Computer Science

3. Khan Academy [11] 1) Masterpieces old and new 2) Full-length SAT 3) Measuring the Universe 4) Art history basics 5) Basic geometry

Procedures After two evaluators tested the selected courses with the 40 UDL categories independently, they discussed the accessibility scores to reach mutual consensus. There are three different types of scores – ○: all courses tested in a platform satisfied the category, △: at least one course tested in a platform did not satisfy the category, ⅹ: all courses did not satisfy the category.

User Study The user study was conducted at the center for students with disabilities located in a private university in a mid-sized city (Figure 1). The study was divided into the mobile session and the PC session.

Figure 1. User study with four participants with visual impairment



ID Gender Age LB YB YS YP P1 Male 18 18 4.5 10

P2 Male 26 Legally 10 4.0 2

P3 Female 24 blind 13 5.0 4

P4 Female 21 21 3.0 13

ID Session MOOC Platform

Device & Screen reader

P1 Mobile Coursera iPhone5+VoiceOver

P2 Mobile edX iPhone4S+VoiceOver

P3 Mobile Khan Academy iPhone5+VoiceOver

P4 PC edX PC+SenseReader[18]

Participants Four participants with visual impairment (two males and two females) were recruited for the user study (Table 2). Participants’ ages ranged from 18 to 26 years old (mean =22.25, s.d.=3.5) and all participants are legally blind. They are university students and owned smartphone devices and PCs. All participants indicated that they have had prior experienced with online courses. They have enough English competencies for understanding MOOCs in English, but their level of confidence varies to some extent.

Apparatus All participants used their own smartphone devices and PC. In the mobile session, three participants used iPhone with VoiceOver and took a respective course assigned to them. In the PC session, one participant used Windows 7 with SenseReader (Korean screen reader) [18] and took a course only in edX due to the limitation of experiment time (Table 3). The course titles used in the user study are as follows: “Jazz Improvisation – Coursera”, “The Science of Everyday Thinking – edX”, “Art History Basics – Khan Academy”

Procedures Pre-interview: We first asked basic questions about the profile and background of each participant, such as “Have you heard about the MOOCs?” and “Have you taken any online courses?”

Experiment: In the experiment, a series of pre-defined tasks were assigned to the participants. During the process of observing task completion, we conducted in-depth interviews for deeper understanding about barriers when the participants were in trouble with certain tasks. We selected the following seven tasks from registration to participation that deem necessary to take courses in MOOCs:

1) Sign up / Login 2) Search the pre-assigned course 3) Register the course 4) Take a first lecture in the course

: Find and play a video material. Control the sound volume and speed. Check functions of captions, transcription, and auto-translation.

5) Participate in a discussion forum : Write, edit, and remove the post and comment.

6) Submit an assignment (only tested in PC) : Upload some files to the assignment. This task is only for PC because mobile devices have many limitations to complete homework.

7) Submit a quiz

Post-interview: We asked the following questions after the completion of the experiment: (1) what are the pros and cons of MOOCs from their experiences; (2) whether they found MOOCs useful from both accessibility and learning perspectives; and (3) how MOOCs can be improved to increate accessibility to users with disabilities.

RESULTS

Manual Testing Tables 4 and 5 show the result of manual testing by human-evaluators. In terms of the testing platform environment, only 52.5% of the categories were satisfied on the PC and 51.7% were satisfied on the mobile. The best combination of configuration is edX on PC (○: 57.5%, △: 15.0%, ⅹ: 27.5%) even though there is a marginal difference.

User Study

Sign up / Login In the mobile session, all participants failed to sign up because the “Sign up” button was hidden in the dropdown menu. All participants had to perform the task of sign up and login with the researchers’ support. The participant in the PC session successfully achieved the task without major difficulties.

Search a course In the mobile session, all participants failed to search the given course. In Coursera, there was no button for the search function. In edX, while P2 was able to query the course name, too many results were provided in a screen. Although P3 also queried the course title, the focus of the screen reader did not move to the search results.

In the PC session, P4 succeeded in searching the course title easily using the find function (Ctrl+F).

Register the course All participants were successful in the enrollment of the given courses.

Take a first lecture in the course In Coursera and Khan Academy on the mobile, the lectures were played successfully. But, P2 could not play the lecture in edX because VoiceOver read the play button as a “button”.

Table 2. Profiles of participants: LB (Level of Blindness), YB (Years of Blindness), YS (Years of Smartphone usage),

and YP (Years of PC usage)

Table 3. Configuration of platform and environment



Environ-ment Platform ○ (%) △ (%) ⅹ (%)

PC

C 50.0 20.0 30.0

E 57.5 15.0 27.5

K 50.0 0.0 50.0

Total 52.5 11.7 35.8

Mobile

C 47.5 20.0 32.5

E 57.5 5.0 37.5

K 50.0 0.0 50.0

Total 51.7 8.3 40.0

Total 52.1 10.0 37.9

P3 with Khan Academy succeeded to adjust the sound volume and to control the play speed.

Although P4 also had trouble in finding a play button in the PC session, she carried out playing the lecture by instinct without formal guidelines or experience.

Participate in a discussion forum In the mobile session, P1 with Coursera was not able to write a new post because VoiceOver read the button for submitting new posts as a “button”. In case of edX, while P2 could enter the discussion forum, he could not distinguish between discussion materials and course materials. The reading task was possible with Khan Academy, but it was hard to find a function of comment writing.

In the PC session, P4 succeeded in all sub-tasks and answered that this forum was not an ordinary bulletin board.

Submit an assignment (only tested in PC) P4 was successful in uploading some files to the assignment.

Submit a quiz In the mobile session, it was difficult to enter the quiz menu and to grasp the structure of quiz such as the test stem and multiple choices in Khan Academy. In edX and Khan Academy, P1 and P3 did not fulfill the task because the focus forcedly moved to the first part in the screen after selecting one item. There was no problem with the forced moving of focus in the PC session with edX. P4, however, answered that it was difficult to find a starting point of the quiz with the screen reader. The system did not provide any feedback for successful submission to users. Although this issue is not related with task success, it could cause additional problems.

DISCUSSION

How well do the current MOOCs support the accessible web on the PC and mobile (iPhone) platforms? According to the results, over one-third of UDL categories were not satisfied (Table 5). All participants answered that

Principle & Category Index

PC Mobile C E K C E K

1. Equitable use

1 ○ ○ ○ ○ ○ ○

2 ○ ○ ○ ○ ○ ○

3 △ △ ⅹ △ △ ⅹ

2. Flexible use

4 ○ ○ ○ ○ ○ ○

5 ⅹ ⅹ ⅹ ⅹ ⅹ ⅹ

6 ○ ○ ○ ○ ○ ○

7 ○ ○ ○ ○ ○ ○

8 △ △ ⅹ △ ⅹ ⅹ

9 △ △ ⅹ ⅹ ⅹ ⅹ

10 ○ ○ ○ ○ ○ ○

3. Simple and intuitive use

11 ○ ○ ○ ○ ○ ○

12 ○ ○ ○ ○ ○ ○

13 ⅹ △ ⅹ ⅹ ⅹ ⅹ

14 ○ ○ ○ ○ ○ ○

15 △ △ ⅹ △ ⅹ ⅹ

16 ○ ○ ○ ○ ○ ○

17 ○ ○ ○ ○ ○ ○

18 ○ ○ ○ ○ ○ ○


20 ⅹ △ ⅹ ⅹ △ ⅹ

4. Perceptible information


22 ○ ○ ○ ○ ○ ○

23 ○ ○ ○ ○ ○ ○


25 ○ ○ ⅹ ○ ○ ⅹ

26 △ ⅹ ⅹ △ ⅹ ⅹ

5. Tolerance for error

27 ○ ○ ○ ○ ○ ○

28 ⅹ ○ ⅹ ⅹ ○ ⅹ


6. Technical and physical effort



32 ○ ○ ○ ○ ○ ○


34 ○ ○ ○ ○ ○ ○


7. Leaner community and support

36 △ ○ ⅹ △ ○ ⅹ

37 ○ ○ ○ △ ○ ○

8. Instructional climate 38 △ ○ ○ △ ○ ○

39 ○ ⅹ ⅹ ○ ⅹ ⅹ

40 △ ○ ○ △ ○ ○

Table 4. The result of Manual testing (C: Coursera, E: edX, K: Khan Academy / ○: all courses tested in a

platform satisfied the category, △: at least one course tested in a platform did not satisfy, ⅹ: all courses did not

satisfy)

Table 5. The summary of result (Abbreviations and symbols are the same as Table 4)



MOOC sites had poor accessibilities, which means MOOC platforms need to improve accessibility and universal design issues. All participants in the user study failed to complete the first task, “Sign up / Login.” Overall, it appeared that users with visual impairment would not be able to take the benefits of the MOOCs’ services with the mobile environment, as expressed by the participants:

“If I use this site from today, it would be hard without someone’s help.” – P2

“Other people with visual impairment will give up on using this platform because the structure is very complex.” – P3

What are the differences in the accessibility of MOOCs between PC and mobiles platforms? Overall, PC (○: 52.5%, △: 11.7%, ⅹ: 35.8%) has better accessibility and universal design for learning than mobiles (○: 51.7%, △: 8.3%, ⅹ: 40.0%). In terms of task completion, the task completion speed in the PC was faster than the mobile’s. The mobile platform, however, had the advantage of detecting a spatial position of items on the screen.

Task Completion Speed The difference in task completion speed was found in the tested MOOCs due to input and search systems. When P4 used the screen reader program (SenseReader [18]) in the PC environment, she could search and move the focus quickly by pushing the Tab, Up and Down keys of the keyboard. On the contrary, swipe gestures such as left to right were used to move the focus in the mobile environment. Since the shortcut find function (“Ctrl + F key” in Windows) was only supported in the PC environment, the participant (P4) with the PC environment could perform the tasks faster.

Spatial Position of Items in the Screen In edX, the participants with the PC and mobile spent much time on finding a play button in the “Take a lecture” task. In addition, they answered that the position of the play button was not like the play button in other websites. We asked about their workarounds: “In this situation, how do you cope with?” and the responses were:

“In this case, I have to ask someone’s help.” – P4 (PC)

“I memorize the position of the abnormal button with my finger, then, turn off the VoiceOver using a shortcut. And I click the position manually.” – P2 (Mobile)

What are the main barriers that limit the accessibility of MOOCs to users with visual impairment? We found not only several minor issues, but critical barriers from the user study. Minor issues have a slight effect to perform the given tasks, for example volume controller and indicator for the essential items in the registration form. In this study, we mainly discuss four critical barriers that appear to have serious impact on the completion of the assigned tasks.

Dropdown Menu Dropdown menu is a toggleable function that provides users with a pre-defined list of items only when the mouse is rolled

over (Figure 2). Because screen reader programs read this menu as a single button, it is easy for people with visual impairment not to detect this as a toggleable menu. In this study, the issue with the dropdown menu was a main barrier to find the “Sign up / Login” buttons, which were somewhat hidden in the dropdown menu. In severe cases, there was no sound output in the mobile Khan Academy when the dropdown menu was focused.

“Even though I checked all elements from top of the screen, I could not find the Sign up button.” – P1

Dynamic Webpage Structure When the participants submitted a quiz or assignment in edX, the platform provided feedback of successful submission without page reloading. Asynchronous JavaScript and XML (AJAX) is one of the web development techniques that can send and get data from a server in the background, and change the web element without the transition of webpages. While AJAX gives lively and dynamic effects to the webpage, this seriously inhibits the web content accessibility. The participants with visual impairment could not catch the webpage event.

“(After submitting a quiz) How much do I have to wait? Does it finish?” – P4

Repetitive Elements It is desirable that one single page should contain information as needed because visually impaired people have to listen to all elements of webpage using the screen reader. This study found that the participants had no choice but to read repetitive information since the category information about courses in MOOCs was usually positioned at the top of each webpage. This repetitive structure made the participants spend much time to even simply pass and feel

Figure 2. The example of dropdown menu in top-left of the page



easily tired. To skip this information, we proposed the use of invisible buttons for people with visual impairment:

“Same information about categories is presented over and over again every time the web-page is loaded. It is really annoying.” – P2

“I cannot find the contents that I want because of too many elements. I need more simple and organized interfaces.” – P3

Illogical Focus Movement The visually impaired are disturbed by the illogical stream of focus, which causes troubles with structuring the webpage. For example, when the participant selected an answer in a question of the objective quiz, the focus moved to the top of the screen forcedly in the mobile webpage of edX. The participant first thought that there was one question in the quiz. Since he moved the focus down for a long time, he realized the existence of unsolved questions. Also in Khan Academy, the focus moved to the bottom of the screen in a forceful manner, which was the similar problem found in edX.

“Due to the illogical focus movement, it takes long long time to finish all questions in the quiz.” – P2

What accessibility functions should be added in MOOCs for visually impaired users? All participants would like to improve basic accessibilities in MOOC platforms. Among them, critical accessibility features are as follows:

Translation Participants want to have the auto-translation feature in the form of translated text files. Some participants answered that dubbing the textual information is desirable.

“It would be better if all lectures and translated scripts can be downloadable. The movie player in my computer (GOM Player) is compatible with my screen reader. Thus it can read a caption file. I used to watch foreign movies in my computer.” – P2

Invisible Button for Visual Impaired Users Some participants proposed the use of invisible buttons for the visual impaired users. An invisible button is an object accessed only by screen readers and can perform a specific function. For example, Google instant search [9], which is the function showing search results as the user starts to type, can be off by selecting the invisible button for slow input speed. Thus, only screen reader users can control the instant search function in the main page directly. Naver [14] provides a more practical feature with invisible buttons. Using the invisible button, users can skip headline contents to go directly to main contents (Figure 3). We expect that many repetitive elements in MOOCs can be greatly reduced with the use of invisible buttons.

How are MOOCs useful for learning to the visually impaired users? While many accessibility problems were discovered in the user study, all participants indicated that MOOC platforms are useful for people with disabilities. The participants anticipated that many people with disabilities might give up in the initial stage due to the complex structure of the MOOCs. Some participants were concerned about their language skills as currently most MOOCs contents were offered in English:

All participants said that the PC-based MOOCs would be much easier to use than the mobile-based MOOCs. One participant preferred the mobile platform due to the affordances of mobility to learn anytime and anywhere:

“If there are helpful contents, I will learn using this platform with my friends’ help.” – P4

“I am not good at English. Therefore, I can use the platform well if it provides translation.” – P2

“I prefer mobile to PC because I like to listen this lecture on the move anytime and anywhere.” – P1

CONCLUSIONS & FUTURE WORKS This research has attempted to evaluate the accessibility of MOOC platforms from UDL perspectives and to discover the main obstacles of the accessible MOOCs. This research was conducted through two phases. In the first phase, the manual testing by two experts was performed. The findings indicate that over 1/3 of the UDL categories were not satisfied, and that the PC environment had better accessibility and universality than the mobile environment. In the second phase, we conducted a user study with four legally-blind participants. We analyzed the causes of poor accessibility

Figure 3. The example of invisible button. There is the button on the left-top of the left figure. If users click the button, the focus is moved to the main contents like the

right figure.



through observing the process of their task completion, and uncovered their struggles and accessibility issues.

Despite these findings, there remains a range of limitations to be resolved in future research. First, in the current study, the accessibility evaluation of the mobile environment was conducted based on the mobile web with the responsive web technology. Nowadays, however, many mobile web services are developed as mobile applications for iOS and Android. Further studies on the evaluation for mobile applications are needed. Second, the participants had to complete the pre-defined tasks in one session. While there are some advantages with simple and low cost usability testing, there are also drawbacks to find accessibility problems at deeper levels. Thus, it is necessary to conduct a prolonged study where participants take one or more MOOC courses with active participation. From this approach, we can expect to collect quantitative data such as users’ clickstream data and logs. Third, this study focused on the users with visual impairment and did not include other types of disabilities. Hence, there should be some cautions to generalize findings of this study to other context of users with disabilities. The small number of users is another limitation, which we plan to expand with more users with other types of disabilities such as hearing impairment and physical disabilities needing assistive inputs.

MOOCs were launched with the great promise to provide open and accessible learning opportunities to a wide range of learners. Despite such promises, this study reveals that the current MOOCs platforms do not address the issue of accessibility and universal design for people with disabilities. As MOOCs as an open education movement becomes a major trend in teaching and learning for the next decade, we need to explore how to make open courses and resources more accessible and universal to learners with disabilities.

ACKNOWLEDGMENTS We would like to thank all the participants and the Center for students with disabilities in Daegu University. This research was supported by the MSIP (Ministry of Science, ICT and Future Planning), Korea, under the “ICT Consilience Creative Program” (IITP-2015-R0346-15-1007) supervised by the IITP (Institute for Information & communications Technology Promotion).

REFERENCES 1. Abou-Zahra, S. Web accessibility evaluation. Web

Accessibility. Springer, 2008, 79-106. 2. Al-Mouh, N. A., Al-Khalifa, A. S., and Al-Khalifa, H.

S. A first look into MOOCs accessibility. Computers

Helping People with Special Needs. Springer, 2014, 145-152.

3. Bohnsack, M., and Puhl, S. Accessibility of MOOCs. Computers Helping People with Special Needs. Springer, 2014, 141-144.

4. Burgstahler, S. Universal design of instruction (UDI): Definition, principles, guidelines, and examples. Retrieved August 10, 2015 from http://goo.gl/wDTcoQ

5. Coursera. http://www.coursera.org 6. edX. http://www.edx.org 7. Elias, T. Universal instructional design principles for

moodle. The International Review of Research in Open and Distributed Learning 11, 2 (2010), 110-124.

8. Elias, T. Universal instructional design principles for mobile learning. The International Review of Research in Open and Distributed Learning 12, 2 (2011), 143-156.

9. Google instant results as you type. https://support.google.com/websearch/answer/186645

10. Iniesto, F., Rodrigo, C., and Moreira Teixeira, A. Accessibility analysis in MOOC platforms. A case study: UNED COMA and UAB iMOOC. Libro de Actas del V Congreso Internacional sobre Calidad y Accesibilidad de la Formación Viertual CAFVIR (2014), 545-550.

11. Khan Academy. http://www.khanacademy.org 12. Mace, R. Universal design: Housing for the lifespan of

all people, 1988. 13. Meyer, A., Rose, D. H., and Chall, J. S. Learning to

read in the computer age, vol. 3. Brookline Books, 1998.

14. Naver. http://www.naver.com 15. Parr, C. MOOC creators criticize courses’ lack of

creativity, 2013. https://goo.gl/FvQREe 16. Scott, S., McGuire, J., and Shaw, S. Principles of

universal design for instruction. Storrs: University of Connecticut, Center on Postsecondary Education and Disability (2001).

17. The Center for Universal Design. The Principles of Universal Design, Version 2.0. Raleigh, NC: North Carolina State University, 1997.

18. XVision Technology, SenseReader. http://xvtech.com



http://goo.gl/wDTcoQ

http://www.coursera.org/

http://www.edx.org/

https://support.google.com/websearch/answer/186645

http://www.khanacademy.org/

http://www.naver.com/

https://goo.gl/FvQREe

are massive open online courses (moocs) really open to

Documents