TOWARDS QUALITY IN E-LEARNING QUALITY ASSURANCE - EPPROBATE INTERNATIONAL COURSEWARE LABEL

Marjan Miloshevic

Danijela MiloshevicTechnical Faculty Čačak - University of Kragujevac, Čačak, Serbia

{marjan, danijela}@tfc.kg.ac.rs

Suzana Loshkovska

Faculty of Computer Science and Engineering Skopje, Macedonia suzana.loshkovska@finki.ukim.mk

2

Content

Quality of E-learning Certification approaches and quality

schemes DL@WEB – BADEN Epprobate

o Aim, objectiveso Procedureo Self-assessment documento Evaluation

Conclusion

3

E-Learning – all present paradigm

E-Learning

Formal education

NON FORMAL EDUCATION

LLL

4

The issue of quality

The standard quality assurance (QA) procedures are not sufficient.

QA in e-learning is a non issue for many, especially for the QA agencies

Numerous international projects were conducted to form a comprehensive, yet usable framework for quality assurance of learning.

Specialized organizations have developed their own benchmarking procedures and tools and established labels as brands

5

Quality of E-Learning – The Terms

There are several definitions of quality of e-learning, and each depends on its scope, objectives, focus and the methodology of the quality approach.

In this context, quality of e-learning is related to all: processes, products and services for learning, education and training supported by the use of information and communication technologies.

6

Issues of internationality

We need an unified label

We have... Language issues Cultural

differences Law issues

7

International Quality Labels

Label name Level Review process Type Reviewers

UNIQUe Universities Publicly documented, on site Quality mark Internal

E-xcellence

University program

Publicly documented; on-line and on-site

(optionally)Benchmark Internal

ECBCheck University program

Documents and accompanying  web-site are in construction phase

Quality Mark Internal

SEVAQ+ Universities, Vocational

studies

Supporting documents and web-site are are

available; on-line

Self-evaluation

tool

Stakeholders (teachers,

students, ...)

ProcertCourseware

for IT specialists

Documents are not publicly available Benchmark N/A

epprobate General courseware

Quality grid is public; review conducted on-line Quality Mark Internal and

external

8

BADEN – Gathering Western Balkan for Improved Education

BAlkan Distance Education Network BASIC GOALS:

affirmation and use of professional knowledge via e-learning and distance education

Еducation аccessible to all, no matter of place Improvement of academic space in WB region

9

BADEN’s goals  Development of informatics society in the region of Balkan

through improving and promoting quality e-learning and distance education,

Modernization of education by promoting regional and international cooperation by connecting institutions and individuals as well as by exchange of information in the fields of e-learning and distance education,

Tuning into The European Higher Education Area (EHEA), Raising the level of pedagogical and technical knowledge

and skills in the fields of e-learning and distance learning, Strengthening of educational and civil sector for successful

application of e-learning and distance education,

10

Goals of BADEN (2)  Fostering and cherishing research spirit for further

development of e-learning and distance education, Strengthening of responsibility within pedagogic

practice in the field of e- learning and distance education,

More effective employment and improvement of economic development,

Raising social awareness about the significance of e-learning and distance education in all aspects of formal and non-formal education

Fostering and strengthening of responsibility for individual development and improvement of status of smaller and wider communities through activities of protection human and minority rights, labor, racial and sex equality and right on life in healthy and ecologically safe environment.

11

Activities

Education through professional and research seminars and summer schools to enable providing mastering skills and knowledge in the fields of e-learning and distance education,

Organization of professional lectures, trainings, symposiums and conferences with topics about e-learning and distance education

Promotion of distance learning through cooperation and participation on manifestations of domestic, regional and international importance, competitive, propaganda and commercial character,

Participation in realization of projects at national, regional and international level as well as in other fields which have the need for implementing this kind of education, independently  or collectively, applying at funds in the country and abroad,

Promoting and placing of innovative and creative professional and research solutions,

12

Activities

Carrying out commercially readymade solutions, individual courses or moderation services in the fields of e-learning and distance education,

Publishing activity from the primary and related fields which include application of e-learning techniques to realize their activities,

Creating and continual improvement of electronic knowledge basis from the fields of e-learning and distance education,

Promotion of the lifelong learning concept by applying e-learning and distance education,

Creating and realizing accredited programs for professional training of the employed in education.

13

epprobate

Self-paced e-Learning products and services are mainly based on courseware. At present, the worldwide market reached $32.1 billion -

with a five-year compound annual growth rate of 9.2% according to Ambient Insight!

There is a huge provision of eLearning courseware in the world. Every day different kind of new material like Web Based

Training (WBT), Computer Based Training (CBT), Game Based Learning (GBL), Videos, Open Educational Resources (OER), Mobile Learning Apps etc. are added.

For the learner it becomes very difficult to distinguish what is of good quality and worth buying. 

14

epprobate

epprobate (from the Latin 'approbare', to approve) is the first international quality label for eLearning courseware.

The quality label is an initiative of three organisations: Learning Agency Network (LANETO), Agence Wallonne des Télécommunication

(AWT)  e-Learning Quality Service Center (eLQSC).

15

epprobate process

1. Contacting the National Partner (BADEN for Western Balkan).

2. A short review – feasibility study.

3. The National Partner gives a recommendation and sends an offer

4. Reviewers who are able to evaluate the courseware from the points of view of eLearning pedagogy, content accuracy, technical delivery, and acceptability by the target group are selected. The producer is asked to help identify representatives of the target

group Payment for review is paid at this point

5. Producer completes a Self Assessment Document answering a range of questions about the courseware and gives access to the courseware to the review team.

16

epprobate process

1. After the reviewers have evaluated the courseware and reached an initial set of conclusions, the head reviewer contacts the producer and sends a report to brief the producer for an on-line meeting with the review team.

2. During the meeting the producer receives feedback from the reviewers about the courseware in terms of the quality criteria of the epprobate Quality Grid, and has an opportunity to provide additional information about the courseware to address any concerns.

3. After this meeting the review team revises their report and sends it together with their recommendation as to whether the label should be awarded or not to the international awarding committee.

17

Epprobate process

1. The international awarding committee is responsible for the final decision about the awarding of the epprobate label and it will consider the recommendations of the review team within the wider context of its knowledge of global e-learning standards.

2. The producer receives formal written feedback from the international awarding committee, and if the courseware has been successful in its application it will be officially labelled by epprobate.

3. Each year an epprobate award for excellence will be awarded to the courseware judged to be most outstanding amongst those awarded the epprobate label during that year.

18

Epprobate quality grid

Each courseware is evaluated for

A. Course designB. Learning designC. Media designD. Content

19

A. COURSE DESIGN

Provision of course information, learning objectives and instructional guidance The courseware provides learners with all the

information needed, including clear and appropriate learning objectives and sufficient instructional guidance.

Constructive alignment The learning objectives, instructional strategies and

assessment processes (where provided within the courseware) are congruently aligned with each other, so that the instructional strategies used can be seen to be effectively assisting learners in achieving the intended learning objectives.

20

Self-evaluation

Assessment and importance

Comments

For each topic the producer fills information in the following table

Example

Assessment and importance

Comments

C3This course is intended to meet the needs of students who wish to have a basic introduction to the nature of design thinking. It is expected that the audience will therefore be self selecting. There is no attempt made to adapt the materials to individual needs though the activities allow for expression of personal perspectives.

21

Review

Assessment and importance

Good features Issues Other comments

For each topic, each reviewer fills information in the following table

There is no intention for any single criterion to be essential, but a courseware supplier in their self assessment document indicates to what extent they meet a specific criteria.

To every characteristic is assigned a mark:A - Exceeds the requirements of the criterionB - Meets the requirements of the criterionC - Meets some but not all of the requirements of the criterionD - Fails to meet the requirements of the criterion

22

Review - example

Assessment and importance

Good features Issues Other comments

A1 The content and activities are well balanced and form a mix of examples and individual tasks, followed by a narrative guidance.

Assessment is modelled to be mostly self-conducted in form of a practical task or discussion.

23

B. LEARNING DESIGN

1. Learner needs The instructional strategies meet the learner's needs, and are matched to the

learner’s profile (skill level, age group, specific constraints, etc.)

2. Personalisation The courseware allows personalization of the learning process and supports the

development of learner autonomy.

3. Instructional strategies The courseware provides an appropriate range of instructional strategies that

motivate and engage learners and support active learning. The instructional strategies support learning through an appropriate range of the

forms of learning: Acquisition – where learning is done by listening to, reading, or watching materials that are

presented in such a way as to gain the attention and interest of the learner Inquiry - where the learner is provided with a focus for enquiry, resources, and guidance for

the learning process Practice - where the learner is provided with interactive exercises, games, opportunities to

interact with representations of the domain, or other activities that provide feedback Communication - where the learner is encouraged and supported in communicating with

other learners (formally and/or informally) and/or with tutors (e.g. coaching) Construction – where the learner is provided with opportunities to learn, or to demonstrate

the outcome of their learning, through producing something - such as an essay, or a design.

24

C. MEDIA DESIGN

1. Media integration The utilization of media (i.e. text, pictures, audio, and video) and

tools effectively enhances comprehension of the course content and empowers implementation of the instructional strategies.

2. Interface The courseware interface (in terms of navigation and web

design) is user friendly, corresponds to current practice, and allows learners to efficiently monitor their progress through the course.

3. Interoperability and technological standards The courseware complies with common technical standards (e.g.

SCORM, IMS) thus supporting interoperability.

25

D. CONTENT

1. Accuracy and values of content The content is an accurate representation of the domain from a

particular perspective or range of perspectives. Where appropriate issues of values are addressed openly. The content is written so as avoid culturally biased terms and examples.

2. Intellectual property rights  The courseware supplier has the rights to use the materials in

the courseware. Where material is provided as an OER the rights of use and reuse are clearly stated with an appropriate Creative Commons license.

3. Legal compliance The content abides by all relevant national and international

legislation with reference to content, including laws related to bias, slander, and promotion of specific values.

26

Conclusion

There is a certain progress made in quality assurance in e-learning. The challenges remains to be faced with,

such as internationalization, distinction in QA scale, and usage of objective and comprehensive – yet not too complex – methodology.

27

Conclusion

Epprobate is an initiative that tries to handle mentioned challenges. It is an international label, which engages

numerous national partners, it treats e-learning at the courseware level, making the QA available to different types of producers (universities, education agencies, companies) and it uses a methodology that compounds all vital criteria through a well defined procedure.