quality & effectiveness of e-learning courses – some experiences from singapore lim kin chew...
TRANSCRIPT
Quality & Effectiveness of E-Learning Courses – Some
Experiences from Singapore
Lim Kin Chew16 – 17 Dec 2010
7th International E-Learning Conference for a Knowledge-Based Society
Srisakdi Charmonman IT CenterAssumption University
Bangkok, Thailand
Contents
• Introduction• What is E-Learning Quality?• What is E-Learning Effectiveness?• Quality initiatives in Singapore• What more can we do to improve E-Learning
Quality?• Conclusion
This is my blog on Quality Matters in E-Learning. I started this blog some time in 2007 mainly to share my experiences and information on learning objects and e-learning in general.
http://eunisim.blogspot.com
Introduction
Request for ConsultancyReceived e-mail request from a government body’s e-
learning Team:
1. E-learning features that support the pedagogical design of e-learning content / instructional objectives
2. Criteria for e-learning effectiveness3. Evaluation of the various teaching & learning
domains, i.e. facilitation; transference; retention of learning, etc
4. Key competencies which e-learning aims to equip learners
Some deductions• Quality and effectiveness of e-learning or e-
training are becoming important issues, not just the development of contents or the use of LMSs
What is E-Learning Quality?• The predominant view is that quality relates
to obtaining the best learning achievements (50%). Together with 'something that is excellent in performance' (19%), this primarily pedagogical understanding was more widespread than options related to best value for money or marketing.
(from “Quality in e-Learning – use and dissemination of quality approaches in European e-Learning – A study by the European Observatory – www.cedefop.eu.int)
Business Times, Monday, 3 Sep 2001BizIT Section
Business Times is a newspaper in Singapore that deals mainly with business matters.
Quotation from BT journalist, Chua Hui Min:
“Effectiveness, rather than technology, is
the key, and defining this is perhaps both
the most difficult and important step in
deciding how to invest in e-learning.”
Entities where e-learning is implemented widely
• ICT Masterplan for schools• Polytechnics and Universities• Corporate bodies• Singapore Armed Forces• Civil Service College
The ICT Masterplan Journey - 4The ICT Masterplan Journey - 4
The ICT Masterplan Journey - 1• Managed by the Ministry of Education (www.moe.gov.sg)• E-learning was given a major boost with the implementation
of the following IT Masterplans for Education:– 1st Masterplan for IT in Education (1997 – 2002) – MP1– 2nd Masterplan for IT in Education (2003 – 2008) – MP2– 3rd Masterplan for IT in Education (2009 – 2014) – MP3
• 1st Masterplan for IT in Education: – Build the strong foundation for schools to harness ICT – Provide basic ICT infrastructure– Equip teachers with a basic level of ICT integration &
competency– 363 schools had fundamental building blocks in place to
use ICT in the curriculum– 30% of ICT usage in curriculum
The ICT Masterplan Journey - 2• The Second Information Technology in Education Study (or
SITES 2) conducted by the International Association for the Evaluation of Educational Achievement (or IEA) in 1999 reflected this paradigm change – accepting ICT as a tool in education.
• It reported that Singapore stood out favourably in terms of the level of teachers’ knowledge of ICT-based instructional practice.
• They had more avenues to develop their ICT skills than their counterparts in most other countries.
• Singapore principals were very positive towards ICT in schools.
• They had an attitude score of 90 on a scale of 0 to100. • Mental barriers of old mindsets had been successfully
breached.
The ICT Masterplan Journey - 3• 2nd IT Masterplan for Education:
– built on foundation from MP1 to strive for an effective and pervasive use of ICT in education by, for example,
• strengthening the integration of ICT into the curriculum,
• establishing baseline ICT standards for students, and • seeding innovative use of ICT among schools.
The ICT Masterplan Journey - 5The ICT Masterplan Journey - 5
The ICT Masterplan Journey - 6The ICT Masterplan Journey - 6
The ICT Masterplan Journey - 7• West Zone Sharing of Resources Project, WeSHARE
in short, is such an example of a first digital repository project developed for teachers by teachers in 2006.
• The project has now been expanded to every zone as part of the Inter-cluster Sharing of Resources project, or iSHARE. To date, a total of 12,450 teachers have gone into iSHARE both to upload and use the more than 66,000 resources in the repository.
• These resources have been metatagged using the CEMS (Content Exchange Metadata Standard) that is based on Dublin Core Metadata Set.
The ICT Masterplan Journey - 8
Lessons learnt:• A gap continues to exist between familiarity
with ICT and translating this into effective teaching.
• Teachers need to still base effective outcomes on sound pedagogical principles when they use ICT tools to bring out a learning point.
The ICT Masterplan Journey - 9Wrong use of technology:• Students were asked to use the Internet to search for
information. However, they were given only 15 minutes to discuss what they have found before they put up a PowerPoint presentation. What’s more important? Discussion or using technology?
• This use of ICT neither transforms nor enhances the learning experience.
Better use of technology:• There were well constructed lessons where teachers
require their primary students to use simple technological tools like discussion forums to create and write stories that teachers and their classmates could critique.
The ICT Masterplan Journey - 10• Masterplan for ICT in Education 3 Learning
Outcomes:– Strengthen competencies for self-directed learning– Tailor learning experiences according to the way that each
student learns best– Encourage students to go deeper and advance their
learning– Learn anywhere
Source: http://www.moe.gov.sg/media/speeches/2008/08/05/opening-address-by-dr-ng-eng-h-1.php
Polytechnics & Universities - 1• 5 polytechnics in Singapore:
– Singapore Polytechnic– Ngee Ann Polytechnic– Temasek Polytechnic– Nanyang Polytechnic– Republic Polytechnic
• All polytechnics and the Institute of Technical Education use e-learning widely.
• Every polytechnic has a dedicated department to look after the design and development of e-learning content and use of other ICT technologies like video conferencing.
• No centralization of e-learning contents.
Polytechnics and Universities - 2• Universities:
– National University of Singapore (NUS)– Nanyang Technological University (NTU)– Singapore Management University (SMU)– SIM University (UniSIM) – Private University– Singapore University of Technology and Design (SUTD)
• Quality of e-learning is only one component in the Quality Assurance Framework for Universities (QAFU). All universities must go through this QAFU audit. QAFU was set up in 2003.
• Responsibility of e-Learning rests with the individual university’s department responsible for e-learning.
• There is very little sharing or collaboration amongst all the universities. Partly this is because of rivalry.
Source: http://www.moe.gov.sg/media/press/2003/pr20030528.htm
Corporate Bodies -1• E-LEAP – E-Learning Early Adopters’
Programme:– Introduced by the IDA in 2002.– Purpose is to encourage companies and corporate
bodies to use e-Learning for staff training– Funding provided for non-governmental
companies to implement e-Learning
• E-Learning Competency Centre (ECC) helped to ascertain the courseware quality
Corporate Bodies - 2
CoursewareQuality
Corporate Bodies - 3
Source: International Journal of The Computer, the Internet and Management Vol. 12 No. 2 (May - August,
2004) pp 64 -71
Corporate Bodies - 4
• 2003 - 2004, the ECC developed some Courseware Evaluation Guidelines for the E-LEAP.
• Guidelines are on gathering evaluation opinions on 10 criteria
Corporate Bodies - 5–10 Quality assessment criteria
i. Interfaceii. Navigationiii. Operationiv. Contentv. Presentationvi. Practicevii.Feedbackviii.Assessmentix. Engagement Techniquesx. Support
Singapore Armed Forces – SPOT-ON - 1
• 1997 – Introduced the Self-Paced On Time On Need (SPOT-ON) programme
• Intention is to tailor as much training as possible in accordance to individual needs, rate of learning and time of convenience
• Initially the move was more of distance learning packages using manuals and CD-ROMs
• When the Internet became popular, e-Learning becomes an extension of SPOT-ON.
Singapore Armed Forces – SPOT-ON - 2
• 1999 - the Battalion Tactics Course (BTC) went online.• 2000 – the National Service Command and Staff Course
(NSCSC) went online.• The Instructional Technology Centre at SAFTI developed
a five-point management indicator to analyze the following:
• course results, • internal validation reports, • focus group surveys, • interviews and the • instructors’ feedback.
Singapore Armed Forces – SPOT-ON - 4Singapore Armed Forces – SPOT-ON - 4
The P’s ExplanationPurpose The effect of e-Learning on meeting the
organization's aim and course objectivesPlayers The effect of e-Learning on the students
and instructorsPedagogy The effect of e-Learning on adult
educationProcess The effect of e-Learning on training
system processes ranging from curriculum design to delivery
Performance The quality of the end-product
Civil Service of the Singapore GovernmentSome of the e-learning programmes for the civil servants are organized and offered by the Civil Service College of Singapore (http://www.ipamonline.com)
They also managed the quality & effectiveness of all the e-learning courses.
Ministries and statutory boards can choose from over 200 titles in the comprehensive library of e-Learning courses. Public organisations implementing e-Learning with Open Academy can now enjoy a 50% development grant on courses.!
CIVIL SERVICE COLLEGE - IPAM OPEN ACADEMY
Model used in SIM University
Quality & Effectiveness Model for UniSIM
Use of Kirkpatrick’s 4-level Model(done by ASTD in 1996)
Level 1 (Reaction) 92%
Level 2 (Learning) 34%
Level 3 (Behaviour) 11%
Level 4 (Results) 2%
Most of the e-learning courses do have some sort of evaluations done by the learners but these tend to be at Level 1 of the Kirkpatrick’s Model.
There is no available information on surveys done on the effectiveness.
What more can we do to improve e-Learning quality?
• Promote the awareness of research work on quality of e-learning e.g. the ISO/IEC JTC1 SC36 Working Group 5 on Quality assurance and descriptive frameworks
• Create collaborative projects involving quality and learning measurements
• Disseminate case studies on quality and learning measurements
• Share resources on quality and learning measurements
• Carry out quality audit of e-learning courses
What more can we do to improve e-Learning quality?
Move from activity measures to performance measure
Source:
http://www.elearningguild.com/pdf/2/110303mgt-h%5b1%5d.pdf
Conclusion - 1• All organizations doing e-learning know the importance and
value of checking the quality of e-learning.• However, there is no consensus of how to go about doing it in
a systematic way.• Some organizations, especially the government bodies, are
very interested in measuring the performance of participants who have completed e-learning courses.
• Schools are more interested in the pedagogical values that the students have attained.
• Quality checking and/or measurements of e-learning courses in the corporate bodies depend on the initiatives of their individual Human Resource Department.
• Much work need to be done in raising the need to monitoring the quality of e-learning programmes and also to measure it.
Conclusion - 2
From Robert Wager (1997):To rise from zeroTo big campus heroTo answer these questions you’ll strive to answer the following questions:
Where am I going,How shall I get there, andHow will I know I’ve arrived?
