next steps for excellence in the quality of e-learning
DESCRIPTION
The development of e-learning has progressed to a stage where it is becoming part of mainstream provision in higher education. Therefore the issue of assessing and sustaining the quality of e-learning must now come to the fore. Quality assessment in higher education is well-established in relation to learning and teaching generally, but what methods can be used to establish quality in the domain of e-learning? The E-xcellence methodology for assessing quality in e-learning (EADTU 2009) is securing recognition by European and international learning organisations. It was designed to be applied to the design and delivery of e-learning in both distance learning and blended learning contexts. It supports a range of uses, from accreditation by external agencies to process improvement through internal review. The methodology presents principles of good practice in six domains of e-learning: strategic management; curriculum design; course design; course delivery; student support; and staff support. A total of 33 benchmark statements cover these domains, and are supported by a handbook for practitioners and guidance for assessors. The handbook includes principles for quality e-learning and exemplars of good practice. Amongst the tools is an online ‘QuickScan’ self-evaluation questionnaire based on the E-xcellence benchmarks which is highly valued as a focus for collaborative review of e-learning programmes. The e-learning landscape has changed since the E-xcellence methodology was first developed. In particular, the use of Open Education Resources (OECD 2007) and the application of social networking tools (Mason & Rennie 2008) were not explicitly considered in the original benchmarks. Accordingly, the E-xcellence NEXT project was instigated to produce and evaluate a revision of the benchmark criteria, associated handbook and exemplars. This paper describes the project process and initial recommendations. A consultation exercise was carried out among E-xcellence participants. Feedback from this was brought to participatory workshops at a European Seminar on QA in e-learning in June 2011. Following this exercise, the benchmark statements were revised and are now available in beta version. The project resources (Quickscan and manual) are being used for a series of self-evaluation and assessment seminars held at European higher education institutions. Feedback from these assessment seminars will be used to finalise materials for publication late in 2012. At that point the E-xcellence Next project will offer to the higher education community a set of self-evaluation and quality assessment tools which are fully updated to encompass social networking, Open Educational Resources and other recent developments in e-learning.TRANSCRIPT
Next Steps for Excellence in the
Quality of e-Learning
ALT-C, 11-13th Sept 2012
Jon Rosewell, Karen Kear, Keith WilliamsDept of Communications and Systems, Faculty of Maths Computing and Technology, The Open University, UK
E-xcellence project 2005–presentFunded by EU Lifelong Learning programme
Managed by EADTU• E-xcellence 2005-06
– Development and trialling of criteria, handbooks and methodology
• E-xcellence plus 2008-09 – Dissemination to institutions and to QA agencies in 9 European
countries• E-xcellence NEXT 2011-12
– Continuing dissemination and updating of criteria and resources
E-xcellence: modes of use• Informal self-evaluation
– Use Quickscan• Local seminar
– Local use of Quickscan with justification for rating– Meeting: institution, project team, national QA agency– Improvement roadmap
• Full assessment– As above but part of formal accreditation– Evidence provided for benchmarks
E-xcellence NEXT: updating• General updating of manual
– Clarifying language / terminology• Deal with emerging trends
– Convergence between distance and F2F blended modes– Social networking in HE– Use of Open Educational Resources
• Process:– Quickscan comments from partners– Participatory workshops– Feedback from local seminars
E-xcellence NEXT: social networking• How might social networking contribute to high quality in
e-learning?• What risks to quality might arise?• Which of the existing E-xcellence quality benchmarks
might apply in this context?• Are any new benchmarks needed to cover this
scenario?
Social networking• Where:
– Online communication: forums, blogs, wikis, …– Sites: Facebook, Twitter, LinkedIn, …
• Why:– Learning: social learning, collaborative work– Building communities: motivation, progress, social
• Issues:– Public (Facebook etc) or walled-garden (VLE)?– Boundaries and invasion of student space?
Revised benchmarks – social networking
Curriculum design
10.Curricula are designed to enable participation in academic communities via online social networking tools. These online communities provide opportunities for collaborative learning, contact with external professionals and involvement in research and professional activities.
Indicators – social networking• There are institutional policies relating to the provision of online community
spaces for student-student and student-teacher interactions.• Curriculum designers specify clearly the educational role that student-student
interaction plays in their programmes.• Criteria for the assessment of student online collaboration exist and are applied
consistently across programmes and courses.
At excellence level:• Teaching staff are supported by formal and informal staff development activity
in the use of online tools for community building.• The institution works closely with professional bodies in the development of
online professional communities.• Innovative assessment approaches, such as online collaborative work, peer
assessment and self-assessment, form a part of the institution’s practice in this area.
E-xcellence NEXT: OERs• How might OERs contribute to high quality in e-learning?• What risks to quality might arise?• Which of the existing E-xcellence quality benchmarks
might apply in this context?• Are any new benchmarks needed to cover this
scenario?
ProvenanceReputation
Brand
ProvenanceReputation
Brand
creation use
user recommendationpeer review
OERrepository
OERrepository
Quality points
checking
Quality Dimensions
ContentAccuracyCurrencyRelevance
Pedagogic EffectivenessLearning objectivesPrerequisitesLearning designLearning stylesAssessment
Reusability & opennessFormat & interoperabilityLocalisationDiscoverability: metadataDigital preservationAccessibility
Ease of useClarityVisual attractiveness, engagingClear navigationFunctional!
Revised benchmarks – OERsCourse design
14.OER material is selected with regard to learning outcome, tailored if necessary for fit to the learning context, and integrated with other learning materials. OER materials are subject to the same review processes as other course materials.
Indicators – OERs• The institution has a policy for use of independent learning
materials from a number of quality assured sources, including OER.
• Course materials obtained from OER are judged fit for purpose by students and external assessors.
• There is a principled approach to judging the quality of material obtained from an OER repository.
• There is a process for tracking intellectual property rights associated with e-learning components.
At excellence level• E-learning components are contributed to repositories as OER.
Updating E-xcellence resources• Work done (published end Sept):
– Revised benchmarks and Quickscan– Manual
• Edited for language, relevance to blended learning• Including social networking and OER
• Work still to be completed:– Update assessors’ notes
Local seminars – purpose• To discuss with HE institutions the quality of e-learning
on the basis of the benchmarks• To explore with QA agencies how to incorporate e-
learning into their frameworks• To exchange ideas during an on-site visit• To improve process:
– Exchange experience on the E-xcellence framework and the Quickscan
– Collect feedback on tools
Local seminars – format• Preparation
– Participants: managers, staff members, course designers, tutors, students– Decide programme to be assessed– Select some or all benchmarks – Team meets to complete QuickScan self-evaluation
• Seminar– First day: local team meet with assessors– Second day: local team, assessors and national QA agency
• Report– From assessors– Roadmap for improvement from institution
Local seminars 2011-12• Russia MESI University, Moscow• Lithuania Kaunas University of Technology• Poland Akademia Górniczo-Hutnicza, Krakow • Cyprus Open University of Cyprus, Nicosia• Latvia Riga Technical University• Portugal Universidade Aberta, Lisbon • Greece Hellenic Open University, Patras
Local seminar feedback (1)• Framework
– Quickscan is valuable to structure discussion– Completeness of the framework is appreciated
• Team working– People exchange perspectives with other departments
• External perspective– Exchange of experience between the evaluators and staff
was valuable– New ideas surfaced for course design
Local seminar feedback (2)• Reflection
– A valued ‘moment of reflection’ on quality– People become aware of choices and implementations– Gives insight into strengths and weaknesses
• Analysis– Opportunity to formulate e-learning policy– Provides foundations for decision making
Comments and feedback?
Web: http://www.eadtu.nl/e-xcellencelabel/
Email: [email protected]
Thank you for your attention
• ‘Social networking’ can be interpreted broadly to cover a range of online communication processes– e.g. via forums, blogs, wikis
• It can also be interpreted more narrowly to focus on social network sites that provide accessible tools– e.g. Facebook, Twitter, LinkedIn
What do we mean by ‘social networking’?
Why use social networking?• Social networking has two primary purposes in
education:– facilitating learning
• social learning theories• focused pedagogic function such as group work,
peer assessment– building communities
• motivation and progress• informal and social
Social networking tools
• Forums discussion and debate• Wikis co-creation of resources• Blogs reflection, sharing and
feedback• Social network sitessense of community
• Public (Facebook etc) or walled-garden (VLE)?– Boundaries and invasion of student space?
Social network sites
Benefits• many students already
use them regularly• seen as more social,
informal and flexible
Challenges• privacy issues • lack of control • blurring of boundaries
between social and academic life.
OER use-cases• Life-long learner finds material for independent study• Individual teacher uses assets in own material• Course uses podcasts from iTunes U• Course uses a 10-hour unit• Entire 100-hour module reused, with new assessment• Course and assignments in OER; tutorial / marking /
accreditation offered for fee• Consortium develops material for own use and ‘frees’ it
Trends toward Open Educational Practice?use create
teacher centred learner centred
transmission constructivism
(sage on stage) (guide on side)
focus on outcome focus on process
standardised personalised learning
individual social/ peer learning
Capability maturity model:
Use OERs Adapt OER material Create OER material
See, for example, OPAL OEP Guidehttp://opal.innovationpros.net/publications/guide/