modeling units of assessment for sharing assessment process information: towards an assessment...
Post on 21-Dec-2015
216 views
TRANSCRIPT
Modeling Units of Assessment for Sharing Assessment Process Information:
towards an Assessment Process Specification
Yongwu Miao, Peter Sloep, and Rob Koper
Educational Technology Expertise CentreOpen University of The Netherlands
ICWL 2008, 20-22 August 2008, Jinhua, Zhejiang, China
Overview
• Where are we from: OUNL
• What we have done: EML => IMS LD
• What is missing in IMS LD: Assessment
• What is our objective: A New Specification
• What is the current status: Conceptual Model
• How can it be used: Olympic Games
• Summary
Where are we from: OUNL
The Open University of the Netherlands (OUNL)
– Started in 1984; national institute;– Open distance education provider
• 7 faculties, 24000 students• 24 study centres in Netherlands and Belgium• education delivered using a variety of
technologies (print, cd-rom, internet, face to face contact sessions, practical rooms, etc.)
– R&D: Educational Technology Expertise Center
What we have done: EML => IMS LD
• Educational Modelling Language (EML)• IMS Learning Design (LD): An e-learning technical specification
IMS LD is derived from EML
IMS Global Learning Consortium, Inc.OUNL is contributing member (voting rights) of IMS
– Also Apple, Blackboard, Microsoft, Cisco, Sun, Texas Instruments, …
1998
* Information model
* Best practice guide
* XML binding
2003
What we have done: EML => IMS LD
Basic idea:Develop a ‘language’ to describe the learning/teaching processes with associated content in a formal, semantic way, so that it can be interpreted by computers.
Compare: music notationThis description should be independent of any delivery platform => can be deployed in any platform that provides the adequate functionality
What we have done: EML => IMS LD
Learning object:- Book,- Audio & video
Whichnotationsare needed to describe a lesson plan?
What we have done: EML => IMS LD
IMS LD, an education-specific process modelling language, can be used to describe a wide variety of pedagogical strategies or approaches in terms of a play including acts and role-parts (based on metaphor of theatrical play)
•People act in different roles•working towards certain objectives•by performing learning and/or support activities•within an environment, consisting of learning objects and services used in the performance of the activities.
Moving focus from what to learn to how to learn: Lesson plan or learn-flow
play
Act 1 Act 2 Act 3 Act 4 Act 5
Role-part 1
Role-part 2
Role-part 3
Role-part …
Role Activity Environment
Learning objects
Learning services
Activity-Description
method
components
What we have done: EML => IMS LD
What we have done: EML => IMS LD
LD authoring tool
LD-compatiblerun-time environment
actual or imaged pedagogical processes
a formal pedagogical model
multiple executions of the same modelcarried out by different groups
the events that can be observed or expected
represented in LD(in the form of XML)
Run-time
Design-time
author
learner,teacher,and other stakeholder
What is missing in IMS LD: Assessment
LD can not explicitly support various types of assessment.
Assessment components within the Educational Modeling Language were excluded when LD was adopted by IMS, because of the existence of IMS Question and Test Interoperability (IMS QTI).
QTI is also an open e-learning technical standard, which describes a data model for the representation of assessment item/test and their results. It defines a set of interaction types which can be used to specify basic question types (e.g., multiple-choice, fill_in_the_blank, and slider) and complicated question types through combination.
What is our objective: A New Specification
1. facilitating representation, understanding, communication, comparison, and reuse of a variety of assessment practices at design-time;
2. guiding and scaffolding learner, tutors, and other stakeholders to perform a variety of activities at run-time by providing guidance and awareness information, by configuring a workspace for carrying out prescriptive tasks, by controlling and changing the sequence of activities based on the execution state and circumstantial information, and by orchestrating the efforts made by different roles/participants.
Develop a high-level assessment process modeling language for facilitating both experts and practitioners to share assessment process information by extending QTI and LD with an additional layer.
What is the current status: Conceptual Model
• Semantic aggregation model• Conceptual structure model• Process structure model
Developed a conceptual model, the core of the specification, which can be represented as:
The first validation study has been conducted through investigating whether the conceptual model of specification meets the requirements of completeness, flexibility, adaptability, and compatibility.
How can it be used: in Olympic Games
It is not necessary to develop a software applications for supporting each assessment
How it can be used: in Olympic Games
rate 2.1
perform1
Athlete Rater 1 Rater 2 Rater n
rate 1.1 rate n.1
rubricvalues 1.1
rubricvalues 2.1
rubricvalues n.1
rate 2.5
perform5
rate 1.5 rate n.5
rubricvalues 1.5
rubricvalues 2.5
rubricvalues n.5
collect and calculate5
score5 and total
…
…
…
calculate1
decisionrules
score1 publish and view result1
…
publish and view result5
system
assessmt.form
assessmt.form
assessmt.form
assessmt.form
assessmt.form
assessmt.form
How it can be used: in Olympic Games
Use existing Run-time Environment
Player.1 Player.2 Player.n
internet
Dispatcher/Service broker
IMS LD engine IMS QTI engine
A A
Assessment service
UoL managementand enactment service
…
an assessment model
multiple instances
server
How can it be used: in Olympic Games
An assessment model can be reused and modified to support other assessment
Summary
IMS LD can support interoperability and reusability of a variety of pedagogical models and systems.
IMS LD provides insufficient expressiveness to model integrated learning, teaching and assessment processes.
A high-level assessment process modeling language can be developed by extending IMS QTI and IMS LD with an additional layer.
The conceptual model of the specification has been developed and validated.