an ontology-based learning design assistant

An Ontology-based Learning Design

Assistant

Valéry Psyché1, 2, Jacqueline Bourdeau1, Roger Nkambou2,1 LICEF Research Center, TELUQ, 100 Sherbrooke West,

Montreal (QC) H2X 3P2, Canada2 GDAC Research Center, UQAM, P.O.B. 8888, succ. C. V.,

Montreal (QC) H3C 3P8, Canada

{valery.psyche, jacqueline.bourdeau}@licef.ca; [email protected]

mailto:[email protected]


Psyché, Bourdeau and Nkambou – LORNET Workshop at ITS 2008 2

Plan

INTRODUCTION RESEARCH CONTEXT AND ISSUES

• Research context and state of the art• Formulation of the research issues and

proposition RESULTING DELIVERABLES

• An ontology of LD theories• CIAO, the ontology-based LD assistant

EVALUATION• Results of the qualitative evaluation conducted

CONCLUSION• Main contributions are summarized, and the

further work is outlined


Introduction

Research goal To describe the contribution of Ontological

Engineering (OE) in the engineering of ITS/ TEL

In order to illustrate this evidence: We have chosen the assistance to learning designers

as an example of a situation where the OE is useful

The challenges addressed are the following: “How is it possible to assist learning designers during

their frequently complex tasks when authoring systems offer neither any of the assistance required to design semantically valid learning scenarios in an educational theories viewpoint nor the means to make such a semantic validation?”

Authoring systems fail to offer access to learning design theories and to offer services related to them


Plan









Research Context and State of the Art

Murray’s classification of authoring systems (Murray T., 1999-2003) Authoring systems can be organized according to the

type of learning systems or intelligent tutors they produce, and they focus either on pedagogy or performance

Pedagogy-oriented systems are actually based on pedagogical modeling, while

Performance-oriented systems are based on the creation of a learning environment to materialize pedagogical models

According to our research subject, our focus is on pedagogy-oriented authoring systems



In pedagogy-oriented authoring systems Most instructional strategies, on which rely the

pedagogical model, are based on behaviorist / empiricist paradigm (Jonassen D.H. & Reeves T.C, 1996)

Numerous representation methods are used to model the pedagogical expertise, but at the end,the resulting models cannot be modified

There is usually no representation method based on declarative knowledge as in ontologies.

The more flexible systems are “intelligent and adaptive hypermedia”



Particularities of “intelligent/adaptive hypermedia” pedagogy-oriented authoring systems They used hyperlinks in order to facilitate the

presentation/ sequencing of the content presented to the learner

• Hyperlinks can be intelligently filtered, sorted and annotated with respect to a learner’s model (Brusilovsky, 1998)

• Hyperlinks can be based on curriculum sequencing, multiple knowledge or instructional strategies

Their strategies are mostly based on the cognitivist/rationalist paradigm

And, they are increasingly based on eLearning/LD standards and semantic web technologies

But • They fail to provide designers with a pedagogical

model based on a multi-paradigms/theories representation


Research Issues and proposition

Main hypothesis The primary cause of this lack in authoring

systems is a non-explicit representation of the learning design (LD) theories/paradigms in their pedagogical model

In order to test this hypothesis, we designed: An ontology of Learning Design Theories An Ontology-based Learning Design Assistant

which provide designers with the assistance they need when using authoring systems • This agent is called CIAO

We favor an OE approach based on the Semantic Web approach To take advantage of its well-established and

standardized technologies


Plan









Resulting deliverablesConceptualization of the

LD Theories Ontology

Main steps of OE methodStep 3. Conceptualization

Step 6. Formalization

Step 9. Operationnalization

Deliverables

Conceptual ontology

Formal ontology

Operational ontology through the ontology-based LD assistant



LD TheoriesOntology The conceptual ontology is built from:

A requirements document which contains:– Usage context– Information sources– Ontology users and usage scenarios – …

A « core ontology » also called « baseline ontology » (Staab S. et al., 01) which contains:

– little but important candidates terms The reuse and integration of ontologies or

knowledge models– EML/IMLSLD


Resulting deliverablesFormalization of the


Deliverables

Conceptual ontology

Formal ontology






Resulting deliverablesFormalization of the


The formal ontology is translated in: RDF(S)/OWL format as shown in examples:

• Classes : – Theory; learning_design

• Sub-classes :– empiricist_paradigm; learning_activity

• Properties:– has_paradigm; part_of_learning-design

• Instance:– Gagné; Gagné_principle1


Resulting deliverablesCIAO, the Ontology-based LD Assistant:

towards an Operationalization of the LD Theories Ontology

Deliverables

Conceptual ontology

Formal ontology






Resulting deliverablesOperationalization of the LD Theories Ontology

An operational ontology was needed in order to exploit CIAO’s services

Initially stored in a repository in an OWL file, The ontology was then partitioned and converted into

two RDF/RDFS files: 1. an RDFS file for the ontology class scheme (or T-box)

2. an RDF file for the ontology instances (or A-box). The T-box contains the axioms that describe the ontology

classes and properties (i.e. terminological declaration) while

The A-box holds statements pertaining to individuals in the field (i.e. assertion data).

The repository that includes the ontology is composed of a stack of Storage And Inference Layers (SAIL)


Resulting deliverablesCIAO, the Ontology-based LD Assistant: towards an

Operationalization of the LD Theories Ontology

Hozo/Protégé

Operationalizationthrough the

Implementation of Ontology-based

LD assistantCIAO


Resulting deliverablesCIAO, the Ontology-based LD Assistant: towards an

Operationalization of the LD Theories Ontology CIAO

Interacts with the operational ontology and with Sesame in order to • Gives access to the assistance services:

– Exploration, Search by query, Export, Validation of LD scenario

• Accessible at :http://www.licef.ca/CIAO

http://www.licef.ca/CIAO


Resulting deliverablesCIAO, the Ontology-based LD assistant

Ontology Documentation

Class Hierarchy

Class Description

Ontology Repository

The exploration service



The Search service: Users can query the

ontology using 3 modes SeRQL is an SQL-

based language that works with ontologies in RDF(S)

Parameter-based

request

Predefined request

Free request in

SeRQL


Resulting deliverablesCIAO, the Ontology-based LD assistant The validation service

Two types of analysis can be performed on a LD scenario• The syntactic analysis

– Allow users to check whether their scenarios are compliant with the IMS-LD standard

» Ex. Rule Rsy#0 : For each LD, there must have at least one « method »

• The semantic analysis – A partial semantic validation on a non annotated

LD scenario, according to the educational paradigms.

» Ex. Rule RseP#2: Detection of sequential activity (learning ou support activity) in an « act »

– A complete semantic validation on an annotated LD scenario based on educational theories:

» Ex. Gagné-Briggs’s Theory of Instruction case



Associated use case

MOT+LD

CIAO



Associated use case

Constructing an LD scenario

with

Syntactic and semantic analysis

of a IMS-LDLD scenariousing CIAO

MOT+LD

CIAO


Plan









Evaluation of deliverablesEvaluation Process of

CIAO and the LD theories ontology An evaluation by expert inspection was conducted 5 experts in the field of Learning Design participated in the

evaluation The experimental laboratory called LORIT at the TELUQ

enabled the observation and collection of multimedia data from multiple sources

The protocol which led to the collection of expert advices involved the following 3 stages: (1) Learning about the subject of the evaluation, (2) Talking while discovering CIAO and the ontology and (3) Answering interview questions The last 2 steps were recorded in order to collect audio

and video data with screenshots to match the experts’ actions.

The instruments provided to the experts were: An assessment guide, a graphical view of the ontology, IMS-LD scenarios for the syntactic and the semantic analysis


Evaluation of deliverablesEvaluation Results of

CIAO and the LD theories ontology

Experts agreed that the services provided by CIAO and the LD theories ontology were useful, flexible and interoperable In particular, the results indicate a complete convergence

of views about the usefulness of the following services:• Exploration, • Research by predefined queries• Syntactic and semantic analysis of LD scenario

Finally, experts found that CIAO lacked usability in its graphical interface.

The experts’ assessment led to the following recommendations: To improve the CIAO’s usability as it is an obstacle to

mastering and using the system services To make LD scenarios repositories available to learning

designers


Plan









Conclusion

This research introduced The limitations for learning designers when

performing their task The lack of representation of pedagogical and

declarative knowledge in authoring systems The designers’ need for assistance to design learning

scenarios. Our solution contains

The engineering of an ontology of LD theories The design and implementation of an ontology-based LD

assistant named CIAO which provide a set of services to the learning designers

Concerning the evaluation, results show that: CIAO lacked usability The LD theories ontology and CIAO’s services were

useful, relevant, and flexible


Conclusion

Future work planned concerning the ontology To extend the scope of the ontology from three to

a greater number of theories (including eclectic theories).

In order to refine the CIAO’s analysis service by adding new semantic validation rules in accordance with LD theories.

Future developments planned concerning CIAO To complete CIAO’s implementation in order to

make it a proactive system (coaching in pedagogical situations)

To perform CIAO’s integration into the TELOS system

An Ontology-based Learning Design

Assistant

Valéry Psyché1, 2, Jacqueline Bourdeau1, Roger Nkambou2,1 LICEF Research Center, TELUQ, 100 Sherbrooke West,

Montreal (QC) H2X 3P2, Canada2 GDAC Research Center, UQAM, P.O.B. 8888, succ. C. V.,

Montreal (QC) H3C 3P8, Canada

{valery.psyche, jacqueline.bourdeau}@licef.ca; [email protected]





Performance-Oriented Authoring Systems (Murray,1999-2003) Device Simulation and Equipment Training Systems

• E.g. DIAG, RIDES, MITT-Writer, ICAT, SIMQUEST, XAIDA Domain Expert Systems

• E.g. Demontr8, D3 Trainer, Training Express Special Purpose Systems

• E.g. IDLE-Tools/IMap, LAT

Pedagogy-Oriented Authoring Systems Curriculum Sequencing and Planning Systems

• E.g. DOCENT, IDE, ISD Expert, Expert CML, CREAM-Tools Tutoring Strategies Systems

• E.g. Eon, GTE, REDEEM, SmartTrainer AT Multiple Knowledge Types Systems

• E.g. CREAM-Tools, DNA, ID-Expert, IRIS, XAIDA Intelligent/Adaptive Hypermedia Systems

• E.g. CALAT, GETMAS, Interbook, MetaLinks


Resulting deliverablesUsage context

In Ontology-based Authoring Environment authors could benefit from accessing theories to: 1) 1) make design decisions (macro, micro) after reflection

and reasoning, 2) 2) communicate about or explain their design decisions,3) 3) check consistency among design decisions, intra-

theory and inter-theories, 4) 4) produce scrutable learning environments, 5) 5) use heuristical knowledge grounded in theoretical

knowledge. Useful functionalities could include:

1) 1) asking the system what theories apply best to this or that learning situation/goal,

2) 2) asking the system to show examples, 3) 3) asking the system for advice on whether this element

of a theory can be combined to an element from another theory, the risk in doing so, other preferable solutions, etc.


Resulting deliverablesInformation sources

Domain definition and documentation « Instructional Theories in Action: lessons illustrating,

selected theories and models » (Reigeluth, 93) « Instructional-Design Theories and Models: An

Overview of their Current Status » (Reigeluth, 83) « Instructional-Design Theories and Models: A New

Paradigm of Instructional Theory » (Reigeluth, 99) « Cognition and Learning » (Greeno et al., 96) « Theories Into Practice (TIP) » (Kearsley, 94-04)

Integration and reuse of ontologies/knowledge models: EML (Koper, 01; Koper, 03) IMS-LD (IMS Global Learning Consortium, 02)


Resulting deliverables: Usage scenariosscenario based on the following objective:

« Learners will be able to define focal lenght »

Usage scenario # 2Scenario # 2.1 –Theory of Instruction (Gagné-Briggs)

Scenario # 2.2 –Component display

Theory (Merrill)

Scenario # 2.3 –Inquiry Teaching Theory (Collins)

Preriquisite Identify focal point Remember – Instance – Fact, Use - Concept

Terminology about lens

Instructional strategy Expository Expository and Practice Inquiry Dialogue

Learning material (resource)

Workbook with visual representation

Printed Study Sheet Lenses, magnifying glass, lenses

Evaluation Quiz Fill in the blank test Question test

Activities based on the theory:

1) Informing the learner of the lesson objective,

2) Presenting the stimulus material with distinctive features,

3) Providing learning guidance,

4) Eliciting performance, 5) Providing informative

feedback,6) Assessing

Performance

1) Introduction, 2) Reference Example, 3) Definition, 4) Learning Tip, 5) Practice, 6) Study Problems, 7) Feedback

1) Preliminary, 2) Forming hypothesis, 3) Testing hypothesis, 4) Entrapping

Students, 5) Varying cases

systematically, 6) Questioning

authority, 7) Testing hypothesis



LD TheoriesOntology

The conceptual LD Theories Ontology in UML




Describes Type of content

Describes how units of studies are modeled

Describes theories, principles and

models of instruction

Describes how learners learn

Reuse and integration of ontologies or knowledge models: The EML pedagogical model



LD Theories Ontology Reuse and integration of ontologies or knowledge

models: The IMS-LD model


Resulting deliverables Formalization of the LD Theories Ontology

Description of the class « theory »

<owl:Class rdf:ID="theory"> <rdfs:comment rdf:datatype="http://www.w3.org/2001/XMLSchema#string"> EML: Describes the theories, principles and models of instruction as they described are in the literature or as they are conceived in the head of practitioners.</rdfs:comment> </owl:Class>


Resulting deliverablesFormalization of the LD Theories Ontology

Description of the class « learning_design »

<owl:Class rdf:about="#learning_design"> <rdfs:comment rdf:datatype="http://www.w3.org/2001/XMLSchema#string"> A learning design is a description of a method enabling learners to attain certain learning objectives by performing certain learning activities in a certain order in the context of a certain learning environment. A learning design is based on the pedagogical principles of the designer and on specific domain and contexts variables. </rdfs:comment> </owl:Class>



Description of the sub class « empiricist_paradigm »

<owl:Class rdf:ID="empiricist_paradigm"> <rdfs:comment rdf:datatype="http://www.w3.org/2001/XMLSchema#string">EML: According to the empirical approach, as typified by Locke and Thorndike, all reliable knowledge is based on experience. The assumption is that behaviour is predictable, given a specific environmental conditions, and that processes can be analysed in isolation. ...</rdfs:comment> <rdfs:subClassOf> <owl:Class rdf:ID="paradigm"/> </rdfs:subClassOf></owl:Class>



Description of the sub class« learning_activity »

<owl:Class rdf:ID="learning_activity"> <owl:disjointWith> <owl:Class rdf:ID="support_activity"/> </owl:disjointWith> <rdfs:subClassOf> <owl:Class rdf:ID="activities"/> </rdfs:subClassOf> <rdfs:comment rdf:datatype="http://www.w3.org/2001/XMLSchema#string"> IMS-LD: A learning activity is directed at attaining a learning objective per individual user. Any user performs a learning activity only once (until completion)</rdfs:comment> </owl:Class>



Description of the sub class « events_of_instruction »

<owl:Class rdf:about="#Gagne_events_of_instruction"> <rdfs:subClassOf> <owl:Class rdf:about="#support_activity"/> </rdfs:subClassOf> <rdfs:comment rdf:datatype="http://www.w3.org/2001/XMLSchema#string"> According to Robert Gagne, there are nine events that activate processes needed for effective learning. These events should satisfy or provide the necessary conditions for learning and serve as the basis for designing instruction and selecting appropriate media. Gagne believes all lessons should include this sequence of events. </rdfs:comment></owl:Class>



Extracted from the ontology in OWL Description of the property «has_paradigm»

<owl:ObjectProperty rdf:ID="has_paradigm"> <rdfs:range rdf:resource="#paradigm"/> <rdfs:domain rdf:resource="#theory"/> <owl:inverseOf> <owl:ObjectProperty rdf:ID="is_paradigm_of"/> </owl:inverseOf></owl:ObjectProperty>



Description of the property «part_of_learning-design»

<owl:ObjectProperty rdf:ID="part_of_learning-design"> <rdfs:domain> <owl:Class> <owl:unionOf rdf:parseType="Collection"> <owl:Class rdf:about="#learning_objectives"/> <owl:Class rdf:about="#prerequisites"/> <owl:Class rdf:about="#components"/> <owl:Class rdf:about="#method"/> </owl:unionOf> </owl:Class> </rdfs:domain> <rdfs:range rdf:resource="#learning_design"/> </owl:ObjectProperty>



Description of the instance « theorist»

<theorist rdf:ID=“Gagné"> <is_theorist_of rdf:resource="#Gagne_Briggs_Theory"/> <rdfs:comment rdf:datatype="http://www.w3.org/2001/XMLSchema#string"> Although Gagne's earlier work reflected behaviorist thought, he is considered to be an experimental psychologist who is concerned with learning and instruction. In 1965, Gagne published The Conditions of Learning which outlined the relation of learning objectives to appropriate instructional designs. Gagne's idea is tied to Skinner's idea of sequenced learning events as displayed in his Nine Events of Instruction. </rdfs:comment></theorist>



Description of the instance «Gagne_principle»

<Gagne_principle rdf:ID="Gagne_principle1"> <is_principle_of rdf:resource="#Gagne_Briggs_Theory"/> <rdfs:comment rdf:datatype= "http://www.w3.org/2001/XMLSchema#string"> Different instruction is required for different learning outcomes. </rdfs:comment> </Gagne_principle>


Resulting deliverablesExploration Service: Class Hierarchy


Resulting deliverablesExploration Service: Class Description


Resulting deliverablesExploration Service: Ontology Documentation


Resulting deliverablesExploration Service: Ontology repository


Resulting deliverablesResearch service: predefinied queries


Resulting deliverablesResearch service: parameter-based queries


Resulting deliverablesResearch service: by coding in SeRQL


Resulting deliverables Syntactic validation service

Rule Rsy#0For each LD, there must have at least one « method »

Trigger condition

The LD does not have « method » associated with: learning-design ( ( method))

Structure of the rule

For each learning-design Check if there is 1 method ; Si YES, go to rule Rsy1 ; IF NOT { Display message ER0 ET stop the validation}

Error message #0 associated (ER0)

“The LD is NOT syntactically CORRECT. For each LD, there must have AT LEAST 1 method”

Method


Resulting deliverables Partial semantic validation service

Rule RseP#2. Detection of sequential activity (learning ou support activity) in an « act »

Trigger condition There are learning or support activities in sequence in the act : activity-structure structure-type="sequence" ( (learning-activity = N) (support-activity = M))

Equivalent XML-IMSLD <activity-structure identifier="AS-introduction" number-to-select="2" structure-type="sequence"> <learning-activity-ref ref="fuel-valve-lesson-intro"/> <learning-activity-ref ref="fuel-valve-theory"/></activity-structure>.

Structure of the rule FOR EACH activity-structure IF structure-type="sequence"THEN { READ the value of number-to-select; COUNT the number N of learning-activity-ref; COUNT the number M of support-activity-ref; DISPLAY the RC2 message GO to RseP#3}

Recommandation #2(RC2)

“ You have (number-to-select) sequential activities: N learning et M teaching activities. A non exhaustive list of theories which talks about “sequencing of instruction is”: Gagné-Briggs Th., Collin’s Th...”

R

C

P Learning Activity#4

C

R

Learning Activity#1

Activity Structure#1

Learner role#1


Resulting deliverables Complete semantic validation service

an ontology-based learning design assistant

Documents