integrating technical and training documentation workshop ... · technical training, authoring...

Integrating Technical and Training Documentation

Workshop ITS 2002 Conference

http://www-eurisco.onecert.fr/events/callworkshop.htm

Organising committee:

Yvonne Barnard, Monique Grandbastien, Robert de Hoog & Cyrille Desmoulins

Yvonne Barnard

EURISCO International (European Institute of Cognitive Sciences and Engineering)

4, avenue Edouard Belin, 31400 Toulouse, France

Tel. +33 (0) 5 62 17 38 27, Fax +33 (0) 5 62 17 38 39

Email: [email protected]

Monique Grandbastien,

INA P-G and LORIA Université Henri Poincaré Nancy 1, France

Email:[email protected]

Robert de Hoog

University of Amsterdam, Dept. of Social Science Informatics, the Netherlands


Cyrille Desmoulins

CLIPS IMAG & Université Joseph Fourier Grenoble 1, France


Keywords

Technical training, authoring systems, human factors, instructional design, learning environments, ontologies,meta-data

- 2 -2

Preface

In many complex operational domains, such as the operation of transport systems, military systems, aeronauticsand maintenance work, the actual work and the training to perform the tasks are separate worlds. Technicalmanuals, operational manuals and training materials are not developed in a coordinated way and they may differconsiderably, although the topics treated are the same. Some of the consequences are inefficient ways ofproducing material, leading to high development costs. Inconsistencies between different documents can lead topotential safety-related problems. Lack of coordination can result in ineffective ways of training and lack oftransfer of training, because procedures at work differ from the ones trained. Re-using technical material foreducational purposes, and vice versa, can be a solution, but it should be done in such a way that the specificneeds of both processes are well served.

The purpose of the workshop ”Integrating technical and training documentation” is to demonstrate and to discussdifferent approaches with people from different backgrounds, such as computer science, artificial intelligence,human factors and educational sciences, in order to develop new ideas to progress on this line of research. Re-using material for different purposes is the key-theme, addressing both practical questions of methods,organisation, tools and techniques as well as more fundamental questions about the differences between workand training and new approaches to bring these processes together. Several multi-disciplinary approaches areneeded to enable the development of innovative methods and tools:

• Advanced computer science techniques for handling (store, analyse, index, search, retrieve) of technicalmaterial (text, pictures, diagrams, video, sound).

• Use of knowledge engineering approaches, such as ontology development.

• Indexing mechanisms for marking-up material, both for operational and training uses.

• Structured pedagogical approaches for developing training and authoring training material.

• The use of advanced multimedia techniques for producing training material.

All papers included in this volume report on work in which a combination of these approaches were used.

Developing training material consists of several phases. The papers in this volume all contribute to one or moreof theses phases. In order to be able to re-use material, it is necessary to have it in the format of small objectswhich are indexed in a useful way. Using meta-data and ontologies are the main techniques which are describedin several papers. The use of standards plays an important role here. The developer of training material needs tosearch for suitable technical documentation. Several papers describe search processes, and related indexingmechanisms. Technical material can be re-used in different ways. Fragments from technical manuals can formthe raw material for creating training material, but they can also be transformed, such as into spoken dialogue orother multi-media forms. In order to be able to re-use technical material for educational purposes, the developerof training material will need a clear didactic frame (a scenario or a template) in which the material can beinserted in order to create lessons or training material. Innovative methods and tools to support this process aredescribed in several papers. Finally the training material has to be constructed in its definitive form, and providethe trainees with high-quality learning experiences. Examples of the final creation of training material, (re-)usingtechnical material are amply given by the authors of the papers.

The research presented in this volume is generally aimed at developing technology for enabling the integrationof technical and training material. By using standards, tools and methods, developers of training material aresupported in their task. But it is not only a matter of innovative software and emerging standards. The research isalso driven by underlying ideas on training and learning. The ultimate aim of integrating technical and trainingdocumentation is to produce innovative training material, which brings work and learning closer together.

- 3 -3

Workshop schedule

09.00 – 09.30 Introduction

09.30 – 10.30 Re-using technical material and indexing, paper presentations and discussion

10.30 – 11.00 Hands-on experience with indexing and document analysis tools in small groups

11.00 – 11.30 Coffee break

11.30 – 12.00 Continue with hands-on experience

12.00 – 12.30 Discussion on the experiences with working the tools

13.00 – 15.00 Lunch

15.00 – 16.00 Developing training documents from technical material, paper presentation anddiscussion

16.00 – 17.00 Hands-on experience with the training scenarios and training material development toolsin small groups

17.00 – 17.30 Coffee break

17.30 – 18.00 Discussion on the experiences with working with the tools

18.00 – 18.30 Discussion on future research in the area of integrating technical and training material

- 4 -4

Contents

Producing learning objects with XML-XSLT technology

Jean-Pierre David, Cécile Guilloux &Alexandre Flament

5

Re-using technical manuals for instruction: document analysis in the IMAT project

Robert de Hoog, Bob Wielinga, Suzanne Kabel, Anjo Anjewierden, Frans Verster, YvonneBarnard, Paolo DeLuca, Cyrille Desmoulins & Johan Riemersma

15

Re-using technical manuals for instruction: creating instructional material with the tools ofthe IMAT project

Robert de Hoog, Suzanne Kabel, Yvonne Barnard, Guy Boy, Paolo DeLuca, Cyrille Desmoulins,Johan Riemersma & Daniëlle Verstegen

27

A scenario tool for coupling human and computerised expertise in the development oftraining material for complex equipment maintenance

Yvonne Barnard, Cyrille Desmoulins & Monique Grandbastien

39

Generating Training and Assistive Dialogues for Astronauts from International SpaceStationTechnical Documentation

Gregory Aist & Beth Ann Hockey

49

Maintenance training design: Less is more

Lesley Jacobs, Anja van der Hulst & Stefan van der Stigchel

55

Personalized eLearning and eCoaching in WINDS

Marcus Specht, Milos Kravcik, Roland Klemke, Leonid Pesin & Rüdiger Hüttenhain

63

On the Social Rational Mirror’s architecture: Semantics and pragmatics of educationalinteractions

Daniele Maraschi, Germana M. Da Nobrega & Stefano A. Cerri

77

- 5 -5

Producing learning objects with XML-XSLT technology

Jean-Pierre David*, Cécile Guilloux** & Alexandre Flament**

*ARCADE team, CLIPS-IMAG GRENOBLE

[email protected]

**Centre de Développement ARIADNE de l’UJF à Grenoble

Abstract

We present a method for producing learning objects, using the technologies of information and communication,which results from modelling learning activities. The study of two case of learning objects, their expression inthe open standard XML and the use of the same tool for authoring them, prove the generic aspect of the chosenapproach.

Keywords

Modelling of educational activity, learning object, structured documents, learning objects generator, XML, XSL

Introduction

We shall speak about learning objects as a hypermedia document which allow the user to make an educationalactivity. We are interested in the definition of class of learning objects to justify the creation of generators ofthese objects. The research thus concerns several points: how to obtain a model of learning object?; How toexpress such a model? and What are the tools which can produce activities accorded to this model?

We have the conviction that for creating resources, teachers should be helped by authoring tools which containan educational methodology.

For creating such a software allowing teachers to produce learning objects, our team, having chosen proprietaryformats as our first productions, has yet determinedly opted for the interoperability of the data and thus for theirexpression in open standards.

Therefore, the emergence of the XML language in 1998, and the first recommendation for XSLT in 1999 gavepossibilities for expressing tree structures and transformation of these structures. The Document Type Definitionwhich defines a class of XML documents allows to express generic models. These standardised languages offerthe interoperability of all the produced data expressed through them.

We detail first of all the experience acquired through the production of autoevaluation exercises with thegenerator GenEval. We present then the GenDiapo project which is an application of the method built fromGenEval. For this class of documents, an editor has been developed and adapted for authoring too theautoevaluation exercises.

Modelling and producing self assessed exercise

Story of the GenEval project

In 1994, UJF Teachers of Mathematic and physics created exercises prototype in TooBook, based on a selfassessment pedagogical approach.

- 6 -6

These prototypes had been validated with an audience of teachers and students, it was observed that it lackedmethodology and tool to facilitate the production of this kind of exercises by teachers themselves. In an otherpart, the ARCADE team, since the production of its interactive environment for the learning of the algorithmichad be concerned to design authoring tools addressing teachers who were not computer specialists. For thesereasons, our team has been involved to design authoring tools within the framework of the European project ofResearch &Development Ariadne.

The « exercise » Learning Object

What is a self assessed exercise?

The educational activity which has to be modelised is a session of steered works in autonomy for the resolutionof a problem. The student is in situation to solve an exercise, with accesses to relevant information to understandstudied concepts, with orientations and progressive methodological helps. He can use or not the availabletheoretical or practical helps, he assesses his work, comparing the complete answer with his own. In this class oflearning objects “self assessed steered works ", the content on which the activity is supported must bedistinguished of the activity itself, which will be the same on all the contents.

The model is described on one hand by the elements which compose it and on the other by the breaking down ofthe tasks which are offered to the learner.

Prototype of the student interface

An interface, allowing the student activity according to the data of an exercise, was directly made in html, Javaand javascript. One Java applet remembers the time spent on every stage of the activity. Javascript code delaysthe access to the information, open a window on a definition of the glossary …

Figure 1: screenshot of an exercise.

All the information to understand the response is organized to be delivered on demand. A glossary is accessibleby hyperlinks on key terms of the domain of the exercise. At the end of every question, the student is invited toassess himself according to the criteria of competence defined by the teacher and at any time, he can consult theoutcome of his work in terms of time spent on every part of the exercise, the evaluation which he gave himselfon all the criteria and the total results by question.

Components of an exercise

The exercise is composed by a set of elements, themselves made up by a set of sub elements. A tree of elementsrepresents the whole exercise. The contents of the element outcome builds itself dynamically during the passageof the exercise by a student.

- 7 -7

Statement

illustration Animation

Method 1

Theoretical help

AUTOEVALUATION

“did you know it?”

Practical help

Glossary

Question1

outcome Concepts?

Short answer


Detailed answer


Method 2

Question2


Figure 2: Components of an exercise.

Task offered to the student

The scenario of activity expresses itself by a set of tasks which are offered to him at a given moment. Thefollowing scheme describes the tasks which are proposed to the student in the exercise.

Reading the statement of the exercise

Asking for the answer according to m2

Reading the "Did you know it?" rubrik

Solving a question

Choosing a solving method m1or m2 ...

Asking for theoretical help in m1

Self assessment

Asking for theoretical help in m2

Asking for the answer according to m1

Reading the glossary, the bibliography

Making a quizz on the concept of the exercise

Asking for practical help in m1

Asking for practical help in m2

Figure 3 : Task offered to a student which runs an autoevaluation exercise

The scenario is confidentially connected to the structure of the components of the exercise, but reflects additionalpedagogical choices. In the set of task presented in the scheme above, the order in which these tasks will beexecuted by the student is not compulsory, it will be simply suggested by the interface. But the teacher candecide that the kind of help, or the complete answer, are not available during that moment, to force the student towork by himself. The description of this kind of scenario can be expressed in various formalisms of tasks(DIANE, JSD, MAD). One can notice behind these formalisms the existence of a structure of tree which willafterward allow to implement them with XML.

- 8 -8

Using XML to express the model

The components of the exercise were expressed in the XML language, specific tags were defined and the treelikebrake down follows the oriented object design of the component of the exercise.

<?xml version="1.0" standalone="no"?><exercise_geneval_ii>

<head_gii><name>Etude statistique de lancers de

dé.</name><author>Jean-Pierre David</author><discipline>informatique</discipline><module>conception d'applications interactives

</module><aim>approche par objets</aim><duration>90</duration>

</head_gii><body_gii>

<autoevaluation_criterions><criterion>

<name>réutilisation</name><id>REUSE</id><statement>Capacité à réutiliser les

objets. </statement>

</criterion></autoevaluation_criterions>

/* Styles exercice geneval2.css */ /* Titre de la page */.head { font: 18pt sans-serif; font-weight:bold; color:#000060}

/* Concepts */.formulaire {font-family: sans-serif; color: #000060}.valid {text-align: center ; font-size: 14pt; }.return {border: 0pt;text-align: center; color: red; margin-top:15pt} /* enoncé */ /* texte de l'enoncé */.statement {font: 16pt serif; margin: 10pt} /* texte des remarques eventuelles */.nota {font: 14pt serif; margin-left: 10pt} /* illustration */ /* corps de l'illustration */.illst {color: #ffffff; background-color: #200000; padding:10px; text-align: center} /* Texte de l'illustration */.textillst {font: 14pt serif; text-align: center}

Figure 4: XML Description of an exercise and Style sheet for the presentation.

The presentation of the components on a web page is described by cascaded style sheet. So, the contents and thestructure of an exercise are expressed in a XML file whereas the presentations of the corresponding web pagesare described in CSS files, modifiable independently. The description of the class of the exercises GenEval isexpressed by a DTD.



<!ELEMENT head_gii (name, author, discipline, module, aim, duration)>

<!ELEMENT body_gii (autoevaluation_criterions, general_statement, question+, proposed_concepts?, glossary?,bibliography?, did_you_know_it?)>

<!ELEMENT autoevaluation_criterions (criterion+)>

<!ELEMENT criterion (name, id, statement?)>

Figure 5: DTD extract of an exercise.

Creating an exercise in XML

Since the emergence of the XML standard, numerous tools appeared for editing XML documents. These editorsverify that documents are well formed and valid compared to the DTD which defines their class. For an author,to create an autoévaluation exercise there is only a need to edit its structure and its contents by means of aneditor XML

- 9 -9

Figure 6: Creating an exercise in an XML editor

Using XSLT for getting web pages

Expressing transformation with XSLT

To unify the mode of description of a document and its transformation in HTML, we chose to express this one inXSLT language. All these languages are standards recommended by the W3C. Furthermore, the environment ofmodification of the model (meta environment) and the environment of creation of an exercise can be the same,this demonstrates the power of the current standardization. Any editing tool, on all the software or materialplatforms, can be used to create and modify geneval.dtd, exo.xml and geneval.xsl files. Furthermore, the sameediting tool allows to edit the DTD, the files XML and the files XSLT.

<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform"><xsl:template match="head_gii"><html><head> <meta name="GENERATOR" content="Generated by Geneval 3 - UJF Grenoble - 2000"/>

<link rel="stylesheet" type="text/css" href="_cover.css" title="cover"/></head><body> <h1> <xsl:apply-templates select="child::name"/></h1> <table> <tr> <td class="info"><xsl:value-of select="$WORD_AIM" disable-output-escaping="yes"/></td> <td><xsl:apply-templates select="child::aim"/></td> </tr> <tr> <td class="info"><xsl:value-of select="$WORD_AUTHOR" disable-output-escaping="yes"/></td> <td><xsl:apply-templates select="child::author"/></td> </tr> </table>

Figure 7 : Part of a XSLT script to transform XML file into HTML file

A XSLT processor transforms a XML file, according to the XSLT scripts, for producing the final html code,which will be used through the browsers for the end users. A lot of this kind of XSLT processors are available byfreeware and open source.

- 10 -10

Transforming Process to produce an exercise

Enseignant

Moteur de transformation

Editeur XML

Geneval.dtd Exo.xml

geneval.xsl

Exercice d’autoévaluation

Publicateurti

8

Etudiant

navigateur

geneval.css

Figure 8 : Transforming Process to produce an exercise

This version of genEval is thus made up by a DTD, by style sheets CSS and of scripts XSLT. It requires notspecific tools except a XML editor and a XSL processor. The production process of an exercise is described bythe following plan

Modelling and producing learning diaporama

The GenDiapo Project

A team of computer specialists who want to produce multimedia documents for their students noticed that somepart of their documents were duplicated in lecture notes and slides which are difficultly maintainable and mustbe continuously synchronized and adapted to different audiences.

The idea then came up to structure in the same document all the information of the course they deliver,indicating which audience addresses certain parts of the document. In addition, they have to differentiate whichpart belong to the summary for the slides and which part belong to the development for the lecture notes.

So, a model of structured doc ument was defined with a scenario of the activity offered to the student on the slideshow by adding contents, index, glossary …

This learning object was named hyperdiaporama. The initial text will be marked by levels of abstraction of theinformation. A lecture note or a set of slides will be produced by the same process than previously, according tothe level chosen by the author.

The hyperdiaporama Learning object

Teaching with a hyperdiaporama

The objective to model an hyperdiaporama object is more oriented to the simplification of the teacher’s workthan to the originality of the activity of learning it. Indeed, the aim is to define a model of a document able tocontain information in a exhaustive way, but which will be delivered in a selective way. So, the sameinformation will be reused or reorganized automatically according to demand, this will introduce more flexibilityinto the production of educational documents of expositive type.

- 11 -11

Prototype of the student interface of a hyperdiaporama

Figure 9: Un écran d’un hyperdiaporama.

These web pages allow the teacher to present his course and the learner to navigate sequentially or according tothe table, to consult the glossary…

Components of a hyperdiaporama

A hyperdiaporama results of the transformation of following components, which structure the course:

� Elements : they are distributed into two groups:• Hierarchical elements :

♦ Section• Content elements:

♦ Non-terminal elements :� Definition, Demonstration, Remark� Note� Example

♦ terminal elements:� Paragraph, Image,Code, List

� Category define the audience of each part of the course� Detail level: can take the value developed or abstract

The publication for a given audience takes into account the category of elements while the choice of a level ofdetail for the publication will determine the selection of elements to publish on a paper or on web pages. So, inthe same document, the teacher can indicate, for every information, which specific audience he addresses and ifhe wants to produce a slide show on web pages or a complete course on pdf file.

With these functionalities one avoids copying and pasting when producing documents of text or of slide showtype, a process we make permanently and which is very difficult to synchronise with the evolutions of thecontents.

Using open standards for expressing a hyperdiaporama

As for GenEval, the structure of a class of documents was described by a DTD, a document produced by theteacher according to the model of this DTD appears in the form of a XML file, the parameters of publication arealso defined in a file XML. To produce the final code HTML or PDF, which will be useful in the currentnavigators or printable, an XSLT processor transforms the document.xml file with the specific scripts written inXSLT. The transformation produces automatically, according to the predefined style of presentation, thehyperdiaporama and the hyperlinks allowing to navigate, as well as the hyperlinks towards the glossary.

- 12 -12

GenDoc: a generic authoring tool for producing Learning Objects

To facilitate the chain of all the stages of the transformation, from the edition phase to the publication phase, andto avoid the author to work at the level of XML tags, the authoring tool GenDoc was developed. This tool wasdeveloped from the Java open source XML editor MERLOT of Channel-Point. Next to it the vision of the tree ofXML elements was added a styled view of the whole document.

Figure 10: Screenshot of the authoring tool GenDoc.

Adding other Learning object model in GenDoc

The modularity of the GenDoc architecture allows its adaptation to a new model of learning object defined by aDTD and their transformation scripts. Practically, a system of plugins allows to adapt the styled view to the DTDand to associate a publication to the model. For the moment, these adaptations must be made by a programmer,because XSL script have to be embedded into java code of the plugins.

The next step of our approach should be the design of a meta-tool, for adding more easily new learning objectmodels. So the pedagogical point of view of the teacher on a learning activity, translated into a XML modelcould be taken into account by our GenDoc tool, without the technical aspects described in this paper, but onlyby using a friendly user interface.

Bibliographie

[1.] Yolaine BOURDA et Marc HELIER

« Métadonnées et XML : applications aux objets pédagogiques » TICE2000, Troyes, octobre 2000.

[2.] J.P. David, A. Cogne

"Mutualisation de Production de documents hypermédias et mise en œuvre pédagogique"

New technologies of Information and Communication in Engineering Education and in Industry,

NTCIF'98 ROUEN, November 1998.

[3.] J.P. David et A. Dutel

"Modélisation et réalisation d’un générateur d’exercices hypermédias"

Colloque "Hypermédias et apprentissage" Chatenay-Malabry 1996.

- 13 -13

[4.] J.P. David, A. Cogne et A.Dutel

"Hypermedia exercises prototyping and modelising" Conférence internationale CALISCE'96 San Sebastian,juillet 1996.

[5.] J.P. Pernin,V. Guéraud. "MARS : Modèle-Associations-Représentation-Scénario. Un modèle de conception d'applications pédagogiquesinteractives ",

7èmes journées de l’ingénierie de l’interaction Homme-Machine, Cepadues éditions,Toulouse oct 1995.

[6.] J.M. CAGNAT, V. GUÉRAUD, J.P. PEYRIN.

"the Arcade Laboratory : an environment to help teach algorithms." (Bulletin SIGCSE ACM, décembre 1990).

- 14 -14

- 15 -15

Re-using technical manuals for instruction:

document analysis in the IMAT1 project

Robert de Hoog*, Bob Wielinga*, Suzanne Kabel* †,

Anjo Anjewierden*, Frans Verster*, Yvonne Barnard ‡ †,

Paolo DeLuca §, Cyrille Desmoulinsª, Johan Riemersma†

* Intelligent Systems Laboratory University of Amsterdam, The Netherlands

‡ EURISCO, France

§ Tecnopolis CSATA Novus Ortus, Italy

ª LORIA, France

† TNO Human Factors, TNO Physics and Electronics Laboratory, The Netherlands

Address, e-mail: [email protected]

Abstract

This paper describes the result of the IMAT project, in particular the tools that were developed for analysingtechnical manuals. The goal of the project was to build a toolset that can support the re-use of the content oftechnical manuals for instruction. In many domains where equipment has to be maintained and repaired, thebuyer of the equipment has to create the training material, because there are no publishers for textbooks for thoselimited domains. As a consequence, the major source of content is the technical manual delivered by the vendor.Unfortunately these manuals are most of the time not immediately suitable for training and instruction, leading totedious and time consuming reworking of the source material. By providing a set of tools that allow the breakingup of the technical manual into smaller fragments, indexing them, storing them in a multi-media data base, the“up front” work needed for re-use can be automated to a large extent, thus saving time.

Keywords

re-use, manuals, instruction, ontologies, document analysis

Introduction

The IMAT project (de Hoog, Barnard & Wielinga, 1999) had as its main aim the development of a set ofproducts for supporting the effective and efficient re-use of the content of technical manuals for instructionalpurposes. In addition, the creation of a structure to facilitate the storing and exchange of experience gained whileworking with this material was taken up. The approach was to take a technical manual as delivered by the 1 The IMAT project (ESPRIT 29175) was a project partly funded by the European Union. Partners in the project were:Intelligent Systems Laboratory, University of Amsterdam (NL), TNO Human Factors and TNO Physics and ElectronicsLaboratory (NL), Loria (F), Renault (F), AFPA (F), ETRA (E), Tecnopolis CSATA Novus Ortus (I), Royal Dutch Airforce(NL), EURISCO (F). Apart from the authors, many other people contributed to the project from the organisations listedabove. We would like to thank them all for their efforts.

- 16 -16

manufacturer of the equipment, break it up into small fragments (homogeneous content) and store thesefragments, after indexing them in a meaningful way, in an object oriented database. This database withfragments can be used to retrieve content when creating training material with an authoring environment chosenby the user.

The project delivered a collection of inter-operable tools consisting of:

• A database schema for describing fragments

• A Document Analysis Tool that analyses, fragments and indexes technical manuals

• A Database Facilities Tool for managing the database, including versioning of fragments

• An Authoring Environments Interface Tool for retrieving fragments from the database and transferringselected fragments to an authoring environment; also used for additional indexing and annotation offragments

• An Ontology Development Tool for creating ontologies in a structured way

• A set of ontologies for describing fragments, domain specific as well as generic, including an extensiveinstructional ontology

• An Instructional Scenarios Tool supporting the creation of skeletal lessons for maintenance tasks. Theskeletal lessons can be exported to the Authoring Environments Interface Tool and used for guidingdatabase queries whose results can be stored in the skeletal lesson

The tools were used at three different user sites, each representing different approaches to creating trainingmaterial and teaching: maintenance of anti-aircraft weapons, car repair and maintenance of traffic controlequipment.

For a better understanding of the design and workflow of the IMAT tools Figure 1 below can be used.

Figure 1: Workflow and structure of the IMAT toolset

The grey boxes are the tools that were developed in the IMAT project. The yellow boxes are the ontologies,which are at the core of the system, but these are described only briefly for reasons of space. The green boxes

General workflowGeneral workflow

Sourcedocument

Documentanalysis

Domainontologies

Generic /instructionalontologies

JasmineOO Database

Authoringenvironmentinterface

Authoringenvironment

Data basefacilities

Othersource material

Scenariotool


SourcedocumentSourcedocument

Documentanalysis

3Documentanalysistool

Domainontologies


JasmineOO Database

Authoringenvironmentinterfacetool


Data basefacilities


Scenariotool

Org .mem. tool

Indexingtool

2

56

7

4

2

10



DocumentanalysisDocumentanalysis

Domainontologies


JasmineOO Database



Data basefacilities


Scenariotool



DocumentanalysisDocumentanalysistool

Domainontologies


JasmineOO Database



Data basefacilities


Scenariotool

2

5

3




Domainontologies


JasmineOO Database



Data basefacilities


Scenariotool



Documentanalysis


Domainontologies


JasmineOO Database



Data basefacilities


Scenariotool

Org .mem. tool

Indexingtool

2

56

7

2

10




Domainontologies


JasmineOO Database



Data basefacilities


Scenariotool




Domainontologies


JasmineOO Database



Data basefacilities


Scenariotool

Covered in this paper




Domainontologies


JasmineOO Database



Data basefacilities


Scenariotool



Documentanalysis


Domainontologies


JasmineOO Database



Data basefacilities


Scenariotool

Org .mem. tool

Indexingtool

2

56

7

4

2

10




Domainontologies


JasmineOO Database



Data basefacilities


Scenariotool




Domainontologies


JasmineOO Database



Data basefacilities


Scenariotool

2

5

3




Domainontologies


JasmineOO Database



Data basefacilities


Scenariotool



Documentanalysis


Domainontologies


JasmineOO Database



Data basefacilities


Scenariotool

Org .mem. tool

Indexingtool

2

56

7

2

10




Domainontologies


JasmineOO Database



Data basefacilities


Scenariotool




Domainontologies


JasmineOO Database



Data basefacilities


Scenariotool


- 17 -17

represent commercial-of-the shelf software used. In fact much more was used than shown, but this would clutterthe picture.

The basic workflow is that a source document is analysed by the Document analysis tool, which breaks it up insmall parts (fragments) and indexes these fragments. The fragments are exported to the Muli-media data basefrom which the fragments can be retrieved with the Authoring environment interface tool. This tool also providesfunctionalities for additional indexing of fragments and adding “lessons learned” like notations to fragments withthe Indexing tool and the Organisational memory/learning tool. The retrieval can be guided from an instructionalviewpoint by using the Instructional scenarios tool. Finally the retrieved fragments can be copy/pasted into theAuthoring environment chosen by the author of the instructional material.

This paper will be describe the set of tools that support the analysis and fragmenting of the technical manual, orthe “up front” work that is needed for re-use (the dashed box in Figure 1). The accompanying paper (de Hoog etal., 2002b) will show how the results of the document analysis process can be used for creating instructionalmaterial.

The multi-media database

For storing fragments from technical manuals a commercial product was used: the object-oriented Jasmine®system available from Computer Associates. It allows the storage and retrieval of text, images, sound and video.During the IMAT project only the text and images facilities were used.

Underlying the database is a database schema that is derived from the ontologies that were developed. Figure 2shows the simple window with which Jasmine can be switched on and off. For most users this is the only visiblepresence of the database.

Figure 2: Control window for Jasmine

The Jasmine® system has performed well. The only drawback is that there is not yet a SQL comparable standardin the Object Orientation world hat permits the replacement of Jasmine® with another system. As a consequencethe IMAT tools are coupled to Jasmine®. This holds in particular for the Authoring Environments InterfaceTool, the Indexing Tool and the Organisational Memory and Learning Tool. For document analysis this link isweaker as that tool produces as output an ODQL file that can be loaded by any data base system supportingODQL.

Managing the database consists of creating contexts for applications and managing fragments, which works withthe data base facilities tool. An important part of this is the loading of ontologies that are relevant for the context(or domain, e.g., a particular type of car) from which the fragments in the technical manual are extracted. Figure3 shows how ontologies are loaded for a context. The “Domain topics” and “Domain keywords” are domainspecific, all other ontologies are generic and applicable across domains but related to instruction.

- 18 -18

Figure 3: Loading ontologies in the Data Base Facilities Tool

The document analysis tool

The goal of the document analysis tool is to break up a technical manual into smaller text and image fragments,providing them with an index for identification and exporting the fragments to the Jasmine database. The mainwindow of the tool is shown in Figure 4.

Though the document analysis process can be carried out fully automatic, from user evaluations it became clearthat some control over and insight in the behaviour of the tool was desirable. The “Set context” button allows theuser to identify a context in which the material will be stored in the database (this should have been created bythe tool described Figure 3). The “Set document” button makes it possible to select (part of) a document that isgoing to be analysed. The tool requires that the source document is available in PDF. By clicking on the “Textanalysis” button the selected document is automatically fragmented and indexed with topics and keywords. Thisprocess relies heavily on the domain and generic ontologies.

Figure 4: Main window of the Document Analysis Tool

The tool pops up a window with the HTML version of the analysed PDF document (see Figure 5).

- 19 -19

Figure 5: HTML version of PDF document

Through another option the user can also view which fragments the analysis process has identified (see Figure6).

Figure 6: View on the results of the text analysis process

In Figure 6 the coloured lines indicate the fragments that were identified, while the margins show the structuraltype of the fragment (e.g., item or paragraph). Clicking on the “Image analysis” button in Figure 5 performs theanalysis of the images in the source text. This process converts the images to SVG, which are stored in thisformat in the database. The transformation to SVG makes it possible to search the content of an image (like

- 20 -20

lines, boxes, text etc.), something that is not possible with Bitmap images. Figure 7 shows the results ofanalysing a wiring schema.

Figure 7: Result of analysing a picture

The different colours in Figure 7 indicate what has been recognized, shown in the list of elements in the upperright hand corner. This includes lines, rectangles etcetera. Text is recognized by using OCR techniques. Bymeans of colours it is indicated how reliable the OCR is, as this is not 100% certain for documents having a loworiginal quality. As no mistakes are allowed in technical diagrams, the user must have a clue about the quality ofthe OCR.

The Document Analysis tool includes a search facility with which the user can have a look at the createdfragments. Figure 8 shows an example window of this part of the tool.

- 21 -21

Figure 8: Fragments search facility of the Document Analysis Tool

In Figure 8 a search is performed on the topic “Automatic data processor (ADP) microcomputer”. The topmostpart of the window shows that two fragments matching this topic were found. The middle part of the windowdisplays the content of the first fragment. It is important to understand that this text fragment has beenautomatically extracted and indexed from the original technical manual and that these two fragments can belocated in different chapters of the manual.

If the user is satisfied with the results of the document analysis process, the button “Build DB” can be clickedand the fragments (text and images) are automatically loaded in the Jasmine data base (if it is running, see Figure2).

The ontology development support tool

Ontologies play a key role in the process, as they are needed for indexing fragments. Support for ontologydevelopment is concerned with domain specific ontologies, that is, creating and modifying ontologies which areunique for a particular area, e.g., cars, traffic control equipment, anti-aircraft guns etc.. Three forms of supportwere available:

• Automatic creation of an ontology from a structured parts list of the equipment under consideration orfrom the Table of Contents from the technical manual, followed by manual fine-tuning.

• Creating the ontology “by hand” using the development tool.

• Modifying ontologies loaded in the database using the database facilities.

As the first type of support is very dependent on the domain, this has been offered as a tailored service onlyduring the IMAT project for the users. The second and third types are available in the tools delivered by IMAT.

- 22 -22

Creating an ontology “by hand” can be done by using the development tool that can be activated by clicking onthe menu item that appears in the dropdown menu under the heading “Tools” in Figure 4, this results in thewindow shown in Figure 9.

Figure 9: The window of the ontology development support tool included in the document analysis tool.

This tool allows for the creation of two different types of hierarchies, Is-a and Part-of (upper part of window).For each concept in the hierarchy its location (“Super” or “Part of”) and other attributes can be defined (left handside of lower part of the window). These are automatically converted into CML (= Conceptual ModellingLanguage, right hand side of lower part of window). These CML defined hierarchies are than available to beused in the database schema and the Authoring Environments Interface Tool. Of course, this support tool canalso be used to modify ontologies.

The other way to modify ontologies is by using the database facilities outlined in Figure 3. When activating thecorrect function, the window in Figure 10 is available.

- 23 -23

Figure 10: Modifying ontologies using the Dat base Facilities tool

In Figure 10 it is shown how user remarks can be added to elements/concepts in the ontologies.

Taken together, the combination of both ways of working, constitute the ontology development tool supportprovided by IMAT. From the experience gained in the project, it is clear that creating a domain ontology fromscratch is a major task, which requires expertise and time. The preferred solution is to create the domainontology to a large extent automatically and let it be checked and tuned by an expert. Within the framework ofthe IMAT project time and effort was lacking to find a more or less generic way of doing this for a wide range ofdomains. Nonetheless, when a parts list or a well-structured Table of Content of the manual is available, someautomation can be achieved, albeit in a customised way.

The instructional ontology

In order to focus the work on instruction it was necessary to development an ontology that enables the analysisand indexing of fragments from an instructional perspective. At the start of the project several initiatives in thisarea were just emerging. During the project the development of the instructional ontology has been done with aneye on the ADL/SCORM, LOM and Ariadne projects. The IMAT instructional ontology is to a large extentcomparable with the vocabularies from those quarters, but allows for a more detailed instructional indexing offragments. The “instructional ontology” is a set of related ontologies. For reasons of space we deal only brieflywith the ontologies.

Fragment ontology

This ontology represents the syntactical view on a fragment, “what it is” in terms of the source document. Forexample, a fragment “is a picture” or a fragment “ is in BMP format”.

- 24 -24

Description ontology

Fragment description type is an attribute of a fragment that reflects the perspective from which a fragment waswritten, and it contains a description of the content of a fragment. For example, a fragment was written from astructural perspective, describing a system’s physical features, and the content concerns a definition of acomponent.

Instructional roles ontology

This ontology describes the possible instructional roles a fragment can play in the instructional material that hasto be created. For example can be a “picture” (from the fragment ontology), but in the new instructional contextit can play the role of an “example” or it can be part of an “exercise”. This notion of “roles” represents thepragmatic aspects of content. It enables re-use of the original content of the manual for instructional purposes.This ontology consists of several components:

• A learning goal describes the desired outcome of an instructional curriculum in terms of knowledge,skills and attitudes that are necessary to carry out certain functions or tasks (e.g., perceptual knowledgenormal/deviant behavior).

• Knowledge is information acquired and stored, in an organised manner, in the mind. Knowledge typestands for the knowledge that is necessary for a learner to appropriate in order to gain the learning goal.

• Knowledge is communicated to the learner by means of instructional strategies. Instructional strategiesrefer to the effective way of teaching or to characteristics of an effective learning environment (e.g.,procedural knowledge).

• Instructional strategies are refined into instructional activities. Instructional activities stand for theactivities performed by instructor and learner (e.g., coaching).

• Instructional activities are refined into instructor actions and learner actions. Instructor actions standfor everything the instructor does within a certain instructional activity, while learner actions stand foreverything the learner does within a certain instructional activity (e.g., provide explanation). The nounsin instructor and learner actions, represent the way fragments are going to be used in instructionalmaterial. For example, the learner action “study illustration” indicates that the fragment that is going tobe put in the training material, will be used as an illustration.

Domain ontologies

The domain ontologies differ from the ones described above because they are closely tied to a unique piece ofequipment. This holds in particular for anti-aircraft and traffic control equipment. For cars the link with specificequipment is less strong, as there are many different instances (brands, types) of the general class of cars. Againthese ontologies are only described briefly.

Anti-aircraft guns

Two systems were included: the Flycatcher anti-aircraft gun and the Hawk anti-aircraft missile. The ontologiesfor these domains were developed in the IMAT project, using Tables of Content and parts lists. The first stepwas carried out by a computer program. The output was refined with the support of domain experts.

Cars

The ontology for cars is a large ontology with hundreds of concepts. An interesting aspect is that it not onlycontains “pure” car concepts, but also concepts related to the maintenance task like “repairing tool”.

Traffic control equipment

The traffic controller built by ETRA is a piece of equipment that controls traffic lights. It consists of a physicalsystem (the equipment) and software with which the behavior of the controller can be programmed. As aconsequence, the ontology had to reflect this distinction.

System ontology

- 25 -25

An ontology of system types and an ontology of system locations have been developed to describe systems in ageneral, domain independent, fashion (e.g., subunit or container). These ontologies are not directly useful orvisible to the author of training material, but they are of use in document analysis. Another argument to developthem was to standardize terms used across domains.

When comparing instructional ontologies and domain ontologies, it becomes clear that the distinction betweenthe two is relative. For example, the description ontology also deals with functional and structural aspects, asdoes the system ontology. Likewise, the domain ontologies sometimes contain concepts with an instructionalflavor such as “repairing tool” and “programming”.

The domain ontologies have played a very important part in the IMAT project. They were the cornerstones forautomatically indexing fragments as performed by the Documents analysis tool, which in turn was the mainprerequisite for making retrieval possible. As instructional indexing cannot (yet) be done in an automatic way,domain concepts really hold the key to the operational effectiveness of the IMAT toolset.

Conclusion

The tools were used in three contexts: maintenance/repair of cars, traffic control equipment and militaryequipment (rockets and guns). In all contexts the source material was quite bulky and previously most of thecreation of training material was done by literally copying, cutting and pasting from the paperbased source.Substantial parts of this source material was processed by the document analysis tool and stored in the multi-media database. Generally speaking the document analysis tools did a good job. Transforming paper intoelectronic form proved to be a benefit in itself. Substantial time was saved in handling electronic material insteadof cumbersome piles of paper.

Nonetheless some technical problems are quite formidable. The almost endless variety in document types makesit very difficult to build analysis tools that can be applied to any document that comes along. In the foreseeablefuture there were always be a necessity for some tailoring of the analysis tool, depending on the manual thatmust be fragmented. In addition, automatic indexing still has its limits. Automatic indexing with instructionalmark up is still in the future and has to be done “by hand”. Without a well-structured parts lists, or table ofcontent the building of a domain ontology can be a substantial amount of work, even if editing tools areavailable. This is probably less of a problem for technical domains, but it could be an obstacle when one wants totransfer the approach to other, less well-defined, areas.

References

ADL, http://www.adlnet.org

ARIADNE, http://ariadne.unil.ch

Hoog, R. de, Y. Barnard & B. Wielinga (1999). IMAT: re-using Multi-media Electronic TechnicalDocumentation for Training. In: J.Y. Roger, B. Stanford-Smith & P.T. Kidd (Eds), Business and Work inthe Information Society: New Technologies and Applications. IOS Press, p. 415 – 421.

Hoog, R. de, S. Kabel, Y. Barnard, G. Boy, P. DeLuca, C. Desmoulins, J. Riemersma & D. Verstegen (2002b).Re-using technical manuals for instruction: creating instructional material with the tools of the IMATproject. This volume.

IMS, http://www.imsproject.org/

- 26 -26

- 27 -27

Re-using technical manuals for instruction: creating instructionalmaterial with the tools of the IMAT1 project

Robert de Hoog*, Suzanne Kabel* †, Yvonne Barnard ‡ †, Guy Boy‡,

Paolo DeLuca §, Cyrille Desmoulinsª, Johan Riemersma †, Danielle Verstegen †

* Intelligent Systems Laboratory University of Amsterdam, The Netherlands

‡ EURISCO, France

§ Tecnopolis CSATA Novus Ortus, Italy

ª CLIPS IMAG & Université Joseph Fourier Grenoble 1, France

† TNO Human Factors, TNO Physics and Electronics Laboratory, The Netherlands

Address, e-mail: [email protected]

Abstract

This paper describes the result of the IMAT project, in particular the tools that were developed to support thecreation of instructional material re-using content from a technical manual. The goal of the project was to build atoolset that can support the re-use of the content of technical manuals for instruction. In many domains whereequipment has to be maintained and repaired, the buyer of the equipment has to create the training material,because there are no publishers for text books for those limited domains. As a consequence, the major source ofcontent is the technical manual delivered by the vendor. Unfortunately these manuals are most of the time notimmediately suitable for training and instruction, leading to tedious and time consuming reworking of the sourcematerial. After storing the fragmented content of a manual in a multi-media database, the author of the trainingmaterial has to retrieve it and insert it in an authoring environment. This process is supported by tools enablingretrieval, creating instructional scenarios, and classifying and storing experience with the material during trainingas well as regular maintenance.

Keywords

re-use, manuals, information retrieval, ontologies, instructional scenarios

Introduction

The IMAT project (de Hoog, Barnard & Wielinga, 1999) had as its main aim the development of a set ofproducts for supporting the effective and efficient re-use of the content of technical manuals for instructionalpurposes. In addition, the creation of a structure to facilitate the storing and exchange of experience gained whileworking with this material was taken up. The approach was to take a technical manual as delivered by themanufacturer of the equipment, break it up into small fragments homogeneous content and store these fragments,

1 The IMAT project (ESPRIT 29175) was a project partly funded by the European Union. Partners in the project were:Intelligent Systems Laboratory, University of Amsterdam (NL), TNO Human Factors and TNO Physics and ElectronicsLaboratory (NL), Loria (F), Renault (F), AFPA (F), ETRA (E), Tecnopolis CSATA Novus Ortus (I), Royal Dutch Airforce(NL), EURISCO (F). Apart from the authors, many other people contributed to the project from the organisations listedabove. We would like to thank them all for their efforts.

- 28 -28

after indexing them in a meaningful way, in an object oriented database. This database with fragments can beused to retrieve content when creating training material with an authoring environment chosen by the user.

The project delivered a collection of inter-operable tools consisting of:

• A database schema for describing fragments

• A Document Analysis Tool that analyses, fragments and indexes technical manuals

• A Database Facilities Tool for managing the database, including versioning of fragments

• An Authoring Environments Interface Tool for retrieving fragments from the database and transferringselected fragments to an authoring environment; also used for additional indexing and annotation offragments

• An Ontology Development Tool for creating ontologies in a structured way

• A set of ontologies for describing fragments, domain specific as well as generic, including an extensiveinstructional ontology

• An Instructional Scenarios Tool supporting the creation of skeletal lessons for maintenance tasks. Theskeletal lessons can be exported to the Authoring Environments Interface Tool and used for guidingdatabase queries whose results can be stored in the skeletal lesson

The tools were used at three different user sites, each representing different approaches to creating trainingmaterial and teaching: maintenance of anti-aircraft weapons, car repair and maintenance of traffic controlequipment.

For a better understanding of the design and workflow of the IMAT tools figure 1 below can be used.

Figure 1: Workflow and structure of the IMAT toolset

The grey boxes are the tools that were developed in the IMAT project. The yellow boxes are the ontologies,which are at the core of the system. The green boxes represent commercial-of-the shelf software used. In factmuch more was used than shown, but this would clutter the picture.

The basic workflow is that a source document is analysed by the Document analysis tool, which breaks it up insmall parts (fragments) and indexes these fragments. The fragments are exported to the Muli-media data basefrom which the fragments can be retrieved with the Authoring environment interface tool. This tool also providesfunctionalities for additional indexing of fragments and adding “lessons learned” like notations to fragments withthe Indexing tool and the Organisational memory/learning tool. The retrieval can be guided from an instructionalviewpoint by using the Instructional scenarios tool. Finally the retrieved fragments can be copy/pasted into the


Sourcedocument

Documentanalysis

Domainontologies


JasmineOO Database



Data basefacilities


Scenariotool



Documentanalysis


Domainontologies


JasmineOO Database



Data basefacilities


Scenariotool

Org .mem. tool

Indexingtool

2

56

7

4

2

10




Domainontologies


JasmineOO Database



Data basefacilities


Scenariotool




Domainontologies


JasmineOO Database



Data basefacilities


Scenariotool

2

5

3





Domainontologies


JasmineOO Database



Data basefacilities


Scenariotool



Documentanalysis


Domainontologies


JasmineOO Database



Data basefacilities


Scenariotool

Org .mem. tool

Indexingtool

2

56

7

2

10




Domainontologies


JasmineOO Database



Data basefacilities


Scenariotool




Domainontologies


JasmineOO Database



Data basefacilities


Scenariotool





Domainontologies


JasmineOO Database



Data basefacilities


Scenariotool



Documentanalysis


Domainontologies


JasmineOO Database



Data basefacilities


Scenariotool

Org .mem. tool

Indexingtool

2

56

7

4

2

10




Domainontologies


JasmineOO Database



Data basefacilities


Scenariotool




Domainontologies


JasmineOO Database



Data basefacilities


Scenariotool

2

5

3





Domainontologies


JasmineOO Database



Data basefacilities


Scenariotool



Documentanalysis


Domainontologies


JasmineOO Database



Data basefacilities


Scenariotool

Org .mem. tool

Indexingtool

2

56

7

2

10




Domainontologies


JasmineOO Database



Data basefacilities


Scenariotool




Domainontologies


JasmineOO Database



Data basefacilities


Scenariotool


2

56

7

2

10




Domainontologies


JasmineOO Database



Data basefacilities


Scenariotool




Domainontologies


JasmineOO Database



Data basefacilities


Scenariotool


- 29 -29

Authoring environment chosen by the author of the instructional material. In this paper we will describe the toolsat the “down stream” side of the process (shown by the dashed box in Figure 1), which support the re-use of thestored material from the manual. The accompanying paper (de Hoog et al., 2002a), provides details about the “upfront” part: analysing and fragmenting the technical manual and storing the content in a database.

The authoring environments interface tool

In order to be able to retrieve fragments from the technical manual that were extracted by using the DocumentAnalysis Tool, the Authoring Environments Interface tool was developed. The tool is used as a mechanism fortransferring selected fragments from the database to the “authoring environment” (e.g. MS Word®) a personcreating instructional material is using. This can be any application that supports the copy/paste facility in MSWindows®. The start window of the tool is shown in Figure 2.

Figure 2: The main window of the Authoring Environments Interface Tool

The “tabs” in the window in Figure 2 make it possible to search for the four main types of fragment (text,pictures, video, sound), but simultaneous retrieval can be done activating the set boxes to the left under “Search”.The main window is used for displaying retrieved material on the empty canvas. By clicking on “Set criteria” theuser can specify what is searched for. This is done with the window in Figure 3.

- 30 -30

Figure 3: Window for defining search criteria

The slots in the window in Figure 3 represent the search terms that can be used: topics, keywords, descriptiontype, knowledge type and use. The “+” and “-“ push buttons open lists of predefined terms, which are derivedfrom the ontologies, that can be selected by the user. After filling all or only a few of the slots, the user clicks onthe “Retrieve” button, after which the search is started. In the “Search log” section, a count is given of thenumber of fragments matching the query. After the search is over a “Display” button appears, which after beingpushed, brings the user back in the main window. This shows a list of retrieved fragments in the upper part, eachof which can be displayed in the canvas by activating the associated set box. Figure 4 shows a window in whicha piece of text from the traffic controller manual is displayed. Figure 5 displays a picture that has been retrieved.

- 31 -31

Figure 4: Display of retrieved text fragment

Figure 5: Display of retrieved picture

- 32 -32

For pictures the tool offers possibilities to scale it, to show thumbnails when many pictures are retrieved, as wellas selecting part of a picture and re-use it in the authoring environment. When the picture is an analysed diagrampresent as SVG in the database, search can also be performed on the content of the picture. Thus searching for aparticular component with “text” and “picture” selected, will return all fragments (text and pictures) containingthis component, even if the bitmap version of the picture was not explicitly indexed as containing thiscomponent.Transfer to the authoring environment can be done by using the Clipboard (copy-paste) facility or bystoring the fragment in a file and retrieving it later from the file in the authoring environment.

This is the basic functionality of the tool. However, it contains additional facilities, which will be describedbelow.

The indexing support tool

Originally it was planned to develop a separate tool for indexing fragments. However, when the projectprogressed it became clear that indexing would take place either in an automatic way or by hand. Following thelogic of the workflow, it became clear that indexing by hand should be tied to existing fragments. As fragmentsare stored in the database and are already indexed on structure and domain topic, the best strategy was tointegrate the indexing tool into facilities for accessing and managing the database. As a consequence there is nota separate “indexing tool” available in the IMAT toolset. The indexing facility is integrated in the AuthoringEnvironments Interface tool and the Data Base Facilities.

In the Authoring Environments Interface Tool a retrieved fragment can be given additional indexes (“mark up”)using the instructional ontology. This is shown in Figure 6.

Figure 6: Indexing a fragment in the Authoring Environments Interface

In Figure 6 a retrieved picture (not shown) is going to be indexed with a particular knowledge type: perceptualknowledge of structure with visual identification of parts. With this facility any fragment that has been createdwith the Document Analysis Tool, or has been entered into the database in another way, can be indexed.Indexing can also be done with the Data Base Facilities tool (see de Hoog et al., 2002a).

- 33 -33

Both processes are very similar. However, in the Data Base Facilities Tool the person doing the indexing hasmore discretion than in the Authoring Environments Interface Tool, because also Keyword and Topic indexingcan be changed, which should be protected because they are key indexes which cannot be allowed to be changedby any arbitrary user.

Organisational memory/learning tool

The organisation memory/learning tool serves the need to store in a systematic way experiences gained withtechnical documentation and instruction. As these experiences are bound to content, it was a logical decision tocouple these experiences to content in the shape of fragments. Broadly speaking two types of experiences can bedistinguished:

• experience in using a fragment, during work as well as during instruction

• detecting mistakes in fragments that should be corrected.

Each type should be handled in a different way. As most experience in using a fragment is welcome, it should beaccessible to anyone using the fragment. Consequently the facility to annotate fragment from a use perspectivewas incorporated in the Authoring Environments Interface Tool, because this is the location where retrieval takesplace. Figure 7 shows the window that can be activated in the Authoring Environment Interface Tool forcollecting and storing use experiences.

Figure 7: Annotation of a fragment in the Authoring Environments Interface Tool

In Figure 7 there are four categories of experiences that can be entered: usability, usefulness, evaluation andadditional information. Any user can enter comments/experiences with this facility and, when a fragment isretrieved by another user, these annotations can be displayed on request (see the “notes” indication in thewindow in Figure 4). In this way an efficient information exchange mechanism concerning fragments betweenusers of the documentation has been put into place.

Though the annotation facility should be accessible for everyone using fragments, this does not hold for updatingfragments. All manuals contain mistakes that are discovered during use, but the cycle time between the detection

- 34 -34

of a mistake and an official update from the manufacturer is often very long. In the mean time it is necessary tostore the mistake without changing the “official” content of the technical documentation. The IMAT Data BaseFacilities Tools allows for this, but only for persons authorized to do it. By using the Data Base Facilities toolversions of fragments can be created. A fragment containing a mistake can be edited and the modified fragmentstored as a draft.

In Figure 8 a text fragment has been retrieved (left hand pane in window). Assume that during using thetechnical documentation it was discovered that “lamp D3” is wrong and should be “lamp D4”. The authorizedperson can now change this in the left hand pane and store the corrected fragment as a “Draft Revision” (checkboxes, right below). When retrieving the original fragment using the Authoring Environments Interface Tool,this will show the existence of two versions of the fragment: the original one and the draft revision. When themanufacturer sends out an authorized correction the original fragment can be discarded and the draft revisionelevated to “original” status.

Figure 8: Creating draft revisions of fragments when mistakes are detected

Together these facilities provide the users with a versatile and powerful environment for codifying andexchanging experiences. This is particularly true when the database runs in a real server-client context whichmakes asynchronous and geographically dispersed sharing of knowledge about the usefulness and correctness ofcontent of technical manuals, for instructional as well as operational use, possible. Nonetheless a cautionary notehas to be struck. One of the original objectives was to achieve automatic updating of fragments used ininstructional material created with an arbitrary authoring environment. This turned out to be too difficult as it isimpossible to keep track of where fragments end up after they were retrieved from the database. This is the pricethat had to be paid for flexibility at the authoring end of the workflow. Had the project settled for a very tightintegration with one particular authoring environment (say Toolbook® or Authorware®), this could have beenrealized. However, the drawback, forcing IMAT users to shift from the current authoring environment in use to anew, unfamiliar, one, was seen as too serious.

- 35 -35

Instructional scenarios tool

During the project the user partners stipulated that in re-using material they felt a need for guidance at apedagogical level, in particular the structuring of (a set of) lessons according to sound instructional knowledgeabout teaching maintenance skills. Following this requirement a tool was developed that allows a user to developa skeletal structure for lessons in a certain domain. Based on theoretical work, part of the construction of askeletal lesson is done automatically as a choice at one step in the process, constrains the choice later on. Figure9 shows the interface of the Instructional scenarios tool.

Figure 9: The initial steps in the Instructional scenarios tool

In Figure 9 the top-level bars from left “Learning goal” to right “Material use”, are the sequential steps inconstructing a skeletal lesson following the instructional ontology (see de Hoog et al., 2002a). First a learninggoal is selected and the tool automatically selects the fitting fragment description (“Definition; Location ofcomponent”). Next the user selects a knowledge type (“declarative knowledge, knowledge of structure,composition”) and an instructional strategy (not shown in Figure 9). All these selections are either madeautomatically or represent a selection from a wider range of options which are constrained by previous choices.After choosing a particular instructional strategy the remaining columns are derived automatically (see Figure10).

- 36 -36

Figure 10: Complete scenario

The content of the column “Instructional activities” contains the skeletal lesson consisting of a sequence ofthings that have to be done. The column “I/L actions” specifies which kind of activities are expected frominstructors and learners during a phase of the skeletal lesson. The final column “material use” indicates what rolethe material used plays in the related phase. For example, during the “Presentation” activity the material usedplays mainly a definitional role. After a skeletal lesson has been identified it can be saved to a file for laterreference.

Apart from this “stand alone” use, the Instructional scenario tool can also be used in conjunction with thedatabase. The structure constructed in Figure 10 can be transferred to a data base context, which contains thematerial that could be used to retrieve relevant material. If this option is chosen, the Authoring EnvironmentsInterface Tool shows a “bag” with the name that was given to the skeletal lesson (see Figure 2). This “bag” canbe opened and then it shows the structure of the skeletal lesson developed using the scenario tool. This isdepicted in Figure 11.

- 37 -37

Figure 11: Authoring Environments Interface Tool with an open bag

In Figure 11 the “instructional material bag” has been opened showing the different actions and activities definedin the skeletal lesson. The user can now employ the characterizations from the skeletal lesson, e.g., knowledgetype and material use, together with a topic or keyword to guide the retrieval, that is setting search criteria just asin Figure 3. The retrieved and selected fragments can now be stored in the relevant sections of the bag, instead ofbeing copy-pasted to the authoring environment. When finished, the bag can be exported to a file structure thatreflects the structure of the skeletal lesson. This allows the author of instructional material to access the materialat a later date, without the need to start up the database. It should be stressed that this facility is very important asthe working of the Authoring Environments Interface Tool assumes that when a new query is fired the results ofthe previous query are erased. This forces the user to decide on what to do with fragments (copy-paste them tothe authoring environment or not) before the next query is started. The bag mechanism provides a “storage”facility with which this decision can be postponed if needed. The Instructional Scenarios Tool (“wizard”) canalso be activated directly from the Authoring Environments Interface Tool.

This turns out to be a powerful way of guiding people, who are not pedagogical experts, in building goodlessons. The link with the database and the Authoring Environments Interface Tool guarantees an easy andnatural connection between structure and content, by “pre-fabricating” the kind of meaningful queries that can beentered.

Using the IMAT tools

Though this paper is mainly devoted to a description of a part of the tools, we will briefly say something aboutthe application of the tools in practice.

- 38 -38

For creating the instructional material several different authoring environments were used: MS Powerpoint®,MS Word®, Toolbook®, and an HTML editor. The size of the training material ranged from 50 – 100Powerpoint® sheets to a complete Toolbook® application. From the latter a page is shown in Figure 12.

Figure 12: Part of the Toolbook® instruction built with the IMAT toolset

Figure 12 is part of an application that is used at AFPA for instructing learners in the area of car body repair.

When measured against expected benefits it was shown that the use of the toolset led to:

• A faster production of training material

• Less expenses when updating training material

• Improved consistency between training material and technical documentation

• New ways of training and teaching, in particular the migration from paper based to computer basedtraining

Moreover the tools were extensively used in the daily practice of the people who created training material andgive maintenance courses. When asked, they confirm that the tools have added value in their work.

References

R. de Hoog, Y. Barnard & B. Wielinga (1999). IMAT: re-using Multi-media Electronic TechnicalDocumentation for Training. In: J.Y. Roger, B. Stanford-Smith & P.T. Kidd (Eds), Business and Work inthe Information Society: New Technologies and Applications. IOS Press, p. 415 – 421.

Hoog, R. de, B. Wielinga, S. Kabel, A. Anjewierden, F. Verster, Y. Barnard, P. DeLuca, C. Desmoulins & J.Riemersma (2002a). Re-using technical manuals for instruction: document analysis in the IMAT project.

- 39 -39

A scenario tool for coupling human and computerised

expertise in the development of training material

for complex equipment maintenance

Yvonne Barnard*, Cyrille Desmoulins** & Monique Grandbastien***

* EURISCO International, France [email protected]

** CLIPS IMAG & Université Joseph Fourier Grenoble 1, France, [email protected]

*** INA P-G and LORIA Université Henri Poincaré Nancy1, France, [email protected]

Abstract

This paper describes a scenario-based approach to assist authors to produce training material from originaltechnical documents in the domain of complex systems maintenance. This approach offers the authors technicaland teaching expertise through the use of ontological indexes, in order to build lesson scenarios and to deriveuseful fragments from technical manuals to be inserted into training material. The scenario tool we developed isdescribed and its use and rationale are presented, as well as its links to others tools developed in the frameworkof a European project on integrating manuals and training.

Keywords

authoring systems, cognitive approaches, human factors, instructional design, instructional meta-data,ontologies, vocational training

Introduction

In ITS and computer based training research it became clear that developing computer programs that can teacheffectively requires large amounts of effort from experts from different disciplines. Instructional design methods(e.g. Gagné, Briggs, & Wager, 1992) and tools (e.g. Designers Edge from Allen Communication) can supportthe developer, but still a lot of human expertise must be put into the design. Our research focuses on the supportof training material developers in the field of complex equipment maintenance. Training is a key factor for thecompetitiveness of industry in general and the high-tech industries in particular. Companies spend significantamounts of resources to provide vocational training tailored to the products and services they produce.

This paper provides an in depth analysis of the process for developing technical training documents. Then itshows how each step of the development process can be automated or partly assisted. Such an assistance hasbeen implemented into a set of tools in the IMAT project (de Hoog, Barnard & Wielinga, 1999). Among thesetools, we describe the so-called “scenario tool” which provides a computer-assisted framework to create trainingmaterial from technical documents produced by the manufacturers. Moreover the same environment allows thetraining document designer to add his own teaching expertise to the selected technical documents, the data baseof technical documents is enriched by annotations represented as additional indexes. We conclude on thebenefits of this set of tools for designing training documents adapted to the changing needs of the professionalsand on the perspectives of a corporate training memory that can be initiated through the use of such tools incompanies.

- 40 -40

Training for maintenance work

Developing educational programs asks for different kinds of expertise such as domain knowledge, pedagogicexpertise, and knowledge about the students to be taught. In general education, in domains such as mathematicsor history, the different forms of expertise are usually available. However, in specialist forms of training this ismuch more of a problem. Our research described in this paper is concerned with training for maintenancetechnicians. Characteristics and problems of training for maintenance technicians are:

• Keeping training up-to-date: technical systems and its components change very rapidly.

• Dealing with variety: many modern products are on the market with a variety of configurations, there isusually not just one single system available, but systems are more and more customised, asking fordifferent maintenance procedures.

• Training for small groups: courses for specialists usually have a very limited amount of trainees.

• Training has to be very specific and is often safety critical: trainees need to learn exactly what to do andhow to do it, making mistakes in maintenance tasks can be dangerous or costly.

• A constant need for training and re-training: nowadays technicians are not only to be trained when theyare young, then progress towards expertise, and stay on their job for the rest of their working life.People change jobs more readily, so the company has to deal with a continuous stream of untrainednewcomers.

• No training material on the market: training of maintenance skills is generally not part of the publiccurriculum and therefore not an attractive market for educational publishers, forcing companies toproduce their own training material.

• More theoretical training has to be combined with practical work: this means that the training site ispreferably close to the workshop. Furthermore, maintenance is usually done at local sites, so it would bebest at least part of the training could be done locally.

These characteristics lead to the situation where higher demands are put on the quality of training while at thesame time less time is available to produce training. Teachers in maintenance are usually experiencedtechnicians themselves, sometimes with additional didactic schooling. Sometimes they work only part-time as aninstructor, spending the rest of their time doing maintenance work, or coaching inexperienced technicians at theworkshop. These people are scarce. Good, self-explanatory learning material, which can be easily updated, istherefore very welcome in technical domains. Computer-based training, centrally developed and locallydelivered seems to offer an excellent solution to several of the problems. However, who has to develop thelearning material? Again it comes back to the problem of the availability of experienced technicians, who haveto provide the domain and didactic expertise. Developing training material in technical domains furthermore asksfor a high level of multi-media use, especially complex graphics, schemes and photos are needed to show thetrainees where components are located, how to remove, repair and install them, what the functions are ofsubsystems etc.

Development of training material

Training material for technical training can consist of different kinds of documents, in paper- or in electronicformat, such as hand-outs for trainees, syllabi, transparencies, electronic slide presentations (e.g. PowerPoint),courseware and trainer documents. Training documents contain texts, pictures, schemes and diagrams. Alsoanimations or videos of processes can be used. Training documents are usually produced in an electronic form,but can be converted to paper-based end-products which are given to the trainees. The content of technicaltraining documents is usually well structured, focused on the technical system and its components, which thetrainee has to learn to maintain. The main part of training documents consists of (Barnard, Riemersma, & deHoog, 1998):

• structural descriptions of the components and their connections

• functional descriptions of components

• procedures for maintenance and repair

• safety measures

- 41 -41

• faults and diagnostic procedures

The work described here comes partly from the European IMAT-project (Integrating Training And Manuals)1 inwhich tools and methods for the creation of training material from fragments of the technical manual aredeveloped (de Hoog, Barnard & Wielinga, 1999). The user organisations involved in the project faced all of theproblems of maintenance training described before. Training material had to be developed for maintenancetechnicians in the domains of weapon systems, car repair and traffic systems maintenance. These are verydifferent domains, but with most of the problems mentioned above in common. The partners involved haddeveloped paper-based material and PowerPoint slides, often at high costs in terms of effort, or had not evenbegun to develop training material, which meant that they had to provide a large amount of support to beginningtechnicians during their job.

The project aimed to provide support for developing training material faster, cheaper and better. To reach thesegoals, the following principles were applied:

• (Re-)use already existing material and domain expertise;

• Automate as many parts of the development process as possible;

• Facilitate centralised development, allowing for easy local customisation;

• Make collaborative development easy, and minimising the effort of domain experts.

These principles could be used because of the specific nature of technical, maintenance training. In the firstplace, although training material is hard to find, technical material is always available. Complex systems havehuge amounts of technical manuals, both for operations and for maintenance purposes. These manuals aredifficult to use, and cannot simply be given to trainees to serve as learning material. However, a large quantity ofdomain expertise is stored in them, and if one could get it out in a form that is suited for training, muchdevelopment time could be gained.

Technical training is not the same as general school education. It is usually very structured; many of the thingsthat have to be learned follow the same pattern. For example, for each component of a system you have to knowthe location, its function, its connection with other components, how to remove and replace it. Thereforetraining, also when given by a teacher in a classroom, usually follows the same patterns, treating many differentcomponents, but always in the same way. The learning goals are usually addressed by using a rather limited setof didactic strategies. There is also much communality between technical maintenance training for very differentsystems. Using these characteristics of technical training, domain expertise from the technical manuals was takenas the basis for development of training material and didactic expertise was standardised and made available.

Lessons can be described in the form of scenarios, indicating which steps have to be taken by both trainers andtrainees to reach the learning goals. For each step of a scenario, a fragment of technical material can be selectedwhich provides the domain knowledge. For example, when the lesson-scenario describes first to teach thefunction of a component and next the structural position, the learning material should contain a functionaldiagram and a picture of the position of the component in the equipment. Because the didactics in maintenancetraining are a rather structured set, didactic ontologies could be developed (Kabel, 2001). These didacticontologies served as the basis for the design and implementation of a scenario-tool, providing the developer oftraining material with didactic expertise and supporting him or her to select the right kind of information on thedomain.

This scenario-tool is part of the set of IMAT tools, methods and scenarios, developed to assist companies in thetransition from paper-based training to a training based on effective and efficient use of available electronicdocumentation. The tools provide capabilities to analyse and segment electronic documents on the basis ofcharacteristics of graphics and text and the relations between them. Coherent segments of information are storedin a company-wide database. Ontologies providing meta-data about an application domain, together withdidactic knowledge about classification of instructional elements and scenarios, are used to index and annotatethe database. The IMAT approach also provides support to develop a corporate memory and feedback structure.The database with indexed and, if needed, annotated fragments taken from maintenance manuals, provides the

1 The IMAT project (ESPRIT 29175) was partly funded by the European Union. Partners in the project were: IntelligentSystems Laboratory, University of Amsterdam (NL), TNO Human Factors and TNO Physics and Electronics Laboratory(NL), Loria (F), Renault (F), AFPA (F), ETRA (E), Tecnopolis CSATA Novus Ortus (I), Royal Netherlands Air Force (NL),EURISCO (F).

- 42 -42

developer of training material with an easy accessible source for finding the raw material needed for a particulartopic. We will not describe in this paper how the database is constructed, but focus on the use of the database bythe scenario-tool.

Activities in the development of training documents

There are three major steps in the development of training material:

1. Set up a lesson structure

In this step the developer creates a structure or scenario in which the content can be filled in. For example alesson on a component starts with the objectives of the lesson, the location of the component in the system, thefunction of the component, the possible malfunctions, and the repair methods. The structure can have the form ofa template, for example a set of PowerPoint sheets where each of these lesson elements can be inserted. In alarge organisation, templates can be made available at a centralised level, taking into account the house style,ensuring that the lessons given at a decentralised level are homogeneous etc. The more standardised andstructured these lesson structures are, the easier it is for the developer to look for the right kind of content (Boot& Barnard, 2000). The scenario-tool described in the sections below supports the developer in creating lessonstructures. There are several reasons for providing support in the form of scenarios:

• To provide on the one hand standard ways of creating instructional material and organising training inorder to make this process more efficient, while on the other hand allowing for customisation accordingto the specific needs of the user.

• To provide standard good practice ways of training, especially for those instructors and developers ofinstructional material who are inexperienced or who have little background in educational science.

• To show instructors and developers of instructional material how to set up new and innovative ways oftraining.

• To work towards further industrialising the process of training development in technical domain.

2. Search and retrieve content

In order to fill in the slots in a lesson scenario, the developer has to find the right source of content. This isusually a complex and time-consuming task. For example, it would be very enlightening for trainees to see apicture of a malfunction in a system, but it is not easy to find a photograph of a faulty component. For technicaltraining, much of the content may come from the maintenance or the operational manuals. If the original manualdoes not contain material that is good enough, sometimes material can be found in the manual of another system,because particular components look alike. In order to shorten the time the developer has to search for content,the IMAT tools take technical manuals, fragment them, index them and store them in a database. The trainingmaterial developer can now search for fragments on a particular topic (such as component X) or with specialcharacteristics (such as a front view of the component).

The IMAT tools allow the developer to search for fragments, using several kinds of indexes. He or she cansearch on topics or keywords, on descriptions of the kinds of fragments (such as short or elaborate ones), onmedia (text, pictures, audio or video) and on instructional indexes such as material use and knowledge type.During the search the developer can also index the fragments with these last kinds of indexes, for example if heor she finds a fragment to be used as an example, the tag "example" can be added to this fragment. In this wayseveral developers can build up instructional knowledge. The selected fragments can fill in the lesson scenarioslots as defined in step 1. This scenario structure in the IMAT database is called an “instructional bag”. Ascreendump of the tool with which fragments can be searched is given in figure 1.

3. Process content

After the raw material has been gathered, the lesson needs to be further processed in an authoring environment.This can be a very simple environment like a word processor or a presentation program, but it is also possible touse a special authoring language to create CBT or a tool to create web-based training. In any case, the developerhas to use his or her creativity and knowledge about presentation techniques to create useful and effectivetraining material. In the IMAT system, the user can directly drag a fragment to a standard Windows authoringenvironment, but the usual way is to take the instructional bag with the fragments and use these for furtherprocessing. The scenario serves as a template for the lesson to be made, the developer knows for which purpose

- 43 -43

the fragment was chosen and the actions the instructor and the trainees are supposed to perform on this piece oftraining material.

Figure 1: IMAT fragment search tool

The scenario-tool

In this section we will describe in more detail the working of the scenario-tool. In IMAT this scenario-tool hasbeen developed to guide the developer to set up lesson-structures (Verstegen et al., 2001). The developer startswith a certain topic he or she wants to make a lesson about, for example a part of the motor of a car. Thescenario tool supports the developer in defining a scenario by combining pre-defined instructional patterns. Thescenario is gradually elaborated by selecting, adapting or creating the scenario elements (learning goal,knowledge type, instructional strategy, instructional activities, instructional steps and actions) (Kabel,Riemersma & Wielinga, 2000, Chen 1995). Depending on the choices of the user, an appropriate selection isoffered to user in each step. For example, if the user has determined an instructional strategy, he or she will beprovided with the list of instructional activities which belong to this strategy.

With the support of the scenario-tool, the developer has to perform the following steps for each topic:

1. Set a learning goal, to be chosen from a list of learning goals suited for technical training.

2 . Determine the knowledge type the trainee has to acquire, for example perceptual knowledge orperforming skills, choosing from a list of knowledge types belonging to the chosen learning goal.

3. Choose an instructional strategy, for example exploration or modelling, again choosing from a list ofstrategies belonging to the chosen knowledge type.

4. Determine a set of instructional activities, the activities the instructor and the trainee have to perform,for example setting goals, asking questions, feedback etc. The list of activities is presented whichbelong to the chosen instructional strategy.

5. Define accompanying instructor and learner actions, a further refinement of the instructional activities,for example present a learning goal, present a summary, read an explanation, memorise instructions.

When the developer has made all the choices, he or she is provided with a scenario, a standard lesson structurewhich contains advice on the kinds of fragments with which the lesson can be build. For example fragments onprocedures, tools, or tests. The scenario can also be edited. When the scenario is ready, it can be stored in thedatabase. Next the developer will use it to select and retrieve useful fragments from the database.

- 44 -44

Step 1: set a learning goal

The scenario-tool starts to provide the user with an empty screen in which the scenario will be constructed step-by-step (figure 2). Most of the work for the user consists of choosing the appropriate items, upon which the toolsuggest limited sets of other categories from which to chose.

One column ofthescenario

Figure 2: Scenario tool, with empty screen

Figure 3: Learning goal choice

- 45 -45

The developer starts with selecting a learning goal from a (collapsible) tree, figure 3.

Upon validation of the dialog box, the “Learning Goal” column will be filled in with the item chosen, then the“Fragment Description” column will be filled in with appropriate text automatically and the next column will beavailable for modifications.

Step 2 Determine the knowledge type

The next step is to choose the knowledge type (figure 4). The user can modify at any time any previous (andeditable) columns, but every change in a previous column may cause an update of next columns. For example, ifyou change the “Learning Goal” column, every next column will be cleared (unless you have chosen the sameitem). The knowledge type to be chosen is restricted by the learning goal selected at the previous step. Then ifyou choose another learning goal, you have access to other knowledge types.

Step 3 Choose an instructional strategy

The next step is to fill in the instructional strategy column. The instructional strategies which can be selected arerestricted by the knowledge type’s choice. If you choose another knowledge type, you will get differentinstructional strategies. In this step, one instructional strategy has to be selected.

Step 4 Determine a set of instructional activities & Step 5 Define accompanying instructor and learner actions

Upon completion, the following part of the scenario will be automatically filled in (figure 5). Instructionalactivities, instructor and learner actions and material use are filled in with appropriate contents regarding theinstructional strategy you have chose. Each cell in the “I/L Actions” is prefixed with “Learner” or “Instructor”depending of the role who performs the action in the scenario. The scenario can be completed by instantiating“Action description” column with free text. This text will not be used for the search of instructional fragmentsbut it will helps the user to complete the scenario with concrete and precise actions related to the specifiedinstruction. After completing the scenario, the tool is able to save it into the database. The scenario can also beexported to an HTML document and printed out.

Apart from this “stand alone” use, the Instructional scenario tool can also be used in conjunction with thedatabase which contains the fragments from the technical manuals. The user can now employ the

Figure 4: Knowledge type choice

- 46 -46

characterizations from the skeletal lesson, e.g., knowledge type, material use and description type, as searchcriteria, together with a topic or key word. Concretely, he or she can directly launch the appropriate fragmentsearch using a pop-up menu on each action. The retrieved and selected fragments are then automatically storedin the relevant action of the instructional bag, instead of being copy-pasted to the authoring environment. Whenfinished, the bag can be exported to a file structure that reflects the structure of the skeletal lesson. This allowsthe author of instructional material to access the material at a later date, or to share it with other authors. The linkwith the database and fragments search tool guarantees an easy and natural connection between structure andcontent, by “pre-fabricating” for the author the kind of meaningful queries that can be entered.

Using scenarios to index fragments

An important function of the scenario tool is to get useful fragments that are indexed in different ways, byknowledge type, material use (derived from instructor and learner actions), fragment description type and topicto be taught. It is possible to search in the database for fragments that are marked-up with these fragmentattributes. For example one can look for a fragment, which is about component X (topic), about how to takeprecautions (knowledge type), is a reminder (material use), and has a task/procedural description of a condition(fragment description type).

In the document analysis process, the fragment description type is derived automatically to a large extent. Mostfragments are therefore indexed on fragment description type. Knowledge type and material use, on the otherhand, have to be indexed manually by the training material developer. Therefore, it might be possible that not allattributes have values, so the developer has to search for fragments based on a subset of values. However, thescenario tool can also help authors to index fragments, thus providing more support for a next cycle ofdevelopment or for another author. In figure 6 the uses of the scenario tool are summarised.

Figure 5: The filled in scenario

- 47 -47

There are three ways of initiating the instructional indexing process:

• Standard use

If a developer first starts working with the database of fragments, the knowledge type and material use attributeswill have no values. During the selection of fragments, the developer can mark-up the fragments with thisinformation. This means that the knowledge type and material use tags are added to the fragments. So the nexttime a developer uses the database, he/she can use these search terms. In the IMAT tools, the retrieval and theindexing process are integrated; it is possible to perform the mark-up during the authoring process.

• Inverse use

The developer can use the Scenario Tool to initiate the instructional indexing by searching for relevant fragmentsfor a given scenario without the instructional indexes (because there are none or few ones), keeping in mindthese instructional indexes and then index the retrieved fragments with these scenario instructional indexes. Heor she can also obviously index without using the Scenario Tool in this “inverse” way. This approach is moredriven by the instructional point of view of the development work, in which the main task of the developer is notindexing.

• Extensive use

It is also possible to index (nearly) all the material with instructional mark-up before an authoring process. Thismight be useful in a situation where a central training centre has a database, which is to be used on location inorder to produce training material locally. In this case, someone at the centre is devoted to do the indexing.

However, even if these three different ways of initiating the instructional indexing are concretely available, theusers’ feedback on the IMAT tools indicate that the extensive use is rarely possible and that on-the-fly indexing(standard or inverse) is preferred. Note that fragments can have more than one value for knowledge type andmaterial use. For example a fragment might be used as an explanation as well as an illustration. Fragments canalso be annotated with the experiences in use, such as the usability, clearness, quality etc. of the fragment. Tothis purpose an annotation system is available, based upon an ontology of organisational learning.

Conclusions

In technical training, work and training are often closely coupled. The things trainees are taught during theirtraining should be directly applicable in the actual maintenance work they are going to perform. Furthermore,there is a trend towards more integrated forms of training and working, on-the-job-training and just-in-time-training. Trainees are not only trained in the beginning of their career as maintenance engineers in a classroom

Figure 6: Scenario Tool and relation with user and other tools

Database offragments

Retrieval tool

authoring/indexing

composing scenario

components

fragmentsscenario withsearch indexes

requests

Scenario Tool

- 48 -48

setting, but more and more training takes place at the workshop during the actual work. People also receive re-training when systems are modified or new systems arrive and are sometimes trained just before a new orspecialised task has to be performed in which they are not proficient.

By the scenario-based approach we described in this paper we allow developers of training material to re-usefragments as they appear in the technical manual and to face the above mentioned new challenges. They canplace these fragments just as they are in the context of the training material, adding explanations if necessary. Itis easy to build scenarios which simulate the normal course of maintenance work, instantiated with the “real”text and pictures from the technical manual, thus providing the trainees with realistic cases. In addition, thefragment search, which is very time consuming, is incorporated directly in the core-task of the author of trainingmaterial, i.e. designing a lesson.

By the ontology-based approach that has been adopted for all the tools in the project, we aim at promoting a highlevel modelling of training activities resulting in the sharing of general and domain specialized teachingexpertise.

In the IMAT tools an extensive annotation facility is provided, both at the fragment, the scenario and theinstructional bag level, which was not further described in this paper (see Barnard & Boy, 2001). It can beviewed from every screen in the tools. This facility allows authors to comment on each others work, and torevise material after feedback from the training or from the workshop. By adding annotations, users can build-upcollaboratively knowledge and experience about developing and using training material. By initiating anannotation process through the scenario tool, we hope to help training material designers to be on the frontlinefor building the coming knowledge society, promoting collaborative and distance authoring and experiencesharing.

Developers of training material at several training centres have tried out the scenario-tool, but now we need morefeedback from users in order to adapt these tools to their changing needs. Finally, we believe that such processesmaking a systematic and heavy use of digital resources combined with human expertise should help workers intheir life-long learning processes in rapidly changing industries.

References

Barnard, Y.F., & Boy, G. (2001). Organisational memory. ESPRIT 29175, IMAT report.

Barnard, Y.F., Riemersma, J.B.J. & Hoog, R. de (1999). The use of IETM's in training for maintenanceengineers. Proceedings of the 21th Interservice/Industry Training, Simulation and Education Conference,pp. 1136-1144.

Boot, E.W., & Barnard, Y.F. (2000). A building block method for structured courseware development. In: J.Bourdeau & R. Heller (Eds.) Proceedings Ed-Media 2000, World Conference on Educational Multimedia,Hypermedia & Telecommunications. Charlottesville: AACE, p. 1566-1568.

Chen, H. (1995). A methodology for characterising computer-based learning environments. InstructionalScience, 23, 183-220.

Gagné, R.M., Briggs, L.J., & Wager, W.W. (1992). Principles of Instructional Design (4th ed.). Forth Worth:Harcourt Brace Jovanovich College.

Hoog, R. de, Barnard, Y.F. & Wielinga, B., (1999). IMAT: Re-using multimedia electronic technicaldocumentation for training. In Roger, J.-Y. et al (Eds.) Business and Work in de Information Society: NewTechnologies and Applications, IOS Press, p 415-421.

Kabel, S. (2001). The added value of ontology-based instructional mark-up. In: J. Moore et al. (Eds.) Artificialintelligence in education. IOS Press, AI-ED 2001 San Antonio.

Kabel, S., Riemersma, J. & Wielinga, B. (2001) Final set of ontologies. IMAT deliverable O3.

Verstegen, D., Barnard, Y.F., Kabel, S., Riemersma, J., Desmoulins, C., & Grandbastien, M. (2001). Scenariosand Guidelines, version 2. IMAT deliverable R.IV.3.

- 49 -49

Generating Training and Assistive Dialoguesfor Astronauts from International Space Station

Technical Documentation

Gregory Aist and Beth Ann Hockey

Research Institute for Advanced Computer Science (RIACS),Mail Stop T27A-2, NASA Ames Research Center, Moffett Field CA 94035-1000 USA

{aist, bahockey}@riacs.eduhttp://www.riacs.edu/

Abstract

Human spaceflight requires extensive and thorough procedures that cover a wide range of activities to beperformed both in space and on the ground. The technical documentation containing these procedures is afruitful source for potential training material, since crewmembers and mission control personnel undergo longand extensive training in the course of preparation for particular mission assignments. One of the main goals ofthe RIALIST group at Ames is to support Human Exploration and Development of Space (HEDS) by applyingspoken dialogue technology to training and task performance on the ground and in space. We are moving toaddress this goal by developing a system aimed at taking technical documentation from the International SpaceStation and producing spoken dialogues geared towards astronaut training and task support. This system is in theearly stages of development – six months as of June 2002 – and is still at the design and early prototyping stage.In this paper, we introduce the scope and nature of tasks performed on board the International Space Station,describe our system architecture, discuss some of the key issues that have arisen in building the early prototypes,and lay out future challenges.

Keywords

astronaut training, task support, spoken dialogue systems

Introduction

Efficient and effective training is key to human spaceflight. Spaceflight is a complicated endeavor carried outwith people and machines operating in tandem. A wide variety of tasks are performed on board the InternationalSpace Station. For example, the ISS on-board status report of March 28, 2002(http://www.spaceref.com/news/viewsr.html?pid=5093) included the following activities shown in Figure 1.

Current astronaut task practices include a mix of high- and low-fidelity simulations. There are time delaysbetween training & use. Some procedures are performed frequently, while some are performed quiteinfrequently. There are lots and lots of manuals. Finally, in some cases, another person is reading the checklist tothe person performing the task.

- 50 -50

For each of these types of tasks, there are procedures associated with them. For these procedures, the trainingshould match task; ideally perhaps the same computer assistant should be used for both training and for taskperformance. We are thus developing a spoken dialogue system for training and task support that is aimed attasks typically performed on board the International Space Station – with a training deployment envisionedinitially, but also directed at eventual on orbit deployment. In related work which includes in-task support,Rudnicky et al. (1996) describe supporting vehicle maintenance with speech interfaces; the Mission RehearsalExercise (USC n.d.) aims at supporting “mission-oriented training” for military personnel.

System Architecture

We developed a procedure assistant consisting of the following components (Figure 2). First, the speechrecognizer (Nuance, using specially compiled language models (Dowding et al. ACL 2001)), recognizes theuser’s speech and emits a string of words which are passed to the Gemini parser (Dowding et al. ACL 1993).

· “live interactive TV exchange with Bath Elementary School in Bath,North Carolina.”

· “unloading of Progress 7P, using the IMS (inventory managementsystem)”

· “transferr[ing] the Russian radiobiology experiment RBO-2 "Bradoz"from the Progress and install[ing] it in the SM.”

· “solid sorbent air sampler” activity

· collecting “FMK (formaldehyde monitoring kit) badges”

· “Part 2 of the inspection of the docking assembly…”

· “… the second part of the BINS strap-down inertial guidance systemupdate/correction using the VP2 and PUMA viewfinder/sighting devices.”

· “reviewed the updated procedures of the MSS (mobile servicer system)operations”

· “completed a dry run of the S0 installation”

· “In preparation for the 8A spacewalks, … performed a thoroughcheckout of two SAFER (simplified air for EVA rescue) units”

· “prepared the Joint Airlock (A/L) for the coming EVAs, among elseunstowing EMU (extravehicular mobility unit) equipment bag and EMUservicing kit, and attaching the equipment bag at a designated location in theA/L, ready for use. EMU reconfiguration and spacesuit sizing is scheduled fortomorrow.”

· “Bursch completed the weekly FFQ (food frequency questionnaire) onthe MEC laptop, which keeps track of the crew's nutritional input, …”

· “while Onufrienko did the regular BRPK-1 condensate water separator.”

· “The crewmembers performed their daily physical exercise.”

· “Today's target areas for the CEO program…” (photography): LakeEyre, Australia; South Sandwich Islands; North Patagonian Glaciers; SouthPatagonian Glaciers.

Figure 1. Daily activities on board the International Space Station, March28, 2002; http://www.spaceref.com/news/viewsr.html?pid=5093.

- 51 -51

Gemini emits logical forms that are passed to the discourse manager (DM). The discourse manager uses atranslation function to change resolved logical forms into actions that the procedure manager (back end) canhandle. The procedure manager loads a new procedure into memory, and maintains it in memory as a liststructure. The procedure manager also navigates between steps in a procedure, and generates task state reportsthat are then passed to the dialogue manager. For example, a task state report might be[procedure_loaded(camera), “Take pictures of the Earth.”]. The dialogue manager takes the task state report,constructs a combination of logical form (LF) and/or word strings, and passes that to the generator. Thegenerator then constructs strings for the text-to-speech engine (Festival) to emit, based on the command receivedfrom the dialogue manager. Each procedure is stored as a separate Prolog file, in order to allow for dynamicallyswitching between procedures.

Procedure data structure

We designed a procedure that walks the user through taking a picture of a view of Earth from space. Theprocedure is specified as a tree, with each node consisting of a description of the node and a list of its children.The description of the node is written as an unanalyzed imperative. A portion of the camera procedure is shownin Figure 3.

Separate left and right lists of nodes are kept so that the current position in the node’s children can be maintained– the head of the right list is always the current position in that list. The procedure is loaded into memory as aProlog term with the structure of subprocedures left in unexpanded form, and instantiated when needed via a callto procedure/5.

Speech recognizer

Word strings Word strings

Parser

Speech synthesizer

Dialogue manager

Logical form (LF)

Generator

Logical form (LF) and/or word strings

Procedure manager

Camera

Physical exam …(Other procedures)

Procedures

Actions Task state reports

Figure 2. System architecture.

- 52 -52

Current Capabilities of the Intelligent Procedure Assistant

The intelligent procedure assistant currently has the following capabilities:

1. Load a procedure into the procedure manager.

2. Say again - repeat a prompt.

3. Next step - move to the next step in the procedure.

/* procedure(Name, Features, Left, Right, Parent) */procedure(photograph_the_earth,

[description: ’Photograph the earth.'], [], [procedure(unpack_camera, _, _, _, _), procedure(install_battery, _, _, _, _), procedure(install_memory, _, _, _, _), procedure(prepare_camera, _, _, _, _), procedure(take_picture_of_madagascar, _, _, _, _), procedure(unprepare_camera, _, _, _, _), procedure(uninstall_memory, _, _, _, _), procedure(print_pictures, _, _, _, _)], _).

procedure(unpack_camera,[description:'Unpack the camera.'], [], [procedure(undo_velcro_strap, _, _, _, _), procedure(unzip_both_sides, _, _, _, _), procedure(remove_camera, _, _, _, _)], _).

procedure(undo_velcro_strap,[description:'Remove the velcro strap.'], [], [], _).

procedure(unzip_both_sides,[description:'Unzip both sides of the camera case.'], [], [], _).

procedure(remove_camera,[description:'Remove the camera from the camera case.'],[], [], _).

procedure(install_battery, [description:'Install the battery.'], [], [procedure(open_battery_compartment, _, _, _, _), procedure(remove_battery_from_charger, _, _, _, _), procedure(put_battery_in_camera, _, _, _, _), procedure(close_battery_compartment, _, _, _, _)], _).

procedure(open_battery_compartment, [description:'Open the battery compartment.'], [], [], _).procedure(remove_battery_from_charger, [description:'Remove the battery from the charger.'],[], [], _).procedure(put_battery_in_camera, [description:'Put the battery in the camera.'], [], [], _).procedure(close_battery_compartment, [description:'Close the battery compartment.'], [], [], _).procedure(install_memory, [description:'Install the memory in the camera.'],

[], [procedure(remove_memory_from_camera_case, _, _, _, _), procedure(put_memory_in_camera, _, _, _, _)], _).

procedure(remove_memory_from_camera_case, [description:'Remove the memory from thecamera case.'], [], [], _).procedure(put_memory_in_camera, [description:'Put the memory in the camera.'], [], [], _).procedure(prepare_camera, [description:'Prepare the camera for photography.'],

[], [procedure(turn_camera_on, _, _, _, _), procedure(remove_lens_cap, _, _, _, _)], _).

procedure(turn_camera_on, [description:'Turn the camera on.'], [], [], _).procedure(remove_lens_cap, [description:'Remove the lens cap.'], [], [], _).…

Figure 3. (Part of) the camera procedure.

- 53 -53

4. Drill down - unpack a procedure into its component parts.

Current procedures available include a procedure to take pictures using a digital camera, and a procedure toconduct the initial portion of a periodic physical exam.

Example Dialogue

A hypothetical example dialogue is shown below. (The system can engage in dialogues such as these, althoughthe details may vary.)

User: Load camera procedure.

System: Camera procedure loaded. First, unpack the camera.

User: Now what?

System: Now, install the battery.

User: How do I do that?

System: To install the battery, first open the battery compartment.

User: Please explain.

System: No more information available about how to open the battery compartment.

User: Say that again.

System: Open the battery compartment.

User: Next step.

System: Remove the battery from the charger.

User: What now?

System: Put the battery in the camera.

…

Key Issues

We discuss some of the key issues facing us as we develop this intelligent procedure system.

Handling long and detailed procedures. Some of the procedures we are working with may be quite detailed,and extend for many steps. Thus we have implemented lazy evaluation of procedures - instantiate only what youneed of each procedure, and load the rest on demand. Such long procedures also present demands on thedialogue system to maintain the user’s awareness of where he or she is in the procedure.

Representing the task structure of procedures. We chose to use lists of lists to represent task structure, becauseprocedures may contain other procedures and this structure facilitates moving from step to step or drilling downto another level of detail.

Representing the meaning of a procedure. Right now, we use an unanalyzed imperative as the first take on whata procedure means; eventually, we would like to use logical form as representation of meaning of procedure.Logical form would enable us to construct the system to handle more sophisticated use of the intenal meaningsof the procedures. For example, a user could check off an item by paraphrasing it: “Okay, I’ve opened thecamera case.”

Matching semantics of the software to objects in the real world. This grounding problem is mostly solved if aperson is executing the procedure – we can rely on him or her to figure out what the number 7 wrench incompartment 23 is. On the other hand, grounding objects becomes quite interesting if human & robot(s) shareexecution. Perhaps a robot might ask the human to ground an object by pointing to it, for example. Or, if itemsare already barcoded, a robot could read the barcodes to confirm that an object it had selected was the onereferred to by the intelligent procedure assistant.

- 54 -54

Future Challenges

We now discuss some future challenges. These range from the simple to the complex.

Multiple domains / procedures. The intelligent procedure assistant should be able to work with procedures froma variety of areas, as laid out in Figure 2. Challenges here include access to a wide-coverage lexicon, and

Shared execution of procedures. In some cases, multiple people or robots would be using the system,necessitating further research into spoken dialogue design.

Translating procedures into Prolog clauses. What if procedures are written in multiple formats? – an issueother workshop attendees may face as well. Should we translate from one format to another? Such anarrangement might be needed if the original procedures are fixed, but for newly written or revised procedures,we would prefer to have them written in a standard format (such as XML) and then write a single translator thatwould take those procedures and generate Prolog code such as that shown in Figure 3.

Markup mechanisms. Mechanisms for marking up documentation for use by the dialogue system includegetting lexical items suitable for use by the grammar out of the technical documentation.

System improvement from prior performance. Finally, we would like to use data from training runs to improvethe system for later training and future deployment.

Conclusion

One of the main goals of the RIALIST group at Ames is to support Human Exploration and Development ofSpace (HEDS) by applying spoken dialogue technology to training and task performance on the ground and inspace. We are moving to address this goal by developing a system aimed at taking technical documentation fromthe International Space Station and producing spoken dialogues geared towards astronaut training and tasksupport. In this paper, we have introduced the scope and nature of tasks performed on board the InternationalSpace Station, described our system architecture, discussed some of the key issues in this system, and laid outfuture challenges. Furthermore, workshop participants will be provided a hands-on experience with our system,which converts a technical procedure written as a set of logical clauses into a spoken dialogue interaction aimedat astronaut training and support.

References

1. J. Dowding, B. A. Hockey, J. M. Gawron, and C. Culy. 2001. Practical issues in compiling typed unificationgrammars for speech recognition. ACL 2001.

2. J. Dowding, M. Gawron, D. Appelt, L. Cherny, R. Moore, and D. Moran. 1993. Gemini: A natural languagesystem for spoken language understanding. ACL 1993.

3. Rudnicky, A. I., Reed, S. D., and Thayer, E. H. 1996. SpeechWear: A mobile speech system.http://www.speech.cs.cmu.edu/air/papers/speechwear.ps

4. USC. N.d. Mission Rehearsal Exercise. http://www.ict.usc.edu/misreh.html

- 55 -55

Maintenance training design: Less is more

Lesley Jacobs, Anja van der Hulst & Stefan van der Stigchel

TNO-FEL

P.O. Box 96864, 2509 JG The Hague, The Netherlands

e-mail: [email protected]

Abstract

Maintenance training, nowadays, primarily consists of the use of real equipment as single training device. In thispaper, arguments will be presented in favour of the use of medium fidelity type trainers (e.g. CBT) over highfidelity type trainers such as the equipment itself. Such medium fidelity trainers not only cost less, they alsoprovide an environment allowing learners to be more actively involved in troubleshooting practice, they allowmore learners to be involved simultaneously and the application of such facilities consumes less of theinstructors time. All in all, in our opinion ‘lesser’ training facilities, such as CBT and computer basedsimulations, provide better means to facilitate training for troubleshooting skills.

Keywords

Maintenance training, Troubleshooting, Computer Based Training, Fidelity, Case based training

Introduction

Few fields are more traditional than that of maintenance training. Where operational tasks are trained more andmore with generic trainers, games, or Computer Based Training (CBT), a great majority of maintenance tasksstill are trained on the ‘real thing’, the real operational equipment. Instructors have always used the realequipment for troubleshooting practice and for mechanical adjustment tasks and see no other ways of providingtrainees with adequate practice.

However, using real equipment is neither a practical nor a very cost effective choice for training, for severalreasons; 1) The ‘real thing’ is generally extremely expensive, with usually as a consequence that too few systemsare procured for training. 2) The limited training systems that are available have a capacity of just one student ata time (low bandwidth). For the rest of a class remains a passive status of on-looker. 3) On top of that,troubleshooting practice on the real systems makes high demands on instructor capacity. Each malfunction has tobe inserted individually by inserting a fault or faulty component. This consumes a lot of instructor time andtrainer time– and thus furthermore reduces the time available for troubleshooting practice.

All these factors make that students receive very little hands-on practice. The instructor may show the effects ofsome malfunctions but the student gains very little experience in the actual troubleshooting. If troubleshootingpractice was to be provided with CBT based facilities, an entire class could be involved in practicesimultaneously. Students would be able to practice as much as they need, in their own pace and the underlyingsystem theory about function, structure and behaviour could be directly related to that practice. But, CBT cannever equal the fidelity level of the real apparatus.

- 56 -56

So the question addressed in this paper is: is it really needed to provide a 100% fidelity training environmentwith the real apparatus? To gain an answer, in the context of the BOOT1 project we reviewed quite a bit ofliterature describing empirical studies after transfer of training of low and medium fidelity maintenance trainers(see: van der Hulst, van den Nieuwenhuyzen and van der Stigchel, in progress).

In this paper we will document the outcomes of this literature review and come up with a number of generalrequirements for facilities for maintenance training. On the basis of those requirements and a more thoroughanalysis of the knowledge and skills involved in troubleshooting we will provide guidelines for the design offacilities for maintenance training.

100% fidelity for maintenance training, is it really needed?

It certainly isn’t. In describing fidelity requirements for maintenance training, we’ll limit the discussion totroubleshooting. This maintenance task can be seen as the systematic tracing of faults in equipment. The taskrequires a thorough knowledge of the functioning of equipment and its separate components. It requiresknowledge about the structure of the system. It requires knowledge about the incidence of certain faults and thebehaviour that is related to those faults. Students have to learn to apply this knowledge by actually trying to tracesimulated malfunctions. For this they need training systems and the question is, must these training systems bethe full fidelity real equipment?

In several studies the effects of lowering fidelity in training simulators for troubleshooting have beeninvestigated. Authors typically distinguish between functional and physical fidelity of the simulators. Physicalfidelity in maintenance trainers refers to the degree to which the components of simulated system ‘look like’those of the real equipment. The medium/low physical fidelity representations used in the studies are generally(photo)graphical or symbolic replications of components. Functional fidelity refers to the degree to which thesimulated components ‘act like’ those of the real equipment. Medium/low functional fidelity is behaviour notalike the actual behaviour. It can, for instance, be the absence of behaviour where the real system would displaydeviant behaviours.

In fidelity experiments, researchers compared the transfer of training of training on the real equipment and thatof training with medium or even low fidelity trainers. Several studies indicate that lowering physical fidelityonly marginally affects student performance. Cicchinelli, Harmon, Keller and Kostenette, (1980) for instance,compared a three-dimensional medium physical/high functional fidelity trainer for electronic equipmenttroubleshooting with training on the actual equipment. It was observed that the performance of students trainedon either of the trainers was exactly the same. A second experiment by the same authors included yet another(two dimensional) medium physical fidelity trainer again revealed no differences in learning outcome whencompared to the actual device based training.

The same observation was made by the several groups that measured training effectiveness of an mediumphysical/high functional fidelity general purpose trainer for maintenance (Spangeberg, 1974; McGuirk, Pipe, andMiller, 1975; Wright and Campbell, 1975). On the other hand, Allen, Hays and Buffardy (1986) notice a smallbut significant effect on one of their performance measures in an experiment on troubleshooting an electro-mechanical device.

Lowering functional fidelity, however, seems to severely affect student performance, as was shown in anexperiment conducted by Allen, Hays and Buffardi (1986). In an experiment focusing on troubleshooting anelectro-mechanical device, they varied three (low, medium, high) levels of functional and physical fidelity,leading to nine experimental conditions. They conclude that functional fidelity ‘is a very potent determinant ofperformance’. Decreasing levels of functional fidelity were associated with substantially longer solution times.

10 experiments by Rouse and Hunt (1984) with the so called TASK/FAULT system for teaching diagnostics ofelectronic networks have shown that training for troubleshooting can be enhanced with low physical andmedium/high functional fidelity computer based simulation. The notion of ‘mixed fidelity’: low to mediumfunctional fidelity simulations when mixed with real equipment practice reduce the need for high physicalfidelity hardware in training electronic switchboard maintenance. Diesel generator troubleshooting in a nuclearpower plant has been studied and it was shown that mixed fidelity simulation can provide effective training. Anexperiment by Macdonald et al. (1983) with three levels of functional and physical fidelity for electronic circuits

1 BOOT is a Dutch acronym and stands for ‘decision support for the selection of facilities for education and training’.

- 57 -57

and two proficiency levels showed no significant differences between experimental and control groups, onlyproficiency levels matter.

For the training of troubleshooting one might conclude that computer based simulations can be as good as the‘real thing’. Many of the above mentioned experiments used symbolic representations of the components(medium physical fidelity) which didn’t seem to affect the learning outcome significantly when compared totraining with the real equipment. Functional fidelity is essential though. Indeed, when simulating equipment fortroubleshooting training, it is essential to have a simulated component display highly realistic behaviour.

In conclusion, the empirical literature reveals that a medium physical fidelity level approach can be as good as afull fidelity approach. This implies that for the training of troubleshooting essentially the use of the realoperational equipment is not a necessity. One might seriously consider alternatives in the form of CBT whileusing symbolic representations of the components of the equipment. With the CBT’s advantages of highbandwidth, active involvement of the students and limited investment of instructor time. A ‘medium fidelityapproach’ might provide more educational value for definitely less money.

In the following sections we’ll elaborate on the ‘medium fidelity approach’ and come up with some concepts forthe instructional design of ‘medium fidelity’ facilities for maintenance training. To get to indications for theinstructional design (ISD) of training equipment for troubleshooting we’ll firstly outline the nature ofmaintenance task, and thus the knowledge and skills to be targeted.

Cognitive aspects in maintenance tasks

Typical maintenance tasks include: operational skills to use the equipment, testing the operational status of theequipment, access a certain component in the equipment, disassembly, repair of components and troubleshootingin case of malfunctioning equipment. The latter is probably the most challenging maintenance task, also from atraining point of view. As indicated before we will limit the discussion to troubleshooting.

Troubleshooting requires a thorough knowledge on the structure and the functioning of the equipment, systembehaviour and occurrence of faults (including fault cause), fault tracking, tracing and isolation. Troubleshootingmostly takes place during corrective maintenance, this means after the occurrence of a malfunction. To a smallerextent troubleshooting occurs also during preventive maintenance tasks and reconfiguration tasks, for instancewhen a certain component gets replaced by a new component type.

In accordance with Halff (1990b) we would like to make a division in three cognitive components that are usedduring troubleshooting tasks. The three components are mental models, procedures and fault isolation ortroubleshooting skills. Of course, the latter should be the most important component to convey in a trainingmethod for troubleshooting. However, mental models form the basis for reasoning about the structure andfunctioning of the equipment, especially during troubleshooting and different kind of procedures also play animportant role in troubleshooting. So, all three types of cognitive components enhancing maintenance skillsshould be addressed in the design of a training method for troubleshooting.

I. Mental models

In short, troubleshooting is regarded as a complex cognitive skill. Mental models seem to be particularlyimportant for the performance of complex cognitive skills that encompass reasoning about processes or devicessuch as in troubleshooting or fault isolation (van Merriënboer, 1997). Therefore a special focus in trainingmethods for troubleshooting should lie on the teaching of mental models. Mental models of equipment refer tocognitive structures used to reason about the equipment being maintained (Halff, 1990a). In terms of equipmenta mental model should include the structure of the equipment, the function and the physical appearance of theequipment. The strongest advantage of mental models lies in the fact that they provide a representation of theequipment in the working memory of a maintainer that can be "run" to understand the equipment, solveproblems or predict events. In the case of troubleshooting a mental model forms the basis for hypothesis testingduring fault isolation.

II. Procedures

As second cognitive component procedures were mentioned. Procedures in maintenance vary from standardprocedures for checking or (dis)assembly of components as listed in job aids or checklists, diagnosticprocedures, to emergency procedures that must be performed under high stress conditions. Essentially,

- 58 -58

procedures are series of actions to be carried out in a predefined order. Sometimes a procedure may have certainchoice points where a particular sequence of actions is chosen.

Despite the many differences in procedures within and across maintenance domains, two general thoughts cometo mind when thinking on maintenance and maintenance training, how to handle procedures and how to handlethe huge amount of (different) procedures. It is important to realise that, regardless of the differences, proceduresinfluence the existing knowledge structures and therefore the mental model of the equipment.

III. Troubleshooting skills

The third and most important cognitive component consists of troubleshooting skills. In literature, there is afairly coherent picture of skilled troubleshooting (Halff, 1989). When a fault mode in one or more of thecomponents of the equipment occurs, the troubleshooting task is to decide upon a sequence of actions that isolateand repair the malfunctioning components of the equipment. The actions in the solution sequence can be definedby the following possibilities: observe the outcomes of some components, observe the state of some components,manipulate the state of some components and repair or replace certain components (Halff, 1990b).

Instructional design, a medium fidelity approach

The question now rises to what extent a medium fidelity type of training such as CBT could provide a goodlearning environment for troubleshooting.

I. Mental models

A method suggested to help learners develop good mental models is to provide various conceptual models of theequipment during training (Hagemann, Mayer & Nenninger (1998) in Allessi & Trollip, 2001). Such models canbe represented in the form of component or block diagrams, goal-plan models, causal models, concept maps,hierarchies, etc.

An essential aspect of the development of mental models is to guarantee the consistency of the conceptualmaterial presented to the students in training and the technical documentation to be used during operationalpractice. Lack of consistency may well lead to confusion and consequently towards inadequate mental models.Therefore it is essential to reuse as much of the conceptual information as available in the documentation of theequipment. Many elements such as component and/or block diagrams shall be readily available, other conceptualmaterial may need to be derived from the manual, hereby trying to maintain a direct link to the originaldocumentation.

A second essential aspect of the development of mental models is to provide dedicated support to helpunderstand the relation between the various models. One may use a symbolic representation to reason aboutpossible faulty components, but, once this component is found, a realistic graphical representation is needed toactually locate the faulty component. Hence training should help convey an understanding of the relationsbetween the various conceptual models.

With this we have come to the discussion of the media nature to be used for such training. Real equipmentcannot convey any conceptual representations and as such is not an optimal mean to build a mental model.Hence, such conceptual representations are usually added on paper. Paper based methods (usually) provideconceptual models by means of block diagrams, equipment specifications and static graphics. CBT and computerbased simulations not only provide better means for displaying conceptual models e.g. by providingmultidimensional component overviews, dynamic graphic displays, video and schematic representations, butalso for the practice of causal reasoning, exhibition of complex functions of components and display of normaland faulted behaviour of components. On top of that multimedia technology allows easy linking of the variouselements of conceptual models (see the previous guideline). This suggests that multimedia technology, with theexcellent capacity for animation, diagrams and the like, has great potential for supporting the development ofmental models (Allessi & Trollip, 2001).

II Procedures

The most common method to train procedures is by applying a drill and practice method. That is, parts of theprocedure are demonstrated or described to the student and subsequently, students are supposed to do it

- 59 -59

themselves and are required to practice that part until mastery. Again, the parts are integrated until the fullprocedure is mastered.

Practice for (dis)assembly procedures is generally done on the real equipment1. Some test or diagnosticprocedures, however, can well be taught by means of medium fidelity simulations . That is, interactive symbolicrepresentations of the equipment with an underlying simulation model that is able to display both correct andfaulty behaviour of the components concerned. Such a simulation has as advantage that it may be augmentedwith dedicated facilities to help analyse students’ performance. An example of such facility is the CAI system(Kuiper, 1995) that incorporates a norm model (the correct procedure) that is used to analyse studentperformance. With such a facility it is possible to provide dedicated feedback and, if necessary, remediation.These instructional measures can hardly be provided while using the real equipment.

A note should be made in regard to procedures with strong sensory-motor aspects. For these types of skills,sufficient hands-on experience with the real equipment remains the only way to obtain the knowledge and skillrequired.

When using (interactive) representations of the equipment, again one should strive for consistency with thoseused to establish mental models. In this manner one can easily relate the merely theory oriented instructionalmaterial necessary to build mental models with the practice needed to master procedures.

As with the construction of mental models, in training for procedures it is again essential to guarantee theconsistency of the procedures presented to the students and those described in the technical documentation. Mosttechnical documentation incorporates test and diagnostic procedures. When such documented procedures areembedded in computer based simulation, it should be possible to visualise the relation between a step in adocumented procedure and an action on the simulation. One might, for instance, think of highlighting a step inthe documented procedure on a student action on the simulation or vice versa.

III Troubleshooting

Troubleshooting skills are considered as a mixture of highly knowledge based methods and weak contextindependent skills (Halff, 1990b). Troubleshooting skills are usually taught trough several case basedtroubleshooting exercises, and practice forms probably one of the most effective methods for troubleshootingtraining. One of the most challenging parts in training methods for troubleshooting is probably the sequencing ofproblems or cases to be used in the training environment.

Especially for troubleshooting skills density of practice is important. Exactly here several advantages of CBTand computer simulations over training on the real equipment come forward. By using CBT, the bandwidth ismuch broader, more students can work actively with the CBT instead of watching passively how a teacher orfellow student operates on the real equipment. In case of safety procedures CBT offers safe mean to obtain somehands-on experience before going into real practice. Furthermore, the real task environment often serves astraining on the job facility for the training of troubleshooting skills, and these skills will only be obtaineddepending on the faults and malfunctioning of the equipment in the present work environment. CBT andcomputer simulations offer the possibilities to train a much broader scope of troubleshooting exercises. Finally, acase based training method seems very well suitable for the training of troubleshooting, since troubleshooting isa special form of skilled problem behaviour. CBT could even contain a whole library of cases where either theinstructor or a learner could select a range of troubleshooting cases, for example from simple to complex,troubleshooting cases on certain components, rare or regular troubleshooting cases, etc. Arrived at the step ofcase based troubleshooting CBT, one step further could be an adaptive case based training environment fortroubleshooting, where the training environments sets up a sequence of troubleshooting cases, adapted to theeducational and contextual level of the learner.

Here too we plea for consistency. CBT implies medium level fidelity. The student interacts on a simulationunderlying a graphical or symbolical representation of the operational equipment. Ideally even this simulationshould be consistent with other instructional material in its presentation of the equipment to the student. Thesymbolic representations used in building a mental model, the interactive one for procedure training as well asthe equipment representation within the simulation should be as consistent and similar as possible.

1 Although examples are available of fruitful assembly practice on 3D models embedded in CBT.

- 60 -60

Integration of mental models, procedures and troubleshooting skills

The final question now rises how to integrate the training of mental models, procedures and troubleshootingskills into a coherent set of training methods. A first obvious step would be the creation of proper mental models.For this step theory, conceptual maps and the main or basic procedures of a system should be presented to thelearner. Also a basic introduction to the real system can be included during this phase of training. Note should betaken on the fact that whenever the real system is used in training the system must be handled as much aspossible under the student’s control of operation. A second step would include the use of CBT generated cases.Depending on the maintenance domain, it is possible to create a curriculum of troubleshooting cases, varyingfrom simple to complex, component specific or otherwise useful case categories for the specific domain. Bycreating ‘learning modules’ around a certain topic, it will be possible to create small loops from theory, practiseand testing of knowledge of skill. In this type of training environment, the real equipment will only be used inthose situations when it is necessary from an instructional point of view, instead of using the equipment as singleand main training device.

Potentials

Of course the mere use of medium fidelity simulations or CBT can never replace the real hands on experience,especially for sensory-motor skills. However, training methods for troubleshooting seem to be very well suitedfor medium fidelity type trainers such as CBT and computer simulation. Arguments in favour of this statement,include:

• the effectiveness studies mentioned in the first part of the paper, i.e. medium physical fidelity does notmean less transfer of training when compared to high fidelity trainers;

• cost effectiveness;

• high bandwidth;

• from passive to active learning;

• less time demands on instructors (shift from instruction to coaching);

• possible embedding of technical documentation;

• presentation advantages for conceptual modelling;

• training of procedures in a safe environment;

• possibility to train all troubleshooting cases or a larger set of cases then possible in real life.

A final argument includes the huge amounts of technical documentation available in the maintenance field. Thetechnical documentation of the equipment forms almost always the direct basis for the input and creation of thetraining materials. Especially in the maintenance field there is a strong trend from paper based documentation to(interactive) electronic technical manuals. It goes almost without saying that all the advantages mentioned in thispaper for the use of CBT in troubleshooting training hold even stronger when the source of technicaldocumentation is not paper, but electronically available source material. It certainly shall be easier to keep theCBT element derived from the documentation up-to –date.

Promising work on the integration of (electronic) manuals and training in the field of maintenance includesseveral research projects, such as the European IMAT project (http://imat.swi.psy.uva.nl/) and the work in thefield of classification standards for electronic technical manuals (http://navycals.dt.navy.mil/ietm/ietm.html).The two highest classes of interactive electronic manuals, IETM class 4 and 5, provide several possibilities tointegrate and link the electronic documentation directly to CBT. Thus enhancing many of the advantagesmentioned in this paper for medium fidelity trainers over high fidelity as training method for troubleshooting.

- 61 -61

Conclusions

Based on the literature review, our work1 in the field of maintenance training and troubleshooting for the DutchNavy and Royal Netherlands Airforce we come to the conclusion that medium fidelity based training methodscan provide a good educational and cost effective training method for troubleshooting. Many experiments onhigh versus medium fidelity trainers were conducted during the eighties and one can conclude that this kind ofexperiments certainly were not an important research topic during the nineties. However, the conclusions infavour of medium fidelity trainers for troubleshooting derived from these studies would probably hold evenstronger when conducted today, since the technical possibilities of CBT have evolved enormously since then.

References

Allen, J.A., Hays, R.T. & and Buffardi, L.C. (1986). Maintenance Training Simulation Fidelity and individualDifferences in Transfer of Training. Human Factors, 28(5), 497-509.

Allessi, S.M. and Trollip, S.R. (2001). Multimedia for learning (3rd ed.). Allyn & Bacon, Massachusetts.

Cicchinelli, L.F., Harmon, R.A., Keller, R. and Kostenette, J. (1980) Relative cost and training effectiveness ofthe three-dimensional simulator and actual equipment. (TECH. Rep. AFHRL-TR-80-24). BrooksAirforce Base, Tex. Air force Human resources Laboratory.

Halff, H.M. (1989). Prospects for automating instructional design. Arlington, VA: Halff Resources, Inc.

Halff, H. M. (1990a). Teaching troubleshooting procedures. Arlington, VA: Halff Resources, Inc.

Halff, H. M. (1990b). Automating maintenance training. Arlington, VA: Halff Resources, Inc.

IETMs: http://navycals.dt.navy.mil/ietm/ietm.html

IMAT: http://imat.swi.psy.uva.nl/

Kuiper, H. (1995). An instructional support system for training simulators. PhD thesis. (ISBN 90-9008918-7)

McGuirk, R.D., Pieper, W.J. and Miller, G.G. (1975). Operational Tryout of a General Purpose Simulator (Tech.Rep. AFHRL-TR-75-13). Brooks Air Force Base, TX.: Air Force Human Resources Laboratory.

McDonald, L.B., Waldrop, G.P., & White, V.T. (1983). Analysis of fidelity requirements for electronicequipment maintenance. (technical report NAVTREAQUIPCEN 81-C-0065-1) Orlando, FL: Navaltraining equipment center.

Merriënboer, J.J.G. van, (1997). Training complex cognitive skills: a four-component instructional design modelfor technical training. Englewood Cilffs, New Jersey: Educational Technology Publications, Inc.

Rouse, W. B. and Hunt, R. M. (1984). Human problem solving in fault diagnosis tasks.In Rouse, W. B., editor (1984). Advances in Man-Machine Systems Research Vol. I. Greenwich, CT: JAIPress Inc., pp 195-222.

Spangeberg, R.W. (1974). Tryout of a General Purpose Simulator in the Air National Guard TrainingEnvironment (Tech. Rep. AFHRL-TR-74-92) Brooks Air Force Base, TX: Air Force Human ResourcesLaboratory.

van der Hulst, A., van den Nieuwenhuyzen, N. & van der Stigchel, S. (in progress) BOOT kennisbasis; Richtlijnvoor opbouw, validatie en beheer. Den Haag TNO-FEL rapport.

Wright, J. and Campbell, J. (1975). Evaluation of the EC-II Programmable Maintenance Simulator in T-2C-Organizational Maintenance Training (Rep. No. NADC=75083-40). Washington, D.C.: Naval AirSystems Command.

1 TNO participates in consortia (e.g. the European Esprit project IMAT) and conducts research projects on the re-use andintegration of technical documentation for maintenance training, medium versus high fidelity simulation research and futurepossibilities towards the integration of IETM’s and CBT or e-learning.

- 62 -62

- 63 -63

Personalized eLearning and eCoaching in WINDS

Marcus Specht, Milos Kravcik, Roland Klemke, Leonid Pesin, Rüdiger Hüttenhain

Fraunhofer Institute for Applied Information Technology FIT

Group for Information in Context (ICON)

D-53754 Sankt Augustin, Germany

{specht, kravcik, klemke, pesin, huettenhain}@fit.fraunhofer.de

ABSTRACT

This paper introduces one approach to e-Learning describing the Adaptive Learning Environment (ALE) forweb-based authoring and learning. The system provides a new methodological approach to design education onthe web. ALE will be used to build a large knowledge base supporting Architecture and Civil EngineeringDesign Courses and to experiment a comprehensive Virtual University of Architecture and Engineering Designin the project WINDS. Here we present the basic concepts of the system, outline the system architecture andshow the possibilities for adaptive learning and instruction in two pedagogical approaches called expository andexploratory learning. The system combines classical structuring of learning materials based on reusable andsharable learning objects with and indexing approach for browsing the learning materials based on a alternativestructure we call the course index. The course index is a collection of important concepts for a course subject ora whole domain.

Keywords

Web based learning, educational hypermedia, interactive learning environment, adaptive systems, e-Learning

Introduction

Web-based Intelligent Design and Tutoring System (WINDS) [1] is an ongoing European project1 with theobjective to implement a learning environment integrating an intelligent tutoring system, a computer instructionmanagement system and a set of cooperative tools. We have developed the current version of the system and 21European university partners prepare more than 20 on-line courses that should be used by students from spring2002.

The main objective of WINDS is to contribute to the reorganization of the pedagogical, cultural, and functionalaspects of design education at the university level. The traditional approach to design teaching shows somefrequent problems that increase learning time and reduce knowledge retention. WINDS will provide supportinstruments for a pedagogically more adequate approach to design teaching. As far as contents are concerned,the system will focus on the problem of cultural integration. The main benefits of WINDS courses are seen as

• general advantages of e-learning, such as worldwide access to learning material and contact to teachers,independent personal working times, well- tested material reused

• open European University for Architecture, Construction Management, and Environmental Engineering

1 Web based intelligent design system (WINDS) is EU funded project in the 5th framework of the IST programme # IST-1999-10253.

- 64 -64

• focus on teaching professional skills (not only textbook knowledge) by means of specialized tools andthe integration of best practices

• support for cooperation among students (peers) as well as between students and teachers

• special support for course authors, such as semiautomatic index creation, and the facilitation ofcreating learning objects with a corporate WINDS style from templates

From a pedagogical point of view the WINDS platform mainly supports:

• the easy creation of flexible learning objects, which can be reused in a variety of learning contexts anddifferent curricula

• the flexible and personalized creation of goal based learning materials and courseware comparable to[2]

• the semiautomatic connection of multiple learning activities concerning design artefacts and dynamiclearning objects (individual learning, cooperative learning)

• the connection of a closed knowledge corpus provided by the teacher, with the open and dynamiclearning resources in the WWW, and a living repository of knowledge from the community of learners.

• the integration of workplace software [3] and meta-cognitive learning tools [4]

• various relations between learning objects

• a highly detailed learner model and history supporting personalized coaching and tutoring activities

For reaching those goals we have taken into account several of the existing standards and semi standards onlearning objects, learner models and their storage and exchange for example [5, 6]. We can build on ourexperience with development of adaptive hypermedia environments like ISIS-Tutor [7], ELM-ART [8] and ACE[2].

ALE produces individualized courseware for the students depending on their current state of knowledge, theirpreferences and learning styles. The author specifies the metadata according to which such adaptation methodslike additional explanation, prerequisite explanation, comparative explanation, explanation variants or sortingcan be implemented, taking into account the user model. In this way, the system can adapt the sequence oflearning objects according to the chosen learning strategy. For instance, a concrete example can precede orcomplement an abstract statement if the student needs it. To reduce the cognitive overload of the learners variousannotation techniques are implemented. As a difference to earlier adaptive learning management systems ALEalso integrates the possibilities of using methods of user modeling and data mining for building expertisenetworks and supporting human resources management in knowledge management for organizations. Highquality expert finding and colearner finding algorithms can build on detailed user and learner models from ALE.

This paper describes an integrated authoring and learning environment implemented in the WINDS project for avirtual university for teaching and learning architecture, construction management, and environmentalengineering. First we introduce the fundamental concepts and learning objects used in the system. Then the mainALE components and some examples of interaction and adaptation are described. Finally we summarize thepaper and outline future perspectives.

Learning Objects

Considering the amount of information that will be collected in the WINDS university storing and collecting thematerial in a reusable way is essential. Based on the analysis of several approaches to formulation of reusablelearning objects and sharable content objects (for example [9]) we defined the WINDS learning objects to fulfilseveral purposes:

• to be the basis for a consistent content generation by the WINDS system

• to enable for a maximum of reusability of learning objects

• to allow for a maximum of flexibility for the dynamic generation of online lessons and courses

Besides being highly reusable and compliant to ongoing standard definitions like SCORM 1.2, IMS, and LOMthe learning objects had to

- 65 -65

• reflect the variety of pedagogical approaches applied by the teachers and authors in the WINDSuniversity

• give the possibility of applying a wide spectrum of methods of adaptive hypermedia like adaptivenavigation and adaptive presentation

The WINDS content management and authoring tool supports authors in building independent learning objectsand structuring them in a default course hierarchy. Nevertheless the underlying knowledge representation andstorage allows for reusing and handling learning objects independently and combining them in new dynamicallygenerated courses later on. This assures that the system is able to produce individualized courseware for studentsdepending on their current state of knowledge, their preferences and learning styles. Based on the currentdefinition of learning objects the system allows for generating adaptive educational hypermedia courses withpersonalized curriculum sequences and personalized content selection on a level of units and pages, and track allinteractions of learners and teachers with those learning objects.

The basic building blocks in the WINDS system should provide the basis for consistent content generation withmaximum flexibility for dynamic generation of online courses. This means the key feature required from thesebuilding blocks is their reusability.

Following these aims four basic types of learning objects are defined in WINDS:

• Course Units are top-level elements that have only subunits but no super-units.

• Learning Units provide the means for course structuring.

• Learning Elements are the basic chunks of information with templates for different pedagogicalpurposes. Specifically we use Paragraphs - the contents of the course, Exercises - the practice tasks,Tests - online assessment. The authors compose each learning element as a sequence of atomic partsthat are reusable basic data components, e.g. text, image, audio, video.

• Index Terms are the fundamental terms of a common glossary for a course.

With course units, learning units, and learning elements course authors can create an arbitrary structuredhierarchy of learning objects which also represent the default learning path for the students. The course authorscan also specify relations between learning objects based on a subset of the Dublin core recommendations [10].These include prerequisite relations, part_of, and related_to relations between course units, learning units, andlearning elements. All these learning objects can be associated with a subset of the Learning Object Metadatadefined in the IEEE LOM specification [5], which is currently reworked to be fully compliant with the SCORM1.2 specification.

- 66 -66

Figure 11: Comparison paragraph in edit mode

Learning elements as the leafs of the course hierarchy are built of single content blocks. A content block caneither be a text segment (ASCII, HTML, other), a URL, or a file that is uploaded to the server. By uploadingfiles authors can easily integrate existing materials like animations, video, images, audio and other binaryformats into learning elements. Figure 1 shows a learning element in editing mode with three content blocks: twopictures as examples and one text content block for describing differences and similarities between the twoimages. Figure 2 shows the same paragraph in view mode like the student will see it.

Every content block has a pedagogical role what gives additional possibilities for reusing and even recompilingdynamic learning elements from content blocks of different pages. By taking into account the pedagogical role ofeach content block also pedagogical patterns for learning elements can be identified. As shown in figure 1 andfigure 2 the author created a page that compares two examples and explains something in an additional contentblock that has the role of fact.

- 67 -67

Figure 12: Comparison paragraph in view mode

Additionally all learning objects in a course unit will be linked dynamically by the underlying index defined bythe course authors. The index reflects a common understanding of the domain and the subject matter. The indexterms are described by a term name, a description, synonyms, and relations to other terms. Figure 3.1 shows anediting form for an index term with several relations to other terms. The available relations are definable by anadministrator of the system and the indices are usable by all authors for an easy import of pre-existing indicesfrom other courses and adaptations to individual definitions. When the author publishes a learning object orpreviews it in view with terms mode the system parses all materials in this learning objects and storesoccurrences of index terms in the database. This allows giving the learner different approaches for workingthrough the learning materials.

Figure 13: Editing an index term

By combining different methods for structuring content in learning objects and by relations to index terms thelearning management system basically the structure of the WINDS learning objects supports two mainpedagogical approaches:

• Expository learning, where the learners are guided though a curriculum and are supported withcoaching components for self directed learning.

• Exploratory learning, where the learners do not follow the default learning path or some adaptedlearning path through a curriculum but explore an alternative structure that is somehow orthogonal tothe learning unit structure.

- 68 -68

Figure 3.2: Browsing the course with index terms

Additionally to learning objects the teachers define and to the index terms that can be developed by the courseauthors cooperatively each course can have a set of external document and web resources connected. This allowsthe authors to integrate online materials into their courses and though give a reference to further materials oronline examples. Like the learning objects of a course all documents are indexed with the index terms of thecourse. In the course player the students can therefore always browse other learning objects and externaldocuments where the current index terms occur.

WINDS Modules

A user of the WINDS system can have one or more of the following three roles assigned:

• Student

• Author (teacher)

• Administrator

Each role has access to one special environment – a student to the learning environment, an author to theauthoring environment and an administrator to the administration environment. Additionally each user canaccess Communities, Tools, News and Help.

Collaboration

The learners are not only passive receivers of knowledge but can themselves contribute to the corpus writingprivate or public annotations, providing feedback to the author, or discussing course related issues withcolleagues and teachers in a discussion forum. In this way students can answer some questions themselves andcan cultivate their ability to formulate problems and give argumentations.

- 69 -69

The main tools provided by the system for collaboration purposes are:

• BSCW folders [11] as a common place to store and exchange documents, support common projects andsubmit homework.

• Discussion forums created by authors for each course.

• Chat room for synchronous communication.

An important feature of the system is the awareness support: the user can see which other students haveregistered for a certain course and even who is currently working in it.

Course Assessment and Homework

Students can be assessed when they solve tests and exercises. Tests are evaluated immediately and automatically.Various types of tests like single- and multiple-choice, picture hotspots, gap fillings and matching pairs areprovided.

More complicated exercises (e.g. design tasks or Bayesian network exercises) are carried out with specialexternal tools, which should be installed locally on the student’s computer or downloaded as an applet. Theresult should be submitted as homework to the student’s folder.

Authoring Environment

The main components of the authoring interface [13, 14] are the navigation tree and the content frame. Thenavigation tree gives an overview of all the courses authored and additionally all courses, learning elements,content blocks, index terms, documents and users in the system can be searched.

A course has three main folders:

• The Units folder includes all the learning units and learning elements forming the structure and contentof the course. The contents can be nested in arbitrary depth and can contain arbitrary learning elements.Authors can provide only text based readings of only exercise or different tests and can link andcombine them in learning units.

• The Index is at the beginning a universal department index prepared in advance and chosen by theauthor who can enhance it. Administrators can create master indices that can be imported into singlecourses. Furthermore the systems allows teachers to export their indices to a standardized XML formatthat enables the system to directly integrate external documents or application in which theses indexterms are used as annotation tags. As one example in WINDS we use 3D VRML worlds with integratedreferences to index terms which allows the students to browse from a specified point in a VRML worldto index terms and to the related learning objects.

• The Documents are supplementary sources of information relevant to the course. Specialised portalswith up-to-date materials can be referenced here. The possibility to view external documents enablesthe system to include highly dynamic information into the courseware. In the further development ofthe system this will allow students and teachers to monitor important sources of information on theWWW and integrate it directly with a standardized curriculum.

The Content Catalog contains all the learning elements created by the author in the system. These materials canbe reused.

- 70 -70

Figure 4: Editing a learning unit in the WINDS authoring environment

In the content frame the following forms are provided and are accessible via different tabs:

• Edit to change the structure and content of the course. The learning objects can be added, moved,copied, disconnected or deleted within the course structure. The author can publish the current learningunit and copy it to a buffer as well as edit the learning elements.

• View to display the learning objects in the same way like students will see them.

• Terms (just for learning elements, not for learning units) to show the content of learning elements or(external) documents with index term entries highlighted and displaying the corresponding termdescriptions on demand.

• Relations to specify prerequisite units and alternative versions in other languages other relations can beadded.

• Meta Data to provide information related to Learning Object Metadata [5] (general, life cycle,technical, educational, rights).

The course index can also be used to view external documents and learning elements with index termshighlighted and hyperlinked to the term’s explanation. Therefore by browsing from learning elements to indexterms and then to external documents, back to index terms and so on, we expect the students to haveindividualized learning paths based on their interests. Nevertheless the system can keep track of all inspectedindex terms, learning objects, and external documents to follow the students learning process.

Cooperative development of a shared understanding

When creating an online course with the authoring tools in ALE the system especially supports the developmentof a shared understanding of the index terms of the current course and of other courses. By using the contentcatalogue the authors can explore and search the complete index terms database. This allows for finding allrelated index term definitions and explore their curricular context and also to contact and discuss relevantdefinitions with other authors. Additionally by previewing external documents with the index terms of a coursehighlighted similarities and differences can be highlighted by the course authors. Beside searching for relatedindex terms the system supports the reuse and search for courses, learning elements, content blocks, and expertswith a wide variety of criteria. This allows teachers to effectively search the content database for all kinds ofobjects and contact points to create their online courses.

- 71 -71

Figure 14: Viewing a learning unit in the WINDS learning environment

Adaptive Learning Environment

The WINDS learning environment is designed to support various learning strategies. Paragraphs containmaterials for expository (explanatory) education. Discovery learning is encouraged not only by hyperlinks butalso by index terms and their interconnection with learning objects and external documents. Collaborationfacilities promote constructivistic learning approaches.

The presented content can be adapted to the current knowledge level of the student on the base of the UserModel and the structure of learning objects. The user model contains for each user all his or her educationalevents, called user episodes. These episodes include events of different types:

• all the user’s actions with the learning objects

• all evaluations of completed tests made by the system

• teacher’s reviews of homework submitted by the student

• self assessment and self estimation of learners

On the base of all these episodes the system can infer conclusions about the user’s knowledge and goals. Theseinferred episodes are stored in the user model as well.

Thus the user model always reflects the current state of the user’s progress. The information is available both forthe teacher to control the student’s study process and for the system to adapt the course presentation andnavigation for the student.

When a user logs in he or she enters Course Registration with the list of all published courses where theregistered courses are distinguished and the ratio of the seen as well as tested study materials are indicated.These bar charts are displayed also when the student works on the course to illustrate the study progress. Thestudent can display also more detailed Course Overview showing the structure of the whole course with thecurrent states of all learning objects - interaction history, tested knowledge, and user’s readiness. This overviewis also permanently available for the student. From the Course Registration page the student can also access a listof all the seen learning elements together with the date of the last access for each of them.

The learning environment in WINDS consists of the following parts:

- 72 -72

Learning Object Display

The main part of the screen is occupied by the content of the current learning object. Its complete path in thecourse tree is displayed together with navigation buttons to support orientation and navigation in the course.

A learning element can be displayed with or without emphasized index terms hyper-linked with theirexplanations. Content blocks in learning elements may be represented by texts, images, hyperlinks, multimedia,or special formats (e.g. Word, PDF, PPT). Some of these content blocks can be displayed as icons activatingadditional window possibly with a note or more detailed information.

Annotated Course Structure

In another frame of the WINDS learning environment the course structure overview is displayed. It supportsstudent’s orientation in the course and helps to choose suitable learning objects for the next study providingannotations for the learning objects.

Learning Object States

The states of the learning objects in WINDS can be considered from several points of view or in severaldimensions. The system distinguishes three of them: interaction history, tested knowledge, and user’s readiness.Additionally some other characteristics can be defined depending on the followed objectives, e.g. current task orcontext. Interaction history and tested knowledge can be expressed by quantitative information defining theextent of the seen learning object or the tested knowledge, user’s readiness is nominal information and dependson the prerequisites specified by the author and on the interaction history of the user.

Annotation Techniques

To represent various states or dimensions of an information space various visual emphasis techniques can beused. Some studies (e. g. [12]) of visual perception recommend the following visual attributes for different kindof information:

• Nominal information:o Textureo Color (Hue)o Position

• Quantitative information:o Lengtho Position

Currently we have a preliminary annotation solution using icon and text annotations. Another possible solutionin the WINDS system is to represent the various states of a learning object in different parts of one icon bydifferent colors. Other applicable annotation techniques include hiding and presentation without hyperlinks.These can be also student’s options.

Course Index

With each learning object corresponding index terms are displayed. Index terms provide means to interrelateheterogeneous course contents and to find individualised paths through the learning materials. The WINDScourse index component maintains the index terms together with their respective descriptions, synonyms,different types of relations (e.g. is_a, part_of, related_to) between terms, their occurrences in the coursematerials as well as in external documents. The student can access all this information about a specific term bychoosing it in the Index frame.

The index component can retrieve and highlight occurrences of index terms within the course materials as wellas within registered external documents (such as web sites).

External Documents

External documents relevant to the course domain explain in more detail some specific issues or provide up-to-date information like specialized portals do. Such external documents serve also as resources for homework and

- 73 -73

projects. These materials can go into more details than the course or give alternative views of the domain. In theDocuments frame the student can choose an external document related to the current learning object and viewthis document either with or without emphasized index terms.

Feedback and Recommendation Area

In the feedback and recommendation area the users can see what their progress is on the current topic and howmuch of the current learning unit they already have successfully worked on. Additionally severalrecommendation strategies will be implemented in the next months which supports learners in recommendationon course navigation (prerequisite warnings, next step recommendations), learning style support (learningmaterial preferences, learning activity selections) and cooperation initialisation (co-learner finding, expertfinding and tutor support).

Coaching Facilities

In ALE a pedagogical agent was implemented to give individualized recommendations to students dependentfrom their knowledge, their interests and their media preferences [15]. The implicitly given teaching goals werepromoting and retaining learner motivation and evoking students’ interest even in lessons they otherwise mighthave avoided.

Beside recommendations of learning material to study the recommendations of the pedagogical agent whereextended to propose cooperative activities and contact to other students that are willing to support others. Table 1shows some type of recommendations given by the agent:

Table 1: Recommendations given by a pedagogical agent in ALE

Student request Agent’s selection and teaching strategy

Anything interesting here? Find a learning material of the current unit, corresponding with student’spreferences. Following this proposal, the student may acquire the current unitby means of a type of learning material that he or she claimed to be especiallyconvenient.

Anything else? Find a learning material of the current unit, to complete knowledge about thisunit. This proposal draws the student’s attention to a learning material of thecurrent unit that has not yet been seen.

What is related? Find a unit related to the current unit and corresponding with student’spreferences. This proposition may attract the student’s interest because it showsthe idea of the current concept in the light of a global domain the student hasclaimed to be especially interested in.

Show me something new! Find a completely new, unrelated, unvisited chapter. A student, who is bored orannoyed by the section he or she is just working at, may like to follow this link.

Contact advanced student! Find a student that declared that he/she is willing to support others and hasexcellent knowledge in the current unit or lesson.

Discuss with otherstudents!

Find students that are willing to cooperate and those have a similar knowledgeof the current unit by comparing the overlay model of the current lesson.

Another feature attached to the agent was the visualization of the learner model. Learners could access a barchart display of the current knowledge model in the system. From the bar chart they could directly jump to testsabout a section to proof that they have better knowledge than assumed by the system. In the experiment donewith ADI and the user model visualization the usage of the agent components were voluntary and the learnersrarely used this feature. Therefore additional evaluations have to be done. The overall results showed thatstudents following the learning material recommendations of the agent gained a better understanding, had morefun, and worked more intense [15].

- 74 -74

Cooperative knowledge collection and adaptive co-learner finding

With the integration of student workspaces into the ALE framework new courses can only consist of a givencurriculum structure prepared by a teacher while the contents and learning materials are completely collected bythe students. This allows teachers to implement basic courseware that is integrated with the growing knowledgeof the learner community and develops towards a living repository of knowledge which includes frequentlyasked questions, best practice solutions, and open space courseware.

Students can ask the assistant for online co-learners and topic experts. Furthermore with every presentation of alearning unit the currently available students that are willing to cooperate are shown to the student. The selectionof appropriate co-learners is based on a comparison of the student’s knowledge model and the other students’knowledge model. Students that are willing to cooperate with others can get a contact point to very similarstudents or to very different students in sense of there curriculum expertise. This can lead to very heterogeneouslearner groups that consist of topic experts in selected areas or to very homogenous learner groups that consist ofexperts on the same topic. The expertise of a student on a certain topic is measured with four main criteria: Thenumber of wrong test answers when working on knowledge tests, the time to answer sufficiently enoughknowledge tests, the collection of additional learning material in student workspaces that has high request ratesby many other students, and the overall score in the curriculum.

The learner groups can work in discussion forums, synchronous moderated chat sessions or communicate viaintegrated email.

Conclusions and Further Research

In this paper we gave an overview of the current state of the ALE system. Its authoring platform supportsreusability of the learning objects. The learning environment presents them as adaptive educational hypermediacourseware. One of the primary objectives of the WINDS project is to make the authoring process relativelysimple and to support sharing of resources. Currently several tens of authors test the usability of the system andwith their comments and suggestions help us to make the system more advanced and reliable.

The potential of such a system is high. The created learning objects can be delivered in a variety of ways, fromclassical web-based training to combinations of classroom events and online seminars, or even as personalizedbooks. Beside the possibility for cross media publishing and the flexible combination of learning objects inindividualized curricula the indexing system and the connection of the learning objects allows for personalizedcoaching of students. In the next few months we will especially extend the coaching and functionality forpedagogical agents supporting the students while learning. Different tutorial and navigation support strategieswill be implemented that allow the teacher to adapt the adaptive method to the target group of his/her course andhis/her pedagogical framework.

For enabling open exploration of contents the system will mainly support on a level of index terms that wherealready seen or explored by a learner and give recommendation on the next steps. In more strictly focusedtraining settings the systems will keep the student close to the default path of a curriculum based on the teacher’sspecifications.

References

[1] WINDS project: http://www.winds-university.org/

[2] Specht, M.; Oppermann, R.: ACE, Adaptive courseware environment, New Review of Hypermedia andMultimedia, 1998/4, p. 141-161.

[3] Nemetschek AG, The O. P. E. N. platform, Technical white paper. 1999, Nemetschek AG: Munich.

[4] Giretti, A. ASA: A Conceptual Design- Support System. Engineering Applications Of ArtificialIntelligence, 1997. 10(1): p. 99- 111

[5] LTSC IEEE: Draft Standard for Learning Object Metadata, IEEE P1484.12/ D6.1, 18 April 2001.

[6] Aviation Industry CBT Committee: AICC Guidelines & Recommendations for CMI,http://www.aicc.org/pages/down-docs-index.htm, AICC.

- 75 -75

[7] Brusilovsky P., Pesin L. "An intelligent learning environment for CDS/ISIS users". Proceedings of theInterdisciplinary Workshop on Complex Learning in Computer Environments (CLCE 94). University ofJoensuu (Finland), May 1994; p.29. http://cs.joensuu.fi/~mtuki/www_clce.270296/Brusilov.html

[8] ELM-ART project. Online at http://www.psychologie.uni-trier.de:8000/projects/ELM/elmart.html

[9] Cisco Systems, Inc. Reusable Learning Object Strategy, Version 3.1. Online athttp://www.cisco.com/warp/public/- 10/wwtraining/elearning/learn/whitepaper_docs/rlo_strategy_v3-1.pdf,22 April 2000.

[10] Dublin Core Metadata Initiative (DCMI) Documents, http://dublincore.org/documents/, DCMI.

[11] Bentley, R., Appelt, W., Busbach, U., Hinrichs, E., Kerr, D., Sikkel, K., Trevor, J. and Woetzel, G. (1997),"Basic support for cooperative work on the World Wide Web", Int. J. Human-Computer Studies, pp. 827-846.

[12] Bertin, J. Semiologie Graphique. Paris: Editions de l’Ecole des Hautes Etudes en Sciences (Lesreimpressions). 1967/1999.

[13] M. Specht, M. Kravcik, L. Pesin, R. Klemke (2001): Authoring Adaptive Educational Hypermedia inWINDS. Proceedings of ABIS2001, Dortmund, Germany, October 8-10, 2001.

[14] M. Specht, M. Kravcik, L. Pesin, R. Klemke (2001): Integrated Authoring Environment for Web BasedCourses in WINDS. Proceedings of ICL2001, Villach, Austria, September 28, 2001.

[15] Schoech, V., M. Specht, et al. (1998). ADI – An Empirical Evaluation of a Pedagogical Agent. WorldConference on Educational Multimedia ED-MEDIA98, Freiburg, Germany.

- 76 -76

- 77 -77

On the Social Rational Mirror’s architecture:

Semantics and pragmatics of educational interactions

Daniele Maraschi, Germana M. Da Nobrega, Stefano A. Cerri

LIRMM, Laboratoire d’Informatique, de Robotique et Micro-électronique de Montpellier

161, rue Ada – 34392 Montpellier – Cedex 2 – France

{maraschi,nobrega,cerri}@lirmm.fr

Abstract

The paper presents the overall architecture developed in two different educational application contexts allowingto manage both semantic and pragmatic aspects of student and teacher interactions with the tutoring system. Amethodology will be exposed along with some other project considerations. All the discussion will be exploitedpresenting in parallel two practical applications.

Introduction

Starting from a general overview of existent tutoring systems having an interface on the Web, we can observethat the methodology used to design them follows mainly the classical approach of Web site development. Wethink this kind of approach does not take really in consideration pedagogical requirements coming from theteaching experience. In this article we describe our approach taking into account innovative technologies todevelop dialogue based tutoring systems on the Web interface. In such a context, we have developed twoapplications deployed in the domain of e-commerce. From the design perspective, both the applications followthe guidelines of our approach, while from the functional objectives, they are essentially complementary. Thecombined solution offers powerful and innovative tools in the new dialogue oriented Web interaction forpedagogical purposes.

The first application considered was developed in the context of a European Project [1] with the main goal toprovide a course to learn the e-commerce vocabulary in the English language by students coming from the Eastof Europe. The ambition of this type of application, as explained in [2], is manifold: allow to capitalise fromdynamically available Web Information, not just from locally generated learning resources; allow as much aspossible Agents to engage both in dialogues with learners and with teachers-developers-experts that wish toenrich, modify, customise etc., on the fly the XML documents generating dialogues with learners; transform thecurrent HTML Web technologies into fully Conversation Oriented, server-based dynamic technologies in orderto multiply the potential use world-wide of produced CALL material.

The second application is a Web-served learning environment, called PhiInEd, to assist both the planning and theexecution phases of a course. The work within PhiInEd is based upon educational theories that privilegeautonomous learning from experience, by trial and error, founding the discipline Discovery Learning. The serverhas been experimented in a student class on Business Contracts. In a complementary paper [10] we report theexperiment. In this paper we introduce PhiInEd through a hypothetical example, and we present the maincomponents of its architecture.

P.R.I.M.A Architecture overview

We have formalised an architecture, called P.R.I.M.A. (Program-centered Reusable Interactive MultimediaArchitecture) describing both the fundamental software components and their organisation. We can identify

- 78 -78

three main temporal phases: a dynamic Information acquisition meant to maintain a continuously updatedinformation repository in order to produce Knowledge; the structuring of Information into Knowledge phasemeant to organise and to provide semantical notations to the Information collected; and the Knowledge reusephase meant to be the preferential source to build learning materials. In all of these phases, other mechanisms areconsidered. For example, since all of them have to provide a user interface in order to interact with the user, theorganisation of such interfaces is described by the Logic, Content and Style (LCS) methodology (Figure 1) thatmodularise a single HTML page into three different files, one structuring the page content, the XML, anotherone structuring the layout on the Web browser, the XSL and the last one structuring the page interaction in Java.The same methodology has already used to develop an XML course starting from the correspondent offlineversion on CD-ROM [3].

Figure 1. The LCS methodology

One of the advantages is to associate in one-to-many relations different components: for example, many Webpage contents described in different XML files can be associated to the same XSL file providing a uniformlayout, or in the other hand, the same content described by an XML file can be displayed in different way on theWeb browser. Another mechanism considers the web interface itself ( in this case the XML/XSL page ) themessage structure of the dialogue between the user and the cognitive agent managing the interaction. This meansthat the Web page have to allow the same expressiveness freedom to the two ways of communication: from userto the agent and viceversa. Moreover, they have to facilitate the pragmatic comprehension by explicitly labellingof such messages.

Basic framework

Since our intention is to provide an easy access to the platform via the Web, we have chosen HTTP as the basicprotocol, but this is not enough since, as stressed in [4], HTTP is a stateless protocol which make difficult totrack the user session and hence to remember the history of the user interaction. So we have developed theplatform using the Java Servlet framework that allows to deploy applications listening on the same socket port ofthe Apache web server and, at the same time, consider a sequence of web interaction like a whole user session.This provides the main advantage to build next Web pages using information coming from the past and not onlyfrom the last page visited, as in the case of many scripting languages. Session tracking is important both from thepedagogical point of view, for example to monitor the student progression, and from dialogue point of viewsince it allows to maintain the partner dialogue knowledge, as justified in [5].

The Servlet framework is also used to implement the LCS methodology explained before allowing to mix theLogic component with the Content and Style ones; this mean, in other words, to program dynamically interactionof an XML and/or an XSL page. The XML programming allows to manipulate dynamically the page contentproposed to the user; for example an XML file structuring an exercise will contain, along with other information,the solutions, but is not really useful to show them immediately to the student. In analogue way, XSLprogramming allows to program the page layout; this can be useful, for example, to adapt the page content to aparticular learning strategy that the system considers more efficient for the student.

- 79 -79

Serendipitous Information acquisition

We state as a precondition that the Course content, view as a set of information, is already available on the Webtaking into account also the advantage of Web dynamics allowing to access to a continuously updatedinformation source. For this reason, we have placed some searching Agents, based on the Madkit agent platform[6], collecting the content of Web pages (Figure 2) coming from serendipitous queries to most common websearch tools, like Google [7] and Altavista [8], and saved in a database.

Figure 2. Serendipitous Information search by Agents

We have placed a filtering parser that transform the HTML page in a XML-like tree structure, called DOM, andselect only the text content nodes discarding those concerning the images and other multimedia types as shownin (Figure 3).

Figure 3. Structure parsing of an HTML page

- 80 -80

Structuring Information into Knowledge

All the information collected in HTML pages stored in the database is essentially not structured. For this reason,a person with a particular role, called the Semantic Author, is charged to attribute semantic notations to theinformation collected. An XML web editor was implemented and presented along with the HTML page previewin order to allow the Semantic Author to do the process manually. The XML web editor, developed in [9],allows to define XML templates describing the component structure of a document; such a structure can bemodified immediately in the case the Semantic Author considers it not semantically appropriated for the HTMLpage content shown on the left-hand side of Figure 4.

For each HTML page a new XML document is generated describing semantically the information. In this way, aconsistent set of XML documents is stored in a different database.

Figure 4. The XML web semantic editor

- 81 -81

Knowledge reuse

Once a significant collection of XML files is set up, it is possible to reuse such Knowledge asking for precisemeaningful document components in order to built the content of a new Web page. In this context, a user playinga different role, called the Course Author, is charged to built a new set of XML documents composing a newCourse content for the domain of e-commerce. Only the resulting XML pages will be used to deploy the finalWeb application along with the layout described by XSL files and the user interaction implemented in Java.

Of course, in the case of changing the goal, correspondingly also the criteria used to structure the new XMLpages necessarily change, but neither the Course Author process nor the Semantic Author process describedabove can be automatically managed by the architecture. This means that human contribution will be alwaysnecessary in order to end successfully the process.

The PhiInEd application

Figure 5: Phases of a course supported by PhiInEd.

Within the server PhiInEd, two phases of a course are currently taken into account, namely, Planning andRunning (Figure 5). Planning a course consists on the elaboration of a Plan by the one who administrates thecourse, to which we refer as the Teacher. Running a course consists on the execution of the Plan, by the oneswho follow the course, to which we refer as the Learners, guided by the Teacher. The result of the work in thePlanning phase, called Plan, is composed of a sequence of Lessons to be studied by Learners. Executing a Planfor the Learners means to study the Lessons from the Plan. We consider the existence of a Plan as a requirementfor a course to be run within PhiInEd. The result of the work in the Running phase is called a ReasoningFramework, that represents knowledge to be used by a machine in order to assist humans to reason in thecorresponding knowledge domain. In a complementary paper [10], we detail each phase, as well as report anexperiment carried out with a class of twenty-seven students in Law.

The server PhiInEd has been designed in such a way to support both the individual work and the communicationbetween participants of a course. Therefore, the main components of the server's architecture are: the Frameworkfor Individual Work (FIW) and the Message Manager (MM). Hereafter we detail each one.

- 82 -82

The Framework for Individual Work

The Framework for Individual Work (FIW) supports, as suggested by its name, the work that each participant ofa course is supposed to perform by himself. For the Teacher, the FIW supports the elaboration of the Plan duringthe Planning phase, while for the Learner, the FIW supports the elaboration of the Reasoning Framework, as aresult of the study of Lessons from the Plan.

Let us explain, by means of a hypothetical example, the rationale underlying individual work. Let us supposethat the subject of study for Learners is the so-called trading conditions. Learners are instructed to play the roleof a lawyer who intends to get assistance from a machine, in order to analyse, verify, and write contracts ontrading conditions. According to the business, they are informed that it is necessary to distinguish, for instance,products from services. Also, products considered dangerous are supposed to be treated differently from thoseconsidered normal, and so on. Let us now suppose that Learners find out that they would need distinguishedassistance, in cases they deal with trading conditions for sellers, or buyers. Maybe buyers of dangerous productsintend to impose to sellers the responsibility of the transport. On the other hand, maybe sellers do not want at alllawyers to consider such a constraint when writing their trading conditions. The example is illustrated in Figure6: the same discourse could be used by lawyers (Learners) to write both sellers' and buyers' trading conditions,but the discourse is constrained differently according to each context of interest (seller or buyer).

Figure 6: (Hypothetical) Trading Conditions in different contexts: buyers and sellers.

According to the example, the resulting Reasoning Framework is composed of: (i) the terms, in a hierarchy,establishing the discourse, (ii) the contexts that are defined by a number of logical constraints among thoseterms, and eventually (iii) the documents that are classified according to contexts, and are supposed to respectthe constraints of their corresponding contexts.

One important feature of a Reasoning Framework is the dialectics between documents and contexts. From theone hand, existing documents may be used as a source of information in order to define contexts. From the otherhand, once they are defined, contexts may “accept" or not arriving documents. By accept we mean that adocument verifies the constraints in a context. Also, one might think in a context as something evolvingcontinuously, possibly due to coming documents. In order to actually assist a human to reason in a domainstarting from the Reasoning Framework, a machine should provide him to build and to revise contexts, as well asto verify/create documents with respect to contexts.

At this point, we are able to present how individual work is actually provided by the FIW. This is accomplishedby means of FIW's three main sub-components, namely, Terms, Folders, and Documents.

The component Terms provides the edition of hierarchies of labels. Functionalities like theaddition/suppression/movement of labels are provided. A label created through this component as a part of ahierarchy, has the same name of the component itself: a Term.

Like the component Terms, the component Folders provides the edition of labels organised as hierarchies.However, a label created through this component as a part of a hierarchy is called a Folder. A Folder represents

- 83 -83

what we have called above a context. The Folder components provide as well the creation of Constraints, aslogical relations among Terms.

Figure 7: Document Description in Terms Hierarchy.

The component Terms provides the verification/creation of a document with respect to the Constraints in aFolder. One must point out what Terms he has observed as Present or Absent in the case of an existingdocument, or what Terms he wants to be Present or Absent, in the case of a document being created. Let usrecall the example about trading conditions. Suppose that the Terms observed as Present in a certain documentare “dangerous" and “Buyers transports". The verification of such a document with respect to the Sellers' contextfrom our example is shown in Figure 7. The two Terms are highlighted because the context “rejects" both Termssimultaneously Present. By reject we mean that the Constraints in the Folder are violated (Figure 8).

Figure 8: Folder representing Sellers' context and its Constraints.

- 84 -84

The list of Terms together with their Present/Absent values is called a Description. In PhiInEd, we areparticularly interested in documents from Web pages. The structure that represents a document in the system iscalled a Document. A Document is then a list composed of a Description, a URL, a Name, and Comments.Name is a string identifying the Document. Comments is also a string. The creation of a Document is providedby the component Folders, since the user must point out in which Folder he wants to classify the Document. Themaintenance of a Document is provided by the component Documents.

Data generated through the FIW component are all stored as a tree in an XML file.

The Message Manager

The Message Manager (MM) is composed of two sub-components, namely, Sender/Receiver and HistoryManager. The Sender/Receiver is responsible for the composition and sending a message from a participant toanother, as well as for retrieving the new messages arriving for a participant. The composition of a messageprovides the attachment of the XML file with the Reasoning Framework. Sending a message causes the storageof message information in a Repository containing all the waiting messages. Whenever a participant asks theretrieval of waiting messages, the pending messages that are addressed to him are removed from the Repository.

The Sender/Receiver communicates with the History Manager, which is responsible for storing/retrieving themessages of a participant. Whenever a message is sent, message information is also stored in the History of thesender. Whenever a message is received by a participant, message information is stored in his History as well,after being removed from the Repository by the Sender/Receiver. The History of a participant is organised as asequence of Dialogues (Figure 9).

Figure 9: Dialogues in a Learner’s History.

- 85 -85

Conclusions

The goal of the paper is to show that enhancing Web technologies we would profit from several opportunitiesoffered in order to:

• use Information available on the Web is a necessary complement to the generation of ad hoc proprietarylearning material;

• transform the resulting learning – authoring process into dynamic and conversational ones, in order tobypass the current limitation of a page-centred client dominated view.

In the first application Information is transformed into Knowledge by the creation of XML files in whichsemantics is organised through XML tags. In the second application Learners use Web pages as a socialinformation source to produce explicitly organised knowledge resulting in a Reasoning Framework. Thecomplementarity of the two applications lies on the fact that in the PhiInEd application the course content issupposed to be achieved in HTML pages composing the Plan. The methodology presented in the first applicationcan be applied in order to compose the content Lessons of a Course.

References

[1] Larflast EU Project. http://www.larflast.bas.bg/

[1] Cerri, S. A.; Dikareva, S.; Maraschi, D.; Trausan-Matu, S.; "Web Server Based Architectures for Language Learning:LARFLAST Agents generating CALL Dialogues". Conference on Computer Assisted Language Learning - TheChallenge of Change, University of Exeter, September 1- 3, 2001.

[2] Maraschi, D., Cerri, S.A., Martinengo, G., “CD-ROM -> Web-Generated : reverse engineering of an interactiveMultimedia Course”. IEEE Int. Conf. On Advanced Learning Technologies (ICALT). Okamoto, T., Hartley, R., Kinsukand Klus, J. P. (eds.). Madison, Wisconsin, USA. 6-8 August 2001. pag. 203-204.

[2] Kinsuk, Han, B., Hong Hong and Patel, A., “Student Adaptivity in TILE: A Client-Server Approach”. IEEE Int. Conf.On Advanced Learning Technologies (ICALT). Okamoto, T., Hartley, R., Kinsuk and Klus, J. P. (eds.). Madison,Wisconsin, USA. 6-8 August 2001. pag. 297-300.

[3] Cerri S. A., “Shifting the focus from control to communication: the STReams OBjects Environments model ofcommunicating agents", Collaboration between human and artificial societies, vol. 1624 in Lecture Notes in ArtificialIntelligence, J. Padget, Ed., Springer-Verlag, Berlin Heidelberg, New York, pag. 71-101, 1999.

[3] Gutknecht O., J. Ferber, "Madkit : A generic multi-agent platform", AGENTS'00 : 4th Int. Conf. on AutonomousAgents, Barcelona, Spain, June 2000; ACM Press, pag. 78-79, 2000.

[4] Google web search: http://www.google.com/

[5] Altavista web search. http://www.altavista.com/

[6] Wittman P., "A Methodology for porting traditional CBT Courses on the Web", Master Thesis, Univ. of Montpellier II,Montpellier, France, 2002.

[7] G. M. da Nóbrega, S. A. Cerri, and J. Sallantin. On the social rational mirror: learning e-commerce in a web-servedlearning environment. In Intelligent Tutoring Systems 6th International Conference, ITS 2002 Biarritz, France, June2002. (submitted).