target authoring tools

TUG-KMI

Authoring System in TARGETwww.reachyourtarget.org

Georg Ottl

Knowledge Management InstituteCognitive Science Section

April 29, 2010

Georg Ottl April 29, 2010 /37

TUG-KMI

Outline

Research environment

Competence performance assessment

Experts competence structure modeler

Probabilistic graphical modelsFactor graphs


TUG-KMI

Transformative, Adaptive, Responsive and enGagingEnvironmenT (TARGET)

I Serious game based learningenvironment

I Enterprise CompetenceDevelopment

I Improve competences in theproject management andinnovation domain


TUG-KMI

Five key concepts of TARGET


TUG-KMI

TARGET Learning Process


TUG-KMI

Role of TUG-KMI in TARGET

I TUG-KMI responsible for TARGET learning processI TUG-KMI responsible for workpackage competence

developmentI Competence performance assessment componentI Story adaptation/interventions

I Integration competence development/TARGET learningprocess


TUG-KMI

Competence performance assessment mockup


TUG-KMI

Outline






TUG-KMI


CompetencesProblems Competence

Assessment


TUG-KMI

TARGET competence performance assessment

I Interpret observable performance in game experiences inregards to a competence state1.

I Include motivational state emotional state in interpretation

I Competence assessment as basis for macro andmicroadaptive2 interventions and adaptations.

I Computational model to automatically assess competencestate.

1Klaus Korossy. “Modeling Knowledge as Competence and Performance”.In: Knowledge Spaces: Theories, Empirical Research, Applications. Ed. byDietrich Albert and Josef Lukas. Mahwah, NJ: Lawrence Erlbaum Associates,1999, pp. 103–132.

2Dietrich Albert et al. “Microadaptivity within Complex Learning Situations- a Personalized Approach based on Competence Structures and ProblemSpaces”. In: Proceedings of the international Conference on Computers inEducation (ICCE 2007). 2007.


TUG-KMI

TARGET competence performance assessment modelauthoring

I Interpretation of the game experiences in terms ofcompetences can be done by a social community throughinspection.

I Creation of a model by using the social communityobservations input (cold start problem)

I Experts create a model to automatically interpret performance


TUG-KMI

TARGET competence performance assessmentrequirements

I Knowledge model/competence state exchange with HRMSystems such as SAP.

I Assessment in realtime3 to enable microadaptive interventions.

3O. Conlan et al. Realtime Knowledge Space Skill Assessment forPersonalized Digital Educational Games. IEEE, 2009, pp. 538–542.


TUG-KMI

Basic principle of probabilistic assessment of thecompetence state

1. If the learner has the competence ci , than increase thelikelihood of all competence states γci containing ci anddecrease the likelihood of all competence states γ 6 ci .

2. If the learner does not have the competence ci , than decreasethe likelihood of all competence states γci containing ci andincrease the likelihood of all competence states γ 6 ci .


TUG-KMI

Assessment calculation complexity reduction

I No structure. Possibly 2n states to be updated on everyperformance observation

I Definition of a partial order relation on competencesexploiting the properties of the “PrerequesiteOf” relation typereduces amount of possible competence states to be takeninto consideration.

I Can experts or the community directly create a competenceassessment model??

Authoring tools can help to create a model for competenceassessment.


TUG-KMI

Mathematical and computational model for competenceassessment

I Nondeterministic assessment4

I Traditional, multiplicative update rule56

I Belief propagation networks such as Bayesian Networks7

4C. Hockemeyer. “A Comparison of non-deterministic procedures for theadaptive assessment of knowledge”. In: Psychologische Beitrage 44.4 (2002),pp. 495–503.

5Jean-Claude Falmagne and Jean-Paul Doignon. “A class of stochasticprocedures for the assessment of knowledge”. In: British Journal ofMathematical and Statistical Psychology 41 (1988), pp. 1–23.

6Jean-Claude Falmagne and Jean-Paul Doignon. “A markovian procedurefor assessing the state of a system”. In: Journal of Mathematical Psychology32.3 (1988), pp. 232–258.

7M. Villano. “Probabilistic Student Models: Bayesian Belief Networks andKnowledge Space Theory”. In: Proceedings of the Second InternationalConference on Intelligent Tutoring Systems. Springer, 1992, 491–498.


TUG-KMI

Outline






TUG-KMI

Current state Competence Modeler

I Support to create competence assessment models

I Using the well studied PrerequesiteOf relation8

I Support experts (psychologists) to create knowledgestructures.

8Dietrich Albert et al. Knowledge Structures. Ed. by Dietrich Albert. NewYork: Springer Verlag, 1994.


TUG-KMI

Hasse diagram visualization

Figure: Popular knowledge space visualizations


TUG-KMI

Current state Competence ModelerProblems solved and Related Problems

I Computer supported Hasse diagramm creation

I Visualizations done with the Java Universal Network/GraphFramework (JUNG) framework9

9J. Madadhain et al. “Analysis and visualization of network data usingJUNG”. In: Journal of Statistical Software 10 (2005), pp. 1–35.


TUG-KMI

Hasse diagram visualization

→

Figure: Version 0.13 and 0.16


TUG-KMI


TUG-KMI

Current state Competence ModelerProblems solved and Related Problems

I Creation of a Hasse diagram reduced to the problem ofcalculating the minimal transitive reduction of a graph whichwas shown to have the same complexity as calculation of thetransitive closure of a graph10.

I Effective calculation and detection of cycles by maintainingthe online topological order of the graph11

I Visualizations done with the JUNG12

10A. V. Aho, M. R. Garey, and J. D. Ullman. “The Transitive Reduction of aDirected Graph”. In: SIAM Journal on Computing 1.2 (1972), pp. 131–137.

11David J. Pearce and Paul H. J. Kelly. “A dynamic topological sortalgorithm for directed acyclic graphs”. In: J. Exp. Algorithmics 11 (2006),p. 1.7.

12J. Madadhain et al. “Analysis and visualization of network data usingJUNG”. In: Journal of Statistical Software 10 (2005), pp. 1–35.


TUG-KMI

Current State Competence ModelerOn-Line Demo Afternoon

I https://dev-css.tu-graz.ac.at/


https://dev-css.tu-graz.ac.at/

TUG-KMI

Outline






TUG-KMI

Probabilistic Graphical ModelsWhatfor?

I Simple way to visualize the structure of a probabilistic modelI Graphical representation allows insights into the properties of

the modelI Insights into conditional independence properties

I Complex computations can be expressed in terms of graphicalrepresentations; use of graph based inference algorithms thatexploit graph properties for calculation.


TUG-KMI

Graph Terminology

I A graph comprises verticesV = (a, b, c , d) connectedby edges

DefinitionA graph G is a pair G = (V ,E ), where V is a (finite) set ofvertices and E ⊆ V × V is a (finite) set of edges.


TUG-KMI

DefinitionA graph G is called undirected iff

∀A,B ∈ V : (A,B) ∈ E ⇒ (B,A) ∈ E (1)

Two ordered pairs (A,B) and (B,A) are identified and representedby only one undirected edge.

DefinitionA graph G is called directed iff

∀A,B ∈ V : (A,B) ∈ E ⇒ (B,A) 6∈ E (2)

An edge (A,B) considered to be a directed edge from A towardsB


TUG-KMI

Visual Representation Graph Models

Figure: Undirected GraphFigure: Directed Graph


TUG-KMI

Probabilistic Graphical Models (1/2)Whatfor?

I In a probabilistic graph model every vertice represents arandom variable

I The edges express probabilistic relationships between thevariables

I Directed Graphical probabilistic ModelsI Bayesian Networks

I Undirected Graphical Probabilistic ModelsI Markov Random FieldsI Loose coupling between statistical variables.


TUG-KMI

Graphical probabilistic models

I Question: Can directed probabilistic models such as asBayesian networks be used for assessment. How does believepropagation relate to the classical update rule?

I Question: How is the relation between directed andundirected probabilistic graphical models?

I Use of directed and undirected graphical probabilistic modelsto assess the players state

I Efficient sum and dot product calculation.


TUG-KMI

A flexible probabilistic graphical model, the Factor Graph

I Factor Graphs13 as a single representation for directed andundirected graphical probabilistic models

I Factor Graphs were successfully applied for Bayesian Networksand Markovian Models

I Multiple applications in artificial intelligence and signalprocessing based on Factor Graphs

13Frank Kschischang et al. “Factor Graphs and the Sum-Product Algorithm”.In: IEEE Transactions on Information Theory 47 (2001), pp. 498–519.


TUG-KMI

Factor graph conversion (1/3)

Example 1:(A simple probabilistic graph)

Let f (S1,S2,S3) be a function of threevariables, and suppose that f can beexpressed as a productf (S1,S2, S3) = p(S1)p(S2)p(S3|S1, S2)

S1

S3

S2


TUG-KMI


Example 1:(A factor graph)

Let f (S1,S2,S3) be a function of threevariables, and suppose that f can beexpressed as a productf (S1,S2, S3) = p(S1)p(S2)p(S3|S1, S2)

S1

f

S3

S2


TUG-KMI


I Efficient algorithms available to calculate probabilities(Sum-Product algorithm)

I Makes extensive use of “conditional independent” propertiesI Parallelization possible

I Approximative algorithms

I Efficient marginalization


TUG-KMI

Thank You!


TUG-KMI

[1] A. V. Aho, M. R. Garey, and J. D. Ullman. “The TransitiveReduction of a Directed Graph”. In: SIAM Journal onComputing 1.2 (1972), pp. 131–137.

[2] Dietrich Albert et al. Knowledge Structures. Ed. byDietrich Albert. New York: Springer Verlag, 1994.

[3] Dietrich Albert et al. “Microadaptivity within ComplexLearning Situations - a Personalized Approach based onCompetence Structures and Problem Spaces”. In:Proceedings of the international Conference on Computers inEducation (ICCE 2007). 2007.

[4] O. Conlan et al. Realtime Knowledge Space Skill Assessmentfor Personalized Digital Educational Games. IEEE, 2009,pp. 538–542.

[5] Jean-Claude Falmagne and Jean-Paul Doignon. “A class ofstochastic procedures for the assessment of knowledge”. In:

Georg Ottl April 29, 2010 Page 36/37

TUG-KMI

British Journal of Mathematical and Statistical Psychology41 (1988), pp. 1–23.

[6] Jean-Claude Falmagne and Jean-Paul Doignon. “Amarkovian procedure for assessing the state of a system”. In:Journal of Mathematical Psychology 32.3 (1988),pp. 232–258.

[7] C. Hockemeyer. “A Comparison of non-deterministicprocedures for the adaptive assessment of knowledge”. In:Psychologische Beitrage 44.4 (2002), pp. 495–503.

[8] Klaus Korossy. “Modeling Knowledge as Competence andPerformance”. In: Knowledge Spaces: Theories, EmpiricalResearch, Applications. Ed. by Dietrich Albert andJosef Lukas. Mahwah, NJ: Lawrence Erlbaum Associates,1999, pp. 103–132.


TUG-KMI

[9] Frank Kschischang et al. “Factor Graphs and theSum-Product Algorithm”. In: IEEE Transactions onInformation Theory 47 (2001), pp. 498–519.

[10] J. Madadhain et al. “Analysis and visualization of networkdata using JUNG”. In: Journal of Statistical Software 10(2005), pp. 1–35.

[11] David J. Pearce and Paul H. J. Kelly. “A dynamictopological sort algorithm for directed acyclic graphs”. In: J.Exp. Algorithmics 11 (2006), p. 1.7.

[12] M. Villano. “Probabilistic Student Models: Bayesian BeliefNetworks and Knowledge Space Theory”. In: Proceedings ofthe Second International Conference on Intelligent TutoringSystems. Springer, 1992, 491–498.


TUG-KMI

Acronyms

JUNG Java Universal Network/Graph Framework


target authoring tools

Documents

tugkmi role of tug

performance georg ottlapril

tugkmi transformative

target tugkmi responsible

competence state1

competence structures

tugkmiauthoring system

knowledge spaces