quality estimation of adaptive tutoring systems volgograd state technical university cad department...
TRANSCRIPT
Quality estimation of adaptive tutoring systems
Volgograd State Technical UniversityCAD departmentPavel Vorobkalov
Faculty advisor Shabalina O.A.
Volgograd 2007
2
The aim and tasks
The aim:Quality management of learning process using adaptive tutoring systems
Tasks:1. To analyze problems of quality of adaptive tutoring systems2. To develop method of quality estimation of adaptive tutoring systems3. To design model of adaptive learning process4. To develop criteria of quality estimation of adaptive learning5. To implement computer-aided system of quality estimation of adaptive
tutoring systems
3
Quality management system of adaptive tutoring systems
Developer
Development Framework
Quality estimation system
Users
Adaptive tutoring systems
Users’Knowledge
User requirments
Users’ Knowledge
Criteria Values
4
Analysis of approaches and methods of quality estimation of adaptive tutoring systems
‘as a whole’ approach Questionnaires Return of investments as a result of
education Common quality criteria
(ex., according to ISO-9127) Standard Conformance
‘layered’ approach Estimation of interface layer Estimation of adaptation models Estimation of adaptation decisions Estimation of adaptation techniques Estimation of leaning content
To interpret results for further development
To generalize results
To use both approaches
5
Model of human-computer interaction during adaptive learning process in layered approach
1 layer 2 layer
Adaptation mechanism
Building adaptation
models
System adaptation
Adaptive applicationData
collection mechanism
Hi-level conclusions:· user is confused;· user couldn’t complete the task.
Low-level user information:· data from input devices;· information about task completion;· quiz answers.
Adaptation decisions:· popup help window;· changing hyperspace.
Adaptation process
Representation as a «white box» needed
?
?
6
Method of estimation and prediction of quality of adaptive tutoring system
1. Collecting data about learning process using adaptive tutoring system
2. Building model of learning process3. Analysis of built model
1. Computation of criteria of quality estimation of learning process2. Expert estimation of learning process (detecting issues)
4. Learning course modification1. Modification of concepts those are problematic2. Testing changed concepts by student groups3. Calculating new parameters of learning process model4. Computation of predicted results of changed learning process (quality
prediction)
7
- 1 student strategy
- 2 student strategy
Table of symbols:
Ci- unitConsists of learning content and a quiz
Cj- sectionSuperset of some units
C1
Domain area structure
C2 C3 C4 C5
C6
C7
C8
C9
C10
C11
C16
C12
C13 C14 C15 C17
C18 C19 C20
- learning dependency
- union
Adaptive learning process
Choosing learning strategy
Generation of page
Learning course development
Observing student
Student
Developer
8
Модели процесса обучения
Model Major advantages and disadvantages
Psychological models, models based on cognition theory
+Well known–No formal method of building model
Grammar-based models (BNF, EBNF) +Advanced approach–Difficult to estimate
Models based on algebraic and differential equations
+Well formalized–High level of abstraction
Machine models (Mur, Mille machines) +Wide range of described tasks–Hard to interpret
Net models (Markov chains, Petri nets) +Easy to interpret–Big, hard to build
1 2
21
2
1
Concept 1 Concept 2
Concept 3
Concept 4
1 2
1 2
2
Quiz 1 Quiz 2
Quiz 3
1
1 1
2
22
(Identity, Kind) = (1, ”Learned”)
(Identity, Kind) = (2, ”Learned”)
Concept 5
9
1 2Concept 1 Concept 2
Concept 3
Concept 4
1 2
1 2
Quiz 1 Quiz 2
Quiz 3
1
1 1
2
22
(Identity, Kind) = (1, ”Current”)
(Identity, Kind) = (2, ”Learned”)
Concept 5
Learning process model based on Petri nets
Set of colors:
kcccC ,,, 21
,
where
ic
– color of a token, an arc;
k
– count of Petri net colors.
KindIdentityci ,
,
where
Identity
– color component that identifies a
student’s role;
Kind
– color component that defines kind of token.
Kindlk
,
where
l
– count of roles of students that collaborate
during learning process using adaptive tutoring
system;
Kind
– count of different kinds of token.
.
.
10
Quantitative parameters of learning process
To random values correspond to each transition:
firing time of transition (learning duration in seconds)
0, DZD
;
delta of knowledge level
1,0, KLKLRKL
.
Probability distribution of 2D random value is:
,
where – probability distribution of 2D
lognormal random value , , and , – mean and standard
deviation values of
X
and
Y
, accordingly, and – correlation factor between
X
and
Y
.
Lognormal distribution of 1D random value
X with mean and standard
deviation
1.0X
.
XXxf ,,
x
Concept 1
Concept 2
Test 1
1
1 1
(D, KL)
1
1
As a quantitative parameters learning duration D and knowledge level KL have been chosen
Knowledge level KL1
Knowledge level KL2=KL1+KL
11
Building the learning process model
p2 p3
p4
Built learning process model
p1
t1 t2
t4
p2 p3
p4
p1
t1 t2
t4
p3
p4
p1
t1
t4
Studying strategy 1Use frequency 0.6
Studying strategy 2Use frequency 0.4
The algorithm
Collecting data about student’s actions
Preprocessing the data
Building Petri net model using join operation
User knowledge level for a unitLearning duration for a unit
Learning process model
0
00
Probability 0,6
Probability 0,4
12
Quantitative parameters of Petri net inferring learning process
Learning durationComputer experience
Correlation between net parameters and user characteristics
C fan
C density
C flex
p2 p3
p4
p1
t1 t2
t3
t4
Quantitative parameters of Petri net:
P – number of placesT – number of transitionsR – number of routes
Difficulty
Density
Flexibility
P
TC fan
1
PP
TCdensity
P
RC flex
17
,16
7fa
nC
23,0166
7
densityC
5,06
3flexC
p5
t5
t6
p6
t7
1
13
Criterion of adaptive decisions balance of tutoring system
Probability h11
Probability h21
Computing of balance criterion B
where i – number of adaptation strategy;
Hi – relative frequency of the strategy;
N – number of possible strategies.
where hij – probability of variant on the route corresponding to strategy i
,logi
iNi HHB
p2 p3
p4
p1
t1 t2
t4
0
ijj
i hH ,
Адаптацияне сбалансированаАдаптацияне сбалансирована
14
Expert estimation of learning process
C2
C3
C4
C1
t1 t2
t3
C5
t4
t5
C6
t6
KL(t1)D(t1)
KL(t2)D(t2)
KL(t3)D(t3)
1
KL(t6)D(t6)
KL(t5)D(t5)
KL(t4)D(t4)
Ra
tio
n i
s a
sc
en
din
g
KL(t3)D(t3)
=0,54311
KL(t1)D(t1)
= 0,25155
KL(t2)D(t2)
=0,71271
KL(t4)D(t4)
=0,80275
KL(t6)D(t6)
=0,82273
KL(t5)D(t5)
= 0,85255
Changes are required
Learning results - ?
KL(t1) D(t1),t1:
KL(t3)D(t3),t3:
15
Экран «Редактирование модели»
16
Экран «Запуск моделирования»
17
Экран «Результаты моделирования»
18
Conclusions
1. Problems of quality of adaptive tutoring systems and approaches to their solution have been analyzed
2. Method of quality estimation of adaptive tutoring systems has been developed
3. Model of adaptive learning process has been designed4. Criteria of quality estimation of adaptive learning have been created5. The prototype of computer-aided system of quality estimation of
adaptive learning has been implemented