inted praesentation

Knowledge Management Institute

1

Georg Öttl Valencia, 08/03/2010 NON-INVASIVE DATA TRACKING IN EDUCATIONAL GAMES

Non-invasive data tracking in educational

games

Combination of Logfiles and Natural

Language Processing

Stephanie B. Linek, Georg Öttl, Dietrich Albert


2


Outline

• Introduction

• Core factors of enjoyment in educational games

• Available data sources

• Open framework for a combined assessment

methodology


3


Introduction

• Genre: adaptive educational games• Transformative, Adaptive, Responsive and enGaging EnvironmenT

• Joy as essential factor in game based learning• Do not destroy the overall game-play experience

• Address player’s needs

• Non-invasive user monitoring and data-tracking


4


Core factors for enjoyment in games and

educational games

• Address the player’s need for reasonable parasocial

interaction

• Address the player’s individual preferences and

actual mood

• Address players capabilities


5


Reasonable parasocial interaction

• Communication preferences are reflected by the

selected game genres

• Provide options to address different communication

preferences for a player

• Adaption to the communication preferences of the

player


6


Individual preferences and actual mood

• Mood management• Media experiences are used for regulation of one’s own mood [1][2]

• Mood monitoring• Communication preferences provide evidence about mood

• Identification of player specific mood preferences

[1] Zillmann, D. (1988). Mood management through communication choices. American Behavioral Scientist, 31, 327-340.

[2] Zillmann, D. (2006). Dramaturgy for emotions from fictional narration. In J. Bryant & P. Vorderer (Eds.), Psychology of

entertainment (215-238). Mahwah, New Jersey: Lawrence Erlbaum Associates.


7


Players capabilities

• Adapt to the players skills

• Adapt to the players progress during time

• Ongoing monitoring of the skills and cognitive

capabilities


8


Data tracking in educational games

• Questionnaires• Evaluation of the overall quality of the game

• Presented before and after the game

• Online behavior monitoring without interrupting the

ongoing learning process• Logfiles

• Natural language


9


Data tracking by logfiles

• Automatic protocol of actions in a computer based

environment

• Huge amount of behavioral information


10


Defining associated logfile parameters

Logfile-Data / Probs (Potential) Meaning/interpretation

Mouse-clicking rate (number of clicks/time unit) Activity of the user: confusion/nervousness vs.

ambition

Frequency of tool-usage (of the different available

tools)

Activity and direction/purpose of user-behavior

Frequency help-seeking behaviour Confusion / overload of the learner

Sequencing of material (in relation to the task) Approximation to the solution vs. disorientation

Interaction frequency/rate with NPC or another

player

Affiliation/Intensity of parasocial interaction with an

NPC / another player: positive (sympathy) vs

negative (aversion)

Length of Interaction: Time spent on interaction with

an NPC or another player



negative (aversion)

Eye-contact between player’ avatar and the

NPC/another player’s avatar



negative (aversion)

Number of characters (NPCs and/or other player’s

avatars) the player is interacting with

Integration in the (virtual) community/group

(Integration-Index)


11


Pros and Cons of Logfiles

Pros

•Objective measurement of

user behavior

•Does not disrupt gameplay

•Non reactive method

without demand effects

Cons

•Too much data /

information overload

•Ambiguous data without

subjective meaning – hard

to interpret


12


Data tracking in Natural Language

• Natural language used in • Team speak

• Multiparty chat conversations

• Private 1:1 chats

• Connection between the words an individual uses and mind

[3][4] [5]• Attempt to algorithmically understand the subjective meaning of text with

Natural Language Processing (NLP)

[3] Jackendoff, R. S. (1985). Semantics and Cognition (Current Studies in Linguistics) (p. 304). The MIT

Press.

[4] Noam Chomsky. (1969). Aspects of the theory of syntax. MIT Press.

[5] Schank, R. C. (1972). Conceptual Dependency: A Theory of Natural Language Understanding. Cognitive

Psychology, 631(3), 552-631.


13


Defining associated NLP parameters

• Identify locations, persons, organizations, …

• Identify the topic

• Identify fascinating, acceptable and unsuitable answers [6]

• Interpret the semantic orientation [7]

• Identify gender, deception status and culture [8]

[6] Huang, J., Zhou, M., & Yang, D. (2007). Extracting chatbot knowledge from online discussion forums. In

Proc. of IJCAI (pp. 423-428).

[7] Hatzivassiloglou, V., & McKeown, K. R. (1997). Predicting the semantic orientation of adjectives.

Proceedings of the 35th annual meeting on Association for Computational Linguistics - (pp. 174-181).

Morristown, NJ, USA: Association for Computational Linguistics.

[8] Chung, C. K., & Pennebaker, J. W. (2007). The psychological functions of function words. Social

communication: Frontiers of social psychology, 343-359.


14


Pros and Cons of NLP

Pros

•Works autonomous

•Provides different

qualitative information than

logfiles

Cons

•Error prone

•Ambiguous outcomes

•Mostly relies on supervised

machine learning


15


Combination of logfile analysis and NLP

• Logfiles• Quantitative objective data regarding the activity of the user

• NLP• Qualitative information


16


Multimodal classification using logfile and NLP

data

• Analyse common properties of both data sources

• Augment necessary additional information

• Combination of both• Multimodal classification task

• Indicator selection based on empirical psychological studies


17


Open framework for a combined assessment

methodology

Defining associations between NLP

generated information and potential

subjective meaning

Pool of interesting psychological variables

subjective meaning

Behavioural logfile

parameterNLP indicators

Defining rules for multimodal

classification: Analysis of common

properties and augmenting information

Defining associations between logfile

parameters and potential subjective

meaning

Theoretical and empirical research

as basis for the definition of appropriate indicators, variables, associations and rules


18


Conclusion

• Multimodal classification of logfiles and NLP will lead

to a more holistic view of the user/player

• Future research on games and game-based learning

might lead to new insight on the association of user-

behaviour and subjective meaning

• A open framework in the sense that other additional

data sources could be incorporated and connected


19


OUTLOOK

TARGET : www.reachyourtarget.org

http://www.reachyourtarget.org/


20


Thank You For Your Attention!

Questions?

Stephanie Linek: [email protected]

Georg Öttl: [email protected]

mailto:[email protected]





21


References

[1] Zillmann, D. (1988). Mood management through communication choices. American Behavioral Scientist, 31, 327-340.

[2] Zillmann, D. (2006). Dramaturgy for emotions from fictional narration. In J. Bryant & P. Vorderer (Eds.), Psychology of entertainment (215-238). Mahwah, New Jersey: Lawrence Erlbaum Associates.

[3] Jackendoff, R. S. (1985). Semantics and Cognition (Current Studies in Linguistics) (p. 304). The MIT Press.

[4] Noam Chomsky. (1969). Aspects of the theory of syntax. MIT Press.

[5] Schank, R. C. (1972). Conceptual Dependency: A Theory of Natural Language Understanding. Cognitive Psychology, 631(3), 552-631.

[6] Huang, J., Zhou, M., & Yang, D. (2007). Extracting chatbot knowledge from online discussion forums. In Proc. of IJCAI (pp. 423-428).

[7] Hatzivassiloglou, V., & McKeown, K. R. (1997). Predicting the semantic orientation of adjectives. Proceedings of the 35th annual meeting on Association for Computational Linguistics - (pp. 174-181). Morristown, NJ, USA: Association for Computational Linguistics. doi: 10.3115/976909.979640.

[8] Chung, C. K., & Pennebaker, J. W. (2007). The psychological functions of function words. Social communication: Frontiers of social psychology, 343-359.


22


Defining the interesting psychological

variables

• Subjective psychological indicators • Related to meaningful parasocial interaction

• Related to mood management

• Related to the adaptivity on the learner characteristics

• General Psychological variables• Curiousness of the player

• Confusion

• Positive versus negative mood

• Cognitive resources

• Objective behavioural correlates have to be found

inted praesentation

Documents

educational games4

educational games6

educational games8

player georg ttlvalencia

noninvasive user monitoring

players individual preferences

players skills

experience address players