learning analytics and serious games: trends and ... · game technology & concepts ... •...

21
© author(s) of these slides including research results from the KOM research network and TU Darmstadt; otherwise it is specified at the respective slide 7-Nov-14 Prof. Dr.-Ing. Ralf Steinmetz KOM - Multimedia Communications Lab ACM Workshop on Serious Games, Orlando, 2014 Learning Analytics and Serious Games: Trends and Considerations ACM Multimedia Serious Games Workshop Nov 7, 2014 Laila Shoukry, M.Sc. Dr. Stefan Göbel [email protected] [email protected]

Upload: dohuong

Post on 26-Apr-2018

221 views

Category:

Documents


6 download

TRANSCRIPT

Page 1: Learning Analytics and Serious Games: Trends and ... · Game technology & concepts ... • Description & model for Serious Games ... Intelligence in Education, 17(2):121-144, 2007

© author(s) of these slides including research results from the KOM research network and TU Darmstadt; otherwise it is specified at the respective slide

7-Nov-14

Prof. Dr.-Ing. Ralf Steinmetz

KOM - Multimedia Communications Lab

ACM Workshop on Serious Games, Orlando, 2014

Learning Analytics

and Serious Games:

Trends and Considerations

ACM Multimedia Serious Games Workshop Nov 7, 2014

Laila Shoukry, M.Sc. Dr. Stefan Göbel

[email protected] [email protected]

Page 2: Learning Analytics and Serious Games: Trends and ... · Game technology & concepts ... • Description & model for Serious Games ... Intelligence in Education, 17(2):121-144, 2007

KOM – Multimedia Communications Lab 2

Serious Games – Team

Stefan Krepp (1.9.14)

Sabrina Radke

Robert Konrad

Christian Reuter, Florian Mehm, Michael Gutjahr, Sandro Hardy, Tim Dutz, Laila Shoukry, Stefan Göbel

Martin Knöll Interdisciplinary research area UNICO, architecture „Serious Games“

Urban Health Games since 2011

15 groups EXIST uniworlds

Page 3: Learning Analytics and Serious Games: Trends and ... · Game technology & concepts ... • Description & model for Serious Games ... Intelligence in Education, 17(2):121-144, 2007

KOM – Multimedia Communications Lab 3

Approach Game technology & concepts

+ further RTD concepts application areas

Characteristics

Real data & real users

Complex, interdisciplinary

Fun & Characterizing Goal

Personalization & adaptation

Authoring, control & evaluation

Serious Games – Games more than fun

SG, KWT 2014

Page 4: Learning Analytics and Serious Games: Trends and ... · Game technology & concepts ... • Description & model for Serious Games ... Intelligence in Education, 17(2):121-144, 2007

KOM – Multimedia Communications Lab 4

Serious Games – Research Field

Overall Aim: Maximise effects & fun

characterizing goal (health..)

user / game experience

Serious Game

• Game state, game world • Game Design, gameplay

• Single / Multiplayer • Offline / Online / Mobile

State Monitoring

• (mobile) sensing • Context awareness

• Player state & behaviour • Psychophysiologic data

Knowledge Base

• Description & model for Serious Games • Game patterns & interaction templates

• User profile, player / learner model • (dynamic) Game Data, e.g. vital data

• (domain) knowledge, situation/adaption base

Adaptation

• Adaptive control • Adaptive gameplay • Difficulty adaptation • Procedural Content

Adaptive

Serious Games

Page 5: Learning Analytics and Serious Games: Trends and ... · Game technology & concepts ... • Description & model for Serious Games ... Intelligence in Education, 17(2):121-144, 2007

KOM – Multimedia Communications Lab 5

Outline ‚Learning Analytics & Serious Games‘

Page 6: Learning Analytics and Serious Games: Trends and ... · Game technology & concepts ... • Description & model for Serious Games ... Intelligence in Education, 17(2):121-144, 2007

KOM – Multimedia Communications Lab 6

Definition – What is Learning Analytics ?

http://edtechreview.in/event/87-webinar/835-can-learning-analytics-enable-personalized-learning

“Learning analytics is the measurement, collection,

analysis and reporting of data about learners and

their contexts, for purposes of understanding and

optimizing learning and the environments in which

it occurs.” George Siemens 2011

Page 7: Learning Analytics and Serious Games: Trends and ... · Game technology & concepts ... • Description & model for Serious Games ... Intelligence in Education, 17(2):121-144, 2007

KOM – Multimedia Communications Lab 7

Motivation

http://www.openequalfree.org/gamification-versus-game-based-learning-in-the-classroom/10082

Why Learning Analytics & Serious Games?

• Evaluation of Serious Games

• Justifying expense in learning contexts

• Objective and cost-effective approach

• Evaluation with Serious Games

• Provide a big amount of gameplay data

• Interactive and engaging nature

Stealth Assessment

• Enable insight about learner attributes

and learning progress

Page 8: Learning Analytics and Serious Games: Trends and ... · Game technology & concepts ... • Description & model for Serious Games ... Intelligence in Education, 17(2):121-144, 2007

KOM – Multimedia Communications Lab 8

Conceptual Approach Learning Analytics & SG

Page 9: Learning Analytics and Serious Games: Trends and ... · Game technology & concepts ... • Description & model for Serious Games ... Intelligence in Education, 17(2):121-144, 2007

KOM – Multimedia Communications Lab 9

Modelling for Learning Analytics in SG

https://www.linkedin.com/pulse/article/20140320222540-1265384-show-what-you-know-the-future-of-

competency-based-learning

• Content

• Competence-based Knowledge Space Theory

(CbKST)

• Requires learning domains to be modelled as a

prerequisite competency structure

• Users

• Open Learner Model (OLM)

• Presenting to the learner an understandable

visualization of his current knowledge state

• Proven to improve learning outcomes

• Player Model by Bartle

• Achiever, Explorer, Killer, Socializer

• Content & Users

• Narrative Game-Based Learning Objects (NGLOB)

• Additionally considers player type and narrative

aspects

• Triple vector: Narrative, Gaming and Learning Context

Page 10: Learning Analytics and Serious Games: Trends and ... · Game technology & concepts ... • Description & model for Serious Games ... Intelligence in Education, 17(2):121-144, 2007

KOM – Multimedia Communications Lab 10

Conceptual Approach Learning Analytics & SG

Page 11: Learning Analytics and Serious Games: Trends and ... · Game technology & concepts ... • Description & model for Serious Games ... Intelligence in Education, 17(2):121-144, 2007

KOM – Multimedia Communications Lab 11

Choosing and Capturing Data I

Recording data depends

on

• Learning goals, tasks

and setting

• Game genre, mechanic

and platform

• Single-Player vs.

Multiplayer

• additional social

component in

collaborative learning

• Fun vs. Learning

(effects)

Designing games „with

analytics in mind“

www.storytec.de StoryTec Authoring Environment with StoryPlay – Learning Analytics Tool

Page 12: Learning Analytics and Serious Games: Trends and ... · Game technology & concepts ... • Description & model for Serious Games ... Intelligence in Education, 17(2):121-144, 2007

KOM – Multimedia Communications Lab 12

Choosing and Capturing Data II

Data modalities and interactions

• Multimodal Learning Analytics

• Includes biometric data and other multimodal

data for assessing motivation, fun and

collaboration aspects in learning settings

• Mobile and Ubiquitous Learning Analytics

• Data of mobile game-based learning appliances

• Interaction with mobile devices

• Considering contextual information

Page 13: Learning Analytics and Serious Games: Trends and ... · Game technology & concepts ... • Description & model for Serious Games ... Intelligence in Education, 17(2):121-144, 2007

KOM – Multimedia Communications Lab 13

Conceptual Approach Learning Analytics & SG

Page 14: Learning Analytics and Serious Games: Trends and ... · Game technology & concepts ... • Description & model for Serious Games ... Intelligence in Education, 17(2):121-144, 2007

KOM – Multimedia Communications Lab 14

Aggregating and Analyzing Data

„Aggregation Model“

• using semantic rules to map game actions or states to meaningful (machine-readable)

expressions under which similar events are grouped

Analyzing data depends on learning context and application

• By instructor (via browser/analyzer)

• Automatic Analysis (for intelligent tutoring systems and adaptive Serious Games)

• Measures to be derived:

• Gaming: general in-game performance, in-game learning, in-game strategies,

player type

• Learning: general traits and abilities of the learner, general knowledge, situation-

specific state, learning behaviors, learning outcomes

• Rules and algorithms (applied during learning sessions) governing the interpretation of

in-game sources of evidence to infer competencies and to update competency models

• Data Mining and Machine Learning approaches can be used for identifying solution

strategies, error patterns and player goals

Page 15: Learning Analytics and Serious Games: Trends and ... · Game technology & concepts ... • Description & model for Serious Games ... Intelligence in Education, 17(2):121-144, 2007

KOM – Multimedia Communications Lab 18

Conceptual Approach Learning Analytics & SG

Page 16: Learning Analytics and Serious Games: Trends and ... · Game technology & concepts ... • Description & model for Serious Games ... Intelligence in Education, 17(2):121-144, 2007

KOM – Multimedia Communications Lab 19

Deploying Results for Learning Analytics in SG

Visualization

• visualizations of narrative structure,

player model and skill tree

• graphs, Hasse Diagrams, Heat Maps

• for games, a special need for real-time

operation, extensibility and

interoperability

Adaptation

• macro-adaptivity: system responds by

choosing the appropriate next learning

object or narrative event

• micro-adaptivity: adjusting aspects

within a learning task like task diffculty

or feedback type

Page 17: Learning Analytics and Serious Games: Trends and ... · Game technology & concepts ... • Description & model for Serious Games ... Intelligence in Education, 17(2):121-144, 2007

KOM – Multimedia Communications Lab 21

Questions & Contact

Page 18: Learning Analytics and Serious Games: Trends and ... · Game technology & concepts ... • Description & model for Serious Games ... Intelligence in Education, 17(2):121-144, 2007

KOM – Multimedia Communications Lab 22

References

• N. R. Aljohani and H. C. Davis. Learning analytics in mobile and ubiquitous learning environments. In 11th • World Conference on Mobile and Contextual Learning: mLearn 2012, Helsinki, Finland, 2012. • R. S. J. D. Baker and K. Yacef. The State of Educational Data Mining in 2009 : A Review and Future Visions. 1(1):3-16, 2009. • P. Blikstein. Multimodal learning analytics. Proceedings of the Third International Conference on • Learning Analytics and Knowledge - LAK '13, page 102, 2013. • S. Bull, Y. C. Y. Cui, a.T. McEvoy, E. Reid, and W. Y. W. Yang. Roles for mobile learner models. The 2nd IEEE International

Workshop on Wireless and Mobile Technologies in Education, 2004. Proceedings., pages 124-128, 2004. • S. Bull and J. Kay. Open learner models. In Advances in Intelligent Tutoring Systems, pages 301-322. Springer, 2010. • G. K. Chung and D. S. Kerr. A Primer on Data Logging to Support Extraction of Meaningful • Information from Educational Games: An Example from Save Patch. CRESST Report 814. National • Center for Research on Evaluation, Standards, and Student Testing (CRESST), page 27, 2012. • A. Cooper. Learning Analytics Interoperability – The Big Picture In Brief. Learning Analytics Community Exchange, 2014. • E. Duval. Attention Please!: Learning Analytics for Visualization and Recommendation. LAK '11, pages 9-17, New York, NY, USA,

2011. ACM. • A. Dyckho and D. Zielke. Design and Implementation of a Learning Analytics Toolkit for Teachers. Journal of . . . , 15:58-76, 2012. • G. Dyke, K. Lund, and J.-J. Girardot. Tatiana: An Environment to Support the CSCL Analysis Process. In Proceedings of the 9th

International Conference on Computer Supported Collaborative Learning – Volume 1, CSCL'09, pages 58{67. International Society of the Learning Sciences, 2009.

• B. M. Eun Young Ha Jonathan Rowe and J. Lester. Recognizing Player Goals in Open-Ended Digital Games with Markov Logic Networks. Plan, Activity and Intent Recognition: Theory and Practice, 2014.

• J.-C. Falmagne, D. Albert, C. Doble, D. Eppstein, and X. Hu. Knowledge Spaces: Applications in Education. Springer Science & Business, 2013.

• I. Garcia, A. Duran, and M. Castro. Comparing the eectiveness of evaluating practical capabilities through hands-on on-line exercises versus conventional methods. In Frontiers in Education Conference, 2008.

• FIE 2008. 38th Annual, pages F4H{18{F4H{22, Oct • 2008. • S. Goebel, M. Gutjahr, and S. Hardy. Evaluation of Serious Games. Serious Games and Virtual Worlds in Education, Professional

Development, and Healthcare, page 105, 2013.

Page 19: Learning Analytics and Serious Games: Trends and ... · Game technology & concepts ... • Description & model for Serious Games ... Intelligence in Education, 17(2):121-144, 2007

KOM – Multimedia Communications Lab 23

References

• S. Goebel, V. Wendel, C. Ritter, and R. Steinmetz. Personalized, adaptive digital educational games using narrative game-based learning objects. pages 438-445. Springer, 2010.

• S. L. . B. H. Jan Plass. Assessment mechanics: design of learning activities that produce meaningful data.New York University, CREATE Lab, Apr. 2013.

• K. E. B. E. W. Joseph Grafsgaard Joseph Wiggins and J. Lester. Predicting Learning and Aect from Multimodal Data Streams in Task-Oriented Tutorial Dialogue. In Seventh International Conference on Educational Data Mining, 2014.

• D. S. Kerr and G. K. W. K. Chung. Using Cluster Analysis to Extend Usability Testing to Instructional Content. CRESST Report 816. National Center for Research on Evaluation, Standards, and Student Testing (CRESST), May 2012.

• M. D. Kickmeier-Rust and D. Albert. An Alien's Guide to Multi-Adaptive Educational Computer Games. Informing Science, 2012. • M. D. Kickmeier-Rust and D. Albert. Learning analytics to support the use of virtual worlds in the classroom. In Human-Computer

Interaction and Knowledge Discovery in Complex, Unstructured, Big Data, pages 358{365. Springer, 2013. • K. R. Koedinger, R. Baker, K. Cunningham, A. Skogsholm, B. Leber, and J. Stamper. A data repository for the EDM community: The

PSLC DataShop. Handbook of educational data mining, pages 43-55, 2010. • J. Konert, K. Richter, F. Mehm, S. G•obel, R. Bruder, and R. Steinmetz. Pedale - a peer education diagnostic and learning

environment. Journal of Educational Technology & Society, 2012. • K. Korossy. Modeling knowledge as competence and performance. Knowledge spaces: Theories, empirical research, and

applications, pages 103{132, 1999. • J. Lester, B. Mott, J. Rowe, and J. Sabourin. {learner modeling to predict real-time aect in serious games. Design Recommendations

for Intelligent Tutoring Systems, page 201, 2013. • S. M. Letourneau. Intensive data at CREATE -Research approaches & future directions. In Simulations and Games Learning

Analytics and Data Mining Workshop, 2013. • R. Levy. Dynamic Bayesian Network Modeling of Game Based Diagnostic Assessments. CRESST Report, 2014. • C. Martin, O. Galibert, M. Michel, F. Mougin, and V. Stanford. NIST Smart Flow System User's guide. • M. Minovic and M. Milovanovic. Real-time learning analytics in educational games. pages 245{251. ACM, • 2013. • R. J. Mislevy, R. G. Almond, and J. F. Lukas. A brief introduction to evidence-centered design. cse report 632. US Department of

Education, 2004. • A. Mitrovic and B. Martin. Evaluating the eect of open student models on self-assessment. International Journal of Articial

Intelligence in Education, 17(2):121-144, 2007. • A. Morrison, P. Tennent, and M. Chalmers. Coordinated visualisation of video and system log data. In Coordinated and Multiple

Views in Exploratory Visualization, 2006. Proceedings. International Conference on, pages 91-102. IEEE,2006.

Page 20: Learning Analytics and Serious Games: Trends and ... · Game technology & concepts ... • Description & model for Serious Games ... Intelligence in Education, 17(2):121-144, 2007

KOM – Multimedia Communications Lab 24

References

• K. Nurmela, E. Lehtinen, and T. Palonen. Evaluating CSCL log les by social network analysis. In Proceedings of the 1999 conference on Computer support for collaborative learning, page 54. International Society of the Learning Sciences, 1999.

• H. Ogata, M. Li, B. Hou, N. Uosaki, M. M. El-Bishouty, and Y. Yano. SCROLL: Supporting to share and reuse ubiquitous learning log in the context of language learning. Research & Practice in Technology Enhanced Learning, 6(2), 2011.

• S. Oviatt, A. Cohen, and N. Weibel. Multimodal learning analytics: Description of math data corpus for icmi grand challenge workshop. In Proceedings of the 15th ACM on International conference on multimodal interaction, pages 563{568. ACM, 2013.

• J. L. Plass, B. D. Homer, C. K. Kinzer, Y. K. Chang, J. Frye, W. Kaczetow, K. Isbister, and K. Perlin. Metrics in Simulations and Games for Learning. In Game Analytics, pages 697{729. Springer, 2013.

• M. Prensky. Computer games and learning: Digital game-based learning. Handbook of computer game studies, 18:97-122, 2005.

• C. Reuter, F. Mehm, S. G•obel, and R. Steinmetz. Evaluation of Adaptive Serious Games using Playtraces and Aggregated Play Data. Proceedings of the 7th European Conference on Game Based Learning (ECGBL) 2013, (October):504{511, 2013.

• A. Serrano, E. J. Marchiori, A. del Blanco, J. Torrente, and B. Fernandez-Manjon. A framework to improve evaluation in educational games. In Global Engineering Education Conference (EDUCON), 2012 IEEE, pages 1-8. IEEE, 2012.

• A. Serrano-Laguna, J. Torrente, P. Moreno-Ger, and B. Fernandez-Manjon. Application of learning analytics in educational videogames. Entertainment Computing, 2014.

• V. J. Shute, M. Ventura, M. Bauer, and D. Zapata-Rivera. Melding the power of serious games and embedded assessment to monitor and foster learning. Serious games: Mechanisms and eects,

• pages 295-321, 2009. • G. Siemens and R. S. J. Baker. Learning Analytics and Educational Data Mining : Towards Communication and Collaboration.

2010. • K. K. Tatsuoka. Architecture of knowledge structures and cognitive diagnosis: A statistical pattern recognition and classication

approach. Cognitively diagnostic assessment, pages 327-359, 1995. • T. P. Vendlinski, G. C. Delacruz, R. E. Buschang, G. Chung, and E. L. Baker. Developing high-quality assessments that align

with instructional video games (CRESST Report 774). . National Center for Research on Evaluation, Standards, and Student Testing, 2010.

• A. V. Wallop. Althea Vail Wallop CS 229 Final Project Report Learning to Predict Disengagement in Educational iPad Application. pages 1-5, 2010.

• J.-D. Zapata-Rivera and J. E. Greer. Interacting with inspectable bayesian student models. International Journal of Articial Intelligence in Education, 14(2):127-163, 2004.

Page 21: Learning Analytics and Serious Games: Trends and ... · Game technology & concepts ... • Description & model for Serious Games ... Intelligence in Education, 17(2):121-144, 2007

KOM – Multimedia Communications Lab 25

Websites

http://www.zoodles.com http://www.google.com/analytics http://piwik.org/ http://secondlife.com/ http://opensimulator.org/