(abstracts) - educational data mining...and challenges of privacy-preserving edm, ethics-aware...
TRANSCRIPT
INVITED TALKS
(abstracts)
Proceedings of the 8th International Conference on Educational Data Mining 1
Proceedings of the 8th International Conference on Educational Data Mining 2
Behind the Scenes of Duolingo
Luis Von AhnDuolingo and Carnegie Mellon University
Matt StreeterDuolingo
ABSTRACTWith over 100 million users, Duolingo is the most populareducation app in the world in Android and iOS. In the firstpart of this talk, we will describe the motivation for creatingDuolingo, its philosophy, and some of the basic techniquesused to successfully teach languages and keep users engaged.The second part will focus on the machine learning and nat-ural language processing algorithms we use to model studentlearning.
Proceedings of the 8th International Conference on Educational Data Mining 3
Proceedings of the 8th International Conference on Educational Data Mining 4
Personal Knowledge/Learning Graph
George SiemensUniversity of Texas Arlington
andAthabasca University
Ryan BakerTeachers College
Columbia Universitybaker2@exchange.
tc.columbia.edu
Dragan GasevicSchools of Education and
InformaticsUniversity of Edinburgh
ABSTRACTEducational data mining and learning analytics have to datelargely focused on specific research questions that provideinsight into granular interactions. These insights have beenabstracted to include the development of predictive models,intelligent tutors, and adaptive learning. While there areseveral domains where holistic or systems models have pro-vided additional explanatory power, work around learninghas not created holistic models with the level of concrete-ness or richness required. The need for both granular andintegrated high-level view of learning is further influencedby distributed, life long, multi-spaced learning that todaydefines education. Drawing on social and knowledge graphtheory, we propose the development of a Personal Knowl-edge/Learning Graph (PKLG) - an open and learner-ownedprofile that addresses cognitive, affective, and related ele-ments that reflect what a learner knows, is able to do, andprocesses through which she learns best. This talk will in-troduce PKLG, detail required technical infrastructure, andarticulate how it would interact with established learningsoftware.
Proceedings of the 8th International Conference on Educational Data Mining 5
Proceedings of the 8th International Conference on Educational Data Mining 6
Educational Neuroscience as a Tool to UnderstandLearning and Learning Disabilities in Mathematics
Pekka RäsänenNiilo Mäki InstituteJyväskylä, Finland
ABSTRACTBecoming numerate is considered as one of the fundamentalskills needed in the modern technology-driven society. Thelatest OECD (2013) report states that aAIJThe way we live
and work has changed profoundly aAS and so has the setof skills we need to participate fully in and benefit from ourhyper-connected societies and increasingly knowledge-basedeconomies.aAIJ The societies invest a lot on education withvarying results. For some reasons there still are persons donot reach even a basic level of skills in numeracy or literacyirrespective of the recent advances in education, educationalresearch and educational technologies.
Persons who fail in learning numeracy, even though theyhave had an opportunity to learn and who, based on theirother skills, should have learnt, we call as having specificlearning disabilities (SLD), developmental dyscalculia (DD).This discrepancy between learning opportunities, generalskills and poor performance in mathematics, has intriguedresearchers now more than a century. From the early begin-ning of the research there has been ideas that it has some-thing to do how the brain of these persons have organized,failed to develop or damaged.
The recent developments in research methodologies, espe-cially in brain imaging and statistical technologies, haveopened new windows to analyze these brain related hypothe-ses. In my presentation I will open some of these windowswith examples from functional brain imaging to longitudinalstudies based on multivariate statistical analysis.
The new windows show different views from the DD. Fromone perspective the DD looks like a unitary construct withvery specific symptoms in numerical processing. This viewhas been more typical within the brain imaging research.The other views show a complex where myriad of factorsfrom genetic to learning experiences each contribute witha small share to the large variation of the individual skills.This view has been more typical in behavioural and cog-
nitive studies, especially in longitudinal research. Whethera common ground can be reached, and what it needed forthat, is discussed.
ReferencesSyvaoja, H., Tammelin, T., Ahonen, T. Rasanen, P., Tolva-nen, A., Kankaanpaa, A., and Kantomaa, M.T. (in press).Internal consistency and stability of the CANTAB neuropsy-chological test battery in children. Psychological Assess-ment.
Aunio, P. and Rasanen, P. (in press). Core numerical skillsfor learning arithmetic in children aged five to eight years.European Early Childhood Education Research Journal.
Rasanen, P., Kaser, T., Wilson, A., von Aster, M., Maslov,A., and Maslova, U. (in press, Jan 2015). Assistive Technol-ogy for Supporting Learning Numeracy. In B O’Neill and A.Gillespie, (eds.) Assistive Technology for Cognition. Cur-rent Issues in Neuropsychology. London: Psychology Press.
Hannula-Sormunen, M. M., Lehtinen E., and Rasanen P. (inpress, May 2015). Children’s preschool subitizing, sponta-neous focusing on numerosity and counting skills as predic-tors of mathematical performance 6-7 years later at school.Mathematical Thinking and Learning.
Rasanen, P. (2014). Computer-assisted Interventions on Ba-sic Number Skills. In R. Cohen Kadosh and A. Dowker,(eds.) The Oxford Handbook of Mathematical Cognition.Oxford: Oxford University Press.
Mazzocco, M., and Rasanen, P. (2013). Contributions oflongitudinal studies to evolving definitions and knowledgeof developmental dyscalculia. Trends in Neuroscience andEducation, 2, 65-73.
Zhang, X., Koponen, T., Rasanen, P., Aunola, K., Lerkka-nen, M., and Nurmi, J. (2013). Linguistic and Spatial SkillsPredict Early Arithmetic Development via Counting SequenceKnowledge. Child Development, 85(3), 1091-1107.
Price, G., Holloway, I., Rasanen, P., Vesterinen, M., andAnsari, D. (2008). Impaired parietal magnitude processingin Developmental Dyscalculia. Current Biology, 17(24).
Proceedings of the 8th International Conference on Educational Data Mining 7
Proceedings of the 8th International Conference on Educational Data Mining 8
INVITED PANELS
(abstracts)
Proceedings of the 8th International Conference on Educational Data Mining 9
Proceedings of the 8th International Conference on Educational Data Mining 10
Industry Panel: The Future of Practical Applications ofEDM at Scale
Ryan BakerTeachers College
Columbia Universitybaker2@exchange.
tc.columbia.edu
John CarneyCarney Labs LLC
Piotr MitrosedX
Bror Saxberg(moderator)
Kaplan [email protected]
John StamperCarnegie Mellon University
and PSLC [email protected]
ABSTRACTThis mixed panel of different professionals working in EDMwill be a conversation about increasing the connection be-tween research and real-world applications. What’s going onnow to scale techniques for use ”out there”in the field? Whatshould researchers, funders, regulators, publishers, trainers,schools/universities and others be doing to get ready forpractical work? What’s in the way that we can usefullystart work to address? We’ll ask the audience to engagein this conversation as well - what’s in your way to movingwork from research environments to practically help learnersat scale - and to generate more useable data at scale?
Proceedings of the 8th International Conference on Educational Data Mining 11
Proceedings of the 8th International Conference on Educational Data Mining 12
Ethics and Privacy in EDM
Dragan GasevicUniversity of Edinburgh
Taylor Martin (moderator)National Science Foundation
Zach PardosUC Berkeley
[email protected] Pechenizkiy
John StamperCMU and PSLC DataShop
Osmar ZaianeUniversity of Alberta
ABSTRACTEducational data mining is inherently falls into the categoryof the so-called secondary data analysis. It is common thatdata that have been collected for administrative or someother purposes at some point is considered as valuable forother (research) purpose. Collection of the student gener-ated, student behavior and student performance related dataon a massive scale in MOOCs, ITSs, LMS and other learningplatforms raises various ethical and privacy concerns amongresearches, policy makers and the general public. This panelis aimed to discuss major challenges in ethics and privacyin EDM and how they are addressed now or should be ad-dressed in the future to prevent any possible harm to thelearners. Several experts are invited to discuss the potentialand challenges of privacy-preserving EDM, ethics-aware pre-dictive learning analytics, and availability of public bench-mark datasets for EDM among others.
Proceedings of the 8th International Conference on Educational Data Mining 13
Proceedings of the 8th International Conference on Educational Data Mining 14
Grand Challenges for EDM and Related Research Areas
Ryan Baker (moderator)Teachers College
Columbia Universitybaker2@exchange.
tc.columbia.edu
Peter Brusilovsky (UMInc)
School of InformationSciences
Pittsburgh [email protected]
Dragan Gasevic (SoLAR)Schools of Education and
InformaticsUniversity of Edinburgh
Neil T. Heffernan (AIED)Department of Computer
ScienceWorcester Polytechnic Institute
Mykola Pechenizkiy(IEDMS)
Department of ComputerScience
Alyssa Wise (ISLS)Faculty of Education
Simon Fraser [email protected]
ABSTRACTEducational data mining (EDM) and Learning analytics arestill rather young research areas. The goal of this panel isto share different views on what major challenges researchesneed to address in EDM, learning analytics and related re-search areas including but not limited to User modeling, AIin Education, and Learning Sciences. The representatives ofthe corresponding communities are invited to discuss whatgrand challenges we should aim to address for the next fiveyears, and which of these challenges are old and which arenew, which of them peculiar to one distinct research areaand which of them are shared across two or more researchareas.
Proceedings of the 8th International Conference on Educational Data Mining 15
Proceedings of the 8th International Conference on Educational Data Mining 16
JEDM TRACK PAPERS
(abstracts)
Proceedings of the 8th International Conference on Educational Data Mining 17
Proceedings of the 8th International Conference on Educational Data Mining 18
Metrics for Evaluation of Student Models
Radek PelánekMasaryk University Brno
ABSTRACTResearchers use many different metrics for evaluation of per-formance of student models. The aim of this paper is to pro-vide an overview of commonly used metrics, to discuss prop-erties, advantages, and disadvantages of different metrics, tosummarize current practice in educational data mining, andto provide guidance for evaluation of student models. In thediscussion we mention the relation of metrics to parameterfitting, the impact of student models on student practice(over-practice, under-practice), and point out connectionsto related work on evaluation of probability forecasters inother domains. We also provide an empirical comparisonof metrics. One of the conclusion of the paper is that somecommonly used metrics should not be used (MAE) or shouldbe used more critically (AUC).
Proceedings of the 8th International Conference on Educational Data Mining 19
Proceedings of the 8th International Conference on Educational Data Mining 20
Multi-Armed Bandits for Intelligent Tutoring Systems
Benjamin ClementInria, France
Didier RoyInria, France
Pierre-Yves OudeyerInria, France
[email protected] LopesInria, France
ABSTRACTWe present an approach to Intelligent Tutoring Systemswhich adaptively personalizes sequences of learning activ-ities to maximize skills acquired by students, taking intoaccount the limited time and motivational resources. At agiven point in time, the system proposes to the students theactivity which makes them progress faster. We introducetwo algorithms that rely on the empirical estimation of thelearning progress, RiARiT that uses information about thedifficulty of each exercise and ZPDES that uses much lessknowledge about the problem.
The system is based on the combination of three approaches.First, it leverages recent models of intrinsically motivatedlearning by transposing them to active teaching, relying onempirical estimation of learning progress provided by spe-cific activities to particular students. Second, it uses state-of-the-art Multi-Arm Bandit (MAB) techniques to efficientlymanage the exploration/exploitation challenge of this op-timization process. Third, it leverages expert knowledgeto constrain and bootstrap initial exploration of the MAB,while requiring only coarse guidance information of the ex-pert and allowing the system to deal with didactic gaps inits knowledge. The system is evaluated in a scenario where7–8 year old schoolchildren learn how to decompose numberswhile manipulating money. Systematic experiments are pre-sented with simulated students, followed by results of a userstudy across a population of 400 school children.
Proceedings of the 8th International Conference on Educational Data Mining 21
Proceedings of the 8th International Conference on Educational Data Mining 22
Developing Computational Methods to Measure and TrackLearners’ Spatial Reasoning in an Open-Ended Simulation
Aditi MallavarapuUniversity of Illinois at
Leilah LyonsUniversity of Illinois at
Tia ShelleyUniversity of Illinois at
Brian SlatteryUniversity of Illinois at
Moira ZellnerUniversity of Illinois at
Emily MinorUniversity of Illinois at
ABSTRACTInteractive learning environments can provide learners withopportunities to explore rich, real-world problem spaces,but the nature of these problem spaces can make assessinglearner progress difficult. Such assessment can be useful forproviding formative and summative feedback to the learn-ers, to educators, and to the designers of the environments.This work adds to a growing body of research that is apply-ing EDM techniques to more open-ended problem spaces.
The open-ended problem space under study here is an en-vironmental science simulation. Learners were confrontedwith the real-world challenge of effectively placing green in-frastructure in an urban neighborhood to reduce surfaceflooding. Learners could try out different spatial arrange-ments of green infrastructure and use the simulation to testeach solution’s impact on flooding. The learners’ solutionsand the solutions’ performances were logged during a con-trolled experiment with different user interface designs forthe simulation. As with many open-problem spaces, analyz-ing this data was difficult due to the large state space, manygood solutions, and many alternate paths to those good so-lutions.
This work proposes a procedure for reducing the state spaceof solutions defined by spatial patterns while maintainingtheir critical spatial properties. Spatial reasoning problemsare a problem class not yet examined by EDM, so this worksets the stage for further research in this area. This work alsodetails a procedure for discovering effective spatial strategiesand solution paths, and demonstrates how this informationcan be used to give formative feedback to the designers ofthe interactive learning environment.
Proceedings of the 8th International Conference on Educational Data Mining 23
Proceedings of the 8th International Conference on Educational Data Mining 24
Move your lamp post: Recent data reflects learnerknowledge better than older data
April GalyardtUniversity of Georgia
Ilya GoldinPearson
ABSTRACTIn educational technology and learning sciences, there aremultiple uses for a predictive model of whether a studentwill perform a task correctly or not. For example, an in-telligent tutoring system may use such a model to estimatewhether or not a student has mastered a skill. We analyzethe significance of data recency in making such predictions,i.e., asking whether relatively more recent observations ofa student’s performance matter more than relatively olderobservations. We investigate several representations of re-cency, such as the count of prior practice in the AFM model,and the proportion of recent successes with exponential andbox kernels. We find that an exponential decay of a pro-portion of successes provides the summary of recent practicewith the highest predictive accuracy over alternative models.As a secondary contribution, we develop a new logistic re-gression model, Recent-Performance Factors Analysis, thatleverages this representation of recent performance, and hashigher predictive accuracy than existing logistic regressionmodels.
Proceedings of the 8th International Conference on Educational Data Mining 25
Proceedings of the 8th International Conference on Educational Data Mining 26