NeilS

pencerBruce

Flickr

The use of data and analytics toguide the improvement of learningexperiences

International Forum on Big Data Analysisfor Learning Improvement

Dr Abelardo Pardo (@abelardopardo)The University of Sydney

slideshare.net/abelardo_pardo

Abelardo Pardo The use of data and analytics to guide the improvement of learning experiences 2

Cygnus921

flickr.com

LALandscape

What toimprove?

Collection,Algorithms,Interventions

Stakeholders’Views

Abelardo Pardo The use of data and analytics to guide the improvement of learning experiences 3

Cygnus921

flickr.com

LALandscape

Learning Analytics

Measurement, collection, analysisand reporting of data about learnersand their contexts, for purposes ofunderstanding and optimizinglearning and the environments in whichit occurs.

Abelardo Pardo The use of data and analytics to guide the improvement of learning experiences 4

(SoLAR, Society for LearningAnalytics Research)

Abelardo Pardo The use of data and analytics to guide the improvement of learning experiences 5

VanessaOflickr.com

“Historically, humanity has made sense of the worldthrough discourse, dialogue, artifacts, myth, story,and metaphor. While those sensemaking approacheswon’t disappear, they will be augmented by data andanalytics.”

(Siemens, 2014 www.elearnspace.org/blog)

Discipline emerged in recent years

with large potential for impact.

Abelardo Pardo The use of data and analytics to guide the improvement of learning experiences 6

www.solaresearch.orgAbelardo Pardo The use of data and analytics to guide the improvement of learning experiences 7

Intl. Conference onLearning Analytics and Knowledge

lak15.solaresearch.orgAbelardo Pardo The use of data and analytics to guide the improvement of learning experiences 8

Abelardo Pardo The use of data and analytics to guide the improvement of learning experiences 9

LJubarflickr.com

Fork in the road:uptake?

Abelardo Pardo The use of data and analytics to guide the improvement of learning experiences 10

Derable

flickr.com

High impact area: evolution, revolution, tangibleimprovements.

Abelardo Pardo The use of data and analytics to guide the improvement of learning experiences 11

Yoakimflickr.comFail at large scale and

become marginal

Abelardo Pardo The use of data and analytics to guide the improvement of learning experiences 12

diffusorflickr.com

How big is “big data”?

Few terabytes (today)

Abelardo Pardo The use of data and analytics to guide the improvement of learning experiences 13

jseliger1flickr.com

Abundant personaldata

Millions of records

Personalization+Interventions

Abelardo Pardo The use of data and analytics to guide the improvement of learning experiences 14

Gee311

flickr.com

InstitutionalLevel

LearningExperience

LearningActivity

Macrolevel: Data manipulated at the institutionallevel

Abelardo Pardo The use of data and analytics to guide the improvement of learning experiences 15

JISC

InfoNetflickr

Abelardo Pardo The use of data and analytics to guide the improvement of learning experiences 16

TulaneP

ublicR

elationsflickr.com

Microlevel: detailedobservation of single context

Abelardo Pardo The use of data and analytics to guide the improvement of learning experiences 17

LearningExperience:multiple tools,variety of activities,address severallearning outcomes.

Abelardo Pardo The use of data and analytics to guide the improvement of learning experiences 18

Cygnus921

flickr.com

LALandscape

What toimprove?

“We need betterevaluation toacknowledge andesteem excellencewhen it occurs, as itdoes.”

Abelardo Pardo The use of data and analytics to guide the improvement of learning experiences 19

Bencrow

eFlickr

(Hattie, 2009)

Where should I focus?

Abelardo Pardo The use of data and analytics to guide the improvement of learning experiences 20

visible-learning.org/hattie-ranking-influences-effect-sizes-learning-achievement/hattie-ranking-teaching-effects

(LastAccessed

Oct.

2013)

Abelardo Pardo The use of data and analytics to guide the improvement of learning experiences 21

JansonH

ews

flickr.com

New

Assessments?

Abelardo Pardo The use of data and analytics to guide the improvement of learning experiences 22

Ashley

Fisherflickr.com

21st century skills

Critical thinkingTeam work skillsCommunication skillsInformation literacyCreativity and Innovation

Discourse Analysis

Abelardo Pardo The use of data and analytics to guide the improvement of learning experiences 23

(De Liddo et al., 2011)

Abelardo Pardo The use of data and analytics to guide the improvement of learning experiences 24

PaulM

ayneflickr.com

Learning Dispositions:Tendency to behave incertain way when learning.

(Buckingham Sum, Deaking Crick, 2012)

Abelardo Pardo The use of data and analytics to guide the improvement of learning experiences 25

Cygnus921

flickr.com

LALandscape

What toimprove?

Collection,Algorithms,Interventions

Abelardo Pardo The use of data and analytics to guide the improvement of learning experiences 26

(Macfadyen & Dawson, 2010)

Abelardo Pardo The use of data and analytics to guide the improvement of learning experiences 27

(Romero, Ventura & García, 2008)

Abelardo Pardo The use of data and analytics to guide the improvement of learning experiences 28

ChefC

ookeFlickr.comObserve while

working oncourse

activities

A clearly identifiedenvironment within

your computer

Abelardo Pardo The use of data and analytics to guide the improvement of learning experiences 29

Kevin

Davies

flickr.com

(Pardo & Delgado Kloos, 2011)

InstrumentationPersonal Computer

ComputerTool 1

ComputerTool 2

ComputerTool 3

ComputerTool 4

Authenticated

Virtual Computer

ComputerTool

ComputerTool

ComputerTool

ComputerTool

ComputerTool

EventDataset

PersonalData

Encoding

Report

Infer

Act

InstructorStudent

Abelardo Pardo The use of data and analytics to guide the improvement of learning experiences 30

Abelardo Pardo The use of data and analytics to guide the improvement of learning experiences 31

Horia

Varlanflickr.com

Tool usage vs academic performance

(Romero-Zaldívar et al., 2012)

Abelardo Pardo The use of data and analytics to guide the improvement of learning experiences 32

deepwarren

Flickr

Closer to the student

From “app” to “companion”

Finished focus group. Potential for studentuptake.

Abelardo Pardo The use of data and analytics to guide the improvement of learning experiences 33

Abelardo Pardo The use of data and analytics to guide the improvement of learning experiences 34

puthoOrP

hotographyflickr.com

LMS interactions, quizzes, response times, sessions,hints requested, questionnaires, interviews, requests,demographics, high school grades, etc.

(Bienkowski, M., Feng, M., & Means, B., 2012)

Abelardo Pardo The use of data and analytics to guide the improvement of learning experiences 35

Naezm

iflickr.com

Learning Record Stores. Tin Can API

Abelardo Pardo The use of data and analytics to guide the improvement of learning experiences 36

Mr.

Velocipedeflickr.com

Statisticalanalysis(small)

Predictivealgorithms

ClusteringAlgorithms

RelationshipMining

Data Distillation DiscoverywithModels

(Baker and Yacef, 2009)

Design Principles for Interventions

• Integrated with the activity. Connected withconcepts and assessment.

• Interventions must support learning, notbecome an additional burden.

• Provide measures that are considereddesirable and why.

• Discuss the process with the students.

Abelardo Pardo The use of data and analytics to guide the improvement of learning experiences 37

(Wise, 2014)

Abelardo Pardo The use of data and analytics to guide the improvement of learning experiences 38

Brian

Talbotflickr.com

Personalization

How to connect events, user data with actions?

Example: Rule based models

Abelardo Pardo The use of data and analytics to guide the improvement of learning experiences 39

DanielO

lnesflickr.com

IF CONDITION THEN ACTION

Abelardo Pardo The use of data and analytics to guide the improvement of learning experiences 40

DanielO

lnesflickr.com

IF video not watched and Thursday THEN reminder

Abelardo Pardo The use of data and analytics to guide the improvement of learning experiences 41

Cygnus921

flickr.com

LALandscape

What toimprove?

Collection,Algorithms,Interventions

Stakeholders’Views

Abelardo Pardo The use of data and analytics to guide the improvement of learning experiences 42

Allan

Lordeflickr.com

Analytics bydesign. Increasein complexity

Abelardo Pardo The use of data and analytics to guide the improvement of learning experiences 43

jbgeronimiflickr.com

Instructor manipulates multiple data sources

Abelardo Pardo The use of data and analytics to guide the improvement of learning experiences 44

Space

&LightFlickr

• Data far away from instructor• Needs curation process

Abelardo Pardo The use of data and analytics to guide the improvement of learning experiences 45

MichaelD

unnflickr.com

Analytics as Self-reflection Tool

The Student Point of View

• Transparency in the assessment

• Detailed account of strategies

• Goal setting

Abelardo Pardo The use of data and analytics to guide the improvement of learning experiences 46

lrargerichFlickr

Abelardo Pardo The use of data and analytics to guide the improvement of learning experiences 47

RobertH

uffstutterflickr.com

• Learning Analytics: discipline with potential.

• From extremely beneficial to irrelevant.

• Objective: personalization and Interventions.

• Use data for the “meso”-level (Course).

• Comprehensive observation offers insight intostrategies.

• Data in multiple sources, formats, and far away.

• Design and deployment more complex.

• Explore use by students.

NeilS

pencerBruce

Flickr

The use of data and analytics toguide the improvement of learningexperiences

International Forum on Big Data Analysisfor Learning Improvement

Dr Abelardo Pardo (@abelardopardo)The University of Sydney

slideshare.net/abelardo_pardo

ReferencesSiemens, G. (2013).Learning Analytics: The Emergence of a Discipline.American Behavioral Scientist, 57(10), 1380–1400.

John HattieVisible Learning: A Synthesis of Over 800 Meta-Analyses Relating to AchievementTaylor & Francis, 2009

Macfadyen, L. P., & Dawson, S. (2010).Mining LMS data to develop an “early warning system” for educators: A proof of concept.Computers & Education, 54(2), 588–599

Romero, C., Ventura, S., & Garcia, E. (2008).Data mining in course management systems: Moodle case study and tutorial.Computers & Education, 51(1), 368–384.

Pardo, A., Delgado Kloos, C., 2011,Stepping out of the box. Towards analytics outside the Learning Management SystemInternational Conference on Learning Analytics and Knowledge, pp, 163-167, ACM New York, USA

Romero-Zaldívar, V.A., Pardo, A., Burgos, D., Delgado Kloos, C., 2012Monitoring Student Progress Using Virtual Appliances: A Case StudyComputer & Education, 58(4):10580–1067, 2012

Abelardo Pardo The use of data and analytics to guide the improvement of learning experiences 49

References 2Liddo, A. De, Shum, S., Quinto, I., 2011Discourse-centric learning analytics.In Proceedings of the International Conference on Learning Analytics and Knowledge.

Shum, S. B., and Crick, R. D., 2012.Learning Dispositions and Transferable Competencies: Pedagogy, Modelling and Learning Analytics.In S. Buckingham Shum, D. Gaševic, and R. Ferguson (Eds.), International Conference on LearningAnalytics and Knowledge (pp. 92–101). ACM Press.

Bienkowski, M., Feng, M., & Means, B., 2012.Enhancing Teaching and Learning Through Educational Data Mining and Learning Analytics. USDepartment of Education.

Baker, R. S. J. D., and Yacef, K., 2009.The state of educational data mining in 2009: A review and future visions.Journal of Educational Data Mining, 1(1), 3–17.

Wise, A., 2014Designing Pedagogical Interventions to Support Student Use of Learning Analytics.In A. Pardo & S. D. Teasley (Eds.), Proceedings of the International Conference on Learning Analytics andKnowledge. ACM Press.

Abelardo Pardo The use of data and analytics to guide the improvement of learning experiences 50