mooc research at tu delft

MOOC RESEARCH AT TU DELFTTHE LEARNING ENVIRONMENT & BEYOND

DAN DAVIS TU DELFT WEB INFORMATION SYSTEMS GROUP, LAMBDA LAB LAMBDA-LAB

WHO ARE WE?

Guanliang Chen PhD Researcher

Dan Davis PhD Researcher

Claudia Hauff Assistant Prof.

Geert-Jan Houben Professor

Research backgrounds in user modeling, information retrieval, big data processing and learning technologies.

OUR GOALS1. Gain actionable insights into learner behaviors at scale 2. Increase our knowledge about learners by looking

beyond the learning platform 3. Design and implement interventions that enable

adaptive learning at scale

OUTLINE1. Beyond the learning platform · WebSci ’16 2. Learning transfer · L@S ’16 3. Personality in MOOCS · UMAP ’16 4. Learners -> Earners · In Progress 5. Learning Paths · EDM ’16 6. Study Strategies · EC-TEL ’16 7. Feedback- Pilot · LAK ‘16 Workshop 8. Adaptive Feedback · In Progress

Learn

er Mo

delin

gSR

L & Su

pport

1. BEYOND THE LEARNING PLATFORM

· Can we link our learners to their social web accounts? · Do social Web platforms enable us to observe user attributes relevant to the online learning experience?

Guanliang Chen, Dan Davis, Jun Lin, Claudia Hauff, and Geert-Jan Houben. Beyond the MOOC platform: Gaining Insights about Learners from the Social Web. ACM WebScience, pp. 15-24, 2016.

Over 1M learners

2—42% ID’d / course

2. LEARNING TRANSFER:

Major findings: · A small subset of learners (8%) display learning transfer. · Existing learning transfer findings from the workplace & classroom setting mostly also hold in MOOCs.

Guanliang Chen, Dan Davis, Claudia Hauff, and Geert-Jan Houben, Learning Transfer: does it take place in MOOCs?, ACM Learning At Scale, pp. 409-418, 2016.

do learners apply what they learned in practice?

3. PERSONALITY IN ONLINE LEARNING

Guanliang Chen, Dan Davis, Claudia Hauff, and Geert-Jan Houben. On the Impact of Personality in Massive Open Online Learning. ACM UMAP 2016.

Also to appear at LWMOOCs III 2016.

· Does personality impact learner engagement, learner behavior and learner success in the context of MOOCs? · Can learners’ personalities be predicted based on their behaviors exhibited in a MOOC platform?

4. FROM LEARNERS TO EARNERS· Are MOOC learners able to solve real-world paid tasks from an online work platform with sufficient quality?

(In Press) Guanliang Chen, Dan Davis, Markus Krause, Efthimia Aivaloglou, Claudia Hauff, and Geert-Jan Houben. From Learners to Earners: Enabling MOOC Learners to Apply their Skills and Earn Money in an Online Market Place.

· Currently building recommender system for scalable implementation in TU DelftX Data Analysis MOOC

5. EXECUTED VS. DESIGNED LEARNING PATH

Dan Davis, Guanliang Chen, Claudia Hauff, and Geert-Jan Houben. Gauging MOOC Learners’ Adherence to the Designed Learning Path. EDM 2016.

To what extent do learners adhere to a MOOC’s designed learning path?

1. VIDEO INTERACTIONS2. BEHAVIOR PATTERN CHAINS3. EVENT TYPE TRANSITIONS

1. VIDEO INTERACTIONSDO LEARNERS WATCH VIDEO LECTURES IN THE PRESCRIBED ORDER?

VIDEO INTERACTIONS

Non-Passing

Passing

Week 1 Week 2Week 3

FP101xDesigned Lecture Order

Executed Paths

2. BEHAVIOR PATTERN CHAINSWHAT ARE THE MOST COMMON SEQUENCES IN THE EXECUTED LEARNING PATHS?

VIDEO QUIZ PROGRESS FORUM

WATCH START VIEW STARTSUBMITEND

SUBMITEND

EIGHT-STEP CHAINFORUMEND

EXAMPLE

LECTUREWATCH QUIZSTART QUIZSUBMIT QUIZSUBMIT

QUIZEND PROGRESS LECTUREWATCH

MOTIFA RECURRING OR REPEATED ELEMENT

MOTIF FREQTOTAL

FREQPASS

FREQFAIL

QUIZ COMPLETE 552,363(29.4%)

328,995(30.8%)

223,368(27.7%)

1.

BINGE WATCHING 149,784(8%)

59,498(5.6%)

90,286(11.2%)

2.

LECTURE -> QUIZ COMPLETE 100,179(5.3%)

50,415(4.7%)

49,764(6.2%)

3.

QUIZ COMPLETE -> FORUM 99,828(5.3%)

67,722(6.3%)

32,106(4%)

4.

FP101x

XQUIZ events only with at least 1 X = SUBMIT

WATCH events only

WATCH event(s) followed by XQUIZ events w/ >1 X = SUBMIT

XQUIZ events w/ >1 X = SUBMIT followed by XFORUM events

3. EVENT TYPE TRANSITIONSHOW DO STUDENTS NAVIGATE BETWEEN CERTAIN EVENT TYPES?

EVENT TYPE TRANSITIONS

Frame101x Non-PassingMarkov ModelState Visualization

(EXECUTED)

EVENT TYPE TRANSITIONS

Frame101x Non-Passing

(EXECUTED)

Frame101x Passing

EVENT TYPE TRANSITIONS(EXECUTED)

Frame101x Non-Passing Frame101x Passing32%

35%

VIDEO

QUIZEND

QUIZSUBMIT

FORUMSUBMIT

FORUMEND

PROGRESS

FORUMSTART

QUIZSTART VIDEO

QUIZEND QUIZ

SUBMIT

FORUMSUBMIT

FORUMEND

PROGRESS

FORUMSTART

QUIZSTART

21%28%

44%

44%

6. RETRIEVAL PRACTICE & STUDY PLANNING IN MOOCS

D. Davis, G. Chen, Tim v.d. Zee, C. Hauff, and G.J. Houben. Retrieval Practice and Study Planning in MOOCs: Exploring Classroom-Based Self-Regulated Learning Strategies at Scale. ECTEL 2016.

· To what extent can findings from the learning sciences apply to MOOC learners?

· Do learners engage with self-regulated learning (SRL) interventions?

· Does the engagement have an effect on learners’ test performance?



Retrieval Practice Study Planning



Retrieval Practice Study PlanningRESULTS: NONCOMPLIANCE

(NEED MORE ENGAGING INTERVENTIONS)

7. PERSONALIZED FEEDBACK @ SCALE

Dan Davis, Guanliang Chen, Ioana Jivet, Claudia Hauff, and Geert-Jan Houben. Encouraging Metacognition & Self-Regulation in MOOCs through Increased Learner Feedback. LAL Workshop 2016.

ENCOURAGING METACOGNITION & SRL THROUGH INCREASED LEARNER FEEDBACKDo learners change their behavior when confronted with their learning performance relative to that of successful learners?

8. PERSONALIZED FEEDBACK @ SCALE

(In Progress)

ENCOURAGING METACOGNITION & SRL THROUGH INCREASED LEARNER FEEDBACKDo learners change their behavior when confronted with their learning performance relative to that of successful learners?

LOOKING FORWARD· NEW INNOVATIONS IN FEEDBACK

-> What & how do learners want & need -> Value in cultural context?

· PATTERN MINING -> Positive deviance -> New sub-groupings (beyond passing v. non-passing) -> How can we nudge to a “better” path (& should we?)

· BEYOND THE PLATFORM -> Where can we look to gain meaningful learner insights? -> What can we do with this information?

THANK YOU

BIT.LY/WIS-LEARNING-ANALYTICS

LAMBDA-LAB

DAN DAVIS TU DELFT WEB INFORMATION SYSTEMS GROUP, LAMBDA LAB

mooc research at tu delft

Education