Engaging with Massive Online Courses

Ashton Anderson , Jure Leskovec Stanford University

Daniel Huttenlocher , Jon kleinberg Cornell University

MOOC introduction

Massive open online course

MOOC platforms :Coursera

Some means: ML1..3and PGM1..3

Main points

Taxonomy of engagement styles base on behavior

Investigate forum relates to participate

Large-scale deployment of badges

Taxonomy of engagement styles

• Fundament activities Viewing lecture Handing in an assignment for credit（ additional activities :upgraded quizzes and forum participation）

• Natural styles of engagement Viewers .primarily watch lectures ,handing few assignment Solvers . primarily hand in assignment viewing few All-rounders . Balance between Collectors . main download Bystanders .registered for course below low threshod.

Taxonomy of engagement styles

Problems:• 1. Not sharp boundaries a ： activity of assignment l: lectures• 2. Engagement and Grades Most students receive zero Different course between ML and PGM(do all course work More challenging)

Taxonomy of engagement styles

Define thresholds c0>=1 and 0<Q0<Q1<1• a Bystander if (a + l) <c0; otherwise, they are• a Viewer or Collector if a/(a + l) <Q0; depending on• whether they primarily viewed or downloaded lectures, respectively,• an All-rounder if Q0 < a/(a + l) < Q1;• a Solver if a/(a + l) >=Q1.

Time of interaction --correlate of their behavior

• Archaeologists： first action in the class is after the end date of the class

Grades and student engagement

• overall final grade distributionin the two classes

Grades and student engagement

• Median number of actions of students with a given final grade in PGM2 and ML2.

Composition of near-perfect students

• Solver , who perhaps know the material,• All-rounder , who watch lecture finish quiz assign

Course forum activity

• Question Which types of students visit the forums?

Course forum activity

Develop an analysis framework can clarify how the forums be used.

1. Clarify the forum conversational structure2.The thread  form of participation3.How Stronger and weaker student interact4.Identify feature based on the context of post

Course forum activity

• If this number is close to k, it means that many students are contributing

• if it is a constant or a slowly growing function of k, then a smaller set of students are contributing repeatedly to the thread.

Course forum activity

Course forum activity

• Estimate a student’s eventual activity level from their forum post.

All forum post for the first two weeks the course. Estimate words (W)

A large-scale badge experiment

• Two large-scale interventions(ML3)

Design and implement badge system

Run randomized experiment that presentation Of badge.

badge system

• Milestone badge :user win badge once they perform amount of activity.

• Badge types: bronze ,silver ,gold and diamond• Award badge types : some actions (cumulative badge),authoring post or thread(accumulative great

achievement)One time badges

badge system• Effects of the badge system on forum

engagement• heavier tail—indicating that more users took more actions• certain features of the distribution were stable prior to the striking

difference exhibited by ML3, in which badges were offered.• Didn’t show qualitatively significant differences in engagement

between the three runs of the class.

Badge Presentation Experiment

• Question:How and why badge produce incentive effects.Do user view the badge as goal to be achieved

for intrinsic personal reasonsWere they viewed as signal social status

Badge Presentation ExperimentBadge treatment conditionsa) Top byline

b) Thread byline

c) Badge ladder

Badge Presentation Experiment

• Effect of badge treatment conditions.

Badge-ladder clearly had the most significant effect. Top-byline and thread-byline were less significant but still

performed better than we’d expect from null treatments

Conclusion

Future work:1.Predictive models of student behavior and grade.2.Persinalization and recommendation mechanisms .3.Further exploring badges.……..

Thank you