On the Impact of Personalityin Massive Open Online Learning

Guanliang Chen, Dan Davis, Claudia Hauff, Geert-Jan Houben

Web Information Systems, TU Delft

Whythis research?

PersonalityMOOC-basedlearning

Classroom-based learning

- Conscientious learners are likely to succeed.

- …matters

MOOCs

Whythis research?

PersonalityMOOC-basedlearning

Classroom-based learning

- Conscientious learners are likely to succeed.

- …matters

Do learners' personalities impact their learning &

learning behaviour in the MOOC setting?

MOOCs

What research questions?

What research questions?

1 Does personality impact learner engagement, learner behaviour and learner success in the context of MOOCs?

What research questions?

1 Does personality impact learner engagement, learner behaviour and learner success in the context of MOOCs?

Can learners' personalities be predicted based on their behaviour exhibited on a MOOC platform? 2

MethodData

Gathering

FeatureEngineering

CorrelationAnalysis

Prediction

MethodData

Gathering

FeatureEngineering

CorrelationAnalysis

Prediction

MOOCEX101x +

Learningdata

PersonalityTest

CourseSurveys

Priorknowledge

Personalityinformation

MethodData

Gathering

FeatureEngineering

CorrelationAnalysis

Prediction

Big Five Personality Test

Extraversion

Conscientiousness Being organised, disciplined and achievement-oriented

A high degree of sociability and assertiveness

Agreeableness Being helpful to others, cooperative and sympathetic

Neuroticism Emotional stability, the level of anxiety and impulse control

Openness Being intellectual curious and prefer variety and novelty

MethodData

Gathering

FeatureEngineering

CorrelationAnalysis

Prediction

Big Five Personality Test

Extraversion

Conscientiousness Being organised, disciplined and achievement-oriented

A high degree of assertiveness

Agreeableness Being cooperative

Neuroticism Emotional stability,anxiety

Openness Being prefer

MethodData

Gathering

FeatureEngineering

CorrelationAnalysis

Prediction

Big Five Personality Test

Extraversion

Conscientiousness Being and

A high degree of assertiveness

Agreeableness Being cooperative

Neuroticism Emotional stability,anxiety

Openness Being intellectual curious and prefer variety and novelty

MethodData

Gathering

FeatureEngineering

CorrelationAnalysis

Prediction

MethodData

Gathering

FeatureEngineering

CorrelationAnalysis

Prediction

Video-based features- Time watching video material - # videos skipped - …

Quiz-based features- Time solving quizzes - # Questions learners attempted to solve - …

Forum-based features- # New forum questions - # Forum replies - …

MethodData

Gathering

FeatureEngineering

CorrelationAnalysis

Prediction

Video-based features- Time watching video material - # videos skipped - …

Quiz-based features- Time solving quizzes - # Questions learners attempted to solve - …

Forum-based features- # New forum questions - # Forum replies - …

In total, there are 20 features.

MethodData

Gathering

FeatureEngineering

CorrelationAnalysis

Prediction

MethodData

Gathering

FeatureEngineering

CorrelationAnalysis

Prediction

Low prior-knowledge group

High prior-knowledge groupLearners

MethodData

Gathering

FeatureEngineering

CorrelationAnalysis

Prediction

18 features 5 personality traits

Spearman’s rankcorrelation analysis

Low prior-knowledge group

High prior-knowledge groupLearners

MethodData

Gathering

FeatureEngineering

CorrelationAnalysis

Prediction

MethodData

Gathering

FeatureEngineering

CorrelationAnalysis

Prediction

Learning features

Personality traits

Input

RegressionModel

Output

Gaussian Processes

Ramdom Forests

ResultsCorrelation

Analysis

Prediction Accuracy

Prediction Accuracy

ResultsCorrelation

Analysis

Low prior-knowledge group E A C N OTime watching video material (in min.) 0.00 -0.04 0.15 0.03 -0.18

Time solving quizes (in min.) -0.02 0.02 0.07 0.02 -0.18# Questions learners attempted to solve -0.04 -0.04 0.15 0.03 -0.17

# New forum questions 0.07 0.04 0.01 0.04 0.00# Forum replies 0.12 0.12 0.00 0.01 0.03

# Total forum postings 0.11 0.10 0.03 0.03 0.02Forum browsing time -0.10 0.00 0.06 -0.04 -0.13

Forum accesses -0.11 -0.04 0.06 -0.05 -0.16# Forum interactions 0.10 0.11 0.03 0.04 0.03

Total time on-site -0.02 -0.03 0.12 0.01 -0.19Average time responded to videos 0.09 -0.04 -0.04 -0.06 -0.1Average time responded to quizs 0.03 0 -0.14 -0.1 -0.15

# Videos skipped 0.02 0.07 -0.14 -0.04 0.18# Videos sped up 0.00 -0.03 0.10 -0.01 -0.02

Maximum session time -0.02 -0.05 0.10 0.00 -0.17Average session time 0.00 -0.03 0.05 -0.01 -0.08

Standard deviation session time 0.04 -0.01 0.11 0.02 -0.16Average between-quizzes time 0.03 0.10 0.00 0.02 -0.10

Standard deviation between-quizzes time 0.01 0.06 0.04 0.01 -0.14Final score -0.06 -0.07 0.12 0.07 -0.15

Prediction Accuracy

ResultsCorrelation

Analysis

Low prior-knowledge group E A C N OTime watching video material (in min.) 0.00 -0.04 0.15 0.03 -0.18

Time solving quizes (in min.) -0.02 0.02 0.07 0.02 -0.18# Questions learners attempted to solve -0.04 -0.04 0.15 0.03 -0.17

# New forum questions 0.07 0.04 0.01 0.04 0.00# Forum replies 0.12 0.12 0.00 0.01 0.03

# Total forum postings 0.11 0.10 0.03 0.03 0.02Forum browsing time -0.10 0.00 0.06 -0.04 -0.13

Forum accesses -0.11 -0.04 0.06 -0.05 -0.16# Forum interactions 0.10 0.11 0.03 0.04 0.03

Total time on-site -0.02 -0.03 0.12 0.01 -0.19Average time responded to videos 0.09 -0.04 -0.04 -0.06 -0.1Average time responded to quizs 0.03 0 -0.14 -0.1 -0.15

# Videos skipped 0.02 0.07 -0.14 -0.04 0.18# Videos sped up 0.00 -0.03 0.10 -0.01 -0.02

Maximum session time -0.02 -0.05 0.10 0.00 -0.17Average session time 0.00 -0.03 0.05 -0.01 -0.08

Standard deviation session time 0.04 -0.01 0.11 0.02 -0.16Average between-quizzes time 0.03 0.10 0.00 0.02 -0.10

Standard deviation between-quizzes time 0.01 0.06 0.04 0.01 -0.14Final score -0.06 -0.07 0.12 0.07 -0.15

Only openness to experience and conscientiousness are significantly correlated with a number of features.

Prediction Accuracy

ResultsCorrelation

Analysis

Low prior-knowledge group E A C N OTime watching video material (in min.) 0.00 -0.04 0.15 0.03 -0.18

Time solving quizes (in min.) -0.02 0.02 0.07 0.02 -0.18# Questions learners attempted to solve -0.04 -0.04 0.15 0.03 -0.17

# New forum questions 0.07 0.04 0.01 0.04 0.00# Forum replies 0.12 0.12 0.00 0.01 0.03

# Total forum postings 0.11 0.10 0.03 0.03 0.02Forum browsing time -0.10 0.00 0.06 -0.04 -0.13

Forum accesses -0.11 -0.04 0.06 -0.05 -0.16# Forum interactions 0.10 0.11 0.03 0.04 0.03

Total time on-site -0.02 -0.03 0.12 0.01 -0.19Average time responded to videos 0.09 -0.04 -0.04 -0.06 -0.1Average time responded to quizs 0.03 0 -0.14 -0.1 -0.15

# Videos skipped 0.02 0.07 -0.14 -0.04 0.18# Videos sped up 0.00 -0.03 0.10 -0.01 -0.02

Maximum session time -0.02 -0.05 0.10 0.00 -0.17Average session time 0.00 -0.03 0.05 -0.01 -0.08

Standard deviation session time 0.04 -0.01 0.11 0.02 -0.16Average between-quizzes time 0.03 0.10 0.00 0.02 -0.10

Standard deviation between-quizzes time 0.01 0.06 0.04 0.01 -0.14Final score -0.06 -0.07 0.12 0.07 -0.15

Only openness to experience and conscientiousness are significantly correlated with a number of features.

Conscientious learners (self-disciplined, striving for success) are likely to be more engaged with more learning materials.

Prediction Accuracy

ResultsCorrelation

Analysis

Low prior-knowledge group E A C N OTime watching video material (in min.) 0.00 -0.04 0.15 0.03 -0.18

Time solving quizes (in min.) -0.02 0.02 0.07 0.02 -0.18# Questions learners attempted to solve -0.04 -0.04 0.15 0.03 -0.17

# New forum questions 0.07 0.04 0.01 0.04 0.00# Forum replies 0.12 0.12 0.00 0.01 0.03

# Total forum postings 0.11 0.10 0.03 0.03 0.02Forum browsing time -0.10 0.00 0.06 -0.04 -0.13

Forum accesses -0.11 -0.04 0.06 -0.05 -0.16# Forum interactions 0.10 0.11 0.03 0.04 0.03

Total time on-site -0.02 -0.03 0.12 0.01 -0.19Average time responded to videos 0.09 -0.04 -0.04 -0.06 -0.1Average time responded to quizs 0.03 0 -0.14 -0.1 -0.15

# Videos skipped 0.02 0.07 -0.14 -0.04 0.18# Videos sped up 0.00 -0.03 0.10 -0.01 -0.02

Maximum session time -0.02 -0.05 0.10 0.00 -0.17Average session time 0.00 -0.03 0.05 -0.01 -0.08

Standard deviation session time 0.04 -0.01 0.11 0.02 -0.16Average between-quizzes time 0.03 0.10 0.00 0.02 -0.10

Standard deviation between-quizzes time 0.01 0.06 0.04 0.01 -0.14Final score -0.06 -0.07 0.12 0.07 -0.15

Only openness to experience and conscientiousness are significantly correlated with a number of features.

Learners with high openness (being intellectually curious, prefer variety) are less likely to spend time on the learning material.

Conscientious learners (self-disciplined, striving for success) are likely to be more engaged with more learning materials.

Prediction Accuracy

Correlation Analysis

ResultsHigh prior-knowledge group E A C N OTime watching video material (in min.) -0.08 -0.07 0.09 0.05 -0.01

Time solving quizes (in min.) -0.09 -0.10 0.10 0.04 -0.03# Questions learners attempted to solve -0.13 -0.08 0.08 0.00 -0.03

# New forum questions -0.04 0.04 0.10 -0.03 0.03# Forum replies -0.02 0.03 0.15 0.08 0.03

# Total forum postings -0.03 0.02 0.15 0.02 0.03Forum browsing time -0.11 -0.04 0.02 -0.03 -0.06

Forum accesses -0.14 -0.06 0.03 -0.04 -0.04# Forum interactions -0.02 0.02 0.15 0.03 0.03

Total time on-site -0.07 -0.07 0.11 0.04 -0.03Average time responded to videos 0.05 -0.02 -0.01 0.06 0.03Average time responded to quizs 0.03 -0.05 0 0.05 0.02

# Videos skipped 0.09 0.08 -0.09 -0.04 0.00# Videos sped up 0.03 0.00 0.06 0.09 0.06

Maximum session time -0.04 -0.04 0.11 0.04 -0.04Average session time 0.06 -0.05 0.03 0.06 -0.04

Standard deviation session time -0.03 -0.07 0.12 0.04 -0.04Average between-quizzes time 0.00 -0.06 0.09 0.06 0.00

Standard deviation between-quizzes time -0.03 -0.07 0.09 0.07 0.00Final score -0.12 -0.05 0.07 0.01 -0.01

Prediction Accuracy

Correlation Analysis

ResultsHigh prior-knowledge group E A C N OTime watching video material (in min.) -0.08 -0.07 0.09 0.05 -0.01

Time solving quizes (in min.) -0.09 -0.10 0.10 0.04 -0.03# Questions learners attempted to solve -0.13 -0.08 0.08 0.00 -0.03

# New forum questions -0.04 0.04 0.10 -0.03 0.03# Forum replies -0.02 0.03 0.15 0.08 0.03

# Total forum postings -0.03 0.02 0.15 0.02 0.03Forum browsing time -0.11 -0.04 0.02 -0.03 -0.06

Forum accesses -0.14 -0.06 0.03 -0.04 -0.04# Forum interactions -0.02 0.02 0.15 0.03 0.03

Total time on-site -0.07 -0.07 0.11 0.04 -0.03Average time responded to videos 0.05 -0.02 -0.01 0.06 0.03Average time responded to quizs 0.03 -0.05 0 0.05 0.02

# Videos skipped 0.09 0.08 -0.09 -0.04 0.00# Videos sped up 0.03 0.00 0.06 0.09 0.06

Maximum session time -0.04 -0.04 0.11 0.04 -0.04Average session time 0.06 -0.05 0.03 0.06 -0.04

Standard deviation session time -0.03 -0.07 0.12 0.04 -0.04Average between-quizzes time 0.00 -0.06 0.09 0.06 0.00

Standard deviation between-quizzes time -0.03 -0.07 0.09 0.07 0.00Final score -0.12 -0.05 0.07 0.01 -0.01

Again, no significant correlation between behavioural features and agreeableness and neuroticism can be observed.

Prediction Accuracy

Correlation Analysis

ResultsHigh prior-knowledge group E A C N OTime watching video material (in min.) -0.08 -0.07 0.09 0.05 -0.01

Time solving quizes (in min.) -0.09 -0.10 0.10 0.04 -0.03# Questions learners attempted to solve -0.13 -0.08 0.08 0.00 -0.03

# New forum questions -0.04 0.04 0.10 -0.03 0.03# Forum replies -0.02 0.03 0.15 0.08 0.03

# Total forum postings -0.03 0.02 0.15 0.02 0.03Forum browsing time -0.11 -0.04 0.02 -0.03 -0.06

Forum accesses -0.14 -0.06 0.03 -0.04 -0.04# Forum interactions -0.02 0.02 0.15 0.03 0.03

Total time on-site -0.07 -0.07 0.11 0.04 -0.03Average time responded to videos 0.05 -0.02 -0.01 0.06 0.03Average time responded to quizs 0.03 -0.05 0 0.05 0.02

# Videos skipped 0.09 0.08 -0.09 -0.04 0.00# Videos sped up 0.03 0.00 0.06 0.09 0.06

Maximum session time -0.04 -0.04 0.11 0.04 -0.04Average session time 0.06 -0.05 0.03 0.06 -0.04

Standard deviation session time -0.03 -0.07 0.12 0.04 -0.04Average between-quizzes time 0.00 -0.06 0.09 0.06 0.00

Standard deviation between-quizzes time -0.03 -0.07 0.09 0.07 0.00Final score -0.12 -0.05 0.07 0.01 -0.01

Again, no significant correlation between behavioural features and agreeableness and neuroticism can be observed.

Conscientious learners are likely to be active in forum discussion, while extraneous leaners not.

Prediction Accuracy

Correlation Analysis

ResultsHigh prior-knowledge group E A C N OTime watching video material (in min.) -0.08 -0.07 0.09 0.05 -0.01

Time solving quizes (in min.) -0.09 -0.10 0.10 0.04 -0.03# Questions learners attempted to solve -0.13 -0.08 0.08 0.00 -0.03

# New forum questions -0.04 0.04 0.10 -0.03 0.03# Forum replies -0.02 0.03 0.15 0.08 0.03

# Total forum postings -0.03 0.02 0.15 0.02 0.03Forum browsing time -0.11 -0.04 0.02 -0.03 -0.06

Forum accesses -0.14 -0.06 0.03 -0.04 -0.04# Forum interactions -0.02 0.02 0.15 0.03 0.03

Total time on-site -0.07 -0.07 0.11 0.04 -0.03Average time responded to videos 0.05 -0.02 -0.01 0.06 0.03Average time responded to quizs 0.03 -0.05 0 0.05 0.02

# Videos skipped 0.09 0.08 -0.09 -0.04 0.00# Videos sped up 0.03 0.00 0.06 0.09 0.06

Maximum session time -0.04 -0.04 0.11 0.04 -0.04Average session time 0.06 -0.05 0.03 0.06 -0.04

Standard deviation session time -0.03 -0.07 0.12 0.04 -0.04Average between-quizzes time 0.00 -0.06 0.09 0.06 0.00

Standard deviation between-quizzes time -0.03 -0.07 0.09 0.07 0.00Final score -0.12 -0.05 0.07 0.01 -0.01

Again, no significant correlation between behavioural features and agreeableness and neuroticism can be observed.

Overall, behavioural features extracted from MOOC log traces are less correlated with personality traits than lexical or behavioural features

extracted from social media traces.

Conscientious learners are likely to be active in forum discussion, while extraneous leaners not.

Prediction Accuracy

Correlation Analysis

ResultsEvaluation: Spearman’ rank correlation coefficient

Prediction Accuracy

Correlation Analysis

ResultsEvaluation: Spearman’ rank correlation coefficient

Significant correlations (indicating usable predictions) are achieved for 4 of the 5 personality traits.

Prediction Accuracy

Correlation Analysis

ResultsEvaluation: Spearman’ rank correlation coefficient

Significant correlations (indicating usable predictions) are achieved for 4 of the 5 personality traits.

The correlation coefficients tend to increase with increasing course weeks.

Prediction Accuracy

Correlation Analysis

ResultsEvaluation: Spearman’ rank correlation coefficient

Significant correlations (indicating usable predictions) are achieved for 4 of the 5 personality traits.

The correlation coefficients tend to increase with increasing course weeks.

Extraversion and neuroticism achieve the highest prediction accuracy.

Summary

Summary1 Various learning features are correlated with

the personality traits of conscientiousness and openness.

Summary1 Various learning features are correlated with

the personality traits of conscientiousness and openness.

The predictions are statistically significant for 4 out of the 5 personality traits and improve as more data becomes available. 2

http://bit.ly/wis-learning-analytics

Thanks for your attention!