How can Cognitive Science improve Online Learning?

Joseph Jay WilliamsJoseph_Williams@berkeley.eduwww.JosephJayWilliams.com/education

CognitiveScience.Co/learnwww.LearningResearch.net

1

Research Landscape

• Qualitative analyses

• Randomized Controlled Trials

• Longitudinal studies

• Educational Data Mining

• Cognitive Science

2

Cognitive Science

3

• Learning: “Adding” vs. “Integrating” information.• Before: Problem-based learning• During: Request Explanations• After: Assessments as instructional tools – Testing & Mixing

• Learning that promotes learning • Motivation• Strategies• Online search

• Resources: • Review of Cognitive Science with practical value: www.josephjaywilliams.com/education• Wiki with resources on Online Education: sites.cognitivescience.co/learn• Contact between researchers & practitioners: www.learningresearch.net

4

Preview

5

Add object

to bucket

Integrate webpage

with internet

Content Exercise

Transfer is rareLearning: Add vs. Integrate

• Consider:– Scenario 1: General invading a fortress can’t use full force, part of force insufficient– Scenario 2: Doctors destroying an internal tumor can’t use strong rays, but weak

rays aren’t enough

• Transfer extremely low (Gick & Holyoak, 1984)

• Motivated MBA students appeared to learn extremely well, but failed to transfer to face-to-face in real life (Gentner, Loewenstein, & Thompson, 2003)

• Transfer is so rare that it’s not a plausible goal (Detterman, 1993)

6

Transfer is rare

• Problem Based Learning (Hmelo-Silver, 2006; Needham & Begg, 1998; Schwartz, 1998)

7

How do you…?Is it possible to…?

Before: Start with Questions & Problems

8

Why…? Free study, Think Aloud…

Learning Measures

(Williams & Lombrozo, 2010)

(Williams, Lombrozo, & Rehder, in press)

(Williams, Walker, Lombrozo, 2012)

During: Request explanations

9

Statistics problem

• Learn a university’s ranking system from examples(Schwartz & Martin, 2004, Belenky & Nokes, 2011)

Sarah was ranked higher.

Rule for ranking John Ranked? Tom

Higher personal score 85% > 69%

Points above average 6% > 4%

Points below maximum –5% > –18%

Number of deviations above average (Z-score)

0.75 < 1.3

85% in HistoryMin of 67%, Max 90% Class Average 79%Standard Deviation 8%

69% in PhysicsMin of 42%, Max 87% Class Average 65%Standard Deviation 3%

>

Accuracy change

10

Single Anomaly Multiple Anomalies0.0

0.1

0.2

0.3

0.4

0.5Free Study

Explain

Accu

racy

impr

ovem

ent f

rom

Pr

e-te

st to

Pos

t-Te

st

Write Thoughts

Williams, Walker, & Lombrozo, 2012

Key Findings

• Explaining increased:• Discovery of principles (Williams & Lombrozo, 2010)

• Use of existing knowledge (Williams & Lombrozo, 2013)

• Explanation’s effect was selective:• Same or even worse memory • Impaired learning if patterns were unreliable (Williams, Lombrozo,

Rehder, in press)

• Similar effects in 5 year olds (Walker, Williams, Lombrozo, & Gopnik, under revision)

Online Mathematics Exercises

• Khan Academy:

• Explain why that solution is correct.

• Here is another student/teacher’s explanation.• Grade both.• Rate how similar they are.

Benefits of explanation

• Instructor guided & learner generated• Learning without feedback• Abstract principles

• The “Testing Effect” (Roediger & Karpicke, 2006)

14

Immediate test: Study+Study ~= Study+TestAfter hours, days, weeks: Study+Study < Study+Test

Learners claim: Study+Study > Study+Test

After: Use Assessments as Instructional Tools

• Mixing Effect (Rohrer, 2009)

• Ten Benefits of Testing (Roediger et al, 2011)

15

After: Efficient Assessments use Mixing Effect

• Support Randomized Experiments or A/B Testing• Precise delivery & control• Quantitative measures of learning

• Ecological validity• Evidence-based decisions• Fidelity• Scalability• Iterative improvement

16

Real-World Laboratories

• Motivation• Learning Strategies• Online search

17

Learning that improves Learning

• Beliefs about intelligence (Dweck, 2006)• Do you agree that…– Your intelligence is something very basic about you that

you can’t change very much. (Fixed Theory).– No matter how much intelligence you have, you can always

change it quite a bit. (Malleable Theory).

• Teach a malleable theory?

18

Increasing motivation

19

Randomized Controlled Trial: Middle, High School, Community College. (Paunesku, Romero et al, 2011; 2012)

Increasing motivation

• Before: Preparatory questions• During: Explanations• After: Applying concept

Currently applying these principles to learning a “Growth Mindset”

Next directions:• Create video versions for MOOCs, Khan Academy, students• Change feedback in Khan Academy exercises

20

Cognitive + Social Psychology

• Teach learning strategies through videos in MOOCs

• Transfer? (Clark & Mayer, 2011)

• Assessment?

• Explanation; Reciprocal Teaching (Palinscar & Brown, 1984)

• Change “theory” of learning• Habit & Behavior Change (Fogg, 2002)

• Technology (Rose et al, 2008)

21

Learning Strategies

• Sophisticated uses• “Theory”: Fishing vs.

Toolkit for Problem-solving

22

Content Exercise

Online search

• Rate the plausibility of each answer.• Predict Accuracy. • Rate similarity.• Grade explanations.

23

1 2 3 4 5 6 7

“Best” vs. “Big” Data

Resources

• Selection of Cognitive Science research applicable to (Online) Education• www.josephjaywilliams.com/education• Wiki with resources on Online Education: sites.cognitivescience.co/learn• Contact between researchers & practitioners: www.learningresearch.net• Comprehensive Wiki & Newsletter on K-12 Ed-Tech www.edsurge.com• Policy Prescriptions for schools National Center for Education and the Economy www.ncee.org Surpassing Shanghai, by Mark Tucker• Institute of Education Sciences “evidence-based education” www.whatworks.ed.gov• E-learning in Industry & Workforce, Online Corporate Training & Development

www.elearningguild.com www.astd.org

Questions?