self-regulated learning: are learners effective and efficient, and how can they be more so?

Self-regulated learning: Are learners effective and efficient, and how can they be more so?

Non-Majors Biology Leadership ConferenceSeptember 29th, 2012, Seattle, WA

Veronica Yan, M.A.University of California, Los Angeles

[email protected]

Self-regulated learning

• An especially relevant topic for college students• Increasingly important in this technological age• And, important across the lifespan, not just college

students

What leads to effective and efficient self-regulated learning?

1. Accurate monitoring of one’s current state of knowledge (monitoring)

• Learned information: Should be able to judge that they can terminate study

• Not yet learned information: Should be able to understand they need to continue studying it

2. Use of effective learning strategies (control)• Quantity of study hours is not sufficient• How that time is spent is critical

Accurate metacognition

Meta-level

Monitor Control

Adapted from Nelson and Naren’s (1990) framework for metamemory. From Bjork, Dunlosky, Kornell (in press).

How do we explore issues of metacognition?

• Ask them• Surveys and questionnaires about study behaviors

and theories about learning• In the lab: during and after learning experiences

• Observe study decisions• Planning study schedule, what to study next, self-

pacing• Mostly in the lab

Outline• What do learners need to know, and do they know it?

• Basics of memory• Focus: a) Testing, and b) Spacing

• Why don’t they know it already? • Current performance vs. long-term retention• Theory-based vs. experience-based cues

• Theories might be inaccurate or not activated• Subjective experiences may be misleading

• Implications for learners and instructors

Understanding memory• Information is not stored like a literal video tape

• New information is related to old information; we have to be an active participant during the learning process

• Memories are not retrieved by hitting a ‘playback’ button• Retrieval involves a reconstructive process (Bartlett,

1932)• Retrieval is cue dependent: Accessibility depends on

current cues

• Retrieval itself has consequences for our memories• Retrieval is a ‘memory modifier’ (Bjork, 1994)

So what leads to long-term learning?

1. Students must put in study time No study, no learning

Benefits of studying

….In the late 1980’s, the population of sea otters in the North Pacific Ocean began to decline. Of the two plausible explanations for the decline—increased predation by killer whales or disease—disease is the more likely. After all, a concurrent sharp decline in the populations of seals and sea lions was almost certainly caused by a pollution related disease, which could have spread to sea otters, whereas the population of killer whales did not change noticeably….

Prose passages (“The Sun” or “Sea Otters”; 256 and 275 words long) from a reading comprehension section of a test-prep book for the TOEFL

Within the text were 30 idea units that were used for the basis of scoring

Roediger & Karpicke (2006)

Study Study Study Study

Study Test Test Test

Final Test

Final Test

5 mins

…

1 week

5 mins

…

1 week

* No feedback given after tests

Benefits of studying

5 mins 5 mins 5 mins 5 mins



Read entire passage 14.2

times


times

Results

5 mins 1 week0.000.100.200.300.400.500.600.700.800.901.00

SSSSSTTT

Retention Interval

Pro

por

tion

Rec

alle

d • 5 mins: Those that studied more, remembered more

---




Final Test

Final Test

5 mins

…

1 week

5 mins

…

1 week



times


times

Benefits of studying?



* No feedback given during tests




Final Test

Final Test

5 mins

…

1 week

5 mins

…

1 week



times


times

Benefits of studying? testing



* No feedback given during tests


Results

5 mins 1 week0.000.100.200.300.400.500.600.700.800.901.00

SSSSSTTT

Retention Interval

Pro

por

tion

Rec

alle

d • 5 mins: Those that studied more, remembered more

• 1 week: Those that were tested retained the information they learned; those that simply read and reread did not


Understanding memory• Information is not stored like a literal video tape

• New information is related to old information; we have to be an active participant during the learning process

• Memories are not retrieved by hitting a ‘playback’ button• Retrieval involves a reconstructive process (Bartlett,

1932)• Retrieval is cue dependent

• Retrieval itself has consequences for our memories• Retrieval is a ‘memory modifier’ (Bjork, 1994), a

potent learning event that strengthens what we retrieve

Do learners understand the benefits of testing?

SSSS STTT3.00

3.50

4.00

4.50

5.00

5.50

6.00

Rat

ing

(1-7

)

Benefits of studying testing

5 mins 1 week0.000.100.200.300.400.500.600.700.800.901.00

SSSSSTTT

Retention Interval

Pro

por

tion

Rec

alle

dRate: How much will you remember in one week’s

time?


Do learners understand the benefits of testing?• Survey evidence :If you quiz yourself while you study (either using a

quiz at the end of a chapter, or a practice quiz, or flashcards, or something else), why do you do so?

a. I learn more that way than I would through rereadingb. To figure out how well I have learned the information I’m

studyingc. I find quizzing more enjoyable than readingd. I usually do not quiz myself

Kornell & Bjork (2007), survey of 472 undergraduates

Do learners understand the benefits of testing?• Survey evidence :If you quiz yourself while you study (either using a

quiz at the end of a chapter, or a practice quiz, or flashcards, or something else), why do you do so?

a. I learn more that way than I would through rereadingb. To figure out how well I have learned the information I’m

studyingc. I find quizzing more enjoyable than readingd. I usually do not quiz myself

Kornell & Bjork (2007), survey of 472 undergraduates

18% 68%

4%9%

Do learners understand the benefits of testing?Which of the following statements do you agree with most: Quizzes are…a. …not useful for anything except to tell me what I do and do not know.

[check only]b. …useful because I learn more for those questions that I answered

incorrectly. [wrong only]c. …useful because I learn more for both questions that I answer

correctly and incorrectly. [all]d. …good only if I get the answers correct. [correct]

Also asked people about their theory of intelligence (Dweck, 1999):

• Fixed: Intelligence is innate, and cannot change• Growth: Intelligence can be increased with effort

Yan, Thai & Bjork (in prep)

Do learners understand the benefits of testing?

Yan, Thai & Bjork (in prep)

Fixed Growth0.00

0.10

0.20

0.30

0.40

0.50

0.60

Check knowledge onlyAll questionsOnly incorrectly answered ques-tionsOnly correctly answered ques-tions

Pro

por

tion

of R

esp

ond

ents

What about errors?

• What is retrieved becomes strengthened• Good, if people get answers correct• Bad, if people get answers wrong

• Consequences of making errors:• The wrong answer becomes strengthened

• Errors persist

• And, having now generated an error, this can compete, or interfere, with the correct answer

What about errors?

“Errorless Learning”

“The two primary reasons for using this approach in education are 1) the students don’t repeatedly make

errors, therefore they don’t establish an “error history” that is later difficult to break; and 2) minimizing errors

reduces emotional and aggressive behavior that can occur following errors, or avoidance of the work altogether. ”

– From a blogpost by Karen Mahon, instructional designer, Ed.d. in Educational Psychology

What about errors?

Kornell, Hays & Bjork (2009)

Whale: _______ Whale: MammalTest first

8 secs 5 secs

Whale: MammalRead- only

13 secs

Final Test

Whale: ???

* Chance of correct guesses during pre-test: 3%

Final Test

Whale: ???

What about errors?

Kornell, Hays & Bjork (2009)

Three Four Five SixExperiment

00.10.20.30.40.50.60.70.80.9

1

Read-onlyTest first

Why?

How can…1) taking time out of one’s study and,2) creating interference,…lead to better learning of the correct answer?

Semantic activation hypothesis: • Generating a guess activates a semantic

network• Which leads to more elaborate encoding of the

correct answer when it is given

What do people think about errors?

Huelser & Metcalfe (2012)

Read Generate0

0.5

1

1.5

2Actual

Believed

Mea

n R

ank

ing

What do people think about errors?

• Even after the final test, (in which they do better on the tested pairs), learners still think that studying is better than testing.

Huelser & Metcalfe (2012)

Read Generate0

0.5

1

1.5

2Actual

Believed

Mea

n R

ank

ing

Does this work in the classroom? Kapur & Bielaczyc (2012)• 7th grade mathematics classes from 3 Singapore

public schools, a range of mathematics ability• Experimental Group: “Productive Failure” (PF):

• 6 periods of generating errors with no instruction from the teacher

• 1 period of actual instruction by the teacher

• Control Group: “Directed Instruction” (DI): • 7 periods of cycling through lectures, practice,

homework and feedback

Does this work in the classroom? Kapur & Bielaczyc (2012)• 7th grade mathematics classes from 3 Singapore

public schools, a range of mathematics ability• Experimental Group: “Productive Failure” (PF):

• 6 periods of generating errors with no instruction from the teacher

• 1 period of actual instruction by the teacher

• Control Group: “Directed Instruction” (DI): • 7 periods of cycling through lectures, practice,

homework and feedback

0%, 7% and 16%

91%, 93% and 92%

Does this work in the classroom? Kapur & Bielaczyc (2012)• Post-test: 3 well-structured problems, one complex

problem, one graphical representation item

Well-structured Complex Representational Flexibility

00.10.20.30.40.50.60.70.80.9

PFDI

Question type

Tes

t P

erfo

rman

ce

*

*

What about errors?

“Errorless Learning”

“The two primary reasons for using this approach in education are 1) the students don’t repeatedly make

errors, therefore they don’t establish an “error history” that is later difficult to break; and 2) minimizing errors

reduces emotional and aggressive behavior that can occur following errors, or avoidance of the work altogether. ”

– From a blogpost by Karen Mahon, instructional designer, Ed.d. in Educational Psychology

Summary: Testing Effect

• Testing is good, not just for assessment, but:• As a learning tool

• Testing involves retrieval, and retrieval itself is a powerful memory modifier

• Even when errors are generated or pre-instruction• As long as corrective feedback is provided• Enables more elaborative encoding of correct

answers; makes connection with pre-existing knowledge

So what leads to long-term learning?

1. Students must put in study time No study, no learning

2. Learning is not just about study time, but alsoi. what you do in that study time, and

e.g., active engagement, elaboration, reorganization, retrieval/testing

ii. how you schedule that study timee.g., repetition can be more or less effective depending on distribution of practice

To space or to mass practice?

• Restudying information:

Study Study Study

Study Study Study

Test

Test

Massed Spaced0.40.50.60.70.80.9 Immediate test


• Restudying information:

Study Study Study

Study Study Study

Test

Test

Massed Spaced0.40.50.60.70.80.9

1

…

…

Massed Spaced0.40.50.60.70.80.9 Immediate test Delayed test

Why is spacing good for LTM

• When people want to learn some information, they study it repeatedly

• How these repetitions are scheduled can make a big difference:• Spacing out repetitions leads to forgetting in

between presentations• Because forgetting has occurred, retrieval now

occurs during subsequent presentations – this retrieval strengthens the learning of the information


• The spacing effect is one of the most robust findings in cognitive psychology • Words, facts • Across seconds, to months and years

• But learners often do not realize this• Item-by-item predictions of future recall

(Zechmeister & Shaughnessy, 1980)

• Massed study makes items feel more fluent

• Questionnaires


Survey:Imagine that in the course of studying, you

become convinced you know the answer to a certain question. What would you do?

• Make sure to study it (or test yourself on it) again later

• Put it aside and focus on other material

36%

64%

Kornell & Bjork (2007)

Spacing works for repetitions; but what about generalization?

• Spacing is great for strengthening information that you’ve studied before

• What is taught in the classroom, especially in the sciences, is more complex concepts

• Not just a memorization of facts, but students must learn more general principles and a conglomerate of related concepts

Inductive learning• Ability to generalize concepts and categories

from exposure to multiple exemplars, and apply to new exemplars

• A natural option then is to teach one concept at a time, immerse students in one lesson so that they can understand it more fully.

Gentoo

Where’s the Gentoo?

• Blocking/massing allows the learner to notice characteristics that unify a category

• Interleaving/spacing makes doing so difficultGentoo Gentoo Gentoo Gentoo

Gentoo Lachesis Reinhard Gentoo

Hypothesis

“Spacing is the friend of recall but the enemy of induction.”

-Ernst Rothkopf

Kornell & Bjork (2008): Artists

You will be shown 72 paintings for 3 seconds each. The paintings will be by twelve artists, with six pictures per artist. Try to learn to recognize which artist painted which picture based on their style.

Later, you’ll be shown 48 new paintings, which you haven’t seen before. You’ll have to identify who painted each one.

BLOCKED

INTERLEAVED

Pessani

Wexler

Schlorff

Stratulat

Hawkins

Mylrea

FeedbackTest

Blocked Interleaved0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

Results

Actual Responses

Which do you think helped you learn the artists’ styles better?

Blocked About the same Interleaved

Blocked Same Interleaved0

0.10.20.30.40.50.60.70.8

Blocked Same Interleaved0

0.10.20.30.40.50.60.70.8

• Original hypothesis: Blocking/massing highlights similarities

• New hypothesis: Interleaving/spacing highlights differences

LewisLewis Schlorff Hawkins

LewisLewis Lewis Lewis

Interleaving benefits differentiation

Interleaving benefit is robust

• The interleaving benefit has now been repeatedly demonstrated • In children (Vlach, Sandhofer, & Kornell, 2008)

• In older adults (Kornell, Castel, Eich, & Bjork, 2010)

• Induction of bird families (Wahlheim, Dunlosky & Jacoby, 2011)

• Induction of human voices (Yan, Vetter, & Bjork, in prep)

• Mathematics concepts (Rohrer & Taylor, 2007; Taylor & Rohrer, 2010)

Interleaving Math Learning4 types of problems

Taylor & Rohrer (2010)

Interleaving Math Learning

Taylor & Rohrer (2010)

100%

81%

Do learners understand interleaving?• After experience: No, not even after they have

taken a test!• What about a priori beliefs? • Tauber, Dunlosky, Rawson, Wahlheim, and

Jacoby (2012):• Ss first familiarized with 6 different birds from

8 bird families • 30 minutes to study in the order they chose

Do learners understand interleaving?

% birds massed:% Ss massed:Run length:

82%78%5.0 birds/family

# birds selected: 57

Tauber et al. (2012)

Summary: Spacing and Interleaving

• Distributing and interleaving practice is beneficial for learning• Spacing benefits learning by introducing

retrieval • Interleaving benefits learning by highlighting

differences between categories, enabling discrimination• Interleaving concepts within a topic is good• Evidence is less clear for interleaving different

topics

What are effective study strategies?

• Testing vs. re-studying• Spacing vs. massing• Interleaving vs. blocking

“Desirable Difficulties”(Bjork, 1994)

• Learning NOT made easy, but rather more effortful

However, learners often do not understand this!


• Basics of memory• Focus: a) Testing and, b) Spacing


• Theories might be inaccurate or not activated• Subjective experiences may be misleading


Current performance vs. long-term retention• Performance during study can be an unreliable

index of long-term retention• When study is massed or blocked, current

performance can look great, and learning can feel very easy

• However, this does not guarantee great long-term retention

• Learners often judge learning on current performance…

• …And instructors too!




• Theories might be inaccurate• Subjective experiences may be misleading


Why are people inaccurate?By the time students enter college, they have a good two decades or so of learning under their belts! • Have developed beliefs about their memories

• Theories inaccurate• Theories not activated, e.g.,

• That more studying leads to more learning (Kornell & Bjork, 2009)

• That we forget over time (Koriat, Bjork, Sheffer, & Bar, 2004)

Stability bias

Learners are very sensitive to item relatedness

But insensitive to retention interval

Why are people inaccurate?By the time students enter college, they have a good two decades or so of learning under their belts! • Have developed beliefs about their memories

• Theories inaccurate• Theories not activated, e.g.,

• That more studying leads to more learning (Kornell & Bjork, 2009)

• That we forget over time (Koriat, Bjork, Sheffer, & Bar, 2004)

• Subjective experience at the time of study misleads judgments

If it feels easy, it must be!• How can people forget about forgetting?

• Foresight biasLight: Lamp

Light: ??(Koriat & Bjork, 2005)

• Fluency: Subjective ease of processing• Might be based upon valid cues, e.g. how familiar

material is• But may also be based upon invalid cues, e.g.

perceptual fluency

Fluency

television

Prediction?

Rhodes & Castel (2008)

Fluency

camel

Prediction?


Fluency

radio

Prediction?


Fluency


PredictionRecall

Individual differences in fluency effects• Is everyone prey to the heuristic that ‘easily learned =

easily remembered’? • Replicated the

font size study

• Compared fixed and growth theorists

Miele, Finn & Molden (2011)

Pre

dict

ed L

earn

ing

Growth TheoristsFixed Theorists

Summary: Why don’t learners know better? • Disconnect between current performance and

long-term retention• Current performance may be high for reasons

unrelated to real learning• Subjective experience can mislead learners to

• Develop inaccurate theories of learning• Not activate accurate theories of learning• Be fooled by fluency

• Not everyone is fooled by fluency (growth theorists may be less susceptible)




• Theories might be inaccurate• Subjective experiences may be misleading


Implications for instructors and for learnersFour take-home points to remember if you forget everything else: • Frequent testing• Spacing/Interleaving• Performance/fluency can be misleading• Attitudes toward learning

Implications for instructors and for learnersFour take-home points to remember if you forget everything else: • Frequent testing

• For instructors: Low stakes tests, used as pedagogical tools rather than simply assessment; doesn’t have to be time-consuming – mini clicker quizzes

• For learners: Self-testing

Implications for instructors and for learnersFour take-home points to remember if you forget everything else: • Frequent testing• Spacing/Interleaving

• For instructors: • Most textbook teaching: Learn one concept, answer

many questions on that concept; then move onto another

• Better? End-of-chapter questions should intermix questions based upon earlier chapters; revisit old material

• For students: Interleave study; intermix modules

Implications for instructors and for learnersFour take-home points to remember if you forget everything else: • Frequent testing• Spacing/Interleaving• Performance/fluency can be misleading

• For instructors and learners: Understand that current performance is not always a good index of long-term learning

• For learners: Self-testing without study aids

Implications for instructors and for learnersFour take-home points to remember if you forget everything else: • Frequent testing• Spacing/Interleaving• Performance/fluency can be misleading• Attitudes toward learning

For [instructors and] learners: • Appreciating that intelligence can be increased with

effort (growth mindset) can affect susceptibility to fluency effects and lead to better study practices

• Appreciate that there are study techniques that can improve long-term retention, and which apply to all learners

The good news

• We all have an enormous capacity to learn, and there is great potential to upgrade self-regulated learning

• There is now a great amount of literature on how to improve learning and memory

• These cognitive principles apply to all learners

• Exciting time to be an instructor! • And now I’m excited to hear from you!

Thank you! Acknowledgments: Robert BjorkElizabeth BjorkBjork Human Learning and Forgetting Lab

James S McDonnell FoundationPearson Education

self-regulated learning: are learners effective and efficient, and how can they be more so?

Documents

srl learning

learning time

lifespan learning

sophisticated learning

study time

learners effective

study behaviors

selfregulated learningan