Kerry Lee, PhD.

Head of Research

Educational & Cognitive Development Lab

National Institute of Education

Applied Cognitive Development Lab

2

Cognitive

underpinnings

of math

proficiency

Study 1:

Individual

differences

in algebraic

problem

solving

Study 2:

Influence

of

executive

functioning Study 3:

Development of

working memory,

executive

functioning &

math abilities

Intervention

study

Behavioral &

fMRI study of

the role of

inhibitory

functions in the

acquisition of

more advanced

strategies

The impact

of

executive

interference

Working

memory,

test anxiety,

and math

performance Visual-

spatial

short term

vs.

working

memory

Algebraic

strategies

Teachers’

perception

of different

algebraic

strategies

Pupils’

understanding

of model

solutions

fMRI study of

strategic

differences I & II

Content

Relations of working memory, executive

functions and academic performance

Development of executive functions

Improving updating capacity

Math Performance in Singapore

Singapore has performed well in

international comparisons of

mathematics achievement

Trends in International Mathematics and

Science Study

Programme for International Student

Assessment

Around 5.5% of children struggle with

math on entry to primary schools

Contributing Variables

System

Societal expectation

Education system

Effort and quality of teachers

Individual

Social or motivational (e.g., Ashcraft, Kirk, &

Hopko, 1998)

Biological (see Geary, 1993, for a review)

Cognitive

Working Memory at Work

259 + 36 = ?

764 / 4 = ?

Sir Humphrey's longest sentence

from “Yes, Minister!” "Well, it's clear that the committee has

agreed that your new policy is a really

excellent plan but in view of some of

the doubts being expressed, may I

propose that I recall that after careful

consideration, the considered view of

the committee was that while they

considered that the proposal met with

broad approval in principle, that some

of the principles were sufficiently

fundamental in principle and some of

the considerations so complex and

finely balanced in practice, that, in

principle, it was proposed that the

sensible and prudent practice would be

to submit the proposal for more detailed

consideration, ...”.

Theories of Working Memory

Close relations between attention and WM Multiple component

model ○ Baddeley & Logie,

1999, Baddeley & Hitch, 1974)

Embedded processes model ○ (Cowan, 1988, 1999)

Controlled attention network ○ (Engle, Kane, &

Tuholski, 1999)

ACT-R model ○ (Lovett, Reder, &

Lebiere, 1999, Anderson, Reder, & Lebiere, 1996)

Symbolic computational models Executive

process/interactive control model ○ (Kieras, Meyer,

Mueller, & Seymour, 1999, Meyer & Kieras, 1997)

SOAR architecture ○ (Young & Lewis,

1999, Laird, Newell, & Rosenbaum, 1987)

Long term working memory framework ○ (Ericsson &

Delaney, 1999)

From Miyake and Shah (1999)

• WM as emergent property – Interactive cognitive

subsystems model

• (Barnard, 1985, 1999)

– Controlled and automatic processing architecture

• (Schneider & Detweiler, 1987)

– Biologically based model

• (O’Reilly, Braver, Cohen, 1999)

Baddeley (2000)

Executive Functions

Updating: replacing

old information with

new while retaining

the relevant

Switching: shifting

from one

strategy/domain of

knowledge to another

Inhibiting: resisting or

ignoring interference

from unwanted

information

Working Memory and

Mathematical Performance

Central executive measures predicted

early mathematical performance

○ Bull, Johnston, and Roy (1999), Bull and

Scerif (2001)

Standardised working memory scores

predicted children’s academic standing

in mathematics with 83% accuracy

○ Gathercole and Pickering (2000)

Cognitive Underpinnings

12

Cognitive

underpinnings

of math

proficiency

Study 1:

Individual

differences

in algebraic

problem

solving

Working

memory

Study 2:

Influence

of

executive

functioning

Updating

(WM)

Inhibition

Switching

Lee et al. (2004) Jn Exp Child Psych

Lee et al. (2009) Jn Edu Psych

Cognitive Underpinnings

13

Cognitive

underpinnings

of math

proficiency

Study 1:

Individual

differences

in algebraic

problem

solving

Working

memory

Study 2:

Influence

of

executive

functioning

Updating

(WM)

Inhibition

Switching

Will improving working memory

capacity also improve children’s

academic performance?

1. Correlational findings

2. Intervention time-point

Lee & Ng (2009)

Mathematics education: A

Singapore journey

Cognitive Underpinnings

14

Cognitive

underpinnings

of math

proficiency

Study 1:

Individual

differences

in algebraic

problem

solving

Working

memory

Study 2:

Influence

of

executive

functioning

Updating

(WM)

Inhibition

Switching

Will improving working memory

capacity also improve children’s

academic performance?

- Intervention time-point

Study 3:

Development of

working memory,

executive

functioning &

math abilities

Complications

The structure of executive functions

may vary with age

○ Early replication with children yielded

consistent finding similar to Miyake et

al. (2000)

Shifting, inhibition, updating

○ More recent studies are equivocal

The Structure of Executive Functions Lehto et al. (2003)

Huizinga et al. (2006)

Wiebe et al. (2008)

Complications

The structure of executive functions may vary with age

○ Early replication with children yielded consistent finding similar to Miyake et al. (2000) Shifting, inhibition, updating

○ More recent studies are equivocal

The nature of school mathematics changes with age

○ From numeracy, arithmetic, geometry, to algebra and calculus

A Cohort- Sequential Study

Examined the nature of executive

functioning from Kindergarten (5.5 year

olds) to Secondary 3 (14.5 year olds)

Examined the relationship between

executive functioning and mathematical

attainment

Questions of Interest

Does the structure of executive

functions vary with age? If so, how?

Does the relation between executive

functioning and mathematical

performance vary from Kindergarten to

Sec 3?

Time Point 1 Time Point 2 Time Point 3 (Age in Years) 5

6 7 9 11

12 13

Design

~ 673 children

spread over 4 cohorts,

81 school at Wave 4

Time Point 4

7

8

10

9

11

8

10

12

14

Constructs Tested

K2

Grade 9

Executive

functioning Mathematics

1 factor?

2 factors?

3 factors?

Basic numeracy

Understanding of

math patterns

Arithmetic

Algebra

Inhibition,

updating,

switching

Instruments

Executive functioning Inhibitory efficiency

○ Flanker

○ Simon

○ Antisaccade Mickey

Switching efficiency ○ Switch conditions from

Flanker and Simon

○ Picture–symbol

Updating capacity ○ Animal Updating

○ Mr. X

○ Listening Recall

Standardised mathematical tasks Wechsler Individual

Achievement Test ○ Number Operations

○ Mathematical Reasoning

Curricular based mathematical tasks

○ Growing number patterns

○ Function machines

○ Arithmetic and algebraic word problems

Participants were shown an

unknown number of animals one

at a time. They were then asked

to remember the last 2, 3, or 4

animals

Which were the

last two

animals that

you saw?

Congruent: 20 trials

Incongruent: 20 trials

Mixed: 28 trials

x 3

•Instruction

•Only the middle fish

is hungry

•If the middle fish is

swimming to the right

press the right key to

feed

•Picture symbol pair presented in

1 of the 4 corners of computer

screen

•Top corners - Does the

picture contain an animal?

•Bottom corners - Does the

picture contain a number?

Non switch blocks (21

trials)

Switch blocks (21 trials)

Predictable switch

(33 trials x 2)

Numerical Operations

Numerical Operations

Evaluate the ability to identify and write

numbers

Grade level Typical items from the WIAT – Numerical Operations

Kindergarten to

Primary 1

Lower Primary

(P1 to P3)

Upper Primary

(P4 to P6)

Secondary

(S1 and above)

1 2 3 _ 5 6 7 8 9

4 + 5 = __ 150

- 25

4 X 3 = __

.4 + .6 = __

-14 + (-16) = __

200% of 80 = __

2x - 15 = 3 – x

x = __

○ Raw scores

○ Multitrait-

Multimethod Model

○ Explicit accounting

for sources of

variance at manifest

level

DV = raw scores from

the incongruent,

congruent, switch, &

no switch conditions

Modelling the Data

Findings

Age 5 6 7 8

Model U(IS) U(IS) U(IS) U(IS)

• Factor structure varies with age

• 2-factors to 3-factors

• First sign of multifactor structure at 10, but does not stabilise till 14

Age 9 10 11 12

Model U(IS) U(IS) U(IS) 3F

Age 13 14

Model U(IS) 3F

Lee, Bull, & Ho (2013) Child Development

Findings & Conclusions

Executive functioning develops with age

Increases in capacity

Reduction in inhibitory and switch costs

○ Task dependent

Changes in the structure of executive functions

○ Gradual differentiation

During early to mid childhood, executive

functioning is closely associated with

processing speed

The two constructs become more distinct with age

Concurrent Relations

WMU

Mr. X Animal

Updating

Listening

Recall

Numeric

Operations

Concurrent Relations

With the exception of the youngest children, the

cross-sectional findings indicate a strong relation

between WMU and mathematical performance

Cross-sectional relations peaked at Grades 1 and

2

Surprising that relation at K1 was relatively small

Support the view that earlier math skills are more

dependent on other fundamental numeric abilities

Later math skills, acquired via schooling, are more

dependent on general cognitive abilities

Age K2 P1 P2 P3 P4 P5 P6 S1 S2 S3

Num Op on

WMU 0.16 0.66 0.63 0.50 0.47 0.55 0.50 0.49 0.59 0.55

Predictive Relations: K2

The predictive findings show that doing well in these early years depends less on what one has learned in mathematics in the previous year, but more on WMU capacity Perhaps indicate that

once basic numeracy is mastered, it contributes little to performance in arithmetic computation

Predictive Relations: P6

WMU did not predict subsequent performance in mathematics from S2 to S3 With increasing

expertise, there is a reduction in reliance on effortful executive processes

The increase in complexity places more demands on domain specific knowledge than on WMU capacities

Patterns of Growth

Mathematics Children who had higher scores at kindergarten had

lower averaged rates of growth ○ Suggesting that children with lower initial performance

do tend to catch up, although not necessarily achieving parity

Updating Rates of growth did not differ significantly across

individuals

Updating and mathematics Children who entered kindergarten with higher

updating capacity improved in their mathematics performance faster than did children with lower updating capacity

Conclusions

Several important findings

Relation between Updating

and Math peaked at P1 and

P2

Children with higher

updating capacity improved

in their mathematics

performance faster

Predictive relations between

Updating and Math were

significant from K2 to S1

Performance in math at P1

and P2 not reliant on earlier

math performance

Implications

For secondary school

students, math content

knowledge is more

important than

underlying cognitive

capacity

For the first years of

primary schooling,

updating capacity is

important

○ Can updating capacity

be improved? How?

Design Parameters

Overall Approach Game Play

Targeted updating

capacity rather than

working memory per se

Intervention not involving

counting or overtly

mathematical content

Adaptive algorithm for

progression

Fun and engaging

Uses visual stimuli to

reduce reliance on verbal

recoding

Format

47

An

Exa

mp

le

Seven Games Monster Smash

Treasure Hunter

Ant Rush

Food Mania

Continuous performance paradigm Keep track paradigm

Seven Games

Continuous performance

paradigm

Keep track paradigm

Post Bear

Alien Toy Factory

Greedy Goldfish

CogMed

Klingberg et al. (2005); Thorell et al.

(2009)

Improved performance on WM and

intelligence measures

Holmes, Gathercole, and Dunning (2009);

Holmes et al. (2010); Dunning, Holmes, and

Gathercole (2013)

Improved performance on memory

measures, but not in intelligence

Cogmed Strong visuo-spatial

component

Targets various

components of WM

Research questions

How do Cogmed and our Updating

intervention programme compare in

improving working memory?

Do improvements in working memory/

updating result in better mathematical

performance?

Participants & Design

Participants 86 7-year-olds with working memory and mathematical

difficulties

Three conditions Experimental – Updating (n = 32)

○ Averaged 23 sessions, 3-4 sessions per week, 30 min per session

Experimental – CogMed (n = 25) ○ Averaged 24 sessions, 3-4 sessions per week, 45 min

per session

Active control (n = 28) ○ Averaged 22 sessions, 3-4 sessions per week, 30 min

per session

Passive control (n = 26) ○ Business as usual

Tasks & Procedure Screening

Pre-test Working and short-term memory measures

○ Animal Updating, Corsi Blocks (Block Recall), Letter Rotation, Backward Letter Recall, Forward Letter Recall

Standardised mathematics measures ○ WIAT Numerical Operations, Math Fluency Addition and Subtraction

Intelligence measure ○ Raven’s Coloured Progressive Matrices

Covariates ○ Language: Bilingual Language Assessment Battery (BLAB)

○ Literacy: Schonell Reading Test

Intervention

Immediate post-test Week after termination of intervention

Follow-up post-test ~ 6 months after termination of intervention

Results

Significant improvement on Corsi Blocks at immediate posttest

No significant differences at delayed post-test

Results

Significant

differences

between

updating

intervention

and Cogmed

and active

control groups

at the long-

term post-test

Results

Mathematical performance & fluid

intelligence

No significant differences at post-test

No differences in results when various

covariates (age, language and

intelligence measures) were included

Summary of Findings

Both intervention programmes tended to

improve WM immediately after the

intervention, but results were not

statistically significant except for Corsi

Blocks

Improvements in WM were not

translated into gains in mathematical

performance immediately and 6-months

after intervention training

Next Steps

For intervention that works, why is

generalisation to math performance poor?

Cannot deploy newly developed capabilities

Can deploy new capabilities, but do not know

when to deploy

Why do some intervention work better than

others?

Dosage

One-to-one coaching

Targeted capabilities

Some Remaining Questions

What leads to differences in factor

differentiation and improvement in

executive efficiency? What develops?

The role of inhibition

Important in both theory and everyday

experiences, but typically fail to predict

academic performance

Are we measuring the wrong aspect of

inhibition?

A Larger Context

Understanding optimization

Social and emotional skills

Executive functions

Understanding and usage of knowledge

Psychological wellbeing

Cultural nature of learning

Creativity and student dispositions

Applied Cognitive Development Lab

Principal investigators

Kerry Lee

Rebecca Bull

Research Scientists/Fellows Ang Su Yin

Fannie Khng

Ng EeLynn

Research Assistants Jeremy Ng

Jennifer Ang

Juliana Koh

Lim Aik Meng

Tay Jia Xin

Project manager Yvonne Ng

These studies are supported by grants

from the Office of Educational

Research, and the Centre for Research

in Pedagogy and Practice, CRP8/05KL, CRP4/07KL, OER 49/08KL,

OER17/11KL

Content

Neuroimaging as an experimental tool

Neuroimaging as a pedagogical tool

The Teaching of Math

Two algebraic problem solving methods are

taught in Singapore schools

Symbolic algebra is taught in secondary school

The model method is taught in primary school

Doing the Right Thing?

Considerable time and effort are expanded on teaching the model method in the primary years

Is it worthwhile? Yes

○ Children can solve algebraic problems earlier

○ The model method help children acquire formal algebra

No ○ Children are taught to do the same thing twice

○ Multiple methods confuse children

Programme evaluation ○ Impracticable

Model method has been part of the national curriculum for over ten years

How can we provide information

to guide the curriculum?

Four studies

What do teachers think of the model

method?

How do children use the schematics?

Does the model method hinder or facilitate

students’ acquisition of formal algebra?

Do the two methods engage similar

cognitive processes?

Teacher’s Perception

Aims

Qualitative study to ascertain teachers’

perception of the model method in relation to

symbolic algebra

To find out whether secondary teachers

capitalise on pupils’ knowledge of the model

method and use it as a bridge to formal

algebra

Findings

Primary teachers tended to perceive the model method as an important problem solving tool

Secondary teachers tended to view it as a ‘primary school’ or “child-like” method and thought it a hindrance to the learning of formal algebra

Pedagogy used in secondary school did not tend to capitalise on what students know about the model method

See Ng, Lee, Ang, & Khng (2007) Redesigning Pedagogy

Model Method Analysis System

Do children use models as concrete receptacles or flexible containers?

Children are given simple word problems Magnitude of numbers

varied across four sets: units, tens, hundreds, thousands

Magnitude also increased within sets

Findings

ones thousands

Increasing

magnitude

Lee, Ng, Khng, Ng Lan Kong (2013) Frontline Education Research

cf.

Validation

data

Functional Magnetic Resonance

Imaging

Measures haemodynamic response Performance of cognitive

task

Localised changes in cortical tissues ○ Increased metabolism

○ Vasodilation

○ Changes in blood oxygenation level

○ Changes in tissue magnetic properties

Allows visualisation of brain activity that correlates with the performance of cognitive tasks

Two Experiments

Experiment 1

Focused on the first stage of algebraic word

problem solving

○ Word problem to model representation

Experiment 2

Second stage

○ Model representation to solution

Experiment 1

Word problem

James has 50 fewer watches than Mike.

How many watches does James have?

Model solution

J

M

50

• Formal algebraic

or Symbolic solution

J = M - 50

Information given

does not

encourage

computation

Control Tasks

Model condition

James has a short rectangle and the

number 50 while Mike has a long rectangle

Symbolic

There are two brands of watches: James

and Mike. The former runs on the M - 50

mechanism.

○ J = M - 50

50 J

M

Procedure

James has 50 fewer watches than Mike. How many watches does James have?

50 J

M

1s

8s

3s

Y/N

Randomised

Sym Mod Mod Sym Mod Sym

x 24 trials, no

more than 3 of

each condition

appeared

consecutively

Ctr-Blc

Ctr-Blc

Counter-balanced

James has a short

rectangle and the

number 50 while Mike

has a long rectangle.

or

Participants

18 (10 males, 20 to 25 years)

Screening criteria

○ As or Bs at Ordinary-Level examination

○ High accuracy on practice trials (> 90%) and

low variation across strategies (< 5%)

Results Similarities between the model & symbolic

methods

Differences Between Methods

Areas activated by the

symbolic method Time course of signal

changes in the precuneus

Threshold set at p < .001

(uncorrected)

-0.4

-0.2

0

0.2

0.4

0.6

0.8

1

1.2

1 2 3 4 5 6 7 8 9

Para

mete

r E

sti

mate

s

SC SE MC ME

Lee et al. (2007) Brain Research

Experiment 2

Results – Expm 2

Lee et al. (2010) ZDM

Conclusions

The symbolic method activated areas

associated with visual attention and

perhaps of procedural recruitment

Although perceived as being more

concrete or more “visual” in nature, our

findings suggest no preferential

recruitment of visual areas

Implications

There seems to be a disconnection between the teaching of algebra in primary and secondary schools Some teachers not making best use of children’s

existing knowledge

Though children still have difficulties implementing their knowledge of variables in the context of models, they seem to have a good understanding of how they could be used

Execution of formal algebraic strategy is more resource intensive than the model method Make sense to leave its introduction to the

secondary school years

Content

Neuroimaging as an experimental tool

Neuroimaging as a pedagogical tool

Imaging as a Pedagogical Tool?

Near infrared spectroscopy (NIRS) Measures BOLD responses

Possibility of examining pedagogical issues in situ

Logic of experiments Performance of cognitive

task

Localised changes in cortical tissues ○ Increases in metabolism

○ Vasodilation

○ Changes in blood oxygenation level

○ Oxy vs. deoxy Hb have different light absorption properties

Allows visualisation of brain activity that correlates with the performance of cognitive tasks

Proof of Concept Study

Problem size effect

RT and ACC differences

WM mediated

Is NIRS sensitive to differences

resulting from task difficulties?

Depth of penetration

Frontal unit

Method

Participants

21 healthy, right-handed adult participants

Instrument

16-channel NIRS

4 tri-wavelength (730nm, 805nm, and

850nm) LEDs and 10 detectors, frequency =

3Hz

Task

Manipulated task difficulty by varying the

magnitude of the operands

75 questions

5 x 15 randomized blocks

Results

Classroom Application?

Applied Cognitive Development Lab

Principal investigator

Kerry Lee, PhD

Research Fellow/Research Scientist Ang Su Yin, Ph.D

Fannie Khng, Ph.D

Research Assistants Jeremy Ng, BSSc (Hons)

Muhammad Nabil Azhar, BSSc (Hons)

Zhang Siran , BSSc (Hons)

Graduate students Catherine Leong, BA (Hons)

Imelda Suryadarma, BSSc (Hons)

Ng Ee Lynn, BSSc (Hons)

Project manager Sharmila Singaram

Collaborators NIE

○ Ng Swee Fong, PhD., Kenneth Poon, PhD

NTU ○ Ringo Ho, PhD

University of Aberdeen ○ Rebecca Bull

NUS/Duke ○ Michael Chee, MBBS; Steven Graham, PhD

Studies are supported by grants from the

Centre for Research in Pedagogy and

Practice and the Office of Educational

Research: CRP8/05KL, CRP4/07KL, OER

49/08KL