Does Playing Lead to Educational Benefits?

The Case of Chess Giovanni Sala Fernand Gobet

Department of Psychological Sciences

Overview of Talk

!  Why study the effects of chess instruction? !  Brief review of previous research !  A meta-analysis of experiments on the benefits

of chess instruction !  A three-group design experiment

!  Game of Go as active control group !  Conclusions

Why Study the Effect of Chess Instruction?

Poor Results in Mathematics !  In many Western countries, pupils perform poorly in

mathematics !  Stevenson, Lee and Stigler (1986)

!  Chinese teenagers perform vastly better than US ones

!  2013 Pisa results (maths) 1.  Shanghai 26. UK 2.  Singapore 36. USA 3.  Hong-Kong 4.  Taiwan 5.  Korea

Play: A Magic Bullet? !  A nice solution would be for children to learn

mathematics (and other skills) when playing !  Playing is fun, and children would be more

motivated !  They could learn important concepts without

realising it !  Implicit learning

!  They could also acquire general cognitive skills !  Concentration !  Intelligence

Caillois (1957) “In a general way, play seems to consist in training the body, the character, or the intelligence, without any predetermined end. Thus, the more the game is removed from reality, the greater is its educational value, for play does not teach formulas; it develops aptitudes.”

Chess: The Ideal Game? !  Factors specific to chess

!  Diversity of pieces help maintain attention !  Chess offers an optimal trade-off between

complexity and simplicity !  Balance between tactics and strategy is ideal !  Chess combines numerical, spatial, temporal and

combinatorial aspects !  These factors may foster

!  Attention !  Problem solving and decision making

!  There is some overlap with mathematics

Chess and Mathematics !  Basic arithmetic

!  Value of pieces !  Control of squares

!  Cartesian geometry !  Coordinate system (a, b, ..., h) × (1, 2, ..., 8)

!  Geometric series !  One places one grain of rice on the first square,

two on the second, four on the third, and so on, doubling each time. What is the total?

or

Chess and Mathematics !  Geometry

!  Euclidian vs. city-block distance !  Reti (1921)

The Question of Transfer !  Near transfer

!  Transfer to a similar domain: algebra to calculus

!  Far transfer !  Transfer to a non-similar domain: Latin to mathematics

!  Is there far transfer from chess to other domains, such as mathematics? !  Chess community: Yes! !  Scientific community: We don’t know

Gobet and Campitelli (2006) !  Systematic review of literature on transfer and

chess teaching !  Disappointing results !  Few scientifically valid studies !  Only a couple of studies were peer-reviewed !  No clear evidence for transfer

State of the Art in 2004 “More rigorous research needs to be done before one can be confident that board games have positive effects on instruction in general. It is our hope that a review of the literature 10 years from now will have more empirical data to report.”

State of the Art in 2015 “More than a century ago, Thorndike and Woodworth (1901) showed that transfer between domains was very difficult indeed, and more recent data support this view. […] Transfer is difficult even between subdisciplines of the same field.”

A Meta-analysis of Experiments on the Benefits of Chess Instruction

Sala and Gobet (2016) http://www.sciencedirect.com/science/article/pii/S1747938X16300112

What is a Meta-Analysis?

!  A statistical procedure to compare the effects of a treatment (e.g., drugs, chess instruction) between studies, and to calculate an overall effect size.

!  The aim of is to quantitatively evaluate the available empirical evidence about a specific phenomenon.

What is a meta-analysis?

Calculating the Effect Sizes...

! 

... and Putting Them Together

! 

Main Inclusion Criteria

!  Experimental design !  Chess instruction !  Academic (mathematics or reading) or

cognitive outcome !  Presence of a control group !  K-12 pupils

The Studies

!  24 studies !  Conducted from 1976 to 2015

!  40 effect sizes !  5,221 participants in total

Results

! 

However...

!  Only a few studies had an active control group (a.k.a. placebo group) !  Necessary for controlling placebo effects

!  Hattie’s (2009) categorization !  The mean value of the effectiveness of

educational interventions is g = 0.4 !  “Zone of desired effects”: g > 0.4

The Ideal Experiment

Random Allocation

Pre-test Pre-test Pre-test

Treatment Group

Do-Nothing Group

Placebo Group

Post-test Post-test Post-test

!  Standard design in medicine

!  Rarely used in education

"  Never used with chess instruction!

Fried and Ginsburg (1979) !  Lack of pre-test is

partly compensated by randomisation !  But no guarantee

that groups are equivalent at the beginning

Random Allocation

Pre-test Pre-test Pre-test

Treatment Group

Do-Nothing Group

Placebo Group

Post-test Post-test Post-test

Frank (1979) !  No placebo

group !  Cannot reject the

effect of unspecific factors

!  e.g. participation in experiment; experimenters’ expectations

Random Allocation

Pre-test Pre-test Pre-test

Treatment Group

Do-Nothing Group

Placebo Group

Post-test Post-test Post-test

An Experiment Using Go as Active Control Group

Sala, Gobet, Trinchero and Ventura (2016)

Method (I) !  Participants

!  52 fourth graders in 3 classes of a primary school in Italy

!  Mean age of the participants = 9.3 years

!  Mathematics measures !  A six-item test based on the IEA-TIMSS, a well

validated mathematical test (Mullis & Martin, 2013) !  Measures both procedural knowledge and problem-

solving ability !  Metacognitive skills

!  Italian version of Panaoura and Philippou’s (2003) questionnaire

Method (II) !  The 3 classes were randomly assigned to:

!  Experimental group: 15 hours of chess lessons during school hours

!  Active control group (placebo group): 15 hours of Go lessons during school hours

!  Passive control group: regular school activities only

!  Importantly, chess and Go replaced part of the hours originally dedicated to mathematics and sciences

Chess and Go Lessons !  Followed a pre-arranged teaching protocol

!  Basic rules of the games !  Tactical exercises !  Playing complete games

!  Focused mainly on problem-solving situations !  e.g. finding the correct move !  Evaluating the strengths/weaknesses in a

particular position !  Were taught by the same instructor

!  An experienced teacher !  Both a chess and Go trainer

Design

Results: Mathematics !  No significant differences between the three

groups in the pretest scores !  An analysis of covariance (ANCOVA) used

the pretest scores as a covariate !  Significant effect of the covariate, p < .001 !  Significant effect of group, p < .04

!  Control group > Go group, p < .02 !  Chess group > Go, marginally, p < .09

!  No difference between the control and the chess group, p < .487

Results: Metacognition !  No significant differences between the three

groups, p < .617 !  An analysis of covariance (ANCOVA) used

the pretest scores as a covariate !  No significant effect of group, p < .694

Discussion !  Chess-Go study: Effects on mathematical

ability !  Not better than regular school activities !  But chess better than Go !  This suggests that some chess-related skills

generalized to mathematics !  Chess effects are not only placebo effects! !  No effect with metacognitive skills

!  Caution! It’s just one study.

General Discussion (I) !  Overall, it seems that the cognitive and

educational benefits of chess playing are minimal

!  Moreover, little is known about the mechanisms leading to such benefits

!  Perhaps focus should be on different aspects !  Social development !  Emotional development

Discussion (II) !  Different programmes could be developed

!  Informing chess instruction with school curricula !  Using several board games to teach specific

concepts !  Making the link explicit

!  Only board games were investigated !  Other types of games or play might more beneficial

Discussion (IV) !  The last word to Caillois (1957)

“But it is never the function of play itself to develop these faculties. The purpose of play is play.”

Thank you!