a field experiment exploring the ... - tierweb.nl papers/tier wp 14-02.pdf · tier-maastricht...
TRANSCRIPT
A Field Experiment
Exploring the Effectiveness of Summer Learning
Sofie J. Cabus & Emily van Gool
TIER WORKING PAPER SERIES
TIER WP 14/02
A Field Experiment
Exploring the Effectiveness of Summer Learning1
Sofie J. Cabus2
Emily van Gool
Top Institute for Evidence Based Education Research (TIER)
TIER-Maastricht University Kapoenstraat 2
6211 KW Maastricht, the Netherlands Tel. +31 (0)43 388 84471 or +31 (0)6 1469 33 42
Master Evidence Based Innovation in Teaching (MEBIT)
MEBIT-Maastricht University Kapoenstraat 2
6211 KW Maastricht, the Netherlands
Abstract
This paper deals with the effectiveness of summer learning programs wherein parents play a
particular role in creating a learning environment at home. A fairly simple and low-cost
evidence-based field experiment is set-up in order to estimate the effectiveness of summer
learning for children who prepare their selves in kindergarten for the transition to the first grade
of Dutch primary compulsory education. Children assigned to the intervention group received a
summer book with exercises dealing with math and language. They could practice in the summer
book for about 6 weeks. Parents were explicitly asked to assist their child with troublesome
exercises. And teachers did assess children's educational proficiency before the start of the
summer break. Standardized test scores of the treated and untreated children have been measured
before and after the intervention. Our findings show that language proficiency increased
significantly (+5.5 points) in the treated group compared to the untreated group. We did not find
a significant effect on math proficiency. It is also argued from the results that using teachers'
assessment provides better statistical modeling than using pre-test scores.
1 We are grateful to Wim Groot, Henriette Maassen van den Brink, MEBIT and TIER/DTPA participants, and the
children, the parents, and the school staff of the two schools that took part in the intervention. The authors acknowledge financial support of Platform31. The usual caveat applies. 2 Corresponding author
2
1 Introduction
The literature indicates the short run detrimental effects of the summer break on school-related
skills (e.g. Cooper et al., 1996). It is in this respect that the previous literature deals with the
concept of ‘summer learning loss’ or ‘summer learning gap’ (e.g. Alexander et al., 2001; 2007).
Alexander et al. (2001; 2007) also argue the ‘lasting consequences of the summer learning gap’,
hereby pointing to the negative effect of summer learning loss on children’s long run educational
outcomes. Several authors indicate that successful summer learning programs could potentially
oppose these detrimental effects (Borman and Dowling, 2006; McClelland et al., 2006;
Matsudaira, 2008). The idea of successful summer learning programs is that, in case summer
learning would increase children's proficiency level with respect to math and language, among
other academic, cognitive and emotional skills, children experience a better transition to the next
grade. Several determinants are discussed. First, the effectiveness of summer learning programs
might be different between educational subjects (e.g. math or language). Cooper et al. (1996)
have performed a meta-analysis including 39 studies. The authors find that, overall, children's
proficiency test scores decline over the summer break, and that math proficiency is more prone
to these harmful effects than language proficiency. Second, compared to wealthy families,
numerous studies have shown that children from poor families have, in general, rather low
proficiency scores on math and language (Cooper et al., 1996; Burkam, 2004; Currie, 2009). The
literature also indicates that children from immigrant families, who speak foreign languages at
home, or live in poor neighborhoods, experience more difficulties in school than natives (Jencks
and Mayer, 1990). Third, the literature has special attention for children's background, in general,
and parental involvement, in particular, within the scope of summer learning. Fan and Chen
(2001) discuss in their meta-analysis the positive association between parental involvement and
the educational outcomes of their children. Epstein and Sheldon (2002) and Sheldon (2007)
confirm that `parenting', i.e. parents creating a learning environment at home, is an important
determinant of children's academic/proficiency performance. Additionally, Fan and Chen (2001)
argue that parental aspirations with respect to children’s academic achievement are more
important than parenting. Fourth, considering parental involvement in relationship to aspirations,
previous research also discussed the differences in perceptions on how important education is.
Tinto (1975) and Finn (1989) conceptually discuss the process of student withdrawal, and
indicate that commitment to the school, peers and teachers may differ between boys and girls,
3
and for different ethnic groups. The meta-analysis of Hattie (2009) confirms these findings,
indicating that boys are more at-risk than girls for having skills deprivation in reading and
writing during the primary school curriculum. Hurtado and Carter (1997) and Donkers and van
der Velden (2012) additionally discuss the racial climate at the school site, potentially affecting
students’ feelings of self-worth.
Various forms of summer learning are discussed in the previous literature. For instance, children
could enroll in summer courses taught by familiar teachers at the school site, or summer camp, in
order to combat their backlog at school (Burkam et al., 2004; Borman and Dowling, 2006). But
also the parents can stimulate their children in learning at home during the summer break
(Epstein and Sheldon, 2002; Sheldon, 2007). In particular the latter form of summer learning is
explored in this paper. It is in this respect that a fairly simple and low-cost evidence-based field
experiment is set-up in order to estimate the effectiveness of summer learning for children who
prepare their selves in kindergarten for the transition to the first grade of Dutch primary
compulsory education. The evidence-based set-up of the field experiment uses a control group,
an intervention group, and the evolution of standardized ability test scores over time. Children
assigned to the intervention group received a summer book with exercises dealing with math and
language. The summer book is in line with the final educational goals set by the Ministry of
Education, Culture and Sciences with respect to kindergarten (i.e. admission criteria for primary
education). The intervention group could practice during the summer break of 2012 for about 6
weeks. The parents were asked to report in the summer book each week whether the child had
been learning or not. Furthermore, parents were explicitly asked to provide their children with
assistance to solve particular exercises. At the end of the summer holidays, children handed in
their summer book to the first grade teacher, and received a small gift from the school to reward
their work.
Before and after the summer break of 2012, both treated and untreated children had to perform
the same nationally standardized ability test (CITO). The pre-test scores and the post-test scores
could then be easily compared between the intervention and the control group as well as across
time periods. Also the parents were invited in both time periods to discuss the intervention and
its results.
4
The field experiment has been applied on 41 children, 6 year olds, and enrolled in the final year
of kindergarten in two separate schools in the Netherlands, both closely located to each other,
and each school part of one management structure. The Netherlands is an interesting case study
for our intervention for at least two reasons. First, standardized ability testing is performed
during the school curriculum, including the final year of kindergarten, in order to provide the
teacher with appropriate feedback with respect to children's proficiency. The ability test is in line
with the final goals set for kindergarten. This makes test scores highly comparable across time
periods and between children. And second, recent policy debate in the Netherlands discusses
whether or not the summer break should be set shorter in order to combat the summer learning
loss. In this respect, the evolution of the control group's test scores before and after the summer
break could be highly interesting (i.e. did untreated students fell behind or not?).
This paper proceeds as follows. A brief overview of the literature is presented in Section 2. The
course and design of the intervention is discussed in Section 3. In Section 4 the data and the
results of the pilot study are discussed. Section 5 concludes.
2 Discussion in the Literature
The discussion in the literature on children’s transition to primary education puts forth two
angles: (1) being ready for the transition to first grade, briefly school readiness, is the result of
`human biology' (e.g. Snow, 2006; Gadeyne et al., 2008); and (2) school readiness is the result of
the interaction between the individual child and his/her parents, the teacher, and the comunnity
(e.g. Mashburn and Pianta, 2006). The former theory on school readiness implies that neither
parents nor the school staff can influence the child's readiness for transition. The latter theory
argues that children can be encouraged by teachers, parents, and the community wherein he/she
lives.
We explain both sides of the theory on school readiness and its theoretical thoughts on the
effectiveness of summer learning. First, we consider the `human biology model', its name
corresponding to children's school readiness as an individual characteristic. Here, school
readiness is most commonly defined as having academic, cognitive and emotional skills (among
others, see Heaviside and Farris, 1993; Ladd et al. 2006; Mashburn and Pianta, 2006). The main
questions underlying readiness are: "[...] what makes children "ready" for their first year of
5
formal schooling? [...] As children enter kindergarten, what tasks must they master to adjust to
school, and progress successfully in this context? (Ladd et al., 2006, p.116)." School readiness
in the human biology model cannot be influenced by parenting and schooling but will evolve
with ageing of the child. Therefore, an intervention as summer learning would not be effective.
Second, we discuss the `interactive model', directly referring to the interactive field between
parents and their children (home), teachers and peers (school), and communities (institutions or
neighborhood). Here, school readiness is the final product of systemic factors (e.g. family and
school institutions) and individual characteristics (e.g. gender, age, and innate ability), and its
complex interaction. As such, it is not only the child's `speed of development' with respect to
academic and cognitive functions (cf. human biology), but also the cummunity as a whole that
plays a role in mastering school-related proficiencies. In this respect, Mashburn and Pianta (2006,
p.169) conclude that: "[...] school readiness is best conceived as an aspect of child functioning
that is determined by the quality and type of interactions that a child engages in with others, and
social relationships between children and parents, parents and teachers, and teachers and
children are identified as the conduits through which children become motivated and interested
in school activities and acquire competencies associated with school readiness." Thus, parenting
and schooling can support the child with being ready for the transition to primary education, for
example, by additional exercises or individual guidance. In line with this model, summer
learning could make a real difference.
3 The course and design of the intervention
3.1 Working in the summer book
In order to combat potential summer learning loss, children received a book with exercises to be
solved during the summer holidays. Note that, in the Netherlands, the school year starts mid-
August after 6 weeks of summer holidays. The summer book consists of 45 pages in total. The
summer book helps the treated child to obtain the final goals of kindergarten with respect to math
and language as stipulated by the Ministry of Education, Culture and Science. However, the
summer book provides, additionally, also children to learn the first letters and numbers that
children learn in first grade (i.e. first year of primary education). For every day of the summer
break, there is a different page, so that a child can work every day on a different exercise. After 6
days (one week), the children are explicitly asked in an exercise to search for a letter or number
6
they have learned during the week. For instance: "Where did you hear the letter M this week?
Can you draw a painting about this on this page."
3.2 Asking for parental involvement
The previous literature indicates the effectiveness of parental (community) involvement in
positively influencing student behavior both at home and at school (e.g. Epstein and Sheldon,
2002; Sheldon, 2007; Patall et al., 2008; Sheridan et al., 2010). In line with these findings, we
acknowledge that parents (and the neighborhood wherein they live) have an important role in
successfully supporting the child's learning during the summer break. Therefore, we explicitly
asked the parents to assist their child with troublesome exercises in three different ways. First,
we organized an information session at the school site just before the holidays to inform the
parents about the course of the intervention. The other way was written in the summer book. On
the first page, one finds a description of the intervention for the parents of the children, and the
question to provide their child with assistance. Here, we provide a transcript of the first Dutch
phrases: "Dear parents or caregivers, This summer book provides exercises that your child can
make during the summer break. Some exercises can be done independently by the child, other
exercises will be too difficult and will need your assistance." And third, parents were also asked
to write down in the summer book on the 7th day of each week how well the child had been
practicing in that week.
3.3 Rewarding and intrinsic motivation
Rewards may stimulate participants of the intervention group to increase intrinsic motivation
(Deci et al., 1999). There are several types of rewards, such as tangible rewards (the free ticket
for the playground) or verbal rewards (the positive feedback). Deci et al. (1999) discuss that
tangible rewards are less effective than feedback when it comes to `free choice' behavior (i.e. the
child freely chooses to do the exercises). As follows, we have dealt with the reward incentive in
the intervention in order to increase the motivation of the children and their parents. After the
summer break, the intervention ends, and parents were asked to hand in the summer book to the
primary school teacher. In case the child did hand in his/her summer book, they were given a
reward together with feedback on the summer book. The reward was a small gift from the school,
namely: a free ticket for the playground in the neighborhood. The feedback also included a
sticker on the summer book. Note that all children handed in their summer book. There is one
specific paragraph in the letter to the parents (i.e. the first page of the summer book) that refers to
7
this reward. It was also mentioned during the information session before the summer holidays.
However, it was only mentioned that their child would receive a small reward without stipulating
what exactly this reward would be. It was also mentioned that their child would receive feedback
on the summer book when handed in after the summer holidays. As such, at the start of the
intervention the reward was verbally mentioned, and repeated in the summer book, but not
specific, and, thus, not tangible.
3.4 The Outcome variable
The intervention uses the standardized exam (CITO) that every child makes before the transition
to primary education. In order to provide the teacher with appropriate feedback with respect to
children's proficiency, standardized ability testing is common practice in the Netherlands. The
ability test is in line with the learning goals of kindergarten set by the Ministry of Education,
Culture, and Science. This makes test scores highly comparable across time periods and also
between schools. The standardized CITO exams are widely used in the Netherlands, and are
considered reliable and valid for measuring children's proficiency. Note that the June and August
standardized CITO exam was exactly the same in order to measure learning gains over the
summer break.
3.5 The assignment of treatment and control Group
We are interested in the effect of summer learning on children's language and math proficiency
test scores. Therefore, we performed an evidence-based education research aiming at children
enrolled in the final school year of kindergarten. Two Dutch schools (further abbreviated by
School A and School B) participated in a pilot case in 2012. Both schools are under one
management structure, and, consequently, are liable to similar school policies. Further note that
the intervention was implemented in a fast and prompt way in January 2012, so that parents did
not choose a school based on our intervention.
Our assignment rule of children to the intervention group or the control group was fairly simple:
children enrolled in School A were assigned to the intervention group, whereas children in
School B to the control group. Random assignment of children was not possible owing to
practical and ethical reasons. We argue the advantages and disadvantages of such an assignment
rule, and deal with the disadvantages.
8
3.5.1 Advantages
Parental involvement was part of the set-up of our intervention. As such, we strongly wish to
avoid communication between parents of children in the intervention group with parents of
children in the control group. Communication between parents from treated and untreated
children would induce spill-over effects, for example, by exchanging the summer book or
sensitization of parents with respect to summer learning. This exchange of information would
directly violate the stable unit treatment value (SUTVA) assumption arguing that treatment may
not be spilled over from treated to untreated students (Rubin, 1974).
In case both the intervention group and the control group would have been mixed in schools by
random assignment, it would have been impossible to support the SUTVA. In contrast, by taking
School A as the intervention school and School B as the control school, we could control for
exchanging information. In addition, we explicitly did not tell parents that School B was the
control group at information sessions for parents in School A.
3.5.2 Disadvantages
Most importantly, non-random assignment can create underlying differences between the
intervention group and the control group that bias the estimation results (i.e. omitted variables
bias) (Rubin, 1974). As previously argued, there are several determinants that make `children at-
risk' of poor performance in school, and, consequently, school failure. These determinants
include among others: gender; ethnicity, cultural differences, and class composition;
socioeconomic status; parental involvement; perception of teachers with respect to school
readiness and school failure; and educational and school policy (for an elaborated discussion, see
Hattie, 2009). Ideally, treated and untreated students are, on average, highly comparable with
respect to these determinants. Of some characteristics, we have information so that we can
control for it in a multivariate regression (i.e. gender, ethnicity and socioeconomic status).
However, we do not observe all determinants of children's proficiency test scores, like innate
ability, or the general dislike of school. It is in this respect that we indicate that, in general,
parents choose the kindergarten at closest distance to their home. This distance criterion, i.e. the
two schools closely located to each other in the same neighborhood, make one believe that, on
average, treated children should be comparable to untreated children.
9
To further overcome this disadvantage of our experimental set-up, we have measured children's
math and language proficiency before the intervention took place: June and January 2012. Before
the summer, children developed their proficiency in school and at home, so that we are able to
measure the success rate of schooling, which is a combination of "teachers' valued added", the
innate ability to learn, as well as environmental factors (e.g. parental involvement and class size
composition) and children's characteristics (e.g. gender and migrant status).
In the first instance, we control only for the June pre-test in a multivariate regression,
additionally controls for unobserved determinants of proficiency test scores. We can formally
express this multivariate regression as:
∑ (1)
where subscript i∈{1,2,...N} denotes all individuals assigned to either the control group (I=0) or
the treatment group (I=1); children's proficiency scores in August (i.e. the post-test);
children's proficiency scores June (i.e. the pre-tests); a vector of individual characteristics;
and the error term. The effect of interest is β, which is the effectiveness of summer learning.
Second, we also look at the evolution in test scores from January over June to August 2012. This
visualizes the performance gains of students over the school year, including learning at school
(value added of teaching). Ideally, students' performance gains over time in the control group and
the treatment group are, on average, the same in order to support the parallel time trend
assumption (Rubin, 1974). This assumption implies that, in absence of the intervention, children
proficiency would have moved parallel over the summer break. Note that, as a result of different
teachers in School A and School B, differences in performance gains between the treated and
untreated students can be expected between January and June. However, by definition, and with
reference to the summer learning gap, we expect no evolution in the control group during the
summer break, and a positive evolution in the treatment group. Having additional information on
the pre-test, we can control for the pre-trend in a multivariate regression.
3.6 Teachers’ perception of being ‘at-risk’
We acknowledge that proficiency test scores are, in fact, screen shots of a particular moment in
time. It can be that, at the pre-test or the post-test, a child was ill or tired, in turn having bad test
scores. Alternatively, recent policy debate in the Netherlands argues for an increasing share of
10
teachers' perception in formulating advice on children's educational proficiency. Harlen (2005)
indicates the costs and benefits of teachers' assessment in a literature review. She concludes that,
overall, teachers' perception on children proficiency may be unreliable and biased, but, also, that
it should be considered against the potentially low validity or reliability of tests. The main
advantage of teachers' assessment with respect to the child's performance is that it is not a screen
shot. Instead, their assessment may be a good indicator for children's performances in the first
year of the primary school.3
Therefore, as a robustness check, we formulate an alternative multivariate model specification
replacing the pre-test score with an indicator of `being at-risk' (of low performance on math or
language), as to predict the post-test scores. The at-risk indicator is based on teachers' assessment
of children's proficiency. Therefore, the intervention included a questionnaire for the teacher of
the control group and the treatment group. Before the pre-test in June was made, teachers had to
report the `at-risk status' of every child on a five point Likert scale (i.e. 1 denotes no risk of
failing the proficiency test; 5 denotes high risk of failing the proficiency test). The teacher in
both the control group and the treatment group filled in the at-at-risk status of the child, in
general, and with respect to math, language, learning attitude, and behavior, in particular. Having
the at-at-risk status of every child, we can control for it in a multivariate regression. We then
replace with and rewrite equation (1) as:
∑ (2)
The effect is again the effectiveness of the intervention.
3.7 A short note on statistical power
In case the statistical power would be rather low, we could falsely conclude that our intervention
did not have any effect (i.e. the type-2 error). We use the G-power software to first calculate the
sample size when one would like to have a statistical power of at least 0.80, and error probability
of 0.05. Our finding indicates that for a small effect size of 0.2, we need at least 191 children in
total. Increasing the effect size to 0.5, we need 26 children in total. For large effect sizes of 0.8,
we need 7 children in total. As such, we need at least an effect size of 0.5 in order to correctly
3 Note that the proficiency test scores are based on the standardized exams administered by the Ministry of
Education, Culture and Science, so that teachers cannot directly influence the post-test results because of their assessment within the scope of this intervention.
11
accept insignificant results. Further note that we can enforce the statistical power by controlling
for individual and family characteristics in a multivariate regression.
4 Data and Results
The intervention group consists of 22 children, and the control group of 19 children. We
summarize the descriptive statistics of the individual characteristics in Table 1. We also include
the difference between the intervention group and the control group and the corresponding
significance level. In general we do not find any significant differences between the intervention
group and the control group with respect to the observed characteristics, except for age in months.
About 47 percent of the children in the control group is male, compared to 41 percent in the
treatment group. Children are, on average, 6 year olds at the start of primary education, with a
significant mean difference between the control group (77 months) and the treatment group (73
months) of -4 months. Most children have the Dutch ethnicity: 42 percent and 55 percent in the
control and treatment group, respectively. There are slightly more (about 6 percentage points)
Moroccan and Turkish children in the control group than in the treatment group. About 79
percent (control group) and 68 percent (intervention group) of parents have a diploma of
secondary or higher education; 5 percent (control and intervention group) a diploma of pre-
vocational education; and 16 percent (control group) and 27 percent (intervention group) a
diploma of only primary education. To conclude, we have information on cultural differences
(difference between the child and his parents with respect to having the Dutch culture or not) at
the side of one or both parents. We indicate a mean difference between control and treatment
group of 8 percentage points at father's side, and -11 percentage points at mother's side.
Next, we plot the evolution of the children's proficiency scores on a graph (max. sore is 108 for
language and 137 for math). Figure 1 deals with math proficiency scores and Figure 2 with
language proficiency scores. We distinguish between the evolution of treated children (dotted
line) and untreated children (solid line) over the period January to August 2012. These figures
clearly show the learning gains in school of children from January to June.
First, consider the results with respect to language proficiency. The treatment group increased
their language proficiency from 64 to 68, i.e. an increase of about 4 points over the summer
break. The language proficiency scores of the control group were unchanged (pre-test and post-
12
test about 63/108). If we now compare the post-test scores on language between the intervention
group and the control group, we observe a significant difference of +5.5 points.
For math proficiency, we observe a minor difference between the treated and untreated children
of -1.5 points and not significant. The descriptive statistics show that untreated children slightly
increased their math proficiency over the summer break from 85 to 87. This increase is not
statistically significant. The intervention group did not increase their math proficiency (pre-test
and post-test about 85/137).
As follows, we include all control variables, namely age, gender, education of the parents, and
cultural differences, into a multivariate regression. Note that we correct the standard error for
heteroskedasticity (χ²=14.13;P-value=0.0002). We estimate two specifications: one specification
additionally controlling for the pre-test scores of June, and one specification additionally
controlling for teachers' perception on the at-risk status of children with respect to language
deficiency (see Section 3). Full model estimation is available with the author upon request.
Following the first specification, we find that our intervention `summer learning' has a small to
medium effect size of 0.27 on language proficiency test scores significant at 10 percent level.
Following the second specification, we estimate a medium effect size of 0.44 significant at 5
percent level.
In conclusion, we find that teachers' assessment is a strong and highly significant predictor of the
level of the post-test scores. From the results we argue that using teachers' assessment provides
better statistical modeling than using pre-test scores, as it pushes up the statistical power of our
analyses to 0.86. These findings can have two underlying explanations: (1) teachers' perception
on children's educational deficiency is directly related to these children their performances (i.e.
teachers' value added; see also Nye et al., 2004; Jussim and Harber, 2005). Or (2), irrespectively
of the teachers' value added, teachers can better indicate than test scores whether a child is
skilled enough for making the transition to primary education (e.g. Okpala, 2007).
5 Conclusion
The literature indicates that children proficiency test scores decline over the summer break (e.g.
Cooper et al., 1996). Summer learning could oppose these detrimental effects on school-related
skills. The literature additionally indicated the positive impact of parental involvement on
achievement (e.g. Epstein and Sheldon, 2002; Sheldon, 2007; Patall et al., 2008). As such, we
13
explicitly incorporated the role of the parents into the intervention by providing them with the
learning material, namely: the summer book. Previous research also argued that summer learning
programs can only be effective in the `interactive model', but not in the model of `human
biology' (Ladd et al., 2006; Mashburn and Pianta, 2006). The former model stipulates that school
readiness can be influenced by providing children with additional exercises, whereas the latter
theory argues that ageing is the main process underlying children's readiness for primary
education.
This paper explores the set-up of a fairly simple and low-cost intervention in order to estimate
the effectiveness of summer learning for 6 year olds children in transition to the first grade of
primary education in the Netherlands. It consists of a summer book helping children to master
the final learning goals of kindergarten set by the Ministry of Education, Culture and Science.
The intervention has been tested in two Dutch schools including a small sample of 41 children,
information on their parents, and teachers’ perception on being at-risk. The effectiveness of the
intervention ‘summer learning’ is promising with respect to language proficiency, but not with
respect to math proficiency. Our findings show that, owing to the intervention, language
proficiency increased significantly (+5.5 points) in the treated group compared to the untreated
group. We observe a summer learning gap for children in the control group, as they did not
increase in points, but remain stable over time. These findings are robust to two different model
specifications and control variables. With respect to math proficiency, we did not observe any
significant effect of the intervention.
Further research should deal with larger sample sizes in order to confirm the results. It could also
explore summer learning as a potential alternative for grade retention, a common practice in the
Netherlands.
14
6 Tables and figures
Table 1: Descriptive statistics of children characteristics and outcome variables
Control
Group
Intervention
Group Difference P-value
(1) (2) (2)-(1)
1. Children characteristics
Gender (%) (male=1) 0.47 0.41 -0.06 0.687
Age (Months) 76.933 73.442 -3.491 0.010*
Country of origin
Netherlands (%) 0.4211 0.5455 0.1244 0.4395
Morocco (%) 0.1053 0.0455 -0.0598 0.4759
Turkey (%) 0.1053 0.0455 -0.0598 0.4759
Other (%) 0.3684 0.3636 -0.0048 0.9755
Education of the parents
Secondary and higher⁽¹⁾ (%) 0.7895 0.6818 -0.1077 0.4505
Pre-vocational education⁽¹⁾ (%) 0.0526 0.0455 -0.0072 0.9179
Primary education (%) 0.1579 0.2727 0.1148 0.3884
Cultural differences⁽²⁾
Father's side 0.1053 0.1818 0.0766 0.5015
Mother's side 0.2105 0.0909 -0.1196 0.2915
2. Outcome variables
proficiency test 84.6842 85.5455 0.8613 0.8294
math June (max. 137)
proficiency test 62.8421 64.2727 1.4306 0.6441
language June (max. 108)
proficiency test 86.8947 85.4091 -1.4856 0.7171
math August (max. 137)
proficiency test 62.947 68.455 5.508 0.1235
language August (max. 108)
Note 1: The pre-vocational and primary education level indicate parents without a valid secondary school-leaving
certificate.
Note 2: Cultural differences denote that the mother, the father, or both is an ethnic minority.
15
Figure 1: Evolution of math proficiency test scores over time (January, June and August
2012).
Figure 2: Evolution of language proficiency test scores over time (January, June and
August 2012).
16
7 References
Alexander, K.L., Entwisle, D.R., Olson, L.S.(2001). Schools, Achievement, and Inequality: A
Seasonal Perspective. Educational Evaluation and Policy Analysis 23, 171-91. doi:
10.3102/01623737023002171
Alexander, K.L., Entwisle, D.R., Olson, L.S.(2007). Lasting consequences of the summer
learning gap. American Sociological Review 72(2), 167-180. doi: 10.1177/000312240707200202
Borman, G.D. and Dowling, N.M. (2006). The longitudinal achievement effects of multi-year
summer school: Evidence from the Teach Baltimore Randomized Field Trial. Educational
Evaluation and Policy Analysis 28, 25-48. doi: 10.3102/01623737028001025
Burkam, D.T., Ready, D.D., Lee, V.E. and LoGerfo, L.F. (2004). Social-class differences in
summer learning between kindergarten and first grade: model specification and estimation.
Sociology of Education 77, 1-31. doi: 10.1177/003804070407700101
Cooper, H., Nye, B., Charlton, K., Lindsay, J., Greathouse, S.(1996). The effects of Summer
Vacation on Achievement Test Scores: A Narrative and Meta-Analytic Review. Review of
Educational Research 66, 227-68. doi: 10.3102/00346543066003227
Currie, J. (2009). Health, wealthy, and wise: socioeconomic status, poor health in childhood, and
human capital development. Journal of Economic Literature 471, 87-122.
Deci, E.L., Koestner, R. and Ryan, R.M. (1999). A meta-analytic review of experiments
examining the effects of extrinsic rewards on intrinsic motivation. Psychological Bulletin 125(6),
627-668. doi: 10.1037/0033-2909.125.6.627
Donkers, J., and van der Velden, R. (2012). Why are migrant students better off in certain types
of educational systems or schools than others? European Educational Research Journal 11(1),
11-44.
Epstein, J.L., and Sheldon, S.B. (2002). Present and accounted for: improving student attendance
through family and community involvement. The Journal of Educational Research 95(5), 308-
318. doi:10.1080/00220670209596604
17
Fan, X. and Chen, M. (2001). Parental involvement and students' academic achievement: a meta-
analysis. Educational Psychology Review 13(1), 1-22. doi: 10.1023/A:1009048817385
Finn, J.D. (1989). Withdrawing from School. Review of Educational Research 59, 117-42. doi:
10.3102/00346543059002117
Gadeyne, E., Onghena, P., Ghesquière P. (2008). Child and Family Characteristics Associated
With Nonpromotion in Preprimary Education. Exceptional Children 74(4), 453-469.
Hattie, J. (2009). Visible learning: A synthesis of over 800 meta-analyses relating to achievement.
London: Routledge.
Harlen, W. (2005). Trusting teachers' judgements: research evidence of the reliability and
validity of teachers' assessment for summative purposes. Research Papers in Education 20(3),
245-270. doi:10.1080/02671520500193744
Heaviside, S., and Farris, E. (1993). Public school kindergarten teachers' views on children's
readiness for school (Report No. FRSS-46).Washington, DC: U.S. Department of Education,
National Center for Education Statistics.
Hornby, G. and Lafaele, R. (2011). Barriers to parental involvement in education: an explanatory
model. Educational Review 63(1), 37-52. doi:10.1080/00131911.2010.488049
Hurtado, S. and Carter, D.F. (1997). Effects of college transition and perceptions of the campus
racial climate on Latino college students' sense of belonging. Sociology of Education 70(4), 324-
345.
Jencks, C. and Mayer, S.E. (1990). The social consequences of growing up in a poor
neighborhood. In: L. Lynn Jr. and M. G. H. McGeary (Eds.), Inner-city poverty in the United
States. Washington D.C.: National Academy Press.
Jussim, L., and Harber, K.D. (2005). Teacher expectations and self-fulfilling prophecies:
Knowns and unknowns, resolved and unresolved controversies. Personality and Social
Psychology Review 9(2), 131-155. doi: 10.1207/s15327957pspr0902_3
18
Ladd, G., Herald, S.L. and Kochel, K.P. (2006). School readiness: are there social prerequisites?
Early Education and Development 17(1), 115-150. doi:10.1207/s15566935eed1701_6
Mashburn, A.J. and Pianta R.C. (2006). Social Relationships and School Readiness. Early
Education and Development 17(1), 151-176. doi: 10.1207/s15566935eed1701_7
Matsudaira, J.D. (2008). Mandatory summer school and student achievement. Journal of
Econometrics 142, 829-850. http://dx.doi.org/10.1016/j.jeconom.2007.05.015
McClelland, M., Acock, A., Morrison, F. (2006). The impact of kindergarten learning-related
skills on academic trajectories at the end of elementary school. Early childhood research
Quarterly 21, 471-490. http://dx.doi.org/10.1016/j.ecresq.2006.09.003
Nye, B., Konstantopoulus, S., and Hedges, L.V. (2004). How large are teacher effects?
Educational Evaluation and Policy Analysis, 26, 237-257. doi: 10.3102/01623737026003237
Okpala, C. (2007). The Perceptions of Kindergarten Teachers on Retention. Journal of research
in childhood Education 21(4). doi:10.1080/02568540709594603
Patall, E.A., Cooper, H. and Robinson, J.C. (2008). Parent involvement in homework: A research
synthesis. Review of Educational Research 78(4), 1039-1101. doi: 10.3102/0034654308325185
Rubin, D. (1974). Estimating causal effects of treatments in randomized and nonrandomized
studies. Journal of Educational Psychology 66, 668-701. doi: 10.1037/h0037350
Sheldon, S.B. (2007). Improving student attendance with school, family, and community
partnerships. Journal of Educational Research 100(5), 267-275. doi: 10.3200/JOER.100.5.267-
275
Sheridan, S.M., Knoche, L.L., Edwards, C.P., Bovaird, J.A. and Kupzyk, K.A. (2010). Parent
Engagement and School Readiness: Effects of the Getting Ready Intervention on Preschool
Children's Social--Emotional Competencies. Early Education & Development 21(1), 125-156.
doi: 10.1080/10409280902783517
Snow, K.L. (2006). Measuring school readiness: Conceptual and practical considerations. Early
Education & Development, 17(1), 7-41. doi: 10.1207/s15566935eed1701_2
19
Tinto, V. (1975). Dropout from higher education: A theoretical synthesis of recent research.
Review of Educational Research 45, 89-125. doi: 10.3102/00346543045001089
TIER WORKING PAPER SERIES
TIER WP 14/02 © TIER 2014
ISBN 978-94-003-0067-5