literacy boost bangladesh - resource centre · literacy boost bangladesh endline report may 2013...
TRANSCRIPT
1
Literacy Boost
Bangladesh
Endline Report May 2013
Jarret Guajardo, Moazzem Hossain, Binoy Kumar Deb Nath, and Amy
Jo Dowd
Acknowledgement to Sk. Md. Abu Bakar, Abdul Quddus Miah, Khalilur
Rahman, Gulam Kabir, and Education department for their excellent support
Special thanks to our team of enumerators: Raju Ahmmed, Asma Akter,
Moushumi Akter, Nasrin Akhter, Rinky Akter, Abdul Gaffar, Md. Uzzal Hossain,
Sheikh Ahmmed Ishtiak, Rafiqul Islam, Khandakar Nusrat Jahan, Lovely Khatun,
Mst. Rahana Khatun, Salma Khatun, Oheduzzaman Liltu, Ahsan Kabir Masum,
Rukaya Mustary, Nasrin Parvin, Shamima Parvin, Obaidur Rahman, Md. Shohel
Rana, Md. Shahidur Rahman Shiplu, Mahmuda Sultana, and Rubana Yasmin.
2
Table of Contents Acronyms and Select Definitions ................................................................................................................................... 1
Reading Outcome Definitions ....................................................................................................................................... 2
Executive Summary ....................................................................................................................................................... 3
Introduction ................................................................................................................................................................... 6
Context ................................................................................................................................................................................................ 7
Methodology ....................................................................................................................................................................................... 8
Sample Selection ...................................................................................................................................................... 8
Instruments and Measurement ................................................................................................................................. 9
Analysis ................................................................................................................................................................. 12
Attrition Analysis ......................................................................................................................................................... 12
Endline Descriptive Statistics Analysis ......................................................................................................................... 14
Student Characteristics .................................................................................................................................................................. 14
School Characteristics .................................................................................................................................................................... 17
Home Literacy Environment Data ............................................................................................................................................... 18
Literacy Boost Participation .......................................................................................................................................................... 21
Impact on Emergent Literacy Skills .............................................................................................................................. 22
Individual Skill Analysis ................................................................................................................................................ 28
Letter Identification ......................................................................................................................................................................... 28
Most Used Words ........................................................................................................................................................................... 29
Pseudo Words .................................................................................................................................................................................. 29
Readers ............................................................................................................................................................................................... 30
Fluency ................................................................................................................................................................................................ 31
Accuracy ............................................................................................................................................................................................. 32
Reading Comprehension ................................................................................................................................................................ 33
One Additional Way to Look at the Data: Readers with Comprehension ...................................................................... 34
Trends in Reading Skill Data from Multilevel Regression Models ............................................................................... 35
LB Reading Activities and Reading Achievement ..................................................................................................................... 35
Classroom and Community Channels for Reading Skill Improvement .............................................................................. 36
Conclusion ................................................................................................................................................................... 38
1
Acronyms and Select Definitions
ANOVA Analysis of variance: Statistical models used to analyze variation and
differences among and between groups
ECCD Early Childhood Care and Development
ES Effect size: The effect size is a measure of the magnitude of an observed
difference, expressed in standard deviations in order to compare across different
types of measures.
GPS Government Primary School
HLE Home literacy environment: Hess and Halloway (1984) identified five
dimensions of the home literacy environment that are theoretically related to
reading achievement in children: value placed on literacy, press for achievement,
availability and use of reading materials, reading with children, and opportunities for
verbal interaction.
ICC Inter-cluster Correlation: the proportion of variation in a measure that is
explained by the fact that all observations (students) are clustered in units
(schools/classrooms)
ICT Information Communication Technology
LB Literacy Boost
MUW Most-Used Words: see below
PTA Parent-Teacher Association
RBM Reading Buddies and Mentoring
RNGPS Recognized Non-Governmental Primary School
RWC Reader with Comprehension: see below
SC Save the Children
SES Socio-economic Status
SMC School Management Committee
WPMC Words per Minute Correct: see below
2
Reading Outcome Definitions
Letter Identification The number of letters (out of all 50 letters of the Bangla alphabet) for
which the child either correctly gave the name, the sound, or a word
that begins with that letter.
Most Used Words The number of words (out of 20 of the most frequently used words in
children’s grade 3 textbook) correctly read aloud by the child.
Pseudo Words The number of nonsense words (out of 20) correctly read aloud by the
child, as a test of children’s decoding skills.
Reader A child who is able to read the oral reading passage independently, here
defined as reading at least five words correctly in the first 30 seconds of
the sub-test. Readers were allowed to continue reading until they
finished the passage or refused to read any further; non-readers were
stopped and read the passage by the assessor.
Fluency Tested during the oral reading passage sub-test, fluency is defined as the
number of words read correctly per minute. As assessors marked the
child’s progress at 30 seconds, this measure is calculated by counting
the number of words correct at the 30 second mark and multiplying this
number by two.
Accuracy1 Tested during the oral reading passage sub-test, the percentage of the
total words in the passage read correctly by students.
Reading Comprehension Children’s ability to correctly answer 10 questions following the
administration of the oral reading passage sub-test.
Readers with
Comprehension Defined as at least 75% of reading comprehension questions correctly
answered (in this case, at least 4 of 5 questions correct) and either at
least one standard deviation of fluency lower than the average for
children who scored 4 of 5 comprehension questions correct or at least
one standard deviation of recognition lower than the average for
children who scored 4 of 5 comprehension questions correct, based on
baseline scores. Thus, to meet this threshold children must score at
least 4 of 5 comprehension question correct and read either at least
33.2 words per minute correct or with 96% recognition.
1 In the body of Literacy Boost assessment research in over 15 countries, as well as in much of the international
education community’s work on early grade reading assessments, ‘accuracy’ is the standard term to refer to the number or percentage of words read correctly from the oral reading passage sub-test of reading assessments. However, there are multiple definitions of what ‘accuracy’ means, and in particular there may be some controversy within Bangladesh. This report will use the term ‘accuracy’ to refer to the abovementioned definition, as a proxy measure of children’s ability to recognize words while reading connected text, acknowledging that other definitions of accuracy exist.
3
Executive Summary
In 2011, Save the Children (SC) began the planning for the 2012 implementation of Literacy
Boost in Meherpur District of Bangladesh. This intervention focuses on working with teachers
and communities to improve children’s reading skills. A baseline assessment of 623 children in
grade 3 was taken in February-March 2012, and in March 2013, follow-up data on 465 (75%) of
the same children was collected to investigate change in Bangla reading skills. This report details
the change in Bangla reading skills of students who were in grade 3 during the 2011 baseline and
who were in grade 4 during the 2013 endline. The assessment tested children’s skills in letter
identification, reading of most-used words and pseudo words, oral reading fluency and accuracy,
and reading comprehension. Inter-rater reliability for each of these skills was excellent. The 465
assessed children come from 18 Literacy Boost schools, which received one year of teacher
training and community activities, and 14 nearby comparison schools, which received only the
standard Save the Children Basic Education Sponsorship interventions.
Attrition analysis reveals that attrition rates were similar for Literacy Boost and comparison
students overall, but when disaggregating by school type Literacy Boost students were
significantly more likely than comparison students to be absent in Government Primary Schools
(GPS) but significantly less likely to be absent in Madrasas. Overall, the oldest students and
those with lower reading accuracy scores were the most likely to be absent on the day of
endline assessment. Although these associations are correlational only and causation
cannot be established, older children and those struggling to learn to read
connected text may be more at-risk of not attending or dropping out of school, and
thus may need more special support to continue their education than their
younger and better-reading peers.
Of the remaining sample of 255 Literacy Boost children and 210 comparison children, the two
groups are equal on almost all measures and thus comparable for a difference-in-difference
impact analysis. When investigating the change in children’s reported home literacy
environment (HLE), children reported no substantial change in availability of different types of
print in their household, and diversity of print remains limited to textbooks, religious books,
and storybooks with low prevalence of coloring books and comics. In terms of changes
between baseline and endline in the number of household members reported to be engaging in
literacy-promoting activities the number of household members reported to been seen reading
by children almost doubles for both Literacy Boost and comparison students, and the increase
in the number of household members reading to the child is also substantial. This increase is
very important for supporting children’s literacy development, but more work
remains to be done. For example, although children reported that nearly all
household members encouraged them to study (90%), they also reported that
fewer than half of household members told stories to them. As storytelling does
4
not require literacy, this is an activity that nearly all household members can
engage children in to help boost their reading skills.
Children were also asked what they do most often to learn to read better, and their answers
were classified into one of five categories by assessors. Children overwhelmingly gave a
response linked to practicing/studying in the community or with the help of a family or
community member. This may indicate that children naturally view learning to read as
an activity that takes place primarily in the community and with their family
members.
Analysis of the impact of the Literacy Boost intervention reveals that overall, the Literacy
Boost only modestly helped children improve their fluency in RNGPS and their
fluency and accuracy in Madrasas. However, Literacy Boost significantly helped the
subset of children who struggled at baseline improve their decoding skills, make
the jump to being able to read connected text independently, and establish a
degree of fluency, accuracy, and even reading comprehension. Nevertheless, it will be
important to better understand why Literacy Boost did not significantly help children master
their letters and most-used words and why Literacy Boost was unable to raise the reading
scores of all children (not only primarily those who struggled at baseline). Some potential
explanations include statistically equal Literacy Boost and comparison changes in children’s in
book-borrowing from school, reading to others in the community, seeing household members
read, and having household members read to them. Evidence from multilevel regression analysis
examining the relationship between the frequency of Literacy Boost student participation in
Literacy Boost community activities also revealed that frequency of meeting with reading buddy
was significantly correlated with all skill gains except reading comprehension. Given the
strong association of frequency of reading buddy participation with reading skill
gains, the fact that comparison schools also had a reading buddy and mentoring
system may explain the limited overall impact of Literacy Boost versus the
comparison group.
An additional impact analysis was performed looking at equity. No noteworthy results were
found for girls, the poorest of the poor, the HLE-deprived, or those children with large
chore/workloads. As the poor, the HLE-deprived, and those with large
chore/workloads exhibit lower endline reading skill performance than their peers,
more work remains to be done to ensure that these groups not only benefit from
Literacy Boost but actually close the gap with their better-off peers.
Aside from reading buddies, frequency of reading camp attendance was associated with gains in
becoming a reader, fluency, accuracy, and becoming a reader with comprehension. Frequency
of participation in make and take activities during reading camps was linked to reading score
5
gains in most used words, pseudo words, and accuracy. Book bank borrowing frequency was
linked to fluency and reading with comprehension gains. Using an index of participation in
Literacy Boost community activities, children in the lowest quintile of participation were
predicted to gain nothing or less than their peers who fell in higher levels of participation. This
is evidence that Literacy Boost community activities may have aided children’s
reading skill acquisition.
Impact and differences between Literacy Boost and comparison samples aside, children only
met two of the reading skills benchmarks set at baseline, in fluency and reading comprehension.
Children still exhibit gaps in their lower-order skills of letter knowledge and most-used and
pseudo words, and a divergence of readers and nonreaders occurs for higher-order skills like
fluency and accuracy measured during the oral reading passage. Literacy Boost should focus
on closing these lower-order skill gaps, and on supporting the subset of children
who are still falling behind, perhaps through engaging remedial community
activities or through partnering them with some of the strongest readers. In
general, the formative assessment component of teacher training should be
emphasized to help teachers identify children at risk of falling behind early and to
diagnose how to help these children keep up with the class. Teacher training should
also give teachers practical suggestions for instructing classes with varied levels of
students.
Finally, correlations between student background and reading skills gains were examined
through multilevel regression analysis. This analysis indicates that older children and those who
repeated one or more grades did not improve their scores as much between baseline and
endline as their younger peers and those students who had never repeated a grade. Both
older children and those who have repeated may need special remedial help in
order to keep pace with other children.
6
Introduction
In Bangladesh, reading is a competency expected to be mastered by primary school students in
both Bangla and English. The curriculum expectations set out that children should be fluent in
reading both languages by the end of grade two, but gathering of reading data occurs only
during Ministry of Education grade 5 terminal examinations. Save the Children staff observe that
reading with comprehension only occurs by grade 4 in Bangla and grade 5 or later for English.
As there is not yet an official focus on children’s reading with comprehension prior to grade 5,
Save the Children decided to study the reading skills of children in grade 3 to learn more about
how children are learning, or struggling to learn, to read in the earlier grades. This series of
studies was also meant to test the Literacy Boost (LB) intervention’s potential for improving
children’s learning.
The Literacy Boost program includes teacher training, community reading activities, and age-
appropriate local language material creation to support emergent literacy skills among early-
grade children. These skills include concepts about print, letter awareness, single word reading
of most used words, reading fluency, reading accuracy, and reading comprehension.2
In 2011, Save the Children began the planning for the 2012 implementation of Literacy Boost in
Meherpur District of Bangladesh. This intervention focuses on working with teachers and
communities to improve children’s reading skills. The Literacy Boost Assessment component
features a set of adaptable early grade assessments used to detail the skills present when
Literacy Boost begins, to inform implementation of teacher training and community activities,
and to chart progress throughout the intervention. Thus, a September 2011 baseline
assessment of children’s Bangla and English reading skills was conducted. Based on the context
and initial 2011 assessment results, the teacher training and community action components
were adapted and implementation in Bangla began in 2012. Just prior to implementation, a
second baseline of 623 children in grade 3 was taken in February-March 2012.
In March 2013, follow-up data on 465 (75%) of the same children were collected to investigate
change in Bangla reading skills. As of early 2013, implementation of English-language activities
had not yet commenced and the English skill-level of hired assessors was insufficient to collect
quality data on children’s English reading skills, and thus the 2013 endline assessment was not
conducted in English. This report details the change in Bangla reading skills of students who
were in grade 3 during the 2011 baseline and who were in grade 4 during the 2013 endline.
These children come from 18 Literacy Boost schools, which received one year of teacher
training and community activities, and 14 nearby comparison schools, which received only the
standard Save the Children Basic Education Sponsorship interventions of Reading Buddies and
Mentoring (RBM), as well as School Management Committee (SMC) capacity-building, parent
awareness, and school drop-out prevention.
2 Writing is not yet a part of the Literacy Boost program, but may be integrated in the future.
7
Key interventions focused on grades 1-3 that are continuing to be implemented in the 18
Literacy Boost schools to date include:
1. a 7-module teacher training on explicit reading instruction
2. provision of community-based Book Banks
3. A Reading Buddy and Mentoring system both in schools and Reading Camps in which
older children read to and with younger children
4. conducting regular community reading awareness sessions with parents, schools, and
SMCs
5. conducting weekly Reading Camps run by trained Reading Camp Leaders
6. conducting Story Time activities in the community
7. conducting Community Read-A-Thon/ Reading Festivals
At the time of this endline assessment in the month of March 2013, Appendix A details the
activities that had been implemented by Save the Children and the dates and frequency of the
intervention component.
Context
SC Bangladesh’s sponsorship-funded program known as Shishuder Jonno (“For the children” in
Bengali), aims to ensure that children in Meherpur learn and develop to their full potential. The
program provides support to children and their families at every stage of life through four core
programs: Early Childhood Care and Development (ECCD), Basic Education (BE), School
Health and Nutrition (SHN) and Adolescent Development (AD), following Save the Children
International’s Common Approach for Sponsorship Program (CASP) modules. In addition to
the four core programs, Shishuder Jonno also implements cross-cutting Child Protection (CP)
and Community Mobilization (CM) activities across the implementation area, as well as select
innovation and pilot projects that may be scaled up within the program or in other Save the
Children programs throughout Bangladesh. Literacy Boost pilot is the part of Basic Education
program.
Since 2008, Shishuder Jonno has been operating in all three upazilas (sub-districts) of Meherpur.
In the sub-district of Meherpur Sadar, SC works through direct delivery. In Mujibnagar and
Gangni sub-districts, SC through partner NGO Jagoroni Chakra Foundation (JCF).
Situated in Khulna Division, Meherpur is located approximately 245 km northwest of the
Bangladeshi capital Dhaka and is the smallest District in the country. It consists of three
upazilas: Meherpur Sadar, Mujibnagar and Gangni. Within the upazilas there are two
municipalities, 18 unions and 249 villages. Although Meherpur shares a 118 km border with
India, migration of the 591,436 inhabitants of Meherpur to India is rare.
8
Despite certain climatic advantages, including fertile land with more than one growing season
and a slightly higher land elevation level in comparison to other parts of the country, many
marginalized communities in Meherpur face the socioeconomic challenges. The adult literacy
rate in Meherpur is 37.4% (compared to national average of 53% UNESCO 2007). The primary
education survival rate to Grade is 41.1% compared to the national rate of 51.9% (PDEP II). In
spite of high primary school enrollment rates, only about 58% students are regularly attending
school and dropout rates are high. Moreover, there are tremendous disparities between
advantaged and disadvantaged groups and, generally rural areas, are considered the least
advantaged.
Methodology
Sample Selection
There are 6 unions (including an urban union hereafter referred to as the Meherpur
‘municipality’) and 115 schools within these unions in the Meherpur sub-District. All schools in
the sample receive services from Save the Children that include: active SMC training and PTA
formation; promotion of enrollment and attendance via celebrations/campaigns, the
establishment of playgrounds, children’s fifth grade performance recognition awards; Reading
Buddies and Mentoring, and support to the MoE’s supplementary reading materials program,
parent awareness, and school drop-out prevention.
When selecting the schools for Literacy Boost intervention, the Save the Children team and
partners on the ground placed priority in having the Literacy Boost pilot spread across
Meherpur, so the sample was stratified by union. Sixteen schools in Meherpur in which Save the
Children supports another intensive intervention with information communications technology
(ICT) and other elements were excluded from the list of possible pilot sites. From the 99
remaining schools, 5 were randomly selected from each union and municipality for a total of 30
schools. Three schools were then randomly assigned to treatment and two randomly assigned
to control. After this selection, the team considered the mix of school types – government
primary school (GPS), registered non-government primary school (RNGPS), Madrasa. After
observing that there were no Madrasas in the control group, the team randomly selected two
more from among the nine Madrasas in the district. This yielded a total of 18 treatment schools
and 14 control schools in the research sample. Among the treatment group are two Madrasas,
seven RNGPS and nine GPS. This enabled an investigation of how Literacy Boost works across
school types, although with limited statistical power for Madrasas due to their small number.
Within each of the 32 schools where data was collected, 20 children – 10 boys and 10 girls – in
grade 3 were sampled at baseline. If there were more than these 20 children in the classroom,
then this was done randomly. A grade 3 class was randomly selected within the school if there
was more than one grade 3 classroom at the school. The resulting sample had 623 students,
350 in Literacy Boost schools and 273 in comparison schools. At endline, an attempt was made
9
to assess as many of these same children as could be found at the school on the day of the
follow-up assessment. As a result, 255 Literacy Boost children and 210 comparison children
were assessed at endline for a total baseline-endline sample of 465 children. Table 1 details how
these groups are spread across the school types. A more thorough discussion of attrition will
be presented in the Attrition Analysis section of this report.
Table 1: Number of Children in Literacy Boost and Control Schools at Baseline and Endline, by
School Type
School type Literacy
Boost
Baseline
Literacy Boost
Endline
(% remaining)
Comparison
Baseline
Comparison
Endline (%
remaining
Madrasa 35 29 (83%) 37 22 (59%)
Government Primary School 200 132 (66%) 78 65 (83%)
Registered Non-Government
Primary School
115 94 (82%) 158 123 (78%)
Instruments and Measurement
School profile data were collected via direct observation and a survey of school headmasters
and grade 4 Bangla teachers at all but one school in the sample. This data includes information
on school facilities & construction, classroom print environment, observed teacher practices,
and self-reported teacher practices. The classroom observation and teacher interview tool in
particular was piloted in order to balance out data collected on out-of-school factors
hypothesized to influence children’s literacy acquisition and to better understand the extent to
which in-school factors explain children’s learning. However, as no reliability analysis was
performed, the tool needs further refinement, and observed teachers were not necessarily
those who had received Literacy Boost training, a discussion of the results is confined to the
Appendix (Appendix B) and is to be considered a preliminary attempt to explain the
relationship between teacher practices and children’s reading outcomes.
For the student assessment, all students in the sample were asked about their background
characteristics (age, household possessions, household building materials, health indicators,
etc.). Students were also asked about their family members and their reading habits (who they
had seen reading in the week prior to the assessment, who had read to them, etc) and students
in the Literacy Boost sample only were asked about their participation in Literacy Boost
community activities such as Reading Camps, Reading Buddies, and Book Bank borrowing.
After collecting this background data, all students were given a reading assessment testing seven
outcomes administered through five sub-tests: letter knowledge, single word recognition
10
(reading of most used words), decoding (reading of pseudo words), reading fluency & accuracy3
(words per minute read correctly and total percentage of passage read correctly; both within
the same sub-test), and a set of comprehension questions linked to the fluency & accuracy
passage. The same set of comprehension questions were administered for both those students
who could read at least five words correctly in 30 seconds (reading comprehension) and those
who could not and thus had the assessor read to them (listening comprehension). These
assessments were developed and pilot tested prior to baseline data collection using the Literacy
Boost Toolkit Assessment Component. All assessment instructions were given in Bangla. Table
2 offers examples of school survey items, background and home literacy indicators, and a
detailed description of reading indicators.
3 In the body of Literacy Boost assessment research in over 15 countries, as well as in much of the international
education community’s work on early grade reading assessments, ‘accuracy’ is the standard term to refer to the number or percentage of words read correctly from the oral reading passage sub-test of reading assessments. However, there are multiple definitions of what ‘accuracy’ means, and in particular there may be some controversy within Bangladesh. This report will use the term ‘accuracy’ to refer to the abovementioned definition, as a proxy measure of children’s ability to recognize words while reading connected text, acknowledging that other definitions of accuracy exist.
11
Table 2: Data Collected
School Survey Examples
Classroom observation General pedagogical techniques, reading techniques, types of
reading skills addressed, student participation
Teacher interview General pedagogical techniques, types of reading skills addressed
General/ School
Observation Teacher turnover, school facilities, presence of SMC/PTA
Student background Examples
General Sex, age, language spoken at home
School-related Distance to school, repetition history, previous ECD attendance
Socioeconomic status Household amenities/possessions
Children’s Time Type of chores, amount of time spent on chores, amount of time
spent studying
LB Participation (LB students only) frequency of participation in community
activities
Home Literacy Environment
Access to print Types of materials present in home
Reading at home Presence and percentage of family members who children see
read, and who read
Reading Outcome Description
Alphabet knowledge Number of letters/sounds known of 50
Single Word Reading Number of single words read correctly of 20
Decoding Number of pseudo-words read correctly of 20
Fluency Number of words in an 86-word connected text read correctly
per minute, calculated based on the child’s progress at 30 seconds4
Accuracy Percentage of words in an 86-word connected text read correctly
(only for readers)
Listening
Comprehension
Number of comprehension questions answered correctly of 5
after listening to a text read aloud by the assessor (only for non-
readers)
Reading
Comprehension
Number of comprehension questions answered correctly of 5
after reading a text read aloud (only for readers)
4 When 30 seconds had elapsed since the children began reading the connected text, assessors were instructed to
quickly calculate if the children had read at least five words correctly. Children who had read at least five words correctly in 30 seconds (hereafter referred to as readers) were allowed to continue reading until they finished the text. Children who had not read at least five words correctly in 30 seconds (hereafter referred to as nonreaders) were stopped by the assessors, at which point the assessors read the passage to them and asked the same set of comprehension questions for a test of listening comprehension.
12
To test inter-rater reliability, or the consistency with which children’s responses were
recorded the same way between different assessors, 14% of students (n=64) were assessed by
two assessors simultaneously. Long one-way analysis of variance (ANOVA) techniques were
used to calculate the intra-class correlation within pairs of assessors for a measure of reliability.
Table 3 presents the results below. Using Fleiss’ benchmarks for excellent (ICC>0.75), good or
fair (0.75>=ICCA>0.4), and poor (0.4>=ICC).5 The inter-rater reliability for all sub-tests
falls within the excellent range, and thus different assessors consistently scored
children’s responses to the literacy assessment.
Table 3: Inter-rater reliability by sub-test
Sub-test Inter-rater
Reliability (ICC)
Reliability Rating
Letters .99 Excellent
MUW .99 Excellent
Pseudo Words .99 Excellent
Reader .93 Excellent
WPMC .94 Excellent
Accuracy .99 Excellent
Reading Comprehension .98 Excellent
Analysis
This report will use comparison of means through clustered t-tests to assess the comparability
of the two groups and consider differences between boys and girls scores. Finally, this report
will consider relationships between reading skills and background factors using regression
models that account for clustering in schools.
Attrition Analysis
Was attrition during endline different in Literacy Boost schools than in comparison schools?
This endline assessment includes the same 18 Literacy Boost schools and 14 comparison
schools from neighboring communities that were assessed in the 2012 sample. At baseline 623
students were assessed, but only 465 students could be located at end-line6. While the attrition
rate in the Literacy Boost sample (27%) was slightly higher than the comparison sample (23%),
this finding is not statistically significant at a 5% level and thus this observed difference in
samples cannot be generalized to the population of fourth grade students in the 18 Literacy
Boost and 14 comparison schools as a whole.
5 Fleiss JL. The Design and Analysis of Clinical Experiments. New York, NY: John Wiley & Sons Inc;1986 . 6 Please note, this does not necessarily mean that students dropped out. There are many possible reasons for
student absence, and we can only note that the students were absent on the particular day of the assessment. The
following findings do not establish why students were absent.
13
Were students who were absent from the endline assessment different in some way than those who
were present?
Of all the variables collected about students’ socioeconomic status, home literacy environment,
and reading skills, only two variables collected at baseline robustly predict the type of student
who might have attritted at endline. The technique used to arrive at these findings was a
multilevel logistic regression analysis, with students clustered in schools.
Age was found to be significantly positively associated with a higher likelihood of attrition and
children’s baseline reading accuracy scores was found to be significantly negatively related to a
higher likelihood of attrition. That means that older children and those with lower reading
accuracy scores at baseline were more likely to have attritted at endline. Note that these
associations are correlational only and causation cannot be established. However, within the
same grade, those children who are older and those struggling to learn to read
connected text may be more at-risk of not attending or dropping out of school, and
thus may need more special support to continue their education than their
younger and better-reading peers who are in the same grade.
Figure 1 below displays the relationship between two of these variables, age and accuracy, and
the likelihood of attrition at endline.
Were students in Literacy Boost schools who were absent at endline different from those who were
present at endline?
Another important area for analysis looks at how those LB students who were not present at
endline differed from LB students who were present at endline. This will have implications for
who in Literacy Boost schools are still struggling with attendance. While trends were largely
0%
10%
20%
30%
40%
50%
0% 50% 100%
Like
liho
od
of
attr
itio
n
Accuracy Score
Figure 1: Likelihood of Attrition by Age and
Accuracy Score
8 years old
10 years old
14
similar for LB and comparison students, Figure 2 below indicates that lack of previous ECD
attendance is significantly correlated with likelihood of attrition at endline. While not
conclusive evidence, it may be that Literacy Boost students with no prior ECD
experience may need special attention through remedial activities, and this
evidence can be used to impress upon parents and ministry officials alike the
importance of ECD.
Endline Descriptive Statistics Analysis
Student Characteristics
Were students who were present at both baseline and endline similar enough for
comparison?
Table 4 summarizes student background data, presenting the averages for all Literacy Boost and
comparison school students as collected at baseline and endline. Of the sample of 465 students
who were assessed at both baseline and endline, t-tests using clustered standard errors to
account for the clustering of children in schools reveal that Literacy Boost and comparison
children are remarkably similar on average. It is possible that the few differences displayed in
the table below are due purely to chance. Thus, children in Literacy Boost and
comparison groups are similar enough to perform a difference-in-difference impact
evaluation of the Literacy Boost intervention.
27%
20%
0%
5%
10%
15%
20%
25%
30%
No ECD ECD
Like
liho
od
of
attr
itio
n
Figure 2: Likelihood of Attrition by
Prior ECD Attendance for LB
Students Only
15
Table 4: 2012 Baseline and Endline Student Background Data by LB/Comparison Group
Variable
Baseline
LB
Baseline
Control Significance
Endline
LB
Endline
Control Significance
Age in years 9.4 9.4 10.2 10.2
Prior ECD attendance 59% 56%
64% 58%
Has repeated a grade 34% 44%
40% 59%
# of time transferred
schools 12% 5% * 16% 12%
# of household possessions 3.2a 3.1a
4.9b 4.7b
Hhold has electricity 73% 69%
76% 68%
Hhold has refrigerator 3% 0%
3% 3%
Hhold has TV 45% 46%
49% 44%
# hhold members 4.5 4.6
3.4 3.4
Minutes to school 15.7 14.1
Reports distance to school
is 'far'
27% 37%
Absent at least one day last
week 44% 46%
34% 33%
# of types of chores/work
reported 2.3 2.2
Time spent on morning
chores/workc 0.9 0.9
Time spent on afternoon
chores/workc 0.2 0.2
Time spent on evening
chores/workc 0.8 0.9
Time spent on weekend
chores/workc 1.4 1.5
Minutes of study per day 170.1 180.0 202.6 206.5
Significant difference between LB and control groups at *p<.05 a Total # of possessions of 6 b Total # of possessions of 9 c 0 = No time, 1 = A short period of time, 2 = A long period of time
The endline student background data reveal that children are now about one year older than at
baseline, for an average age of slightly over 10 years on average. Over half of children have
attended some form of ECD – BRAC (62%), Islamic Foundation (20%), and SC Preprimary
(14%) are the most frequently reported type of ECD. About half of students have ever
repeated a grade. Boys are significantly more likely to have repeated a grade (58% for boys
versus 39% for girls) and thus are significantly older on average (10.5 years for boys versus 10
years for girls). Fifteen percent of students have changed schools – mostly because of their
parents’ wishes or due to lack of higher grade levels at their old school. Of nine possessions
16
asked after (electricity, refrigerator, TV, cow, goat, poultry, land, bike, and motorbike), children
report having about half on average, with a normal distribution.
Curiously, children report living with slightly less household members on average than at
baseline. This may be due to an actual change, but it may also be due to children’s evolving
perceptions of who they count as a household member, or to a change in the questionnaire
itself.7 The question about distance to school was changed in light of the trouble children had
estimating the precise number of minutes it took them to reach school, and instead they were
asked if school was very close, neither very close nor very far, or very far. One-third of
children reported that school was ‘very far’ for them to reach. At baseline, a similar proportion
of children reported that it took them 20 minutes or more to arrive at school.
One-third of children reported being absent from school at least one day in the week prior to
assessment, down about ten percentage points from baseline statistics. Of those who missed at
least one day, children reported being absent most frequently for leisure (29%), sickness (23%),
work (19%), or because school was closed (15%). On average, children report performing two
types of chores, sometimes three. Boys are slightly less likely to perform chores than girls in
general, and are more likely to perform chores such as tending animals, watering plants, and
tending the land/working in the fields. Girls are more likely to clean, cook, or look after siblings
(findings that are statistically significant). When asked whether they spend ‘no time,’ ‘a short
period of time,’ or ‘a long period of time’ on chores during the morning, afternoon, evening,
and weekends, children appear to be most busy on the weekends followed by the weekday
mornings and evenings. While girls are more likely to perform chores and perform more types
of chores on average than boys, girls and boys appear to spend an equal amount of time on
chores.
Finally, children’s study time appears to have increased from a baseline average of about three
hours per day to just under three and a half hours per day. As with any of the other variables
that exhibited a change in average value between baseline and endline, the change could either
reflect reality or could be the result of measurement error when asking young students to
estimate precise quantities. As the students have matured from baseline to endline, we will
assume that the endline values are more accurate (contain less error) and hence we will use
these endline values when analyzing the data later in this report.
7 At baseline, assessors used a grid to record this information on paper. However, at endline data was collected
with electronic tablets, which changes the nature of the question – instead of immediately recording all household members the child lives with, assessors must ask all questions about each household member one by one until the child cannot think of anyone else.
17
School Characteristics
Table 5 below summarizes the results of the school observation checklist. Equal numbers of
schools are urban between the two sample groups, but the Literacy Boost sample contains a
much higher percentage of Government Public Schools (GPS) than the comparison sample.
Conversely, the comparison sample contains a much higher percentage of registered non-
governmental primary schools (RNGPS) than the Literacy Boost sample. The proportion of
Madrasas is almost equal, although school observation data for one comparison madrasa was
left out of the statistics reported below due to data entry error.
Table 5: School Characteristics8
Variable LB Comparison
Urban 17% 15%
GPS 56% 31%
RNGPS 33% 62%
Madrasa 11% 8%
Classroom has blackboard 100% 100%
Estimated % of children who can see board 98% 100%
Distance to district center in km 9.0 9.6
Distance to nearest paved road in km 0.42 0.42
School has electricity 83% 77%
School has water point 100% 100%
Water point is treated 94% 100%
School has latrine for students 100% 100%
School has separate latrine for girls 94% 100%
Latrine has anal cleansing material 89% 92%
Latrine is clean 89% 100%
School has hand washing station 89% 100%
School has library 100% 100%
Students can borrow library books 94% 100%
Total # of teachers transferred in and out this
year 0.3 0
School has SMC 100% 100%
School has PTA 100% 100%
School infrastructure appears to be present in general, although in most cases the quality of the
infrastructure was not taken into account. Nearly all classrooms have a blackboard that an
estimated 100% of children can see, and most all schools have electricity, treated water points,
and clean latrines for students (including separate latrines for girls) with anal cleansing material
8 For 31 of the 32 schools; data for one madrasa is missing due to data entry error.
18
and hand washing stations. Very importantly, nearly all schools have libraries from which
students can borrow books, as well as SMCs and PTAs. Teacher transfer is very low. No
notable differences between Literacy Boost and comparison schools appear to exist. Findings
from the pilot teacher observation and interview can be found in Appendix B.
Home Literacy Environment Data
Just as important as investigating children’s school and classroom environment is examining
their home literacy environment (HLE). The HLE is not only about materials in the home, but
how those materials are used to engage the child in reading and learning. Hess and Holloway
(1984) identified five dimensions of the home literacy environment that are theoretically related
to reading achievement in children. The first is value placed on literacy, which the Literacy Boost
assessment operationalizes by asking children whether they see anyone reading at home. The
second is press for achievement, is operationalized as individuals telling the student to study. The
third is the availability and use of reading materials, operationalized as the amount of printed
materials at home. The fourth dimension is reading with children, operationalized by asking the
children whether anyone reads to them at home. The last is opportunities for verbal interaction,
operationalized as family members telling stories to learners.
In terms of the third dimension of availability and use of reading materials, Figure 3 below shows
that children’s self-reported presence of various types of print in the household remained fairly
stable over time. Textbooks are not displayed here as all children reported having them in the
household. Statistically significant differences that existed between Literacy Boost and
comparison children at baseline with regard to religious books and storybooks seem to have
disappeared by endline as comparison students reported significantly higher prevalence of these
materials at endline. It also appears that both Literacy Boost and comparison children
experienced a significant increase in the prevalence of coloring books in the household,
although the percentage of children reporting these in their homes remains low. In terms of
comparability between Literacy Boost and comparison samples, it appears Literacy
Boost children had more religious books and storybooks at baseline, giving them a
significantly higher number of types of reading materials available at baseline than
comparison students. As an important difference, this will be controlled for during
impact analysis.
19
Significant difference between LB and control groups at baseline at *p<.05
Figure 4 does appear to show changes between baseline and endline in the number of
household members reported to be engaging in specific activities corresponding to the
remaining four dimensions of the HLE. For the number of household members reported to
been seen reading by children, the number almost doubles for both Literacy Boost and
comparison students, and the increase in the number of household members reading to the
child is also substantial. This is even more remarkable factoring in that children reported less
household members on average at endline than at baseline. This increase is very important
for supporting children’s literacy development, but more work remains to be done.
For example, although children reported that nearly all household members
encouraged them to study (90%), they also reported that fewer than half of
household members told stories to them. As storytelling does not require literacy,
this is an activity that nearly all household members can engage children in to help
boost their reading skills.
Significant difference between LB and control groups at *p<.05
Note that household members in school was not collected at endline, and household members encouraging the
child to study and telling stories to the child was not collected at baseline.
89%
0% 8%
74%
14% 2%
91%
1% 8%
73%
12% 2%
0%
20%
40%
60%
80%
100%
Religiousbooks
Magazines Newspapers Storybooks ColoringBooks
Comics% c
hild
ren
re
po
rtin
g ty
pe
of
pri
nt
Figure 3: Types of Reading Materials Reported in the
Home
Baseline LB
Endline LB
Baseline Control
Endline Control
* * *
*
1.1* 0.9
2.0 1.5
3.1
1.5
0.9* 0.8
1.9 1.5
3.0
1.5
0.0
0.5
1.0
1.5
2.0
2.5
3.0
3.5
seen reading reading to child encouragingchild to study
telling story tochild
Nu
mb
er
of
ho
use
ho
ld
me
mb
ers
Figure 4: Number of household members…
Baseline LB
Endline LB
Baseline Control
Endline Control
20
In terms of sex, girls report more family members reading to them and telling them stories on
average than boys. And again, Literacy Boost children enjoyed a baseline advantage
over their comparison peers in terms of number of household members seen
reading. This difference will be controlled for during endline analysis. There were
no other significant difference between Literacy Boost and comparison students.
Children were also asked about their own literacy habits, again displaying substantial changes
between baseline and endline. Many less children reported borrowing books from school at
endline than at baseline, especially children in the Literacy Boost sample. On the other hand,
many more children reported reading a book to others in the week prior to the assessment,
and the increase was especially marked for comparison children. It should be noted that these
questions changed between baseline and endline – at baseline children were asked if they
borrow books from school and read to others in general, but at endline children were asked if
they borrow books from school and read to others in the past week. In addition, the endline
question for borrowing books from school specifically referenced the RBM program (linked to
Book Banks), and SC Bangladesh staff indicate that the school-based Book Banks were
discontinued in favor of the community-based Book Banks. This may explain the large change in
children’s responses, especially for the school book-borrowing data. However, the issue of the
difference in change between Literacy Boost and comparison children still remains unexplained.
Significant difference between LB and control groups at **p<.01
Note that sharing books with others in the community in the past week was not collected at endline
Finally, children were asked what they do most often to learn to read better, and their answers
were classified into one of five categories by assessors (see Figure 6 below).9 Children
9 All components to this question exhibited inter-rater reliability of 0.8 and above.
70%
39%
29% 30%**
62%
76%
35% 26%
55%**
71%
0%
10%
20%
30%
40%
50%
60%
70%
80%
Borrows booksfrom school
Read book toothers in
community in pastweek
Shared book withothers in
community in pastweek
% o
f ch
ildre
n r
ep
ort
ing
hab
it
Figure 5: Child Literacy Habits by Sample
Group
Baseline LB
Endline LB
Baseline Control
Endline Control
21
overwhelmingly gave a response linked to practicing/studying in the community or with the help
of a family or community member. Half of students also mentioned practicing/studying a specific
lower-order component of reading such as letters, sounds, words, etc. No significant
differences exist between Literacy Boost and comparison children in these
responses, which could indicate that children naturally view learning to read as an
activity that takes place primarily in the community and with their family
members.
Literacy Boost Participation
The last set of background questions was asked only of children in the Literacy Boost sample.
These questions asked children about their participation in various Literacy Boost community
activities. Nearly all children in the Literacy Boost sample said they had borrowed books from
book banks in the past year, had reading buddies, and had attended reading camps in the past
year. Of those who said they participated in these activities, children said they borrowed books
from the book banks between three and four times per month, said they read with their
reading buddy about three times per week, and said they attended a reading camp between
three and four times per month. About two-thirds of children who had attended the reading
camps said they had created make-and-take materials at the camps, 81% ‘sometimes’ and 19%
‘always.’
50%
16% 29%
80%
7%
0%
20%
40%
60%
80%
100%
letters, sounds,words, etc.
reading books in school/ with helpof teacher
in community/ withhelp of
family/communitymember
another way% o
f ch
ildre
n g
ivin
g re
spo
nse
re
late
d t
o c
ate
gory
Figure 6: Q. What do you do most often to learn to read
better? A. I practice/study...
22
Impact on Emergent Literacy Skills
In this section we examine gain scores in each literacy skill to assess the impact of Literacy
Boost on student learning outcomes. In Table 6, baseline & endline data are presented for both
Literacy Boost and comparison students. Furthermore, the change from baseline to endline is
presented (gain scores), along with the initial benchmark10 set during the baseline assessment
and whether the benchmark was met. Clustered t-tests are were used to test whether the
findings are generalizable to the population, and standardized effect sizes are also reported to
give a comparable sense of magnitude of the difference in Literacy Boost and comparison gain
scores across all skills.
10
A Note about Benchmarks: While Save the Children has used this approach to reading assessment in
Bangladesh, Ethiopia, Guatemala, Haiti, Malawi, Mali, Mozambique, Nepal, the Philippines, Sri Lanka, South Africa,
Vietnam, Uganda, Yemen, and Zimbabwe, comparison across countries and languages is less helpful than more
detailed contextual information for setting expectations of impact. For each measure used in these assessments,
the baseline established the upper end of the range of scores (the 75th percentile) as a benchmark, or a reasonable
estimate of what is currently possible among these children.
96% 92% 96%
64%
0%
20%
40%
60%
80%
100%
Borrowed book bankbooks in past year
Has reading buddy Attended readingcamp last year
Created make and takematerials at reading
camps
% o
f ch
ildre
n r
ep
ort
ing
par
tici
pat
ion
Figure 7: Participation in LB Community Activities
23
Table 6: Baseline/Endline Scores with Benchmarks, Significance, and Effect Sizes
A B C D E F G H Ib
Reading
Skill Group
Baseline
Scoresa
Endline
Scoresa
Benchmark
from
Baseline
Benchmark
met?
Change
from
Baseline
to
Endline
Sig. Diff.
of Gain
Scores
Between
Groups
Effect
Size of
Diff in
Gain
Scores
Letter
Identification
(of 50)
LB 44.4 44.5
48 No
0.1
None 0.05 Comparison 45 44.7 -0.3
Most Used
Words (of 20)
LB 13.9 17.2 18 No
3.3 None 0.03
Comparison 14 17.2 3.2
Pseudo Words
(of 20)
LB 9.8 13.7 14 No
3.9 None 0.04
Comparison 9.3 13 3.7
% Readers LB 83% 88%
N/A N/A 5
None 0.18 Comparison 85% 83% -2
Fluency (Words
Per Minute)
LB 28.3 56.7 42 wpm Yes
28.4 None 0.22
Comparison 29.1 51.6 22.5
Accuracy (% text
read correctly)
LB 70% 81% 92% No
11 ~ 0.22
Comparison 73% 78% 5
Reading
Comprehension
(of 5)
LB 1.6 3.2 2 Yes
1.6 None -0.03
Comparison 1.5 3.3 1.8
% Readers with
Comprehensionc
LB 9% 35% N/A N/A
26 None -0.13
Comparison 7% 39% 32
Significant difference between LB and control groups at ~p<.1, *p<.05 a Neither baseline nor endline scores exhibited any statistically significant difference between LB and comparison
students b Widely cited statistician Jacob Cohen describes effect sizes of .2 as small, .5 as medium, and .8 as large. Cohen, J.:
Statistical Power Analysis for the Behavioral Sciences. (2nd ed.) 1988. c See footnote 12.
At first glance, it appears that Literacy Boost impact was limited or absent altogether. While
benchmarks were met for fluency and reading comprehension and just barely unmet for
accuracy and lower-order skills, little evidence exists here to attribute this student learning to
the Literacy Boost intervention. No statistically significant differences exist between Literacy
Boost and comparison groups either at baseline or at endline. When comparing the gain scores
between the two groups, the difference is statistically insignificant and either very small in effect
size (letters, MUW, pseudo words) or negative in effect size11 (reading comprehension, and
11
The effect size is a measure of the magnitude of an observed difference, expressed in standard deviations in order to compare across different types of measures. So, as Literacy Boost exhibited a statistically significant effect size of 0.46 in RNGPS for fluency, this means that Literacy Boost children improved almost a half standard deviation more than comparison children improved.
24
proportion of children reading with comprehension12), meaning that the comparison children
gained more over time than the Literacy Boost children in these skills. Only the proportion of
readers (children reading at least five words correct in the first 30 seconds of reading a grade-
level text), fluency, and accuracy scores exhibit positive but still relatively small effect sizes. And
of these three, only accuracy is marginally insignificant, while the other two are completely
statistically insignificant.
However, this initial glance of the overall impact may obscure the complete story on a number
of levels. First, impact may have been different in different types of schools. Second, it may be
important to control for significant differences between Literacy Boost and comparison
samples, such as HLE as noted in the previous section. And finally, impact may have differed
between children with high and low baseline scores.
First, a preliminary glance at disaggregation by school type is warranted with respect to
statistical significance and effect size. Table 7 displays the results of just such an analysis. Impact
appears nonexistent among GPS children. It is difficult to hypothesize an explanation for this
finding without more information. It is true that for some reason Literacy Boost GPS schools
experienced both a higher rate of attrition and an association of attrition with poor
performance in more reading skills than comparison GPS schools. However, the baseline scores
of non-attriting GPS Literacy Boost children are statistically equal to the baseline scores of
nonattriting GPS comparison children. Another potential explanation is that children’s ‘gains’ in
book-borrowing from school, reading to others in the community, seeing household members
read, and having household members read to them, are statistically equal between comparison
children and Literacy Boost children. However, this is true for all three types of schools, not
only GPS. SC Bangladesh staff note that GPS schools tend to have much higher class sizes, more
teacher shortages, and inadequate classroom facilities in early grades. This factor could
potentially mute the impact of Literacy Boost in GPS versus other types of schools. However,
Literacy Boost has exhibited impact even with large class sizes in other countries.13
A final hypothesis is that the Literacy Boost program may be most effective for contexts in
which schools are performing worse on average than Bangladeshi GPS schools in Meherpur –
that is, the GPS system, although not sufficient to help most children learn to read in the early
grades, may be ‘sufficient’ to wash out the positive impact of Literacy Boost seen in other more
severe contexts. Baseline data collected from grade 2 children also in March-April 2013
12
To calculate the definition of ‘readers with comprehension,’ the Literacy Boost research team standard definition
of at least 75% of reading comprehension questions correct (in this case, at least 4 of 5 questions correct) and
either at least one standard deviation of fluency lower than the average for children who scored 4 of 5
comprehension questions correct or at least one standard deviation of recognition lower than the average for
children who scored 4 of 5 comprehension questions correct, based on baseline scores. Thus, to meet this
threshold children must score at least 4 of 5 comprehension question correct and read either at least 33.2 words
per minute correct or with 96% recognition. 13
See Dowd and Mabeti, 2011.
25
indicates that children in GPS schools often outperform their peers in RNGPS and Madrassas.
Whatever the reason, the explanation for lack of Literacy Boost impact among
GPS children will be an important research question for the upcoming qualitative
study of Literacy Boost in Meherpur.
The other side of the story is Literacy Boost impact in RNGPS and Madrasas. At baseline,
compared with GPS schools, RNGPS had consistently slightly lower reading scores and
Madrasas consistently had much lower reading scores across sub-tests. Table 7 indicates that
impact in RNGPS was not only consistently positive (with the exception of proportion of
readers with comprehension) but attained moderate levels of effect size with statistical
significance or only marginal insignificance. In Madrasas, little evidence for impact in the lower-
order reading skills exists, but effect sizes are moderate in the higher-order skills of connected
text fluency and accuracy. It may be that, due to the much smaller sample size, these effect sizes
are rendered statistically insignificant but would otherwise be significant given a larger sample
size of Madrasa students. As will be seen, multilevel regression analysis supports this
explanation.
Table 7: LB – Comparison Gain Score Difference Effect Size by School Type
GPS RNGPS Madrasa
Letter Identification -0.23 0.34* 0.05
Most Used Words -0.2 0.26~ 0
Pseudo Words -0.2 0.26~ 0.16
% Readers 0.07 0.32 0.06
Fluency 0.01 0.46* 0.47
Accuracy -0.05 0.41* 0.42
Reading Comprehension -0.28~ 0.16 0.17
% Readers with Comprehension -0.15 -0.17 0.17
Significant difference between LB and control groups at ~p<.1, *p<.05
Effect sizes calculated through Stata’s listcoef command after multilevel regression controlling for a variety of
factors (see appendices E, F, & G). The effect size reported here measures the size of the difference in children’s
scores over time between Literacy Boost and comparison samples.
Appendices C-F present the results of multilevel regression analysis of the impact of Literacy
Boost on gain scores, controlling for a range of variables thought to be unaffected by the
Literacy Boost intervention and clustering standard errors to account for the clustering of
children within schools. As Table 8 indicates, the impact for GPS is neither practically nor
statistically significant.14 However, while many results for RNGPS are statistically insignificant
14
Statistical significance denotes confidence with which one can generalize the finding from a sample (in this case, a sample of students) to the broader population (in this case, the entire cohort of 4
th graders in the 32 schools).
Practical significance denotes an intuitive sense that the difference between Literacy Boost and comparison children was ‘enough,’ for example if Literacy Boost children leaned 13.3 words more than comparison children learned, this is a large difference and practically significant, while if Literacy Boost children learned only 1.4 letters
26
when controlling for other variables, the effect sizes on letters and fluency remain significant .
Of these two, only the impact on fluency is practically significant – a moderate effect size of
0.42 standard deviations translates into an additional 13 words per minute in fluency
attributable to Literacy Boost. The previous results for Madrasas are also robust – moderate
effect sizes for connected text fluency and accuracy translate to an additional 18 words per
minute and an additional 15 percent of the text correctly read, attributable to Literacy Boost.
Table 8: Literacy Boost Coefficient Magnitude,15 Statistical Significance, and Effect Size from
Multilevel Regression Model, by School Type
GPS RNGPS Madrasa
Mag. Sig. ES Mag. Sig. ES Mag. Sig. ES
Letters -0.02
-0.01 1.4 * 0.29 -0.5
-0.07
Most Used Words 0.1
0.04 0.7
0.17 0.7
0.1
Pseudo Words 0.3
0.07 1.4
0.29 0.2
0.03
Reader (% of sample) -0.7%a
-0.02 7.2%a
0.21 6.4%a
0.12
Fluency (words per
minute correct) -1.2
-0.1 13.3 ~ 0.42 18.3 * 0.62
Accuracy (% of text
correct) -1.5%a
-0.07 7%a
0.26 14.9%a ~ 0.38
Reading Comprehension 0
0 0.1
0.09 0
-0.01
RWC (% of sample) 5.7%a
0.11 -10.6%a
-0.21 3.1%a
0.09
Significant difference between LB and control groups at ~p<.1, *p<.05 a Percentage point difference between LB and comparison groups
Table 9 below displays the results of similar regression analysis run for all schools, but looking
only at those children with low reading scores at baseline. The magnitude of difference in gain
scores, level of statistical significance, and effect size in standard deviations is given for both the
baseline nonreaders (those who could not read at least five words of the oral reading passage
correctly in 30 seconds) and for children scoring in the lowest two quintiles of baseline most-
used word scores.16 It appears that the Literacy Boost program had the most impact on
children who were weaker readers, including nonreaders, at baseline in terms of pseudo words,
more than comparison children, this is a small difference and thus practically insignificant, even if it is statistically significant. 15
The Literacy Boost coefficient represents the predicted difference in scores between LB and comparison children, controlling for the variables enumerated in Appendices C-F. Thus, for RNGPS the magnitude of the coefficient on fluency is 13.3, meaning that controlling for these variables, Literacy Boost children are predicted to have increased their reading speed by 13.3 words per minute more than the increase by comparison children. 16
Analysis of correlations between all sub-test scores revealed that most-used words was the most highly correlated sub-test with all others. The lowest two quintiles of baseline most-used word scores include all nonreaders but one, plus an additional 134 children.
27
proportion of children able to meet the definition of ‘reader,’ fluency, accuracy, and reading
comprehension. Significant effect sizes range from a small 0.22 to a moderate 0.51.
Table 9: Literacy Boost Coefficient Magnitude,17 Statistical Significance, and
Effect Size from Multilevel Regression Model for Low-Scoring Children at
Baseline
Nonreaders
(16% of sample)
Lowest Two Quintiles of Baseline
MUW Scores (45% of sample)
Sub-Test Mag. Sig. ES Mag. Sig. ES
Letters 0.3 0.04 0.7 .13
Most Used Words 1.2 0.24 0.8 0.21
Pseudo Words 1.9 ~ 0.36 1.1 ~ 0.22
Reader (% of
sample)
24.7%a * 0.47 18%a * 0.37
Fluency (words per
minute correct)
5.9 0.21 13.8 ** 0.45
Accuracy (% of text
correct)
20%a * 0.51 15.6%a ** 0.45
Reading
Comprehension
0.6 ~ 0.39 0.4 * 0.27
RWC (% of sample) 0.5%a .02 -2.9%a -0.07 Significant difference between LB and control groups at ~p<.1, *p<.05, **p<.01
a Percentage point difference between LB and comparison groups Overall, it appears that while Literacy Boost only modestly impacted children
overall in RNGPS and Madrasas in terms of fluency and accuracy, the Literacy
Boost program significantly helped children who struggled at baseline improve
their decoding skills, make the jump to being able to read connected text
independently, and establish a degree of fluency, accuracy, and even reading
comprehension.
Nevertheless, more work remains to be done. Given that RNGPS and Madrasas
were behind GPS in all skills, it will be important to better understand why Literacy
Boost did not significantly help children with their lower-order skills in order to
adjust the program to better ensure that children reach grade 4 having fully
mastered these skills. It will also be necessary to understand why Literacy Boost
was unable to raise the reading scores of all children, not just primarily those who
struggled at baseline. This will be very important in order to make Literacy Boost a
17
The Literacy Boost coefficient is an expression of the predicted difference in scores between Literacy Boost and comparison children, controlling for the variables enumerated in Appendices C-F. Thus, for nonreaders the magnitude of the coefficient on accuracy is 20%, meaning that controlling for these variables, Literacy Boost children are predicted to have increased the percentage of text read correctly by 20 percentage points more than the increase by comparison children.
28
more effective program for helping all Bangladeshi children learn to read with
comprehension in the early grades of primary school.
Potential explanations for these findings that warrant investigation include the fact that children
were assessed in grades 3 and 4 – perhaps when teachers and curriculum have largely turned
away from lower-order skills. In addition, the fact that comparison schools enjoyed the Save the
Children-sponsored Reading Buddy and Mentoring program, as well as the Reading for Children
pre-primary program, may have undermined the suitability of the 14 comparison schools as
valid controls for impact analysis.
Equity Impact Analysis
In addition to standard impact evaluation, an equity impact analysis was performed. This analysis
looked at Literacy Boost impact specifically with respect to traditionally disadvantaged groups
such as girls, the poorest of the poor, the HLE-deprived, and those children with large
chore/workloads. To conduct this analysis, endline data was used to construct indices to place
children into quintiles of socio-economic status (SES) and chore/work load, and baseline data
was used to construct an index to place children into quintiles of HLE.18
No noteworthy results were found for girls, the poorest of the poor, the HLE-deprived, or
those children with large chore/workloads. As the poor, the HLE-deprived, and those
with large chore/workloads exhibit lower endline reading skill performance than
their peers, more work remains to be done to ensure that these groups not only
benefit from Literacy Boost but actually close the gap with their better-off peers.
Individual Skill Analysis
Letter Identification
This section will analyze each of the eight outcomes one by one, highlighting both progress and
need for additional improvement. First, in terms of letter knowledge, children could name
about 45 of the 50 Bangla letters at baseline, and made nearly no additional progress by endline.
The benchmark of 48 letters remains unmet, although by grade 4 the benchmark should
arguably be all 50 letters. To ensure children learn all of their letters as early as
possible, moving forward Literacy Boost should help teachers and community
action facilitators focus on the most difficult letters (t, F, H, s, r, p, E, O, o,
T, and X) as well as give teachers the tools to conduct effective formative
18
Endline data was used for SES and chore/workload indices as it is assumed to be more accurate as the children have matured since baseline. Baseline data was used for the HLE index as it is assumed that the Literacy Boost intervention may change the HLE over time.
29
assessment in order to better target the specific deficiencies of the children in their
classroom.
No statistically significant differences between LB and comparison groups.
Most Used Words
At baseline, children could read only about 14 of the 20 most used words from their grade 3
level textbook. By endline, children had increased their ability to read this group of common
words by three words, still short of the benchmark of 18 words read correctly. Again, as these
are the 20 most common words in their textbook, a benchmark of all 20 words read correctly
may be more appropriate. To get children there, teachers and community action
facilitators will need to continue to find a way to integrate practice of these lower-
order skills into their instruction, even as the curriculum may de-emphasize them
for higher grade levels. Children struggled the most with the words Kivq, Avb‡›`,
¯‹z‡j, FZz, and †`q.
No statistically significant differences between LB and comparison groups.
Pseudo Words
44.4 44.5
0.1
45.0 44.7
-0.2 -10.0
0.0
10.0
20.0
30.0
40.0
50.0
Baseline Endline Gain
lett
ers
co
rre
ct o
f 5
0
Figure 8: Letter Knowledge
LB
Control
Benchmark: 48 letters
13.9
17.2
3.3
14.0
17.2
3.2
0.0
5.0
10.0
15.0
20.0
Baseline Endline Gain
mo
st u
sed
wo
rds
corr
ect
of
20
Figure 9: Most Used Words
LB
Control
Benchmark: 18 words
30
Children’s performance when asked to decode pseudo words in Bangla was weaker than their
ability to read single real words, indicating that the children had memorized the shape of those
familiar words rather than knowing how to spell them out and break them down into syllables
and phonemes.. This is not surprising given their imperfect letter knowledge. At baseline,
children could decode about 9 or 10 of the pseudo words on average, and by endline had the
ability to decode an additional four words. However, this did not meet the benchmark of 14
words. Again, children will need more explicit instruction, modeling, and practice
with letters, phonological awareness, and decoding. Children struggled the most
with KziK, Zzbb, jwK‡Kv, gbvq, and we`c.
No statistically significant differences between LB and comparison groups.
Readers
At baseline, about 84% of children could read at least five words correctly of a grade-level
passage in 30 seconds. These students were designated as ‘readers’ and allowed to continue
reading for as long as it took them to finish the passage. Those who could not meet this
threshold were designated as ‘non-readers’ and were stopped and read the passage by the
assessors. Nothing much changed between baseline and endline with regard to this measure. In
fact, in the comparison group so many children who had met the threshold at baseline failed to
do so at endline that the proportion of readers within the comparison group actually declined.19
This indicates that there is a substantial subset of children who have not mastered
their basic skills and may need remedial support in order to keep pace with their
grade 4 peers.
19
This could be due to assessor error. According to these children’s baseline fluency scores, most of them should have been classified as nonreaders. Some assessors may also have committed errors at endline, but the fact that this error was able to override any gain in proportion of readers is noteworthy.
9.8
13.7
3.9
9.3
13.0
3.7
0
2
4
6
8
10
12
14
16
18
20
Baseline Endline Gain
pse
ud
o w
ord
s co
rre
ct o
f 2
0
Figure 10: Pseudo Words
LB
Control
Benchmark: 14 words
31
No statistically significant differences between LB and comparison groups.
Fluency
Fluency is measured by counting the number of words students read correctly in the first 30
seconds of reading the grade level passage and multiplying it by two. This is one measure where
students experienced large gains. Children started with an average of about 28-29 words
correct per minute, but just about doubled their speed between baseline and endline to surpass
the benchmark of 42 words per minute correct. As fluency is a critical ingredient to
becoming a good reader, this is an excellent achievement. However, children who
have not yet achieved the benchmark of 42 words per minute correct, and
especially those who have not met the threshold for ‘reader,’ deserve continued
focused attention in order to bring them up to speed with their peers.
83% 88%
4%
85% 83%
-2% -10%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Baseline Endline Gainpro
po
rtio
n o
f st
ud
en
ts r
ead
ing
at
leas
t 5
wo
rds
corr
ect
in 3
0 s
eco
nd
s
Figure 11: Readers
LB
Control
32
Includes all children in the sample, nonreaders assigned their wpmc score based on their 30 second progress. No
statistically significant differences between LB and comparison groups.
Accuracy
Accuracy is a skill related to fluency, and measures the overall number of words a student reads
correctly in a given passage, regardless of the time it takes him or her to finish the passage. At
baseline children were able to read about 71% of the passage. Between baseline and endline,
this is the one skill that approached a, but did not meet the criteria of a statistically significant
improvement between Literacy Boost and comparison children overall. The gain score of the
Literacy Boost students was more than double the gain score of comparison students.
Nevertheless, the benchmark of 92% of the passage read correctly was not met.
Significant difference between LB and control groups at ~p<.1
Includes all children in the sample; nonreaders assigned the percentage of text they had completed correctly when
stopped.
28.3
56.7
28.4 29.1
51.6
22.5
0.0
10.0
20.0
30.0
40.0
50.0
60.0
Baseline Endline Gain
wo
rds
pe
r m
inu
te c
orr
ect
Figure 12: Fluency
LB
Control
Benchmark: 42 WPMC
70%
81%
11%
73% 78%
5%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Baseline Endline Gain
% o
f co
nn
ect
ed
te
xt c
orr
ect
ly r
ead
Figure 13: Accuracy
LB
Control
Benchmark: 92% correct
~
33
Figure 14 below suggests one major reason for the failure to achieve this benchmark is the
substantial amount of children unable to read connected text at all. That is, the zero or very
low accuracy scores of these children pull down the overall average. These children must be
supported, perhaps through engaging remedial community activities or through
partnering them with some of the strongest readers. In general, the formative
assessment component of teacher training should be emphasized to help teachers
identify children at risk of falling behind early and to diagnose how to help these
children keep up with the class. Teacher training should also give teachers practical
suggestions for instructing classes with varied levels of students.
Figure 14: Frequency of Accuracy Scores
Reading Comprehension
After reading the grade-level passage, children were asked a set of five comprehension
questions. At baseline, children could answer only about 1.5 of these questions correct on
average. By endline, children had improved their average number of correct comprehension
questions for an average of about three questions correct, surpassing the benchmark of two
questions correct. This progress should be built upon, and teachers, community
action facilitators, and family members need to focus not only on whether children
can read, but if they can engage with text not only to understand the literal
meaning but to predict, synthesize information, make inferences, and evaluate.
01
02
03
04
05
0
Pe
rcen
t
0 .2 .4 .6 .8 1Accuracy w/ nonreader scores
34
Includes all children in the sample; nonreaders coded as 0. No statistically significant differences between LB and
comparison groups.
One Additional Way to Look at the Data: Readers with
Comprehension
Students reading with comprehension, with understanding, is the ultimate goal of Literacy
Boost. As such, a new composite measure to focus attention on this goal as well as to track
progress in terms of equity, in terms of all children reading with comprehension, is displayed
below in Figure 16. ‘Reading with comprehension’ is defined as reading a grade-level passage
with such skilled comprehension that the child correctly answers at least 75% of
comprehension questions, or in the case of five questions, 80% of comprehension questions. To
ensure that children who correctly answer this number of questions have done so as a result of
sufficiently skilled reading (as opposed to guessing), the threshold of ‘reading with
comprehension’ is not only defined as the reading with at least 75-80% comprehension in
addition to scoring either at least one standard deviation below the average fluency
corresponding to 75-80% comprehension or at least one standard deviation below the average
accuracy corresponding to 75-80% comprehension.
Figure 16 shows that very few children qualified as ‘readers with comprehension’ at baseline.
However, by endline there was a large increase in readers with comprehension, such that about
37% of children in the sample meet this definition. While this is a massive improvement
over the baseline, more grade 4 children should be able to read a grade 3 level text
with at least 80% comprehension. Reading comprehension in general was an area
that did not demonstrate any Literacy Boost impact. Thus, it will be necessary to
modify the Literacy Boost program so that the large number of children who are
able to read with decent speed and accuracy also understand what they are reading
and can interact with the text on a deeper level to be truly able to read to learn.
1.4
2.8
1.5 1.3
2.8
1.5
0.0
0.5
1.0
1.5
2.0
2.5
3.0
Baseline Endline Gain
# o
f co
mp
reh
en
sio
n q
ue
stio
ns
corr
ect
ly
answ
ere
d o
f 5
Figure 15: Reading Comprehension
LB
Control
Benchmark: 2 questions
35
No statistically significant differences between LB and comparison groups.
Trends in Reading Skill Data from Multilevel Regression Models
This section examines results from a series of regressions with clustered standard errors to
account for the clustering of children within schools in order to investigate relationships
between reading outcomes on one hand and student background, channels for reading skill
improvement, and intensity of LB participation on the other hand. The aggregate model can be
found in Appendix C, and the models for each of the school types in Appendices D, E, and F.
First, in terms of associations with student background, age and having repeated at least one
grade were both negatively correlated with two or more reading skill gains. This means that
older children and those who repeated one or more grades did not improve their scores as
much between baseline and endline as their younger peers and those students who had never
repeated a grade. Both older children and those who have repeated may need special
remedial help in order to keep pace with other children.
LB Reading Activities and Reading Achievement
Appendix G displays regressions based on data from Literacy Boost students only. These
regression examine reading skill gains and the intensity of participation in Literacy Boost
community activities. As almost all Literacy Boost children reported participating in these
community activities, we use the continuous variables of the frequency of participation was
used for book banks (# of times borrowed per month last year), reading buddies (# of times
meet per week), and reading camps (# of times attended reading camps per month last year).
Whether students created make-and-take materials at reading camps ‘sometimes’ or ‘always’ is
also examined.
9%
35%
26%
7%
39%
32%
0%
5%
10%
15%
20%
25%
30%
35%
40%
45%
Baseline Endline Gain
% o
f st
ud
en
ts m
ee
tin
g cr
ite
ria
for
're
adin
g w
ith
co
mp
reh
en
siu
on
'
Figure 16: Readers with Comprehension
LB
Control
36
Based on univariate multilevel regressions of each individual skill gain on each of these
participation variables, the variable positively associated with the greatest number of skill gains
is frequency of meeting with reading buddy (all skills except reading comprehension). Frequency
of reading camp attendance was associated with gains in becoming a reader, fluency, accuracy,
and becoming a reader with comprehension. Make and take frequency was linked to most used
words, pseudo words, and accuracy gains. Book bank borrowing frequency was linked to
fluency and reading with comprehension gains. Given the strong association of frequency
of reading buddy participation with reading skill gains, the fact that comparison
schools also had a reading buddy and mentoring system may explain the limited
impact of Literacy Boost.
The abovementioned four variables were also combined into an index of participation in
Literacy Boost community activities by adding the first three (book bank, reading buddies, and
reading camp participation), with reading camp participation weighted by 1.5 for make-and-take
‘sometimes’ and by 2 for make-and-take ‘always.’ This index was then converted into quintiles
of participation, and those quintiles used in a multivariate model with the standard set of
control variables (Annex F). Figure 17 below displays some of the results: children in the lowest
quintile of participation were predicted to gain nothing or less than their peers who fell in
higher levels of participation. This is evidence that Literacy Boost community activities
may have aided children’s reading skill acquisition, and the association shown in
Figure 17 also holds for becoming a reader and fluency.
Classroom and Community Channels for Reading Skill Improvement
Finally, a series of multilevel regressions were run on the entire sample of children. These
regressions investigated possible ‘channels’ for reading skill improvement – channels such as
0
5
10
15
20
1 2 3 2 5
Gai
n S
core
Participation Quintile
Figure 17: MUW and Accuracy Gains by LB
Community Activity Participation Quintiles
MUW
Accuracy
37
changes in children’s literacy habits and HLE. It should be noted that there was no association
between Literacy Boost and these potential channels, as shown in Table 10. All five changes in
child literacy habits and HLE favor the comparison group, and four of the five differences are
statistically significant or marginally insignificant. It appears that for some reason
comparison children were improving their literacy habits and experiencing positive
changes in their HLE at the same or higher rate than Literacy Boost students, and
this will be important to investigate during the qualitative study of Literacy Boost.
Table 10: Changes in Child Literacy Habits and HLE by Sample
Group
Potential Channel LB Control Significance
Book borrow gain -40% -21% ***
Read to others gain 24% 36% *
% See read gain 23% 30% ~
% Read to gain 18% 23%
N types of reading materials
gain 10% 28% ~
Because few or no potential channels for literacy skill acquisition were correlated with the
Literacy Boost intervention, the regressions in Appendix H examine the association between
these channels for children’s literacy skill improvement in general for the sample as a whole.
Only a handful of variables were robustly associated with at least two reading skill gain scores.
Figure 18 illustrates one of these variables – teachers’ possession of an updated balanced
scorecard or other type of student tracking sheet and the number of types of teaching
techniques observed in the language arts lesson (techniques such as call on students, lecture,
games, song, role plays, etc.). Teachers with updated tracking sheets and those observed to use
more types of techniques on the day of observation had children who on average exhibited
higher gains in pseudo words and fluency. This relationship was also present with tracking sheet
and most used words, accuracy, and reading comprehension. In addition, the estimated
percentage of children who were encouraged to participate and children’s reported gain in the
percentage of household members seen reading over time were positively associated with
higher reading comprehension gains. These associations, while correlational rather than
causal, can nevertheless be used to guide the second year of Literacy Boost
intervention as factors that should be emphasized to the extent possible with
teachers and community members.
38
Conclusion
This report examined how children’s skills changed between February-March 2012 to March-
April 2013 for 465 grade 4 students. As the original sample comprised 623 grade 3 students,
attrition was 25%, with similar rates between Literacy Boost and comparison students overall
but with much higher attrition among LB students than comparison students in GPS. In both
sample groups, the oldest students and those with lower reading accuracy scores were the
most likely to be absent on the day of endline assessment. Thus, older children and those
struggling to learn to read connected text may be more at-risk of not attending or
dropping out of school, and thus may need more special support to continue their
education than their younger and better-reading peers.
The sample of remaining students is statistically equivalent on almost all observed background,
school environment, and home literacy environment characteristics. Among all children,
diversity of print in the household remains limited at endline, although the number of
household members engaging in literacy-promoting activities increased substantially between
baseline and endline. However, more room for improvement remains: as fewer than
half of household members told stories to children in the past week, families should
be encouraged to more frequently engage in this practice as it does not require
literacy but can help boost children’s reading skills.
Analysis of the impact of the Literacy Boost intervention reveals that overall, the Literacy
Boost only modestly helped children improve their fluency in RNGPS and their
fluency and accuracy in Madrasas. However, Literacy Boost significantly helped the
subset of children who struggled at baseline improve their decoding skills, make
the jump to being able to read connected text independently, and establish a
degree of fluency, accuracy, and even reading comprehension. Nevertheless, it will be
0
1
2
3
4
5
6
7
8
1 3 5
Gai
n in
nu
mb
er
of
pse
ud
o w
ord
s co
rre
ct
Number of types of teaching techniques observed in lesson
Figure 18: Pseudo Word Gain by Updated Tracking Sheet and
# of Types of Teaching Techniques Observed Used in Lesson
No updated tracksheet
Updated tracksheet
39
important to better understand why Literacy Boost did not significantly help children master
their letters and most-used words and why Literacy Boost was unable to raise the reading
scores of all children (not only primarily those who struggled at baseline). Although there are
many possible explanations for this finding, one of the most likely important factors may be
similar out of school experiences between Literacy Boost and comparison students. The change
in many aspects of children’s home literacy environment and community literacy practices was
similar between the two groups, and the comparison group of children also enjoyed SC’s RMB
program, participation in which was strongly correlated with reading skills gains.
Other evidence suggests that Literacy Boost community activities may be helping children
improve their reading. Frequency of borrowing books from Book Banks as well as participation
in RBM, reading camps, and make and take activities during reading camps was linked to
stronger gains in a variety of reading skills between baseline and endline. However, many of the
benchmarks set at baseline for children to achieve have not yet been realized. Only the fluency
and reading comprehension benchmarks were met, and children still exhibit gaps in their lower-
order skills of letter knowledge and most-used and pseudo words. A divergence in skill
development between readers and nonreaders also exists for higher-order skills like fluency
and accuracy. Furthermore, equity analysis reveals that students from the poorest households,
those from households with the fewest reading materials and lowest prevalence of reading
habits, and those with large chore/workloads exhibit lower still perform worse in their reading
skills at endline than their better-off peers. And finally, multilevel regression analysis indicates
that older children and those who repeated one or more grades did not improve their scores
as much between baseline and endline as their younger peers and those students who had
never repeated a grade. Given this information, Literacy Boost should focus on:
closing lower-order skill gaps.
supporting the subset of children who are still falling behind, including older
children and those who have repeated a grade. This could be done through
engaging remedial community activities or through partnering them with
some of the strongest readers.
finding a way to better support the poorest students, the most HLE deprived
students, and those with large chore/workloads. It will be essential to ensure
that these children have access to both in-school and out-of-school
opportunities to practice their reading skills.
the formative assessment component of teacher training, to help teachers
identify children at risk of falling behind early and to diagnose how to help
these children keep up with the class.
practical suggestions for teachers on how to more effectively manage and
instruct classes with varied levels of students.
40
Appendix A. Literacy Boost Activities prior to Endline Assessment
Date of
Intervention
/ Start
Intervention Frequency /
Number of
Interventions
I. LB Teacher Training
20-30 May,
2011
Overall LB orientation to Bangladesh Sponsorship Basic
Education team.
Once
04-08 April,
2012
First Teacher Training:
78 (51 females & 27 males) grade one-three language teachers
and their four supervisors attended CBSDP training venue in
three separate clusters.
Session: Introduction to Development and Instruction for
Children
Once
22-24 May,
2012
Second Teacher Training:
81 (51 females & 30 males) grade one-three language teachers
and their four supervisors attended CBSDP training venue in
three separate clusters.
Session: Formative Assessment
Once
19-21 June,
2012
Third Teacher Training:
80 (51 females & 29 males) grade one-three language teachers
and their four supervisors attended CBSDP training venue in
three separate clusters.
Session: Letter Knowledge
Once
17-19 July,
2012
Fourth Teacher Training:
76 (48 females & 28 males) grade one-three language teachers
and their four supervisors attended CBSDP training venue in
three separate clusters.
Session: Phonemic Awareness
Once
28-30 August,
2012
Fifth Teacher Training:
81 (52 females & 29 males) grade one-three language teachers
and their four supervisors attended CBSDP training venue in
three separate clusters.
Session: Vocabulary
Once
27 September-
01 October,
2012
Sixth Teacher Training:
78 (49 females & 29 males) grade one-three language teachers
and their four supervisors attended CBSDP training venue in
three separate clusters.
Session: Reading Fluency and Accuracy
Once
12-15
November,
2012
Seventh and final Teacher Training:
72 (44 females & 28 males) grade one-three language teachers
and their four supervisors attended CBSDP training venue in
three separate clusters.
Session: Reading Comprehension and Conclusion
Once
41
May, 2012 - Development of LB classroom observation checklist
- Encourage trained teachers to establish Reading corner and
print rich environment
May to onward
Continuous
Date of
Intervention /
Start
Intervention Frequency / Number
of Interventions
II. Community Action
1. Enhancing the Literacy Environment
29-November,
2011
Three days Basic training for 20 Field Officers, 26 Lead
Volunteers and 115 Reading Camp Volunteers.
Content covered Reading Camp Session, Use LB Flipbook for
Community awareness, Overall Community Action operational
modality
Detail execution
frequency:
Volunteers training Information 2012.pdf
27-29
February,
2012
15 Reading Camp Sessions Development Once
July-Aug 2012 LB materials creation / development / procurement 2 times
Jan, 2012
Sep 2012
Book banks filled with materials distributed Twice
From March
to December,
2012
A total of 2697 children’s parents/caregivers attended Literacy
Awareness sessions facilitated by 26 Lead volunteers in 115
Reading Camp session
8 times round the
year and disseminated
16 strategies
2. Community Reading Activities
From January
2012 to date
115 Reading camps conducted by 1 Reading Camp Volunteer
and supervised by 24 Lead Volunteers regularly and 20 Field
Officers occasionally
Activities covered: Open play, Song, Story including Reading for
Children, Activity based learning, Make and Take, Journaling
(Drawing, Writing), Book borrowing through Reading Buddy
and Mentoring
every Thursday/ Friday
afternoon for 100
minutes
From January-
December,
2012
A total of 2697 children from grade one to three have benefited
from the reading camp sessions through weekly reading camp
sessions since its inception
Round the year (22
solid Reading Camp
sessions but actual
held 40 sessions)
From March
to December,
2012
A total of 2697 children attended Literacy Awareness sessions
facilitated by 26 Lead volunteers in 115 Reading Camp session
7 times round the
year and disseminated
14 strategies
Initially Jan-
Dec 2012
Thereafter
from Jan, 2013
Reading buddies
1300 pairs formed across 115 Reading Camp and borrowed new
books from weekly Reading Camp sessions
Ongoing
42
to date
From Feb
2012 to date
Story time
Weekly, the elderly members of the community are invited to
tell traditional oral stories to children at the Reading Camps.
every Thursday/ Friday
afternoon
From Feb
2012 to date
Children read books from Supplementary Reading materials and
Primers and tell oral stories to fellow children in the same
language and experience on dialogic reading.
From their homes, in
school and reading
camps
December
2012
Community Read-A-Thon/ Reading Festival celebrated in 18
school points, Literacy fair and reading competition were
organized involving 2088 children, 2263 parents and 408
community stakeholders including School Management
Committee, Community Core Groups, School Teachers and
Community elites.
4 times in different
locations
3. Raising Reading Awareness
5-8 June, 2012 Bangladesh Literacy Boost experience shared in Regional Forum
on ‘Partnering for Quality Education and Learning Outcomes’,
Bangkok
Once
16 July, 2012 Literacy Boost baseline findings sharing and orientation
workshop with Schools, ministry counterparts and community
Once
December,
2011
Technical team presented 2011 Literacy Boost baseline to the
Ministry of Primary and Mass Education for getting LB
implementation permission.
Once
March and
October 2012
Orientation workshops on Literacy Boost Program with EC
delegates, Save the Children Education Sector colleagues
Twice
September
2012
624 SMC members oriented on Literacy Boost activities Once
December
2012
Published Reading Camp Bulletin participated by 2088 children,
26 Lead volunteers and 115 Reading Camp Volunteers
Once and
disseminated all
working schools and
Reading Camps
43
Appendix B: Findings from pilot teacher observation and interview
A pilot classroom observation and teacher interview tool was employed as a preliminary
attempt to balance data collected on out-of-school factors hypothesized to influence children’s
literacy acquisition and to better understand the extent to which in-school factors explain
children’s learning. For this exercise, grade 4 Bangla language arts teachers were observed and
interviewed by the leaders of the assessment teams. As the Literacy Boost intervention targets
grades 1-3, grade 4 teachers were sometimes, but not always, targeted for Literacy Boost
teacher training due to the fact that Bangladeshi teachers are often responsible for more than
one grade. Despite the imperfect alignment of grade 4 teachers Literacy Boost teacher training,
grade 4 teachers were chosen to be observed and interviewed due to the assumption that it
would be easier to link them to the children being assessed than grade 3 teachers, due to
teacher turnover and transfer. Thus, as no reliability analysis was performed, the tool needs
further refinement, and observed teachers were not necessarily those who had received
Literacy Boost training, the following results should be considered a preliminary attempt to
explain the relationship between teacher practices and children’s reading outcomes.
Figure 19 and Tables 11-14 present the observation and interview findings. Although the
teachers may not have received Literacy Boost teacher training, it may be useful to note the
current school environment that these children had been exposed to for the two months prior
to the endline assessment, and to find correlations between aspects of that environment and
children’s reading scores. In terms of comparing between LB and comparison classrooms, due
to the very small sample size of 31 schools (data for one school was erroneously collected and
could not be used), practically all differences are statistically insignificant. However, it may
instead be useful to compare the simple average differences in practical terms (does the
difference ‘seem’ large), especially because nearly the entire population of grade 4 language arts
classrooms from LB and comparison schools was observed.
The grade 4 Bangla language arts classrooms were observed to have high levels of presence of
books and posters. Reading charts and student work was present in about half of observed
classrooms, and songs, newspapers, and learning corners were rare. The only noteworthy
difference between LB and comparison schools appears to be in student work, where LB
classrooms have slightly more student work displayed than comparison classrooms. Because of
this, LB schools had slightly higher print-rich language arts classrooms than comparison schools.
Although grade 4 classrooms were not targeted by the Literacy Boost intervention,
teachers from grades above grade 3 should be encouraged either directly or
indirectly through their trained grade 1-3 peers to enrich their classroom print
environment so that any gains from Literacy Boost in grades 1-3 are sustained
throughout primary school.
44
Teachers’ Bangla language arts classes were observed for at least one half-hour on the day of
assessment for the 31 of 32 schools. Tables 11-14 present the findings from the classroom
observation, contrasted with the results of a short teacher interview in the case of Tables 11
and 12. First, teachers rarely exhibited teaching of lower-order skills such as letters, syllables,
sounds, and vocabulary. This is to be expected in grade 4, although a small amount of LB
teachers were observed to be covering this topic.
A high percentage of teachers were observed to include reading of connected text in their
lesson on the day of assessment. No major differences between LB and comparison teachers
was observed in terms of content. Interestingly, when teachers were interviewed about the
different skills they usually covered in their lessons many more of them mentioned lower-order
skills. The lack of either observed or reported coverage of letters is concerning, as
children have not fully mastered their letters. Teachers may also be over-reporting
the amount of time and attention they give to other lower-order skills. Thus,
although Literacy Boost did not previously target grade 4 teachers, as will be seen a
significant proportion of students are entering grade 4 without yet having mastered
their lower-order skills and will need continuous help, both from teachers and
community activities, to finish primary school learning to read with
comprehension.
94%
56%
11%
78%
17%
56%
22%
92%
62%
8%
85%
8%
46%
23%
0%10%20%30%40%50%60%70%80%90%
100%
% o
f cl
assr
oo
ms
wit
h m
ate
rial
Figure 19: Type of Print/Materials in Grade 4 Classroom by Sample
Group
LB
Control
45
Table 11: Reading Skill Content Covered via Observation and Self-Reporting
Literacy Boost Comparison Overall
Content covered Observed
Self-
reported Observed
Self-
reported Observed
Self-
reported
Letters 11% 6% 0% 15% 5% 10%
Syllables 22% 56% 15% 62% 20% 59%
Sounds 17% 33% 15% 54% 14% 42%
Rhymes Not
collected 61%
Not
collected 62%
Not
collected 61%
Vocabulary 11% 44% 8% 39% 9% 42%
Reading 72%
Not
collected 77%
Not
collected 76%
Not
collected
N of different skills
taught 1.3 1.4 1.2 1.7 1.3 1.5
Table 12 appears to indicate that children in both Literacy Boost and comparison schools are
entering grade 4 classrooms where teachers use only a fraction of the most engaging teaching
techniques. Not surprisingly, calling on students, lecture, and discussion were among the most
popular teaching techniques observed and reported by teachers. Worryingly, engaging
techniques like songs, games, role plays, and use of materials like flashcards were rarely
observed. The reading of stories was not widespread. Effective strategies such as summarizing
the lesson at the end of the class were also uncommon. Future intervention may want to
consider training upper-primary teachers to integrate more engaging and effective
strategies into their reading instruction, especially since one in every six students
cannot read at least five words correctly of connected text in 30 seconds upon
reaching grade 4.
Table 12: Teaching Techniques Observed and Self-Reported
Literacy Boost Comparison Overall
Teaching Technique Observed
Self-
reported Observed
Self-
reported Observed Self-reported
Call on students 67% 56% 85% 39% 74% 48%
Lecture 67% 72% 54% 77% 61% 74%
Story time 28% 44% 46% 54% 36% 48%
Song 6% 17% 15% 15% 10% 16%
Game 0% 17% 0% 31% 0% 23%
Role play 6% 0% 0% 8% 3% 3%
Discussion 67% 83% 69% 100% 68% 90%
Flashcards 17% 22% 8% 62% 13% 39%
Lesson summary 28%
Not
collected 8%
Not
collected 19% Not collected
Total number of types of
techniques 2.8 3.1 2.8 3.8 2.8 3.4
46
Comparison teachers appeared to be more likely to use the call on students and story time
techniques, while LB teachers appeared more likely to use the lecture and lesson summary, but
both groups of teachers exhibited a similar number of techniques used overall.
Table 13 presents observation data on how the grade 4 teachers read to the students. For each
one of the techniques listed here, under 50% of all teachers were observed to use the
technique, except for the asking of evaluative questions. Inferential questions and reading in an
engaging manner were the two least prevalent reading techniques observed, and explaining new
vocab and involving a discussion of letters and/or sounds while reading were also rare. If
children are to read to learn, they will need to be challenged to practice engaging
more deeply with text and will also need teachers who can model such
engagement. Comparison teachers appear slightly more likely to ask questions during and
after reading, as well as to ask summary and evaluative questions and to have students guess
new words. LB teachers appear slightly more likely to ask questions before reading and to ask
factual questions.
Table 13: Dialogic/Active Reading Techniques Observed
Technique Literacy Boost Comparison Overall
Asked questions before reading 50% 39% 45%
Asked questions during reading 39% 62% 48%
Asked questions after reading 33% 46% 39%
Asked factual questions 56% 31% 45%
Asked summary questions 17% 54% 32%
Asked inferential question 11% 8% 10%
Asked evaluative questions 50% 62% 55%
Read in an engaging way 11% 15% 13%
Explained new vocab 22% 23% 23%
Discussed letters/sounds 22% 31% 26%
Had students guess new words 44% 54% 48%
Total number of techniques 3.6 4.2 3.8
Table 14 shows that when students read, no one type of reading dominated. Pair and group
reading, echo reading, and sequential reading were the most prevalent, with very little silent
reading and no dialogic reading between students. Comparison classrooms seem to contain
slightly more individual, pair/groups, and sequential reading, while LB classrooms seem to
involve more echo reading.
47
Table 14: Types of Student Reading Observed
Type of Reading Literacy Boost Comparison Overall
Silent 22% 15% 19%
Individual 28% 39% 32%
Pairs/groups 39% 54% 45%
Echo 56% 39% 38%
Sequential 33% 46% 39%
Dialogic 0% 0% 0%
Total number of types of reading 1.8 1.9 1.8
48
Appendix C: Mutilevel Regression Model, Overall
(1) (2) (3) (4) (5) (6) (7) (8)
VARIABLES/ Gain Scores: Letters MUW Pseudo Words Reader WPMC Accuracy
Reading
Comp RWC
LB school 0.622~ 0.570~ 0.787 0.0656~ 7.861* 0.0667* 0.106 -0.0508
(0.326) (0.298) (0.492) (0.0342) (3.301) (0.0321) (0.146) (0.0550)
Age in years -0.0358 -0.375* -0.410~ -0.0250 -2.666** -0.0194 -0.0364 0.0128
(0.192) (0.160) (0.237) (0.0160) (0.901) (0.0134) (0.0854) (0.0248)
5 quantiles of hle -0.0237 -0.147~ -0.316* -0.00343 -0.839 -0.000751 0.0579 0.0352*
(0.111) (0.0763) (0.149) (0.00972) (0.808) (0.00807) (0.0544) (0.0172)
Repeated at least one grade -0.689* -0.460 -0.437~ -0.0117 -2.139 -0.0113 -0.211~ -0.0487
(0.324) (0.283) (0.255) (0.0235) (1.706) (0.0202) (0.112) (0.0300)
Distance to Nearest Tar Road
in KM 0.00170 -0.222 -0.142 -0.0251~ -3.370* -0.0104 -0.0937 -0.0289
(0.168) (0.178) (0.202) (0.0135) (1.409) (0.0111) (0.0717) (0.0249)
# of grades at the school -0.165 -0.293~ -0.131 -0.0170~ -1.195 -0.00940 -0.126** -0.0260*
(0.178) (0.151) (0.124) (0.00960) (0.719) (0.00951) (0.0351) (0.01000)
Baseline score -0.553*** -0.517*** -0.403*** -0.491*** -0.00107 -0.358*** -0.518*** -0.578***
(0.0504) (0.0340) (0.0307) (0.0696) (0.0662) (0.0543) (0.0559) (0.0718)
Constant 25.77*** 16.04*** 13.69*** 0.727** 61.89*** 0.576** 2.862** 0.167
(3.380) (2.170) (2.824) (0.208) (13.23) (0.169) (0.989) (0.287)
Observations 448 448 448 448 448 448 448 448
R-squared 0.445 0.429 0.263 0.268 0.065 0.186 0.207 0.131
Robust standard errors in parentheses, *** p<0.001, ** p<0.01, * p<0.05, ~ p<0.1
Due to lack of statistical significance, variables included in the model but not shown here include sex, SES quintiles, ecd attendance,
chore/work quintiles, far distance to school, missing school in past week, having previously transferred schools, school electricity, and
household size
49
Appendix D: Mutilevel Regression Model, GPS
(1) (2) (3) (4) (5) (6) (7) (8)
VARIABLES Letters MUW
Pseudo
Words Reader WPMC Accuracy
Reading
Comp RWC
LB school -0.0275 0.126 0.275 -0.00704 -1.874 -0.0148 0.00317 0.0569
(0.376) (0.220) (0.935) (0.0385) (1.622) (0.0264) (0.121) (0.0431)
Female = 1 Male = 0 -0.438 0.185 0.205 0.0599 1.184 0.0564 0.638* 0.135
(0.265) (0.244) (0.446) (0.0412) (2.937) (0.0362) (0.253) (0.0935)
Age in years -0.0500 -0.269 -0.0410 -0.0302 -2.609** -0.0250 0.0325 0.0868~
(0.153) (0.255) (0.266) (0.0178) (0.857) (0.0163) (0.143) (0.0487)
Attended ECD program -0.229 -0.344 -0.139 0.0629* 3.127 0.0385 0.0295 0.0116
(0.388) (0.456) (0.657) (0.0283) (3.529) (0.0227) (0.178) (0.0598)
Repeated at least one grade -0.568 -0.239 -0.459 0.0137 0.114 0.00431 -0.219 -0.0973*
(0.358) (0.219) (0.482) (0.0311) (2.290) (0.0269) (0.181) (0.0414)
# of times transferred schools 0.804** 0.456 0.757 0.0135 1.291 0.00913 0.362 0.0608
(0.246) (0.351) (0.499) (0.0418) (3.805) (0.0356) (0.274) (0.107)
Missed school in the past week -0.626~ 0.125 0.0944 0.0329 3.053 0.0191 -0.305 -0.0631
(0.324) (0.260) (0.552) (0.0603) (3.533) (0.0449) (0.213) (0.0768)
Distance to Nearest Tar Road in KM 0.0571 -0.356*** -0.235 -0.0296~ -0.00929 -0.0174 -0.214** -0.0744***
(0.157) (0.0780) (0.165) (0.0147) (1.260) (0.0108) (0.0532) (0.0139)
School has Electricity -1.227*** -0.300 -0.241 0.123* -1.069 0.0341 -0.246~ -0.155**
(0.257) (0.404) (0.537) (0.0488) (2.574) (0.0334) (0.132) (0.0479)
# of grades at the school -0.0478 -0.105 -0.180 -0.000944 1.837* 0.00319 -0.116* -0.0487*
(0.0749) (0.0872) (0.279) (0.00841) (0.741) (0.00651) (0.0476) (0.0164)
Baseline scores -0.410*** -0.531*** -0.366*** -0.471** 0.0895 -0.377** -0.690*** -0.587***
(0.0630) (0.0568) (0.0558) (0.149) (0.0589) (0.0995) (0.0584) (0.0815)
Constant 20.77*** 15.89*** 10.89** 0.543~ 39.80*** 0.544* 2.522 -0.352
(3.241) (2.848) (3.315) (0.302) (9.419) (0.227) (1.735) (0.556)
Observations 191 191 191 191 191 191 191 191
R-squared 0.458 0.607 0.303 0.328 0.107 0.293 0.375 0.211
Robust standard errors in parentheses, *** p<0.001, ** p<0.01, * p<0.05, ~ p<0.1
Due to lack of statistical significance, variables included in the model but not shown here include SES quintiles, far distance to school,
50
number of household members, HLE quintiles, and chore/work quintiles.
Appendix E: Multilevel Regression Model, RNGPS
(9) (10) (11) (12) (13) (14) (15) (16)
VARIABLES Letters MUW
Pseudo
Words Reader WPMC Accuracy
Reading
Comp RWC
LB school 1.398* 0.655 1.407 0.0720 13.32~ 0.0704 0.135 -0.106
(0.639) (0.560) (0.927) (0.0501) (6.477) (0.0454) (0.287) (0.102)
Age in years -0.221 -0.385~ -0.612 -0.0111 -2.194 -0.00532 0.0541 -0.0292
(0.264) (0.190) (0.372) (0.0228) (1.797) (0.0164) (0.120) (0.0265)
5 quantiles of hle 0.0177 -0.199~ -0.617**
-
0.00315 -0.948 0.00599 -0.0260 0.0421
(0.124) (0.109) (0.193) (0.0169) (1.543) (0.0139) (0.0641) (0.0285)
Attended ECD program 0.622~ 0.0250 -0.107 -0.0123 1.291 -0.0316 -0.0655 -0.0215
(0.329) (0.477) (0.531) (0.0478) (4.978) (0.0384) (0.135) (0.0578)
Missed school in the past week -0.0267 -0.157 0.0338
-
0.00513 -3.525 -0.0250 0.164 0.00168
(0.618) (0.339) (0.670) (0.0599) (5.142) (0.0470) (0.192) (0.0648)
Reported school was far away -0.0981 -0.667 -0.0687 -0.0210 -2.380 -0.0209 -0.136 -0.0298
(0.563) (0.517) (0.715) (0.0397) (4.782) (0.0394) (0.205) (0.0634)
Distance to Nearest Tar Road in KM 1.060 0.455 0.656 0.104 -4.342 0.0623 -0.345 -0.0515
(0.949) (0.684) (1.228) (0.0626) (6.385) (0.0620) (0.280) (0.120)
School has Electricity -0.205 -0.322 -0.331 -0.0222 1.230 -0.0272 0.394~ 0.108
(0.557) (0.402) (0.802) (0.0484) (5.138) (0.0419) (0.208) (0.0907)
# of grades at the school -0.214~ -0.196 -0.0931 0.0161 -0.585 0.0153 0.00533 0.0153
(0.107) (0.114) (0.165) (0.0114) (0.947) (0.0100) (0.0482) (0.0174)
Baseline Scores -0.620***
-
0.492*** -0.388***
-
0.496*** 0.0534 -0.328*** -0.341**
-
0.635***
(0.0494) (0.0458) (0.0447) (0.0777) (0.0999) (0.0665) (0.0847) (0.150)
Constant 30.17*** 15.73*** 15.42** 0.588* 54.91* 0.377~ 1.276 0.457
(3.973) (2.503) (3.997) (0.242) (23.58) (0.202) (1.233) (0.350)
Observations 215 215 215 215 215 215 215 215
R-squared 0.616 0.448 0.272 0.338 0.078 0.211 0.156 0.130
51
Robust standard errors in parentheses, *** p<0.001, ** p<0.01, * p<0.05, ~ p<0.1
Due to lack of statistical significance, variables included in the model but not shown here include SES quintiles, number of household
members, sex, having ever repeated a grade, having transferred schools, and chore/work quintiles.
Appendix F: Multilevel Regression Model, Madrasa
(17) (18) (19) (20) (21) (22) (23) (24)
VARIABLES Letters MUW
Pseudo
Words Reader WPMC Accuracy
Reading
Comp RWC
LB school -0.492 0.709 0.175 0.0637 18.35* 0.149~ -0.0171 0.0313
(3.026) (1.740) (1.455) (0.0500) (5.293) (0.0622) (0.274) (0.102)
Female = 1 Male = 0 -0.844 0.0296 0.481 0.198 2.988 0.0454 -0.0559 -0.213
(1.413) (1.346) (1.941) (0.177) (9.628) (0.118) (0.650) (0.223)
Age in years 0.925 -0.368 -1.011~ 0.0224 1.072 0.0126 -0.168 -0.00418
(1.136) (1.018) (0.342) (0.109) (3.310) (0.0876) (0.337) (0.0699)
5 quantiles of hle -1.101~ -0.201 0.251 0.0197 -1.754* -0.00640 0.104 0.0250
(0.453) (0.244) (0.691) (0.0151) (0.533) (0.0149) (0.211) (0.0408)
Attended ECD program 2.850* 2.142~ 0.470 -0.223~ -7.726 -0.180 -0.376 0.169
(0.719) (0.747) (1.813) (0.0775) (7.792) (0.0885) (0.474) (0.0746)
Repeated at least one grade -2.852* -1.147 -0.567 -0.0643 -5.795 -0.0417 -0.789 -0.133
(0.734) (1.070) (1.432) (0.139) (3.765) (0.141) (0.458) (0.124)
# of times transferred schools 1.234 -0.302 0.166 0.173 -4.337 0.0253 0.842 -0.120
(1.424) (0.928) (1.356) (0.157) (8.054) (0.128) (0.675) (0.0638)
Missed school in the past week -0.421 -0.770 -0.169 -0.262 -21.16 -0.178 -0.884 -0.178~
(1.678) (1.275) (2.519) (0.181) (10.62) (0.115) (0.421) (0.0571)
Reported school was far away 3.379* 2.617 -0.244 -0.0372 12.83~ 0.0269 -0.347 -0.119
(0.625) (1.164) (2.378) (0.171) (4.497) (0.193) (0.831) (0.199)
Bangla letter knowledge -0.600~ -0.555* -0.497* -0.448* -0.173 -0.334 -0.680
(0.218) (0.103) (0.103) (0.123) (0.237) (0.176) (0.301)
Constant 17.79 9.061 17.89* -0.110 9.287 0.0743 3.220 0.165
(21.24) (10.30) (3.656) (1.141) (21.35) (0.825) (3.410) (0.566)
Observations 50 50 50 50 50 50 50 50
R-squared 0.421 0.405 0.331 0.348 0.295 0.239 0.216 0.227
Robust standard errors in parentheses, *** p<0.001, ** p<0.01, * p<0.05, ~ p<0.1
52
Due to lack of statistical significance, variables included in the model but not shown here include SES quintiles, number of household
members, and chore/work quintiles. School-level variables were not included in this model due to missing data from one Madrasa.
Appendix G: Multilevel Regression Model, Participation in LB Activities
(1) (2) (3) (4) (5) (6) (7) (8)
VARIABLES Letters MUW
Pseudo
Words Reader WPMC Accuracy Reading Comp RWC
5 quantiles of lbpart 0.0363 0.388* 0.363 0.0433* 1.903~ 0.0342* 0.00725 -0.00555
(0.189) (0.172) (0.335) (0.0183) (0.987) (0.0130) (0.0787) (0.0208)
Age in years 0.546 0.209 0.0162 -0.0276 -2.731~ -0.00326 0.00113 0.0456
(0.375) (0.283) (0.346) (0.0248) (1.431) (0.0217) (0.143) (0.0380)
5 quantiles of ses_2 -0.779* -0.336 -0.541 -0.0242 -0.961 -0.0204 -0.0638 0.0118
(0.294) (0.246) (0.321) (0.0203) (1.522) (0.0147) (0.0742) (0.0235)
5 quantiles of hle 0.0250 -0.364* -0.699** -0.00742 -1.595 -0.00675 -0.0450 0.00386
(0.178) (0.132) (0.219) (0.0197) (1.141) (0.0144) (0.0843) (0.0230)
5 quantiles of choreload_2 0.518* 0.397* 0.162 0.00831 -0.816 0.00965 -0.0763 -0.0206
(0.195) (0.185) (0.240) (0.0131) (1.407) (0.00834) (0.0806) (0.0227)
Repeated at least one grade -0.162 -0.213 0.495 0.0510 -3.422 0.0302 -0.181 -0.0973*
(0.617) (0.366) (0.421) (0.0483) (2.610) (0.0347) (0.173) (0.0345)
# of times transferred schools 0.0617 -0.565 -1.257 -0.0400 3.129 -0.0722 0.201 0.0202
(0.528) (0.433) (0.839) (0.0526) (4.553) (0.0438) (0.141) (0.0817)
Number of Household Members 0.560~ 0.491~ 0.465 0.00819 1.218 0.0169 0.0783 0.0120
(0.290) (0.278) (0.287) (0.0262) (1.580) (0.0188) (0.130) (0.0216)
Distance to Nearest Tar Road in KM 0.266 0.150 0.105 0.00728 -3.098* 0.0143 -0.118 -0.0607*
(0.241) (0.145) (0.257) (0.0128) (1.449) (0.0104) (0.0974) (0.0263)
School has Electricity -1.925** -1.338~ -0.725 -0.141~ -3.867 -0.0978~ 0.288 0.174~
(0.577) (0.663) (0.674) (0.0756) (3.476) (0.0525) (0.340) (0.0906)
Constant -3.803 1.669 4.933 0.267 66.67** 0.0629 2.076 -0.129
(4.321) (3.364) (4.248) (0.307) (20.25) (0.269) (1.368) (0.424)
Observations 248 248 248 248 248 248 248 248
R-squared 0.127 0.100 0.095 0.077 0.086 0.080 0.044 0.058
Robust standard errors in parentheses, *** p<0.001, ** p<0.01, * p<0.05, ~ p<0.1
Due to lack of statistical significance, variables included in the model but not shown here include number of grades at the school, reporting the
53
school to be far from home, missing school in the past week, prior ecd attendance, and sex.
Appendix H: Multilevel Regression Model, Channels for Reading Skills Improvement
(1) (2) (3) (4) (5) (6) (7) (8)
VARIABLES Letters MUW
Pseudo
Words Reader WPMC Accuracy
Reading
Comp RWC
Reading to others in community gain -0.551
-1.052*
0.0456
pctread2ugain 1.771~ 0.915
-0.460~ -0.176*
nreadmatsgain 0.263~
5 quantiles of hle 0.238 -0.216* -0.472**
0.0301
# of activities/lesson components reported
in language arts class -0.159 -0.281*
-0.0128~
0.0164
# of reading skills reported addressed in
lesson 0.429~ 0.257
# of types of student reading observed 0.389 0.517*
-
0.0527*
Teacher has an updated balanced
scorecard/other type of tracking sheet 0.633 0.699~ 1.779***
15.41*** 0.103** 0.582*** 0.113
# of types of effective reading strategies
used by teacher during reading
-0.252*
-1.629*
# of activities/lesson components
observed in language arts class
0.425*
4.072**
# of reading skills addressed in lesson
-0.597**
-3.259* -0.0273~ -0.0771 -0.0308
# of types of print and learning materials in
the classroom
1.488*
-
0.0796~
-
0.0389*
Estimated % of children encouraged to
participate in class
0.311*** 0.118***
pctnseereadgain
0.520* 0.190~
Constant
-
4.785* 3.938*** 7.171*** 0.0226 47.36*** 0.157*** 1.049* 0.0614
Observations 446 446 446 446 446 446 446 446
R-squared 0.115 0.072 0.083 0.003 0.094 0.033 0.063 0.069
54
*** p<0.001, ** p<0.01, * p<0.05, ~ p<0.1
Variables included in the model but not displayed here include sex, age, ses quintiles, chore/work quintiles, prior ecd
attendance, hle quintiles, having ever repeated a grade, number of household members, having transferred schools, having
missed at least one day of class in past week, reporting school is far from home, distance to nearest tar road, school electricity,
and number of grades at school.