literacy boost metro manila - resource centre · 1 literacy boost metro manila endline report...
TRANSCRIPT
1
Literacy Boost
Metro Manila
Endline Report September 2013
Adrian Badiable, Jarret Guajardo, Rachael Fermin, and Elvie Jo C. Robis
Acknowledgement to Richelle Gutierrez Dela Pena, Literacy Boost
community volunteers, and the teachers and students of Cielito Zamora
Memorial School for their hard work and excellent support
Special thanks to our team of enumerators:
Maria Luisa Lazaga, Monaliza Leonardo, Francis Masilang, Dave Mark Valerio,
Marly Guttierez, Anna Rose Pineda, Lea Llanita, Hanni Francisco, Francis
Macalinao, Jojimar Tabiolo, Mario Padida, Jr., Abigail Mallari, Ivy Gaviola, Geraldine
Rodriguez.
1
Table of Contents
Executive Summary .......................................................................................................................................... 1
Introduction ....................................................................................................................................................... 5
Context ........................................................................................................................................................... 6
Implementation History .............................................................................................................................. 6
Methodology .................................................................................................................................................. 7
Sample Selection .......................................................................................................................................... 7
Instruments and Measurement .................................................................................................................. 8
Analysis ......................................................................................................................................................... 8
Attrition Analysis .............................................................................................................................................. 8
Endline Descriptive Statistics Analysis ...................................................................................................... 10
Student Characteristics ............................................................................................................................ 10
Home Literacy Environment Data ......................................................................................................... 11
Literacy Boost Participation .................................................................................................................... 14
Sex Differences .......................................................................................................................................... 15
Impact on Emergent Literacy Skills ............................................................................................................ 15
Overall Impact ............................................................................................................................................ 15
Evidence from Focus Groups .................................................................................................................. 19
Equity Analysis ............................................................................................................................................ 21
Individual Skill Analysis.................................................................................................................................. 24
Concepts about Print .................................................................................................................................... 24
Letter Awareness .......................................................................................................................................... 25
Word Recognition: Most Used Words ....................................................................................................... 26
Readers and Nonreaders ............................................................................................................................ 27
Fluency and Accuracy ................................................................................................................................... 28
Reading Comprehension .............................................................................................................................. 30
One New Way to Look at Data: Readers with Comprehension ............................................................. 33
Trends in Reading Skill Data from Multilevel Regression Models ...................................................... 34
LB Reading Activities and Reading Achievement ................................................................................ 34
Literacy Boost Home Literacy Environment Channels for Reading Skill Improvement ............ 35
2
Conclusion ...................................................................................................................................................... 36
Appendix A. Literacy Boost Activities prior to Endline Assessment............................................. 38
Appendix B: Inter-rater Reliability ......................................................................................................... 40
Appendix C: Student Background Characteristics ............................................................................. 41
Appendix D: Multilevel Regression Analysis ....................................................................................... 43
Appendix E: Details of Focus Group Discussions (FGDs) with Primary Stakeholders ............. 45
1
Executive Summary
This report examines the results of a follow-up student background survey and reading
assessment conducted in June 2013 as part of Save the Children’s Literacy Boost program. The
Literacy Boost program includes teacher training, community reading activities, and age-
appropriate local language material creation to support emergent literacy skills among early-
grade children. In September 2012 a baseline survey and reading assessment was conducted
before the commencement of these activities, covering 261 grade 2 students throughout 14
sections of Cielito Zamora Memorial Elementary School in the Caloocan district of Manila in
the Philippines. The 14 grade 2 sections from which students were assessed were randomly
split into seven ‘full intervention’ sections designated to receive Literacy Boost, and seven
additional sections received no intervention at all to the extent possible. Following up with as
many of the original students as possible, the June 2013 survey and reading assessment of 283
grade 3 students serves to measure how students’ reading skills have changed over time as well
as to estimate the impact of eight months of the Literacy Boost intervention.
However, three issues undermined the ability to estimate program impact. First, randomization
of sections placed the two highest-performing sections in the LB group. Second, due to the fact
that both intervention and comparison groups were together in the same school, it is inevitable
that a degree of sample contamination occurred. Third and finally, by the time of the endline
assessment it was possible to find only 168 of the original 261 students assessed at baseline
(64%). Thus, an additional 115 students were randomly resampled who had been enrolled in
grade 2 with the cohort of students assessed at baseline in order to improve the sample size.
Attrition Analysis
The attrition rate is significantly higher among Literacy boost students (43%) than among
comparison students (28%). Using a form of transportation other than walking to arrive at
school and having exchanged books with others in the community in the past week were found
to be significantly associated with a higher likelihood of attrition. More information is necessary
to explain these findings. In addition, students with lower baseline Filipino most used word
scores and students who did not qualify readers in English at baseline were also significantly
more likely to attrit at endline. Students who struggle with their basic Filipino and
English reading skills may be more at-risk of not attending or dropping out of
school, and thus may need more special support to continue their stronger peers.
Background Characteristics and Home Literacy Environment
Literacy Boost and comparison children are remarkably similar on average, enough for a
difference-in-difference impact evaluation of the Literacy Boost intervention. Children’s self-
reported presence of various types of print in the household increased for most types of print
over time. However, many students still face a lack of variety of reading materials at home, and
also low levels of household members telling stories, singing, and playing games with students.
2
Literacy Boost should redouble its Parental Awareness Workshop efforts in order
to improve the household literacy environment for students.
In terms of student literacy habits, more children reported learning to read outside of school or
with family community members as well as having read to another person in the community in
the week prior to the assessment. A slightly lower proportion of students reported exchanging
books with others, similar to the proportion who reported borrowing books from the school
library at endline. No statistically significant differences exist between Literacy Boost and
comparison groups in terms of either student literacy habits or community activity
participation. It is likely that this is a product of both sample contamination as well as
small sample size.
Participation in Literacy Boost community activities is low among all students, and more
efforts should be made to expand Reading Camp coverage in terms of geography,
scheduling, and number of volunteers. In addition, more qualitative information
will need to be gathered to explain the even lower Book Bank borrowing rates
among students.
Impact Evaluation of Literacy Boost
With the exception of proportion of English readers with comprehension, no significant impact
is found overall between Literacy Boost and comparison groups. While results are not
statistically significant in the vast majority of skills, the fact that effect sizes are
either positive or practically zero when excluding the two high-performing Literacy
Boost sections indicates that the Literacy Boost program may have had a small
degree of impact overall. Beyond deficiencies in the program itself, sample
contamination may have undermined the ability to perceive a difference between
Literacy Boost and comparison groups.
No benchmarks set at baseline were met. While many benchmarks are reasonable,
those benchmarks set at 100% of items are unlikely to ever be reached and thus
letter knowledge and Filipino most-used words may be considered nearly-achieved.
However, more work remains to be done for the majority of other skills, especially
giving children enough exposure to actual books to increase their mastery of
concepts about print. It will be important to better understand why limited or no
impact was found. This can be done through combining evidence from this
quantitative analysis with qualitative evidence from focus group discussions. The
Literacy Boost program may need to be improved and lessons from the first year
of piloting the program applied so that Literacy Boost can demonstrate results for
students in its second phase with a more rigorous research design.
3
From focus group discussions conducted with students, teachers, community facilitators, and
parents, there is qualitative evidence that Literacy Boost has made a positive impact on
processes and learning outcomes. It does not appear that students attending Reading Camps
are either those with the highest or lowest baseline scores and thus the major challenge is that
not all children attend Reading Camps and therefore have the opportunity to benefit. From
analysis of focus group feedback, Literacy Boost should:
expand Reading Camp opportunities for all children by recruiting additional
volunteers and expanding coverage in terms of geography and scheduling
identify safe Reading Camp venues with adequate facilities
facilitate the provision of as many new titles as possible at Book Banks
better educate all stakeholders that the objective of the Reading Camps is to
provide a more informal and fun environment for children to practice their
reading skills, not an extension of classroom time
encourage parents to help their children with a variety of aspects of reading,
especially in terms of asking comprehension questions
further investigate whether or not messages related to literacy are getting
through to parents, and how these messages can be better conveyed
invest more in advertising Parental Awareness Workshops, and make
accommodations such as a young children’s play area at the workshop so
that those with young children to be able to attend
offer teachers refresher trainings and provide new titles for school libraries
Equity Analysis
The strongest baseline drivers of inequality in this context appear to be SES and HLE, followed
by work and chore load, although gender is also an issue with boys scoring significantly lower
than girls in a few skills. Literacy Boost helped boys catch up to girls in a few lower-order skills
such as concepts about print and Filipino and English most-used words. Suggested activities,
games, and materials should be provided to teachers and community volunteers to
engage disadvantaged groups of children, and perhaps extra remedial help can be
offered to students at-risk of falling behind.
Student Skill Profile
Based on analysis of students’ individual skills, Literacy Boost programming should:
include familiarization on parts of a story to participants of Reading Camps
train teachers and community reading activity facilitators to ensure all
students know all letters of the alphabet especially ñ, ng, l, q, j, & f.
highlight letter knowledge and phonemic awareness during Reading Camp
Sessions, ensuring active participation of all students, not just those that
know their letters
4
continue providing technical support and guidance to teachers to help
students learn and practice their decoding skills as well as provide ample
opportunity inside and outside of the classroom for students to practice
reading and memorizing basic, frequently used words such as these
give students ample time to practice reading through such initiatives as
reading buddies in order to build their automaticity
further enhance readers’ opportunity for practice of reading enjoyable and
relevant materials
facilitate the provision of mother-tongue based reading materials as a good
complement to the implementation of Mother-Tongue-Based Multilingual
Education program of DepEd and its schools
help teachers and community volunteers focus on asking different types of
questions before, during, and after stories so that students can better
develop their literal and deeper comprehension skills, especially
understanding the details of the story, making inferences based on the
information in the story, and summarizing the story
Looking at the relationship between background characteristics and reading skills, older
children and those from larger households did not improve their scores as much between
baseline and endline as their younger peers and those students from smaller households. Both
older children and those from larger households may need special remedial help in
order to keep pace with other children.
There were no significant relationships between reported participation and reading skill gains
over time. However, frequency of Reading Camp attendance was positively associated with
gains in letter knowledge and Filipino fluency. This is additional evidence that Literacy
Boost Reading Camps may have helped students improve some of their reading
skills.
Finally, there was no statistically significant association between Literacy Boost and potential
home literacy environment channels for impact. This may be due to potential sample
contamination between Literacy Boost and comparison groups, but students are
also not reporting improvements in their household literacy environment and
literacy habits in most measures. Literacy Boost will need to apply lessons from
focus groups held with parents to determine what more can be done to help
parents better engage in literacy acquisition promoting activities.
5
Introduction
This report examines the results of a follow-up student background survey and reading
assessment conducted in June 2013 as part of Save the Children’s Literacy Boost program. The
Literacy Boost program includes teacher training, community reading activities, and age-
appropriate local language material creation to support emergent literacy skills among early-
grade children. In September 2012 a baseline survey and reading assessment was conducted
before the commencement of these activities, covering 261 grade 2 students throughout 14
sections of Cielito Zamora Memorial Elementary School in the Caloocan district of Manila in
the Philippines. The 14 grade 2 sections from which students were assessed were randomly
split into seven ‘full intervention’ sections designated to receive the teacher training, community
activities, and material creation components of Literacy Boost, and seven additional sections
received no intervention at all to the extent possible. Following up with as many of the original
students as possible, the June 2013 survey and reading assessment of 283 grade 3 students
serves to measure how students’ reading skills have changed over time as well as to estimate
the impact of eight months of the Literacy Boost intervention.
Skills tested in this assessment include concepts about print (CAP), letter awareness, single
word reading of most used words (MUW), reading fluency, reading accuracy, reading
comprehension, and listening comprehension. The key research questions to be explored in
this report include the following:
1. How has the sample of students changed over time?
o Are the students who were able to be found at endline different than those who
were not able to be found? If so, how?
o Did the attrition rate differ between intervention and comparison sections?
2. Of the students who were able to be found at endline, how comparable are baseline
background characteristics and reading skills in intervention sections versus comparison
sections?
3. What can the endline assessment tell us about students’ reading skills?
o What does this mean for continuing Literacy Boost programming in this school?
4. Did the Literacy Boost program exhibit impact on students’ reading skills?
o For which types of students was impact the greatest/least?
o Does this impact result in more equitable outcomes for traditionally
disadvantaged groups?
5. How does students’ development of reading skills over time vary by student
background and community literacy environment?
o What does this mean for targeting Literacy Boost’s various intervention
components?
To investigate these questions, this report first describes the context and implementation
history of Literacy Boost in the Cielito Zamora Memorial School. Next, this report gives an
overview of the research methods used; including sampling, measurement, and analysis. The
report will then analyze the attrition of the sample over time and how intervention and
comparison groups have or have not remained statistically similar. The report will then present
results from impact analysis investigating the extent to which Literacy Boost appears to have
6
improved students’ reading skills. After this, students’ endline scores for each of the reading
skills will be analyzed to determine which skills students have mastered and which require
additional improvement. Finally, the report will investigate any correlations between baseline-
endline reading skill development and student background or community literacy environment
variables using multilevel regression analysis.
Context
The urban district of Caloocan North is located in the northern part of the Metro Manila area
in the Philippines. The vast majority of the population speaks Filipino (Tagalog)1 as their native
language. Caloocan is one of the more economically disadvantaged in the Metro Manila area;
and school was disrupted, property damaged, and residents killed in typhoon flooding in the
period immediately preceding the assessment. Other challenges faced by elementary school
students in this area include being in large classes, having short contact time with teachers
(approximately 4 hours per day), and exposure to violence in the community in addition to
extremes of pollution and a generally unsanitary environment.
Implementation History
All Literacy Boost and comparison sections in this school received the same School Health and
Nutrition interventions from Save the Children. See appendix A for a complete timeline of
Literacy Boost intervention components.
Country Office staff report the following challenges with respect to implementation:
- The original idea with the consultants was to train a core group who will serve as
trainers for the Reading Awareness Workshops for parents and the Reading Camp for
children. This core group was composed of child advocate volunteers from former
sponsorship impact areas in Paranaque, Taguig and Pateros, who have had the
experience implementing reading programs in their respective areas. Initially, they
served as support to the small group of trained volunteers from Barangay 177 who
were inexperienced and lacking in confidence in the first Reading Awareness
Workshops for parents. But because of the geographic distance of the core group’s
residence from the Caloocan impact area, this arrangement was not sustained. There
were only 53 adults and 33 youth who were trained.
- As the months went by, another challenge was the turnover of the volunteers, who
found regular jobs, or had to quit volunteering because of school.
- A perennial challenge for implementers of a program which includes parents is their
involvement in activities. They would allow their children to attend the Reading Camp,
but would not attend or commit to the Reading Awareness Workshops.
- Although Reading Camps began in November, there were not a lot of children then
attending then. Only 4 Reading Camps (RCs) were open in November 10 (2 RCs) and
Nov. 24 (2 more RCs), with only about 30 children in each RC site. More children
1 This report will use the term Filipino, as the reading assessment used is based on the grade 2 Filipino textbook. The differences between Filipino and Tagalog are minor, as Filipino is essentially a formalized version of Tagalog.
7
started to pour in after the Christmas vacation when more kids knew about the LB
program.
Methodology
Sample Selection
The sample for the September 2012 baseline assessment encompassed 261 grade 2 students,
divided between 14 sections in the same large urban school of Cielito Zamora Memorial
Elementary. In each of the 14 sections, 20 children in grade 2 were sampled. Ten boys and ten
girls were randomly selected where there were more than 20 children present in a section on
the day of assessment. Due to random assessor error in recording the correct student code, 19
observations from the baseline sample were dropped, yielding the final baseline sample size of
261 students. Of the 14 grade 2 sections, seven were randomly assigned to receive the full
Literacy Boost intervention (n = 133) and seven were likewise randomly assigned to receive no
Literacy Boost intervention (n = 128) as the comparison sections.
However, three issues with the research design merit discussion. First, randomization was
performed without taking into account the fact that students are effectively randomly
distributed among grade 2 sections except in the case of two sections. Students in these two
sections were assigned to these sections due to their high academic performance. During the
randomization, by chance both of these sections were assigned to the Literacy Boost group.
Therefore, during impact analysis tests will be conducted both including and
excluding these two high-performing sections to determine whether this affects the
results.
Second, due to the fact that both intervention and comparison groups were together in the
same school, it is inevitable that a degree of sample contamination occurred. With regard to
teacher training, only teachers from Literacy Boost sections were trained (the only anomaly
was that one Literacy Boost section teacher did not receive training due health issues).
However, teachers may have informally exchanged ideas and techniques related to Literacy
Boost. Furthermore, with regard to community activities an equally (small) proportion of
comparison students mentioned borrowing books from Literacy Boost Book Banks as Literacy
Boost students, and only a slightly lower proportion of comparison students reported having
attended a Reading Camp as Literacy Boost students. Impact analysis will attempt to
control for the community activity sample contamination, but cannot control for
any sample contamination from teacher training or other spillover effects.
Third and finally, by the time of the endline assessment it was possible to find only 168 of the
original 261 students assessed at baseline (64%). Save the Children Philippines staff believe that,
due to the mobile nature of families in the Caloocan area, a large proportion of students moved
to other areas (and perhaps schools) as their families searched for work. Thus, an additional
8
115 students were randomly resampled who had been enrolled in grade 2 with the cohort of
students assessed at baseline, in order to bring the total number of endline students to 20
assessed per each of the 14 grade 2 sections.
Instruments and Measurement
For the student assessment, all students in the sample were asked about their background
characteristics (age, household possessions, household building materials, health indicators,
etc.). Students were also asked about their family members’ reading habits (who they had seen
reading in the week prior to the assessment, who had read to them, etc) as well as about their
participation in Literacy Boost community activities such as Reading Camps attendance and
Book Bank borrowing.
After collecting this background data, all students were also given a literacy test composed of
six components administered through five sub-tests: concepts about print, letter awareness,
single word recognition (reading of most used words), reading fluency & accuracy (words per
minute read correctly and total percentage of passage read correctly; both within the same sub-
test), and a set of comprehension questions linked to the fluency & accuracy passage. The same
set of comprehension questions were administered for both those students who could read
independently (reading comprehension) and those who could not and thus had the assessor
read to them (listening comprehension). Children who could read independently (readers)
were defined as those who could read at least five words correct in the first 30 seconds of the
fluency & accuracy sub-test and finish the full passage. Those children who could not read
independently (nonreaders) were those who either read less than five words correct in the first
30 seconds of the fluency & accuracy passage or who could not finish reading the passage.
The most used words, fluency & accuracy, and comprehension sub-tests were conducted in
both Filipino and English, the former being the native language of nearly all of the students. All
assessment instructions were given in Filipino. For a discussion of inter-rater reliability, please
see Appendix B.
Analysis
This report will use comparison of means through clustered t-tests to assess the comparability
of intervention and comparison groups and consider differences between boys and girls scores.
Finally, this report will consider relationships between reading skills and background factors
using multilevel regression models that account for clustering in schools.
Attrition Analysis
This endline study assessed grade 3 students from the same seven Literacy Boost grade 2
sections and seven grade 2 comparison sections from Cielito Zamora Memorial School that
were assessed in the 2012 sample. At baseline 261 students were assessed, but only 168
9
students could be located at endline2. In order to determine whether or not the Literacy Boost
and comparison endline sample is still comparable, it is first necessary to test whether attrition
rates were different between the two sub-samples. Using clustered t-tests, the attrition rate is
significantly higher among Literacy boost students (43%) than among comparison students
(28%). This finding holds even excluding the two high-performing Literacy Boost sections, and
also remains marginally insignificant even controlling for a variety of baseline background
characteristics. Thus, it will be necessary to retest the comparability of the remaining
sample of Literacy Boost and comparison students.
Of all the variables collected about students’ socioeconomic status, home literacy environment,
and reading skills, two background variables and two reading outcome variables collected at
baseline robustly predict the type of student who might have attritted at endline. The technique
used to arrive at these findings was a multilevel logistic regression analysis, with students
clustered in schools.
Using a form of transportation other than walking to arrive at school was found to be
significantly positively associated with a higher likelihood of attrition. More information is
needed to understand the explanation for this, but not walking to school is significantly
correlated with socio-economic status (SES) and these students’ families may have the means to
relocate to other areas. Another explanation is that students not walking to school may live
farther from the school and may drop out if the distance becomes too much of a burden. The
other background variable robustly associated with higher likelihood of attrition is having
exchanged books with others in the community in the week prior to the assessment. More
information is necessary to explain this unintuitive finding as well. Finally, students with lower
baseline Filipino most used word scores and students who did not qualify readers in English at
baseline were also significantly more likely to attrit at endline. Note that these associations are
correlational only and causation cannot be established. However, students who struggle
with their basic Filipino and English reading skills may be more at-risk of not
attending or dropping out of school, and thus may need more special support to
continue their stronger peers.
Figure 1 below displays the relationship between two of these variables, walking to school and
Filipino most used word score, and the likelihood of attrition at endline.
2 Please note, this does not necessarily mean that students dropped out. There are many possible reasons for
student absence, and we can only note that the students were absent on the particular day of the assessment. The
analysis presented here does not establish why students were absent.
10
Endline Descriptive Statistics Analysis
Student Characteristics
Table 1 summarizes student background data, presenting the averages for all Literacy Boost and
comparison students as collected at baseline and endline. Of the sample of 168 students who
were assessed at both baseline and endline, excluding the 24 students from the two high-
performing sections, t-tests assuming unequal variances reveal that Literacy Boost and
comparison children are remarkably similar on average. Only in terms of chore frequency and
breakfast size are the two groups significantly different (see Appendix C for detailed statistics
on students’ time and health). It is possible that this small number of differences is due purely to
chance. Thus, children in Literacy Boost and comparison groups are similar enough
to perform a difference-in-difference impact evaluation of the Literacy Boost
intervention.
0%
20%
40%
60%
80%
100%
0 10 20
Pre
dic
ted
like
liho
od
of
attr
itio
n
Baseline Filipino MUW score
Figure 1: Likelihood of Attrition by How Student Arrives at School and
Filipino MUW Score
Does not walk toschool
Walks to school
11
Table 1: Student Background Characteristics by Sample Group (excluding sections 1 & 2)
Baseline Endline
Variable LB Control Sig. N LB Control Sig. N
% Female 44% 50%
144
Age in years 7.6 7.4
140 8.5 8.2
144
Walks to school 73% 75%
144
Previously attended ECD 81% 91%
139 76% 78%
141
Years of ECD 1.2 1.2
94 2.2 2.3
81
Has repeated a grade 14% 12%
144 14% 12%
142
# of household members 3.5 3.0
144 2.9 3.2
144
SES index 2.5 2.6
140 2.5 2.6
144
Chores/work index 1.7 1.7
144 2.5 2.4
144
Frequency of work1 1.0 1.1
36
Frequency of chores1 2.3 2.5 * 142
Frequency of chores and work1
3.7 3.3
144
Frequency of study1 2.7 2.7
142 3.4 3.4
144
Frequency of play1
2.7 2.4
144 * Statistically significant difference between Literacy Boost and comparison groups at p<0.05 1 Students were asked whether they spend no time (0), a little time (1), or a lot of time (2) on the activity in
question for each time period of before school, after school, and on the weekend. If necessary, it was clarified with
students that a little time meant less than two hours, and a lot of time meant two or more hours.
In terms of children’s time (see Appendix C for details), the only significant difference between
Literacy Boost and comparison students is that comparison students report spending less time
on average on work or chores after school than Literacy Boost students.
Home Literacy Environment Data
Just as important as investigating children’s school and classroom environment is examining
their home literacy environment (HLE). The HLE is not only about materials in the home, but
how those materials are used to engage the child in reading and learning. Hess and Holloway
(1984) identified five dimensions of the home literacy environment that are theoretically related
to reading achievement in children. The first is value placed on literacy, which the Literacy Boost
assessment operationalizes by asking children whether they see anyone reading at home. The
second is press for achievement, is operationalized as individuals telling the student to study. The
third is the availability and use of reading materials, operationalized as the amount of printed
materials at home. The fourth dimension is reading with children, operationalized by asking the
12
children whether anyone reads to them at home. The last is opportunities for verbal interaction,
operationalized as family members telling stories to learners.
In terms of the third dimension of availability and use of reading materials, Figure 2 below shows
that children’s self-reported presence of various types of print in the household increased for
most types of print over time (except for textbooks, for which prevalence approached 100% at
both baseline and endline). The increase was most marked for religious materials. Endline
prevalence of reading materials was similar between Literacy Boost and comparison groups
with the possible exception of presence of storybooks in the household. This difference is
marginally insignificant, meaning there is a greater chance that this difference was found by
chance. It is reported here because it indicates that for some reason more comparison students
reported having storybooks in their homes, which may fit with evidence displayed in Figure 3
that comparison students report a significantly larger percentage of household members seen
reading than in Literacy Boost households.
~Significant difference from control group at baseline at p<0.1
Figures 2 and 3 indicate that many students still face a lack of variety of reading materials at
home, and also low levels of household members telling stories, singing, and playing games with
students. Literacy Boost should redouble its Parental Awareness Workshop efforts
in order to improve the household literacy environment for students.
31%~ 33% 37%
23%
64%
46% 39% 39%
26%
53%
0%
10%
20%
30%
40%
50%
60%
70%
Storybooks Coloring books Newspapers Magazines Religiousmaterials
% o
f sa
mp
le
Figure 2: Types of Reading Materials Reported in the Household, by Sample Group (excluding
high-performing sections)
Baseline LB
Endline LB
Baseline Control
Endline Control
13
*Significant difference from control group at baseline at p<0.05
In terms of comparability between Literacy Boost and comparison samples, it
appears that comparison students may have a slight advantage in terms of
storybooks in the household and percentage of household members seen reading.
These differences did not exist at baseline. In fact, the finding about percentage of
household members seen reading comes about because that percentage actually
dropped over time among the Literacy Boost sample. More information is needed
to explain this finding, but household literacy environment will nevertheless be
controlled for during impact analysis.
Children were also asked about their own literacy habits, and Figure 4 below displays modest
change in the three indicators of this area over time. More children reported learning to read
outside of school or with family community members as well as having read to another person
in the community in the week prior to the assessment. A slightly lower proportion of students
reported exchanging books with others, similar to the proportion who reported borrowing
books from the school library at endline. No statistically significant differences exist between
Literacy Boost and comparison groups. This may be due to sample contamination or to
the indirect link between Literacy Boost programming and students’ literacy
habits.
48%*
79%
44% 42% 32%
56% 62%
82%
47% 42% 30%
59%
0%
20%
40%
60%
80%
100%
Seenreading
Encouragingstudent to
study
Reading tothe student
Telling storyto the
student
Singing tothe student
Playinggame withthe student
ave
rage
% o
f h
ou
seh
old
me
mb
ers
Figure 3: Proportion of Household Members
Engaging in Literacy Activities, by Sample Group (excluding high-performing sections)
Baseline LB
Endline LB
Baseline Control
Endline Control
14
Note that sharing books with others in the community in the past week was not collected at endline
Literacy Boost Participation
The last set of background questions asked children about their participation in various Literacy
Boost community activities. Based on the information presented in Table 2, participation in
Literacy Boost community activities is low, and more efforts should be made to expand
Reading Camp coverage in terms of geography, scheduling, and number of
volunteers. In addition, more qualitative information will need to be gathered to
explain the even lower Book Bank borrowing rates among students.
77%
17%
44%
86%
13%
51%
14%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Reports learning toread primarily
outside ofschool/withcommunitymembers
Exchanged bookswith others in
community in pastweek
Read to others incommunity in past
week
Borrows books fromschool library
% o
f sa
mp
le
Figure 4: Student Literacy Habits (excluding high-performing sections)
Baseline
Endline
15
Table 2: Literacy Boost Community Activity Participation by Sample Group (entire endline
sample)
Variable LB Control Sig. N
Has borrowed book from Book Banks 7% 7%
270
# of times borrowed Book Bank book per week (Book
Bank borrowers) 2.1 1.2
13
Has attended Reading Camp 22% 17%
273
Frequency of Reading Camp attendance (RC attendees)a 2.7 2.7
53
How many other children attended Reading Camp (RC
attendees) 27 21
18
Missed a Reading Camp session (RC attendees) 43% 78% ** 51
Did make and take at Reading Camp (RC attendees) 55% 61%
52
Frequency of make and take for (RC attendees) 0.9 0.8 52 ** Statistically significant difference between Literacy Boost and comparison groups at p<0.01 level a 1 = Once, 2 = Rarely, 3 = Many times, 4 = Always
Strangely, although participation is often higher among the Literacy Boost sample than the
comparison sample, there were no statistically significant differences between the two groups,
with the exception of the percentage of students who reported missing a Reading Camp
session. It is likely the findings of similar participation rates between Literacy Boost
and comparison students is a product of both sample contamination as well as
small sample size.
Sex Differences
There are no statistically significant differences between Literacy Boost and comparison
students for the majority of background variables. In terms of students’ time, it appears that
boys may be more likely to engage in play other than computer games than girls, and spend
more time playing before school and on the weekends. Girls may spend more time on work or
chores on the weekend, and spend more time studying overall.
With home literacy environment, girls may be more likely to have read to others in the
community in the week prior to the assessment. Finally, there are no statistically significant
differences between boys and girls regarding Literacy Boost community activity participation.
Impact on Emergent Literacy Skills
Overall Impact
In this section we examine gain scores in each literacy skill to assess the impact of Literacy
Boost on student learning outcomes. In Tables 3 and 4 below, baseline & endline data are
16
presented for both Literacy Boost and comparison students. Furthermore, the change from
baseline to endline is presented (gain scores), along with the initial benchmark3 set during the
baseline assessment and whether the benchmark was met. Clustered t-tests are were used to
test whether the findings are generalizable to the population, and standardized effect sizes4 are
also reported to give a comparable sense of magnitude of the difference in Literacy Boost and
comparison gain scores across all skills.
With the exception of proportion of English readers with comprehension,5 no significant impact
is found overall between Literacy Boost and comparison groups when excluding the two high-
performing sections from the analysis. The letter knowledge gain score difference between
Literacy Boost and comparison groups is marginally insignificant when not controlling for other
factors (see Appendix D for regression tables), but becomes completely insignificant once
controlling for these factors. All other overall results, whether in terms of statistical significance
or effect size (magnitude), do not change by much whether controlling for other factors. As
expected, including the high-performing sections in the analysis raises the effect sizes of the
different performance between Literacy Boost and comparison groups, but does not change
statistical significance.
3 A Note about Benchmarks: While Save the Children has used this approach to reading assessment in Bangladesh,
Ethiopia, Guatemala, Haiti, Malawi, Mali, Mozambique, Nepal, the Philippines, Sri Lanka, South Africa, Vietnam,
Uganda, Yemen, and Zimbabwe, comparison across countries and languages is less helpful than more detailed
contextual information for setting expectations of impact. For each measure used in these assessments, the
baseline established the upper end of the range of scores (the 75th percentile) as a benchmark, or a reasonable
estimate of what is currently possible among these children. 4 The effect size is a measure of the magnitude of an observed difference, expressed in standard deviations in order
to compare across different types of measures. So, as Literacy Boost exhibited a statistically significant effect size of
0.46 in RNGPS for fluency, this means that Literacy Boost children improved almost a half standard deviation more
than comparison children improved. 5 To calculate the definition of ‘readers with comprehension,’ the Literacy Boost research team standard definition
of at least 80% of reading comprehension questions correct (in this case, at least 8 of 10 questions correct) and
either at least one standard deviation of fluency lower than the average for children who scored 8 of 10
comprehension questions correct or at least one standard deviation of recognition lower than the average for
children who scored 8 of 10 comprehension questions correct, based on baseline scores. Thus, to meet this
threshold in Filipino children must score at least 8 of 10 comprehension question correct and read either at least
29.5 words per minute correct or with 91% recognition.
17
Table 3: Baseline/Endline Filipino Scores with Benchmarks, Significance, and Effect Sizes
(excluding high-performing sections)†
A B Ca Da E F G Hb Ib, c
Reading
Skill Group
Baseline
Scores
Endline
Scores
Benchmark
from
Baseline
Benchmark
met?
Change
from
Baseline
to
Endline
Sig. Diff.
of Gain
Scores
Between
Groups
Effect
Size of
Diff in
Gain
Scores
Concepts about
LB 7 6.8 9 No
-0.2 None 0.08
Comparison 6.8 6.5 -0.3
Letter
Identification
(of 56)
LB 47.5 52.9
56 No
5.4
None 0.13 Comparison 50.9 52.4 1.5
Filipino Most
Used Words (of
20)
LB 12.8 18.9
20 No
6.1
None 0.08 Comparison 15 18.5 3.4
% Filipino
Readers
LB 79% 90% 100% No
12% None -0.01
Comparison 77% 88% 11%
Filipino Fluency
(Words Per
Minute)f
LB 35.9 51.2
62 No
15.7
None 0.11 Comparison 38.6 48.3 9.8
Filipino Accuracy
(% text read
correctly)f
LB 68% 88%
99% No
20%
None 0.07 Comparison 69% 84% 16%
Filipino Reading
Comprehension
(of 10)f
LB 4 4.8 6d No
0.8 None 0.01
Comparison 3.7 4.4 0.8
% Filipino
Readers with
Comprehensione
LB 10% 19%
None N/A
10%
None 0.11 Comparison 10% 15% 5%
† Inclusion of the two high-performing sections does not significantly alter the results a Neither baseline nor endline scores exhibited any statistically significant difference between LB and comparison
students b Calculated through multilevel regression analysis accounting for clustering of students in schools c Widely cited statistician Jacob Cohen describes effect sizes of .2 as small, .5 as medium, and .8 as large. Cohen, J.:
Statistical Power Analysis for the Behavioral Sciences. (2nd ed.) 1988. d Recalculated to include baseline zero scores (nonreaders) e See footnote 5. f Averages include zero scores of nonreaders.
18
Table 4: Baseline/Endline English Scores with Benchmarks, Significance, and Effect Sizes
(excluding high-performing sections)†
A B Ca D E F G Hb Ib,c
Reading
Skill Group
Baseline
Scores
Endline
Scores
Benchmark
from
Baseline
Benchmark
met?
Change
from
Baseline
to
Endline
Sig. Diff.
of Gain
Scores
Between
Groups
Effect
Size of
Diff in
Gain
Scores
Concepts about
LB 7 6.8 9 No
-0.2 None 0.08
Comparison 6.8 6.5 -0.3
Letter
Identification
(of 56)
LB 47.5 52.9
56 No
5.4
None 0.13 Comparison 50.9 52.4 1.5
English Most
Used Words (of
20)
LB 13.9 16.7
19 No
2.8
None 0.07 Comparison 13.9 15.9 2
% English
Readers
LB 50% 75% 100% No
25% None 0.03
Comparison 54% 74% 20%
English Fluency
(Words Per
Minute)f
LB 29.8 48.7
68 No
18.9
None 0.05 Comparison 32.7 49.3 16.7
English Accuracy
(% text read
correctly)f
LB 43% 67%
93% No
24%
None 0.01 Comparison 45% 66% 21%
English Reading
Comprehension
(of 10)f
LB 2.7 4 6d No
1.2 None 0.21
Comparison 2.9 3.1 0.2
% English
Readers with
Comprehensione
LB 8% 23%*
None N/A
15%
* 0.38 Comparison 14% 9% -5%
Significant difference from control group at *p<.05 † Inclusion of the two high-performing sections does not significantly alter the results a Baseline scores did not exibit any statistically significant difference between LB and comparison students b Calculated through multilevel regression analysis accounting for clustering of students in schools c Widely cited statistician Jacob Cohen describes effect sizes of .2 as small, .5 as medium, and .8 as large. Cohen, J.:
Statistical Power Analysis for the Behavioral Sciences. (2nd ed.) 1988. d Recalculated to include baseline zero scores (nonreaders) e See footnote 5. f Averages include zero scores of nonreaders.
While results are not statistically significant in the vast majority of skills, the fact
that effect sizes are either positive or practically zero when excluding the two high-
performing Literacy Boost sections indicates that the Literacy Boost program may
have had a small degree of impact overall. Beyond the program itself, two other
factors may contribute to this very limited impact. First, as evidence exists that
comparison children also received Literacy Boost community activities this may
19
undermine the ability to perceive a difference between the learning of Literacy
Boost and comparison groups. Second, as will be explored in the equity impact
analysis section, impact may have been concentrated among subsets of the school
population.
No benchmarks set at baseline were met. While many benchmarks are reasonable,
those benchmarks set at 100% of items are unlikely to ever be reached and thus
letter knowledge and Filipino most-used words may be considered nearly-achieved.
However, more work remains to be done for the majority of other skills, especially
giving children enough exposure to actual books to increase their mastery of
concepts about print.
It will be important to better understand why limited or no impact was found. This
can be done through combining evidence from this quantitative analysis with
qualitative evidence from focus group discussions. The Literacy Boost program
may need to be improved and lessons from the first year of piloting the program
applied so that Literacy Boost can demonstrate results for students in its second
phase with a more rigorous research design.
Evidence from Focus Groups
In addition to the quantitative assessment, a series of focus groups was held with students
(grade 2, 3, and 4), teachers, community volunteers, and parents. For more details on the focus
group discussions, see Appendix E.
The results of these focus groups indicate that reading is an enjoyable activity for students, who
associate it with learning and intelligence. However, they point to pronouncing and
understanding English words as a source of difficulty in reading. There appears to be a pattern
whereby younger students (grade 2 and 3) read mostly with others, especially family members,
and do so at home and using school books. Starting in grade 3, children begin to branch out
from the home and read in other places such as Reading Camps, and by grade 4 students are
reading alone and doing so in a wide variety of places, including public parks, and with a wider
variety of types of print. Literacy Boost should facilitate this process through
continuation of Reading Camps and increasing the variety of materials available in
Book Banks.
Students report that their family members encourage them to read, and when reading with
family members students mostly mention help with pronunciation, but also say that their
parents ask them questions about stories. Nevertheless, the focus appears to be on
pronunciation, and Literacy Boost should encourage parents to help their children
20
with a variety of aspects of reading, especially in terms of asking comprehension
questions.
The grade 2 and 3 students identified reading activities in school associated with Save the
Children, and said they enjoy reading itself and games. Many students were able to participate in
reading camps, and enjoy them because the facilitators are ‘nice,’ ‘smart,’ and ‘teach them well.’
Students who attend the Reading Camps say they learn a lot from the camps and like the
reading, drawing, playing, coloring activities and practicing writing. The grade 3 students
specifically mentioned improving not only their reading skills, but their writing skills. However,
students are sometimes annoyed and distracted by the presence of younger children at the
camps, and the grade 4 students said they are sometimes unable to attend due to other
commitments such as church. Reading Camps should be scheduled at times that are
most convenient for children, or sessions at different times during the week be
offered if resources exist, perhaps by recruiting additional volunteers.
Reading also appears to be a priority for parents, and many encourage their children to attend
Reading Camps and have a positive view of them. Their children enthusiastically share what
they did at the Reading Camp, and remind the parents when it is time to go again. Parents
report that their children are now more interested in reading and writing, more disciplined
about study, and read faster. Other positive spillovers include improvements in children’s
moods. Parents, along with Reading Camp Facilitators, suggest that venues may need to be
safer, better ventilated, have bathroom facilities, and have more new books for children.
Literacy Boost should work to identify adequate Reading Camp venues and
facilitate the provision of as many new titles as possible to the Book Banks at the
Reading Camps.
Parents attending the Parental Awareness Workshops appear to confuse the messages from
those workshops with other child rights/child protection workshops held by Save the Children.
Especially because household literacy practices do not appear to be changing much,
it will be important for Literacy Boost to understand whether or not messages
related to literacy are getting through to parents, and how these messages can be
better conveyed.
Parents report not attending the awareness workshops because they were not aware of them.
Those who do not attend are even willing to adjust their schedule to attend, but others
mention the presence of young children at home as a barrier to attendance. Literacy Boost
should invest more in advertising Parental Awareness Workshops, especially
through tapping the networks of existing participants to recommend others to
attend. Perhaps accommodations should also be made (such as another volunteer
21
organizing a children’s play area at the workshop) so that those with young
children to be able to attend.
Teachers reported the training sessions useful for them, although it appears much of the
material may not have been new but rather reinforced what they have learned through other
trainings. They appear to appreciate most the terminology and framework of the Components
of Reading as well as Read Aloud methods, but do not find activities that take a lot of class time
useful, such as role plays. Teachers report most frequently using Read Aloud methods and
spelling through sounds. They suggest refresher trainings and new titles as the best
way Literacy Boost can support their efforts.
Finally, the Reading Camp Facilitators also report improved reading skills, specifically that the
children have become more conscious with their vowel and consonant sounds. They report
that children are enthusiastic and many attend, but this is also a challenge for the facilitators
when there is an average of 50 to 80 children in their sessions every week. They also note that
there are many younger children being brought in by their parents even if the lessons will not
be appropriate for their age. From facilitator feedback, Literacy Boost should recruit and
train more Reading Camp Facilitators as well as provide new books, new activities
& materials, and more training for current facilitators on how to lead the camps as
well as fuller awareness of Save the Children programs.
Overall, there is qualitative evidence that Literacy Boost has made a positive impact on
processes and learning outcomes. However, important questions remain about which students
benefitted, and how many. From the LB community activity evidence presented earlier, it
appears that only about one-fifth of students are attending. Using multilevel regression analysis,
it does not appear that attending students are either those with the highest or lowest baseline
scores. Thus, the major challenge appears to be that not all children attend Reading Camps and
therefore have the opportunity to benefit. Additional challenges appear to be lack of new
materials, especially books, and the perceived need for more structured learning at the Reading
Camps. Literacy Boost should expand Reading Camp opportunities for all children,
facilitate the provision of new books, and should better educate all stakeholders
that the objective of the Reading Camps is to provide a more informal and fun
environment for children to practice their reading skills, not an extension of
classroom time.
Equity Analysis
In addition to standard impact evaluation, an equity impact analysis was performed. This analysis
looked at Literacy Boost impact specifically with respect to both students who struggled at
baseline as well as traditionally disadvantaged groups such as girls, the poorest of the poor, the
22
HLE-deprived, those children with large chore/workloads, and those children without previous
ECD experience. To conduct this analysis, endline data was used to construct indices to place
children into quintiles of socio-economic status (SES) and chore/work load, and baseline data
was used to construct an index to place children into quintiles of HLE.6
First, the series of multilevel regressions used for the impact analysis above was re-run to
examine Literacy Boost impact among students who struggled the most at baseline, defined as
either those who were classified as nonreaders in English or those who scored in the bottom
two quintiles of English most-used words.7 These results were similar to the overall impact
analysis, and thus it appears that Literacy Boost exhibited the same limited impact on students
who were struggling as it did for the entire population of grade 2 students.
Table 5 below presents an outline of which groups had significantly lower scores at baseline.
Boys scored significantly lower than girls in a few skills, and the poorest of the poor scored
lower than the less-poor in all of the lower-order skills. Those students from the most
deprived home literacy environments scored lower than their peers from richer home literacy
environments in many skills, and those students with larger work and chore loads (the busiest)
scored significantly lower than those with lighter work and chore loads in a few skills. The only
skill for which students lacking previous ECD experience scored lower than those with ECD
experience was proportion of English readers with comprehension. Thus, the strongest
drivers of inequality in this context appear to be SES and HLE, followed by work
and chore load.
6 Endline data was used for SES and chore/workload indices as it is assumed to be more accurate as the children have matured since baseline. Baseline data was used for the HLE index as it is assumed that the Literacy Boost intervention may change the HLE over time. 7 English most-used words was used as one definition of students struggling at baseline due to its high correlation with most of the other Filipino and English reading skills.
23
Table 5: At baseline, which groups are disadvantaged?
Sub-test Sex SES HLE Work ECD
CAP Boys Poorest
Least busy
Letters
Poorest
Filipino MUW
Poorest Most deprived Busiest
English MUW Boys Poorest
Filipino Reader
Busiest
English Reader
Filipino WPMC
English WPMC Most deprived
Filipino Accuracy
English Accuracy Most deprived
Filipino Reading Comp
Most deprived Busiest
English Reading Comp Boys
Filipino RWC
Most deprived
English RWC Poorest Without ECD
Multilevel regression analysis controlling for the same factors as the overall impact analysis
reveals little evidence that the Literacy Boost program had positive impacts on equity in this
school. In some cases, inequities widened as evidence suggests that girls and those from richer
home literacy environments benefitted disproportionately from Literacy Boost in some higher-
order skills. However, Literacy Boost did help boys catch up to girls in a few lower-order skills
such as concepts about print and Filipino and English most-used words. Figure 5 below displays
this dynamic for English most-used words.
Excludes two high-performing sections
3.4 1.8 1.4
3.4
0
5
10
15
20
Boys Girls Boys Girls
LB Control
MU
W c
orr
ect
Figure 5: English MUW Baseline and Gain Scores, by Sample Group and Sex
Gain
Baseline
24
During the second phase of Literacy Boost implementation, it will be necessary to
give even more attention to targeting existing inequalities. Suggested activities,
games, and materials can be provided to teachers and community volunteers to
engage disadvantaged groups of children, and perhaps extra remedial help can be
offered to students at-risk of falling behind. This will be very important in order to
make Literacy Boost a more effective program for helping all children in these
Caloocan schools learn to read with comprehension in the early grades of primary
school.
Individual Skill Analysis
This section takes an in-depth look at the results of each sub-test and its constituent items.
Concepts about Print
Average for entire baseline and endline samples shown due to lack of statistically significant differences between
groups. Magnitude of average gain shown at top of graph
The first sub-test of the reading assessment consisted of 10 ‘concepts about print’ (CAP)
questions. These questions concern familiarity with books – where to start, which way to read,
what is a letter, what is a word, etc. Just like the baseline results, students in the sample as a
whole still found CAP questions 10 and 4 the most difficult (‘where is the end of the story?’
And ‘where is the beginning of the story?’). Comparing the baseline and endline results,
students still performed best on CAP questions 2 and 5 (‘open the book’ and ‘show me a
word’). It appears students are generally familiar with the concept of words as well as basic
dynamics of books (where the cover is and how to open them). Students still appear to
experience difficulty in identifying the beginning and end of the story. Both the Literacy Boost
and comparison groups registered a slight decline from their baseline percentage. These
results may indicate that reading activities should include familiarization on parts
-0.4
-2
0
2
4
6
8
10
# o
f C
AP
qu
est
ion
s co
rre
ct
Figure 6: CAP Baseline Score and Gain
Gain
Baseline
25
of a story. The next phase of Literacy Boost implementation should give emphasize
to teaching the basic parts of a story to participants of Reading Camps.
Letter Awareness
Average for entire baseline and endline samples shown due to lack of statistically significant differences between
groups. Magnitude of average gain shown at top of graph
The next sub-test examined students’ letter awareness. Students were shown a chart of all
uppercase and lowercase letters and asked to name the letter, pronounce the letter sound, or
give a word that begins with that letter. Students know the majority of their letters, but only
36% know all 56 upper and lowercase letters, nevertheless this percentage has increased from
its baseline figure of 28%. As shown in Figure 7, both the Literacy Boost and comparison groups
increased by about two and a half letters on average. The group also exemplified a noteworthy
increase in letter identification wherein 95% of the assessed student can recognize at least 80%
of both lower and upper case letters, this compared to the baseline result where one in five
students could not name more than four upper and four lower case letters.
Table 6 below presents the letters that were correctly identified only 90% or less of the time
by students. It appears confusion still exists between the uppercase ‘I’ and the lowercase ‘l’.
Likewise, students still struggle with consonants Ñ and NG in their upper and lowercase forms.
Nevertheless, when compared with the baseline result, the number of less recognized letters
was reduced from six upper case to just two upper case, and six lower case to just four lower
case letters. Finally, except for ‘NG’, all upper case and lower case letters have improved
ratings during endline assessment. The next phase of Literacy Boost implementation
should train teachers and community reading activity facilitators to ensure all
students know all letters of the alphabet specifically the identified letters in Table 6.
Further, reading camp sessions should highlight letter knowledge and phonemic
awareness, and ensure active participation of all students, not just those that know
their letters.
2.3
0
10
20
30
40
50
# o
f le
tte
rs c
orr
ect
Figure 7: Letters Baseline Score and
Gain
Gain
Baseline
26
Table 6: Upper and Lowercase Letters
Identified with 90% Frequency or Less by Students Uppercase Baseline
% Correct
Endline
% Correct
Lowercase Baseline
% Correct
Endline
% Correct
Ñ 54% 65% ñ 64% 67%
NG 83% 77% l 65% 77%
I 88% - q 72% 79%
Q 89% - ng 84% 77%
J 89% - j 88% -
W 90% - f 89% -
Word Recognition: Most Used Words
The most used words (MUW) sub-test consists of a chart of 20 words that the student is asked
to read. These 20 words were identified as ‘most used’ by tabulating the number of times a
word appeared in students’ language arts textbooks. Table 7 shows that among the 4 Filipino
words which students found difficult during the baseline assessment, only one remained below
90% during the endline assessment.
Table 7: Frequency of Correct Reading
of Filipino and English Most Used Words
Filipino Baseline
% Correct
Endline
% Correct
English Baseline
% Correct
Endline
% Correct
pook 82% 86% sentences 43% 40%
anu-ano 85% - these 56% 69%
mga 85% - read 61% 79%
ng 85% - think 62% 75%
words - 80%
For English, students struggled most with sentences, these, read and think for both assessments,
nevertheless as shown in the table, three out of the four words showed improved percentages
during the endline assessment. Furthermore, students were able to correctly read half of the
English words at least 85% of the time compared to none during the baseline assessment.
Finally, the average most used words read correctly improved to 83% from 75% for English and
94% from 70% for Filipino.
27
Average for entire baseline and endline samples shown due to lack of statistically significant differences between
groups. Magnitude of average gain shown at top of graph
Figure 8 above shows that Literacy Boost and comparison students improved at about the same
rate, although they have yet to fully master their reading of words in isolation, especially in
English. Literacy Boost programming should continue providing technical support
and guidance to teachers to help students learn and practice their decoding skills as
well as provide ample opportunity inside and outside of the classroom for students
to practice reading and memorizing basic, frequently used words such as these.
Readers and Nonreaders
After the most used words sub-test, students are then asked to read aloud a passage of
connected text of about 100 words in length. This passage is based on the most used words
and passages found in students’ language arts textbooks. At this point in the assessment,
assessors classify students as either ‘readers’ or ‘nonreaders.’ Readers are defined as students
who were able to read at least 5 words correctly in the first 30 seconds of reading. Figure 9
shows the increase in percentage of Filipino and English Readers from baseline. All other
students were classified as nonreaders, and were read the passage by assessors.
4.8 2.1
0
5
10
15
20
Filipino English
# o
f M
UW
wo
rds
corr
ect
Figure 8: MUW Baseline Score and Gain
Gain
Baseline
28
Average for entire baseline and endline samples shown due to lack of statistically significant differences between
groups. Magnitude of average gain shown at top of graph
Because some students have not yet fully mastered their letters and most-used words, it is not
surprising that all students cannot yet pass the 5 words correct in 30 seconds threshold to
qualify as readers. After addressing remaining gaps in letter knowledge and single-
word reading, it will be necessary to give students ample time to practice reading
through such initiatives as reading buddies in order to build their automaticity.
Fluency and Accuracy
Average for entire baseline and endline samples shown due to lack of statistically significant differences between
groups. Magnitude of average gain shown at top of graph
Fluency (words per minute read correctly) and accuracy (percent of the passage read correctly)
are presented together here because they are measured together in the oral reading passage
11% 21%
0%
20%
40%
60%
80%
100%
Filipino English% o
f sa
mp
le r
ead
ing
at le
ast
5 w
ord
s co
rre
ct in
30
se
con
ds
Figure 9: Proportion of Sample Reading at Least 5 Words Correct in 30 Seconds
(Reader), Baseline and Gain
Gain
Baseline
6.3
8.3
0
10
20
30
40
50
60
70
80
Filipino English
Wo
rds
corr
ect
pe
r m
inu
te
Figure 10: Words Correct per Minute Among Readers, Baseline and Gain
Gain
Baseline
29
sub-test. For those students classified as readers, the number of words students read correctly
in a minute is tracked for fluency. As the student continues to read after the first minute, the
total number of words read correctly from the passage as a whole, no matter how long it takes
the student, is computed for accuracy. This section presents this data for readers only8 in order
to better understand the strengths and weaknesses of students who can read a degree of
connected text.
Average for entire baseline and endline samples shown due to lack of statistically significant differences between
groups. Magnitude of average gain shown at top of graph
Figures 10 and 11 show the average fluency and accuracy for the subset of students defined as
readers (baseline n=205 and 146 of 261 for Filipino and English, respectively; endline n=251 and
217 of 284 for Filipino and English, respectively).9 Baseline and endline results provide similar
trend, wherein fluency and accuracy in reading Filipino and English words increased from its
baseline figures. From speeds of 52-62 words correctly per minute in baseline it increased to
58-71 words per minute. Likewise, the readers’ accuracy improved to 97-90% from its previous
rate of 90-85%. Students could read the English passage with greater speed, but with less
accuracy, than the Filipino passage. There are many possible reasons for this: Filipino words are
longer, students may rely more on sight-reading of English words and more slowly decode
Filipino; or are more familiar with English connected text and read it more confidently but
without native accuracy, or because they apply Filipino pronunciation rules to English, etc. The
next Literacy Boost Assessment should consider re-arranging the order of the
tool’s sections. Reading of English Passage should be administered first instead of
the Filipino passage. This is to test whether reading of relatively longer Filipino
words and passage at the start is a possible cause for fewer words read in a minute.
Nonetheless, continued implementation of Literacy Boost should further enhance 8 As opposed to including the zero scores of the nonreaders, which would lower the average scores for this data. 9 Assessors were instructed to stop the oral reading passage sub-test if the student had failed to read at least five words correctly in 30 seconds. The Tangerine electronic data collection software captures the student’s progress at 60 seconds, so this measure was not recorded for these students and thus their fluency measures were coded as 0. Accuracy scores for these students were deemed to be especially unreliable, and thus also coded as 0.
5% 6%
0%
20%
40%
60%
80%
100%
Filipino English
% o
f te
xt r
ead
co
rre
ctly
Figure 11: Accuracy among Readers, Baseline and Gain
Gain
Baseline
30
readers’ opportunity for practice of reading enjoyable and relevant materials.
Furthermore, the provision of mother-tongue based reading materials will be a
good complement to the implementation of Mother-Tongue-Based Multilingual
Education program of DepEd and its schools - pushing further the improvement of
Filipino fluency of readers in the succeeding years.
Reading Comprehension
After finishing the reading passage, readers were asked the full list of ten comprehension
questions linked to the passage (one summary question, six factual comprehension questions,
two inferential comprehension questions, and one evaluative question). Figure 12 presents
readers’ reading comprehension scores.
Average for entire baseline and endline samples shown due to lack of statistically significant differences between
groups. Magnitude of average gain shown at top of graph
On average, Literacy Boost and comparison students answered around 50 percent of reading
comprehension questions correctly. Increases in scores from baseline to endline was marginal
or even nonexistent in the case of English, one the reason for which is the addition of new
readers (children who did not meet the 5 words correct in 30 seconds threshold at baseline
but did at endline). However, excluding these new readers does not change average endline
reading comprehension scores much – they rise from 5.5 to 5.7 in Filipino and from 5.3 to 5.6
in English.
The following section explores students’ scores by type of comprehension question to suggest
where students need the most help with their reading comprehension. Figures 13 and 14
present readers’ comprehension skills broken down by comprehension question type.10 A
10 For both nonreaders and readers, ten comprehension questions were asked: one summary question (scored correctly if the student recalled at least two of the four story details from the Filipino story or of the three story details from the English story), six non-yes-or-no literal questions, two non-yes-or-no inferential questions, and
0.5 -0.1
-2
0
2
4
6
8
10
Filipino English# o
f re
adin
g co
mp
reh
en
sio
n q
ue
stio
ns
corr
ect
Figure 12: Reading Comprehension among Readers, Baseline and Gains
Gain
Baseline
31
similar pattern presents itself for Filipino and English reading comprehension scores for readers.
Readers were able to improve their scores in at least 2 types of questions at the endline
assessment, nevertheless they still performed the worst in literal and inferential comprehension.
Similar to the baseline result, a larger portion of students still fail to summarize the English
passage than the Filipino passage.
Looking closely at Figure13, Filipino reading comprehension (by type of question) improved by
4 to 8 percentage points from its baseline except for the summary question which recorded a
decrease of 9 percentage points. The same tool was used for the baseline and endline
assessment. In any case, in Filipino it appears students need the most help
understanding the details of the story, and making inferences based on the
information in the story.
Average for entire baseline and endline samples shown due to lack of statistically significant differences between
groups.
For the English reading comprehension (Figure 14), the endline assessment showed nearly the
same performance of students as at baseline. In English, students need help not only
understanding the details and making inferences, but also summarizing the story.
one evaluative question (scored correctly if the student provided information from the text in order to justify his/her opinon).
87%
45%
35%
74% 78%
51%
43%
78%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Summary Literal Inferential Evaluative
Figure 13: Frequency of Filipino Reading Comprehension Questions Answered Correctly
Baseline
Endline
32
Average for entire baseline and endline samples shown due to lack of statistically significant differences between
groups.
Overall, it appears that readers still need to continue on improving their ability to understand
and express details from the text. For the result of literal and inferential questions (both Filipino
and English), when carefully analyzed, most of the students were able to easily answer the
‘what, when and where’ questions, the answers to which can be found directly from the text of
the story. In contrast, ‘how and why’ questions were more difficult for students, which would
require a good understanding of the entire story. Teachers and community volunteers
should focus on asking different types of questions before, during, and after stories
so that students can better develop their literal and deeper comprehension skills.
In terms of assessment, the next Literacy Boost assessment should replace terms
in some of the questions with more familiar words.
64%
48%
56%
77%
67%
45%
58%
75%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Summary Literal Inferential Evaluative
Figure 14: % of English Reading Comprehension Questions Answered Correctly
Baseline
Endline
33
One New Way to Look at Data: Readers with Comprehension
Significant difference from control group at *p<.05.
Students reading with comprehension, with understanding, is the ultimate goal of Literacy
Boost. As such, a new composite measure to focus attention on this goal as well as to track
progress in terms of equity, in terms of all children reading with comprehension, is displayed
above in Figure 15. ‘Reading with comprehension’ is defined as reading a grade-level passage
with such skilled comprehension that the child correctly answers at least 80% of
comprehension questions. To ensure that children who correctly answer this number of
questions have done so as a result of sufficiently skilled reading (as opposed to guessing), the
threshold of ‘reading with comprehension’ is not only defined as the reading with at least 80%
comprehension in addition to scoring either at least one standard deviation below the average
fluency corresponding to 80% comprehension or at least one standard deviation below the
average accuracy corresponding to 80% comprehension.
Figure 15 shows that very few children qualified as ‘readers with comprehension’ at baseline.
However, by endline there was a large increase in readers with comprehension, such that about
a quarter of Literacy Boost students meet this definition in Filipino and about a third do in
English. In English, the LB group gained statistically more readers with comprehension than the
comparison group. We may attribute the Literacy Boost program interventions in
improving the English Reading Comprehension of students.
49%
63%
35%
50%
65%
73%
42%
62%
36%
27%
41%
37%
25%
15%
26%
25%
16%
9%
24%
13%
11%
12%
33%*
14%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Endline
Baseline
Endline
Baseline
Endline
Baseline
Endline
Baseline
Co
mp
aris
on
LBC
om
par
iso
nLB
Filip
ino
Engl
ish
Figure 15: Filipino and English Readers with Comprehension Tiers
Emergent Beginner Reader with Comprehension
34
While this represents an improvement over the baseline, more grade 3 children
should be able to read a grade 2 level text with at least 80% comprehension. Thus,
it will be necessary to modify the Literacy Boost program so that the large number
of children who are able to read with decent speed and accuracy also understand
what they are reading and can interact with the text on a deeper level to be truly
able to read to learn.
Trends in Reading Skill Data from Multilevel Regression Models
This section examines results from a series of standard t-tests and regressions with clustered
standard errors to account for the clustering of children within schools in order to investigate
relationships between reading outcomes on one hand and student background, channels for
reading skill improvement, and intensity of LB participation on the other hand. Appendix D
presents the results of multilevel regression analysis of the impact of Literacy Boost on gain
scores, controlling for a range of variables thought to be unaffected by the Literacy Boost
intervention and clustering standard errors to account for the clustering of children within
schools.
First, in terms of associations with student background, age and number of household members
were negatively correlated with three or more reading skill gains. This means that older
children and those from larger households did not improve their scores as much between
baseline and endline as their younger peers and those students from smaller households. Both
older children and those from larger households may need special remedial help in
order to keep pace with other children.
Additionally, girls often gained more in their reading skills over time than boys, and students
with larger work and chore loads gained more than those with lighter work and chore loads.
For a couple of skills, students with previous ECD experience gained more than those without
such experience, and those with greater body mass index (BMI) gained more than those with
lower BMI. Strangely, there is also a consistent negative association between self-reported
amount of time spent studying and reading skills gains over time. That is, the more time
students report spending on study, the lower their reading skill gains are predicted to be. More
information is required to explain this latter unintuitive finding. It should be noted that these
results are for the sample as a whole, meaning that these findings apply to both Literacy Boost
and comparison students.
LB Reading Activities and Reading Achievement
Another set of multilevel regressions examined reading skill gains and endline scores and the
intensity of participation in Literacy Boost community activities among Literacy Boost students
only. The variables of interest used in these regressions include self-reported frequency of
35
borrowing from Book Banks (# of times usually borrowed from Book Bank per week) and self-
reported frequency of Reading Camp attendance (never, once, rarely, many times, or always).
Whether students created make-and-take materials at reading camps ‘sometimes’ or ‘always’
was also examined.
There were no significant relationships between reported participation and reading skill gains
over time. However, frequency of Reading Camp attendance was positively associated with
gains in letter knowledge and Filipino fluency, as shown in Figure 16 below. As the regressions
used to generate Figure 16 control for a variety of factors including baseline scores, this is
some evidence that Literacy Boost Reading Camps may have helped students
improve some of their reading skills.
Relationship shown for both LB and comparison students
Literacy Boost Home Literacy Environment Channels for Reading Skill
Improvement
Finally, a series of t-tests were run on the baseline-endline sample of children (n=168). These
tests investigated possible ‘channels’ for reading skill improvement – channels such as changes in
children’s literacy habits and HLE. It should be noted that there was no statistically significant
association between Literacy Boost and these potential channels, as shown in Table 8. This
may be due to potential sample contamination between Literacy Boost and
comparison groups, but students are also not reporting improvements in their
household literacy environment and literacy habits in most measures. Literacy
Boost will need to apply lessons from focus groups held with parents to determine
0
5
10
15
20
25
30
Never Once Rarely Many Times Always
Gai
n
Frequency of Reading Camp Participation
Figure 16: Predicted Endline Endline Scores by Reported Reading Camp
Frequency of Participation
Letters
Filipino WPMC
36
what more can be done to help parents better engage in literacy acquisition
promoting activities.
Table 8: Change in Potential Community Channels for Literacy Boost Impact, by Sample
Group
Variable
Change
in LB
Change in
Control Significance
Frequency of study 0.6 0.7
# of types of reading materials at home 0.8 0.4
Learns to read primarily outside school 9% 5%
Exchanged books with others in community in
past week -8% -5%
Read to others in community in past week -1% 8%
% hhold members seen reading -6% -2%
% hhold members encouraging student to study -6% 0%
% hhold members reading to the student -8% -14%
% hhold members telling story to the student 5% -4%
% hhold members singing to the student -4% -8%
% hhold members playing game with the student -4% 2%
Conclusion
This study served to evaluate Save the Children’s first implementation of Literacy Boost in an
urban context, taking the first deep look at children’s home literacy environment, literacy skills,
and the relationship between the two in a large school in the city of Manila. The lessons learned
from this study will serve as an important resource for adapting the Literacy Boost program to
other urban areas in the Philippines and other countries. By analyzing the challenges of a
problematic research design, high attrition, and low community activity participation present in
this pilot study the Literacy Boost program can be enhanced to more effectively help all
students in Manila and other cities learn to read with comprehension.
Recommendations
Based on background statistics and focus group discussions, Literacy Boost programming
should:
redouble its Parental Awareness Workshop efforts in order to improve the
household literacy environment for students
gather more qualitative information to explain the even lower Book Bank
borrowing rates among students
37
expand Reading Camp coverage in terms of geography, scheduling, and
number of volunteers
identify safe Reading Camp venues with adequate facilities
facilitate the provision of as many new titles as possible at Book Banks
better educate all stakeholders that the objective of the Reading Camps is to
provide a more informal and fun environment for children to practice their
reading skills, not an extension of classroom time
encourage parents to help their children with a variety of aspects of reading,
especially in terms of asking comprehension questions
further investigate whether or not messages related to literacy are getting
through to parents, and how these messages can be better conveyed
invest more in advertising Parental Awareness Workshops, and make
accommodations such as a young children’s play area at the workshop so
that those with young children to be able to attend
offer teachers refresher trainings and provide new titles for school libraries
Based on analysis of students’ individual skills, Literacy Boost programming should:
include familiarization on parts of a story to participants of Reading Camps
train teachers and community reading activity facilitators to ensure all
students know all letters of the alphabet especially ñ, ng, l, q, j, & f.
highlight letter knowledge and phonemic awareness during Reading Camp
Sessions, ensuring active participation of all students, not just those that
know their letters
continue providing technical support and guidance to teachers to help
students learn and practice their decoding skills as well as provide ample
opportunity inside and outside of the classroom for students to practice
reading and memorizing basic, frequently used words such as these
give students ample time to practice reading through such initiatives as
reading buddies in order to build their automaticity
further enhance readers’ opportunity for practice of reading enjoyable and
relevant materials
facilitate the provision of mother-tongue based reading materials as a good
complement to the implementation of Mother-Tongue-Based Multilingual
Education program of DepEd and its schools
help teachers and community volunteers focus on asking different types of
questions before, during, and after stories so that students can better
develop their literal and deeper comprehension skills, especially
understanding the details of the story, making inferences based on the
information in the story, and summarizing the story
38
Appendix A. Literacy Boost Activities prior to Endline Assessment
Date of
Intervention / Start
Intervention Number
of days
I. LB Teacher Training
October 27, 2012 1st Teachers training
Topics covered:
1. Introduction to Save the Children
2. What is Literacy Boost?
3. Introduction to Reading Development & Instruction for
Young Children’
4. Vocabulary
5. Comprehension
2 days
November 12, 2012 Read Aloud workshop for all teachers of CZMS as part
of their In-service training
1 whole
day
November 17, 2012
2nd teachers’ training
Review of Vocabulary and Comprehension modules
Formative Assessment
1
December 8, 2012 3rd Teachers’ Training
Phonemic Awareness
Letter Knowledge
1
March 18 Demo-teaching 3 (non-
consecutiv
e;
duration is
2 hours
per
session
April 29-30 4th Teachers’ Training
1. Review of past modules
2. Formative Assessment part 2
3. Reading Fluency
4. Addressing language issues in the literacy classroom
2
II. Community Action
1. Enhancing the Literacy Environment
October 30, 2012 Soft launch of Literacy Boost in CZMS: Read Aloud for all
students
Turn-over of Book Banks to CZMS
Months of October -
November
Book Banks installed all-around barangay 177
2. Reading Camp for children
October 13, 2012 Training of Reading Camp facilitators
Topic: Phonemic Awareness, Read-Aloud
1 day
November 5, 2012 Reading Camp facilitators training 1 whole
day
November 10, 2013 1st Reading Camp held in 5 reading camp sites: 1) Blessed
39
Date of
Intervention / Start
Intervention Number
of days
Mary Day Care Center, 2) Little Star DCC, 3) The Light
DCC, 4) Maria Luisa DCC, 5) Franville DCC
November 17, 2012 Reading Camp facilitators training to cover remaining
Reading Sessions
1 whole
day
November 21, 2012 Reading Month celebration: Reading Camp conducted in
2 separate sections in CZMS
Half day
March 20-21, 2013 Training of new batch of Reading Camp facilitators 2 full days
3. Reading Awareness Workshops for Parents
September 27, 2012 TOT: Reading Awareness Workshops for Parents* 2 full days
October 1-2, 2012 Training of community volunteers for the Reading
Awareness workshops for parents
2 full days
October 6, 2012 Orientation of Day Care Workers on the Literacy Boost
program
- The orientation was more about an introduction
to the program and what their role would be in
the Reading Camps. The DC workers now serve
as the adult support to the youth facilitators of
Reading Camp and they also supervise the lending
of books in the Book Banks.
3 hours
October 30, 2012 Soft launching of Literacy Boost program in CZMS: 3 hours
November 10, 2012 1st Reading Awareness Workshop for parents 2 hours
November 27-28,
2012
Refresher sessions for Parent volunteers of Reading
Awareness Workshops
2 full days
December 15 Year-end review for community volunteers of LB Half day
*Participants of the TOT were former Child Advocate Volunteers of “older” impact areas like Paranaque, Taguig
and Pateros. Ideally, they will eventually train the succeeding volunteers for the Reading Camp and Reading
Awareness workshops for Parents. Unfortunately, this initial design did not happen because of many factors, such
as the distance of the new site (Caloocan) and the time and availability of the trained trainors.
40
Appendix B: Inter-rater Reliability
To test inter-rater reliability, or the consistency with which children’s responses were
recorded the same way between different assessors, 10% of students (n=28) were assessed by
two assessors simultaneously. Long one-way analysis of variance (ANOVA) techniques were
used to calculate the intra-class correlation within pairs of assessors for a measure of reliability.
Table B presents the results below. Using Fleiss’ benchmarks for excellent (ICC>0.75), good or
fair (0.75>=ICCA>0.4), and poor (0.4>=ICC).11
Table B: Inter-rater reliability by sub-test
Sub-test Inter-rater
Reliability (ICC)
Reliability Rating
Concepts about Print .90 Excellent
Letters .99 Excellent
Filipino MUW .24 Poor
English MUW .99 Excellent
Filipino Reader or Nonreader 1.0 Excellent
English Reader or Nonreader 1.0 Excellent
Filipino Fluency .33 Poor
English Fluency .65 Good/Fair
Filipino Accuracy .88 Excellent
English Accuracy .92 Excellent
Filipino Reading Comprehension .93 Excellent
English Reading Comprehension .96 Excellent
Filipino Listening Comprehension .96 Excellent
English Listening Comprehension .90 Excellent
The inter-rater reliability for most sub-tests falls within the excellent range, with
the exception of English fluency (good/fair) and Filipino fluency and most-used
words (poor). Thus, while assessors were consistent in their interpretation of
students’ answers overall, there may be added noise in the data for Filipino fluency
and most-used words and English fluency, which will make it more difficult to
detect relationships with other variables. However, as assessed students were
randomly allocated among assessors, the distribution of this noise should not be
systematically related to students’ scores and thus not bias the average score for
these sub-tests. In the future, more inter-rater reliability exercises should be
undertaken for scoring Filipino most-used words and fluency in both languages.
11 Fleiss JL. The Design and Analysis of Clinical Experiments. New York, NY: John Wiley & Sons Inc;1986 .
41
Appendix C: Student Background Characteristics
Table C.1: Relative Amount of Time Reported Spent on Work/Chores,
Study, and Play
Before school After school On weekends Altogether N
Work 1.1 1.1 1.4 3.6 283
Study 1.0 1.2 1.2 3.4 283
Play 0.6 0.7 1.2 2.5 283
Averages for entire endline sample shown due to lack of significant differences between LB
and comparison groups
Scale: 0 = no time, 1 = a little time/less than 2 hours, 2 = a long time/2 hours or more
Table C.2: How Children Report Spending Their Time
Outside of School
Proportion N
Housework 88% 283
Caring for children 47% 283
Working for money 4% 283
Playing computer games 20% 283
Other types of play 71% 283
Homework 76% 283
Other types of study 86% 283
Averages for entire endline sample shown due to lack of
significant differences between LB and comparison groups
Table C.3: Heath Indicators by Sample Group (excluding high-performing sections)
Baseline Endline
Variable LB Control Sig. N LB Control Sig. N
Size of breakfast 1.8 2.1 * 142 2.2 2.4
144
Had eaten within last 4 hours 86% 81%
123 74% 70%
128
Ill on day of assessment 9% 13%
97 1% 7%
113
In pain on day of assessment 14% 8%
140 22% 11% ~ 140
Reports safe water purification method 35% 26%
144 19% 17%
144
Height in centimeters 119.1 120.1
141 121.9 121.5
144
Weight in kilograms 20.7 21.3
141 23.9 23.8
143
42
BMI 14.5 14.6 141 16.0 16.1 143
43
Appendix D: Multilevel Regression Analysis
(1) (2) (3) (4) (5) (6) (7) (8)
VARIABLES CAP Gain Letter Gain
Filipino MUW Gain
English MUW Gain
Filipino Reader Gain
English Reader Gain
Filipino Fluency Gain
English Fluency Gain
Literacy Boost 0.207 0.995 0.609 0.371 -0.00550 0.0142 2.943 1.230
Sex (1=Female) 0.468 2.045* 0.965 0.932 0.108* 0.0627 5.043 2.325
Endline SES quintiles 0.273 0.0552 -0.220 -0.132 -0.000136 0.0231 0.248 2.451
Baseline HLE quintiles 0.0670 -0.364 -0.134 0.0926 -0.0190 0.0266 -0.981 0.662
Endline chore/work load 0.481* 0.246 0.0305 0.237 0.00223 0.0518** 2.454 2.801~
Attended ECD/Preschool -0.072 0.0580 -0.618 0.547 -0.0907 0.135* -5.947 3.172
Age in Years -0.263~ -0.410 -1.021* -0.787* -0.0122 -0.0344 -0.796 -2.111
Number of Household Members -0.470* -0.179 -0.253 -0.350 0.00950 -0.0397~ -1.593 -1.962
Has Repeated a Grade 0.167 -0.905 0.379 0.341 -0.0613 -0.106 -8.831 -6.814
Frequency of study -0.5*** -0.436 -0.127 -0.491 -0.0210 -0.0717* -4.947* -5.307*
Frequency of play 0.0398 -0.0278 0.133 0.159 -0.00699 -0.0131 -0.837 -0.890
Baseline Child Literacy Habits -0.213 -0.0890 -0.125 -0.302 -0.0397 0.00733 0.197 3.453
Size of Breakfast 0.534** -0.0076 -0.478 -0.343 -0.0482 0.00495 0.0423 1.831
Body Mass Index 0.0400 0.320* 0.0965 0.0439 0.0182~ -0.00686 -0.312 -1.404
Baseline Score -0.7*** -0.68*** -0.899*** -0.579*** -0.604*** -0.520*** -0.494*** -0.283***
Constant 5.914* 36.3*** 26.83*** 18.14** 0.654* 0.882* 64.27* 68.67**
Observations 138 138 138 138 137 138 137 138
R-squared 0.536 0.817 0.821 0.494 0.506 0.430 0.334 0.243
Robust standard errors in parentheses *** p<0.001, ** p<0.01, * p<0.05, ~ p<0.1
44
Appendix D: Multilevel Regression Analysis (Continued)
(9) (10) (11) (12) (13) (14)
VARIABLES Filipino
Accuracy Gain English
Accuracy Gain Filipino Reading
Comprehension Gain English Reading
Comprehension Gain Filipino
RWC Gain English
RWC Gain
Literacy Boost 0.0241 0.00479 0.0177 0.729 0.0441 0.168*
Sex (1=Female) 0.0976~ 0.0712 1.026* 0.678~ 0.0626 0.0723
Endline SES quintiles -0.00328 0.0253 0.0899 0.147 -0.0123 0.0227
Baseline HLE quintiles -0.0107 0.0213 0.0706 0.107 0.0364 -0.00175
Endline chore/work load 0.000452 0.0469** 0.262 0.347* -0.00674 -0.000721
Attended ECD/Preschool -0.0289 0.107* -0.00318 0.260 0.0305 0.0676
Age in Years -0.0141 -0.0362 -0.275 -0.289 -0.0211 -0.0517
Number of Household Members -0.00475 -0.0392~ -0.353* -0.714** -0.0619~ -0.0651*
Has Repeated a Grade -0.0714 -0.0802 0.443 -0.447 0.0233 0.120
Frequency of study -0.0151 -0.0635* -0.721** -0.750** -0.108** -0.0658**
Frequency of play -0.0108 -0.000325 0.249~ -0.323* 0.0111 -0.0328
Baseline Child Literacy Habits -0.0464 0.0103 -0.478 -0.112 -0.0824 -0.00156
Size of Breakfast -0.0344 0.0252 -0.0650 -0.215 -0.00762 -0.00896
Body Mass Index 0.0156 -0.00409 0.00121 0.0390 -0.00224 -0.00333
Baseline Score -0.639*** -0.466*** -0.524*** -0.640*** -0.703*** -0.900***
Constant 0.685* 0.722** 6.788* 8.079* 0.825 0.930~
Observations 138 138 138 138 138 138
R-squared 0.522 0.397 0.400 0.526 0.379 0.529
Robust standard errors in parentheses *** p<0.001, ** p<0.01, * p<0.05, ~ p<0.1
45
Appendix E: Details of Focus Group Discussions (FGDs) with Primary
Stakeholders
Literacy Boost Program
Period Covered: 05 to 09 August 2013
Area: Barangay 177, Caloocan City
INTRODUCTION
Following its first year of implementation, Save the Children’s Literacy Boost Program conducted an Assessment that
included a series of FGDs carried out from 05 to 09 August 2013. Target groups for this activity included partner school
Cielito Zamora Memorial School pupils from Grades 2 to 4 and teachers who participated in the Literacy Boost training
for teachers. Likewise, community members, specifically Reading Camp Facilitators and parents of children
participating in Reading Camps were made part of this undertaking.
A total of 7 FGDs, with a minimum of 5 and a maximum of 8 participants, and a key informant interview (KII) with the
Literacy Boost Focal Person from Cielito Zamora Memorial School were completed. Number of participants per FGD are
listed below:
Date Conducted FGD Groups # of
Participants
05 Aug; AM 1. Grade 2 Pupils 5
06 Aug; PM 2. Grade 3 Pupils 8
05 Aug; AM 3. Grade 4 Pupils 6
06 Aug; AM 4. Teachers who participated in the Literacy Boost training
sessions
Grade 1 4
Grade 3 4
09 Aug; PM 5. Parents 2
09 Aug; PM 6. Reading Camp Facilitators (children and youth) 6
09 Aug; AM 7. Reading Camp Facilitators (adults) 7
Since there were only two (2) parents who took part in the FGD intended for them, another session is being organized
to complete the information that they can share with the Program. Difficulties were encountered at setting a date as
there are classes during the weekdays where parents are busy with their tasks at home. More so, the nearest Saturday,
17 August 2013, was celebration of the barangay fiesta. However, this FGD will still be scheduled and completed.
Conducted by:
ELVIE JO C. ROBIS
Date: 21 August 2013