leveling the playing field: strategies for schools …...leveling the playing field: strategies for...
TRANSCRIPT
Leveling the Playing Field:
Strategies for Schools Serving At-Risk Students in Elementary Schools in Massachusetts
by
Lucy Boyd, Amy Chandran, and Jonathan Hui
Taubman Center for State and Local Government
June 2016
Taubman Center Working Paper
WP –2016 – 02
PolicyAnalysisExercise
LevelingthePlayingField:
StrategiesforSchoolsServingAt-RiskStudentsin
ElementarySchoolsinMassachusettsAnalysisprovidedfortheMassachusettsDepartmentofElementary
andSecondaryEducation
LucyBoyd,AmyChandran,JonathanHui CandidatesforthedegreeofMasterinPublicPolicy,May2016.
Advisor:ProfessorJoshuaGoodman
ThisPAEissubmittedinpartialfulfillmentoftherequirementsforthedegreeofMasterin
PublicPolicy.Itreflectstheviewsoftheauthor(s)andshouldnotbeviewedasrepresenting
theviewsofthePAE'sexternalclient(s),northoseofHarvardUniversityoranyofitsfaculty.
2
Acknowledgements
First,wewouldliketothankProfessorJoshuaGoodmanattheHarvardKennedySchoolforhis
directionandsupport.Heprovidedinvaluableguidancethroughthedevelopmentoftheproject,
theanalysisandthecreationofthefinalproduct.Histechnicalexpertiseinempiricalmethodsand
research,aswellashistopicalexpertiseineducationreformmadethisprojectpossible.
Second,wewouldliketothankCarrieConawayandKendraWinnerattheMassachusetts
DepartmentofElementaryandSecondaryEducationfortheirsupportthroughoutthisproject.They
providedtechnicalsupportandadvice,andgrantedusaccesstoDESEdatathatenabledour
researchandanalysis.
Moreover,wewouldliketothanktheadministrationandteachersoftheHyannisWestElementary
School(Barnstable),SumnerSchool(Boston),CarltonM.ViveirosElementarySchool(FallRiver),
NewtonSchool(Greenfield),theMaryM.LynchSchool,KensingtonInternationalSchool,the
WarnerSchool(allSpringfield)andtheBelmontStreetCommunitySchool(Worcester)for
participatinginthisstudy.Theprovidedinvaluableinsightintothestrategiesusedtoimplement
bestpracticestoserveat-riskstudentsinhigh-performingschools.
Furthermore,wewouldliketothankourpeersintheSocialandUrbanPolicySeminaratthe
HarvardKennedySchoolfortheirconstantfeedbackandinsightintoourmethodologyandwriting.
Finally,wewouldliketothankourfamiliesandpeersfortheirunwaveringsupportthroughthis
project.Yourpatience,supportandencouragementwereverymuchappreciated.
3
TableofContents
Acknowledgements..............................................................................................................................2
I.ExecutiveSummary..........................................................................................................................4
II.Context................................................................................................................................................6
III.ExistingEvidenceonSupportingAt-RiskStudents............................................................7
IV.Methodology....................................................................................................................................8
V.FindingsfromAnalysis..................................................................................................................91. DataDrivenInstruction......................................................................................................................92. Wraparoundservices.........................................................................................................................103. Behavior&DisciplineManagement..............................................................................................114. AcademicInterventionsandEnrichment...................................................................................125. TeacherCulture,TrainingandEmpowerment.........................................................................136. ParentalEngagement.........................................................................................................................157. CultureofAttendance........................................................................................................................168. MonitoringSiteVisitRubrics...........................................................................................................17
VI.Recommendations......................................................................................................................18Recommendation1:EmbedComprehensiveandCohesiveSystems..........................................191.1.DataDrivenInstruction................................................................................................................................191.2.BehaviorManagement..................................................................................................................................201.3.WraparoundServices....................................................................................................................................211.4.AcademicInterventionsandEnrichment..........................................................................................22
Recommendation2:Createstrategiestocultivateapositiveschoolculture..........................242.1.TeacherTraining,CultureandEmpowerment...............................................................................242.2.ParentEngagement.........................................................................................................................................262.3.CultureofAttendance....................................................................................................................................27
Recommendation3:ImproveMonitoringSiteVisitRubric...........................................................283.1.AddAdditionalContinuumPointstoPortionsofMSVRubric...............................................283.2.AddAdditionalIndicatorsforAttendanceStrategies.................................................................29
FrameworkforImplementation..................................................................................................30
VII.Conclusion....................................................................................................................................32
AppendixI:References....................................................................................................................33
AppendixII:DetailedLiteratureReview...................................................................................35
AppendixIII:MethodologyforQuantitativeAnalysis...........................................................47StageI:SchoolMatching.............................................................................................................................47StageII:FindingsfromQuantitativeAnalysisandHypothesisDevelopment..........................47StageIII:CaseStudies..................................................................................................................................49StageIV:AnalysisofMonitoringSiteVisits..........................................................................................50Limitationsofthestudy..............................................................................................................................50
AppendixIV:SchoolSample...........................................................................................................52
AppendixV:SelectionofIndicatorsforStatisticalAnalysis...............................................55SelectedIndicators.......................................................................................................................................55OmittedIndicators.......................................................................................................................................56
AppendixVII:InterviewProtocol................................................................................................58
AppendixVII:StakeholderConsiderations...............................................................................61
4
I.ExecutiveSummaryDistrict and school leaders struggle to overcome the educational challenges associated with the
demographic factors of race and poverty.While these demographic characteristics have become
commonpredictorsofstudentachievement,thisneednotbethecase.SomeMassachusettspublic
schoolsachievehigheducationaloutcomeswiththesametypesofhigh-riskstudents–how?
Thisreportthereforeasks:
What strategies can be implemented in low-performing elementary schools in
Massachusetts serving high-risk students to improve student and school-wide
outcomes?
By focusing inonhigh-performingLevel1and2 (Level1/2)schools inMassachusetts that serve
studentbodieswithademographicmake-upsimilartolowerperformingLevel4and5(Level4/5)
schools, we have identified common practices that improve student performance. This report
outlinesandanalyzesthesebestpracticesasutilizedbyelementaryschoolswithlargepopulations
ofminority and socio-economically disadvantaged students.Whilemany low-performing schools
may already be aware of some of these practices, our recommendations focus on their
implementationandoperationalization.Basedonthisstudyandanalysis,weproposeadoptionof
threemajorrecommendations:
1. Embedcomprehensiveandcohesivesystemsintodailyschooloperationsthat focus
onstudentperformance.
2. Createstrategiestocultivateapositiveschoolculture.
3. Adddata points for these best practices toMonitoring Site Visit Rubrics for school
evaluations.
The first two recommendations are broken down into seven total action steps with detailed
descriptions of school-level implementation and examples. These recommendations seek to go
beyondexistingliteraturebydescribinghowtooperationalizethesebestpracticesasexemplified
byhigh-performingschools.
The third recommendationdetails specific improvements that canbemade to theDepartmentof
ElementaryandSecondaryEducation(DESE)’sMonitoringSiteVisit(MSV)Rubricthatisusedfor
evaluating Level 4 and 5 schools. The improvements align the rubric with the best practices
describedinthefirsttworecommendations.
Throughtheserecommendations,thisreportisintendedtoassistDESEtobettersupportandassist
low-performing schools that are struggling to meet ambitious yearly targets of achieving 100%
proficiencyinMathandEnglishtestsby2016-2017.
OverviewoftheRecommendations:
Inanalyzing implementationofthesebestpractices, twoaspectsofschooloperationsemergedas
core to the effectiveness of the educational environment: systems and culture. High-performing
schools consistently implement systems around data driven instruction, behavior management,
wraparound services, and academic interventions and enrichment opportunities. Secondly, these
5
schools work deliberately to create a positive school culture, particularly in terms of teacher
training and collaboration, parental engagement, and student attendance, leading to teacher
empowerment and higher teacher retention. A summary overview of these specific
recommendationsforschoolsisprovidedinFigure1below.
Furthermore,thelastrecommendationaimstoimproveDESE’sabilitytoevaluateandsupportlow
performing schools when implementing these recommendations.We recommend DESE improve
theMonitoringSiteVisit rubricby:1)addingadditionalcontinuumpoints to theMSVrubric that
reflectspecificbestpracticesand2)addinganadditionalrubricrowaroundstudentabsences.
6
II.Context
This report investigates Level 1 and 2 schools serving students fromminority and economically
disadvantaged backgrounds to develop an understanding of the strategies schools can use to
overcome demographic challenges associated with student success. It is provided to the
Massachusetts Department of Elementary and Secondary Education (DESE), which has
responsibilityforreviewinglevelsandaddressingissuesthatemergeinunder-performingschools.
Morespecifically,itseekstosupporttheworkoftheDESEOfficeofPlanningandResearch,which
providesanalysis,research,andtoolstoinformdecision-makingandsupportschoolimprovement.
WeusedDESE’sLevelingSystem to identifyhighand lowperforming schools.TheLevel System,
reviewed annually, takes into account absolute test scores, fluctuations and growth in these test
scores,andevidenceofwhethergapsbetween lowandhighperformingstudentswithinaschool
have increasedordecreased.Aschool isclassifiedasLevel1 if it ismeetinggap-narrowinggoals
either in absolute performance (weighted at 75%) and growth (weighted at 25%) for all sub-
groups of students. The top 80% of schools are categorized as Level 1 or Level 2, while the
remaining20%arecategorizedasLevel3,4,or5schools.ThelowestperformingLevel3schools
aredesignatedasLevel4schoolsiftheyremaininthelowestperforminggroupforaperiodoffour
or more years. Designation as a Level 5 school is rare, and reserved for the consistently low
performingschoolsthatshownosignsofimprovement,asdeterminedbytheBoardofElementary
andSecondaryEducation.
Toanswerourresearchquestion,welookedatschoolscategorizedasLevel4or5andfoundLevel
1and2schoolswithsimilarstudentdemographics(seeFigure2).Weprioritizedtheproportionof
studentsfromeconomicallydisadvantagedandminoritybackgroundsforthiscomparisonbecause
ofthestrongassociationsbetweenpoverty,race,andlowstudentachievement.
Toassessthecurrent
practicesinplaceatLevel4
schools,weusedexisting
datafromtheMonitoring
SiteVisits(MSV),whichare
stateorderedobservations
atLevel4schoolsasthey
implementaturnaround
plan.Thisrubricincludes
thefollowingfour
turnaroundfocusareas:1)
Leadership,shared
responsibility,and
professionalcollaboration
2)Intentionalpracticesfor
improvinginstruction3)Student-specificsupportsandinstructiontoallstudentsand4)School
cultureandclimate.BasedontrendsintheMSVrubricdata,weidentifiedgapsinstrategiesused
betweenLevel4andLevel1and2schools.Thisallowedustofocusouranalysisand
recommendationsonstrategiesLevel4schoolsarenotalreadyusing.
Figure2:DemographicDataofSampledLevel1/2Schools
SchoolName %ofStudents
Econ.Disadv. Minority
CharlesSumnerSchool,Boston 57.7% 85.9%
CarltonM.ViveirosSchool,FallRiver 68.9% 32.6%
HyannisWestSchool,Barnstable 59.4% 38.4%
NewtonSchool,Greenfield 63.6% 21.9%
KensingtonSchool,Springfield 76.4% 82.7%
MaryM.LynchSchool,Springfield 77.5% 88%
WarnerSchool,Springfield 67% 78.4%
BelmontStreetSchool,Worcester 71.5% 67%
7
III.ExistingEvidenceonSupportingAt-RiskStudents
Thereisalargebodyofquantitativeandqualitativeacademicliteratureregardingbestpracticesto
support student development during elementary school. However, since much of the literature
focusesongeneralstudentpopulationsandnotourspecificsub-populationofat-riskstudents,itis
oflimitedvalueindeterminingparticularstrategiesfortheschoolsinfocusinthisreport.Inorder
toaddressthisresearchgap,thisstudycombinesexistingacademicliteraturewithoriginalanalysis
ofschool-leveldatainMassachusetts.
Overall, existing literature (seeAppendix II) on supporting at-risk students suggests that school-
level, student-centered systems are essential to improving student achievement. Systems that
increase instructional time, increase use of student-level data, and enhance human capital
developmentall contribute to improvedstudentoutcomes.Within theareaofhumancapital, the
literature also illustrates that increased accountability and training, coupled with intrinsic
incentives (such as empowerment) and extrinsic incentives (such as increased pay), improves
teacherretentionandquality.
Studies have also shown that provision of extensive social-emotional and non-academic
interventions are associated with student achievement. Students in wraparound zones, for
example, see significant improvements in test scores, and teaching students leadership skills and
providing them leadership opportunities could yield long-term benefits. Furthermore, school
culture has also been shown to be highly influential. For example, consistent reinforcement of
positivebehaviorisassociatedwithstudentgrowth.Lastly,evenexternalfactorsofculture(outside
thedirectschoolenvironment)cancontributetostudentlearning.Forinstance,networkingevents
oractivitiesinthecommunitythatimprovesocialcapitalofparentscouldhavepositiveeffectson
studentoutcomes.
Existingliteraturefurtherpointstoseveralfactorsthatareoutsideofthescopeofourresearchand
willthereforenotbeaddressedinthisreport.Forexample,thereisextensiveresearchontherole
of leadership development and career preparation on students’ college and career success, but
since this report focuses on school performance only, the impact on college and workplace
performance will not be addressed. Lastly, existing studies also highlight the importance of
financial incentives and resources such as pay structures or resource allocation. However, since
these are district-level decisions, they are outside of the scope of this report, which focuses on
school-levelsystems.
AfullsummaryoftheacademicliteratureconsideredforthisstudyisincludedinAppendixII.This
summary is focusedon studies concernedwith best practices at the school-level rather than the
community-levelordistrict-level,asthiswasmostrelevanttoourresearchquestion.
8
IV.Methodology
Our research involved four stages described below. A full description of our methodology is
includedinAppendixIII.
Stage I: SchoolMatching:To effectively compare high-performing and low-performing schools
withsimilardemographicmake-up(proportionofstudentsconsideredeconomicallydisadvantaged
or from a minority background), we identified thresholds across these indicators. Using these
thresholds,weidentifiedLevel1and2schoolswithsimilardemographicstoLevel4and5schools.
The thresholds establishedmeant that at least53%of a school’s studentbodywaseconomically
disadvantagedor55%camefromminoritybackgrounds.
StageII:QuantitativeAnalysis&HypothesisDevelopment:UsingschooldatacapturedbyDESE
inSchoolandDistrictProfiles,weranregressions todeterminewhat indicatorswererelevant to
schoolperformance.Wetestedvariablessuchasprincipalretention,teacherretention,numberof
studentsdisciplined,percentageof staff evaluated, andvariousothers, toestablishwhich school-
level characteristics significantly and meaningfully correlated with greater success among
Massachusetts Elementary Schools. Controlling for those variables, including student
demographics,thetwoschool-levelcharacteristicsthatstoodoutwerestudentattendancerateand
teacher retention rate. Using this analysis and existing academic literature, we formulated
hypothesesregardingthecontributionofthesetwocharacteristicstohigherschoolperformanceto
testinStageIII.
Stage III: Interviews, FieldVisits, andQualitativeAnalysis:We invited Level 1 and 2 schools
above our two demographic thresholds to participate in our study. We conducted interviews
and/or visits with principals, teaching staff, administrative staff and specialty teachers in eight
Level1and2schools,andincertainschoolswereabletoviewclassroomoperations.Thisallowed
us to identify and investigate specific best practices common across these high-performing
elementaryschools.
StageIV:AnalysisofMonitoringSiteVisitReports:Oncethesebestpracticeswerecompiled,we
analyzed Monitoring Site Visit (MSV) reports to identify gaps in the implementation of these
practices inLevel 4 schools as comparedwithLevel 1 and2 schools. In comparingour findings
with detailed analysis ofMSV reportswe refined our understanding of key differences between
highandlowperformingschoolsservinghigh-riskpopulationsandtailoredourrecommendations
toaddressthesegaps.
StageI:School
Matching
StageII:Quantitative
Analysis&
Hypothesis
Development
StageIII:Interviews,
FieldVisits
and
Qualitative
Analysis
StageIV:Analysisof
MSV
reports
9
V.FindingsfromAnalysisThissectionpresentsasummaryofthefindingsfromacrossourcasestudies,MSVanalysisand
literaturereview.Thesefindingsformedthebasisforformulationofspecificrecommendations.
Thisanalysisfoundthatwhilelow-performingschoolsgenerallyhaveasenseofwhatworkswell
forhigh-riskstudents,high-performingschoolsaremuchmoreintentionalandstrategic.Our
findingsareorganizedbelowbyfirstexplainingMSVreportexpectationsforthelistedbestpractice
followedbyadescriptionofthegapwithLevel1and2schools.
1. DataDrivenInstruction
Data usage in instruction falls within Turnaround Practice 2: Intentional Practices Improving
Instruction in theMSVrubric. Inorder toachieve thehighest “Sustaining” rating in indicator2.5:
StudentAssessmentDataUse,DESE requires teachersand leaders to consistentlyusebenchmarks
andstateassessmentstomakeschool-widedecisionsregardingschool-widepractices.Relatedly,in
order to achieve the highest “Sustaining” rating in indicator 2.6: Teacher Progress Assessment
Practices, DESE requires teachers and leaders to work collaboratively using a variety of
assessmentstodeterminestudentprogresstowardsintendedoutcomes.
InLevel1/2schoolinterviews,teachersdescribedusingdatain
allcollaborativemeetings,usuallyheldonceortwiceperweek.
At Viveiros in Fall River, teachers analyze the data of the
studentsstrugglingmost(Tier3)weeklyandtwiceamonthin
literacyandmath.AtKensingtonElementary,teachersuseunit
tests and a re-teaching cycle based on trends in student
performance on specific questions. At Warner Elementary,
teachers look at data collaboratively twice a week and administer unit tests followed by a re-
teachingcycle.AtMaryLynchElementary,theschoolreliesonA-net,aneducationalnon-profit,to
provideassessmentsalignedwithstatestandards.Atallotherschoolsinterviewed,similarpatterns
of consistent, frequent and responsive use of data built into the teaching cyclewere evident. In
severalinterviews,teachersalsoreferencedusinginformaldata,suchasgradedhomeworkorexit
tickets,onadailybasistoassessifstudentsareprogressingtowardsteachingobjectives.
MSV reports indicate that Level 4 schools often provide collaborative time for teachers to plan
usingdata.Inaddition,theysuggestleadersusestateassessmentstomakeschool-widedecisionsat
thebeginningoftheyear.Level4schoolsaretypicallyratedas“providing”evidenceofdatadriven
instruction, but not necessarily consistently. One report noted, “Coacheswere looking at data to
make decisions on school-wide practices. However, not all staff examined data to inform school
practices.”Thissuggestsinconsistentschool-wideuseofdata.Anotherreporthighlighted,“…School
leadersuseddatatoidentifyvocabularyasakeyareaforimprovementthisyear.However,itisnot
clearfromthedatathatschoolleadersconsistentlyusestudentresultsonbenchmarkandcommon
assessments—in addition to state assessment data…” Several otherMSVs report that the school
was “providing” evidence of using state assessments but not other benchmarks throughout the
year,muchlessonaweeklybasislikeLevel1and2schoolsinterviewed.
AtViveirosinFallRiver,
teachersanalyzedataof
theirTier3students
weekly,andtherestof
thestudentstwiceper
month.
10
2. Wraparoundservices
Provision of coordinated wraparound services falls
withinTurnaroundPractice4:SchoolClimateandCulture.
In order to achieve the highest rating of “Sustaining”,
DESE requires that school leaders and staff share
“individual and mutual responsibility” for providing
students with comprehensive social-emotional services.
This includes using a systemic approach, involving the
assessment of student and family needs throughout the
year. In Level 1 and 2 schools, these systems are often
coordinated through a single point of contact, responsible for the referral and monitoring of
students receiving external services. In Springfield, this is the responsibility of the district-wide
Student-TeacherAssistanceProgram.Inotherschools,WrapAroundZone(WAZ)servicesareoften
coordinatedthroughoneschoolcounselorordean.
In2012,DESEconductedastudyontheeffectofWAZsonstudentperformance.Thesearestate-
funded non-academic supports that address the climate and culture of the school, as well as
students’ social and emotional capacity. It found significant improvements in standardized test
scores,especiallyforthirdandfourthgraders,butnoimprovementinsuspensionrates,attendance,
or retention (DESE, 2012). Therefore, it concluded that WAZs improve immediate academic
performance for students facing social or emotional challenges, but their long-term effects on
behaviorarelessclear.OurownresearchshowsthatschoolswithcoordinatedandeffectiveWAZ
systemshavehigherstudentattendancerates,lowersuspensionrates,andhigherretentionrates.
These findings suggest WAZ services can have strong effects, particularly when implemented
alongsideotherstudentsupportsystems.
AcademicliteraturereviewsalsosuggestthatWAZservicescouldmakeasignificantdifferenceto
theeducationalachievementofat-riskstudents.Forexample,astudyconductedinChicagoPublic
Schoolsdemonstratedanassociationbetweenoutsidefactors,suchasstrongersocialnetworksfor
parents or decreased crime rates, and improved student achievement and attendance. This and
other studies provide evidence that student outcomes are not only the result of internal school
organization, but external community factors as well. Wraparound Services that provide
coordinated and effective service provision to address the social, emotional, and behavioral
repercussionsofnegativeexternalfactorscouldthereforeimpactstudentachievementandschool
culture.
MSV reports indicate thatwhile low-performing schools oftenprovide amenuof comprehensive
services for students and families, there is no cohesive system that refers students, tracks their
progress,andevaluatestheeffectivenessoftheseservices.Onereportindicatesthat“therewasno
evidence that school leaders and staff tookpart in identifying family needs or services”; another
notes that “such services seem to be limited to some counseling services, but these services are
neitherwidespreadnor systemic.” Indeed,whilemanyLevel4/5 schoolsprovideaccess to these
services,theyareoftennotcoordinatedorlimitedtoone-offindividualreferrals.
InmanyLevel1/2schools,
thesesystemsare
coordinatedthroughasingle
pointofcontact,whois
responsibleforthereferral
andmonitoringofstudents
receivingexternalservices.
11
3. Behavior&DisciplineManagement
Implementationof a school-widebehaviorplan fallswithinTurnaroundPractice4:SchoolClimate
andCulture.DESErequiresfourkeyelementsbeincludedinaschool-widebehaviorplan:1)asetof
highbehavioralexpectationsaredefined;2)asystemofpositivebehaviorsupportsisdeveloped;3)
proceduresareimplementedbyamajorityofthestaff;and4)dataisconsistentlyusedtomonitor
behavior. Our case studies reveal that high-performing schools fulfill these criteria by
demonstrating a coherent system of high expectations while still providing teachers with
autonomy.
While our quantitative analysis indicates the relationship between student discipline rate and
schoolLevelwasnotstatisticallysignificant,themagnitudeofcorrelationwasstilllarge.Amere1
percentincreaseintherateofstudentsreceivingdisciplinaryactioncorrelatedwithadecreasein
the probability that a school is high-performing by 0.80 percentage points (see Appendix III for
empiricaldetails).This suggests thathigherperformingschools tend to take lessdrasticpunitive
measures such as suspensions, and based on our interviews, likely focus more on in-school
discipline systems thatbuildapositive culture.This is further substantiatedbyexisting research
and consistent qualitative findings on behavior management from interviews that indicate
behavioralsystemstobeanimportantfactor.
A2014studyexaminingtheimpactofinstillingacultureofhighbehavioralexpectationsintofailing
publicschoolstestedwhetherthesehighexpectationsproducedharderworkingandmorefocused
students. The schools that implemented these practices increasedmath achievement by 0.15 to
0.18 standard deviations per year. This suggests that implementation of a culture of high
expectationscouldclosetheblack-whiteachievementgapinmathwithinthreeyears.However,the
effects on readingwere onlymarginal and statistically insignificant (Fryer, 2014). Other studies
havealsoshownthatdevelopingconsistentschool-widestructuresforpositivebehaviorsupports
contributetoimprovedstudentbehavior.
Our interviews with high-performing schools also indicate the importance of positive
reinforcement and providing students with a clear set of expectations. Many of the schools use
tiered behavior models to provide this clarity and systematic approach. Some schools run a
‘responsive classroom’model topositively communicateexpectations. Inaddition, all the schools
evidencepositivefeedbacksystemstorewardorrecognizegoodorkindbehavior.
Analysis of theMSV reports indicate that Level 4 schools generally understand best practices in
behaviormanagementandoftenoutlinehighexpectationsforstudents.Moreover,manyLevel4/5
schoolsattempttoimplementpositivebehaviorsupports.Whiletheseschoolscanoftenarticulate
what behavioral interventions are needed, implementation is generally inconsistent and lacks
structure.Forexample,oneMSVreportnotes,“StakeholdersstatedthattheschoolusedPBISforits
behaviorplan…However,thereisnoevidenceofhowconsistentlytheplanisbeingimplementedor
monitored.”Indeed,aconstantthemeacrossMSVreportsisthatschoolsknowwhattodo,butfail
toimplementaschool-widestructuretoreinforcebehavioralexpectationsforstudents.
Interviewswithhigh-performingschoolsalsoindicatedtheimportanceofpositive
reinforcementandprovidingstudentswithaclearsetofexpectations.
12
4. AcademicInterventionsandEnrichment
Another aspect ofwherehigh and lowperforming schools differ is theprovisionof tutoring and
enrichment opportunities for students. These include Extended Learning Time, such as tutoring
beforeandafterschool,aswellasactivitiesinnon-coreacademicareas.ExtendedLearningTimeis
addressedinTurnaroundPractice4:wherea“Sustaining”ratingrequires,“Allstudentshaveaccess
to expanded learning opportunities that are well defined and well supported, and high-needs
students are targeted for participation in these programs.” The availability of tutoring is also
alludedtoinTurnaroundPractice3undertheindicatorofaMulti-tieredSystemofSupport:wherea
“Sustaining” rating requires, “Leaders and teachers activelyuse established systemswith criteria
and protocols for identifying students for interventions and enrichment…” which includes the
processesofidentification,makingdecisionsabout,andmonitoringinterventions.
ExaminationoftheMSVsrevealthatacademictutoringopportunitiesarenotavailableoroflimited
availabilityinmostLevel4schools.Forinstance,atoneschoolextendedlearningopportunitiesand
mentoring programs are not available to all students because of staffing restrictions and lack of
funding. SomeMSVreportshighlight the intention toprovidebeforeorafter-school tutoring,but
theseprogramshavenot yetbegun.Other reports indicate tutoringhasbeendiscontinuedorno
mentionofitsexistenceismade.
Moreover, the types of tutoring offered also limits availability and/or take up. SomeMSVs only
mentiontheexistenceofintensivetutoringasanacademicinterventionforstudentsstrugglingthe
most. InoneschoolwhereCityYearandStarfishruntutoringbeforeandafterschool, interviews
with educators reflect a desire for more resources and programs so that tutoring can become
“formalized operationally, [and]well-disciplined.” There are exceptions. A small number of low
performingschoolsprovidetutoring,butattendanceandalignmentwithcurriculumisnotclearin
theMSVs.
Similarly, evidence of implementation of enrichment
opportunities is sparse. Some MSV reports highlight the fact
that schools are focused on academics and intend to provide
enrichment opportunities later down the road. Other schools
show evidence of limited enrichment opportunities, for
instance, forstudentswho“mightbeready toworkonhigher
level things”. The small number of Level 4 schools that have
implementedenrichmentopportunitiesdonotalwaysmonitor
these for effectiveness. Many reports make no mention of
enrichmentopportunitiesorexplicitlynotethesewerelacking.
By contrast, enrichment and tutoring opportunities are consistently available in the Level 1/2
schools. These include a range of programs such as clubs or elective subjects or providing
additionaltutoringhoursbeforemajortesting.Enrichmentopportunitiesareoftenledbyteachers
with particular interests in subjects outside the traditional curriculum. Inmost of these schools,
programsaremadeavailablethroughgrantsorpartnershipswithorganizationsinthecommunity.
Forinstance,Reeboksponsorsamorningexerciseprogramatoneschoolvisited.
[InLevel1/2schools]
enrichmentopportunities
areoftenledbyteachers
withparticularinterests
insubjectsoutsideofthe
traditionalcurriculum.
13
5. TeacherCulture,TrainingandEmpowerment
Teacher culture falls underTurnaroundPractice4:SchoolClimateandCulture, and is specifically
dealt with in indicator 4.5: Trusting Relationships. Achieving a rating of “Sustaining” requires
teacherstoactina“solutions-oriented”mannerandcollaborateonplanning,analysis,assessment,
anddecision-making.Teacher training fallswithinTurnaroundPractice2:IntentionalPracticesfor
Improving Instruction, and is specifically noted in indicator 2.4:ClassroomObservationDataUse.
Achievingaratingof“Sustaining”requiresteachersareobservedandreceiveactionablefeedback
thatisconsistentlyreviewed.
TeacherRetention:
Ourquantitative analysis indicates that a 1% increase in the teacher retention rate is associated
with a 0.86 percentage point increase in the probability of a school being rated as a Level 1/2
school. Higher teacher retention rates are thereforemeaningfully associated with the likelihood
thataschoolwillberankedLevel1or2.Whilethecausaldirectionofthisrelationshipisn’tclear
from quantitative analysis alone, there is consistent and strong evidence from interviews
conducted that teacher retention is a cause of high student achievement - often expressed in
statementsabouttheimportanceofveteranteachers.
The existing academic literature regarding teacher retention is heavily focused on teacher
incentivesandteacherdevelopment.Forexample,areviewofWashingtonD.C.’scity-wideIMPACT
system, a robust teacher evaluation system with performance pay, found the system improved
outcomesforbothlowandhighperformingteachersandencouragedstrongteacherstoremainin
schools(Dee,2015).Certainaspectsofthisteacherevaluationsystem,suchasfrequentobservation
and feedbackonclassroombestpracticesandadditionalprofessionaldevelopment forstruggling
teachers,couldbeimplementedwithinindividualschools.
Another study demonstrated associations between teacher retention and four elements of staff
culture (Shen,1997).These included:1)hiringmoreexperienced teachers;2) increasing teacher
salaries;3)emphasizingtheintrinsicmeritsofteaching;and4)empoweringteacherstoinfluence
schooldecisionsandpolicies.ManyLevel1/2schoolsconfirmedthesefindingswhendiscussingthe
importance of flexible hiring decisions, veteran teachers, and frequent check-ins with teachers.
However,more research is required tobetterunderstandwhat factors improve the likelihoodof
retainingteachers.
TeacherTraining:
TheMSVreportsalsoindicatethatLevel4/5schoolsoften
have a teacher training system in place that includes
observation and feedback, but is inconsistent.
Additionally,thereappeartobegapsinaccountability,as
instructional leaders do not always review observation
datawithteachersconsistently.OneMSVnotes, “insome
cases, staff reported not being completely aware of the
schoolwideexpectationsforobservationsorhowthesedataareusedtodriveimprovementatthe
school-level.” Another suggests, “There was evidence that classroom observation data are being
used by the individual teachers being observed, but data are not reviewed by the Instructional
Level1/2schoolsemphasized
theimportanceofagradual
releasemodelfordeveloping
strugglingornewteachers.
14
LeadershipTeamoranyotherschool-widemechanism.”Ingeneral,mostteachersareobservedin
their classrooms. The gaps in performance thereforemay result from thedifferences in the data
review and feedback piece of the instructional model. Lastly, MSV reports do not report on the
existence of a gradual releasemodel for teacher training. Gradual releasemodels train teachers
through model teaching by an instructional leader, co-teaching, and finally, observation and
feedbackthattapersoffasthetraineetakescontroloftheclassroom.However,variousLevel1/2
schools emphasized the importance of a gradual release model as a strategy for assisting and
developingstrugglingornewteachers.
TeacherCulture:
MSVreportsindicatethatLevel4/5schoolsareoftendescribedashavinga“positive”staffculture
with teacher-leader communication, but collaboration is often inconsistent and/or not strategic.
OneMSVreportnotes,“Althoughtheprofessionaldevelopmentschedulewasassessedandplanned
overthesummer,therewasnoevidencethatthistimeorCommonPlanningTimewasconsistently
reevaluated…” Another MSV reports, “Although teachers have blocks of time specifically for
planning,notimewasallottedforcollaboration.”Thisindicatesthatwhilelow-performingschools
build collaborative time into their schedules, they are not consistently using this time in a
productiveway.
In analyzing the MSV reports, two trends in teacher empowerment and culture emerge in low-
performingschool:1)teacherslackvoiceinschool-widedecisions;and2)communicationispoor
between administration and staff. As noted in various reports, atmany Level 4 schools, teacher
input is not sought for strategic decisions such as teaching schedules, the structure for
interventions,andwhatinitiativestheschoolshouldpursue.Somenotethatteacherbuy-intothe
school leaders’ vision is inconsistent. Furthermore, many schools struggle with establishing
consistent communication between school leaders and staff; for example, one report quotes a
teachersaying,“there’sa lotofchangeswe’renotmadeawareof.”Bycontrast,Level1/2schools
commonlyhostweeklymeetings and regular individual check-inswith teachers, conduct teacher
climatesurveys,andincludeteachersintheschedule-makingprocess.
VerticalIntegration:
WhilenotexplicitlyaddressedintheMSV
reports,aconsistentthemeacrosshigh
performingschoolsistheimportanceofa
verticallyintegratedteacherculturethat
enablesexpertisesharingacrossgrade-levels.
Indeed,thecamaraderieandwillingnessof
teacherstosupportoneanotherwasobserved
throughtheirinteractionsduringtheinterview
process.Animportantaspectofthisstrongcollegiateand‘team’approachisthepossibilityand
Level1/2schoolshostweeklymeetingsandregularindividualcheck-inswithteachers,
conductteacherclimatesurveys,andincludeteachersintheschedule-makingprocess.
InLevel1/2schools,verticalintegration
providesteacherswithabetter
understandingofhowrigorouslya
particularareaofthecurriculumhas
beencoveredandinsightintowhat
teachingstrategiesproveeffectivefor
particularstudents.
15
executionof‘verticalintegration’acrosstheteachingteamsofdifferentgrades.Whileallschools
(regardlessofLevel)havemandatoryCommonPlanningTime,highperformingschoolsoften
dedicatespecifictimetointegratestaffverticallyandshareknowledgeacrossgrade-levels.
Inschoolswithhigh-riskpopulations,thisapproachisparticularlyvaluablefortworeasons.First,
theissuesfacedbyhigh-riskstudentsareoftentimesnotspecifictograde-levelandstaywiththem
throughadolescence.Therefore, knowledge sharingaboutparticular studentneedsacrossgrade-
levelsmaybeofevenmorevalueinthesecontexts.Second,wherestudentsstruggleacademically
(as a result of external issues, for example), vertical integration allows for greater long-term
monitoring of student progress. For example, in Level 1/2 schools, vertical integration provides
teacherswithabetterunderstandingofhowrigorouslyaparticularareaofthecurriculumhasbeen
coveredandinsightintowhatteachingstrategiesproveeffectiveforparticularstudents.
6. ParentalEngagement
Parental engagement falls within Turnaround Practice 4: School Climate and Culture, and is
specificallyaddressedinthesub-indicator4.6:CommunityEngagement,whichlaysoutfivespecific
aspectsofparentalengagement.The importanceofstrategic,systematicand focusedengagement
withparentsemergedasakeydifferentiatorofsuccessbetweenhighandlowperformingschools
servingstudentsatrisk,whencomparingtheMSVreportswithdatacollectedduringschoolvisits,
ParentinvolvementismixedacrossLevel4schools.Some
schools promote regular engagement through weekly or
monthly opportunities while other Level 4 schools only
engage parents sporadically or are just beginning to
initiate more deliberate engagement opportunities. For
example, oneMSVnotes, “According tooneof the school
leaders,parentswhohadneverbeen in the schoolbefore
werestartingtocomeinforevents,suchastheShiningStarsassemblies.”
Inaddition,thetypeofparentalengagementisimportant.Asonereportnoted,“Familyengagement
is primarily focusedonbuilding social relationships andminimally on academic supports…most
effortsappeartohavebeensocialinnatureandlessfocusedoninformationforparentstosupport
students’academicprogress.”ManyoftheMSVreportscitethesocial,ratherthanacademic,focus
of parental engagement. There are exceptions to this which includes a number of schools that
exitedLevel4toLevel3orhigherinthe2015-16year.Anotherengagementstrategythatiseither
notevidencedintheMSVs,ornotedasonlybeginningtobeimplementedbysomeLevel4schools,
isvisitstoparentsattheirhome.
By contrast, qualitative data on Level 1/2 schools
suggestsmorestrategicandconsistentengagementwith
parents. For example, one school holds “shadow days”
twiceayear(inadditiontoanumberofotherevents),in
which parents are invited to come and sit in on classes
andfollowtheirchildthroughtheday.Anotherschoolis
in the process of hosting meetings (at the principal and
ManyoftheMSVreportscite
thefocusonengaging
parentssociallyratherthan
throughacademic
opportunities.
‘ShadowDays’allowparents
towitnesstheirchild’s
learningenvironmentand
academicexperience.
16
vice-principal level)with every parent of a studentwho recently took the PARCC assessment to
explain the results and review the child’s academic progress. All Level 1/2 schools interviewed
haveadesignatedparentliaisonorstaffmemberwhocoordinatesparentengagement.
Parentalengagementemergesasparticularly important in thecontextofhigh-risk students fora
numberofreasons.First,anecdotalevidencesuggeststhatengagingparentsinconversationsabout
their child’s academic performance is reflected, at least in the short term, in improvements in
studentbehavior.Developingtheconceptofa‘partnership’ofresponsibilitybetweenteachersand
parents increases the likelihood that parents engage their child about their schooling in a
substantiveway at home. Second, existing literature shows that there is an association between
student performance and parents’ social capital. Social gatherings for parents to expand their
networkandexposuretoadditionalresourcescouldhaveaneffectonstudentperformance.Third,
parental engagement can improve student attendance (discussed below). Finally, academic
researchindicatesthatforhigh-riskstudents,thegainsfromeducationalinterventionsarestrongly
tiedtotheacademicabilitiesoftheparents.Therefore,activitiesthatengageparentsacademically
couldimprovestudentperformance.
7. CultureofAttendance
AttendanceisnotdirectlyaddressedinMSVreports,
but both our quantitative and qualitative analysis
indicates that it is an important component of a
successfulschool.Ourstatisticalanalysisshowsthat
a 1% increase in student attendance is associated
with a 7.4 percentage point increase in the
probability of a school being rated as a Level 1/2
school. This suggests that a strong culture of
attendance is strongly associated with being a Level
1/2 school. Existing literature indicates the causal relationship of this finding: an increase in
instructionaltimeprovided(Dobbie&Fryer2011)andadecreaseinstudentabsences(Goodman
2014) are associatedwith significant increases in student achievement. The reason is thatwhen
individualstudentsareabsent,teachersmustcatchthemupduringclasstimeindividually,which
takesawayfromwholegrouplearningandhindersschool-levelperformance(Goodman,2014).
Additionally,everyLevel1/2schoolinterviewedtracksattendanceandholdsstudentsandfamilies
accountable in someway. Daily phone calls aremade to absent student homes, either by a staff
memberor throughanautomatedsystem,at every school. Staff also indicated that theybelieved
thisiskeytoensuringbetterstudentattendance.Manyschoolshaveadditionalincentiveprograms
such as a rewards system for collective or individual attendance goals. In addition, Level 1/2
schoolsuse family-centricattendancestrategies toaddresschronicabsenteeism, suchasvisits to
thehomeandmeetingswithparents.Finally,manyoftheschoolshaveonestaffmemberincharge
oftrackingabsenteeismdata.
Amere1%increaseinstudent
attendanceisassociatedwitha
7.4pointincreaseintheprobabilityofa
schoolbeinghigh-performing.
17
8. MonitoringSiteVisitRubrics
Observing and collating the findings above highlighted gaps in the MSV rubric. While the MSV
rubric is not meant to provide operational support and instead only collect data on school
turnaroundprogress,therearegapsinthetypesofdatatherubriccollects.Whenweanalyzedthe
MSV reportswith the collated setofbestpractices acrosshigh-performing schools,we identified
gaps in the data collection process. Additional continuum points and rubric rows aligned to our
recommendationswill enablecollectionofmorepreciseandhelpful informationabouta school’s
progress.
18
VI.Recommendations
Basedonourquantitativeandqualitativeanalysisofschoolperformance,werecommendschools
servingat-riskstudentpopulations:
1. Embed comprehensive and cohesive systems into daily school operations that focus on
studentperformance;and,
2. Createstrategiestocultivateapositiveschoolculture.
Theserecommendationsandtheirsub-recommendationsemergedfromanalysisofthefindings
describedaboveandencapsulatetheimplementationofbestpracticesbyhigh-performingschools
ascomparedwiththepracticesoflow-performingschools.AsanalysisoftheMSVreports
emphasizethedifficultylow-performingschoolshaveinoperationalizingbestpractices,these
recommendationsarecraftedashighlyspecificactionssteps.Furthermore,whileexistingliterature
alsoidentifiessomeofthesebestpractices,thissectionaimstoillustratethemwithpractical
examplesofhowthesehaveandcanbeimplementedinelementaryschools.Allofour
recommendationsaresummarizedinFigure1onpagefour.
ThethirdrecommendationisaimedatenablingDESEtobettersupportlow-performingschools
throughtheirmonitoringandevaluationprocess.Thisfinalrecommendationtoimprovethe
MonitoringSiteVisitrubricswillallowforbettercollectionofdataandimprovedsupportforthe
lowestperformingschoolsonastatewidelevel.Together,therecommendationssupportschoolsin
implementingbestpracticesandallowDESEtomonitorschools’progressindoingso.This
alignmentisdescribedinFigure3below:
Figure3:AlignmentofSchool-LevelandDESERecommendations
19
Recommendation1:EmbedComprehensiveandCohesiveSystems
We recommend that schools develop and embed comprehensive and cohesive systems for data
driven instruction, behavior management, and the provision of interventions and services. Our
analysisshowsthatLevel1/2andLevel4/5schoolsdonotdiffergreatlyinthetypesofservicesor
initiatives available, but Level 1/2 schools articulate and instigate coherent strategies in
operationalizing these initiatives, and so provide consistent and comprehensive support. This
recommendationthereforeproposesthatinitiativesshouldnotbeimplementedinsilosorad-hoc
ways; rather, they should exist as part of a cohesive school-wide system.
1.1.DataDrivenInstruction
1.1.1. Administer formative assessments multiple times per year and conduct weekly
analysis of student data (at minimum). The notion of data driven instruction should not be
limited to state assessments or summative assessments. Whether schools create their own
formative assessments to use throughout the year, or rely on an outside source, collecting and
analyzing student progress data frequently is a consistent characteristic of Level 1/2 schools.
Weekly analysis can be focused on studentwork or involve using tools such asWilson Literacy
Training. In weekly collaboration meetings, which occur in most schools regardless of Level,
teachersshould lookatstudentdataduringeverymeeting ideallywithan instructional leaderAt
someschools, teachersbroughtactualstudentworktoeachof thesemeetings.This increasesthe
accountability of using data and the rigor with which it is evaluated. By using data on a more
frequentandconsistentbasis,teachersarebetterequippedtomanipulatelessonplanstostudent
needsandbucketstudentsintolearninggroupsastheirneedsshift.Frequentandconsistentuseof
dataisespeciallyimportantforat-riskpopulationsthathavespecificandacuteacademicneedsthat
canbetargetedbyteachers.
AtWarnerElementary in the teacher common room, there is a visual representationof
everystudent’smostrecenttestscoresonacardwiththeirphoto.Thesephotocardsare
placedwithinproficiencycategoriesonthewallsofthestaffroomwiththegoalbeingthat
studentswillgraduallyimprovethroughouttheyearwitheachnewformativeassessment.
Thisvisual cue for teachers is aconstantreminderofwhichstudentsneed assistance in
which areas; and provides a concrete action aligned with their progress against data
measures.
Testingdataisalsoreviewedsystematicallyaftertestsarecompleted.Whenanumberof
students get something incorrect, teachers reteach (or have another teacher reteach)
theseproblematicareasinthefollowingweeks.
AtViveirosinFallRiver,teachersbringinformaldata(gradedhomework,shortresponses,
exit tickets, etc.) to every collaboration meeting and instructional leadership meeting.
Theyusethisdatatoidentifytrendsinstudentlearningacrossclassroomsandrestructure
theirlessonsaccordingly.
ImplementationExample:VISUALCUESANDMONITORING
20
1.2.BehaviorManagement
1.2.1. Implement consistent, positive school-wide behavior management structures that
include clear points of contact and procedures for escalation. Many low-performing schools
articulate a vision for behavior management but leave it up to teachers to design their own
behaviorstandards for theirownclassrooms.Widedisparitiesbetweenclassroomstrategies lead
to inconsistent implementationof school-wide expectations,which canbe confusing for students
and lead to misbehavior. Thus, we recommend that schools design core behavior structures,
uniformacross all classrooms,while allowing teachers autonomy to supplement the school-wide
structure with classroom-specific incentives. Examples of these core structures used by various
Level1/2schoolsincludebuddyrooms,wherestudentsaresenttoapartnerteacher’sclassroom
foratime-out,specificphysicalspacesforstudentstoregroupevenbeforebehavioralissuesarise,
andvisualrepresentationsofbehavioralmonitoring.Forexample,someschoolshaveacolor-coded
system for students tomonitor their own behavior as they progress throughout the day. These
behavioralinterventionsfollowadistinctescalationpathfromtheclassroomtotheprincipal.
Atoneschool,certainbehaviors,suchastalkingback,wouldbedealtwithinsidethe
classroom.Anothersetofmoredetrimentalmisbehavior,suchasatantrum,wouldbe
dealtwithbysendingthestudenttoa“buddyroom”wheretheteacherisexpectingthem.
Finally,studentswhoconsistentlymisbehave,ordemonstratemoreegregiousbehaviors
suchasendangeringotherstudents,aremonitoredbyaschoolleader,whofrequently
checksinwiththem.Oftentimes,thesestudentswillalsotakepartinsocialskills
workshops,workinaseparateclassroomasneeded,andworkwithaschoolcounselorto
overcomebehavioralissues.
ImplementationExample:ESCALATIONLADDER
At theSumner School, theVicePrincipalhasresponsibility for thebehavioral,aswellas
socialandemotional,wellbeingofstudents.Sheconductsdailycheck-inswithstudentsin
the mornings, with an emphasis on positive reinforcement. She also keeps a list of
studentswho struggle themostwithbehaviorandchecks inwith themmore frequently
throughouttheday.HerclearroleallowsstudentstoaccesstheVicePrincipalwhenthey
were‘havinganoffday’,oralternativelyneedsomeotherformofnon-academicsupport.
AtHyannisWestElementary,atieredapproachisconsistentlyimplementedacrossthe
school.Thebasisofthesystemisapositiveinterventionapproach(facilitatedbythe
ResponsiveClassroommethod),inwhichstudentsarefrequentlyrewardedforgoodand
kindbehavior.Inaddition,recognitionisgiveninassembliesfor“randomactsof
kindness”.AspartofTier1,theexpectationsforbehavioraremadeclearforallstudents.
Forinstance,expectationsforhowtobehaveinthehallwaysaredisplayedinthe
corridors.Tier2involvesidentifyingstudentswhoneedmoresupportandcreating
individualinterventionplansasneeded.Forexample,astudentmayhaveadaily“check-
in”and“check-out”withateacherwhospecificallyasksthemwhattheyneedtoget
throughthedayproductively,andthenfollows-upthroughout.Inaddition,theschool
counselorprovidesregularskillsbasedworkshopsforstudentswhostrugglewith
particularsocialskills.Staffandcommunitymembersalsoparticipateinmentoring
studentswhoneedgreaterassistance,volunteeringonaweeklybasistohavealunchor
breakfastwiththestudent.Tier3ofthebehaviorsystemprovidesaseparateclassroom
wherestudentscangoformoreintensivecounseling,tutoring,orotherinterventions.
ImplementationExample:TIEREDAPPROACH;SINGLEPOINTOFCONTACT
21
1.2.2.Conductreviewsofdisciplinedataconsistentlytoevaluatemanagementstructure.We
recommendthatLevel4/5schoolsreviewdisciplinedataaspartofgrade-levelteammeetings.Our
quantitative analysis suggests a strong association between the decrease in students disciplined
andtheprobabilityofschoolsbeingaLevel1/2school.Reviewingstudent-leveldatatounderstand
the causes of disciplinary issues and assessing the trends in behavior management across the
schoolmayhighlight clear areaswhere school-wide systems are consistently failing. This review
should be used to inform changes to the behavior management structure and assess on-going
opportunitiesforimprovementofschool-widesystems.
1.3.WraparoundServices
1.3.1.Createasystemforreferral,monitoring,andevaluationofwraparoundservices,
ideallythroughonepointofcontact.EvidencefromMassachusettsWraparoundZones(WAZ)
demonstratesaclearassociationbetweenWAZservicesandstudentachievement;however,the
mereprovisionofservicesisnotadequate.InWAZschools,acohesivesystemensuresthat
studentsarenotreceivingduplicateservicesandarebeingreferredtothemostappropriateservice
providers.Inaddition,schoolscanmonitortheprogressofeachindividualstudentreceiving
servicesandevaluatetheoutcomesofprogramparticipation.MSVreportsindicatethatteachers
whoarenottrainedtoaccuratelyidentifystudentneedsprimarilyhandlereferralsonaone-off
basis,whichcanresultinduplicativeorinadequateserviceprovision.
Schoolsshouldidentifyonepointofcontact,usuallyacounselororviceprincipal,whohandlesall
studentorfamilyparticipationinoutsideservices.Thispersonshouldberesponsiblefortraining
teacherstoidentifystudentneeds,handlingallreferrals,communicatingwithfamiliesandservice-
providers,andmonitoringtheoutcomesoftheseservices.Theyshouldactasago-toresourcefor
teacherswhohavequestionsorconcernsandmakeregularcontactwithteacherstomonitorand
identifyschool-wideneeds.
HyannisWestkeepsadailylogofbehaviorincidents.Whenstudentsareexitedfroma
classroom,thisabsenceisrecordedinordertoprovidedataonhowmuchschoolingis
missed.Moreseriousmisdemeanorsarereportedinadistrictwidedatasystem,anda
counselorandprincipalreviewthisdataroutinely.
ImplementationExample:ADAILYLOG
SumnerSchoolinBostonconductstwice-annualreviewofstudentdisciplinedatato
addressthecausesofdisciplinaryissuesandtodirectstudentstospecificinterventions
basedontheirdisciplineprofile.Studentsarereferredtosocial-emotionalorbehavioral
servicesbasedonthosedata.Inaddition,thesereviewsareusedasanopportunityto
addressschool-widetrendsanddevelopappropriatesystemicinterventionsthatrespond
tothese.
ImplementationExample:REVIEWOFDATA
22
1.4.AcademicInterventionsandEnrichment
1.4.1.Developconsistentopportunitiesforteacherstotutorallstudentswithdiverseneeds.
Addressingthediverseneedsofstudentsthroughvariedandmulti-tieredacademic interventions
ensures that schools are meeting every child’s needs. Many low-preforming schools focus
interventioneffortsontargetedsub-groups,suchasELLorspecialeducationstudents,butdonot
have systems to identify specific needs across the entire student body. Schools should develop
systems to identify individual students’ needs – whether through data analysis or classroom
observations – and establish tutoring groups and content based on this evaluation. In Level 1/2
schools, tutoring is viewed as most effective when provided by teachers, or as part of the
responsibilitiesofspecialty instructional leaders,ratherthananoutsideandpossibly freesource.
Additionally, Level 1/2 schools oftenprovide tutoring in cycles that are alignedwithunits being
taught and individual student needs. Therefore, if a student needs tutoring for writing but not
readingcomprehension,theyattendthesetutoringsessionswhentheyareoffered,ratherthanan
ad-hocrangeofservicesthatmayormaynotberelevant.
AtViveirosinFallRiver,acounselorreceivesallreferralsfromschoolstaffand
coordinateswithallexternalagencies.Teachersareawareofthespecificprocessesand
requirementstoreferstudentsforservices.Thecounselorisresponsiblefordocumenting
andtrackingreferralstoanexternalserviceprovider.Ratherthanjustbeingan
intermediary,thecounselorfollowsuptoevaluatethestudent’sprogressandliaises
betweentheserviceprovider,teachers,andfamiliesonwhetherthoseservicesare
appropriateandhavedeliveredthedesiredoutcomes.Ifnot,thenchangesaremade.
ImplementationExample:ASYSTEMOFCOORDINATION
AttheBelmontStreetCommunitySchool,eachstudenthasanacademicdevelopmentplan,
aswellasageneralinterventionplanforbothacademicandnon-academicservices.
Interventionsmayincludeadditionalreadingpracticewithateacheroradditionalstudy
skillstime,dependingonthechild’sneeds.Theseinterventionsaredeterminedthrough
datareviewsandmonitoredconsistentlybygrade-levelteachers.
ImplementationExample:STUDENTCENTEREDINTERVENTIONS
AtSumnerinBoston,teacherstutorinsix-weekcyclesandre-evaluatethetutoringgroups
basedondatathatiscollectedthroughouttheunit.Thesecyclesarebasedoncontent,such
asoperationsorwriting,ratherthanstudenttype,suchasELLorSPED.Thetutoringoffered
isalignedwiththecurriculumbeingtaught.Insomecases,teachersalsoprovidetutoringfor
highperformingstudentsinordertochallengeorpushthembeyondthecurrentclassroom
content.Thisensuresthatalltypesofstudentneedsarebeingmet.
ImplementationExample:ALIGNINGTUTORINGANDCURRICULUM
23
1.4.2.Developpartnershipswithexternalproviderstocreateopportunitiesforenrichment
and learning. Many schools that do not have the financial or logistical capacity to provide
enrichmentservicesbyteachersrelyonexternalprovidersforsuchservices.Externalpartnerships
canbedevelopedwith churches, sister schools, local communityorganizations, oruniversities to
provide free or subsidized enrichment programs for students. Schools lacking resources should
considerengaginginlocalpartnershipstoprovidesuchopportunities.
HyannisWestElementarypartnerswitharangeofcommunityservicesandorganizations.
Forexample,membersoftheBigBrotherandBigSisterprogramattendtheParentOpen
House evening at thebeginningof the year. Parentswho feel that their childrenwould
benefitfromtheprogramprovidepermissionfortheirchildtobeassigneda“bigbrother”
or “big sister” to visit the student at lunch or after school. The school also welcomes
‘interns’ or volunteers from two different local high schools who become regular
classroom visitors and are available to assist in reading to small groups or with
organizational aspects of the classroom. The Local Rotary Club is also involved in the
school and provides funds to support literacy programs and guest readers to different
classrooms.Inaddition,thelocalPoliceDepartmentprovidesan‘adoptivepoliceofficer’to
regularly visit the school in order to demonstrate the positive role of the police in the
community.
ImplementationExample:BUILDCOMMUNITYPARTNERSHIPS
24
Recommendation2:Createstrategiestocultivateapositiveschoolculture.
We recommend that schools cultivate a positive school culture through intentional and explicit
strategiesclearlylinkedtodesiredoutcomes.Ouranalysisshowsthatpositiveschoolculturedoes
notonlyimpactstudentbehaviorandparentengagement,butalsostudentattendanceandteacher
retention, both of which are empirically associated with improvements in overall school
performance.WhileMSVreportsoftenindicateapositiveschoolcultureinLevel4schools,thereis
rarely a coherent or intentional plan linking various initiatives with desired outcomes, such as
teachersatisfaction.Ratherthansimplyimplementingalitanyofunalignedinitiatives,schoolsmust
strategically evaluate how those initiatives lead to their desired outcomes. We therefore
recommend that schools develop intentional strategies to strengthen school culture through:
parentengagement, teacher training, culture,empowermentandvertical integration,andstudent
attendance.
2.1.TeacherTraining,CultureandEmpowerment
2.1.1.Implementagradualreleasetrainingmodel.Teacherswhostruggleintheclassroomcan
negatively affect culture and are likely not to be retained. Therefore, a strong teacher-training
model can directly promote a positive culture. Because MSV data reveals that classroom
observationsforLevel4schoolsareinthemidtolowrangeontheirrubric,agradualreleasemodel
couldensurethat the low-rangeteachers improvemorequickly thantheyotherwisemight.Level
1/2schoolsoftenuseagradualreleasetrainingmodelforstrugglingornewteachers.Theschools
we interviewed generally use a 6 to 8week cycle depending on howmuch support the teacher
needs.Themodelteachinglastsaroundtwoweeks,theco-teachingmodellastsaroundtwoorthree
weeksandthentheobservationphaselastsaslongasnecessaryandtapersoffgradually.Basedon
observationdata,thesetimeframesshouldbeadjustedasrequiredforindividualteachers.
2.1.2.Structureandevaluatecollaborationtimeto increaseproductivityandeffectiveness.
Scheduling collaboration time isnot an issue formostLevel4/5 schools;however, ensuring that
thistimeisusedeffectivelycanbeachallenge.Ingeneral,theprincipalorinstructionalleadersin
Level1/2schoolsprovideastructureforthesemeetingsandguidanceonhowthetimeshouldbe
used. Then, an instructional leader or coach runs themeetings. School leaders also consistently
evaluatethisstructurebysittinginonmeetingstoensureitiseffectiveandproductive.Asaresult,
usingthistimeeffectivelynotonlyimprovesstudentoutcomes,butleadstomoresatisfiedteachers
aswell.
AtViveirosinFallRiver,teachersarepairedwithaninstructionalleaderwhomodel
teachesthefirstfewweeksoflessons.Then,theytransitiontoaco-teachingmodel,and
finally,theyshifttoasingleleadteacherclassroomwithobservationfeedbackcycles.
Check-insanddebriefsareincludedthroughouttheentirecycletoensureteachersare
learningfromtheprocess.After6-8weeks,theinstructionalteamlooksatstudentdatato
decidewhichteachersneedadditionalsupportthroughthismodel.
ImplementationExample:GRADUALRELEASEMODEL
Whilevariousmodelsarepossible,onepotentialscheduleforcollaborationtimewould
seethefirst10minutesdevotedtostudent-leveldataanalysis.Thenext20minutes
devotedtotrendsinlearningacrossclassrooms,andthefinal20minutesprovidedfor
teacherstopairoff,revampandgetpeerfeedbackontheirlessonplansforthecoming
week.
ImplementationExample:STRUCTURINGCOLLABORATIVETIME
EFFECTIVELY
25
2.1.3. Empower teachers with autonomy and flexibility through frequent communication
withschoolleaders.ManyLevel1/2schoolsdevelopstrategiesandsystemstoempowerteachers
andgivethemavoice.Providingteacherswiththisgreaterinfluenceandautonomycanencourage
them to stay at the school. School leaders empower teachers by allowing them to make key
decisionsregardingaspectsof teachingsuchas interventionplans,parentengagementstrategies,
or classroom-level reward systems. Furthermore, these schools develop consistent systems for
communication between teachers and school leaders to analyze teacher observation data and
providefeedback.
2.1.4. Dedicate specific planning time to ‘vertical’ meetings, particularly at the end and
beginning of the academic year. Many schools focus collaborative time on grade-level team
meetingsbutprovidefewopportunitiesforcollaborationbetweengrade-levelsteachingteams.In
ordertoensurethatallteacherscanmosteffectivelyaddressachild’sneeds–bothacademicand
social-emotional–,vertical integrationandcommunicationbetweengrade-levels is imperative.At
thesemeetings, teachers should inform the student’s next teacher of specific student needs and
ensurethecontinuationofservicesandinterventions.
Verticalintegrationtimeshouldalsobeusedtoallowforbestpracticesharing.Providingteachers
with theopportunity to learn fromothermoreexperienced teachers,particularlywhere teachers
mayhavetaughtthesamestudentsorcurriculumpreviously,canbeofgreatvalue.InvariousLevel
1/2schools, there isevidenceof consistentandon-goingmeetingsacross staff groups thatallow
teacherstosolicithelpwithparticularstudentsandsubjectmatters.Giventhehigh-needsofat-risk
populations,havingextrainsightsintoparticularstrategiesfordealingwithaspecificchild’sneeds
maybeofsignificantvalue.
AtCarltonViveiros,teachersaregiventheautonomytodesigntheirownspecials
curriculum,teachingstudentsspecialtyelectivesbasedontheirowninterests.
AttheNewtonSchoolinGreenfieldandtheWarnerSchoolinSpringfield,teachersare
giventheflexibilitytodesigntheirownteachingschedulesandworkalongsideschool
administrationtodesigninterventionschedulesthatmeettheirstudents’specificneeds.
Furthermore,theentiredistrictatSpringfieldPublicSchoolscompleteOrganizational
HealthInventory(OHI)surveys,whichareusedforteacherstoassessschoolcultureand
communicationwithschoolleaders.
ImplementationExample:EMPOWERINGTEACHERSINTHEIRCLASSROOMS
AtViveirosinFallRiver,aschoolleadersitsinonmeetingseachweektoevaluatetheir
progressandefficiency.ThisisusuallytheEnglishLanguageArtsinstructionalleader,
Mathleader,orprincipal.Theydonotuseaspecificrubrictoevaluatethesemeetings,but
theseobservationsallowthemtointerveneandprovidefeedbackshouldstaffmembers
needit.Sometimestheywillleadthemeetingtomodelhowcollaborationtimeshouldbe
run.Allofthesecollaborationmeetingsincludestudent-leveldataanalysis,whichteachers
collectinformallythroughouttheweekandbringwiththemtoeverymeeting.
ImplementationExample:EVALUATINGEFFECTIVENESS
26
2.2.ParentEngagement
2.2.1.Dedicatespecificresourcesandplanningeffortstofosterconsistentopportunitiesfor
parentalengagement.Wherepossible,designatespecificresponsibilitiesforparentalengagement
tooneortwostaffmemberstoprovideschoolstheabilitytoplanandexecuteparentalengagement
in more consistent and strategic ways. Create predictable patterns of parental engagement to
enhance parental responsibility, signal high expectations for learning, and deter non-attendance.
Level1/2implementingthesestrategiesdidsoacrossavarietyofforums,includingthroughtheir
parent teacher councils (or equivalents) as well as a program of activities and engagements
throughouttheyear.
2.2.2. Focus engagement opportunities with parents on academic, as well as social or
recognition-based, events. Many Level 4/5 schools mention award ceremonies or social
gatheringsforparents.However, it is justas importantthoughthatparentsare involvedwiththe
academic aspects of school aswell. Thiswill better equip themwith content knowledge and an
understandingoflearningprocessestosupporttheirchildathome.ManyLevel1/2schoolsdothis
byschedulingeventssuchas“ScienceFairs”or“MathNights”oraninteractiveeventbasedaround
aunitbeingtaughtinthecurriculum.Thesebecomeopportunitiesforstudentsandtheirparentsto
engagewithacademicallyrelevantmaterialandshare inthe learningprocesstogetherwithinthe
schoolenvironment.High-performingschoolsalsowelcomedparent involvement in theeveryday
functioningofaclassroom,particularly ifastudentwasstruggling.Someschools formallyextend
thisinvitation,butmostemphasizethatparentsarewelcometoobserveandparticipateinthedaily
runningof the classroomat any time. Socialor recognitionbasedevents also still takeplaceand
usuallyincludesomesortofinformationorresourcesharing.
AtSumnerElementary,theprincipalhiredaparentwhowasalsoapara-teacheralready
staffedattheschooltotakeresponsibilityforengagingandimprovingtheparentcouncil.
Shewasalsobilingualwhichgreatlyhelpedwithcommunication.Theprioritywasto
diversifytheparentcouncilwithmoreHispanicandBlackparents.Now,thegoalisto
engageparentsmoreintheclassroom.Thisinvolvesinvitingparentstoschooltoworkon
projectswiththeirchildrenorshadowthemintheclassroom.Tostartthisinitiative,the
principalhasassignedtwoteachersinthesecondgradetopilottheprogram.Oncethey
havecreatedafluidsystemforparentengagementintheclassroom,theywillrolluptothe
othergrades.Overall,settingshort-termgoalswithoneortwocoordinatorsforparent
engagementhasworkedwellforSumner.
ImplementationExample:STRATEGICENGAGEMENTANDACHIEVEABLEGOALS
Inoneschool,classroomteachersinvitedparentstwiceayearto‘shadow’studentsfora
schooldaytoseedailyclassroompractices.Classroomteachersalsosenthomeprojects
withstudentsthatareintentionallytailoredforparentcollaboration.
Toengageparentssocially,aschoolleaderinvitesthemtoa‘MuffinsforMoms’or‘Donuts
forDads’breakfastwheretheycansocializewitheachotherandreceiveresourcesor
informationaboutschoolprocessessuchasbehaviorexpectations.
ImplementationExample:CATERINGFORACADEMICANDSOCIALENGAGEMENT
27
2.3.CultureofAttendance
2.3.1. Create a family-centered system for addressing chronic absenteeism. To address
chronic absenteeism, Level 1/2 schools dig deeply intowhat the root cause of such absenteeism
could be. Is it transportation?Mistrust of school? Parent’s schedules? Level 1/2 schools create a
profile for each child who is chronically absent and communicate with families consistently to
identifyandsolvetheproblem.
2.3.2.Developasystemtosupportdailyphonecallsincasesofstudentabsence.Thispractice
iswidely implemented among Level 1/2 schools and considered imperative tomaintaining high
attendance. Interviews indicated that for high-needs populations where students face transient
circumstances and lack consistent transportation to school, high expectations and accountability
areessentialformotivatingconsistentattendanceandovercomingexternalbarriers.
AtHyannisWestElementary,theyhaveimplementedanumberofapproachestodealwith
absenteeism,and,inparticular,withchronicabsenteeism.Firstly,theyhavemovedfroma
systemofrewardingstudentsonayearlybasisforperfectattendance,toprovidingmore
regular opportunities for recognition of perfect attendance and punctuality, including
through monthly student assemblies, and posting student names in a bi-monthly
newslettertofamilies.Inaddition,theschoolhas increasedprovisionofbreakfastinthe
classroomsinthemorningtotrytoencouragestudentstocomeontimetoschool.
The school counsellor sends letters home after 5, 7 and 10 days of absence. When a
student has been absent for 10 days, the family is called in for a meeting, in order to
determine thecauseof the absenceanddevelop solutions, suchas transportationneeds
forexample.ArepresentativefromtheLocalDistrictAttorney’sofficeisalsoinvitedtothe
meeting,notasapunitivemeasure,butratherasawayofwideningtherangeofsolutions
available.
ImplementationExample:AMULTI-PRONGEDAPPROACHFORCOMBATTINGABSETEEISM
Allschoolsinterviewedhadimplementedadailycallsysteminsomeway.Insomeschools
this task was performed by the school nurse, and in others a dedicated parent liaison
communicated on a daily basis with absent students’ families. In other cases, such as
across Boston Public Schools, the district had implemented an automated call system,
however,itwasnotedthatthepurposeofthiswasasafetymeasurefirstandforemost.
ImplementationExample:ADAILYCALLSYSTEM
28
Recommendation3:ImproveMonitoringSiteVisitRubric
Our final recommendation focuses on specific changesDESE canmake to theMSV rubric. These
recommendationsarebasedongapsidentifiedinthequantitativeandqualitativeanalysisofdata
fromLevel1/2schoolsandtheevaluationofLevel4/5schools.Whilethisisnotpartofouroriginal
research question for what schools can do, it is in line with the over-arching goal of improving
support for lowperformingschools.Bycollectingbetterdata in theMSVrubric,webelieveDESE
willbeabletobetteridentifyschool-levelgapsandrespondaccordingly.
3.1.AddAdditionalContinuumPointstoPortionsofMSVRubric
3.1.1.Indicator1.5:InstructionalLeadershipandImprovement.Thecurrentcontinuumpoints
includedina“Sustaining”rankingareweeklyordailyobservations,timelyandactionablefeedback,
anddatacollection tomonitorprogress.DESEshouldconsider includingagradual releasemodel
forneworstrugglingteachersinordertoincreaseimplementationofinstructionalbestpractices.
Level 1/2 schools consistently cite the gradual release model as highly effective for improving
teacherqualityineveryclassroom.
3.1.2: Indicator 2.2: Instructional Schedule: The current continuum points included in a
“Sustaining” rankingdescribe collaboration “across grade-levels and content areas”.DESE should
consider including languagearoundvertical integrationsothatcollaborationtimebetweengrade-
levelsisalsomeasuredandreviewed.Basedonourinterviews,atminimumonceperyear,teachers
shouldcollaboratewith teachers fromothergradesand/ormentor teachers todiscussparticular
students and learn from one another about implementation of best practices and coverage of
specificcurriculum.
3.1.3Indicator2.5:StudentAssessmentDataUse.Thecurrentcontinuumpoints included ina
“Sustaining” ranking describe “consistent” use of benchmarks, common assessments and state
assessments. DESE should consider including language around frequency and variety of data
collected. In our interviews, teachers from Level 1/2 schools describe using both informal data
dailyorweeklyandformaldataattheendofeachunit.Werecommendteacherscollectformaldata
aftereachunitandinformaldataonaweeklybasis.Thisdatashouldbeusedtoadjustupcoming
unitsandlessonsbasedonstudentmisunderstanding.
SAMPLE LANGUAGE (SUSTAINING): Struggling or new teachers are paired with an
instructionalleaderforasix-weekgradualreleasemodel.
SAMPLELANGUAGE(SUSTAINING):Teachersanalyzeinformaldataweeklyandformaldata
attheendofeachunittoinformtheirinstructionimmediatelyforthenextweekorunit.
SAMPLELANGUAGE (SUSTAINING): At least onceper year, teachers between grade-levels
meettodiscussindividualstudentneedsandbestpractices.
29
3.1.4:Indicator2.6:TeacherProgressAssessmentPractices.MostLevel4/5schoolsareaware
thattheyshouldanalyzestudentdatatoinforminstruction.OneprimarydifferencebetweenLevel
4/5 schools and Level 1/2 schools is the frequency with which teachers analyze student data
collaboratively as a team. At Level 1/2 schools, teachers analyze data collaboratively at least
weekly.Thecurrent“Sustaining”languagesaysteacherswork“individuallyandcollaboratively”to
determine student progress. This does not include language around frequency.We suggest data
analysisoccurweeklyincollaborationmeetings,evenifonlywithinformaldata,whichisespecially
helpfulforstrugglingteachers.
3.1.5: Indicator 4.1: School Wide Behavior Plan: Most schools seem to have a school wide
behavior plan of some sort, regardless of Level. Currently, the “Sustaining” rating states that the
plan should be implemented by a “majority” of the staff and subsequentlymonitored by school
leadership. The Level 1/2 schools we spoke with had behavior plans that included a ladder of
escalation for student behavior and a single point of contact for studentswho struggle themost
withbehavioralissues.Wesuggestincludingmoredescriptionofwhatthisplanshouldlooklikein
the MSV report. We also believe that “majority” should be changed to “all” to avoid gaps in
consistencybetweenclassrooms.
3.2.AddAdditionalIndicatorsforAttendanceStrategies
Higher attendance rates are correlated with an increase in the probability of being a Level 1/2
school. Our interviews with Level 1/2 schools also reveal the importance of a family-centric
approach to attendance accountability that includes student profiles and home visits for the
chronicallyabsent. Inaddition,dailyphonecallsareconsistentlyabestpracticeacrossLevel1/2
schools as an accountability and information gatheringmechanism. This should be a new rubric
rowunderturnaroundpracticefour.
SAMPLE LANGUAGE (SUSTAINING): Teachers bring student data to weekly collaboration
meetingsforanalysisasateam.
SAMPLELANGUAGE (SUSTAINING): Absent students’ families are called daily; chronically
absentstudentsaregivenaprofileandreceiveahomevisitoncetheyhavemissed10school
days.
SAMPLE LANGUAGE (SUSTAINING): Schools implement a school wide behavior plan that
includesa)aclear ladderofescalationfromclassroomtoprincipal forstudentbehaviorand
b) a single point of contact for students who struggle the most with behavioral issues. All
teachers and leaders, including part-time and specials teachers, implement this plan
consistently.
30
FrameworkforImplementation
Inordertosupportimplementationtheserecommendations,wedevelopedthefollowingquestions
forschoolstoassesstheirneedsandnextsteps.Sinceschoolsvaryinexistingcapacityandneed,we
believe that individual school leadership teams are best placed to determine a particular and
specificplan.Schoolsshouldassesstheirowncapacityandneedstoidentifypriorityareastofocus
on,anddevelopatimelineforimplementingtherecommendationsoutlinedabove.Tothatend,we
recommend that schools ask themselves the following reflection questions to analyze their
strengths and weaknesses. We recommend that schools first focus on impact to identify the
initiativesthatwillproducethegreatestgainsfortheirteacherandstudentpopulations.Following
this, we recommend that schools analyze the feasibility of implementing those initiatives given
their resources, the school’s existing capacity, aswell as district-level constraints. School leaders
shouldprioritize initiatives thathave thehighest impact,andthose thatareeasiest to implement
giventheircurrentsituation.
Werecommendthatschoolleadersusethisframeworktoguidediscussionwiththeirstaff,parents,
and students to strategically develop a unique implementationplan that accurately reflects their
needs and capabilities. Many high-performing schools indicated that they found it ineffective to
attemptallthereformstheydesiredatonce;instead,theyfocusedononeortwokeypriorities,and
then used those as a foundation to build up other reforms and initiatives. This set of reflection
questionsallowsschoolstoidentifytheirprioritiesanddevelopasequenceforaction.
After schools identify their priority initiatives, they can refer to Appendix VII, a summary of the
recommendationsaredividedamongstthekeystakeholdersthatprincipalsneedtopartnerwith,in
ordertoachievedesiredresults.Thereareseparatesummariesforschoolleadership,teachers,and
the counseling department based onwhich stakeholderswe believe aremost closely aligned for
implementing each recommendation. Principals can use this guide to consider the stakeholders
theyneedtoengagewithinordertoachieveeachofthesepriorities.
31
32
VII.Conclusion
This report outlines twomajor recommendations aimed at improving the systems and culture of
low-performing schools and one additional recommendation for improving the MSV rubrics in
evaluating these schools. Within these two areas of systems and culture, we have highlighted
specific sub-recommendations for school leaders and provided examples of how these best
practices can be operationalized at the school-level. Specifically, we recommend low-performing
schools focus on building systems to support data driven instruction, behavior management,
wraparoundservices, and tutoringandenrichment.Werecommend that theseschoolsalso focus
onintentionallyfosteringastrongandpositivecultureintheareasofteachercommunicationand
empowerment,parentengagementandattendance.
Basedoninterviewswithhighperformingprincipalswhohaveturnedaroundschoolsthemselves,
it is not recommended that all best practices are implemented at once; instead they should be
implementedinasequencetheschool leaderbelievesbestsuitstheirschool. InAppendixVIIwe
haveprovided additional guidance for school leaders tousewhenbeginning the implementation
process that will enable them to determine which stakeholders they should partner with to
implementspecificaspectsoftheserecommendations.
In education, there is general consensus on turnaround best practices. This is apparent in the
indicators includedon theMSV rubrics and in the turnaroundbest practicesdocumentsusedby
DESE. However, where minimal gaps have been identified in the MSV rubrics, our third
recommendationseeks toaddress these.While thespecificsof this thirdrecommendationdonot
directly respond to the initial research question asked, we believe that strengthening the
monitoringandevaluationprocessesofschools,toensureallbestpracticesarecapturedinDESE’s
reviewofschoolsisimperativetoprovidingoptimalsupportandfeedback.
While consensus may exist on what best practices are, there is little available detailing what
implementation should look like,especially inschoolsservinghigh-riskchildren.Through lengthy
interviews with school leaders and teachers, we focused on identification of trends in
implementation. In thisdocument,wehaveprovideddetailsonhow to concretelyoperationalize
thesepotentiallyvaguenotionsofimprovingsystemsandculture.Coupledwithvividexampleson
the school-level, this document seeks to adds additional value to the existing literature by
highlightinghowlowperformingschoolscanbetterserviceatriskstudents.
The reality in Massachusetts, however, is that poverty and race are inextricably tied to school
performanceofmostpublic schools.Schoolswithhigh levelsofpovertyandminorityenrollment
arestillmorelikelythannottobeunder-performing,andtherealitiesofpovertycontinuetobring
obstaclestostudentlearning,bothinandoutofschool.Whilewebelieveourrecommendationswill
produce significant improvements in school performance, our children need much more. The
realities of poverty and race cannot be solely addressed by education reform; it requires a
comprehensive understanding of the types of poverty and the diverse challenges faced by our
studentsandfamilies.
33
AppendixI:References
Abdulkadiroglu,Atila;Angrist,Joshua;Dynarski,Susan;Kane,ThomasandPathak,Parag(2011).
AccountabilityandFlexibilityinPublicSchools:EvidencefromBoston'sChartersandPilots.The
QuarterlyJournalofEconomics126:2,699-748.
Barrera-Osorio,Felipe;Bertrand,Marianne;Linden,LeighandPerez-Calle,Francisco(2011).Improving
theDesignofConditionalTransferPrograms:EvidencefromaRandomizedEducationExperimentin
Colombia.AmericanEconomicJournal:AppliedEconomics3:2167-195.
Bettinger,EricP.,BridgetTerryLong,PhilipOreopoulos,andLisaSanbonmatsu(2012).TheRoleof
ApplicationAssistanceandInformationinCollegeDecisions:ResultsfromtheH&RBlockFAFSA
Experiment.TheQuarterlyJournalofEconomics127:3,1205-1242.
Bryk,A.S.,BenderSebring,P.,Allensworth,E.,Luppescu,S.andEaston,J.Q.(2010).“TheInfluenceof
CommunityContext.”InA.S.Bryk,P.SebringBender,E.Allensworth,S.LuppescuandJ.Q.Easton(Eds.),
OrganizingSchoolsforImprovement:LessonsfromChicago(pp.137-196).Chicago,IL:Universityof
ChicagoPress.
Chakrabarti,Rajashri(2014).IncentivesandResponsesUnderNoChildLeftBehind:CredibleThreatsand
TheRoleofCompetition.JournalofPublicEconomics110:124-146.
Chetty,Raj,Friedman,JohnandJonahRockoff(2014).MeasuringtheImpactsofTeachersII:Teacher
Value-AddedandStudentOutcomesinAdulthood.AmericanEconomicReview104:9,2633–2679.
Deming,David(2009).EarlyChildhoodInterventionandLife-CycleSkillDevelopment:Evidencefrom
HeadStart.AmericanEconomicJournal:AppliedEconomics1:3,111-134.
Duflo,Esther;Hanna,RemaandRyan,Stephen(2012).IncentivesWork:GettingTeacherstoCometo
School.AmericanEconomicReview102:4,1241-78.
Fairlie,RobertandRobinson,Jonathan(2013).ExperimentalEvidenceontheEffectsofHome
ComputersonAcademicAchievementamongSchoolchildren.AmericanEconomicJournal:Applied
Economics5:3,211-40.
Figlio,DavidandHart,Cassandra(2014).CompetitiveEffectsofMeans-TestedSchool
Vouchers.AmericanEconomicJournal:AppliedEconomics6:1,133-156.
Fredriksson,Peter;Ockert,Bjorn;andOosterbeek,Hessel(2013).Long-TermEffectsofClassSize.The
QuarterlyJournalofEconomics128:1,249-285.
FryerJr.,Roland(2011).FinancialIncentivesandStudentAchievement:EvidencefromRandomized
Trials.TheQuarterlyJournalofEconomics126:4,1755-1798.
Goodman,Joshua(2012).TheLaborofDivision:ReturnstoCompulsoryMathCoursework.HKSFaculty
ResearchWorkingPaperSeries12-032.
Goodman,Joshua(2014).FlakingOut:StudentAbsencesandSnowDaysasDisruptionsofInstructional
Time.NBERWorkingPaper20221.
Jackson,KiraboC.,Johnson,RuckerandPersico,Claudia(2014).TheEffectofSchoolFinanceReformson
theDistributionofSpending,AcademicAchievement,andAdultOutcomes.NBERWorkingPaper20118.
Kuhn,PeterandWeinberger,Catherine(2005).LeadershipSkillsandWages.JournalofLaborEconomics
23:3,395-436.
Kuhn,PeterandWeinberger,Catherine(2005).LeadershipSkillsandWages.JournalofLabor
Economics.23:3,395-436.
Marsh,JulieA.,JohnF.PaneandLauraS.Hamilton.MakingSenseofData-DrivenDecisionMakingin
Education:EvidencefromRecentRANDResearch.SantaMonica,CA:RANDCorporation,2006.
http://www.rand.org/pubs/occasional_papers/OP170.html.
NancyFreyandDouglasFisher,“BetterLearningthroughStructuredTeaching:AFrameworkforthe
34
GradualReleaseofResponsibility,”AssociationforSupervisionandCurriculumDevelopment,
http://eric.ed.gov/?id=ED509133.
RolandG.FryerJ.InjectingCharterSchoolBestPracticesintoTraditionalPublicSchools:EvidenceFrom
FieldExperiments*QuarterlyJournalofEconomics(2014).2014;129(3):1355-1407.
2014_injecting_charter_school_best_practices_into_traditional_public_schools.pdf
Shen,J.(1997).Teacherretentionandattritioninpublicschools:EvidencefromSASS91.TheJournalof
EducationalResearch,91(2),81.Retrievedfromhttp://search.proquest.com.ezp-
prod1.hul.harvard.edu/docview/204546375?accountid=11311
Sugai,GeorgeandRobertR.Horner."APromisingApproachforExpandingandSustainingSchool-Wide
PositiveBehaviorSupport."SchoolPsychologyReview35,no.2(06,2006):245-259.
http://search.proquest.com.ezp-prod1.hul.harvard.edu/docview/219656206?accountid=11311
35
AppendixII:DetailedLiteratureReviewThisAppendixincludesdetailedsummariesandanalysesofliteraturerelevanttothisstudy.ThesedetailedliteraturereviewsaresummarizedinSectionIVofthereport.
1-SystemsandStructuresinSchools
Studies on the impact of systems and structures in schools focus on the allocation ofresources, thedevelopmentof programs, and external partnerships. First, studieson theimpactofresourceallocationinschoolsshowvaryingeffectsonstudentoutcomes.First,anincrease in instructional timeprovided(Dobbie&Fryer2011)andadecrease instudentabsences (Goodman 2014) are associated with significant increases in studentachievement. Second, a decrease in class size is also associated with increases in bothcognitiveandnon-cognitiveoutcomes(Fredrikssonetal.,2013),measuredbycomparingstudents above and below a particular threshold for class size. Furthermore, a studyconducted by Fryer to inject charter school practices in traditional public schools alsoshowed that by improving human capital (and removing ineffective teachers andadministrators),providingadditionaltutoringinterventionsbasedondataandusingdatato drive instruction is associated with closing the math achievement gap in only threeyears. Finally, a study conducted in Chicago Public Schools by Bryk demonstrated anassociation between outside factors, such as collective efficacy of the community, theavailabilityofoutsideconnections,andvariouscommunityfactorssuchascrimerateandpercentageofstudentsneglectedorabusedwithimprovementsbothinstudentoutcomesand student attendance. This suggests that student achievement (and thus, schoolperformance)isnotonlyaresultofactionsandactivitieswithinaschool,butthatwithinthecommunityexternaltotheschool.Furthermore, studies on returns to schooling and schoolprogrammingdemonstrate thattherearefactorsthatmaynotbecapturedinouranalysis.Forinstance,astudyconductedexaminingtheHeadStartprogramshowsitseffectsonlowincomestudentswhogoontoelementaryschool.Participationinprogramslikethisisnotevidencedinourdata,sothismay present a confounding factor. Moreover, this study highlighted significant othervariables,includingmeasuresofamother’sintellectualabilities,whichincreasedtheabilityof students tomaintain increases in test scores for example.The longer-term impacts ofreturns to pre-Kindergarten and early education programs may be a systematic factoracrosstheStateofMassachusetts, thatcannotbeadequatelyaddressedoraccountedforwithintheboundariesofthepresentstudy.
FinancialresourcesTheEffectofSchoolFinanceReformsontheDistributionofSpending,Academic
Achievement,andAdultOutcomes.C.KiraboJackson,RuckerJohnson,ClaudiaPersico.Summary:Thisstudyexaminedtheeffectofschoolfinancereforminitiativesmeanttodecreasethespendinggapbetweenhighandlow-incomeschooldistricts.Itexaminedfourtypesofschoolfinancereforms:adequacy-basedcourt-orderedreforms(basedonargumentthatresourcesforlow-incomedistrictswereinsufficient),equity-basedcourt-orderedreforms(basedonargumentthatresourcesbetweenhighandlow-incomedistrictswereinequitable),reformsthatentailhightaxprices(suchasspendinglimits),andreformsthatentaillowtaxprices(incentivestoincreaselocaltaxes).Itusedpaneldata
36
toexaminetheeffectsofthesereformsonschoolspendingforbothhighandlow-incomedistricts.Finally,italsoexaminedtheeffectsofhigherspendingresultingfromthesereforms,andhowtheyimpacthighschoolgraduationratesandadultoutcomes.Conclusion:Thisstudyfoundthatwhileallreformsreducedinequalityinspending,theydifferedinpracticalimpacts.Adequacy-basedcourt-orderedreformsincreasedoverallspendingforallschools,whileequity-basedcourt-orderedreformshadlittleeffectonoverallspendinglevels.Thestudyshowedthatwhileequity-basedcasesreducedinequalitybyaround$800perstudent,italsodecreasedtheabsolutelevelofspendingbyaround$500perpupil.Ontheotherhand,inadequacy-basedcases,thegaponlydecreasedbyaround$400perstudent,buttheabsolutelevelofspendingincreasedbyover$1,000forlow-incomedistricts.Furthermore,reformsthatentailedhightaxpricesreducedlong-runspendingbyalldistricts,whilethosethatentailedlowtaxpricesincreasedspendinggrowth,especiallyforlow-incomedistricts.Spendinglimitpolicies,forexample,didreduceinequality,butitdecreasedlow-incomedistrictspendingbyalmost$900perstudentinyears10-20.Reward-for-effortmatchinggrants,however,increasedoverallspendingbyaround$100perstudent,withgreaterincreasesforlower-incomedistricts.Thestudyfurtherfoundthata20%increaseinper-pupilspendingperyearforall12yearsofschoolingledtoa0.9morecompletedyearsofeducation,25%higherearnings,anda20percentagepointdecreaseinadultpoverty.RelevancetoPAE:Theseresultsaresomewhatrelevant,asanexaminationofMassachusetts’schoolfinancereforms,andthesubsequenttrendsindistrictspending,couldexplainInstructionaltime&studentabsencesDobbie,W.andR.G.Fryer(2011).Gettingbeneaththeveilofeffectiveschools:Evidence
fromNewYorkCity.WorkingPaper17632,NationalBureauofEconomicResearch.Hoxby,C.M.andS.Murarka(2009).CharterschoolsinNewYorkCity:Whoenrollsand
howtheyaffecttheirstudents’achievement.WorkingPaper14852,NationalBureauof
EconomicResearch.Mostacademicstudiesofinstructionaltimearguethatincreasedinstructionaltimeandlongerschooldayscontributetoimprovedstudentoutcomes.HoxbyandMuraka(2009)arguethatalongerschoolyearhasasignificanteffectonstudentachievement(eachschooldayincreaseisassociatedwitha0.02standarddeviationincreaseinstudentachievement).ThisfindingisfurthersubstantiatedbystudiesbyDobbieandFryer(2011).FlakingOut:StudentAbsencesandSnowDaysasDisruptionsofInstructionalTime.
JoshuaGoodman.Summary:Thisstudyusesstudentandschoolfixedeffects,aswellasinstrumentalvariables,toexaminetheeffectofstudentabsencesandschoolclosuresonstudentachievement.Thestudyreliesontheassumptionthatmoderatesnowfallleadstostudentabsences,whileheavysnowfallleadstoschoolclosures.Conclusion:Thisstudyshowedthatstudentabsencesareassociatedwithdecreasesinstudentachievement,whileentireschoolclosuresdonotimpactstudentoutcomes.Itfoundthateachadditionalabsencewasassociatedwitha0.05standarddeviationdecrease
37
inmathachievement,butthattherewasnosignificantimpactfromeachadditionaldayofclosure.Thestudyexplainsthatthisisconsistentwithteachingpracticeswherecoordinateddisruptionssuchasschoolclosuresarewell-managed,butaredifficulttoadapttosituationswhereindividualstudentsareabsentandcausespecificdisruptions.Thisisbecauseondayswherestudentsareallabsent,teacherscanadjustinstructiontorecovermisseddays,butondayswhereindividualstudentsareabsent,theyarelikelytomissthematerialalreadycovered.RelevancetoPAE:ThisishighlyrelevanttoourPAE,asstudentabsencescouldlikelybeakeyfactorindeterminingstudentperformance.Themoredaysstudentsareatschool,themoreinstructiontheyreceive.WewouldbeinterestedinstudyingstudentabsencedataacrossMassachusetts,andexaminewhetherLevel1/2and4/5schoolsservingsimilardemographicshavesimilarattendancecharacteristics.TeacherqualityMeasuringtheImpactsofTeachersI:MeasuringBiasinTeacherValue-Added
Estimates.Chetty,FriedmanandRockoff.ANDMeasuringtheImpactsofTeachersII:TeacherValue-AddedandStudentOutcomesin
Adulthood.Chetty,FriedmanandRockoff.ThisstudyisnotrelevanttoourPAE,asitaddressesadultoutcomes,andnotimmediatestudentachievement.CurriculumGoodman,Joshua(2012).TheLaborofDivision:ReturnstoCompulsoryMath
Coursework.HKSFacultyResearchWorkingPaperSeries12-032.ThisisnotrelevanttoourPAEasitmeasurestheimpactofstate-levelcurriculumrequirements.SincewearestudyingdifferentialimpactswithinMassachusetts,thefindingsofthisstudyarenotrelevanttoourscope.ClassSizeFredriksson,Peter;Ockert,Bjorn;andOosterbeek,Hessel(2013).Long-TermEffectsof
ClassSize.TheQuarterlyJournalofEconomics128:1,249-285.Summary:ThisstudyusesaregressiondiscontinuityandIVdesigntoexaminetheeffectsofclasssizesbetweentheagesof10-13forstudentsinSwedenontheirachievementatages13,16and18.Itfurtherexamineslongtermeffectssuchaseducationalattainment,wagesandearningsbetweentheagesof27-42.Conclusion:Thestudyfindsthatsmallerclasssizesbetweentheagesof10-13arebeneficialforbothcognitiveandnoncognitiveoutcomesatage13,andforacademicachievementatage16.Itfindsthatclassesabovetheclasssizethresholdareassociatedwithastatisticallysignificant0.033standarddeviationdecreaseincognitiveachievementatage13,a.0265standarddeviationdecreaseinnon-cognitiveachievementatage13,andadecreaseof0.0233standarddeviationsinacademicachievementatage16.
38
RelevancetoPAE:Thisstudysuggests,particularlyfortheagegroupwearestudying,thatclasssizedoesaffectstudentachievement.WhilethisstudywasdoneinSwedenandholdslessexternalvalidityfortheUnitedStates,thedemographiccharacteristicsofbothcountriesaresimilarenoughthatthesefindingscouldbeextrapolatedtoanAmericancontext.Wewillthereforeattempttofocusonclasssizeasapossiblevariableanddeterminantofschoolperformance.TechnologyFairlie,RobertandRobinson,Jonathan(2013).ExperimentalEvidenceontheEffectsof
HomeComputersonAcademicAchievementamongSchoolchildren.American
EconomicJournal:AppliedEconomics5:3,211-40.Summary:Thisstudyrandomlyselectedstudentsfrom5Californiaschooldistrictsthatdidnotpreviouslyhavecomputeraccessathome,andstudiedchangesinperformancebeforeandaftercomputersweregiventothem.Computerswereprovidedduringthesecondquarteroftheschoolyear;thusfirstquarterdatawasusedaspre-treatmentresults,andthird/fourthquarterdatawereusedaspost-treatmentresults.Theeffectsofcomputeraccesswerethenmeasuredagainststudents’outcomesontests,attendance,discipline,andsurveysoneffort.Conclusion:Thisstudyfoundthatthereisnosignificanteffect,andinmanycases,noeffectatall,ofcomputeraccessathomeonstudentoutcomes.Furthermore,studentsevenreportthathavingacomputerathomedoesnotincreasethelikelihoodthattheywilluseitforwordprocessingorresearch.Itfurtherdoesnothaveanyeffectonacademicoutcomes.ThisstudyisnotrelevanttoourPAEasitaddressestheeffectofhomecomputers,whichoutofthescopeofourstudy.Ourstudyonlystudiesdistrictandschoolpracticeswithintheircontrol;theprovisionofhomecomputersisnot,andthereforeisoutsidethescopeofthisstudy.ReturnstoSchoolingDemingDavid(2009)EarlyChildhoodInterventionandLife-CycleSkillDevelopment:
EvidencefromHeadStartAmericanEconomicJournal:AppliedEconomicsSummary:Thisstudyexaminesthelong-termimpactofHeadStart,anationwidepreschoolprogramforpoorchildren.UsingdatafromtheNationalLongitudinalSurveyofYouth,theauthorcomparessiblingswhodifferedinparticipationintheprogram.Hefindsthattherearemoderatetolargelong-termimpacts,intermsofclosingthegapbetweenchildrenwithmedianandbottomquartilefamilyincomes,whichiscomparabletoothermodelprograms.DetailsoftheStudy:ThedatausedwasfromtheNationalLongitudinalMother-ChildSupplement,trackingparticipantsfrom1984to1990.Thewithin-familydifferencesmodel,andcontrolforothervariantssuchasmaternalworkhistoryorchildcarearrangements.Findings:Thelong-termimpactofHeadStartisabout0.23standarddeviationsonasummaryindexofyoungadultoutcomes,withlargerimpactsforrelativelydisadvantagedchildren.Thisisaroundone-thirdofthegapbetweenthebottompermanentincome
39
quartileandthemedianintheCNLSYsample;itis80%aslargeasgainsfrom“model”preschoolprograms.Resultsarerobusttovariousalterationstothestudydesign.Thestudycommentsthatwhiletestscoresarecommonlyusedbenchmarks,thelinksbetweentheseandadultoutcomesisnotwellunderstood.Thestudythereforealsoshedlightonlife-cyclebenefitsofearly-skillformation.Thestudyshowsthataninitialtestscoregain,atages5-6years,fadesoutby11-14,andthisisparticularlysowithAfricanAmericanandverydisadvantagedchildren.However,thesechildrenexperiencethelargestlong-termbenefits.Thestudyhighlightsthefade-outeffectsontestscoresforstudentswhosemothershadlowAFQTscores.Forchildrenwhosemothersscoredonestandarddeviationbelowtheaverageonacognitivetestscore,thelong-termeffectofHeadStartis0.28standarddeviations,andyettheirnettestscoregainisessentiallyzero.Thus,aprojectionoffuturebenefitsforthesechildrenbasedsolelyontestscoregainswouldgreatlyunderstatetheimpactoftheprogram.Relevance:Thisstudymaybeparticularlyrelevantinhighlightingconfoundingfactorsintermsofwhichschoolshavehighertestscoresormoresuccessfuloveralloutcomesdespitedemographics.Participationinheadstartbyasignificantportionofthestudentpopulationmaybeanimportantfactorthatweshouldacknowledgewouldbeidealtocontrolfor.Inaddition,italsoshowstheimportanceofotherfactorsnotcapturedbyourdata,suchastheintelligenceofthemother,intermsofstudentsuccessandabilitytoretainbenefitsfromlearning.CardDavid(1993)UsingGeographicVariationinCollegeProximitytoEstimatethe
ReturntoSchooling,PrincetonUniversityThisstudyisnotrelevanttoourPAEinsofarasitisexaminingthecausaleffectofschoolonlaterincomeandearningsinlife.Byprovidingevidencethateducationdoeshaveapositiveimpactontheearningsofmalesitgroundsthecasefortheimportanceofeducation,butthisdoesseemasrelevant.Other:Gelber,Isen,Kessler(2014)TheEffectsofYouthEmployment:EvidencefromNewYork
CitySummerYouthEmploymentProgramLotteries.NationalBureauofEconomic
ResearchSummary:NotrelevanttoourPAEasitanalyzestheeffectsofemploymentprogram;notinthescopeofimprovingeducationoutcomes.
2–HumanCapital
Studies on human capital focus on teacher incentives and teacher development. On theteacher level, performance pay is a type of incentive that is highly debated.WashingtonD.C.’s city-wide IMPACT system, a robust teacher evaluation system coupled withperformancepay,hasbeenfoundtoincreasetheperformanceoflowperformingandhighperforming teachersaswell as encourage strong teachers to remain in the schools (Dee,2015).While aperformancepay system isup to the stateordistrict level to implement,certain characteristics of such a teacher evaluation system could be implemented in
40
individualschools.Forexample,theteacherevaluationsysteminWashingtonD.C.includesfrequentobservationandfeedbackonclassroombestpractices.Intermsofstaffculture,astudy(Shen1997)alsodemonstratedassociationsbetweenfourkeyelementswithincreasedteacherretention.Thoseinclude:1)hiringmoreexperiencedteachers;2)increaseteachersalaries;3)emphasizetheintrinsicmeritsofteaching;and4)empoweringteachersininfluencingschooldecisionsandpolicies.
Teacher/LeaderAutonomyAbdulkadiroglu,Atila;Angrist,Joshua;Dynarski,Susan;Kane,ThomasandPathak,
Parag(2011).AccountabilityandFlexibilityinPublicSchools:EvidencefromBoston's
ChartersandPilots.TheQuarterlyJournalofEconomics126:2,699-748.Summary:Thispaperisfocusedonthedifferencesinstudentachievementbetweencharterschoolsversuspilotschools.Charterschoolsareautonomousentitiesthatarenotboundbyunioncontracts.Pilotschoolsareinnovationschoolsthatarestillboundtounioncontracts.Thisaffectscompensation,hours,expectations,etc.Thisdoesnotnecessarilyfitnicelyintoan“incentive”categoryperse,butcouldspeaktothebenefitsofautonomyofbothleadersandteachers.Inthatway,autonomythenbecomesanincentive-yourelinquishunionboundcontractsandsecurityformoreautonomyandflexibility.RelevancetoPAE:Wecouldargueformoreautonomyintheseschoolsforprincipalsandteachers,andalsolongerschooldays,butthatisslightlyastretchhere.Itisdifficulttoknowhowmuchautonomyeachcharterprovidesetc,butthelengthofstudentschooldayisabsolutelyrelevant.Duflo,Esther;Hanna,RemaandRyan,Stephen(2012).IncentivesWork:Getting
TeacherstoCometoSchool.AmericanEconomicReview102:4,1241-78.Summary:TeacherabsenteeisminIndia,andmanydevelopingcountries,isalargeissue.Toaddressthis,thisstudymonitoredandcompensatedteachersfromincreasedattendanceatschool.Theymonitoredviacamerasandcompensatedfinancially.Thiswasarandomizedcontroltrial.Theteacherstreatedthoughareinnon-traditionalschoolswhoarenotpartofthepoliticallyactiveteacher’sgroup.Theyarepara-teachersinstead,andtheschoolsarerunbynonprofitgroupsetcratherthanthegovernment.Conclusion:Thetreatmentgroup’sabsenteeismfellby21percentagepointsandstudenttestscoresincreasedby0.17standarddeviations.Theyfoundthatteachersrespondstrongly(highelasticity)tofinancialincentivesinthismodel.Becausetheincentiveswereprovidedfornon-traditionalschoolstoteacherswhodonotmakeupthelargepoliticallyactiveteachingblock,itishardtotellwhetherthiswouldworkwithregularteachers.RelevancetoPAE:Financialincentivesseemtoworkintermsofgettingeliminatingabsenteeism.Thisthoughiscontext-specific:IndiahasanunusuallyhighabsenteeismproblemsotheeffectonthemarginmaynotbeaslargeinplacesliketheU.S.whereabsenteeismisnotassevereonaverage.Additionally,thiswasimplementedwithnon-traditionalschoolswhoemploypara-teachers(signedonyearlycontracts)ratherthanthepoliticallyactiveteachingblockintraditionalschools.Thisisequivalenttounionizedversusnon-unionizedintheU.S.andbyextensionperhaps,charterversusnon-charter.One
41
thingtoconsiderisfinancialincentivesforadditionalresponsibilitiesortasks-moredataanalysisforXamount?moreafter-schooltutoringforYamount?takeoncoachingthreeteachersforZamount?WewouldneedtolookatU.S.specificstudieswithfinancialincentivestotrulyunderstandhowpowerfulfinancialincentivescanbeinthisculturethough.
3–Social-EmotionalandAcademicInterventionsforStudents
Non-cognitiveskilldevelopmentisadifficultoutcometomeasure.Onelarge, longitudinalstudythoughlookedattheassociationbetweenexhibitingleadershipskillsinhighschooland wages. It found that those who occupy two leadership positions in high school onaverage earn 6.9 percent higher wages ten years later and are more likely to holdmanagerial positions in life (Kuhn,2005).This study isunique inmeasuring leadershipsskillsbefore entering the labormarket and so is relevant toourquestionof school-levelbestpractices.Importantly,thestudyfoundthatleadershipskillsarecapturingsomeeffectassociatedwith“sociability”andnotcognitiveability.Howdoyouteachleadershipingradeschool?Thereislittleconsensusbehindananswertothisquestion.Wecaninferfromthisstudy though that providing students in early grades with the skills needed to occupyleadership positions in high school could lead to improved outcomes later in life. Moregenerally, teaching leadership skills in school could have profound effects on long-termstudentoutcomes.Intermsoftheimpactofnon-cognitiveskilldevelopmentonlowincomeorminoritystudents,therearenomajorstudiesonthistopic.Secondly, the Department of Elementary and Secondary Education in Massachusettsconducted a study on the effect ofWrap Around Zones (WAZ) on student performance.Thesearestate-fundednon-academicsupportsthataddresstheclimateandcultureoftheschool as well as students’ social and emotional capacity. They found that significantimprovements in standardized test scores, especially for third and fourth grades, but noimprovement in suspension rates, attendance or retention (DESE, 2012). Therefore, theWAZsupportsareeffective for increasing immediateacademicperformance for studentsfacing social or emotional challenges, but it is less clear whether this affects long-termbehavior.Lastly, studies on incentives for improving student outcomes range from intrinsic andextrinsic rewards for students. To identify school-level best practices, looking at teacherspecific and student specific incentives are most relevant because they are under thecontroloftheschoolratherthanthelocalorstatelevel.Onthestudentlevel,onaverage,financial incentives forpoorandminority studentsare statistically zero, and likelynot asustainable intervention(Fryer,2011).Young,secondgradestudentswere foundtoreadmore when they were paid for reading more books. Other students, who were offeredpayment for increasing their grades or test scores, were not found to increase theirperformance.Therationaleisthatstudentsdonotcompletelyunderstandhowtoincreasetestscoresorgrades,butdoknowhowtoreadthebook.Thisfindingsupportstheideathatstudentsrespondpositivelytoincentivesforspecificactions,suchasreadingmorebooksor memorizing vocabulary words, that can lead to better outcomes like test scores orgrades.Ontheschool-level, thiscould take the formofdailyrewardsystems,publicdatacollectionandtracking,oracademiccompetitions.
42
Non-cognitiveSkillDevelopmentKuhn,PeterandWeinberger,Catherine(2005).LeadershipSkillsandWages.Journal
ofLaborEconomics23:3,395-436.Summary:Thisisalongitudinalstudythatregressesadultwagesonindicatorsofleadershipskillstakenbeforelabormarketentrywhilecontrollingforcognitivemeasures(testscoresprimarily),familybackground,andhighschoolfixedeffects.Thepurposeistoevaluatetheadditionaleffectof“noncognitive”skillssuchasleadershipexhibitedinhigh-schoolonfutureearnings.Thefundamentalassumptionisthattheseleadershipindicators(captain,classpresident,etc)arebasedrealleadershipskill(ie.thecoachchoosesacaptainbecausetheyexhibitleadershipskillsratherthannepotismorsomeotherreason).Additionally,theyalsousedaself-surveytoassessleadershipskillsasanindicator.Conclusion:Studentswhoexhibitleadershipskillsinhighschoolearnsignificantlymore10yearslater-6.9%higherwageonaverageifastudentwasbothapresidentandcaptainofaclub/team.Thisisnotthecaseforteamorclubmembersonlywhichsupportstheirhypothesisthatitstheleadershipskills(ratherthantheotherskillsacquiredfrombeingonateamorinacertaintypeofclub)thatdifferentiatesthesestudents.Studentswhoinhabitleadershippositionsinhighschoolarealsomorelikelytoholdmanagerialpositions11yearslaterinlife.Overall,wecanconcludethatonaverage,controllingforfamilyandhighschoolandcognitivevariables,studentswhoexhibitleadershipskillsinhighschoolgoontoearnhigherwagesonaveragehaveahigherprobabilityofoccupyingmanagerialpositionslaterinlife.Forourpurposes,itisdifficulttohowtheseleadershipskillsareattained-innateortaught?Andwhen-K-8?orduringhigh-schoolleadershipopportunities?Ifweassumethatstudentsarechosenorelectedtoleadershippositionsorperceivethemselvesasleadersbasedonrealleadershipskills,thenthissuggestsleadershipskillsareeitheracquiredormanifestedbeforetheyoccupysaidleadershipposition.Ifthatisthecase,thenthereisanargumenttobemadethatleadershipissomethingthatcanbedevelopedduringadolescentandyoungadultyears.Thestudyfindsthatstudentswhoattendschoolswithmoreleadershipopportunitiesearnmorelateronaverage.Theeffectthoughisonlystatisticallysignificantforstudentswhoexhibitleadershipskillsbeforetenth-grade,suggestingacomplementaritybetweeninnateandacquiredleadershipskills.RelevancetoPAE:Wecouldarguestudentsneedmoreleadershipopportunitiesinschool,butalsothattheyshouldreceivemoreleadershiptrainingearlieron.Ingeneral,thisstudyismostrelevanttohighschool,butwecouldextrapolatetoearliergradesandtheleadershipskillsandopportunitiesbeingpresentedtostudentsthere.
• Movingforward,weshouldtakenoteofschoolsthatdeliberatelydevelopleadershipskillsinstudentsorprovideopportunitiesforleadershippositionstobeheld.
• FallsRiverVivieroshasimplementedcurriculumaroundcharacterbuilding,butnotdistinctlyleadershipskills.Theresponsiveclassroommaycultivateleadershipsskills(empathy,self-control,patienceetc)butitdependsonhowyoudefineleadershipskill.
43
TransparencyofInformation/PersonalAssistanceBettinger,EricP.,BridgetTerryLong,PhilipOreopoulos,andLisaSanbonmatsu
(2012).TheRoleofApplicationAssistanceandInformationinCollegeDecisions:
ResultsfromtheH&RBlockFAFSAExperiment.TheQuarterlyJournalofEconomics
127:3,1205-1242.ThisarticleisirrelevanttoourPAE.ItisconcernedwiththeincreaseincollegeenrollmentandfinancialreceiptwhenprovidedpersonalassistanceandinformationregardingcollegecostandtheFAFSA.Whileitprovidesexcellentinsightintothebenefitsofprovidingpersonalassistanceforincreasingparticipationincertainprograms(healthcare,governmentsupports,etc),thisisnotrelevanttoourPAEeither.Thiscouldbeusedasevidenceforincreasingpersonalassistanceforparentsinschoolchoicemodels,soperhapsamentioninafootnoteoranappendix,butotherwise,thisresearchdoesnotapplytoourPAE.
4–Culture&Incentives
School culture envelopes the culture built for both students and teachers. The mostrelevant studies look at specific best practices on the school-level. In a study by RolandFryer,heanalyzestheimpactofinjectingempiricallybackedcharterschoolbestpracticesinto traditional public schools that were previously failing. The five best practices thatwere“injected”intotraditionalpublicschoolswereincreasedtimeofschooldayandyear,improved human capital (teachers and leadership), more student level differentiationthrough tutoring, frequent use of data to inform instruction, and a culture of highexpectations. The schools that implemented thesepractices increasedmath achievementby0.15 to0.18standarddeviationsperyear. Inotherwords, theycouldclose theblack-whiteachievementgap inmathwithinthreeyears.Theeffectsonreadingweremarginalandstatisticallyinsignificant(Fryer,2014).Thestudentpopulationswereprimarilypoor,BlackandHispanicstudents.Furthermore,otherstudieshavealsoshownthatdevelopingconsistentschoolwidestructuresforpositivebehaviorsupports,specifically,contributetoimproved student behavior. Because of these significant gainswith a student populationsimilartothepopulationwearestudyinginMassachusetts,wehavereasontobelievethatschoolsimplementingsome,ifnotall,ofthesebestpracticesarebetterequippedtoserveatriskstudents.
RolandFryer.InjectingCharterSchoolBestPracticesintoTraditionalPublicSchools:
EvidencefromFieldExperiments.HarvardUniversity.ExaminestheeffectivenessofinjectingcharterschoolbestpracticesintotraditionalpublicschoolsinHouston,Texas.ThebestpracticesweredeterminedfromFryer’sotherpapercalled“GettingBeneaththeVeilofEffectiveSchools:EvidencefromNewYorkCity”.Thispaperfoundsignificantincreasesineffectivenessfortraditionalresourcebasedmodels(classsize,perpupilexpenditure,non-certifiedteachersetc)doesnotexplainvariation.Thefollowingfivepractices,ascollectedinfouryearsofcasestudiesandqualitativeanalysis,explainoverfiftypercentofvariationinhighperformingcharterschools.
44
BestPracticesinclude:
• IncreasedTime.Lengthenedschooldaysbyonehourandschoolyearsbytendays.StudentswereincentivizedtoattendschoolonSaturdays.
• ImprovedHumanCapital.19of20principalswereremovedaswellas46%oftheteachingstaff.
• MoreStudentLevelDifferentiation.Allfourth,sixthandninthgraderswereprovidedatutorandothergradeswereprovidedadditionaltutoringbasedontheMATCHmodel.
• FrequentUseofDatatoInformInstruction.Interimassessmentswererequiredeverythreetofourweeks,threecumulativeassessmentswereprovided,andassistanceinanalyzingandpresentingdatawasprovidedtoteachers.
• ACultureofHighExpectations.Providedclearexpectationsforleadership,arubricforschoolandclassroomenvironment,andstudent-parent-teachercontracts.Specificstudentperformancegoalsweresetforeachschoolandtheprincipalwasheldaccountableandincentivizefinancially.
Conclusion:Allstatisticalapproachesleadtothesamebasicconclusions.Injectingbestpracticesfromcharterschoolsintolowperformingtraditionalpublicschoolscansignificantlyincreasestudentachievementinmathandhasmarginal,ifany,effectonEnglishLanguageArtsachievement.Themathresultsweresuchthattheycouldeliminatetheachievementgapinthreeyearstime.
Bryk,A.S.,BenderSebring,P.,Allensworth,E.,Luppescu,S.andEaston,J.Q.(2010).“The
InfluenceofCommunityContext.”InA.S.Bryk,P.SebringBender,E.Allensworth,S.
LuppescuandJ.Q.Easton(Eds.),OrganizingSchoolsforImprovement:Lessonsfrom
Chicago(pp.137-196).Chicago,IL:UniversityofChicagoPress.StudyoftestscoreandschoolattendancegrowthinChicagofoundnumeroussocialfactorsthatleadtosubstantialgrowth:
1. Religiousparticipation2. Collectiveefficacy(communitysocialcapital)3. Outsideconnections4. Crimerateofimmediateneighborhood5. Percentageofstudentsabusedorneglected
Trust:whiletrustdoesnotcontributedirectlytostudentlearning,itcreatesthefabricbywhichschoolimprovementstrategiesbecomesustainable,andfacilitatescoreorganizationalchange.
• Trustallowsfortheexpansionandsustainabilityofessentialsupports• Smallerschools/academiesenablerelationaltrust• TrustisbuiltonteacherANDstudentretention
45
CompetitionChakrabarti,Rajashri(2014).IncentivesandResponsesUnderNoChildLeftBehind:
CredibleThreatsandTheRoleofCompetition.JournalofPublicEconomics110:124-
146.Summary:TheyusedRegressionDiscontinuityDesign(RD)totestwhetherAYPincreasedperformanceinmath,reading,andattendance.ThepurposewastoinvestigatewhetherhighstakestestingandthethreatofclosurebyNCLBencouragedschoolsto:
• Focusonhighstakesstakessubjects(readingandmath)atthesacrificeoflowstakessubjects?
• Focusonlyonstudentsrightaroundthecutoffforproficiency?Withexpenseforthosegroupsofstudentsattheend?
• Focusmoreonsub-groups?• DoschoolsrespondtocompetitioninducedbyproximitytoAYPadequateschools?• Increasetestparticipation,attendance,andgraduation?
Conclusions:
• Focusonhighstakesstakessubjects(readingandmath)atthesacrificeoflowstakessubjects?NO.Infact,spillovereffectsfromfocusingonreadingseemedtohaveincreasedperformanceinlanguageartsdespitebeingalow-stakessubject.
• Focusonhighstakestestinggrades?YES.Therewasadecreasedinthirdgradebecausefourthgradeisatestingyear.
• Focusonlyonstudentsrightaroundthecutoffforproficiency?Withexpenseforthosegroupsofstudentsattheend?NO.
• Focusmoreonsub-groups?NO.Actuallyledtodeteriorationofperformanceforeconomicallydisadvantagedandspecialeducation.Overall,performanceforsub-groupofWhitesincreasedthemost.
• DoschoolsrespondtocompetitioninducedbyproximitytoAYPadequateschools(duetothreatofstudentstransferringtoAYPpassschools)?YES.AYPfailedschoolsfacingmorecompetitionperformedbettermorebroadlyandstrongly.Theyachievedstatisticallysignificantincreasesinattendance,graduationrates,andtestparticipationaswellasgreatershiftstotherightofstudentsbeingproficientoradvancedproficient.Thecredibilityofthethreatswasalsofoundtomatter.Additionally,itwasfoundthatdistrictsdidnotallocatemoreresourcestowardsAYPfailedschoolseithersoitisnotaresourceallocationissue.
• Increasetestparticipation,attendance,andgraduation?NO.• Overall,schoolsincreasedreadingperformanceandmathperformanceiftheyfailed
onthosemeasurestobeginwith.Therefore,itseemstheyrespondedtotheincentiveofnotfailingAYPagainbasedonwhattheyfailedbefore.Thisdidnotimprovetheperformanceoverallthoughforothersubjectsorothernon-testinggradesforexample.
RelevancetoPAE:Sincethesearedistrict-levelandstate-levelresults,itisdifficulttopulloutbestpractices.Thatbeingsaid,thenotionthateconomicallydisadvantagedstudentsandspecialeducationstudentssufferwhenhighstakestestingisinplaceisworthconsideringwhenschoolsdecidehow/when/whytoimplementtheirowntestinganddatacollection.Additionally,italsoencouragesschoolstotakespecialcareofthosespecialsub-groupsinpreparationforanupcomingstandardizedtest.Additionally,theideaofcompetitionwasveryinteresting.Inbothcases(mathandreadinginducedfailures)increasedcompetitionincreasedperformance.Forbestpracticeswithinschool,perhaps
46
thisfurtherencouragesdatacollectionbetweenclassroomsortheteacherobservation-feedbacksystem.Thefactthatperpupilspendingwasunassociatedwithincreasesinperformancealsoconfirmsthathypothesisthatthebestpracticesarefromwithinschoolsamongthepersonnelversusexternalvariables.Overall,becausethisismoreconcernedwithdistrict-levelandstate-levelpolicies,wecanonlyextrapolategeneralcharacteristicsofthesystemratherthandiscretebestpractices.Figlio,DavidandHart,Cassandra(2014).CompetitiveEffectsofMeans-TestedSchool
Vouchers.AmericanEconomicJournal:AppliedEconomics6:1,133-156.Conclusion:Increasedcompetitionthroughvouchersincreasesstudentperformanceandarenotassociatedwithstudentcompositionorresourceallocation.RelevancetoPAE:ThisisnotentirelyrelevanttoourPAEbecauseMassachussettsdoesnothaveavouchersystem,butthenotionofcompetitionasanincentiveisaninterestingone.Keyquestionstoaskare:
• Aretherewaystoincreaseasenseofcompetitionforschoolswithoutavouchersystem?Perhapsthelevelingsystemisadequate.
• Arecharterschoolsasufficientcompetitiveforce?• Aretherewaystoinduceasenseofcompetitivenesswithinschoolsperhaps?• Perpupilspendingonceagainisnotassociatedwithincreasedstudentperformance
-whatarebestpracticesthatDONOTcostadditionalmoney?
47
AppendixIII:MethodologyforQuantitativeAnalysisOurresearchinvolvedfourkeysteps:SchoolMatching,HypothesisDevelopment,CaseStudiesandQuantitativeReviewofFindings.TheSchoolMatchingphaserequiredidentifyingandgroupingschoolsthathavesimilarEconomicandMinorityStatus.TheHypothesisDevelopmentstageusesthesegroupingsandconductsdataanalysistoidentifycorrelativerelationshipsbetweenschoolpracticesandperformance,andhowthosedifferbetweenschoolswithinthesegroups.Furthermore,theCaseStudystagewaswherewetestedthehypothesescreatedthroughdataanalysis,andusedqualitativeresearch(casestudies,interviewsandfieldvisits)toevaluatethevalidityofourhypotheses,refiningthem,andconcludingbyidentifyingbestpracticesthatdifferentiatehigh-andlow-performingschoolswithsimilardemographiccharacteristics.Finally,theAnalysisofMonitoringSiteVisitReportswillexaminestrategiesemployedbyLevel4schoolsandtoidentifyareasofgrowthforthoseschoolsbasedonourhypothesisdevelopmentandcasestudystages.
StageI:SchoolMatching
TheSchoolMatchingstageinvolvesgroupingLevel1/2andLevel4/5schoolswithsimilarEconomicandMinoritystatustogether,inordertoconductacomparativeanalysisofdemographicallysimilarschools.Weidentifiedeconomicdisadvantage(FreeandReducedPriceLunchenrollment)andminoritystatusastwokeycharacteristicsforassignmentofgroups,asthesearecommonlyreferencedpredictorsofstudentperformance.ThesecomparisongroupsconsistofLevel1/2andLevel4/5schools,wheretheproportionofstudentsofeconomicallydisadvantagedstudentsattendingwaswithin3.5percentagepointsofeachother.AcutoffthresholdwasalsodeterminedbycomparingthedemographicdataofLevel4/5schools.OursampleonlyincludedLevel1/2schoolsthatmettheminimumthresholdofeconomicallydisadvantagedandminoritystudents,setatthe10thpercentilemarkoftheproportionofeconomicallydisadvantagedandminoritypopulationsamongLevel4/5schools.Weusedthe10thpercentilemarkinordertoeliminateanyoutliersamongLevel4/5schools;tobeincludedinthesample,Level1/2schoolsthereforemusthaveatleast53%economicallydisadvantagedpopulationor55%minoritypopulation.AfulllistingofthesampleschoolscanbefoundinAppendixIV.Analysisofthesecomparisongroupsrevealedthatsufficientnumbersofdemographicallysimilarschoolsforcomparisononlyexistamongstelementaryschools.Therewereveryfewmiddleandhighschoolsthatmetthethresholdforcomparison.Asaresult,ouranalysisandcomparisonfocusesmainlyonelementaryschools.Indeed,highperformingschoolswithbothhighpovertyandhighminoritypopulationsisananomaly.Thus,thisstudywillfocusonstrategiesthatfocusspecificallyonservingthesepopulations,andschoolsthathavedemonstratedsuccessdoingso.
StageII:FindingsfromQuantitativeAnalysisandHypothesisDevelopment
Withthematchinggroupscreated,weuseddatatocomparetheschoolswithinthosegroupsandidentifythepracticesthatdifferentiateLevel1/2andLevel4/5schoolswithineachgroup.Hypothesisdevelopmentwillfirstrelyonliteraturereviewofexistingworkonschoolreformtoidentifydriversofschoolimprovement.Wethenconductedananalysisofschool-leveldatatoestablishcorrelativerelationshipsbetweenpracticesandstudentoutcomes.
48
Basedontheliteraturereview,wedeterminedthefollowingindicatorsarepossibledriversanddeterminantsofschoolperformance.Wedidnotincludeanyindicatorsthatwouldbeendogenoustoschoolperformance,orwouldaffectaschool’sLevelratingthroughfactorsotherthanitself.AfulldescriptionofomittedvariablesandanexplanationfortheiromissioncanbefoundinAppendixV.Theselectedindicatorsforanalysiswere:
1) AttendanceRate2) Percentageofstudentintakesperyear(studentsthatenrolledafter1stdayof
school)3) Principalretentionrate4) Teacherretentionrate5) Percentageofstaffevaluatedperyear6) Percentageofteachersteachingintheirlicensedsubjectarea7) PercentageofteachersincoresubjectareasthatareHighlyQualified8) Percentageofstudentsdisciplined
Amultivariateregressionwasusedtoanalyzehoweachofthesefactorscorrelatewiththeprobabilityofaschoolbeingahighperforming(Level1or2)school,controllingforallothervariables.Analysissuggestedthatonlytwofactors–studentattendancerateandteacherretentionrate–hadastatisticallysignificanteffectonaschool’slikelihoodofbeingahigh-performingschool.Theregressionfunctionsarebelow:
(1) γ(DummyvariableforLevel1/2status)=β0+β1(AttendanceRate)
(2) γ(DummyvariableforLevel1/2status)=β0+β1(StudentIntakeRate)
(3) γ(DummyvariableforLevel1/2status)=β0+β1(PrincipalRetentionRate)
(4) γ(DummyvariableforLevel1/2status)=β0+β1(TeacherRetentionRate)
(5) γ(DummyvariableforLevel1/2status)=β0+β1(%ofStaffEvaluated)
(6) γ(DummyvariableforLevel1/2status)=β0+β1(%ofTeachersinLicensedArea)
(7) γ(DummyvariableforLevel1/2status)=β0+β1(%ofCoreTeachersHighlyQualified)
(8) γ(DummyvariableforLevel1/2status)=β0+β1(StudentDisciplineRate)
(9) γ(DummyvariableforLevel1/2status)=β0+β1(AttendanceRate)+β2(Student
IntakeRate)+β3(PrincipalRetentionRate)+β4(TeacherRetentionRate)+β5(%of
StaffEvaluated)+β6(%ofTeachersinLicensedArea)+β7(%ofCoreTeachers
HighlyQualified)+β8(StudentDisciplineRate)Theanalysisdemonstratedtheassociationsbetweenvariousindicatorsandtheprobabilityofaschoolbeingahigh-performing(Level1or2)school.Thisanalysissuggestedthatonlythestudentattendancerateandteacherretentionratehadastatisticallysignificanteffectonaschool’slevelstatus.Thefulltableofresultsisasfollows:
49
Table1:EffectofschoolindicatorsontheprobabilityofbeingaLevel1/2school
(allfiguresrepresentpercentagepointincreasesinprobabilityofbeingaLevel1/2school)
Variable (1) (2) (3) (4) (5) (6) (7) (8) (9)
Studentattendancerate 12.8*** - - - - - - - 7.4***
%studentintakes - 3.2*** - - - - - - -0.79
Principalretentionrate - - 0.28*** - - - - - 0.096
Teacherretentionrate - - - 1.4*** - - - - 0.86**
%staffevaluated - - - - - - - 0.033
%teachersinlicensedarea - - - - - 0.89** - - -1.1*
%coreteachershighlyqualified - - - - - - 0.97** - 0.094
%ofstudentsdisciplined - - - - - - - 2.5*** 0.80
StageIII:CaseStudies
Inordertoidentifybestpracticesspecifictohigh-performingschoolsandtosupplementourliteraturereviewandquantitativeanalysis,weconductedfieldvisitsandcasestudiesofLevel1and2schoolsinoursample.Wevisitedandinterviewedtheprincipal,schoolleadersandteachersatsixelementaryschools,andusedthosefindingstoidentifycommonthreadsandpracticesacrossoursampleofhigh-performingschools,thatwereeithernotevidencedbyorunavailablefromthequantitativedata.Ateachschool,weconductedinterviewswiththeprincipalandschoolleaders,includingassistantprincipals,instructionalcoaches,andgrade-levelleadteachers.Atsomeschools,separateteacherfocusgroupswerealsoconductedtogaugealignmentbetweentheadministrationandtheinstructionalstaff.Ourinterviewprotocolwasbasedonourinitialhypothesisdevelopment,andincludedquestionsaboutschoolculture,humancapital,familyandcommunityinvolvement,instructionanduseofdata,andservicesforstudents.ThefullinterviewprotocolcanbefoundinAppendixVI.Theeightschoolswevisitedorinterviewedwere:1. CarltonM.VivierosSchool,FallRiver2. NewtonSchool,Greenfield3. MaryLynchElementarySchool,Springfield4. WarnerSchool,Springfield5. KensingtonInternationalSchool,Springfield6. BelmontStreetCommunitySchool,Worcester7. CharlesSumnerElementarySchool,Boston8. HyannisWestElementary,Barnstable
50
StageIV:AnalysisofMonitoringSiteVisits
OneofthelimitationsofthisinterviewapproachwasthelackofadirectcomparisonpointwithLevel4or5schools.Whilewecouldidentifycommonaspectsofschoolcultureandpracticeassociatedwithhigh-performingschoolsinoursample,withoutaccesstoqualitativedataaboutLevel4and5schools,therewasnowayofconfirmingthesepracticeswereonesthatdifferentiatedhigh-performingfromlow-performingschools.Inordertoaddressthis,weconductedananalysisoftheMonitoringSiteVisitreports(MSVs)thatDESEconductsinordertoevaluatetheprogressofLevel4schools.TheseMSVsidentifystrategiesforperformanceandgrowth,basedonfourturnaroundpracticessetbyDESE,andindicatetheschool’splannedstrategiesandassesstheexecutionandimplementationofthosestrategies.ThefouroverarchingstrategieshighlightedbytheMSVsasareasforimprovementincluded:1. Leadership,SharedResponsibility,andProfessionalCollaboration2. IntentionalPracticesforImprovingInstruction3. Student-SpecificSupportsandinstructiontoAllStudents4. SchoolClimateandCultureEachofthesecategoriescomprisedofanumberofindicators,alongwhichschoolswereratedaseithershowing:“NoEvidence”,“Developing”,“Providing”,“Sustaining”or“CoherentImplementation”.Further,qualitativereportsofaschool’sachievementagainsttheseindicatorsisgiveninthedetailoftheMSV.Usingthismoredetailedanalysis,wesummarizedourfindingsfromthecasestudiesandMSVreportreviewsfor8keystrategies.
Limitationsofthestudy
Thoughbasedonrigorousanalysisofboththequantitativeandqualitativedataavailabletous,thisresearchissubjecttoanumberoflimitations.Whilethesearevaried,threemainareaspresentasparticularlyimportantwhenconsideringourfindingsandrecommendations.Thefirstisthesizeofthesampleavailable.Asnoted,theschoolsbeingexaminedareananomalytogeneralpatternsacrossdemographyandoutcomesineducation.Therefore,thestudy,bydesignissomewhatlimitedinitsinternalvalidity–thesamplewasnotsufficientlylargeenoughtoprovidestrongevidenceofthepatternsobserved.Similarly,theselectionforinterviewwasnotrandom.AllschoolsthatwereLevel1or2andhadsimilardemographicstotypicalLevel4or5schoolswereinvitedtoparticipateinthestudy.Onlythoseschoolsthatrepliedwereinterviewed;therefore,itispossiblethataselectionbiasinthesetypesofschoolsisalsoatwork.Nevertheless,giventhesmallnumberofschoolsintheoriginalpool,itisunlikelythiswillhaveaffectedresultstoodramatically.Asecondlimitationarisesoutofthenatureofthedataitself.Itbecamecleartous,fromspeakingwithprincipalsandteacherswhohadpreviouslyhadexperienceinLevel4/5schools,thatthecategoryof“economicdisadvantage”isawideone.ThismeansthatwhilesomeoftheschoolsthatareperformingattheLevel1/2standardlooksimilaronthefaceofthedata,toLevel4/5schools,theremaybemoreconsistent,substantivedifferencesthatreallyaccountforthedifferencesinperformance.Inparticular,thetypeofpovertyoreconomicdisadvantagediffersdependingonthesuburbandarea,whetheraschoolislocatedregionallyorinanurbancenter,forexample.ItwasnotedtousthatinLevel4/5schools,someofthesamestrategieshadbeenappliedwithoutsuccess,simplybecausethe
51
issuesfacingstudents,forexample,intermsofhowpervasiveviolenceintheircommunitywas,weresomuchmoreserious.Thethirdlimitationofthestudyissimplythatthereareanumberofareas,highlightedbytheliteraturereviewconducted,aswellasourschoolvisits,thatwedonothaveclearorsystematicdataabout.Alargeareathatwasnotexaminedindepthbyourteamwasthebudgetaryandfinancialconstraintsofdifferentschools.Similarly,therewasevidencethatdistrict-wideleadershipplayedaroleinmakingcertainaspectsofschoolculturemoreorlesspervasive.These,andmanyotheraspectsofschooloperationsarelikelyimportanttocontrolfororexaminetounderstandpreciselywhattheleversforsuccessare.
52
AppendixIV:SchoolSample
Level School Type EDIS%
Minority
%
2014TestScores:
Average
Proficientor
above
2014Test
Scores:
Proficientor
aboveinELA
2014Test
Scores:
Proficientor
aboveinMath
1
Lowell-SChrista
McAuliffeElementary Public 54.7 51.4 53 43 63
1
Springfield-AliceBBeal
Elementary Public 58.6 64.3 60 58 62
1
Fitchburg-Crocker
Elementary Public 61.4 56.3 62 64 60
1
Lawrence-South
LawrenceEast
ElementarySchool Public 66.3 89.3 54.5 44 65
1
Lowell-CharlotteM
MurklandElementary Public 67.5 33.1 51.5 40 63
1 FallRiver-JohnJDoran Public 66.8 34.4 43 43 43
1
Springfield-DanielB
Brunton Public 67.2 78.8 47 40 54
1
Lawrence-FrancisM
Leahy Public 66.4 97.4 40 31 49
1
WestSpringfield-PhilipG
Coburn Public 69.9 22.1 56.5 46 67
1
FallRiver-CarltonM.
ViveirosElementary
School Public 68.9 32.6 43.5 39 48
1
Worcester-UnionHill
School Public 71 68.6 46 42 50
1
Lawrence-GerardA.
Guilmette Public 70.7 95.8 54.5 46 63
1
Worcester-Belmont
StreetCommunity Public 71.5 67 37.5 36 39
1
Chicopee-GenJohnJ
Stefanik Public 77.9 80.3 59.5 56 63
1
Springfield-MaryM
Lynch Public 77.5 88 51.5 47 56
1 Boston-JosephHurley Public 72.3 78.4 62 59 65
2
Quincy-SnugHarbor
CommunitySchool Public 55.1 21.2 42 36 48
2 Boston-JosiahQuincy Public 56.8 32.2 61.5 54 69
2
NewBedford-ElwynG
Campbell Public 55 40.7 52.5 51 54
2
Peabody-WilliamA
WelchSr Public 55.1 42.9 51.5 52 51
2
NewBedford-SgtWmH
CarneyAcademy Public 55.8 48.7 54.5 54 55
2 Brockton-EdgarBDavis Public 55.3 75.3 43 48 38
2 Boston-CharlesSumner Public 57.7 85.9 45 40 50
53
2
FallRiver-WilliamS
Greene Public 59.3 23.1 43.5 40 47
2
Barnstable-Hyannis
WestElementary Public 59.4 38.4 55 49 61
2
Greenfield-Newton
School Public 63.6 21.9 53 36 70
2 Taunton-HHGalligan Public 63.2 36.1 59.5 59 60
2
Springfield-Frederick
Harris Public 63.9 75.3 52 41 63
2
NewBedford-JamesB
Congdon Public 64.7 52.5 45.5 42 49
2
Holyoke-LtClayre
SullivanElementary Public 66.3 79 37.5 39 36
2
Springfield-FrankH
Freedman Public 66.1 82.9 49 55 43
2 Springfield-Warner Public 67 78.4 64.5 53 76
2 Lawrence-JohnKTarbox Public 68.4 98.3 45 38 52
2
NewBedford-Carlos
Pacheco Public 72.1 62.6 49 38 60
2
Springfield-ArthurT
Talmadge Public 72.1 75.6 49 49 49
2
Springfield-Kensington
Avenue Public 76.4 82.7 38.5 40 37
4 Boston-HenryGrew Public 55 85.2 30 28 32
4
Boston-WilliamEllery
Channing Public 54.6 92.2 28 28 28
4 Boston-Mattahunt Public 58.4 94.5 13.5 10 17
4
Lawrence-Oliver
PartnershipSchool Public 58.9 96.1 37 28 46
4 Boston-JohnWinthrop Public 63.9 91.1 16.5 18 15
4
Athol-Royalston-
Riverbend-SandersStreet
School Public 68.2 16.9 23.5 23 24
4
Worcester-ElmPark
Community Public 66.4 63.4 18 22 14
4
Lawrence-UPAcademy
LeonardMiddleSchool Charter 66.5 97.9 44.5 53 36
4
NewBedford-
Hayden/McFadden Public 70.1 57.1 25.5 20 31
4
Lawrence-Community
DayArlington Charter 69.3 97.7 30 26 34
4
FallRiver-Samuel
Watson Public 71.8 33.1 21 23 19
4 Springfield-WhiteStreet Public 77.4 81.1 47.5 40 55
4
Springfield-WilliamN.
DeBerry Public 80.8 95.2 30 21 39
4
Springfield-Milton
BradleySchool Public 79.2 95.9 20.5 20 21
5 NewBedford-John Public 60.8 55.1 44 43 45
54
AveryParker
5 Boston-PaulADever Public 66 85.7 34 27 41
5 Boston-JohnPHolland Charter 64.7 85.6 19.5 14 25
5
Holyoke-Morgan
Elementary Public 84.2 97.2 15.5 19 12
55
AppendixV:SelectionofIndicatorsforStatisticalAnalysis
SelectedIndicators
AttendanceRate:theaveragepercentageofthenumberofdayspresentforallstudents.Thisindicatorwasselectedforanalysisasourresearchandliteraturereviewhasshownthatincreasedstudentattendanceandlearningtimeisassociatedwithimprovedlearningoutcomes.WewillthereforeanalyzestatewidedataforMassachusettstodeterminewhetherthisisalsoassociatedwithaschool’saccountabilitylevel.Studentintakerate:thepercentageofallenrolledstudentsthatenrolledafterthefirstdayoftheschoolyear.Thisindicatorwasselectedforanalysisasahigherpercentageofstudentsthatentertheschoolafterthebeginningoftheyearislikelytoaffectschoolperformance;itisreasonabletoassumethatstudentsthatenteraschoolafterthefirstdaywillneedmoretimetoadjusttoanewsettingandcurriculum,andmaysufferacademicallyasaresult.Whileeffectscouldbebothpositiveornegative,webelievethisisarelevant,exogenousvariablethatshouldbeexamined.WewillthereforeanalyzestatewidedataforMassachusettstodeterminewhetherthisisalsoassociatedwithaschool’saccountabilitylevel.Principalretentionrate:thepercentageofprincipalsthatremainattheschoolfortheentireyearandreturnforthebeginningofthenextyear.Thisindicatorwasselectedforanalysisasconsistencyandstabilityinschoolleadershipislikelytoaffectstudentoutcomes.Consistentschoolleadershipmeansthatteachersandstudentsaremorelikelytobeaccustomedtotheadministration’spolicies,andoutcomesaremorelikelytorepresenttheleadership’spriorities.WewillthereforeanalyzestatewidedataforMassachusettstodeterminewhetherthisisalsoassociatedwithaschool’saccountabilitylevel.Teacherretentionrate:thepercentageofteachersthatremainattheschoolfortheentireyearandreturnforthebeginningofthenextyear.Thisindicatorwasselectedforanalysisasconsistencyandstabilityininstructionalstaffislikelytoaffectstudentoutcomes.Consistencyininstructionalstaffmeansthatteacherscancontinuallyimproveinsteadofhavingtoberetrainedyearafteryear,andcanfocusonimprovementratherthanspendingtimeintransition.WewillthereforeanalyzestatewidedataforMassachusettstodeterminewhetherthisisalsoassociatedwithaschool’saccountabilitylevel.Percentageofstaffevaluatedperyear:thepercentageofteachingstaffthatreceivedanevaluationthepreviousyear.Thisindicatorwasselectedforanalysisaswebelievethataccountabilityincentivesarelikelytobeassociatedwithstudentoutcomesandschoolperformance.Ifteachersarebeingevaluated,theyareheldaccountabletotheirperformance,incentivizingthemtoimprovetheirownperformance.This,then,willinturnimpactstudentoutcomesandschoolperformance.WewillthereforeanalyzestatewidedataforMassachusettstodeterminewhetherthisisalsoassociatedwithaschool’saccountabilitylevel.Percentageofteachersteachingintheirlicensedsubjectarea:thepercentageofteachersthatarelicensedinthespecificsubjectareathattheyareassignedto.Thisindicatorwasselectedforanalysisaswebelievethatteachersthatteachintheirspecificlicensedsubjectarea,inwhichtheyareexperts,aremorelikelytobeeffective.Forexample,teacherstrainedandcertifiedtoteachmathematicsaremorelikelytobeeffective
56
mathematicsteachersthanteacherstrainedandcertifiedinliterature.WewillthereforeanalyzestatewidedataforMassachusettstodeterminewhetherthisisalsoassociatedwithaschool’saccountabilitylevel.PercentageofteachersincoresubjectareasthatareHighlyQualified:thepercentageofteachersincoreclasses(ELA,math,socialstudies,science,foreignlanguages,arts)thatholdaMassachusettsteachinglicenseanddemonstratesubject-mattercompetencyinthesubjecttheyteach.ThisindicatorwasselectedforanalysisaswebelievethatHighlyQualifiedteachersaremorelikelytobeeffectivethannon-HighlyQualifiedteachers.Teachersthathavebeencertifiedasexpertsoftheircraftshouldbemoreeffectivethanthosethathavenotreceivedsuchcertification.WewillthereforeanalyzestatewidedataforMassachusettstodeterminewhetherthisisalsoassociatedwithaschool’saccountabilitylevel.Percentageofstudentsdisciplined:thepercentageofallenrolledstudentsthatreceivedadisciplinarydisposition,includingin-schoolsuspensions,out-of-schoolsuspensions,permanentexpulsions,andremovalstoalternatesettings,suchasalternativeschooling,homeorcommunityservice.Thisindicatorwasselectedforanalysisaswebelievethatstudentdisciplineisastrongdeterminantofschoolperformance.Schoolswherestudentsarenotdisciplinedarelikelytohavepoorercultures,andseesignificantinstructionaltimewastedondisciplinaryinterventions.Furthermore,theseschoolsalsoexpendmoreresourcesaddressingdisciplineissues,takingresourcesawayfrominstruction.WewillthereforeanalyzestatewidedataforMassachusettstodeterminewhetherthisisalsoassociatedwithaschool’saccountabilitylevel.
OmittedIndicators
Studentdropoutrate:thepercentageofenrolledstudentsthatdroppedoutofschoolanddidnotreturntoanyschoolbeforeOctober1ofthenextyear.Thiswasnotincludedasstudentdropoutrateislikelytobeanendogenous;theperformanceandlevelofaschoolislikelytoimpactthestudentdropoutrate.Studentsatpoorperformingschoolsaremorelikelytodropout.Thisisthereforeanunreliabledeterminantofthevariablesthatcontributetoschoolperformance;iftheirperformanceaffectsthisvariable,wecannotconcludethatthevariablewill,inreverse,alsoaffectschoolperformance.Studentretentionrate:thepercentageofenrolledstudentsthatreturnedtoenrollinthesamegradeastheprecedingyear.Thiswasnotincludedasstudentretentionrateislikelytobeanendogenous;theperformanceandlevelofaschoolislikelytoimpactthestudentretentionrate.Wecannotdeterminewhetherstudentshavetorepeatayearbecauseofpoorschoolperformance,orwhetherstudentsrepeatingleadstoaschoolperformingbetter.Furthermore,wecannotdeterminewhetherstudentsareretainedbecauseaschoolhasrigorousassessmentpractices,orwhetherstudentsaresufferingfrompoorteaching.Thisisthereforeanunreliabledeterminantofthevariablesthatcontributetoschoolperformance;iftheirperformanceaffectsthisvariable,wecannotconcludethatthevariablewill,inreverse,alsoaffectschoolperformance.Studentchurnrate:thepercentageofallenrolledstudentsthatenteredorexitedtheschoolafterthefirstdayofschoolandbeforethelastdayofschoolofthatyear.Thiswasnotincludedasstudentchurnratealessreliableindicatorofstudentstabilitythanthestudentintakerate.Studentsthatleavebeforetheendoftheschoolyeardonotaffecta
57
school’stestscoresforthatcurrentyear.Thisisthereforeanirrelevantvariabletomeasure.Studentenrollmentstability:thepercentageofallenrolledstudentsthatremainedattheschoolfortheentireyear.Thiswasnotincludedasstudentenrollmentstabilitywasnotausefulindicatorofstudentperformance.Whetherastudentleavestheschoolbeforethelastdayoftheyeardoesnotaffecttheirperformanceattheircurrentschool,andthereforehasnobearingontheirtestscoresandacademicperformanceatthatschool.Ageofteachers:theproportionofteachersineachagegroup.Thiswasnotincludedaswebelievethatwhilethenumberofyearsofteachingexperiencemaycontributetostudentoutcomes,teachers’ageitselfisunlikelytohaveanybearingonstudentperformance.Raceofteachers:theproportionofteachersofeachracialgroup(Black,White,Hispanic,Asian,NativeAmerican,Biracial,Other).Thiswasnotincludedaswebelievethatwhilestudentsmayrespondbettertoteachersofthesamerace,theproportionofteachersofanyraceattheentireschoolisirrelevant.Thereisnodatamatchingstudentandteacherraces,andthusaschool-widedatapointisnotusefulforanalysis.Teacherevaluationratings:thepercentageofteachersreceivingexemplary,proficient,needsimprovementorunsatisfactoryevaluationratings.Thiswasnotincludedaswebelievethatteacherevaluationratingsarelikelytobeendogenoustoschoolperformance.Lowperformingschoolsoftenhavelowperformingprincipals,whoassignratingstoteachers.Teacherevaluationsaresubjectiveandarerelativetooverallschoolperformance,notcomparedtoschoolsacrosstheentireCommonwealth.Studentattritionrate:thepercentageofstudentsenrolledataschoolattheendofaschoolyearthatdidnotenrollonthefirstdayofthefollowingschoolyear.Thiswasnotincludedasstudentattritionrateislikelytobeanendogenous;theperformanceandlevelofaschoolislikelytoimpactthestudentattritionrate.Studentsatpoorperformingschoolsaremorelikelytoleaveortransfertoadifferentschool.Thisisthereforeanunreliabledeterminantofthevariablesthatcontributetoschoolperformance;iftheirperformanceaffectsthisvariable,wecannotconcludethatthevariablewill,inreverse,alsoaffectschoolperformance.Thefollowingindicatorswereomittedbecausetheyareabsolutenumbers,notpercentages,andarethereforenotasusefulforcomparison:
• Numberofstudentsabsenttendaysormore• Numberofstudentschronicallyabsent• Numberofstudentswith10ormoreunexcusedabsences• Numberofteachersevaluated• Numberofstudentsreceivingdifferenttypesofdisciplineinterventions
58
AppendixVII:InterviewProtocol
InterviewProtocolThisisthelistofquestionsweintendtouseduringourinterviewswhileconductingresearchforthe
purposeofanalyzingeffectivestrategiesusedinschoolsservingdisadvantagedpopulations.
Introduction:
Mynameis(LucyBoydorJonathanHuiorAmyChandran)fromHarvardUniversity,andIamaskingyou
totakepartinaresearchstudywiththeMassachusettsDepartmentofElementaryandSecondary
Education.WearelookingtocompareLevel1/2schoolsservingdemographicallysimilarpopulationsas
variousLevel4/5schoolsacrosstheState.Thisinterviewisintendedtoprovideuswithabetterpicture
aboutthespecificwaysyou,andyourschool,havebeenabletoachievehighlevelsofstudent
performanceandaneffectiveschoolenvironment.Theinterviewwilllastabout30minutes.Beinginthis
studyisvoluntary.Pleasetellmeifyoudonotwanttoparticipateatanytime.Youcanskipquestions
thatyoudonotwanttoanswerorstoptheinterviewatanytime.
Ifyousorequest,IwillkeepthedataIcollectconfidential,andwillnotshareyourpersonalinformation
orspecificdatayouprovidewithanyoneoutsidetheresearchteam.Otherwise,Iwouldliketobeableto
includesomeofyourresponsesinasetcasestudytobepublished,wheretheseareparticularlypertinent
toourfindings.
Wouldyoubehappyforustousequotesorfactsfromthisinterviewtohighlightkeyfindingsinourdata
collectionprocess?
(Ifso)Isusingyournameandresponsespublicallyokaywithyou?Alternatively,mayweciteyour
responsesanonymously?
Areyouhappyformetorecordthisconversation?
Ifyouhaveanyquestions,youcanalsocontactmyresearchadvisor,JoshGoodman,whocanverifythe
detailsortheresearchprojectscopeorprovidefurtherclarification:[email protected].
OpeningQuestionforAllInterviewees:
1. Overall,whatdoyoubelievearetheprimarymechanismsbywhichyourschoolachieveshigh
studentperformance?
ForPrincipals:
Nowwearegoingtoaskaseriesofspecificquestionsthatrelatetothefollowingfocusareas:student
culture,humancapital,datadriveninstruction,tutoring,andadditionalservices.Wewillendwitha
seriesofspecificquestionsaboutservingstudentsfromdiversebackgrounds.
1. StudentCultureBestPractices:
a. Whatarespecificwaysinwhichyouincreasestudentattendancerates?Whichhave
beensuccessful?Whichhavenot?Howdoyouknowthis?
b. Yourschool’sattritionratesarealsorelativelylow.Howhaveyoumanagedtokeep
studentturnoverlow?Whataresomespecificpracticesyourschoolhasused?
c. Pleasedescribeyourschool’sdisciplinarysystemandprocedures(PossibleProbes:Do
youtrackstudentdisciplinedata?Isthereanincentivestructure?)
d. Pleasedescribethestudentcultureofyourschool.Pleasebeasspecificaspossible.
i. Wouldyoucallyourschool’scultureoneofhighexpectations?Whatarethese
expectations?
59
ii. Arethereanyspecificactivitiesorenvironmentalpromptsinplacethatyou
believeareespeciallyimportantinbuildingthisculture?
e. Canyoudescribeparentandcommunityinvolvementintheschool.
i. Howdoyourparentandcommunityengagementstrategiesimpactschool
culture?Whataresomestrategiesthathaveworkedforyou?
2. HumanCapitalBestPractices:
a. Whatarespecificwaysinwhichyouaddressteacherretention?(PossibleProbe:Asa
principal,whatisyourstrategyforimprovingtheteacherexperienceatyourschool?)
b. Pleasedescribehowyourschooltrainsteachersanddevelopstheirteachingskills.
i. HowmanyhoursofProfessionalDevelopmentdoteachersengageineach
year?
c. Howmanyclassroomobservationsdoteachersreceiveeachyear?
d. Whatistheaveragenumberofteacherabsencesperyear?
3. DataDrivenInstruction:
a. Pleasedescribetheroleofdatainyourinstructionhere.(PossibleProbe:Areteachers
collectingdatafrequently?Ifso,whatdotheydowiththisdataanddoesitinformtheir
instruction?)
i. Whatdodatameetingslooklike?Dotheyallsharecommonpractices?Ifso,
whatarethey?
ii. Whatkindsofdataareyourteacherscollectinganddiscussing?
b. Isthereanyoneontheadministrativeorteachingstaffthathasparticularresponsibility
fordatacollectionandanalysis?Ifnot,howisthisprocessmanaged?
4. AcademicsandTutoring:
a. Howmanyhoursperdayarestudentsinschool,excludingafterschoolprogramming?
b. Howmanyschooldaysperyearisyourschoolinsession?DoyouhostSaturdayschool?
c. Doesyourschoolprovideextraresourcestostudents;Iamthinkingoftutoring,extra
homeworkassistancehours,specialopportunityclassesorotheropportunitiesfor
students?
d. (Ifyestoabove)Pleasedescribeyourschool’stutoring(orother)program,ifitexists.
(PossibleProbe:Isit1:1,grouptutoringorsomeotherformat?Wholeadsthetutoring?
Howarestudentsidentifiedfortutoring?)
5. OtherServices:
a. Doesyourschooluseanyothertypesofnon-academicservicesin-house?Examples
includesupportwithclimateandcultureoftheschool,socio-emotionalneedsof
students,orbehavioralneedsofstudents?Thismighttaketheformofaschoolor
counselor,forexample.
i. (Ifyes)Howmanyhoursdoeseachservicetypicallyspendservicingstudents
perweek?
b. Doyouhavealistingofyourextracurricularprogramsandactivities?Whichone(s)
wouldyouhighlightasyourmostsuccessfulorpopular?Howdoyouknowthey
contributetotheholisticdevelopmentofyourstudents?
6. FinalFollowupQuestions:
a. Asyouareaware,wearelookingatyourschoolbecauseitservesastudentbodythatis
demographicallysimilartosomeschoolsthathavenotbeenassuccessfulasyours.We
arethereforeparticularlyinterestedinstrategiesthatyouseeasimportanttoservinga
studentbodywithalargeproportionofchildrenfromeconomicallydisadvantaged
backgroundsand/orethnicminorities.Isthereanythingyourschoolhasdoneto
addressthesegroupsinparticular,oranythingyoubelieveiskeytoacultureof
successfullyservingthesepopulations?
b. YourschoolhasachievedLevel½statusintheDepartment’srankingservice,whatdo
youseeashavingbeenmostimportanttoachievingthissuccess?
60
c. Isthereanythingthatwehaven’tdiscussedthatyouthinkhasbeenparticularly
importantforyourschool’ssuccess,oranythingelseyouwouldliketomention?
ForTeachers:
Nowwearegoingtoaskaseriesofspecificquestionsthatrelatetothefollowingfocusareas:student
culture,humancapital,anddatadriveninstruction.
1. StudentCultureBestPractices:
a. Pleasedescribethestudentcultureofyourschoolandmorespecifically,thecultureof
yourclassroom.Pleasebeasspecificaspossible.
i. Arethereanyspecificactivitiesorenvironmentalpromptsinplacethatyou
believeareespeciallyimportantinbuildingthisculture?
b. Whathasyourschooldonetomakesurestudentsarecomingtoschoolandarestaying
fromyeartoyear?
2. HumanCapitalBestPractices:
c. Howdoesyourschoolsupportteachers?Doyoufeelthatthisisaparticularstrengthof
theschool?Why?
d. Whatpracticesdoesyourschoolputinplacetoretainitsteachers?Doyouthinkthese
areeffective?
e. Pleasedescribetheprofessionaldevelopmentandteachertrainingimplementedat
yourschool.(PossibleProbes:Doyouhaveamentorteacher?Howoftenareyou
observed,ifatall?Doyouengageinafterschooltrainingwithotherteachers?)
f. Howmanyhours(approximately)ofprofessionaldevelopmentareyourequiredto
undertakeperyear?
3. DataDrivenInstruction:
g. Areyouengagedincollectingdataaboutteachingpracticesoryourinstructionona
regularbasis?
h. Ifso,pleasedescribetheroleofdatainyourinstructionhere.(PossibleProbe:Howdo
youusedatatoinformyourinstruction?)
4. AcademicSupportforStudents:
a. Doesyourschoolhaveparticularprocessesfordealingwithstudentswhoarestruggling
academically,orisacase-by-caseassessmentmadeforstudentneeds?
b. Whatsupportsystemsareinplaceforstudentswhomayneedextraassistance?
c. Howeffectivedoyouthinkthesesystemsareinbridgingachievementgapswithinyour
classroom?Howdoyouknowthis?
5. FinalFollowupQuestions:
d. Asyouareaware,wearelookingatyourschoolbecauseitservesastudentbodythatis
demographicallysimilartosomeschoolsthathavenotbeenassuccessfulasyours.We
arethereforeparticularlyinterestedinstrategiesthatyouseeasimportanttoservinga
studentbodywithalargeproportionofchildrenfromeconomicallydisadvantaged
backgroundsand/orethnicminorities.Isthereanythingyourschoolhasdoneto
addressthesegroupsinparticular,oranythingyoubelieveiskeytoacultureof
successfullyservingthesepopulations?
e. YourschoolhasachievedLevel½statusintheDepartment’srankingservice,whatdo
youseeashavingbeenmostimportanttoachievingthissuccess?
f. Isthereanythingthatwehaven’tdiscussedthatyouthinkhasbeenparticularly
importantforyourschool’ssuccess,oranythingelseyouwouldliketomention?
Thankyousomuchforyourtime.Ifyouhaveanyquestionsorfeedback,pleasefeelfreetoreachoutto
61
AppendixVII:StakeholderConsiderationsThefollowingbriefguidesbreakdownourrecommendationsbywhichgroupofpeopleprincipalswillneedtocollaboratewithtoimplementthem.Theygroupsare:theleadership
team,teachers,andcounseling/supportstaff.
TheLeadershipTeam
Areaof
ImprovementIdeal
BehaviorManagement
• Oneschoolleaderisresponsibleforhandlingstudentmisbehaviorthatmustbedealtwithoutsideoftheclassroomaspartofauniformescalationmodel.
o Seerecommendation1.2.1. • Oneschoolleadertracksstudentdisciplineandbehaviordataand
theleadershipteamevaluatesthisdataconsistentlyduringleadershipmeetings.
o Seerecommendation1.2.2.
TeacherCollaboration
• Atleastoneschoolleaderattendsaportionofteachercollaborationmeetingstoevaluateproductivityandeffectiveness.
• Seerecommendation2.1.2.
TeacherEmpowerment
• Schoolleadersmeetwithteachersonaconsistentbasisforcheck-insandconductteachersatisfactionsurveys.
o Seerecommendation2.1.3.
DataDrivenInstruction
• Schoolleadersreviewformalandinformaldatawithteacherstoprovidefeedbackandsupportinadaptinglessonstostudentneeds.
o Seerecommendation1.1.1.
ParentEngagement • Oneortwoschoolleadersareresponsibleforcreatingastrategytoengageparentsbothsociallyandacademically.Theyworkwithteachersineachgrade-leveltodelegatetasksandscalebestpractices.
o Seerecommendation2.2.1.
StudentAttendance • Oneschoolleaderisresponsibleforrecognizingstudentsonamonthlybasisforperfectornearperfectattendance.
o Seerecommendation2.3.1.• Oneschoolleader,orarotatingteamofleaders,callhomesdailyof
studentswhoareabsent.o Seerecommendation2.3.2.
EnrichmentOpportunities
• Oneschoolleaderisresponsibleforreachingouttolocalorganizationstoorchestrateenrichmentopportunitiesforstudentsafterschool.
o Seerecommendation1.4.2.
62
Teachers
Areaof
ImprovementIdeal
TeacherTraining • Veteranorhighperformingteacherssupportstrugglingornewteachersthroughagradualreleasemodel.
• Seerecommendation2.1.1.
TeacherCollaboration
• Grade-levelteacherscollaborateduringcommonplanningtimeweekly,includingspecialistsandinterventionists.
o Seerecommendation2.1.2.• Grade-spanteacherscollaborateatkeypointsthroughouttheyear.
o Seerecommendation2.1.4.
TeacherEmpowerment
• Teachersaregivenautonomyandflexibilityoverschedulingandresponsibilities.
o Seerecommendation2.1.3.
Tutoring • Teachersprovidetutoringincyclesthroughouttheyearbasedonspecificstudentneedsthatarere-evaluatedattheendofeachcycle.
o Seerecommendation1.4.1.
BehaviorManagement
• Everyteacherhasaclassroomlevelbehavioralsystemthatfeedsintoaschool-widebehavioralsystemthroughaseamlessescalationladder.
o Seerecommendation1.2.1.
DataDrivenInstruction
• Teacherscollectformalandinformaldataonaweeklybasisandevaluatestudent-leveldatainweeklycollaborationmeetingsandwithaninstructionalleader.
o Seerecommendation1.1.1.
63
TheCounselor,SocialWorkerorSupportPersonnel
Areaof
ImprovementIdeal
StudentAttendance • SocialWorkerorCounselorshouldkeepprofilesofstudentswhoarechronicallyabsentandworkwithteachersandleadershiptodevelopindividualinterventioniststrategies.• Seerecommendation2.3.1.
BehaviorManagement
• SocialWorkerorCounselorworkscollaborateswithschoolleaderresponsibleforstudentbehaviordatatoreferstudentsforsocial,emotional,orbehavioralservices.
o Seerecommendation1.2.2.
WraparoundServices
• SocialWorkerorCounseloractsascentralpointofcontactforallreferralservicesthatconnectsteachers,leaders,andserviceproviderstooneanother.
• SocialWorkerorCounselormonitorstheeffectivenessofthissystemforeachchildandensuresduplicateservicesarenotbeingprovided.
o Seerecommendation1.3.
StudentEnrichment • SocialWorkerorCounselorisresponsibleforreachingouttolocalorganizationstoorchestrateenrichmentopportunitiesforstudentsafterschool.
o Seerecommendation1.4.2.