stem instruction improvement programs improve student … · response to calls to increase both...

STEM Instruction Improvement Programs Improve Student Outcomes

How should teachers spend their STEM-focused professional learning time? To answer this question, we analyzed a recent wave of rigorous new studies of STEM instructional improvement programs. We found that programs work best when focused on building knowledge teachers can use during instruction: knowledge of the curriculum materials they will use, knowledge of content and how content can be represented for learners, and knowledge of how students learn that content. We argue that such learning opportunities improve teachers’ professional knowledge and skill, potentially by supporting teachers in making more informed in-the-moment instructional decisions.

Suggested citation: Hill, Heather, Kathleen Lynch, Kathryn Gonzalez, and Cynthia Pollard. (2019). STEM Instruction Improvement Programs Improve Student Outcomes. (EdWorkingPaper: 19-135). Retrieved from Annenberg Institute at Brown University: http://www.edworkingpapers.com/ai19-135

Heather C. HillHarvard University

Kathleen LynchBrown University

Kathryn E. GonzalezHarvard University

Cynthia PollardHarvard University

VERSION: September 2019

EdWorkingPaper No. 19-135

STEMInstructionImprovementProgramsImproveStudentOutcomesHeatherC.Hill,KathleenLynch,KathrynE.Gonzalez,CynthiaPollard

SuggestedCitation:Hill,H.C.,Lynch,K.,Gonzalez,K,&Pollard,C.(forthcoming).STEMInstructionImprovementProgramsImproveStudentOutcomes.PhiDeltaKappan.

AbstractHowshouldteachersspendtheirSTEM-focusedprofessionallearningtime?Toanswerthisquestion,weanalyzedarecentwaveofrigorousnewstudiesofSTEMinstructionalimprovementprograms.Wefoundthatprogramsworkbestwhenfocusedonbuildingknowledgeteacherscanuseduringinstruction:knowledgeofthecurriculummaterialstheywilluse,knowledgeofcontentandhowcontentcanberepresentedforlearners,andknowledgeofhowstudentslearnthatcontent.Wearguethatsuchlearningopportunitiesimproveteachers’professionalknowledgeandskill,potentiallybysupportingteachersinmakingmoreinformedin-the-momentinstructionaldecisions.

HowshouldteachersspendtheirSTEM-focusedprofessionallearningtime?Recentnationalsurveydata(Baniloweretal.,2018)suggeststeachersofSTEM(science,technology,engineeringandmathematics)devotetheirprofessionallearningtimetostudyingstatestandards,analyzinginstructionalmaterials,deepeningtheirunderstandingofcontentandstudentthinkingaboutcontent,learningaboutassessment,andstudyingstudentdata.Whatwedon’tknowistheextenttowhichsuchactivitiesimprovestudents’academicoutcomes.InarecentsynthesisoftheresearchliteratureonSTEMinstructionalimprovement,wesetouttodeterminejustthat.Wefoundtheprogramsthatworkbesttendtofocusonbuildingknowledgeteacherscanuseduringinstruction:abouthowtousecurriculummaterials,aboutcontentandhowcontentcanberepresentedforlearners,andabouthowstudentslearnthatcontent.Wearguethatsuchlearningopportunitiesimproveteachers’professionalknowledgeandskill,potentiallybysupportingteachersinmakingmoreinformedin-the-momentinstructionaldecisions.AStudyofStudiesConductingsynthesesoftheliteratureoninstructionalimprovementprogramsisnotnew.Suchsynthesesgenerallyhavetwogoals:tocalculatetheaveragesizeofprogrameffectsonstudentoutcomes,andtolearnwhetherspecificprogramcharacteristicspredictvariationintheseeffects.However,upuntilrecently,attemptstoperformsynthesesusingonlyrigorousstudies–thosethatcompareparticipatingteacherstoasimilarsetofnon-participants,typicallyviarandomization–returnedtoofewstudiestoachievethelattergoal.Forinstance,Yoonandcolleagues(2007)couldfindonlynineELA,math,andsciencestudiesthatmetfederalWhatWorksClearinghouse(WWC)evidencestandards.InSTEM,Gerstenetal.(2014)locatedonlyfivemathematicsprofessionaldevelopmentstudiesthatmetWWCstandards.Whilethesenumberswouldallowthemeta-analysisauthorstocalculatetheoveralleffectoftheseprogramsonstudentoutcomes,neithercouldlinkspecificprogramfeaturestothoseoutcomes.However,followingcallsforstrongerresearchintotheimpactofeducationalinterventions(e.g.,Shavelson&Towne,2001),federalresearchportfoliosbeganintheearly2000stoprioritizeresearchmethodsthatallowforcausalinference,andtorequireimprovedstudentoutcomesasanindicatorofprogramsuccess.Dollars’andscholars’turntowardusingcausalmethodsandexaminingstudent-levelimpactshasresultedinawealthofnewstudies.Thesenewstudies,wereasoned,wouldpermitmoreextensiveempiricalanalysesthanpriorsyntheses.Withthisinmind,wesetouttoreviewstudiesonSTEMinstructionalimprovementprograms.WefocusedonSTEMbecauseofitsimportancetothenationaleconomy(Atkinson&Mayo,2010;NationalCommissiononMathematicsandScienceTeaching,2000),andbecausewewantedtoprovideevidenceonSTEM-specificprofessionallearningactivities,ratherthanmoregeneralactivities.BecausemanySTEMinstructionalimprovementprogramsweredevelopedin

responsetocallstoincreasebothstudentunderstandingofcoredisciplinaryideasandstudentengagementwithkeydisciplinarypracticessuchasinquiry,argumentationandproof(NGA,2010;NCTM,2000;NRC2012),wereasonedthatsuchpracticeswouldnotnecessarilycarryacrossdisciplines.Asistypicalinareviewofthiskind,weconductedextensivedatabasesearches,combedthrougholderresearchsyntheses,andcontactedtheprincipalinvestigatorsofstudieswithunpublishedfindings.Werequiredstudiestoeitherfeaturerandomassignmentofteachersorschoolstothenewprogramoracontrolgroup,ortohaveamatchedcomparisongroupofteachersidentifiedbeforedatacollectionbegan(fordetails,seeLynch,Hill,Gonzalez&Pollard,2019).Intheend,welocated89studiesofprogramsthatcontainedprofessionaldevelopmentforteachers;ofthese,71alsocontainednewcurriculummaterialsforteacherstouseinclassrooms,suggestingthatprogramdevelopersoftenpairedprofessionaldevelopmentwithnewclassroommaterials.Becausemanyconsidercurriculummaterialstobeanimportantsourceofteacherprofessionallearninginandofthemselves(Ball&Cohen,1996),wealsoincludedsixstudiesofnewcurriculummaterialsbutnoassociatedprofessionaldevelopment;anotherthreestudiescomparedcurriculumwithandwithoutprofessionaldevelopmentandwereincludedinthecountabove.Welocated95studiesintotal.Wethenreadthroughthesestudiesandcreatedadatasetthatcontainedseveralpiecesofinformationfromeachstudy.

• Programtype:Focusonprofessionaldevelopment,curriculummaterials,orboth.• Assessmenttype:Studentoutcomesmeasuredviastateordistrictstandardizedtest,

otherstandardizedtest,orresearcher-designedassessment.Forteacherprofessionaldevelopment,wefurthercodedseveralotherfeatures:

• ThelengthofthePD;• Thefocus(orfoci),includingtopicslikeimprovingteachercontentknowledge,

integratingtechnologyintotheclassroom,andlearninghowtousecurriculummaterials;

• Theactivitiesthatteachersengagedinduringtheprofessionaldevelopment,suchasreviewingstudentwork,observingademonstrationofinstruction,orworkingthroughstudentcurriculummaterials;

• Theformatoftheprofessionaldevelopment,forinstancewhetheritwasdeliveredduringasummerworkshop,containedcoaching,orinvolvedonlinelearning.

Forcurriculummaterials,wecodedforthefractionoftheoriginalcurriculumitwastoreplace,whethertherewasanybackgroundinformationaboutcontentorstudentlearningembedded

inthematerials,andwhetherthematerialscontainedkitsforstudentexperimentsandactivities.Followingcoding,wesetouttocalculateanaverageeffectsizeonstudenttestscoresacrossprograms,andtounderstandwhetheranyprogramcharacteristicspredictedstrongerorweakerstudentoutcomes.Weusedastatisticaltechniquecalledmeta-analysistodoso;wedescribethedetailsofthisanalysisinLynch,Hill,GonzalezandPollard(2019).ProfessionalLearningProgramsImproveStudentOutcomesFigure1showstheaverageeffectsizeacrossallSTEMinstructionalimprovementprograms,bothoverallandbrokendownbythetypeofassessmentusedintheevaluation.Acrossall95studies,theaverageprogramproducesa+8-percentiledifferenceintherankoftheaveragetreatment-andcontrol-groupstudent.Thiseffectismuchlarger(+14percentiles)forresearcher-designedassessmentsthanforstate-standardizedandotherstandardizedassessments(+2percentilesand+3percentiledifferences,respectively).Forallassessmenttypes,ouranalysisshowsthatthedifferencebetweenthetreatmentandcontrolgroupsisnotlikelytobezero–meaningtheseeffectsarestatisticallysignificant,thoughinthecaseofstandardizedassessments,notlarge.Acriticalquestionforouranalysiswaswhethersomeprogramfeaturesoutperformedothers,intermsofboostingstudentoutcomes.Figure2showsthatprogramspoststrongeraverageeffectswhentheyfeatureprofessionaldevelopmentpairedwithnewcurriculummaterials(anaverage+10percentilerankdifferencebetweentreatmentandcontrol)asopposedtofeaturingeithercurriculummaterialsorprofessionaldevelopmentalone(anaverage+6percentile-rankdifferencebetweentreatmentandcontrol).Figure3,whichshowstheaverageimpactsassociatedwithdifferentprofessionaldevelopmentprogramfoci,explainsthisfindinginmoredepth.Here,wecomparedprogramwithspecificcharacteristicstoseewhetheranyledtostrongerstudentoutcomesthanprogramswithoutthesecharacteristics.Wealsodescriptivelycomparedtheimpactsofprogramswiththesecharacteristicsrelativetotheaverageprogramimpactinthe95-studydataset.Twoprogramfoci–helpingteacherslearnhowtousecurriculummaterials,andimprovingteachers’contentknowledge,pedagogicalcontentknowledge,andknowledgeofstudentlearning–postedbetterstudentoutcomes(oftwoandthreepercentilepointsbetterthantheaverageprogram,respectively)thanprogramswithoutthesefoci.ThissuggeststhatthecombinedeffectofcurriculumandprofessionaldevelopmentinFigure2mayresultfromteacherslearninghowtousethematerialsandimprovingtheircontentknowledgeandknowledgeofstudentsalongtheway.Thishypothesisissupportedbyamorequalitativereadoftheincludedstudies;manythatfeaturedbothcurriculummaterialsandprofessionaldevelopmentoftenengagedteachersinsolvingmathematicsproblems,takingpartinscientificinvestigation,watchingfacilitatorsmodelinstruction,andstudyingstudentwork.Relationshipsbetweenotherfociandstudentoutcomeswerestillpositive,butouranalysisdidnotindicatetheyweredifferent,onaverage,thanprogramsthatdidnotincludethesefeatures.

Inthecaseofintegratingtechnologyandcontent-specificformativeassessment,whichpostedstrongerabsolutegainsthanotherfoci,thisinabilitytodifferentiatetheeffectfromzerowasduetothesmallnumberofstudiesincludedinthesecategories.Figure4showsaverageimpactsassociatedwithdifferentprofessionaldevelopmentprogramformats.Threeformats–same-schoolcollaboration(+2percentiles),implementationmeetings(+4percentiles),andsummerworkshops(+2percentiles)–yieldedstrongergainsonstudentassessmentsthanprogramswithouttheseformats.Same-schoolcollaborationoccurredwhenteachersparticipatedintheprofessionaldevelopmentsessionalongsideotherteachersintheirschool,andimplementationmeetingsallowedteacherstoconvenebrieflywithotheractivityparticipantsafterthestartofimplementationtodiscussobstaclesandaidstoputtingtheprogramintopractice.Professionallearningwithanonlinecomponentyieldedlowerimpactsonstudentlearning(-4percentiles)thanprogramsthatwereentirelyfacetoface.Andprogramswithcoaching,apopularapproachtoinstructionimprovementinmanydistricts,yieldedimpactssimilartoprogramswithoutcoaching.However,fewprogramsfocusedonextended1:1coaching;instead,coachingappearedmoreasanadd-ontotraditionalprofessionaldevelopment.Ouranalysesdetectednopositiveornegativeassociationsbetweentheactivitiesteachersengagedduringprofessionaldevelopmentandthesizeofprogrameffects;thisincludedactivitiessuchasreviewingstudentwork,solvingproblems,developingcurriculummaterials,andreviewingbothgenericandtheirownstudents’work.Thesamewastrueforfeaturesofnewcurriculummaterials–nofeaturesofthosematerialsoutperformedothers.Finally,thedurationoftheprofessionaldevelopmentwasunrelatedtostudentoutcomes.Becausethiswascontrarytoourexpectation,soweconductedseveralanalysestoassesspossiblethresholdeffects(e.g.,morethan10hours)oracurvilinearrelationship(e.g.,professionaldevelopmentismaximallyeffectivewhenbetween20and40hours),butwefoundnoevidenceforeither.Afterreading95studiesdescribingSTEMinstructionalimprovementprograms,whatcanwesay?Thecharacteristicssignificantlyassociatedwithabove-averagestudentgainsincluded:

• Professionaldevelopmentfocusedonnewcurriculummaterials;• Programsaimedatimprovingteachers'content/pedagogicalcontentknowledge,

orunderstandingofhowstudentslearn;• Programscontainingspecificformats,including:

• Meetingstotroubleshootanddiscussclassroomimplementationoftheprogram;

• Summerworkshopsthatallowconcentratedlearningtime;• Same-schoolparticipationandcollaboration.

Programswithonlysomeorfewofthesecharacteristicsmaystillhavepositiveeffects;however,whenprogramsdidincludethesecharacteristics,studentoutcomeswereimprovedabovetheaverageprogrameffect.Webelievethattheresultsofthismeta-analysishighlighttheimportanceofprofessionalknowledgeforteaching.Withotherscholars(Ball,Thames&Phelps2008;Shulman,1986;Lampert,2001),weviewprofessionalknowledgeinteachingasknowinghowthecontent,studentthinking,andcurriculumcometogether,andthenmakinggoodinstructionaldecisionsbasedontheparticularsofthesituation.Programsthatoutperformothersinouranalysistendedtofocusongrowingthisformofknowledgeasopposedtogeneralpedagogicalknowledge,ormoreperipheraltopicsliketechnology.Whatcan’twesay?Meta-analyseshavetheadvantageofexaminingprogramsimplementedacrossawidevarietyofcontexts,providingsomerobustnesstofindings.However,thismeta-analysisislimitedinwhatitcansayaboutprofessionallearningsystems“ontheground”inU.S.schoolsanddistricts.Tostart,eachprogramweexaminedwasimplementedinaspecificcontext,orasmallsetofcontexts;whethertheprogramwouldsucceedinanothercontextisanopenquestion.Critically,ananalysisfoundaslighttrendtowardsmallerimpactsoftheseprogramsinhigh-povertysettings,suggestingthatinterventionsmayworkbetter,onaverage,indistrictsservingmoreadvantagedstudents.However,wefoundnofurtherinteractionsbystudentrace,ethnicity,districttype(urban,suburbanorrural),andsizeofthetreatmentgroup.Thatsaid,otheraspectsofdistrictandschoolcontextmayinteractwithprogramefficacy.Weknowfromstudiesofpolicyimplementation,forinstance,thatleadershipandpeersupportmatterquiteabitinencouragingteachertake-upanduseofnewinstructionalpractices(e.g.,Matsumura,Garnier,&Resnick,2010;Wanless,Patton,Rimm-Kaufman&Deutsch2013),andthepresenceofcompetinginstructionalguidanceandinitiatives(e.g.,instructionalpacingguides,conflictingadviceonwhatandhowtoteach)tendstodampenteacherchange(Hill,Corey&Jacob,2018).Thestudieswereviewedcontainednoinformationaboutthesefactors,however,sowecouldnottestthemformallyinourmodels.Second,theprogramsdescribedheretendedtobesmall,intensive,enrolledvolunteerteachers,andwereoftenledbyuniversityacademicsorresearchers.Bycontrast,localprofessionaldevelopmentcancontainmyriaddifferentofferings,withteachersspreadingtheirtimeacrossseveraldifferentsettings(summerworkshops,grade-levelteammeetings),topics(ELA,mathematics),andinsessionsledbyotherteachersorschoolordistrictleaders.Insomesystems,teachershaveatleastpartialchoiceovertheprofessionaldevelopmenttheyengage,whileinothersystemstheyhaveverylittle.Oneimplicationofthesedifferencesbetweenthestudysampleandtypicalpracticeisthatwedon’tyetknowwhetherthefeaturesthat“work”inouranalysiswillworkintypicalU.S.schools.Doesaprogramfocusoncontentandpedagogicalknowledgeimprovestudentoutcomes?Doesworkingthroughnewcurriculummaterialsyieldbenefitstotheaverage

teacher?Alotdependsuponthequalityoflocalimplementation,andinourestimation,howdeeplyteachersengagewiththesubjectmatterandattempttoimprovetheircraft.STEM-focusedprofessionallearninginwidercontextAsnotedintheintroduction,teachersofSTEMengageinawidevarietyofprofessionallearningactivities,ofteninasingleyear.Thisleadstoanimportantquestionfordistricts:howtomakemoretimeforthekindsoflearningopportunitiesthatpostedbettergains?Teachersalreadyreportfeelingoverwhelmedbythesheervolumeofreformandever-increasinginstructionalresponsibilities(AFT,2017;Valli&Beuse,2007),andit’slikelythatscalingbackoreliminatinganactivitywillbenecessarytomakeroomformoreefficaciousformsofprofessionallearning.Basedonevidenceintheliterature,wewouldnominateeliminating“datateammeetings,”1whereteachersstudystudentdatainhopesofindividualizingandimprovinginstruction.Inareviewseparatefromthisone,nineevaluationsofprogramsthatfeaturedteachers’studyofdataproducedonlytwopositive(andonenegative)resultslikelytobestatisticallydifferentfromzero,outofatotal19impactanalysesrelatingprogramparticipationtostudenttestscoreoutcomes.Aswell,qualitativeresearchsuggestthatteachersstudyingdatadoesnotitselfleadtoneworimprovedinstructionaltechniques(Barmore,2018;Goertz,Oláh,L.N.,&Riggan,2009),andourownobservationsofandparticipationinschool-baseddatateamssuggeststhatteamdiscussionsoftenascribepoorstudentperformancetofactorsotherthaninstructionitself(e.g.,alackofbackgroundknowledge,abadweek,troubleathome)(seealsoGoertzetal.,2009).Yetrecentnationalsurveysoflocalprofessionaldevelopmentsuggestthatschoolshavemadelargeinvestmentsinhavingteachersstudystudentassessmentdata(e.g.,Banilower,etal.,2018).Repurposingthesemeetingstowardbuildingexpertiseincurriculummaterialsandcontentseemsnatural;wecaution,however,thatdistrictswillhavetodosocarefully,usingroutinesandstructuresthatfocusattentionsquarelyandindepthoninstruction.ConclusionThattheseSTEMinstructionalimprovementprogramsbooststudentoutcomesshouldbeareasonforoptimismamongpolicymakersandleaders.Ourfindingsmay,forinstance,helpshapehowstatesanddistrictschoosetospendTitleIIdollars,fundsaimedatimprovingteacherquality.Theyalsosuggesthowleadersmaynarrowthescopeofteacherprofessionallearninginwayslikelytoincreasetheefficiencyandimpactofthoseefforts.Finally,ourfindingsalsosuggesttheimportanceofteachers’professionalknowledgetostudentoutcomes,ahypothesessupportedbystudieslinkingteacherknowledgetostudentoutcomes(e.g.,Baumert,2010;Hill,Rowan&Ball,2005)andthathasimplicationsforteacherhiringandretention.

1Wedifferentiate“datause”programsfrom“formativeassessment”programsinthattheformertypicallyusesdatafrominterimorbenchmarktestswhilethelatterhelpsteacherscreatetheirownassessments,eitherfromitembanks,curricularassessments,orbasedonprinciplesofgoodassessment.Programsthatfeaturedcontent-specificformativeassessmentwereincludedinouranalysis.

ReferencesAmericanFederalofTeachers(2017).2017educatorqualityoflifesurvey.Washington,DC:AFT.Atkinson,RobertD.andMayo,MerrileaJoyce,RefuelingtheU.S.InnovationEconomy:FreshApproachestoScience,Technology,EngineeringandMathematics(STEM)Education(December9,2010).TheInformationTechnology&InnovationFoundation,Forthcoming.AvailableatSSRN:https://ssrn.com/abstract=1722822Ball,D.L.,&Cohen,D.K.(1996).Reformbythebook:Whatis—ormightbe—theroleofcurriculummaterialsinteacherlearningandinstructionalreform?EducationalResearcher,25(9),6-14.Ball,D.L.,Thames,M.H.,&Phelps,G.(2008).Contentknowledgeforteaching:Whatmakesitspecial.JournalofTeacherEducation,59(5),389-407.Banilower,E.R.,Smith,P.S.,Malzahn,K.A.,Plumley,C.L.,Gordon,E.M.,&Hayes,M.L.(2018).Reportofthe2018NSSME+.ChapelHill,NC:HorizonResearch,Inc.Barmore,J.M.(2018).JourneyfromDataintoInstruction:HowTeacherTeamsEngageinData-DrivenInquiry.Cambridge,MA:HarvardUniversity.Baumert,J.,Kunter,M.,Blum,W.,Brunner,M.,Voss,T.,Jordan,A.,...&Tsai,Y.M.(2010).Teachers’mathematicalknowledge,cognitiveactivationintheclassroom,andstudentprogress.AmericanEducationalResearchJournal,47(1),133-180.Gersten,R.,Taylor,M.J.,Keys,T.D.,Rolfhus,E.,&Newman-Gonchar,R.(2014).Summaryofresearchontheeffectivenessofmathprofessionaldevelopmentapproaches.(REL2014–010).Washington,DC:U.S.DepartmentofEducation,InstituteofEducationSciences,NationalCenterforEducationEvaluationandRegionalAssistance,RegionalEducationalLaboratorySoutheast.Retrievedfromhttp://ies.ed.gov/ncee/edlabsGoertz,M.E.,Oláh,L.N.,&Riggan,M.(2009).Fromtestingtoteaching:Theuseofinterimassessmentsinclassroominstruction.Philadelphia,PA:UniversityofPennsylvania.Hill,H.C.,Corey,D.L.,&Jacob,R.T.(2018).DividingbyZero:ExploringNullResultsinaMathematicsProfessionalDevelopmentProgram.TeachersCollegeRecord,120(6),n6.Hill,H.C.,Rowan,B.,&Ball,D.L.(2005).Effectsofteachers’mathematicalknowledgeforteachingonstudentachievement.AmericanEducationalResearchJournal,42(2),371-406.Lampert,M.(2001).Teachingproblemsandtheproblemsofteaching.YaleUniversityPress.

Lynch,K.,Hill,H.C.,Gonzalez,K.E.,&Pollard,C.(2019).StrengtheningtheResearchBasethatInformsSTEMInstructionalImprovementEfforts:AMeta-Analysis.EducationalEvaluationandPolicyAnalysis,0162373719849044.Matsumura,L.C.,Garnier,H.E.,&Resnick,L.B.(2010).Implementingliteracycoaching:Theroleofschoolsocialresources.EducationalEvaluationandPolicyAnalysis,32(2),249-272.NationalCommissiononMathematicsandScienceTeachingforthe21stCentury(US),&ChairGlenn.(2000).Beforeit'stoolate:AreporttothenationfromtheNationalCommissiononMathematicsandScienceTeachingforthe21stCentury.UnitedStates.U.S.DepartmentofEducation,Washington,D.C.Retrievedfromhttps://files.eric.ed.gov/fulltext/ED441705.pdfNationalCouncilofTeachersofMathematics.(2000)PrinciplesandStandardsforSchoolMathematics.Reston,VA:NCTMNationalGovernorsAssociation,CouncilofChiefStateSchoolOfficers.(2010).CommonCoreStateStandards.Washington,DC:Author.NationalResearchCouncil,BoardonScienceEducation.(2012).AframeworkforK-12scienceeducation:Practices,crosscuttingconcepts,andcoreideas.Washington,DC:TheNationalAcademiesPress.Retrievedfromhttp://www.nap.edu/catalog.php?record_id=13165Riener,C.,&Willingham,D.(2010).Themythoflearningstyles.Change:Themagazineofhigherlearning,42(5),32-35.Shavelson,R.J.,&Towne,L.(Eds.).(2001).Scientificresearchineducation.Washington,DC:TheNationalAcademiesPress.Wanless,S.B.,Patton,C.L.,Rimm-Kaufman,S.E.,&Deutsch,N.L.(2013).Setting-levelinfluencesonimplementationoftheResponsiveClassroomapproach.PreventionScience,14(1),40-51.Yoon,K.S.,Duncan,T.,Lee,S.W.-Y.,Scarloss,B.,&Shapley,K.(2007).Reviewingtheevidenceonhowteacherprofessionaldevelopmentaffectsstudentachievement(Issues&AnswersReport,REL2007–No.033).Washington,DC:U.S.DepartmentofEducation,InstituteofEducationSciences,NationalCenterforEducationEvaluationandRegionalAssistance,RegionalEducationalLaboratorySouthwest.Retrievedfromhttp://ies.ed.gov/ncee/edlabs

Figure1

Figure2

Figure3

Figure4