effects of the my teaching partner intervention in secondary school classrooms
DESCRIPTION
Effects of the My Teaching Partner Intervention in Secondary School Classrooms. Joseph P. Allen Robert C. Pianta University of Virginia. Co-Collaborators: Amori Mikami Anne Gregory. Project Team: Chris Hafen Sharon Deal Judith Wasserman Rachel Boren Janetta Lun. Context. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Effects of the My Teaching Partner Intervention in Secondary School Classrooms](https://reader035.vdocuments.net/reader035/viewer/2022062810/56815ac3550346895dc88d35/html5/thumbnails/1.jpg)
Effects of the My Teaching Partner Intervention in Secondary School Classrooms
Joseph P. AllenRobert C. Pianta
University of VirginiaCo-Collaborators:Amori MikamiAnne Gregory
Project Team:Chris HafenSharon DealJudith WassermanRachel BorenJanetta Lun
![Page 2: Effects of the My Teaching Partner Intervention in Secondary School Classrooms](https://reader035.vdocuments.net/reader035/viewer/2022062810/56815ac3550346895dc88d35/html5/thumbnails/2.jpg)
ContextNumber of Secondary School Students in U.S.: 24
million
Number of Secondary School Classesbeing taught each week 6 million
% of 9th graders who won’t finish High school by the end of 12th grade 25%
Number of programs in ‘What Works’ Clearinghouse with demonstrated efficacy improving teaching quality enough to improve student achievement in these classrooms
0
![Page 3: Effects of the My Teaching Partner Intervention in Secondary School Classrooms](https://reader035.vdocuments.net/reader035/viewer/2022062810/56815ac3550346895dc88d35/html5/thumbnails/3.jpg)
Key Questions• Can we identify teacher-student
interaction qualities that predict student engagement and achievement?
• Can we change these qualities?• Will changes lead to sustainable student
achievement gains?• What are the mechanisms of change?
![Page 4: Effects of the My Teaching Partner Intervention in Secondary School Classrooms](https://reader035.vdocuments.net/reader035/viewer/2022062810/56815ac3550346895dc88d35/html5/thumbnails/4.jpg)
Key Questions• Can we identify teacher-student
interaction qualities that predict student engagement and achievement?
• Can we change these qualities?• Will changes lead to sustainable student
achievement gains?• What are the mechanisms of change?
![Page 5: Effects of the My Teaching Partner Intervention in Secondary School Classrooms](https://reader035.vdocuments.net/reader035/viewer/2022062810/56815ac3550346895dc88d35/html5/thumbnails/5.jpg)
Classroom Learning Assessment & Scoring System- Secondary (CLASS-S)
Emotional Support
Positive ClimateTeacher Sensitivity
Regard for Adolescent Perspectives
Negative Climate
Instructional Support
Instructional Learning FormatContent
UnderstandingAnalysis & Problem
SolvingQuality of Feedback
Classroom Organization
Behavior ManagementProductivity
Student OutcomesStudent Engagement
![Page 6: Effects of the My Teaching Partner Intervention in Secondary School Classrooms](https://reader035.vdocuments.net/reader035/viewer/2022062810/56815ac3550346895dc88d35/html5/thumbnails/6.jpg)
Evaluation Design• 43 teachers within 8 schools (640 students)
(The control condition in an RCT).
• 1 focal classroom selected per teacher
• Predicting Future Achievement after Covarying Baseline Achievement Test Scores
• Teacher Demographics:– 64% female– 83% White, 8% African-American; 6% Mixed-Ethnicity; 3% Other– 54 middle school, 34 high school– 35% BA degree; 65% at least a year of course work beyond BA– Average 8 years of teaching experience
![Page 7: Effects of the My Teaching Partner Intervention in Secondary School Classrooms](https://reader035.vdocuments.net/reader035/viewer/2022062810/56815ac3550346895dc88d35/html5/thumbnails/7.jpg)
Classroom Characteristics• School type: 39% High school;
61% Middle School
• Subject: 52% Language/Social Studies; 48% Math/Science
• Average class size: 23 students
• Gender: 47% girls 53% boys
• Ethnicity: 23% African American2% Asian4% Hispanic70% European-American
![Page 8: Effects of the My Teaching Partner Intervention in Secondary School Classrooms](https://reader035.vdocuments.net/reader035/viewer/2022062810/56815ac3550346895dc88d35/html5/thumbnails/8.jpg)
Observational Assessment of Classroom Environment
• Videotaped observations of a classroom– spread throughout course of year
• Two 20-minute segments per class session/tape– Each tape rated by 2 raters
• Coded Using CLASS-S System• High inter-rater reliabilities; ICC’s range from
– .73 - .82 for overarching domains– .50 -.78 for specific dimensions (all but one dimension > .64)
![Page 9: Effects of the My Teaching Partner Intervention in Secondary School Classrooms](https://reader035.vdocuments.net/reader035/viewer/2022062810/56815ac3550346895dc88d35/html5/thumbnails/9.jpg)
Student Academic Success• Score on State “Standards of Learning” End of
Year Subject Test
• The measure by which schools/students are judged for accreditation/graduation.
• Extensive seven-year validation/standardization process.
![Page 10: Effects of the My Teaching Partner Intervention in Secondary School Classrooms](https://reader035.vdocuments.net/reader035/viewer/2022062810/56815ac3550346895dc88d35/html5/thumbnails/10.jpg)
Analytic Approach• Multi-level modelling • All models covary:
– Student factors:• Grade level• Gender• Family poverty status
– Classroom factors:• Classroom size
– Teacher Factors• Teacher experience• Teacher education• Teacher gender and race
• Moderating effects of covariates were also examined.
![Page 11: Effects of the My Teaching Partner Intervention in Secondary School Classrooms](https://reader035.vdocuments.net/reader035/viewer/2022062810/56815ac3550346895dc88d35/html5/thumbnails/11.jpg)
Predicting Student AchievementAchievement
Emotional Positive Climate .22**Support: Negative Climate -.04
Teacher Sensitivity .16*Regard for Adol. Perspectives .21**
Classroom Behavior Management .06Organization: Productivity .15
Instructional Content Understanding .12Support: Analysis & Prob. Solving .18*
Instructional Learning Formats .22**Quality of Feedback .09
Composite of Significant Dimensions Above .32***
![Page 12: Effects of the My Teaching Partner Intervention in Secondary School Classrooms](https://reader035.vdocuments.net/reader035/viewer/2022062810/56815ac3550346895dc88d35/html5/thumbnails/12.jpg)
Key Questions
✔Can we identify teacher-student interaction qualities that predict student engagement and achievement?
• Can we change these qualities?• Will changes lead to sustainable student
achievement gains?• Why?
![Page 13: Effects of the My Teaching Partner Intervention in Secondary School Classrooms](https://reader035.vdocuments.net/reader035/viewer/2022062810/56815ac3550346895dc88d35/html5/thumbnails/13.jpg)
MyTeachingPartner Overview• Consultant and teacher work together
using the CLASS-S in cultivating: – Observation– Reflection– Development of knowledge and expertise
Classroom Observation Teaching Practice
Knowledge Expertise Support
![Page 14: Effects of the My Teaching Partner Intervention in Secondary School Classrooms](https://reader035.vdocuments.net/reader035/viewer/2022062810/56815ac3550346895dc88d35/html5/thumbnails/14.jpg)
The Steps of the Consultancy
![Page 15: Effects of the My Teaching Partner Intervention in Secondary School Classrooms](https://reader035.vdocuments.net/reader035/viewer/2022062810/56815ac3550346895dc88d35/html5/thumbnails/15.jpg)
MTPS Website
www.mtpsecondary.net
![Page 16: Effects of the My Teaching Partner Intervention in Secondary School Classrooms](https://reader035.vdocuments.net/reader035/viewer/2022062810/56815ac3550346895dc88d35/html5/thumbnails/16.jpg)
Detailed Video Examples
www.mtpsecondary.net
![Page 17: Effects of the My Teaching Partner Intervention in Secondary School Classrooms](https://reader035.vdocuments.net/reader035/viewer/2022062810/56815ac3550346895dc88d35/html5/thumbnails/17.jpg)
Key Questions
✔Can we identify teacher-student interaction qualities that predict student engagement and achievement?
• Can we change these qualities?• Will changes lead to sustainable student
achievement gains?• What are the mechanisms of change?
![Page 18: Effects of the My Teaching Partner Intervention in Secondary School Classrooms](https://reader035.vdocuments.net/reader035/viewer/2022062810/56815ac3550346895dc88d35/html5/thumbnails/18.jpg)
Evaluation Design88 classrooms
45 Tx. 43 Control(Classrooms Randomized within school)
2237 Students
Assessed Across 2 Years
![Page 19: Effects of the My Teaching Partner Intervention in Secondary School Classrooms](https://reader035.vdocuments.net/reader035/viewer/2022062810/56815ac3550346895dc88d35/html5/thumbnails/19.jpg)
Evaluation DesignTreatment group:
Year 1:– Introductory Workshop (late summer)– Ongoing consultancy– ~ 2 days total in-service timeYear 2:– Booster Workshop (late summer) only + Web site
access• Control group:
Usual in-service practice.
![Page 20: Effects of the My Teaching Partner Intervention in Secondary School Classrooms](https://reader035.vdocuments.net/reader035/viewer/2022062810/56815ac3550346895dc88d35/html5/thumbnails/20.jpg)
Intervention Effect on Change in Classroom Qualities
Intervention Target January MarchApril/May
Overall Teacher-Student Interaction Composite
ns
Student Engagement ns
![Page 21: Effects of the My Teaching Partner Intervention in Secondary School Classrooms](https://reader035.vdocuments.net/reader035/viewer/2022062810/56815ac3550346895dc88d35/html5/thumbnails/21.jpg)
Intervention Effect on Change in Classroom Qualities
Intervention Target January MarchApril/May
Overall Teacher-Student Interaction Composite
ns ns
Student Engagement ns ns
![Page 22: Effects of the My Teaching Partner Intervention in Secondary School Classrooms](https://reader035.vdocuments.net/reader035/viewer/2022062810/56815ac3550346895dc88d35/html5/thumbnails/22.jpg)
Intervention Effect on Change in Classroom Qualities
Intervention Target January MarchApril/May
Overall Teacher-Student Interaction Composite
ns ns .19*
Student Engagement ns ns .34*
![Page 23: Effects of the My Teaching Partner Intervention in Secondary School Classrooms](https://reader035.vdocuments.net/reader035/viewer/2022062810/56815ac3550346895dc88d35/html5/thumbnails/23.jpg)
Year 1 Change in OverallTeacher-Student Interactions
Pre-Test Post-Test3.553.603.653.703.753.803.853.903.954.00
Control GroupMyTeachingPartner
Standardized Effects: Baseline = .45***Intervention = .19*
MTPS participation predicts higher quality teacher-student
interactions
![Page 24: Effects of the My Teaching Partner Intervention in Secondary School Classrooms](https://reader035.vdocuments.net/reader035/viewer/2022062810/56815ac3550346895dc88d35/html5/thumbnails/24.jpg)
Key Questions
✔Can we identify teacher-student interaction qualities that predict student engagement and achievement?
✔Can we change these qualities?• Will changes lead to sustainable student
achievement gains?• What are the mechanisms of change?
![Page 25: Effects of the My Teaching Partner Intervention in Secondary School Classrooms](https://reader035.vdocuments.net/reader035/viewer/2022062810/56815ac3550346895dc88d35/html5/thumbnails/25.jpg)
Year 1Intervention Effects on Achievement
• No relation of intervention to either baseline or exit achievement test scores in Year 1 (all p’s > .35).
• Why?– No evidence we changed the classroom until the
very end of the year when most teaching was past.
![Page 26: Effects of the My Teaching Partner Intervention in Secondary School Classrooms](https://reader035.vdocuments.net/reader035/viewer/2022062810/56815ac3550346895dc88d35/html5/thumbnails/26.jpg)
Year 2Change in Achievement
Pre-Test Post-Test460.00465.00470.00475.00480.00485.00490.00495.00
Control GroupMyTeachingPartner
Standardized Effects: Pre-test = .54***Intervention = .22*
MTPS is predicting increases in End of Course Achievement
Tests
![Page 27: Effects of the My Teaching Partner Intervention in Secondary School Classrooms](https://reader035.vdocuments.net/reader035/viewer/2022062810/56815ac3550346895dc88d35/html5/thumbnails/27.jpg)
Year 2Intervention Effects on Achievement
• Real-world effect size = .22 SD increment in Achievement Test scores • Average ‘Bump’ of students in MTP from 50th to 59th percentile in
achievement• If effect applies equally at all parts of achievement spectrum (as
appears to be the case): a .22 SD boost would reduce failure rates from:
14% without the intervention to
10% with it
Reducing the number of failing students each year by 29%
*** This occurs in the year AFTER the intervention year (i.e., sustainability), across diverse subject matter/content areas.
![Page 28: Effects of the My Teaching Partner Intervention in Secondary School Classrooms](https://reader035.vdocuments.net/reader035/viewer/2022062810/56815ac3550346895dc88d35/html5/thumbnails/28.jpg)
Key Questions
✔Can we identify teacher-student interaction qualities that predict student engagement and achievement?
✔Can we change these qualities?
✔Will changes lead to sustainable student achievement gains?
• What are the mechanisms of change?
![Page 29: Effects of the My Teaching Partner Intervention in Secondary School Classrooms](https://reader035.vdocuments.net/reader035/viewer/2022062810/56815ac3550346895dc88d35/html5/thumbnails/29.jpg)
A Preliminary Mediational Analysis
“My Teaching Partner”
Intervention
Intervention
Observed Change in Student
Achievement
*
Environmental Outcome
![Page 30: Effects of the My Teaching Partner Intervention in Secondary School Classrooms](https://reader035.vdocuments.net/reader035/viewer/2022062810/56815ac3550346895dc88d35/html5/thumbnails/30.jpg)
A Preliminary Mediational Analysis
“My Teaching Partner”
Intervention
InterventionIntervention
Target
Observed Change in Student
Achievement
Environmental Outcome
??
![Page 31: Effects of the My Teaching Partner Intervention in Secondary School Classrooms](https://reader035.vdocuments.net/reader035/viewer/2022062810/56815ac3550346895dc88d35/html5/thumbnails/31.jpg)
Mediational Analyses• Assessed via Multi-level Structural Equation
Modelling, followed up via parametric bootstrapping analysis (Preacher et al., 2010)
• Focus on target of intervention (Teacher-student interactions assessed via CLASS-S)
• Using Centered/Standardized data for ease of interpretation.
![Page 32: Effects of the My Teaching Partner Intervention in Secondary School Classrooms](https://reader035.vdocuments.net/reader035/viewer/2022062810/56815ac3550346895dc88d35/html5/thumbnails/32.jpg)
“My Teaching Partner”
Intervention
InterventionIntervention
TargetStudent
Outcome
.37** .16**
Change in Student Achievement
MTP-S Effect as Mediated via Observed Interactions
Initial Model (Simple Direct Effects)
.12*
Observed Teacher-Student
Interactions
![Page 33: Effects of the My Teaching Partner Intervention in Secondary School Classrooms](https://reader035.vdocuments.net/reader035/viewer/2022062810/56815ac3550346895dc88d35/html5/thumbnails/33.jpg)
“My Teaching Partner”
Intervention
InterventionIntervention
Target
.06* *
StudentOutcome
.37** .16**
Change in Student Achievement
MTP-S Effect as Mediated via Observed Interactions
.06 ns
Initial Model (Simple Direct Effects)Final Model (Including Mediated Effect)
.12*
Observed Teacher-Student
Interactions
![Page 34: Effects of the My Teaching Partner Intervention in Secondary School Classrooms](https://reader035.vdocuments.net/reader035/viewer/2022062810/56815ac3550346895dc88d35/html5/thumbnails/34.jpg)
Limitations• Design only supports causal interpretations for outcomes, not
for mediating processes with analyses thus far.
• Some Attrition Took Place (though it was unrelated to the intervention in every possible way we could test).
• Modest statistical significance with small sample
![Page 35: Effects of the My Teaching Partner Intervention in Secondary School Classrooms](https://reader035.vdocuments.net/reader035/viewer/2022062810/56815ac3550346895dc88d35/html5/thumbnails/35.jpg)
Conclusions• We CAN identify elements of the classroom environment that predict
student achievement.
• We CAN change these environmental factors.
• If we do, student achievement will change as well, eventually.
• Changes can be sustained over time and in new classrooms, post-intervention.
• We can identify potential mechanisms of change linked to the intervention.
• Which has implications for cost effectiveness…
![Page 36: Effects of the My Teaching Partner Intervention in Secondary School Classrooms](https://reader035.vdocuments.net/reader035/viewer/2022062810/56815ac3550346895dc88d35/html5/thumbnails/36.jpg)
*BOE = Back of Envelope
Potential Significance – Costs vs. Benefits(BOE* Calculation)
Resources per classroom Estimated Cost
20 Teacher hours No additional cost to system (in lieu of Regular In-service)
1 Teacher-consultant per 20 teachers Maximum of $3,500 per teacher including benefits (Potentially offset by ongoing teacher supervision personnel costs)
Video equipment $200 per teacher
TOTALS: Maximum of $3,700/23 children = $160/child (i.e., < 2% of annual per pupil expenditures) **
BenefitsAverage ‘Bump’ in achievement of ALL students
from 50th to 59th percentileReduction of 1 course failure per classroom
(**Results may apply to multiple classrooms taught by a teacher)
![Page 37: Effects of the My Teaching Partner Intervention in Secondary School Classrooms](https://reader035.vdocuments.net/reader035/viewer/2022062810/56815ac3550346895dc88d35/html5/thumbnails/37.jpg)
MyTeachingPartner Secondary
Replication is ongoing with the support of IES
Further information available at:www.myteachingpartner.net