lindsay clare matsumura helen garnier brian junker lauren resnick donna diprima bickel june 30, 2010...
TRANSCRIPT
LINDSAY CLARE MATSUMURAHELEN GARNIERBRIAN JUNKER
LAUREN RESNICKDONNA DIPRIMA BICKEL
June 30, 2010Institute of Educational Sciences Conference
Evidence on the Implementation and Effectiveness of the
Content-Focused Coaching® Program
Context for the Study
• Literacy coaching is widespread, little evidence that coaching influences instruction and student learning
Research shows that quality of coaching varies significantly across schools
Standards for coach qualifications often not followed (IRA, 2004; 2006)
What it means to be a “coach” is variably defined (Duessen et al, 2007)
Coaching resources used in a diffuse way
Content Focused Coaching
Intensive literacy-coach professional development program developed by the Institute for Learning (IFL) 3 days a month over the academic year led by IFL
fellowsGoals of the coach training
Develop coaching skills Build subject matter knowledge and pedagogical skills
to assist Ts to enact more rigorous reading comprehension lessons Improving quality of class discussions about texts (Questioning
the Author, Beck & McKeown, 2006)
Content-Focused Coaching®• Coaches work withIFL Trainers 3x month
• Principals and District staff attend
District
School
Classroom
Coaches work with Ts: • Weekly in grade-level teams and individually
• Monthly in classrooms to model, observe and co-teach
Ts enact QtA lessonswith Ss in their classroom
Study Design
• Three year study (2006-2009)
• Urban district in Texas 91% of students eligible for free-lunch 80% Hispanic, 15% African American 40% English language learners
• Lowest-performing schools randomly assignment to treatment (n=15) and comparison (n=14) conditions
Data Sources
• Data sources include:
Teacher surveys (baseline and end of each study year) Frequency of participation in literacy coaching Satisfaction with coaching Content of the coaching activities
Coach and principal interviews (once a year)
Classroom observations (twice a year) Quality of text discussions Rigor of text discussions and lesson activities
Student test scores Degrees of Reading Power assessment (twice a year) Texas Assessment of Knowledge and Skills
Overview of the Talk
Part 1: Influence of a school’s social resources on the implementation of the Content-Focused Coaching (CFC) program (Y1) Principal leadership Norms for the professional community Teacher experience
Part 2: Effectiveness of the CFC program (Y1 and Y2): Ts participation in coaching Observed text discussions Ss reading achievement (all Ss; ELLs)
Part 1: Influence of a School’s Social Resources of Teachers’ Coaching
Participation
• Regression analyses based on T survey responses (N=96)
Social resources assessed at baseline Principal leadership Norms for the professional community Professional experience
Teachers’ coaching participation assessed at the end of Y1 Overall frequency of Ts’ participation in coaching Ts perception of the usefulness of coaching Type of coaching activities
• Qualitative analyses of CFC coach interviews
Principal Leadership is Key
• You take the principals to the [CFC] trainings, but honestly, it still comes down to if the principal doesn’t really want the coach to do these things, doesn’t value the coach doing these things, isn’t leading the way so the coach can follow, it just isn’t going to happen.
Principal Leadership
• Principals’ willingness to share leadership significantly predicted:
Frequency of T participation in coaching (p<.05 )
Type of coaching received by Ts Planning and reflecting on instruction (p<.05 ) Lesson enactment (p<.05)
T belief that the coaching they received had improved their practice (p<.01 )
Principals’ past active participation in literacy reforms did NOT predict T participation in coaching
Mechanisms by Which Principals Influence Coaches’ Work
• Interviews with CFC coaches indicate that principals positively influenced their work by:
Actively supporting and participating in the CFC program The day I came the P introduced me to the faculty. She told them that CFC was vital for us to
change our ways of thinking and that it was going to take some time…and that we would be very patient and not despair. They would get it and everybody is learning. She was learning…I was in a learner-student role and they were gonna be in the same role.
CFC isn’t the Ps agenda. It’s happening at her school and she knows that if she tries to block it she will get into trouble, but she is not going to pave the way for me. If the Ts are reluctant or hesitant she is not going to help.
Publicly identifying the coach as a resource for Ts Referring Ts to coach for literacy related questions Holding faculty-wide PD sessions, inviting coach to serve on
leadership committees, watching the coach model lessons in Ts classrooms
Allowing coaches to manage their own schedules• Ps negative relationship with Ts impeded coaches’ work
Norms of the Professional Community
• Strong existing culture of T collaboration negatively predicted:
Type of coaching received by Ts Planning and reflecting on instruction (p<.05 ) Lesson enactment (p<.05 )
T belief that the coaching they received had improved their practice (p<.05 )
Mechanisms by Which Ts Professional Community Influence Coaches’ Work
• Interviews with CFC coaches suggest that: In a few schools Ts were organized against coaching (n=2)
[This school] has a reputation to oust their coach within a year or two. They don’t like coaches at this campus so the longest a coach has been here is two years and then they’re out, they’re gone…
In a few schools with strong professional communities reform goals were not aligned with CFC (n=3)
In some schools with a very weak professional culture, Ts were interested in working with coaches to alleviate their isolation (n=5)
Contrary pattern detected in some schools (n=5)
Teachers’ Years of Experience
• Less experienced teachers participated more frequently in coaching (p<.05 )
New Ts were reported to be more receptive to coaching than more veteran teachers (n=10) New teachers are really positive and appreciative of
getting extra support.
I’m just another person coming into her school trying to save her school…She’s seen my kind so many times before she’s sick of us. So I don’t expect her to be my best buddy anytime soon.
Part 2: Effectiveness of the CFC Program (Y1 and 2)
• What is the influence of the CFC program on teachers’ coaching experiences, reading comprehension instruction and students’ reading achievement?
Participants
• Students (N=1754) 4th and 5th grade 91% eligible for free or reduced price lunch 80% Hispanic; 15% African American 40% English language learners (ELLs)
• Teachers (N=98) 7 years average teaching experience 38% master’s degree 56% teach in both English and Spanish
Analyses
• Hierarchical linear growth models: Amount and type of coaching received by Ts T belief that coaching helped improve their
practice T surveys (baseline and end of each year)
Quality of instruction Observed text discussions (fall and spring of each year)
Student achievement Texas Assessment of Knowledge and Skills (end of each year) Degrees of Reading Power Assessment (fall and spring of each
year)
Effect of CFC on T Participation in Coaching
• Ts in the CFC schools by the end of Y2 Participated more frequently in coaching
ES=.89; p<.000 Were more likely to believe that the coaching they
received was useful to them for improving their practice ES=.95; p<.000
Figure 1:Coach Observed Teacher for 30 Minutes
0
10
20
30
40
50
60
70
CFCF06
CompF06
CFCS08
CompS08
Never
1-3 Year
< 6-8 weeks
Effect of CFC on Types of Coaching Activities
• Ts at the end of Y2 in the CFC schools participated in coaching activities that emphasized
o Building knowledge of the theory underlying effective reading instructiono ES=.70, p<.016
o Planning and reflecting on instructiono ES=.94, p<.002
o Lesson enactmento ES=.91, p<.001
o Differentiating instructiono ES=.76, p<.007
Observations of Classroom Discussions
• T and Ss participation Percent of Ss participating in the discussion T connects Ss contributions Ss connect to each others’ contributions T presses Ss to explain their answers using evidence
from the text Ss use evidence from the text to explain their answers
• Rigor of the lesson Quality (‘grist’) of text discussed Ss opportunity to analyze and interpret a text
Effect of CFC on Instruction
• T and Ss participation Percent of Ss participating in the discussion
ES=.35, p<.005 T connects Ss contributions
ES=.46, p<.003 Ss connect to each others’ contributions
ES=.38, p<.025 T presses Ss to explain their answers using evidence
ES=.33, p<.049 Ss use evidence from the text to explain their answers
ES=.43, p<.006
Effect of CFC on Instruction
• Rigor of the lesson Quality (‘grist’) of the text discussed
ES=.52, p<.012 Ss opportunity to analyze and interpret a text
ES=.39, p<.011
Figure 2. Observation Ratings of Quality of Participation in Class Text DiscussionsFall 2006 to Spring 2008 (Cohort 1, n=98)
Note. Significant change over time indicated in graph by **p<.01. Within-time comparisons indicated the following differences:
Fall 2006, Spring 2007: No differences detected. Fall 2007, Spring 2008: CFC>Comparison, p<.01.
**
Figure 3. Observation Ratings of Rigor of Class Text Discussions Fall 2006 to Spring 2008 (Cohort 1, n=98)
Note. Significant change over time indicated in graph by *p<.05. Within-time comparisons indicated the following differences:
Fall 2006, Spring 2007, Spring 2008: No differences detected. Fall 2007: CFC>Comparison, p<.01.
*
*
• No effect of CFC was detected on student achievement for the total sample of students (N=1754)
• CFC positively predicts ELL student achievement on the TAKS
ES=.61, p<.01
Effect of CFC on Student Achievement
Percent of Ss participating in the discussion ES=.85, p<.000
Ss use evidence from a text to support their answers ES=.33, p<.026
Quality of the text discussed ES=.45, p<.08
Dimensions of Instruction Associated with
Improved Achievement for ELL Students
• ELLs are the fastest growing subgroup in the U.S.
• The reading achievement of ELL students is very low 7.5% proficient 70% below-basic
Recent NAEP Findings (2007)
• Contextual factors in schools significantly influenced the initial implementation of CFC Principal leadership played a key role Less experienced Ts were more receptive to coaching Some evidence of a negative relationship between the
strength of the existing professional norms and coaches’ work with Ts
• CFC mostly showed positive effects on desired outcomes Strong effect on Ts coaching experiences and attitude
toward coaching Moderate effect on reading comprehension instruction Moderate effect on reading achievement of ELL students
only (40% of the sample)
Summary of Findings
Publications
Matsumura, L.C., Garnier, H., Resnick, L.B. (in press). Implementing literacy coaching: The role of school social resources. Educational Evaluation and Policy Analysis
Matsumura, L.C., Garnier, H.E., Correnti, R., Junker, B., & Bickel, D.D. (in press). Investigating the effectiveness of a comprehensive literacy-coaching program in schools with high teacher mobility. Elementary School Journal.
Matsumura, L.C., Sartoris, M., Bickel, D., & Garnier, H. (2009). Leadership for literacy coaching: The principal’s role in launching a new coaching program. Educational Administration Quarterly, 45(5), 655-93.
For more information: [email protected]