2013 ITQ Final Report: Teacher and Student Impact

March 2014

1

Table of Contents

Executive Summary ..................................................................................................................... 4 Evaluation Report ....................................................................................................................... 10 Program Impact on Teachers ...................................................................................................... 10

Teacher Surveys ................................................................................................................. 10 Teaching Strategies and Skills ............................................................................................ 12 Scale 1: Teacher Ratings of Student Engagement ............................................................. 16 Scale 2: Teacher Ratings of Student Skills Related to Writing and Reading. .................... 16 Scale 3: Teacher Ratings of Implementation of Strategies ................................................ 17 Working with Poet Mentors ............................................................................................... 18 Challenges .......................................................................................................................... 21 Teacher perceptions of program activities ......................................................................... 22

Teacher Survey Summary .......................................................................................................... 26 Poet Mentor Surveys .................................................................................................................. 28 Poet Mentor Survey Summary ................................................................................................... 36 Program Impact on Students ...................................................................................................... 37

Student Survey Analysis ..................................................................................................... 37 Summary Student Survey Analysis .................................................................................... 41

Summary of Principal Interviews ............................................................................................... 41 Student Achievement Analysis .................................................................................................. 42

Data Analysis ...................................................................................................................... 43 Summary of Student Achievement Analyses ............................................................................. 45 Summary of Evaluation .............................................................................................................. 45 Appendix A: ITQ Student Survey Poet Mentor Class 2012-2013 ............................................. 47 Appendix B: ITQ Student Survey Comparison Class 2012-2013 .............................................. 48 Appendix C: ITQ Teacher Survey 2012-2013 ........................................................................... 49 Appendix D: ITQ Poet Mentor Survey 2012-2013 .................................................................... 58 Appendix E: Teacher Survey Scale Items .................................................................................. 66 Appendix F: Mean Scores for Outcome Variables by Treatment Group ................................... 67 Appendix G: Regression Models with Treatment Group ........................................................... 68

2

List of Tables & Figures Figure 1. Comparison of Teacher and Student Ratings of Skills Gained ....................................... 6 Figure 2. Teacher Responses by School ....................................................................................... 10 Figure 3. Ethnicity of Respondents ............................................................................................... 11 Figure 4. Subjects Taught ............................................................................................................. 11 Figure 5. Write Now Program Skills ............................................................................................ 12 Figure 6. Teaching Skills .............................................................................................................. 13 Figure 7. Literacy Seminars Attended 2013-2014 ........................................................................ 17 Figure 8. Frequency of Collaboration with Poet Mentors 2013-2014 .......................................... 18 Figure 9. Time Spent Collaborating with Poet Mentors ............................................................... 19 Figure 10. Supporting Poet Mentors ............................................................................................. 19 Figure 11. Impact of the Poet Mentor on Students ....................................................................... 20 Figure 12. Poet Mentor Strategies ................................................................................................ 21 Figure 13. Challenges Encountered by Teachers ......................................................................... 22 Figure 14. Satisfaction with Write Now Program ........................................................................ 23 Figure 15. Comparison between 2011-2012 and 2012-2013 Teacher Survey Responses ............ 24 Figure 16. Time Commitment for Write Now Program ............................................................... 24 Figure 17. Teacher Program Improvement Suggestions ............................................................... 25 Figure 18. Satisfaction with Lesson Study ................................................................................... 27 Figure 19. Poet Mentor Schools.................................................................................................... 28 Figure 20. Poet Mentor Classes .................................................................................................... 28 Figure 21. Poet Mentor Teaching Strategies ................................................................................. 29 Figure 22. Improvement in Teaching Skills ................................................................................. 30 Figure 23. Poet Mentor Ratings of Gains in Students' Skills ........................................................ 31 Figure 24. Impact on Classroom Climate ..................................................................................... 32 Figure 25. Poet Mentor Impact on Teachers ................................................................................. 33 Figure 26. Teacher Activities During Poet Mentor's Lessons ...................................................... 34 Figure 27. Challenges Encountered by Poet Mentors ................................................................... 35 Figure 28. Poet Mentor Program Improvement Suggestions........................................................ 36 Figure 29. Student Perceptions of their Teacher's Behavior and Opinions .................................. 37 Figure 30. Student Perceptions of Engagement ............................................................................ 38 Figure 31. Student Perceptions of Growth in Skills ..................................................................... 39 Figure 32. Student Perceptions of Impact of Poet Mentor............................................................ 40 Table 1. Summary of Regression Models ....................................................................................... 9 Table 2. Teacher Ratings in Poet Mentor and Non-Poet Mentor Classes 2013-2014 .................. 14 Table 3. Teacher Ratings of Student Skills in Poet Mentor vs. Non-Poet Mentor Classes 2013-2014............................................................................................................................................... 15 Table 4. Scale 1 Mean Ratings ..................................................................................................... 16 Table 5. Scale 2 Mean Ratings ..................................................................................................... 16 Table 6. Scale 3 Mean Ratings ..................................................................................................... 17 Table 7. Student Demographic Information ................................................................................ 42 Table 8. Summary of Regression Models ..................................................................................... 45

3

Executive Summary

The Write Now ITQ Program is completing the final year of a three year grant in the Twin Rivers Unified School District. The program combined targeted professional development for teachers with the placement of poet mentors in classrooms to facilitate spoken word activities and provide support to students and teachers. The evaluation of the program included teacher, poet mentor, and student surveys, along with interviews with principals from participating schools, and analyses of student level achievement data. The following are highlights from the full evaluation report. Teacher Perspectives

• Teacher survey results indicate that teachers found the Write Now program helped them improve their teaching practice and provided new methods for them to connect with their students. Over 80% of the teachers agreed that Write Now impacted their teaching and increased their skills in the following ways:

o Learning how spoken word can contribute to students’ literacy skill development, o Learning new instructional strategies to engage students, o Learning creative techniques to build students’ literacy skills, and o Learning new ways to connect with their students on a personal basis

• Similarly, 80% or more of the teachers agreed that their teaching skills and perspectives

had improved in the following areas: o Permanently changing their approach to teaching, o Reinvigorating their commitment to teaching, o Gaining information/skills specifically from attending the Write Now literacy

seminars, o Encouraging students to incorporate their own experiences in their class work,

and o Helping students connect class content to their lives.

• Teachers were also asked to compare student improvement in their classes with a Poet

Mentor Residency to those in a similar non-residency class. While teachers reported general improvement among all their students, they reported larger gains in the students in Poet Mentor classrooms in a number of areas, including:

o Becoming more respectful of one another, o Becoming more engaged in class content and classroom activities, o Becoming more interested in going to college, o Seeing more connections between class work and their own lives, o Increasing their desire to express themselves creatively, o Participating more in class discussions, and o Attending class more regularly.

4

• The differences between teacher’s ratings of students in poet-mentor classes compared to non-poet mentor classes were statistically significant for the following items:

o Looking forward to coming to class, o Writing for enjoyment, and o Enjoying writing.

• Teacher ratings of both their own skills and their students’ skills were compared across the three years of the program and their ratings were consistent across time with little or no variation. On targeted skills teachers gave consistently higher ratings to students in their poet mentors classes when compared to their non-poet mentor classes.

• Finally, it is important to note that participating teachers were asked, each year, to choose their class with the most at-risk youth for the Poet Mentor residency. Consequently, the improvements that both teachers and students report specifically about classrooms with poet mentors are particularly relevant to the impact of the intervention on this population of students.

Student Perspectives Students of the treatment teachers, in both their classes with and without poet mentors, were generally positive about their relationships with their teachers, with at least three quarters of the students perceiving their teacher:

o Believed they would be successful, o Encouraged students to express their opinions in both class discussions and

writing, and o Noticed when they were absent.

Students in classrooms with poet mentors responded more positively on each of the survey items, compared to the other students. Further, they were more likely than the students in the non-poet mentor classes to report that they looked forward to coming to class and were both more interested in writing than they had been at the beginning of the school year and perceived themselves to be more skilled writers by the end of the school year. Students in the poet mentor classes were asked a series of additional questions specifically about how they perceived the contribution of the poet mentor to their enjoyment and skill development. Over 75% of the students in these classes reported that the poet mentor had:

o Helped them understand the value of being a good writer and speaker to their lives,

o Helped them become a better writer and speaker, and o Made them want to come to class.

Survey items were compared across teacher and students (poet mentor and non-poet mentor classes) and there was general agreement between the groups about the skills that students and teachers felt they had gained from being a part of the program. For example, 70% of teachers agreed that the program had improved student writing skills and 63% of students in poet mentor classes agreed with this same statement (See Figure 1). This suggests that both teachers and students agreed that students showed growth in key skills through the program and that this

5

agreement was stronger between students with poet mentors and their teachers than for students without poet mentors. Figure 1. Comparison of Teacher and Student Ratings of Skills Gained

Poet Mentor Perspectives Poet mentors reported seeing increases in student skills related to creative writing, the ability to express themselves, and making connections between school work and their own lives. Overall, poet mentors reported that students in their classes seemed engaged and were less resistant to class work and participating in class discussions by the end of the school year. Poet mentors reported that their own teaching skills and knowledge of teaching had increased greatly by participating in the program. Specifically poet mentors felt that they had gained skills related to helping students express their ideas and become better at creative writing. Poet mentors felt that they provided a positive role model to students. Poet mentors also reported that the teachers they worked with learned new skills and were more able to connect with students in their classes. For example one poet mentor said, “There has been an increase in strategies and techniques used in the classroom. The program has been tremendous in affecting the lives of teachers and students.” This aligns with what teachers reported in terms of gaining new skills which allowed them to make stronger connections with their students and make academic content more relevant to students. Principal Interviews Interviews were conducted with two of the principals participating in the Write Now Program. Both of the principals described the positive impact the program had on teachers and students at their schools. They spoke about how the program had transformed teachers and given them new

79%

63%

54%

62%

73%

56%

51%

55%

95%

70%

75%

80%

0% 20% 40% 60% 80% 100%

Incorporate personal experiences and opinions

Improve writing skills

Improve verbal skills

Understand the connection between course content and

everyday lives

Teachers that Agree

Students without Poet Mentors that Agree

Students with Poet Mentors that Agree

6

skills and strategies to engage students. Both principals also described how the program provided a new and constructive way for students and teachers to communicate and relate to one another. One of the principals said, “For an urban school the SAYS program has really supported our efforts to create a positive culture…The student voices have really effected teachers.” When asked how the program would be sustained in the coming years, both of the principals expressed a desire to continue the program if they were able to find adequate resources and funding to do so. Student achievement analyses In depth analysis of student achievement data was conducted to determine how the Write Now Program had impacted student achievement on CSTs, CMAs, 7th Grade Writing CSTs, CAHSEE, and GPA. Regression models were developed comparing treatment grouping variables (comparing year 3 students in a treatment teacher class to other groups of students). The three treatment models tested were: (a) Students in treatment teachers’ classrooms with and without a poet mentor; (b) students in treatment teachers’ classrooms compared to other students in the district (in non-treatment schools); and (c) students in treatment teachers’ classrooms compared to other students in the treatment schools. Regression Analysis Independent variables included in each model include the treatment group indicator, student demographics (home language, ethnicity, whether the student qualifies for free/reduced price lunch), disability status, whether the student’s ethnicity matched that of the teacher, and the previous year’s measure of the dependent variable (for example, when 2013 ELA CST scores were the variable of interest, the student’s 2012 ELA CST score was entered as an independent variable to control for students’ ability when they entered the class). For the models focused on the treatment schools, teacher effects were controlled for by entering a “dummy variable” 1 for each teacher and the models comparing across treatment and non-treatment schools controlled for school effect by entering dummy variables for each school. We report any relationships that were significant at the p<.10 level. Model 1: Students within treatment teachers’ classes. When comparing students in poet mentor residence classes to those in treatment teacher classes without a poet mentor, regression analyses find no differences in outcomes between the classes except on the outcome variables of 7th grade writing scores and CMA scores, where students in the poet mentor classes had statistically significant higher scores (p<.10) than those in the non- poet mentor treatment teacher courses. The Beta (standardized regression coefficient) for 7th grade writing scores was .433 and the b (unstandardized regression coefficient) was .546. This means that students in poet mentor classrooms had 7th grade writing scores that were, on average, half a point higher than students in non-poet mentor classrooms, when controlling for the other independent variables. The writing grade is scored on a four point (1-4) scale. The Beta for CMA scores was .696 and the b was 150.45. This means that a student in a poet mentor class showed, on average, a 150 point higher CMA score than a student in a non-poet mentor class. This is a large effect size for CMA scores which ranged from 157 to 515 in the sample and had a standard deviation of 65. So, students in poet mentor classes had CMA scores that were on average over two standard deviations higher

1 A dummy variable is a statistical term which refers to a unique variable created to capture variance within individuals and or settings.

7

than students in non-poet mentor treatment classes. It is important to note that the number of students in each of these models was quite small (N<50), making it difficult to confidently generalize from these results. Further, particularly with small numbers of students in each group, the large variations among students who take the CMA, in terms of their specific disabilities, also complicates interpretation of these results. Given this, the findings are promising and suggest that the program is having a positive impact for students who take the CMA. Because most of the outcome variables in model one showed no differences between students in the poet mentor classes and those in other treatment teacher classes (and the Ns and effect sizes are small where significant differences were found), both the poet mentor class and non-poet mentor class students were combined as “treatment” students in models comparing treatment and non-treatment students. This provides as large a treatment group as possible, to avoid masking possibly significant relationship. For example, even though the students in the Poet Mentor classes showed significantly higher 7th grade writing and CMA scores, restricting the treatment group to only these students would result in the test group Ns being 29 and 42 respectively for these outcome measures. Model 2: Students within treatment teachers’ classes compared to those in non-treatment schools. When comparing treatment students to non-treatment students at other schools in the district, students in treatment teachers’ classrooms had significantly lower CMA scores when compared to other students in the district. The Beta for CMA scores was -.181 and the b was -56.41. This means that students in non-treatment schools in the district had, on average, a 56 point higher CMA score when compared to treatment students. Again, this pattern of results is difficult to interpret. Because the types of disabilities of students classified as “disabled” vary widely in terms of their potential impact on student learning, interpreting differences in CMA scores is problematic. While controlling for the previous year’s CMA score should control for some of this variation, it is unclear whether the significant findings reflect true differences, particularly when comparing treatment students to non-treatment students in other schools, or if the effect may be due to other factors. There were no other significant effects of treatment group found in this model. Model 3: Students within treatment teachers’ classes compared to those in non-treatment classes within the treatment schools. There were no significant impacts of being in a treatment teacher class in year 3 for any of the outcome variables in the model comparing treatment students to other students within the treatment schools. A summary of regression model results is displayed in Table 1 below.

8

Table 1. Summary of Regression Models

ITQ Models

1. Treatment students: (Pm vs Non-PM)

2. Treatment students vs. students in non-treatment schools

3. Treatment students vs. other students in treatment schools

Outcome variables (2013) p (n) p (n)

p (n)

ELA CST Scale score N/S (509) N/S (2365) N/S (1241) ELA CMA score 0.059 (37)a .017 (152)b N/S (50) 7th grade writing 0.055 (33)a N/S (368) N/S (101) CAHSEE ELA N/S (204) N/S (625) N/S (354) Overall GPA N/S ( 634) N/S (2741) N/S (1579) Academic GPA N/S (628) N/S (2704) N/S (1553)

Note: Any results with a p<.10 are reported. a = significant difference with treatment (PM) students showing higher scores on this variable. b = significant difference with non-treatment students showing higher scores on this variable. Summary The Write Now program focused at least as much on developing teachers’ abilities to connect with and engage their students (which all survey measures suggest was successfully accomplished) as it did on technical professional development focused on content specific instructional knowledge and pedagogy. Further, the pedagogy addressed was specific to writing, which is only one component of the skills measured by the majority of the outcome measures. It seems unrealistic to expect to find significant impacts on student test scores when engagement and writing skills are the primary focus of the program and the majority of the student achievement outcome measures encompass a considerably different range of skills and knowledge. We suggest that the meaningful results of the program are better captured by the qualitative information provided by the surveys than by the quantitative results reflected by test scores and GPA.

9

Evaluation Report This report summarizes the evaluation data collected in the 2012-13 school year for the Write Now ITQ project at Twin Rivers Unified School District. Data collection included teacher and student surveys, interviews with principals, and an analysis of student achievement data as measured by GPA, CST, and CAHSEE scores. Teacher and student survey data provides information on overall program impact. Student achievement data analyses illustrate how the program impacted participating students’ academic outcomes.

Program Impact on Teachers Teacher Surveys Participant teachers were surveyed at the beginning and end of Year 2. The pre-survey was sent to 23 teachers and 20 teachers responded (87%). The post-survey was sent to 24 teachers (one teacher joined the program during the year) and 20 completed the survey (83%). Unfortunately, of the 20 teachers who entered the pre-survey, 19 teachers skipped 10 of the 15 questions making pre-post comparisons difficult. The questions that were answered on the pre-survey included basic background information. Teachers were asked if this was their first year participating in the Write Now Program and the majority (18 of 19) said that it was not. The teachers were asked to identify the school in which they were working on both the pre- and post-survey. Six of the teachers who completed the post-survey were from schools other than Vista Nueva, Grant and Martin Luther King. Apparently teachers in the program had moved to other schools during the course of the year. The number of teachers who completed the survey pre and post by school are shown in Figure 2 below. Figure 2. Teacher Responses by School (Pre- and Post)

6

3

10

1

5

11

1

0 2 4 6 8 10 12

Other

Martin Luther King Middle School

Grant High School

Vista Nueva

Pre Post

10

Participating teachers were asked to identify their ethnicity on the surveys and responses are displayed below (Figure 3). Approximately 47% of the teachers responding to the survey identified themselves as African American, 23% white, 17% Hispanic, and 12% Asian-American. Figure 3. Ethnicity of Respondents

When asked to identify what subject(s) they taught the majority of teachers (94%) were either teaching English or Special Education, with a small number (6%) teaching in a continuation setting (See Figure 4). Figure 4. Subjects Taught

When asked how long they had been teaching, eighteen of the twenty teachers responded to this item. On average, they have been teaching for twelve years, three of the seventeen teachers had

0.0% 5.0%

10.0% 15.0% 20.0% 25.0% 30.0% 35.0% 40.0% 45.0% 50.0%

African American Asian American White, Non-Hispanic

White, Hispanic heritage

Other (please specify)

0% 10% 20% 30% 40% 50%

An English teacher

A Special Education teacher

A continuation/alternative school teacher

11

been teaching for three or four years, however the majority of teachers had been teaching for six or more years. Teaching Strategies and Skills In the pre and post-survey, teachers were asked a variety of questions about the Write Now Program and how it had impacted their teaching practices and their students. On some items teachers were asked to respond separately for their poet mentor and non-poet mentor classes. As mentioned above, only one of the teachers who entered the pre-survey answered these questions so we are unable to report pre-post survey comparisons for these items and only responses from the post-survey will be reported here. In the post-survey teachers were asked when they began participating in the Write Now Program. The majority (13 of 20) had been involved in the program since 2009 or 2010 and most had been participating in the program for three or more years. Teachers were asked: “Reflecting on your experiences as a teacher participating in the Write Now Program, to what extent do you agree with each of the following statements” (See Figure 5 below). The majority of teachers (82%) either agreed or strongly agreed with these eight statements regarding the Write Now Program. For example, nineteen of the twenty teachers responded that they strongly agreed or agreed with the statement: “I learned how spoken word can contribute to improving students’ literacy skills.” Figure 5. Write Now Program Skills (N=20)

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

It helped me improve my students' reading skills.

It helped me improve my students' writing skills.

It deepened or changed my understanding of writing.

I learned new ways to connect with my students on a personal basis.

I learned creative techniques to build my students' literacy skills.

I learned new instructional strategies to engage my students.

It helped me improve my students' public speaking skills.

I gained an appreciation of how spoken word can contribute to improving students' literacy skills.

Strongly agree Agree Neither agree nor disagree Disagree Strongly disagree

12

In the post-survey, teachers were also asked to rate their agreement with a series of statements regarding how the Write Now Program had impacted their teaching and their students’ work (See Figure 6). Overall, teachers either agreed or strongly agreed (78 %) with these eleven statements. The responses show that teachers rated highly the skills they gained from working with a poet mentor. Teachers reported that they had learned new techniques to help students connect experiences from their own lives with their writing and class work. Teachers also reported that the program made permanent changes in the way they will teach and that participating reinvigorated their commitment to teaching. The item with the least amount of agreement was the statement: “It helped me convey high expectations to my students” which seven teachers (65%) rated as “neither agree nor disagree.” Figure 6. Teaching Skills (N=20)

0% 10% 20% 30% 40% 50% 60% 70%

It helped me assist my students in understanding the connections between course content and their

everyday lives and culture.

It helped me encourage my students to incorporate their own experiences and opinions in their class writing

assignments and class discussions.

I learned new strategies to differentiate my writing instruction to meet my students' individual needs.

It helped me establish a culture of respect in my classroom(s).

It helped me convey high expectations to students to encourage them to achieve academically.

I gained valuable information/skills during the Literacy Seminars.

I gained valuable information/skills from having a Poet Mentor in my classroom.

It encouraged me to collaborate more with other teachers.

It reinvigorated my commitment to teaching.

It made me a better writer.

It made permanent changes in the way I will teach in the future.

Strongly Agree Agree Neither Agree nor Disagree Disagree Strongly disagree

13

Improving Teacher Quality Teachers were asked about different teaching strategies they were applying in both their poet mentor classes and their non-poet mentor classes. When asked to respond to the item: “Over this past school year (2012-13), how often in a typical month, have each of the following occurred in your class or classes?” the majority of teachers (92%) responded that they were using these strategies in their poet mentor classes and most (76%) were also using them in their non-poet mentor classes. (See Table 2). This is an increase from the previous year when the overall percentages were 76% for poet mentor classes, and 61% for non-poet mentor classes. The two statements with the highest agreement for non-poet mentor classes were, “Gave students writing assignments to be completed in class,” (83%) and “Asked for students’ opinions about materials or assignments” (83%). This suggests that teachers were using these two strategies consistently in both poet-mentor and non-poet mentor classes. One of the goals of the program was to move from teacher driven to student centered instruction and teachers’ responses suggest that this was taking place. When these percentages for teacher ratings of their skills were compared to previous year’s survey results, it was found that there was a high level of agreement across years2. Teachers showed consistently high ratings for these items in all three years. Table 2. Teacher Ratings in Poet Mentor and Non-Poet Mentor Classes 2013-2014 (N=19)

I have done the following activities at least 2-3 times a month in my:

Poet Mentor Class (%)

Non-Poet Mentor

Class (%) Explicitly integrated different art forms into your classroom activities 94.7 77.8 Had students read poetry 100.0 77.8 Had students write poetry 94.8 72.3 Gave students writing assignments to be completed in class 99.9 83.3 Gave students writing assignments to be completed as homework 73.7 72.2 Had students work in groups to develop and complete writing assignments 89.5 77.8 Had students work in groups to critique or discuss one another’s work 89.5 66.7 Had students read their writing out loud to the class 100.0 72.3 Used materials that students have suggested for classroom discussion or assignments 89.5 77.7 Used students’ writing as examples for your classroom instruction 89.5 77.8 Asked for students’ opinions about materials or assignments 94.7 83.3 Overall

92.3 76.3

2 See 2012 ITQ Annual Report: Teacher and Student Impact for complete results from the previous year (http://education.ucdavis.edu/post/cees-projects).

14

Teachers were asked a series of questions that required them to rate the skills of their students in a variety of areas. In general, teachers reported at the time of the post-survey that students in their poet mentors classes (91%) were doing these specific target skills more often than students in their non-poet mentor classes (75%) (Table 3). Several of the items received 100% agreement from teachers, such as students “became more respectful of one another,” and “increased their desire to express themselves creatively.” This suggests that teachers felt the program was successful in creating a more positive classroom climate that encouraged students to express themselves and connect with classroom activities and academic content. Differences between the teachers’ rating of the poet mentor and non-poet mentor classes were compared using a T-Test and were found to be statistically significant (p<.001). Interestingly, these overall ratings are very similar to the previous year in which teachers rated their poet mentor classes at 88% overall for these skills and their non-poet mentor classes at 74% overall on these skills and activities. Table 3. Teacher Ratings of Student Skills in Poet Mentor vs. Non-Poet Mentor Classes 2013-2014 (N=19)

The students did each of the following over the past year: Poet Mentor Class (%)

Non-Poet Mentor Class

(%) Became more respectful of one another 100.0 88.9 Became more engaged in the content and classroom activities 94.7 77.8 Became more interested in going to college 94.7 77.8 Could see more connections between class work and their own lives 95.0 83.3 Increased their desire to express themselves creatively (e.g. music, writing, speaking, dance, visual arts, etc.) 100.0 83.3 Participated more in classroom discussions 94.7 77.8 Attended class more regularly 89.5 72.2 Completed class assignments more frequently 78.9 77.8 Gained interest in getting feedback on their writing 78.9 50.0 Exhibited more self-control 89.5 66.7 Became more curious and interested in the world 89.5 77.8 Became less resistant to writing personal essays 89.5 66.7 Gained skills in writing personal essays 84.2 77.8 Became less resistant to expository writing 78.9 66.7 Gained skills in expository writing 89.5 77.8 Became less resistant to creative writing (e.g. fiction or poetry) 100.0 83.3 Gained skills in creative writing 94.7 72.2 Became less resistant to reading their work aloud 94.7 72.2 Gained skills in reading aloud 89.5 77.8 Overall 90.8 75.1

15

In order to gain a better understanding of how teacher ratings may have changed over time, scales were developed from the teacher survey to compare teachers’ responses across the three years of the project. Scales were developed in three areas: student engagement, student reading and writing skills, and teacher implementation of strategies. The individual items that make up each scale are listed in Appendix E. Scale 1: Teacher Ratings of Student Engagement Correlations between the items within this scale were moderate (Pearson’s R = .79). This scale captures teachers’ ratings of overall student engagement which as the Program Director notes is a precursor to successful teaching which was a primary goal of the grant. Examples of individual items in this scale include: Became more engaged in the content and classroom activities; Could see more connections between class work and their own lives. Comparisons between 2011, 2012 and 2013 demonstrate increases in student engagement with writing. Furthermore, teachers reported that students were exhibiting engagement and connecting with class content more in their PM classes in all three years. T-tests were used to analyze the mean ratings across the three years and there were significant differences (p<.05) between poet mentor and non-poet mentor classes across the three years (Table 4). Table 4. Scale 1 Mean Ratings

Year PM Classes Non-PM Classes 2011 2.85 2.56 2012 3.11 3.00 2013 3.32 2.88 Overall 3.09 2.81 Scale 2: Teacher Ratings of Student Skills Related to Writing and Reading. Correlations between the items within this scale were moderately high (Pearson’s R = .89). Comparisons between 2011, 2012 and 2013 showed that teachers reported that students were exhibiting these skills more in their PM classes in all three years. Examples of individual items in this scale include: Became less resistant to expository writing; Gained skills in expository writing. T-tests were used to analyze the mean ratings across the three years and there were significant differences between poet mentor and non-poet mentor classes across the three years. When the mean ratings for the three years in this scale were analyzed using T-tests there were significant differences (p<.02) between poet mentor and non-poet mentor classes (Table 5). Table 5. Scale 2 Mean Ratings

Year PM Classes Non-PM Classes 2011 2.84 2.65 2012 2.99 2.72 2013 3.30 2.87 Overall 3.04 2.74

16

Scale 3: Teacher Ratings of Implementation of Strategies Correlations between the items within this scale were low to moderate (Pearson’s R = .63). T-tests were used to analyze the mean ratings across the three years and there were significant differences between poet mentor and non-poet mentor classes in 2012 and 2013 but not in 2011 (p<.05) (Table 6). Examples of individual items in this scale include: Had students work in groups to develop and complete writing assignments; Had students work in groups to critique or discuss one another's work; Used materials that students have suggested for classroom discussion or assignments. This is the only scale that showed significant change in teacher ratings between the three years. Teachers in classes with poet mentors rated themselves consistently more highly on implementing these strategies than they did for their non-poet mentor classes. Table 6. Scale 3 Mean Ratings

Year PM Classes Non-PM Classes 2011(N=27) 3.10 3.13 2012 (N=19) 3.19 2.85 2013 (N=18) 3.64 3.26 On the post-survey, teachers were also asked about their participation in the Literacy Seminars, and how the Literacy Seminars and Poet Mentor Residencies helped them with teaching strategies that address reading and writing skills. In general, teachers were very satisfied with both the Literacy Seminars and Poet Mentor Residencies in terms of improving their teaching skills. Teachers were asked what percentage of the literacy seminars they attended. The majority of teachers (90%) responded that they had attended “most” or the “vast majority” of the Literacy Seminars (See Figure 7).

Figure 7. Literacy Seminars Attended 2013-2014 (N=18)

0.0%

10.0%

20.0%

30.0%

40.0%

50.0%

60.0%

Vast majority - 90- 100% Most - 75 - 89% Some - 50-74% A few - less than 50%

17

Working with Poet Mentors In terms of the participation of the poet mentors inside the treatment classrooms, the majority (61%) of teachers said that their poet mentors were in their classrooms for the “vast majority” of their assigned weekly residency dates. Another 21% of teachers said that the poet mentors were in their classrooms for their weekly residency date “most” of the time.

In addition to the Literacy Seminars and in-class residency, when asked about the frequency of collaborating with poet mentors on their own time, teachers reported having a range of experiences. Thirty percent of teachers responded that they collaborated once a week, 17% of teachers said they collaborated three times per month, and 33% said they collaborated two times per month. Another 17% of teachers said that they collaborated with their poet mentors less than once per month and 6% said that they collaborated together once a month (Figure 8). When compared to the 2012-2013 survey results, there appears to be more consistency in terms of how much time was spent collaborating. For example, in 2011-2012, 37% of teachers said they were collaborating with their poet mentors at least once a week however 26% said that they were collaborating with their poet mentors less than once per month. This is higher than the current year’s percentage of 16%. This suggests that as the teachers and poet mentors developed a partnership that their willingness to meet one-on-one and collaborate on lessons increased. Figure 8. Frequency of Collaboration with Poet Mentors 2013-2014 (N=18)

In terms of the actual time spent collaborating with their poet mentor, 39% of teachers were spending 15 minutes or less collaborating, 44% were spending 15-30 minutes, and 11% were spending between 30 and 60 minutes collaborating (See Figure 9). These responses represent an overall increase in the amount of time teachers were spending collaborating with their poet mentors from the previous year. In the 2011-2012 year survey results showed the majority of teachers (53%) were spending less than 15 minutes collaborating with their poet mentors.

0.0%

5.0%

10.0%

15.0%

20.0%

25.0%

30.0%

35.0%

At least once a week

3 times per month

2 times per month

1 time per month Less than once per month

18

Figure 9. Time Spent Collaborating with Poet Mentors (N=18)

When asked about the types of activities that they participated in when their poet mentor was in the classroom, the majority of teachers (83%) said that they “always” or “often” supported the poet mentor’s lesson. Seventy-seven percent said that they participated in the lesson with their students. 38% of the teachers said they co-taught with their poet mentor. This is an increase from 2012 when 26% of teachers responded that they co-taught lessons with their poet mentor. Other responses to this item are shown in Figure 10 below. Figure 10. Supporting Poet Mentors (N=18)

39%

44%

11% 6%

15 minutes or less 16 - 30 minutes 31 - 60 minutes 1 hour or more

1%

1%

22%

22%

38%

77%

83%

0% 10% 20% 30% 40% 50% 60% 70% 80% 90%

Observed the lesson without participating

Used the class time to catch up on other responsibilities

Taught the lesson with the Poet Mentor observing and providing feedback afterwards

Taught the lesson with the Poet Mentor providing assistance

Co-taught a lesson

Participated in the lesson with my students.

Supported the Poet Mentor's lesson (e.g. supervised, helped students, posted examples)

19

When asked about the impact of the poet mentor on their class, the majority of teachers (83%) said that the poet mentor gave their students a positive role model, and encouraged students to reflect on their own lives (“to a great extent” or “somewhat”). Another 78% of teachers said that the poet mentors had improved their students’ confidence in their own writing (“to a great extent” or “somewhat”). The other answer options were “a little” and “not at all.” These choices are not included in the summary display for this item (Figure 11). Figure 11. Impact of the Poet Mentor on Students (N=18)

Teachers were also asked to what extent they had helped the poet mentor they worked with to learn new teaching strategies or learn classroom management skills and the majority (67%) said that they had helped in both of these areas either “to a great extent” or “somewhat.” Responses are shown below in Figure 12. This demonstrates that not only were teachers learning from their poet mentors but poet mentors were learning important teaching strategies and skills from their partner teachers.

33%

44%

44%

50%

61%

67%

67%

72%

78%

83%

83%

33%

17%

44%

28%

28%

11%

17%

22%

11%

6%

11%

0% 10% 20% 30% 40% 50% 60% 70% 80% 90%

I helped the Poet Mentor learn about classroom management.

I helped the Poet Mentor learn teaching strategies.

Increased regular student attendance.

Building bridges between me and my students.

Encouraging my students to write more, overall, not just poetry.

Helped my students recognize why writing is important to their lives.

Building a safe and respectful classroom community.

Encouraged my students to participate more in class.

Improved my students' confidence in their own writing and speaking abilities.

Encouraging my students to reflect on their own lives both in discussion and writing.

Gave my students a positive role model.

Somewhat To a great extent

20

Figure 12. Teacher Helped Poet Mentor Learn New Strategies (N=18)

Challenges Teachers were asked to identify the primary challenges they had encountered while they participated in the Write Now Program. They were given a list of potential challenges and asked to say whether this item was “very challenging,” “somewhat challenging,” or “not an issue.” Overall, the majority of teachers (90%) said that most of the items were “not an issue.” The item that teachers identified as the most challenging was “Having enough time to incorporate spoken word poetry and still meet my course objectives/pacing guide.” The items that teachers identified as not being an issue were, “Developing good rapport with the poet mentor,” “Opening my classroom to a poet mentor,” and “Lack of school site support.” These are the same items that teachers found both challenging and not an issue during the 2011-2012 school year (Figure 13).

0

5

10

15

20

25

30

35

40

45

Learn new teaching strategies Learn classroom management skills

To a great extent

Somewhat

A little

Not at all

21

Figure 13. Challenges Encountered by Teacher (%) (N=18)

Teacher perceptions of program activities Teachers were asked to rate their satisfaction with the Write Now Program in a variety of areas (Figure 14). Overall, the majority of teachers (N=16, 88%) were either “satisfied” or “somewhat satisfied” with the program. The only item that received more than two teacher ratings of “very unsatisfied” was “The reliability of the poet mentor to be in class when scheduled.”

0% 20% 40% 60% 80% 100%

Finding time to collaborate with the Poet Mentor outside of class

Having enough time to incorporate spoken word poetry and still meet …

Students' lack of interest in spoken word poetry

Drawing connections between the spoken word content and traditional …

Clarity about the project process

Clarity about the project goals

My own lack of comfort with poetry or spoken word

Lack of department support

Lack of district support

Lack of school site support

Opening my classroom to a Poet Mentor

Developing good rapport with the Poet Mentor

Not an issue

Somewhat Challenging

Very challenging

22

Figure 14. Rate Your Satisfaction with Write Now Program (N=18)

When comparisons were made to the 2011-2012 survey responses, teachers appeared to be equally satisfied with the overall program components including strategies used by poet mentors, and the content of the Literacy Seminars (Figure 15). Overall, teacher ratings of satisfaction with the content taught by poet mentors increased from 77% to 89% between the two years.

11%

6%

17%

6%

11%

6%

6%

6%

6%

33%

28%

22%

22%

22%

17%

56%

61%

61%

67%

67%

72%

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

The support and structure provided by the program

The strategies used by the Poet Mentors

The reliability of the Poet Mentor to be in class when scheduled

The content taught by the Poet Mentors

The content of the Literacy Seminars

The support provided by my school's administration

Very Unsatisfied Somewhat Unsatisfied Somewhat Satisfied Satisfied

23

Figure 15. Comparison between 2011-2012 and 2012-2013 Teacher Survey Responses (Percent “Satisfied” or “Somewhat Satisfied”)

Teachers were also asked to rate the amount of time required for different components of the Write Now Program. The majority of teachers (78%) said that there was not enough time available for collaborating with their poet mentor outside of class. A large number (59%) also responded that there was not enough time devoted to giving class time to the poet mentors. By contrast the majority (94%) of the teachers said the right amount of time was required for the Literacy Seminars, and 72% said the right amount of time was required for the Summer Institute (Figure 16). Figure 16. Time Commitment for Write Now Program (N=18) (%)

89%

89%

89%

77%

82%

94%

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Content taught by Poet Mentors

Strategies used by Poet Mentors

Content of Literacy Seminars

2011-2012 2012-2013

0%

6%

11%

0%

22%

35%

72%

94%

78%

59%

17%

6%

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Time available for collaborating with the Poet Mentors outside of class

Class time required for Poet Mentor Residency

Summer Institute

Literacy seminars

Not enough time Right amount of time Too much time

24

When asked if they planned to continue using the strategies learned in the Write Now Program 100% of the teachers who responded said yes. Given the fact that the Write Now Program only has one more year in these schools, teachers were asked if they were interested in becoming a teacher leader to help develop and deliver trainings for other teachers/schools on the strategies they had learned in the program. The majority of teachers (61%) said yes, 17% said maybe, and another 17% said they were not sure. Only 6% of the teachers said that they were not interested in becoming teacher leaders. Teachers were asked how they would change the Write Now Program if they could and the majority (94%) said they would increase opportunities to collaborate with poet mentors. Another 67% of teachers said that they would “provide more time for poet mentors to work in the classroom”, and “build on what we have done so far by continuing the program for another year or two” (See Figure 17). Figure 17. Teacher Program Improvement Suggestions (N=18)

50.0%

50.0%

50.0%

56.0%

67.0%

67.0%

94.0%

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Improve the training of poet mentors in teaching strategies and classroom management

Improve the climate of trust and mutual respect among poet mentors, teachers, and program

providers

Spend more time in literacy seminars on instructional techniques and practicing new skills

Spend more time training on how to manage discussing divisive topics (such as race) with

students

Provide more time for Poet Mentors to work in the classroom (e.g. more days per week or longer class

time)

Build on what we have done so far by continuing the program for another year or two

Increase opportunities/settings for poet mentors and teachers to collaborate

25

Teacher Survey Summary Student Skills In the spring 2013 (post) survey administration, teachers reported improvements among their students in both the poet mentor and in their non-poet mentor classes on nearly every behavior/skill. All of the teachers (100%) completing the survey said that students in their poet mentor classes had become more respectful of one another, had increased their desire to express themselves creatively, and became less resistant to creative writing. Students in the poet mentor classes were perceived as having improved more than their comparison class cohort in all the skill areas highlighted in the survey, however the gains for students in non-poet mentor classes were substantial (73% overall) with the exception of one area, “Gained interest in getting feedback on their writing.” Teacher perceptions of program activities Teachers were generally very satisfied with their experiences in both the Literacy Seminars and having the poet mentors in residence in their classrooms. Most (82%) felt they learned more about teaching, writing, spoken word activities, and strategies to engage with their students. The majority (95%) agreed that as part of the Write Now Program they learned how spoken word can contribute to improving students’ literacy skills and public speaking skills, and learned new ways to connect with students on a personal basis. Most teachers reported that they were using skills learned in the program in their classrooms regularly – 92% in poet mentor classes, and 76% in non-poet mentor classes. Summary of teacher perceptions Based on responses to the 2013 teacher survey, it appears that the Write Now Program is valued by teachers both for what they are learning and for the contributions it makes directly (via the poet mentor) and indirectly (via strategies they are implementing) to their students’ learning. Overall, teachers reported using the strategies they learned in the program in both their poet mentor (92%) and non-poet mentor (76%) classes. These responses are higher than in last year’s teacher surveys where 76% of teachers reported using the strategies learned in poet mentor classes, and 61% in non-poet mentor classes. This suggests that teachers are implementing the strategies across their classes and using them more consistently. Lesson Study The final section of the survey asked teachers about their participation in Lesson Study as part of the ITQ Write Now Program. When asked if they had been involved in Lesson Study, 56% (N= 10) of the teachers responded that they had not, and 44% (N=8) responded that they had participated. When asked how often they met with their lesson study group, teachers said that they met with their group from one to ten times (average 3.5). When asked to rate specific statements related to their experience with lesson study, most teachers either agreed or strongly agreed with six of the eight statements. All of the teachers who responded to the survey responded that Lesson Study gave them valuable opportunities to work together, collaborate and re-teach lessons. The statements that teachers felt were least true were: “We had an opportunity to teach our lesson, revise and re-teach” and “Lesson study allowed me to work more deeply with my poet mentor.” Results are displayed below in Figure 18.

26

Figure 18. Satisfaction with Lesson Study (N=8)

12.5%

25.0%

25.0%

12.5%

12.5%

12.5%

25.0%

25.0%

37.5%

37.5%

37.5%

50.0%

50.0%

50.0%

62.5%

Lesson study allowed me to work more deeply with my poet-mentor.

We had an opportunity to teach our lesson, revise and reteach.

It gave me an opportunity to stretch my skills and try something new.

The experience helped me focus on student responses to the lessons I teach.

Lesson study increased my ability to create culturally relevant writing lessons.

The experience increased my interest in collaborating with other teachers.

Lesson study increased my ability to connect spoken word with traditional essay writing.

Overall, I found the lesson study experience to be time well spent.

Lesson study allowed me to work more deeply with other teachers.

Agree Strongly Agree

27

Poet Mentor Surveys

Poet mentors were sent an end of the year survey in May of 2013, at the same time as teachers were surveyed. The survey was sent to eight poet mentors and six poet mentors completed the survey. The majority (5 of 6) of the poet mentors who completed the survey said they were working at Grant High School, although poet mentors were also working in three other school settings and were working at multiple schools (See Figure 19).

Figure 19. Poet Mentor Schools (N=6)

When asked to identify how many classes they had worked with as part of the ITQ Write Now Program, poet mentors responses ranged from 1 to 5 with two classes receiving the highest number of responses (See Figure 20). Figure 20. Poet Mentor Classes (N=6)

5

2 2

1

0

1

2

3

4

5

6

Grant High School Martin Luther King Middle School

Vista Nueva Career and Technology High School

Other

0

1

2

3

1 2 3 4 5

Num

ber o

f Poe

t Men

tor R

espo

nses

Number of Classes per Poet Mentor

28

When asked how many of the literacy seminars they attended, poet mentor responses were similar to teacher responses. The majority (83%) of poet mentors said they had attended most or the vast majority of the literacy seminars. Poet mentors were also asked about teaching strategies that they had used while working with students, and how often they used the strategies (once a week, 3 times per month, 2 times per month, 1 time per month, or less often). All of the poet mentors (100%) said that they were using whole class teaching either once a week or 3 times per month. The other strategies were not being used as consistently. Co-teaching with the teacher was used by most (59%) of the poet mentors once a week or three times a month. At least once a month 88% of poet mentors said they observed the teacher’s lesson and provided feedback (See Figure 21). Figure 21. Poet Mentor Teaching Strategies (N=6)

Poet mentors were asked to rate the extent to which the ITQ Write Now Program had improved their skills in certain areas and the majority (99%) responded that their skills in these areas had improved “somewhat” or “to a great extent.” On two of the items, knowledge of teaching, and teaching skills, all of the poet mentors rated their skills as improving “to a great extent” (Figure 22).

6%

6%

12%

24%

24%

65%

18%

29%

18%

35%

6%

53%

29%

35%

53%

6%

41%

29%

29%

41%

24%

88%

29%

Coaching teacher

Observing the teacher's lesson and providing feedback

Co-teaching with teacher

Collaborating with the teacher outside of class time

One-on-One tutoring with students

Small group work with students

Whole class teaching

At least once per week 3 times per month 2 times per month

1 time per month <1 time per month

29

Figure 22. Improvement in Teaching Skills

Poet mentors were asked to rate the skill levels of their students following their participation in the ITQ Write Now Program. The majority of poet mentors (90% overall) responded that their students had gained skills either “somewhat” or to a “great extent” in all of the areas outlined. 100% of the poet mentors said that their students had “become more curious and interested in the world.” Other areas of note include 83% of poet mentors responding that following participation in the ITQ Write Now Program they felt that their students had improved to “a great extent” in the following areas: “could see more connections between class work and their own lives,” “increased their desire to express themselves creatively,” and “gained skills in creative writing.” Responses to this item are displayed in Figure 23 below and are consistent with findings from the student and teacher surveys.

0% 20% 40% 60% 80% 100% 120%

Knowledge of spoken word

Ability to help teachers connect with students

Ability to help teachers improve students' writing

Understanding of students' cultures

Classroom management skills

Knowledge of academic writing

Knowledge of student learning

Ability to impact school culture

Ability to facilitate respectful, open communication among students

Ability to help students express their ideas in writing

Ability to help students express their ideas verbally

Knowledge of teaching

Teaching skills

To a Great Extent

Somewhat

30

Figure 23. Poet Mentor Ratings of Gains in Students' Skills

0% 20% 40% 60% 80% 100%

Became more engaged in the content and classroom activities

Completed class assignments more frequently

Became more respectful of one another

Attended class more regularly

Came to class on time more frequently

Exhibited more self-control

Gained skills in expository writing

Became less resistant to reading their work aloud

Became more interested in going to college

Participated more in classroom discussions

Gained interest in getting feedback on their writing

Became less resistant to writing personal essays

Gained skills in writing personal essays

Became less resistant to expository writing

Became less resistant to creative writing (e.g. fiction or poetry)

Gained skills in reading aloud

Could see more connections between class work and their own lives

Increased their desire to express themselves creatively (e.g. music, …

Gained skills in creative writing

Became more curious and interested in the world

To a great extent

Somewhat

31

Poet mentors were also asked to rate the extent to which they felt that their presence in the classroom as part of the ITQ Write Now Program had impacted both students and the classroom climate. Similar to teacher ratings, the majority (83%) of poet mentors responded that they felt their participation had “to a great extent” provided students with a positive role model. Another 83% of poet mentors said that their presence in the classroom had “to a great extent” encouraged students to participate more in class (Figure 24).

Figure 24. Impact of Poet Mentors on Classroom Climate

Poet mentors were asked to think about how the program had impacted the classroom teachers they worked with, and specifically how they interacted with students. The majority (89% overall) of poet mentors responded that they felt the program had either “somewhat” or “to a great extent” increased teachers’ abilities to connect with their students and make content more relevant. For example, 100% of the poet mentors felt that teachers participating in the program had increased their abilities to encourage students to use their own experiences and opinions in class discussions. Responses to this item are shown in Figure 25 below.

17%

67%

67%

83%

83%

67%

17%

33%

17%

17%

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Increasing students' school attendance

Building a bridge between teachers and their students

Building a safe and respectful classroom community

Encouraging students to participate more in class

Providing students with a positive role model

To a great extent Somewhat

32

Figure 25. Poet Mentor Impact on Teachers

Poet mentors had a range of responses to items asking them to rate how often they did a variety of activities with their partner teachers. Poet mentors were asked to respond for each separate teacher they worked with but we are presenting overall responses across teachers and poet mentors. The responses showed that, according to poet mentors, teachers most often supported their lessons by helping students. The majority of poet mentors (66%) reported that the teachers they worked with either “always” or “often” participated in the lesson alongside the students. This mirrors the teachers’ responses about what they did during the poet mentor lessons. Poet mentors did not report that teachers co-taught with them often. Responses to this item are found in Figure 26 below and show that there were a wide range of responses to this item, indicating that poet mentors had different experiences with different partner teachers.

33.3%

33.3%

33.3%

33.3%

33.3%

16.7%

66.7%

66.7%

66.7%

66.7%

66.7%

83.3%

100.0%

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Ability to help students see the connection between course content and their everyday lives and cultures

Ability to choose instructional materials that are relevant to students' everyday lives and cultures

Ability to encourage students to use their own experiences and opinions in class discussions

Use of alternative teaching strategies to engage students

Ability to connect with their students

Understanding of spoken word as an expressive art form that can build students' literacy skills

Ability to encourage students to use their own experiences and opinions in their class writing

assignments

Somewhat To a Great Extent

33

Figure 26. Teacher Activities During Poet Mentors’ Lessons

On the final section of the survey, poet mentors were asked about challenges they had encountered while participating in the program and ways they might improve the program. In terms of challenges, the majority of poet mentors (80%) identified not having enough class time to work with students as either “somewhat challenging” or “very challenging.” The other area highlighted as a challenge in poet mentors’ responses was finding enough time to collaborate with teachers outside of class, which 67% said was “somewhat challenging.” The items that poet mentors rated as “not an issue” most frequently (88%) were: “Teachers’ knowledge of spoken word,” and “Teachers’ attitudes towards spoken word.” Responses to this item are displayed in Figure 27 below.

0% 20% 40% 60% 80% 100%

Use the class time to catch up on other responsibilities

Participate in the lesson alongside the students

Co-teach with you

Observe the lesson without participating

Support your lesson by helping students, posting examples, etc.

Always

Often

Sometimes

Seldom

Never

34

Figure 27. Challenges Encountered by Poet Mentors

Poet mentors identified similar items to teachers when asked for their suggestions for program improvement. The majority (83%) of poet mentors responded that they wanted to build on what the program had done so far and continue working as poet mentors. Another 50% of poet mentors responded that they would like to see more time provided for poet mentors to work in the classroom. Poet mentors’ responses to how they would improve the program are displayed in Figure 28 below.

0% 20% 40% 60% 80% 100%

Finding time to collaborate with the teacher(s) outside of class

Having enough class time to work with students

Teachers' expectation of their students

Students' lack of interest in spoken word poetry

Lack of department support

Lack of school site support

Developing good rapport with the teacher(s)

Clarity about the project goals

Clarity about the project process

Teachers' attitudes toward spoken word poetry

Teachers' knowledge of spoken word poetry

Very challenging

Somewhat challening

Not an issue

35

Figure 28. Poet Mentor Program Improvement Suggestions

Poet Mentor Survey Summary In the post survey, poet mentors reported seeing increases in student skills related to creative writing, the ability to express themselves, and making connections between school work and their own lives. Overall, poet mentors reported that students in their classes seemed engaged and were less resistant to class work and participating in class discussions. Poet mentors reported that their own teaching skills and knowledge of teaching had increased greatly by participating in the program. Specifically poet mentors felt that they had gained skills related to helping students express their ideas and become better at creative writing. Poet mentors felt that they provided a positive role model to students. Poet mentors also reported that the teachers they worked with learned new skills and were more able to connect with students in their classes. This aligns with what teachers reported in terms of gaining new skills which allowed them to make stronger connections with their students and make academic content more relevant to students.

0.0%

16.7%

16.7%

16.7%

33.3%

50.0%

83.3%

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Spend more time in literacy seminars on instructional techniques and practicing new skills

Provide more training for Poet Mentors in teaching strategies and classroom management

Improve the climate of trust and mutual respect among poet mentors, teachers, and program …

Spend more time training on how to manage discussing divisive topics (such as race) with …

Increase opportunities/settings for poet mentors and teachers to collaborate

Provide more time for the Poet Mentors to work in the classroom (e.g. more days per week or longer …

Build on what we have done so far by continuing the program for another year or two

Response Percent

36

Program Impact on Students Student Survey Analysis

Students in both the classes with the poet mentor and those in the teacher selected comparison classes were asked to complete a paper and pencil survey near the end of the school year (Spring 2012-13). Teachers administered the survey and sent the hard copies to the Center for Education and Evaluation Services (CEES) for analysis. Survey items were analyzed using the Mann Whitney U test, the results of which are discussed in this section.

Students were asked about things that may reflect their teachers’ efforts to relate to and engage with students. Specifically, they were asked the following: My teacher: (a) understands what my life is like, (b) encourages me to write about my own experiences and opinions, (c) notices when I’m not here, (d) encourages students to express opinions in class discussions, (e) believes I will be a success, (f) includes examples of my racial, ethnic, or cultural background in class lesson, and (g) encourages me to write about my own experiences. Both Poet Mentor class and non-Poet Mentor class students were asked these questions, and the results can be found in Figure 29 below.

Figure 29. Student Perceptions of their Teacher's Behavior and Opinions. My teacher: (Percent “Totally” or “Kind of” Agree) (N=367)

Statistically significant differences (p<.05) are indicated by *

For all statements, the majority of students, regardless of whether they were in a poet mentor or non-poet mentor class, reported a fairly high level of agreement. In both the poet mentor classes and the non-poet mentor classes, the statement, “My teacher believes I will be a

55%

69%

74%

73%

78%

72%

84%

62%

70%

78%

79%

82%

83%

90%

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Understands what my life is like

Includes examples of my racial, ethnic, or cultural background in class lessons

Notices when I'm not here

*Encourages me to write about my own experiences

*Encourages me to write about my own opinions

Encourages students to express opinions in class discussions

*Believes I will be a success

Poet Mentor Class Non-Poet Mentor Class

37

success,” had the largest percentage of students either “totally” agreeing or “kind of” agreeing, 93.0% and 85.2% respectively. Contrarily, the statement, “My teacher understands what my life is like,” had the smallest percentage of students either “totally” agreeing or “kind of” agreeing for both types of classes with more students in the poet mentor classes (66.3%) than those in the comparison classes (55.7%) agreeing with the statement. Statistically significant differences (Mann Whitney U Test) in level of agreement between the students in poet mentor and non-poet mentor classes appeared for the statements: My teacher “believes I will be a success” (p=.019), “encourages me to write about my own experiences” (p=.012), “encourages me to write about my own opinions” (p=.001). These results suggest that teachers are perceived as caring about all their students, but that the students do not necessarily equate this with high expectations.

Students were also asked about other aspects of their engagement in school, classroom and class activities, and expectations. Results suggest that, by the end of the school year, students in each type of class (with and without poet mentors) were more interested in going to college (almost 80% of both groups said they wanted to attend college “more/a lot more”), although just over half of the students said they are “more” or “a lot more” interested in school (Figure 30). Students in the poet mentor classes were significantly (p<.05) more likely to report that they now look forward to coming to class (57.0%), enjoyed writing (48.1%), and wrote for personal enjoyment more (51.3%) than was true in the comparison classes (47.0%, 33.3%, and 38.2%, respectively).

Figure 30. Percent of Students Responding "More/A Lot More" to "Compared to the beginning of the school year, I now:"

Statistically significant differences (p<.05) are indicated by *

33%

38%

45%

52%

78%

42%

47%

52%

54%

77%

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

*Enjoy writing

*Write for personal enjoyment

*Look forward to coming to this class

Am interested in school

Want to attend college

Poet Mentor Class Non-Poet Mentor Class

38

Additionally, students were asked about their perceptions regarding whether they had improved their communication skills and if their grades in English had improved. As Figure 31 demonstrates, more than half the students in both groups felt they had improved their grades and gained skills in writing and expressing themselves in both writing and verbally. The largest perceived skill gains were in writing with 56.8% of the comparison students reporting they were “better” or “a lot better” writers and 66.7% of the students in the poet mentor classes reporting the same (a statistically significant larger proportion, p<.05). Another statistically significant difference appeared for the item, “I express myself when I talk.” More students in the poet mentor classes (55.7%) reported that they were “better” or “a lot better” on this skill than comparison students (51.1%).

Figure 31. Percent of Students Reporting They are "better" or "a lot better" Compared to the Beginning of the Year in Each Area.

Statistically significant differences (p<.05) are indicated by *

Taken together, these results suggest that most students believe they had gained skills and were fairly well engaged in school and writing. Moreover, this suggests that the treatment teachers’ self-reported similarities in activities and strategies in both their classes (poet mentor and non-poet mentor) are positively received by their students in both classes. Generally, students in their poet mentor classes appear somewhat more engaged and interested in writing than those in the comparison classes. This is significant given the fact that teachers selected their most resistant students who had little or no interest in writing.

51%

51%

54%

56%

5%

54%

61%

63%

0% 10% 20% 30% 40% 50% 60% 70%

My grades in English are

*I express myself when I talk

I express myself when I write

*My writing skills are

Poet Mentor Class Non-Poet Mentor Class

39

To explore more about how students responded to the poet mentors, in particular, an extra set of questions was included in the surveys of the poet mentor classes, asking specifically how the experience of having a poet mentor influenced the student in various areas (Figure 32). Just as teachers reported, students found the poet mentors to be very engaging, encouraging them to come to class and helping them improve their skills in writing, speaking, and connecting with both their teacher and one another. More than three-quarters of the students responded that having the poet mentor in class made them want to come to class more, helped them understand the importance of writing and public speaking, and assisted them in becoming better writers. The item with the lowest percent of “totally” or “kind of” responses was for “made me enjoy speaking” (65.5%).

Figure 32. Percent of Students Responding "Kind of" or "Totally" to "Having a poet mentor in my class" items

17.3%

20.1%

20.9%

21.9%

24.1%

25.5%

26.3%

28.8%

30.9%

29.5%

27.0%

23.4%

26.3%

30.6%

27.0%

25.2%

27.0%

23.4%

0% 5% 10% 15% 20% 25% 30% 35%

Made me enjoy writing

Helped me connect with my teacher

Made me want to come to class

Helped me become a better speaker

Helped me become a better writer

Helped me to understand how being a good public speaker will help me in my

life, outside of school

Helped me to understand how being a good writer will help me in my own life,

outside of school

Helped me connect with other students

Made me enjoy speaking

Kind of

Totally

40

Summary Student Survey Analysis

Student survey responses indicate that students respond positively to having a poet mentor in their classes and that they perceive that they are making improvements in English, particularly in the area of writing. Students in poet mentor classes also reported that they expressed themselves both in writing and when speaking more clearly after being in a poet mentor class. In addition, over fifty percent of the students in the poet mentor classes said that they looked forward to coming to class. Treatment students in both poet mentor and non-poet mentor classes reported that they were more interested in attending college when compared to the beginning of the year. Overall, the student surveys suggest that treatment students are engaged in the program and feel that they had made improvements in their writing and speaking skills.

Summary of Principal Interviews

Principals from four schools participating in the ITQ Write Now Program were contacted by the evaluation team to be interviewed about their experiences having the program at their school sites. Two of the principals agreed to be interviewed and provided their feedback about the program. Both of the principals described the positive impact the program had on teachers and students at their schools. They spoke about how the program had transformed teachers and given them new skills and strategies to engage students. For example, one of the principals mentioned, “They (teachers) have an increased understanding of the lives of urban kids.” One of the principals described how the teachers had really grown professionally during the program and how including students’ voices and perspectives had increased the effectiveness of teachers. One of the principals talked about an individual teacher, “She was always an effective teacher but now to call her effective would be an understatement. The students’ voices have really impacted teachers.” When asked to provide specific examples of how the program had positively impacted students, one of the principals mentioned, “It gives kids the tools to express their feelings and relate to what they are doing in their English classes.” Both principals also spoke about how the program provides a new and constructive way for students and teachers to communicate and relate to one another. In terms of school culture, the principals interviewed described a culture of inclusion where students were accepted and allowed to express themselves. One of the principals said, “It really brought everyone together. Seeing kids from a variety of backgrounds participating together is powerful.” Both of the principals mentioned how the program had helped develop a more positive school culture for students and teachers. When asked how the program would be sustained in the coming years, both of the principals expressed a desire to continue the program if they were able to find adequate resources and funding to do so.

41

Student Achievement Analysis The district provided student level data for all students in the district. The data included demographic information as well as data on CST scores, CMA scores, CAHSEE scores, GPA, and a 7th Grade Writing CST scores. Student level data for these same variables was available across years so it was possible to control for the prior years’ achievement in the analyses. For the purpose of the student achievement analysis, students were divided into four groups: (a) treatment teacher classes with poet mentors; (b) treatment teacher classes, (c) non-treatment teacher classes in the treatment schools and, (d) non-treatment teacher classes in non-treatment schools. Demographic information for each group can be found in Table 7. Table 7. Student Demographic Information (%)

Treatment Teacher with Poet Mentor

Treatment Teacher

Non-Treatment Teacher/

Treatment School

Non-Treatment Schools

N 339 467 1558 3640 Grade Level

7 12% 3% 9% 13% 8 9% 13% 12% 14% 9 14% 25% 21% 16%

10 27% 36% 17% 18% 11 20% 15% 21% 20% 12 17% 8% 19% 19%

Gender Male 61% 53% 53% 53%

Female 39% 47% 47% 47% Ethnicity

Asian 22% 24% 26% 10% Black/African

American 33% 25% 22% 20%

Hispanic/Latino 37% 45% 42% 41% White 8% 8% 9% 28% Other 1% 1% 1% 1%

Family Income* Low Income 89% 85% 88% 77%

Not Low Income 11% 15% 12% 23% Disability Status No Disability 73% 89% 85% 84%

Disability 27% 11% 15% 16% Home Language

English 57% 52% 46% 63% Non-English 43% 48% 54% 36%

*Family income is determined based on whether or not students qualified for free or reduced price lunch.

42

For clarification, treatment schools in the sample include Grant High School, Martin Luther King Middle School, and Vista Nueva Career and Technology School. There are non-treatment students in the treatment schools, who are not in a treatment teacher’s class. The non-treatment schools include all other schools in the Twin Rivers Unified School District, with the exception of seven junior high schools which are not included in the analyses. As can be seen in the demographic table above, even when these junior high schools are not included there are still more 7th and 8th graders in the non-treatment schools than in the treatment schools. This method was used to attempt to make the five groups used in the analysis as well matched as possible. These groups also allow for comparisons between treatment teachers’ students (with and without poet mentors) and students in treatment schools and non-treatment schools. Data Analysis For descriptive purposes, mean scores were calculated for the four treatment groups for all outcome variables (Appendix F). The outcome variables are CST scores, CMA scores, 7th Grade CST writing scores, CAHSEE scores, and both overall and academic GPA. Three regression models were used to analyze the achievement data to determine if differences existed on the outcome measures between the four treatment groups. All regression models controlled for previous years’ scores, ethnicity, socioeconomic status, and home language. In addition, an ethnicity match variable was created which showed when teacher and student ethnicity was the same. Regression Analysis Independent variables included in each model include the treatment group indicator, student demographics (home language, ethnicity, whether the student qualifies for free/reduced price lunch), disability status, whether the student’s ethnicity matched that of the teacher, and the previous year’s measure of the dependent variable (for example, when 2013 ELA CST scores were the variable of interest, the student’s 2012 ELA CST score was entered as an independent variable to control for students’ ability when they entered the class). For the models focused on the treatment schools, teacher effects were controlled for by entering a dummy variable for each teacher and the models comparing across treatment and non-treatment schools controlled for school effect by entering dummy variables for each school. We report any relationships that were significant at the p<.10 level. Model 1: Students within treatment teachers’ classes. When comparing students in poet mentor residence classes to those in treatment teacher classes without a poet mentor, regression analyses find no differences in outcomes between the classes except on the outcome variables of 7th grade writing scores and CMA scores (alternative assessment for special education students), where students in the poet mentor classes had statistically significant higher scores (p<.10) than those in the non- poet mentor treatment teacher courses. The Beta (standardized regression coefficient) for 7th grade writing scores was .433 and the b (unstandardized regression coefficient) was .546. This means that students in poet mentor classrooms had 7th grade writing scores that were, on average, half a point higher than students in non-poet mentor classrooms, when controlling for the other independent variables. The writing grade is scored on four point (1-4) scale. The Beta for CMA scores was .696 and the b was 150.45. This means that a student in a poet mentor class showed, on average, a 150 point higher CMA score than a student in a non-poet mentor class.

43

This is a large effect size for CMA scores which ranged from 157 to 515 in the sample and had a standard deviation of 65. So, students in poet mentor classes had CMA scores that were on average over two standard deviations higher than students in non-poet mentor treatment classes. It is important to note that the number of students in each of these models was quite small (N<50), making it difficult to confidently generalize from these results. Further, particularly with small numbers of students in each group, the large variations among special education students who take the CMA also complicate interpretation of these results. Because most of the outcome variables in this model showed no differences in achievement between students in the poet mentor classes and those in other treatment teacher classes (and the Ns and effect sizes are small where significant differences were found), both the poet mentor class and non-poet mentor class students were combined as “treatment” students in models comparing treatment and non-treatment students. This provides as large a treatment group as possible, to avoid masking possibly significant relationships due to low numbers of treatment students. For example, even though the students in the Poet Mentor classes showed significantly higher 7th grade writing and CMA scores, restricting the treatment group to only these students would result in the test group Ns being 29 and 42 respectively for these outcome measures. Model 2: Students within treatment teachers’ classes compared to those in non-treatment schools. When comparing treatment students to non-treatment students at other schools in the district, students in treatment teachers’ classrooms had significantly lower CMA scores when compared to other students in the district. The Beta for CMA scores was -.181 and the b was -56.41. This means that students in non-treatment schools in the district had, on average, a 56 point higher CMA score when compared to treatment students. Again, this pattern of results is difficult to interpret. Because the types of disabilities of students in receiving special education services vary widely in terms of their potential impact on student learning, interpreting differences in CMA scores is problematic. While controlling for the previous year’s CMA score should control for some of this variation, it is unclear whether the significant findings reflect true differences, particularly when comparing treatment students to non-treatment students in other schools, or if the effect may be due to other factors that were not adequately controlled for in the models. There were no other significant effects of treatment group found in the regression models using the dichotomous measure of whether a student was in a treatment teachers’ class in year 3 when compared to other students in the district. Model 3: Students within treatment teachers’ classes compared to those in non-treatment classes within the treatment schools. There were no significant impacts of being in a treatment teacher class in year 3 for any of the outcome variables in the model comparing treatment students to other students within the treatment schools. A summary of regression model results is displayed in Table 8 below.

44

Table 8. Summary of Regression Models

ITQ Models

1. Treatment students: (Pm vs Non-PM)

2. Treatment students vs. students in non-treatment schools

3. Treatment students vs. other students in treatment schools

Outcome variables (2013) p (n) p (n)

p (n)

ELA CST Scale score N/S (509) N/S (2365) N/S (1241) ELA CMA score 0.059 (37)a .017 (152)b N/S (50) 7th grade writing 0.055 (33)a N/S (368) N/S (101) CAHSEE ELA N/S (204) N/S (625) N/S (354) Overall GPA N/S ( 634) N/S (2741) N/S (1579) Academic GPA N/S (628) N/S (2704) N/S (1553)

Note: Any results with a p<.10 are reported. a = significant difference with treatment (PM) students showing higher scores on this variable. b = significant difference with non-treatment students showing higher scores on this variable.

Summary of Student Achievement Analyses In summary, the regression analyses show that treatment group had a positive impact on CMA scores and 7th Grade CST Writing scores for students who were in treatment teachers’ classrooms with poet mentors when compared to students in treatment teacher’s classrooms without a poet mentor. By contrast the opposite result was found when treatment students were compared to all other students in the district. Students in treatment teachers’ classrooms had significantly lower CMA scores than other students in the district. There were no other significant effects of treatment group found in these models.

Summary of Evaluation Survey responses from teachers, students and poet mentors show that there were consistent levels of satisfaction with, and growth in skills from participation in the program. All three groups of individuals reported that the program had increased their skills in specific ways. For example teachers and poet mentors reported that their skills in teaching, connecting with students and making content relevant to students’ lives had increased. Students who participated in the program perceive that they are making improvements in English, and more specifically in writing. Students in the program also reported that they express themselves both in writing and when speaking more clearly after participation in the treatment teacher class. Students who participated in the program also reported that they were more interested in attending college when compared to the beginning of the year. Overall, students in both poet mentor and non-poet mentor classes reported being engaged in the program and perceive improvements in their writing and speaking skills.

45

Although results from the student achievement data are mixed, this is not surprising given that comparisons were being made both across the district, across schools, and across classrooms. These environments are complex and it is difficult to capture all of the variance associated with teacher, student and school differences. It is also difficult to make direct links between the program components and goals and the achievement level outcome variables (CST, CMA, CAHSEE, and GPA). The most closely aligned student achievement data was the 7th grade writing assessment but unfortunately this measure was only given to a small subset of students.

46

Appendix A: ITQ Student Survey Poet Mentor Class 2012-2013

47

Appendix B: ITQ Student Survey Comparison Class 2012-2013

48

Appendix C: ITQ Teacher Survey 2012-2013

49

50

51

52

53

54

55

56

57

Appendix D: ITQ Poet Mentor Survey 2012-2013

58

59

60

61

62

63

64

65

Appendix E: Teacher Survey Scale Items

Teacher Survey Scales – Individual Survey Items Included in Each Scale Scale 1 – Teacher Ratings of Student Skills Related to Engagement PM vs. Non-PM (Pearson’s R=.79) Became more respectful of one another Became more engaged in the content and classroom activities Became more interested in going to college Could see more connections between class work and their own lives Increased their desire to express themselves creatively (e.g. music, writing, speaking, dance, visual arts, etc.) Participated more in classroom discussions Attended class regularly Came to class on time more frequently Completed class assignments more frequently Gained interest in getting feedback on their writing (Removed from scale, low correlation, SD above 1) Exhibited more self-control (Removed from scale, low correlation, SD above 1) Became more curious and interested in the world

Scale 2 – Teacher Ratings of Student Academic Skills related to Spoken Word/Writings PM vs. Non-PM (Pearson’s R=.89)

Became less resistant to writing personal essays Gained skills in writing personal essays Became less resistant to expository writing Gained skills in expository writing Became less resistant to creative writing (e.g. fiction or poetry) Gained creative writing skills Became less resistant to reading their work aloud

Scale 3 – Teacher Strategies PM vs. Non-PM (Items specifically related to poetry and spoken word were removed because it was assumed that these would be used more frequently in PM classes.) Pearson’s R=.63 Gave students writing assignments to be completed in class Gave students writing assignments to be completed as homework Had students work in groups to develop and complete writing assignments Had students work in groups to critique or discuss one another's work Had students read their writing out loud to the class Used materials that students have suggested for classroom discussion or assignments Used students' writing as examples for your classroom instruction Asked for students' opinions about materials or assignments

66

Appendix F: Mean Scores for Outcome Variables by Treatment Group

Year Treatment Teacher

with Poet Mentor Treatment Teacher

without Poet Mentor

Non-Treatment Teacher/

Treatment School

Non-Treatment School

Mean CST Score 2012

321.21(N=228) 343.68(N=369) 325.88(N=1238) 339.05(N=2687)

Mean CST Score 2013

315.51(N=228) 339.52(N=399) 317.65(N=1114) 332.50(N=2485)

Mean CST Writing

Score 2013 2.690(N=29) 2.417(N=12) 2.270(N=115) 2.633(N=415)

Overall GPA 2.200 (N=331) 2.374 (N=464) 2.323 (N=1544) 2.247 (N=3231)

Academic GPA 2.133(N=329) 2.348 (N=463) 2.272 (N=1519) 2.150 (N=3182)

CAHSEE 341.59(N=95) 378.83(N=167) 354.06(N=262) 365.39(N=642)

Mean CMA Scaled Score

2012

292.25 (N=43)

291.75 (N=12)

276.22

(N=81) 289.79

(N=206)

Mean CMA Scaled Score

2013

275.46 (N=46)

260.25

(N=8) 269.66

(N=66) 278.76

(N=186)

67

Appendix G: Regression Models with Treatment Group

Table 1. Model One: Treatment Teachers with and without Poet Mentors 2013 CST Scores N=509 Variables B Std. Error Beta Sig CST ELA 2012 .690 .029 .722 .000 Free/Reduced Lunch

2.408 3.512 .018 .493

Language .059 2.809 .001 .983 Ethnicity 1.349 5.967 .006 .821 Ethnicity Match (Teacher/Student)

1.934 3.965 .016 .626

Treatment Group (PM/No PM)

-3.164 2.966 -.032 .287

Table 2. Model One: Treatment Teachers with and without Poet Mentors 2013 CMA Scores N=37 Variables B Std. Error Beta Sig CMA 2012 .721 .166 .696 .000 Language 34.323 46.647 .195 .470 Ethnicity 12.590 41.944 .125 .767 Ethnicity Match (Teacher/Student)

150.452 75.377 .780 .059

Treatment Group* (PM/No PM)

.721 .166 .696 .059

Table 3. Model One: 2013 CST 7th Grade Writing Scores N=33 Variables B Std. Error Beta Sig CST ELA 2012 -.001 .003 -.097 .659 Free/Reduced Lunch

-.842 .387 -.372 .039

Language .106 .225 .094 .642 Ethnicity* Ethnicity Match (Teacher/Student)

-.075 .459 -.033 .872

Treatment Group (PM/No PM)

.546 .272 .433 .055

*N’s too small to include ethnicity variable.

68

Table 4. Model One: 2013 CAHSEE Scores N=204 Variables B Std. Error Beta Sig CST ELA 2012 .466 .029 .869 .000 Free/Reduced Lunch

5.729 3.318 .063 .086

Language -3.290 2.614 -.052 .210 Ethnicity -.102 9.165 .000 .991 Ethnicity Match (Teacher/Student)

2.179 3.769 .031 .564

Treatment Group (PM/No PM)

3.260 3.312 .047 .326

Table 5. Model One: GPA Overall N=634 Variables B Std. Error Beta Sig Overall GPA2011-12

.847 .023 .878 .000

Free/Reduced Lunch

.017 .049 .007 .723

Language -.081 .038 -.045 .033 Ethnicity -.095 .081 -.025 .238 Ethnicity Match (Teacher/Student)

.006 .051 .004 .907

Treatment Group (PM/No PM)

-.006 .040 -.003 .877

Table 6. Model One: GPA Academic N=628 Variables B Std. Error Beta Sig Academic GPA 2011-12

.775 .024 .818 .000

Free/Reduced Lunch

.030 .056 .011 .591

Language -.084 .043 -.045 .050 Ethnicity -.134 .091 -.034 .139 Ethnicity Match (Teacher/Student)

-.003 .058 -.002 .963

Treatment Group (PM/No PM)

-.068 .046 -.036 .138

69

Table 7. Model Two: Treatment Teachers vs. Non-Treatment Schools CSTs 2013 N=2365 Variables B Std. Error Beta Sig CST ELA 2012 .761 .013 .763 .000 Free/Reduced Lunch

-.684 1.622 -.005 .674

Language -.576 1.320 -.006 .663 Ethnicity .015 1.666 .000 .993 Ethnicity Match (Teacher/Student)

-2.982 1.090 -.038 .006

Treatment Group (PM/No PM)

11.387 9.379 .077 .225

Table 8. Model Two: CMA N=152 Variables B Std. Error Beta Sig CMA 2012 .509 .062 .544 .000 Free/Reduced Lunch

5.330 12.552 .027 .672

Language 7.575 9.256 .054 .415 Ethnicity -12.117 11.264 -.079 .284 Ethnicity Match (Teacher/Student)

2.543 5.188 .035 .625

Treatment Group (PM/No PM)

-56.141 23.250 -.181 .017

70

Table 10. Model Two: 7TH Grade Writing—Treatment Teachers vs. All Other Students in Non-Treatment Schools N=368 Variables B Std. Error Beta Sig CST ELA 2012 .005 .001 .367 .000 Free/Reduced Lunch

8.841E-5 .098 .000 .999

Language .092 .066 .072 .163 Ethnicity .089 .087 .057 .312 Ethnicity Match (Teacher/Student)

.035 .062 .030 .575

Treatment Group (PM/No PM)

-.243 .205 -.055 .238

Table 11. Model Two: CAHSEE N=625 Variables B Std. Error Beta Sig CST ELA 2012 .440 .016 .768 .000 Free/Reduced Lunch

-.294 1.948 -.004 .880

Language -.337 1.704 -.005 .843 Ethnicity -5.492 2.635 -.073 .038 Ethnicity Match (Teacher/Student)

2.270 2.178 .035 .298

Treatment Group (PM/No PM)

2.909 13.893 .038 .834

Table 12. Model Two: Overall GPA N=2741 Variables B Std. Error Beta Sig Overall GPA 2011-12

.863 .010 .908 .000

Free/Reduced Lunch

.053 .019 .025 .006

Language -.005 .017 -.003 .782 Ethnicity -.019 .021 -.010 .353 Ethnicity Match (Teacher/Student)

.002 .014 .001 .893

Treatment Group (PM/No PM)

.015 .088 .006 .869

71

Table 13. Model Two: Academic GPA N=2704 Variables B Std. Error Beta Sig Academic GPA 2011-12

.823 .010 .880 .000

Free/Reduced Lunch

.037 .021 .017 .073

Language -.009 .018 -.005 .619 Ethnicity -.013 .022 -.006 .570 Ethnicity Match (Teacher/Student)

-.002 .015 -.002 .884

Treatment Group (PM/No PM)

.056 .098 .021 .569

Table 14. Model Three: CSTs—Treatment Teachers vs. All Others in Treatment Schools N=1242 Variables B Std. Error Beta Sig CST ELA 2012 .700 .019 .716 .000 Free/Reduced Lunch

.161 2.450 .001 .948

Language -1.585 1.707 -.016 .353 Ethnicity .205 3.481 .001 .953 Ethnicity Match (Teacher/Student)

-.225 2.822 -.003 .937

Treatment Group (PM/No PM)

-3.157 5.551 -.028 .570

Table 15. Model Three: CMA’s 2013 N=52 Variables B Std. Error Beta Sig CMA 2012 -.124 .237 -.117 .604 Free/Reduced Lunch

11.738 29.565 .068 .694

Language -8.184 30.213 -.062 .788 Ethnicity -52.101 36.541 -.257 .164 Ethnicity Match (Teacher/Student)

29.796 22.760 .435 .200

Treatment Group (PM/No PM)

-16.810 100.452 -.088 .868

72

Table 16. Model Three: 7th Grade Writing N=103 Variables B Std. Error Beta Sig CST ELA 2012 .002 .002 .132 .287 Free/Reduced Lunch

-.371 .375 -.099 .326

Language -.025 .127 -.024 .843 Ethnicity .242 .311 .125 .439 Ethnicity Match (Teacher/Student)

.202 .318 .311 .527

Treatment Group (PM/No PM)

-.164 .359 -.085 .648

Table 17. Model Three: CAHSEE N=359 Variables B Std. Error Beta Sig CST ELA 2012 .442 .024 .778 .000 Free/Reduced Lunch

3.885 2.838 .041 .172

Language 1.814 2.067 .028 .381 Ethnicity -6.552 4.197 -.052 .119 Ethnicity Match (Teacher/Student)

6.623 3.133 .107 .035

Treatment Group (PM/No PM)

-3.108 6.096 -.048 .610

Table 18. Overall GPA N=1580 Variables B Std. Error Beta Sig Overall GPA 2011-12

.858 .015 .010 .661

Free/Reduced Lunch

.010 .033 .004 .771

Language -.049 .024 -.027 .044 Ethnicity -.054 .048 -.016 .264 Ethnicity Match (Teacher/Student)

.018 .038 .015 .642

Treatment Group (PM/No PM)

.109 .086 .052 .206

73

Table 19. Model Three: Academic GPA N=1554 Variables B Std. Error Beta Sig Academic GPA 2011-12

.804 .016 .798 .000

Free/Reduced Lunch

.004 .037 .002 .908

Language -.064 .027 -.035 .017 Ethnicity -.031 .053 -.009 .561 Ethnicity Match (Teacher/Student)

.001 .042 .001 .979

Treatment Group (PM/No PM)

.038 .094 .018 .689

74