csu center for teacher quality assessing teacher preparation outcomes for program improvement and...

CSU Center for Teacher Quality

Assessing Teacher Preparation Outcomes forProgram Improvement and Institutional Accountability

CSU Academic Council MeetingBurlingame, California

November 30, 2006

Teacher EducationProgram

Outcomes

Outcome Two:Effects of a

Program on Its Graduates'

Teaching as Reported byIts TeachingGraduates

Outcome One: Intrinsic Qualities of a Program as Reported by Its

Graduates When They Complete

the Program

Outcome Three:Effects of a Program onIts Graduates' Teaching

as Reported by TheirK-12 Supervisors

Outcome Four: Effects of a Program on

Its Graduates' Teaching asMeasured by

Valid, Reliable Assessments of Performance

Outcome Five: Participation

and Persistence in the Profession of Teaching by a

Program's Graduates

Outcome Six:K-12 Pupil Learning Outcomes that Can

Validly be Traced to Teacher Education

1 2

3

45

6

Question A: Should other important outcomes of teacher education programs be added to the CSU Mosaic?

Question B: Should the outcomes in the Mosaic be modified to make it more valuable for other institutions?

Assessing and Improving Multiple Significant Outcomes of Teacher Preparation Longitudinally.

Center for Teacher Quality The California State University System

The CSU Mosaic:

1


Outcomes



the Program

1

The CSU Mosaic:

2

Evaluating CSU’s Teacher Preparation Programs:The Exit Survey

Key features: Campus administrators have real-time, online access to campus data Campus administrators can add own questions to the base survey 6,000+ graduates from 22 campuses participated in 2006

Evaluation Question Answered by Graduates Exiting All Credential Programs in the CSU

CSU System:All Programs

What is your overall evaluation of your Teaching Credential Program?

I learned a lot in my CSU credential program. 62.5%

I learned quite a bit that was important. 31.2%

The CSU program included relatively little substance. 5.3%

The CSU professional prep. program offered nothing of value. 1.1%

3


Outcomes






the Program

1 2

The CSU Mosaic:

4

Composite E-2: CSU Teachers Assess the Overall Value and Quality of Fieldwork in Credential Programs (K-12)

6%

6%

56%

64%

58%

68%

59%

64% 23%

22%

20%

23%

23%

24% 14%

9%

12%

9%

13%

11%

6%

4%

3%

3%

0% 25% 50% 75% 100%

System 01-02

Campus 01-02

System 02-03

Campus 02-03

System 03-04

Campus 03-04

Whe

re a

nd W

hen

Teac

hers

Wer

e Pr

epar

ed

Percentages of Teaching Graduates Whose Preparation Was Assessed One Year After CSU Completion

Well Prepared Adequately Prepared Somewhat Prepared Not-at-All Prepared

87 %

81 %

88 %

81 %

87 %

80 %

5


Outcomes






the Program



1 2

3

The CSU Mosaic:

6

18%15% 15% 16% 15% 14%

44%39% 40% 38% 38% 36%

36%44% 44% 45% 45% 48%

2%2%2%2%2%3%

1990-00 N = 798

2000-01 N = 1,303

2001-02 N = 1,489

2002-03 N = 1,279

2003-04 N = 1,452

2004-05 N = 1,252

CSUTeachersWere WellPrepared

CSUTeachersWereAdequatelyPreparedSomewhatPrepared

Not-at-AllPrepared

Total Blue = 80 % 83 % 84 % 83 % 83 %

84 %

100%

25%

50%

75%

CSU Effectiveness in Preparing K-8 Teachers to Teach Reading-Language Arts: Assessments by the Employment Supervisors of

CSU First Year Teachers in Six Cohorts

7


Outcomes






the Program






1 2

3

4

The CSU Mosaic:

8

Teacher Candidate Performance Assessments in California

Soon, the State will Require Each Institution to Assess Each Candidate’s Performance Using Option One or Two:

►

► Based on the Same State-Adopted Teaching Performance Expectations as TPA ► Developed by Stanford University in a Consortium of 12 Universities ► Each Candidate’s Performance Tasks will Resemble the TPA Pedagogical Tasks ► Candidate Responses will be Assessed by Institutional Faculty and Guest Assessors

► Institutional Faculty are Being Trained to Score Performances Validly and Reliably

(1) Option One is Called Teaching Performance Assessment (TPA)

► Based on State-Adopted Teaching Performance Expectations ► Developed by Educational Testing Service under a State Contract ► Each Candidate will Perform in Relation to Four Complex Pedagogical Tasks ► Candidate Responses will be Assessed by Institutional Faculty and Guest Assessors ► Institutional Faculty are Being Trained to Score Performances Validly and Reliably ► Candidate Responses will be Assessed by Institutional Faculty and Guest Assessors ► Institutional Faculty are Being Trained to Score Performances Validly and Reliably

(2) Option Two is Performance Assessment for California Teaching (PACT)

9


Outcomes






the Program








Program's Graduates

1 2

3

45

The CSU Mosaic:

10

CSU Evaluations of Graduates’ Retention in Teaching

CSU has Two Evaluations in Progress --

(1) A Large-Scale Evaluation of Teachers’ Reasons for Staying or Leaving

► Based on a Large-Scale Survey of Teachers Who Stayed and Who Left Teaching ► Focus of the Evaluation was on Teachers’ Reasons for Staying or Leaving ► Working Conditions in K-12 Schools Were the Primary Reasons Cited by Teachers ► Teacher Preparation was a Less-Important Factor in Decisions to Stay or Leave

(2) An Analysis of State Data on CSU Teacher Employment Patterns

► A CSU Proposal is Pending Before the California Department of Employment. ► This Analysis will Include All of CSU’s Teaching Graduates Since 1999. ► Analysis will Focus on Patterns of Retention, Transfer and Attrition from Teaching. ► CSU is Waiting for Data Files from the California Department of Employment.

11

What role did teacher education play in teachers’ decisions to leave?

Survey Question:How much did each of the factors below affect your decision to leave teaching?

This factor affected teachers’ decisions

“A Lot” or “Somewhat”

My credential program coursework did not prepare me to be successful in this school. 13.3%

My student teaching experiences did not prepare me to be successful in this school. 12.2%

Finding: Teacher preparation ranked 33rd and 34th among 35 factors

A Study of Teacher Retention in California

Survey Question: How much did each of the factors below affect your decision to remain in the classroom?

This factor affected teachers’ decisions

“A Lot” or “Somewhat”

My credential program coursework prepared me to be successful in this school. 57.7%

What role did teacher education play in teachers’ decisions to stay?

Finding: Teacher preparation ranked 9th among 35 factors

12


Outcomes






the Program








Program's Graduates



1 2

3

45

6

The CSU Mosaic:

13

600

605

610

615

620

625

CSU-Prepared New TeachersNew Teachers Prepared Outside CSU

Teaching English Learners Effectively: Effectiveness of CSU Prepared Teachers and Other Teachers in Terms of Academic Learning by Their Students Who are English Learners in California Schools

600

605

610

615

620

625

600

605

610

615

620

625Teaching Language Skills in Grade 5 Teaching Literacy in Grade 5 Teaching Math in Grade 5

Start of Grade 5

Similarities Between the Two Groups of Teachers ► The two groups of teachers had from one to four years of teaching experience. ► The two groups taught in the same urban schools in a large California district. ► This analysis focuses on English Learners taught by the new CSU Teachers (N=1,297) or by the other Teachers (N=9,846).

Preliminary Findings ► Posttest score differences between two groups of English learners were statistically significant (.05), but the groups began the fifth grade at equivalent levels of proficiency. ► In language, literacy and math instruction for ESL students in grade 5, new teachers from CSU were prepared to be more effective than a matched group of teachers from outside the CSU.

Start of Grade 5

Start of Grade 5

End of Grade 5

End of Grade 5

End of Grade 5

14

613

618

623

628

633

CSU-Prepared New TeachersNew Teachers Prepared Outside CSU

Teaching Low-Income Students Effectively: Effectiveness of CSU Prepared Teachers and Other Teachers in Terms of Academic Learning by Their Students Who are Low Income in California Schools

613

618

623

628

633

613

618

623

628

633

Teaching Language Skills in Grade 5 Teaching Literacy in Grade 5 Teaching Math in Grade 5

Similarities Between the Two Groups of Teachers ► The two groups of teachers had from one to four years of teaching experience. ► The two groups taught in the same urban schools in a large California district. ► This analysis focuses on students from low-income families taught by the new CSU Teachers (N=1,297) or by the other Teachers (N=9,846).

Preliminary Findings ► Two groups of low-income students were significantly different in pretest scores (.05) and posttest scores (.05). Low-income children taught by new CSU teachers were less proficient when the year began. ► The increased effectiveness of new CSU teachers enabled a low-performing group of low-income students to become more proficient than their counterparts at the end of the instructional year.

End of Grade 5

End of Grade 5

End of Grade 5

Start of Grade 5

Start of Grade 5

Start of Grade 5

15


Outcomes






the Program








Program's Graduates



1 2

3

45

6

Question A: Should other important outcomes of teacher education programs be added to the CSU Mosaic?

Question B: Should the outcomes in the Mosaic be modified to make it more valuable for other institutions?

Assessing and Improving Multiple Significant Outcomes of Teacher Preparation Longitudinally.

Center for Teacher Quality The California State University System

The CSU Mosaic:

16

csu center for teacher quality assessing teacher preparation outcomes for program improvement and...

Documents