DEVELOPING DEEPER AND MORE MEANINGFUL

EVALUATION SYSTEMS

Lisa Hood, Center for the Study of Education Policy

Funded by a grant through the McCormick Foundation

Teacher Evaluation in the Early Grades: Developing a PreK-3

Support Services for Danielson Framework for Teaching

Danielson Early Supports Project Staff

Lisa Hood, lhood@ilstu.edu Deb Kasperski, dkasper@ilstu.edu

Erika Hunt, elhunt@ilstu.eduPam Rosa, pam.rosa@cecillinois.org

Websitehttp://leadershiplinc.illinoisstate.edu/

District Level – Teacher Evaluation in the Early

Grades Danielson Framework for Teaching has

been validated for Grades 4 and above, yet many districts were planning to use the Danielson FfT for early childhood teachers (PreK-grade 3)

Is this a valid and reliable tool for PreK-3 teachers?

What tiered assessments are appropriate for PreK-3 classrooms?

Developing Early Learning Supports for Danielson

Framework for Teaching (FfT) Fall 2012:Review and adaptation by PreK-3

administrators, teachers, and stakeholders Critical Attributes and Examples

Winter 2012: NBCT’s developed examples and coded PreK-3 videos using adapted Framework

Winter 2013: Training developed for pilot principal and teachers

March 15, 2013: Feedback from early childhood stakeholders

April 26, 2013: Second round of feedback from early childhood stakeholders

June 2013: National feedback of framework supports solicited at NAEYC conference, San Francisco

Fall 2013: Pilot of framework supports and training

Piloting the 2013 FfT with Early Learning

Supports Pilot will occur in seven selected schools

throughout the region (Northern, Central, and Southern Illinois) in fall 2013.

Prior to pilot, selected principals and teachers will undergo a three day pilot training that will utilize three inter-connected modules.

Module 1: Understanding and using the 2013 Danielson FfT in the context of early

learning (PreK-3rd grade)

Present the need for piloting the 2013 FfT Early Learning supports to enhance early learning teaching practice growth opportunities

Identify the structure/priorities of the 2013 FfT Domains/Components and Critical Attributes

Understand shifts in professional practice for different levels of performance and the link to the 2013 FfT Framework Components/Critical Attributes

Explore and connect to Early Learning Grade Band examples at four different levels of practice

Module 2: Applying the 2013 Danielson FfT instrument through observation-based

evidence collection in early learning teaching settings

Understand observation-based data collection within the Plan-Teach-Reflect-Apply (PTRA) Professional Improvement Cycle for supporting and improving teaching practice

Develop/enhance observation process skills for collecting, sorting, and scoring evidence focused upon early learning Classroom Environment and Instruction

Determine Master Scores for early learning Classroom Environment and Instruction teaching practices using 2013 FfT Components and Critical Attributes

Module 3: Applying the 2013 Danielson Framework for Teaching instrument

through conversation-based evidence collection in early learning teaching

settings Understand data collection and artifacts of practice and

student learning within Professional Improvement Cycle Enhance skills focused upon early learning Planning and

Preparation practices, review of Classroom Environment and Instruction, and Responsibilities

Determine Master Ratings for Planning/Preparation and Responsibilities using FfT Components and Critical Attributes and protocols to confirm inter-rater reliability

Extend knowledge of professional learning-focused coaching language, conversations, and protocols of practice to support planning and reflection conversations

New ProposalsNew Proposals

February 2013: Submitted a proposal to validate the 2013 Danielson Framework for Teaching for PreK-3rd grade teachers

April 2013: Submitting a proposal to validate the training and professional development modules for principals and teachers associated with the early learning supports for the 2013 Danielson FfT.

7/25/11

Purpose of Proposed Validation Study

To ensure that FfT provides accurate and reliable data that PreK-3rd grade teachers and their supervisors can use to identify strengths and weaknesses

To help teachers and supervisors use the data from the FfT to identify appropriate professional development mechanisms to promote teachers’ growth.

To utilize data from the early learning Framework’s observation along with other with student growth data so that teachers who are observed to be “Unsatisfactory, Needs Improvement, Proficient, or Excellent” have appropriate student growth that correlates with the observation ratings

Other Benefits of Proposed

Validation Study Construct a student/parent survey Explore other student growth measures Increase principals’ understanding of

different teaching pedagogies related to early childhood education

Purpose of Proposed Professional

Development Project Develop and validate professional development

training and supplemental materials to support Danielson FfT for PreK-3rd grade teachers.

To use the data from validation study to identify appropriate professional development mechanisms to promote teachers’ growth and principals’ understanding of early childhood education (PreK-grade 3).

Training geared to principals, other persons evaluating teachers, the PreK-3rd grade teachers

Group Activity

Question One: What do evaluators of PreK-grade 3 teachers need to know about early learning (PreK-grade 3) classrooms?

Question Two: What do principals need to know in general about doing effective evaluations that will grow teacher’s practice?

Building Evaluation Capacity Within P-12 Principal Programs

and Partners

Approval of new Principal Endorsement Programs – a

beginning not an end Annual reports required by Illinois Educator

Licensure Board program data (e.g., number of faculty, number

of graduates, etc.) formative data that is internal to the programs

and varies by institution 2011 - Recommendations by P-20 Council

Committee on Teacher and Leadership Effectiveness for Principal Preparation Performance Indicators\ Tied to new P-20 Data Longitudinal System

Principal Preparation Program Performance Indicators

Principal Performance Indicators Recommended by P-20 Council Teacher and Leadership Effectiveness Committee (2011)

1. Percentage of graduates that pass the required state principal endorsement certification exam Note: This is a competency based assessment that will be administered to all candidates

2. Percentage of program graduates that obtain principal or assistant principal positions within 1, 3 and 5 years of graduation

3. Performance evaluations of program graduates using the four category rating system once placed as principals and assistant principals in IllinoisNote: This data will be available after implementation of the new statewide requirements in the 2012-13 school year.

4. Data and indicators of student growth within 1, 3 and 5 years of placement as an assistant principal or principalNote: According to Illinois requirements, this measure will be incorporated into performance evaluation ratings of Illinois principals and assistant principals.

5. Results of learning climate surveys in schools where principals and assistant principals are placed within 1, 3 and 5 years of graduationNote: A state standardized learning climate survey will be developed

6. Percentage of program graduates retained as principals, assistant principals, superintendents or district leadership roles within 1, 3 and 5 years of placement

Principal Preparation Program Continuous Improvement

Process Project Project Staff

Lisa Hood, Project Director Erika Hunt, Project Staff Alicia Haller, Evaluation Coach Joe Pacha, Evaluation Coach Brad White, Illinois Education Research Council, Team Consultant

Project Partners: University of Illinois at Urbana-Champaign Southern Illinois University at Edwardsville North Central College Loyola University New Leaders for New Schools

Logic Model: Illinois Principal Preparation Program Continuous Improvement and Review Process

Inputs:(What to invest)

FacultyTimeMoney ResearchMaterialsTechnologyPartners

Activities: (What we do)

Review Principal Preparation Programs (Quality Program Assessments and Feedback.

Support university personnel in their continuous improvement processes (Process Improvements)

Help faculty implement best practices for improving programs (Knowledge Building)

Participation:(Who we reach)

Principal Preparation Program Faculty members and ISBE

University Partners, Faculty, and ISBE

Faculty Members and ISBE

Outputs: Outcomes-Impact

Short Term:(What the short term results are)

Improved candidate selection

Improved courses and programs

Improved internship

Improved graduates of programs

Medium Term:(What the medium term results are)

Candidates prepared through quality coursework

Candidates ready to perform internship assessments successfully

Candidates prepared to lead schools in improving learning for all students

Long Term:(What the ultimate impacts are)

Improved learning for ALL students

Improved teacher practices

Improved school leadership

20

Evaluation: 1. Focus – 2. Collect Data – 3. Analyze and Interpret – 4. Report – Repeat the Cycle

Focus1. Program Requirements2. Internship Requirements3. Internship Assessments4. Coursework Requirements5. Staffing Requirements6. Candidate Selection

Collect Data1.Data Collected for Program Improvement2.Analysis of the Program Improvement Data3.Program Changes Based on Data Analysis 4.Summative INTERNSHIP Assessment Data: (Number of candidates; listing of each and all assessments with the scoring of each candidate on the assessment and number of times each candidate needed to complete the assessment successfully; length of the internship for each candidate)5.Listing of Courses and Course Syllabi6.Number of candidates and Faculty (FTE)7.Number of Candidate Applications8.Number of Candidates Selected9.Number of Candidate Completers10.Graduate’s School Data Analysis over time

Logic Model: Illinois Principal Preparation Program Continuous Improvement and Review Process

Analyze the results1.Analysis and planning for program improvement based on data 1, 2, &3.2.Analysis and implementation for improvement for Internship and candidate success based on data 4, 5, 6, 7, 8 and 9.3.Analysis and planning for improvements based on data 10.

Report 1.Program Improvement Plan2.Internship Improvement Plan3.Candidate Selection Improvement Plan4.Coursework and Staffing Improvement Plan

Implementation of a Program Theory Evaluation (PTE) model to identify how policies, practices, and processes have changed as a result of the process of redesigning;

Creation of an evaluation plan of Principal Preparation Programs; Alignment of current data and creation of needed data sources to

continuously inform evaluation plan process; Coaching, technical assistance, and tools to inform the evaluation

process Collaboration with the ISBE to provide input to the annual reporting

process and requirements for Principal Preparation Programs.

Project Objectives

Project Activities

Programs will receive technical assistance through evaluation coaches to help build internal capacity

Programs will be provided with: Tools to inform the evaluation process (e.g., Self-

Assessment Rubric, Sample Data Instruments) Technical assistance (e.g., Coaching and Program

Team Meetings) Meeting facilitation Timeline planning Data analysis Reporting framework

Two Project Phases

Phase One: Initial Assessment Phase

Phase Two: Continuous Improvement Phase

Phase One: Initial Assessment Phase

Beginning March 2013, evaluation coaches will convene program faculty and partners.

Step One: Completion of a self-assessment process developed by project staff

Step Two: Use self-assessment instrument to capture baseline information on: partnerships with districts and other community

members content and pedagogy assessments internship

Step Three: Share results with program team and develop action plan

Evaluation coaches and program faculty/partners will begin testing the program’s theory of change and whether it is having the intended effects on principal candidates and new principals. Assessment of the program’s “continuous

improvement evaluation plan” will be conducted to determine appropriate level of outcomes

Center staff and evaluation coaches, in partnership with program faculty and partners, will document the process and any on-going program modifications made.

Phase Two: Continuous Improvement Phase

Next Steps

Interest with funding a multi-phase statewide evaluation of principal preparation programs

Proposed development of a framework for a statewide evaluation of principal preparation Advisory committee to assist with development

of Evaluation Framework Statewide forum to collect input into draft

conceptual framework Development of an Request for Proposal (RFP) to

select an external evaluator for the statewide evaluation

Group Activity

Question One: What are the highest priority issues to keep in mind when developing a statewide evaluation framework for principal preparation?

Question Two: What do you foresee will be the benefits and challenges to doing a statewide evaluation of principal preparation programs?

Questions?

Thank you!

Please remember to complete an evaluation form before you leave. Evaluation form includes questions on

challenges and successes of the new P-12 Principal Endorsement programs.