BUCKET EVALUATION TRAINING

Empowering Effective TeachersHillsborough County Public Schools

Objectives and Purpose Participants will evaluate teacher buckets for…

Balance Consistency Accuracy & Fairness

Participants will… Discuss and organize information and evidence from a

teacher’s bucket to establish trends and increase the accuracy of final evaluation ratings.

Participants will… Demonstrate understanding of norms, protocols &

evaluation practices through a journal/reflection page to evaluate the efficacy of the training module.

Agenda Overview of Training & Materials

Discussion on Evaluating Buckets Activity - Evaluate Bucket 1 (Grade 1) Activity – Evaluate Bucket 2 (MS Social

Studies) Activity – Evaluate Bucket 3 (HS Science)

Journal and Last Minute Reminders

Training Packet Interactive copy of PowerPoint presentation Activity Buckets

Activity 1 – Grade 1 Review Bucket Rate Bucket Activity 2 – Middle School Social Studies Activity 3 – High School Science Evidence Collection Tool (optional) Rating Worksheet (required) Journal Page Survey form (to be completed at the conclusion

of training) Survey

Facts and Myths of Teacher Evaluation

Facts Evaluators examine the entire

bucket looking for trends when assigning final evaluation ratings.

Evidence can be taken from summary reports and pre-appraisal forms to further establish trends. Evaluators may examine scripting notes to look for trends and must read journal entries for clarification.

Teachers are not evaluated based on any observed growth (this is not a growth model). Certain components may follow the growth model (i.e. 2a, 2d).

Myths Individual component ratings

are averaged. Ratings are disregarded if the

observer is suspected to be a high/low rater.

Evaluators only look at their own observation data.

One data source is weighed heavier than another.

An equal number of Accomplished and Developing ratings automatically constitute an overall Developing rating.

Journal items are not taken into consideration during the final evaluation.

The Devil in the Details (Part 1)

Component Rating Type Date

(3b) Using Questioning and Discussion Techniques

A Formal (Principal)

10/12/2011

(3b) Using Questioning and Discussion Techniques

D Informal (Principal)

1/27/2012

(3b) Using Questioning and Discussion Techniques

A Formal (Peer) 12/13/2011

(3b) Using Questioning and Discussion Techniques

D Informal (Principal)

1/6/2012

(3b) Using Questioning and Discussion Techniques

A Informal (Peer) 3/24/2012

Examine the ratings below and assign a final evaluation score.

Click on the link below to enter your rating: https://www.surveymonkey.com/s/Session1DD1

Summary of Participant Responses

Responses will be collected and summarized in a chart which will be displayed below.

Ratings

Item 1Item 2Item 3Item 4

RESULTS

The Devil in the Details (Part 2)

Component Rating

Type Date

(3b) Using Questioning and Discussion Techniques

A Formal (Principal)

10/12/2011

(3b) Using Questioning and Discussion Techniques

D Informal (Principal)

1/27/2012

Teacher’s questions were low level questions asked in rapid succession. The teacher attempted to call different students but very few students were responding to the

teacher’s questions. Teacher called Susie and Jose quite a few times because they were starting to get off task.

(3b) Using Questioning and Discussion Techniques

A Formal (Peer) 12/13/2011

(3b) Using Questioning and Discussion Techniques

D Informal (Principal)

1/6/2012

Students did not get any chance to discuss with one another. All conversations were between the teacher and a student. Teacher used a system for calling names

inconsistently. The level of student participation was low. Most of the teacher’s questions were low level recall questions. Towards the end of the lesson John was the

only one responding to the teacher’s questions. (3b) Using Questioning and Discussion

TechniquesA Informal (Peer) 3/24/2012

The teacher asked a combination of high and low level questions. The teacher framed a couple questions to promote student thinking. Two students responded. The teacher

asked Mary to respond directly to Jack’s question. Mary, unsure of her answer, responded to the teacher. The teacher repeated this process a few more times with

differing results. The teacher called on different students who were raising their hands. The teacher directed questions to students who were not paying attention to get them

back on track.

Devil in the Details (Part 2) Peer Summary

3b. Using Questioning and Discussion Techniques - Discussions enable students to talk to one another, without ongoing mediation by the teacher. The teacher reminds students what is necessary to work the calculations and worked several equations on the board.

Principal Summary Teacher's questions are mostly

directed to individuals working in small groups. Teacher assists students and clears up misunderstandings. Most questions are low-level, but designed to seek thinking in Math and to create opportunities for expanded discussion and understanding. The lesson is largely based on questions and answers and subsequent discussions. The teacher randomly calls on students. All students participate and are actively engaged in the question and answer session.

Click on the link below to enter your rating: https://www.surveymonkey.com/s/Session1DD2

Summary of Participant Responses

Participant responses will be collected and summarized in a chart which will be displayed below. Ratings

Item 1Item 2Item 3Item 4

RESULTS

Pre-Evaluation Documents Compile all documents:

Formal scripting notes Teacher-provided or peer-collected artifacts

from formal observation cycle (tests, worksheets, seating charts, student work)

Summary Forms Informal scripting & comments Collected extenuating circumstances (email) Bucket PDF or printout Teachers’ Journal EntriesNote: Prior to assigning ratings, check for last-minute observer or journal entries.

Interpreting Outliers Outliers

An outlier is an observation that lies an abnormal distance from other values in a random sample from a population.

A single outlier (Exemplary or Requires Action) does not necessarily mean that either peer/administrator is rating high/low and therefore will bear less weight towards the overall evaluation rating. Individual outliers still deserve the same level of analysis as the rest of the ratings.

Requires Action ratings will most likely include comments or would be highlighted as an area of strength/focus in the observation summary report. Use them to determine trends in teacher practice based on specific evidence.

Example of an OutlierComponent Rating Type Date

(3a) Communicating with Students

D Formal (Peer) 9/12/2011

(3a) Communicating with Students

D Informal (Peer) 1/27/2012

(3a) Communicating with Students

A Formal (Principal) 12/13/2011

(3a) Communicating with Students

D Informal (Peer) 1/6/2012

(3a) Communicating with Students

E Informal (Principal) 3/24/2012

Where do you go?Component Rating Type Date

(3c) Engaging Students in Learning A Formal (Peer) 9/12/2011(3c) Engaging Students in Learning A Informal (Peer) 1/27/2012(3c) Engaging Students in Learning D Formal (Principal) 12/13/2011(3c) Engaging Students in Learning D Informal (Peer) 1/6/2012(3c) Engaging Students in Learning D Informal (Principal) 3/24/2012

1. The strength of an evidence depends upon the degree of impact towards overall student learning

2. Frequency – teachers may request as many observations as needed but the power relies upon the evidence. A few E’s, A’s, D’s, RA’s a few days apart does not necessarily show the teacher’s overall performance throughout the year.

3. Dates and interval between observations – be mindful of observations done early in the year and the start of a semester (high school).

4. Formal versus Informal or Peer versus Principal – evidence is evidence (reliability of evidence is strengthened by comments and summaries from other components).

Evaluation Protocols RA ratings require comments. If the bucket does not contain the

required number of completed observations, communicate this concern to Jessica Doherty.

Ratings on all observations are final. No changes will be made to previous observation ratings unless a genuine error was made, and the teacher was rated lower. Send a request to Jessica Doherty so that the teacher’s appraisal may be reactivated.

A 3-week window will be opened to teachers for requests to review specific components in their bucket using a specified form.

Peers will have until the start of the next school year to respond. Teachers will be notified whether or not there has been any changes in their bucket.

New peers evaluate first few buckets with buddy peer until they feel comfortable.

Activity 1 – 1st Grade1. Individually evaluate the 1ST

Grade Bucket.2. Highlight discussion points for

whole group discussion. You may use your worksheets to take notes and keep track of your ratings. You may click on the rubric and critical attributes icons above for reference.

3. Once finished, click on the link below to enter final ratings.Rate Bucket

4. With time to spare, discuss your ratings (including supporting trends/evidence) with your buddy peer.RESULTS

Activity 2 – MS Soc. Studies1. Individually evaluate the

Middle School Social Studies Bucket.

2. Highlight discussion points for whole group discussion. You may use your worksheets to take notes and keep track of your ratings. You may click on the rubric and critical attributes icons above for reference.

3. Once finished, click on the link below to enter final ratings.Rate Bucket

4. With time to spare, discuss your ratings (including supporting trends/evidence) with your buddy peer.

Activity 3 – HS Science1. Individually evaluate the

High School Science Bucket.2. Highlight discussion points for

whole group discussion. You may use your worksheets to take notes and keep track of your ratings. You may click on the rubric and critical attributes icons above for reference.

3. Once finished, click on the link below to enter final ratings.Rate Bucket

4. With time to spare, discuss your ratings (including supporting trends/evidence) with your buddy peer.

JournalReflect on the contents of this training and review your materials. What are some important ideas, best

practices and critical protocols that you found most helpful in this training?

What are you taking out of this training? How are you going to apply the knowledge and skill you learned towards improving your practice/role in teacher evaluation?

Final Thoughts and Survey Please click on the link: Training Evaluation Q&A Important Dates

May 24, 2012 – Final Run for Peer Buckets and Journals- (ongoing) Teachers review buckets for any missing

observations- (ongoing) Teachers acknowledge completed observationsMay 24, 2013 – All observations must be completed

June 10, 2013 – All peers hit “SUBMIT” to send evaluation resultsJune 11-28, 2013 – Bucket Review Period

Evaluation Video by Dr. David Steele.

THANK YOU FOR YOUR ACTIVE PARTICIPATION!

Bucket Training Team 2013