professional development for evaluators wednesday, july 31 leveraging performance management to...

PROFESSIONAL DEVELOPMENT FOR EVALUATORS

Wednesday, July 31

Leveraging Performance Management to Support School

Priorities

Getting Started: Revising Goals

Please consider the goals – are there any revisions you would suggest?

Characteristics of high-quality goals:SMARTAligned to school prioritiesHigh leverageIncludes mid-year benchmark

Today’s Agenda

Collecting Evidence A Closer Look at Observations

A Closer Look at Artifacts

Lunch

EvaluationsBreakouts

Planning for the work at your schoolManaging Directed Growth & Improvement plans

Closing

What progress have we made toward the goals of the training?

Goal 1:Evaluators will know how to implement the new system (technical 5-step implementation) and how to get it done and done well (evaluation best practices).

What progress have we made toward the goals of the training?

Goal 2:Evaluators will leave with concrete, specific plans for implementation in their school, including how to:Use the evaluation system to further their school prioritiesTalk with teachers about the evaluation system

What progress have we made toward the goals of the training?

Goal 3:Evaluators will leave knowing the specific responsibilities of evaluators and teachers under the new evaluation system.

Evidence:Observation & Artifacts

Evidence is collected throughout the cycle

Every educator is an active participant in an evaluation

Process promotes collaboration and continuous learning

Continuous Learning

All Evidence is entered in EDFS

Both educators and evaluators can upload artifacts

Evaluators upload both observation evidence & feedback

Evidence should be collected on …

Progress on Ratings on OVERALL (2) Goals (4) Standards RATING

Evidence Collection

Evidence should be explicitly related to specific standards & goals.

When appropriate, evaluator provides feedback to educator using language from the rubric.

Evaluator does not rate educator practice after each piece of evidence, but during the formative or summative evaluation.

11

Evidence vs. Feedback

Evidence Feedback

• Facts• Aligned to Rubric

• Ex 1: Lesson plan submitted as artifact

• Ex 2: Ms. Y said, "Yes, I know. You are working hard. You can do this.”

• Claim & Interpretation• Aligned to Rubric

• Ex 1: The lesson plan does not include a closing activity. How is summarizing incorporated into the lesson?

• Ex 2: Ms. Y's response show that she continues to foster a safe intellectual environment where students take academic risks.  

Evidence from Observations

Observations

Turn & Talk:

What have your experiences with observation been like?

What the Research Says . . .

Evaluators need multiple opportunities and settings to observe and assess educator practice

Multiple observations paired with timely feedback are a key part of a strong evaluation system

15

Observing Lessons … CAUTION! …

We observe all the time, which means that it is easy to do, but hard to do objectively.

Remember

• We tend to see what we want to see

• We have to be aware of and avoid bias, particularly personal idiosyncrasies

• We have to avoid preconceptions

• Observation should be based on agreed upon criteria

Observation Bias

What is bias? What are some experiences you have had with bias?

What are some examples of bias you have about teaching?

What can we do to diminish the influence of bias in our work?

Principles of Brief Observations

Frequent

Focused

Varied

Useful and Timely Feedback

Strategies for Collecting Evidence

Identify a focus ahead of time Goals, specific Elements

Record evidence, not judgment Quotations, observed actions or movements by

teacher and students, times, numbers, literal descriptors, etc…

Be an efficient note-taker Establish abbreviations, paraphrase

When possible, meet with the teacher Check in briefly to ask questions/give feedback

19

Examples

Observations & Tools from Sara

Observations & Tools from Mary

Other forms of Observations: Team Feedback

Sara’s tracking tool

CHS approach to scheduling

Systems for Tracking Observations

Observations in the EDFS

BPS developed an observation tool in EDFS

Evidence and feedback should be entered in EDFS, and tagged to an appropriate area of the rubric

Observations may only be used in formative or summative evaluations if the educator received feedback within 5 days of the observation

Observations: EVALUATOR View

Observations: EVALUATOR View

Observations: EVALUATOR View

Next version of the observation tool

Next version of the observation tool

Observations: EVALUATOR View

Observations: Educator &Evaluator View

Observing Practice

Chris McCloud7th Grade Math Teacher

Strengths Needs

•Improving student performance in math for students who enter my class performing below grade levelIndicators/Elements:

•Teaching classrooms with diverse needs (especially meeting the needs of student with an IEP) Indicators/Elements:

•Family outreach and communication Indicators/Elements:

•Additional support for implementing the revised MA Curriculum framework Indicators/Elements:

•Improving communication with families for whom English is a second language Indicators/Elements:

•Strengthen leadership skills Indicators/Elements:

C. McCloud’s self-assessment against the rubric

34

Chris McCloud’s Goals:

SL: Based on the fact that my 7th grade ELL students averaged 30% on my pre-assessment, my goal is that they will average at least 80% on each of the six unit assessments that they will take before May 15.

PP: In order to support my ELL students in averaging 80% on unit assessments, I will consistently identify and teach symbols, key terms and other math vocabulary, and use daily exit tickets that measure both vocabulary and conceptual understanding. I will measure my progress through student vocabulary notebooks and tracking exit ticket performance data.  

Observation of Chris McCloud: Selective Scripting

1. Read over the two excerpted elements from the rubric (I-A-4. Well-Structured Lessons, II-A-2. Student Engagement)

2. Watch the video.3. While you are watching the video, collect

evidence that will allow you to rate Mr. McCloud on the two elements. Use the worksheet to selectively script evidence related to each of the elements.

4. When you are finished, complete the worksheet and rate Mr. McCloud.

How did it go?

Value of Feedback

Targeted, Ongoing Feedback

Improved Educator Practice

Improved Student Achievement

Characteristics of Effective Feedback

FOCUSED: feedback should focus on what was observed

EVIDENCE-BASED: feedback should be grounded in evidence of practice

ACTIONABLE: feedback should reinforce effective practice and identify areas for continued growth

TIMELY: feedback should be provided shortly after the observation

Constructing Feedback

Chris McCloud:

Using your observation notes and the rubric: Draft two points of feedback for ChrisFocus on feedback that is

FocusedEvidence-based, and Actionable

Feedback conversations

What has been your experience with feedback conversations?

Volunteers for a roleplay?

Feedback conversations: Resources

Beyond the Scoreboard, from Educational Leadership

Talk About Teaching, by Charlotte Danielson

Leverage Leadership, by Paul Bambrick-Santoyo

Difficult Conversations, by Stone, Patton & Heen

Reflect

What are the implications of what we just discussed for your practice? For your school

community?

Evidence from Artifacts

Collecting Evidence: What can you see?

In the classroom Outside of the classroom

In artifacts/conversations

• Well-structured lessons

• Student engagement

• Classroom management

• Differentiated instruction

• A safe learning environment

• Professional collaboration

• Family and community engagement

• Reflection

• Results of data analysis

• Adjustments to practice

Collecting and Assessing Evidence - Artifacts

Artifacts may include: Tracking & analysis of student assessment data Student work Lesson plans

Identify & chart possible sources of evidence in the standard you are assigned. What kind of documents would be useful for teachers upload?

Artifacts: Clarifying Expectations

How many would you like people to submit?

What kind of documents?

How detailed of a rational would be useful?

Artifacts Guidance

A suggested format for rationales:

Step 1: Identify the standard, indicator, element, or goal that the artifact addresses.

Step 2: Describe the artifact and identify the section that directly connects to the standard, indicator, element, or goal.

Step 3: Highlight the artifact’s impact on student learning.

Step 4: Specify the evidence of professional growth or proficiency the artifact provides.

Artifacts Guidance

Look over the examples on p. 4-6What is strong? Anything more you would add?

Zil’s SLG and Action Steps

Zils SLG Artifact

Mid Year Check Point

Artifacts: Examples

Looking at these artifacts, what evidence do you have of the teacher’s practice?

Trio Activity

Thinking about a specific team or educator, think about which artifacts you would suggest they upload.

Consider what kinds of artifacts would show…-elements that cannot be observed?-evidence of growth in areas noted for improvement?-progress on goals?-performance on each standard?

Artifacts on EDFS: Educator & Evaluator View

Artifacts: Educator & Evaluator View

Artifacts: Educator & Evaluator View

Artifacts: Educator & Evaluator View

Artifacts: Educator & Evaluator View

Artifacts: Educator & Evaluator View

Artifact Tips

Start an electronic folder for artifacts on your desktop

Keep file names short (fewer than 32 characters)

Upload videos somewhere else, and include a link

Genius Scan – app for turning pictures into PDFs

Artifact Tips Cont’d

All artifacts have to be uploaded to EDFS – even if the hard copy is not scanned, it has to be entered as evidence and a hard copy kept in a binder.

You can comment on artifacts through EDFS, but the online system is not a substitute for face-to-face conversation.

Reflect

What are the implications of what we just discussed for your practice? For your school

community?

Evaluations

Evaluations

The basicsEvaluations on EDFSExamples

What is a formative assessment?

A mid-plan check-in rating on progress towards each goal rating on each standard

A document that considers all evidence observations to date artifacts to date

It may be used to change a plan If there is a significant change in practice,

however, this is not mandatory. The plan may continue until the summative.

What is a summative evaluation?

A summary at the end of the plan rating on progress towards each goal rating on each standard

A document that considers all evidence the formative assessment observations & artifacts since the formative consistency or change in practice since the

formativeThe overall rating determines the next

plan

Not the “evaluation” of the past

Evaluations are NOT write-ups of individual, formal observations.

Formative Assessment & Summative Evaluation

Meetings Upon request of educator or evaluator Required for Summative if the overall

rating is NI or Unsatisfactory Recommended for all educators

Evaluations in EDFS

Formative Assessment

Step 1

Step 2

Evaluations in EDFS

Evaluations in EDFS: Rating Goals

Evaluations in EDFS: Rating Standards

Evaluations in EDFS: Overall Rating

Evaluations in EDFS: Prescriptions

Prescriptions for any standard in which an educator is rated NI or UNSAT.

Evaluations in EDFS: Comments

Comments are optional for both the evaluator & the educator

Evaluations in EDFS: Release

Evaluations in EDFS: Sign-Off

Evaluation Examples

Read one of the examples you have.

What do you notice about how evidence is used in the write-up?

Rationales for Ratings

Standard I. Curriculum, Planning, & Assessment 

Rating 

Exemplary, Proficient, Needs Improvement, Unsatisfactory

RationaleClaim/Connection to rubric: Evidence 1:

Evidence 2:

Evidence 3:

Rationale: What claim can you make? What is the evidence to support it (artifacts, observations)?

Reflect

What are the implications of what we just discussed for your practice? For your school

community?

Breakouts

Group 1: Planning for evaluation work at your

school

Group 2: Working with teachers on improvement or

directed growth plans

Planning the work at your school

Mary’s document

Planning the work at your school

Guiding Questions:What has your school done to support staff with the evaluation process? What additional supports do you think would be helpful?What are ways of structuring this support at the school? What timelines should guide the work?How can you and your team find the time to do this well?

Working with teachers rated NI or U

You can change the plan at the time of the formative assessment (though a formative assessment does not HAVE to change the plan).

Timelines and Requirements

Directed Growth and Improvement plans

Considerations:•Choosing length•Using a calendar to calculate dates of formative assessment and summative evaluation•Writing goals from prescriptions•Developing the Action Plan, including supports and timelines•Collection of evidenceSupports: “Did I” sheets and timelines

http://educatoreffectiveness.weebly.com

• For more information, visit: EDFS: http://eval.mybps.org/ http://educatoreffectiveness.weebly.com

• Email questions, comments and feedback to: bpsevaluation@boston.k12.ma.us

• MA Department of Elementary and Secondary Education (DESE) Evaluation Site:

http://www.doe.mass.edu/edeval/

Office of Educator Effectiveness

• Ross Wilson, Assistant Superintendent for Educator Effectiveness

• Chason Ishino, Implementation Specialist

• Jared Joiner, Implementation Specialist

• Emily Kalejs Qazilbash, Implementation Specialist

• Angela Rubenstein, Implementation Specialist

• Kris Taylor, Implementation Specialist

• Jenna Costin, EDFS On-line System Coordinator

• Jen Kozin, Data Analyst

Evaluator Training Facilitators: Mary Driscoll, Edison K-8Sara Zrike, Hurley K-8

Contact us if you have questions

Ross Wilson, Asst. Superintendent for Educator Effectiveness

Network Implementation Specialists BPS email

A Emily Qazilbash eqazilbash

B Nicole Ireland nireland

C TBD eval@mybps.org

D Angela Rubenstein arubenstein

E Jared Joiner jjoiner

F Kris Taylor ktaylor

High Schools

Chason Ishino cishino

EDFS tech support

Jenna Costin jcostin

Data analysis

Jen Kozin jkozin

What progress have we made toward the goals of the training?

Goal 1:Evaluators will know how to implement the new system (technical 5-step implementation) and how to get it done and done well (evaluation best practices).

What progress have we made toward the goals of the training?

Goal 2:Evaluators will leave with concrete, specific plans for implementation in their school, including how to:Use the evaluation system to further their school prioritiesTalk with teachers about the evaluation system

What progress have we made toward the goals of the training?

Goal 3:Evaluators will leave knowing the specific responsibilities of evaluators and teachers under the new evaluation system.

Closing

Please complete the exit ticket.

Thank you so much for participating, and please be in touch!