Measuring Student Growth and its Role in the Evaluation Process

Keep the Goal in Mind

• Continuously improving the quality of instruction so students are learning at high levels

• Recall our Hattie work:– Want to get information to Long Term Memory (LTM)– Automaticity with information is critical to get

information to LTM– To achieve automaticity requires collective effort – There are characteristics of “expert

teachers”(experts in the area of instruction)

Recall: Characteristics of Expert Teachers

• High level of pedagogical knowledge

• Higher ability to improvise and adjust instruction based on the situation

• Deep understanding of why students are successful or are unable to perform a task

• More apt to provide appropriate learning strategies that engage, challenge intrigue without overwhelming Hattie, 2009

Recall Characteristics of Expert Teachers (cont.)

• More apt to anticipate and plan for the challenges students will encounter when learning new tasks

• Higher ability to improvise when things are not going as expected

• Accurate in hypothesis generation about what is causing students success or failure

• Distinct passion Hattie, 2009

How to Accomplish the Goal

• Measure student growth in multiple ways and aggregate data

• Use multiple points of data measured over time• Include teacher behaviors outside of the classroom• Ensure teacher evaluation provides an accurate

representation of teacher abilities• Use the evaluation system data to enhance

teacher’s ability to teach • Support teachers (ensure there is alignment at all

levels – align principal and central office evaluation systems

Marzano & Toth (2013)

Arguments for and Against Measuring Student Growth

• Student growth is an explicit part of Race to the Top legislation

• Literature cited in support of measuring student growth within the context of teacher evaluation: (Winters, 2012; Chetty, Friedman, and Rockoff, 2011) – Supports combining with other evaluation methods

and good policy

• Common term used: Value Added Measure (VAM) – How much has a student learned since a specific designated time?

Student Growth and Teacher Effectiveness (cont.)

• This concept of using student growth to determine teacher effectiveness is not a new concept

• Having it included as a mandatory component of the teacher evaluation system is new

The Impact of Teacher Effectiveness(Marzano)

Average School/Average Teacher

50th 50th

Highly Ineffective School/ Highly Ineffective Teacher

50th 3rd

Highly Effective School/Highly Ineffective Teacher

50th 37th

Highly Ineffective School/ Highly Effective Teacher

50th 63rd

Highly Effective School/Highly Effective Teacher

50th 96th

Highly Effective School/Average Teacher

50th 78th

Student Growth and Teacher Effectiveness (cont.)

• Argument against:– “Assumes student learning is measured well by a

given test, is influenced by the teacher alone, and is independent from the growth of classmates and other aspects of the classroom context. None of these assumptions is well supported by current evidence.” (Darling-Hammond, Amrein-Beardsley, Haertel, and Rothstein, 2012, p. 8)

Student Growth and Teacher Effectiveness (cont.)

• Three criticisms: 1. Value-Added Models (VAM) are inconsistent

– Research on VAM scores indicates they can change year to year

2. VAM scores differ significantly depending the way it is calculated

– Equations used to compute VAMs differ in the variables used to predict student outcomes

3. Teacher ratings based on VAMs do not reflect the many influences that contribute to or hinder student progress

– Arguments that teacher effectiveness is not a stable enough construct to be uniquely identified

Question: Where do we go from here?

Answer: Keep the goal in mind!(Continuously improving the quality of instruction so students are learning at high levels)

Some Things Frowned Upon

• To think about student growth absent of the way kids are sorted into schools

• What that means: – Student performance is calculated by

determining the difference of the student’s performance from the beginning of the year to the end of the year

– This would work against teachers who are teaching in lower performing districts

Questions to Answer

• What test or measure is going to be used to evaluate student achievement and then student growth?

• What population are you measuring student growth?

• Who will calculate the student growth (a vendor)?

Selecting Assessments

• Recommendations typically include using the state assessment plus another reliable and valid assessment– Validity – test measures what it claims to

measure (Ex. use of a blood pressure cuff is valid since it is measuring blood pressure and not something else)

– Reliability – extent to which an assessment tool produces stable and consistent result

Selecting Assessments (cont.)

• District developed assessments? (end of course assessments)– Are the tests up to the task?– Do the meet the qualities of reliability and

validity?

• At this point, it has been found that most district created assessments are not up to the task

Selecting Assessments (cont.)

• If you use these assessments in the teacher evaluation process you will want to be more lenient in how they get used to determine teacher effectiveness – Trend data

-OR-– Teachers whose data are at the lowest

percentile could be deemed “less than effective”

Selecting Assessments (cont.)

• What about using CBM (Curriculum Based Measures) like aimsweb or DIBELS?– DIBELS has addressed the issues of

reliability, validity, how to control for low vs. high performing districts

– DIBELS Accountability Decisions (AD)– DIBELS Pathways of Progress

Selecting Assessments (cont.)

• What about the secondary level? (CBM is more readily available at the elementary and middle school grades)– True– There are issues of reliability and validity of

CBM’s at the secondary level

• List the assessments that are being used across levels (elementary and secondary) to determine student growth within the teacher evaluation process?

• In light of the information presented thus far, what further questions, wonderments, or concerns do you have?

Activity

Questions Left to Answer

• What test or measure is going to be used to evaluate student achievement and then student growth?

• What population are you measuring student growth?

• Who will calculate the student growth (a vendor)?

Implementation Example

• Caution: This example is based on the assessments that were available to me when I was a building administrator

• Elementary: – DIBELS data (would use DIBELS AD and

Pathways of Progress– NWEA, MAP data for grades 2+– State assessment data

• I would try to steer clear from using district created assessments

But don’t forget student growth is only a piece of how to accomplish our goal of continuous improvement on the quality of instruction!

Recall How to Accomplish the Goal

• Measure student growth in multiple ways and aggregate data

• Use multiple points of data measured over time• Include teacher behaviors outside of the classroom• Ensure teacher evaluation provides an accurate

representation of teacher abilities• Use the evaluation system data to enhance

teacher’s ability to teach • Support teachers (ensure there is alignment at all

levels – align principal and central office evaluation systems

Marzano & Toth (2013)

These things will be discussed on 5/20!

Gots Wants