student growth in the non-tested subjects and grades: options for teacher evaluators
DESCRIPTION
Student Growth in the Non-Tested Subjects and Grades: Options for Teacher Evaluators Elena Diaz-Bilello, Center for Assessment Michael Cohen, Denver Public Schools Ruth Chung Wei, Stanford University Scott Marion, Center for Assessment Stuart Kahl, Measured Progress NCSA New Orleans - PowerPoint PPT PresentationTRANSCRIPT
Measured Progress ©2012
Student Growth in the Non-TestedSubjects and Grades:
Options for Teacher Evaluators
Elena Diaz-Bilello, Center for AssessmentMichael Cohen, Denver Public SchoolsRuth Chung Wei, Stanford UniversityScott Marion, Center for Assessment
Stuart Kahl, Measured Progress
NCSA New Orleans
June 25, 2014
Measured Progress ©2012
Alternative Assessment Strategiesfor Evaluating Teaching Effectiveness
(AASETE)
Stuart KahlMeasured Progress, Inc.
2014 NCSA, New Orleans
Measured Progress ©2012
The AASETE objective was to:
design a research-based system for using performance assessments along with other instruments to measure student academic growth, which in turn, could be used with other measures of teaching effectiveness for purposes of teacher evaluation in non-tested subjects and grades.
Measured Progress ©2012
The Research
• >12,000 students in three states
• Approx. 250 teachers
• >600 classrooms
• 18 subject/grade-level/state combinations
• Pre- and post-testing
• Prior achievement data (test scores and grades)
• Comparison of growth models
Measured Progress ©2012
Major Findings
• moderate to high correlations among
results for different growth models applied to data from the same (or equated) tests
• quite variable correlations among indicators based on the same model, but different end-of-year tests (state vs. AASETE) or based on the same model applied to different test components (multiple-choice vs. performance)
Measured Progress ©2012
Findings (continued)
• only slightly higher correlations among indicators with sophisticated scaling of student scores as opposed to raw (or linearly transformed) student scores
• moderate to high correlations among indicators based on simple growth or simple prediction models and those based on more sophisticated models
Measured Progress ©2012
Features of AASETE-Recommended Approach
• common end-of-course (or interim) assessments across teachers, schools, districts; multiple assessment components, including performance
• less sophisticated analyses easily run with commonly used software packages such as Excel
Measured Progress ©2012
Features (continued)
• a simple prediction model subtracting students’ predicted scores on a common end-of-course measure from the actual scores on that measure, and aggregating (averaging) the differences at the teacher level
• human judgment in deciding if the student growth for a particular teacher is adequate given the unique characteristics of the teacher’s students, the other unique contextual factors of the teacher’s situation, and previous growth indicators for the teacher
Measured Progress ©2012
A System for Using Student Academic Growth
in the Evaluation of Teaching Effectiveness in
the Non-Tested Subjects and Grades
A Guide for Education Policy Makers and Evaluators of Teachers
Measured Progress, Inc.May 2014
This document was prepared by Measured Progress, Inc. with funding from the Bill and Melinda Gates Foundation, Grant No. OPP1029363. The content of the publication does not necessarily reflect the views of the Foundation.
Measured Progress ©2012
Table of Contents
Preface …………………………………………………………… 3
Acknowledgments …………………………………………… 5
Measured Progress ©2012
Contents (continued)
Student Academic Growth and Teacher Evaluations …… 6
The Problem in the Non-Tested Subjects and Grades … 6
What Are the Options for the Non-Tested Subjects and Grades ……………………………………………………………… 7
Test-Based Value-Added/Growth Indicators ……… 7Student Learning Objectives (SLOs) …………………… 8Comparisons of Approaches …………………………… 9
Human Judgment and Multiple Measures ………………… 12Interpreting Normative Data …………………………… 12Weighing Multiple Measures …………………………… 13
Measured Progress ©2012
Contents (continued)
The Recommended “Simple Regression” Approach …… 15
Common Assessments …………………………………… 15
Why Not Simple Pre-Post Growth? ……………………. 16
Simple Prediction/Regression ………………………… 16
Numbers of Students and Teachers …………………… 18
Outcome or End-of-Course Measures .……………… 19
More on Performance Components …………………… 21
Measured Progress ©2012
Contents (continued)
Predictor Variables …………………………………………… 22Predictors in General …………………………………… 22Predictors to Use …………………………………………
23
Associated Analyses and Checks …………………………23
Some Final Words on the Generation, Interpretation,and Use of Value-Added/Growth Statistics ……………
25More on the Proposed Method
…………………… 25How Much Work Is It?
……………………………… 26
Measured Progress ©2012
Contents (continued)
Appendix A: Overview and Recommendations of AASETE Study ………………………………28
Appendix B: Instructions for Performing Regression-Based Growth Analysis at the Teacher Level Using Excel …………
31
Measured Progress ©2012
Measured Progress ©2012
Measured Progress ©2012
Measured Progress ©2012
How Much Work Is It?
• End-of-course testing already happening
• Data management systems in place with data
• Capability to access and use data
• One new Excel analysis to learn
• The challengesPolicy Human judgment
• District assessment program
Measured Progress ©2012
P.O. Box 1217, Dover, NH 03821-1217 | Web: measuredprogress.org | Office: 603.749.9102 It’s all about student learning. Period.
Thank you!