Data’s Face—Assessments for a Progressive Learning

Institution

Nick WadeValley View 365-U, Romeoville, IL

March 16, 2011

Learning Targets

Educators will become familiar with the research and practices of quality assessments and best practices.

Educators will learn and understand how to effectively align curricular objectives with a proper measurement tool.

Educators will learn and understand how to competently disaggregate data from the quality measurement tool.

A Little Bit About Your Amazing Speaker…

Bachelor of Arts, December 2006, Double Major: English and Secondary Education

Master of Education, May 2010, Leadership and Administration

Doctor of Education, November 2011, Educational Leadership – Superintendent Track

“I don’t mind being called the smartest man in the world. I just wish it wasn’t this one.”

~Ozymandias, The Watchmen by Alan Moore

Important quote or statistic I wish to emphasize.

What Has Not Been Happening?• Schools have not been

collecting and analyzing data in a way that would show a consistent level of development from grade level to grade level to be at the necessary proficiency level to succeed.

• Decision and/or policy makers need to pay attention to any and all numerical patterns in order to effectively monitor and evaluate programs, instructional practices, and the curriculum in order to best decide on how to proceed.

“Just the facts, ma’m.”• 70% of high school students graduate.• Of those 70%, only a third are prepared for

college.• Of that third, 40% must take remedial courses.• 2/3 of college/university professors believe what

is taught in high school is not applicable to college.

• The United States ranks tenth in industrialized nations for college completion.

SOURCE: The National Governor’s Association, National Assessments of Educational Progress

What Should We Do?• The American public education system

is comprised of a minimum of fifty state systems.

• No Child Left Behind (NCLB) established the fundamental goal of offering an equitable education to all students, but it also brought to light a number of important issues like the disconnect between the assessment itself and what is being taught in the classrooms (Achieve, 2002).

• Despite the state-based standardized assessments, there is “one” its results can be quantified and related between states, districts, schools, and teacher—the ACT.

Why the ACT?• Due to this one national

assessment all students take their junior year, it becomes logical to use that particular assessment as a benchmark.

• According to Rick Stiggins, the public education system should “take advantage” of the invaluable amount of research and data standardized assessments have been producing to “rethinking the relationship between assessment practices and effective schools” (Stiggins, 2004, pg. 22).

What Say You Skeptics?• The skeptics may cry that focusing on a

standardized assessment has ruined the schools.

• If standardized assessments are ruining our schools, then how come a) the students have not become proficient in mastering the regarded college and career readiness skills, and b) there has not been significant growth within this particular assessment across America?

• If the ACT is a misguided and supposedly simple assessment, then the results, across the board, should have reflected this notion. Otherwise, there’s a significant gap in curriculum, instruction, and the belief system of current educators.

“As someone who has spent his entire career doing research, writing and thinking about educational testing and assessment issues, I would like to conclude by summarizing a compelling case showing that the major uses of tests for student and school accountability during the past 50 years have improved education and student learning in dynamic ways. Unfortunately, that is not my conclusion.”

~Robert Linn, a creator and scholar of standardized tests

The System• “High-stakes tests without

supportive classroom assessment environments” will harm students, both struggling and succeeding from reaching the next personal benchmark (Stiggins, 2004, pg. 24).

• Use the College and Career Readiness Standards established by the ACT as the foundation for designing and administering such a system.

The Assessments• “Experts have described the distinct uses and

limitations of summative, formative, and interim assessments” (Perie, Marion, Gong, & Wurtzel, 2007).

• Summative assessments are given at the end of a particular unit and are reflective of the unit’s goals established by the course it’s under.

• Formative assessments are given significantly more frequently and usually at the creation of a particular instructor.

• The interim assessments are administered in a timely and purposeful manner that should effectively be measuring students’ achievement within the standards established by the curriculum.

The Quantitative• “Experts have described the distinct uses

and limitations of summative, formative, and interim assessments” (Perie, Marion, Gong, & Wurtzel, 2007). These are used primarily to provide quantitative data regarding mastery of taught information, and would primarily be used to evaluate the effectiveness of a particular curriculum or instructor.

• A valid and reliable formative assessment is aligned to the purpose and design of the unit’s summative. The results should allow teachers to modify instruction and effective use of them can provide a “positive effect on students learning” (Black & Wiliam, 1998).

Disaggregation of Data

• Bolingbrook High School’s English Department has collected a semester’s worth of data from both interim and summative assessments.

• The following graphs represent various sub-groups in addition to showing their development based on certain units in the curriculum.

• This data can help develop long-term goals.

Honors and Standard Tracks

Short Story Unit Fahrenheit 451or The Secret Lifeof Bees

Non-Fiction Unit Poetry Review

Male and Female

Short Story Unit Fahrenheit 451or The Secret Lifeof Bees

Non-Fiction Unit Poetry Review

Ethnicity

Short Story Unit Fahrenheit 451or The Secret Lifeof Bees

Non-Fiction Unit Poetry Review

Summative Score Reports

Holistic Conclusions• Written summative assessments

show a significant drop in comparison to the reading passages. As of now, we have three (3) written summatives for second semester.

• Scores improved in units that did not utilize large portions of text (i.e. a novel). As of now, second semester has the students reading at least three (3) plays and one novel.

• Summative assessment scores are closely related to the interim assessment scores, thus, informing us the rigor of the summatives are still in line with the demands of the ACT.

Analytical Conclusions• Students in all sub-groups have

shown growth from beginning to the end of the semester (averaging a 6% increase).

• No particular achievement gap has closed despite all sub-groups increasing consistently and gradually.

• Asian-American/Pacific Islander and Multi-Racial improved the most of all the ethnic sub-groups with a 7-8% increase.

• All male sub-groups, excluding Asian-American/Pacific Islanders, improved at least 9% with African-Americans the most at 12%.

Analytical Conclusions, Cont’d• All female sub-groups have not

increased more than 6%. Is it possible the curriculum and our instructional activities are geared more towards males, and females are being left behind?

• The Special Education Department has yet to disaggregate their data, but we will personally start keeping track second semester to avoid the wait at the request of senior leadership.

• The goal for second semester is for there to be a semester average, for each sub-group, of at least 7.5%.

• Next year, specific goals will be made depending on the student’s sub-group data from this year (details to come).

Questions About the Units• What particular units are showing

trends of being beneficial in comparison to other ones? What factors may contribute to this and how can we address them for second semester?

• Teachers have been spending different amounts of time on particular units. What are some reasons for this and are they showing to be helpful or hurtful towards student proficiency?

• What are the summative assessments telling us about our students, in addition to the interim assessments?

• How have we been using formative assessments to guide instruction?

Vocabulary• Students have been doing

relatively well in increasing the skills associated with reading and grammar, yet the scores are increasing at a tortoise-like speed, thus, indicating a clear block to the students' progress overall.

• What a few teachers did was change the vocabulary words the students struggled with the most (i.e. pessimistic or optimistic) and simply changed it into a synonym they would most likely recognize.

• There was an immediate 6-9% increase and it must be noted these problems were not “vocabulary in context.”

Long Term Goals• Create and implement a new vocabulary-based curriculum

that utilizes the most commonly used words on the ACT and SAT.

• Create and implement a new grammar- and writing-based curriculum that utilizes the demands and rigor of the ACT to improve writing and English scores.

• Continue collecting data based on students’ growth in certain units and begin adjusting pacing guides and the units themselves to more efficiently nurture student growth.

• All goals and be measured based on item analyses of particular skills from the interim and summative assessments.

ReferencesAchieve. (2002). No child left behind: Meeting challenges, seizing opportunities, improving achievement

(Achieve Policy Brief No. 5). Washington, D.C.: Author.Black, P., & Wiliam, D. (1998). Inside the black box: Raising standards through classroom assessment.

London: King’s College.Linn, R. “Assessment and Accountability.” Educational Researcher. February 2000, p. 14.Marzano, R., Pickering, D., & Pollock, J. (2001). Classroom instruction that works. Alexandria: ASCD

Press.Perie, M., Marion, S., Gong, B., & Wurtzel, J. (2007). The role of interim assessments in a comprehensive

assessment system. Aspen: Aspen Institute.“Rising to the Challenge: Are High School Graduates Prepared for College and Work?” Achieve (2005),

available online at http://www.achieve.org/files/pollreport.pdf (accessed November 7, 2010).Robert Linn. “Assessment and Accountability.” Educational Researcher. February 2000, p. 14.Schmidt, W.H., McKnight, C.C., Houang, R.T., Wang, H.A., Wiley, D.E., Cogan, L.S., et al.

(2001). Why schools matter: A cross-national comparison of curriculum and learning. San Francisco: Jossey-Bass.

Stiggins, Rick. (2004). New Assessment Beliefs for a New School Mission. Phi Delta Kappan, 86 (1), 22-27.

Tomlinson, C., & McTighe, J. (2006). Integrating differentiated instruction and understanding by design. Alexandria: ASCD Press.

Wagner, Tony. (2008). The Global Achievement Gap: Why Even Our Best Schools Don’t Teach the New Survival Skills Our Children Need—And What We Can Do About It. Basic Books: New York.

Walberg, H.J., Wang, M.C. (2001) Tomorrow’s Teachers. McCutchan Publishing Corporation: Berkley.