Everett Public Schools

Power of Proficiency Scaling

Jo Anne Buiteweg, Director of Learning Management Systems

Jana Sanchez, STEM Mathematics Facilitator

ATI Summer 2014

Entry TaskWrite down some of your thoughts for these two questions; be prepared to share.1.What does a “2” represent? (approaching proficiency)

- For a teacher?- For a student?- For a parent?- For a transcript reviewer?

2.What does a “4” represent? (beyond proficiency)- For a teacher?- For a student?- For a parent?- For a transcript reviewer?

“Clearly a better method for developing and scoring assessments is needed – one that ensures that the scale (the size of an inch) stays the same from one assessment to the next and that a teacher applies the same logic to scoring of each assessment.”

Marzano, Robert J. Formative Assessment and Standards-Based Grading 2010

• How does“ Depth of Knowledge” (DOK) as defined by Norman Webb facilitate the implementation of standards-aligned program?

• How is the “Depth of Knowledge” a foundation for the construction of a proficiency scale to “unpack” clusters/ standards?

• How can we use the implementation of CCSS and NGSS as a way to infuse standards-focused practices?

• What implications does scaling have as the foundation for building leveled assessments and instructional plans?

• How can instructional leaders utilize proficiency scaling as a focal point for a more coherent system for curriculum, assessment and instruction?

Session’s Overview

Backwards Design Model

Norman Webb’s Depth of KnowledgeAdapted from the model used by Norm Webb, University of Wisconsin, to align standards with assessments

•The degree of depth or complexity of knowledge reflected in the content standards and assessments

•How deeply a student needs to understand the content for a given response/assessment

• Addresses the content being assessed and the depth to which we expect students to demonstrate understanding of that content.

• Is a reference to the complexity of mental processing that must occur to answer a question, perform a task, or generate a product

• Is about cognitive complexity, not difficulty• Is not grade contextualized• Offers a pathway to rigor

Webb’s Depth of Knowledge (DOK)

Created based on work of Webb, Norman L. and others. “Web Alignment Tool” 24 July 2005. Wisconsin Center of Educational Research. University of Wisconsin-Madison. 2 Feb. 2006

Depth of Knowledge depends on more than the verb. The complexity also depends on what the verb is acting on. For example, “draw” is in the DOK level 1 sector. But a student who draws a blueprint of a new building is doing more than recall of information.

Explain also can be at different levels--explain by repeating a definition (DOK level 1), explain by putting a paragraph into your own words (DOK level 2), or explain by describing an analysis of the factors contributing to the economic down turn of the US (DOK level 3).

Bloom’s and Webb’s

Different models to describe cognitive rigor•Bloom – What type of thinking (verbs) are needed to complete the task?

•Webb – How deeply do you have to understand the content to successfully interact at a given depth? How complex is the content?

Cognitive Rigor MatrixThis matrix from the Smarter Balanced Content Specifications for Mathematics draws from both

Bloom’s (revised) Taxonomy of Educational Objectives and Webb’s Depth-of-Knowledge Levels below.

DOK Levels

Depth of Knowledge

• Recall— Identify this utensil. • Concept— Explain the function of

the fork.• Strategic— Identify two examples

of when a fork would not be the best utensil for a type of food and explain why.

• Extended— Design an investigation to determine the optimal number and length of tines for a salad fork.From: Lois Barnes

SREB/HSTW

Depth of Knowledge

• Recall— Identify the type of tree.• Concept— Explain the function of

the leaves.• Strategic— Explain how a drought

might affect the growth of the tree.• Extended— Design an investigation

of seedling growth to determine the best fertilizer for this type of tree.

From: Lois Barnes

SREB/HSTW

Card Sort Activity?

• Working with an elbow partner label each task/prompt with the appropriate DOK level and justify your decision.

Cognitive Complexity vs Difficulty What is the difference between Cognitive Complexity and Difficulty Level?

•Difficulty refers to how many students answered the question correctly. •High Order Thinking refers to how many steps it takes to answer the question.

Add:4,678,895+ 9,578,885•What is the DOK?

DOK Snapshot• DOK is a scale of cognitive demand• DOK is not an exact science• DOK is not about difficulty but how much thinking is required

for the student to complete the prompt/task• DOK is about the item/standard not the student• The context of the item/standard must be considered to

determine the DOK level not just a look at what verb was chosen.

• DOK is lowered when too much information is given

How can we be consistent in applying DOK at our grade level?

Depth of Knowledge = Cognitive Demand = Rigor

•How much and what kind of “thinking” is called for in each CCSS cluster, for classroom instruction and on assessments?

•What kinds of “thinking” is called for approaching the cluster and beyond the cluster?

Proficiency ScalingThe process of identifying and developing the cognitive demand or level of rigor for a given standard.- Starting with the standard: educators use a framework …

(Bloom’s, Costa’s , Webb’s DOK or combination) …as way to build “a rigorous rubric-based approach in the interest of valid and reliable assessing” which informs both teacher and student

- If Proficient is the standard:- What is Advanced?- What is a Basic?

Marzano, Robert J. Formative Assessment and Standards-Based Grading 2010

Common CoreGrades K Grades 1 Grades 2

Understand addition as putting together and adding to, and understand subtraction as taking apart and taking from.1. Represent addition and subtraction with objects, fingers, mental images, drawings2, sounds (e.g., claps), acting out situations, verbal explanations, expressions, or equations.2. Solve addition and subtraction word problems, and add and subtract within 10, e.g., by using objects or drawings to represent the problem.3. Decompose numbers less than or equal to 10 into pairs in more than one way, e.g., by using objects or drawings, and record each decomposition by a drawing or equation (e.g., 5 = 2 + 3 and 5 = 4 + 1).4. For any number from 1 to 9, find the number that makes 10 when added to the given number, e.g., by using objects or drawings, and record the answer with a drawing or equation.

Represent and solve problems involving addition and subtraction.1. Use addition and subtraction within 20 to solve word problems involving situations of adding to, taking from, putting together, taking apart, and comparing, with unknowns in all positions, e.g., by using objects, drawings, and equations with a symbol for the unknown number to represent the problem.22. Solve word problems that call for addition of three whole numbers whose sum is less than or equal to 20, e.g., by using objects, drawings, and equations with a symbol for the unknown number to represent the problem.

Represent and solve problems involving addition and subtraction.

Use addition and subtraction within 100 to solve one- and two-step word problems involving situations of adding to, taking from, putting together, taking apart, and comparing, with unknowns in all positions, e.g., by using drawings and equations with a symbol for the unknown number to represent the problem.

Advanced Cognitive task extending from standard; requiring decision-making, expressing reasoning, or applying what has been explicitly taught in new contexts

Proficient 

Standard as defined by the state including expectations for content, process, skills, and/or performance to be explicitly taught.

Basic 

Cognitive step just before standard that is explicitly taught; includes concepts broken into distinct segments, foundational skills and key vocabulary.

Foundational

With help, partial success of Basic or Proficient content and/or elements preceding Basic level.

Recall/Reproduction

DOK 1

Skills/

Co

ncep

tsD

OK

2

Strategic Thinking

DOK 3

Ext

end

ed

Th

inki

ng

DO

K 4

ProficiencyProficiency Scale level

does not equal DOK level

K-Math

Advanced Cognitive task extending from standard; requiring decision-making, expressing reasoning, or applying what has been explicitly taught in new contexts Count backwards from a given number by ones.Write numbers from various starting points beyond 20 and continue the number pattern.Write given numbers out of sequence above 20.

Proficient

Standard as defined by the state including expectations for content, process, skills,

and/or performance to be explicitly taught.Know number names and the count sequence. (MTH.K.CC.KNNCS)Count to 100 by ones (MTH.K.CC.KNNCS.1)Count to 100 by tens (MTH.K.CC.KNNCS.1)Count forward beginning from a given number within the known sequence (instead of having to begin at 1). (MTH.K.CC.KNNCS.2)Write numbers from 0 to 20. (MTH.K.CC.KNNCS.2)Represent a number of objects with a written numeral 0-20 (with 0 representing a count of no objects). (MTH.K.CC.KNNCS.3)

Basic

Cognitive step just before standard that is explicitly taught; includes concepts broken into distinct segments, foundational skills and key vocabularyUses the pattern of 1-9 to count within a decade (e.g. in twenties, in thirties, etc)Writes numbers from 0 to 20 using a tool (e.g. number line, 100s chart)Represents a number of objects with a visual tool 0-20 (refer to a number line or number card). Recognizes the numbers from 0 to 20.

Foundational With help, partial success of Basic or Proficient content and/or elements preceding Basic level.

Advanced Cognitive task extending from standard; requiring decision-making, expressing reasoning, or applying what has been explicitly taught in new contexts

Proficient 

Standard as defined by the state including expectations for content, process, skills, and/or performance to be explicitly taught.

Basic 

Cognitive step just before standard that is explicitly taught; includes concepts broken into distinct segments, foundational skills and key vocabulary.

Foundational

With help, partial success of Basic or Proficient content and/or elements preceding Basic level.

Difference Between Scales and Rubrics

• Scales are built for teachers use in planning assessments and instruction

• Scales are tied to standard – independent of performance task

• Basic tasks are deliberate performance expectations not written to be “lacking” or “missing” proficient elements

• Rubrics are tied to specific performance expectations– Rubrics are smaller picture– Proficiency Scales are the bigger picture

Build Summative Assessments and Assessment Maps

Build summative assessments and assessment maps• Reliable and Valid• Items/Prompts aligned to standards• Items/Prompts leveled• Scoring and reporting variables determined

Fix 7: Don’t organize by type but by standardFix 8: Don’t grade unclear standards; provide clear expectationsFix 10: Don’t rely on weak assessments; use quality assessments

Leveraging Scaling

Scaling and

Leveling

29

Curriculum & Instruction

Rigorous, Informative Assessment

Reporting Variables/Grading Practices

Is school about ACTIVITIES or LEARNING?

Schimmer’s Key Questions

Is school about POINTS or EVIDENCE?Is learning an EVENT or a PROCESS?

Emphasize COMPLETION or STANDARDS?

Shift from Completion to Standards

Emphasize COMPLETION or STANDARDS?

Fix 1: Don’t include student behaviorFix 2: Don’t reduce score for late workFix: 14: Don’t summarize evidence over time: emphasize recent achievement.

Is school about ACTIVITIES or LEARNING?

Shift from Activities to Learning

Fix 5: Don’t reduce grade based on attendanceFix 6: Don’t include “group scores”Fix 9: Don’t assign grades through comparison to others

Shift from Points to Evidence

Fix 3: Don’t add “extra” pointsFix 4: Don’t punish with grades (dishonesty)Fix 11: Don’t rely on the mean; use other measures and professional judgmentFix 12: Don’t include 0s – use “I” for insufficient evidence and gather information to make determinations

Shift from Event to Process

Is learning an EVENT or a PROCESS?

Fix 13: Don’t use formatives in grade; use only summative evidenceFix 15: Don’t leave students out of the learning process – they should play key roles in the assessment process”

Welcome, Encourage, Inspire to Build Confident Learners