DOK and HOTSDOK and HOTS

Presented By:Presented By:

Cisely ScottCisely Scott

Julian DavenportJulian Davenport

DATE: June 23, 2011

BENCHMARK: DOK, HOTS, and Questioning Strategies

OBJECTIVE:

Today we will examine the role Higher Order Thinking, Depth of Knowledge, and Questioning Strategies on student achievement.

ESSENTIAL QUESTION:

How does increasing the rigor in your questioning enhance the performance of your students in the classroom? What are the long term implications of using these strategies?

VOCABULARY:

Depth of Knowledge, Cognitive Complexity (low, medium, high), Difficulty vs. Complexity, HOT Verbs, Questioning Strategies, Inquiry Based Learning,

AGENDA: •What the Research Says….•What is Depth of Knowledge?•Application to FCAT•Elements of Inquiry•Activity-Developing a HOT Question•How Do We Ensure Use in the Classroom? •Journal Reflection: How does the use of HOTs questions, DOK, and Questioning Strategies affect teacher performance and student achievement?

EXIT SLIP: IN YOUR JOURNALConnect: How does the use of HOTs questions, DOK, and Questioning Strategies affect teacher

performance and student achievement?Connect: 1 question you still have about FCAT 2.0.

HOME LEARNING:

Review today’s lesson and develop your next steps to share this information with teachers at your school

Common Board Configuration (CBC)Common Board Configuration (CBC)

ESSENTIAL QUESTION (EQ)ESSENTIAL QUESTION (EQ)

How does increasing the rigor in teacher questioning enhance the performance of your students in the classroom? What are the long term implications of using these strategies?

REACHING YOUR VISIONREACHING YOUR VISION

This presentation will help you to This presentation will help you to reach your vision as a coach by reach your vision as a coach by providing you with an overview providing you with an overview DOK, HOT, and Questioning DOK, HOT, and Questioning Strategies so that you’re able to Strategies so that you’re able to monitor for and model these monitor for and model these strategies within the classroom.strategies within the classroom.

What the Research What the Research Says……Says……

Teachers are the KeyTeachers are the Key

“Teachers must be the primary driving force behind change. They are best positioned to understand the problems that students face and to generate possible solutions.”

James Stigler and James Hiebert,The Teaching

Gap

Quality Instruction Makes A Difference

“Good teaching can make a significant difference in student achievement, equal to one effect size (a standard deviation), which is also equivalent to the affect that demographic classifications can have on achievement.”

Paraphrase Dr. Heather Hill, University of Michigan

Differences in Instruction

“Our research indicates that there is a 15% variability difference in student achievement between teachers within the same schools.”

Deborah Loewenberg Ball, Dean of Education, University of Michigan

“What Matters Very Much is Which Classroom?”

“If a student is in one of the most effective classrooms he or she will learn in 6 months what those in an average classroom will take a year to learn. And if a student is in one of the least effective classrooms in that school, the same amount of learning take 2 years.”

Research has indicated that... “teacher quality trumps virtually all other influences on student achievement.”

(e.g., Darling-Hammond, 1999; Hamre and Pianta,2005; Hanushek, Kain, O'Brien and Rivken, 2005;Wright, Horn and Sanders, 1997)

So what are effective teachers So what are effective teachers doing differently? What should you doing differently? What should you

be looking for as a coach?be looking for as a coach?

Discuss with your neighbor the answer Discuss with your neighbor the answer to this question. (5-10 mins). Come to this question. (5-10 mins). Come up with a list of 10 things to share.up with a list of 10 things to share.

Rigorous ClassroomRigorous ClassroomStandards-based teachingStandards-based teaching

Look for Lots of high level activity- - - Look for Lots of high level activity- - - High Levels of QuestioningHigh Levels of Questioning ReflectingReflecting AnalyzingAnalyzing Doing experiments (science)Doing experiments (science) Doing exploratory activitiesDoing exploratory activities Discussing/DebatesDiscussing/Debates WritingWriting Working in groupsWorking in groups

And And a Scoring Guide available to all students for all major a Scoring Guide available to all students for all major assignmentsassignments

Sets expectationsSets expectations

12

Rigorous ClassroomRigorous Classroom

Only by creating a culture of high Only by creating a culture of high expectations and providing support expectations and providing support so students can truly succeed do you so students can truly succeed do you have a rigorous classroom.have a rigorous classroom.

Barbara Blackburn 2008Barbara Blackburn 2008

13

What is Depth Of Knowledge?What is Depth Of Knowledge?

Cognitive Demand

• The kind and level of thinking required of students to successfully engage with and solve a task • Ways in which students interact with content

Why Depth of Why Depth of Knowledge?Knowledge?

Focuses on complexity of content standards in order to successfully complete an assessment or task.

The outcome (product) is the focus of the depth of

understanding.

Why Use a Depth of Knowledge?Why Use a Depth of Knowledge?

•Used to determine the level of Used to determine the level of the expected outcomes of the the expected outcomes of the Next GenerationNext Generation Sunshine Sunshine State Standards and State Standards and benchmarksbenchmarks

•Determines Determines the complexity of the complexity of FCATFCAT items items (success with (success with items leads to AYP)items leads to AYP)

Verbs are Verbs are notnot always used always used appropriately... appropriately...

Words like explain or analyze have to be considered in context.

•“Explain to me where you live” does not raise the DOK of a simple rote response.

• Even if the student has to use addresses or landmarks, the student is doing nothing more than recalling and reciting.

DOK DOK isis about what follows the verb... about what follows the verb...

What comes after the verb is more important than the verb itself.

“Analyze this sentence to decide if the commas have been used correctly” does not meet the criteria for high cognitive processing.”

The student who has been taught the rule for using commas is merely using the rule.

Same Verb—Three Different DOK LevelsSame Verb—Three Different DOK Levels

DOK 1- Describe three characteristics of metamorphic rocks. (Requires simple recall)

DOK 2- Describe the difference between metamorphic and igneous rocks. (Requires cognitive processing to determine the differences in the two rock types)

DOK 3- Describe a model that you might use to represent the relationships that exist within the rock cycle. (Requires deep understanding of rock cycle and a determination of how best to represent it)

Activity

In your groups, use the word explain, analyze, or any of the other words on the ladder to come up with a low complexity question and a higher complexity question using the verb. (5 minutes)

DOK DOK isis about intended outcome, about intended outcome, notnot difficulty difficulty

DOK is a reference to the complexity of mental processing that must occur to answer a question, perform a task, or generate a product.

• Adding is a mental process.• Knowing the rule for adding is the intended

outcome that influences the DOK.• Once someone learns the “rule” of how to

add, 4 + 4 is DOK 1 and is also easy.• Adding 4,678,895 + 9,578,885 is still a DOK 1

but may be more “difficult.”

DOK is DOK is notnot about difficulty... about difficulty...

• Difficulty is a reference to how many students answer a question correctly.

“How many of you know the definition of exaggerate?” DOK 1 – recall If all of you know the definition, this question is an easy question.

“How many of you know the definition of vociferous?” DOK 1 – recall If most of you do not know the definition, this question is a difficult question.

DOK DOK isis about complexity about complexity

• The intended student learning outcome determines the DOK level.

• Every objective in the science and mathematics frameworks has been assigned a DOK level.

• Instruction and classroom assessments must reflect the DOK level of the objective or intended learning outcome.

Quick QuizQuick Quiz1) Give an example of a statement

that uses a verb that “sounds” like a high DOK but is used inappropriately.

2) Fill in the blanks: What _____ the verb is more _____ than the verb itself when deciding the DOK level.

3) What is the difference between difficulty and complexity?

4) What really determines the DOK level?

Copy this down in your notebook?

Quick Quiz AnswersQuick Quiz Answers1) Give an example of a statement that uses a

verb that “sounds” like a high DOK but is used inappropriately. answers vary

2) Fill in the blanks: What follows the verb is more important than the verb itself when deciding the DOK level.

3) What is the difference between difficulty and complexity? answers vary, but do not rely on the verb

4) What really determines the DOK level? the intended learning outcomes

How does this apply to FCAT?How does this apply to FCAT?

Correlation to FCATCorrelation to FCAT

Percentage of Points by CognitivePercentage of Points by Cognitive Complexity Level for FCAT SSS ReadingComplexity Level for FCAT SSS Reading

FCAT Science 2.0FCAT Science 2.0Here is the new breakdown, based on Webb’s DOK:Here is the new breakdown, based on Webb’s DOK:

Note: A high complexity question in 5Note: A high complexity question in 5thth grade may not be a grade may not be a high complexity question in 8high complexity question in 8thth grade. The complexity is grade. The complexity is ultimately based on the cognitive demand placed on the ultimately based on the cognitive demand placed on the student. Cognitive complexity also may depend on the level of student. Cognitive complexity also may depend on the level of the distractor. the distractor.

Cognition Classification Cognition Classification SchemesSchemes

Classification schemes for cognition allow communication about the nature and characteristics of thinking and learning.

Examples: Bloom’s Taxonomy and Webb’s Depth of Knowledge

What is Webb’s Depth of Knowledge?What is Webb’s Depth of Knowledge?

Webb’s Levels of Cognitive ComplexityWebb’s Levels of Cognitive Complexity

Low ComplexityLow Complexity relies on the recall and relies on the recall and recognition of previously learned concepts and recognition of previously learned concepts and principlesprinciples

Moderate ComplexityModerate Complexity involves more flexible involves more flexible thinking and choice among alternativesthinking and choice among alternatives

High ComplexityHigh Complexity requires students to engage in requires students to engage in more abstract reasoning, planning, more abstract reasoning, planning, analysis, judgment, and analysis, judgment, and creative thinkingcreative thinking

Complexity v. DifficultyComplexity v. Difficulty

The degree of challenge of FCAT items is currentlyThe degree of challenge of FCAT items is currentlycategorized in two ways:categorized in two ways: Cognitive ComplexityCognitive Complexity

Determined by what the item requires the student to recall, Determined by what the item requires the student to recall, understand, analyze, and do. understand, analyze, and do.

Assume student is familiar with basic concepts of the task Assume student is familiar with basic concepts of the task and focuses on the task.and focuses on the task.

Depends on the task not the student.Depends on the task not the student.

• Item DifficultyItem Difficulty Depends on the percentage Depends on the percentage of students likely to answer of students likely to answer

correctly.correctly.• Easy – More than 70%Easy – More than 70%• Average – 40% to 70%Average – 40% to 70%• Challenging – Less than 40%Challenging – Less than 40%

Low ComplexityLow ComplexityDOK Level

1 Low Low complexity complexity items rely heavily on the items rely heavily on the recall and recognitionrecall and recognition of of

previously learned concepts and principles. previously learned concepts and principles. Items typically specify what the student is to do, which is often to Items typically specify what the student is to do, which is often to

carry out some carry out some procedure that can be performed mechanicallyprocedure that can be performed mechanically. . It is It is NOTNOT left to the student to come up with an left to the student to come up with an original method or original method or

solutionsolution.. Skills requiredSkills required to respond to low complexity items include: to respond to low complexity items include:

• solving a one-step problem;solving a one-step problem;• computing a sum, difference, product, or quotient;computing a sum, difference, product, or quotient;• evaluating a variable expression, given specific values for the variables;evaluating a variable expression, given specific values for the variables;• recognizing or constructing an equivalent representation;recognizing or constructing an equivalent representation;• recalling or recognizing a fact, term, or property;recalling or recognizing a fact, term, or property;• retrieving information from a graph, table, or figure;retrieving information from a graph, table, or figure;• identifying appropriate units or tools for common measurements;identifying appropriate units or tools for common measurements;• performing a single-unit conversion;performing a single-unit conversion;• reproducing a diagram or standard representation;reproducing a diagram or standard representation;• completing a routine procedurecompleting a routine procedure, such as measure temperature; and, such as measure temperature; and• calculating using a common formula.calculating using a common formula.

Moderate ComplexityModerate ComplexityDOK Level

2 FCAT Mathematics moderate complexity items involve FCAT Mathematics moderate complexity items involve more flexible more flexible

and choice among alternativesand choice among alternatives.. Items require a response that Items require a response that goes beyond the habitualgoes beyond the habitual, is not explicitly , is not explicitly

specified in the text, and specified in the text, and ordinarily has more than a single stepordinarily has more than a single step.. The student is expected to decide what to do using informal methods of The student is expected to decide what to do using informal methods of

reasoning and problem solving strategies and to reasoning and problem solving strategies and to bring together skill and bring together skill and knowledge from various domainsknowledge from various domains..

Skills requiredSkills required to respond to moderate complexity items include: to respond to moderate complexity items include:• solving a problem requiring multiple operations;solving a problem requiring multiple operations;• solving a problem involving spatial visualization and/or reasoning;solving a problem involving spatial visualization and/or reasoning;• selecting and/or using different representations, depending on situation and purpose;selecting and/or using different representations, depending on situation and purpose;• retrieving information from a graph, table, or figure and using it to solve a problem;retrieving information from a graph, table, or figure and using it to solve a problem;• determining a reasonable estimate;determining a reasonable estimate;• extending an algebraic or geometric pattern;extending an algebraic or geometric pattern;• providing a justification for steps in a solution process;providing a justification for steps in a solution process;• comparing figures or statements;comparing figures or statements;• representing a situation in more than one way;representing a situation in more than one way;• formulating a routine problem, given data and conditions;formulating a routine problem, given data and conditions;• explaining relationships between facts, terms, or properties;explaining relationships between facts, terms, or properties;• describing and explain examples and non-examples of science concepts;describing and explain examples and non-examples of science concepts;• making decisions and comparisons;making decisions and comparisons;• selecting a procedure according to criteria and perform procedure;selecting a procedure according to criteria and perform procedure;• organizing, representing, and interpreting data.organizing, representing, and interpreting data.

High ComplexityHigh ComplexityDOK Level

3 High complexity items make heavy demands on student thinking.High complexity items make heavy demands on student thinking. Students must engage in Students must engage in abstract abstract reasoning, planning, analysis, reasoning, planning, analysis,

judgment, and creative thoughtjudgment, and creative thought. . Items require that the student think in a Items require that the student think in a more abstract and more abstract and

sophisticated way.sophisticated way. Skills required Skills required to respond correctly to high complexity items to respond correctly to high complexity items

include:include:• performing a procedure having multiple steps and multiple decision pointsperforming a procedure having multiple steps and multiple decision points;;• solving a non-routine problem (as determined by grade-level appropriateness);solving a non-routine problem (as determined by grade-level appropriateness);• solving a problem in more than one way;solving a problem in more than one way;• describing how different representations can be used for different purposes;describing how different representations can be used for different purposes;• generalizing an algebraic or geometric pattern;generalizing an algebraic or geometric pattern;• explaining and justifying a solution to a problem;explaining and justifying a solution to a problem;• describing, comparing, and contrasting solution methods;describing, comparing, and contrasting solution methods;• providing a mathematical justification;providing a mathematical justification;• analyzing similarities and differences between procedures and concepts;analyzing similarities and differences between procedures and concepts;• formulating an original problem, given a situation;formulating an original problem, given a situation;• formulating a mathematical model for a complex situation;formulating a mathematical model for a complex situation;• analyzing or producing a deductive argument;analyzing or producing a deductive argument;• identifying research questions and design experimentsidentifying research questions and design experiments• developing a scientific model for a complex situation; anddeveloping a scientific model for a complex situation; and• forming conclusions and generalizations from experimental data.forming conclusions and generalizations from experimental data.

How can I How can I REALLYREALLY understand the understand the “depth” of the benchmarks?“depth” of the benchmarks?

Resources to help teachers gain a better Resources to help teachers gain a better understanding of the benchmarks understanding of the benchmarks areare available.available.

Professional development, PLCs, Lesson Professional development, PLCs, Lesson Study, common team planning, and Study, common team planning, and instructional coaching will support instructional coaching will support teachers as they transition to the NGSSS. teachers as they transition to the NGSSS.

What role should my textbook play as a What role should my textbook play as a resourceresource??

Elements of Inquiry:Elements of Inquiry:Checks for Understanding and Checks for Understanding and

Questioning StrategiesQuestioning Strategies

Top 10 Reasons to Ask Top 10 Reasons to Ask QuestionsQuestions

1.1. Increases student retention of knowledge, skills, and Increases student retention of knowledge, skills, and

understandings.understandings.

2.2. Fosters an active learning environment to maintain student Fosters an active learning environment to maintain student

engagement. engagement.

3.3. Stimulates students’ interest and curiosity in content.Stimulates students’ interest and curiosity in content.

4.4. Encourages both individual and group accountability.Encourages both individual and group accountability.

5.5. Assesses what students already know.Assesses what students already know.

6.6. Diagnoses difficulties or road blocks to learning.Diagnoses difficulties or road blocks to learning.

7.7. Helps students monitor their own understandings.Helps students monitor their own understandings.

8.8. Supports students in making conjectures and arguing validity. Supports students in making conjectures and arguing validity.

9.9. Allows students to hear different explanations by their peers.Allows students to hear different explanations by their peers.

10.10.Develops student communication skills and social development.Develops student communication skills and social development.

Quality QuestioningQuality Questioning Quality questioning frames, delivers, times, Quality questioning frames, delivers, times,

solicits, and responds to questions in a way that solicits, and responds to questions in a way that reveals as much information as possible about reveals as much information as possible about what students know or do not know.what students know or do not know.

It’s not, necessarily, about asking more questions. It’s not, necessarily, about asking more questions.

In fact, research shows that teachers ask a lot of In fact, research shows that teachers ask a lot of questions in the course of a day…approximately questions in the course of a day…approximately one every 72 seconds, on average.one every 72 seconds, on average.

However, almost half of these questions are However, almost half of these questions are actually answered by the teacher, not the actually answered by the teacher, not the students.students.

Ask Questions BetterAsk Questions Better Pose a question to the whole class, then call a student by name to Pose a question to the whole class, then call a student by name to

respond, “Question First, Name Last.” respond, “Question First, Name Last.” Make and maintain eye contact with students.Make and maintain eye contact with students. Consider a “no hands up” approach.Consider a “no hands up” approach. Use a random technique for selecting students to avoid always Use a random technique for selecting students to avoid always

calling on the same students. calling on the same students. Limit if not eliminate unison response questions.Limit if not eliminate unison response questions. Provide students with “wait time” to think before responding.Provide students with “wait time” to think before responding. Redirect student questions to other students to answer whenever Redirect student questions to other students to answer whenever

possible.possible. Ask all students regardless of gender, ethnicity, and/or Ask all students regardless of gender, ethnicity, and/or

socioeconomic status equally challenging questions.socioeconomic status equally challenging questions. Vary pace of questioning as necessary to ensure lesson Vary pace of questioning as necessary to ensure lesson

momentum.momentum.

ActivityActivity

1.1. Each group will select a benchmark from Each group will select a benchmark from either the Math or Science Test Item either the Math or Science Test Item Specifications. Specifications.

2.2. Groups will develop a Higher Order Groups will develop a Higher Order Thinking Question.Thinking Question.

3.3. Each group will present.Each group will present.4.4. Using Think-Pair-Share, groups will Using Think-Pair-Share, groups will

discuss whether it is low, moderate, or discuss whether it is low, moderate, or high complexity and why or why nothigh complexity and why or why not

5.5. Using the whiteboards, you will vote L, M, Using the whiteboards, you will vote L, M, or H, and be asked to explain why. or H, and be asked to explain why.

How do we ensure that teachers How do we ensure that teachers utilize Higher Order Thinking in the utilize Higher Order Thinking in the

Classroom? Classroom? Plan higher order questions in your Plan higher order questions in your

common planning sessions.common planning sessions. Include HOTS questions in Include HOTS questions in

promethean/smartboards to make promethean/smartboards to make sure that teachers address the sure that teachers address the questions. questions.

Journal ReflectionJournal Reflection

How does the use of HOTs questions, DOK, and Questioning Strategies affect teacher performance and student achievement?