common core standards, content complexity, and depth of knowledge (dok) october 16, 2015 third...
Post on 18-Jan-2016
223 Views
Preview:
TRANSCRIPT
Common Core Standards, Content
Complexity, andDepth of Knowledge
(DOK) October 16, 2015Third Annual MENA ConferenceAmerican UniversityDubai, United Arab Emirates
Norman L. Webb
Wisconsin Center for Education Products and Services (WCEPS)
Common Core Standards, Content Complexity, and Depth of Knowledge (DOK)
Norman L. WebbSenior Research Scientist Emeritus
Wisconsin Center for Education ResearchUniversity of Wisconsin-Madison
andWisconsin Center for Education Products and
Services
Attainment To
Readiness
The Common Core Standards (CCS) add focus and coherence to the K-12 curriculum to help ensure students are college and career ready by the end of high school.
Outline
Part 1: History of Content Complexity
Part 2: Depth of Knowledge
Part 3: Analysis and Implications
Complexity of the Common Core
Standards
PART 1: History of Content Complexity
Higher performing countries on TIMSS
more effectively used cognitively
demanding tasks and had students engage in critical thinking
and reasoning.(Hiebert and Stigler, 2004)
What does“content complexity”
mean?
What factors make educational material simple vs. complex?
Content Complexit
y
Processing of
concepts and skills
Prior knowledge
and experience
Sophisticationof material
Number of parts
Application of
content structure
Content complexity is important because…
• Alignment
• Validity of assessments
• Clarity of communication
• Truth in advertising
Ralph Tyler’s Behavioral Aspect of the Objectives (course dependent)
1. Understanding of important facts and principles
2. Familiarity with dependable sources of information
3. Ability to interpret data
4. Ability to apply principles
5. Ability to study and report results of study
6. Broad and mature interests
7. Social attitudes
Bloom’s Taxonomy (1956)
Knowledge Recall of specifics and generalizations; of methods and processes; and of pattern, structure, or setting.
Comprehension
Knows what is being communicated and can use the material or idea without necessarily relating it.
Application Use of abstractions in particular and concrete situations.
Analysis Make clear the relative hierarchy of ideas in a body of material or to make explicit the relations among the ideas or both.
Synthesis Assemble parts into a whole.
Evaluation Judgments about the value of material and methods used for particular purposes.
Bloom’s Revised Taxonomy (2001)
OLD NEWKnowledge Remember
Comprehension Understand
Application Apply
Analysis Analyze
Synthesis Evaluate
Evaluation Create
Krathwol and Anderson, 2001
Knowledge Dimension of Revised Bloom’s Taxonomy
• Factual knowledge
• Conceptual knowledge
• Procedural knowledge
• Metacognitive knowledge
(Krathwohl, 2002)
Survey of Enacted Curriculum, ELA Cognitive
LevelsRecall Provide facts, terms, definitions,
conventions; describe, etc.
Demonstrate/Explain
Follow instructions; give examples; etc.
Analyze/Investigate
Categorize, schematize; distinguish fact from opinion; make inferences, draw conclusions; etc.
Evaluate Determine relevance, coherence, logical, internal consistency; test conclusions; etc.
Generate/Create Integrate, dramatize; predict probable consequences; etc.
PISA: Three Clusters Representing Understanding
of Mathematics
Reproduction cluster
Knowledge of facts and common problem representation
Connection cluster
Applying problem solving to non-routine situations in familiar contexts
Reflection cluster
Planning solution strategies and implementing them in problem settings with more elements and more unfamiliar contexts.
PISA: Three Science Competencies
• Identify scientific issues
• Explain phenomena scientifically
• Use scientific evidence
Why another system??
Standards-based educatio
n
Need for alignment
Need for appropriate system to describe content
complexity
Part 1: Summary
• Content complexity is worth considering
• Many different language systems and frameworks have been developed to describe content complexity
• DOK was developed to best fit the needs of describing content complexity as expressed in learning expectations (standards)
• DOK was developed to inform alignment studies (of standards and assessments)
Complexity of Common Core Standards
Part 2: Depth-of-Knowledge System
Depth of Knowledge (Webb, 1997)
Level 1:
Recallrecall of a fact, information, or procedure, etc.
Level 2:
Skills/Conceptsuse information or conceptual knowledge, connecting ideas, etc.
Level 3:
Strategic thinkingrequires reasoning, developing a plan; may not have a single correct answer or approach, etc.
Level 4:
Extended thinkingoften an investigation or project; involves extended time spent on complex problems, etc.
Depth of Knowledge (Webb, 1997)
ReadingLevel 1:
Recall of a fact, recognize text feature, verbatim recall, etc.
Level 2:
Draw meaning from text using structure and features, comprehend, connect ideas, etc.
Level 3:
Reason, conduct analysis to make inferences, read critically to attest to internal logic, implied values, identify abstract connections between texts, etc.
Level 4:
Often an investigation or project; involves extended time spent on complex problems, etc.
Why DOK?
Rationale for 4 levels
Based on contemporary learning models
Takes into account multiple factors that contribute to content complexity
Recognizes inherent differences in the structure of different disciplines
Provides a common language that is a good fit for today’s education world
Which of these means about the same as the word gauge?
a.balance
b.measure
c.select
d. warn Grade 5
Grade 8
A triangle has 0 diagonals, a quadrilateral has 2 diagonals, a pentagon has 5 diagonals, and a hexagon has 9 diagonals. If the pattern continues, how many diagonals will a octagon have?
Sides 3 4 5 6
Diagonals
0 2 5 9
A. 11B. 14C. 18D. 20
26
Which of these conclusions is best supported by information from the passage? a. If a candidate meets the personal and educational
qualifications and is in fair physical shape, his or her chances of becoming an agent are very good.
b. Compared with other law enforcement agencies in the country, the F.B.I. has a low success rate for tracking down and apprehending suspected offenders.
c. The job of an agent is not for everyone; it takes someone with special training who is not afraid of danger and doesn’t mind being socially isolated at times.
d. The life of a federal investigator is not as interesting as most people think; agents spend most of their time working at desks.
121 1) 190 13 2) 200 32 3) 290 + 34 4) N
Grade 4
None of the above
a)
b)
c)
d)
It’s still a level 1
(AP Photo/Matt Rourke)
As of 2007, Marc Umile had recited (from memory) more digits of Pi (over 15,000) than any other North American.
COMPLEXITY
LOW HIGH
DIFFICULTY
LOW
HIGH
Levels of content complexity
NOT Levels of importance
NOT Levels of progression/sequence
Hess’ Cognitive Rigor Matrix
Bloom’s Revised Taxonomy of Cognitive ProcessDimensions
Webb’s Depth-of-Knowledge (DOK) Levels
Level 1Recall &
Reproduction
Level 2Skills &
Concepts
Level 3Strategic Thinking/ Reasoning
Level 4Extended Thinking
Remember
Understand
Apply
Analyze
Evaluate
Create
This isn’t quite how we
see it.
IMPORTANT: It’s more than just
verbs! (We didn’t m
ake this.)
Content complexity is
continuous and generally
decided by group analyses.
Part 2: Summary
• DOK provides a common language to describe content complexity in the context of today’s education world
• Difficulty and complexity are related but different
• There are several common misunderstandings and misrepresentations of DOK
• With practice, there’s an increase in consistency when applying DOK
Complexity of the Common Core Standards
Part 3: Analysis and Implications
Common Core Standards (CCS)
and DOK
What distribution of DOK do we see in the CCS?
Standards for Mathematical Practice
1. Make sense of problems and persevere in solving them.
2. Reason abstractly and quantitatively.3. Construct viable arguments and critique the
reasoning of others.4. Model with mathematics.5. Use appropriate tools strategically.6. Attend to precision.7. Look for and make use of structure.8. Look for and express regularity in repeated
reasoning.
1. Thinking and reasoning
2. Argumentation
3. Communication
4. Modeling
5. Problem posing and solving
6. Representation
7. Using symbolic, formal and technical language and operations
8. Use of aids and tools
Mathematical Competency Used for Pisa
Critical Areas
Domains
Clusters
Standards
Structure of CCS Mathematics
Grade 7 CCS Mathematics StandardsLevel Description DOK
7.RP Ratios and Proportional Relationships7.RP.1 Compute unit rates associated with ratios of fractions, including
ratios of lengths, areas and other quantities measured in like or different units. For example, if a person walks 1/2 mile in each 1/4 hour, compute the unit rate as the complex fraction 1/2/1/4 miles per hour, equivalently 2 miles per hour.
7.RP.2 Recognize and represent proportional relationships between quantities. a. Decide whether two quantities are in a proportional relationship, e.g., by testing for equivalent ratios in a table or graphing on a coordinate plane and observing whether the graph is a straight line through the origin. b. Identify the constant of proportionality (unit rate) in tables, graphs, equations, diagrams, and verbal descriptions of proportional relationships. c. Represent proportional relationships by equations. d. Explain what a point (x, y) on the graph of a proportional relationship means in terms of the situation, with special attention to the points (0, 0) and (1, r) where r is the unit rate.
7.RP.3 Use proportional relationships to solve multistep ratio and percent problems. Examples: simple interest, tax, markups and markdowns, gratuities and commissions, fees, percent increase and decrease, percent error.
41
Mathematics 7.RP.2
Recognize and represent proportional relationships
a. Decide whether two quantities are in a proportional relationship
b. Identify the constant of proportionality (unit rate)
c. Represent proportional relationships by equations.
d. Explain the meaning of a point (x, y) on the graph of a proportional relationship
Depth of Knowledge of All Mathematics Common Core State Standards
Grade Total DOK Level
1 2 3 4 3 25 24 24 3 0 4 28 28 16 5 0 5 26 26 18 3 0 6 29 29 20 2 0 7 24 18 22 8 0 8 28 26 25 9 0
Number and Quantity 27 27 15 0 0 Algebra 27 26 21 7 0 Functions 28 27 24 4 0 Geometry 43 24 36 19 1 Statistics and Probability 31 27 29 7 0 TOTAL 316 282 250 67 1 Percentage of Total Standards at DOK Level (Standards may cover a range of DOK levels)
89% 79% 21% < 1%
Sato, E, Lagunoff, R, and Worth, P. (March 4, 2011) SMARTER Balanced Assessment Consortium Common Core State Standards analysis: Eligible content for the summative assessment. Final report submitted to the SMARTER Balanced Assessment Consortium. San Francisco, CA: West Ed. http://www.k12.wa.us/SMARTER/pubdocs/SBAC_CCSS_Eligible_Content_Final_Report_030411.pdf downloaded October 17, 2011.
43
44
Florida Analysis of DOK Levels of CCSS
Mathematics for High School Total #
Standards
DOK Level
1 2 3 4
Number and Quantity 27 13 14
Functions 28 2 23 3 Algebra 27 7 14 6 Geometry 43 5 24 14 >1Statistics and Probability
31 3 27 1
Total 156 30 102 24 Percent 19% 65% 15% Smarter Balanced % 84% 80% 24% 1%
Reading Standards for Literature K–5
Reading Standards for Informational Text K–5
Reading Standards: Foundational Skills (K–5)
Writing Standards K–5
Speaking and Listening Standards K–5
Language Standards K–5
K-5 Language Arts Standards
45
CCS Grade 7 Reading (Inf) & Language Standards
(p. 1)
46
Level Description DOKRI Reading Standards for Informational TextRI.7.KID Key Ideas and DetailsRI.7.KID.1
Cite several pieces of textual evidence to support analysis of what the text says explicitly as well as inferences drawn from the text.
RI.7.KID.2
Determine two or more central ideas in a text and analyze their development over the course of the text; provide an objective summary of the text.
RI.7.KID.3
Analyze the interactions between individuals, events, and ideas in a text (e.g., how ideas influence individuals or events, or how individuals influence ideas or events).
RI.7.KID.1
Cite several pieces of evidence
to support analysis of what the text says explicitly
as well as inferences drawn from the
text.47
Depth of Knowledge of Eligible ELA Common Core State Standards
Grade Total DOK Level
1 2 3 4 3 35 20 27 24 7 4 35 18 26 28 9 5 35 16 30 29 12 6 36 13 25 31 15 7 36 14 25 31 18 8 36 13 25 31 18
9–10 36 13 25 31 18 11–12 36 13 25 31 21
TOTAL 285 120 208 236 118 Percentage of Total Standards at DOK Level (Standards may cover a range of DOK levels) 42% 73% 83% 41%
Sato, E, Lagunoff, R, and Worth, P. (March 4, 2011) SMARTER Balanced Assessment Consortium Common Core State Standards analysis: Eligible content for the summative assessment. Final report submitted to the SMARTER Balanced Assessment Consortium. San Francisco, CA: West Ed. http://www.k12.wa.us/SMARTER/pubdocs/SBAC_CCSS_Eligible_Content_Final_Report_030411.pdf downloaded October 17, 2011.
48
Florida Analysis of CCS DOK LevelsEnglish Language Arts Grades K-2
K-2Areas
Total #Standards
DOK Level
1 2 3 4
Read Lit 27 4 19 4 0
Read Inform
30 3 20 7 0
Read Found
10 7 3 0 0
Writing 21 0 9 9 3
Spk List 18 3 11 4 0
Language 16 6 7 3 0
Total 122 23 69 27 3
Percent 19% 56% 22% 2%49
Florida Analysis of CCS DOK LevelsEnglish Language Arts Grades 3-5
Grades 3-5Areas
Total #Standards
DOK Level
1 2 3 4
Read Lit 27 0 15 12 0Read Inform 30 0 15 14 1
Read Found 6 3 3 0 0Writing 29 0 3 23 3Spk List 18 1 7 10 0Language 18 6 7 5 0Total 128 10 50 64 4Percent 8% 39% 50% 3%
50
Florida Analysis of CCS DOK LevelsEnglish Language Arts Grades 6-8
Grades 6-8Areas
Total #Standards
DOK Level
1 2 3 4
Read Lit 27 0 8 19 0Read Inform 30 0 11 19 0
Writing 30 0 4 20 6Spk List 18 0 5 13 0Language 18 0 14 4 0Total 123 0 42 75 6Percent 0% 34% 61% 5%
51
Florida Analysis of CCS DOK LevelsEnglish Language Arts Grades 9-12
Grades 6-8Areas
Total #Standards
DOK Level
1 2 3 4
Read Lit 18 0 3 15 0Read Inform 20 0 5 15 0Writing 20 0 1 11 8Spk List 12 0 2 10 0Language 12 0 7 5 0Read History 20 0 5 15 0Read Sci Tech
200 8 12 0
Write HST 18 0 1 9 8Total 140 0 32 92 16Percent 23% 66% 11%
52
1. Clarify important learning outcomes and priorities
2. Plan and deliver effective instruction
3. Select and design appropriate assessments
4. Align standards, instruction, and assessments.
Strategies For Applying DOK To Help Improve Learning
53
Deconstruct standards and expectations
Outline learning trajectory
Map to given time
Anticipate learning struggles
Clarify Important Learning Outcomes and Priorities
54
2.OA.3. Determine whether a group of objects (up to 20) has an odd or even number of members, e.g., by pairing objects or counting them by 2s; write an equation to express an even number as a sum of two equal addends. (Grade 2 Operations and Algebraic Thinking)
DOK 1:
Define: odd group
even equation
number equal
addend sum
Count to 100
Count by 2
Deconstruction of a Standard
55
DOK 1 (continued):
Understand the number line
Place numbers on a number line
Use skip counting
Pair objects
Compare numbers
Identify a number as odd or even
Identify an equation
DOK 2:
Write an equation
Deconstruction (continued)
56
Assessment Items: Factors that Can Influence DOK
• Question• DOK 1 What, where, find, compute• DOK 2 Why or why not, compare, organize• DOK 3 Imply, infer, project, generalize
• Context• DOK 1 None• DOK 2 Familiar and relevant• DOK 3 Unfamiliar, multiple
• Application• DOK 1 Routine or none• DOK 2 Typical, routine, basic• DOK 3-4 Fix, verify, and justify; hypothetical
But REMEMBER: The DOK of an
item depends on the main
challenge for a typical student.
How Does Item Type Influence DOK?
• multiple choice
• constructed response
• cloze
• hot text
• fill-in-the-blank
• sequencing
• drag-and-drop
• multi-part itemBut REMEMBER: The DOK of an
item depends on the main
challenge for a typical student.
Practice in Using DOK
59
Look at the drawing. The numbers alongside each column and row are the total values of the symbols within each column and row. What should replace the question mark?
Grade 8:
A. 23
B. 25
C. 28
D. 30
E. 32
Is there a match among
1. Complexity of outcome not process
2. Challenge of instructional activity
3. Necessary performance on assessment
Check on Alignment
61
Summary and Takeaway Themes
• DOK is a valuable common language, with ongoing training and discussion
• Complexity and difficulty are related but different
• It’s critical to consider alignment
• Consider central performance in objectives
• Consider item type (MC, CR, cloze, etc.)
• Consider appropriate distribution by DOK level
Website
webbalign.org
Alignment Tool
watv2.wceruw.org
top related