ctb cadds sally valenzuela director, publishing strategic initiatives ctb/mcgraw-hill
DESCRIPTION
CTB CADDS Sally Valenzuela Director, Publishing Strategic Initiatives CTB/McGraw-Hill. Abstract. Evidence-based item development approaches require item authors to make explicit how items provide validity evidence to support claims. - PowerPoint PPT PresentationTRANSCRIPT
CTB CADDSSally ValenzuelaDirector, Publishing Strategic InitiativesCTB/McGraw-Hill
2
Abstract
Evidence-based item development approaches require item authors to make explicit how items provide validity evidence to support claims.
Advances in technology require increased consideration of accessibility and interoperability during item authoring.
Refined item development processes are needed to meet these requirements.
3
CADDS
Serves as a bridge between standards and assessment
Provides opportunities for systematic unpacking of standards for item development
Documents steps in reaching assessment goals Maintains focus at all steps on evidence to support
interpretations and uses of test information
4
CADDS
Details information about the content to be assessed
Describes item types that will be used Uses detailed specifications for each item/task Increases direction for item writers
5
CADDS
1. Define the intended inferences and decisions to be based on test scores.
2. Define the achievement construct.3. Draft performance level descriptors (expectations of
students).4. Define the evidence to be elicited by the item pool.5. Complete item writer assignments to meet the item
pool specification requirements.6. Complete item creation (authoring and editing).7. Field test assessment items and tasks.8. Implement the operational test.
6
INFERENCE about students
(Step 1)
Achievement Construct (Step 2)
Assessment item/task
creation (Step 6)
Item Writer Assignments
(Step 5)
Performance Level
Descriptors (Step 3)
Item Pool Evidence(Step 4)
Pilot, field, operational
testing (Step 7/8)
DESIGNDEVELOPMENT
7
CADDS
1.Define intended inferences and uses of the assessment data.
• Adoption of CCSS has created need for transition assessments.
• Test designs are incorporating new item types.• Reporting requirements are changing.• “Alignment” issues abound.
8
CADDS
2.Define the test construct(s) that will become assessment targets.
• What cognitive tasks are required by the standard?• How do we consider students progression of learning?• What is the instructional context for a given standard?• How do we incorporate performance levels?
9
CADDS
3.Develop initial proficiency level descriptors to guide development and interpretation of test scores.
• Identify source(s) of PLDS.• Define the role of PLDs in item specifications and authoring.
10
CADDS
4.Define the evidence.1. Item pool or test blueprint2. Specifications3. Item/task templates4. Instructions to item writers
5.Develop items and performance tasks based on specifications.1. specifications templates2. cognitive task frameworks3. tagging for accessibility and interoperability
11
CADDS
6.Refine items and tasks through collaborative review by stakeholders.
7.Field test items and tasks in appropriate small- or large-scale settings.
8.Implement the operational test and continue the design and specification validation process.
12
Implementation
Expansion of item specifications template
Performance description Rules for source materialsPerformance level descriptors Rules for item/task problem
Grade level placement Rules for response requirements
Assessment targets/standards Administration requirements21st century skills Accessibility requirementsCognitive rigor Administrator directionsProblem/processing type Technology requirementsPerformance description Rules for source materials
13
Implementation Articulation of cognitive tasks during item
development Cognitive Task Frameworks
• Traditional (DOK, Bloom’s)• Within CCSS (Conley)
– Mathematical Practice Standards– Selected ELA standards
• Cognitive Rigor Matrix (Hess)
14
Item Authoring
Clearly defined elements for authors• construct evidence requirements• parameters for student responses• options for item type, cognitive demand, other variable
factors Articulated focus on evidence, validity and
accessibility
15
CCSS StandardModeling with Geometry G-MGApply Geometric Concepts in Modeling Situations.
Model with Mathematics
Make sense of problems.
Attend to precision.
Item 1
Item 2
Item 3
Above Proficient
Proficient
Basic
Item 1
Item 2
Item 3
Above Proficient
Proficient
Basic
Item 1
Item 2
Item 3
Above Proficient
Proficient
Basic
16
Item Development Implications
Explicit articulation of evidence to support claims Rigorous specifications development More direction for item writers Clearer distinction among item attributes Refinement of item development plans