curriculum-based measurement (cbm) for special educators: workshop agenda

Response to Intervention

www.interventioncentral.org

Curriculum-Based Measurement & Special Educators: Assessing Student Foundation Academic Skills

Jim Wrightwww.interventioncentral.org



Curriculum-Based Measurement (CBM) for Special Educators: Workshop Agenda

Curriculum-Based Measurement: An Introduction

Writing IEP Goals for CBM

CBM Reading, Math Computation, & Writing

Charting & Evaluating Data Using CBM

CBM Web Resources



Curriculum-Based Measurement: An Introduction


www.interventioncentral.org 4

Curriculum-Based Measurement: Advantages as a Set of Tools to Monitor RTI/Academic Cases

• Aligns with curriculum-goals and materials• Is reliable and valid (has ‘technical adequacy’) • Is criterion-referenced: sets specific performance levels for specific

tasks• Uses standard procedures to prepare materials, administer, and

score• Samples student performance to give objective, observable ‘low-

inference’ information about student performance • Has decision rules to help educators to interpret student data and

make appropriate instructional decisions• Is efficient to implement in schools (e.g., training can be done quickly;

the measures are brief and feasible for classrooms, etc.)• Provides data that can be converted into visual displays for ease of

communicationSource: Hosp, M.K., Hosp, J. L., & Howell, K. W. (2007). The ABCs of CBM. New York: Guilford.



• Reading fluency• Reading comprehension• Math computation• Writing• Spelling• Phonemic awareness skills• Early math skills

Among other areas, CBM Techniques have been developed to assess:



CBM Student Reading Samples: What Difference Does Fluency Make?

• 3rd Grade: 19 Words Per Minute





Measuring General vs. Specific Academic Outcomes

General Outcome Measures…• Track the student’s increasing proficiency on

general curriculum goals such as reading fluency. Example: CBM-Oral Reading Fluency (Hintz et al., 2006).

• Are most useful for longer-term measurement (e.g., to set and track IEP goals over the timespan of a school year).

Sources: Burns, M. K., & Gibbons, K. A. (2008). Implementing response-to-intervention in elementary and secondary schools: Procedures to assure scientific-based practices. New York: Routledge. Hintz, J. M., Christ, T. J., & Methe, S. A. (2006). Curriculum-based assessment. Psychology in the Schools, 43, 45-56.



Measuring General vs. Specific Academic Outcomes

Specific Sub-Skill Mastery Measures…• Track short-term student academic progress

with clear criteria for mastery (Burns & Gibbons, 2008). Example: Letter Identification.

• Are helpful in assessing whether the student has acquired short-term skills whose acquisition may require weeks rather than months.

Sources: Burns, M. K., & Gibbons, K. A. (2008). Implementing response-to-intervention in elementary and secondary schools: Procedures to assure scientific-based practices. New York: Routledge. Hintz, J. M., Christ, T. J., & Methe, S. A. (2006). Curriculum-based assessment. Psychology in the Schools, 43, 45-56.



Problem: IEPs Often Use ‘Mastery Measurement’• The student is expected to master a series of ‘short-term’

objectives• The methods used to track student progress change as the

student masters various objectives• These informal ‘mastery tests’ have technical limitations:

1. They fail to provide information about students’ maintenance of mastered skills

2. They are not in equivalent ‘units’--so different measures cannot be compared directly with one another

3. There is little or no information about the reliability and validity of these ‘informal’ measures

4. They are unable to show GENERAL student progress toward a long-term goal in mastering academic skills

Source: Stecker, P. M., & Hosp, M. (2005). Applications of Progress-Monitoring to IEP and Program Development. Presentation at the 2005 CEC Annual Conference & Expo, Baltimore, MD.



Solution: CBM Measures Can Serve as Valid, Reliable ‘Long-Term’ Academic Measurement Tools

• Various types of Curriculum-Based Measurement can:– Track student progress toward a fixed year-end goal for

improvement– Have been demonstrated to be a valid and reliable measure of

student fluency in targeted basic academic skills– are timed and uses standard directions and scoring criteria—so

each probe administered is an equivalent ‘unit’ and can be compared with past and future student probes. Results can also be charted for visual feedback

– Allows the IEP to be less cluttered with assessment methods, thus making the document more manageable

Source: Stecker, P. M., & Hosp, M. (2005). Applications of Progress-Monitoring to IEP and Program Development. Presentation at the 2005 CEC Annual Conference & Expo, Baltimore, MD.



‘Elbow Group’ Activity: How Might CBM Measures Be Useful in Your Practice as a Special Educator?

In your ‘elbow groups’:• Discuss the general concept

of CBMs and how they might have a use in monitoring students with whom you work.

• In particular, how can such measures be useful to assess IEP goals?

• Be prepared to share your discussion points with the larger group.

• Aligns with curriculum-goals and materials• Is reliable and valid (has ‘technical adequacy’) • Is criterion-referenced: sets specific

performance levels for specific tasks• Uses standard procedures to prepare materials,

administer, and score• Samples student performance to give objective,

observable ‘low-inference’ information about student performance

• Has decision rules to help educators to interpret student data and make appropriate instructional decisions

• Is efficient to implement in schools (e.g., training can be done quickly; the measures are brief and feasible for classrooms, etc.)

• Provides data that can be converted into visual displays for ease of communication



Curriculum-Based Measurement: Oral Reading Fluency

Jim Wrightwww.interventioncentral.org



In your packet, for CBM reading/literacy…

Here are key resource and activity materials:• Oral Reading Fluency (ORF): Directions: pp. 2-7• Research-Based Norms: ORF: p. 17• CBM Literacy Measures: Listing: pp. 19-21• ORF Record Form: pp. 41-42

13



Reading: 5 Big Ideas• Phonemic Awareness• Alphabetics• Fluency with Text/General Outcome Measure• Vocabulary• Comprehension



Five Big Ideas in Reading• “Phonemic Awareness: The ability to hear and

manipulate sounds in words. • Alphabetic Principle: The ability to associate sounds

with letters and use these sounds to form words. • Fluency with Text: The effortless, automatic ability

to read words in connected text. • Vocabulary: The ability to understand (receptive) and

use (expressive) words to acquire and convey meaning. • Comprehension: The complex cognitive process

involving the intentional interaction between reader and text to convey meaning.”

SOURCE: University of Oregon: http://reading.uoregon.edu/big_ideas/trial_bi_index.php



http://www.nichd.nih.gov/publications/nrppubskey.cfm



NRP Conclusions Regarding Importance of Oral Reading Fluency:

“An extensive review of the literature indicates that classroom practices that encourage repeated oral reading with feedback and guidance leads to meaningful improvements in reading expertise for students—for good readers as well as those who are experiencing difficulties.”-p. 3-3



CBM Reading Probes: Preparation



CBM Reading : Preparation



CBM Reading Assessment: Preparation• Decide on ‘measurement pool’ (e.g., basal

reading series, literature selections sorted according to readability)

• Prepare examiner and student copies of passages

• Select passages randomly from larger library when administering to student



CBM Reading Measurement Pool: Silver Burdett & Ginn (1989)

Book 1: All Through the Town.Grade 1Book 2: Out Came the Sun…..Grade 1Book 3: Morning Bells….……..Grade 1Book 4: Make A Wish…………Grade 1Book 5: A New Day……………Grade 1Book 6: Garden Gates………..Grade 2Book 7: Going Places…………Grade 2Book 8: Castles of Sand………Grade 3Book 9: On the Horizon……….Grade 3Book 10: Silver Secrets……….Grade 4Book 11: Dream Chasers……..Grade 5Book 12: Wind by the Sea…….Grade 6Wind by the Sea…….Grade 6



CBM Reading Probes: Example



CBM Reading Probes: Administration



CBM Reading : Administration



CBM Reading Probes: Administration

• 3 passages selected at random from probe collection

• Stopwatch• Pencil, pen, or marker• Quiet, non-distracting location

Materials needed:



CBM Reading Assessment: Administration

• Sit at table next to or across from student• Read off standardized directions• Start stopwatch after (a) student reads first

word, or (b) examiner provides first word



CBM Reading Assessment: Administration

• Supply correct word if student hesitates for longer than 3 seconds

• Mark errors on examiner passage• Mark student stopping point in passage (“]”)

at end of 60 seconds• Tell student to stop reading



CBM Reading Probes: Scoring



CBM Reading : Scoring



CBM Reading Assessment: Scoring

Words are counted as correct if . . .

• the student repeats a correctly read word

• the student self-corrects within 3 seconds

• variant pronunciation of a word is due to dialectical differences or speech articulation issues




Words are counted as incorrect if they are. . .

• mispronunciations

• substitutions (e.g., home for house)

• omissions

• hesitations of greater than 3 seconds

• word transpositions




Words read aloud are ignored if. . .

• the student inserts them into the text



CBM Reading Assessment: Computing Correctly Read Words

• subtracting number of errors (E) from

• total read words (TRW) during timed minute --words read up to end bracket in passage

Number of correctly read words (CRW) is calculated by:



CBM Reading Assessment: Computing Correctly Read Words

• TRW=74TRW=74• Errors=5Errors=5• CRW=69CRW=69



CBM Reading Assessment: Omitted Text Adjustment:

1. Count up the number of words omitted in a segment of a passage

2. Subtract all but one of those omitted words from the total word count (TRW)

3. Repeat for additional omitted passages

4. Count each omission as single error when calculating correctly read words (CRW)



CBM Reading Assessment: Scoring Example/Omitted Text

• TRW=74TRW=74• Omitted Omitted

Words=10Words=10• New TRW=64New TRW=64• Errors=6Errors=6• CRW=58CRW=58



CBM Reading Assessment: Selecting Median Values

Franklin Jones Mrs. Larrossa

3-2 SB&G: Bk 9

9/23 49 Lvl 4-1 Probe 14 45 92%

9/23 64 Lvl 4-1 Probe 23 61 95%

9/23 42 Lvl 4-1 Probe 32 40 95%



CBM: Math Computation



In your packet, for CBM Math…

Here are key resource and activity materials:• CBM Math Computation: Directions: pp. 7-11• Research Norms: Math Computation: p. 17• Early Math Fluency: Listing: p. 24• Math Computation & Concepts/Applications:

Listing: p. 30-31

39



CBM Math Computation Probes: Preparation



CBM Math Computation Sample Goals• Addition: Add two one-digit numbers: sums to 18• Addition: Add 3-digit to 3-digit with regrouping

from ones column only

• Subtraction: Subtract 1-digit from 2-digit with no Subtract 1-digit from 2-digit with no regroupingregrouping

• Subtraction: Subtract 2-digit from 3-digit with Subtract 2-digit from 3-digit with regrouping from ones and tens columnsregrouping from ones and tens columns

• Multiplication: Multiply 2-digit by 2-digit-no Multiply 2-digit by 2-digit-no regrouping regrouping

• Multiplication: Multiply 2-digit by 2-digit with Multiply 2-digit by 2-digit with regrouping regrouping



CBM Math Computation Assessment: Preparation

• Select either single-skill or multiple-skill math probe format.

• Create student math computation worksheet (including enough problems to keep most students busy for 2 minutes)

• Create answer key




• Advantage of single-skill probes:– Can yield a more ‘pure’ measure of student’s

computational fluency on a particular problem type




• Advantage of multiple-skill probes:– Allow examiner to gauge student’s adaptability

between problem types (e.g., distinguishing operation signs for addition, multiplication problems)

– Useful for including previously learned computation problems to ensure that students retain knowledge.



CBM Math Computation Probes: Administration



CBM Math Computation Probes: Scoring



CBM Math Computation Assessment: Scoring

Unlike more traditional methods for scoring math computation problems, CBM gives the student credit for each correct digit in the answer. This approach to scoring is more sensitive to short-term student gains and acknowledges the child’s partial competencies in math.



Math Computation: Scoring Examples

2 CDs3 CDs

12 CDs



Math Computation: Scoring

Placeholders Are CountedNumbers Above Line Are Not Counted



Question: How can a school use CBM Math Computation probes if students are encouraged to use one of several methods to solve a computation problem—and have no fixed algorithm?

Answer: Students should know their ‘math facts’ automatically. Therefore, students can be given math computation probes to assess the speed and fluency of basic math facts—even if their curriculum encourages a variety of methods for solving math computation problems.



The application to create CBM Early Math Fluency probes online

http://www.interventioncentral.org/php/numberfly/numberfly.php



Examples of Early Math Fluency (Number Sense) CBM Probes

Quantity Discrimination

Missing Number

Number Identification

Sources: Clarke, B., & Shinn, M. (2004). A preliminary investigation into the identification and development of early mathematics curriculum-based measurement. School Psychology Review, 33, 234–248.

Chard, D. J., Clarke, B., Baker, S., Otterstedt, J., Braun, D., & Katz, R. (2005). Using measures of number sense to screen for difficulties in mathematics: Preliminary findings. Assessment For Effective Intervention, 30(2), 3-14



Curriculum-Based Measurement: Writing

Jim Wright




In your packet, for CBM Writing…

Here are key resource and activity materials:• CBM Writing: Directions: pp. 12-16• Research Norms: Writing: Total Words Written:

p. 17

56



CBM Writing: Preparation



CBM Writing Assessment: Preparation

• Select a story starter• Create a CBM writing

probe: a lined sheet with the story starter at the top



CBM Writing Assessment: Preparation

• Create or collect story starters that students will find motivating to write about. (And toss out starters that don’t inspire much enthusiasm!)

• Avoid story starters that allow students simply to generate long lists: e.g., “What I want for my birthday is…”

Story Starter Tips:



CBM Writing Probes: Administration



CBM Writing Probes: Scoring



CBM Writing Assessment: Scoring

I woud drink water from the ocean and I woud eat the fruit off of the trees. Then I woud bilit a house out of trees, and I woud gather firewood to stay warm. I woud try and fix my boat in my spare time.

Total Words:

Total Words = 45Total Words = 45




Total Words: Useful for tracking a student’s fluency in writing (irrespective of spelling, punctuation, etc.)




CBM Writing Assessment: ScoringCorrectly Spelled Words:

Correctly Spelled Words = 39Correctly Spelled Words = 39




Correctly Spelled Words: Permits teachers to (a) monitor student spelling skills in context of writing assignments, and (b) track student vocabulary usage.




Correct Writing Sequences: Most global CBM measure. Looks at quality of writing in context.





Correct Writing Sequences:

Correct Writing Sequences = 37Correct Writing Sequences = 37



Team Activity: How Can Teachers Use CBM-Writing in Their Classrooms?At your table:

• Discuss the CBM-Writing assessment methods presented in this workshop.

• What are the strengths of CBM-Writing as a classroom assessment tool?

• What other methods might teachers use to supplement CBM-Writing to gather more complete information about students’ writing skills?



Creating Progress-Monitoring Charts for CBM Data



In your packet, for CBM Charting…

Here are key resource and activity materials:• CBM Charting Example: pp. 43-47• CBM ORF Charting Form: p. 48

74



Charting Example: Jared M.

Background. Your Teacher Support Team has completed a CBM survey-level screening in reading for Jared M., a 4th grader. According to his teacher, Jared reads at the beginning 2nd-grade level. An initial RTI Team meeting is held on Monday, January 20 th. At that meeting, an intervention is designed … Your team schedules a follow-up RTI Team meeting for Monday, March 10th, about six instructional weeks from the date of the initial meeting.




• 1. Survey-Level Assessment. On Jared’s attached CBM Student Record Form, review the Survey-Level assessment results. For each level of CBM probe administered, circle the median Correctly Read Words (CRWs), Errors (E), and Percentage of Correctly Read Words (%CRWs). Consult Table 1 on the Record Form to identify the student’s Mastery, Instructional, and Frustration levels of reading.



Step 1: Survey-Level Assessment

Example: JaredM

ASTERYINSTRUCT.

INSTRUCT.FRUSTR.




• 2. Set up the graph. At the top of your monitoring graph, put in these date-spans for each of the instructional weeks during which Jared will be monitored:

Baseline: 1/13-1/17

Week 4: 2/10-2/14 Week 8: 3/17-3/21 Week 12: 4/14-4/18

Week 1: 1/20-1/24 Week 5: 2/24-2/28 Week 9: 3/24-3/28

Week 2: 1/27-1/31 Week 6: 3/3-3/7 Week 10: 3/31-4/4

Week 3: 2/3-2/7 Week 7: 3/10-3/14 Week 11: 4/7-4/11



Jared: Recording Start and End Dates for Each Week



Baseline

Intervention: Week 1

Intervention: Week 2





1 131 17

1 201 24

1 271 31

2 32 7

2 102 14

2 242 28

3 33 7

3 103 14

3 173 21

3 243 28

3 314 4

4 74 11

4 144 18



Charting Example: Jared M.• 3. Determine & chart the

student’s baseline reading rate. On the Record Form, review Jared’s Baseline assessment information.

• Notice that the ‘Book/Reading Level’ is not filled in for the Baseline observations. Find the highest instructional reading level that the student attained on the Survey-Level assessment. Write that book level in the appropriate blank in the Baseline observations.




3. 3. Determine & chart the student’s baseline reading rate (Cont.)

• Circle the median CRW and E for each of the Baseline observations.

• On the progress-monitoring graph, chart the median CRWs and Es for all 3 observations.

• Circle this middle Baseline datapoint on your chart.



Jared: Chart Baseline Data & Select ‘Baseline Rdng Rate’

1 131 17

1 201 24

1 271 31

2 32 7

2 102 14

2 242 28

3 33 7

3 103 14

3 173 21

3 243 28

3 314 4

4 74 11

4 144 18

70

67

72Baseline Reading Rate=70



Charting Example: Jared M.• 4. Set a performance goal. To compute Jared’s

performance goal in reading:– Use Table 2 on the Record Form to identify the rate of

progress that Jared should make each week in goal-level (3rd-Grade) reading material.

– You will recall that your RTI Team has decided to monitor Jared’s reading for six weeks before holding a follow-up meeting. To compute how much Jared’s reading rate should increase in that time, multiply his expected weekly progress by the number of weeks that he will be monitored.

– Add Jared’s expected reading progress to his baseline reading rate. This combined figure is Jared’s reading goal.



Step 2: Compute a Student Reading Goal

Example: Jared

3rd Gr-Book 2

70

1 20 07

3 7 07

6

1.5

9

79



Charting Example: Jared M.5. Plot the ‘Aim-Line’. To graph a 6-week ‘aim-line’: • Draw a vertical dividing line (‘start-line’) at the point where the

intervention will begin (start of Week 1). • Draw a second dividing line on the graph (‘end-line’) that

marks the conclusion of six weeks of monitoring (end of Week 6).

• On the start-line, mark an ‘X’ at the point that is equal to the value of your circled baseline datapoint.

• Mark Jared’s reading goal with an ‘X’ at the appropriate spot on the end-line.

• Now draw a straight line between the start-line and end-line ‘X’s. This is your chart’s aim-line.



7970

Jared: Plot the CBM ‘Aim-Line’

1 131 17

1 201 24

1 271 31

2 32 7

2 102 14

2 242 28

3 33 7

3 103 14

3 173 21

3 243 28

3 314 4

4 74 11

4 144 18

XX

Aim-Line




6. Plot Jared’s progress-monitoring data. Review Jared’s CBM data for the first six weeks of progress-monitoring.

• Circle the median CRWs and Es. • Plot these values on the chart. What conclusions do

you draw from the chart? • Based on these data, should the RTI Team

recommend changing Jared’s intervention? Keep it in place with no changes? Why?



Example: Jared

Step 4: Progress-Monitoring



Instructional

Jared: Intervention Phase 1: Weeks 1-6

1 131 17

1 201 24

1 271 31

2 32 7

2 102 14

2 242 28

3 33 7

3 103 14

3 173 21

3 243 28

3 314 4

4 74 11

4 144 18

XX

Mastery



Writing IEP Goals for CBM



In your packet, for CBM IEP goals…

Here are key resource and activity materials:• Format for Writing CBM IEP Goals: p. 31

98



CBM: Elements of IEP Goals

An IEP goal written for curriculum-based measurement has three components:

1. Conditions under which the data is collected2. Student behavior to be directly observed &

measured3. Criterion for judging the student’s performance

99



Writing CBM Goals in Student IEPs (Wright, 1992)

Source: Wright, J. (1992). Curriculum-based measurement: A manual for teachers. Retrieved on September 4, 2008, from http://www.jimwrightonline.com/pdfdocs/cbaManual.pdf



IEP Goals for CBA/CBM: READING

Reading In [number of weeks until Annual Review], when given a randomly selected passage from [level and name of reading series] for 1 minute

Student will read aloud

At [number] correctly read words with no more than [number] decoding errors.

Conditions BehaviorPerformance Criterion



IEP Goals for CBA/CBM: Written Expression

Written Expression

In [number of weeks until Annual Review], when given a story starter or topic sentence and 3 minutes in which to write

Student will write

A total of:[number] of words or[number] of correctly spelled words or [number] of correct word/writing sequences




IEP Goals for CBA/CBM: Spelling

Spelling In [number of weeks until Annual Review], when dictated randomly selected words from [level and name of spelling series or description of spelling word list] for 2 minutes

Student will write

[Number of correct letter sequences]




The Special Educator & Data Collection: Expectations



Bringing Special Education IEPs into Alignment with RTI: Expectations

At future Annual Review meetings, many districts across the nation have decided to apply the same standards when evaluating the quality of the special education programs of IEP students as those used under RTI to judge the quality of general education.

In particular, special educators will be asked to provide a description of how the student program on the IEP is delivered in a manner consistent with RTI principles.

107



Bringing Special Education IEPs into Alignment with RTI: Expectations (Cont.)

At the Annual Review, the special education teacher will be expected to demonstrate how he or she:

1. Defined the student academic or behavioral challenge in clear, specific, measurable terms.

2. Selected an evidence-based intervention practice or program to address the identified student concern.

3. Selected one or more methods to assess the student’s progress during the intervention; calculated the student’s baseline performance level; set a goal for improvement.

4. Collected progress-monitoring data on the student at least weekly to judge whether the intervention was effective.

5. Collected information on the integrity, or quality, with which the intervention was implemented.

108



Bringing Special Education IEPs into Alignment with RTI: Expectations (Cont.)

At the Annual Review, the special education teacher will be expected to demonstrate how he or she:

1. Defined the student academic or behavioral challenge in clear, specific, measurable terms.

2. Selected an evidence-based intervention practice or program to address the identified student concern.

3. Selected one or more methods to assess the student’s progress during the intervention; calculated the student’s baseline performance level; set a goal for improvement.

4. Collected progress-monitoring data on the student at least weekly to judge whether the intervention was effective.

5. Collected information on the integrity, or quality, with which the intervention was implemented.

109

At your tables:• Review the expectations for

using RTI principles to create quality academic or behavioral programs for students with IEPs-with particular attention to items 3 & 4.

• What supports will you need as a special educator to attain these goals in your own practice?

• Be prepared to report out.

curriculum-based measurement (cbm) for special educators: workshop agenda

Documents