Monitoring the Progress of Special Education

Students

The Power of Progress Monitoring:What ALL special educators need to know and be able to do

Agenda for Today Homework Review & Standards Selection Intervention Demos –Matthew & Joy Objectives & Big Idea Tools for Progress Monitoring

Curriculum Based Measures (CBMs) Other Standardized Norm-Referenced Tools Rubrics & Scoring Guides

Partner Progress Monitoring & Charting Practice Goal Writing Using Data Decision Rules

Learning Objectives

Students will draw comparisons between the Common Core and Colorado Academic Standards

Students will administer and chart two different Curriculum Based Measures (Maze & CBM Math) to monitor student progress over time.

Students will plot results using free charting tools

Students will apply instructional decisions to the results obtained through their progress monitoring.

Clock PartnersActivity 1

Homework Review

David Tower’s Goals

Let’s Talk – What do you suppose David’s mom is most concerned about? What is your biggest concern for David?

Specific Target Behaviors: ______________________________ Setting Event Strategies – Reduce impact Antecedent Strategies - Decrease likelihood of behavior Behavior Teaching Strategies - Increase replacement

behavior Reinforcement Strategies – What is attained or avoided?

Suggested interventions: _______________________________

Now, What about Standards?

Comparing Common Core and Colorado Academic Standards

Compare the kindergarten Common Core standard for Foundational Reading Skills on the next slide with its comparable Colorado Academic Standard under Reading, Writing and Communicating Standard 1 - Oral Expression and Listening:

Activity 2

9 o’clock partner

Research-based Intervention Demo 1

Matthew – Understanding Shape

Research-based Intervention Demo 2

Joy – Reading Comprehension

What is Progress Monitoring?

An on-going, systematic approach to gathering academic and behavioral data.

Green Board Quiz: Why Monitor Progress?

Evaluate response to intervention Encourage data-based decision-making Measure learning outcomes on a frequent

basis Help schools establish more effective

programs for children who have not benefited from previous programming.

In other words, it tells us if our interventions are working

Activity 3

A Good Progress Monitoring Tool…

• is administered frequently

• is easy and quick

• has multiple, comparable versions (to eliminate a practice effect)

• is sensitive to student growth

• provides timely results

• Can be charted

• is called a PROBE

Today’s Big Idea

If you want to measure change, don’t change the

measure.

LOOK FOR MULTIPLE, COMPARABLE VERSIONS

General Outcome Measures

Specific skill assessments that can be linked to achievement in broader academic domains, like reading, spelling, or math.

Initial Studies (2004-05) Examined reliability and validity of four potential

measures

3 7

Quantity Discrimination

● ● ● ●● ● ● ●

Quantity Array

Number ID

0 ___ 2 3

Missing Number

Anne Foegen, Ph.D., 2006

6

The Search for a Stronger GOM for Math: Quantity Discrimination

Quantity Discrimination

“Look at the paper in front of you. In each row there are some boxes with numbers in them.” (Point to the first set of boxes in the top row)

“I want you to tell me the number that is bigger.”

“Look at the paper in front of you. There are boxes with dots in them. (Point to the first box.)

“How many dots are in each box?”

The Search for a Stronger GOM for Math: Quantity Array

“Look at the paper in front of you. Each box has three numbers and a blank.” (Point to the first box.)

“What number goes in the blank?”

The Search for a Stronger GOM for Math: Missing Number

“Look at the paper in front of you. There are numbers in boxes.” (Point to the first box.)

“What number is this?”

The Search for a Stronger GOM for Math: Number Identification

Your Vote?

Anne Foegen, Ph.D., 2006

3 7

Quantity Discrimination

● ● ● ●● ● ● ●

Quantity Array

Number ID

0 ___ 2 3

Missing Number

6

Quantity Array did not prove to be a reliable and valid tool for K-1 screening or PMing.

Activity 3Cont.

Compare These Two Assessments

Of Nonsense Word Fluency

Activity 4

6:00 o’clock partner

DIBELS Next Nonsense Word Fluency

Scholastic Teacher Resource

Purposes of Assessment

Screening Diagnostic Progress Monitoring Outcome

Why does DIBELS NWF meet evidence for Progress Monitoring while the Scholastic assessment does not?

How can the Scholastic measure be effectively used?

www.rti4success.org/progressMonitoringTools

January 2009

Why Progress Monitor? • Progress monitoring has been extensively researched in Special

Education (Fuchs & Fuchs, 1986) • Students showed improved reading scores when teachers:

Action Effect Size

Monitored progress +.70

Graphed progress +.80

Used decision rules +.90

(0.2 - 0.3 small 0.5 medium 0.8 - 1.0 large)

IEP Goals Page

Progress Monitoring

Writing Legal & Educationally Meaningful IEP Goals

• • • Area of Need: Reading Fluency

Condition: Given Grade-3 reading material

Specific behavior: Will fluently and accurately read aloud

Goal baseline: 50 Words Correct per Minute

Goal target: 101 Words Correct per Minute*

*Using one approach to establishing goals, the teacher calculates a 1.5-word increase across 34 weeks left in

school year and adds it to the current baseline. Goal is set at 101 words read correctly per minute: (1.5 x

34) + 50 = 101. Adapted from Stecker, Lembke, Saenz 2007

NOT

As an evaluation method:

Grades

Grade levels

Observation

Standardized test scores (e.g. WJ-III, WIAT-2)*

Percentages (with caution)

Evaluation tools with a small scale (e.g. 0-4)

* Tests designed with 3 or more forms for progress monitoring are permissible (e.g. TOSWRF, TOSCRF)

WHAT

TOWRITE

As an evaluation method:

Curriculum Based Measures (CBM)

Rubrics

Timer/ Observation Data

Standardized Formative Assessments (Acuity, MAP)

Standardized Diagnostic Assessments with multiple, equitable forms

WHAT

TOWRITE

Progress Reporting

Make it simple

May write: “See attached graph/chart”

IEP goals & progress reports use the same data collection

Break

Methods to Evaluate & Chart Growth

from CBMs to Rubrics

Commonly Used PM ToolsCBM Probes: Early Reading (ISF, PSF, LSF, NWF) ORF Reading Mazes Math Probes Writing (CWS/CIWS)

Other Measures: Standardized Commercial Assessments Scoring Guides and Rubrics Timer/Observation Data

PracticeUsing CBMs, Commercial Tools, Scoring

Guides, Rubrics to Monitor Progress

Oral Reading Fluency12:00 o’clock Partner

Activity 5

Oral Reading Fluency (ORF) Individually administered test of accuracy and

fluency with connected text

Measured in Words Correct per Minute (WCPM)

Timed for 1 minute

Baseline is the average of 2 probes or the median of 3 probes (depending on measure used)

Materials needed: Stopwatch or timer Clipboard Student & administrator copies of probes Pencil or pen

Oral Reading Fluency (ORF)

Today, we are using 1st grade Fundations’ ORF probes from Wilson Reading Systems

Oral Reading Fluency (ORF) Start your timer or stopwatch when the student

says the first word of the passage. The title is not counted. If the student fails to say the first word after 3 seconds, say the word and mark it as incorrect, then start your timer.

The maximum time for each word is 3 seconds. If the student does not provide the word within 3 seconds, say the word and mark the word as incorrect.

Follow along on the examiner scoring sheet. Put a slash(/) over words read incorrectly.

Scoring

Oral Reading Fluency (ORF) At the end of 1 minute, place a bracket

( ] )after the last word provided by the student and say “Stop.”

Record the total number of words read correctly on the bottom of the scoring sheet by counting the number of words the student attempted (up to the bracket). Then count the number of errors made. Subtract errors from total number of words attempted. Result is number of words read correctly.

Oral Reading Fluency (ORF)

Say:

When I say “Begin,” start to read the story to me. If you come to a word you do not know, do your best to get it. I am going to time you while you read, but most importantly read it well so that you can tell me all about it afterward. Are you ready? Begin.”

Go ahead and practice with your partner.

University of Maryland (CBM reading passages)

The CBM measures are free to download and use. The CBM measures, teacher scoring sheets, administration instructions and scoring instructions are on the internet:

http://www.glue.umd.edu/~dlspeece/cbmreading 

Letter Sound Fluency Test (practice list, 12 lists) Oral Reading Fluency passages for grade 1 (15 passages) Oral Reading Fluency passages for grade 2 (19 passages) Oral Reading Fluency passages for grade 3 (14 passages) Oral Reading Fluency passages for grade 4 (14 passages)

Vanderbilt University (CBM reading passages)

CBM measures are free, except for copying costs and postage. The CBM measures, scoring sheets, administration instructions, and scoring instructions are available: Letter Sound Fluency Test for kindergarten (5 tests) Word Identification Fluency Test for grade 1 (20 tests) CBM Reading passages for grades 1-8 (30 passages per grade) Maze Fluency passages for grades 1-6 (30 passages per grade)

Mail: Diana Phillips Peabody #328

230 Appleton PlaceNashville, TN 37203-5721

Phone: 615-343-4782

Math CBMsAIMSweb, EasyCBM, Intervention Central, Teacher Made

M-CBM

M-CBM Math FactDirections (Grades 1-6)

1. Students have a Math Fact Probe and a pencil.

2. Say to the student(s):We’re going to take a 2-minute Math Fact test. I want you to write

your answers to: <Addition and subtraction> <Multiplication and division> <Addition, subtraction, multiplication, and division>

problems. Look at each problem carefully before you answer it.”

When I say ‘BEGIN,’ write your answer to the FIRST problem (demonstrate by pointing) and work ACROSS the page. Then go to the next row.

Try to work EACH problem. If you come to one YOU REALLY DON’T KNOW HOW TO DO, put an 'X’ through it and go to the next one.

If you finish the first side, turn it over and continue working. Are there any questions? (Pause)”

Directions Continued3. Say, “BEGIN” and start your stopwatch/timer.

4. If testing in groups, walk around and monitor students to ensure they are not skipping problems, are working across the page, and continue to write answers to the problems during the test time.If a student is excessively skipping problems they should know how to do,

say to the student:

“Try to work EACH problem. You can do this kind of problem so don’t skip or put an ‘X’ over it.”

If a student is not working across the page, say to the student.

“Work across the page. Try to work each problem in the row.”

If a student stops working before the test is done, say to the student.

“Keep doing the best work you can.”

5.At the end of 2 minutes, say, “Stop. Put your pencils down.” Monitor to ensure students stop working.

Scoring Digits Correct

Scoring Digits Correct

Scoring Digits Correct

Math CBM Scoring PracticeActivity 6

3 o’clock partner

Setting GoalsUsing Chartable Data

Goal-Setting Methods

Local or nationally established benchmarks

Local or nationally established growth rates that reflect typical increases in performance by grade level

Intra-individual framework that uses the student’s current growth rate to establish the target growth rate

Goal-setting method varies by CBM tool used

Setting Realistic, but Ambitious

Goals

+ 1/2

Projected growth

Measurable IEP goals that are Compatible with CBM

use Once the goal has been calculated, a Long Range Goal (LRG) can be written

using the following formula:

In _______ (number of weeks until next IEP review) when presented with stories from Level ________ (level number in which the student currently reads 30-60 wpm) in ______________ (name of reading series) _________ (student's name) will read aloud _________ words correctly.

The following format can be used to write an IEP objective:

Each successive week, when presented with a random selection from Level _________ (same as for LRG) ______________ (name of reading series) __________ (student's name) will read aloud at an average increase* of _____ wpm.

* The average weekly increase is the aimline and can be obtained from calculating the numeric value of the aimline (see determining the trend [slope]).

www.specialconnections.ku.edu/cgi-bin/cgiwrap/specconn/main.php?cat=assessment&section=main&subsection=cbm/iepgoals

Mark Semmler’s Excel Graphing Tools

Multi-Student Chart

Charting Results

Excel spreadsheets

Three provided: CCSD cbmExcelChart aimline and trendline.xls CCSD cbmExcelChart aimline no trendline.xls CCSD Multi Student PM Charting Tool

Decision RulesFormulating “Next Step” Decisions

4-Point Rule

When at least 6 points have been collected, examine the 4 most recent data points. If all 4 are above Aim Line, increase goal. If all 4 are below Aim Line, make an

instructional change. If the 4 data points are both above and

below the Aim Line, continue collecting data until the 4-Point Rule or Trend-Line Rule can be applied.

Decision Rules

Decision Rules

Trend-Line Rule

When at least 8 data points have been collected, review trend of current performance and compare to Aim Line (Goal Line). If trend of student progress is steeper than Aim

Line, raise goal. If trend of student progress is less steep than Aim

Line, make an instructional change.

Standardized Formative Assessments

Standardized Formative Assessments

Rubrics

When To Use

Rubrics are a good alternative when no validated, or strongly validated, measure is available:

Secondary math/ problem solving Writing Behavior

Points on rubric need a wide point spread in order to show growth over time

Behavior Tools

Timer/ Observation Data www.specialconnections.ku.edu/cgi-bin/cgiwrap/specconn/main.php?cat=assessment&section=teachertools

Event Recording (Frequency)

Duration

Latency

Momentary Time Sampling

Permanent Product Measurement (e.g. homework)

SEL Rubrics

Univ of Kansas

Special Connections

Tool

Cherry Creek

SEL Rubric Example

Cherry Creek

SEL Rubric Example

Standard Rubric Applied to Writing

Sample

Whole class, small group, or individually administered

Use same standardized instructions and story starters/prompts as CBM Writing

One minute to plan, either 3 or 7 minutes to write

Monitoring Writing Intervention Progress

(MWIP)

Homework Pick one of the two IEP goals you wrote for David Tower to practice

the graphing of progress data using Excel.

You will need to make-up at least six data points. For the sake of ease, pretend this is last year’s data. Plot the baseline score at January 1, 2011 and the goal for June 1, 2011. Data points should be plotted at intervals that make sense for the skill being taught (You decide: monthly, twice-monthly, weekly, twice-weekly or daily).

Graph David’s progress by using one of the Excel charting tools created by CCSD SpEd Coordinator Mark Semmler. These can be downloaded at http://ccsdrti.pbworks.com/Charting-Tools

cbmExcelChart aimline and trendline.xls cbmExcelChart aimline no trendline.xls CCSDMultiStudentPMChartingTool.xls

Print out your goal and chart to turn in.

HomeworkWrite your third and final one-page field experience summary. Be sure that

your observation, interview, or practice links an intervention to an assessment.

Example 1: Interview a school interventionist (SpEd, ELA, Title I, or Reading Recovery teacher). Ask him or her what screening or diagnostic assessments were used to place students in their intervention, what progress monitoring assessments are used to measure growth, and what outcome assessment will be used to exit the child from the intervention.  

Example 2: Observe an intervention with a student (academic or behavioral). Write what you see in objective terms. Comment on its effectiveness: Was the purpose of the lesson clear? Did instruction appear to be systematic and explicit? Was assessment used to inform instruction? Was the child engaged? 

Example 3: Summarize your own experience providing an intervention. Provide the date when it occurred, how many students, your purpose, and what assessment did you use to determine the appropriateness of the selected activity? Were students engaged? How effective was it? What could you do to improve the lesson in the future?

Homework Continue to work on your Research-Based

Interventions /Toolkit. It is due in hard copy by (40 activities + 10 Programs) on August 20th.

Reading Math Written Expression Social or Emotional Behavior Spelling Functional Academic, Transition or Job

Skills