data-based decision making · data-based decision making progress monitoring and measuring fidelity...

DATA-BASED DECISION MAKING

Progress Monitoring and Measuring Fidelity At Tier 2

1

2

Progress Monitoring

Essentials

Monitoring Effect

Monitoring Fidelity

Comprehensive Assessment System 3

Screener Diagnostic Progress Monitoring

(Formative Assessment)

Problem Solving Model

Progress Monitoring Essentials 4

What makes an assessment able to be used for progress monitoring? Efficient (easy to give, easy to understand) Global indicator of skills Reliable and valid Recent (old data is useless data)

Monitoring Student Progress 5

Tier 2 Student Progress Monitoring System: Assess specific skills (not high-inference skills; observable

behaviors) Sensitive to small increments of growth over time Administered efficiently and repeatedly (quickly and

easily) Easily summarized in teacher/parent-friendly formats for

communication purposes (graphs) Structured so that comparisons can be made across

students Have direct relevance to intervention development for

specific skills

Monitoring is Two-fold 6

1. Monitoring outcomes (is it working?) 2. Monitoring fidelity/implementation (are we doing

what we said we would?)

Data Collection Progress Monitoring & Frequency

Person Collecting/ Interpreting Data

Once/day Several times/day

Teachers with Behavioral Expertise

Paraprofessionals School Counselors

School Psychologists

Once/day Several times/week

1-2 times/week

Teachers Paraprofessionals School counselors

School psychologist

1-2 times/month Monthly

3 times/year

Teachers All staff

-Behavior Tracking Forms (ODRs) -Suspension, Expulsion reports

*SET, BoQ, PIC, TIC

Daily Progress Reports Checklists, ODRs

Permanent Products *BAT, Checklists

Direct Observation Behavior Rating Scale

*BAT, Checklists, Interviews

7 *indicates tool for fidelity/implementation

8

Progress Monitoring

Essentials

Monitoring Effect

Monitoring Fidelity

Monitoring Effect of Interventions: Two questions

9

1. Is it working for most students (overall effectiveness)?

2. Is it working for one particular student (individual effectiveness)?

Has to be effective for most before you can really determine if

it’s effective for one

10

0

20

40

60

80

100

120

student 1Linear (student 1)

Improve fidelity and redo Intervention

Not sure if Intervention would

have worked

Intervention didn’t work

Measuring Fidelity

Fidelity was good

Fidelity was unclear, not good,

or don’t know

Checking for good fidelity avoids misattributing poor response to diet when poor fidelity/implementation is the culprit

Goal Not Met

12

0

20

40

60

80

100

120

student 1Linear (student 1)

13

0

20

40

60

80

100

120

student 1student 2student 3student 4

14

0

20

40

60

80

100

120

student 1student 2student 3student 4

15

Evaluate overall effectiveness of Tier 2 For instance, here is CICO

Examples 16

ODRs Daily points on a behavior tracking card Daily self-report or checklist Percentage of work completion Others?

18

19

Progress Monitoring

Essentials

Monitoring Effect

Monitoring Fidelity

Monitoring Effect 20

1. Check fidelity 2. Check if it’s working for most students (is it

effective at all?) 3. Check if it’s working for a particular student

Fidelity? Most students? One student?

Measuring Fidelity: Did it occur?

21

Direct Assessment Components of an intervention are clearly outlined in

operational terms and directly observed. Often calculate a total percentage of number of

components implemented

Indirect Assessment Assess degree of implementation without directly

observing instruction/intervention. Typically based on self-reports, interviews, and review of permanent products.

Checklists/Observations 22

Checklists can break down the components, mark if implemented and calculate a % of parts implemented

Examples of Indirect Assessment

24

Look at… self reports (rating scales, checklists) on implementation lessons plans to ensure instruction was planned for and

provided attendance to ensure student was present to receive

instruction student products to see that students participated in the

instruction

25

10/12 35 45 Fire drill ended early, 10/13 45 45

Social Skills

School-Generated Checklists… 27

Daily Behavior Plan Assessment—CICO Teacher

28

Fidelity 29

Fidelity is not an evaluation tool (there are other processes for that)

Coaching tool No surprises (share what’s on it) It’s about following the “prescription”, not fooling or

evaluating people

Measuring Fidelity: Did it match?

Can be more than just ‘Did it occur?’. Can also ask if the intervention targeted the student’s deficits.

If the student has a deficit in academics, would an intervention targeting social skills be appropriate? If the function is peer avoidance, would a study hall

(for adult tutoring) be appropriate?

Is Check In/Check Out appropriate for a student who doesn’t want adult attention?

Measuring Fidelity: Did it match?

Measuring Fidelity: Did it match?

STAY ON TARGET! • Ensure the intervention is

on target by doing good problem analysis and using research-based instruction.