assessment and tier iii rti foundations training june-september 2010 erin banks and amy roberts,...

Assessment and Tier III

RtI Foundations TrainingJune-September 2010

Erin Banks and Amy Roberts, School Psychologists

QuickTime™ and a decompressor

are needed to see this picture.

22

What is the Responsiveness to Instruction Model?

• A student with academic delays is given one or more research-validated interventions.

• The student's academic progress is monitored frequently to see if those interventions are sufficient to help the student to catch up with his or her peers.

• If the student fails to show significantly improved academic skills despite several well-designed and implemented interventions, this failure to 'respond to intervention' can be viewed as evidence of an underlying Learning Disability.

www.interventioncentral.com

33

Why RtI?

• One advantage of RtI in the diagnosis of educational disabilities is that it allows schools to intervene early to meet the needs of struggling learners.

• Another advantage is that RtI maps those specific instructional strategies found to benefit a particular student. This information can be very helpful to both teachers and parents.

www.interventioncentral.com

What is Curriculum Based Measurement?

• Curriculum Based Measurement is a form of Curriculum-Based Assessment (CBA).

• CBM is a method of monitoring student educational progress through direct assessment of academic skills

• CBMs measure basic skills in reading, mathematics, spelling, written expression and readiness skills– Teacher gives the student brief, timed samples, or

“probes,” made up of academic material that the student is expected to learn

Adapted from www.aimsweb.com

CBMs

• Because CBM probes are quick to administer and simple to score, they can be given frequently to provide continuous progress data. The results are charted and provide for timely evaluation based on hard data

• Early literacy skills (phonics and phonological awareness) are downward extensions of CBM (General Outcome Measures - GOM).

Using Curriculum Based Using Curriculum Based Measures as General Measures as General Outcome MeasuresOutcome MeasuresIt’s about using General Outcome It’s about using General Outcome Measures (GOMs) for formative Measures (GOMs) for formative assessment/evaluation to:assessment/evaluation to:

•• Inform teaching Inform teaching

ANDAND

• • ensure accountability.ensure accountability.

It’s different from, but related to, It’s different from, but related to, summative high-stakes summative high-stakes testing/evaluation, which:testing/evaluation, which:

• • Doesn’t inform teaching.Doesn’t inform teaching.

• • Mostly used for Mostly used for accountability/motivation.accountability/motivation.

Adapted from www.aimsweb.com

Formative EvaluationsFormative Evaluations

Formative Evaluation: Process of assessing student achievement during instruction to determine whether an instructional program is effective for individual students.

• When students are progressing, continue using your instructional programs. • When tests show that students are not progressing, you can change your instructional programs in meaningful ways.

Adapted from www.aimsweb.com

Summative AssessmentsSummative Assessments

SumSummativemative Assessment: come at the end of Assessment: come at the end of a program of studya program of study• Tells you what information the child Tells you what information the child has mastered for the year without has mastered for the year without ability to change instruction ability to change instruction

• Big tests that include a wide range of Big tests that include a wide range of information covered over a long period information covered over a long period of timeof time• High-stakes tests (EOGs)High-stakes tests (EOGs)

Adapted from RtI in the Classroom

Universal (school-wide) screening Universal (school-wide) screening using CBMs allows us to add systematic using CBMs allows us to add systematic Formative Evaluation to current Formative Evaluation to current practice.practice.

• For Teachers (and Students)For Teachers (and Students)• Early Identification of At Risk Early Identification of At Risk StudentsStudents

• Instructional PlanningInstructional Planning• Monitoring Student ProgressMonitoring Student Progress

• For ParentsFor Parents• Opportunities for Opportunities for Communication/InvolvementCommunication/Involvement

• AccountabilityAccountability

• For AdministratorsFor Administrators• Resource Allocation/Planning and Resource Allocation/Planning and SupportSupport

• AccountabilityAccountability

Using Curriculum Based Using Curriculum Based Measurement as a General Measurement as a General Outcome MeasureOutcome Measure

Adapted from www.aimsweb.com

Using Curriculum Based Using Curriculum Based Measurement as a General Measurement as a General Outcome Measure: ResearchOutcome Measure: Research

• Curriculum-Based MeasurementCurriculum-Based Measurement (CBM) was (CBM) was developed more than 20 years ago by developed more than 20 years ago by Stanley Deno at the University of Stanley Deno at the University of Minnesota through a federal contract to Minnesota through a federal contract to develop a reliable and valid measurement develop a reliable and valid measurement system for evaluating basic skills system for evaluating basic skills growth.growth.

• CBM is supported by more than CBM is supported by more than 25 years 25 years of school-based researchof school-based research by the US by the US Department of Education.Department of Education.

• Supporting documentation can be found Supporting documentation can be found in in 100s of articles, book chapters, and 100s of articles, book chapters, and books in the professional literaturebooks in the professional literature describing the use of CBM to make a describing the use of CBM to make a variety of important educational variety of important educational decisions. decisions.

Adapted from www.aimsweb.com

Summary of Research Summary of Research Validating Curriculum Based Validating Curriculum Based MeasurementMeasurementReliableReliable and and validvalid indicator of student indicator of student

achievementachievement

Simple, efficientSimple, efficient, and of , and of shortshort

duration to facilitate frequent duration to facilitate frequent

administration by teachersadministration by teachers

Provides assessment information that Provides assessment information that

helps teachers plan better instructionhelps teachers plan better instruction

Sensitive to the improvementSensitive to the improvement of of

students’ achievement over timestudents’ achievement over time

Easily understoodEasily understood by teachers and by teachers and

parentsparents

Improves achievementImproves achievement when used to when used to

monitor progressmonitor progress

Adapted from www.aimsweb.com

Curriculum Based Measurement: Advantages

• Direct measure of student performance

• Helps target specific areas of instructional need for students

• Quick to administer

• Provides visual representation (reports) of individual student progress and how classes are acquiring essential reading skills

• Sensitive to even small improvements in performance

Adapted from www.aimsweb.com

Advantages continued…

• Capable of having many forms

• Monitoring frequently enables staff to see trends in individual and group performance—and compare those trends with targets set for their students.

• Correlates strongly with “best practices” for instruction and assessment, and research-supported methods for assessment and intervention.

Curriculum Based Curriculum Based Measurement: Things to Measurement: Things to Remember Remember

•Standardized tests to be given, scored, Standardized tests to be given, scored, and interpreted in a standard way.and interpreted in a standard way.

• Researched with respect to psychometric Researched with respect to psychometric properties to ensure accurate measures of properties to ensure accurate measures of learning.learning.

• Are sensitive to improvement in brief Are sensitive to improvement in brief intervals of time.intervals of time.

• Tell us how students earned their scores Tell us how students earned their scores (qualitative information).(qualitative information).

• Designed to be as short as possible to Designed to be as short as possible to ensure “do-ability.”ensure “do-ability.”

• Are linked to decision making for Are linked to decision making for promoting positive achievement and promoting positive achievement and problem-solving.problem-solving. Adapted from www.aimsweb.com

1515

Curriculum Based Measurement

• CBM has been shown to posses high levels of reliability – Reliability - the extent to which the measurements of a test remain consistent over repeated tests of the same subject under identical conditions

• 42 one-minute CBM type assessments in reading, math, and written expression for grade K-5 were found to have reliability coefficients between .90-.99 with just three one-minute administrations (Jenkins, 2002)

1616

Curriculum Based Measurement

• Discriminant Validity - Does it appear to measure what it’s supposed to measure?

And Doesn’t associate with constructs that shouldn’t be

related.

• Several studies have demonstrated the ability of CBM to differentiate between students receiving special education services, students receiving Chapter 1 services, and students not receiving any of those services (Deno, Marston, Shinn, and Tindal, 1983; Marston and Deno, 1982; Shinn and Marston, 1985; and Shinn, Tindal, Spira, and Marston, 1987).

1717

Other types of CBMs - AIMSweb

• Similar reading probes to DIBELS that include reading fluency and early literacy

• Adds a reading comprehension piece (MAZE)

• Also has math computation, math concepts and applications, early numeracy and writing components

AIMSweb: Tests of Early Numeracy

Number Identification• Name single digit and double

digit numbers • One minute timed task• Administer to:

– Kindergarten – fall– Kindergarten – winter– Kindergarten – spring– 1st Grade – fall

– 1st Grade – winter

1818

AIMSweb: Tests of Early Numeracy

Quantity Discrimination

1919

• Identify greater number• One minute timed task• Administer to:

– Kindergarten – fall– Kindergarten – winter– Kindergarten – spring– 1st Grade – fall– 1st Grade – winter – 1st Grade - spring

AIMSweb: Tests of Early Numeracy

Missing Number

2020

• Identify missing number

• One minute timed task

• Administer to:

– Kindergarten – fall

– Kindergarten – winter

– Kindergarten – spring

– 1st Grade – fall

– 1st Grade – winter

– 1st Grade - spring

AIMSweb: MAZE (Reading Comprehension)

• Student reads passage silently for 3 minutes

• Every 7th word is replaced with three choices

• Student circles correct choice

• Can be group administered

AIMSweb: Data Management

2222

2323

AIMSweb: Data Management

Other types of CBMs: www.interventioncentral.org

• Website has many CBM probes available for free

• You can create multiple forms of early literacy and numeracy probes

• Many national norms available for comparison

2424

CBM Videos

• AIMSweb training videos – Letter Sound Fluency– Nonsense Word Fluency– Phoneme Segmentation Fluency– Quantity Discrimination

QuickTime™ and a decompressor

are needed to see this picture.

Tier III

QuickTime™ and a decompressor

are needed to see this picture.

What has happened to get us to this point?

• Student has not made sufficient progress with Tier I (4-6 weeks with pre and post test) and Tier II interventions (4-6 weeks with progress monitoring every 2 weeks)

• Now they go to RTI assistance team to receive more intensive interventions

Requirements of Tier III

• 8 week minimum time frame with 2 interventions (4 weeks each)

• Intervention 3-4 times a week for at least 30 minutes (in addition to the core)

• Progress monitoring 2 times a week

• Data check after 4 weeks

1st Meeting• 1st Step - Parent invitation needs to be sent

home PRIOR to the meeting• Well defined target behavior:

– Enabling skills for written expression: mechanics and expression

– Enabling skills for math: number sense, facts, computation, applications, and problem solving

– Enabling skills for behavior: social skills, work completion, compliance, and problem solving skills

• Define the problem by examining effects of environment, curriculum, instruction, and learner

Environment

Environment: things in the child’s environment that is interfering with learning– Attendance issues, family life, transience

– Issues in the classroom

QuickTime™ and a decompressor

are needed to see this picture.

Curriculum• Curriculum:

– Consider if curriculum appropriate for student. Consider sequence of objectives, teaching methods, and practice materials provided.

– What does the core look like in the classroom?

– What extra services is the child receiving?

– Does the student need more systematic instruction?

– Has he been exposed to numerous curriculums due to transience?

QuickTime™ and a decompressor

are needed to see this picture.

Instruction

• Instruction: – Manner in which teacher uses curriculum – consider instructional techniques, presentation style, questioning, feedback techniques.

– Does he benefit from small group, 1:1 instruction?

Learner

• Learner:• Student skill – necessary prerequisite skills needed to be successful; i.e. Lacks fluency for comprehension or lacks phonemic awareness which affects fluency and comprehension

• Student process – capacity to learn and problem solving techniques; i.e. Has difficulty with working memory or processing

QuickTime™ and a decompressor

are needed to see this picture.

RIOT

• How do you investigate why the problem is occurring?– RIOT:

• R - Review information: records, cumulative folders, etc.

• I - Interview others: parent, teacher, student

• O - Observe: student in specific class or core

• T - Test: basic skill levels

– Assign who will be in charge of the investigation. Set the next meeting time.

Meet Bubba

QuickTime™ and a decompressor

are needed to see this picture.

Bubba’s BaselineMeasuring Tool: Dog-A-Rama Scale

• Monday = 0 pounds lost• Wednesday = .5 pounds lost• Friday = 0 pounds lost

• Median Score = 0 pounds lost (baseline)(No intervention has started yet!)

QuickTime™ and a decompressorare needed to see this picture.

Bubba’s Weight Loss Goal

Goal Statement:Bubba will lose 25 pounds by 8 weeks, as measured by the Dog-O-Rama scale.

Hypothesis:The gap between Bubba’s current and desired weight will be decreased if he begins a High Protein Dog Food diet.

Bubba’s InterventionIntervention:

– Bubba will eat 1/2 cup of High Protein Dog Food at 7:30 AM and a 1/2 cup of High Protein Dog Food at 5:30 PM.

– Stranger Test(does this make sense to someone off the street?)

QuickTime™ and a decompressorare needed to see this picture.

Bubba’s Progress MonitoringMeasuring Tool:

Bubba’s progress will be measured by weighing him 2 times a week using the Dog-A-Rama scale to determine his pounds lost per week.

Decision-Making Rule:If 5 or more data points are below the aimline, change intervention; if 5 or more data points are above the aimline; continue intervention.

QuickTime™ and a decompressorare needed to see this picture.

Review Bubba’s Intervention

Baseline = 0 pounds

Data Points Below Aimline:

5

Data Points Above Aimline:

3

Is he making progress toward his goal?

Should the vet change his intervention?

YES!

Intervention ChangeSecond Intervention:

Bubba will participate in an exercise program consisting of walking 30 minutes, 3 times a week.

Evaluation Tool:Continue to measure progress using the Dog-A-Rama scale, weighing Bubba 2 times a week.

QuickTime™ and a decompressor

are needed to see this picture.

Bubba’s Weight Loss Review

Goal = 25 pounds

Data Points Above Aimline: 1

Data Points Below Aimline: 8

Did Bubba make progress?

Did he meet his goal?

What should we do next?

Update on Bubba• Bubba is losing weight!

• May choose to combine diet & exercise for most effective intervention

QuickTime™ and a decompressor

are needed to see this picture.

How Does Bubba Relate to the RtI Process?

Obtaining Baseline:• Gathered prior to intervention• Provides comparison for progress data• Helps set goals• Give CBM 3 times• Put the scores in numerical order and take the median score to get rid of outliers

• Get a new baseline if needed due to new information obtained from RIOT-ing

Review Baseline

• Team meets again within 2 weeks to document results

• Based on these results, set goal based on the data

• Baseline may be gathered between Tier II and Tier III to expedite process

Goal Setting• Short-term goals: where do you want them to be at the end of 8 weeks (intervention term)?

• Long-term goals: where do you want them to be long-term, by the end of the year?

Short-term goals are graphed

Goal Setting• May use:

– Grade-Level norms– RSS District norms– AIMSweb national aggregate norms– Rate of Improvement/Growth Rate

• Make short-term goal realistic based on baseline data but also high enough to get them to where they need to be (grade level) (at 10%ile, goal is 25%ile)

Setting goals for students should put them on a TRAJECTORY to be on-grade level.

• Ex. By the end of 8 weeks, Johnny will increase his reading fluency to 100 correct words per minute on grade level reading passages

Reading Fluency Growth Rates (Doug and Lynn

Fuchs)Realistic Growth Rates

• 1 - 2words/week• 2 -1.5words/week• 3 -1 word/week• 4 -.9 words/week• 5 -.5 words/week

Ambitious Growth Rates

• 1 - 3 words/week• 2 - 2 words/week• 3 - 1.5 words/week• 4 - 1.1 words/week• 5 - .8 words/week

Goal Setting - Tier III paperwork

• Choose your goal• Document the goal (where you want the student to be)

• Document their baseline (current level of performance)

• Calculate the difference• Document method used for goal setting Ex. AIMSWeb Aggregate winter norms at 25th percentile (short-term goal)

Intervention

• Match intervention to problem.• Intervention should be developed with the expectation that it will be altered in some way as a result of the progress monitoring data– Increase time– Increase frequency– Increase intensity before changing intervention completely (unless child has flatlined)

• No intervention works all of the time for every student

Intervention Plan

What intervention will be used? (Use the stranger test!)

– Ex. Florida Center for Reading Research (FCRR) P.001

Arrangements: where, who, how much, and with what?

– Ex. 4 times a week in hallway alcove

Persons responsible: who will deliver the intervention and who will progress monitor?

– Ex. Mrs. Teacher will provide intervention and Amy Roberts will progress monitor

Intervention Plan…

• What will be used to measure progress and how often will it be measured?– Ex. AIMSweb reading CBM will be used to progress monitor 2x a week.

Intervention will be changed when…

• The majority of progress monitoring data points are below the child’s goal/aimline (5 out of 8)

• After 4 weeks there will be a data check to see what the data is telling us.

How to Create a Graph for Progress Monitoring

• Set up the graph– Write the student’s name on the graph– Label the vertical axis with what you are

assessing (i.e. words correct per minute)– Label the horizontal axis with dates of

assessment– Title the graph

• Establish baseline– Administer probe 3 times– Select the median (the middle number)– Mark the baseline by placing a dot on the

vertical axis

Graphing Continued…

• Set the goal/target– Determine the expected rate of progress (e.g. 2

words per week, 110 words by end of 8 weeks) and plot on graph

• Draw the aimline– Using a red colored pencil and a ruler, connect

the baseline data point with the target data point

• Measure and plot student performance– Administer probes or assessments according to

plan– Place the data point on the graph– Connect the data point to the previous point

Graphing• Analyze student performance

– Determine whether the student is making adequate progress• If 5 out of 8 data points are above aimline, raise the goal

• If 5 out of 8 data points are below aimline, the student is not making adequate progress and need a change in intervention

• If the data points are along the aimline, the student is progressing right on target and continue intervention and progress monitoring

– If a change in intervention is required, draw a squiggle line from the top of the graph to the bottom to indicate intervention change

Last Page

• After 8 weeks, meet back and document child’s level before the intervention and their level after the intervention and if they achieved their goal

• Check decision: – No longer an area of concern– Continue intervention plan– Modify intervention plan– Move to Tier IV

Requirements to go to Tier IV

a. Two interventions with data indicating performance below the 15th%ile based on Rowan Salisbury norms.

b. Performance that is more than two times below the grade mean (50th%ile). For example, if the child has an ORF of 25 and the 50th%ile is 100, the child’s performance is more than 2xs below the grade mean or 50th%ile.

c. The intervention is showing mild success but it is so intensive that it cannot be maintained without special education assistance.

d. Growth rate indicating that of insufficient progress (their aimline and trend line will never cross).

That’s All Folks!