secondary literacy: universal screening and progress monitoring
DESCRIPTION
David Putnam, Jr., Ph.D. Associate Director, C & I Tigard Tualatin School District. Secondary literacy: Universal Screening and Progress Monitoring. Provide a rationale and framework for literacy intervention at the secondary level Examine the Maze and its relationship to OAKS - PowerPoint PPT PresentationTRANSCRIPT
SECONDARY LITERACY:UNIVERSAL SCREENING AND PROGRESS MONITORING
David Putnam, Jr., Ph.D.Associate Director, C & ITigard Tualatin School District
OBJECTIVES OF THE PRESENTATION Provide a rationale and framework for literacy
intervention at the secondary level Examine the Maze and its relationship to OAKS Detail the process for using Maze for Universal
Screening, program evaluation, and identifying students in need of additional support
Describe progress monitoring effectiveness, and procedures for analyzing performance and instructional decision making
WHAT’S THE PROBLEM? Teaching reading is often considered an
elementary school task despite….. More than 8 million students in grades 4 – 12 are
struggling readers (USDoE 2003). 40% of HS students cannot read well enough to
benefit from their textbooks (NAEP, 2005). In Oregon in 2006-07, 33% of 8th graders and 35%
of 10th graders (35%) did not meet OAKS reading.
The problem is more severe when we disaggregate data by racial and special program subgroups.
• High Expectations for Student Achievement--And Always Increasing
• Students w Moderate to Severe Educational and/or Behavioral Needs--Big Prerequisite Skill Deficits
• Students with a Long History of Failure--Poor Motivation and Lots of Escape Driven Behavior
• General Education Teachers with Limited Support Skills and Instructional Materials
• Students’ Programs Being Driven by Graduation Requirements Rather Than Instructional Needs
Mark R. Shinn, Ph.D. & Madi Phillips, Ph.D.NASP, 2007
SECONDARY EDUCATION FOR MANY STUDENTS
TTSD SECONDARY LITERACY PLAN: Focus resources on teaching
literacy strategies proven to increase achievement for all students across all content areas
Execute a comprehensive literacy intervention model to address students in need of strategic and intensive interventions
Use a Three Tier Model morphed to secondary schools
EFFECTIVE SECONDARY INSTRUCTION:A THREE TIER MODEL
All students, IN EVERY TIER, have access to embedded literacy strategies across content areas
Strategies: Frayer Model Anticipation Guide Word Sorts DR/TA or KWL Group Summarizing Definition Word
Chart Differentiated
Assessment
TIER I: WHAT DO ALL STUDENTS RECEIVE?
Core Curriculum Access to Content
Literacy Strategies A limited number of
students are monitored by the Literacy Specialist
Target = 80% of student population
TIER II: WHAT DO STUDENTS RECEIVE IN ADDITION TO THE CORE?
Content Literacy Strategies Across the Content Areas
Strategic Intervention Middle School: Soar
to Success High School: Literacy
Strategies Classes
Target = >15%
Student Population
TIER III: WHAT DO STUDENTS RECEIVE IN ADDITION TO THE CORE?
Content Literacy Strategies Across the Content Areas
Comprehensive reading and writing support LANGUAGE! (High
School) LANGUAGE! (Middle
School)
WHAT IS UNIVERSAL SCREENING? Universal screening is the
process of efficiently assessing ALL students in a timely manner to analyze: The effectiveness of
curriculum, instruction, and school organization
Students’ level of proficiency in essential academic areas
Identify student that MAY need additional help
WHY UNIVERSAL SCREENING?DETERMINE PROGRAM EFFECTIVENESS
Are 80% of our students meeting the benchmark? 80% by ethnicity? By program sub-group? By subject?
WHY UNIVERSAL SCREENING? PROGRAM EVALUATION
Helps you to determine if the core curriculum needs to be addressed Intensity Fidelity Targeted Group size Instructional skills
WHY UNIVERSAL SCREENING?IDENTIFY STUDENTS IN NEED OF SUPPORT
Periodic and universal screening ensures that no students “fall through the cracks”
Strategic support: Students are placed in a program that provides moderate intervention and progress monitored every 2 weeks
Intensive support: Students are placed in an intervention that is intense and progress monitored every 2 weeks
WHAT UNIVERSAL SCREENERS ARE USED WITH SECONDARY STUDENTS IN TTSD?
MAZE OAKS Grades Attendance Office Discipline Referrals
(ODRs)
WHAT IS MAZE? Multiple-choice cloze task
Grade-level passage w/ every 7th word replaced by 3 word choices in parenthesis
Student reads silently and selects as many correct words as they can in 3 minutes
Curriculum-Based Measurement test that is “INDICATOR” of overall reading health Combines fluency, comprehension, and
all other subsumed reading skills Can be administered to a group;
scored later Easy & quick to administer, multiple
forms
EXAMPLE OF MAZE PASSAGE
WHY MAZE? Allows for screening/assessing ALL
students, ALL groups of Students, and School-wide literacy in time for intervention
Can use same test to monitor progress
Frequent progress monitoring increases academic achievement
Maze scores are a predictor of performance on OAKS AND NOW HS graduation
TABLE OF PROBABLE SUCCESS (MN)Maze Correct
Choices3 minutes
Writing: CWS minus IWS7 minutes
Probability of Passing
Minnesota Basic Skills Test
4 9 10%7 33 20%10 53 30%12 70 40%14 83 50%16 100 60%19 116 70%22 137 80%26 162 90%37 210 100%
Critical values corresponding to likelihood of passing 8th grade Minnesota Basic Skills Test – Doug Marston, et al.
MAZE/OAKS CORRELATIONS IN TTSD:SPRING MAZE OAKS BEST SCOREGrade Median
ScorePassage 1 Passage 2 Passage 3
6 .660 .607 .668 .636
7 .689 .615 .649 .706
8 .684 .634 .701 .661
• All correlations moderate to high• Relatively consistent across passages• Median correlations “in the middle”
MAZE AND PROBABILITY OF SUCCESS ON OAKS (TTSD)
Grade
Fall Maze Score Needed
for 85% Probability of Passing OAKS
Reading in Spring
Spring Maze Score Needed for 85%
Probability of Passing OAKS
Reading in Spring
6 20 33
7 20 33
8 21 37
VARIABILITY ACROSS PASSAGES
6th Grade 7th Grade 8th Grade0.00
5.00
10.00
15.00
20.00
25.00
30.00
Single Maze Passages vs Median Scores, Fall, 2008
# o
f Cor
rect
Wor
ds
MAZE SCREENING IN MS IN TTSD All students screened 3 times per year
Three, 3 minute tests will be given each time Screening assessment will occur in
(Matrix/Trek/LA class) Tests will be scored and data entered by
(Classified Staff/Parent volunteers/Electronically)
Data will be used for program evaluation and to place students in support
Students in support will be monitored
HOW DO YOU ANALYZE EFFECTIVENESS OF ACADEMIC PROGRAMS?
Focused on MAZE, OAKS and Grades Queried ESIS for a demographic file
with student name, ID #, ethnicity, program subgroup
Merged demographic file with data file for each measure
Created an Excel template organized by all subgroups
HOW DO YOU ANALYZE PROGRAM EFFECTIVENESS USING OAKS AND MAZE?
Core Data Analysis MAZE, OAKS, Grades blank template MAZE, OAKS, Grades Data Example
HOW TO IDENTIFY STUDENTS IN NEED OF INTERVENTION?
Initial Screening: Screening process initiated when academic
skills fall at or below the 35% on OAKS, AND/OR
In Middle Schools: Bottom 20% of students on the MAZE-CBM/Maze Benchmark scores
Screen further with San Diego Quick, SRAI, and curriculum placement tests, when appropriate
HOW TO IDENTIFY STUDENTS IN NEED OF INTERVENTION?
Post Screening Diagnostics and Placement: 6-Minute Solution--check for fluency &
accuracy; then, San Diego Quick to determine level of SRAI
to use; then, SRAI to gauge comprehension skills; then, Language! placement tests are
administered for students with the most significant reading needs
HOW DO YOU IDENTIFY THE LOWEST 20% OF STUDENTS WITH MAZE DATA?
Example Excel file
Example of IPAS School Student list
PROGRESS MONITORING
What is progress monitoring? What are the effects of progress
monitoring? How do you conduct progress
monitoring at the secondary level? How do you decide if the intervention
is working?
WHAT IS PROGRESS MONITORING? An on-going, systematic approach to
gathering academic and behavioral data to evaluate response to intervention, thereby
allowing data-based decision-making regarding instruction and learning outcomes on a frequent basis.
help schools establish more effective programs for children who have not benefited from previous programming.
In other words, it tells us if our interventions are working
EFFECTS OF PROGRESS MONITORING
• Progress monitoring has been extensively researched in Special Education (Fuchs & Fuchs, 1986)
• Students showed improved reading scores when teachers:• monitored their progress (+.70 effect size; ≈ 25th
50th %ile. Like it!)• graphed their reading scores (+.80 effect size. Love
it!)• used decisions rules to determine whether to make a
change in instruction (+.90 effect size. Gotta have it!)
EFFECTS OF PROGRESS MONITORING
CBM with decision rules (Fletcher, et.al., 2006) “goal raising rule” for students responding well:
effect size .52 (≈ 25th 40th %ile) “change the program rule” for students not
responding well: effect size .72 (≈ 25th 50th %ile) Results in teachers planning more comprehensive
reading programs
Additional support for effectiveness in General Education(Fuchs, et al., 1994)
PROGRESS MONITORING IN SECONDARY SCHOOLS
Select assessment tools Maze
Determine how often to progress monitor Every 2 weeks
Identify & Train staff to: Administer & score Reading Teacher Input & Analyze data Instructional Coordinator
Use the data Intervention planning at “20%” monthly meetings Student feedback
Sanford & Putnam (2007)
32
MEET MONTHLY TO CONSIDER…
1. Continuing (Student is making progress, but, continues to need support)
2. Intensifying (Intervention is not working and should be revised), or
3. Referring for Special Education Evaluation (Intensive intervention is proving unsuccessful)
4. Exiting (Intervention no longer needed)
EBIS Intervention Progress Monitoring
24
56
25
0
5
10
15
20
25
30
9/ 15/ 2
007
10/15/
2007
11/14/
2007
12/14/
2007
1/ 13/ 2
008
2/ 12/ 2
008
3/ 13/ 2
008
4/ 12/ 2
008
5/ 12/ 2
008
6/ 11/ 2
008
Monitoring/ Correct
Baseline/ Benchmark
Aimline
Trendline
Grade: 8
Probe Name: Maze Liz Ryan
Teacher Name: Goldman
School: The School of Rock
Corre
ct Re
spon
ses/
Min
.
Intervention: Soar
Intervention Change:
Language C
3-4 Data Points
Below the Aimline!
EBIS Intervention Progress Monitoring
24 5 6
12
16
20
26 25
0
5
10
15
20
25
30
35
9/ 15/ 2
007
10/15/
2007
11/14/
2007
12/14/
2007
1/ 13/ 2
008
2/ 12/ 2
008
3/ 13/ 2
008
4/ 12/ 2
008
5/ 12/ 2
008
6/ 11/ 2
008
Monitoring/ Correct
Baseline/ Benchmark
Aimline
Trendline
Grade: 8
Probe Name: Maze Liz Ryan
Teacher Name: Armeli
School: The School of Rock
Corre
ct Re
spon
ses/
Min
.
Intervention: Language C
Intervention Change:
Language C
Now that’s WORKIN’!
EBIS Intervention Progress Monitoring
24
56
76
8
25
0
5
10
15
20
25
30
9/ 15/ 2
007
10/15/
2007
11/14/
2007
12/14/
2007
1/ 13/ 2
008
2/ 12/ 2
008
3/ 13/ 2
008
4/ 12/ 2
008
5/ 12/ 2
008
6/ 11/ 2
008
Monitoring/ Correct
Baseline/ Benchmark
Aimline
Trendline
Grade: 8
Probe Name: Maze Rachael Cifuentes
Teacher Name: Helton
School: The School of Rock
Corre
ct Re
spon
ses/
Min
.
Intervention: Language C
Intervention Change:
Language C
3-4 Data Points Below the Aimline!
Consider SPED
Referral
EXIT FROM INTERVENTION WHEN:
Maze scores indicate 4 or more data points above the aimline AND are at or above the 50th percentile; AND
Grade+ scores are at or above the 5th stanine; AND
OAKS scores are at or above the 35th percentile
INTENSIFY READING INTERVENTIONS WHEN: Progress monitoring indicates 4 data
points below the aimline (maze). Slope is flat or decreasing AND scores
are below 50th percentile (maze). Grade+ scores at or below 3rd stanine.
MAKE A PLAN Select Measures Decide
Who will assess students? Who will record & graph the information? Who will make instructional decisions?
Get Training Establish
Decision rules Team Process Schedule for assessment
REFERENCES FOR MAZE AIMSweb
www.aimsweb.org Easy CBM
http://easycbm.com/ National Center on Student Progress
http://www.studentprogress.org/ Intervention Central
www.interventioncentral.org David Putnam, Jr., Ph.D.
QUESTIONS?