1. 1. 1 Effective Assessment for Screening, Progress Monitoring, and Diagnosis Lynn S. Fuchs and Douglas Fuchs Vanderbilt University
2. 2. 2 Assessment Process of collecting information for the purpose of formulating decisions. Formal assessment: Objective measurements of abilities, skills, strategies Informal assessment: Based on inferences a professional draws as a function of unsystematic observations in the relevant context Both are important. Focus of this presentation: Formal Assessment
3. 3. 3 The Importance of Formal Classroom Reading Assessment Formal assessment helps the teacher: Identify which students are at risk for long-term reading problems. Identify which students are making inadequate progress in response to the core reading program. For students making inadequate progress, build effective individualized programs. Diagnose reading strengths and weaknesses on component skills to differentiate instruction. Provide students with appropriate reading material. Ensure ALL students achieve reading success to enhance learning across the subject areas.
4. 4. 4 In This Presentation 1. Draw distinctions between two formal assessment systems Mastery Measurement General Outcome Measurement 2. Explain three decisions made on the basis of General Outcome Measurement Screening Progress Monitoring Diagnosis
5. 5. 5 In This Presentation 3. Differentiate between two approaches to General Outcome Measurement Curriculum Sampling Approach Performance Indicator Approach
6. 6. 6 In This Presentation 1. Draw distinctions between the two major forms of formal classroom assessment Mastery Measurement General Outcome Measurement
7. 7. 7 MASTERY MEASUREMENT Tracks Mastery of Short-term Instructional Objectives Determines the sequence of skills in an instructional hierarchy For each skill, develops a criterion- referenced test To implement Mastery Measurement, the teacher
8. 8. 8 First-Grade Phonics 1. Letter recognition 2. Linking sounds to letters 3. Initial and final consonants 4. Short vowels 5. Digraphs and blends 6. Long vowel patterns 7. Other vowel patterns 8. High-frequency word recognition 9. Decoding
9. 9. 9 Multidigit Addition Mastery Test
10. 10. 10 Problems with Mastery Measurement Hierarchy of skills is logical, not empirical. Assessment does not reflect maintenance or generalization. Methods rely on single-skill testing. So , number of objectives mastered does not relate well to performance on important criterion measures like high-stakes tests.
11. 11. 11 General Outcome Measurement (GOM) was designed to address these problems GOM makes no assumptions about instructional hierarchy for determining measurement (i.e., GOM fits with any instructional approach) GOM incorporates automatic tests of retention and generalization
12. 12. Sampling performance on year-long curriculum for each GOM Avoids need to specify a skills hierarchy Avoids single-skill tests Automatically assesses maintenance and generalization SO THAT: GOM scores relate well to performance on high-stakes tests and true reading success
13. 13. 13 In This Presentation 2. Explain three decisions made on the basis of General Outcome Measurement Screening Progress Monitoring Diagnosis
14. 14. 14 What Decisions Do Teachers Make Using GOM Assessments? Screening Progress Monitoring Diagnosis
15. 15. 15 Screening What Is Screening? Measures students to identify those in danger of scoring poorly on end-of-year high-stakes tests and experiencing poor long-term outcomes. Identifies students early so that intervention is provided well before the end of the year. Purpose is to improve long-term outcomes.
16. 16. 16 Screening What Happens During Screening? A tool is selected. A cut-point is set. Students who score below the cut-point are designated as at risk for unsatisfactory long- term outcomes. Conducted once at the beginning of the year or periodically across the school year.
17. 17. Progress Monitoring What Is Progress Monitoring? Progress monitoring involves more frequent (e.g., periodic, monthly, or weekly) measurement of students reading performance. The frequent assessments are used to quantify the rate of improvement (the students progress) toward competence in the grade-level curriculum. Progress monitoring permits the teacher to evaluate the effects of instruction and, when instruction is producing inadequate results, the teacher uses the progress-monitoring data to redesign the instructional program in a timely way, to ensure strong outcomes by the end of the year.
18. 18. 18 Progress Monitoring What Happens With Progress Monitoring? Select a measure. Assess students frequently (weekly or monthly or periodically). For each student, graph scores against time. For each student, quantify rate of improvement (slope: increase per week or month or period). Use the information to inform instructional decisions. Determine whether student is improving adequately. If not, determine how to build a better, differentiated program.
19. 19. 19 What We Look For With GOM Progress Monitoring Increasing scores Indicate the student is becoming a better reader Flat scores Indicate the student is not profiting from the instructional program and requires a change in the instructional program
20. 20. 20 Sarahs Reading Progress 0 20 40 60 80 100 120 140 160 180 WordsReadCorrectly Sarah Smith Reading 2 Sep Oct Nov Dec Jan Feb Mar Apr May
21. 21. 21 Jessicas Reading Progress 0 20 40 60 80 100 120 140 160 180 WordsReadCorrectly Jessica Jones Reading 2 Sep Oct Nov Dec Jan Feb Mar Apr May
22. 22. 22 Diagnostic Assessment Tool used to measure a students strengths and weaknesses and to differentiate instruction. The goal is to identify productive targets for instruction. Conducted periodically.
23. 23. 23 In This Presentation 3. Differentiate between two approaches to General Outcome Measurement Curriculum Sampling Approach Performance Indicator Approach
24. 24. 24 Two GOM Approaches for Representing Year-Long Performance Method #1: The Curriculum-Sampling Approach Systematically sample items from the annual curriculum (illustrated in Math GOM) Method #2: The Performance Indicator Approach Identify a global behavior that correlates highly with important outcomes and serves as a proxy for competence with the many skills taught in the annual curriculum
25. 25. 25 The Curriculum-Sampling Approach The critical skills that constitute the curriculum are specified. The relative importance of each skill is determined. For each skill, assessment items are designed. The items are organized into the test such that the weight given to each skill on the assessment reflects the relative importance of that skill. Each alternate form of the test is created in the same way to represent the annual curriculum with equivalent difficulty.
26. 26. 26 Example: GOM Curriculum-Sampling Approach The 100-Point Skills Battery Administered on 3 testing occasions each year At each testing occasion, the test is a parallel form of equivalent difficulty, sampling the entire years curriculum in the same way, but each time with different items.
27. 27. 27 100-Point Skills Battery (Curriculum-Sampling Approach) Grade K Strand Items Weight Format Setting Oral Blending 5 5 Produced One-on-one Rhyming 5 5 Produced One-on-one Segmentation 5 5 Produced One-on-one High-Frequency Word Reading 10 10 Produced One-on-one Letter Recognition 10 10 Produced One-on-one Letter Sounds 10 20 Produced One-on-one Phonics Word Reading 10 20 Produced One-on-one Grammar, Usage & Mech. 5 5 Multiple Choice Group Vocabulary 5 10 Multiple Choice Group Comprehension 5 20 Multiple Choice Group
28. 28. 28 100-Point Skills Battery (Curriculum-Sampling Approach) Strand # Items Weight Format Setting Phonemic Awareness 5 5 Produced One-on-one Phonics 5 5 Produced One-on-one Decoding 10 20 Produced One-on-one High-Frequency Word Reading 25 25 Produced One-on-one Grammar, Usage and Mechanics 5 5 Multiple Choice Group Vocabulary 5 10 Multiple Choice Group Comprehension 10 20 Multiple Choice Group Spelling 5 10 Produced Group Dictation Grade 1
29. 29. 29 100-Point Skills Battery (Curriculum-Sampling Approach) Grade 2 Strand # Items Total weight Format Setting Phonics 5 10 Multiple Choice Group Word Analysis 5 10 Multiple Choice Group Spelling 5 10 Multiple Choice Group Grammar, Usage and Mechanics 5 10 Multiple Choice Group Vocabulary 5 20 Multiple Choice Group Comprehension 20 40 Multiple Choice Group
30. 30. 30 100-Point Skills Battery (Curriculum-Sampling Approach) Grade 3 Strand # Items Total weight Format Setting Phonics 5 10 Multiple Choice Group Word Analysis 5 10 Multiple Choice Group Grammar, Usage and Mechanics 5 10 Multiple Choice Group Spelling 5 10 Multiple Choice Group Vocabulary 5 20 Multiple Choice Group Comprehension 5 20 Multiple Choice Group
31. 31. 31 100-Point Skills Battery (Curriculum-Sampling Approach) A (gradually-increasing) screening cut-score is applied at each assessment occasion. Students scoring below the cut- point are identified as at risk for reading difficulties. Interventions are designed. The 3 scores collected across the school year are graphed to represent student progress for each student in the class. At each assessment occasion, the strand scores are used diagnostically to identify strengths and weaknesses for differentiating instruction.
32. 32. 32 Screening Cut-Points for 100-Point Skills Battery Gr K Gr 1 Gr 2 Gr 3 Benchmark 1 10 10 10 10 Benchmark 2 40 40 40 40 Benchmark 3 85 85 85 85
33. 33. 100-Point Skills Battery (Curriculum-Sampling Approach) A (gradually-increasing) screening cut-score is applied at each assessment occasion. Students scoring below the cut- point are identified as at risk for reading difficulties. The 3 scores collected across the school year are graphed to represent student progress for each student in the class. Students whose progress falls short are provided differentiated instruction or intervention. At each assessment occasion, the strand scores are used diagnostically to identify strengths and weaknesses for differentiating instruction or providing intervention.
34. 34. 34 The Performance Indicator Approach to GOM Assessment Tasks that potentially serve as an overall indicator of reading competence at a grade level are identified. Research is conducted to determine which task relates best to the important, long-term outcomes. Alternate forms for that best task are created so each alternate form is different but is equivalent, representing the same level of difficulty for that task.
35. 35. 35 The GOM Performance Indicator Approach Example: At 2nd grade, students read a 2nd-grade passage aloud for 1 minute. The score (number of words read correctly) represents the students overall reading competence. This serves as a direct measure of passage reading fluency, but more importantly, it functions as an indicator of overall reading competence. Students who score poorly when reading text aloud in a fixed time are the same students who have poor skill with Decoding Word recognition Vocabulary Comprehension Measures have been validated to reflect overall reading competence at the various elementary grades.
36. 36. 36 The GOM Performance Indicator Approach Kindergarten Letter-Sound Fluency Teacher: Say the sound that goes with each letter. Time: 1 minute p U z u y i t R e w O a s d f v g j S h k m n b V Y E i c x
37. 37. 37 Kindergarten Letter-Sound Fluency Alternate-passage stability (3 weeks): .92 - .94 Criterion validity with WRMT: .58 - .71 Predictive validity with CBM (Fall 1 to Spring 1): .68 Predictive validity with CBM (Fall K to Spring 1): .54 Predictive validity with TerraNova (Fall 1 to Spring 1): .53 Predictive validity with TerraNova (Fall K to Spring 1): .43
38. 38. 38 The GOM Performance Indicator Approach Grade 1 Word-Identification Fluency Teacher: Read these words. Time: 1 minute. two for come because last from ...
39. 39. 39 Grade 1 Word-Identification Fluency Alternate-passage stability (3 weeks): .97 Criterion validity with WRMT: .75 - .93 Predictive validity with CBM (22-30 weeks): .68 - .87 Predictive validity with TerraNova (22-30 weeks): .62 - .76
40. 40. 40 The GOM Performance Indicator Approach Grades 2-3: Passage Reading Fluency Number of words read aloud correctly in 1 minute on end-of-year passages
41. 41. 41 Jason Fry ran home from school. He had to pack his clothes. He was going to the beach. He packed a swimsuit and shorts. He packed tennis shoes and his toys. The Fry family was going to the beach in Florida. The next morning Jason woke up early. He helped Mom and Dad pack the car, and his sister, Lonnie, helped too. Mom and Dad sat in the front seat. They had maps of the beach. Jason sat in the middle seat with his dog, Ruffie. Lonnie sat in the back and played with her toys. They had to drive for a long time. Jason looked out the window. He saw farms with animals. Many farms had cows and pigs but some farms had horses. He saw a boy riding a horse. Jason wanted to ride a horse, too. He saw rows of corn growing in the fields. Then Jason saw rows of trees. They were orange trees. He sniffed their yummy smell. Lonnie said she could not wait to taste one. Dad stopped at a fruit market by the side of the road. He bought them each an orange. Passage Reading Fluency
42. 42. 42 Grades 2-3: Passage Reading Fluency Alternate-passage stability (3 weeks): .92 Criterion validity with WRMT: .70 - .89 Predictive validity with CBM (22-30 weeks): .72 - .86 Predictive validity with TerraNova (22-30 weeks): .65 - .72
43. 43. 43 These GOM Performance Indicators Produce accurate, meaningful information about students academic levels and their rates of improvement. Are sensitive to student improvement. Correspond well with high-stakes tests. When used to monitor progress and inform instructional decisions, result in improved student outcomes.
44. 44. 44 The GOM Performance Indicator Approach A (gradually-increasing) screening cut-score is applied at each assessment occasion. Students scoring below the cut- point are identified as at risk for reading difficulties. The 3 scores collected across the school year are graphed to represent student progress for each student in the class. At each assessment occasion, the strand scores are used diagnostically to identify strengths and weaknesses for differentiating instruction.
45. 45. 45 Grades K-1 Benchmark Cut-Offs for Fluency Assessment Grade K Grade 1 Benchmark 1 0 15 Benchmark 2 12 40 Benchmark 3 30 75 Type of Measure Letter Sounds High-Frequency Word Identification
46. 46. 46 Grades 1-3 Benchmark Cut-Offs for Passage Reading Fluency Gr 1 Gr 2 Gr 3 Benchmark 1 N/A 79 99 Benchmark 2 47 100 120 Benchmark 3 82 117 137
47. 47. 47 The GOM Performance Indicator Approach A (gradually-increasing) screening cut-score is applied at each assessment occasion. Students scoring below the cut- point are identified as at risk for reading difficulties. The 3 scores collected across the school year are graphed to represent student progress for each student in the class. At each assessment occasion, the strand scores are used diagnostically to identify strengths and weaknesses for differentiating instruction.
48. 48. 48 GOM Assessment Systems Can rely exclusively on curriculum-sampling measures Can rely exclusively on performance indicators Can integrate the curriculum-sampling and performance indicator approaches
49. 49. 49 Major Advantages Curriculum Sampling Maps nicely with high-stakes tests Provides strand scores for diagnosis of strengths and weaknesses Performance Indicator Takes less time, so can be administered more frequently, which is necessary for designing differentiated instructional programs
50. 50. 50 Integrating Curriculum Sampling Approach with Performance Indicator Approach Curriculum sampling - Used with all students periodically (3 times per year) For students who evidence risk on curriculum-sampling approach, performance indicators - Administered on weekly, with scores graphed against time. - This allows the teacher to monitor the effects of instruction more carefully for at-risk students and use the data to build more effective programs.
51. 51. Your Questions 51