assessing the instructional level for writing david parker, kristen mcmaster, and matthew burns
TRANSCRIPT
![Page 1: Assessing the Instructional Level for Writing David Parker, Kristen McMaster, and Matthew Burns](https://reader035.vdocuments.net/reader035/viewer/2022062404/5514b84d550346f06e8b6599/html5/thumbnails/1.jpg)
Assessing the Instructional Level for Writing
David Parker, Kristen McMaster, and Matthew Burns
![Page 2: Assessing the Instructional Level for Writing David Parker, Kristen McMaster, and Matthew Burns](https://reader035.vdocuments.net/reader035/viewer/2022062404/5514b84d550346f06e8b6599/html5/thumbnails/2.jpg)
Activity
1. Topic:– White: Describe why nuclear fission has
been easier to do than nuclear fusion.– Peach: Describe why this conference will
be useful for your practice.– Pink: Describe the events of your last
family vacation.
2. Pencils down: Think for 30 seconds3. Write!!!
![Page 3: Assessing the Instructional Level for Writing David Parker, Kristen McMaster, and Matthew Burns](https://reader035.vdocuments.net/reader035/viewer/2022062404/5514b84d550346f06e8b6599/html5/thumbnails/3.jpg)
Activity
• Count # Words Written
• Results:Sample White Peach Pink
1
2
3
4
5
Average
![Page 4: Assessing the Instructional Level for Writing David Parker, Kristen McMaster, and Matthew Burns](https://reader035.vdocuments.net/reader035/viewer/2022062404/5514b84d550346f06e8b6599/html5/thumbnails/4.jpg)
Activity
• The Findings:– Did the Peach and Pink writers write more?– Who was more on-task?
• Why’d we do it?– Simulate the right amount of challenge
• Think of the kids!!!
![Page 5: Assessing the Instructional Level for Writing David Parker, Kristen McMaster, and Matthew Burns](https://reader035.vdocuments.net/reader035/viewer/2022062404/5514b84d550346f06e8b6599/html5/thumbnails/5.jpg)
Overview
1. Introduction– Why writing?– Why instructional level?– Purpose of this study
2. Method– Who, what, how?
3. Results– What was found
4. Discussion– Why it matters, limitations, what next?
![Page 6: Assessing the Instructional Level for Writing David Parker, Kristen McMaster, and Matthew Burns](https://reader035.vdocuments.net/reader035/viewer/2022062404/5514b84d550346f06e8b6599/html5/thumbnails/6.jpg)
IntroductionWhy Writing??
National Report Cards on Writing, 2003; 2008
![Page 7: Assessing the Instructional Level for Writing David Parker, Kristen McMaster, and Matthew Burns](https://reader035.vdocuments.net/reader035/viewer/2022062404/5514b84d550346f06e8b6599/html5/thumbnails/7.jpg)
Introduction
Why does writing proficiency matter?
– Enhances learning in content area courses (Bangert-Drowns, Hurley, & Wilkinson, 2004).
– College Entrance, Job Obtainment/Performance (National Commission on Writing,
2004; 2005).
![Page 8: Assessing the Instructional Level for Writing David Parker, Kristen McMaster, and Matthew Burns](https://reader035.vdocuments.net/reader035/viewer/2022062404/5514b84d550346f06e8b6599/html5/thumbnails/8.jpg)
Introduction
Problem: Detecting writing problems in late elementary or middle school, makes it more difficult to remediate (Baker,
Gersten, & Graham, 2003)
Solution: Start Intervening Early!!!!
![Page 9: Assessing the Instructional Level for Writing David Parker, Kristen McMaster, and Matthew Burns](https://reader035.vdocuments.net/reader035/viewer/2022062404/5514b84d550346f06e8b6599/html5/thumbnails/9.jpg)
IntroductionEnter the Instructional Level!!!
First, some background knowledge
![Page 10: Assessing the Instructional Level for Writing David Parker, Kristen McMaster, and Matthew Burns](https://reader035.vdocuments.net/reader035/viewer/2022062404/5514b84d550346f06e8b6599/html5/thumbnails/10.jpg)
IntroductionWhat is the Instructional Level?
![Page 11: Assessing the Instructional Level for Writing David Parker, Kristen McMaster, and Matthew Burns](https://reader035.vdocuments.net/reader035/viewer/2022062404/5514b84d550346f06e8b6599/html5/thumbnails/11.jpg)
Introduction• Theoretical Foundation
– Vykotsky (1978)– Betts (1946)– Gravois & Gickling (BP-V; 2008)
• Measurement Tools– Curriculum-based Measurement (CBM; Deno, 1985; Marston, 1989)
• For early writers (Coker & Ritchey, 2009; McMaster, Du, Yeo, Deno, Parker, & Ellis, 2009)
• Assessment – Curriculum-based Assessment (Gickling &
Havertape, 1981; Gickling, Shane, & Croskery, 1989)
![Page 12: Assessing the Instructional Level for Writing David Parker, Kristen McMaster, and Matthew Burns](https://reader035.vdocuments.net/reader035/viewer/2022062404/5514b84d550346f06e8b6599/html5/thumbnails/12.jpg)
IntroductionEmpirical Findings
• Reading:– 93-97% correctly read words (Treptow, McComas, & Burns, 2007; Gickling
& Armstrong, 1978)
• Improved on-task behavior, task completion, and reading comprehension
• 4x Faster growth rates (Burns, 2007)
• Math:– 14-31 Correct Digits (2/3rd Graders); 24-49 Correct
Digits (4/5th Graders)• Highest growth slopes (Burns, VanDerHeyden, & Jiban, 2006).
![Page 13: Assessing the Instructional Level for Writing David Parker, Kristen McMaster, and Matthew Burns](https://reader035.vdocuments.net/reader035/viewer/2022062404/5514b84d550346f06e8b6599/html5/thumbnails/13.jpg)
Introduction
There is NO Instructional Level for writing!
Purpose: To identify potential estimates of the instructional level for writing.
![Page 14: Assessing the Instructional Level for Writing David Parker, Kristen McMaster, and Matthew Burns](https://reader035.vdocuments.net/reader035/viewer/2022062404/5514b84d550346f06e8b6599/html5/thumbnails/14.jpg)
MethodParticipants
– 5 classrooms from 2 urban schools– 85 1st grade students
• 51% male
• 41% White; 28% Black; 26% Hispanic
• 57% Free/Reduced Lunch
• 17% special education services
Setting– Classrooms
![Page 15: Assessing the Instructional Level for Writing David Parker, Kristen McMaster, and Matthew Burns](https://reader035.vdocuments.net/reader035/viewer/2022062404/5514b84d550346f06e8b6599/html5/thumbnails/15.jpg)
MethodMeasures1.Curriculum-based Measurements
– Two Types1. Picture-Word2. Sentence Copy
– Scoring Procedures1. Words Written2. Words Spelled Correctly3. Correct Word Sequences
2.Test of Written Language
![Page 16: Assessing the Instructional Level for Writing David Parker, Kristen McMaster, and Matthew Burns](https://reader035.vdocuments.net/reader035/viewer/2022062404/5514b84d550346f06e8b6599/html5/thumbnails/16.jpg)
Method
Picture-Word Prompt (McMaster, Du, & Petursdottir, 2009)
![Page 17: Assessing the Instructional Level for Writing David Parker, Kristen McMaster, and Matthew Burns](https://reader035.vdocuments.net/reader035/viewer/2022062404/5514b84d550346f06e8b6599/html5/thumbnails/17.jpg)
Method
Sentence-Copy Prompt (McMaster, Du, & Petursdottir, 2009)
![Page 18: Assessing the Instructional Level for Writing David Parker, Kristen McMaster, and Matthew Burns](https://reader035.vdocuments.net/reader035/viewer/2022062404/5514b84d550346f06e8b6599/html5/thumbnails/18.jpg)
MethodProcedure
– Weekly progress monitoring data• 12 weeks• Teacher-administered• Students practiced then completed prompts for 3
minutes
Fidelity and Agreement– Collected for teacher administration as well as
prompt scoring – Teacher administration fidelity: 100%– Agreement: generally > 90%
![Page 19: Assessing the Instructional Level for Writing David Parker, Kristen McMaster, and Matthew Burns](https://reader035.vdocuments.net/reader035/viewer/2022062404/5514b84d550346f06e8b6599/html5/thumbnails/19.jpg)
MethodData Analysis (an 8-step plan)1.Establish Reliability of Accuracy/Fluency Metrics
2.Establish Validity of Promising Metrics
3.Compute Growth Slopes
4. Identify top 1/3rd Slopes
5.Compute Mean Start for top 1/3rd Slops
6.Create Categories
7.Establish Reliability of Categories
8.Establish Validity of Categories
Part 1: Find promising measures and scoring procedures
![Page 20: Assessing the Instructional Level for Writing David Parker, Kristen McMaster, and Matthew Burns](https://reader035.vdocuments.net/reader035/viewer/2022062404/5514b84d550346f06e8b6599/html5/thumbnails/20.jpg)
MethodData Analysis (an 8-step plan)1.Establish Reliability of Accuracy/Fluency Metrics
2.Establish Validity of Promising Metrics
3.Compute Growth Slopes
4. Identify top 1/3rd Slopes
5.Compute Mean Start for top 1/3rd Slopes
6.Create Categories
7.Establish Reliability of Categories
8.Establish Validity of Categories
Part 2: Find Instructional Levels
![Page 21: Assessing the Instructional Level for Writing David Parker, Kristen McMaster, and Matthew Burns](https://reader035.vdocuments.net/reader035/viewer/2022062404/5514b84d550346f06e8b6599/html5/thumbnails/21.jpg)
MethodData Analysis (a 8-step plan)1.Establish Reliability of Accuracy/Fluency Metrics
2.Establish Validity of Promising Metrics
3.Compute Growth Slopes
4. Identify top 1/3rd Slopes
5.Compute Mean Start for top 1/3rd Slops
6.Create Categories
7.Establish Reliability of Categories
8.Establish Validity of Categories
Part 3: Examine promise of instructional levels
![Page 22: Assessing the Instructional Level for Writing David Parker, Kristen McMaster, and Matthew Burns](https://reader035.vdocuments.net/reader035/viewer/2022062404/5514b84d550346f06e8b6599/html5/thumbnails/22.jpg)
Table 1. Means, Standard Deviations, and Correlation Coefficients for Fluency and Accuracy Scores for Sentence Copy and Picture-Word Prompts and Accompanying Scoring Procedures.
Fluency Accuracy
Probe 2 Probe 3 Probe 2 Probe 3
Prompt
ProcedureM SD M SD r M SD M SD r
Picture-Word
Words Written 17.0 8.4 18.4 8.6 .71*
Words Spelled Correctly 13.4 7.7 15.0 8.6 .67* 76.1 23.4 77.8 24.6 .52*
Correct Word Sequences 11.9 8.6 13.1 9.1 .67* 54.2 28.6 55.9 26.5 .46*
Sentence Copy
Words Written 16.7 7.1 16.8 7.7 .71*
Words Spelled Correctly 12.8 6.6 13.3 7.1 .74* 74.6 25.1 78.8 19.7 .60*
Correct Word Sequences 11.9 7.3 12.6 8.1 .70* 59.8 29.6 64.6 26.7 .56*
![Page 23: Assessing the Instructional Level for Writing David Parker, Kristen McMaster, and Matthew Burns](https://reader035.vdocuments.net/reader035/viewer/2022062404/5514b84d550346f06e8b6599/html5/thumbnails/23.jpg)
Correlation with TOWL-3 Total
Prompt
Procedure
Fluency Raw
r1
Category
ρ1
Picture-Word
Words Written .32* .36*
Words Spelled Correctly .48* .46*
Correct Word Sequences .52* .50*
Sentence Copy
Words Written .26 .21
Words Spelled Correctly .42* .46*
Correct Word Sequences .46* .48*
Table 2. Criterion-related Validity Coefficients between Scoring Procedures for Each Prompt and the Test of Written Language-3 (TOWL-3) Total Score.
![Page 24: Assessing the Instructional Level for Writing David Parker, Kristen McMaster, and Matthew Burns](https://reader035.vdocuments.net/reader035/viewer/2022062404/5514b84d550346f06e8b6599/html5/thumbnails/24.jpg)
Prompt
ProcedureMean SD SE
Fluency Criteria
(3 minute probe)
Picture-Word
Words Written 14.46 8.68 1.64 11-18
Words Spelled Correctly 11.43 7.03 1.33 9-14
Correct Word Sequences 10.93 8.56 1.62 8-14
Sentence Copy
Words Written 16.25 6.58 1.24 14-19
Words Spelled Correctly 13.39 6.52 1.23 11-16
Correct Word Sequences 13.32 7.86 1.49 10-16
Table 3. Derivation of and Estimates for Fluency Instructional Level Criteria for Scoring Procedures within Prompt Types.
![Page 25: Assessing the Instructional Level for Writing David Parker, Kristen McMaster, and Matthew Burns](https://reader035.vdocuments.net/reader035/viewer/2022062404/5514b84d550346f06e8b6599/html5/thumbnails/25.jpg)
Probe 2 Probe 3
Frustration Instructional Independent Frustration Instructional Independent.
κPrompt-Procedure
N % N % N % N % N % N %
Picture-Word
Words Written 19 23.8 19 23.8 42 52.5 21 25.3 15 18.1 47 56.6 .46*
Words Spelled Correctly 24 30.0 18 22.5 38 47.5 23 27.7 12 14.5 48 57.8 .46*
Correct Word Sequences 30 37.5 20 25.0 30 37.5 29 34.9 17 20.5 37 44.6 .46*
Sentence Copy
Words Written 24 29.6 30 37.0 27 33.3 23 28.8 24 30.0 33 41.2 .37*
Words Spelled Correctly 29 35.8 24 29.6 28 34.6 29 36.2 28 35.0 23 28.8 .46*
Correct Word Sequences 32 39.5 25 30.9 24 29.6 31 38.8 20 25.0 29 36.2 .47*
Table 4. Number and Percentage of Fluency Scores Categorized as Frustration, Instructional, and Independent and Kappa Coefficients.
![Page 26: Assessing the Instructional Level for Writing David Parker, Kristen McMaster, and Matthew Burns](https://reader035.vdocuments.net/reader035/viewer/2022062404/5514b84d550346f06e8b6599/html5/thumbnails/26.jpg)
Correlation with TOWL-3 Total
Prompt
Procedure
Fluency Raw
r1
Category
ρ1
Picture-Word
Words Written .32* .36*
Words Spelled Correctly .48* .46*
Correct Word Sequences .52* .50*
Sentence Copy
Words Written .26 .21
Words Spelled Correctly .42* .46*
Correct Word Sequences .46* .48*
Table 2. Criterion-related Validity Coefficients between Scoring Procedures for Each Prompt and the Test of Written Language-3 (TOWL-3) Total Score.
![Page 27: Assessing the Instructional Level for Writing David Parker, Kristen McMaster, and Matthew Burns](https://reader035.vdocuments.net/reader035/viewer/2022062404/5514b84d550346f06e8b6599/html5/thumbnails/27.jpg)
Discussion
Conclusion:– Consistent with previous research for reading
(Burns, 2007; Gickling & Armstrong, 1978) and math (Burns, VanDerHeyden, &
Jiban, 2006), criteria are plausible that indicate a student will make optimal growth in writing skill.
Implications:– MORE research!– Instructional decision-making
![Page 28: Assessing the Instructional Level for Writing David Parker, Kristen McMaster, and Matthew Burns](https://reader035.vdocuments.net/reader035/viewer/2022062404/5514b84d550346f06e8b6599/html5/thumbnails/28.jpg)
DiscussionLimitations
– Conceptual issues• CBM (General Outcome Measure) vs. CBA (Specific Subskill
Measure)?• Material difficulty?
– Generalizability?? (only 1st graders?)– Criterion for “high-responders”– Ongoing research with early CBM-Ws
Future Research– Investigate effects of instructional level prospectively (vis.
Intervention)?– Which measure is most informative?– Appropriate criteria?
![Page 29: Assessing the Instructional Level for Writing David Parker, Kristen McMaster, and Matthew Burns](https://reader035.vdocuments.net/reader035/viewer/2022062404/5514b84d550346f06e8b6599/html5/thumbnails/29.jpg)
Questions?