data-based instruction for beginning writers: tools ...mo-case.org/resources/documents/spring collab...

2/19/2017

Lembke et al., 2016 1

Data-Based Instruction for Beginning Writers: Tools, Learning, and Collaborative Support

Assessment and Decision Making

Erica Lembke, Kim Moore, Alex Smith, Erica Mason, Sara Eldred, Carol Garman

University of Missouri

[email protected]

dbitlc.missouri.edu/

• DBI Team

– University of Missouri:• Carol Garman, Kim Moore, Abby Carlisle, Apryl Poch, Alex Smith

– University of Minnesota: • Kristen McMaster, Dana Brandes, Pyung-Gang Jung, Mike Herriges,

Jaehyun Shin, Kyle Wagner

• DBI-TLC is a $1.5 million Professional Development Grant funded by the Institute for Education Sciences, US Department of Education

Acknowledgements

What is DBI-TLC?

• DBI = Data-Based Instruction

• TLC = Tools, Learning, and Collaborative Support

• Development activities/partnerships with 2 school districts

• To support teachers’ use of Data-Based decision making (DBI)

• To improve outcomes for students, focusing on early writing development

Why Implement DBI?

• Some students do not respond to research-based interventions.

• These students require more intensive, individualized instruction.

• DBI provides a framework to individualize instruction.

• When teachers use DBI correctly, student achievement can improve.

Why Writing?

• Proficient writing is essential to learning in school and to later vocational success.

• Learning to write is critical to

– Overall literacy development

– Students’ ability to communicate what they know

– Integrating knowledge and thinking critically

And that’s where DBI comes in!

• In this project, we provide

– Tools for assessing young writers’ performance and progress

– Tools for providing early writing intervention

– A process for systematic decision-making to individualize instruction for struggling writers

2/19/2017


What Components of Writing are Addressed with DBI?

• Text generation

– Turning ideas into text (words, sentences, passages)

– Includes idea generation, word choice, content, text structure, genre

Text Generation

Transcription

Constrained by Attention and Memory

Self Regulation

Simple View of Writing

• Transcription

– Translating sounds, words, sentences, and passages into print

– Includes handwriting or typing, spelling, and mechanics

• Self-regulation

– What writers do to meet their writing goals

– Includes goal setting, planning, organizing, self-monitoring, self-evaluating, revising, and self-rewarding

Eight Steps of DBI

Overview of CBM

• Curriculum-based measurement (CBM) entails simple, efficient procedures that provide global indicators of student performance and progress in core academic domains.

Overview of CBM

• There are four key characteristics of CBM (Deno, 1985):

1. Reliable and valid

2. Simple and efficient

3. Easy to understand

4. Inexpensive

How is CBM different from other writing assessments?

CBM can…

• supplement informal diagnostics by providing different information

• be given frequently (weekly or daily)

• be relatively easy to administer and score

• measure very small increments of student progress

• signal the need for further diagnosis and intervention

How is CBM different from other writing assessments?

• It is important to remember that CBM is used as a general outcome measure, or a global indicator of writing ability, as opposed to a specific skill measure.

2/19/2017


CBM: An Index of Academic Health

Nu

mb

er o

f C

orr

ect

Seq

uen

ces

Combination

Text Generation

Self Regulation

Transcription

CBM for Beginning WritersTask Level of

Language

Grade

Level

Scoring Method

Word

Dictation

Word 1, 2, 3 Words Written (WW)

Words Spelled Correctly (WSC)

Correct Letter Sequences (CLS)

Picture-

Word

Sentence 1, 2, 3 WW

WSC

Correct Word Sequences (CWS)

Story

Prompt

Passage 2, 3 WW

WSC

CWS

CBM for Beginning Writers Word Dictation

• Word Dictation prompts are designed to capture transcription skills at the word level.

• Each Word Dictation prompt contains 30 words and is administered individually for 3 minutes.

Word Dictation Word Dictation Form Construction

WD Form Guidelines

• Total items

• Word types

• Stratification

• Counterbalancing

2/19/2017


Picture Word• Picture-Word Prompts are designed to capture

transcription and text generation at the sentence level.

• Each Picture-Word prompt contains 12 pictures/words and is group-administered for 3 minutes.

Story Prompts• Story Prompts are designed to capture transcription

and text generation at the passage level.

• Students generate a story in response to the Story (open-ended sentence) Prompt.

• Each prompt is group-administered for 3 minutes.

Administering CBM-W

• Be sure to have all materials needed prior to administration (timer, pencils, etc.)

• Administer each CBM task according to standardized administration directions for the prescribed length of time

* Administration directions are available on our DBI website

Administering CBM-W

• Standardized and validated administration procedures for Word Dictation:

“I would like you to write some words for me. I will read each word two times, and then you will write the word on your paper. When you are finished with one word, move down a line and get ready for the next word.”

Administering CBM-W

• Standardized and validated administration procedures for Picture Word:

“I’m going to ask you to write some sentences. You will write one sentence for each picture in your packet. Keep your pencils down. First, let’s name the picture on the front of your packet…”

Administering CBM-W

• Standardized and validated administration procedures for Story Prompt:

“I’m going to ask you to write a story. Before you write, I want you to think about the story. First, you will think, then you will write…Your story can be real or made up but needs to use the story starter. Think about the story you will write that starts like this…READ STORY STARTER.”

2/19/2017


Scoring CBM-W

• It is very important to score the student’s writing sample as soon as possible after administration using the specific scoring procedures as outlined

• We recommend using red and blue pencils for scoring.

*Our comprehensive Scoring Guide is available on our website

WD Scoring Video

Scoring Procedures (PW & SP)

• Words spelled correctly (WSC):

– The number of correctly spelled words written in the sample.

– Words spelled correctly are counted regardless of whether they are used correctly within the context of the sentence (e.g., “I saw a bare when I was camping.”)

– Some words (such as proper nouns) must be capitalized to receive credit

Scoring CBM-W

• Correct Word Sequences (CWS): Any two adjacent words that are correct in terms of spelling, grammar, capitalization, and punctuation

• Incorrect Word Sequences (IWS): Any two adjacent words that are incorrect in terms of spelling, grammar, capitalization, and punctuation

Scoring PW and SP

• Read the entire written sample.

• Count the number of words written.

• Underline (in red) incorrectly spelled words

• For Story Prompt, mark the end and/or beginnings of each sentence with a blue|.


• Example:

The man saw a bare in the tree and begn to run . | then the bare chased him.

2/19/2017



Example:

• ^The^man^saw^a bare in^the^tree^and

begn to^run. ^| then ^ the bare chased ^him. ^

WW = 17

WSC = 16

CWS = 12

IWS = 7

Picture Word Scoring Practice

Selecting a CBM Task

• Select a CBM Task using the form “Selecting an Appropriate CBM Task”

*Available on our website.

Selecting an Appropriate CBM Task

Administer and Score 3 Different CBM Prompts of the Same Measure and

Determine Median score• You can administer the prompts in one day or

across 3 days—preferably all within one week.

• To score the three baseline prompts, use the standardized scoring directions.

• To determine the median score, rank order the three scores from lowest to highest.

• The middle score is the median.

Initial Benchmarks

2/19/2017


Step 2: Set an Ambitious Long-Term Goal

a. Determine an end-date

b. Determine long-term

goal

c. Plot the long-term goal

d. Draw a goal line

Step 2a: Determine an End Date

• Determine the end-date of the instructional

period in which you will monitor progress.

– The instructional period is often the end of the

semester or school year.

– For students receiving special educational

services, the end date would often be the end of

the IEP cycle.

Step 2b: Determine Long-Term Goal

• Determine the level at which you expect the

student to perform at the end of this

instructional period.

• This level is the student’s long-term goal.

End-of-Year Benchmark

• Identify the end-of-year CBM benchmark for typically developing students at the grade level the student is being monitored.

• Use this benchmark as the long-term goal.

Step 2c &2d: Plot and Draw the Goal Line

• Plot the long-term goal point on the

last date of the instructional period.

• Draw a goal line by connecting the

baseline and the long-term goal.

Graphing

• Graphed data will provide you with a clear picture of a student’s progress

• Consider using Excel for graphing• The graph will allow you to:

– Set reasonable and ambitious goals,– Monitor the appropriateness of the student’s goal,– Judge the adequacy of the student’s progress,– Determine the effectiveness of the student’s writing

instructional program,– Use decision rules to make changes to the student’s

instructional program when needed.

2/19/2017


Step 3: Implement High Quality Instruction

Step 4: Monitor Progress Toward the Goal

Step 5: Use Decision Rules

a. Consider features of the

student’s performance and

progress (level, trend,

variability)

b. Make a decision based

on student’s data by

comparing the trend line to

the goal line

5a. Three Features For Evaluating Student Performance

• Level

• Trend

• Variability

Why are these features important?

• Indicators of student growth

Level

• Level . . . Compare student’s data points to the goal line

–Ask yourself:

• Are most points above the goal line?

• Are most points below the goal line?

• Are most points near the goal line?

Variability

• Variability . . . The amount of “bounce” between data points around the goal line

–Ask yourself:

• Is there a lot of variability between scores?

• Are the scores relatively stable?

• Great variability can make it difficult to determine if the student is making sufficient progress

2/19/2017


5b. Make a decision based on

student’s data by comparing

trend line to goal line

Let’s Practice! – Example #1

Phase Line 1Phase Line 2

Trend Line A

Trend Line B

0

5

10

15

20

25

0 2 4 6 8 10 12 14 16 18 20

Identify the following: Level, Trend, Variability, and Decision



Trend Line A Trend Line B

0

5

10

15

20

25

0 2 4 6 8 10 12 14 16 18 20




Trend Line A

Trend Line B

0

2

4

6

8

10

12

14

16

0 2 4 6 8 10 12 14 16 18 20


Step 6: Generate Hypotheses

Why is the student not

making sufficient

progress?

What instructional

change(s) need(s) to be

made?

• If an instructional change is needed:

– Use self-check questions

Generate Hypotheses

2/19/2017


Generate Hypotheses

• The student needs a change to

– Setting or format (instructional arrangement)

– Delivery (instructional explicitness)

– See Decision-Making Rubric

Generate Hypotheses

• The student needs a change to content

– See Transcription Scope and Sequence (Writing Intervention Toolkit), or other curricular materials

Generate Hypotheses

• The student needs a change in focus

– See Instructional Alignment Tool and Diagnostic Checklists (Writing Intervention Toolkit)

Step 7: Make an Instructional Change

Determine a change

using the Decision-

Making Rubric

Document change on

your Decision Log

Draw in next

intervention phase line

Insert Second Intervention Phase Line Step 8: Repeat Steps 4-7

2/19/2017


DBI in Action: A Teacher’s Perspective

Questions???

• Visit http://dbitlc.missouri.edu/ for more information

• Visit arc.missouri.edu to order the materials

Thank you for attending.

http://dbitlc.missouri.edu/

data-based instruction for beginning writers: tools ...mo-case.org/resources/documents/spring collab...

Documents