Power of Policy: Establishing the Conditions for Successful MTSS

Daniel J. Reschly, PhDProfessor of Education and Psychology, Emeritus

Vanderbilt University

dan.reschly@gmail.com

Reschly MTSS 12

Vandy is#1 in Education

25 ConsecutiveLosing Seasons??Then9-4 20129-4 20132-10 2014

Vanderbilt Is NOT AFootball Power

Reschly MTSS 2

• I have none, but I am open to all offers

• I am not cheap, but I am reasonable

Conflict of Interest

Reschly MTSS 3

BASICS Multi-Tiered System of Supports

• Purpose: Improve results • Close gaps between expected/needed results and

current results• Pragmatic, focus on what works, not on theory or

even expert determined “best practices.”• Problem Solving • Differences

– RTI focused primarily on raising bottom half, NCLB (ESEA, 2002)

– MTSS focused on closing gaps broadly in education and human services settings

Reschly MTSS 4

Old Model: Refer-Test-Place

New Model: Prevention and Early Identification/Treatment

Reschly MTSS 5

Importance of Prevention

• “ There is substantial evidence with regard to both behavior and achievement that early identification and intervention is more effective than later identification and intervention.” Nat’l Academy Report on Disproportionality p. 5, Donovan & Cross, 2002

• “If antisocial behavior is not changed by the end of grade 3, it should be treated as a chronic condition much like diabetes. That is, it cannot be cured, but managed with the appropriate supports and continuing intervention.” (Walker et al., 1995, p. 6)

Reschly MTSS 6

Prevention continued

• Reading intervention with a 2nd grader is like changing the direction on a speedboat, with a 5th grader it is like changing the direction of an oil tanker. (Ed Week May 13, 2009, p.11

Reschly MTSS 7

KTG: Initial Sound Fluency Fall to January 05-06 Yr.

Benchmark: Winter KTG25 sounds correct/min.

New KTG Teacher and Traditional Instruction

Results for class are not satisfactoryMust improve curriculum, instruction or both

On all class-wide graphs look at level and progress

BenchmarkAka WaterLine

Reschly MTSS 8

KTG: Initial Sound Fluency Fall to January 05-06 Yr.

Benchmark: Winter KTG25 sounds correct/min.

Experienced Teacher Direct Instruction

Results for class are satisfactory. CanUse results to assess individual student risk

WaterLine

Reschly MTSS 9

Phoneme Seg. Fluency: Jan to May 05-06 Yr.

Benchmark: 35 correct

New KTG Teacher and Traditional Instruction

Results for class are not satisfactory. Must improve curriculum, instruction or both

WaterLine

Reschly MTSS 10

Phoneme Seg. Fluency: Jan to May 05-06 Yr.

Benchmark: May 35 per minute

Experienced Teacher Direct Instruction

WaterLine

Reschly MTSS 11

Nonsense Word Fluency: Jan to May 05-06 Yr.

Benchmark: 25 correct per minute

New KTG Teacher and Traditional Instruction

Results for class are not satisfactory. Must improve curriculum, instruction or both

WaterLine

Reschly MTSS 12

Nonsense Word Fluency: Jan to May 05-06 Yr.

Benchmark: 25 correct per minute

Experienced Teacher Direct Instruction

Results for class are satisfactory. CanUse results to assess individual student risk

WaterLine

Reschly MTSS 13

Second Grade Oral Reading FluencyBenchmarks: Early 2nd=42 WCM; Winter=71 WCM End of 2nd=90 to 95

??

Students needing greater Gen’l Ed monitoring and Interventions

Good results? Poor results? Level is unsatisfactory. Progress is good.Consider results at early 2nd grade

Reschly MTSS 14

Basics: MTSS Principle 1: Science

Principle I: Scientifically-based instruction and interventions matched to student needs + good fidelity + sufficient time to be effective

What is scientifically-based? Is this sufficient? Why? Response? Effective for all? “Programs” are not MTSS Difference between programs and MTSS

Reschly MTSS 15

Basics. Principle 1:Scientifically-Based (ESEA, 2002)

Scientifically-based research was defined in the law as, (A) applies rigorous, systematic, and objective procedures to obtain

valid knowledge relevant to reading development, reading instruction, and reading difficulties;

(i) employs systematic, empirical methods that draw on observation or experiment;

(ii) involves rigorous data analyses (iii) reliable across observers, researchers, student groups, settings

(iv) accepted by a peer-reviewed journal or approved by a panel of independent experts through a comparably rigorous, objective, and scientific review

Reschly MTSS 16

Basics MTSS Process cont.

• Principle II: Formative evaluation including,– Frequent progress monitoring (how frequent?)– Progress compared to goals (benchmarks, what?)– Decision rules applied to modify interventions

• Progress exceeds goals, raise goals• Progress does not meet goals, improve intervention• Constant interaction between interventions and progress

– Formative eval is main difference vs current practices

Reschly MTSS 17

Basics. Principle 2: Progress Monitoring and Formative Evaluation

• Producing a 0.7 to + 1.0 SD Effect– Robust effects of progress monitoring and formative

evaluation– Data, graphs, decision rules, changes are key differences

between the usual program and RTI process– Absent data, no accountability, students gains assumed and

ineffective programs continued– Data provide feedback to everyone----supports commitments

to improved results

Reschly MTSS 18

Basics. MTSS Process?

• Principle III: Data-based decision making using student RTI data, including – General education classroom

instruction/intervention, curricular success– Need for more intensive services, academic and/or

behavioral– Referral for specialized services– Special education eligibility– Annual review and special education exit

• Broadly, data-based decision making at multiple levels, but not just any data

Reschly MTSS 19

Barriers to MTSS Implementation Districts reported that the three primary

obstacles to implementing MTSS as are follows: Insufficient teacher training in

instruction and behavior interventions

Lack of intervention resources

Lack of data, knowledge, and skills for

tracking/chartingSource: Spectrum K12 Solutions, 2009, 2010, 2011, 2012, 201

Reschly MTSS 20

POLICY CHOICE: STATES

• Does State Policy Make A Difference?• Does achievement across states merely reflect

demographic and SES differences? • Educational instruction results are about the

same regardless of methodology.• Teacher results do not vary much, SES and

demographics are much stronger influences• CONSIDER NAEP results 1992-2013

Reschly MTSS 21

-5-4

-3 -3-2 -2 -2

-1 -1

01

2 2 2 23 3 3

4 4 4 4 4 4

6 6 6 6 6 67 7 7

8 8 8 8 89 9 9

1010111111

1213

17

19

21

-10

-5

0

5

10

15

20

25

NM AK OK WI

ME IA SD ND

WV VT MI

MO UT ID NE IL

MT

WY IN NH AZ OR SC TX

USA MN RI PA NJ

MA

OH

NV LA TN CT VA AR WA NY

MS

GA CO CA NC HI AL KY DE DC FL MD

NAEP 4TH Grade Reading Change 1992-2013

USA +6MI +1

Reschly MTSS 22

-5-4

-3-2

-1

01

23

45

67

89

1011

1213

1415

1617

1819

2021

-10

-5

0

5

10

15

20

25

NM AR OKWI

ME IASD

NDWV

VT MI IDMOUTNE

IL MTWY

AZ INNH

OR SCTX

USAMN

RI PANJ

MA

LA NVOH

AR CTTNVAWA

GAMSNY

CACO

AL HINC

DE DC FL MD

1992-2013 NAEP 4TH Grade Change

Reschly MTSS 23

NCES Reading Report Card 2013

• Categories– < Basic: Less than partial mastery of prerequisite

knowledge and skills fundamental to proficient work at the grade level

– Basic: Partial mastery of ……..– Proficient: Solid academic performance. Demonstrated

competency over challenging subject matter– Advanced: Superior performance

National Center for Educational Statistics (2013). Institute of Educational Sciences, US Department of Education, Washington DC.

Reschly MTSS 24

49

2047 50

2147

69 70

34

30

29

33 32

33

33

20 23

34

17

33

17 16

34

17 9 6

25

418

3 2 12 3 2 1 7

0

20

40

60

80

100

120NAEP 2013 Reading Levels by Group

AdvancedProficientBasic< Basic

Reschly MTSS 25

215

221

225

211

216

221 221223

217

221

224

216 216

220

223

219 219 219221

225

222

219221221

219

225 224

232

217

227

223

229

224226

221

200

205

210

215

220

225

230

235

USA IL IN IA MD MI MN NE NJ OH PA WI

Big Ten States NAEP 4th Grade Reading

199220032013

Reschly MTSS 26

221219

225 224

232

217

227

223

229

224226

221

205

210

215

220

225

230

235

USA IL IN IA MD MI MN NE NJ OH PA WI

Big Ten States 2013 NAEP 4th Grade Reading

Reschly MTSS 27

Revolution in Federal General andSpecial Education Legislation

1960-70s To 2000sAssistance Results[___________________________________________

]ESEA EHA NCLB/Procedures IDEA 2004Number Served Outcomes

Improvement

Reschly MTSS 28

Public Policy: What Is It?

• Public Policy: courses of action, regulatory measures, laws, and funding priorities concerning a given topic promulgated by a governmental entity or its representatives. Public policy includes constitutional law, legislative acts, executive branch regulations, and judicial decisions. All interact in dynamic ways.

• Values• Priorities• Actions

Reschly MTSS 29

WHY???

• WHY better results in AL than MI• What did AL do that MI did not do?• This is improvement. What about overall

level?• What could be done in Michigan state public

policy to promote better results

Reschly MTSS 30

Table Responses

• What could be done in Michigan state public policy to promote better results

1. 2. 3.4.5.6.

Reschly MTSS 31

What did states with significant gains do?

• MD?• FL?• AL• MA?• CO?• What are some laggard states doing now?

Reschly MTSS 32

State Policies that Produce Gains

• Standards focusing on key indicators of reading instruction

• Curriculum frameworks that identify what is to be taught, materials, instructional routines

• Assessments aligned closely to standards and curriculum frameworks

• View of teaching? Highly individualistic, creative, personal style OR ??????

Reschly MTSS 33

• Teacher preparation standards aligned with state standards

• Rigorous review of teacher preparation programs

• Reading instruction competency test (focused on SBRI and assessment of progress)

• Sanctions for programs that do not produce competent teachers.

State Policies that Produce Gains cont.

Reschly MTSS 34

Teacher Preparation• Teacher education reform is essential• Promising beginnings, but a huge gap exists between

what is known to be effective and what is taught.• Note history of medical education

• Flexner, A. ( 1910). Medical Education in the United States and Canada: A report to the . Carnegie Foundation for the Advancement of Teaching. Higher Education. Boston: The Merrymount Press. (downloaded on July 14, 2011 from http://www.carnegiefoundation.org/sites/default/files/elibrary/Carnegie_Flexner_Report.pdf)

Reschly MTSS 35

Perspective

• Avoid demeaning teachers. We need good teachers

• Avoid bashing teacher preparation. We need good preparation programs

• Reform is essential

Reschly MTSS 36

Centrality of Teachers and Teacher Preparation

Teacher effects are significant, especially for at-risk students and students with disabilities. Tennessee Value Added Assessment System: Three years of

highly effective teachers overcome effects of low socioeconomic status

Teacher qualifications (e.g., degree level) have modest to trivial effects.

Teacher practices have large effects. Research-based teaching practices exist but are not taught in

most teacher preparation programs. Improved teacher preparation and professional

development are prerequisites to improved achievement.

Reschly MTSS 37

Current Teacher Preparation??

• Note Secretary Duncan’s recent statements• Current teacher preparation largely dominated by

– Student centered philosophy—unstructured teaching– Radical constructivist, social constructivist, – Philosophy rather than science– Rejection of objective data, tests, progress monitoring,

etc.• Enormous changes needed in teacher preparation• Schools and publishing companies must do significant

amount of teacher preparationReschly MTSS 38

Focus on What Works Research Foundations from Meta-Analysis

Treatment Effect Size Applied Behavior Analysis + 1.00 Formative evaluation: CBM+

Graphing+Decision Rules+ Reinf. + 1.00 Explicit Instruction and Problem Solving + .70 to 1.50 Comprehension Strategies +1.00 Math Interventions +.60 to 1.10 Writing Interventions +.50 to .85 Matching instruction to learning styles?? 0.00 Note, these effect sizes are stable across cultural groups

Reschly MTSS 39Reschly RTI 40

40

What Does NOT Work (Kavale, 2007; Lloyd et al., 1999; Marzano, 2007)

Perceptual motor training Matching instruction to presumed cognitive

strengths Training cognitive weaknesses (e.g., working

memory) to improve achievement Special classes for students with high incidence

disabilities (exceptions?) Unstructured instruction with learners who

have limited prior knowledgeReschly MTSS 40

Reschly RTI 41

NZ 18.9

USA 9.9

OCEDAverage12.7

Ed Week 1-12-11, p. 14

Reschly MTSS 41Reschly RTI 42Reschly RTI 42

General Instructional Principle

• Instruction at student’s knowledge/skill level

LowerPriorKnowledge

Higher PriorKnowledge

Needs Complete, ExplicitSystematic

Can Profit fromIncompleteImplicitLess Structured

Reschly MTSS 42

John Hattie’s Work on “What Works”

20082011

2013Reschly MTSS 43

Teacher as Activator vs. Facilitator (updated list)

Teacher as Activator Teacher as Facilitator

Teaching students self-verbalization / questioning (.76)

Inductive teaching (.33)

Teacher Clarity (.75) Simulation and gaming (.32)

Reciprocal teaching (.74) Inquiry-based teaching (.31)

Feedback (.74) Smaller classes (.21)

Metacognitive strategies (.67) Individualized instruction (.22)

Direct instruction (.59) Web-based learning (.18)

Mastery learning (.57) Problem-based learning (.15)

Providing worked examples (.57) Discovery methods in math (.11)

Providing goals (.50) Whole language (.06)

Frequent / effects of testing (.46) Student control over learning (.04)

Graphic / Behavioral organizers (.41)

Reschly MTSS 44

Hattie Effect Sizes (> 800 Meta-Analyses)• Default Effect Size =.4 Standard Deviation• Which of these interventions exceed > .4?

• Acceleration (skipping one grade) Y N• Inquiry/Discovery Instruction Y N• Student-teacher relationships Y N• Whole language reading instruction Y N• Class Size Y N• Retention (one year) Y N• Classroom management Y N• Peer Tutoring Y N• Phonics Instruction Y N• Formative Evaluation Y N• Worked Examples Y N• Reciprocal Teaching Y N

Reschly MTSS 45

Hattie Effect Sizes (> 800 Meta-Analyses)

• Default Effect Size =.4 Standard Deviation• Which of these interventions exceed > .4? Effect• Acceleration (skipping one grade) Y N + .88• Inquiry/Discovery Instruction Y N + .31• Student-teacher relationships Y N + .72• Whole language reading instruction Y N + .06• Class Size Y N + .21• Retention (one year) Y N - .16• Classroom management Y N + .52• Peer Tutoring Y N + .55• Phonics Instruction Y N + .58• Formative Evaluation Y N + .90• Worked Examples Y N + .57• Reciprocal teaching Y N + .74

How did you do?

Reschly MTSS 46

Direct, Explicit Teacher Directed Instruction, http://rea.mpls.k12.mn.us/BEAT_THE_ODDS_-

_Kindergarten_Teachers.html

• Teach all elements of the task• Break task into components—as far as needed• How explicit? Explicit enough for the student to

make good progress– Teacher Models Skill, using multiple examples and

non-examples– Teacher and student perform task together– Student performs task with feedback– Student independently practices task to automaticity– Integrate skills with prior skills and competencies

• See YouTube Direct Instruction videosReschly MTSS 47

Importance of Scientifically Based Reading Instruction

• Reading is a gateway competence.• Referrals to special education are dominated by students

with reading problems.• Reading competence is related to several important

educational outcomes.– Science of reading instruction– Chall, 1967, 2000; National Reading Panel, 2000; Snow, Burns,

and Griffin, 1998– Florida Center for Reading Research (http://www.fcrr.org/)– The University of Texas at Austin, Vaughn Gross Center for

Reading and Language Arts (http://www.texasreading.org/utcrla/)

• Science of reading instruction markedly underutilized48Reschly MTSS 48

Scientifically-based Instruction in Reading

Reading Curricula content-Snow et al, 1998 Phonemic Awareness Phonics Fluency Vocabulary Comprehension PLUS Direct, systematic instruction Universal screening and formative evaluation

Problem of teacher preparation VU-TQ Center Innovation configurations, reading,

classroom behavior, inclusive services, learning strategies (Reschly, et al., 2007)

Reschly MTSS 49

Snow, C. E., Burns, M. S. &Griffin, P. (Eds.) (1998).

Preventing reading difficulties in young children. Washington DC: National Academy Press.

51

Chall, J. .S. (1967). Learning to read: The great debate. New York: McGraw-Hill.•Research review 1900-1965•Early Reading, K-3•Code vs Meaning Emphasis•Phonics or Whole Word•Code superior, especially forstruggling readers

Lamented the generally poorPreparation of teachers to teach reading

Reschly MTSS 51

15% 11% 7% 11%13%

43%

IHEs and SBRI Five Components

Sample N=72

5 Components• Phonemic

awareness• Phonics• Fluency• Vocabulary• Compre-

hension

Source National Council onTeacher Qualityhttp://www.nctq.org/nctq

N=11 N=8N=5

N=8 N=9

N=31

# of Components TaughtWell

5 4 3 2 1 0 52Reschly Reschly MTSS 52

Effects to Teacher Preparation on Mathematics Achievement (NCTQ Walsh)

Reschly MTSS 53

Early reading preparation (NCTQ WALSH

National n = 609 programs (UG and G)

Massachusetts n = 15 programs (UG and G)

71% of U.S. programs are not providing elementary candidates with practical, research-based training in reading instruction methods. Reschly MTSS 54

Preparation of Special Education Teachers in Scientifically-Based Reading

Instruction in 27 IHEs (Reschly et al., 2007)

PA

Phonics

V C

SI EIProgressMonitoring

SBRRNCLBPrinciples

FL

Integration UniversalScreening

%

Reschly MTSS 55

Reading Course Syllabi: Projects

Explain your philosophy of literacy. Develop bulletin board to motivate children to read. Produce journal explaining personal experience in

learning to read. Analyze the social justice implications of literacy

Reschly MTSS 56

Criterion: Approximately 2 SD below the mean, or a score of <70

Standard Score of 70 has a percentile rank of about 2

Reliable Teacher EffectsTop and Bottom 20%

Middle 40%

Top 20%Bottom 20%

VALUE ADDED

Reschly MTSS 57

Resources for Academic and Behavioral Interventions

Vaughn-Gross Reading Center, Sharon Vaughnhttp://www.texasreading.org/utcrla/

Florida Reading Center-Torgesen/Wagnerhttp://www.fcrr.org/

Progress Monitoring: http://www.studentprogress.org/default.asp

Intervention Central-James Wrighthttp://www.interventioncentral.org/

Center on Instruction (Reading, Math, Writing, etc)http://www.centeroninstruction.org/

Reschly MTSS 58

Levine Study and Conclusions

• Levine, A. (2006). Educating school teachers. Washington DC: The Education Schools Project. Downloaded December 10, 2006 from http://www.edschools.org/pdf/Educating_Teachers_Report.pdf

• A majority of teachers are prepared at the education schools with the lowest admission standards and least accomplished professors. p. 27.

• Education schools used as cash cows• Wild wild west of higher education

Reschly MTSS 59

Improving Teacher Preparation

• Facts re Teacher Preparation True False1. More teachers now than in 1985 T

F2. Current teachers are aging T

F3. Increased female dominance T

F4. Teachers are increasingly diverse T

F5. Academic ability of tchrs is increasing T F6. Significant attrition continues T

F Reschly MTSS 60

What can you do to improve teacher preparation in your district• Teaching is largely a local market• What institution(s) prepare the majority of

teachers in your district?• What can be done to influence these

institutions re SBRI?

Reschly MTSS 61 62

Innovation Configurations (IC) (Hall & Hord, 1987; Roy & Hord, 2004)

Vanderbilt University’s TQ Center Ics CEDAR Scientifically Based Reading Research (Smartt &

Reschly) Classroom Organization and Behavior Management

(Oliver & Reschly) Inclusive Practices (Holdheide & Reschly) Learning Strategies (Schumaker) Problem Solving (Reschly & Wood-Garnett) Mathematics (McGraner, VanDerHeyden, & Holdheide) Assessment (Hosp)

www.tqsource.orgReschly MTSS 62

SBRI Innovation ConfigurationSee HandoutKey Essential

ComponentNone

Code = 0Mention

OnlyCode = 1

Mention Plus Readings/

TestsCode = 2

Plus Assignments

Code = 3

Plus Supervised

PracticeCode = 4

SBRI and Federal Policy

Phonemic Awareness

Phonics

Fluency

Vocabulary

Reschly MTSS 63

Problem Solving Teacher Preparation Innovation ConfigurationSee HandoutKey Essential

ComponentNone

Code = 0

Mention Only

Code = 1

+ Readings/

TestsCode = 2

+Assignments

Code = 3

+ Supervised

PracticeCode = 4

Attitudes, Beliefs, Essential Inform.

Instructional Competencies

Clsm Organ & BehMgmt

Problem SolvingCompetencies

Collaboration in Classroom, School, and District RTI

Reschly MTSS 64

PROBLEM SOLVING CHART

Does the *%$# thingwork?

Don’t mess with it! You Idiot! Did you mess with it?

Does anyone

else know?

Will you catch hell?

Hide it!

You poor slob! Ignore it

Can you blame somebody else?

NO PROBLEM

Yes

Yes

Yes

Yes

Yes

No

No

No

No

No

Reschly MTSS 65

Application of Innovation Configurations

• Institution of higher education (IHE) faculty self-assessment, self-reflection, course improvement

• IHE department heads, deans, and other university administrators interested in ensuring high-quality instruction in teacher preparation programs

• State departments of education seeking to unify instruction statewide with common language and goals consistent with federal policy (e.g., Maryland and Colorado)

Reschly MTSS 66

Application of Innovation Configurations cont.

• Design of professional development • Professional association standards• State licensure and teacher education program

approval requirements

Reschly MTSS 67

Fidelity of Implementation• RTI Process, PBIS, SBRI, etc: Good results with good

fidelity, BUT• Fidelity cannot be assumed• Development of protocols, checklists• Daily checklist completed• Direct observation when results fall short of goals• Do we, educational professionals, need checklists??

Reschly MTSS 68

Gawande, A. (2009). The Checklist Manifesto: How to Get Things Right. New York: MacMillan.

Reschly MTSS 69

Gawande’s Checklist Manifesto

• Checklists: Force function, that is, remind us to do routine behaviors that are essential to accomplishing complex tasks

• Many examples– Aviation checklists-3 long checklists to get to runway– Medicine-routine administration of pediatric

immunizations, no distractions, 4 steps before – Medicine-surgery procedures (infection, bleeding,

anesthesia)

Reschly MTSS 70

Gawande’s Checklist Manifesto cont.

• Central line insertion in medicine: Tube to a major vein

• Routine Procedure, but infection rates too high—lead to enormous negative consequences

• Checklist developed that reduced infection rates by 66%– Wash hands thoroughly with anti-bacterial

soap– Clean patient’s skin with chlorhexidine

antiseptic– Sterile drapes over the patient– Wear mask, hat, sterile gown, gloves– Put sterile dressing over the insertion sit

Reschly MTSS 71

Gawande’s Checklist Manifesto cont.

• Surgery checklist-international study involving wide range of hospitals

• Three checklists regarding routine behaviors– Seven item checklist before anesthesia– Seven item checklist prior to incision– Five item checklist at the conclusion of the surgery

• Outcomes? Significant decrease in surgical complications

Reschly MTSS 72

Acceptance of Checklists by Professionals

• Mixed, often negative reactions: Reasons???• Survey of Gawande’s participants was generally

positive– Lower for surgeons, higher for nurses– 50% to 70% depending on specialty endorsed

continued use– Question? If you were the surgical patient, would

you want the checklist used?– Answer: 94% said yes

• Checklists work, but acceptance is mixed

Reschly MTSS 73

Discussion: Improving Fidelity

• Current district efforts?

• Acceptance of checklists?

Reschly MTSS 74

Commitment to Better Results

• Shared commitment• Potential for growth• Celebrate progress• Work collaboratively to improve results• Work hard AND have a good time

Reschly MTSS 75

Three things that are real: God, human folly, and laughter; The first two are beyond our comprehensionSo we must do what we can with the third. John F. Kennedy

Best wishes to you for a great meeting and year

President Kennedy on Humor

Reschly MTSS 76