The Big Picture:“Getting it Going” and
“Keeping it Going”
Susan Barrett
Implementer Partner,
OSEP Center on PBIS
pbis.org
www.scalingup.org
Problem Statement
“We give schools strategies & systems for developing positive, effective, achieving, & caring school & classroom environments, but implementation is not accurate, consistent, or durable. Schools need more than training.”
Sustainability + Scaling
Organizational capacity for & documentation of accurate (90%) &
expandable implementation of evidence-based practice across desired context
(e.g., district, classroom, school-wide, nonclassroom)
over time w/ local resources & systems for continuous regeneration.
Education• 65 million kids
• 6 million teachers and staff
• 100,000 schools
• 3,143 counties
• 60 states & U.S. jurisdictions
Expansion 7,800 Schools !!!
Maryland 600 schools North Carolina 548 schools
Illinois 744 schools Colorado 405 schools
Florida 250 schools New York 322 schools
Michigan 248 schools Ohio 221 schools
New Hampshire 141schools West Virginia 215 schools
Virginia 210 schools Louisiana 285 schools
Missouri 300 schools Oregon 398 schools
Start with the end in mind…
• What will it take to have 100,000 replications that produce increasingly effective outcomes for 100 years?– Fixsen
Creating Implementation Capacity
• Start with too many overqualified people• “Generation 1” practitioners become:
– Generation 2 interviewers, trainers, coaches, evaluators
– Generation 3 administrators, directors, and leaders
– Generation 4 state and federal officials(Fixsen, 2008)
Educational Initiatives
• Guiding Principles (Coyne, 2008)– Promoting evidence based practices– Supporting change at the systems level
(feasible, consistent and relevant to local needs)
– Developing local capacity to sustain effective practices over time
Leadership Team
FundingVisibility Political
Support
Training Coaching Evaluation
Local School Teams/Demonstrations
SWPBS Implementers’ Blueprint Elements
MemoTo: School Administrators
From: District Administrators
In keeping with the new state initiative, this fall we will be implementing an exciting new district initiative of SNI in place of LYI. All in-service days previously scheduled for LYI will be rescheduled as staff development for SNI. The $500 for release time and materials for LYI will be discontinued and provided instead for SNI. By the way, you will need to create local SNI teams that meet weekly. The former members of your LYI team would be perfect for this new team. Your new SNI binders will be coming next week. Have a great year!!!
14 Initiatives• School Counseling Services• Second Step• FBA/BIP’s• School Health• Social Skills• Bully proofing• Anger Management• Student Intervention Plans• Behavioral Contracting• Character Education• 504 Plans/IEP• CICO• Responsive Classroom• Expanded School Mental Health
Competing or Coordinated
• Need for a framework, the anchor, for all school improvement efforts
• Common language, Common logic
School counseling services Second Step FBAs/BIPs School health services Social skills, bully proofing, and/or anger management groups
Student Intervention Plans
UNIVERSAL
TARGETED
INTENSIVE
School wide PBIS
Check-in/Check-out Section 504 Plans and/or IEPs
Health Education Voluntary State Curriculum Behavioral contracting
Alternative programs
Character Education
Responsive counseling School mental health services Bullying Prevention
Expanded School Mental Health Initiatives and Interagency Partnerships
School counseling services Second Step FBAs/BIPs School health services Social skills, bully proofing, and/or anger management groups
Student Intervention Plans
UNIVERSAL
TARGETED
INTENSIVE
School wide PBIS
Check-in/Check-out Section 504 Plans and/or IEPs
Health Education Voluntary State Curriculum Behavioral contracting
Alternative programs
Character Education
Responsive counseling School mental health services Bullying Prevention
Expanded School Mental Health Initiatives and Interagency Partnerships
Lead
ersh
ip T
eam
Stu
dent
Ser
vice
s T
eam
Fam
ily
ES
MH
Wrap T
eam
Empower Educators
• We need to be better consumers!
• Say “No”, unless . . .
Show evidence of need
Office referrals are increasing every day!!!
“Minor disrespect and disruption are increasing over time and are most likely during the last 15 minutes of our block periods when students are engaged in independent seatwork. This pattern is most common in 7th and 8th grades, involves many students, and appears to be maintained by peer attention.”
The playground is out of control !!!!The students won’t listen to anyone
and are fighting all of the time!!!
“Major & minor referrals have increased by 50% during lunch time on the playground. The referrals are mostly 4th and 5th graders, and disrespect and aggressive behavior are the highest problem behaviors. Peer attention is the motivation.”
DATA is your friend
• 3 Parts– Coherent office referral procedures– Computer Application– Decision Making
Using Data -Using Data -
• Does your Data give you an accurate Does your Data give you an accurate picture? Reliable? What is your confidence picture? Reliable? What is your confidence level? Do you share with all staff?level? Do you share with all staff?
• General Data Decision RulesGeneral Data Decision Rules• Are you able to make precision statements? Are you able to make precision statements?
Do you present to faculty to get buy in?Do you present to faculty to get buy in?• How do you know when to move “up the How do you know when to move “up the
triangle”?triangle”?
Improving Decision-Making
Problem SolutionFrom
To ProblemProblem
SolvingSolution
Information
Key features of data systems that work.
• The data are accurate and valid
• The data are very easy to collect (1% of staff time)
• Data are presented in picture (graph) format
• Data are used for decision-making• The data must be available when decisions need to be made
(weekly?)
• Difference between data needs at a school building versus data needs for a district
• The people who collect the data must see the information used for decision-making.
Why Collect Discipline Information?
• Decision making
• Professional Accountability
• Decisions made with data (information) are more likely to be (a) implemented and (b) effective
What data to collect for decision-making?
• USE WHAT YOU HAVE– Office Discipline Referrals/Detentions
• Measure of overall environment. Referrals are affected by (a) student behavior, (b) staff behavior, (c) administrative context
• An under-estimate of what is really happening
• Office Referrals per Day per Month
– Attendance– Suspensions/Expulsions– Vandalism
Office Discipline Referral Processes/Form
29• Coherent system in place to collect office
discipline referral data– Faculty and staff agree on categories– Faculty and staff agree on process– Office Discipline Referral Form includes needed
information• Name, date, time• Staff• Problem Behavior, maintaining function• Location
When Should Data be Collected?
• Continuously
• Data collection should be an embedded part of the school cycle not something “extra”
• Data should be summarized prior to meetings of decision-makers (e.g., weekly)
• Data will be inaccurate and irrelevant unless the people who collect and summarize it see the data used for decision-making.
Organizing Data for “active decision-making”
• Counts are good, but not always useful
• To compare across months use “average office discipline referrals per day per month”
0
10
20
30
40
50
60
70
Refe
rrals
Aug Sept Oct Nov Dec Jan Feb Mar Apr May Jun
School Month
Total Ref versus Ref/Day/MoNV High School
January 10
Change Report OptionsChange Report Options1.41.82.72.52.753.4900.000
Using Data for On-Going Problem Solving
• Start with the decisions not the data
• Use data in “decision layers” (Gilbert, 1978)– Is there a problem? (overall rate of ODR)– Localize the problem
– (location, problem behavior, students, time of day)
– Get specific
• Don’t drown in the data
• It’s “OK” to be doing well
• Be efficient
Is there a problem?
• Office Referrals per Day per Month
• Attendance
• Faculty Reports
SWIS Data Summary
06-07
SWIS summary 06-07 (Majors Only)1974 schools; 1,025,422 students; 948,874 ODRs
Grade Range
Number of Schools
Mean Enrollment per school
Mean ODRs per 100 per school day
K-6 1288 446 .34 (sd=.37)
(1 / 300 / day)
6-9 377 658 .98 (sd=1.36)
(1/ 100 / day)
9-12 124 1009 .93 (sd=.83)
(1/ 107 / day)
K-(8-12) 183 419 .86 (sd=1.14)
(1/ 120 / day
Interpreting Office Referral Data:Is there a problem?
• Absolute level (depending on size of school)– Middle, High Schools (> 1 per day per 100)– Elementary Schools (> 1 per day per 300)
• Trends– Peaks before breaks?– Gradual increasing trend across year?
• Compare levels to last year– Improvement?
0
5
10
15
20 A
ve R
efe
rrals
per
Day
Sept Oct Nov Dec Jan Feb Mar Apr May Jun
School Months
Office Referrals per Day per MonthLast Year and This Year
Middle School
N= 495
What systems are problematic?• Referrals by problem behavior?
– What problem behaviors are most common?
• Referrals by location?– Are there specific problem locations?
• Referrals by student?– Are there many students receiving referrals or only a small number of
students with many referrals?
• Referrals by time of day?– Are there specific times when problems occur?
Defining the ProblemWhat Problem Behaviors Are
Occurring?Referrals by Problem Behavior
0
5
10
15
20
Ta
rdy
Fo
rge
/Th
eft
Dre
ss C
od
eT
ob
acco
Alc
oh
ol
Co
mb
ustib
lB
om
bA
rso
nW
ea
po
ns
Oth
er
Un
kn
ow
nD
rug
sT
ech
Iap
pO
ut b
ou
nd
sG
an
gM
ino
rM
-Prp
tyM
-Oth
er
M-D
ress
M-T
ech
M-T
ard
yM
-Un
kn
ow
nIn
ap
p la
nL
yin
gV
an
da
lM
-Dis
rup
tio
nH
ara
ss
Pro
p d
am
M-I
na
pp
la
nM
-Co
nta
ct
M-D
isre
sp
tS
kip
Dis
resp
tA
gg
/Fig
ht
Dis
rup
tio
n
Nu
mb
er
of
Re
ferr
als
Referrals: 52
Clarifying the ProblemWhen Are Problem Behaviors
Occurring?Referrals by Time
0
2
4
6
8
10
12
14
16
7:0
0 A
M
8:0
0 A
M
9:0
0 A
M
10:0
0 A
M
11:0
0 A
M
12:0
0 P
M
1:0
0 P
M
2:0
0 P
M
3:0
0 P
M
4:0
0 P
M
5:0
0 P
M
Nu
mb
er
of
Refe
rrals
Referrals: 138
Clarifying the ProblemWhere Are Problem Behaviors
Occurring?
Referrals by Location
0
20
40
60
80
100
Com
mon
Bath
rm
Bus Z
n
Park
lot
Specia
l evt
Oth
er
Unknow
n
Off
ice
Locker
rm
Off
-Cam
pus
Sta
diu
m
Café
Lib
rary
Hall
Gym
Cla
ss
Bus
Ply
gd
Nu
mb
er
of
Refe
rrals
Referrals: 166
Clarifying the ProblemWho Is Engaging in Problem
Behaviors?Referrals by Student
0
2
4
6
8
10
12
1 2 3 5 10
11
12
13
17
18
20
21
22
23
24
25
34
35
37
38
39
40
42
47
48
49
50
52
54
56
57
60
62
65
72
73
75
78
79
85
86
88
89
93
94
96
97
99
100
95
84
91
98
87
67
26
55
51
36
41
Student No.
Nu
mb
er
of
Refe
rrals
Students: 60 Referrals: 96
Data-Based Decision Making
• Is there a problem?• What areas/systems are involved?• Are there many students or a few involved?• What types of problem behaviors are
occurring?• When are the behaviors most likely to occur?
• What is the most effective use of our resources to solve this problem?
CollectCollect and Useand Use
DataData
Review Status and
Identify Problems
Develop andRefine
Hypotheses
Discuss andSelect
Solutions
Develop andImplementAction Plan
Evaluate andRevise
Action Plan
Problem Solving Foundations
Team Initiated Problem Solving (TIPS) Model
Precision Problem Statements(What are the data we need for a decision?)
• Precise problem statements include information about the five core “W” questions.– What is problem, and how often is it
happening– Where is it happening– Who is engaged in the behavior– When the problem is most likely– Why the problem is sustaining
Primary versus Precision Statements
• Primary Statements– Too many referrals– September has more
suspensions than last year
– Gang behavior is increasing
– The cafeteria is out of control
– Student disrespect is out of control
• Precision Statements– There are more ODRs
for aggression on the playground than last year, and these are most likely to occur during first recess, with a large number of students, and the aggression is related to getting access to the new playground equipment.
What are the data you are most likely to need to move from a Primary to a Precise
statement?• What problem behaviors are most common?
– ODR per Problem Behavior• Where are problem behaviors most likely?
– ODR per Location• When are problem behaviors most likely?
– ODR per time of day• Who is engaged in problem behavior?
– ODR per student• Why are problem behaviors sustaining?
– No graph
What other data may you want?
• ODR by staff• ODR by IEP• ODR by grade• ODR by gender by grade• Faculty subjective impressions• Academic performance• Attendance• Information about home status
Decision Making
• Is there a problem?• What areas/systems are involved?• Are there many students or a few involved?• What kinds of problem behaviors are occurring?• When, where and with whom are these
behaviors most likely?
• What is the most effective use of our resources to address this problem?
Primary to precise
• Primary– Kids are noisy in the hallway going to reading
• Precise– Many second graders coming from reading
are too loud from room 13 to room 22 and their noise is maintained by peer attention. We also have very little adult supervision available
Using Data to Build Solutions
• Prevention: How can we avoid the problem context?– Who, When, Where– Schedule change, curriculum change, etc
• Teaching: How can we define, teach, and monitor what we want?– Teach appropriate behavior– Use problem behavior as negative example
• Recognition: How can we build in systematic reward for desired behavior?
• Extinction: How can we prevent problem behavior from being rewarded?
• Consequences: What are efficient, consistent consequences for problem behavior?
• How will we collect and use data to evaluate (a) implementation fidelity, and (b) impact on student outcomes?
Data BaseSWIS
Decision Making•Precision Statements•Build Solutions
Evaluation Questions
School Improvement
Goals and Objectives
Marketing and Dissemination•PresentationsNewsletters
Annual Reports
Educational Initiatives
• Guiding Principles (Coyne, 2008)– Promoting evidence based practices– Supporting change at the systems level
(feasible, consistent and relevant to local needs)
– Developing local capacity to sustain effective practices over time
Keep it Going!!
• Create Useful Systems of Data Collection• Establish Community of Practice
– We will be more efficient and effective if we share and learn together- Local NETWORKs
• Apply three tiered logic to your work• Showcase and celebrate outcomes!!
– Cost benefit
• Fidelity!!• Action Plan, process tools like BOQ, staff surveys!!!