module 1 part 1 : using data to problem-solve implementation issues

Module 1 Part 1 : Using Data to Problem-Solve Implementation Issues

Session 2 Phase I Team Training

Presented by the MBI Consultants

Acknowledgements

Much of the content and ideas of this workshop stems from the work of others. Special thanks to the work of Tim Lewis,

George Sugai, Rob Horner, Lori Newcomer, the professors at the University of Oregon, National Center on Positive Behavioral Interventions and Support, and the Quaglia Institute for Student Aspirations.

BE RESPONSIBLE - Make yourself comfortable & take care of your

needs - Address question/activity in group time before discussing “other” topics - Use your team time wisely - Return promptly from breaks BE RESPECTFUL - Turn cell phones to “off” or “vibrate” - Listen attentively to others BE PREPARED - Ask questions when something is unclear - Be an active participant

OUR EXPECTATIONS FOR TRAINING

Trainer will raise his/her hand

Participants will raise their hand and wait quietly

ATTENTION SIGNAL

PHASE I TRAINING MATRIX

FORMAT OF PRESENTATIONS

Lecture with slides

Workbook/work time

& Action plan

development

Workbook guides

Activity =

Please read =

Critical Components of MBI Commit to a common purpose and approach to

discipline—creating a safe and welcoming culture that includes student voice and family/community involvement

Establish and maintain team… with administrator support, participation and leadership

Establish a clear set of positive expectations and behaviors

Establish procedures for teaching expected behavior Establish a continuum of procedures for encouraging

expected behaviors Establish a continuum of procedures for

discouraging inappropriate behaviors Establish a system for using data to make

decisions, progress monitor, and problem-solve

MBI DATA

Use data to assess current status Self-assessment Survey (SAS)

Use data to assess implementation fidelity

Team Implementation Checklist (TIC) School-wide Evaluation Tool (SET) Benchmarks of Quality (BoQ)

Use data to assess impact (outcome) on students

Office Discipline referrals

PBISApps.org

Summary of Currently Available Online Tools

External Evaluation/Research:• SET for Tier 1• ISSET for Tier 2 and Tier 3

Internal Evaluation/Action Planning:• BAT for Tier 2 and Tier 3• TIC mostly for Tier 1, does have some on Tiers 2 &

3• BoQ for Tier 1• Early Childhood Benchmarks of Quality (ECBoQ) for

Tier 1 • SAS mostly for Tier 1, does have some on Tiers 2 &

3• Monitoring Advanced Tiers Tool (MATT)

Main Ideas

Decisions are more likely to be effective and efficient when they are based on data.

The quality of decision-making depends most on the first step (defining the problem to be solved)

Define problems with precision and clarity

Main Ideas

Data help us ask the right questions…they do not provide the answers: Use data to Identify problems Refine problems Define the questions that lead to solutions

Data help place the “problem” in the context rather than in the students.

Main Ideas

The process a team uses to “problem solve” is important: Roles:

Facilitator; Recorder; Data analyst; Active member Organization

Agenda: Old business (did we do what we said we would do); New business; Action plan for decisions.

What happens BEFORE a meeting What happens DURING a meeting What happen AFTER a meeting

Agenda, data summaryUpdates, identify problem, problem solveMinutes posted, tasks completed

TEAM PROBLEM-SOLVING METHODTEAM PROBLEM-SOLVING METHOD

TIPS II Meeting Minutes and Problem-Solving Action Plan FormToday’s Meeting: Winter MBI Training Date, time, location: Facilitator: Minute Taker:

Data Analyst: Next Meeting: Date, time, location: Facilitator: Minute Taker: Data Analyst:

Team Members (bold are present today________________________________________________________________

Information for Team, or Issue for Team to Address

Discussion/Decision/Task (if applicable) Who? By When?

Administrative/General Information and Issues

Implementation and Evaluation

Precise Problem Statement, based on review of data

(What, When, Where, Who, Why)

Solution Actions (e.g., Prevent, Teach, Prompt, Reward, Correction, Extinction,

Safety)Who? By When?

Goal, Timeline, Decision Rule, & Updates

Problem-Solving Action Plan

Agenda for NEXT Meeting

1. 2.3.

    Implementation and EvaluationPrecise Problem Statement, based on

review of data(What, When, Where, Who, Why)

Solution Actions (Prevent, Teach, Prompt, Reward,

Correction, Extinction, Adaptations, Safety)

Who? By When? Goal with Timeline

Fidelity of Imp measure 

Effectiveness of Solution/Plan 

    Not started Partially Imp Imp Fidelity Done

Goal Met Better Same Worse

Agenda for Today:

1. Analyze TIC and SAS Results to determine current level of implementation . Previously Defined Problems/Solutions (Update)

1

2

Decision-making at many levels

Whole school Small groups or school areas Individual student

------------------------------------------------------- Same basic process

Carol

SWIS

Implementation Measures

Interpreting the TIC and SASEarly Childhood Benchmarks of Quality

(ECBoQ)*

Team Implementation Checklist - Version 3.1

Sugai, G, Horner, R.H., Lewis‐Palmer, T.,& Rossetto Dickey, C. (2011). Team Implementation Checklist, Version 3.1, University of Oregon. Retrieved fromhttps://www.pbisapps.org/Applications/Pages/PBIS-Assessment.aspx

22 numbered items that focus on the universal level of prevention (Tier 1) although it has 3 items to assess progress toward implementing Tiers 2 and 3.● For example, item 20 asks if

“Personnel are able to provide behavior expertise for students needing Tier II and

Tier III support.”

Does your team need to complete a current TIC?

If so, complete now!

Pick one item that is “in Progress” or “Not Yet Started” and develop an action statement for that item (Who will do What by When)*Remember this is on your TIPS Meeting Form*

Self-Assessment Survey (SAS)

Todd, A.W., Sugai, G., and Horner, R.H. (2003). Effective Behavior Support (EBS) Survey. University of Oregon, Retrieved from

https://www.pbisapps.org/Applications/Pages/PBIS-Assessment-Surveys.aspx#sas

The SAS is an annual assessment used by schools to identify the staff perception of the implementation status and improvement priority for school-wide, classroom, non-classroom and individual student systems. Results of the SAS are effective in identifying the staff priorities for Action Planning.

COMMON FEATURES TIC AND SAS

Establish Commitment Establish and Maintain Team Self‐Assessment Prevention Systems (defining and teachingexpectations, rewarding appropriate behavior,

andresponding to violations) Classroom System (a new category) Information System Build Capacity for Function‐based Support

Graphics from PBISApps:Team Implementation Checklist Total Score

(Percentage of Points)

TIC Subscale Report

Subscales

Reg

ular

M

eetin

gs

Cla

ssro

om

Dis

cipl

ine

dat

a

Inte

nsiv

e In

divi

dual

Team Implementation Checklist Items

Establish & Maintain Team

3. Team Established (Representative).

2

4. Team has regular meeting schedule, effective operating procedures

2

5. Audit is completed for efficient integration of team with other teams/initiatives addressing behavior support.

1

SAS-Did you remember to bring yours?

Measures implementation level and priority for improvement using 46 items across four systems:

• School‐wide (18 items)• Specific Setting (9 items)• Classroom (11 items)• Individual Student (8 items) Often is used as part of staff development with

initial PBIS training. Measures “Priority for Improvement,” as well as

staff perceptions of level of implementation (“in place” or “partially in place” or “not in place”) Many schools continue to use this as a measure of

progress over time.

Graphics from PBISApps: Self-Assessment Survey: Overall Status

 In Place  59%

Partial 35%

Not 6%

High 19%

Medium 31%

Low 50%

Subscale Report

Expectations Defined

Expectations Taught

Reward System

Violations System

Monitoring Management District Support

Implementation

Average

11/11/2011 97% 100% 91% 76% 80% 67% 64% 77%

Item Analysis Current Status

Feature Improvement Priority

In Place

Partial

 Not  

System: schoolwide High Medium Low

96 % 4 % 0 % 1. A small number (e.g. 3-5) of positively and clearly stated student expectations or rules are defined.

21 % 17 % 63 %

65 % 25 % 10 % 2. Expected student behaviors are taught directly.

44 % 28 % 28 %

50 % 46 % 4 % 5. Consequences for problem behaviors are defined clearly.

28 % 40 % 32 %

38 % 38 % 25 % 6. Distinctions between office v. classroom managed problem behaviors are clear.

33 % 29 % 38 %

SAS Report Color Code Key for Current StatusWhite = 80% or more of staff marked item as “In Place”.Yellow = 51-79% of staff marked item as “In Place”.Red = 0 -50% of staff marked item as “In Place”.Red Review:Red Review:

Review the features in red and determine 2 features which staff identified as “High” priority. List here:Yellow Review:Yellow Review:Review the yellow features and identify 1 or 2 features the team could address to easily improve implementation. List here:White Review:White Review:Review the white features and 1 or 2 features that your school can celebrate.

Self-Assessment Survey Report Analysis

Schoolwide System OnlySchool: Date: Use the worksheet below to discuss the results of your school’s Self-Assessment Survey.

1. Self-Assessment Survey Report Analysis

2. Review and Update Team Action Plan 3. How will you share the SAS Results with Staff? (Add to your team action plan)

Work Time

Complete the “Problem-Solving Implementation Issues – Part 1”

section of your workbook