Taking EBPs to School

Developing and testing a framework for

applying common elements of evidence

based practice to school mental health

Aaron Lyon, Ph.D.

Kristy Ludwig, Ph.D.

Eric J. Bruns, Ph.D.

Ericka Weathers, M.A.

Elizabeth McCauley, Ph.D.

17th Annual Conference on Advancing School Mental Health

Salt Lake City, UT

October 25, 2012

Acknowledgements

Shannon Dorsey

Lucy Berliner

Doug Cheney

Ann Vander Stoep

Michael Pullmann

Janine Jones

Kelly Thompson

Nick Canavas

Seattle Public Schools

Public Health of Seattle & King County

School-based practitioners!

Collaborators: Funding:

• Institute of Education Sciences (R305A120128)

• American Psychological Foundation

• City of Seattle Office for Education

• Bill and Melinda Gates Foundation

Dr. Lyon is an investigator with the Implementation Research Institute (IRI), at the George Warren

Brown School of Social Work, Washington University in St. Louis; through an award from the National

Institute of Mental Health (R25MH080916-01A2) and the Department of Veterans Affairs, Health

Services Research & Development Service, Quality Enhancement Research Initiative (QUERI).

Overview

1. Background and Rationale

The Why and How of EBP in school-based

mental health (SBMH)

2. The Developing Protocol

The Brief Intervention for School Clinicians

(BRISC)

3. Initial input from Experts on BRISC

Results from Stakeholder Interviews and Nominal

Group Process (NGP)

4. Discussion

Evidence-Based Practice in School

Mental Health

School-based mental health (SBMH) offers

accessible services, particularly for historically

underserved youth (Burns et al., 1995; Kataoka et al.,

2007; Lyon et al., under review)

SBMH offers reduced stigma for service seeking (Nabors &

Reynolds, 2000)

SBMH can lead to improvements in a variety of

mental health, academic, and other functional

outcomes

E.g., improved GPA for users vs. similar non-users (Walker et

al., 2010)

EBPs in School: Room for Improvement

School-based services are unlikely to be

evidence-based (Evans & Weist, 2004; Rones &

Hoagwood, 2000)

Recent meta-analysis of SBMH programs for

low-income, urban youth revealed low levels of

effectiveness, some iatrogenic effects (Farahmand

et al., 2011)

Growing emphasis on increasing the use of EBP

in SBMH

Few interventions delivered in schools have been

designed for or tested in authentic education sector

service delivery settings (Wong, 2008)

Most “evidence-based” MH interventions developed in

more “traditional” outpatient settings

Simultaneously…

EBP developers have paid insufficient attention to the

school context and how it might influence effective

service delivery (Ringeisen et al., 2003)

EBPs in School: Room for Improvement

“It is important to emphasize that promotion of

effective mental health practices in schools…

involves more than simply ‘trumpeting’ the

selection of ‘evidence-based’ approaches, most of

which have not been examined for their

effectiveness, palatability, durability, affordability,

transportability, and sustainability in real-world

school or clinic settings” (Paternite, 2005; p.660)

EBPs and SBMH: Getting beyond the rhetoric

EBP Transportability and SBMH

Commonly-cited concerns about the

applicability of EBP to new contexts are

relevant to SBMH

Substantial need for flexibility

Treatment engagement / duration variability

Ability of EBP to address the full range of client

problems (type and severity)

SBMH service accessibility may make concerns about the

cultural relevance of EBP even more important

Dubrow et al. (2004)

Leveraging Evidence in Novel Contexts: Integrative Approach

SBMH can be Enhanced using EBP in a Variety of Ways

Family Check-Up (Connell, Dishon et al., 2007; Dishon

et al., 2008)

Preventive intervention that combines elements of EBP (i.e.,

evidence-based assessent, Motivational Interviewing,

behavioral parent training)

Identifies at-risk youth

Brief intervention: Conducts family assessment and

provides feedback to families in three meetings

Assessment

Initial Interview

Feedback

Common Elements & Common Factors

Common elements

Generic components/procedures of treatment

(e.g., exposure, psychoeducation) cut across

distinct treatment protocols

Common factors

Personal and interpersonal components (e.g.,

alliance, therapist effects) common to all

interventions are responsible for treatment

outcomes

Barth, R. P., Lee, B. R., Lindsey, M. A., Collins, K.S., Strieder, F., & Chorpita, B. F. et al. (2011).

Evidence-based practice at a crossroads: The emergence of common elements and factors.

Research on Social Work Practice.

Distillation and Matching Model (DMM)

ESTs can be distilled into common practice

elements/modules and matched to client

characteristics

1. Distillation (interventions as composites of

strategies)

Technique identification

Evidence accumulation

2. Matching (summarizing relevant considerations for

intervention selection)

Gauge association between content and study

characteristics (e.g., client age, gender, ethnicity)

(Chorpita, Daleiden, & Weisz, 2005)

Modular Psychotherapy: An

Application of the DMM

Shifting the primary goal from “using evidence-

based practices” to “getting positive outcomes” (Chorpita, Bernstein, & Daleiden, 2008)

Modular therapies more acceptable to providers

than standard EBT (Borntrager et al., 2009)

More flexible than traditional manuals with

regard to the timing of Tx delivery (McHugh et al.,

2009)

More effective than standard-arranged EBT or

usual care (Weisz et al., 2012)

Implementation with school-based providers within an Expanded School MH framework (Weist et al., 2009)

Higher use of EBP, but no impact on practitioner attitudes or youth outcomes

Application in school-based primary care (Stephan et al., 2010)

Resulted in provider behavior change

SBMH providers with greater knowledge about common elements may provide higher-quality services (Stephan et al., 2012)

Modular psychotherapy pilot in Seattle’s school-based health centers (Lyon et al., 2011)

A Common Elements Approach may be Relevant to SBMH

Modular psychotherapy pilot (Lyon et al., 2011)…

7 SBHC counselors selected 66 students for tracking

Primary presenting problem:

Depression – 75%; Anxiety – 14%; Mixed Dep. & Anx. – 11%

487 Total sessions across 66 students

Mean # sessions per student = 7.4 (range: 1-24, median: 6,

mode: 3)

In 94% of sessions, students received at least one

standardized assessment measure

Modular Psychotherapy Pilot A Common Elements Approach may

be Relevant to SBMH

Pilot identified a need for a brief intervention

model (3-4 sessions) to maximize intervention-

setting fit

During 2009 modular psychotherapy pilot, modal

number of sessions was 3

Large caseloads, sole practitioner

Frequent disruptions

Engagement difficulties

Some clinicians struggled to determine which modules

to select/prioritize

Many students (60%+) with subclinical presentations

Applying Common Elements to a Brief, School-Based Intervention

BRIEF INTERVENTION FOR

SCHOOL CLINICIANS (BRISC)

Typical School-Based Approach Innovative Approach

Intervention is often crisis-driven

(Langley et al., 2010)

Structured / systematic

identification of treatment targets

Focused on providing nondirective

emotional support (Lyon et al., 2011)

Focused on skill building /

problem solving

Interventions do not systematically use

research evidence (Evans & Weist,

2004; Rones & Hoagwood, 2000)

All intervention elements are

evidence-based

Standardized assessments are used

infrequently (Weist, 1998; Lyon et al.,

2011)

Utilizes standardized assessment

tools for progress monitoring

Structure of BRISC

Weekly Standardized

Assessments

and Homework

Activities

Emotion

Identification

Psychoeducation 1) Depression,

2) Anxiety, or

3) Anger

Engagement/ Motivation

Enhancement

Communication

Analysis Problem Solving Cognitive

Restructuring Coping Strategies /

Mood Changing

Motivation

Assessment

Model Requirements

Systematic / structured

intervention

Adaptable/flexible (but

evidence-based)

intervention delivery

Efficiency

Engagement

Specific treatment

target identification

Modularized Approach

Stepped Care / Brief Treatment

Structure

Culturally-Informed Treatment

Engagement/Motivation Strategies

Problem Solving Orientation

Assessment and Monitoring

Intervention Elements

Original BRISC Components

BRISC Protocol: Session 1

Identify the problem and target emotion

Assess impact on functioning (i.e. school, peers, family)

Normalize problem/empathize with the experience and emotion/psychoeducation

Determine willingness to change/conduct a quick motivational assessment

Propose working together

BRISC Protocol: Session 2

Review problem/emotion and related

interference

Provide additional psychoeducation about specific emotion (i.e. depression, anxiety, anger)

Provide cognitive triad and problem solving framework

Select module and begin this session if time permits

BRISC Protocol: Session 3

Review cognitive triad and related problem

situations that occurred recently

Discuss new coping strategeis implemented

Implement module Problem solving

Communication analysis

Coping strategies/mood changing strategies

Cognitive restructuring

Assign practice exercise to try during the week

BRISC Protocol: Session 4

Review practice exercise

Assess outcome of implemented modules or identify difficulties with implementation

Discuss future implementation of strategy

Review tools and coping strategies

Create a plan for responding to future difficulties and negative moods

Refer for continued services if needed and/or connect with school/outside resources

Framework for BRISC Integration into Existing Systems

BRISC

Tier 3

Tier 2

Tier 1

BRISC Inputs (from the school & other systems) Seeks school staff (e.g., teacher, administrator) input /

assessment about presenting problems and optimal pull-out

timing for individual students

Promptly evaluates student academic

functioning to determine whether psych Sx are interfering

/ if academic Fx should be an explicit target

Assesses/understands

where BRISC fits into existing approaches to dealing with Bx’l health

and options (e.g., PBIS)

BRISC Outputs (into the school & other systems)

Develops individualized teacher-communication plans for students and

coaches them through their execution Communicates directly

with teachers and parents about BRISC

skill targets and methods of supporting them. Provides a post-BRISC

progress report to key school staff. Provides an everyday language description of

the BRISC program (for parents, teachers, etc.)

Referral to or coordination with intensive services during or following

(for MH) BRISC implementation

Links individual BRISC targets to relevant existing universal programs and communicates with program liaisons

Note: Tiers 1, 2, & 3 within the Education context are largely comparable to Primary, Secondary, and Tertiary Prevention

Indiv. Student Level

Indiv. Student Level

System Level

System Level

BRISC System Integration

Project Overview: Year 1

Study 1: Expert Input

Key Informant interviews

Nominal Group Process at a national Summit

Study 2: Initial feasibility testing (project personnel)

Analyze findings (behavioral change, response to

BRISC)

Revise BRISC protocol

Project Overview: Year 2 and Year 3

Study 3: Protocol

validation w/school based

mental health providers

Description of usual care

(24 cases)

BRISC training and trial

(24 cases)

Analyze findings

Revise BRISC protocol and

training

Study 4: Randomized

pilot study with School

Based Mental Health

providers

30 students BRISC

30 students TAU

Analyze findings

Prepare for larger

randomized study

RESULTS FROM KEY

INFORMANT INTERVIEWS CONDUCTED JUNE-JULY 2012

Methodology

Thirteen interviews conducted between June 2012

and August 2012

Six with national experts

Two with staff of Seattle Public Schools

Two with members of provider organizations

Two with staff from Public Health in King County

One with a SBMHC counselor

Data analyzed by UW research coordinator and

graduate student.

Atlas.ti was used to code data into themes.

Results of interviews

Theme1: Integration of Mental Health Into Schools

Theme 2: Implementing Research Based Mental

Health Treatment in Schools

Theme 3: Student Engagement and Cultural and

Linguistic Responsiveness

Theme 4: Monitoring and Feedback, Use of School

Data

Theme 1:

Integration of Mental Health Into Schools

Need to understand specific services in each tier available in the school to promote adequate integration of BRISC

“The success of the intervention is dependent upon the adequacy and understanding of Tier 1 supports.”

Need to know how will intersect with other providers of individual support/counseling (e.g., school psychologists)

How will the need for more intensive services be identified after the proposed four sessions.

Integration of Mental Health Into Schools Referral/Screening

Within each school, work with teachers and school

personnel to develop clear profile of target

students and referral processes

Important to teach school staff who / how to refer

in order to make referrals more systematic.

School teams and teachers could identify students

for referral based on academic and behavioral

issues.

Integration of Mental Health Into Schools Referral/Screening

“Mental health screening wouldn’t be effective in getting the right kids (the 15% with academic problems). Do outreach. Tell school staff and students about the school-based health center, mental health services, how therapy could be helpful, what kinds of problems I can help with. Many of the students

I treat are self-referrals or referrals from friends.”

Integration of Mental Health Into Schools Focus on Academics

Underscore the focus on improving both

social/emotional and academic outcomes

“It is not compelling to say that simply reducing anxiety will lead to

learning. Because the kid is probably still not learning. The problem we

have in mental health is that it is not its goal to improve learning.

Mental health in schools thus has an added burden.”

“It is hard to have an indirect influence on academics through the

relationship between emotions and academics.”

Theme 2: Implementing Research Based

Mental Health Treatment in Schools

Clinician training on how to use the modules and how to move between modules in individualized way is critical

Structured guidance or a protocol for how to refer out and how to collaborate with school staff also are necessities.

“Clinician has to be flexible and collaborative. Need to be a part of a team.”

“Clinician needs to be prepped to work in a school setting and the school prepped to integrate the effort of the clinician.”

Implementing Research Based Mental

Health Treatment in Schools BRISC Intervention content

BRISC “homework” (practice activities) must

be individualized to the student

support given to complete it – in school if

possible

Individualization is also key regarding the

modules that are used

Clarity needed on how to select the modules to

be used with each student

Implementing Research Based Mental

Health Treatment in Schools Individualization is crucial

“You need to be able to evaluate what you are going to

work on with each student. Is it academics or do you

need to clean up what is the barrier (depressed,

ADHD, needs glasses, has a learning issue).

We need to do a differential assessment. I am not going

to treat his depression. I am going to propose we

need an active education intervention. His depression

may lift if he does better in school.”

Implementing Research Based Mental

Health Treatment in Schools Strengths and Challenges

Seven respondents voiced support for the approaches

in the BRISC protocol:

Four core modules (4)

BRISC homework (3)

Psychoeducation (3)

Session 1 objectives (3)

Seven respondents addressed challenges that are

likely to be faced with BRISC.

BRISC is a radical practice change and not all clinicians will

buy-in (n=5)

Four sessions is too few to learn new skills (n=2)

Student Engagement and Cultural Linguistic

Responsiveness

Need to determine the types of students that are

appropriate as well as inappropriate for BRISC.

The intervention would be most appropriate for resilient

students; Students with poor attendance may not be

appropriate

Students with transient or situational problems may be

inappropriate for BRISC

Need to consider how to flexibly and appropriately

include families in the process

Student Engagement and Cultural Linguistic

Competence Considerations of Family Involvement

“Create an individualized option for families. They can come in or be on the speaker phone. Send parents a report of progress. This needs to happen with every

student.”

Monitoring and Feedback, Use of School

Data

Monitor a flexible mix of social/emotional and

academic outcomes, per the needs of the

student.

Develop an online database with the capacity

to monitor and provide feedback on an array

of variables

Monitoring and Feedback, Use of School Data

“Although mental health treatment may not directly affect academic performance, it is only one step away and academic outcomes such as completing homework, attending school, studying for a test, finishing a book

could easily be used as goals to set and monitor if it is a major concern that is identified.”

PRIORITY ACTION STEPS

FROM ADVISORS BRISC INTERVENTION

DEVELOPMENT SUMMIT

AUGUST 20-21, 2012, SEATTLE

Nominal Group Process at BRISC

Development Summit

Organized by four topic areas

2 groups per track, 7-8 people per group

Participants will be asked to give input on:

1. What is a strength of BRISC as presented thus far?

2. What is a concern/weakness or something you think has not been well

addressed?

3. What is your advice to the project? What is an action step you would

recommend to the project team?

After brainstorming, each group submitted top 3

recommendations

Large group then voted on highest priority action steps for the

BRISC team

Priority recommendations:

Integration of Mental Health into Schools

Stay small- keep focus on developing a targeted

intervention to exist in a tiered system (18 votes)

Ensure BRISC is connected to a building-level

oversight team (16 votes)

Clarify consistent “language” that will be used

around BRISC (16 votes)

Use focus groups: students, parents, and teachers

Try to develop terms that are as accessible – as if your

were describing BRISC to parents and youth

Priority recommendations: Evidence-Based

Treatment

Incorporate academic interventions into the modules

AND focus on monitoring academic success (17

votes)

Carefully plan how this will integrate into clinician

workflow and organizational structure (15 votes)

Operationalize cultural responsivity within an

evidence-based structure (14 votes)

Develop and apply implementation pre-conditions

(School and clinician level) (11 votes)

Priority recommendations: Student

Engagement

Define BRISC fidelity in a way that promotes

flexible intervention with students (15 votes)

Incorporate training and support for clinicians to

provide culturally responsive treatment (14 votes)

Integrate Motivational Interviewing/engagement

strategies as flexible components rather than a tool

or required module (13 votes)

Priority Recommendations:

Relevant Outcomes and Use of Data

Create a BRISC steering committee/project team that includes providers, school reps, families/students (22 votes)

Use brief measures that can facilitate student identification of academic and socio-emotional outcomes to focus on (e.g., Top Problems Checklist) (14 votes)

Align goals and measurement strategies with on-going school routines (12 votes)

Data inventory and make use of existing SPS data systems to extent possible (12 votes)

DISCUSSION

Discussion: Asking the Hard Questions

How best to build on previous research on modular approaches in schools (e.g., Weist et al., 2009; Stephan et al., 2010)

SBMH providers with greater EBP knowledge provided higher quality services, but…

Use of an EBP-based quality framework did not improve student outcomes

What more can we do?

How do we avoid “scope creep”

Want to attend to recommendations about the need to integrate fully into school context and RtI framework, but…

We also need to focus on developing an effective individual treatment delivery platform

Discussion: Asking the Hard Questions

Focusing on academic outcomes as a part of SBMH intervention How does a MH provider effectively do this? Measure academic progress?

Provide academic interventions?

Solving the confidentiality dilemma – sharing data with key individuals in a useful and respectful way

RtI / Tier integration: Understanding the full range of services available, and

knowing what to do with that information

Discussion: Asking the Hard Questions

Building youth engagement / commitment Can this be done in an initial session?

In 4 sessions?

Training / maintaining cultural sensitivity in BRISC What is the best way to “Incorporate training and

support for clinicians to provide culturally responsive treatment”

…while also maintaining a focus on using evidence based treatment elements?

HOW HAVE YOU

INCORPORATED EVIDENCE

BASED ELEMENTS OF

TREATMENT IN YOUR SMH

PROGRAM?

For More Information

Aaron Lyon: lyona@uw.edu

Kristy Ludwig: ludwik01@uw.edu

Eric J. Bruns: ebruns@uw.edu