Preventing Child Maltreatment: Using successful implementation strategies and tracking outcomes for children and families

Professor Diane DePanfilisUniversity of Maryland School of Social Work

2

Key Concepts in Today’s Session

• Implementation science

– Implementation in the “real world” is complex

– Stages of implementation

– Implementation teams

– Implementation drivers

• Illustrations from implementing Family Connections in NYC and beyond

Why is this important?

3

When we implement efficacious practices, it is important that we focus on implementation, not just on whether outcomes are achieved as intended.

4

Implementation Science

“Children and families cannot benefit from interventions they do not experience”

This is called the Implementation Gap

5

What does this mean we do differently?

• Consider what works• Diligently apply what

works• Have teams ready to

break through barriers

6

What is “Implementation?”

• A specified set of activities designed to put into practice an activity or program.

• A strategic, purposeful approach, not a one-time event, for making a change.

• A process for bridging the gap between “what we know” and “what we do.”

Brief context:Family Connections

What is Family Connections?

1See DePanfilis & Dubowitz, 2005; DePanfilis 20009; DePanfilis, Filene, & Brodowski, 2009

Family Connections (FC)

• FC Core Components– Intake– Outreach & engagement– Emergency/Concrete services– Comprehensive family assessment

(assessment instruments)

– Outcome driven service plans with SMART goals

– Tailored change focused intervention• Minimum of 1 hour per week of change

focused interventions• Advocacy/service facilitation

– Multi-family activities (optional)– Service plan evaluation (at least every

90 days)– Case closure

Is a multi-faceted community-based program that works with vulnerable families in their homes, in the context of their neighborhoods, to help them meet the basic needs of their children to prevent child maltreatment and achieve safety, well-being, and permanency outcomes.

Brief HistoryOriginal Demonstration• 1996-2002

Federally Funded Replication• 2003 – 2009

Adaptations with New Target Populations (2009 – Present)• Grandparents• Trauma• Safety

Wider Replication Nationally• 2009 - 2014

10

2 Administration for Children’s Services (ACS) Initiative on Evidence Based Practice in Child Welfare

Expression of Interest for Funded Preventive Services Programs to convert programs to an Evidence Based, Evidence Informed, or

Promising Practice. (June 2012)

11

Nine Preventive Programs Were Approved to Implement Family Connections

• Astor Services for Children & Families

• Cardinal McCloskey Community Services

• Catholic Guardian Services

• Episcopal Social Services

• The Family Center• Leake and Watts

Services, Inc.• New Alternatives for

Children• Northside Center for

Child Development• St. Dominic’s Home

April 2013

12

13

NYC - FCC Logic Model

Inputs

ACS Funding

Eligibility Criteria and

Referral Procedures

TrainedStaff &

LeadershipTeams

ImplementationPlanning

Intermediate Outputs

Emergency Assistance(initial & ongoing)

Comprehensive FamilyAssessment (4-6 weeks)

Outcome Driven Service Plans (SMART goals) (by week 6-8)

Change Focused Intervention-Minimum 1 hour per week

change focused intervention-Advocacy/service facilitation

Final

Outputs

1,536Target

Families

Short-Term/ Intermediate Outcomes

Increase Protective Factors

• Parenting Attitudes• Family Strengths/Functioning• Social Support• Family Resources• Home Safety & Stability

Decrease Risk Factors

• Parental Stress• Caregiver Risks/Needs (FASP)• Child Risks/Needs (FASP)• Family Risks/Needs (FASP)

Long-Term Outcomes

Increase childSafety

(Prevent Child

Maltreatment)

AchievePermanency

(PreventPlacement)

Intake/Outreach/Engagement

Evaluation of Change/Case Closure(90 days post Plan)

14

Areas of Assessment (Instruments)BASELINE ONLY (used for assessment but not to measure change over time)

• DEPRESSIVE SYMPTOMS (Center for Epidemiologic Studies – CES-D)

• ALCOHOL/DRUG PROBLEM SCREENING (CAGE-AID)

• DOMESTIC VIOLENCE SCREENING (ACS SCREEN)

• READINESS FOR CHANGE (Readiness to Change Index –REDI)

• PARENTING ATTITUDES (Adult-Adolescent Parenting Inventory – AAPI-2)

• SOCIAL SUPPORT (Support Functions Scale- SFS)

• FAMILY RESOURCES (Family Resource Scale – FRS)

• FAMILY FUNCTIONING (Family Functioning Style Scale-FFSS)

• PARENTING SRESS (Parenting Stress Index-Short Form – PSI-SF)

• HOME SAFETY-STABILITY (Developed for FCC)

Using Computer Assisted Self-Interviews

• Enhance empowerment and engagement

• Illustrate “success” to families and to staff

• Enhance motivation to change

15

16

3 Implementation Science

Stages, Teams, & Drivers

17

Implementation Science Purpose

Implementation Science provides frameworks for successful implementation so that:– Children and families benefit from interventions and

experience positive outcomes

– Workers are supported to learn new skills, manage change, and identify barriers

– Organizations are responsible for creating hospitable environments for change and supporting workers

NATIONAL IMPLEMENTATION RESEARCH NETWORK (NIRN)

Fixsen, D. L., Naoom, S. F., Blase, K. A., Friedman, R. M. & Wallace, F. (2005). Implementation Research: A

Synthesis of the Literature. Tampa, FL: University of South Florida, Louis de la Parte Florida Mental Health

Institute, The National Implementation Research Network (FMHI Publication #231).

Download all or part of the monograph at:http://www.fpg.unc.edu/~nirn/resources/detail.cfm?resourceID=31

Framework used by ACS & the NYC-FCC

19

What works to support successful implementaton?

• The combination of:

– Stage-matched implementation activities

– “Drivers” or core components that promote competency, organizational support, and leadership

– Teams that provide organized capacity to lead and support the change effort

20

Review of Implementation Science Core Concepts

Overview of implementation stages & drivers

1Stages

2Drivers

3 Exemplars

21

Implementation Stages

Most implementation initiatives take between 2-4 years

1

22

Stages of Implementation

Full Implementation(Sustainability & Effectiveness)

Initial Implementation

Installation

Exploration

23

2 Implementation Drivers

The core components that will increase the effectiveness of implementation

See Kaye, DePanfilis, Bright, & Fisher, 2012

24

Staff Selection

Training

Coaching (and

Supervision)

Performance Assessment (Fidelity)

Systems Intervention

LEADERSHIP

Decision Support Data System

Integrated & Compensatory

CO

MPETEN

CY

ORG

AN

IZATIO

N

Facilitative Administration

© Fixsen & Blase, 2008

Implementation Drivers

25

Overall Objectives for Building Competency for NYC Family Connections’ Practice

• To build knowledge and skills related to the core components of Family Connections.

• To practice working with “sample” families starting at Intake and ending at Case Closure.

26

Overview of Initial Training and Coaching Plan

_________________________________________________________________________

April May June July

Initial OrientationApril 9 & 12

Supervisors and

Directors Training

April 22-23 & 29-30

Phase 1 Core Training (intake and

engagement)May 1 or

May 3

Practicum 1 Practice

Introducing FC to Current Families

5/6 – 5/29 CoachingAs needed

Phase 2 Core Training

(family assessment/service

planning)May 20-21 or May

30-31

Practicum 2 Practice conducting

comprehensive family assessment including use of instruments;

practice developing a FC Case plan.

Onsite CoachingJune

Phase 3 Core Training (change focused

intervention; evaluating change; closure decision

making)July 9 & 10 or July 11

& 12

Continued Practicum

Coaching with Supervisors –

ongoing on at least a monthly basis

(N=122)

(N=42) (N=103) (N=104) (N=103)

Coaching is unlocking a person’s potential to maximize their own performance. It is helping them to learn rather than teaching them ”

- John Whitmore, 2002

27

28

3*

Examples

Key Take Home Points

“Implementation occurs in the context of community”

(Fixsen et al., 2005)

Organizational Readiness, Climate, and Culture

29

30

Organizational Climate

Climate is the psychological impact and individual perceptions of the work environment

• Stress – emotional exhaustion, role overload, role conflict

• Engagement

• Functionality (e.g. role clarity, cooperation/cohesion)

31Aarons et al., 2011; Glisson & James, 2002)

Readiness of the Organization to Change

32

8%

46%34%

12%

Preparedness of Organization to Make Change

Not prepared at all Somewhat prepared Prepared Fully prepared

Organizational Climate

34

Contrast 1 year later

Not prepared at all Somewhat prepared Prepared Fully Prepared I don't think our organization needs to make any changes

0.0%

5.0%

10.0%

15.0%

20.0%

25.0%

30.0%

35.0%

40.0%

45.0%

50.0%

8.2%

45.9%

33.7%

12.2%

1.1%

23.9%

32.6%

41.3%

1.1%

Figure 9. Preparedness of Organization to Make Change

May-13 (n = 98) May-14 (n = 92)

TRUE or FALSE?TRUE or FALSE?Readiness to Change Matters?

Stages of Change*

• Pre-contemplation• Contemplation• Preparation• Action• Maintenance

36

*Transtheoretical model of change is relevant to change initiatives – see: Prochaska, J. O., & DiClemente, .C. (1982); Prochaska, J. O., DiClemente, C.C., & Norcross, J.C. (1992)

Factors that affect individual attitudes to implement EBPs

• Intuitive Appeal• Requirements to

change• Openness to new

practices• Perceived

divergence from what I’m used to

37Aarons (2004))

FC Philosophical Principles and Current Practice

38

Not at all Slight extent Moderate extent Great extent Very great extent0.0%

10.0%

20.0%

30.0%

40.0%

50.0%

60.0%

0.011

0.106

0.404

0.479

1.1%1.1%

10.5%

41.5%

46.3%

1.1% 1.1%

7.4%

43.2%47.4%

0.011

0.105

0.326

0.558

Extent to Which FC Philosophical Principles Drive Current Practice

Helping Alliance Strengths-BasedEmpowerment Approach Cultural Differences

39

A year later – Attitudes Stay Consistent

Not at all Slight extent Moderate extent Great extent Very great extent0.0%

10.0%

20.0%

30.0%

40.0%

50.0%

60.0%

1.1%2.3%

9.1%

31.8%

55.7%

4.5%

14.8%

35.2%

45.5%

4.5%

14.8%

31.8%

48.9%

3.2%

12.5%

34.1%

50.0%

Figure 3. Extent to Which FC Philosophical Principles Drive Current Practice May 2014 (n = 88)

Helping Alliance Strengths-Based Empowerment Approach Cultural Differences

TRUE or FALSE?TRUE or FALSE?Coaching & Supervision Matters?

41

Findings from a Review of Fidelity

• In general, NYC programs achieved higher levels of fidelity to the core program components at 6 months post implementation compared to other programs that have implemented Family Connections.

• Across the FCC, some programs did better than others and within programs, some supervisory units achieved higher performance.

TRUE or FALSE?TRUE or FALSE?Fidelity improves outcomes?

Moderating Effect of Fidelity (JBA, 2011)

• Families at sites with higher fidelity scores on program structure reported significantly greater reductions in parenting stress, caregiver depressive symptoms, and need for support.

• Families at sites with higher philosophical principle fidelity scores showed less need for social support over time and smaller decreases in the number of critical dimensions of family functioning.

• Families at sites with higher administrative activities fidelity scores demonstrated greater reductions in child internalizing behaviors and improvements in parental attitudes to parent-child role reversal over time but lower levels of improvement on family functioning.

44

Integrated and Compensatory

Implementation Drivers

• Not a linear process• Drivers overlap and

interact with each other

• Drivers may be more or less salient at different stages

• Strength of one driver can help to compensate for weakness in another

Staff Selection

Training

Coaching and

Supervision

Performance Assessment (Fidelity)

Systems Intervention

LEADERSHIP

Decision Support Data

System

Integrated &

Compensatory

Facilitative Administration

CO

MPETEN

CY

ORG

AN

IZATIO

N

Questions/Reflections?

46

Selected References• Aarons, G.A. (2004). Mental health provider attitudes toward adoption of evidence-based practice: The

evidence-based practice attitude scale (EBPAS). Mental Health Services Research, 6, 61-74. • Aarons, G. A., Hurlburt, M., & Horowitz, S. M. (2011). Advancing a conceptual model of evidence-based

practice implementation in public service sectors. Administration and Policy in Mental Health and Mental Health Services Research, 38(1), 4-23.

• DePanfilis, D. (2009). Using prevention science to reduce the risk of child neglect. Children Australia, 34(1), 40-44.

• DePanfilis, D., & Dubowitz, H. (2005). Family Connections: A program for preventing child neglect. Child Maltreatment,10, 108-123.

• DePanfilis, D., Filene, J. H., & Brodowski, M. L. (2009). Introduction to Family Connections and the national replication effort. Protecting Children, 24(3), 4-14.

• Fixsen, D. L., Blasé, K. A., Naoom, S. F., & Wallace, F. (2009). Core implementation components. Research on Social Work Practice, 19, 531-540.

• Fixsen, D. L., Naoom, S. F., Blase, K. A., & Friedman, R. M. (2005). Implementation research: A synthesis of the literature. Tampa, FL: University of South Florida, Louis de la Parte Florida Mental Health Institute, The National Implementation Research Network (FMHI Publication #231.

• Glisson, C., Green, P., & Williams, N.J. (2012). Assessing the organizational social context (OSC) of child welfare systems: Implications for research and practice. Child Abuse and Neglect, 36, 621 – 632.

• James Bell Associates (2011). National cross site evaluation of the replication of Family Connections: Final evaluation report. Washington, DC: USDHHS, ACY, ACYF, Office of Child Abuse and Neglect.

• Kaye, S., DePanfilis, D., Bright, C. L., & Fisher, C. (2012). Applying implementation drivers to child welfare systems change: Examples from the field. Journal of Public Child Welfare, 6,512-530.

Follow up Contact information:

Diane DePanfilis ddepanfilis@ssw.umaryland.edu410-409-6959