Maximizing Potential: Measuring What Matters

Randi Maines Walters, Ph.D., MSW, LCSW-CChildren’s Bureau, ACYF, ACF

NHSTE Symposium, May 22,2013

2

Greetings & Acknowledgements

Greetings From:

Joe Bock, Acting Associate Commissioner

Jane Morgan, Director of Capacity Building Division

Special Thanks To:

• Nancy Dickinson

• Freda Bernotavicz

• Robin Leake

• Mary McCarthy

5/22/2013

Why NHSTE Matters So Much

Measuring Performance: How Do We Know Training Is Making An Impact?

Your Contribution To The Field

Building The Evidence Base For Training That Improves Outcomes For Children and Families

“THE WAY TO MEASURE THE VALUE OF SOMETHING IS NOT BY WHAT YOU GET OUT OF IT, IT’S WHAT YOU BECOME FROM IT.”

‘

.”

Oliver Wendell Holmes

4 5/3/2013

Maximizing Whose Potential?

• The potential of vulnerable of children and families

• The potential of the people that serve vulnerable children and families

• The collective potential of the organizations whose missions is to develop the workforce in order to best serve children and families

“….for you will never be what you ought to be until they are what they ought to be…”

Martin Luther King

What Matters?

Integrating Safety, Permanency, and Well-Being

Professional Development of Staff

Organizational Culture and Climate

Measuring What Matters: Safety, Permanency & Well BeingIntegrating Safety, Permanency, and Well Being

Focus On Outcomes

Using Data To Drive Outcome Performance in Key Areas

ACYF’s Priority:

INTEGRATING WELL-BEING WITH SAFETY AND PERMANENCY TO ACHIEVE BETTER OUTCOMES FOR CHILDREN, YOUTH, AND FAMILIES

WELL-BEING

PERMANENCYSAFETY

Supportive, responsive relationships promote healing and recovery and reinforce growing social and emotional skills

Nurturing environments provide security and promote positive outcomes

Systems and policies promote and sustain screening, assessment, the use of evidence-based interventions, progress monitoring, and continuous quality improvement

Assessment drives individualized treatment plan with evidence-based interventions

Systematic approaches to teaching coping skills and social skills

Intensive Intervention

Targeted Social and Emotional

Supports

Stress Reducing and Developmentally

Appropriate Environments

Safe, Supportive, and Responsive Relationships

Knowledgeable and Effective Workforce

Healing and RecoverySOCIAL AND EMOTIONAL WELL-BEING FOR CHILDREN, YOUTH, AND FAMILIES

Adapted from the Technical Assistance Center on Social Emotional Intervention for Children and the Center on the Social and Emotional Foundations for Early Learning

TITLE IV-E CHILD ABUSE AND NEGLECT DEMONSTRATION PROJECTS

TITLE IV-E CHILD WELFARE DEMONSTRATION PROJECTS

• HHS may waive title IV-E requirements for States with approved projects, allowing them to use funds flexibly and reinvest savings

• HHS prioritized well-being and addressing trauma as the focus of the demonstrations

• States are encouraged to align screening, assessment, and evidence-based interventions with the needs and characteristics of the target population in order to achieve improved well-being

INTEGRATING SAFETY, PERMANENCY, AND WELL-BEING• Knowledge building and developing practice

– Training staff and foster parents– Providing supports to staff to address secondary trauma

• Validated screening & assessment

– Screening and continual functional assessment that gathers information from multiple sources

• Case planning and management

– Requires sensitive and responsive relationship between child and social worker, birth parents, foster parents, etc.

• Scaling-up of evidence-informed services

– Skilled mental health providers available– Increasing capacity to deliver trauma-focused mental health treatment

• Cross-system partnerships and system collaboration

– Work with Medicaid and mental health respond to trauma-informed needs being identified

Conradi, L; et al. (2011). Promising practices and strategies for using trauma-informed child welfare practice to improve foster care placement stability: A breakthrough series collaborative. Child Welfare. 90(6):207.

MATCHING POPULATIONS, OUTCOMES, AND APPROACHES: IV-E DEMONSTRATION PROJECT EXAMPLES

Population

Children, 8-17

Children, 13-17

Children, 2-7

Screening& Assessment

- UCLA PTSD Index

- Strengths & Difficulties Questionnaire

- Child & Adolescent Needs & Strengths

- Strengths & Difficulties Questionnaire

- Child & Adolescent Needs & Strengths

- Trauma Symptoms Checklist for Young

Children- Infant Toddler Emotional

Assessment- CBCL

EBIs

Trauma-Focused Cognitive Behavioral

Therapy

Multisystemic Therapy

Parent-Child Interaction Therapy

Outcomes

- Behavior problems

- PTS symptoms

- Depression

- Delinquency/Drugs- Peer problems

- Family cohesion

- Conduct disorders

- Parent distress

- Parent-child interaction

INTEGRATING SAFETY, PERMANENCY, AND WELL-BEING FOR CHILDREN AND FAMILIES MEANS:

1. Focus on child & family level outcomes

2. Monitor progress for reduced symptoms and improved child/youth functioning

3. Proactive approach to social and emotional needs

4. Developmentally specific approach

5. Promotion of healthy relationships

6. Build capacity to deliver EBPs

15

Measuring What Matters: Focus On Outcomes

Standardized Risk and Safety Assessments

Caseworker Visits with Parents and Children

Placement Stability

Timeliness of Permanency

Assessment of Need and Services To Meet Those Needs

17

CB Reading Lists

• Leaps and Bounds with Implementation Science

– Getting To Outcomes With Abe & Co.

– Understanding Stages of Implementation and Drivers of Change with Dean and Karen

– Illuminating Inner and Outer Context and Seeing Coaching as a Retention Strategy with Greg

– The Science of Training and Development in Organizations (Eduardo Salas, Scott Tannebaum, Kurt Kraiger, and Kimberly Smith-Jentsch, 2012)

5/22/2013

18

Implementation Framework For Strategic Improvement

• Experimenting with Integrative Workplans

• Theories of Change and Why We Can’t Do Without Them

• Training: Necessary But Not Sufficient

5/22/2013

Measuring What Matters: Professional Development of Staff

Broadening The Perspective

From Training To Professional Development

From Training To Blended Learning Systems

From Individual Performance To Organizational Outcomes

Measuring What Matters: Organizational Improvement

Culture and Climate

The Good, The Bad, The Ugly of The Marriage Between Bureaucracies and People Caring-People Changing Organizations

Training Evaluation

22

Research On Training (Salas, et al, 2012)

• Properly Designed Training Works

• The Way Training Is Designed, Delivered and Implemented Can Greatly Influence Its Effectiveness

• Training is a systematic process

– What Matters Before

– What Matters During

– What Matters After Training

5/22/2013

Six Core Principles of Improvement

Carnegie Foundation

Improvement Research

http://www.carnegiefoundation.org/improvement-research

24

Principles of Improvement

• Make the work problem specific and user centered

• Variation in performance is the core problem to address

• See the system that produces the current outcomes

5/3/2013

Principles of Improvement

• We can not improve at scale what we cannot measure

• Anchor practice improvement in disciplined inquiry

• Accelerate the improvement through networked communities

“Continuous effort—not strength or intelligence---is the key to unlocking potential.”

Winston Churchill

Reading List Referenced During Presentation

Aarons, G.A., Hurlburt, M., & Horwitz, S.M. (2011). Advancing a conceptual model of evidence-based practice implementation in public service sectors. Adm Policy Ment Health 38:4-23.

Fixsen, D. L., Naoom, S. F., Blase, K. A., Friedman, R. M., & Wallace, F. (2005). Implementation research: A synthesis of the literature. National Implementation Research Network

Glisson, C., & Schoenwald, S. K. (2005). The ARC organizational and community intervention strategy for implementing evidence-based child mental health treatments. Mental Health Services Research , 7 (4), 243-260.

Wandersman, et al. (2008). Bridging the gap between prevention research and practice: The Interactive Systems Framework for dissemination and implementation. American Journal of Community Psychology, 41, 171-181.

Wiseman, S., Chinman, M., Ebener, P. A., Hunter, S., Imm, P., & Wandersman, A. (2007). Getting to Outcomes: 10 steps for achieving results-based accountability. RAND Corporation.