Promoting Science-based Approaches: Bridging Research

and Practice by Integrating Research to Practice Models and

Community-Centered Models (ISF)

Abraham Wandersmanwandersman@sc.eduU. Of Connecticut

April 2010

MAKING A DIFFERENCE

MAKING A DIFFERENCE

• HOW DO WE GET THERE?

THE 2015 TARGET DATE FOR ELIMINATING SUFFERING AND

DEATH DUE TO CANCER:

AMBITOUS GOALS

• Dr. von Eschenbach: I believe we are at what I call a strategic inflection in biology, which means we're at a point of unprecedented growth in three key areas related to cancer research: knowledge, technology, and resources. The integration of growth in these three sectors provides an opportunity for exponential progress. To achieve this progress, we must set a clear direction and focus our efforts into a cohesive strategy.

• The goal of eliminating suffering and death due to cancer provides this focus. It does not mean "curing" cancer but, rather, it means that we will eliminate many cancers and control the others, so that people can live with -- not die from -- cancer. We can do this by 2015, but we must reach for it. We owe it to cancer patients around the world -- and their families -- to meet this challenge.

May 16, 2003 BenchMarks

HEALTHY PEOPLE 2010

Healthy People 2010 Objectives

• Target: 1.0 new case per 100,000 persons.

• Baseline: 19.5 cases of AIDS per 100,000 persons aged 13 years and older in 1998. Data are estimated; adjusted for delays in reporting.

• Target setting method: Better than the best.

• Data source: HIV/AIDS Surveillance System, CDC, NCHSTP.

Persons Aged 13 Years and Older, 1998

New AIDS Cases

13-1. Both

Genders

Females* Males*

Rate per 100,000

TOTAL 19.5 8.8 30.8

Race and ethnicity

American Indian or Alaska Native 9.4 4.5 14.5

Asian or Pacific Islander 4.3 1.2 7.8

Asian DNC DNC DNC

Native Hawaiian and other Pacific Islander DNC DNC DNC

Black or African American DNC DNC DNC

White DNC DNC DNC

 Hispanic or Latino 33.0 13.8 52.2

Not Hispanic or Latino DNC DNC DNC

Black or African American 82.9 48.5 122.9

White 8.5 2.2 15.2

Family income level

Poor DNC DNC DNC

Near poor DNC DNC DNC

Middle/high income DNC DNC DNC

Sexual orientation DNC DNC DNC

In 2007, there were 42,495 new cases of HIV/AIDS in adults, adolescents, (2500)

DATA - EVIDENCE

WHY IS EVIDENCE/SCIENCE NOT USED MORE?

Expanding Research and Evaluation Designs…for QII

Carolyn M. Clancy, MDDirector, AHRQ

September 13, 2005

Publication

Bibliographic databases

Submission

Reviews, guidelines, textbook

Negative results

variable

0.3 year

6. 0 - 13.0 years50%

46%

18%

35%

0.6 year

0.5 year

9.3 years

It takes 17 years to turn 14 per cent of original research to the benefit of patient care

Dickersin, 1987

Koren, 1989

Balas, 1995

Poynard, 1985

Kumar, 1992

Kumar, 1992

Poyer, 1982

Antman, 1992

Negative results

Lack of numbers Expert

opinion

Inconsistentindexing

17:14

Original research

Acceptance

Implementation

Treatments Thought to Work but Shown Ineffective

• Sulphuric acid for scurvy• Leeches for almost anything• Insulin for schizophrenia• Vitamin K for myocardial infarction• HRT to prevent cardiovascular

disease• Flecainide for ventricular

tachycardia• Routine blood tests prior to surgery• ABMT for late stage Breast CA

BMJ February 28 2004; 324:474-5.

THE GAP BETWEEN SCIENCE AND PRACTICE

• IN THE DOCTOR’S OFFICE

• OVERALL 54.9% RECEIVED RECOMMENDED CARE

ASCH ET AL STUDY, NEJM, 2006

POSSIBLE SOLUTION

• VA MEDICAL SYSTEM HAS 67% RECOMMENDED CARE

SYSTEM HAS ELECTRONIC MEDICAL RECORDS, DECISION SUPPORT TOOLS, AUTOMATED ORDER ENTRY, ROUTINE MEASUREMENT AND REPORTING ON QUALITY, INCENTIVES FOR PERFORMANCE

As Yogi Berra supposedly said,

"In theory there is no difference between theory and practice, but in practice there is."

* Why is there a gap between science and practice?

* What is the dominant scientific paradigm for developing research evidence and disseminating it?

* Why is this science model necessary but not sufficient?

* What is the responsibility of the practitioner to deliver evidence-based interventions and what is their capacity to do so?

* What is the responsibility of funders to promote the science of evidence-based interventions and to promote the practice of effective interventions in our communities?

How can evaluation help providers, local CBOS and coalitions, health districts, and state agencies reach results-based accountability?

Two Routes to Getting To Outcomes (GTO):

A) Bridging Science and PracticeB) Empowerment Evaluation

Research To Practice Practice To Research

CLOSING THE GREAT DIVIDE

2. With an emphasis on risk and protective factors, review relevant infor-mation—both from fields outside prevention and from existing preventive intervention research programs

3. Design, conduct, and analyze pilot studies and confirmatory and replication trials of the preventive intervention program

4. Design, conduct, and analyze large-scale trails of the preventive intervention program

5. Facilitate large-scale implementation and ongoing evaluation of the preventive intervention program in the community

1. Identity problem or disorder(s) and review information to determine its extent

Feedback Loop

FIGURE 1.1 The preventive intervention research cycle. Preventive intervention research is represented in boxes three and four. Notre that although information from many different fields in health research, represented in the first and second boxes, is necessary to the cycle depicted here, it is the review of this information, rather than the original studies, that is considered to be part of the preventive intervention research cycle. Likewise, for the fifth box, it is the facilitation by the investigator of the shift from research project to community service program with ongoing evaluation, rather than the service program itself, that is part of the preventive intervention research cycle. Although only one feedback loop is represented here, the exchange of knowledge among researchers and between researchers and community practitioners occurs throughout the cycle.

Gates Foundation

Preventive Intervention

Vaccine/Drug

Mechanism

Syringes

Physician

Health System

Support System

Medical Schools

Government Funding

Emergency Departments, Hospitals, and Clinics

HOMES

Childcare Centers and Schools

Community Centers

Clinics and Pharmacies

Media

Care Coordination by Health Department

Nurse and Environmental Case Management

Community Education

Surveillance and Evaluation

Surveillance and Evaluation

Family Education

Parent and Neighborhood Organizing

Clinical Quality Improvement

Public Communications

Visual Description of Alianza’s Community Action Plan*

Family obtains medications/equipment

Recruitment at Asthma Community Center

Community Health Worker home visiting

& social referrals program (1, 2, 3, 4)

Community-Based, Educational and

Promotional Programs (5, 6)

School-based ALA Asthma Programs:

Open Airways, A is for Asthma (8)

Parent Organization (7)

EMPOWERED & EDUCATED

FAMILIES

Alianza Steering Committee & Workgroups

Recruitment at Local Clinic

Nurse Coordinator performs initial

evaluation & enters into registry (9,10)

Primary Doctor evaluates and

prescribes medications/ equipment

Refer to specialist

Financial Incentives & Quality

Improvement (14)

Expanded Medications/ Equipment

Coverage (15)

Child takes medication

IMPROVED SYMPTOMS CONTROL

& QUALITY OF LIFE

1:1 or Group Asthma

Education (11,12)

PACE (Physician Asthma Care

Education) (13)

Expanded Specialty Referrals

Legend:

COMMUNITY ACTIVITIES

CLINICAL ACTIVITIES

POLICY CHANGES

EDUCATIONAL ACTIVITIES

*Activities numbered 1-15 are cross-referenced with the Program Update summaries

From Research to “Best Practices” in Other Settings and Populations

Larry GreenAmerican Journal of Health Behavior, 2001

1) Process

2) Control

3) Self-Evaluation

4) Tailoring Process and New Technology

5) Synthesizing Research

Getting to Outcomes

1) Needs/Resources 2) Goals 3) Best Practice 4) Fit 5) Capacities 6) Plan 7) Process Evaluation 8) Outcome Evaluation 9) CQI 10) Sustain

“Prevention Science” Intervention

Basic research

Efficacy

Effectiveness

Services Research

Practice

Community

Organizational Systems

1) Schools

2) Health Agencies

3) Community Coalitions

Prevention Support System

(Funders)

Training

Technical Assistance

Funding

Green Characteristics

1) Process

2) Control

3) Self-Evaluation

4) Tailoring Process and new Technology

5) Synthesizing Research

io

Distilling the Information—Prevention Synthesis & Translation System

Supporting the Work—Prevention Support System

Putting It Into Practice—Prevention Delivery System

Synthesis

General Capacity Building

Innovation-Specific Capacity Building

General Capacity Use

Innovation-Specific Capacity Use

Macro Policy

Climate

Funding

Existing Research and Theory

Translation

ROUTE B:

EMPOWERMENT EVALUATION

What Can Steve Spurrier Teach Us about Loving Evaluation?

Forms

ComprehensiveCommunity Plan

Chairpersons ConsolidateWork of Individual

Committees

ReligionEducationBusiness Parents Youth HealthMediaGrassroots/ Neighborhood

Criminal Justice

Conduct Needs Assessment

Lead Agency

Ad Hoc Committee Of Community Leaders

Forms Committees

Resulting In

Resulting In

Plan Implementation

Impact on Community Health Indicators

Resulting In

Implementation

COALITION

FORMATION

MAINTENANCE

OUTCOMES

Figure 2. Overview of the development of a community coalition.

Table 1. Evaluation of MPA by Developmental Phases, Ecological Levels, and Stages of Readiness

Ecological Levels

Developmental phases and measures

Intra-personal

Inter-personal

Organizational Community PublicPolicy

Stages of readiness

Phase 1: Coalition formation

Forecast X Initial mobilization andestablishing organizational structure

Meeting Effectiveness Inventory X X X

Project Insight Form X X X

Committee survey X X X

Needs Assessment Checklist X X

Plan Quality Index X X X

Phase 2: Plan implementation

Tracking of Actions X X X Building capacity for action and implementing

Prevention Plus III X X X X X

Policy Analysis Case Study X X

Phase 3: Impact

Key Leader survey X X X Refining and institutionalizing

Community survey X X

Trend data X X

Level of Institutionalization Scale

X X X

Outcome Evaluation

A B

Shoot

No ResultsPlanImplement

ReadyAim

Ready Aim Close

Plan Implement CQI

Shoot

Shoot

ResultsPlan Implement

Ready Aim Hit

Empowerment Evaluation: An evaluation approach that aims to increase the probability of achieving program success by:

a)Providing program stakeholders with tools for assessing the planning, implementation, and self-evaluation of their program, and

b)Mainstreaming evaluation as part of the planning and management of the program/organization.

EE PrinciplesCore Principles of Empowerment Evaluation

Principle 1: Improvement

Principle 2: Social Justice

Principle 3: Inclusion

Principle 4: Democratic participation

Principle 5: Capacity building

Principle 6: Organizational learning

Principle 7: Community ownership

Principle 8: Community knowledge

Principle 9: Evidence-based strategies

Principle 10: Accountability

Accountability Questions Relevant Literatures

1. What are the underlying needs and conditions that must be addressed? (NEEDS/RESOURCES)

1. Needs/Resource Assessment

2. What are the goals, target population, and objectives? (i.e., desired outcomes)? (GOALS)

2. Goal Setting

3. What science (evidence) based models and best practice programs can be used in reaching the goals (BEST PRACTICE)?

3. Consult Literature on Science Based and Best Practice Programs

4. What actions need to be taken so the selected program “fits” the community context? (FIT)

4. Feedback on Comprehensiveness and Fit of Program

5. What organizational capacities are needed to implement the program? (CAPACITIES)

5. Assessment of Organizational Capacities

6. What is the plan for this program (PLAN) 6. Planning

7. Is the program being implemented with quality (PROCESS)

7. Process evaluation

8. How well is the program working? (OUTCOME EVALUATION)

8. Outcome and Impact Evaluation

9. How will continuous quality improvement strategies be included? (IMPROVE)

9. Total Quality Management; Continuous Quality Improvement

10. If the program is successful, how will it be sustained? (SUSTAIN)

10. Sustainability and Institutionalization

What Is Getting To Outcomes?• By Matthew Chinman, Pamela Imm & Abraham

Wandersman

• A system based on ten empowerment evaluation and accountability questions that contain elements of successful programming

• Published by the RAND Corporation (quality review)

• Available at no cost at: http://www.rand.org/publications/TR/TR101/

• “Best Practice Process” - CSAP

The Getting To Outcomes Process

#1Needs/

Resources

#2Goals

#3Best

Practices

#4Fit

#5Capacities

#6Plan

#7Process

#8Outcome

Evaluation

#9Improve

#10Sustain

GTO-04 Manual

GTO-04 Manual

GTO-04 Manual

• WINNERS Example

GTO-04 Manual

• Up to date model program descriptions

GTO-04 Manual

• Risk and protective factor based

Uses of GTO• Individual Program Level (e.g., WINNERS)

– Still a guide to planning, implementation, evaluation– Use data to continually improve – Determine effectiveness in one program

• Coalition Level (e.g., CDC grant)– Each committee monitors own programs– Direct TA for program improvement– Fulfill the whole coalition’s accountability requirements

• State/Federal Level (e.g., S.C. SIG grant)– Monitor several similar programs at once across large area– Aggregate program data for state-wide reporting and within

state comparisons– Highlight specific technical assistance needs across the state

GTO®

2009

Using Getting to Outcomes to improve communities' capacity to conduct high quality prevention programming: A Center for Disease Control & Prevention Empirical Example

Chinman et al (2008) American Journal of Community Psychology

GTO Demonstration & Evaluation

• Purpose: Evaluate a 2-yr GTO intervention to improve prevention capacity and program performance

• CDC-funded participatory research grant

• Sample: 2 prevention coalitions (SC, CA) involving 10 programs & 268 coalition staff

• Design: Quasi-experimental; mixed methods

• Within each coalition, assign by program• (GTO: 2 SC+ 4 CA v. Comparison: 2 SC+2 CA)

• intervention: participate in GTO• comparison: usual practice

Getting To Outcomes Evaluation: Conclusions

GTO improved practitioner capacity & performance of tasks associated with high quality prevention (planning, evaluation, etc.) and programs that used GTO showed greater outcomes Those with greater exposure to GTO demonstrated more gains in

capacity TA hours show that practitioners mostly want & got help with

evaluation activities GTO can be difficult to absorb without ongoing TA Organizational issues a major factor

Conversion to “learning organization” not complete Resources are significant barrier to adoption, implementation, and

sustainability Incentive structure within which coalitions operate not aligned with

CQI

GTO Evaluation: Conclusions

Technical assistance to use the steps is critical to the success of GTO

Organizational issues can be a major factor Lack of resources pose significant barriers to

adoption, implementation, and sustainability

“Prevention Science” Intervention

Basic research

Efficacy

Effectiveness

Services Research

Practice

Community

Organizational Systems

1) Schools

2) Health Agencies

3) Community Coalitions

Prevention Support System

(Funders)

Training

Technical Assistance

Funding

Green Characteristics

1) Process

2) Control

3) Self-Evaluation

4) Tailoring Process and new Technology

5) Synthesizing Research

io

Distilling the Information—Prevention Synthesis & Translation System

Supporting the Work—Prevention Support System

Putting It Into Practice—Prevention Delivery System

Synthesis

General Capacity Building

Innovation-Specific Capacity Building

General Capacity Use

Innovation-Specific Capacity Use

Macro Policy

Climate

Funding

Existing Research and Theory

Translation

EXAMPLE COMBINING

*BRIDGING RESEARCH AND PRACTICE (ISF)

AND

*GTO

GTO®

2009

Teen Pregnancy PreventionThe Promoting Science Based Approaches Project

CDC Adolescent Reproductive Health Team

The Barriers to Use of Science-Based Approaches (SBA)

• Funding for training and materials• Implementation funding• Fear of controversy• Lack of motivation (why use SBA?)• Suitability for own community• Ease of implementation• Loyalty to current strategies

Philliber, Nolte & Schauer, in prep

The Challenge

• Teen pregnancy field has growing number of effective prevention programs

• However, programs are not being implemented as widely nor as effectively as needed to combat rising teen pregnancy

io

Distilling the Information—Prevention Synthesis & Translation System

Supporting the Work—Prevention Support System

Putting It Into Practice—Prevention Delivery System

Synthesis

General Capacity Building

Innovation-Specific Capacity Building

General Capacity Use

Innovation-Specific Capacity Use

Macro Policy

Climate

Funding

Existing Research and Theory

Translation

PSBA Activities: National Grantees

• Develop trainings & other tools to build capacity to use SBA

• Use tools to build capacity of state & regional grantee organizations

• Disseminate information about SBA to a broad audience

GTO

PSBA Activities: State Coalitions and RTCs

• More intensive– Provide targeted technical assistance to small

number of local organizations (5-10) to increase their capacity to use SBA locally

• Less intensive– Provide information and resources re: SBA to

broad audiences within state/region through newsletters, websites, etc.

GTO

io

Distilling the Information—Prevention Synthesis & Translation System

Supporting the Work—Prevention Support System

Putting It Into Practice—Prevention Delivery System

Synthesis

General Capacity Building

Innovation-Specific Capacity Building

General Capacity Use

Innovation-Specific Capacity Use

Macro Policy

Climate

Funding

Existing Research and Theory

Translation

GTO System Model

To AchieveDesired

Outcomes

Training +

QI/QA +

Tools +

TA +

=Current Level of Capacity

+

GTO Steps: (1) Needs & Resources; (2) Goals & Desired Outcomes; (3) Science-based practices; (4) Fit; (5) Capacity ; (6) Plan; (7) Implementation & Process Evaluation; (8) Outcome evaluation; (9) Continuous Quality Improvement; and (10) Sustainability

Actual Outcome

s Achieved

Levels & AIDS TreatmentCountry State Health District FQHC Provider

Accountability Question

1. NEEDS/ RESOURCES

2. GOALS

3. EVIDENCE-BASED PRACTICES

4. FIT

5. CAPACITY

6. PLAN

7. IMPLEMENTATION

8. OUTCOME EVALUATION

9. CQI

10. SUSTAINABILITY

The GTO PANORAMA

GTO Steps

1. Needs and Resources2. Goals and Objectives3. Best Practices4. Fit5. Capacities6. Plan7. Process Evaluation8. Outcome Evaluation9. CQI10. Sustainability

GTO Steps

GTO Content

1. Needs and Resources2. Goals and Objectives3. Best Practices4. Fit5. Capacities6. Plan7. Process Evaluation8. Outcome Evaluation9. CQI10. Sustainability

Substance Abuse Specific

Content

GTO Steps

GTO Content Domains

1. Needs and Resources

2. Goals and Objectives

3. Best Practices

4. Fit

5. Capacities

6. Plan

7. Process Evaluation

8. Outcome Evaluation

9. CQI

10. Sustainability

Substance Abuse Specific

Content

GTO Steps

Systems of Care

Performance Contracting

Emergency Preparedness

Specific ContentUnderage

Drinking Specific ContentYouth

Development Specific ContentPatient

Centered Care Specific ContentTeen

Pregnancy Specific Content

GTO Content Library

GTO Levels

1 Fit

1 Best Practices

1 Needs and Resources

1 Goals and Objectives

1 Capacities

1 Plan

1 Process Evaluation

1 Outcome Evaluation

1 CQI

1 Sustainability

GTO Steps

INDIVIDUAL

Substance

Abuse Specific Content

1 Fit

1 Best Practices

1 Needs and Resources

1 Goals and Objectives

1 Capacities

1 Plan

1 Process Evaluation

1 Outcome Evaluation

1 CQI

1 Sustainability

GTO Steps

ORGANIZATIONAL

Substance

Abuse Specific Content

1 Fit

1 Best Practices

1 Needs and Resources

1 Goals and Objectives

1 Capacities

1 Plan

1 Process Evaluation

1 Outcome Evaluation

1 CQI

1 Sustainability

GTO Steps

COUNTY

Substance

Abuse Specific Content

1 Fit

1 Best Practices

1 Needs and Resources

1 Goals and Objectives

1 Capacities

1 Plan

1 Process Evaluation

1 Outcome Evaluation

1 CQI

1 Sustainability

GTO Steps

STATE

Substance

Abuse Specific Content

1. Needs and Resources

2. Goals and Objectives

3. Best Practices

4. Fit

5. Capacities

6. Plan

7. Process Evaluation

8. Outcome Evaluation

9. CQI

10. Sustainability

Substance Abuse Specific

Content

GTO Steps

NATIONAL

Systems of Care

Performance Contracting

Emergency Preparedness

Specific ContentUnderage

Drinking Specific ContentYouth

Development Specific ContentPatient

Centered Care Specific ContentTeen

Pregnancy Specific Content

GTO Content Library

GTO Support System

Training

Technical AssistanceQI/QA

Tools

1 Fit

1 Best Practices

1 Needs and Resources

1 Goals and Objectives

1 Capacities

1 Plan

1 Process Evaluation

1 Outcome Evaluation

1 CQI

1 Sustainability

GTO Steps

INDIVIDUAL

Substance

Abuse Specific Conten

t

1 Fit

1 Best Practices

1 Needs and Resources

1 Goals and Objectives

1 Capacities

1 Plan

1 Process Evaluation

1 Outcome Evaluation

1 CQI

1 Sustainability

GTO Steps

ORGANIZATIONAL

Substance

Abuse Specific Conten

t

1 Fit

1 Best Practices

1 Needs and Resources

1 Goals and Objectives

1 Capacities

1 Plan

1 Process Evaluation

1 Outcome Evaluation

1 CQI

1 Sustainability

GTO Steps

COUNTY

Substance

Abuse Specific Conten

t

1 Fit

1 Best Practices

1 Needs and Resources

1 Goals and Objectives

1 Capacities

1 Plan

1 Process Evaluation

1 Outcome Evaluation

1 CQI

1 Sustainability

GTO Steps

STATE

Substance

Abuse Specific Conten

t

1. Needs and Resources

2. Goals and Objectives

3. Best Practices

4. Fit

5. Capacities

6. Plan

7. Process Evaluation

8. Outcome Evaluation

9. CQI

10. Sustainability

Substance Abuse Specific

Content

GTO StepsNATIONAL

Systems of Care

Performance Contracting

Emergency Preparednes

s Specific Content

Underage Drinking Specific ContentYouth

Development Specific ContentPatient

Centered Care Specific ContentTeen

Pregnancy Specific Content

GTO Content Library

As Yogi Berra supposedly said:

“It’s déjà vu all over again.”

* Why is there a gap between science and practice?

* What is the dominant scientific paradigm for developing research evidence and disseminating it?

* Why is this science model necessary but not sufficient?

* What is the responsibility of the practitioner to deliver evidence-based interventions and what is their capacity to do so?

* What is the responsibility of funders to promote the science of evidence-based interventions and to promote the practice of effective interventions in our communities?

How can evaluation help providers, local CBOs and coalitions, health districts, and state agencies reach results-based accountability?

As Yogi Berra supposedly said,

"If you see a fork in the road, take it."

References

• Chinman, M., Hunter, S. B., Ebener, P., Paddock, S. M., Stillman, L., Imm, P., Wandersman, A. (2008). The Getting To Outcomes Demonstration and Evaluation: An Illustration of the Prevention Support System. American Journal of Community Psychology, 206-224.

*  Wandersman, A. (2003) Community science: Bridging the gap between science and practice with community-centered models. American Journal of Community Psychology, 31, 3/4, 227-242.

• Wandersman, A., Duffy, J., Flaspohler, P., Noonan, R., Lubell, K., Stillman, L., et al. (2008). Bridging the gap between prevention research and practice: The Interactive Systems Framework for Dissemination and Implementation. American Journal of Community Psychology, 41, 171-181.

• Lesesne et al (2008) Promoting Science Based Approaches to teen pregnancy prevention. American Journal of Community Psychology, 379-392.

• Wandersman, A. (2009) Four keys to success (theory, implementation, evaluation, resource/system support): High hopes and challenges in participation. American Journal of Community Psychology. 43 (1/2), 3-21.