The Implementation Process: Perspectives from Frontline

Providers and ManagersJoAnn E. Kirchner, MD Louise E. Parker, PhD

Laura Bonner, PhDElizabeth M. Yano, PhD, MSPH

Mona Ritchie, MSW

Agenda

Formative Evaluation– TIDES

Methods Findings

Formative Evaluation

Types of formative evaluation– Developmental evaluation– Implementation-focused evaluation– Progress-focused evaluation– Interpretive evaluation

The “process” as a product

Formative Evaluation: TIDES Developmental evaluation

– Pre-implementation stakeholder interviews with MH and PC leadership

Implementation-focused evaluation – Mid-implementation qualitative interviews – Early feedback of initial findings to implementation team

Progress-focused evaluation– Program penetration and utilization

Interpretive evaluation– Site case studies– Cost analysis– Qualitative interviews

Methods

Conducted 106 qualitative semi-structured interviews during field visits with– Managers across four participating medical centers and three participating regional networks– Providers, administrators, and consumers

across five participating facilities

Analysis– Data management package– Coded by two-investigator teams– Independent coding with resolution of conflicts

Methods

Here we report findings from– 19 medical center and regional network managers– 49 Frontline providers and administrators

Concerning their perspectives on the QI program implementation process

Results

Local Site Resources

Implementation process

Local Frontline Staff:Attitudes Behaviors

Penetration and Sustainability

Local Outcomes

Spread

Importance of site characteristics

Attitudes/beliefs/experiences– Perceived need for the intervention– Competing needs– Staff open to innovation– MH/PC relationship

• Expected would affect implementation success but did not

Importance of site characteristics

Resources– Perceived time to

• Utilize the program Marketing opportunity

• Participate in implementation activities– Organizational structure

– Staffing– Prior QI experience– Tools (e.g., access to informatics support)

Who to Involve

All Informants – Medical Center Managers– Frontline Clinical Managers– Frontline Primary Care Providers (MD, PA, APN)

Frontline Providers

– RNs, LPNs– Mental Health Providers

Key Roles in Implementation

Leadershipleadership support is obviously also critical. Unless the leaders are able to

translate the importance of the initiative, it's probably not going to take place for a variety of reasons. One of which is that there are a thousand things that need to be done, priorities need to be set and it’s leaderships responsibility to make sure that those priorities are delineated and made clear to everybody, what is it that we’re going to do and what is it we're not going to do - Chief of Staff

– Importance of leaders• valuing the program

I think it’s [leadership support] important because if the leadership shows that it’s important enough for them to spend their time, then it implies that it’s important enough for us to spend our time - MH Provider, Physician

Key Roles in Implementation• initiating the program

• “cheerleading”Managers just have to go back out and continue doing cheerleading

sessions. A little more cheerleading. You go back, and you ... when you see that that’s where you are, you have to just go back and reemphasize. Okay. We’ve gotten to here, we’ve got to be over here or...you guys are doing a great job here, but we could do a little more here, and it’s that cheerleading. You know, 90 percent I think of what managers do are cheerleading people on - Network Manager

• resourcesI feel that anything that you have in place, if the head of anything...is not

going to support you, give you what you need, be there for you in every sense of the word, it’s not worth having. I think it is doomed to fail if you don’t have that in place - Network Manager

• importance of management at multiple levels

Key Roles in Implementation

But in absence of clinic perceived need for the program, leadership will have an uphill battle

Key Roles in Implementation Champion

– Who should be a champion?Does that person have credibility with the other staff, you know, and

that’s not measured in what letters you have behind your name. That’s do people listen to you when you speak. I don’t know how you put a degree on that, but that’s what you need - Primary Care, RN

Colleague who has experienced the program High status peers Someone interested in quality improvement and in

improving mental health care in primary care setting Credible Able to communicate well to groups Self-driven Good rapport

Key Roles in Implementation Role/activities of the champion

He’s [the local champion] been the person who’s at our meetings reminding us about the program…And I think the more you do that, the more you get people using it - Primary Care, RN

– Serve as the local expert and resource person– Encourage people to attend related meetings– Remind people about the program– Provide ongoing marketing

Needs of the champion– Time

Key Roles in Implementation Leadership as a champion

– Roles of champion, leadership, and opinion leaders are not distinct in the views of clinical staff and managers

– Thus, implementation researchers may be creating artificial distinctions

Site Readiness

Low hanging fruit

What about the fruit higher in the tree?

Informing Readiness

Message will depend on how ready sites are for a particular intervention (i.e., perceived need)

Regardless of readiness, different audiences (i.e., managers versus frontline) have different needs to promote readiness

Importance of ongoing marketing for allSometimes in the VA, you’ve got to tell me three times to do it because if

you tell me three times, it was really important. If you tell me once and I never hear from you again, then it probably wasn’t important and was a passing fad and I’ll kind of wait – Chief of Staff

PLAN

DO

STUDY

ACT

1st Generation Sites

2nd Generation Sites

1st Generation Sites• sustainability• penetration

Ongoing Formative Evaluation

Developmental evaluation– Site level needs assessment tool

Implementation-focused evaluation– Site level fidelity monitoring tool

Progress-focused evaluation– Program penetration and utilization through National

Depression Monitor

Interpretive evaluation– Formative evaluation of regional dissemination of TIDES– Assessment of the sustainability effort