continuous quality improvement plan for evidence-based ... · standards. quality assurance is a...

16
Continuous Quality Improvement Plan for Evidence-Based Health Interventions (QTAC CQI Plan) The following document outlines the policies, procedures, protocols and mechanisms used to ensure the quality delivery of evidence-based interventions (EBIs) for health that are promoted (1) by the New York State Evidence-Based Health Programs Quality & Technical Assistance Center, a unit of the Center for Excellence in Aging & Community Wellness, State University of New York at Albany, hereafter known as "QTAC", and (2) by organizations that have signed the QTAC Partnership MOU, hereafter known as the “Partner”. All program fidelity activities are part of the overall quality assurance and improvement strategies required of QTAC Partners. More specifically, this document sets forth the minimum standards for our Partners as well as performance indicators for each standard. We encourage you to consider ways to not only meet but to exceed these program quality and fidelity standards. If you have questions, please contact the QTAC at the Center for Excellence in Aging & Community Wellness by email at [email protected] or by phone at 518-442-5530 or 877-496- 2780. Thank you for your continued commitment to quality program delivery.

Upload: others

Post on 02-Aug-2020

5 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Continuous Quality Improvement Plan for Evidence-Based ... · standards. Quality assurance is a data-driven process for team decision-making, problem-solving and program improvement

Continuous Quality Improvement Plan for Evidence-Based Health Interventions

(QTAC CQI Plan)

The following document outlines the policies, procedures, protocols and mechanisms used to

ensure the quality delivery of evidence-based interventions (EBIs) for health that are promoted

(1) by the New York State Evidence-Based Health Programs Quality & Technical Assistance

Center, a unit of the Center for Excellence in Aging & Community Wellness, State University of

New York at Albany, hereafter known as "QTAC", and (2) by organizations that have signed the

QTAC Partnership MOU, hereafter known as the “Partner”.

All program fidelity activities are part of the overall quality assurance and improvement

strategies required of QTAC Partners.

More specifically, this document sets forth the minimum standards for our Partners as well as

performance indicators for each standard. We encourage you to consider ways to not only

meet but to exceed these program quality and fidelity standards.

If you have questions, please contact the QTAC at the Center for Excellence in Aging &

Community Wellness by email at [email protected] or by phone at 518-442-5530 or 877-496-

2780. Thank you for your continued commitment to quality program delivery.

Page 2: Continuous Quality Improvement Plan for Evidence-Based ... · standards. Quality assurance is a data-driven process for team decision-making, problem-solving and program improvement

QTAC CQI Plan (QTAC Partnership MOU ~ Appendix A)

TABLE OF CONTENTS

What is a Quality Assurance Plan? 1 What is Continuous Quality Improvement? 1 What is Program Fidelity? 1 Why is Program Fidelity a critical part of program quality? 2 What is the Purpose of the QTAC CQI Plan? 2 What is the Role of Partners in the CQI Plan? 2

Using the RE-AIM Framework in Planning Quality Assurance Systems 3 RE-AIM Definitions

CQI Performance Indicators 4 Reach – Defining Our Target Population 4 Efficacy/Effectiveness– Participant Outcomes 4 Adoption – Partner Commitment 6 Implementation – Quality Programming 6 Maintenance – Sustainable Systems 8

When Performance Indicators Are Not Met – Process Improvement Strategies 9

Tracking Our Efforts 10 Required Tracking 10 Suggested Tracking to Enhance Program Quality 12

Documents Referenced 14

ABBREVIATIONS

ALED Active Living Every Day

EBI Evidence-Based Intervention REFERENCES TO EBI THROUGHOUT THIS DOCUMENT INCLUDE ALL PROGRAMS CURRENTLY BEING PROMOTED BY THE NYS QTAC

LSC Lifestyle Coach PROGRAM FACILITATORS OF NYS DPP

NYS DPP New York State Diabetes Prevention Program

Portal NYS QTAC Partner Portal FOR MORE INFORMATION, SEE BOX ON PAGE 10.

SPERC Stanford Patient Education Research Center THE DEVELOPERS OF SEVERAL LICENSED EBIS CENTERED AROUND SELF-MANAGEMENT, I.E., CHRONIC DISEASE SELF-MANAGEMENT

PROGRAM S-SMP Stanford Self-Management Program

WWE-SD Arthritis Foundation Walk With Ease Self-Directed Program ALL REFERENCES THROUGHOUT THIS DOCUMENT ARE REGARDING THE WWE SELF-DIRECTED PROGRAM, WHICH IS CURRENTLY BEING

PROMOTED BY THE QTAC.

Page 3: Continuous Quality Improvement Plan for Evidence-Based ... · standards. Quality assurance is a data-driven process for team decision-making, problem-solving and program improvement

[RETURN TO TABLE OF CONTENTS]

QTAC CQI Plan (QTAC Partnership MOU ~ Appendix A) 1 (Revised 1/29/14)

What is a Quality Assurance Plan?

Quality Assurance Plans are structured written approaches that assist all members of the

program delivery team (administrators, coordinators, implementers, evaluators, funders, etc.)

to describe, measure, and evaluate program delivery to ensure that participants receive

effective, quality services; and that programs are delivered consistent with established

standards. Quality assurance is a data-driven process for team decision-making, problem-

solving and program improvement to ensure that minimum program standards are met and

provide information to improve program delivery and outcomes. Program Fidelity activities are

critical to ongoing quality assurance and improvement strategies.

What is Continuous Quality Improvement?

Continuous Quality Improvement (CQI) is a cyclical process that includes:

Planning that utilizes performance objectives to monitor program delivery and goals

Monitoring programs by obtaining ongoing Partner and participant feedback and

collecting program data to inform decision-making

Evaluation that allows the program team to determine best practices and problem-solve

barriers and challenges to quality program delivery

Utilization of data from multiple sources to obtain the most accurate picture of the

program in an effort to enhance outcomes for all stakeholders, including participants,

program facilitators and trainers, organizations and funders

Corrective action recommendations based upon collected data compared to quality

standards which when implemented help ensure and improve overall program quality

and performance and enhance participant satisfaction

What is Program Fidelity?

At the organizational/agency/program delivery level, program fidelity refers to how closely staff

and others (i.e., program facilitators, trainers and evaluators) follow the program that the

developers provide. This includes consistency of delivery as intended as well as in program

timing and costs.

The programs promoted by the QTAC have clearly defined core program elements and methods

that, when followed closely, help to ensure the quality of the program and enhance program

outcomes for both individuals and organizations. Developing and monitoring program fidelity

using strategies that are measurable and sustainable is crucial to both the effectiveness and

quality delivery of evidence-based interventions for health.

Page 4: Continuous Quality Improvement Plan for Evidence-Based ... · standards. Quality assurance is a data-driven process for team decision-making, problem-solving and program improvement

[RETURN TO TABLE OF CONTENTS]

QTAC CQI Plan (QTAC Partnership MOU ~ Appendix A) 2 (Revised 1/29/14)

Why is Program Fidelity a critical part of program quality?

Program fidelity is critical to the integrity of the program and quality outcomes. Program

delivery that is not consistent with the underlying empirical research that supports it decreases

the likelihood that the program will result in the desired outcomes. Poor quality fidelity can

also result in a range of unintended effects for participants, facilitators and trainers. To deliver

the highest quality program faithful to the evidence supporting it, fidelity monitoring must be a

thoughtful, comprehensive part of an overall Quality Assurance/Quality Improvement (QA/QI)

plan.

What is the Purpose of the QTAC CQI Plan?

1. To define the QTAC expectations of quality assurance (QA) and program fidelity for

organizations implementing programs in partnership with the QTAC.

2. To frame these expectations in a broader context using the RE-AIM framework.

3. To specify designated roles, responsibilities and timelines for quality program

planning, delivery and follow-up.

4. To identify performance indicators.

5. To orient Partners to QTAC CQI Plan standards.

6. To identify measures and mechanisms used in the periodic review of performance

indicators.

7. To standardize protocols and performance indicators for quality improvement.

8. To provide a mechanism for review of the effectiveness of quality improvement

strategies.

What is the Role of Partners in the CQI Plan?

The CQI Plan is written for our Partners as they are the critical implementers of the strategies

outlined. By signing the QTAC Partnership MOU, Partners commit to quality assurance activities

that include program fidelity monitoring and reporting. Partner commitment to quality

programming is critical to the implementation, adoption and maintenance of programs that

obtain the intended outcomes for target audiences.

Partners provide crucial information to the ongoing CQI process and Plan. Partners illuminate

challenges in delivery at the local level and problem-solve with the QTAC to overcome

challenges and ensure that participants receive the highest quality experience.

Page 5: Continuous Quality Improvement Plan for Evidence-Based ... · standards. Quality assurance is a data-driven process for team decision-making, problem-solving and program improvement

[RETURN TO TABLE OF CONTENTS]

QTAC CQI Plan (QTAC Partnership MOU ~ Appendix A) 3 (Revised 1/29/14)

Using the RE-AIM Framework in Planning Quality Assurance Systems

The RE-AIM framework’s five components (Reach, Efficacy/Effectiveness, Adoption,

Implementation and Maintenance) are the basis for this plan. Each component builds on the

next helping to maximize quality program delivery at every level.

RE-AIM Definitions (re-aim.org)

Reach activities assess the extent to which a program reaches the intended target population.

Efficacy/Effectiveness activities assess whether a program is achieving the same participant outcomes

and having the same impact as in the original research design.

Adoption activities assess the extent to which implementation partner sites choose to continue

delivery of the program and the amount of organizational support dedicated to ongoing delivery – this

includes supporting program facilitators.

Implementation monitoring activities assess the extent to which the program is implemented with

fidelity, i.e., as intended in the real world in different settings and with different personnel.

Maintenance activities assess the extent to which participants sustain program benefits and how well

the program is sustained within partner sites over the long term.

Efficacy/ Effectiveness

Reach

Implementation

Maintenance

Building Programs and Policies with a Large Public Health Impact

Adoption

Page 6: Continuous Quality Improvement Plan for Evidence-Based ... · standards. Quality assurance is a data-driven process for team decision-making, problem-solving and program improvement

[RETURN TO TABLE OF CONTENTS]

QTAC CQI Plan (QTAC Partnership MOU ~ Appendix A) 4 (Revised 1/29/14)

CQI Performance Indicators

Reach – Defining Our Target Population

In order to document our reach and secure future funding for programming, the QTAC collects

basic demographic information on participants in all of our programs as well as on the program

facilitators trained. For more information about reach information for your targeted

population, please contact the QTAC.

R E A C H

Data Collection Point Partner Indicator QTAC/State Indicator

Participant Information Survey (a voluntary contact/ demographic/health status survey completed by participants before or at the first workshop session) Program facilitator Application and facilitator’s Portal record Participant enrollment data RECEIVED THROUGH THE QTAC PARTNER

PORTAL “Add New Participant” button OR THROUGH THE “Enroll” buttons at

https://www.ceacw.org/find-a-workshop NYS DPP Participant Eligibility

Form

Partners will collect and return to QTAC Participant Information Surveys from 75% of workshop participants Partner will collect and enter into the Partner Portal contact and demographic information collected via a program facilitator Application for 100% of all facilitators who successfully pass the training

QTAC supported programs will reach 80% of targeted numbers of participants by program

50% of target population will have access to at least 1 EBI (Evidence-Based Intervention) within 30 miles of their home

5% increase annually in number of professional organizations and health care providers recommending EBIs as reported by registered participants or by professional staff directly registering participants for QTAC promoted EBIs

FOR NYS DPP only – 50% of workshop participants are eligible due to a blood-based diagnosis and/or past diagnosis of gestational diabetes

Efficacy/Effectiveness– Participant Outcomes

Evidence-Based Interventions have proven positive outcomes that have been tested and

confirmed in randomized control trials, translational studies and widespread dissemination.

We can reasonably expect that if program delivery is consistent with the intent of the

developers and the evidence-base, our program participants will achieve similar outcomes to

those demonstrated in research conducted for the individual programs (e.g., improved health

and symptoms, increased self-efficacy, increases in physical activity, weight loss, decreased

Page 7: Continuous Quality Improvement Plan for Evidence-Based ... · standards. Quality assurance is a data-driven process for team decision-making, problem-solving and program improvement

[RETURN TO TABLE OF CONTENTS]

QTAC CQI Plan (QTAC Partnership MOU ~ Appendix A) 5 (Revised 1/29/14)

healthcare utilization, etc.). The strong focus on program delivery fidelity then helps to ensure

that we can expect similar outcomes for our participants.

Because positive program effects are a valued outcome, we do ask that you collect for all

programs, as part of your Participant Information Survey, basic information about the health

and quality of life of the participants as well as unique outcomes/efficacy measures for each

intervention based on the findings from the original research. The QTAC will re-collect this

information from random samples of participants six months or more following the completion

of their program experience to determine to what extent, if any, program participation has

impacted these valued outcomes. All outcomes data will be reported in aggregate only. This

information may also be compared to state and national data. In some cases we are also able

to consult with partners to determine additional outcomes of value and help to design tools to

measure those outcomes (please email [email protected] or call 877-496-2780 for more

information).

Data Collection Details: To appropriately administer outcomes surveys, you will need to have

program participants complete the Participant Information Survey prior to the start of their first

program session. This may be done by asking participants to arrive approximately 30 minutes

early for their first session, or by holding a pre-workshop/program session (known as a Session

Zero), or by inputting the data electronically through the QTAC Partner Portal upon participant

registration. All data will be forwarded to the QTAC where it will be de-identified and securely

stored for a period of up to three years.

EFFICACY

Data Collection Point Partner Indicator QTAC/State Indicator

Participant Information Survey and/or supplemental outcomes information as designated by program

Partners will collect and return to QTAC information from 75% of workshop participants, using the Participant Information Survey and/or supplemental outcomes information as designated by program

QTAC will collect follow-up data for random samples of 10% of workshop participants completing the Participant Information Survey and/or supplemental outcomes information as designated by program Analysis of these findings will indicate positive impact on at least 50% of responding program participants for each measure

Page 8: Continuous Quality Improvement Plan for Evidence-Based ... · standards. Quality assurance is a data-driven process for team decision-making, problem-solving and program improvement

[RETURN TO TABLE OF CONTENTS]

QTAC CQI Plan (QTAC Partnership MOU ~ Appendix A) 6 (Revised 1/29/14)

Adoption – Partner Commitment

Successful broad dissemination of an EBI rests largely on Partner’s adoption of the program(s).

Adoption refers to an ongoing commitment to quality delivery of the program and an

understanding and commitment of the resources needed to achieve this.

Through current funding, the QTAC is able to assist Partners with training and technical

assistance. However, the future of the program rests largely on the resource commitment of

Partners and ongoing funding/reimbursement mechanisms. To that end, the QTAC is collecting

information about the dedicated resources of our Partners for delivery of EBI.

ADOPT I ON

Data Collection Point Partner Indicator QTAC/State Indicator

Partner resource information collected in “Add New Workshop” Partner Portal entries

Partners report utilizing non-grant related resources for the delivery of a QTAC promoted EBI in at least 50% of their workshops

50% of Partners report utilizing non-grant related resources for the delivery of a QTAC promoted EBI

Implementation – Quality Programming

Delivery of the program that is consistent with the evidence-base is critical to the quality and

fidelity of an EBI. The implementation phase of the program is where we have the most

significant impact on our participants and begin a change process that leads to improved

quality of life, health outcomes and public health impact. Understanding and improving our

program delivery is critical to quality implementation and therefore data collected in this phase

is key to continuous quality improvement efforts. Information related to workshop location,

duration, session length, group size (if applicable), facilitators, and participants are all important

to high quality implementation of EBI.

For details, please review Developing a High Quality Workforce to Deliver Evidence-Based

Interventions (EBIs) for Health (QTAC, 2014).

Page 9: Continuous Quality Improvement Plan for Evidence-Based ... · standards. Quality assurance is a data-driven process for team decision-making, problem-solving and program improvement

[RETURN TO TABLE OF CONTENTS]

QTAC CQI Plan (QTAC Partnership MOU ~ Appendix A) 7 (Revised 1/29/14)

Data Collection Point Partner Indicator QTAC/State Indicator

IMPLEMENTATION

“Workshops” Partner Portal entries Workshop participant attendance record Check-in Call Form (see page 11: Program Fidelity Check-in Call for details)

Participant Satisfaction Surveys

“Add New Workshop” Partner Portal entry and workshop participant attendance record will be completed for 100% of workshops delivered in partnership with the QTAC Workshop attendance record will indicate: at least 75% of workshop

attendees complete the workshop

at least 75% of workshops began with 10 or more participants registered or will have a QTAC-approved Waiver for Reduction in Class Size request in place where applicable

100% of all programs are led by required number of facilitators with valid training credentials

Partners ensure 75% of program facilitators respond to QTAC-initiated Check-in Call(s) for each program series Partners collect Participant Satisfaction Surveys from at least 60% of workshop participants

Workshop completers will average 75% by program for all Partners reporting; and 65% of workshops will begin with 10 or more participants registered 100% of all programs are led by required number of facilitators with valid training credentials Check-in calls will be conducted by the QTAC according to program protocol for at least 75% of all workshops. 75% of program facilitators will achieve a minimum fidelity score as indicated by summed scores from Check-in Call forms 75% of workshop participants completing Participant Satisfaction Survey will indicate satisfaction with programming

Page 10: Continuous Quality Improvement Plan for Evidence-Based ... · standards. Quality assurance is a data-driven process for team decision-making, problem-solving and program improvement

[RETURN TO TABLE OF CONTENTS]

QTAC CQI Plan (QTAC Partnership MOU ~ Appendix A) 8 (Revised 1/29/14)

Maintenance – Sustainable Systems

Maintenance refers to the willingness and ability of organizations and systems to commit to EBI

in the future. Many factors affect the extent to which sustainability occurs including resources,

program demand, program capacity and the long term outcomes achieved by both participants

and systems. Much of this data, including attrition rates and resources, are tracked through

other mechanisms previously cited in this document. Our focus here is the extent to which

quality delivery of the EBI is embedded in our partner organizations through shared resources

and quality program capacity mechanisms.

Data Collection Point Partner Indicator QTAC/State Indicator

MAINTENANCE

Partner resource information collected in “Add New Workshop” Partner Portal entries

Data collected through Partner Portal

Partners report utilizing non-grant related resources for the delivery of a QTAC promoted EBI in at least 50% of their workshops

At least 50% of program facilitators will facilitate the minimum number of programs/workshops required to remain active in a 12-month period 50% or more of program Trainers will maintain certification in their respective program At least 75% of active program Delivery Personnel (program facilitators, trainers and program coordinators) will attend an annual Quality Assurance meeting in person or via webinar, hosted by the QTAC or a QTAC Partner

50% of Partners report utilizing non-grant related resources for the delivery of a QTAC promoted EBI

At least 50% of program facilitators will facilitate the minimum number of programs/workshops required to remain active in a 12-month period 50% or more of program Trainers will maintain certification in their respective program

Page 11: Continuous Quality Improvement Plan for Evidence-Based ... · standards. Quality assurance is a data-driven process for team decision-making, problem-solving and program improvement

[RETURN TO TABLE OF CONTENTS]

QTAC CQI Plan (QTAC Partnership MOU ~ Appendix A) 9 (Revised 1/29/14)

When Performance Indicators Are Not Met – Process Improvement Strategies

Performance indicators are a critical component of the CQI in that they provide standards by

which each individual Partner and the overall QTAC and State effort can be measured and

improved. At times one or more of these performance indicators will fall short of these

standards despite our best efforts. When performance indicators are less than expected we are

compelled to consider possible explanations and ways to improve implementation and delivery

strategies using tools and standards embedded within the CQI and its performance indicators.

This represents an opportunity to review our processes to ensure that we are doing all that is

needed to meet performance standards.

Data supporting performance indicators is regularly observed; however, a formal semi-annual

analysis and review of performance data is made by the QTAC for each Partner. In addition, the

QTAC reports project performance to the Statewide Management Group on a semi-annual basis

or as requested by the group.

When a Partner does not meet one or more performance indicators, QTAC will work in

consultation with the Partner to identify and implement improvement strategies. A progress

review will take place 3 and 6 months following the implementation of improvement strategies

to ensure progress is being made and/or to address any additional barriers to improvement.

This process will continue until the Partner and the QTAC are confident that the indicator will

be reached.

Similarly, when the QTAC does not meet one or more performance indicators, the Statewide

Management Group for the QTAC will work with QTAC to complete an analysis and plan for

improvement which will be shared with all Partners for review and comment before a final

decision is rendered and implemented. QTAC performance indicators will be reviewed

quarterly and reported to the Statewide Management Group and Partners.

Page 12: Continuous Quality Improvement Plan for Evidence-Based ... · standards. Quality assurance is a data-driven process for team decision-making, problem-solving and program improvement

[RETURN TO TABLE OF CONTENTS]

QTAC CQI Plan (QTAC Partnership MOU ~ Appendix A) 10 (Revised 1/29/14)

Tracking Our Efforts

CQI and Program Fidelity are data driven processes. The tools, mechanisms and measures

utilized by the QTAC to track our program quality are provided to all of our Partners at the time

the QTAC Partnership MOU is executed. The QTAC is available to provide further guidance and

technical assistance to enhance your use of these

data and tracking tools for your organizational

reporting needs.

The following is a list of items that Partners are

required to track and/or assist the QTAC to track

in order to enhance our understanding of the

quality of your local program. It is followed by a

list of suggested activities to enhance program

quality.

NOTE: The majority of required data will be

gathered and entered into the NYS QTAC Partner

Portal.

REQUIRED TRACKING

Delivery Personnel Information: Facilitators

QTAC Partner will enter and update facilitator

information in the “Delivery Personnel” section of

the QTAC Partner Portal. QTAC Partner Program

Coordinator will make sure each program

facilitator trainee completes the QTAC program-

specific facilitator application (i.e., S-SMP Leader

Application) prior to being trained and will

maintain a copy in local program files. Program

Coordinators will also ensure previously trained facilitators have completed this form and will

maintain a copy in local program files. Information collected on this form must be entered

and updated by the Partner in the Partner Portal “Delivery Personnel” entries.

Delivery Personnel Information: Trainers (S-SMP ONLY)

Partner Program Coordinator will ensure that all S-SMP Master Trainers forward a copy

of their Stanford-signed Master Trainer Agreement and Certification form to the QTAC

upon receipt.

Partner will notify the QTAC of any changes (additions, deletions, revisions) in Master

Trainers or T-Trainers working under their Stanford License/QTAC Partnership MOU.

NYS QTAC PARTNER PORTAL HTTPS://PORTAL.CEACW.ORG/LOGIN

The QTAC Partner Portal is an online

registration and data management

tool developed to assist QTAC

Partners with local delivery of NYS

QTAC promoted evidence-based

health programs. Partners with a fully

executed QTAC Partnership

Memorandum of Understanding

(MOU) are the primary users of the

Portal. All partners will receive an

orientation and login information for

the use of the Portal. NYS QTAC

Partners also have the ability to

generate reports for their workshops.

Reporting functions for the Portal

include aggregate data reports on the

type of workshops held, location,

attendance and completion rates and

basic demographic profile of

participants.

Page 13: Continuous Quality Improvement Plan for Evidence-Based ... · standards. Quality assurance is a data-driven process for team decision-making, problem-solving and program improvement

[RETURN TO TABLE OF CONTENTS]

QTAC CQI Plan (QTAC Partnership MOU ~ Appendix A) 11 (Revised 1/29/14)

Participant Information Surveys and Attendance Log

Attendance records and demographic/contact/health status data are collected at the local level

and returned to the QTAC directly via the Partner Portal and/or mail for entry into the portal

and in some cases National Data Systems. QTAC de-identifies all participant information prior

to sharing through any other data system. Hard copies of data are held at the QTAC for three

years and then are destroyed.

Participant Satisfaction Surveys

These surveys are collected at the final session of every workshop/program. These surveys

provide workshop/program participants with an opportunity to give feedback to program

planners about the workshop/program held at a given location. Together with Program Fidelity

Check-in Calls, participant Attendance Logs and Fidelity Observation Checklists (if utilized), this

information assists in understanding the workshop/program experience for participants while

providing information that enhances the quality of programming and illuminates needed areas

for improvement.

Other Workshop Data Packet Forms

In addition to the aforementioned forms/surveys, an EBI may have additional required data

forms for participants or Partners to complete. Please be sure to download all required

workshop/program data forms from the Portal. To access program workshop forms located in

the QTAC Partner Portal, authorized QTAC Partner personnel must login to the Portal, click the

“Show” button beside any scheduled program workshop, and a details page will show with a

box containing all “Workshop Data Packet” forms for download. These forms are uniquely pre-

coded for that particular workshop.

Program Fidelity Check-in Call

Once a program/workshop series begins, workshop facilitators will receive a phone call(s) (or

email) from the QTAC or designated staff, asking a few questions related to program fidelity.

This is referred to as a “Program Fidelity Check-in Call”. Check-in Calls are one of several

strategies to obtain information about program facilitator performance. The Check-in Call

allows for the most flexibility and cost-savings to the QTAC, Program Coordinator and Partner

organization as well as provides the added benefit that if challenges are identified they can be

corrected in real time.

Program Coordinators must notify all workshop facilitators that they will receive a Program

Fidelity Check-in Call(s) from the QTAC during their workshop and that it is imperative that they

respond. Facilitators should expect the call/email response to take between 10-15 minutes.

Upon completion of the Check-in Call, QTAC personnel, including program trainers, will

carefully review these forms and will conduct appropriate follow-up with Program Coordinator

Page 14: Continuous Quality Improvement Plan for Evidence-Based ... · standards. Quality assurance is a data-driven process for team decision-making, problem-solving and program improvement

[RETURN TO TABLE OF CONTENTS]

QTAC CQI Plan (QTAC Partnership MOU ~ Appendix A) 12 (Revised 1/29/14)

as needed. The data is entered into a secure database and aggregate reports are generated on

a semi-annual basis to assist in development of Quality Assurance meetings, other facilitator

support activities, and to ensure QTAC Partner is meeting CQI Indicators. (For program-specific

details, such as when Check-in Calls are conducted, see “Collecting Information about

Facilitator Performance” section of Developing a High Quality Workforce to Deliver Evidence-

Based Interventions (EBIs) for Health (QTAC, 2014).) To further support program fidelity, QTAC

encourages Program Coordinators to use the Program Fidelity Check-in Call Form to make

additional Check-in Calls to their facilitators during the course of a workshop series.

Leader Training Information (S-SMP ONLY)

BEFORE Leader Training Begins

o Partner will add Training to NYS QTAC Partner Portal and notify the QTAC of the following information:

Training program name, location, dates, times Training Coordinator name, phone number, email Names of two Master Trainers co-leading the Training or request for

assistance in recruiting Inform NYS QTAC if you would like the training listed on the NYS QTAC

Calendar (https://www.ceacw.org/calendar) making it public so other agencies may send trainees to attend.)

o Partner will verify ALL trainees have completed a QTAC S-SMP Leader Application

and will review this application carefully, sign, and file in local program files

o Partner will follow the QTAC Leader Screening Protocol

DURING Leading Training, Partner will make sure:

o Trainees daily initial the QTAC Leader Training Roster (sign-in sheet)

AFTER Leading Training, Partner does the following:

o Send QTAC a copy of the trainee-initialed Leader Training Roster (sign-in sheet).

The following information is included on the Leader Training Roster: dates and

location of Leader training, names of ALL trainees (including non-completers)

and whether or not they successfully passed the training, names of the two

Master Trainers conducting the training.

o Program Coordinator updates trainees’ information in the Partner Portal.

o Send QTAC copies of the completed Leader Training Evaluation Forms (Master

Trainers will have these forms).

SUGGESTED TRACKING TO ENHANCE PROGRAM QUALITY

There is additional information that local programs and QTAC can collect to better understand

the program experience for participants and delivery personnel. Several suggested additional

activities that may be performed locally and/or by QTAC staff are listed below. It is hoped that

these suggested activities will assist local programs and QTAC in identifying areas for

Page 15: Continuous Quality Improvement Plan for Evidence-Based ... · standards. Quality assurance is a data-driven process for team decision-making, problem-solving and program improvement

[RETURN TO TABLE OF CONTENTS]

QTAC CQI Plan (QTAC Partnership MOU ~ Appendix A) 13 (Revised 1/29/14)

improvement and potential system enhancements that will strengthen program delivery

throughout NYS.

Fidelity Observations/Program Monitoring

Observations are conducted at the local level using standard QTAC forms and are performed by

trained fidelity monitors/personnel. This information will help you make quick course

corrections with facilitators if problems arise during the workshop/program – particularly

related to S-SMP delivery. Observations are generally needed when a newly trained facilitator

would like additional feedback and/or training support as they develop their skills.

Observations are at times requested by delivery personnel, program coordinators or program

administrators because a Check-in Call or other incidence has brought a potential facilitator

performance issue to the attention of program staff and/or QTAC. It is imperative that

facilitators receive timely feedback related to in-person observations. QTAC has a training to

help staff conducting in-person observations to provide this feedback. Please retain your copy

of Fidelity Observation Checklists in a secure location.

Facilitator Interview Data

Interviews are conducted by local program staff when a program facilitator requests contact or

when a report is made that requires follow-up (such a report can be made by another facilitator

or trainer, a participant or other workshop/program observer). The QTAC will inform local

program staff if a report is made through collected data and/or directly to the QTAC. A plan for

follow-up, either by local program personnel or the QTAC, will be developed and appropriate

follow-up documented.

Facilitator Exit interviews

Local program personnel may conduct exit interviews with inactive facilitators, that is, those

who have not facilitated a workshop/program in more than 12 months, using the QTAC

Program Facilitator Exit Interview Form. Exit interviews provide your program with additional

information about the quality and fidelity of your program. Feedback from the exit interviews is

shared, as appropriate, with regional and local program personnel and the QTAC.

Participant Phone Interviews

A phone interview with a program participant may be conducted when requested by a

participant or when a participant attends but does not complete the workshop/program.

Phone interviews with non-completers provide you with additional information about the

quality and fidelity of your program. Phone interviews are conducted by local program

personnel. Feedback from participant phone interviews is shared as appropriate with regional

and local program personnel and the QTAC. Contact the QTAC for the appropriate form.

Page 16: Continuous Quality Improvement Plan for Evidence-Based ... · standards. Quality assurance is a data-driven process for team decision-making, problem-solving and program improvement

[RETURN TO TABLE OF CONTENTS]

QTAC CQI Plan (QTAC Partnership MOU ~ Appendix A) 14 (Revised 1/29/14)

Documents Referenced Developing a High Quality Workforce to Deliver EBIs for Health (QTAC, 2014)

Located in QTAC File Cabinet: HTTPS://WWW.CEACW.ORG/QTAC/Q-FILES-QTAC

NYS DPP Participant Eligibility Form

This form must be downloaded from the QTAC Partner Portal*. You may view a SAMPLE form by clicking here.

Other S-SMP Forms SPERC Master Trainer Agreement and Certification form HTTP://PATIENTEDUCATION.STANFORD.EDU/LICENSING/CDSMPMT_AUTH.PDF

Fidelity Observation Checklists (located in S-SMP “Fidelity” folder) QTAC Leader Screening Protocol QTAC Leader Training Roster (will be available via the Portal soon) BOTH LOCATED IN S-SMP “TRAINING” FOLDER OF S-SMP FILE CABINET: HTTPS://WWW.CEACW.ORG/QTAC/Q-FILES-SMP

Participant Attendance Log/ Record

The “Attendance Log” must be downloaded from the QTAC Partner Portal* for each workshop/program series scheduled. This information is then recorded each week by the Partner into the Portal’s workshop “Schedule” section.

Participant Information Survey This survey must be downloaded from the QTAC Partner Portal*.

Participant Satisfaction Surveys This survey must be downloaded from the QTAC Partner Portal*.

Program Facilitator Application ALED: ALED Facilitator Application LOCATED IN ALED “TRAINING” FOLDER OF ALED FILE CABINET HTTPS://WWW.CEACW.ORG/QTAC/Q-FILES-ALED

NYS DPP: QTAC Lifestyle Coach Application LOCATED IN NYS DPP “TRAINING” FOLDER OF NYS DPP FILE CABINET HTTPS://WWW.CEACW.ORG/QTAC/Q-FILES-NYSDPP

S-SMP: S-SMP Master Trainer Application S-SMP Leader Application

BOTH LOCATED IN S-SMP “TRAINING” FOLDER OF S-SMP FILE CABINET HTTPS://WWW.CEACW.ORG/QTAC/Q-FILES-SMP

WWE Self-Directed: Not applicable

Program Facilitator Remediation Form

Located in QTAC File Cabinet: HTTPS://WWW.CEACW.ORG/QTAC/Q-FILES-QTAC

Program Facilitator Exit Interview Form

Located in QTAC File Cabinet: HTTPS://WWW.CEACW.ORG/QTAC/Q-FILES-QTAC

Program Fidelity Check-In Call Documentation Form

Located in the QTAC File Cabinet: http://ceacw.org/docs/QTAC/SMP/Check-InCall.pdf

QTAC Partnership MOU Please contact the QTAC to become a QTAC Partner by signing a QTAC Partnership MOU and appropriate program exhibits. A sample of these documents is located in QTAC File Cabinet: HTTPS://WWW.CEACW.ORG/QTAC/Q-FILES-QTAC

*NOTE: To access program workshop forms located in the QTAC Partner Portal, authorized QTAC Partnerpersonnel must login to the Portal, click the “Show” button beside any scheduled program workshop, and a details page will show with a box containing all “Workshop Data Packet” forms for download. These forms are uniquely pre-coded for that particular workshop. Please contact the QTAC for any assistance with accessing these forms.