Final Canadian National Delphi Consensus Results - What Are The Appropriate National Clinical

Pharmacy Key Performance Indicators (cpKPI) For Canadian Hospital Pharmacists?

  Olavo Fernandes BScPhm, ACPR, PharmD, FCSHP Director of Pharmacy- Clinical, University Health Network, Toronto ON

Assistant Professor (Status)- Leslie Dan Faculty of PharmacySean K. Gorman, BSc(Pharm), ACPR, PharmD

Regional Coordinator - Clinical Quality and Research, Pharmacotherapeutic Specialist – Critical Care Interior Health Authority, Clinical Associate Professor – Faculty of Pharmaceutical Sciences, UBC

Kent Toombs BSc(Pharm), ACPRClinical Pharmacy Manager, Capital District Health Authority, Halifax, NS

Canadian Hospital Pharmacy Leadership Conference , June 8, 2013

2

ObjectivesTo outline the key elements of the national consensus process in developing clinical pharmacy key performance indicators (cpKPI) for hospital pharmacists

including consensus definition, selection criteria for cpKPI,  critical topic/ activity foci and pre-Delphi candidate cpKPI)

To report the final results of the recent national Delphi consensus phase to establish a final suite of cpKPI

To summarize the next phases and communication plans in the national cpKPI process :

1. exploring interprofessional/ external stakeholder feedback,2. national information capture/ measurement systems, 3. cpKPI knowledge translation kit4. practical definition and measurement questions5. pan-Canadian communication/ Manuscript publications / posters

Overall Goal of the National cpKPI Collaborative / National Consensus Process

To develop a core set of national clinical pharmacy KPI for hospital pharmacists via a systematic national evidence-informed consensus process

3

Key Performance Indicators (KPI)

What is it?“Quantifiable measures that reflect the critical success factors of an organization” 1

Quantitative measures of quality

Why is it important?Elevate professional accountability & transparency Serve to improve quality of care

1. Doucette D, Millen B. Should Key Performance Indicators for Clinical Pharmacy Services Be Mandatory, Can J Hosp Pharm 2011; 64(1):55-57. 4

5

Rationale for clinical pharmacy KPI (cpKPI)

GAP: currently NO established national or international consensus on what constitutes a KPI for clinical pharmacy services

Rationale: To advance practice toward desired evidence-informed patient outcomes

cpKPI will serve to better define minimum standards and permit benchmark comparisons within and between organizations

National cpKPI CollaborativeDefinition of cpKPI

Five pillars/ characteristics of cpKPI:1. Reflect a desired quality practice and 2. A metric with a link to direct patient care and 3. Link to evidence of impact on meaningful patient

outcomes and4. A pharmacy/ pharmacist sensitive metric5. Feasible to measure

• Clinical metric would have to fulfill all 5 pillars to qualify as a candidate cpKPI

Hierarchy of Study Outcomes (AHRQ)

Level 1: Clinical and QoL outcomes• Morbidity, mortality, adverse events

Level 2 : Surrogate outcomes• I.e. blood glucose, blood pressure, cholesterol

Level 3: Measureable variables with an indirect or unestablished connection to target outcome• I.e. medication disease state knowledge

Level 4: Indirect variables • I.e. patient satisfaction, “potential adverse events”

Should Align with Local Consensus or Guidelines for Prioritization of Hospital Pharmacist Activities

6 Domains1. Pharmaceutical care patient care process2. Operational patient care supporting activities3. Drug information4. Teaching/Education/Learning5. Research6. Service (clinical and pharmacy committees)

*Extracted from UHN Pharmacist Pyramid-Prioritization of Pharmacist Activities Draft

Optimal National cpKPI

Literature: 1.Evidence2.Process

CSHP 2015/ CPhA Blueprint

Front-line Staff/Leaders

Peer Hospital Best Practices

Pharmacy Leadership

Information Gathering - Prior to Consensus Building

Proposed Timeline

CSHP endorsed concept

KPIWG formed

Information Gathering

Survey Development

Delphi Process

Consensus Meeting

May 2011

Aug 2011 Feb 2013

Pre-Delphi Delphi Post-Delphi

Dec’12-Mar’13 We are here

Key National Process Milestones1. National consensus definition – cpKPI (Aug 2011)2. National Crude Inventory of candidate cpKPI / metrics (started Jan

2012)3. National Information-gathering Process: Workshops/ Information

sessions-Front line feedback (Feb 2012- Nov 2012)4. Outcome and Process Debates/ Finalized Evidence summary tables

(June-July 2012)5. A priori consensus cpKPI selection criteria (ideal attributes)– “Slavik

11” (Finalized July 2012) 6. Key cpKPI Critical Activity / Topic Areas – “Doucette 8” (Finalized

August 2012) 7. Final Pre-Delphi Candidate cpKPI list (October 2012)8. Selection of National Delphi Panel members (November 2012)9. Delphi Panel Process – Round 1-3 (Dec 21, 2013- Mar 8 2013)

• cpKPI Live Meeting (February 5, 2013)

11

Slavik -11- Consensus Criteria – Ideal Attributes Based on high quality literature evidence(e.g. Observational data vs. RCT vs. systematic review) Relevant impact on clinically important outcomes(e.g. Surrogate versus clinical endpoints, effect size of intervention) Best-suited to pharmacist’s role (e.g. Identifies pharmacist-specific clinical role vs. GP vs. RN) Attributable to direct patient care(e.g. Marker of clinical intervention, not distribution) Specific to pharmaceutical care process(e.g. Related to generally-accepted PC processes) Aligned with professional goals, objectives, practices(e.g. Accreditation Canada ROPs, standards, CSHP Vision 2015, etc.)

12

Slavik -11- Consensus Criteria – Ideal Attributes

Accepted disease-based quality indicator(e.g. ACEI or BB for HF, VTE prophylaxis in hospitalized patients) Feasible to measure(e.g. Reliable measurement systems can/could be put in place) Efficient to measure(E.g. Acceptable time commitment, useable) Valuable quality measure(E.g. Prevalent, impactful problem with practical, proven interventions) Generalizability(E.g. Versatile enough to be applied in large, academic and small community sites)  13

Delphi panelist priority ranking of consensus cpKPI selection criteria- Final – Mar 2013

14

Bringing the evidence all together with extrapolation………Bond et. al. (2007) Observational

Study Clinical Pharmacy & Mortality 1. admission drug histories 2. medical rounds participation 3. CPR team participation

Kaboli PJ et al. (2006) Systematic Review4. attendance on patient care

rounds5. patient interviews and

assessments6. medication reconciliation7. discharge “counselling”

(patient medication education)8. follow-up after discharge

RCT Outcome FindingsGillespie U et al. 2009- RCTIntegrated Intervention pharmaceutical

care Integrated Intervention 1. post-discharge hospital visits (ED +

readmissions)2. emergency department visits3. drug related readmissions

Makowsky MJ et al. 2009- RCT1. “overall quality score” 2. 3 and 6 month all-cause readmission

(hospital or ED visit after index hospital admission)

Chisholm-Burns MA et al 2010, systematic review w/ focussed meta-analysesHbA1c , LDL Cholesterol, Blood PressureAdverse Drug Events

Evidence Summary Tables

Discussion: specific group suggestions to modify or concur with the follow sections

• Strengths and Limitations • Application/Synthesis: How does this study inform the cpKPI

selection process  (methods, cpKPI selection criteria, and candidate cpKPI)?

• What are the patterns (similarities and differences) compared to other key papers?

Purpose: August- used to refresh and focus outcome evidence for streamlining ; Nov-  Used by Delphi panelists to support ranking and decision making

16

Levels of Evidence

1. Observational Studies2. Systematic Reviews3. Randomized Controlled Trials

18

PRACTICE QUESTION

Does pharmacist-led comprehensive pharmaceutical care reduce morbidity (& other meaningful patient outcomes) for elderly hospitalized patients?

A Comprehensive Pharmacist Intervention to Reduce Morbidity in Patients 80 years or Older Gillespie U et al. Arch Intern Med 2009; 169(9):894-900.

Objectives: assess the effectiveness of interventions performed by ward-based pharmacists on morbidity and overall use of (secondary) hospital care

Design:prospective, single centre, unblinded, randomized control trialpatient- unit of randomization, central centre

Setting: 2 acute internal medicine wards (university teaching hospital) in Uppsala, SwedenDuration: Oct 2005-June 2006Patients:

Patients 80 years or older admitted to 2 acute care internal medicine wardsWritten informed consentSample size calculation : 400 patients

A Comprehensive Pharmacist Intervention to Reduce Morbidity in Patients 80 years or Older Gillespie U et al. Arch Intern Med 2009; 169(9):894-900.

Patients randomized to:intervention (comprehensive care by hospital pharmacist)• Ward based clinical pharmacists1. comprehensive patient interview, BPMH, admission medication

reconciliation, 2. pharmaceutical care drug review (Cipolle method) to identify and

resolve DTPs, physician interventions on drug selection, dosages, monitoring….1. Addressed: indication, effectiveness, safety and adherence2. DTPs discussed on ward rounds

3. Patients received education and discharge counselling/ reconciliation

4. pharmacist discharge letter communicated to primary care physicians by pharmacists

5. Follow up telephone call 2 months after dischargecontrol: standard care without pharmacist involvement by physicians and nurses

Results: Major Outcomes Gillespie U et al. Arch Intern Med 2009; 169(9):894-900.

Patients Evaluated (n=368, 182 intervention / 186 control) over a 12 month period

Post-Discharge Hospital Visits (ED + readmission)↓ 16% intervention group

(quotient 1.88 vs. 2.24, 95% CI 0.72-0.99)

Emergency Department Visits:↓ 47% intervention group

(quotient 0.35 vs. 0.66, 95% CI 0.37-0.75)

Drug Related Readmissions:↓ 80% intervention group

(quotient 0.06 vs. 0.32, 95% CI 0.10-0.41)

Aside: Balancing Measures- Readmissions Alone and Mortality :- No significant difference

Bringing the evidence all together with extrapolation………Bond et. al. (2007) Observational

Study Clinical Pharmacy & Mortality 1. admission drug histories 2. medical rounds participation 3. CPR team participation

Kaboli PJ et al. (2006) Systematic Review4. attendance on patient care

rounds5. patient interviews and

assessments6. medication reconciliation7. discharge “counselling”

(patient medication education)8. follow-up after discharge

RCT Outcome FindingsGillespie U et al. 2009- RCTIntegrated Intervention pharmaceutical

care Integrated Intervention 1. post-discharge hospital visits (ED +

readmissions)2. emergency department visits3. drug related readmissions

Makowsky MJ et al. 2009- RCT1. “overall quality score” 2. 3 and 6 month all-cause readmission

(hospital or ED visit after index hospital admission)

Chisholm-Burns MA et al 2010, systematic review w/ focussed meta-analysesHbA1c , LDL Cholesterol, Blood PressureAdverse Drug Events

Doucette 8- Consensus Critical Activity / Topic Areas

1. Pharmaceutical Care – Integrated (DTP assessment/ care plan/ monitoring)

2. Medication Reconciliation- BPMH/Med History Taking3. Medication Reconciliation- Admission Reconciliation4. Medication Reconciliation- Discharge Reconciliation5. Team (or Patient) Rounds6. Discharge Patient Education / Counselling 7. Post Discharge Follow-Up8. Disease or Drug Specific – Best Practice Quality

Indicators

23

Doucette 8- Consensus Critical Activity / Topic Areas

• Dot voting: 20 dots per person• Assign proportionatelyQuestion: • Will measuring a cpKPI in this “critical

activity topic area” be useful to advance clinical pharmacy practice to improve the quality of patient care?

• Semchuk-26 Draft Candidate KPI list24

DEMOGRAPHICS OF cpKPI DELPHI PANEL

25

Other= Administration, Pharmacy Association, Oversee multiple sites, Regional Health Authority, Long Term Care and Rehabilitation Centre, District health authority with tertiary and community practice.

Teaching hospital Tertiary care

hospital

Community hospital Academia Other Clinic

69% (18)

35% (9)27% (7) 27% (7)

23% (6)

12% (3)

What is your practice setting (check all that apply)?

88% (23)

12% (3)

Do you work primarily with pediatrics or adults?

Pediatrics

Adults

How many years of experience do you have as a licensed Pharmacist?

0% (0)

0-5 years 6-10 years 11-15 years 16-20 years 20+ years

65% (17)

8% (2)8% (2)

19% (5)

Other: MBA, BSc (Pharmacology), EXTRA Fellowship (CFHI) Certified Health Executive (CHE), MBA, Post PharmD Residency, Certified Geriatric Pharmacist

What is your educational background?

BScPhm PharmD Residency (ACPR) Other Masters

Degree

100% (26)

54% (14) 54% (14)

23% (6) 19% (5)

DOUCETTE 8 – 20 Dot Voting RESULTS

30

Doucette 8- Consensus Critical Activity / Topic Areas1. Pharmaceutical Care – Integrated (DTP assessment/ care

plan/ monitoring)2. Medication Reconciliation- BPMH/Med History Taking3. Medication Reconciliation- Admission Reconciliation4. Medication Reconciliation- Discharge Reconciliation5. Interprofessional (team) patient care rounds6. Discharge Patient Education / Counselling 7. Post Discharge Follow-Up8. Disease or Drug Specific – Best Practice Quality Indicators

• Used to create “Semchuk 26” candidate cpKPI list

31

Grape Analogy: BUNDLES AND CRITICAL ELEMENTS

Modified Delphi Process Methodology

A Delphi technique is a structured process commonly used to develop consensus healthcare quality indicators It was developed to minimize influence from more vocal group members, and utilizes surveys or questionnaires instead of discussion.

frequently used with expert panels to generate consensus on healthcare issues

To arrive at consensus, a modified Delphi technique will be used.

This ‘modified” technique is an iterative process that builds consensus using three rounds of anonymous panelist ratings with a live/tcon meeting

34

Delphi RoundsA. Standardized Orientation

• Audio PowerPoint + Mandatory Pre-ReadingB. Round 1

• Demographic Information; Panelist ranks Semchuk 26 cpKPI, For each Slavik 11 and Overall Ranking, Suggest new cpKPI

C. Round 2• Review R1 aggregate summary/ report card for each cpKPI• Frequency Graphs Summary• Review anonymous qualitative comments• Panelist re-ranks all cpKPI

D. Live Meeting – Debate and Discussion to inform individual rankings• identify meet other panelists for the first time

E. Round 3Review Feb 5 Live Minutes , R2 summaries (as above), Final Rankings

35

Delphi Rounds

1. Individual cpKPI ratings 2. Opportunity to suggest additional candidate cpKPI (round 1 only

to allow panel feedback)3. Ranking of priority of “Doucette 8” Critical Activities and “Slavik

11” Selection Criteria4. Combining cpKPI, Creating New cpKPI by modifying working (ie

cpKPI 27, 28, 30)

• Threshold for consensus consideration: • 75% of panelists assign a rating of 7-9 on the 9 point Likert scale• MAGIC NUMBER = 20

36

Round 1 Qualitative Panelist Discussion Themes

• Qualitative discussion themes while comparing cpKPI included: 1. Varying degrees of sensitivity to pharmacists’

contribution2. Varying degrees of feasibility of measurement3. Varying degrees of generalizability across practice

areas (i.e.. psychiatry, surgery) as well as across different types of hospitals (i.e. urban versus rural)

4. Inter-relationships between: medication reconciliation cpKPIs; discharge/ inpatient counselling cpKPIs

37

Round 1 – 3 New cpKPI Submitted by Panelists

1. cpKPI #27: combined pharmacist admission Med Rec + BPMH Number (or proportion) of patients who receive formal documented admission medication reconciliation by a pharmacist (includes a pharmacist-BPMH OR pharmacist-BPMH-review as part of reconciliation as well as resolution of identified discrepancies).

2. cpKPI #28: Proactive bundle; Number (or proportion) of patients receiving “proactive comprehensive, direct patient care by a pharmacist in collaboration with the health care team” (Makowsky Collaborate RCT Bundle).

3. cpKPI #29: Time on Ward Committed decentralized clinical pharmacist time per patient day per patient service.

38

Live Feb 5: Meeting Issues and Controversies

1. How to optimally handle process of care vs. disease/drug-specific indicators?• High Value Action “DTP resolved” as a subset

2. Grape Theory: Bundles and Critical Elements3. Number vs. proportion4. A priori Suite properties5. “High Risk vs. All Patients”

 

Final Delphi ResultsRound 3 Final Rankings

8 cpKPI have officially met consensus 6/8 Doucette Categories represented with combos  

 

Final 8: cpKPI Number and Description

Proportion of patients who receive formal documented discharge medication reconciliation and resolution of identified discrepancies by a pharmacist (#11)

Number (or proportion) of patients who receive formal documented admission medication reconciliation by a pharmacist (combined BPMH) (#27)

Number (or proportion) of patients for whom clinical pharmacists have completed (executed/implemented) a pharmaceutical care plan (#27)Number (or proportion) of pharmacists who actively participate in interprofessional patient care rounds to improve medication management

Final 8: cpKPI Number and Description

Number of total drug therapy problems (DTPs) resolved by pharmacists

Number (or proportion) of patients receiving "proactive comprehensive, direct patient care by a pharmacist in collaboration with the health care team" (Makowsky Collaborate RCT Proactive Bundle) (#28)

Number (or proportion) of hospitalized patients who receive medication counselling by a pharmacist at discharge

Number (or proportion) of patients who have received in person education from a pharmacist about their disease(s) and medication(s) during their hospital stay

How do the final national clinical pharmacy key performance indicators align with national consensus selection criteria?

43

National cpKPI CollaborativeNext Steps 7 Post-Delphi Phases1. cpKPI knowledge translation kit- practical getting started kit

• Final 8 cpKPI-specific measurement summaries, background, 7 step change management framework

2. Final 8 : Practical Outstanding Questions• Wording, outstanding questions, practical definitions, practical measurement issues

3. Exploring external stakeholder feedback• Interprofessional : physicians, nurses, Ministry of Health, pharmacists- US,

UK, NZ, Aus, patients, hospital administrators4. National information capture systems / measurement systems (“apps”)5. Pan-Canadian Communication of Final Delphi Results6. 2 Manuscript Publications/ 7 Conference Abstracts7. Formal “Pilot” Sites