Practice-based Learning & Improvement: How to Incorporate

Trainees in QI/Patient Safety Activities as Part of GME

Joseph Conigliaro, MD, MPH, FACPJoseph Conigliaro, MD, MPH, FACP Center for Enterprise Quality and SafetyCenter for Enterprise Quality and SafetyAssociate Chief Medical Officer, Quality Associate Chief Medical Officer, Quality

and Patient Safetyand Patient Safety

ObjectivesUnderstand the definition and scope of QI and patient safetyBecome familiar with the epidemiology of adverse events in healthcareBuild a good foundation for understanding systems approach to patient safetyLearn about high reliability organizationsUnderstand and review quality and process improvement activitiesSummarize skills neededWhat about litigation?

Goal

Create a curriculum and/or teaching techniques that can be incorporated into your clinical practice

Practice Based Learning and Improvement

Residents must be able to investigate and evaluate their practices, appraise and assimilate scientific evidence, and improve their patient care practices.

Residents are expected to: 1. Analyze practice experience and perform practice-based

improvement activities using a systematic methodology

2. Obtain and use information about their own population of patients and the larger population from which their patients are drawn

PBLI Steps

Monitor practice Monitor practice

Reflect on or analyze practice to Reflect on or analyze practice to identify learning or improvement identify learning or improvement needs needs

Engage in learning or plan Engage in learning or plan improvement improvement

Apply new learning or improvement Apply new learning or improvement

Monitor impact of learning or Monitor impact of learning or improvement improvement

IHI Learning Domains

Learning DomainsLearning Domains

Health care as a Health care as a process, systemprocess, system

Variation and Variation and measurement measurement

Customer/Beneficiary Customer/Beneficiary knowledgeknowledge

Leading, following and Leading, following and making changes in making changes in healthcarehealthcare

CollaborationCollaboration

Social context and Social context and accountabilityaccountability

Developing new, locally Developing new, locally useful knowledgeuseful knowledge

Professional subject Professional subject mattermatter

Institute of Medicine Report Elevated Patient Safety Agenda

Top 10 Causes of DeathTop 10 Causes of Death

#1 Heart Disease#1 Heart Disease

#2 Cancer#2 Cancer

#3 Cerebrovascular#3 Cerebrovascular

#4 Pulmonary#4 Pulmonary

#5 #5 MEDICAL ERRORSMEDICAL ERRORS

#6 Pneumonia/influenza#6 Pneumonia/influenza

#7 Diabetes#7 Diabetes

#8 Motor Vehicles#8 Motor Vehicles

#9 Suicide#9 Suicide

#10 Kidney Disease#10 Kidney Disease

30,000-98,000 Medical Errors30,000-98,000 Medical Errors

The IOM Challenge

Challenged the health care system to reengineer Challenged the health care system to reengineer itself to ensure better outcomesitself to ensure better outcomes

Need a system thatNeed a system that– Integrates careIntegrates care– Emphasizes preventionEmphasizes prevention– Utilize evidence-based medicine and measures of Utilize evidence-based medicine and measures of

service and qualityservice and quality

Definition and Scope of Patient Safety

Definitions of error, adverse event, and Definitions of error, adverse event, and patient safety; and why they matterpatient safety; and why they matter

Scope of patient safetyScope of patient safety– What we call itWhat we call it– System focus, not individualSystem focus, not individual– What we measureWhat we measure– What we hope to achieveWhat we hope to achieve

VA National Center for Patient Safety

Patient safety is the identification and control of hazards that could cause harm to patients

Patient safety is the prevention of harm or injury to patients

Is Patient safety a euphemism for medical error?– Not really - medical error is poorly defined and often a

euphemism for blaming an individual

Patient safety is about providing a safe environment in which to practice

How can there be quality healthcare if it isn’t safe?

What is patient safety?

Semantics?

What do you call the following?– Wrong leg is amputated– Wrong medication is dispensed– Diagnosis is “too late” to save patient with meningitis– You almost go into the wrong room to do a lumbar

puncture

Hold judgment, but consider– Incidence and prevalence vary widely in major journals– Focus is reducing harm to the patient

Working definitions of patient safety, errors, and adverse events

VA: Adverse events are: – Are untoward incidents, therapeutic misadventures,

iatrogenic injuries or other adverse occurrences directly associated with care or services provided within the jurisdiction of a medical center, outpatient clinic or other facility.

– May result from acts of commission or omission (e.g., administration of the wrong medication, failure to make a timely diagnosis or institute the appropriate therapeutic intervention, adverse reactions or negative outcomes of treatment, etc.).

Human error is irrelevant

Patient Safety EpidemiologyHow many adverse events?

How many close calls?

Inpatient vs outpatient?

Very dependent on definitions and methodology

And consider the research evidence…

Retrospective studies (Brennan, et al, 1991))Retrospective studies (Brennan, et al, 1991))– 2-4% of hospitalizations2-4% of hospitalizations– 10-40% including close calls10-40% including close calls

Cross-sectional (Ely, et al 1995)Cross-sectional (Ely, et al 1995)– 50% with survey of Family Practice docs 2-20 years 50% with survey of Family Practice docs 2-20 years

experienceexperience

Prospective studies (Gopher, 1991; Andrews, Prospective studies (Gopher, 1991; Andrews, 1999)1999)– ICU observation: 1.7 events/patient/dayICU observation: 1.7 events/patient/day– Internal Medicine rounds: 50% of all admitted Internal Medicine rounds: 50% of all admitted

patients with 1-10 eventspatients with 1-10 events

96 - 98% Reliability in Hospitals?

What would What would 99.9%99.9% reliability reliability mean?mean?– 1 hour of unsafe drinking water every month1 hour of unsafe drinking water every month– 2 unsafe plane landings per day at O’Hare Airport in 2 unsafe plane landings per day at O’Hare Airport in

ChicagoChicago– 16,000 pieces of mail lost every hour16,000 pieces of mail lost every hour– 22,000 checks deducted from the wrong bank account 22,000 checks deducted from the wrong bank account

each houreach hour– 20,000 incorrect prescriptions every year20,000 incorrect prescriptions every year – 500 incorrect operations each week500 incorrect operations each week

**Multiply above numbers by 20 to 40X ~ Hospital Multiply above numbers by 20 to 40X ~ Hospital ReliabilityReliability

Systems Approach to Patient Safety

Systems model examples

Systems versus Person-Focused

No accountability?

Patient Outcomes

Individual &Organizational

Outcomes

Technologyand Tools

Organization

EnvironmentTasks

Person

Technologyand Tools

Organization

EnvironmentTasks

PersonPROCESSES:* care process

* other processes

It’s usually the system!(from UW-Madison Systems Engineering Initiative in Patient Safety)

What is the difference between focusing on the person and focusing on the

system?Person approachPerson approach– Focus on individualsFocus on individuals

– Blaming individuals for Blaming individuals for forgetfulness, inattention, or forgetfulness, inattention, or carelessness, poor productioncarelessness, poor production

– Methods: poster campaigns, Methods: poster campaigns, writing another procedure, writing another procedure, disciplinary measures, threat disciplinary measures, threat of litigation, retraining, of litigation, retraining, blaming and shamingblaming and shaming

– Target: IndividualsTarget: Individuals

System approachSystem approach– Focus on the conditions and Focus on the conditions and

environment in which environment in which individuals workindividuals work

– Building fault tolerance in a Building fault tolerance in a system of work to reduce system of work to reduce harm or mitigate its effectsharm or mitigate its effects

– Methods: creating better Methods: creating better systemsystem

– Targets: System (team, Targets: System (team, tasks, workplace, tasks, workplace, organization, physical organization, physical environment)environment)

High Reliability Organizations

Main theoretical construct in safety Main theoretical construct in safety literatureliterature

Learning organizations that make Learning organizations that make “everything everybody’s business”“everything everybody’s business”

Lessons learned from industryLessons learned from industry– Nuclear PowerNuclear Power– AviationAviation

Patient Safety Challenges

Medicine Views Errors as Failings Which Medicine Views Errors as Failings Which Deserve Blame - FaultDeserve Blame - Fault– Train and blame mentality pervadesTrain and blame mentality pervades

– Corrective Actions Focus on IndividualCorrective Actions Focus on Individual

No Blood No Foul PhilosophyNo Blood No Foul Philosophy– Many in health care ignore or downplay close callsMany in health care ignore or downplay close calls

– Is experience the best teacher? Who pays the tuition Is experience the best teacher? Who pays the tuition for learning from experience of managing for learning from experience of managing complications?complications?

A More Productive Approach

People Don’t Come to Work to Hurt People Don’t Come to Work to Hurt Someone or Make a MistakeSomeone or Make a Mistake

Systems Issues > Individual’s Fault or Systems Issues > Individual’s Fault or ProblemProblem

Common vulnerabilities that can be found Common vulnerabilities that can be found and fixed for EVERYONE, not just one and fixed for EVERYONE, not just one person/placeperson/place

Awareness and Shame May be Largest Hurdles

1999 1999 Survey at VA and Private Healthcare Survey at VA and Private Healthcare OrganizationsOrganizations– Only 27% Agreed that Errors were a Serious Only 27% Agreed that Errors were a Serious

ProblemProblem– 49% “Ashamed” by Error49% “Ashamed” by Error

1999 IOM report concurs1999 IOM report concurs

Multi-Causal Theory “Swiss Cheese” Diagram (Reason, 1991)

“Culture of Safety” and “High Reliability Organizations”

Safety is always on the “agenda” – Safety is always on the “agenda” – especially for top managementespecially for top management

Embrace information from close calls and Embrace information from close calls and hazard analysishazard analysis

Communication up and down the “food Communication up and down the “food chain”chain”

If you are not sure it is safe, it is not safeIf you are not sure it is safe, it is not safe

Empirical Evidence that Culture and Attitudes are Key

Effective system fixes with evidence meet Effective system fixes with evidence meet resistance from frontlinesresistance from frontlines– Sign the site for wrong-site surgerySign the site for wrong-site surgery– High concentration potassium removed from wardsHigh concentration potassium removed from wards

Root cause analyses keep finding the issuesRoot cause analyses keep finding the issues– Communication between various disciplinesCommunication between various disciplines– Failure to “speak up” when something looks “out of Failure to “speak up” when something looks “out of

whack” whack” 

Close Calls and High Reliability Organizations

Close calls 10-100 times > adverse eventsClose calls 10-100 times > adverse events

Noting them and dealing with them is a Noting them and dealing with them is a marker for HRO, culture of safetymarker for HRO, culture of safety

People more willing to analyze and delve People more willing to analyze and delve into close calls (less shame?)into close calls (less shame?)

Three Related Concepts

Human error is a symptom of trouble Human error is a symptom of trouble deeper in the system (it is the starting deeper in the system (it is the starting point, not the end)point, not the end)

To explain failure, do NOT try to find To explain failure, do NOT try to find where people went wrongwhere people went wrong

Find how people’s assessment and action Find how people’s assessment and action made sense at the time, given the made sense at the time, given the circumstances that surrounded them circumstances that surrounded them

Fundamental Skill Sets Trainees Need to be

Successful

Skill Sets that are Needed

Process Improvement Approaches Process Improvement Approaches and Toolsand Tools

Research Methodology & DesignResearch Methodology & Design

Measurement and Data ManagementMeasurement and Data Management

Skills in Change ManagementSkills in Change Management

Process Improvement Approaches and Tools

Demming ModelDemming Model

CQI and the PDSACQI and the PDSA

FADEFADE

Six SigmaSix Sigma

Lean ThinkingLean Thinking

Industrial engineering principlesIndustrial engineering principles

Patient Safety

Safety StandardsSafety Standards– PracticesPractices– EquipmentEquipment

Sentential EventSentential Event– Root Cause Analysis (RCA)Root Cause Analysis (RCA)

Failure AnalysisFailure Analysis– FMEAFMEA

Process ImprovementProcess Improvement

Change Management

All Too Often NeglectedAll Too Often Neglected– To affect positive change you need to engage To affect positive change you need to engage

people from multiple disciplines to change people from multiple disciplines to change their behaviors and the way they have their behaviors and the way they have functioned in the past.functioned in the past.

Understands motivational theoryUnderstands motivational theory

Can help lead the change processCan help lead the change process

VehiclesDepartment SpecificDepartment Specific– M&MM&M– Exit Rounds– Log and learning plan– Yearly reviewYearly review– Improvement project– Chief Resident LeadersChief Resident Leaders

CEQSCEQS– FellowshipFellowship– RCA and other activitiesRCA and other activities– Audit and Feedback sessionsAudit and Feedback sessions

Both (?)– Practice-based Small Group Learning Program– Evidence-based Medicine Curriculum

Vehicles

Example

Exit Rounds

Mortality & Morbidity Conference

Practice-based Small Group Learning Program

Evidence-based Medicine Curriculum

Log and learning plan

Improvement project

Other Program not listed

Arseneau R. Exit rounds: a reflection exercise. Acad Med 1995;70:684-7.

Exit Rounds

Group session with attendingGroup session with attending

Each resident reviews a discharged Each resident reviews a discharged patient for whom he/she was responsible patient for whom he/she was responsible

Describes what was learned from caring Describes what was learned from caring for that patientfor that patient

Mortality & Morbidity Conference

Focus on adverse on teaching servicesFocus on adverse on teaching servicesAnalyzes causes and consequences of each eventAnalyzes causes and consequences of each eventShould result in proposals for actions to avoid recurrence of similar Should result in proposals for actions to avoid recurrence of similar eventseventsM&M case assigned to each resident who analyzes case in M&M case assigned to each resident who analyzes case in terms of his or her own practice behaviors that could be terms of his or her own practice behaviors that could be improvedimprovedResident presents these issues during conferenceResident presents these issues during conference

Ziegelstein RC, Fiebach NH. “The Mirror” and “The Village”: Ziegelstein RC, Fiebach NH. “The Mirror” and “The Village”: a new method for teaching practice-based learning and improvement a new method for teaching practice-based learning and improvement and systems-based practice. Acad Med 2004;79:83-8.and systems-based practice. Acad Med 2004;79:83-8.

Practice-based Small Group Learning Program

Residents meet to review current Residents meet to review current information about a specific clinical information about a specific clinical problem and to reflect on their problem and to reflect on their experiences and challenges with it.experiences and challenges with it.

Group discussion is stimulated by Group discussion is stimulated by prepared material and led by a prepared material and led by a trained peer facilitatortrained peer facilitator

Foundation for Medical Practice Education (www.fmpe.org/en/programs/pbsg.html)Foundation for Medical Practice Education (www.fmpe.org/en/programs/pbsg.html)

Evidence-based Medicine Curriculum

Residents rotate as leaders of group session to discuss application Residents rotate as leaders of group session to discuss application of EBM to one of their own patientsof EBM to one of their own patients

As prep, residents develop focused clinical question, conduct a lit As prep, residents develop focused clinical question, conduct a lit search, critically appraise evidence, and apply to care of their own search, critically appraise evidence, and apply to care of their own patientspatients

Test using different scenarios or abstracts to assess ability to – compose relevant, concise, & searchable clinical questions

– conduct efficient literature search

– choose relevant & methodologically sound evidence

– calculate statistics relevant to diagnosis & treatment

Green ML, Ellis PJ. Gen Intern Med 1997;12:742-50.Smith CA, et al J Gen Intern Med 2000;15:710-5.

Log and Learning Plan

Working with a mentor, residents keep a log of Working with a mentor, residents keep a log of significant events or clinical surprises and significant events or clinical surprises and develop a plan to address learning needs develop a plan to address learning needs revealed by these events.revealed by these events.

Resident submits written analysis of critical incident or pattern of practice behaviors

Related learning plan are rated against specific criteria

Fung Kee Fung M. et al An internet-based learning portfolio in resident education: the Fung Kee Fung M. et al An internet-based learning portfolio in resident education: the KOALATM multicentre rogramme. Med Educ 2000;34:474-9.KOALATM multicentre rogramme. Med Educ 2000;34:474-9.ACGME. Advancing education in Practice-based Learning & Improvement. ACGME, 2004.

Improvement Project

Residents work with a mentor to identify an Residents work with a mentor to identify an aspect of their own practice that needs to be aspect of their own practice that needs to be improvedimproved

Implement the improvementImplement the improvement

Determine its effectiveness during senior yearDetermine its effectiveness during senior year

Completes it within 12 months

Rater uses a checklist to assess it

Lough JRM, Murray TS. Audit and summative assessment: a completed audit cycle. Lough JRM, Murray TS. Audit and summative assessment: a completed audit cycle. Med Educ 2001;35:357-63.Med Educ 2001;35:357-63.

At UKExample Perform in your program (yes/no)

Exit Rounds Ortho-postoperative conference

Surgery

Mortality & Morbidity Conference

Path Cardiology Otolaryn Pulm/CC

Diag Radiology OMS Surgery Nephr

Ortho Medicine Anesthesia

Practice-based Small Group Learning Program

Path Otolary Psych Pulm/CC

Ortho Cardiology Anesthesia Nephr

Evidence-based Medicine

Curriculum

Path Cardiology Otolary Pulm/CC

Ortho (Journal Clubs) Anesthesia Surgery Nephr

Log and learning plan Diag Rad Cardiology Otolary

Ortho Anesthesia

Improvement project Path OMS-prn Oto lary

Pulm/Critical Care Anesthesia Nephr

Other Program not listed Pulm/Critical Care-Systems Based Practice Project Nephr CQI

National Peer Review Protection?

Patient Safety and Quality and Patient Safety and Quality and Improvement ActImprovement Act– June 2005June 2005– Established broad confidentiality and privilege Established broad confidentiality and privilege

protections for info reported to PSOsprotections for info reported to PSOs– Data, reports, records, memos, analyses, Data, reports, records, memos, analyses,

written or oral statements that are reportedwritten or oral statements that are reported

Litigation?

Federal Rules of Evidence– Relevance (similarity and timing)– Danger of unfair prejudice

Admissibility as evidence– Process measures– Outcomes measures– Rankings

Specificity vs Aggregation

Kesselheim, et al.JAMA. 2006;295:1831-1834.

Conclusions

QI/Patient Safety activities should be focused on the system and not the individualPatient Safety is proactive with a focus to prevent patient harmBlame freeEngage trainees in Institution wide initiatives and aggregate data as needed

Useful Web Sites

http://www.improvementskills.orghttp://www.improvementskills.org (excellent tutorial site – (excellent tutorial site – free or $10 if you want CME)free or $10 if you want CME)http://www.patientsafety.govhttp://www.patientsafety.gov (tools and links to improve (tools and links to improve patient safety)patient safety)http://www.pqe.orghttp://www.pqe.org (partnership for quality education – (partnership for quality education – joint sponsor by RWJ and Pew Trust)joint sponsor by RWJ and Pew Trust)http://www.mceconnection.org/mce/http://www.mceconnection.org/mce/ (managed care (managed care education connection)education connection)http://www.ihi.orghttp://www.ihi.org (Institute for Healthcare Improvement) (Institute for Healthcare Improvement)http://www.ahrq.govhttp://www.ahrq.gov (Agency for Healthcare Research (Agency for Healthcare Research and Quality)and Quality)http://www.hce.orghttp://www.hce.org (Health Care Excel) (Health Care Excel)