Making Healthcare Safer 2:Critical Analysis of the Evidence for

Patient Safety Practices

Paul G. Shekelle, MD, PhD, on behalf of the Patient Safety Practice Team

Patient safety is a big problem

• 44,000 to 80,000 people die in the US each year due to diagnostic errors

• 68,000 people die of decubitus ulcers

• Many thousands more die from other preventable causes, e.g.,

– Teamwork and communication errors

– Failure to receive evidence-based interventions

• Costs of these errors are more than $30 billion (€23 billion)

Making Health Care Safer I

In 2000, the Agency for Healthcare Research and Quality commissioned a systematic review of patient safety practices

The report identified some early evidence-based safety practices:•General clinical topics•Organization, structure, and culture•Systems issues and human factors•Patient role

• Adverse drug event prevention• Infection control• Surgery, anesthesia,

perioperative medicine• Safety practices for

hospitalized or institutionalized elders

This report highlighted the enormous gap between what was known and what needed to

be known

Fast forward 10 years...

2000 2010

MHCS I MHCS II

MHCS II was a 4-year project conducted in 4 stages:•Developing a conceptual framework•Reviewing the evidence•Conducting a technical expert panel•Making recommendations

The conceptual framework comprised 6 factors for evaluating a PSP

• Explicit description of the theory on which the PSP rests

• Description of the PSP, including staff roles

• Measurement of contexts

• Details of implementation

• Assessment of outcomes and unexpected effects

• Assessment of influence of context on effectiveness

Source: Shekelle et al., Annals of Internal Medicine 2013

The Technical Expert Panel included 21 stakeholders and methods experts

Alyce Adams, Phd

Peter Angood, MD

David Bates, MD

Leonard Bickman, PhD

Pascale Carayon, PhD

Sir Liam Donaldson, MD

Naihua Duan, PhD

Donna Farley, PhD

Trisha Greenhalgh, MD

John Haughom, MD

Eileen Lake, PhD

Richard Lilford, PhD

Kathleen Lohr, PhD

Gregg S. Meyer, MD

Marlene Miller, MD

Duncan Neuhauser, PhD

Gery Ryan, PhD

Sanjay Saint, MD

Steve Shortell, PhD

David Stevens, MD

Kieran Walshe, PhD

The evidence review had 3 key concerns

• Context

• Implementation

• Adoption

Context has at least 4 domains

External factors

Organizational

structural characteristics

Management Tools

Teamwork, leadership,

patient safety culture

• Regulatory requirements

• Payments or

penalties, e.g., Public

reporting• National

campaigns, collaboratives, or sentinel

events

• Size, complexity,

• Location

• Financial Status

• Existing infrastructure

• Training resources

• Internal audit and feedback

Why is context important?

QualityImprovement

Research

Translating Research into Practice Is Envisioned as a Linear Process

Hospital

Hospital

QualityImprovement

Research

Bettercancer care

Reducedadverse drug

events

Hospital

Hospital

Translating Research into Practice Is Envisioned as a Linear Process

But That’s Not How it Works

Improving Patient Safety Is Like Gardening

Is this the right soil?

Has the soil been prepared?

Will it get enough sun?

Will it get enough water?

Some Plants that Grow Well inOne Environment…

…Will Always Fail in Another

Most Plants Will Survive in Different Environments

Most Plants Will Survive in Different Environments, but Not Thrive

Most Plants Will Survive in Different Environments, but Not Thrive

Unless the Environment Is Adapted to Support its Growth

MHCS II sought to understand the role of contextual factors in the implementation and

effectiveness of interventions to improve patient safety

How we conducted the reviews...

We searched for evidence in 2 ways

• We sought and extracted data about context, implementation, and unintended harms

– From studies that evaluated the effectiveness of safety strategies

• We identified implementation studies

– Studies that focused on elements of implementation processes shown or thought to influence the success of the intervention

Topic Refinement

• Began with a list of 158 potential PSPs

– 79 MHCS 1 topics + NQF 2010, Joint Commission, Leapfrog

• Conducted processes to combine and eliminate to reach 97 potential PSPs

– internal team triage

– conference with TEP

In-depth reviews (18) Brief reviews (23)

Review process involved several steps

1. Search for existing systematic reviews

– Assessing their potential relevance, quality

2. If relevant reviews found If no reviews found

3. Update search

or abbreviated searchFull search

We then characterized each strategy by...

• Scope of the underlying problem

• Strength of evidence about effectiveness

• Evidence or potential for harmful consequences

• Rough cost estimate

• Estimate of difficulty of implementation

Some examples...

Chapter 19. Preventing In-Facility Falls

The Problem:

• 1.3-8.9 falls per 1,000 bed-days occur in acute care hospitals

• 30%-50% of in-facility falls are associated with reports of injuries

• In-patient falls have been associated with $13,000 in increased costs and an increase of 6.3 days’ length of stay

What is the Patient Safety Practice?

Most in-facility fall prevention programs aremulticomponent interventions:

• Falls risk assessment

• Post fall review

• Patient education

• Staff education

• Footwear advice

• Scheduled and supervised use of toilet

• Medication review to assess for medications that affect alertness and balance

And many additional intervention components exist

Why should this Patient Safety Practice work?

Falls prevention interventions have no real identifiedconceptual framework, BUT

• Falls are understood to have a multifactorialetiology

• Attention to multiple risk factors is expected to have better outcomes than targeting a single factor

• Falls usually result from a combination of patient-specific risk factors (e.g., age>85, history of falling) and environmental risk factors (e.g., poor lighting)

• Insufficient staff time is often allocated for thorough fall risk assessment

What are the beneficial effects of the Practice?

Evidence from 3 systematic reviews and an updatesearch supported similar conclusions:

• Multi-component in-facility falls prevention programs significant reduce fall rates

• These programs reduce fall rates more effectively than do any single component interventions in acute care settings

And the interventions have not been associated withany actual adverse effects.

How has the Practice been implemented

and in what contexts?

Effective implementation in hospitals and seems toinvolve a number of consistent themes:

• Leadership support

• Engagement of front line clinical staff in intervention design

• Multidisciplinary committees to oversee interventions

• Pilot testing the intervention

• Informational technology systems

• Changing the prevailing attitude about the inevitability of falls

• Education and training of clinical staff

Cited as one of the strongest factors

for success

Chapter 24. Rapid-Response Systems

The Problem:

• Unrecognized deterioration in the clinical status of in-patients can progress to cardio-respiratory arrest,

• These arrests are associated with poor prognosis, YET

• Clear signs and symptoms predicting the arrest are often present

What is the Patient Safety Practice?

Rapid response systems:

• Combine improved recognition of deterioration and a critical care team to respond

• Comprise 4 components

– Criteria for notifying the response team (e.g., vital signs) and a system for activating it

– The response team (medical emergency team)

– Feedback loop to analyze event data for quality improvement

– Administrative component to coordinate staff, resources, equipment, and education

Why should this Patient Safety Practice work?

• Most cardio-respiratory arrests have clear antecedents indicating deterioration

• Yet these indications are not recognized or recognition is delayed

• Even when signs are recognized, a variety of systemic barriers prevent response

THEREFORE

• Improving the recognition process by defining criteria and spreading this knowledge should result in earlier recognition and intervention

What are the beneficial effects of the Practice?

Evidence from 1 high-quality systematic review and an update search showed:

• Among adults, implementation of a RRS was associated with reduction in cardiopulmonary arrest inpatients in general wards

• Among children, implementation of a RRS was associated with reduction of both cardiopulmonary arrest and in-hospital mortality

Effects are small but most studies were not welldesigned and didn’t do long-term assessment.

How has the Practice been implemented

and in what contexts?

• Most studies occurred in academic medical centers

• 5 major themes identified:

– Education on the medical emergency team

– Expertise

– Support of medical and nursing staff

– Nurses’ familiarity with and advocacy for the patient

– Nurses’ workload

• Improved implementation is associated with

– Technology and tools

– Staff and training

– Barriers and facilitators

Summary table

Topic

Scope of theProblem

Targeted bythe PSP

(Frequency/Severity)

Strength ofEvidence forEffectivenessof the PSPs

Evidence orPotential for

HarmfulUnintended

ConsequencesEstimate of

Cost

Implementation Issues:How Much Do WeKnow?/How Hard Is It?

FallsPrevention

Common/Low

High Moderate (increased use of restraints and/or

sedation

Moderate Moderate/Moderate

RapidResponse

Common/high

Moderate Low Moderate Moderate/moderate

Finally, an expert panel assessed each strategy for the priority for adoption

• They concluded 22 were ready to be encouraged for adoption

– 10 strongly encouraged

– 12 encouraged

Strategies strongly encouraged for adoption

• Pre-op and anesthesia checklists

• Bundles to prevent central line-associated infections

• Interventions to reduce urinary catheter use

• Bundles to prevent ventilator-associated pneumonia

• Hand hygiene

• The do-not-use list for hazardous abbreviations

• Multicomponent interventions to prevent pressure ulcers

• Barrier precautions to prevent health care-associated infections

• Use of real-time ultrasonography for central line placement

• Interventions to improve prophylaxis for venous thromboembolisms

Strategies encouraged for adoption (1)

• Multicomponent interventions to reduce falls

• Use of clinical pharmacists to reduce adverse drug events

• Documentation of patient preferences for life-sustaining treatment

• Obtaining informed consent to improve patients’understanding of the potential risks of procedures

• Team training

• Medication reconciliation

• Practices to reduce radiation exposure from fluoroscopy and CT

• Use of surgical outcome measures and report cards

Strategies encouraged for adoption (2)

• Rapid response systems

• Use of complementary methods for detecting adverse events or medical errors to monitor for patient safety problems

• Computerized provider order entry

• Use of simulation exercises in patient safety efforts

Future Research Priorities

• Sufficient data about the costs of PSPs to support cost-effectiveness analyses or return-on-investment analyses

• More patient safety measures for ambulatory care

• Better measures of the major causes of harm