Studying hospital quality using mixed methods

Elizabeth H. Bradley, PhDYale University

School of Medicine

2

Disclosure and Acknowledgments

The research is funded by the Agency for Health Care Research and Quality (#R01HS10407-01), and the Donaghue Medical Research Foundation (#02-102)

Conducted in collaboration with Genentech and the National Registry of Myocardial Infarction (NRMI) Investigators

3

Purpose of the presentation

Provide an example of using mixed methods to identify determinants of hospital quality

Highlight the benefits and special challenges in employing mixed methods

4

Background

ACC/AHA guidelines recommend beta-blocker prescription for patients hospitalized with acute myocardial infarction (AMI)

However, many patients do not receive beta-blockers after AMI, and hospitals vary substantially in rates of beta blocker use and improvement in those rates over time

5

Improvement in hospital beta-blocker rates, 1996-1999

0

1020

30

4050

60

70

8090

100

-0.5

0

-0.4

5

-0.4

0

-0.3

5

-0.3

0

-0.2

5

-0.2

0

-0.1

5

-0.1

0

-0.0

5

0.00

0.05

0.10

0.15

0.20

0.25

0.30

0.35

0.40

Change in beta-blocker use

Nu

mb

er o

f H

osp

ital

s

6

Objective

To identify success factors in hospitals’ increased rates of beta-blocker use for patients with AMI… what works?

7

Mixed methods

Qualitative study

Site visits in higher/lower performersIn-depth interviews with key staff

Quantitative study

Closed-ended survey of random sample of hospitals

8

Qualitative study

Site visits with in-depth interviews (n=45) in 8 hospitals with higher versus medium/lower performance in beta-blocker rates• 14 physicians • 15 nurses• 11 quality improvement staff• 5 administrators (senior and mid-level)

Constant comparative method for data analysis of qualitative data

9

A taxonomy to classify QI efforts along key dimensions

Goals – content, specificity, sharednessAdmin support – philosophy, resourcesClinical support – physician, nurse, ancillarySystems design – standing orders, pathways,

reminders, care coordinators, etc.Data feedback – validity, timeliness

Contextual factors – size, teaching status, system affil, financial constraints, market and regulatory pressures, etc.

10

Hypotheses about “what works”

In beta-blocker performance, presence of clinical champions and administrative support for quality improvement are more important than systems design interventions

Data feedback, especially when it is physician-specific, can be viewed as punitive and can backfire as an improvement effort

11

Quantitative study

Cross-sectional study of 234 hospitals from those participating 30+ months during Apr 96-Sept 99 in the National Registry of Myocardial Infarction (reflects 54.2% response rate)

Patients: 60,363 treated for AMI in these hospitals during 1998-1999, the years just after beta-blocker recommendations were widely published

Telephone survey of QI directors at each hospital

Hierarchical models to estimate p (high-rate hospital)

12

Hospital sample (n=234)

Beta-blocker ratesMean, range 60%;19% - 89%

Urban 83%

Teaching 39%

Annual AMI volume (median) 137 patients

13

QI efforts

Type of QI effort Prevalence

Standing orders 57%Clinical pathways 58%Educational efforts 76%QI teams 80%Care coordinators 50%Reminder forms 28%Computer support 34%

14

QI efforts (continued)

Type of QI effort Prevalence

Data feedback reports 97%Quarterly reports 81%Public display of data 34%Data reports last 6 months 39%Data are physician-specific 11%Has physician champion 92%

15

QI efforts and performance

QI effort Adj OR p-value

Standing orders 2.3 .066

Physician champion 10.5 .001

Admin support [1-5] 2.0 .009

Data feedback thatis physician-specific 0.1 .001

16

Qualitative study benefits

The qualitative study augmented conceptual background for quantitative study:

1. Taxonomy with which to characterize QI efforts, a multifaceted intervention

2. Hypotheses about systems interventions, clinical champions, administrative support, and data feedback

3. Comprehensible language for survey design

17

Special challenges of using mixed methods

Integration the qualitative and quantitative work – benefit comes from their integration but easy to split them off

The “juicy” ideas in qualitative work can be difficult to measure (organizational change, culture, etc.) and test in larger samples

Qualitative work often slows down and delays the quantitative work

Publishing mixed methods – a special challenge (length, reviewers’ tolerance for unknown methods)

18

Why use mixed methods?

“Grounds” our work, so that we ask the important questions, have realistic hypotheses, and use sensible language

Increases the potential that research will be more easily translated into practice

19

Conclusions

Mixed methods studies are particularly advantageous for some, but certainly not all, topic areas

Research inquiries that involve multifaceted interventions, interdisciplinary interactions, or innovation and organizational change are good candidates for mixed methods