academyhealth, june 9, 2008 washington, dc weifeng weng, gerald k. arnold,

Comparison of Physician Rankings on Performance Quality Composites in the Care of Hypertensive Patients

AcademyHealth, June 9, 2008Washington, DC

Weifeng Weng, Gerald K. Arnold, Eric S. Holmboe, Rebecca S. Lipner

ABIM and Maintenance of Certification (MOC)

ABIM certifies physicians in internal medicine and its subspecialties

Certification is time-limited: 10 year duration Renew certificates through MOC program MOC requires demonstration of

– Professionalism– Lifelong Learning– Cognitive Expertise– Practice performance

Practice Improvement Module (PIMTM)

Performance Report

Performance Report

Improvement

Chart audit Patient survey

Impact

plan

do

study

act

Practice survey

Based on Picker patient and CAHPS

surveys

Based on Wagner’s Chronic Care Model & IHI’s Idealized

Office Design

Evidence-based guidelines

Research Question In P4P programs

– Clinical measures dominate – Patient survey and Practice system survey

measures used less frequently– Typically, rewards awarded by relative ranking– Typically not all three data streams used

Do physician performance rankings (and rewards) vary considerably when different combinations of the three data streams are used?

Methods

Physician database with patient-level data Standardized composite scores for the three

data streams(1) Chart audit

(2) Patient survey

(3) Practice systems

Super-composite scores: Combine composites Examine changes in physician rankings

Physician and Patient Samples

659 Physicians– Mean Age: 44 (SD = 6.4), 26% female– 61% general internists, 39% subspecialists (largely

nephrologists and cardiologists)– 29% in solo practice

Patients– Chart audit: 13,096 patients, age 18-75, 51% male – Patient survey: 14,913 patients, age 18-75, 53% male

Chart Audit Individual Measures

Measures

Physicianlevel ICC

N forRm

=0.85Mean (SD) Rho 95% CIBlood pressure control 0.52 (0.19) 0.12 0.10 to 0.14 42LDL control 0.55 (0.20) 0.16 0.14 to 0.18 29Complete lipid profile 0.82 (0.22) 0.43 0.39 to 0.46 8Urine protein test 0.81 (0.26) 0.64 0.60 to 0.68 3Annual serum creatinine test 0.88 (0.16) 0.36 0.32 to 0.40 10DM co-morbidity documentation or screen test 0.93 (0.15) 0.59 0.54 to 0.64 4Aspirin therapy for eligible patients 0.74 (0.25) 0.40 0.36 to 0.44 9Smoking status and cessation advice and treatment 0.96 (0.08) 0.36 0.30 to 0.42 10Counseling for diet and physical activity 0.89 (0.18) 0.51 0.47 to 0.56 5

Outcome variables are risk adjusted for co-morbidity conditions:BP control <130/80 for pts with dm or stroke co-morbidities, <140/90 for rest.LDL control <100 for pts with major risks, <130 for pts with other risks, <160 for the rest.

Patient Survey Individual Measures

Measures

Physicianlevel ICC

N for Rm

=0.85Mean (SD) Rho 95% CIOverall hypertension care 0.88 (0.12) 0.20 0.17 to 0.23 22Encouraging/answering questions 0.83 (0.14) 0.19 0.16 to 0.22 24Providing information on Medication side effects 0.73 (0.17) 0.17 0.14 to 0.19 28Providing information on foods to eat & avoid 0.60 (0.17) 0.12 0.10 to 0.14 41Providing information on taking medication properly 0.86 (0.13) 0.19 0.16 to 0.22 24

Practice System Individual Measures

Measures (# of measures) Mean SD Alpha

Information Management (27) 0.54 0.24 0.89

Patient activation & communication (28) 0.56 0.23 0.88

Access & communication with patients (7) 0.63 0.27 0.73

Safety & efficiency (11) 0.72 0.24 0.82

Practice team (8) 0.72 0.32 0.82

Consultation & referral (4) 0.62 0.31 0.65

Practice system survey of 89 questions

Distribution of Physician Performance Composite Scores

C+P+SC+SC+PSystem (S)

Patient (P)

Chart (C)

-8

-4

-6

0

-2

4

2

6

Correlations among Composites

Chart Patient System C+P C+S C+P+S

Chart (C) 1.00 0.22 0.19 0.76 0.72 0.66

Patient (P) 1.00 0.15 0.77 0.22 0.63

System (S) 1.00 0.20 0.78 0.66

C+P 1.00 0.59 0.84

C+S 1.00 0.87

C+P+S 1.00

15% 18%

10% 10% 12%

15%15%

9%12%

14%

0%

10%

20%

30%

40%

Patient System C+P C+S C+P+S

Percent who change rankings by more than one quartile*

Baseline: Chart

* One quartile counts for 164 rank positions

Ranks better than chart Ranks worse than chart

8% 8%

2% 3% 2%

7% 8%

2%0.3%0.3%

0%

5%

10%

15%

20%

Patient System C+P C+S C+P+S

Percent who change rankings by more than two quartiles*

* Two quartile counts for 329 rank positions

Baseline: Chart

Ranks better than chart Ranks worse than chart

Examples of Extreme Discordance of Performance – more than three quartiles

Ranks and (z scores)

Physician Chart (C) Patient (P) System (S) C+P C+S C+P+S

1 24 592 83 356 15 187

(1.3) (-1.2) (1.1) (0.08) (2.4) (1.2)

2652 287 4.5 644 586 538

(-3.6) (0.23) (1.7) (-3.4) (-1.9) (-1.7)

Rank 1 = Best; Rank 659 = Worst

Conclusions

Measuring multiple dimensions in the quality of patient care is complex– Very moderate correlations among three data

streams

– Rankings change considerably depending on combinations

A profile that incorporates more than one aspect of patient care tells a different story than any one of them alone

Limitations and Future Research

Self-report data for chart and system data Participants are volunteers Need more robust risk adjusters Investigate other analytic approaches for

combining individual measures into composites

Investigate stability of pass/fail decisions