Next Steps in Measuring Next Steps in Measuring Clinical Quality Clinical Quality
Joe V. Selby, MDJoe V. Selby, MD
Division of ResearchDivision of Research
Kaiser Permanente Northern CaliforniaKaiser Permanente Northern California
Differences in Clinical Quality – Differences in Clinical Quality –
Diabetes CareDiabetes Care
Plan A Plan B
Retinal Screening (P) 41.4 45.3
Hb A1c Testing (P) 72.7 75.1
Hb A1c Control (O) 61.9 55.2
Monitoring Nephropathy (P) 28.5 36.1
LDL-C Testing (P) 60.7 69.1
LDL-C Control (O) 29.1 36.7
*HEDIS Health Plan Summary Data
What We KnowWhat We Know WhatWhat We MeasureWe Measure
The Chasm in Clinical Quality AssessmentThe Chasm in Clinical Quality Assessment
Quantitative effects of many process measures and of differences in outcomes on survival and on non-fatal complications in populations – from clinical trials
Processes of care not known to be related to outcomes or effectiveness
Semi-quantitative outcomes (Hb A1c >9.5%, LDL-C < 100) that hide more effectiveness differences than they reveal
Population Rates for Simple Process Population Rates for Simple Process Measures do NOT Consistently Reflect Measures do NOT Consistently Reflect
Clinical Benefit in those PopulationsClinical Benefit in those Populations
Point #1Point #1
PacifiCare Texas
Kaiser Permanente No. California
Pacific Health Research Institute
U. Michigan
Indiana U.
UCLA
UMDNJ
CDC
Centers for Disease Control - Sponsor and Data Coordinating Center
TTranslating ranslating RResearch esearch IInto nto AAction for ction for DDiabetesiabetes
A multi-center cohort study of diabetes in managed care settingsA multi-center cohort study of diabetes in managed care settings
10 health plans 10 health plans (n=500 to 2000(n=500 to 2000per plan)per plan)
67 physician67 physiciangroups with groups with > 50 > 50 membersmembersin sampling in sampling frameframe
The TRIAD Sampling SchemeThe TRIAD Sampling Scheme
(Sampling scheme: Aimed for equal numbers from each physician group within health plan, so from 50 - 1500 per physician group)
TRIAD Data (2000-2001)TRIAD Data (2000-2001)
Patient Surveys (telephone or mailed) – 11,928 respondents
Chart Reviews – 8,757 patients
Medical Director Surveys – health plan and provider group directors
Four Measures of Disease Management Four Measures of Disease Management Intensity – from Health Plan and Provider Intensity – from Health Plan and Provider
Group Director SurveysGroup Director Surveys
Use of diabetes registries
Use of clinician reminders
Performance feedback to physicians
Diabetes care management:Guideline use
Patient reminders,
Patient education
Use of care/case managers
Provider Group Performance Difference (%) Provider Group Performance Difference (%) (80(80thth – 20 – 20thth Percentile of Dis Mgmt Intensity) Percentile of Dis Mgmt Intensity)
PROCESS MEASURESPROCESS MEASURES
Care Management
Performance Feedback
Diabetes Registry
MD Reminders
Hb A1c Test 11 0.001 9 0.0001 9 0.01 4 0.07
LDL-C Test 13 0.0001 8 0.001 11 0.01 2 0.59
Retinal Exam 7 0.01 8 0.001 4 0.13 7 0.001
Urine Albumin
16 0.0001 11 0.0001 13 0.01 10 0.01
Foot Exam 8 0.01 6 0.01 3 0.45 5 0.05
Aspirin Advised
0 0.99 1 0.74 3 0.30 3 0.38
adjusted for patient age, sex, race, education/income, diabetes treatment andduration, comorbidities, SF-12 (PCS), health plan disease mgmt intensity
Provider Group Performance DifferencesProvider Group Performance Differences (80 (80thth – 20 – 20thth Percentile of Dis Mgmt Intensity) Percentile of Dis Mgmt Intensity)
INTERMEDIATE OUTCOMES INTERMEDIATE OUTCOMES
Care Management
Performance Feedback
Diabetes Registry
MD Reminders
Hb A1c (%) 0.1 0.71 -0.1 0.74 -0.1 0.55 0 0.74
Syst. Blood Pressure (mmHg)
2 0.01 1 0.22 3 0.01 1 0.22
LDL-cholesterol (mg/dL)
2 0.06 2 0.70 2 0.46 0 0.70
adjusted for patient age, sex, race, education/income, diabetes treatment and duration, comorbidities, health plan disease mgmt intensity
Moreover,
Provider Group intensity of disease management also unrelated to the appropriateness* of treatment for each condition
Provider Group Quality Scores based on process measures were unrelated to provider group levels of control of blood pressure, LDL-C or Hb A1c
*Proportion in control or on appropriately aggressive pharmacotherapy
Point #2
Even if we measure evidence-based Even if we measure evidence-based processes or outcomes, the potpourri of processes or outcomes, the potpourri of indicators within and across diseases indicators within and across diseases
don’t readily yield a measure of overall don’t readily yield a measure of overall clinical benefitclinical benefit
Differences in Clinical Quality Differences in Clinical Quality (hypothetical) based on evidence-based (hypothetical) based on evidence-based
processes/ outcomesprocesses/ outcomes
Plan A Plan B
Patients Using Aspirin (%) 41 54
Mean Hb A1c (%) 8.1 7.5
Mean LDL-C (mg/dL) 106 131
Mean SBP (mmHg) 141 136
Flu Shot Past 12 mos (%) 67 54
How Do We Quantify the Net Benefit?How Do We Quantify the Net Benefit?
Each of these differences represents a predictable change in expected survival and complications (i.e., each measures a clinical benefit )
But practical questions remain:
Which is more important, the difference in Hb A1c levels or the difference in BP control?
Should plans, providers work to improve multiple measures modestly, or drive one indicator toward the optimal for all patients?
We need a composite, quantitative measure of net clinical benefit that can be compared across plans, provider groups, systems.
Quality-adjusted life-year
The QALY
A common metric for measuring clinical quality (both survival and quality of life)
Across interventions (using aspirin, BP lowering)
Across perspectives (patient, provider, purchaser)
Across diseases (diabetes, CHF, CAD, asthma)
Across activities (e.g., chronic disease care, prevention)
Where Do QALY’s Come From?Where Do QALY’s Come From?
Creating a Quantitative Metric for Creating a Quantitative Metric for DiabetesDiabetes
NaturalHistoryModel
SystolicBlood
Pressure
HemoglobinA1C
AspirinUse
LDL-Cholesterol
Expected Survival &
Complicatons
Adjusted Life-expectancy
RiskAdjusters
Potential Advantages of ModelPotential Advantages of Model
Expresses quality in familiar metric – life expectancy
Requires clinical trial evidence clearly evidence-based
Allows exploration to explain differences, which emphasizes population importance of various indicators
Potential Disadvantages/Questions Potential Disadvantages/Questions
Will require extensive explanation and transparency of the model to gain acceptance
New evidence will have to be incorporated over time, potentially altering metrics across years
Because it takes a population or public health perspective, will not capture quality of care well for rare conditions (because prevalence too small)
Questions of whether and how to adjust for case-mix differences between population will have to be addressed