levels of evidence from diabetes registries registry-based epidemiology? john m. lachin professor of...
TRANSCRIPT
LEVELS OF EVIDENCE FROM DIABETES REGISTRIES
Registry-based Epidemiology?
John M. Lachin
Professor of Biostatistics, Epidemiology and Statistics
The Biostatistics Center
The George Washington University
EuBIRO-D vs. USA
Ciao Fabrizio e Massimo
No regional or national healthcare program
No national or regional registriesHMO network
Translating Research into Action for Diabetes
Comparative Effectiveness ResearchAgency for Healthcare Quality and Research: patient satisfaction, quality of life
National Institutes of Health: Clinical outcomes
GRADE study
Science and Uncertainty
Jacob Bronowsky:
All information is imperfect. We have to treat it with humility... Errors are inextricably bound up with the nature of human knowledge…
The degree of uncertainty is controlled through the application of the scientific method,
and is quantified through statistics.
Statistical Test of an Hypothesis
Null Hypothesis (H0): The hypothesis to be disproven
The hypothesis of no difference.
Alternative Hypothesis (H1): The hypothesis to be proven
The hypothesis that a difference exists.
Two types of errors:Type I: False positive, probability Type II: False negative, probability
Power = 1 -
Factors that Affect and Power
Selection and Observational/Experimental Bias
Poor study design or executionMissing dataReproducibility (precision) of assessments
Missing DataThe Fundamental Issue - BIAS
Numerators and denominators may be biased
Estimates of population parameters, differences between treatments or exposure groups may be biased.
Statistical analyses, p–values and confidence limits may be biased.
p = 0.05 may mean a false positive error rate () much greater than 0.05;
N=800, 20% missing in treated/exposed, true ≈ 0.40.
Can’t Statistics Handle This?Not definitively.The magnitude of the bias can not be
estimated, no correction possible.Analyses can be conducted under certain
assumptions.But there is no way to prove that the
assumptions apply.Best way to deal with missing data is to
prevent it.
Sample Size Adjustments
Can adjust sample size to allow for losses-to-follow-up and missing data, e.g. increase N by 10% if expect 10% losses
BUT, this adjusts only for the loss of information,
NOT for any bias introduced by missing data.
Precision or Reliability of Measures
Reliability coefficient = proportion of total variation between subjects due to variation in the true values.
1 - = proportion of variation due to random errors of collection, processing and measurement.
Reliability ()
Po
we
r
Impact of ReliabilityPower decreases as decreases.
Impact of ReliabilityIf N is the sample size needed for a
precise measure then N/ is needed for an imprecise measure.
1.0 0.9 0.8 0.7 0.6 0.5
1/ 1.0 1.11 1.25 1.43 1.67 2.0
Impact of ReliabilityMaximum possible correlation between Y
and X is a function of the respective reliabilities: Max(R2) = x y
x y Max(R2)
1.0 0.9 0.90
0.9 0.9 0.81
0.9 0.7 0.63
0.9 0.5 0.45
0.7 0.7 0.49
0.7 0.5 0.35
Impact of Misclassificationsm = fraction of treatment or exposure
misclassifications, or fraction of outcomes misclassified
N/(1-2m)2 is needed
m 0 0.1 0.8 0.7 0.6 0.5
1/(1-2m)2 1.0 1.56 2.78 6.25 25.0 ∞
Randomized Clinical Trial Randomization:
•Subjects assigned to each treatment independently of patient characteristics
•No selection bias. Treatment groups expected to be similar for all variables measured and unmeasured.
•No confounding of the experimental treatment with other uncontrolled factors
•May infer a cause – effect relationship between treatment and the outcome, provided the trial is of good quality.
Randomized Clinical Trial Precisely defined population
Precisely defined exposure (the treatments)
Precisely defined outcome measure
Results clearly interpretable
Observational Study
Many types, e.g. case-control study
Prospective cohort study
No randomized controls
Maybe a precisely defined population
Maybe a precisely defined exposure (the treatments)
Maybe a precisely defined outcome measure
Observational Study
Many potential biases
Selection bias – composition of groups
Confounding with other factors
Statistical adjustments substituted for randomization
Observational Study
Necessary in settings where a randomized study is impossible
Smoking and lung cancer
Generally describe an association between the exposure factor and an outcome that may not represent a causal relationship.
Difficult to establish causality, though possible with replication of a highly specific association.
Observational Evidence
The essential issues with observational evidence is the degree to which an observed relationship can or can not be explained by
•other variables,
•other mechanisms, or
•biases
– even after statistical adjustment
Confounding
When the study factor (groups) are associated with another (confounding) factor that is a direct cause of the outcome.
Coffee consumption and cancer.
Coffee consumption confounded with smoking.
Higher fraction of smokers among coffee drinkers.
Statistical Adjustment for Confounding
Regression or stratification model including the study factor and the possible confounding factor(s)
Assumes that the operating confounding factors have been identified and measured.
Assumes that the regression model specifications are correct.
Statistical Adjustment for Confounding
Estimates the association of the factor with the outcome IF the confounding factor were equally distributed among the groups.
Difference in cancer risk between coffee drinkers and non-drinkers IF the fraction of smokers was the same among drinkers and non-drinkers.
Coffee drinking and smoking are alterable. Thus, the results would have a population interpretation.
Statistical Adjustments
NOT all covariate imbalances introduce bias, in which case adjustment itself introduces bias.
Gender inherently confounded with body weight
Gender adjusted for body weight estimates the gender difference if males and females had the same weight distribution.
Statistical Adjustments
Adjustment for weight provides a biased estimate of the overall male:female difference in risk in the population
But weight-adjusted estimate describes the additional male:female difference in risk, if any, that is associated with gender differences other than weight
Of mechanistic interest.
Omitted Covariates
Observational study can only adjust for what has been measured.
Adjustment for observed factors can not eliminate bias due to imbalances in unmeasured covariates.
Inappropriate CovariatesAnalysis should follow the prospective
history of covariates
Statistically invalid to define a covariate over a period of exposure that goes beyond the observation of an event.
Example, mean HbA1c over 5 years as a predictor of outcomes observed during the 5 years.
Rather, use the mean HbA1c up to the time of each successive event.
Confounding by Indication
In some cases, however, exposure to a factor (e.g. drug) may be confounded with the indications leading to the exposure.
Example: statins indicated in the presence of hyperlipidemia.
Recent data suggests that statin use may also increase risk of T2D in IFG/IGT.
But is the increased risk due to the statin use or the prior history of hyperlipidemia?
Confounding by Indication
In other cases an adjusting factor (e.g. dose) may likewise be confounded with an indication.
Example: Hemkens et al. analysis of the association of insulin glargine vs. human insulin with cancer in a German claims database.
14% decrease in age, gender adjusted risk.
But substantial dose imbalance.
14% increase in risk when also adjusted for dose.
Reasons for Dose ImbalanceConfounding by indication, or allocation
bias.
High or low glargine (or human insulin) dose may be determined by unmeasured patient factors that are differentially distributed within groups.
e.g. high glargine dose only administered to severely ill patients.
Impossible to statistically adjust for such confounding
Adjusted analysis results are biased.
Registries
Many types:
100% population captured, e.g. public health care system
Non-random subsample, e.g. insurance provider or hospital based
In latter case, registry population may not represent the full population of interest
Inherently prospective
But no standardized follow-up schedule
Registries
Relies on data capture in conjunction with the administration of medical care
No specific exposure of interest when established, in epidemiological sense
No specific outcome measure of interest.
Rather medical status and treatment recorded (possible exposures) and other major morbidities and mortality recorded (possible outcomes).
Registries
Epidemiologic analyses may be attempted.
But, difficult to precisely define exposure to a factor:
When is a subject
First at risk of being exposed (e.g. when is a drug introduced to the market?)
Actually first exposed (e.g. starts drug)
Removed from exposure (e.g. off drug)
Confounding by indication often an issue
Registries
Coding, classification of events may not be standardized
Often no adjudication
May be difficult to determine whether or exactly when an outcome event occurred, e.g. macroalbuminuria is “interval-censored”
May be difficult to determine when subject no longer at risk (right censored)
Incidence may be difficult to assess reliably.
Registries - Uses
Prevalence
Distribution of patient status or conditions in the population
Cross-sectional associations
If “representative” but not proportionally, weighted analyses can provide estimates in the broader population.
Disadvantaged populations (poverty, uninsured) may not be represented
Registries - Epidemiology
Exposure to a factor and outcomes
Open to many biases.
Statistical adjustments may be inadequate.
But, a registry can be the foundation for first-rate epidemiologic studies.
Registries - Epidemiology
Nested case-control studies
Sub-sample of possible cases that is carefully adjudicated
Sub-sample of possible controls (matched by follow-up time) also verified.
Exposure (risk) and confounding factors also verified.
Registries - Epidemiology
Prospective cohort studies
Identify eligible subjects -- representative of the registry (general) population
Formally enroll subjects (consent) with a systematic follow-up schedule
Careful characterization of exposure (risk) and confounding factors
Specific outcome reporting (assessments) with adjudication.
Registries - EpidemiologyEmbedded cohort study
Identify eligible subjects
Enroll subjects (consent)
Establish a schedule of assessments to be conducted as part of routine care
Send notices to patients when visits due
Capture exposure (risk) and confounding factors
Identify possible outcomes through medical reports, with subsequent adjudication.
Registries - Epidemiology
A hybrid
Establish an embedded cohort study.
Also implement a formal prospective study in a sub-sample.
The latter can serve as a quality check on the former.
Registries - Epidemiology
LARGE Sample Size
N needed to detect a rare outcome (e.g. fulminant hepatotoxicity, or angioedema)
If risk is 1 in 10,000, need N = 29,956 to be 95% confident that at least one case will be observed.
If wished to have 85% power to detect a 50% increased risk, at least 75 events required.
N = 836,000 followed for 1 year!!
Conclusions
Registry can provide superior descriptions of quality of care and distribution of factors in broad population of interest.
Not as rigorous as a formal prospective epidemiologic study, but can form the basis for such studies.
Affords opportunities for large sample sizes needed to detect rare outcomes.