randomized controlled trials clinical research center samsung medical center
DESCRIPTION
Randomized controlled trials Clinical Research Center Samsung Medical Center. Randomized controlled trials. Eliseo Guallar, MD, DrPH [email protected] Juhee Cho, MA, Ph.D. [email protected] Gee Young Suh, MD, Ph.D. [email protected]. Cohort Study: issues for concern. Confounding - PowerPoint PPT PresentationTRANSCRIPT
Randomized controlled trials
Clinical Research CenterSamsung Medical Center
1
Randomized controlled trials
Eliseo Guallar, MD, [email protected]
Juhee Cho, MA, [email protected]
Gee Young Suh, MD, Ph.D. [email protected]
2
Presenter’s Name
Date
Confounding
Selection bias
Misclassification of exposure and out-come
Cohort Study: issues for con-cern
3
Presenter’s Name
Date
In the design of the studyRandomizationRestrictionMatching
In the analysisStandardizationStratification Multivariate models Inverse probability weighting
Sensitivity analysis
Methods to control for confounding
4
Presenter’s Name
Date
Learning objectives
To review the key aspects of designing RCTs To review the design methods used to protect
against selection bias, information bias, and confounding used in RCTs
To understand the basic analytical methods to protect against selection bias in RCTs
To discuss ethical issues, reporting standards, conflicts of interests, and other issues related to the role of RCTs in clinical research
5
Presenter’s Name
Date
Friedman LM, Furberg CD, DeMets DL. Fundamentals of clinical trials. 3rd ed. New York, Springer-Verlag, 1998
6
Presenter’s Name
Date
Piantadosi S. Clinical trials. A methodological perspective. 2nd ed. New York, Wiley, 2005
7
Presenter’s Name
Date
10
Presenter’s Name
Date
11
Presenter’s Name
Date
12
Presenter’s Name
Date
James Lind and Scurvy
Aboard Salibury, 1747 Non-randomized Parallel group 6 groups (n=2)
Quart of cider a-day 25 drops of elixir vitriol x 3/day Two spoonfuls of vinegar x 3/day Sea water, half a pint a day Two oranges and lemon for 6 days Bigness of nutmeg x 3/day
13
Presenter’s Name
Date
14
Presenter’s Name
Date
Medical Research Council. BMJ 1948;2:769-8215
Presenter’s Name
Date
An experiment designed to assess relative efficacy of a test intervention in comparison to one or more alternative interventions, in comparable groups of human beings
Clinical trial: Definition
16
Presenter’s Name
Date
Participants are assigned to one of two or more interventions using an explicit method that assures the assignment will be random, or by chance
“Similar” to flipping a coin
What is randomization?
17
Presenter’s Name
Date
Pharmaceutical Drug treatment, preventive treatment, natural or
synthetic products Device
Prosthesis, ICD, thermal balloon Procedure
Surgery, laser, radiological intervention Behavior change
Smoking cessation, dietary change, exercise Other
Counseling, information provision
Types of interventions in clinical trials
18
Presenter’s Name
Date
Intensity (dose), duration, and frequency of the intervention
Feasibility of blinding Single intervention vs. combination of inter-
ventions Compliance with intervention Generalizability to clinical practice Balance between efficacy and safety
Choice of interventions in clinical trials
19
Presenter’s Name
Date
Primary vs. secondary outcomes Main endpoint – sample size
Clinical outcomes vs. surrogate markers Clinical importance Cost of measure Length of follow-up Number of patients
Single clinical outcomes vs. composite outcomes Mortality as an outcome Adverse events
Not powered to detect difference in side effects Trails included in applications for drug approval
Choice of outcome measures in clinical trials
20
Presenter’s Name
Date
Single center Participants recruited at one site Single site usually also responsible for data col-
lection, management, analysis
Multicenter Participants recruited at >1 site Usually has data coordinating center and other
resource centers
Single vs. multicenter trials
21
Presenter’s Name
Date
22Van den Berghe N Engl J Med 2001;345:1359
N=1548, surgical ICU
Presenter’s Name
Date
23
N=6104, 42 hospitals, both medical and surgical
THE NICE-SUGAR Study Investigators N Engl J Med 2009;360:1283
Presenter’s Name
Date
Basic structure of parallel group randomized controlled trial
Assess eligibility
Randomize
Test intervention Comparison
Follow-up for outcomes Follow-up for outcomes
24
Presenter’s Name
Date
Clear case definition Balance
Number of cases eligible for the trial Risk of outcome Likelihood of benefit from intervention Generalizability
Design adequate sample size Consider stratification Consider run-in period
Selection of study participants (in-clusion criteria)
25
Presenter’s Name
Date
A study treatment may be harmful High risk of adverse reaction to intervention Unacceptable risk of assignment to placebo
Active treatment unlikely to be effective At low risk of outcome Type of disease unlikely to respond Taking a treatment that interferes with intervention
Unlikely to adhere to intervention Unlikely to complete follow-up Practical problems with following protocol
Reasons for excluding participants from a trial
26
Presenter’s Name
Date
27
Presenter’s Name
Date
The PROTECT Investigators N Engl J Med 2011;364:1305-1428
Presenter’s Name
Date
29The PROTECT Investigators N Engl J Med 2011;364:1305-14
Presenter’s Name
Date
INTEVENTION
30The PROTECT Investigators N Engl J Med 2011;364:1305-14
Presenter’s Name
Date
Outcome Measurement
31The PROTECT Investigators N Engl J Med 2011;364:1305-14
Presenter’s Name
Date
Clear Definition of Outcome
32The PROTECT Investigators N Engl J Med 2011;364:1305-14
Presenter’s Name
Date
Descriptive characteristics of participants Key risk factors for outcome or variables that
define key subgroups Baseline value of outcome variable BE PARSIMONIOUS Establish a bank of biological materials
Baseline measurements in random-ized clinical trials
33
Presenter’s Name
Date
34The PROTECT Investigators N Engl J Med 2011;364:1305-14
Presenter’s Name
Date
35The PROTECT Investigators N Engl J Med 2011;364:1305-14
Presenter’s Name
Date
First step in testing a new treatment in hu-mans
Usually conducted in healthy volunteers Closely monitored Designed to determine:
Metabolic and pharmacologic effects in humans Side effects with increasing doses Pharmacokinetics, drug metabolism, and phar-
macological effects
Types of drug clinical trials: Phase I
36
Presenter’s Name
Date
Suntharalingam G, et al. N Engl J Med 2006;355:1018-2837
Presenter’s Name
Date
Early controlled clinical studies to obtain pre-liminary data on efficacy for a particular indi-cation
Conducted in a relatively small number of pa-tients
Well controlled, closely monitored Designed to determine:
Preliminary data on efficacy Short term data on side effects and risks
Types of drug clinical trials: Phase II
38
Presenter’s Name
Date
Performed after preliminary data on efficacy from phase II studies is obtained
Include several hundred to several thousand patients
Designed to determine: Efficacy Safety Information for extrapolating results to the gen-
eral population and for physician labeling
Types of drug clinical trials: Phase III
39
Presenter’s Name
Date
Evaluate the long term safety and efficacy of a drug
Usually after licensure granted by the FDA for the indication under study
May address: Different doses or schedules of administration Other patient populations or other stages of the
disease Use of the drug over a longer period of time
Types of drug clinical trials: Phase IV
40
Presenter’s Name
Date
RandomizationMasking (blinding)Intention to treat analysis
Key methodological tools in RCTs
41
Presenter’s Name
Date
Participants are assigned to one of two or more interventions using an explicit method that assures the assignment will be random, or by chance
“Similar” to flipping a coin
What is randomization?
42
Presenter’s Name
Date
Randomization
43
Treatment
Group
Control
Group
Treatment
Group
Treatment
Group
Control
Group
Control
Group
Presenter’s Name
Date
Protects against selection bias and con-founding
Results in groups similar on known and un-known prognostic factors (on average)
Provides a basis for standard statistical anal-ysis
Adds credibility to study findings
Randomization
44
Presenter’s Name
Date
Grady D, et al. JAMA 2002;288:49-5745
Presenter’s Name
Date
46
Presenter’s Name
Date
van Vollenhoven, et al. Lupus 1999;8:181-747
Presenter’s Name
Date
Sacks H, et al. Am J Med 1982;72:233-40
Historical vs. randomized controls
48
Presenter’s Name
Date
Table of random numbers Computer-generated list of treatment as-
signments Other methods designed to be random are
subject to bias (eg, birth date, medical record number, etc)
How is randomization done?
49
Presenter’s Name
Date
50
Presenter’s Name
Date
Inappropriate Randomization Meth-ods
Assigning patients alternately to treatment group is not random assignment
Assigning the first half of the population to one group is not random assignment
Assignments by methods based on patient characteristics such as date of birth, order of entry into the clinic or day of clinic attendance, are not reliably random
Presenter’s Name
Date
Random procedure is the same as a haphaz-ard procedure
Randomization ensures comparable study groups
Differences in baseline characteristics of in-terventions groups indicates a breakdown in randomization process
A study without randomization is invalid
Misconceptions about ran-domization
52
Presenter’s Name
Date
Blocks - Assignment ratio is enforced within each block. (eg, 1:1)
E.g.: EECC, ECEC, ECCE, CEEC, CECE, and CCEE
Randomly permuted blocks A block of 4 patients may be assigned to one of
EECC, ECEC, ECCE, CEEC, CECE, and CCEE with equal probabilities of 1/6 each.
Used to avoid serious imbalances in the numbers of participants assigned to each group.
Permuted Block Randomiza-tion
53
Presenter’s Name
Date
Attempts to assure comparability of groups on important prognostic vari-ables (where you cannot leave things up to chance)
Stratified randomization
54
Presenter’s Name
Date
55
Presenter’s Name
Date
Limit to variables believed to influence out-come
Small number of variables In multicenter trials, use center as a stratifica-
tion factor Adds to logistical complexities Larger number of strata leads to greater de-
partures from expected ratio Small block sizes can mitigate this problem but
lead to others
Stratification considerations
56
Presenter’s Name
Date
57The PROTECT Investigators N Engl J Med 2011;364:1305-14
Presenter’s Name
Date
One or more of the investigators, care providers, outcome assessors, or patients does not know the intervention the partici-pant is assigned to or receives
A study is “double masked” when the inves-tigator and the participant are unaware of the assignment
Masking or blinding
58
Presenter’s Name
Date
Issues leading to Blinding
Most investigators have firm views about which of a range of alternative treatments is more effective and of-ten, which is more appropriate for particular groups of patients.
a strong temptation by investigators to channel particular groups of patients to particular treatments (channeling effect )
A risk of the investigators subconsciously losing their objectivity in their assessments of treatment effects simply because of their clear preference for particular treatments
Risk of having other forms of bias, which can be satis-factorily controlled by proper blinding
Presenter’s Name
Date
Potential Biases due to not Blind-ing
Patient biasCare Provider biasAssessor biasLaboratory biasAnalysis and Interpretation bias
Presenter’s Name
Date
Patient Bias
the patient's knowledge that the patient is receiving a "new" treatment may substantially affect the pa-tient's subjective assessment
there is a subject x disease interaction in at least some diseases (and virtually all diseases)
thus, the patient's knowledge of the treatment being received may affect the outcome of the study
Presenter’s Name
Date
Care Provider Bias
the care provider's knowledge of which treatment a patient is receiving may affect the way the provider
– deals with the patient
– treats the patient
these differences may give the patient infor-mation (even if incorrect) about the treatment the patient is receiving, which then may affect the outcome of the study
Presenter’s Name
Date
Assessor Bias
the assessor's knowledge of which treatment the patient is receiving may affect the way the asses-sor assesses outcome
such a bias would directly affect the validity of the conclusions of the study
if the assessment is done while the patient is still receiving treatment, this may provide the patient with information about the treatment being re-ceived
Presenter’s Name
Date
Laboratory Bias
the knowledge of which treatment the patient re-ceived may affect the way in which the test is run or interpreted, or be retested.
although this is most severe with subjectively graded results (pathology slides, photographs, ECG, etc.), this can also be a problem with "objective tests" such as laboratory assays which may be run subtly differently by the technician.
Presenter’s Name
Date
Analysis and Interpretation bias
knowledge of the treatment group may affect the results of the analysis of the data by
seeking an explanation of an "anomalous” finding when one is found contrary to the study hypothesis
accepting a "positive" finding without fully exploring the data
knowledge of the treatment group may affect the deci-sions made by external monitors of a study by
terminating a study for adverse events because they fit the ex-pectations of the monitors
terminating a study for superiority of treatment because it fits the expectations of the monitors
Presenter’s Name
Date
66The PROTECT Investigators N Engl J Med 2011;364:1305-14
Presenter’s Name
Date
Summary odds ratios obtained for perinatal trials using randomization allocation schemes that were
adequate, unclear or inadequate
Ratio of Level of allocation odds ratiosconcealment (95% CI)
Adequate 1.00 (referent)Unclear 0.67 (0.60 – 0.75)Inadequate 0.59 (0.48 – 0.73) p<0.001
Inadequate allocation concealment
Schulz KF, et al. JAMA 1995;273:408-41267
Presenter’s Name
Date
68
Presenter’s Name
Date
Interpretations MDs Textbooks
Single blindingParticipants 75% 74%
Double blindingParticipants & providers 38% 43%Participants & investigators NA 21%Participants & outcome
assessors 5% 14%
Common MD definitions of blinding - survey
Devereaux PJ, et al. JAMA 2001;285:2000-3 69
Presenter’s Name
Date
70The PROTECT Investigators N Engl J Med 2011;364:1305-14
Presenter’s Name
Date
Once randomized always analyze Primary analysis include patients as part of
the group to which they were originally ran-domized (“Intention-to-treat” analysis)
Even if they did not get or take the treatment they were assigned
Even if they took the treatment assigned to the other group
Secondary analysis include patients in the treatment group corresponding to their actual treatment (“as treated” analysis)
Benefits of randomization are lost Study more like an observational study
“Intention-to-treat” analysis
71
Presenter’s Name
Date
72JAMA 2010, 303(15): 1483
Presenter’s Name
Date
Intention-to-treat analysis respects the ran-domized assignment
Intention-to-treat analysis introduces a bias to-wards the null
The reason patients do not comply with their assignment or use another intervention may be related to the outcome
E.g.: Sicker patients may stop taking an active drug; if counted as part of the “no treatment” group, would make “no treatment” look worse than active drug.
Why do an intention-to-treat analysis?
73
Presenter’s Name
Date
Coronary Drug Project. N Engl J Med 1980;303:1038-41
Coronary Drug Project: Effect of compliance on outcome
74
Presenter’s Name
Date
75
Presenter’s Name
Date
76The PROTECT Investigators N Engl J Med 2011;364:1305-14
Presenter’s Name
Date
Crossover design Factorial design Cluster randomization Non-inferiority trials Large simple trials
Design variations
77
Presenter’s Name
Date
Participants administered one intervention, then “crossed over” to receive a second, and perhaps subsequently crossed over again to receive a third and even a fourth.
Participant serves as his/her own control FDA (1977) recommends that in most cases
this design should not be used, due to fre-quent misapplications and misanalyses
Crossover trials
78
Presenter’s Name
Date
AB/BA crossover trial
Bendtsen L, et al. Neurology 2004;62:1706-1179
Presenter’s Name
Date
Crossover RCTs are appropriate for … Chronic conditions with “stable” characteristics Non-curative interventions (the condition reverts
to baseline levels without treatment) Interventions whose effects can be measured after
“short” course
Crossover RCTs are not appropriate for … Acute condition, self limited conditions (post-op
pain, reaction to a traumatic event, common cold)
Indications for crossover trials
80
Presenter’s Name
Date
Advantages Variability reduced because each participant is
used twice Fewer participants needed Evaluates each intervention in each participant
Disadvantages Period-treatment interaction: Effects in first period
may carry over into second period Each patient is followed up for a longer period of
time, increasing the risk of losses to follow-up
Advantages and disadvantages of crossover trials
81
Presenter’s Name
Date
Evaluate 2 interventions compared with con-trol in a single experiment
2 x 2 3 x 2 Incomplete or partial factorial (some empty cells)
Factorial trials
82
Presenter’s Name
Date
2 x 2 Factorial design
A+(active)
A-(control)
B+(active) A+B+ A-B+
B-(control) A+B- A-B-
Treatment
83
Presenter’s Name
Date
2x2 factorial RCT
http://www.icmaedu.com/img/img_profess_study.jpg84
Presenter’s Name
Date
Cost and effort reduced compared to 2 separate trials
Informative and efficient if little or no in-teraction, or interaction is important to understand
Factorial trials - Advantages
85
Presenter’s Name
Date
Possibility of interaction - impact of interac-tion on sample size
PHS: -carotene and aspirin may both affect CVD and cancer
WHI: diet and hormones may impact > 1 disease Data monitoring more complicated if one in-
tervention affects 2 outcomes (eg, HRT and breast cancer and CVD)
Problems with noncompliance, recruitment, complexity in general
Factorial trials – disadvantages
86
Presenter’s Name
Date
Randomization Scheme
PHYSICIANS' HEALTH STUDY
22,071
Randomized
11,037
Aspirin
11,037
Aspirin placebo
5,517
Beta-carotene
5,520
Beta-caroteneplacebo
5,519
Beta-carotene
5,515
Beta-caroteneplacebo
Presenter’s Name
Date
FACTT Design(Fluid and Catheter Treatment Trial)
88
N=1000 PAC CVC
Liberal A B 497
Restricted C D 503
513 487
Presenter’s Name
Date
89N Engl J Med 2006;354:2564-75
Presenter’s Name
Date
90N Engl J Med 2006;354:2213-24.
Presenter’s Name
Date
Cluster Randomization
Randomization is done not by individual but larger groups
Hospital ICU Regions Country
91
Presenter’s Name
Date
Lancet 2005;365:2091-97
Presenter’s Name
Date
93N Engl J Med 2009;361:335-44
Presenter’s Name
Date
94N Engl J Med 2009;361:335-44
Presenter’s Name
Date
Designed to show that test intervention is “equivalent” to comparison
Special sample size issues Comparison usually is standard intervention
Equivalence (noninferiority) trials
95
Presenter’s Name
Date
Mayer SA, et al. N Engl J Med 2008;358:2127-3796
Presenter’s Name
Date
Large Simple Trials(Pragmatic Trials)
Features Very large number of patients Broad eligibility criteria Minimal data collection Easily administered intervention
Rationale Modest benefit require large sample size Treatment interactions unlikely so baseline charac-
teristics and interim response are not needed Less precision tolerated
97
Presenter’s Name
Date
Efficacy vs Effectiveness
Efficacy extent to which an intervention (technology,
treatment, procedure, service, or program) produces a beneficial result under ideal condi-tions.
Effectiveness whether the interventions are effective in “real-
world” conditions or “natural” settings
98
Presenter’s Name
Date
99NEJM 2005;353:1209
Presenter’s Name
Date
Impossible to predict outcome for an individ-ual from population-based studies
BUT Is this individual so different, Are the available interventions so different, Are the possible outcomes so different,
… that the evidence in a given high quality RCT can be dismissed as irrelevant?
External validity
100
Presenter’s Name
Date
Use of informed consent Children, proxy
Data and safety monitoring Trials in special populations
Ethical issues
101
Presenter’s Name
Date
102
Presenter’s Name
Date
103http://www.consort-statement.org
Presenter’s Name
Date
Failure to publish Ethical and scientific responsibility
Selective outcome reporting Conflict of interest Trial registration
Other ongoing challenges in RCT research
104
Presenter’s Name
Date
Combination chemotherapy vs.monotherapy with alkilating agentsin advanced ovarian cancer
Simes RJ. Stat Med 1987;6:11-29
Presenter’s Name
Date
Time to publication of protocolssubmitted to the Royal Prince AlbertEthics Committee (Sidney, Australia)
Stern JM, Simes RJ. BMJ 1997;315:640-645
Presenter’s Name
Date
Vickers A, et al. Control Clin Trials 1998;19:159-166
Presenter’s Name
Date
If you torture your data long enough, they will tell you whatever you want to hear.
James L. Mills
Presenter’s Name
Date
Subgroup analysis
When enough comparisons are made some comparisons will be statistically significant even if there are differences between groups just by chance.
Pre-defined subgroup analyses deter-mined by biologic plausibility will be in-formative
Should be interpretated with caution Hypothesis generating
Presenter’s Name
Date
0 60
25%
50%
75%
100%
30Number of Comparisons
1 or more comparisons as significant
Presenter’s Name
Date
ISIS-2
17,187 patients with suspected AMI Placebo, SK, aspirin, SK+aspirin
Mortality significantly better in combina-tion therapy group
When subgroup analysis according to 12 signs of Zodiac
Overall benefit of aspirin (p<0.00001) For Gemini and Libra
• Increased mortality 9 13%
Presenter’s Name
Date
http://www.who.int/ictrp/en/112
Presenter’s Name
Date
113http://clinicaltrials.gov
Presenter’s Name
Date
Learning objectives
To review the key aspects of designing RCTs To review the design methods used to protect
against selection bias, confounding and information bias used in RCTs Randomization Blinding
To understand the basic analytical methods to protect against selection bias in RCTs Intention-to-treat analysis
114