Interim Monitoring inInterim Monitoring inRandomized TrialsRandomized Trials

Interim Monitoring inInterim Monitoring inRandomized TrialsRandomized Trials

• WhyWhy alter/stop a clinical trial early? alter/stop a clinical trial early?

• WhoWho should decide? should decide?

• WhatWhat should be monitored? should be monitored?

• How oftenHow often should you monitor? should you monitor?

• What What statistical methodsstatistical methods to use? to use?

• Fascinating examples...Fascinating examples...

• WhyWhy alter/stop a clinical trial early? alter/stop a clinical trial early?

• WhoWho should decide? should decide?

• WhatWhat should be monitored? should be monitored?

• How oftenHow often should you monitor? should you monitor?

• What What statistical methodsstatistical methods to use? to use?

• Fascinating examples...Fascinating examples...

A Little HistoryA Little HistoryA Little HistoryA Little History

• NIH task force on administration of NIH task force on administration of multicenter trials (Greenberg Report, 1966)multicenter trials (Greenberg Report, 1966)– Funding AgencyFunding Agency– Trial ChairTrial Chair– Steering or Executive CommitteesSteering or Executive Committees– Statistical or Coordinating CenterStatistical or Coordinating Center– Policy Advisory BoardPolicy Advisory Board• Data Safety and Monitoring Board, orData Safety and Monitoring Board, or• Data Monitoring CommitteeData Monitoring Committee

• NIH task force on administration of NIH task force on administration of multicenter trials (Greenberg Report, 1966)multicenter trials (Greenberg Report, 1966)– Funding AgencyFunding Agency– Trial ChairTrial Chair– Steering or Executive CommitteesSteering or Executive Committees– Statistical or Coordinating CenterStatistical or Coordinating Center– Policy Advisory BoardPolicy Advisory Board• Data Safety and Monitoring Board, orData Safety and Monitoring Board, or• Data Monitoring CommitteeData Monitoring Committee

Why Stop a Trial Early?Why Stop a Trial Early?Why Stop a Trial Early?Why Stop a Trial Early?

• Benefit clearly demonstratedBenefit clearly demonstrated

• Harm clearly demonstratedHarm clearly demonstrated

• Not possible to demonstrate benefitNot possible to demonstrate benefit– trial design flawedtrial design flawed– low enrollment, high noncompliance, low enrollment, high noncompliance,

poor data, high drop-outpoor data, high drop-out– no difference between groupsno difference between groups

• Research question answered by Research question answered by another studyanother study

• Benefit clearly demonstratedBenefit clearly demonstrated

• Harm clearly demonstratedHarm clearly demonstrated

• Not possible to demonstrate benefitNot possible to demonstrate benefit– trial design flawedtrial design flawed– low enrollment, high noncompliance, low enrollment, high noncompliance,

poor data, high drop-outpoor data, high drop-out– no difference between groupsno difference between groups

• Research question answered by Research question answered by another studyanother study

Which Trials Should be Monitored?Which Trials Should be Monitored?Which Trials Should be Monitored?Which Trials Should be Monitored?

• Most phase III trials, unlessMost phase III trials, unless– no possibility of harmno possibility of harm– duration too short to be practicalduration too short to be practical

• Most phase III trials, unlessMost phase III trials, unless– no possibility of harmno possibility of harm– duration too short to be practicalduration too short to be practical

Who Should Decide?Who Should Decide?Who Should Decide?Who Should Decide?

• InvestigatorsInvestigators

• SponsorSponsor

• IndependentIndependent monitoring board, without monitoring board, without conflict of interestconflict of interest– expertsexperts -- investigatorsinvestigators– ethicistsethicists -- ? representative of sponsor ? representative of sponsor– statisticiansstatisticians -- ? lay persons? lay persons

• InvestigatorsInvestigators

• SponsorSponsor

• IndependentIndependent monitoring board, without monitoring board, without conflict of interestconflict of interest– expertsexperts -- investigatorsinvestigators– ethicistsethicists -- ? representative of sponsor ? representative of sponsor– statisticiansstatisticians -- ? lay persons? lay persons

What Should You Monitor?What Should You Monitor?What Should You Monitor?What Should You Monitor?

• Issues early in the trialIssues early in the trial– trial designtrial design– recruitmentrecruitment– compliancecompliance– loss to follow-uploss to follow-up– data quality and timelinessdata quality and timeliness– information for other studiesinformation for other studies

• Issues early in the trialIssues early in the trial– trial designtrial design– recruitmentrecruitment– compliancecompliance– loss to follow-uploss to follow-up– data quality and timelinessdata quality and timeliness– information for other studiesinformation for other studies

What Should You Monitor?What Should You Monitor?What Should You Monitor?What Should You Monitor?

• Issues later in the trialIssues later in the trial– primary and secondary outcomesprimary and secondary outcomes• clear benefit or harmclear benefit or harm• no conditional powerno conditional power

– adverse eventsadverse events– side effectsside effects– subgroupssubgroups

• Issues later in the trialIssues later in the trial– primary and secondary outcomesprimary and secondary outcomes• clear benefit or harmclear benefit or harm• no conditional powerno conditional power

– adverse eventsadverse events– side effectsside effects– subgroupssubgroups

How Can a Trial Be Altered?How Can a Trial Be Altered?How Can a Trial Be Altered?How Can a Trial Be Altered?

• Early alterationsEarly alterations– change entry criteria change entry criteria – increase sample size or prolong durationincrease sample size or prolong duration– change definition of outcome (RUTH)change definition of outcome (RUTH)– dose adjustment (TIMI II)dose adjustment (TIMI II)

• Goals of these changesGoals of these changes– result in a successful trial (definitive result)result in a successful trial (definitive result)– change original protocol as little as possiblechange original protocol as little as possible

• Timely, high quality data crucialTimely, high quality data crucial

• Early alterationsEarly alterations– change entry criteria change entry criteria – increase sample size or prolong durationincrease sample size or prolong duration– change definition of outcome (RUTH)change definition of outcome (RUTH)– dose adjustment (TIMI II)dose adjustment (TIMI II)

• Goals of these changesGoals of these changes– result in a successful trial (definitive result)result in a successful trial (definitive result)– change original protocol as little as possiblechange original protocol as little as possible

• Timely, high quality data crucialTimely, high quality data crucial

How Can a Trial Be Altered?How Can a Trial Be Altered?How Can a Trial Be Altered?How Can a Trial Be Altered?

• Later alterationsLater alterations– increase duration of the trialincrease duration of the trial– modify the trialmodify the trial• stop one arm of the interventionstop one arm of the intervention• terminate high risk groupsterminate high risk groups• add safety measuresadd safety measures

– terminate the trial earlyterminate the trial early

• Later alterationsLater alterations– increase duration of the trialincrease duration of the trial– modify the trialmodify the trial• stop one arm of the interventionstop one arm of the intervention• terminate high risk groupsterminate high risk groups• add safety measuresadd safety measures

– terminate the trial earlyterminate the trial early

• Often enough to achieve goalsOften enough to achieve goals

• Not so often that there is no new dataNot so often that there is no new data

• Typically every 6-12 months or when Typically every 6-12 months or when an additional 20% of the expected an additional 20% of the expected outcomes have occurredoutcomes have occurred

• Often enough to achieve goalsOften enough to achieve goals

• Not so often that there is no new dataNot so often that there is no new data

• Typically every 6-12 months or when Typically every 6-12 months or when an additional 20% of the expected an additional 20% of the expected outcomes have occurredoutcomes have occurred

How Often to Monitor?How Often to Monitor?How Often to Monitor?How Often to Monitor?

• DSMB usually does not share interim DSMB usually does not share interim results with sponsor, investigatorsresults with sponsor, investigators

• Open sessionOpen session– DSMB, sponsor, investigators, FDADSMB, sponsor, investigators, FDA– recruitment, retention, data qualityrecruitment, retention, data quality– overall findingsoverall findings

• Closed sessionClosed session– DSMB members onlyDSMB members only– between group findingsbetween group findings– discussion regarding modificationsdiscussion regarding modifications

• DSMB usually does not share interim DSMB usually does not share interim results with sponsor, investigatorsresults with sponsor, investigators

• Open sessionOpen session– DSMB, sponsor, investigators, FDADSMB, sponsor, investigators, FDA– recruitment, retention, data qualityrecruitment, retention, data quality– overall findingsoverall findings

• Closed sessionClosed session– DSMB members onlyDSMB members only– between group findingsbetween group findings– discussion regarding modificationsdiscussion regarding modifications

ConfidentialityConfidentialityConfidentialityConfidentiality

Statistical MethodsStatistical MethodsStatistical MethodsStatistical Methods

• Perform tests of significance and Perform tests of significance and

stop the trial if any p<.05stop the trial if any p<.05

• Simple, but wrongSimple, but wrong

total teststotal tests overall alphaoverall alpha

11 .05.05

22 .08.08

55 .14.14

1010 .20.20

2020 .35.35

• Perform tests of significance and Perform tests of significance and

stop the trial if any p<.05stop the trial if any p<.05

• Simple, but wrongSimple, but wrong

total teststotal tests overall alphaoverall alpha

11 .05.05

22 .08.08

55 .14.14

1010 .20.20

2020 .35.35

Interim Analyses in the CDPInterim Analyses in the CDPInterim Analyses in the CDPInterim Analyses in the CDP

Z ValueZ ValueZ ValueZ Value

+2+2

+1+1

00

-1-1

-2-2

+2+2

+1+1

00

-1-1

-2-2

10 20 30 40 50 60 70 80 90 10010 20 30 40 50 60 70 80 90 100

Month of Follow-upMonth of Follow-up

Statistical MethodsStatistical MethodsStatistical MethodsStatistical Methods

• Perform tests of significance and Perform tests of significance and adjust the test-wise alphaadjust the test-wise alpha

• Multiple approachesMultiple approaches– BonferroniBonferroni– Classical sequential methodsClassical sequential methods– Group sequential methodsGroup sequential methods• PocockPocock• Haybittle and PetoHaybittle and Peto• O’Brien and FlemmingO’Brien and Flemming• Lan and DeMetsLan and DeMets

• Perform tests of significance and Perform tests of significance and adjust the test-wise alphaadjust the test-wise alpha

• Multiple approachesMultiple approaches– BonferroniBonferroni– Classical sequential methodsClassical sequential methods– Group sequential methodsGroup sequential methods• PocockPocock• Haybittle and PetoHaybittle and Peto• O’Brien and FlemmingO’Brien and Flemming• Lan and DeMetsLan and DeMets

Group Sequential MethodsGroup Sequential MethodsGroup Sequential MethodsGroup Sequential Methods

• O’BrienO’Brien -- small small ii for early tests for early tests

FlemmingFlemming gradually increasing gradually increasing ii

N=5 interim tests; N=5 interim tests; = .05 = .05

initial initial ii=.00001; =.00001; ff=.046=.046

• Lan-Lan- spending function spending function

DeMetsDeMets defined by N previous “looks”defined by N previous “looks”

proportion of data/time proportion of data/time betweenbetween

• O’BrienO’Brien -- small small ii for early tests for early tests

FlemmingFlemming gradually increasing gradually increasing ii

N=5 interim tests; N=5 interim tests; = .05 = .05

initial initial ii=.00001; =.00001; ff=.046=.046

• Lan-Lan- spending function spending function

DeMetsDeMets defined by N previous “looks”defined by N previous “looks”

proportion of data/time proportion of data/time betweenbetween

Stopping BoundariesStopping Boundaries

54.5

4.23.5

2

-5-4.5

-4.2-3.5

-2

-6

-4

-2

0

2

4

6

1st Look 2nd Look 3rd Look 4th Look 5th Look

Stop for HarmStop for Benefit

ZZ

Curtailed SamplingCurtailed SamplingCurtailed SamplingCurtailed Sampling

Compute p(reject HCompute p(reject Hoo given data so far) given data so far)Deterministic Curtailed SamplingDeterministic Curtailed Sampling• assume all future outcomes in treatedassume all future outcomes in treated• assume all future outcomes in placeboassume all future outcomes in placebo

Stochastic Curtailed SamplingStochastic Curtailed Sampling

• assume Hassume Hoo true true

• assume Hassume Ha a truetrue

Compute p(reject HCompute p(reject Hoo given data so far) given data so far)Deterministic Curtailed SamplingDeterministic Curtailed Sampling• assume all future outcomes in treatedassume all future outcomes in treated• assume all future outcomes in placeboassume all future outcomes in placebo

Stochastic Curtailed SamplingStochastic Curtailed Sampling

• assume Hassume Hoo true true

• assume Hassume Ha a truetrue

Interim MonitoringInterim MonitoringInterim MonitoringInterim Monitoring

• NOT NOT simply a statistical issuesimply a statistical issue

• Must weigh:Must weigh:– Possible baseline differences in groupsPossible baseline differences in groups– Possible bias in assessment of outcomePossible bias in assessment of outcome– Impact of missing dataImpact of missing data– Differential co-intervention or Differential co-intervention or

noncompliancenoncompliance– Internal consistency of findingsInternal consistency of findings– Impact of early termination on medical Impact of early termination on medical

practice and public healthpractice and public health

• NOT NOT simply a statistical issuesimply a statistical issue

• Must weigh:Must weigh:– Possible baseline differences in groupsPossible baseline differences in groups– Possible bias in assessment of outcomePossible bias in assessment of outcome– Impact of missing dataImpact of missing data– Differential co-intervention or Differential co-intervention or

noncompliancenoncompliance– Internal consistency of findingsInternal consistency of findings– Impact of early termination on medical Impact of early termination on medical

practice and public healthpractice and public health

Nuts and BoltsNuts and BoltsNuts and BoltsNuts and Bolts

• Board chosen earlyBoard chosen early

• Data Monitoring PlanData Monitoring Plan– board membersboard members– variables and analysesvariables and analyses– frequency of monitoringfrequency of monitoring– statistical methodsstatistical methods– guidelines for decisionsguidelines for decisions

• Timely, accurate and complete dataTimely, accurate and complete data

• Board chosen earlyBoard chosen early

• Data Monitoring PlanData Monitoring Plan– board membersboard members– variables and analysesvariables and analyses– frequency of monitoringfrequency of monitoring– statistical methodsstatistical methods– guidelines for decisionsguidelines for decisions

• Timely, accurate and complete dataTimely, accurate and complete data

Beta-blocker Heart Attack TrialBeta-blocker Heart Attack TrialBeta-blocker Heart Attack TrialBeta-blocker Heart Attack Trial

• Subjects - 3,837 persons 5-21 days Subjects - 3,837 persons 5-21 days after MIafter MI

• Intervention - propranolol 180-Intervention - propranolol 180-240mg/day vs placebo240mg/day vs placebo

• Follow-up - 25 of planned 34 monthsFollow-up - 25 of planned 34 months

• Outcome - mortalityOutcome - mortality

• Subjects - 3,837 persons 5-21 days Subjects - 3,837 persons 5-21 days after MIafter MI

• Intervention - propranolol 180-Intervention - propranolol 180-240mg/day vs placebo240mg/day vs placebo

• Follow-up - 25 of planned 34 monthsFollow-up - 25 of planned 34 months

• Outcome - mortalityOutcome - mortality

AnalysesAnalyses MonthMonth DeathsDeaths ZZ Critical Critical ValueValue

11 1111 5656 1.681.68 5.885.88

22 1616 7777 2.242.24 5.045.04

33 2121 126126 2.372.37 3.793.79

44 2828 177177 2.302.30 3.193.19

55 3434 247247 2.342.34 2.642.64

66 4040 318318 2.822.82 2.302.30

77 4848

AnalysesAnalyses MonthMonth DeathsDeaths ZZ Critical Critical ValueValue

11 1111 5656 1.681.68 5.885.88

22 1616 7777 2.242.24 5.045.04

33 2121 126126 2.372.37 3.793.79

44 2828 177177 2.302.30 3.193.19

55 3434 247247 2.342.34 2.642.64

66 4040 318318 2.822.82 2.302.30

77 4848

Beta-blocker Heart Attack TrialBeta-blocker Heart Attack TrialBeta-blocker Heart Attack TrialBeta-blocker Heart Attack Trial

Coronary Drug ProjectCoronary Drug ProjectCoronary Drug ProjectCoronary Drug Project

• Subjects - 8,341 men post-MISubjects - 8,341 men post-MI• Interventions -Interventions - estrogen 2.5 and 5.0 mg QDestrogen 2.5 and 5.0 mg QD

dextrothyroxine 6 mg QDdextrothyroxine 6 mg QD

clofibrate 1.8 gm QDclofibrate 1.8 gm QD

niacin 3.0 gm QDniacin 3.0 gm QD

placeboplacebo

• Follow-up - 1.5 to 2.5 of planned 5 yearsFollow-up - 1.5 to 2.5 of planned 5 years

• Outcomes - death, MI, cancer, VTEOutcomes - death, MI, cancer, VTE

• Subjects - 8,341 men post-MISubjects - 8,341 men post-MI• Interventions -Interventions - estrogen 2.5 and 5.0 mg QDestrogen 2.5 and 5.0 mg QD

dextrothyroxine 6 mg QDdextrothyroxine 6 mg QD

clofibrate 1.8 gm QDclofibrate 1.8 gm QD

niacin 3.0 gm QDniacin 3.0 gm QD

placeboplacebo

• Follow-up - 1.5 to 2.5 of planned 5 yearsFollow-up - 1.5 to 2.5 of planned 5 years

• Outcomes - death, MI, cancer, VTEOutcomes - death, MI, cancer, VTE

Coronary Drug ProjectCoronary Drug ProjectCoronary Drug ProjectCoronary Drug Project

CEE 5 mgCEE 5 mg PlaceboPlacebo RRRR

( n=1,119)( n=1,119) (n=2,789)(n=2,789)

CHD eventCHD event 11.0%11.0% 7.5%7.5% 1.51.5

PE or DVTPE or DVT 3.5% 3.5% 1.5%1.5% 2.3*2.3*

Total mortalityTotal mortality 9.7% 9.7% 8.2%8.2% 1.21.2

JAMA, 1970JAMA, 1970

CEE 5 mgCEE 5 mg PlaceboPlacebo RRRR

( n=1,119)( n=1,119) (n=2,789)(n=2,789)

CHD eventCHD event 11.0%11.0% 7.5%7.5% 1.51.5

PE or DVTPE or DVT 3.5% 3.5% 1.5%1.5% 2.3*2.3*

Total mortalityTotal mortality 9.7% 9.7% 8.2%8.2% 1.21.2

JAMA, 1970JAMA, 1970

Physicians’ Aspirin StudyPhysicians’ Aspirin StudyPhysicians’ Aspirin StudyPhysicians’ Aspirin Study

• Subjects - Subjects - 22,071 physicians22,071 physicians

• Interventions - aspirin 325 mg QODInterventions - aspirin 325 mg QOD

beta-carotene 50 mg QODbeta-carotene 50 mg QOD

• Follow-up - Follow-up - 5 of planned 7 years5 of planned 7 years

• Outcomes - Outcomes - main = CVD deathmain = CVD death

secondary = MI, strokesecondary = MI, stroke

• Subjects - Subjects - 22,071 physicians22,071 physicians

• Interventions - aspirin 325 mg QODInterventions - aspirin 325 mg QOD

beta-carotene 50 mg QODbeta-carotene 50 mg QOD

• Follow-up - Follow-up - 5 of planned 7 years5 of planned 7 years

• Outcomes - Outcomes - main = CVD deathmain = CVD death

secondary = MI, strokesecondary = MI, stroke

Physicians’ Aspirin StudyPhysicians’ Aspirin StudyPhysicians’ Aspirin StudyPhysicians’ Aspirin Study

OutcomeOutcome AspirinAspirin PlaceboPlacebo RRRR p valuep value

CVD deathCVD death 81 81 83 83 1.01.0 .87.87

MIMI 139139 239239 0.60.6 .00001.00001

StrokeStroke 119 119 98 98 1.21.2 .15.15

ischemicischemic 9191 8282 1.11.1 .50.50

hemorrhhemorrh 2323 1212 2.12.1 .06.06

OutcomeOutcome AspirinAspirin PlaceboPlacebo RRRR p valuep value

CVD deathCVD death 81 81 83 83 1.01.0 .87.87

MIMI 139139 239239 0.60.6 .00001.00001

StrokeStroke 119 119 98 98 1.21.2 .15.15

ischemicischemic 9191 8282 1.11.1 .50.50

hemorrhhemorrh 2323 1212 2.12.1 .06.06

Coronary Arrhythmia Coronary Arrhythmia Suppression TrialSuppression Trial

Coronary Arrhythmia Coronary Arrhythmia Suppression TrialSuppression Trial

• 1727 of planned 4400 subjects1727 of planned 4400 subjects after MIafter MI

with ventricular ectopywith ventricular ectopy

• Flecainide, encainide or moricizine vs. pboFlecainide, encainide or moricizine vs. pbo

• Mean follow-up Mean follow-up 1 year of1 year of planned 5 years planned 5 years

• Outcomes - mortality from arrhythmia, Outcomes - mortality from arrhythmia,

total mortalitytotal mortality

• 1727 of planned 4400 subjects1727 of planned 4400 subjects after MIafter MI

with ventricular ectopywith ventricular ectopy

• Flecainide, encainide or moricizine vs. pboFlecainide, encainide or moricizine vs. pbo

• Mean follow-up Mean follow-up 1 year of1 year of planned 5 years planned 5 years

• Outcomes - mortality from arrhythmia, Outcomes - mortality from arrhythmia,

total mortalitytotal mortality

Coronary Arrhythmia Coronary Arrhythmia Suppression TrialSuppression Trial

Coronary Arrhythmia Coronary Arrhythmia Suppression TrialSuppression Trial

OutcomeOutcome F/EF/E PlaceboPlacebo pp

N randomizedN randomized 730730 725725

Arrhythmic deathArrhythmic death 3333 9 9 .0006.0006

Total deathTotal death 5656 22 22 .0003.0003

OutcomeOutcome F/EF/E PlaceboPlacebo pp

N randomizedN randomized 730730 725725

Arrhythmic deathArrhythmic death 3333 9 9 .0006.0006

Total deathTotal death 5656 22 22 .0003.0003

Canadian Atrial Fibrillation StudyCanadian Atrial Fibrillation StudyCanadian Atrial Fibrillation StudyCanadian Atrial Fibrillation Study

• 383 of planned 660 subjects with AF383 of planned 660 subjects with AF

• randomized to warfarin or placeborandomized to warfarin or placebo

• follow-up 1.2 of planned 3.5 yearsfollow-up 1.2 of planned 3.5 years

• results of two other large trials availableresults of two other large trials available

• 383 of planned 660 subjects with AF383 of planned 660 subjects with AF

• randomized to warfarin or placeborandomized to warfarin or placebo

• follow-up 1.2 of planned 3.5 yearsfollow-up 1.2 of planned 3.5 years

• results of two other large trials availableresults of two other large trials available

Findings of Other TrialsFindings of Other TrialsFindings of Other TrialsFindings of Other Trials

STROKE RATESTROKE RATE

WarfarinWarfarin PlaceboPlacebo

AFASK TrialAFASK Trial 2.0%2.0% 5.5%5.5%

SPAF TrialSPAF Trial 1.6%1.6% 8.3%8.3%

STROKE RATESTROKE RATE

WarfarinWarfarin PlaceboPlacebo

AFASK TrialAFASK Trial 2.0%2.0% 5.5%5.5%

SPAF TrialSPAF Trial 1.6%1.6% 8.3%8.3%

HERS DSMB ReportHERS DSMB ReportMonitoring for VTEsMonitoring for VTEs

-6

-4

-2

0

2

4

6

6 mo 1.5 yr 2.5yr 3.5 yr End

Stop for HarmStop for Benefit

ZZ

.. . . . . .

..

. .

HERS DSMB ReportHERS DSMB ReportMonitoring for CHD DeathMonitoring for CHD Death

-6

-4

-2

0

2

4

6

6 mo 1.5 yr 2.5yr 3.5 yr End

Stop for HarmStop for Benefit

ZZZZ

. .. . .

. .... .

SummarySummarySummarySummary

• Interim monitoring very importantInterim monitoring very important

• Should be planned in advanceShould be planned in advance

• Should be performed wellShould be performed well

• Any change in trial protocol should Any change in trial protocol should be carefully considered, weighing be carefully considered, weighing many issuesmany issues

• Interim monitoring very importantInterim monitoring very important

• Should be planned in advanceShould be planned in advance

• Should be performed wellShould be performed well

• Any change in trial protocol should Any change in trial protocol should be carefully considered, weighing be carefully considered, weighing many issuesmany issues