Topics in Clinical Trials (8) - 2012

J. Jack Lee, Ph.D.Department of BiostatisticsUniversity of Texas M. D. Anderson Cancer Center

Monitoring Response Variables

Monitor toxicity: ethical considerations, protect the safety of participantsMonitor the primary endpoint

Shows intervention is clearly better Early stopping due to efficacy

Shows intervention is harmful Early stopping due to adverse effect

Shows intervention is unlikely to be beneficial Early stopping due to futility

No clear indication one way or the other Continue the trial as planned Adjust the sample size based on observed effect, etc.

Data monitoring committee (DMC) / Data Safety and Monitoring Board (DSMB)

Fundamental Point

During the trial, response variables need to be monitored for early dramatic benefits or potential harmful effects.Preferably, monitoring should be done by a group of independent investigators.Although many techniques are available to assist in monitoring, none of them should be used as the sole basis for the decision to stop or continue the trial.

Data Monitoring Committee (DMC)

Composition Why not include the study investigator?

Conflict of interest, lack of objectivity Knowing the interim result may affect the clinical

equipoise and the subsequent trial conduct Credibility

Independent group of experts in the field including clinicians, statisticians, patient advocates, etc.

Institutional DMC or a special DMC assembled for the trial

Responsibility of DMC

Ensure the safety of participantsEnsure the integrity of the trial monitor the accrual of the trial Examine the randomization process Evaluate the compliance status

Make decision to continue or terminate the trial based on all data and informationOversee the trial on behalf of the sponsorProvide a service to the regulatory agency such as NCI or FDA

Operation of DMC

Meeting schedule Before or in the early phase of the trial Correspond to the time of pre-defined interim

analyses End of study

Format and Confidentiality Open session

All invited: accrual, logistic, data quality, adherence, toxicity, etc.

Closed session DMC + study statistician DMC only

Executive session DMC + study PI

Decisions During the Trial

Planned Interim Analysis Group sequential design

Unplanned Interim Analysis Group sequential design – alpha spending function Conditional Power Stochastic Curtailment Predictive Power / Predictive Probability

Interim Decisions Stop the trial due to efficacy

superiority, one way or another Stop the trial due to futility

Lack of efficacy, no difference between groups Continue the trial as planned Sample size re-estimation

Conditional Power ApproachExample 1: Compare 2 Antibiotics

Goal: compare the response rate of pipracillin (tx1) vs. clindamycin (tx2)

Design: RCT with 47 pts/arm. 80% power to detect an absolute difference of 25% (85% vs. 60%) with 1-sided a= 5% (2-sided a= 10%)Interim result:

ResponseNo Yes

Tx 1 4 22 26

Tx 2 3 18 21

7 40 47

Question: Can the trial be stopped early?

Estimation

The 95% CI of P1 – P2 covers 0.25. If the goal of this trial is to detect a 25% difference in response rate, cannot stop the trial due to “no difference.”CI calculation does not reflect the nature of interim analysis. If use “repeated CI” (e.g. Jennison & Turnbull), the CI will be wider.

1

2

1 2

ˆ 22 / 26 0.846

ˆ 18 / 21 0.857

ˆ ˆ 0.011, 95% exact CI = (-0.33, 0.27)

P

P

P P

Conditional Power by B-ValueLan & Wittes, Biometrics 44:579-585, 1988.

1 2

0 a

1

Let , ,..., ( , 1)

For testing H : 0 versus H : 0,

Let / be the test stat. with pts

/ be the information time,

( ) and ( )

Key equation : (1) ( ) [ (1) ( )]

1. ( ) an

N

n

n ii

n N

X X X N

Z X n n

t n N

B t Z t E Z N

B B t B B t

B t

d (1) ( ) are indep. normally distributed.

2. [ ( )] , [ (1) ( )] (1 )

3. [ ( )] , [ (1) ( )] (1 )

B B t

E B t t E B B t t

Var B t t Var B B t t

/ 2 / 2

The conditional power (1) | ( ), is

( ) Pr( (1) | ( ), ) Pr( ( ( ) (1 ),1 ) )

( ) (1 ) 1 ( )

1For two-sided test

( ) (1 ) ( ) (1 )( ) 1 ( ) ( )

At e

1

t

1

h

p

p

B Z B t

C B Z B t N B t t t Z

Z B t t

t

Z B t t Z B t tC

t t

1 2

47

interim, we have

(.846)(.154) (.857)(.143)~ (.846, ), ~ (.857, ),

26 21.011

.106(.846)(.154) (.857)(.143)

26 21( .106) 0.006, ( 0) 0.008

.85 .60( 2.83 or 2.8

(.85)(.15) (.60)(.40)

47 47

p p

p N

P N P N

Z

C C

C Z

3) 0.34

Stochastic CurtailmentLan, Simon, Halperin: 82 Comm in Stat.-Seq:1:207-

219Davis, Hardy: 94, J. Clin. Epi 47:1033-1042

One might stop the trial early and Reject Ho if cond. Prob P(Z(1) R| Z(t), Ho) ≥ g Accept Ho if cond. Prob P(Z(1) A| Z(t), H1) ≥ g’

Early termination can be due to efficacy or futility Overall type I error rate ≤ /a g Overall type II error rate ≤ /b g’ With a small number of looks, the error rates will be even

less

Boundaries of stochastic curtailment

1

1 θ(1 )( ) ,

Under : θ 0, Under : θo

Z Z t tZ t

tH H Z Z

Stochastic Curtailment and Conditional Power

Under H1: = 1.960 + 1.282 = 3.242, cannot stop before t=0.64

0.0 0.2 0.4 0.6 0.8 1.0

-10

-8

-6

-4

-2

0

2

4

6

8

10

0.0 0.2 0.4 0.6 0.8 1.0

-10

-8

-6

-4

-2

0

2

4

6

8

10

Information Time

Z V

alu

eStopping boundary for 2-sided alpha= 0.05 power= 0.9 Condition Power= 0.9

Stop, Conclude H1

Stop, Conclude H1

Stop, Conclude Ho

Early Stopping Using Predictive Power

Similar to the conditional power approach but integrate over the possible values of the conditioning parameter

Predictive Power = ( ) ( | )p nC f X d

Early Stopping Using Predictive Probability (PP)

At interim, compute the predictive probability, i.e., a positive result at the end of trial if the current trend continues given the current data. If PP is very high, stop the trial now

and declare treatment is working, If PP is very low, stop the trial now and

declare treatment is not working, Otherwise, continue.

Frequentist Approach

Bayesian Approach

==

Example 2: Predictive Probability in Breast Cancer

Buzdar AU et al, JCO 23, 3676-3685, 2005

Disease: HER-2 (+) Breast Cancer

Agent: A: 4 cycles of paclitaxel + 4 cycles of fluorouracil, epirubicin, and

cyclophosphamide B: same as A + weekly trastuzumab for 24

weeks

Statistical Design: phase II

Sample Size: 164 (maximum)

Primary Endpoint: pathological complete response (pCR)

Method: Standard two-stage design with Predictive Probability Interim Monitoring by

DSMB Results: After 34 pts completed therapy, DSMB stopped the trial.

With 34 Patients

Arm CR rates

A 4 / 16 (25%)

B 12/ 18 (67%)

With 42 Patients

Arm CR rates

A 5 / 19 (26%)

B 15/ 23 (65%) P=0.016

Review of Randomized Phase II TrialsLee and Feng, JCO 23, 4450-4457, 2005

56%

3%

Comparing Fixed vs. Random P

Assume we observed x/n successes.What is the probability of observing X/N successes in the future? Assuming the estimated P is

fixed random

What are the differences?Which one do you prefer?

0 1 2 3 4 5 6 7 8 9 10

0.0

0.0

50

.10

0.1

50

.20

0.2

50

.30

Observed x/n = 5/10, Fixed P

X in future N=10

Pro

ba

bili

ty

0 1 2 3 4 5 6 7 8 9 10

0.0

0.0

50

.10

0.1

50

.20

0.2

50

.30

Observed x/n = 5/10, Prior=Beta(.5, .5)

X in future N=10

Pro

ba

bili

ty

0 1 2 3 4 5 6 7 8 9 10

0.0

0.0

50

.10

0.1

50

.20

0.2

50

.30

Observed x/n = 5/10, Prior=Beta(.5, .5), Fixed P (blue), Random P (red)

X in future N=10

Pro

ba

bili

ty

0 1 2 3 4 5 6 7 8 9 10

0.0

0.0

50

.10

0.1

50

.20

0.2

50

.30

Observed x/n = 8/10, Fixed P

X in future N=10

Pro

ba

bili

ty

0 1 2 3 4 5 6 7 8 9 10

0.0

0.0

50

.10

0.1

50

.20

0.2

50

.30

Observed x/n = 8/10, Prior=Beta(.5, .5)

X in future N=10

Pro

ba

bili

ty

0 1 2 3 4 5 6 7 8 9 10

0.0

0.0

50

.10

0.1

50

.20

0.2

50

.30

Observed x/n = 8/10, Prior=Beta(.5, .5), Fixed P (blue), Random P (red)

X in future N=10

Pro

ba

bili

ty

Available Software

East (www.cytel.com)ADDPLAN (www.addplan.org)PEST http://www.rdg.ac.uk/mps/mps_home/software/pest4/pest4.htm

PASS http://www.ncss.com/passsequence.html

S+SeqTrial

http://www.statsci.com/products/seqtrial/default.asp

GLUMIP for internal pilot studyhttp://www.soph.uab.edu/coffey

M.D. Anderson (biostatistics.mdanderson.org)

PEST 4 (Planning and Evaluation of Sequential Trials)

This package includes facilities for:A range of designs to detect: superior efficacytreatment equivalence or non-inferiorityfutilitysafety concerns Flexible interim monitoring information-based to guarantee power Test statistics calculated within PEST 4 and displayed with the design boundariesAdjustment for prognostic factors using stratification or covariate adjustmentFinal analysis providing valid p-values, estimates and confidence intervalsSimulation for illustration and exploration of accuracy

Derry FA et al. Efficacy and safety of oral sildenafil (Viagra) in men with erectile dysfunction caused by spinal cord injury. Neurology. 1998 Dec;51(6):1629-33.

Figure 1. Single triangular sequential design shows original and adjusted continuation regions. Z is a measure of the observed advantage of sildenafil versus placebo and its expected value is the log odds ratio ([THETA]) multiplied by the information (V) collected up to that point. V is the information collected up to the interim analysis point about [THETA], as contained in the current value of Z. The expected value of the variance of Z is V. The arrow marks the critical point along the bottom boundary. To the left of this point sildenafil is significantly worse than placebo; to the right of this point there is evidence of no difference.

GROUP SEQUENTIAL TESTS in PASS

Group Sequential Tests of MeansThis module calculates sample size and power for group sequential designs used to compare two treatment means. The program allows you to vary the number and times of interim tests, the type of alpha spending function, and the test boundaries. It also gives you complete flexibility in solving for power, significance level, sample size, or effect size. The results are displayed in both numeric reports and informative graphics.

Group Sequential Tests of ProportionsThis module calculates sample size and power for group sequential designs used to compare two proportions. The program allows you to vary the number and times of interim tests, the type of alpha spending function, and the test boundaries. It also gives you complete flexibility in solving for power, significance level, sample size, or effect size. The results are displayed in both numeric reports and informative graphics.

Group Sequential Tests of Survival CurvesThis module calculates sample size and power for group sequential designs used to compare two survival curves. The program allows you to vary the number and times of interim tests, the type of alpha spending function, and the test boundaries. It also gives you complete flexibility in solving for power, significance level, sample size, or effect size. The results are displayed in both numeric reports and informative graphics.

East East is software for the design, simulation and monitoring of clinical trials using group sequential and adaptive methodologies. East allows investigators to easily design superiority, futility only, and non-inferiority trials, for all endpoints, with complete confidence that the type-1 error and power of the study will be protected. East's simulation capability aids clinicians and statisticians in understanding the trade-offs between trial designs so that designs can be compared and investigators can choose the best one. And East's interim monitoring module will perform all the necessary calculations for exact inference at an interim analysis. By designing, simulating and monitoring clinical trials using East, investigators can take advantage of flexible approaches that will allow them to identify futile trials and terminate them, fast-track effective therapies, and salvage underpowered studies by performing sample-size reassessment, without jeopardizing the statistical integrity of the trial.

S+SeqTrial

S+SEQTRIAL offers a complete computing environment for applying group sequential methods, including: A fully object-oriented language with

specialized objects (such as design objects, boundary objects, and hypothesis objects)

and methods (such as operating characteristics and power curve plots);

Easy comparative plots of boundaries, power curves, average sample number (ASN) curves, and stopping probabilities;

User-selected scales for boundaries: sample mean, z-statistic, fixed sample p-value, partial sum, error spending, Bayesian posterior mean, and conditional and predictive probabilities;

Stopping Rule Computation

The unified family of group sequential designs, which includes all common group sequential designs: Pocock (1977), O’Brien & Fleming (1979), Whitehead triangular and double triangular (Whitehead & Stratton, 1983), Wang & Tsiatis (1987), Emerson & Fleming (1989), and Pampallona & Tsiatis (1994);A new generalized family of designs. S+SEQTRIAL includes a unified parameterization for designs, which facilitates design selection, and includes designs based on stochastic curtailment, conditional power and predictive approaches;Applications including normal, Binomial, Poisson, survival, one-sample and two-sample; One-sided, two-sided, and equivalence hypothesis tests, as well as new hybrid tests;Specification of the error spending functions of Lan & DeMets (1989) and Pampallona, Tsiatis, & Kim (1993);Arbitrary boundaries allowed on different scales: sample mean, z-statistic, fixed sample p-value, partial sum, error spending, Bayesian posterior mean, and conditional and predictive probabilities; Exact boundaries computed using numerical integration.

Meta-Analysis

Cochrane data: randomized trials before 1980 of cortico-steroid therapy in premature labor and its effect on neonatal death

• What is the overall conclusion of the effect of corticosteroid in reducing neonatal death by combining the data from all trials?

No. Trial ev.trt n.Trt ev.ctrl n.ctrl

1 Auckland 36 532 60 538

2 Block 1 69 5 61

3 Doran 4 81 11 63

4 Gamsu 14 131 20 137

5 Morrison 3 67 7 59

6 Papageorgiou

1 71 7 75

7 Tauesch 8 56 10 71

Models: Assume Yi is the measure of treatment effect (e.g., log(OR) for Trial i)

Fixed-effect model: A common, fixed q2( , )i iY N s

2

2

| , ( , )

| , ( , )

i i i i i

i

Y s N s

N

Random-effect model: Random qi from each trial

2 2( , )i iY N s 2 2 2 2| Y, , ( (1 ) , (1 )) where /( )

is a shrinkage factor for study

i i i i i i i i i

th

N B B Y s B B s s

i

Estimation: Fixed Effect Model

Frequentist’s method When is assumed known

Bayesian method

2is

2

1 1

1

ˆ / with 1 /

ˆ ( ,1/ )

k k

MLE i i i i ii i

k

MLE ii

WY W W s

N W

20

2 2 20 0 01 1 1

201 1

Assume the prior for is (0, )

| , , ~ ( /( + ), 1 /( + ))

/( + ) is the posterior mean

k k k

i i i ii i i

k k

B i i ii i

N

Y s N WY W W

WY W

Fixed Effects: Mantel-Haenszel Method

windows(record=T)library(rmeta)data(cochrane)steroid.MH <- meta.MH(n.trt, n.ctrl, ev.trt, ev.ctrl,names=name, data=cochrane)summary(steroid.MH)Fixed effects ( Mantel-Haenszel ) meta-analysisCall: meta.MH(ntrt = n.trt, nctrl = n.ctrl, ptrt = ev.trt, pctrl = ev.ctrl, names = name, data = cochrane)------------------------------------ OR (lower 95% upper)Auckland 0.58 0.38 0.89Block 0.16 0.02 1.45Doran 0.25 0.07 0.81Gamsu 0.70 0.34 1.45Morrison 0.35 0.09 1.41Papageorgiou 0.14 0.02 1.16Tauesch 1.02 0.37 2.77------------------------------------Mantel-Haenszel OR =0.53 95% CI ( 0.39,0.73 )Test for heterogeneity: X^2( 6 ) = 6.9 ( p-value 0.3303 )

Fixed Effects: Mantel-Haenszel Method

StudyAucklandBlockDoranGamsuMorrisonPapageorgiouTauesch

Summary

Deaths/N(steroid)36 / 532

1 / 694 / 81

14 / 1313 / 671 / 718 / 56

Deaths/N(placebo)

60 / 5385 / 61

11 / 6320 / 137

7 / 597 / 75

10 / 71

OR0.580.160.250.700.350.141.02

0.53

0.1 0.5 1.0 1.5 2.0

Forest Plot

Funnel Plot

-2.0 -1.5 -1.0 -0.5 0.0

log(OR)

Siz

e: 1

/se

-4 -3 -2 -1 0 1

log(OR)

Siz

e: 1

/se

Estimation: Random Effect Model

Frequentist’s method When is assumed known

When is unknown: Restricted MLE (REML)

22 2

1 1ˆ ( ) ( ) / ( ) with ( ) 1 /( )

k k

MLE i i i i ii iW Y W W s

2 2 2 2 2 2

1 1

2 2

1 1

1

ˆˆFind REML of as the solution to ( )( ( ) ) / ( ) 1

ˆ ˆ ˆ ˆ = ( ) / ( ) with ( ) 1/( )

ˆ ~ ( , 1 / ( ))

k k

R i i R i ii i

k k

R i R i i i R i Ri i

k

R ii

kW Y s W

k

W Y W W s

N W

2

2 2 2

2 2

ˆ ˆ ˆFind empirical Bayes estimator of (1 ) where /( )

ˆ ˆ~ ( , (1 )) It ignores the uncertainly of the hyperparameter { , }

R R R Ri i R i i i i i R

R Ri i i i

B B Y B s s

N s B

Estimation: Random Effect Model (cont.)

Bayesian method Let

The posterior distribution is only analytically tractable for conjugate distributions. Otherwise, requires MCMC using BUGS or WinBUGS.

2 2

21 k

2 2 2 2

2B

~ (0, ) and ~ gamma( , ),

Then, the joint posterior distribution for ={ , , ... , , } is

( | , ) ( | , ) ( | , ) ( ) ( )

The posterior distribution of is

ˆ ( | , )

i i i ii

N a c d

V

p V Y s p y s p p p

E Y s

2i

2

, { ( ) }iP V d d d

Random Effect: DerSimonian-Laird Method

steroid.DSL <- meta.DSL(n.trt, n.ctrl, ev.trt,ev.ctrl,names=name, data=cochrane)summary(steroid.DSL)Random effects ( DerSimonian-Laird ) meta-analysisCall: meta.DSL(ntrt = n.trt, nctrl = n.ctrl, ptrt = ev.trt, pctrl = ev.ctrl, names = name, data = cochrane)------------------------------------ OR (lower 95% upper)Auckland 0.58 0.38 0.89Block 0.16 0.02 1.45Doran 0.25 0.07 0.81Gamsu 0.70 0.34 1.45Morrison 0.35 0.09 1.41Papageorgiou 0.14 0.02 1.16Tauesch 1.02 0.37 2.77------------------------------------SummaryOR= 0.53 95% CI ( 0.37,0.78 )Test for heterogeneity: X^2( 6 ) = 6.86 ( p-value 0.334 )Estimated random effects variance: 0.03

Random Effect: DerSimonian-Laird Method

StudyAucklandBlockDoranGamsuMorrisonPapageorgiouTauesch

Summary

Deaths/N(steroid)36 / 532

1 / 694 / 81

14 / 1313 / 671 / 718 / 56

Deaths/N(placebo)

60 / 5385 / 61

11 / 6320 / 137

7 / 597 / 75

10 / 71

OR0.580.160.250.700.350.141.02

0.53

0.1 0.5 1.0 1.5 2.0

Forest Plot

Random Effect: Bayesian Method

Data: list(lor=c(-5.47791E-01, -1.80359E+00, -1.40416E+00, -3.56675E-01, -1.05494E+00, -1.97490E+00, 1.65293E-02), sd=c(2.20347E-01, 1.11021E+00, 6.10841E-01, 3.72186E-01, 7.14875E-01, 1.08252E+00, 5.12081E-01), N=7)

Model: model{ for (j in 1 : N) { sinv[j] <- 1/(sd[j]*sd[j])

lor[j] ~ dnorm(beta[j], sinv[j]) beta[j] ~ dnorm(mu, tau) } mu ~ dnorm(0, 0.000001)

tau ~ dgamma(0.001,0.001) sigma <- 1/tau }

Initial Values: list(mu=0,tau=1)

>beta mean sd MC_error val2.5pc median val97.5pc start samplebeta[1] -0.5947 0.1851 0.00852 -0.943 -0.5929 -0.21830 5001 5000beta[2] -0.7186 0.3894 0.01536 -1.721 -0.6649 -0.12390 5001 5000beta[3] -0.7487 0.3418 0.01643 -1.632 -0.7003 -0.22140 5001 5000beta[4] -0.5680 0.2457 0.01045 -1.023 -0.5759 -0.02555 5001 5000beta[5] -0.6854 0.3278 0.01350 -1.468 -0.6507 -0.13080 5001 5000beta[6] -0.7396 0.3940 0.01698 -1.797 -0.6777 -0.17330 5001 5000beta[7] -0.5231 0.3002 0.01257 -1.018 -0.5512 0.20430 5001 5000> mu mean sd MC_error val2.5pc median val97.5pc start samplemu -0.6526 0.2448 0.01234 -1.174 -0.6392 -0.2289 5001 5000

> sigma mean sd MC_error val2.5pc median val97.5pc start samplesigma 0.1226 0.4387 0.01376 0.0007608 0.01902 0.8716 5001 5000

> exp(mu) mean sd MC_error val2.5pc median val97.5pcmu 0.5206902 1.277366 1.012416 0.3091280 0.5277144 0.795408

Random Effect: Bayesian Method

Summary of Meta-Analysis Result