EXAMINING THE RELIABILITY AND USEFULLNESS OF INTERIM ANALYSIS

DATA: CASE OF ANTIPSYCHOTIC DRUG TRIALS

Jonathan Rabinowitz (1), Nomi Werbeloff (1), Stephen Levine (2)

 

(1) Bar Ilan University, Ramat Gan, Israel

(2) Haifa University

2

What is NEWMEDS?NEWMEDS is

an international consortium of scientists that has launched one of the largest ever research academic-industry collaboration projects

to find new methods for the development of drugs for schizophrenia and depression.

EFPIA companies: H Lundbeck A/S, Abbott, AstraZeneca AB, Eli Lilly and Company Ltd, Janssen Pharmaceutica NV, Novartis Pharma AG, Orion Corporation, Pfizer Limited, F. Hoffmann-La Roche AG, Institut de Recherches Servier

Universities: King’s College London (UK), Karolinska Institutet (Sweden), The University of Cambridge (UK), Central Institute of Mental Health (Germany), CSIC (Spain), The University of Manchester (UK), Bar Ilan University (Israel)

SME‘s

Psynova Neurotech Ltd (UK), deCODE genetics (Iceland), GABO:mi (Germany)

3

Who supports NEWMEDS?

NEWMEDS is funded by the Innovative Medicines Initiative (IMI), a unique large scale public-private partnership between the European Union (represented by the European Commission) and the pharmaceutical industry (represented by the European Federation of Pharmaceutical Industries and Associations, EFPIA).

IMI aims to put Europe at the forefront of biopharmaceutical innovation and to support more efficient discovery and development of better medicines for patients.

www.imi.europa.eu

4

The Consortium

5

Key data for NEWMEDS

Start date: 1 September 2009

Runtime: 5 years

Structure: 11 Workpackages

Clusters: System based animal models (A)

Translational Technology (B)

Patient Stratification (C)

6

Schizophrenia Database

Data from: Astra Zeneca, Janssen, Lilly, Lundbeck, Pfizer

64 Industry sponsored studies 34 placebo controlled

30 active comparator25,900 patients

16,105 study drug7,119 active comparator2,676 placebo

1 NIMH sponsored study CATIE 1,493 patients

1 European Union sponsored study EUFEST 498 patients

7

Depression Database

Data from: Astra Zeneca, Lundbeck, Pfizer and Lilly                 

        39 placebo controlled Industry sponsored studies

                                           12,217 patients

                                             8,260 active drug

                                             3,957 placebo

8

Methodological questionInterim analyses yield effect size information that assist in deciding about potential sample size adjustments.

To what extent are the results of the first cohort of patients to be enrolled similar to the results of the subsequently enrolled patients?

In the rush to complete studies quality may decline reducing effect sizes over time.

Method: Studies split at midpoint of enrolment.

Results: Difference in placebo vs. active treatment difference in effect size (Cohen’s d) between cohorts: Mean Median Antipsychotic trials -0.01 -0.03Antidepressant trials 0.05 0.04

Overall the first and second cohorts enrolled yielded similar results. 9

Effect size differences placebo vs. antipsychotic treatment in first and second cohort of patients enrolled ordered by difference:

>Negative difference greater effect first cohort, positive greater effect second cohort

>9 trials differences were positive, in 4 differences were negative, where first cohort showed more difference these differences were small (less than -.08).

>In 7 of the 20 studies the first cohort well outperformed the second cohort (from -12 to -.44).

Effect size differences placebo vs. antidepressant treatment in first and second cohort of patients enrolled ordered by difference:

>In 22 trials the differences were positive, in 17 the differences were negative.

>In 12 studies these differences were small (less than .08 in favor of either cohorts).

Using first cohort results to re-estimate sample size

-Repower study, as if designing a new study.Ignores the fact that first cohort results are fixed.

-Repower second cohort based on conditional probability Makes use of the fact that first cohort results are fixed.

Sample size needed for second cohort estimated using formula for adding z scores.

1. Convert Cohen’s d to z = (d*sqrt(n))/2

2. Z needed for second cohort=(1.96*SQRT(n1^2+n2^2)-n1*z)/n1

(Based on formula for to sum weighted z scores.)

3. Compute P of z p=(1-NORMSDIST(z))*2

4. Compute sample size using d at time 1 and the p value from previous step.

(a) Using power software. (b) Resampling-bootstrapping (sample sizes increased until obtaining needed p value).

April 20, 2023 14

Study ES 1,p= , n=placebo/activeES 2, p= , n=placebo/activeDifference of ES2-ES1

Repowering study based on first cohort, total needed for study and additional patients needed for second cohort, if any.

Conditional probability

Sample size calculation for second part 90%, 2-tail 

   Boot-strapping 90%, 2-tail 

A

ES1=.25, p=.15, n=134 (32/102)ES2=.18, p=.30, n=137 (38/99) Diff=-.07  950 (n=230/620)

Add: 950-137=813

 Z needed 1.31, p=.19576-137=439

 450-137=313 

B

ES1=.33, p=.22, n=58 (18/40)ES2=.08, p=.74, n=63 (25/38)Diff=-0.25   438 (n=146/292)

Add: (438-63)=385

 Z needed 1.45, p=.15309-63=246 

 280-63= 217  

C

ES1=.24, p=.089, n=205 (50/155)ES2=.35, p=.015, n=203 (48/155)Diff=.11  976 (n=244/732)

Add: (976-203)=773

Z needed 1.06, p=.29508-203=305 

400-203=197 

D

ES1=.34, p=.21, n=59 (21/38)ES2=.53, p=.06, n=55 (n=17/38)Diff=.19 411 (n=137/274)

Add: (411-55)=356

Z needed 1.46, p=.14300-55=245

270-55= 215  

April 20, 2023 15

Conclusion

Results suggest that earlier cohorts in studies of antipsychotic and antidepressant medications can reliably serve to estimate trial effects in interim analyses and to efficiently adjust sample size calculations using conditional probability.