potential and limitations of real-life data in clinical research...2016/08/31 · hr = 0.74 (0.62...

Potential and limitations of real-life data

in clinical research

Marc Buyse, ScD

• Motivating example

• Appeal and problems of real life data

• Confounding

• Causal inference

• I have a dream

Outline

Outline



• Confounding


• I have a dream

First line treatment of MBC

Bevacizumab for metastatic breast cancer

Paclitaxel

Bevacizumab

Paclitaxel

First line treatment of MBC


Paclitaxel

Bevacizumab

Paclitaxel

Choice

?


Large difference in PFS


No difference in OS

Bevacizumab for MBC

E2100 AVADO RIBBON-1

No. of patients 722 488 1237

Geography US (90%) Ex-US US (50%)

Randomization

ratio (BV:PL) 1:1 1:1 2:1

Chemotherapy Paclitaxel

weekly Docetaxel

Capecitabine,

Docetaxel/nab-Paclitaxel,

Doxorubicin/Epirubicin

Primary Endpoint PFS PFS PFS

PFS hazard ratio 0.48

P < 0.001

0.62

P < 0.001

0.67

P < 0.001

Key Secondary

Endpoints OS, ORR

OS, ORR,

1-yr survival

OS, ORR,

1-yr survival

Ref: O’Shaughnessy et al, ASCO 2010

11

Overall Survival, Pooled Population

Non-BV

(n=1008)

BV

(n=1439)

Median, mo 26.4 26.7

HR (95% CI) 0.97 (0.86–1.08)

1-yr survival

rate (%) 77 82

Ref: O’Shaughnessy et al, ASCO 2010

Bevacizumab for MBC

ESME

Ref: Delaloge et al, Ann Oncol 2016; 27: 1725-32.

Bevacizumab for MBC

ESME Hazard ratio = 0.67 (0.60 – 0.75)

PFS

HR = 0.74 (0.62 – 0.81)

PFS

HR = 0.64 (0.57 – 0.71)

Bevacizumab for MBC

real life data 3426 patients

clinical trial data 2477 patients

PFS

HR = 0.74 (0.62 – 0.81)

OS

HR = 0.67 (0.60 – 0.75)

PFS

HR = 0.64 (0.57 – 0.71)

OS

HR = 0.97 (0.86 – 1.08)

Bevacizumab for MBC

real life data 3426 patients

clinical trial data 2477 patients

Ref: Buyse and Vansteelandt, Ann Oncol 2016; 28: 182.



• Confounding


• I have a dream

Appeal of real life data

• Reflect reality rather than trials conducted with

– carefully monitored centers

– highly selected patients

– tightly controlled procedures

– biased « explanatory » analyses

• Large numbers

• Available anyway

Can be used to describe patterns of care

Can be used to detect rare treatment toxicities

Limitations of

real life data

• Messy data

• Missing data

(not at random)

• Uncontrolled treatment

choices

• Heterogeneity across

centers / physicians

Cannot be used to assess

treatment efficacy due to

CONFOUNDING

Limitations of

• Messy data

• Missing data

(not at random)

• Uncontrolled treatment

choices

• Heterogeneity across

centers / physicians

Cannot be used to assess

treatment efficacy due to

CONFOUNDING

real life data

• Artificial settings

• Hard to generalize

• Obscenely expensive

But, cannot be replaced to

assess treatment efficacy

through RANDOMIZATION

clinical trial data



• Confounding


• I have a dream

Confounding

Bevacizumab Time to death

Age

Confounding

Time to death

Sex

Bevacizumab

Age

etc…

Measured confounders in MBC

Time to death Bevacizumab

Confounders

Age

Sex

Type of metastases

Metastatic sites

Number of metastatic sites

Synchronous metastases

Time from diagnosis

Therapy at diagnosis

ER status

Grade

Period of care

Region of care

Regressing on measured confounders


Confounders

Stratifying for measured confounders


Confounders

Using propensity scores


Confounders

Bevacizumab for MBC

ESME

HR = 1

Ref: Delaloge et al, Ann Oncol 2016; 27: 1725-32.

What about unknown confounders?


Confounders

Unknown

confounders



• Confounding


• I have a dream

X Y

U

X explanatory variable (treatment)

Y dependent variable (outcome)

U confounders

What about unknown confounders?

Identify an « instrumental variable »

X Y

Z

U

X explanatory variable (treatment)

Y dependent variable (outcome)

U confounders

Z instrumental variable

Requirements for instrumental variable Z

X Y

Z

U

Z not independent of X inclusion restriction

Requirements for instrumental variable Z

X Y

Z

U

Z not independent of X

Z independent of U

Z independent of Y X

inclusion restriction

exclusion restrictions

Does smoking cause poor health?

Smoking Poor health

Tobacco tax

Age

Tax not independent of smoking

Tax independent of age

Tax independent of poor health smoking

Time to death

Instrumental

variable

Difficult to find plausible instrumental variable for

bevacizumab treatment !

Bevacizumab

Confounders

Does bevacizumab prolong survival in MBC?

Does oxaliplatin prolong survival in MCC?

R

FOLFOX4

FOLFOX7 x 6 cycles

sLV5FU2

Ref: Tournigand et al, J Clin Oncol 2006;24: 394.


Cycles every 14 days; doses in mg/m²

LV 400 5-FU 2400

LV 400 5-FU 2400-3000

LV 400 5-FU 600 5-FU 600

Oxali 130

H0 H2 H24 H48

5FUb 400

H0 H2 H24 H48

H0 H2 H24 H48

5FUb 400 5FUb 400

LV 400

Oxali 130

FOLFOX4

FOLFOX7

sLV5FU2


No significant difference

Ref: Tournigand et al, J Clin Oncol 2006;24: 394-400.


FOLFOX4 27%

Confounding by drugs given upon progression

FOLFOX7

Oxaliplatin

55%

70%

Irinotecan

61%

32%

Others

33%

Expected

100%

Expected

0%


FOLFOX4 78% 46% 18%

Confounding by successive treatment lines

FOLFOX7 73% 39% 20%

Second

line

Third

line

Fourth

line

Per-protocol

oxaliplatin

reintroduction

40%

Confounding effects on survival !


Measured confounders in OPTIMOX1

Time to death Oxaliplatin

Confounders

ECOG Performance status

ACE

LDH

ALK

Treatments given on progression

Age

Sex

Primary tumor location


Synchronous metastases

Adjuvant chemotherapy

Time-dependent covariates (TDC)

Jan 2011

0

0

1

0

1

1

2

2 Tumour

progression

Oxaliplatin

reintroduction

Irinotecan

second-line

treatment

Aug 2011 Sep 2012 Apr 2013

Ref: de Gramont et al, J Clin Oncol 2007;25:3224-9.

PH model for survival

Survival HR

(P-values)

ECOG performance status

0 vs 1-2

0.62 (P < 0.0001)


1 vs > 1

0.78 (P = 0.019)

LDH

ULN

0.55 (P < 0.0001)

ALK

ULN

0.78 (P = 0.031)


Survival HR

(P-values)

Adjusted survival HR

(P-values)

ECOG performance status

0 vs 1-2

0.62 (P < 0.0001)

0.60 (P < 0.0001)


1 vs > 1

0.78 (P = 0.019)

NS

LDH

ULN

0.55 (P < 0.0001)

0.60 (P < 0.0001)

ALK

ULN

0.78 (P = 0.031)

NS

Time to tumour progression - 3.99 (P < 0.0001)

Time to oxaliplatin reintroduction - 0.56 (P < 0.0001)

Time to irinotecan use - 0.40 (P < 0.0001)

PH model for survival with TDC


Oxaliplatin Time to death

Preference

Ref: Brookhardt & Schneeweiss, Int J Biostat 2007;3:14-32.

Prognostic

factors


Centers have widely different policies for oxaliplatin use

Oxaliplatin Time to death

Preference

Ref: Brookhardt & Schneeweiss, Int J Biostat 2007;3:14-32.

Prognostic

factors

Preference not independent of oxaliplatin use

Preference independent of prognostic factors

Preference independent of time to death oxaliplatin


Survival by preference

for oxaliplatin reintroduction

% patients with

oxaliplatin

reintroduction

Nr of

centres

Nr of

patients

Median

survival

(95% C.I.)

Hazard ratio

for survival

(95% C.I.)

0% 18 37 14.8 mos 1.00

1 – 20% (mean 15%)

6 100 16.8 mos 0.91

(0.56 – 1.48)

21 – 40% (mean 34%)

12 227 19.4 mos 0.78

(0.50 – 1.21)

> 40%

(mean 55%) 21 257 22.6 mos 0.59

(0.38 – 0.91)




ℎ𝑟 = 1 − 0.7 ∙ 𝑝 ℎ𝑟

𝑝



• Confounding


• I have a dream

The current nightmare

?

Adapted from JC Soria, ECCO 18 / ESMO 40, Vienna, September 2015

Total cost per new molecular entity : $ 1.8 billion

Real world data

Fixes (painful and incremental)…

Ref: Najafzadeh and Schneeweiss, NEJM 2017; 376;13: 1203-5.

Dreaming of a disruptive future

Total cost per new molecular entity :

CRO centric clinical research

Patient centric clinical research

Dreaming of a disruptive future

Obscenely expensive and

increasingly irrelevant

Cost-effective and

increasingly relevant

potential and limitations of real-life data in clinical research...2016/08/31 · hr = 0.74 (0.62...

Documents