Translational research: bench to where?

John P.A. IoannidisProfessor and Chairman, Department of Hygiene and Epidemiology,

University of Ioannina School of Medicine, Ioannina, GreeceProfessor of Medicine (adjunct), Tufts University School of Medicine,

Boston, USA

Definitions

• Translational research = conveying and making use of information from the bench to the bedside and from the bedside to the bench

• The bench should be extended to include the computer and the cyberspace

• The bedside should be extended to include the streets

Translation of major basic science promises

• Medical progress depends on important discoveries from basic biomedical research

We have limited empirical data on: how frequently these promises are materialized reach the stage of clinical application reach the stage of routine clinical use what is the time frame for this translation

Highly-promising basic science publications

25, 190 articles (published in 1979-1983 in Nature, Science, Cell, JEM, JCI, JBC)

562 articles (retrieved key word search)

153 potentially eligible articles (full text)

101 original articles that made clear promises for immediate clinical translation

The rate of publishing an RCT or positive RCT

decreased after 12-15 yrs had elapsed from the promise

Translation into clinical research

RCT Positive RCT

Current Licensed Clinical Use

Of the 27 technologies that had a published RCT:

only 5 (actually 4) are in licensed clinical use one 1 is in wide clinical use

• ACE inhibitors/ hypertension (Nature 1980)• Pergolide mesylate/ Parkinson’s disease (Science 1979)• Recombinant IL-2/cancer (JEM 1980)• Alpha 1 antithrypsin/ emphysema (JCI 1981)• Naloxone/ shock treatment (Science 1979)

Contopoulos-Ioannidis et al. Am J Med 2003 and Ioannidis JP. J Translational Med 2004

Translation of highly promising basic science research into clinical research occurs only sparingly and with considerable time lag

What are the problems underlying this attrition?

Major postulated problems of current “molecular” research

• Small sample sizes• Small effect sizes• Large number of biological factors• Old-epidemiology problems: confounding,

misclassification

All these result in questionable replication validity

Quanta of small effects

Ioannidis, Trikalinos, and Khoury, Am J Epidemiol 2006 and Zeggini et al. Science 2007

Total genetic information (subjects or alleles)

100005000

40003000

20001000

500400

300200

10050

40

Cu

mu

lative o

dds r

atio

543

2

1

,5,4,3

,2

,1

,05,04,03,02

DISEASE/GENE

Nephropathy/ACE

Alcoholism/DRD2

HTN/Angiotensinogen

Parkinson/CYP2D6

Lung cancer/GSTM1

Schizophrenia/DRD3

Down dementia/APOE

Lung cancer/CYP2D6

Ioannidis et al, Nature Genetics 2001

Non-replicated diminishing effects

Total genetic information (subjects or alleles)

3000020000

100005000

40003000

20001000

500400

300200

100

Cu

mu

lative

od

ds r

atio

4

3

2

1

,9,8

DISEASE/GENE

IHD/APOE

NTD/MTHFR

Ischaemic stoke/ACE

ICVD/APOE

Bladder cancer/NAT2

MI/PAI-1

NIDDM/KIR6.2-BIR

IHD/ACE

The other side: don’t give up early…

…but don’t give up on anything?

Counting fish in the sea of gene-disease associations

Multiplier Parameter

>10000000 Gene variants

>1000 Diseases

>10 Outcomes

>10 Subgroups

>10 Genetic contrasts

>10 Investigators

1 quadrillion Candidate analyses

12,000,000 interacting variants means…

102085 analyses

I need 8 slides to write all the zeros

• 1000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000

Wisely choosing our targets: “biological plausibility”?

• Just in the year 2002 studies were published addressing the relationship of the APOE epsilon polymorphism with familial Alzheimer’s disease; sporadic Alzheimer’s disease; colorectal cancer; fatty liver; atherosclerosis; hyperlipidemia; acute ischemic stroke; spina bifida; coronary artery disease; normal tension glaucoma; hypertension; Parkinson’s disease, diabetic nephropathy; pre-eclampsia; hepatitis C-related liver disease; cerebrovascular disease; coronary artery disease post-renal transplantation; non-specified cognitive impairment; childhood nephrotic syndrome; spontaneous abortion; multiple sclerosis; alcohol withdrawal; cognitive dysfunction after coronary artery surgery; alcoholic chronic pancreatitis; alcoholic cirrhosis; macular toxicity from chloroquine; macular edema; aortic valve stenosis; vascular dementia; type II diabetes mellitus; and migraine.

ROC curve of luciferase activity ratios for epidemiological associations for the same

gene variants

0,0 0,2 0,4 0,6 0,8 1,0

0,0

0,2

0,4

0,6

0,8

1,0

Ioannidis and Kavvoura Genet Med 2006

Invoking external evidence in support: too perfectly agreeable to be true

• We screened 46 studies on microarrays addressing major cancer outcomes in humans

• We scrutinized the comments made in the discussion regarding external evidence that supported or was against the findings of specific genes or groups thereof in the identified molecular signatures

• Supportive comments outnumbered comments against the research findings by 9 to 1 (270 vs. 29)

• 17% of the comments did not even pertain to the same gene as found in the study

• 53% did not pertain even to the same disease

Ioannidis, Polyzos and Trikalinos Eur J Cancer 2007

Waves of evidence

microcosms

Post-study odds of a true finding are small

• When effect sizes are small

• When studies are small

• When field are “hot” (many furtively competitively teams work on them)

• When there is strong interest in the results

• When databases are large

• When analyses are more flexible Ioannidis JP. PLoS Medicine 2005

A research finding cannot reach credibility over 50% unless

u<R

i.e., bias must be less than the pre-study odds

Readily available, available, hidden, and very well hidden data

Kyzas, Loizou, Ioannidis. JNCI 2005

What do we want after all?

• The written editorial policies of the 25 most-cited journals ask routinely for novelty, importance and significance as criteria for acceptance of manuscripts; only one (JAMA) mentions the need to discuss limitations

• Cell: “unusual significance”, “raise provocative questions”

• PNAS: “exceptional importance”• EMBO Journal: “manuscripts that deserve urgent

publication”• Biochemistry-US: “publication of inconclusive

results is discouraged”

Ioannidis, J Clin Epidemiol 2007

How about from the bedside to the bench:Five scenarios for academic medicine

Academia versus/and/or industry

Patsopoulos et al. BMJ 2006

Potential conflicts in driving translational research

Patsopoulos et al. BMJ 2006

Brain drain and brain deficit

Clinical evidence

and burden of disease

Swingler et al. BMJ 2003

Medicine losing (lost) control of the basic sciences

ICRAM, BMJ 2004

Mentoring: how to get the best of it?

• In many fields <20% of faculty members have had a mentor

• Women have an extra difficulty to find mentors

• Academics: Solitary individual paths with no parallel

Choosing academic medicine for a career? You must be kidding!

• Graduate degree or special fellowship with protected time

• Mentor role

• Start early!!

• Interest declines as people progress through clinical training

The news from the industry:Few new chemical entities

Service. Science 2004

In search of blockbusters

Service. Science 2004

JCI, November 2006

The glacial pace of clinical translation

Ioannidis, JAMA 2005, Contopoulos-Ioannidis et al. (submitted)

Evolution and translation of research findings does not have to be a roundtrip

from bench to nowhere

PLoS Clinical Trials 2006

Learning from the past: suggestions• Promote multidisciplinary communication• There are too many discoveries; we need more replication

and more stringent and determined pursuit of replicated research promises

• Foster systematic, evidence-based approaches to research. • Acknowledge in earnest the difficulty and even the failures

of the scientific enterprise.• Examine what pathways have led to specific successes and

failures in translation.• Synthesize evidence systematically from many studies and

teams of investigators and anticipate this integration from the design phase of research.

• Give credit to original ideas, good quality work and robust methodology rather than to impressive claims and magazine hype.