Why should we trust scientists about food and health?

The value of observational studies Professor Kay-Tee Khaw FMedSci

Identifying the environmental causes of disease: how should we decide what to believe and when to take action?

A report of an Academy of Medical Sciences working group

Main outputs

1.Five key recommendations

2. Detailed guidelines for:• Researchers• Editors• Science or medical writers• Clinicians and healthcare practitioners• Policy managers• Funders

Cartoon that illustrates the confusion that sometimes accompanies research into the causes of disease.

Why are we interested in causes?

1.ExplanationWhy did this happen?

2.PredictionCan we predict what is likely to happen in future?

3. Alter the course of eventsCan we change outcome?

4.Action: changes to policy and practice

Key research implications

1. Many environmental influences e.g. diet, are not open to experimental manipulation for practical and ethical reasons.

2. We have to rely mainly on non-experimental research strategies

3. Possible explanations for associations apart from causality include chance, confounding or bias.

Meat consumption and colon cancer incidence in various countries

Armstrong and Doll (1975) Environmental factors and cancer incidence in different countries. International Journal of Cancer Research. 15, 617

Pairs of nesting storks and births over time in Sweden

From association to cause

The Bradford Hill guidelines provide a useful framework for interpreting whether associations are likely to be causal:

• Strength• Consistency• Specificity• Temporality• Biological gradient• Plausibility• Coherence• Experiment• Analogy

Bradford Hill A (1965) The environment and disease: association or causation? Proceedings of the Royal Society of Medicine, 58, 295-300.

Main varieties of non-experimental research

1. Case-control studies2. Prospective cohort studies3. Ecological designs4. Natural experiments

May be combined with animal models and

randomised controlled trials

Main methods of the working groupExamples of associations from non-experimental research which we still believe are causal

• Smoking and lung cancer.• Lipids and coronary artery disease.• Folic acid and neural tube defects.• Foetal alcohol syndrome.

Examples of associations from non-experimental research which we no longer believe are causally related

• Caffeine in pregnancy.• Antioxidant Vitamin supplements and mortality.• Early alcohol use and later alcohol abuse or

dependency.

Key features of associations from observational studies for which causality is accepted as robust1. Very large effect or rare and unusual outcome.

2. Detailed attention to alternative non-causal explanations.

3. Multiple research designs (including natural experiments).

4. Causal inference tested in multiple populations and settings differing in risk characteristics.

5. Confirmatory findings from animal models and human experiments pointing to likely mediating biological processes.

Key lessons from observational studies in which associations have not been subsequently substantiated

1. Most misleading findings result from small scale, poor quality studies, often not subjected to peer review, nor published in scientific journals, and not replicated.

2. Non-experimental studies are especially susceptible to bias when there are selection effects stemming from personal choices e.g. lifestyle, supplements.

3. Particular caution when major interests are concerned e.g commercial considerations or strongly held clinical views.

4. Causal claims, even from good studies, need to be based on similar findings from multiple studies using varying designs.

Main conclusions

Randomised controlled trials can minimise but not exclude confounding and bias

Most of what we do is not supported by evidence from randomized controlled trials for ethical and feasibility reasons and generalizability sometimes an issue

We usually have to make policy relying on on other types of research design and understand strengths and limitations when interpreting and applying findings

Judgement as to what action is justified by available evidence: quantitative assessments/ sensitivity analyses of consequences of different options

Always uncertainty: new cohorts, new exposures, factors, new interactions: need for continuing surveillance and research: dynamic process

‘All scientific work is incomplete – whether it beobservational or experimental. All scientific work is liable to be upset or modified by advancing knowledge. That does not confer upon us a freedom to ignore the knowledge we already have or to postpone the action that it appears to demand at a given time.’

Sir Austin Bradford Hill CBE FRS, 1965