Lesson learnt from IMI-PROTECTBenefit-risk assessment of medications,

and adaptation to vaccines

Perspective on Benefit-Risk Decision-Making in Vaccinology Conference24th June 2014Fondation Merieux, Annecy, France

Shahrul Mt-Isa

School of Public Health

Follow us

Disclaimers

“The processes described and conclusions drawn from the work presented herein relate solely to the testing of methodologies and representations for the evaluation of benefit and risk of medicines.

This report neither replaces nor is intended to replace or comment on any regulatory decisions made by national regulatory agencies, nor the European Medicines Agency.”

2

PROTECT is receiving funding from the European Community’s Seventh Framework Programme

(F7/2007-2013) for the Innovative Medicine Initiative (www.imi.europa.eu)

3

IMI-PROTECT Work Package 5Benefit-risk integration and representation

The licensing challenge in medical decision-making

• The task of regulators (e.g. EMA, FDA) is to make a good and defensible decision on which medicines should receive a license for which indications, based on the available evidence of risks and benefits.

• It is increasingly important to be able to justify and explain these decisions to patients and other stakeholders.

• Whose value preferences take priority?

• Can more formal approaches of decision-making, and especially more modern methods of graphical display help regulators do these better?

• Which quantitative approach(es) to use?

4

Methodologies available

5

Non –quantitative

All B-R assessment approaches

Benefit-risk assessment framework

Metric indices for B-R assessment

NNTNNH

AE-NNTRV-NNH

Impact numbersMCE

RV-MCEMARNEAR

Estimation techniques

QALYDALYHALE

Q-TWiST

UT-NNTINHBBRRGBR

Principle of 3TURBO

Beckmann

BLRANCB

Decision treeMDP

MCDASMAASBRAM

CUIDI

Trade-off indices

PROACT-URLASF

BRATFDA BRF

CMR-CASSCOBRASABRE

UMBRAOMERACT 3x3

Descriptive framework

Quantitative framework

Threshold indices Health indices

SPMCVCA

DCE

DAGsPSMCPMITC

MTCCDS

Approaches excluded and not appraised

Utility survey techniques

Mt-Isa et al. Balancing benefit and risk of medicines: a systematic review andclassification of available methodologies. Pharmacoepidemiology and Drug Safety 2014. DOI: 10.1002/pds.3636.

Recommendation Roadmap

6

Planning

Evidence gathering and data preparation Analysis

Exploration

Conclusion and dissemination

• critical issues

• think & discuss purpose and context

• documentation

• foundations for future analyses and updates

• relevant evidence

• data collection

• data aggregation

• missing/incomplete data

• Evaluate data

• Quantify benefits and risks

• Weigh or integrate

• robustness

• sensitivity

• assumptions and uncertainties

• other consequences

• impact or added value to the RMPs

• communicate results/consensus

• any influence on future actions

• transparent audit trail

• ensures "big picture" is not lost

Stakeholders and decision-makers

Patients

• Make decisions for themselves

Healthcare providers

• Make decisions based on prescribing lists

Health technology assessors

• Makes decisions on cost-effectiveness

Regulators

• Makes decisions on quality, safety, efficacy and benefit-risk balance to individuals and public health

Pharmaceutical companies

• Makes decisions on what to develop for which licenses to apply

7

Perspectives

• Individual vs public health

• Stakeholders:

– who the decision maker is (the first party),

– who the decision is to be made for (the first or the second party), and

– any other stakeholders involved (the third party) in the decision making.

• Highlight relevant benefits and risks

• Affects different population differently

8

Rimonabant case study

Drug of interest Rimonabant

Indication Weight loss in obese and overweight patients with co-morbidities in adults (>18y)

Severe side effect

Increased risk of psychiatric disorders

Regulatoryhistory

2006 Approved in June2009 Voluntary withdrawal in January bythe MAH

Comparators Placebo, orlistat, sibutramine9

It is an interesting case study because: Despite some benefits in

patients, risk of psychiatric disorders emerged post-marketing, resulting

first in label changes and then voluntary withdrawal in the EU.

Rimonabant value tree

10

Median [Range] absolute valuesIndirect treatment comparison

Criteria Placebo Orlistat Sibutramine Rimonabant

Achieving 10% Weight loss (proportion)

0.11 [0.10,0.14] 0.24 [0.15,0.34] 0.47 [0.16,0.80] 0.40 [0.22,0.62]

Reduction in HDL Cholesterol (mg/dL)

1.02 [0.95,1.10] 1.92 [1.12,2.75] 2.63 [1.19,4.11] 2.12 [0.26,3.93]

Cardiovascular disorders (proportion)

0.02 [0.01,0.02] 0.11 [0.00,1.00] 0.04 [0.00,1.00] 0.03 [0.00,0.99]

Psychiatric disorders (proportion)

0.01 [0.01,0.01] 0.00 [0.00,0.03] 0.02 [0.00,0.11] 0.02 [0.00,0.06]

Gastrointestinal disorders(proportion)

0.05 [0.05,0.06] 0.19 [0.06,0.50] 0.09 [0.02,0.35] 0.07 [0.04,0.15]

11

• AKA Impact Numbers

• England and Wales 2006 >20y

• Assumptions: target popn similar to placebo arm, ~62% overweight+, ~70% eligible Rx, target 10% coverage

Attributable Population assessment Performance assessment Impact estimation

Population Impact Measures (PIM)

Median (95% CI) Achieved a 10% weight loss in one year Any psychiatric disorders

PAR 0.54 (0.47, 0.60) 0.55 (-0.06, 0.87)

PIN-ER-t 1338016 (1052444, 1663270) 27190 (-2633, 98730)

EIN 6 (5, 9) 253 (-1911, 8.5e+8)

NEPP (target 10%) 291781 (207657, 402174) 6140 (-247, 53674)

12Verma et al. Population Impact Analysis: a framework for assessing the population impact of a risk or intervention. J Public Health (Oxf). 2012 Mar; 34(1):83-9. doi: 10.1093/pubmed/fdr026.

PIM Trade-off

13

• NCB and BRR

• Single benefit vs single risk

-200

-150

-100

-50

0

50

100

150

200

250

300

350

400

.1 .125 .25 .5 1 2 4 8 10

Relative importance of benefit to risk

NCB (x1000) of NEPP.92

.96

1

.1 .125 .25 .5 1 2 4 8 10

Pr(B>R) of NEPP

-3000

-2000

-1000

1

1000

2000

3000

4000

5000

.1 .125 .25 .5 1 2 4 8 10

Relative importance of benefit to risk

BRR of NEPP

Multi-criteria methodology requirements

• Most multiple criteria methods for benefit-risk assessment require:

– Favourable (benefit) and unfavourable (risk) effects data

From clinical trials, post-marketing surveillance, epidemiological studies

– Preference data (weights, value functions)

14

SMAA illustration: Scoring and weighting

15

Outcome:

Achieved 10%

weight loss

Measure:

40%

(range 22% - 62%)

Value(measure):

50%

(range 28% - 78%)

0.0

2.0

4.0

6.0

8.0

10.0

Den

sity

0.2 0.3 0.4 0.5 0.6

0.0

2.0

4.0

6.0

8.0

Den

sity

0.3 0.4 0.5 0.6 0.7

Weight space:

57%

(range 21% - 100%)

0.0

0.5

1.0

1.5

2.0

2.5

Den

sity

0.2 0.4 0.6 0.8 1.0

0.0

1.0

2.0

3.0

4.0

5.0

Den

sity

0.0 0.2 0.4 0.6 0.8

BR

Contribution

29% (range

9% - 68%)

Distributions of utilities

16

• Non-missing weights

model elicited from

study team

• Drugs

• Placebo

• Orlistat

• Sibutramine

• Rimonabant

Performance ranking

17

• Non-missing weights

model

• Drugs

• Placebo

• Orlistat

• Sibutramine

• Rimonabant

• Interactive version

allows own weightshttp://public.tableausoftware.com/vie

ws/Finalwave2dashboard-

fullrangeweight/Dashboardutilitydensi

ty?:embed=y

Direct preference elicitation

• Simple

• Unstructured

• May result in inconsistency

• Lack transparency

18

Structured elicitation methods

available – not covered

Remarks

• Formally structured benefit-risk assessment can aid with transparency and communication of benefits and risks

• These methodologies do not make decisions themselves. They support decision-making and are not intended to replace medical expertise.

• Stakeholders such as patients and public involvement may add value and would lead to more clinically relevant decisions.

19

Final thoughts…

1. Estimate PIMS

2. Use within multi-criteria analysis method e.g. MCDA or SMAA

20

RESOURCES

21

Follow us

Acknowledgements

• The research leading to these results was conducted as part of the PROTECT consortium (Pharmacoepidemiological Research on Outcomes of Therapeutics by a European ConsorTium, www.imi-protect.eu) which is a public-private partnership coordinated by the European Medicines Agency.

• The PROTECT project has received support from the Innovative Medicine Initiative Joint Undertaking (www.imi.europa.eu) under Grant Agreement n° 115004, resources of which are composed of financial contribution from the European Union's Seventh Framework Programme (FP7/2007-2013) and EFPIA companies’ in kind contribution.

22

Work Package 5 of IMI-PROTECT

23

Public Private

Imperial College (co-leader) Merck KGaA (co-leader)

EMA AMGEN

DHMA AstraZeneca

MHRA Bayer

Mario Negri Institute GSK (deputy co-leader)

GPRD Lilly

La-SER Novartis

IAPO Novo Nordisk

Pfizer

Roche

Sanofi-Aventis

Takeda

24

Click or scan me!

http://www.imi-protect.eu/results.shtml#