Value of Information analysis for trials of treatments for rare diseases: Early insights from the InSPiRe Project Dr Jason Madan, Warwick Medical School.

InSPiRE: Innovative methodology for small populations research

InSPiRe : Development of novel and methods for the design and analysis of clinical trials in rare diseases / small populations

WP1 - Research in early phase dose-find trials in small populations

Sarah Zohar, INSERM UMR1138, Paris

WP2 - Research in decision-theoretic designs for clinical trials in small

populations

Nigel Stallard Warwick Medical School,

Coventry

WP3 - Research in confirmatory trials for small populations and personalised

medicines

Martin Posch

Medizinische Universität, Vienna

WP4 - Research in use of evidence synthesis in the planning and interpretation of clinical trials in small populations and rare diseases

Tim Friede University Medical Center

Gottingen

This project has received funding from the European Union's Seventh Framework Programme for research, technological development and

demonstration under grant agreement no 602144

InSPiRe Team Dinner in Vienna, June 2014

Small populations - challenges for policy and research

EU defines a rare disease as affecting <5 in 10000 6000 rare diseases identified, affecting around 30 million EU citizens Orphan drugs benefit from specific drug development legislation in US and EU 18% of Orphan drugs cost >£100K per patient per year. Stratified and personalised medicine approaches can increase the number of conditions with rare disease status.

Is Decision-making in Rare Diseases Different?

Paulden et al review the literature on decision factors and proposed decision-making frameworks for orphan drugs: - rule of rescue supports the non-abandonment of patients with severe diseases if alternatives are not available (irrespective of cost). - equity principle argues that resource allocation should be based on need, distribution of health, and magnitude of benefit (no special treatment for orphan drugs) - rights-based approach states that individuals have a right to a minimum level of healthcare (even if their disease is rare). Cost-effectiveness is the application of these principles, not a separate factor.

Paulden et al, Pharmacoeconomics 2015

Proposed framework for aiding coverage decision

Effectiveness evidence requirements for orphan drugs Some authors that orphan drugs should provide the same level of evidence. Proposed decision frameworks often assume high quality evidence not available. - Smaller trials - Intermediate outcomes (e.g. oncology trials that do not look at long-term survival)

Decision – Theoretic Trial Design

• Mainstream statistical literature on trial design follows frequentist paradigm • Define null hypothesis • Design trial to control probability of type I / type II error (false positive

/ false negative) • Desired error rates set by convention (e.g. 5% type I , 10% type II). • Frequentist hypothesis testing, no synthesis with other data sources

• Decision-theoretic trial design statistical literature: • Bayesian framework • Decision involving choice between mutually exclusive alternatives • Uncertain payoff / utility function to maximise.

Rare diseases can make conventional approaches to trial design unfeasible.

Statistical literature on decision-theory and rare disease trials

• We carried out a systematic review of the literature on decision theoretic designs for pilot studies and small clinical trials (Hee et al 2015)

• 67 articles identified (up to October 2014)

• 8 explicitly mention Value of Information analysis • Range of decision-making perspectives – commercial, regulatory, societal

or none.

• Three types of study design – single stage, multi-stage, portfolio / multi-arm.

Key issues from the Statistical Literature on decision-theoretic trial design

• Computational methods for identifying optima

• Choice and structure of the utility function

• Construction of appropriate priors

A rule of thumb for optimal sample size

Let N be the total population for whom two treatments are being considered Cheng et al (2003) show that the optimal trial size tends asymptotically to O(N1/2) where the primary endpoint follows a Bernouilli distribution Stallard et al (2015) extend this result to situations where the primary endpoint follows any distribution from the Exponential family. Applicability of this result in small populations limited by - restrictive assumptions about utility functions - asymptotic applicability of findings.

Characteristics of small population decisions: - Difficulties in recruiting large numbers of patients - Multiple trials competing for same patient group

- Multiple regulatory frameworks with different models of interaction

between decision-makers - Strategic interaction between industry and regulators. - Regulator and reimbursement roles often separate. - How to set reimbursement rules to optimise industry-sponsored

trials in rare diseases?

VoI for rare diseases – Structuring the decision problem

Strategic interaction of sponsor and regulator decision-making

B

b

Consider a proposed trial of a new treatment T. Let B = the true individual incremental net benefit of T relative to standard of care = prior distribution for B n = the sample size of the proposed trial = Required assurance level for net benefit = Posterior mean for individual net benefit after trial C(n) = Cost of conducting the trial N = The total population that are eligible for T. p = The price paid to the sponsor for each patient treated.

( 0)A p bU I BN C

( 0)R p b

U NI B p

( 0)S p bU NI p C

Societal, commercial and regulatory objective functions

Societal objective function

Regulatory objective function

Commercial objective function

Need to consider who sets which decision parameters, and in which order, to arrive at optima.

Example: α fixed, regulator chooses p, sponsor chooses n. If sponsor goes first, n* = p* = 0 If regulator goes first, can identify pareto optima by finding solutions where = - (assuming convexity).

Isoutils for regulator and sponsor, with heavy line depicting pareto optima, and small circles depicting global optimum for the regulator

/ 0.3E B

/ 0.1E B

Moving to more realistic decision scenarios

Can use similar approaches to explore more realistic regulatory frameworks for trials in rare diseases. E.G : • Make price a function of the posterior mean and the probability this is zero

• Make total net benefit non-linear in population size

• Introduce asymmetric priors.

• Extend to portfolios of treatments Value of Information methods in this area need to take the ongoing debate on decision-making frameworks for rare disease treatments into account.

References

http://www.eurordis.org/ Cheng, Y., Fusheng, S., and Berry, D. A. (2003). Choosing sample size for a clinical trial using decision analysis. Biometrika 90, 923 Hee, S. W., Hamborg, T., Day, S., Miller, F., Madan, J., Posch, M., Zohar, S., and Stallard, N. (2015). Decision theoretic designs for small trials and pilot studies: a review. SMMR (online access) Stallard N, Miller D, Hee SW, Madan J, Zohar S and Posch M (2015). Determination of the optimal sample size for a clinical trial accounting for the population size. Submitted to Biometrics. Paulden, M., Stafinski, T., Menon, D., & McCabe, C. (2015). Value-Based Reimbursement Decisions for Orphan Drugs: A Scoping Review and Decision Framework. PharmacoEconomics, 33(3), 255-269.