Economic Evaluation of Public Health Interventions: Challenges &

Recommendations

Helen Weatherly

University of York, UK

Elements of my presentation

• Undertaking economic evaluations of public health interventions

– Challenges for analysts and options for dealing with the challenges

• Using economic evaluations of public health interventions

– Challenges for decision makers and options for dealing with the challenges

Challenges for analysts in the economic evaluation of public health interventions

• Review of the literature

• Focus on four methodological challenges

• Options for dealing with the challenges

Review of the literature

• NHS EED online database

• Free access

• Summaries of economic evaluations studies by health economists

• Papers published 2000 – 2005

• 1,264 studies found

• 154 relevant studies

Public Health Areas

0%

2%

4%

6%

8%

10%

12%

14%

16%

18%

20%

Accidents HIV/AIDS Low birth weight

Smoking Alcohol Sexually transmitted infections

Ante & post natal

visiting

Teenage pregnancy

Drug abuse Obesity & physical activity

Youth suicide

prevention

Public health area

% of

studies

Types of Economic Evaluation

Cost consequence analysis (CCA)

36%

Cost-effectiveness analysis (CEA)

37%

Cost-utility analysis (CUA)27%

Economic Evaluation Setting

0

5

10

15

20

25

30

35

40

45

50

Community Medical Home Workplace School Transport Prison

%

Setting

Target Group

General population35%

Working age population27%

Children14%

Women10%

Older people10%

Ethnic minority3%

Men1%

Study Perspective

Not stated24%

Societal31%

Heatlh service, health care payer, third party

payer32%

Multiple3%

Patient1%

Government1%

Hospital, local, health

department, provider

8%

Types of Costs

0

20

40

60

80

100

120

140

160

180

Analytical Challenges Posed by the Evaluation of Public Health

Interventions

• Estimates of relative treatment (programme) effects

• Measurement and valuation of outcomes

• Equity

• Inter-sectoral effects

Estimates of Relative Treatment (Programme) Effects

• Aim to obtain unbiased estimates of effect

• Focus on populations/communities rather than specific individuals

• Preference for Randomised Controlled Trials (RCTs)

– In practice, few RCTs

– RCT measured outcomes tend to be short term. Benefits from public health interventions may be realised over the longer term

Measurement and Valuation of Outcomes

• Requires measuring and valuing outcomes over the longer term

• QALYs increasingly popular

• Are other outcomes relevant to public health interventions?

– Effects beyond target individuals/groups

– Non-health benefits

• Consider which outcomes measurement and valuation methods are valid

Equity Considerations

• Growing literature, although not generally applied in the context of economic evaluations

• Standard economic evaluations focus on efficiency (i.e. health maximisation) rather than on equity (i.e. the distribution of health gains)

Inter-Sectoral Effects

• Current literature does not cast the net very broadly

• Evaluations of PH interventions need to consider both public and private costs as effects of public health interventions are often wide-ranging

• Need to explore the ‘ripple effects’ (e.g. bicycle helmets)

Recommendations

• Attribution of outcomes

• Measuring and valuing outcomes

• Equity considerations

• Inter-sectoral costs and consequences

Attribution of Outcomes

• Conduct RCTs. If cannot be undertaken, fill gaps using natural experiments & non-experimental data

• Match outcomes in RCTs with those available in long-term observational studies

• Build on quantitative methods for data synthesis and extrapolation

- extending time frames

- combination of effects from different designs

- econometric modelling

Measuring & Valuing Outcomes

• Debate about theoretical & value propositions underlying various forms of economic evaluation

- money valuation

- superQALY

- cross-sector compensation test

• Perform a cost-consequences analysis alongside

• Continue research on developing a more general measure of well-being

Equity Considerations

• Narrative discussion

• Health inequality impact assessment

– Impact of an intervention on health inequality

• Opportunity cost analysis of equity

– Where the most cost-effective option is judged inequitable, calculate the opportunity cost of not selecting that option

• Equity weighting of health outcomes

– Is a QALY is a QALY is a QALY?

• Conduct pilot studies on methods for incorporating equity concerns

Inter-sectoral Costs & Consequences

• Quantify inter-sectoral impacts of interventions in a CCA. Could assess need for budgetary transfers

• Consider broader impact of public health interventions on the voluntary sector & private individuals

• Conduct an analysis of costs & consequences by beneficiary group, e.g. by health status etc

• Assess whether a general equilibrium approach would be more suitable for the evaluation of PH interventions

Use of economic evaluations of public health evaluations by decision makers:

Challenges & recommendations

• Decision makers objectives

– Comparing public health interventions with standard health care interventions

• Motivating and sustaining behavioural change

• Enhancing transferability of studies

Adapted from: Guide to NICE methods, NICE 2008

Element of Assessment NICE HTA reference case NICE public health reference case

Defining decision problem

The scope developed by NICE The scope developed by NICE

Comparator Therapies routinely used in NHS Therapies routinely used in public sector

Perspective on costs NHS and Personal Social Services (PSS) Public sector, including the NHS & PSS

Perspective on effects All health effects on individuals All health effects on individuals

Type of economic evaluation

Cost-effectiveness analysis (CEA) Primary analysis CEASecondary analysis CCA, CBA

Synthesis of evidence on outcomes

Based on a systematic review Based on a systematic review

Measure of health effects

Quality-adjusted Life Years (QALYs) QALYs

Source of data for measurement of HRQoL

Reported directly by patients &/or carers

Reported directly by patients &/or carers

Source of preference data for valuation of changes in HRQoL

Representative sample of the public Representative sample of the public

Discount rate Annual rate 3.5%, costs & health effects

Annual rate 3.5%, costs & health effects

Equity position Additional QALY same weight regardless of other characteristics of individuals receiving health benefit

Additional QALY same weight regardless of other characteristics of individuals receiving health benefit

Motivating and sustaining behavioural change

• Beyond neoclassical economics

– Overcoming market failure

• Implementing public health interventions

– Targeting provision

• Concerns about the distribution of health

• Use of e.g. tariffs to reward implementation

– Incentives for public health and individual behaviour

• E.g. Nudge economics

Transferability of studies

• CE studies of public health interventions are rare compared to evaluations of standard care

• Maximise use of the few studies available

– Comparability of methods for economic evaluation of public health interventions vs. those for standard care

– Encourage transparent, full, consistent approaches to reporting of study methods including quantifying uncertainty

• Reference case

– Support research to improve methods of assessment of PH interventions

– Incentives to improve data availability

Summary (i)

• In principle the general methods of economic evaluation can be applied to public health interventions

• The current literature is disappointing and represents many missed opportunities

• Efforts need to be made in improving the effectiveness evidence base, through RCTs and observational studies

• Economic evaluations in this area need to pay a lot more attention to inter-sectoral effects and equity considerations

Summary (ii)

• Prior to using the studies, decision makers need to identify their goals

• Stakeholder involvement. Incentives to engage in the process

• Transferability issues

– Suitability for use in different settings/commissioning tasks

– Complexities inherent in each study, quantification of uncertainty

– Communicating accessible information to users

– Role of technical analysts in use of studies

References

• Cookson R, Drummond M, Weatherly H. Explicit incorporation of equity considerations into economic evaluation of public health interventions. Health Economics Policy and Law. 2009;4(2):231-45.

• Marks L, Cave S, Hunter D, Mason J, Peckham S, Wallace A, Mason A, Weatherly H, Melvin K. Public health governance and primary care delivery: a triangulated study. Report prepared for the NIHR Service Delivery and Organisation Programme, 2010.

• Neumann PJ, Jacobson PD, Palmer JA. Measuring the value of public health systems: The disconnect between health economists and public health practitioners. American Journal of Public Health 2008, 98, 12, 2173-2180.

• Phillips C, Fordham R, Marsh K et al. Exploring the role of economics in prioritization in public health: What do stakeholders think? The European Journal of Public Health 2010.

• Weatherly H, Drummond M, Claxton K, Cookson R, Ferguson B, Godfrey C, Sculpher M, et al. Methods for assessing the cost-effectiveness of public health

interventions: key challenges and recommendations. Health Policy 2009;93:85-92.