eupha 2.lessons learned from published economic evaluations by_mathijsvandenberg
DESCRIPTION
Pre conference workshop Economic Evaluations of Public Health Interventions Amsterdam, EUPHA 2010 Public health economics was one of the themes of a pre conference at the 3rd European Public Health Conference in Amsterdam that took place from 10-13 November of 2010. Around 40 people participated at this pre conference. In four presentations the main topics in Public Health economics were introduced and illustrated. Economics is concerned with allocation of scarce resources in society over alternative uses. Some different types of evaluations were shown. The preference (utility) based health measure QALY (Quality Adjusted Life Years) was explained and discussed. In general methods for economic evaluations can be applied for evaluation of Public Health interventions. This was illustrated by a presentations on the economic impact of prevention strategies in tackling obesity. This study showed some good results in improving population health and decreasing health expenditure. However in many Public Health areas the effectiveness of public health interventions is still limited and should be assessed carefully concerning assumptions, costs calculated and models used. More attention should be paid to inter-sectoral effects, equity considerations and a societal perspective in performing economic evaluations. Finally the involvement of relevant stakeholders is key to the success of Prevention. The chair of this meeting concluded that Public Health and Economics could make a good couple. However for a longstanding relationship, we should put more effort in the evidence base of Public Health interventions. It is important that Public Health interventions demonstrate value for money!TRANSCRIPT
1 11 November 2010
Lessons learned from published economic evaluations in public health
Matthijs van den Berg
Johan Polder
Ardine de Wit
2 Lessons learned from published economic evaluations in public health | 11 November 2010
Content
1. Introduction
2. RIVM project
3. Cost-effectiveness of public health interventions
4. Critical issues in public health economic evaluations
Lessons learned from published economic evaluations in public health | 11 November 2010
3
Cost-effectiveness is quite the thing
● Science
– Exponential increase in cost-effectiveness studies
– Both absolute and relative
● Practice
● Policy
Lessons learned from published economic evaluations in public health | 11 November 2010
4
Exponential increase cost-effectiveness studies
0
10000
20000
30000
40000
50000
1950 1955 1960 1965 1970 1975 1980 1985 1990 1995 2000 2005
Lessons learned from published economic evaluations in public health | 11 November 2010
5
Exponential increase proportion c-e studies
0,0
0,1
0,1
0,2
0,2
0,3
0,3
1950 1955 1960 1965 1970 1975 1980 1985 1990 1995 2000 2005
Lessons learned from published economic evaluations in public health | 11 November 2010
6
Cost-effectiveness is quite the thing
● Science
● Practice
– Dutch manual for evidence-based guideline development:
› Cost-effectiveness should be one of the ‘other considerations’that should be taken into account before coming to recommendations (CBO, 2007)
– Guidelines more frequently use cost-effectiveness considerations
› Cardiovascular risk management
● Policy
Lessons learned from published economic evaluations in public health | 11 November 2010
7
Dutch guideline ‘Cardiovascular risk management’
● Guideline:
– CVD 10 year mortality risk > 5%: lifestyle advice
– CVD 10 year mortality risk >10%: pharmacological treatment
● Recommendations are based on analyses of cost-effectiveness and budget impact
● The threshold for cost-effectiveness is set on € 20.000 per QALY gained, which corresponds with the value of ƒ 40.000 in the 1998-guideline, corrected for inflation
Lessons learned from published economic evaluations in public health | 11 November 2010
8
Cost-effectiveness is quite the thing
● Science
● Practice
● Policy
– Dutch policy document on public health: ‘Being healthy, and staying healthy; a vision of health and prevention’
– Health care insurance board: judgement framework for reimbursement decisions on the basic health care package
Lessons learned from published economic evaluations in public health | 11 November 2010
9
● One of the main principles in public health policy making
– “Forms of preventive intervention that do not work will be stopped,…
– and promising forms of intervention will be investigated to establish how (cost-)effective they are....
– Forms of intervention that are known to be (cost-)effective should, in principle, be funded through the appropriate insurance systems, subject to the usual budgetary constraints.”
Lessons learned from published economic evaluations in public health | 11 November 2010
10
● Dutch Health Insurance Board uses a judgement framework for which interventions to cover, consisting of 4 criteria:
– Necessity
– Effectiveness
– Cost-effectiveness
– Feasibility
● Health Council of the Netherlands uses frameworks for decisions on new vaccinations and screenings
– Cost-effectiveness is a main assessment criterion in both frameworks
Lessons learned from published economic evaluations in public health | 11 November 2010
11
Take-home message
● Cost-effectiveness is hot, especially in public health, but don’t take the result of a cost-effectiveness analysis (i.e. an ICER) for granted
● Read cost-effectiveness analyses of preventive interventions very carefully and ask yourself questions like:
– Are the assumptions on effectiveness well-founded and realistic?
– Which costs are included and which are ignored?
– What are critical choices in the model that was used?
Lessons learned from published economic evaluations in public health | 11 November 2010
12
RIVM project ‘Cost-effectiveness of prevention’
● Aim of the project
– To identify potentially cost-effective preventive interventions that have not yet been systematically implemented in the Netherlands.
– i.e. to find opportunities for public health policy
● Basic idea of the project:
– Systematic and frequent literature search
– Read and appraise the cost-effectiveness analyses
– Write a report or a paper
Lessons learned from published economic evaluations in public health | 11 November 2010
13
Lessons learned from published economic evaluations in public health | 11 November 2010
14
Lessons learned from published economic evaluations in public health | 11 November 2010
15
Number and type of prevention-CEAs
● In 2008 we found over 230 papers studying the cost-effectiveness of a preventive intervention or program
● About half of the included studies used QALY’s of DALY’s as outcome measure
Lessons learned from published economic evaluations in public health | 11 November 2010
16
What diseases do prevention-CEAs focus on?
– Infectious diseases: 32%
– Cancers: 21%
– Cardiovascular diseases: 10%
– Mental- and behaviour diseases: 7%
– Musculoskeletal system: 7%
– Rest 23%
● Vaccination gets disproportionally much attention in cost-effectiveness analyses relative to the burden of infectious diseases
● Mental- and behavioural diseases cause large burden of disease, while they were the subject of only 7% of included studies
Lessons learned from published economic evaluations in public health | 11 November 2010
17
How cost-effective were the studied interventions?
>80% lower than 50,000/(QA)LY
Median: €12,000/(QA)LY
● What do you conclude from this chart?
Lessons learned from published economic evaluations in public health | 11 November 2010
18
How cost-effective were the studied interventions?
● Almost all preventive interventions have favourable cost-effectiveness ratios
– Is prevention always cost-effective?
– Is there a publication bias?
– Do researchers work up to favourable ICERs?
Lessons learned from published economic evaluations in public health | 11 November 2010
19
Some critical comments
● Based on observations in economic evaluations of public health interventions
● Three categories
– Costs (numerator of the ICER)
– Effects (denominator of the ICER)
– Models
Lessons learned from published economic evaluations in public health | 11 November 2010
20
Costs
● Which costs are included?
– Health care perspective (only health care costs and health benefits) or societal perspective (all costs and all benefits)
– The societal perspective is preferred
– Public health measures have impact outside the healthcare sector
– Most CEA’s on prevention use the health care perspective, ignoring very relevant cost categories
Lessons learned from published economic evaluations in public health | 11 November 2010
21
Effects
● Is the intervention scenario realistic?
– What is the reach of the intervention? What participation rate is assumed?
– Case: colorectal cancer screening using colonoscopy
Lessons learned from published economic evaluations in public health | 11 November 2010
22
•Unpleasant bowel
preparation
•Uncomfortable
proceduce
•Risk of bowel
perforation
Lessons learned from published economic evaluations in public health | 11 November 2010
23
Participation rate
● Is the intervention scenario realistic?
– What is the reach of the intervention? What participation rate is assumed?
– Case: colorectal cancer screening using colonoscopy
– How many people would participate in such a screening?
Lessons learned from published economic evaluations in public health | 11 November 2010
24
Lessons learned from published economic evaluations in public health | 11 November 2010
25
Participation rate
● Systematic review of cost-effectiveness analyses of colonoscopy screening
– Model assumptions on uptake: ~85%
– Real life data for participation: ~30%
● Conclusion:
– In many CEAs model assumptions on screening uptake were more positive than real life data suggest
Lessons learned from published economic evaluations in public health | 11 November 2010
26
Adherence
Lessons learned from published economic evaluations in public health | 11 November 2010
27
Adherence
● Systematic review of medication CEAs found that many studies used adherence rates from clinical trials which notoriously overestimate adherence found in the general population
● However in a small subset of studies that varied adherence in sensitivity analyses, the impact on the cost-effectiveness ratio was substantial
● With policymakers increasingly turning to these studies for guidance, failure to account for nonadherence may lead to suboptimal resource allocation strategies
Lessons learned from published economic evaluations in public health | 11 November 2010
28
Uncertainty in effectiveness
Lessons learned from published economic evaluations in public health | 11 November 2010
29
Uncertainty in effectiveness
● “The aim of this paper was to determine the cost-effectiveness of a Dutch school-based smoking education program.
● The incremental cost-effectiveness ratio of the school program was estimated at €19,900 per quality adjusted life year gained.
● Main problem in estimating the cost-effectiveness was the lack of proper effectiveness data on daily smokers among adolescents.”
Lessons learned from published economic evaluations in public health | 11 November 2010
30
Lessons learned from published economic evaluations in public health | 11 November 2010
31
Long term effects
Lessons learned from published economic evaluations in public health | 11 November 2010
32
Long term effects
Lessons learned from published economic evaluations in public health | 11 November 2010
33
Lessons learned from published economic evaluations in public health | 11 November 2010
34
● “The range of effectiveness of the vaccine was considered between30% en 90% with a positive baseline at 80%.”
● “The costs of the vaccine was assumed to be $450 for three doses and ranged from $300 to $2000.”
● “Vaccination was found to be potentially cost-effective with an ICER of -$2384 relative to standard treatment.”
› However, such a vaccine is not available at the moment
› This is a purely hypothetical excercise
Lessons learned from published economic evaluations in public health | 11 November 2010
35
Models
●Time horizon
–Many modelling studies on the cost-effectiveness of preventive interventions have very long time horizons (E.g. 80 yrs, 100 yrs, lifetime)
–Necessary to capture all health benefits (e.g. stop smoking interventions and the prevention of lung cancer)
–Shorter time horizons would make the ICERs of many preventive interventions very unfavourable
Lessons learned from published economic evaluations in public health | 11 November 2010
36
Models
Lessons learned from published economic evaluations in public health | 11 November 2010
37
Models
●Discount rates
– Most international CEAs use same discount rate for both costs and effects (e.g. 4% or even 6%)
– Dutch guidelines prescribe different discount rates for costs and effects (4% and 1.5%)
– This has a large influence on the ICER, making prevention a lot more cost-effective
Lessons learned from published economic evaluations in public health | 11 November 2010
38
●Case: cost-effectiveness of HPV vaccination
– Dozens of recently published CEAs
– Including several Dutch model studies
– Most of these conclude favourable cost-effectiveness
– However, de Kok et al., 2009 does not
Lessons learned from published economic evaluations in public health | 11 November 2010
39
De Kok et al.: “In conclusion, many uncertainties still exist about the effects of HPV vaccination on HPV-related diseases. Our cost-effectiveness analysis shows that in the Netherlands, a country with low cervical cancer incidence and mortality, HPV vaccination is not cost-effective […].”
Lessons learned from published economic evaluations in public health | 11 November 2010
40
Coupé et al.: “In a sensitivity analysis, de Kok et al. reported an ICER of € 19 700 per QALY at discount rates of 4% for costs and 1.5% for effects. The difference in reported ICERs betweenthe two studies can thus be largely explained by the discount rates that were used.
Lessons learned from published economic evaluations in public health | 11 November 2010
41
Postma: “I conclude that the conclusion by de Kok et al. is misleading, should be re-visited and would probably better be formulated as “In the Netherlands, HPV vaccination is likely to be cost-effective if compared with screening alone”, […]”
Lessons learned from published economic evaluations in public health | 11 November 2010
42
●Moreover
– Assumptions concerning participation rate in HPV vaccination were about 85%
– While the real participation rate in the Netherlands appeared to be about 50%
– In what way would that influence the cost-effectiveness ratio?
Lessons learned from published economic evaluations in public health | 11 November 2010
43
Another example: rotavirus vaccination
Lessons learned from published economic evaluations in public health | 11 November 2010
44
Lessons learned from published economic evaluations in public health | 11 November 2010
45
Lessons learned from published economic evaluations in public health | 11 November 2010
46
Lessons learned from published economic evaluations in public health | 11 November 2010
47
● “The intervention had a gross incremental cost-effectiveness ratio of AUD$ 3.70 (95% uncertainty interval (UI) $2.40, $7.70) per DALY.”
● “When the present value of potential savings in future health-care costs was considered, the intervention was ‘dominant’ […].”
● “Restricting televised advertisements targeting children couldpotentially be one of the most cost-effective population-basedobesity prevention interventions available to governmentstoday.The ICER was most sensitive to effectiveness assumptions.”
● “Limited evidence of the effectiveness of this intervention was a keyconcern”
Lessons learned from published economic evaluations in public health | 11 November 2010
48
● “The most relevant study to assess the effectiveness of a reductionin advertising was a randomized controlled trial that compared the impact of exposure to different food advertisements on food and beverage selections in 288 5- to 8-year old children on holidaycamp for a period of 2 weeks in Quebec in 1982.
● The children were exposed to sweet commercials, no commercials, fruit commercials or nutritional public service announcements. Children who viewed sweet commercials chose (and ate) significantly more sweets over fruit as snacks compared withchildren in the other three groups (75 versus 67%, 64 and 65%, respectively).”
● “Whether te reduction in BMI would be maintained over the lifetimeof the child is unknown and difficult to predict.”
Lessons learned from published economic evaluations in public health | 11 November 2010
49
Conclusions
●Don’t take the result of a cost-effectiveness analysis (i.e. an ICER) for granted
●Please read cost-effectiveness analyses of preventive interventions very carefully
– Are the effect assumptions realistic?
– Are all relevant costs included?
– What are critical model choices?