an exercise on the feasibility of carrying out secondary economic analyses

HEALTH ECONOMICS, VOL. 5: 155- 165 (1996)

ECONOMIC EVALUATION

AN EXERCISE ON THE FEASIBILITY OF CARRYING OUT SECONDARY ECONOMIC ANALYSES

TOM JEFFERSON’, MIRANDA MUGFORD2, ALASTAIR GRAY AND VITTORIO DEMICHEL14 ’Army Medical Directorate, Ministry of Defence;

National Perinatal Epidemiology Unit, Oxford; Centre for Socio-legal Studies, WoIfson College, Oxford; Institute of Medical Statistics, University of Pavia, Italy.

SUMMARY

Purchasers of health services need up to date information on cost-effectiveness of interventions to help in prioritising spending. But economists have not yet developed a formal methodology for reviewing and summing up evidence from individual economic evaluations which may have been conducted at different times and in different places, or indeed for assessing whether such systematic reviews are possible in this context.

This paper discusses the problems of reviewing available economic information, using a body of literature on the economics of influenza vaccination to illustrate some relevant issues. First, the paper examines alternative methods for adjusting prices to take into account differences in currency and time periods: Retail Price Indices are compared with health specific inflation indices, and exchange rates are compared with Purchasing Power Parity (PPP) data. For the studies reviewed, the choice of conversion method made little practical difference.

Secondly, the paper explores the possibility of summarising the results of a review in terms of quantities of resources used, rather than prices. This method is constrained by the available data, but could, be more generally useful as it allows direct Comparison of underlying technologies, and calculation of costs by attaching local unit costs to the resources associated with an intervention. These two exercises highlight many of the problems that arise in generalising from economic studies. Both methods need to be developed further if they are to be useful to decision makers.

KEY WORDS-secondary economic evaluations; resource allocation

INTRODUCTION

With increasing frequency, agencies purchasing health services can use the results of economic analyses to aid decision-making concerning resource allocation. This is reflected in a dramatic increase in studies wholly or partly containing economic evaluations. ’ As the number of evaluations grows, so does the need to consider their evidence systematically, especially in areas where a number of studies may have been performed.

Although the methods of systematic review have increasingly been applied to the epidemiological literature on the effectiveness of alternative

interventions, so far the process of reviewing and summing up economic evidence has been little developed, despite the considerable number of studies available. It is not clear whether ‘systematic reviews’ akin to those performed on epidemiological literature can be applied to economic evaluations. First, substantial methodological variations have been demonstrated between economic evaluations that are superficially comparable, raising questions about the reliability of cost- effectiveness ‘league tables’ and other comparative devices.*-* Second, doubts have been expressed about the theoretical basis for transferring results from the setting in which an existing study was

Address for correspondence: Dr. T. 0. Jefferson, Keogh Barracks, Ash Vale, Hants GU12 5RR, UK, Tel 01252 340355; Fax 01252 340 348

CCC 1057-92301961020155-1 1 0 1996 by John Wiley & Sons, Ltd.

Received 17 October 1994 Accepted 26 January I996

156 T. JEFFERSON ET AL

performed into a decision-making process in another setting.’ Third, it is not clear what techni- cal options exist for summarising and transfemng results from a number of economic evaluations.

Despite these concerns, the pressure to collate, summarise and interpret existing studies continues, especially where decision-makers may lack the resources to commission full economic evaluations that are context-specific. ’O If feasible, these ‘secondary’ analyses of existing economic evaluations would have three advantages. First, they would condense and present in a readily under- standable format large amounts of critical information which are dispersed in numerous published and unpublished sources. Secondly, reviews are a vehicle for updating the state of knowledge in a particular field and provide a stimulus for increasing methodological quality. Thirdly, they may be cheaper than repeating studies.

This paper explores the possibility of developing a methodology to sum up evidence about cost and cost-effectiveness from pre-existing work, and thereby to perform ‘secondary’ economic evaluations. It focuses on two main issues. First, it considers whether, when comparing different studies performed at different times and places, the methods used for converting cost data into a standard currency make much difference to the conclusions of the analysis, in the sense of affect- ing the relative position of cost estimates from the different studies. Second, it explores whether studies give enough detail about quantities of resource inputs associated with an intervention to allow these to be compared directly, and the costs of care in different settings to be calculated from the resource data using a single set of unit prices. Both questions are explored with reference to a systematic review of economic evaluations of vaccination against influenza A and B. This area was chosen because it is a complex and recurring public health problem with potentially significant resource implications in industrialised nations.

A greatly simplified example may help to illustrate some of these issues: suppose an American Congressional Committee is considering an influenza immunization programme in 1990, and the only available economic evaluation of Influenza immunization was performed in Russia in 1980. It reported a vaccine cost of 6 roubles in 1980 prices, and that immunizing a cohort of 10,OOO people would result in 1500 fewer physician consultations at 20 roubles per consultation, 1

hospital admission of 20 days at 100 roubles per day, and 1500 working days lost at 50 roubles per day. Hence costs=60,000 roubles, and benefits = (3000 + 2000 + 75,000) 80,000 roubles, so that the ratio of benefits to costs is = 1.3 : 1. If the dollar-rouble exchange rate was parity in 1980 and health sector prices in the USA rose by 80% during the 1980s, a straight translation of this study would give costs of $108,000 and benefits of $144,OOO, with the benefit-cost ratio remaining at 1.3 : 1. However, if it was known that a vaccine dose cost $12 in the USA in 1990 prices, that a physician consultation was $50 per consultation, a hospital admission was $300 per day, and a working days lost was equivalent to $100, then costs would equal $120,000, and benefits would equal (75,000 + 6,000 + 150,000 = ) $231,000, so that the ratio of benefits to costs would rise to 1.9 : 1.

These two approaches can be located within the general framework of cost and production functions in health care. By focusing on the reported costs of production derived from local relative prices in a specified setting, and then translating these costs to a different setting using some shadow exchange rate or conversion factor, the former approach explicitly recognises that relative prices will affect the particular configuration of factor inputs adopted; however, it may be highly dependent on the appropriateness of the conversion factor used. By focusing on the observed combination of factor inputs associated with a particular intervention in a specified time and place, and then attaching to these factor inputs a different set of relative prices reflecting local conditions, the latter approach avoids the necessity of using an exchange rate or purchasing power parity which may not accurately measure the social opportunity cost of the particular bundle of factor inputs in the new setting, but ignores the possibility that the prevailing relative prices in the new location might have altered the factor mix.

These problems have been identified and dis- cussed extensively in other areas, such as project appraisal in developing countries,” and it is important to note that either procedure could contradict the first or second order conditions that would theoretically be required for profit maximis- ation in a comparative static analysis under conditions of perfect competition: namely, that each input is utilised up to a point at which the value of the marginal product is equal to its price, and that marginal increases in either of a range of inputs yield diminishing net returns. However,

FEASIBILITY OF SECONDARY ECONOMIC ANALYSES 157

this should be viewed in the context of the uncertainty which prevails over specifying the objective function of health providers or empirically esti- mating production or cost functions in health care in different settings.13

METHODS

Literature review The systematic review of all economic evalua-

tions of vaccination against influenza began with a MEDLINE CD-ROM search in all European languages since 1966, using the following key words to identify studies: influenza, prevention of, cost-benefit , cost-effectiveness, economic evaluation, cost. All bibliographies of retrieved studies were searched, and the manufacturers of the vaccine and researchers active in the field of the economics of prevention of communicable dis- eases were questioned in an effort to extend search coverage.

In order to derive economic data only from methodologically valid studies a check-list of entry criteria to the secondary analysis was used. The criteria were aimed at separating reliable from unreliable studies and were proposed in a previously published overview of economic evaluations of Hepatitis B vaccination.8 These were as follows: 1.

2.

3. 4.

5.

The Study should have a clear aim, viewpoint and timespan; The design should be consistent with study aim(s) and coherent methods, results and conclusions; Costs should be itemised; Marginal and sensitivity analyses should have been carried out; All the above should have been clearly and unequivocally stated in the text.

The application of discounting was not included as a criterion in this study, as the costs and benefits of influenza prevention accrue in a short timespan due to the yearly antigenic drift of the causative agent (drift is a minimal yearly change in viral antigenic configuration which limits specificity and hence probably the efficacy of each new vaccine). The criteria for including studies had the effect of reducing the numbers of observations in the review: quantity of data was sacrificed in favour or improved reliability and consistency.

All the studies included in the secondary analysis were classified by type of economic evaluation, using definitions current in standard texts.14 In order not to bias cost estimation by double counting (or over-representation of the same study), studies which re-used resource estimates derived from other previously published articles were eliminated from the analysis. Cost estimates were stratified by dividing the reference populations into: elderly (aged more than 65), working age (aged 18 to 60) and general (all ages). Where evaluations presented estimates both for general and working age populations, only one set of estimates was used. For each population group the benefit-cost ratios were listed and the distribution of the estimates was identified: the 25th, median and 75th percentile values were described as ‘low’, ‘medium’ and ‘high’ estimates. All variables to which the primary models were sensitive were identified.

Price standardisation All costs quoted in the studies for resource

inputs and results were extracted and standardized to 1987 US dollars. Converting health service costs over space or time into another price basis is a procedure which relies on a number of assumptions: for instance that the underlying technology or relative factor prices remain broadly constant. These assumptions are unlikely to hold in a wider economy over a period of time, or between economies at different levels of development. However, the next and perhaps more important question is how significant any deviations from these assumptions are likely to be in practice, that is, in terms of the interpretations one can put on the findings within a narrow sector of the economy in a limited time period, or between similar economies. This is a question which can only be examined and resolved empirically.

In this study, two methods for standardising costs were compared to see whether they resulted in significantly different results: in the first method, official exchange rates were used to translate costs into US dollars, and the US Con- sumer Price Index was used to place these costs on a 1987 basis; in the second method, health specific price indices for each country of study were used to place costs on a 1987 basis, and health specific Purchasing Power Parities (PPP) were used to translate costs into US d o i i a r ~ . ~ ~ . ~ ~

The rationale for exploring alternative methods of standardising prices was as follows: retail price


indices or a GDP deflator may not accurately measure changes in a sub-sector of the economy such as the health sector, and in general the preferred method of recalculating prices in the health sector-for example, in an economic evaluation -is by deflating or inflating them using a health specific price index. However, it is not clear in practice what difference this choice makes. Similarly, the most accessible mechanism for converting prices between national currencies is the official exchange rate prevailing between two currencies at a point in time, but these reflect many things other than purchasing power parities. Medi- cal care-specific PPPs would be a preferred method, assuming they were available and reliable. Then, if a British study yields a cost per life year gained of 10,OOO UK pounds in 1985 prices for treatment x, and the objective is to express this in terms of US dollars in 1987 prices, the procedure would be to convert pounds to dollars using the relevant medical care-specific PPP, then inflate to 1987 prices using the US medical care-specific price index.

Medical care-specific PPPs have previously been used to make international com arisons of aggregate real health expenditures. 17tiB)They have been criticised on the grounds that a relatively small number of items are used to calculate the health services component of medical PPPs. It also has to be assumed that the aggregate PPP is appropriate for a specific intervention.

To gauge the differences that may exist between these price conversion methods when used in the context of a secondary evaluation, all costs quoted in the studies were converted into a reference currency and year using both methods, and the results were compared.

The resource comparison approach To extract information on the quantities of

resources relating to direct and indirect benefits, one of the authors (VD) reviewed the methods and results sections of all studies included in the influenza vaccination database. Studies were included only if they contained clear statements concerning resource estimates which had subse- quently been used and costed in the relevant studies. Authors were not asked to provide sup- plementary information, as the aim of the study was to ascertain what can be generalised in a secondary evaluation from published or available primary research.

Once a range of resource estimates had been identified from the systematic review, a standard set of unit costs was attached to these resource volumes, the standard costs being derived from a cost-of-illness study that two of the authors (TOJ and VD) have recently carried out in the Emilia region of Italy. The results were then used in a series of evaluations performed on a simulated general population cohort of 1,000,000 individuals. The efficacy of the vaccine was held at 80% through- out, based on estimates derived from primary literature. The benefit to cost ratios (BCRs) arising from the different resource estimates were ranked, and variation around the median was measured in terms of the 25th and 75% centiles. Benefit to cost ratios at high and low attack rates (100 and 250 per lo00 population) were also examined.

RESULTS

Literature search A total of 28 studies that could be defined as

economic evaluations of vaccination against influenza were identified, of which 15 were excluded on the basis of the criteria applied. Of the 13 studies included, 4 were published or carried out in the period 1990 to the present, 4 in the period 1985-89, 2 were published between 1980-84, and 3 were published before 1980. The 13 included studies related to a total of 6 countries, 8 from America and one each from 5 other countries. All the identified studies are listed in the Appendix, which also gives information on whether they were included or excluded and, if excluded, on what grounds.

Price standardisation Before comparing alternative methods of

standardising prices to a common currency and year, all key variables of interest were collated from the identified studies to assess how much inter-study range existed. These data are reported in Tables 1-3, which deal respectively with studies of the general population, of the elderly and of the population of working age (18-60). The 75th percentile, median and 25th percentile values are reported as High, Medium and Low estimates. Thus in Table 1, of the six studies which reported a baseline influenza incidence rate on which to perform an economic evaluation, the


Table 1. Distribution of estimates from 6 economic evaluations of immunization for influenza in the general population (monetary estimates adjusted to 1993 US dollars using exchange rates and the US CPI)

Variable Number of High Low

observations (75th percentile) Medium (25th percentile)

Incidence per 100,000 at risk Effectiveness (Percent) Direct cost per case ($1993) Indirect cost per case ($1993) Vaccine cost per dose ($1993) Cost per healthy life-year gained ($1993) Cost per avoided case ($1993) Benefit-Cost Ratio

6 5 4 4 4 3

2 -

26,600 70 60

225 17

462 -

5.1

13,800 60 15 72 14

136 - -

12,982 46 14.6 6.3 6

135 - 5.0

High estimate (corresponding to the 75th percentile of the observations retrieved) was 26,600 per million population, and the Low estimate was 12,982 per million. Similar differences between studies can be observed for the assumed effectiveness of the vaccine, the direct and indirect costs per case reported, costs per vaccine dose, and the resulting costs per healthy life year gained or benefit-cost ratio. For the purposes of these descriptive tables costs are expressed in US 1993 $s, calculated on the basis of exchange rates and the US CPI.

Variation in these costs may indicate differences both in use of resources and in relative prices between resource inputs. So for instance, a high direct cost estimate in the elderly population is derived from Patriarca’s evaluation of care in private nursing homes in the USA and is con- stantly higher, even after price adjustment (Appendix A, reference 4).

Equally Sauras’s evaluation of the impact of influenza immunisation on a rural community in Spain (Appendix A, reference 12) is responsible

for the low direct and indirect cost per case estimates in Table 1. It is worth noting in passing that none of the economic evaluations of influenza vaccination in the elderly made estimates of indirect costs. This may be due to the current uncertainty over how to estimate production losses or their proxies in the elderly and minors.lg

As Tables 1-3 indicate, there is substantial variation between studies in the values attached to epidemiological and cost variables, and in the resulting benefit-cost ratios. However, as noted earlier, it is not clear that different price standardisation methods add significantly to this variation. In all, the studies included in the analysis provided a total of 33 cost estimates, and these could be translated from their year and country of origin into US $s using either a general retail price index and exchange rate, or health specific price indices and PPPs. When this was done, and the cost estimates were then ranked, the Spearman Rank Correlation between the two different methods was 0.933 (F‘ = 0.O00). Figure 1 shows a plot of these results, indicating that, in this context, the methodological

Table 2. Distribution of estimates from 3 economic evaluations of immunization for influenza in the elderly population (monetary estimates adjusted to 1993 US dollars using exchange rates and the US CPI)

Variable Number of High Low

observations (75th percentile) Medium (25th percentile)

Incidence per 100,000 at risk Effectiveness (Percent) Direct cost per case ($1993) Indirect cost per case ($1993) Vaccine cost per dose ($1993) Cost per healthy life-year gained ($1993) Cost per avoided case ($1993) Benefit-Cost ratio

16,000 70

984

11 133 132

-

-

- 70

160

5

28

-

-

9.1

3,000 60

145

4 126 27

-

-


Table 3. Distribution of estimates from 4 economic evaluations of immunization for influenza in the working age population (monetary estimates adjusted to 1993 US dollars using exchange rates and the US CPI)

LI1 Y 30 -- E Y = 25 0 u1

% g

3 15

5i *O

!i lo

$8 h m

Variable

rn m

8

m --

m

-- w

--

rn m

--

8

Number of observations

High (75th percentile) Medium

Incidence per 100,OOO at risk Effectiveness (Percent) Direct cost per case ($1993) Indirect cost per case ($1993) Vaccine cost per dose ($1993) Cost per healthy life-year gained ($1993) Cost per avoided case ($1993) Benefit-Cost Ratio

10,000 80 91 707 28 - - 16.7

2,500 60

114 22 133 82 1.3

-

Low (25th percentile)

3,000 30 46 83 3 - - 0.7

35 T

rn 0 I

0 5 10 15 20 25 30 35 51W7 RANK USING RPI AND EXCHANQE RATES

Figure 1. Rank correlation between 33 costs from different years and .countries, expressed in $1987 using a) CPI and exchange rates, and b) health specific price indices and PPPs

differences concerning choice of price index and currency convertor do not appear to have major empirical consequences. This result compares with a Spearman Rank Correlation of 0.858 between aggregate health expenditure in the 22 OECD countries converted using exchange rates and using health specific PPPs as reported by Gemitham and Jonsson. '*

Resource comparison Ten studies in the database contained extractable

these studies were the resource data for comparable items. Data that did not refer to adult populations were excluded, and a number of resource items were excluded on the grounds that they were only reported in one primary study: these included days of antibiotic treatment; diagnostic procedures during hospital stay or outpatient visits; hospitalisation for very high risk cases; loss of leisure time; school days lost for children and disability days.

Only four resource items were reported in at least two studies: number of consultations. num-

data on resource volumes associated with the interventions being evaluated, but in only six of

ber of hospital admissions, hospital length of stay and number of working days lost. In general point

FEASIBILITY OF SECONDARY ECONOMIC ANALYSES 161 Table 4. Resource estimates contained in 6 economic evaluations of influenza immunisation

Point estimates Range estimates

Resource item Minimum Maximum Minimum Maximum

Consultations per case 0.2 1.2 0.4 3.6 Hospital admissions per 100,000 population 0.12 0.12 0.012 0.073 Hospital length of stay per admission 9.0 10.0 5.4 12.0 Working days lost per case 2.8 10.0 3.2 3.4

0.2 -

estimates were reported more frequently than range estimates, and very few studies reported resource use by age group or by severity of case. All authors claimed that the resource estimates were derived from actual data gathered retrospectively. All data relating to length of stay and admission rates refer to admissions for pneumonia or pneumonia and influenza.

Table 4 summarises the variations in resource estimates extracted from the 6 studies. When point estimates were reported, the largest inter-study variations were observed around consultations (ranging from 0.2 to 1.2 per case) and working days lost (ranging from 2.8 to 10 days per case). When range values were reported, there was also substantial inter-study variation around consultations and hospital length of stay (ranging from 5.4 days to 12 days per admission).

Figures 2 and 3 show how benefit-cost ratios change with the different resource estimates and attack rates used. It can be seen that the number of doctor-patient consultations and work days lost do not appear to greatly affect the benefit-cost ratio, whereas length of hospital stay has more influence. Benefit-cost ratios range from 0.3 to 1.2 for the low attack rate scenarios and 1.1 to 3.6 for the high attack rate scenarios, and are thus significantly narrower than the ranges calculated when the benefit-cost ratios were compared using the price standardisation method. As one might expect, there appears to be an inverse linear relationship between estimated incidence/attack rates and benefit-cost ratios. This

LOW MEDIUM HIGH

1.2

% y 0.8 E a c W

rn 0.6

0.4


0

2 8 1 5 - - No. of consultations

r

I ~ LOW MEDIUM HIGH

0 1

Estimator of mource use

Figure 3. Changes in Benefit-Cost Ratios as resource consumption estimates vary: based on medium cost estimates and high influenza incidence estimates

relationship, which was not evident in the review of the economics of HB vaccination,* is reassur- ing as it supports the view that consistent methodology was used across the primary evaluations that satisfied OUT entry criteria.

DISCUSSION

This paper has identified and illustrated a number of issues which arise when attempting to perform ‘secondary’ analysis of existing economic evaluations. The first issue which needs to be tackled is that of terminology. Here, the initial definition of an ‘economic evaluation’ was broad, covering all studies which deal with the use of resources, whether self-standing or as part of a larger study such as a randomised control trial. Additionally, the term ‘secondary’ research is used here to mean the use of a systematic methodology for identify- ing, pooling and summing up evidence from previous evaluative studies. This approach draws on well-established methodological principles such as are laid out in Light and Pillemer’s seminal work on summarising available evidence.m

Light and Pillemer set out some an initial check- list before attempting a summing-up on any topic.

Firstly, the aim of the review should be defined. The aim will dictate the methods used in the review itself. If the objective is exploratory-that is, to explore what is known about the subject in question-then the inclusion criteria should be as comprehensive as possible. If, however, the review is ‘hypothesis-testing’ -that is, being used to address a pre-specified question-then stricter inclusion criteria are required.

Both approaches require a first stage, of identi- fying and gathering as many studies on the subject (overview) as possible. The second stage may then be either exploratory or hypothesis-testing. Attempts to generalise by applying the results of an overview from one context to another can be located in the latter category, and it is only in this context that we believe the term ‘secondary economic evaluation’ can be used.

An example of an exploratory review is the study of the international literature on the economics of Hepatitis B (HB) prevention,* where the ranges of costs and of HB incidences in different risk groups were plotted to describe variability.

This distinction between primary and secondary research in economic evaluation is particularly difficult, given the range of steps and diversity of sources of data required for a typical economic

FEASIBILRY OF SECONDARY ECONOMIC ANALYSES 163

evaluation. One finding of the present literature review is the infrequency with which authors describe precisely the research methods used for finding secondary data used in their studies. This also makes it difficult to test the validity of the studies, by source checking or by replication.

Despite these problems, the proportion of studies excluded on methodological grounds alone from this systematic review of economic evaluations on influenza immunization was lower than in the review of economic evaluations of Hepatitis B vaccination,* indicating that the general quality of economic literature on the prevention of influenza is comparatively good. All the economic evaluations found in the review reported favourable baseline benefit-cost ratios for vaccination against influenza, and hence at a first broad level of pooling it can be stated that there is general agree- ment in the available methodologically reliable literature that prevention of influenza A and B by vaccination is worthwhile, especially in high incidence scenarios (intra- or interepidemic).

Such a statement is however conditional on the level of clinical effectiveness of the vaccine. Vaccine effectiveness estimates used in the literature range from 30% to 80%, and the cost estimates rest upon these estimates. As no overview of effectiveness of influenza vaccination has been carried out, it must be concluded that the results of the economic literature, regardless of their quality, are only valid at the levels of clinical effectiveness assumed by the studies.

Regardless of how costs are standardised, substantial variability remains in costs and benefit- cost ratios, and an initial lesson from this exercise is that such variability should be reflected in the pooled estimates and not disguised in a single point estimate of overall cost-effectiveness or cost-benefit.

A number of methodological issues arise concerning the feasibility and validity of such secondary economic analyses. These include the specification of the health care context, quantities of inputs and variation in combinations of factor inputs, standardisation of price levels and currency, and technology levels. Despite genuine interest in the question,” such methodological difficulties appear up to now to have hindered systematic efforts to perform secondary evaluations. However, some work has been done in this field, as has been shown by Brown and Fintor in their overview of the economic evaluation of screening for breast cancer, in which the authors

reconciled differences in cost estimates in prior evaluations as part of a systematic review.’’ Their study however involved recalculating economic variables and even re-running the original compu- ter models with data from primary studies.

Systematic review in economic evaluation has not been widely used as a formal process, and it appears that funding agencies such as the UK Medical Research Council and NHS National Research and Development Programme are plac- ing increasing emphasis on the prospective study design, in which economic evaluations are performed alongside and as part of randomised controlled trials.” This may indeed be the best way forward-prospective economic studies will always give more control over the kinds of data collected and may also give economists influence over the definition of outcome measures, end- points, and other aspects of study design.

However, such prospective studies may be costly, or be insufficiently powered to show significant cost differences. Questions will continue to be asked concerning the transferability of their results to different geographical settings. Where several prospective studies of an intervention exist, the issues of whether or how their results may be combined will inevitably continue to arise.

The comparative neglect of secondary analyses of economic evaluations can be contrasted with the rapid growth of overviews or meta-analyses of clinical randomised control trials, which address similar issues, and have commanded widespread scientific acceptance. This may be due to the fact that effectiveness issues are seen as ‘hard’ or quantitative science whereas economic analysis is still seen as a mainly qualitative exercise, subject to too many assumptions and methodological uncertainties, and over-tolerant of opacity in reporting methods and results. No claim is being advanced here that a straightforward way has been found to sum up economic data from many sources. However, the present study has suggested that there is scope for the development of these methods in health economics. Progress may require a more coherent theoretical framework linked to cost and production function theory, as suggested in the introduction. The scope for summing-up economic evaluations could then be assessed in relation to this framework, while inclusion criteria could be tied to the increasingly explicit reporting guidelines now being urged for economic studies.

164 T. JEFFERS

REFERENCES

1. Elixhauser, A. (ed) Health care cost-benefit and cost-effectiveness analysis (CBA/EA). From 1979 to 1990: a bibliography. Medical Care 1993;

2. Gerard, K. Cost-utility in practice: a policy maker’s guide to the state of the art. Health Policy 1992;

3. Mason, J., Drummond, M. F. and Torrance, G. Some guidelines on the use of cost-effectiveness league tables. British Medical Journal 1993; 306 :

4. Drummond, M. F., Torrance, G. and Mason, J. Cost- effectiveness league tables: more harm than good? Social Science andktedicine 1993; 37: 33-40.

5. Udvarhelyi, S., Colditz, A., Rai, A. and Epstein, A M. Cost-effectiveness and cost-benefit analyses in the medical literature. Are methods being used correctly? Annals of Internal Medicine 1992: 116:

6. Adams, M. E., McCall, N. T., Gray, D. T., Orza, M. J. and Chalmers, T. C. Economic analysis in randomized controlled trials. Medical Care 1992;

7. Drummond, M. F., Brandt, A., Luce. B. and Rovira, J. Standardising methodologies for economic evaluation in health care. International Journal of Technology Assessment in Health Care

8. Jefferson, T. 0. and Demicheli, V. Is vaccination against Hepatitis B efficient? A review of world literature. Health Economics 1994; 3: 25-38.

9. Mooney, G. Key issues in health economics Lon- don: Harvester Wheatsheaf, 1994.

10. Mugford, M., Jefferson, T. 0. and McGuire, A. How can we generalize from the results of economic evaluations? Paper presented at the United Kingdom Health Economics Study Group 1994 (unpublished).

11. Little, I. M. D. and Mirrlees, J.A. Project appraisal and planning for developing countries. London: Heinemann, 1974.

12. Heathfield, D. F. and Wibe, S . An introduction to cost and production functions. London: Macmillan, 1987.

13. McGuire, A., Henderson, J. and Mooney, G. The economics of health care: an introductory text. London: Routledge Kegan Paul, 1988.

14. Drummond, M. F., Stoddart, G. L. and Torrance, G. W. Methods for the economic evaluation of health care programmes. Oxford: Oxford Univer- sity Press, 1987.

15. Blades, D. and Roberts, D. A Note on the New OECD Benchmark Purchasing Power Parities for 1985. OECD Economic Studies 1987: 183-197.

16. Ward, M. Purchasing power parities and real expenditures in the OECD. Paris: OECD, 1985.

~ ~ ( s u P P ) : JS1-149.

21: 249-79.

570-2.

238-44.

30: 231-38.

1993; 9: 26-36.

;ON ET AL

17. Parkin, D., McGuire, A. and Yule, B. Aggregate health care expenditures and national income: is health care a luxury good? Journal of Health Economics 1987; 6: 109-127.

18. Gerdtham, U.-G. and Jonsson, B. Conversion factor instability in international comparisons of health care expenditure. Journal of Health Economics 1991; 10: 227-234.

19. Koopmanschap, M. A. and Rutten, F. H. Indirect costs in economic studies. Confronting the confu- sion. Pharmacoeconomics 1993; 4: 446-54.

20. Light, R. J. and Pillemer, D. Summing up. The science of reviewing research. London: Harvard University Press, 1984.

21. Brown, M. L. and Fintor, L. Cost-effectiveness of breast cancer screening: preliminary results of a systematic review of the literature. Breast Cancer Research and Treatment 1993; 25: 113- 118.

22. Advisory Committee on Health Technology Assess- ment. Assessing the efects of health technologies: principles, practice and proposals. London: Department of Health, 1992.

22. Mason, J. and Drummond, M. Reporting guidelines for economic studies. Health Economics 1995; 4: 85-94.

APPENDIX A STUDIES IDENTIFIED IN THE INFLUENZA VACCINATION REVIEW

Included

1. %no, P. and Desfontaine, M. Vaccinations en Milieau Militaire. Etude cout/efficacite. 1993, unpublished.

2. Hocking, B. Benefits and costs of influenza immunization in the workforce. Asia Pacific Human Resource Management November 1989; 87-93.

3. Levy, E. and Levy, P. La vaccination contre la grippe des personnes d’age actif (25-64 ans): une etude cout- benefice. Revue Epidemiologique et Sante’ Publique 1992; 40: 285-295.

4. Patriarca, P. A., Arden, N. H., Koplan, J. P. and Goodman, R. A. Prevention and control of type A influenza infections in nursing homes. Benefits and costs of four approaches using vaccination and amantadine. Annals of Internal Medicine 1987;

5. Helliwel, B. E. and Drummond, M. F. The costs and benefits of preventing influenza in Ontario’s elderly. Canadian Journal of Public Health 1988;

6. Gavira, F. J. and Lardinois, R. Analisis de renta- bilidad de la vacunacion antigripal en una poblacion rural @a Victoria, Cordoba). Medicina Clinica (Barcelona) 1990; 94: 777-81.

7. Maucher, J. M. and Gambert, S . R. Cost-effective analysis of Influenza vaccination in the elderly. Age 1990; 13: 81-85.

107: 732-40.

79: 175-80.


8. Smith, J. W. G. and Pollard, R. Vaccination against Influenza: a five-year study in the Post Office. Journal of Hygiene 1979; 83: 157-80.

9. Riddiough, M. A., Sisk, J. E. and Bell, J. C. Influenza vaccination: cost-effectiveness and public policy. Journal of the American Medical Associa- tion 1983; 249: 3189-3195.

10. Office of Technology Assessment, US Congress. Cost-effectiveness of influenza vaccination. Wash- ington: Government Printing Office, 1982.

11. Kavet, J. A perspective on the significance of pandemic Influenza. American Journal of Public Health 1977; 67: 1063-70.

12. Sauras, I. A. Analisis economico del coste de la gripa aplicado a la provincia de Huesca. Revista Sanitaria Higiene Publica 1987; 61: 1017- 1028.

13. Koplin, A. N., Francis, B. J. et al. Administrative costs of the Influenza control program of 1976-1977 inIllinois. Medical Care 1979; 17: 201-09.

Excluded: 1. Perez-Tirse, J., Gross, P. A. Review of cost-

benefit analyses of influenza vaccine. Pharmacoe- conomics 1992; 2: 198-206. (Review).

2. Schoenbaum, S. C., McNeil, B. J. and Kavet, J. The swine-influenza decision. New England Journal of Medicine 1976; 295: 759-65. (Contains estimates from earlier study).

3. Anonymous. Cost-effectiveness of flu vaccine. Congressional Office of Technology Assessment. Connecticut Medicine 1982; 46. (Summary of reference already in the study).

4. Klarman, H. E. and Guzick, D. Economics of influenza. In Selby , P.(ed). Influenza: virus, vaccines and strategy. New York Academic Press Inc. 1976: 255-70. (Contains estimates from earlier study).

5. Evans, D. B., Hensley, M. J. and O’Connor, S . J. Influenza vaccination in Australia: a review of the economic evidence for policy recommendations. Medical Journal of Australia 1988; 149: 540-543. (Review).

6. Anonymous. Prophylaxis viewed economically. (Translated from German) Zahnartzliche mitteilun- gen 1974; 19: 997-98. (Presents no data).

7. Fedson, D. S. The influenza vaccination demonstra- tion project: an expanded policy goal. Infection Control and Hospital Epidemiology 1990; 11: 351 and following. (Contains estimates from earlier study).

8. Helcl, J., Gopfertova, D. and Heinz, F. Economic relevance of Influenza and other acute respiratory disease (translated from Czech). Czekoslovenka Epidemiologia. 1988; 6 : 340-49. (Viewpoint of eastern European economy).

9. Shevchenko, L. F., Sinyak, K. M., Sazhok, R. A., Stepanchuck, and Ryzhevskaya, N. I. Economic losses due to Influenza and other acute respiratory illnesses and possibilities of reducing them (translated from Russian). 1974, unpublished. (Viewpoint of eastern European economy).

10. Anonymous. Economic losses in Influenza Epidemics (translated from Czech). Czeck Ministry of Health 1982, unpublished. (Viewpoint of eastern European economy and presents insufficient data).

11. Elo, 0. Cost-benefit studies of vaccination in Finland. International Symposium on Immuniz- ation: Benefit versus risk factors, Brussels 1978. Development of Biological Standards 1979; 43:

12. Kouznetsov, V. S., Gouzyukina, T. V. and Orlova, E. R. L’efficence economique et medico-sociale des innovations et la pratique de la protection de la sante’. La Sanfe’ Publique 1990; 33: 11-16. (Viewpoint of eastern European economy).

13. Fedson D. Clinical practice and public policy for Influenza and Pneumococcal vaccination for the elderly. Clinics in Geriatric Medicine 1992; 8: 183- 199. (Contains estimates from earlier studies).

14. McBean, A. M., Babish, D. and Warren, J. L. The impact and cost of Influenza in the elderly. Archives of Internal Medicine 1993; 153: 2105- 11. (Descriptive study containing inconsist- ent cost methodology).

15. Fedson, D. S. Influenza and Pneumococcal vaccination: clinical efectiveness, value to society and variations between countries in public policies and clinical practice. 1991. (Unpublished review).

419-428.

an exercise on the feasibility of carrying out secondary economic analyses

Documents