1Danish Centre for Health Technology Assessment & 2EUnetHTA Secretariat, National Board of Health
6 April 2010
Prepared for the Working Conference ‘Health Services
Research in Europe’, Thursday 8 & Friday 9 April 2010
Health Technology Assessment 1Camilla Palmhøj Nielsen, 1Tina Maria Funch and 2Finn Børlum Kristensen
Draft report
Health Services Research and Health Technology Assessment – draft report
6 April 2010 2
Disclaimer: The research leading to these results has received funding from the European
Community's Seventh Framework Programme (FP7/2007-2013) under grant agreement
n°223248. Sole responsibility lies with the authors and the European Commission is not
responsible for any use that may be made of the information contained therein.
Health Services Research and Health Technology Assessment – draft report
6 April 2010 3
Contents
Health Technology Assessment ................................................................................................. 5
Introduction.................................................................................................................................. 5
Health technology assessment (HTA)................................................................................ 5
HTA and Health Services Research................................................................................... 5 Objective ............................................................................................................................. 6
Methods .............................................................................................................................. 6
Overview over the literature ........................................................................................................ 9
Existing Research....................................................................................................................... 15
The content of analysis in HTA................................................................................................. 15
Economic evaluation......................................................................................................... 16 Ethics................................................................................................................................. 17
Legal aspects.................................................................................................................... 18 Organisational aspects ..................................................................................................... 19
Social aspects................................................................................................................... 19
Best practice in undertaking HTA..................................................................................... 20
HTA products ............................................................................................................................ 21
Horizon scanning/early warning ....................................................................................... 22
Rapid assessment ............................................................................................................ 23 Mini HTA ........................................................................................................................... 23
Core HTA .......................................................................................................................... 23 Adaptation toolkit .............................................................................................................. 24
Life cycle perspectives of health technologies ......................................................................... 24
HTA and specific groups of technologies ................................................................................. 25
Pharmaceuticals ............................................................................................................... 25 Screening .......................................................................................................................... 26
Development of HTA capacity and HTA programmes ............................................................. 27
HTA Capacity .................................................................................................................... 28
HTA Programmes ............................................................................................................. 28
Priority setting ................................................................................................................... 29 Policy-HTA links................................................................................................................ 29
Input for future research agenda .............................................................................................. 32
Closing remarks.......................................................................................................................... 33
Health Services Research and Health Technology Assessment – draft report
6 April 2010 4
Literature...................................................................................................................................... 33
Health Services Research and Health Technology Assessment – draft report
6 April 2010 5
Health Technology Assessment
Introduction
Health technology assessment (HTA)
The International Network of Agencies for Health Technology Assessment (INAHTA) defines
health-care technology as: “…prevention and rehabilitation, vaccines, pharmaceuticals and
devices, medical and surgical procedures, and the systems within which health is protected and
maintained.” Technology assessment in health care is defined as: “…a multidisciplinary field of
policy analysis. It studies the medical, social, ethical and economic implications of development,
diffusion, and use of health technology”. The EUnetHTA Project has added the following
explanatory clarification that emphasizes the process and aims of an assessment: Health
technology assessment (HTA) is a multidisciplinary process that summarises information about
the medical, social, economic and ethical issues related to the use of a health technology in a
systematic, transparent, unbiased, robust manner. Its aim is to inform the formulation of safe,
effective, health policies that are patient focused and seek to achieve best value. Despite its
policy goals, HTA must always be firmly rooted in research and the scientific method. The
practice of HTA within this definition varies considerably across national settings. It informs
policy- and decision-making in specific political, economic and institutional contexts. In order to
be useful HTA has to be designed with processes and outputs that fit the relevant context.
HTA and Health Services Research
HTA is a field that connects clinical research with health services research (HSR) in order to
provide a broad based input to policy-making and to the development of an evidence-based
health care practice. HTA is not in itself research, but is building on both clinical research and
on health services research. However, HTA is sometimes combined with clinical research
and/or HSR to answer policy questions satisfactorily.
The working definition of HSR used in this report is:
HSR is the multidisciplinary field of scientific investigation that studies how social factors, financial systems, organisational structures and processes, health technologies and personal behaviours affect access to health care, the quality and cost of health care and, ultimately, the health and wellbeing of citizens
Health Services Research and Health Technology Assessment – draft report
6 April 2010 6
In the field of HSR HTA is obviously related to analysis of health technologies, but with strong
links to the full field of HSR in order to analyse different prerequisites for and consequences of
the use of health technologies.
Objective
This chapter provides an overview over health services research (HSR) of relevance for health
technology assessment (HTA) in Europe. Based on review of published research we identified
the trends in health services research in relation to HTA so far. In addition we provide input to a
future research agenda by describing the research called for in the existing literature. In order to
give input to a future research agenda we focus on development of: 1) the content of analysis in
HTA (e.g. analysis of economy, organisation, ethics, legal aspects and social aspects), 2) the
‘HTA products’ developed to adequately meet the needs of policy-makers (e.g. early
warning/horizon scanning, rapid assessment, mini-HTA, core HTA), 3) themes/problems where
HTA should be developed to be able to address them more comprehensibly in future (e.g. fast
evolving technologies, information and communication technologies, the life cycle of
technologies, and link between policy and HTA), 4) development of HTA in relation to specific
technologies, and 5) development of HTA capacity and of HTA programmes. To give input to a
future research agenda we discuss need for development in relation to theoretical approaches
and in research methodologies in relation to HTA. Basically these themes concerns two different
contributions:
HSR as part of HTA methods to analyse consequences of use of technologies
HSR as analyses and improvement of HTA as a tool to provide input to policy-making
Methods
This section describes the methods we applied in this literature review presenting existing
research and input for a future research agenda. At this stage the draft report builds on a
literature review. Later input from the conference will be used in the report
Health Services Research and Health Technology Assessment – draft report
6 April 2010 7
Literature review
In this section, the methods used in the literature review concerning HSR of relevance for HTA
are summarised, including search strategies and inclusion/exclusion criteria used for the
selection of relevant literature.
Initially we piloted different search strategies to find the strategy with the largest sensitivity and
specificity. In general it was difficult to define a strategy which captured relevant articles without
losing too many. Based on these pilot literature search we decided to search exclusively in
PubMed. This decision was made since the search seemed to catch a large majority of the
relevant literature. However the consequence of this strategy is that we have not specifically
searched in specialised databases for articles concerning the ‘HTA disciplines’ e.g. health
economy, organisation, and ethics, and we therefore risk losing the specialised research within
each discipline. To compensate for this we decided that the only Mesh term used for the
literature search was ‘technology assessment’. This ensured a high sensitivity but a low
specificity of the search, and therefore allowed us to find the large majority of relevant articles
included in the rather broad search. The search was conducted on the 19th February 2009 and
was limited to include articles dating back ten years. The Literature search included 3360
references. These references were scanned at title/abstract level by two researchers and were
in this process reduced to 555 articles. The next step was to review each article and to decide
whether to include or exclude the article. This process reduced the number of relevant articles
to 195 articles. These articles were supplemented with articles from a recent volume of
International Journal of Technology Assessment in Health Care (vol.25, 2009, SUPPL. 2) which
contain 14 articles summarising the findings from the EUnetHTA project. Not all articles are
directly cited in this draft version of the report.
Criteria to select the relevant articles were the following:
Inclusion:
Articles containing HSR within the area of HTA or debating future needs for HSR within HTA. All
languages included.
Exclusion:
Articles which did not include HSR within HTA (clinical research, health impact assessment,
assessment of specific technologies, articles from outside Europe, priority-setting within health
care not specifically concerning HTA).
Health Services Research and Health Technology Assessment – draft report
6 April 2010 8
Articles from outside Europe were excluded from this systematic review, but some were still
used to show the state-of-the art, to put the European HSR into perspective, and to give input to
the section on future research agenda. Within HTA the European and non-European research is
closely related and it is useful to take the non-European research into account when providing
input to a future research agenda.
The systematic literature search was supplemented with grey literature. This includes all
products from the EUnetHTA project (www.eunethta.net) and a number of recent reports
commenting on the development of HTA. These reports were selected because they are
examples of the most recent research within HTA including HSR and because they specifically
address the need for development within HTA including need for research. The included grey
literature is listed in appendix 1.
The articles and the grey literature were analysed and distributed into relevant categories
reflecting the following themes:
The content of analysis in HTA
o Economic evaluation
o Ethics
o Legal aspects
o Organisational aspects
o Social aspects
o Best practice in undertaking HTA
HTA products
o Horizon scanning/early warning
o Rapid assessment
o Mini-HTA
o Core HTA
o Adaptation toolkit
Health Services Research and Health Technology Assessment – draft report
6 April 2010 9
Life cycle perspectives of technologies
HTA and specific technologies
o Pharmaceuticals
o Screening
Development of HTA capacity and HTA programmes
o HTA capacity
o HTA programmes
o Priority setting within HTA programmes
o Links between policy and HTA
Overview over the literature
This part of the report will give a descriptive, graphical overview of the literature included in the
literature review in relation to selected variables. The variables chosen for classifying the
articles included in this rerview were: 1) the nationality of the first author, 2) the number of
nationalities involved in the research 3) the institution(s) involved in the research, 4) the topics
of the research, 5) the journal where the research was published, and 6) the distribution of
articles in the decade 2000-2009.Overall this overview intend to provide a visual overview of
characteristics of the body of research undertaken in relation to HTA.
Health Services Research and Health Technology Assessment – draft report
6 April 2010 10
Nationality of first author
0
20
40
60
80
100
120
Brittis
hIri
sh
Germ
an
Dutch
Swedish
Norwegia
n
Danish
Finnish
Frenc
h
Spanis
h
Italia
n
Greek
Israeli
Belgian
Swiss
Austri
an
Hungar
ian
Canadia
nUS
Austra
lian
This diagram provides an overview over which countries are active in publishing research in
relation to HTA, and it is a clear tendency that British research are very active within HSR/HTA.
Also it is shown that a large number of European countries are involved in this field. The
Canadian, US, and Australian articles are included, because there are European co-authors.
Number of nationalities involved in the research
Health Services Research and Health Technology Assessment – draft report
6 April 2010 11
Number of nationalities involved
0
20
40
60
80
100
120
140
160
1 2 3 4 or more
This diagram shows to what extent the research is based on collaboration between researchers
from different countries. Even if most research is based in one country, it is still clear that
collaboration between researchers from different countries is established. This is partly due to a
long tradition of (EU-supported) projects among HTA institutions in Europe.
Types of institutions involved in the research
Health Services Research and Health Technology Assessment – draft report
6 April 2010 12
Institution involved in the research
0
20
40
60
80
100
120
140
160
University HTA institution Consultancy Industry Other
This diagram provides an overview over the scientific environment in which the research was
undertaken. Mixed scientific environment is identified in 88 of 215 cases. Even if the universities
are most active it is remarkable that research in relation to HTA is characterised by involvement
of other actors – primarily HTA institutions. However, also private consultancies and industry
has published relevant research.
Topic
Health Services Research and Health Technology Assessment – draft report
6 April 2010 13
Research topic
0
10
20
30
40
50
60
70
Economics Widerimpacts of
technologies
Best practice HTAproducts
Developmentof HTA
methodology
HTA andspecific
groups oftechnologies
HTA capacityand HTA
programmes
HTA policylinks
This diagram provides an overview over the research topics identified in the analysis of the
literature. The diagram shows the especially research in health economics and in the links to
policy have been conducted during the last 10 years. However, a broad field of topics have
been covered.
Publisher
Health Services Research and Health Technology Assessment – draft report
6 April 2010 14
Journal
0
20
40
60
80
100
120
Intl J
Tec
hnol A
sses
s Hea
lth C
are
Health P
olicy
J Hea
lth S
erv R
es P
olicy
Health E
cono
mics
Eur J
Health E
con
BMJ
EUnetH
TA
NHS R&D H
TA P
rogr
amme
Other
Not surprisingly, the International Journal of Technology Assessment in Health Care is the most
used journal for publication of HSR in relation to HTA, but also other journals or programmes
are represented in the list of publishers.
Year of publication
Health Services Research and Health Technology Assessment – draft report
6 April 2010 15
Year of Publication
0
5
10
15
20
25
30
35
2000 2001 2002 2003 2004 2005 2006 2007 2008 2009
This diagram provides an overview over the number of articles published and the year of
publication of research in relation to HTA. The number of identified articles varies between 12
and 30 pr. year.
Existing Research
This part of the report has the purpose of presenting the content of the identified research. As
earlier indicated the research is categorized into different topics of relevance to HSR in the area
of HTA.
The content of analysis in HTA
This part includes research within the ‘HTA disciplines’ and articles concerning best practice in
undertaking HTA.
Health Services Research and Health Technology Assessment – draft report
6 April 2010 16
Economic evaluation
Economic evaluation is the comparison of two or more alternatives courses of action in terms of
both their costs and consequences. Economists usually distinguish several types of economic
evaluation, differing in how consequences are measured:
Cost minimisation analysis
Cost benefit analysis
Cost effectiveness analysis
Cost utility analysis
Economic evaluation is a well established research discipline with lots of different methods
available. However, in spite of the great amount of research, there is still disagreement
concerning the methods to apply when conducting an economic evaluation as part of an HTA.
Hence, even though the economic aspect might have well established methodologies available,
there is still lack of clarity about recommendations and best practice concerning economic
evaluation in relation to HTA.
This literature review revealed that a lot of research is being done in the area of health
economics in relation to health technology assessment. This was the area in which the greatest
body of literature was identified. In total 60 articles were identified to provide an overview of the
European research in this area. Among these articles 37 reported specific studies and were
relevant to provide an overview of the current research activity in this area, while the rest solely
point to or debate areas of future research.
14 studies concerned different aspects of economic modelling (1-14). 4 studies concerned
different aspects of guidelines of how to carry out economic evaluations and how to present
them (1-4). 5 studies concerned cost-effectiveness analysis or different aspects of such analysis
(10, 15-21). 3 studies concerned methods of how to estimate learning effects, that is the
consequences of increasing effectiveness over time (22-24). 4 studies concerned the
assessment of effects. One study concerned the assessment of utility (25-28), one study
concerned the assessment of quality of life (5), one study concerned assessments of
willingness to pay (6), and one study concerned discrete choice experiments (29). Finally, 4
studies concerned the use of economic evaluations and the influence on decision making (30-
33).
Health Services Research and Health Technology Assessment – draft report
6 April 2010 17
In addition EUnetHTA has contributed to analysis of how provide core elements of an economic
analysis which can be used to adapt to different settings (7-11).
Thus, this literature review showed, that economic modelling, guidelines, cost-effectiveness
analysis, the effects of learning curves, estimation of effect and utility, the use of economic
evaluation, influence on decision making, and use of economic analysis in different settings
were the themes that make up the majority of the current research activity dealing with
economic evaluation in relation to HTA.
Other studies dealt with the use of systematic reviews and evidence and the assessment of
uncertainty.
The studies primarily applied quantitative methods or review quantitative methods. Those
studies, that concern the use of economic evaluations and the influence on decision making,
were the only studies that apply qualitative methods or a combination of qualitative and
quantitative methods.
The studies were predominantly conducted in university environments. Also, the research
activity was most prevalent in the United Kingdom.
Additionally, this review revealed that a lot of research is also undertaken in especially Canada
and US. This body of research overall reflected the same research themes as the current
European research.
Ethics
The ethical domain of HTA deals with prevalent morals, values and behavioral models of the
society relevant for the technology. It has been argued that ethics should be part of HTA since
its inception in the beginning of the 1970’ies. However, today there is still little agreement on the
methods for integrating ethics in HTA, and only few HTAs report on these issues.
A variety of methods to integrate ethics in HTA exits, e.g. causistry, coherence analysis, and
principlism. Both qualitative methods taken from existing research disciplines and methods
developed specifically for the integration of ethics in HTA were applied.
Working groups within both INAHTA and HTAi have tried to come to some agreement of which
methods that are most appropriate to approach the ethical issues, and most recently EUnetHTA
Health Services Research and Health Technology Assessment – draft report
6 April 2010 18
developed methodology to address the ethical domain as part of the Core model. However, still
only few HTAs address these issues.
6 articles relevant for the ethical domain were identified in the literature search.
4 articles were methodological articles that provided and addressed specific methods of how to
include the ethical issues in HTA (12-15). Also relevant for the consideration of ethics, 2 articles
address criteria for how qualitative research should be evaluated (16, 17). 2 articles discuss why
ethics should be part of HTA and the reasons for the lack of its inclusion (18, 19). In EUnetHTA,
ethics were included in the core model (7-11).
Several articles addressed ethics and discussed the opportunities and challenges in relation to
specific technologies that are considered controversial or ethically complex (e.g. genetic
screening).
All of the articles use qualitative methods or discuss qualitative methodology.
The research is mainly undertaken at universities.
In addition, this literature search revealed that research and debate concerning the ethics in
HTA is undertaken in Canada and Australia.
Legal aspects
The legal aspects of HTA address legislation questions concerning the technology under
assessment. For an issue to be considered a legal one, one must be able to point out the legal
source (stipulation, convention or agreement) that makes the issue legally relevant. Also, this is
what separates the legal domain from domains concerning ethics, social issues and safety.
No articles explicitly focusing on the legal aspects were identified in the literature review. Only
one European article address the role of legal aspects in relation to genetic screening. The
article proposes an Internet citizens’ jury as a method to address ethical, societal and legal
implications (34). Additionally one Canadian study also focusing on the legal aspects in relation
to genetic screening was identified in the literature (35). EUnetHTA included legal aspects in the
core model (7-11).
Health Services Research and Health Technology Assessment – draft report
6 April 2010 19
Organisational aspects
The organisational aspect of HTA focuses on the delivery modes of the assessed technologies.
This includes aspects of e.g. the management, financing and controlling issues, and can thus
contribute to assessments by clarifying challenges and barriers in implementing health
technologies.
One European article identified in the literature search addressed the organisational aspects of
HTA. The authors aimed to make a case for a greater emphasis in research on how health
services are managed, organised and delivered, and discuss the theoretical differences
between and within disciplines, and discuss their implications for research methods (20). In
addition organisational issues are incorporated into the EUnetHTA core model (7-11) and the
adaptation tool kit (21-25)
One article underlined the future need for inclusion of organisational analysis in HTA due to
demand from policymakers (36).
Social aspects
The literature revealed a great deal of mixing of different concepts such as social aspects and
patient perspective under this heading. According to EUnetHTA the social domain of HTA takes
the patient as a point of departure in the analysis of the manifold social implications of a health
technology. The analysis should reveal the resources needed when using a technology and the
consequences of its use in the life of the patients.
No European articles focusing explicitly on the assessment of social aspects were identified in
the literature review. One Danish study presented a research design including a citizens’ jury
that aims at exploring ethical, social and legal implications (34).
However, two Belgian studies concern the use of qualitative research in HTA, how knowledge of
the nature of this type of research can be improved, become more trustworthy and how the
contribution can be increased and criteria for judging this type of research (16). EUnetHTA has
included social aspects as part of the core model and introduces a broader range of literature on
this topic, which has not been included in this review (7-11).
EUnetHTA included social aspects in the core model (7-11).
Health Services Research and Health Technology Assessment – draft report
6 April 2010 20
Additionally, three Canadian studies, two American studies, and one Australian study focused
on the social aspects, how to assess the social issues, how to incorporate the assessment in
HTA and how to evaluate methods (26-29).
Best practice in undertaking HTA
Seven articles analysed and debated best practice in undertaking HTA in general. One article
from the ECHTA/ECAHI project describes the current best practice in undertaking and reporting
HTA (37). One articles described the time-trends in health technology assessment from 1989 to
2002 and thereby gave an overview over the content of HTA reports (38) Another gave en
overview over cross-national HTA processes (39), while a further article gave an international
comparison of the definition and practical application of HTA (40). An statistical analysis
analysed what factors that influenced assessment methods in HTA (41). On article discussed
key principles for the improved conduct of HTA (42), while the last identified article debates
advantages and disadvantages of standardization of evidence requirements for HTA (43).
In addition the EUnetHTA products – mainly the core model - could be seen as a further
implementation of the conclusions from the ECHTA/ECAHI project, and contributed to
developing a more standardized form for reporting HTA (7-11).
Transparency
As part of the discussion on best practice, a lot of attention has been given to transparency
issues – both in relation to use of methods and to reporting of HTA. An example of this is the
INAHTA checklist which promotes different requirements for creating increased transparency in
the reporting of HTA (30).
As the implementation of HTA in EU member states increased, this led to a series of EU funded
projects from 1993-2002, including EUR-ASSESS and the ECHTA/ECAHI. The
recommendations of these projects are currently being implemented through the EUnetHTA
project, and include a core model of HTA reports and an adaption toolkit to expand proper
sharing and production of information – including promoting transparency.
Health Services Research and Health Technology Assessment – draft report
6 April 2010 21
Five articles dealing with issues of transparency were identified. In one article two HTA reports
on the same topic are compared in order to address issues of transparency and variations in
HTA methods (31). The authors concluded, that efforts to guarantee transparency in the original
studies, and in the HTA reports themselves, needed to be taken, as such lack of transparency
may affect policy decisions. Another article presents the INAHTA checklist for HTA reports (30).
The three latter articles were debating articles. One article reviewed and discusses the issues
associated with standardisation taking into account the perspectives of multiple stakeholders
(32), while two articles debates issues of transparency in NICE technology assessments in the
case of drug evaluations (33, 34).
Three reports debated and critiqued the lack of transparency in HTA reports in general and
opted for more focus on transparency issues (44-46)
HTA products
A variety of different types of health technology assessments have emerged, typically from the
work of HTA agencies. A lot of literature addresses HTA production in general. However, a
more limited literature addresses specific ‘HTA products’ which have been developed to meet
the needs of users of HTA.
Mini HTA and rapid assessments are two types of assessments that have emerged as
responses to the existing conflict between traditional thorough HTAs and the need to produce
input to decision making in a timely fashion. Political and administrative decision makers require
a faster production of HTA.
Horizon scanning/early warning is a HTA product that was developed in response to the needs
of assessing technologies in the early phases of adaption before thorough research have been
undertaken.
Core HTA and an adaptation toolkit for ‘translation’ of foreign HTA reports has been develop as
part of the EUnetHTA project with the aim of reducing duplication of work and promoting already
existing foreign reports.
Health Services Research and Health Technology Assessment – draft report
6 April 2010 22
Horizon scanning/early warning
Health technologies in an early stage of its life cycle sometimes spread rapidly in a health care
system despite lack of convincing evidence. Also, new technologies exist, that may be
underused resulting in lack of benefit for the patients.
Horizon scanning systems/early warning systems is a method evolved from the work of HTA
agencies. The International Information Network on New and Emerging Health Technologies
(EuroScan) is an international collaboration established in 1999. EuroScan defines the focus of
horizon scanning as those technologies that are not yet adopted by the healthcare system and
those that are in the phase of adoption. The purpose of horizon scanning is to provide input to
decision making that is timely and relevant.
The members of EuroScan have agreed on a common terminology, classification and
understanding of their activities. Their activities consist of five main components which are the
identification and filtering of technologies, prioritization, early assessment, dissemination, and
monitoring the assessed technologies.
Nine articles concerning different aspects of horizon scanning were identified. One of these
articles reported from a workshop and subsequent discussions, that reviewed the achievements
and progress of Euroscan (35), and one article provided an overview of processes and practices
of horizon scanning (36). Six articles evaluate different aspects of what constitutes a proper
horizon scanning. This includes the use of selection criteria in horizon scanning, the sources to
search in the process of identifying technologies, and the effectiveness and accuracy of HSS
(37-40). Two articles explore aspects of adaption of established horizon scanning to Danish
horizon scanning (41, 42). Finally, one article explores the intrascientific citation (bibliometric
impact) of the Swedish early warning reports (SBU Alerts) and the science base of these reports
(43).
EUnetHTA produced a newsletter on new and emerging health technologies for European
policy makers in collaboration with Euroscan (44). The process was reported in an article which
described the process of producing a newsletter (45).
Health Services Research and Health Technology Assessment – draft report
6 April 2010 23
Rapid assessment
Rapid assessment is a tool to produce input to decision making in a timely fashion. The rapid
assessments are typically carried out by HTA agencies within a timeframe of 6 months or less.
A number of programs exist. However, they vary in scope and methods. At this point, no
common definition and methodology of rapid assessment exists and the quality of the
assessments varies.
The literature review identified only two European articles concerning rapid assessments.
One German article compares different rapid assessment programs with respect to scope,
methods and time to complete assessments, and introduces and discusses a model for
processing rapid assessment in the German context (46). Likewise, a British study describes a
method for the rapid appraisal of new interventional procedures and compares its conclusions
with those derived from a slower, more thorough method (47).
Additionally, one Canadian article and one Australian article were identified concerning the
impact and methods of rapid assessments (48, 49).
Mini HTA
Mini HTA is a tool mainly used in hospital settings when making decisions about the uptake of
new technologies. It typically consists of a form containing a number of questions corresponding
to the domains of a thorough HTA. This tool aims at providing input to decision making within
few weeks adjusted to the local settings.
Only two articles concerning mini HTA were identified in the literature search (50, 51). Both
studies were Danish studies undertaken by Danish University Hospitals. Both studies evaluated
the use of HTA and the attitudes of decision makers towards the tool. The studies revealed that
different versions of the tool, and with varying quality, are commonly used in hospital settings in
Denmark.
Core HTA
Core HTA built on recommendations from previous European HTA projects and developed a
specific structure for undertaking and reporting HTA. The main aim was to avoid duplication of
Health Services Research and Health Technology Assessment – draft report
6 April 2010 24
HTA reports in Europe since there are many examples of reports on technologies produced
synchronically across Europe – and also across the rest of the world.
The core model was reported in five different publications. One model for medical and surgical
prcedures (7), and one for diagnostic technologies (8). A handbook introducing the model was
published (9), and two articles presented the idea behind the core model (10), and the testing of
the model during the development phase (11).
Adaptation toolkit
EUnetHTA also developed an adaptation toolkit. This product was a further development of an
already existing activity among the HTA agencies, which used foreign HTA reports and
‘translated’ them into the relevant setting. The aim to avoid as much works as possible by
reusing e.g. the systematic review from the foreign report. EUnetHTA worked to structure the
activity by publishing a glossary of HTA adaption terms (22, 23). Also the adaptation toolkit itself
(21) and two additional articles described the development of the toolkit (24, 25).
Life cycle perspectives of health technologies
Many existing healthcare interventions diffused before the establishment of the current
assessment and evaluation procedures. Assessment of ineffective or inappropriately applied
practices is growing as a priority for international health policy, both for improved quality of care
and sustainability of resource allocation. It might be difficult for assessments to have an impact
on an established practice (52). On the other hand, assessments of resource consuming
technologies that remain in use are needed in order to determine disinvestments and to improve
the overall quality of care and sustainable resource allocation (53-55).
The literature review identified four articles concerning aspects related to the assessment of
technologies already in use including disinvestments. One of these articles introduced a method
of constructive medical technology assessment that aim to change the development and
diffusion of a medical device to improve its later clinical effectiveness, illustrated by the case of
heart assist devices (52). The three other articles discussed disinvestment in relation NICE
evaluations. These articles pointed out, that NICE had a tendency to focus on the assessment
of new technologies while the assessments of existing technologies are given less priority (55).
Health Services Research and Health Technology Assessment – draft report
6 April 2010 25
This concerned the aim of the establishment of NICE which partly was to ensure the overall best
use of resources in the NHS so that patients receive the greatest benefit (54). Hence, NICE
recommendations may not necessarily increase the overall efficiency of resource allocation
decisions in the NHS due to lack of focus on the full life cycle of health technologies (54). This
dilemma called for an overall increased focus on the assessment of technologies already in use
(55). One article added to this debate by exploring the policy options available to NICE as it
prepared to launch a programme to meet the NHS request for guidance on disinvestment. All of
the possible options present different challenges, as NICE will need to collaborate in new ways
with partners inside, and perhaps outside, the NHS (53).
EUnetHTA also focused on the life cycle perspective in relation to assessment of technologies,
but so far concentrated on the earlier stages of the life cycle by focusing on evidence generation
on promising health technologies (56-58). However the core model encouraged that new
technologies should always be compared to existing technologies, and the model can also be
used to assess possible obsolete technologies.
HTA and specific groups of technologies
In this section two groups of technologies are addressed. The first group concern primarily
pharmaceuticals that possess challenges to HTA in terms of the ability to provide input to
decision making in a timely matter and address the literature that concern Coverage with
Evidence Development (CED) as an option to overcome this obstacle. The other group concern
screening methods that represent a group of technologies that in many cases by practitioners
and to the general public is received as harmless.
Pharmaceuticals
To provide assessments of technologies as an input to a rational decision making process in a
timely matter is always a challenging task. A sufficient amount of studies for thorough
assessments are usually not available until the technologies already are in use. On the other
hand technologies that are in early phases of their lifecycle may lack sufficient studies for
thorough assessments which affect the quality of assessments. Hence, the balance between
Health Services Research and Health Technology Assessment – draft report
6 April 2010 26
too early and too late assessment of technologies is a challenge for HTA (see also paragraph
on life cycles).
The cases of pharmaceuticals often make up a great body of this discussion especially in
relation to the evaluation of the economic benefits of new pharmaceuticals. Resulting from this
discussion, the possibility of providing pharmaceuticals while ongoing evaluation is still
undertaken has been put forward. This has been referred to as Coverage with Evidence
Development (CED) and refers to the reimbursement of pharmaceuticals given that an ongoing
evaluation of the benefits is undertaken.
Even though the possibilities of providing pharmaceuticals in terms of CED, the issue can be
generalized to other health technologies, as the challenge to provide assessments of
technologies as an input to a rational decision making process in a timely matter relates to all
new technologies. CED gives an opportunity to provide an input to the decision making process,
but with certain reservations.
4 articles concerning CED were identified in the literature search. One article address the
tension between pharmaceutical regulators and healthcare reimbursement authorities and CED
as an option to overcome this issue (59). One article discusses CED as a possibility for NICE
recommendations to the NHS (60). One study explores the conceptual and policy issues in
general relation to CED and discusses issues involved in operationalizing CED in practice and
presents criteria for which technologies may be most suitable for CED (61), and finally one
article comments on this study (62).
Screening
Screening is the application of a test to detect potential disease or condition in a person who
has no known signs or symptoms of that disease or condition. The goal of screening is the early
detection of disease or risk factors of disease so that intervention can reduce morbidity and
mortality from the involved disease (63). Methods of screening represent a group of
technologies essentially different from clinical practice. The main issues being that screening is
carried out in apparently healthy persons, so the moral duty is essentially different from that of
typical clinical practice (64). Also the technologies are typically received as harmless by
practitioners and the general public resulting in a growing demand for and uncontrolled use of
Health Services Research and Health Technology Assessment – draft report
6 April 2010 27
these technologies. In some cases traditions of use of screening technologies is a challenge
(65).
In the literature review a series of articles concerning HTA and screening were identified carried
out as a specific theme in IJTAHC. The objectives of these articles were to explore the impact of
HTA on health policy and practice in the case of preventional technologies in nine different
countries illustrated by three cases of screening: Mammography screening for breast cancer,
screening for prostate cancer, and the use of ultrasound screening in pregnancy (63-71).
Mammography screening represents a technology with a surprising lack of active policies
concerning implementation despite the weigh of evidence that indicates benefits. Only Sweden,
UK and The Netherlands had at the time of the publishing of the articles implemented such
policies (64, 65, 67, 68).
Ultrasound screening in pregnancy represented a technology that had spread despite lack of
assessment showing benefit. The technology was perceived as an appealing, apparently safe
and relatively cheap technology resulting in a pressure on demand side from both patients and
physicians (65).
Screening for prostate cancer presented a technology that (despite no assessment program
supported such screening so far] was supported by many physicians resulting in a growing use
of the technology (65).
An overall concern was the case of opportunistic screening that is when screening tools is used
as diagnostic tools outside the context of a national screening program. This practice was
reported extremely difficult to control (65).
Development of HTA capacity and HTA programmes
An always important issue for the HTA community is to build capacity and develop existing
programmes to ensure sustainability of HTA in Europe. Related themes are 1) how to ensure
that an HTA programme prioritises assessment of the right technologies to ensure as much
benefit of HTA as possible, and 2) how to improve the links between policy and HTA to improve
the impact of HTA.
Health Services Research and Health Technology Assessment – draft report
6 April 2010 28
HTA Capacity
Internationally there is a growth in HTA activities. However, despite this growth many European
countries have none or only limited HTA capacity, especially low and middle income countries.
There are several obstacles to the introduction of HTA. In many countries, there is a general
lack of awareness of HTA and the manpower to conduct assessments is insufficient. Hence, an
important aspect of capacity building is the establishment of educational programs to promote
training and knowledge in HTA. Moreover data availability in these countries is low and the
quality and validity of morbidity and mortality data is questionable. For these reasons,
developing countries are dependent on studies carried out in other countries. However, while
some data might be transferable, such as clinical and epidemiological data, resource utilization,
costs, cost-effectiveness data, prices, ability and willingness to pay vary from country to country.
These issues are particularly relevant due to the recent uptake of several new member
countries in the European Union. The literature review revealed, that some efforts, European
and international, are undertaken in areas of capacity building. Furthermore, one of the main
goals of the EUnetHTA Collaboration is to increase transparency and transferability of HTAs.
The literature review revealed that European initiatives of capacity building are undertaken in
the case of Hungary (72, 73) and Estonia (74).
As a part of the ECHTA project, Working Group 5 provided an overview of current HTA
education and training programs in Europe revealing that education programs are limited in the
EU12 countries. Additionally, the working group developed a curriculum of a European Master
of science in HTA (75). Likewise, the Ulysses Program, a training program developed in
corporation between Canadian, Spanish and Italian HTA agencies, aimed at evaluators who will
produce HTA and decision makers who will use HTA (76).
EUnetHTA published a handbook on capacity building to support the development of HTA in
Europe (77).
HTA Programmes
Also the description and analysis of different organizational aspects of HTA programs seem to
be of increasing interest. The particular organizational features of the body producing HTA
Health Services Research and Health Technology Assessment – draft report
6 April 2010 29
reports may influence the diffusion of knowledge and the effect of those activities on policy
making.
The literature search revealed two European articles concerned with the performance and
description of HTA programs. One article focused on the differences and similarities of HTA
agencies {448}, the other article analyse what constitutes an “optimal” HTA program {3070}.
EUnetHTA published a survey on characteristics of existing HTA organisations to provide
information on how to develop vigorous HTA organisations in Europe (78).
Priority setting
The resources for HTA within existing HTA programmes fall short of that needed to evaluate all
health care technologies. Therefore, for resources to be used cost effectively, priorities are set
by the HTA agencies concerning which technologies to assess. Several methods exist to make
this prioritization, and some are more transparent and systematic than others. Different HTA
agencies apply different methods for priority setting.
In total four European articles concerning different aspects of priority setting were identified in
the literature search. Two articles evaluated the methodology of two priority setting approaches
in UK and the Netherlands respectively (79, 80), and one article described and evaluated the
relative importance of the different sources used for priority setting by the NHS HTA Programme
(81). Finally one study aimed at developing a method (PATHS) for economic evaluation and at
the stage of research prioritization (82).
PolicyHTA links
The development of links between policy-making and HTA has been a built-in theme in relation
to HTA since the first HTAs were produced in the 1970ties. This was a result of the fact that
HTA was ‘invented’ to give input to policy-making at different levels. Thus development of and
research in the policy loop - ‘policy-HTA-policy’ - is an important theme since the utilisation of
HTA is necessary to justify the use of resources on production of HTA. In short…. if policy-
makers do not see the benefit of HTA and find it useful when making decisions, HTA is not
viable. Therefore researchers and HTA institutions has shown a growing interest in developing
Health Services Research and Health Technology Assessment – draft report
6 April 2010 30
HTA so that it is useful to policy-makers and in documenting the impact of HTA on policy-
making.
44 contributions (articles, reports, book chapters) were identified which deal with the links
between policy and HTA.
A total number of 11 publications addressed the policy-HTA links in a UK setting. In one
comprehensive report the impact of the NHS HTA Programme was assessed (47). The overall
conclusions were that the programme has had considerable impact on knowledge generation,
perceived impact on policy and to some extend on practice. One article evaluated the impact of
a technology appraisal process in the South and West Development and Evaluation Committee
by measuring awareness, influence, and quality of reports among clinical and managerial staff
and concluded that the process was perceived to have an impact on policy decisions among the
staff. However, impact on practice could not be identified with routine data (48). Another article
debated the use of evidence (mainly economic evidence) in the development of local health
policies, and based their conclusions on an in-depth study of Health Improvement Programmes
in England. The questionnaire based study concluded that the main ways of increasing the use
of evidence were to produce more evidence-based national guidance and to disseminate
summaries to local decision makers (49). In addition to these three publications a total of eight
articles analysed the policy-HTA links specifically in relation to NICE processes. Two articles
analysed the implementation and impact of NICE guidance. One looked at impact on GP
prescribing and found only little impact unless the guidance coincided with information from
other sources (50). The other studied the extent and pattern of implementation using multiple
methods and concluded that implementation of the guidance has been variable. However,
guidance was more likely to be implemented when there was strong professional support, a
strong, stable evidence base, no increased or unfunded costs in organisations with good
systems for tracking guidance implementation, and where the professionals involved was not
isolated (51). Two article discussed the general procedures of NICE (52, 53), while two other
article analysed the use of cost-effectiveness analysis/economic appraisal and discusses the
consequences for the use of NICE appraisals (54, 55). One examined concerns from patient
groups in relation to NICE decision-making processes (56), and finally did one article analyse
NICE’s use of cost effectiveness as an exemplar of a deliberative process which include both
scientific context-free evidence about the general clinical potential of a technology, scientific
context-sensitive evidence about particular evidence in realistic scenarios, and colloquial
Health Services Research and Health Technology Assessment – draft report
6 April 2010 31
evidence to into a context and to supply the best evidence short of scientific evidence to fill in
any relevant gaps (57)
A series of articles analysed HTA and its influence on health-care priority setting based on case
studies in England/Wales, France, the Netherlands, and Sweden (58-62). The overall
conclusion on this 2004 overview was that translating HTA into policy is a highly complex
business, and that its influence on policy making remains marginal (58).
Two articles investigated the usefulness and impact of HTA in hospital settings in France and
Italy, and the French article concluded positively that HTA has had significant impact on the
implementation of technologies (63, 64).
Six different articles addressed the questions of impact (65, 66), priority setting for adoption of
health technologies (67), integration of HTA recommandations into organisational and clinical
practice (68), needs of decision makers (69),and lessens learnt from a user perspective in
relation an HTA programme. These studies took place in different European countries.
Furthermore seven articles investigated different themes in relation to the policy HTA link (70-
76).
A series of articles, letters and replies debated the links between policy and HTA by focusing on
why HTA reports building on the same methods and material in relation to PET in oncology led
to different conclusions and policy decisions (77-79). Further the authors discussed the
requirement for documentation in relation to future HTAs of diagnostic methods.
Finally EUnetHTA published a book containing eight separate chapters analysing the policy-JTA
links in Europe. This included reviews of e.g. the impact of HTA on policy-making, the needs
and demands of policy-makers, and an overview of the producers of HTA in Europe (80).
Stakeholder involvement in HTA
Five European articles concerned stakeholder involvement in HTA processes. Two articles
specifically focused on the involvement of consumers. One article gave an overview of
consumer involvement in processes of INAHTA member organisations (81), while the other
describes consumer involvement in the NHS HTA program (82). Both articles described current
status of involvement and particularly the second article emphasized the need for explicit,
inclusive and reproducible methods for supporting consumer involvement. The same theme is
Health Services Research and Health Technology Assessment – draft report
6 April 2010 32
dealt with in a Canadian article which identified what consumer organisations considered
meaningful involvement, what the current practices in Candian HTA processes were, and
developed a model for involvement based on priorities and needs (83). Two articles analysed
stakeholder involvement more generally in NICE processes (84, 85). The articles analysed the
processes where NICE was created and discusses the unique way of involving stakeholders,
but also concludes that there is a need for more flexible approaches to stakeholder involvement
in order to ensure legitimate and transparent processes. Two Canadian articles also analysed
stakeholder involvement – one as a means to improve the impact of HTA (86), and one a
involving the values and judgements of stakeholders in policy coverage decisions (87). The final
article described the processes developed in the EUnetHTA project with the purpose of
involving stakeholders in the further development of European collaboration in relation to HTA.
The article focused on stakeholder involvement in common processes across a large number of
HTA institutions and included European umbrella organisations in the development of a
stakeholder policy (88).
Input for future research agenda
Several inputs for the future research agenda concerning HTA in relation to health services
research in Europe were identified in the literature review.
Some research themes relate to the methodology relevant to the specific elements of HTA,
some relate to ways of improving the influence of HTA on health policy making while others
relate to the assessment of technologies at different stages in their life cycle and assessment of
technologies that challenge the current methodological framework of HTA.
In addition to the outcome of the literature review that is presented in this report, the
achievement of the Health Services Research Europe Working Conference the 8th and 9th of
April will provide even more insight and result in further inputs to this research agenda.
A thorough presentation of the input for future research agenda concerning HTA in relation to
health services research in Europe will thus be provided after the Conference.
However, to provide a basis for the parallel session concerning HTA at the conference, three
agendas will be presented and discussed in more detail at the conference. These are:
Assessing the wider impact of health technologies (e.g. organisational aspects)
Health Services Research and Health Technology Assessment – draft report
6 April 2010 33
Improving the links between HTA and policy
Assessing technologies which challenge the common HTA methodology
Closing remarks
This version of the report should be considered a draft version of a final report on Health
Technology Assessment in relation to Health Services Research in Europe that will be provided
after the Health Services Research Europe Working Conference in Hague.
The finial report will provide a thorough presentation of the future research agenda concerning
HTA in relation to health services research in Europe incorporating the result of the literature
review, the achievements of the Working Conference and finally the results of the stakeholder
survey undertaken as a part of the project.
Literature
1. Philips Z, Bojke L, Sculpher M, Claxton K, Golder S. Good practice guidelines for decision-analytic modelling in health technology assessment: A review and consolidation of quality
assessment. Pharmacoeconomics. 2006;24(4):355-71.
2. Cooper N, Coyle D, Abrams K, Mugford M, Sutton A. Use of evidence in decision models: An
appraisal of health technology assessments in the UK since 1997. J Health Serv Res Policy.
2005 Oct;10(4):245-50.
3. Philips Z, Ginnelly L, Sculpher M, Claxton K, Golder S, Riemsma R, et al. Review of
guidelines for good practice in decision-analytic modelling in health technology assessment.
Health Technol Assess. 2004 Sep;8(36):iii,iv, ix-xi, 1-158.
4. Spiegelhalter DJ, Myles JP, Jones DR, Abrams KR. Bayesian methods in health technology
assessment: A review. Health Technol Assess. 2000;4(38):1-130.
Health Services Research and Health Technology Assessment – draft report
6 April 2010 34
5. Vallejo-Torres L, Steuten LM, Buxton MJ, Girling AJ, Lilford RJ, Young T. Integrating health
economics modeling in the product development cycle of medical devices: A bayesian
approach. Int J Technol Assess Health Care. 2008 Fall;24(4):459-64.
6. Brennan A, Chick SE, Davies R. A taxonomy of model structures for economic evaluation of
health technologies. Health Econ. 2006 Dec;15(12):1295-310.
7. Barton P, Bryan S, Robinson S. Modelling in the economic evaluation of health care:
Selecting the appropriate approach. J Health Serv Res Policy. 2004 Apr;9(2):110-8.
8. Griffin S, Bojke L, Main C, Palmer S. Incorporating direct and indirect evidence using
bayesian methods: An applied case study in ovarian cancer. Value Health. 2006 Mar-
Apr;9(2):123-31.
9. Griffin S, Claxton K, Hawkins N, Sculpher M. Probabilistic analysis and computationally
expensive models: Necessary and required? Value Health. 2006 Jul-Aug;9(4):244-52.
10. Karnon J. Alternative decision modelling techniques for the evaluation of health care
technologies: Markov processes versus discrete event simulation. Health Econ. 2003
Oct;12(10):837-48.
11. Grieve R, Hutton J, Green C. Selecting methods for the prediction of future events in cost-
effectiveness models: A decision-framework and example from the cardiovascular field. Health
Policy. 2003 Jun;64(3):311-24.
12. Bloom BS, de Pouvourville N, Libert S. Classic or bayesian research design and analysis.
does it make a difference? Int J Technol Assess Health Care. 2002 Winter;18(1):120-6.
Health Services Research and Health Technology Assessment – draft report
6 April 2010 35
13. Boas G, van der Stel H, Peters H, Joore M, Anteunis L. Dynamic modeling in medical
technology assessment. fitting hearing aids in the netherlands. Int J Technol Assess Health
Care. 2001 Fall;17(4):618-25.
14. Claxton K, Neumann PJ, Araki S, Weinstein MC. Bayesian value-of-information analysis. an
application to a policy model of alzheimer's disease. Int J Technol Assess Health Care. 2001
Winter;17(1):38-55.
15. Sprague S, Quigley L, Adili A, Bhandari M. Understanding cost effectiveness: Money
matters? J Long Term Eff Med Implants. 2007;17(2):145-52.
16. Eckermann S, Briggs A, Willan AR. Health technology assessment in the cost-disutility
plane. Med Decis Making. 2008 Mar-Apr;28(2):172-81.
17. Adam T, Koopmanschap MA, Evans DB. Cost-effectiveness analysis: Can we reduce
variability in costing methods? Int J Technol Assess Health Care. 2003 Spring;19(2):407-20.
18. IJzerman MJ, Reuzel RP, Severens HL. Pre-assessment to assess the match between cost-
effectiveness results and decision makers' information needs: An illustration using two cases in
rehabilitation medicine in the netherlands. Int J Technol Assess Health Care. 2003
Winter;19(1):17-27.
19. Kristiansen IS, Stavem K, Linnestad K, Pedersen KM. Evaluation of medical methods--can
we rely on cost-benefit-analysis? Tidsskr Nor Laegeforen. 2003 Mar 6;123(5):657-60.
20. Fenwick E, Claxton K, Sculpher M. Representing uncertainty: The role of cost-effectiveness
acceptability curves. Health Econ. 2001 Dec;10(8):779-87.
Health Services Research and Health Technology Assessment – draft report
6 April 2010 36
21. Zivin JG. Cost-effectiveness analysis with risk aversion. Health Econ. 2001 Sep;10(6):499-
508.
22. Ramsay CR, Grant AM, Wallace SA, Garthwaite PH, Monk AF, Russell IT. Statistical
assessment of the learning curves of health technologies. Health Technol Assess. 2001;5(12):1-
79.
23. Ramsay CR, Wallace SA, Garthwaite PH, Monk AF, Russell IT, Grant AM. Assessing the
learning curve effect in health technologies. lessons from the nonclinical literature. Int J Technol
Assess Health Care. 2002 Winter;18(1):1-10.
24. Ramsay CR, Grant AM, Wallace SA, Garthwaite PH, Monk AF, Russell IT. Assessment of
the learning curve in health technologies. A systematic review. Int J Technol Assess Health
Care. 2000 Autumn;16(4):1095-108.
25. Brauer CA, Rosen AB, Greenberg D, Neumann PJ. Trends in the measurement of health
utilities in published cost-utility analyses. Value Health. 2006 Jul-Aug;9(4):213-8.
26. O'Brien BJ, Spath M, Blackhouse G, Severens JL, Dorian P, Brazier J. A view from the
bridge: Agreement between the SF-6D utility algorithm and the health utilities index. Health
Econ. 2003 Nov;12(11):975-81.
27. Feeny D, Blanchard C, Mahon JL, Bourne R, Rorabeck C, Stitt L, et al. Comparing
community-preference-based and direct standard gamble utility scores: Evidence from elective
total hip arthroplasty. Int J Technol Assess Health Care. 2003 Spring;19(2):362-72.
28. Mullahy J. Live long, live well: Quantifying the health of heterogeneous populations. Health
Econ. 2001 Jul;10(5):429-40.
Health Services Research and Health Technology Assessment – draft report
6 April 2010 37
29. Ratcliffe J, Longworth L. Investigating the structural reliability of a discrete choice
experiment within health technology assessment. Int J Technol Assess Health Care. 2002
Winter;18(1):139-44.
30. Williams I, McIver S, Moore D, Bryan S. The use of economic evaluations in NHS decision-
making: A review and empirical investigation. Health Technol Assess. 2008 Apr;12(7):iii, ix,x, 1-
175.
31. Bryan S, Williams I, McIver S. Seeing the NICE side of cost-effectiveness analysis: A
qualitative investigation of the use of CEA in NICE technology appraisals. Health Econ. 2007
Feb;16(2):179-93.
32. Hoffmann C, Graf von der Schulenburg JM. The influence of economic evaluation studies on
decision making. A european survey. the EUROMET group. Health Policy. 2000 Jul;52(3):179-
92.
33. Williams IP, Bryan S. Cost-effectiveness analysis and formulary decision making in england:
Findings from research. Soc Sci Med. 2007 Nov;65(10):2116-29.
34. Moldrup C. Medical technology assessment of the ethical, social, and legal implications of
pharmacogenomics. A research proposal for an internet citizen jury. Int J Technol Assess
Health Care. 2002 Summer;18(3):728-32.
35. Potter BK, Avard D, Graham ID, Entwistle VA, Caulfield TA, Chakraborty P, et al. Guidance
for considering ethical, legal, and social issues in health technology assessment: Application to
genetic screening. Int J Technol Assess Health Care. 2008 Fall;24(4):412-22.
36. Battista RN, Hodge MJ. The evolving paradigm of health technology assessment:
Reflections for the millennium. CMAJ. 1999 May 18;160(10):1464-7.
Health Services Research and Health Technology Assessment – draft report
6 April 2010 38
37. Velasco M, Perleth M, Drummond M, Gurtner F, Jorgensen T, Jovell A, et al. Best practice
in undertaking and reporting health technology assessments. working group 4 report. Int J
Technol Assess Health Care. 2002 Spring;18(2):361-422.
38. Draborg E, Gyrd-Hansen D. Time-trends in health technology assessments: An analysis of
developments in composition of international health technology assessments from 1989 to
2002. Int J Technol Assess Health Care. 2005 Fall;21(4):492-8.
39. Garcia-Altes A, Ondategui-Parra S, Neumann PJ. Cross-national comparison of technology
assessment processes. Int J Technol Assess Health Care. 2004 Summer;20(3):300-10.
40. Draborg E, Gyrd-Hansen D, Poulsen PB, Horder M. International comparison of the
definition and the practical application of health technology assessment. Int J Technol Assess
Health Care. 2005 Winter;21(1):89-95.
41. Draborg E, Andersen CK. What influences the choice of assessment methods in health
technology assessments? statistical analysis of international health technology assessments
from 1989 to 2002. Int J Technol Assess Health Care. 2006 Winter;22(1):19-25.
42. Drummond MF, Schwartz JS, Jonsson B, Luce BR, Neumann PJ, Siebert U, et al. Key
principles for the improved conduct of health technology assessments for resource allocation
decisions. Int J Technol Assess Health Care. 2008 Summer;24(3):244,58; discussion 362-8.
43. Hutton J, Trueman P, Facey K. Harmonization of evidence requirements for health
technology assessment in reimbursement decision making. Int J Technol Assess Health Care.
2008 Fall;24(4):511-7.
44. Kanavos P, Persson U, Drummond M. The future of health technology assessment in
europe. The Initiative for Sustainable Healthcare Financing in Europe; 2009. Report No.: 2.
Health Services Research and Health Technology Assessment – draft report
6 April 2010 39
45. Sorenson C, Drummond M, Kanavos P. Ensuring value for money in health care
the role of health technology assessment in the european union. United Kingdom: World Health
Organisation 2008, on behalf of the European Observatory on Health Systems and Policies;
2008.
46. Sorenson C, Kanavos P, Drummond M. Ensuring value for money in health care: The role of
HTA in the european union. Luxembourg: Financing Sustainable Healthcare in Europe; 2007.
47. Hanney S, Buxton M, Green C, Coulson D, Raftery J. An assessment of the impact of the
NHS health technology assessment programme. Health Technol Assess. 2007 Dec;11(53):iii,iv,
ix-xi, 1-180.
48. Dixon S, Coleman P, Nicholl J, Brennan A, Touch S. Evaluation of the impact of a
technology appraisal process in england: The south and west development and evaluation
committee. J Health Serv Res Policy. 2003 Jan;8(1):18-24.
49. Weatherly H, Drummond M, Smith D. Using evidence in the development of local health
policies. some evidence from the united kingdom. Int J Technol Assess Health Care. 2002
Fall;18(4):771-81.
50. Wathen B, Dean T. An evaluation of the impact of NICE guidance on GP prescribing. Br J
Gen Pract. 2004 Feb;54(499):103-7.
51. Sheldon TA, Cullum N, Dawson D, Lankshear A, Lowson K, Watt I, et al. What's the
evidence that NICE guidance has been implemented? results from a national evaluation using
time series analysis, audit of patients' notes, and interviews. BMJ. 2004 Oct 30;329(7473):999.
52. Paul JE, Trueman P. 'Fourth hurdle reviews', NICE, and database applications.
Pharmacoepidemiol Drug Saf. 2001 Aug-Sep;10(5):429-38.
Health Services Research and Health Technology Assessment – draft report
6 April 2010 40
53. Buxton M. Implications of the appraisal function of the national institute for clinical
excellence (NICE). Value Health. 2001 May-Jun;4(3):212-6.
54. Towse A, Pritchard C. National institute for clinical excellence (NICE): Is economic appraisal
working? Pharmacoeconomics. 2002;20 Suppl 3:95-105.
55. Williams I, Bryan S, McIver S. How should cost-effectiveness analysis be used in health
technology coverage decisions? evidence from the national institute for health and clinical
excellence approach. J Health Serv Res Policy. 2007 Apr;12(2):73-9.
56. Devlin N, Appleby J, Parkin D. Patients' views of explicit rationing: What are the implications
for health service decision-making? J Health Serv Res Policy. 2003 Jul;8(3):183-6.
57. Culyer AJ. NICE's use of cost effectiveness as an exemplar of a deliberative process. Health
Econ Policy Law. 2006 Jul;1(Pt 3):299-318.
58. Oliver A, Mossialos E, Robinson R. Health technology assessment and its influence on
health-care priority setting. Int J Technol Assess Health Care. 2004 Winter;20(1):1-10.
59. Stevens A, Milne R. Health technology assessment in england and wales. Int J Technol
Assess Health Care. 2004 Winter;20(1):11-24.
60. Orvain J, Xerri B, Matillon Y. Overview of health technology assessment in france. Int J
Technol Assess Health Care. 2004 Winter;20(1):25-34.
61. Berg M, van der Grinten T, Klazinga N. Technology assessment, priority setting, and
appropriate care in dutch health care. Int J Technol Assess Health Care. 2004 Winter;20(1):35-
43.
Health Services Research and Health Technology Assessment – draft report
6 April 2010 41
62. Carlsson P. Health technology assessment and priority setting for health policy in sweden.
Int J Technol Assess Health Care. 2004 Winter;20(1):44-54.
63. Lettieri E, Masella C, Nocco U. Budgeting and health technology assessment: First evidence
obtained from proposal forms used to submit the adoption of new technology. Int J Technol
Assess Health Care. 2008 Fall;24(4):502-10.
64. Bodeau-Livinec F, Simon E, Montagnier-Petrissans C, Joel ME, Fery-Lemonnier E. Impact
of CEDIT recommendations: An example of health technology assessment in a hospital
network. Int J Technol Assess Health Care. 2006 Spring;22(2):161-8.
65. Britton M, Jonsson E. Impact of health technology assessments. some experiences of SBU.
Int J Technol Assess Health Care. 2002 Fall;18(4):824-31.
66. Oortwijn WJ, Hanney SR, Ligtvoet A, Hoorens S, Wooding S, Grant J, et al. Assessing the
impact of health technology assessment in the netherlands. Int J Technol Assess Health Care.
2008 Summer;24(3):259-69.
67. Shani S, Siebzehner MI, Luxenburg O, Shemer J. Setting priorities for the adoption of health
technologies on a national level -- the israeli experience. Health Policy. 2000 Dec;54(3):169-85.
68. Gagnon MP, Sanchez E, Pons JM. Integration of health technology assessment
recommendations into organizational and clinical practice: A case study in catalonia. Int J
Technol Assess Health Care. 2006 Spring;22(2):169-76.
69. Andradas E, Blasco JA, Valentin B, Lopez-Pedraza MJ, Gracia FJ. Defining products for a
new health technology assessment agency in madrid, spain: A survey of decision makers. Int J
Technol Assess Health Care. 2008 Winter;24(1):60-9.
Health Services Research and Health Technology Assessment – draft report
6 April 2010 42
70. Moret-Hartman M, van der Wilt GJ, Grin J. Health technology assessment and ill-structured
problems: A case study concerning the drug mebeverine. Int J Technol Assess Health Care.
2007 Summer;23(3):316-23.
71. Packer C, Simpson S, Stevens A, EuroScan: the European Information Network on New
and Changing Health Technologies. International diffusion of new health technologies: A ten-
country analysis of six health technologies. Int J Technol Assess Health Care. 2006
Fall;22(4):419-28.
72. Hartz S, John J. Contribution of economic evaluation to decision making in early phases of
product development: A methodological and empirical review. Int J Technol Assess Health
Care. 2008 Fall;24(4):465-72.
73. van der Wilt GJ, Rovers M, Straatman H, van der Bij S, van den Broek P, Zielhuis G. Policy
relevance of bayesian statistics overestimated? Int J Technol Assess Health Care. 2004
Fall;20(4):488-92.
74. Hutton J, McGrath C, Frybourg JM, Tremblay M, Bramley-Harker E, Henshall C. Framework
for describing and classifying decision-making systems using technology assessment to
determine the reimbursement of health technologies (fourth hurdle systems). Int J Technol
Assess Health Care. 2006 Winter;22(1):10-8.
75. Drummond M, Weatherly H. Implementing the findings of health technology assessments. if
the CAT got out of the bag, can the TAIL wag the dog? Int J Technol Assess Health Care. 2000
Winter;16(1):1-12.
76. May C. Mobilising modern facts: Health technology assessment and the politics of evidence.
Sociol Health Illn. 2006 Jul;28(5):513-32.
Health Services Research and Health Technology Assessment – draft report
6 April 2010 43
77. Hojgaard L. Are health technology assessments a reliable tool in th analysis of the clinical
value of PET in oncology? who audits the auditors? Eur J Nucl Med Mol Imaging. 2003
May;30(5):637-41.
78. Van Tinteren H, Hoekstra OS, Boers M. The need for health technology assessments of
PET. Eur J Nucl Med Mol Imaging. 2003 Oct;30(10):1438,9; author reply 1439.
79. Kristensen FB, Adams E, Briones E, Coburn D, Facey K, Hansen NW, et al. Health
technology assessment of PET in oncology: Re eur J nucl med mol imaging 2003; 30:637-641.
Eur J Nucl Med Mol Imaging. 2004 Feb;31(2):295,7; author reply 297-8; discussion 298.
80. Garrido MV, Kristensen FB, Nielsen CP, Busse R. Health technology assessment and
health policy-making in europe: Current status, challenges and potential. United Kingdom:
World Health Organization on behalf of the European Observatory on Health Systems and
Policies; 2008.
81. Hailey D, Nordwall M. Survey on the involvement of consumers in health technology
assessment programs. Int J Technol Assess Health Care. 2006 Fall;22(4):497-9.
82. Royle J, Oliver S. Consumer involvement in the health technology assessment program. Int
J Technol Assess Health Care. 2004 Fall;20(4):493-7.
83. Pivik J, Rode E, Ward C. A consumer involvement model for health technology assessment
in canada. Health Policy. 2004 Aug;69(2):253-68.
84. Culyer AJ. Involving stakeholders in healthcare decisions--the experience of the national
institute for health and clinical excellence (NICE) in england and wales. Healthc Q.
2005;8(3):56-60.
Health Services Research and Health Technology Assessment – draft report
6 April 2010 44
85. Milewa T. Representation and legitimacy in health policy formulation at a national level:
Perspectives from a study of health technology eligibility procedures in the united kingdom.
Health Policy. 2008 Mar;85(3):356-62.
86. McGregor M, Brophy JM. End-user involvement in health technology assessment (HTA)
development: A way to increase impact. Int J Technol Assess Health Care. 2005
Spring;21(2):263-7.
87. Abelson J, Giacomini M, Lehoux P, Gauvin FP. Bringing 'the public' into health technology
assessment and coverage policy decisions: From principles to practice. Health Policy. 2007
Jun;82(1):37-50.
88. Nielsen CP, Lauritsen SW, Kristensen FB, Bistrup ML, Cecchetti A, Turk E, et al. Involving
stakeholders and developing a policy for stakeholder involvement in the european network for
health technology assessment, EUnetHTA. Int J Technol Assess Health Care. 2009 Dec;25
Suppl 2:84-91.
References
1. Thurston SJ, Craig D, Wilson P, Drummond MF. Increasing decision-makers' access to
economic evaluations: Alternative methods of communicating the information. Int J Technol
Assess Health Care. 2008 Spring;24(2):151-7.
2. Drummond M, Manca A, Sculpher M. Increasing the generalizability of economic evaluations:
Recommendations for the design, analysis, and reporting of studies. Int J Technol Assess
Health Care. 2005 Spring;21(2):165-71.
Health Services Research and Health Technology Assessment – draft report
6 April 2010 45
3. Soto J. Health economic evaluations using decision analytic modeling. principles and
practices--utilization of a checklist to their development and appraisal. Int J Technol Assess
Health Care. 2002 Winter;18(1):94-111.
4. Hjelmgren J, Berggren F, Andersson F. Health economic guidelines--similarities, differences
and some implications. Value Health. 2001 May-Jun;4(3):225-50.
5. Stein K, Fry A, Round A, Milne R, Brazier J. What value health?: A review of health state
values used in early technology assessments for NICE. Appl Health Econ Health Policy.
2005;4(4):219-28.
6. Frew EJ, Wolstenholme JL, Whynes DK. Comparing willingness-to-pay: Bidding game format
versus open-ended and payment scale formats. Health Policy. 2004 Jun;68(3):289-98.
7. FinOHTA, Finnish Office for HTA. EUnetHTA HTA CORE MODEL FOR MEDICAL AND
SURGICAL INTERVENTIONS 1.0R WORK PACKAGE 4. Finland: EUROPEAN NETWORK
FOR HEALTH TECHNOLOGY ASSESSMENT; December 2008.
8. FinOHTA, Finnish Office for HTA. EUnetHTA HTA CORE MODEL FOR DIAGNOSTIC
TECHNOLOGIES 1.0R WORK PACKAGE 4. Finland: EUROPEAN NETWORK FOR HEALTH
TECHNOLOGY ASSESSMENT; December 2008.
9. FinOHTA, Finnish Office for HTA. HTA core model handbook. Finland: EUnetHTA; 2008.
10. Lampe K, Makela M, Garrido MV, Anttila H, Autti-Ramo I, Hicks NJ, et al. The HTA core
model: A novel method for producing and reporting health technology assessments. Int J
Technol Assess Health Care. 2009 Dec;25 Suppl 2:9-20.
Health Services Research and Health Technology Assessment – draft report
6 April 2010 46
11. Pasternack I, Anttila H, Makela M, Ikonen T, Rasanen P, Lampe K, et al. Testing the HTA
core model: Experiences from two pilot projects. Int J Technol Assess Health Care. 2009
Dec;25 Suppl 2:21-7.
12. Saarni SI, Hofmann B, Lampe K, Luhmann D, Makela M, Velasco-Garrido M, et al. Ethical
analysis to improve decision-making on health technologies. Bull World Health Organ. 2008
Aug;86(8):617-23.
13. Autti-Ramo I, Makela M. Ethical evaluation in health technology assessment reports: An
eclectic approach. Int J Technol Assess Health Care. 2007 Winter;23(1):1-8.
14. Hofmann B. Toward a procedure for integrating moral issues in health technology
assessment. Int J Technol Assess Health Care. 2005 Summer;21(3):312-8.
15. Moldrup C. Medical technology assessment of the ethical, social, and legal implications of
pharmacogenomics. A research proposal for an internet citizen jury. Int J Technol Assess
Health Care. 2002 Summer;18(3):728-32.
16. Leys M. Health care policy: Qualitative evidence and health technology assessment. Health
Policy. 2003 Sep;65(3):217-26.
17. Leys M. Health technology assessment: The contribution of qualitative research. Int J
Technol Assess Health Care. 2003 Spring;19(2):317-29.
18. Hofmann BM. Why ethics should be part of health technology assessment. Int J Technol
Assess Health Care. 2008 Fall;24(4):423-9.
19. ten Have H. Ethical perspectives on health technology assessment. Int J Technol Assess
Health Care. 2004 Winter;20(1):71-6.
Health Services Research and Health Technology Assessment – draft report
6 April 2010 47
20. Fulop N, Allen P, Clarke A, Black N. From health technology assessment to research on the
organisation and delivery of health services: Addressing the balance. Health Policy. 2003
Feb;63(2):155-65.
21. NCCHTA, NIHR Coordinating Centre for HTA. EUnetHTA HTA ADAPTATION TOOLKIT
WORK PACKAGE 5. UK: EUROPEAN NETWORK FOR HEALTH TECHNOLOGY
ASSESSMENT; October 2008.
22. NCCHTA, NIHR Coordinating Centre for HTA.
Glossary of HTA adaptation terms. . November 2007.
23. Rosten C, Chase DL, Hicks NJ, Milne R, European network for Health Technology
Assessment (EUnetHTA). Enhancing understanding: The development of a glossary of health
technology assessment adaptation terms. Int J Technol Assess Health Care. 2009 Dec;25
Suppl 2:42-7.
24. Turner S, Chase DL, Milne R, Cook A, Hicks NJ, Rosten C, et al. The health technology
assessment adaptation toolkit: Description and use. Int J Technol Assess Health Care. 2009
Dec;25 Suppl 2:37-41.
25. Turner S, Chase DL, Milne R, Cook A, Hicks NJ, Rosten C, et al. The adaptation of health
technology assessment reports: Identification of the need for, and development of, a toolkit to
aid the process. Int J Technol Assess Health Care. 2009 Dec;25 Suppl 2:28-36.
26. Street JM, Braunack-Mayer AJ, Facey K, Ashcroft RE, Hiller JE. Virtual community
consultation? using the literature and weblogs to link community perspectives and health
technology assessment. Health Expect. 2008 Jun;11(2):189-200.
27. Bridges JF, Jones C. Patient-based health technology assessment: A vision of the future. Int
J Technol Assess Health Care. 2007 Winter;23(1):30-5.
Health Services Research and Health Technology Assessment – draft report
6 April 2010 48
28. Potter BK, Avard D, Graham ID, Entwistle VA, Caulfield TA, Chakraborty P, et al. Guidance
for considering ethical, legal, and social issues in health technology assessment: Application to
genetic screening. Int J Technol Assess Health Care. 2008 Fall;24(4):412-22.
29. Lehoux P, Williams-Jones B. Mapping the integration of social and ethical issues in health
technology assessment. Int J Technol Assess Health Care. 2007 Winter;23(1):9-16.
30. Hailey D. Toward transparency in health technology assessment: A checklist for HTA
reports. Int J Technol Assess Health Care. 2003 Winter;19(1):1-7.
31. Porzsolt F, Kajnar H, Awa A, Fassler M, Herzberger B. Validity of original studies in health
technology assessment reports: Significance of standardized assessment and reporting. Int J
Technol Assess Health Care. 2005 Summer;21(3):410-3.
32. Hutton J, Trueman P, Facey K. Harmonization of evidence requirements for health
technology assessment in reimbursement decision making. Int J Technol Assess Health Care.
2008 Fall;24(4):511-7.
33. Poole C, Agrawal S, Currie CJ. Let cost effectiveness models be open to scrutiny. BMJ.
2007 Oct 13;335(7623):735.
34. Maynard A. Transparency in health technology assessments. BMJ. 2007 Mar
24;334(7594):594-5.
35. Simpson S, Packer C, Carlsson P, Sanders JM, Ibarluzea IG, Fay AF, et al. Early
identification and assessment of new and emerging health technologies: Actions, progress, and
the future direction of an international collaboration--EuroScan. Int J Technol Assess Health
Care. 2008 Fall;24(4):518-25.
Health Services Research and Health Technology Assessment – draft report
6 April 2010 49
36. Wild C, Langer T. Emerging health technologies: Informing and supporting health policy
early. Health Policy. 2008 Aug;87(2):160-71.
37. Murphy K, Packer C, Stevens A, Simpson S. Effective early warning systems for new and
emerging health technologies: Developing an evaluation framework and an assessment of
current systems. Int J Technol Assess Health Care. 2007 Summer;23(3):324-30.
38. Douw K, Vondeling H. Selection of new health technologies for assessment aimed at
informing decision making: A survey among horizon scanning systems. Int J Technol Assess
Health Care. 2006 Spring;22(2):177-83.
39. Simpson S, Hyde C, Cook A, Packer C, Stevens A. Assessing the accuracy of forecasting:
Applying standard diagnostic assessment tools to a health technology early warning system. Int
J Technol Assess Health Care. 2004 Summer;20(3):381-4.
40. Douw K, Vondeling H, Eskildsen D, Simpson S. Use of the internet in scanning the horizon
for new and emerging health technologies: A survey of agencies involved in horizon scanning. J
Med Internet Res. 2003 Jan-Mar;5(1):e6.
41. Douw K, Vondeling H, Oortwijn W. Priority setting for horizon scanning of new health
technologies in denmark: Views of health care stakeholders and health economists. Health
Policy. 2006 May;76(3):334-45.
42. Douw K, Vondeling H, Sorensen J, Jorgensen T, Sigmund H. "The future should not take us
by surprise": Preparation of an early warning system in denmark. Int J Technol Assess Health
Care. 2004 Summer;20(3):342-50.
43. Lundberg J, Brommels M, Skar J, Tomson G. Measuring the validity of early health
technology assessment: Bibliometrics as a tool to indicate its scientific basis. Int J Technol
Assess Health Care. 2008 Winter;24(1):70-5.
Health Services Research and Health Technology Assessment – draft report
6 April 2010 50
44. Simpson S, Wild C. On the horizon european newsletter on new and emerging health
technologies. EUnetHTA; 2008.
45. Wild C, Simpson S, Douw K, Geiger-Gritsch S, Mathis S, Langer T. Information service on
new and emerging health technologies: Identification and prioritization processes for a european
union-wide newsletter. Int J Technol Assess Health Care. 2009 Dec;25 Suppl 2:48-55.
46. Perleth M, Luhmann D, Gibis B, Droste S. Rapid assessments"--quick evaluation of medical
technology. Gesundheitswesen. 2001 Mar;63 Suppl 1:S79-84.
47. Warren V. Health technology appraisal of interventional procedures: Comparison of rapid
and slow methods. J Health Serv Res Policy. 2007 Jul;12(3):142-6.
48. Hailey D, Corabian P, Harstall C, Schneider W. The use and impact of rapid health
technology assessments. Int J Technol Assess Health Care. 2000 Spring;16(2):651-6.
49. Watt A, Cameron A, Sturm L, Lathlean T, Babidge W, Blamey S, et al. Rapid reviews versus
full systematic reviews: An inventory of current methods and practice in health technology
assessment. Int J Technol Assess Health Care. 2008 Spring;24(2):133-9.
50. Ehlers L, Vestergaard M, Kidholm K, Bonnevie B, Pedersen PH, Jorgensen T, et al. Doing
mini-health technology assessments in hospitals: A new concept of decision support in health
care? Int J Technol Assess Health Care. 2006 Summer;22(3):295-301.
51. Folkersen J, Pedersen PH. Attitudes to the use of a decision support method when
introducing new medical technology at the university hospital of copenhagen. Ugeskr Laeger.
2006 May 22;168(21):2069-74.
Health Services Research and Health Technology Assessment – draft report
6 April 2010 51
52. Hummel JM, van Rossum W, Verkerke GJ, Rakhorst G. Medical technology assessment:
The use of the analytic hierarchy process as a tool for multidisciplinary evaluation of medical
devices. Int J Artif Organs. 2000 Nov;23(11):782-7.
53. Pearson S, Littlejohns P. Reallocating resources: How should the national institute for health
and clinical excellence guide disinvestment efforts in the national health service? J Health Serv
Res Policy. 2007 Jul;12(3):160-5.
54. Walker S, Palmer S, Sculpher M. The role of NICE technology appraisal in NHS rationing.
Br Med Bull. 2007;81-82:51-64.
55. Maynard A, Bloor K, Freemantle N. Challenges for the national institute for clinical
excellence. BMJ. 2004 Jul 24;329(7459):227-9.
56. HAS, French National Authority for Health. EUnetHTA web-based toolkit to facilitate
european collaboration on evidence generation on promising health
technologies WORK PACKAGE 7-STRAND A. . DECEMBER 2008.
57. Quentin F, Carbonneil C, Moty-Monnereau C, Berti E, Goettsch W, Lee-Robin SH, et al.
Web-based toolkit to facilitate european collaboration on evidence generation on promising
health technologies. Int J Technol Assess Health Care. 2009 Dec;25 Suppl 2:68-74.
58. Carbonneil C, Quentin F, Lee-Robin SH, European network for Health Technology
Assessment (EUnetHTA). A common policy framework for evidence generation on promising
health technologies. Int J Technol Assess Health Care. 2009 Dec;25 Suppl 2:56-67.
59. McCabe C, Claxton K, O'Hagan A. Why licensing authorities need to consider the net value
of new drugs in assigning review priorities: Addressing the tension between licensing and
reimbursement. Int J Technol Assess Health Care. 2008 Spring;24(2):140-5.
Health Services Research and Health Technology Assessment – draft report
6 April 2010 52
60. Chalkidou K, Hoy A, Littlejohns P. Making a decision to wait for more evidence: When the
national institute for health and clinical excellence recommends a technology only in the context
of research. J R Soc Med. 2007 Oct;100(10):453-60.
61. Hutton J, Trueman P, Henshall C. Coverage with evidence development: An examination of
conceptual and policy issues. Int J Technol Assess Health Care. 2007 Fall;23(4):425-32.
62. Tunis SR, Chalkidou K. Coverage with evidence development: A very good beginning, but
much to be done. commentary to hutton et al. Int J Technol Assess Health Care. 2007
Fall;23(4):432-5.
63. Oortwijn W, Banta HD, Cranovsky R. Introduction: Mass screening, health technology
assessment, and health policy in some european countries. Int J Technol Assess Health Care.
2001 Summer;17(3):269-74.
64. Gray JA. Evidence-based screening in the united kingdom. Int J Technol Assess Health
Care. 2001 Summer;17(3):400-8.
65. Banta HD. Health policy, health technology assessment, and screening in europe. Int J
Technol Assess Health Care. 2001 Summer;17(3):409-17.
66. Faisst K, Schilling J, Koch P. Health technology assessment of three screening methods in
switzerland. Int J Technol Assess Health Care. 2001 Summer;17(3):389-99.
67. Jonsson E, Banta HD, Schersten T. Health technology assessment and screening in
sweden. Int J Technol Assess Health Care. 2001 Summer;17(3):380-8.
68. Banta HD, Oortwiin W. Health technology assessment and screening in the netherlands:
Case studies of mammography in breast cancer, PSA screening in prostate cancer, and
ultrasound in normal pregnancy. Int J Technol Assess Health Care. 2001 Summer;17(3):369-79.
Health Services Research and Health Technology Assessment – draft report
6 April 2010 53
69. Mousiama T, Loakimidou S, Largatzi E, Kaitelidou D, Liaropoulus L. Health technology
assessment in the area of prevention: Selected screening cases in greece. Int J Technol Assess
Health Care. 2001 Summer;17(3):338-57.
70. Perleth M, Busse R, Gibis B, Brand A. Evaluation of preventive technologies in germany:
Case studies of mammography, prostate cancer screening, and fetal ultrasound. Int J Technol
Assess Health Care. 2001 Summer;17(3):329-37.
71. Vermeulen V, Coppens K, Kesteloot K. Impact of health technology assessment on
preventive screening in belgium: Case studies of mammography in breast cancer, PSA
screening in prostate cancer, and ultrasound in normal pregnancy. Int J Technol Assess Health
Care. 2001 Summer;17(3):316-28.
72. Gulacsi L. The time for cost-effectiveness in the new european union member states: The
development and role of health economics and technology assessment in the mirror of the
hungarian experience. Eur J Health Econ. 2007 Jun;8(2):83-8.
73. Gulacsi L, Boncz I, Drummond M. Issues for countries considering introducing the "fourth
hurdle": The case of hungary. Int J Technol Assess Health Care. 2004 Summer;20(3):337-41.
74. Gibis B, Artiles J, Corabian P, Meiesaar K, Koppel A, Jacobs P, et al. Application of
strengths, weaknesses, opportunities and threats analysis in the development of a health
technology assessment program. Health Policy. 2001 Oct;58(1):27-35.
75. Antes G, Briones E, Britton M, Burnand B, Engel G, Gallo P, et al. Education and support
networks for assessment of health interventions. working group 5 report. Int J Technol Assess
Health Care. 2002 Spring;18(2):423-46.
Health Services Research and Health Technology Assessment – draft report
6 April 2010 54
76. Lehoux P, Battista RN, Granados A, Gallo P, Tailliez S, Coyle D, et al. International master's
program in health technology assessment and management: Assessment of the first edition
(2001--2003). Int J Technol Assess Health Care. 2005 Winter;21(1):104-12.
77. Catalan Agency for Health Technology Assessment and Research. Catalan Health Service.
Department of Health Autonomous Government of Catalonia.
EUnetHTA work package 8. EUnetHTA handbook on health technology assessment capacity
building. Barcelona (Spain): EUROPEAN NETWORK FOR HEALTH TECHNOLOGY
ASSESSMENT; 2008.
78. Moharra M, Kubesch N, Estrada MD, Parada T, Cortés M, Espallargues M, et al. EUnetHTA
survey report on HTA organisations. . 2008.
79. Shepherd J, Briggs J, Payne L, Packer C, Kerridge L, Ashton-Key M. Setting the future
policy agenda for health technology assessment: A specialty mapping approach. Int J Technol
Assess Health Care. 2007 Fall;23(4):405-13.
80. Oortwijn WJ, Vondeling H, van Barneveld T, van Vugt C, Bouter LM. Priority setting for
health technology assessment in the netherlands: Principles and practice. Health Policy. 2002
Dec;62(3):227-42.
81. Chase D, Milne R, Stein K, Stevens A. What are the relative merits of the sources used to
identify potential research priorities for the NHS HTA programme? Int J Technol Assess Health
Care. 2000 Summer;16(3):743-50.
82. Townsend J, Buxton M, Harper G. Prioritisation of health technology assessment. the
PATHS model: Methods and case studies. Health Technol Assess. 2003;7(20):iii, 1-82.