central dimensions of clinical practice evaluation: efficiency, appropriateness and effectiveness -...

Journal of Evaluation in Clinical Practice, 2, 2, 131 -1 52

Central dimensions of clinical practice evaluation: efficiency, appropriateness and effectiveness - II Andrew Miles BSc (Hons) MSc MPhil PhD‘, Declan O’Neill MPH FAFPHM MFPHM2 and Andreas Poiychronis MB BCh3 ‘Deputy Director, Centre for t h e Advancement of Clinical Practice, European Institute of Health and Medical Sciences, University of Surrey and Visiting Professor in Health Services Research, Faculty of Science & Health, University of East London, London, UK ‘Director of Health Improvement, New South Wales Health Department, Sydney, Australia 3Senior House Physician, Department of Medicine, Guy’s Hospital, Guy’s and St. Thomas’ Hospitals’ N H S Trust, London, UK

Correspondence Professor A. Miles Deputy Director Centre for the Advancement of

Clinical Practice European Institute for Health and

Medical Sciences University of Surrey Stag Hill Campus Guildford GU2 5XH UK

Keywords: clinical guidelines, clinical effectiveness, evidence-based medicine, practice variations

Accepted for publication: 21 May 1996

Abstract That a treatment selected for a given condition works, or that i t works better than alternative treatments, or that i t was selected because it works as well as but is cheaper than alternative treatments, should be of pivotal concern to clinicians and is of central concern to patients and to health care managers. Attempts to address these concerns have resulted in what is now widely termed the ‘effectiveness movement’. The protagonists of the movement have been concerned to create a culture ofevaluation and inquiry within which the formulation of evidence-based clinical guidelines and their introduction into routine practice have played a prominent part. The need to ensure cosf effectiveness of clinical intervention has been at least as emphasized as the need to ensure the clinical effectiveness of health care interventions. Although cost-effectiveness analyses are now an indispensable feature of practice guideline formulation and treatment evaluation, few studies have examined any deterioration in patient outcome associated with successful cost containment. An adequate understanding of the concept of clinical effectiveness and the associated aims of the ‘effectiveness movement’ is central to an understanding of the future nature and extent of health service provision, not simply in the UK but also internationally. Having examined the concepts of efficiency and appropriateness previously (O’Neill, Miles & Polychronis 1996, Journal of Evaluation in Clinical Practice 2 , 13-27) we move in this second of two articles to a detailed explanation of the concept of effectiveness, and to an examination of the derivation and use of clinical practice guidelines, concluding with a consideration of the role of practice guidelines in ensuring the cost effectiveness of health care intervention. The reservation is expressed that a ‘guidelines culture’, when established, will be manipulated by health care commissioners for largely political purposes, creating a systematic bias in the purchasing process that will actively disadvantage a range of patient groups.

(C; 1996 Blackwell Science 131

A. Miles eta/

Introduction

The case for introduction of treatments of proven effectiveness into routine clinical practice requires automatic consideration in modem health services, and this is where the complexity begins. To address this complexity, we pose four questions: (i) why is clinical effectiveness research necessary? (ii) what are clinical guidelines and how are they derived? (iii) can effective care be afforded? (iv) how can effective care be implemented? We structure this second part of our review accordingly (the first part being the paper by O’Neill, Miles & Polychronis 1996).

Why is effectiveness research necessary?

Because of the phenomenon of wide variations in clinical practice and wide variations in clinical outcome.

Maynard (1994) has pointed out that medical practice and the health care reform systems share a principal characteristic - the absence of an adequate scientific base. As a consequence, Maynard (1994) says, clinicians intervene in different ways when confronted with patients of similar age, sex and condition. In advancing such a contention, Maynard (1994) echoes the view of the Institute of Medicine: ‘where considerable research has been done and good methods have been applied to analyse it, honest clinicians may come to different conclusions using the same evidence’ (Field & Lohr 1990). The direct con- sequences of this phenomenon are the well-recognized differences in medical opinion which frequently appear to be contradictory, with variations in clinical practice causing increasing concern.

Definitive variations in clinical practice

‘Startling variations’ (Hopkins 1990) have been identified in medical and surgical practice between doctors and between geographical regions (Wennberg & Git- telsohn 1982; UK-TIA Study Group 1983; Chassin et al. 1986a; Wennberg et al. 1987, 1989; Pernn et al. 1989; Wilken et al. 1989). When clinical outcome was examined in this context, geographical differences in mortality were identified, with the mortality associated with conditions recognized as appropriate for medical intervention being three times higher in some

areas of England and Wales than in others (Charlton et al. 1983).

This startling variation in mortality was shown in definitive fashion by Charlton & Lakhani (1986) in a study which examined the mortality risk associated with hernia, gallstones and appendicitis in a population age range of 5-64 years. A 10-fold variation between district health authorities in England and Wales was evident from the results. A wide variation was similarly observed in ‘mortality from asthma examined in a population range of 5-44 years.

Study of variations in clinical practice in terms of surgical procedure has demonstrated wide variation in the rates at which surgical procedures are performed. Saunders et af. (1989), for example, observed a 2-fold variation in the tonsillectomy rate between two adja- cent small areas in Scotland. Concern about this matter had been expressed in the 1930s and 1940s when election to tonsillectomy was examined in different areas of Kent (Glover 1938; Hampton 1989). Additional study of election to surgery for glue ear demonstrated that a child in Oxford, Wessex or South West Thames region was twice as likely to be elected for surgery than a child in Trent or South Western region, with essentially similar variations in surgery rates being observed elsewhere. When hysterectomy rates were examined, 3-fold differences were identifiable between general practices in Oxfordshire, with cardiac surgery rates varying 5-fold between English regions. The cholecystectomy rate in Canada has been observed to be 5 times that reported in the UK, and hysterectomy is 7 times more common in the USA than in the UK (King’s Fund Institute 1986; Hamp- ton 1995). Variations in elective surgery rates have also been reported for caesarian section, cataract extraction and for a multiplicity of other surgical procedures and hospital admissions (Saunders et al. 1989). Continued variability extends through pre- scribing practices (Gosney & Tallis 1984), surgical and anaesthetic practices (Buck et al. 1987) and consultant supervision of junior staff (Charlton et al. 1983).

A further interesting example is provided by the Randomised Intervention Treatment of Angina (RITA) Trial, which demonstrated that in one centre 50% of patients undergoing angiography proceeded to coronary artery bypass grafting of percutaneous transluminal coronary angioplasty, while at a different centre 85% required one or other form of intervention

132 0 1996 Blackwell Science, Journal of Evaluafion in Clinical Practice, 2. 2, 131-152

Central dimensions of clinical practice evaluation - It

(Hampton 1995; Henderson et al. 1995). As Hampton (1995) points out, the scientific data relating to the value of intervention were available to all the physicians involved in the trial, but widely differing views were found in terms of indications for proper use of coronary angiography. Gray & Hampton (1994) demonstrated marked differences in the use of coronary arteriography in a single region within the UK. In one centre in Trent examined by these authors, patients undergoing angiography were observed to have essentially ‘mild’ symptoms, to be on simple treatment, to have treadmill tests that were not strongly positive, and who were therefore judged to have relatively mild coronary disease. At a separate centre within the Trent region, Gray & Hampton (1994) observed angiography being performed in patients with more severe symptoms, who were treated with more anti-anginal drugs, who were more likely to have strongly positive exercise tests, and were found to have more extensively diseased coronary arteries.

Population risk and the differing needs of individual patients will, of course, contribute to the incidence of clinical conditions and to variations in clinical practice directly related to variations in the clinical status and needs of individual patients. Shaw ( 1 989) is convinced, however, that the extent of variation in clinical outcome is related to definitive differences in the technical effectiveness of medical care. Hopkins (1990) is in general agreement with this contention and, while accepting that variation in clinical practice will be generated by differing patients’ needs, suggests that this variation may normally be considered to be small.

Dependence of clinical effectiveness on the ‘style’ of practice

The extraordinary variations in clinical practice that have been consistently demonstrated in the literature continue to occur not only in the UK but also internationally (Krakauer et al. 1995) and are likely to be causally related to corresponding variations in the style of clinical practice and as such may be independent of the objective clinical appropriateness of intervention (Hopkins 1989, 1990).

What, then, is the aetiology of this differing ‘style’ of practice? McIntyre (1985) has pointed out that doctors work on the basis of personal, rather than

collective, responsibility and that this rnodus operundi acts to reinforce the belief in a personal approach. As a consequence, doctors will treat in an individual manner and in a way that they, personally, think best. They can frequently be observed to do so despite the fact that there may be multiple approaches to the same problem and where consensus/evidence-based clinical guidelines exist. It is now generally acknowledged that this problem has been complicated by the difficulties in maintaining adequate familiarity with a rapidly increasing medical literature. It is not difficult to see, therefore, how this prevailing situation generates variations in clinical practice and how it con- tributes to delays in the implementation of medical research evidence that can result in suboptimal care for patients (Haines & Jones 1994).

Variations in the ‘style’ of clinical practice and their relationship to the exercise of clinical judgement

The critics of the new ‘scientific medicine’ frequently contest that the effectiveness movement represents science at all (Charlton 1993), emphasizing the cen- trality of the exercise of clinical judgement in the context of the immediate experience of the individual patient, to whom the application of general research evidence may be judged to be inappropriate. The problem of applicability is a real one and is derived from the methodological limitations of the randomized controlled trial (Charlton 1996). However, evidence exists to show that a great many clinicians will invoke the term clinical judgement if specifically asked to describe the reasoning they have employed in reaching a clinical decision (Feinstein 1967), simply because it is unlikely that they have employed a for- mally identified method of design and evaluation. This is an indefensible position which some colleagues consider to represent a failure of medical education. So what, then, do the words design and evaluation mean to doctors in the context of the routine care of everyday patients? In a series of excellent articles, which still retain their original importance, Feinstein (1964a4, 1967) argues that acts of patient care are analogous in many ways to a laboratory experiment. Every patient undergoing treatment, he says, does so in a ‘baseline state’, and manifests a clinical outcome subsequent to the selected clinical intervention. Routine clinical activities, unlike routine laboratory

0 1996 Blackwell Science, Journal ol Evaluafion in Clinical Pracfice, 2 , 2. 131-152 133

A. Miles eta/

experiments, Feinstein (1964a-d) argues, lack both a concurrent control group and an innovative goal. In the clinical situation, the ‘control’ comparison is derived directly from the doctor’s knowledge of former patients who have presented with an essentially similar clinical picture. The ‘goal’ of the experiment, this author says, derives from the doctor’s aim to repeat (or exceed) in current patients the clinical outcome that he has achieved in former patients. An essential difference, Feinstein (1967) argues, is that these ‘experiments of patient care frequently fail to be invested with the extent of scientific attention that would characterize the typical laboratory experiment’. In designing an experiment, a laboratory scientist would refer to the accumulated body of knowledge in the literature, and refer directly to recently published research and ongoing work of independent scientists. Clinicians should similarly refer to the findings of evidence-based medicine (Polychronis et al. 1996) and to the recommendations of clinical practice guidelines.

I t might be argued that, in the latter half of the present century, doctors have failed in large measure to recognize the acts of prediction and intervention in patient care as essential scientific challenges, and have instead been disproportionately concerned with investigating the pathobiological basis and aetiology of disease. The primary duty of a clinician is to change nature, rather than explain how nature works, ‘but the basic ideas and predictive principles of patient care interventions have been downgraded and regarded as secondary, applied activity, unworthy of first-rate minds’ (Feinstein 1994). Feinstein ( 1 967) urged clinicians to facilitate the development of scientifically based clinical practice and to study clinical phenom- ena in a direct fashion, with accompanying specifica- tion of the importance of varying clinical data and development of methods with which to improve scientific integrity. He additionally urged clinicians to proceed to identification of appropriate taxonomic methodologies for data classification, and to the development of such methods de now, where necessary. Such activities would develop intellectual models and pragmatic methods for articulation of the clinical process, its recapitulation and utilization of its results in quantitative analysis.

Feinstein ( 1 994) is nevertheless clear that modem clinical judgement retains the pivotal importance in

patient care that it has historically held, but that its scientific challenges in data and taxonomy have been essentially ignored or overlooked in parallel with a much greater emphasis on quantitative models deriving from non-clinical sources. This author emphasizes that modem clinical investigators have the opportunity to explore a multiplicity of areas of clinical practice amenable to basic scientific investigation which are intellectually exciting and in real terms protective of the professional art. In this way, doctors ‘can probe their own clinical knowledge and experience, inti- mately examine the events and relationships of patient care, identify the pertinent evidence, organise suitable taxonomic classifications, develop the clinometric indexes and do appropriate scientific analyses for the unique and fundamental characteristics of clinical activities that still occur as clinical judgement’ (Fein- stein 1994). The central utility of clinical effectiveness research data in this context is clear.

Delays in implementation of research evidence into routine clinical practice

Where definitive research evidence exists of proven applicability to given patient groups, delays in its implementation into routine clinical practice can result in suboptimal care of patients (Haines & Jones 1994) and therefore to a probability of suboptimal clinical outcome. There is a well-documented history in medicine of spectacular delays between the discovery of unequivocally effective treatments and their routine use. The effectiveness of lemon juice in the prevention of scurvey, demonstrated by Lancaster in 1601, did not receive confirmation by Lind until 1747 and failed to be adopted fully by the Royal Navy until 1795 and by the merchant marine until 1865 (Mos- teller 1981; Haines & Jones 1994). A more recent example is afforded by the 13-year delay between the publication of the results from cumulative meta-analysis of randomized controlled trials of the clinical effectiveness of thrombolysis in acute myocardial infarction and recommendation of this intervention by the clinical press (Antman et al. 1992; Haines & Jones 1994). Further examples are provided by the delay in use of dexamethasone in the management of premature labour (Donaldson 1992) and the use of thromboprophylaxis in orthopaedic surgery (Laverick et al. 1991).

134 0 1996 Blackwell Science, Journal of Evaluation in Clinical Practice, 2 , 2 , 131 -1 52

Central dimensions of clinical practice evaluation - II

In continuity with such historical tradition, changes in modem clinical behaviour in direct response to definitive research evidence of proven relevance to routine practice have yet to be documented. Penn & Steer (1991) have reported the results of a survey in which consultant obstetricians were interviewed in relation to their willingness to change practice based on research evidence. These authors found that, although 83% of these clinicians responded positively and appeared committed to introducing definitive results deriving from randomized controlled trials, little evidence existed to show that real change had occurred in their everyday practice. Prophylactic corticosteroid administration in the management of premature labour was demonstrated more than a decade ago to result in significant decreases in neo- natal mortality and respiratory distress (see Chalmers 1993), but Donaldson (1992) has shown that, in the Northern region at least, steroids were administered in less than 20% of cases where a clinical indication existed. A similarly low percentage of obstetricians were shown by the Maternity Services Working Party (1992) to employ antibiotic prophylaxis at caesarian section. Early induction in pre-labour rupture of the membranes - a practice strongly associated with increased caesarian section rates - is still used in a great many hospitals (Chalmers et al. 1989 a,b; Han- nah 1993), providing yet another example of suboptimal care deriving from a failure to conform personal professional practice to acknowledged clinical standards.

Recent reports demonstrate continuing deficiencies of this nature in clinical practice. Paterson-Brown et al. (1995) reported that 37% of English obstetric units surveyed in their study demonstrated a lack of access to either Effective Care in Pregnancy and Childbirth or the Oxford Database of Perinatal Trials. Significant differences were identified by these authors in such availability when non-teaching district general hospitals (DGHs) and teaching hospitals were directly compared [33/74 (45%) versus 3/24 (125) respectively; P = 0.011. The authors further demonstrated that Effective Care in Pregnancy and Childbirth was available in 51 YO of DGHs and 79% of teaching hospitals, compared with the availability of the Oxford Database of Perinatal Trials which was available in only 16% and 62%, respectively. Interestingly, Paterson-Brown and associates identified ‘non-awareness’, ‘expense’

and ‘perceived lack of need’ as the reasons underlying the unavailability.

Implications for access of patients to ‘best’ care

The results of this study have immediate implications for patient access to so-called ‘best’ care. Indeed, while non-teaching and teaching hospitals show different ‘cultures’, and frequently demonstrate different approaches to general operational management as well as to patient management, all patients have equal right of access to effective care, irrespective of the nature of the health care organization. Paterson- Brown et al. (1995) were able to conclude not only that there has been inadequate dissemination of medical knowledge in obstetrics but also that there exists a reluctance of obstetricians to conform their personal clinical practice to definitive medical evidence from meta-analysis of randomized controlled trials of proven relevance to routine clinical care.

Such results are disturbing and reflect some intrin- sic elements of the prevailing culture that act to maintain the status quo with all of its characteristics of variability in practice and inadequacies in sound, scientific base. Such a clinical scenario has increasing implications for clinical risk management. The association of obstetricians with ‘unscientific practices’ is well recognized (House of Commons Health Com- mittee 1992) and Chalmers (1993) has warned that obstetricians may be medico-legally compromised by any identifiable failure to exercise clinical judgement in accordance with definitive medical evidence of proven relevance to routine clinical care.

Doctors are, of course, quintessentially determinist by nature and disproportionately reliant on personal experience and clinical judgement rather than on clinical research evidence, for the reasons we have discussed, so that the process of medical decision making is characteristically ‘essentially interpretive and therefore irreducible to probabilities, no matter how rigorously derived’ (Tanenbaum 1994). As a consequence of this situation, many doctors will challenge medical research evidence on the basis that it lacks scientific integrity, is idealist, is transitory or represents a combination of these things (Hunter 1995). At the present time, we wonder for how much longer the delays in the delivery of effective clinical care of proven relevance to given patient categories

10 1996 Blackwell Science. Journal ol Evaluafion in Clinical Pracfice. 2. 2, 131-152 135

A. Miles eta/

will be tolerated by patients (Rigge 1996; Kelson 1996) and, indeed, by the State itself.

State enquiry into clinical practice

In this context it is interesting to consider the actions of the Select Committee on Science and Technology of the House of Lords in summoning evidence on 31 January 1995 from Dr Iain Chalmers, Director of the UK Cochrane Centre and Professor Trevor Sheldon, Director of the NHS Centre for Reviews and Dis- semination. The evidence given ran to some 15 printed pages. Among the most spectacular pieces of evidence given to the Select Committee were those delivered by Chalmers himself, which were later sensationally reported to the general public by the press (Lightfoot & Rogers 1995; Stuttaford 1995). Let us examine the Hansard itself:

‘When I was a clinician I was fairly homfied to find out that I was actually killing my patients by using a technique which I had been advised to use in medical school. That was an extremely sobering experience: when you take on trust what your teachers have been telling you and you find through trying to apply it that your patients are suffering and dying unnecessarily. I imagine that most of the doctors here have been upset a t some stage in their careers to find something which they really believed in because they had read it in a textbook turned out not only not to be useful, but actually harmful. Once you have had that experience, you start to ask questions about where the information being delivered to us practitioners comes from ... As you saw from the Antman study - which has been copied to you and presented in part by Trevor Sheldon: because reviewers have not used scientific methods, advice on some lifesaving therapies has been delayed for more than a decade while other treatments have been recommended long after controlled research has shown them to be harmful. These researchers looked at American textbooks and review articles; but here is one from my own home town, published in 1987. The statement that the clinical benefits of thrombolysis whether expressed as improved patient survival or preservation of left ventricular function remain to be established is lethal [The sentence quoted is ‘lethal’ only in the

sense that it might discourage practitioners from administering thrombolysis, the clinical benefits of which were in fact established in 1983 - Editors]. I think the 2nd Edition of the Oxford Textbook of Medicine has sold about 50000 or 60000 copies. Goodness knows how many hundreds of thou- sands of people have read that information but that is what is available to us. When members of the public are exposed to the new phenomenon called evidence-based health care, they find it astonishing that we health professionals have let our house get into such a muddle.. . When Archie Cochrane was a medical officer in a prisoner of war camp, he received a leaflet urging him to defend his clinical freedom. His comment was that he would have willingly traded some of his clinical freedom for some reliable information about how to treat his patients. . . The Cochrane Centre had as its opening statement Archie Cochrane’s criticism that the medical profession’s house was not in order. He could have applied it equally well to nursing, to physiotherapy or to speech therapy. All of the health professionals are in a similar mess.. .’

(House of Lords 1995) These concerns do not isolate Chalmers and Sheldon, although few colleagues would feel comfortable in articulating them so spectacularly.

What are clinical guidelines and how are they derlved?

Most doctors acknowledge that clinical standards, clinical audit and the ability to define clinical outcomes are ‘robust values in the development of medical practice’ and are ‘concepts in touch with the real world’, giving the medical profession ‘something to aspire to’ (Smith 1994). The term ‘evidence-based medicine’, however, remains unfamiliar and poorly understood, and the advent of clinical practice guidelines has generated no small anxiety within the medical profession. The utility of clinical guidelines remains a matter of considerable debate and con- troversy (Delamothe 1993; Hart 1993; 1994, Hurwitz 1995) and reactions are typically varied. The reaction of the traditional elites is characterized by Sibylline warnings and dramatic exhortations. Indeed, as Dans (1994) has said of clinical guidelines:

136 0 1996 Blackwell Science, Journal of Evaluation in Clinical Practice. 2 , 2 , 131-152


They have been hailed as the salvation from the wide variations in medical care and simulta- neously lamented as harbingers of the ruination of Medicine as we know it ... those who are fearful focus on the hazards of navigating between the Charybdis of proliferating guidelines and the Scylla of third-party payers and lawyers read to use the rules against them.

but provides useful context in pointing out that guideline development and availability needs to be kept in perspective: ‘they are neither saviours nor villains, just imperfect attempts to care for patients better. Their development can act as beacons into a still vast scientific unknown’.

Importance of terminology

Terminology is of no small importance in fostering clinical guideline acceptance and implementation. The use of the word protocol, for example, is considered unwise in that a protocol is interpreted by most clinicians as an unduly restrictive manual of clinical practice which directly affects the exercise of clinical judgement (Hart 1993). The word recommendation may ‘lack conviction’, and the word policy may be unsuitable if that policy has ‘a history of being incorporated into doctors’ contractual obligations’ (Onion & Walley 1995). Dickinson (1995) has suggested that the term clinical guidelines may have a ‘negative ring’, invoking images of ‘medical cook- books’ in clinicians’ minds. As an alternative, Dick- inson (1 995) advances the description expert practice support. We join with Onion & Walley (1995) in pre- ferring the term clinical practice guidelines.

General characteristics of clinical practice guidelines

Comparison of the definition of clinical standards by Irvine & Donaldson (1994) with the now widely accepted definition of clinical guidelines by the Insti- tute of Medicine (1992) shows a quintessential similarity. Clinical standardr have been defined by the former investigators as ‘complex aggregations of criteria built up into a series of statements describing good clinical practice’ and clinical guidelines have been defined by the latter authors as ‘systematically developed statements to assist practitioner and patient decisions about appropriate healthcare for specific

clinical circumstances’. Both entities are highly inter- related with the common aim of ensuring and increasing the effectiveness and appropriateness of clinical care and the efliciency with which such improvements in the clinical service are delivered to patients.

Levels of flexibility in clinical decision making: clinical standards, clinical guidelines and clinical options

Guidelines typically consist of statements describing different aspects of the patient’s clinical presentation and, with reference to these, indicate the nature of intervention to be followed or, at the very least, considered. In considering medical decision making, Eddy (1990) has described the characteristics and utility of clinical standards, clinical guidelines and clinical options. Eddy (1990) employs the term clinical standards in describing interventions which are to be considered appropriate in all circumstances and which therefore preclude flexibility in the individual clinician. This author employs the term clinical guideline in describing an intervention where a degree of flexibility is afforded, depending on the given clinical circumstances. ,Clinical standards and clinical guidelines are therefore distinguished from clinical options which are characterized by their essential lack of definitive guidance, a property inherent in the high degree of flexibility that they afford to the individual clinician. Grimshaw & Russell (1 993) recommend preferentially the categorizations advanced by Irvine & Donaldsori (1994) who describe mandatory elements, near mandatory elements and optional elements.

Mandatory, near mandatory and optional elements

In accordance with this categorization, mandatory elements within clinical guidelines are recognized as representing those components which can be demonstrated to have been derived from a solid scientific base and which have been shown to have integral relation to patient clinical outcome. A prominent example in this context is early salicylate administration following acute myocardial infarction (MI), in the absence of specific contra-indication. ISIS-2 has definitely established that this intervention diminishes subsequent mortality (ISIS-2 1988). This intervention

8 1996 Blackwell Science, Journal olEvaluation in Clinical Practice, 2, 2. 131-152 137

A. Miles et a/

in these clinical circumstances is therefore to be considered a mandatory element in the treatment protocol for MI. Conversely, in other clinical circumstances where effectiveness data of such definite derivation are absent, alternative management strategies are likely to be available and in such circumstances would represent the optional elements of the clinical protocol.

Clinical guidelines will therefore contain elements which are mandatory by virtue of a definitive scientific demonstration of their absolute indication in given clinical circumstances, but will also contain optional elements of management strategy because of the lack of a solid scientific base. On this basis, a further consideration of the characteristics of clinical guidelines is possible.

Irvine et al. (1986) argued that clinical guidelines could be considered under two simple, broad categories: deterministic clinical guidelines and branching clinical guidelines. Deterministic clinical guidelines are characterized by a fixed number of elements which are to be followed rigidly. This category of guidelines therefore precludes deductive clinical decision making and as a consequence will have limited clinical applicability, being reserved for highly specific clinical circumstances. They are typically useful, and may typically be seen employed in situations of acute medical or surgical emergency. Branching clinical guidelines are more commonly known as algorithms or treatment flow-charts, although they are sometimes presented as text (Margolis et al. 1989; Schoenbaum & Gottlieb 1990).

Validity of clinical practice guidelines

Woolf (1992) insisted that, for their validity, clinical guidelines must be derived from sound evidence which might in given circumstances be supplemented by expert opinion. They should be considered as ‘works in progress’, this author says, and are useful in dis- cussing with doctors the areas of their clinical practice which do not systematically incorporate important advances in the field. Eddy (1990) had earlier made clear that the relative validity of the elements of clinical guidelines will depend absolutely on the integrity or ‘scientific weight’ of the data from which they are derived and which explain the clinical outcome that they may be expected to produce.

Grimshaw & Russell (1993) are similarly definitive in insisting that a central factor in the integrity and validity of clinical guidelines will be rigorous development based on an available scientific base. These authors recognize, however, that, where a sound scientific base does not yet exist, validity may be con- ferred by beset clinical judgement, although failure to adopt a rigorous approach may result in the formulation of ineffective or even dangerous clinical practice. In considering the methodology which may be employed in synthesis of clinical evidence, Grim- shaw & Russell (1993) identify five principal techniques: (i) expert opinion, (ii) unsystematic literature review, (iii) ungraded systematic review, (iv) graded systematic review, and (v) formal meta-analysis.

Expert opinion

Within the United Kingdom, clinical guidelines have often been formulated by expert groups in the absence of a specific, formal review of the relevant clinical literature. The formulation of clinical guidelines in this manner will therefore rely substantially on the working knowledge of the participating clinicians in terms of available research and in terms of individual clinical experience. Haynes et al. (1984) pointed out that the clinicians’ knowledge of published research is highly variable and frequently incomplete and can be based upon a misinterpretation of presented results. Indeed, Sackett (1986), in considering this precise scenario, was able to identify four principal factors associated with an overestimation of clinical effectiveness. Formulation of clinical guidelines according to this particular methodology therefore represents guideline development under what Grimshaw & Russell (1993) have described as ‘unevaluative conditions’. Such guidelines may be expected to demonstrate inherent bias and may result in a perpetuation of opinion-based, rather than evidence-based, clinical practice.

Unsystematic review

Grimshaw & Russell (1993) describe an unsystematic review as one which lacks a definitive research strategy, lacks explicit inclusion criteria and lacks formal methods of synthesizing evidence. In some ways, guideline development through unsystematic review

138 Cj 1996 Blackwell Science. Journal of Evaluation in Clinical Practice, 2 , 2. 131 -1 52

Central dimensions of clinical practice evaluation - I 1

may be considered to be superior to guideline development through so-called ‘expert opinion’, in that it expands the evidence base significantly, though not to the extent that all important studies within the literature can be guaranteed as having been included. It is clear that the non-application of appropriate study inclusion criteria can generate literature which will vary somewhat in its overall utility, and selection bias is therefore a principal and persistent characteristic of guideline development through the process of unsystematic review.

Systematic review

In direct contrast to the unsystematic review, the systematic review is characterized by the application of a definitive literature search strategy and the definition and application of inclusion criteria such as, for example, minimum levels of methodological design of published studies. Mulrow (1994) has elegantly described the rigorous methodology with which research data of relevance to clinical decision making are located and crystallized. The overall process frequently necessitates collaboration on an international scale and is a time-consuming exercise (Grimshaw et al. 1995). Take, for example, the recent Antiplatelet Trialists’ collaboration which involved the aggrega- tion of original patient data on some I30000 subjects in over 300 randomized controlled trials (Antiplatelet Trialists’ Collaboration 1994ax).

Ungraded systematic reviews are characterized by the employment of explicit standards in judging the scientific validity and clinical utility of published research studies, and actively exclude the introduction of methodologically unsound work. Grimshaw & Russell ( 1 993) cite the example of Haynes et al. (1 984) in this context, where the latter authors reviewed 248 studies published between 1970 and 1983 for potential inclusion in a systematic review of the effectiveness of continuing medical education. When rigorous inclusion criteria were applied in the review process, only seven studies of the total of 248 proved suitable for election to final analysis.

Sackett (1986) has described the principles of the graded systematic review, which is characterized by the active exclusion of research from sources other than the randomized controlled trial. The central methodological characteristic of this process involves

the grading of evidence in which research studies are ranked by design. A major randomized trial with low error risk factors would be ascribed a level 1 ranking. A series of cases with no available control data would be ascribed a level of 5 ranking. The development of clinical guidelines employing such trials can therefore grade recommendations for best clinical practice with direct reference to the graded integrity of the research data. An A-graded practice recommendation would therefore be recognized in this context as deriving from at least one level-1 study.

Meta-analysis

Clinical guidelines crystallized from systematic reviews of the research literature are in many cases appropriate for immediate introduction into routine practice. Many clinicians will be persuaded on this basis to modify their personal clinical practice, but others will express reservations in relation to the scientific integrity of the evidence on which the guidelines are based, and the process by which they have been derived. Meta-analysis is preached with evan- gelical zeal as a source of infallible dogma for clinical decision making. The technique is certainly considered to represent a powerful method of major utility in situations where contradictory data have been generated by trials which have examined essential similar subjects, and where data from several trials require combination and analysis. Methodological deficiencies in this process remain a source of concern to many clinicians, however (Eysenck 1994, 1995; Thompson 1994; Villar et al. 1995; Egger & Smith 1999, and such concerns are articulated by many doctors as a legitimate reason for non-introduction of scientific evidence into routine clinical practice. Tra- ditional elites will act similarly and employ these concerns as the perfect excuse for preservation of a clinical practice that is based on personal, professional experience and therefore on personal, individual clinical judgement. In such a situation, it is clearly necessary to examine frankly the basis of the reasoning underlying the anxieties associated with the use of this technique in clinical evidence synthesis.

Eysenck (1995, 1995) has elegantly articulated the difficulties that warrant serious consideration. The inclusiveness of meta-analysis represents an immediate concern, this author says, in that meta-analysis can

Q 1996 Blackwell Science, Journal ol Evalualron in Clrnrcal Pracbce. 2. 2. 131-152 139

A. Miles eta/

examine data which are ‘good, bad and indifferent’ (Eysenck 1994). Smith et al. (1980) were among the first investigators to pioneer the technique of metaa- nalysis, advancing its utility in precluding subjectivity in outcome analysis. Studies which might warrant the overall description of ‘unsound’ or ‘bad’, for example, those with clear methodological deficiencies in the study design, are frequently excluded from the process of outcome analysis by reviewers, with the process of exclusion involving a subjective judgement. Smith el al. (1980) correctly emphasized that the process of formulation of exclusion and inclusion criteria by reviewers can frequently include a major bias, in that reviewers may seek to include or exclude in order to ‘favour a favourite hypothesis or vested ideological interest’. To avoid such difficulties, Smith et al. (1980) suggested comprehensive inclusiveness. Clearly, such a policy is highly problematic in its own right. Indeed, much research will have received unfavourable criticism on objective grounds. If a study has simply not employed objectively recognized statistically valid sampling techniques, for example, then it warrants exclusion from analysis on objective grounds. The discovery of subjectivity in any analysis generates immediate concern, but to be considered in addition is what Eysenck (1994) describes as ‘intimate knowledge of the field, the participants, the problems that arise, the reputation of different laboratories, the likely trustworthiness of individual scientists. . .’. Meta- analysis is therefore criticized by some authors as routinely failing to consider these highly relevant factors and thus, while precluding one form of subjective bias, cumulatively aggregating, in systematic fashion, any inherent biases in the original research. These biases may or may not, Eysenck (1 994) argues, balance out.

Eysenck has written previously in relation to these problems (Eysenck 1978,1984, 1992) and his concerns have been considered widely by the scientific community. This author has recently articulated the principal methodological difficulties that should be considered when performing meta-analyses, and certainly when interpreting their results. These include non-linearity of regressions, multivariate versus uni- variate effects, restriction of coverage, non-homo- geneity of summarized data, grouping of different causal factors resulting in estimates of equivocal meaning, and impaired ability to detect discrepancies

because of a theory directed approach. Eysenck (1994) insists that ‘meta-analysis may not be the one best method for studying the diversity of fields for which it has been used’ and believes that meta-analyses are ‘often employed in an effort to recover something from poorly designed studies, studies of insufficient statistical power, studies that give erratic results and those resulting in apparent contradictions’. Eysenck (1994) recognizes that careful investigation can preclude such pitfalls but, in the process of doing so, subjective bias can be introduced, and this is the very process that meta-analysis is theoretically designed to avoid. In his concluding text, this author provides a sobering thought:

If a medical treatment has an effect so recondite and obscure as to require meta-analysis to establish it, I would not be happy to have it used on me. It would seem better to improve the treatment, and the theory underlying the treatment.

In an important review, Clarke & Stewart (1995) agree that all possible biases should be minimized in the process of meta-analysis, and undertake an evaluation of the methods through which this may be achieved. These authors give specific consideration to the use of individual patient data (IPD meta-analysis), a methodological approach that has been described as a ‘yardstick’ against which other fornis of systematic review can be measured, and also discuss the relevance to meta-analysis of aggregate, or tabular, data. Such methodologies have already provided reliable evidence in areas of clinical uncertainty. In one of Eysenck’s most recent examinations of the technique of meta-analysis (Eysenck 1995), the utility of alternative methodologies, such as best-evidence synthesis and theory-directed analysis, in the derivation of medical evidence is suggested.

Clinical acceptability of clinical practice guidelines

Grimshaw & Russell (1993) have emphasized that the nature and characteristics of guideline development groups will be important factors in the validity and subsequent acceptability of formulated guidelines. These investigators have described three principal categories of guideline development groups: internal groups, intermediate groups and external groups.

An internal group is characterized by having as its

140 8 1996 Blackwell Science, Journal of Evaluation in Clinical Practice, 2, 2 , 131-1 52

Central dimensions of clinical practice evaluation - I 1

members the clinicians who will subsequently use the developed guidelines. An intermediate group differs from an internal group in having as its members the representatives of the clinicians who will subsequently use the developed guidelines. An external group excludes from its membership the clinicians who will subsequently use the developed guidelines. It has been suggested that guideline implementation is most likely to occur when the guidelines have been developed through the internal group method, in that the sense of ownership of the produce will enhance the like- lihood of operational use. However, only two (Put- nam & Curry 1985; North of England Study of Standards and Performance in General Practice 1992) of four studies (Sommers et al. 1984; Putman & Curry 1985, 1989; North of England Study of Standards and Performance in General Practice 1992) identified by Grimshaw et al. (1995) provided supporting data in this context.

Importance of speciality composition

An important factor in determining the nature of developed clinical practice guidelines is the speciality composition of internal, intermediate and external groups. Leape et al. (1992) demonstrated differences in the clinical practice guidelines for carotid endarterectomy advanced by a panel consisting solely of surgeons, and another consisting of surgeons, neurologists and other specialties. Similar observations have been reported by Scott & Black (1991) in a study examining clinical guideline development of cholecystectomy .

Potential bias of peer group, Delphi technique and consensus conference methods

Three principal methodologies have been described for development of clinical practice guidelines: the peer group method, the Delphi method and the consensus conference method (Fink et al. 1984; Auder et al. 1990). Grimshaw & Russell (1 993) have expressed concern in relation to the possible biases that can be introduced through each one of these particular methods, noting that little evidence is available which enables examination of their relative merits (Russell et al. 1994). Grimshaw & Russell (1993) advise against the use of primary method as a classification factor

when considering guideline validity, and instead focus on the work of Woolf (1992) in characterizing four secondary methodologies related directly to the nature of clinical decision making: informal consensus- based guideline development, formal consensus-based guideline development, evidence-based guideline development and explicit guideline development.

Informal consensus

Informal consensus-based clinical practice guideline development is characterized by its use of poorly defined and frequently implicit criteria for decision making, with resultant guidelines therefore deriving from ‘global subjective judgement’ (Eddy 1990). This technique lacks complex analytical processes, enabling rapid formulation of guidelines. The concurrent lack of data on which to base the recommended practice frequently poses difficulties in convincing other clinicians of the validity of the developed guidelines. The formulation of clinical practice guidelines via informal consensus has been widely employed within the United Kingdom (New- ton et al. 1992).

Formal consensus

A more structured approach to guidelinc development characterizes the formal consensus-based method of clinical practice guideline development, but there is a similar failure of this method to demonstrate a direct association of the guideline development process with an established evidence base (Woolf 1992).

Evidence-based method

The envelopment of clinical practice guidelines in direct association with an established evidence base characterizes this method (Woolf et al. 1990). In circumstances where the evidence base employed has not been rigorously derived, expert opinion may be introduced into the process, although it is always necessary that this is documented.

Explicit method

The explicit method has been infrequently utilized to date. It involves evaluation of the benefits, risks and

0 1996 Blackwell Science, Journal of Evaluation in Clinical Practice, 2 , 2 , 131-152 141

A. Miles eta/

costs of potential clinical interventions through use of an estimation of the probability and value of each potential outcome, a so-called ‘balance sheet’ being formulated to facilitate patient choice (Eddy, 1990; Grimshaw & Russell 1993).

Enquiries into clinical practice guideline derivation and validity

Clinical practice guidelines are considered valid when they lead to predicted health gain at predicted cost. Grimshaw & Russell (1993; Grimshaw et af. 1995) are clear that valid guidelines can lead to changes in the process and outcome of care if appropriately dis- seminated and implemented, but warn that the development of invalid guidelines can lead to the use of ineffective interventions, the misuse of resources and the potential for a deterioration in the patient’s health. The authors identify three factors of key relevance in ensuring the development of valid guidelines. Firstly, the scientific evidence must be derived in a systematic fashion through the use of explicit search strategies and inclusion criteria and the employment of rigorous methods of data synthesis. Secondly, the guideline development panel examining the scientific evidence so derived must include most (if not all) of the relevant disciplines, but few end users of the guideline. Thirdly, the guideline development panel must ensure that its recommendations demonstrate an explicit link with the evidence on which they are based.

The ‘quality’ of the evidence on which guidelines are based is therefore central to the validity of the guideline and central to the concerns of clinicians who are increasingly exhorted to employ practice guidelines in routine care. Key concerns frequently articulated by practitioners include the reliability of the techniques, such as meta-analysis, used to synthesize evidence from multiple sources and the relevance of the results of highly selective randomized controlled trials in the decision-making process when confronted with the immediate experience of the highly individual patient. Polychronis et af. (1996a, b), for example, illustrate how the results of meta-analysis can be advanced as ‘evidence’ when methodological problems in the process have led to the production of results which do not represent evidence at all. Intra- venous magnesium sulphate, for example, was

advanced as an ‘evidence-based, effective, safe, simple and inexpensive’ intervention that ‘should be introduced into clinical practice without delay’, while later studies concluded that ‘until further research is presented there are no grounds for routine use of magnesium for patients with acute myocardial infarction’. Such events fail to inspire the busy clinician and create cynicism. Polychronis et al. (1996) also defend the role of personal practitioner experience and deterministic medical reasoning in circumstances where general research evidence is judged inapplicable to the individual patient in the individual clinical consultation. Evidence-based medicine, increasingly advanced as the advent of salvation from ineffective clinical practices and spiralling costs, is more limited in its applicability in everyday care than its protagonists and ideologues would admit.

Grey zones of clinical practice and the limitations of evidence-based medicine

Evidence-based medicine provides only minor assis- tance in what Naylor (1995) has described as the many ‘grey zones’ of clinical practice, where the evidence in relation to risk-benefit ratios of competing clinical options is incomplete or contradictory. Naylor (1995) believes this particular intellectual terrain to be poorly charted, while recognizing that the RAND research demonstrates its considerable potential.

The development of criteria for assessment of clinical appropriateness of RAND involves constitution of panels of clinical experts who rate hundreds of hypothetical cases on a risk-benefit scale (Brook el al. 1986; Park et af. 1989). Published research deriving from these exercises demonstrates that the ratings for some clinical scenarios inevitably cluster in what might be described as the ‘uncertain’ middle of the scale, and the existence of disagreement within the panel can result in a failure to produce definitions of clinical appropriateness for other clinical scenarios.

Naylor (1995) has demonstrated that, when the ratings produced by a given nation’s experts are applied to charts audited in the same country, a notable proportion of ‘grey-zone procedures’ is revealed. This author has additionally demonstrated how differing results can be generated when cross- panelists’ ratings are applied to the same data. US panels are characterized by being ‘action-orientated’

142 cQ 1996 Blackwell Science, Journal of Evaluation in Clinical Practice. 2. 2, 131 -1 52


in areas of uncertainty when compared, for example, to UK panels. Canadian panels are associated with an intermediate policy. It is also acknowledged that surgeons view surgery with significantly less uncertainty than physicians, and that generalists are significantly more conservative than specialists (Kahn et al. 1988a,b; Brook et al. 1988; Leape et al. 1992; Fraser et al. 1993).

These findings may be considered to demonstrate not only the role of evidence, but in addition the roles of inference and experience. One potential scenario is that panelists may therefore pool ignorance as much as distil wisdom (Scott & Black 1991) and the decision-roles for equivocality versus inappropriateness are themselves arbitrary (Phelps 1993). If evidence were required from randomized controlled trials in support of each indication, the ‘grey zone’ would be significantly extended (Lomas et al. 1989).

Naylor (1995) points out soberingly that ‘the boundaries of the grey zones are themselves uncertain, varying with the evidence and its interpretation. Clinical medicine seems to consist of a few things we know, a few things we think we know - but probably don’t; and lots of things we don’t know at all’. Many clinical actions are therefore undertaken in the absence of evidence to guide them, and as a consequence one might expect what has been described by Naylor (1 995) as ‘paralytic indecisiveness’ to be more frequent in routine clinical practice than actually seems to be the case. Doctors are, of course, rather accustomed to decision making in circumstances of clinical uncertainty and become confident in this ‘educated guesswork’ to the point where it is easy to confuse personal opinion with evidence or personal ignorance with genuine scientific uncertainty (Naylor

Naylor (1995) is clear that clinical guideline writers act unhelpfully if they fail to distinguish between fact and fervour. Indeed, Naylor (1 995) points out that ‘if clinical guidelines and other trappings of evidence- based medicine are to be credible, they must distil the best evidence about what ought to be done in practice in ways that honestly acknowledge what we do not know about a topic , . . more generally, the prudent application of evaluative sciences will affirm, rather than obviate, the need for the art of medicine.

1995).

Can effective care be afforded?

Possibly. That studies of clinical effectiveness must be con-

ducted in parallel with studies of cost-effectiveness is now increasingly acknowledged by clinicians (Eddy 1992; Drummond 8i Maynard 1993; Sheldon & Bor- owtiz 1993; Williams 1994). Such a recognition derives from the inescapable reality that limited resources have to be utilized efficiently. Grimshaw et al. (1995) emphasize that, if the development of clinical practice guidelines proceeds without due reference to cost, practices may be recommended that result in large increases in cost for little corresponding increases in health. Difficulties can arise, however, as Buchan (1993) has pointed out, ‘when effective care (as defined scientifically) is not affordable care (as defined politically). Clinicians are trained to believe that they should provide ideal, rather than reasonable, care to the people they have identified as needing it’, although consultants are now recognizing that clinical decisions have automatic resource implications and that, in the reformed NHS, clinical judgement has to be set within the priorities and guidelines agreed in a broader context of the decisions about community needs and service constraints that are made by purchasers, trust boards and clinical directorates (Bailey 1995). This is ‘a job for white coats as well as white collars’ (Bottomley 1994).

Clinicians generally believe themselves to be oper- ating effectively and that it is the resource limitation, imposed by management, that limits the quality of their care (Hopkins 1990). Initially, cost-effectiveness analyses were viewed, along with decision analysis and other forms of health policy analysis, as artificial, simplistic and dehumanizing (Ransohoff & Feinstein 1976; Feinstein 1977; Schwartz 1979; Brett 1981; Asch & Hershey 1995), and many clinicians find such exercises difficult to apply in the setting of clinical consultation with the individual patient. This difficulty derives as Asch & Hershey (1995) point out, from the recognition that patients can vary widely in their objective needs and personal preferences, and the use of analyses which employ an average set of goals may therefore not represent to them an approach in the best interests of all patients. The well-recognized variation in referral rates, surgical operations and lengths of stay demonstrate that resources are used

10 1996 Blackwell Science, Journal ol Evaluation in Clinical Practice. 2 , 2 , 131-152 143

A. Miles eta/

variably by clinicians, and where resource over-utilization resulting from an inappropriate nature and level of care is identified, the resources could be released to buffer inappropriate under-utilization elsewhere or to extend or improve an existing service.

The interest in economic evaluation of health care service provision has increased in parallel with the increased limitations on health care budgets. This interest in economic evaluation has been demonstrated not only by a substantial and increasing volume of associated publications (Backhouse et al. 1992; Elixhauser et al. 1993; Department of Health 1994) but also by the implementation of indicative drug budgets, general practitioner fundholding and specific contracts for clinical services. Drummond (1 995) has suggested that medical audit may also be considered to be a component of the cost-effectiveness programme in that the definition of audit states the need to increase quality of care within available resources.

Drummond (1995) has pointed out that, despite a proliferation in audit and guideline activities, there is to date limited evidence of economic data being employed in the determination of clinical policy or in the evaluation of its impact. This author insists that, in an ideal world, economic evaluation of guidelines and clinical policies would be an integral component of the studies which generate them. The inclusion of economic analysis within clinical trials is recognized as an essentially modem concept and a ‘relatively young science’ (Drummond 1995). In an important work within the inaugural edition of the Journal of Evaluation in Clinical Practice, Drummond (1995) concurs with Williams (1993) in warning that the way medical audit is implemented will have a critical influence on whether it will deliver an improvement in the health of the population that is commensurate with its costs. Drummond (1995) is clear that, unless medical audit and guideline development is func- tionally influenced by economic thinking, cost-effectiveness of health care provision cannot ‘be guaranteed, and a continued lack of reference may result in increased inefficiency and thus cost ineffec- tiveness of health care provision.

Health care rationing may need to be more aggressive if the development of clinical guidelines and new evidence-based treatments proceeds without reference to cost. The report of the Swedish Ministry

‘

of Health and Social Affairs Healthcare and Medical Priorities Commission (1993) accepted rationing of health care as an inevitability and one resulting from a growing gap between health need and health resources. In this gap, the expanding but expensive capacity of modem medicine to diagnose and treat disease plays an important part. The Commission maintained that the developing processes of prioritization, which are fundamental to health care rationing, must be exercised with reference to three core principles: (i) human beings are equally valuable, (ii) society must pay special attention to the needs of the weakest and most vulnerable, and (iii) cost-efliciency, and gaining the greatest .return, for the amount of money spent, must prevail. As Caplan (1995) has pointed out, ‘these simple principles have serious and important con- sequences for policy’. Certainly, the articulation of these principles precludes a variety of current prioritization strategies, simply for moral reasons. Indeed, ‘plans that would bridge the gap between need and resource availability by throwing the poor, the elderly, the sinful or the prematurely born into the gap to close it are simply unacceptable. Solutions that permit the wealthy, the socially prominent, or those who hold positions of power to walk across the gap on their wallets are also dismissed as morally unacceptable’ (Caplan 1995).

Ham (1995) believes that the British approach to health care rationing will be based on clinical guidelines and therefore linked to effectiveness research and guideline development. The process of health care rationing by exclusion, a principal feature of the Oregon approach, is unlikely to be adopted in the UK, Ham (1995) argues. Instead, the UK system is likely to model itself on key aspects of the approach adopted in New Zealand which centres on the utility of clinical guidelines in cost-effective, clinically effective, medical decision making. At the present time, however, the UK system seems to be based on what Ham (1995) has described as ‘rationing by muddling through’, an approach which some authors believe to be not without some merit, given the difficulties of priority setting and the paucity of technical solutions to complex problems (Hunter 1993; Ham 1995).

The Government views clinical audit, and the efforts to develop a knowledge-based, knowledge-led NHS, as pivotal processes in the development of priority-setting methodologies for health care

144 0 1996 Blackwell Science, Journal of Evaluation in Clinical Practice, 2 , 2 , 131-152


rationing, maintaining that ‘the issue of rationing core services does not arise while there is scope for further improvements in effectiveness (Ham, 1995). Ham (1995) views the significance of this statement as 2- fold in that the term rationing is employed preferentially to priority-setting and also in that the reference to core services may imply that some services may be considered non-core.

Ham (1995) notes that the Department of Health has advised purchasers increasingly to base their commissioning activities with direct reference to available clinical guidelines, advice which has heigh- tened the profile and necessity of effectiveness research within the UK. The difficulty, as Ham (1995) outlines, is in identifying ‘the best ways of influencing and changing clinical practice and implementing research findings’.

Initial thinking on the mechanisms through which this objective could be achieved led to the suggestion that ‘a key to improving quality in the NHS is for healthcare commissioners to stop purchasing activity and to begin purchasing evidence-based protocols’, on the basis that ‘purchasing procedures or even packa- ges of care guarantees nothing about the expected content and scientific basis of those services, whereas when a protocol is purchased we know that, if it is followed, the most effective set of actions will be taken (Sheldon & Borowitz 1993). Miles et al. (1995) warned, however, that even the purchasing of protocols for thrombolytic treatment in eligible patients following acute myocardial infarction would be an inappropriate and dangerous development of the purchasing function because i t would create a pre- cedent that could lead to the introduction of inappropriate care and to the exercise of a selectivity that could actively disadvantage patients.

This is clearly what might be described as a typical ‘clinical’ or even ‘conservative’ view. NHS health care commissioners will manifestly refuse to share it. For them, evidence-based medicine is of core importance in developing a clinical guideline-based NHS which is more concerned with the cost containment associated with systems of managed care than it is concerned with achieving quantifiable increases in medical quality. Such a strategy, that is, the incremental introduction of a system of managed care, would necessarily start with the establishment of a ‘guidelines’ culture and would therefore be characterized by

efforts on the part of the NHS to develop guidelines across all specialties. This is currently happening (NHS Executive 1996a,b), and a guideline-based service, when successfully established, may well be subsequently manipulated in a way which guarantees the primacy of cost limitation. Indeed, as Polychronis et al. (1996b) point out, the results of evidence-based medicine have a very clear potential for use in an apparently value-free, apolitical cost-containment exercise whereby only treatments for which formal evidence of effectiveness exists will be purchased. But, as these same authors emphasize, there are at least two factors which determine the availability of formal evidence: the degree of effort invested in research and therefore the number of randomized controlled trials performed, and the ease with which clinical outcomes can be quantified and assessed. This scenario could well lead to some drug treatments being purchased simply because there is formal evidence of their effectiveness rather than because they are actually superior to alternative treatments for which formal evidence of effectiveness does not exist.

Such considerations are unlikely to be counte- nanced by health care commissioners obsessed with the desire to contain costs. Their propensity to employ the results of evidence-based medicine lo disinvest in certain treatments where no formal evidence of effectiveness exists, and their failure to invest in treatments for which formal evidence of effectiveness exists but which are resource-positive in nature, is highly likely to characterize the future of health care purchasing on the effectiveness model. Indeed, a recent example has been provided by Pinching (1 996) who reminds us that the latest evidence-based treatment of HIV disease deriving from the Delta and ACTG175 trials will increase the costs of care per patient by 50-75%. Health care commissioners have indicated that, while such treatments may be used, their costs must be found from within existing allo- cations; despite the fact that these are reducing in the face of increasing caseloads. This is a clear example of a selective use of effectiveness data. To the question we posed at the beginning of this section: ‘Can effective care be afforded?’, it would seem that the answer may well, with time, become ‘no’.

0 1996 Blackwell Science, Journal olEvaluation in Clinical Practice, 2 , 2, 131-152 145

A. Miles eta/

How can effective care be implemented?

Through a combination of methods. The frequency with which the term guideline or

protocol has been employed in clinical articles has increased 10-fold during the last decade (Hurwitz 1994), but it is well recognised that, within the UK, the rate of proliferation of guidelines has hardly been matched by a similar rate of guideline introduction into routine c!inical practice. The situation is essentially similar within the United States where some 20 000 healthcare standards and clinical practice guidelines have been advanced by some 500 organizations (Leone 1993; Hurwitz 1994) but with similar disparity between availability and implementation.

Implementation of definitive medical research evidence of proven relevance to given patient categories into routine clinical practice is in theory simply a question of a clinician ‘doing something differently’. In practice, however, i t is a complex matter. Treasure (1994) has pointed out that ‘a randomised controlled trial, however beautifully conducted, may have very little to say about how (or whether) the findings should be implemented in clinical practice - a topic of concern in the cost and audit conscious healthcare systems that most countries have these days’.

A simple demonstration to clinicians of the super- iority of one mode of practice over another is of itself insufficient as a method of promoting the implementation of changes in practice within clinical organizations (Stocking 1992, 1994). Indeed, pro- posed innovations in clinical practices may con- ceivably have benefits or disadvantages unrelated to the patient and may instead be seen as having greater relevance to the clinician’s own security. In this context, we might reflect on the words of Lilleyman (1994) who has said ‘Hell hath no fury like a vested interest paraded as a moral principle’. Innovations in clinical practice may well involve the learning of new skills and the abandonment of principles which have been learned early in professional training and which have been protected tenaciously ever since. An extensive summary by Rogers (1983) of some 7000 studies of change management included reports which documented the management of changes in routine clinical practice. The process of change implementation is well recognized to be far more complex in practice than might be anticipated in theory.

Methodologies for implementation of change

A great deal of research attention has been devoted to the development and evaluation of methods through which change may be introduced into routine clinical practice. Methods have been examined in this context which might prove effective either singly or in combination (Rogers 1983; Horder et al. 1986; Domenghetti et al. 1988; Pettigrew 1988; Lomas et al. 1989, 1991; Lomas 1990; Mugford et al. 1991; Grim- shaw et af. 1995, for review). The availability of new knowledge alone has been shown to be ineffective in precipitating change (Georgiades & Phillimore 1975; Stolz 1981; Lee et af . 1985). Other interventions, in isolation, are similarly unlikely to prove effective. The ability of clinical audit to introduce change, for example, is thought to be less effective when employed as a single intervention (Lomas et al. 1991; Stocking 1992, 1994) than when employed in combination with various forms of continuing clinical education (Fowkes et al. 1986a,b; Tierney et al. 1986; Lomas et al. 1991; Avorn et af . 1992; Anderson et al. 1994), especially taking into consideration a range of other supporting methods (Porras & Hoffer 1989; Paxton et af . 1993). Grimshaw et al. (1995) are in agreement with the contention that educational intervention is a powerful mechanism in change introduction. These authors cite specific research which indicates the dependence of educational intervention in this context on the active participation of clinicians. The optimum strategy for change implementation has yet to be defined, a consequence of insufficient research data. Systematic reviews of available data are currently being prepared by the Cochrane collaboration, and these will focus on the relative effectiveness of opinion leaders, outreach visits, audit and feedback and patient-mediated interventions (Freemantle et af . 1995). Miles et af . (1996b) have described a method of change introduction which combines direct clinical education with a highly operational means of introducing evidence-based changes in clinical practice into routine clinical practice, in an incrementally increasing manner. These authors believe that, on the basis of evidence so far available, permanent change in routine clinical practice can be achieved by

Direct education of clinicians by an educator that they perceive as authoritative, in a clinical setting

146 0 1996 Blackwell Sclence, Journal of Evaluation in Clinical Pracfice. 2 , 2 , 131-1 52

Central dimensions of clinical practice evaluation - I I

related to the educational topic, followed by one- to-one continuous feedback over an augmented period of time, with multi-disciplinary involvement. Purchaser involvement (Miles et al. 1995) and direct intervention by patients (Rigge 1995) is likely to potentiate this mechanism in significant fashion.

Conclusion

The development of modern clinical care is dependent on the integration of clinical art with clinical science. The continuing synthesis of evidence in relation to what constitutes optimum clinical appropriateness and clinical effectiveness is partnered by research aimed a t identifying the optimum means of increasing the efficiency with which those improvements in care can be delivered to patients. I t is increasingly recognized that the clinical audit mechanism (Miles el al. 1996a,b) represents a powerful tool in the character- ization of the appropriateness and effectiveness of provider clinical services and of the efficiency with which they are delivered to patients. When conducted with rigorous methodology (Miles er al. 1996a,b), the clinical audit mechanism enables comparison and contrast of characterized local practice with so-called 'best' practice, and facilitates the introduction of evidence-based change into routine clinical practice in a direct fashion. The importance of such developmental work to patients when viewed either as individuals or as epidemiological populations is incontrovertible. All such developments must proceed, however, with a n adequate input of what Drummond (1995) has described as 'economic thinking'. This is a logical and acceptable approach which will generate tangible improvements in patient care with maximization of access for reasonable population health.

Note

This paper is expanded upon in chapters 2 & 5 of the volume Effective Clinical Practice (eds. A. Miles & M. Lugon) to be published by Blackwell Science in late 1996.

Acknowledgements

Professor Miles and D r Polychronis thank D r Paul

Bentley and Dr M,yriam Lugon for helpful advice and criticism during the time of writing.

References

Anderson F.A., Wheeler H.B. & Goldberg R.J. (1994) Changing clinical practice: Prospective study of the impact of continuing medical education and quality assurance programmes on the use of prophylaxis for venous thromboembolism. Archives of Internal Medicine 154, 669-677.

Antiplatelet Trialists' Collaboration (l994a) Collaborative overview of randomised controlled trials of antiplatelet therapy. I: Prevention of death, myocardial infarction and stroke by prolonged antiplatelet therapy in various categories of patients. British Medical Journal 308, 8 1 - 106.

Antiplatelet Trialists' Collaboration ( I 994b) Collaborative overview of randomised trials of antiplatelet therapy. 11: Maintenance of vascular graft or arterial patency by antiplatelet therapy. British Medical Journal308, 159- 168.

Antiplatelet Trialists' Collaboration (1994~) Collaborative overview of randomised trials of antiplatelet therapy. 111: Reductions in venous thrombosis and pulmonary emho- lism by antiplatelet prophylaxis among surgical and medical patients. British Medical Journal 308, 235-246.

Antman E.M., Lau J., Kupelnick B., Mosteller F. & Chal- mers T.C. (1992) A comparison of results of meta-analyses of randomised controlled trials and recommendations of clinical experts. Joitrnal of the American Medical Association 268, 240-248.

Asch D.A. & Hershey J.C. (1995) Why some health policies don't make sense at the bedside. .4nnal.v of Internal Medicine 122, 846850.

Auder A.M., Greenfield S. & Field M. (1990) Medical practice guidelines: current activities and future directions. Annals of Internal Medicine 113, 709-714.

Avorn J., Soutmerai S.B. & Everitt, D.E. (1992) A rdndo- mized trial of a program to reduce the use of psychoactive drugs in nursing homes. New England Journal of Medicine

Backhouse M.E., Backhouse R.J. & Edey S.A. (1992) Eco- nomic evaluation bibliography. Health Economics 1

Bailey, J. (1995) Rethinking Consultants: Time for change in traditional working practices. British Medical Journal

Bottomley V. (1994) NHS priority setting. Extract from speech of Secretary of State for Health to the BMA/Kings Fund/Patients Association Conference. In: Political challenge or political veneer? Health Care Analysis 2, 5- 12.

327, 168-1 73.

(SUPPI.), 1-236.

310, 788-790.

10 1996 Blackwell Science, Journal of Evaluafron in Cknrcal Practice. 2, 2, 131-152 147

A. Miles eta1

Brett A.S. (1981) Hidden ethical issues in decision analysis. New England Journal of Medicine 305, 1150-1 152.

Brook R.H. (1991) Measures of Appropriateness. Quality Care and Health Status Available from RAND. RAND, Santa Monica. CA.

Brook R.H., Chassin M.R., Fink A., Solomon D.H. & Kosecoff J.B. (1986) A method for the detailed assessment of the appropriateness of medical technologies. International Journal of Technological Assessment in Health Care 2, 53-64.

Brook R.H., Kosecoff J.B., Park R.E., Chassin M.R., Winslow C.M. & Hampton J.R. (1988) Diagnosis and treatment of coronary heart disease: doctors’ attitudes in the USA and UK. Lancet ii, 750-753.

Buchan H. (1993) Clinical guidelines: Acceptance and promotion. Quality in Health Care 2, 213-214.

Buck N., Devlin H.B. & Lunn J. (1987) Report of a Con- fidential Enquiry into Perioperative Deaths (CEPOD). Nuffield Provincial Hospitals Trust/Kings Fund, Lon- don.

Caplan A.L. ( 1 995) Straight talk about rationing. Annals of Internal Medicine 122, 795-796.

Central Statistics Office (1994) Health Statistics. HMSO, London.

Chalmers I. (1 993) Underuse of antenatal corticosteroids and future litigation. Lancet 341, 699.

Chalmers I., Hetherington J. & Elbourne D. (1989a) Methods in Synthesising Evidence to Evaluate Effects of Care in Pregnancy and Childbirth. Oxford University Press, New York, pp. 39-65.

Chalmers I., Enkin M. & Kierse M. (1989b) Effective Care in Pregnancy and Childbirth. Oxford University Press, Oxford.

Charlton B.G. (1993) Management of Science. Lancet 342,

Charlton B.G. (1996) The future of clinical research: away from megatrials and towards methodologically rigorous studies and representative sampling. Journal of Evaluation in Clinical Practice, in press.

Charlton J.R.H. & Lakhani A. (1986) Avoidable Deaths Study. Birmingham inter-authority comparisons. Uni- versity of Birmingham Press.

Charlton J.R.H., Silver R.M. & Hartley R.M. (1983) Geo- graphical variation in mortality from conditions amenable to medical intervention in England and Wales. Lancet i, 691-696.

Chassin M.R, Brook R.H. & Park R.E. (1986) Variations in the use of medical and surgical services by the Medicare population. New England Journal of Medicine 314, 285- 290.

Clarke M. & Stewart L.A. (1995) Systematic reviews of randomised controlled trials: the need for complete data.

99-100.

Journal of Evaluation of Clinical Practice 1, 119-126. Dans P.E. (1994) Credibility, cookbook medicine, and

common sense: Guidelines and the College Annals of Internal Medicine 120, 966-968.

Delamothe T. (1993) Wanted: Guidelines that doctors will follow. British Medical Journal 307, 218.

Department of Health (1994) Department of Health Register of Cost-eflectiveness Studies. HMSO, London.

Dickinson E. (1995) Using market principles for healthcare development. Quality in Health Care 4, 40-44.

Domenghetti G., Luraschi P., Gutmiller F., Pedrinis E., Casablanca A., Spinelli E. & Repetto F. (1988) Effect of information campaign by the mass media on hysterectomy rates. Lancet ii, 1470-1473.

Donaldson L.J. (1992) Maintaining excellence: the preservation and development of specialised services. British Medical Journal 305, 1280-1284,

Drummond M.F. (1994) Economic Analysis Alongside Controlled Trials: An Introduction for Clinical Research- ers. Department of Health, London and Leeds. HMSO, London.

Drummond M.F. (1995) The role of health economics in clinical evaluation. Journal of Evaluation in Clinical Practice 1, 71-75.

Drummond M.F. & Maynard A. (1993) Purchasing and Providing Cost-effective Health Care. Churchill Living- stone, Edinburgh.

Eddy D.M. (1990) Practice policies - where do they come from? Journal of the American Medical Association 263,

Eddy D.M. (1992) A Manual for Assessing Health Practices and Designing Practice Policies: The Explicit Approuch. American College of Physicians, Philadelphia.

Egger M. & Smith G.D. (1995) Misleading meta-analysis. British Medical Journal 310,152-154.

Elixhauser A., Luce B.R., Taylor W.R. & Reblando J. (1993) Health care CBA/CEA: an update on the growth and composition of the literature. Medical Care 31

Eysenck H.J. (1978) An exercise in mega-silliness. American Journal of Psychology 33, 517.

Eysenck H.J. (1984) Meta-analysis: an abuse of research integration. Journal of Special Education 18, 41-59.

Eysenck H.J. (1992) Meta-analysis: sense or non-sense? Pharmaceutical Medicine 6, 1 13- 1 19.

Eysenck H.J. (1994) Meta-analysis and its problems. British Medical Journal 309, 789-792.

Eysenck H.J. (1995) Meta-analysis or best evidence synthesis: Journal of Evaluation in Clinical Practice 1, 29-36.

Feinstein A.R. (1964a) Scientific methodology in clinical medicine: I. Introduction, principles and concepts. Annals of Internal Medicine 61, 564-579.

1 265- 1 275.

(SUPPI.), JSI-JSI 1.

148 0 1996 Blackwell Science, Joumalol Evaluation in Clinical Practice, 2 , 2 , 131-152


Feinstein A.R. (1964b) Scientific methodology in clinical medicine: 11. Classification of human disease by clinical behaviour. Annals of Internal Medicine 61. 757-78 1.

Feinstein A.R. (19644 Scientific methodology in clinical medicine: 111. The evaluation of therapeutic response. Annals of Internal Medicine 61,944-965.

Feinstein A.R. (1964d) Scientific methodology in clinical medicine: IV. Acquisition of clinical data. Annals o/ Internal Medicine 61, 1162-1 193.

Feinstein A.R. (1967) Clinical Judgement. Williams and Wilkins, Baltimore.

Feinstein A.R. (1977) Clinical biostatistics XXXIX. The haze of Bayes, the aerial palaces of decision analysis and the computerised Ouija board. Clinical Pharmacology and Therapeutics 21,482496.

Feinstein A.R. ( 1 994) Clinical judgement revisited: the dis- traction of quantitative models. Annals of Internal Medi- cine 120, 799-805.

Field M. & Lohr K. (1990) Clinical Practice Guidelines: Directions for a New Programme. National Academy Press, Washington.

Fink A,, Kosecoff J., Chassin M. & Brook R.H. (1984) Consensus methods: characteristics and guidelines for use. American Journal of Public Health 74, 979-983.

Fowkes F.G.R., Hall R. & Jones J.H. (1986a) Trial of strategy for reducing the use of laboratory tests. British Medical Journal 292, 883-885.

Fowkes F.G.R., Davies E.R. & Evans K.T. (1986b) Multi- centre trial of four strategies to reduce use of a radi- ological test. Lancet 1, 367-370.

Fraser G.M., Pilpel D., Hollis S., Kosecoff J.B. & Brook R.H. (1993) Indications for cholecystectomy: the results of a consensus panel approach. Quality Assurance in Health Care 5, 75-80.

Georgiades, N.J. & Phillimore L. (1975) The myth of hero- innovator and alternative strategies for organisation change. In Behaviour Modification with the Severely Handicapped (eds C. Kiernan & J. Woodford). Associated Scientific Press.

Glover J.A. (1938) The incidence of tonsillectomy in school children. Proceedings of the Royal Society of Medicine xxxi, I21 9-1 236.

Gosney M. & Tallis R. (1984) Prescription of contra-indicated and interacting drugs in elderly patients admitted to hospital. Lancet ii, 564567.

Grimshaw J. & Russell I. (1993) Achieving health gain through clinical guidelines. I: Developing scientifically valid guidelines. Quality in Healthcare 2, 243-248.

Grimshaw J.M., Freemantle N., Wallace S., Russell I., Hunvitz B., Watt I., Long A. & Sheldon T. (1995) Developing and implementing clinical practice guidelines. Quality in Healthcare 4, 55-64.

Haines A. & Jones R. (1994) Implementing findings of research. British Medical Journal 308, 1488-1492.

Ham C. (1995) Healthcare rationing: the British approach seems likely to be based on guidelines. British Medical Journal 310, 1483-1484.

Hampton J.R. (1989) Inappropriate investigation and treatment in clinical practice. In Papers bused on a conference held in November 1988. Royal College of Physi- cians, London.

Hampton J.R. (1995) Practice variations, appropriateness and decision analysis. Quarterly Journal of Medicine 88, 365-367.

Hannah M.E. (1993) Active management of pre-labour rupture of membranes at 37 + weeks. In Pregnancy and Childbirth Module. Cochrane Database of Systematic Reviews: Review No. 03272, 24 March, 1993. Published through Cochrane Updates on Disk, Oxford. (Update Software, Spring 1993.)

Hart 0. (1993) Protocols or guidelines, or neither? British Medical Journal 306, 8 16.

Haynes R.B., Roms D.A., McKibbon A. & Tugwell P. (1984) A critical appraisal of the efficacy of continuing medical education. Journal of the American Medical Association 251, 61-64.

Henderson R.A., Raskino C.L. & Hampton J.R. (for the RITA Trial Participants) ( 1 995) Variations in the use of coronary arteriography in the UK: the RITA Trial Cor- onary Arteriogram Register. Quarterly Journal of Medi- cine 88, 167-173.

Hopkins A. (1 989) Appropriate Investigation and Treatment in Clinical Practice. Royal College o f Physicians, London.

Hopkins A. (1990) Measuring the Quality of Medical Care. Royal College of Physicians, London.

Horder J., Bosanquet N. & Stocking B. (1986) Ways of influencing the behaviour of general practitioners. Journal of the Royal College of General Practitioners 36, 5 17-52 I .

House of Commons Health Committee (1992) Maternity Services. HMSO, London.

House of Lords (1995) Minutes of Evidence Taken Before the Select Committee on Science and Technology. Sub-committee 1: Medical Research and the NHS Reforms. Ses- sions 1994-1995 31 January 1995. HMSO, London.

Hunter D.J. (1993) Rationing Dilemmas in Healthcare. National Association of Health Authorities and Trusts, Birmingham.

Hunter D. (1995) Effective practice. Journal of Evaluation in Clinical Practice 1, 131-134.

Hurwitz B. (1994) Clinical guidelines: Proliferation and medicolegal significance. Quality in Health Care 3, 3 7 4 .

Hurwitz B. (1995) Clinical guidelines and the law: advice, guidance or regulation? Journal of Evaluation in Clinical Practice 1, 49-60.

0 1996 Blackwell Science, Journal ol Evaluation in Clinical Practice, 2, 2, 131-152 149

A. Miles eta/

Institute of Medicine (1992) Guidelines for Clinical Practice: From Development to Use. National Academic Press, Washington DC.

Irvine D. & Donaldson L. (1994) Quality and standards in healthcare. In Quality Assurance in Medical Care (eds J.S. Beck, I.A.D. Bouchier & I.T. Russell). Royal Society of Edinburgh, Edinburgh.

Irvine D.H., Russell I.T. & Hutchinson A. (1986) Perfor- mance review in general practice. In Pursuit of Quality: (eds D.A. Pendleton, T.P.C. Schofield & M.L. Marinker). Royal College of General Practitioners, London.

ISIS-2 (1988) Second International Study of Infarct Survival Collaborative Group. Randomised trial of intravenous streptokinase, oral aspirin, both or neither among 17,187 cases of suspected myocardial infarction. Lancet ii, 349- 360.

Jarvis E. (1850) The influence of distance from and proxi- mity to an insurance hospital on its use by any people. Boston Med Surg Journal 42, 226-232.

Kahn K.L., Park R.E. & Brook R.H. (1988a) The effect of comorbidity on appropriateness ratings for two gastro- intestinal procedures. Journal ofClinica1 Epidemiology 41, 115-122.

Kelson M. (1996) Evidence-based medicine and clinical audit: perceptions and expectations of the patient. In Evaluation and Development in Clinical Care (ed A. Miles). Royal Society of Medicine Press Ltd, London.

King's fund institute (1988) Research Report - Health Care Variations. Assessing the Evidence. KFC.

Krakauer H., Bailey Clifton R., Cooper H., Wai-Kouk Y., Skellan K.J. & Kattakkuzhy G. (1995) The Systematic assessment of variations in medical practices and their outcomes. Public Health Reports 110, 2-1 2.

Laverick M.D., Croal S.A. & Mollan R.A.B. (1991) Orthopaedic surgeons and thromboprophylaxis. British Medical Journal 303, 504550.

Leape L.L., Park R.G., Kahan J.P. & Brook R.H. (1992) Group judgements of appropriateness: the effect of panel composition. Quality Assurance in Health Care4, 151-1 59.

Lee T.H., Pearson S.D., Johnson P.A., Garcia T.B., Weis- berg R.N., Guadagnoli R., Cook F. & Goldman L. (1985) Failure of information as an intervention to modify clinical management - a time series trial in patients with acute chest pain. Annals of Internal Medicine 122, 434- 437.

Leone A. (1993) Medical practice guidelines are useful tools in litigation. Medical Malpractice, Law and Strategy 10,

Liberati A., and Members of the Steering Group and Coordinating Centre for Appropriateness of Hospital Use and the European Biomed Project (1995) Interna- tional Journal of Quality in Health Care 7, 185-276.

1-6.

Lightfoot L. & Rogers L. (1995) Hundreds Killed by Doctors Relying on Out-dated Manuals. The Sunday Times, 5 February, pp. 1-2.

Lilleyman J.S. (1994) Clinical standards in reformed NHS. Archives of Disease in Childhood 71, 275-276.

Lomas J., Anderson G.M. & Domnick-Pierre K. (1989) D o practice guidelines alter practice? The effect of a consensus statement on the practice of physicians. New England Journal of Medicine 321, 1306-1 3 1 1.

Lomas J., Enkin M. & Anderson G.M. (1991) Opinion leaders vs audit and feedback to implement practice guidelines. Delivery after previous Caesarian section. Journal of ihe American Medical Association 265, 2202-2207.

McIntyre N. (1985) Fostering the critical attitude in Medicine. Australian Clinical Review, 139-143.

Margolis C.Z., Cook C.D., Barak N., Adler A. & Geerts- man A. (1989) Clinical algorithms reach paediatric decision making more effectively than prose. Medical Care 27, 576-592.

Maternity Services Working Party (1 992) Interventions Subgroup: Results of the Questionnaire Sent to Obste- tricians in Scotland. Home and Health Department, Scottish Office, Edinburgh.

Maynard A. (1994) Management in Medicine. Medical Audit News 4, 140.

Miles A., Bentley D.P., Polychronis A., Price N. & Grey J.E. (l996a) Clinical audit in the National Health Service: fact or fiction? Journal of Evaluation in Clinical Practice 2,29- 35.

Miles A., Bentley D.P., Price N.. Polychronis A., Grey J.E. & Asbridge J. (1996b) The total health care audit system: a systematic methodology for clinical practice evaluation and development in NHS provider organizations. Journal of Evaluation in Clinical Practice 2, 37-64.

Mosteller F. (1981) Innovation and evaluation. Science 21 I ,

Mugford M., Banfield P. & O'Hanlan N. (1991) Effects of feedback of information on clinical practice: A review. British Medical Journal 303, 398-402.

Mulrow C.D. (1994) Rationale for systematic reviews. British Medical Journal 309, 597-599.

Naylor D.C. (1995) Grey zones of clinical practice: Some limits to evidence-based medicine. Lancet 345, 840-842.

NHS Executive (l996a) Promoting Clinical Effectiveness. A Framework for Action in and through the NHS. NHS Executive, Leeds.

NHS Executive (1 996b) Clinical Guidelines. Using Clinical Guidelines to Improve Patient Care within the NHS. NHS Executive, Leeds.

North of England Study of standards and performance in General Practice (1992) Medical audit in general practice: effects on doctors' clinical behaviour and the health of

881-886.

150 0 1996 Blackwell Science, Journal of Evaluafion in Clinical Praclrcs, 2 , 2. 131-152


patients with common childhood conditions. British Medical Journal 30, 1480-1488.

O’Neill D., Miles A. & Polychronis A. (1996) Central dimensions of clinical practice evaluation: efficiency, appropriateness and effectiveness - I. Journal of Evalua- tion in Clinical Practice 2, 13-27.

Onion C.W.R. & Walley T. (1995) Clinical guidelines: Development, implementation and effectiveness. Post- graduate Medical Journal 71, 3-9.

Park R.E., Fink A. & Brook R.H. (1989) Physician ratings of appropriate indications for three procedures: Theore- tical indications vs indications used in practice. American Journal of Public Health 19, 445-447.

Paterson-Brown S., Fisk N.M. & Wyatt C. (1995). Uptake of meta-analysis overviews of effective care in English obstetric units. British Journal of Obstetrics and Gynae-

Penn N.J. & Steer P.J. ( 1 991) How obstetricians manage the problem of pre-term delivery with special reference to the pre-term breech. British Journal of Obstetrics and Gynaecology 98, 53 1-534.

Perrin J.M.. Homer C.J. & Berwick D.M. (1989) Variations in rates of hospitalisation of children in three urban communities. New England Journal of Medicine 320, 1183-1 187.

Pettigrew A. (1988) The Management of Strategic Change. Blackwell Scientific, Oxford.

Phelps C.E. (1993) The mcthodological foundations of studies of the appropriateness of medical care. New England Journal of Medicine 329, 1241-1245.

Polychronis A,, Miles A. & Bentley D.P. (1996) Evidence- based medicine: Reference? Dogma? Neologism? New Orthodoxy? Journal of Evaluation in Clinical Practice 2,

Putnam R.W. & Curry L. (1985) Impact of patient care appraisal on physician behaviour in the office setting. Canadian Medical Association Journal 132, 1025-1029.

Putnam R.W. & Curry L. (1989) Physician participation in establishing criteria for hypertension management in the office: will patient outcome be improved? Canadian Medical Association Journal 140, 806809.

Ransohoff D.F. & Feinstein A.R. (1976) Is decision analysis useful in clinical medicine? Yale Journal of Biological Medicine 49, 165-168.

Rigge M. (1996) Clinical audit and the availability of evidence-based care: the patient’s view. In Effective Clinical Practice (eds A. Miles & M. Lugon). Blackwell Science Ltd, Oxford.

Rogers B. (1983) Dffusion of Innovation. 3rd edn. Free Press, New York.

Russell I.T., Grimshaw J.M. & Wilson B.J. (1994) Epide- miological and statistical issues in quality assurance. In

cology 102,297-301.

1-3.

Quality Assurance in Medical Care (eds J.S. Beck, I.A.D. Bouchier & I.T. Russell. Royal Society of Edinburgh, Edinburgh.

Sackett D.L. (1986) Rules of evidence and clinical recommendations on the use of anti-thrombolytic agents. Chest

Saunders D., Coulter A. & McPherson K. (1989) Variations in Hospital Admission Rates: a Review of the Literature. Kings Fund Project Paper No. 79. Kings Fund, London.

Schoenbaum C.S. & Gottlieb L.K. (1990) Algorithm based improvement of clinical quality. British Medical Journal 301, 1374-1376.

Schwartz W.B. (1979) Sounding board decision analysis: a look at the chief complaints. New England Journal of Medicine 300, 556559.

Scott E.A. & Black N.A. (1991) Appropriateness of cholecystectomy - a consensus panel approach. Gut 32, 1066- 1070.

Shaw C. ( 1 989) Medical Audit - a Hospital Handbook. Kings Fund Centre KFC 89, 11. London.

Sheldon T.A. & Borowitz M . (1993) Changing the measure of quality in the NHS: From purchasing activity to purchasing protocols. Quality in Health Care 2. 149-150.

Smith M.L., Glass G.V. & Miller T.I. (1980) The Benejit.7 uf Psychotherapy. John Hopkins Press, Baltimore.

Smith R. (1994) Medicine’s core values. British Medical Journal 309, 1 247- 1248.

Stocking B. (1992) Promoting change in clinical care. Quality in Health Care 1, 5 6 6 0 .

Stocking B. (1994) How d o we change systems of care in response to the findings of clinical audit? In Professional and Management Aspects of Clinical Audit (ed A. Hop- kins), pp. 131-138. Royal College of Physicians, Lon- don.

Stolz S.B. (1981) Adoption of innovations from applied behavioural research: Does anybody care? Journal of Applied Behaviour Analysis 14, 491-505.

Stuttaford T. (1995) Are Our Doctors Dangerously Out of Date? The Times, February 7 , p. 17.

Swedish Ministry of Health and Social Affairs: Health Care and Medical Priorities Commission (1993). No Easy Choices: The Difficult Priorities of Healthcare. Swedish Government Official Reports, Stockholm, Sweden.

Tanenbaum S.J. (1994) Knowing and acting in medical practice: the epistemological politics of outcome research. Journal of Health Politics Policy and Law 19, 27-44.

Thompson S.G. (1994) Why sources of heterogeneity in meta-analysis should be investigated. British Medical Journal 309, I35 1-1 355.

Treasure T. (1994) From research to practice. Lancer 344. 41 7 4 1 8.

UK-TIA Study Group (1983) Variation in the use of

86 (SUPPI.) 2-35.

iD 1996 Blackwell Science. Journal ol Evaluafion in Clinical Practice. 2 , 2, 131-152 151

A. Miles eta/

angiography and carotid endarterectomy by neurologists in the UK: TIA aspirin trial. British Medical Journal 286, 5 14-51 I.

Villar J., Carroli G. & Belizan J.M. (1995). Predictive ability of meta-analyses of randomised controlled trials. Lancet

Wennberg J.E. & Gittelsohn A. (1982) Variations in medical care among small areas. Scientijk American 246,120-135.

Wennberg J.E., Freeman J.L. & Culp W.J. (1981) Are hospital services rationed in New Haven or over-utilised in Boston? Lancet i, 1185-1 189.

Wennberg J.E., Freeman J.L., Shelton R.M. & Bub0lzT.A. (1989) Hospital use and mortality among Medicare beneficiaries. Annals of Internal Medicine 122, 434-431.

Wilken D., Metcalfe D.H. & Marinker M. (1989). The meaning of information on G P referral rates in hospital. Community Medicine ii, 65-10.

345,112-176.

Williams A. (1993) Quality assurance from the perspective of health economies. Proceedings of the Royal Society of Edinburgh 101B. 105-1 14.

Williams A. (1994) How should information on cost-effectiveness influence clinical practice? In Outcomes into Clinical Practice (ed T. Delamothe. BMJ Publishing Group, London, pp. 99-107.

Woolf S.H. (1992) Practice guidelines - a new reality in medicine. 11-methods of developing guidelines. Archives of Internal Medicine 152, 946-952.

Woolf S.H., Battista R.N., Anderson G.M., Logan A.G. and Wang E. (1990) Assessing the effectiveness of pre- ventive manoeuvres: Analytic principles and systematic methods in reviewing evidence and developing clinical practice recommendations. Journal of Clinical Epide- miology 43, 891-905.

152 0 1996 Blackwell Science, Journal of Evaluation in Clinical Practice, 2 , 2, 131-152

central dimensions of clinical practice evaluation: efficiency, appropriateness and effectiveness -...

Documents