medical education for a changing future: new concepts for revising texts

Medical Education 1987, 21, 320-333

Medical education for a changing future: new concepts for revising texts

P. E . POLITSER

Departments of Health Policy and Manaxement and Biostatistics, Harvard School ofPublic Health, Bostori, Massachusetts

Summary. Recent psychological research suggests how to simplify texts and translate quantitative concepts into the language of clinical medicine.

Key words: *Concept formation; “education, medical, undergraduate; “writing; decision- making; problem-solving; probability learning; curriculum

Introduction

Innovations in medical education often seem to follow changes in our perceptions of medical information and knowledge. During the 195os, for example, the danger of sharp disciplinary boundaries became apparent. Educators realized that solving medical problems required integrat- ing knowledge from different fields. Subse- quently, many medical schools developed interdisciplinary curricula.

Today perceptions of medical knowledge and information are again changing, and medical educators have argued that we again need new curricula to prepare students for the future (Matheson & Lindberg 1984; Sox et al. 1984). But today’s problems differ from yesterday’s.

First, in the USA recent cost constraints pressure doctors to use information more efficiently-to reduce diagnostic testing and the length of hospital stays. The Baconian ideal of exhaustively acquiring information no longer seems appropriate. Instead we must teach students to reduce unnecessary costs.

Correspondence: Peter E. Politser M D PhD, 677 Huntington Avenue, Boston, Massachusetts OZI I S , USA.

Second, acquiring all medical knowledge no longer seems practical. According to some statistics, a standard medical text contains about 20oooo facts (Pauker et al. 1984) and students during the basic science years must learn about 24 facts per hour (Russell et al. 1984). Whether or not these figures are taken seriously, little doubt remains that medical knowledge far exceeds human learning capacities, that new material is continually added to already overburdened curricula, and that the resulting information over- loads interfere with learning (Russell et al. 1984). Many researchers conclude that we can no longer hope to train ‘omniscient’ doctors, only ‘safe’ ones, who know where their expertise ends and when to seek consultation. Some argue that to help doctors cope with the future expansion of medical knowledge we must teach them problem-solving skills as well as facts.

Unfortunately, serious questions remain about the efficacy of teaching problem-solving and efficient information use. This paper reviews the obstacles facing such education and reviews a promising direction for research. Recent psychological studies of human judgement and choice suggest how we can revise texts to simplify knowledge and translate quantitative concepts of efficient information use into the language of clinical medicine.

Coping with new cost constraints: can we teach doctors how to use information more efficiently?

Recent reports from the American Association of Medical Colleges stimulated several efforts to teach doctors probabilistic concepts to help them

N e w concepts foy revising texts 3 2 1

use information more efficiently. These concepts include the impact of disease prevalence, test sensitivity and specificity on the predictive value of a test. Doctors are also commonly taught how decision thresholds and cost-effectiveness methods can help them choose the best management stategies based on the least amount of data (see Elstein et al . 1985).

This instruction clearly helps students learn to formulate policies for diagnosing and treating diseases, and these policies can improve decisions for certain groups of patients. But whether it helps individual clinicians to apply probabilistic reasoning skills to individual patients is ques- tionable. Early efforts to teach efficient test use that appeared promising (Griner 1979) were generally more intent on pressuring doctors to reduce costs than on helping them determine when costs were justified, and attempts to teach them how to use information more accurately and efficiently had little impact. Knowledge of the statistical characteristics of tests was not associated with improved test selection (Greenland et a / . 1979), and students did not perform more efficiently on patient management problems after receiving statistical and cost-awareness training (Williams et al . 1984). Students often d o not see decision-making instruction as relevant, and they do not appear to incorporate decision- making concepts into practice (Cebul et al . 1984).

There are many explanations for the difficulty of teaching probabilistic concepts and related decision-making skills. In the USA, doctors have had financial incentives to overutilize tests, lacked adequate role models and administrative support, relied on test routines, reassured patients by ordering more tests and feared legal problems if they did not (Williams et a / . 1984). Now that cost constraints in the USA have elimi- nated some of the motivation to overutilize tests, some researchers suggest that teaching decision- making concepts there will be more effective. So far, little evidence supports this.

Even if cost constraints reduce motivational obstacles, other factors are likely to continue to hinder decision-making education. These include individual cognitive characteristics. For example, some doctors may consistently order more tests because they wish to avoid uncertainty or post-decisional regret about failing to detect a disease that is actually present (Feinstein

1985). Also, doctors may simply not need much training in probability concepts. Some researchers argue that people are too smart for such education to help, that they already adequately consider probabilistic factors in their decisions. These researchers acknowledge that people do not provide good numerical estimates ofprobabilities, but argue that this alone does not condemn human judgment. We may find differences in the brightness of a candle difficult to quantify yet intuitively appreciate them very well (Fischoff & MacGregor 1982). However, other researchers believe just the opposite-that people are not too smart. They assert that probabilistic reasoning is foreign to human logic and that human information-processing abilities are too limited to allow it. According to both argu- ments, probabilistic education is doomed.

Another theory is that faulty reasoning skills are not hereditary but acquired: that people are not in principle incorrigible, but that most of those presently around us are (Fischoff & MacGregor 1982). Doctors may already have learnt faulty ways to make decisions and have difficulty changing them (Cebul et al . 1984). Thus, we may need new types ofdoctors, trained before medical school to think probabilistically.

However, many investigators believe that the educational methods, rather than the individuals, are at fault. They point out that statistical concepts are usually taught as pure theory, involving abstract terms like test sensitivity and specificity, with no logical explanations of their meaning. They also note that the benefits ofusing quantitative concepts on a case-by-case basis may be too small to merit the mental effort they require. For example, using optimal decision-making principles may improve cost-effectiveness in very simple problems (Fryback & Thornbury 1978). but such reasoning may be very difficult in complex ones. Educational methods do not distinguish which problems probabilistic concepts are likely to resolve.

Clearly, many factors can interfere with decision-making education, including the reluctance of doctors to use numbers, the complexity of decision-making concepts, and the lack of adequate explanations of them. Later I explain how revisions in standard texts could help remove these interferences. The proposed revisions would translate numerical probabilities into

3 22 P. E . Politsev

common verbal expressions, logically explain statistical concepts and transform ‘optimal’ quantitative methods for using information into more easily used inference rules that perform nearly as well.

Coping with changes in knowledge: can we teach ‘problem-solving’?

The rapid growth of medical knowledge provides another incentive for simplifying educational methods. Many educators have noted that students can no longer hope to memorize all the essential facts since these are too numerous and are changing too quickly. ‘Problem-solving’ education is often advocated to teach students to acquire and use knowledge more efficiently. For example, Barrows & Tamblyn (1980) identified some faulty heuristics used by aspiring doctors: collecting excessive data, generating too many or too few hypotheses, and processing hypotheses inefficiently (e.g. serially rather than in multi- ples). As one remedy, Kassirer (1982) suggested that we ‘preach what we practice’, i.e. that expert doctors should make more effort to vocalize their thoughts and teach novices how they think.

Other researchers doubt that problem-solving education is effective. Experts and non-experts use the same general types of problem-solving skills (Elstein e ta l . 1978) and although experts do have additional, more specific abilities (improved memory and more sophisticated heuristics, enabling them to ignore extraneous information and solve problems more quickly Uohnson 19861) many of these permit almost immediate recognition of patterns. These skills do not involve a more teachable, deliberative activity and seem almost inseparable from knowledge. Also, performance is more highly correlated with knowledge than with diagnostic strategies (Elstein et a / . 1978). Thus, some investigators feel that problem-solving heuristics cannot be effectively taught independently of knowledge.

Perhaps this belief in the futility of problem- solving education is overstated. It is partly based on correlational, not causal, reasoning. Also, some expert judgment ir deliberative and hence potentially teachable. One non-medical study taught hypothesis-generation and testing, problem-solving and decision-making skills inde-

pendently of content and found that this improved intellectual performance (Herrnstein et al . 1986), and one medical study showed that problem-solving courses can help doctors perform better on problem-solving examinations (Babbot & Halter 1983). Some investigators suggest that we might even be able to teach pattern recognition if we can better understand the skills it requires (Gale & Marsden 1981).

Unfortunately, the literature on teaching pattern recognition is speculative, and the evidence supporting problem-solving education in medicine is not yet very convincing. The medical study mentioned was poorly controlled. Until better evidence supports these newer methods (and even when it does), it may be necessary to help students cope with expanding knowledge by improving more conventional methods. This paper presents specific suggestions for revising texts to accomplish this: simplifying texts and improving displays of knowledge. The latter may also foster more efficient problem-solving heuristics and lessen the need to teach them directly.

Improving the presentation of information in texts

The following section reviews recent psychological research suggcsting how text revisions could help overcome three obstacles to the teaching of problem-solving or quantitative concepts. First, to avoid the difficulty doctors have using numbers to express uncertainty, texts could more effectively select words to communicate disease probabilities and explain the impact of tests on them. Second, to overconic the difficulties ofteaching complex quantitative concepts, texts could translatc them into simpler and more familiar criteria for evaluating information. Third, to cope with increases in the quantity of medical knowledge, texts could reduce its burden and encourage efficient problem-solving by reorganizing it and displaying it in bctter ways.

( I ) Texts can more adequately C o i r i m i i i i c n f c probabilistic concept7ts

Early in their training, students often consider very unlikely diagnoses (Harrows & Taniblyn

New conceptsfor vevisirrg texts 323

1980). Weaknesses in appreciating disease probabilities plague even experienced doctors and may cause them to perform unnecessary diagnostic tests. One reason is that textbooks often provide little information about disease probability, and the information they do provide often seems vague. For example, in describing findings associated with acute myocardial infarction, Harvey et a / . (1976) state:

‘Patients may present with chest pain, shock, pulmonary edema, or sudden death. Occasiortally the patient may initially appear with one of the complications of myocardial infarction, such as systemic embolization or pericarditis. Severe pro- longed chest pain of characteristic location is usually present.’ (p. 257, emphases added).

Such vague descriptions of uncertainty can impair communication and education, but, until recently, it was not clear how we could improve them. The numerical meanings of such verbal expressions of likelihood vary greatly for different individuals (Bryant & Norman 1980; see Schwartz & Griffin 1986). For example, ‘occasionally’ might mean a 5 % chance to one person and a 50% chance to another. Accor- dingly, some researchers have argued that we should replace words with numbers. However, choosing and remembering the appropriate numbers are unfamiliar tasks, requiring more mental effort than words. Numbers may also communicate unwarranted precision. Thus, verbal expressions of likelihood might be preferable ifa consensus could be reached on their meaning.

Recent research suggests that a consensus on some verbal expressions of likelihood may be possible. Kong et a l . (1986) found that doctors, medical students and other medical professionals had almost the same group medians for some probability expressions; words like almost certain, probable, likely, possible or improbable. Variations between individuals existed, but they might be reduced by eliminating some words with a very high degree of disagreement (e.g., liable to happen, not unreasonable) and developing an ‘agreed-upon correspondence’ between numbers and other verbal expressions-for example, agreeing that ‘almost certain’ means about a 90% chance, ‘probable’ means about

70%, etc. The author plans to publish a code based on common usage and suggests this may gradually ‘catch on’ just as do the meanings of most words encoded in dictionaries. Medical educators could play a prominent role in dis- seminating such information.

Words could also be preferable to numbers in helping doctors revise their opinions about disease likelihood after receiving additional information; teaching them numerical calculations (Bayes’ theorem) has had limited success. Words are more naturally used with the (verbal) inference rules in clinical algorithms that help doctors revise their actions based on new data, and verbal expressions of likelihood may help people avoid somejudgmental biases. One study found that people using verbal expressions made better predictions and were less influenced by irrelevant information than those using numbers (Zimmer 1983).

Unfortunately, we lack agreement on the meaning of words expressing changes in disease probabilities, and such terms are often confused with the probabilities themselves. For example, words like ‘indeterminate’ and ‘intermediate’ are often used indiscriminately. To correct this, Pat- ton (1986) recently suggested the post-test probability of disease (e. g. positive predictive value) be called ‘indeterminate’ if it is unchanged from the pretest probability and ‘intermediate’ if it is between 25 and 75%. His choice of words is arguable; however, the need to eliminate vague word usage and to achieve a consensus on some words is not. We need research to evaluate the numerical significance of expressions like ‘test A makes disease X more likely’, ‘much more likely’, etc. If some consensus is possible, a dictionary of the meaning of such modifiers could conceivably reduce biases.

Another possible approach is to help doctors revise disease likelihoods without requiring numerical computation by verbally explaining statistical concepts, i.e. explaining why test A makes disease X much more likely. One study explained the impact of test sensitivity and specificity on disease probability in terms of the reasons for false negative or positive tests (for example, false negatives often occur with the haemoccult test for colorectal cancers because bleeding from these tumours is typically intermittent and false positives are less common as

3 24 P. E. Politrev

long as the patient follows dietary restrictions; thus, a negative result generally has less diagnostic significance than a positive one) (Politser 1984). Such explanations, even in multiple testing situations, often had a significant impact in helping doctors revise their probabilities. However, the explanations were more successful where statistical concepts were merely ignored than where prior misconceptions existed. This suggested that we need to correct misconceptions and perhaps teach statistical concepts early in the medical curriculum, before prior misconceptions have developed (this has been done, but not with adequate logical explanations). Finally, while the study did find that some explanations of statistical data were helpful, locating these explanations usually required a detailed explora- tion of the research literature. They were seldom available in standard texts. T o teach doctors about the impact of statistical factors like test sensitivity and specificity on disease probabilities, text authors need to search carefully for logical explanations and researchers presenting such data should try to provide them. Again, this may help doctors better revise their probabilities without making complex calculations.

( 2 ) Texts murt translate optimal quantitative criteria for evaluating information into movefamiliar rules

In reasonably complex medical problems, revising probability judgments one finding at a time is not advisable or feasible, even if we provide better explanations of statistical data. Inter- test dependencies and the cognitive complexity of the tasks will pose serious problems. However, in these cases texts can sometimes replace statistical concepts with more familiar logical rules. Some ofthese can perform nearly as well as optimal statistical principles. For example, if patient findings are positively correlated, conjunctive rules (e.g. diagnose disease X iffind- ings A and B are abnormal) or disjunctive rules (e.g. diagnose disease Y if findings A or B are abnormal) can closely approximate optimal statistical ones (e.g. diagnose disease X if the cumulative significance of findings A and B exceeds a given threshold, determined by regres- sion analysis) (Einhorn 1970). Thus, when tests measure similar things (and hence are correlated), more familiar logical criteria could pro-

mote adequate information use while requiring less mental effort than statistical computation.

Nevertheless, selecting a particular logical rule may be critical as different rules have different risks. Conjunctive rules minimize false positive errors (diagnosis of a disease when it is not present) but also lower true positive rates because they require that both tests be abnormal to diagnose a disease. Thus, they should be used when it is important to avoid a false positive. The opposite is true for disjunctives (wherein we diagnose a disease if either is abnormal). These should be used when it is most important to avoid false negatives (i.e. missing a disease that is present).

Even though missing a disease is often our major worry and disjunctive rules (or variants of them, like ‘at least two of five’ positive findings) would seem most appropriate, texts are heavily biased toward teaching conjunctive rather than disjunctive concepts. They typically provide conjunctive lists of findings (A and B and C, etc.) associated with a given disease (cfTable 2 ) . Per- haps they do this because experience more naturally teaches conjunctive concepts to people (including those who write texts) (Bourne & Guy 1968). However, the uncritical presentation of such concepts is not likely to promote efficient information use. Doctors may naturally learn conjunctive rules through experience in seeing diseases and their manifestations, so they may have some idea of how frequent such events may be (i.e. that findings A and B usually both occur in certain diseases). However, learning the frequency of disjunctive events (i.c. that ‘A or B’ usually occur) is cognitively more difficult. This is because learning the frequency of a disjunctive event Pr(A or B) = I -Pr (not A and not B). Thus, it requires learning the probability of ‘not A and not B’ and then realizing that the probability of ‘A or B’ equals its complement (I-Pr (not A and not €3)). I t not only requires the effort of comput- ing a complement but also requires noticing the absence of events, something people do poorly (Einhorn & Hogarth 1978). Thus, people systematically underestimate the likelihood of disjunctives (Politser 1981). As a result, disjunctives may surprise us, appear unduly important and promote inappropriate decisions. O n e example of this occurs when a doctor is asked the likelihood that ‘at least one’ test is falsely positive

N e w conceptsfor revising texts 325

when 20 independent tests are performed (this is a disjunctive event). Many are astonished to learn that the true probability of this event is so high (63 YO) and far exceeds the false positive rate for an individual test ( 5 % ) . One might argue that this is surprising only because tests are not ordinarily independent. However, Inglefinger et a\. (1983) provide examples of common laboratory profiles where the assumption is valid and where this phenomenon still evokes surprise. More important, unwarranted surprise over a test abnor- mality can lead to unnecessary, costly and even dangerous follow-up testing.

An obsession with conjunctive concepts and a tendency to ignore disease variations may also cause undue fascination with ‘unusual’ case presentations. Such presentations may seem more surprising than they really should, leading to unnecessary work-up. Sometimes these cases are even deemed worthy of publication.

Finally, excessive focus on conjunctive concepts may lead to a myopic view of the patient. Trained clinicians, wishing to adopt a ‘coherent’ formulation of a presenting problem, tend to distort reports of patient responses, especially ambiguous ones, to make them mutually consis-

Table I. A criteria table representation of knowledge concerning systemic lupus erythematosus (SLE) (from Kingsland & Lindberg 1986). This table summarizes required (REQD), exclusionary (EXCL), and other clinical findings relevant to the diagnosis of SLE (CLIN). The last category provides criteria for a definitive, probable or possible diagnosis in terms of the major or minor findings present. This table summarizes the variability in disease presentations more adequately than many other knowledge representations in texts.

Major decision elements Minor decision elements

( I ) Nephritis ( I ) Fever ( 2 ) Serious C N S disease (3) Pleuritis. O R poly-articular

(4) Malar rash (4) Photosensitivity ( 5 ) Auto-immune haemolytic anaemia

( 2 ) Arthritis, oligo-articular

O R pericarditis ( 3 ) Alopecia

( 5 ) Mucosal ulcerations (6) Hypergammaglobulinaemia, > = I . 8 gm% (7) Hypocomplementemia (8) Thrombocytopoenia. <Ioo,ooo/cmm

(9) Discoid lupus erythematosus (10) False positive VDRL or RPR ( I I ) Raynaud’s phenomenon (1.2) Circulating anticoagulant (13) Lupus band test

O R leukopenia. <3,5oo/cmm

CLIN

REQD

EXCL

Definite A. 4 majors B. 3 majors,

incl no. I or no. z z minors C. z majors,

incl no. I or no. z 4 minors D. 3 majors, incl no. I and no. z ANA+ O R LE cell+ OR anti-Sm+ O R anti-DNA+ Sclerodactyl y Arthritis, erosive anti-RNP+, with ENA>=I:Io,ooo A N D with anti-Sm

Probable A. I major

3 minors B. 4 minors C . z majors z minors

ANA+ O R LE cell+ O R anti-Sm+ O R anti-DNA+ Same

Possible A. 3 minors B. I major

z minors

N o requirements

None

3 26 P. E .

tent (Watts 19x0). If texts more adequately explained how variable disease presentations may be and avoided strictly conjunctive lists of findings, they might reduce this tendency to neglect inconsistent responses and even prevent some diagnostic errors.

This could be accomplished by more complete or systematic discussions of the common range of disease presentations. Kingsland & Lindberg (1986) systematically group evidence for systemic lupus erythematosus into four categories: major, minor, required and exclusionary (see Table I). Then they suggest that ifat least four of five ‘major’ findings are present this disease should be considered ‘definite’; ifat least three of thirteen minor elements are present it should be considered ‘possible’. Although these ‘criteria tables’ were not developed for educational pur- poses, they could help communicate the variability of disease presentations. The next section discusses the need for research to explore ways to simplify them for educational use.

Politser

( 3 ) Carefirlly chosen displays in texts can simplify knowledge and-foster more efficient information use

Presenting medical knowledge to foster more efficient information use is clearly not sufficient. We must also seek ways to present large amounts of knowledge so that it is more easily learnt and applied. This is not an entirely new suggestion. Kriel et a / . (1986) and Coles (1985) have argued that information should be presented in a format that can be more effectively translated into practice. This section suggests possible ways to accomplish this, based on rccent psychological research.

Traditional text presentations of disease criteria can be difficult to learn even when they do communicate disease variations through disjunctive rules and their variants. Examples include the IISM 111 diagnostic criteria for psychiatric diagnosis wherein one criterion for major depression is the simultaneous occurrence of ‘at least 3 of 7’ neurovegetative signs of depression (e.g. insomnia, weight loss, diminished libido) (American Psychiatric Association 1980). The complex diagnostic criteria in DSM I11 have apparently proven too difficult for most doctors, and even trained psychiatrists still do not always apply them correctly (Jampala e t a / . 1986).

One possible reason for this problem is that such criteria imply distinct boundaries for each disease category; this is not how people naturally categorize objects or represent them in memory (Bordage & Zacks 1984). Thus, an important task for future research is to find ways to reorga- nize knowledge in texts according to the way people naturally learn. Iloctors typically structure knowledge of specific diseases around general categories or prototypes (e.g. major depression and dysthymia are instances of ‘depression’). Rather than trying to learn complete lists of all the diagnostic criteria for each specific disorder, medical students might learn best by first reading several illustrations of one general category (e.g. affective disorders like major depression or dysthymia) and learning to distinguish these cases from other general categories (e.g. common features of non-affective psychiatric disorders like schizophrenia). Texts could describe general traits like neurovegetative symptoms in characterizing depressions and symptonis like mood-incongruent delusions in characterizing schizophrenia. Exam- ples of more specific distinctions (e.g. major depressions vs dysthymia) might then be more easily learnt through examples of more specific distinguishing traits (e.g. based on the duration, number and coincidence of neurovegetative signs of depression). This implies that text information for novices should be reorganized hierarchically from more general diagnostic distinctions (affective disorders vs schizophrenia) to more subtle ones (specific types of depression). Many texts do have separate chapters on each general category of disease, but these rarely have succinct and thoughtfully organized compari- sons of ‘typical’ disease presentations, proceed- ing from the more general to the more specific. In more advanced texts, written for students with more expertise, extremely specific information, such as presented in Table I , might then be more appropriate.

Knowledge should be siniplified to proniote efficient heuristics for usins information as well as for learning it. Because novices cannot edit irrelevant information so quickly as experts, we may need to present it in simplified forms until they acquire better natural editing skills. Texts cur- rently present inforniation in many different ways, but often with no apparent rationale. For

New rorrceptrfov vevisirg fextx 327

example, they may list multiple findings for a single disease (Table z), show multiple diseases for a single finding or syndrome (Table 3) or list diseases in pairs (Table 4) or in tables (Table 5 ) . Some texts also present knowledge in the form of algorithms, and some even present it exclusively in this form (Eiseman & Wotkyns 1978). These displays may foster different heuristics for using clinical information and knowledge, and each may be more useful in some situations than in others.

Some types of problems tend to be more difficult than others. The likelihood that a case will be difficult should influence the way we display the relevant knowledge. Kriel e t a / . (1986)

Table 2. Typical display ofmultiple (from Harvey ~t al. ry76)

suggest that clinicians reason from findings to a given disease rather than from diseases to findings and that listing multiple diseases for a given finding or syndrome (as in Table 3) would be most useful clinically. However, other research suggests that clinicians reason both ways. I n easier problems, they reason from findings to diseases, as when clinicians recognize patterns. In difficult problems, they reason from diseases to findings (Patel & Grocn 1986). This occurs in hypothetico-deductive reasoning, when the clinician, after formulating a disease hypothesis, asks what findings would confirm it. Such reasoning should be aided by listing multiple findings for a given disease (Table 2 ) . Texts

findings associated with a given disease

Clinical manifestations of constrictive pericarditis

History Physical examination

X-ray

Electrocardiogram

Laboratory findings Catheterization

Oedema, ascites Elevated venous pressure Collapsing venous pulse Small, quiet heart Early diastolic knock Diastolic heartbeat Oedetna, ascites, hepatomegaly Paradoxical pulse Small or slightly enlarged heart Irregularly or triangularly shaped heart Dilated superior vena cava Calcium in pericardium Low voltage ST- and T-wave changes Atrial fibrillation Low serum albumin, lymphopenia Early diastolic dip followed by diastolic

plateau in ventricular pressure Thickened pericardium detected by:

catheter in R.A. angiocardiography

Table 3. Typical display of multiple diseases associated with a given disease (or syndrome) (from Harvey et a / . 1976)

Causes of congestive heart failure

(I) Valvular heart disease (2) Hypertension in the systemic circulation (3) Hypertension in the pulmonary circulation (4) Pericardial disease ( 5 ) Myocardial disease (6) Miscellaneous rare conditions including high output states, e.g.

thyrotoxicosis, arterio-venous fistula, and anaemia

328 P. E . Politser

Table 4. Pairwise display of diseases with their distinguishing findings (from Harvey e t a l . 1976)

Differential features of myocardial and pericardial disease

Myocardial disease Pericardial disease

History Family history of heart

Alcoholism, malnutrition,

Embolism

Systemic viral illness Physical exam Gallops, S3 and S4 Pansystolic murmur PMI at LBD Prominent t, wave in JVP Pulsus alternans ECG Abnormal Q R S complexes Atrial arrhythmias ST-T changes LVH with strain Ventricular arrhythmias

disease

recent pregnancy

Asymptomatic

Tuberculosis, chest trauma, metastatic disease

Chest pain relieved by leaning forward Systemic viral illness

Friction rub Pulsus paradoxus Inspiratory increase of venous pressure Absent PMI or PMI inside LBD Collapsing y descent of JVP

Normal Low voltage Atrial arrhythmias ST-T changes

should consider including such displays, especially for difficult diagnoses where hypothetico-deductive reasoning is likely to be needed. O f course, such displays are conjunctive lists of findings and, used alone, may not adequately communicate the variability of disease manifestations. They may need to be supple- mentcd by the other displays, previously discussed.

Recent psychological research also suggests how we might better choose between various types of knowledge displays. Different displays may foster different heuristics for using knowledge, and there are trade-offs between effort and accuracy in using different heuristics. In general, those that require less mental effort tend to be less accurate. However, in different situations the performance of heuristics also may vary (Payne & Johnson 1985). For example, strategies resembling algorithms consider findings serially to eliminate (or confirm) diseases. They can greatly reduce mental effort because they can rule out (or rule in) diseases with only a few findings, and they may be fairly accurate early in the diagnostic process when we are considering many diseases, because they tend to

eliminate only very unlikely ones. However, their accuracy is liable to suffer once we have ruled out the unlikely diseases and must weigh the significance of each finding to choose between the few remaining diagnoses. With an algorithmic (serial elimination) strategy, a single unreliable finding might lead us to eliminate (or confirm) prematurely a hypothesis before considering contradictory findings appearing later in the algorithm. To illustrate these dangers, sup- pose a patient with depressed mood is evaluated according to the algorithm proposed by the American Psychiatric Association (1980) (see Fig. I ) . The following steps would be followed:

(I) Assess possible organic findings to rule out (rule in) an organic aetiology;

(2) Assess possible non-affective psychotic features to rule out non-affective psychiatric syn- dromes like schizophrenia;

(3) Assess features of the depressive syndrome to rule out major affective disorder; (4) Assess whether or not all episodes were

depressive to rule out bipolar disorder; and so forth.

In step I of the algorithm, we could prc- maturely accepf an organic diagnosis (e.g.

New concepts for revising texts 329

Table 5 . Display of multiple diseases and findings (from Harvey e l al. 1976)

Red eye: differential diagnosis

Sign Conjunctivitis Anterior uveitis Acute glauceoma

Vision

Conjunctival injection Discharge

Pain

Pupil size Pupillary response to light lntraocular pressure

Corneal appearance Anterior chamber depth Commentst

Normal or intermittent blurring that clears with blinking Diffuse

Significant, often with crusting of lashes None or minimal

Normal Normal

Normal (sterilize tonometer after use) Clear Normal

If viral, spontaneous resolution; if bacterial use topical bacteriocidal agent; if allergic or irritative, withdraw offending agent; all have good prognosis

Slightly blurred

Circumcorneal (ciliary flush) None

Moderately severe; stabbing or aching Normal or irregular Normal or irregular constriction Normal or slightly low

Clear or slightly hazy Normal

Systemic work-up indicated if recurrent; treat with topical steroids and cycloplegics

Marked blurring

Diffuse and circumcorneal

None

Very severe; often nausea and vomiting Mid-dilated and fixed Minimal or no reaction

Elevated

Steamy; opaque Shallow

Immediate therapy with pilocarpine and osmotic agents; surgical or laser iridectomy is only definitive treatment; visual prognosis dependent on duration and severity of attack

t Topical steroid preparations should never be used to treat a red eye without a specific indication since they may cause the development of glaucoma, cataracts or herpetic keratitis.

dementia in an elderly patient) without considering psychiatric findings later in the algorithm suggesting that depression is causing a pseudo- dementia. We could thus miss many dangerous, treatable disorders. Likewise, if the patient has several ill-defined somatic complaints, w e might presume an organic diagnosis, perform super- fluous tests, and perhaps even hospitalize the patient unnecessarily. Patients with chronic somatic complaints due to psychiatric disease often prompt a fruitless search for organic causes.

Later in the algorithm, at step 4, we could also erroneously eliminate bipolar disorder because the patient’s past episodes o f illness appear largely depressive and the student does not recognize (or inquire about) manic symptoms.

Empirical research in other medical areas also points towards the potential dangers of algorithms. Computer-derived algorithms appear far less accurate than experienced clini-

cians in the diagnosis o f myocardial infarction (Poretsky e t a / . 1985) .

Texts should recommend algorithms to sift through complex information early in the diagnostic process and not to establish a final diagnosis. They should also recommend use of algorithms (or protocols) t o guide the choice of medical tests or treatments only as a preliminary guide. Although algorithms are n o w often recommended to monitor cost-effectiveness, they may inadvertently reduce the quality o f care, particularly in difficult cases.

For difficult problems, such as those arising later in the diagnostic process, w e should use displays fostering other judgmental strategies. For example, after reaching the stage ofconsider- ing major affective disorders in the algorithm of Fig. I , w e could refer the student t o a display of hypotheses in pairs along with their distinguishing findings, as shown in Table 6 for unipolar us

330

Full depressive or manic syndrome sustained for at least 2 weeks or any duration if manic and hospitalized

P. E . Politrev

N~

Differential diagnosis of mood disturbance

disorder

Primary degenerative dementia with depressive t- features or organic affective syndrome

* disorder Disturbance all depressive

Depressed irritable or expansive mood predominant clinical feature

Known organic aetiology Yes

No

Psychotic features

~

4 1 ca lvladaptive reaction to a

psychosocial stressor with Atypical

No * affective disorder

Major affective disorders

No

No 4 Bil: Major

depression disc

Yes

r-- I Duration of mood disturbance

at least 2 years

Other. specific affective disorders

Adjustment 1 disorder V^^ L1^,^.

mood Figure I. Typical display of knowledge organized by an algorithm (from American Psychiatric Association 1980)

New conceptsfor revising texts 331

Table 6. Criteria for differentiating biopolar and unipolar major depression (adapted from Kolb & Brodie 1982)

Criteria Biooolar Unioolar - Family history +

Age of onset Younger Older Sex Equal Female>male Manic attack + (e.g. elevated expansive or irritable mood, increased activity, more talkative, pressured speech, flight of ideas, racing thoughts, decreased need for sleep, distractability. excessive activities with unrecognized painful consequences--e.g. buying sprees, sexual indiscretion, foolish business investments, reckless driving) Manic response when given tricyclic antidepressants or L-dopa Theraoeutic and oreventive resoonse with lithium

-

- + + -

bipolar major depression. Such displays may foster heuristics that consider a larger number of findings and do not depend on an unreliable few to eliminate an entire class of diagnoses. (Table 6 helps the student learn to enumerate the manic symptoms and other findings that could distinguish the two disorders.) Considering pairs of diagnoses, while sometimes requiring more effort than algorithms, may not require excessive effort once the number of viable diagnostic hypotheses is reduced, and it should prove more accurate. Thus, we might recommend that the student use an algorithm early and then refer to information concerning distinguishing features of the most likely pairs of diagnoses. We could present such information either in pairs (Table 3 or 6) or in tables (Table 5 ) .

Displays of pairs of diagnoses and their distinguishing features may also be more useful than algorithms for more advanced texts aimed at developing the more refined diagnostic skills characteristic ofexperts. Kassirer & Gorry (1978) suggested that experienced clinicians use a focused approach within their own areas of expertise, often asking questions that would dis- criminate the most likely hypotheses. Con- versely, outside their areas of expertise, the same doctors often use a more methodical approach. Thus, methodical algorithms might be accepta- ble for novices. However, once their expertise expands different knowledge representations may prove more useful. We may need different types of display for different phases of learning.

Conclusion

This paper has suggested that, since we may have limited abilities to teach efficient information use and problem-solving directly, we might

accomplish similar goals indirectly through revisions in medical texts. Authors could choose bet- tcr words to express probabilities, improve diagnostic criteria and simplify medical knowledge. Many of the guidelines presented are preliminary ones, based on past psychological research. It is hoped that these suggestions will stimulate further research to refine them. Ultimately, these investigations may help us understand how to make large volumes of knowledge more manageable and foster more efficient information use.

Although this paper has only presented guidelines for revising standard texts, similar principles have been applied to the design of computer knowledge management systems and electronic texts (Politser et al . 1987). These can deliver knowledge to students in many different forms and with even more flexibility, and we have demonstrated improvements in accuracy as well as reductions in the time required to make diagnoses with their use.

Many specific questions about the best ways of providing text information remain unanswered. These require additional research. However, in this era of growing knowledge and escalating costs, the immediate need for change is clear. If medical students must learn 24 facts each hour, not allowing for sleep, then learning is not a simple matter of motivation, nor of rewarding our students for their effort, nor of chastising them for their lack of it. With human capacities strained beyond their limits, we need to discover educational strategies that reduce effort, not demand more. Text revision may accomplish this. It will not eliminate the effort from learning, but it may simplify knowledge and facilitate better types of learning for the same amount of effort. We must begin the research needed to

332 P. E . Politrev

guide these revisions soon if we hope to train safe and efficient physicians for the future.

Acknowledgements

The author is recipient of NIH Research Career Development Award LMooRo from the National Library of Medicine. This research was also sup- ported in part by grant LMo4132 from the National Library of Medicine.

The author wishes to thank Frederick Mosteller, Milton Weinstein, David Hoaglin, Cleo Youtz, Suzanne Salem and Mari Mazour for their comments and assistance.

References

American Psychiatric Association (1980) Diagnostic and Statistical Manual of Mental Disorders (3rd edition). American Psychiatric Association, Washington, DC.

Anderson J.R., Boyle C.F. & Reiser B.J. (1985) Intelligent tutoring systems. Science 228, 456-62.

Babbot D. 81 Halter W.D. (1983) Clinical problem- solving skills of internists trained in the problem- oriented system. Journal of Medical Education 58, 947-53.

Baron J., Hershey J . C . & Beattie J . (1987) Heuristics and biases in diagnostic reasoning. 11: Information and confirmation. Organizational Behavior and Human Peformance (in press).

Barrows H . 81 Tamblyn R.M. (1980) Problem-based Learning: A n Approach to Medical Education. Springer, New York.

Berwick D.M. 81 Thibodeau L.A. (1983 Receiver operating characteristic analysis of diagnostic skill. Medical Care 21, 8 7 6 8 5 .

Bordage G. & Zacks R. (1984) The structure ofmedical knowledge in the memories of medical students and general practitioners: categories and prototypes. Medical Education 18, 40616.

Bourne L.E. 81 Guy D.E. (1968) Learning conceptual rules. 11: The role of positive and negative instances. Journal of Experimental Psychology 77. 488-94.

Bryant G.D. & Norman G.R. (1980) Expressions of probability: words and numbers. N e w EnglandJour- nal of Medicine 302, 41 I .

Cebul R.D., Beck L.H., Carroll J.G., Eisenberg J. M., Schwartz J.S., Strasser A.M. & Williams S.V. (1984) A course in clinical decision making adapta- ble to diverse audiences. Medical Decision Making 4, 2 8 6 9 6 .

Coles C.R. (1985) A study o f t h e relationships between curriculum and learning in undergraduate medical education. PhD thesis, University of Southampton.

Einhorn H.J. (1970) The use of non-linear, non-com- pensatory models in decision-making. Psychological Bulletin 73, 221-39.

Einhorn H.J. & Hogarth R. (1978) Confidence in the judgement: persistence of illusion of validity. Psychological Review 85, 395-416.

Eiseman B. & Wotkyns R.S. (eds) (1978) Surgical Deci- sion Making. W. B. Saunders, Philadelphia.

Elstein A.S., Dawson-Saunders B. & Belzer L. (1985) Instruction in medical decision-making. A report of two surveys. Medical Decision-Making 5. 229-3 3.

ElsteinA.S., Shu1manL.S. & SprafkaS.S. (1978) Medi- cal Problem Solving. Harvard University Press, Cambridge, Massachusetts.

Feinstein A.R. (1985) The chagrin factor and qualita- tive decision analysis. Archives of Internal Medicine

First M.B., Soffer L.J. & Miller R.A. (1985) QUICK

(Quick Index to Caduceus Knowledge): using the Internist-1 /Caduceus Knowledge Base as an electronic textbook of medicine. Computers and Biomedical Research 18, 137-65.

Fischoff B. & MacGregor D. (1982) Subjective confidence in forecasts. Journal ofForecastirig I , I 55-72.

Fryback D. & Thornbury J.R. (1978) Informal use of decision theory in radiological patient management. Radiology 129. 385-8.

Gale J. 81 Marsden P. (1981) Clinical problem solving: the beginning of the process. Medical Education 16, 22-26.

Greenland P., Mushlin A.I. & Griner P.F. (1979) Dis- crepancies between knowledge and use of diagnostic studies in asymptomatic patients. Journal sf Medical Education 54, 863-9.

Griner P.F. (1979) Use of laboratory tests in a teaching hospital: long-term trends, reduction in use and relative cost. Annals of Internal Medicine 9, 243-8.

Haight S.A., Connelly D.P., Gatewood L.C. &Burke M.D. (1983) Decision making and laboratory test utilization: expert-novice differences. In: IEEE Pro- ceeding o f the S ix th Annual Symposium on Computer Applications in Medical Care (ed. by G. S. Cohen). Washington, D.C.

Harvey A.M., Johns R.J., McKusik V.A., Owens A.H. & Ross R.S. (1976) The Principles and Practice of Medicine (20th edition). Appleton-Century- Crofts, New York.

Herrnstein R.J., Nickerson R.S., de Sanchez M. & Swets J . A . (1986) Teaching thinking skills. American Psychologist 41, 1279-89.

InglefingerJ.A., Mosteller F., Thibodeau L.A. &Ware J .H. (1983) Biostatistics in Clinical Medicine. Mac- millan, New York.

Jampala C.V., Sierles F.S. & Taylor M.A. (1986) The Impact of DSM I11 on Psychiatric Diagnosis, Annual Meeting of the American Psychiatric Associa- tion, May 1 3 .

Johnson E. (1987) Expertise and decision under uncertainty. In: The NatnreofExpertise (ed. by M. Chi, R. Glaser & M . Farr). Lawrence Erlbaum & Associ- ates, Hillsdale, New Jersey.

Kassirer J. (1982) Teaching clinical medicine by itera- tive hypothesis testing: let’s preach what we practice. N e w Englandjournal of Medicine 309. 92 1-3.

Kassirer J. 81 Gorry A. (1978) Clinical problem-solv-

145, 1257-9,

New concepts for revising text5 3 3 3

ing: a behavioral analysis. Annals oflnternal Medicine

Kingsland L.C. & Lindberg D.A.B. (1986) Thecriteria form of knowledge representation in medical artificial intelligence. In: M E D I N F O 86 (ed. by R. Solomon, B. Blum & M. Jorgensen), pp. 12-6. Elsevier Science Publishers, BV North Holland, IFIFIMIA.

Kolb L. & Brodie K. (1982) Modern Clinical Psychiafvy. W. B. Saunders, Philadelphia.

Kong A., Barnett D., Mosteller F. & Youtz C. (1986) How medical professionals evaluate expressions for probabilities. New EnglandJournal of Medicine 315, 74c-4.

Kong A. (1986) A Pilot Study on Probability Expres- sions. Memo NS-593, Department of Statistics, Harvard University, Boston.

Kriel J.R. & A’Beckett Hewson M.G. (1986) Concep- tual frameworks in preclinical and clinical textbooks. Medical Education 20, 94-101.

Matheson N. & Lindberg D.A.B. (1984) Subgroup report on medical information science skills.Journa1 ofMedica1 Education 59, I 55-9.

Patel V.L. & Groen G.J. (1986) Knowledge-based solu- tion strategies in medical reasoning. Cognitive Science 10, 91-116.

Patton D. (1986) ‘Indeterminate’ us ‘intermediate’. Medical Decision Making 6, 100.

Pauker S.G., Gorry G.A. , Kassirer J.P. & Schwartz W.B. (1984) Towards the simulation of clinical cognition: taking a present illness by computer. In: Readings in Medical Artificial Intelligence: The Firsr Decade (ed. by W. J. Clancey & E. H . Shortcliffe). Addison-Wesley Publishing Co., Reading, Massachusetts.

Payne J. W. &Johnson E. (1985) Effort and accuracy in choice. Managernent Science 31, 395-414,

Politser P. E. (198 I ) Decision analysis and clinical judgment: a reevaluation. Medical Decision M a k i q I,

Politser P.E. (1984) Explanations of statistical concepts: can they penetrate the haze of Bayes? Methods oflnformation in Medicine 23, 99-108.

Politser P.E., Gastfriend D., Bakin D. & Nguyen L.

899 245-5.

3 62-89.

(1987) An intelligent display system for psychiatric diagnosis in primary care. Medical Care (in press).

Poretsky L., Leibowitz I.H. & Friedman S.A. (1985) The diagnosis of myocardial infarction by computer derived protocol in a municipal hospital. Angiol- Ogy, 165-70.

Resnick L.B. (1983) Mathematics and science learning: a new conception. Science 220, 477-8.

Russell I.J., Hendricson W.D. & Herbert R.J. (1984) Effects of lecture information density on medical student achievement. Journal of Medical Education 59. 881-9.

Schwartz S. & Griffin 7. (1986) Medical Thinking. Springer-Verlag, Berlin.

Shanteau J. (1975) Averaging versus multiplying com- bination rules of inference judgment. Acta Psycho- logica 39, 83-9.

Sox H.C. Jr., Morgan W.L. Jr., Neufeld V.R., Sheldon G.F. & Tonesk X. (1984) Subgroup report on clinical skills. Journal of Medical Education 59, 139-47.

Sujan M. (1985) Consumer knowledge: effects on evaluation strategies mediating consumer judgments. Jourrial of Consutner Research 12, 3 1-46.

Thorngate W. (1980) Efficient decision heuristics. Behavioral Sciente 25, 2 1 ~ 2 5 .

Watts F.N. (1980) Clinical judgement and clinical training. British Journal of Medical Psychology 53, 9 5- I 08.

Williams S.V., Eisenberg J.M., Kitz D.S., Carroll J .G., Beck L.H., Rubin S.I. & Luff G.E. (1984) Teaching cost-effective diagnostic test use to medical students. Medical Care 22, 535-42.

Zimmer A.C. (1983) Verbal us numerical processing of subjective probabilities. In: Detision Making Under Uncertainly (ed. by R. W. Sholz). Elsevier Science Publishers. Amsterdam.

Receioed 1 2 November 1986; editorial comments to author 19 Defemher 1986; accepted for publication 3 March 1986

medical education for a changing future: new concepts for revising texts

Documents