supplemental digital appendix 1
TRANSCRIPT
Supplemental digital content for Kline JA, Neumann D, Raad S, et al. Impact of patient affect on physician estimate of probability of serious illness and test ordering. Acad Med. 201X;XX(X).
Copyright © by the Association of American Medical Colleges. Unauthorized reproduction is prohibited. 1
Supplemental Digital Appendix 1
Rationale, Detailed Methods, Protocol, and Supplemental Results
Rationale
This protocol examines the potential diagnostic value that lies in the nonverbal information provided by
patient facial affect, as interpreted by physicians. It is postulated that patients’ facial affect contributes to
the initial evaluation of patients with symptoms that raise suspicion of an immediate threat to life. Acute
pulmonary embolism (PE) represents a prototype disease on this issue.[2,3] Because PE can present with
complaints caused by numerous other conditions (e.g., chest pain, shortness of breath, fatigue), and
because PE is the third leading cause of cardiovascular death in the US, and can kill suddenly, clinicians
in emergency departments (EDs) tend to over test patients with possible PE.[4] For example, using a
National database, Feng et al examined outcomes of ED patients undergoing computerized tomographic
pulmonary angiography (CTPA) scans—the primary imaging modality used to diagnose and exclude
PE—only 3.7% went on to have a diagnosis of PE.[5] Over use of CTPA has negative consequences,
including radiation exposure, injury to kidneys related to iodinated contrast, false positive testing and
cost.[6-8] Reasons for over testing include medicolegal concerns, perceived normative behaviors of peers,
patient expectations and the belief in a zero tolerance for error culture [4,9-11]
Within this context, the overarching hypothesis of this study is that the quality of diagnostic testing in
emergency care is strongly influenced by the strength of the clinician’s initial belief that a patient has an
immediate threat to life. This form of metacognition has also been referred to as fast, or system one
decision-making.[12-14] Here, we refer to the quality of diagnostic testing generally as less testing,
especially with CTPA scanning, among patients who have no discernable diagnosis on longitudinal
follow-up.[2]
Supplemental digital content for Kline JA, Neumann D, Raad S, et al. Impact of patient affect on physician estimate of probability of serious illness and test ordering. Acad Med. 201X;XX(X).
Copyright © by the Association of American Medical Colleges. Unauthorized reproduction is prohibited. 2
In the emergency care setting, the initial belief about illness acuity is based on a small set of objective
data, including the patients’ chief complaint, age, gender past medical history, vital signs, and is also
determined by the clinicians’ interpretation of the patients’ overall appearance, which includes the
patient’s affect.[15,16] In emergency care, 87% clinicians use implicit clinical judgment over evidence to
make final dispositions, and this number increases to 93% in low-risk patients.[17]
Prior literature has suggested that patients with acute illness portray different emotions from healthy
subjects: disgust, fear and increased frontalis muscle tone in response to airway resistance. [18] In
previous work, we found decreased facial affect variability in patients with significant cardiopulmonary
disease, compared to those without disease. Additionally, clinicians who perceived that their patients had
smiled, rated them as having a lower probability of acute PE than patients they believed not to have
smiled.[1,19] Although the design of this previous study prevents us from making any inferences about
the diagnostic utility of facial expression analysis in clinical practice, it nevertheless, laid the groundwork
for further studies to explore the concept of using patients’ facial expressions as an additional objective
tool to assess pretest probability and improve clinician-patient communication. This is relevant given that
70% clinicians have identified communication with patients as the most important factor to reduce
unnecessary advanced imaging[20].
The overarching goal of minimizing unnecessary testing is also shared by low risk patients with chest
pain or dyspnea; one study revealed that these patients have communicated to their providers that they
desire less testing for PE than clinicians.[21] To our knowledge, no study has tested if viewing a patient’s
facial affect changes the belief of a clinician in the probability of significant disease or desire to order a
diagnostic test. Accordingly, we designed this protocol to test the hypothesis that patient affect can
change physician pretest probability of significant cardiopulmonary disease in a population of emergency
Supplemental digital content for Kline JA, Neumann D, Raad S, et al. Impact of patient affect on physician estimate of probability of serious illness and test ordering. Acad Med. 201X;XX(X).
Copyright © by the Association of American Medical Colleges. Unauthorized reproduction is prohibited. 3
patients previously selected for CT pulmonary angiography. Another aim is to compare this effect
between results of a standardized test of affect recognition, training level, specialty, and gender of the
clinicians.
Methods and Protocol
A. Overall design:
This was a pilot study conducted at Indiana University affiliated hospitals (IU Health University
Hospital, IU Health Methodist Hospital, Sidney and Lois Eskenazi Hospital). The protocol was
designed with awareness of domains used to assess effectiveness of instruments to assess
physician decision-making: 1. The conceptual framework (emergency clinicians must use limited
information to make rapid decisions), 2. The theoretical framework (the patient’s faces affects
decisions), 3. Content validity (faces of real patients recorded in real time watching a standardized
stimulus), 4. Construct validity (visual analogue scale pre and post assessment of physician global
belief of patient illness and desire to order a specific test) and 5. Internal consistency, reliability
and reproducibility (multiple observers, novice to expert of either gender from two
specialties).[22] The study has two parts: 1. Collection of patient videos and 2. Presenting the case
histories and videos to clinicians to determine the effect of affect on their belief in presence or
absence of significant CP disease. This study had approval from the institutional review board
(IRB) and all patients and physicians signed an informed consent form.
B. Patient videos
To systematically elicit changes in patients’ facial expressions under standardized conditions,
patients were presented with five visual stimuli on the laptop (Figure 1). Four were standardized
still photos from the International Affective Picture Set (IAPS): three of which have been shown
Supplemental digital content for Kline JA, Neumann D, Raad S, et al. Impact of patient affect on physician estimate of probability of serious illness and test ordering. Acad Med. 201X;XX(X).
Copyright © by the Association of American Medical Colleges. Unauthorized reproduction is prohibited. 4
to elicit strong positive Valence responses from normal persons (baby, embrace, puppies) and one
neutral to serve as a baseline (cup).[23] Patients also viewed a 26 s video clip from a “Best of”
America’s Funniest Home Videos (a cat that flips after being taunted by a bird and an excited dog
falling in a pool); We added this video as a stimulus since it had won a funniest video award in a
nationwide voting and we therefore expected it would be likely to elicit a strong (positive)
emotional response that would notably alter facial expressions. The point of interest was when the
dog falls in the pool at approximately 25 s into the video. Each of the standardized IAPS images
was projected for 4 s, separated by 1 s of a black screen. The patients’ faces were recorded only
when the image was projected. Patients were placed in semi-Fowler’s position, and the camera of
a small laptop computer (MacBook Air®, Inc., Apple Inc. Cupertino, CA) is positioned
approximately 18 inches in front of the subject. The computer’s webcam recorded participants’
facial expressions. The computer was programmed using Mac OS X to demonstrate the six-slide
presentation shown above. The cup, typically associated with a neutral response, is the control
stimulus used to establish a baseline facial expression, and was always presented first. The
order of the subsequent four slides was automatically shuffled by the operating program to present
a different order for each patient. We pre-tested, revised and refined the image collection methods
in 15 consenting patients undergoing CTPA. We found that optimal lighting conditions required a
portable photography umbrella-reflecting hood to provide uniform indirect lighting (LimoStudio-
Photography Photo Portrait Studio 600W Day Light Umbrella Continuous Lighting Kit, Rancho
Cucamonga, CA). This setup produced reliable image quality.
We obtained video images of 75 subjects, of whom 11 had the criterion standard definition of
acute CP disease. All subjects were called for additional approval before showing their videos to
physicians. We reasoned that at least 10 patients with disease positive (in this case, the definition
of significant CP disease) would be necessary to generate a meaningful receiver operating
Supplemental digital content for Kline JA, Neumann D, Raad S, et al. Impact of patient affect on physician estimate of probability of serious illness and test ordering. Acad Med. 201X;XX(X).
Copyright © by the Association of American Medical Colleges. Unauthorized reproduction is prohibited. 5
characteristic curve for each clinician tested. Table 1 shows the clinical characteristics of the
enrolled patient population. The IRB asked the investigators to reconsent all patients, during
which 2 patients withdrew, leaving 73 videos for the final work.
C. Criterion standard for significant cardiopulmonary disease
Using previously defined methods, patient disease outcomes were determined from usual medical care
processes by 1) structured evaluation of the medical record including all radiographic and laboratory
results as well as physician documentation, 2) supplemented by results of a telephone call to ask the
patient if he or she had any new diagnosis 3) Data from these two sources were then reviewed by three
independent clinicians who were blinded to each other’s opinions and had unfettered access to the entire
research database (for further clarification as needed).The three clinicians used a previously defined
explicit definition of a cardiopulmonary emergency (CPE, (Table 2) including any emergent thoracic
diagnoses that can be detected on CTPA requiring immediate treatment to prevent imminent deterioration:
this includes PE, defined as a filling defect in a segmental or larger pulmonary artery on enrollment
CTPA or within 90 days or high probability V/Q scan or ultrasound-proven DVT resulting in treatment
according to published guidelines;[24-26] pneumonia, aortic dissection or aneurysm, pneumothorax, new
thoracic mass or cardiac tamponade. Clinicians indicated a binary Yes/no determination regarding the
presence or absence for each of the 75 patients.[2] The final outcome of CPE required agreement of 2/3
clinicians and was found in 11/75 patients.
D. Experimental protocol
The experiment was conducted as a “Survey” in REDCap electronic data collection system that included
the Diagnostic Assessment of Nonverbal Accuracy (DANVA) test of facial affect recognition, followed
by standardized case reports of 75 patients shown to clinicians. [27]
Supplemental digital content for Kline JA, Neumann D, Raad S, et al. Impact of patient affect on physician estimate of probability of serious illness and test ordering. Acad Med. 201X;XX(X).
Copyright © by the Association of American Medical Colleges. Unauthorized reproduction is prohibited. 6
Physicians filled their personal information, including their specialty and years of training (Figure 2).
Then, they were shown a presentation in PowerPoint that displayed a series of 24 slides of faces with
different emotional states. The pictures were from the Diagnostic Analysis of Nonverbal Accuracy Scale
(DANVA: Nowicki & Duke, 1994)[28]. Each slide depicted a specific facial expression and was shown
for two seconds. Physicians viewed each slide and answered the dominant emotion shown among four
options being displayed: Happy, Sad, Angry, or fearful. The entire test took about 2 minutes. The
DANVA served as a measure to determine physicians’ innate ability to recognize emotions from facial
expressions. The initial part of the affect recognition test is shown in Figure 3.
After submitting the affect recognition test, physicians were directed to the main part of the survey
comprising 75 videos of patients’ faces and corresponding questions and visual analogue scales (VAS)
before and after watching the video. Physicians could watch the videos all at once or in batches. Each
subject video was logged in a spreadsheet for that physician until the physician had completed all eligible
subject videos. Physicians first read a description of subject medical history (age, chief complaint, past
medical history and vital signs) that was prepared by consensus of two authors. All case histories are
given as an addendum in this supplement. The content and importance of the case history is justified by
prior evidence showing that emergency clinicians generate the majority of diagnostic hypothesis on these
criteria, and that novice clinicians tend to over-rely on these objective data in hypothesis
generation.[29,30] Then, physicians answered a set of 2 questions based on their interpretation of subject
medical history using a visual analogue scale. The first VAS asked, “What is the probability that this
patient has a life threatening disease process” (e.g., myocardial infarction pulmonary embolism, aortic
dissection, infection with sepsis, pneumothorax, etc…). The second (#2) VAS asked, “What is your
certainty that you will order a computerized tomographic (CT) scan of the chest with intravenous (IV)
Supplemental digital content for Kline JA, Neumann D, Raad S, et al. Impact of patient affect on physician estimate of probability of serious illness and test ordering. Acad Med. 201X;XX(X).
Copyright © by the Association of American Medical Colleges. Unauthorized reproduction is prohibited. 7
contrast?” Physicians then watched the video of the patient’s face then answered the same set of questions
after viewing the video. This process is shown in Figure 5.
E. Recruitment of clinicians
Residents of postgraduate years 1-3 were eligible, as were fellows and faculty with any year of
experience. Clinicians were approached by one of the study authors and were given a verbal description
of the project and instructions on how to use the test instrument. In the process of obtaining informed
consent, authors attempted to distribute forms equally between residents, fellows, faculty with equal
representation of emergency medicine and internal medicine. Each completed the DANVA with a study
associate present, but they were able to take the remaining portion of the survey at their leisure.
Clinicians were enrolled while working in the emergency department, intensive care setting, on the wards,
or by appointment through email. Enrollment of physicians took place at any time that they were
available. (Emergency, and Internal Medicine) They had control of when to start and stop the viewing the
patient videos.
F. Data analysis and sample size
The primary analysis was the measurement of the clinician marks on the two sets of visual analogue
scales (Set #1 given after the case history and set #2 given after the video), 0-10 cm with 0 representing
0% and a 10 representing 100% measured in two sets (#1 and #2). The videos belonged to a subgroup of
patients enrolled in a larger pulmonary embolism registry and they all underwent CTPA scanning. Their
disposition was home, observation, or admission to the medicine wards or intensive care unit. Three
physician observers examined the patients’ medical records to determine whether they had a serious
cardiopulmonary emergency (CPE) diagnosed within 45 days. The explicit definitions of CPE were Acute
Coronary Syndrome (ACS), Pulmonary Embolism (PE), or other significant cardiopulmonary diagnosis
Supplemental digital content for Kline JA, Neumann D, Raad S, et al. Impact of patient affect on physician estimate of probability of serious illness and test ordering. Acad Med. 201X;XX(X).
Copyright © by the Association of American Medical Colleges. Unauthorized reproduction is prohibited. 8
as highlighted in table 2. Additionally, we will calculate a receiver operating characteristic curve for each
VAS and report the mean and standard deviation area under the ROC for all clinicians before and after
viewing the patient videos. Recognizing gender differences in decision-making style, we planned in
advance to stratify responses according to physician gender. [31]
For the first aim of the study, testing the extent to which clinicians change their opinion about patient
prognosis and need for diagnostic testing based upon the patient’s facial affect, we sought to test for a 10
mm difference between set #1 and set #2 for the first VAS using a paired t-test, representing a 10%
difference in the clinician’s belief that the patient is experiencing a potentially life-threatening illness.
This difference is justified as significant on the basis that the minimum change in diagnostic certainty for
CT scanning of emergency department patients with dyspnea is approximately 10%.[32] Similarly, we
will examine the change in mean area under the ROC for the two VAS scales with a paired t-test. Prior
work with similar patients undergoing CTPA scanning using clinician entered VAS data has found that
clinicians will mark approximately a 16±15% degree of certainty that patient has a life threatening
condition[33]. The following analysis, as shown below, indicated need for 20 clinicians from each
specialty (emergency medicine and internal medicine including intensive care) at each training level. Thus
at least 30 clinicians from each specialty were enrolled (PGY1, PGY3 and staff), assuming a worst-case
of 30% drop-out.)
Probability of type I error, Alpha = 0.05, probability of type II error (Beta) = 0.2 or 80% power to detect a
10 mm difference from zero with a standard deviation = 15 mm requires an estimated minimum sample
size = 20. Because we want to have 20 pairs of either gender, we estimate a minimum sample size of 50
physicians with complete data
Project Administration
Supplemental digital content for Kline JA, Neumann D, Raad S, et al. Impact of patient affect on physician estimate of probability of serious illness and test ordering. Acad Med. 201X;XX(X).
Copyright © by the Association of American Medical Colleges. Unauthorized reproduction is prohibited. 9
The project was administered by the Indiana University School of Medicine. The research team
comprised of emergency and internal medicine attending physicians and residents, a research professor, a
clinical research specialist, and a medical student. A biostatistician was also involved in the study design
and data management. The residents and medical student that were part of the research team conducted
physician enrollment and data collection.
Ethical Considerations
Ethical approval was granted by the IRB at Indiana University. All patients gave written informed consent
to shoot a film of their faces for the purpose of research. Similarly, all clinicians selected for the survey
were required to sign a consent form before starting the survey. A copy of the informed consent was
offered to every physician. Clinicians may have experienced temporary anxiety in trying to formulate a
decision about the patient but this was temporary. There were no potential risks for physical discomfort
involved in participating in this study, as it was observational and for data collection only. To reduce
anxiety, clinicians were assured that their results were anonymous and that they were going to be
informed about patient outcomes if they wish. Clinicians, as well as patients, could opt out of the study at
any time. All physicians and patients were adults at least 21 years of age.
Data Management and Data Quality
Research electronic and data capture (REDCap) software was used for entering, editing, and charting
survey records.
Supplemental digital content for Kline JA, Neumann D, Raad S, et al. Impact of patient affect on physician estimate of probability of serious illness and test ordering. Acad Med. 201X;XX(X).
Copyright © by the Association of American Medical Colleges. Unauthorized reproduction is prohibited. 10
RESULTS SUPPLEMENTAL TO MAIN PAPER
Overview of All Clinicians
We recruited 179 clinicians, who collectively completed 4,524 patient modules. Seeing the patients’ faces
on video caused an increase in VAS reported probability of CPE in 35%, and decrease in 48% and no
change in 17%. The video increased desire for CTPA in 35%, decreased it in 43% and caused no change
in 22%. To determine the absolute magnitude of effect (increase or decrease) of the patients’ faces on
physician pretest probability, we subtracted the first VAS from the second VAS (VAS after seeing face –
VAS before seeing face) for each of 4,524 case encounters for the change probability of a CPE and
change in desire to order a CTPA. Negative values were converted to positive to yield the absolute VAS
change caused by viewing the patient faces: for pretest probability of CPE this was 10.2 (SD 3.8) and for
desire for CTPA this was 10.0 (SD3.7). Seeing the video caused a >10% change in VAS probability of
emergency in 1,695/4,524 (37%) and >10% change in VAS need for CT in 1,624/4,524 (36%).
We compared features of the 50 physicians who completed the module to the 129 who started but did not
complete the module. Completers were older than non-completers (34 vs. 31 years, ∆=3, 95% CI for
difference in means 0.1 to 5.7) faculty [(24/50 (48%) vs. 22/129 (17%) ∆=31%, 95% CI 16 to 46%], and
more likely to be from emergency medicine [34/50 (68%) vs. 33/129 (26%) ∆=42%, 95% CI 27 to 56%.
Female gender distribution was relatively similar (17/50 (34%) vs. 48/129 (37%) ∆=3%, 95% CI -13 to
18%. Completers had a slightly higher DANVA score (18.9 vs. 18.0, ∆ = 0.9, 95% CI 0.1 to 1.7). For the
50 completers, the mean first and second VAS score for CPE were 45 (14) and 43 (16) and for CT were
36 (13) and 35 (14). For the 13 non-completers who scored more than half of the 73 videotaped patients,
the mean first and second VAS score for CPE was 48 (12) and 46 (12) and for CT was 43 (13) and 43
Supplemental digital content for Kline JA, Neumann D, Raad S, et al. Impact of patient affect on physician estimate of probability of serious illness and test ordering. Acad Med. 201X;XX(X).
Copyright © by the Association of American Medical Colleges. Unauthorized reproduction is prohibited. 11
(13). For the 113 non-completers who scored less than half of the patients, the mean first and second VAS
score for CPE were 53 (12) and 48 (13) and for CT was 47 (11) and 43 (13).
Discussion
Here, we present a methodology to assess the contribution of patient affect to clinical judgment, which we
believe to be conceptually similar to gestalt reasoning, system I (rapid) reasoning, and expert
intuition.[12-14,17] Rapid and accurate pretest probability assessment can improve appropriateness of
diagnostic test ordering.[34] Pretest probability assessment is a major factor that guides clinicians to make
decisions about diagnostic testing and prognosis. One tool that clinicians use to assess pretest probability
of disease is the overall appearance of their patients. This protocol will allow insight into the magnitude
and direction of looking at the patients’ faces on physician pretest probability assessment.
Reference List
(1) Kline JA, Neumann DM, . Relationship of patient affect to pretest probability for pulmonary
embolism. Journal of Thrombosis and Haemostasis 13[S2], 978. 2015.
(2) Kline JA, Shapiro NI, Jones AE, Hernandez-Nino J, Hogg MM, Nelson RD, Troyer JL. Outcomes
and radiation exposure of emergency department patients with chest pain and shortness of breath
and ultralow pretest probability. Ann Emerg Med 2014;63:281-8.
(3) Schuur JD, Carney DP, Lyn ET, Raja AS, Michael JA, Ross NG, Venkatesh AK. A top-five list
for emergency medicine: a pilot project to improve the value of emergency care. JAMA Intern
Med 2014;174:509-15.
Supplemental digital content for Kline JA, Neumann D, Raad S, et al. Impact of patient affect on physician estimate of probability of serious illness and test ordering. Acad Med. 201X;XX(X).
Copyright © by the Association of American Medical Colleges. Unauthorized reproduction is prohibited. 12
(4) Mozaffarian D, Benjamin EJ, Go AS, Arnett DK, Blaha MJ, Cushman M, de FS, Despres JP,
Fullerton HJ, Howard VJ, Huffman MD, Judd SE, Kissela BM, Lackland DT, Lichtman JH,
Lisabeth LD, Liu S, Mackey RH, Matchar DB, McGuire DK, et al. Heart disease and stroke
statistics--2015 update: a report from the American Heart Association. Circulation 2015;131:e29-
322.
(5) Feng LB, Pines JM, Yusuf HR, Grosse SD. U.S. trends in computed tomography use and
diagnoses in emergency department visits by patients with symptoms suggestive of pulmonary
embolism, 2001-2009. Acad Emerg Med 2013;20:1033-40.
(6) Mitchell AM, Kline JA, Jones AE, Tumlin JA. Major Adverse Events One Year After Acute
Kidney Injury After Contrast-Enhanced Computed Tomography. Ann Emerg Med 2015;66:267-
74.
(7) Adams DM, Stevens SM, Woller SC, Evans RS, Lloyd JF, Snow GL, Allen TL, Bledsoe JR,
Brown LM, Blagev DP, Lovelace TD, Shill TL, Conner KE, Aston VT, Elliott CG. Adherence to
PIOPED II investigators' recommendations for computed tomography pulmonary angiography.
Am J Med 2013;126:36-42.
(8) Schissler AJ, Rozenshtein A, Kulon ME, Pearson GD, Green RA, Stetson PD, Brenner DJ,
D'Souza B, Tsai WY, Schluger NW, Einstein AJ. CT pulmonary angiography: increasingly
diagnosing less severe pulmonary emboli. PLoS One 2013;8:e65669.
(9) Studdert DM, Mello MM, Sage WM, DesRoches CM, Peugh J, Zapert K, Brennan TA. Defensive
medicine among high-risk specialist physicians in a volatile malpractice environment.[see
comment]. JAMA 2005;293:2609-17.
Supplemental digital content for Kline JA, Neumann D, Raad S, et al. Impact of patient affect on physician estimate of probability of serious illness and test ordering. Acad Med. 201X;XX(X).
Copyright © by the Association of American Medical Colleges. Unauthorized reproduction is prohibited. 13
(10) Lucas FL, Sirovich BE, Gallagher PM, Siewers AE, Wennberg DE. Variation in cardiologists'
propensity to test and treat: is it associated with regional variation in utilization? Circ Cardiovasc
Qual Outcomes 2010;3:253-60.
(11) Rothberg MB, Class J, Bishop TF, Friderici J, Kleppel R, Lindenauer PK. The Cost of Defensive
Medicine on 3 Hospital Medicine Services. JAMA Intern Med 2014.
(12) Norman G, Sherbino J, Dore K, Wood T, Young M, Gaissmaier W, Kreuger S, Monteiro S. The
etiology of diagnostic errors: a controlled trial of system 1 versus system 2 reasoning. Acad Med
2014;89:277-84.
(13) Croskerry P, Petrie DA, Reilly JB, Tait G. Deciding about fast and slow decisions. Acad Med
2014;89:197-200.
(14) Croskerry P. A universal model of diagnostic reasoning. Acad Med 2009;84:1022-8.
(15) Pelaccia T, Tardif J, Triby E, Ammirati C, Bertrand C, Charlin B, Dory V. Insights into
emergency physicians' minds in the seconds before and into a patient encounter. Intern Emerg
Med 2015;10:865-73.
(16) Wiswell J, Tsao K, Bellolio MF, Hess EP, Cabrera D. "Sick" or "not-sick": accuracy of System 1
diagnostic reasoning for the prediction of disposition and acuity in patients presenting to an
academic ED. Am J Emerg Med 2013;31:1448-52.
(17) Calder LA, Arnason T, Vaillancourt C, Perry JJ, Stiell IG, Forster AJ. How do emergency
physicians make discharge decisions? Emerg Med J 2015;32:9-14.
Supplemental digital content for Kline JA, Neumann D, Raad S, et al. Impact of patient affect on physician estimate of probability of serious illness and test ordering. Acad Med. 201X;XX(X).
Copyright © by the Association of American Medical Colleges. Unauthorized reproduction is prohibited. 14
(18) Ritz T, Claussen C, Dahme B. Experimentally induced emotions, facial muscle activity, and
respiratory resistance in asthmatic and non-asthmatic individuals. Br J Med Psychol 2001;74 Part
2:167-82.
(19) Kline JA, Neumann D, Haug MA, Kammer DJ, Krabill VA. Decreased facial expression
variability in patients with serious cardiopulmonary disease in the emergency care setting. Emerg
Med J 2014;32:3-8.
(20) Kanzaria HK, Hoffman JR, Probst MA, Caloyeras JP, Berry SH, Brook RH. Emergency physician
perceptions of medically unnecessary advanced diagnostic imaging. Acad Emerg Med
2015;22:390-8.
(21) Geyer BC, Xu M, KABRHEL C. Patient preferences for testing for pulmonary embolism in the
ED using a shared decision-making model. Am J Emerg Med 2014;32:233-6.
(22) Legare F, Moher D, Elwyn G, LeBlanc A, Gravel K. Instruments to assess the perception of
physicians in the decision-making process of specific clinical encounters: a systematic review.
BMC Med Inform Decis Mak 2007;7:30.
(23) Lang PJ, Bradley MM, Cuthbert BN. International affective picture system: Affective ratings of
pictures and instruction manual.
(IAPS): Affective ratings of pictures and instruction manual. Gainesville, FL: University of Florida; 2008.
Report No.: Technical Report A-8.
(24) PIOPED Investigators. Value of the ventilation/perfusion scan in acute pulmonary embolism.
JAMA 1990;263:2753-9.
Supplemental digital content for Kline JA, Neumann D, Raad S, et al. Impact of patient affect on physician estimate of probability of serious illness and test ordering. Acad Med. 201X;XX(X).
Copyright © by the Association of American Medical Colleges. Unauthorized reproduction is prohibited. 15
(25) Courtney DM, Miller CD, Smithline HA, Klekowski N, Hogg MM, Kline JA. Prospective multi-
center assessment of interobserver agreement for radiologist interpretation of multidetector CT
angiography for pulmonary embolism. Journal of Thrombosis & Haemostasis 2010;8:533-40.
(26) Kearon C, Akl EA, Comerota AJ, Prandoni P, Bounameaux H, Goldhaber SZ, Nelson ME, Wells
PS, Gould MK, DENTALI F, Crowther M, Kahn SR. Antithrombotic therapy for VTE disease:
Antithrombotic Therapy and Prevention of Thrombosis, 9th ed: American College of Chest
Physicians Evidence-Based Clinical Practice Guidelines. Chest 2012;141:e419S-e494S.
(27) Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic data capture
(REDCap)--a metadata-driven methodology and workflow process for providing translational
research informatics support. J Biomed Inform 2009;42:377-81.
(28) Nowicki S, Duke MP. Individual differences in the nonverbal communication of affect: The
Diagnostic Analysis of Nonverbal Accuracy Scale. J Nonverbal Behav 1994;18:9-35.
(29) Pelaccia T, Tardif J, Triby E, Ammirati C, Bertrand C, Dory V, Charlin B. How and when do
expert emergency physicians generate and evaluate diagnostic hypotheses? A qualitative study
using head-mounted video cued-recall interviews. Ann Emerg Med 2014;64:575-85.
(30) Schubert CC, Denmark TK, Crandall B, Grome A, Pappas J. Characterizing novice-expert
differences in macrocognition: an exploratory study of cognitive work in the emergency
department. Ann Emerg Med 2013;61:96-109.
(31) Calder LA, Forster AJ, Stiell IG, Carr LK, Brehaut JC, Perry JJ, Vaillancourt C, Croskerry P.
Experiential and rational decision making: a survey to determine how emergency physicians make
clinical decisions. Emerg Med J 2012;29:811-6.
Supplemental digital content for Kline JA, Neumann D, Raad S, et al. Impact of patient affect on physician estimate of probability of serious illness and test ordering. Acad Med. 201X;XX(X).
Copyright © by the Association of American Medical Colleges. Unauthorized reproduction is prohibited. 16
(32) Pandharipande PV, Reisner AT, Binder WD, Zaheer A, Gunn ML, Linnau KF, Miller CM, Avery
LL, Herring MS, Tramontano AC, Dowling EC, Abujudeh HH, Eisenberg JD, Halpern EF,
Donelan K, Gazelle GS. CT in the Emergency Department: A Real-Time Study of Changes in
Physician Decision Making. Radiology 2015;150473.
(33) Kline JA, Stubblefield WB. Clinician gestalt estimate of pretest probability for acute coronary
syndrome and pulmonary embolism in patients with chest pain and dyspnea. Ann Emerg Med
2014;63:275-80.
(34) Roy PM, Durieux P, Gillaizeau F, Legall C, rmand-Perroux A, Martino L, Hachelaf M, Dubart
AE, Schmidt J, Cristiano M, Chretien JM, Perrier A, Meyer G. A computerized handheld
decision-support system to improve pulmonary embolism diagnosis: a randomized trial. Ann
Intern Med 2009;151:677-86.
(35) Lucassen W, Geersing GJ, Erkens PM, Reitsma JB, Moons KG, Buller H, van Weert HC. Clinical
Decision Rules for Excluding Pulmonary Embolism: A Meta-analysis. Ann Intern Med
2011;155:448-60.
(36) Runyon MS, Richman PB, Kline JA. Emergency medicine practitioner knowledge and use of
decision rules for the evaluation of patients with suspected pulmonary embolism: variations by
practice setting and training level. Acad Emerg Med 2007;14:53-7.
(37) Kline JA. Making the black box more useful. Emerg Med J 2014.
(38) Hojat M, Louis DZ, Markham FW, Wender R, Rabinowitz C, Gonnella JS. Physicians' empathy
and clinical outcomes for diabetic patients. Acad Med 2011;86:359-64.
Supplemental digital content for Kline JA, Neumann D, Raad S, et al. Impact of patient affect on physician estimate of probability of serious illness and test ordering. Acad Med. 201X;XX(X).
Copyright © by the Association of American Medical Colleges. Unauthorized reproduction is prohibited. 17
(39) Steinhausen S, Ommen O, Antoine SL, Koehler T, Pfaff H, Neugebauer E. Short- and long-term
subjective medical treatment outcome of trauma surgery patients: the importance of physician
empathy. Patient Prefer Adherence 2014;8:1239-53.
Supplemental digital content for Kline JA, Neumann D, Raad S, et al. Impact of patient affect on physician estimate of probability of serious illness and test ordering. Acad Med. 201X;XX(X).
Copyright © by the Association of American Medical Colleges. Unauthorized reproduction is prohibited. 18
Table 1: Clinical characteristics of enrolled patients (N=73)
Female gender 45 61.64 %
White race 38 52 %
Triage code >2 %
Admitted to hospital 23 31.5%
Age (years) 49 SD 17
Heart rate (beats/min) 95.1 SD 17.48
Systolic BP (mm Hg) 142.96 SD 30.82
SpO2 97.5 SD 2.62
Respiratory rate (breaths/min) 19.1 SD 4.3
Temperature (degrees C) 36.84 SD 0.55
Supplemental digital content for Kline JA, Neumann D, Raad S, et al. Impact of patient affect on physician estimate of probability of serious illness and test ordering. Acad Med. 201X;XX(X).
Copyright © by the Association of American Medical Colleges. Unauthorized reproduction is prohibited. 19
Table 2: Definitions of cardiopulmonary emergency
Criterion standard for ACS Definition was consistent with published guidelines. This included:
myocardial infarction, defined by either > 0.1 uV of ST segment
elevation in two contiguous leads on 12 lead electrocardiogram, or a
troponin I concentration that was above the 99th percentile with <10%
coefficient of variability (standard deviation/mean) with no prior history
of prior chronic troponin elevation., death thought to be from ACS
(autopsy not required), need for revascularization (angioplasty, stent
placement, or surgical bypass grafting), or a coronary stenosis on cardiac
catheterization >60% not amenable to stent placement and requiring new
medical management. A positive stress test or nuclear cardiology test
followed by new medical management in the absence of any other
criteria was not considered ACS.
Criterion standard for PE Diagnosis of PE required the medical record to demonstrate evidence of
a filling defect in a pulmonary artery on pulmonary vascular imaging
together with a clinical plan to treat, or evidence of a compression
ultrasound positive for deep vein thrombosis in the setting of
nondiagnostic pulmonary vascular imaging
Criterion standard for other
significant cardiopulmonary
diagnosis
Major adverse cardiac events (dysrhythmia requiring treatment, acute
heart failure, or pericarditis with effusion), aortic aneurysm or dissection,
other pulmonary diagnosis (pneumothorax, pneumonia), dangerous
mediastinal processes, or hemorrhage related to treatment.
Supplemental digital content for Kline JA, Neumann D, Raad S, et al. Impact of patient affect on physician estimate of probability of serious illness and test ordering. Acad Med. 201X;XX(X).
Copyright © by the Association of American Medical Colleges. Unauthorized reproduction is prohibited. 20
Figure 1: Visual stimuli shown to patients
Supplemental digital content for Kline JA, Neumann D, Raad S, et al. Impact of patient affect on physician estimate of probability of serious illness and test ordering. Acad Med. 201X;XX(X).
Copyright © by the Association of American Medical Colleges. Unauthorized reproduction is prohibited. 21
Figure 2: Physicians’ personal information
Supplemental digital content for Kline JA, Neumann D, Raad S, et al. Impact of patient affect on physician estimate of probability of serious illness and test ordering. Acad Med. 201X;XX(X).
Copyright © by the Association of American Medical Colleges. Unauthorized reproduction is prohibited. 22
Figure 3: Affect Recognition Test
Supplemental digital content for Kline JA, Neumann D, Raad S, et al. Impact of patient affect on physician estimate of probability of serious illness and test ordering. Acad Med. 201X;XX(X).
Copyright © by the Association of American Medical Colleges. Unauthorized reproduction is prohibited. 23
Figure 4: Example of case history
Supplemental digital content for Kline JA, Neumann D, Raad S, et al. Impact of patient affect on physician estimate of probability of serious illness and test ordering. Acad Med. 201X;XX(X).
Copyright © by the Association of American Medical Colleges. Unauthorized reproduction is prohibited. 24
Figure 5: Example of what clinicians viewed. On the left is the survey with the standardized history at the top. The
patient’s video appeared after the clinician recorded his or her initial VAS estimates from the standardized history.
The clinician then revised the VAS estimates after seeing the patient face. (Although all patients gave written
informed consent to have their faces shown for teaching and publication, this patient’s eyes are obscured to help
protect privacy).
Supplemental digital content for Kline JA, Neumann D, Raad S, et al. Impact of patient affect on physician estimate of probability of serious illness and test ordering. Acad Med. 201X;XX(X).
Copyright © by the Association of American Medical Colleges. Unauthorized reproduction is prohibited. 25
STANDARDIZED CASE HISTORIES—ALL PATIENTS
Faces 1
Age: 59
Primary symptom: Mental Status Change
PMH: Asthma, hypertension, meningioma and seizures
Temp: 36.7
HR: 91
Resp: 21
SpO2: 97
BP: 163/88
HT(cm): 175.26
WT(kg): 109.8
Faces 2
Age: 74
Primary symptom: Dyspnea
PMH: Hypertension, anemia, GI bleed, and anxiety
Temp: 36.0
HR: 88
Resp: 14
SpO2: 100
BP: 160/67
HT(cm): 157.5
WT(kg): 90.7
Faces 3
Supplemental digital content for Kline JA, Neumann D, Raad S, et al. Impact of patient affect on physician estimate of probability of serious illness and test ordering. Acad Med. 201X;XX(X).
Copyright © by the Association of American Medical Colleges. Unauthorized reproduction is prohibited. 26
Age: 57
Primary symptom: Dyspnea
PMH: PE and DVT on anticoagulation therapy, CAD, Hypertension, prior stroke, and HIV
Temp: 36.6
HR: 108
Resp: 18
SpO2: 99
BP: 121/83
HT(cm): 182.9
WT(kg): N/A
Faces 4
Age: 38
Primary symptom: Dyspnea
PMH: Pulmonary embolism, CAD, diabetes, hypertension and sleep apnea
Temp: 36.7
HR: 95
Resp: 20
SpO2: 99
BP: 189/102
HT(cm): 185
WT(kg): 179.3
Faces 5
Age: 49
Primary symptom: substernal chest pain
Supplemental digital content for Kline JA, Neumann D, Raad S, et al. Impact of patient affect on physician estimate of probability of serious illness and test ordering. Acad Med. 201X;XX(X).
Copyright © by the Association of American Medical Colleges. Unauthorized reproduction is prohibited. 27
PMH: Pulmonary embolism and DVT, CAD, hypertension, sleep apnea, depression, GERD, personality disorder
unspecified
Temp: 37.0
HR: 110
Resp: 24
SpO2: 97
BP: 126/110
HT(cm): 185.4
WT(kg): 147.7
Faces 6
Age: 35
Primary symptom: Substernal chest pain
PMH: Smoker and pulmonary embolism
Temp: 37.4
HR: 95
Resp: 18
SpO2: 95
BP: 134/79
HT(cm): 160.0
WT(kg): 113.7
Faces 7
Age: 85
Primary symptom: Dyspnea
PMH: Hypertension, atrial fibrillation, and HLD
Temp: 38
HR: 93
Supplemental digital content for Kline JA, Neumann D, Raad S, et al. Impact of patient affect on physician estimate of probability of serious illness and test ordering. Acad Med. 201X;XX(X).
Copyright © by the Association of American Medical Colleges. Unauthorized reproduction is prohibited. 28
Resp: 20
SpO2: 95
BP: 179/58
HT(cm): 188.0
WT(kg): 102.3
Faces 8
Age: 48
Primary symptom: Dyspnea
PMH: Prior history of pulmonary embolism and DVT, hypertension, and CHF
Temp: 36.8
HR: 108
Resp: 17
SpO2: 100
BP: 219/146
HT(cm): 195
WT(kg): 181.9
Faces 9
Age: 82
Primary symptom: Dyspnea
PMH: Asthma and hypertension
Temp: 36.8
HR: 114
Resp: 21
SpO2: 99
Supplemental digital content for Kline JA, Neumann D, Raad S, et al. Impact of patient affect on physician estimate of probability of serious illness and test ordering. Acad Med. 201X;XX(X).
Copyright © by the Association of American Medical Colleges. Unauthorized reproduction is prohibited. 29
BP: 168/81
HT(cm): 167.6
WT(kg): 79.3
Faces 10
Age: 49
Primary symptom: Dyspnea
PMH: COPD, diabetes, and hypertension
Temp: 36.4
HR: 91
Resp: 14
SpO2: 89
BP: 134/87
HT(cm): 167
WT(kg): 104.04
Faces 11
Age: 54
Primary symptom: Dyspnea
PMH: Diabetes, Hypertension, and GERD
Temp: 36.8
HR: 83
Resp: 18
SpO2: 97
BP: 214/104
Supplemental digital content for Kline JA, Neumann D, Raad S, et al. Impact of patient affect on physician estimate of probability of serious illness and test ordering. Acad Med. 201X;XX(X).
Copyright © by the Association of American Medical Colleges. Unauthorized reproduction is prohibited. 30
HT(cm): 180.32
WT(kg): 138.6
Faces 12
Age: 39
Primary symptom: Pleuritic chest pain
PMH: Hypothyroidism, peptic ulcer disease, eating disorder unspecified, and anxiety disorder
Temp: 36.6
HR: 92
Resp: 15
SpO2: 100
BP: 111/78
HT(cm): 157.5
WT(kg): 38.1
Faces 13
Age: 31
Primary symptom: Weakness
PMH: Paraplegia, hypertension, and rheumatoid arthritis
Temp: 38.0
HR: 126
Resp: 27
SpO2: 96
BP: 73/50
HT(cm): 182.9
Supplemental digital content for Kline JA, Neumann D, Raad S, et al. Impact of patient affect on physician estimate of probability of serious illness and test ordering. Acad Med. 201X;XX(X).
Copyright © by the Association of American Medical Colleges. Unauthorized reproduction is prohibited. 31
WT(kg): 95.5
Faces 14
Age: 19
Primary symptom: Dyspnea
PMH: GERD, current oral contraception
Temp: 36.7
HR: 122
Resp: 18
SpO2: 100
BP: 137/95
HT(cm): 160.0
WT(kg): 74.6
Faces 15
Age: 40
Primary symptom: Substernal chest pain,
PMH: Asthma and diabetes
Temp: 36.7
HR: 105
Resp: 26
SpO2: 100
BP: 112/76
HT(cm): 157
WT(kg): 112.9
Supplemental digital content for Kline JA, Neumann D, Raad S, et al. Impact of patient affect on physician estimate of probability of serious illness and test ordering. Acad Med. 201X;XX(X).
Copyright © by the Association of American Medical Colleges. Unauthorized reproduction is prohibited. 32
Faces 16
Age: 74
Primary symptom: Dyspnea
PMH: CAD, pulmonary embolism, hypertension, CHF, hyperlipidemia, atrial fibrillation, and history of lung
nodule
Temp:
HR: 75
Resp: 16
SpO2: 94
BP: 126/50
HT(cm): 162.6
WT(kg): 98.4
Faces 17
Age: 45
Primary symptom: Syncope or Seizure
PMH: Prothrombin mutation, on anticoagulation therapy for DVT, history of TIA, and gastric ulcer
Temp: 36.4
HR: 108
Resp: 12
SpO2: 98
BP: 116/46
HT(cm): 173
WT(kg): 105.2
Supplemental digital content for Kline JA, Neumann D, Raad S, et al. Impact of patient affect on physician estimate of probability of serious illness and test ordering. Acad Med. 201X;XX(X).
Copyright © by the Association of American Medical Colleges. Unauthorized reproduction is prohibited. 33
Faces 18
Age: 77
Primary symptom: Dyspnea
PMH: COPD
Temp: 36.2
HR: 85
Resp: 17
SpO2: 95
BP: 129/54
HT(cm): 175
WT(kg): 71.9
Faces 19
Age: 58
Primary symptom: Dyspnea
PMH: CAD, hypertension, CHF, and hyperlipidemia
Temp: 36.7
HR: 71
Resp: 14
SpO2: 97
BP: 133/75
HT(cm): 182
WT(kg): 92.4
Faces 20
Age: 49
Supplemental digital content for Kline JA, Neumann D, Raad S, et al. Impact of patient affect on physician estimate of probability of serious illness and test ordering. Acad Med. 201X;XX(X).
Copyright © by the Association of American Medical Colleges. Unauthorized reproduction is prohibited. 34
Primary symptom: Pleuritic Chest Pain
PMH: On anticoagulation therapy for PE and DVT, asthma, COPD, and fibromyalgia
Temp: 37.0
HR: 100
Resp: 20
SpO2: 99
BP: 139/73
HT(cm): 165.4
WT(kg): 100.8
Faces 21
Age: 50
Primary symptom: Hemoptysis
PMH: COPD
Temp: 36.8
HR: 59
Resp: 16
SpO2: 97
BP: 157/97
HT(cm): 177.8
WT(kg): 66
Faces 22
Age: 33
Primary symptom: Syncope or Seizure
Supplemental digital content for Kline JA, Neumann D, Raad S, et al. Impact of patient affect on physician estimate of probability of serious illness and test ordering. Acad Med. 201X;XX(X).
Copyright © by the Association of American Medical Colleges. Unauthorized reproduction is prohibited. 35
PMH: On anticoagulation therapy for pulmonary embolism, Gitelman Syndrome, and Uterine/Cervical Cancer s/p
treatment
Temp: 37.3
HR: 99
Resp: 16
SpO2: 99
BP: 111/68
HT(cm): 167.6
WT(kg): 77.3
Faces 23
Age: 32
Primary symptom: Pleuritic chest pain
PMH: Lupus, history of DVT, Addison’s disease, autoimmune hepatitis, anemia, vitamin D deficiency
Temp: 36.1
HR: 108
Resp: 16
SpO2: 99
BP: 117/74
HT(cm): 165.1
WT(kg): 86.4
Faces 24
Age: 60
Primary symptom: Substernal Chest Pain
Supplemental digital content for Kline JA, Neumann D, Raad S, et al. Impact of patient affect on physician estimate of probability of serious illness and test ordering. Acad Med. 201X;XX(X).
Copyright © by the Association of American Medical Colleges. Unauthorized reproduction is prohibited. 36
PMH: COPD, CAD, diabetes, hypertension, paroxysmal atrial fibrillation, hypothyroidism, CHF and history of
endometrial carcinoma s/p treatment
Temp: 37.8
HR: 134
Resp: 14
SpO2: 98
BP: 202/82
HT(cm): 172
WT(kg): 99.1
Faces 25
Age: 64
Primary symptom: Dyspnea
PMH: Hypertension
Temp: 36.2
HR: 111
Resp: 30
SpO2: 98
BP: 179/114
HT(cm): 167.6
WT(kg): 69.6
Faces 26
Age: 67
Primary symptom: Substernal chest pain
Supplemental digital content for Kline JA, Neumann D, Raad S, et al. Impact of patient affect on physician estimate of probability of serious illness and test ordering. Acad Med. 201X;XX(X).
Copyright © by the Association of American Medical Colleges. Unauthorized reproduction is prohibited. 37
PMH: CAD s/p CABG and multiple PCIs, diabetes, hypertension, hyperlipidemia, recently diagnosed
adenocarcinoma of colon, left BKA, and history of renal cell carcinoma s/p nephrectomy
Temp: 36.8
HR: 98
Resp: 20
SpO2: 97%
BP: 123/66
HT(cm): 176
WT(kg): 72.7
Faces 27
Age: 60
Primary symptom: Dyspnea
PMH: Hypertension, hyperlipidemia, hypothyroid, and glioblastoma undergoing chemotherapy
Temp: 36.5
HR: 68
Resp: 18
SpO2: 97%
BP: 117/64
HT(cm): 162.6
WT(kg): 78.4
Faces 28
Age: 29
Primary symptom: Dyspnea
PMH: Asthma and Hypertension
Supplemental digital content for Kline JA, Neumann D, Raad S, et al. Impact of patient affect on physician estimate of probability of serious illness and test ordering. Acad Med. 201X;XX(X).
Copyright © by the Association of American Medical Colleges. Unauthorized reproduction is prohibited. 38
Temp: 37.2
HR: 126
Resp: 23
SpO2: 96
BP: 149/73
HT(cm): 180
WT(kg): 91
Faces 29
Age: 20
Primary symptom: Substernal chest pain
PMH: Smoker
Temp: 36.9
HR: 135
Resp: 18
SpO2: 100%
BP: 143/94
HT(cm): 156
WT(kg): 76.5
Faces 30
Age: 41
Primary symptom: Dyspnea
PMH: CAD, diabetes, hypertension, PVD with right below knee amputation, CKD stage 3
Temp: 36.4
HR: 104
Supplemental digital content for Kline JA, Neumann D, Raad S, et al. Impact of patient affect on physician estimate of probability of serious illness and test ordering. Acad Med. 201X;XX(X).
Copyright © by the Association of American Medical Colleges. Unauthorized reproduction is prohibited. 39
Resp: 18
SpO2: 100%
BP: 229/93
HT(cm): 185.4
WT(kg): 88.2
Faces 31
Age: 33
Primary symptom: Dyspnea
PMH: Pulmonary embolism, obstructive sleep apnea
Temp: 36.8
HR: 87
Resp: 17
SpO2: 100%
BP: 158/89
HT(cm): 165
WT(kg): 105.2
Faces 32
Age: 36
Primary symptom: Pleuritic chest pain
PMH: On anticoagulation therapy for PE and DVT, Factor 5 Leiden variation, other genetic induced clotting
disorders
(PAI-1, MTHFR), asthma, hypertension, GERD
Temp: 36.6
HR: 77
Supplemental digital content for Kline JA, Neumann D, Raad S, et al. Impact of patient affect on physician estimate of probability of serious illness and test ordering. Acad Med. 201X;XX(X).
Copyright © by the Association of American Medical Colleges. Unauthorized reproduction is prohibited. 40
Resp: 23
SpO2: 97
BP: 121/76
HT(cm): 162.6
WT(kg): 72.7
Faces 33
Age: 28
Primary symptom: Substernal chest pain
PMH: On anticoagulation for pulmonary embolism, C-section within last month, and asthma
Temp: 37.0
HR: 81
Resp: 16
SpO2: 100%
BP: 102/82
HT(cm): 165
WT(kg): 80
Faces 34
Age: 59
Primary symptom: Dyspnea
PMH: COPD, chronic interstitial PNA, prior pulmonary embolism, HTN, CKD, depression
Temp: 37.7
HR: 135
Resp: 24
SpO2: 100%
BP: 190/69
Supplemental digital content for Kline JA, Neumann D, Raad S, et al. Impact of patient affect on physician estimate of probability of serious illness and test ordering. Acad Med. 201X;XX(X).
Copyright © by the Association of American Medical Colleges. Unauthorized reproduction is prohibited. 41
HT(cm): 170.2
WT(kg): 116.1
Faces 35
Age: 81
Primary symptom: Dyspnea
PMH: Hypertension
Temp: 36.4
HR: 73
Resp: 24
SpO2: 96%
BP: 127/67
HT(cm): 193
WT(kg): 88.7
Faces 36
Age: 18
Primary symptom: Dyspnea
PMH: No past medical history
Temp: 36.2
HR: 88
Resp: 18
SpO2: 97
BP: 136/70
HT(cm): 160
WT(kg): 73.8
Supplemental digital content for Kline JA, Neumann D, Raad S, et al. Impact of patient affect on physician estimate of probability of serious illness and test ordering. Acad Med. 201X;XX(X).
Copyright © by the Association of American Medical Colleges. Unauthorized reproduction is prohibited. 42
Faces 37
Age: 35
Primary symptom: Pleuritic chest pain
PMH: Smoker, on anticoagulation therapy for DVT
Temp: N/A
HR: 101
Resp: 18
SpO2: 98%
BP: 138/86
HT(cm): 170.2
WT(kg): 114.1
Faces 38
Age: 76
Primary symptom: Dyspnea
PMH: Hypertension, history of DVT, metastatic prostate cancer with current oral chemotherapy, and anxiety
Temp: 36.4
HR: 77
Resp: 20
SpO2: 98
BP: 152/91
HT(cm): 173
WT(kg): 69.4
Faces 39
Supplemental digital content for Kline JA, Neumann D, Raad S, et al. Impact of patient affect on physician estimate of probability of serious illness and test ordering. Acad Med. 201X;XX(X).
Copyright © by the Association of American Medical Colleges. Unauthorized reproduction is prohibited. 43
Age: 53
Primary symptom: Syncope or Seizure
PMH: Breast cancer s/p mastectomy and current chemotherapy
Temp: 36.6
HR: 80
Resp: 17
SpO2: 97%
BP: 125/58
HT(cm): 157
WT(kg): 64
Faces 40
Age: 21
Primary symptom: Pleuritic chest pain
PMH: Smoker and prior history of pulmonary embolism
Temp: 37.2
HR: 79
Resp: 16
SpO2: 96%
BP: 139/75
HT(cm): 175.3
WT(kg): 181.8
Faces 41
Age: 32
Primary symptom: Pleuritic chest pain
Supplemental digital content for Kline JA, Neumann D, Raad S, et al. Impact of patient affect on physician estimate of probability of serious illness and test ordering. Acad Med. 201X;XX(X).
Copyright © by the Association of American Medical Colleges. Unauthorized reproduction is prohibited. 44
PMH: Asthma, personality disorder unspecified, anxiety disorder
Temp: 37.1
HR: 80
Resp: 20
SpO2: 98%
BP: 125/80
HT(cm): 167.6
WT(kg): 102.3
Faces 42
Age: 53
Primary symptom: Dyspnea
PMH: COPD, smoker, and history of pericarditis
Temp: 36.8
HR: 107
Resp: 17
SpO2: 95%
BP: 123/76
HT(cm): 170
WT(kg): 153.2
Faces 43
Age: 50
Primary symptom: Syncope or Seizure
PMH: Diabetes, Hypertension, GERD, Hepatitis C, schizophrenia, and prior history of PE per patient
Supplemental digital content for Kline JA, Neumann D, Raad S, et al. Impact of patient affect on physician estimate of probability of serious illness and test ordering. Acad Med. 201X;XX(X).
Copyright © by the Association of American Medical Colleges. Unauthorized reproduction is prohibited. 45
Temp: 37.0
HR: 96
Resp: 18
SpO2: 100%
BP: 157/87
HT(cm): 182
WT(kg): 78.6
Faces 44
Age: 66
Primary symptom: Substernal chest pain
PMH: PVD s/p LE angioplasty within one week, hypertension, CHF, COPD, smoker, and anxiety disorder
Temp: 36.7
HR: 93
Resp: 20
SpO2: 97%
BP: 136/59
HT(cm): 174
WT(kg): 62.7
Faces 45
Age: 34
Primary symptom: Dyspnea
PMH: Lupus, antiphospholipid antibody, and smoker
Temp: 36.8
HR: 91
Supplemental digital content for Kline JA, Neumann D, Raad S, et al. Impact of patient affect on physician estimate of probability of serious illness and test ordering. Acad Med. 201X;XX(X).
Copyright © by the Association of American Medical Colleges. Unauthorized reproduction is prohibited. 46
Resp: 16
SpO2: 100%
BP: 105/65
HT(cm): 265
WT(kg): 77.8
Faces 46
Age: 91
Primary symptom: Pleuritic chest pain
PMH: History of DVT, hypertension, PVD, smoker, CKD, NSCLC, and history of bladder cancer
Temp: 36.1
HR: 81
Resp: 22
SpO2: 96%
BP: 168/93
HT(cm): 152.4
WT(kg): 59.1
Faces 47
Age: 51
Primary symptom: Pleuritic chest pain
PMH: COPD, hypertension, CREST syndrome, interstitial lung disease
Temp: 37.0
HR: 106
Resp: 20
Supplemental digital content for Kline JA, Neumann D, Raad S, et al. Impact of patient affect on physician estimate of probability of serious illness and test ordering. Acad Med. 201X;XX(X).
Copyright © by the Association of American Medical Colleges. Unauthorized reproduction is prohibited. 47
SpO2: 96%
BP: 176/97
HT(cm): 170.2
WT(kg): 90.4
Faces 48
Age: 34
Primary symptom: Hemoptysis
PMH: History of pulmonary embolism, diabetes, and hypertension
Temp: 36.8
HR: 85
Resp: 16
SpO2: 99
BP: 118/68
HT(cm): 154.9
WT(kg): 84.2
Faces 49
Age: 69
Primary symptom: Cough
PMH: Leukemia with ongoing therapy, DVT on anticoagulation
Temp: 37.1
HR: 95
Resp: 18
SpO2: 95%
Supplemental digital content for Kline JA, Neumann D, Raad S, et al. Impact of patient affect on physician estimate of probability of serious illness and test ordering. Acad Med. 201X;XX(X).
Copyright © by the Association of American Medical Colleges. Unauthorized reproduction is prohibited. 48
BP: 125/74
HT(cm): 165
WT(kg): 60.4
Faces 50
Age: 70
Primary symptom: Pleuritic chest pain
PMH: CAD, history of pulmonary embolism, anxiety disorder
Temp: 37.5
HR: 74
Resp: 20
SpO2: 98%
BP: 158/93
HT(cm): 177
WT(kg): 69.1
Faces 52
Age: 35
Primary symptom: Pleuritic chest pain
PMH: HIV, smoker, and non-cirrhotic portal hypertension with recent TIPS revision under conscious sedation
Temp: 37.2
HR: 77
Resp: 14
SpO2: 98%
BP: 128/48
HT(cm): 172
Supplemental digital content for Kline JA, Neumann D, Raad S, et al. Impact of patient affect on physician estimate of probability of serious illness and test ordering. Acad Med. 201X;XX(X).
Copyright © by the Association of American Medical Colleges. Unauthorized reproduction is prohibited. 49
WT(kg): 59.1
Faces 53
Age: 43
Primary symptom: Dyspnea
PMH: orthopedic surgery within last month
Temp: 36.7
HR: 90
Resp: 16
SpO2: 98%
BP: 144/85
HT(cm): 165
WT(kg): 81.9
Faces 54
Age: 27
Primary symptom: Dyspnea
PMH: Asthma
Temp: 36.7
HR: 102
Resp: 20
SpO2: 97%
BP: 149/85
HT(cm): 152.4
WT(kg): 88.2
Supplemental digital content for Kline JA, Neumann D, Raad S, et al. Impact of patient affect on physician estimate of probability of serious illness and test ordering. Acad Med. 201X;XX(X).
Copyright © by the Association of American Medical Colleges. Unauthorized reproduction is prohibited. 50
Faces 55
Age: 57
Primary symptom: Substernal chest pain
PMH: HIV, rheumatoid arthritis, smoker, anxiety disorder
Temp: 36.8
HR: 88
Resp: 16
SpO2: 94%
BP: 113/69
HT(cm): 162
WT(kg): 80
Faces 56
Age: 51
Primary symptom: Dyspnea
PMH: Diabetes, Hypertension, Depression, and history of pericarditis
Temp: 36.8
HR: 110
Resp: 26
SpO2: 98%
BP: 142/102
HT(cm): 160
WT(kg): 91.9
Faces 57
Supplemental digital content for Kline JA, Neumann D, Raad S, et al. Impact of patient affect on physician estimate of probability of serious illness and test ordering. Acad Med. 201X;XX(X).
Copyright © by the Association of American Medical Colleges. Unauthorized reproduction is prohibited. 51
Age: 57
Primary symptom: Dyspnea
PMH: History of multiple DVTs with recent cessation of anticoagulation, hypertension, hyperlipidemia, and
anxiety/depression
Temp: 36.2
HR: 121
Resp: 28
SpO2: 100%
BP: 166/113
HT(cm): 160
WT(kg): 59.1
Faces 58
Age: 34
Primary symptom: Pleuritic chest pain
PMH: Smoker
Temp: 36.7
HR: 86
Resp: 16
SpO2: 98
BP: 146/72
HT(cm): 169.5
WT(kg): 75.7
Faces 59
Age: 60
Supplemental digital content for Kline JA, Neumann D, Raad S, et al. Impact of patient affect on physician estimate of probability of serious illness and test ordering. Acad Med. 201X;XX(X).
Copyright © by the Association of American Medical Colleges. Unauthorized reproduction is prohibited. 52
Primary symptom: Dyspnea
PMH: History of PE and DVT, COPD, CAD, diabetes, hypertension, PVD with right foot amputation, ESRD on
dialysis, atrial fibrillation, GERD, Hepatitis C, and obstructive sleep apnea
Temp: 36.2
HR: 76
Resp: 12
SpO2: 100%
BP: 182/94
HT(cm): 172.7
WT(kg): 100
Faces 60
Age: 55
Primary symptom: Pleuritic chest pain
PMH: Anxiety and personality disorder unspecified
Temp: 36.4
HR: 104
Resp: 28
SpO2: 98%
BP: 123/73
HT(cm): N/A
WT(kg): 72.6
Faces 61
Age: 74
Primary symptom: Syncope or Seizure
Supplemental digital content for Kline JA, Neumann D, Raad S, et al. Impact of patient affect on physician estimate of probability of serious illness and test ordering. Acad Med. 201X;XX(X).
Copyright © by the Association of American Medical Colleges. Unauthorized reproduction is prohibited. 53
PMH: CAD
Temp: 36.2
HR: 72
Resp: 17
SpO2: 98%
BP: 122/80
HT(cm): 177.8
WT(kg): 76.2
Faces 62
Age: 24
Primary symptom: Dyspnea
PMH: Smoker, elective abortion 1 month ago
Temp: 36.6
HR: 88
Resp: 22
SpO2: 99%
BP: 105/56
HT(cm): 149.9
WT(kg): 117.5
Faces 64
Age: 32
Primary symptom: Dyspnea
PMH: Anxiety disorder
Temp: 36.9
Supplemental digital content for Kline JA, Neumann D, Raad S, et al. Impact of patient affect on physician estimate of probability of serious illness and test ordering. Acad Med. 201X;XX(X).
Copyright © by the Association of American Medical Colleges. Unauthorized reproduction is prohibited. 54
HR: 107
Resp: 24
SpO2: 96%
BP: 159/99
HT(cm): 167.6
WT(kg): 77.2
Faces 65
Age: 50
Primary symptom: Pleuritic chest pain
PMH: On anticoagulation therapy for DVT and PE, antithrombin deficiency, asthma, COPD, and hypertension
Temp: 36.9
HR: 89
Resp: 14
SpO2: 100%
BP: 154/119
HT(cm): 167.6
WT(kg): 154.5
Faces 66
Age: 54
Primary symptom: Pleuritic chest pain
PMH: CAD, hypertension, lung cancer with associated brain and skeletal metastasis undergoing chemotherapy and
radiation, smoker
Temp: 36.8
HR: 95
Supplemental digital content for Kline JA, Neumann D, Raad S, et al. Impact of patient affect on physician estimate of probability of serious illness and test ordering. Acad Med. 201X;XX(X).
Copyright © by the Association of American Medical Colleges. Unauthorized reproduction is prohibited. 55
Resp: 16
SpO2: 100%
BP: 116/91
HT(cm): 165.1
WT(kg): 64.3
Faces 67
Age: 59
Primary symptom: Pleuritic chest pain
PMH: Hypertension, obstructive sleep apnea, depression
Temp: 37.5
HR: 83
Resp: 19
SpO2: 97
BP: 145/91
HT(cm): 188
WT(kg): 157.8
Faces 68
Age: 46
Primary symptom: Dyspnea
PMH: Asthma, hypertension, obstructive sleep apnea, GERD, depression, personality disorder
Temp: 37.2
HR: 66
Resp: 18
SpO2: 97%
Supplemental digital content for Kline JA, Neumann D, Raad S, et al. Impact of patient affect on physician estimate of probability of serious illness and test ordering. Acad Med. 201X;XX(X).
Copyright © by the Association of American Medical Colleges. Unauthorized reproduction is prohibited. 56
BP: 107/61
HT(cm): 162.5
WT(kg): 109
Faces 69
Age: 45
Primary symptom: Substernal chest pain
PMH: Hypertension, DVT on anticoagulation therapy
Temp: 36.5
HR: 70
Resp: 21
SpO2: 98
BP: 137/95
HT(cm): 160
WT(kg): 74.3
Faces 70
Age: 61
Primary symptom: Dyspnea
PMH: Diabetes, hypertension, HLD, stroke, Rheumatoid arthritis, and Wolff Parkinson White
Temp: 36.4
HR: 112
Resp: 21
SpO2: 99
BP: 232/137
HT(cm): 167.64
Supplemental digital content for Kline JA, Neumann D, Raad S, et al. Impact of patient affect on physician estimate of probability of serious illness and test ordering. Acad Med. 201X;XX(X).
Copyright © by the Association of American Medical Colleges. Unauthorized reproduction is prohibited. 57
WT(kg): 86.4
Faces 71
Age: 28
Primary symptom: Dyspnea
PMH: Smoker, history of pulmonary embolism
Temp: 36.7
HR: 95
Resp: 16
SpO2: 99%
BP: 143/82
HT(cm): 170.2
WT(kg): 71.7
Faces 72
Age: 62
Primary symptom: Cough
PMH: COPD, CAD, hypertension, smoker, anxiety
Temp: 39.0
HR: 130
Resp: 18
SpO2: 83%
BP: 101/71
HT(cm): 169.6
WT(kg): 56.7
Supplemental digital content for Kline JA, Neumann D, Raad S, et al. Impact of patient affect on physician estimate of probability of serious illness and test ordering. Acad Med. 201X;XX(X).
Copyright © by the Association of American Medical Colleges. Unauthorized reproduction is prohibited. 58
Faces 73
Age: 44
Primary symptom: Dyspnea
PMH: COPD, hypertension, and anxiety
Temp: 37
HR: 117
Resp: 36
SpO2: 97%
BP: 136/89
HT(cm): 152.4
WT(kg): 76.6
Faces 74
Age: 52
Primary symptom: Dyspnea
PMH: CAD, diabetes, hypertension, paroxysmal atrial fibrillation, obstructive sleep apnea, factor 8 deficiency
Temp: 38.8
HR: 94
Resp: 21
SpO2: 96%
BP: 155/73
HT(cm): 167.6
WT(kg): 118.2
Faces 75
Age: 46
Supplemental digital content for Kline JA, Neumann D, Raad S, et al. Impact of patient affect on physician estimate of probability of serious illness and test ordering. Acad Med. 201X;XX(X).
Copyright © by the Association of American Medical Colleges. Unauthorized reproduction is prohibited. 59
Primary symptom: Pleuritic chest pain
PMH: Hypertension and smoker
Temp: 36.4
HR: 85
Resp: 18
SpO2: 99%
BP: 142/96
HT(cm): 172.7
WT(kg): 89.5
Supplemental digital content for Kline JA, Neumann D, Raad S, et al. Impact of patient affect on physician estimate of probability of serious illness and test ordering. Acad Med. 201X;XX(X).
Copyright © by the Association of American Medical Colleges. Unauthorized reproduction is prohibited. 60
Supplemental Digital Appendix 2
Figure 1A
Supplemental digital content for Kline JA, Neumann D, Raad S, et al. Impact of patient affect on physician estimate of probability of serious illness and test ordering. Acad Med. 201X;XX(X).
Copyright © by the Association of American Medical Colleges. Unauthorized reproduction is prohibited. 61
Figure 1B
Plot of the physician visual analogue scale (0-100) result for the estimate of the desire to order a
computerized tomographic pulmonary angiogram after reading the standardized case history (Figure 1A),
followed by the score after seeing the patient’s face (Figure 1B). Plots show the median and 1st-3rd
quartile (X axis) for 50 physician completers for each of 73 patients (Y axis).
Supplemental digital content for Kline JA, Neumann D, Raad S, et al. Impact of patient affect on physician estimate of probability of serious illness and test ordering. Acad Med. 201X;XX(X).
Copyright © by the Association of American Medical Colleges. Unauthorized reproduction is prohibited. 62
Supplemental Digital Appendix 3
Figure 2A
Supplemental digital content for Kline JA, Neumann D, Raad S, et al. Impact of patient affect on physician estimate of probability of serious illness and test ordering. Acad Med. 201X;XX(X).
Copyright © by the Association of American Medical Colleges. Unauthorized reproduction is prohibited. 63
Figure 2B
Plots (boxes) of visual analogue (VAS) data from 50 completer physicians for the probability of
cardiopulmonary emergency (Figure 2A) and desire for CTPA (Figure 2B). The boxes are organized from
the least experienced physicians in the upper left corner) to the most experienced in the lower right hand
corner in terms of diagnostic accuracy caused by looking at the patient face. In each box, the X-axis
represents patient number, and the left to right direction indicates increasing initial VAS value after
reading the case report. The vertical bars represent the deflection in VAS caused by seeing the patient
Supplemental digital content for Kline JA, Neumann D, Raad S, et al. Impact of patient affect on physician estimate of probability of serious illness and test ordering. Acad Med. 201X;XX(X).
Copyright © by the Association of American Medical Colleges. Unauthorized reproduction is prohibited. 64
face. Red lines are PE+ and black bold are other CPE+ patients; light gray vertical lines represent CPE-
patients.