classic papers in medical informatics
TRANSCRIPT
![Page 1: Classic Papers in Medical Informatics](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a65860f7f8b9a931a8b66f9/html5/thumbnails/1.jpg)
Click to edit Master title style
Classic Papers in Medical Informatics
Michael Hogarth, MD, FACPProfessor, Internal Medicine
Professor, Pathology and Laboratory MedicineChair, Health Informatics Graduate Program
UC Davis
![Page 2: Classic Papers in Medical Informatics](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a65860f7f8b9a931a8b66f9/html5/thumbnails/2.jpg)
Objective
• Identify “classic” papers in medical informatics• Considerations– What is meant by ‘classic’?• What metric/measurement
– Author, citation, topic, historical
• Is it the most impactful?• Is it an innovation?
– What is “the discipline”?• Medical informatics, clinical informatics, biomedical
informatics, health informatics, ?
![Page 3: Classic Papers in Medical Informatics](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a65860f7f8b9a931a8b66f9/html5/thumbnails/3.jpg)
Prior Art
• Characterizing the “impact” of papers and topics is not new
2011
2013
1965
![Page 4: Classic Papers in Medical Informatics](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a65860f7f8b9a931a8b66f9/html5/thumbnails/4.jpg)
A classic method: “Impact Factor”
• Journal Impact Factor (JIF) – obtained from the Journal Citation Reports (Thomson Reuters)– “a measure of journal prestige and impact”– Uses the Science Citation Index (SCI), a citation database
originally developed by the Institute for Scientific Information in 1964
– Calculated by dividing the number of citations in the present year by all the articles published in the two prior years
#citations of journal articles this year
#of papers in the journal in the prior 2 years
![Page 5: Classic Papers in Medical Informatics](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a65860f7f8b9a931a8b66f9/html5/thumbnails/5.jpg)
A newer method - Eigenfactor
• Eigenfactor ranks the influence of journals using network theory and an approach similar to Google’s PageRank algorithm
• Journals are considered influential if they are cited by other influential journals
(Journal A cites Journal B 4 times)
![Page 6: Classic Papers in Medical Informatics](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a65860f7f8b9a931a8b66f9/html5/thumbnails/6.jpg)
Are “bibliometrics” a correct metric?
![Page 7: Classic Papers in Medical Informatics](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a65860f7f8b9a931a8b66f9/html5/thumbnails/7.jpg)
JAMIA Most Cited
• Earliest article – 1994
• None after
• Only lists JAMIA articles
• Citations counts can change over time – – ?shouldn’t a classic
article remain classic forever?
![Page 8: Classic Papers in Medical Informatics](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a65860f7f8b9a931a8b66f9/html5/thumbnails/8.jpg)
Most Cited – Journal of Biomedical Informatics
• Only papers since 2010• Only articles from JBI• Citation counts change
with time
http://www.journals.elsevier.com/journal-of-biomedical-informatics/most-cited-articles/
![Page 9: Classic Papers in Medical Informatics](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a65860f7f8b9a931a8b66f9/html5/thumbnails/9.jpg)
Expert searcher vs. Content expert
• Informatics experts asked by AHRQ to create a Health IT “bibliography” of ‘seminal articles’ in 11 categories
• Using the same categories, an expert librarian searcher compiled a list of ‘best’ health informatics articles using information seeking and retrieval tools
• Lists overlapped by only 27%
2009
![Page 10: Classic Papers in Medical Informatics](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a65860f7f8b9a931a8b66f9/html5/thumbnails/10.jpg)
So what did I do?
I tossed all that aside...
![Page 11: Classic Papers in Medical Informatics](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a65860f7f8b9a931a8b66f9/html5/thumbnails/11.jpg)
My Approach – educator and discipline “expert”
• Authors and Papers– Authors - individuals who were/are influential in the discipline
• Even if they didn’t publish many papers (ie, Morris Collen)• Individuals often noted to be important by myself and my peers
– Papers – Important innovations and/or new knowledge in the discipline
• Classic = “Timeless”– I reached as far back as the beginning of the discipline– Principles in the paper are ‘timeless’ – relevant today as they were
when published
• Discipline– “medical” (clinical) informatics
![Page 12: Classic Papers in Medical Informatics](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a65860f7f8b9a931a8b66f9/html5/thumbnails/12.jpg)
Some Biases
• I decided to select a “Top 20”– Why not a top 10? – just couldn’t do it...
• Historical articles are being evaluated by me without a personal context (I was not in the discipline before 1995)
![Page 13: Classic Papers in Medical Informatics](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a65860f7f8b9a931a8b66f9/html5/thumbnails/13.jpg)
So what are you getting?
• In my opinion, these are papers that every student of medical (clinical) informatics should have read!
• I can’t be wrong – it is my opinion!
![Page 14: Classic Papers in Medical Informatics](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a65860f7f8b9a931a8b66f9/html5/thumbnails/14.jpg)
![Page 15: Classic Papers in Medical Informatics](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a65860f7f8b9a931a8b66f9/html5/thumbnails/15.jpg)
1) “Reasoning Foundations of Medical Diagnosis”(Ledley and Lusted, 1959)
• First paper analyzing the reasoning processes inherent in medical diagnosis
• Introduced the use of Bayesian probability and value theory (game theory) as elements in making a medical diagnosis and choosing a therapy
• The paper clearly outlines the derivation of a form of Bayes that can be used to determine the probability that a patient (P) had a disease (f) given a set of symptoms (G)
• Paper also outlines how to use game theory to choose the treatment of greatest value to the patient
![Page 16: Classic Papers in Medical Informatics](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a65860f7f8b9a931a8b66f9/html5/thumbnails/16.jpg)
Dr. Robert Ledley -- a pioneer
• Physicist and DDS• Pioneered the use of digital
computers in biology and medicine
• First exposed to computer programming through his wife (SAEC)
• Led the development of the Automatic Computerized Transverse Axial (ACTA) scanner...the CT scan!
• National Medal of Technology (1997)
![Page 17: Classic Papers in Medical Informatics](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a65860f7f8b9a931a8b66f9/html5/thumbnails/17.jpg)
2) “Computers in Medical Data Processing”(Ledley and Lusted, 1960)
• Outlines the utility of using ‘electronic computers’ in medical diagnosis
• The first time the notion of a ‘computer network’ is mentioned in the context of clinical computing• “a great significance of a national health computer
network cannot be overestimated”
• First time the notion of using clinical ‘data’ to optimize individual care is mentioned in our literature– “patient’s like mine” or evidence-based
practice
![Page 18: Classic Papers in Medical Informatics](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a65860f7f8b9a931a8b66f9/html5/thumbnails/18.jpg)
3) “Digital Computer as Aid to Differential Diagnosis”(Lipkin, et al. 1961)
• First example of decision support in healthcare• Used structured data (information codes) • Used weights for relevant findings (ie,
‘pathognomonic = 10; positive sickle-cell prep is a 10 for sickle cell dz)
• DSS accomplished by ‘matching’ the input findings to those extracted (and coded) from a textbook (Wintrobe)
• A ranked the diagnosis list by ‘matched items’• Computer provided correct diagnosis in 57% of the
cases, but the correct diagnosis was at least in the differential list for all cases
‘information codes’ used to represent data
![Page 19: Classic Papers in Medical Informatics](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a65860f7f8b9a931a8b66f9/html5/thumbnails/19.jpg)
4) “Medical Records that Guide and Teach” (Weed; 1968)
• Paper describing “problem oriented medical records”– As a core component of using the
chart to evaluate the quality of care being rendered
• Introduced the notion of a “flowsheet” with key data over time for a particular problem
• One of my favorite quotes in the discipline:
![Page 20: Classic Papers in Medical Informatics](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a65860f7f8b9a931a8b66f9/html5/thumbnails/20.jpg)
5) “Design and Implementation of a Clinical Data Management System”(Greenes, Pappalardo, Marble, and Barnett; 1969)
• Paper outlines the nature of clinical data and the best way to ‘store’ it in a computer– “criteria for the design of a clinical data
management system include flexibility in its interface with its environment, the capability of handling variable length text string data, and of organizing it in tree-structured files”
– “with the exception of laboratory data, much of the clinical information in the medical record is generally recorded in narrative or free text form”
– “the expense and inefficiency of writing, debugging, and modifying such programs have been serious obstacles..”
• The first paper describes the “MGH Utility Multi-Programming System” (MUMPS)
![Page 21: Classic Papers in Medical Informatics](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a65860f7f8b9a931a8b66f9/html5/thumbnails/21.jpg)
The importance of MUMPS
• An interpreted language (real-time interpreter)– Recognition that clinical data management
environments do not require ‘pure central processing’
– Interpreters make it easier to have cross platform software systems
• Combining logic/computation and data storage in the same language– Provides significant speed and robustness
• Arrays as data storage – fast, simple• Multi-user by design
– multiple users executing different parts of the program at the same time using the same interpreter...
– Uses a ‘re-entrant’ interpreter
• Estimated that >70% of all health records in the US today are stored in a MUMPS-based system (Epic, VISTA, Meditech)
![Page 22: Classic Papers in Medical Informatics](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a65860f7f8b9a931a8b66f9/html5/thumbnails/22.jpg)
6) “Computer-based consultations in clinical therapeutics”(Shortliffe, 1975)
• The first example of an “expert system” based on artificial intelligence principles used in healthcare– Decision rules stored in a knowledge base– Rule system included a rule-acquisition
capability where new information could be incorporated by modifying rules in the decision rules database
• Demonstrated performance superior to infectious disease experts– MYCIN chose correctly 65% of the time– Faculty specialists: 42.5%-62.5%
• Introduced the notion of “explanations” for expert systems– critical for a system to explain its
decisions/recommendations to retain user (clinician) trust
![Page 23: Classic Papers in Medical Informatics](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a65860f7f8b9a931a8b66f9/html5/thumbnails/23.jpg)
7) “An Application-Independent Subsystem for Free-Text Analysis”
(Fenichel and Barnett; 1975)
• The first paper to describe “free text analysis” (natural language processing) on clinical records
• Implemented in MUMPS• No evaluation, though
![Page 24: Classic Papers in Medical Informatics](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a65860f7f8b9a931a8b66f9/html5/thumbnails/24.jpg)
8) “Quality Assurance through Automated Monitoring and Concurrent Feedback Using a Computer-Based Medical Information System”
(Barnett, et al; 1978)
• The first report of computerized clinical records being used for quality improvement
• An early example of “alerts and reminders” as an effective decision support tool
• The Harvard Community Health Plan implemented COSTAR in 1970– HCHP was a ‘pre-paid’ health plan (HMO)– COSTAR was the first clinical information
system to be broadly implemented
![Page 25: Classic Papers in Medical Informatics](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a65860f7f8b9a931a8b66f9/html5/thumbnails/25.jpg)
9) “Origins of Medical Informatics”(Morris Collen; 1986)
• A historical perspective of the discipline by one of its “founding fathers”– 1950-1970’s– Describes the genesis of the discipline,
including how it became known as “informatics” in the US
– Origins of information science• Hollerith, Billings and the 1890 census• Punch-card electro-mechanical machine
used to process 62milliion records in 3 years
• Hollerith in 1896 created Tabulating Machine Company, renamed IBM Corp in 1924
![Page 26: Classic Papers in Medical Informatics](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a65860f7f8b9a931a8b66f9/html5/thumbnails/26.jpg)
10) “Surveillance Using a Hospital Information System”(Classen, et al.;1991)
• A paper summarizing the use of the “Health Evaluation through Logical Processing” (HELP) for surveillance– HELP included an integrated clinical record (with data from
multiple ancillary systems)– Also has a decision support module for optimizing antibiotic
prophylaxis preoperatively
• One of the first reports of a computer based decision support system improving outcomes– Increased the appropriate use of peri-operative antibiotic
prophylaxis from 40% to 58%– Post-surgical wound infection rate decreased from 1.8% to 0.9%
• One of the first CPOE systems to be implemented routinely in a large institution– 1975 all medication ordering was computerized at LDS hospital– Implemented ‘drug alerts’ (90% resulted in changes)– Observed seizure rate for a particular antibiotic was 0.3%
compared to a national average of 2% at other medical centers
• Early use of ‘predictive analytics’– Using data from 150,000 admissions, a logistic regression model
was developed to predict hospital-acquired infection– Model correctly predicted in 65% of patients in a 1,000 patient
cohort
![Page 27: Classic Papers in Medical Informatics](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a65860f7f8b9a931a8b66f9/html5/thumbnails/27.jpg)
11) “As We May Think: The Concept Space and Medical Hypertext”(Cimino, Elkin, Barnett; 1992)
• Described the notion of “hypertext” navigation of clinical data (pre-Web)– The WWW was first described by Tim
Berner’s Lee in a proposal he wrote for his boss at the CERN lab in 1988, Mosaic was first introduced in 1993.
• Highlighted common problems with hypertext navigation– We see these often in the web – cognitive
overhead required to review multiple web page retrieval results to find if one (or more) has what you want/need
• Introduced the notion of the ‘concept space’ (semantic space)– An early description of what might be
considered a ‘semantic web’
![Page 28: Classic Papers in Medical Informatics](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a65860f7f8b9a931a8b66f9/html5/thumbnails/28.jpg)
12) “Physician Inpatient Order Writing on Microcomputer Workstations”
(Tierney, McDonald, et al.; 1993)
• One of the first examples of a randomized controlled trial (RCT) of an informatics intervention
• First large-scale study in medical informatics – 5,219 patients– 68 IM teams of students, residents, faculty
• First demonstration of the utilization/productivity improvement from physician order entry– The value of providing information “at the point of
ordering” to influence the utilization of resources
• Key Findings– Intervention reduced ‘charges’ by 12.7% -- ie, CPOE
can lead to more appropriate ordering– Length of stay reduced 0.89 days for intervention
teams– Intervention Interns spent 33min per 10-hours
*longer* to enter orders than the control group did to write them
![Page 29: Classic Papers in Medical Informatics](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a65860f7f8b9a931a8b66f9/html5/thumbnails/29.jpg)
13) “Doctors, Patients, and Computers: Will Information Technology Dehumanize Health-Care Delivery?
(Shortliffe, 1993)
• Explores the “human dimension” in using computers in healthcare– The resistance of physicians to using
computers for clinical (although they use computers for many other things)
– The perceptions of patients when they encounter a ‘computer-using’ physician
– Whether computers will ‘dehumanize’ healthcare
– The changing nature of society when it comes to acceptance of technology
– Predicts that computers will ‘recede into the environment’ as ubiquitous computing emerges
![Page 30: Classic Papers in Medical Informatics](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a65860f7f8b9a931a8b66f9/html5/thumbnails/30.jpg)
14) “A General Natural-Language Text Processor for Radiology”(Friedman et al; 1994))
• Describes the first ‘successful’ NLP system used in a clinical environment
• 230 radiology reports spanning 4 diseases (neoplasm, CHF, bacterial pneumonia, COPD)
• System has a recall and precision of 70% and 87% respectively
![Page 31: Classic Papers in Medical Informatics](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a65860f7f8b9a931a8b66f9/html5/thumbnails/31.jpg)
15) “Effect of Computerized Physician Order Entry and Team Intervention on Prevention of Serious Medication Errors”
(Bates, Leape, et al.; 1998)
• The most cited article in the discipline
• Compared two interventions for preventing serious medication errors – one intervention was CPOE
• Showed CPOE reduces non-intercepted serious medication errors by 55%
• Demonstrated the patient-safety benefit from CPOE
![Page 32: Classic Papers in Medical Informatics](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a65860f7f8b9a931a8b66f9/html5/thumbnails/32.jpg)
16) “Types of Unintended Consequences Related to Computerized Provider Order Entry”
(Campbell, Sittig, Ash, et al; 2006)
• One of the first studies focused on unintended problems/errors caused by health information technology
• Focused on CPOE• Cases from 5 large academic
medical centers• Unintended consequences
– More/new work – 19.8%– Workflow issues – 17.6 %– System demands – 14.8%
• Overhead in maintaining the system, keeping up with regulatory compliance, etc..
![Page 33: Classic Papers in Medical Informatics](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a65860f7f8b9a931a8b66f9/html5/thumbnails/33.jpg)
17) “Clinical Classification and Terminology: Some History and Current Observations”
(Chute; 2000)
• One of my favorite papers• A comprehensive ‘story’ of
terminological systems in healthcare
• A treatise on the rationale for the use of concept representation and terminologies in healthcare
• An excellent definition of “clinical terminology”:
![Page 34: Classic Papers in Medical Informatics](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a65860f7f8b9a931a8b66f9/html5/thumbnails/34.jpg)
18) “Electronic Health Records in Ambulatory Care – A National Survey of Physicians”
(DesRoches, et al; 2008)
• A survey of physician use of EHRs that revealed only 14% used any kind of EHR
• Was an important moment for the country and its policy makers with regards to EHR adoption
• Showed that a large fraction of practicing physicians are in small groups (1-3 physicians) and adoption in that segment was extremely low
![Page 35: Classic Papers in Medical Informatics](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a65860f7f8b9a931a8b66f9/html5/thumbnails/35.jpg)
19) “Decrease in Hospital-wide Mortality Rate After Implementation of a Commercially Sold Computerized Physician Order Entry System”
(Longhurst, et al; 2010)
• Demonstrated that CPOE reduced hospital-wide mortality– Estimated decrease in
adjusted mortality by 20%
– 36 fewer deaths over an 18-month period
![Page 36: Classic Papers in Medical Informatics](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a65860f7f8b9a931a8b66f9/html5/thumbnails/36.jpg)
20) “Grid Binary LOgistic REgression (GLORE): Building shared models without sharing data
(Wu, Ohno-Machado, et al; 2012)
• A paper describing a method for performing decomposable regression models against distributed remote clinical data repositories
• Fulfills the original vision promoted by Ledley and Lusted 55 years ago– a ‘national computer network’ where
clinical data could be used to optimize the decisions being made by a physician in the context of a specific patient
![Page 37: Classic Papers in Medical Informatics](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a65860f7f8b9a931a8b66f9/html5/thumbnails/37.jpg)
Honorable Mention:“What it will take to achieve the as-yet-unfulfilled promises of
health information technology”(Kellerman and Jones; Health Affairs, 2012)
the largest benefit of health IT will be overall system productivity improvement....
![Page 38: Classic Papers in Medical Informatics](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a65860f7f8b9a931a8b66f9/html5/thumbnails/38.jpg)
Questions?