Experience in improving healthcare decision-making with health IT: integrating theory, research, and practice

Matthew Samore, MDVA Salt Lake City Health Care SystemProfessor of Internal MedicineAdjunct Professor of Biomedical InformaticsUniversity of Utah

Acknowledgments Michael Rubin, MD, PhD Kim Bateman, MD Brian Sauer, PhD Lucy Savitz, PhD Tom Greene, PhD R. Scott Evans, PhD Randall Rupper, MD, MPH

Salt Lake VA Informatics, Decision Enhancement, ,and Surveillance (IDEAS) Center Selected Investigators and Collaborators Jonathan Nebeker, MD Charlene Weir, PhD Frank Drews, PhD

Partners: University of Utah, VA Salt Lake City, Healthinsight, CaduRx, Intermountain Healthcare

Funding: VA HSR&D REA 08-264 AHRQ R01 HS15413 AHRQ 1R18HS017308-01

Thesis of this talk Theory and models provide scientific

underpinnings for generalization Which supports comparative effectiveness research

For health services research and epidemiology: Use of models understood

For clinical decision support: Not so much

Health information technology ≠ informatics Sub-disciplines such as cognitive informatics crucial

More succinctly expressed:

“The difference between theory and practice is that in theory there is no difference but in practice there is”

Statement of the problem:

“A disproportionate amount of literature on the benefits [of health information technology] that have been realized comes from a small set of early-adopter institutions that implemented internally developed health information technology systems..”

Chaudhry et. al. Ann Intern Med. 2006;144:742-752

Addressing generalizability

In what contexts will effects generalize? What accounts for variability in results? Why are impacts lower in magnitude or

narrower in scope in larger trials compared to single institution studies?

How to incorporate information about implementation, adoption, formative evaluation?

Relevance to comparative effectiveness research (CER) CER priorities

Directly focused on health information technology Compare the effectiveness of alternative redesign

strategies—using decision support capabilities, electronic health records, personal health records

Indirectly tied to health information technology Compare the effectiveness of various strategies

To control MRSA To control healthcare associated infection To enhance patients’ adherence to medication regimens

Addressing CER challenges

Need to explicitly formulate causal question Determining identifiability Defining level of inference Validating methods to reduce bias

Conceptual frameworks (THEORY) Natural & engineered systems

Co-evolution Cognitive processing

Information overload ↔ fit-to-workflow Cyclical models of control

Feedback and feed-forward

System co-evolution

Fundamental theorem in informatics

C. Friedman J Am Med Inform Assoc. 2009;16:169-170

Proposed modification: Computers plus humans create a distinct

socio-technical system Characteristics are not equivalent to other

industries

Relevance

Level of inference needed to assess causal effect of health information technology: Socio-technical system

Potential benefits of simulation

Cognitive processing Motivation, mental models, tasks, goals

Influenced by social context Lack of fit-to-workflow experienced as:

Information overload Interruptions

Relevance Cognitive informatics methods

Task analysis Direct observation Match implementation strategy to task

complexity

Second law of thermodynamics as applied to cognition: Humans seek states of reduced

cognitive effort Workarounds

As cognitive load increases, automatic processing systems kick-in

For those who believe that there is a Simpson’s quote for every situation

“In this house, we obey the laws of thermodynamics!

Homer Simpson’s response when his daughter builds a perpetual motion machine in which energy increases with time

Contextual Control Model

Feed-back systems not sufficient Need to anticipate and predict

Pure feedback systems subject to loss of stability Time horizon is long in strategic control

modes Relevance

Link between decision support and surveillance Surveillance contributes feedback and feed-

forward capabilities

Feedback & feed-forward decision support

Illustrative experience with decision support for antimicrobial prescribing Two different technologies studied

Clinical task: Management of patient with acute

respiratory infection in outpatient setting Whether or not to prescribe an antibiotic Choosing the antibiotic Diagnostic label Impact of perceived or actual patient demand

Application of theory to practiceimplementation of electronic health records in rural settings Socio-technical system

Hook was electronic prescribing Stepwise approach to adoption Accommodating variation

Readiness to change Social context and clinic culture Encouraging play Avoiding information overload

Community intervention plus clinical decision support system

Standalone

algorithms on handheld computers

Community randomized trial

Samore MH et. al. JAMA. 2005 Nov 9;294(18):2305-14.

Effect on prescribing any antibiotic

Clinical decision support system integrated with computerized clinic order entry

Algorithm usually

triggered by ordering antibiotic

Clinic randomized trial

AlgorithmEnabled

AlgorithmDisabled

-80

-70

-60

-50

-40

-30

-20

-10

0

10

20

30

40

50

Rel

ativ

e Pr

eval

ence

Red

uctio

n Fr

om B

asel

ine

Effect on macrolide prescriptions

Interpretation Deciding whether to prescribe an antibiotic

and choosing the drug involve different cognitive processes

Given that decision to prescribe an antibiotic is made Possible to embed correct choice in workflow

Feed-forward decision support needed to impact the “is this a situation that warrants an antibiotic” decision

Relevant to drug-drug interaction alerting

Recommendations and conclusions Models fundamental to translation of

research into practice Incorporation of theory and models into

comparative effectiveness research Role of simulation