www.mc.vanderbilt.edu/criss

Teaching & TrainingFaculty and staff provide guidance in

theories, methods and tools related to human factors through simulation-based

training and assessment.

Highly interdisciplinary and collaborative, CRISS conducts basic and applied research in healthcare informatics, patient safety and clinical quality, and designs and evaluates health information technology, care processes and medical devices.

Simulation-Based Performance Assessment

Handover Tool Development, Implementation and Evaluation Safety Labels & SignsPREDICT*: Computerized Decision Support

The Epidemiology of Perioperative Non-Routine Events Patient-Reported Non-Routine Events (NREs) Evidence-Based Cardiovascular ICU Protocol Implementation

Center for Experiential Learning & Assessment Cognitive Aids to Support Emergency Situations in Pediatric Surgery

Pre-op UI Redesign Usability Testing for Safety Enhanced Design Blood Product Verification Systems Usability Test

Patient-Centered Outcomes Research Institute (PCORI) National Institutes of Health (NIH) Agency for Healthcare Research and Quality (AHRQ) National Patient Safety Foundation (NPS) Food and Drug Administration (FDA), CDRH Anesthesia Patient Safety Foundation (APSF) National Institutes of Standards and Technology (NIST) Center for Disease Control (CDC) Veterans Affairs Health Services Research & Development Department of Energy (DOE)

Injecting Healthcare with Human Factors

Communication & Decision Making

We investigate team communication, coordination, adaptive problem solving, culture and effectiveness, and individual and group performance-shaping factors,

to generate improved clinical care processes and outcomes.

Design & UsabilityCRISS investigators design and evaluate medical devices and health information technology. We have collaborated with

the VA, other Vanderbilt centers and outside vendors to develop and

improve the user experience.

Work Analysis & Improvement

Using human factors engineering, cognitive psychology, biomedical

engineering and implementation science, CRISS studies performance during patient

care to understand how and why care deviates from optimal.

Core Faculty

Sources of Research Support

Background•Patientcaretransitions(i.e.patienthandovers),whilevitaltopatientsafety,arevariableanderrorprone.•NeonatalIntensiveCareUnit(NICU)patientsareespeciallyvulnerabletomedicalerrorsyet,toourknowledge,therearenopublishedstudiesonICU-to-ORtransitionsofcare.

Clearlyandsuccinctlydisplayingsafety-criticalinformationisvitaltosuccessfulpatient–centeredcare.Wehaveappliedourexper-tiseinhumanfactorsengineering,usabilityanalysisandgraphicdesigntoimprovesafetylabelingandsignageindiversehigh-riskclinicaldomains.RecentworkincludesaredesignedhospitalbloodbankTransfu-sionAdministrationRecord(TAR)form,asafetylabelforepiduralbags,safetytapeforhighalertinfusionmedicationsinOB-GYN,labelsforuncrossmatchedtraumabloodbagsandbloodrefrig-eratorsinthepediatricOR,andatraumabloodorderingposter.

ProblemTailoringmedicationtopatients’geneticresponses(pharmacogenetics)isnew.Raisingawarenessofpharmacogeneticadversedruginteractionsischallenging.InterventionUsingsimulationweevaluatedfourCDSmodes(active,passive,directionalandinstructional)onclinicians’useofpharmaco-geneticinformationduringprescribing.

Thisstudyaimstoimproveoperatingroomemergencyresponsebyreducingrisksassociatedwithmiss-stepsandomissionsinthedeliveryofcareduringpediatricemer-gencieswhentimelyresponseiscritical.Checklists,aformofcognitiveaid,havebeentoutedasapotentialsolutiontoclinicianknowledgegapsandfailurestoadheretostandardsoftreatmentthatoccurduringcrisissituation.

Current Application•Unstructuredlayoutanddataentryprocess•Difficultmanualentryoflabs•Out-of-workflowconfirmation•Noclearpatientstatusindicators

CRISSresearchersconductedsummativeusabilitytests(SUT)accordingtoCFR170.314(g)(43)safetyenhanceddesigncriteriaforMeaningfulUse2toevaluateVUMCusabilityofuserinterfacesininpatientandoutpatientEHRs.AreportwassubmittedtotheONC.The71separatetestsessionsinvolved60participants(registerednurses,residentphysicians,nursepractitioners,medicalfellows,andattendingphysicians).

Matt Weinger, MDCRISS DirectorHIT and Technology Usability, Event Analysis

Matt Shotwell, PhDAssistant Professor (Biostatistics)Statistical Computing and Inference

Arna Banerjee, MDAssociate Professor of Anesthesiology, Assistant Professor of SurgeryEducation and Training

Shilo Anders, PhDResearch Assistant Professor (Anesthesiology)HIT UI Design and Evaluation

Jason Slagle, PhDResearch Assistant Professor (Anesthesiology)Task, Workload and Event Analysis

Scott Watkins, MDAssistant Professor (Pediatric Cardiac Anesthesiology)Pediatric Anesthesia Simulation Programs

Dan France, PhDResearch Associate Professor (Anesthesiology and Medicine)Healthcare System Modeling

Anne Miller, PhDAssociate Professor (Human Factors)Distributed Cognition, Teamwork

Amanda Lorinc, MDAssistant Professor (Pediatric Anesthesiology)Patient Safety, Quality Improvement

Laurie Lovett Novak, PhD, MHSAAssistant Professor (Biomedical Informatics)Impact of HIT on Work Patterns

MethodsThisstudyevaluatestheeffectsofthemulti-disciplin-aryparticipatorydesignandlow-intensityimplemen-tationofastructuredNICU-to-ORhandovertoolandprocessonthequalityofhandoversandthefrequencyoftransition-of-carerelatedevents.Toevaluatetheintervention,trainedobservers,usingastructuredassessmenttool,willobserveandscoreac-tualNICU-to-ORhandoversbefore,duringandaftertheinterventionandcollectdataonnon-routineevents.

FindingsPreliminaryfindingsidentifycommonbarrierstoeffectivehandovers:distrac-tions/interruptions;unclearexpectations;communicationbarriers;competingtasks;productionpressure;andlackofstructure.Wehopetoimprovehandoverqualityanddecreasesignificantnon-rou-tineeventsbydevelopingatoolthatcanbeeasilyimplementedinothercareunits.

Problem•Bloodproductadministrationprocessissubjecttoerrors.

Method•Scenario-basedsimulationusabilitytestoftwocommer-cialtechnologies.•19nursesand3anesthesiaproviders.

Thestudyexaminestheuseofbothpaperandelectronicemergencychecklistsinsimulatedpediatricemergenciestotesttheirefficacyandeaseofuse.Analysisaboutperformanceandsubjectivedatafromquestionnaires,observationsandretrospectivevideodatawillhighlightconcernsaroundpedi-atricemergenciesintheOR,useanddesignofthechecklists,andtheirimpactonworkflowandparticipantresponse.

Anetworkofhigh-fidelitysimulationprogramsofferingall-daysimulationcoursesmandatoryforboard-cer-tifiedanesthesiologistswhodesireMaintenanceofCertificationinAnes-thesia(MOCA)addresseschallengesofmeasuringclinicalcompetency.

CRISScollaboratescloselywithVanderbilt’sCenterforExperientialLearningandAssessment(CELA)—amultipurpose,high-fidelitysimulationfacilitywith12fullyequippedclinicalexamrooms,a4-bedICUorED,andanORsuite.It’sequippedwithtwocompletecontrolrooms,mannequin-basedsimulation,andthelatestinvirtualrealitysimulatorsandpartialtasktrainers.CELAaffordsanoptimalenviron-mentforresearchinteaching.

Findings•Significantdifferencesbe-tweenthetwohand-heldbarcodescanningproducts.•Useofeitherdevicewaslessefficientthanmanualchecking.•Usabilityissuesincludedpooraccesstosubtasks,lackofprocessfeedback,inadequateerrormessaging,andconfusingdeviceinteractions.

Resultswilldelineatethere-quirementsoffuturevalidationstudiesofclinicalperformanceassessmentandprovideguid-anceforsimulation-basedcompetencyassessmentacrossmultiplespecialties.

*PharmacogenomicResourceforEnhancedDecisionsInCare&Treatment

Clinicalpracticefrequentlyvariesfromoptimalcare,yetmedicalerrorsthatdonotcausepatientharmareoftennotreported.WeintroducedtohealthcaretheconceptoftheNon-RoutineEvent(NRE),modeledaftersafetyprocessesinthenuclearpowerindustrywhereeverydeviationfromstandardoperatingprocedures(SOPs)isreportedandinvestigated.Inhealthcare,anNREisdefinedas“any aspect of clinical care perceived by clinicians or trained observers as deviating from optimal care for that patient in that clinical situation.”Inaseriesofstudies,wehaveshownthatperioperativeclinician-reportedNREs:1)arefrequent(≥1NREinupto40%ofallcareperiodsstudied);2)captureawidecross-sectionofsystemfailures;and3)areassociatedwithincreasedclinicianworkloadandwithsignificantpatientphysiologicaldisturbances.NREsprovideawindowonsystemsafetyandcanbeusedasadependentvariableinsafetyinterventions.

InastudyfundedbythePatientCenteredOutcomesResearchInstitute(PCORI),CRISShascollectedNREsfrompatients,familymembersandpatients’cliniciansinfourmedicalsettings:ambulatorysurgery,interventionalcardiology,pediatriconcology,andpediatriccardiacsurgery.Patient-andclinician-reportedNREswerecommon.Ininterventionalcardiologycases,70%containedpatient-reportedNREs(thehighestofourfoursettings)while47%containedclinician-reportedNREs.PatientNREsrarelyover-lappedwithclinicianNREs.Acrossallsettings,mostpatientNREsreflecteddeficientcaredeliveryprocesses– i.e.,thecarewasnotpatient-centered.

ResultsActive,directionalCDSpromotedgreateruseofpharmacoge-neticinformation,butmayleadtosub-optimalresultsinsituationuncertainty.

Redesigned UI•Progressivedisclosureofmodularqueries•Personalizedaccesswithabilitytoauditpriorentries•Clear,easyconfirma-tionprocess•Improvedabilitytoscanforpertinentpositivesandalerts

Event Detection & Analysis Methods

shilo.anders@vanderbilt.edumatt.weinger@vanderbilt.edu jason.slagle@vanderbilt.edu scott.watkins@vanderbilt.edumatt.shotwell@vanderbilt.edudan.france@vanderbilt.eduarna.banerjee@vanderbilt.edu anne.miller@vanderbilt.eduamanda.lorinc@vanderbilt.edu laurie.l.novak@vanderbilt.edu

Inmultipleprojectsinvari-ouscaredomains(e.g.,theICU,cardiology,periopera-tiveservices,andoncology),CRISSprospectively(e.g.,>900video-recordingsandinterviewsofsurgicalcases)andretrospectively(i.e.,semi-structuredinterviews)applieseventdetectionandanalyticalmethodstoassesstheetiolo-gyandimpactofadverseeventsandNon-RoutineEvents(NREs)onthequalityandsafetyofclinicalcareaswellastheirrelationshipswithvariousper-formance-shapingfactorsandclinicaloutcomes.

Summary of Sample Non- Routine EventThesurgeonendsupperforminganunsafeact(i.e.,standingonanupsidedowntrashcantoadjustequipment)thatdisruptedthesurgery,endan-

geredthesurgeonandpatient,anddistractedtheentireORteam.

PurposePost-surgicaldischargefromtheCVICUwithin48hourssignificantlyreducestherisksofcomplicationssuchasVAP,deliriumandcatheter-relatedinfections.MethodWeimplementedacollaborativeprospectivepathwaytrackingtoolandprocessthatsupportedthecommunicationofpatientprogressinandbetweenmorningrounds.ResultsPatientswithlowgoalscoresandpatientswithhighteamadherencescoreshadshorterlengthsofICUandHospitalstay.

Figure 1. NRE Schematic

•  A “non-routine event” (NRE) is defined as any event that is perceived by clinicians, patients and/or skilled observers to deviate from ideal care for that specific patient in that specific clinical situation (Figure 1).1

In previous studies, we have shown that NREs: •  can be reliably collected prospectively from physicians

and nurses in various care settings 2,3 •  are frequent (from 15% to 50%) •  are often associated with patient impact or injury 4

•  provide data about the nature and severity of process deficiencies that could cause future patient injury

•  Thus far, we know very little about how patients and their families/caregivers view ‘non-routine’ events in perioperative care.

•  Determine what aspects of their clinical encounters do patients and families view as “non-routine” and that reflect low care quality or safety issues.

•  Elucidate the factors that influence the reporting of perioperative NREs and affect the nature of the NREs that are reported.

•  Determine whether NREs obtained from patients/families add to evidence about clinical system failure modes beyond that obtained from clinicians caring for the same patients.

DEPARTMENT OF ANESTHESIOLOGY

1. Weinger MB, Slagle J: Human factors research in anesthesia patient safety: Techniques to elucidate factors affecting clinical task performance and decision-making. JAMIA 2002; 9(6): S58-63

2. Oken A, Rasmussen MD, Slagle JM, Jain S, Kuykendall T, Ordonez N, Weinger MB: A facilitated survey instrument captures significantly more anesthesia events than does traditional voluntary event reporting. Anesthesiology 2007; 107(6), 909-22

3. Rayo M, Smith P, Weinger MB, Slagle JS: Assessing medication safety technology in the intensive care unit. Proc Hum Factors Ergon Soc 2007; 51:692-96

4. Slagle JM, Anders S, Porterfield E, Arnold A, Calderwood C, Weinger MB: Significant physiological disturbances associated with non-routine event containing and routine anesthesia cases. J Patient Safety (in press)

This study was also supported by Academic Program Support funds from VUMC to CRISS (Center for Research and Innovation in Systems Safety).

•  We refined our previous NRE collection tool, the Comprehensive Open-ended Non-routine Event Survey (CONES),2 for use with patients based on a thematic analysis of patient/caregiver focus groups, input from our team’s patient representatives, and pilot testing. (Figure 2)

•  Preoperatively, we obtained written consent and demographic data from patients having elective ambulatory surgery (discharged within 23 hours).

•  Postoperatively, trained investigators collected NREs from the patients and also from their anesthesia providers, surgeons and perioperative nurses.

•  Patients were surveyed with the Patient-specific CONES tool prior to discharge and then again by phone approximately one-week later.

•  In the 140 patients studied (age 58±14 yrs, 54% male), there were a total of 160 clinician-reported NREs and 87 patient-reported NREs. There were 214 unique NREs.

•  Eighty-two surgical cases (59%) contained clinician-reported NREs while 57 cases (41%) contained patient-reported NREs. (See examples, Figure 3)

•  Both the patient and at least one clinician reported an NRE in 39 cases (28%), although they were rarely about the same event.

•  CRNA’s were most likely to report an NRE (69% of cases), OR nurses and surgeons reported NREs in 53% and 55%, respectively, while anesthesia residents reported NREs least often (34%).

•  More than half (54%) of all NREs involved care tasks/processes, 23% involved clinicians’ actions/inactions and 13% involved technology.

•  The themes of the NREs are shown in Table 1.

•  In this preliminary study, 40% of ambulatory surgery patients reported NREs and these included clinical care deviations, communication failures, and service deficiencies.

•  NRE collection from perioperative patients is feasible and appears a valuable source of quality and safety data.

•  Patient NREs rarely overlapped with clinician NREs. •  Most patient NREs represented deficient care delivery processes – The care

provided was not patient-centered! •  These methods show promise for understanding and evaluating patient-

centered perioperative care processes.

Patient-Reported Perioperative Non-Routine Events Matthew B. Weinger, MD*; Jason M. Slagle, PhD; Amanda Lorinc, MD; Gina Whitney, MD; Eric Porterfield, MS;

Krys Dworski, Eva Cassidy, and the PNRE Project Team (Funded by the Patient-Centered Outcomes Research Institute)

Table 1. Key Themes from Focus Groups and their Occurrence in Cases and NREs*

•  Did anything happen that you did not expect or want to happen?

•  Did you get the information you needed to know at the right time?

•  Did anything happen that caused you stress, worry, or concern?

Figure 2. P-CONES Probes for Ambulatory Surgery

•  I got lost in the hospital trying to find…

•  The lights and/or noise disrupted my sleep.

•  [Clinician] was condescending/ rude/didn’t listen.

Figure 3. Sample Patient NREs

Key Non-Routine Event Theme % of Cases (n = 140)

% of NREs (n = 214)

Diagnostic and Therapeutic Issues (unfamiliarity with the patient’s condition, mistakes and errors, diagnostic delays or misdiagnoses, etc.) 47.1 46.3

Health Care Process Deficiencies (unexpected care, failure to get access, delays in treatment, care disruptions or variability, etc.) 44.3 43.9

Communication of Health Information (getting the wrong amount of information, or wrong content, or mistimed delivery, etc.) 32.1 31.3

Environment of Care (available food choices, incorrect diet, cleanliness, etc.) 17.9 14.0 Staffing Issues (too few nurses, adequately trained providers unavailable, etc.) 18.6 13.1 Patient-Provider Relationship (dismissal of patient concerns, not talking with or listening to patients, being rude or inflexible, etc.) 15.0 10.7

No Relevant Focus Group Theme 5.6 8.4 * Both patient- and clinician-reported NREs included.

Methods

Study Aims

Introduction Figures & Tables Results

Conclusions

References & Acknowledgements

•  I was “dumped on the sidewalk” before I was ready to go home.

•  An allergy band was never placed on me.

•  I couldn’t breathe when I woke up from surgery.

•  My inpatient diabetes management was “not very good.”

•  Did anything happen to you that was frustrating, inconvenient, or distracting?

•  Did the people taking care of you know what you needed?

•  Were you worse off afterwards than you expected to be?

•  Did you understand what was supposed to happen to you?

•  Was everything you needed made available to you?

Key themes for patient-reported NREs in surgical setting.

Datacollectedincludednumberoftaskssuccessfullycompletedwithoutassistance;timetocompletethetasks;numberandtypesoferrors;devia-tionsfromidealpathways;andparticipants’satisfactionofthesystems.Conclusions: Tasktimeswerehighlyvariable(upto5min.tocomplete);andmosttaskswerecompletedwithmorethan75%successrates(butwithbetween0–75%oferrorsgoingundetected).Mosterrorsandinefficienciesweretheresultofpoorlabeling,confusingnavigation,poorlayoutandlackofstandardization.Recommendationsweremadetoreduceerrors,andimproveefficiencyandsatisfaction(reducefrustrationandconfusion).

Sample pre-op workspace screen. (in development)

Unsafe act due to poor ergonomics.

CELA control room looking onto test bays.

Simulation testing of scanning device.

Sample pages of guide.Online evaluation form.

Example of complex neonatal intensive care unit setting.

Clopidogrel advisor for intermediate metabolizers.

Paper form used to support collaborative communications.

How everyday practice can deviate from best practice.

Blood ordering poster.

Medication reconciliation success rates.