theroleofhigh-fidelityteam-basedsimulation in acute care ... · a comprehensive analysis that...

The Role of High-Fidelity Team-Based Simulationin Acute Care Settings: A Systematic ReviewSarah Armenia, MS1 Loka Thangamathesvaran, BS1 Akia D. Caine, MD1 Neil King, MD1

Anastasia Kunac, MD, FACS2 Aziz M. Merchant, MD, FACS1

1Division of General Surgery, Department of Surgery, RutgersUniversity, New Jersey Medical School, Newark, New Jersey

2Division of Trauma and Surgical Critical Care, Department ofSurgery, Rutgers University, New Jersey Medical School,Newark, New Jersey

Surg J 2018;4:e136–e151.

Address for correspondence Aziz M. Merchant, MD, FACS, Division ofGeneral Surgery , Department of Surgery, New Jersey Medical School,Rutgers Biomedical and Health Sciences, 185 South Orange Avenue,MSB G530, Newark, NJ 07103 (e-mail: [email protected]).

Teamwork and communication have been identified ascritical components of safe healthcare systems.1 Previousstudies across several industries have recognized simulationas an effective way of improving these skills, particularly inthe acute care setting where ad hoc teams form rapidly andrequire efficient collaboration.2,3 The increasing complexity

of simulation has enabled the assessment and developmentof technical and nontechnical skills in a diverse spectrum ofacute care settings.4–7 High-fidelity simulation specificallyinvolves the use of a computerized full-bodymannequin thatcan give dynamic, physiologic feedback and can be pro-grammed to provide realistic responses.8 This technology

Keywords

► teaching► simulation training► team training► high fidelity

simulation training► education► simulation-based

medical education

Abstract Introduction High-fidelity team-based simulation has been identified as an effectiveway of teaching and evaluating both technical and nontechnical skills. Several studieshave described the benefits of thismodality in a variety of acute care settings, but a lackof standardized methodologies has resulted in heterogeneous findings. Few studieshave characterized high fidelity simulation across a broad range of acute care settingsand integrated the latest evidence on its educational and patient impact.Methods The MEDLINE, EMBASE, Cochrane Library, and PsycINFO databases weresearched for empirical studies from the last 10 years, investigating high fidelity team-based simulation in surgical, trauma, and critical care training curricula.Results Seventeen studies were included. Interventions and evaluations were com-prehensively characterized for each study and were discussed in the context of fouroverarching acute care settings: the emergency department/trauma bay, the operat-ing room, the intensive care unit, and inpatient ad hoc resuscitation teams.Conclusions The use of high-fidelity team-based simulation has expanded in acute careand is feasible and effective in a wide variety of specialized acute settings, including theemergency department/trauma bay, the operating room, the intensive care unit, andinpatient ad hoc resuscitation teams. Training programs have evolved to emphasize team-based, multidisciplinary education models and are often conducted in situ to maximizeauthenticity. In situ simulations have also provided the opportunity for system-levelimprovement and discussions of complex topics such as social hierarchy. There is limitedevidence supporting the impact of simulation on patient outcomes, sustainability ofsimulation efforts, or cost-effectiveness of training programs. These areas warrant furtherresearchnow that the scopeofutilizationacross acutecare settingshasbeencharacterized.

receivedJanuary 9, 2018accepted after revisionJune 29, 2018

DOI https://doi.org/10.1055/s-0038-1667315.ISSN 2378-5128.

Copyright © 2018 by Thieme MedicalPublishers, Inc., 333 Seventh Avenue,New York, NY 10001, USA.Tel: +1(212) 584-4662.

Review ArticleTHIEME

e136

mailto:[email protected]

https://doi.org/10.1055/s-0038-1667315

https://doi.org/10.1055/s-0038-1667315

has facilitated several acute care team-based training pro-grams and subsequently a growing body of research on theireffectiveness. However, without standardized interventionand evaluation methodologies, the heterogeneity of thesestudies necessitates systematic analysis.

Previous reviews of the literature on simulation in acutecare settings have focused on specific learning objectives,participant populations, or clinical environments withinacute care. A review by Boling and Hardin-Pierce integratedresearch specifically on knowledge and confidence followinghigh-fidelity simulation in critical care training.5 In anotherreview, Tan et al analyzedmultidisciplinary team simulationspecifically in the operating room.9 Warren et al reviewedthe effectiveness of simulation on satisfaction and learningoutcomes in nurse practitioner programs.10 Other reviewshave separated their analysis by technical versus nontechni-cal skills. Gjerra et al and Lewis et al reviewed the impact ofteam-based simulation on nontechnical skills specifically.3,4

A comprehensive analysis that appropriately reflects thebreadth of participant populations, types of skills assessed,and scope of acute care settings is therefore necessary.

The purpose of this review was to synthesize the bestavailable evidence on the utilization of high-fidelity team-based simulation in a broad scope of acute care settings. Thegoal was to explore the full scope of application of thismodality to surgical, trauma, and critical care training cur-ricula, to compare intervention and evaluation characteris-tics by acute care setting, and to integrate existing evidencefrom the last 10 years on actual patient outcomes. Theresearch questions were as follows:

(1) What is the scope of acute care settings in which high-fidelity team-based simulation is being utilized, and howdothecharacteristicsof thesesimulationsdifferbysetting?

(2) How does in situ versus off site simulation study designcompare in acute care team-based simulation training?

(3) How doesmultidisciplinary team design impact the effec-tiveness of acute care team-based simulation training?

(4) What translational progress has been made over the last10 years in evaluating the impact of acute care simula-tion training on actual patient outcomes?

Methods

Search StrategyAn initial search ofMEDLINEwas conducted to identify indexterms and keywords pertaining to “team-based simulation.”An extensive second search using all identified index termsand keywords was then performed in the following data-bases: MEDLINE, EMBASE, Cochrane Library, and PsycINFO.Keywords included “simulation,” “surgical procedures,operative,” “general surgery,” “trauma,” and “critical care.”Studies were limited to those in the English language withfull text available. Due to the innovative and technology-driven nature of the subject, searches were limited to a 10-year period, including studies from January 1, 2008 toMarch 11, 2018 (date the search was performed). Finally,reference lists of all articles included thus far were searchedfor additional relevant citations. These articles were then

imported to a reference management system for full-textreview using pre-established inclusion and exclusion criteria(►Table 1). Since the objective of the search was to synthe-size the available evidence regarding team-based simulationas a primary intervention, only empirical studies wereincluded, and other types of studies, such as literaturereviews or editorials, were excluded. A flow diagram illus-trating the article selection process is included (►Fig. 1).

Assessment of QualityThe articles were then assessed by two independentreviewers, the first author (Sarah Armenia) and secondauthor (Loka Thangamathesvaran), using the Critical Apprai-sal Skills Program (CASP) to standardize the assessmentprocess.11 CASP is a 10-question checklist used to evaluateresearch studies and offers a systematic way to criticallyevaluate methodology across independent reviewers.Screening reliability between the two reviewers wasassessed using Cohen’s kappa at the abstract and full-textlevels.12 Any discrepancies were resolved by discussion withother co-authors, and the final consensus was confirmed bythe senior author (Aziz M. Merchant).

Data SynthesisAfter thematic analysis, four distinct clinical environmentsubtypes were identified under the umbrella of acute care:emergency department/trauma bay, operating room, inten-sive care unit, and inpatient ad hoc resuscitation/code teams.Information for setting subtype categorization was found ineither the objective, description of the study setting andparticipants, or the methodology for studies with in situinterventions. In studies that were conducted offsite, settingsubtype information was found in the objective or themethodology (which included descriptions of the specificenvironment that the intervention was attempting to simu-late). Two studies with in situ interventions did not simulatea fixed clinical environment for simulations and were there-fore assigned to a separate category. These studies designedunannounced simulations that were triggered at variouslocations throughout the medical center resulting in the

Table 1 Inclusion and exclusion criteria

Inclusion criteria Exclusion criteria

& Peer-reviewed papers& Published from 2008 to2018

& Published in any country& Published in English& Empirical studies thatinvestigate technical andnon-technical skills viateam-based simulationtraining

& Acute care setting(emergency department/trauma bay; operatingroom; intensive care units;and ad hoc resuscitation/code teams).

& Incomplete reports (onlyabstract available; confer-ence proceedings)

& Review articles& Studies where instrumentdesign and/or validationwas the primary endpoint.

The Surgery Journal Vol. 4 No. 3/2018

Team-Based Simulation in Acute Care Armenia et al. e137

formation of ad hoc teams sent to various inpatient settings.Studies categorized into these four acute care setting sub-types were then further characterized by distinguishingfeatures of their simulated patient populations (i.e., pediatrictrauma patients) or their study participants (i.e., militarypersonnel deployed in Iraq) as seen in ►Fig. 2.

For each acute care setting, characteristics of the inter-vention and evaluation were tabulated systematically. Eachintervention was characterized by using the simulationtechnology, whether it was conducted in situ or off site,and the scope of clinical scenario(s) simulated (►Table 2).Each evaluation was characterized by the type of skill

Fig. 1 Flow diagram of article selection process.

Fig. 2 The spectrum of acute care settings where high-fidelity simulation is feasible for team-based training and further categorization of studypopulations and/or clinical contexts. Abbreviations: SICU, surgical intensive care unit; PICU, pediatric intensive care unit; NICU, neonatalintensive care unit; PCICU, pediatric cardiac intensive care unit; ICU intensive care unit.


Team-Based Simulation in Acute Care Armenia et al.e138

Table

2Cha

racteristics

oftheinterven

tion

s:simulationtech

nology,

contex

t,an

dscen

ario(s)

Acu

teca

resetting

Source

sSimulationtech

nology

Insitu

Description

ofclinical

scen

ario(s)

Emergen

cyde

partm

ent/

trau

maba

y

Briggs

etal.,20

17High-fide

litytrau

masimulation(STR

ATUS

Cen

terforSu

rgical

Simulation)

Blun

ttrau

mafrom

amotor

vehicleaccide

nt;multiple

pene

tratinginjuries

from

abroke

n-plate

glasswindow

Cap

ella

etal15

High-fide

litytrau

masimulation(CarilionClin

icCen

ter

forExpe

rien

tial

Learning

)Uns

table

patien

taftermotorvehicleacciden

twith

pene

tratinginjuries;s

imila

rscen

arios(referredto

but

notprov

ided

)

Falcon

eet

al16

SimBa

by™,P

ediaSIM,a

ndSimMan

®(M

edical

Educ

a-tion

alTe

chno

logies

Inco

rporated

,Sa

rasota,Fla)

Infant

withhe

adinjury;C

hild

withape

netratingwou

ndto

theba

ck;ad

olesce

ntwithmultitrau

mainclud

ingan

unstab

lepe

lvic

frac

ture

Miller

etal17

SimMan

®3G

(Lae

rdal,Stavan

ger,Norw

ay)

♦Pa

tien

twithblun

tab

dominal

trau

main

obviou

ssh

ock

withan

intact

airw

ayan

daFA

STex

aminationpo

sitive

for

intrap

eriton

ealfreefluid;p

atient

withpe

netratingch

est

injury

who

arrive

dwitho

utintrav

enous

acce

ssan

drequ

ired

adva

nced

airw

ayman

agem

ent,tube

thoracos

tomy,

andpe

ricardioce

ntesisforstab

ilization

Patterso

net

al18

High-fide

litysimulator

(not

spec

ified

)♦

Trau

maan

dmed

ical

simulations

basedon

high-risk

clinical

cases(not

spec

ified

)

Steine

man

net

al19

SimMan

®3G

(Lae

rdal,Stavan

ger,Norw

ay)

♦Th

reeprep

rogram

med

,β-tested

,15-min

blun

ttrau

-matic

shoc

kscen

arios(not

spec

ified

)

Ope

rating

room

Ace

roet

al20

Simulated

OReq

uipp

edwithaSimMan

®3G

(Lae

rdal,

Stavan

ger,Norw

ay)

Preg

nant

simulated

patien

tin

hemorrhag

icshoc

k,blee

ding

from

aca

rotidinjury,u

ltim

atelylead

ingto

cardiacarrest

Hoa

nget

al21

Hum

an-W

orn,

Partial-T

ask,

Surgical

Simulator,also

knownas

the“C

utSu

it”(Strateg

icOpe

ration

sInc.,Sa

nDiego

,CA);Cad

avers

♦Simulated

trau

mainvo

lvingon

eor

twocasualties

per

scen

ario

everyda

y,withincrea

sing

complex

ity,ba

sedon

conc

epts

taug

htea

rlierthat

day;

fina

l6-h

masscasu

alty

even

ton

thelast

day

Huser

etal22

MET

IiST

AN

(CAEHea

lthc

are,

Sarasota,FL)mod

ified

toho

ldfive

troc

ars

♦Ve

ntricu

larfibrillationin

asimulated

patien

tdo

cked

toa

robo

t

Kellicu

tet

al23

High-fide

litysimulator

(not

spec

ified

);med

ical

mou

lage

applicationto

simulatecasu

alties

♦Tw

enty

trau

mascen

arioswerecreatedfrom

theBa

ghda

dCo

mba

tSu

pportHospitaltraum

apa

tient

databa

se

Intens

ivecare

units(adu

lt,

pediatric;cardiac,

andsu

rgical)

Figu

eroa

etal24

New

born

HAL®

,Ped

iatric

HAL®

(Gau

mard,Miami,FL);

SimMan

®(Lae

rdal,Stav

ange

r,Norway)

Postop

erativeNorwoo

dpa

tien

twithaccide

ntal

extuba

-tion

;Glenn

patien

twithpo

stop

erativehe

morrhag

icstroke

;Fon

tanpa

tien

twithlowcardiacou

tput;B

Tsh

unt

withacutethrombo

sis;

postope

rative

Ebstein’srepa

irwithun

stab

leSV

T;po

stop

erativeTO

Fpa

tien

twithlow

cardiacou

tput

synd

rome;

andjetve

ntila

tion

lead

ingto

ECMO

Gun

drosenet

al25

SimMan

®2G

(Lae

rdal,Stavan

ger,Norw

ay)

♦Se

ptic

shock

Pascua

letal26

(Con

tinue

d)



Table

2(Con

tinue

d)

Acu

teca

resetting

Source

sSimulationtech

nology

Insitu

Description

ofclinical

scen

ario(s)

SimMan

®(m

alean

dfemale)

withtech

nicalm

anipula-

tion

usingSimMan

Softwareve

rsion3.3.1(Lae

rdal,

Stav

ange

r,Norway)

Ana

phy

laxiswithtens

ionpn

eumotho

rax;

Septic

shoc

kfrom

Clostridium

difficile

colitis;Myo

cardialinfarction

withdiab

etic

ketoacidos

is;he

morrhag

icshock

with

abdo

minal

compa

rtmen

tsynd

rome;

deteriorating

trau

matic

braininjury

withstatus

epile

pticus

Reed

etal27

Baby

Ann

e®man

ikin

used

forlow-fide

litysimulation

(Lae

rdal,Stavan

ger,Norw

ay);Prem

ie,New

born

and

PediatricHAL®

simulatorsus

edforhigh

-fide

litysimula-

tion

(Gau

mard,Miami,FL)

♦Hom

ebirth(esopha

geal

intuba

tion

,hy

pothermia,

hypo

glyce

mia);resp

iratoryfailu

re(sep

sis/pne

umon

ia);

cardiogen

icsh

ock;

ECPR

(cardiac,

gene

ralsurgery);p

ost

sterno

tomych

estco

mpression

s;trache

ostomy

dislod

gmen

t;ve

ntricu

larfibrillation;

suprave

ntricu

lar

tach

ycardia;

acutepu

lmon

aryhe

morrhag

e;tens

ion

pneu

motho

rax;

pneu

motho

raxpo

stsurfac

tant;myelo-

men

ingoc

eleself-ex

tuba

tion;

Prem

ieself-ex

tubation;

fentan

yl-in

duce

drigidch

est;an

dap

nea/brad

ycardia/

desaturation

even

t

Stoc

keret

al28

SimBa

by™

(Lae

rdal,Stav

ange

r,Norway)

♦Re

spiratoryprob

lems(respiratory

arrest,blocked

endo

trache

altube

,pne

umotho

rax,

andseve

reasthma

exacerbation);cardiacproblem

s(cardiacarrest,pe

ri-

cardiale

ffus

ion,

thrombo

sedarterio-pulmon

arysh

unt,

post-operativelow

cardiacou

tput

state,

andpo

st-sur-

gicalc

ardiactampo

nade

);an

dge

neralP

ICUproblem

s(hyp

erka

lemic

rhythm

disturba

ncean

dsuprave

ntricu

lar

tach

ycardia)

Inpatient

adho

cresuscitation

team

s/co

deteam

s

And

reatta

etal29

MET

IPed

iaSIM

(CAEHea

lthc

are,Sa

rasota,FL);Sim

Baby

™(Lae

rdal,Stavan

ger,Norw

ay)

♦Se

psis(immun

osup

pressedan

dno

rmal

patien

ts);

respiratorydistress

(bronc

hiolitis,p

neum

onia);

increa

sedintracranial

pressure/herniation(intracran

ial

mass,

intracranial

trau

ma,

andmen

ingitis,

seizures);

anap

hylactic

shock;an

dca

rdiogen

icsh

ock(con

gestive

heartfailu

re,c

onge

nitalh

eart

disease,

andmyo

carditis)

Barbeito

etal30

SimMan

®3G

(Lae

rdal,Stav

ange

r,Norway)

♦Cardiacarrest

Abbrev

iations

:BT,

Blaloc

k–Th

omas–T

aussig;E

CMO,e

xtraco

rporea

lmem

brane

oxyg

enation;

ECPR

,extraco

rporea

lcardiop

ulmon

aryresu

scitation;

FAST

,foc

usedassessmen

twithsono

graph

yfortrau

ma;

PICU,

pediatricintensivecare

unit;ST

RAT

US,

simulation,

training

,research

andtech

nology

utilization

system

;SV

T,supraven

tricular

tach

ycardia;

TOF,tetralog

yof

fallo

t.



assessed (technical, nontechnical, or both), the Kirkpatrick’slevel(s) of evaluation (►Table 3), the simulation scoringinstrument, and the debriefing process (►Table 4). TheKirkpatrick model of evaluation has been used previouslyto assess evidence in educational research and provides asystematic way of categorizing learning outcomes.13 In thismodel, Kirkpatrick Level 1 evaluates participant satisfaction;Level 2 evaluates knowledge acquisition; Level 3 evaluatesparticipant behavior change; and Level 4 evaluates improvedpatient outcomes. It is possible for an evaluation to covermultiple Kirkpatrick’s levels of evaluation as seenin ►Table 3.

Results

Overview of the Included StudiesSeventeen studies met the inclusion criteria. The studiesoriginated from four countries: the United States (n ¼ 14),Norway (n ¼ 1), England (n ¼ 1), and Germany (n ¼ 1). Thetypes of journals covered a broad spectrum of disciplines:surgical education (n ¼ 5), surgery (n ¼ 2), endourology(n ¼ 1), pediatric cardiology (n ¼ 1), intensive and critical

care nursing (n ¼ 1), intensive and critical care medicine(n ¼ 1), perinatology (n ¼ 1), pediatric critical care (n ¼ 1),simulation in healthcare (n ¼ 1), emergency medicine(n ¼ 1), trauma (n ¼ 1), andqualityandsafety (n ¼ 1). Journalquality was assessed using the SCImago Journal Rank (SJRIndicator), which is based on the number of citations receivedbya journal andthequalityof thejournals thosecitations camefrom. The studieswere categorized into four acute care settingsubtypes: emergencydepartments/traumabays (n ¼ 6), oper-ating rooms (n ¼ 4), intensive care units (n ¼ 5), and inpati-ent ad hoc resuscitation teams (n ¼ 2). Six studies assessedonly technical skills, 1 study assessed only nontechnical skills,and 10 studies assessed both. Five of the studies usedvalidatedinstruments for these assessments. Eleven studies implemen-ted their simulations in situ, and six studies conducted thesimulations in offsite simulation centers. Fifteen studies hadmultidisciplinary participants; one study consisted of onlynurses; and one study consisted of only advanced practi-tioners. Evaluation was done at several Kirkpatrick levels—the effect on learning (Level 2) was most frequently evaluated(13 of 17 studies) followed by the effect on reaction (Level 1),consisting of 9 of 17 studies.

Table 3 Characteristics of the evaluation of effect of each simulation intervention by type(s) of skills evaluated and the fourKirkpatrick levels of evaluationa for each acute care setting

Acute caresetting

Sources Technicalskills

Nontechnicalskills

Both Kirkpatrick’s levels of evaluationb

Reaction Learning Behavior Outcomes

Emergencydepartment/trauma bay

Briggs et al.,2017

♦ ♦

Capella et al15 ♦ ♦ ♦

Falcone et al16 ♦ ♦ ♦

Miller et al17 ♦ ♦ ♦

Pattersonet al18

♦ ♦

Steinemannet al19

♦ ♦ ♦ ♦

Operating room Acero et al20 ♦ ♦ ♦

Hoang et al21 ♦ ♦

Huser et al22 ♦ ♦

Kellicut et al23 ♦ ♦ ♦

Intensive careunits (adult,pediatric; cardiac,surgical)

Figueroaet al24

♦ ♦

Gundrosenet al25

♦ ♦

Pascual et al26 ♦ ♦ ♦

Reed et al27 ♦ ♦ ♦ ♦

Stocker et al28 ♦ ♦ ♦

Inpatient ad hocresuscitationteams/code teams

Andreattaet al29

♦ ♦ ♦ ♦ ♦

Barbeito et al30 ♦ ♦ ♦

aAdapted from Kirkpatrick.13bLevel 1: Reaction (participant satisfaction), Level 2: learning (knowledge, skills and attitudes), Level 3: behavior (translation of learning to clinicalsetting), and Level 4: outcome (patient outcomes).



Table 4 Characteristics of the instrument used to score simulation interventions (including whether it is validateda) and thedebriefing process following the simulation

Acute care setting Sources Simulation scoring instrument Debriefing process

Technical skills Nontechnical skills

Emergency department/Trauma bay

Briggs et al14 Clinical checklist; times tospecific task completion

NOTSSa; T-NOTECHSa None (retrospective studydesign)

Capella et al15 N/A; Resuscitations pre-and post-training werescored, not simulations

N/A; resuscitations pre- andpost-training were scored,not simulations

Videotapes (of resuscita-tions pre- and post-training)reviewed immediately aftersimulation

Falcone et al16 Instrument developed byHolcomb et al., 2001

Not assessed Videotapes reviewedimmediately aftersimulation

Miller et al17 Not assessed CTSa Immediately after simula-tion; Focused on teamwork

Patterson et al18 Not scored; concepts dis-cussed in debriefing

Modified ANTSa Immediately after simula-tion; focused on teamworkand system-level safetythreats

Steinemann et al19 Clinical process parameterschecklist

T-NOTECHSa Videotapes reviewedimmediately after simula-tion; focused on teamwork

Operating room Acero et al20 Number of mitigation stepscompleted; indirectlyassessed through question-naire (testing clinicalknowledge)

Not assessed Videotapes reviewedimmediately after both“cold” and “warm”simulations

Hoang et al21 Disposition time and criticalerrors made (assessed atthree time points forcomparison)

Not assessed None

Huser et al22 Times to specific taskcompletion

Not assessed Same day as simulation;Focused on teamwork andsystem-level safety threats

Kellicut et al23 Prehospital, triage, andresuscitation evaluationchecklists

Component of triage andresuscitation evaluationchecklists

Videotapes reviewedimmediately after simula-tion; Focused on teamwork

Intensive care units (adult,pediatric; cardiac, andsurgical)

Figueroa et al24 Clinical process parameterschecklist

Principles of Team STEPPSassessed

Immediately aftersimulation

Gundrosen et al25 Clinical checklist; times tospecific task completion

ANTSa Videotapes reviewedimmediately aftersimulation

Pascual et al26 ECCS; indirectly assessedthrough written examina-tion pre- and post-course

TLIS Videotapes reviewedimmediately aftersimulation

Reed et al27 Clinical checklist Not assessed Immediately after simula-tion; Individual, equipmentand system-level issues

Stocker et al28 Not scored; Assessedthrough self-evaluation

Not scored; assessedthrough self-evaluation

Based on the Children’sHospital Boston SimulationProgram teaching princi-ples of crisis resourcemanagement

Inpatient ad hoc resuscita-tion teams/code teams

Andreatta et al29 Not scored; assessedthrough self-evaluation andindirectly through survivalrates longitudinally

Not scored; assessedthrough self-evaluation

Videotapes reviewedimmediately aftersimulation

Barbeito et al30 Not scored; Conceptsdiscussed in debriefing

Not scored; concepts dis-cussed in debriefing

Videotapes reviewedimmediately after simula-tion; focused on teamworkand system-level threats

Abbreviations: ANTS, Anesthetists’ Non-Technical Skills; CTS, clinical teamwork scale; ECCS, emergency clinical care skills; NOTSS, non-technicalskills for surgeons; Team STEPPS, Team Strategies and Tools to Enhance Performance; TLIS, Team Leadership-Interpersonal Skills; T-NOTECHS,modified non-technical skills scale for trauma.aIndicates the instrument has been validated.



Characteristics of Emergency Department/Trauma BaySimulationsSix studies developed training programs that simulated acrisis within the emergency department or trauma baysetting.14–19 One study simulated emergent care in thepediatric population;18 one study simulated trauma in thepediatric population;16 and the remaining four studiessimulated trauma in adult populations.14,15,17,19 Studydesign was heterogeneous with a wide variety of outcomemeasures. One study used simulation as a means of identify-ing latent safety threats and made changes at the systemlevel.18 One study assessed sustainability, observing that thescored behaviors returned to baseline after simulationsstopped.17 Most studies evaluated simulations, but Milleret al, Steinemann et al, and Capella et al observed actualtrauma resuscitations as part of the study design.15,17,19 Onestudy assessed technical skills only;16 another assessednontechnical skills only,17 and the remaining four studiesassessed both skillsets.14,15,18,19 Evaluation was done atseveral Kirkpatrick levels—notably, two of the four studiesincluded in this review that evaluated the effect on outcomes(Level 4) were from this acute care setting subgroup.15,19

Characteristics of Operating Room SimulationsFour studies developed training programs that simulated acrisis requiring operative care as part of the interven-tion.20–23 One study simulated an operative emergencyoccurring during active deployment in Iraq.23 One studysimulated an operative emergency on a Navy Shipboard as apart of pre-deployment training.21 The remaining twostudies developed interventions that simulated crisesoccurring during ongoing operations.20,22 One study simu-lated an intraoperative code during general surgery,20 andanother study simulated an intraoperative code duringrobotic surgery, while the robot was actively docked.22 Asummary of simulations in this setting is providedin ►Table 5. Three studies included a didactic and simula-tion component, while one study included only a simulationcomponent.20,21,23 One of the four studies evaluated thesustainability of the training after 5 months.21 The operat-ing room subgroup had the largest proportion of studiesthat assessed technical skills only (three of four stu-dies).20–22 The remaining study assessed both skillsets.23

This emphasis on technical skills was reflected in theevaluations—all four studies used clinical checklists andtimes to task completion. Nontechnical skills were evalu-ated as part of a larger checklist in one study.23 Evaluationwas done at several Kirkpatrick levels —all four studiesevaluated the effect of simulation on the learning level(Level 2), and two studies also evaluated the effect on thereaction level (Level 1).20,23 A summary of evaluationcharacteristics can be found in ►Tables 3 and 4. One studyaddressed system-level issues—the simulation of resuscita-tion during robotic surgery prompted the formation of aflow diagram by a multidisciplinary team after the firstsimulation.22 This flow diagram contributed to better out-comes in the second simulation.

Characteristics of Intensive Care Unit SimulationsFive studies developed training programs that simulated acrisis within intensive care units of various subspecial-ties.24–28 The studies simulated a general intensive careunit,25 surgical intensive care unit,26 pediatric intensivecare unit,28 neonatal intensive care unit,27 and a pediatriccardiac intensive care unit,24 respectively. A summary ofsimulations in this setting is provided in ►Table 5. Studydesign was heterogeneous—one study used pre- and post-intervention evaluations,26 and one study use a randomi-zation process to compare a didactic versus simulation-based curriculum.25 Four studies assessed both technicaland nontechnical skills,24–26,28 and one study assessed onlytechnical skills.27 The evaluation tools for these parametersvaried—several studies used a clinical process checklist fortechnical skills,24,25,27 whereas one study used a previouslyvalidated score sheet developed by the National Registry ofEmergency Medical Technicians.26 Nontechnical skills werealso evaluated in a variety of ways—one study assessedthese skills through participant self-evaluation;28 onestudy used Anesthetists’ Non-Technical Skills (ANTS);25

one study used Team Leadership-Interpersonal Skills(TLIS);26 and Figueroa et al assessed principles of TeamStrategies and Tools to Enhance Performance (TeamSTEPPs).24 Evaluation was done at several Kirkpatricklevels. All five studies evaluated the effect of simulationon the learning level (Level 2); three studies also evaluatedon the reaction level (Level 1);26–28 and one study alsoevaluated on the behavior level (Level 3).27 Reed et alevaluated changes in behavior qualitatively and indirectly,associating a decrease in the number of full codes followingsimulation with an increased aptitude for managingdecompensating patients earlier in the process.27 Onestudy addressed system-level issues and conducted corre-sponding quality improvement efforts throughout thesimulation period.27 A summary of evaluation character-istics can be found in ►Tables 3 and 4.

Characteristics of Inpatient Ad Hoc ResuscitationTeam/Code Team SimulationsTwo studies developed training programs to simulate crisesthat would trigger the response of hospital-wide ad hocresuscitation teams/code teams to an inpatient unit.29,30

One study simulated a pediatric cardiopulmonary arrest(CPA),29 and the other study simulated an adult CPA.30 Asummary of simulations in this setting is providedin ►Table 5. One study evaluated only technical skills,29

while the other study evaluated both.30 The ad hoc resus-citation team subgroup had the highest proportion of highKirkpatrick levels of evaluation, with both studies evaluat-ing the effect of simulation on the behavior level (Level 3)and outcomes level (Level 4). In addition, the study byAndreatta et al was the only study included in this reviewthat evaluated on all four Kirkpatrick levels.29 Only four ofseven studies in this review evaluated on the outcomes level(Level 4).15,19,29,30 A summary of evaluation characteristicscan be found in ►Tables 3 and 4.



Table

5Su

mmaryof

includ

edstud

ies

Stud

yStudydesign

Participan

tsObjective

Interven

tion

Outcomemea

sures

Resu

lts

SJR

indicatora

Ace

roet

al20

Pre-/pos

t-interven

tion

stud

ySing

lesite

171ORstaffmem

bers

(surgeryreside

nts,

anesthesia

reside

nts,

periop

erativenu

rses)

Evalua

tewhe

ther

ahigh

-fide

lityman

ne-

quin

improv

esteam

performan

cein

ahigh

-risk

surgical

emergen

cy

Exsang

uina

tion

sce-

narious

inghigh

-fide

-lityman

nequ

in

Team

performan

ceof

eigh

tmitigationstep

sat

baselin

e(“co

ld”)

vsde

briefi

ngan

ddida

ctic

session(“warm”)

Team

training

using

high

-fide

litysimula-

tion

iseffectivein

training

ORstaffin

ahigh

-risksu

rgical

emergen

cy

0.98

3

And

reatta

etal.,

2016

Long

itud

inal,m

ixed

-metho

ds

Sing

lesite

252reside

nten

coun

-ters

(som

eredu

ndan

cy)

Evalua

teviab

ility

and

effectiven

essof

asimulation-ba

sed

pediatricmoc

kco

deprogram

onpa

tien

tou

tcom

es;ev

alua

tereside

nts’

confi

denc

ein

performing

resu

scitations

Moc

kpe

diatric

code

sat

increa

sing

ratesov

era48

-mon

thpe

riod

Self-assessmen

tda

ta;

hospital

reco

rdsfor

pediatricCPA

survival

ratesthroug

hout

stud

ydu

ration

Survival

rates

increa

sedto

approx

.50

%co

rrelatingwith

theincrea

sednu

mbe

rof

moc

kco

desan

dremaine

dstab

lefor

3years

1.35

9

Barbeito

etal30

Post-in

terven

tion

stud

ySing

lesite

>30

0(87ph

ysicians,

100nu

rses,2

1resp

iratorytherap

ists,

10ad

ministrativestaff,

remaind

erun

spec

ified

)

Iden

tify

oppo

rtun

ities

forsystem

optimiza-

tion

usingan

insitu

simulation-ba

sedqu

al-

ityim

prov

emen

tprogram

Simulated

unan

-no

unce

dcardiacarrest

sessions

Tech

nicala

spec

tsof

session;

structural

and

system

sba

sedha

zards

andde

fects

Insitu

simulationcan

iden

tify

andmitigate

latent

hazardsan

dde

fectsin

theho

spital

emergen

cyrespon

sesystem

0.56

7

Briggs

etal14

Retrospec

tive

coho

rtstud

y20

team

s(surgical

and

emergen

cyroom

resi-

dents;

emergen

cyde

partmen

tnu

rses;

emergen

cyservices

assistan

ts)

Evalua

tetheeffectsof

team

lead

ers’

non-

tech

nica

lskills

ontech

nica

lperform

ance

ofclinical

tasksus

ing

simulated

scen

arios

Twosepa

rate

high-

fide

lity,

simulated

trau

mascen

arios

Non

tech

nicalskills

(suc

has

commun

ica-

tion

,lead

ersh

ipan

dteam

work)

usingthe

Mod

ified

Non

tech

nical

SkillsScaleforTrau

ma

system

Non

tech

nicalskills

oftrau

mateam

san

dtrau

malead

ersde

te-

riorateas

clinical

sce-

narios

progress

0.98

3

Cap

ella

etall15

Pre-/pos

t-interven

tion

stud

y28

surgeryreside

nts,6

facu

ltysurgeo

ns,80

emergen

cyde

part-

men

tnu

rses

Evalua

teifform

alteam

training

improv

esteam

beha

viorsin

trau

maresuscitation;

evalua

teifthis

improv

emen

tincrea

sesefficien

cyan

dim

prov

esclinical

outcom

es

Didac

ticsessions

;Mul-

tidisciplin

arysimula-

tion

sessions

Team

workdo

mainrat-

ings

(lea

dership,situa-

tion

mon

itoring,

mutua

lsup

port

and

commun

ication);time

tode

finitive

man

agem

ent

Structured

trau

ma

resuscitationteam

training

augm

entedby

simulationim

prov

esteam

performan

ce

0.98

3

Falcon

eet

al16

Long

itud

inal;p

re-/

post-in

terven

tion

160(ped

iatric

sur-

geon

s,em

ergen

cymed

icineph

ysicians

,

Evalua

tetheim

pact

ofmultidisciplin

arysimu-

lation

training

in

Mon

thly

high

-fide

lity

trau

masimulations

over

1-year

period

Scoringtool

assessing

numbe

rof

completed

tasksin

four

area

s:

Skillsrelatedto

airw

ayman

agem

ent,initial

trau

maassessmen

t,

1.02

6



Table

5(Con

tinue

d)

Stud

yStudydesign

Participan

tsObjec

tive

Interven

tion

Outco

memea

sures

Resu

lts

SJR

indicator

a

stud

ySing

lesite

surgery/pe

diatricresi-

dents,

nurses,critical

care

fello

ws,

parame-

dic,

respiratory

therap

ists)

pediatrictrau

mateam

performan

ceairw

ayman

agem

ent,

brea

thing,

circulation

anddisability

cervical

spineprec

au-

tion

san

dpe

lvic

frac-

ture

reco

gnitionan

dman

agem

ent

improv

edafterteam

training

Figu

eroa

etal.,

2012

Pre-/pos

t-interven

tion

stud

ySing

lesite

37(residen

ts,2

3nu

rses,5

resp

iratory

therap

ists)

Evalua

tewhe

ther

aprev

iously

valid

ated

team

worksystem

usingsimulation-based

team

training

(SBT

T)wou

ldhe

lpim

prov

epe

rcep

tion

ofteam

-work,

confi

denc

e,an

dco

mmun

icationdu

ring

post

pediatriccardiac

surgerycardiacarrest

Sixsimulated

post-

pediatric

cardiacsur-

gery

scen

arios(airway,

neurologican

dcardiac

emergen

cies)

Survey

spe

rformed

before,im

med

iately

after,an

d3months

afterpa

rticipation

SBTT

iseffectivein

improv

ingco

mmun

i-cation

andincrea

sing

confi

denc

eam

ong

mem

bersof

amulti-

disciplin

aryteam

dur-

ingcrisisscen

arios

0.78

7

Gun

dro-

senet

al25

Pre-/pos

t-interven

tion

stud

ySing

lesite

72nu

rses

Evalua

tetheus

eof

insitu

simulationto

exploreteam

compe

-tenc

eof

ICUnu

rses

Participan

tsrand

o-

mized

toeither

lec-

ture-based

orsimulation-ba

sed

teac

hing

ofseptic

shock

intheICU

“Tea

mworking

”an

d“situa

tion

awaren

ess”

evalua

tedby

two

blinde

draters

Insitu

simulationmay

befeasible

forasses-

sing

compe

tenc

ein

ICUs;

Nostatistica

llysign

ificant

differen

cebe

twee

nlearning

grou

ps

0.56

4

Hoa

nget

al21

Prospec

tive

observa-

tion

alstud

yUSNav

ymed

ical

per-

sonn

el(dep

loye

dph

y-sician

s,co

rpsm

en,

nurses,n

urse

anesthetists)

Evalua

tetheab

ility

ofa

simulation-ba

sed

training

course

topro-

duce

sustaine

dim

prov

emen

tin

team

-work,

commun

ication,

know

ledg

ean

dtrau

ma

man

agem

ent;

decrea

setimene

eded

toco

mpletetasks;

decrea

seerrors

Simulated

trau

ma

usingaHum

an-W

orn

Partial-T

askSu

rgical

Simulator

and

cada

vers

Timeto

dispos

ition

andcritical

errors

mad

edu

ring

simulation

Cou

rsede

mon

strated

sustaine

dim

prov

e-men

t;canim

prov

etrau

macare

prov

ided

byNav

ymed

ical

per-

sonn

elto

wou

nded

servicemem

bers

0.98

3

Hus

eret

al22

Post-in

terven

tion

stud

ySing

lesite

18(nurses,

anesthe-

siologists,u

rologists,

gyne

cologists)

Evalua

teac

uteem

er-

genc

yman

agem

entin

anORdu

ring

arobotic-

assisted

surgeryof

ahu

man

simulator

Simulated

emergen

cydu

ring

robo

tic-assisted

surgeryof

ahu

man

simulator

Timeto

startof

chest

compressions

,remov

alof

robo

tic

system

,first

Prob

lemsthat

aros

edu

ring

thefirstem

er-

genc

ysimulationwere

solved

andim

prov

e-men

tswereno

ted

1.08

9

(Con

tinue

d)



Table

5(Con

tinue

d)

Stud

yStud

ydesign

Participan

tsObjective

Interven

tion

Outcomemea

sures

Resu

lts

SJR

indicatora

defibrillationan

dsta-

bilizationof

circulation

during

repe

tition

ofsimulationafter

debriefi

ng

Kellicu

tet

al23

Post-in

terven

tion

stud

y22

0de

ploy

edpe

rson

-ne

l(ph

ysicians

,nurse

anesthetists,ph

ysician

assistan

ts,nu

rses,

med

ics,

ORtech

ni-

cian

s,othe

rmed

ical

supp

ortpe

rson

nel)

Evalua

teane

wed

uca-

tion

alan

dteam

-train-

ingprog

ram

ina

comba

tthea

teran

dassess

staffpe

rcep

tion

follo

wingtraining

Simulationtraining

mod

elspe

rformed

inthefield(Iraq)

Ano

nymou

ssurvey

sco

mpleted

post-

training

Surgical

Team

Assess-

men

tca

nbe

success-

fully

implem

entedin

anau

stere,

hostile

environm

entby

inco

r-po

rating

simulation

training

mod

elsan

dTe

amST

EPPs®

conc

epts

1.17

4

Miller

etal17

Pre-/pos

t-interven

tion

stud

ySing

lesite

39multidisciplinary

team

s(traum

asu

r-ge

onsan

dreside

nts;

EDph

ysicians

andresi-

dents;

EDnu

rses;

tech

nician

s;ph

arma-

cists;

clerks;respira-

tory

therap

ists

Evalua

tewhe

ther

anin

situ

trau

masimulation

program

couldbe

implem

entedan

dwhe

ther

thiswou

ldim

prov

eteam

work

andco

mmun

ication

Wee

klytrau

masimu-

lation

sfor8wee

ksClin

ical

Team

work

Scale(CTS

)was

used

toco

mpa

reprev

iously

observed

trau

maac

ti-

vation

sto

thos

eac

ti-

vation

sdu

ring

either

adida

ctic-onlype

riod

orsimulation-on

lype

riod

Improve

men

tswere

notice

din

allc

ompo

-ne

ntmea

suresdu

ring

thein

situ

simulation

interven

tionph

asebu

tthisob

served

bene

fit

declined

afterthe

simulationprogram

stop

ped

1.59

3

Pascua

let

al26

Pre-/pos

t-interven

tion

stud

ySing

lesite

12ad

vanc

edprac

ti-

tione

rs(APs)

Evalua

tewhe

ther

human

patien

tsimula-

tor-b

ased

training

isus

eful

inestablishe

dICUAPs

Five

scen

ariosus

inga

human

patien

tsimula-

tor(m

ixed

lead

eran

dob

server

roles)

Emergen

cycare

skills

(airway

-breathing

-cir-

culation

sequ

ence

;reco

gnitionof

shoc

k;pn

eumotho

rax,

etc.)

Hum

anpa

tien

tsimu-

latortraining

inestab-

lishe

dsu

rgical

ICUAPs

improv

eslead

ersh

ip,

team

work,

andself-

confi

denc

eskillsin

man

agingmed

ical

emergen

cies

N/A

Patterso

net

al18

Post-in

terven

tion

stud

ySing

lesite

218he

althca

reprov

iders

Iden

tify

latent

safety

threatsat

ahigh

errate

than

laboratory-based

training

;Re

inforce

team

worktraining

ina

pediatricED

90in

situ

simulations

ofcritical

patien

tsco

nduc

tedov

er1year

Obs

erve

dlatent

safety

even

ts(suc

has

mal-

func

tion

ingeq

uip-

men

tor

know

ledg

ega

ps);blinded

vide

oreview

usingamod

i-fied

Ane

sthe

tists’Non

-Te

chnicalSkills

scaleto

assess

team

beha

viors

Insitu

simulationisa

prac

ticalm

etho

dfor

detectionof

latent

safety

threatsan

dto

reinforcetraining

beha

viors

2.54



Table

5(Con

tinu

ed)

Stud

yStudydesign

Participan

tsObjective

Interven

tion

Outco

memea

sures

Resu

lts

SJR

indicatora

Reed

etal27

Post-in

terven

tion

stud

ySing

lesite

>50

0NICUstaff(neo

-na

tal/ca

rdiac/su

rgical

attend

ings

;neo

natal

fello

ws;ne

onatalnu

rse

prac

titione

rs;p

ediatric

residen

ts;nu

rses;

resp

iratorytherap

ists)

Evalua

teteam

-based

simulationtraining

intheNICUsetting

High-

andlow-fide

lity

simulationin

theNICU

(18case

scen

arios)

cond

uctedfora4-year

period

Qua

litativeiden

tifica-

tion

ofsystem

sissu

esan

dothe

rarea

sne

ed-

ingim

prov

emen

t

Team

-based

simula-

tion

training

isfeasible

andrealisticin

abu

syNICUwithap

prop

riate

plan

ning

and

implem

entation.

0.90

6

Steine

-man

net

al19

Pre-

post-in

terven

tion

stud

ySing

lesite

137team

mem

bers

(surgeo

ns;em

erge

ncy

physicians

;reside

nts;

physicianassistan

ts;

nurses;resp

iratory

therap

ists;em

ergen

cyde

partmen

ttech

nician

s)

Evalua

teim

pact

ofan

HPS

-based

,insitu

team

training

course

onteam

commun

ica-

tion

,coo

rdination,

and

clinical

efficacy

oftrau

maresuscitation

4-hHPS

-based

curricu-

lum

(web

-based

dida

c-ticfollo

wed

byHPS

training

inem

erge

ncy

depa

rtmen

t)

Performan

cech

ange

sdu

ring

HPS

-based

and

actual

trau

ma

resuscitations

Improv

emen

tin

team

-workrating

san

dclini-

caltaskspee

dan

dco

mpletionrates

0.98

3

Stoc

ker

etal28

Pre-po

stinterven

tion

stud

ySing

lesite

219PICUprov

iders

(nurses;

cardiologists;

intens

ivists;a

nesthe

-tists;

surgeo

ns;allie

dhe

alth

professiona

ls)

Evalua

tetheim

pact

ofan

embe

dde

dsimula-

tion

-based

team

train-

ingprogram

onpe

rceive

dpe

rfor-

man

ce;E

valuatethe

effect

over

differen

tph

ases

oftheprog

ram

3ph

aseprogram

ofsimulated

critical

even

tsov

er2years

Evalua

tion

question

-na

ire(assessing

impa

cton

team

work,

commun

icationskills,

assessmen

tskills,

spe-

cifictech

nica

lskills,

confi

denc

e)

Thereisa6-

to12

-mon

thlearning

curve

intheim

plem

entation

ofan

embe

dde

dmul-

tidisciplin

aryteam

training

prog

ram;

repe

ated

expo

sure

tosimulationismost

bene

ficial

tocrisis

resource

man

agem

ent

training

versus

asing

leisolated

expo

sure

0.69

2

Abbrev

iation

s:CPA

,cardiop

ulmon

aryarrest;C

RM,c

risisreso

urce

man

agem

ent;EC

MO,e

xtraco

rporea

lmem

brane

oxyg

enation;

ED,e

merge

ncyde

partm

ent;HPS

,hum

anpa

tien

tsimulated

;ICU,inten

sive

care

unit;N

ICU,n

eona

talinten

sive

care

unit;O

R,o

peratingroom

;PICU,p

ediatricintensivecare

unit;S

BTT,

simulation-based

team

training

;SJR,S

CIm

agoJourna

lRan

k;Te

amST

EPPS

®,T

eam

Strategiesan

dTo

olsto

Enha

ncePe

rforman

cean

dPa

tien

tSa

fety.

a SJR

(SJR

indica

tor)isamea

sure

ofscientificinflue

nceof

scho

larlyjourna

lsthat

acco

unts

forb

oththenu

mber

ofcitations

rece

ived

byajourna

land

theim

portan

ceor

prestigeof

thejourna

lswhe

resu

chcitations

comefrom

.Allrank

sarefrom

2016

(mos

trece

ntda

taav

ailable).



Discussion

In Situ Simulation versus Off Site SimulationExperiencesThe majority (11 of 16 studies) developed in situ simulationprograms rather than utilizing a designated simulationcenter.17–19,21–23,25,27–30 Simulation in the real work settinghas been identified as particularly valuable because it bringstogether all the elements of the care team and the environ-ment.30 In situ simulation therefore facilitates observation ofthe delivery of care as it happens, rather than how wespeculate it may happen or as it should happen if didactictools were to be followed precisely.31

The value of in situ simulation was particularly wellillustrated by studies seeking to identify system-levelissues. Patterson et al acknowledged the role of in situsimulation in the training and evaluation of technical andnontechnical skills, but emphasized the unique ways inwhich the modality could be used to evaluate systemcompetence and identify latent conditions that predisposeto medical error.18 This can be explained by the inherentoverlap of in situ simulation and system-level evaluation inexamining the conditions under which individuals work tobuild defenses and to avert or mitigate errors.32 Theseauthors conducted recurring in situ simulations to discoversafety threats and system issues in this environment. Thesimulations served not only as a way to identify these issuesbut also as means of experiential learning that amplified theability improve clinical processes. The authors noted thatthe in situ simulations prompted the identification of alatent threat in almost every simulation performed. Theycontrasted this rate of identification to that observed in thelaboratory setting and attributed the difference to a moretime-pressured environment and ability to test the actualclinical care system, including equipment, processes, andstaff response. Beyond technical skills, Patterson et al notedthat in situ simulation provided a means to continuouslyreinforce nontechnical skills (such as communication andteamwork skills).

The Multidisciplinary Evolution of Teamwork TrainingThe literature on simulation in healthcare has graduallyevolved from evaluations of individuals to evaluations ofteams.33–37 A similar progression occurred as early studiesassessing specific skills performed by individuals were fol-lowed by studies evaluating nontechnical skills and team-work. As the emphasis on teamwork increased, the value ofincorporating team members from multiple disciplines insimulation activities became evident. In a review of high-fidelity simulation in critical care training by Boling andHardin-Pierce, only 3 of 17 included articles were categor-ized as having a “mixed” population, in comparison to theremaining 14 articles with homogeneous populations ofeither nurses or physicians.5 However, the importance ofmultidisciplinary teamwork is now better reflected in theparticipants of simulation literature, as illustrated by 14 of16 included articles in this review assessing teams of multi-disciplinary participants.

Some studies adopted amultidisciplinary approach at theearliest possible stages of their intervention, consulting withmultiple stakeholders of the clinical team to refine thelearning objectives and simulated scenarios. Barbeito et alconducted several interviews with residents, intensive careunit staff, members of their critical care committee andhospital leadership, critical care unit nurses and nursingaids, and other personnel.30 This direct involvement acrossdisciplines was of great benefit to the goals of the study, asparticipants were invested in the process of organizationalchange and felt encouraged to communicate system-levelissues. The authors noted that some of the issues identified indebriefings were already well known to providers, and theprogram simply facilitated a formal way in which solutionscould be implemented. For example, several experiencednurses had noticed occasional delays in establishing intra-venous access during resuscitations. Protocols for the use ofintraosseous devices were tested and refined during simula-tion and later implemented in actual codes.

Patterson et al also recognized the value of multidisci-plinary training and discussion of social dynamics in theidentification of latent safety threats at the system level.18

These entities were described as system-based threats topatient safety that can materialize at any time and arepreviously unrecognized by healthcare providers, unit direc-tors, or hospital administration.38 Many of these previouslyunrecognized issues were brought to the attention of phy-sician staff by nursing staff and prompted multidisciplinaryproblem solving. This collaboration catalyzed a shift inculture that emphasized safety and broke down implicitauthority gradients. This paradigm shift was tested system-atically by the purposeful addition of “mistakes” in multipledomains to simulations. During debriefings, discussion cen-tered around times when team members did not feel com-fortable addressing these issues despite knowing mistakeswere being performed. Several team members describedfeeling an authority gradient during resuscitations, andthis issuewas addressed from amultidisciplinary standpointin the same manner as other latent safety threats. This studyemphasizes the role of the debriefing process not only inskill-based improvement, but also in the acquisition of ashared mental model, one of the focal concepts of qualityimprovement efforts.16,39

Recent studies have continued to address the gaps inprevious training platforms by modifying the culture toemphasize teamwork and expanding the scope of partici-pants to reflect the importance of multidisciplinary train-ing.18,21 This paradigm shift is demonstrated by the structureand learning objectives of the Surgical Trauma TrainingCourse (S2T2C).21 This curriculum was developed to fill thegaps of traditional training by adhering to a team-basededucational approach. This model included all personnelfrom corpsmen to surgeons participating in United StatesNavy pre-deployment training and demonstrated an empha-sis onmultidisciplinary team training. This emphasis reflectsrecent discussions of sociological fidelity in the context ofsimulation, which describes the interactions between lear-ners as a means of creating authenticity and social



realism.40,41 As multidisciplinary team training is bettercharacterized in the literature, there is more discussion ofsimulation as an opportunity to discuss social dynamics,hierarchy, power relations, and other factors affecting inter-professional teamwork.15,16,42,43

Evaluation of Effect on Patient OutcomesThe underlying motivation of the authors of each study indesigning the simulation training programs was ultimatelyto improve patient outcomes. However, the majority ofauthors acknowledged that this goal was beyond the scopeof their study objectives and/or would require substantiallymore statistical power to demonstrate. Aside from the fourstudies that attempted to evaluate patient out-comes,15,19,29,30 all authors ended with a discussion ofways in which future research could expand upon theirfindings to assess actual patient impact. Since only fourstudies attempted to correlate their findings with institu-tional patient outcomes, only these four studies were iden-tified as Level 4 evaluations in accordance with Kirkpatrick’slevels of evaluation in ►Table 3.13 Two of these studiesconducted simulations of the trauma bay,15,19 and theremaining two studies conducted simulations of inpatientcodes requiring ad hoc resuscitation teams.29,30 Given thecost and administrative burden of developing simulationprograms, these data are becoming increasingly importantto support the experimental data already published.

The studies that assessedpatientoutcomesused avarietyofdifferent outcome measures reflecting the unique clinicalenvironment of each acute care setting. Some studies havedrawn indirect conclusions by monitoring clinical parametersbefore, during, and/or after the simulation and comparingoutcomes. Andreatta et al used institutional pediatric CPAsurvival rates as a metric for patient outcomes and comparedthese rates tomatchednational averages.19,29The institutionalsurvival rate also served as a proxy for the effectiveness of theintervention, as they compared these data longitudinallybefore and after the simulation program. At the beginning ofthe study (whenonly 10 “informalmockcodes”hadbeen run),the pediatric CPA rate was 33%. However, they observed asignificant increase to �50% within 1 year, which also corre-lated temporally with increased frequency of mock codeswhen plotted. The authors also compared survival rates bythe type of arrhythmia triggering the code and noted theimprovements correlated with the times those rhythmswerebeing simulated inmockcodescenarios. This informationwas interpreted as evidence that the content being taught inthe simulationswas translating to improvedpatientoutcomes.

Steinemann et al also analyzed clinical process para-meters, including time to completion and reporting of keyelements of the primary trauma survey, focused abdominalultrasound, times in and out of the ER, number and type ofprocedures performed, units of blood transfused, and delaysto patient transfer.19 Corresponding patient data were alsorecorded, such as gender, morbidity, mortality, and length ofstay. While the authors observed significant improvementsin mean teamwork scores and objective parameters, such asspeed and completeness of resuscitation, no significant

change was noted in global clinical endpoints, such asmortality, morbidity, or length of stay. The study thereforeobserved significant differences in Level 3 evaluations (trans-lation of learning to clinical setting as illustrated byimproved objective parameters), but did not observe signifi-cant differences in Level 4 evaluations (patient outcomes).13

Other studies assessed potential impact on patient out-comes more qualitatively by continuously evaluating thesystem-level effect of the simulation program.15,30 Barbeitoet al achieved this by designing the intervention in a reitera-tive way to both identify system-level issues and to assessattempts at resolving these issues over time.30 This facili-tated an indirect assessment of patient outcomes throughmonitoring the consequences of process changes. For exam-ple, debriefings revealed inefficiency in the way sampleswere collected, and laboratory studies were ordered duringmock codes. This issue was addressed by the creation of astandard set of laboratories (“code labs”) that were thenautomatically ordered during codes. This process changewasthen introduced into actual codes, and a decreased incidenceof laboratory order entry errors was observed after imple-mentation of the new workflow. Capella et al reported asimilar pattern of results in their Level 3 and Level 4 evalua-tions, although their experimental setupwas different.15 Theauthors observed significantly decreased times to task com-pletion (times from arrival to computed tomography [CT]scanner, endotracheal intubation, and operating room), butpatient outcome data were not significantly differentbetween the two groups (intensive care unit length of stay,hospital length of stay, complication rate, andmortality rate).Both studies cited a small sample size as the principle reasonfor not observing significant differences in patient outcomesgiven the changes in objective parameters observed.15,30

Critique of Current EvidenceThere are very few randomized studies of simulation in teamtraining, particularly in the acute care setting. Onlyone of theincluded articles in this review randomized participants.25

Gundrosen et al randomized nurses that were being intro-duced to a new clinical guideline via either lecture-based orsimulation-based teaching methods and evaluated the effecton non-technical skills.25 Another common limitation of thecurrent simulation literature is the lack of a control group. Itis therefore challenging to rule out whether the effects seenwere due to chance or characteristics specific to the parti-cipants in the intervention group. Some studies haveattempted to address this by using a pre- and post-interven-tion design where the study group served as their owncontrol.20,21 The S2T2C used a prospective observationalstudy design where the participants served as their owncontrols.21 Similarly, Acero et al used a “cold” simulation toevaluate participants’ baseline knowledge and skills, latercomparing these results to a “warm” simulation after formaltraining had been given.20 Other groups have instead dis-cussed why a static control and experimental arm is notfeasible or desirable in the trauma setting, since in reality,trauma groups change composition dynamically.17 Asanother alternative, some studies did not utilize control



groups, but instead intentionally formed other comparablegroups.26 For example, Pascual et al validated their intensivecare curriculum for advanced practitioners by havingrecently graduated critical care fellows participate and serveas the “gold standard comparison group.”26

Very few studies evaluatedwhether the observed effects ofthe interventions were sustainable. Those studies thatassessed sustainability had mixed findings. Miller et al con-ducted in situ simulations with the goal of improving team-work and communication skills in the trauma setting.17 Theauthors evaluated the effect of the simulations in severalphases, including a “potential decay phase” in which sustain-ability was assessed after the simulation training had ended.All observed benefits had declined, and the authors concludedthat, while an in situ simulation program can be effective inimproving teamwork and communication in the clinical set-ting, these benefits are lost if the simulation program is notcontinued. Hoang et al evaluated technical and nontechnicalskills in simulated combat via the S2T2C at different points intime.21 In contrast, the authors observed improved teamworkandcommunicationskills upon thecompletionof theS2T2Caswell as after 5 months had passed. However, despite sustain-ment of significantly improved disposition times 5 monthslater, these times did increase, indicating the necessity ofrefresher courses to optimize training outcomes. Other studiesevaluated sustainability indirectly, through clinical outcomemeasures assessed after the simulation period had ended.

Sustainment was also assessed through longitudinal studydesign.28 Assessing the impact of an embedded simulationteam training program in a pediatric intensive care unit,Stocker et al observed a 6- to 12-month learning curve.28

The authors concluded that repeated exposure to simulationis the most beneficial in crisis resource management training,and single, isolated exposuremaynot be sufficient. However, alimitation of this study was the use of participant self-report-ing to assess effectiveness.

Need for Future ResearchAlthough the quantity of simulation-based research has con-tinued to increase steadily, the quality is highly variable, andfurther research issorelyneeded.44,45Amajorbarrier to furtherimplementation of simulation in acute care training is theassociated cost. Very few studies provide information on thecosts of initiating and running simulation programs, but thisinformation is necessary for the justification of this investmentin an era of tightening healthcare budgets. For example, Aceroet al reported a cost of $3.8 million in the building of theirinstitution’s simulation center and an additional $1.5 millionannually for associated operating expenses.20 Other financialand administrative burdens requiring further investigationinclude the opportunity costs of removing participants fromclinical duties and/or occupying clinical areas (especially oper-ating rooms) for in situ simulations. Finally, as previouslydiscussed, randomized studies with larger sample sizes andthe statistical power to evaluate the impact of simulations onactual patient outcomes are necessary. Data illustrating statis-tically significantchanges inpatientoutcomes, suchas lengthofstay, would enable more sophisticated cost analyses exploring

the utility of wider implementation of simulation programs.More research on the sustainability of these outcomeswill alsobe necessary to model the future impact of these investments.

Study LimitationsThis study is not without limitations. Although the searchprocess was rigorous, it is still possible that some relevantstudies weremissed and therefore not included in this review.Thepredefinedsearch strategymayhave leftoutkeywords thatwould have potentially captured additional relevant studies.For example, the use of the predefined terms “surgery,”“trauma,” and “critical care” and the subsequent applicationof predefined inclusion and exclusion criteria did not yieldobstetrics andgynecology simulation research, although simu-lated acute care scenarios may be created in this context. Inaddition, the review synthesized a relatively small amount ofstudies that each had relatively small sample sizes. This poten-tially limits thestrengthandgeneralizabilityofconclusionsandthe accurate identification of themes. However, the number ofstudies included is comparable with integrative reviews ofsimilar scope such as Boling̀and Hardin-Pierce (17 studies),5

Gjerra et al (13 studies),3 and Warren et al (10 studies).10

Conclusions

High-fidelity team-based simulation is feasible in a widevariety of acute care settings, including emergency depart-ments/trauma bays, operating rooms, intensive care units ofmultiple types, and inpatient ad hoc resuscitation teams. It isan effective means of training and/or evaluating multidisci-plinary teams inboth technical andnontechnical skills andhasthe capacity to facilitate organizational- and system-levelchange. It is also a way of involving the input of multiplestakeholders and can improve multidisciplinary teamwork.Studies over the last 10 years havebeenheterogeneous in bothintervention and evaluation design, and there is still a paucityof validated instruments available for this context. However, amore standardized approach to team-based simulation isnecessary to generate generalizable conclusions and to pro-vide evidence-based guidance for future simulation planning.These conclusions could also enhance the role of low- andmedium-fidelity team-based simulation in environmentswhere high-fidelity simulation is not possible due to logisticor financial reasons. As our understanding of “psychologicalfidelity” improves, the elements most critical to developingmultidisciplinary teamwork skills can be reproduced in lowerfidelity simulations, such as task trainers, computer-basedsystems, and virtual reality systems. Finally, there are cur-rently no studies that have demonstrated significant improve-ments in patient outcomemetrics, such asmortality or lengthof stay. The impact on patient outcomes and the sustainabilityof simulation efforts are areas that warrant further research.

Conflicts of InterestThere are no conflicts of interest.

AcknowledgmentThis research received no external funding.



References1 WeaverSJ,DySM,RosenMA.Team-training inhealthcare:anarrative

synthesis of the literature. BMJ Qual Saf 2014;23(05):359–3722 GeorgiouA, LockeyDJ. The performance and assessment of hospital

trauma teams. Scand J Trauma Resusc Emerg Med 2010;18:663 Gjeraa K, Møller TP, Østergaard D. Efficacy of simulation-based

trauma team training of non-technical skills. A systematic review.Acta Anaesthesiol Scand 2014;58(07):775–787

4 Lewis R, Strachan A, Smith MM. Is high fidelity simulation the mosteffective method for the development of non-technical skills innursing?Areviewof the currentevidence.OpenNurs J 2012;6:82–89

5 Boling B, Hardin-Pierce M. The effect of high-fidelity simulationon knowledge and confidence in critical care training: an inte-grative review. Nurse Educ Pract 2016;16(01):287–293

6 Doolen J, Mariani B, Atz T, et al. High-fidelity simulation inundergraduate nursing education: a review of simulationreviews. Clin Simul Nurs 2016;12:290–302

7 Al-Ghareeb AZ, Cooper SJ. Barriers and enablers to the use of high-fidelity patient simulation manikins in nurse education: anintegrative review. Nurse Educ Today 2016;36:281–286

8 Cant RP, Cooper SJ. Simulation-based learning in nurse education:systematic review. J Adv Nurs 2010;66(01):3–15

9 Tan SB, Pena G, Altree M, Maddern GJ. Multidisciplinary teamsimulation for the operating theatre: a review of the literature.ANZ J Surg 2014;84(7-8):515–522

10 Warren JN, Luctkar-Flude M, Godfrey C, Lukewich J. A systematicreview of the effectiveness of simulation-based education onsatisfaction and learning outcomes in nurse practitioner pro-grams. Nurse Educ Today 2016;46:99–108

11 CASPCriticalAppraisal Skills Programme. 2014.Available at: http://www.phru.nhs.uk/Pages/PHD/CASP.htm. Accessed March 11, 2018

12 McHugh ML. Interrater reliability: the kappa statistic. BiochemMed (Zagreb) 2012;22(03):276–282

13 Kirkpatrick D. Evaluating Training Programs: The Four Levels. SanFrancisco: Berrett-Koehler; 2014

14 Briggs A, Raja AS, Joyce MF, et al. The role of nontechnical skills insimulated trauma resuscitation. J Surg Educ 2015;72(04):732–739

15 Capella J, Smith S, Philp A, et al. Teamwork training improves theclinical care of trauma patients. J Surg Educ 2010;67(06):439–443

16 FalconeRAJr,DaughertyM,SchweerL, PattersonM,BrownRL,GarciaVF. Multidisciplinary pediatric trauma team training using high-fidelity trauma simulation. J Pediatr Surg 2008;43(06):1065–1071

17 Miller D, Crandall C, Washington C III, McLaughlin S. Improvingteamwork and communication in trauma care through in situsimulations. Acad Emerg Med 2012;19(05):608–612

18 Patterson MD, Geis GL, Falcone RA, LeMaster T, Wears RL. In situsimulation: detection of safety threats and teamwork training in ahighriskemergencydepartment.BMJQualSaf2013;22(06):468–477

19 Steinemann S, Berg B, Skinner A, et al. In situ, multidisciplinary,simulation-based teamwork training improves early trauma care.J Surg Educ 2011;68(06):472–477

20 Acero NM,Motuk G, Luba J, et al. Managing a surgical exsanguina-tion emergency in the operating room through simulation: aninterdisciplinary approach. J Surg Educ 2012;69(06):759–765

21 Hoang TN, Kang J, Siriratsivawong K, et al. Hyper-realistic, team-centered fleet surgical team training provides sustained improve-ments in performance. J Surg Educ 2016;73(04):668–674

22 Huser AS,MüllerD, Brunkhorst V, et al. Simulated life-threateningemergency during robot-assisted surgery. J Endourol 2014;28(06):717–721

23 Kellicut DC, Kuncir EJ, Williamson HM, Masella PC, Nielsen PE.Surgical Team Assessment Training: improving surgical teamsduring deployment. Am J Surg 2014;208(02):275–283

24 Figueroa MI, Sepanski R, Goldberg SP, Shah S. Improving teamwork,confidence, and collaboration among members of a pediatric cardi-ovascular intensive care unit multidisciplinary team using simula-tion-based team training. Pediatr Cardiol 2013;34(03):612–619

25 Gundrosen S, Solligård E, Aadahl P. Team competence amongnurses in an intensive care unit: the feasibility of in situ simula-tion and assessing non-technical skills. Intensive Crit Care Nurs2014;30(06):312–317

26 Pascual JL, Holena DN, Vella MA, et al. Short simulation trainingimproves objective skills in established advanced practitionersmanaging emergencies on the ward and surgical intensive careunit. J Trauma 2011;71(02):330–337, discussion 337–338

27 Reed DJ, Hermelin RL, Kennedy CS, Sharma J. Interdisciplinaryonsite team-based simulation training in the neonatal intensivecare unit: a pilot report. J Perinatol 2017;37(04):461–464

28 Stocker M, Allen M, Pool N, et al. Impact of an embeddedsimulation team training programme in a paediatric intensivecare unit: a prospective, single-centre, longitudinal study. Inten-sive Care Med 2012;38(01):99–104

29 Andreatta P, Saxton E, Thompson M, Annich G. Simulation-basedmock codes significantly correlate with improved pediatricpatient cardiopulmonary arrest survival rates. Pediatr Crit CareMed 2011;12(01):33–38

30 Barbeito A, Bonifacio A, Holtschneider M, Segall N, Schroeder R,Mark J; Durham Veterans Affairs Medical Center Patient SafetyCenter of Inquiry. In situ simulated cardiac arrest exercises to detectsystem vulnerabilities. Simul Healthc 2015;10(03):154–162

31 Mondrup F, Brabrand M, Folkestad L, et al. In-hospital resuscita-tion evaluated by in situ simulation: a prospective simulationstudy. Scand J Trauma Resusc Emerg Med 2011;19:55

32 Reason J. Human error: models and management. BMJ 2000;320(7237):768–770

33 Lee SK, Pardo M, Gaba D, et al. Trauma assessment training with apatient simulator: a prospective, randomized study. J Trauma2003;55(04):651–657

34 Holcomb JB, Dumire RD, Crommett JW, et al. Evaluation of traumateam performance using an advanced human patient simulatorfor resuscitation training. J Trauma 2002;52(06):1078–1085,discussion 1085–1086

35 King DR, Patel MB, Feinstein AJ, Earle SA, Topp RF, Proctor KG.Simulation training for amass casualty incident: two-year experienceat the Army Trauma Training Center. J Trauma 2006;61(04):943–948

36 Block EF, Lottenberg L, Flint L, Jakobsen J, Liebnitzky D. Use of ahuman patient simulator for the advanced trauma life supportcourse. Am Surg 2002;68(07):648–651

37 Hogan MP, Pace DE, Hapgood J, Boone DC. Use of human patientsimulation and the situation awareness global assessment tech-nique in practical trauma skills assessment. J Trauma 2006;61(05):1047–1052

38 Alfredsdottir H, Bjornsdottir K. Nursing and patient safety in theoperating room. J Adv Nurs 2008;61(01):29–37

39 Davies JM. Team communication in the operating room. ActaAnaesthesiol Scand 2005;49(07):898–901

40 Boet S, Bould MD, Layat Burn C, Reeves S. Twelve tips for asuccessful interprofessional team-based high-fidelity simulationeducation session. Med Teach 2014;36(10):853–857

41 Sharma S, Boet S, Kitto S, Reeves S. Interprofessional simulatedlearning: the need for ‘sociological fidelity’. J Interprof Care 2011;25(02):81–83

42 Dieckmann P, Gaba D, Rall M. Deepening the theoretical founda-tions of patient simulation as social practice. Simul Healthc 2007;2(03):183–193

43 SydorDT, BouldMD, Naik VN, et al. Challenging authority during alife-threatening crisis: the effect of operating theatre hierarchy. BrJ Anaesth 2013;110(03):463–471

44 Cook DA, Hatala R, Brydges R, et al. Technology-enhanced simula-tion for health professions education: a systematic review andmeta-analysis. JAMA 2011;306(09):978–988

45 McGaghie WC, Issenberg SB, Cohen ER, Barsuk JH, Wayne DB.Translational educational research: a necessity for effectivehealth-care improvement. Chest 2012;142(05):1097–1103



http://www.phru.nhs.uk/Pages/PHD/CASP.htm

http://www.phru.nhs.uk/Pages/PHD/CASP.htm

theroleofhigh-fidelityteam-basedsimulation in acute care ... · a comprehensive analysis that...

Documents