Use of Simulation in MOC: NOT Ready for Prime TIME

Dr Kevin Lachapelle Adair Chair in Surgical Education

9th Annual Meeting of the ACS-AEI Consortium

High Stakes Simulation for MOC: Not Ready for Prime Time

• 1) What problem?

• 2) Little Data on Simulation for MOC

• 3) Simulation good for Formative Assessment, less for Summative Assessment

• 4) Needs to be aligned with Scope of practice/patient outcomes

• 5) Mandatory does not equal Buy in

• 6) Logistics and Cost

Problem?

STS Database

• 1 500 000 patients subjected to CABG 2000-2009

• predicted mortality 2.4%

• Observed mortality 2.4% to 1.9%

• Reduction in

• Stroke

• haemorrhage

• sternal wound infection

• 2000 complaints

• 200 physician interviews

• 3% of files result in disciplinary action

• communication

• sexual

Data on Simulation in MOC

• Association Between Licensure Examination Scores and Practice in Primary Care

R Tamblyn JAMA 288(23) 2002

• Association Between MOC Examination and Quality of Care for Medicare Beneficiaries

E Holmboe Archives Internal Medicine 168 (3) 2008

• American Association of Anesthesiology

• American Board of Internal Medicine

• Interventional Cardiology

Experience with Mandatory Simulation

• Learner outcome of Anesthesiologist SBE for MOC

• 8 studies

• Most Kirkpatrick 1, no comparison to other methodologies

“There is a paucity of empirical data that support the use of simulation for certification of trainees and practicing surgeons”

Simulation, Learning and Assessment

• What do we know ( we think)

JAMA. 2011;306(9):978-988

• Screen 10903 articles • 609 articles • Pre-test, intervention, post-test with a

comparison group • 35 000 participants. • Knowledge, skills, behaviour, ? Patient

outcome

Meta-analysis

• Majority in SimCentre • Majority Novice/beginner

• Students 27% • Residents 24.7% • Nursing 11.8% • Physicians 16.2%

• Technical skills 67%

17

FLS + FLS - Surgeon Age 38.2 50.4 Rate of BDI 0.47% 0.14%

• Simulator For Assessment

• Specifically validated for purposes of assessment

• Distinguish Beginner, Intermediate, Fully trained, Expert

• Correlate with actual clinical outcome

Impact of New Technologies ?

• Virtual reality

• 3D printing

• Live Tissue

Scope of Practice

• Changes during career

• Narrows, more specific

• Focus on Patient outcomes

• Work-based Assessment

• Shows competency, performance gaps

Tyranny of Mandatory

• Buy in equals success

• build on successful projects

Logistics and Cost

• 77 000 Canadian Physicians

• Impact? ( one sim activity /year)

• Capacity? ( Sim Centre across Canada)

• Cost $156 000 000!