ipssw pre-conference workshop 7 (2016, glasgow)
TRANSCRIPT
Simulation Research:Improving Rigor, Impact, and
Methodology
May 8, 2016
Glasgow, UK
IPSSW Pre-Conference Workshop
International Network for Simulation-based Pediatric Innovation, Research and Education
Introduction / Faculty
• Ralph MacKinnon
• Todd Chang
• Vinay Nadkarni
• David Kessler
Disclosures
• No relevant financial conflicts of interest
• INSPIRE receives funding/support from
International Network for Simulation-based Pediatric Innovation, Research and Education
International Network for Simulation-based Pediatric Innovation, Research and Education
Mission
We aim to improve the delivery of medical care to acutely ill children
by answering important research questions pertaining to resuscitation,
technical skills, behavioral skills, debriefing and simulation-based
education
International Network for Simulation-based Pediatric Innovation, Research and Education
Who are you?
• Where do you work with Simulation?
• What is your simulation experience?
• What is your research experience?
International Network for Simulation-based Pediatric Innovation, Research and Education
Learning Objectives
• Define the 2 types of simulation-based research
• Evaluate characteristics of a simulation-based research question or protocol that are conducive to quality research and output
• Identify 6 specific challenges in simulation-based research that threaten rigor and quality of research output
Objectives
1. Develop a study design using simulation as the subject of research
2. Develop a study design using simulation as an investigative methodology
3. Use Reporting Guidelines to improve study methodology
International Network for Simulation-based Pediatric Innovation, Research and Education
Schedule
International Network for Simulation-based Pediatric Innovation, Research and Education
Didactic Discussion Small group work Break
Closing Discussion & Networking
Objectives
1. Develop a study design using simulation as the subject of research
2. Develop a study design using simulation as an investigative methodology
3. Use Reporting Guidelines to improve study methodology
International Network for Simulation-based Pediatric Innovation, Research and Education
International Network for Simulation-based Pediatric Innovation, Research and Education
2 Types of Simulation Research
Research subject
(e.g. Team training)
Research method
(e.g. Test new defibrillator)
International Network for Simulation-based Pediatric Innovation, Research and Education
1: Simulation as the research subject
• Is simulation an effective training methodology?
• How do we maximize effects on patient outcomes?
2: Simulation as the research method
• Leverages standardization of simulation
• Questions that might not be safe, feasible, ethical, timely in traditional research
Simulation as the subject of research
International Network for Simulation-based Pediatric Innovation, Research and Education
International Network for Simulation-based Pediatric Innovation, Research and Education
INSPIRE Research Themes
Simulation as the subject
Debriefing Develop/assess/implement effective techniques for debriefing real/sim events
IPE, Teamwork, Communication
Develop/assess/implement effective techniques for team training
Procedural, Psychomotor Skills
Develop/assess/implement effective techniques for skills development retention
Simulation as the subject
• Research examining whether or not specific features of simulation experiences are effective
International Network for Simulation-based Pediatric Innovation, Research and Education
International Network for Simulation-based Pediatric Innovation, Research and Education
Simulation as the subject
What we know…
• 609 studies
• 23% RCT’s
• 12% multicenter studies
• 5% reported patient and/or healthcare outcomes
• Sim is an effective educational intervention compared to no intervention
International Network for Simulation-based Pediatric Innovation, Research and Education
Pediatric Research
• 57 studies
• 1/3 RCT
• compare sim to nointervention
• Large pooled effective size
• Knowledge
• Skills in simulation
• Time to completion
• < 10% on patient outcome
International Network for Simulation-based Pediatric Innovation, Research and Education
2 Lessons
International Network for Simulation-based Pediatric Innovation, Research and Education
1. Avoid the null comparison • (Don’t compare to no intervention)
Varying Instructional Design
1. Feedback ***
2. Repetitive Practice ***
3. Distributed Practice
4. Curriculum Integration
5. Clinical variation
6. Range of difficulty
7. Individualized learning
8. Multiple learning strategies
9. Defined Outcomes
10. Valid simulator
International Network for Simulation-based Pediatric Innovation, Research and Education
2 Lessons
International Network for Simulation-based Pediatric Innovation, Research and Education
2. Measure patient outcomes • (Or at least provider behavior)
Knowledge/skills
Attitudes/perceptions
Learners’ satisfaction/reaction
Yadley 2011, BEME 2005, Miller 1990, Kirkpatrick 1967
Δ practice
Δ behavior
Knowledge/skills
Attitudes/perceptions
Learners’ satisfaction/reaction
Yadley 2011, BEME 2005, Miller 1990, Kirkpatrick 1967
Popul-ation
Patient
Δ practice
Δ behavior
Knowledge/skills
Attitudes/perceptions
Learners’ satisfaction/reaction
Yadley 2011, BEME 2005, Miller 1990, Kirkpatrick 1967
Framing your Research Outcomes
• Goal of medical education• To transmit knowledge, impart skills, and inculcate
the values of medicine
that will improve health outcomes
Cooke, Irby, O’Brien. Carnegie Foundation Report: Educating Physicians: A call for reform of medical school and residency. 2010
International Network for Simulation-based Pediatric Innovation, Research and Education
Selecting outcomes
• Important to patient• Measurable• Dependent on provider skills/behaviors
• Quality• Satisfaction, comfort
• Procedure related **• Success/failure• Complications (Erb’s Palsy, Line Infections)
• Survival • Cardiac resuscitation, obstetrics, NICU
International Network for Simulation-based Pediatric Innovation, Research and Education
INSPIRE Examples
International Network for Simulation-based Pediatric Innovation, Research and Education
Infant Lumbar Punctures
• Essential pediatric procedure
• Decreasing experience
• Novice success rate 35%
• Bench top simulator• Low cost
• Easy to use
Auerbach et al Pediatric Emergency Care 2013, Gaies Pediatrics 2007
International Network for Simulation-based Pediatric Innovation, Research and Education
Infant Lumbar Punctures
• Instructional Design principles: • Distributed Practice (JITT)
• Repeated Practice
• Defined Outcomes
• Outcomes selected:• Clinical Lumbar Puncture success
International Network for Simulation-based Pediatric Innovation, Research and Education
Research Question
• Population
• Intervention
• Comparison
• Outcome
International Network for Simulation-based Pediatric Innovation, Research and Education
Research Question
• P – 1st year residents (‘interns’)
• I – Mastery Learning LP Training + Just-in-Time Training on LP Simulator just prior to clinical LP
• C – Mastery Learning LP Training only
• O – Clinical LP success rate
International Network for Simulation-based Pediatric Innovation, Research and Education
Auerbach et al, Pediatric Emergency Care 2013
Overall Performance Score
International Network for Simulation-based Pediatric Innovation, Research and Education
Objectives
1. Develop a study design using simulation as the subject of research
2. Develop a study design using simulation as an investigative methodology
3. Use Reporting Guidelines to improve study methodology
International Network for Simulation-based Pediatric Innovation, Research and Education
Simulation as an investigative methodology
International Network for Simulation-based Pediatric Innovation, Research and Education
International Network for Simulation-based Pediatric Innovation, Research and Education
INSPIRE Research Themes
Simulation as the method
Technology Develop/assess/implement novel technologies designed to improve processes of care and pediatric patient outcomes
Acute Care and Resuscitation
Develop/assess/implement novel techniques for improving care of pediatric patients
Human Factors Assess the role of human factors when providing care to pediatric patients
Patient Safety Explore the key variables that influence patient safety and assess strategies to mitigate
Simulation as the method
• The simulated environment is used as an experimental model to study factors affecting human and systems performance in healthcare.
International Network for Simulation-based Pediatric Innovation, Research and Education
International Network for Simulation-based Pediatric Innovation, Research and Education
Why use simulated instead of real scenarios
• Standardization of “noise”
• Patient- age, clinical status, parent actions
• Individuals- experience, stress, fatigue
• Team- structure, numbers
• Environment- structure, distractions, noise
• Technology- equipment availability, size, safety
• Ask questions that might not be safe, feasible, ethical, timely in traditional research
Threats to internal validity
International Network for Simulation-based Pediatric Innovation, Research and Education
Threats to validity
• Educator training
• Simulator Selection
• Scenario Design
• Confederates
• Realism (physical, conceptual and emotional)
• Debriefing
• Video Capture and Review
• Study OutcomesCheng et al, Pediatrics, 2014
• ADD SLIDES ON EDUCATIONAL DESIGNS- e.g. solomon 4, cluster randomized etc…(martin’s paper, selin workshop)
• Ethics- rct not always possible. (perfume example)
INSPIRE Examples
International Network for Simulation-based Pediatric Innovation, Research and Education
Technology
• A QCPR simulator was
used to show that
Feedback Devices
improved CPR
performance
International Network for Simulation-based Pediatric Innovation, Research and Education
Systems/Human Factors
• Individuals
• Teams
• Environments
• Technological Factors
• Systems Factors
• Patient Factors
International Network for Simulation-based Pediatric Innovation, Research and Education
SEIPS 2.0. Ergonomics . 2013 November ; 56(11) Holden, Carayon et al
International Network for Simulation-based Pediatric Innovation, Research and Education
SEIPS 2.0. Ergonomics . 2013 November ; 56(11) Holden, Carayon et al
International Network for Simulation-based Pediatric Innovation, Research and Education
New Environments
International Network for Simulation-based Pediatric Innovation, Research and Education
International Network for Simulation-based Pediatric Innovation, Research and Education
Safety / Quality
• In-situ simulation• Mobile unit brought into work environment
• Use real life equipment in “context”
• Work with real life team
• Identify latent safety threats
• Provider level - insufficient staffing/training
• Systems level - missing/broken equipment
International Network for Simulation-based Pediatric Innovation, Research and Education
Direct comparison: same patient at different hospital
Crash Testing, not Normal Testing
• Identify and remediate threats to safety • Active
• Latent: unrecognized errors in design or organization
International Network for Simulation-based Pediatric Innovation, Research and Education
Objectives
1. Develop a study design using simulation as the subject of research
2. Develop a study design using simulation as an investigative methodology
3. Use Reporting Guidelines to improve study methodology
International Network for Simulation-based Pediatric Innovation, Research and Education
Reporting Guidelines
• Standardized reporting will enhance quality of studies and assist journal reviewer/editors
• Explanation and elaboration document will support implementation of reporting guidelines
• Reporting guidelines currently under review• Eventual adoption for author guidelines
International Network for Simulation-based Pediatric Innovation, Research and Education
Reporting Guidelines
• Extensions to prospective trial design:
• CONSORT (randomized-control)
• STROBE (observation cohort)
International Network for Simulation-based Pediatric Innovation, Research and Education
How do we use Reporting Guidelines?
International Network for Simulation-based Pediatric Innovation, Research and Education
Objectives
1. Develop a study design using simulation as the subject of research
2. Develop a study design using simulation as an investigative methodology
3. Use Reporting Guidelines to improve study methodology
International Network for Simulation-based Pediatric Innovation, Research and Education
What changes will you be making?
International Network for Simulation-based Pediatric Innovation, Research and Education
What changes will you be making?
International Network for Simulation-based Pediatric Innovation, Research and Education