The Veterans Health Administration (VHA) initiated a national directive for facilities to establish a core group of providers to perform airway management outside of the OR.1

Respiratory Therapists (RTs) have been identified in the directive as a preferred airway responder. At the VA Medical Center in Iowa City, RTs are the primary responders for airway management on off hours and weekends. The RTs at the VA do not consistently demonstrate the knowledge, skills, and confidence to perform effective airway management in emergent situations.

1 Veterans Health Administration. (2012). Out of operating room airway management. (Directive 2012-032). Washington, DC: Department of Veteran Affairs. Retrieved from http://www.va.gov/LMR/MOUs/AFGE.asp 2 Kirkpatrick, D.L. (1996). Evaluating training programs: The four levels. San Francisco, CA: Berret-Koehler Publishers. 3 Riggs, M.L., Warka, J., Babasa, B., Betancourt, R., & Hooker, S. (1994). Development and validation of self-efficacy and outcomes expectancy scales for job related, applications. Education and Psychological Measurement, 54 (3), 793-802. 4Durning, S.J., Cation, L.J., Markert, R.J., & Pangaro, L.N. (2002). Assessing the reliability and validity of the mini–clinical evaluation exercise for internal medicine residency training. Academic Medicine, 77 (9), 900-904. 5 Kim, J., Neilipovitz, D., Cardinal, P., Chiu, M., & Clinch, J. (2006). A pilot study using high-fidelity simulation to formally evaluate performance in the resuscitation of critically ill patients: the university of Ottawa critical care medicine high-fidelity simulation, and crisis resource management. Critical Care Medicine, 34 (8), 2167-2174. 6 Issenberg, S.B., McGaghie, W.C. Petrusa, E.R., Gordon, D.L. & Scalese, R.J. (2005). Features and uses of high-fidelity medical simulations that lead to effective learning: a BEME systematic review*. Medical Teacher, 27 (1), 10-28.

The VHA mandated airway management training occur, and this Capstone Project accomplished this by establishing competency in the RT staff to perform airway management. This methodology was successful in meeting the directive by providing training that was adapted to meet needs specific to RTs and utilizing established methods to evaluate learning and knowledge acquisition. High-fidelity simulation was a useful tool to provide risk-free training for the RTs in core skills that are difficult to establish with experiential learning.6 This project design offers an effective way to evaluate training to demonstrate:

1. Relevance of the training to learners. 2. Acquisition of knowledge and skills. 3. Transfer of knowledge and skills for job performance. 4. Tangible results to the organization as a result of training.2

Conceptual Framework Program evaluation design utilizing Kirkpatrick’s Evaluating Training Programs: The Four Levels. 2 Pre-Program Self-efficacy and outcomes expectancy based needs assessment questionnaire concerning airway management skills, knowledge, and outcomes. 3 Step Educational Program Based on VHA SimLEARN Program A computerized didactic module developed by VHA to teach content matter specific to airway management and airway management devices. A technical skills workshop which incorporates a high-fidelity simulation manikin to practice using basic and advanced airway management devices. Use of high-fidelity simulation with debriefing to evaluate clinical decision making and behavior during clinically based airway management. Post-Program Ongoing evaluation of self-efficacy and outcomes expectancy. Repeat evaluation of technical skills and simulation testing.

Evaluation Tools Permission granted for use of each tool. IRB approval granted. The Self-Efficacy and Outcomes Expectancy Scale is a general scale to measure work specific ability to perform a job and the likely consequences the expected behavior will produce. 3 (Adapted) Cronbach’s α=.86 The Mini-CEX is an observation evaluation tool that utilizes a 9 point rating scale to delineate performance appraisal of general competency of patient care and procedural skills.4 (Adapted) Cronbach’s α=.90 The Ottawa Global Rating Scale is an observational evaluation tool that utilizes a 7 point likert scale for performance appraisal of ideal behaviors and overall function during crisis resource management. 5 ICC=.61

Acknowledgements: I would like to thank my advisor Dr. Joseph and mentor Mary Eagan, BSN, for their guidance and support during this project. I would also like to thank the respiratory therapists at the Iowa City VA Medical Center for their willingness to participate fully in this training program.

Mitch Dotson, CRNA, MSN, DNP Student Veterans Affairs Medical Center, Iowa City, Iowa

Successfully improved RT knowledge, skill level, and confidence in performing airway management. Acquisition of airway management knowledge in a safe and effective learning atmosphere through the use of patient simulation. Identification of gaps in knowledge and skill to foster a desired need for continued learning in airway management by the RT staff. Empowerment of the RT staff by providing RT specific training to an often neglected professional group.

FUTURE PLANS Provide feedback to VHA to improve nationally produced training materials for simulation training in airway management for expansion to other clinicians (ER physicians, pulmonary fellows). Evaluate code blue and rapid response team data at the VA to determine if established competence in the RT staff translates into decreased morbidity and mortality in emergency situations.

To design and implement a training program to establish RT competency in cognitive and procedural airway management to institute a core group of providers to safely perform airway management.

Expected outcomes of the training include improving RT knowledge, technical

skills, and confidence when performing airway management

Kirkpatrick’s Four Levels of Evaluating Training Programs

Level 1 Reaction

• How did the participants react to the training? • Post-training reaction questionnaire

Level 2 Learning

• To what extent did participants improve knowledge or skill? • Post-didactic test • Mini-CEX evaluation of technical skills testing • Mini-CEX evaluation of simulation testing • Simulation performance appraisal during debriefing

Level 3 Behavior

• Did behavior change as a result of training?

• Ottawa Global Rating Scale evaluating behavior • Post-training re-assessment self-efficacy and outcomes expectancy

questionnaire • Participant interview during simulation debriefing for planned behavior

change

Level 4 Results

• What benefits does the organization receive? • Competent RTs to perform airway management evidenced by:

• 80% passing score on didactic test • Mini-CEX scores >4 on technical skills and simulation testing • Ottawa GRS scores ≥ 5

Baseline Assessment Evaluation of needs assessment and self-efficacy and outcome

expectancy questionnaire to prioritize training needs.

Self-Efficacy RT Outcome Measures (0-100) N=13 Pre Post Confidence of skill set to perform airway management 76.9 82.3 Adequacy of training to perform airway management 77.6 93 Knowledge of difficult airway algorithm 63.8 81.5 Bag Mask Ventilation (BMV) 86.9 90 Laryngeal Mask Airway (LMA) 78.4 88.4 Direct Laryngoscopy (DL) 76.9 84.6 Video Laryngoscopy (VL) 72.3 91.5

Testing Scores N=13 Average Score on didactic test: Aim 80% 100% pass rate of computerized test (1 repeat test) Score on technical skills test: Aim > 4 (9 point scale) BMV=8.4 LMA=8.5 DL=8.3 VL=8.6 Scores on simulation testing: Aim > 4 (9 point scale) 6.5 Ottawa GRS scores: Aim ≥ 5 (7 point scale) 5.2

Reaction Questionnaire RT Outcome Measures (0-100) N=13 Post-Program Belief that technical skills of airway management improved 97.6 Belief that knowledge of airway management improved 98.4 Understanding of appropriate airway management clinical actions improved 97.6 More confident to perform airway management 96.9