The Institute for Patient Safety and Medical Simulation

Baptist Health/Auburn University

Montgomery, Alabama

Are We Supposed to Be Measuring Something?

Randy Johnson, Ph.D. & Judi Miller, MSN, RN

WHO ARE WE?

Hospital Based Simulation Center 3 Hospitals in Montgomery,

Alabama 3,000+ Employees 450 Patients

Community Focused

Faith Based

THE INSTITUTE FOR PATIENT SAFETY AND MEDICAL SIMULATION

22,500 Square Feet 2 ICU/ED/Med-Surg

Simulation Rooms OR Suite 12 Bed Simulation

Lab Pediatric

Simulation Lab 2 Pre-Hospital

Simulation Rooms Cardiac Simulation

Lab 6 Task Training

Rooms

OUR PARTNERS

Baptist Health Auburn University Residency Programs Area Schools of Nursing Pre-Hospital Providers Alabama Department of

Public Health Healthcare Career

Programs

WE ARE ALL ABOUT ERROR REDUCTION!

Institute of Medicine Reports Josie King Story Crew Resource Management

http://www.baptistfirst.org/careers/

“With the growing knowledge that millions of patients are being harmed daily throughout the world as a consequence of preventable adverse events, the urgency could not be greater for patient safety solutions that will help practitioners and health care organizations avoid these tragic occurrences.”

- Dennis S. O’Leary

Making Health Care Safer

OVERVIEW

The SMART Training Model

Kirkpatrick’s typology Levels of Training

Evaluation Organizational Change Literature Review

Medicine Aviation

AMS

RT

ynergisticedicalndesource

eam

TrainingEvaluating

Process

Application

Knowledge

Experience

Making Health Care

Safer

The SMART Training Model

CREW RESOURCE MANAGEMENT (CRM)

Process

CREW RESOURCE MANAGEMENT (CRM)

Flight 1549 January 15, 2009

CREW RESOURCE MANAGEMENT (CRM)

“CRM is not and never will be the mechanism to eliminate error and assure safety in a high-risk endeavor…” Helmreich, Merritt,

and Wilhelm, (1999) The International Journal of Aviation Psychology

CREW RESOURCE MANAGEMENT (CRM)

“The safety of operations is influenced by professional, organizational, and national cultures and safety requires focusing each of these toward an organizational ‘safety culture’ that deals with errors nonpunitively and proactively.” Helmreich, Merritt, and Wilhelm, (1999) The International Journal of Aviation Psychology

CREW RESOURCE MANAGEMENT (CRM)

Team Communication Process (Protocol) and

Patterned Communications

PURPOSE-BASED DECISION MAKING

Application

PURPOSE-BASED DECISION MAKING

• Emotional Hijacking• Working Memory

PURPOSE-BASED DECISION MAKING

Emotional Hi-jacking “Emotions can quickly

hijack other brain processes, diverting attention, and determining what will be remembered, and what won't.” (Eide, F., Eide, B., http://stanford.wellsphere.com/brain-health-article/emotional-hijack/469427)

PURPOSE-BASED DECISION MAKING

Working Memory “The ability to hold in mind information in the face of potentially interfering distraction in order to guide behavior.”

Jarrold & Towse, “Individual Differences in Working Memory,” 2006

Working Memory—Nor-mal Lapses “The occurrence of lapses may be critical in occupational groups with a need to maintain a high level of alertness… Lapses in these occupations can have disastrous consequences, including multiple fatalities.”

Peiris, Davidson, & Bones, “Event-based Detection of Lapses of Responsiveness, 2008

PURPOSE-BASED DECISION MAKING

EVIDENCE-BASED PRACTICE

Knowledge

SIMULATION

Experience

Simulation Used in SMART Training Is:

1. Team Based

2. Realistic

Simulation Provides the

Opportunity to Apply SMART Principles in

Realistic Settings

AND . . .

Facilitates Individual and

Team Learning in a Non-Threatening,

Ethical, and Realistic

Environment

SIMULATION

It’s All About

Teamwork

KIRKPATRICK’S TYPOLOGY--1976

EVALUATING TRAINING

KIRKPATRICK’S TYPOLOGY

Evaluating Training: A Multi-Level Approach

KIRKPATRICK’S TYPOLOGY

1. Reaction of the Student to The Training

Was the training worthwhile, relevant, and interesting?

KIRKPATRICK’S TYPOLOGY

1. Reaction of the Student to The Training

Data Can be Captured Through: Feedback Forms (Student

Evaluations) Online Feedback Verbal Reaction

KIRKPATRICK’S TYPOLOGY

2. Is There Evidence of Student Learning?

Assessment of Learning is Based on Ability, Knowledge, and

Attitudes

KIRKPATRICK’S TYPOLOGY

2. Is There Evidence of Student Learning?

Data Collection: - Attitudinal Questionnaires - Assessment Testing (Pre/Post) - Observation and/or Interview - Trained Observers in Simulation

KIRKPATRICK’S TYPOLOGY

3. Is There Behavioral Evidence of Student Learning?

Is There a Transfer of Knowledge

or Application of Skills to the Job?

KIRKPATRICK’S TYPOLOGY

3. Is There Behavioral Evidence of Student Learning?

Data Collection: - ANTS - SMART Champions - Trained Observers in Simulation

KIRKPATRICK’S TYPOLOGY

4. Is There Evidence of a Training Effect in The Organization/Environment

Is There a Clear Correlation Between Training and Organizational Impact?

KIRKPATRICK’S TYPOLOGY

4. Is There Evidence of a Training Effect in The Organization/Environment

Data Collection: - Highly Valued - Difficult to Collect - Few Studies

OUR RESULTS

Baptist Medical Center East – PACU Staffing: Five or six nurses per day Patient Population: 3,996 patients

during the 24 month study Study Period: November 2006—

October 2008 Control: November 2006—October 2007

(2,024 Patients) Experimental: November 2007—October

2008 (1,972 Patients)

Patient Safety:“Freedom from accidental injury,” or “avoiding injuries or harm to patients from care that is intended to help them.” Ensuring patient safety “involves the establishment of operational systems and processes that minimize the likelihood of errors and maximizes the likelihood of intercepting them when they occur.”

Complication:“An injury caused by medical management rather than by the underlying disease or condition of the patient.” In general, adverse events prolong the hospitalization, produce a disability at the time of discharge, or both.

OUR RESULTS

Patient Safety Indicants:OUR RESULTS

Reintubation: An unexpected reintubation is defined as an intubation that occurs in PACU after anesthesia has extubated the patient in surgery and the patient has been able to maintain their airway prior to placement in the PACU.

The six “areas of evidence” used are: 1. Face Validity, 2. Precision, 3. Minimum Bias, 4. Construct Validity, 5. Fosters Real Quality Improvement, 6. Application

Patient Safety Indicants:OUR RESULTS

Unplanned Admissions to a Critical Care Unit: These admissions are defined as an unexpected decline in patient status and requiring a higher level of care. All PACU patients identified in advance as requiring postoperative ICU/CCU care were excluded from the data.

The six “areas of evidence” used are: 1. Face Validity, 2. Precision, 3. Minimum Bias, 4. Construct Validity, 5. Fosters Real Quality Improvement, 6. Application

Patient Safety Indicants:OUR RESULTS

Post-operative Complications: These are defined as bleeding or cardiovascular issue(s) requiring unplanned treatment. Untreated, such complications would lead to a rapid worsening in the patients’ condition.

The six “areas of evidence” used are: 1. Face Validity, 2. Precision, 3. Minimum Bias, 4. Construct Validity, 5. Fosters Real Quality Improvement, 6. Application

OUR RESULTS

The six “areas of evidence” used are: 1. Face Validity, 2. Precision, 3. Minimum Bias, 4. Construct Validity, 5. Fosters Real Quality Improvement, 6. Application

Nov 2006 Oct 2007 Nov 2007 Oct 2008N

ov

em

be

r 20

07

Experimental Group

Control Group

SM

AR

T Im

ple

me

nta

tio

n N

ov

20

07

OUR RESULTS

The six “areas of evidence” used are: 1. Face Validity, 2. Precision, 3. Minimum Bias, 4. Construct Validity, 5. Fosters Real Quality Improvement, 6. Application

Control Period Percentage Post Treament Percentage NormalizedMeasure Nov 06 -Oct 07 of instances Nov 07 - Oct 08 of instances Impact Practical Impact

Total Patients 2024 1972Post operative complication 27 1.33% 14 0.71% reduction of 13 47%Unexpected Admission to CCU 6 0.30% 2 0.10% redution of 4 66%Re-intubation 6 0.30% 0 0.00% reduction of 6 100%

SELECTED RESEARCH

SELECTED RESEARCH: NURSINGSimulator Effects on Cognitive Skills and

Confidence Levels(Brannan & Bezanson, 2008) 107 baccalaureate nursing students –

junior level Traditional lecture method vs Human

patient simulators Treating patients with acute myocardia

infarction

SELECTED RESEARCH: HEALTH CARE

Kao, L., & Thomas, E. (2008). "Navigating Towards Improved Surgical Safety Using Aviation-Based Strategies". Journal of Surgical Research, 134, 327-335.

Morey, J., Simon, R., Jay, G., Wears, R., Salisbury, M., Dukes, K., et al. (2005). "Error Reduction and Performance Improvement in the Emergency Department through Formal Teamwork Training: Evaluation Results of the MedTeams Project".

SELECTED RESEARCH: HEALTH CARE

Baker, C., Pulling, C., McGraw, R., Dagnone, J., Hopkins-Rosseel, D., & Medves, J. (2008). "Simulation in Interprofessional Education for Patient-Centered Collaborative Care". Journal of Advanced Nursing 372-379.

Davenport, D., Henderson, W., Mosca, C., Khuri, S., & Mentzer, R. (2007). "Risk-Adjusted Morbidity in Teaching Hospitals Correlates with Reported Levels of Communication and Collaboration on Surgical Teams but Not with Scale Measures of Teamwork Climate, Safety Climate, or Working Conditions". The Journal of the American College of Surgeons 205(6), 778-784.

Grogan, E., Stiles, R., France, D., Speroff, T., Morris, J., Nixon, B., et al. (2004). "The Impact of Aviation-Based Teamwork Training on the Attitudes of Health-Care Professionals". The Journal of the American College of Surgeons, 199(6), 843-849.

Guerlain, S., Turrentine, F., Bauer, D., Calland, J., & Adams, R. (2008). "Crew Resource management Training for Surgeons: Feasibility and Impact". Cognition, Technology & Work, 10, 255-264.

SELECTED RESEARCH: HEALTH CARE

Nishisaki, A., Keren, R., & Nadkarni, V. (2007). "Does Simulation Improve Patient Safety?: Self-Efficacy, Competence, Operational Performance, and Patient safety". Anesthesiology Clinics 25, 225-236.

Sundar, E., Sundar, S., Pawlowski, J., Blum, R., Feinstein, D., & Pratt, S. (2007). "Crew Resource Management and Team Training". Anesthesiology Clinics 25, 283-300.

Wetzel, C., Kneebone, R., Woloshynowych, M., Moorthy, K., & Darzi, A. (2006). "The Effects of Stress on Surgical Performance". The American Journal of Surgery 191(5-10), 5.

SELECTED RESEARCH: AVIATION

O'Connor, P., Campbell, J., Newon, J., Melton, J., Salas, E., & Wilson, K. A. (2008). "Crew Resource Management Training Effectiveness: A Meta-Analysis and Some Critical Needs". The International Journal of Aviation Psychology 18(4), 353-368.

Salas, E., Burke, C., Bowers, C., & Wilson, K. (2001). “Team Training in the Skies: Does Crew Resource Management (CRM) Training Work?”. Human Factors 43(4), 641-674.

Salas, E., Wilson, K., Burke, C., & Wightman, D. (2006). “Does Crew Resource Management Training Work? An Update, an Extension, and Some Critical Needs”, Human Factors 48(2), 392-412.