From “Learning” Objectives to Outcomes

Joanie Selman, MSN, RNStephen F. Austin State University School of Nursing

Objectives of Session

1. Discuss core components to a healthcare simulation.

2. Discuss the importance of outcomes evaluation and challenges to traditional assessments.

3. Discuss the importance of validity, reliability and feasibility as it relates to assessment.

4. Discuss types of assessments and their application in healthcare education.

DeWitt School of Nursing at Stephen F. Austin State

UniversityNacogdoches, Texas

Designated as a Laerdal Center of Educational Excellence

Physical Space

9,000 square foot simulation lab opened Jan. 2010*10 room Med-Surg lab*4 room OB/pedi lab and Nursery*ER with 12 gurney spaces*Assessment lab with 10 exam tables*14-station control room

Equipment5-3Gs1-SimMan1-SimNewB2-SimBaby17 VitalSims6-Virtual IVsComing soon- SimMom, SimKid

How do I start this?Start simple. Do not overwhelm yourself.

Use the Nursing Process !1. Assessment2. Planning3. Intervention4. Evaluation

Assessment• Assess the learning needs of your students to

build scenario objectives. – Start broad and narrow it down.

• Look at ….. – Common disease/complication from your area.

• Choose problem that is it crucial for students know how to handle.

– School’s objectives– Course objectives– Unit objectives– Objectives related to specific disease process

Example-Assessment phase• Med-surg area• Chest Pain

• Very Common chief complaint of clients• Students must learn interventions to care for

this client• Not enough clinical time for every student to

experience a client with chest pain.

Planning• Plan the objectives for the scenario so that you

can achieve the goal for your students.• Based on objectives of

– The School– The Course– The Unit– What do I want for the students to accomplish

(do) in this scenario? Remember Students learn by DOING !

Example-Planning phase for chest pain (acute coronary syndrome) scenario

• School of nursing outcome objectives– Graduates of the program will:

• Apply knowledge of the physical, social, and behavioral sciences in the provision of nursing care based on theory and evidence based practice.

• Course objectives– Students will:

• Apply concepts from nursing, scientific, and humanistic disciplines to the care of an adult client in medical-surgical situations.

• Unit objectives– Students will:

• Apply knowledge of pathophysiology of cardiovascular diseases to the plan of care for clients with cardiovascular disease, and vascular disorders.

• Formulate an appropriate nursing plan of care for clients with selected cardiovascular Disease, and vascular disorders.

Scenario objectives for chest pain(Acute coronary syndrome)

Specific to this scenario:• What do I want the students to learn by “doing” ? • What interventions do I want to “see” the students

accomplish?• What are best-practices that the students should

demonstrate?• What do I want to see in terms of prioritization and critical thinking?

Example –ACS scenarioStudent learning objectives

• Demonstrates infection control practices.• Identifies signs and symptoms of ACS by proper assessment• Identifies primary nursing dx• Prioritizes and implements Physician Orders appropriately and

safely by applying cardiac chest pain protocol• Initiates relevant cardiac and respiratory monitoring• Implements the "5 rights" of medication administration• Recognizes signs and symptoms of adverse reaction to

nitroglycerin tablets• Implements therapeutic communication• Demonstrate teaching techniques to help clients

understand lifestyle choices that effect health.

Assessment tool

• From the scenario objectives, make an assessment tool in the form of a “check-off” list for instructors and student observers to use during simulation.

Example- ACS scenario assessment

tool

InterventionsWriting the SIM

Ready, Set, Go

• You have your objectives• You know what you want to see• You know what you want the students to do

You’re ready to write the scenario

Writing the scenario

-- Remember --Objectives guide everything

– Mannequin actions– Vital signs, set-up– “Patient” voice– MD orders– Medications– Chart contents– Available supplies– Pre-Sim activities

Components of a Simulation

You’re finished planning and writing

NOW WHAT ???• Pre-Sim Activities/Assignment • Running the Sim• Debriefing

Researchers suggest that equal time be spent on each.

Pre-SIM activity Lecture on the topic

Help students prepare Assignment to complete

on topic before the SIM

article in a journal case study

“dress rehearsal” with other students or faculty

Running the SIM Keep it real. Room, mannequin, pt voice Pre-brief students

act exactly like a real life setting give them report – like at a hospital

Give a few minutes after report for them to plan as a team before starting scenario

As mannequin voice give “hints” if student is really struggling.Give student “Observer” check-list

De-brief immediately

Faculty facilitate the discussion. Guide students in reflective learning.

Have observer use assessment tool. Let observer identify positives as well as negatives

of the simulation activity. Constructive feedback is important.

Redirect students away from poor choices. Praise students for positive choices.

Group discusses application of scenario to clinical practice

Evaluation of Outcomes• Did the students meet the objectives?• Did this teaching strategy work?• How do I assess their learning?

Faculty Evaluation of students learning Traditional way of evaluation is an exam.Simulation necessitates non-traditional evaluation.

Evaluation of Outcomes

Use the assessment tool of the instructor and the observer as a tool to measure the student’s accomplishment of the objectives of the SIM.

Evaluation of Outcomes

The assessment tool has internal validity…. it was created from the scenario objectives,

therefore it is measuring what you intend to measure

The assessment tool has external validity…it is based on real world interventions and

expectations of nurses in specific medical occurrences.

Evaluation of Outcomes

• The assessment tool has reliability.– It is consistent

• From group to group• From observer to observer• From instructor to observer

Constant, Repeatable and Objective Evaluation

Evaluation of Outcomes

Assessment tool items can be analyzed and data generated for a quantitative evaluation “view” • For example:

– 90 % introduced themselves to the patient– 92 % instructed patient to take NTG under tongue– 85% assessed blood pressure before giving NTG

Beneficial to instructor to identify gaps in learning

Evaluation of Outcomes• Evaluation of Student’s perception of the

scenario and learning experience.– Use surveys for the students to complete soon

after the SIM.– Short, concise and easy to answer.

Student evaluation of SIMQuestion Disagree No opinion Agree

1. I understand the purpose and objectives of the simulation. 0.00% 0.00% 100.00%

2. The scenario(s) resembled a real-life situation. 1.33% 0.00% 98.67%

3. The simulation provided a variety of ways to learn the material. 0.00% 5.33% 94.67%

4. I had a chance to work with my peers during the simulation. 0.00% 5.33% 94.67%

5. The simulation helped me learn to prioritize. 0.00% 1.33% 98.67%

6. The debriefing session was helpful as a learning activity. 1.33% 0.00% 98.67%

7. I was able to participate in the debriefing session. 1.33% 1.33% 97.33%

8. The feedback was constructive. 0.00% 2.67% 97.33%

9. The simulation will help me better care for healthcare clients. 0.00% 0.00% 100.00%

10. The simulation made me feel more confident in caring for healthcare clients. 2.67% 4.00% 93.33%

Student Evaluation• Have a free text comment section on the evaluation. Actual student comments…….

“I really enjoy SIMs and I always learn a lot from them. I wish we had more and I will continue to learn and feel more comfortable when entering a patient's room.”

“I always learn things that I know I won’t forget. It is good to have these simulations because we don't always come across some of these situations when we are actually in clinicals. Because of these simulations, I know i will be prepared when I come across the same problems.”

“The feedback in the debriefing session was very informative without being condemning. Good job!”

FeasibilityReasonable to use SIMs?

Cons: Mannequins costly Labor intensive for faculty Instructor training needed Space needed

Pros: Helps with shortage of clinical opportunities Reproducibility of clinical experience Great teaching tool for Tactile/Kinesthetic learners Opportunity for students to gain confidence Encourages teamwork Prepares students for the workforce

FeasibilityReasonable to use SIMs?

With current expectations of students, school administrators and employers, Feasible or not…Simulations are the future and are needed to help fulfill these expectations.

Simulation

A powerful teaching tool

It’s worth the trouble

IT’S WORTH IT !!

– To the instructor

– To the students

– To the future employer

– To the client

questions

Thank You So much !

Contact informationJoanie Selman, MSN, RN

selmand@sfasu.edu936-674-7896

DeWitt School of NursingStephen F. Austin State University

Nacogdoches, Texas