the quest for quality: monitoring and measuring outcomes in continuing nursing education session 2:...

1

The Quest for Quality:Monitoring and Measuring Outcomes in Continuing Nursing Education

Session 2: Selecting Provider Unit Outcomes Measures

Pamela S. Dickerson, PhD, RN-BC, FAANDirector of Continuing Education, Montana Nurses Association

President, PRN Continuing Education

Presented by Ohio Nurses Association and Montana Nurses Association

2

Disclosures• Purpose: Participants will effectively develop

learning and provider unit processes that lead to measurable outcomes demonstrating quality patient care and/or enhanced professional development of registered nurses.

• Criteria for Successful Completion: Participate in the entire webinar. Complete the evaluation and registration information and return to ONA as directed.

• Conflict of Interest: There is no conflict of interest for any planner or presenter for this activity.

3

Objective• Determine appropriate outcome measures to

guide the work of your approved provider unit.

4

Self-AssessmentI know how to Strong

= 3Moderate = 2

Weak = 1

Set goals for my provider unit

Develop a quality outcome measure

Assess learning activities in relation to provider unit goals

Evaluate quality outcome measures as a way to determine provider unit effectiveness

Total Score =

5

• My total score was closest to a:

• 4• 7• 9• 12

6

7

Our Ultimate Goal• Reflected in ANA/NNSDO Nursing Professional

Development Scope and Standards of Practice Model

• Reflected in ANCC Accreditation Program Model

8

© 2010 ANA & NNSDO – Used with permission

9

Accreditation Conceptual Framework, ANCC 2013 (used

with permission)

Nursing Profession

al Developme

nt: Continuing Nursing Education

Educational Design Process• Assessment of Learning Needs • Planning and Faculty• Design Principles • Achievement of Objectives

Educational Design Process• Assessment of Learning Needs • Planning and Faculty• Design Principles • Achievement of Objectives

Quality Outcomes• Evaluation Process• Evaluation Participation• Goals for Improvement• Value/Benefit to Nursing Professional Development

Quality Outcomes• Evaluation Process• Evaluation Participation• Goals for Improvement• Value/Benefit to Nursing Professional Development

Structural Capacity

• Commitment• Accountability• Leadership• Resources

Structural Capacity

• Commitment• Accountability• Leadership• Resources

10

What Is A Quality Measure?

• “Quality measures are metrics, indicators or ratings which provide some insight into the quality of care that patients are receiving in a particular setting or place”. - Pennsylvania Health Care Quality Alliance

• Following that line of thought, quality measures in continuing education are metrics that provide some insight into the quality of the work of our provider units that contributes to quality patient care and nursing professional development.

11

Quality Measures Abound

• Common examples of quality measures:

o Your most recent performance appraisal

o Amount of money in your checking account

o Number of dust bunnies under your bed

o Amount of gas in your car

o Your child’s PSAT score

o Amount of chocolate in your house

12

Examples of Outcomes Measures:Health Care Organizations

• Morbidity

• Mortality

• Healthcare Facility-Acquired Infections

• Readmission Rates

• Patient Satisfaction (HCAHPS)

• Quality of Life

13

Examples of Outcomes Measures:Nursing (NDNQI)

Clinical• Falls• Symptoms, e.g. pain• Central line infections• Catheter-associated

UTIs• Ventilator associated

pneumonia rate• Pressure ulcers• Other…

Professional• Nurse hours per

patient day• Nursing skill mix• RN education and

certification• Retention• Work environment• Other…

14

Measuring Outcomes

• Outcome: “The impact of structure and process on the organization…and the value/benefit to nursing professional development.” (ANCC, 2013).

• Outcome Measurement: “The process of observing, describing, and quantifying predefined indicators of outcomes of performance.” (ANA/NNSDO, 2010)

15

Quality Outcomes Measure

• A measurable component of the structure or process of a provider that demonstrates the effectiveness of the provider unit.

• Examples:o Number of nurse planners (PU outcome measure)o Number of learning activities that include active

learner participation (PU outcome measure)o Patient care outcome related to educational

content (professional development/impact on care outcome measure)

16

Relationship of Quality Outcome Measures to Goals

• Goal: What the provider unit will accomplish in the year

• Outcome Measure: What evidence will be collected to determine goal achievement – how will we know if we’re successful?

17

Example• PU Goal: Improve the diversity of learning activities

available to nurses in our target market area.

• Outcome measures might include:o Number of “live” activities compared to previous

yearo Number of web-based activities compared to

previous yearo Number of interprofessional activities compared to

previous yearo Number of published independent studies compared

to previous year

18

Comparison with a Learning Activity

• For a learning activity, the goal might be that learners will provide more person-centered care to patients with dementia.

• Measurable outcomes might be:o Observation of behaviors o Report of family memberso Decreased number of care-related complaints o Calmer behavior of patients

19

Evaluation in Context: ActivityTarget

AudienceNeeds

Assessment

Gap Analysis

Purpose

Objectives

Content/ Teaching Methods

Short-term

Evaluation

Outcome Long term

evaluation)

20

Name of Session: Facilitator:

Date: Location:

Attendance (to be inserted for summative) Contact Hours:

Please indicate your achievement of each objective:

Yes No

1. ___ ___

2. ___ ___

Please provide your response to each item:

The facilitator was effective in teaching the subject. ___ ___

This session provided a good way for me to

learn this content. ___ ___

What did you learn today that will strengthen your practice or personal behavior?

What one change will you make in your practice as a result of participating in this activity?

Generic Evaluation

21

Generic Evaluation (cont’d)

What barriers do you think will make it difficult to implement what you have learned?

What needs to happen to overcome these barriers?

What more would you like to learn about this topic?

Comments:

Suggestions for future topics (please indicate what you would like to know about and WHY this is important in your practice):

22

Evaluation in Context: Provider UnitGoals for Provider

Unit

Outcome Measures

Provider Unit Structure &

Function

Learning Activities

Activity Evaluation

Outcome Measurement for

Provider Unit

PSD, 2013

23

Building Blocks

• Activity outcomes are specific to one learning activity. (EDP 13)

• Activities should be designed with provider unit goals in mind.

• Outcomes measures allow you to look at data in the aggregate to determine if provider unit goals are met. (QO 4, 5)

24

Your Turn• Write a goal for your provider unit for 2014.

25

Your Turn• Based on the goal you wrote, write the outcome

measure or measures that will provide evidence of achieving that goal (or not).

26

Provider Outcomes Measures: Ideas

• Cost-effectiveness of PU operations (may include return on investment)

• Turnover of nurse planners• Number of educational activities provided• Types of educational activities offered• Quality of learning activities

o Learner perceptiono Faculty feedbacko Effective in filling the identified gap

27

Example

• Provider unit goal for 2013: Increase the number of nurse planners so that there is a nurse planner for every clinical area.

• Outcome measures:o Number of nurse plannerso Representation of clinical areas among nurse

planners

28

What Was The Outcome?

• December, 2012 – 6 nurse planners for hospital with 1400 nurse employees; ED, critical care areas, OR/PACU, and palliative care did not have nurse planner representation

• January, 2014 – 10 nurse planners plus primary nurse planner – all clinical areas have nurse planner representation

29

Outcomes - Visually

Nurse Planners Clinical Areas Represented0

2

4

6

8

10

12

14

20122014

30

Professional Development Outcomes Measures: Ideas

• Leadership skills• Healthy work environment• Healthy lifestyle / role modeling• Critical thinking skills• Nurse advocacy

o For patientso For the professiono For the community – health promotion

31

Goal Example• Provide quality continuing education to all of our

nurses.

• Is this an effective goal statement?o Yeso No

32

33

Goal Example• Provide quality continuing education

• This is very broad – what is quality? What kind of measurement could I use to evaluate it? What does success look like?

• Commonly used: Maintain approved provider status – so what?

• Hint: If you can’t think of an outcome measure for it, it probably isn’t a strong PU goal!

34

Goal Example• Provide targeted continuing education to hospice

nurses in 2014 based on gap analysis data to increase patient’s perceptions of quality of care at the end of life by the end of the year.

• Is this an effective goal statement?o Yeso No

35

36

Why Is This Goal Stronger? • Provide targeted continuing education to hospice

nurses in 2014 based on gap analysis data to increase patient’s perceptions of quality of care at the end of life by the end of the year.

• Outcome is clear• Outcome measures can be selected:

o Number of hospice nurses engaged (PU)o Number of activities based on gap analysis

data (PU)o Patient perceptions of QOL (NPD)

37

Revisit Your Goal• Is the outcome clear? Will you know when

success has been achieved?

• Are there clearly evident outcome measures? Do you know what data you will collect to demonstrate success?

• Are your outcomes related too The work of your provider unit? (QO 4)o Professional development of nurse learners? (QO 5)

38

Our Accountability in Nursing Professional Development

• Standard 3 (Outcomes Identification):

o Develops outcomes that reflect professional role competence, learning and change

o Documents outcomes, including those that demonstrate learning and program impact.

ANA/NNSDO, 2010

39

Our Accountability in Nursing Professional

Development

• Standard 6 (Evaluation):

o Selects valid, reliable, and relevant methods and instruments to measure processes and outcomes.

o Synthesizes evaluation data, trends, and expectations to guide decision-making about changes and improvement of all components of NPD practice

ANA/NNSDO, 2010

40

Our Accountability in Nursing Professional Development

• Standard 16 (Leadership):

o Influences decision-making bodies to maintain and improve quality nursing and professional development programs

o Promotes the professional development program mission, goals, action plans, and outcome measures.

ANA/NNSDO, 2010

41

Self-AssessmentI know how to Strong

= 3Moderate = 2

Weak = 1

Set goals for my provider unit

Develop an outcome measure

Assess learning activities in relation to provider unit goals

Evaluate outcome measures as a way to determine provider unit effectiveness

Your Score:

42

• My total score was closest to a:

• 4• 7• 9• 12

43

44

Summary• Outcomes measures for your provider unit help

you to define and measure goal achievement.

• Quantifying and sharing of outcomes data is expected as part of our role as nursing professional development specialists.

45

Questions and Discussion

46

For Additional Information

Pam Dickerson, PhD, RN-BC, FAAN

pdickerson@aol.compam@mtnurses.org

Zandra Ohri, MA, MS, RN

zohri@ohnurses.org

47

References• American Nurses Association & National Nursing Staff

Development Organization (2010). Nursing Professional Development: Scope and Standards of Practice. Silver Spring, MD. Nursesbooks.org

• American Nurses Credentialing Center. (2013). 2013 ANCC Primary Accreditation Application Manual for Providers and Approvers. Revision 3.0. Silver Spring, MD. Author.

• National Database for Nursing Quality Indicators. (2013). Data. Retrieved from https://www.nursingquality.org/data.aspx

• Pennsylvania Health Care Quality Alliance. (2013). Our Measures. Retrieved from http://www.phcqa.org/measures/