1

DOCUMENTING EVIDENCE OF EFFECTIVENESS IN YOUR APPROVED PROVIDER UNIT

FALL, 2014 WEBINAR SERIES

Educational Design Process

Ohio Nurses Association andMontana Nurses Association

(c) ONA, 2014

2

SERIES PLANNERS & FACULTY

Pam Dickerson, PhD, RN-BC, FAAN Director of Continuing Education, Montana Nurses

Association

Terry Pope, MS, RN President, Nursing Institute for Continuing Education

Zandra Ohri, MA, MS, RN Director of Continuing Education, Ohio Nurses Association

(c) ONA, 2014

3

SESSION 3

EDUCATIONAL DESIGN PROCESS: PROVIDING THE EVIDENCE

Objective: Identify evidence to validate the effectiveness of your provider unit in use of the educational design process for your learning activities.

(c) ONA, 2014

4

DISCLOSURES

Requirements for successful completion: Attendance at event & turning in an evaluation form.

Conflict of interest: Planners & faculty have declared no COI

The Ohio Nurses Association (OBN-001-91) is accredited as a provider of continuing nursing education by the American Nurses Credentialing Center’s Commission on Accreditation.

(c) ONA, 2014

5

OF NOTE

This webinar is designed to increase your effectiveness in documenting evidence of your work related to the EDP criteria – it is NOT intended to address the criteria themselves.

If you have specific questions about things like how to do a needs assessment, assess for conflict of interest, or negotiate a commercial support agreement, please contact your nurse peer review leader.

(c) ONA, 2014

6

(c) ONA 2014

7

OVERVIEW OF EDP CRITERIA DOMAINS

EDP 1 & 2: Assessment

EDP 3-6: Planning

EDP 7-11: Design Principles

EDP 12-13: Achievement of Objectives

(c) ONA, 2014

8

EDP 1 AND 2: ASSESSMENT OF LEARNING NEEDS

Criterion: CNE activities are developed in response to, and with consideration for, the unique educational needs of the target audience.

EDP 1 – Method of assessing learning needs of the target audience

EDP 2 – Use of data to develop a learning activity to address a gap in knowledge, skill, or practice

(c) ONA, 2014

9

EDP 3 – 6: PLANNING

Criterion: Planning for each educational activity must include one nurse planner and one other planner. One of the planners must have appropriate subject matter expertise for the educational activity.

EDP 3 – Process to select a planning committeeEDP 4 – Process to identify a conflict of interestEDP 5 – Process for resolution of a conflict of interestEDP 6 – Process used to determine requirements for

successful completion

(c) ONA, 2014

10

EDP 7 - 11: DESIGN PRINCIPLES

Criterion: The educational design process incorporates measurable educational objectives, best available evidence, and appropriate teaching methods.

EDP 7 – Developing measurable objectives related to change in practice or nursing professional development

EDP 8 – Selection of content based on best available evidence

EDP 9 – How content integrity is maintainedEDP 10 – Precautions with commercial support / sponsorshipEDP 11 – Selection of teaching methods to achieve purpose

and objectives(c) ONA, 2014

11

EDP 12 – 13: ACHIEVEMENT OF OBJECTIVES

Criterion: A clearly defined method that includes learner input is used to evaluate the effectiveness of each educational activity. Results from the activity evaluation are used to guide future activities.

EDP 12 – How summative evaluation data are used to guide future activities

EDP 13 – How evaluation data demonstrated change in nursing practice or nursing professional development

(c) ONA, 2014

12

WRITING TO THE CRITERIA

First – provide a description2-3 paragraphs, or about ½ pageClearly articulate your process, focusing on key words

Then – provide an exampleOne specific instance – the more detail, the betterFocus on how you implemented the process describedKeep internal consistency in mind – will your sample

activities support your description and example?

(c) ONA, 2014

13

EDP 1: ASSESSMENT OF LEARNING NEEDS

Nurse Planner’s methods of assessing the current learning needs of the target audience

Key Points

Methods used by the activity’s nurse planner

Current learning needs

Target audience for a specific learning activity

(c) ONA, 2014

14

EDP 1: THINGS TO CONSIDER

First: What is the opportunity for improvement or the problem in practice?

Second: What outcome is desired to address this issue?

Third: Who needs this information?

Fourth: Why do they need it? ** this is your needs assessment for an activity!

Be sure you aren’t doing “wants” or “likes” assessments!

(c) ONA, 2014

15

EDP 2: ASSESSMENT OF LEARNING NEEDS

How the Nurse Planner uses data collected to develop an educational activity that addresses the identified gap in knowledge, skills, and/or practice.

Key points:Use of dataGap analysisDevelopment of activity

(c) ONA, 2014

16

EDP 2: THINGS TO CONSIDER

What is a “gap analysis”?

Why does it matter?

When planning an activity, how does this process form the foundation for determining the desired outcome, then planning the activity?

(c) ONA, 2014

17

EDP 3: PLANNING

The process used to select a planning committee for an educational activity, including why an individual member was chosen.

Key points:Planning committeeSelection – process and a specific member

(c) ONA, 2014

18

EDP 3: THINGS TO CONSIDER

Planning committees should reflect relevance to the activity being planned – be cautious of “generic” planning committees

Remember internal congruence – if you state that speakers are always on the planning committee, reviewers will look for that in your sample activities

What is the role of the planning committee? Provide clear evidence.

(c) ONA, 2014

19

EDP 4: PLANNING

The process used to identify all actual and potential conflicts of interest for all members of the planning committee, presenters, authors, and content reviewers.

Key points:Use of a standard processKnowledge of conflict of interest (COI)Relevance for everyone who can control content

(c) ONA, 2014

20

EDP 4: THINGS TO CONSIDER

Clearly define COI – remember that it relates to people, not organizations

Focus on identification of the COI – how do you do that?

Don’t digress into other topics

(c) ONA, 2014

21

QUESTION

Is “completion of the BIO/COI form” adequate to identify conflict of interest?

Yes

No

(c) ONA, 2014

22

EDP 5: PLANNING

The process for resolution of an actual or potential conflict of interest and the outcome achieved.

Key points:Resolving a COI – what are the options?What is achieved when you exercise those options?

NOTE: If you have never had to resolve a COI, answer this by providing an “as if” example – not applicable is not acceptable!

(c) ONA, 2014

23

EDP 6: PLANNING

The process utilized during the planning phase of the educational activity to determine how participants will successfully complete the learning activity.

Key points:Part of planning processCriteria for successful completion – not disclosure, but

decision about what these will beWhy that decision was made

(c) ONA, 2014

24

EDP 7: DESIGN PRINCIPLES

How measurable educational objectives are developed that address the change in nursing practice or professional development.

Key points:Measurable objectives

What can be “measured”Avoid 2 or more verbs in one objective

Address change in practice or professional developmentRelevance to identified gap, purpose, and outcome measure (c) ONA,

2014

25

EDP 7: THINGS TO CONSIDER How do you develop objectives for your learning activities?

Who does this? If the presenter, how do the nurse planner and planning committee have input?

How do you assure that the objectives are tied to the gap data and desired outcome?

How do you assure that the objectives focus on the expectation for the learner, not the presenter?

How are the objectives related to the teaching methods?

How do the objectives relate to change in nursing practice or professional development?

(c) ONA, 2014

26

QUESTION ABOUT OBJECTIVES

Is it ok to give a blank planning table to a speaker to be filled out and returned to the nurse planner?

Yes

No

(c) ONA, 2014

27

WORKING WITH THE PLANNING TABLE

What’s the risk of the speaker completing this form independently?

How does the speaker know what’s been identified as the practice gap or desired outcome?

What if you get data back that isn’t “right”?

(c) ONA, 2014

28

EDP 8: DESIGN PRINCIPLES

How the content of the educational activity is selected based on best available current evidence

Key Points:Content selectionBased on “best available evidence”

(c) ONA, 2014

29

EDP 8 - EXAMPLE

The speaker completes the planning table with the objectives and content. The nurse planner reviews the content to be sure it is on target with the expected presentation and will help the learner achieve the stated outcome. The speaker also provides 3-5 references that are dated within the past 5 years. If references are not provided, or if they are out-of-date, the nurse planner will contact the speaker to get better references. (c) ONA, 2014

30

WHAT DO YOU THINK?

Is this a strong example to address EDP 8?

Yes

No

(c) ONA, 2014

31

THINGS TO THINK ABOUT Is it always appropriate for the speaker to do this

work independently?

What about content in relation to objectives and teaching methods?

What’s “magic” about 5 years?

Just because a reference is dated within the past 5 years, does that mean it’s the “best available evidence”?

What are credible sources of evidence? (c) ONA, 2014

32

EDP 9: DESIGN PRINCIPLES

How content integrity is maintained for CNE activities, including what precautions are taken to prevent bias and how those precautions are implemented

Key Points:Content integrityBias Implementing precautions

(c) ONA, 2014

33

WHAT IS “CONTENT INTEGRITY”?

Actual or potential conflicts of interest are resolved

Content is based on best on best available evidence

Independence from organization providing commercial support or sponsorship

Freedom from promotional activity

(c) ONA, 2014

34

EDP 10: DESIGN PRINCIPLES

In the presence of commercial support/sponsorship, how additional precautions are taken to maintain content integrity for CNE activities, including what precautions are taken to prevent bias and how those precautions are implemented.

Definition of commercial supportDefinition of sponsorship

Not applicable if you never have had and never expect to have commercial support

(c) ONA, 2014

35

EDP 10: THINGS TO THINK ABOUT

“Standard” commercial support / sponsorship agreement

What else do you need to consider?

What precautions do you take:Before an activityDuring an activity

Note: If you have received commercial support or sponsorship within the past 2 years, describe your process, provide an example, AND attach an activity file with the commercial support/sponsorship evidence.

(c) ONA, 2014

36

EDP 11: DESIGN PRINCIPLES

How teaching methods were selected that are appropriate to achieve the purpose and objectives of the CNE activity

Key Points:Selection of teaching methods for an activityRelationship of teaching methods to other components and

desired outcomeRelationship of teaching methods to criteria for successful

completion

(c) ONA, 2014

37

EDP 12: ACHIEVEMENT OF OBJECTIVES

How summative evaluation data for an educational activity were used to guide future activities.

Key points:Summative evaluationUse of data

(c) ONA, 2014

38

EDP 12: SUMMATIVE EVALUATION

Things to consider:

What evaluation methods are you using?

Are the methods themselves giving you helpful data?

Who compiles evaluation data?

Who analyzes evaluation data?

Is there a “post-action” report or planning committee meeting to analyze the outcome?

What do you do with the evidence from the summative evaluation to guide future activities?

(c) ONA, 2014

39

EDP 13: ACHIEVEMENT OF OBJECTIVES

How evaluation data were collected to measure change in nursing practice or nursing professional development

Key Points:Evaluation data useMeasuring changeNursing practice? Professional development?

(c) ONA, 2014

40

EDP 13: MEASURING OUTCOMES

Refer to initial gap analysis and needs assessment data

What was the original “problem in practice” or opportunity for improvement?

How do you know that issue has been addressed – or not?

(c) ONA, 2014

41

SUMMARY

The educational design process is the heart and soul of your provider unit.

Providing evidence of adherence to criteria demonstrates your commitment to quality continuing education to enhance nursing practice and professional development.

Coming next: Quality Outcomes

(c) ONA, 2014

42

FOR FURTHER INFORMATION

Pam Dickerson, PhD, RN-BC, FAANDirector, Continuing EducationMontana Nurses AssociationTelephone: 406-465-9126pam@mtnurses.org

Zandra Ohri, MA, MS, RNDirector, Continuing EducationOhio Nurses AssociationTelephone: 614-448-1027zohri@ohnurses.org

(c) ONA, 2014