program planning committees: tools for building …€¦ · – tools for patient management ......
TRANSCRIPT
Elizabeth A. Lindsay MS, PhD
Program Planning Committees: Tools for Building Capacity
4th National Accreditation Conference
Toronto, Ontario
September 21, 2012
Learning Objectives
After active engagement in this workshop, participants will be better able to:
• Identify recommended competencies for planning committee members
• Discuss planning committee roles and responsibilities with a focus on building capacity and meeting standards
• Identify and develop practice planning committee solutions for use in faculty development
Agenda
A. Background for the workshop – 20 min • Why are planning committees important? • Is there currently a problem with planning processes? • What do committee members need to be able to do – what is the scope of their
roles? • Overview of the “Gatekeeper Study” results B. Small group discussion – 20 min Objectives - to discuss specific competencies and tools directed at selected aspects
of the planning process for large or small group learning C. Full group discussion - 35 min • Presentation of examples of tools to support planning processes • Examples of tools recommended in small group discussion D. Evaluation/Feedback – 5 min.
What do they presently do?
Usual Practice for Planning One-day Programs
• 2-3 face-to- face meetings before event
• E-mail communication to finalize faculty, titles & objectives
• Occasionally, 1 debriefing meeting
Impact of Group Learning on Clinical Practices
• Multiple reviews show us the challenges of demonstrating impact of educational programs on competency and performance and there are pressures to improve the impact of our programs as well as to expand the type of activities we offer.
• However, group learning continues to be the predominant CME/CPD
activity for most physicians, as reported by the MainCert and MainPro programs in Canada
• Recent study of family physicians in the Ottawa region demonstrated that they are quite satisfied with the CME/CPD options available to them and believe that group learning does impact their clinical practices….little pressure to change.
• Expectations for the planning processes for accredited group learning and on-line learning are clearly defined – how well are we doing them? Those who evaluate applications for accreditation can advise us.
Seems appropriate that we try to improve impact
How?
Identify ingredients that can increase impact
Transfer this knowledge and skill to those who build or lead the programs
Step 1 Clarify target
audience and carry out
needs assessment process
Step 2 Identify subject area for
content and write learning
objectives
Step 3 Outline content
area to reflect learning
objectives and choose
format/methods/faculty
Step 4 Invite faculty and share
learning objectives,
recommendations for learning
formats
Step 5 Execute sessions
according to plan and evaluate
sessions, report to planning
committee
Step 6 Feedback and
other learning from past
meetings Synthesize
results
Steps for
Building
an
Accreditable
Program
What do we know about impact of group learning?
Group learning ( lectures, workshops, small groups, rounds) Effect size average - 6-10% * Key factors that determine impact - amount and type of interaction;
– less complex behaviours and more serious outcomes* Previously noted factors that enhance impact: needs based; relevance to practice reflection before, during and after meetings commitment to change at educational events combined with other interventions sequential repetition
*Forsetlund et al 2009
What do planning committees need to be able to do? From the AAMC/SACME Harrison survey*….
Toward a more effective product: Enhancing didactic activities • Practice enablers (patient material, flow-sheets, algorithms etc at or after course) • Follow-up method post-course to reinforce learning
Examples used in the survey • Pre-course planning ( needs assessment, objectives, choose methods, evaluate) • In-course enhancement
– Meaningful interactivity – Simulations – Tools for patient management
• Follow-up methods post-course to reinforce learning • Sequential learning
* Association of American Medical Colleges. Academic CME in the US and Canada: The 2010 AAMC/SACME Harrison survey p. 13
Basic Assumptions for the Gatekeeper Study
Investigators: Elizabeth Lindsay, Paul Hendry, Michael Allen, Heather Armson , Jatinder Takhar
• Health care providers who serve on planning committees and as presenters are the “gatekeepers” to improving effectiveness of group learning programs
• There is a relationship between program design, implementation processes and potential impact of group learning programs on participants
We Applied Behavior Change Frameworks* to Guide our Study Questions
Are physicians who serve on planning committees
Aware of best practices in program design?
Willing to apply new planning processes?
Do physicians who serve on planning committees believe that selected planning processes are:
Important?
Feasible?
Will make affect impact of the program?
*PRECEDE (Green) and Theory of Planned Behavior (Azjen)
Study Participants
• 52 physicians who had participated in planning committees at the University of Ottawa over an 2.5 yr period.
Respondents
• Family physicians - 17
• Other specialists - 15
Total 32/62%
Competency areas selected for study**
• Assessed information that describes the learning needs of potential participants in the program eg. gaps between current and recommended clinical practices
• Created learning objectives for the program that state what the learner will be able to do as a result of participating in the specific sessions.
• Recommended types of learning formats that work best for different types of objectives eg for knowledge based or skills based outcomes
• Discuss objectives and recommended learning formats with faculty/teachers to ensure mutual understanding
• Encourage faculty/teachers to present the level of evidence supporting educational messages (eg. Meta-analysis, RCT, expert opinion)
• Identify strategies that will facilitate practice integration of recommended best practices or address barriers to implementation
• Audit programs to assess existence of bias or compliance with accreditation standards
**Selected from Alliance for CME list of competencies for CME/CPD professionals
Study participants
Frequency
Scale 1-5
Willingness
Scale 1-5
Importance
Scale 1-5
Feasibility
Scale 1-5
Impact
Scale 1-5
Mean Responses
4 or 5
Mean Responses
4 or 5
Mean Responses
4 or 5
Mean Responses
4 or 5
Mean Responses
4 or 5
Assess needs
4.0 78% 4.2 81% 4.9 100% 4.1 77% 4.6 90%
Create learning objectives
4.2 78% 4.4 81% 4.6 100% 4.4 90% 4.2 87%
Recommend
learning formats
4.2 81% 4.3 81% 4.8 99% 4.4 90% 4.5 93%
Discuss obj &form with presenters
3.4 56% 4.1 78% 4.6 97% 4.1 80% 4.3 87%
Encourage careful data presentation
2.9 31% 4.1 75% 4.5 87% 4.1 70% 3.8 58%
Identify integration strategies
3.3 41% 4.1 78% 4.5 90% 3.9 69% 4.1 76%
Audit for bias and standards
2.0 16% 3.5 53% 4.3 90% 3.6 54% 3.4 43%
Conclusions
• High levels of willingness to improve
• Strong belief in the importance of all factors but
• Not as strong regarding belief of potential impact
• Lots of variability regarding feasibility
Purpose and process of small group discussion
Focus on one element of the planning process
Considering our expectations regarding planning committees – what can we do to support them?
Describe and create a list of specific processes and tools you use or think we should use to support planning committees
Topics for small group discussion
Composition of the Planning Committee – The Chair and Preparation
Composition of the Planning Committee – The Committee Members
Competencies and Pre-work
Structure and Process of Planning Meetings – During meetings
Structure and Process of Planning Meetings – Between Meetings
Choose effective presenters and helping them prepare
Choose educational strategies that go beyond didactic presentations with
Q&A
Full Group Discussion
With the package of resources as a basis --- add the ideas raised by each of the small groups
Topics for small group discussion
Composition of the Planning Committee – The Chair and Preparation
Qualities of a high functioning chair
Preparation Ideas – evidence for importance of improving impact
Composition of the Planning Committee – The Committee Members
Competencies and Pre-work Content Experts
Audience Representatives
Education Designer
Event Planner
Topics for small group discussion
Structure and Process of Planning Meetings – During meetings
Agenda should reflect time for different steps in the process in order to set up expectations for the meeting process eg. Time for looking at needs assessment information
Use a planning template that can be completed during discussion at the meeting, columns for Gap and Objective, Title, Presenters – event planner can do some of this with help from the chair or another committee member
Structure and Process of Planning Meetings – Between Meetings
The event planner, education designer and chair can work on planning template between meetings to ensure the process moves forward in a timely manner
Individual members of the committee may have tasks that they have agreed to complete eg. Check on availability of presenter or find a particular resource – ideally meeting planner should follow-up to gather this information and apply it to the planning template.
Topics for small group discussion
Choose effective presenters and helping them prepare
Presenters strengths – ideas for how to find them – Knowledgeable and creditable regarding content
– Demonstrates good understanding of the clinical setting in which learner works and the challenges faced there
– Clarity regarding content and presentation format
– Sufficient interaction to ensure understanding eg. Able to answer questions appropriately
– Create template letters of invitation to potential presenters to ensure draft objectives and title are included and invite feedback
Choose educational strategies that go beyond didactic presentations with
Q&A
Match objective with learning format – see Michie
What makes a workshop, a workshop?
Improving interaction – Tip sheets
Resources
• Strauss S, Tetroe, J, Graham ID. Knowledge Translation in Health Care: Moving from evidence to practice. 2009. Wiley-Blackwell. Oxford.
• Green LW, & Kreuter MW. Health Program Planning: An educational and ecological.
4th edition 2005. McGraw Hill. Toronto.
• Ajzen I. The theory of planned behavior. Organizational Behavior and Human Decision Processes, 1991;50:179-211
• Michie S, Johnson M, Francis J, Hardeman W, Eccles M. From theory to intervention: Mapping theoretically derived behavioural determinants to behaviour change techniques. Applied Psychology:An International Review, 2008, 57(4):660-680.
• Michie S, Johnson M, Abrahm C, Lawton R. Parker D, Walker A, on behalf of the “Psychological Theory” Group. Making psychological theory useful for implementing evidence based practice: a consensus approach. Qual Saf Health Care 2005;14:26-33.
• Moore, DE. How physicians learn and how to design learning experiences for them: An approach based on an interpretive review of evidence. In:Hager, M, Russell S & Fletcher SW (Eds.). Continuing education in the health professions: Proceedings of a conference. A Report for the Josiah Macy Foundation, 2008; http://www.josiahmacyfoundation.org/index.
Contents of Handout
1. Title page - University of Ottawa Office of Continuing Medical Education (OCME)
Handbook for Planning Committees A How‐to‐Guide for the development of an education program that may be accredited for Family Physicians (MainPro 1) and Specialists (MOC)
2. Examples of slides for discussion of KT, evidence of impact of group learning.
3. Excel planning template for listing of topics, gaps and objectives
4. Tip Sheets – Example – Interactivity
5. Invitation template to participate on a planning committee, including roles, responsibilities, time commitment, timelines and expectations regarding response to communication.
6. K to A framework
7. Michie matching of objectives and methods