![Page 1: Transform your clinical competency committee with learner ... · Transform your clinical competency committee with learner coaching Matthew Kelleher, MD, MEd (@ ... Warm EJ, Mathis](https://reader035.vdocuments.net/reader035/viewer/2022070706/5e9b60130083f77cb2119c02/html5/thumbnails/1.jpg)
Transform your clinical competency committee with learner coaching
Matthew Kelleher, MD, MEd (@kelzj3)
Benjamin Kinnear, MD (@Midwest_MedPeds)
Eric Warm, MD (@CincyIM)
Date: 10/21/17
![Page 2: Transform your clinical competency committee with learner ... · Transform your clinical competency committee with learner coaching Matthew Kelleher, MD, MEd (@ ... Warm EJ, Mathis](https://reader035.vdocuments.net/reader035/viewer/2022070706/5e9b60130083f77cb2119c02/html5/thumbnails/2.jpg)
I do not have an affiliation (financial or otherwise) with a
pharmaceutical, medical device or communications organization.
Je n’ai aucune affiliation (financière ou autre) avec une
entreprise pharmaceutique, un fabricant d’appareils
médicaux ou un cabinet de communication.
2
![Page 3: Transform your clinical competency committee with learner ... · Transform your clinical competency committee with learner coaching Matthew Kelleher, MD, MEd (@ ... Warm EJ, Mathis](https://reader035.vdocuments.net/reader035/viewer/2022070706/5e9b60130083f77cb2119c02/html5/thumbnails/3.jpg)
Learning Objectives
By the end of this workshop, we hope you are able to:
1. Evaluate your own competency committee as operating under
a problem identification model or a developmental model.
2. Use coaching techniques with your learners.
3. Use your competency committee to guide coaching strategies
for learner improvement.
3
![Page 4: Transform your clinical competency committee with learner ... · Transform your clinical competency committee with learner coaching Matthew Kelleher, MD, MEd (@ ... Warm EJ, Mathis](https://reader035.vdocuments.net/reader035/viewer/2022070706/5e9b60130083f77cb2119c02/html5/thumbnails/4.jpg)
Think-pair-share
What does your CCC do to help residents:
• Improve
• Learn
• Grow
4
![Page 5: Transform your clinical competency committee with learner ... · Transform your clinical competency committee with learner coaching Matthew Kelleher, MD, MEd (@ ... Warm EJ, Mathis](https://reader035.vdocuments.net/reader035/viewer/2022070706/5e9b60130083f77cb2119c02/html5/thumbnails/5.jpg)
Observable Practice Activities (OPAs)
Process-Based (POPA)
Mid/End of Rotation Feedback (formative)
Entrustment (1-5)
Content-Based (COPA)
Reporting Milestones
External Reporting Longitudinal Assessment Formative > Summative
Mapping Mapping
Warm EJ, Mathis BR, Held JD, Pai S, Tolentino J, Ashbrook L, Lee CK, Lee D, Wood S, Fichtenbaum CJ, Schauer D, Munyon R, Mueller C. Entrustment and Mapping of Observable Practice Activities for Resident Assessment. J Gen Intern Med. 2014 Feb 21. [Epub ahead of print] PubMed PMID: 24557518
Average of 3,987 milestone assessments, 4325 narrative words per resident!!!!
![Page 6: Transform your clinical competency committee with learner ... · Transform your clinical competency committee with learner coaching Matthew Kelleher, MD, MEd (@ ... Warm EJ, Mathis](https://reader035.vdocuments.net/reader035/viewer/2022070706/5e9b60130083f77cb2119c02/html5/thumbnails/6.jpg)
6
Our CCC
Mountain of milestone data
![Page 7: Transform your clinical competency committee with learner ... · Transform your clinical competency committee with learner coaching Matthew Kelleher, MD, MEd (@ ... Warm EJ, Mathis](https://reader035.vdocuments.net/reader035/viewer/2022070706/5e9b60130083f77cb2119c02/html5/thumbnails/7.jpg)
Month of Residency
AggregateEntrustment
Blue = Resident Performance
Red = Predicted Score
N= 3320Number of
Assessments
• Trends from start of residency to present• Comparison to the predicted score
![Page 8: Transform your clinical competency committee with learner ... · Transform your clinical competency committee with learner coaching Matthew Kelleher, MD, MEd (@ ... Warm EJ, Mathis](https://reader035.vdocuments.net/reader035/viewer/2022070706/5e9b60130083f77cb2119c02/html5/thumbnails/8.jpg)
Blue = Resident Performance
Red = Predicted Score
Milestones Assessed in past 6
months
• Snapshot of the past 6 months• Gives us an idea of what we will be reporting
to ACGME and ABIM
EntrustmentScale
![Page 9: Transform your clinical competency committee with learner ... · Transform your clinical competency committee with learner coaching Matthew Kelleher, MD, MEd (@ ... Warm EJ, Mathis](https://reader035.vdocuments.net/reader035/viewer/2022070706/5e9b60130083f77cb2119c02/html5/thumbnails/9.jpg)
• Z-score = number of standard deviations Observed (blue line) is from Predicted (red Line)
Blue = Resident Performance
Red = Predicted Score
![Page 10: Transform your clinical competency committee with learner ... · Transform your clinical competency committee with learner coaching Matthew Kelleher, MD, MEd (@ ... Warm EJ, Mathis](https://reader035.vdocuments.net/reader035/viewer/2022070706/5e9b60130083f77cb2119c02/html5/thumbnails/10.jpg)
X-Bar Chart
SD Chart
"AUC"= 3.06
Z-Score (plus 3)
Standard Deviation within
the Subgroup (Competencies)
“Area under the curve”
![Page 11: Transform your clinical competency committee with learner ... · Transform your clinical competency committee with learner coaching Matthew Kelleher, MD, MEd (@ ... Warm EJ, Mathis](https://reader035.vdocuments.net/reader035/viewer/2022070706/5e9b60130083f77cb2119c02/html5/thumbnails/11.jpg)
3rd Year Resident at 36 Months
N= N= "AUC"=2470 815 3.06
![Page 12: Transform your clinical competency committee with learner ... · Transform your clinical competency committee with learner coaching Matthew Kelleher, MD, MEd (@ ... Warm EJ, Mathis](https://reader035.vdocuments.net/reader035/viewer/2022070706/5e9b60130083f77cb2119c02/html5/thumbnails/12.jpg)
There’s more!!!
12
1
6
11
16
21
26360 Patient Care
360 Teamwork360 Professionalism
360 Efficiency
Patient Evaluations
Testing Overall
DM Overall
Diabetes Process
Diabetes Outcome
Diabetes Change
Prevention Quality
Prevention Change
Absolute Work UnitsWork Intensity
CitizenshipSelf Professionalism
Self Patient Care
Self Patient Care
Self Systems Based…
Self Systems Based…
Self Systems Based…
Self Systems Based…
Self Knowledge
Self Communication
Self Efficiency
Self Professionalism
Self TeamworkSelf Patient Care
1
6
11
16
21
26360 Patient Care
360 Teamwork
360 Professionalism
360 Efficiency
360 Overall
Patient Evaluations
Testing Overall
DM Quality
DM Change
Prevention Quality
Prevention Change
Absolute Work Units
Work Intensity
Citizenship
![Page 13: Transform your clinical competency committee with learner ... · Transform your clinical competency committee with learner coaching Matthew Kelleher, MD, MEd (@ ... Warm EJ, Mathis](https://reader035.vdocuments.net/reader035/viewer/2022070706/5e9b60130083f77cb2119c02/html5/thumbnails/13.jpg)
AND MORE!!!!!!
13
Patient Evaluations Your Score Your Rank Class Average Stand. Dev Range
Number of Evaluations Total 22.00 14 21.77 6.79 8.00-37.00
Average Visits per Patient 2.36 22 2.65 0.31 2.17-3.41
Physician Explains 5.59 19 5.66 0.17 5.30-6.00
Physician Listens 5.36 24 5.68 0.19 5.30-5.94
Physician Gives Instructions 5.59 21 5.73 0.18 5.30-6.00
Physician Knows History 5.59 15 5.59 0.17 5.00-5.82
Physician Respects Patient 5.52 23 5.76 0.17 5.39-6.00
Physician Spends Enough Time 5.45 21 5.62 0.20 5.17-5.94
Physician Calls With Results 5.17 18 5.30 0.33 4.32-5.78
*Rate on Scale of 0-10 9.14 19 9.23 0.34 8.30-9.79
Would Recommend To Others (scale 1-4) 3.90 9 3.85 0.09 3.63-4.00
Average/Overall Rank 22
Testing Score Rank Average Stand. Dev Range
ITE1 73 3 63.13 6.92 48-77
ITE2 80 4 68.00 9.09 45-84
GIM 1 73 2 59.00 9.68 38-76
Cardiology 62 16 66.00 7.06 44-76
Rheumatology 71 4 62.00 8.74 44-76
GI Liver 80 4 70.00 9.75 51-89
Endocrinology 57.00 10.00 40-78
Mathis Testing Overall 72 3 62.00 6.00 54-77
Number Average Stand. Dev Range
Number of notes closed > 24 hours 33 32.69 33.86 4 - 136
Number of notes closed > 7 days 0 2.88 6.24 0-26
Number of refills completed in 48 hours 90% 99% 3.91% 90%-107%
Number of completed patient evaluations 22 21.77 6.79 13728
Percentage of eligble AAP Attendance 100% 96% 10% 60%-100%
Citzenship Rank 15
Patient assessments!
Knowledge! Citizenship!
![Page 14: Transform your clinical competency committee with learner ... · Transform your clinical competency committee with learner coaching Matthew Kelleher, MD, MEd (@ ... Warm EJ, Mathis](https://reader035.vdocuments.net/reader035/viewer/2022070706/5e9b60130083f77cb2119c02/html5/thumbnails/14.jpg)
Clinical Competency Committee
CollectPre-
reviewReview Deep dive
Reporting
Feedback
![Page 15: Transform your clinical competency committee with learner ... · Transform your clinical competency committee with learner coaching Matthew Kelleher, MD, MEd (@ ... Warm EJ, Mathis](https://reader035.vdocuments.net/reader035/viewer/2022070706/5e9b60130083f77cb2119c02/html5/thumbnails/15.jpg)
15
If dashboard looks good.No red flags in narrative.
If underperforming on dashboard.If red flags in narrative.
![Page 16: Transform your clinical competency committee with learner ... · Transform your clinical competency committee with learner coaching Matthew Kelleher, MD, MEd (@ ... Warm EJ, Mathis](https://reader035.vdocuments.net/reader035/viewer/2022070706/5e9b60130083f77cb2119c02/html5/thumbnails/16.jpg)
Assessment ecosystem
Direct observation
Entrustment/OPAs Milestones data
CCC
ACGME
Semi-annual PD meeting
![Page 17: Transform your clinical competency committee with learner ... · Transform your clinical competency committee with learner coaching Matthew Kelleher, MD, MEd (@ ... Warm EJ, Mathis](https://reader035.vdocuments.net/reader035/viewer/2022070706/5e9b60130083f77cb2119c02/html5/thumbnails/17.jpg)
Two CCC paradigms
17Hauer KE, Chesluk B, Iobst W, et al. Reviewing Residents' Competence: A Qualitative Study of the Role of Clinical Competency Committees in Performance Assessment. Academic medicine : journal of the Association of American Medical Colleges. 2015.
Problem identification Developmental
![Page 18: Transform your clinical competency committee with learner ... · Transform your clinical competency committee with learner coaching Matthew Kelleher, MD, MEd (@ ... Warm EJ, Mathis](https://reader035.vdocuments.net/reader035/viewer/2022070706/5e9b60130083f77cb2119c02/html5/thumbnails/18.jpg)
“Red flags” and “low-flyers” Helping all residents to improve
Focus on global performance Examine specific strengths/weaknesses
Competence is implicit, just avoid failure
Competence based on comparison to milestones
Feedback delivered by report with minimal follow-up
Feedback delivered by longitudinal mentor/coach
18Hauer KE, Chesluk B, Iobst W, et al. Reviewing Residents' Competence: A Qualitative Study of the Role of Clinical Competency Committees in Performance Assessment. Academic medicine : journal of the Association of American Medical Colleges. 2015.
![Page 19: Transform your clinical competency committee with learner ... · Transform your clinical competency committee with learner coaching Matthew Kelleher, MD, MEd (@ ... Warm EJ, Mathis](https://reader035.vdocuments.net/reader035/viewer/2022070706/5e9b60130083f77cb2119c02/html5/thumbnails/19.jpg)
CCC questions to discuss Answer + Explanation Why
What assessment data does your
program collect that can be used for
specific learner feedback?
What challenges do you foresee in
shifting your CCC to a
Developmental Model? (time,
training, etc) What challenges have
you already faced?
What output/feedback does your
CCC provide to learners? Is it
specific and actionable for learners?
If not, how could that be changed?
Who delivers the feedback? Is this
the right person?
19
![Page 20: Transform your clinical competency committee with learner ... · Transform your clinical competency committee with learner coaching Matthew Kelleher, MD, MEd (@ ... Warm EJ, Mathis](https://reader035.vdocuments.net/reader035/viewer/2022070706/5e9b60130083f77cb2119c02/html5/thumbnails/20.jpg)
Group sharing
Name some coaches you’ve had in life, what
made them great?
20
![Page 21: Transform your clinical competency committee with learner ... · Transform your clinical competency committee with learner coaching Matthew Kelleher, MD, MEd (@ ... Warm EJ, Mathis](https://reader035.vdocuments.net/reader035/viewer/2022070706/5e9b60130083f77cb2119c02/html5/thumbnails/21.jpg)
Draw the following?
21
Asking Questions
Giving Answers
Learner is Expert
You’re the
Expert
1. Mentor2. Coach3. Friend4. Facilitator5. Counselor6. Consultant7. Advisor
![Page 22: Transform your clinical competency committee with learner ... · Transform your clinical competency committee with learner coaching Matthew Kelleher, MD, MEd (@ ... Warm EJ, Mathis](https://reader035.vdocuments.net/reader035/viewer/2022070706/5e9b60130083f77cb2119c02/html5/thumbnails/22.jpg)
Draw the following?
22
Asking Questions
Giving Answers
Learner is Expert
You’re the
Expert
1. Coach2. Mentor3. Friend4. Counselor
Ben’s hockey coach
Law
yer
![Page 23: Transform your clinical competency committee with learner ... · Transform your clinical competency committee with learner coaching Matthew Kelleher, MD, MEd (@ ... Warm EJ, Mathis](https://reader035.vdocuments.net/reader035/viewer/2022070706/5e9b60130083f77cb2119c02/html5/thumbnails/23.jpg)
23
![Page 24: Transform your clinical competency committee with learner ... · Transform your clinical competency committee with learner coaching Matthew Kelleher, MD, MEd (@ ... Warm EJ, Mathis](https://reader035.vdocuments.net/reader035/viewer/2022070706/5e9b60130083f77cb2119c02/html5/thumbnails/24.jpg)
What is coaching?
“helping someone get from point A to point B”
24
“Coaching is the art of facilitating
the performance, learning, and
development of another.”
![Page 25: Transform your clinical competency committee with learner ... · Transform your clinical competency committee with learner coaching Matthew Kelleher, MD, MEd (@ ... Warm EJ, Mathis](https://reader035.vdocuments.net/reader035/viewer/2022070706/5e9b60130083f77cb2119c02/html5/thumbnails/25.jpg)
Many domains can be coached
• Resiliency
• Self-directed learning
• Academic Pursuits
• Leadership
Coaching
competence
![Page 26: Transform your clinical competency committee with learner ... · Transform your clinical competency committee with learner coaching Matthew Kelleher, MD, MEd (@ ... Warm EJ, Mathis](https://reader035.vdocuments.net/reader035/viewer/2022070706/5e9b60130083f77cb2119c02/html5/thumbnails/26.jpg)
Why is Coaching so Important?
Norman G. The adult learner: a mythical species. Acad Med. 1999;74(8):886–889.Davis D, Mazmanian P, Fordis M, Harrison R, Thorpe K, Perrier L.Accuracy of physician self-assessment compared with observed measures of competence. JAMA.2006;296(9):1094–
1102.
![Page 27: Transform your clinical competency committee with learner ... · Transform your clinical competency committee with learner coaching Matthew Kelleher, MD, MEd (@ ... Warm EJ, Mathis](https://reader035.vdocuments.net/reader035/viewer/2022070706/5e9b60130083f77cb2119c02/html5/thumbnails/27.jpg)
27
• Feedback should be a reflective dialogue • Performance monitored, reflections shared and validated,
activities planned, and follow-up negotiated and monitored.
“In conventional assessment programmes, adherence to minimum standards can suffice for promotion and graduation. In programmatic assessment individual excellence is the goal and the mentor (coach) is the key person to promote such excellence.”
![Page 28: Transform your clinical competency committee with learner ... · Transform your clinical competency committee with learner coaching Matthew Kelleher, MD, MEd (@ ... Warm EJ, Mathis](https://reader035.vdocuments.net/reader035/viewer/2022070706/5e9b60130083f77cb2119c02/html5/thumbnails/28.jpg)
28
Kerri PalamaraMcGrath, MD
![Page 29: Transform your clinical competency committee with learner ... · Transform your clinical competency committee with learner coaching Matthew Kelleher, MD, MEd (@ ... Warm EJ, Mathis](https://reader035.vdocuments.net/reader035/viewer/2022070706/5e9b60130083f77cb2119c02/html5/thumbnails/29.jpg)
Coaching in Medical Education
Consistent, Longitudinal
Built on a relationship of trust
Not a supervisor or evaluator
Understand the system and curriculum
Do not need to be experts
Facilitate reflection and prioritize goals
Active Listening
Relentlessly Positive
29Telio S, Ajjawi R, Regehr G. The “educational alliance” as a framework for reconceptualizing feedback in medical education. Acad Med. 2015; 90(5): 609-614.Gawande A. Personal best. The New Yorker, 3 October 2011.
![Page 30: Transform your clinical competency committee with learner ... · Transform your clinical competency committee with learner coaching Matthew Kelleher, MD, MEd (@ ... Warm EJ, Mathis](https://reader035.vdocuments.net/reader035/viewer/2022070706/5e9b60130083f77cb2119c02/html5/thumbnails/30.jpg)
Coaching Sessions
Check In
Get Curious: Reflect and Challenge
Explore: Build Motivation and
Confidence
Accountability: Narrow and Commit
30
Kerri Palamara McGrath, MD
![Page 31: Transform your clinical competency committee with learner ... · Transform your clinical competency committee with learner coaching Matthew Kelleher, MD, MEd (@ ... Warm EJ, Mathis](https://reader035.vdocuments.net/reader035/viewer/2022070706/5e9b60130083f77cb2119c02/html5/thumbnails/31.jpg)
31
![Page 32: Transform your clinical competency committee with learner ... · Transform your clinical competency committee with learner coaching Matthew Kelleher, MD, MEd (@ ... Warm EJ, Mathis](https://reader035.vdocuments.net/reader035/viewer/2022070706/5e9b60130083f77cb2119c02/html5/thumbnails/32.jpg)
Let’s Review and Reflect
32
![Page 33: Transform your clinical competency committee with learner ... · Transform your clinical competency committee with learner coaching Matthew Kelleher, MD, MEd (@ ... Warm EJ, Mathis](https://reader035.vdocuments.net/reader035/viewer/2022070706/5e9b60130083f77cb2119c02/html5/thumbnails/33.jpg)
Practice Coaching
33
Unsupervised Practice
Indirect Supervision
![Page 34: Transform your clinical competency committee with learner ... · Transform your clinical competency committee with learner coaching Matthew Kelleher, MD, MEd (@ ... Warm EJ, Mathis](https://reader035.vdocuments.net/reader035/viewer/2022070706/5e9b60130083f77cb2119c02/html5/thumbnails/34.jpg)
Practice Coaching
34
ICS3 Appropriate utilization and completionof health records.PC-5 Manages patients with progressiveresponsibility and independence.PC-3Requests and provides consultative care
![Page 35: Transform your clinical competency committee with learner ... · Transform your clinical competency committee with learner coaching Matthew Kelleher, MD, MEd (@ ... Warm EJ, Mathis](https://reader035.vdocuments.net/reader035/viewer/2022070706/5e9b60130083f77cb2119c02/html5/thumbnails/35.jpg)
Practice Coaching
35
15 - Escalate care in a critical or unstable patient10 - Demonstrate communication to healthcare team members in timely manner24 - Respond to pages in timely manner45 - Takes leadership role of teaching healthcare team66 - Role model effective communication in challenging situations 168 – Demonstrate empathy and compassion in relieving pain and suffering
![Page 36: Transform your clinical competency committee with learner ... · Transform your clinical competency committee with learner coaching Matthew Kelleher, MD, MEd (@ ... Warm EJ, Mathis](https://reader035.vdocuments.net/reader035/viewer/2022070706/5e9b60130083f77cb2119c02/html5/thumbnails/36.jpg)
Practice Coaching
Narrative is so important
36
![Page 37: Transform your clinical competency committee with learner ... · Transform your clinical competency committee with learner coaching Matthew Kelleher, MD, MEd (@ ... Warm EJ, Mathis](https://reader035.vdocuments.net/reader035/viewer/2022070706/5e9b60130083f77cb2119c02/html5/thumbnails/37.jpg)
What we are doing now
1. Changing the data we review in CCC
(more specific)
2. Planning faculty development in our CCC
3. Planning to recruit coaches
4. Strategizing incentives for coaches
5. Brainstorming on how to measure
change
6. Coaching a handful of residents to pilot
the process
37
![Page 38: Transform your clinical competency committee with learner ... · Transform your clinical competency committee with learner coaching Matthew Kelleher, MD, MEd (@ ... Warm EJ, Mathis](https://reader035.vdocuments.net/reader035/viewer/2022070706/5e9b60130083f77cb2119c02/html5/thumbnails/38.jpg)
38
Coaching questions to discuss Answer + Explanation Why
How many coaches will you need? Ratio to learner?
Who can be a coach? Who cannot?
How often should coach and learners meet? Will it be mandatory?
What training will coaches
need? How often? Other
requirements?
How will you recruit and retain
coaches? What incentive can
you offer?
How will you match learners to
coaches?
How do you know if coaches are
effective?
![Page 39: Transform your clinical competency committee with learner ... · Transform your clinical competency committee with learner coaching Matthew Kelleher, MD, MEd (@ ... Warm EJ, Mathis](https://reader035.vdocuments.net/reader035/viewer/2022070706/5e9b60130083f77cb2119c02/html5/thumbnails/39.jpg)
“Feedback cultivates insight. Coaching promotes
performance change”
- Dr. Marygrace Zetkulic (yesterday)
39
![Page 40: Transform your clinical competency committee with learner ... · Transform your clinical competency committee with learner coaching Matthew Kelleher, MD, MEd (@ ... Warm EJ, Mathis](https://reader035.vdocuments.net/reader035/viewer/2022070706/5e9b60130083f77cb2119c02/html5/thumbnails/40.jpg)
Team
If you want peer mentorship as you transform your CCC into a
developmental model that fuels coaching, email us:
40