nailed it! doctor of pharmacy students’ self-awareness of ......nailed it! doctor of pharmacy...
TRANSCRIPT
![Page 1: Nailed it! Doctor of Pharmacy Students’ Self-Awareness of ......Nailed it! Doctor of Pharmacy Students’ Self-Awareness of Performance on Objective Structured Clinical Examinations](https://reader033.vdocuments.net/reader033/viewer/2022060602/60560b223326a87b7e4d3213/html5/thumbnails/1.jpg)
Nailed it! Doctor of Pharmacy Students’ Self-Awareness of Performance on Objective
Structured Clinical Examinations (OSCE)
Kara Sermersheim, PharmD, BS
Kimberly K. Daugherty, PharmD, BCPS
Sarah E. Raake, PharmD, MSEd, BCACP, LDE
Sullivan University College of Pharmacy & Health Sciences
Louisville, KY
Disclosures: The speaker has no actual or potential conflicts of interest in relation to this presentation
![Page 2: Nailed it! Doctor of Pharmacy Students’ Self-Awareness of ......Nailed it! Doctor of Pharmacy Students’ Self-Awareness of Performance on Objective Structured Clinical Examinations](https://reader033.vdocuments.net/reader033/viewer/2022060602/60560b223326a87b7e4d3213/html5/thumbnails/2.jpg)
Objectives
1. Discuss Doctor of Pharmacy (PharmD) students’ self-awareness related to their performance on Objective Structured Clinical Examinations (OSCE).
2. Identify the Doctor of Pharmacy students who are not self-aware based on study data.
3. Discuss how this data could be used to develop student self-awareness skills
![Page 3: Nailed it! Doctor of Pharmacy Students’ Self-Awareness of ......Nailed it! Doctor of Pharmacy Students’ Self-Awareness of Performance on Objective Structured Clinical Examinations](https://reader033.vdocuments.net/reader033/viewer/2022060602/60560b223326a87b7e4d3213/html5/thumbnails/3.jpg)
Background: Sullivan Itself
− Sullivan University College of Pharmacy and Health Sciences− 3-year accelerated program
− Louisville, KY
− First graduating class in 2011
− Objective Structured Clinical Examinations (OSCEs) − Experiences that link didactic learning to simulated real-world cases
![Page 4: Nailed it! Doctor of Pharmacy Students’ Self-Awareness of ......Nailed it! Doctor of Pharmacy Students’ Self-Awareness of Performance on Objective Structured Clinical Examinations](https://reader033.vdocuments.net/reader033/viewer/2022060602/60560b223326a87b7e4d3213/html5/thumbnails/4.jpg)
Why?Self-assessment of OSCE performance may help students develop self-awareness
in the practice setting
![Page 5: Nailed it! Doctor of Pharmacy Students’ Self-Awareness of ......Nailed it! Doctor of Pharmacy Students’ Self-Awareness of Performance on Objective Structured Clinical Examinations](https://reader033.vdocuments.net/reader033/viewer/2022060602/60560b223326a87b7e4d3213/html5/thumbnails/5.jpg)
Self-awareness
![Page 6: Nailed it! Doctor of Pharmacy Students’ Self-Awareness of ......Nailed it! Doctor of Pharmacy Students’ Self-Awareness of Performance on Objective Structured Clinical Examinations](https://reader033.vdocuments.net/reader033/viewer/2022060602/60560b223326a87b7e4d3213/html5/thumbnails/6.jpg)
Self-awareness
− Conscious knowledge of one’s own character, feelings, motives, and desires
− CAPE Outcome 4.1: students should display self-awareness defined as the ability to examine and reflect on personal knowledge, skills, abilities, beliefs, biases, motivation, and emotions that could enhance or limit personal and professional growth
− There is currently limited data relating OSCEs to student self-awareness
Oxford Online Dictionary. Definition of self-awareness. https://www.oxfordlearnersdictionaries.com/us/definition/english/self-awareness. (accessed 25 Mar 2020).Accreditation Council for Pharmacy Education. Accreditation standards and key elements for the professional program in pharmacy leading to the doctor of pharmacy degree. https://www.acpe-accredit.org/pharmd-program-accreditation/ (accessed 2019 Sept 16).
![Page 7: Nailed it! Doctor of Pharmacy Students’ Self-Awareness of ......Nailed it! Doctor of Pharmacy Students’ Self-Awareness of Performance on Objective Structured Clinical Examinations](https://reader033.vdocuments.net/reader033/viewer/2022060602/60560b223326a87b7e4d3213/html5/thumbnails/7.jpg)
Objective Structured Clinical Examinations (OSCEs)
![Page 8: Nailed it! Doctor of Pharmacy Students’ Self-Awareness of ......Nailed it! Doctor of Pharmacy Students’ Self-Awareness of Performance on Objective Structured Clinical Examinations](https://reader033.vdocuments.net/reader033/viewer/2022060602/60560b223326a87b7e4d3213/html5/thumbnails/8.jpg)
OSCEs as natural place to assess self-awareness
− OSCEs in pharmacy education are experiences that link didactic learning to simulated real-world cases
− Self-assessment of OSCE performance may help students develop self-awareness in the practice setting
![Page 9: Nailed it! Doctor of Pharmacy Students’ Self-Awareness of ......Nailed it! Doctor of Pharmacy Students’ Self-Awareness of Performance on Objective Structured Clinical Examinations](https://reader033.vdocuments.net/reader033/viewer/2022060602/60560b223326a87b7e4d3213/html5/thumbnails/9.jpg)
Past examples of students lacking self-awareness in OSCEs
− Underconfident learner
− Overconfident leaner
![Page 10: Nailed it! Doctor of Pharmacy Students’ Self-Awareness of ......Nailed it! Doctor of Pharmacy Students’ Self-Awareness of Performance on Objective Structured Clinical Examinations](https://reader033.vdocuments.net/reader033/viewer/2022060602/60560b223326a87b7e4d3213/html5/thumbnails/10.jpg)
Sample case (Directions to student)
![Page 11: Nailed it! Doctor of Pharmacy Students’ Self-Awareness of ......Nailed it! Doctor of Pharmacy Students’ Self-Awareness of Performance on Objective Structured Clinical Examinations](https://reader033.vdocuments.net/reader033/viewer/2022060602/60560b223326a87b7e4d3213/html5/thumbnails/11.jpg)
Self-awareness survey
![Page 12: Nailed it! Doctor of Pharmacy Students’ Self-Awareness of ......Nailed it! Doctor of Pharmacy Students’ Self-Awareness of Performance on Objective Structured Clinical Examinations](https://reader033.vdocuments.net/reader033/viewer/2022060602/60560b223326a87b7e4d3213/html5/thumbnails/12.jpg)
Grading checklist (SP completes)Pass Fail
Checklist item 1 X
Checklist item 2 X
Checklist item 3 X
Checklist item 4 X
Checklist item 5 X
Skills Section Score 80%
Checklist item 1 X
Checklist item 2 X
Checklist item 3 X
Checklist item 4 X
Communication Section 25%
Overall Score (80% skills + 20% communication) 69%
Return to the Pharmacist (RTP) No
Overall Student Performance
![Page 13: Nailed it! Doctor of Pharmacy Students’ Self-Awareness of ......Nailed it! Doctor of Pharmacy Students’ Self-Awareness of Performance on Objective Structured Clinical Examinations](https://reader033.vdocuments.net/reader033/viewer/2022060602/60560b223326a87b7e4d3213/html5/thumbnails/13.jpg)
Sample SP checklist and comments
![Page 14: Nailed it! Doctor of Pharmacy Students’ Self-Awareness of ......Nailed it! Doctor of Pharmacy Students’ Self-Awareness of Performance on Objective Structured Clinical Examinations](https://reader033.vdocuments.net/reader033/viewer/2022060602/60560b223326a87b7e4d3213/html5/thumbnails/14.jpg)
Why?We do these cases to assess patient care skills as well as soft skills such as,
communication, professionalism, empathy, and potentially self-awareness.
![Page 15: Nailed it! Doctor of Pharmacy Students’ Self-Awareness of ......Nailed it! Doctor of Pharmacy Students’ Self-Awareness of Performance on Objective Structured Clinical Examinations](https://reader033.vdocuments.net/reader033/viewer/2022060602/60560b223326a87b7e4d3213/html5/thumbnails/15.jpg)
Pharmacist patient care process
Joint Commission of Pharmacy Practitioners. Pharmacists’ patient care process. https://www.pharmacist.com/sites/default/files/files/PatientCareProcess.pdf. (accessed 2020 Mar 23).
![Page 16: Nailed it! Doctor of Pharmacy Students’ Self-Awareness of ......Nailed it! Doctor of Pharmacy Students’ Self-Awareness of Performance on Objective Structured Clinical Examinations](https://reader033.vdocuments.net/reader033/viewer/2022060602/60560b223326a87b7e4d3213/html5/thumbnails/16.jpg)
Case “buckets”
Collect Assess Plan Implement Follow-up
Communication/Law X X X
Patient Counseling X X
Patient Medication Recommendation
X X X X
Provider Recommendation
X X X X
Skills X X
![Page 17: Nailed it! Doctor of Pharmacy Students’ Self-Awareness of ......Nailed it! Doctor of Pharmacy Students’ Self-Awareness of Performance on Objective Structured Clinical Examinations](https://reader033.vdocuments.net/reader033/viewer/2022060602/60560b223326a87b7e4d3213/html5/thumbnails/17.jpg)
Grading Checklist (SP completes)Pass Fail
Collect Checklist item 1 X
Assess Checklist item 2 X
Assess Checklist item 3 X
Plan Checklist item 4 X
Implement Checklist item 5 X
Skills Section Score 80%
Checklist item 1 X
Checklist item 2 X
Checklist item 3 X
Checklist item 4 X
Communication Section 25%
Return to the Pharmacist (RTP) No
Overall Score (80% skills + 20% communication) 69%
Overall Student Performance
![Page 18: Nailed it! Doctor of Pharmacy Students’ Self-Awareness of ......Nailed it! Doctor of Pharmacy Students’ Self-Awareness of Performance on Objective Structured Clinical Examinations](https://reader033.vdocuments.net/reader033/viewer/2022060602/60560b223326a87b7e4d3213/html5/thumbnails/18.jpg)
The ProjectStudy Design & Statistics
![Page 19: Nailed it! Doctor of Pharmacy Students’ Self-Awareness of ......Nailed it! Doctor of Pharmacy Students’ Self-Awareness of Performance on Objective Structured Clinical Examinations](https://reader033.vdocuments.net/reader033/viewer/2022060602/60560b223326a87b7e4d3213/html5/thumbnails/19.jpg)
Methods
– Retrospective review
– Single institution
– 3-year program
– Single cohort– 4 OSCEs completed
– 23 discrete cases
Joint Commission of Pharmacy Practitioners. Pharmacists’ patient care process. https://www.pharmacist.com/sites/default/files/files/PatientCareProcess.pdf. (accessed 2020 Feb 22).
![Page 20: Nailed it! Doctor of Pharmacy Students’ Self-Awareness of ......Nailed it! Doctor of Pharmacy Students’ Self-Awareness of Performance on Objective Structured Clinical Examinations](https://reader033.vdocuments.net/reader033/viewer/2022060602/60560b223326a87b7e4d3213/html5/thumbnails/20.jpg)
• Reliability• Validity
Self-awareness survey
• Reliability• Validity
OSCE cases
![Page 21: Nailed it! Doctor of Pharmacy Students’ Self-Awareness of ......Nailed it! Doctor of Pharmacy Students’ Self-Awareness of Performance on Objective Structured Clinical Examinations](https://reader033.vdocuments.net/reader033/viewer/2022060602/60560b223326a87b7e4d3213/html5/thumbnails/21.jpg)
Obtaining validity and reliability
– Microsoft Forms– Survey
– 23 individual OSCE cases
– Utilized 15 faculty members
![Page 22: Nailed it! Doctor of Pharmacy Students’ Self-Awareness of ......Nailed it! Doctor of Pharmacy Students’ Self-Awareness of Performance on Objective Structured Clinical Examinations](https://reader033.vdocuments.net/reader033/viewer/2022060602/60560b223326a87b7e4d3213/html5/thumbnails/22.jpg)
Validity and reliability
– Lawshe method– Average CVR for all 23 cases was 0.74 (range 0.36-0.92)
– Sixteen cases were content valid (CVR ≥ 0.75)
– Seven cases had a CVR between 0-0.74
– Cronbach’s Alpha– Average Cronbach alpha for all 23 cases was 0.646 (range 0.193-0.887)
– Four cases had a Cronbach alpha of <0.5
– Nine cases were between 0.5-0.699
– Ten cases had a Cronbach alpha >0.7
![Page 23: Nailed it! Doctor of Pharmacy Students’ Self-Awareness of ......Nailed it! Doctor of Pharmacy Students’ Self-Awareness of Performance on Objective Structured Clinical Examinations](https://reader033.vdocuments.net/reader033/viewer/2022060602/60560b223326a87b7e4d3213/html5/thumbnails/23.jpg)
Validity and reliability
– Spearman-Brown – The overall self-awareness survey (nail it/fail it) was found to be reliable
(0.829)
– Only the return to the pharmacist (0.743) and overall student performance (0.611) subscores were found to be reliable
Eisinga R, te Grotenhuis M, Pelzer B. The reliability of a two-item scale: pearson, Cronbach or spearman-brown. Int J Public Health. 2012;58(4), 637–642.
![Page 24: Nailed it! Doctor of Pharmacy Students’ Self-Awareness of ......Nailed it! Doctor of Pharmacy Students’ Self-Awareness of Performance on Objective Structured Clinical Examinations](https://reader033.vdocuments.net/reader033/viewer/2022060602/60560b223326a87b7e4d3213/html5/thumbnails/24.jpg)
Data reviewed using Chi-squared
OSCE Grade
Failed (≤69.4%)
Passed (≥69.5%)
Survey Nail it (1)
Fail it (0)
Eisinga R, te Grotenhuis M, Pelzer B. The reliability of a two-item scale: pearson, Cronbach or spearman-brown. Int J Public Health. 2012;58(4), 637–642.
![Page 25: Nailed it! Doctor of Pharmacy Students’ Self-Awareness of ......Nailed it! Doctor of Pharmacy Students’ Self-Awareness of Performance on Objective Structured Clinical Examinations](https://reader033.vdocuments.net/reader033/viewer/2022060602/60560b223326a87b7e4d3213/html5/thumbnails/25.jpg)
Identifying the disconnect
Overall Performance
OSCE Score
Failed
(≤69.4%)
Passed
(≥69.5%)
Survey
Nail it (1)
Fail it (0)
Return to the Pharmacist
SP Answer
No (0) Yes (1)
Survey
Yes (1)
No (0)
![Page 26: Nailed it! Doctor of Pharmacy Students’ Self-Awareness of ......Nailed it! Doctor of Pharmacy Students’ Self-Awareness of Performance on Objective Structured Clinical Examinations](https://reader033.vdocuments.net/reader033/viewer/2022060602/60560b223326a87b7e4d3213/html5/thumbnails/26.jpg)
The ProjectResults & Limitations
![Page 27: Nailed it! Doctor of Pharmacy Students’ Self-Awareness of ......Nailed it! Doctor of Pharmacy Students’ Self-Awareness of Performance on Objective Structured Clinical Examinations](https://reader033.vdocuments.net/reader033/viewer/2022060602/60560b223326a87b7e4d3213/html5/thumbnails/27.jpg)
Overall-Self Awareness
![Page 28: Nailed it! Doctor of Pharmacy Students’ Self-Awareness of ......Nailed it! Doctor of Pharmacy Students’ Self-Awareness of Performance on Objective Structured Clinical Examinations](https://reader033.vdocuments.net/reader033/viewer/2022060602/60560b223326a87b7e4d3213/html5/thumbnails/28.jpg)
Case breakdown and odds Ratio
![Page 29: Nailed it! Doctor of Pharmacy Students’ Self-Awareness of ......Nailed it! Doctor of Pharmacy Students’ Self-Awareness of Performance on Objective Structured Clinical Examinations](https://reader033.vdocuments.net/reader033/viewer/2022060602/60560b223326a87b7e4d3213/html5/thumbnails/29.jpg)
Limitations
– Single cohort of students
– Single institution
– Low stakes
![Page 30: Nailed it! Doctor of Pharmacy Students’ Self-Awareness of ......Nailed it! Doctor of Pharmacy Students’ Self-Awareness of Performance on Objective Structured Clinical Examinations](https://reader033.vdocuments.net/reader033/viewer/2022060602/60560b223326a87b7e4d3213/html5/thumbnails/30.jpg)
ConclusionsAction Plan(s)
![Page 31: Nailed it! Doctor of Pharmacy Students’ Self-Awareness of ......Nailed it! Doctor of Pharmacy Students’ Self-Awareness of Performance on Objective Structured Clinical Examinations](https://reader033.vdocuments.net/reader033/viewer/2022060602/60560b223326a87b7e4d3213/html5/thumbnails/31.jpg)
Action Plan
– Data-driven report card
– Moving to modular dashboard feedback mechanism
– Face-to-face meeting
![Page 32: Nailed it! Doctor of Pharmacy Students’ Self-Awareness of ......Nailed it! Doctor of Pharmacy Students’ Self-Awareness of Performance on Objective Structured Clinical Examinations](https://reader033.vdocuments.net/reader033/viewer/2022060602/60560b223326a87b7e4d3213/html5/thumbnails/32.jpg)
Watch list
Last Name Overall Score Skills Communication RTP Collect Assess Plan Implement Follow-up/Monitor Notes Advisor
Learner 1 Smith
Learner 2 XX Jones
Learner 3 Dennis
Learner 4 Lawrence
Learner 5 Jennings
![Page 33: Nailed it! Doctor of Pharmacy Students’ Self-Awareness of ......Nailed it! Doctor of Pharmacy Students’ Self-Awareness of Performance on Objective Structured Clinical Examinations](https://reader033.vdocuments.net/reader033/viewer/2022060602/60560b223326a87b7e4d3213/html5/thumbnails/33.jpg)
Future directions
– Student remediation– Develop activities
– Faculty advisor meetings
– Identify areas of self-improvement
– Learner and faculty development opportunities
– Review and improvement of OSCE cases not deemed valid
![Page 34: Nailed it! Doctor of Pharmacy Students’ Self-Awareness of ......Nailed it! Doctor of Pharmacy Students’ Self-Awareness of Performance on Objective Structured Clinical Examinations](https://reader033.vdocuments.net/reader033/viewer/2022060602/60560b223326a87b7e4d3213/html5/thumbnails/34.jpg)
Final take-away
– Nail it/Fail it surveys are a useful tool to identify students who are not self-aware
![Page 35: Nailed it! Doctor of Pharmacy Students’ Self-Awareness of ......Nailed it! Doctor of Pharmacy Students’ Self-Awareness of Performance on Objective Structured Clinical Examinations](https://reader033.vdocuments.net/reader033/viewer/2022060602/60560b223326a87b7e4d3213/html5/thumbnails/35.jpg)
Acknowledgements
– Katie F. Leslie, PhD, MS Director of The Center for Health and Wellness and Clinical and Administrative Sciences Associate Professor
![Page 36: Nailed it! Doctor of Pharmacy Students’ Self-Awareness of ......Nailed it! Doctor of Pharmacy Students’ Self-Awareness of Performance on Objective Structured Clinical Examinations](https://reader033.vdocuments.net/reader033/viewer/2022060602/60560b223326a87b7e4d3213/html5/thumbnails/36.jpg)
Nailed it! Doctor of Pharmacy Students’ Self-Awareness of Performance on Objective
Structured Clinical Examinations (OSCE)
Kara Sermersheim, PharmD, BS
Kimberly K. Daugherty, PharmD, BCPS
Sarah E. Raake, PharmD, MSEd, BCACP, LDE
Sullivan University College of Pharmacy & Health Sciences
Louisville, KY
Disclosures: The speaker has no actual or potential conflicts of interest in relation to this presentation