utilizing multiple grader rubrics for practical assessment of student performance
TRANSCRIPT
1ExamSoft Assessment Conference | #ASSESS2015
ExamSoft Assessment Conference
#ASSESS2015
2ExamSoft Assessment Conference | #ASSESS2015
Utilizing Multiple Grader Rubrics for Practical Assessment of Student Performance
Sherry Jimenez, Ed.D. Sarah Zahl, Ph.D.
Assistant Dean for Educational Development Director of Educational Assessment
3ExamSoft Assessment Conference | #ASSESS2015
Agenda
• Audience Response• MUCOM Brief Overview • Multiple Grader Rubrics
– Assessment Types and Examples– Tracking outcomes in rubrics
• Recommendations• Small Group- Pair/Share• Q&A
Audience Response
7ExamSoft Assessment Conference | #ASSESS2015
Audience Response
8ExamSoft Assessment Conference | #ASSESS2015
MUCOM Overview
• Competency Based Curricular Design• Systems based/integrated curriculum
– Course Director, multiple internal faculty and external clinicians
– e.g. Cardiovascular course + OSCE• iPad exclusive• Building a culture of evidence among
faculty/students/administrators
9ExamSoft Assessment Conference | #ASSESS2015
Tracking Student Performance
• Competencies• COMLEX blueprint• System• Discipline• Learning Domains• Course specific outcomes
10ExamSoft Assessment Conference | #ASSESS2015
National Board of Osteopathic Medical ExaminersFundamental Osteopathic Medical Competencies
COMLEX Blueprint
11ExamSoft Assessment Conference | #ASSESS2015
Use of Rubrics
• Lab Practicals: Osteopathic Manipulative Medicine (OMM) Lab
• Simulations• Objective Structured Clinical Examinations (OSCE)• Small Group Evaluations: Clinical Medicine
12ExamSoft Assessment Conference | #ASSESS2015
Small Group Evaluations
13ExamSoft Assessment Conference | #ASSESS2015
Small Group Evaluations – Grading Process
14ExamSoft Assessment Conference | #ASSESS2015
Small Group Evaluations – Grading Process
• Complete the administrative planning process externally before using ExamSoft
• Pre-assign faculty graders and students• Pre-assign assessment area (region)• Prepare extra materials and guides• Communication
15ExamSoft Assessment Conference | #ASSESS2015
OSCE
• Humanistic rubric, SOAP note, SP evaluation, self-assessment
16ExamSoft Assessment Conference | #ASSESS2015
Lab Practicals
• Paper Very complex online process streamlined process in ExamSoft
Old process: multiple stations, partner flip, multiple regions, multiple graders and students at each station
New process: one station, partner flip only, one region (pre-determined but surprise to student)
17ExamSoft Assessment Conference | #ASSESS2015
Outcomes
• Culture shift –modifying structure • Positive feedback – ease of evaluation• Immediate faculty feedback – inter-rater • Timeliness of student feedback• Pairing rubric data with written exam data to develop
longitudinal reports • Continuous refinement
18ExamSoft Assessment Conference | #ASSESS2015
Group/Pair Share
• How are you currently assessing student performance in practical settings?
• How could you incorporate multiple grader rubrics in your curriculum?
19ExamSoft Assessment Conference | #ASSESS2015
Discussion
• Debrief and share
• Questions?
20ExamSoft Assessment Conference | #ASSESS2015
Thank You