promoting the development of clinical skills throughout the continuum of medical education...
TRANSCRIPT
![Page 1: Promoting the Development of Clinical Skills throughout the Continuum of Medical Education University of North Carolina – Chapel Hill School of Medicine](https://reader036.vdocuments.net/reader036/viewer/2022062515/56649d0d5503460f949e1b01/html5/thumbnails/1.jpg)
Promoting the Development of Clinical Skills throughout the Continuum of Medical Education
University of North Carolina – Chapel Hill School of MedicineNovember 9, 2011
![Page 2: Promoting the Development of Clinical Skills throughout the Continuum of Medical Education University of North Carolina – Chapel Hill School of Medicine](https://reader036.vdocuments.net/reader036/viewer/2022062515/56649d0d5503460f949e1b01/html5/thumbnails/2.jpg)
Ann C. Jobe, MD,MSNExecutive Director
Clinical Skills Evaluation Collaboration (CSEC)
![Page 3: Promoting the Development of Clinical Skills throughout the Continuum of Medical Education University of North Carolina – Chapel Hill School of Medicine](https://reader036.vdocuments.net/reader036/viewer/2022062515/56649d0d5503460f949e1b01/html5/thumbnails/3.jpg)
Clinical Skills in Practice
• The physician-patient encounter is central to the identity of physicians in the US
• Clinical skills of trainees and young physicians have been described as deficient since at least the 1970’s
• Good evidence supports the diagnostic and therapeutic value of the clinical encounter but…
• …..Technology, fragmented care, reimbursement, and practice culture affect the clinical encounter
Weiner,A. & Nathonson M; JAMA 1976; 236:852-855Verghese, A et al; Annals Int Med 2011;155:550-553
![Page 4: Promoting the Development of Clinical Skills throughout the Continuum of Medical Education University of North Carolina – Chapel Hill School of Medicine](https://reader036.vdocuments.net/reader036/viewer/2022062515/56649d0d5503460f949e1b01/html5/thumbnails/4.jpg)
Clinical Skills in Practice
• The clinical encounter is often buried in process measures, such as HEDIS or other guidelines
• The ritual value of the clinical encounter is important, and must be balanced by its documented utility
• The environment determines most of what and how trainees learn about the clinical examination
Weiner,A. & Nathonson M; JAMA 1976; 236:852-855Verghese, A et al; Annals Int Med 2011;155:550-553
![Page 5: Promoting the Development of Clinical Skills throughout the Continuum of Medical Education University of North Carolina – Chapel Hill School of Medicine](https://reader036.vdocuments.net/reader036/viewer/2022062515/56649d0d5503460f949e1b01/html5/thumbnails/5.jpg)
COMMUNICATION
• The essence of the patient-physician relationship
• Includes communicating verbally, non-verbally, as well as actions and interactions during a physical examination
![Page 6: Promoting the Development of Clinical Skills throughout the Continuum of Medical Education University of North Carolina – Chapel Hill School of Medicine](https://reader036.vdocuments.net/reader036/viewer/2022062515/56649d0d5503460f949e1b01/html5/thumbnails/6.jpg)
Communication
• It is all about COMMUNICATING with patients and families and health professionals
• It is all about improving communication to improve the quality and safety of health care
![Page 7: Promoting the Development of Clinical Skills throughout the Continuum of Medical Education University of North Carolina – Chapel Hill School of Medicine](https://reader036.vdocuments.net/reader036/viewer/2022062515/56649d0d5503460f949e1b01/html5/thumbnails/7.jpg)
Why Assess Communication Skills?
• Essential physician competency • (LCME, ACGME, ABMS, USMLE)
• Clinical outcomes require effective
communication
• Public expectations: need for more
information and supportive interactions.
• Quality measures now incorporate
patient-centeredness
![Page 8: Promoting the Development of Clinical Skills throughout the Continuum of Medical Education University of North Carolina – Chapel Hill School of Medicine](https://reader036.vdocuments.net/reader036/viewer/2022062515/56649d0d5503460f949e1b01/html5/thumbnails/8.jpg)
Patient-Centered Communication
• Exploring the patient’s illness experience
• Understanding the patient as a whole person
• Picking up on patient cues
• Involvement of the patient in problem definition• Involvement of the patient in decision-making
• (now >50% expect such involvement)
• Finding common ground regarding management• Enhancing the doctor/patient relationship by
being responsive to the patient IOM,2001; Street,2008
![Page 9: Promoting the Development of Clinical Skills throughout the Continuum of Medical Education University of North Carolina – Chapel Hill School of Medicine](https://reader036.vdocuments.net/reader036/viewer/2022062515/56649d0d5503460f949e1b01/html5/thumbnails/9.jpg)
Communication Skills
• Prospective study of 80 medical outpatients with new or previously undiagnosed conditions
• Internists asked to list their differential diagnoses and to estimate their confidence in each diagnostic possibility • after the history, • after the physical examination, and • after the laboratory investigation.
![Page 10: Promoting the Development of Clinical Skills throughout the Continuum of Medical Education University of North Carolina – Chapel Hill School of Medicine](https://reader036.vdocuments.net/reader036/viewer/2022062515/56649d0d5503460f949e1b01/html5/thumbnails/10.jpg)
Communication Skills
• In 61 of 80 cases (76%), the leading diagnosis after taking the history agreed with the diagnosis accepted at the time the record was reviewed• The physical examination led to the diagnosis in 10 patients (12%)• The laboratory investigation led to the diagnosis in 9 patients
(11%)
• These data support the concept that most diagnoses are made from the medical history
![Page 11: Promoting the Development of Clinical Skills throughout the Continuum of Medical Education University of North Carolina – Chapel Hill School of Medicine](https://reader036.vdocuments.net/reader036/viewer/2022062515/56649d0d5503460f949e1b01/html5/thumbnails/11.jpg)
Communication Skills
• Authors suggest that more time should be devoted to improving history-taking skills during clinical training.
Peterson MC, Holbrook JH, Hales D, Smith NL, Staker LV: Contributions of the history, physical examination, and laboratory investigation in making medical diagnoses.
West J Med 1992 Feb; 156:163-165
![Page 12: Promoting the Development of Clinical Skills throughout the Continuum of Medical Education University of North Carolina – Chapel Hill School of Medicine](https://reader036.vdocuments.net/reader036/viewer/2022062515/56649d0d5503460f949e1b01/html5/thumbnails/12.jpg)
Communication Skills
• Numerous publications confirm that poor skills in patient communication are associated with:• Lower levels of patient
satisfaction• Higher rates of complaints• Increased risk of
malpractice claims• Poorer health outcomes
![Page 13: Promoting the Development of Clinical Skills throughout the Continuum of Medical Education University of North Carolina – Chapel Hill School of Medicine](https://reader036.vdocuments.net/reader036/viewer/2022062515/56649d0d5503460f949e1b01/html5/thumbnails/13.jpg)
High level skills in “bedside medicine” – “clinical skills” • Ability to elicit a patient’s
story/history• Correct use of evidence-based
PE maneuvers in a focused manner based on history
• Ability to synthesize information gathered
• Ability to communicate and negotiate plans for management
are the cornerstone of patient safety and quality of care
![Page 14: Promoting the Development of Clinical Skills throughout the Continuum of Medical Education University of North Carolina – Chapel Hill School of Medicine](https://reader036.vdocuments.net/reader036/viewer/2022062515/56649d0d5503460f949e1b01/html5/thumbnails/14.jpg)
Why Does It Matter?
• Initiatives focused on improving clinical skills, especially communication – through teaching and assessment - will be most successful in improving the quality and outcomes of care provided by health professionals
![Page 15: Promoting the Development of Clinical Skills throughout the Continuum of Medical Education University of North Carolina – Chapel Hill School of Medicine](https://reader036.vdocuments.net/reader036/viewer/2022062515/56649d0d5503460f949e1b01/html5/thumbnails/15.jpg)
Comprehensive Program
• Overarching Competencies and Objectives
• Map for addressing teaching and assessing throughout the continuum of education• Course content• Assessment methodologies
![Page 16: Promoting the Development of Clinical Skills throughout the Continuum of Medical Education University of North Carolina – Chapel Hill School of Medicine](https://reader036.vdocuments.net/reader036/viewer/2022062515/56649d0d5503460f949e1b01/html5/thumbnails/16.jpg)
AAMC Recommendations ForClinical Skills Curricula For Undergraduate
Medical Education(2008)
• Professionalism• The ability to understand the nature of, and demonstrate
professional and ethical behavior in, the act of medical care.
• Patient Engagement and Communication Skills• The ability to engage and communicate with a patient, develop a
student-patient relationship, and communicate with others in the professional setting
• Biomedical Knowledge Application Skills• The ability to apply scientific knowledge and method to clinical
problem solving.
![Page 17: Promoting the Development of Clinical Skills throughout the Continuum of Medical Education University of North Carolina – Chapel Hill School of Medicine](https://reader036.vdocuments.net/reader036/viewer/2022062515/56649d0d5503460f949e1b01/html5/thumbnails/17.jpg)
AAMC Recommendations ForClinical Skills Curricula For Undergraduate
Medical Education(2008)
• History Taking• The ability to take a clinical history, both focused and
comprehensive.
• Patient Examination• The ability to perform a mental and physical examination
• Clinical Testing• The ability to select, justify and interpret selected clinical tests and
imaging
• Clinical Procedures• The ability to understand and perform a variety of basic clinical
procedures
![Page 18: Promoting the Development of Clinical Skills throughout the Continuum of Medical Education University of North Carolina – Chapel Hill School of Medicine](https://reader036.vdocuments.net/reader036/viewer/2022062515/56649d0d5503460f949e1b01/html5/thumbnails/18.jpg)
AAMC Recommendations ForClinical Skills Curricula For Undergraduate
Medical Education(2008)
• Diagnosis• The ability to diagnose and explain clinical problems in terms of
pathogenesis, to develop basic differential diagnosis, andto learn and demonstrate clinical reasoning and problem identification.
• Clinical Information Management• The ability to record, present, research, critique and manage
clinical information
• Clinical Intervention• The ability to understand and select clinical interventions in the
natural history of disease, including basic preventive, curative and palliative strategies
![Page 19: Promoting the Development of Clinical Skills throughout the Continuum of Medical Education University of North Carolina – Chapel Hill School of Medicine](https://reader036.vdocuments.net/reader036/viewer/2022062515/56649d0d5503460f949e1b01/html5/thumbnails/19.jpg)
AAMC Recommendations ForClinical Skills Curricula For Undergraduate
Medical Education(2008)
• Prognosis• The ability to understand and formulate a prognosis about the
future events of an individual’s health and illness basedupon an understanding of the patient, the natural history of disease, and upon known intervention alternatives.
• Personalizing Clinical Care• The ability to provide clinical care within the practical context of a
patient’s age, gender, personal preferences, family, healthliteracy, culture, religious perspective, and their economic circumstances
![Page 20: Promoting the Development of Clinical Skills throughout the Continuum of Medical Education University of North Carolina – Chapel Hill School of Medicine](https://reader036.vdocuments.net/reader036/viewer/2022062515/56649d0d5503460f949e1b01/html5/thumbnails/20.jpg)
Core Competencies & Assessment
• Patient Care/Clinical Skills• Students must be able
to provide care that is compassionate, appropriate, and effective for treating health problems and promoting health
![Page 21: Promoting the Development of Clinical Skills throughout the Continuum of Medical Education University of North Carolina – Chapel Hill School of Medicine](https://reader036.vdocuments.net/reader036/viewer/2022062515/56649d0d5503460f949e1b01/html5/thumbnails/21.jpg)
Core Competencies & Assessment
• Interpersonal & Communication Skills• Students must
demonstrate interpersonal and communication skills that facilitate effective interactions with patients and their families and other health professionals
![Page 22: Promoting the Development of Clinical Skills throughout the Continuum of Medical Education University of North Carolina – Chapel Hill School of Medicine](https://reader036.vdocuments.net/reader036/viewer/2022062515/56649d0d5503460f949e1b01/html5/thumbnails/22.jpg)
Developing a Comprehensive Program
• Types of assessments
• Examinees
• Timing of assessments
![Page 23: Promoting the Development of Clinical Skills throughout the Continuum of Medical Education University of North Carolina – Chapel Hill School of Medicine](https://reader036.vdocuments.net/reader036/viewer/2022062515/56649d0d5503460f949e1b01/html5/thumbnails/23.jpg)
• Types of assessments• Formative
• Designed to provide feedback to facilitate acquisition of new skills or improvement of performance
• Part of continuous professional development• Part of performance and quality improvement
![Page 24: Promoting the Development of Clinical Skills throughout the Continuum of Medical Education University of North Carolina – Chapel Hill School of Medicine](https://reader036.vdocuments.net/reader036/viewer/2022062515/56649d0d5503460f949e1b01/html5/thumbnails/24.jpg)
• Types of assessments• Summative
• “High stakes”• Associated with an important decision – like
graduation, licensure, certification or credentialing• Utilized to distinguish between those who are
competent and those who are not
![Page 25: Promoting the Development of Clinical Skills throughout the Continuum of Medical Education University of North Carolina – Chapel Hill School of Medicine](https://reader036.vdocuments.net/reader036/viewer/2022062515/56649d0d5503460f949e1b01/html5/thumbnails/25.jpg)
• Types of assessments• “Snapshot”
• One time assessment
• Longitudinal• Repeated over various periods of time
![Page 26: Promoting the Development of Clinical Skills throughout the Continuum of Medical Education University of North Carolina – Chapel Hill School of Medicine](https://reader036.vdocuments.net/reader036/viewer/2022062515/56649d0d5503460f949e1b01/html5/thumbnails/26.jpg)
• Timing of assessments• At planned intervals for promotion decisions• Ongoing for continuous professional
development and/or performance improvement• One-time “snapshot” for initial licensure • Repeat assessment for license renewal• For credentialing or granting privileges• Review for re-entry into practice
![Page 27: Promoting the Development of Clinical Skills throughout the Continuum of Medical Education University of North Carolina – Chapel Hill School of Medicine](https://reader036.vdocuments.net/reader036/viewer/2022062515/56649d0d5503460f949e1b01/html5/thumbnails/27.jpg)
Program Elements
• Depend on PURPOSE of the assessment
and• LEVEL of the
examinee
![Page 28: Promoting the Development of Clinical Skills throughout the Continuum of Medical Education University of North Carolina – Chapel Hill School of Medicine](https://reader036.vdocuments.net/reader036/viewer/2022062515/56649d0d5503460f949e1b01/html5/thumbnails/28.jpg)
Assessing Skills and Performance
• What is included in an assessment of skills and performance?
• What are some of the assessment methods and how are they assembled?
• How do the methods perform against the criteria for good assessment?
![Page 29: Promoting the Development of Clinical Skills throughout the Continuum of Medical Education University of North Carolina – Chapel Hill School of Medicine](https://reader036.vdocuments.net/reader036/viewer/2022062515/56649d0d5503460f949e1b01/html5/thumbnails/29.jpg)
Miller’s Pyramid for Assessing Clinical
Competence
Does
Knows
Shows How
Knows How
Knowledge
Performance
Competence
Action
![Page 30: Promoting the Development of Clinical Skills throughout the Continuum of Medical Education University of North Carolina – Chapel Hill School of Medicine](https://reader036.vdocuments.net/reader036/viewer/2022062515/56649d0d5503460f949e1b01/html5/thumbnails/30.jpg)
Kirkpatrick Criteria
4. ResultsChange in organizational practice
Benefits to patients/clients
5. BehaviorTransfer learning to workplace
Learners apply new knowledge and skills
6. LearningChange attitudes/perceptions
Change knowledge/skills
7. ReactionCustomer satisfaction related to participation in
educational activities
![Page 31: Promoting the Development of Clinical Skills throughout the Continuum of Medical Education University of North Carolina – Chapel Hill School of Medicine](https://reader036.vdocuments.net/reader036/viewer/2022062515/56649d0d5503460f949e1b01/html5/thumbnails/31.jpg)
Simulation
• Simulation• Real patients are
replaced with realistic but artificial experiences
• Trainee interacts with the re-creations
• Judgments are made about their performance
![Page 32: Promoting the Development of Clinical Skills throughout the Continuum of Medical Education University of North Carolina – Chapel Hill School of Medicine](https://reader036.vdocuments.net/reader036/viewer/2022062515/56649d0d5503460f949e1b01/html5/thumbnails/32.jpg)
Simulation
• Methods can be divided according to how faithful they are to reality• Intermediate fidelity
• Task specific models
• Instructor driven models
• High fidelity • Virtual reality
• Standardized patients (SPs)
![Page 33: Promoting the Development of Clinical Skills throughout the Continuum of Medical Education University of North Carolina – Chapel Hill School of Medicine](https://reader036.vdocuments.net/reader036/viewer/2022062515/56649d0d5503460f949e1b01/html5/thumbnails/33.jpg)
Method: Task Specific Models
• Designed around a specific task• Venipuncture model• Animal cadavers
• Usually not automated• Relatively inexpensive
![Page 34: Promoting the Development of Clinical Skills throughout the Continuum of Medical Education University of North Carolina – Chapel Hill School of Medicine](https://reader036.vdocuments.net/reader036/viewer/2022062515/56649d0d5503460f949e1b01/html5/thumbnails/34.jpg)
Method: Instructor Driven Models
• Physical representation
• Responses driven by an instructor
• Little feedback
• Moderate cost
![Page 35: Promoting the Development of Clinical Skills throughout the Continuum of Medical Education University of North Carolina – Chapel Hill School of Medicine](https://reader036.vdocuments.net/reader036/viewer/2022062515/56649d0d5503460f949e1b01/html5/thumbnails/35.jpg)
Method: Virtual Reality Simulators
• Simple physical representation
• Sensing device that informs computer of user actions
• Computer models realistic reactions• 3D imaging• Haptics
![Page 36: Promoting the Development of Clinical Skills throughout the Continuum of Medical Education University of North Carolina – Chapel Hill School of Medicine](https://reader036.vdocuments.net/reader036/viewer/2022062515/56649d0d5503460f949e1b01/html5/thumbnails/36.jpg)
Method: Standardized Patients
• Individuals trained to portray a patient• Scripted and standardized
• USMLE Step 2 CS example• Integrated Clinical Encounter
• Data gathering• SP completing checklists
• Written communication• Doctor rating a patient note
• Communication & Interpersonal skills
• SP Rating
• Spoken English• SP Rating
![Page 37: Promoting the Development of Clinical Skills throughout the Continuum of Medical Education University of North Carolina – Chapel Hill School of Medicine](https://reader036.vdocuments.net/reader036/viewer/2022062515/56649d0d5503460f949e1b01/html5/thumbnails/37.jpg)
Ideal Assessment of Communication Skills
• Evidence-based construct• Assessment instrument consists of observable
behaviors• Realistic stimuli
• SPs trained to use instrument reliably
• Appropriate scoring decisions
![Page 38: Promoting the Development of Clinical Skills throughout the Continuum of Medical Education University of North Carolina – Chapel Hill School of Medicine](https://reader036.vdocuments.net/reader036/viewer/2022062515/56649d0d5503460f949e1b01/html5/thumbnails/38.jpg)
Putting it Together: Objective Structured Clinical Examination
(OSCE)• Multiple stations
• Each focused on a specific aspect of competence
• Stations might include• Manikins
• SPs
• ECG or X-ray interpretation
• Heart sounds
• Animal cadavers• Anastomosis• Laparoscopic vessel ligation
• Simulators
“In a way the OSCE is not an examination method; rather it is an examination format or framework into which many different types of test methods can be incorporated”
Ian Hart, 2001
![Page 39: Promoting the Development of Clinical Skills throughout the Continuum of Medical Education University of North Carolina – Chapel Hill School of Medicine](https://reader036.vdocuments.net/reader036/viewer/2022062515/56649d0d5503460f949e1b01/html5/thumbnails/39.jpg)
Putting it Together: OSCE
• Stations are usually short: 10-15 minutes
• Test is composed of 8-25 stations• Round-robin format
• At a bell, examinees rotate to next station
• Can accommodate as many examinees as stations
• Total score is calculated across all stations
![Page 40: Promoting the Development of Clinical Skills throughout the Continuum of Medical Education University of North Carolina – Chapel Hill School of Medicine](https://reader036.vdocuments.net/reader036/viewer/2022062515/56649d0d5503460f949e1b01/html5/thumbnails/40.jpg)
Work-based Methods
• Work-based assessment
• Real patient encounters
• Trainees are observed
• Judgments are made about their performance
“When your work speaks for itself, don't interrupt.”
Henry Kaiser
![Page 41: Promoting the Development of Clinical Skills throughout the Continuum of Medical Education University of North Carolina – Chapel Hill School of Medicine](https://reader036.vdocuments.net/reader036/viewer/2022062515/56649d0d5503460f949e1b01/html5/thumbnails/41.jpg)
Work-based Assessment
• Foundation Programme (in the UK)• Two-year program
• Bridge between medical school and advanced training
• Series of clinical placements
• Assessment Purpose• Determine fitness to progress to
the next level• Identify trainees in difficulty• Provide feedback• Establish accountability
• Three methods • Mini-Clinical Evaluation
Exercise (mCEX) • Directly Observed
Procedures (DOPs)• Case-Based Discussion
(CbD)
![Page 42: Promoting the Development of Clinical Skills throughout the Continuum of Medical Education University of North Carolina – Chapel Hill School of Medicine](https://reader036.vdocuments.net/reader036/viewer/2022062515/56649d0d5503460f949e1b01/html5/thumbnails/42.jpg)
Mini-Clinical Evaluation Exercise (mCEX)
• Process• List of patient problems
• Trainee picks a patient
• Assessor observes the encounter
• Focused clinical task
• Assessor rates:• Hx, PE, Communication, Clinical
Judgment, Professionalism, Organization/Efficiency
• Assessor provides feedback
• Takes 15-20 minutes
![Page 43: Promoting the Development of Clinical Skills throughout the Continuum of Medical Education University of North Carolina – Chapel Hill School of Medicine](https://reader036.vdocuments.net/reader036/viewer/2022062515/56649d0d5503460f949e1b01/html5/thumbnails/43.jpg)
Directly Observed Procedures (DOPs)
• Process• List of procedures• Trainee picks a patient• Assessor observes the
encounter • Procedure
• Assessor rates:• Preparation, Sedation,
Asepsis, Technical skill, etc.
• Assessor provides feedback
• Takes 15-20 minutes
![Page 44: Promoting the Development of Clinical Skills throughout the Continuum of Medical Education University of North Carolina – Chapel Hill School of Medicine](https://reader036.vdocuments.net/reader036/viewer/2022062515/56649d0d5503460f949e1b01/html5/thumbnails/44.jpg)
Case-Based Discussion (CbD)
• Process• List of patient problems• Trainee picks 2 case records
• Assessor selects one
• Discussion centered on the trainee’s notes
• Assessor rates:• Diagnosis, Treatment,
Planning, Professionalism, etc.
• Assessor provides feedback
• Takes 15-20 minutes
![Page 45: Promoting the Development of Clinical Skills throughout the Continuum of Medical Education University of North Carolina – Chapel Hill School of Medicine](https://reader036.vdocuments.net/reader036/viewer/2022062515/56649d0d5503460f949e1b01/html5/thumbnails/45.jpg)
Putting it Together: Work-based Assessment
• An OSCE “on the hoof”• Multiple encounters are
needed• Captured as feasible
during clinical training
• Multiple examiners are needed
• Encounters can be made to conform loosely to a problem list
• Ongoing, longitudinal assessments
![Page 46: Promoting the Development of Clinical Skills throughout the Continuum of Medical Education University of North Carolina – Chapel Hill School of Medicine](https://reader036.vdocuments.net/reader036/viewer/2022062515/56649d0d5503460f949e1b01/html5/thumbnails/46.jpg)
Criteria for Judging an Assessment
• How do simulation and work-based assessment perform against the criteria?• Validity • Reliability• Equivalence• Educational effect• Opportunity for feedback• Feasibility
![Page 47: Promoting the Development of Clinical Skills throughout the Continuum of Medical Education University of North Carolina – Chapel Hill School of Medicine](https://reader036.vdocuments.net/reader036/viewer/2022062515/56649d0d5503460f949e1b01/html5/thumbnails/47.jpg)
Validity
• What is validity?• Degree to which the
inferences based on scores are correct
• Does the test measure what it is supposed to measure?
• Simulation • Good content coverage
• Rare conditions• Errors cause no harm
• Good fidelity
• Work-based methods• Excellent content
coverage• Includes difficult to
simulate conditions
• High fidelity
![Page 48: Promoting the Development of Clinical Skills throughout the Continuum of Medical Education University of North Carolina – Chapel Hill School of Medicine](https://reader036.vdocuments.net/reader036/viewer/2022062515/56649d0d5503460f949e1b01/html5/thumbnails/48.jpg)
Reliability
• What is reliability?• If an assessment process
is repeated with the same trainees, they should get the same scores
• Physician performance varies considerably from patient to patient• The trainee must be
observed with several patients
• Assessors differ in stringency• The trainee must be
evaluated by different examiners
![Page 49: Promoting the Development of Clinical Skills throughout the Continuum of Medical Education University of North Carolina – Chapel Hill School of Medicine](https://reader036.vdocuments.net/reader036/viewer/2022062515/56649d0d5503460f949e1b01/html5/thumbnails/49.jpg)
Equivalence
• What is equivalence?• To compare examinees
they must have taken assessments that are equal in difficulty
• Fairness • Comparable meaning
• Simulation• Different examinees can
be given the same items• Security
• Statistical techniques help with different versions
• Work-based methods• Equivalence is a
problem that can be mitigated but not eliminated
![Page 50: Promoting the Development of Clinical Skills throughout the Continuum of Medical Education University of North Carolina – Chapel Hill School of Medicine](https://reader036.vdocuments.net/reader036/viewer/2022062515/56649d0d5503460f949e1b01/html5/thumbnails/50.jpg)
Educational Effect
“Students respect what you inspect.”
• Both simulation and work-based methods signal the importance of working with patients• Drives learning
![Page 51: Promoting the Development of Clinical Skills throughout the Continuum of Medical Education University of North Carolina – Chapel Hill School of Medicine](https://reader036.vdocuments.net/reader036/viewer/2022062515/56649d0d5503460f949e1b01/html5/thumbnails/51.jpg)
Opportunity for Feedback
• Feedback is critical to learning • General education (Hattie,
1999)• Meta-analysis of 12 meta-
analyses
• Feedback is among the largest influences on achievement
• Medical education (Veloski et al., 2006)
• Feedback alone is effective in 71% of studies
• Simulation• Amount of feedback varies
by method• Depends on deployment• Lower for instructor driven
methods • Higher for model driven
methods
• Work-based methods• Trainees rarely observed• Provides an excellent
opportunity for feedback following observation
![Page 52: Promoting the Development of Clinical Skills throughout the Continuum of Medical Education University of North Carolina – Chapel Hill School of Medicine](https://reader036.vdocuments.net/reader036/viewer/2022062515/56649d0d5503460f949e1b01/html5/thumbnails/52.jpg)
Feasibility
• There are significant resource constraints in most educational programs
• Simulation• Purchase, maintenance,
logistics
• Case development
• SP/Observer training
• Work-based methods• Faculty development
• Logistics
![Page 53: Promoting the Development of Clinical Skills throughout the Continuum of Medical Education University of North Carolina – Chapel Hill School of Medicine](https://reader036.vdocuments.net/reader036/viewer/2022062515/56649d0d5503460f949e1b01/html5/thumbnails/53.jpg)
Summary: Assessment of Skills and Performance
• Trainees must ‘show how’ • Simulation
• Can produce equivalent scores
• Work-based methods • Cover more patient problems• Can be more feasible
• Both methods• Require multiple patients and
examiners • Have positive educational
effects• Provide opportunities for
feedback
![Page 54: Promoting the Development of Clinical Skills throughout the Continuum of Medical Education University of North Carolina – Chapel Hill School of Medicine](https://reader036.vdocuments.net/reader036/viewer/2022062515/56649d0d5503460f949e1b01/html5/thumbnails/54.jpg)
Finding Opportunities
• Seeking out the “best practices” already in place across the organization
• Disseminating and seeding what is working to other areas
• Finding ways to maximize synergy of work already in place
![Page 55: Promoting the Development of Clinical Skills throughout the Continuum of Medical Education University of North Carolina – Chapel Hill School of Medicine](https://reader036.vdocuments.net/reader036/viewer/2022062515/56649d0d5503460f949e1b01/html5/thumbnails/55.jpg)
Opportunities Along the Continuum
• Assessment of team member performance
![Page 56: Promoting the Development of Clinical Skills throughout the Continuum of Medical Education University of North Carolina – Chapel Hill School of Medicine](https://reader036.vdocuments.net/reader036/viewer/2022062515/56649d0d5503460f949e1b01/html5/thumbnails/56.jpg)
Opportunities Along the Continuum
• Assessment of outcomes of a team’s performance
![Page 57: Promoting the Development of Clinical Skills throughout the Continuum of Medical Education University of North Carolina – Chapel Hill School of Medicine](https://reader036.vdocuments.net/reader036/viewer/2022062515/56649d0d5503460f949e1b01/html5/thumbnails/57.jpg)
Opportunities Along the Continuum
• Assessment of individual team members – using “standardized team members”
![Page 58: Promoting the Development of Clinical Skills throughout the Continuum of Medical Education University of North Carolina – Chapel Hill School of Medicine](https://reader036.vdocuments.net/reader036/viewer/2022062515/56649d0d5503460f949e1b01/html5/thumbnails/58.jpg)
Opportunities Along the Continuum
• Assessment of teams composed of members of several health professions
![Page 59: Promoting the Development of Clinical Skills throughout the Continuum of Medical Education University of North Carolina – Chapel Hill School of Medicine](https://reader036.vdocuments.net/reader036/viewer/2022062515/56649d0d5503460f949e1b01/html5/thumbnails/59.jpg)
Opportunities Along the Continuum
• Standardized Patient assessments/ OSCEs & simulations for:• Incoming residents• Residents moving into
supervisory roles• Residents at completion of
residency• New medical staff –
credentialing review and privileging
• Individuals who are re-entering practice
![Page 60: Promoting the Development of Clinical Skills throughout the Continuum of Medical Education University of North Carolina – Chapel Hill School of Medicine](https://reader036.vdocuments.net/reader036/viewer/2022062515/56649d0d5503460f949e1b01/html5/thumbnails/60.jpg)
Opportunities Along the Continuum
• “Secret Shoppers” -standardized patients in clinical settings assessing clinical skills of:• Residents• Faculty• New medical staff –
credentialing review and privileging
• Individuals upon re-entry into practice
![Page 61: Promoting the Development of Clinical Skills throughout the Continuum of Medical Education University of North Carolina – Chapel Hill School of Medicine](https://reader036.vdocuments.net/reader036/viewer/2022062515/56649d0d5503460f949e1b01/html5/thumbnails/61.jpg)
Most Important Consideration
• A Comprehensive Program based on • Well defined Purpose and Goals• Overarching Competencies and
Objectives• A detailed “Map” that covers the
timing and methodologies of assessments across the continuum
• Focused efforts on gaps in teaching and assessment
• A well thought out evaluation of the program
• Providing data and evidence supporting the benefit to patients and improvement in care
![Page 62: Promoting the Development of Clinical Skills throughout the Continuum of Medical Education University of North Carolina – Chapel Hill School of Medicine](https://reader036.vdocuments.net/reader036/viewer/2022062515/56649d0d5503460f949e1b01/html5/thumbnails/62.jpg)
Why Does It Matter?
• Initiatives focused on improving clinical skills, especially communication – through teaching and assessment - will be most successful in improving the quality and outcomes of care provided by health professionals
![Page 63: Promoting the Development of Clinical Skills throughout the Continuum of Medical Education University of North Carolina – Chapel Hill School of Medicine](https://reader036.vdocuments.net/reader036/viewer/2022062515/56649d0d5503460f949e1b01/html5/thumbnails/63.jpg)
THANK YOU
Let us continue on the journey together –
improving how we care for our
patients