kirk lalwani, md, frca, mcr associate professor, chair, resident evaluation and clinical competence...
TRANSCRIPT
Milestones: Are You Ready for July 1st ?
Kirk Lalwani, MD, FRCA, MCR
Associate Professor, Chair,
Resident Evaluation and Clinical Competence Committee,
Anesthesiology and Perioperative Medicine.
On behalf of the AMIGOS(Anesthesiology Milestones Implementation
Group Operatives)
Disclosure
No conflict of interest
Some of my slides are borrowed from the faculty educational resources on the ACGME website
Implementing Milestones
Objectives:
Outline the Next Accreditation System (NAS)
Define a Milestone
Review our Milestones strategy
Discuss assessment of Anesthesiology Milestones
Abbreviations
NAS- Next Accreditation System
CLER- Clinical Learning Environment
ACGME- Accreditation Council for Graduate Medical Education
RRC- Residency Review Committee
Accreditation Council for Graduate Medical Education
© 2013 Accreditation Council for Graduate Medical Education Information Current as of December 2, 2013
The Next Accreditation System
Why ‘Next Accreditation System’ (NAS)?
“Self-regulation is a fundamental professional responsibility, and the system for educating physicians answers to the public for the graduates it produces.”
“The Next GME Accreditation System – Rationale and Benefits”
Nasca T.J., Philibert I., Brigham T., Flynn T.C.
N Engl J Med 2012; 366:1051-1056
Goals of NAS
Strengthen resident development in Professionalism Communication Skills Systems Based Practice Practice Based Learning
Enhance public accountability More explicit definition of a good physician (Milestones) Patient safety (Clinical Learning Environment Review Program)
Reduce burden required for accreditation
‘Next Accreditation System’ …in a Nutshell
Accreditation on the basis of educational outcomes
NAS : Advance from an episodic ` biopsy ’ model to annual data collection or ` continuous oversight ’.
RRCs will measure compliance through the evaluation of annual program data elements.
The Next Accreditation System
ContinuousObservations
Identify Opportunities for Improvement
Program MakesImprovement(s)
AssessProgramImprovement(s)
PromoteInnovation
© 2013 Accreditation Council for Graduate Medical Education Information Current as of December 2, 2013
The Next Accreditation System
We will now have annual data collection
Trends in annual data Milestones Resident and Faculty Surveys Resident and Faculty Scholarly activities Case Log Data ITE / ABA Examination pass rates
Clinical Learning Environment Review Visits
• Oversight of • Transitions of Care• Duty Hours Policy• Fatigue Management and Mitigation• Education and Monitoring of Professionalism• Involvement in Institutional Quality Improvement and
Safety initiatives• Supervision policies
Additional information available at: http://www.acgme-nas.org/CLER
Milestones
Click icon to add picture
What are Milestones?
Specific behaviors, attributes, or outcomes in the six general competency domains to be demonstrated by residents during residency.
(ACGME and Specialty Boards)
Skill and knowledge-based development that commonly occurs by a specific time.
Why ????
ACGME requires it Program Accreditation
Who, When, and How???? All residents in all ACGME-accredited programs
Subspecialty fellows from July 2015
New assessment methodology as well as existing tools
6-monthly reporting of Milestones to ACGME in January and July
Reporting starts July 2014
Expected Benefits of Milestone Assessments
For Residents
Clarify expectations
Feedback should identify specific areas to work on
Earlier identification of under-performers
Aspirational goals for residents who exceed expectations
Expected Benefits of Milestone Assessment
For the Program Curriculum development Accreditation requirements Earlier identification of under-performers
For the Public Better definition of what a physician can do at the
completion of training Program is accountable to a common standard Possible use for board certification
17
MinimalistCCC assigns milestone levels using existing tools
ComprehensiveProgram changes assessment system, invests time in faculty development and education to improve rater expertise and CCC validity
PragmaticProgram implements new milestone aligned tools, which help residents understand expectations
Milestones Development Strategy
Process: June 2013Assembled AMIGOS Group from RECCC, faculty, and residents to represent all clinical
areas
Survey to rank importance of each Milestone, following which 17 / 25 chosen
Small groups assigned 2-3 Milestones each to develop instruments or methodology
Large group review and feedback
Education office worked with Evalue to incorporate tools into system
General Strategy“Good now is better than perfect later….”
Priorities
Ease of use Should not be burdensome for residents, faculty, RECCC or education
office staff No PAPER ! App for ‘on-the-go’ assessments - procedures, H&P
Implementation feasibility Ready by July 2014 Integrate with existing system (Evalue)
Ensuring adequate data to assess Milestones comprehensively A Milestone for every competency 17/25 Milestones chosen
Work in progress, as refinements will be made and new tools added over time
Which?
Which Milestones did the AMIGOS select?
Medical Knowledge (MK): 1
MK1- Knowledge of biomedical, clinical, epidemiological, and social sciences as outlined in the American Board of Anesthesiology Content Outline
ITEAKTABA Basic ExaminationACLS certificationDirect Clinical Observation
Patient Care (PC): 8 of 10
PC1-Preanesthetic Evaluation, Assessment, and Preparation PC2- Anesthetic choice and conduct PC3- Periprocedural pain management PC4- Management of perianesthetic complications PC5- Crisis management PC6- Triage and management of critically ill patient in a non-
operative setting PC7- Acute, chronic, and cancer related pain consultation and
management PC8- Technical skills: Airway management PC9- Technical skills: Monitoring and Equipment PC10-Technical skills: Regional anesthesia
Example of PC-8 Assessment
PC-8 (Technical skills: Airway management)Direct Clinical Observation (DCO)
Daily evaluation form Airway management checklist
Objective Structured Clinical Examination (OSCE) / Objective Structured Assessment Test (OSAT)
Simulation / Standardized Patient (SP)
Professionalism (Prof): 3 of 5
Prof1- Responsibility to patients, families, and society
Prof2- Honesty, integrity, and ethical behavior Prof 3- Commitment to institution,
department and colleagues Prof4- Receiving and giving feedback Prof 5- Responsibility to maintain personal,
emotional, physical and mental health
Example of Prof-1 Assessment
Prof 1 (Responsibility to Patient, Family, and Society)360 degree evaluations
Patients Healthcare team members Peer evaluations Administrative staff
Direct Clinical Observation (DCO) Daily evaluation form Written comments
Interpersonal and Communication Skills (ICS): 2 of 3
ICS1- Communication with patients and families
ICS2- Communication with other professionals
ICS3- Team and Leadership skills
Practice Based Learning (PBL): 2 of 4
PBL1 Incorporates quality improvement and patient safety initiatives into personal practice
PBL2- Analysis of practice to identify areas in need of improvement
PBL3- Self-directed learning PBL4- Education of patient, family, students,
residents and other health professionals
Systems Based Practice (SBP) 1 of 2
SBP1- Systems-based approaches to patient care
SBP2- Coordination of patient care within the healthcare system
How will we Assess Milestone Levels? Milestones are a summary of how a resident is
progressing based on information from other sources
Milestones are not assessment tools to be used for gathering detailed information
They do not replace end-of-rotation forms, simulation, 360’s, Daily evaluations etc.
More pieces of data allow for more precision
There will be a minimum number of evaluations required for procedural assessments
Clinical Competency Committee (RECCC)
Clinical Competency Committee
End-of-Rotation
Evaluations
Peer Evaluations
SelfEvaluations
Case Logs
Student Evaluations
Patient/Family
Evaluations
ProfessionalismIndicators like
pharmacy, billing, evals
etc
Nursing and Ancillary
Personnel Evaluations
Assessment of Milestones
Clinical Workplace Evaluations
Mock Orals
OSCE
ITESimLab
UnsolicitedComments
Anesthesiology Milestones
Level 4 is the Graduation Target Not every resident will achieve Level 4 in every Milestone
Residents required to substantially meet most Milestones at Level 4
Residents are not expected to achieve Level 5 during residency
Residents may achieve a level of competency in specific Milestones sooner than expected
Milestone Template
PC1. Preanesthetic Evaluation, Preparation
Level 1 Level 2 Level 3 Level 4 Level 5
Performs comprehensive H&P
Identifies medical issues that may affect anesthesia care
Identifies elements and process of informed consent
Identifies medical issues relevant to anesthesia care
Optimizes preparation of non-complex patients
Obtains informed consent but recognizes when assistance is needed
Identifies medical issues relevant to subspecialty anesthesia care
Optimizes preparation of complex or subspecialty patients
Obtains informed consent of patients with complex problems or requiring subspecialty anesthesia care
Performs assessment of complex or critically ill patients with conditional independence
Obtains informed consent in complicated clinical situations with conditional independence
Independently performs assessment for all patients
Independently serves as a consultant to other members of the healthcare team
Consistently ensures that informed consent is obtained by using all available resources
General Competency Developmental Progression or
Set of Milestones Subcompetency
Milestone
Instruments to Measure Milestones Daily Evaluation Form for OR (Evalue) Daily Evaluation Form for ICU (Evalue) Daily Evaluation Form for Pain (Evalue) Technical Skills forms (Evalue) (‘App’?)
Arterial line CVP Line Nerve Block / Epidural / Caudal Airway Management
Peer evaluations Assessment of Patient Interaction Regional assessments Simulation assessments
‘QUASAR’ Project:Quality in Anesthesiology, Systems Assessment and Research
Resident ‘QUASAR’ Project: Quality in Anesthesiology, Systems Assessment and Research Quality Improvement Systems Based Practice Practice-Based Learning
CA1- Background research, identify area of study
CA2- Identify quality measure to be studied and change to be implemented, study effects, refine
CA3- Write summary, submit abstract to national conference, or write a manuscript for publication
Summary
Next logical extension of competency-based evaluation
Mostly familiar, not burdensome Levels 1 and 2 do NOT signify a ‘Fail’ grade Evaluations will be more specific and focused Direct clinical observation to evaluate
procedures and patient interactions
We Need Your Data !
APOM NEEDS YOU
Acknowledgements AMIGOS
Nicole Conrad Oumou Diallo Judy Freeman Julio Gonzalez Karen Hand Izumi Harukuni Amy Miller Juve Diana Kim Ed Kahl Dean Lao David Larsen Kim Mauer Michele Noles Annie Riley Peter Schulman Katie Seligman Chris Swide Linda Wylie David Wilson Glenn Woodworth