APPD 2012 Spring Meeting

The Brink of a New Era in

Pediatric Medical Education: Will We Transform our Future?

Carol Carraccio, MD,MA

Director, Competency-based Assessment Programs American Board of Pediatrics

• Carol Carraccio has nothing to disclose

Why a New Era?

• The Pediatrics Community is in a pivotal position to connect initiatives whose “whole” >> “the sum of its parts” – New Accreditation System coupled with a

forward thinking RC – APPD LEARN – IIPE Partnership – PD School – Partnerships beyond Peds (AAMC, NBME) – Education as a continuum rather then silos

Building Blocks for the Continuum

• Milestones

• EPAs

Pediatrics Milestone Project

• Initiative spearheaded by ACGME and ABP • Charge:

– Refine the 6 ACGME competencies in the context of the specialty

– Define expected performance levels by year of training

– Identify or develop tools to assess performance

Where Are We Now?

• Completed the first of the 3 parts of the charge • Milestone Project document sent to Program

Directors • A pdf version is available on the ABP website at:

https://www.abp.org/abpwebsite/publicat/mileston

es.pdf

What Have We Learned So Far? APPD Survey

Strongly Disagree/ Disagree

Neutral Strongly Agree/ Agree

Helpfulness of Background (n = 124)

5.3% 18.4% 59.6%

Likely to Pursue References (n=114)

14.1% 32.5% 53.47%

Yes No

Interest in Collaboration to Study Milestones (n=115)

71.8% 28.2%

What Have We Learned So Far?

• Responses to open-ended questions – Concerns:

• Length/practicality • Educational jargon

– How PDs and APDs are using them

• Faculty development • Semi-annual reviews • Remediation • Incorporating them into assessments • Setting ILP goals

Where Are We Going?

• Not very far without your help

• Please engage with us in studying the milestones and EPAs

Studying Milestones & EPAs

• The Perfect Example of A Challenge Bringing Lots of Opportunities – Collaboration – Our learning & professional development – Being part of something bigger – Educational innovation – Informing and transforming the future of

medical education

So How Do We Get There?

• With a Vision, a Strategic Plan, and a

Collaborative and Coordinated Effort

• Through partnerships – APPD LEARN- IIPE – NBME Pilot Project – With colleagues across the educational

continuum

What Direction Do We Take?

• Study specific milestones – When do learners demonstrate the behaviors

described? – Learning roadmap & formative feedback

• Study the integration of behaviors across competencies – Is the learner a “good doctor”? – Reframe and ground the competencies in a

clinical context (EPAs)

“Entrustable Professional Activities”

• Professional life activities that define a specialty • Ground the competencies in our everyday work

(competencies in context) • Activities lead to some observable outcome • Complexity of these activities requires an

integration of knowledge, skills and attitudes across competency domains

• Example: Care of the well newborn

Ten Cate O, Scheele F. Competency-based postgraduate training: Can we bridge the gap

between theory and clinical practice. Acad Med;2007:82:542-547

The Value of EPAs

• Can be mapped to all competencies and

judiciously mapped to specific sub-competencies and their milestones

• Look at assessment from a different perspective that complements the view from the level of milestones

Milestones + EPAs: Critical Combination for Assessment

• Milestones assess how well a learner can accomplish some smaller part of a competency – A granular approach that is helpful for

feedback & remediation

• EPAs integrate competencies within a clinical context and assess clusters of behaviors – A holistic approach to insure that one can put

all of the pieces together to care for patients

The Value of EPAs

• New and valuable perspective on assessment of competencies – ENTRUSTMENT

• Can this learner practice this EPA without direct supervision?

• What level of supervision is necessary? • More intuitive question/answer than whether

someone is competent in PBLI, SBP…

EPAs: Getting Them Right

• Generic language in new RC requirements that

refers to EPAs • Survey of APPD Membership

– 6 questions – Draft list

• What’s missing? • What should be omitted?

Take Home Messages

• Milestones and EPAs complement each other and the combination is critical to meaningful assessment of learners

• Collaborations and partnerships to learn from what we are doing is the only path forward to true innovation and transformation

• We need your help to light the way

Thank You!