The best health for all. The best care for all.

Competence by Design

Kenneth A. Harris, MD FRCSC

Deputy CEO & Executive Director, Office of Specialty Education

February 3, 2016

Discussion Topics

• Overview of CanMEDS 2015 and CBD

• Change fundamentals

• The road to CBD

• Assessment & ePortfolio

• Minimum Data Set (MDS)

• Ongoing collaboration

• How to reach us

• Available resources

2/6/2017 3

Key Concepts

Define Milestones & EPAs

2/6/2017 5

Milestones within an EPA

2/6/2017 6

Milestones and EPAs within Four Stages of Residency

2/6/2017 7

2/6/2017 8

Assessment

• The nature of assessment will change, but the method of service delivery will not • CBD is a hybrid approach to competency-based/outcomes

focused training

• Time will be utilized as a resource to structure training experiences and maintain a high standard of service delivery

2/6/2017 9

Specific Themes

2/6/2017 10

Specialties Implementing CBD July 2017

• Anesthesiology

• Otolaryngology – Head and Neck Surgery

2/6/2017 11

Readiness

• Readiness is multifaceted and involves a variety of partners beyond the Royal College

• Activities are underway with PGDs & CBME Leads to reach shared goal of implementation by July 1st 2017

2/6/2017 12

A Collaborative Approach with Local Champions & Change Agents

Deans, PG Deans

• Align visits for outreach with schools, collaboratively with Clinical Chairs, Hospitals & Ministries

Program Directors

• Co-create the design for their discipline in SC CBD workshops & bring it home as local champions

Faculty

• Faculty development resources and toolkit

Residents

• Regular engagement sessions with RDoCs and FMRQ

CBME National Faculty Leads

• Close collaboration with College

PG Managers, Program Administrators

• ICRE & advisory groups

2/6/2017 13

The best health for all. The best care for all.

Change Fundamentals

Rhonda St. Croix

Change Advisor

How people change

2/6/2017 15

Essentials for successful change

•Shared purpose

•Lead by example & build ownership/commitment

• Build distributed network for the change •Leverage relationships & connections

• Change spreads via peer-to-peer influence starting with early innovators & champions

•Change behaviours, get culture • Small set of inter-connected behaviours creates high impact

•Gather feedback to improve

• A deep cycle of learning & innovation

2/6/2017 16

Top strategies that work Create & share your CBD story – compelling why

• Link CBD to needs & issues people care about

• What’s the payoff

Build your local network for CBD – create the waves

• Local leadership/sponsorship & change agents

• Give people time to understand, ask questions, address concerns/doubts & make CBD their own

• Liaison roles, retreats, meetings, committees, communiques

Make progress visible

• early disciplines implementing July 1

Translate CBD into simple new behaviours by role

• EPAs, assessments, competence cte, real time feedback

Start now (try/test) & evolve

Support – locally and Royal College

2/6/2017 17

The best health for all. The best care for all.

The Road to CBD

Dr. Jolanta Karpinski

Associate Director, Specialties Unit

2/6/2017 19

Trying things on

Working the workshops

Cohort rollout

What Does CBD Rollout Mean? Creation of new version of specialty specific documents in CBD format

Royal College approval of new version of specialty standards

Training programs begin applying new standards to a group of residents

Residents trained under new standards Program faculty teach and assess

interact with Royal College using CBD standards

Residents trained in CBD system enter CPD PGME offices and programs apply CBD policies and procedures

2/6/2017 20

Cohort Implementation Plan

2/6/2017 21

2/6/2017 22

2/6/2017 23

CBD is in

our future

Activation Workshops

and design

Implementation

planning

CBD is

our

reality! 36 specialties and subspecialties

Cardiology Anatomical Path Anesthesiology

Clin Imm and Allergy Cardiac Surgery Forensic Path

General Surgery Critical Care Med Medical Oncology

Geriatric Medicine Emergency Med Otolaryngology – Head

and Neck Surgery Neo-Perinatal Med Gastroenterology

Nuclear Medicine General Int Med Surgical Foundations Obs and Gyn General Path

PMR (Rehabilitation) Internal Medicine

Plastic Surgery Nephrology

Psychiatry Neurosurgery

Respirology Pediatrics

Rheumatology Radiation Onc

Urology

Working the workshops

2/6/2017 24

CBD workshops

Specialty committee supported to attend

• All members and up to 3 guests

Work supported by Royal College CBD team

• Clinician educator, writer

Workshops entail

• Reinventing discipline in CBD framework - Developing EPAs and work-based program assessment for the discipline

- Creating new document suite with CanMEDS 2015 and CBD content and format

• Planning for implementation

2/6/2017 25

Core Components of CBME • Competencies are clearly articulated.

• Competencies are arranged progressively.

• Learning experiences facilitate the progressive development of competencies.

• Teaching practices promote the progressive development of competencies.

• Assessment practices support and document the progressive development of competencies.

2/6/2017 26

2/6/2017 27

2/6/2017 28

Examples of EPAs

2/6/2017 29

In the real world Driving on G1

In medicine Running codes

Progression in Internal Medicine

• Assessing, diagnosing, and initiating management for patients with common acute medical presentations in acute care settings

• Assessing, diagnosing and managing patients with complex or atypical acute medical presentations

• Assessing and managing patients in whom there is uncertainty in diagnosis and/or treatment

2/6/2017 30

Oversimplification of current system

Rotations • Rotation specific objectives

In-training evaluation reports • Global assessment • Timely? • ?linked to clinical activities

Progression through training • Onus to demonstrate lack of progression

2/6/2017 31

Oversimplification of CBD

• Training is sequenced in stages

• Supervisors and residents focus on EPAs designated for that stage

• Clinical and other training experiences are the basis to acquire the skills to achieve those tasks

• Supervisors document observations of resident’s performance on EPAs

• Coaching model: what can I do better?

• Competence Committees review progress

• Residents progress through stages

2/6/2017 32

Assessment in CBD

• Assessment for learning

• Re-emphasis on assessment in the workplace

• Assessment focused on discrete clinical tasks

• Assessment for purposes of learning

• Requirement for demonstration of competence

• Assessment as collection of evidence of progression

• Distinct promotion from one stage to another - Competence Committees

• Tracking and coaching along the pathway

2/6/2017 33

Assessment for learning

• Role of coaching

• Guidance for learning

• Frequent

• Link to deliberate practice

2/6/2017 34

Assessment for progression

2/6/2017 35

Assessment of EPA Achievement

• EPA Assessment Encounters

• Point in time

• Single rater

• Specific context

2/6/2017 36

Putting it together: Wisdom of Crowds

2/6/2017 37

Assessment of EPA Achievement

• EPA Assessment Encounters

• Point in time

• Single rater

• Specific context

2/6/2017 38

EPA Achievement - Multiple times - Multiple raters - Multiple contexts

Getting ready for implementation

2/6/2017 39

Implementation ideas: plan

2/6/2017 40

Implementation ideas

2/6/2017 41

Sample of some CBD Resources – • Introduction to competency-based medical education (CBME) and

Competence by Design (CBD): Video (7:33)

• Understanding Entrustable Professional Activities (EPAs): Video (5:11)

• CBD Cheat Sheet – brief summary of all key CBD concepts

• EPA Fast Facts – provides a brief overview of EPAs and how will be used within CBD

• Competence by Design (CBD): An Overview (March 2016 PPT)

• Answering your CBD Questions: Part 1 - The Rationale

• Answering your CBD Questions: Part 2 – CBD Implementation

• Answering your CBD Questions: Part 3 – Glossary of CBD and CBME Terms

2/6/2017 42

Implementation ideas: do

Reinforce current strengths

• Value of clinical experiences

• Supervisor interaction with resident

New behaviours

• Focused, explicit clinical tasks

• More frequent “small” assessments

• Resident is “the last patient of the day”

• New language: “let’s go learn!”

New structures

• New clinical experiences

• Competence Committees

2/6/2017 43

The best health for all. The best care for all.

Assessment Essential Assessment Elements

Minimum data set (MDS) ePortfolio

Jane Fulford, CPA, CGA

Chief Information Officer

Assessment Essential Elements Key Points

• Agreement with the ‘Six Essential Elements of the CBD Assessment Strategy’

• Use of the assessment forms is not mandatory, provided faculties are in compliance with both the required use of the Six Essential Elements and the transfer of data related to the minimum data set

2/6/2017 45

Six Essential Elements of the Assessment Strategy

1. Assessment requirements, as defined by the Specialty committee, and

inclusive of:

• EPAs, as defined by the Specialty Committee

• CanMEDS-based milestones, as defined by the Specialty Committee

2. Increased emphasis on direct and indirect observation

3. Many low-stakes observations of focused clinical tasks

4. Narrative, actionable, timely, concrete recorded feedback

5. Curation, collation, and group decision-making by a Competence Committee

6. Stages and progression of increasing entrustment, facilitated by group

entrustment decisions at the Competence Committee level

2/6/2017 46

Minimum Data Set (MDS) | Key points

•An agreed upon sample of assessment data will be transferred from PGME offices to the Royal College via a third party vendor as a method to: • Maintain oversight of the CBD system

• Enable Royal College operational functions (e.g., exam planning and credentialing)

• Ensure continuous quality improvement

2/6/2017 47

Minimum Data Set | Key Points • Agreed to the transfer of a specific data set to the Royal College

• Domain I – Progression, credentialing, examinations and planning • Learner identified to RC at point of entry into PGME

• RC Status Categories for learners identified by Faculty/Program, as decided by Competence Committee: • Resident promoted to exam-eligible and,

• Resident is certification-eligible

• Domain II and IV – Learning Analytics, Benchmarking and CQI of overall system and CBD redesign • Aggregate, de-identified learner assessment and system-tracked data shared with

the Royal College will include: • EPA Achieved status, time-stamped

• Learner Action (promote to next stage), time stamped

2/6/2017 48

Minimum Data Set | Key Points

•Domain III – Progressive Accreditation System • Detailed discussion deferred while further development work continues on the new standards and accreditation process

2/6/2017 49

Minimum Data Set | Key Points

Data Stewardship Committee

• Ensure oversight of types of assessment data collected and transferred to the Royal College

• Facilitate discussion and transparency among key stakeholders in the new CBD system and safeguard privacy concerns

• Continuous review of data points shared within MDS in order to ensure appropriateness and ability of the MDS to answer key questions, ensure rigorous standards, and measure outcomes of the CBD system

2/6/2017 50

2/6/2017 51

Your Input Matters

Share your input with the Royal College via:

• cbd@royalcollege.ca

• www.facebook.com/TheRoyalCollege

• https://twitter.com/Royal_College

• www.linkedin.com

For more information, visit:

• www.royalcollege.ca/cbd

• www.royalcollege.ca/cbd/resources

2/6/2017 52