Preconference workshop Practical tips when implementing Entrustable Professional Activities

Olle ten Cate, PhD Center for Research and Development of Education

University Medical Center Utrecht, the Netherlands

Thai Medical Education Conference, Mahidol University, Siriraj Hostpital, Bangkok, Thailand, December 11-13, 2019

Overview – from 8.30 am to 12.00 noon 1. Welcome and getting acquainted – 15’

2. A small quiz – 15’

3. Introduction to CBME and EPAs – 35’

4. How to identify and reach consensus about suitable EPAs – 10’

5. Demonstrating focusgroup with nominal group technique – 45’

6. Break – 30’

7. Entrustment as assessment – 15’

8. The entrustment-based discussion technique – exercise – 30’

9. Wrap up – 15’

Getting aquainted

A small quiz Turn to your neighbor. In turn, try to explain:

1. What is the essence of competeny-based education?

2. What is a competency framework? Can you name an example?

3. What is an EPA?

4. What are the 5 “Dreyfus” stages of competence development?

5. What are the common 5 levels of supervision, used for EPAs?

Brief introduction to CBME and EPAs

Why Competency-Based Medical Education? There simply is no other pursuit of medical training. Despite:

- ..struggles to define competence

- ..difficulties in assessment of medical competence

- ..critics of CBME frameworks in the literature

Competency-Based Medical Education

• Better, broader, more specific description of the physician

• A move from assuming competence to assessing competence

• Licensing and physicians and registering specialists only when they meet standards, based on competence, not just time in training

• An imperative for high quality, safe health care

Two examples:

CanMEDS framework (Canada) >

< ACGME framework (USA)

competent

10-20% Group at

risk

Progress Test scores 2005-2009 for all Dutch radiology residents

PGY 1 PGY 2 PGY 3 PGY 4 PGY 5 Ravesloot et al 2012

Trainees who may

accellerate

CBME challenges

General acceptance worldwide, but..

• CBME frameworks can become analytical and detailed

• Competencies are sometimes rather abstract and general

• Clinical teachers do struggle with assessment

Units of professional practice (tasks) that may be entrusted to a learner to execute unsupervised, once he or she has demonstrated the required competence

Shift of focus:

from individual competencies to the work that must be done;

EPAs integrate all necessary competencies

Entrustable Professional Activities

Ten Cate 2005

E.P.A. • Entrustable: acts that require trust – by colleagues,

patients, public

•Professional: confined to occupations with extra-ordinary qualification and right

•Activities: tasks that must be done

EPAs ground competencies in daily practice

person-descriptors

knowledge, skills, attitudes, values

• content expertise • health system knowledge • communication ability • management ability • professional attitude • scholarly skills

Competencies

work-descriptors

essential units of professional practice

• discharge patient • counsel patient • lead family meeting

• design treatment plan • Insert central line • Resuscitate patient

EPAs

Competencies versus EPAs

Person Work

the ability to do something successfully or efficiently*

that something that is (trusted to be) done successfully or efficiently

*Oxford dictionary

Medical expert

Communicator

Collaborator

Leader

Health advocate

Scholar

Professional

++

+

+

+

+

+

++

++

+

+

+

+

++

++

+

++

++

+

++

+

++

EPA1 EPA2 EPA3 EPA4 EPA5

..th

en

co

mp

ete

nci

es

can

be

infe

rre

d

if assessment is focused on EPAs..

EPAs require multiple competencies

Pangaro & ten Cate 2013

PERSON DESCRIPTORS

WORK DESCRIPTORS

Growth of competence over time

training deliberate professional practice

proficient

expert

competent

advanced

novice

Dreyfus & Dreyfus 1986; ten Cate et al, 2010

Ready for unsupervised practice

EPA5

Competency curves of one trainee for various EPAs

training deliberate professional practice

EPA1

EPA4

EPA2

EPA3

Compe- tence

thres- hold

Justified entrustment decisions

Loss of trust

Entrustment decisions: Five levels of supervision, reflecting increasing trust in trainee autonomy

1. Be present but no permission to enact EPA

2. Practice EPA with direct (pro-active) supervision

3. Practice EPA with indirect (re-active) supervision

------------------------------------------------------[threshold]---

4. Unsupervised practice allowed (distant oversight)

5. May provide supervision to junior learners

Ten Cate et al 2010

Ad-hoc decisions of entrustment occur daily in clinical education Summative decisions of entrustment are based on multiple workplace-based assessments and focus on increased autonomy. Sometimes called a STAR

Portfolio of:

trainee Jones

PGY1 PGY2 PGY3 PGY4

EPA a 1 2 2 2 3 4 4 5

EPA b 1 1 2 2 2 3 3 4

EPA c 2 2 3 4 5 5 5 5

EPA d 2 3 4 4 4 4 5 5

An individualized workplace curriculum

Graded supervision allows for

1 Observing the activity

2 Acting with direct, pro-active supervision present in the room

3 Acting with (re-active) supervision available within minutes

4 Acting unsupervised, i.e. under clinical oversight

5 Acting as the supervisor to a junior

Recommended full EPA Description 1 Title of the EPA

2 Specification and limitations

3 Potential risks in case of failure

4 Most relevant domains of competence

5 Required knowledge, skills, attitude and experiences

6 Information sources to assess progress and support summative entrustment

7 Which entrustment-supervision expected at which stage of training?

8 Time period to expiration if never practiced

1. Title

• This just reflects work or activity to be done in health care (not stated as skill or ability)

• Avoid adjectives that describe skill or competence

• A short as possible, as long as needed

2. Specifications and Limitations

• Description of what is included in activity, what is not included, and context. Try to make a bulleted or numbered list of components chronologically

• Are there any limitations regarding complexity or setting applicable when the learner will be formally entrusted (certified) with this EPA?

3. Potential risks in case of failure

• List possible complications and adverse events if done improperly

• Think of harm to patients, undue costs or wasted resources, psychological damage to team or hospital, etc.

• Damage to the learner

• (Limit list to 2-5 most likely adverse events)

4. Most Relevant Competency Domains

• Map to institutional or national competencies framework (if any) and learning objectives. This to help guide assessment.

• If CanMEDS is used, try to limit to 2-4 roles that seem most relevant for this EPA

5. Required Knowledge, Skills, Attitudes and Experience • Which knowledge, skills, attitudes (KSAs) are expected before

student can be trusted to carry out is EPA? This information will guide students / residents.

• Experiences are essential prior education, completed tests, number of performed procedures etc.

• These are KSAs and experiences that must be completed before summative entrustment decision about decrease of supervision can be considered.

6. Assessment Information Sources to assess progress and make summative decisions • What information should be used to determine progress and

ground a summative entrustment decision

• 4 common WBA assessment procedures: (i) short observations (MiniCEX, DOPS etc), (ii) case-based discussion, (iii) Longitudinal evaluation (multi-source feedback or 360O

evaluation), (iv) evaluation of products

• Think of how many of these must be satisfactory before a summative entrustment decision.

7. Entrustment / Supervision Level expected at which stage of training • When are trainees expected to reach which level of

entrustment or supervision for this EPA? In many countries: level 3 (indirect supervision) at end of med school, level 4 (unsupervised practice) at end of residency.

• There are more detailed levels in between.

8. Expiration date

• If the EPA involves major risks, competence should be maintained. How long a period of non-practice should lead to expiration of certification at level 4?

• Consequence: the practitioner will require supervision (level 2 or 3) again until readiness for unsupervised practice is warranted.

Example: Resuscitation of the multiple trauma patient in the Emergency Room

2 Resuscitation of trauma patients of all age groups, in the Emergency Room. Active participation in the trauma team.

Assessment and control of vital functions. Pain management in trauma patients.

No limitations

3 Unnecessary suffering for patient; Failure of resuscitation teamwork; Increasing preventable morbidity or even mortality

4 Medical expert

Communicator

Collaborator

Manager

Health advocate

Scholar

Professional

5 Trauma mechanisms & pathophysiology; Organization of trauma care; Collaboration in the trauma team; Trauma

diagnoses & treatment; Primary & secondary survey; Trauma airway management; Emergency IV1 & IO2 access;

Emergency thoracostomy; Hemorrhage / massive transfusion; Emergency Room registration procedures

6 5 SPOs and 5 trauma CBDs (different days and assessors), incl. trauma airway management, emergency IV & IO

access and emergency thoracostomy; LPO over >3 weeks (MSF); 2 trauma simulator achievement tests passed

7 Level 4 (unsupervised practice) in PGY 4 of anesthesiology training

8 Six months after non-practice

Identifying and reaching consensus about suitable EPAs

How does a program arrive at set of valid EPAs? Frequently used methods:

- Local expert committee meetings

- Document analysis (analyzing existing descriptions of a profession / specialty)

- Focus Groups using a Nominal Group Technique

- Delphi studies: expert consultation about a predefined set of EPAs en multiple rounds of consultation

Identifying and reaching consensus about suitable EPAs

How does a program arrive at set of valid EPAs? Frequently used methods:

- Local expert committee meetings

- Document analysis (analyzing existing descriptions of a profession / specialty)

- Focus Groups using a Nominal Group Technique

- Delphi studies: expert consultation about a predefined set of EPAs en multiple rounds of consultation

Focus group using a nominal group technique Procedure

1. Careful selection of (5 to 10) participants; well informed about the EPA concept

2. Plan a meeting of 1.5 hours minimum. Facilities: arrangement around a table; flip chart; pen and notebook for everyone; skilled facilitator. Optional: audio recording; ethical approval and informed consent if journal publication is considered

3. Start with summarizing purpose and summarizing EPA concept (focus on issues of size and number)

4. Provide 5 (to 10) minutes in silence to have each participant generate as many EPAs as possible – write down on their notebook

5. Ask first participant to name one (no more) relevant EPA. Clarify if needed, and write on the flipchart, visibly for all. Then ask next participant for one tha is not identical. And so on, until all proposed EPAs are exhausted. Number the EPAs.

6. Finally let everyone vote for essential EPAs and optional EPAs

7. Optional: report the results later to the group and have everyone react on priotities and comprehensiveness

Volunteers for fishbowl focus group demonstration

EPAs in the service years after UME

• What are the most important activities that newly graduates will have to do when they work in a rural primary practice for the service years?

• All take five minutes to write down at least 3

• Then watch the focusgroup session with a section of participants

• Later, others can add ideas

Results from a similar exercise in Islamabad, Pakistan, October 2019

Assessment approaches in the workplace

1. Short practice observations (miniCEX, DOPS)

2. Longitudinal practice observations, with 360O

3. Case-based/entrustment-based discussions

4. Product evaluation (EHR, reports)

Entrustment decision making

How to decide when a learner can work start working unsupervised?

The flow of workplace-based assessment data

feedback

Multiple times ad hoc entrustment

with EPA executions

Observation

evaluation

Clinical Competency committee (CCC) or

Entrustment Committee receives and evaluates

request for a summative entrustment decision, drawing on portfolio

data

PORTFOLIO

Yes!

Entrustment decision making ties learner assessment with patient care

The trust concept in EPA-based assessment

• Trusting someone is making yourself vulnerable

• Calculated risk that adverse events are acceptable

• Graduates will be certified to carry out activities that supervisors have not been able to observe and leaners may have never encountered

• Entrustment decisions require estimation of adaptive competence to cope with unfamiliar situations

• Entrustment-based discussions aim at estimating risk

Entrustment-Based Discussions

The Clinical Teacher, 2017

Entrustment-based discussion procedure Four steps; 10-15 min

1. Explain the activity. The learner explains the case or EPA

2. Show depth of knowledge. Learner relates findings to prior knowledge (anatomy, (patho)physiology, diagnostic tests, indications, therapeutic options etc)

3. Show awareness. How was the learner prepared to cope with potential risks and complications with this EPA?

4. What if…? What would the learner have done if the patient were different (in gender, culture, medical history) or unexpected findings arose?

Group work

1. Establish groups of 3 (including an observer)

2. Read the two scenarios about your neighbor kids and think with each of at least two questions that will inform your decision to trust (5’)

3. Choose one scenario and conduct one EBD (10’)

4. Debrief, led by observer (5’)

5. Change roles, conduct and debrief one more EBD (15’)

6. Plenary discussion (15’)

Plenary wrap-up

• Smal quiz

• Introduction to EPAs

• Consensus procedures and focus group with NGT exercise

• Entrustment based discussion exercise

• Interested in more? Think of the 3-day International EPA

course 2020 in Kuala Lumpur (or Washington, Utrecht Bogota)

3-day intensive international course Ins and Outs of EPAs

• March 5-7, 2020, Kuala Lumpur, Malaysia (after Ottawa conference)

• April 30-May 2, 2020, Washington DC, USA

• June 11-13, 2020, Utrecht, the Netherlands

• September 17-19, 2020, Bogotá, Colombia