blind-dating in pt education where is the trust?...m m mm domains of competencies (englander r, et...

9/30/2019

1

BLIND-DATING IN PT EDUCATION – WHERE IS

THE TRUST?Scott Burns, PT, DPT, OCS, FAAOMPT

Mary Sinnott, PT, DPT, MEdRebecca Vernon, PT, DPT, NCS

Jane Fagan, PT, DPT, OCS

What are we talking about today?

Objectives

• Discuss and explore the historical background of entrustable professional activities in healthcare.

• Examine the possible uses in physical therapy education.

• Explore the relationship between competency based education and entrustable professional activities.

• Explore the development and implementation of an approach in a PT program.

• Apply the 5 stage approach to developing entrustableprofessional activities for physical therapy

Background on Entrustable Professional Activities and Competence-Based Education

CBE = Competence-Based Education

• A progression of competence along a series of defined milestones on the way to the explicit outcome goals of training (Frank, 2010)

• Competencies should be: (ten Cate, 2005)– Specific

– Comprehensive (include knowledge, attitude, and skill)

– Durable

– Trainable

– Measurable

– Related to professional activities

– Connected to other competencies.

9/30/2019

2

Miller’s Pyramid of Assessment of Competence (1990)

KNOWS(Knowledge)

KNOWS HOW(Competence)

SHOWS HOW (Performance)

DOES(Action)

Temple’s experience

• Skills Checks, lab practicals, clinical simulations for psychomotor components of competencies

• Written examinations and oral presentations to assess knowledge

• Attitudes?

– APTA Professionalism: Core Values Self Assessment

– Developed a Professional Behaviors Self Assessment using the work of May et al (1995) and Santasier et al (2007)

Generic Abilities (May et al)

• Commitment to learning• Interpersonal skills• Communication skills• Effective use of time and resources• Use of constructive feedback• Problem solving• Professionalism• Responsibility• Critical thinking• Stress management

Entrustable Professional Activities and CBE

• First identified by ten Cate in 2005

• Need to achieve competence in order to be trustworthy

– What is the threshold and what is the progression?

• EPAs are independently executable, observable, and measurable

– EPAs translate competencies into practice

– EPAs usually require multiple competencies in an integrated and holistic nature (ten Cate, 2013)

EPA Definition (AAMC, 2014)

• An EPA is an activity that all entering residents should be expected to perform on day 1 of residency without direct supervision.

• EPAs are units of professional practice, defined as tasks or responsibilities that trainees are entrusted to perform unsupervised once they have attained sufficient specific competence.

The EPA(The desired outcome)

Patient Care

Knowledge of Practice

Practice-based Learning and Improvement

Interpersonal and

Communication Skills

Professionalism System-based

PracticeIPP/IPC

Personal and Professional

Development

C1

C2

C1

C2

C1

C2

C1

C2

C1

C2

C1

C2

C1

C2

C1

C2

M

M

M M M M M M M M M M M M M M M


Domains of Competencies (Englander R, et al. Acad Med. 2013;88:1088-1094)

Individual Competencies (AAMC Core Entrustable Professional Activities for Entering Residency. 2014)

Entrustable Learner Milestones (AAMC Core Entrustable Professional Activities for Entering Residency. 2014)

Pre-entrustable Learner Milestones (AAMC Core Entrustable Professional Activities for Entering Residency. 2014)

9/30/2019

3

Examples of Core EPAs from AAMC

• Perform a H&P

• Prioritize a differential diagnosis

• Document a clinical encounter in the patient record

• Collaborate as a member of an interprofessional team

• Perform general procedures of a physician

• Identify system failures and contribute to a culture of safety

Levels of “Entrustment” / Supervision

1. Not allowed to practice EPA

a. Inadequate knowledge/skill

b. Adequate knowledge, some skill; allowed to observe

2. Allowed to practice EPA only under proactive, full supervision

a. As coactivity with supervisor

b. With supervisor in room ready to step in as needed

3. Allowed to practice EPA only under reactive/on-demand supervision

a. With supervisor immediately available, all findings double checked

b. With supervisor immediately available, key findings double checked

c. With supervisor distantly available (e.g., by phone), findings reviewed

4. Allowed to practice EPA unsupervised

5. Allowed to supervise others in practice of EPA

Chen HC, et al.

The EPA(The desired outcome)

Patient Care

Knowledge of Practice

Practice-based Learning and Improvement

Interpersonal and

Communication Skills

Professionalism System-based

PracticeIPP/IPC

Personal and Professional

Development

C1

C2

C1

C2

C1

C2

C1

C2

C1

C2

C1

C2

C1

C2

C1

C2

M

M



Domains of Competencies (Englander R, et al. Acad Med. 2013;88:1088-1094)

Individual Competencies (AAMC Core Entrustable Professional Activities for Entering Residency. 2014)

Entrustable Learner Milestones (AAMC Core Entrustable Professional Activities for Entering Residency. 2014)

Pre-entrustable Learner Milestones (AAMC Core Entrustable Professional Activities for Entering Residency. 2014)

Professionalism to Professional Identity

• The Carnegie Foundation (2010)

– “Professional identify formation – the development of professional values, actions, and aspiration – should be the backbone of medical education.”

• Crossley et al (2009)

– “A necessary foundation for professionalism”

Professional Self-Identity

• PSI: The extent to which an individual feels like a member of the profession they chose

– Dornan et al: “an [appropriate] state of mind that include confidence, motivation, and a sense of professional identity”

– The inclusion of a new identity with the one they entered school with

– Core values internalization

Professional Self-Identity

• PSIQ: Professional Self-Identity Questionnaire developed by Crossley and Vivekananda-Schmidt

• Used by the team to survey PSI three main themes

– Participation in the professional role

– Being recognized as a quasi-professional

– Extra-curricular activities

9/30/2019

4

Professional Identity Pyramid (Cruesset al. 2016)

KNOWS(Knowledge)

KNOWS HOW(Competence)

SHOWS HOW(Performance)

DOES(Action)

IS(Identity)

Knows the behavioral norms expected of a health care professional

Knows when individual behaviors are appropriate

Demonstrates the behaviors expected of a professional under supervision

Consciously demonstrates the behaviors expected of a professional

Consistently demonstrates the attitudes, values, and behaviors expected of one who has come to “think, act, and feel like a professional”

Development of the CBE/EPA model

WHY DID WE BEGIN TO LOOK AT THESE CONCEPTS IN OUR PROGRAM?

• Academic programs have rubrics, assessments and goals that determine level of competence

• Behavior may be one that is more challenging to assess objectively

Difficult to predict

• Students who demonstrate competence in academic setting, but unsure how they will respond in clinical setting

• More recent trend of students struggling with professional behaviors in the clinic

How will they perform?

9/30/2019

5

What SPARC’d this conversation?

• Two student types

– Both struggle during the program

• But for different reasons

Which student do you feel more comfortable going to the clinic?

Student 1 Student 2

Looking at students in a 2x2

+ Competence+ Trustworthiness

+ Competence- Trustworthiness

- Competence- Trustworthiness

- Competence+ Trustworthiness

Easy and straight forward for programs





A bit more muddy• Tends to be very personable and likeable

• Don’t want to see them be unsuccessful





Tends to be very muddy

• Has all the ability but lacks the trust to perform when needed

• Very difficult to ‘catch’ them in a traditional academic rubric





9/30/2019

6

So our examples fit here





Match making

ONCE THEY ARRIVE IN THE CLINIC AND MEET THEIR CLINICAL INSTRUCTOR

Potential Positives

• Wonderful learning experiences for student/CI

• Optimal development of student

• Mutually beneficial to the clinical site

• Enhanced partnership between program and site

9/30/2019

7

Potential Negatives

• Strain between student/CI

• Emotionally and mentally taxing

• Suboptimal learning experiences

• Time commitment increases and can affect clinic, patients and faculty

• Increased strain between program and clinical site

A lot on the line

Expectation gap CSM and ELC

• Over the last few years

– Attended conference presentations regarding the idea of CBE and EPA

– Read and discuss more articles on CBE and EPA

9/30/2019

8

So we initiated a plan

• Student Progression and Readiness Across the Curriculum (SPARC)

– Formalized method to foster development (academic and non-academic) of our students prior to heading into 1st full time clinical

Student Progression and Readiness Across the Curriculum (SPARC)

– Use elements from the EPA and CBE literature to construct assess these elements for each student

– Also assess the progression (or lack of progression) over time

– Establish a remediation plan during the didactic portion of the curriculum


– Started with readiness for 1st full-time clinical experience

– Quickly expanded to across the entire curriculum

• Identified key time pointsLevel of

Competency for end of

Year 1

Level ofCompetency for

CE 1 (end of year 2)

Level of Competency for Terminal Clinical

Experiences

Level of Competency for

Graduation

9/30/2019

9


– Under each them, we created key areas we wanted to focus on

• Based on student, alumni, employer surveys and faculty feedback

Level ofCompetency

for end of Year 1




Experiences


Graduation

Sub-theme examples:• Introduce themselves appropriately• Empathy• Ask for assistance when needed• Document appropriately for encounter


– Then for each theme the faculty discussed appropriate progression from beginner/novice to entry-level

Level ofCompetency

for end of Year 1




Experiences


Graduation

Sub-theme examples:• Introduce themselves appropriately to audience• Empathy• Ask for assistance when needed• Document appropriately for encounter

Beginner/Novice Entry-level


– ACAPT produced a working document that created more sub-themes and levels for competence prior to 1st full-time clinical

https://acapt.org/docs/default-source/public-docs/ksas-and-levels-of-competency-for-considering-student-readiness.pdf?sfvrsn=47da8bd8_2


– Refined our list of sub-themes and used the ACAPT document as a benchmark for student readiness prior to 1st full-time clinical exp


Level ofCompetency

for end of Year 1




Experiences


Graduation


– Then began to map out what our expectations would be for our other key time points

• End of 1st year

• Final didactic semester prior to terminal clinical exp

• Completion of terminal clinical exp


Level ofCompetency

for end of Year 1




Experiences


Graduation


– Adaptation of existing assessments to reflect these elements deemed important by the program

• Modifications to assessments in fieldwork/ICE, semester evaluations, graduate and employer surveys

– Developed a program wide curricular assessment strategy that incorporates feedback from student, faculty, clinical faculty




9/30/2019

10


– Goal: Formative assessments to better identify areas of strength and areas for further development. Foster progression and development of sub-themes


– Additional opportunity: Early identification of students that may need additional resources in order to obtain the desired level


– Additional opportunity: Early identification of students that may need additional resources in order to obtain the desired level

Providing this student with individually tailored resources for success:• Could include more practice

opportunities• Specific ICE experiences• Practice with

communication

Implementation of the CBE/EPA Model

Goals

• Reduce subjectivity

• Initiate objective documentation from multiple stakeholders, each with multiple data points

• Improved ability to identify areas of development for individual students earlier on

• Embed competency based assessments at deliberate time points to act as readiness indicators

• Coordinate the efforts of all stakeholders with clear student expectations

Review of Existing Readiness Indicators

• Academic Year 1– Basic science courses, intro clinical courses

– ICE curriculum

• Academic Year 2– Clinical management courses

– ICE Curriculum

• Professional Behaviors– Do we just expect them to have a professional identity?

(some will)

9/30/2019

11

SPARC – AY 1

• ICE Experiences 1-2– Cognitive, Psychomotor, Affective Objectives– Clinical faculty input

• Student self-reflections– ICE, KSAPs, professional development

• Faculty notes– safety, professional behaviors

• Clinical Simulations– Deliberate carryover and integration within formative assessments

• End of Year 1 Practical Examination with SP

End of Year 1 Practical Exam

• 1 experience across multiple courses

• Standardized patients

• Development of KSAPs expected at end of year 1

• Deliberate practice opportunities– Case development

– Lab and equipment

– Faculty presence

SPARC – AY 2

• ICE Experiences 3-5– Cognitive, Psychomotor, Affective Objectives– Clinical faculty input

• Student self-reflections– ICE, KSAPs, professional development

• Faculty notes– safety, professional behaviors

• Clinical Simulations– Clinical Management of Neuromuscular Conditions– Clinical Management of Musculoskeletal Conditions– Management of the Complex Patient (simulation center)

Purposeful Coordination

• Development of a “Portfolio” System

– Central repository

– Collection of information from key stakeholders

– Key time points

– Ability to develop an individualized student plan

• Student account– self reflections, rubrics, assessments

• Faculty access– notes– flags

• Clinical faculty input– ICE rubrics– full-time CE summaries

Catalyst Points

• Time points and milestones

• Initiation of communication and coordination

• Deliberate remediation strategies– Knowledge

– Skills

– Abilities

– Professional behavior development

9/30/2019

12

IDENTIFYING THE NEED FOR REMEDIATION

Student examples

Student examples

• Student 1

– Concerning academic performance

• Low performer on written exams

• Assignments: lacking ability to synthesize and apply information

– Student struggling with consistent competency

• Competency-based assessment (i.e. practical exams)

• Need for repeat of practical exams in multiple courses

Student examples

• Student 1

– Not confident that student knows or knows how

– Can this student be trusted in the clinic?

Student examples

• Student 1

– Remediation plan

• Outline of specific behaviors/domains of concern» Communication

» Clinical reasoning

» Clinical skills

• Clear expectations and consequences» Case review and problem-solving bi-weekly with faculty

mentor

» Participation in part-time integrated clinical experiences

» Written reflections with specific/individualized prompts

Student examples

• Student 1

– Academic Outcome

• Expectations of academic remediation plan were met

• Student proceeds to first full-time CE– Remediation plan carried forward to full-time CE

– Bi-weekly phone calls with DCE to review clinical cases/case analysis/clinical decision making

– Monthly phone conference with DCE, CI, and student

9/30/2019

13

Student examples

• Student 1– CE Outcome

• CE 1 Successful completion!!!– Exceeds expectations for CPI criteria for CE 1

» Communication• “Student has come a long way with communication;

significantly more verbal with both staff and patients; accepts feedback easily, and communicates with a lot more confidence”

» Clinical reasoning• “Consistently improving in this category”

» Clinical Skills• “Significant improvement in examination skills; able to

see the big picture; fully grasps how to determine a POC based on client presentation.”

Student examples

• Student 2

– Acceptable academic performance

• Assessed via written exams

• Assignments

– Competence with clinical skill set in various settings

• Competency-based assessment (i.e. practical exams)

Student examples

• Student 2

– Student knows and knows how

Student examples

• Student 2

– Inconsistent attendance and participation in class

– Demonstration of unprofessional behaviors in academic and clinical settings

– Student does not consistently “show how”

– Can this student be trusted in the clinic?

Student examples

• Student 2– Remediation plan

• Outline of specific behaviors/domains of concern» Professional behavior

» Accountability

» Communication

• Clear expectations and consequences» Punctuality

» Attendance

» Adherence to part-time clinical site requirements

» Written reflections with specific/individualized prompts

Student examples

• Student 2– Outcomes:

• Academic remediation implemented spring 2019

• Expectations NOT met by start of first full-time clinical experience

• 1st full-time CE cancelled and student placed in a remedial part-time clinical experience – Learning contract instituted for remedial pre-CE1

» Additional responsibilities as lab TA under direct supervision of DPT faculty

» CPI utilized for remedial part-time CE

» Bi-Weekly conference calls scheduled with student, CI, and DCE

9/30/2019

14

Student examples

• Student 2

– Outcomes:

• Did it work?

References

• El-Haddad C, Damodaran A, McNeil HP, Hu W. The ABCs of entrustable professional activities: an overview of ‘entrustable professional activities’ in medical education. Intern Med J. 2016; 46(9). 1006-10.

• Chesbro SB, Jensen GM, Boissonnault WG. Entrustable Professional Activities as a Fraework for Continued Professional Competence: Is Now the Time? Phys Ther. 2018; 98(1): 3-7.

• Miller GE. The assessment of clinical skills/competence/performance. Acad Med. 1990; 65 (9): S63-S67.

• Cruess RL, Cruess SR, Steinert Y. Amending Miller’s Pyramid to Include Professional Identity Formation. Acad Med. 2016; 91: 180-185.

• Shorey S, Lau TC, Lau LST, Ang E. Entrustable professional activities in health care education: a scoping review. Med Edu. 2019; https://doi.org/10.1111/medu.13879

• Kwan J, Crampton R, Mogensen LL, Weaver R, van der Vleuten CP, Hu WC. Bridging the gap: a five stage approach for developing specialty-specific entrustableprofessional activities. BMC Med Edu. 16: 117.

References

• Frank JR, Mungroo R, et al. Toward a definition of competency-based education in medicine: A systematic review of published definitions. Medical Techer. 2010;32:631-637.

• ten Cate, O. Entrustability of professional activities and competency-based training. Med Educ. 2005;30:1176-1177.

• May W, Morgan BJ, et al. Model for ability-based assessment in physical therapy education. Jour Phys Ther Educ. 1995;9(1):3-6.

• Santasier AM, Plack MM. Assessing professional behaviors using qualitative data analysis. Jour Phys Ther Edu. 2007;21:29-39.

• ten Cate, O. Nuts and bolts of entrustable professional activities. Jour Grad Med Educ. 2013; 157-158.

• Association of American Medical Colleges. Core entrustable professional activities for entering residency. AAMC, 2014

• Chen HC, van den Broek S, ten Cate O. The case for entrustable professional activities in undergraduate medical education. Academic Medicine. 2015;90(4):431-463

References

• Englander R. Toward a common taxonomy of competency domains. Acad Med. 2013;88:1088-1094.

• Cooke M, Irby DM, O’Brien C. Educating physicians: A call for reform of medical school and residency. San Francisco, Calif: Jossey-Bass; 2010.

• Vivekananada-Schmidt P. A model of professional self-identity formation in student doctors and dentists: a mixed method study. BMC Medical Education. 2015;15:83

• Crossley J, Vivekananda-Schmidt P. The development and evaluation of a professional self identity questionnaire to measure evolving professional self-identity in health and social care students. Med Teach. 2009;31:3603-e607.

• Timmerberg JF, Dole, R, Silberman N, et al. Physical Therapist Student Readiness for Entrance Into the First Full-time Clinical Experience: A Delphi Study. PTJ 2019; 99(2): 131-146.

• Chesbro SB, Jensen, GM, Boissonnault WG. Entrustable Professional Activities as a Framework for Continued Professional Competence: Is Now the Time? PTJ 2018; 98(1): 3-7

THANK YOU QUESTIONS, COMMENTS, DISCUSSION

https://doi.org/10.1111/medu.13879

blind-dating in pt education where is the trust?...m m mm domains of competencies (englander r, et...

Documents