development, evaluation and remediation · clinical reasoning tool development definition of...

1

Clinical Reasoning Tool Development, Evaluation and Remediation

• Clinical Reasoning Tool development

• Testing the effectiveness of the Tool

• Using the Clinical Reasoning Tool

• Remediating clinical reasoning

3

Clinical Reasoning

Tool Development

Background

Canadian faculties of medicine use Chart Stimulated Recall to:

– identify critical thinking and reasoning skills

– stimulate reflective practice

– provide feedback

– improve documentation skills

– help demonstrate and evaluate roles and competencies

– help structure a teaching session

– identify gaps in knowledge

Dept. Family Medicine, University of Alberta (2009)

Background Health regulated colleges across Canada use different methods to evaluate their registrants’ clinical reasoning.

Background

Sample of registrant quotes following peer assessment:

“The chart review was fine, but I wanted more feedback and discussion about the charts.”

“I feel that my actual clinical work was not explored”

“I am looking forward to the addition of a more clinical part to the assessment process”

Goal:To develop a reliable and valid tool to evaluate speech language pathologists’ and audiologists’ clinical reasoning across all practice areas as part of the Quality Assurance Program.

Clinical Reasoning Tool Development

Definition of Clinical Reasoning

“Clinical reasoning describes the process by which registrants collect and evaluate information, come to an understanding of a patient problem or situation, plan and implement interventions, evaluate outcomes, and reflect on and learn from the process.”


“What a hypothetical, typical registrant who exercises average care, skill, and judgment would do in similar circumstances and thereby serve as a comparative standard.”

Definition of Reasonable:

Testing the Effectiveness of the Tool

11


Face and Content Validity

The Tool and the Guide were reviewed by 5 stakeholder focus groups:

• Managers and Administrators

• Randomly selected registrants

• University Clinical Educators

• Registrants recently peer assessed (SLPs and audiologists)

• Peer Assessors (SLPs and audiologists)

The groups recommended changes and additions which were incorporated.

They agreed that the content of the Clinical Reasoning Tool was valid,

that is, it measures clinical reasoning.


Participants: Peer Assessment 2017

• 45 clinicians completed the Clinical Reasoning Tool

• 3 French and 42 English administrations

• 9 Audiologists – providing services in clinic settings

• 36 Speech Language Pathologists– 9 provided adult services

– 8 provided preschool services

– 9 provided services to school boards

– 10 provided services through SHSS, LHIN and/or private

13


Methodology

• Two Clinical Reasoning Tools were completed at the site visit– The clinician selected one patient record (chart)

– The peer assessor selected another patient record

• Both administrations of the Tool were audio recorded

• The peer assessor scored the Tools

• The audio files and scores were uploaded to Box (cloud storage)

14


Measures:

1) Pre-site visit clinician survey

2) Post-site visit clinician survey

3) Post-site visit peer assessor confidence survey

4) Inter-rater reliability

5) Clinician and peer assessor comments

15


Clinician Survey Results

Surveys used a 5 point Likert Scale

2 Tailed t-test used to measure change between pre and post

One question showed a significant change from pre to post

• Do you think a measure of clinical reasoning skills can be valid(it actually measures clinical reasoning)?

2 Tailed t-test t = 2.2953, df = 44, p = 0.03153

17


Peer Assessment Confidence Survey (post-administration)

“How confident are you in your determination of clinical reasoning?”

5 point Likert scale to measure responses

Results:

N = 44

Mean 4.6 – between ‘confident’ and ‘very confident’

Range 3-5

18


Inter-Rater Reliability

• The audio recordings were sent to a second peer assessor working in the same practice area for scoring.

• The peer assessors were not allowed to discuss their finding with each other.

Results

Scores were allocated:2 = complete clinical reasoning

1 = questionable clinical reasoning

0.5 = Inadequate clinical reasoning

19


Inter-Rater Reliability

Results show good inter-rater reliability (alpha coefficient > .70)

20


Inter-Rater Reliability – Calibration Exercise

• All peer assessors listened to 8 audio recordings and voted anonymously on clinical reasoning

• Results of the discussions:

– Listen to the content of clinicians’ responses

– What is and what is not clinical reasoning

– Additional variables to be aware of: lack of preparation and atypical communication skills

21


Clinical Reasoning Tool Trial Results:

• 43 clinicians were found to have clinical reasoning

• 2 clinicians were found not to have clinical reasoning

22

Clinical Reasoning Tool

To date, 129 registrants have completed 258 CRTs in the peer assessment process and 1.5% were found to need assistance.

Or . . . . .

98.5% clinicians demonstrated clinical reasoning

23

Next Steps

• Council approved the Clinical Reasoning Tool for the peer assessment process

• The CRT and Guide are posted on the website for all registrants (link)

• Clinical reasoning indicator added to the 2020 SAT

• Inform the membership – today’s eforum!

• CRT Remediation Program approved by the Quality Assurance Committee (QAC)

• The CRT will be implemented in the Mentorship Program in 2020

http://caslpo.com/members/quality-assurance-program/peer-assessment-program

The Clinical Reasoning Tool

26


The Clinical Reasoning Tool is

available for all registrants to use

Website-> Registrants -> Quality Assurance -> Peer Assessment

• The Clinical Reasoning Tool

• The Clinical Reasoning Tool Guide

http://www.caslpo.com/sites/default/uploads/files/TL_EN_Clinical_Reasoning_Tool.pdf

http://www.caslpo.com/sites/default/uploads/files/TL_EN_CRT_Guide.pdf

Scenarios and Polling Questions

28

29

• How did the background information you collected direct your assessment?

• Response: I always do an artic assessment with artic kids

• Sufficient Clinical Reasoning?


CLINICAL REASONING PROCESSES

THE MEMBER. . .

• collected sufficient information.

• applied background/clinical information in the decision

making process.

• linked information from one phase of intervention to the

next.

• incorporated the patient’s context and situation into areas

of intervention and decision making

• considered options and provided a reasonable rationale to

eliminate other options

• was flexible in their approach regarding the patient, their

needs or other intervention options

• provided a reasonable rationale to explain why they did

what they did

31

• How did the background information you collected direct your assessment?

• Response: The boy was outgoing and happy to come with me, he had reasonable attention, he was able to tolerate a formal articulation assessment and I got the information I needed to develop a therapy plan




THE MEMBER. . .



making process.


next.








what they did

33

• What was unique about this patient?

• Response: Nothing really, he was a typical 80-year-old with a hearing loss who needed hearing aids




THE MEMBER. . .



making process.


next.








what they did

35

• What was unique about this patient?

• Response: The 80-year-old patient with a hearing loss came from a large family who got together often. His wife said that he is isolating himself at family gatherings and not participating, which is not typical behaviour for him. She was concerned that he was missing out on important conversations.




THE MEMBER. . .



making process.


next.








what they did

37

• How did you decide to continue with or move onto another section of the management plan?

• Response: This patient has advanced dementia and cannot tolerate a texture upgrade. There are real safety issues. However, the dietitian and I met with the family to discuss what foods she liked and disliked and we monitor her intake. The dietitian and I meet at weekly rounds to discuss the patient, and we do joint re-assessments when nursing inform us there is an issue




THE MEMBER. . .



making process.


next.








what they did

39

• How did you decide to continue with or move onto another section of the management plan?

• Response: She’s got severe swallowing problems and dementia so we haven’t moved on.




THE MEMBER. . .



making process.


next.








what they did

Remediating Clinical Reasoning

41

Remediating Clinical Reasoning

Remediation will differ for clinicians who show:

• Clinical reasoning in most but not all areas

Vs.

• Minimal clinical reasoning

• No clinical reasoning

42

Clinical Reasoning

Remediation Program

Much of the academic research focusses on teaching clinical reasoning to students, but the Quality Assurance Committee needed to develop a program for experienced professionals who need help in this area.

43

Clinical Reasoning Remediation Program

As part of the solution, adult learning principles (Kearsley 2010) were incorporated;

– making the program as self-directed as possible

– ensuring that it is relevant and practical

– incorporating learning by doing

– respecting the clinician and building on success

44


• The remediation program involves a trained peer coach working with the clinician at their place of work using the clinician’s patient records.

• The Quality Assurance Committee decides how long the remediation program will last.

45


• The program focusses on critical thinking (Kelly and Crawford

2007), that is thinking ahead, thinking in action and thinking back (reflection).

• With reflection the clinician verbalizes their thought processes. Through ‘thinking out loud’, tacit knowledge is made explicit and solutions are developed.

46


Following the constructs of self-direction and building on success, the clinician reviews and selects patient records where they have successfully used the clinical reasoning processes from the CRT. For example:

– evidence showing they collected sufficient information

– applied it and linked information from one phase of intervention to the next

– That they took the patient’s unique context into account and showed a flexible approach regarding the patient’s needs

– Considered other intervention options

– In summary, they could provide a reasonable rationale for why they did what they did.

47


• The peer coach then helps the clinician through a series of questions to problem solve (Nembhard 2014) those areas where clinical reasoning is lacking.

• Discussion and Practice Scenarios are included. Different clinical scenarios will trigger critical and reflective thinking and problem solving.

• Progress is measured through re-administering the CRT.

48

Scenarios – SLP, Pre-School

A very young child has been referred for delayed language. The child arrived to the assessment with her father. The father spoke Ukrainian and was adamant that his child speak Ukrainian at home. How is this new piece of clinical information going to impact on your clinical reasoning and decision making?

What further information would you collect?

How would you apply the information in assessment?

What is the unique patient context?

What options would you consider?

How would you apply the information in treatment?

How would you involve other professionals?

Scenarios – Audiologist

A 15 year old boy is culturally deaf and the mother wants to pursue cochlear implants. You are not sure from the 15 year old’s response if this is what he wants.

What further information would you collect?

How would you apply the information in assessment?

What is the unique patient context?

What options would you consider?

How would you apply the information in treatment?

How would you involve other professionals?

Practice Advice Team

Alexandra CarlingDirector of Professional Practice & QA

[email protected]

Samidha JoglekarAudiology Advisor

[email protected]

Sarah Chapman-JayAdvisor, SLP and QA

[email protected]

David BeattieConseiller sur l’exercisede l’orthophonie

[email protected]

mailto:[email protected]

Thank You!

Additional contact information and resources:[email protected] | 416 975-5347, ext. 226

Follow CASLPO on social media:|

facebook.caslpo.com youtube.com/user/caslpo

www.caslpo.com

Publications:www.caslpo.com/express

www.caslpo.com/members/resources

mailto:[email protected]

https://www.youtube.com/user/caslpo

http://www.caslpo.com/

http://www.caslpo.com/express

http://www.caslpo.com/members/resources



THE MEMBER. . .



making process.


next.








what they did


55


56


57


58


59


60


61


62


63

development, evaluation and remediation · clinical reasoning tool development definition of...

Documents