conducting workplace based assessments philip dacosta consultant histopathologist member, rcpath wba...
TRANSCRIPT
Conducting workplace based assessments
Philip DaCosta
Consultant Histopathologist
Member, RCPath WBA Working Group
Outline
Motivating trainers and trainees Organising the assessment process Conducting an assessment Providing feedback Workplace-based assessment as a
formative process
“It’s not all alien…”
Build on existing structures Day-to-day clinical work/encounters Supervisory staff (medical, scientific and
BMS) Use existing opportunities to train and
assess WBA formalises existing supervision and
improves feedback
Motivation
Clarity about process and method of assessment about number of assessments about trainee’s role about the assessor’s role about documentation and evidence
Expectation monitoring arrangements linkage to the ARCP broad coverage of professional activities
Organisation
Cohort of assessors (appropriate to the relevant tasks)
Training and familiarity with assessment tools Pacing of assessment process
planner for year linkage to other processes (portfolio reviews, ESR,
ARCP) remedial processes
The mechanics…
KNOWS
KNOWS HOW
SHOWS
DOES
Miller’s pyramid of clinical competence
Knowledge
Competence
Performance
ActionCBD
ECE
Mini-CEX
DOPS
Mini-PAT
DOPS ECE Mini-CEX CbD
Real time - HOW people do their work Reflective - WHY they did what they did
Consider the focus of the assessment
DOPS ECE Mini-CEX
CbD
Real time - HOW people do their workReflective - WHY
they did what they did
Multi-source feedback - global and team-based assessment
ePATH-SPRAT (MSF)
Direct Observation of Practical Skills (DOPS)
Case-based Discussion (CbD)
Evaluation of Clinical Events (ECE)
Mini-Clinical Evaluation Exercise (Mini-CEX)
Chemical Pathology
3 during training
Minimum 6 per year for first 2 years
Minimum 6 per year
Minimum 6 per year
Minimum 6 per year
Histopathology 3 during training
Minimum 6 per year up to end of stage C
Minimum 6 per year
Minimum 6 per year
-
Medical Microbiology & Virology
3 during training
Minimum 6 per year
Minimum 6 per year
- -
Number of WBAs during training
Opportunities and scenarios
Assessment process
Feedback
Guidance notes for assessors and
trainees
Curriculum
Schedules of scenario complexity
Assessment form
Standards for Assessment
Assessment form & feedback
guidance
Personal Development
Plan
Portfolio
Undertaking assessments
“Overview of WBA” document
Customised for each specialty
Similar content Introduction to WBA Outline of WBA tools Who can be an assessor? Standards for assessment Record keeping Examples of scenarios
Hyperlinks to relevant guidance notes and forms
Guidance note for each WBA tool
Aimed at assessors and trainees Customised for each specialty
and each tool Content:
Explanation of the WBA tool Suitable cases Who can be an assessor How the assessment works Standards for assessment Outcome of assessment Feedback Record keeping
Opportunities and scenarios
Assessment process
Feedback
Guidance notes for assessors and
trainees
Curriculum
Schedules of scenario complexity
Assessment form
Standards for Assessment
Assessment form & feedback
guidance
Personal Development
Plan
Portfolio
Opportunities & scenarios
Opportunities - examples of DOPS
Direct observation of practical skills (DOPS) - Chem Path use of pipette use of balance preparation of buffer measurement of glucose using meter urinalysis using ‘dipstick’ use of blood gas machine manual (specify), e.g. osmometry use of bilirubinometer performance of sweat test insertion of long line supervision of dynamic function tests
Direct observation of practical skills (DOPS) - Histopathology Specimen cut up (e.g. a cancer resection, a group of skin
biopsies) Autopsy procedures (e.g. initial evisceration, dissection of
heart or other system, taking samples for toxicology) Set up and use of microscope Use of camera and specimen photography Handling and reporting of frozen section Systematic microscopic assessment of biopsy/cytology
specimen Practical reporting procedures (e.g. completion of proformas,
notification of case for MDT discussion, diagnostic coding, ensuring copies are sent to relevant individuals)
Taking a fine needle aspirate
Opportunities - examples of DOPS
Direct observation of practical skills (DOPS) - Microbiology sample handling and preparation (including safe laboratory
practice) safe disposal microscopy and staining identification: presumptive or definitive use of selective media ability to distinguish between significant and non-significant
pathogens and normal commensal flora ability to spot and separate mixtures of organisms, obtaining
pure culture antimicrobial sensitivity testing: setting up, reading and
interpretation practical aspects of therapeutic drug monitoring serology molecular methodologies
Opportunities - examples of DOPS
Evaluation of clinical/management events (ECE) - Chem Path Presenting audit findings and leading discussion on the action
required Observation of a trainee led teaching event Demonstration and presentation of cases at MDT Presentation at ‘grand round’ Referring a case for a specialist opinion Providing clinical biochemistry advice in response to enquiry (primary
and secondary care) – by letter, by phone Use of critical incident/non-conformity reporting procedures Presenting findings and leading discussion on the action required Preparing a business case Writing clinical guidelines Appointment of staff
Opportunities - examples of ECE
Opportunities - examples of ECE
Evaluation of clinical/management events (ECE) - Histopathology
evaluating and reporting a histopathology or cytology case evaluating and presenting autopsy findings to a clinical supervisor demonstrating autopsy findings to clinical team presenting cases in a MDT meeting/clinicopathological conference presenting cases at a morbidity/mortality meeting or ‘grand round’ referring a case for specialist opinion providing clinicopathological advice in response to an enquiry use of the call/recall and failsafe systems in cervical cytology
screening use of critical incident reporting procedures handling a patient safety issue (e.g. specimen misidentification) presenting audit findings and leading discussion on the action
required.
Opportunities - examples of CbD
Case-based discussion (CbD) Discussion of a case relating to specific areas of
clinical practice: eg. lipidology or nutrition on chemical pathology eg. blood stream inections or serological diagnosis in
microbiology eg. complex case requiring immunohistochemistry in
histopathology Involvement in critical incident or patient safety event Case involving diverging diagnostic opinions Case raising health and safety/risk management issues Evaluation of QC/QA data
How to use opportunities for workplace-based assessment
Numerous opportunities/scenarios available - refer to guidance notes
Be realistic about assessment duration complexity appropriateness
Sampling
Opportunities and scenarios
Assessment process
Feedback
Guidance notes for assessors and
trainees
Curriculum
Schedules of scenario complexity
Assessment form
Standards for Assessment
Assessment form & feedback
guidance
Personal Development
Plan
Portfolio
Undertaking assessments
Conducting the assessment
Ensure both you and the trainee have read and understood the guidance
Check appropriateness of event Adequate time to undertake assessment
and provide feedback Appropriate place for feedback Have the assessment form with you and
make notes on it
The assessment form
Information about the assessment …
The assessment …
Feedback and outcome …
Outcome - satisfactory / unsatisfactory …
SATISFACTORY
UNSATISFACTORY
A “global” judgement required to comply with “Gold Guide” and ARCP requirements. In essence, does the trainee need
to repeat the assessment…
Personal calibration
The “experienced” trainer All assessors will know what constitutes acceptable
(ie. safe) clinical care/standards Assessors bring their experience to the assessment
Refer to documentation “Standards for assessment tools” Curriculum Suggested complexity of scenarios at different stages
of training Consistency within cohort of trainees
educational supervisors will receive assessment forms from other assessors and will undertake assessments on a range of trainees
Standards for assessment
Trainees must be assessed against the standard expected of a trainee at the end of the stage of training that they are in
The form offers a grading scale from 1–6: 1–2 Below expectations 3 Borderline 4 Meets expectations 5–6 Above expectations
“Standards for assessment tools” provides definitions and grade descriptors
Process Standardisation
Triangulation Checklist -v- Global rating Discussion amongst assessors College monitoring and QA processes
Benefits of using the assessment form Gives structure to the encounter Encourages a systematic approach
Generic forms, so some items will be not applicable (“unable to comment”)
Reduces bias Helpful in providing feedback Helpful in identifying developmental points Helpful in commenting on behaviour
Opportunities and scenarios
Assessment process
Feedback
Guidance notes for assessors and
trainees
Curriculum
Schedules of scenario complexity
Assessment form
Standards for Assessment
Assessment form & feedback
guidance
Personal Development
Plan
Portfolio
Feedback
Providing feedback
Immediate! Part of assessment process Consider using modified Pendleton framework
what trainee felt went well … what assessor observed went well … what trainee would do differently in future … developmental suggestions from assessor …
Focus on what was done and observed Use the form as part of the feedback process
Using the assessments as a formative process
What they know … How they do things …. How they present themselves … How they learn … How they are progressing …
Enthuse them ….
Focus on good feedback
Trainees consistently report that “senior review” has major educational impact (as well as optimising patient care)
The educational impact depends on good feedback
Using structured feedback can encourage self appraisal, confidence and planned learning by the trainee
Paperwork
College intend to implement a web-based system from August 2008
For now, use the paper based systems 3 (4) copies of assessment form (whether successful or
unsuccessful) one into trainee’s portfolio one to educational supervisor one to RCPath assessment office (part of the WBA
standardisation and validation process) (one copy to assessor, if they ask for it. Sensible to do this if
assessment outcome is unsatisfactory) Educational supervisor can use information from all
assessments in drawing up their report for the ARCP process, even though the trainee will only present successful WBAs as evidence to the ARCP panel
Summary
Understand the assessment tools and process
Use your existing people, places and opportunities
Train the trainers/assessors and trainees Allay their anxieties Use it as a formative process