handouts presentation - multivie · presentation handouts ... • daily, weekly, monthly blood bank...
TRANSCRIPT
PresentationHandouts
(9108-QE)
Competency Assessment: Finding the
Good, the Bad and the Ugly
October 6, 2012 10:30 AM - 12:00 PM
Event Faculty List
Event Title: 9108-QE: Competency Assessment: Finding the Good, the Bad and the UglyEvent Date: Saturday, October 6, 2012 Event Time: 10:30 AM to 12:00 PM
Director L. Jeanne Wall, MEd, MT(ASCP)SBB Consultant Wall Quality Management [email protected] Disclosures: No
Moderator Anne Chenoweth, MBA, MT(ASCP)CM, CQA(ASQ) Technical Specialist AABB [email protected] Disclosures: No
Speaker Judith Sullivan, MS, MT(ASCP)SBB, CQA(ASQ)Quality Source Consultant Blood Systems, Inc. [email protected] Disclosures: No
Speaker Regina Castor, BS, MT(ASCP)SBB Blood Bank Technical Specialist Immucor [email protected] Disclosures: No
06/11/2012
1
Competency Assessment: Examples
Regina Castor BS MT(ASCP) SBB cm
Objectives
• Provide examples and/or strategies for assessing competence for the following groups:
– Clinical Lab Staff
– Donor Center Staff
– Nursing Staff
– Physicians
CLIA ’88 Requirements
• Direct Observation
• Routine Patient Test Performance
• Performance of Instrument Maintenance/Function Checks
If 10 procedures x 10 staff members x 6 elements of CLIA…
06/11/2012
2
Direct Observation– Questions that come to mind?
• Who does the observation?– Manager– Senior Tech– Peer– “Secret Shopper”
• What / How much do we choose to observe?– Routine/Critical tasks– Pre‐analytical, Analytical, Post Analytical– Tasks that are new, changed
• How often do we observe it?– Look at regulations
• How do we document it?– Can vary, but it must be done
Initial Training
Direct Observation
• The observation must be documented
– Create a checklist from the procedure
Don’t forget this is a good time to critique procedures
Creating a checklist = an internal audit!
06/11/2012
3
Donor Center
Audit Checklist:*PPE*Bag Inspected*Bag hung properly*Tubing inserted*Hemostat applied*Sample port remains below donor arm
etc.
An Example of Direct Observation Competency Assessment
Procedure for Issuing Blood:
06/11/2012
4
Could include “What if“ questions to satisfy Problem solving requirement
Performance of Instrument Maintenance and Instrument checks
• ‐Review steps in the procedures to assure you are
“saying what you do, and doing what you say”
Make your procedure into a checklist
Remember this is a good opportunity to audit procedures for following manufacturer’s directions
Examples of Instrument Maintenance/Function Checks
• Calibration of Scales
• Testing Temperature High/Lows, Alarm activation checks
• Serofuge Calibrations
• Daily, Weekly, Monthly Blood Bank automation checks
• Sterile docking devices
• Maintenance on plasma thawers
• Cooler Validations
• Hgb level detection devices
• Decontamination
06/11/2012
5
Use QA schedules to determine who and what to observe
CLIA ’88 Requirements
• Monitoring the recording and reporting of test results
• Most Transfusion Services and Donor Centers do some sort of record review– Define how this review is done in your Competency P&Ps
Example of Documenting
Monitoring and Recording resultsPROCEDURE STATES:
“Daily test and QC results are reviewed. Any reporting of results or test performance that is found to be in non‐
compliance with procedures will be documented on a non conformance form. Non conformance forms for each tech are reviewed prior to determining annual competency.”
The inspector should be able to follow the documentation trail to see how the process works
06/11/2012
6
2011 Blood Bank Competency Assessment Summary
1. Direct Observation of routine patient test performance
Tech 1 Tech 2 Tech3 Tech4 Tech5 Tech 6 Tech 7
ABO/Rh and antibody screen 11/1/2011 11/1/2011 11/1/2011 11/1/2011 Sep‐11 12/6/2011 12/23/2011
Other
2. Monitoring, recording and reporting of test results
Tech 1 Tech 2 Tech3 Tech4 Tech5 Tech 6 Tech 7
ABO/Rh 0223IH9 0223IH11 0207IH16 0225;IH10 0206IH13 12/6/20110225;IH30
Antibody Screen 0223IH9 0223IH11 0207IH16 0225;IH10 0206IH13 12/6/20110225;IH30
Compatibility 0223IH9 0223IH11 0207IH16 0225;IH10 0206IH13 12/6/20110225;IH30
Antigen Typing 2/26/11 6/30/11 1/18/11 2/25/11 7/23/11NA NA
Antibody ID 0105IH31 0811IH11 0131IH6 0801;IH21 NA NA
TRXN 0317;IH16 0228IH24
Courtesy of Lea Tolzmann, Winter Haven Hospital, Winter Haven, FL
All Routine Tasks
Accession # of test reviewed
CLIA Header
CLIA Header
CLIA ’88 Requirements
• Review of intermediate test results, QC records, Proficiency testing results, preventive maintenance records
Again, most are doing this but don’t document it well from a competency standpoint
06/11/2012
7
Proficiency Testing
• Be aware that simply rotating Lab Proficiency surveys (such as CAP or API) among staff members and checking the results against the expected results in and of itself does not satisfy all 6 of the CLIA requirements
Proficiency Testing
• If you are observing Proficiency sample testing as a direct observation event, are you treating it like routine patient samples?
Proficiency samples must be treated like routine samples
Are we there yet?????
06/11/2012
8
Assessment of test performance through testing previously analyzed specimens, internal blind testing samples, or external
proficiency testing samples
• Internally prepared samples
– Can be time consuming to prepare
– Less Cost
– Variable results depending on storage and use
– Unique patient samples ??
• Can you get enough sample for multiple techs?
• How stable will the samples be?
– Document results – create worksheet? Use downtime recording procedures to audit process
Assessment of test performance through testing previously analyzed specimens, internal blind testing samples, or external
proficiency testing samples
• Externally Prepared Samples
– Available from some Blood Suppliers as a value added service
– Purchased tech competency products (as opposed to Lab Proficiency products)
• Scalable
• CEUs offered
• Can be used for multiple techs
• ≠Proficiency products‐ more flexibility on how used
Uses of Competency Samples
• Assign prior to releasing to task– Initial competency assessment
• Assign if no proficiency survey has been assigned for a given task in a given year– Assure that all techs have had a challenge sample on all routine procedures per year
• Assign as part of corrective action– For failure on Proficiency tests, failure on annual competencies, Root Cause corrective action
06/11/2012
9
Uses of Competency Samples
• Assign based on new methodologies
– New automation, new potentiator for tube method, etc.
– Can be ordered as needed
CLIA ’88 Requirements
• Assessment of Problem Solving Skills
• Oral interviews
• Skills Labs
• Written Case Studies
– These may be included in some purchased competency or proficiency products
– Include infrequently occurring events
» Txn Rxns
» Adverse Donor reactions
» Massive Transfusion Scenarios
» Disaster Drills
06/11/2012
10
Tracer Audits
• Effective at looking at a variety of documents, staff• Can be Retrospective
– Following a unit being transfused back to time of collection, reagents used, QC, equipment used, results recording, etc.
• Can be forwarded looking– Ex: following equipment from purchase to discontinuation– Crossmatch sample collection through transfusion– Donor collection through processing/labelling
Tracer audits can easily incorporate several elements of CLIA by looking at a variety of records
Physician
• Blood Order
• Consent
• Proper Utilization?
Phlebotomy
• Proper labeling
• Turn around Time (order to receipt)
•Was the sample processed correctly
Blood Bank
• Equipment use
• Reagent QC/Receipt
• Results reporting
Nursing
• Time out process (NSPG #1)
• Proper use of filter
• Transfusion protocol
• Transfusion Reaction reporting (if indicated)
Transfusion Audit
06/11/2012
11
Transfusion Audit
Example of facility that uses Microsoft Access to document audits
Nursing Competency
• Record Review– Donor deferral
– Transfusion documentation
– Transfusion Reactions
• Direct observations– Collecting blood
– Transfusing units
– Use of blood warmers
– Transporting Blood
– Learning Labs/Simulator
Nursing Competency
• Example of Annual Module assigned through Hospital’s Learning Management System
– Staff required to review and complete quiz
06/11/2012
12
Physician Competency
• Ordering Practices (TJC requirement)– Is blood ordered at appropriate time– Is the order in compliance with Transfusion policy– Use of emergent blood
• Physician Order– Are all required elements documented
• Dated/timed• Signature• Reason for transfusion• Type and Cross vs. Transfuse• Special requirements for transfusion, how well are those documented
Physician Competency
• Consent
– Is it complete
• Dated and timed before transfusion occurred
– Are all elements being documented – Risks
– Benefits
– Alternatives
– What about refusal process
06/11/2012
13
Example of Documentation
Example of Documentation
Conclusions
• Make checklists from the procedures
• Audits can incorporate multiple CLIA requirements
• Include all staff that relate to transfusion medicine, not just the testing personnel
• Tracer audits are valuable tools
• Document, Document, Document
06/11/2012
1
Competency Assessment: Finding the Good, The Bad, and the UglyJudy Sullivan, MS, MT(ASCP)SBBAABB Annual MeetingOctober 2012
Objectives
• Discuss requirements related to competency assessment
• Describe what assessors, inspectors, and surveyors look for as evidence of compliance
• Identify methods to meet the requirements for competency assessment
3
What Is Competence?
• A cluster of related abilities, commitments, knowledge, and skills that enable a person to act effectively in a job or situation.
• Competence indicates sufficiency of knowledge and skills that enable someone to act in a wide variety of situations.
www.businessdictionary.com
06/11/2012
2
Once Competent, Always Competent, Right?
• Dedicated Staff– Complacency
– SOP drift
• Rotating Staff– Loss of familiarity
– Infrequent activities
Not Necessarily!
What Drives Competency Assessment for Laboratory Staff
It’s the right thing to do!!!!
It’s required by regulation!
CLIA Competency Assessment Key Requirement
493.1413(b)(8)(9) & 1451(b)(8)(9)
Technical Consultant/Supervisor Responsibilities• Evaluating the competency of all testing
personnel and assuring that the staff maintain their competency to perform test procedures and report test results promptly, accurately and proficiently
06/11/2012
3
CLIA Competency Assessment
Competency for all tests must include:• Direct observations of routine
patient test performance, including patient preparation, if applicable, specimen handling, processing and testing
8
CLIA Competency Assessment
• Monitoring the recording and reporting of test results
• Review of intermediate test results or worksheets, quality control records, proficiency testing results, and preventive maintenance records
9
CLIA Competency Assessment
• Direct observation of performance of instrument maintenance and function checks
• Assessment of test performance through testing previously analyzed specimens, internal blind testing samples or external proficiency testing samples
• Assessment of problem solving skills
06/11/2012
4
Frequency
• At least semiannually during the first year the individual tests patient specimens
• At least annually thereafter unless test methodology or instrumentation changes– Prior to reporting patient test results, the individual's
performance must be reevaluated to include the use of the new test methodology or instrumentation
11
CAP
GEN.55500 Competency Assessment
• The competency of each person to perform his/her assigned duties is assessed
• CLIA elements cited
NOTE: The competency of each person to perform the duties assigned must be assessed following training before the person performs patient testing.
12
CAP
GEN.57000 Competency Corrective Action• If an employee fails to demonstrate
satisfactory performance on the competency assessment, the laboratory has a plan of corrective action to retrain and reassess the employee's competency.
• Evidence of Compliance:Records of corrective action to include evidence of retraining and reassessment of competency
06/11/2012
5
What Drives Competency Assessment for Other Staff
It’s the right thing to do!!!!
It’s required by Standards!
AABB
STD 2.1.2 Training• The blood bank or transfusion service shall have
a process for identifying training needs and shall provide training for personnel performing critical tasks.
Not Just Testing Personnel!
AABB
STD 2.1.3 Competence• Evaluations of competence shall be performed
before independent performance of assigned activities and at specified intervals.
06/11/2012
6
Training
≠
Competency
What Do Assessors / Inspectors Look For?
18
First Things First
• Is there a Policy, Process or Procedure Addressing Training and Competency? – Laboratory
• General policies• CLIA elements incorporated
– Blood Bank • SOPs specific for the testing performed
06/11/2012
7
19
The Next Steps
• How is competency determined?– Who can assess competency?
• Is it defined?• No self-assessments allowed!!!
– What constitutes an assessment?• 3 P’s, Tools, Checklists, Guidance
– What tests are being evaluated?• Must include routine tests every year• Can’t pick and choose
– How is it documented?• Does practice match SOPs?
20
Beware the Regulations!
• Testing Personnel– All routine tests
– All CLIA elements MUST be used for evaluation
• Other Personnel– Facility-specified
21
Considerations
• New employees– Competency assessment separate from training– Assessed twice in the first year
• Incumbents– Annual assessment
• Documentation for staff that work on all shifts• Tests being evaluated
– Is there any distinction made for testing that may be provided on day shift vs after hours
– Special testing
06/11/2012
8
Competent or Not?
• Statement of competency
• If not competent, what was done?– Does practice
match SOP?
23
I Don’t Have Time for This!!
24
Let’s Go Back to Basics
• Direct observation of performance
• Direct observation of instrument maintenance/function checks
• Monitoring recording and reporting of test results
• Review of worksheets, QC records, PT results, PM records
• Testing of previously analyzed specimens
• Assessment of problem solving skills
06/11/2012
9
Direct Observation Checklist
• SOP in checklist form
• Performed at conclusion of training on that SOP
• Future use:– Semi-annual competency assessment
– Annual competency assessment
Instrument Maintenance
• Direct observation of performance of instrument maintenance and function checks
• Which instruments?
• Who does it?
Blood Bank Staff ReagentQC
ProvueQC
KB QC SpecialAntigenQC
Provue PM
CellwasherPM
Daytime FT X X X X X
Daytime FT X X X X X
Daytime FT X X X X X
Evening FT X X X
Evening FT X X X
Evening FT X X X
Evening FT X X X
Nights FT X X X
Nights FT X X X
Nights FT X X X
PT X X
PT X X
PRN X X
PRN X X
Staff to QC/PM Matrix
06/11/2012
10
Who Is Watching?
• Someone who– Has been trained
– Can determine competent vs non-competent behavior
– Is motivated to do it right
It Doesn’t Always Have to Be the Supervisor!!
Just Remember…
Those who assess competency must also have their competency assessed IF they perform critical
tasks!
And That Includes the Supervisor!!
Duh! I’m watching…Of course they are going to do it
right!!
Explain to me…
06/11/2012
11
31
Let’s Go Back to Basics
• Direct observation of performance
• Direct observation of instrument maintenance/function checks
• Monitoring recording and reporting of test results
• Review of worksheets, QC records, PT results, PM records
• Testing of previously analyzed specimens
• Assessment of problem solving skills
32
Aren’t You Doing This Already?!
• Include as part of the competency policy
• Include as part of direct observation– Recording and reporting of test results
– QC records
– Preventive maintenance records
– Worksheets
• Document!
33
Let’s Go Back to Basics
• Direct observation of performance
• Direct observation of instrument maintenance/function checks
• Monitoring recording and reporting of test results
• Review of worksheets, QC records, PT results, PM records
• Testing of previously analyzed specimens
• Assessment of problem solving skills
06/11/2012
12
34
Previously Analyzed Specimens
• Patient samples
• Proficiency testing samples– Must be rotated among all individuals performing
testing
– Document as part of competency assessment
Danger, Danger, Will Robinson!!!
DO NOT share PT samples with other staff until
AFTER the results have been received from the PT provider
36
Let’s Go Back to Basics
• Direct observation of performance
• Direct observation of instrument maintenance/function checks
• Monitoring recording and reporting of test results
• Review of worksheets, QC records, PT results, PM records
• Testing of previously analyzed specimens
• Assessment of problem solving skills
06/11/2012
13
Assessment of Problem Solving Skills
• Written test or quiz
• Case Studies
• Scenarios: What would you do if…?
• Staff narrative
Unsuccessful Result
• Process for Remediation– Actions to take
• Removing employee from testing until competency is demonstrated
– Documentation
– Recurrence
– Re-assessment
– Determination • Completion at end of each assessment
Documentation
• Trackable and Traceable– Checklist not sufficient– Each assessment requirement must be
documented• Direct observation checklists• Title and date of record review• Title, date, sample ID if using PT• Graded test/quiz• Assessor name(s) and dates• Employee name
06/11/2012
14
Documentation
• Determination of competency– Signature/date of supervisor– Signature/date of individual performing competency
assessment if different from the supervisor– Signature/date of employee– Include a statement of competency by the supervisor
and the employee
41
Example Competency Statements
• (employee sign/date)___________ certify that I am fully trained and competent to perform the roles listed above
• (assessor sign/date)___________ attest that the staff member listed above is fully trained and competent to perform the roles listed above
I’m Done! Right?
NO!!
06/11/2012
15
New Procedures
• Changes in test methods or instrumentation
• Prior to reporting patient test results, the individual’s performance must be re-evaluated to include the use of the new test methodology or instrumentation
Revised Procedures
• Determine type of Training – Read SOP and sign
– Perform in service and read & sign
– Full training and competency assessed • Trainer requirements/Trainee requirements/DO and
Evaluation
Words of Wisdom
In a hierarchy, every employee tends to rise to his level of incompetence.
Work is accomplished by those employees who have not yet reached their level of
incompetence.
~ Laurence J Peter
