responding to an aavld accreditation site visit … to a site visit report general: 1) address the...

Responding to an AAVLD Accreditation

Site Visit Report

AAVLD Accreditation Committee October 2012

Presentation Outline

1) Overview of the AAVLD Accreditation

• Initiating the Accreditation Process • Preparing for an AAVLD accreditation site visit • The AAVLD Accreditation site visit

2) Accreditation Site Visit Reports

• Site Visit Team Draft Report • AAVLD Accreditation Committee Report • Responding to the Accreditation Report

3) Question and Answer Session with AC members

Initiating the AAVLD Accreditation Process

1) Laboratories without prior accreditation 2) Laboratories renewing accreditation

AAVLD public webpage: www:aavld.org/ • AAVLD Requirements for an Accredited Veterinary Medical Laboratory • AAVLD Application for Accreditation

AAVLD Members’ webpage • AAVLD Requirements Checklist Document • Guideline for Auditees Document

AAVLD Requirements Audit Checklist (a self-assessment tool)

*All highlighted Requirements must have a n associated laboratory document and/or record

How to submit the application materials

What to submit: • The Application and completed Checklist • Quality Manual

plus specific documents if requested by the site visit team: • System SOPs • Recent Management Reviews • Internal Audits • Corrective Action logs and reports • Preventive Action logs and reports

How to submit the application materials

Who to submit to: • The Site Visit Team members

When to submit: • The site team must review your Quality Manual before scheduling (finalizing) dates for a site visit.

How to submit: • Electronic (e.g. USB drive or a secure web-page) • Hardcopies may be requested (one or more)

Who is the AAVLD Site Visit Team

• Two to five AAVLD peers trained as auditors and with individual expertise consistent with your laboratory’s services. • Individuals are selected from the AAVLD Accreditation Committee and AAVLD auditor pool. • Typically one team member was on the prior visit if you are renewing your accreditation.

Preparing for a Site Visit

Communication with the Site Team Chairperson:

• Scheduling the audit dates

• Responsibilities for site visit expenses

• On-site travel and housing arrangements

• Arranging the site visit agenda Schedule all laboratory sections and services Schedule stakeholder meeting(s):

• Clients, Diagnostic Laboratory Board • Ag Commissioners, Regulatory Veterinarians • University Administration

The Site Visit

(1) Introduction of Laboratory staff & site team

(2) Laboratory audit of the AAVLD requirements and the laboratory’s own quality policies & procedures

• Combination of vertical and horizontal auditing • Review of records typically includes:

• Random, representative cases • Training , competency, and proficiency records • Equipment Maintenance and Calibration Logs • Validation documentation • Client feedback, client surveys • Corrective Action Reports and follow-up

(3) Exit Meetings

General Audit Approach

• Fact-finding on behalf of the Accreditation Committee

• A sampling exercise; the collection of objective evidence; identification of systemic issues within the Quality System.

• Interactive process – OK to ask auditors for clarification and explanation (not to ask for specific direction or comparison to other labs).

• Auditors are looking for “what you do right;” laboratory commendations are an important part of the process.

• No surprises - not an adversarial process. • Most importantly - audits are opportunities to evaluate and improve your quality system.

Following the Site Visit

The laboratory:

• Provides the Accreditation Committee with an evaluation of the site visit team.

• Reviews the DRAFT report for factual correctness.

The site visit team:

• Prepares the DRAFT Site Visit Report

• Provides the Director with the DRAFT report for factual review.

Introduction • Background / Executive Summary • Overview / Current Accreditation Status • Response to previous audit and accomplishments

The Draft Site Visit Report

Draft of the Nonconformances

The Final Site Visit Report (following committee discussion and voting)

Introduction • Background / Executive Summary • Overview / Current Accreditation Status • Response to previous audit and accomplishments

Commendations Requirements and Recommendations Suggested Best Practices

Nonconformances

Example:

Responding to a Site Visit Report

General: 1) Address the issues, not the observation

2) Use your corrective action process, including assignment of timelines and follow-up monitoring for the effectiveness of the resolution:

• Every nonconformance must be addressed, but all do not have to be completed when submitting the response.

• Action items must have timelines (“we will do it” is not acceptable); state what do you plan to do, how you plan to do it, what is the timeline.

• Provide evidence that the correction has been made. 3) Disputing a nonconformance: Must be done in the language of the AAVLD standard, and where applicable your own policy and procedure documents.

Top 10-ways to not dispute a finding

1) We don’t think this is a nonconformance because we don’t think this is a realistic requirement.

2) The auditor never asked us about that.

3) The auditor didn’t understand our answer.

4) We have no idea what you’re talking about.

5) We fixed it before the auditors left.

6) We do that all the time and its never been a problem.

7) Our clients like it that way.

8) We already knew about this one from a prior audit.

9) You didn’t write us up for it last time.

10) That specific requirement doesn’t really work for us.

*Instead, phrase your dispute in the language of the AAVLD standard, and where applicable your own policy and procedure documents, and include evidence.

Example Nonconformance

Laboratory’s Resolution

Incomplete Response/ Lacks

documentary Evidence

Response Documentary Evidence

AAVLD 4.10.2.2: Failure to

appropriately record observations

and test data

Make changes to the physical process of data

collection

Ink pens are now accessible to staff at locations where data

is recorded

1. Corrective Action Report.

2. Monitoring /Internal Audit Schedule to verify effectiveness of the resolution.

Response Examples

Requirement: (AAVLD 4.10.2.2) Observations, data and calculations shall be clearly and permanently recorded and identifiable to the specific test at the time they are made.

Requirement: (QM 201.2) Observations and data shall be recorded in permanent blue or black ink at the time they are made. When a QC value or environmental measure falls outside the acceptance limits, it shall be identified by circling in red ink or highlighting in yellow. Nonconformance 1: Positive control values outside of the accepted range were not circled in red or highlighted per QM 201.2 requirements, example: PCR worksheets: AI rPCR on 9/12/12, 9/13/12 and 9/14/12; NDV rPCR on 9/20/12; and PRRS PCR on 9/7 and 9/10 through 9/14/12.

Nonconformance 2: Examples of failure to record test data at the time of the observation were observed. Serology QC log (SerQCLog005) data for NDV HI Positive control titer and Negative control were entered on 9/3/12 and then with a down arrow for all additional test dates for the week. Inappropriate Response: the technicians have been reminded of the lab policy to highlight out-of-spec readings and enter QC log data individually for each run.

Nonconformance 1: Positive control values outside of the accepted range were not circled in red or highlighted per QM 201.2 requirements, example: PCR worksheets: AI rPCR on 9/12/12, 9/13/12 and 9/14/12; NDV rPCR on 9/20/12; and PRRS PCR on 9/7 and 9/10 through 9/14/12.

Response: CAR12-121 – complete sets of pens/highlighters (blue/black, red, yellow) designated for QC entry have been placed in areas of all test sections where QC logs and worksheet entry is performed. Also see CAR12-198 (AAVLD 4.8.1 Control of Nonconforming Testing) related to the failure to respond to the out-of-range assay controls detected during the investigation of the nonconformance.

CAR12-121: Nonconformance to Records requirements

Requirement: QM 201.2 - Observations and data shall be recorded in permanent blue

or black ink at the time they are made. When a QC value or environmental measure falls

outside the acceptance limits, it shall be identified by circling in red ink or highlighting in

yellow.

Evidence: AAVLD audit identified 9 instances where failed controls were not highlighted

on worksheets and a log with a single data entry for multiple dates of data. (see AAVLD

NC 4.10.2.2). Internal audit during investigation identified 4 more instances of non-

highlighted out-of-range data in 2 other sections (records 12-121a, 12-121b attached).

Investigation: QM and 3 technical staff responsible for data capture performed an

internal audit of worksheets to assess the extent of the problem (also see follow-up

CAR12-198). A fishbone analysis identified equipment and process as the root cause:

colored pens are frequently not located where data is recorded (they are in various

techs lab coat pockets or in the laundry room). Rather than find them, the data is

circled with the available pen or data entry is deferred until a highlighter-pen can be

located (and then is sometimes forgotten about).

Resolution: 1. Allow highlighting by circling in blue ink; 2. Provide colored pens to every

technician 3. Place dedicated pens in data record areas.

Corrective Action Plan:

Responsible individual: QM Due Date: 10/1/2012 Follow-up: 1/15/2012 audit

Policy / Procedure

1. SOP (QM 201.2) requires data entry in specific colors; out-of-range data are highlighted to draw attention.

2. Data entry must be performed at the bench where worksheets are at time of the data collection (QM 201.2).

Equipment 1. Highlighters are rarely at the bench. a. Some techs collect pens in their

pockets, so there are never any where I need them when I need them.

2. It’s a disruption to go find pens at the desk, retrieve them from coats in the

laundry room, or go to the storeroom.

Personnel

1. Training: current for QM 201.2

2. Please don’t make us stuff our pockets with pens and timers; can’t we all agree to share the few already available.

Environment

1. We are trying to keep clutter off the benches, so pens are generally kept in the desk area or lab coat pockets.

CAR12-121 NC 4.10.2.2

CAR12-121: Nonconformance to Records requirements

Requirement: QM 201.2 - Observations and data shall be recorded in permanent blue

or black ink at the time they are made. When a QC value or environmental measure falls

outside the acceptance limits, it shall be identified by circling in red ink or highlighting in

yellow.

Evidence: AAVLD audit identified 9 instances where failed controls were not highlighted

on worksheets and a log with a single data entry for multiple dates of data. (see AAVLD

NC 4.10.2.2). Internal audit during investigation identified 4 more instances of non-

highlighted out-of-range data in 2 other sections (records attached).

Investigation: QM and 3 technical staff responsible for data capture performed an

internal audit of worksheets to assess the extent of the problem (also see follow-up

CAR12-198). A fishbone analysis identified equipment and process as the root cause:

colored pens are frequently not located where data is recorded (they are in various

techs lab coat pockets or in the laundry room). Rather than find them, the data is

circled with the available pen or data entry is deferred until a highlighter-pen can be

located (and then is sometimes forgotten about).

Resolution: 1. Allow highlighting by circling in blue ink; 2. Provide colored pens to every

technician 3. Place dedicated pens in data record areas.

Corrective Action Plan:

Responsible individual: QM Due Date: 10/1/2012 Follow-up: 1/15/2012 audit

Requirement: (AAVLD 4.10.2.2) Observations, data and calculations shall be clearly and permanently recorded and identifiable to the specific test at the time they are made.

Requirement: (QM 201.2) Observations and data shall be recorded in permanent blue or black ink at the time they are made. When a QC value or environmental measure falls outside the acceptance limits, it shall be identified by circling in red ink or highlighting in yellow.

Nonconformance 2: Examples of failure to record test data at the time of the observation were observed. Serology QC log (SerQCLog005) data for NDV HI Positive control titer and Negative control were entered on 9/3/12 and then with a down arrow for all additional test dates for the week.

Response: Investigation indicated the same root cause and resolution (designated pens at accessible sites) and so is included with CAR12-121. Internal audit of QM 201.2 is scheduled for the first quarterly audit of 2013 per Internal Audit Schedule 2013.

Nonconformance Laboratory’s Resolution


Documentary Evidence


AAVLD 5.2: Incomplete Training

Records; no evidence of competency evaluation

Create a new form to capture employee

training including area for specific criteria used to evaluate competency.

A new training form which captures

criteria for competency

evaluation is now being used


2. Example(s) of the new form in use.

Example 2:





AAVLD 5.4.2.3: Technical SOPs do

not include all required elements

Revise SOP on SOPs to include all required

elements.

The Document Format SOP has been revised. All

technical SOPs will be reviewed and

revised to include all required elements.

1. Corrective Action report.

2. Copy of the new SOP. 3. Action plan and

timeline for review and revision of all SOPs.

Example 3:





AAVLD 5.3: Facility design

creates a biosafety concern by venting

air from the histology area into

offices

HVAC modification with air exchange re-

engineering.

“The ventilation system has been

retrofitted and now vents to the outside”


2. Photograph (diagram) of the remodel.

3. Environmental monitoring records.

CAR12-092 HVAC Remodel CAR12-092 30-day EH&S Air Quality Checks

Example 4:





AAVLD 5.6.3: Materials used in conjunction with testing activities are not stored in a manner that ensures proper performance and/or accuracy.

Correct acceptable ranges for refrigerators

A procedure for establishing acceptable

temperature ranges has been written.

Temperature ranges have been corrected

for the reagent storage refrigerator.


2. SOP for establishing acceptable measurement ranges.

3. Training records for the new SOP.

4. Audit record of verifying acceptable ranges for all equipment used for storing and handling critical reagents.

5. Revised temperature monitoring forms in use.

Example 5:





AAVLD 5.4.4.2: Spreadsheets used for reagent calculations are not subject to appropriate checks

Verify spreadsheet calculations; lock

appropriate cells to prevent modification:

establish a cross-reference to technical

SOP & document review.

“The PCR spreadsheets have been checked for

accuracy & are now in the annual PCR

SOP document review schedule”


2. Copy of the revised SOP including cross-reference to controlled spreadsheets.

3. Training record for revised SOP.

4. Internal audit of compliance for all spreadsheets used for calculations.

5. Document review schedule for calculation spreadsheets.

Example 6:





AAVLD 5.4.4.3b: Integrity and confidentiality of data is compromised by posting and sharing of passwords

Cyber-security training and implementation.

Training on passwords and

computer security is being scheduled for

all employees .


2. Training Plan 3. Timeline for completion

of training. 4. Schedule for internal

audit measuring effectiveness of the training.

Example 7:





AAVLD 4.11.3: Corrective actions associated with prior internal audit finding (shared passwords) not discharged within an appropriate and agreed-upon time frame.

Monitor Action Plans and timeframes assigned for

Corrective Actions. Audit effectiveness of the

resolution(s).

“Action plans and timeframes are a component of our CAPA process. The QM has set-up a

quarterly schedule to monitor for

incomplete or unfinished assigned

actions”


2. Copy of CAPA SOP(s) and associated form(s)

3. Example of record documenting “Outstanding Action Item Review” & response by QM.

4. Schedule for auditing the effectiveness of the resolution.

Example 8:





AAVLD 4.10.1.2: Traceability records (AAVLD 5.6.2a) not readily retrievable

(located and provided after the

audit was completed)

Establish a storage and/or indexing system for records related to

certificates and documentation of

reference materials.

Each section maintains their own records on reference materials. They are now aware of the need to document

reference organisms and maintain certificates.


2. Section-specific or system-wide record management (indexing) plans.

3. Training record (or other evidence of “awareness”, e.g. quiz results.

4. Copy of internal audit of record retrievability.

Example 9:





AAVLD 5.5.8: Equipment taken out of service was not properly labeled; had no date or initials on a small “Do not use” note taped on the equipment.

Provide documentation to dispute the

nonconformance (no Corrective Action

required)

Disputed nonconformance:

“The label on the equipment is

compliant with the lab’s SOP QM 144.1

on Equipment Use & Maintenance, and with AAVLD 5.5.8”

1. Copy of SOP QM 144.1 documenting how malfunctioning equipment is taken out of service, labeled, and stored.

2. Digital photograph of the equipment and label (if deemed additionally necessary).

Example 10:

AAVLD 5.5.8 Equipment that has been subjected to overloading or mishandling, or gives suspect results, or has been shown to be defective or outside specified limits, shall be taken out of service, clearly labeled or marked, and appropriately stored until it has been repaired and shown to perform correctly.

QM 144.1 A “Do not use” sign shall be appended to equipment determined to be defective. The sign shall remain in place until repairs are complete and correct performance verified. All calibration, verification, and repair history shall be recorded in the equipment log.

Format for submitting responses Submit Response as a digital file (coordinated set of files)

including:

Response document (Requirement, Nonconformance, Response) including cross-reference to associated documents/files containing the:

Corrective Action Reports, Timeline for Completion, and Monitoring Plans as appropriate.

Evidence: example documents, digital images, graphics, etc.

A PDF of the Response Document containing hyperlinks to PDF’s of the CARs and additional supporting documentation is encouraged.

Example:

How to get your AAVLD Accreditation questions answered?

• Immediate issues: Chair of the Accreditation Committee • Frequently Asked Questions: submit question to the Accreditation Committee chairperson’s office ([email protected]) or to any accreditation committee member. Responses will be posted ~quarterly on the AAVLD member website under Accreditation / FAQs.

mailto:[email protected]

Future Accreditation Committee Efforts

1) Development of a template for submitting the Audit Report Responses.

2) Ongoing training of auditors to ensure consistency and standardization throughout the AAVLD audit process.

3) Continued development of the FAQ process (based on users’ feedback and needs).

4) Additional suggestions from AAVLD members?

responding to an aavld accreditation site visit … to a site visit report general: 1) address the...

Documents