CPA is a UKAS
company
Martin Stearn, UKAS Senior Assessment Manager
ISO 15189:2012 and Susceptibility Testing
BSAC User Day 22 June 2016
© UKAS June 2016
CPA is a UKAS
company
• Discuss the various clauses within the ISO 15189:2012 Standard
relevant to antimicrobial susceptibility testing
• Provide some clarity regarding UKAS assessment process
• Not to tell you what you need to do or how to do it
Objectives
2
CPA is a UKAS
company
• Discuss the various clauses within the ISO 15189:2012 Standard
relevant to antimicrobial susceptibility testing
5.5 Examination processes
ISO 15189:2012 Requirements
3
CPA is a UKAS
company
4.3 Document control
4.5 Examination by referral laboratories
4.6 External services and supplies
4.7 Advisory services
4.9 Identification and control of non-conformities
4.10 & 4.11 Corrective & preventive action
4.12 Continual improvement
4.13 Control of records
4.14 Evaluation and audits
4.14.2 Periodic review of procedures
4.14.3 Assessment of user feedback
4.14.4 Staff suggestions
4.14.5 Internal audit
4.16.6 Risk management
ISO 15189:2012
Management requirements
4
CPA is a UKAS
company
5.1.5 Training
5.1.6 Competence assessment
5.1.8 Continuing education and professional development
5.2.3 Storage facilities
5.3.1 Equipment
5.3.1.2 Acceptance testing
5.3.1.3 Instructions for use
5.3.1.4 Calibration and metrological traceability
5.3.1.5 Maintenance and repair
5.3.1.6 Incident reporting
5.3.1.7 Equipment records
5
ISO 15189:2012
Technical requirements
CPA is a UKAS
company
5.3.2 Reagents and consumables
5.3.2.2 Reception and storage
5.3.2.3 Acceptance testing
5.3.2.4 Inventory management
5.3.2.5 Instructions for use
5.3.2.6 Adverse incident reporting
5.3.2.7 Records
5.4.3 Request form information
5.5 Examination procedures
5.5.1.1 General
5.5.1.2 Verification of examination procedures
5.5.1.3 Validation of examination procedures
5.5.1.4 Measurement uncertainty of measured quantity values
6
ISO 15189:2012
Technical requirements
CPA is a UKAS
company
5.5.3 Documentation of examination procedures
5.6 Ensuring the quality of examination results
5.6.2 Quality control (IQC)
5.6.3 Interlaboratory comparisons (EQA)
5.6.4 Comparability of examination results
(procedures, equipment, sites as applicable)
5.7.1 Review of results
5.8 Reporting of results
5.8.2 Report attributes
5.8.3 Report content
5.9 Release of results
5.9.2 Automated selection and release of results
5.9.3 Revised reports
5.10.3 Information system management
7
ISO 15189:2012
Technical requirements
CPA is a UKAS
company
• Laboratories are free to use whatever method they select.
• ISO 15189:2012 does not prescribe any particular method or the
use of a ‘gold standard’ or the need to change to the latest method.
• Neither do UKAS, we’ll assess your methods and your competence
to use them.
• ISO 15189:2012 does specify the need to select methods which
have been validated and verified to achieve the right result.
Method selection
8
CPA is a UKAS
company
5.5.1 Selection, verification and validation of examination processes
5.5.1.1 General
The laboratory shall select examination procedures which have been
validated for their intended use.
‘Shall’ = you must
(only 321 ‘shalls’ in the Standard!)
‘Should’ = you shall unless you can justify not doing so
(24)
NOTE: Preferred procedures are those specified in the instructions for use
of in vitro medical devices or those that have been published in
established/authoritative textbooks, peer-reviewed texts or
journals, or in international consensus standards or guidelines, or
national or regional regulations. (Standard methods)
ISO 15189:2012
9
CPA is a UKAS
company
Confirmation, through the provision of objective evidence, that the
requirements for a specific intended use or application have been
fulfilled.
Validation
10
The method achieves the intended outcome because;
• Evidence A
• Evidence B
• Evidence C
• Evidence D
The method works
The zones of growth inhibition achieved by the AST method
accurately demonstrate susceptibility or resistance for that organism
CPA is a UKAS
company
Confirmation, through the provision of objective evidence, that the
requirements for a specific intended use or application have been
fulfilled.
Validation (again)
The method achieves the intended outcome because;
• Evidence A; published in established/authoritative textbooks, peer-
reviewed texts or journals,
• Evidence B; international consensus standards or guidelines
• Evidence C; national or regional regulations.
• Evidence D: records of who has validated the procedure
The method works
The zones of growth inhibition achieved by the AST method accurately
demonstrate susceptibility or resistance for that organism
11
CPA is a UKAS
company
5.5.1 Selection, verification and validation of examination processes
5.5.1.3 Validation of examination procedures
The laboratory shall validate examination procedures derived from
the following sources:
a) Non-standard methods
b) Laboratory designed or developed methods
c) Standard methods used outside their intended scope
• Applying breakpoint criteria to related or even unrelated organisms
a) Validated methods subsequently modified
• Alteration of the method in any way (e.g. incubation
time/temperature, disc concentration, media, inoculum density)
ISO 15189:2012
12
CPA is a UKAS
company
Verification
We (the laboratory) can achieve the intended outcome because;
• Evidence X
• Evidence Y
• Evidence Z
The method works in our hands
The zones of growth inhibition we achieve by using the validated AST
method accurately demonstrate the susceptibility or resistance of our
isolates to which we have applied the AST method
Confirmation, through provision of objective evidence, that specified
requirements (as demonstrated by validation) have been fulfilled.
13
CPA is a UKAS
company
Verification (again)
We (the laboratory) can achieve the intended outcome because;
• Evidence X; performance characteristics (we can do) demonstrate
the performance claims (what is expected) have been met in our lab
• Evidence Y; documented verification procedure
• Evidence Z; records of review & authorisation of the verification data
The method works in our hands
The zones of growth inhibition we achieve by the AST method
accurately demonstrate the susceptibility or resistance of the isolates to
which the method has been applied
Confirmation, through provision of objective evidence, that specified
requirements have been fulfilled.
14
CPA is a UKAS
company
5.5.1 Selection, verification and validation of examination processes
5.5.1.2 Verification of examination procedures
The laboratory shall obtain information from the manufacturer/method
developer for confirming the performance characteristics (expected
zone diameters, MICs) of the procedure.
What are you aiming to achieve? What are the intended outcomes of
the method being used?
Appropriate control strains (ATCC/NCTC) subjected to the method
should achieve the intended result (zone diameter, MIC) when we do
the test.
ISO 15189:2012
15
CPA is a UKAS
company
5.5.1 Selection, verification and validation of examination processes
5.5.1.2 Verification of examination procedures
Validated examination procedures used without modification shall be
subject to independent verification by the laboratory before being
introduced into routine use.
Generally acceptable for retrospective verification of historical methods
(inc. BSAC), e.g. demonstrable ability to meet performance claims
through quality assurance.
What evidence do you have to demonstrate new methods (inc.
EUCAST) have been verified as fit for purpose before being applied to
clinical isolates and informing clinical decisions.
ISO 15189:2012
16
CPA is a UKAS
company
5.5.1 Selection, verification and validation of examination processes
5.5.1.2 Verification of examination procedures
The independent verification by the laboratory shall confirm, through
obtaining objective evidence (in the form of performance
characteristics) that the performance claims for the examination
procedure have been met. The performance claims for the
examination procedure confirmed during the verification process shall
be those relevant to the intended use of the examination results.
How can you show the assessors that you can accurately determine
the susceptibility of your isolates when using the validated AST
method?
ISO 15189:2012
17
CPA is a UKAS
company
5.5.1 Selection, verification and validation of examination processes
5.5.1.2 Verification of examination procedures
The laboratory shall document the procedure used for the
verification and record the results obtained.
Show us your verification procedure. Is the technical approach to
verification sufficient and suitable for the intended use of the method?
Staff with the appropriate authority shall review the verification
results and record the review.
Who has reviewed the data and signed off the verification? Are they
competent to do so?
ISO 15189:2012
18
CPA is a UKAS
company
uncertainty (of measurement)
parameter, associated with the result of a measurement, that
characterizes the dispersion of the values that could reasonably be
attributed to the measurand (VIM definition)
VIM:
International Vocabulary of Basic and General Terms in Metrology
(ISO)
Martin:
How sure are you that the measurement is correct?
Measurement uncertainty
19
CPA is a UKAS
company
5.5.1.4 Measurement uncertainty of measured quantity values
The laboratory shall determine measurement uncertainty for each
measurement procedure in the examination phase used to report
measured quantity values on patients’ samples.
Where examinations include a measurement step but do not report a
measured quantity value, the laboratory should calculate the
uncertainty of the measurement step where it has utility in assessing
the reliability of the examination procedure or has influence on the
reported result
ISO 15189:2012
20
CPA is a UKAS
company
Metrological Traceability
Property of a measurement result whereby the result can be related to
a reference through a documented unbroken chain of calibrations, each
contributing to the measurement uncertainty (VIM)
Applicable to:
• Calibration of callipers, rulers or whatever else is measuring zone
diameters
• How sure are you of thee diameters?
• Can you demonstrate traceability?
Traceability
21
CPA is a UKAS
company
5.3.1.4 Equipment calibration and metrological traceability
The laboratory shall have a documented procedure for the calibration of
equipment that directly or indirectly affects examination results
Where this is not possible or relevant, other means of providing
confidence in the results shall be applied
- use of certified reference materials
ISO 15189:2012
22
CPA is a UKAS
company
• Entirely up to you
• Not applicable (e.g. MU)
– OK, so how have you come to that conclusion?
• Often see overarching/standalone procedures on MU, validation, verification & traceability/calibration
– OK, so how have you applied that to your AST method
• Need to consider how each applies to AST, do the work and document it
23
Documentation
CPA is a UKAS
company
• All of the above should be covered during the ISO 15189:2012 initial
assessment
• Highly likely that findings of non-conformity will be raised in relation to
AST
• Laboratory is expected to propose suitable improvement action at the
assessment to address and clear the findings
• Evidence of improvement action needs to be submitted (12 weeks) for
review by the Technical Assessor
• All findings must be cleared before grant of accreditation can be made
Initial assessment
24
CPA is a UKAS
company
Schedule of Accreditation
25
CPA is a UKAS
company
Schedule of Accreditation
26
CPA is a UKAS
company
Extension to Scope (ETS)
27
• Proposed changes to your assessed and accredited scope need to be
communicated to your UKAS Assessment Manager
• ETS process is required to add or change tests or methods on your
published Schedule of Accreditation
• New AC6 required detailing the additions / changes plus supporting
documentation, must include:
• SOPs
• Validation & verification data and summary
• Uncertainty of measurement estimation
• Measurement traceability chain
CPA is a UKAS
company
Extension to Scope (ETS)
28
• Many ETS applications run parallel to surveillance / reassessments
• Might be additional site time, often accommodated in existing effort
• If appropriate can be done by remote assessment, if so need:
• IQC performance data and summary
• EQA performance summary
• Training / competency records
• System suitability / evaluation summary
• Any other specific documentation
CPA is a UKAS
company
Extension to Scope (ETS)
29
CPA is a UKAS
company
Extension to Scope (ETS)
30
• UKAS has assessed your competence
to use BSAC methodology
• Your susceptibility testing using BSAC
methodology will be accredited
• UKAS has not assessed your competence
to use EUCAST methodology
• Your susceptibility testing using EUCAST
methodology will not be accredited until
ETS for this has been granted
CPA is a UKAS
company
Extension to Scope (ETS)
31
Inform UKAS early in the
change control process
• UKAS can arrange to assess your
competence to use EUCAST
methodology through ETS process
• Grant of ETS before clinical use
• Your susceptibility testing using
EUCAST methodology will be
accredited on the day you go live
CPA is a UKAS
company
Martin Stearn, UKAS Senior Assessment Manager
Thanks for listening
Any questions?
© UKAS June 2016