The New EP23 ‒ Laboratory Quality Control Based on

Risk Management,Approved Guideline

Luann Ochs, MS

Senior Vice President – Operations

Clinical and Laboratory Standards Institute

Objectives

• Describe the various types of control processes.

• Identify CLSI document EP23-A as a resource for developing a laboratory quality control (QC) plan based on risk management.

• Review key aspects of risk management.

2

3

Today’s Quality Control

• Advantages– QC monitors the end product (result) of the entire test system.– QC has target values: if assay recovers target, then everything

is assumed stable (instrument, reagent, operator, sample).

• Disadvantages– When a problem is detected, one must go back and reanalyze

patients since last “good” QC.– If results are released, then results may need to be corrected.– For Point of Care devices, does traditional QC work for every

test?

• Need to get to fully automated analyzers that eliminate errors up front– Until that time, need a robust QC plan (QCP)

4

Types of Quality Control

• “On-Board” or Analyzer QC – built-in device controls or system checks

• Internal QC – laboratory-analyzed surrogate sample controls

• External QC – blind proficiency survey

• Other types of QC – control processes either engineered by a manufacturer or enacted by a laboratory to ensure result reliability

5

6

Quality Control Limitations

• No single QC procedure can cover all devices, because the devices may differ.

• QC practices developed over the years have provided laboratories with some degree of assurance that results are valid.

• Newer devices have built-in electronic controls, and “on-board” chemical and biological controls.

• QC information from the manufacturer increases the user’s understanding of device’s overall quality assurance requirements.

The Quality Control Toolbox

• Every QC tool has its strengths and weaknesses (there is no perfect QC tool).

• Implement a combination of tools in order to properly control a test.

• EP23 explains the strengths and weaknesses of the different QC processes, and helps the laboratory determine the right combination of tools: The Right QC

7

CLSI Document EP23

• Laboratory Quality Control Based on Risk Management; Approved Guideline

• James H. Nichols, PhD, DABCC, FACB, Chairholder of the document development committee

• EP23 describes good laboratory practice for developing a QCP based on the manufacturer’s risk mitigation information, applicable regulatory and accreditation requirements, and the individual health care and laboratory setting.

8

The Scenario

• CLSI document EP23 provides guidance on developing an appropriate QCP that will:

– Save time and money.– Use electronic and/or integrated QC features.– Use other sources of QC information.– Conform to one’s laboratory and clinical use of the test.

9

Developing a QCP

10

MEASURING SYSTEM INFORMATION

Medical Requirements forthe Test Results

Regulatory andAccreditationRequirements

Test System Information- Provided by the Manufacturer- Obtained by the Laboratory

Information AboutHealth Care andTest Site Setting

PROCESSRisk Assessment

OUTPUTQuality Control Plan

PROCESSPost-implementation Monitoring

Measuring System Information

• Gather information from several sources:– Medical requirements for the test results

• Allowable performance specifications via physicians– Regulatory and accreditation requirements

• Clinical Laboratory Improvement Amendments – Test/test system information

• User’s manual, reagent package insert, literature• Prior laboratory data

– Health care and testing site settings• Temperature conditions, operator training programs

11

Developing a Process Map

• Break down all phases of the test/test system into steps, so that weak points can be identified.

• Each step can be analyzed to find potential failure

modes that could present significant risk to patients.

• Process can then be further analyzed to see if controls can be put into place to avoid the failures.

12

Process Map

13

Developing a Process Map (cont’d)

• Compile this information. • A process map or a fishbone diagram is one example.

14

Incorrect Test Result

1 Samples

2 Operator

3 Reagents

5Measuring

System

4Laboratory Environment

Sample Integrity

Sample Presentation

- Lipemia- Hemolysis- Interfering subtances- Clotting- Incorrect tube

- Bubbles- Inadequate volume

Operator Capacity

Operator staffing

Atmospheric Environment

Utility Environment

- Training- Competency

- Short staffing- Correct staffing

- Dust- Temperature- Humidity

- Electrical- Water quality- Pressure

Reagent Degradation- Shipping- Storage- Used past expiration- Preparation

Quality Control Material Degradation- Shipping- Storage- Used past expiration- Preparation

Calibrator Degradation- Shipping- Storage- Used past expiration- Preparation

Instrument Failure

Inadequate Instrument Maintenance

- Software failure- Optics drift- Electronic instability

- Dirty optics- Contamination- Scratches

Identify Potential Hazards

The Risk Assessment

• Once the process map is created, examine it for places where errors could occur.

• Five major areas: – Samples– Operator– Reagents– Laboratory environment– Measuring system

15

Key Process Steps

Think about what steps can be taken to reduce errors “unrelated” to the actual testing

of the sample.

16

PREPARATIONPre-analytical

Pre-examination

TESTINGAnalytical

Examination

RESULT REPORTINGPost-analytical

Post-examination

Perform the Risk Assessment

• Identify the potential failures and their causes.– Review the process map, fishbone diagram, manufacturer’s

instructions, etc.

• Assess each potential failure.• Where a failure could occur, add an element to

the QCP that will reduce the possibility of that failure, making residual risk acceptable.– For some types of failures, the manufacturer’s information

may already have a quality check in place.

17

Perform the Risk Assessment

• “Risk Assessment” and “Risk Management” are formal terms for what you’re already doing.

18

19

Risk Definition

• Risk – the chance of suffering or encountering harm or loss (Webster's Dictionary and Thesaurus. Ashland, OH: Landall, Inc.; 1993).

• Risk, can be estimated through a combination of the probability of occurrence of harm and the severity of that harm (ISO/IEC Guide 51).

• Risk, essentially is the potential for an error to occur that could lead to patient/staff harm.

Life is a Continual Risk Assessment

20

You assess risk every day,

all the time, usually without even

thinking about it.

In the Laboratory

21

You do this every day!

Assess the Risk of Harm Due to Failures

22

Defective IVD:Hazard

Incorrect ordelayed

test result:Hazard

Incorrect ordelayed medical

treatment:HazardousSituation

Injury or death:Harm

Perform the Risk Assessment (cont’d)

• Construct a table; see which types of errors are detected and which ones are not. – If not detected, it must be included in the QCP.

• For each possible failure, assess the likelihood of that failure occurring, and the severity of consequences if it occurs.– Do this for each identified failure.– Use all of the information gathered in order to make

these assessments.

23

Risk Assessment Worksheet

24

Frequency (1 – 5) Example(Sometimes called “Probability”)

25

Common Terms

Rating Example (ISO 14971)

Practical Example

Frequent 5 ≥ 1/1,000 More than 1x/week

Probable 4 < 1/1,000 and ≥1/10,000

Once every few months

Occasional 3 < 1/10,000 and ≥1/100,000

Once a year

Remote 2 < 1/100,000 and ≥1/1,000,000

Once every few years

Improbable 1 < 1/1,000,000 and

≥10,000,000

Unlikely to ever happen

The laboratory must decide – and it may be different for different tests!

Severity (1 – 5) Example

26

Common Terms

Rating Possible Description(ISO 14971)

Catastrophic 5 Results in patient deathCritical 4 Results in permanent injury of life-

threatening injurySerious 3 Results in injury or impairment

requiring professional medical intervention

Minor 2 Results in temporary injury or impairment not requiring professional

medical interventionNegligible 1 Inconvenience or temporary discomfort

Detectability (1 – 5)

What is the likelihood that the control process detects or prevents the failure?

27

Common Terms

Rating Example

Low 5 Control is ineffective  4 Control less likely to detect the

failure  3 Control may or may not detect the

failure  2 Control almost always detects the

failureHigh 1 Control can detect the failure

Risk Acceptability Matrix

28

Probability of Harm

Negligible Minor Serious Critical Catastrophic

Frequent Unacceptable Unacceptable Unacceptable Unacceptable Unacceptable

Probable Acceptable Unacceptable Unacceptable Unacceptable Unacceptable

Occasional Acceptable Acceptable Unacceptable Unacceptable Unacceptable

Remote Acceptable Acceptable Acceptable Unacceptable Unacceptable

Improbable Acceptable Acceptable Acceptable Acceptable Unacceptable

Severity of Harm

Criticality

• Multiply Frequency x Severity x DetectabilityExample: Probable (4) x Catastrophic (5) x High likelihood to detect failure (1) = 20

29

Criticality Result

Low <10

Mid 10 – 20

High >20

Higher criticality numbers must have quality control actions in place.

Assemble the Quality Control Plan

• Use the information gathered earlier to assess all of the identified risks and their control measures.

• Construct the QCP.

• Include each of the identified QCP actions in the QCP.

30

Monitor QCP for Effectiveness

• Verify that the QCP that is put in place actually works

• Continue to monitor errors and control failures.

• If an error occurs:– Take the appropriate corrective action.– Investigate the cause of the error.– Once the cause is understood, evaluate whether any

changes need to be made in the QCP.

31

Monitor QCP for Effectiveness (cont’d)

• Review any complaints that the laboratory receives from health care providers. – These complaints may include pointing out another

source of QC “failure” that must be addressed.

• For patient safety, the QCP should be reviewed and monitored on an ongoing basis to ensure that the QCP is optimal.

32

Summary

• A QCP is necessary for result quality, and each QCP is unique.

• A QCP is the industry standard. It depends upon the extent to which the device’s features achieve their intended purpose in union with the laboratory’s expectation for ensuring quality results.

• Once implemented, the QCP is monitored for effectiveness and modified as needed to maintain risk at a clinically acceptable level.

33

EP23 Companion Products

34

Risk Assessment WorksheetImplementation Workbook

Plus – More fully worked examples coming soon

www.clsi.org