the quality system pamela youde nethersole eastern ... 3_basic... · pamela youde nethersole...

Basic QC Concepts

Lo Yun Chuen, SO (M)Chemical Pathology Laboratory

Pamela Youde Nethersole Eastern Hospital10 September 2014

Purchasing & Inventory

AssessmentOccurrence Management

Information Management

Process Improvement

Customer Service

Facilities & Safety

The Quality SystemOrganization Personnel Equipment

Documents & Records

Process Control

(QC & EQA) & Specimen

Management

Module 1: Core Laboratory / Rapid Response Laboratory - Automation / Instrumentation

Module 2: General equipmentModule 3: Quality control: Basic

concept, IQC, EQAP, case studyModule 4: Quality Management in

Laboratory Operation

Quality Management System

A quality management system can be defined as “coordinated activities to direct and control an organization with regard to quality”.

This definition is used by the International Organization for Standardization (ISO) and by the Clinical and Laboratory Standards Institute (CLSI).

In a quality management system, all aspects of the laboratory operation, including the organizational structure, processes and procedures, need to be addressed to assure quality.

Quality Management System Total Quality Management

Most healthcare organizations have adopted this concept

is a management approach to long-term success through customer satisfaction.

Six Sigma New trend Six Sigma seeks to improve the quality of process

outputs by identifying and removing the causes of defects (errors) and minimizing variability

Productivity Vs quality

ImplementingQuality Management

does notguarantee

anERROR-FREE

Laboratory

But it detects errors that may occur and prevents them from recurring

Organization

Personnel Equipment

Purchasing &

Inventory

Process Control

Information Management

Documents&

Records

Occurrence Manageme

ntAssessmen

t

Process Improvement

Customer Service

Facilities &

Safety

Laboratories notimplementing aquality managementsystem guaranteesUNDETECTED ERRORS

Total Quality Management (TQM)

Definition:Management philosophy and approach

Focus on processes and their improvements in order to satisfy customer need

TQM

Components:1. Quality Laboratory Process (QLP)2. Quality Control (QC)3. Quality Assurance (QA)4. Quality Improvement (QI)5. Quality Planning (QP)

TQM

QLP:Refer to analytical processes, general

policies, practices and procedure that define how all aspects of the work are done

TQMQC: Emphasize statistical control procedures and

non-statistical check procedures

Non-statistical checks including linearity checks, reagent and calibrator checks and temperature monitors

To assess the validity of results and control release of results in real time within the laboratory

TQMQA: Primarily measures performance characteristics

that beyond the borders of laboratory to areas of direct and immediate concern of our customers

Include turnaround time, specimen identification, patient identification and test utility

Determine the quality of the results generated by laboratory

A overall management plan to guarantee the integrity of system

Quality System

Quality Assurance

Quality Control

TQM

QI: Problems identified Re-plan the process to prevent the problem from

recurringQP: QP provides the focus on customers and

emphasizes the importance of understanding their needs and expectations,

Lead to the definition of quality goals.

TQMSummary:QC+QA: Detect the problems early to

prevent their consequences

QI+QP: Provide structured solving process to identify and rectify the problems to document new QLP to achieve quality requirements

New QLP measured and monitored through QC and QA

TQM

Quality System Overview

The Quality Assurance Cycle

•Data and Lab Management•Safety•Customer Service

Patient/Client PrepSample Collection

Sample Receipt and Accessioning

Sample TransportQuality Control

Testing

RecordKeeping

ReportingPersonnel CompetencyTest Evaluations

Why we need Quality Control?

Source of Laboratory Errors

Plebani and Carraro. Clin Chem 43:1348,1997

Source of Error Plebani et al

Pre -analytical 68 %

Analytical 13 %

Post-analytical 19 %

Why do laboratory errors occur?

QualityControl &

Assessment

PoorWorkload

Management

Understaffed

Non-validatedTests

InadequateAttentionTo Detail

Time Pressures

Poor Results Verification

Poor Sample Control

PoorQuality

Management

Quality control

Why we need QC? To detect, reduce and correct errors in

analytical process before release of patient results in order to produce accurate and precise results

Quality control

Why we need QC?

QC is one of the most important variables reviewed as part of the Laboratory Accreditation

ISO Family of QM Standards

Testing or

Calibration Laboratory

ClinicalLaboratory

Industry

Environment

Quality Control Quality Control

Components:1. Procedure2. Control materials3. Data interpretation

Normal Distribution

All values are symmetrically distributed around the mean

Characteristic “bell-shaped” curve Assumed for all quality control statistics

Normal Distribution

Freq

uenc

y

4.7’ 4.8’ 4.9’ Mean 5.1’ 5.2’ 5.3’

X

Normal Distribution

02468

10121416

# of

Obs

erva

tions

192 194 196 198 200 202 204 206 208 210 212

Serum glucose (mg/dL)

Mean

Accuracy and Precision The degree of fluctuation in the measurements

is indicative of the “precision” of the assay. The closeness of measurements to the true

value is indicative of the “accuracy” of the assay. Quality Control is used to monitor both the

precision and the accuracy of the assay in order to provide reliable results.

Accuracy Vs. PrecisionAccuracyHow well a measurement agrees with an accepted value

PrecisionHow well a series of measurements agree with each other

Precise and inaccurate

Precise and accurate

Precision and AccuracyPrecision and Accuracy

Imprecise and inaccurate Measures of Dispersion or Variability

There are several terms that describe the dispersion or variability of the data around the mean:

• Range• Variance• Standard Deviation• Coefficient of Variation

Range

Range refers to the difference or spread between the highest and lowest observations.

It is the simplest measure of dispersion. It makes no assumption about the shape

of the distribution or the central tendency of the data.

Calculation of Variance (S2)

222

1N)X(X2 /dlmgS 1

Calculation of Variance Variance is a measure of variability about

the mean. It is calculated as the average squared

deviation from the mean. the sum of the deviations from the mean,

squared, divided by the number of observations (corrected for degrees of freedom)

Degrees of Freedom

Represents the number of independent data points that are contained in a data set.

The mean is calculated first, so the variance calculation has lost one degree of freedom (n-1)

Calculation of Standard Deviation

mg/dlS 1N)x(x 2

1

variance

Calculation of Standard Deviation

The standard deviation (SD) is the square root of the variance it is the square root of the average squared

deviation from the mean SD is commonly used (rather than the

variance) since it has the same units as the mean and the original observations

SD is the principle calculation used in the laboratory to measure dispersion of a group of values around a mean

Standard Deviation and Probability

For a set of data with a normal distribution, a value will fall within a range of: +/- 1 SD 68.2% of the time +/- 2 SD 95.5% of the time +/- 3 SD 99.7% of the time

68.2%

95.5%99.7%

Freq

uenc

y-3s- 2s -1s Mean +1s +2s +3s

X

Standard Deviation and Probability

In general, laboratories use the +/- 2 SD criteria for the limits of the acceptable range for a test

When the QC measurement falls within that range, there is 95.5% confidence that the measurement is correct

Only 4.5% of the time will a value fall outside of that range due to chance; more likely it will be due to error

Calculation of Coefficient of Variation

The coefficient of variation (CV) is the standard deviation (SD) expressed as a percentage of the mean

Ideally should be less than 5%

100x meanSDCV

Monitoring/Interpretation of QC Data

Monitoring QC Data

Use Levey-Jennings chart Plot control values each run, make

decision regarding acceptability of runMonitor over time to evaluate the

precision and accuracy of repeated measurements

Review charts at defined intervals, take necessary action, and document

Levey-Jennings Chart

A graphical method for displaying control results and evaluating whether a procedure is in-control or out-of-control

Control values are plotted versus time Lines are drawn from point to point to

accent any trends, shifts, or random excursions

Levey-Jennings Chart- 20 - 15 - 10 - 5 0

+3SD

+2SD

+1SD

Mean

-1SD

-2SD

-3SD0

0.4

0.8

1 .2

Levey-Jennings Chart -Record Time on X-Axis and the Control Values on Y-Axis

80859095

100105110115

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24Con

trol

Val

ues

(e.g

. mg/

dL)

Time (e.g. day, date, run number)

Levey-Jennings Chart -Plot Control Values for Each Run

80859095

100105110115

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24Con

trol

Val

ues

(e.g

. mg/

dL)

Time (e.g. day, date, run number)

Levey-Jennings Chart Calculate the Mean and Standard Deviation;

Record the Mean and +/- 1,2 and 3 SD Control Limits

80859095

100105110115

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24

Mean

Day

+1SD

+2SD

+3SD

-1SD

-2SD-3SD

Levey-Jennings Chart -Record and Evaluate the Control Values

80

85

90

95

100

105

110

115

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24

Mean

Day

+1SD

+2SD

+3SD

-1SD

-2SD

-3SD

LJ chart

Aims : All assayed values for QC material should

be plotted and compared with the limits To detect and monitor process variation

over time Tool for long term monitoring of a process

such as shift and imprecision

LJ chart False rejection problems with LJ charts that use mean

+/-2s as control limits.- No. of control per run=1, false rejection rate=5%- No. of control per run=2, false rejection rate=9%- No. of control per run=3, false rejection rate=14%- No. of control per run=4, false rejection rate=18%

Review the chart using Westgard Rules based on statistical methods

To detect random error and systemic error

Systematic errorVs

Random error

Random error Vs Systematic error

Random error- create imprecision with variable manner

Systematic error- create bias with constant manner Trend and shift


Once error happened, consider: ParameterReagent/calibrator InstrumentMethodology Sample


Examples:

Random errors: Lack of maintenance Inconsistent reagent preparation Intermittent dripping (sample probe/reagent

probe) Faulty lamps/detectors Scratched reagent mixers ISE selectivity deteriorated; etc


Trends:


Shift :

Findings Over Time Ideally should have control values clustered

about the mean (+/-2 SD) with little variation in the upward or downward direction

Imprecision = large amount of scatter about the mean. Usually caused by errors in technique

Inaccuracy = may see as a trend or a shift, usually caused by change in the testing process

Random error = no pattern. Usually poor technique, malfunctioning equipment

When does the Control Value Indicate a Problem?

Consider using Westgard Control RulesUses premise that 95.5% of control values

should fall within ±2SDCommonly applied when two levels of

control are usedUse in a sequential fashion

Westgard Rules(Generally used where 2 levels of

control material are analyzed per run)12S rule13S rule22S rule

R4S rule41S rule10X rule

Westgard multi-rules

Warning forpossible systematicor random error


Identify systematicor random error


Identify systematicerror


Identify random error





Westgard Multi-rule approach

Combination of rules decided by individual laboratory applying on individual test

When a rule is violatedWarning rule = use other rules to inspect

the control pointsRejection rule = “out of control”

Stop testing Identify and correct problem Repeat testing on patient samples and controls Do not report patient results until problem is solved

and controls indicate proper performance

Solving “out-of-control” problems Policies and procedures for remedial

action

Troubleshooting

Alternatives to run rejection

LJ chart

Summary: 5% values will exceed 2SD when no error

existsDetect random error or systematic error

based on Westgard rules 100% error detection and 0% false

rejection cannot be achievedUsually power of error detection around

90% and false rejection around 5%

QC Planning

Single rule? Multi-rules?

Number of control measurements?

Concept of Six-Sigma

Total Analytical Error, TE

TE = Inaccuracy + Imprecision

Total Allowable Error (Medical Usefulness) should be greater than TE (Method / Assay Performance)

Source:http://www.qcnet.com/Portals/0/PDFs/BVValues1Final.pdf

What is Six Sigma?General goal for process performance

Six ‘Sigmas’ of process variation should fit within tolerance limits or quality requirement of product

2 different methodologies: Outcome measure, Predictive measure

Six SigmaOutcome measure – defect rate in term of

defects per million (DPM) - convert DPM to Sigma-metric using

standard conversion table - applicable to pre-analytic & post-analytic

lab processes - difficult to apply in analytical process

(true value?, out 2SD?, out TEa#?, medical mistake?, fatality?)

# TEa – Allowable total error

Six SigmaTotal Error Error

PercentProcess Sigma

1,000,000 100,000 10% 2.78

1,000,000 10,000 1% 3.83

1,000,000 5,000 0.5% 4.08

1,000,000 1,000 0.1% 4.59

1,000,000 500 0.05% 4.79

1,000,000 100 0.01% 5.22

1,000,000 50 0.005% 5.39

1,000,000 10 0.001% 5.76

1,000,000 5 0.0005% 5.92

1,000,000 1 0.0001% 6.25

Six Sigma Predictive measure – measure variation of

the process - calculate Sigma-metric: (Quality

Requirement – Bias) / Variability (SD or CV)

- Sigma also called Capability Index (Cps) - Quality requirement (TEa or ALE*) from

proficiency testing criteria or other suitable source (e.g. RCPA, CLIA & etc)

- Bias & variability from method validation, peer comparison, EQAP result or even routine QC data

* ALE – Allowable Limit of Error

Six Sigma

Analyte Conc. Precision (%CV)

Bias (% Bias) TEa Limit (%) Calculated Sigma

3 2.2 6.7 10 1.53 1.0 6.7 10 3.33 2.2 2.7 10 3.3

500 4.2 5.8 10 1.0500 4.2 5.8 20 3.4

Six Sigma Same bias, lower variability = Higher

Sigma Same variability, lower bias = Higher

Sigma Same bias & variability, less stringent

quality requirement = Higher Sigma

Sigma Error Rate3 66,803 / 106

4 6280 / 106

5 233 / 106

6 3.4 / 106

Capability Index

Based on Six Sigma concept

Definition:

In most cases can assume bias=0

Capability Index

Capability Index Capability Index

Capability Index Six Sigma

Capability Index

Cps<3 Incapable

Cps between 3 and 4 Barely capable

Cps between 4 and 6 Capable

Cps>6 Highly capable(World Class)

Capability IndexQC rules application based on Cps value: Cps>6-Any single rule Cps=5-Single rule QC with 2.5SD limits with 2-

3 controls Cps=4-Single rule QC with 2.5SD limits or

multi-rule QC with 3-6 controls Cps=3-Multi rule QC with 6-8 controls or More!! Cps<3-Nothing to do Consider change

methodology

Capability IndexSummary: Identify the performance of the assay

Select the appropriate QC rule to maximize the error detection

Select the appropriate number of QC run per day to reduce costs

Desirable laboratory QC system

Set up QC program including the QC material selection, QC limit set up, QC rule and no. of QC run per day selection ;etc

Follow program

Document all activities-QC procedures, QC failure and remedial actions

Take home message

Basic TQM structure

LJ QC chart setup and interpretation

Applications of capability index

References

1. R Pang. A practical Guide to Internal Quality Control. July 2010

2. CA Burtis and ER Ashwood (eds). Chapter 17 Quality Management. Clinical Chemistry Tietz textbook 1999. WB Saunders Co.

3. http://www.Westgard.com