machine validation for clinical biochemistry analyzer...machine validation for clinical biochemistry...

21
Machine Validation for Clinical Biochemistry Analyzer Submitted 25 October 2016, Under Review Lenon Scotch 1 , Allen Matubu 2 , Danai Tavonga Zhou 1,3 1. University of Zimbabwe College of Health Sciences, Department of Medical Laboratory Sciences, P.O. Box AV 178, Avondale, Harare, Zimbabwe 2. University of Zimbabwe, College of Health Sciences, Department of Medicine, P.O. Box AV 178, Avondale, Harare, Zimbabwe 3. Africa University, Faculty of Health Sciences, Medical Laboratory Sciences, P. O Box 1320, Mutare, Zimbabwe Emails: [email protected] , [email protected] , [email protected], OR [email protected] ABSTRACT Unreliable results issued by many laboratories have dire consequences such as unnecessary treatment, treatment complications, failure to provide proper treatment, delay in correct diagnosis and additional, unnecessary testing. Optimum operating conditions are difficult to control in clinical laboratories and these negatively affect the function of clinical machines and quality of laboratory results. Therefore validation is necessary to determine a machine’s optimum operating conditions. Our cross-sectional study aimed to evaluate performance specifications of a new department analyzer compared to those established by the manufacturer before comparing the new machine’s performance to an old pre-validated machine serving as the Gold Standard. Blood samples (N=173) were analyzed for glucose, creatinine and urea, using the Gold Standard chemistry analyzer and the new machine respectively. The results obtained were assessed for accuracy, linearity, carryover, reference ranges, analytical measurable ranges, clinical reportable ranges, specificity and sensitivity to determine validity of the new machine. From our results accuracy, linearity, carryover, reference ranges, analytical measurable ranges, clinical reportable ranges, specificity and sensitivity were all within manufacturers’ stated claims. Results were also comparable to the Gold Standard machine.

Upload: others

Post on 17-Mar-2020

23 views

Category:

Documents


3 download

TRANSCRIPT

Page 1: Machine Validation for Clinical Biochemistry Analyzer...Machine Validation for Clinical Biochemistry Analyzer Submitted 25 October 2016, Under Review Lenon Scotch1, Allen Matubu2,

Machine Validation for Clinical Biochemistry Analyzer

Submitted 25 October 2016, Under Review

Lenon Scotch1, Allen Matubu

2, Danai Tavonga Zhou

1,3

1. University of Zimbabwe College of Health Sciences, Department of Medical Laboratory Sciences,

P.O. Box AV 178, Avondale, Harare, Zimbabwe

2. University of Zimbabwe, College of Health Sciences, Department of Medicine, P.O. Box AV 178,

Avondale, Harare, Zimbabwe

3. Africa University, Faculty of Health Sciences, Medical Laboratory Sciences, P. O Box 1320,

Mutare, Zimbabwe

Emails: [email protected], [email protected], [email protected], OR

[email protected]

ABSTRACT

Unreliable results issued by many laboratories have dire consequences such as unnecessary

treatment, treatment complications, failure to provide proper treatment, delay in correct

diagnosis and additional, unnecessary testing. Optimum operating conditions are difficult to

control in clinical laboratories and these negatively affect the function of clinical machines

and quality of laboratory results. Therefore validation is necessary to determine a machine’s

optimum operating conditions. Our cross-sectional study aimed to evaluate performance

specifications of a new department analyzer compared to those established by the

manufacturer before comparing the new machine’s performance to an old pre-validated

machine serving as the Gold Standard. Blood samples (N=173) were analyzed for glucose,

creatinine and urea, using the Gold Standard chemistry analyzer and the new machine

respectively. The results obtained were assessed for accuracy, linearity, carryover, reference

ranges, analytical measurable ranges, clinical reportable ranges, specificity and sensitivity to

determine validity of the new machine. From our results accuracy, linearity, carryover,

reference ranges, analytical measurable ranges, clinical reportable ranges, specificity and

sensitivity were all within manufacturers’ stated claims. Results were also comparable to the

Gold Standard machine.

Page 2: Machine Validation for Clinical Biochemistry Analyzer...Machine Validation for Clinical Biochemistry Analyzer Submitted 25 October 2016, Under Review Lenon Scotch1, Allen Matubu2,

Introduction

Background and literature review

Validation is the process of determining the performance characteristics of method or

procedure or process [1]. It is a prerequisite for judgment of the suitability of produced

analytical data for the intended use. This implies that a method may be valid in one situation

and invalid in another [2]. The process is intended to check that development and verification

procedures of an analyzer meet initial requirements, specifications, and regulations. It is a

process of establishing evidence that provides a high degree of assurance that the machine

accomplishes its intended requirements [1, 3]. Validation often involves acceptance of fitness

for purpose with end users and other product stakeholders.

Validation is important and can be used during accreditation of clinical laboratories and

certification, using common standards and practices. Regulatory agencies require that a

laboratory validate an instrument before it is put into use for patient testing [4, 5]. The

Clinical Laboratory Improvement Amendment (CLIA) regulations state that the laboratory

must demonstrate that it can obtain performance specifications comparable to those

established by the manufacturer when the laboratory introduces a new clinical chemistry

analyzer. Reliable data depends on a robust system design, which is initially validated by the

manufacturer using formal study procedures and subsequently verified by the end user

laboratory focusing on the laboratory’s specific patient populations [1, 2, 6].

Zimbabwe has established independent and internationally credible accreditation bodies

for example the Standards Association of Zimbabwe (SAZ). Standards are accomplished

through the establishment of a world-wide network of national accreditation bodies, which

Page 3: Machine Validation for Clinical Biochemistry Analyzer...Machine Validation for Clinical Biochemistry Analyzer Submitted 25 October 2016, Under Review Lenon Scotch1, Allen Matubu2,

will through Multilateral Agreements (MLAs) eventually; ensure that the competence of

certification bodies, inspection bodies and laboratories are assessed on the same principles,

regardless of their location. In Clinical Chemistry laboratories assessments are based on the

harmonized ISO standards for example ISO 15189 Certification for Medical Laboratories

[3,7].

From a medical perspective, the value of an automated clinical chemistry analyzer is to

provide physicians and other health care providers with reliable clinical data for patient

management. All results must be statistically and medically comparable on any clinical

chemistry analyzer [8, 9]. Before reporting patient test results, the laboratory needs to

demonstrate the accuracy and precision of its analyzer/s [6, 10].

Laboratory results influence 70%-75% of medical diagnosis hence quality of laboratory

service directly affects the quality of health care. Laboratory results must be as accurate as

possible, and laboratory operations must be reliable, and reporting must be timely in order to

be useful in a clinical or public health setting. If inaccurate results are provided, the

consequences can be very serious [11], for example, there may be unnecessary treatment,

treatment complications, failure to provide the proper treatment, delay in correct diagnosis

and additional and unnecessary diagnostic testing [9].

We carried out a validation procedure for a new machine at the University of Zimbabwe,

College of Health Sciences, in the Department of Medical Laboratory Sciences. The

validation involved a comparison between two machines: an old machine which we refer to

as the Gold Standard and a new machine. The aim of this report is to avail information to

other laboratories about the practical aspects of machine validation and share results of a

validation process in our setting. Literature on validation processes in clinical chemistry

laboratories in our setting is very scarce.

Page 4: Machine Validation for Clinical Biochemistry Analyzer...Machine Validation for Clinical Biochemistry Analyzer Submitted 25 October 2016, Under Review Lenon Scotch1, Allen Matubu2,

Materials and Methods

Laboratory Methods

The procedure focused firstly on the laboratory’s own procedures using the new analyzer.

During this process, measurements of common chemistry analytes such as: glucose,

creatinine and urea were performed. This was to ensure that the equipment was operating

within established parameters, giving reproducible results that meet pre-determined

specifications [12]. The validation procedure also verified the reportable range of test results

for the new machine’s test system and showed whether manufacturer's reference intervals are

appropriate for our laboratory's patient population [13, 14].

Accuracy Testing

In order to verify accuracy: glucose, creatinine and urea levels of 173 serum specimens were

determined using the new analyzer. Results, obtained using the new analyzer were compared

with the manufacturer’s insert details of accuracy determination [10]. Standard deviation and

coefficient of variance had to meet manufacturer’s stated claims [5]. Results were then

compared with those obtained from the Gold Standard chemistry analyzer. Standard deviation

and coefficient of variances were used to compare the two machines [5, 9, 10]. Limits of

acceptability for glucose, creatinine and urea levels for new analyzer were set by the

Laboratory Quality Assurance Department [4, 5, 10].

Precision

Glucose, creatinine and urea levels of multi sera controls were determined using the new

chemistry analyzer. Coefficient of variances were determined and used to assess the precision

of the machine [4]. Within run and between run reproducibility were determined by running

the negative pathological controls and normal controls as follows: For within run, 20

Page 5: Machine Validation for Clinical Biochemistry Analyzer...Machine Validation for Clinical Biochemistry Analyzer Submitted 25 October 2016, Under Review Lenon Scotch1, Allen Matubu2,

replicates of pathological control and 20 replicates positive control were tested in one run

each. For between run reproducibility, both pathological and normal controls were tested 5

times per day for 4 days to obtain a total of 20 replicates each [6, 9].

For the between run and within run precision, the results from the normal and pathological

controls were used to calculate the coefficient of variance (CV). The CV obtained was

compared to the manufacturer’s CV. The laboratory CV had to be less than or equal to the

manufacturer’s stated CV [4, 6, 10].

Linearity Testing

Glucose, creatinine and urea levels of five (5) serum specimens for each parameter were

determined using Gold Standard analyzer and new chemistry analyzer respectively to assess

the linearity of the machine. When plotted, the values had to be equidistant from each other

[16, 17, 18]. Six specimens were tested three (3) times each for each parameter and the data

was plotted immediately to identify and correct any outliers [18]. Data was plotted in

regression analysis program. The Linearity spreadsheet performed all necessary calculations

[18]. Mean values from the Gold standard were plotted on the X-axis [17, 18]. Mean values

for the new analyzer were plotted on the Y-axis [18]. The slope and intercept were calculated

using linear regression. This can also be done using the Excel slope and intercept functions

[17]. Using slope and intercept, a predicted Y value complementary to each X value was

calculated [18]. The predicted Y values were plotted versus the corresponding known X

values on the same graph. A straight line was drawn to connect all the predicted Y points on

the graph [18]. Each measured Y value was subtracted from the associated predicted Y value.

The difference is the systematic error due to non-linearity [18]. Systematic errors were then

compared to 50% of the total error. Systematic errors had to be less than 50% of the total

error [17] to prove validity of the new machine.

Page 6: Machine Validation for Clinical Biochemistry Analyzer...Machine Validation for Clinical Biochemistry Analyzer Submitted 25 October 2016, Under Review Lenon Scotch1, Allen Matubu2,

Analytical Measurable Range (AMR)

Multi Sera control samples from linearity procedure were used [9, 13]. The lowest glucose,

urea and creatinine levels of the control sample were diluted to verify the low end of

Analytical Measurement Range (AMR) [13]. The highest sample values for glucose, urea and

creatinine levels obtained from linearity were used as the high end of the AMR [9]. Five

samples were run each in duplicate and the results averaged [14]. Data was evaluated

immediately to identify and correct any problems [9]. The reportable range had to lie within

the manufacturer’s Analytical Measurable Range (AMR) [9, 14]. The manufacturer’s upper

limit was accepted if the known sample was within the percent Total Allowable Error (TEa)

of laboratory’s AMR upper limit [9]. Measured values also had to be within TEa of the Gold

Standard machine values [8, 14]. The manufacturer’s lower limit could be accepted if the

known sample was within the minimum detectable difference or percent TEa of the lower

limit [8]. Measured values also had to be within TEa of the Gold Standard machine values

[8].

Clinical Measurable Range (CRR)

Four dilutions were made to cover the clinical measurable range, since minimum amount of

dilution was ideal and since accuracy decreases with increasing dilution [8, 14]. The

maximum value of dilution allowed could not exceed the manufacturer’s recommendations

for dilution [14]. Any sample that did not give a numerical value beyond this allowed

dilution is reported as greater than the upper end of the CRR [8]. A sample (H) with a very

high concentration of the analyte and a sample (L) with a very low concentration were

chosen. Eleven replicates of the low sample and ten replicates of the high sample were run in

the following order and no other sample could be run within the series

L1/L2/L3/H1/H2/L4/H3/H4/L5/L6/L7/L8/H5/H6/L9/H7/H8/L10/H9/H10/L11. A passing

Page 7: Machine Validation for Clinical Biochemistry Analyzer...Machine Validation for Clinical Biochemistry Analyzer Submitted 25 October 2016, Under Review Lenon Scotch1, Allen Matubu2,

status was given if carryover was less than the error limit which was based upon three times

the low-low SD [4, 8]. The manufacturer’s stated sensitivity was used.

Reference Ranges

To verify or transfer a published range, the 20 specimens for glucose, creatinine and urea

with normal values were analyzed. Ranges were determined using a non-parametric statistical

method to determine the 95% reference limits. The lower and upper reference limits were

defined as the 2.5th

and 97.5th

percentiles, respectively [13, 14].

Chemistry Analyzer operations

The new analyzer was run according to manufacturer’s instructions found in its operating

manual. Parameters were selected and tested for validation purposes, for example: urea,

creatinine and glucose. These parameters have the most diagnostic value and can be

confirmed using the manufacturers’ co-efficient of variances (CVs).

Study Design and Ethical Considerations

The cross sectional analytical study was carried out on 173 samples although minimum

sample size for validation is as low as 20. We hoped to improve power of study and therefore

the generalizability of our results to other laboratories in our setting. Blood samples in plain

tubes and fluoride tubes were used. Permission to carry out the proposed project was sought

from relevant authorities whilst ethical clearance was sought from the Joint Research Ethics

Committee of the University of Zimbabwe, College of Health Sciences and Parirenyatwa

Group of Hospitals (JREC).

Page 8: Machine Validation for Clinical Biochemistry Analyzer...Machine Validation for Clinical Biochemistry Analyzer Submitted 25 October 2016, Under Review Lenon Scotch1, Allen Matubu2,

Laboratory Processing and Analysis

Blood samples in plain and fluoride tubes were de-identified for confidentiality and assigned

numerical identifiers. Samples were centrifuged at 3000 revolutions per minute (rpm) for five

minutes and serum and plasma were aspirated using a micropipette. The serum and plasma

were aliquoted into serum pots and stored in a refrigerator at 2 -8 OC. Samples were thawed

at room temperature on the day of assaying. Samples were assayed on the new chemistry

analyzer and Gold Standard chemistry analyzer and results compared.

Results

Precision

The within-run precision and between-day precision results showed that the CVs for urea,

glucose and creatinine on normal and pathological control samples were within

manufacturer’s stated values (Tables 1 and 2 and Figs 1 and 2).

Table 1: Within-run Precision for New Analyzer

Analyte

Expected Results Observed Results

Acceptability

Within- run

Mfg’s

Precision

25% of

CLIA

Normal Control

CV%

Control Path

CV%

Urea 1% 2.25% 0.07% 0.04% Acceptable

Creatinine 10% 3.75% 0.03% 0.02% Acceptable

Glucose 1% 2.50% 0.04% 0.02% Acceptable

Page 9: Machine Validation for Clinical Biochemistry Analyzer...Machine Validation for Clinical Biochemistry Analyzer Submitted 25 October 2016, Under Review Lenon Scotch1, Allen Matubu2,

Table 2: Between-day Precision for New Analyzer

Analyte

Expected Results Observed Results

Acceptability Between-day

Mfg’s

Precision

33% of

CLIA

Normal

Control

CV%

Control Path

CV%

Urea 1% 2.97% 0.16% 0.02% Acceptable

Creatinine 10% 4.95% 0.002% 0.053% Acceptable

Glucose 1% 3.3% 0.31% 0.122% Acceptable

Figure 1: Correlation of glucose results between new and gold standard analyzer

(Accuracy)

Figure 2: Correlation of creatinine results between new and gold standard analyzer

(Accuracy)

Page 10: Machine Validation for Clinical Biochemistry Analyzer...Machine Validation for Clinical Biochemistry Analyzer Submitted 25 October 2016, Under Review Lenon Scotch1, Allen Matubu2,

Accuracy

Results for the measurement of glucose, creatinine and urea on the new analyzer were

accurate based on the Coefficient of Variance and Standard Deviation (SD) obtained for these

analytes (Fig 3). Results were within manufacturer’s claims (Table 3, 4) and were accepted

on the basis of Total Allowable Error (TEa) values for urea, glucose and creatinine (Table 5).

Table 3: Accuracy-1 for New Analyzer

Analyte

Total

Allowable

Error

Correlation

Coefficient

(R)

Linear

Regression

Statistics

Error Index

Range

% of

Error

Indices

-1.0 to 1.0 Acceptability

Expected

>0.975 Slope Intercept

Expected

-1.0 to1.0

Expected

≥ 95%

Urea 0.714mmol/

l or 9%

0.999 0.990 -0.128 -0.94-0.84 95% Acceptable

Creatinine 26.52umol/l

or 26.52%

0.997 0.970 1.724 -0.15-0.19 95% Acceptable

Glucose 0.33mmol/l

or 10%

1.000 1.002 -0.038 -0.34-0.21 95% Acceptable

Table 4: Accuracy-2 for New Analyzer

Analyte

Total

Allowable

Error

Correlation

Coefficient

(R)

MDP Error Index Range

Worst

Sigma

Metric Acceptabilit

y

Expected

>0.975

Expected

-1.0 to1.0

100% of

Error

Indices:

-1.0 to 1.0

Expected

:

>2.0

Urea 0.714mmol/

lor 9%

0.999 -0.3-0.2 yes 15.2 Acceptable

Creatinin

e

26.52umol/l

or 15%

0.997 -0.319-0.1 yes 65.7 Acceptable

Glucose 0.33mmol/l

or 10%

1.000 -0.017-

0.0428

yes 19.7 Acceptable

Page 11: Machine Validation for Clinical Biochemistry Analyzer...Machine Validation for Clinical Biochemistry Analyzer Submitted 25 October 2016, Under Review Lenon Scotch1, Allen Matubu2,

Table 5: Table of Total Allowable Error Recommended for Validation requirements:

Minimum Recommended Validation requirements for Chemistry Total Allowable

Error (TEa) for Some Common Analytes (http://www.dgrhoads.com/db2004/ae2004.php)

Analyte

Total Error Limits (whichever is greater)

(whichever is greater) Precision

Percentage Minimum detectable

difference or absolute value

Short Term

25% TE

Long Term

33% TE

Albumin ± 10% (1) ±0.2 g/dL, 2.0 g/L (4) 2.5% 3.3%

Alk. Phos ± 30% (1) ±5.0 U/L (4) 7.5% 9.9%

ALT ± 20% (1) ±5.0 U/L (4) 5.0% 6.6%

Amylase ± 30% (1) ±5.0 U/L (4) 7.5% 9.9%

AST ± 20% (1) ±5.0 U/L (4) 5.0% 6.6%

Bilirubin, Direct ± 20% (2) ± 0.4 mg/dL , 6.84 umol/L (2) 5.0% 6.6%

Bilirubin, Total ± 20% (1) ± 0.4 mg/dL, 6.84 umol/L (1) 5.0% 6.6%

Calcium ± 8.3% (4) ± 1.0 mg/dL, 0.25 mmol/L (1) 2.08% 2.74%

Chloride ± 5% (1) ±2.0 mmol/L (4) 1.25% 1.65%

Cholesterol ± 10% (1) ±3.0 mg/dL, 0.08 mmol/L (4) 2.50% 3.3%

Creatinine ± 15% (1) ± 0.3 mg/dL, 26.52 µmol/L (1) 3.75% 4.95%

Creatine Kinase ± 30% (1) ±5.0 U/L (4) 7.5% 9.9%

Glucose (serum/

CSF) ± 10% (1) ± 6.0 mg/dL, 0.33 mmol/L (1) 2.50% 3.3%

HDL Cholesterol ± 30% (1) ± 2.0 mg/dL, 0.05 mmol/L (4) 7.5% 9.9%

Lactate ± 10% (9) ± 0.2 mmol/L (9) 2.5% 3.3%

Lactate

Dehydrogenase ± 20% (1) ±5.0 U/L (4) 5.0% 6.6%

LDL-Chol. Calc ± 30% (5) Not available 7.5% 9.9%

LDL-Chol. Meas. ± 30% (2) Not available 7.5% 9.9%

Lipase ± 30% (2) ±8.0 U/L (4) 7.5% 9.9%

Magnesium ±25% (1) ±0.2 mg/dL, 0.08 mmol/L (4) 6.25% 8.25%

Phosphorus ±10.7% (2) ± 0.3 mg/dL, 0.097 mmol/L (2) 2.68% 3.53%

Potassium ± 12.3%* (4) ± 0.5 mmol/L (1) 3.08% 4.06%

Protein, Total

(Serum) ±10% (1) ± 0.2 g/dL, 2.0 g/L (4) 2.5% 3.3%

Protein, Total

(CSF) ±20% (11) Not available 5.0% 6.6%

Sodium ± 3.1%* (4) ± 4.0 mmol/L (1) 0.78% 1.32%

Transferrin ±20% (2) ±5.0 mg/dL, 0.05 g/L (7) 5.0% 6.6%

Triglycerides ±25% (1) ±4.0 mg/dL, 0.05 mmol/L (4) 6.25% 8.25%

Urea (BUN) ±9% (1) ± 2.0 mg N/dL, 0.714 mmol/L

1) 2.25% 2.97%

Uric Acid ±17% (1) ±0.5 mg/dL, 30 µmol/L (4) 4.25% 5.61%

Vitamin B-12 ± 30% (6) ±30 pg/mL, 22.2 pmol/L (6) 7.5% 9.9%

Page 12: Machine Validation for Clinical Biochemistry Analyzer...Machine Validation for Clinical Biochemistry Analyzer Submitted 25 October 2016, Under Review Lenon Scotch1, Allen Matubu2,

Linearity results

The linearity results obtained for urea, glucose and creatinine were acceptable (Figs 4, 5 and

6) implying that there was no significant difference in the performance of the Gold Standard

and new analyzer (Table 6)

Figure 3: Correlation of urea results between new and gold standard analyzer for

accuracy

Figure 4: New analyzer glucose linearity scatter plot

Page 13: Machine Validation for Clinical Biochemistry Analyzer...Machine Validation for Clinical Biochemistry Analyzer Submitted 25 October 2016, Under Review Lenon Scotch1, Allen Matubu2,

Figure 5: New analyzer creatinine linearity scatter plot

Table 6: Linearity for New Analyzer

Analyte

Linear Regression

Statistics

Allowable

Systematic Error Linear Range

Verified Evaluation

Slope Intercept 50% of TEa

Urea 0.059 -2.901 7.5 1.2-39.3 mmol/l Linear

Creatinine 1.623 -110.619 4.5% 10-1062 umol/l Linear

Glucose 0.044 -2.272 5% 0.6-31.57 mmol/l Linear

Analytical Measurable Range (AMR) and Clinical Reportable Range (CRR)

The AMR and CRR for urea, glucose and creatinine were within manufacturer’s AMR ranges

and within reportable ranges. For CRR after 1:10 dilution the lower and upper limits on

verified values obtained and the ranges had were the same as AMR and reportable range

(Table 7, Graph 6).

Expected Urea Concentration (mmol/L)

Page 14: Machine Validation for Clinical Biochemistry Analyzer...Machine Validation for Clinical Biochemistry Analyzer Submitted 25 October 2016, Under Review Lenon Scotch1, Allen Matubu2,

Table 7: Analytical Measurable Range (AMR) and Clinical Reportable Range (CRR)

for New Analyzer

Analyte Mfg’s

AMR

Low

Value

Verified

High

Value

Verified

Reportable

Range Dilutions CRR

DAIDS

Toxicity

Grade 4

Urea 1-40

mmol/L

7.5 8.63 1-40 1:10 1-40 Requires

dialysis

Creatinine 9-2420

umol/L

96.6 130 9-2420 1:10 9-2420 3.5xULN

Glucose 0.3-28

mmol/L

4.1 6.03 0.3-28 1:10 0.3-28 >27.75

Figure 6: New analyzer urea linearity scatter plot

Carryover

Carryover results obtained for urea, glucose and creatinine were less than the error limit of

three times low-low standard deviation (Table 8).

Table 8: Carryover for New Analyzer

Analyte Low-

low

Mean

High-

low

Mean

Low-low

Standard

deviation

High-low

Standard

deviation

Error

limit

Carryover Status

Urea 7.58 7.68 0.045 0.110 0.134 0.1 Pass

Creatinine 91.68 91.8 0.045 0.071 0.134 0.12 Pass

Glucose 2.706 2.722 0.005 0.008 0.016 0.016 Pass

Expected Urea Concentration (mmol/L)

Page 15: Machine Validation for Clinical Biochemistry Analyzer...Machine Validation for Clinical Biochemistry Analyzer Submitted 25 October 2016, Under Review Lenon Scotch1, Allen Matubu2,

Sensitivity and Specificity

Sensitivity and Specificity results were adopted from the manufacturer’s instrument package

inserts (Table 9). Interference substances such as: icterus, haemolysis, lipemia, ascorbic acid

and bilirubin were assumed to have no significant effects on glucose, creatinine and urea

levels.

Table 9: Summary of manufacturer’s claims on Specificity and Sensitivity for New

Analyzer

Analyte

Specificity (Interfering Substances) Sensitivity Confidence

Urea Icterus – No significant effect

Hemolysis – No significant effect

Lipemia – No significant effect; >Abs flagging may

occur

Ascorbic acid- No significant effect

Billirubin- No significant effect

1 mmol/L 99.7%

Creatinine Icterus – No significant effect

Hemolysis – No significant effect

Lipemia – No significant effect; >Abs flagging may

occur

Ascorbic acid- No significant effect

Billirubin- No significant effect

9 umol/L 99.7%

Glucose

Icterus –No significant effect

Hemolysis- No significant effect

Lipemia- No significant effect

Ascorbic acid- No significant effect

Billirubin- No significant effect

0.3

mmol//L

99.7%

Reference ranges

The reference ranges; results obtained on established patients’ reference ranges had to be

comparable with manufacturer’s reference ranges. Verified ranges for urea and glucose were

within manufactures’ stated ranges (Table 10).

Page 16: Machine Validation for Clinical Biochemistry Analyzer...Machine Validation for Clinical Biochemistry Analyzer Submitted 25 October 2016, Under Review Lenon Scotch1, Allen Matubu2,

Table 10: Reference ranges for New Analyzer

Analyte

Mfg’s

reference

range

Reference range

cited

Verified

reference

Range

Established

patients reference

range

%verified

expected

(>/= 90%

Urea

6.38-8.63

mmol/L

Mfg’s reference

range

7.5-8.63

mmol/L

2-62.5 mmol/L 100%

Creatinine

77.9-105.3

ummol/L

Mfg’s

Reference

ranges

96.6-130

ummol/L

33-1726 ummol/L 85%

Glucose

3.9-6.4

mmol/L

Mfg’s reference

range

4.1-6.03

mmol/L

4.1-6.4 mmol/L 100%

Discussion

Validation aims to establish evidence that provides a high degree of assurance that the

machine accomplishes its intended requirements [1, 3, 15]. Overall results for all parameters

measured were within the manufactures’ stated claims. The within-run precision and

between-day precision results agreed with manufacturer’s stated Coefficient of Variances

(CVs) for urea, glucose and creatinine on normal and pathological control samples (Tables 1

and 2). The Coefficient of Variance (CV) and Standard Deviation (SD) for urea, glucose and

creatinine fell within defined Total Allowable Error (TEa). TEa values were calculated values

adopted from minimum recommended validation requirements for chemistry Total Allowable

Error (Tea) listed in Table 5 [8, 9, 16]

The measurement of glucose, creatine and urea with new analyzer was accurate based on

the correlation coefficient (CV) and Standard Deviation (SD) obtained for these analytes. CV

and SD had to be within manufacturer’s claims (Table 3), and were acceptable on the basis of

Total Allowable Error (TEa) values for urea, glucose and creatinine (Table 5). Error indices

and % error index lay within expected range (Table 3). Medical Decision Point (MDP) error

Page 17: Machine Validation for Clinical Biochemistry Analyzer...Machine Validation for Clinical Biochemistry Analyzer Submitted 25 October 2016, Under Review Lenon Scotch1, Allen Matubu2,

index ranges were acceptable and the 100% error indices fell within the manufactures’ stated

claims.

The linearity results obtained for urea, glucose and creatinine were also acceptable (Figs 4,

5 and 6). i.e. there was no significant difference in the performance of the Gold Standard and

new analyzer (Table 6). Systemic errors had to fall within acceptable limits and meet

manufacturer’s stated claims in order to show that a machine’s status or capacity correlates

with that of the Gold Standard analyzer. The systemic error were less than 50% total error

because validation protocols states that the systemic error of the machine being validated

should be less than 50% total error for it to pass the validation [17]. There was no significant

difference between the results produced by the new analyzer and those produced by the Gold

Standard Analyzer in our study.

The Analytical Measurable Ranges and Clinical Reportable Ranges (AMR and CRR) for

urea, glucose and creatinine had to be within manufacturer’s AMR ranges and within

reportable ranges. For CRR after 1:10 dilution the lower and upper limits on verified values

was the same as AMR and reportable range (Table 7, to show that the new machine is reliable

[13]. Carryover results obtained for urea, glucose and creatinine fell below the error limit of

three times low-low standard deviation (Table 8) suggesting that the machine’s error is within

expected range and that its error was insignificant.

Sensitivity and Specificity results were adopted from the manufacturer’s instrument

package inserts. Interference substances such as icterus, haemolysis, lipemia, ascorbic acid

and bilirubin were assumed to have no significant effects on glucose, creatinine and urea

(Table 9). Reference ranges results obtained on established patients’ reference ranges were

either lower or higher than manufacturer’s reference ranges due to different environmental

and operating conditions. Verified ranges for urea and glucose were within manufactures’

Page 18: Machine Validation for Clinical Biochemistry Analyzer...Machine Validation for Clinical Biochemistry Analyzer Submitted 25 October 2016, Under Review Lenon Scotch1, Allen Matubu2,

stated ranges and creatinine higher upper limit and lower limit were within manufacturers’

range (Table 10).

Conclusion

The validation results for the new analyzer showed that machine’s accuracy, precision,

carryover and verified reference ranges were within the manufacturer’s stated claims. The

machine produced results that were more or less comparable to those of the Gold Standard

analyzer. We could safely conclude that the new machine produces acceptable, comparable

results for patient management. Therefore the manufacturer’s claims were proved to be true

under the environment which assessments were made.

Limitations

The greatest challenge encountered during the study was the unavailability of literature from

other validation studies because validation reports are deemed private and confidential hence

they cannot be published. The manufacturer’s claims were the only available information for

comparison.

Recommendations for future

This paper will allow fellow clinical scientists in our setting to validate their analyzers using

tried and tested methods that are not too costly and can be carried out with minimal human

and chemical resources. Sharing of further information as published papers is also one way to

share skils and knowledge necessary to carry out this vital pre-analytical quality assurance

exercise.

Page 19: Machine Validation for Clinical Biochemistry Analyzer...Machine Validation for Clinical Biochemistry Analyzer Submitted 25 October 2016, Under Review Lenon Scotch1, Allen Matubu2,

Acknowledgements

1. University of Zimbabwe, College of Health Sciences, Department of Medical Laboratory

Sciences, academic and technical staff for assistance with the practical work

2. University of Zimbabwe, College of Health Sciences, Department of Community Medicine

for study design and statistical analyses

3. Parirenyatwa Group of Hospitals, Department of Clinical Chemistry for technical

assistance

References

1. Global Harmonization Task Force: Quality Management Systems, Process Laboratory

Validation Guidance (GHTF/SG3/N99-10), (2004) 2nd

Ed, 3.

2. S. Berwouts, M.A. Morris, E. Dequeker, Approaches to quality management in a genetic

testing laboratory, European Journal of Human Genetics, 18, 2010, 1-19.

3. D. Burnett, International Organization for Standardization (ISO) 15189:2003 quality

management, evaluation and continual improvement. Clinical Chemistry and Laboratory

Medicine, 44, 2006, 133-739.

4. Clinical and Laboratory Standards Institute (CLSI), Evaluation of Precision Performance

of Quantitative Measurement Methods, (3rd

Ed.) (CLSI document EP5-A2, ISBN 1-56238-

542-9, Pennsylvania, USA,2004).

5. W. W. Hauck, W. Kock, D. Abernethy, R. L.Williams, Making sense of trueness,

precision, accuracy and uncertainty. Pharmacopeial Forum, 34, 2008, 838-842.

6. Clinical and Laboratory Standards Institute (CLSI), User verification of performance for

precision and trueness (2nd

Ed.) (CLSI document EP15-A2 (ISBN 1-56238-574-7,

Pennsylvania,USA, 2005).

Page 20: Machine Validation for Clinical Biochemistry Analyzer...Machine Validation for Clinical Biochemistry Analyzer Submitted 25 October 2016, Under Review Lenon Scotch1, Allen Matubu2,

7. International Organization for Standardization (ISO), Medical laboratories-Particular

requirements for quality and competence (ISO 15189, 2007).

8. Clinical and Laboratory Standards Institute (CLSI), Estimation of total analytical error for

clinical laboratory methods, (CLSI EP21-A, 2003).

9. A. Menditto, M. Patriarca, B. Magnusson, Understanding the meaning of accuracy,

trueness and precision, Accreditation Quality Assurance, 12, 2007, 45-47.

10. Clinical and Laboratory Standards Institute (CLSI), Evaluation of the linearity of

quantitative measurement approved guideline (2nd

Ed.), (CLSI document EP6-A, ISBN 1-

56238-498-8, Pennsylvania, USA, 2005).

11. International Organization for Standardization (ISO), General requirements for the

competence of testing and calibration of laboratories, (ISO/IEC 17025, 2005)

12 D. Burnett, C. Blair, Standards for the medical laboratory-harmonization and subsidiarity,

Clinical Chemistry Acta, 309, 2001 137-145.

13. Clinical and Laboratory Standards Institute (CLSI), How to define and determine

reference intervals in the clinical laboratory, (CLSI document C28-A, Pennsylvania, USA,

2000).

14. Clinical and Laboratory Standards Institute (CLSI), Defining, establishing, and verifying

reference intervals in the clinical laboratory proposed guideline (3rd

Ed.), (CLSI document

C28-P3 ISBN 1-56238-663-8, Pennsylvania, USA, 2001).

15. D. Burnett, C. Blair, M. R. Haeney, S. L. Jeffcoate, K. W. Scott, D. L. Williams, Clinical

pathology accreditation: standards for the medical laboratory, Journal of Clinical Pathology,

55, 2002, 729-733.

16. A. J. Saah, D. R. Hoover, “Sensitivity” and “specificity” reconsidered: the meaning of

these terms in analytical and diagnostic settings. Annals of Internal Medicine, 126(1), 1997,

91-94.

Page 21: Machine Validation for Clinical Biochemistry Analyzer...Machine Validation for Clinical Biochemistry Analyzer Submitted 25 October 2016, Under Review Lenon Scotch1, Allen Matubu2,

17. Clinical and Laboratory Standards Institute (CLSI), Evaluation of the linearity of

quantitative measurement procedures: A statistical approach, (CLSI document C6-A,

Pennsylvania, USA 2003), 23(16).

18. Clinical and Laboratory Standards Institute (CLSI). Evaluation of precision performance

of quantitative measurement methods; Approved guideline (2nd

Ed.)