Module 6:QC Basic Rules and Charts

Analysis, Charts and Interpretation of Rules

Learning Objectives

Upon completion of this module you will be able to:

• Define: Control materials, Calibration materials, Levey-Jennings Chart, Westgard Single Rule and Multi-Rule Criteria, Systemic Error, & Random Error

• Describe a quality control sample in terms of assayed versus unassayed and their uses

Upon completion of this module you will be able to:

• Discuss the following Westgard rules: – 13s

– 12s

• Apply the Westgard rules listed above to decide whether an analytic run is accepted or rejected.

• List which Westgard rules detect systematic versus random error.

Learning Objectives

– 22s

– 6x

– 7t

Definitions

• Control materials: Protein-based materials made or manufactured with a composition similar to patient samples that have or have not been analyzed for concentration so they have expected or no expected values.

Definitions

• Certified reference material (CRM)– “A reference material that has one or more values

certified by a technically valid procedure and is accompanied by, or is traceable to, a certificate or other document that is issued by a certifying body.” [CLSI] NBS (National Bureau of Standards)

• Calibrator: purified solution used to adjust electrical output of an instrument and should trace back to a CRM.

• Levey-Jennings Chart: A visual presentation of daily quality control values plotted on a chart using mean, standard deviation, and +/- 3 Std Dev range criteria

• Westgard Single Rule and Multi-Rule Criteria: A set of rules used to evaluate quality control values as acceptable or not

Definitions

• Systemic Errors: Errors in the test system usually caused by a malfunction that affects all tests

Definitions

• Random Errors: Errors that statistically occur unpredictably and do not indicate system malfunction but do indicate an individual test or component malfunction

Need for Quality Control Samples

• Patient results are “unknowns”. Analysis of the patient sample produces a number.But…is the result valid?

• Patient may have previous result. But…– Change from previous may be expected– No change from previous result may be expected

Quality control samples are “knowns.” They have been made or manufactured with a composition similar to patient samples and have been analyzed for concentration before they are put into use so they have expected results.

Therefore, if Quality Control samples are analyzed and expected results are obtained, we can assume our system is operating correctly. orIf Quality Control samples are analyzed and do not give expected results, what should be done?

Need for Quality Control Samples

• Preserved to maintain accurate and precise results

• Assayed –provided with mean and s for analytes

• Unassayed – you must determine mean and s

Quality Control Samples Are

Why is it advisable to run controls at different levels?

Interactive Opportunity

• Analyzed solely for quality control purposes

• Not for calibration

• Should be used for qualitative and quantitative testing as known results to check if method to measure patient results is reliable.

Quality Control Sample Use

Is it sufficient to use only one calibrator to set the output of an analyzer?

Interactive Opportunity

Qualitative Versus Quantitative Results

• What are some examples of qualitative tests in the laboratory?

• What quality control samples would you use to verify the test is working so that patient results may be reported?

QC Results and Expectations for Qualitative Tests

• Quality Controls will be a known positive and known negative material

• The results will be recorded in a log book with the date, techs initials, and either ‘positive’ or ‘negative’ written in by the tech

• Reason for running controls with qualitative tests is to check that the reagent(s) are working as they are supposed to and give accurate results

Gaussian distribution

QC Results and Expectations for Quantitative Tests

What does the “Internal” Quality Control on a test cartridge indicate?

Interactive Opportunity

QC Sample Analysis as an Alarm System

QC sample result should: • Indicate when the test system is not

working properly

• Not indicate a false alarm

• It should be 100% reliable

Common QC Sample Rules Single Rule:

• Accept the analytical run if both quality control results fall within mean +/- 2 standard deviations range.

• Reject the analytical run and troubleshoot the problem if one or both quality control results are outside +/- 2 standard deviations from the mean.

QC Sample Result Outside of Expected Range

QC value < +2 SD or > -2 SD from the mean– Expected most of the time– With 95% confidence. This means 95% of

acceptable results are in this range. 5% could still be acceptable i.e. false alarm

QC value > + 3 SD or < - 3 SD from the mean– Usually indicates a problem and should be

followed up on

Historical Use of QC Sample Results

• Shewart developed statistical tools to monitor quality

• Levey and Jennings – quality standards for the laboratory

• Westgard - established rules for monitoring quality control

Levey-Jennings Chart

Levey-Jennings Chart

Quality Control Result Limits• Mean is the central line.

• One standard deviation above = + 1 s

• One standard dev below = - 1s

• 2 standard dev above = + 2s

• 2 standard dev below = -2s

• 3 standard dev above = + 3s

• 3 standard dev below = -3s

Calculating Confidence Limits on the Chart

• For example, Control 1 has a mean of 200 and a standard deviation of 4 mg/dL.

• What is the 95% confidence limit or mean + 2 s limit?– The upper control limit, mean + 2s would be:

200 + 2*4 = 208 mg/dL

– The mean – 2s, lower control limit would be:200 - 2*4 = 192 mg/dL

95% confidence limit is 192-208 mg/dL

Quality Control Sample Rules

• Decision Criteria• In control versus out of control• Control limits

– Mean + 2s– 95% limits– 5% false rejection– If 1 result exceeds 2s but not 3s and no other

shifts or trends, it is usually a warning but not a reason to reject the results.

QC Sample Measured with each Analytical Run

How long is a run?• Manual• Automated method

Laboratory Quality Control Policy (Quality Manual)stating your laboratory will not report results withoutappropriate QC.

Quality control Analysis SOP• How many control samples• How often run• How documented• How problems are solved• What Westgard rules are used.

QC Sample Analysis Procedure

Lab Results Need to Be...

• Accurate

• Precise

• Quality Control Rules should help to assess accuracy and precision.

• After verification with QC Rules, lab results also need to be reported timely.

Accuracy• Closeness of the measured result to the true

value

• Your control values may show precision but be inaccurate

• Instrument can repeatedly get the same value, but it’s not the right value

30

Precision

Precision is: The reproducibility or closeness of results to each other

31

How can we check for accuracy and precision?

Can you have precision without accuracy?

Can you have accuracy without precision?

Interactive Opportunity

Possible Causes for Random Errors

• Can be due to variations in line voltage, pipetting, dispensers, contamination, volume dispensed, bubbles in lines of reagents, etc*…

• A random error can be either positive or negative

• The direction and exact magnitude of the random error cannot be exactly predicted

• Random Errors affect reproducibility or precision of a test system

1 3s Rule Violation:• Usually indicates a Random Error

• When reviewing statistical data remember when using +/- 3 SD, there is a 0.3% chance of getting a control value outside the +/- 3 SD range

• Before running patient specimens you must troubleshoot AND then rerun the control and the values must be within +/- 2SD

• Check that controls are in date and well mixed*

Random Errors and Possible Causes

Westgard Rules

• Random error detected by 1 3s

• This means that 1 QC value exceeds -3s• Could be below -3s or above + 3s

2 2s Rule Violation:• Usually indicates a Systematic Error• Remember, the first time single control value is > or

< 2 SD it serves as a warning and you can run and report out your patient values

• BUT, the second sequential time the control value is > or < 2 SD, you must trouble shoot and find the reason for the error BEFORE you can run patient specimens and report out patient values

• Check that controls are in date and well mixed

Possible Causes for Random Errors

• Systematic error detected by 2 2s

• Could be below -2s or above + 2s

Westgard Rules

• Systematic Errors include bias/shifts, & trends

• These errors affect accuracy of the test system.

• Usually 6x and 7t Rule Violations

• Can be due to calibration lot changes, temperature changes in incubator unit, light source deterioration, electronics, reagent lot changes, etc.

• Systematic errors are always in one direction on a L J chart.

Systematic Errors

• A systematic difference or systematic error between an observed value and some measure of the truth.

• 6x Rule violations:– Usually indicate an error with your test system– On day 7, after 6 days of one or more control

values above or below the mean*, you must trouble shoot and find/correct error before running patient specimens

*Note the values of the 6 days are within +/- 2sd

Bias/Shift: 6x Rule Violation

• Systematic Error detected with 6x: shift

• 6 results are on the same side of the mean

Westgard Rules

• Systematic error when a series of control values are treated as a time series

• 7t Rule violations:– Usually indicate an error with your system– On day 8, after 7 days of one or more control

values moving either upward or downward, you must trouble shoot and find/correct error before running patient specimens

Trend: 7t Rule Violation

• Systematic error detected with 7t: trend

• 7 results falling in a pattern, up or down

Westgard Rules

Levey Jennings Charts: Evaluation of Results

• Single rule QC – Procedure uses a single criterion or single set of control limits,

such as a Levey-Jennings chart with control limits set as either the mean plus or minus 2 standard deviations (2s) or the mean plus or minus 3s. There is more than 1 single rule but each rule is assessed individually.

• Multi-rule QC– Uses a combination of decision criteria, or control rules, to

decide if an analytical run is in-control or out-of-control – Uses 5 different control rules to judge acceptability of an

analytical run

Westgard Rules:Single or Multi Rule QC

• Many laboratories follow the Westgard Single-Rule policy.

• The rules are assessed individually to check for random and systematic errors.

Westgard Rules

• These rules work best together in a multi-rule system with automated analyzers

• Rules evaluated in a certain order• Starts with the warning rule (12s ) that triggers the

need to look for other violations:

Westgard Multirule System

• This decision tree helps to decide whether to reject or accept a run. If you answer yes, it means that rule has been violated.

Westgard Multirules

• Describe a control in terms of – Assayed (mean and s provided)– Unassayed (mean and s not provided)– Use (validation of run not calibration)

• Calculate 95% limits for quality control samples

• Detect errors in quality control using 95% and Westgard rules

Key Points

• Levey-Jennings chart include values and markings on the x (days or time) and y axis (concentration mean and s)

Key Points

• Westgard rules increase ability to detect systemic errors.

Key Points

• Westgard Rules increase ability to detect random errors.

Key Points

• Westgard Rule, 1 2s is provided as a Warning

Key Points

• Shewhart WA. Economic Control of Quality of Manufactured Product. New York; D. Van Hostrand Company, Inc., 1931.

• Levey S, Jennings ER. The use of control charts in the clinical laboratory. Am J Clin Pathol 1950;20:1059-66.

• Henry RJ, Segalove M. The running of standards in clinical chemistry and the use of the control chart. J Clin Pathol 1952;27:493-501.

• Westgard JO, Groth T, Aronsson T, Falk H, deVerdier C-H. Performance characteristics of rules for internal quality control: probabilities for false rejection and error detection. Clin Chem 1977;23:1857-67.

References

• Westgard JO, Barry PL, Hunt MR, Groth T. A multi-rule Shewhart chart for quality control in clinical chemistry. Clin Chem 1981;27:493-501.

• Westgard JO, Barry PL. Cost-Effective Quality Control: Managing the Quality and Productivity of Analytical Processes. Washington, DC:AACC Press

• www.westgard.com

References