aacc talk on calibration verification 2009
Post on 07-May-2015
2.728 Views
Preview:
TRANSCRIPT
AACC Short Course 73214How to Fulfill the CLIA Requirements
for Calibration, Calibration Verification, and Reportable Range
Anthony Killeen, MD, PhDWilliam Castellani, MD
(in conjunction with CAP)
This PowerPoint Slide Set
• http://tinyurl.com/kmq8vu
Outline
• Calibration Verification• Linearity• CVL Program at the CAP• Examples and Troubleshooting• Cases• Discussion of Selected CAP Checklist
Questions
Calibration Verification
• Periodic verification that the calibration of the analytical system remains valid
• Typically assessed by comparing test results with appropriate target values
• If the calibration changes, then patient test result values will also change
Linearity
• From CLSI document EP6-A (2003)– A quantitative analytical method is linear when
there exists a mathematically verified straight-line relationship between the observed values and the true concentrations or activities of the analyte.
– …if the true amount of an analyte in a sample were to double, clinicians expect that would be reflected in a doubling of the measured value.
– (Notice that there’s no reference to accuracy)
Calibration Verification vs. Linearity
• Calibration verification is the process of verifying agreement between calibrators (or other materials of known analyte concentrations) and measured values
• Calibrators should be traceable to a reference method to ensure accuracy
• Linearity evaluation does not require knowledge of the “true” analyte concentration
• “Linearity” does not appear in CLIA
Analytical Measurement Range vs. Linearity
• AMR is the range of analyte values that a method can directly measure without any dilution, concentration, or other pretreatment of the specimen that’s not part of the usual assay process
• A method should be linear across the AMR
Linearity as Defined in EP6-A vs. Instrument Response
• Linearity typically refers to the final analytical result, not the raw instrument output. A plot of analyte concentration vs. the raw instrument output may not be linear (e.g., competitive immunoassay)
• Linearity as used here is a straight-line relationship between “true” analyte concentrations and measured concentrations
Formal Calibration Verification/Linearity Program
• College of American Pathologists– CVL Surveys
• Other programs e.g., Maine Standards
Advantages to Participating in a CVL Program
• Supplements proficiency testing (PT)• Linearity testing often has smaller absolute
limits for error based on medically or analytically relevant criteria than does PT
• Challenges the full calibration range• Can detect problems earlier than QC or PT• Calibration verification is required by CLIA,
CVL program provides the materials and data analysis
Linearity Study Materials
• Generally 5 samples that span the AMR• Preparation– Individually spiked samples– Serial dilutions of a high concentration sample– Admixtures of low and high concentrations (CAP)– At least 2 replicates at each level to assess
random error
CAP Linearity Assessment
• Uses a polynomial method described in EP6-A• For statistical approach see – EP6-A CLSI/NCCLS Document, Wayne, PA (2003)– Kroll MH et al., Arch Pathol Lab Med 124:1331-8, 2000
Calibration Verification Evaluation
• Results reported as “verified” or “different”• If “verified”, then differences between your
means and the peer-group means are within the allowable error limits in the range specified
• If “different”, then at least one of your values exceeds the peer-group allowable error limits
Tabular Data
Graphical Data
Peer GroupSummary
Result
Calibration Verification Report Example
Difference = 100 * (Your Mean – Peer Mean) / Peer Mean
Allowable Error = larger of: One half of goal for total error OR minimum detectable difference
100% 100%
Note that CLIA Allowable Error for PT is ± 10% for albumin
Calibration not verified
Need to re-calibrate
High biases
Proficiency Testing Data from Same Lab
CAP Linearity Evaluations
• Standard• Extended Range• Dilution• Results– Linear– Nonlinear– Imprecise
Result
Tabular Data
Graphical Data
Extended Range Linearity Evaluation
• Extended range evaluation is used if– Survey includes an additional high specimen that
is spiked and not part of the admixture set– If there is a large gap in the relative
concentrations of the admixture ratios such that a specimen(s) may have a large influence on the fitted regression line, the specimen(s) will be considered to be extended range
Scoring Extended Range Specimens
• Specimens evaluated for linearity excluding the extended range specimen
• If a range that includes the highest non-extended range specimen is linear, then the extended range specimen is evaluated
• If the mean of the extended range specimen value falls on the extrapolated line (within allowable error) then the linear range will include the extended range specimen
Scoring Extended Range Specimens (2)
• The extended range specimen is not used to calculate the best-fit line
Dilution Validation Linearity Evaluation
• Performed if– Lab reports that it diluted at least one specimen– Enough undiluted specimens (normally 5) are
reported for evaluation using the standard linearity evaluation
EXAMPLES AND TROUBLESHOOTING
Linearity demonstrated for LN-01 to LN-05 only
Linearity Example 1: Standard Evaluation, specimens excluded from the linear range
Linearity Example 1: Standard Evaluation, specimens excluded from the linear range
Example 1: Troubleshooting
• Does the linear range cover the AMR? If the high specimen is above the AMR, did you dilute? Was the dilution protocol followed?
• If the linear range does not cover the AMR, then there may be problems with reagents, specimen handling, or the test system
• Check QC, PT, calibration data• Address identified problems and re-run linearity• Consider adjusting AMR to cover the linear range
Example 2: Extended Range Evaluation, extended range sample not linear
Example 2: Extended Range Evaluation, extended range sample not linear
Example 2: Troubleshooting
• Confirm that the linear range covers the AMR• Check the dilution protocol. Did you dilute?
Dilution can introduce variability• If the linear range does not cover the AMR,
then there may be problems with reagents, specimen handling, or the test system
Example 3: Standard Evaluation, nonlinear data
(Best appreciated with graphical data)
Example 3: Standard Evaluation, nonlinear data
Example 3: Troubleshooting
• Is the peer group generally linear?• If the peer group is generally linear then there
may be problems with specimen handling or the test system
• Review QC, calibration, PT data• Eliminate specimen or reagent handling errors• Diagnose fix any identified test system failures• Re-run linearity study
Example 4: Dilution Validation Evaluation, nonlinear trend in non-diluted samples
Example 4: Troubleshooting• (Because non-diluted samples are nonlinear, no
evaluation of the diluted sample is performed• Is the peer group generally linear?• If the peer group is generally linear then there
may be problems with specimen handling or the test system
• Review QC, calibration, PT data• Eliminate specimen or reagent handling errors• Diagnose fix any identified test system failures• Re-run linearity study
Example 4: Standard Evaluation, large replicate imprecision
Example 5: Standard Evaluation, large replicate imprecision
Example 5: Troubleshooting
• Pattern suggests pipetting problems which should be carefully investigated
• Fix any identified test system failures• Re-run linearity study
Example 6: Standard Evaluation, only difference plot shows replicate imprecision (similar problem to Example 5)
Looks OK?
Example 6: Standard Evaluation, only difference plot shows replicate imprecision (similar problem to Example 5)
Example 6: Troubleshooting
• Pattern suggests pipetting problems which should be carefully investigated
• Fix any identified test system failures• Re-run linearity study
Example 7: Standard Evaluation, Imprecision from poor fit
Example 7: Standard Evaluation, Imprecision from poor fit
Here, the term “imprecision” refers to the inability to find a subset of data displayinglinearity with adequate precision. Points zigzag around the best-fit line.
Example 7:Troubleshooting
• Unusual pattern, may be difficult to diagnose• Is the peer group linear? If not, call Customer
Service• Highest value may be diluted and a dilution
error is masking loss of recovery at the high end of the AMR
• Specimen handling problems may cause this
Example 8: Extended Range Evaluation, Imprecise in non-extended range, extended range sample(s) not evaluated
Example 8: Extended Range Evaluation, Imprecise in non-extended range, extended range sample(s) not evaluated
Large gap indicates extended range
Imprecise. E.R. specimennot evaluated
LN-24
• Creatinine linearity survey• Uses human serum spiked with pure
crystalline creatinine• Accuracy assigned values (NIST) allow for
accuracy based grading in addition to calibration verification and linearity checks
Case 1. Calibration Verification (AST)
Linearity study show the data were linear
Case 2. Calibration Verification (calcium)
Case 3. Calibration Verification
Case 4. Linearity (testosterone)
Executive Summary Page from CVL Survey
SOME CAP CHECKLIST QUESTIONS
CHM. 13000
• Are calibration procedures for each method adequate, and are the calibration results documented?
CHM.13100
• Are high quality materials with method- and matrix-appropriate target values used for calibration and calibration verification whenever possible?
CHM.13125
• Are all calibration materials used for non-FDA cleared assays documented as to quality?
• NOTE: Standards used to prepare calibrators for non-FDA-cleared assays require certificates of purity from the vendor, or a check on purity as part of the initial assay validation process. The laboratory should document the accuracy of a new lot of calibrators by checking the new lot against the current lot.
CHM.13400• Are criteria established for frequency of recalibration or
calibration verification, and the acceptability of results?• • NOTE: Criteria typically include: • At changes of reagent lots for chemically or physically active or
critical components, unless the laboratory can demonstrate that the use of different lots does not affect the accuracy of patient/client test results and the range used to report patient/client test data
• QC fails to meet established criteria• After major maintenance or service• When recommended by the manufacturer• At least every 6 months
CHM.13500
• Is the method system recalibrated when calibration verification fails to meet the established criteria of the laboratory?
CHM.13600
• Is validation of the analytical measurement range (AMR) performed with matrix-appropriate materials which include the low, mid and high range of the AMR, and is the process documented?
CHM.13700
• Are criteria established for validating the analytical measurement range, and is compliance documented?
GEN.42085• Is the reportable range verified/established for each
analytic procedure before implementation?• NOTE: The reportable range includes all results that may be
reliably reported, and embraces two types of ranges: • The ANALYTICAL MEASUREMENT RANGE (AMR) is the
range of analyte values that a method can directly measure on the specimen without any dilution, concentration, or other pretreatment not part of the usual assay process
• The CLINICALLY REPORTABLE RANGE (CRR) is the range of analyte values that a method can measure, allowing for specimen dilution, concentration, or other pretreatment used to extend the direct analytical measurement range
Summary of Topics Covered
• Calibration Verification• Linearity• CVL Program at the CAP• Examples and Troubleshooting• Cases• Discussion of Selected CAP Checklist
Questions
top related