Download - Resmeth(Bias Ppt) Arianne Letada
-
8/12/2019 Resmeth(Bias Ppt) Arianne Letada
1/35
Use of recovery and bias information inanalytical chemistry and estimation of itsuncertainty contribution
Thomas P.J.Linsinger
Trends in Analytical Chemistry
Arianne G. Letada
MS-Chemistry
-
8/12/2019 Resmeth(Bias Ppt) Arianne Letada
2/35
BACKGROUND-LAB METHOD FLOW
Method
Validation
Method
Transfer
Method
Development
Approved
-
8/12/2019 Resmeth(Bias Ppt) Arianne Letada
3/35
Definitions
Validation is the processof demonstrating orconfirming the performance characteristicsof amethod of analysis.
A process of evaluating method performanceand demonstrating that it meets a particularrequirement.
Validation applies to a specific operator,
laboratory, and equipmentutilizing the methodover a reasonable concentration range andperiod of time.
-
8/12/2019 Resmeth(Bias Ppt) Arianne Letada
4/35
Why Method Validation?
To minimize analytical and instrumental errors
To give reliable and reproducible results in
accordance with the given specifications of the
test method
To ensure the quality of the test results
To meet accreditation requirement
Objective evidence for defense againstchallenges
To be assured of the correctness of results
-
8/12/2019 Resmeth(Bias Ppt) Arianne Letada
5/35
Method bias is generally recognized as intrinsic
part of method validation
There are different points of view on how to
estimate potential bias
whether to correct for it how to accommodate it
its uncertainty in the uncertainty budget for a
particular measurement.
-
8/12/2019 Resmeth(Bias Ppt) Arianne Letada
6/35
Introduction
It is a truism in the analytical community that
not every method that should work in principlewill also work in practice and deliver accurate
results.
Analytical methods therefore have to be
validated (either in-house or by laboratoryintercomparison) to demonstrate the reliability
of the results obtained by these methods.
-
8/12/2019 Resmeth(Bias Ppt) Arianne Letada
7/35
Parameters for Method Validation
Accuracy
Precision (repeatability, reproducibility) Specificity
Limit of detection
Limit of quantitation
Linearity and range
-
8/12/2019 Resmeth(Bias Ppt) Arianne Letada
8/35
Definition
IUPAC
Trueness: Closeness of the
agreement between the
average value obtained froma large series of test results
and an accepted reference
value
Bias: The differencebetween the limiting mean
and the true value
ISO Guide 99
Trueness: Closeness of
agreement between the
average of an infinite number
of replicate measured
quantity values and a
reference quantity value.
Bias: Systematic
measurement error or its
estimate, with respect to a
reference quantity value.
-
8/12/2019 Resmeth(Bias Ppt) Arianne Letada
9/35
This review will focus on the estimation of bias
and its uncertainty, essential for assessment ofthe significance of an eventual bias, and the
use of uncertainty in the estimation of
measurement uncertainties.
Particular emphasis is given not only to peer-reviewed literature but also to guidelines
issued by international organizations and
accreditation bodies.
-
8/12/2019 Resmeth(Bias Ppt) Arianne Letada
10/35
Estimation of bias
b = xmeasxref eq. 1
This bias may be a function of the analytecontent.
b = xmeas / xref eq.2
The methods of bias estimation differ in thesource of the reference value.
-
8/12/2019 Resmeth(Bias Ppt) Arianne Letada
11/35
Requirements in selecting the
materials for bias estimation
Materials shall resemble real-life samples as
closely as possible Materials with different analyte levels shall be
available
The reference values shall be reliable and
their uncertainties as small as possible.
-
8/12/2019 Resmeth(Bias Ppt) Arianne Letada
12/35
Certified reference materials (CRMs): well-designedintercomparisons or measurements with another method ofdemonstrated accuracy
Advantage
high reliability of the
reference values with low
uncertainties.
Therefore recommended
for bias estimation (e.g.,IUPAC, ISO 17025 and
Eurachem)
Disadvantages
only in few cases are
materials at different
concentration levelsavailable for a certain
matrix.
due to the longer storage
of CRMs, trade-offsbetween stability and
realistic presentation
must frequently be made
-
8/12/2019 Resmeth(Bias Ppt) Arianne Letada
13/35
Harmonised guidelines for the use of recovery informationin analytical measurement: use of surrogates (spikes) asoptions for estimating recoveries
Advantages
materials with different
analyte concentrations
are readily available the uncertainties in the
analyte contents are
generally low
Disadvatages
spikes (isotopically
labeled or not) generally
do not reach anequilibrium in the spiked
samples and hence result
in heterogeneous
samples
The recovery of the surrogate is
therefore likely to be greater than
that of the native analyte, so a
bias in an estimated recovery may
arise
-
8/12/2019 Resmeth(Bias Ppt) Arianne Letada
14/35
Meija and Mester Isotope-dilution mass-
spectrometry: their review showed that equalbehavior cannot be expected for trace metalsor organometallic substances.
Analytical Methods Committee: which
recognized that all methods for estimatingrecovery are unsatisfactory in some
circumstances
-
8/12/2019 Resmeth(Bias Ppt) Arianne Letada
15/35
ISO TS 21748: Guide to the use of repeatability,reproducibility and trueness estimates in measurementuncertainty estimation
Advantages
only possible for a single
laboratory to demonstrate
bias control with respectto other laboratories, as it
usually has to assume
that the results of the
other laboratories are
correct.
Disadvantages
the lower reliability of the
assigned values
-
8/12/2019 Resmeth(Bias Ppt) Arianne Letada
16/35
Testing for significance
If the bias obtained is smaller than the expanded uncertainty of
this bias, there is in reality no evidence of a bias. However, even if
found statistically significant, a bias may still be deemed
practically insignificant if it is small compared to the measurement
uncertainty of the measurement in question.
The Eurachem Guide on measurement uncertainty defines this
small as not larger than one-third of the largest uncertaintycomponent.
This general recommendation has been confirmed by Monte Carlosimulations showing that uncertainties are significantly
underestimated if a bias larger than one third of the other
uncertainty contributions remains uncorrected.
-
8/12/2019 Resmeth(Bias Ppt) Arianne Letada
17/35
Uncertainty of bias
ucorr = u2meas+ u2b eq.3ub= u2meas,b+ u2ref eq.4
These two equations illustrate the basic
principle of estimating the uncertainty of biasnamely that it can never be smaller than the
uncertainty of the reference value.
It also breaks down the estimation of bias
uncertainty into two sub-problems, namely
determination of the uncertainty of the
reference value and estimation of the
measurement variation.
-
8/12/2019 Resmeth(Bias Ppt) Arianne Letada
18/35
Disadvantages
First, it treats uncertainties of recovery as
combined uncertainty, thus forgoing anydeeper insight into source of the potential bias;
and,
second, it implicitly assumes that the results
obtained from the samples during the study arevalid for future, yet unknown, samples.
-
8/12/2019 Resmeth(Bias Ppt) Arianne Letada
19/35
Bettencourt et al., showed that deeper insight
of the source of bias can also be gained fromvalidation data. They applied an intricate analytical design that aimed at
separating uncertainties from re-extraction of the extracted
sample, sample-processing recovery and extraction
recovery. In this way, they managed to identify the main sources of
bias, which subsequently could be used for a targeted
method optimization. Their approach is close to the
bottom-up approach of uncertainty estimation
Maroto et al., who constructed Youden plots forsamples of various weights used to obtain an estimate
of the constant bias, which later on were combined
with the uncertainty from the proportional bias.
-
8/12/2019 Resmeth(Bias Ppt) Arianne Letada
20/35
The second problem is not specific to biasestimation but extends to all aspects of method
validations; samples always have to be chosen
to reflect samples encountered in daily use.
The issue of bias is in this respect no differentfrom determination of repeatability,
intermediate precision or limit of quantification.
-
8/12/2019 Resmeth(Bias Ppt) Arianne Letada
21/35
-
8/12/2019 Resmeth(Bias Ppt) Arianne Letada
22/35
Barwick and Ellison discussed how to
accommodate variation betweenconcentrations, matrices and spike vs. incurred
samples in the general uncertainty model. They gave guidance on how to include additional
uncertainty sources corresponding to differences betweenrecovery of the spiked and the incurred analyte and
changes of recovery with concentration and matrix.
This is done by estimating uncertainty contributions for all
of these variations and including them in the overall
uncertainty model
Eurachem Guide on measurement uncertainty
recommended inclusion of these uncertainties
-
8/12/2019 Resmeth(Bias Ppt) Arianne Letada
23/35
Bot et al. used a nested design to evaluate meanuncertainty of recovery, variation of recovery due to
difference in matrices and variation of recoverydepending on the content of endocrine disruptors in a
sample (sediments and waters). Their uncertainties are
estimated from ANOVA. They generally found that
variation of recovery depending on the analyte contentwas negligible.
The problem of matrix mismatch can sometimes be
overcome by using materials from proficiency tests
(PTs) for assessing method trueness. PT samples are frequently closer to real-life samples
than spikes or CRMs, and the problem of matrix
mismatch is hence less prominent.
-
8/12/2019 Resmeth(Bias Ppt) Arianne Letada
24/35
Desenfant and Priel explained the use of PT results
for the estimation of uncertainty of bias. The
uncertainty of bias is the standard deviation of thebiases in the various rounds of the PT scheme and
must be included in the overall uncertainty.
If the average bias is not zero, an additional
uncertainty contribution relating to this bias must beadded. However, Desenfant and Priel discouraged
this latter approach. While the use of PT data certainly
has its advantages, one main disadvantage is that
assigned values are often less reliable.
-
8/12/2019 Resmeth(Bias Ppt) Arianne Letada
25/35
Correction versus non-correction of
bias
There is evidence that recovery correction
leads in most cases to better comparable
results, but legislation is sometimes unclear or
even explicitly states that results should not be
corrected for recoveries.
Recoveries in organic analysis are oftenregarded as acceptable if they are 70110%
-
8/12/2019 Resmeth(Bias Ppt) Arianne Letada
26/35
It is understood that any correction of bias
should be a method of last resort only. Removal of a bias should always be preferred
over accommodation of a bias.
Magnusson and Ellison gave four criteria that
need to be fulfilled to warrant bias correction:
(1) evidence of a significant effect;
(2) a causal relationship;
(3) the estimation of the bias must be sufficiently
accurate; and,
(4) correction must reduce the uncertainty.
-
8/12/2019 Resmeth(Bias Ppt) Arianne Letada
27/35
Guide to the Expression of Uncertainty in
Measurement(GUM) explicitly states thatknown biases must be corrected for, and only
in exceptional circumstances is it acceptable to
increase the uncertainty to allow for the bias.
Eurachems translation of the GUM foranalytical chemistry also states that known
biases must be corrected for.
IUPACalso follows this policy. Its guide on theuse of recovery information distinguishes
between rational and empirical methods
-
8/12/2019 Resmeth(Bias Ppt) Arianne Letada
28/35
TheAsia Pacific Laboratory Accreditation Cooperation (APLAC)
states that results must be corrected for recovery. Results not
corrected for recovery are only traceable to the specific workinginstruction.
TheAMCstates even more drastically that results obtainedwithout correction for recovery are necessarily empirical
The same conclusions are drawn by theAustralian National
Pathology Accreditation Advisory Council (NPAAC). Eurolab,Nordtest, the Nordic Committee on Food Analysis (NMKL),
European Accreditation (EA) and the American Association for
Laboratory Accreditation (A2LA) agree that bias must be
corrected for.
This unanimous agreement between the institutions in Europe,Asia and the Americas that results must be corrected for
recoveries is encouraging with respect to the ultimate goal of
achieving comparability of measurement results world-wide.
-
8/12/2019 Resmeth(Bias Ppt) Arianne Letada
29/35
Use of the uncertainty of bias
According to the law of error propagation,
uncertainties in the basic analytical procedure
and the uncertainty of bias/recovery must be
combined to obtain the full measurement
uncertainty.
There is agreement among international bodiesthat the uncertainty of the bias correction is a
part of the measurement uncertainty.
-
8/12/2019 Resmeth(Bias Ppt) Arianne Letada
30/35
The EA Guidelines for uncertainty estimation
state explicitly that in general, the uncertaintyassociated with the determination of the bias isan important component of overall uncertainty.
Lyn et al. showed that ignoring bias, in this
case from sample preparation, underestimateduncertainty.
Feinberg and Laurentiefound that inclusion of
uncertainties of the recovery factors increaseduncertainties.
-
8/12/2019 Resmeth(Bias Ppt) Arianne Letada
31/35
Guidance exists on the issue of how uncorrected bias
should be included in an uncertainty statement.
Magnusson and Ellison devoted an entire review articleto the treatment of uncorrected bias. Also, Phillips and
Eberhardt, ODonnell and Hibbert and Synek
investigated several options for allowing for
uncorrected bias. A recovery of 100% does not eliminate the recovery
factor from the basic measurement equation. The
problem basically is that, if the variation in results of a
method is high enough or if the uncertainties of thecertified values of the CRMs used are large enough,
any bias will remain undetected.
-
8/12/2019 Resmeth(Bias Ppt) Arianne Letada
32/35
Maroto et al. found that ignoring non-significant
bias results in underestimations of theuncertainty, if the uncertainty of the bias is
large and if the bias contributes significantly to
the overall uncertainty.
Ellison and Barwick also explicitly stated that arecovery of 100% has an uncertainty, but they
also warned of double counting so they
advised planning validation experiments
carefully.
-
8/12/2019 Resmeth(Bias Ppt) Arianne Letada
33/35
IUPAC, Eurachem and Nordtest follow this line by
stating explicitly that an uncertainty of recovery needs
to be included in the overall uncertainty even for 100%
recovery or a bias of zero.
The NPAACguide qualifies this point of view by
recommending inclusion of the uncertainty of bias only
when significant.
TheISO explicitly assumes that the uncertainty of the
bias check is negligible compared to the other
uncertainty sources, so does not have to be included. If
this is not the case, an additional uncertaintycontribution must be added.
-
8/12/2019 Resmeth(Bias Ppt) Arianne Letada
34/35
Hasselbarththe uncertainty of bias is not a part of theoverall measurement uncertainty. The condition is that
a full uncertainty budget is available and bias is testedas final step in an exhaustive evaluation of
measurement uncertainty.
In this case, all uncertainty sources are already
included and no additional uncertainty needs to beadded when no significant bias is found.
He agrees that, whenever bias is tested in a within-
laboratory or a between-laboratory validation
procedure, the model still includes the recovery factorand its uncertainty must be included even if recovery is
found to be 100%.
-
8/12/2019 Resmeth(Bias Ppt) Arianne Letada
35/35
Conclusions and suggested procedure
1. Materials that are sufficiently close to real-life samples with
sufficiently accurately assigned reference values shall be selectedfor the assessment of method bias.
2. Method bias shall be estimated.
3. The uncertainty of the bias shall be estimated. This estimation
includes in all cases the uncertainty of the assigned value and a
contribution of the variation of the measurement results used forthe bias estimation.
4. The significance of the bias is tested by comparing the bias
determined with its expanded uncertainty estimated. If the bias is
found to be significant and no further method optimizations are
performed, results, in general shall be corrected for this bias.5. The uncertainty of the bias is included in the uncertainty budget
for measurements from this method, regardless of whether or not
the bias was found to be significant. If no bias correction is applied
for insignificant bias, an additional uncertainty contribution
accounting for this uncorrected bias has to be added