the role of proficiency tests in the estimation of...

The Role of Proficiency Tests in the Estimation of Measurement Uncertainty of

PCDD/PCDF and PCB Determination by Isotope Dilution Methods

e-mail: [email protected]

Stefano Raccanelli, Environmental Ethical Chemist, Italy

Measuring uncertainty

For the official food and feed control for levels of PCDD/Fs and PCBs non-compliance is defined as exceedance of legal limits beyond reasonable doubt taking into account the measurement uncertainty.

Courtesy of G. Scortichini

As a consequence when checking for non-compliance in official control, the same results of different laboratories can cause different assessment of results due to application of different measurement uncertainties.



(2) Top-down approach Global approach, which combines precision (within-laboratory reproducibility), trueness (bias) and proficiency tests (PTs) studies to calculate measurement uncertainty.

(1) Bottom-up approach Metrological approach based on the identification of the sources of uncertainty and the calculation of their standard uncertainties, then adequately combined. Suggested by ISO and adapted by EURACHEM to the analytical chemistry.


(0) Empirical general approach Based on 10000 measurements, the Horwitz curve. Mentioned also by ISO.

Empirical equation that relate the concentration of chemical to percentual variation coefficient (uncertainty)

For concentrations below 120 ppb Thompson proposed

(0) Horwitz relation

Type A evaluation (of uncertainty) Evaluation of uncertainty by the statistical analysis of series of observations.

Type B evaluation (of uncertainty): Evaluation of uncertainty from probability density functions based on experience or other information.

(1) Bottom-up approach

Uncertainty sources identified

• repeatability/reproducibility (type A) • calibration curve (type A) • calibration curve drift (type B) • volume (type B) • standard purity (type B) • sample weighing (type B)

Quantifying uncertainty components (Type A) EXAMPLE: PCB-126 in beef fat

Repeatability uncertainty

Number of tests = 18 Mean recovery = 112.3 % Standard deviation = 7.0% Relative standard deviation = 6.2%


ni

s ) x ( u i r =

Relative uncertainty = 0.0147

Intercept = -0.0396 Slope = 1.0364 R = 1.0000 Residual standard deviation = 0.2611

Calibration curve uncertainty

Injected amount (pg) Signal ratio

0 0.00

0.5 0.49

2 2.10

10 9.93

40 41.74

200 207.19

Quantifying uncertainty components (Type A) EXAMPLE: PCB-126 in beef fat


[ ]

21

2

/ −

−=∑=

n

yys

n

iii

xy


Maximum acceptable variation of RRFi from in-house method:

± 20%

Calibration curve drift uncertainty

Maximum acceptable variation of RRFi from EPA method 1668B 2008 ± 30%

Quantifying uncertainty components (Type B) EXAMPLE: PCB-126 in beef fat


( )3

id

xxu ∆= Relative uncertainty = 0.0577

Labelled compound solution (dilution 1): Relative uncertainty = 0.0128

Labelled compound solution (dilution 2): Relative uncertainty = 0.0173

Labelled compound solution (spiking): Relative uncertainty = 0.0173

Relative uncertainty (sum) = 0.0277

Spiking volume uncertainty



( )6

iv

xxu ∆=

12

Standard purity uncertainty Standard solution tolerance taken from analytical certificate: ± 5%



( )3

ip

xxu ∆=


Sample weighing uncertainty Maximum tolerance taken from calibration certificate of analytical balance: ± 0.3 mg



( )3

iw

xxu ∆=


222222)()()()()()()()(

+

+

+

+

+

==

•

wwu

ppu

vvu

ddu

ccu

rru

yyuyu c

c

For models involving only a product or a quotient, the combined standard uncertainty is given by:

Where etc., are the uncertainties of the parameters

expressed as relative standard deviations. rru )(

Calculating the combined expanded uncertainty

The expanded uncertainty is usually calculated using a coverage factor k = 2. ) y ( u k ) y ( U c × =

Uncertainty source Relative uncertainty

Repeatability 0.062*

Calibration curve 0.046*

Calibration curve drift 0.058

Volume 0.028

Standard purity 0.029

Sample weighing 0.0005

Relative combined uncertainty

0.105

Relative expanded uncertainty (k=2)

0.209

* In routine analysis u(xr) and u(xc) are related to the number of replicates

Quantifying TOTAL uncertainty EXAMPLE: PCB-126 in beef fat


* In routine analysis u(xr) and u(xc) are related to the number of replicates

Combined and expanded uncertainty

PCB-126 in beef fat

r

prsampler N

Nxuxu ×= )()(

r

pcsamplec N

Nxuxu ×= )()(

Np: number of replicates in

precision study

Nr: number of replicates in

routine analysis

Repeatability

Calibration curve


Four approaches can potentially be used

How uncertainty estimated for each individual congener could be propagated to TEQ?

1) The square root of the sum of squares (RSS):

2) The SUM:

3) The average of (uci*TEFi)i=congener

4) The median of (uci*TEFi)i=congener

∑=

=29

1

2)*()(i

ciic uTEFTEQu

∑=

=29

1)*()(

iciic uTEFTEQu

Only one is statistically correct: in fact the square root of the sum of squares (RSS) provides the most realistic estimates

Is this a simplified

approach?

Black-box approach

(2) Top-down approach

Uncertainty results from combination of two terms: a. Best estimate of global precision (s)

b. Best available estimate of global bias (E) and its uncertainty (uE)

𝑢𝑢𝑐𝑐 = 𝑓𝑓( 𝑠𝑠 ⏟𝑎𝑎

; 𝐸𝐸 ; 𝑢𝑢(𝐸𝐸)��𝑏𝑏

)

(2) Top-down approach

Global precision is evaluated as the standard deviation of results of repeated measures on the same sample, in a period long enough to include all possible sources than can influence the results: - Different lot of reagents/solvents; - Different analyst; - Different environmental conditions, - Different instrumental calibrations; - Different instrument or after service…..

a) Best estimate of global precision (s)

a) Best estimate of global precision (s)

Same reagent lot

same series

Same instrument

Same Laboratory

Ripetibility (r)

Ripetibility intra-lab. (Rw)

Riproducibility (R)

Same reagent lot

Same instrument

Same Laboratory

Same Laboratory

Different series

Different reagent lot

Same Laboratory

Different series

Different reagent lot Differnt

instruments

Different series

Differnt instruments

Different reagent lot

Different laboratory

Same instrument

Different series

Courtesy of F. Pecoraro

𝑢𝑢𝑐𝑐 = 𝑓𝑓( 𝑠𝑠 ; 𝐸𝐸 ; 𝑢𝑢(𝐸𝐸)��𝑏𝑏𝑏𝑏𝑎𝑎𝑠𝑠

𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐

)

b) Best estimate of global bias and uncertainty

The best estimate of global bias can be obtained in two ways: I) From CRM analysed several times in different days;

II) From participation to Proficiency Tests (PT); The estimates can be provided in similar way, I will present the PT method (Northtest approach)

𝑢𝑢𝑐𝑐 = 𝑓𝑓( 𝑠𝑠𝑅𝑅𝑅𝑅 ; 𝐸𝐸 ; 𝑢𝑢(𝐸𝐸) )

b) Best estimate from participation to PT: of global bias and its uncertainty

Participating to several PTs (minimum 6)

Participating to one PT


𝑢𝑢𝑐𝑐 = � 𝑠𝑠𝑅𝑅𝑅𝑅 2� + 𝐸𝐸𝑅𝑅𝑅𝑅𝑅𝑅2 + 𝑢𝑢(𝐸𝐸𝑅𝑅𝑅𝑅𝑅𝑅 )2��

𝑈𝑈 = 2 ∙ 𝑢𝑢𝑐𝑐

Global bias Global precision

Uncertainty of global bias

Calculating the combined expanded uncertainty


(2) Top-down approach Pros: dynamic (updated uncertainty at each PT participation), moderate costs for synergies with other Lab needs (accreditation). Cons: need for participation to some PTs, possible errors in estimation of assigned value and overestimation of assigned uncertainty

(1) Bottom-up approach Pros: rigorous if correctly applied Cons: static (generally done only once), expensive, possible underestimation of error source (e.g. ripetibility)

Pros&Cons of approaches for Measuring uncertainty

(0) Empirical general approach Pros: immediate, no cost Cons: not verified, static, low reliability, possible overestimation

On-going top-down approach

validation data (precision, accuracy)

Quality control (QC)

results from PTs (more than 1 per year)

daily LOQ

white matrix/solvent effect

variation of response factors

Made it a dynamic way to run your Laboratory!

Some features of InterCinD

- the samples are NATURALLY CONTAMINATED MATRIXES (es. soil, sediments, ash, food, wastes…);

- Matrixes are homogeneized and treated in agreement with the international guidelines for reference materials (ISO Guide 34:2009);

- analytes: PCDD/F, PCB-DL, PCB-ICES6, PAH, PBDE, heavy metals;

- Data are required in 3 replicates (evaluation of accuracy & precision).

The interlaboratory circuit InterCinD is organized in agreement with international guidelines and is accreditated ISO 17043

the InterCIND

+ + =

Assessing Performance

j

jjkijki s

xxz

−= ,,

,,

Accuracy: using z-scores

Precision: using “relative range” (r)

ji

jijijijijijiji x

xxxxxxr

,

,3,,2,,1,,3,,2,,1,,

),,min(),,max( −=

Overall objective Is to provide basis for the laboratories to assess their performances evaluated through the z-scores (accuracy) and dispersion of measures (precision).

the participants 2013 (26 countries) 2014 (23 countries)

Num

ber o

f lab

s

the role of proficiency tests in the estimation of...

Documents