MATHMET 2010International Workshop
Bayesian Approach to assign Consensus Values in PT Comparisons
Séverine DemeyerNicolas Fischer
Mathematics and Statistics Division (LNE)
Berlin, June 21 st 2010
2June 21 st 2010 MATHMET 2010, PTB
Outline
� Framework of Proficiency Testing
� PT data
� Standardized approach to assign consensus values: NF ISO 13 528
� The proposed approach: modelling bias
� Methodology
� When to introduce latent predictors of bias?
� Statistical model
� Estimating the model
� Bayesian computation of posterior distributions
� Getting the consensus value, its associated uncertainty and bias
� Conclusion
� Perspectives
3June 21 st 2010 MATHMET 2010, PTB
Framework of the project
� Era-net+ European project entitled « Traceable
measurements for biospecies and ion activity in clinical
chemistry » (JRP 10, TRACEBIOACTIVITY)
� WP 5: PTB, SP, LNE
� Delivery 2: Evaluating a consensus value in proficiency tests.
� Funded by the European Community’s Seventh Framework
Programme, ERA-NET Plus, under Grant Agreement No.
217257.
4June 21 st 2010 MATHMET 2010, PTB
Samples provided by BIPEA
N stable and homogene samples
BTEX, PCB, Triazines
5June 21 st 2010 MATHMET 2010, PTB
PT data
� PT provider: BIPEA (2nd provider in Europe)
� Measurands: concentrations of BTEX, Triazine and PCB in
water
� No associated uncertainties
� 31 participating laboratories
-3
-2
-1
0
1
2
3
4
5
6
1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31
ATRAZ37
DSAZ37
SIMAZ37
TBUTZ37
CYANA37
DIA37
CV=0.300.290.220.200.310.62
Example: results for 6 analytes from triazine family
Z scores
labs
analytes
6June 21 st 2010 MATHMET 2010, PTB
NF ISO 13 528
5.6 Consensus value from participants [see ISO/IEC Guide 43-1:1997, A.1.1 item e)]
5.6.1 General«With this approach, the assigned value X for the test material
used in a round of a proficiency testing scheme is the robust
average of the results reported by all the participants in the
round, calculated using Algorithm A in Annex C.
Other calculation methods may be used in place of
Algorithm A, provided that they have a sound statistical basis and
the report describes the method that is used. »
Statistical methods for use in proficiencytesting by interlaboratory comparisons
7June 21 st 2010 MATHMET 2010, PTB
NF ISO 13 528: Algorithm A
Algorithm A to compute robust means and standard deviations
* *
* * *
if
if
otherwise
i i
i i i
i
x x x
x x x x
x
δ δ
δ δ
− < −= + > +
*1,5sδ =
* 's median ix x=* *1,483 median of the is x x= × −
Initialisation
Iterate till convergence
Outputs:
*
1,25xs
up
= ×
*xConsensus value:
Associated uncertainty:
Bias:*
ix x−
8June 21 st 2010 MATHMET 2010, PTB
Examples
Crossed effect on bias?
Effect of method
on bias?
9June 21 st 2010 MATHMET 2010, PTB
Overview
Proficiency testing
statistical
model
Measurement data
(quality assessment)
Auxiliary information
(survey)
Bayesian estimation
consensus value,
associated uncertainty,
bias
10June 21 st 2010 MATHMET 2010, PTB
Proposed measurement model
jj jZX µ β τ= + +
Consensus value Measurement bias
of laboratory j
Predictors
Nature of Zj ?
Modelling
Bias
Results
11June 21 st 2010 MATHMET 2010, PTB
Construction of predictors
� Depends on the measurand
� Based on experts knowledge (survey,…)
� If a few number of variables can explain bias:
���� Zj are kept as observed variables
� If several variables can explain bias:
���� the observed variables are grouped
���� Zj are latent variables summarizing the observed
variables
���� Zj should capture structures in data
12June 21 st 2010 MATHMET 2010, PTB
Steps
� Collecting measurements
� Collecting additional information on laboratories (survey)
� Converting this information into variables
���� latent variables
� Constructing a statistical model
� Estimating the model
� Validating the model
13June 21 st 2010 MATHMET 2010, PTB
Building latent variables
� Idea: summarizing measurement process + background information on labs
� Blocks = latent (unobserved) concepts, variables
14June 21 st 2010 MATHMET 2010, PTB
Links between questions and blocks
� Latent concepts are measured on observed variables
(the questions)
� Example:
15June 21 st 2010 MATHMET 2010, PTB
Structure of the model
Structural equation modelling
Hierarchical modelling
16June 21 st 2010 MATHMET 2010, PTB
Structural equation modelling
1η
2η
1ξ 2ξ
3ξ4ξ
11111111θθθθ
12121212θθθθ
13131313θθθθ
15151515θθθθ
14141414θθθθη1111
y1111
2y
3y
4y
5y
1 12 2 12 13 14 1
2 23 2
2 3 4
3 44 2
j j j j
j
j j
j jj
ξ ξ ξη π η λ λ λ δη δξ ξλ λ
= + + + +
= + +
1 111 1
515 15
j j j
j j j
y
y
η ε
η
θ
θ ε
= +
= +K
11θ
15θ
(Simultaneous equations)
17June 21 st 2010 MATHMET 2010, PTB
Complete model
j
j
j j
j j
j j
j
Z
Z
H
X
Y
Z
µ β νθ ε
δ
= + +
= +
= Λ +SEM
Results
Consensus value
Auxiliary
data
Endogeneous
latent variables
Bias
18June 21 st 2010 MATHMET 2010, PTB
Model inference: Bayesian approach
� To take into account prior information on parameters
(correlations, variances)
� Estimation algorithm based on posterior conditional
distributions (MCMC)
� Iterating:
� Imputation of latent variables
� Posterior sampling of parameters
19June 21 st 2010 MATHMET 2010, PTB
Proposed Gibbs algorithm to estimate SEM
20June 21 st 2010 MATHMET 2010, PTB
Imputation of latent variables
Results:
Let
21June 21 st 2010 MATHMET 2010, PTB
Posterior conditional distributions
Let
22June 21 st 2010 MATHMET 2010, PTB
Posterior conditional distributionsConjugate models
Normal/Gamma( ), ~ ,k k k k kY Z Y N Z εθ ε θ= + Σ
( ), ~ ,k k k k k k k kH Z H N Z δδ= Λ + Λ Σ
23June 21 st 2010 MATHMET 2010, PTB
Estimation of the model: 3 phases
Phase 3
Consensus value Associated uncertainty Biais
Latent variables(continuous)
Phase 1
continuous
AuxiliaryData
Nominal / binary
Phase 2continuous
Estimating theparameters of the structural model
Estimating theparameters of the hierarchical model
Results
24June 21 st 2010 MATHMET 2010, PTB
Conclusion
� New approach to compute consensus values and their associated uncertainties
� Modelling bias
� Modelling structures in auxiliary data
� To propose different models from ANOVA to SEM to handle structures in the auxiliary information.
� Collaborative work between experts and statisticians.
� Model inference in progress for nominal auxiliary data.
25June 21 st 2010 MATHMET 2010, PTB
Perspectives
� To test the approach with SEM on water pollutants
when the statistical tool works with nominal data
� To adapt the model for creatinine data (another
structure of the auxiliary information)