bayesian approach to assign consensus values in ... - ptb.de€¦ · june 21 st 2010 mathmet 2010,...

MATHMET 2010International Workshop

Bayesian Approach to assign Consensus Values in PT Comparisons

Séverine DemeyerNicolas Fischer

severine.demeyer@lne.fr

Mathematics and Statistics Division (LNE)

Berlin, June 21 st 2010

2June 21 st 2010 MATHMET 2010, PTB

Outline

� Framework of Proficiency Testing

� PT data

� Standardized approach to assign consensus values: NF ISO 13 528

� The proposed approach: modelling bias

� Methodology

� When to introduce latent predictors of bias?

� Statistical model

� Estimating the model

� Bayesian computation of posterior distributions

� Getting the consensus value, its associated uncertainty and bias

� Conclusion

� Perspectives

3June 21 st 2010 MATHMET 2010, PTB

Framework of the project

� Era-net+ European project entitled « Traceable

measurements for biospecies and ion activity in clinical

chemistry » (JRP 10, TRACEBIOACTIVITY)

� WP 5: PTB, SP, LNE

� Delivery 2: Evaluating a consensus value in proficiency tests.

� Funded by the European Community’s Seventh Framework

Programme, ERA-NET Plus, under Grant Agreement No.

217257.

4June 21 st 2010 MATHMET 2010, PTB

Samples provided by BIPEA

N stable and homogene samples

BTEX, PCB, Triazines

5June 21 st 2010 MATHMET 2010, PTB

PT data

� PT provider: BIPEA (2nd provider in Europe)

� Measurands: concentrations of BTEX, Triazine and PCB in

water

� No associated uncertainties

� 31 participating laboratories

-3

-2

-1

0

1

2

3

4

5

6

1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31

ATRAZ37

DSAZ37

SIMAZ37

TBUTZ37

CYANA37

DIA37

CV=0.300.290.220.200.310.62

Example: results for 6 analytes from triazine family

Z scores

labs

analytes

6June 21 st 2010 MATHMET 2010, PTB

NF ISO 13 528

5.6 Consensus value from participants [see ISO/IEC Guide 43-1:1997, A.1.1 item e)]

5.6.1 General«With this approach, the assigned value X for the test material

used in a round of a proficiency testing scheme is the robust

average of the results reported by all the participants in the

round, calculated using Algorithm A in Annex C.

Other calculation methods may be used in place of

Algorithm A, provided that they have a sound statistical basis and

the report describes the method that is used. »

Statistical methods for use in proficiencytesting by interlaboratory comparisons

7June 21 st 2010 MATHMET 2010, PTB

NF ISO 13 528: Algorithm A

Algorithm A to compute robust means and standard deviations

* *

* * *

if

if

otherwise

i i

i i i

i

x x x

x x x x

x

δ δ

δ δ

− < −= + > +

*1,5sδ =

* 's median ix x=* *1,483 median of the is x x= × −

Initialisation

Iterate till convergence

Outputs:

*

1,25xs

up

= ×

*xConsensus value:

Associated uncertainty:

Bias:*

ix x−

8June 21 st 2010 MATHMET 2010, PTB

Examples

Crossed effect on bias?

Effect of method

on bias?

9June 21 st 2010 MATHMET 2010, PTB

Overview

Proficiency testing

statistical

model

Measurement data

(quality assessment)

Auxiliary information

(survey)

Bayesian estimation

consensus value,

associated uncertainty,

bias

10June 21 st 2010 MATHMET 2010, PTB

Proposed measurement model

jj jZX µ β τ= + +

Consensus value Measurement bias

of laboratory j

Predictors

Nature of Zj ?

Modelling

Bias

Results

11June 21 st 2010 MATHMET 2010, PTB

Construction of predictors

� Depends on the measurand

� Based on experts knowledge (survey,…)

� If a few number of variables can explain bias:

���� Zj are kept as observed variables

� If several variables can explain bias:

���� the observed variables are grouped

���� Zj are latent variables summarizing the observed

variables

���� Zj should capture structures in data

12June 21 st 2010 MATHMET 2010, PTB

Steps

� Collecting measurements

� Collecting additional information on laboratories (survey)

� Converting this information into variables

���� latent variables

� Constructing a statistical model

� Estimating the model

� Validating the model

13June 21 st 2010 MATHMET 2010, PTB

Building latent variables

� Idea: summarizing measurement process + background information on labs

� Blocks = latent (unobserved) concepts, variables

14June 21 st 2010 MATHMET 2010, PTB

Links between questions and blocks

� Latent concepts are measured on observed variables

(the questions)

� Example:

15June 21 st 2010 MATHMET 2010, PTB

Structure of the model

Structural equation modelling

Hierarchical modelling

16June 21 st 2010 MATHMET 2010, PTB

Structural equation modelling

1η

2η

1ξ 2ξ

3ξ4ξ

11111111θθθθ

12121212θθθθ

13131313θθθθ

15151515θθθθ

14141414θθθθη1111

y1111

2y

3y

4y

5y

1 12 2 12 13 14 1

2 23 2

2 3 4

3 44 2

j j j j

j

j j

j jj

ξ ξ ξη π η λ λ λ δη δξ ξλ λ

= + + + +

= + +

1 111 1

515 15

j j j

j j j

y

y

η ε

η

θ

θ ε

= +

= +K

11θ

15θ

(Simultaneous equations)

17June 21 st 2010 MATHMET 2010, PTB

Complete model

j

j

j j

j j

j j

j

Z

Z

H

X

Y

Z

µ β νθ ε

δ

= + +

= +

= Λ +SEM

Results

Consensus value

Auxiliary

data

Endogeneous

latent variables

Bias

18June 21 st 2010 MATHMET 2010, PTB

Model inference: Bayesian approach

� To take into account prior information on parameters

(correlations, variances)

� Estimation algorithm based on posterior conditional

distributions (MCMC)

� Iterating:

� Imputation of latent variables

� Posterior sampling of parameters

19June 21 st 2010 MATHMET 2010, PTB

Proposed Gibbs algorithm to estimate SEM

20June 21 st 2010 MATHMET 2010, PTB

Imputation of latent variables

Results:

Let

21June 21 st 2010 MATHMET 2010, PTB

Posterior conditional distributions

Let

22June 21 st 2010 MATHMET 2010, PTB

Posterior conditional distributionsConjugate models

Normal/Gamma( ), ~ ,k k k k kY Z Y N Z εθ ε θ= + Σ

( ), ~ ,k k k k k k k kH Z H N Z δδ= Λ + Λ Σ

23June 21 st 2010 MATHMET 2010, PTB

Estimation of the model: 3 phases

Phase 3

Consensus value Associated uncertainty Biais

Latent variables(continuous)

Phase 1

continuous

AuxiliaryData

Nominal / binary

Phase 2continuous

Estimating theparameters of the structural model

Estimating theparameters of the hierarchical model

Results

24June 21 st 2010 MATHMET 2010, PTB

Conclusion

� New approach to compute consensus values and their associated uncertainties

� Modelling bias

� Modelling structures in auxiliary data

� To propose different models from ANOVA to SEM to handle structures in the auxiliary information.

� Collaborative work between experts and statisticians.

� Model inference in progress for nominal auxiliary data.

25June 21 st 2010 MATHMET 2010, PTB

Perspectives

� To test the approach with SEM on water pollutants

when the statistical tool works with nominal data

� To adapt the model for creatinine data (another

structure of the auxiliary information)