gauge r&r: a gum/metrological/ bayesian perspective dave leblond mbsw-38 may 19, 2015 1

1

Gauge R&R: A GUM/Metrological/ Bayesian Perspective

Dave LeBlondMBSW-38

May 19, 2015

2

Acknowledgements

Thanks to:

• Stan Altan (J&J)

• Bill Porter (PPP LLC)

• Yan Shen (J&J)

• Jyh-Ming Shoung (J&J)

for organizing this session, inviting me, inspiring discussions, and providing the 3 Gauge R&R examples.

3

Outline

• Measurement Uncertainty (MU) from a Bayesian perspective

• Computational considerations

• Examples1. Dissolution measurement

2. Particle size measurement

3. Bioassay measurement

• Conclusions

• Bibliography

4

The GUM perspective

Q: What is U?

A: “A parameter, associated with the result of a measurement, that characterizes the dispersion of the values that could reasonably be attributed to the measurand” (i.e., The part of the result after the ±).

Reportable result = m

Current practice

± U

ISO/GUM compliant practice

5

MU requires a complete probability model

• Let m be an analytical measurement that estimates m

• Let m be the unknown measurand quantity

• fixed, but uncertain

• MU refers to the uncertainty in m (not m)

• ISO: “…values that could reasonably be attributed to the measurand.”

• (Don’t forget to put the u in the mu)

• The posterior, p(m|m), expresses current knowledge about m

• To get p(m|m), we need prior knowledge and Bayes’ rule

• Inference about m requires a Bayesian approach

Reportable result = m ± U

6

U has a probability interpretation

m

p(m|m)

mm-U m+U

• Bayesian interpretation: posterior distribution of m, conditional on m

• Assume symmetrical, location-scale family with scale independent of m

Reportable result = m ± U

m U

m U

p | m d e.g., .

0 95

7

MU needs 2 spaces & 2 experimentsSpace

Observed Parameter

Gauge R&R

Routine Measurement

p(m|m) ∫p(m| ,m s)p(m)p’(s)ds

p(q) p(s)

p(m)

prior

p(y|q,s)

likelihood

y = vector of observed Gauge R&R results q = vector of true levels

s = true analytical imprecision

p(m| ,m s)

m = true but uncertain measurand quantity valuem = analytical measurement that estimates m

posterior

p’(s) = p(s|y) ∫p(y|q,s)p(q)p(s)dq

p’(s)

Expe

rimen

t

8

MU needs 2 spaces & 2 experimentsSpace

Observed Parameter

Gauge R&R

Routine Measurement p(m| ,m s)

p(y|q,s)

p(m|m) ∫p(m| ,m s)p(m)p’(s)ds

p’(s) = p(s|y) ∫p(y|q,s)p(q)p(s)dq

p(q) p(s)

p’(s)p(m)

likelihood prior posterior

Expe

rimen

t

Assume• m is unbiased for m• normal likelihoods• diffuse priors for q and m: p(q) = p(m) = 1

Key outputs• Gauge R&R: p’(s) and U• Routine Measurement: p(m|m) … or at least m ± U

9

Getting ∫p(m| ,m s)p(m)p’(s)ds by MC Integration

l(m|m,s)

likelihood

l(m|m=0,s)

location-scalem arbitrary

*s

MCMC draws

l(m|m=0, *s )

Plug ineach s*

*m

Obtain 1 MC drawfor each *s

*m0P2.5 P97.5

p( *m |m=0)

Estimatepercentiles

p(m) = 1

Diffuseprior

U = ½(P97.5 – P2.5)

Estimate U

10

Sometimes buried in the SAS Log…NOTE: Convergence criteria met.NOTE: Estimated G matrix is not positive definite.NOTE: Asymptotic variance matrix of covariance parameter estimates has been

found to be singular and a generalized inverse was used. Covariance parameters with zero variance do not contribute to degrees of freedom computed by DDFM=KENWARDROGER.

MIXED arbitrarily sets a negative variance estimate to zero, effectively performing model reduction and ignoring uncertainty in the estimate.

The Bayesian prior would disallow a negative variance so that an interval estimate of the variance is available.

Covariance Parameter Estimates

Standard Cov Parm Group Estimate Error Lower Upper

Day(Site) 0.2612 0.5356 0.03723 503733 ExpRu*HPLC(Site*Day) 0.9645 0.5322 0.4129 4.2521 ExpRun(Day) Site G 6.3519 3.5080 2.7176 28.0595 ExpRun(Day) Site I 0.8339 0.9441 0.2028 80.8289 ExpRun(Day) Site L 10.3038 5.2917 4.6221 39.6342 ExpRun(Day) Site T 0 . . . Residual Site G 5.7744 1.1481 4.0502 8.8978 Residual Site I 3.0847 0.6446 2.1285 4.8713 Residual Site L 8.6020 1.6357 6.1194 12.9815 Residual Site T 6.7212 1.2652 4.7965 10.0968

11

Implementing MCMC• An MCMC chain is a correlated multivariate sample from the multivariate posterior

distribution of all parameters in the model, given the data and prior assumptions.

• Independent “noninformative” (wide uniform or normal with huge variance) used for all parameters.

• Square root of variance components as parameters

• Run 3 MCMC chains from different (random) starting points

• Only retain every 40th iteration as a draw (to reduce autocorrelation)

• Discard first 3,000 draws from each chain (“burn-in”)

• Save 10,000 draws from each chain (30,000 total)

• Checked for convergence of the 3 chains

12

Comparison of approachesConsideration MIXED BUGS

Estimation Method REML, ML, MIVQUE0, Type1, Type2, Type3

MCMC, method chosen by BUGS

Denominator df Method BW, CON, KR, RES, SAT, DDF=list No worries

Multiple Comparison Adjustment Method

BON, SCHEFFE, SIDAK, SIMULATE, T No worries

Effect coefficients E1, E2, E3 No worries

Computational optionsCONVF, CONVG, CONVH, DFBW, EMPIRICAL, NOBOUND, RIDGE=, SCORING=, NOPROFILE

No worries

Asymptotic normality? yes No approximations

Iteration convergence Warnings provided Manual

Speed Generally very fast Can be slow (hours)

Syntax mimics… Mixed model algebra Data generationNon-negative bounding of variances Arbitrarily set to zero Handled through prior

13

1. Dataobs i j k l m y1 2 7 20 25 2 81.002 2 7 20 25 3 80.623 2 7 20 25 1 82.46…70 2 12 23 48 3 83.3671 2 12 23 48 2 85.2372 2 12 23 48 1 85.0173 1 1 8 1 2 90.3974 1 1 8 1 3 95.5075 1 1 8 1 1 92.40…142 1 6 11 24 3 81.01143 1 6 11 24 2 86.82144 1 6 11 24 1 91.73145 3 13 32 49 2 87.12146 3 13 32 49 3 88.72147 3 13 32 49 1 86.20…214 3 18 35 72 3 76.74215 3 18 35 72 2 81.62216 3 18 35 72 1 82.42217 4 19 44 73 2 93.51218 4 19 44 73 3 91.59219 4 19 44 73 1 92.07…286 4 24 47 96 3 85.47287 4 24 47 96 2 90.00288 4 24 47 96 1 89.44

i indexes Site (S, 4 unique sites)

j indexes Day within Site (SD, 24 unique Days)

k indexes ExpRun within Site (E, 48 unique ExpRuns)

l indexes HPLC*ExpRun combinations within Site (HE, 96 unique combinations)

m indexes Batch (B, 3 unique batches)

14

1. Statistical model

obs obsi obs m obs j obs k obs l obsy S B SD E HE

j obs

i [ obs ]k obs

l obs

obs i obs

SD ~ N ,sigma.SD , j obs ...

E ~ N ,sigma.E ,k obs ...

HE ~ N ,sigma.HE ,l obs ...

~ N ,sigma

2

2

2

2

0 1 24

0 1 48

0 1 96

0

measurement uncertainty effects“fixed”(but uncertain) effects

obs ...

i obs ...

m obs ...

1 288

1 4

1 3

Set to zero restrictions

S B 4 3 0

Site Batch Day(Site) ExpRun(Day) HPLC*ExpRun(Site*Day)Diss30

15

1. BUGS modelmodel{

# Likelihood for(obs in 1:n.obs){ mu[obs] <- theta + S[i[obs]] + B[m[obs]] + SD[j[obs]] + E[k[obs]]+ HE[l[obs]] y[obs] ~ dnorm(mu[obs],tau[i[obs]]) } for(jj in 1:24){ SD[jj] ~ dnorm(0,tau.SD) } for(ll in 1:96){ HE[ll] ~ dnorm(0,tau.HE) } for(kk in 1:12){ E[kk] ~ dnorm(0,tau.E[1]) } for(kk in 13:24){ E[kk] ~ dnorm(0,tau.E[2]) } for(kk in 25:36){ E[kk] ~ dnorm(0,tau.E[3]) } for(kk in 37:48){ E[kk] ~ dnorm(0,tau.E[4]) }

# Priors theta~dnorm(0,0.000001)

for(site in 1:3){ S[site]~dnorm(0,0.000001) } S[4]<- 0

for(batch in 1:2){ B[batch]~dnorm(0,0.000001) } B[3]<- 0

for(site in 1:4){ sigma[site] ~ dunif(0,100) tau[site] <- pow(sigma[site],-2) sigma.E[site] ~ dunif(0,100) tau.E[site] <- pow(sigma.E[site],-2) }

sigma.SD~dunif(0,100) tau.SD<-pow(sigma.SD,-2)

sigma.HE~dunif(0,100) tau.HE<-pow(sigma.HE,-2)

}

16

1. Joint posterior marginals mean sd 2.5% 25% 50% 75% 97.5% Rhat n.efftheta 87.86865 0.56354 86.75000 87.50000 87.87000 88.23000 88.99000 1.00100 30000S[1] -1.85345 1.10904 -4.06602 -2.57400 -1.85200 -1.14400 0.33300 1.00098 30000S[2] -4.80681 0.75670 -6.31602 -5.29025 -4.80400 -4.32000 -3.29998 1.00101 30000S[3] -6.78676 1.32984 -9.42800 -7.63400 -6.78500 -5.93800 -4.18898 1.00102 29000B[1] 1.67055 0.33997 1.00500 1.44500 1.66700 1.90100 2.34400 1.00098 30000B[2] -0.57506 0.34345 -1.24800 -0.80692 -0.57430 -0.34580 0.10550 1.00100 30000sigma.SD 0.63573 0.42448 0.02972 0.30180 0.58095 0.89732 1.59600 1.00151 3500sigma.E[1] 2.81409 0.91456 1.39297 2.19200 2.68500 3.28100 4.99000 1.00095 30000sigma.E[2] 0.99404 0.57189 0.08034 0.58310 0.94945 1.32700 2.25900 1.00095 30000sigma.E[3] 3.65876 1.09118 2.02900 2.90500 3.48000 4.22000 6.24702 1.00097 30000sigma.E[4] 0.56074 0.44593 0.02289 0.22100 0.46070 0.79032 1.66800 1.00109 14000sigma.HE 0.95419 0.31807 0.21159 0.76710 0.97680 1.17100 1.52400 1.01856 1100sigma[1] 2.47125 0.25124 2.03500 2.29400 2.45200 2.62800 3.01800 1.00114 11000sigma[2] 1.82693 0.19887 1.48300 1.68600 1.81000 1.95200 2.26100 1.00102 30000sigma[3] 3.00323 0.29334 2.49100 2.79800 2.98200 3.18300 3.64602 1.00095 30000sigma[4] 2.65304 0.25404 2.20997 2.47400 2.63400 2.80900 3.20302 1.00106 19000

17

1. Marginal posterior distribution of sigma by site

Posterior for sigma

30,000 mcmc drawssigma

Pe

rce

nt o

f To

tal

0

5

10

15

1 2 3 4 5

G I

L

1 2 3 4 5

0

5

10

15

T

18

1. Marginal posterior distribution of sigma.E by site

Posterior for sigma.E

30,000 mcmc drawssigma.E

Pe

rce

nt o

f To

tal

0

10

20

30

0 5 10

G I

L

0 5 10

0

10

20

30

T

19

1. Marginal Posterior distributions of sigma.SD and sigma.HE

Posterior for sigma.SD

30,000 mcmc drawssigma.SD

Pe

rce

nt o

f To

tal

0

2

4

6

0 1 2 3

Posterior for sigma.HE

30,000 mcmc drawssigma.HE

Pe

rce

nt o

f To

tal

0

2

4

6

0.0 0.5 1.0 1.5 2.0 2.5

20

1. Bivariate posterior kernal density

30,000 drawssigma.SD

sig

ma

.E4

0.1

0

.2

0.3

0.4

0.4

0.5

0.6

0.7

0.8

0.9

1

0.0 0.5 1.0 1.5

0.0

0.5

1.0

1.5

21

1. Variance Components Estimates(SAS, Bayesian)

Variance Components GroupEstimate (REML,

median)Lower95% Upper95%

Day(Site)0.260.34

0.040.0009

5.04E+052.55

DissRun*HPLC(Site*Day)0.960.95

0.410.04

4.252.32

DissRun(Day)

Site G6.357.21

2.721.94

28.0624.90

Site I0.830.90

0.200.006

80.835.10

Site L10.3012.11

4.624.12

39.6339.02

Site T0.000.21

0.000.0005

0.002.78

Residual

Site G5.776.01

4.054.14

8.909.11

Site I3.083.28

2.132.20

4.875.11

Site L8.608.89

6.126.21

12.9813.29

Site T6.726.94

4.804.88

10.1010.26

22

1. Error Budget(SAS, Bayesian)

Site Source % of TotalPosterior quantiles of % of Total

Median(2.5th-97.5th quantile)

G

Residual 43 40(18-65)Day(Site) 2 2(0-18)

DissRun*HPLC(Site*Day) 7 6(0-17)DissRun(Day) 48 49(18-78)

Total 100

I

Residual 60 56(30-81)Day(Site) 5 6(0-33)

DissRun*HPLC(Site*Day) 19 16(1-39)DissRun(Day) 16 16(0-53)

Total 100

L

Residual 43 39(17-64)Day(Site) 1 1(0-11)

DissRun*HPLC(Site*Day) 5 4(0-11)DissRun(Day) 51 54(26-80)

Total 100

T

Residual 85 79(56-94)Day(Site) 3 4(0-24)

DissRun*HPLC(Site*Day) 12 10(1-25)DissRun(Day) 0 2(0-25)

Total 100

23

1. Posterior distribution of Error

To obtain a sample from the posterior distribution of Error:1. Start with 30,000 MCMC draws of the 4 sigmas (a 4-vector for a given site i)2. For each draw vector,

a. simulate a random sample of the 4 error contributors using [2]b. Plug these into [1]

3. The result is 30,000 MCMC draws from the posterior distribution of Error

m Error

Error SD E HE

[1]

i

i

SD ~ N ,sigma.SD

E ~ N ,sigma.E

HE ~ N ,sigma.HE

sigma~ N ,

2

2

2

2

0

0

0

06

[2]

24

Posterior Distribution of Error at 4 Sites

30,000 mcmc drawsError

Pe

rce

nt o

f To

tal

05

1015

2025

-10 0 10

: Site 1 : Site 2

: Site 3

-10 0 10

0510

152025

: Site 4

1. Expanded uncertainties (U) for each site

U = 6.66(SAS: 5.84)

U = 3.72(SAS: 3.21)

U = 8.38(SAS: 7.20)

U = 3.63(SAS: 3.06)

• 95% credible interval of posterior distribution of Error: 0 ± U• Results should be reported as m ± U

25

1. Adjusting for site bias

1.0 1.5 2.0 2.5 3.0 3.5 4.0

80

82

84

86

88

Site Means for Batch C

(based on 30,000 mcmc draws)Site

95

%C

I of S

ite M

ea

nG

I

L

T

• ISO/GUM philosophy: Do everything in your power to adjust for bias• First must decide what “truth” is

• Is one site the “reference” site?• Take the average across sites as the “truth”?

• Determine the bias appropriate for each site.• Adjust reported values from each site by subtracting the site-bias

26

2. Statistical model

obs obsi obs j obs k obs l obsy S B SD SBD

k obs

l obs

obs

SD ~ N ,sigma.SD ,k obs ...

SBD ~ N ,sigma.SBD ,l obs ...

~ N ,sigma

2

2

2

0 1 21

0 1 84

0

measurement uncertainty effects“fixed”(but uncertain) effects

obs ...

i obs ...

j obs ...

1 252

1 7

1 6

Set to zero restrictions

S B 7 6 0

Site Batch Day(Site) Day*Batch(Site)Dv50a

27

2. Joint posterior marginals mean sd 2.5% 25% 50% 75% 97.5% Rhat n.efftheta 3.55274 0.06061 3.43300 3.51300 3.55300 3.59300 3.67200 1.00097 30000S[1] -0.10220 0.06714 -0.23440 -0.14660 -0.10230 -0.05748 0.03012 1.00096 30000S[2] -0.00934 0.06736 -0.14100 -0.05363 -0.00958 0.03472 0.12530 1.00104 21000S[3] 0.10382 0.06718 -0.02814 0.05949 0.10385 0.14870 0.23650 1.00097 30000S[4] -0.01691 0.06750 -0.14980 -0.06217 -0.01745 0.02786 0.11630 1.00108 15000S[5] 0.00924 0.06726 -0.12190 -0.03564 0.00929 0.05394 0.14190 1.00110 13000S[6] -0.01911 0.06704 -0.15110 -0.06396 -0.01881 0.02593 0.11090 1.00118 8500B[1] -0.27059 0.05847 -0.38580 -0.30962 -0.27090 -0.23100 -0.15500 1.00100 30000B[2] 0.35566 0.05898 0.24000 0.31590 0.35510 0.39510 0.47280 1.00101 30000B[3] -0.24313 0.05759 -0.35480 -0.28230 -0.24310 -0.20450 -0.12980 1.00095 30000B[4] -0.04624 0.05878 -0.16140 -0.08572 -0.04664 -0.00697 0.06928 1.00104 23000B[5] 0.04943 0.05874 -0.06520 0.00971 0.04954 0.08884 0.16480 1.00096 30000sigma.SD 0.02372 0.01882 0.00089 0.00914 0.01956 0.03373 0.07047 1.00164 2900sigma.SBD 0.04174 0.02713 0.00200 0.01944 0.03869 0.06058 0.09957 1.00308 2600sigma 0.24960 0.01202 0.22710 0.24130 0.24920 0.25750 0.27440 1.00098 30000

28

2. Variance Component Estimates(SAS, Bayesian)

Variance Components

Estimate Lower95% Upper95% % of Total

Day(SITE_APP)0

0.0004?

8E-7?

0.0050

1(0-7)

BATCH*Day(SITE_APP)

0.000470.0015

0.000064E-6

3.61E+1090.01

12(0-14)

Residual0.060.06

0.050.05

0.080.08

9996(83-100)

Total0.060.07

0.050.05

3.61E+1090.08

100100

Bayesian point estimate is the posterior medianBayesian % of Total is posterior median(central 95% credible interval)

29

2.Posterior for sigma.SBD

30,000 mcmc drawssigma.SBD

Fre

qu

en

cy

0.00 0.05 0.10 0.15

05

00

10

00

20

00

Posterior for sigma.SD

30,000 mcmc drawssigma.SD

Fre

qu

en

cy

0.00 0.05 0.10 0.15

01

00

03

00

0

Posterior for sigma

30,000 mcmc drawssigma

Fre

qu

en

cy

0.20 0.22 0.24 0.26 0.28 0.30

05

00

10

00

20

00

30

2. Bivariate posterior kernal density of 2 error budget components

30,000 drawsDay to Day variance (% of Total)

Da

y*B

atc

h v

ari

an

ce (

% o

f To

tal)

0.01 0.02

0.03

0.04

0.05

0.0

6

0.07

0.08

0.1

0.1

1

0.0 0.5 1.0 1.5 2.0 2.5 3.0

02

46

81

0

31

2. Posterior distribution of Errorm Error

Error SD SBD

SD ~ N ,sigma.SD

SBD ~ N ,sigma.SBD

~ N ,sigma

2

2

2

0

0

0

To obtain a sample from the posterior distribution of Error:1. Start with 30,000 MCMC draws of the 3 sigmas (a 3-vector)2. For each draw vector,

a. simulate a random sample of the Error contributors using [2]b. Plug these into [1]

3. The result is 30,000 MCMC draws from the posterior distribution of Error

[1]

[2]

32

Posterior Distribution of Error

30,000 mcmc drawsError

Pe

rce

nt o

f To

tal

0

2

4

6

-1.0 -0.5 0.0 0.5 1.0

2. Expanded uncertainty (U)• 95% credible interval of posterior distribution of Error: 0 ± U• Future results should be reported as m ± U

U = 0.5031(SAS: 0.5027)

33

3. Statistical model

obs obsi obs j obs k obs l obsy A C R P

k obs

l obs

obs

R ~ N ,sigma.R ,k obs ...

P ~ N ,sigma.P ,l obs ...

~ N ,sigma

2

2

2

0 1 10

0 1 20

0

measurement uncertainty effects“fixed”(but uncertain) effects

obs ...

i obs ...

j obs ...

1 60

1 2

1 6

Set to zero restrictions

A C 2 6 0

Analyst Conc Run(Analyst)Log_assay Plate(Run*Analyst)

34

3. Joint posterior marginals mean sd 2.5% 25% 50% 75% 97.5% Rhat n.efftheta 4.62917 0.03696 4.55700 4.60600 4.62900 4.65200 4.70300 1.00103 36000A[1] 0.02415 0.04812 -0.07098 -0.00528 0.02386 0.05341 0.12050 1.00098 60000C[1] -0.11067 0.02275 -0.15490 -0.12580 -0.11090 -0.09574 -0.06523 1.00100 60000C[2] -0.03164 0.02257 -0.07554 -0.04675 -0.03179 -0.01675 0.01311 1.00100 60000C[3] 0.02867 0.02249 -0.01549 0.01371 0.02860 0.04377 0.07298 1.00099 60000C[4] 0.02143 0.02304 -0.02285 0.00603 0.02111 0.03644 0.06818 1.00099 60000C[5] 0.00480 0.02271 -0.03912 -0.01040 0.00459 0.01977 0.05001 1.00101 52000sigma.R 0.04537 0.03047 0.00243 0.02309 0.04154 0.06171 0.11680 1.00105 25000sigma.P 0.06873 0.01772 0.03955 0.05644 0.06687 0.07890 0.10870 1.00099 60000sigma 0.04523 0.00574 0.03568 0.04117 0.04466 0.04867 0.05811 1.00100 60000deviance -203.08263 10.09257 -219.90000 -210.30000 -204.20000 -197.00000 -180.70000 1.00100 60000

35

3. Variance Components Estimates (in log scale)(SAS, Bayesian)

Variance Components

Estimate Lower95% Upper95% % of Total

Run(Analyst)0.00150.0017

0.00030.000006

1.46790.0136

2121(0-71)

Plate(Analyst*Run)0.00390.0045

0.00170.0016

0.01590.0118

5353(15-83)

Residual0.00190.0020

0.00120.0013

0.00320.0034

2622(8-45)

Total0.00730.0090

0.00320.0051

1.48710.0219

100100

Bayesian point estimate is the posterior medianBayesian % of Total is posterior median(2.5th-97.5 percentiles)

36

3. Posterior distribution of Errorm Error

Error R P

sigma.RR ~ N ,

sigma.PSBD ~ N ,

sigma~ N ,

2

2

2

03

06

06

To obtain a sample from the posterior distribution of Error:1. Start with 60,000 MCMC draws of the 3 sigmas (a 3-vector)2. For each draw vector,

a. simulate a random sample of the 3 error contributors using [2]b. Plug these into [1]

3. The result is 60,000 MCMC draws from the posterior distribution of Error

[1]

[2]

37

m U m U Ue e e e ,e

60,000 mcmc drawsexp(Error)

Per

cent

of T

otal

0

5

10

0.9 1.0 1.1 1.2

Translate to the Relative Potency scale

3. Expanded uncertainty

Error

Per

cent

of T

otal

0

5

10

-0.1 0.0 0.1

+U = +0.0924-U = -0.0924

e-U = 0.912 e+U = 1.097

m U

em em – em-U em+U - em SAS

80 7.1 7.7

102.5 9.0 9.9 7.8

120 10.6 11.6

38

ConclusionsMetrological Approach

Pros• Scientifically sound• Model based (forces analytical introspection)• ISO compliant

Cons• Learning curve for CMC, regulators• Metrological approach still evolving (slowly)• How to deal with site and/or instrument biases?• How to deal with transformed scales of measurement?

Bayesian Version of Metrological ApproachPros• Permits direct probability statements (risk management)• BUGS syntax mimics data generation mechanism• GUM revision moving toward Bayesian perspective

Cons• Steep learning curve for CMC, regulators• Unfamiliar software tools (BUGS, R, STAN, JAGS,…)• MCMC requires care, maybe long computing times

39

Bibliography1. Gelman A, et al (2014) Bayesian data analysis, 3rd edn, CRC Press

2. Burdick R, et al (2005) Design and analysis of gauge R&R studies, SIAM

3. Willink R (2013) Measurement uncertainty and probability, Cambridge University Press [gives a frequentist perspective]

4. Working Group 1 of the Joint Committee for Guides in Metrology (JCGM/WG 1), JCGM 100:2008, GUM 1995 with minor corrections, Evaluation of measurement data — Guide to the expression of uncertainty in measurement.

5. Working Group 2 of the Joint Committee for Guides in Metrology (JCGM/WG 2), JCGM 200:2012, VIM, 3rd edition, 2008 version with minor corrections, International vocabulary of metrology – Basic and general concepts and ssociated terms.

6. Eurachem working group (Editors: S L R Ellison , A Williams) Eurachem/CITAC Guide CG 4 (2012), Quantifying Uncertainty in Analytical Measurement, 3rd edn.

7. Hubert et al (2004, 2007) Harmonization strategies for the validation of quantitative analytical procedures: A SFSTP proposal Part 1, J Pharm Biomed Anal 36, 579-586. Part 2, J Pharm Biomed Anal 45: 70-81, Part 3, J Pharm Biomed Anal 45: 82-96.

8. Feinberg et al (2004) New advances in method validation and measurement uncertainty aimed at improving the quality of chemical data, Anal Bioanal Chem 380:502-514.

9. Howson C, and Urbach P, Scientific reasoning: the Bayesian approach 3rd edn. , Open Court, Chicago, IL [argues that scientific inference requires a Bayesian perspective]

40

“… To endure uncertainty is difficult, but so are most of the other great virtues”.

- Bertrand Russell, 1950

- Thank you for your endurance!!

P.S. Don’t forget to put the u in the mu