![Page 1: Gauge R&R: A GUM/Metrological/ Bayesian Perspective Dave LeBlond MBSW-38 May 19, 2015 1](https://reader035.vdocuments.net/reader035/viewer/2022062421/56649d765503460f94a56a34/html5/thumbnails/1.jpg)
1
Gauge R&R: A GUM/Metrological/ Bayesian Perspective
Dave LeBlondMBSW-38
May 19, 2015
![Page 2: Gauge R&R: A GUM/Metrological/ Bayesian Perspective Dave LeBlond MBSW-38 May 19, 2015 1](https://reader035.vdocuments.net/reader035/viewer/2022062421/56649d765503460f94a56a34/html5/thumbnails/2.jpg)
2
Acknowledgements
Thanks to:
• Stan Altan (J&J)
• Bill Porter (PPP LLC)
• Yan Shen (J&J)
• Jyh-Ming Shoung (J&J)
for organizing this session, inviting me, inspiring discussions, and providing the 3 Gauge R&R examples.
![Page 3: Gauge R&R: A GUM/Metrological/ Bayesian Perspective Dave LeBlond MBSW-38 May 19, 2015 1](https://reader035.vdocuments.net/reader035/viewer/2022062421/56649d765503460f94a56a34/html5/thumbnails/3.jpg)
3
Outline
• Measurement Uncertainty (MU) from a Bayesian perspective
• Computational considerations
• Examples1. Dissolution measurement
2. Particle size measurement
3. Bioassay measurement
• Conclusions
• Bibliography
![Page 4: Gauge R&R: A GUM/Metrological/ Bayesian Perspective Dave LeBlond MBSW-38 May 19, 2015 1](https://reader035.vdocuments.net/reader035/viewer/2022062421/56649d765503460f94a56a34/html5/thumbnails/4.jpg)
4
The GUM perspective
Q: What is U?
A: “A parameter, associated with the result of a measurement, that characterizes the dispersion of the values that could reasonably be attributed to the measurand” (i.e., The part of the result after the ±).
Reportable result = m
Current practice
± U
ISO/GUM compliant practice
![Page 5: Gauge R&R: A GUM/Metrological/ Bayesian Perspective Dave LeBlond MBSW-38 May 19, 2015 1](https://reader035.vdocuments.net/reader035/viewer/2022062421/56649d765503460f94a56a34/html5/thumbnails/5.jpg)
5
MU requires a complete probability model
• Let m be an analytical measurement that estimates m
• Let m be the unknown measurand quantity
• fixed, but uncertain
• MU refers to the uncertainty in m (not m)
• ISO: “…values that could reasonably be attributed to the measurand.”
• (Don’t forget to put the u in the mu)
• The posterior, p(m|m), expresses current knowledge about m
• To get p(m|m), we need prior knowledge and Bayes’ rule
• Inference about m requires a Bayesian approach
Reportable result = m ± U
![Page 6: Gauge R&R: A GUM/Metrological/ Bayesian Perspective Dave LeBlond MBSW-38 May 19, 2015 1](https://reader035.vdocuments.net/reader035/viewer/2022062421/56649d765503460f94a56a34/html5/thumbnails/6.jpg)
6
U has a probability interpretation
m
p(m|m)
mm-U m+U
• Bayesian interpretation: posterior distribution of m, conditional on m
• Assume symmetrical, location-scale family with scale independent of m
Reportable result = m ± U
m U
m U
p | m d e.g., .
0 95
![Page 7: Gauge R&R: A GUM/Metrological/ Bayesian Perspective Dave LeBlond MBSW-38 May 19, 2015 1](https://reader035.vdocuments.net/reader035/viewer/2022062421/56649d765503460f94a56a34/html5/thumbnails/7.jpg)
7
MU needs 2 spaces & 2 experimentsSpace
Observed Parameter
Gauge R&R
Routine Measurement
p(m|m) ∫p(m| ,m s)p(m)p’(s)ds
p(q) p(s)
p(m)
prior
p(y|q,s)
likelihood
y = vector of observed Gauge R&R results q = vector of true levels
s = true analytical imprecision
p(m| ,m s)
m = true but uncertain measurand quantity valuem = analytical measurement that estimates m
posterior
p’(s) = p(s|y) ∫p(y|q,s)p(q)p(s)dq
p’(s)
Expe
rimen
t
![Page 8: Gauge R&R: A GUM/Metrological/ Bayesian Perspective Dave LeBlond MBSW-38 May 19, 2015 1](https://reader035.vdocuments.net/reader035/viewer/2022062421/56649d765503460f94a56a34/html5/thumbnails/8.jpg)
8
MU needs 2 spaces & 2 experimentsSpace
Observed Parameter
Gauge R&R
Routine Measurement p(m| ,m s)
p(y|q,s)
p(m|m) ∫p(m| ,m s)p(m)p’(s)ds
p’(s) = p(s|y) ∫p(y|q,s)p(q)p(s)dq
p(q) p(s)
p’(s)p(m)
likelihood prior posterior
Expe
rimen
t
Assume• m is unbiased for m• normal likelihoods• diffuse priors for q and m: p(q) = p(m) = 1
Key outputs• Gauge R&R: p’(s) and U• Routine Measurement: p(m|m) … or at least m ± U
![Page 9: Gauge R&R: A GUM/Metrological/ Bayesian Perspective Dave LeBlond MBSW-38 May 19, 2015 1](https://reader035.vdocuments.net/reader035/viewer/2022062421/56649d765503460f94a56a34/html5/thumbnails/9.jpg)
9
Getting ∫p(m| ,m s)p(m)p’(s)ds by MC Integration
l(m|m,s)
likelihood
l(m|m=0,s)
location-scalem arbitrary
*s
MCMC draws
l(m|m=0, *s )
Plug ineach s*
*m
Obtain 1 MC drawfor each *s
*m0P2.5 P97.5
p( *m |m=0)
Estimatepercentiles
p(m) = 1
Diffuseprior
U = ½(P97.5 – P2.5)
Estimate U
![Page 10: Gauge R&R: A GUM/Metrological/ Bayesian Perspective Dave LeBlond MBSW-38 May 19, 2015 1](https://reader035.vdocuments.net/reader035/viewer/2022062421/56649d765503460f94a56a34/html5/thumbnails/10.jpg)
10
Sometimes buried in the SAS Log…NOTE: Convergence criteria met.NOTE: Estimated G matrix is not positive definite.NOTE: Asymptotic variance matrix of covariance parameter estimates has been
found to be singular and a generalized inverse was used. Covariance parameters with zero variance do not contribute to degrees of freedom computed by DDFM=KENWARDROGER.
MIXED arbitrarily sets a negative variance estimate to zero, effectively performing model reduction and ignoring uncertainty in the estimate.
The Bayesian prior would disallow a negative variance so that an interval estimate of the variance is available.
Covariance Parameter Estimates
Standard Cov Parm Group Estimate Error Lower Upper
Day(Site) 0.2612 0.5356 0.03723 503733 ExpRu*HPLC(Site*Day) 0.9645 0.5322 0.4129 4.2521 ExpRun(Day) Site G 6.3519 3.5080 2.7176 28.0595 ExpRun(Day) Site I 0.8339 0.9441 0.2028 80.8289 ExpRun(Day) Site L 10.3038 5.2917 4.6221 39.6342 ExpRun(Day) Site T 0 . . . Residual Site G 5.7744 1.1481 4.0502 8.8978 Residual Site I 3.0847 0.6446 2.1285 4.8713 Residual Site L 8.6020 1.6357 6.1194 12.9815 Residual Site T 6.7212 1.2652 4.7965 10.0968
![Page 11: Gauge R&R: A GUM/Metrological/ Bayesian Perspective Dave LeBlond MBSW-38 May 19, 2015 1](https://reader035.vdocuments.net/reader035/viewer/2022062421/56649d765503460f94a56a34/html5/thumbnails/11.jpg)
11
Implementing MCMC• An MCMC chain is a correlated multivariate sample from the multivariate posterior
distribution of all parameters in the model, given the data and prior assumptions.
• Independent “noninformative” (wide uniform or normal with huge variance) used for all parameters.
• Square root of variance components as parameters
• Run 3 MCMC chains from different (random) starting points
• Only retain every 40th iteration as a draw (to reduce autocorrelation)
• Discard first 3,000 draws from each chain (“burn-in”)
• Save 10,000 draws from each chain (30,000 total)
• Checked for convergence of the 3 chains
![Page 12: Gauge R&R: A GUM/Metrological/ Bayesian Perspective Dave LeBlond MBSW-38 May 19, 2015 1](https://reader035.vdocuments.net/reader035/viewer/2022062421/56649d765503460f94a56a34/html5/thumbnails/12.jpg)
12
Comparison of approachesConsideration MIXED BUGS
Estimation Method REML, ML, MIVQUE0, Type1, Type2, Type3
MCMC, method chosen by BUGS
Denominator df Method BW, CON, KR, RES, SAT, DDF=list No worries
Multiple Comparison Adjustment Method
BON, SCHEFFE, SIDAK, SIMULATE, T No worries
Effect coefficients E1, E2, E3 No worries
Computational optionsCONVF, CONVG, CONVH, DFBW, EMPIRICAL, NOBOUND, RIDGE=, SCORING=, NOPROFILE
No worries
Asymptotic normality? yes No approximations
Iteration convergence Warnings provided Manual
Speed Generally very fast Can be slow (hours)
Syntax mimics… Mixed model algebra Data generationNon-negative bounding of variances Arbitrarily set to zero Handled through prior
![Page 13: Gauge R&R: A GUM/Metrological/ Bayesian Perspective Dave LeBlond MBSW-38 May 19, 2015 1](https://reader035.vdocuments.net/reader035/viewer/2022062421/56649d765503460f94a56a34/html5/thumbnails/13.jpg)
13
1. Dataobs i j k l m y1 2 7 20 25 2 81.002 2 7 20 25 3 80.623 2 7 20 25 1 82.46…70 2 12 23 48 3 83.3671 2 12 23 48 2 85.2372 2 12 23 48 1 85.0173 1 1 8 1 2 90.3974 1 1 8 1 3 95.5075 1 1 8 1 1 92.40…142 1 6 11 24 3 81.01143 1 6 11 24 2 86.82144 1 6 11 24 1 91.73145 3 13 32 49 2 87.12146 3 13 32 49 3 88.72147 3 13 32 49 1 86.20…214 3 18 35 72 3 76.74215 3 18 35 72 2 81.62216 3 18 35 72 1 82.42217 4 19 44 73 2 93.51218 4 19 44 73 3 91.59219 4 19 44 73 1 92.07…286 4 24 47 96 3 85.47287 4 24 47 96 2 90.00288 4 24 47 96 1 89.44
i indexes Site (S, 4 unique sites)
j indexes Day within Site (SD, 24 unique Days)
k indexes ExpRun within Site (E, 48 unique ExpRuns)
l indexes HPLC*ExpRun combinations within Site (HE, 96 unique combinations)
m indexes Batch (B, 3 unique batches)
![Page 14: Gauge R&R: A GUM/Metrological/ Bayesian Perspective Dave LeBlond MBSW-38 May 19, 2015 1](https://reader035.vdocuments.net/reader035/viewer/2022062421/56649d765503460f94a56a34/html5/thumbnails/14.jpg)
14
1. Statistical model
obs obsi obs m obs j obs k obs l obsy S B SD E HE
j obs
i [ obs ]k obs
l obs
obs i obs
SD ~ N ,sigma.SD , j obs ...
E ~ N ,sigma.E ,k obs ...
HE ~ N ,sigma.HE ,l obs ...
~ N ,sigma
2
2
2
2
0 1 24
0 1 48
0 1 96
0
measurement uncertainty effects“fixed”(but uncertain) effects
obs ...
i obs ...
m obs ...
1 288
1 4
1 3
Set to zero restrictions
S B 4 3 0
Site Batch Day(Site) ExpRun(Day) HPLC*ExpRun(Site*Day)Diss30
![Page 15: Gauge R&R: A GUM/Metrological/ Bayesian Perspective Dave LeBlond MBSW-38 May 19, 2015 1](https://reader035.vdocuments.net/reader035/viewer/2022062421/56649d765503460f94a56a34/html5/thumbnails/15.jpg)
15
1. BUGS modelmodel{
# Likelihood for(obs in 1:n.obs){ mu[obs] <- theta + S[i[obs]] + B[m[obs]] + SD[j[obs]] + E[k[obs]]+ HE[l[obs]] y[obs] ~ dnorm(mu[obs],tau[i[obs]]) } for(jj in 1:24){ SD[jj] ~ dnorm(0,tau.SD) } for(ll in 1:96){ HE[ll] ~ dnorm(0,tau.HE) } for(kk in 1:12){ E[kk] ~ dnorm(0,tau.E[1]) } for(kk in 13:24){ E[kk] ~ dnorm(0,tau.E[2]) } for(kk in 25:36){ E[kk] ~ dnorm(0,tau.E[3]) } for(kk in 37:48){ E[kk] ~ dnorm(0,tau.E[4]) }
# Priors theta~dnorm(0,0.000001)
for(site in 1:3){ S[site]~dnorm(0,0.000001) } S[4]<- 0
for(batch in 1:2){ B[batch]~dnorm(0,0.000001) } B[3]<- 0
for(site in 1:4){ sigma[site] ~ dunif(0,100) tau[site] <- pow(sigma[site],-2) sigma.E[site] ~ dunif(0,100) tau.E[site] <- pow(sigma.E[site],-2) }
sigma.SD~dunif(0,100) tau.SD<-pow(sigma.SD,-2)
sigma.HE~dunif(0,100) tau.HE<-pow(sigma.HE,-2)
}
![Page 16: Gauge R&R: A GUM/Metrological/ Bayesian Perspective Dave LeBlond MBSW-38 May 19, 2015 1](https://reader035.vdocuments.net/reader035/viewer/2022062421/56649d765503460f94a56a34/html5/thumbnails/16.jpg)
16
1. Joint posterior marginals mean sd 2.5% 25% 50% 75% 97.5% Rhat n.efftheta 87.86865 0.56354 86.75000 87.50000 87.87000 88.23000 88.99000 1.00100 30000S[1] -1.85345 1.10904 -4.06602 -2.57400 -1.85200 -1.14400 0.33300 1.00098 30000S[2] -4.80681 0.75670 -6.31602 -5.29025 -4.80400 -4.32000 -3.29998 1.00101 30000S[3] -6.78676 1.32984 -9.42800 -7.63400 -6.78500 -5.93800 -4.18898 1.00102 29000B[1] 1.67055 0.33997 1.00500 1.44500 1.66700 1.90100 2.34400 1.00098 30000B[2] -0.57506 0.34345 -1.24800 -0.80692 -0.57430 -0.34580 0.10550 1.00100 30000sigma.SD 0.63573 0.42448 0.02972 0.30180 0.58095 0.89732 1.59600 1.00151 3500sigma.E[1] 2.81409 0.91456 1.39297 2.19200 2.68500 3.28100 4.99000 1.00095 30000sigma.E[2] 0.99404 0.57189 0.08034 0.58310 0.94945 1.32700 2.25900 1.00095 30000sigma.E[3] 3.65876 1.09118 2.02900 2.90500 3.48000 4.22000 6.24702 1.00097 30000sigma.E[4] 0.56074 0.44593 0.02289 0.22100 0.46070 0.79032 1.66800 1.00109 14000sigma.HE 0.95419 0.31807 0.21159 0.76710 0.97680 1.17100 1.52400 1.01856 1100sigma[1] 2.47125 0.25124 2.03500 2.29400 2.45200 2.62800 3.01800 1.00114 11000sigma[2] 1.82693 0.19887 1.48300 1.68600 1.81000 1.95200 2.26100 1.00102 30000sigma[3] 3.00323 0.29334 2.49100 2.79800 2.98200 3.18300 3.64602 1.00095 30000sigma[4] 2.65304 0.25404 2.20997 2.47400 2.63400 2.80900 3.20302 1.00106 19000
![Page 17: Gauge R&R: A GUM/Metrological/ Bayesian Perspective Dave LeBlond MBSW-38 May 19, 2015 1](https://reader035.vdocuments.net/reader035/viewer/2022062421/56649d765503460f94a56a34/html5/thumbnails/17.jpg)
17
1. Marginal posterior distribution of sigma by site
Posterior for sigma
30,000 mcmc drawssigma
Pe
rce
nt o
f To
tal
0
5
10
15
1 2 3 4 5
G I
L
1 2 3 4 5
0
5
10
15
T
![Page 18: Gauge R&R: A GUM/Metrological/ Bayesian Perspective Dave LeBlond MBSW-38 May 19, 2015 1](https://reader035.vdocuments.net/reader035/viewer/2022062421/56649d765503460f94a56a34/html5/thumbnails/18.jpg)
18
1. Marginal posterior distribution of sigma.E by site
Posterior for sigma.E
30,000 mcmc drawssigma.E
Pe
rce
nt o
f To
tal
0
10
20
30
0 5 10
G I
L
0 5 10
0
10
20
30
T
![Page 19: Gauge R&R: A GUM/Metrological/ Bayesian Perspective Dave LeBlond MBSW-38 May 19, 2015 1](https://reader035.vdocuments.net/reader035/viewer/2022062421/56649d765503460f94a56a34/html5/thumbnails/19.jpg)
19
1. Marginal Posterior distributions of sigma.SD and sigma.HE
Posterior for sigma.SD
30,000 mcmc drawssigma.SD
Pe
rce
nt o
f To
tal
0
2
4
6
0 1 2 3
Posterior for sigma.HE
30,000 mcmc drawssigma.HE
Pe
rce
nt o
f To
tal
0
2
4
6
0.0 0.5 1.0 1.5 2.0 2.5
![Page 20: Gauge R&R: A GUM/Metrological/ Bayesian Perspective Dave LeBlond MBSW-38 May 19, 2015 1](https://reader035.vdocuments.net/reader035/viewer/2022062421/56649d765503460f94a56a34/html5/thumbnails/20.jpg)
20
1. Bivariate posterior kernal density
30,000 drawssigma.SD
sig
ma
.E4
0.1
0
.2
0.3
0.4
0.4
0.5
0.6
0.7
0.8
0.9
1
0.0 0.5 1.0 1.5
0.0
0.5
1.0
1.5
![Page 21: Gauge R&R: A GUM/Metrological/ Bayesian Perspective Dave LeBlond MBSW-38 May 19, 2015 1](https://reader035.vdocuments.net/reader035/viewer/2022062421/56649d765503460f94a56a34/html5/thumbnails/21.jpg)
21
1. Variance Components Estimates(SAS, Bayesian)
Variance Components GroupEstimate (REML,
median)Lower95% Upper95%
Day(Site)0.260.34
0.040.0009
5.04E+052.55
DissRun*HPLC(Site*Day)0.960.95
0.410.04
4.252.32
DissRun(Day)
Site G6.357.21
2.721.94
28.0624.90
Site I0.830.90
0.200.006
80.835.10
Site L10.3012.11
4.624.12
39.6339.02
Site T0.000.21
0.000.0005
0.002.78
Residual
Site G5.776.01
4.054.14
8.909.11
Site I3.083.28
2.132.20
4.875.11
Site L8.608.89
6.126.21
12.9813.29
Site T6.726.94
4.804.88
10.1010.26
![Page 22: Gauge R&R: A GUM/Metrological/ Bayesian Perspective Dave LeBlond MBSW-38 May 19, 2015 1](https://reader035.vdocuments.net/reader035/viewer/2022062421/56649d765503460f94a56a34/html5/thumbnails/22.jpg)
22
1. Error Budget(SAS, Bayesian)
Site Source % of TotalPosterior quantiles of % of Total
Median(2.5th-97.5th quantile)
G
Residual 43 40(18-65)Day(Site) 2 2(0-18)
DissRun*HPLC(Site*Day) 7 6(0-17)DissRun(Day) 48 49(18-78)
Total 100
I
Residual 60 56(30-81)Day(Site) 5 6(0-33)
DissRun*HPLC(Site*Day) 19 16(1-39)DissRun(Day) 16 16(0-53)
Total 100
L
Residual 43 39(17-64)Day(Site) 1 1(0-11)
DissRun*HPLC(Site*Day) 5 4(0-11)DissRun(Day) 51 54(26-80)
Total 100
T
Residual 85 79(56-94)Day(Site) 3 4(0-24)
DissRun*HPLC(Site*Day) 12 10(1-25)DissRun(Day) 0 2(0-25)
Total 100
![Page 23: Gauge R&R: A GUM/Metrological/ Bayesian Perspective Dave LeBlond MBSW-38 May 19, 2015 1](https://reader035.vdocuments.net/reader035/viewer/2022062421/56649d765503460f94a56a34/html5/thumbnails/23.jpg)
23
1. Posterior distribution of Error
To obtain a sample from the posterior distribution of Error:1. Start with 30,000 MCMC draws of the 4 sigmas (a 4-vector for a given site i)2. For each draw vector,
a. simulate a random sample of the 4 error contributors using [2]b. Plug these into [1]
3. The result is 30,000 MCMC draws from the posterior distribution of Error
m Error
Error SD E HE
[1]
i
i
SD ~ N ,sigma.SD
E ~ N ,sigma.E
HE ~ N ,sigma.HE
sigma~ N ,
2
2
2
2
0
0
0
06
[2]
![Page 24: Gauge R&R: A GUM/Metrological/ Bayesian Perspective Dave LeBlond MBSW-38 May 19, 2015 1](https://reader035.vdocuments.net/reader035/viewer/2022062421/56649d765503460f94a56a34/html5/thumbnails/24.jpg)
24
Posterior Distribution of Error at 4 Sites
30,000 mcmc drawsError
Pe
rce
nt o
f To
tal
05
1015
2025
-10 0 10
: Site 1 : Site 2
: Site 3
-10 0 10
0510
152025
: Site 4
1. Expanded uncertainties (U) for each site
U = 6.66(SAS: 5.84)
U = 3.72(SAS: 3.21)
U = 8.38(SAS: 7.20)
U = 3.63(SAS: 3.06)
• 95% credible interval of posterior distribution of Error: 0 ± U• Results should be reported as m ± U
![Page 25: Gauge R&R: A GUM/Metrological/ Bayesian Perspective Dave LeBlond MBSW-38 May 19, 2015 1](https://reader035.vdocuments.net/reader035/viewer/2022062421/56649d765503460f94a56a34/html5/thumbnails/25.jpg)
25
1. Adjusting for site bias
1.0 1.5 2.0 2.5 3.0 3.5 4.0
80
82
84
86
88
Site Means for Batch C
(based on 30,000 mcmc draws)Site
95
%C
I of S
ite M
ea
nG
I
L
T
• ISO/GUM philosophy: Do everything in your power to adjust for bias• First must decide what “truth” is
• Is one site the “reference” site?• Take the average across sites as the “truth”?
• Determine the bias appropriate for each site.• Adjust reported values from each site by subtracting the site-bias
![Page 26: Gauge R&R: A GUM/Metrological/ Bayesian Perspective Dave LeBlond MBSW-38 May 19, 2015 1](https://reader035.vdocuments.net/reader035/viewer/2022062421/56649d765503460f94a56a34/html5/thumbnails/26.jpg)
26
2. Statistical model
obs obsi obs j obs k obs l obsy S B SD SBD
k obs
l obs
obs
SD ~ N ,sigma.SD ,k obs ...
SBD ~ N ,sigma.SBD ,l obs ...
~ N ,sigma
2
2
2
0 1 21
0 1 84
0
measurement uncertainty effects“fixed”(but uncertain) effects
obs ...
i obs ...
j obs ...
1 252
1 7
1 6
Set to zero restrictions
S B 7 6 0
Site Batch Day(Site) Day*Batch(Site)Dv50a
![Page 27: Gauge R&R: A GUM/Metrological/ Bayesian Perspective Dave LeBlond MBSW-38 May 19, 2015 1](https://reader035.vdocuments.net/reader035/viewer/2022062421/56649d765503460f94a56a34/html5/thumbnails/27.jpg)
27
2. Joint posterior marginals mean sd 2.5% 25% 50% 75% 97.5% Rhat n.efftheta 3.55274 0.06061 3.43300 3.51300 3.55300 3.59300 3.67200 1.00097 30000S[1] -0.10220 0.06714 -0.23440 -0.14660 -0.10230 -0.05748 0.03012 1.00096 30000S[2] -0.00934 0.06736 -0.14100 -0.05363 -0.00958 0.03472 0.12530 1.00104 21000S[3] 0.10382 0.06718 -0.02814 0.05949 0.10385 0.14870 0.23650 1.00097 30000S[4] -0.01691 0.06750 -0.14980 -0.06217 -0.01745 0.02786 0.11630 1.00108 15000S[5] 0.00924 0.06726 -0.12190 -0.03564 0.00929 0.05394 0.14190 1.00110 13000S[6] -0.01911 0.06704 -0.15110 -0.06396 -0.01881 0.02593 0.11090 1.00118 8500B[1] -0.27059 0.05847 -0.38580 -0.30962 -0.27090 -0.23100 -0.15500 1.00100 30000B[2] 0.35566 0.05898 0.24000 0.31590 0.35510 0.39510 0.47280 1.00101 30000B[3] -0.24313 0.05759 -0.35480 -0.28230 -0.24310 -0.20450 -0.12980 1.00095 30000B[4] -0.04624 0.05878 -0.16140 -0.08572 -0.04664 -0.00697 0.06928 1.00104 23000B[5] 0.04943 0.05874 -0.06520 0.00971 0.04954 0.08884 0.16480 1.00096 30000sigma.SD 0.02372 0.01882 0.00089 0.00914 0.01956 0.03373 0.07047 1.00164 2900sigma.SBD 0.04174 0.02713 0.00200 0.01944 0.03869 0.06058 0.09957 1.00308 2600sigma 0.24960 0.01202 0.22710 0.24130 0.24920 0.25750 0.27440 1.00098 30000
![Page 28: Gauge R&R: A GUM/Metrological/ Bayesian Perspective Dave LeBlond MBSW-38 May 19, 2015 1](https://reader035.vdocuments.net/reader035/viewer/2022062421/56649d765503460f94a56a34/html5/thumbnails/28.jpg)
28
2. Variance Component Estimates(SAS, Bayesian)
Variance Components
Estimate Lower95% Upper95% % of Total
Day(SITE_APP)0
0.0004?
8E-7?
0.0050
1(0-7)
BATCH*Day(SITE_APP)
0.000470.0015
0.000064E-6
3.61E+1090.01
12(0-14)
Residual0.060.06
0.050.05
0.080.08
9996(83-100)
Total0.060.07
0.050.05
3.61E+1090.08
100100
Bayesian point estimate is the posterior medianBayesian % of Total is posterior median(central 95% credible interval)
![Page 29: Gauge R&R: A GUM/Metrological/ Bayesian Perspective Dave LeBlond MBSW-38 May 19, 2015 1](https://reader035.vdocuments.net/reader035/viewer/2022062421/56649d765503460f94a56a34/html5/thumbnails/29.jpg)
29
2.Posterior for sigma.SBD
30,000 mcmc drawssigma.SBD
Fre
qu
en
cy
0.00 0.05 0.10 0.15
05
00
10
00
20
00
Posterior for sigma.SD
30,000 mcmc drawssigma.SD
Fre
qu
en
cy
0.00 0.05 0.10 0.15
01
00
03
00
0
Posterior for sigma
30,000 mcmc drawssigma
Fre
qu
en
cy
0.20 0.22 0.24 0.26 0.28 0.30
05
00
10
00
20
00
![Page 30: Gauge R&R: A GUM/Metrological/ Bayesian Perspective Dave LeBlond MBSW-38 May 19, 2015 1](https://reader035.vdocuments.net/reader035/viewer/2022062421/56649d765503460f94a56a34/html5/thumbnails/30.jpg)
30
2. Bivariate posterior kernal density of 2 error budget components
30,000 drawsDay to Day variance (% of Total)
Da
y*B
atc
h v
ari
an
ce (
% o
f To
tal)
0.01 0.02
0.03
0.04
0.05
0.0
6
0.07
0.08
0.1
0.1
1
0.0 0.5 1.0 1.5 2.0 2.5 3.0
02
46
81
0
![Page 31: Gauge R&R: A GUM/Metrological/ Bayesian Perspective Dave LeBlond MBSW-38 May 19, 2015 1](https://reader035.vdocuments.net/reader035/viewer/2022062421/56649d765503460f94a56a34/html5/thumbnails/31.jpg)
31
2. Posterior distribution of Errorm Error
Error SD SBD
SD ~ N ,sigma.SD
SBD ~ N ,sigma.SBD
~ N ,sigma
2
2
2
0
0
0
To obtain a sample from the posterior distribution of Error:1. Start with 30,000 MCMC draws of the 3 sigmas (a 3-vector)2. For each draw vector,
a. simulate a random sample of the Error contributors using [2]b. Plug these into [1]
3. The result is 30,000 MCMC draws from the posterior distribution of Error
[1]
[2]
![Page 32: Gauge R&R: A GUM/Metrological/ Bayesian Perspective Dave LeBlond MBSW-38 May 19, 2015 1](https://reader035.vdocuments.net/reader035/viewer/2022062421/56649d765503460f94a56a34/html5/thumbnails/32.jpg)
32
Posterior Distribution of Error
30,000 mcmc drawsError
Pe
rce
nt o
f To
tal
0
2
4
6
-1.0 -0.5 0.0 0.5 1.0
2. Expanded uncertainty (U)• 95% credible interval of posterior distribution of Error: 0 ± U• Future results should be reported as m ± U
U = 0.5031(SAS: 0.5027)
![Page 33: Gauge R&R: A GUM/Metrological/ Bayesian Perspective Dave LeBlond MBSW-38 May 19, 2015 1](https://reader035.vdocuments.net/reader035/viewer/2022062421/56649d765503460f94a56a34/html5/thumbnails/33.jpg)
33
3. Statistical model
obs obsi obs j obs k obs l obsy A C R P
k obs
l obs
obs
R ~ N ,sigma.R ,k obs ...
P ~ N ,sigma.P ,l obs ...
~ N ,sigma
2
2
2
0 1 10
0 1 20
0
measurement uncertainty effects“fixed”(but uncertain) effects
obs ...
i obs ...
j obs ...
1 60
1 2
1 6
Set to zero restrictions
A C 2 6 0
Analyst Conc Run(Analyst)Log_assay Plate(Run*Analyst)
![Page 34: Gauge R&R: A GUM/Metrological/ Bayesian Perspective Dave LeBlond MBSW-38 May 19, 2015 1](https://reader035.vdocuments.net/reader035/viewer/2022062421/56649d765503460f94a56a34/html5/thumbnails/34.jpg)
34
3. Joint posterior marginals mean sd 2.5% 25% 50% 75% 97.5% Rhat n.efftheta 4.62917 0.03696 4.55700 4.60600 4.62900 4.65200 4.70300 1.00103 36000A[1] 0.02415 0.04812 -0.07098 -0.00528 0.02386 0.05341 0.12050 1.00098 60000C[1] -0.11067 0.02275 -0.15490 -0.12580 -0.11090 -0.09574 -0.06523 1.00100 60000C[2] -0.03164 0.02257 -0.07554 -0.04675 -0.03179 -0.01675 0.01311 1.00100 60000C[3] 0.02867 0.02249 -0.01549 0.01371 0.02860 0.04377 0.07298 1.00099 60000C[4] 0.02143 0.02304 -0.02285 0.00603 0.02111 0.03644 0.06818 1.00099 60000C[5] 0.00480 0.02271 -0.03912 -0.01040 0.00459 0.01977 0.05001 1.00101 52000sigma.R 0.04537 0.03047 0.00243 0.02309 0.04154 0.06171 0.11680 1.00105 25000sigma.P 0.06873 0.01772 0.03955 0.05644 0.06687 0.07890 0.10870 1.00099 60000sigma 0.04523 0.00574 0.03568 0.04117 0.04466 0.04867 0.05811 1.00100 60000deviance -203.08263 10.09257 -219.90000 -210.30000 -204.20000 -197.00000 -180.70000 1.00100 60000
![Page 35: Gauge R&R: A GUM/Metrological/ Bayesian Perspective Dave LeBlond MBSW-38 May 19, 2015 1](https://reader035.vdocuments.net/reader035/viewer/2022062421/56649d765503460f94a56a34/html5/thumbnails/35.jpg)
35
3. Variance Components Estimates (in log scale)(SAS, Bayesian)
Variance Components
Estimate Lower95% Upper95% % of Total
Run(Analyst)0.00150.0017
0.00030.000006
1.46790.0136
2121(0-71)
Plate(Analyst*Run)0.00390.0045
0.00170.0016
0.01590.0118
5353(15-83)
Residual0.00190.0020
0.00120.0013
0.00320.0034
2622(8-45)
Total0.00730.0090
0.00320.0051
1.48710.0219
100100
Bayesian point estimate is the posterior medianBayesian % of Total is posterior median(2.5th-97.5 percentiles)
![Page 36: Gauge R&R: A GUM/Metrological/ Bayesian Perspective Dave LeBlond MBSW-38 May 19, 2015 1](https://reader035.vdocuments.net/reader035/viewer/2022062421/56649d765503460f94a56a34/html5/thumbnails/36.jpg)
36
3. Posterior distribution of Errorm Error
Error R P
sigma.RR ~ N ,
sigma.PSBD ~ N ,
sigma~ N ,
2
2
2
03
06
06
To obtain a sample from the posterior distribution of Error:1. Start with 60,000 MCMC draws of the 3 sigmas (a 3-vector)2. For each draw vector,
a. simulate a random sample of the 3 error contributors using [2]b. Plug these into [1]
3. The result is 60,000 MCMC draws from the posterior distribution of Error
[1]
[2]
![Page 37: Gauge R&R: A GUM/Metrological/ Bayesian Perspective Dave LeBlond MBSW-38 May 19, 2015 1](https://reader035.vdocuments.net/reader035/viewer/2022062421/56649d765503460f94a56a34/html5/thumbnails/37.jpg)
37
m U m U Ue e e e ,e
60,000 mcmc drawsexp(Error)
Per
cent
of T
otal
0
5
10
0.9 1.0 1.1 1.2
Translate to the Relative Potency scale
3. Expanded uncertainty
Error
Per
cent
of T
otal
0
5
10
-0.1 0.0 0.1
+U = +0.0924-U = -0.0924
e-U = 0.912 e+U = 1.097
m U
em em – em-U em+U - em SAS
80 7.1 7.7
102.5 9.0 9.9 7.8
120 10.6 11.6
![Page 38: Gauge R&R: A GUM/Metrological/ Bayesian Perspective Dave LeBlond MBSW-38 May 19, 2015 1](https://reader035.vdocuments.net/reader035/viewer/2022062421/56649d765503460f94a56a34/html5/thumbnails/38.jpg)
38
ConclusionsMetrological Approach
Pros• Scientifically sound• Model based (forces analytical introspection)• ISO compliant
Cons• Learning curve for CMC, regulators• Metrological approach still evolving (slowly)• How to deal with site and/or instrument biases?• How to deal with transformed scales of measurement?
Bayesian Version of Metrological ApproachPros• Permits direct probability statements (risk management)• BUGS syntax mimics data generation mechanism• GUM revision moving toward Bayesian perspective
Cons• Steep learning curve for CMC, regulators• Unfamiliar software tools (BUGS, R, STAN, JAGS,…)• MCMC requires care, maybe long computing times
![Page 39: Gauge R&R: A GUM/Metrological/ Bayesian Perspective Dave LeBlond MBSW-38 May 19, 2015 1](https://reader035.vdocuments.net/reader035/viewer/2022062421/56649d765503460f94a56a34/html5/thumbnails/39.jpg)
39
Bibliography1. Gelman A, et al (2014) Bayesian data analysis, 3rd edn, CRC Press
2. Burdick R, et al (2005) Design and analysis of gauge R&R studies, SIAM
3. Willink R (2013) Measurement uncertainty and probability, Cambridge University Press [gives a frequentist perspective]
4. Working Group 1 of the Joint Committee for Guides in Metrology (JCGM/WG 1), JCGM 100:2008, GUM 1995 with minor corrections, Evaluation of measurement data — Guide to the expression of uncertainty in measurement.
5. Working Group 2 of the Joint Committee for Guides in Metrology (JCGM/WG 2), JCGM 200:2012, VIM, 3rd edition, 2008 version with minor corrections, International vocabulary of metrology – Basic and general concepts and ssociated terms.
6. Eurachem working group (Editors: S L R Ellison , A Williams) Eurachem/CITAC Guide CG 4 (2012), Quantifying Uncertainty in Analytical Measurement, 3rd edn.
7. Hubert et al (2004, 2007) Harmonization strategies for the validation of quantitative analytical procedures: A SFSTP proposal Part 1, J Pharm Biomed Anal 36, 579-586. Part 2, J Pharm Biomed Anal 45: 70-81, Part 3, J Pharm Biomed Anal 45: 82-96.
8. Feinberg et al (2004) New advances in method validation and measurement uncertainty aimed at improving the quality of chemical data, Anal Bioanal Chem 380:502-514.
9. Howson C, and Urbach P, Scientific reasoning: the Bayesian approach 3rd edn. , Open Court, Chicago, IL [argues that scientific inference requires a Bayesian perspective]
![Page 40: Gauge R&R: A GUM/Metrological/ Bayesian Perspective Dave LeBlond MBSW-38 May 19, 2015 1](https://reader035.vdocuments.net/reader035/viewer/2022062421/56649d765503460f94a56a34/html5/thumbnails/40.jpg)
40
“… To endure uncertainty is difficult, but so are most of the other great virtues”.
- Bertrand Russell, 1950
- Thank you for your endurance!!
P.S. Don’t forget to put the u in the mu