validation support for aoac 2015.18 using magruder dataicp • avg ia ratio = 1.38 • several...
TRANSCRIPT
![Page 1: Validation Support for AOAC 2015.18 Using Magruder DataICP • Avg IA ratio = 1.38 • Several values > 1 • Average is approaching the IA, but more variable • 1704 & 1804 = 15](https://reader035.vdocuments.net/reader035/viewer/2022062415/5fbe367bd9c33326ae7b9807/html5/thumbnails/1.jpg)
Validation Support for AOAC
2015.18 Using Magruder Data
(DRAFT)
![Page 2: Validation Support for AOAC 2015.18 Using Magruder DataICP • Avg IA ratio = 1.38 • Several values > 1 • Average is approaching the IA, but more variable • 1704 & 1804 = 15](https://reader035.vdocuments.net/reader035/viewer/2022062415/5fbe367bd9c33326ae7b9807/html5/thumbnails/2.jpg)
Question and assessment:
• Can we use existing ICP-PK Magruder data in
lieu of conducting a full collaborative study?
• Assessment:
1. Reviewed Available Phosphate & Soluble Potash
data since 2016
2. Identified 21 samples
3. Compared data from the common methods
4. Compared data to the AAPFCO IA
5. Provided an evaluation of ICP PK data
![Page 3: Validation Support for AOAC 2015.18 Using Magruder DataICP • Avg IA ratio = 1.38 • Several values > 1 • Average is approaching the IA, but more variable • 1704 & 1804 = 15](https://reader035.vdocuments.net/reader035/viewer/2022062415/5fbe367bd9c33326ae7b9807/html5/thumbnails/3.jpg)
Interchangeability between indicators
Can navigate between stdev, %RSD, Investigational
Allowance (IA) and Horwitz ratios
1. Magruder report provides method stdev
2. IA = analyte stdev x 2.33 • 2.33 = 99% confidence interval, one-tail?
• note: analyte = all method data for that analyte
– combining “pulls-down” better method(s) and “pulls-up” poorer method(s)
3. Solve for analyte stdev = IA / 2.33 • example for Guar = 50; Avail Phosphate analyte stdev = 1.07 / 2.33 = 0.4592
Guar % (use Mean) Avail. Phosphate IA Sol. Potash IA
50 1.07 1.66
OP No. 72 p. 52
![Page 4: Validation Support for AOAC 2015.18 Using Magruder DataICP • Avg IA ratio = 1.38 • Several values > 1 • Average is approaching the IA, but more variable • 1704 & 1804 = 15](https://reader035.vdocuments.net/reader035/viewer/2022062415/5fbe367bd9c33326ae7b9807/html5/thumbnails/4.jpg)
Interchangeability between indicators
4. Generate an IA ratio, which compares the method
stdev to the IA stdev (i.e. method stdev / IA stdev) • If stdev from the method < stdev from the IA; IA ratio will be < 1
• If stdev from the method > stdev from the IA; IA ratio will be > 1
– Can also use the Magruder method %RSD • RSD = stdev / mean x 100
• References: • http://www.magruderchecksample.org/presentations/2018S_IAvsRSDObservations.pdf
• http://www.magruderchecksample.org/presentations/2018W_NewIAMetrics.pdf
– Can compare to Horwitz (i.e. AOAC historic variability)
• calculate Horwitz predicted %RSD = 2C^-0.15 and solve for stdev
![Page 5: Validation Support for AOAC 2015.18 Using Magruder DataICP • Avg IA ratio = 1.38 • Several values > 1 • Average is approaching the IA, but more variable • 1704 & 1804 = 15](https://reader035.vdocuments.net/reader035/viewer/2022062415/5fbe367bd9c33326ae7b9807/html5/thumbnails/5.jpg)
IA ratio for available P2O5 by method
• On average, Grav 41.10 method produced IA ratios < 1
– one of the methods used to help establish the IA
• ICP 41.51 was second best method, but with averaged
IA ratios above 1
– therefore some data variability above the IA
1.64
1.48
1.38
0.83
0.00 0.25 0.50 0.75 1.00 1.25 1.50 1.75
Auto Spec 41.60
Other 41.99
ICP 41.51
Grav 41.10
Averaged P IA Ratio (-Lo and High)
![Page 6: Validation Support for AOAC 2015.18 Using Magruder DataICP • Avg IA ratio = 1.38 • Several values > 1 • Average is approaching the IA, but more variable • 1704 & 1804 = 15](https://reader035.vdocuments.net/reader035/viewer/2022062415/5fbe367bd9c33326ae7b9807/html5/thumbnails/6.jpg)
IA Ratios for P2O5 by Method
Interpretation
• Magruder sample ID on X-axis, IA ratio value on the Y-axis
• Want most, or preferably all, data to fall below IA ratio of 1
• More an indicator of variability/reproducibility than accuracy
• Gravimetric IA ratios approaching ideal
– averaged IA ratio = 0.83, with 81% of data < 1
• (1803 = 3-1-2; 1809 = 3-1-3); (1701 = 18-46-0; 1709 = 12-40-0)
0.00
0.50
1.00
1.50
2.00
2.50
3.00
3.50
4.00
4.50
1602 1607 1608 1609 1612 1701 1704 1707 1709 1710 1802 1803 1804 1805 1806 1809 1810 1811 1812 1902 1903
P IA Ratio Grav Method 41.10
Grav 41.10
IA Ratio
![Page 7: Validation Support for AOAC 2015.18 Using Magruder DataICP • Avg IA ratio = 1.38 • Several values > 1 • Average is approaching the IA, but more variable • 1704 & 1804 = 15](https://reader035.vdocuments.net/reader035/viewer/2022062415/5fbe367bd9c33326ae7b9807/html5/thumbnails/7.jpg)
ICP
• Avg IA ratio = 1.38
• Several values > 1
• Average is approaching the
IA, but more variable
• 1704 & 1804 = 15 and 24% P2O5
• 1609, 1709 & 1805 > 40% P2O5
• Trending in right direction?
Other
• Avg IA ratio = 1.48
• Several values > 1
• Not one single method
Auto Spec
• Surprise?
• Avg IA ratio = 1.64
• Most values > 1
• 41.60 is collaborated
official method
0.00
0.50
1.00
1.50
2.00
2.50
3.00
3.50
4.00
4.501
60
2
16
07
16
08
16
09
16
12
17
01
17
04
17
07
17
09
17
10
18
02
18
03
18
04
18
05
18
06
18
09
18
10
18
11
18
12
19
02
19
03
P IA Ratio - ICP 41.51
ICP 41.51
IA Ratio
0.00
0.50
1.00
1.50
2.00
2.50
3.003.50
4.00
4.50
16
02
16
07
16
08
16
09
16
12
17
01
17
04
17
07
17
09
17
10
18
02
18
03
18
04
18
05
18
06
18
09
18
10
18
11
18
12
19
02
19
03
P IA Ratio - Other 41.99
Other 41.99
IA Ratio
0.00
0.50
1.00
1.50
2.00
2.50
3.00
3.50
4.00
4.50
16
02
16
07
16
08
16
09
16
12
17
01
17
04
17
07
17
09
17
10
18
02
18
03
18
04
18
05
18
06
18
09
18
10
18
11
18
12
19
02
19
03
P IA Ratio - Auto Spectrophotometer 41.60
Auto Spec 41.60
IA Ratio
![Page 8: Validation Support for AOAC 2015.18 Using Magruder DataICP • Avg IA ratio = 1.38 • Several values > 1 • Average is approaching the IA, but more variable • 1704 & 1804 = 15](https://reader035.vdocuments.net/reader035/viewer/2022062415/5fbe367bd9c33326ae7b9807/html5/thumbnails/8.jpg)
IA ratio for soluble K2O by method
• On average, the STPB oxalate produced IA ratios
close to 1
– one of the methods used to help establish the IA
• ICP citrate 50.52 was the second best method, but
with averaged IA ratio above 1
– therefore some data variability above the IA
5.35
1.81
1.61
1.41
1.40
1.11
0.00 0.50 1.00 1.50 2.00 2.50 3.00 3.50 4.00 4.50 5.00 5.50
Flame Ox 50.60
AA Ox 50.30
ICP Ox 50.50
Other 50.99
ICP Citrate 50.52
STPB Ox 50.00
Averaged K IA Ratio (-Lo and High)
![Page 9: Validation Support for AOAC 2015.18 Using Magruder DataICP • Avg IA ratio = 1.38 • Several values > 1 • Average is approaching the IA, but more variable • 1704 & 1804 = 15](https://reader035.vdocuments.net/reader035/viewer/2022062415/5fbe367bd9c33326ae7b9807/html5/thumbnails/9.jpg)
STPB Oxalate
• averaged IA ratio = 1.1
• Several values near 1
• More variable with low
guarantees – 180311 = 2% K2O
– 180911 = 3% K2O
• Otherwise, close to ideal
ICP Citrate
• Avg IA ratio = 1.40
• Data is quite variable
• Several values > 1
Other
• Avg IA ratio = 1.41
• Not a single method
• IA ratio consistently
between 1 to 2
0.00
0.50
1.00
1.50
2.00
2.50
3.00
3.50
4.00
16
02
16
07
16
08
16
10
16
11
17
04
17
07
17
08
17
12
18
02
18
03
18
04
18
06
18
08
18
09
18
10
18
11
18
12
19
02
19
03
19
04
K IA Ratio - STPB Oxalate
STPB Ox 50.00
IA Ratio
0.00
0.50
1.00
1.50
2.00
2.50
3.00
3.50
4.00
16
02
16
07
16
08
16
10
16
11
17
04
17
07
17
08
17
12
18
02
18
03
18
04
18
06
18
08
18
09
18
10
18
11
18
12
19
02
19
03
19
04
K IA Ratio - ICP Citrate
ICP Citrate 50.52
IA Ratio
0.00
0.50
1.00
1.50
2.00
2.50
3.00
3.50
4.00
16
02
16
07
16
08
16
10
16
11
17
04
17
07
17
08
17
12
18
02
18
03
18
04
18
06
18
08
18
09
18
10
18
11
18
12
19
02
19
03
19
04
K IA Ratio - Other
Other 50.99
IA Ratio
![Page 10: Validation Support for AOAC 2015.18 Using Magruder DataICP • Avg IA ratio = 1.38 • Several values > 1 • Average is approaching the IA, but more variable • 1704 & 1804 = 15](https://reader035.vdocuments.net/reader035/viewer/2022062415/5fbe367bd9c33326ae7b9807/html5/thumbnails/10.jpg)
note changes in Y-axis scale
ICP Oxalate
• Avg IA ratio = 1.61
• Oxalate is more difficult
ICP matrix than citrate
• 180812 = 0-0-60
AA Oxalate
• Avg IA ratio = 1.81
• 5 to 6 labs reporting
• Data is quite variable
• Most values between 1 - 3
• 1806 = SRN, 6% K2O
Flame Oxalate
• Avg IA ratio = 5.35
• Yikes!
• Data is highly
variable
• Only 3 to 5 labs reporting
0.00
1.00
2.00
3.00
4.00
5.00
6.00
7.00
16
02
16
07
16
08
16
10
16
11
17
04
17
07
17
08
17
12
18
02
18
03
18
04
18
06
18
08
18
09
18
10
18
11
18
12
19
02
19
03
19
04
K IA - ICP Oxalate
ICP Ox 50.50
IA Ratio
0.00
1.00
2.00
3.00
4.00
5.00
6.00
7.00
16
02
16
07
16
08
16
10
16
11
17
04
17
07
17
08
17
12
18
02
18
03
18
04
18
06
18
08
18
09
18
10
18
11
18
12
19
02
19
03
19
04
K IA Ratio - AA Oxalate
AA Ox 50.30
IA Ratio
0.00
5.00
10.00
15.00
20.00
25.00
16
02
16
07
16
08
16
10
16
11
17
04
17
07
17
08
17
12
18
02
18
03
18
04
18
06
18
08
18
09
18
10
18
11
18
12
19
02
19
03
19
04
K IA Ratio - Flame Oxalate
Flame Ox 50.60
IA Ratio
![Page 11: Validation Support for AOAC 2015.18 Using Magruder DataICP • Avg IA ratio = 1.38 • Several values > 1 • Average is approaching the IA, but more variable • 1704 & 1804 = 15](https://reader035.vdocuments.net/reader035/viewer/2022062415/5fbe367bd9c33326ae7b9807/html5/thumbnails/11.jpg)
What about Horwitz?
• Solve Horwitz predicted RSD for stdev, then multiply by 2.33
for “universal” IA estimate
• K2O IA similar to Horwitz at lower conc; below Horwitz at high
• P2O5 IA is unusual, mostly well below Horwitz & relatively flat
![Page 12: Validation Support for AOAC 2015.18 Using Magruder DataICP • Avg IA ratio = 1.38 • Several values > 1 • Average is approaching the IA, but more variable • 1704 & 1804 = 15](https://reader035.vdocuments.net/reader035/viewer/2022062415/5fbe367bd9c33326ae7b9807/html5/thumbnails/12.jpg)
Magruder ICP-P stdev vs. IA & Horwitz?
• Stdev from Magruder ICP direct available P2O5 data
• Multiply by 2.33 to put on IA scale, then calculate trend line
• Horwitz PRSD solved for stdev, then multiplied by 2.33
• On average, ICP-P data is more similar to Horwitz than to the IA
• Data still quite variable
![Page 13: Validation Support for AOAC 2015.18 Using Magruder DataICP • Avg IA ratio = 1.38 • Several values > 1 • Average is approaching the IA, but more variable • 1704 & 1804 = 15](https://reader035.vdocuments.net/reader035/viewer/2022062415/5fbe367bd9c33326ae7b9807/html5/thumbnails/13.jpg)
Magruder ICP-K stdev vs. IA & Horwitz?
• Stdev from Magruder ICP citrate K2O data
• Multiply by 2.33 to put on IA scale, then calculate trend line
• Horwitz PRSD solved for stdev, then multiplied by 2.33
• On average, ICP-K data is more similar to Horwitz than to the IA
• Data still quite variable
![Page 14: Validation Support for AOAC 2015.18 Using Magruder DataICP • Avg IA ratio = 1.38 • Several values > 1 • Average is approaching the IA, but more variable • 1704 & 1804 = 15](https://reader035.vdocuments.net/reader035/viewer/2022062415/5fbe367bd9c33326ae7b9807/html5/thumbnails/14.jpg)
Summary (Good)
1. Currently, ICP citrate is the second best P & K
method
– some other “official” methods are worse
2. Several Magruder P & K samples/guarantees
– probably enough to satisfy reviewers
3. Wide range of concentrations and matrices
4. Participants likely state, industry, commercial,
private and possibly international labs?
5. Duplicates, so have estimate of repeatability
6. Data is similar to Horwitz, so acceptable
HorRat values are likely
7. Would save LOTS of time
![Page 15: Validation Support for AOAC 2015.18 Using Magruder DataICP • Avg IA ratio = 1.38 • Several values > 1 • Average is approaching the IA, but more variable • 1704 & 1804 = 15](https://reader035.vdocuments.net/reader035/viewer/2022062415/5fbe367bd9c33326ae7b9807/html5/thumbnails/15.jpg)
Summary (Bad) 1. ICP PK data sometimes exceeds the AAPFCO IA
– If becomes official, may need to improve variability or increase IA?
– Relegate to screening method, or not a BPM?
2. Accept data as is, without opportunity for improvement
– No practice samples
– Can’t work with collaborators to improve accuracy or precision
– Can’t un-invite under performing lab(s)
3. Fixed data format/sources vs. dynamic
– Magruder – may have different labs and/or different numbers of
labs reporting for any given sample
4. Statistical challenges?
– Used robust statistics, so not sure about raw statistics
– May be less than 8 labs reporting sometimes, which may
eliminate tests for outliers
– Note < 7 eliminates robust statistics
5. May not meet ISO criteria?
![Page 16: Validation Support for AOAC 2015.18 Using Magruder DataICP • Avg IA ratio = 1.38 • Several values > 1 • Average is approaching the IA, but more variable • 1704 & 1804 = 15](https://reader035.vdocuments.net/reader035/viewer/2022062415/5fbe367bd9c33326ae7b9807/html5/thumbnails/16.jpg)
Conclusions
• Quicker and easier completion of collaborative study,
so the good may outweigh the bad
• Data likely better if did traditional collaborative study
of selecting samples, collaborators, evaluating “pre-
screening” sample data and providing training
– A historical bias?
• Data will most likely meet AOAC validation criteria
• Some data may not meet AAPFCO’s IA criteria
• Look to AOAC for some guidance on data handling
• Looking to AAPFCO Magruder committee for
recommendations
• Acknowledge: Andy Crawford, Frank Sikora, Harold Falls