smokeless tobacco sub-group - coresta · smokeless tobacco sub-group ... karl wagner, ph.d. altria...
Post on 21-Apr-2018
219 Views
Preview:
TRANSCRIPT
Smokeless Tobacco Sub-Group
Technical Report
CORESTA Reference Products
2014 Analysis
January 2015
Author and Sub-Group Secretary:
Karl Wagner, Ph.D.
Altria Client Services Inc., U.S.A.
Sub-Group Coordinator and Study Coordinator:
John Bunch, Ph.D.
American Snuff Company, U.S.A.
Co-Author and Statistical Analysis:
Michael Morton, Ph.D.
Altria Client Services Inc., U.S.A.
STS-CTR 2014 Analysis of CRPs – January 2015 2/45
Table of Contents
Table of Contents ....................................................................................................................... 2
1. Summary ............................................................................................................................ 3
2. Introduction ........................................................................................................................ 3
2.1 Objective ...................................................................................................................... 4
3. Organisation ....................................................................................................................... 4
3.1 Participants .................................................................................................................. 4
3.2 Protocol ........................................................................................................................ 5
3.2.1 Sample Shipment .................................................................................................. 5
3.2.2 Within Laboratory Sample Preparation ................................................................ 5
3.2.3 Sample Analysis and Data Reporting ................................................................... 5
4. Data – Raw ......................................................................................................................... 6
5. Data – Statistical Analysis .................................................................................................. 9
5.1 Exclusion of Outliers ................................................................................................... 9
5.2 Calculation of Repeatability (r) and Reproducibility (R) .......................................... 10
5.3 Calculation of Z-Scores ............................................................................................. 11
5.4 CRP Stability Assessment ......................................................................................... 12
6. Data Interpretations .......................................................................................................... 14
7. Recommendations ............................................................................................................ 15
APPENDIX A: Raw Data Plots for 2014 WG4 Study ............................................................ 16
APPENDIX B: CRP Stability Assessment, Mean Data Plots ................................................. 30
APPENDIX C: Results using Cochran’s Test for Outlier Detection ...................................... 44
STS-CTR 2014 Analysis of CRPs – January 2015 3/45
1. Summary
In May 2014, the CORESTA Smokeless Tobacco Sub-Group (STS) conducted the annual
Working Group 4 (WG4) inter-laboratory study designed to assess the stability of the four
CORESTA Reference Products (CRPs). Eleven laboratories participated in the 2014 study.
The participating laboratories reported the levels of nicotine, pH, moisture (oven volatiles),
and tobacco specific nitrosamines (TSNAs) using CORESTA Recommended Methods
(CRMs). In an initial study conducted in 2010, the STS recommended these analytes as
appropriate markers for monitoring product stability. Tabulated data are presented along with
repeatability (r) and reproducibility (R) and z-scores.
The results from the stability analysis over all study years showed only a few trends.
Specifically:
• In the 2012/2013 report1, a statistically significant increase in moisture was noted for
CRP3 over the study years (2010-2013). This upward trend did not continue in 2014;
however, the 2014 CRP3 moisture results are statistically significantly higher as
compared to 2010 and 2011.
• In the 2012/2013 report, an increase in inter-laboratory variability for pH was noted
for CRP3 over the study years (2010-2013). There is still an indication that this is the
case; however, the trend did not reach the level of statistical significance in 2014.
• In 2012, there was a notable increase in inter-laboratory variability for moisture for
CRP4. This year the lab-to-lab differences were much smaller, possibly because all of
the laboratories ordered the CRPs from North Carolina State University just prior to
testing.
Based on the results taken in their entirety, the conclusion of this study is that all four CRPs
are suitable for continued use as reference products. Lack of any clear and significant trends
in the stability results indicate that storage at −20 °C is an appropriate storage condition for
the four reference products. The recommendations for 2015 are to continue to monitor the
stability of the CRPs and to continue to require all participating laboratories to procure the
CRPs immediately prior to the laboratory phase of the study to mitigate differences in long
term storage conditions where the products may not have been stored at −20 °C.
2. Introduction
In November 2008, the Smokeless Tobacco Sub-Group (STS) was established by
recommendation of the CORESTA Scientific Commission. In 2009, STS Working Group
Three (WG3) cooperated to design and manufacture four CORESTA Reference Products
(CRPs) referred to as CRP1, CRP2, CRP3, and CRP4. These products were intended as
replacements for the Smokeless Tobacco Research Products: 2S3 (Moist Snuff), 1S2 (Dry
Snuff) and 2S1 (Loose-leaf Chewing Tobacco), which were more than ten years old.
At the Amelia Island, Florida, STS meeting (October 2009), Working Group 4 (WG4) was
organized to proceed with the chemical characterization of the four CRPs. The protocol for
the first WG4 study2 was distributed in December 2009 and the study was conducted in 2010.
This study included 43 analytes and did not specify methods of analysis. Many of the results
from this study showed a wide range in analyte levels, which is not unexpected since the
methods were not harmonized. The recommendations from the 2010 study were to monitor
the stability of the CRPs on an annual basis, by determining the levels of nicotine, pH,
1 STS Technical Report: CORESTA Reference Products 2012 and 2013 Analysis, July 2014.
2 STS Technical Report: CORESTA Reference Products 2010 Analysis, February 2014.
STS-CTR 2014 Analysis of CRPs – January 2015 4/45
moisture (oven volatiles) and TSNAs using CORESTA Recommended Methods (CRMs) or
draft CRMs. This annual analysis would allow the STS to determine when the CRPs should
be remanufactured.
At the STS Meeting in Hamburg, Germany (May 2011), it was decided to adopt the
recommendations from the 2010 WG4 study and initiate the first annual analysis of the four
CRPs using CRMs or draft CRMs. The first WG4 study using CRMs or draft CRMs was
conducted in 2011, and a similar study has been carried annually since that time. The focus of
this report is the 2014 study. Repeatability and reproducibility (r & R) and z-scores were
calculated for the 2014 study while the stability analysis includes data from 2010-2014. As
mentioned earlier, methods of analysis were not specified for the 2010 WG4; however, the
data were considered to be similar enough for inclusion in the analysis.
The CRPs continue to be stored at -20°C and distributed by the North Carolina State
University (NCSU) Tobacco Analytical Services Lab under the direction of Dr. Ramsey
Lewis3 and Karen Andres
4.
2.1 Objective
The 2014 participating laboratories were to provide analytical results for pH, moisture (oven
volatiles), nicotine, and the four TSNAs (N-nitrosonornicotine (NNN), N-nitrosoanatabine
(NAT), N-nitrosoanabasine (NAB) and 4-(N-nitrosomethylamino)-1-(3-pyridyl)-1-butanone
(NNK)). This work was conducted using the applicable CRMs referenced in Section 3.
These studies were conducted to support the assessment of stability of the CRPs and to
provide an assessment of inter-laboratory variability. Data were collected from the
participating laboratories and statistically evaluated in basic conformance with the
recommendations of ISO 5725-2:1994 and ISO/TR 22971:2005. The stability analysis was
conducted using the study data from 2010 through 2014. Additionally, z-scores were
calculated for the 2014 data as an additional measure of each laboratory’s performance as
compared to the results of other laboratories.
3. Organisation
3.1 Participants
A list of the participating laboratories is provided in Table 1. The laboratories are listed in
alphabetical order. Letter codes were assigned to each laboratory and do not correspond to
the order in the table below. Since the statistical analysis covers several years, the laboratory
letter codes were maintained between the tables for all years. Furthermore, not all
laboratories participated in all time points or submitted data for all analyses.
Table 1: List of Participating Laboratories in the 2014 WG4 Study
2014 WG4 Participants
Altria Client Services
American Snuff Company
Essentra Scientific Services
Global Laboratory Services
Imperial Tobacco Group, Seita
3 Ramsey_lewis@ncsu.edu
4 Karen_Andres@ncsu.edu
STS-CTR 2014 Analysis of CRPs – January 2015 5/45
2014 WG4 Participants
Japan Tobacco, Inc.
R.J. Reynolds Tobacco Company
Swedish Match Northern Europe
Swisher International, Inc.
Technical Center of Shanghai Tobacco Group Co., Ltd
University of Kentucky
3.2 Protocol
Specific details from the protocol are described below:
3.2.1 Sample Shipment
Laboratories were responsible for procuring approximately 150g of each of the CRPs from
NCSU immediately before starting the 2014 study. Laboratories were requested to store the
samples at approximately 4°C upon receipt if the analyses would be conducted within one
week or to store the samples at approximately -20°C if the analyses would be delayed. The
study start date was May 4, 2014. Laboratories were requested to submit data by May 23,
2014. The samples are identified in Table 2.
Table 2: Sample Identification
Product Type
CRP1 - Swedish style snus pouch
CRP2 - American-style loose moist snuff
CRP3 - American-style loose dry snuff powder
CRP4 - American-style loose-leaf chewing tobacco
3.2.2 Within Laboratory Sample Preparation
The laboratories were directed to remove the samples from cold storage the day before testing
and to not open the samples until equilibrated to ambient temperature. Once samples were
opened, they could be stored at approximately 4°C for up to one week. Additionally:
• The snus pouch (CRP1) was to be cut into two halves and placed directly into the
extraction vessel. Both the tobacco and pouch material were included in the analysis.
• The moist snuff reference product (CRP2) and the dry snuff reference product (CRP3)
did not require sample grinding and were analyzed as received.
• The Loose Leaf reference product (CRP4) was to be ground according the
participating laboratory’s standard procedure.
3.2.3 Sample Analysis and Data Reporting
The WG4 participating laboratories were instructed to conduct triplicate analyses for the
following: pH, moisture (oven volatiles), nicotine, and TSNAs. The laboratories were
requested to use the following CRMs:
• pH: CRM N° 69, Determination of pH in Smokeless Tobacco Products
• Moisture (oven volatiles): CRM N° 76, Determination of Moisture Content (Oven
Volatiles) of Smokeless Tobacco Products
STS-CTR 2014 Analysis of CRPs – January 2015 6/45
• Nicotine: CRM N° 62, Determination of Nicotine in Tobacco and Tobacco Products
by Gas Chromatographic Analysis
• TSNAs: CRM N° 72, Determination of Tobacco Specific Nitrosamines in Smokeless
Tobacco Products by Liquid Chromatography - Tandem Mass Spectrometry
Participating laboratories were requested to document any deviations from the CRMs and
submit the deviations with their results; however, no deviations were reported. All test results
were to be reported on an as-is basis with no correction for moisture content. The results
were not to be rounded and ideally reported to at least one more digit than typically required.
The study results and the comments were to be sent by e-mail to John E. Bunch, Study
Coordinator.
4. Data – Raw
The full data set for the 2014 study is listed in Table 3. The results are presented on an as-is
basis, without correction for moisture. Each analysis includes three replicates. Raw data
plots that include all replicates, without removal of outliers, are given in Appendix A.
Table 3: Full Data Set for 2014 WG4 Study (results are presented on an as-is basis)
Lab Code
Product Nicotine pH Moisture NNN NNK NAT NAB
% % µg/g µg/g µg/g µg/g
A CRP1 1.01 7.91 49.83 0.67 0.23 0.53 0.035
A CRP1 0.96 7.94 50.48 0.70 0.24 0.59 0.041
A CRP1 1.04 7.94 49.64 0.61 0.22 0.57 0.040
A CRP2 1.29 7.67 54.85 1.82 0.50 1.90 0.164
A CRP2 1.30 7.66 54.72 1.85 0.46 1.94 0.177
A CRP2 1.28 7.66 54.88 1.76 0.50 1.94 0.178
A CRP3 2.21 6.59 8.96 8.50 4.85 6.67 0.404
A CRP3 2.18 6.67 9.94 8.29 4.76 6.06 0.410
A CRP3 2.19 6.65 8.81 8.77 4.52 6.58 0.383
A CRP4 1.06 5.89 22.62 1.97 0.46 1.43 0.064
A CRP4 1.06 5.90 22.65 2.18 0.47 1.32 0.069
A CRP4 1.09 5.90 21.93 1.98 0.46 1.29 0.075
F CRP1 0.88 7.76 51.75 0.69 0.21 0.54 0.033
F CRP1 0.73 7.78 52.20 0.68 0.20 0.50 0.029
F CRP1 0.77 7.81 50.32 0.70 0.21 0.52 0.034
F CRP2 1.19 7.641
54.58 1.97 0.47 1.78 0.146
F CRP2 1.17 7.511 54.44 1.80 0.45 1.72 0.147
F CRP2 1.17 7.551 54.42 1.81 0.46 1.81 0.134
F CRP3 2.15 6.51 8.88 8.50 5.22 6.07 0.368
F CRP3 2.13 6.56 8.91 8.62 4.52 6.08 0.372
F CRP3 2.17 6.59 8.81 8.62 4.73 5.50 0.377
F CRP4 1.11 5.61 23.04 1.81 0.39 1.16 0.049
F CRP4 0.95 5.75 23.45 1.96 0.54 1.23 0.057
F CRP4 0.81 5.72 21.37 1.89 0.42 1.22 0.057
G CRP1 1.00 7.99 52.33 0.80 0.23 0.64 0.044
G CRP1 0.99 7.92 52.44 0.72 0.21 0.57 0.036
G CRP1 1.03 7.95 52.73 0.73 0.21 0.58 0.039
G CRP2 1.24 7.71 54.67 2.06 0.48 2.04 0.187
STS-CTR 2014 Analysis of CRPs – January 2015 7/45
Lab Code
Product Nicotine pH Moisture NNN NNK NAT NAB
% % µg/g µg/g µg/g µg/g
G CRP2 1.27 7.70 54.63 2.10 0.47 1.99 0.188
G CRP2 1.26 7.70 54.65 2.10 0.49 2.00 0.191
G CRP3 2.12 6.72 8.91 9.60 3.96 6.53 0.500
G CRP3 2.21 6.72 8.93 9.23 3.98 6.49 0.509
G CRP3 2.14 6.72 8.94 9.49 3.91 6.51 0.483
G CRP4 1.07 5.92 21.23 2.18 0.46 1.40 0.072
G CRP4 1.06 5.92 21.19 2.26 0.41 1.33 0.076
G CRP4 1.07 5.92 21.27 2.26 0.44 1.33 0.067
I CRP1 0.98 7.95 53.67 0.66 0.22 0.54 0.030
I CRP1 0.97 7.95 51.55 0.71 0.22 0.52 0.031
I CRP1 0.92 7.95 51.20 0.70 0.21 0.53 0.032
I CRP2 1.25 7.68 53.70 2.01 0.48 1.79 0.160
I CRP2 1.24 7.67 53.45 1.97 0.46 1.83 0.160
I CRP2 1.25 7.67 53.52 1.94 0.48 1.89 0.157
I CRP3 2.13 6.85 7.67 8.98 4.31 5.53 0.396
I CRP3 2.13 6.84 7.45 8.78 4.23 5.42 0.408
I CRP3 2.14 6.84 7.36 8.47 4.46 5.71 0.389
I CRP4 1.17 5.90 22.03 2.15 0.50 1.33 0.062
I CRP4 1.17 5.89 21.95 2.15 0.50 1.36 0.062
I CRP4 1.15 5.89 21.99 2.17 0.47 1.33 0.058
L CRP1 1.02 7.89 51.41 0.58 0.20 0.48 0.037
L CRP1 1.04 7.92 51.42 0.61 0.21 0.50 0.037
L CRP1 1.03 7.91 49.54 0.63 0.21 0.50 0.036
L CRP2 1.29 7.68 54.59 1.75 0.44 1.69 0.146
L CRP2 1.27 7.69 54.49 1.74 0.44 1.70 0.150
L CRP2 1.28 7.69 54.44 1.73 0.43 1.70 0.150
L CRP3 2.21 6.72 8.44 8.57 4.31 5.82 0.406
L CRP3 2.21 6.72 8.56 8.49 4.24 5.77 0.395
L CRP3 2.20 6.71 8.50 8.77 4.18 5.78 0.403
L CRP4 1.07 5.93 23.28 1.95 0.44 1.20 0.063
L CRP4 1.07 5.93 23.27 1.93 0.43 1.18 0.066
L CRP4 1.07 5.93 23.28 1.93 0.43 1.17 0.061
M CRP1 0.98 8.04 48.61 0.58 0.21 0.49 0.028
M CRP1 0.98 7.95 47.75 0.68 0.23 0.53 0.032
M CRP1 1.01 7.93 48.16 0.66 0.21 0.52 0.029
M CRP2 1.29 7.69 54.88 1.77 0.43 1.72 0.152
M CRP2 1.33 7.70 54.22 1.78 0.47 1.75 0.153
M CRP2 1.32 7.71 54.29 1.79 0.47 1.81 0.145
M CRP3 2.20 6.84 10.28 8.44 4.06 5.79 0.397
M CRP3 2.13 6.83 10.39 8.51 4.12 5.63 0.421
M CRP3 2.15 6.84 10.12 8.10 4.16 5.67 0.398
M CRP4 1.06 5.89 21.44 1.83 0.39 1.27 0.055
M CRP4 1.05 5.91 21.80 1.93 0.46 1.30 0.061
M CRP4 1.13 5.94 22.07 1.75 0.40 1.16 0.059
O CRP1 0.88 7.57 54.34 0.65 0.20 0.52 0.018
O CRP1 0.88 7.57 54.13 0.64 0.21 0.53 0.016
STS-CTR 2014 Analysis of CRPs – January 2015 8/45
Lab Code
Product Nicotine pH Moisture NNN NNK NAT NAB
% % µg/g µg/g µg/g µg/g
O CRP1 0.86 7.58 54.24 0.62 0.20 0.52 0.015
O CRP2 1.16 7.48 54.47 1.66 0.44 1.72 0.109
O CRP2 1.15 7.48 54.41 1.69 0.45 1.76 0.112
O CRP2 1.12 7.49 54.39 1.69 0.46 1.75 0.112
O CRP3 1.95 6.72 10.31 5.76 3.18 4.72 0.225
O CRP3 1.98 6.75 10.33 6.99 3.81 5.73 0.294
O CRP3 1.94 6.73 10.18 5.90 3.13 4.65 0.234
O CRP4 0.93 5.94 21.61 1.65 0.43 1.28 0.042
O CRP4 0.99 5.97 21.79 1.93 0.54 1.49 0.061
O CRP4 0.99 5.96 21.40 1.94 0.44 1.37 0.041
P CRP1 0.89 7.31 50.93 0.69 0.18 0.44 0.032
P CRP1 1.00 7.33 51.04 0.70 0.18 0.44 0.033
P CRP1 0.87 7.35 51.23 0.69 0.18 0.46 0.035
P CRP2 1.18 7.42 53.14 2.00 0.40 1.49 0.158
P CRP2 1.20 7.44 52.77 2.09 0.40 1.47 0.150
P CRP2 1.26 7.47 53.31 1.98 0.39 1.47 0.153
P CRP3 2.05 6.83 8.02 8.86 5.02 6.07 0.432
P CRP3 2.05 6.85 8.49 8.58 4.99 5.91 0.435
P CRP3 2.04 6.82 8.17 8.72 5.05 5.93 0.432
P CRP4 1.23 5.93 21.13 2.18 0.48 1.39 0.069
P CRP4 1.22 5.94 20.98 2.15 0.47 1.39 0.069
P CRP4 1.20 5.92 21.12 2.19 0.48 1.37 0.070
Q CRP1 0.97 7.96 50.20 0.63 0.21 0.45 0.023
Q CRP1 1.06 7.99 50.76 0.61 0.19 0.44 0.027
Q CRP1 0.96 8.05 50.57 0.68 0.23 0.49 0.026
Q CRP2 1.26 7.75 54.51 1.87 0.43 1.65 0.135
Q CRP2 1.25 7.75 54.40 1.87 0.46 1.67 0.139
Q CRP2 1.26 7.75 54.12 1.88 0.44 1.63 0.134
Q CRP3 2.17 6.82 9.20 8.80 4.53 5.41 0.354
Q CRP3 2.18 6.81 9.31 8.64 4.46 5.39 0.357
Q CRP3 2.18 6.81 9.36 8.71 4.61 5.26 0.356
Q CRP4 1.10 5.94 22.23 2.08 0.44 1.17 0.056
Q CRP4 1.13 5.91 22.07 2.08 0.44 1.20 0.053
Q CRP4 1.13 5.91 22.04 2.07 0.45 1.18 0.056
R CRP1 0.83 7.84 50.39 0.68 0.21 0.58 0.035
R CRP1 0.84 7.77 50.40 0.67 0.19 0.56 0.033
R CRP1 0.86 7.73 50.95 0.63 0.19 0.57 0.032
R CRP2 0.92 7.56 54.12 1.82 0.39 1.88 0.156
R CRP2 1.16 7.58 54.23 1.82 0.38 1.89 0.155
R CRP2 1.15 7.56 54.53 1.81 0.40 1.85 0.156
R CRP3 1.83 6.68 8.70 8.75 5.40 6.05 0.453
R CRP3 1.75 6.73 8.77 8.55 5.55 6.05 0.449
R CRP3 1.89 6.72 8.23 8.30 5.50 6.35 0.448
R CRP4 1.09 5.85 21.34 2.44 0.57 1.59 0.079
R CRP4 0.92 5.88 21.22 2.58 1.152
2.10 0.095
R CRP4 0.90 5.91 22.02 2.38 0.64 1.70 0.081
STS-CTR 2014 Analysis of CRPs – January 2015 9/45
Lab Code
Product Nicotine pH Moisture NNN NNK NAT NAB
% % µg/g µg/g µg/g µg/g
S CRP1 1.03 7.88 50.88 0.59 0.18 0.47 0.023
S CRP1 1.02 7.87 52.66 0.59 0.18 0.48 0.022
S CRP1 1.05 7.87 51.59 0.59 0.19 0.48 0.023
S CRP2 1.50 7.70 54.06 1.64 0.39 1.71 0.122
S CRP2 1.50 7.71 54.16 1.65 0.40 1.59 0.130
S CRP2 1.49 7.70 53.64 1.68 0.41 1.65 0.128
S CRP3 2.42 6.80 10.12 6.76 3.19 5.03 0.286
S CRP3 2.47 6.81 9.82 6.87 3.22 5.09 0.294
S CRP3 2.46 6.73 10.06 6.67 3.15 4.77 0.282
S CRP4 1.18 5.73 22.57 1.53 0.34 1.07 0.036
S CRP4 1.23 5.70 22.40 1.61 0.35 1.01 0.037
S CRP4 1.21 5.73 21.80 1.73 0.38 1.07 0.045
1 Eliminated as a Levene’s Test outlier prior to the r&R calculations.
2 Eliminated as an obvious outlier prior to conducting Grubbs’ Test. Reps 1 and 3 were retained.
5. Data – Statistical Analysis
A statistical analysis was conducted in basic conformance with ISO 5725-2:1994 and ISO/TR
22971:2005. A summary of the results from outlier detection and the calculated results for
repeatability (r) and reproducibility (R) are given below in sections 5.1 and 5.2, respectively.
Even though ISO 5725-2:1994 does not suggest calculation of z-scores, z-scores are presented
in section 5.3 so that the participating laboratories would have an additional measure of their
performance compared to their peers. Section 5.4 describes the stability analysis conducted
using data from the 2010-2014 studies. Raw data plots that include all replicates, without
removal of outliers, are given in Appendix A
5.1 Exclusion of Outliers
Procedures outlined in ISO 5725-2:1994 and ISO/TR 22971:2005 were generally used for the
exclusion of outliers. An adaptation of Levene’s Test was used for eliminating laboratories
with overly large repeatability standard deviations and Grubbs’ Test was used to eliminate
laboratories with outlying mean values.
ISO 5725(2) also recommends the use of Mandel’s h and k plots. Mandel’s h statistic is the
same as the statistic used in Grubbs’ Test. Similarly Mandel’s k statistic, associated with
within lab standard deviation, is statistically equivalent to the c-value calculated in Cochran’s
Test ( cnk labs ). However, the critical values associated with Mandel’s h and k statistics do
not make allowance for multiple testing and can therefore, give a false impression of
statistical significance. Thus, Mandels’s h and k statistics do not add fundamentally new
information and may lead to incorrect conclusions. For those reasons, we do not include
Mandel’s h and k plots.
The intent of ISO 5725-2:1994 is to eliminate outliers that exceed a 1% critical value. This
was accomplished by an adaptation of Levene’s Test. Levene’s Test is preferable to
Cochran’s Test, which is recommended in ISO 5725-2:1994, because of Cochran’s Test’s
extreme sensitivity to deviations from normality. Grubbs’ Test and an adaptation of Levene’s
Test were applied at the standard nominal 1% significance level to determine outliers and the
results are shown in Table 4. Levene’s Test is mentioned in ISO/TR 22971:2005 as an
STS-CTR 2014 Analysis of CRPs – January 2015 10/45
alternative to Cochran’s Test. However, Levene’s Test does not directly apply without
adaptation. For more details, see the footnote below.5
Table 4: Outliers among 2014 Data
Analyte Product Levene’s Outliers Lab Grubbs’ Outliers Lab
pH CRP2 F −
The (-) symbol indicates an outlier was not detected.
As noted above, we do not recommend the use of Cochran’s Test; however we recognize that
it is the approach that has been traditionally employed. So, for comparison, we contrast
Cochran’s Test to Morton’s adaptation of Levene’s Test and show the results in Appendix C.
5.2 Calculation of Repeatability (r) and Reproducibility (R)
After removal of outlying data based on numerical data consistency methods (Grubbs’ Test,
Levene’s Test), the final repeatability and reproducibility (r & R) results were calculated. The
r & R results are shown in Table 5 for the 2014 WG4 study. Since this test was conducted to
evaluate the stability of the CRMs, it should be understood that the r&R results reflect both
laboratory variability and product consistency. That is always true to some degree, but is
particularly true in studies such as this in which the product stability is being evaluated and
may play a non-negligible role in the results.
Table 5: Repeatability (r) and Reproducibility (R) for 2014 WG4 Study
Parameter Product No of
Labs* Mean
Repeatability Reproducibility
r % r R % R
Nicotine (%) CRP1 11 0.95 0.113 11.9% 0.249 26.2%
Nicotine (%) CRP2 11 1.25 0.123 9.8% 0.316 25.3%
Nicotine (%) CRP3 11 2.13 0.085 4.0% 0.445 20.9%
Nicotine (%) CRP4 11 1.08 0.166 15.3% 0.287 26.5%
pH CRP1 11 7.82 0.091 NA 0.576 NA
pH CRP2 10 7.64 0.029 NA 0.290 NA
pH CRP3 11 6.74 0.068 NA 0.262 NA
pH CRP4 11 5.88 0.075 NA 0.252 NA
Moisture (%) CRP1 11 51.2 1.96 3.8% 4.60 9.0%
Moisture (%) CRP2 11 54.2 0.54 1.0% 1.50 2.8%
Moisture (%) CRP3 11 9.10 0.66 7.3% 2.51 27.7%
Moisture (%) CRP4 11 22.0 1.16 5.3% 2.03 9.3%
NNN (µg/g) CRP1 11 0.66 0.087 13.2% 0.143 21.7%
NNN (µg/g) CRP2 11 1.85 0.108 5.8% 0.393 21.3%
5 Levene’s Test is commonly used to determine if each of several subpopulations have the same variance. Since
it was designed to test for overall differences, not to determine if the largest variance is significantly greater than
the others, some adaptation is necessary to use the approach to eliminate laboratories whose within lab variation
is too large. Levene’s test was adapted to this purpose by Morton, who presented the approach utilized in this
report at the 2014 CORESTA Congress (Quebec, Canada, presentation ST28, October 14, 2014). Specifically,
the approach taken here is a two-step process with a lab being eliminated as an outlier if both steps are
statistically significant. First, Levene’s Test was run at a nominal -level of 0.02. Second a comparison of the
largest variance to the remaining variances is carried out at a one-sided nominal level of =0.01/number of labs.
Dividing by the number of labs is to account for multiple testing, since it is not known a priori which lab will
have the largest variance. Simulation studies were carried out by Morton and presented at the 2014 CORESTA
Congress and these results demonstrated that this process has an overall -level near 0.01 and is robust to
deviations from normality.
STS-CTR 2014 Analysis of CRPs – January 2015 11/45
Parameter Product No of
Labs* Mean
Repeatability Reproducibility
r % r R % R
NNN (µg/g) CRP3 11 8.29 0.741 8.9% 2.698 32.5%
NNN (µg/g) CRP4 11 2.02 0.236 11.7% 0.677 33.4%
NNK (µg/g) CRP1 11 0.20 0.027 13.2% 0.045 21.8%
NNK (µg/g) CRP2 11 0.44 0.036 8.0% 0.098 22.2%
NNK (µg/g) CRP3 11 4.34 0.490 11.3% 1.951 44.9%
NNK (µg/g) CRP4 11 0.46 0.104 22.7% 0.174 37.9%
NAT (µg/g) CRP1 11 0.52 0.054 10.5% 0.137 26.6%
NAT (µg/g) CRP2 11 1.76 0.096 5.4% 0.419 23.8%
NAT (µg/g) CRP3 11 5.76 0.697 12.1% 1.540 26.7%
NAT (µg/g) CRP4 11 1.31 0.267 20.3% 0.578 44.0%
NAB (µg/g) CRP1 11 0.031 0.006 19.8% 0.020 65.8%
NAB (µg/g) CRP2 11 0.149 0.012 7.8% 0.060 40.0%
NAB (µg/g) CRP3 11 0.386 0.039 10.2% 0.198 51.2%
NAB (µg/g) CRP4 11 0.061 0.015 24.8% 0.037 60.4%
*This is the number of laboratory data sets included after removal of outliers.
NA = Since pH is not a proportional scale, it is not appropriate to calculate % r or % R.
5.3 Calculation of Z-Scores
Although calculation of z-scores is not suggested in ISO 5725-2:1994, z-scores were
calculated so that the participating laboratories could compare their results to those of their
peers. It is expected that most of the data should fall within the range of ±2. A final summary
table of z-scores is presented in Table 6 for the 2014 WG4 study. Outliers detected with
Levene’s test were removed prior to the calculation of the z-scores and are given as “Outlier”
in the tables.
Table 6: Z-Scores for 2014 WG4 Study
Product Lab Nicotine pH Moisture NAB NAT NNK NNN
CRP1 A 0.65 0.52 -0.78 1.14 0.94 1.51 0.05
CRP1 F -1.94 -0.19 0.15 0.18 0.08 0.07 0.68
CRP1 G 0.67 0.64 0.85 1.25 1.70 0.69 1.99
CRP1 I 0.08 0.62 0.61 -0.01 0.21 0.74 0.59
CRP1 L 1.01 0.41 -0.26 0.84 -0.55 -0.17 -1.23
CRP1 M 0.49 0.74 -1.96 -0.11 -0.02 0.61 -0.41
CRP1 O -0.92 -1.22 1.97 -2.06 0.10 -0.20 -0.45
CRP1 P -0.36 -2.41 -0.08 0.37 -1.54 -1.74 0.79
CRP1 Q 0.57 0.87 -0.44 -0.76 -1.21 0.52 -0.42
CRP1 R -1.31 -0.21 -0.40 0.32 1.14 -0.36 0.01
CRP1 S 1.05 0.25 0.34 -1.16 -0.84 -1.67 -1.59
CRP2 A 0.37 0.25 1.14 1.12 1.12 1.29 -0.26
CRP2 F -0.68 Outlier1
0.48 -0.34 0.06 0.44 0.11
CRP2 G 0.04 0.64 0.82 1.85 1.67 1.08 1.76
CRP2 I 0.00 0.35 -1.32 0.45 0.51 0.96 0.94
CRP2 L 0.29 0.48 0.53 -0.04 -0.45 -0.20 -0.77
CRP2 M 0.56 0.61 0.45 0.02 -0.03 0.33 -0.47
CRP2 O -1.00 -1.49 0.37 -1.82 -0.11 0.22 -1.22
STS-CTR 2014 Analysis of CRPs – January 2015 12/45
Product Lab Nicotine pH Moisture NAB NAT NNK NNN
CRP2 P -0.34 -1.88 -2.26 0.20 -1.94 -1.37 1.29
CRP2 Q 0.04 1.09 0.21 -0.64 -0.78 0.06 0.20
CRP2 R -1.60 -0.68 0.12 0.28 0.72 -1.49 -0.21
CRP2 S 2.31 0.64 -0.54 -1.08 -0.77 -1.32 -1.38
CRP3 A 0.39 -1.16 0.20 0.18 1.33 0.54 0.24
CRP3 F 0.11 -2.08 -0.22 -0.20 0.24 0.71 0.31
CRP3 G 0.16 -0.25 -0.15 1.59 1.47 -0.58 1.22
CRP3 I 0.00 1.10 -1.79 0.16 -0.40 -0.01 0.48
CRP3 L 0.49 -0.34 -0.64 0.22 0.07 -0.15 0.34
CRP3 M 0.15 1.03 1.38 0.27 -0.12 -0.34 0.06
CRP3 O -1.12 -0.11 1.39 -1.94 -1.42 -1.42 -2.21
CRP3 P -0.54 0.99 -0.95 0.67 0.42 0.99 0.46
CRP3 Q 0.29 0.77 0.26 -0.44 -0.79 0.28 0.45
CRP3 R -1.96 -0.36 -0.56 0.91 0.77 1.67 0.26
CRP3 S 2.02 0.41 1.08 -1.42 -1.56 -1.69 -1.62
CRP4 A -0.12 0.22 0.64 0.65 0.17 0.07 0.08
CRP4 F -1.38 -2.11 0.98 -0.56 -0.59 -0.14 -0.59
CRP4 G -0.13 0.49 -1.18 0.83 0.21 -0.37 0.91
CRP4 I 0.92 0.18 0.01 -0.04 0.13 0.55 0.57
CRP4 L -0.11 0.60 2.01 0.17 -0.68 -0.43 -0.39
CRP4 M 0.02 0.41 -0.34 -0.25 -0.37 -0.66 -0.81
CRP4 O -1.23 0.91 -0.60 -1.07 0.33 0.20 -0.79
CRP4 P 1.51 0.60 -1.42 0.65 0.35 0.28 0.64
CRP4 Q 0.41 0.49 0.20 -0.52 -0.69 -0.29 0.21
CRP4 R -1.22 0.03 -0.72 1.89 2.52 2.44 1.90
CRP4 S 1.34 -1.81 0.42 -1.75 -1.38 -1.67 -1.73
1. Outliers were determined as described in Section 5 and were removed prior to the calculation of the z-
scores.
5.4 CRP Stability Assessment
The data were evaluated for stability by examining the results from the 2010-2014 WG4
studies for trends over time, for all analytes. Even though the stability trend analysis,
described below, was the primary evaluation of stability, the data were also graphically
evaluated for changes in between-lab variability over the study years. As mentioned in the
introduction, the 2010 study did not specify methods of analysis and the 2011 study was the
first study year initiated to assess stability of the CRPs where methods were specified.
However, after evaluating the data for all five years, it was determined that the 2010 data
were consistent with the data from later years and were included in the stability analysis.
The stability trend analysis included two factors: year of analysis and laboratory. To account
for potential year-to-year correlation in a laboratory’s results, the laboratory variable was
treated as a random factor.
There is also a potential need to account for multiplicity of testing. That is, if four statistical
tests are each carried out with a 5% chance to falsely declare a time trend, then there is just
under a 20% chance that at least one of them will give a false indication of a time trend even
in the ideal case of absolute stability. In this instance there are four reference products and
seven separate analytes, thus twenty-eight comparisons in all. The approach taken was to
correct for testing multiplicity across products for each analyte and not across all analytes.
STS-CTR 2014 Analysis of CRPs – January 2015 13/45
The correction employed was to allow for four tests being conducted not all twenty-eight.
That meant that each test was judged statistically significant if the associated p-value was less
than 5%/4 or 1.25%. Therefore, there is roughly a 5% chance that each analyte will be
declared to show a time trend on one or more of the CRPs by chance alone.
Graphs of the means for all years and all analytes are shown in Appendix B. These graphs
include the complete data set, without removal of outliers, to give the reader a perspective
both on the retained points and those that were excluded as outliers. Table 7 summarizes the
trend analysis of each analyte and each product. Since the testing was done on the mean
values, Grubbs’ Test outliers were identified and removed prior to analysis, but no outliers
were removed for excessive intra-laboratory variation. There were not any statistically
significant trends in the data, though it should be noted that in most years, for most analytes,
there is quite a lot of lab-to-lab variation. This limits the ability to detect statistically
significant trends.
The previous year’s analysis6 showed a slight upward trend in moisture for CRP3. This year
that trend fell just short of statistical significance. The reason for this appears to be because
the 2014 values are generally comparable to 2013 instead of continuing the upward trend.
There is still (on average) statistically higher CRP3 moistures in 2013 and 2014 than in 2010
and 2011.
Table 7: Summary of Stability Analysis
Product Variable Estimated Slope p-value1
CRP1 Nicotine -0.0014 81.0%
CRP2 Nicotine -0.0066 39.8%
CRP3 Nicotine -0.0042 74.3%
CRP4 Nicotine -0.0174 8.1%
CRP1 pH -0.0273 7.0%
CRP2 pH -0.0149 7.7%
CRP3 pH -0.0124 8.4%
CRP4 pH -0.0062 54.5%
CRP1 Moisture2
-0.2948 5.6%
CRP2 Moisture -0.0267 58.4%
CRP3 Moisture 0.1914 1.7%
CRP4 Moisture 0.0774 73.4%
CRP1 NNN -0.0025 81.3%
CRP2 NNN -0.0060 77.3%
CRP3 NNN 0.0632 45.0%
CRP4 NNN -0.0006 98.3%
CRP1 NNK3
-0.0043 1.6%
CRP2 NNK -0.0032 38.6%
CRP3 NNK4
0.0849 8.8%
CRP4 NNK 0.0043 51.9%
CRP1 NAT -0.0073 29.0%
CRP2 NAT -0.0266 24.8%
CRP3 NAT -0.0115 87.7%
CRP4 NAT 0.0104 61.7%
CRP1 NAB4
0.00048 61.0%
6 STS Technical Report: CORESTA Reference Products 2012 and 2013 Analysis, July 2014.
STS-CTR 2014 Analysis of CRPs – January 2015 14/45
Product Variable Estimated Slope p-value1
CRP2 NAB -0.00093 63.1%
CRP3 NAB -0.00145 78.2%
CRP4 NAB4
-0.00013 88.7%
1. P-values are declared statistically significant if p < 1.25%. 1.25% is derived by dividing the
nominal 5% level by 4 to account for testing multiplicity per analyte.
2. Calculated omitting the Lab E result in 2010.
3. Calculated omitting the Lab P result in 2013.
4. Calculated omitting the Lab K result in 2012.
6. Data Interpretations
Though the trend of increased average moisture content noted in last year’s report for CRP3 is
no longer statistically significant, the current average moisture values are still somewhat
higher than those shortly after the product was manufactured. Still this increase was marginal
and equated to an increase of approximately 1% when calculated as an average for all labs.
CRP3 has the lowest moisture content so this product would be most likely to absorb water
over time as compared to the other CRPs. As mentioned earlier, results are presented on an
as-is basis; however, this negligible shift in moisture for CRP3 would not be evident in the
other analyses considering the negligible change in moisture in comparison with lab-to-lab
method variability.
In examining the mean data plots for all years (Appendix B), a number of potential outliers
are evident; therefore, the stability analyses were conducted both with and without the outliers
included, but only shown in the table above with the outliers excluded. There were some
indications of changes in between-lab variability over the study years. Two notable examples
of increased variability between labs are pH for CRP3 and moisture for CRP1. Both have
increased in variability across the reporting period (2010 Lab E was dropped as an outlier for
CRP1).
As reported last year, the inter-laboratory variability for CRP4 moisture has been very high in
some years, such as 2012, but was improved for 2014. This year, one laboratory reported that
a package of CRP4 that was recently received from NCSU appeared to be uncharacteristically
dry. The laboratory confirmed that the moisture content was approximately half of what it
should have been and retested another package (all analyses). Although the original package
of CRP4 was clearly defective a smaller loss of moisture would not have been readily
apparent and any differences in the moisture content determined would have likely been
attributed to method variability.
Due to the product type, CRP4 is the least homogenous of the four CRPs. This suggests that
sample preparation (particle size reduction) and how tobacco aliquots are removed for testing
may be important components of the variability seen within labs for CRP4. Labs reported a
range of techniques ranging from cryogrinding an entire package to simply removing
individual aliquots for testing. Variability may potentially be reduced by generating a
separate homogeneous composite sample for each replicate and removing the aliquot for the
replicate from that homogeneous composite. This could be accomplished by either
cryogrinding an entire package or using a razor blade to cut and mix the entire contents of a
package before removing the sample for analysis. We recommend this approach in the future.
Overall, both nicotine and TSNAs are shown to be stable over the five study years, for all
CRPs. This is not unexpected as both nicotine and TSNAs should be stable in these products
when the products are maintained at −20°C.
STS-CTR 2014 Analysis of CRPs – January 2015 15/45
Based on the results taken in their entirety, the conclusion of this study is that all four CRPs
are suitable for continued use as reference products. Lack of any clear and significant trends
in the stability results indicate that storage at −20°C is an appropriate storage condition for the
four reference products. Furthermore, since the changes in moisture, if any, are small relative
to the inherent method reproducibilities, it is unlikely that they will have an easily detectable
effect on other results.
7. Recommendations
The Smokeless Tobacco Sub-Group recommends that WG4 continue to monitor stability of
the CRPs on an annual basis. As stated earlier, data generated in 2010 will serve as the
baseline for future WG4 stability analyses. A formal analysis of product stability should be
conducted with data obtained from the next WG4 study in the spring of 2015. For future
analyses, we are recommending that all CRPs be obtained from NC State University just prior
to the study to mitigate differences in long term storage conditions by the participating
laboratories. Furthermore, we are recommending that CRP4 be prepared for analysis by
either cryogrinding an entire package or by finely cutting and mixing an entire package with a
razor blade before removing an aliquot for analysis.
STS-CTR 2014 Analysis of CRPs – January 2015 30/45
APPENDIX B: CRP Stability Assessment, Mean Data Plots
STS-CTR 2014 Analysis of CRPs – January 2015 44/45
APPENDIX C: Results Using Cochran’s Test for Outlier Detection
As discussed above, we do not recommend the use of Cochran’s Test; however we recognize that it is
the approach that has been traditionally employed. So, for comparison, we contrast Cochran’s Test to
Morton’s adaptation of Levene’s Test and show the Results in APPENDIX B.
The outliers identified when Cochran’s Test is employed are given in the following table. Note that
the Grubbs’ Test outliers show slight differences as well because Grubbs’ Test was applied after the
removal of intra-lab variation outliers.
Outliers among 2014 data using Cochran’s Test followed by Grubbs’ Test (for comparison
purposes only as this is not the recommended approach)
Analyte Product Cochran’s Outliers -
Lab Grubbs’ Outliers -
Lab
Nicotine CRP2 R,P −
Nicotine CRP4 F,R −
pH CRP2 F,P −
pH CRP4 F S
NAB CRP3 O −
NAT CRP3 O −
NAT CRP4 R −
NNN CRP2 F −
NNN CRP3 O S
Moisture CRP3 A −
Moisture CRP4 F −
The corresponding r&R table is given below when using Cochran’s Test followed by
Grubbs’ Test (for comparison purposes only as this is not the recommended approach).
Parameter Product No of
Labs* Mean
Repeatability Reproducibility
r % r R % R
Nicotine (%) CRP1 11 0.95 0.113 11.9% 0.249 26.2%
Nicotine (%) CRP2 9 1.27 0.035 2.7% 0.280 22.0%
Nicotine (%) CRP3 11 2.13 0.085 4.0% 0.445 20.9%
Nicotine (%) CRP4 9 1.11 0.064 5.8% 0.223 20.1%
pH CRP1 11 7.82 0.091 NA 0.576 NA
pH CRP2 9 7.66 0.020 NA 0.231 NA
pH CRP3 11 6.74 0.068 NA 0.262 NA
pH CRP4 9 5.92 0.044 NA 0.074 NA
Moisture (%) CRP1 11 51.2 1.96 3.8% 4.60 9.0%
Moisture (%) CRP2 11 54.2 0.54 1.0% 1.50 2.8%
Moisture (%) CRP3 10 9.0 0.43 4.7% 2.60 28.8%
Moisture (%) CRP4 10 21.9 0.73 3.3% 1.89 8.6%
NNN (µg/g) CRP1 11 0.66 0.087 13.2% 0.143 21.7%
NNN (µg/g) CRP2 10 1.84 0.078 4.2% 0.408 22.1%
NNN (µg/g) CRP3 9 8.69 0.522 6.0% 0.960 11.1%
NNN (µg/g) CRP4 11 2.02 0.236 11.7% 0.677 33.4%
NNK (µg/g) CRP1 11 0.20 0.027 13.2% 0.045 21.8%
NNK (µg/g) CRP2 11 0.44 0.036 8.0% 0.098 22.2%
NNK (µg/g) CRP3 11 4.34 0.490 11.3% 1.951 44.9%
STS-CTR 2014 Analysis of CRPs – January 2015 45/45
Parameter Product No of
Labs* Mean
Repeatability Reproducibility
r % r R % R
NNK (µg/g) CRP4 11 0.46 0.104 22.7% 0.174 37.9%
NAT (µg/g) CRP1 11 0.52 0.054 10.5% 0.137 26.6%
NAT (µg/g) CRP2 11 1.76 0.096 5.4% 0.419 23.8%
NAT (µg/g) CRP3 10 5.83 0.498 8.5% 1.391 23.9%
NAT (µg/g) CRP4 10 1.27 0.147 11.6% 0.333 26.3%
NAB (µg/g) CRP1 11 0.03 0.006 19.8% 0.020 65.8%
NAB (µg/g) CRP2 11 0.15 0.012 7.8% 0.060 40.0%
NAB (µg/g) CRP3 10 0.40 0.024 6.1% 0.159 39.7%
NAB (µg/g) CRP4 11 0.06 0.015 24.8% 0.037 60.4%
*This is the number of laboratory data sets included after removal of outliers.
NA = Since pH is not a proportional scale, it is not appropriate to calculate % r or % R.
As noted by Morton in a 2014 presentation at the CORESTA Congress, Cochran’s Test tends to
identify more outliers than the adaptation of Levene’s Test. As a general rule, the calculated
repeatability tends to be lower when Cochran’s Test is employed. The estimated reproducibility tends
to be similar with the two approaches, but in some cases can be quite different as illustrated here. For
example, in examining NNN in CRP3, after Lab O is dropped as a Cochran’s Test outlier, then Lab S
is identified as a Grubbs’ Test outlier and the reproducibility is much less than with both of those Labs
included in the analysis.
top related