Robert E. Gladd, James M. LittlefieldInternational Technology Corporation, Oak Ridge Laboratory

alculations of Statistical Process Control(SPC) limits assume representative ran-dom sampling of process output for the

applied to the equipment when warranted to assurecontinuing accuracy of output.

At a data accrual rate of one QC result per detec-tor per collection interval, the practical imperativesof counting instrument control (i.e., procedural,regulatory, and contractual) preclude the use of tra-ditional SPC random sampling techniques. One sim-ply cannot await the amassing of several monthsworth of data before sampling the database and gen-erating statistical control limits, "flying blind" in theinterim. When an operating detector is subjected tosignificant electronic and/or mechanical adjustmentor repair, the continued use of prior parameter esti-mates may be improper; new operating conditionsproperly mandate the collection of a new subset ofdata for QC evaluation to assure effective instru-ment control.

Accelerating the data collection process by per-forming multiple QC counts per detector per day isimpractical given the normal production schedulingdemands of a laboratory and, more importantly,would be inappropriate owing to the inevitable resul-tant biasing of the data by transient environmentalvariables. Clearly, a routine periodic QC sourcecount procedure with provisions for contingency re-counting is the most practical method of acquiringthe necessary instrument reliability QC data.

The Oak Ridge Laboratory of International Tech-nology Corporation (IT/ORL) operates a variety ofalpha, beta, and gamma counters, and presently

Where data accrue rapidly enough for the collectionof randomly sampled subsets to be practicable, ran-dom sampling is unquestionably the method ofchoice for the computation of statistical control indi-ces. In a manufacturing environment where outputis measured in hundreds or thousands of units perhour or day, randomly drawn quality control (QC)data sets are conveniently obtained and SPC limitsmay be derived in a timely fashion.

In the counting room of the environmental ra-dioanalyticallaboratory, however, instrument reli-ability QC process outputs are typically obtained bythe periodic placement of a radioactive standard ineach instrument detector chamber and counting thedisintegrations over a specified interval. Thismethod produces a single "total counts" QC resultper detector per data collection period. Radioactivedisintegrations being characteristically stochasticphenomena, one expects a normally distributed vari-ability of source counts over time. The quality assur-ance objective is to insure that counting instrumentsexhibit an unbiased, random, expected variabilitythat may be taken into account in the error termcomputations of analytical results, and that anyemergent detector performance trends become read-ily apparent so that corrective adjustments may be

Reprint of "Prom the Counting Room", Vol. 1, No.4, 1990

Radioactivity & Radiochemistry

The Counting Room: Special Edition

Figure 1 GEl/Ol Bi-207 source control plot.

analyst with immediate NDEV feedback as data is en-tered into the system. Input data are rapidly checkedfor the most common forms of keyboard input er-rors, such as invalid dates, detector IDs, unauthor-ized user IDs, exact input duplication, and grosslyout-of-range count data stemming from inadver-tently mis-keyed or omitted digits. Statistics are com-puted (for N ~ 2), and the count input data are con-verted to NDEVs and returned to the user in aresults screen window within a few seconds. ND EV sfalling outside :t3 sigma require recounting of thesource, and a second consecutive >ABS(3) sigmaoutlier in the same direction mandates corrective ac-tion. NDEV values falling within the "warning"zones [ABS(2 s NDEV s 3)] are monitored for evi-dence of "runs" indicative of detector trend, or exces-sive sequential swings denoting inordinate volatility.

The initial sequential sample of N = 20 accruesover a period of approximately one month (sourcechecks are not performed on weekends or holidayswhen the instruments are idle). Tentative statisticalcontrol indices are calculated during this data collec-tion interval where 1 < N < 20 to give the analyst anongoing indication of instrument performancewhile awaiting the requisite twenty data values withwhich to generate the "permanent" detector seriesparameter estimates. With the addition of eachN < 20 result, the tentative statistics are recalculatedso that the indices converge toward the final N = 20means and sigmas. The final parameter estimates arewritten to disk from which they are referenced in

monitors the performance of eight instruments con-taining a total of seventy-seven detectors. In Januaryof 1989, an in-house developed computerized sys-tem of count QC data acquisition and SPC analysiswas installed to assist in complying with the require-ments of instrument calibration QC ("IQCDATX'and "IQCSTATS"). IT/ORL QC Procedure specifiesthe use of a minimum of twenty data points for thederivation of SPC parameter estimates and, of neces-sity, a sequential sample of the initial twenty sourcecount values for each detector "series" is used to cal-culate the means and sigmas and construct upperand lower control limits. Individual data points areexpressed as normalized deviates ("NDEV") fromMEAN = 0 and SIGMA = 1, with warning limitsLWL = -2 and UWL = +2, and control limitsLCL = -3 and UCL = +3 (sigmas). Beta and gammasource counts are corrected for temporal source de-cay before statistics are generated. Source back-ground counts are also entered and plotted as nor-malized deviates. NDEV results are returned by theprogram in both tabular and high-resolution controlchart scatterplot format. The user may request of theIQCSTATS module tables, scatterplots, or both, foreither an individual detector or for all instruments.Additionally, the program contains an option for adaily "Last Entry" report which issues a printout ofthe input data and statistical indices for the most re-cent entry in the system for each detector.

The recently completed, fully integrated, interac-tive prototype of the IQCDATA module provides the

28 Reprint of "Prom the Counting Room", Vol. I, No.4, 1990

"Radioanalytic Counting- Instrument Reliability: An Interim Assessment of a Computer-Assisted Statistical Process Control Approach Under Development"

Robert E. Gladd, James M. Littlefield

Figure 2b LB-5100(TA/OO2 alpha, renormalized control plot. Figure 3 LB-5100!TAjOO2 beta control plot.

subsequent operations of the system. These perma-nent statistics may be cleared from the lookup fileupon user request to force recalculation in the eventthat data entry errors are found and corrected by apassword-authorized editor. A "series" continuesfrom the date a detector is placed in service untilsuch time as significant corrective measures are re-quired, at which point the adjustments and/or re-pairs undertaken are documented and the next seriesis assigned an ID for the process to begin anew.

At this writing, the IQC system has been in op-eration for sufficient duration to have acquired de-tector series datasets numbering from approximatelyfifty to over one hundred periodic source check re-sults per detector/series, enough data to provide forthe assessment of the robustness of the IQCSTATSsequential "convenience" sample-derived control sta-tistics. Detectors were chosen randomly from eachinstrument and, with the aid of a random numbertable, N = 20 datasets were drawn from the totalavailable counts for each chamber and comparedwith the original sequential sets for the respectivechamber. The data were entered into a SAS- PC TM

(Statistical Analysis System) program which per-formed independent sample T-tests and "Homoge-neity of Variance" F-tests on the original datasets vs.the randomly drawn results. The nul hypothesis isthat the means and variances of the sequential andrandom subsets do not differ significantly (i.e., thatthey are "statistically equivalent") and that by impli-cation, the method of series sequential sampling of

detector data is indeed serviceable for the derivationof instrument SPC indices.

The results, summarized in Appendix A, are atonce reassuring and cautionary. Of the eight instru-ments, si:x were subjected to statistical re-examina-tion. One proportional alpha/beta counter is a recentacquisition and lacks sufficient operating history toprovide a large enough "N" for meaningful randomsampling. A 32 chamber alpha counter is monitoredby a pulse count calibration procedure in which thesigma is a fixed percentage of the series mean ratherthan the usual square root of the adjusted (N -1)mean squared deviation, rendering a conventionalT -test inapplicable.

Eight initial T -tests were performed on the si:x in-struments reviewed. Two counters are alpha/betacounters, and the T -tests were applied to both alphaand beta source counts (Figures 2a, 2b, and 3). Ofthe eight T -tests, seven reported statistical equiva-lence of parameter estimates (i.e., "nonsignificantdifferences at 0.05 probability level"). Germaniumdetector GEl (Figure 1), while nonsignificant atp = 0.05, was a bit close for comfort, owing to threeextreme source count values in the initial sequentialsample. The initial series count, which was the mostextreme, was bypassed during the random sampling.The remaining two, both well below the mean, wereincluded by the random number table, resulting in anegative shift in the mean and a contraction in thesigma from sequential to random sample. The GEldata were subjected to another round of random

29Reprint of "From the Counting Room", Vol. 1, No.4, 1990

Radioactivity & Radiochemistry

The Counting Room: Special Edition

dent on the alpha plot, one that merits testing for sig-nificance of slope.

Using the SAS regression procedure PROC REG(Appendix A), a test of the simple linear regressionmodel, "y = (a)(X) + b:' for the LB51A/002 datasetyields a statistically significant slope of -4.453262(Prob> ITI, 0.0001) with a model Adjusted R-Squared of 0.2489. Formal statistical analysis con-firms what is readily apparent to the eye; a decliningalpha source count trend is occurring for this detec-tor, one that merits closer scrutiny. Confirmation ofthe statistical significance of the negative trend is ob-tained by deleting three early series results (02/10,02/14,02/15), all with disproportionately highNDEV values, and testing the remaining data via thesame SAS regression model. The revised model,while diluted somewhat in strength, remains statisti-cally significant with a slope of -3.281061(Prob> ITI, still 0.0001) and a model Adjusted R-Squared of 0.1739.

Although LB51A/002 indisputably manifests anegative trend, a concern with "statistical signifi-cance" should be tempered with an awareness of"practical significance:' The Adjusted R-Square sta-tistic, a correlative measure of model predictivestrength, indicates that no more than 24.89% of thealpha source count variability is accounted for by therelationship between the source count results andtheir chronological positions. Experimental removalof the possibly anomalous early extreme values low-ers the predictive power of the linear trend model to17.39%. This "trend", while not to be casually dis-missed, is in fact rather loosely coupled. Leaning tooheavily upon textbook statistical trend-line analysishas its perils, as many a former Wall Street brokerwould unhappily attest. Continued evidence of de-tector trend does, however, warrant inspection of theequipment for contamination or damage, as well asinspection of the source material. Several of our ger-manium detectors recently exhibited a decliningsource count trend (Figure 1); the cause was foundto be a nearly invisible crack in the source casing.

Effective radio analytic counting instrument con-trol is accomplished through a blend of adaptiveSPC empirical methodology and expert judgement.While traditional statistical procedures are usefulanalytical tools, a proper perspective is essential.

sampling, and a second T -test yielded similar mar-ginally "nonsignificant" test statistics. The GEl dataillustrates vividly the impact extreme values have onprocess control statistics, particularly the normalizeddeviates. A few relatively large source count fluctua-tions in the initial twenty results diminish the magni-tudes of the NDEVs for the remaining count values,an important contextual point to keep in mind wheninterpreting the control charts.

One instrument, coded in the SAS output sum-mary (Appendix A) as "LB51X' ("LB5100/TNO02"on the Alpha Count Control Chart-Figure 2a),showed statistically significant sequential-to-randomsample alpha source count shift. This detector seriesdataset was randomly sampled a second time and T-tested again. The repeat T -test results confirmed thesignificant shift in the alpha count parameter esti-mates, raising a pertinent question; are we to con-clude that the SPC parameter estimates derived fromthe first twenty detector series alpha values are inthis instance invalid? If we replace the original meanand sigma in the IQC system lookup file with the sec-ond set of random sample-derived indices andrenormalize the daily alpha counts for this series tothese revised statistics, the revamped control chart(Figure 2b) exhibits an upward shift in the scatter.This is with a barely perceptible expansion of thescatter owing to the slight contraction in the sigma.The "Mean Deviation:' an index of overall detectorbias, rises from -0.78 to -0.02, the new figure indicat-ing that the detector evinces essentially no alphacount reproducibility vertical-axis bias when viewedin the context of the random-sample statistics, animplicit reminder of the inherent methodologicalstrength of the random sample.

Under the revised SPC limits, one early alphadata point (fifth in the series, on 02/14/89, originallyatNDEV = +2.74) now becomes, post-hoc, an "out-lier" at NDEV = +3.71. A comparison of the originalvs. the revised plot reveals a reduction, from seven totwo, in normalized deviation points lying betweenthe warning and control limits (i.e.,ABS (2 < NDEV < 3), one of which is now our retro-active outlier). The "warning zone" points on theoriginal plot were, with the exception of the 02/14 re-sult, all <LWL. A declining count trend is visibly evi-

30 Reprint of "From the Counting Room", Vol. 1, No.4, 1990

Robert E. Gladd, James M. Littlefield

each new set of source and background check data.The "Mean Normalized Deviation" and its standarddeviation are calculated for the entire series dataset.A T -Test is performed to reveal whether or not theoverall series Normalized Mean differs significantlyfrom zero (zero being, you will recall, the normal-ized mean calculated from the initial N = 20 set).Again, a 0.01 Confidence Level is used, and the T-Test is computed under the conservative assumptionof "unequal variances;' a method requiring the com-putation of "approximate degrees of freedom" (ADFon the plot). T -scores falling outside the critical val-ues are noted on the plot with a "T -Test FAILED"warning. The Mean Normalized Deviation is indi-cated on the plot by a broken line. Where a detectorexhibits neither trend nor "Mean N_Dev" bias, theleast squares trend line and the population normal-ized mean "bias line" collapse to the X-axis.

Finally, a "Coefficient of Variation' (C.V.) statis-tic is returned to provide an index of relative vari-ability. Sometimes referred to as the "PercentageStandard Deviation;' this parameter estimate is sim-ply the ratio of Sigma to Mean expressed as a per-centage, enabling the analyst to quickly assess bothday-to-day and overall fluctuations of the detectorseries data. For example, where C. V. = 0.8, the inter-val from the LCL( -3) to the UCL( +3), a range of6 sigma, is equivalent to 4.8% of the mean. Use ofthe C. v: permits the user to easily ascertain both therelative volatility of the series scatter and the propor-tional magnitudes of individual sequential fluctuations.

Our empirical review of the accruing IQC data-base and analytical system finds that the sequentialsampling method employed by the software is in-deed a conceptually sound one and a practical proce-dure yielding SPC estimates of a generally robust na-ture with which to monitor, analyze, and manageinstrument performance. The study further demon-strates that, even under eventualities where originaldetector series parameter estimates are found to bestatistically unrepresentative of the detector seriescount population, adverse impact upon the accuracyof analytical results is highly unlikely. The IQC sys-tem provides the IT/ORL counting room staff withup-to-date and statistically comprehensive decisiondata for optimal equipment management.

Even in the "worst" case found in this study, that ofthe LB5100 alpha series, the difference in the sequen-tial vs. the random-sample means, while "signifi-cantly different" in terms of a formal statistical T-test, is on the order of 0.6 percent. This relative0.6% difference in the means is far less than the cus-tomary 2.5 to 5.0% "expected laboratory precisionparameters" specified for analytical results. When werenormalize the data to the random-sample-derivedmean and sigma, the scatter is shifted upward and allbut one of the individual points remain within thecontrol bounds. Further, our revisionist "outlier"must be viewed in the context of a fundamentalcharacteristic of the normal distribution; 99.7% ofnormally distributed data fall within :t3 standard de-viations of a mean value. We would therefore expect,on average, three out of 1000 points to randomly falloutside of 3 sigma. Only when a detector exhibits re-peated extreme deviations from the mean in thesame direction may we infer the presence of a sys-tematic problem. With respect to the longer-termtrend indicated by a predictively weak but "statisti-cally significant" linear slope, similar interpretivecaution is warranted. Just as transient environmentalvariables may impact a single count, so too mightseasonal environmental influences be substantive fac-tors in observed detector drift. Such trends, whileworthy of closer scrutiny, may tend to slowly flattenout and reverse in response to such subtle long-termbackground influences.

The IQC development system now a place at theIT Oak Ridge Laboratory provides analysts and man-agement with timely and, as indicated by this study,generally robust SPC indicators of detector perform-ance. System enhancements installed since our initialempirical inquiry provide the counting room staffwith a comprehensive set of statistical tools for theevaluation of instrument reliability. The IQC controlchart routine now fits a least-squares trend linethrough the scatter and reports the Regression Corre-lation Coefficient and R-Square values on the plot.The regression coefficient is tested for significanceusing the standard 0.01 Confidence Limit CriticalValues. Correlation indices falling outside the criticalvalues are noted on the plot with a warning state-ment. In a similar manner, the parameter estimatesare now continually reassessed with the addition of

Reprint of "From the Counting Room", Vol. 1, No.4, 1990 31

Radioactivity 6- Radiochemistry

The Counting Room: Special Edition

SAS Output Summary

T -Test and Regression Model Results

ITAS Oak Ridge Laboratory: IQC System Sampling Test.

T -Test Sequential vs. Random Source Check Counts. SAS 12:28 Wednesday, May 31, 1989.

Table

Detector = C2A

Variable: Counts

Table

Detector = C2B

Variable: Counts

TableDetector = GEl

Variable: Counts

T-Test *1

Table

Detector = GEl

Variable: Counts

T-Test #2

27554.75000000

27802.65000000

For HO: Variances are equal, F' = 2.18 with 19 and 19 DF

Table

Detector = L6

Variable: Counts

32 Reprint of "From the Counting Room", Vol. 1, No.4, 1990

"Radioanalytic Counting-Instrument Reliability: An Interim Assessment of a Computer-Assisted Statistical Process Control Approach Under Development"

Robert E. Gladd, James M. Littlefield

SAS Output Summary

T -Test and Regression Model Results

ITAS Oak Ridge Laboratory: IQC System Sampling Test.T -Test Sequential vs. Random Source Check Counts. SAS 12:28 Wednesday, May 31, 1989.

Table

Detector = LB51 A

Variable: Counts

T-Test *1

Table

Detector = LB51 A

Variable: Counts

T-Test *2

Table

Detector

Variable:

Table

Detector = LSCI

Variable: Counts

Table

Detector = LSCI

Variable: Counts

Reprint of "From the Counting Room", Vol. 1, No.4, 1990 33

=LB51B

Counts

Radioactivity & Radiochemistry

The Counting Room: Special Edition

SAS Output Summary, continued

T-Test and Regression Model Results

LB51 00!TAjO02: SAS Proc Reg Output, Linear Trend Regression Analyis

SAS 10:24 Tuesday, June 6, 1989.

c.v. 0.71119

Parameter Estimates

T for HO:Parameter = 0

623.464

-5.181

ParameterEstimate

24674

-4.453262

Variable

INTERCEP

DAYS

Standard Error

39.57598903

0.85953556

DF

1

1

0.0001

0.0001

Table Model: MODEL 1 Tria! Dep. Variable: SOURCE

24610

-3.281061

654..856

-4.098

Table Model: MODEL 1 Trial 2 Dep. Variable: SOURCE

ReferencesGoldin, Abraham 5., Internal Quality Control for Radioassay, USEPA

Office of Radiation Protection Programs, Washington, DC, (1983).

Kanipe, L. G., Handbook for Analytical Quality Control in Radioana-

Iytical Laboratories, TVNUSEPA Interagency Energy-Environment

Research & Development Program (EPA 600/7-77-088), Wash-

ington, DC, (1977).

Freund, R. J., Little, R C., SAS System For Linear Models, SAS Insti-

tute, Cary, NC, (1986).

Steel, R G. B., Torrie, J. H., Principles and Procedures of Statistics,

Second Edition, New York, McGraw-Hili Book Co., (1980).

Diem, K., Lentner, C., Ed., Scientific Tables, Seventh Edition, Ciba-

Geigy, Ltd., Basle, Switzerland, (1970).

R&R

Reprint of "Prom the Counting Room", Vol. 1, No.4, 199034