peer reviewed measurement uncertainty

Upload: theolen

Post on 03-Jun-2018

223 views

Category:

Documents


0 download

TRANSCRIPT

  • 8/12/2019 Peer Reviewed Measurement Uncertainty

    1/7

    Report

    Measurement Uncertaintyncertainty in the broad sense is

    no new concept in chemistry;

    analysts have always sought to

    quantify and control the accuracy of their

    results. Few analysts would dispute that a

    result is of little value without some

    knowledge of the associated uncertainty;

    clearly, without such information, inter

    pretation is impossible.

    Correct interpretation depends on a

    good assessment of accuracy. When esti

    mates of accuracy are optimistic, results

    may appear irreconcilable and may be

    overinterpreted; with unduly pessimistic

    assessments, methods may appear unfitfor a particular purpose and may be opti

    mized when it's not necessary.

    In general, different methods of estimating uncertainty will lead to different values.

    Most estimates of accuracy have been

    based on the standard deviation of a series

    of experiments or interlaboratory compari

    sons,often in association with estimates of

    bias in the form of recovery estimates.

    When individual effects are being consid

    ered, the contribution from random vari

    ability can be estimated from repeatability

    reproducibility or other precision mea

    sures.In addition separate contributions

    from several systematic or random effects

    can be combined linearly or by the root

    sum ofsquares.Finally, the way uncer

    tainty is expressed can vary substantially.

    Confidence intervals, absolute limits, stan

    dard deviations, and coefficients of variance

    are all in common use. Clearly, with so

    many possibilities for estimating and ex

    pressing such a critical parameter, a con

    sensus is essential for comparability.

    The most recent recommendation is

    that accuracy be expressed in terms of a

    quantitative estimate of uncertainty as de

    scribed in the International Standards Or

    ganization's (ISO)Guide to the ExpressionofUncertaintyin Measurement t1) and other

    measurement authorities (2,3).The guide

    is published under the auspices of several

    organizations, includingISO,the Interna

    tional Bureau of Weights and Measures

    (BIPM) the Organization for International

    and Legal Metrology (OIML) and the In

    ternational Union of Pure and Applied

    Chemistry (IUPAC)

    The document lays out a standard ap

    proach to estimating and expressing un-

    Correctinterpreta

    tion ofaccuracyen

    sures thatresultsare

    judged neitheroverly

    optimistically norun

    duly pessimistically.

    Steve EllisonLaboratory of the Government Chemist

    (U.K.)

    Wolfhard WegscheiderUniversity of Leoben (Austria)

    Alex WilliamsEURACHEM Working Group on

    Measurement Uncertainty (U.K.)

    S0003-2700(97)09035-5 CCC: $14.001997 American Chemical Society

    Analytical Chemistry News & Features, October1, 1997 607 A

  • 8/12/2019 Peer Reviewed Measurement Uncertainty

    2/7

    Report

    certainty across manyfieldsof measure

    mentand,in view ofitspedigree, is widely

    accepted by accreditation and certification

    agencies worldwide. This Report deals

    primarily with the provisions and interpre

    tation ofthisdocument, though itisrecog

    nized that different approaches are used

    in otherISOdocuments.

    Defin i t ions

    The definitionofmeasurement uncertainty

    is"aparameter associated with the resultof

    a measurementthatcharacterizesthedis

    persion ofthe valuesthatcouldreasonably

    be attributedtothe measurand"(1,4).

    Thus, measurement uncertainty describes

    a range or distribution ofpossiblevalues.

    For example,82 5describes s rangeeo

    values. Measurement uncertainty, there

    fore,differs from "error", whichisdefined

    as asingle valuethe difference between a

    resultandthe true value.The stated range must also include the

    values the measurand could reasonably

    take,on the basis of the result. That

    makes it quite different from measuresof

    precision, which give only the range

    within which the mean ofaseries of ex-

    perimentswilllie. Precision makes no

    allowance forbias;measurement uncer

    tainty includes random components and

    systematic components. Note that known

    systematic errors, or bias, should be cor

    rected for as fully as possible; failure to

    make such a correction is simply to report

    a result known to be wrong. But an uncertainty associated with each correction fac

    tor remains and must be considered. This

    consideration of systematic effects makes

    measurement uncertainty more realistic

    than measures such as standard error.

    Finally, measurement uncertainty is an

    estimate. Obviously, all statistical calcula

    tions onfinitesamples provide estimates

    of population parameters, but the estimate

    goes deeper thanthis.Devising experi

    ments that can accurately characterize

    uncertainties in method bias and other

    systematic effects is extremely difficult.For example, most derivatizations are pre

    sumed to proceed to completion. How

    certain can the analyst be ofthis?Unfortu

    nately, statistics help little; in practice the

    chemistisoften forced to make an edu

    cated estimate from prior experience

    However itiscrucialtorealize that the

    attempt must be made the correction for

    bias and the uncertainty ofthiscorrection

    factor cannot simply be ignored if compa-

    rability is to be established

    Error and uncertainty. In common parlance, the terms error and uncer

    tainty are frequently used interchange

    ably. However, several significant differ

    ences in the concepts are implied by the

    terms defined byISO(4).Errorisdefined

    as the difference between an individual

    result and the true value of the measur

    and. Error, therefore, has a single value

    for each result. In principle, an error could

    be corrected ifallthe sources of error

    wereknown,though the random partof

    an errorisvariable from one determina

    tion to the next.Uncertainty, on the other hand, takes

    the form ofarange and, if estimated for

    an analytical procedure and a defined

    sample type, may applytoall determina

    tions so described.Nopart of uncertainty

    can be corrected for. In addition, estima

    tion of uncertainty does not require refer

    ence to a true value, onlyto aresult and

    the factors that affect theresult.This shift

    in philosophy marks a concept rooted in

    observable, rather than theoretical, quan-

    Box 1. Calcu lati ng uncert aint y using ISO rules

    Rule1:Alllontributions are eombined in the formoofsandard deviationn sSDs).

    Combining as SDs allows calculating a rigorous combined SD using standard

    forms. It does not imply that the underlying distribution is or needs to benormalevery distribution has an SD. It is not perfectly rigorous to deduce

    a confidence interval from a combinedSD,but in most cases, especially when

    three or more comparable contributions are involved, the approximation is at

    least as good as most contributing estimates.

    Rule2:Uncertainties are combined according to

    in in whichu(y)is the uncertainty ofa valuey,lFu2,- the uncertainties ofthe independent parametersxh x2,... on which it depends, anddy/dXjis the

    partial differential ofywith respectto *,-.When variables are not independent,

    the relationship is extended to include a correlation term (1).

    Rule2establishes the principle of combination of root sum squares. One

    corollaryisthat small components are quickly swampedbylarger contributions,

    making it particularly important to obtain good values for large uncertainties

    and unnecessary to spend time on small components. In pictures, this looks

    like a simple Pythagorean triangle. For the uncertaintiesuxandu2,the

    combinationuccan be visualized.

    Rule3:TheSDobtained ffom Eq. . 1neds to ob multiplied dbycoverage factor

    kto obtain a range called the expanded uncertainty, which includes a large

    fraction of the distribution. For most purposes,k=2 is sufficient (2)and will

    give a range corresponding to an approximately95%confidence interval.

    Similarly,k=3 is recommended for more demanding cases.

    608 A AnalyticalChemistryyews &&eatures, October1, 1991

  • 8/12/2019 Peer Reviewed Measurement Uncertainty

    3/7

    titles. To further illustrate the difference,

    the result ofananalysis after correction

    mayby chancebe very close to the

    value of the measurand, and hence have a

    negligible error. The uncertainty may

    nonetheless still be very large, simply be

    cause the analystisunsure ofthesize of

    the error.Uncertainty and quality assur-

    ance.ISOexplicitly excludesgrosserrors

    of procedurefrom considerationwithinan

    uncertaintyassessment.Uncertainty esti

    mates can realistically apply onlytowell-

    established measurement processesinsta

    tisticalcontrol, and thus they areastate

    ment of the uncertainty expected when

    proper quality control (QC) measures are

    inplace. Itisthus implicit thatQCand qual

    ityassurance (QA) processesbe inplace

    and within specification if an uncertainty

    statementis tobeat allmeaningful.

    Repeatability andreproducibility

    The most generally applied estimates of

    uncertainty at present are those obtained

    from interlaboratory comparisons, particu

    larly those using the collaborative trial

    protocols ofISO5725 (5) and the Associa

    tion of Official Analytical Chemists

    (AOAC)(6).

    For legislative purposes, the collabora

    tive trial reproducibility figure is the closest approach to uncertainty that attempts

    to estimate the full dispersion of results

    obtained by a particular metiiod, and it

    has the considerable advantages of sim

    plicity and generality, though at high cost.

    Another substantial advantage is its objec

    tivity, because itisbased entirely on ex

    perimental observations inarepresenta

    tive range of laboratories. Though it

    serveswellin cases in which the chief

    issue is comparability among particular

    laboratories with a common aim several

    factors leave this approach wantingReproducibility is inevitably a measure

    ofprecision;although it covers a range of

    laboratorybias,it cannot cover bias inher

    entinthe methoditself,nor, in general,

    sample matrix effects. Arguably, these

    effects are not relevant for a standard

    method, which may simply define a proce

    dure that generatesaresult for trade or

    legislative purposes. Many methods, in

    deed fall into this class; even when a

    metiiod purports to determine a specific

    molecular species, there is no guarantee

    that it determines all thatispresent or,

    indeed, any particular species at all.

    An example is the semiquantitative

    AOACmethod for detecting cholesterol.

    Though standardized and properlyac

    cepted for certain trade and regulatory

    purposes on the basis of collaborative trial

    data showing sufficient agreement be

    tween laboratories (7), subsequent work

    using internal calibration (8)has shown

    that method recovery is poorer than the

    reproducibility figure suggests. It follows

    that long-term studies of cholesterol levels

    in food could be expected to misinterpretchanges in apparent level particularly

    nations using different cholesterol

    determination methods Reproducibility

    figures will in treneral suffer from the

    absence ofbiasinformation

    These arguments suggest that repro

    ducibilitywillgenerally underestimate

    uncertaintybut not necessarily.Asingle

    laboratory can have much smaller uncer

    tainties for a determination than the repro

    ducibility figure would indicate, which

    tends to includearange of pooras wellas

    good results. This issue can be put morebluntly: What does the spread of results

    found byahandful of laboratories on a

    specific set of samples at some time in the

    past have todowith the results ofanindi

    vidual laboratorytoday?Indeed, this is the

    very question that mustbeanswered

    quantitatively before any laboratory can

    make use of collaborative trial information

    inaformal uncertainty estimate. It follows

    that reproducibility, although a powerful

    tool, is notapanacea.

    The ISO approachThe approach recommended in the ISO

    guide, outlined below, is based on com

    bining the uncertainties in contributory

    parameterstoprovide an overall estimate

    of uncertainty (Figure 1).

    To begin, write down a clear statement

    ofwhat isbeing measured, including the

    relationship between the measurand and

    the parameters (measured quantities, con

    stants, calibration standards, and other

    influences) on which it depends. When

    Figure 1 . Uncertainty estimation process.

    Analytical Chemistry News & Features, October1, 1997 609 A

  • 8/12/2019 Peer Reviewed Measurement Uncertainty

    4/7

    Report

    Figure 2. Dioxin analysis.

    possible, include corrections for known

    systematic effects. Though the basic spec

    ification information is normally given in

    the relevant standard operating procedure

    or other method description, itisoften

    necessarytoadd explicit terms for factors

    such as operating temperatures and ex

    traction time or temperature, which will

    not normally be included in the basic cal

    culation given in a method description.

    Then, for each parameterinthis rela

    tionship, listthepossible sources of uncer

    tainty, including chemical assumptions.

    Measure or estimate the size of the uncer

    tainty associated with each possible source

    of uncertainty(orforacombinationof

    sources). Combine the quantified uncer

    tainty components, expressed as standard

    deviations, accordingtothe appropriate

    rules (seeBox onp.608 A)),ogive ecom

    bined standard uncertainty, and apply theappropriate coverage factorto givean ex

    panded combined uncertainty

    The most important features are that

    all contributing uncertainty components

    are quantified as standard deviations in

    the first instance, whether they arise from

    random variability or systematic effects;

    also, that estimates of uncertainty from

    experiment, prior knowledge, and profes

    sionaljudgment are treated in the same

    way and given the same weight.

    Quantifyingallcontributing uncertainty

    componentsasstandard deviationspro

    videsaparticularly simple and consistent

    methodof calculationbased on standard

    expressions for combining variances.It is

    justified in principle because, although an

    errorin aparticular casemay besystem

    atic, lackof knowledgeaboutthesizeof the

    errorleadstoaprobability distribution for

    theerror.This distribution canbetreated

    inthesame way asthat ofarandom vari

    able. Treating estimatesofuncertainty fron

    experiment prior knowledge and profes

    sional judgmentthe same wayand giving

    them the same weight ensures thatall

    known factors contributingtouncertainty

    are accounted forevenwhen experimental

    determinationisnot possible

    In principle, this approach overcomes

    many ofthedeficiencies in currently used

    approaches. It is much quicker and lesscostly to apply than a collaborative trial,

    but it can use collaborative trial data ad

    vantageously if available. The approach

    covers all the effects onaresult, system

    atic or random, and it takes into account

    all available knowledge. In addition, it

    mandates a particular form of expression,

    leadingtoimproved comparability in un

    certainty estimates

    However, disadvantages exist. The

    ISO approach, because it requires appro

    priatejudgment, cannot be entirely objec

    tive;to some extent it relies on the experi

    ence of the analyst.Asignificant costin

    time and effort isafactor; estimating un

    certainties on the basis oflocalconditions

    without using published data involves

    more effort than simply quoting a pub

    lished reproducibility figure.The lack of objectivity can be compen

    sated for by third-party review, such as

    quality system assessment, interlabora-

    tory comparisons, in-house QC sample

    results, and certified reference material

    checks. Finally, it should be clear that a

    decision to excludeaparticular contribu

    tion entirely rather than makesomejudg

    ment ofitssize represents a de facto deci

    sion to allocate the contribution a size of

    zero hardly an improvement.

    Cost,too,may berecouped in direct or

    indirect benefits. Uncertainty estimation

    improves knowledge ofanalyticaltech

    niques and principles, formingapowerful

    adjuncttotraining.Knowing themain con

    tributionstouncertainty determines the

    directionof methodimprovement most

    effectively. Efficiency can be improvedwith

    minimal impactonmethod performance.

    Finally, normalQAprocedures,such as

    checking the method for use, maintaining

    recordsofcalibration and statisticalQC

    procedures, should provideallthe required

    data;additional costshouldbe no morethan

    thatofcombining the dataappropriately

    Sources of uncertainty

    Many factors affect analytical results, and

    every one is a potential source of uncer

    tainty. In sampling, effects such as ran

    dom variations between different samples

    and any potential for bias in the sampling

    procedure are components of uncertainty

    affecting thefinalresult. Recovery ofan

    analyte from a complex matrix, or an in

    strument response, may be affected by

    other constituents ofthematrix. Analytespeciation may further compound this

    effect. When a spike is used to estimate

    recovery the recovery of the analyte from

    the sampleiricivdiffer" from the recoverv

    ofthespike Stability effects are also im-

    portant but frequently are not well-known

    Cross-contamination between samples

    and contamination from the

    laboratoryemnrrtnment are pupr nrpcpnt risk s

    Though ISO does not include accidental

    gross cross-contamination in its definition

    610 A Analytical Chemistry News & Features, October1, 1997

  • 8/12/2019 Peer Reviewed Measurement Uncertainty

    5/7

    of uncertainty, as it represents loss of con

    trol ofthemeasurement process, the pos

    sibility of background contamination

    should nonetheless be considered and

    evaluated when appropriate.

    Although instruments are regularly

    checked and calibrated, the limits of accu

    racy on the calibration constitute uncertainties. Calibration used may not accu

    rately reflect the samples presented; for

    example, analytical balances are com

    monly calibrated using nickel check

    weights, although samples are rarelyof

    such high density. Though not large in

    most circumstances, buoyancy effects

    differ between calibration weight and sam

    ple. Other factors include carry-over and

    systematic truncation effects.

    The molarity ofavolumetric solution is

    not exactly known, even iftheparent ma

    terial has been assayed, because some

    uncertainty relating to the assay proce

    dure exists.A widerange of ambient con

    ditions, notably temperature, affects ana

    lytical results. Reference materials are

    also subjecttouncertainty; fortunately,

    most providers of reference materials now

    state the uncertainty in the manner rec

    ommended in the guide.

    The uncritical use of computer soft

    ware can also introduce errors. Selecting

    the appropriate calibration model is im

    portant, and software may not permit the

    best choice. Early truncation and round

    ing offcanalso lead to inaccuracies in the

    final result.

    Operator effectsmaybe significant;

    they can be evaluated either by predicting

    them or by conducting experiments in

    volving many operators. The latterap

    proachwillnot normally detect an overall

    operator bias (for example, a particular

    scale reading may be taken in the same

    manner byagroup of operators similarly

    trained), but the scope of variation can be

    estimated. "Operator effect" could reasonably be considered a proxy for a range of

    poorly controlled input parameters such

    as scale-reading accuracy time and dura

    tion of agitation during extraction and so

    on It follows thataformal mathematical

    model oftheexperimental process would

    not normally include "operator" as an in-

    niit factor but only the specific factors un

    de r ooerator

    Random effects contribute to uncer

    tainty in all determinations, and this en

    try is usually included in the list as a

    matter of course. Conceptualizing every

    component of uncertainty as arising

    from both systematic and random effects

    is also frequently useful; this step avoids

    the most common trap for the unwary

    overlooking systematic effects in the

    effort to obtain good precision measures.Both need to be taken into account,

    though the ISO approach requires only

    the overall value.

    Determinands are not always com

    pletely defined. For example, volumes

    may or may not be defined with refer

    ence to a particular set of ambient con

    ditions. Similarly, the determinand may

    be defined in terms ofarange of condi

    tions.For example, material extracted

    from an acidified aqueous solution at pH

    below 3.0 allows substantial latitude.

    Such incomplete definitions result in thedeterminand itself having a range of val

    ues,irrespective of good analytical tech

    nique, and that range constitutes an

    uncertainty.

    Many common analytes, such as fat,

    moisture, ash, and protein, are defined not

    in terms ofaparticular molecular or

    atomic species but against some essen

    tially arbitrary process. In effect, the re

    sult is simply a response to a stated proce

    dure,expressed in the most convenient

    units. Such measurements are not gener

    ally compared with results from other

    methods; in effect, bias is neglected by

    convention. However, the procedure itself

    may lack full definition or permitarange

    of conditions, giving rise to uncertainties.

    Of course if comparison with other meth

    ods is desired additional sources of un

    certainty including method bias must be

    taken into

    Increasing confidence

    The ISO guide suggests multiplying the

    standard uncertainty by a coverage fac

    torkto express uncertainties when a

    high degree of confidence is desired.

    This representation exactly mirrors the

    situation in conventional statistics, in

    which a confidence interval is obtainedby multiplying a standard deviation for a

    parameter by a factor derived from the

    Student ^-distribution for the appropriate

    number of degrees of freedom.

    The formal approach in the guide re

    quires estimation ofasimilar parameter,

    the "effective degrees of freedom", and

    uses this value in the same way. Though

    the details are beyond the scope of this

    article, some important points can be

    made.

    This parameter is almost invariably

    dominated by the number of degrees of

    freedom in the dominant contribution to

    the overall uncertainty. When the domi

    nant contribution arises from sound and

    well-researched information, effective de

    grees of freedom remain high, normally

    leadingtok= 2for near95%confidencce

    Only where large uncertainty contribu

    tions are based on meager datawillthe

    choice ofkbecome significant.Aprag

    matic approach, therefore, is simply to

    adoptk = 2for routtne work ,ndk k=

    when a particularly high confidence is

    required(2)

    The question of possible distributions

    mustalsobe consideredat thepointofde

    ciding coverage factors. Although the guide

    usesacombination of standard deviations

    based on established error propagation

    theory, the step from standard deviation to

    confidence involves some assumptions.

    The guide takes theviewthat,in mostcir-

    Table 1. Contributions to uncertainty in dioxin analysis.

    Parameter u(RSD) Main contributio n3

    Oss 0.02

    V 0.02

    Ak an d -4SS 0.09

    RRFn 0.08

    "lspk 0.12

    Combine d uncertainty 0.17

    Syringe specification; certified reference solution

    uncertainty

    Density (volume determined by weight)

    Permitted abundance ratio range

    Range permitted by method

    Permined range of spike recovery

    [(0.02) +(0. 02) +(0. 09) + (0.08)2 +(0.12)2]1/2

    (a) Contributions are listed if they contribute more than 10% of the stated uncertainty.(b) Permitted ranges are treated as limits of rectangular distributions and adjusted to SDvalues (1)by dividing by the square root of 3.(c) Recovery of added material is not, in general, fully representative of recovery of analytematerials.

    Analytical Chemistry News & Features, October 1, 1997 61 1 A

  • 8/12/2019 Peer Reviewed Measurement Uncertainty

    6/7

    Report

    cumstances, the centrallimittheoremwill

    apply, and the appropriate distributionwill

    be approximately normal. Certainlyit is

    raretocalculate confidence intervals based

    on other distributionsingeneral analytical

    chemistry, ifonly because it isunusual to

    have sufficient datatojustify other assump

    tions. Nonetheless, when additional knowl

    edge about underlying distributions exists,

    it ismost sensible to basekonthe best

    available information.

    Dioxin example

    The analysis ofdioxinsin the effluentof

    paper and pulp mills by isotope dilution

    MS (Figure2)isagood example (9). For

    the sake ofdiscussion,we willconsider

    only the analysis of 2,3,7,8-tetrachloro-

    dibenzodioxin (2,3,7,8-TCDD) and will

    ignore the normally important uncertain

    ties caused by interference from other

    TCDD isomers,GCintegration, and res

    olution difficulties.Byway of illustration,

    some minor contributions that would

    not normally be included will also be

    examined.

    The basic equation for determining the

    concentrationCxof TCDD is

    Cx=AkQss/AaJiRFnVR%pk

    in whichAk isthe peak area oftheana-

    lyte,Qssis the amount of spike,i4ssis the

    peak area ofthestandard,RRFnis the

    relevant response factor for the relevant

    ion13C-12, Fis the original sample vol

    ume,andi?spkisthe (nominal) recoveryof

    the analyte relative to added material.

    Rakmerits explanation, because it is

    not used in the standard. Because the13

    C-12 calibration spike is added to the

    slurry and is not naturally part ofthesam

    ple, differential behavior is possible. If

    measurable, this behavior would appear

    as imperfect recovery of analyte.Acom

    plete mathematical model of the system

    therefore requires some representation of

    the effect. Because no existing parameter

    in the equation is directly influenced by

    recovery the recovery term has been

    added in the form ofanominal correction

    factor The resultis abasic equation en

    compassing all the main effects on the

    result

    Identification oftheremaining contri

    butions to the overall uncertaintyisbest

    achieved by considering the parameters

    in the equation, any intermediate mea

    surements from which they are derived,

    and any effects that arise from particular

    operations within the method (such as the

    possibility of"spikepartitioning"). Table 1

    lists parameters, calculated uncertainties

    (as relative standard deviations), and

    some contributory factors.

    Informationin Table 1 showsthat uncer

    tainties associated with the physical measurements of volume and mass contribute

    essentially nothingtothe combined uncer

    tainty and thatanyfurther study should be

    directed primarilyat theremainingcompo

    nents. The largest contributionarisesfrom

    the extraction recovery step,in linewith

    most analysts' experience.

    The method studied hereisunusualin

    specifying directcontrol of allthe major

    factors affecting uncertainty, which makes

    itrelativelyeasy toestimate uncertainty as

    longasthe method isoperating withincon

    trol.For most methods currentlyinuse,

    however, such controllimitsarenot closely

    specified.Typically,one ortwocriticalpa

    rameters are given single target values, and

    precision controllimitsare setIt thenfalls

    to the laboratorytoestimate the contribu

    tion ofits ownlevel ofcontrol tothe uncertainty, ratherthan simplydemonstrating

    compliancewithan established set of fig

    ures and an associated, carefully studied,

    uncertainty estimate.

    Legislation and compliance

    Two issues are importantwhenuncer

    taintyisconsidered in the context of legis

    lation and enforcement. The first con

    cerns the simpler problem of whether a

    result constitutes evidence ofnoncompli

    ance with some limit, particularly when

    the limitiswithin the uncertainty quoted.The second issue is the use of uncertainty

    information in setting limits.

    Two instances in compliance are

    clear-cut: Either the result is above the

    upper limit, including its uncertainty,

    which means that the result is in non

    compliance (Figure 3a); or the result,

    including its uncertainty, is between the

    upper and lowerlimits,and is therefore

    in compliance (Figure 3d). For any other

    case, some interpretation is necessary

    and can be made only in the light of the

    and with the knowledge and

    understanding of the end of the

    information

    For example, Figure 3b represents

    probable noncompliance with the limit,

    but noncompliance is not demonstrated

    beyond reasonable doubt. In the case of

    legislation, the precise wording needs to

    be consulted; some legislation requires

    that, for example, process operators dem

    onstrate that they are complyingwitha

    limit. In such acase,Figure 3b represents

    noncompliance with the legislation; compliance has not been demonstrated be

    yond doubt.

    Similarly, if legislation requires clear

    evidence of noncompliance withalimit

    that triggers enforcement, although there

    is no clear demonstration of compliance,

    there is insufficient evidence of noncom

    pliance to support action, as in Figure 3c.

    In these situations, end-users and legisla

    tors must spell out how the situation

    should be handled.

    Upperlimit

    Lowerlimit

    (a)Result above

    limit plusuncertainty

    (b)Result abovelimit, but limit

    withinuncertainty

    (c)Result belowlimit, but limit

    withinuncertainty

    (d)Result belowlimit minusuncertainty

    Figure 3. Uncertainty and compliance limits.

    612 A AnalyticalChemistry News &&eatures, October1, 1991

  • 8/12/2019 Peer Reviewed Measurement Uncertainty

    7/7

    A recent editorial (10)pointed out the

    need to avoid setting limits that cannot be

    enforced without disproportionate effort.

    An important factor to consider is the actual

    uncertainty involved in determining a level

    of analyte; if legislation is to be effective, the

    uncertainty must be small in relation to any

    limiting range. In chemistry, measurementrequirements tend to follow a "best avail

    able"presumption, even when this policy is

    not actually written into legislation; an ex

    ample is the Delaney Clause (11).

    As the state of the art improves, new

    measurements become possible and are

    immediately applied, leading to a situa

    tion in which the best available technol

    ogy is the only acceptable technology. In

    such a situation, uncertainties are, inevi

    tably, hard to quantify well; they will of

    ten be larger than required for the pur

    pose. That legislation takes into account

    the full uncertainty is particularly impor

    tant; failure to include significant compo

    nents may unreasonably restrict enforce

    ment. In particular, the possibility of sys

    tematic effects being considered is vital;

    legislation based on measurement of

    absolute amounts of substance, as in

    most new European environmental legis

    lation, must consider the full range of

    methods and sample matrices that may

    fall within that legislation.

    Another important consideration is theinterpretation of results and their relevant

    uncertainties against limits. Assumptions

    about the handling of experimental uncer

    tainty in interpretation for enforcement

    purposes must be clearly stated in the

    legislation. Specifically, do limits allow for

    an experimental uncertainty or not? If so,

    how large is the allowance?

    A fundamental factor is how well leg

    islators understand uncertainty. The

    need to set limits in some contexts is

    easily understood, such as how much of

    a toxic compound is acceptable in anenvironmental matrix. However, judging

    compliance is trickier, and a better un

    derstanding of analytical uncertainly is

    required. The current move toward spec

    ifying method performance parameters,

    such as repeatability, reproducibility,

    and recovery rather than the method

    itself, is a step in the right direction; but

    these parameters do not necessarily

    cover all of the significant components of

    uncertainty. What is required is the addi

    tional specification of the measurement

    uncertainty to meet the needs of the

    legislation.

    Ellison's workwassupported under contractwith the Department of Trade and Industry as

    part of the National Measurement SystemValidAnalytical Measurement Programme.

    References

    (1) Guide to the ExpressionofUncertaintyinMeasurement;ISO:Geneva,,193;ISBN92-67-10188-9.

    (2) Quantifying Uncertainty ininalyticalMeasurement;Published do nehalf foEURACHEM by Laboratory of the Government Chemist: London,1995;ISBN0-948926-08-2.

    (3) Taylor,B.N.; Kuyatt, C. E.Guidelines forEvaluating and ExpressingthtUncertaintyofNISTMeasurementResults;NIST Technical Note 1297, National Institute of Stan

    dards and Technology: Gaithersburg,MD,1994.

    (4) International VocabularyofBasicanaGeneral Standard Termsin Metrology,ISO:Geneva,1993;ISBN 92-67-10175-1.

    (5) ISO 5725:1986,PrecisionofTestMethods:DeterminationofRepeatability anaReproducibility fora Standard Method byInter-laboratoryTests;ISO:Geneva, 1987.

    (6) Youden,W.H.; Steiner, E. H.StatisticalManualofthe Associationof Official Analytical ChemiststAOAC:Washington, DC,1982.

    (7) Thorpe, C.W.Assoc.Anal.Chem. 1969,52,778-81.

    (8) Lognay, G. C; Pearse,J.;Pocklington, D.;Boenke, A.; Schurer,B.;Wagstaffe,P.J.

    Analyst1995,1201831-35.(9) ReportEPSl/RM/19; Environment Can

    ada: Ottawa, Ontario, 1992.(10) Thompson,M.Analyss1995,120,

    117N-18N.(11) Delaney: Federal Food, Drug and Cos

    meticAct;Food additives amendment,1958.

    Steve Ellison is head of the analytical

    quality and chemometrics section at the

    Laboratory of the Government Chemist

    (U.K.). His research interests include sta

    tistics, validation and measurement un

    certainty, and chemometrics in the con

    texts of regulatory analysis and analyticalchemistry. Wolfhard Wegscheider is profes

    sor of chemistry and dean of graduate

    studies at the University ofLeoben (Aus

    tria) and chair of EURACHEM Austria.

    Alex Williams is chair of the EURACHEM

    Working Group on Measurement Uncer

    tainty Address correspondence about this

    article to Wegscheider at Institute of Gen

    eral and Analytical Chemistry University

    ofLeoben A-8700 Leoben Austria

    (wegschei@unileoben ac at)

    What's newin LC-MS

    PRODUCTS

    Quattro LC

    micromass

    USATel: 508 524-8200 Fax: 508 524-82

    Europe Tel: +31 (0) 294-480484 Fax: +31 (0) 2

    UK/ InternationalTel: +44 (0) 161 945 4

    Fax:+44(0) 161 998 8915

    http://www.mlcromass.co.uk

    CIRCLE 1 ON READER SERVICE C

    http://www.mlcromass.co.uk/http://www.mlcromass.co.uk/