calibration point selection - ncsl · pdf filegeneral operating procedure ... when it carries...
TRANSCRIPT
Calibration Point SelectionWho makes the call?
Presented by: Joe PetersenMetrology EngineerAbbott Laboratories
Calibration Point Selection08/01/2012
2© 2012 Abbott
Common Questions
• Do calibration points have to cover the range of the instrument?
• How many calibration points are needed?
• Are 10, 50, 90 % of full scale appropriate?
• Should zero be used? Is it valid?
Calibration Point Selection08/01/2012
3© 2012 Abbott
Introduction
• Calibration points effect measurement reliability– ISO 17025 addresses many quality issues;
• calibration point selection is not directly addressed• perhaps indirectly addressed by:
– 5.4 Methods, or– 5.7 Sampling
Calibration Point Selection08/01/2012
4© 2012 Abbott
Agenda
• General principles
• Who makes the call?
• A form of sampling?
Calibration Point Selection08/01/2012
5© 2012 Abbott
Learning Objectives
After this session, the participant should be able to:• List key considerations for selecting calibration points• Promote best practices for using a team approach to define
calibration points
Calibration Point Selection08/01/2012
6© 2012 Abbott
Where are calibration point selection requirements defined?
QM
Policy
General Operating Procedure
Specific Calibration Procedure(Work Instruction)
Calibration Point Selection08/01/2012
7© 2012 Abbott
Example of Policy / General Procedure
• “The selection of calibration points will vary with the normal use of the meter ....”– Usage
• “For example, a meter which monitors a fixed value can be spot calibrated at the critical value.”– Single Point Calibration
• “Meters which may indicate any value within their ranges will require a linearity test at full scale and at least three equally spaced divisions on one range, and a full-scale test on each remaining range.”– Full Calibration
• “Other meters, such as those used over a narrow range, may require calibration over a restricted portion of the scale.”– Limited Calibration
– CALIBRATION OR QUALIFICATION OF PANEL METERS, Metrology Bulletin, Oct '64--USN Metrology Engineering Center, Pomona, Calif. ; as published in National Conference of Standards Laboratories, NCSL NEWSLETTER 67-4, 1967
Calibration Point Selection08/01/2012
8© 2012 Abbott
Example of Policy / General Procedure• “In general, for each TI characteristic being calibrated, the smallest number of test
points consistent with obtaining verification of the TI performance over its specified range should be used.”– Risk based approach
• “The number and choice of required test points will vary, depending on the characteristics of the TI.”– Consider design and technology
• “Guidance concerning choice of test points for selected generic classes of equipment is contained in Calibration Requirements Documents (CRDs).”– Specific requirements are in individual calibration procedures
• “In some cases, TI application/use requirements will dictate test point selection.”– Consider use
– REQUIREMENTS FOR PREPARATION OF INSTRUMENT CALIBRATION PROCEDURES• NAVAIR 17-35TR-4• NAVY METROLOGY ENGINEERING CENTER• 15 AUGUST 1990• 3.7.2.10 Calibration Test Points.
Calibration Point Selection08/01/2012
9© 2012 Abbott
Key Considerations for Point Selection
• Measurement Technology
• External Guidance Documents
• Calibrated Range
– full or limited
• Assigned Accuracy
– (maximum permissible error)
• # of adjustment points
• User requirements
– critical usage points may be preferred calibration points
Calibration Point Selection08/01/2012
10© 2012 Abbott
Are 3 points sufficient?Balance calibration guidelines
5 or moreEA-10/18 Guidelines on the calibration of non-automatic weighing instruments
10ASTM E898
typically 4Sartorius Tolerances for Testing Metrological Specifications
typically 4Mettler Balance Tolerances
5SANAS R15-03 (South African National Accreditation System)
5Code of Practice for Industrial Weighing Machines, United Kingdom Weighing Federation
10UKAS LAB145OIML R76-1
Minimum # of Accuracy/Linearity Test Points (excluding zero)
Publication
Calibration Point Selection08/01/2012
11© 2012 Abbott
EURAMET Guidelines on the Calibration of Digital Multimeters
• cg-15
– Previously EA-10/15
Calibration Point Selection08/01/2012
12© 2012 Abbott
EURAMET Guidelines on the Calibration of Digital Multimeters
– 3.4.1 “… consider the working principles of the instrument ...”– “[obtain guidance] from the manufacturer’s instructions,
although the points listed therein should not necessarily be considered exhaustive.”
‘trust but verify’
Calibration Point Selection08/01/2012
13© 2012 Abbott
EURAMET Guidelines on the Calibration of Digital Multimeters
* Based on five parameters and five ranges per parameter.
• Conclusion: The extent of testing should be based on instrument accuracy.
100 to 1145 1/2 digits or more
(e.g. 200,000 counts)2
58 to 704 1/2 digits or less
(e.g. 20,000 counts)1
Recommended # of calibration Points*
Accuracy Class (Resolution)Table
Calibration Point Selection08/01/2012
14© 2012 Abbott
# of calibration Points
• Factors to Consider
– Calibrated Span– Assigned Accuracy– Measurement Technology
• First principles• Empirical
– Cost– Risk
Relative Humidity Performance
-4
-2
0
2
4
0 20 40 60 80 100
% RH
Erro
r, %
RH
Relative Humidity Performance
-4
-2
0
2
4
0 20 40 60 80 100% RH
Erro
r, %
RH
Assume ± 2 %RH
Calibration Point Selection08/01/2012
15© 2012 Abbott
Hind Sight is 20/20
• Given an excessive # of calibration points, it is easy to see which points are unnecessary
• You generally can’t tell you have enough points until you’ve tested more than enough
– Type testing– Retrospective review of historical results
Calibration Point Selection08/01/2012
16© 2012 Abbott
Hind Sight is 20/20Stir Plate Calibrations
-4
-3
-2
-1
0
1
2
3
4
0 200 400 600 800 1000 1200
Speed, rpm
Aver
age
Erro
r, %
of r
eadi
ng .
Average Error, % of reading -- 1 standard deviation Error BarsAverage Error, % of reading -- 1 standard deviation Error Bars
Calibration Point Selection08/01/2012
17© 2012 Abbott
Build on Experience
“Test results showed that measurements made at 50 test points were sufficient to allow accurate predictions of the instrument's performance at all 255 test points specified by the manufacturer.”
A. D. Koffman, H. L. Stott, Modeling and Test Point Selection for a Thermal Transfer Standard [AC Voltage], NCSL Workshop & Symposium, 1993
Calibration Point Selection08/01/2012
18© 2012 Abbott
Points associated with secondary parameters
temperaturerelative humiditywavelengthoptical transmittancefrequencyAC voltage
.. depends upon..
Secondary ParameterPrimary Parameter
Calibration Point Selection08/01/2012
19© 2012 Abbott
Full matrixSpectrophotometer Calibration Points – Commercially Available Filters– Visible Light Region
0.00.20.40.60.81.01.21.41.61.82.0
380 480 580 680 780wavelength, nm
optic
al d
ensi
ty o
r abs
orba
nce
Calibration Point Selection08/01/2012
20© 2012 Abbott
Risk based approach
AC Voltage Calibration PointsEURAMET cg-15/v.01
0102030405060708090
100
10 100 1000 10000 100000 1000000frequency, Hz
mea
sure
men
t poi
nt,
% o
f ful
l sca
leOne Intermediate RangeOthers
Calibration Point Selection08/01/2012
21© 2012 Abbott
Uncertainty increases as calibration points decrease
0.01 - 0.03Multipoint (5-point) calibration
0.02 - 0.03Calibration (2-point) by bracketing
0.3Calibration (1-point), ΔpH = |pH(X) – pH(S)| = 3 and assumed slope
Target uncertainty, U(pH)
Procedure
Source: R.P. Buck, et al., Measurement of pH. Definition, standards, and procedures (IUPAC Recommendations 2002)International Union of Pure and Applied Chemistry
Calibration Point Selection08/01/2012
22© 2012 Abbott
# of calibration Points
• Summary of General Requirements
– Typically three or more to verify linearity– One is suitable for devices which are inherently linear (e.g. flow
totalizers, stopwatches, frequency counters)– Two are suitable for very narrow portions of a range– Five or more may be appropriate for some technologies
• Often recommended in guidance documents
Calibration Point Selection08/01/2012
23© 2012 Abbott
Specific Principles
• Primarily related to technology
– Analog v. Digital– First Principles v. Empirical– Mechanical v. Electronic
Calibration Point Selection08/01/2012
24© 2012 Abbott
Who makes the call?
• Instrument Manufacturer
• Calibration Supplier
• Metrological Confirmation Group
• Instrument Owner / User
Calibration Point Selection08/01/2012
25© 2012 Abbott
Who makes the call?
– Instrument Manufacturer
– Calibration Supplier
– Instrument Owner / User
Going back to key considerations
• Measurement technology – (perhaps biased view)– Perhaps the most expertise
• Likely failure modes
• Available calibration standards• Perhaps an unbiased view of measurement
technology based on experience with multiple manufacturers
• Critical Points
Calibration Point Selection08/01/2012
26© 2012 Abbott
Best practice for using a team approach
1. Obtain input from stakeholders
2. Calibration laboratory proposes calibration points
3. Owner reviews and approves
• Aligns with ISO 17025, 4.4 Review of requests, tenders and contracts
Calibration Point Selection08/01/2012
27© 2012 Abbott
Is calibration point selection a form of sampling?
Calibration Point Selection08/01/2012
28© 2012 Abbott
Is calibration point selection a form of sampling?
• ISO 17025 - 5.7 Sampling
– 5.7.1 The laboratory shall have a sampling plan and procedures for sampling when it carries out sampling of substances, materials or products for subsequent testing or calibration. …
– NOTE 1: Sampling is a defined procedure whereby a part of a substance, material or product is taken to provide for testing or calibration of a representative sample of the whole.
• How many organizations have a defined procedure for selection of calibration points?
Calibration Point Selection08/01/2012
29© 2012 Abbott
Is calibration point selection a form of sampling?
5.7 Sampling – Not Applicable for calibrations(How many have seen this or a similar statement in a quality manual?)
• “Sampling does not apply. This laboratory only performs calibration, not testing.”
• “The laboratory does not sample items submitted for calibration.”
• “Sampling is not a part of our calibration process.
Calibration Point Selection08/01/2012
30© 2012 Abbott
Is calibration point selection a form of sampling?
• Three interpretations of ISO 17025 for point selection
1. a form of sampling covered by 5.7, “Sampling”2. not sampling but is covered by 5.4, “Test and calibration
methods and method validation”3. not covered by 17025
Calibration Point Selection08/01/2012
31© 2012 Abbott
Inadequate Calibration Points
XYZ Calibration Company ISO 17025 Accredited Calibration
Accreditations Universal
Certificate #: 01.02.03 Item: Humidity Meter Manufacturer: Meter Company Model: 87Y21
Temperature Standard Unit Under
Test Error Uncertainty
°C °C °C °C 23.04 23.1 +0.06 0.05
Relative Humidity
Standard Unit Under Test
Error Uncertainty
% RH % RH % RH % RH 50.3 52.8 +2.5 1.4
Approved by: J. Jones 03/30/2013
Device Range:
-10 to 70 °C
0 to 100 %RH
Calibration Point Selection08/01/2012
32© 2012 Abbott
Between Point Uncertainty
• If user specifies range
– Reliability and uncertainty between points is lab’s responsibility
• If user specifies points
– Reliability and uncertainty between points is user’s responsibility
– Mark Kuster, Enough Is Enough: Selecting Points for Range Calibration, Measure Journal, Sep. 2011
Calibration Point Selection08/01/2012
33© 2012 Abbott
Between Point Uncertainty
Sample certification statement:
• If user specifies range
– “Instrument meets manufacturer’s specifications throughout all functions and ranges.”
• If user specifies points
– “Instrument meets manufacturer’s specifications at all points tested.”
Calibration Point Selection08/01/2012
34© 2012 Abbott
Appeal to Accreditation Bodies
• Whether point selection is an element of calibration methods or sampling:– Establish directives to assure adequate calibration points
• Eliminate the quality system gap• Level the playing field
– Prohibit single point certificates that imply a fully calibratedinstrument • (except as appropriate)• Some instruments, purchased new, come with an accredited calibration
Calibration Point Selection08/01/2012
35© 2012 Abbott
Conclusion
• Risk of too few calibration points
– Untested regions– Undetected linearity errors
• Calibration point selection is technology dependent
• Each laboratory should establish a general policy / procedure
• Guidelines should also be defined in specific calibration procedures
• Team approach is desirable
• Accreditation should address point selection