1NCSLI Conference 2013

Inter-Laboratory Comparison Study Using Modular

Instrumentation and Lessons Learned

Author: Dimaries Nieves – National InstrumentsSenior Metrology Engineer

Speaker: Jorge Martins – National InstrumentsPrincipal Metrology Engineer

2NCSLI Conference 2013

Learning Objectives• How can we demonstrate performance and competence of modular instrumentation as part of metrology and accreditation process?

• Propose a process to perform an Inter-laboratory Comparison using a modular instruments.

• Standard Documents for ILC evaluation.

• Review some of the results and lessons learned.

3NCSLI Conference 2013

• Inter-laboratory Comparison (ILC) is a key criterion for laboratory accreditation• ILC is an important asset in:

• Measurement Reliability• Measurement Assurance• Calibration Process Confidence• Measurement Method• Technician Proficiency

• There is no commercial proficiency testing available for Modular Instrumentation.

For the last two years, National Instruments performed an interlaboratory comparison using Modular Instrumentation as the primary standard

4NCSLI Conference 2013

Modular Instruments • Gradually taking an important role in test and measurement

• Enable more cost effective and flexible measurements

• Used in Research, Calibration, Validation and Production test

In metrology and laboratory accreditation process • Need to demonstrate performance

and competence performing regular calibration services.

5NCSLI Conference 2013

NI Inter-Laboratory Comparison Proposal• The ILC was designed based on:

• NCSLI’s RP-15 • ASTM E691-13

• Unit was monitored using the petal or flower model

• NI Certified Calibration Centers were selected to participate

• NIC Metrology Laboratory serves as the pivot Laboratory

6NCSLI Conference 2013

ILC Equipment Bias and Stability• Stability of the reference unit was determined by the pivot Laboratory before starting the ILC process.

• No significant bias was found within the pivot measurements.

• The pivot laboratory monitor the performance of the unit to ensure that any unexpected changes in the traveling standard are promptly detected.

• Comparison of the pivot lab data showed that the unit remained in control throughout the all ILC.

7NCSLI Conference 2013

ILC Reference Value• Reference values were determined using the pivot lab measurements.

• All pivot runs were included in the analysis• Statistical Analysis of the data was performed

• Uncertainties reported by the pivot laboratory include ILC process bias and deviations.

8NCSLI Conference 2013

First ILC Run• NI PXI-4072 Flex DMM and LCR Meter (6 ½ digits DMM)

• NI Published Calibration Procedure• “Verify Mode Only” procedure using NI’s calibration software, Calibration Executive

• Participants use their own chassis, standards, and cables to perform the measurements

•Report only one measurement result per point with an expanded uncertainty at 95% of confidence (k=2).

9NCSLI Conference 2013

For Evaluation

An En value was calculated for each measurement provided for each laboratory

10

NCSLI Conference 2013

Resistance Measurement Verification in 2-wire mode First ILC (2011)

11

NCSLI Conference 2013

Resistance Measurement Verification in 4-wire mode

First ILC (2011)

12

NCSLI Conference 2013

Second ILC Run• NI PXI-4072 FlexDMM and LCR Meter (6 ½ digits DMM) , Chassis and Cables (entire Setup).

• NI Published Calibration Procedure• “Verify Mode Only” procedure using NI’s calibration software, Calibration Executive

• Performed three runs on the unit, doing a manual “Self Calibration” before each run

• Verify that the board temperature was stable at 35 ºC ± 2 ºC before performing the verification

•Report measurement result with an expanded uncertainty at 95% of confidence (k=2).

13

NCSLI Conference 2013

Resistance Measurement Verification in 2-wire mode Second ILC (2012)

Pivot

14

NCSLI Conference 2013

Resistance Measurement Verification in four-wire mode Second ILC (2012)

15

NCSLI Conference 2013

Additional Analysis• Consistency Evaluation

• Within the laboratory

• S = Standard Deviation for one laboratory• Sr = Repeatability standard deviation of the equipment

• p = Number of laboratories participating in the ILC

16

NCSLI Conference 2013

Additional Analysis• Consistency Evaluation

• Between Laboratory

• Where, = Lab average minus the average of the lab averages

• Where, Sx = Standard deviation of the lab averages

17

NCSLI Conference 2013

Within-laboratory consistency (k) for DC Voltage measurement Second ILC (2012)

18

NCSLI Conference 2013

Between-laboratory consistency (h) for DC Voltage measurement

Second ILC (2012)

19

NCSLI Conference 2013

Conclusion• Modular Instruments meet the published measurement specifications, regardless of the combination of chassis, controller and measurement I/O.

• However, for an ILC the higher requirements for repeatability and reproducibility make it advantageous to provide, not just the instrument but, the entire measurement system.

• Including the entire system in the ILC reduces differences between participants, improving the correlation of data.

• Several statistical tools are needed

20

NCSLI Conference 2013

Questions?