javier garcia - verdugo sanchez - six sigma training - w1 analysis of measurement

Analysis of MeasurementPart 1: Introduction

Week 1

Knorr-Bremse Group

About this Module

Based on this technique you can asses and judge t t h b tt th d ib dmeasurement systems much better than described in the ISO 9000 standard.

Part 1: Introduction of Measurement System Analysis Concept definition and describing the basic termsConcept definition and describing the basic terms

Part 2: Attributive Measurements Kappa Analysis

Part 3: Continuous Measurements The method for the Gage R&R Study

Some exercises

Knorr-Bremse Group 16 BB W1 Measurement Intro 08, D. Szemkus/H. Winkler Page 2/39

The DMAIC Cycle

ControlMaintain DefineMaintain

ImprovementsSPC

Control Plans

Project charter (SMART)

Business Score CardQFD VOC

D Documentation QFD + VOC

Strategic GoalsProject strategy

C M

MeasureImproveAI

Baseline AnalysisProcess MapC + E MatrixAnalyze

ImproveAdjustment to the

OptimumFMEA

Measurement System

Definition of critical InputsFMEA

S

FMEAStatistical Tests

SimulationTolerancing y

Process CapabilityStatistical TestsMulti-Vari Studies

Regression

Tolerancing


Content and Terminology

Discrimination P/T Ratio

Terms connected with accuracyT l

Precision to tolerance

R&R % True value Systematic Error / Bias Linearity

R&R % Repeatability and

ReproducibilityLinearity

Terms connected with precision Process capability related i i f h

p Repeatability Reproducibility

variation from the measurement system

Linearity

Stability (over Time) Stability (over Time)


Possible Sources for Process Variation

Observed Process Variation

Actual Process Variation Measurement Variation

Short Term Process Variation

Long Term Process Variation

Variation within a sample

Variation due to Measurement

System

Variation due to Operatorp System p

Repeatability Precision Calibration Stability Linearity

In order to work on the actual process variation, the measurement variation has


to be determined and separated from the process variation

Sources of Measurement Variation

W k M th d

Operator TrainingEase of Data Entry

M eaMechanical instability

Tool Work Methods

Sufficient Work TimeMaintenance Standard

Calibration Frequency

Electrical Instability

Wear

'Measurement Variation'Operator Technique

Standard ProceduresAlgorithm Instabilty

Measurement Variation

Humidity

Cleanliness

Vibration

Line Voltage Variation

Temperature Fluctuation

M ethodsEnvironment Environment


Needed Information How big is the measurement error?

What are the sources of the measurement error? What are the sources of the measurement error?

Is the gauge stable over the time?

Is the gauge suitable for this examination?

How can we improve the measurement system?

Measurement tools (Hardware and Software)Measurement tools (Hardware and Software)

All procedures for using the tools Which operator?

Set-up and handling proceduresp g p

Off-line calculations and data entry

C lib ti f d t h i


Calibration frequency and technique

Effects of Measurement Error

Measurement System

AverageBias -Determined through

Calibration Study

Accuracy

total product measurement= +p

V i bilit

Measurement System Variability - Determined through R&R Study

Variability

222

Precision

222tmeasuremenproducttotal +=


The True Process Variation

Observed Variation (Total Variation)

Actual Process Variation Measurement Variation

Can we observe the truth?


Can we observe the truth?

DiscriminationThe number of decimal places that can be measured by the system.

Increments of measure (scale) should be about one-tenth of theIncrements of measure (scale) should be about one tenth of the

width of the product specification or process variation. th Recent standards require even 1/20th of the specification

Poor Discrimination

1 2 3 4 5

Good Discrimination

1 2 3 4 5


Accuracy

Accuracy - Extent which the average of the measurements deviate from the true value

True valueTrue value Theoretically correct value

NIST t d d / ISO NIST standards / ISO

Bias (systematic error)( y ) Distance between average value of all measurements and

true value

Amount tool is consistently off target

Systematic error or offset Systematic error or offset


AccuracyInstrument accuracy is the difference between the observed average

value of measurements and the master value. The master value isvalue of measurements and the master value. The master value is

an accepted, traceable reference standard (e.g., NIST, ISO).

Master Value(Reference Standard)

Average ValueHow would we investigate the accuracy of a measurement system?

Who takes over this task normally?

What is the typical definition to ypdescribe the estimation of the measurement system accuracy?


Systematic Errors / BiasThe average of measurements differs around a fixed value. Systematic errors are:

Systematic errors (bias) of the operators different operators measure noticeable different average values for an identical h t i ticharacteristic;

Systematic errors (bias) of the instruments different instruments show noticeable different average values for an identicalinstruments show noticeable different average values for an identical characteristic;

Instrument 1Master Value

(Reference Standard)Average Value

Instrument 2Instrument 1(Reference Standard)


Precision

Total variation of a measurement system

The measure of the natural variation of repeated measurementsmeasurements

Notation connected with precision are, random error, i ti d t t t tvariation spread, test retest error,

Repeatability rpt and Reproducibility rpdp y p p y p

222dtMS += rpdrptMS +


Precision: Repeatability rpt The inherent variability of the measurement system

Variation that occurs when repeated measurementsVariation that occurs when repeated measurements are made of the same variable under absolutely identical conditions

Same operator

Same set up Same set-up

Same units

S Same environmental conditions

Short-term

Estimated by the pooled standard deviation of the distribution of repeated measurementsp

Repeatability is usually less than the total variation of the system


the system

Repeatability rptRepeatability describes the variation between successive

measurements of the same part, same characteristic, by the samemeasurements of the same part, same characteristic, by the same

person using the same instrument. Also known as test - retest error;

M V l

used as an estimate of short-term variation.

Master Values

Excellentrepeatability

Poor repeatability


Precision: Reproducibility - rpd

The variation that results when different conditions are used to make the measurements

Different operatorsDifferent operators

Different set-ups

Different test units

Different environmental conditions

Long-term

Estimated by the standard deviation of the averages Estimated by the standard deviation of the averages of measurements from different measurement conditionsconditions


Reproducibility - rpdThe Reproducibility describes the difference in the average of the

measurements made by different persons using the same or differentmeasurements made by different persons using the same or different

instrument when measuring the identical characteristic.

Master Value

Person A

P BPerson B

Person B

Person C

Person A

Person C


Repeatability and Reproducibility What is influenced by system calibration?

RepeatabilityRepeatability

Reproducibility

B th Both

What is easier to fix? Low repeatability

Low reproducibility Low reproducibility

What would we expect at the use of a measuring system in the production? Variation caused by repeatabilityy p y

Variation caused by reproducibility

Both


Both

LinearityA measure of the difference in accuracy or precision over the range

of instrument capability.of instrument capability.

Gage 1:Linearity is an issue here

Gage 2:Linearity is not an issue here

AccuracyAccuracy

00

M t U it M t U itMeasurement Units Measurement Units


Stability

The distribution of measurements

i t t d di t bl

Master Value

remains constant and predictable

over time for both mean and Master Value

Time 1standard deviation

N d ift dd hift l tTime 2

No drifts, sudden shifts, cycles, etc.

Evaluated using a trend chartEvaluated using a trend chart


Examples for Stability

89.689.5

Stable Measurement System

3 0SL=89 48

91

Drift in Measurement System

11

89.489.389.289.1

dual

Val

ue

X=89.07

3.0SL 89.48

90

89

dual

Val

ue

1

X=88.79

3.0SL=89.64

89.088.988.888.7

Indi

vid X 89.07

-3.0SL=88.66

88

87

Indi

vid

1 1

1

-3.0SL=87.95

1050

88.6

1050

87 1 1

Cyclic Measurement System

91.5

90.5

89 5lue

3.0SL=91.11Assuming a Spec. of 89 +/- 3

All values are within the Spec 89.588.5

87.5ndiv

idua

l Va

X=88.49

All values are within the Spec.

But take look to the trends in the charts.

1050

86.5

85.5

In

LB=86.00


Accuracy vs. Precision

Suppose we have a material with a true hardness of 5.0.

Method 1 gives the following 4 readings:Method 1 gives the following 4 readings:3.8; 4.4; 4.2; 4.0

M th d 2 i th f ll i 4 di Method 2 gives the following 4 readings:6.5; 4.0; 3.2; 6.3

Which method is more accurate? Which method is more accurate?

Which method is more precise?

Which method do you prefer? Why?


CorrelationA measure of linear association between two variables, e.g. two

different measurement methods or two different laboratories.different measurement methods or two different laboratories.

Correlation Example: Offset Correlation Example: N Off tOffset

2

No Offset

2

Met

hod

2

Met

hod

2

Method 1 Method 1Correlation Example:Correlation Example:

No Correlation

od 2

Met

ho


Method 1

Measurement Process The ideal measurement system will produce true measurements

every time it is used (Zero Bias, Zero Variance) Quality of the measurement system is characterized by statistical

propertiesTh t h ld i l d The measurement process should include: Certification and SOP

Control: calibration and monitoring

Periodic recertification

Repair and troubleshooting guide

Properties: Must be in statistical control

Variability must be small compared to product specifications

Variability must be small compared to process variation

Increments of measure should be about one-tenth respectively 1/20th


( 5% of the Tolerance) of product specification or process variation

Index - P/T (Precision to Tolerance)

Usually expressed as percent1006/ =TP MS percent100/ =

ToleranceTP

This value addresses, what percent of the Tolerance is taken by the measurement error (Precision).taken by the measurement error (Precision).

It includes both repeatability and reproducibility.

P/T < 10% Measurement system excellent

P/T < 20% Measurement system acceptableP/T 20% Measurement system acceptable

P/T < 30% Measurement system marginal acceptable

Note: 6 standard deviations account for 99,73 % of MS variation. 5.15 standard deviations account for 99% of MS variation. (is/was Industrial


standard). Minitab 15 uses 6 standard deviation by default .

Relation Precision to Tolerance

P/TProduct toleranceLSL USL

V i i f h

Product toleranceLSL USL

Variation of the measurement system

P/T = 20%system

P/T = 100%

P/T = 200%


Use and Interpretation of P/T The P/T-Ratio is the most often used evaluation for judging the precision of a measurement system.y

This is telling us how good the system is working regarding the specification.

But specifications might be to wide or to small

In the area of judging a product to an important customer specification the P/T-Ratio is the best evaluation.

The measurement error MS includes 2 components

Repeatability Variation caused by the measurement device Repeatability Variation caused by the measurement device

Reproducibility Variation caused by the operator

Regarding process capability and process improvement the single use of the P/T-Ratio can create a wrong feeling of

f t


safety.

Index %R&R (Precision to Total Variation)

100&% = MSRR Usually expressed as percent100&% =Total

RR

percent

This value addresses, what percent of the total variation is taken by the measurement error (Precision).taken by the measurement error (Precision).

It includes both repeatability and reproducibility.

%R&R < 10% Measurement system excellent

%R&R < 20% Measurement system acceptable%R&R 20% Measurement system acceptable

%R&R < 30% Measurement system marginal acceptable

%R&R is the best possibility for us, to asses the capability of measurement systems. Based on the %R&R we can decide where to concentrate for


improvements. Process or measurement system

Repeatability & Reproducibility

%R&R

Ob d i ti

%R&R = 20%

Observed process variation

Variation of the measurementmeasurement system

%R&R = 75%

%R&R = 100%


Use and Interpretation of %R&R

%R&R is the best tool for estimating the capability of a measurement system during process improvement activitiessystem during process improvement activities.

The %R&R ratio estimates the capability of the measurement system in relation to the total process variation. That means, it gives us the information on how good the real process / product variation can be identifiedprocess / product variation can be identified.

The %R&R includes 2 components

Repeatability Variation caused by the measurement device

Reproducibility Variation caused by the operatorReproducibility Variation caused by the operator

The use of statistical tools (and their conclusion) can be influenced by high %R&R values Therefore the capability ofinfluenced by high %R&R values. Therefore, the capability of the used measurement system should be checked in advance.


Comparison of P/T and %R&R


Ob d i tiObserved process variation

%R&R = 20%P/T = 20%

%R&R = 50%P/T = 50%

%R&R = 100%P/T = 100% %R&R = 100%P/T = 100%


Measurement system variation



Ob d i tiObserved process variation

P/T = 50% %R&R = 25%

P/T = 100% %R&R = 50%

P/T = 200% %R&R = 100%P/T = 200% %R&R = 100%





Observed process variationprocess variation

P/T = 10% %R&R = 20%

P/T = 20% %R&R = 40%

P/T = 50% %R&R = 100%P/T = 50% %R&R = 100%



Guidelines for the interpretation of %R&R

As soon %R&R has been calculated, you have to decide what shall be aligned on the target value first:decide what shall be aligned on the target value first: Process variation or


Ask yourself if the measurement system you want toAsk yourself, if the measurement system you want to use is capable for process improvement activities.

Th t t ill b d t f The measurement system will be adequate for process improvement activities if: if Cpo < 1,0 and %R&R < 70%

if 1.0 < Cpo > 1,5 and %R&R < 40%po if Cpo > 1,5 and %R&R < 30%


Example for the Relation % R&R - CpA simplified illustration of the relation % R&R and Cp

The evaluation of a process shows the following values:The evaluation of a process shows the following values:Mean = 204,33 with a StDev. = 2,31

The specification limits are LSL 194 and USL 210The specification limits are LSL = 194 and USL = 210This is equivalent to a process capability

with a cp = 1,16 and a cpk = 0.82

The measurement system evaluation results in a R&R = 54,67 %

If the portion of the measurement system variation can beIf the portion of the measurement system variation can be reduced, the overall capability can be predicted as follows:

Observed capability

Gage R&R cp cpk54,67 1,16 0,82 2

obs. Pact. P

RR1CC

&=

Observed capability

30 1,30 0,9215 1,36 0,960 1,38 0,98

2RR-1 &

True capability without measurement error


0 1,38 0,98 measurement error

Example for the Relation % R&R - CpThe difference between cp = 1,16 vs. cp = 1,39 seems to be not big at the first view but

1,40

1,35

Scatterplot of Cp vs %R&R

first view but

%R&R Cp ppmLT54 67 1 16 23204

,

1,30

1 25

Cp

54,67 1,16 2320450 1,20 1737245 1,24 1307740 1,27 10120

1,25

1,20

1 15,35 1,30 805930 1,33 660725 1,35 5581

60504030201001,15

%R&R

Scatterplot of ppm vs %R&R

20 1,36 485815 1,37 436010 1,38 40355 1 39 3852

25000

20000

23200

5 1,39 38520 1,39 3793 15000

10000pp

m

The ppm Long Term estimation shows the effect more obvious > 19 000 ppm

6050403020100

5000

%R&R

4000


> 19.000 ppm %R&R

R&R und CpWhat needs more attention?

Measurement system or process capability?

Process %R&R Cpo What shall be improved?

1 10% 0 5 Process1 10% 0,5 Process

2 40% 1,0 One or the other*

3 60% 1,5 Measurement system

4 70% 5,5 Measurement system*

Process 2: An improvement of the measurement system could beProcess 2: An improvement of the measurement system could be a higher benefit momentarily.

Process 4: Do we really need the effort to improve the


measurement system?

Summary

Discrimination P/T Ratio

Terms connected with accuracyT l

Precision to tolerance

R&R % True value Systematic Error / Bias Linearity

R&R % Repeatability and

ReproducibilityLinearity

Terms connected with precision Process capability related i i f h

p Repeatability Reproducibility

variation from the measurement system

Linearity

Stability (over Time) Stability (over Time)


javier garcia - verdugo sanchez - six sigma training - w1 analysis of measurement

Engineering