10. measurement system analysis (msa)

QUALITY TOOLS & TECHNIQUES

1

TQ TMEASUREMENT SYSTEM

ANALYSIS (MSA)By: -

Hakeem–Ur–RehmanCertified Six Sigma Black Belt (SQII – Singapore)

IRCA (UK) Lead Auditor ISO 9001MS–Total Quality Management (P.U.)

MSc (Information & Operations Management) (P.U.)IQTM–PU

INTRODUCTION TO MEASUREMENT SYSTEM ANALYSIS

2

So far we have learned that the heart and soul of Six–Sigma is that it is adata–driven methodology. How do you know that the data you have used is accurate and precise? How do you know if a measurement is a repeatable and reproducible?

How good are these?

Also known as Measurement System Evaluation (MSE)

Anytime you measure the results of a process you will observe some variation.This variation comes from two sources:

Parts made by any process Method of making measurements

Thus, measuring the same part repeatedly does not result in identicalmeasurement.

MEASUREMENT SYSTEM ANALYSIS: Definition

3

A measurement system may be defined as “thecollection of instruments or gages, standards,operations, methods, fixtures, software, personnel,environment and assumptions used to quantify a unitof measure or fix assessment to the featurecharacteristic being measured; the complete processused to obtain the measurement.

(Automotive Industry Action Group – AIAG 2002 Standard)

MEASUREMENT SYSTEM ANALYSIS…

Whenever you measure anything, the variation that you observe canbe segmented into the following components…

All measurement systems have error. If you don’t know how much of thevariation you observe is contributed by your measurement system, you cannotmake confident decisions.

AccuracyPrecision

Repeatability Reproducibility

Measurement System ErrorUnit-to-unit (true) Variation

Observed Variation

Stability Bias Linearity

ACCURACY Vs PERCISION

5

Two categories of measurement error. ACCURACY refers to how close measurements are to the

"true" value, while PRECISION refers to how close measurements are

to each other.

PERCISION METRICS

6

A precise metric is one that returns the same value of a

given attribute every time an estimate is made.

Precise data are independent of who estimates them or

when the estimate is made.

Precision can be partitioned into two components:

– Repeatability

– Reproducibility

Repeatability and Reproducibility = Gage R+R

PERCISION METRICS…

7

Repeatability is the variation in measurements obtained

with one measurement instrument used several times byone appraiser while measuring the identical characteristic onthe same part.

For example:– Manufacturing: One person measures the purity of multiple samples of the

same vial and gets different purity measures.

– Transactional: One person evaluates a contract multiple times (over aperiod of time) and makes different determinations of errors.

Repeatability

Y

PERCISION METRICS…

8

Reproducibility is the variation in the average of the

measurements made by different appraisers using the samemeasuring instrument when measuring the identicalcharacteristic on the same part.

For example:– Manufacturing: Different people perform purity test on samples from the

same vial and get different results.

– Transactional: Different people evaluate the same contract and makedifferent determinations.

Reproducibility

Operator A

Operator B

Y

ACCURACY METRICS

9

LINEARITY: Linearity is an indication that “gauge response

increases in equal increments to equal increments ofstimulus, or, if the gauge is biased, that the biasremains constant throughout the course of themeasurement process”.

Linearity examines how accurate your measurementsare through the expected range of themeasurements. It answers the question: "Does mygage have the same accuracy across allreference values?”

STABILITY (or DRIFT): Stability (or Drift) is the total variation in the measurements obtained with a measurement system

on the same master or parts when measuring a single characteristic over an extended time period.(AIAG, 2002)

“Control Charts may be used to monitor the stability of a measurement system” “A signal of special cause variation on the charts could indicate the need for calibration of the

measurement system”

BIAS = Observed average value – Reference (True) value Bias, is the difference between the true value (reference value) and the observed average

of measurements on the same characteristic on the same part. (AIAG, 2002) It answers the question: "How accurate is my gage when compared to a

reference value?"

Nominal HighLow

**

*Reference Value (x)

B i

a s

(y

)

0.00

+ e

- e

y = a + b.xy: Bias, x: Ref. Value

a: Slope, b: Intercept

MEASUREMENT SYSTEM ANALYSIS

MEASUREMENT SYSTEM ANALYSIS USING MINITAB

MINITAB offers several commands to help youdetermine how much of your process variationarises from variation in your measurement system.

Gage R&R (Crossed), Gage R&R (Nested)examine measurement system precision.

Gage Linearity and Bias examines gagelinearity and accuracy.

MEASUREMENT SYSTEM ANALYSIS (Cont…)

12

BIAS AND LINEARITY (EXAMPLE): A manufacturer wants to know if a thermometer is taking

accurate and consistent readings at five heat settings (202°,204°, 206°, 208°, and 210°). Six readings are taken at eachsetting.

To find out if the thermometer is taking biasedmeasurements, subtract the individual readings from thereference value. The bias values for measurements taken atheat setting 202° are calculated in the below table.

Thermometer reading

Actual temperature

BIAS

The temperature readings at the202° heat setting are positivelybiased; the thermometer givesreadings that are higher thanthe actual temperature.

202.7 - 202 = 0.7

202.5 - 202 = 0.5

203.2 - 202 = 1.2

203.0 - 202 = 1.0

203.1 - 202 = 1.1

203.3 - 202 = 1.3

MEASUREMENT SYSTEM ANALYSIS (Cont…)

13

BIAS AND LINEARITY (EXAMPLE) (Cont…):

To interpret the linearity of the thermometer data, determine ifthe bias of the thermometer changes across the heat settings.If the data do not form a horizontal line on a scatter plot,linearity is present.

The scatter plot shows that biaschanges as the heat settingsincrease. Temperatures for lowerheat settings are higher than theactual temperatures, while readingsfor higher heat settings are lowerthan the actual temperatures.Because bias changes over the heatsettings, linearity is present in thisdata.

GAGE LINEARITY AND BIAS STUDY

14

EXAMPLE: A plant foreman chose five parts that represented the expected range of themeasurements. Each part was measured by layout inspection to determine its reference (master)value. Then, one operator randomly measured each part twelve times.You obtained the process variation (16.5368) from a Gage R&R study using the ANOVA method.Minitab displays the process variation in the Session window (Total Variation row of the Study Var(6 * SD) column).Open the worksheet GAGELIN.MTW

Choose Stat Quality Tools Gage Study Gage Linearity and Bias Study

GAGE LINEARITY AND BIAS STUDY: EXAMPLE (Cont…)

15

INTERPRETATION RULE: In (Gage Bias) Section; if“Average” P–Value < 5%So, Gage is Bias

In (Gage Linearity) Section;if “Slope” P–Value < 5%,So Gage is producingNonlinear Results

Good Gage must have more linearity than bias

Types of MSA’sMSA’s fall into two categories:

– Attribute

– Variable

Transactional projects typically have Attribute based measurementsystems.

Manufacturing projects generally use Variable studies more often, but douse Attribute studies to a lesser degree.

Attribute

– Pass/Fail

– Go/No Go

– Document Preparation

– Surface imperfections

– Customer Service Response

Variable

– Continuous scale

– Discrete scale

– Critical dimensions

– Pull strength

– Warp

GAUGE REPEATABILITY & REPRODUCIBILITY (R & R) STUDIES

17

Gage repeatability and reproducibility studiesdetermine how much of your observed processvariation is due to measurement system.

GAUGE REPEATABILITY & REPRODUCIBILITY (R & R) STUDIES USING MINITAB

18

Gage repeatability and reproducibility studies determine how much of your observedprocess variation is due to measurement system variation. MINITAB allows you toperform either crossed or nested Gage R&R studies. Use Gage R&R Study (Crossed) when each part is measured multiple times by

each operator. If all operators measure parts from each batch, then use Gage R&R Study

(Crossed). Use Gage R&R Study (Nested) when each part is measured by only one

operator. If each batch is only measured by a single operator, then you must use Gage R&R

Study (Nested). In fact, whenever operators measure unique parts, you have anested design.

MINITAB provides two methods for assessing repeatability and reproducibility:X–bar and R, and ANOVA. (ANOVA is better than X–bar and R method) The X–bar and R method breaks down the overall variation into three

categories: part-to-part, repeatability, and reproducibility. The ANOVA method goes one step further and breaks down reproducibility

into its operator, and operator-by-part (An Operator*Part interaction means that two or

more operators may measure different parts differently) components.

Gage R&R Study (Crossed)METHOD—Gage R&R Study (Crossed): ANOVA MethodEXAMPLE: Ten parts were selected that represent the expected range of the processvariation. Three operators measured the ten parts, three times per part, in a randomorder.Open the worksheet GAGEAIAG.MTWChoose Stat > Quality Tools > Gage Study > Gage R&R Study (Crossed).

The percent contribution from Part-To-Part is larger than that of Total Gage R&R, telling you that much of the variation is due to

differences between parts.

There are large differences between

parts, as shown by the non-level line.

Operator B measures parts inconsistently.

most of the points in the X-bar and Rchart are outside the control limits,indicating variation is mainly due todifferences between parts.

the differences between operators are smallcompared to the differences between parts, but aresignificant. Operator C appears to measure slightlylower than the others.

GUIDELINES FOR MEASUREMENT SYSTEM ACCEPTABILITY

20

According to the Automobile Industry Action Group(AIAG), you can determine whether your measurementsystem is acceptable using the following guidelines.

If the Total Gage R&R contribution in the %Study Varcolumn (% Tolerance, %Process) is: Less than 10% the measurement system is

acceptable. Between 10% and 30% the measurement

system is acceptable depending on the application,the cost of the measuring device , cost of repair, orother factors.

Greater than 30% the measurement system is

unacceptable and should be improved.

If you are looking at the %Contribution column, thecorresponding standards are: Less than 1% the measurement system is

acceptable. Between 1% and 9% the measurement system is

acceptable depending on the application, the cost ofthe measuring device, cost of repair, or otherfactors.

Greater than 9% the measurement system is

unacceptable and should be improved.

According to the AIAG , when the number of distinct categories is 5 or more

it represents an adequate measuring system.

% Tolerance or

% Study Variance

% Contribution System is…

10% or less

10% - 20%

20% - 30%

30% or greater

1% or less

1% - 4%

5% - 9%

10% or greater

Ideal

Acceptable

Marginal

Poor

Here are the Automotive Industry Action Group’s definitions for Gage

acceptance

Gage R&R Study (Crossed)

METHOD—Gage R&R Study (Crossed): ANOVA Method

Two-Way ANOVA Table With Interaction

Source DF SS MS F PPart 9 88.3619 9.81799 492.291 0.000Operator 2 3.1673 1.58363 79.406 0.000Part * Operator 18 0.3590 0.01994 0.434 0.974Repeatability 60 2.7589 0.04598Total 89 94.6471Alpha to remove interaction term = 0.25

If p-value for Operator * Part is > 0.25, Minitabomits this from the full model. Notice there isan ANOVA table without the interaction becausethe p-value was 0.974.

Two-Way ANOVA Table Without Interaction

Source DF SS MS F PPart 9 88.3619 9.81799 245.614 0.000Operator 2 3.1673 1.58363 39.617 0.000Repeatability 78 3.1179 0.03997Total 89 94.6471Gage R&R

%ContributionSource VarComp (of VarComp)Total Gage R&R 0.09143 7.76Repeatability 0.03997 3.39Reproducibility 0.05146 4.37Operator 0.05146 4.37

Part-To-Part 1.08645 92.24Total Variation 1.17788 100.00

Between 1% and 9% the measurement systemis acceptable depending on the application, thecost of the measuring device, cost of repair, orother factors. (AIAG)

Study Var %Study Var %ToleranceSource StdDev (SD) (6 * SD) (%SV) (SV/Toler)Total Gage R&R 0.30237 1.81423 27.86 22.68Repeatability 0.19993 1.19960 18.42 14.99Reproducibility 0.22684 1.36103 20.90 17.01

Operator 0.22684 1.36103 20.90 17.01Part-To-Part 1.04233 6.25396 96.04 78.17Total Variation 1.08530 6.51180 100.00 81.40

Number of Distinct Categories = 4

Between 10% and 30% the measurementsystem is acceptable depending on theapplication, the cost of the measuring device ,cost of repair, or other factors

number of distinct categories is 5represents an adequate measuring system

Gage R&R Study (Crossed)

22

METHOD—Gage R&R Study (Crossed): X–Bar & R MethodEXAMPLE: Ten parts were selected that represent the expected range of the processvariation. Three operators measured the ten parts, three times per part, in a randomorder.Open the worksheet GAGEAIAG.MTWChoose Stat > Quality Tools > Gage Study > Gage R&R Study (Crossed).

22

In the Components of Variation graph, a low percentageof variation (7.13%) is due to the measurement system(Gage R&R), and a high percentage (92.87%) is due todifferences between parts. (See your Session Window)Most of the points in the X–Bar Chart are outside thecontrol limits when the variation is mainly due to part-to-part differences.

Gage R&R Study (Crossed)METHOD—Gage R&R Study (Crossed): X–Bar & R MethodEXAMPLE: (Cont…)

Session window outputX–Bar and R method with GAGEAIAG data

%ContributionSource VarComp (of VarComp)Total Gage R&R 0.09357 7.13Repeatability 0.04073 3.10Reproducibility 0.05284 4.03


Process tolerance = 8


Part-To-Part 1.10412 6.62474 96.37 82.81Total Variation 1.14571 6.87428 100.00 85.93


INTERPRETATION:Look at the %Contribution column in the Gage R%R Table.The measurement system variation (Total Gage R&R) isslightly smaller than what was found for the same data withthe ANOVA method.

The % Study Var column shows that the Total Gage R&Raccounts for 26.70% of the study variation; again slightlysmaller than what was found using the ANOVA method. Insome cases, there is a greater difference in the twomethods because the ANOVA method considers significantOperator by Part interactions whereas the X–Bar and Rmethod does not.

Between 1% and 9% the

measurement system isacceptable depending on theapplication, the cost of themeasuring device, cost ofrepair, or other factors. (AIAG)

Between 10% and 30% the

measurement system isacceptable depending on theapplication, the cost of themeasuring device , cost of repair,or other factors

number of distinct categories is 5 represents an adequatemeasuring system

GAGE R&R STUDY (CROSSED) USING:i. ANOVA METHODii. X–BAR & R METHOD

Three parts were selected that represent theexpected range of the process variation. Threeoperators measured the three parts, threetimes per part, in a random order.

Open the file GAGE2.MTW

EXERCISE

Gage R&R Study (Nested)METHOD—Gage R&R Study (NESTED): X–Bar & R MethodEXAMPLE: Three operators each measured five different parts twice, for a total of 30measurements. Each part is unique to operator; no two operators measured the same part. Youdecide to conduct a gage R&R study (nested) to determine how much of your observed processvariation is due to measurement system variation .Open the worksheet GAGENEST.MTWChoose Stat > Quality Tools > Gage Study > Gage R&R Study (Nested).

Look at the Components of Variation Graph - located inupper left corner. Most of the variation is due tomeasurement system error (Gage R&R), while a lowpercentage of variation is due to differences between parts.

Look at the X-Bar Chart - located in the lower left corner.Most of the points in the X-Bar chart are inside the controllimits when the variation is mostly due to measurementsystem error.

Gage R&R Study (Nested)METHOD—Gage R&R Study (Nested):EXAMPLE: (Cont…)

Gage R&R (Nested) for ResponseSource DF SS MS F POperator 2 0.0142 0.00708 0.00385 0.996Part (Operator) 2 22.0552 1.83794 1.42549 0.255Repeatability 15 19.3400 1.28933Total 29 41.4094

Gage R&R%Contribution

Source VarComp (of VarComp)Total Gage R&R 1.28933 82.46Repeatability 1.28933 82.46Reproducibility 0.00000 0.00


Process tolerance = 10


Part-To-Part 0.52374 3.14243 41.88 31.42Total Variation 1.25045 7.50273 100.00 75.03


INTERPRETING THE RESULTS: Look at the %Contribution columns for Total

Gage R&R and Part-to-Part. The percentcontribution for differences between parts(Part-To-Part = 17.54) is much smaller than thepercentage contribution for measurementsystem variation (Total Gage R&R = 82.46).

The %Study Var column indicates that the TotalGage R&R accounts for 90.81% of the studyvariation. So, most of the variation is due tomeasurement system error; very little is due todifferences between part.

A 1 in number of distinct categories tells youthat the measurement system is not able todistinguish between parts.

QUESTIONS

10. measurement system analysis (msa)

Engineering