measurement systems analysis v1.1

1© 2001 Six Sigma Academy

Measurement Systems Analysis (MSA)


Why Measure?

• To understand a decision:• Meet standards & specifications• Detection/reaction oriented• Short-term results

• Stimulate continuous improvement:• Where to improve?• How much to improve?• Is improvement cost effective?• Prevention oriented• Long-term strategy

“If you cannot measure, you cannot improve!”– Taguchi


Measurement System As A Process

Cleanliness

Temperature

Dimension

Weight

Corrosion

Hardness

Conductivity

Density

Sequence

Timing

Positioning

Location

Set-up

Preparation

Cleanliness

Temperature

Design

Precision

Calibration

Resolution

Stability

Wear

Cleanliness

Vibration

Atmospheric pressure

Lighting

Temperature

Humidity

Compliance-procedureFatigue

AttentionCalculation errorInterpretation

SpeedCoordination

Vision

Knowledge-instrumentDexterity

PeopleEnvironment

MeasurementError

MethodMaterial Machine


What Is An MSA?

Scientific and objective method of analyzing the validity of a measurement system

• A “tool” which quantifies:

1. Equipment Variation

2. Appraiser (Operator) Variation

3. The Total Variation of a Measurement System• MSA is NOT just Calibration• MSA is NOT just Gage Repeatability & Reproducibility (R&R)

Measurement System Analysis is often a “project within a project”


MSA Relationship To DMAIC

Measurement Systems Analysis• Quantitative evaluation of tools and processes used in making

discrete or variable observations

Measurement Systems Control • Established, documented, and continuously carried out • Ensures measurement system maintains an acceptable status • Often referred to as “Long Term Gage Plan”

Define Improve ControlMeasure Analyze

Define Improve ControlMeasure Analyze


MSA - A Starting Point

Before you…• Make adjustments• Implement solutions• Run an experiment• Perform a complex statistical analysis

You should…• Validate your measurement systems• Validate data and data collection systems

MSA quantifies a major source of process variation


Measurement Systems

• Examples• Precision gage• Data collection form• Survey• School entrance exam• Customer satisfaction• On-time delivery report

What is your system ?


Types of Measurement System Analysis

• Operational Definitions• Walking the Process• Gage R&R

• Variable Data• Attribute Data


MSA – Operational Definitions

The Measurement System can be validated using Operational Definitions

constructed by the Project Team to ensure that all measurement takers completely understand what is expected during the data

collection phase.


Developing Operational Definitions

•Operational definitions are descriptions written in a way that ensures consistent interpretation by different people

•The operational definition method of description will be used throughout the DMAIC process


•Operational Definition• The technique of defining an item, process or characteristic using

Operational Definitions is an effective way to communicate between Team Members and other people involved in the project. Because Operational Definitions are so effective, the technique is used in a number of locations within the DMAIC process. Remember, to be effective, an Operation Definition must be written in a way that ensures consistent interpretation by different people.CC


General Example – Operational Definitions

• Examples of Operational Definitions for data collection:

• Record the date that the lease company written notification arrives in the dealership using an MM/DD/YY format.

• List any cosmetic preparation in excess of the standard pre-delivery process required to render the vehicle acceptable for retail consumer sale.

• Record the weight of each package of coffee in ounces by pouring the coffee into the filter and placing the filter and coffee on the scale tray.

• Record the length of time that coffee remains in the urn by recording the actual time of day each time the Brew button is pressed to recharge the urn. Use 24-hour clock and round to the nearest minute.


MSA – Walking the Process

“Walking the Process” is a method of conducting MSA when it is not possible

to perform a Gage R&R.


How to “Walk the Process”

• Develop Operational Definitions for each of the measures to be collected

• Train data collectors prior to beginning the data collection activity• Follow the process from beginning to end and monitor the data

collection activities to determine if data is being collected properly• Continue walking the process until the data compiled accurately

reflects the existing process


Components Of Measurement Error


Components Of Measurement Error

• Resolution/Discrimination• Accuracy (bias effects)• Linearity• Stability (consistency)• Repeatability-test-retest (Precision) • Reproducibility (Precision)

Each component of measurement error can contribute to variation, causing wrong decisions to be made


Categories Of Measurement Error Which Affect Location

Accuracy/ Bias

LinearityStability


Categories Of Measurement Error Which Affect Spread

RepeatabilityReproducibility

Precision


Can change be detected?

Resolution/Discrimination

Resolution?

Accuracy/Bias?

Linearity?

Stability?

Precision (R&R)?

OK

OK

OK

OK


Resolution

• Simplest measurement system problem• Poor resolution is a common issue• Impact is rarely recognized and/or addressed• Easily detected• No special studies are necessary • No “known standards” are needed


Definitions:

• Resolution/Discrimination• Capability to detect the smallest tolerable changes

• Inadequate Measurement Units• Measurement units too large to detect variation present

• Guideline: “10 Bucket Rule”• Increments in the measurement system should be one-tenth the

product specification or process variation


Resolution/Discrimination

1 2 3 4 5

Better Discrimination

1 2 3 4 5

Poor Discrimination

Same process output being measured

1.3

1


Resolution Actions

• Measure to as many decimal places as possible• Use a device that can measure smaller units• Live with it, but document that the problem exists• Larger sample size may overcome problem• Priorities may need to involve other considerations:

• Engineering tolerance• Process Capability• Cost and difficulty in replacing device


Accuracy/Bias

Measurements are “shifted” from “true” value

Resolution?

Accuracy/Bias?

Linearity?

Stability?

Precision (R&R)?

OK

OK

OK

OK


Accuracy/Bias

Difference between the observed average value of measurements and the master value

Master Value(Reference Standard)

AverageValueMaster value is an accepted,

traceable reference standard


Accuracy/Bias

x x

x

xx

x

x

x

x x x

x

xx

x

x

x

x

Less accurateMore accurate


Accuracy/Bias Actions

• Calibrate when needed/scheduled• Use operations instructions• Review specifications• Review software logic• Create Operational Definitions


Linearity

Measurement is not “true” and/or consistent across the range of the “gage”

Resolution?

Accuracy/Bias?

Linearity?

Stability?

Precision (R&R)?

OK

OK

OK

OK


Linearit

Full Range of GageReference Value

No Bias

Observed Average Value

Bias


Linearity Actions

• Use only in restricted range • Rebuild• Use with correction factor/table/curve• Sophisticated study required and will not be discussed in this course


Stability

Measurement drifts

Resolution?

Accuracy/Bias?

Linearity?

Stability?

Precision (R&R)?

OK

OK

OK

OK


Stability

• Measurements remain constant and predictable over time • For both mean and standard deviation

• No drifts, sudden shifts, cycles, etc. • Evaluated using control charts

Time 2

Time 1

Master Value(Reference Standard)


Stability Actions

• Change/adjust components• Establish “life” timeframe• Use control charts• Use/update current SOP


Precision

Repeatability and Reproducibility

Resolution?

Accuracy/Bias?

Linearity?

Stability?

Precision (R&R)?

OK

OK

OK

OK


Good Precision Poor Precision

Precision

2total = 2

product/process + 2repeatability + 2

reproducibility

Master Value

A B

Also known as Gage R&R


Repeatability (A Component Of Precision)

• Variation that occurs when repeated measurements are made of the same item under absolutely identical conditions

• Same: • Operator• Set-up• Units• Environmental conditions

• Short-term


Reproducibility (A Component Of Precision)

The variation that results when different conditions are used to make the measurements

• Different:• Operators• Set-ups • Test units• Environmental conditions • Locations• Companies

• Long-term


R&R Actions

Repeatability• Repair, replace, adjust equipment• SOP

Reproducibility• Training• SOP


Attribute Measurement Studies


Purpose Of Attribute MSA

• Assess standards against customers’ requirements • Determine if all appraisers use the same criteria • Quantify repeatability and reproducibility of operators • Identify how well measurement system conforms to a “known master” • Discover areas where:

• Training is needed• Procedures are lacking• Standards are not defined


Attribute MSA - Excel Method

• Allows for R&R analysis within and between appraisers• Test for effectiveness against standard• Limited to nominal data at two levels


DATE: 1/4/2001Attribute Legend

5 (used in computations) NAME: Acme Employee

1 Pass PRODUCT: Widgets2 Fail BUSINESS: Earth Products

Known PopulationSample # Attribute Try #1 Try #2 Try #1 Try #2 Try #1 Try #2

1 Pass Pass Pass Pass Pass Pass Pass2 Pass Pass Pass Pass Pass Pass Pass3 Pass Pass Pass Pass Pass Pass Pass4 Pass Pass Pass Pass Pass Fail Pass5 Fail Fail Fail Fail Fail Pass Fail6 Fail Pass Pass Pass Pass Pass Pass7 Pass Pass Pass Pass Pass Pass Pass8 Pass Pass Pass Pass Pass Pass Pass9 Fail Fail Fail Fail Fail Fail Fail10 Pass Pass Pass Pass Pass Pass Pass11 Pass Pass Pass Pass Pass Pass Pass12 Pass Pass Pass Pass Pass Pass Pass13 Pass Pass Pass Pass Pass Pass Pass14 Pass Pass Pass Pass Pass Fail Pass15 Fail Fail Fail Fail Fail Pass Fail16 Pass Pass Pass Pass Pass Pass Pass17 Pass Pass Pass Pass Pass Pass Pass18 Pass Pass Pass Pass Pass Pass Pass19 Fail Fail Fail Fail Fail Fail Fail20 Pass Pass Pass Pass Pass Pass Pass21 Pass Pass Pass Pass Pass Pass Pass22 Pass Fail Fail Pass Pass Pass Pass23 Pass Pass Pass Pass Pass Pass Pass24 Pass Pass Pass Pass Pass Fail Pass25 Fail Fail Fail Fail Fail Fail Fail26 Pass Pass Pass Pass Pass Pass Pass27 Pass Pass Pass Pass Pass Pass Pass28 Pass Pass Pass Pass Pass Pass Pass29 Fail Fail Fail Fail Fail Fail Fail30 Pass Pass Pass Pass Pass Pass Pass

Operator #1 Operator #2 Operator #3

Attribute MSA Example

Open file MSA-Attribute.xls

Microsoft Excel Worksheet


Scoring Example

• 100% is target for all scores • <100% indicates training required

• % Appraiser score = repeatability• Screen % Effectiveness Score = reproducibility• % Score vs. Attribute

• individual error against a known population• Screen % Effective vs. Attribute

• Total error against a known population

100.00% 78.57% 100.00%

78.57% 64.29% 71.43%

SCREEN % EFFECTIVE SCORE - > 57.14%

SCREEN % EFFECTIVE SCORE vs. ATTRIBUTE - > 42.86%

% APPRAISER SCORE - >

% SCORE VS. ATTRIBUTE - >


Statistical Report


Statistical ReportContinued


Attribute MSA – MINITAB™ Method

• Allows for R&R analysis within and between appraisers• Test for effectiveness against standard• Allow nominal data with two levels• Allows for ordinal data with more than two levels


MINITAB Method - Data Entry

• Same data as Excel example• Arranged in multiple columns• Data can also be stacked in single column


Attribute Study - MINITAB Analysis

Attribute MSA.mpj

Tool Bar Menu > Stat > Quality Tools > Attribute Gage R&R Study

Attribute MSA.MPJ


Attribute Study - MINITAB AnalysisContinued

1. Select “Multiple Columns” if data is un-stacked

2. Enter number of appraisers and trials

3. Enter name of column with “Known” 4. Select OK

1. Select “Single Column” if data is stacked


Attribute MSA - MINITAB Graphical Output

Bob Sue Tom

70

80

90

100

Appraiser

Per

cent

Within Appraiser

Bob Sue Tom

60

70

80

90

100

Appraiser

Per

cent

Appraiser vs Standard

Assessment AgreementDate of study: 1/03/2001Reported by: JoseName of product: XYZ ReportMisc:

[ , ] 95.0% CI

Percent

Lower variation within appraiser

Higher variation within appraiser

Lower variation appraiser vs. standard

Higher variation appraiser vs. standard

Not included if no “Known”


Attribute MSA – MINITAB Session Window Results

Each Appraiser vs. Standard

Assessment Agreement

Appraiser # Inspected # Matched Percent (%) 95.0% CI

Bob 30 28 93.3 ( 77.9, 99.2)

Sue 30 29 96.7 ( 82.8, 99.9)

Tom 30 24 80.0 ( 61.4, 92.3)

# Matched: Appraiser's assessment across trials agrees with standard.

Assessment Disagreement

Appraiser # Pass/Fail Percent (%) # Fail/Pass Percent (%) # Mixed Percent (%)

Bob 1 3.3 1 3.3 0 0.0

Sue 1 3.3 0 0.0 0 0.0

Tom 1 3.3 0 0.0 5 16.7

# Pass/Fail: Assessments across trials = Pass/standard = Fail.

# Fail/Pass: Assessments across trials = Fail/standard = Pass.

# Mixed: Assessments across trials are not identical.

Between Appraisers


# Inspected # Matched Percent (%) 95.0% CI

30 24 80.0 ( 61.4, 92.3)

# Matched: All appraisers' assessments agree with each other.

All Appraisers vs. Standard


# Inspected # Matched Percent (%) 95.0% CI

30 23 76.7 ( 57.7, 90.1)

# Matched: All appraisers' assessments agree with standard.

Individual vs. Standard

Disagreement assessment(repeatability)

Between appraisers(reproducibility)

Total agreement(against known)


MINITAB Method - Ordinal Data Entry

Ordinal MSA.mtw• Survey data rated on a 1 to 5 scale• Arranged in multiple columns

Minitab Worksheet


Attribute Study - Ordinal

Select “categories of the attribute data are ordered”

Analysis is same as 2 level data


Industrial Attribute MSA Exercise

• Evaluate samples supplied by instructor • Determine the screen and appraiser scores• Interpret the results• Recommend actions iGrafx Professional Document

attributecircles.MPJ


Variables Measurement Studies


Six Step Variables MSA

1. Conduct initial gage calibration (or verification)

2. Perform trials and data collection

3. Obtain statistics via MINITAB

4. Analyze, interpret results

5. Check for inadequate measurement units

6. On-going evaluation• What would be your long-term gage plan ?


Trials And Data Collection

• Generally two to three operators • Generally 5-10 process outputs to measure • Each process output is measured 2-3 times (replicated) by each

operator

Randomization is Critical

1 2 3

P1

1 2 3

P2

1 2 3

P3

1 2 3

P4

1 2 3

P5

Oper1

1 2 3

P1

1 2 3

...

1 2 3

P5

Oper2

1 2 3

P1

1 2 3

...

1 2 3

P5

Oper3


Randomization, Repeats, Replicates

Randomization• Runs are made in an arbitrary vs. patterned order • Average out effects of noise or unknown factors• Tradeoff - Invalid results versus slight inconvenience (if any)

Repeats • Running more than one sample of a single run• Results are averaged

Replication • Running entire experiment in a time sequence• MSA allows for repeatability study


Variables MSA - MINITAB Example

Variable MSA.mtwUSL=1.5LSL=0.5

Replicate 1 Replicate 2(Randomized order)

Variable MSA.MTW


MSA Using MINITAB

10 Process Outputs

3 Operators

2 Replicates• Have Operator 1 measure all

samples once (as shown in the outlined block)

• Then, have Operator 2 measure all samples once

• Continue until all operators have measured samples once (this is Replicate 1)

• Repeat these steps for the required number of Replicates

• Enter data into MINITAB in 3 columns as shown

USL=1.5LSL=0.5

Replicate 1 Replicate 2(Randomized order)


Manipulate The Data

Your data in MINITAB should initially look like this. You will need to STACK your data so that all like data is in one column only

Now you are ready to run the macro for data analysis

Use the commands> Manip> Stack > Stack Blocks of Columns

(Stack all Process Outputs, Operators, and Responses so that they are in one column only)


Note:c10, c11, c12 are the columns in which the respective data are found IN OUR EXAMPLE. You must have ALL data STACKED in these columns

Enter titles

Stacked And Ready For Analysis


Prepare The Analysis

Use the commands> Stat > Quality Tools > Gage R&R Study (Crossed)

Each process output measured by each operator

OR

> Gage R&R Study (Nested)For “destructive

tests” where each process output is measured uniquely by each operator


ANOVA method is preferred• Gives more information

Enter Gage Info and Options

Choose Method Of Analysis


USL - LSL=0.50

USL=1.0

LSL=0.6

USL=1.0

LSL=0.5

Adding Tolerance (Optional)

Upper Specification Limit (USL)

MinusLower Specification

Limit (LSL)

For this example:


Two-Way ANOVA Table With Interaction

Source DF SS MS F P

Part 9 2.05871 0.228745 39.7178 0.00000

Operator 2 0.04800 0.024000 4.1672 0.03256

Operator*Part 18 0.10367 0.005759 4.4588 0.00016

Repeatability 30 0.03875 0.001292

Total 59 2.24912

Gage R&R %Contribution

Source VarComp (of VarComp)

Total Gage R&R 0.004437 10.67

Repeatability 0.001292 3.10

Reproducibility 0.003146 7.56

Operator 0.000912 2.19

Operator*Part 0.002234 5.37

Part-To-Part 0.037164 89.33

Total Variation 0.041602 100.00

StdDev Study Var %Study Var %Tolerance

Source (SD) (5.15*SD) (%SV) (SV/Toler)

Total Gage R&R 0.066615 0.34306 32.66 68.61

Repeatability 0.035940 0.18509 17.62 37.02

Reproducibility 0.056088 0.28885 27.50 57.77

Operator 0.030200 0.15553 14.81 31.11

Operator*Part 0.047263 0.24340 23.17 48.68

Part-To-Part 0.192781 0.99282 94.52 198.56

Total Variation 0.203965 1.05042 100.00 210.08

Number of Distinct Categories = 4

MSA Output:

Gage name:Date of study:Reported by:

Tolerance:Misc:

00.30.40.50.60.70.80.91.01.1 1 2 3

Xbar Chart by Operator

Sam

ple

Mea

n

Mean=0.8075UCL=0.8796

LCL=0.7354

0

0.00

0.05

0.10

0.15 1 2 3

R Chart by Operator

Sam

ple

Ran

ge

R=0.03833

UCL=0.1252

LCL=0

1 2 3 4 5 6 7 8 9 10

0.4

0.5

0.6

0.7

0.8

0.9

1.0

1.1

Part

OperatorOperator*Part Interaction

Ave

rage

1 2

3

1 2 3

0.4

0.5

0.6

0.7

0.8

0.9

1.0

1.1

Operator

By Operator

1 2 3 4 5 6 7 8 9 10

0.4

0.5

0.6

0.7

0.8

0.9

1.0

1.1

Part

By Part

%Contribution

%Study Var %Tolerance

Gage R&R Repeat Reprod Part-to-Part

0

100

200

Components of Variation

Per

cent

Gage R&R (ANOVA) for Response

What does all this mean?

Session Window Graphs


Gage name:Date of study:Reported by:Tolerance:Misc:

00.30.40.50.60.70.80.91.01.1 1 2 3


Sam

ple

Mea

n

Mean=0.8075UCL=0.8796

LCL=0.7354

0

0.00

0.05

0.10

0.15 1 2 3

R Chart by Operator

Sam

ple

Ran

ge

R=0.03833

UCL=0.1252

LCL=0

1 2 3 4 5 6 7 8 9 10

0.4

0.5

0.6

0.7

0.8

0.9

1.0

1.1

Part


Ave

rage

1 2

3

1 2 3

0.4

0.5

0.6

0.7

0.8

0.9

1.0

1.1

Operator

By Operator

1 2 3 4 5 6 7 8 9 10

0.4

0.5

0.6

0.7

0.8

0.9

1.0

1.1

Part

By Part

%Contribution

%Study Var %Tolerance


0

100

200


Per

cent

Gage R&R (ANOVA) for ResponseMSA HealthSide

MSATroubleshoot

Side

Graphical Output - 6 Graphs In All

If only 1 operator, you won’t get these graphs

If nested study, you won’t get this

graph


Destructive Test

Gage name:Date of study:Reported by:Tolerance:Misc:

12

13

14

15

16

17

18 Bill ie Nathan Steve


Sam

ple

Mea

n

Mean=15.15

UCL=17.62

LCL=12.68

0

1

2

3

4

5 Bill ie Nathan Steve

R Chart by Operator

Sam

ple

Ran

ge

R=1.313

UCL=4.290

LCL=0

Billie Nathan Steve

13

14

15

16

17

18

Operator

By Operator

6 7 8 9 10 11 12 13 14 15 1 2 3 4 5Billie Nathan Steve

13

14

15

16

17

18

PartOperator

By Part (Operator)

%Contribution

%Study Var


0

50

100


Per

cent

Gage R&R (Nested) for Response

Operator by process output interaction is not applicable


Graphical Output Metrics

Chart Output• Xbar Chart: Shows sampled process output variety

• Reproducibility/bias/sensitivity• R Chart: Helps identify unusual measurements

• Resolution/repeatability • Bar Chart: Distinguishes R&R from Process Output to Process

Output• Components of variation

These are your leading graphical indicators


Misc:Tolerance:Reported by:Date of study:Gage name:

0

4

3

2

1

0

-1

321


Sam

ple Mea

n

Mean=1.401

UCL=3.654

LCL=-0.8528

0

4

3

2

1

0

321

R Chart by Operator

Sam

ple Ran

ge

R=1.198

UCL=3.915

LCL=0

10 9 8 7 6 5 4 3 2 1

2.0

1.5

1.0

Part


Ave

rage

1 2

3

321

3

2

1

Operator

By Operator

10 9 8 7 6 5 4 3 2 1

3

2

1

Part

By Part

%Contribution %Study Var

Part-to-PartReprodRepeatGage R&R

100

50

0

Components of VariationPerce

nt

Gage R&R (ANOVA) for ResponseBar Charts For Components

Much better

Needs help

Answers: “Where is the variation?”


Closer Look At The Xbar & R Charts

Xbar: at least 50% outside limits; R chart: in control

R Chart: Exposes gage Repeatability, resolution & stability

Xbar Chart: Test of sensitivity,

bias, & population variety


More R Chart Indicators

Both may indicate poor gage resolution

0

0.005

0.004

0.003

0.002

0.001

0.000

321

R Chart

Sam

ple

Ran

ge

R=4.33E-04

UCL=0.001416

LCL=0

0

0.15

0.10

0.05

0.00

321

R Chart by Operator

Sam

ple

Ran

ge

R=0.03833

UCL=0.1252

LCL=0

Randy

Rbar too small?

Plateaus


%Study

%Tolerance

%Contribution

Tabular Output Metrics

Number of Distinct Categories


% Contribution

• Measurement System Variation (R&R) as a percentage of Total Observed Process Variation

• Includes both repeatability and reproducibility

100* onContributi %TOTAL

2R&R

2

% Contribution

1%

9%


% Study Variation

• Looks at standard deviations instead of variance• Measurement System Standard Deviation (R&R) as a percentage of

Total Observed Process Standard Deviation• Includes both repeatability and reproducibility % Study

Variation

10%

30%

100* ationStudy Vari %TOTAL

R&R


AcceptanceCriteria

% Tolerance

• Measurement error as a percent of tolerance• Includes both repeatability and reproducibility• 5.15 Study Variation = 99% % Tolerance

10%

30%

100*Tolerance

*155 Tolerance %

P/T Tolerance to Precision

R&R


Distinct Categories

• Number of divisions that the Measurement System can accurately measure across the process variation

• How well a measurement process can detect process output variation- process shifts and improvement

• Less than 5 indicates Attribute conditions Number of DistinctCategories

10

5

R&R

Output Process*2 Categories Distinct of Number 2

2


Acceptability Summary

Tabular Method

% Contribution

1%

9%

ProcessControl% Study Variation

10%

30%

ProductControl

% Tolerance

10%

30%

Number ofDistinct

Categories

10

5

Desirable to Have All 4 Indicators Say “Go”


Keys To Successful MSA

• Define and validate measurement process• Identify known elements of the measurement process (operators,

gages, SOP, setup, etc.)• Clarify purpose and strategy for evaluation• Set acceptance criteria• Implement preventive/corrective action procedures• Establish on-going assessment criteria and schedules


Gage R&R - Which % Gage R&R Do I Use?

Depending on how variable your process is as compared to tolerance, your % Gage R&R values as a percent of Study variation, Tolerance and Process Variation will be quite different.

For example:

Consider a very stable process with low variability. Percent Tolerance will indicate that your gauge is very good (low % GRR) with high discrimination. On the other hand, when compared to process variation, the GRR will be poor (High % GRR).

As your process improves, you will need to move to more precise gauges if you wish to “see” decreases in variation due to the measuring system. On the other hand, if you truly only want to be able to tell when production is becoming less capable, then you are only interested in the precision of the gauge as it relates to your customer’s specification. See the Appendix at the end of this module for further examples


Buffalo, NY Plant

1.5 mmSix Sigma BB01/01/1998

Gage #020371

Misc:

Tolerance:Reported by:Date of study:

Gage name:

10 9 8 7 6 5 4 3 2 1

1.1

1.0

0.9

0.8

0.7

0.6

0.5

0.4

Part ID


Ave

rage

1

2

3

Gage R&R (ANOVA) for Measure

Buffalo, NY Plant


Gage #020371

Misc:


Gage name:

321

1.1

1.0

0.9

0.8

0.7

0.6

0.5

0.4

Oper ID

By Operator


Buffalo, NY Plant


Gage #020371

Misc:


Gage name:

10 9 8 7 6 5 4 3 2 1

1.1

1.0

0.9

0.8

0.7

0.6

0.5

0.4

Part ID

By Part


Gage R&R, Graphical Output:

• Operator * Part Interaction:• Shows if any given part(s) was hard to manage for any given operator(s)• Appears as though at least two of the operators had trouble measuring part #10• What would the ideal graph look like?

• By Operator:• Shows if any operator(s) had higher or lower readings (on average) than the others• What would the ideal graph look like?

• By Part:• Shows the ability of all of our operators to obtain the same readings for each part• Also shows the ability of our measurement system to distinguish between parts (amount of

overlap)• What would be the ideal graph look like?


Buffalo, NY Plant


Gage #020371

Misc:


Gage name:

0

1.11.00.90.80.70.60.50.40.3

321Xbar Chart by Operator

Sa

mpl

e M

ea

n

X=0.8075

3.0SL=0.8796

-3.0SL=0.7354

0

0.15

0.10

0.05

0.00

321R Chart by Operator

Sa

mpl

e R

ang

e

R=0.03833

3.0SL=0.1252

-3.0SL=0.000


Gage R&R, Xbar & R:

• How do we evaluate the X-bar & R-chart?

• Why are the data points out of control on the X-bar and R chart?


1.080.980.880.780.680.580.480.38

5 4 3 2 1Part Num

Me

asu

re

12

3

1.080.980.880.780.680.580.480.38

10 9 8 7 6Part Num

Me

asu

re

Runchart of Measure by Part, Operator

Minitab, Gage Run Chart:

• Generates a run chart of measurements by operator and part id• Allows us to visualize repeatability and reproducibility within and between

operator and part• The center line is the overall average of the parts

• STAT > Quality Tools > Gage Run Chart


0.0

1.0

2.0

3.0

4.0

5.0

6.0

0.5 0.6 0.7 0.8 0.9 1.0 1.1 1.2 1.3 1.4 1.5 1.6 1.7 1.8 1.9 2.0

Observed Cp

Act

ual

Cp 0%

10%

20%

30%

40%50%

60%

70%

P/T Ratio

P/T Ratio Effect on Capability


Which Might Need The Most Attention?

Measurement System or Process Capability

Process %R&R Obs. Cp Decision ?

1 10% 0.5 ?

2 60% 1.4 ?

3 60% 0.5 ?

4 70% 6.5 ?

% R&R Vs. Capability


Which Might Need The Most Attention?

Measurement System or Process Capability

Process %R&R Obs. Cp Decision ?

1 10% 0.5 Capability

2 60% 1.4 Measurement

3 60% 0.5 Maybe Both

4 70% 6.5 Measurement

*Note: Process Step 4

Would improving %R&R really be worth the effort ?

% R&R Vs. Capability


Handling Poor Gage Capability:

• If a dominant source of variation is repeatability (equipment), you need to replace, repair, or otherwise adjust the equipment.

• If, in consultation with the equipment vendor or upon searches of industry literature, you find that the gage technology that you are using is “state-of-the-art” and it is performing to its specifications, you should still fix the gage. One temporary solution to this problem is to use signal averaging (see next page).

• If a dominant source of variation is operator (reproducibility), you must address this via training and definition of the standard operating procedure. You should look for differences between operators to give you some indication as to whether it is a training, skill, and/or procedure problem.

• Evaluate the specifications. Are they reasonable?• If the gage capability is marginal (as high as 30% of study variation)

and the process is operating at a high capability (Ppk greater than 2), then the gage is probably not hindering you and you can continue to use it.


• Note: If you want to decrease your gage error take advantage of the standard error square root of the sample.

• The signal averaging technique uses:

n

1

• n = the number of repeat measures taken on the same part

• the measurement = the average of “n” readings

• Example: a gage error of 50% can be cut in half if your point estimate is an average of 4 repeat measurements

2/14

1 • This technique should be used as a short term approach to

perform a study, but you must fix the gage.

xxxxxxxxxx

x

xxxx

x xx

Distributionof Means

Distributionof Individuals

Controlling Repeatability:


The Signal-to-Noise Ratio (S/N Ratio) relates the product variation to the measurement system variation. The S/N Ratio should be as large as possible.

The Discrimination Index provides the number of divisions that the Measurement System can accurately measure across the part (sample) variation. If this index is less than 4, then it is inadequate to provide data for a study. If the index is 4, then it is equivalent to a go/no-go gage. We would like to see the value of 5 or greater.

S / N Ratio

P

MS

Discrim=

p

ms

* .141

Other Statistical Indexes


Effects of P/T and S/N Ratios

• The effect of P/T on Cpk• Large P/T reduces the process Cpk from the true value to

some smaller observed value. • The effect of P/T on part assessment

• Large P/T increases the probability that we will misclassify product as defective when it’s really good and vice versa.

• The effect of S/N ratio on control chart sensitivity• small S/N increases the time before an out-of-control

process is detected by a control chart (refer to X-bar & range)

• The Effect of the Discrimination Index• If the Index = 2, only attribute data is available and sample

sizes must be larger.• If the Index is 5 to 10, then discrimination is finer and

sample sizes can be smaller.


Calibration Steps

• Determine if the measurement system needs to be recalibrated

• Determine the minimum number of measurements needed to make this decision

• Take data and make decision• If yes, recalibrate system• Why don’t we just recalibrate?

• Normal variation causes the measurement to be slightly different each time it is used

• Recalibration should be done only when the measurements are off by more than the normal variation

• Recalibrating a system when it is not needed can increase the variability in the measurements


Appendix


Interpreting Variables GR&R Results

Presented on the following slides are four Variable Gage R&R results - % Study (P/TV - Precision to Total Variation) and % Tolerance (P/T - Precision to Tolerance) along with a representative graphical illustration to help visualize the results and any required action to improve the Measurement System. Also discussed is the effect of the GR&R on Cp.

– There are an infinite number of GR&R results(combinations of % Study and % Tolerance) use these four relatively extreme scenarios to help you determine what actions that you need take given your own results. Remember we are looking for GR&R results of < 10%, although anything less than 30% is considered barely acceptable (proceed with caution).

– These graphs are not drawn to scale, therefore, when reviewing this information do not compare the relative size of the histograms between the scenarios, rather, compare the histograms within the scenario to the Spec Limits. Actual data was not used to create these histograms.

– These examples assume 10 parts were selected that represent the long-term capability of the process being investigated. Three operators, 2 trial.

– No assumptions have been made as to the problem with the Measurement System.

– Actual data was not used to calculate the Cp indices. They were visually estimated, but are assumed reasonable.


Scenario #1

9080706050

15% - % Study15% - % Tolerance

Gage Contribution(Precision)

Part Contribution(Part Variation)

Observed(Total Variation)

In this example we observe a GR&R result that is acceptable, where the % Study Variation is the same as the % Tolerance Variation. The results are the same because the relative size of the Total Variation -PV (5.15*sTotal) and the Tolerance- T (USL - LSL) are the same. Therefore, when we take the P/TV or P/T ratio, where P is the Precision of the Gage (5.15* sms) it is well below 30%.

This gage is deemed acceptable, no action is required. The only action is to improve the Process Capability.

Furthermore, the observed Cp of this process is probably close to 1, as it appears 6 standard deviations of the process can fit inside the tolerance once. Finally, as a result of the acceptable GR&R values the observed Cp (what we measure) is considered to be the actual Cp.

LSL USL

Tolerance


Scenario # 2


9080706050




Tolerance

In this example we observe a GR&R where the % Study Variation is the same as the % Tolerance Variation, however the results are extremely unacceptable. The results are the same because the relative size of the Total Variation -TV (5.15*sTotal) and the Tolerance- T (USL - LSL) are the same. Therefore, when we take the P/TV or P/T ratio, where P is the gage contribution (5.15* sms) it is very much above 30%. Thus, indicating the Measurement System can not effectively discern part to part differences. The impact of a poor GR&R results is to inflate the variability of the product standard deviation.

In this example we absolutely need to fix the Measurement System!!!

Finally the observed Cp of this process (using this poor gage) is probably close to 0.5, as it appears that only half of the 6 standard deviations of the process can fit inside the tolerance. The actual Cp is probably much higher maybe closer to 1 or 1.5. If the measurement system were improved and deemed acceptable the observed Cp would reflect actual Cp.

LSL USL


Scenario #3

9080706050




70% - % Study 5% - % Tolerance

LSL USL

Tolerance

Here we observe a GR&R where the % Study Variation is extremely unacceptable and the % Tolerance Variation is very acceptable. How can this be? In this example the Gage Precision - P (5.15* sms) compared to the Total Variation - TV (5.15*sTotal) P/TV is quite large - 70%. However, when we compare the Gage Precision with to the Tolerance (USL - LSL) P/T we observe a very acceptable GR&R - 5%.

Do we need to fix our Measurement System? Well that depends, if we are still looking for process improvement then we should fix the measurement system. If, however, we do not need to improve the process capability then our measurement system is acceptable.

In this example our observed Cp is probably close to 2 (99.73% of our process variability close can fit into our customer tolerance), where as the actual Cp may be significantly higher. If for some reason the PV began to increase to the size of the Tolerance then we would observe our gage as acceptable.


Scenario #4

9080706050





LSL USL

Tolerance

Here we observe a GR&R where the % Study Variation is acceptable and the % Tolerance Variation is very unacceptable. How can this be? In this example the Gage Precision - P (5.15* sms) compared to the Total Variation - PV (5.15*sTotal) P/TV is very small - 5%. However, when we compare the Gage Precision with to the Tolerance (USL - LSL) P/T we observe a very large GR&R - 70%%.

Do we need to fix our Measurement System? Yes, we need to fix the measurement system. In this example, the observed Cp will be the actual Cp and it is probably about 0.2 to 0.4. However, as we work our Six Sigma project and reduce the variability of our KPOV to improve our Process Capability our % Study Variation will become worse (% Tolerance, will remain constant). When our Process Variation is the same size as the Tolerance, both GR&R’s will be 70% and our observed Cp will not reflect the actual. Therefore improvement of the measurement system is required.

measurement systems analysis v1.1

Education

measurement system msa

sigma academy5

sigma academy8

sigma academy14

sigma academy10

sigma academy6

sigma academy15

sigma academy2