UNCLASSIFIED / FOUO

UNCLASSIFIED / FOUO

UNCLASSIFIED / FOUO

UNCLASSIFIED / FOUO

National GuardBlack Belt Training

Module 23

Measurement System Analysis (MSA)

Introduction

UNCLASSIFIED / FOUO

UNCLASSIFIED / FOUO

2

CPI Roadmap – Measure

Note: Activities and tools vary by project. Lists provided here are not necessarily all-inclusive.

TOOLS

•Process Mapping

•Process Cycle Efficiency/TOC

•Little’s Law

•Operational Definitions

•Data Collection Plan

•Statistical Sampling

•Measurement System Analysis

•TPM

•Generic Pull

•Setup Reduction

•Control Charts

•Histograms

•Constraint Identification

•Process Capability

ACTIVITIES• Map Current Process / Go & See

• Identify Key Input, Process, Output Metrics

• Develop Operational Definitions

• Develop Data Collection Plan

• Validate Measurement System

• Collect Baseline Data

• Identify Performance Gaps

• Estimate Financial/Operational Benefits

• Determine Process Stability/Capability

• Complete Measure Tollgate

1.Validate the

Problem

4. Determine Root

Cause

3. Set Improvement

Targets

5. Develop Counter-

Measures

6. See Counter-MeasuresThrough

2. IdentifyPerformance

Gaps

7. Confirm Results

& Process

8. StandardizeSuccessfulProcesses

Define Measure Analyze ControlImprove

8-STEP PROCESS

UNCLASSIFIED / FOUO

UNCLASSIFIED / FOUO

33

Learning Objectives

Understand the importance of good measurements

Understand the language of measurement

Understand the types of variation in measurement systems

UNCLASSIFIED / FOUO

UNCLASSIFIED / FOUO

4

Exercise: The Three Rs

UNCLASSIFIED / FOUO

UNCLASSIFIED / FOUO

55

Examples

The Hale Koa Hotel manager wants to reduce customer check-in time

The VA wants to reduce VA Home Loan Guarantee Program processing errors

The Army Community Service organization wants to improve its customer service performance

A VA Hospital is interested in finding ways to improve in-patient and out-patient care

UNCLASSIFIED / FOUO

UNCLASSIFIED / FOUO

6Measurement System Analysis

Why Is MSA Important?

Our ability to assess the performance of a process we wish to improve is only as good as our ability to measure it

The measurement system is our “eyes” for our process

We need to be able to see the performance of our process clearly in order to improve it

Sometimes, improving the ability to measure our process results in immediate process improvements

Can you trust your measurements to tell you the truth?

UNCLASSIFIED / FOUO

UNCLASSIFIED / FOUO

7

Sources Of Observed Process Variation

The variation due to the measurement system must be identified first, then separated from actual process variation

Observed VariationObserved Variation

Actual Process VariationActual Process Variation Measurement VariationMeasurement Variation

Variance

Due to Instrument

Variance

Due to Instrument

- Repeatability

- Calibration

- Stability

- Linearity

Variance

Due to Operators

Variance

Due to Operators

- Reproducibility

- Long-term Process Variation

- Short-term Process Variation

Observed VariationObserved Variation

Actual Process VariationActual Process Variation Measurement VariationMeasurement Variation

Variance

Due to Instrument

Variance

Due to Instrument

- Repeatability

- Calibration

- Stability

- Linearity

Variance

Due to Operators

Variance

Due to Operators

- Reproducibility

Variance

Due to Operators

Variance

Due to Operators

- Reproducibility

- Long-term Process Variation

- Short-term Process Variation

UNCLASSIFIED / FOUO

UNCLASSIFIED / FOUO

8

Variation Is Additive

s2 Observed = s2 Measurement + s2 Part + s2 Error

s2 Measurement = s2 Observed – s2 Part – s2 Error

s2 Measurement = s2 Repeatability + s2 Reproducibility + s2 Error

Actual values

Measured values

UNCLASSIFIED / FOUO

UNCLASSIFIED / FOUO

9

How might measurementvariation affect these decisions?

Verify process conformity to specifications

Assist in continuous

improvement activities

What if the amount of measurement variation

is unknown

?

Process

Measurement

Process

Measurement

Measurement variation can make our process capabilities appear worse than they are.

Why Worry About Measurement Variation?

Consider the reasons why we measure:

UNCLASSIFIED / FOUO

UNCLASSIFIED / FOUO

10

Measurement Variation

Measurement Variation is broken down into two components: (The two Rs of Gage R&R)

Reproducibility (Equipment or Gage or Operator Variability)

Different individuals get different measurements for the same thing

Repeatability (Equipment or Gage or Operator Variability)

A given individual gets different measurements for the same thing when measured multiple times

The tool we use to determine the magnitude of these two sources of measurement system variation is called Gage R&R

UNCLASSIFIED / FOUO

UNCLASSIFIED / FOUO

11

Inspector A

Inspector B Inspector C

Reproducibility (Operators’ Precision)

Reproducibility is the variation in the average of the measurements made by different operators using the same measuring instrument when measuring the identical characteristic on the same part

222

ogm sss

UNCLASSIFIED / FOUO

UNCLASSIFIED / FOUO

12

Ideal Process Target

Repeatability (Gage Precision)

Repeatability is the variation between successive measurements of the same part, same characteristic, by the same person using the same equipment (gage). Also known as test /re-test error, used as an estimate of short-term variation.

222

ogm sss

UNCLASSIFIED / FOUO

UNCLASSIFIED / FOUO

13

True Values

Bias

Repeatability Reproducibility

Gage R&R Stability

Measurement Error

Observed Measurements

Gage R & R variation is the percentage that measurement variation (repeatability and

reproducibility) represents of the variation observed in the process

Generally recognized criteria for gage acceptability is when Gage R & R variability to process variability is :

Under 10%: Acceptable gage

10% to 30%: Might be acceptable

Over 30%: Gage is unacceptable and should be corrected or replaced

Measurement Error

Operator Operator * Part

Discrimination Linearity

UNCLASSIFIED / FOUO

UNCLASSIFIED / FOUO

14

Master Value (Reference Standard)

Average Value

Bias (Instrument Accuracy)

Bias is the difference between the observed average value of measurements and the master value. The master value is determined by precise measurement typically by calibration tools linked to an accepted, traceable reference standard.

UNCLASSIFIED / FOUO

UNCLASSIFIED / FOUO

15

Time One

Time Two

Stability

Stability = If measurements do not change or drift over time, the instrument is considered to be stable

UNCLASSIFIED / FOUO

UNCLASSIFIED / FOUO

1616

Ruler

Caliper

Micrometer

.28

.279

.2794

.28

.282

.2822

.28

.282

.2819

.28

.279

.2791

Discrimination

Discrimination is the capability of detecting small changes in the characteristic being measured

The instrument may not be appropriate to identify process variation or quantify individual part characteristic values if the discrimination is unacceptable

If an instrument does not allow differentiation between common variation in the process and special cause variation, it is unsatisfactory

UNCLASSIFIED / FOUO

UNCLASSIFIED / FOUO

17

Linearity

A measure of the difference in bias (or offset) over the range of the sample characteristic the instrument is expected to see determines linearity. If the bias is constant over the range of measurements, then linearity is good.

Over what range of values for a given characteristic can the device be used?

When the measurement equipment is used to measure a wide range of values, linearity is a concern.

Measurement Scale

LowEnd

HighEnd

Measurement Variation

UNCLASSIFIED / FOUO

UNCLASSIFIED / FOUO

18

Master Value

Average Value

Inspector A

Inspector B

Inspector C

Master Value

(Reference Standard)

Time One

Time Two

Instrument 2

Instrument 1

.28

.279

.2791

Name That Problem!

1. Discrimination2. Bias/Accuracy3. Repeatability4. Reproducibility5. Instrument Bias6. Stability

UNCLASSIFIED / FOUO

UNCLASSIFIED / FOUO

19

Measurement Systems Analysis TemplateThe Measurement System used to collect data has been calibrated and is considered to have no potential for significant errors. The data collection tool is reliable, can be counted on, has good resolution, shows no signs of bias and is stable.

Type of Measurement

ErrorDescription Considerations to this Project

Discrimination (resolution)

The ability of the measurement system to divide measurements into “data categories”

Work hours can be measured to <.25 hours. Radar usage measure to +- 2 minute.

BiasThe difference between an observed average measurement result and a reference value

No bias - Work hours and radar start-stop times consistent through population.

Stability The change in bias over timeNo bias of work hours and radar usage data.

Repeatability The extent variability is consistentNot an issue. Labor and radar usage is historical and felt to be accurate enough for insight and analysis.

ReproducibilityDifferent appraisers produce consistent results

Remarks in usage data deemed not reproducible, therefore were not considered in determining which radars were used in each op

Variation The difference between parts N/a to this process.Required Deliverable

- Example -

UNCLASSIFIED / FOUO

UNCLASSIFIED / FOUO

20

Measurement Systems Analysis Template

The Measurement System is acceptable with the Total Gage R&R % Contribution <10%

Percent

Part-to-PartReprodRepeatGage R&R

100

50

0

% Contribution

% Study Var

Sam

ple

Range 0.10

0.05

0.00

_R=0.0417

UCL=0.1073

LCL=0

1 2 3

Sam

ple

Mean

10.00

9.75

9.50

__X=9.7996

UCL=9.8422

LCL=9.7569

1 2 3

Part

10987654321

10.00

9.75

9.50

Operator

321

10.00

9.75

9.50

Part

Average

10 9 8 7 6 5 4 3 2 1

10.00

9.75

9.50

Operator

1

2

3

Gage name:

Date of study :

Reported by :

Tolerance:

Misc:

Components of Variation

R Chart by Operator

Xbar Chart by Operator

Response by Part

Response by Operator

Operator * Part Interaction

Gage R&R (ANOVA) for ResponseGage R&R

%Contribution

Source VarComp (of VarComp)

Total Gage R&R 0.0015896 3.70

Repeatability 0.0005567 1.29

Reproducibility 0.0010330 2.40

Operator 0.0003418 0.79

Operator*Part 0.0006912 1.61

Part-To-Part 0.0414247 96.30

Total Variation 0.0430143 100.00

Study Var %Study Var

Source StdDev (SD) (6 * SD) (%SV)

Total Gage R&R 0.039870 0.23922 19.22

Repeatability 0.023594 0.14156 11.38

Reproducibility 0.032140 0.19284 15.50

Operator 0.018488 0.11093 8.91

Operator*Part 0.026290 0.15774 12.68

Part-To-Part 0.203531 1.22118 98.13

Total Variation 0.207399 1.24439 100.00

Number of Distinct Categories = 7

- Example -Optional BB Deliverable

UNCLASSIFIED / FOUO

UNCLASSIFIED / FOUO

21

Takeaways

It is important to be able to rely on the accuracy and precision of the measurement system to make good decisions

Understand the various types of measurement system variation

Eliminate as much of the variation in the measurement system as possible to focus on and improve the true cause of variation in process performance

UNCLASSIFIED / FOUO

UNCLASSIFIED / FOUO

22

What other comments or questions

do you have?