quality by design - dgs quality assurancedgs-qualityassurance.com/pdf/faro measurement...

Rolls-Royce

Willow Farm Faro Measurement Analysis, August 2008

Denis Sexton

Rolls-Royce

The purpose of this study is to investigate and identify the major sources of

measurement variation in the Willow Farm Faro Arm Measurement process. Once

the significant factors have been tested and established, a part specific inspection

method can be developed and a conventional gauge R&R study can be carried out

and evaluated.

This project follows on from an initial trial study carried out at Willow Farm in

September 2007. The recommendations from the previous study have been followed,

as well as some additional investigations raised from background information

gathered from the previous experience of the study leader. Additional reference

material was obtained from the “MSA guide” and Brainstorming activities from other

Faro studies elsewhere.

The first part of this measurement variation study (carried out over 2 days in April

2008) covers location indicators while the second part (carried out in August 2008) is

the GRR study looking at dispersion indicators. No previous location indicator studies

were available, so these were initiated prior to the spread variation study.

Resolution was the first issue to consider and approve ( which was that the Faro had

discrimination of at least one tenth of the values being appraised). The 12 step

DMAIC process was used for this study.

Overview

Rolls-Royce

The 12 Step DMAIC Process

Rolls-Royce

Step A: Identify Project CTQs

Critical CTQ = Robust Faro Measurement System

Faro Arm Gauge R&R studies demonstrate poor variation capability

D CIAM

Rolls-Royce

Step B: Team Charter

Business Case: (BIG Y – Net Sales, Customer Satisfaction, Compliance/Legal,

Employee Satisfaction).

Customer Satisfaction has been impacted negatively as the dimensional capability of the

Faro Arm at Willow farm does not meet the minimum gauge R&R requirements (as defined

by the RR performance measures). Of the four criteria defined, the previous study failed

on three indicators.

Problem Statement: With poor gauge capability the results obtained from the Faro arm

are compromised as it is not clear how much of the variation results from equipment, part,

inspector, method of measurement or an interaction of these.

Goal statement: To improve the capability of the Faro measurement system (or to provide

guidance on what needs to be done to improve it) by analytical evaluation to provide a

more streamlined inspection throughput, and by allowing more of the product currently

being measured by hand instruments to be moved to the Faro arm. An acceptable result

will show at least “Marginal Acceptability” on each of the four key performance measures.

Timeline: Define (Mar 08),Measure (April 08), Analyse ( July 08), Improve (July 08), Control

(August 08)

D CIAM

Rolls-Royce

Team members & roles

Denis Sexton – Study Leader Rolls Royce

Colin Ries – Sponsor Rolls Royce

David Attenborough - Team Member Ceva

Ray Armstrong - Team Member Ceva

Chris Welch - Team Member Ceva

Tech Advise / Support – Faro UK

Reference Material

Faro Basic Training Workbook

Previous Rolls Royce Gauge R&R Report from September 2007

AIAG Measurement Systems Analysis Reference Manual (second edition 1995)

D CIAM

Rolls-Royce

Step C: Process Map SIPOC D CIAM

Suppliers Inputs Outputs

(Providers of the

required resources)

(Resources required

by the process)

(Deliverables from

the process)

Requirements Requirements

Rolls-Royce Information Knowledge Input variables

Ceva Inspectors Experience for experimental

Faro UK Tech Support Training trials at WF

Internal Customer

(Derby/ Ansty)

Measurement

Faro Arm data for the

Work Station Knowledge Identification

Parts Experience of major

Drawings Training causes of

variation Final Customer

(shipyard)

Satisfactory Robust

Work instructions Correct Gauge Measurement

for part specific Current R&R Study System

Inspection Understood

Process Customers

(Top level description of the activity) (Stakeholders who place the

requirements on the outputs)

Determine

possible

causes of

variation.

Design

Experiments

and Trials to

investigate

causes of

variation

Develop

InspectionP

rocess to

minimise

variation

Customer Thinking

Project Flow

Rolls-Royce

Project Overview

Like every process, the distribution that can be used to describe the measurement system’s variation can be classified by:

Location Indicators• Bias -The difference between the reference value and the observed

average measurement. • Stability -The change in bias over time (used for calibration interval). • Linearity - The change in bias across the range of the instrument (or

bias with respect to size).

Spread or width Indicators• Repeatability -The variation in measurements obtained from one

appraiser on one part, with one instrument. • Reproducibility -The variation in average measurements from

multiple appraisers on one part, with one instrument.

Shape of Distribution- Normal distribution is assumed for this type of study. Normality assumption will be tested during this study and effects considered as part of the review of results..

D CIAM

Rolls-Royce

• 1. Using the Willow farm Faro Arm with calibrated reference standards, carry out a series of measurement screening trials over two days to confirm which of the factors (if any) identified from the initialbrainstorming activity are significant enough to use in a formal experiment. NB Follow guidelines used in MSA Reference Manual.

• 2. Design experimental matrix in Minitab to test all the data gathered from the measurement trials are processed efficiently.

• 3. Carry out the analysis including hypothesis and post hoc analysis for significance, and draw conclusion(s) from the data .

• 4. Use lessons learnt as an input into development of a sample inspection plan for the Faro Arm, on a specific part or family of parts.

• 5. Carry out Repeatability and Reproducibility with multiple appraisers and parts. Set up should be as per the “normal production inspection process”, which will be based on the information obtained in the location studies and the inspection plan / SOP developed.

• 6. Compare results with previous studies and make recommendations for continuous improvement steps as required.

Project Overview Task Order

D CIAM

Rolls-Royce

CTQ (Y)

Faro Arm Part Inspector

Probe size & method Day 1 & 2 Position Form Procedure Experience

Ball Dia Arm Position

Faro Capable of measuring within spec

D CIAMStep 1: Select CTQ Characteristics

Project CTQ (Y) = Robust Faro Measurement System

Project y = Reduce variability of Faro Arm by identifying and

removing assignable sources of variation from inspection process.

Point location

Rolls-Royce

CTQ Elements

# Of Defect Opportunities

Per Unit

Defect

Specifications

(Performance Standards)

Target

Project y Measure

Project y Operational

Definition

Output Characteristic

Output Unit

Step 2: Determine The Defect Definition

Robust Faro Measurement System

Acceptable Gage R&R Results as per RR requirements (see slide 18)

As per Part Drawing (TBA) and standardise work instruction (TBA)

R&R for SC dim’s shall be per RR Requirements (see slide 18)

Nominal dim’s defined on Part drawing.

USL and LSL as defined on Part Drawing.

Out of specification “RED INDICATOR” for any R&R per project measures.

2 per dimension chosen (Dimensions on Drawings, plus form).

D CIAM

Rolls-Royce

Step 3: Validate Initial Measurement System

On Y D CIAM

As the whole project is based around the validation of a measuring system, the validation

stage of this project involved determining the major sources of location variation, as this was

necessary prior to involving spread or width indicators. The initial trials involved the study

leader determining some baseline measurements under various operational conditions over

two days (29th and 30th April 2008). The output of the location studies should indicate what

factors may need to be considered when developing the standard measurement method

(inspection plan). The multiple appraiser measurements were carried out on Tuesday 5th

August 2008 with bore measurements as these types of features had demonstrated the

most variation in the past. Form was considered an important requirement with feature size.

Determine Possible

Sources Of Bias Variation

Location (Bias) Studies

Develop Standard Procedure

Select Representative

Measurand

Dispersion (spread) Studies

Develop

Standard

Measurement

Methodology

Rolls-Royce


On Y D CIAM

t Variation

Measuremen

Environment

Measurements

Methods

Material

Machines

Personnel

Experience

Training

Probe size

C ondition

C alibration

Master A rtifact

Weight of part

Ty pe of Part

Part Location

Gage Mounting

Part C lamping

Program Mode

O rder of Measure

V ariance of Data

Ty pe of F eature

Probing Strategy

Time of Day

C leanliness

Temperature

Gage Location

Cause and Effect Diagram for Faro Measurement Variation

Rolls-Royce

• Initial measurements were done on a 40mm calibrated ring gauge (as this was the

largest round calibrated reference standard available), and later with a 100mm slip

gauge as length reference. Finally a linearity study was conducted with slip piles

from 100 mm to 300 mm. The reference standards were measured multiple times at

various positions inside the measurement envelope of the arm, both parallel and

perpendicular to the surface plate/table (refer photos). This was to allow various

functions of the gauge to be tested. The procedure was repeated over two days to

check bias and stability. On day 1 the Faro was mounted in the middle of the table

and a 3mm ball used, while on day 2 a 6mm ball was used and the gauge location

on the table was moved to the corner. These reference standard parts as well as a

larger disc part (RU51317 for larger diameter and PCD measurements) were used

under the conditions identified by brainstorming (with Faro staff). Based on these

initial findings, predicator and response variables for a DOE were defined.

• A four factor two level factorial experimental orthogonal matrix was developed in

Minitab for analysis. Predicator variables chosen were probe diameter, gauge

position on table, day, and probing strategy. The probe diameter and Faro arm

position were blocked by day as it was considered a more efficient way to process

the data. Initial response variable was diameter. Based on the findings of the DOE a

2 One way ANOVA” . Later studies involve form (roundness) and linear distance.


On Y D CIAM

Rolls-Royce


On Y D CIAM

Rolls-Royce

Day 1 Faro Position Day 2 Faro Position


On Y

A

BB

A

D CIAM

A = Initial position of 40mm Ring Gauge

B = Secondary position of 40mm Ring Gauge

Rolls-Royce

Previous “Best Case” study on similar

diameter (170.688-171.166mm) gave a

R&R result of 3.83 % (tol) on a spec width

of 0.508mm and a process R&R of 33.39 %

RU 52314

Maximum number of “distinct categories” was 3 on this feature, and 4

on any feature of previous study. It was stated that the parts were chosen

at random, and did not represent the process or product variation.

Step 4: Establish Previous Product Capability

% Contribution was 11.15% on this feature

D CIAM

The previous studies did not include form (roundness,cylindricity,

straightness, or flatness).

Rolls-Royce

Step 5: RR Define Performance Objectives D CIAM

A = % Tolerance

Fail >30%

Marginal 20-30%

Good 10-20%

Ideal 0-10%

B = % Study Variation

Fail >30%

Marginal 20-30%

Good 10-20%

Idea 0-10%

C = Number of distinct

Categories

Fail 1-5

Marginal 6-10

Good 11+

D = % ContributionFail >9%

Marginal 1-9%

Good <1%

Rolls-Royce

Step 6: Identify Potential X’s D CIAM

t Variation

Measuremen

Environment

Measurements

Methods

Material

Machines

Personnel

Experience

Training

Probe size

C ondition

C alibration

Master A rtifact

Weight of part

Ty pe of Part

Part Location

Gage Mounting

Part C lamping

Program Mode

O rder of Measure

V ariance of Data

Ty pe of F eature

Probing Strategy

Time of Day

C leanliness

Temperature

Gage Location

Cause and Effect Diagram for Faro Measurement Variation

Rolls-Royce

Step 7: Screen Potential Causes (First Screening

Findings)

D CIAM

Change in Ball Diameter also included a change in day and

Faro arm position on table (three blocked variables).

Initial matrix included 4 factors at 2 levels with 1 response .

Post hoc assumptions of normality of residuals met

The most significant main effects on ring diameter were

number of Probe points and probe pattern BUT

No factor or interaction significant at the 95 % (or 99%) CI

Rolls-Royce

Step 7: Screen Potential Causes (First Screening) D CIAM

StdOrder RunOrder CenterPt Blocks Ball Dia Probe Points Probe Pattern Part Location Diameter

13 1 1 2 6 3 Equi B 40.0047

15 2 1 2 3 3 Non Equi B 39.9934

10 3 1 2 6 4 Equi A 39.9945

14 4 1 2 3 4 Equi B 39.9985

9 5 1 2 3 3 Equi A 39.9977

11 6 1 2 6 3 Non Equi A 39.9978

16 7 1 2 6 4 Non Equi B 40.0024

12 8 1 2 3 4 Non Equi A 39.9996

4 9 1 1 6 4 Non Equi A 40.005

3 10 1 1 3 3 Non Equi A 40.0529

1 11 1 1 6 3 Equi A 40.0129

8 12 1 1 3 4 Non Equi B 40.0084

5 13 1 1 3 3 Equi B 39.9989

6 14 1 1 6 4 Equi B 40.0039

7 15 1 1 6 3 Non Equi B 40.0302

2 16 1 1 3 4 Equi A 39.9959

Rolls-Royce

Me

an

of

Dia

me

ter

63

40.0100

40.0075

40.0050

40.0025

40.0000

43

Non EquiEqui

40.0100

40.0075

40.0050

40.0025

40.0000

BA

Ball Dia Probe Points

Probe Pattern Part Location

Main Effects Plot (data means) for Diameter

Step 7: Screen Potential Causes (DOE summary) D CIAM

Rolls-Royce

Ball Dia

43 Non EquiEqui BA

40.02

40.01

40.00

Probe Points

40.02

40.01

40.00

Probe Pattern

40.02

40.01

40.00

Part Location

3

6

Dia

Ball

3

4

Points

Probe

Equi

Non Equi

Probe Pattern

Interaction Plot (data means) for Diameter


Interaction evident between ball diameter

and probe pattern / part location on ring diameter.

Rolls-Royce

Te

rm

Effect

AB

A

BCD

D

CD

ABC

BC

AC

BD

AD

B

C

ACD

ABD

0.0200.0150.0100.0050.000

0.02005

A Ball Dia

B Probe Points

C Probe Pattern

D Part Location

Factor Name

Pareto Chart of the Effects(response is Diameter, Alpha = .05)

Lenth's PSE = 0.00763125


Factor and interactions not significant

At 95% confidence interval

Rolls-Royce

3 or

more

factors

Comparing Means

1 Factor

1-sample

Z-test

Two way

ANOVA

ANOVA

GLM

One way

ANOVA

1-sample

t-test

2-sample

t-test

Paired

t-test

1 Sample 2 Samples 2 or

more

samples

2 Factors

not known known independent paired

D CIAMStep 7: Hypothesis testing of means.

Rolls-Royce

Step 7: General Linear Model:

Length versus Ball Dia, Position

Factor Type Levels Values

Ball Dia fixed 2 3, 6

Probe Points fixed 2 3, 4

Probe Pattern fixed 2 Equi, Non Equi

Part Location fixed 2 A, B

Analysis of Variance for Diameter, using Adjusted SS for Tests

Source DF Seq SS Adj SS Adj MS F P

Ball Dia 1 0.0000023 0.0000023 0.0000023 0.01 0.924

Probe Points 1 0.0004030 0.0004030 0.0004030 1.65 0.225

Probe Pattern 1 0.0004275 0.0004275 0.0004275 1.75 0.213

Part Location 1 0.0000158 0.0000158 0.0000158 0.06 0.804

Error 11 0.0026868 0.0026868 0.0002443

Total 15 0.0035354

S = 0.0156287 R-Sq = 24.00% R-Sq(adj) = 0.00%

Unusual Observations for Diameter


D CIAM

Rolls-Royce

Step 7: Screen Potential Causes (First Screening)

Residual

Pe

rce

nt

0.040.020.00-0.02-0.04

99

90

50

10

1

N 16

AD 0.543

P-Value 0.137

Fitted Value

Re

sid

ua

l

40.01540.01040.00540.00039.995

0.04

0.02

0.00

-0.02

Residual

Fre

qu

en

cy

0.040.030.020.010.00-0.01-0.02

8

6

4

2

0

Observation Order

Re

sid

ua

l

16151413121110987654321

0.04

0.02

0.00

-0.02

Normal Probability Plot Residuals Versus the Fitted Values

Histogram of the Residuals Residuals Versus the Order of the Data

Residual Plots for Diameter

D CIAM

Rolls-Royce

Step 7: Screen Potential Causes

(Second Screening Findings) D CIAM

Secondary matrix included 2 factors, both at 2 levels.

Two Response variables (Diameter and Form).

Post hoc assumptions of normality or residuals show a

non-normal condition for diameter on the Anderson Darling Test Which

may need to be re-tested on a larger (+50) data set.

No factor or interaction significant at the 95 % (or 99%) CI

Rolls-Royce

StdOrder RunOrder CenterPt Blocks Ball Probing Ring Dia Form StdOrder RunOrder CenterPt Blocks

17 1 1 1 3 Equi 39.9991 0.0021 32 21 1 1 6 Non Equi 40.032 0.0019

10 2 1 1 6 Equi 40.0022 0.0013 15 22 1 1 3 Non Equi 39.9704 0.0017

34 3 1 1 6 Equi 40.0039 0.0016 21 23 1 1 3 Equi 40.0007 0.0035

29 4 1 1 3 Equi 39.9994 0.0001 8 24 1 1 6 Non Equi 39.9979 0.0037

14 5 1 1 6 Equi 40.0055 0.0001 23 25 1 1 3 Non Equi 39.9959 0.0006

28 6 1 1 6 Non Equi 39.9961 0.0011 6 26 1 1 6 Equi 40.0036 0.0002

25 7 1 1 3 Equi 39.995 0.0011 11 27 1 1 3 Non Equi 39.9946 0.0024

22 8 1 1 6 Equi 40.0049 0.0011 39 28 1 1 3 Non Equi 40.0166 0.0022

40 9 1 1 6 Non Equi 40.0254 0.0013 30 29 1 1 6 Equi 40.0027 0.0012

38 10 1 1 6 Equi 40.0035 0.0006 16 30 1 1 6 Non Equi 40.0028 0.0008

20 11 1 1 6 Non Equi 40.0031 0.0013 31 31 1 1 3 Non Equi 39.997 0.0019

1 12 1 1 3 Equi 40.0003 0.002 37 32 1 1 3 Equi 40.0001 0.0044

19 13 1 1 3 Non Equi 39.9882 0.0008 35 33 1 1 3 Non Equi 40.0091 0.002

4 14 1 1 6 Non Equi 40.0229 0.0043 33 34 1 1 3 Equi 39.9978 0.0029

3 15 1 1 3 Non Equi 39.9582 0.0007 36 35 1 1 6 Non Equi 40.0078 0.0021

26 16 1 1 6 Equi 40.0014 0.003 12 36 1 1 6 Non Equi 40.02 0.0003

27 17 1 1 3 Non Equi 39.9987 0.0023 18 37 1 1 6 Equi 40.0029 0.0015

24 18 1 1 6 Non Equi 39.9438 0.0006 13 38 1 1 3 Equi 39.9967 0.0028

9 19 1 1 3 Equi 39.9978 0.0003 7 39 1 1 3 Non Equi 40.0019 0.0016

5 20 1 1 3 Equi 39.9976 0.0056 2 40 1 1 6 Equi 40.0026 0.0033

Step 7: Screen Potential Causes (Second Screening) D CIAM

Rolls-Royce

Me

an

of

Rin

g D

ia

63

40.005

40.004

40.003

40.002

40.001

40.000

39.999

39.998

39.997

39.996

Non EquiEqui

Ball Probing

Main Effects Plot (data means) for Ring Dia

Probing

Me

an

Non EquiEqui

0.00250

0.00225

0.00200

0.00175

0.00150

3

6

Ball

Interaction Plot (data means) for Form

Me

an

of

Form

63

0.0021

0.0020

0.0019

0.0018

0.0017

0.0016

0.0015

Non EquiEqui

Ball Probing

Main Effects Plot (data means) for Form

Probing

Me

an

Non EquiEqui

40.006

40.004

40.002

40.000

39.998

39.996

39.994

39.992

3

6

Ball

Interaction Plot (data means) for Ring Dia


Rolls-Royce

3 or

more

factors

Comparing Means

1 Factor

1-sample

Z-test

Two way

ANOVA

ANOVA

GLM

One way

ANOVA

1-sample

t-test

2-sample

t-test

Paired

t-test

1 Sample 2 Samples 2 or

more

samples

2 Factors

not known known independent paired

D CIAMStep 7: Hypothesis Testing of Means

Rolls-Royce

Step 7: Two Way ANOVA: Ring Diameter

Two-way ANOVA: Ring Dia versus Ball, Probing

Source DF SS MS F P

Ball 1 0.0007217 0.0007217 3.12 0.086

Probing 1 0.0000312 0.0000312 0.13 0.716

Interaction 1 0.0001314 0.0001314 0.57 0.456

Error 36 0.0083240 0.0002312

Total 39 0.0092082

S = 0.01521 R-Sq = 9.60% R-Sq(adj) = 2.07%


Ball fixed 2 3, 6

Probing fixed 2 Equi, Non Equi

Te

rm

Standardized Effect

B

AB

A

2.01.51.00.50.0

2.028

A Ball

B Probing

Factor Name

Pareto Chart of the Standardized Effects(response is Ring Dia, Alpha = .05)


D CIAM

Rolls-Royce

Step 7: Post Hoc Residual Analysis (Second Screening)

Residual

Pe

rce

nt

0.020.00-0.02-0.04-0.06

99

90

50

10

1

N 40

AD 3.464

P-Value <0.005

Fitted Value

Re

sid

ua

l

40.00540.00039.995

0.02

0.00

-0.02

-0.04

-0.06

Residual

Fre

qu

en

cy

0.020.00-0.02-0.04-0.06

30

20

10

0

Observation Order

Re

sid

ua

l

4035302520151051

0.02

0.00

-0.02

-0.04

-0.06



Residual Plots for Ring Dia

D CIAM

Rolls-Royce

Analysis of Variance for Form, using Adjusted SS for Tests

Two-way ANOVA: Form versus Ball, Probing

Source DF SS MS F P

Ball 1 0.0000024 0.0000024 1.48 0.232

Probing 1 0.0000007 0.0000007 0.41 0.527

Interaction 1 0.0000037 0.0000037 2.30 0.138

Error 36 0.0000574 0.0000016

Total 39 0.0000640

S = 0.001262 R-Sq = 10.41% R-Sq(adj) = 2.94%

Step 7: Two Way ANOVA: Form


Ball fixed 2 3, 6

Probing fixed 2 Equi, Non Equi

Te

rm

Standardized Effect

B

A

AB

2.01.51.00.50.0

2.028

A Ball

B Probing

Factor Name

Pareto Chart of the Standardized Effects(response is Form, Alpha = .05)


D CIAM

Rolls-Royce

D CIAMStep 7: Post Hoc Residual Analysis (Second Screening)

Residual

Pe

rce

nt

0.0040.0020.000-0.002

99

90

50

10

1

N 40

AD 0.476

P-Value 0.227

Fitted Value

Re

sid

ua

l

0.002500.002250.002000.001750.00150

0.0030

0.0015

0.0000

-0.0015

-0.0030

Residual

Fre

qu

en

cy

0.00240.00120.0000-0.0012-0.0024

8

6

4

2

0

Observation Order

Re

sid

ua

l

4035302520151051

0.0030

0.0015

0.0000

-0.0015

-0.0030



Residual Plots for Form

Rolls-Royce

Step 7: Screen Potential Causes (Third Screening

Findings) D CIAM

Post hoc assumptions of normality of residuals met

Position of 100mm slip was significant (p= 0) with

Position “1” being significantly* different to “2” and “3”.

Position was independent on tip (sphere) diameter which

was not significant.

*NB At the 95 % CI

Final Matrix included 2 factors, one at 2 levels, and one

with 3 levels. One Response variables (Length).

Further investigation needed but in the meantime the position of

The part in working envelope should be defined.

Rolls-Royce

1

2 3

1 = Initial position of 100mm Slip Gauge

2 = Secondary position of 100mm Ring Gauge

3 = Third position of 100mm Ring Gauge


Rolls-Royce

StdOrder RunOrder PtType Blocks Ball Dia Position Length

16 1 1 1 6 1 100.005

14 2 1 1 3 2 100.002

22 3 1 1 6 1 100.006

19 4 1 1 3 1 100.016

23 5 1 1 6 2 100.001

25 6 1 1 3 1 100.025

30 7 1 1 6 3 100.007

21 8 1 1 3 3 99.987

18 9 1 1 6 3 100.01

15 10 1 1 3 3 99.994

26 11 1 1 3 2 100.004

17 12 1 1 6 2 99.999

6 13 1 1 6 3 100.01

28 14 1 1 6 1 100.006

2 15 1 1 3 2 100.008

5 16 1 1 6 2 100.002

3 17 1 1 3 3 99.994

12 18 1 1 6 3 100.006

4 19 1 1 6 1 100.004

11 20 1 1 6 2 99.998

13 21 1 1 3 1 100.01

29 22 1 1 6 2 100.001

9 23 1 1 3 3 99.998

10 24 1 1 6 1 100.002

1 25 1 1 3 1 100.012

8 26 1 1 3 2 100.009

27 27 1 1 3 3 99.997

7 28 1 1 3 1 100.013

24 29 1 1 6 3 100.006

20 30 1 1 3 2 100.01Position

Me

an

321

100.015

100.010

100.005

100.000

99.995

3

6

Dia

Ball

Interaction Plot (data means) for Length

Me

an

of

Le

ng

th

63

100.010

100.009

100.008

100.007

100.006

100.005

100.004

100.003

100.002

100.001

321

Ball Dia Position

Main Effects Plot (data means) for Length

Step 7: Screen Potential Causes (Third Screening)D CIAM

Rolls-Royce


Ball Dia fixed 2 3, 6

Position fixed 3 1, 2, 3

Two-way ANOVA: Length versus Ball Dia, Position

Source DF SS MS F P

Ball Dia 1 0.0000075 0.0000075 0.58 0.452

Position 2 0.0004301 0.0002151 16.75 0.000

Interaction 2 0.0008818 0.0004409 34.34 0.000

Error 24 0.0003082 0.0000128

Total 29 0.0016276

S = 0.003584 R-Sq = 81.06% R-Sq(adj) = 77.12%

Step 7: Two Way ANOVA:

Length and Ball Diameter

Individual 95% CIs For Mean Based on Pooled Std Dev

Position Mean -+---------+---------+---------+--------

1 100.010 (-*-)

2 100.003 (-*--)

3 100.001 (-*-)

-+---------+---------+---------+--------

100.000 100.010 100.020 100.030

Factor and interaction IS significant

D CIAM

Rolls-Royce

Step 7: Post Hoc Residual Analysis (Third Screening)

Standardized Residual

Pe

rce

nt

420-2

99

90

50

10

1

N 30

AD 0.403

P-Value 0.335

Fitted Value

Sta

nd

ard

ize

d R

esid

ua

l

100.015100.010100.005100.00099.995

2

0

-2

Standardized Residual

Fre

qu

en

cy

3210-1-2

6.0

4.5

3.0

1.5

0.0

Observation Order

Sta

nd

ard

ize

d R

esid

ua

l

30282624222018161412108642

2

0

-2



Residual Plots for Length

D CIAM

Rolls-Royce

Step 7: Screen Linearity

The final Study was carried out by measuring Various length

of Slip piles from 100mm to 300mm (in increments of 50mm)

to test for gauge linearity.This is the range over which most

of the part diameters are measured on the Faro Arm

A simple linear OLS regression was be used to test linearity

over this range.

The Correlation Coefficient is 1 (100%) and residuals meet

normality requirements.

Rolls-Royce

Linearity Analysis by Regression: Slip Lengths (100 -300 mm)

The regression equation is Slip Length = 50.0 + 50.0 Part

Predictor Coef SE Coef T P

Constant 50.0053 0.0025 20029.09 0.000

Part 49.9999 0.0008 66421.92 0.000

S = 0.00714133 R-Sq = 100.0% R-Sq(adj) = 100.0%

Analysis of Variance

Source DF SS MS F P

Regression 1 225000 225000 4.41187E+09 0.000

Residual Error 43 0 0

Total 44 225000

Unusual Observations

Obs Part Slip Length Fit SE Fit Residual St Resid

6 1.00 100.025 100.005 0.002 0.020 2.87R

R denotes an observation with a large standardized residual.


Rolls-Royce

0.020.010.00-0.01-0.02

99

90

50

10

1

Residual

Pe

rce

nt

300250200150100

0.02

0.01

0.00

-0.01

Fitted Value

Re

sid

ua

l

0.0160.0080.000-0.008

10.0

7.5

5.0

2.5

0.0

Residual

Fre

qu

en

cy

454035302520151051

0.02

0.01

0.00

-0.01

Observation Order

Re

sid

ua

l

Normal Probability Plot Versus Fits

Histogram Versus Order

Residual Plots for Slip Length


Rolls-Royce

Measurement

Variation

Probe

Diameter

Part

Position

on Table

Faro

Position

on table.

Equi spaced

hole probing

Diameters Lengths

Type of

Feature

Probe

Strategy

Environm

ent

Inspector

or

Inspector

interaction

Vacuum

Pressure

Form


Rolls-Royce

D CIAMLocation Observations

• Temperature, and environmental factors were not significant on response data

gathered over two separate days. Significant differences between Probe

diameter or gauge location on table may show a difference between “day”, as

these factors were blocked by day.

• Vacuum clamp pressure was not significant as long as arm did not move, but

should be added to inspection procedure.

• For some applications ball diameter MAY be significant on measurand

diameter (p=0.086) and should be considered when developing inspection

plan.

• For some applications probe strategy MAY be significant on measurand

diameter (although not in this study) a minimum of points should be used to

give form and be equi-spaced as this allows a common approach.

• For length measurements, position position on table relative to measurand is

significant and should be defined when developing inspection plans.

• Not part of this study, but important to the set up is Faro ball qualification. In

addition a reference standard should be used at set up to show any shifts in

bias (stability) over time.

Step 7: Identify Vital Few X’s

Rolls-Royce

Step 7: Identify Vital Few X’s

D CIAMSpread Observations

If the Repeatability is is large compared to Reproducibility the reasons may be:

• The faro arm or probing head may need maintenance / Calibration.

• The location of part may need to be designed to be more rigid.

• The clamping (if utilised) of part needs to be improved.

• There is excessive within-part variation (form errors).

If the Reproducibility is is large compared to Repeatability the reasons may be:

• The appraiser needs to be better trained in how to locate the part.

• The alignment of the part the part co-ordinate system may be unsatisfactory

(miss probe on part or part moved after alignment).

• There is Program corruption or part moved between appraisers.

Rolls-Royce

RU 51338

159.258-159.512

Step 8: Determine Transfer Function y = f(x)

Faro Constraining Schematic D CIAM

Rolls-Royce

Step 9: Quantify New Improved Performance

– DIA THIS PROJECTD CIAM

Part-to-PartReprodRepeatGage R&R

100

50

0

Perc

ent

% Contribution

% Study Var

% Tolerance

0.02

0.01

0.00

Sam

ple

Range

_R=0.00968

UCL=0.02492

LCL=0

Chris Denis Ray

159.400

159.375

159.350

Sam

ple

Mean

__X=159.36887

UCL=159.37878

LCL=159.35897

Chris Denis Ray

54321

159.400

159.375

159.350

Part

RayDenisChris

159.400

159.375

159.350

Inspector

54321

159.400

159.375

159.350

Part

Avera

ge

Chris

Denis

Ray

Inspector

Gage name:

Date of study :

Reported by :

Tolerance:

M isc:

Components of Variation

R Chart by Inspector

Xbar Chart by Inspector

Dia RUB51338 by Part

Dia RUB51338 by Inspector

Inspector * Part Interaction

Gauge RR for RU51338

Rolls-Royce

Gage R&R

%Contribution

Source VarComp (of VarComp)

Total Gage R&R 0.0000368 4.73

Repeatability 0.0000294 3.79

Reproducibility 0.0000074 0.95

Inspector 0.0000000 0.00

Inspector*Part 0.0000074 0.95

Part-To-Part 0.0007401 95.27

Total Variation 0.0007768 100.00

Process tolerance = 0.254

Study Var %Study Var %Tolerance

Source StdDev (SD) (6 * SD) (%SV) (SV/Toler)

Total Gage R&R 0.0060633 0.036380 21.75 14.32

Repeatability 0.0054233 0.032540 19.46 12.81

Reproducibility 0.0027114 0.016268 9.73 6.40

Inspector 0.0000000 0.000000 0.00 0.00

Inspector*Part 0.0027114 0.016268 9.73 6.40

Part-To-Part 0.0272041 0.163224 97.61 64.26

Total Variation 0.0278716 0.167230 100.00 65.84

Number of Distinct Categories = 6

AB

C

D

Current Study - Internal Ring

Gage R&R - Diameter 159.258 /159.512 – Tolerance 0.254mm

RU 51338

Step 9: Quantify New Improved Performance

– DIA THIS PROJECTD CIAM

Rolls-Royce

Step 10: Comparison from PREVIOUS PROJECT D CIAMP

erc

ent


100

50

0

% Contribution

% Study Var

% Tolerance

Sam

ple

Range 0.010

0.005

0.000

_R=0.00292

UCL=0.00953

LCL=0

Chris Dave Ray

Sam

ple

Mean 170.92

170.91

170.90

__X=170.91054

UCL=170.91602

LCL=170.90505

Chris Dave Ray

Part

10987654321

170.930

170.915

170.900

Name

RayDaveChris

170.930

170.915

170.900

Part

Avera

ge

10 9 8 7 6 5 4 3 2 1

170.92

170.91

170.90

Chris

Dave

Ray

Name

Gage name: Faro A rm

Date of study : 16th A ug 07

Reported by : B Dunscombe

Tolerance:

M isc:


R Chart by Name

Xbar Chart by Name

171 by Part

171 by Name

Name * Part Interaction

Dia 170.688 / 171.196 mm

Rolls-Royce

%Contribution


Total Gage R&R 0.0000105 11.15



Name 0.0000035 3.77

Part-To-Part 0.0000837 88.85



Study Var %Study Var %Tolerance

Source StdDev (SD) (6 * SD) (%SV) (SV/Toler)

Total Gage R&R 0.0032410 0.0194461 33.39 3.83

Repeatability 0.0026375 0.0158249 27.17 3.12


Name 0.0018836 0.0113015 19.41 2.22

Part-To-Part 0.0091496 0.0548975 94.26 10.81

Number of Distinct Categories = 3 B

A

D

C

Gage R&R - Diameter 170.688 / 171.196 – Tolerance 0.508mm

Previous Study - Dimension 6 – Internal Ring

RU 52314

Step 11: Determine Previous Capability

Performance – PREVIOUS PROJECT D CIAM

Rolls-Royce

Step 11: Determine Process Capability Improved

Performance – FORM (Roundness) THIS STUDY D CIAM


100

50

0

Perc

ent

% Contribution

% Study Var

% Tolerance

0.008

0.004

0.000

Sam

ple

Range

_R=0.002796

UCL=0.007198

LCL=0

Chris Denis Ray

0.04

0.02

0.00

Sam

ple

Mean

__X=0.01707UCL=0.01993

LCL=0.01421

Chris Denis Ray

54321

0.04

0.02

0.00

Part

RayDenisChris

0.04

0.02

0.00

Inspector

54321

0.04

0.02

0.00

Part

Avera

ge

Chris

Denis

Ray

Inspector

Gage name:

Date of study :

Reported by :

Tolerance:

M isc:


R Chart by Inspector

Xbar Chart by Inspector

Form RUB51338 by Part

Form RUB51338 by Inspector

Inspector * Part Interaction

Gauge RR for RU51338

Rolls-Royce

Step 11: Determine Process Capability Improved

Performance – FORM (Roundness) THIS STUDY D CIAM

Gage R&R

%Contribution


Total Gage R&R 0.0000032 1.27



Inspector 0.0000002 0.07

Part-To-Part 0.0002483 98.73



Study Var %Study Var %Toleranc

Source StdDev (SD) (5.15 * SD) (%SV) (SV/Toler)Total

Gage R&R 0.0017877 0.0092068 11.27 3.62

Repeatability 0.0017366 0.0089434 10.95 3.52


Inspector 0.0004246 0.0021865 2.68 0.86

Part-To-Part 0.0157571 0.0811490 99.36 31.95

Total Variation 0.0158582 0.0816696 100.00 32.15

Number of Distinct Categories = 12

A

B

C

D

Part location during studies

Rolls-Royce

Step 12: Implement Process Controls To Maintain

Performance D CIAM

Develop Specific Inspection plans for pertinent parts with information gathered from

experiments and instrument manufacturer guidelines. The inspection plan should define what

type of equipment used for the part / feature, the set up and all other factors which could affect

results. In this case position on table (linear), Faro probing strategy equi-spacing is

recommended and form should always be recorded (even if not used for reporting).

Maintain traceability to reference standard by measuring a common master Ring / Slip at each

set up and add results to calibration X Bar & R Chart. Include this in inspection procedure/plan.

Ensure that future GRR studies address the issue of form indicators to check generated process

data variations are not a function of out of form condition. This is an underlying assumption of

this type of study and the statistics (ANOVA) assume a normality of distribution which should be

tested. Ensure that parts / features selected for future GRR studies are representative of the

variation across the whole range of process or specification limits AND that special cause

variation in the measurement process is eliminated (should be addressed in inspection plan).

With the same raw data, the product specification was reduced to the point where the %R&R as a

product of tolerance was 20% the occurred where product spread was 0.15mm (total). This is

typical of what can be achieved with typical FARO manual probing. For tighter diameter control,

alternate gauging should be utilised (CMM / electronic bore). Diameters are normally the most

challenging feature for co-ordinate based inspection systems like Faro arms.

quality by design - dgs quality assurancedgs-qualityassurance.com/pdf/faro measurement...

Documents