design of experiments (doe) in new product design

Design of Experiments (DOE)

in

New Product Design & Development

31st Annual International Test & Evaluation Symposium

7 October 2014

Arlington, VA

14-DFSSLE-10AAir Academy Associates

Office: +1 719-531-0777

Fax: +1 719-531-0778

www.airacad.com

www.airacad.com/airacadvideo.aspx

©2014 Air Academy Associates, LLC. Do Not Reproduce.

Simplify, Perfect, Innovate

Purpose of this Presentation:

to debunk the myth that


cannot or should not be used in

New Product Design and Development



Key Points

Concept of a Transfer Function

Relationship Between DOE and Transfer Functions

Examples of DOE-generated Transfer Functions in R&D

Design Techniques Using Transfer Functions

• Expected Value Analysis

• Parameter (Robust) Design

DOE in Modeling and Simulation



Transfer Function: The Bridge to Innovation

Where does the transfer function come from?

• Exact transfer Function

• Approximations

- DOE

- Historical Data Analysis

- Simulation

Processy (CTC)

X1

X2

X3

s

y = f1 (x1, x2, x3)

= f2 (x1, x2, x3)

Parameters

or Factors

that

Influence

the CTC



Exact Transfer Functions

• Engineering Relationships

- F = ma

- V = IR

R2R1

The equation for current (I) through

this DC circuit is defined by:

where N: total number of turns of wire in the solenoid

: current in the wire, in amperes

r : radius of helix (solenoid), in cm

: length of the helix (solenoid), in cm

x : distance from center of helix (solenoid), in cm

H: magnetizing force, in amperes per centimeter

2222 )x5(.r

x5.

)x5(.r

x5.

2

NH

r

x

The equation for magnetic force at a distance

X from the center of a solenoid is:

9V

21

21

21

21

)(

RR

RRV

RR

RR

VI




• “Interrogates” the process

• Changes “I think” to “I know”

• Used to identify important relationships

between input factors and outputs

• Identifies important interactions between

process variables

• Can be used to optimize a process

• An optimal data collection methodology



Purposeful changes of the inputs (factors) in order to observe

corresponding changes in the output (response).

Run

1

2

3

.

.

X1 X2 X3 X4 Y1 Y2 . . . . . . Y SY

Inputs

A = X1

B = X2

D = X4

C = X3

Y1

Outputs

.

.

.

.

.

.

PROCESS

What is a Designed Experiment?

Y2



DOE Helps Determine How Inputs Affect Outputs

A1 A2

y

i) Factor A affects the average of y

B1

B2

y

ii) Factor B affects the standard deviation of y

C2

C1

y

iii) Factor C affects the average and the

standard deviation of y

D1 = D2

y

iv) Factor D has no effect on y



What Makes DOE so Powerful?(Orthogonality: both vertical and horizontal balance)

AB

+

+

-

-

-

-

+

+

AC

+

-

+

-

-

+

-

+

A Full Factorial Design for 3 Factors A, B, and C, Each at 2 levels:

BC

+

-

-

+

+

-

-

+

ABC

-

+

+

-

+

-

-

+

Run A B C

1 - - -

2 - - +

3 - + -

4 - + +

5 + - -

6 + - +

7 + + -

8 + + +



Famous Quote

“All experiments (tests) are designed;

some are poorly designed,

some are well designed.”

George Box (1919-2013), Professor of Statistics, DOE Guru



Design of Experiments (DOEs): A Subset of All Possible Test Design Methodologies

The Set of All Possible Test Design

Methodologies (Combinatorial Tests)

Orthogonal or

Nearly

Orthogonal

Test Designs

(DOEs)

One

Factor

At a

Time

(OFAT)

Best Guess

(Oracle)

Boundary Value Analysis

(BVA)

Equivalence Partitioning (EP)



Motivation for DOE from Dr. Gilmore (DOT&E)

1. One of the most important goals of operational testing is to

characterize a system’s effectiveness over the operational envelope.

2. I advocate the use of DOE to ensure that test programs are able to

determine the effect of factors on a comprehensive set of

operational mission-focused and quantitative response variables.

3. Future test plans must state clearly that data are being collected to

measure a particular response variable (possibly more than one) in

order to characterize the system’s performance by examining the

effects of multiple factors … and clearly delineating what statistical

model (e.g., main effects and interactions) is motivating … the

variation of the test.

4. Confounding factors must be avoided.

5. Another pitfall to avoid is relying on binary metrics as the primary

response variable.



• Total # of Combinations = 35 = 243

• Central Composite Design: n = 30

Modeling Flight

Characteristics

of New 3-Wing

Aircraft

Pitch )

Roll )

W1F )

W2F )

W3F )

INPUT OUTPUT

(-15, 0, 15)

(-15, 0, 15)

(-15, 0, 15)

(0, 15, 30)

(0, 15, 30)

Six Aero-

Characteristics

Value Delivery: Reducing Time to Market for New Technologies

Patent Holder: Dr. Bert Silich



CL = .233 + .008(P)2 + .255(P) + .012(R) - .043(WD1) - .117(WD2) + .185(WD3) + .010(P)(WD3) -

.042(R)(WD1) + .035(R)(WD2) + .016(R)(WD3) + .010(P)(R) - .003(WD1)(WD2) -

.006(WD1)(WD3)

CD = .058 + .016(P)2 + .028(P) - .004(WD1) - .013(WD2) + .013(WD3) + .002(P)(R) - .004(P)(WD1)

- .009(P)(WD2) + .016(P)(WD3) - .004(R)(WD1) + .003(R)(WD2) + .020(WD1)2 + .017(WD2)2

+ .021(WD3)2

CY = -.006(P) - .006(R) + .169(WD1) - .121(WD2) - .063(WD3) - .004(P)(R) + .008(P)(WD1) -

.006(P)(WD2) - .008(P)(WD3) - .012(R)(WD1) - .029(R)(WD2) + .048(R)(WD3) - .008(WD1)2

CM = .023 - .008(P)2 + .004(P) - .007(R) + .024(WD1) + .066(WD2) - .099(WD3) - .006(P)(R) +

.002(P)(WD2) - .005(P)(WD3) + .023(R)(WD1) - .019(R)(WD2) - .007(R)(WD3) + .007(WD1)2

- .008(WD2)2 + .002(WD1)(WD2) + .002(WD1)(WD3)

CYM= .001(P) + .001(R) - .050(WD1) + .029(WD2) + .012(WD3) + .001(P)(R) - .005(P)(WD1) -

.004(P)(WD2) - .004(P)(WD3) + .003(R)(WD1) + .008(R)(WD2) - .013(R)(WD3) + .004(WD1)2

+ .003(WD2)2 - .005(WD3)2

Ce = .003(P) + .035(WD1) + .048(WD2) + .051(WD3) - .003(R)(WD3) + .003(P)(R) - .005(P)(WD1)

+ .005(P)(WD2) + .006(P)(WD3) + .002(R)(WD1)

Aircraft Equations



Cycles to failure

Clamp Size(8,24)

Radial Gap(.045, .090, .135)

Temperature(90, 180)

Bolt Tension/Torque (7.5K, 30K)

Repair

Clamp

Leak Repair Clamp Process



Repair Clamp Regression Results

Transfer Function:

y = 676-600B-583C+525BC,

B=Radial Gap

C=Temperature



Surface Plot of the Transfer Function

90

105

120

135

150

165

180

10

510

1010

1510

2010

2510

0.02

0.03

0.03

0.04

0.05

0.06

0.06

0.07

0.08

0.09

0.09

0.10

0.11

0.12

0.12

0.13

0.14

0.15

0.16

Temp

Re

spo

nse

Val

ue

Radial Gap

cycles until failure

Y-hat Surface Plot Radial Gap vs Temp

Constants: Clamp = 16 Torque = 17213

2010-2510

1510-2010

1010-1510

510-1010

10-510



Contour Plot

90

95

100

105

110

115

120

125

130

135

140

145

150

155

160

165

170

175

180

0.02

0.03

0.03

0.04

0.05

0.06

0.06

0.07

0.08

0.09

0.09

0.10

0.11

0.12

0.12

0.13

0.14

0.15

0.16

Temp

Radial Gap

cycles until failure

Y-hat Contour Plot Radial Gap vs Temp

Constants: Clamp = 16 Torque = 17213

2250-2500

2000-2250

1750-2000

1500-1750

1250-1500

1000-1250

750-1000

500-750

250-500

0-250



Fusing Titanium and Cobalt-Chrome

Courtesy Rai Chowdhary

©2014 Air Academy Associates Do Not Reproduce.

20

Case Study: OnTech Self-Heating Container

• Self-heating

• Activated by button

on bottom of can

• Used for hot

beverages and

soups

• Disposable

• Environmentally

compatible

Key Features (VOC)Identify



Case Study: General Design Concept

Water for reaction

Energy release

Calcium Oxide (CaO)

Beverage

Convection

Point of activation

Design



Case Study: Transfer Functions

DesignWall thickness (X1)

CaO mass (X2)

H20 volume (X3)

Y1=f1(X1, X2, X3)

Example: “Time to use” and “Can temp” as a function of

“Wall thickness”, “CaO mass”, and “H2O volume”

Time to use (Y1)

Y2=f2(X1, X2, X3) Can temp (Y2)

How do we find the functions f1 and f2?

• First principle equations

(Physics / Engineering equations)

• Analytical Models (Simulation and Regression)

FEA, CFD, etc.

• Empirical models (Design of Experiments)



The Value of Transfer Functions

Simple and compact way of understanding relationships between

performance measures or response variables (Y’s) and the factors

(X’s) that influence them.

Allows us to

• Predict the response variable (y), with associated risk levels, before any

change in the product or process is made.

• Assess the product/process capability in the presence of uncontrolled

variation or noise using Monte Carlo Simulation (DFSS tool: Expected

Value Analysis).

• Understand the impact of the factors (sensitivity analysis)

• Optimize performance easily using DFSS tools such as parameter

design and tolerance allocation.

Greatly enhance one’s knowledge of a product or process.

In general, they are the gateway to systematic innovation.

Provide a meaningful metric for the maturity in DFSS for any

organization.



Expected Value Analysis (EVA)

EVA is the technique used to determine the characteristics of the output

distribution (mean, standard deviation, and shape) when we have

knowledge of (1) the input variable distributions and (2) the transfer

functions.

X1

X2

X3

y1

y2y2 = f2 (X1, X2, X3)

y1 = f1 (X1, X2, X3)

Variation in the inputs causes variation in the output.



y = x2x

Expected Value Analysis Example

What is the mean or expected value of the y (output) distribution?

What is the shape of the y (output) distribution?

6

2

y = f(x)

?



Expected Value Analysis Example(Cycle Time)

Step 1 Step 2 Step 3Total Time =

S1 + S2 + S3

Spec/Goal:

Complete in

<= 16 hrs

S1

S2

S3

T

1 3

hrs

1 2 8

hrs

10

hrs

1

7 13

The simulated results are shown on the next page.

• What is the expected value (mean) of the total cycle time?

• What is the shape of the output distribution?

• Approximately what percent of the time will it take longer than 16 hours to complete all 3 steps?



Expected Value Analysis Example(Cycle Time) (EVA Results)



Parameter Design (Robust Design)

Process of finding

the optimal mean

settings of the

input variables to

minimize the

resulting dpm.

LSL USL

1

X1

Y

X22

LSL USL

1

X1

Y

X22

LSL USL

init

X1

Y

X2 init

new

new

LSL USL

init

X1

Y

X2 init

new

new



Parameter Design (Robust Design)

X1 X2

Changing the mean

of an input may

possibly reduce the

output variation!

T

X1 X2

X

If you’re the

designer,

which setting

for X do you

prefer?

X

T



Plug Pressure (20-50)

Bellow Pressure (10-20)

Ball Valve Pressure (100-200)

Water Temp (70-100)

Reservoir Level (700-900)

Nuclear

Reservoir

Level

Control

Process

Robust Design Simulation* Example

* From SimWare Pro by Philip Mayfield and Digital Computations



Prior to Robust Design(defect rate is 61%)



After Robust Design(defect rate is 0.0004%)



Applications of Modeling and Simulation

Simulation of stress and vibrations of turbine

assembly for use in nuclear power generation

Simulation of underhood thermal cooling for decrease

in engine space and increase in cabin space and comfort

Evaluation of dual bird-strike on aircraft engine

nacelle for turbine blade containment studies

Evaluation of cooling air flow behavior

inside a computer system chassis

Power

Automotive

Electronics

Aerospace



Multidisciplinary Design Optimization (MDO): A Design Process Application



Summary of "Modeling the Simulator"

Perform Screening Design

Using the Simulator if

necessary

Perform Expected Value Analysis,

Robust Design, and Tolerance

Allocation Using Transfer Function

Build Prototype to Validate

Design in Real World

Perform Modeling Design Using the

Simulator to Build Low Fidelity Model

Validate Design Using

the Simulator

Optimized Simulator

Optimized Design

Critical Parameters ID'd

Transfer Function on

Critical Parameters



Thank You

Questions

Colorado Springs, Colorado



Examples of Computer Aided Engineering (CAE) and Simulation Software

Mechanical motion: Multibody kinetics and dynamics

ADAMS®

DADS

Implicit Finite Element Analysis: Linear and nonlinear

statics, dynamic response

MSC.Nastran™, MSC.Marc™

ANSYS®

Pro MECHANICA

ABAQUS® Standard and Explicit

ADINA

Explicit Finite Element Analysis : Impact simulation,

metal forming

LS-DYNA

RADIOSS

PAM-CRASH®, PAM-STAMP

General Computational Fluid Dynamics: Internal and

external flow simulation

STAR-CD

CFX-4, CFX-5

FLUENT®, FIDAP™

PowerFLOW®



Examples of High Fidelity Simulation Models

Preprocessing: Finite Element Analysis and

Computational Fluid Dynamics mesh generation

ICEM-CFD

Gridgen

Altair® HyperMesh®

I-deas®

MSC.Patran

TrueGrid®

GridPro

FEMB

ANSA

Postprocessing: Finite Element Analysis and

Computational Fluid Dynamics results visualization

Altair® HyperMesh®

I-deas

MSC.Patran

FEMB

EnSight

FIELDVIEW

ICEM CFD Visual3 2.0 (PVS)

COVISE



Applying Modeling and Simulation to Automotive Vehicle Design

IDENTIFY

CTCs, CDPs

SCREENING DESIGN

(DOE PRO)

NASTRAN RADIOSS MADYMO

Integrated processes with high fidelity

CAE analyses on HPC servers

Examples of CTCs:

y1 = weight of vehicle

y2 = cost of vehicle

y3 = frontal head impact

y4 = frontal chest impact

y5 = toe board intrusion

y6 = hip deflection

y7 = rollover impact

y8 = side impact

y9 = internal aerodynamics (airflow)

y10 = external aerodynamics (airflow)

y11 = noise

y12 = vibration (e.g., steering wheel)

y13 = harshness (e.g., over bumps, shocks)

y14 = durability (at 100K miles)

Examples of Critical Design Parameters (CDPs or Xs):

x1 = roof panel material

x2 = roof panel thickness

x3 = door pillar dimensions i beam

x4 = shape/geometry

x5 = windshield glass

x6 = hood material, sizing and thickness

x7 = under hood panel material, sizing and thickness

Many, Many x’sThe critical

few CDP’s

Safety CTCs

with constraints

specified by

FMVSS

(Federal Motor

Vehicle Safety

Standards)

RADIOSS

DYNA

MADYMO

no federal

requirements

on these CTCs

CFD

NASTRAN

t1

t2



Applying Modeling and Simulationto Automotive Vehicle Design (cont.)

MODELING DESIGN

(DOE PRO)

NASTRAN RADIOSS MADYMO

High Fidelity Models

MONTE CARLO

SIMULATION

(DFSS MASTER)

Response Surface Models

Low Fidelity Models

VALIDATION

Robust

Designs

CDPs, CTCs

CDPs

NASTRAN RADIOSS

MADYMO

High Fidelity Models



GEMS LightSpeedTM CT Scanner

GE's First DFSS System ('98):

Full Use of Six Sigma/DFSS Tools Leading-Edge Technology

• Key customer CTQs identified- Image quality- Speed- Software reliability- Patient comfort

• Disciplined systems approach: 90 system CTQs• 33 Six Sigma (DMAIC) or DFSS projects/studies• Scorecard-driven• Part CTQs verified before systems integration

• World's first 16-row CT detector• Multi-slice data acquisition• 64-bit RISC computer architecture• Long-life PerformixTM tube

ResultsBetter image quality

- Earlier, more reliable diagnoses- New applications; vascular imaging, pulmonary embolism,

multi-phase liver studies,…• Much faster scanning:

- Head: from 1 min to 19 sec (9 million/yr)- Chest/abdomen: from 3 min to 17 sec (4 million/yr)

• Clinical productivity up 50%• 10x improvement in software reliability• Patient comfort improved – shorter exam time• Development time shortened by 2 years• High market share; significant margin increase

"Biggest breakthrough in CT in a decade," Gary Glazer, Stanford

Head Abdomen



Sample of Who Has Used OurDFSS Methods and Tools

design of experiments (doe) in new product design

Documents