chee418/801 - module 3: design of experimentsnunez/mastertecnologiastelecomunicacion/tema… · 4...

CHEE418/801 - Module 3:Design of Experiments

James [email protected]

Queen’s University

November, 2010

J. McLellan (Queen’s) CHEE418/801 Fall 2010 Module 2 November, 2010 1 / 125

Outline1 Motivation and Definitions

MotivationTerminology

2 Two-Level Factorial DesignsDefinitionAnalyzing Two-Level Factorial DesignsDesigning Two-Level Factorial Experimental DesignsAssessing Effects Statistically

3 Blocking of Experimental Designs4 Fractional Factorial Designs

Overview and DefinitionsAliasing StructureEstimating Effects

5 Higher-Order DesignsCentral Composite DesignsFace-Centred Central Composite DesignsBox-Behnken Designs


Definition - Experimental Design

An experimental design is a disciplined plan for collecting data

What should we observe, and how should we perturb the process?

How can we maximize the information content of the data?


Motivation

Experimental design is an integral component of quality improvement, andsupports improvement in:

product design

process design

process operation

Experimental design is an important tool for learning more about physicalsystems because it helps provide clear insights into physical behaviour.


Process Investigations


The Iterative Nature of Process Investigations


Relationship to this course


Why not use routine operating data?

Routine operating data frequently do not contain sufficient information ofinterest due to:

the limited range of operating variables due to tight controlI values don’t vary significantly, so the effects of the variables may not

be seen

systematic relationships between operating variablesI arising from process control and/or other operating policies

coincidental or correlation relationships that don’t necessarilyrepresent cause and effect

Without trying to fix most operating variables, and perturbing thevariables of interest, it is very difficult to conclude that the relationships inthe data represent cause and effect behaviour.

Historical data are used to gain insight into process behaviour, however itis important to recognize that the behaviour observed may not representcause and effect relationships.


Active Versus Passive Data Collection

Active Data Collection

we actively intervene in the process and cause changes

Passive Data Collection

we passively observe, without introducing perturbations into theprocess

The only way to ensure that our observations represent cause and effect isto introduce perturbations (”causes”) and observe the responses(”effects”)


Terminology Used in Design of Experiments

Responses - measurable outcomes of interest

frequently have more than one response variable being considered

e.g., in melt grafting - degree of grafting, grafting efficiency

Factors - controllable variable thought to have an influence on theresponse(s)

deliberately manipulated to determine effect on response(s)

e.g., in melt grafting - screw speed, initiator type, temperature

we have referred to these previously as regressors / explanatoryvariables

Level - value of a setting of a factor

Test run - set of factor level combinations for one experimental run


Terminology Used in Design of Experiments

Covariates - variables affecting process or product performance whichcannot be or are not controlled

e.g., melt grafting - humidity, room temperature, quench watertemperature

Extraneous Variation - variation in measured response values in anexperiment that is attributable to sources other than the deliberateperturbations we have made in the levels of the factors

e.g., melt grafting - variability in the quench water temperature andhumidity

Design - selection of test run factor levels - the set of experimental runsthat we will conduct

Effect - the effect of factors on the response, measured by the change inaverage response values under two or more factor level combinations


Example - of Terminology

Wave Solder Process - the wave solder process is producing too manydefective items. Investigate the extent to which conveyor speed,temperature and flux density affect the occurrence of defects. Roomtemperature and humidity are not controlled, and the operators work on12 hour shifts.

Reponse -

Factors -

Covariates -


Considerations in Planning an Experimental Investigation

What are the objectives of the investigation?

What are the performance characteristics of interest?

What responses will be used to assess these characteristics?

What factors will be deliberately manipulated?

What is the operating region for conducting our experiments? Howfar can we adjust factors?

What other variables may influence our results?

Will it be possible to conduct additional tests in the future if we needmore data?

What sets of operating conditions are to be tested?

In what order will the tests be carried out?


Considerations in Planning an Experimental Investigation

How can we assess the effects of the factors?I Eliminate possible systematic bias by trying to include as much as

possible all factors suspected of having an effect in the list of factors tobe perturbed

I Consider running a screening study in which we try to identify the maineffects of each factor while reducing the number of experimental runswe conduct.

What types of relationships do we think exist? Quadratic? Linear?How can we do our experimental runs in order to assess whetherthese types of relationships exist?

I we need two points to identify a line, three points to identify curvature,...


One-factor-at-a-time Investigations

We start at a nominal operating point, conduct experiments by varying thefirst factor, then keep it fixed and conduct experiments in which thesecond factor is varied, and so forth...

Why is this a problem? Consider an example of reactor yield vs.temperature and concentration - find the maximum yield.


One-factor-at-a-time Investigations

If we adjust one factor at a time, we will miss the synergies betweenconcentration and temperature, and we will not locate the true value ofthe maximum yield.

one-factor-at-a-time testing does not account for possible interactionsbetween the effects of the variables

the yield surface contours are rotated ellipses, which havecross-product terms indicating the presence of two-factor interactions


Two-Level Factorial Designs

Suppose we have k factors being investigated, and we have an operatingregion of interest defined by low and high limits for each factor:



We should conduct an experiment at every combination of high and lowvalues for all factors:

Runs Coded Values

L,L −1,−1L,H −1, 1H,L 1,−1H,H 1, 1


Coding Variables

Coding simplifies the calculations and helps realize the advantages of theexperimental design. The standard coding is:

x =xuncoded − xuncoded12range(xuncoded)

For this coding:

−1 corresponds to the low limit of interest

+1 corresponds to the high limit of interest

the average of the upper and lower limits is the midpoint of theinterval of interest

In order to take advantage of the properties of the experimental design, weneed to work in coded variables.


Coding for Qualitative Factors

Sometimes the factors being investigated are not numerical, but areinstead qualitative:

catalyst types A and B

catalyst preparation I and II

suppliers A and B

machines I and II

These factors can be coded as −1 and +1:

e.g., −1 for catalyst type A, +1 for catalyst type B



If we have k factors under investigation, a two-level factorial design willconsist of 2k runs

this is the number of combinations of high and low values (two levels)for k factors

These designs are known as 2k designs. This notation identifies thenumber of levels (2) and the number of factors (k).



Why should we place the runs at the limits of the region of interest?

think of the variance of the slope parameter estimate in a straight linemodel

V ar(β1) =σ2ε∑n

i=1(xi − x)2

placing the xi values as far as possible from the average minimizesthe variance of the parameter estimates

leads to improved precision of the parameter estimates

effectively maximizes the signal in the data relative to the noise



Why should we place the runs at the limits of the region of interest...

In the multiple regression case:

placing the points as far from the average point as possible maximizesthe determinant of XTX

covariance matrix of the parameter estimates is based on inverse ofXTX and the area of the joint confidence region is proportional to1/√det(XTX)

maximizing the determinant minimizes the area of the jointconfidence region and yields the most precise parameter estimates

Parameter estimates contain information about the effects of the factors,so precision in the parameter estimates translates into precision in theknowledge of the effects of the factors.


General Factorial Designs

We can define factorial designs for a number of factors, each at possiblydifferent numbers of levels. If we have k factors, each considered at mi

levels, then a general factorial design consists of experimental runs at allpossible combinations of the levels for each factor, having:

m1 ·m2 ·m3 · · · ·mk =k∏i=1

mi

experimental runs.

Examples:

2k - two-level factorial design

3k - four-level factorial design

The number of runs can get very large very quickly!


Randomization

When implementing a designed experiment, the runs should be conductedin a completely randomized manner. Why?

to guard against systematic trends caused by other variables whichcould lead to misinterpretation of the results or biased results

examplesI systematic noise component associated with increasing temperaturesI slow drift in one of the instrumentsI all high temperature runs conducted on the day shift, all low

temperature runs on the night shift - confounding of effects


Information Provided by a Designed Experiment

Given m distinct sets of factor levels (runs) in the experimental design, wecan estimate:

the overall average response

m− 1 pieces of information about the effects of the factors on theresponse

This is often viewed as providing m− 1 independent pieces of informationabout the process. The overall average is not viewed as a piece ofinformation about the factor effects.

Link to regression - for m distinct sets of experimental runs, we canestimate the intercept parameter and m− 1 other parameters for a total ofm parameters.


Example - Using a 2k Factorial Design to InvestigateReactor Yield

We want to investigate the effect of temperature (T ) and concentration(C) on chemical reactor yield.

prepare a 22 factorial design in T and C - 4 runs

Runs Coded Values

L,L −1,−1L,H −1, 1H,L 1,−1H,H 1, 1


Example - Using a 2k Factorial Design to InvestigateReactor Yield

We can obtain the following information from the experimental datacollected using the 22 design in T and C:

main effects - effect of C on yield and T on yield (2 pieces ofinformation)

interaction effect - effect of C ∗ T on yield (1 piece of information)

total of 3 pieces of information from 4 runs

remaining run helps provide overall average yield


Main Effects

The main effect of a factor is the average influence of a change in level ofthe single factor on the response.

For a 2-level factorial design,(Main Effect

of a Factor

)=(Average of Responses

at High Level of Factor

)−(Average of Responses

at Low Level of Factor

)

Main Effect = yfactor=+1 − yfactor=−1


Main Effects - Chemical Reactor Example

For temperature:

average yield at high T is 70average yield at low T is 57main effect is 70− 57 = 13


Main Effects Plot from JMP

JMP will generate main effects plots that summarize graphically the maineffects. These are found in the Prediction Profiler which you can selectfrom the Fit Model output window by clicking on the triangle besideResponse, and selecting Factor Profiling ‖ Profiler. Here is the plot forthe Chemical Reactor Yield problem.

55

60

65

70

y63.5

-1

-0.5 0

0.5 1

0x1

-1

-0.5 0

0.5 1

0x2

Prediction Profiler


Interaction Effects

Interaction is the extent to which the influence of one factor on theresponse depends on the level of another factor – e.g., T ∗ C. Visually, forthe reactor example,


Interaction Effects - Chemical Reactor Example

The influence of temperature at high concentration is slightly larger thanthe influence of temperature at low concentration ⇒ mild interactioneffect.

The interaction effect between T and C is

12

[(∆yield from T

at high concentration

)−(

∆yield from T

at low concentration

)]=

12

(14− 12) = 1


Interaction Effects - Definition

For two factors, x1 and x2, the interaction effect is

12

[(effect of factor 1 on response

at high level of factor 2

)−(effect of factor 1 on response

at low level of factor 2

)]Why divide by 2? To place the assessment of the interaction effect on the

same basis as that of the main effects.


Interaction and Main Effects - Graphically

We can return to the interaction plot and visualize the main effects as well:


Interaction Profiles from JMP

JMP will generate interaction profiles which are the same as the plots onthe previous slide. To generate these plots, click on the triangle byResponse in the model fit output, and select Factor Profiling ‖Interaction Plots. Here is the plot for the chemical reactor yield example:

55606570y

55606570

y

x1

-1

1

-1 0 0.5 1

-11

x2

-1 0 0.5 1

x1x2

Interaction Profiles


Using Regression to Estimate Effects

We can estimate the main and interaction effects by fitting a regressionmodel. For the chemical reactor yield example using the 22 design, we canestimate the main effects and 2-factor interaction by fitting the followingmodel to the data (x1 is C, x2 is T ):

Y = β0 + β1x1 + β2x2 + β12x1x2 + ε

Computing the main effect of factor 1 (C) in terms of the model:difference between average yield at high C (x1 = 1) and average yield atlow C (x1 = −1):

1

2{[β0 + β1(1) + β2(1) + β12(1)(1)] + [β0 + β1(1) + β2(−1) + β12(1)(−1)]}

−1

2{[β0 + β1(−1) + β2(1) + β12(−1)(1)] + [β0 + β1(−1) + β2(−1) + β12(−1)(−1)]}

= 2β1


Using Regression to Estimate Effects

In the general case, with k factors, to obtain the main and 2-factorinteraction effects from the 2k design, fit a first-order plus 2-factorinteraction model to the data:

Y = β0 + β1x1 + β2x2 + · · ·+ βkxk + β12x1x2 + · · ·+ βk−1,k + ε

From the model, we have:

main effect of factor i is 2βi2-factor interaction effect between factors i and j is 2βij


Example - Chemical Reactor Yield

Form the X matrix:

X =

1 −1 −1 11 1 −1 −11 −1 1 −11 1 1 1

The observation vector is:

y =

60547268

and the parameter estimate vector is β =

63.5−2.56.50.5

.


Example - Chemical Reactor Yield

Using the estimated regression parameters, the effects are:

main effect of x1 = 2(−2.5) = −5main effect of x2 = 2(6.5) = 13interaction effect x1x2 = 2(0.5) = 1


The Effects Representation

The effects representation is another approach sometimes used in industryto compute effects for 2-level factorial designs.

Steps:

1 Form the data table

x1 x2 x1x2 y

−1 −1 1 601 −1 −1 54−1 1 −1 721 1 1 68



2 Compute the weighted sum of factor column values multiplying theircorresponding response column values. For example, for column 1(the x1 column), (−1) ∗ 60 + 1 ∗ 54 + (−1) ∗ 72 + 1 ∗ 68 = −10

x1 x2 x1x2 y

−1 −1 1 601 −1 −1 54−1 1 −1 721 1 1 68−10 26 2 wtd sums



3 The effect for the column (factor or interaction) is obtained bydividing the weighted sum by 2k−1 where k is the number of factors.For example, for column 1, the main effect for factor 1 is(−10)/22−1 = (−10)/2 = −5.

x1 x2 x1x2 y

−1 −1 1 601 −1 −1 54−1 1 −1 721 1 1 68−10 26 2 wtd sums

−5 13 0.5 effects



Caution - If you use the effects representation approach, check to makesure that the design you are analyzing is a proper 2k design.

You may need to divide by something other than 2k−1 if there are replicateruns in the dataset.


Calculating Effects

In industry, you will find several approaches used for calculatingeffects -

I effects representationI formal definitionI regression

The approach that is used is likely a reflection of how the materialwas learned (e.g., did you learn regression first?) and where youlearned it (e.g., statistics department vs. engineering department).

The regression approach is a fail-safe approach as long as youremember how the parameters are related to the effects.


Two-Level Factorial Designs - the 23 Case

If we have 3 factors and we construct a 23 design, we have 8 runs and wecan obtain the following information:

we have 8− 1 = 7 pieces of independent information

main effects - for 3 factors = 3 pieces of information

2-factor interaction effects: x1x2, x1x3, x2x3 = 3 pieces ofinformation

3-factor interaction effect: x1x2x3 = 1 piece of information

total of 7 pieces of information


The 23 Design - Pictorially


23 Design Example - Chemical Reactor Yield

We are investigating the effect of concentration, catalyst type andtemperature on yield in a chemical reactor. We have 3 factors so we use a23 design.

Determine

main effects - due to x1 (concentration), x2 (catalyst type - I or II),x3 (temperature)

two-factor interaction effects

three-factor interaction effects - higher-order interactions such asthese are usually not significant



The main effect of catalysttype is the difference betweenthe average yield on the backface of the cube (catalysttype = II) and the averageyield on the front face(catalyst type = I)...



Main effect of catalyst type...



The two-factor interaction effect between catalyst type and temperature is

given by 12

[(effect of cat type

at high T

)−(effect of cat type

at low T

)]

Two-factor interaction effect for cat type and T is 12 [11.5− (−8.5)] = 10.



The two-factor interaction effect is the difference between the averages onthe following two planes:



The two-factor interaction can information can be summarized usinginteraction plots - here the yields reported at each end of the lines is theaverage over the high and low values of the concentrations.



We can also visualize two-factor interaction effects using interaction plots -the lines show the change in yields for catalyst type I and type II at hightemperature and low temperature. The yield at catalyst type I and low Tis the average of the yields for the two concentrations (high, low) at thesecatalyst type and temperature conditions.


Design Decisions for Two-Level Factorial Designs

1 High and low levels for each factorI draw from process understanding, objectives of investigation, results

from previous investigations, historical operating data

2 Number of runs at each factor level3 Whether to include centre point runs - centre point runs can help

I estimate inherent noise varianceI assess curvature over the experimental region

4 Maintaining a balanced designI preserve the ”cancellation” structure of runs - minimize correlation

between parameter estimates


Deciding on the Number of Runs

Adding runs strengthens the signal-to-noise in the data

think of estimating the mean - as we add more runs, the precision ofthe estimate improves - the variance of X is σ2

X/n

precision of our predictions depends on the noise variance, thenumber of runs that we conduct, and how we choose theexperimental run conditions

as the number of runs is increased, the precision increases

Goal - perform enough runs so that the precision of the predicted effects issufficient to allow detection of a certain effect size



We use hypothesis tests (and confidence intervals) to decide on thestatistical significance of effects -

hypothesis test is ”effect is not significant” (the null hypothesis)

there are two types of risk with this hypothesisI Type I error - erroneous conclusion that the effect is statistically

significant (we incorrectly reject the null hypothesis that it isn’t) - thisis sometimes known as ”alpha-risk”

I Type II error - failure to detect a significant effect (failure to correctlyreject the null hypothesis) - sometimes referred to as ”beta-risk”

analogy - statistical quality control charts



There are expressions to relate precision to number of runs. Here is oneexample for reference -

The number of runs n required at each factor level in a 2k factorial designto detect an effect of size ∆ is

n = 2(Zα/2 + Zβ)2(σε

∆

)2

where

σ2ε is the inherent noise variance

α is the Type I error risk (false detection)

β is the Type II error risk (failure to detect)

Zα/2 is the value of the standard Normal random variable with uppertail probability of α/2


Constructing 2k Designs

The best approach is to use a systematic methodology - the one below iscalled the standard order

1 Start with the first factor, and alternate −1,+1,−1,+1, . . .2 For the second factor, alternate with every pair of runs:−1,−1,+1,+1,−1,−1,+1,+1, . . .

3 For the third factor, alternate levels every four runs:−1,−1,−1,−1,+1,+1,+1,+1,−1, . . .

4 For the fourth factor, alternate every eight runs, and so on.


Centre-Point Runs

Adding centre-point runs to the two-level factorial design improves thedesign by providing a way of estimating the noise variance directly.Centre-point runs are those with factor levels set to 0 in coded form.

Benefits -

provide replicates to enable estimation of inherent noise variation

allow assessment of curvature of the response surface - therelationship between the response and the factors

I compare average of ”corner” values to average at the centre of thedesign - if there is a significant difference, there is curvature presentand further experimentation to estimate a fully quadratic model shouldbe considered


Centre-Point Runs

Centre-point runs don’t contribute any additional information about mainor interaction effects

to see this, think of effects representation - centre-point run responsemeasurement get multiplied by 0 and don’t contribute to theweighted sum

To assess curvature of the response surface -

calculate average of the 2k runs at the corners of the experimentalregion

calculate the average of the replicate runs at the centre

use a t-test for differences in means to compare the average at thecorners to the average at the centre - assume that the observations atthe corners have the same variance as the observations at the centre(and estimate the noise variance from the replicates at the centre)


Properties of Two-Level Factorial Designs

1 Parameter estimates are uncorrelated - XTX is diagonal2 Parameter estimates have uniform precision

I uniform precision = same precisionI entries in XTX are identical (equal to 2k)I including centre points improves the precision of the intercept estimate,

but doesn’t change the precision of the other parameter estimates

3 Optimality - for any two-level experimental design, two-level factorialdesigns

I provide the most precise parameter estimatesI provide the most precise predicted responses for any prediction at a

point in the experimental region


Properties of Two-Level Factorial Designs

4 Two-level factorial designs allow the estimation ofI main effects - the terms linear in x - e.g., β1x1

I interaction effects - two-factor (e.g., β12x1x2), three-factor (e.g.,β123x1x2x3), and higher

I but not estimation of quadratics - in order to estimate quadratics, weneed at least three levels in the experimental design


Maintaining Balance in Two-Level Factorial Designs

The combinations of ±1 values in the two-level factorial designs are chosencarefully to obtain cancellation between different columns of XTX. Thiscancellation leads to uncorrelated parameter estimates. The balance alsoensures uniform precision of estimated parameters and predicted responses.

Balanced Designs

designs in which each level of every individual factor appears the samenumber of times in combination with each of the levels of every otherfactor - i.e., same number of runs at each corner of the box

e.g., 24 design - low level of x1 appears 8 times with low level of x2,and so forth

changing the balance in an experimental design can alter theproperties provided by the design



An exercise to try out - start with a 22 design, and imagine you are fittingthe model Y = β0 + β1x1 + β2x2 + β12x1x2.

1 Check that your XTX matrix is diagonal.

2 Now add an extra run at one of the run conditions in your design, sothat you have 5 runs. What does XTX look like?

3 You can actually calculate what the correlation will be between theparameter estimates without knowing the noise variance. To do this,first calculate P = (XTX)−1. Now take the (1, 2) element of theinverse matrix P, and divide it by the square root of the (1, 1)element, and the square root of the (2, 2) element. The ratio is thecorrelation between β0 and β1.



4 Try adding another row, without maintaining the balance. Calculatethe correlation again.

5 You can calculate correlations between other parameters by taking the(i, j)th element of the P matrix, and dividing by the square root ofP(i, i) and P(i, i).


Calculating the Precision of Estimated Effects

We will want to decide whether an estimated effect is statisticallysignificant - i.e., is the effect large relative to the background noise in theexperiments?

Approaches for calculating precision and assessing statistical significance -

treat the effects assessment as a regression problem and decidewhether the parameters are statistically significant

calculate the variance of the estimated effects from the fundamentaldefinition of the effects (i.e., the differences in average responses)

Remember - precision is indicated by variance of the estimated effect.


Precision of Predicted Effects - Starting from Regression

If we use regression to estimate a model

Y = β0 + β1x1 + · · ·+ βkxk + β1β2x1x2 + · · ·+ ε

the estimated main effect for x1 is 2β1, the estimated two factorinteraction between x1 and x2 is β12, and so forth.

The variance of an estimated effect is:

V ar( ˆeffecti) = V ar(2βi) = 4V ar(βi)

If the data have come from a 2k design,

V ar(βi) =σ2ε

2k

V ar( ˆeffecti) =4σ2

ε

2k=

σ2ε

2k−2


Precision of Predicted Effects - Starting from Regression

Where does V ar(βi) = σ2ε

2kcome from ?

in a 2k design, there will be 2k rows of ±1s

in XTX, the diagonal entries come from a column of X on its side(transposed) multiplying the same column - we will end up adding1(1) and −1(−1) over the n = 2k rows in the experimental design

this will give 12k

in (XTX)−1

since the covariance matrix for the parameter estimates is

(XTX)−1σ2ε , this gives us σ2

ε

2kas the variance for the parameter

estimates


Precision of Predicted Effects - from Formal Definition ofEffects

The key idea in developing the variance of predicted effects is that for twoindependent random variables, Y1 and Y2,

V ar(k1Y1 + k2Y2) = k21V ar(Y1) + k2

2V ar(Y2)

Recall that for a 2-level factorial design,(Main Effect

of a Factor

)=(Average of Responses

at High Level of Factor

)−(Average of Responses

at Low Level of Factor

)

Main Effect = yfactor=+1 − yfactor=−1



The variance of the estimated main effect is:

V ar( ˆmain effect) = V ar(yfactor=+1 − yfactor=−1)= V ar(yfactor=+1) + V ar(yfactor=−1)

What is V ar(yfactor=+1 − yfactor=−1)?

factor is at high conditions for half of the runs in the design, i.e.,2k/2 = 2k−1 runs

variance of the sample average is σ2/n where σ2 is the variance ofthe (sample random) variables added up to compute the average, andn is the number of variables added up to calculate the average

in our case, V ar(yfactor=+1) = σ2ε

2k−1



Putting it all together,

V ar( ˆmain effect) = V ar(yfactor=+1) + V ar(yfactor=−1)

=σ2ε

2k−1+

σ2ε

2k−1

= 2σ2ε

2k−1

=σ2ε

2k−2

This is the same result as that obtained from the regression approach.


Summary - Variance of Predicted Effects

The precision of the predicted effect is:

V ar( ˆmain effect) =σ2ε

2k−2

where σ2ε is the noise variance.

We don’t usually know the noise variance, and we estimate it as s2ε using

MSE from the regression

replicates

from insignificant effects - more to come ...


Testing for Significant Effects

We want to see whether an effect is zero or not, i.e., whether it isstatistically significant. We can test for significance using confidenceintervals or hypothesis tests.

Hypothesis Test -

H0 : effect = 0, Ha : effect 6= 0

test statistic: | ˆeffects ˆeffect

|

compare to tν,α/2 where ν is the degrees of freedom of the estimateof the noise variance


Obtaining Estimates of the Noise Variance

There are several possible methods - in descending order of preference -

from replicate runs in the current experimental design

from replicate runs from a previous experimental results (provided weare convinced that the previous experiments were conducted underthe same general conditions as the current experiments)

from the MSE if a regression model has been fit (best if number ofparameters p is not close to the number of runs n)

from non-significant effects - more on this shortly


Obtaining Estimates of the Noise Variance

If we have replicates for estimating variance in the current design -

pool if there is more than one replicate set (e.g., replicate sets at thecentre and corners of the experimental region)

can test for constant variance using Bartlett’s test (not covered inthis course)


Testing Significance of Effects Without a Noise VarianceEstimate

In screening studies, investigators often are interested in performing arough screening to determine significant factors:

they will often fit all of the main and interaction effects, so that for a2k design, they are fitting β0 and 2k−1 parameters, i.e., fitting 2k

parameters from 2k unique run conditions

this leaves no residuals - no way of estimating the noise variance

some of the estimated effects may just be random noise effects -others may represent deterministic effects - how can we determinewhich are just noise without needing a noise variance estimate for ahypothesis test?

Answer - use a Normal probability plot


Using Normal Probability Plots to Test Significance ofEffects

Premise - some of the estimated effects may just be random noise effects -they are non-zero because of noise in the data.

Normal probability plots -

are used to determine whether observations come from a Normaldistribution

are generated by ordering the data in ascending order, and calculatingthe cumulative fraction that each observation represents

are formed by plotting cumulative fraction versus observation onspecial Normal probability graph paper

if the observations come from a Normal distribution they will lie alonga line on the Normal probability plot



Steps -

1 estimate effects

2 place effects in ascending order, and assign a rank i to each effect,from 1 to n (the number of estimated effects)

3 calculate cumulative fractions Pi for each estimated effect: Pi = i−0.5n

4 plot cumulative fraction Pi versus estimated effect on a Normalprobability plot

5 effects that are not statistically significant (i.e., random noise) shouldlie along a line on the Normal probability plot

6 effects that are statistically significant will lie off any line present onthe plot - they aren’t random noise



Effects that are not statistically significant are considered to representnoise

form a line centred about zero - remember that we assume the noiseis Normally distributed with a mean of zero

in theory, the slope of the line is σε, the noise standard deviation

Significant effects -

will not lie on the straight line

often appear as kinks, or a steeper tail


Using JMP to Assess Effects Using Normal ProbabilityPlots

Instead of plotting the estimated effects vs. cumulative fractions, JMP plotsestimated effects vs. the value from a Normal distribution that would correspondto the cumulative fraction. For example, if we have 5 estimated effects, the 3rdestimated effect corresponds to the 50% cumulative fraction of the data. JMPwill plot this vs. 0, which is the value in a Normal distribution corresponding tothe 50% cumulative fraction.

Other points

the red and blue lines pass through (0, 0) with slope of σε, the estimatednoise standard deviation

the red line uses the MSE from fitting the effects using regression

the blue line estimates the noise variance from small (near zero inmagnitude) estimated effects


Using JMP to Assess Effects Using Normal ProbabilityPlots

For the solder defects example -

Significant factors are flux density,conveyer speed, pot temperature,and conveyer speed*pot temperature

Normal Plot

-10

-5

0

5

10

t Rati

o

conveyer speedpot temp

flux density

conveyer speed*pot temp

-3 -2 -1 0 1 2 3

Normal Quantile

Blue line is Lenth's PSE, from the estimates population.

Red line is RMSE, Root Mean Squared Error from the residual.

Normal Plot


Blocking of Experimental Designs

The scenario - we have put together a 23 design to investigate meltgrafting of functional groups on polymers - our factors are temperature x1,screw speed x2, and initiator type x3. The 23 design has 8 runs, but wecan only do 4 runs during the day shift, and 4 runs during the night shift.

if we do all of the high temperature runs during the day, and the lowtemperature runs at night, the effect of temperature is confounded(aliased) with shift effect - we won’t be able to determine conclusivelywhether the effect we saw is due to temperature, or shift

How can we divide up the runs to avoid confusing trends in the data dueto the factors to trends that might be due to shift effects?



Solution -

solution - do the runs for which x1x2x3 = +1 during the day, and dothe runs for which x1x2x3 = −1 during the night

this solution confounds shift effect with the high-order interactioneffect x1x2x3 but we suspect that this three-factor interaction effectis not significant so we are prepared to accept this

dividing runs in an experimental design in this way is called blocking

blocking is an approach for dividing runs in an experimental designinto groups in such a way that the basis for dividing is confoundedwith effects that we think are insignificant



In the previous example, we divided a 23 design into two blocks of 4 runseach. We can divide larger designs (e.g., 24 design) into more blocks ifnecessary, but we need to specify more blocking relationships.

Example - dividing 24 design into 4 blocks of runs -

need two blocking relationships - for block I, use the runs for whichx1x2x3 = +1 and x1x3x4 = +1for block II, use runs for which x1x2x3 = +1 and x1x3x4 = −1for block III, use runs for which x1x2x3 = −1 and x1x3x4 = +1for block IV , use runs for which x1x2x3 = −1 and x1x3x4 = −1

Any block effects will be confounded with x1x2x3 and x1x3x4 (and oneother implied relationship which we will discuss in the context of fractionalfactorial designs).



Blocking situations arise when we don’t have the ability to havehomogenous conditions to run our experimental design

e.g., two batches of catalyst having to be used in our 23 design whenwe hadn’t planned to include catalyst batch as an effect

e.g., having to use more than one piece of equipment - e.g., extruder,analytical instrument - to conduct the runs in the designed experiment

Blocking groups experimental runs into blocks in which the conditions areas homogenous as possible within each block

conditions are the general circumstances of the experimental runs,and not the run levels in the design.


Motivation and Introduction

Scenario - Suppose we have four factors: A, B, C, and D and we haveenough money in our budget to run 8 experiments.

a full two-level factorial design for 4 factors will be a 24 design whichhas 16 runs

can we choose 8 runs from the full 24 design that will get us most ofthe information that we want?

the 24 design will let us estimateI main effects: A, B, C, and DI two-factor interactions: AB, AC, AD, BC, BD, CDI three-factor interactions: ABC, ABD, ACD, BCDI the four-factor interaction: ABCD



For each effect in the 24 design, we have 8 runs at the high level, and 8runs at the low level, for the effect

e.g., 8 runs for which A = +1 and 8 runs for which A = −1e.g., 8 runs for which ABCD = +1 and 8 runs for whichABCD = −1

Idea - the four-factor interaction, ABCD, is not likely to be significant -often only two-factor interactions are significant, and three-factor andhigher interactions are not significant

Solution - choose the 8 runs in the 24 design for which ABCD = +1



What are the implications of choosing a design consisting of the 8 runs forwhich ABCD = +1:

we will not be able to estimate the ABCD effect - it only has onevalue (+1) in the design

if ABCD = +1, then ABC and D must be the same value - e.g., ifABC = +1 and D = +1, then ABCD = +1; alternatively, ifABC = −1 and D = −1, then ABCD = +1 ⇒ ABC = D

similarlyI AB = CDI AC = BDI AD = BCI A = BCD, B = ACD, C = ABD, D = ABC



The 8 run design we obtained from the full 24 factorial design is called a24−1 fractional factorial design

it is a half-fraction of the full design

notation - the 4 in 24−1 tells us the full design from which thefraction was taken

notation - the 1 in 24−1 tells us the fraction - the 1 means a 12

fraction - one way to think of the 1 is that we are obtaininginformation for a fourth factor, D, by throwing away information onABC that we would get from a 23 design by setting D = ABC



Fractional factorial designs have an aliasing structure

AB = CD

AC = BD

AD = BC

A = BCD, B = ACD, C = ABD, D = ABC

that tells us the implications of throwing away the 8 runs from the 24

design. We need to check the aliasing structure to make sure that wedon’t confound effects that we are interested in with other effects.



For our 24−1 fractional factorial design, we have

1 a defining relation: ABCD = +12 an aliasing structure

I AB = CDI AC = BDI AD = BCI A = BCD, B = ACD, C = ABD, D = ABC

We can construct our 24−1 design by creating a 23 design in A, B, C, andthen letting D = ABC.


2k−p Fractional Factorial Designs

Fractional Factorial Design - A 2k−p fractional factorial design is a(12

)pfraction of the full 2k design

the ”2k−p” design has 2k−p runs

the 2k−p design requires p defining relations to determine the runsfrom the full 2k design that are to be used

the easiest way to determine the run conditions is to start with thesmaller, 2k−p design runs and then use the defining relations todetermine the run levels for the other factors

I in our 24−1 example, we started with a 23 design (since we wereconsidering a 24−1 design and 4− 1 = 3), and then used the definingrelation ABCD = +1 to obtain D = ABC


Blocking Variables and Aliasing

A blocking variable is a combination of factors whose value is heldconstant in the fractional factorial design -

in our 24−1 design example, the blocking variable was ABCD and itsvalue was held at + in the fractional factorial design

blocking variables are also sometimes called defining contrasts

the defining relation is written asblocking variable1 = · · · = blocking variablep = +1

Aliases are those combinations of factors having the same levels as othercombinations of factors in the fractional factorial design.


Determining the Aliasing Structure

Determining the aliases in the fractional factorial design uses two rules -here, X is a blocking variable (e.g., ABCD):

1 X ∗ 1 = X (i.e., multiplying a term by +1 leaves their valuesunchanged

2 X ∗X = 1 (i.e., multiplying a term by itself always yields the value+1)

Example: 23−1 half fraction -

we have one blocking variable ABC, and the defining relation isABC = +1multiply both sides of the defining relation by A:A ∗ABC = A ∗+1⇒ BC = A, i.e., A = BC



Continuing in our 23−1 example,

B ∗ABC = B ∗+1⇒ AC = B, i.e., B = AC

C ∗ABC = C ∗+1⇒ AB = C, i.e., C = AB

Conclusion - all main effects are aliased (confounded) with two-factorinteractions in this design

when we calculate the main effect (e.g., of A), it represents thecombined effect of A and BC - if we assume that BC is insignificant,then we can interpret the estimated effect as representing A, but thisis an assumption!.

This is an example of the law ”you don’t get something for nothing” - ifyou eliminate runs, you are eliminating information and will have lessinformation as a result.



1 To obtain effects aliased with the main effects -I multiply each blocking variable in the defining relation by the first

factorI multiply each blocking variable in the defining relation by the second

factor and so forth ...

2 To obtain effects aliased with two-factor interactions -I multiply each blocking variable in the defining relation by a two-factor

interaction of interestI repeat this procedure for each two-factor interaction

3 This procedure can be repeated for three-factor interactions andhigher-order interactions if necessary (but it usually isn’t necessary todo this).


The Resolution of a Two-Level Fractional Factorial Design

Two-level fractional factorial designs are classified by the amount ofaliasing they contain - this is a reflection of how small a fraction we havetaken of the original design.

The resolution of a two-level fractional factorial design is the length of thesmallest blocking variable in the defining relation.

e.g., in our 24−1 example, the defining relation was ABCD = +1, wehave one blocking variable containing 4 factors, and thus theresolution of our design is 4.


Resolution of a Two-Level Fractional Factorial Design

Resolution III Designs -

the smallest blocking variable contains 3 factors ⇒ at least one maineffect is aliased with a two-factor interaction, but main effects are notaliased with other main effects

Resolution IV Designs -

the smallest blocking variable contains 4 factors ⇒ at least one maineffect is aliased with a three-factor interaction, at least one two-factorinteraction is aliased with another two-factor interaction, but maineffects are not aliased with other main effects

We can have Resolution V designs in which the length of the smallestblocking variable in the defining relation is 5, and so forth.


Example - 25−2 Design

this is a(

12

)2 = 14 fraction of the full 25 design

we will need two blocking variables to reduce the number of runsfrom 32 to 8choose runs for which ABCD = +1 AND CDE = +1: ABCD andCDE are the blocking variables

the defining relation is ABCD = CDE = +1these blocking variables imply a third blocking variable -

I since ABCD = +1 and CDE = +1, then (ABCD)(CDE) = +1 soABCD ∗ CDE = ABE = +1

the full defining relation, with the two blocking variables that wechose, and the implied blocking relation, isABCD = CDE(= ABE) = +1



in the defining relation, the implied blocking variables are typicallyindicated by enclosing them in parentheses: (= ABE)since the smallest blocking variable (”word”) in the defining relationcontains 3 factors, this is a Resolution III design

to get the aliasing structure, we determine the aliasing from each ofthe blocking variables in the defining relation:

I from ABCD = +1, we have A = BCD, B = ACD, C = ABD,D = ABC, AB = CD, AC = BD, and so forth

I from CDE = +1, we have C = DE, D = CE, E = CDI from (= ABE) = +1, we have A = BE, B = AE, E = AB



what would happen if we had chosen ABCDE = +1 andABCD = +1 as the blocking variables?

I my motivation was to use the longest words for blocking variables thatI could in the defining relations

the implied blocking variable is ABCDE ∗ABCD = E = +1 - oops- this is no good! I won’t be able to estimate the main effect forfactor E.

when you take smaller fractions from a two-level factorial design, youneed to balance off making the blocking variables as long as you can,against making them shorter to avoid situations where the impliedblocking variables are unacceptably small.



note that we could also have used ACDE = ABC = +1 as thedefining relation, which would imply the blocking variable BDE = +1there are other combinations of blocking variables that could be usedas well



this is also a quarter fraction: p = 2 here so we need 2 blockingvariables in our defining relation

blocking variables - try ABCE = +1 and ADEF = +1 - hopefullyour implied blocking variable will also be 4 factors long

implied blocking variable: ABCE ∗ADEF = BCDF = +1 - this isgood - we still have Resolution IV design

defining relation: ABCE = ADEF (= BCDF ) = +1smallest blocking variable in the defining relation has length of 4 -this is a Resolution IV design



to list the run conditions for the 6 factors, start with a 24 design in A,B, C and D (the 4 comes from 6− 2)

from the defining relation, E = ABC and F = ADE - use theserelationships to define the run conditions for factors E and F

and randomize when the experiments are actually conducted


One more piece of notation

In fractional factorial designs, the resolution of the design is sometimesnoted as a subscript.

For example:

26−2IV is a resolution IV 26−2 design

23−1III is a resolution III 23−1 design


Saturated Designs

Fractional factorial designs try to get information about effects of primaryinterest (usually main effects, plus sometimes two-factor interactions). Byadjusting blocking variables, sometimes we can get particularly efficientdesigns.

Saturated designs

focus on single-factor effects

give up on interaction effects - even two-factor interactions areconfounded with main effects

are Resolution III designs

example: 27−4III design provides information on 7 main effects in 8 runs!


Saturated Designs

Saturated designs

are screening designs - they provide information on main effects butnothing else

don’t provide for estimating error - they can be augmented withcentre-point replicates - otherwise, we need to use Normal probabilityplots to analyze the significance of estimated effects

Saturated designs, in which we obtain information about k main effects ink + 1 runs, are not available for every number of factors - the saturatednature is a consequence of how we are able to pick blocking variables inthe defining relation.


Plackett-Burman Designs

Plackett-Burman designs are a special family of saturated designs thatcome with numbers of runs that are multiples of 4 - this is a reflection ofhow they were developed.

Examples -

12 run design - used when we have between 7 and 11 factors

16 run design - used when we have between 12 and 15 factors - this isactually a 215−11 design!


Estimating Effects in Fractional Factorial Designs

Effects are estimated for two-level fractional factorial designs in the sameway as they are for full two-level factorial designs. The approaches are:

1 Regression Approach - fit a model in main effects, two-factorinteractions and more as required. Remember that the estimated i-theffect is 2βi, where βi is the i-th estimated parameter.

2 Effects representation - follow the same procedure as before, butremember that the total number of runs (in a fractional factorialdesign without replicates) is 2k−p, so you will need to divide by thisquantity instead.

3 Formal definitions - follow the definitions for main or interactioneffects, but remember that the number of runs at a high or lowcombination will be 1

22k−p.


Estimating Effects in Fractional Factorial Designs

The fail-safe approach is regression analysis, but you may sometimes beasked to use a different approach.

The key difference is in the interpretation of the estimated effects -

the estimated effect represents the effect of the factor in question plusall effects aliased with that factor

e.g., in the 23−1III design, we had ABC = +1 as the defining relation -

this means that the estimated main effect for factor A represents:main effect for A+ effect of BC .


Precision of Estimated Effects

Precision for two-level fractional factorial designs is handled the same wayas for full two-level factorial designs. For a 2k−p design, the precision of an

estimated effect is 4 σ2ε

2k−pwhere σ2

ε is the noise variance.

We can obtain as estimate of the noise variance from:

replicates

MSE in the regression

directly from insignificant effects

The key difference from the full two-level factorial design case is thenumber of runs being used.


Testing Significance of Effects

The statistical significance of effects in fractional two-level factorialdesigns is tested in the same way as for full two-level factorial designs.The only difference is the number of runs involved.

Normal probability plots are used more often to look for significanteffects in fractional two-level factorial designs - this is because weoften don’t have extra runs from which to estimate the noise variance.


Properties of Two-Level Fractional Factorial Designs

Two-level fractional factorial designs are:

balanced designs - same number of runs at high and at low levels ofeach factor

parameter estimates will be uncorrelated, but remember that theparameter estimates represent the effects of the factor in question andall effects aliased with it - e.g., for the 23−1

III design, A = BC so the

βA parameter represents the effect A+BC. However, it will beuncorrelated with βB which represents the effect B +AC.

extensions of two-factorial designs - Foldover Designs and reversingthe levels of a single factor are ways to add to a fractional factorialdesign to eliminated some of the aliasing in a targeted way


Higher-Order Designs

So far, we have looked at two-level factorial and fractional factorialdesigns. Having two levels in each factor in a factorial arrangement meansthat we can estimate main effects and interaction terms. We can’testimate higher-order terms such as quadratics.

To estimate higher-order terms, we need more levels - at least three levelsin each factor. Designs that support estimation of higher-order terms arecalled Higher-Order Designs. The common higher-order designs are:

central composite design

face-centred central composite design

Box-Behnken design

three (or more)-level factorial design - seen earlier - 3k designs growlarge very rapidly and may not be practical because of this


Central Composite Designs

Central composite designs are formed from a 2k design by adding

centre-point runs

”star points” - runs with a given factor level greater in magnitudethan +1, with all other factors held at their centre points (i.e., zero)

total number of points in a central composite design is 2k + 2k +mfor k factors, with m runs at the centre point.

Star points -

are at a distance (2k)14 from the centre (which is 0)

star points are selected to provide rotatability - uniform precision forresponses an equal distance from the centre point



Example - central composite design in 2 factors -



Advantages -

can build a central composite design from a previous two-levelfactorial design for sequential experimentation

uniform precision of predictions at a given distance from the centre ofthe design

in a full quadratic model, all parameter estimates, except thequadratics, will be uncorrelated

Disadvantage -

star points may lie outside the desired experimental region - eitherneed to shrink the factorial box, or go a bit beyond the operatingregion limits


Face-Centred Central Composite Designs

Face-centred Central Composite Designs are Central Composite Designs inwhich the star points have been constrained to be ±1 - i.e., to lie on theedges of the experimental region.The remainder of the design is the sameas the central composite design.Advantages -

can build a face-centred central composite design from a previoustwo-level factorial design for sequential experimentation

in a full quadratic model, all parameter estimates, except thequadratics, will be uncorrelated

Disadvantage -

the design is no longer rotatable - the precision of predictions won’tnecessarily be uniform at a given distance from the centre


Face-Centred Central Composite Designs

Picture


Box-Behnken Designs

Box-Behnken designs are an alternative for keeping the runs within theboundaries of the experimental region -

resulting design is rotatable or nearly rotatable, meaning thatprecision of predictions will be almost uniform for points a fixeddistance from the centre of the design

Box-Behnken designs are formed by combining two-level factorialdesigns with incomplete block designs

available for 3 or more factors


Example - Higher-Order Designs in 3 Factors

Central Composite Design

x1 x2 x3−1 −1 −11 −1 −1−1 1 −11 1 −1−1 −1 11 −1 1−1 1 11 1 1

−1.68 0 01.68 0 00 −1.68 00 1.68 00 0 −1.680 0 1.680 0 00 0 00 0 0

Face-Centred Central CompositeDesign

x1 x2 x3−1 −1 −11 −1 −1−1 1 −11 1 −1−1 −1 11 −1 1−1 1 11 1 1−1 0 01 0 00 −1 00 1 00 0 −10 0 10 0 00 0 00 0 0


Example - Higher-Order Designs in 3 Factors

Central Composite Design

x1 x2 x3−1 −1 −11 −1 −1−1 1 −11 1 −1−1 −1 11 −1 1−1 1 11 1 1

−1.68 0 01.68 0 00 −1.68 00 1.68 00 0 −1.680 0 1.680 0 00 0 00 0 0

Box-Behnken Design

x1 x2 x3−1 −1 01 −1 0−1 1 01 1 0−1 0 −11 0 −1−1 0 11 0 10 −1 −10 1 −10 −1 10 1 10 0 00 0 00 0 0


Estimating Second-Order Models

Full second-order models will contain main effects, two-factor interactions,and quadratic terms. They can be estimated using regression - e.g., fortwo factors x1 and x2:

Y = β0 + β1x1 + β2x2 + β12x1x2 + β11x21 + β22x

22 + ε

for any of the higher-order designs, the quadratic parameter estimateswill be correlated with each other, and with the intercept parameterestimate

the main effects and interaction terms will be uncorrelated with all ofthe other parameter estimates

second-order models can represent curvature, including maxima orminima, and are the basis for the statistically-based optimizationtechnique called Response Surface Methodology


chee418/801 - module 3: design of experimentsnunez/mastertecnologiastelecomunicacion/tema… · 4...

Documents