sizing mixture ( rsm) designs - asqrube.asq.org/.../sizing-mixture-rsm-designs.pdf · sizing...

1

Sizing Mixture (RSM) Designsfor Adequate Precision via

Fraction of Design Space (FDS)

Pat WhitcombStat-Ease, Inc.612 746 2036

Gary W. OehlertSchool of Statistics

U i it f Mi t

1

612.746.2036fax 612.746.2056

[email protected]

University of Minnesota612.625.1557

[email protected]

Sizing Mixture (RSM) Designs


Review – Power to size factorial designs

Precision in place of powerPrecision in place of powerIntroduce FDS

Sizing designs for precisionTwo component mixtureThree component constrained mixture

2

Sizing designs to detect a difference

Summary


Copyright 2009 Stat-Ease, Inc.

2

Sizing Factorial Designs

KnownFactors

UnknownFactors

T i i l

Screening KnownFactors

UnknownFactors

T i i l

ScreeningDuring screening and characterization the emphasis is on identifying factor effects

yes

Factor effectsand interactions

Response

Curvature?

Screening

no

Trivialmany

Vital fewCharacterization

Optimizationyes


Response

Curvature?

Screening

no

Trivialmany


Optimization

is on identifying factor effects.

What are the important design factors?

For this purpose power is an ideal metric to evaluate design suitability.

3

Surfacemethods

Confirm? Backup

Celebrate!

no

yes

Verification

Surfacemethods

Confirm? Backup

Celebrate!

no

yes

Verification


12

One Factor

Factorial Design – Power23 Full Factorial Δ=2 and σ=1

11

12

9

10

11

R1

-1.00

-0.50

0.00

0.50

1.00

-1.00

-0.50

0.00

0.50

1.00

8

9

10

R

1

A: A B: B Δ

4

-1.00 -0.50 0.00 0.50 1.00

8

A: ASizing Mixture (RSM) Designs


3


Power is reported at a 5.0% alpha level to detect the specified signal/noise ratio.Recommended power is at least 80%.

R1Signal (delta) = 2.00 Noise (sigma) = 1.00 Signal/Noise (delta/sigma) = 2.00

5

A B C57.2 % 57.2 % 57.2 %

The following three slides dig into how power is computed.


One Replicate of 23 Full Factorial C = (XTX)-1 matrix

The design determines the standard error of the coefficient:

0.125 0 0 0 0 0 0 0⎛⎜

⎞⎟

C

0

0

0

0

0

0

0.125

0

0

0

0

0

0

0.125

0

0

0

0

0

0

0.125

0

0

0

0

0

0

0.125

0

0

0

0

0

0

0.125

0

0

0

0

0

0

0.125

0

0

0

0

0

0

⎜⎜⎜⎜⎜⎜⎜⎜⎜

⎟⎟⎟⎟⎟⎟⎟⎟⎟

=

6

( ) ( )i i i

i 2 2i ii

t-value = SE c ˆ 0.125 ˆβ β β

= =β σ σ


0 0 0 0 0 0 0 0.125⎜⎝

⎟⎠


4

NonCentrality Parameter23 Full Factorial Δ=2 and σ=1

The reference t distribution assumes the null hypothesis of Δ = 0. The noncentrality parameter (2.828) defines the t distribution under the alternate hypothesis of Δ = 2

( )( )

ii

i 2 2ii ii

2

2noncentrality = c ˆ c ˆ

1

0 125 1

Δβ

=σ σ

=

distribution under the alternate hypothesis of Δ = 2.

7

( )( )20.125 1

1 2.8280.3536

= =



noncentral tα=0.05,df=4 with noncentrality parameter of 2.828

Power = 57.2%

2.828

8Sizing Mixture (RSM) Designs


5

Factorial Design – PowerTwo Replicates of 23 Full Factorial Δ=2 and σ=1

Power is reported at a 5.0% alpha level to detect the specified signal/noise ratio.Recommended power is at least 80%.

R1Signal (delta) = 2.00 Noise (sigma) = 1.00 Signal/Noise (delta/sigma) = 2.00

9

A B C95.6 % 95.6 % 95.6 %


Two Replicates of 23 Full FactorialC = (XTX)-1 matrix

The design determines the standard error of the coefficient:0.0625 0 0 0 0 0 0 0⎛

⎜⎞⎟

C

0

0

0

0

0

0

0.0625

0

0

0

0

0

0

0.0625

0

0

0

0

0

0

0.0625

0

0

0

0

0

0

0.0625

0

0

0

0

0

0

0.0625

0

0

0

0

0

0

0.0625

0

0

0

0

0

0

⎜⎜⎜⎜⎜⎜⎜⎜⎜

⎟⎟⎟⎟⎟⎟⎟⎟⎟

=

10

( ) ( )i i i

i 2 2i ii

t-value = SE c ˆ 0.0625 ˆ

β β β= =

β σ σ


0 0 0 0 0 0 0 0.0625⎜⎝

⎟⎠


6

NonCentrality ParameterTwo Replicates of 23 Full Factorial Δ=2 and σ=1

ii 2

Δβ

( )( )

ii 2 2

ii ii

2

2noncentrality = c ˆ c ˆ

1

0.0625 1

1 4 0

β=

σ σ

=

11

4.00.25

= =


Factorial Design – PowerTwo Replicates of 23 Full Factorial Δ=2 and σ=1

noncentral tα=0.05,df=12 with noncentrality parameter of 4.0

Power = 95.6%

4.0



7

Conclusions

Factorial DOE

D i i dDuring screening and characterization (factorials) emphasis is on identifying factor effects.


For this purpose power is an

13







14


Summary



8

Sizing Response Surface Designs

KnownFactors

UnknownFactors

T i i l

Screening KnownFactors

UnknownFactors

T i i l

Screening

yes


Response

Curvature?

Screening

no

Trivialmany


Optimizationyes


Response

Curvature?

Screening

no

Trivialmany


Optimization

When the goal is optimization emphasis is the fitted surface.

How well does the surface represent true behavior?

F hi i i

15

Surfacemethods

Confirm? Backup

Celebrate!

no

yes

Verification

Surfacemethods

Confirm? Backup

Celebrate!

no

yes

Verification

For this purpose precision (FDS) is an good metric to evaluate design suitability.(Assuming model adequacy; i.e. insignificant lack of fit.)


1. Estimate the designed for polynomial well.

2. Give sufficient information to allow a test for lack of fit.H i d i i t th ffi i t i th d l

Design Properties

Have more unique design points than coefficients in the model.Provide an estimate of “pure” error.

3. Remain insensitive to outliers, influential values and bias from model misspecification.

4. Be robust to errors in control of the component levels.

5. Provide a check on model assumptions, e.g., normality of errors.

16

6. Generate useful information throughout the region of interest,i.e., provide a good distribution of .

7. Do not contain an excessively large number of trials.( ) 2ˆVar Y σ



9

FDSFraction of Design Space

Fraction of Design Space:Calculates the volume of the design space havingCalculates the volume of the design space having a prediction variance (PV) less than or equal to a specified value.The ratio of this volume to the total volume of the design volume is the fraction of design space.Produces a single plot showing the cumulative f ti f th d i th i (f

17

fraction of the design space on the x-axis (from zero to one) versus the PV on the y-axis.


FDS – PVFraction of Design Space

Prediction Variance:

( )ˆvar y

PV is a function of:

x0 – the location in the design space (i.e. the x coordinates for all model terms).

( )( ) ( ) 10

0 0 02

var −= = T T

yPV x x X X x

s

18

X – the experimental design (i.e. where the runs are in the design space).



10

FDS – StdErrFraction of Design Space

Prediction standard error of the expected value:

( )ˆ( )

( ) ( )

( ) ( )0

100 0 02

ˆ0 0

ˆvar −= =

= =

T T

y

yPV x x X X x

s

sStdErr x PV x

s


FDS PlotFraction of Design Space

1. Pick random points in the design space.

2 Calculate the standard error of the expected value2. Calculate the standard error of the expected value

3. Plot the standard error as a fraction of the design space.

( )01y T T

0 0

SEx X X x

s−

=

FDS Graph

0.800.901.00

20Fraction of Design Space

StdE

rr M

ean

0.00 0.25 0.50 0.75 1.00

0.000.100.200.300.400.500.600.70



11

Simplex-LatticeAugmented and 4 Replicates

A: A1.00

0.572

FDS Graph1.00

0.00 0.00

0.47

0.57 0.57

0.57

2

StdE

rr M

ean

0.25

0.50

0.75

0.47

0.57

Mix section 2 21

B: B1.00

C: C1.000.00

StdErr of Design

0.470.470.570.57

0.572 2

Fraction of Design Space

0.00 0.25 0.50 0.75 1.00

0.00





22


Summary



12

Two Component MixtureLinear Model

7

Two Component Mix

4

5

6

R1

2

2

Sizing Mixture (RSM) Designs 23

3

0

1

0.25

0.75

0.5

0.5

0.75

0.25

1

0

Actual A

Actual B

2


7

Two Component MixThe solid center line is the fitted model;is the expected value or mean

prediction.y

4

5

6

R1

The curved dotted lines are the computer generated confidence limits, or the actual precision.

d Is the half-width of the desired confidence interval, or the desired precision. It is used to create the outer straight lines.

ˆ ±y d


Note: The actual precision of the fitted value depends on where we are predicting.

3

0

1

0.25

0.75

0.5

0.5

0.75

0.25

1

0

Actual A

Actual B


13


Evaluate the standard error as a fraction of design space for this design.

4

5

6

7

R1

Two Component Mix

2

2


25

3

01

0.250.75

0.50.5

0.750.25

10

Actual AActual B

2


Click on “Evaluate” and then “Graphs”:



14


Want a linear surface to represent the true response value within ± 0.64 with 95% confidence.

The overall standard deviation for this response is 0.55.

Enter:

d = 0 64


d = 0.64

s = 0.55

a (α) = 1 – 0.95 = 0.05


Only 53% of the design space is precise enough to predict the mean within ± 0.64.



15

Define PrecisionHalf Width of the Confidence Interval

Confidence interval on the expected value:

The mean response is estimated and the precision of the

Statisticaldetail

The mean response is estimated and the precision of the estimate is quantified by a confidence interval:

We will use the half-width of the confidence interval (d) to define the precision desired:

( )ydfy t s ˆ,2

ˆα±

p

Mix section 4 29

( )ˆ ˆ,2,2

= =y ydfdf

dd t s or stαα

What Precision is Needed?Mean Confidence Interval Half-Width

Statisticaldetail

InputHalf-width of confidence interval: d

( )

( )

ˆ,2

1

ˆ

−

±

= ydf

T T

y d

d t s

X X

α

Estimate of standard deviation: s

Mix section 4 30

( )

( ) ( )

ˆ 0 0

1ˆ0 0

−

=

= =

T Ty

y T T

s s x X X x

sStdErr Mean FDS x X X x

s


16


Want a linear surface to represent the true response value within ± 0.64 with 95% confidence.

Statisticaldetail


.05 ,52

ˆ

2

For 95% confidence 2.571, 0.64 & 0.55

0.64 0.252.571

= = =

= = =ydf

t d s

dstα

Mix section 4 31

( )

,2

ˆ 0.25 0.450.55

= = =

df

ysStdErr Mean FDS

s

FDS – StdErr Mean PlotFraction of Design Space

1. Pick random points in the design space.

2 Calculate the standard error of the expected value

Statisticaldetail

2. Calculate the standard error of the expected value


( )01y T T

0 0

SEx X X x

S−

=

FDS Graph

0.901.00

Mix section 4 32Fraction of Design Space

StdE

rr

0.00 0.25 0.50 0.75 1.00

0.000.100.200.300.400.500.600.700.80


17


53% is precise enough, i.e.53% has a StdErr ≤ 0.45


d = 0.64s = 0.55α = 0.05


53% of the design space h h d i d i i

7

Two Component Mix

53%has the desired precision, i.e. is inside the solid straight (red) lines.

4

5

6

R1 ˆ ±y d

53%


3

0

1

0.25

0.75

0.5

0.5

0.75

0.25

1

0

Actual A

Actual B


18

Sizing for PrecisionFDS ≥ ?

How good is good enough? Rules of thumb:• For exploration want FDS ≥ 80%• For verification want FDS ≥ 90% or higher!

What can be done to improve precision?• Manage expectations; i.e. increase d• Decrease noise; i.e. decrease s• Increase risk of Type I error; i.e. increase alphayp ; p• Increase the number of runs in the design






36


Summary



19

Mixture Simplex Quadratic Model

Most of the action

y 4A 4B 8C 16AC= + + +12

14

Most of the action occurs on the A-C edge because of the AC coefficient of 16. The quadratic coefficient of 16 means that the response is 4 units

A (1) B (0) C (1)

0

2

4

6

8

10

R

1


response is 4 units higher at A=0.5, C=0.5 than one would expect with linear blending.

A (0)

B (1)

C (0)

Mixture Simplex Quadratic Model

14

This illustrates the

y 4A 4B 8C 16AC= + + +

4

6

8

10

12

4β

C 8β =

AC14

4β =

This illustrates the quadratic blending along the A-C edge as C increases from zero to one.


0

2

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

C

A 4β =


20

Mixture ConstrainedDesign for Quadratic Model

0.0 ≤ A ≤ 1.00.0 ≤ B ≤ 1.0

A: A1.000

0.0 ≤ C ≤ 0.1Total = 1.0

0.000 0.000


B: B1.000

C: C1.0000.000

StdErr of Design

Mixture ConstrainedQuadratic Model (1 replicate)

Power at 5 % alpha level for effect ofTerm StdErr* VIF Ri-Squared 1 s 2 s 3 s

% % %A 0.87 3.44 0.7094 5.0 % 5.0 % 5.0 %B 0.87 3.44 0.7094 5.0 % 5.0 % 5.0 %C 233.34 2722.29 0.9996 5.0 % 5.0 % 5.0 %

AB 2.86 2.19 0.5442 22.9 % 67.3 % 94.7 %AC 258.98 1092.98 0.9991 5.0 % 5.0 % 5.0 %BC 258.98 1092.98 0.9991 5.0 % 5.0 % 5.0 %

*Basis Std. Dev. = 1.0


Power is bottomed out at alpha!


21


6. Generate useful information throughout the region of interest.

Question: Will predictions, using the quadratic model from this design, be precise enough for our purposes?

To know the truth requires an infinite number of runs; most likely this will exceed our budget.

So the question is how precisely do we need to estimate the response?


estimate the response?

The trade off is more precision requires more runs.


Confidence interval on the expected value:

The mean response is estimated and the precision of theThe mean response is estimated and the precision of the estimate is quantified by a confidence interval:

We will use half-width of the confidence interval (d) to define the precision desired:

( )ydfy t s ˆ,2

ˆα±


p

( )ˆ ˆ,2,2

= =y ydfdf

dd t s or stαα


22

InputHalf-width of confidence interval: d

What Precision is Needed?Confidence Interval Half-Width

Estimate of standard deviation: s

( )

ˆ

,2

1−

=

=

ydf

T T

dst

s s x X X x

α


( )

( ) ( )

ˆ 0 0

1ˆ0 0

−

=

= =

y

y T T

s s x X X x

sStdErr Mean FDS x X X x

s


Want quadratic surface to represent the true response value within ±10 with 95% confidence.p

The standard deviation for this response is 7.8.

.05 ,72

ˆ

,2

For 95% confidence 2.365, 10 & 7.8

10 4.232.365

= = =

= = =ydf

t d s

dstα


( )

,2

ˆ 4.23 0.547.8

= = =

df

ysStdErr Mean FDS

s


23


Only 58% of the design space has StdErr ≤ 0.54

d = 10


d 0s = 7.8α = 0.05

Mixture ConstrainedQuadratic Model (2 replicates)

Power at 5 % alpha level for effect ofTerm StdErr* VIF Ri-Squared 1 s 2 s 3 s

% % %A 0.62 3.44 0.7094 5.0 % 5.0 % 5.0 %B 0.62 3.44 0.7094 5.0 % 5.0 % 5.0 %C 164.99 2722.29 0.9996 5.0 % 5.0 % 5.0 %

AB 2.02 2.19 0.5442 47.1 % 96.5 % 99.9 %AC 183.12 1092.98 0.9991 5.0 % 5.0 % 5.0 %BC 183.12 1092.98 0.9991 5.0 % 5.0 % 5.0 %

*Basis Std. Dev. = 1.0


Adding a replicate does little to increase power!


24

Mixture ConstrainedWhy Does Power Not Increase?

Quadratic, linear or combinations of them model response. Model Coefficients are Correlated!

14

4

6

8

10

12

14

Linear

Quadratic


Individual coefficients can not be resolved!

0

2

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

Constraint

C


Want quadratic surface to represent the true response value within ± 10 with 95% confidence.


.05 ,2

ˆ

2

,2

0For 95% confidence , 10 & 7.8

10 4.792.086

2.086= = =

= = =ydf

t d s

dstα


( )

,2

ˆ 4.79 0.617.8

= = =ysStdErr Mean FDS

s


25


100% of the design space has StdErr ≤ 0.61


Precision Depends On

The size of the confidence interval half-width (d):A larger half-width (d) increases the FDS.

The size of the experimental error σ:A smaller s increases the FDS.

The α risk chosen:A larger α increases the FDS.

Choose design appropriate to the problem:


Choose design appropriate to the problem:Size the design for the precision required.


26

Conclusions

Factorial DOE Mixture Design andResponse Surface Methods

During screening and characterization (factorials) emphasis is on identifying factor effects.


When the goal is optimization (usually the case for mixture design & RSM) emphasis is on the fitted surface.

How well does the surface represent true behavior?



For this purpose precision (FDS) is a good metric to evaluate design suitability.





52


Summary



27

Detecting a Difference of Δwherever it may occur

Δ


What is the probability of finding a difference ≥ Δif it occurs between any two points in the design space?

FPDS – StdErr Diff PlotFraction of Paired Design Space

1. Pick random pairs of points in the design space.

2 Calculate the standard error of the difference2. Calculate the standard error of the difference


( ) ( ) ( )( )1 21 1 1ˆ ˆy y T T T T T T

1 1 2 2 1 2

SEx X X x x X X x 2 x X X x

s− − −− = + −

FPDS Graph

1.60

1.80

2.00

Sizing Mixture (RSM) Designs 54Fraction of Paired Design Space

Std

Err

Diff

0.000 0.250 0.500 0.750 1.000

0.00

0.20

0.40

0.60

0.80

1.00

1.20

1.40

1.60


28

Revisit – Mixture ConstrainedDesign for Quadratic Model

0.0 ≤ A ≤ 1.00.0 ≤ B ≤ 1.0

A: A1.000

0.0 ≤ C ≤ 0.1Total = 1.0

0.000 0.000


B: B1.000

C: C1.0000.000

StdErr of Design


Want to detect a difference of 10 on a quadratic surface with 95% confidence.

The overall standard deviation this for response is 7.8.

.05 ,72For 95% confidence 2.365, 10 & 7.8

10 4.232 365Δ

= Δ = =

Δ= = =

t s

st


( )

,22.365

4.23 0.547.8

Δ= = =

dft

sStdErr Diff FPDSs

α


29


Only 37% of the design space has StdErr ≤ 0.54


Δ = 10s = 7.8α = 0.05


Want to detect a difference of 10 on a quadratic surface with 95% confidence.

The overall standard deviation this for response is 7.8.

.05 ,2

2

20For 95% confidence , 10 & 7.8

10 4.792.086

2.086

Δ

= Δ = =

Δ= = =

df

t s

stα


( )

,2

4.79 0.8

.617

Δ= = =

df

sStdErr Diff FPDSs


30


86% of the design space has StdErr ≤ 0.61


Detecting a Difference of Δ Depends On

The size of the difference (Δ):A larger difference (Δ) increases the FPDS.

The size of the experimental error σ:A smaller σ increases the FPDS

The α risk chosen:A larger α increases the FPDS.

Choose design appropriate to the problem:Si th d i t d t t diff f i t t


Size the design to detect a difference of interest.


31

Conclusions

Factorial DOE Mixture Design andResponse Surface Methods

During screening and characterization (factorials) emphasis is on identifying factor effects.


When the goal is to detect a change in the response emphasis is on differences over the fitted surface.

Does a difference ≥ D occur in the design space?



For this purpose precision (FPDS) is a good metric to evaluate design suitability.





62


Summary



32

Summary

Two criteria covered:

1. Precision of expected value, StdErr Mean:1. Precision of expected value, StdErr Mean:Use the StdErr Mean when the purpose of the DOE is to quantify shape; i.e. functional design (e.g. optimization).

2. Precision to detect a difference, StdErr Diff:Use the StdErr Diff when the purpose of the DOE is t d t t h i th d i


to detect change in the design space.

Summary

Two criteria not covered:

1. Precision of predicted value, StdErr Pred:1. Precision of predicted value, StdErr Pred:Use the StdErr Pred when the purpose of the DOE is to develop a model for prediction.

2. Precision to form a tolerance interval, TI Multiplier:Use the TI Multiplier when the purpose of the DOE is to form a tolerance interval at a point in the d i


design space.(new in Design-Expert version 8)


33

References

[1] Gary W. Oehlert (2000), A First Course in Design and Analysis of Experiments, W. H. Freeman and Company.

[2] Douglas C. Montgomery (2005), 6th edition, Design and Analysis of Experiments, John Wiley and Sons IncWiley and Sons, Inc.

[3] Gary Oehlert and Patrick Whitcomb (2001), Sizing Fixed Effects for Computing Power in Experimental Designs, Quality and Reliability Engineering International, July 27, 2001

[4] Alyaa R. Zahran, Christine M. Anderson-Cook and Raymond H. Myers, Fraction of Design Space to Assess Prediction, Journal of Quality Technology, Vol. 35, No. 4, October 2003

[5] Heidi B. Goldfarb, Christine M. Anderson-Cook, Connie M. Borror and Douglas C. Montgomery, Fraction of Design Space plots for Assessing Mixture and Mixture-Process Designs, Journal of Quality Technology, Vol. 36, No. 2, October 2004.

[6] Steven DE Gryze, Ivan Langhans and Martina Vandebroek, Using the Intervals for Prediction: A Tutorial on Tolerance Intervals for Ordinary Least-Squares Regression,

65

y q g ,Chemometrics and Intelligent Laboratory Systems, 87 (2007) 147 – 154

[7] Raymond H. Myers, Douglas C. Montgomery and Christine M. Anderson-Cook (2009), 3rd

edition, Response Surface Methodology, John Wiley and Sons, Inc.[8] John A. Cornell (2002), 3rd edition, Experiments with Mixtures, John Wiley and Sons, Inc.[9] Wendell F. Smith (2005), Experimental Design for Formulation, ASA-SIAM Series on

Statistics and Applied probability.



Thank You for AttendingThank You for Attendinggg

To obtain a copy of my slides:

Give me your business card

S d t t t@ t tSend a request to [email protected]



sizing mixture ( rsm) designs - asqrube.asq.org/.../sizing-mixture-rsm-designs.pdf · sizing...

Documents