jmp demo slides

Laboratory for Interdisciplinary Statistical Analysis

LISA helps VT researchers benefit from the use of Statistics

Collaboration Walk In Consulting

Experimental Design • Data Analysis • Interpreting ResultsGrant Proposals • Software (R, SAS, JMP, SPSS...)

CollaborationFrom our website request a meeting for personalized statistical adviceGreat advice right now:

Walk-In ConsultingMonday—Friday* 12-2PM for questions

requiring <30 mins*Mon—Thurs during the summerGreat advice right now:

Meet with LISA before collecting your data

Short Courses

g

Designed to help graduate students apply statistics in their research

All services are FREE for VT researchers. We assist with research—not class projects or homework.All services are FREE for VT researchers. We assist with research not class projects or homework.

www.lisa.stat.vt.edu1

Using JMP® for Statistical Analysis Part IIPart II

– Design and Analysis of ExperimentsWandi Huang

Laboratory for Interdisciplinary Statistical AnalysisDepartment of Statistics, Virginia Tech

http://www.lisa.stat.vt.edu/

02/08/2011

2

Course OutlineCourse Outline

• Introduction and Basic Principles

• Introduction to Factorial Designs

• Screening Designs

• Response Surface Designs• Response Surface Designs

• Resources

3

Section Outline ‐ IntroductionSection Outline Introduction

Ob ti l St d D i f E i t (DOE)• Observational Study vs. Design of Experiments (DOE)• Terminology• EffectsEffects• Principles of Good Design • Sequential Nature of DOE• Advantages of DOE

4

DOE DefinitionDOE Definition

A designed experiment is a test or series of tests in g pwhich purposeful changes are made to the input variables of a process or system so that we may observe and identify the reasons for the changes inobserve and identify the reasons for the changes in the output response.

– From Design and Analysis of Experiments by Douglas Montgomery

5

IntroductionIntroduction

• Drawing Conclusions from a Designed Experiment vs• Drawing Conclusions from a Designed Experiment vs. from Observational Study

– A plot that shows a relationship between two variables does not necessarily prove a true cause‐and‐effect relationship

– Correlation between two variables often occurs because they yare both associated with a third factor – lurking variable

– A scatter plot is useful for identifying potential relationships, but designed experiments must be used to verify causalityg p y y

6

IntroductionIntroduction

• Key pointKey point– To determine cause and effect in a system you have to interfere with it (not just passively observe it)to interfere with it (not just passively observe it)

From George Box (Famous Statistician) ‐ 1966From George Box (Famous Statistician) 1966

7

TerminologyTerminology

• Factor• Factor– Also known as input, “X” variable, explanatory variable, or independent variable

– A variable that is controlled for purposes of the experiment. It is believed that the factor will have some relationship with the response.p p

• Level– A particular value of a factor– For example, if temperature is a factor, levels could be 250˚, 300˚ or 350˚ F

8


R• Response– Usually a measurement on a product or process that you are interested inare interested in

– Also known as the output, “Y” variable, response variable, or dependent variable

– An experiment may have multiple responses but often it is best to focus on the most important response(s)

The methodology discussed in this course does not– The methodology discussed in this course does not typically handle non‐continuous response variables

9


Noise Factors/Variables - a factor that potentially impacts the p y presponse but you are not necessarily interested in controlling

Or it is expensive to controlYou want to make the response consistent across settings of p gthe other factors (control factors)This “robustness” minimizes the impact of the noise factorExamples of Noise Factors/Variables

Raw MaterialEnvironmental ConditionsDifferent Suppliers

keep noise variables in mind when designing andanalyzing an experiment!

10

TerminologyGeneral ModelTerminology

x2 xpx1

. . .Factors

Inputs Outputsy

zqz2z1

. . .Noise

11


• Coded vs Uncoded Variables• Coded vs. Uncoded Variables– Using coded variables consists of coding factor levels to a scale where ‐1 is equal to the low level and 1 is equal to q qthe high level

– Provides the factors on a common or dimensionless scale

– Determines the relative impact of the factors within the design space

– Uncoded variables are the original factor levels– Uncoded variables are the original factor levels

– Statistical analysis should be done using coded variables –which JMP does

– Output will be in terms of coded variables12


• Coded vs. Uncoded Variables ‐ examplesCoded vs. Uncoded Variables examples

Factor Uncoded CodedMachine A -1

B 1

Temperature 300˚ -1

350˚ 0

400˚ 1

Time 15 minutes -1

60 minutes 1

EffectsEffects

• Main effect – The change in the response due to the change in a factor levelg– This effect can be linear or curved

• Interaction effect – The change in the response due to simultaneous change inresponse due to simultaneous change in factor levels

14

EffectsEffects95

90A linear main effect for a

85

factor with two levels is simply represented byTo

rque

80

represented by a straight line

75

Low High

Line Speed

15

EffectsEffects95

85

90A main effect for a factor with more than

80

85 with more than 2 levels is similar

Torq

ue

70

75 Note – Only consider the main effects if

T

65

Low Middle High

no interaction is present

Low Middle High

Line Speed16

Interaction EffectInteraction Effect

• If an interaction effect is present, lines on an interaction plot will not be parallel

• If the interaction is rque

present, then using the main effects will be misleading

To

Low HighClutch Setting (High)

Clutch Setting (Low)Line Speed

Low High Clutch Setting (Low)

Principles of Experimental DesignPrinciples of Experimental Design

Good experimental designs will employ the following principles where appropriate:

– Randomization– Replication (instead of just repetition)– Blocking

18


• Randomization both the assignment of the• Randomization – both the assignment of the experimental units to factor combinations and the order in which the individual runs of the experiment are

f d d l d t i dperformed are randomly determined– Randomization is a valuable device for dealing with unavoidable sources of variability and reduce the risk of an unknown or unexpected occurrence jeopardizing accurate conclusions

– It helps to avoid time effectsp– By properly randomizing the experiment, one “averages out” the effects of noise variables that may be presentbe present

19


• Factor combination – a specific combination of factor levelsFactor combination a specific combination of factor levels that is used in an experiment. Also known as a treatment combination

• Experimental Unit ‐ the smallest entity to which a factor combination is applied

• Observational Unit – the entity that is measured– Not necessarily the same as the experimental unit

– If an experimental unit is measured multiple times, each measurement isIf an experimental unit is measured multiple times, each measurement is an observational unit

• A run occurs each time we impose and carry out a particular f bi i i l i l ifactor combination on a particular experimental unit– We can have multiple measurements on each run

20


R li ti Th f f th f t• Replication – The performance of the same factor combination multiple times on different experimental units. Each different experimental p punit is a replicate

• Repetition – is when the same experimental unit is measured more than once or when several units are sampled during the same run.– When repetition occurs the experimental unit and– When repetition occurs, the experimental unit and observational unit are not the same

• An experiment can have replication and/or repetition

21

Replication ExampleReplication Example

• The goal of the Baby Care experiment was to identifyThe goal of the Baby Care experiment was to identify process variables that have a significant impact on product quality– Several factors considered:

– For each factor combination, the levels of each factor were set, then 15 pads were tested

– One factor combination was done at two different times so a total of 30 pads were tested

– The 15 are repetitions, and the factor combination performed at two different times are replicates


• Blocking – is a technique that is used to account for theBlocking is a technique that is used to account for the unwanted variation that could be caused by non‐homogeneous conditions, so they do not distort the analysis of the factors that are of interestthe factors that are of interest– In a designed experiment, a block is a portion of the experimental

material that should be more homogeneous than the entire set of material

– Blocking involves making comparisons among the experimental conditions within each block

– Multiple randomization schemes are needed ‐ both the blocks and within blocks

– One is usually not interested in estimating the effects of the blocking variable

– Examples: machines days operators batches of raw materialExamples: machines, days, operators, batches of raw material

23

Blocking in a Designed ExperimentBlocking in a Designed Experiment

E lExample

I h i l i t l fIn a chemical process experiment only four experimental trials can be made from a single batch. In this experiment we are studying the effects of fourIn this experiment we are studying the effects of four different binder types on properties of the final product. Four batches of materials were made.

24


Batch 1 Batch 2 Batch 3 Batch 4

B1 = 89.5B3 = 93.0

B1 = 88.9B2 = 93.7

B4 = 97.3B3 = 96.9

B1 = 96.7B3 = 101.1B3 93.0

B2 = 89.1B4 = 96.4

B2 93.7B3 = 93.7B4 = 97.4

B3 96.9B1 = 95.1B2 = 95.3

B3 101.1B2 = 97.2B4 = 100.9

Mean = 92.0 Mean = 93.4 Mean = 96.2 Mean = 99.0

25

Blocking in a Designed Experimentg g p

100

105

Binder 1

90

95

Yie

ld

Binder 1Binder 2Binder 3

80

85

90Y Binder 3Binder 4

80Batch 1 Batch 2 Batch 3 Batch 4

Excel output26

Blocking in a Designed Experimentg g p

File → Open 1-1Binder.JMPAnalyze → Fit Y by XAnalyze → Fit Y by X▼Red Triangle → Quantiles▼Red Triangle → Means/ANOVA

27Not SignificantMSE = 11.3


Accounting for the variability in the batches in the statisticalAccounting for the variability in the batches in the statistical analysis

Analyze → Fit Model

Significant

MSE = 2.3

Notice the great reduction in the estimated experimental (MSE) h b t h i d bl ki ff terror (MSE) when batch is used as a blocking effect

Sequential ExperimentationSequential Experimentation

• Experimentation is an iterative processExperimentation is an iterative process– It is often best to do a smaller experiment

initially, and then do additional experiments based on the initial results

• Sir Ronald Fisher once said “the best time to design an experiment is after you’ve done it.”you e do e t

• “It is best not to plan a large ‘all‐encompassing’ experiment at the outset because this is the time when you knowbecause this is the time when you know least about the system.” G.E.P. Box

Sequential ExperimentationSequential Experimentation

Purpose

Screening DesignNarrow down a list of many factors to the

most important onesmost important ones

Understand relationshipsFactorial Design

Understand relationships and interactions between

factors and response

Response Optimize response by interpolating factor Surface Designinterpolating factor

settings30

Advantages of DOEAdvantages of DOE

• More efficient than one‐factor‐at‐a‐time experimento e e c e a o e ac o a a e e pe e

• Avoids misleading conclusions that can occur when interactions are present

• Factors are not confounded with one another– Confounding occurs when you cannot distinguish which factor is

ll i ti threally impacting the response

• Can create a mathematical model useful for prediction

• Yields conclusions that are valid over a range of• Yields conclusions that are valid over a range of experimental conditions– Interpolation is valid (with continuous factors)

31






• Resources

32

Introduction to Factorial DesignsIntroduction to Factorial Designs

• In a standard factorial design, at least one run is made at each of the possible factor combinations

The number of runs required can be quite large!– The number of runs required can be quite large!• 2 factors with 3 levels each: 32=9 factor combinations• 3 factors with 3 levels each: 33=27 factor combinations• 4 factors with 3 levels each: 34=81 factor combinations

– Most common type are those using only 2 levels for each factor – 2k designsfor each factor 2 designs

• k is the number of factors

33

Introduction to Factorial DesignsIntroduction to Factorial Designs

• 2k factorial design is a very important special case of2 factorial design is a very important special case of a factorial design where each of the k factors of interest has only two levels – Useful at early stages of development work as a screening experiment (with not too many factors)

U f l f d t di i t ti b t f t– Useful for understanding interaction between factors

– Can not adequately evaluate curvature in the response function (will discuss this more later)u ct o ( d scuss t s o e ate )

34

ExampleHighTablet Dissolution

Example

Relative Humidity

Example -23 Factorial Design

High

Low

Pan Speed

Inlet Air Humidity

Low High Low

Pan Speed

Study how Pan Speed, Inlet Air Humidity and Relative H idit ff t T bl t Di l tiHumidity affect Tablet Dissolution

35

Example23 Factorial Design - The data table

Example

Run Pan Speed

Relative Humidity

Inlet Air Humidity

Tablet Dissolution

1 9 35 7 89.8

2 9 35 14 89.7

3 9 65 7 84.6

4 9 65 14 87.9

5 11 35 7 87.2

6 11 35 14 88.8

7 11 65 7 70 17 11 65 7 70.1

8 11 65 14 73.1

Create a 23 Factorial Design in JMPCreate a 2 Factorial Design in JMP

• DOE → Full Factorial Designg

Create a 23 Factorial Design in JMPCreate a 2 Factorial Design in JMP

• In this table, enter the values for Tablet ,Dissolution manually

Studying Main Effects in a 2k Factorial DesignStudying Main Effects in a 2 Factorial Design

Recall that a Main Effect is the change in the response due to the change in a factor level.

This is computed by averaging responses across all of the levels of all of the other factors.

There will be a main effect for each factor. Because they are averages, they are only useful when there is no interaction effect present.

39

Studying Main Effects in a 2k

73.187.9

Factorial Design

70.184.6

88 0Y 79 8Y =

88.889.7

88.0LowY = 79.8HighY =

Pan Speed

Low High

89.8 87.2

Pan Speed

So to calculate the main effect of Pan Speed we take the average response at the high level and g p gsubtract the average response at the low level.

The main effect of Pan Speed is –8.2 40


78 9H hY =

Factorial Design

High73.187.9

78.9HighY

70.184.6

Relative Humidity

Likewise, the main effect of Relative

Low88.889.7

of Relative Humidityis –10.0

89.8 87.2

88.9LowY =41


84 9Y =

Factorial Design

73.187.9

84.9HighY

70.184.6Likewise, the

main effect of Inlet Air

88.889.7

of Inlet Air Humidity

is 2High

89.8 87.2 Inlet Air HumidityLow

g

82.9LowY =42

Studying Interaction Effects in a 2k Factorial DesignStudying Interaction Effects in a 2 Factorial Design

Interaction between factors – When the difference inInteraction between factors When the difference in the response between the low level and high level of a factor is not the same at all levels of the other factors

In other words, the main effect for one factor depends pon the level of the other factor.

43

Studying Interaction Effects in a 2k

73.187.986 3Y = 716H h PSY =High

Factorial Design

70.184.6

86.3Low PSY − = 71.6High PSY −

Relative Humidity

88.889.7 Low

Pan Speed

Low High

89.8 87.288.0High PSY − =89.8Low PSY − =

Pan Speed

The average effect (or main effect) of Pan Speed At High Relative Humidity is –14.7 (71.6-86.3)

There is likely an interaction between Pan

Speed and Relative Humidity

The average effect (or main effect) of Pan Speed At Low Relative Humidity is –1.8 (88.0-89.8)

44

JMP: Analysis of DesignJMP: Analysis of Design

• We’re going to examine this interaction in JMP. To do this we need to build a model, which we will be using for the next several slidesfor the next several slides . . .

• The JMP file for the dissolution datashould have a model “built in”should have a model built in

• ▼Red Triangle (next to “Model”in the upper left panel of the data table)→ S i→ Run Script

• If this red triangle is not present,Analyze→ Fit ModelAnalyze → Fit Model

JMP: Analysis of Design

The main effects and two way Make sure the

JMP: Analysis of Design

interactions should already be listed in the Construct Model Effects section

response variable is listed

If not, make sure that

Do the Minimal Reportthe degree is “2”

Click on the 3 factors (using CTRL-click)

Reportemphasis

Click on( g )

Click Macros, then click Factorial to degree

Click on Run

JMP: Interaction PlotJMP: Interaction Plot

Now we can get the plot!Now we can get the plot!▼Red Triangle (next to “Response Tablet

Dissolution”) → Factor Profiling → Interaction PlotsIf the lines are (nearly) parallel,

then there is no interaction.

) g

ConclusionsThere appears to be an interaction between Pan Speed and Relative Humidity. Other interactions are not very strong or non-existent.

Al th i t bi i ff tAlso: there is not a big main effect for Inlet Air Humidity.

47

Interaction Plot ExamplesInteraction Plot ‐ Examples

90No interaction

80

85

90

1

75

80 -1

1 0 1

Strong interaction

Weak interaction

-1 0 1

8590 -1

85

90

707580

175

80

85

-111

-1

-1 0 175

-1 0 1

1

Analysis of Factorial DesignsAnalysis of Factorial Designs

• A first order regression model with interactions is often• A first‐order regression model with interactions is often used in the case of a 2k factorial design

• Model is used with “coded variables” (‐1 to 1)• β0 is defined as the intercept and is the grand average of all

the data• β is defined as the regression coefficient for the ith factor• βi is defined as the regression coefficient for the i factor

0

k k

i i ij i jy x x xβ β β= + +∑ ∑ ∑01 2

i i ij i ji i j

y β β β= < =∑ ∑ ∑

M i TMain Effects

Two-wayInteractions

Analysis of Factorial DesignsAnalysis of Factorial Designs

• Analysis of Variance is used in order to determine if ycollectively the regression coefficients are statistically significant

t t ti ti d t d t i if th i di id l ffi i t• t‐statistics are used to determine if the individual coefficient has a statistically significant effect on the response variable

Hypothesis for testing individualregression coefficients

The test statistic for the hypothesis

0

1

: 0: 0

i

i

HH

ββ

=≠

0

ˆˆ( )i

i

tseββ

=1 iβ

50


Act al b Predicted PlotResponse Tablet Dissolution

Scroll back up in the Fit Model

80

85

90

Tabl

etso

lutio

n A

ctua

l

Actual by Predicted Plot Scroll back up in the Fit Model output window (where we made

the interaction plots)

70

75

Dis

s

70 75 80 85 90Tablet Dissolution Predicted

P=0.0641 RSq=1.00 RMSE=0.7071

Analysis of Variance

If this is not significant, it is not RSquareRSquare AdjRoot Mean Square ErrorMean of ResponseObservations (or Sum Wgts)

0.9988290.9918020.707107

83.98

Summary of Fit likely that the individualt-tests will be significant

Observations (or Sum Wgts) 8

ModelError

Source61

DF426.42000

0 50000

Sum ofSquares

71.07000 5000

Mean Square142.1400

F Ratio

Prob > F

Analysis of Variance Sometimes it is reasonable to use a higher alpha level (e.g., 0.10) while in early stages of Error

C. Total17

0.50000426.92000

0.50000.0641

Prob > F ) y gexperimentation

51


InterceptPan Speed(9 11)

Term83.9

4 1

Estimate0.250 25

Std Error335.60

16 40

t Ratio0.0019*0 0388*

Prob>|t|

Parameter Estimates

Pan Speed(9,11)Relative Humidity(35,65)Inlet Air Humidity(7,14)Pan Speed*Relative Humidity

-4.1-4.9750.975

-3.225

0.250.250.250.25

-16.40-19.90

3.90-12.90

0.0388*0.0320*0.15980.0493*

A statistical test on whether or not βi is = 0 is the same

Pan Speed*Inlet Air HumidityRelative Humidity*Inlet Air Humidity

0.1750.6

0.250.25

0.702.40

0.61120.2513

A statistical test on whether or not βi is 0 is the same thing as testing whether or not the ith factor is significant.

These parameter estimates will only be in coded form if you’ve

Caution:p y y

created the design in JMP and entered in the response data.If you bring in the data from Excel and analyze it, be sure to

enter the factors in coded variables. 52

JMP: Removing Insignificant Model TermsJMP: Removing Insignificant Model Terms

Go back to the “Fit Model” dialog.

Select the two non-Select the two nonsignificant interactions and click on Remove.

Then click on Run again.

53

JMP: Removing Insignificant Model TermsJMP: Removing Insignificant Model Terms

Here is our final model:

RSquareRSquare Adj

0.9915090 980188

Summary of Fit

Response Tablet Dissolution

Measures the model’s capability to fit the present data. R t th ti f i ti iRSquare Adj

Root Mean Square ErrorMean of ResponseObservations (or Sum Wgts)

0.9801881.099242

83.9 8

Analysis of Variance

Represents the proportion of variation in the response data that is explained by the model

ModelErrorC. Total

Source 4 3 7

DF 423.29500 3.62500

426.92000

Sum of Squares 105.824 1.208

Mean Square 87.5783

F Ratio

0.0019Prob > F

y

All the effects are very significant except for the

main effect of Inlet Air

InterceptPand Speed(9,11)

Term 83.9 -4.1

Estimate0.3886410.388641

Std Error215.88-10.55

t Ratio<.00010.0018

Prob>|t|

Parameter Estimatesmain effect of Inlet Air

Humidity, which is marginally significant

(could remove this effect)Relative Humidity(35,65)Inlet Air Humidity(7,14)Pand Speed*Relative Humidity

-4.975 0.975 -3.225

0.3886410.3886410.388641

-12.80 2.51 -8.30

0.00100.08700.0037

( )

54

Model BuildingModel Building

• Model building is an iterative procedureg p– Goal is to get an accurate, parsimonious representation of reality

• Start the analysis by including all the factors and the interactions between them

l h ff h h f• Eliminate those effects which are not significant– Eliminate non‐significant interactions first

Th li i t i ifi t i ff t if th– Then eliminate non‐significant main effects if they are not included in any significant interactions

“All models are wrong

55

All models are wrong, some are useful.”

– George Box

Model Checking in Factorial DesignsModel Checking in Factorial Designs

• Evaluating Model Adequacy is an important part of the g q y p pregression analysis– It is always important to examine the fitted model to ensure

it provides an adequate approximation to the true systemit provides an adequate approximation to the true system

– We are essentially verifying the assumptions that allow us to do regression

• Normality of the response

• Equal variance of the response

Proceeding with exploration and optimization of the fitted– Proceeding with exploration and optimization of the fitted response surface will likely give poor or misleading results unless the model is an adequate fit

56


• The residuals play an important role inˆ( )y y−The residuals, play an important role in judging model adequacy– Normal quantile plot (also known as the normal probability

( )i iy y

plot) of the residuals checks the normality assumptions• If the residuals plot approximately along a straight line, then the normality assumption is satisfied

– Plot of residuals vs. the predicted response, • If the residuals are scattered randomly on the plot, then it can be assumed that the variance is constant for all values of y. y

57

JMP: Checking for Equal Variance with a Plot f h d l

▼Red Triangle (upper left of Fit Model results window)

of the Residuals

g→ Row Diagnostics → Plot Residual by Predicted

Scroll down to see plot

The residuals havei il d th 0 5

1.0

1.5

Res

idua

l

Residual by Predicted Plot

Response Tablet dissolution

similar spread across thex-axis (but it is hard to besure for the small x values

1 5

-1.0

-0.5

0.0

0.5

Tabl

et d

isso

lutio

n R

since there isn’t a lot of data)-1.5

70 75 80 85 90 95Tablet dissolution Predicted

58


Any pattern in a residuals plot (against run order) suggests

1

2

3

interdependence among the runs

-3

-2

-1

0

1

Res

idua

ls

Individual Measurement of Column 7

Control Chart

Megaphone shape –increasing variability

↓

Stable – no problems →

-45 10 15 20 25 30 35 40 45 50

Sample

-10

-5

0

5

10

15

Res

idua

l

1 0

1.5

2.0

5 10 15 20 25 30 35 40 45 50Sample

-1.0

-0.5

0.0

0.5

1.0

Res

idua

ls ← Curvature oftenindicates “modelmisspecification”

-1.55 10 15 20 25 30 35 40 45 50

Sample

These plots can also be good for detecting outliers

JMP: Checking for Normality of the Residuals

▼Red Triangle (in Fit Model results window)

JMP: Checking for Normality of the Residuals

g→ Save Columns → Residuals

Analyze → Distribution 1.5.01 .05.10 .25 .50 .75 .90.95 .99

Residual Tablet dissolution

Distributions

y* Select “Residual Tablet

Dissolution” for “Y, Columns”* ▼Red Triangle (next to “Residual 0

0.5

1

.01 .05.10 .25 .50 .75 .90.95 .99

▼Red Triangle (next to ResidualTablet Dissolution”)→ Normal Quantile Plot -1

-0.5

0

The normality assumption is satisfiedsince the black points are near thediagonal line

-1.5

-3 -2 -1 0 1 2 3

Normal Quantile Plot

diagonal line.

60

Predictions in JMP

JMP lets you save something called a Prediction Formula

Predictions in JMP

▼Red Triangle (in Fit Model results window)→ Save Columns → Prediction Formula

Return to the data table (leave results window open!) and right click on the column header for “Pred Formula Tablet Dissolution”; choose “Formula” from the menuDissolution ; choose Formula from the menu

61

Predictions in JMPPredictions in JMP

The mathematicalThe mathematical model in terms of

the original variables is shown in the bottom of the

dialog box.dialog box.

If desired, it can be copied/pasted intocopied/pasted into other programs.

62

Optimization in JMP

Go back again to the Fit Model output window

Optimization in JMP

▼Red Triangle → Factor Profiling → Profiler

Scroll down to see the profilerScroll down to see the profiler

▼Red Triangle(next to theProfiler)→ Desirability

Functions

▼Red Triangle(next to theProfiler)→ Maximize Desirability

63

Optimization in JMPOptimization in JMP

80

85

90

Tabl

etso

luti

on0.

725

6563

3

Prediction Profiler

To maximize Tablet

Di l ti P70

75T

Dis

s 90?.

760.

751

abili

ty38

23Dissolution, Pan

Speed and Relative

Humidity should0

0.25

Des

ira

0.99

9

9.5 10

10.5 11 35 40 45 50 55 60 65

35

7 9 11 13

14

0

0.25 0.5

0.75 1

Humidity should be at their low levels and Inlet Air Humidity at9

Pan Speed

Relative

Humidity

Inlet Air

Humidity Desirability

Air Humidity at its high level

The Profiler will tell you theThe predicted response and The Profiler will tell you the factor settings that meet

your objective

The predicted response and its variability is also shown

64

DOE from Beginning to EndDOE from Beginning to End

1. Set up the designp g2. Collect data and input into software (JMP)3. Fit a model with all main effects and

interactions 4. Analysis

1. Model building2. Check for model adequacy on final model3 ANOVA/Parameter estimates/Summary of Fit3. ANOVA/Parameter estimates/Summary of Fit4. Interaction plots5. Prediction Formula6. Optimization

65






• Resources

66

Section Outline ‐ Screening DesignsSection Outline Screening Designs

• Basic principles• Basic principles

• Fractional factorial designs

• Plackett – Burman designs

• Setup in JMPp

• Analysis in JMP

67

Screening DesignsScreening Designs

• A screening design is used when you are interested inA screening design is used when you are interested in determining which factors are most influential on a response with a small number of runs

• More interested in reducing number of factors than their• More interested in reducing number of factors than their interactions

• 2 major types are– Fractional factorial design– Plackett‐Burman design

68


• Confounding (also known as aliasing)– A confounding design is one where 2 or more treatment (main and/or interaction) effects are estimated by the same linear combination of the experimental observations

– Exists when a change in the response is due to multiple effects

– It is not possible to determine which effect is really causing h hthe change

– Can occur accidentally in poorly designed experiments– Done deliberately and systematically in screening designs– In general, we confound main effects and 2 way interactions with higher order interactions

– Most software generated designs will give you the g g g yminimum amount of confounding possible

69

Fractional Factorial DesignsFractional Factorial Designs

Oft b t f ll b f f t• Often best for a smaller number of factors– Requires 4, 8, 16, 32, 64, etc. runs

– Uses a fraction of the runs from a full factorial design– Uses a fraction of the runs from a full factorial design

• Notation ‐ 2k‐p

– k is the number of factors consideredk is the number of factors considered

– p is the level of fractionation

• 23‐1, 24‐1, 25‐1, etc. are half fraction designs

• 24‐2, 25‐2, 26‐2, etc. are quarter fraction designs, , , q g

70


Resolution 3Resolution 4Resolution 5

Design Resolution Table for Fractional Factorial Designs

2 3 4 5 6 7 8 9 10 11 12 13 14 154 Full 1/2

Number of Factors

8 Full 1/2 1/4 1/8 1/1616 Full 1/2 1/4 1/8 1/16 1/32 1/64 1/128 1/256 1/512 1/1024 1/204832 Full 1/2 1/4 1/8 1/16 1/32 1/64 1/128 1/256 1/512 1/102464 Full 1/2 1/4 1/8 1/16 1/32 1/64 1/128 1/256 1/51264 Full 1/2 1/4 1/8 1/16 1/32 1/64 1/128 1/256 1/512128 Full 1/2 1/4 1/8 1/16 1/32 1/64 1/128 1/256256 Full 1/2 1/4 1/8 1/16 1/32 1/64 1/128512 Full 1/2 1/4 1/8 1/16 1/32 1/64

71


-,+,+ +,+,+Example – 23-1

Runs not performed -,+,- +,+,-

p

ctor

CFa

+,-,+

Factor B+,-,--,-,-Runs performed

Factor A

Fractional Factorial Design ExampleFractional Factorial Design Example

• Resistivity of wafery– Five factors in a manufacturing process for an integrated circuit were investigated in a 25‐1 design with the objective f l i h th f t ff t th i ti it f thof learning how these factors affect the resistivity of the

wafer

– 5 factors – each with 2 levels• implant dose, temperature, time, oxide thickness, furnace position

– 25‐1 fractional factorial design – Resolution 5 design2 fractional factorial design Resolution 5 design• 16 runs were done

– Every main effect is confounded with a four‐factor interaction

Plackett‐Burman DesignsPlackett Burman Designs

• Most useful when you havemany factors to study• Most useful when you have many factors to study– Only exist for set number of runs – such as 12, 20, 24, 28, 36, 40, 44, 48, etc.

– Any multiple of 4 that is not a power of 2• Don’t exist for 4, 8, 16, 32, 64, etc.

– Only for 2 levels per factorOnly for 2 levels per factor

• Can only study main effects– Resolution 3 designsg– Some projection properties to a small number of significant factors

74

Plackett‐Burman DesignsPlackett‐Burman Designs

• Example – 10 factors could be studied withExample 10 factors could be studied with a 12 run Plackett‐Burman design

Setup of Screening Designs in JMPSetup of Screening Designs in JMP

Using JMP to setup a screening design will g p g ggreatly simplify the analysis

DOE → Screening DesignThe default is to include a response called “Y”, if you want to change it

dd l i l li k hor add multiple responses click here

Add the appropriate number of Continuous or Categorical factorsContinuous or Categorical factors

You can fix mistakes by clicking on Remove Selected

76


Double click on the Name or Values to change the factorchange the factor name or level –These are the values that will appear in your

output and on all graphs

When you are ready, click Continue

77


Possible designs and their effects

that can bethat can be estimated are

shown

Pick the appropriate design

Scroll down to the bottom and click

Continue

78


Make sure that youTo see what effects are confounded with each

other check here but don’t change the Make sure that you randomize the runs

other check here – but don t change the generating rules or aliasing

Don’t add replicates

When you areWhen you are ready to generate the design click

Make Table

79

Analysis of Screening DesignsAnalysis of Screening Designs

• There are two methods for analyzing screening designs

h l h d ( h (h lf ) l l )• Graphical methods (such as (half‐) normal plot)– This is the only option whennumber of runs ≤ (number of effects + 1)u be o u s ( u be o e ects )

• Standard analysis using the parameter estimates, significance testing and model building– This is possible only whennumber of runs > (number of effects + 1)

80

Analysis of Screening Designs in JMPAnalysis of Screening Designs in JMP

• Standard procedure for analyzing a screening design is:1. Setup the designp g

2. Enter the response data

3. Run scriptp

4. Analysis1. Model buildingg

– Half normal plot

– ANOVA

2 F ll i t2. Follow up experiments

81


• Resistivity example

• DOE →

Screening DesignScreening Design

82



• Enter data for Resistivity

83



JMP file should have a model “built in”

• ▼Red Triangle (next to “Model”• ▼Red Triangle (next to Modelin the upper left panel of the data table)→ Run Script→ Run Script

• If this red triangle is not present,Analyze→ Fit Model andAnalyze → Fit Model and

add all main and 2‐way effects

R th d l• Run the model84


Step 1 – Model Building

(optional) Click to change the default normal plot to a half normal plot

▼Red Triangle → Effect Screening → Normal Plot

(optional) Click to change the default normal plot to a half normal plot

Signif. effects: dose, temperature, time, dose*temperature 85


Final Model

86

Analysis of Screening DesignsAnalysis of Screening Designs

A l i th d i th f t i l• Analysis can then proceed as in the factorial designs case on the most significant terms

• Follow up experiments– Now that the most important factors have been identified, additional experimentation can be done to better understand interactions and possible curvatureand possible curvature

87






• Resources

88

Section Outline –Response Surface Designs

• TerminologyTerminology• Types

– Central Composite Designp g– Box‐Behnken Design

• Setup• Graphical Methods

– Contour PlotR S f Pl t– Response Surface Plot

• Analysis

89

Response Surface DesignsResponse Surface Designs

• Appropriate when you have continuous factors– And you want to be able to interpolate in order to optimize the response

• Best for a smaller number of factors• Best for a smaller number of factors– Most economical for fewer than 6 factors

• Useful when there is curvatureUseful when there is curvature– A non‐linear relationship between the factors and response

l f l d h d• Also RSD are often implemented when prediction is of utmost importance

90


• Center points• Center points– Points at the center of the design region (all the factors at the middle of the high and low level settings)

– Often replicated to get estimates of experimental error

– Used in factorial designs to check for curvature (will ll f d ff )not allow estimation of quadratic effects)

– More often used in response surface designs in conjunction with axial points tomodel curvatureconjunction with axial points to model curvature

91


• Axial points• Axial points– Points along the edges or outside of the design region formed by factorial pointsregion formed by factorial points

– Required if curvature is to be modeled

92

Example ‐ Response Surface iDesigns – 3 Factors

Red points are theBl i t Red points are the factorial points

Blue points are the axial points

G i tGreen point is the center point(s)

93


• 2 major types of Response Surface Designs• 2 major types of Response Surface Designs (RSD)

C t l C it D i (CCD)– Central Composite Designs (CCD)

– Box Behnken Designs (BBD)

94

Central Composite DesignsCentral Composite Designs• A combination of factorial, axial, & center points

• Axial points can be “face centered” – using only combinations of high, low, and middle levels

• Or they can be “outside” – using more extreme factor levels

95

Box‐Behnken DesignBox Behnken Design• For three factors, the points are on the edges of the cube

• More extreme factor combinations are not performed (the corner points)

• Cannot be done sequentiallyCannot be done sequentially

• Appealing when extreme conditions (corners) are not runnable or of interest

96

Setup of RSDSetup of RSD

The set up of a RSD is similar to that of Factorial pand Screening Designs

DOE → Response Surface DesignThe default includes a response called “Y” – go here to change it

dd l i lor add multiple responses

Add the appropriate number of Continuous factors (minimum 2)Continuous factors (minimum 2)

You can change factor namesand factor levels as needed

Click “Continue” to proceed 97

Setup of RSDSetup of RSD

Select the appropriate

design

Then click on Continue

98

Setup of RSDSetup of RSDYou can change the location of

the axial pointsMake sure

that you randomize

the axial points

the runs

Specify the # of additionaladditional

replicates and center points

When you are ready to make the design table

click Make Table

99

Analysis of RSDAnalysis of RSD

• When the curvature in the true response surface is strong enough the first‐order model with interactions is inadequateinteractions is inadequate

• A second‐order model (shown below) will likely be required in these situationsbe required in these situations

20

k k k

i i ij i j ii iy x x x xβ β β β= + + +∑ ∑ ∑ ∑01 2 1

i i ij i j ii ii i j i= < = =∑ ∑ ∑ ∑

LinearEffects

Two-way Interactions

QuadraticEffectsEffects Interactions Effects


M d l b ildi i RSD i th it i• Model building in RSD is the same as it was in factorial designs

• First eliminate non significant quadratic effects• First eliminate non‐significant quadratic effects and interactions before eliminating main effects

• Preserve model hierarchy• Preserve model hierarchy– If a factor is involved in an interaction or quadratic effect, the main effect should be included

101


• Standard procedure for analyzing a RSD is:p y g1. Setup the design2. Enter the response data3. Include all main effects, 2 way interactions and

quadratic terms 4. Analysisy

1. Model building2. Check for model adequacy on final model3. ANOVA/Parameter estimates/Summary of Fit3. ANOVA/Parameter estimates/Summary of Fit4. Graphical Methods

» Interaction plots» Optimizationp» Contour Plots & Response Surface Plots

5. Conclusions102

Example Breadwrapper ExperimentExample – Breadwrapper Experiment

• An experiment was conducted to study theAn experiment was conducted to study the strengh of breadwrapper stock in grams per square

– 3 factors: sealing temperature, cooling temperature, percent additive

A CCD d ith t i t– A CCD was used with one center point

– Objective is to maximize the breadwrapper strengh

Example – Breadwrapper Experiment

• We setup the design and run the experiment just as


we did for factorial and screening designs


• Manually enter the values for Strength

Example – Breadwrapper Experiment Example – Breadwrapper ExperimentExample – Breadwrapper Experiment

• This JMP file should have a model “built in”

• ▼Red Triangle (next to▼Red Triangle (next to“Model” in the upper left panelof the data table) → Run Script)→ p

• If this red triangle isn’t there,use Analyze→ Fit Modeluse Analyze → Fit Model

106

JMP: Analysis of DesignJMP: Analysis of Design1. The main effects, two way interactions, and quadratic terms

3. Make sure the response

should already be listed in the construct model effects section

response variable is listed

4. Do the Minimal Report2. If not, click on Report

emphasis

5 Click on

the 3 factors (using CTRL-click)

Click Macros, 5. Click on Run

,then click

Response Surface

Analysis of RSD


Analysis of RSD

Use output and pprocess knowledge to help determine

which non-significant c o s g caeffects to eliminate

from the model

Fi tFirst, removenon-significant

interactions and quadratic terms

108

Analysis of RSD


Analysis of RSD

Here is the analysis after eliminating the insignificant terms

109

Analysis of RSD

Step 2 – Model Checking (Equal Variance)

Analysis of RSD

▼Red Triangle (upper left of Fit Model results window)▼Red Triangle (upper left of Fit Model results window)→ Row Diagnostics → Plot Residual by Predicted

Scroll down to see plotp

The residuals have similarspread across x-axis

110


Step 2 – Model Checking(Normality)

▼Red Triangle (in Fit Modellt i d )results window)

→ Save Columns→ Residuals

Analyze → Distribution* Select “Residual Strength” for

“Y C l ”“Y, Columns”* ▼Red Triangle (“Residual Strength”) → Normal Quantile Plot

The normality assumption is satisfied since the black points areThe normality assumption is satisfied since the black points are near the diagonal line.

111

Analysis of RSD

Step 3 – ANOVA/Parameter Estimates/Summary of Fit

Analysis of RSD

RSquare Summary of Fit

Final Parameter Estimates

112

Analysis of RSD

Step 4 – Graphical Methods

Analysis of RSD

Graphical methods become even more crucial in RSD because we want to understand better where the optimum is as well as how the response changes.

There are many that we can use: interaction plots, t l t d f l tcontour plots, and response surface plots

Step 4 Graphical Methods (Interaction Plot)Step 4 – Graphical Methods (Interaction Plot)

▼Red Triangle (upper left of Fit Model results window)→ Factor Profiling → Interaction Plot

113

→ Factor Profiling → Interaction Plot


Step 4 – Graphical Methods (Optimization/Profiler)

▼Red Triangle (upper left of Fit Model results window)→ Factor Profiling → Profiler

If necessary, scroll down to the Prediction Profiler

▼Red Triangle (next to Prediction Profiler)g ( )→ Desirability Functions

▼Red Triangle (next to Prediction Profiler)→ Maximize Desirability

114


Step 4 – Graphical Methods (Optimization/Profiler)

To maximize the t th listrength, sealing temp should be

set at its low l l lilevel, cooling temp at mid level, and

percent additive at high level

The Profiler will tell you theThe predicted response and The Profiler will tell you the factor settings that meet

your objective

The predicted response and its variability is also shown

115

Analysis of RSD

Step 4 – Graphical Methods (Contour Plot)

Analysis of RSD

▼Red Triangle (upper left of Fit Model results window)→ Factor Profiling → Contour Profiler

Scroll to the Contour Profiler The predicted response for the specified factor

levels is herelevels is here

The factor levels can be changed

Or if you want to enter a value for the response and see all the factor

levels that will achieve that response, do it here

116



To add contours to the Contour Plot:

▼Red Triangle (next to Contour Profiler) → Contour Grid▼Red Triangle (next to Contour Profiler) → Contour Grid

You then have to specify the contours. Use the

defaults for now.

117

Analysis of RSD


Analysis of RSD

The lines on the plot represent the predicted values of the

response for different valuesresponse for different values of the three factors

118

Analysis of RSD

Step 4 – Graphical Methods (Contour Plot & Response Surface Plot)

Analysis of RSD

To find the region of operability using a given

range of the response, enterrange of the response, enter in values for the Lo Limit

and/or Hi Limit

Low = 9Low = 9

The white area represents factor settings that are within

the specified limits

119



▼Red Triangle (upper left of Fit Model results window)→ Factor Profiling → Surface Profiler

Scroll to the surface profiler, which will be labeled “Drop”

Choose which variables to plotby clicking on buttons in this area

Response Surface Plot(Can be rotated by clicking

and dragging) 120

Analysis of RSD

Step 5 – Conclusions

Analysis of RSD

p

From all the graphical methods determine the best factor levels to meet your objectivey jBe aware the variability in the predicted response can be large

121






• Resources

122

ResourcesResources

• Help menue p e u– Indexes– Tutorials– Books

• JMP documentations

S l D t– Sample Data

123

ResourcesResources

• On‐line resourcesO e esou ces– http://www.jmp.com/about/events/webcasts/for webcasts and recorded demos

– http://www.jmp.com/academic/check out Learning Library

8 Q i k G id• JMP 8 Quick Guide

124

ResourcesResources

• On‐line resourcesO e esou ces– http://www.lisa.stat.vt.edu/Welcome to LISA!

– http://www.lisa.stat.vt.edu/?q=short_coursesLISA short courses

125

Thank YouThank You

126

jmp demo slides

Documents

response data

insignificant model

fit model dialog

prob f

r t th ti f i ti

insignificant model

jmpfinal model signif

fit model output window