common design problems 1.masking factor effects 2.uncontrolled factors 3.one-factor-at-a-time...

Common Design ProblemsCommon Design Problems

1.1. Masking factor effectsMasking factor effects

2.2. Uncontrolled factorsUncontrolled factors

3.3. One-factor-at-a-time testingOne-factor-at-a-time testing

Masking factor effectsMasking factor effects

if variation in test results is on the if variation in test results is on the same order of magnitude as the same order of magnitude as the factor effects, the latter may go factor effects, the latter may go undetected. undetected. This can be addressed through This can be addressed through

appropriate choice of sample size.appropriate choice of sample size. unmeasured covariates (e.g., the unmeasured covariates (e.g., the

effect of time passing, as an effect of time passing, as an instrument degrades) can also lead to instrument degrades) can also lead to variation that masks factor effectsvariation that masks factor effects

Uncontrolled factorsUncontrolled factors

it’s pretty obvious that all variables of it’s pretty obvious that all variables of interest should be included as factorsinterest should be included as factors

sometimes, though, it can be tricky to sometimes, though, it can be tricky to choose the appropriate level of model choose the appropriate level of model granularity. granularity. Sometimes a “high-level feature” (i.e., Sometimes a “high-level feature” (i.e.,

some function of the levels of many some function of the levels of many factors) could be chosen in place of the factors) could be chosen in place of the many factors that influence itmany factors that influence it

this can make it difficult to vary the factor this can make it difficult to vary the factor appropriately, and can limit analysis of appropriately, and can limit analysis of the experimental results.the experimental results.

it can be tempting to vary each factor it can be tempting to vary each factor value independently, holding the others value independently, holding the others constant. However, this does not constant. However, this does not explore the factor space very explore the factor space very effectively. effectively. It would be a terrible local search strategy! It would be a terrible local search strategy!

Looking at the same point in another Looking at the same point in another way, it neglects the possibility that way, it neglects the possibility that interactions between factors could be interactions between factors could be importantimportant

One-factor-at-a-time testingOne-factor-at-a-time testing

Factorial DesignFactorial Design

DefinitionDefinition A Study design in which responses are A Study design in which responses are measured at different combinations of level measured at different combinations of level

of one or more experimental factorsof one or more experimental factors A study design in which treatment consists of A study design in which treatment consists of

two or more factors or independent two or more factors or independent variables A Study in which ll combinations of levels of

two or more independent variables (factors) are measured

A Study design when the combined effects A Study design when the combined effects of two or more factors are investigated of two or more factors are investigated

concurrentlyconcurrently Two or more ANOVA factors are combined in Two or more ANOVA factors are combined in

a single studya single study

FactorsFactors A variable upon which the experimenter A variable upon which the experimenter

believes that one or more response believes that one or more response variables may depend, and which the variables may depend, and which the experimenter can controlexperimenter can control the design of the experiment will largely the design of the experiment will largely

consist of a policy for determining how to consist of a policy for determining how to set the factors in each experimental trialset the factors in each experimental trial

They are denoted as capital lettersThey are denoted as capital letters

Possible values of a factor are called Possible values of a factor are called levelslevels.. in other literature, the word “version” is in other literature, the word “version” is

used for a qualitative (not quantitative) used for a qualitative (not quantitative) levellevel

They are denoted as lowercase levelThey are denoted as lowercase level

TreatmentsTreatments

They represent a particular They represent a particular combination of factors levelcombination of factors level

They are denoted by the same They are denoted by the same combination of the lowercase letters combination of the lowercase letters

that represent the corresponding that represent the corresponding levelslevels

With 3 factors A, B, C, the treatment With 3 factors A, B, C, the treatment corresponding to the combination of corresponding to the combination of

level alevel a11 of A, b of A, b33 of B, andof B, and CC2 2 of C is denoted by aof C is denoted by a11 bb33 c c22

Complete Factorial ExperimentComplete Factorial Experiment

An experiment in which responses are measured An experiment in which responses are measured at all combination of levels of the factorsat all combination of levels of the factors

A complete factorial experiment in which there are a levels of A, b levels of factor B and so on is called a x b x

… factorial experiment

Total number of treatments in an a x b x c … complete factorial

experiment is t = abc…

The Use of Factorial The Use of Factorial DesignDesign

Identify factors with significant effects Identify factors with significant effects on the responseon the response

Identify interactions among factorsIdentify interactions among factors Identify which factors have the most Identify which factors have the most

important effects on the responseimportant effects on the response Decide whether further investigation of Decide whether further investigation of

a factor’s effect is justifieda factor’s effect is justified Investigate the functional dependence Investigate the functional dependence

of a response on multiple factors of a response on multiple factors simultaneously (if and only if you test simultaneously (if and only if you test

many levels of each factor)many levels of each factor)

Advantages Advantages of Factorial Designsof Factorial Designs

1.1. Saves Time & Efforte.g., Could Use Separate Completely

Randomized Designs for Each Variable

2. Controls Confounding Effects by Putting Other Variables into Model

3. Can Explore Interaction Between Variables

ANOVA ANOVA Null HypothesesNull Hypotheses

1.1. No Difference in Means Due to No Difference in Means Due to Factor AFactor AHH00: : 11.... = = 22.... =... = =... = aa....

2.2. No Difference in Means Due to No Difference in Means Due to Factor BFactor BHH00: : ..11.. = = ..22.. =... = =... = ..bb..

3.3. No Interaction of Factors A & BNo Interaction of Factors A & BHH00: AB: ABijij = 0 = 0

Total Variation

Total Variation

ANOVA ANOVA Total Variation Total Variation PartitioningPartitioning

Variation Due to Treatment

Variation Due to Treatment

Variation Due to Random Sampling

Variation Due to Random Sampling

Variation Due to Interaction

Variation Due to Interaction

SSESSESS (Treatment)SS (Treatment)

SS(AB)

SS(Total)SS(Total)

Variation Due to

Factor B

Variation Due to

Factor BSSB

Variation Due to

Factor A

Variation Due to

Factor ASSA

Factorial Patterns

Simple EffectSimple Effect the effect of one factor on only one the effect of one factor on only one

level of another factorlevel of another factor Effect of changing the level of one Effect of changing the level of one

factor while holding the level of the factor while holding the level of the other factor fixedother factor fixed

If the simple effects differ, there is an interaction

Factorial Patterns

Main EffectMain Effect Effect of variation in a single variable, Effect of variation in a single variable,

averaged across all levels of all other averaged across all levels of all other variablesvariables

An outcome that is a consistent An outcome that is a consistent difference between levels of a factor. difference between levels of a factor.

Main Effect of One variable, no Main Effect of One variable, no effect of the othereffect of the other

Main Effect of All VariablesMain Effect of All Variables

Factorial Patterns

InteractionInteraction Effect of variation in a single variable

depends on the specific levels of at least one other variable

It exists when differences on one factor depend on the level you are on another factor

The effects of one factor change depending on the level of another factor

The effect of one factor depends on the level The effect of one factor depends on the level of the other factorof the other factor

Interaction effect

How do we know if there is an How do we know if there is an interaction in a factorial design?interaction in a factorial design? Statistical analysis will report all main Statistical analysis will report all main

effects and interactions.effects and interactions. If you can not talk about effect on one If you can not talk about effect on one

factor without mentioning the other factor without mentioning the other factorfactor

Spot an interaction in the graphs – Spot an interaction in the graphs – whenever there are lines that are not whenever there are lines that are not parallel there is an interaction present!parallel there is an interaction present!

Interaction Effect

1.1. Occurs When Effects of One Factor Occurs When Effects of One Factor Vary According to Levels of Other FactorVary According to Levels of Other Factor

2.2. When Significant, Interpretation of When Significant, Interpretation of Main Effects (A & B) Is ComplicatedMain Effects (A & B) Is Complicated

3.3. Can Be DetectedCan Be DetectedIn Data Table, Pattern of Cell Means in One In Data Table, Pattern of Cell Means in One

Row Differs From Another RowRow Differs From Another Row

In Graph of Cell Means, Lines CrossIn Graph of Cell Means, Lines Cross

Interaction Effect

Diverging trends, from little Diverging trends, from little difference to larger differencedifference to larger difference

Converging trends, from large Converging trends, from large difference to smaller difference.difference to smaller difference.

Cross-over interactions, with Cross-over interactions, with decrease in one variable while the decrease in one variable while the other stays constant or increases.other stays constant or increases.

Interaction effect:Interaction effect:difference in magnitude of response

Interaction effect:Interaction effect:difference in direction of response

Graphs of InteractionGraphs of Interaction

Effects of Motivation (High or Low) & Training Method (A, B, C) on Mean

Learning TimeInteraction No Interaction

AverageAverageResponseResponse

AA BB CC

HighHigh

LowLow

AverageAverageResponseResponse

AA BB CC

HighHigh

LowLow

Assumption of Factorial Assumption of Factorial DesignDesign

Interval/ratio data Normal distribution or N at least 30

Independent observations Homogeneity of variance

Proportional or equal cell sizes

Strategy for factorial analysis

1. A test is performed to see if there is an interaction between the factors

2. If statistically significant interaction is indicated, the simple effect of the factors are examined

separately3. If there is no demonstrable interaction, then inferences are

made about each of the main effect

Test first the higher order interactions. If an interaction is present there is no

need to test lower order interactions or main effects involving those factors. All

factors in the interaction affect the response and they interact

The testing continues with for lower order interactions and main effects for

factors which have not yet been determined to affect the response.

Statistical test infactorial experiment

2-way ANOVA

Example: Study aids for examExample: Study aids for exam A: workbook or notA: workbook or not B: 1 cup of coffee or notB: 1 cup of coffee or not

Workbook (Factor A)Workbook (Factor A)

Caffeine Caffeine (Factor B)(Factor B)

NoNo YesYes

YesYes Caffeine Caffeine onlyonly

BothBoth

NoNo Neither Neither (Control)(Control)

Workbook Workbook onlyonly

Effects of Study Aids for Exams

N=30 N=30 per cellper cell

Workbook (Factor A)Workbook (Factor A) Row Row MeanMeanss

CaffeineCaffeine

(Factor (Factor B)B)

No (a1)No (a1) Yes (a2)Yes (a2)

Yes (b1)Yes (b1) CaffeeCaffee

=80=80BothBoth

=85=8582.582.5

No (b2)No (b2) ControlControl

=75=75BookBook

=80=8077.577.5

Col Col MeansMeans

77.577.5 82.582.5 8080

X X

X X

Factorial patterns Simple EffectSimple Effect

Simple effect of workbook factor at no caffeine Simple effect of workbook factor at no caffeine ((μμ[Ab1])[Ab1])

μμ[Ab1]= 85 – 80= 5[Ab1]= 85 – 80= 5Simple effect of workbook factor at with caffeine Simple effect of workbook factor at with caffeine ((μμ[Ab2])[Ab2])

μμ[Ab2]= 80 – 75= 5[Ab2]= 80 – 75= 5Simple effect of caffeine factor at no workbook (Simple effect of caffeine factor at no workbook (μμ[a1B])[a1B])

μμ[a1B]= 75 – 80= - 5[a1B]= 75 – 80= - 5Simple effect of caffeine factor at with workbook Simple effect of caffeine factor at with workbook ((μμ[a2B])[a2B])

μμ[a2B]= 80 – 85= - 5[a2B]= 80 – 85= - 5

Factorial patterns

Main Effect Main Effect The average of two simple effects The average of two simple effects 1. Main effect of factor A (1. Main effect of factor A (μμ[A])[A])

μμ[A]= {[A]= {μμ[Ab1]+ [Ab1]+ μμ[Ab2]}/2[Ab2]}/2 = ( 5 + 5)/2= 5= ( 5 + 5)/2= 5

2. Main effect of factor B ((2. Main effect of factor B ((μμ[B])[B])μμ[B]= {[B]= {μμ[a1B]+ [a1B]+ μμ[a2B]}/2[a2B]}/2 = { -5 +(-5)}/2= - 5= { -5 +(-5)}/2= - 5

Factorial patterns

Interaction effect (Interaction effect (μμ[AB])[AB])

μμ[AB]= {[AB]= {μμ[Ab1] - [Ab1] - μμ[Ab2]}/2[Ab2]}/2

= ( 5 - 5)/2= 0= ( 5 - 5)/2= 0

μμ[BA]= {[BA]= {μμ[a1B]- [a1B]- μμ[a2B]}/2[a2B]}/2

= { -5 -(-5)}/2= 0= { -5 -(-5)}/2= 0

No Interaction between Factor Caffeine (B) and workbook (A)

Main Effects and Main Effects and InteractionsInteractions

Main effects seen by row and column means; Slopes and breaks.

Interactions seen by lack of parallel lines..

Workbook (Factor A)

86

84

82

80

78

76

74

Mea

n RM

Tes

t Sco

re

No Yes

Without Caffeine

With Caffeine

Factor B

Single Main Effect for B

2.0 1.0

Factor A

25

20

15

10

5

0

Mea

n R

espo

nseSingle Main Effect

B=1

B=2

A

1 2

B1

2

10 10

20 20

(Coffee only)

Single Main Effect for A

2.0 1.0

Factor A

20

16

12

8

4

0

Mean R

esp

onse

Single Main Effect

B=1

B=2

A

1 2

B1

2

10 20

10 20

(Workbook only)

Two Main Effects; Both A & BTwo Main Effects; Both A & B

2.0 1.0

Factor A

35

30

25

20

15

10

5

0

Mean R

esponse

Two Main Effects

B=1

B=2

A

1 2

B1

2

10 20

20 30

Both workbook and coffee

Interaction (1)Interaction (1)

2.0 1.0

Factor A

35

30

25

20

15

10

5

0

Mean R

esponse

Interaction 1

B=1

B=2

A

1 2

B1

2

10 20

10 30

Interactions take many forms; all show lack of parallel lines.

Coffee has no effect without the workbook.


2.0 1.0

Factor A

25

20

15

10

5

0

Mean R

esponse

Interaction 2

B=1

B=2

A

1 2

B1

2

10 20

20 10

People with workbook do better without coffee; people without workbook do better with coffee.


2.0 1.0

Factor A

40

35

30

25

20

15

10

5

0

Mean R

esponse

Interaction 3

B = 1

B = 2

Coffee always helps, but it helps more if you use workbook.

Factorial designsFactorial designsFactor Factor BB

Factor Factor AA

bb11 bb22 bb33 bb44AveraAvera

gege

aa11yy1111 yy1212 yy1313 yy1414

aa22yy2121 yy2222 yy2323 yy2424

aa33yy3131 yy3232 yy3333 yy3434

AveraAveragege

1y

2y

3y

1y 2y 3y4y y

55443322110 xxxxxyij

Effect of A Effect of B No interaction between A and B

Factorial experiment with no Factorial experiment with no interactioninteraction

10 15 20 25 30

Temperature (oC)

0

5

10

15

20

25

50 % RH

80 % RH

Factorial experiment with no interaction

10 15 20 25 30

Temperature (oC)

0

5

10

15

20

25

50 % RH

80 % RH


10 15 20 25 30

Temperature (oC)

0

5

10

15

20

25

50 % RH

80 % RH

10 15 20 25 30

Temperature (oC)

0

5

10

15

20

25


22110 xxyij

0

1

2

Factorial experiment with Factorial experiment with interactioninteraction

10 15 20 25 30

Temperature (oC)

0

5

10

15

20

25

0

1

2

3

21322110 xxxxyij

Factorial designsFactorial designsFactor B

Factor A

b1 b2 b3 b4 Average

A1 y11 y12 y13 y14

AA22 y21 y22 y23 y24

A3 y31 y32 y33 y34

AveragAveragee

1y

2y

3y

1y 2y 3y 4y y

Effect of A Effect of B

5211421032951841731655443322110 xxxxxxxxxxxxxxxxxyij

Interactions between A and B

Source Degrees of freedomDegrees of freedomFactor A Factor B Interactions between A and B Residuals

aa-1 = 2-1 = 2

b - b - 11 = = 33

((aa-1)(-1)(bb-1) = 6-1) = 6

Ab- 1- (a-1) – (b-1)- (a-1)Ab- 1- (a-1) – (b-1)- (a-1)(b-1)(b-1) == 00

Total ab-1 = ab-1 = 1111

Two-way factorial designwith interaction, but without replicationwith interaction, but without replication

A= 3 B= 4

Source Degrees of freedom

Factor A Factor BResiduals

a-1 = 2b - 1 = 3

(a-1) (b-1) = 6

Total ab - 1= 11

Two-way factorial designwithout replication

Without replication it is necessary to assume no interaction between factors!


Factor A Factor B Interactions between A and B Residuals

a-1 = 2 b – 1 = 3

(a-1)(b-1) = 6

ab( r-1) = 12

Total rab – 1 = 23

Two-way factorial designwith interaction (r = 2)

A= 3 B= 4 r= 2

Linear Models for factorial Experiments

Single FactorSingle Factor: : A – a A – a levelslevels

yyijij = = + + ii + + ijij ii = 1,2, ... , = 1,2, ... ,aa; ; jj = = 1,2, ... ,1,2, ... ,rr

01

a

ii

Random error – Normal, mean 0, standard-deviation

i

iAyi when ofmean thei

Overall mean Effect on y of factor A when A = i

y11

y12

y13

y1n

y21

y22

y23

y2n

y31

y32

y33

y3n

ya1

ya2

ya3

yan

Levels of A1 2 3 a

observationsNormal distribution

Mean of observations

1 2 3 a

+ 1

+ 2

+ 3

+ a

Definitions

a

iia 1

1mean overall

a

iiiii a

iA1

1 )en (Effect wh

Two Factor:Two Factor: A A ( (a a levels), levels), B B ((bb levels levels

yyijkijk = = + + ii + + jj+ (+ ())ijij + + ijkijk

ii = 1,2, ... , = 1,2, ... ,aa ; ; jj = 1,2, ... , = 1,2, ... ,bb ; ; kk = = 1,2, ... ,1,2, ... ,rr

0,0,0,01111

b

jij

a

iij

b

jj

a

ii

ij

ijji

ij jBiAy

and when ofmean the

Overall mean

Main effect of A Main effect of B

Interaction effect of A and B

Table of Effects Overall mean, Main, Interaction Effects

Linear Model of factorial Linear Model of factorial design (2 factors)design (2 factors)

ijke

jkkjijky )(

= population mean for populations of all subjects, (= population mean for populations of all subjects, (grand grand mean)mean),,

ααjj = effect of group = effect of group jj in factor A (Greek letter alpha), in factor A (Greek letter alpha),

ββkk = effect of group = effect of group jj in factor B (Greek letter beta), in factor B (Greek letter beta),

αβ αβ jj k k = effect of the combination of group = effect of the combination of group j j in factor A and group in factor A and group kk in in

factor B, factor B,

eeijkijk = individual subject = individual subject kk’s variation not accounted for by any ’s variation not accounted for by any of the of the

effects aboveeffects above


Factor A Factor BInteractions between A and B Residuals

a-1 b - 1

(a-1)(b-1)

ab( r-1)

Total rab - 1

Two-way factorial designwith replications

a

iiA rbSS

1

2̂

b

jjB raSS

1

2̂

a

i

b

jijAB rSS

1 1

2

a

i

b

j

r

kijijkError yySS

1 1 1

2

Analysis of Variance (ANOVA) Table Entries (Two factors – A and B)

Example: Factorial DesignEffects of fatigue and alcohol

consumption on driving performance Fatigue

Rested (8 hrs sleep then awake 4 hrs) Fatigued (24 hrs no sleep)

Alcohol consumption None (control) 2 beers Blood alcohol .08 %

Indicator Variable:Performance errors on closed driving course

rated by driving instructor.

Experimental Research Experimental Research DataData

Alcohol (Factor A)Alcohol (Factor A)

Orthogonal design; n=2Orthogonal design; n=2

Fatigue (Factor Fatigue (Factor B)B)

None(J=1)

2 beers(J=2)

.08 %(J=3)

TiredTired

(K=1)(K=1)24

1618

1820

RestedRested

(K=2)(K=2)02

24

1618

Factorial Example Results

Intox2 beersnoneAlcohol Consumption

25

20

15

10

5

0

Drivin

g E

rrors

Factorial Design

Rested

Fatigued

Main Effects? Interactions? Both main effects and the interaction appear significant

DataDataA (Alcohol A (Alcohol

consumption)consumption)B (fatiged)B (fatiged) CellCell PersonPerson Driving Driving

errorerror

11 11 11 11 22

11 11 11 22 44

22 11 22 33 1616

22 11 22 44 1818

33 11 33 55 1818

33 11 33 66 2020

11 22 44 77 00

11 22 44 88 22

22 22 55 99 22

22 22 55 1010 44

33 22 66 1111 1616

33 22 66 1212 1818

MeanMean 1010

PersonPerson Driving Driving errorerror

MeanMean Different (D)= Different (D)= XXijij--μμ

D²D²

11 22 1010 -8-8 6464

22 44 1010 -6-6 3636

33 1616 1010 66 3636

44 1818 1010 88 6464

55 1818 1010 88 6464

66 2020 1010 1010 100100

77 00 1010 -10-10 100100

88 22 1010 -8-8 6464

99 22 1010 -8-8 6464

1010 44 1010 -6-6 3636

1111 1616 1010 66 3636

1212 1818 1010 88 6464

TotalTotal 728728

SS total

SS ErrorPersonPerson CellCell Driving Driving

errorerrorTreatment Treatment

meanmeanεεijij εε²²

11 11 22 33 -1-1 11

22 11 44 33 11 11

33 22 1616 1717 -1-1 11

44 22 1818 1717 11 11

55 33 1818 1919 -1-1 11

66 33 2020 1919 11 11

77 44 00 11 -1-1 11

88 44 22 11 11 11

99 55 22 33 -1-1 11

1010 55 44 33 11 11

1111 66 1616 1717 -1-1 11

1212 66 1818 1717 11 11

TotalTotal 00 1212

SS A – Effects of AlcoholPersonPerson μμ Level (A)Level (A) μμAA ααjj αα²j²j

11 1010 11 22 -8-8 6464

22 1010 11 22 -8-8 6464

33 1010 22 1010 00 00

44 1010 22 1010 00 00

55 1010 33 1818 88 6464

66 1010 33 1818 88 6464

77 1010 11 22 -8-8 6464

88 1010 11 22 -8-8 6464

99 1010 22 1010 00 00

1010 1010 22 1010 00 00

1111 1010 33 1818 88 6464

1212 1010 33 1818 88 6464

TotalTotal 00 512512

SS B – Effects of FatigueSS B – Effects of FatiguePersoPersonn

μ Level (B) μB βk βk²

11 10 1 13 3 9

22 10 1 13 3 9

33 10 1 13 3 9

44 10 1 13 3 9

55 10 1 13 3 9

66 10 1 13 3 9

77 10 2 7 -3 9

88 10 2 7 -3 9

99 10 2 7 -3 9

1010 10 2 7 -3 9

1111 10 2 7 -3 9

1212 10 2 7 -3 9

TotalTotal 108

Summary Table Source Source SSSS

TotalTotal 728728

AmongAmong 716716

AA 512512

BB 108108

WithinWithin 1212

Check: SSTotal=SSWithin+SSAmong728 = 716+12 SSInteraction = SSAmong –

(SSA+SSB).SSInteraction = 716-(512+108) = 96.

SourSourcece

SSSS dfdf MSMS FF

AA 515122

2 2 252566

128128

BB 101088

11 101088

5454

AxBAxB 9696 22 4848 2424

ErrorError 1212 6 6 22

98.5)6,1,05.( F14.5)6,2,05.( F

Random Effects Random Effects and Fixed Effects and Fixed Effects

FactorsFactors

FIXED VS. RANDOM Fixed Factor:

only the levels of interest are selected for the factor, and there is no intent to generalize to other levelsall population levels are present in the design (eg. Gender, treatment condition, ethnicity, size of community)

Random Factor:the levels are selected at random from the possible levels, and there is an intent to generalize to other levelsthe levels present in the design are a sample of the population to be generalized to (eg. Classrooms, subjects, teacher, school district, clinic)

Example - Fixed EffectsExample - Fixed Effects

Source of Protein, Level of Protein, Weight Source of Protein, Level of Protein, Weight GainGain

DependentDependent Weight Gain Weight Gain

IndependentIndependent Source of Protein,Source of Protein,

BeefBeef CerealCereal Pork Pork

Level of Protein,Level of Protein, HighHigh Low Low

Example - Random EffectsExample - Random EffectsIn this Example a Taxi company is interested In this Example a Taxi company is interested in comparing the effects of three in comparing the effects of three brands of brands of tirestires (A, B and C) on mileage (mpg). Mileage (A, B and C) on mileage (mpg). Mileage will also be effected by will also be effected by driverdriver. The company . The company selects selects b b = 4 drivers at random from its = 4 drivers at random from its collection of drivers. Each driver has collection of drivers. Each driver has n n = 3 = 3 opportunities to use each brand of tire in opportunities to use each brand of tire in which mileage is measured.which mileage is measured.DependentDependent

Mileage Mileage

IndependentIndependent Tire brand (A, B, C),Tire brand (A, B, C),

Fixed Effect FactorFixed Effect Factor Driver (1, 2, 3, 4),Driver (1, 2, 3, 4),

Random Effects factorRandom Effects factor

The Model for the fixed effects The Model for the fixed effects experimentexperiment

where where , , 11, , 22, , 33, , 11, , 22, (, ())11 11 , (, ())21 21 , , (())31 31 , (, ())12 12 , (, ())22 22 , (, ())32 32 , are fixed , are fixed unknown constants unknown constants

And And ijk ijk is random, normally distributed with is random, normally distributed with mean 0 and variance mean 0 and variance 22..

Note:Note:

ijkijjiijky

01111

b

jij

a

iij

n

jj

a

ii

The Model for the case when factor The Model for the case when factor B B is is a random effects factora random effects factor

where where , , 11, , 22, , 33, are fixed unknown constants , are fixed unknown constants

And And ijk ijk is random, normally distributed with mean is random, normally distributed with mean 0 and variance 0 and variance 22..

jj is normal with mean 0 and variance is normal with mean 0 and varianceand and

(())ijij is normal with mean 0 and variance is normal with mean 0 and varianceNote:Note:

ijkijjiijky

01

a

ii

2B

2AB

This model is called a variance components model

1.1. The The EMS EMS for Errorfor Error isis 22..2.2. The The EMS EMS for each ANOVA term contains for each ANOVA term contains

two or more terms the first of which istwo or more terms the first of which is 22..

3.3. All other terms in each All other terms in each EMS EMS contain contain both coefficients and subscripts (the both coefficients and subscripts (the total number of letters being one more total number of letters being one more than the number of factors) than the number of factors)

4. The subscript of 22 in the last term of in the last term of each each EMS EMS is the same as the treatment is the same as the treatment designationdesignation..

Rules for determining Expected Mean Squares (EMS) in an Anova Table

EXPECTED MEAN SQUARES

E(MS) expected average value for a mean square computed in an ANOVA based on sampling theory

Two conditions: null hypothesis E(MS) and alternative hypothesis E(MS) null hypothesis condition gives us the

basis to test the alternative hypothesis contribution (effect of factor or interaction)

EXPECTED MEAN SQUARES

→→1 Factor design:1 Factor design:Source E(MS)Source E(MS)

Treatment A Treatment A 22ee + n + n22

AA

errorerror 22ee (sampling (sampling

variation)variation)

Thus F=MS(A)/MS(e) tests to see if Thus F=MS(A)/MS(e) tests to see if Treatment A adds variation to what might Treatment A adds variation to what might be expected from usual sampling variability be expected from usual sampling variability of subjects. If the F is largeof subjects. If the F is large, , 22

A A 0. 0.

EXPECTED MEAN SQUARESEXPECTED MEAN SQUARES

→ → Factorial design (AxB):Factorial design (AxB):

SourceSource E(MS)E(MS)

Treatment A Treatment A 22ee + (1-b/B)n + (1-b/B)n22

ABAB + nb + nb22AA

errorerror 22ee (sampling variation) (sampling variation)

Thus F=MS(A)/MS(e) does not test to see if Thus F=MS(A)/MS(e) does not test to see if Treatment A adds variation to what might be Treatment A adds variation to what might be expected from usual sampling variability of expected from usual sampling variability of subjects unless b=B orsubjects unless b=B or 22

ABAB = 0 . = 0 .

If b (number of levels in study) = B (number in If b (number of levels in study) = B (number in the population, factor is FIXED; else RANDOMthe population, factor is FIXED; else RANDOM

EXPECTED MEAN SQUARES→→Factorial design (AxB):Factorial design (AxB):



ABAB + + nbnb22

AA

AxBAxB 22ee + (1-b/B)n + (1-b/B)n22

ABAB


IfIf 22ABAB = 0 , = 0 , and B is random, then F = and B is random, then F =

MS(A) / MS(AB) gives the correct test of the MS(A) / MS(AB) gives the correct test of the A effect. A effect.

EXPECTED MEAN SQUARES→ → Factorial design (AxB):Factorial design (AxB):



ABAB + nb + nb22AA

ABAB 22ee + (1-b/B)n + (1-b/B)n22

ABAB


If instead we test F = MS(AB)/MS(e) and it is non If instead we test F = MS(AB)/MS(e) and it is non significant, thensignificant, then 22

ABAB = 0 = 0 and it can be testedand it can be tested

F = MS(A) / MS(e)F = MS(A) / MS(e)

*** More power since df= a-1, *** More power since df= a-1,

df(error) instead of df = a-1, (a-1)*(b-1)df(error) instead of df = a-1, (a-1)*(b-1)

Source df Expected mean square

A I-1 2e + n2

AB + nJ2A

B J-1 2e + n2

AB + nI2B

AB (I-1)(J-1) 2e + n2

AB

error N-IJK 2e

Table 10.3: Expected mean square table for I x J random factorial design

Source df Expected mean square

A (fixed) I-1 2e + n 2

AB + nJ 2A

B (random) J-1 2e + nI 2

B

AB (I-1)(J-1) 2e + n 2

AB

error N-IJK 2e

Table 10.5: Expected mean square table for I x J mixed model factorial design

Mixed and Random Design Tests

General principle: look for denominator General principle: look for denominator E(MS) with same form as numerator E(MS) with same form as numerator E(MS) without the effect of interest:E(MS) without the effect of interest: F = F = 22

effecteffect + other variances /other + other variances /other variancesvariances

Try to eliminate interactions not Try to eliminate interactions not important to the study, test with important to the study, test with MS(error) if possibleMS(error) if possible

The Anova table for the two factor The Anova table for the two factor modelmodel

ijkijjiijky

SourcSourcee

SSSS dfdf MSMS

AA SSSSAAa a -1-1 SSSSAA/(/(aa – 1) – 1)

BB SSSSAAbb - 1 - 1 SSSSBB/(/(aa – 1) – 1)

ABAB SSSSABAB((a a -1)(-1)(b b --

1)1)SSSSABAB/(/(aa – 1) ( – 1) (aa – 1) – 1)

ErrorError SSSSErroErro

rr

abab((nn – – 1)1)

SSSSErrorError//abab((nn – 1) – 1)

The Anova table for the two factor The Anova table for the two factor model (A, B – fixed) model (A, B – fixed)

ijkijjiijky

SourcSourcee

SSSS dfdf MSMS EMSEMS FF

AA SSSSAA a a -1-1 MSMSAA MSMSAA/MS/MSErrorError

BB SSSSAA bb - 1 - 1 MSMSBB MSMSBB/MS/MSErrorError

ABAB SSSSABAB ((a a -1)(-1)(b b --1)1)

MSMSABAB MSMSABAB//MSMSErrorError

ErrorError SSSSErrorError abab((nn – 1) – 1) MSMSErrorError

2

a

iia

nb

1

22

1

b

jjb

na

1

22

1

a

i

b

jijba

n

1 1

22

11

EMS = Expected Mean Square

The Anova table for the two factor The Anova table for the two factor model model

(A – fixed, B - random) (A – fixed, B - random) ijkijjiijky

SourcSourcee

SSSS DfDf MSMS EMSEMS FF

AA SSSSAA a a -1-1 MSMSAA MSMSAA/MS/MSABAB

BB SSSSAA bb - 1 - 1 MSMSBB MSMSBB/MS/MSErrorError

ABAB SSSSABAB ((a a -1)(-1)(b b --1)1)

MSMSABAB MSMSABAB//MSMSErrorError

ErrorError SSSSErrorError abab((nn – 1) – 1) MSMSErrorError 2

a

iiAB a

nbn

1

222

1

22Bna

22ABn

Note:

The divisor for testing the main effects of A is no longer MSError but MSAB.

Factor CFactor B

Factor A

Factor A

10109988776655443322110 xxxxxxxxxxyijk

Factor B Factor C

Three-way factorial Three-way factorial designdesign

Factor A

42203219101189117811671156114511341123111 xxxxxxxxxxxxxxxxxxxx 10 Main effects

31 Two-way interactions

107272972718727094145841441031439314283141 xxxxxxxxxxxxxxxxxxxxxxxx

30 Three-way interactions

Three Factor:Three Factor: A A ((a a levels), levels), B B ((b b levels), levels), C C ((c c levels)levels)

yyijklijkl = = + + ii ++ jj++ ijij ++ kk ++ (())ikik ++

(())jkjk++ ijkijk ++ ijklijkl

= = ++ ii ++ jj++ kk ++ ijij ++ ( (ikik ++ ( (jkjk

++ ijkijk ++ ijklijkl

ii = 1,2, ... , = 1,2, ... ,aa ; ; jj = 1,2, ... , = 1,2, ... ,bb ; ; kk = 1,2, ... , = 1,2, ... ,cc; ; ll = = 1,2, ... ,1,2, ... ,rr 0,,0,0,0,0

11111

c

kijk

a

iij

c

kk

b

jj

a

ii

Main effects Two factor Interactions

Three factor InteractionRandom error

ijkijk = the mean of = the mean of y y when when A A = = ii, , B B = = jj, , C C = = kk

= = ++ ii ++ jj++ kk ++ ijij ++ ( (ikik ++

((jkjk

++ ijkijk

ii = 1,2, ... , = 1,2, ... ,aa ; ; jj = 1,2, ... , = 1,2, ... ,bb ; ; kk = 1,2, ... , = 1,2, ... ,cc; ; ll = = 1,2, ... ,1,2, ... ,rr

0,,0,0,0,011111

c

kijk

a

iij

c

kk

b

jj

a

ii


Three factor Interaction

Overall mean

Anova tablefor 3 factor Experiment

SourceSource SSSS dfdf MSMS FF p p -value-value

AA SSSSAA a - 1a - 1 MSMSAA MSMSAA/MS/MSErrorError

BB SSSSBB b - 1b - 1 MSMSBB MSMSBB/MS/MSErrorError

CC SSSSCC c - 1c - 1 MSMSCC MSMSCC/MS/MSErrorError

ABAB SSSSABAB ((a - 1a - 1)()(b - 1b - 1)) MSMSABAB MSMSABAB//MSMSErrorError

ACAC SSSSACAC ((a - 1a - 1)()(c - 1c - 1)) MSMSACAC MSMSACAC//MSMSErrorError

BCBC SSSSBCBC ((b - 1b - 1)()(c - 1c - 1)) MSMSBCBC MSMSBCBC//MSMSErrorError

ABCABC SSSSABCABC ((a - 1a - 1)()(b - 1b - 1)()(c c - 1- 1))

MSMSABCABC MSMSABCABC//MSMSErrorError

ErrorError SSSSErrorError abc(r - 1)abc(r - 1) MSMSErroErro

rr

Sum of squares entries

a

ii

a

iiA yyrbcrbcSS

1

2

1

2̂

Similar expressions for SSB , and SSC.

a

i

b

jjiij

a

iijAB yyyyrcrcSS

1 1

2

1

2

Similar expressions for SSBC , and SSAC.

Sum of squares entries

Finally

a

iikjABC rSS

1

2

a

i

b

j

c

kijkkiijijk yyyyyr

1 1 1 2 ikj yyy

a

i

b

j

c

k

r

lijkijklError yySS

1 1 1 1

2

a

iiA rbcSS

1

2̂

b

jjB racSS

1

2̂

a

i

b

jijAB rcSS

1 1

2

a

i

c

kikAC rbSS

1 1

2

b

j

c

kjkBC raSS

1 1

2

a

i

b

j

c

kijkABC rSS

1 1 1

2

a

i

b

j

c

k

r

lijkijklError yySS

1 1 1 1

2

Analysis of Variance (ANOVA) Table Entries ( 3 factors – A, B and C)

c

kkC rabSS

1

2̂

The statistical model for 3 factor Experiment

effectsmain effectmean kjiijk/y

error randomninteractiofactor 3nsinteractiofactor 2

ijk/ijkjkikij


Factor A Factor B Factor CInteractions between A and BInteractions between A and CInteractions between B and CInteractions between A, B and C Residuals

a-1 = 2 b – 1 = 5c-1 = 3

(a-1)(b-1) = 10(a-1)(c-1) = 6

(b-1)(c-1) = 15(a-1)(b-1)(c-1) = 30

abc( r -1) = 72

Total n = rabc - 1= 143

Three-way factorial Three-way factorial designdesign

Four Factorial Design

Four Factor:Four Factor:

yyijklmijklm = = + + + + jj+ + (())ijij + + kk + + (())ikik + + (())jkjk+ + (())ijkijk + + ll+ + (())ilil + + (())jljl+ + (())ijlijl + + (())klkl + + (())iklikl + + (())jkljkl+ + (())ijklijkl + + ijklmijklm

= =

++ii + + jj+ + kk + + l l

+ (+ ())ijij + + (())ikik + + (())jkjk + + (())ilil + + (())jljl+ + (())klkl

++(())ijkijk+ + (())ijlijl + + (())iklikl + + (())jkljkl

+ (+ ())ijklijkl + + ijklmijklm

i i = 1,2, ... ,= 1,2, ... ,aa ; ; jj = 1,2, ... , = 1,2, ... ,bb ; ; kk = 1,2, ... , = 1,2, ... ,cc; ; ll = 1,2, ... , = 1,2, ... ,dd; ; mm = 1,2, ... , = 1,2, ... ,rr

wherewhere 0 = 0 = ii = = jj= = ( ())ijij kk = = (())ikik = = (())jkjk= = ( ())ijkijk = = ll= = (())ilil = = ( ())jljl = = ( ())ijlijl = = ( ())klkl = = ( ())iklikl = = ( ())jkljkl = =

(())ijklijkl

and and denotes the summation over any of the subscripts. denotes the summation over any of the subscripts.


Three factor Interactions

Overall mean

Four factor Interaction Random error

Estimation of Main Effects and Estimation of Main Effects and

InteractionsInteractions Estimator of Main effect of a FactorEstimator of Main effect of a Factor

• Estimator of k-factor interaction effect at a combination of levels of the k factors

= Mean at level i of the factor - Overall Mean

Example:Example:

The The main effectmain effect of factor B at level j in a four of factor B at level j in a four factor (A,B,C and D) experiment is estimated by:factor (A,B,C and D) experiment is estimated by:

• The two-factor interaction effect between factors B and C when B is at level j and C is at level k is estimated by:

yyˆjj

yyyy kjjkjk

The The three-factor interactionthree-factor interaction effect between effect between factors B, C and D when B is at level j, C is at factors B, C and D when B is at level j, C is at level k and D is at level l is estimated by:level k and D is at level l is estimated by:

• Finally the four-factor interaction effect between factors A,B, C and when A is at level i, B is at level j, C is at level k and D is at level l is estimated by:

yyyyyyyy lkjklljjkjkljkl

jklikiijjklklilijijkijklijkl yyyyyyyyy

yyyyyyy lkjikllj

The Completely Randomized Design is called balanced

If the number of observations per treatment combination is unequal the design is called unbalanced.

If for some of the treatment combinations there are no

observations the design is called incomplete.

Remarks

Why should more than two levels of a factor be used in a

factorial design?

Two-levels of a factor

10 15 20 25 30

Temperature (oC)

0

5

10

15

20

25

30

10 15 20 25 30

Temperature (oC)

0

5

10

15

20

25

30

Three-levelsfactor qualitativefactor qualitative

22110 xxy

1

Low Medium High

0

2

10 15 20 25 30

Temperature (oC)

0

5

10

15

20

25

30

Three-levelsThree-levelsfactor quantitativefactor quantitative

2210 xxy

Why should not many levels of each factor be

used in a factorial design?

Because each level of each factor increases the number of

experimental units to be used

For example, a five factor experiment with four levels per factor yields 45 = 1024 different

combinations

If not all combinations are applied in an experiment, the design is

partially factorial

Full and fractional Full and fractional factorial designfactorial design

Full factorial designFull factorial design Study all combinationsStudy all combinations Can find effect of all factorsCan find effect of all factors

Fractional (incomplete) factorial Fractional (incomplete) factorial designdesign Leave some treatment groups emptyLeave some treatment groups empty Less informationLess information May not get all interactionsMay not get all interactions No problem if interaction is negligibleNo problem if interaction is negligible

Fractional factorial Fractional factorial designdesign

Large number of factorsLarge number of factors Large number of experimentsLarge number of experiments Full factorial design too expensiveFull factorial design too expensive Use a fractional factorial designUse a fractional factorial design

22k-pk-p design allows analyzing k factors design allows analyzing k factors with only 2with only 2k-pk-pexperiments.experiments. 22k-1k-1 design requires only half as many design requires only half as many

experimentsexperiments 22k-2k-2 design requires only one quarter of design requires only one quarter of

the experimentsthe experiments

common design problems 1.masking factor effects 2.uncontrolled factors 3.one-factor-at-a-time...

Documents

experimental factors

uncontrolled factors

anova factors

factor effects slide

independent variables

study design

masking factor effects

factor value