Common Design ProblemsCommon Design Problems
1.1. Masking factor effectsMasking factor effects
2.2. Uncontrolled factorsUncontrolled factors
3.3. One-factor-at-a-time testingOne-factor-at-a-time testing
Masking factor effectsMasking factor effects
if variation in test results is on the if variation in test results is on the same order of magnitude as the same order of magnitude as the factor effects, the latter may go factor effects, the latter may go undetected. undetected. This can be addressed through This can be addressed through
appropriate choice of sample size.appropriate choice of sample size. unmeasured covariates (e.g., the unmeasured covariates (e.g., the
effect of time passing, as an effect of time passing, as an instrument degrades) can also lead to instrument degrades) can also lead to variation that masks factor effectsvariation that masks factor effects
Uncontrolled factorsUncontrolled factors
it’s pretty obvious that all variables of it’s pretty obvious that all variables of interest should be included as factorsinterest should be included as factors
sometimes, though, it can be tricky to sometimes, though, it can be tricky to choose the appropriate level of model choose the appropriate level of model granularity. granularity. Sometimes a “high-level feature” (i.e., Sometimes a “high-level feature” (i.e.,
some function of the levels of many some function of the levels of many factors) could be chosen in place of the factors) could be chosen in place of the many factors that influence itmany factors that influence it
this can make it difficult to vary the factor this can make it difficult to vary the factor appropriately, and can limit analysis of appropriately, and can limit analysis of the experimental results.the experimental results.
it can be tempting to vary each factor it can be tempting to vary each factor value independently, holding the others value independently, holding the others constant. However, this does not constant. However, this does not explore the factor space very explore the factor space very effectively. effectively. It would be a terrible local search strategy! It would be a terrible local search strategy!
Looking at the same point in another Looking at the same point in another way, it neglects the possibility that way, it neglects the possibility that interactions between factors could be interactions between factors could be importantimportant
One-factor-at-a-time testingOne-factor-at-a-time testing
Factorial DesignFactorial Design
DefinitionDefinition A Study design in which responses are A Study design in which responses are measured at different combinations of level measured at different combinations of level
of one or more experimental factorsof one or more experimental factors A study design in which treatment consists of A study design in which treatment consists of
two or more factors or independent two or more factors or independent variables A Study in which ll combinations of levels of
two or more independent variables (factors) are measured
A Study design when the combined effects A Study design when the combined effects of two or more factors are investigated of two or more factors are investigated
concurrentlyconcurrently Two or more ANOVA factors are combined in Two or more ANOVA factors are combined in
a single studya single study
FactorsFactors A variable upon which the experimenter A variable upon which the experimenter
believes that one or more response believes that one or more response variables may depend, and which the variables may depend, and which the experimenter can controlexperimenter can control the design of the experiment will largely the design of the experiment will largely
consist of a policy for determining how to consist of a policy for determining how to set the factors in each experimental trialset the factors in each experimental trial
They are denoted as capital lettersThey are denoted as capital letters
Possible values of a factor are called Possible values of a factor are called levelslevels.. in other literature, the word “version” is in other literature, the word “version” is
used for a qualitative (not quantitative) used for a qualitative (not quantitative) levellevel
They are denoted as lowercase levelThey are denoted as lowercase level
TreatmentsTreatments
They represent a particular They represent a particular combination of factors levelcombination of factors level
They are denoted by the same They are denoted by the same combination of the lowercase letters combination of the lowercase letters
that represent the corresponding that represent the corresponding levelslevels
With 3 factors A, B, C, the treatment With 3 factors A, B, C, the treatment corresponding to the combination of corresponding to the combination of
level alevel a11 of A, b of A, b33 of B, andof B, and CC2 2 of C is denoted by aof C is denoted by a11 bb33 c c22
Complete Factorial ExperimentComplete Factorial Experiment
An experiment in which responses are measured An experiment in which responses are measured at all combination of levels of the factorsat all combination of levels of the factors
A complete factorial experiment in which there are a levels of A, b levels of factor B and so on is called a x b x
… factorial experiment
Total number of treatments in an a x b x c … complete factorial
experiment is t = abc…
The Use of Factorial The Use of Factorial DesignDesign
Identify factors with significant effects Identify factors with significant effects on the responseon the response
Identify interactions among factorsIdentify interactions among factors Identify which factors have the most Identify which factors have the most
important effects on the responseimportant effects on the response Decide whether further investigation of Decide whether further investigation of
a factor’s effect is justifieda factor’s effect is justified Investigate the functional dependence Investigate the functional dependence
of a response on multiple factors of a response on multiple factors simultaneously (if and only if you test simultaneously (if and only if you test
many levels of each factor)many levels of each factor)
Advantages Advantages of Factorial Designsof Factorial Designs
1.1. Saves Time & Efforte.g., Could Use Separate Completely
Randomized Designs for Each Variable
2. Controls Confounding Effects by Putting Other Variables into Model
3. Can Explore Interaction Between Variables
Advantages Advantages of Factorial Designsof Factorial Designs
1.1. Saves Time & Efforte.g., Could Use Separate Completely
Randomized Designs for Each Variable
2. Controls Confounding Effects by Putting Other Variables into Model
3. Can Explore Interaction Between Variables
ANOVA ANOVA Null HypothesesNull Hypotheses
1.1. No Difference in Means Due to No Difference in Means Due to Factor AFactor AHH00: : 11.... = = 22.... =... = =... = aa....
2.2. No Difference in Means Due to No Difference in Means Due to Factor BFactor BHH00: : ..11.. = = ..22.. =... = =... = ..bb..
3.3. No Interaction of Factors A & BNo Interaction of Factors A & BHH00: AB: ABijij = 0 = 0
Total Variation
Total Variation
ANOVA ANOVA Total Variation Total Variation PartitioningPartitioning
Variation Due to Treatment
Variation Due to Treatment
Variation Due to Random Sampling
Variation Due to Random Sampling
Variation Due to Interaction
Variation Due to Interaction
SSESSESS (Treatment)SS (Treatment)
SS(AB)
SS(Total)SS(Total)
Variation Due to
Factor B
Variation Due to
Factor BSSB
Variation Due to
Factor A
Variation Due to
Factor ASSA
Factorial Patterns
Simple EffectSimple Effect the effect of one factor on only one the effect of one factor on only one
level of another factorlevel of another factor Effect of changing the level of one Effect of changing the level of one
factor while holding the level of the factor while holding the level of the other factor fixedother factor fixed
If the simple effects differ, there is an interaction
Factorial Patterns
Main EffectMain Effect Effect of variation in a single variable, Effect of variation in a single variable,
averaged across all levels of all other averaged across all levels of all other variablesvariables
An outcome that is a consistent An outcome that is a consistent difference between levels of a factor. difference between levels of a factor.
Main Effect of One variable, no Main Effect of One variable, no effect of the othereffect of the other
Main Effect of All VariablesMain Effect of All Variables
Factorial Patterns
InteractionInteraction Effect of variation in a single variable
depends on the specific levels of at least one other variable
It exists when differences on one factor depend on the level you are on another factor
The effects of one factor change depending on the level of another factor
The effect of one factor depends on the level The effect of one factor depends on the level of the other factorof the other factor
Interaction effect
How do we know if there is an How do we know if there is an interaction in a factorial design?interaction in a factorial design? Statistical analysis will report all main Statistical analysis will report all main
effects and interactions.effects and interactions. If you can not talk about effect on one If you can not talk about effect on one
factor without mentioning the other factor without mentioning the other factorfactor
Spot an interaction in the graphs – Spot an interaction in the graphs – whenever there are lines that are not whenever there are lines that are not parallel there is an interaction present!parallel there is an interaction present!
Interaction Effect
1.1. Occurs When Effects of One Factor Occurs When Effects of One Factor Vary According to Levels of Other FactorVary According to Levels of Other Factor
2.2. When Significant, Interpretation of When Significant, Interpretation of Main Effects (A & B) Is ComplicatedMain Effects (A & B) Is Complicated
3.3. Can Be DetectedCan Be DetectedIn Data Table, Pattern of Cell Means in One In Data Table, Pattern of Cell Means in One
Row Differs From Another RowRow Differs From Another Row
In Graph of Cell Means, Lines CrossIn Graph of Cell Means, Lines Cross
Interaction Effect
Diverging trends, from little Diverging trends, from little difference to larger differencedifference to larger difference
Converging trends, from large Converging trends, from large difference to smaller difference.difference to smaller difference.
Cross-over interactions, with Cross-over interactions, with decrease in one variable while the decrease in one variable while the other stays constant or increases.other stays constant or increases.
Interaction effect:Interaction effect:difference in magnitude of response
Interaction effect:Interaction effect:difference in direction of response
Graphs of InteractionGraphs of Interaction
Effects of Motivation (High or Low) & Training Method (A, B, C) on Mean
Learning TimeInteraction No Interaction
AverageAverageResponseResponse
AA BB CC
HighHigh
LowLow
AverageAverageResponseResponse
AA BB CC
HighHigh
LowLow
Assumption of Factorial Assumption of Factorial DesignDesign
Interval/ratio data Normal distribution or N at least 30
Independent observations Homogeneity of variance
Proportional or equal cell sizes
Strategy for factorial analysis
1. A test is performed to see if there is an interaction between the factors
2. If statistically significant interaction is indicated, the simple effect of the factors are examined
separately3. If there is no demonstrable interaction, then inferences are
made about each of the main effect
Test first the higher order interactions. If an interaction is present there is no
need to test lower order interactions or main effects involving those factors. All
factors in the interaction affect the response and they interact
The testing continues with for lower order interactions and main effects for
factors which have not yet been determined to affect the response.
Statistical test infactorial experiment
2-way ANOVA
Example: Study aids for examExample: Study aids for exam A: workbook or notA: workbook or not B: 1 cup of coffee or notB: 1 cup of coffee or not
Workbook (Factor A)Workbook (Factor A)
Caffeine Caffeine (Factor B)(Factor B)
NoNo YesYes
YesYes Caffeine Caffeine onlyonly
BothBoth
NoNo Neither Neither (Control)(Control)
Workbook Workbook onlyonly
Effects of Study Aids for Exams
N=30 N=30 per cellper cell
Workbook (Factor A)Workbook (Factor A) Row Row MeanMeanss
CaffeineCaffeine
(Factor (Factor B)B)
No (a1)No (a1) Yes (a2)Yes (a2)
Yes (b1)Yes (b1) CaffeeCaffee
=80=80BothBoth
=85=8582.582.5
No (b2)No (b2) ControlControl
=75=75BookBook
=80=8077.577.5
Col Col MeansMeans
77.577.5 82.582.5 8080
X X
X X
Factorial patterns Simple EffectSimple Effect
Simple effect of workbook factor at no caffeine Simple effect of workbook factor at no caffeine ((μμ[Ab1])[Ab1])
μμ[Ab1]= 85 – 80= 5[Ab1]= 85 – 80= 5Simple effect of workbook factor at with caffeine Simple effect of workbook factor at with caffeine ((μμ[Ab2])[Ab2])
μμ[Ab2]= 80 – 75= 5[Ab2]= 80 – 75= 5Simple effect of caffeine factor at no workbook (Simple effect of caffeine factor at no workbook (μμ[a1B])[a1B])
μμ[a1B]= 75 – 80= - 5[a1B]= 75 – 80= - 5Simple effect of caffeine factor at with workbook Simple effect of caffeine factor at with workbook ((μμ[a2B])[a2B])
μμ[a2B]= 80 – 85= - 5[a2B]= 80 – 85= - 5
Factorial patterns
Main Effect Main Effect The average of two simple effects The average of two simple effects 1. Main effect of factor A (1. Main effect of factor A (μμ[A])[A])
μμ[A]= {[A]= {μμ[Ab1]+ [Ab1]+ μμ[Ab2]}/2[Ab2]}/2 = ( 5 + 5)/2= 5= ( 5 + 5)/2= 5
2. Main effect of factor B ((2. Main effect of factor B ((μμ[B])[B])μμ[B]= {[B]= {μμ[a1B]+ [a1B]+ μμ[a2B]}/2[a2B]}/2 = { -5 +(-5)}/2= - 5= { -5 +(-5)}/2= - 5
Factorial patterns
Interaction effect (Interaction effect (μμ[AB])[AB])
μμ[AB]= {[AB]= {μμ[Ab1] - [Ab1] - μμ[Ab2]}/2[Ab2]}/2
= ( 5 - 5)/2= 0= ( 5 - 5)/2= 0
μμ[BA]= {[BA]= {μμ[a1B]- [a1B]- μμ[a2B]}/2[a2B]}/2
= { -5 -(-5)}/2= 0= { -5 -(-5)}/2= 0
No Interaction between Factor Caffeine (B) and workbook (A)
Main Effects and Main Effects and InteractionsInteractions
Main effects seen by row and column means; Slopes and breaks.
Interactions seen by lack of parallel lines..
Workbook (Factor A)
86
84
82
80
78
76
74
Mea
n RM
Tes
t Sco
re
No Yes
Without Caffeine
With Caffeine
Factor B
Single Main Effect for B
2.0 1.0
Factor A
25
20
15
10
5
0
Mea
n R
espo
nseSingle Main Effect
B=1
B=2
A
1 2
B1
2
10 10
20 20
(Coffee only)
Single Main Effect for A
2.0 1.0
Factor A
20
16
12
8
4
0
Mean R
esp
onse
Single Main Effect
B=1
B=2
A
1 2
B1
2
10 20
10 20
(Workbook only)
Two Main Effects; Both A & BTwo Main Effects; Both A & B
2.0 1.0
Factor A
35
30
25
20
15
10
5
0
Mean R
esponse
Two Main Effects
B=1
B=2
A
1 2
B1
2
10 20
20 30
Both workbook and coffee
Interaction (1)Interaction (1)
2.0 1.0
Factor A
35
30
25
20
15
10
5
0
Mean R
esponse
Interaction 1
B=1
B=2
A
1 2
B1
2
10 20
10 30
Interactions take many forms; all show lack of parallel lines.
Coffee has no effect without the workbook.
Interaction (2)Interaction (2)
2.0 1.0
Factor A
25
20
15
10
5
0
Mean R
esponse
Interaction 2
B=1
B=2
A
1 2
B1
2
10 20
20 10
People with workbook do better without coffee; people without workbook do better with coffee.
Interaction (3)Interaction (3)
2.0 1.0
Factor A
40
35
30
25
20
15
10
5
0
Mean R
esponse
Interaction 3
B = 1
B = 2
Coffee always helps, but it helps more if you use workbook.
Factorial designsFactorial designsFactor Factor BB
Factor Factor AA
bb11 bb22 bb33 bb44AveraAvera
gege
aa11yy1111 yy1212 yy1313 yy1414
aa22yy2121 yy2222 yy2323 yy2424
aa33yy3131 yy3232 yy3333 yy3434
AveraAveragege
1y
2y
3y
1y 2y 3y4y y
55443322110 xxxxxyij
Effect of A Effect of B No interaction between A and B
Factorial experiment with no Factorial experiment with no interactioninteraction
10 15 20 25 30
Temperature (oC)
0
5
10
15
20
25
50 % RH
80 % RH
Factorial experiment with no Factorial experiment with no interactioninteraction
10 15 20 25 30
Temperature (oC)
0
5
10
15
20
25
50 % RH
80 % RH
Factorial experiment with no interaction
10 15 20 25 30
Temperature (oC)
0
5
10
15
20
25
50 % RH
80 % RH
Factorial experiment with no Factorial experiment with no interactioninteraction
10 15 20 25 30
Temperature (oC)
0
5
10
15
20
25
50 % RH
80 % RH
10 15 20 25 30
Temperature (oC)
0
5
10
15
20
25
Factorial experiment with no Factorial experiment with no interactioninteraction
22110 xxyij
0
1
2
Factorial experiment with Factorial experiment with interactioninteraction
10 15 20 25 30
Temperature (oC)
0
5
10
15
20
25
0
1
2
3
21322110 xxxxyij
Factorial designsFactorial designsFactor B
Factor A
b1 b2 b3 b4 Average
A1 y11 y12 y13 y14
AA22 y21 y22 y23 y24
A3 y31 y32 y33 y34
AveragAveragee
1y
2y
3y
1y 2y 3y 4y y
Effect of A Effect of B
5211421032951841731655443322110 xxxxxxxxxxxxxxxxxyij
Interactions between A and B
Source Degrees of freedomDegrees of freedomFactor A Factor B Interactions between A and B Residuals
aa-1 = 2-1 = 2
b - b - 11 = = 33
((aa-1)(-1)(bb-1) = 6-1) = 6
Ab- 1- (a-1) – (b-1)- (a-1)Ab- 1- (a-1) – (b-1)- (a-1)(b-1)(b-1) == 00
Total ab-1 = ab-1 = 1111
Two-way factorial designwith interaction, but without replicationwith interaction, but without replication
A= 3 B= 4
Source Degrees of freedom
Factor A Factor BResiduals
a-1 = 2b - 1 = 3
(a-1) (b-1) = 6
Total ab - 1= 11
Two-way factorial designwithout replication
Without replication it is necessary to assume no interaction between factors!
Source Degrees of freedom
Factor A Factor B Interactions between A and B Residuals
a-1 = 2 b – 1 = 3
(a-1)(b-1) = 6
ab( r-1) = 12
Total rab – 1 = 23
Two-way factorial designwith interaction (r = 2)
A= 3 B= 4 r= 2
Linear Models for factorial Experiments
Single FactorSingle Factor: : A – a A – a levelslevels
yyijij = = + + ii + + ijij ii = 1,2, ... , = 1,2, ... ,aa; ; jj = = 1,2, ... ,1,2, ... ,rr
01
a
ii
Random error – Normal, mean 0, standard-deviation
i
iAyi when ofmean thei
Overall mean Effect on y of factor A when A = i
y11
y12
y13
y1n
y21
y22
y23
y2n
y31
y32
y33
y3n
ya1
ya2
ya3
yan
Levels of A1 2 3 a
observationsNormal distribution
Mean of observations
1 2 3 a
+ 1
+ 2
+ 3
+ a
Definitions
a
iia 1
1mean overall
a
iiiii a
iA1
1 )en (Effect wh
Two Factor:Two Factor: A A ( (a a levels), levels), B B ((bb levels levels
yyijkijk = = + + ii + + jj+ (+ ())ijij + + ijkijk
ii = 1,2, ... , = 1,2, ... ,aa ; ; jj = 1,2, ... , = 1,2, ... ,bb ; ; kk = = 1,2, ... ,1,2, ... ,rr
0,0,0,01111
b
jij
a
iij
b
jj
a
ii
ij
ijji
ij jBiAy
and when ofmean the
Overall mean
Main effect of A Main effect of B
Interaction effect of A and B
Table of Effects Overall mean, Main, Interaction Effects
Linear Model of factorial Linear Model of factorial design (2 factors)design (2 factors)
ijke
jkkjijky )(
= population mean for populations of all subjects, (= population mean for populations of all subjects, (grand grand mean)mean),,
ααjj = effect of group = effect of group jj in factor A (Greek letter alpha), in factor A (Greek letter alpha),
ββkk = effect of group = effect of group jj in factor B (Greek letter beta), in factor B (Greek letter beta),
αβ αβ jj k k = effect of the combination of group = effect of the combination of group j j in factor A and group in factor A and group kk in in
factor B, factor B,
eeijkijk = individual subject = individual subject kk’s variation not accounted for by any ’s variation not accounted for by any of the of the
effects aboveeffects above
Source Degrees of freedom
Factor A Factor BInteractions between A and B Residuals
a-1 b - 1
(a-1)(b-1)
ab( r-1)
Total rab - 1
Two-way factorial designwith replications
a
iiA rbSS
1
2̂
b
jjB raSS
1
2̂
a
i
b
jijAB rSS
1 1
2
a
i
b
j
r
kijijkError yySS
1 1 1
2
Analysis of Variance (ANOVA) Table Entries (Two factors – A and B)
Example: Factorial DesignEffects of fatigue and alcohol
consumption on driving performance Fatigue
Rested (8 hrs sleep then awake 4 hrs) Fatigued (24 hrs no sleep)
Alcohol consumption None (control) 2 beers Blood alcohol .08 %
Indicator Variable:Performance errors on closed driving course
rated by driving instructor.
Experimental Research Experimental Research DataData
Alcohol (Factor A)Alcohol (Factor A)
Orthogonal design; n=2Orthogonal design; n=2
Fatigue (Factor Fatigue (Factor B)B)
None(J=1)
2 beers(J=2)
.08 %(J=3)
TiredTired
(K=1)(K=1)24
1618
1820
RestedRested
(K=2)(K=2)02
24
1618
Factorial Example Results
Intox2 beersnoneAlcohol Consumption
25
20
15
10
5
0
Drivin
g E
rrors
Factorial Design
Rested
Fatigued
Main Effects? Interactions? Both main effects and the interaction appear significant
DataDataA (Alcohol A (Alcohol
consumption)consumption)B (fatiged)B (fatiged) CellCell PersonPerson Driving Driving
errorerror
11 11 11 11 22
11 11 11 22 44
22 11 22 33 1616
22 11 22 44 1818
33 11 33 55 1818
33 11 33 66 2020
11 22 44 77 00
11 22 44 88 22
22 22 55 99 22
22 22 55 1010 44
33 22 66 1111 1616
33 22 66 1212 1818
MeanMean 1010
PersonPerson Driving Driving errorerror
MeanMean Different (D)= Different (D)= XXijij--μμ
D²D²
11 22 1010 -8-8 6464
22 44 1010 -6-6 3636
33 1616 1010 66 3636
44 1818 1010 88 6464
55 1818 1010 88 6464
66 2020 1010 1010 100100
77 00 1010 -10-10 100100
88 22 1010 -8-8 6464
99 22 1010 -8-8 6464
1010 44 1010 -6-6 3636
1111 1616 1010 66 3636
1212 1818 1010 88 6464
TotalTotal 728728
SS total
SS ErrorPersonPerson CellCell Driving Driving
errorerrorTreatment Treatment
meanmeanεεijij εε²²
11 11 22 33 -1-1 11
22 11 44 33 11 11
33 22 1616 1717 -1-1 11
44 22 1818 1717 11 11
55 33 1818 1919 -1-1 11
66 33 2020 1919 11 11
77 44 00 11 -1-1 11
88 44 22 11 11 11
99 55 22 33 -1-1 11
1010 55 44 33 11 11
1111 66 1616 1717 -1-1 11
1212 66 1818 1717 11 11
TotalTotal 00 1212
SS A – Effects of AlcoholPersonPerson μμ Level (A)Level (A) μμAA ααjj αα²j²j
11 1010 11 22 -8-8 6464
22 1010 11 22 -8-8 6464
33 1010 22 1010 00 00
44 1010 22 1010 00 00
55 1010 33 1818 88 6464
66 1010 33 1818 88 6464
77 1010 11 22 -8-8 6464
88 1010 11 22 -8-8 6464
99 1010 22 1010 00 00
1010 1010 22 1010 00 00
1111 1010 33 1818 88 6464
1212 1010 33 1818 88 6464
TotalTotal 00 512512
SS B – Effects of FatigueSS B – Effects of FatiguePersoPersonn
μ Level (B) μB βk βk²
11 10 1 13 3 9
22 10 1 13 3 9
33 10 1 13 3 9
44 10 1 13 3 9
55 10 1 13 3 9
66 10 1 13 3 9
77 10 2 7 -3 9
88 10 2 7 -3 9
99 10 2 7 -3 9
1010 10 2 7 -3 9
1111 10 2 7 -3 9
1212 10 2 7 -3 9
TotalTotal 108
Summary Table Source Source SSSS
TotalTotal 728728
AmongAmong 716716
AA 512512
BB 108108
WithinWithin 1212
Check: SSTotal=SSWithin+SSAmong728 = 716+12 SSInteraction = SSAmong –
(SSA+SSB).SSInteraction = 716-(512+108) = 96.
SourSourcece
SSSS dfdf MSMS FF
AA 515122
2 2 252566
128128
BB 101088
11 101088
5454
AxBAxB 9696 22 4848 2424
ErrorError 1212 6 6 22
98.5)6,1,05.( F14.5)6,2,05.( F
Random Effects Random Effects and Fixed Effects and Fixed Effects
FactorsFactors
FIXED VS. RANDOM Fixed Factor:
only the levels of interest are selected for the factor, and there is no intent to generalize to other levelsall population levels are present in the design (eg. Gender, treatment condition, ethnicity, size of community)
Random Factor:the levels are selected at random from the possible levels, and there is an intent to generalize to other levelsthe levels present in the design are a sample of the population to be generalized to (eg. Classrooms, subjects, teacher, school district, clinic)
Example - Fixed EffectsExample - Fixed Effects
Source of Protein, Level of Protein, Weight Source of Protein, Level of Protein, Weight GainGain
DependentDependent Weight Gain Weight Gain
IndependentIndependent Source of Protein,Source of Protein,
BeefBeef CerealCereal Pork Pork
Level of Protein,Level of Protein, HighHigh Low Low
Example - Random EffectsExample - Random EffectsIn this Example a Taxi company is interested In this Example a Taxi company is interested in comparing the effects of three in comparing the effects of three brands of brands of tirestires (A, B and C) on mileage (mpg). Mileage (A, B and C) on mileage (mpg). Mileage will also be effected by will also be effected by driverdriver. The company . The company selects selects b b = 4 drivers at random from its = 4 drivers at random from its collection of drivers. Each driver has collection of drivers. Each driver has n n = 3 = 3 opportunities to use each brand of tire in opportunities to use each brand of tire in which mileage is measured.which mileage is measured.DependentDependent
Mileage Mileage
IndependentIndependent Tire brand (A, B, C),Tire brand (A, B, C),
Fixed Effect FactorFixed Effect Factor Driver (1, 2, 3, 4),Driver (1, 2, 3, 4),
Random Effects factorRandom Effects factor
The Model for the fixed effects The Model for the fixed effects experimentexperiment
where where , , 11, , 22, , 33, , 11, , 22, (, ())11 11 , (, ())21 21 , , (())31 31 , (, ())12 12 , (, ())22 22 , (, ())32 32 , are fixed , are fixed unknown constants unknown constants
And And ijk ijk is random, normally distributed with is random, normally distributed with mean 0 and variance mean 0 and variance 22..
Note:Note:
ijkijjiijky
01111
b
jij
a
iij
n
jj
a
ii
The Model for the case when factor The Model for the case when factor B B is is a random effects factora random effects factor
where where , , 11, , 22, , 33, are fixed unknown constants , are fixed unknown constants
And And ijk ijk is random, normally distributed with mean is random, normally distributed with mean 0 and variance 0 and variance 22..
jj is normal with mean 0 and variance is normal with mean 0 and varianceand and
(())ijij is normal with mean 0 and variance is normal with mean 0 and varianceNote:Note:
ijkijjiijky
01
a
ii
2B
2AB
This model is called a variance components model
1.1. The The EMS EMS for Errorfor Error isis 22..2.2. The The EMS EMS for each ANOVA term contains for each ANOVA term contains
two or more terms the first of which istwo or more terms the first of which is 22..
3.3. All other terms in each All other terms in each EMS EMS contain contain both coefficients and subscripts (the both coefficients and subscripts (the total number of letters being one more total number of letters being one more than the number of factors) than the number of factors)
4. The subscript of 22 in the last term of in the last term of each each EMS EMS is the same as the treatment is the same as the treatment designationdesignation..
Rules for determining Expected Mean Squares (EMS) in an Anova Table
EXPECTED MEAN SQUARES
E(MS) expected average value for a mean square computed in an ANOVA based on sampling theory
Two conditions: null hypothesis E(MS) and alternative hypothesis E(MS) null hypothesis condition gives us the
basis to test the alternative hypothesis contribution (effect of factor or interaction)
EXPECTED MEAN SQUARES
→→1 Factor design:1 Factor design:Source E(MS)Source E(MS)
Treatment A Treatment A 22ee + n + n22
AA
errorerror 22ee (sampling (sampling
variation)variation)
Thus F=MS(A)/MS(e) tests to see if Thus F=MS(A)/MS(e) tests to see if Treatment A adds variation to what might Treatment A adds variation to what might be expected from usual sampling variability be expected from usual sampling variability of subjects. If the F is largeof subjects. If the F is large, , 22
A A 0. 0.
EXPECTED MEAN SQUARESEXPECTED MEAN SQUARES
→ → Factorial design (AxB):Factorial design (AxB):
SourceSource E(MS)E(MS)
Treatment A Treatment A 22ee + (1-b/B)n + (1-b/B)n22
ABAB + nb + nb22AA
errorerror 22ee (sampling variation) (sampling variation)
Thus F=MS(A)/MS(e) does not test to see if Thus F=MS(A)/MS(e) does not test to see if Treatment A adds variation to what might be Treatment A adds variation to what might be expected from usual sampling variability of expected from usual sampling variability of subjects unless b=B orsubjects unless b=B or 22
ABAB = 0 . = 0 .
If b (number of levels in study) = B (number in If b (number of levels in study) = B (number in the population, factor is FIXED; else RANDOMthe population, factor is FIXED; else RANDOM
EXPECTED MEAN SQUARES→→Factorial design (AxB):Factorial design (AxB):
SourceSource E(MS)E(MS)
Treatment A Treatment A 22ee + (1-b/B)n + (1-b/B)n22
ABAB + + nbnb22
AA
AxBAxB 22ee + (1-b/B)n + (1-b/B)n22
ABAB
errorerror 22ee (sampling variation) (sampling variation)
IfIf 22ABAB = 0 , = 0 , and B is random, then F = and B is random, then F =
MS(A) / MS(AB) gives the correct test of the MS(A) / MS(AB) gives the correct test of the A effect. A effect.
EXPECTED MEAN SQUARES→ → Factorial design (AxB):Factorial design (AxB):
SourceSource E(MS)E(MS)
Treatment A Treatment A 22ee + (1-b/B)n + (1-b/B)n22
ABAB + nb + nb22AA
ABAB 22ee + (1-b/B)n + (1-b/B)n22
ABAB
errorerror 22ee (sampling variation) (sampling variation)
If instead we test F = MS(AB)/MS(e) and it is non If instead we test F = MS(AB)/MS(e) and it is non significant, thensignificant, then 22
ABAB = 0 = 0 and it can be testedand it can be tested
F = MS(A) / MS(e)F = MS(A) / MS(e)
*** More power since df= a-1, *** More power since df= a-1,
df(error) instead of df = a-1, (a-1)*(b-1)df(error) instead of df = a-1, (a-1)*(b-1)
Source df Expected mean square
A I-1 2e + n2
AB + nJ2A
B J-1 2e + n2
AB + nI2B
AB (I-1)(J-1) 2e + n2
AB
error N-IJK 2e
Table 10.3: Expected mean square table for I x J random factorial design
Source df Expected mean square
A (fixed) I-1 2e + n 2
AB + nJ 2A
B (random) J-1 2e + nI 2
B
AB (I-1)(J-1) 2e + n 2
AB
error N-IJK 2e
Table 10.5: Expected mean square table for I x J mixed model factorial design
Mixed and Random Design Tests
General principle: look for denominator General principle: look for denominator E(MS) with same form as numerator E(MS) with same form as numerator E(MS) without the effect of interest:E(MS) without the effect of interest: F = F = 22
effecteffect + other variances /other + other variances /other variancesvariances
Try to eliminate interactions not Try to eliminate interactions not important to the study, test with important to the study, test with MS(error) if possibleMS(error) if possible
The Anova table for the two factor The Anova table for the two factor modelmodel
ijkijjiijky
SourcSourcee
SSSS dfdf MSMS
AA SSSSAAa a -1-1 SSSSAA/(/(aa – 1) – 1)
BB SSSSAAbb - 1 - 1 SSSSBB/(/(aa – 1) – 1)
ABAB SSSSABAB((a a -1)(-1)(b b --
1)1)SSSSABAB/(/(aa – 1) ( – 1) (aa – 1) – 1)
ErrorError SSSSErroErro
rr
abab((nn – – 1)1)
SSSSErrorError//abab((nn – 1) – 1)
The Anova table for the two factor The Anova table for the two factor model (A, B – fixed) model (A, B – fixed)
ijkijjiijky
SourcSourcee
SSSS dfdf MSMS EMSEMS FF
AA SSSSAA a a -1-1 MSMSAA MSMSAA/MS/MSErrorError
BB SSSSAA bb - 1 - 1 MSMSBB MSMSBB/MS/MSErrorError
ABAB SSSSABAB ((a a -1)(-1)(b b --1)1)
MSMSABAB MSMSABAB//MSMSErrorError
ErrorError SSSSErrorError abab((nn – 1) – 1) MSMSErrorError
2
a
iia
nb
1
22
1
b
jjb
na
1
22
1
a
i
b
jijba
n
1 1
22
11
EMS = Expected Mean Square
The Anova table for the two factor The Anova table for the two factor model model
(A – fixed, B - random) (A – fixed, B - random) ijkijjiijky
SourcSourcee
SSSS DfDf MSMS EMSEMS FF
AA SSSSAA a a -1-1 MSMSAA MSMSAA/MS/MSABAB
BB SSSSAA bb - 1 - 1 MSMSBB MSMSBB/MS/MSErrorError
ABAB SSSSABAB ((a a -1)(-1)(b b --1)1)
MSMSABAB MSMSABAB//MSMSErrorError
ErrorError SSSSErrorError abab((nn – 1) – 1) MSMSErrorError 2
a
iiAB a
nbn
1
222
1
22Bna
22ABn
Note:
The divisor for testing the main effects of A is no longer MSError but MSAB.
Factor CFactor B
Factor A
Factor A
10109988776655443322110 xxxxxxxxxxyijk
Factor B Factor C
Three-way factorial Three-way factorial designdesign
Factor A
42203219101189117811671156114511341123111 xxxxxxxxxxxxxxxxxxxx 10 Main effects
31 Two-way interactions
107272972718727094145841441031439314283141 xxxxxxxxxxxxxxxxxxxxxxxx
30 Three-way interactions
Three Factor:Three Factor: A A ((a a levels), levels), B B ((b b levels), levels), C C ((c c levels)levels)
yyijklijkl = = + + ii ++ jj++ ijij ++ kk ++ (())ikik ++
(())jkjk++ ijkijk ++ ijklijkl
= = ++ ii ++ jj++ kk ++ ijij ++ ( (ikik ++ ( (jkjk
++ ijkijk ++ ijklijkl
ii = 1,2, ... , = 1,2, ... ,aa ; ; jj = 1,2, ... , = 1,2, ... ,bb ; ; kk = 1,2, ... , = 1,2, ... ,cc; ; ll = = 1,2, ... ,1,2, ... ,rr 0,,0,0,0,0
11111
c
kijk
a
iij
c
kk
b
jj
a
ii
Main effects Two factor Interactions
Three factor InteractionRandom error
ijkijk = the mean of = the mean of y y when when A A = = ii, , B B = = jj, , C C = = kk
= = ++ ii ++ jj++ kk ++ ijij ++ ( (ikik ++
((jkjk
++ ijkijk
ii = 1,2, ... , = 1,2, ... ,aa ; ; jj = 1,2, ... , = 1,2, ... ,bb ; ; kk = 1,2, ... , = 1,2, ... ,cc; ; ll = = 1,2, ... ,1,2, ... ,rr
0,,0,0,0,011111
c
kijk
a
iij
c
kk
b
jj
a
ii
Main effects Two factor Interactions
Three factor Interaction
Overall mean
Anova tablefor 3 factor Experiment
SourceSource SSSS dfdf MSMS FF p p -value-value
AA SSSSAA a - 1a - 1 MSMSAA MSMSAA/MS/MSErrorError
BB SSSSBB b - 1b - 1 MSMSBB MSMSBB/MS/MSErrorError
CC SSSSCC c - 1c - 1 MSMSCC MSMSCC/MS/MSErrorError
ABAB SSSSABAB ((a - 1a - 1)()(b - 1b - 1)) MSMSABAB MSMSABAB//MSMSErrorError
ACAC SSSSACAC ((a - 1a - 1)()(c - 1c - 1)) MSMSACAC MSMSACAC//MSMSErrorError
BCBC SSSSBCBC ((b - 1b - 1)()(c - 1c - 1)) MSMSBCBC MSMSBCBC//MSMSErrorError
ABCABC SSSSABCABC ((a - 1a - 1)()(b - 1b - 1)()(c c - 1- 1))
MSMSABCABC MSMSABCABC//MSMSErrorError
ErrorError SSSSErrorError abc(r - 1)abc(r - 1) MSMSErroErro
rr
Sum of squares entries
a
ii
a
iiA yyrbcrbcSS
1
2
1
2̂
Similar expressions for SSB , and SSC.
a
i
b
jjiij
a
iijAB yyyyrcrcSS
1 1
2
1
2
Similar expressions for SSBC , and SSAC.
Sum of squares entries
Finally
a
iikjABC rSS
1
2
a
i
b
j
c
kijkkiijijk yyyyyr
1 1 1 2 ikj yyy
a
i
b
j
c
k
r
lijkijklError yySS
1 1 1 1
2
a
iiA rbcSS
1
2̂
b
jjB racSS
1
2̂
a
i
b
jijAB rcSS
1 1
2
a
i
c
kikAC rbSS
1 1
2
b
j
c
kjkBC raSS
1 1
2
a
i
b
j
c
kijkABC rSS
1 1 1
2
a
i
b
j
c
k
r
lijkijklError yySS
1 1 1 1
2
Analysis of Variance (ANOVA) Table Entries ( 3 factors – A, B and C)
c
kkC rabSS
1
2̂
The statistical model for 3 factor Experiment
effectsmain effectmean kjiijk/y
error randomninteractiofactor 3nsinteractiofactor 2
ijk/ijkjkikij
Source Degrees of freedom
Factor A Factor B Factor CInteractions between A and BInteractions between A and CInteractions between B and CInteractions between A, B and C Residuals
a-1 = 2 b – 1 = 5c-1 = 3
(a-1)(b-1) = 10(a-1)(c-1) = 6
(b-1)(c-1) = 15(a-1)(b-1)(c-1) = 30
abc( r -1) = 72
Total n = rabc - 1= 143
Three-way factorial Three-way factorial designdesign
Four Factorial Design
Four Factor:Four Factor:
yyijklmijklm = = + + + + jj+ + (())ijij + + kk + + (())ikik + + (())jkjk+ + (())ijkijk + + ll+ + (())ilil + + (())jljl+ + (())ijlijl + + (())klkl + + (())iklikl + + (())jkljkl+ + (())ijklijkl + + ijklmijklm
= =
++ii + + jj+ + kk + + l l
+ (+ ())ijij + + (())ikik + + (())jkjk + + (())ilil + + (())jljl+ + (())klkl
++(())ijkijk+ + (())ijlijl + + (())iklikl + + (())jkljkl
+ (+ ())ijklijkl + + ijklmijklm
i i = 1,2, ... ,= 1,2, ... ,aa ; ; jj = 1,2, ... , = 1,2, ... ,bb ; ; kk = 1,2, ... , = 1,2, ... ,cc; ; ll = 1,2, ... , = 1,2, ... ,dd; ; mm = 1,2, ... , = 1,2, ... ,rr
wherewhere 0 = 0 = ii = = jj= = ( ())ijij kk = = (())ikik = = (())jkjk= = ( ())ijkijk = = ll= = (())ilil = = ( ())jljl = = ( ())ijlijl = = ( ())klkl = = ( ())iklikl = = ( ())jkljkl = =
(())ijklijkl
and and denotes the summation over any of the subscripts. denotes the summation over any of the subscripts.
Main effects Two factor Interactions
Three factor Interactions
Overall mean
Four factor Interaction Random error
Estimation of Main Effects and Estimation of Main Effects and
InteractionsInteractions Estimator of Main effect of a FactorEstimator of Main effect of a Factor
• Estimator of k-factor interaction effect at a combination of levels of the k factors
= Mean at level i of the factor - Overall Mean
Example:Example:
The The main effectmain effect of factor B at level j in a four of factor B at level j in a four factor (A,B,C and D) experiment is estimated by:factor (A,B,C and D) experiment is estimated by:
• The two-factor interaction effect between factors B and C when B is at level j and C is at level k is estimated by:
yyˆjj
yyyy kjjkjk
The The three-factor interactionthree-factor interaction effect between effect between factors B, C and D when B is at level j, C is at factors B, C and D when B is at level j, C is at level k and D is at level l is estimated by:level k and D is at level l is estimated by:
• Finally the four-factor interaction effect between factors A,B, C and when A is at level i, B is at level j, C is at level k and D is at level l is estimated by:
yyyyyyyy lkjklljjkjkljkl
jklikiijjklklilijijkijklijkl yyyyyyyyy
yyyyyyy lkjikllj
The Completely Randomized Design is called balanced
If the number of observations per treatment combination is unequal the design is called unbalanced.
If for some of the treatment combinations there are no
observations the design is called incomplete.
Remarks
Why should more than two levels of a factor be used in a
factorial design?
Two-levels of a factor
10 15 20 25 30
Temperature (oC)
0
5
10
15
20
25
30
10 15 20 25 30
Temperature (oC)
0
5
10
15
20
25
30
Three-levelsfactor qualitativefactor qualitative
22110 xxy
1
Low Medium High
0
2
10 15 20 25 30
Temperature (oC)
0
5
10
15
20
25
30
Three-levelsThree-levelsfactor quantitativefactor quantitative
2210 xxy
Why should not many levels of each factor be
used in a factorial design?
Because each level of each factor increases the number of
experimental units to be used
For example, a five factor experiment with four levels per factor yields 45 = 1024 different
combinations
If not all combinations are applied in an experiment, the design is
partially factorial
Full and fractional Full and fractional factorial designfactorial design
Full factorial designFull factorial design Study all combinationsStudy all combinations Can find effect of all factorsCan find effect of all factors
Fractional (incomplete) factorial Fractional (incomplete) factorial designdesign Leave some treatment groups emptyLeave some treatment groups empty Less informationLess information May not get all interactionsMay not get all interactions No problem if interaction is negligibleNo problem if interaction is negligible
Fractional factorial Fractional factorial designdesign
Large number of factorsLarge number of factors Large number of experimentsLarge number of experiments Full factorial design too expensiveFull factorial design too expensive Use a fractional factorial designUse a fractional factorial design
22k-pk-p design allows analyzing k factors design allows analyzing k factors with only 2with only 2k-pk-pexperiments.experiments. 22k-1k-1 design requires only half as many design requires only half as many
experimentsexperiments 22k-2k-2 design requires only one quarter of design requires only one quarter of
the experimentsthe experiments