two-way anova (two-factor crd)

Two-Way ANOVA(Two-Factor CRD)

STAT:5201

Week 5: Lecture 2

1 / 29

Factorial Treatment Structure

A factorial treatment structure is simply the case where treatmentsare created by combining factors.

We can generically refer to the factors with letters like A,B,C , etc.and the number of levels in are a, b, c , etc.

A two-factor factorial has g = ab treatments, a three-factor factorialhas g = abc treatments and so forth.

We have a completely randomized design with N total number ofexperiment units.

As mentioned earlier, we can think of factorials as a 1-way ANOVAwith a single ‘superfactor’ (levels as the treatments), but in mostcases, it is beneficial to consider the factorial nature of the design.

2 / 29

Two-Way ANOVA (Factorial): Balanced Design

Two factors: A with a levels, and B with b levels.

g = a× b treatments altogether, where the treatments are thecombinations of the levels of the two factors.

Completely randomized design with treatments randomly assigned tothe g treatments. No blocking. No nesting.

Example (Factors of DayLength and Climate with a = b = 2)

Four treatments arising from the combination of the two factors g = 4.Treatments randomly assigned to hamsters, with two hamsters in eachtreatment cell or ni = 4 for all i , and N = 8.

Climatecold warm

DayLength longshort

3 / 29

Two-Way ANOVA: Balanced Design

Full Model (includes interaction):

Yijk = µ+ αi + βj + (αβ)ij + εijk with εijkiid∼ N(0, σ2)

for i = 1, . . . , a and j = 1, . . . , b and k = 1, . . . , n

Total number of observations N = nab.

As with the 1-way ANOVA effects model, we have anoverparameterization (we have 1 + a + b + ab parameters for themean structure, but only ab distinct groups), so we need constraintsto make the parameters interpretable.

Sum-to-zero restrictions on parameters:

0 =a∑

i=1

αi =b∑

j=1

βj =a∑

i=1

(αβ)ij =b∑

j=1

(αβ)ij

4 / 29


Full Model (includes interaction):


for i = 1, . . . , a and j = 1, . . . , b and k = 1, . . . , n

Parameters αi , βj and (αβ)ij are fixed, unknown constants.

The sum-to-zero restrictions give µ as the overall mean, or theexpected value of response averaged across all treatments.

The term αi is called the main effect of A at level i .

The term βj is called the main effect of B at level j .

The term (αβ)ij is the interaction effect of A and B in the ijtreatment. The interaction effect is a measure of how far thetreatment mean differs from the additive model. If all (αβ)ij = 0,then we are reduced to the additive model.

5 / 29


Example (Animal-fattening experiment)

Two primary factors: Vitamin B12 (0mg, 5mg) & Antibiotics (0mg, 40mg)

Three animals randomly assigned to each of 4 treatments.Response was weight gain in lbs/week.

Because we can not assume that the two factors do not interact, weshould include interaction in the model.

G.W. Cobb (1998). Introduction to Design & Analysis of Experiments.6 / 29


Example (Animal Fattening example)

2 I/o (j)

!l maid 4d ind~ ~ddrc C 5. Z OM )

~ i3f< '" 0 "b!

#~ (r~{C;OrnCl-)An fl'bld?'c

W(,&hi- )~GU/\. C Ibs/Uld, .30( . I ~I . Do/.0,:)it o 0l . 0 r-I I 2lo

I . 2-1I .. ( ~[. 52-(. SlP[. <)''1

I.)/In J-( 0)d; co 1-0

\J,' ~Jr1(~ 0 f I. I 9 L D 3Gt z 5 (. '2. 2 I.S-f

'-,."J

~/~ 8(L ~;:b'~h(J 0 02 0 03 0 0Lj 0 <.foS- D yofo 0 40":J- '5 oY S- O~ S-(o 0

5" 40l~ C;; L{D/2- -) L(O

f3 =)~12... a:

We- wdI!/ ~ /J1~ w;/tz ~d~)4/s 0 bpz tfhm M ~ {JrJ;/ pJJ ~ 1\

We will fit a model that includes interaction, also known as the‘full model’.

7 / 29

Two-Way ANOVA: Parameter Estimates

Our full-model fitted values: Yijk = µ+ αi + βj + ˆ(αβ)ij

Of course all observations in the same cell have the same fitted value.

Our estimates for µ, αi , and βj under sum-to-zero- constraints aresimilar to what we saw in the additive two-factor CRD...

µ = Y... Overall meanαi = Yi .. − Y... main effects of factor A level iβj = Y.j . − Y... main effects of factor B level j

To get our estimator for (αβ)ij , we should note that the full model(with interaction) allows for effects due to individual combinations ofA and B.

8 / 29

Two-Way ANOVA: Full-Model Fitted Values

The fitted values Yijk in the full model ARE the cell means(i.e. just the mean of all observations in the cell).


The cell means for the animal fattening example give us the fitted valuesfor the full model, or the two-way ANOVA with interaction:

Y11. = Y11k = 1.19 Y21. = Y21k = 1.22

Y12. = Y12k = 1.03 Y11. = Y22k = 1.54

for k = 1, 2, 3

9 / 29


The estimates for the interaction effects, or the ˆ(αβ)ij values,quantify how far the full-model fitted values are from theadditive-model fitted values (or reduced model fitted values).

Thus, we need the additive-model fitted values (we already know howto get these) and the full-model fitted values (just the cell means) toestimate the (αβ)ij values.

Additive model fitted values: Yijk = µ+ αi + βj

Later, when we test for an interaction effect, relatively large ˆ(αβ)ijvalues suggest there is an interaction present. If ˆ(αβ)ij values are allvery small, then no interaction is present, or the additive model issufficient.

10 / 29



Two primary factors: Vitamin B12 (0mg, 5mg) & Antibiotics (0mg, 40mg)

What are the additive-model fitted values? Yijk = µ+ αi + βj

l-D

11 / 29



The additive model (reduced model) finds the best fit (smallest SSE) suchthat the interaction plot has parallel lines.

l-D

The estimate for each ˆ(αβ)ij can be found by subtracting the additive fitfor cell ij from the full-model fit for that cell.

12 / 29



The estimate for each ˆ(αβ)ij can be found by subtracting the additive fit

for cell ij from the full model fit for that cell, or ˆ(αβ)ij=Yij ,full − Yij ,additive

S;m; l~ )r-.qJ(b t \ .::: l , (<1 -- I. 0 "7 - 0, I L-

Ao(P'2-1 ~ I, 2 2. ~ (. 3 Y -==-- - O. /2-AcrfoL 2- :::- i, ")1 -- I. C/2::: O. I L

-- v.. - v y. --y.111~ I' - . +

cJ I".. 'J «: ~Po

13 / 29


S;m; l~ )r-.qJ(b t \ .::: l , (<1 -- I. 0 "7 - 0, I L-


-- v.. - v y. --y.111~ I' - . +

cJ I".. 'J «: ~Po

14 / 29


S;m; l~ )r-.qJ(b t \ .::: l , (<1 -- I. 0 "7 - 0, I L-


-- v.. - v y. --y.111~ I' - . +

cJ I".. 'J «: ~Po

15 / 29



Use the parameter estimates to show that the full-model fitted values aretruly the same as the cell means.

3

Yes~ ~ ~. s~ a4LJ t, 1';, k- -=- Y~ CJl.fJ2 V>'U2a/Yl -3- 1/ •••

-r£- ~'~ ~/V~ S~/rt ~ ~ 6C2/~ JYz ~

(C S to?? - Iv - ?::c r: 0 ~" r:ie» I-r- I C !;'on 5 ,16 / 29

Two-Way ANOVA: Restrictions on Parameters

The estimators we’ve shown here are based on the sum-to-zerorestrictions.

The animal fattening experiment has 4 cell means, and only 4parameters are needed to describe the mean structure(i.e. µ1, µ2, µ3, µ4)

But the full effects model utilizes 9 parameters to describe the 4means: µ, α1, α2, β1, β2, (αβ)11, (αβ)12, (αβ)21, (αβ)22

So, we utilize restrictions to make the parameters uniquely estimable(and interpretable).

17 / 29

Two-Way ANOVA: Restrictions on Parameters

Sum-to-zero restrictions on parameters:

0 =a∑

i=1

αi =b∑

j=1

βj =a∑

i=1

(αβ)ij =b∑

j=1

(αβ)ij

SAS uses a different constraint or restriction, and this affects theinterpretation or the parameters (we will see this soon).

The other parameter to be estimated is σ2 in the full model:

Lito ©~ O~ jJtA/(~ :h k d~ cO a: L c:., It,/:uth"Lf,

_ "Z..D- 2-= fYlSE zz: '0[;;: . _:-£ z (Y~~ - Y:;L~)N -~~S ) N - (~~ )

/l 2D =:

0.62 qg

/\(j~. D. {)(PD2


For the full model with the animal fattening data,σ2 = SSE

12−4 = 0.0298 = 0.0036 and σ = 0.0602

18 / 29

Two-Way ANOVA: SS for Balanced Design

1~/1''')a.. I 2··. hv,J k '" Ai + G-f -' ~. +fy# )'" + ~I~I~ .. , b A 2

i d ~

AN'OIJIt f~ J" IwD-~ tR.{) (6.f-tt ld.r5J;t.Jf,Idr);f~SJ~/~ bJ~ dh/~ Lu~'II~ e~ ~

~~ d-f: 5S ;\II S E(MS)-

I I T It

'/,I

A

8 b -I

19 / 29


Oehlert provides the table with slightly different notation, using the effectsnotation:

180 Factorial Treatment Structure

Term Sum of Squares Degrees of Freedom

Aa∑

i=1

bn(αi)2 a − 1

Bb∑

j=1

an(βj)2 b − 1

ABa,b∑

i=1,j=1

n(αβij)2 (a − 1)(b − 1)

Errora,b,n∑

i=1,j=1,k=1

(yijk − yij•)2 ab(n − 1)

Totala,b,n∑

i=1,j=1,k=1

(yijk − y•••)2 abn − 1

Display 8.3: Sums of squares in a balanced two-way factorial.

from the treatment means. We follow exactly the same program for balancedfactorials, obtaining the formulae in Display 8.3.

The sums of squares must add up in various ways. For example

SST = SSA + SSB + SSAB + SSE .

Also recall that SSA, SSB, and SSAB must add up to the sum of squaresSS partitionsbetween treatments, when considering the experiment to have g = ab treat-ments, so that

a,b∑

i=1,j=1

n(yij• − y•••)2 = SSA + SSB + SSAB .

These identities can provide useful checks on ANOVA computations.We display the results of an ANOVA decomposition in an Analysis of

Variance table. As before, the ANOVA table has columns for source, degreesof freedom, sum of squares, mean square, and F. For the two-way factorial,Two-factor

ANOVA table the sources of variation are factor A, factor B, the AB interaction, and error,so the table looks like this:

20 / 29


Reminder of estimators, shown in Oehlert.8.5 Models with Parameters 177

µ = y•••αi = yi•• − µ = yi•• − y•••βj = y•j• − µ = y•j• − y•••

αβij = yij• − µ − αi − βj

= yij• − yi•• − y•j• + y•••

Display 8.2: Estimators for main effects andinteractions in a two-way factorial.

a − 1 of them can vary freely; there are a − 1 degrees of freedom for factorA. Similarly, the βj values must sum to 0, so at most b − 1 of them can varyfreely, giving b − 1 degrees of freedom for factor B. For the interaction, we Main-effect and

interactiondegrees offreedom

have ab effects, but they must add to 0 when summed over i or j. We canshow that this leads to (a − 1)(b − 1) degrees of freedom for the interaction.Note that the parameters obey the same restrictions as the corresponding con-trasts: main-effects contrasts and effects add to zero across the subscript, andinteraction contrasts and effects add to zero across rows or columns.

When we add the degrees of freedom for A, B, and AB, we get a − 1+ b − 1 + (a − 1)(b − 1) = ab − 1 = g − 1. That is, the ab − 1 degrees Main effects and

interactionspartition between

treatmentsvariability

of freedom between the means of the ab factor level combinations have beenpartitioned into three sets: A, B, and the AB interaction. Within each factor-level combination there are n − 1 degrees of freedom about the treatmentmean. The error degrees of freedom are N − g = N − ab = (n − 1)ab,exactly as we would get ignoring factorial structure.

The Lynch and Strain data had a three by two factorial structure withn = 5. Thus there are 2 degrees of freedom for factor A, 1 degree of freedomfor factor B, 2 degrees of freedom for the AB interaction, and 24 degrees offreedom for error.

Display 8.2 gives the formulae for estimating the effects in a two-wayfactorial. Estimate µ by the mean of all the data y•••. Estimate µ + αi bythe mean of all responses that had treatment A at level i, yi••. To get anestimate of αi itself, subtract our estimate of µ from our estimate of µ + αi. Estimating

factorial effectsDo similarly for factor B, using y•j• as an estimate of µ+βj . We can extendthis basic idea to estimate the interaction terms αβij . The expected value intreatment ij is µ+αi+βj+αβij , which we can estimate by yij•, the observedtreatment mean. To get an estimate of αβij , simply subtract the estimates of

21 / 29

Two-Way ANOVA: SAS Example


STAT:5201 Applied Statistic II +r6YJt ett( 1(' tryfiI( -h'{O/fU eTwo-Factor Experiment

Two Primary Factors: Vitamin B12 (Omgand 5 mg) and Antibioitic (Omg and 40 mg)Response: Weight gain in lbs/week

/*Fit the full model (with interaction)*/proc glm data=anfat plot=diagnostics;

class animal antibiotic vitamin;model gain=vitamin antibiotic vitamin*antibiotic;output out=diagnostics p=predicted r=residual;

Three animals are randomly assigned to each of the 4 treatments as a completely randomized design /

(CRD). Ani, b,ot/L

~rnSAS Program Part 1:

run;

SAS Output from Part 1:

The GLM ProcedureDependent Variable: gain

R-Square0.934046

Coeff Var4.835982

/

SumofDF Squares

3 0.410700008 0.0290000011 0.43970000

~Root MSE~ gain Mean0.060208 1.245000

Mean Square F Value Pr > F

ModelErrorCorrected Total

0.136900000.00362500

37.77 <.0001

" ~I

(}1t~~

F--6.4lSource

Source DF Type I SS Mean Square F Value Pr > Fvitamin 1 0.21870000 0.21870000 60.33 <.0001antibiotic 1 0.01920000 0.01920000 5.30 0.0504vitamin*antibiotic 1 0.17280000 0.17280000 47.67 0.0001

\~;:- DF Type III SS Mean Square F Value Pr > F

\ vitamin 1 0.21870000 0.21870000 60.33 <.0001

antibiotic 1 0.01920000 0.01920000 5.30 0.0504,vitami.n=arrtabfotLc 1 0.17280000 0.17280000 47.67 0.0001! - -- ------ ~---~---- ~-~..-~----~---- ANDVA ·kb/e

1 2 - -fuG-lvr-s wiftt Wt Ie (tL-it5f1-

22 / 29



STAT:5201 Applied Statistic II +r6YJt ett( 1(' tryfiI( -h'{O/fU eTwo-Factor Experiment

Two Primary Factors: Vitamin B12 (Omgand 5 mg) and Antibioitic (Omg and 40 mg)Response: Weight gain in lbs/week

/*Fit the full model (with interaction)*/proc glm data=anfat plot=diagnostics;

class animal antibiotic vitamin;model gain=vitamin antibiotic vitamin*antibiotic;output out=diagnostics p=predicted r=residual;

Three animals are randomly assigned to each of the 4 treatments as a completely randomized design /

(CRD). Ani, b,ot/L

~rnSAS Program Part 1:

run;

SAS Output from Part 1:

The GLM ProcedureDependent Variable: gain

R-Square0.934046

Coeff Var4.835982

/

SumofDF Squares

3 0.410700008 0.0290000011 0.43970000

~Root MSE~ gain Mean0.060208 1.245000

Mean Square F Value Pr > F

ModelErrorCorrected Total

0.136900000.00362500

37.77 <.0001

" ~I

(}1t~~

F--6.4lSource

Source DF Type I SS Mean Square F Value Pr > Fvitamin 1 0.21870000 0.21870000 60.33 <.0001antibiotic 1 0.01920000 0.01920000 5.30 0.0504vitamin*antibiotic 1 0.17280000 0.17280000 47.67 0.0001

\~;:- DF Type III SS Mean Square F Value Pr > F

\ vitamin 1 0.21870000 0.21870000 60.33 <.0001

antibiotic 1 0.01920000 0.01920000 5.30 0.0504,vitami.n=arrtabfotLc 1 0.17280000 0.17280000 47.67 0.0001! - -- ------ ~---~---- ~-~..-~----~---- ANDVA ·kb/e

1 2 - -fuG-lvr-s wiftt Wt Ie (tL-it5f1-

We start by testing the highest-order interaction term, if thatinteraction is not significant, then we move to lower-order terms.

Here, the interaction is significant (p=0.0001), so we will NOTconsider the tests for main effects. The hierarchy principle says theymust now remain in the model regardless of their significance.

Remember to check all assumptions for the model using diagnostics.

23 / 29



See handout on PROC GLM, two-way ANOVA

24 / 29

Two-Way ANOVA: Overparameterization


Writing the factor effects model (a = 2, b = 2, n = 2) as a linearmodel Y = Xβ + ε

Y =

1 1 0 1 0 1 0 0 01 1 0 1 0 1 0 0 01 1 0 0 1 0 1 0 01 1 0 0 1 0 1 0 01 0 1 1 0 0 0 1 01 0 1 1 0 0 0 1 01 0 1 0 1 0 0 0 11 0 1 0 1 0 0 0 1

µα1

α2

β1β2

(αβ)11(αβ)12(αβ)21(αβ)22

+

ε111ε112ε121ε122ε211ε212ε221ε222

But the model is overparametrized and the X matrix is not of fullrank. The OLS estimates could be found by using a generalizedinversed: β = (X ′X )−XY , but we can also fix this by imposingrestrictions on the parameters.

25 / 29

Two-Way ANOVA: Re-written X design matrix

Using sum-to-zero restrictions and re-writing as a full rank designmatrix:

Y =

1 1 1 11 1 1 11 1 −1 −11 1 −1 −11 −1 1 −11 −1 1 −11 −1 −1 11 −1 −1 1

µα1

β1(αβ)11

+

ε111ε112ε121ε122ε211ε212ε221ε222

Other parameters determined by model restrictions:α2 = −α1, β2 = −β1, (αβ)12 = (αβ)21 = −(αβ)11,(αβ)22 = −(αβ)21

26 / 29

Two-Way ANOVA: the Cell Means version of the model

Though not commonly used, on occasion we may write the two-wayANOVA model using the cell means model notation:

Yijk = µij + εijk with εijkiid∼ N(0, σ2)

where i = 1, . . . , a; j = 1, . . . b, k = 1 . . . , nij

- µij is the mean of all units given level i of factor A and level j offactor B.

- µi . =∑

j µijb is the mean response at level i of factor A.

- µ.j =∑

i µija is the mean response at level j of factor B.

- µ.. =∑

i

∑j µij

ab is the overall mean response.

27 / 29


Example (Drill Speed and Feed Rate)

A mechanical engineer is studying the thrust force developed by a drillpress. Two primary factors are investigated; Drill Speed (125, 200) andFeed Rate (0.02, 0.03, 0.05, 0.06). Two runs will be performed at eachcombination performed in random order.

2-/23 @,wo-fodoc- /Vl# {W;1fJ,.~~~) ('~S

~f'U>1-U: rlJUUL ~ 7 4. divU /1.uY.

A IN.MM1I~ ~/~ ;; J~fi7 Ik ~JOLU- ~ 'c a- ~ I'/'~

F~rdDrii{s~ tun 6. ()'5 . - o. o~ D. 0(,

/2- C; z. 7-0 2. 'I') I 2.wO 1- 2. "152.n I 2, f<J 2, ?- z. I 2, '8"~

1'----- ,__-+ "..,-_" ..--,. 0.' " .. -,.",'" -,------ "--'''''-'--'''~'~''''-''-- -.

I200 2. g-~ I 2.3'') 2. "Z? 2 .11'"_---____ 2, Y/.p I 2, 70 2 - ?7- 2 ' 77

b.f. iJ1ntl;~ {UJt>~} D~~ l<~~ IY{ I:~/~. W,'ld :04.

rl/~~~:v..

I, ==-~-f ~ -r$ + (crl)·. rEI'} Lu;f/. ~!t~/o,tfl)

I IJ K- I d jJ o 0 /

i=t "J -_I U L==! z( ) L J - ) "'J r K /

12 ct0~ "d;tJYt S\~ l~ cell

t{ZD

D.C. Montgomery (2005). Design and Analysis of Experiments. Wiley :USA

28 / 29


Example (Drill Speed and Feed Rate)

Thought of as cell means using cell means notation below:

/2) e'lhO+~ 0+q 5 ce (/ yn.eQA'l S _

2) toPuf4c'- ~ ~ .OZ VS. F~ ~ . o 3 ((l.~qtH?s):A..()~e:;32) - t:~~~~)= A -;5:

3) {tfn<t"'---'Dr;U s;uJ /2;'-,,5, z.o» (A//~ F~ 0.62..

A ,,~ c: ~ J +ff(;)) (-1 /l /\ I'- /\) )'f ~ ~((- ~ I = (' -I- ,+ ( II - A -I- ~ -r4, -rt.......cr# i,

--

And the design matrix for X in the cell means model is again of fullrank and no restrictions are needed for parameter estimation.

29 / 29

two-way anova (two-factor crd)

Documents