fixed+random

7/27/2019 Fixed+Random

1/19

- p. 1/19

Statistics 203: Introduction to Regressionand Analysis of Variance

Fixed vs. Random Effects

Jonathan Taylor


2/19

qTodays class

qTwo-way ANOVA

qRandom vs. fixed effects

qWhen to use random effects?

qExample: sodium content in

beer

qOne-way random effects

model

q Implications for model

qOne-way random ANOVA

table

q Inference for

qEstimating 2qExample: productivity study

qTwo-way random effects

model

qANOVA tables: Two-way

(random)

qMixed effects model

qTwo-way mixed effects model


(mixed)

qConfidence intervals forvariances

qSattherwaites procedure

- p. 2/19

Todays class

s Random effects.

s One-way random effects ANOVA.

s Two-way mixed & random effects ANOVA.

s Sattherwaites procedure.
http://www-stat.stanford.edu/~jtaylo/courses/stats203/R


3/19

qTodays class

qTwo-way ANOVA




beer


model



table

q Inference for



model


(random)




(mixed)



- p. 3/19

Two-way ANOVA

s Second generalization: more than one grouping variable.

s Two-way ANOVA model: observations:(Yijk), 1 i r, 1 j m, 1 k nij : r groups in fi rstgrouping variable, m groups ins second and nij samples in(i, j)-cell:

Yijk = + i + j + ()ij + ijk, ijk N(0, 2).

s Constraints:x

ri=1 i = 0

x

mj=1 j = 0

x

m

j=1

()ij = 0, 1 i r

x ri=1()ij = 0, 1 j m.


4/19

qTodays class

qTwo-way ANOVA




beer


model



table

q Inference for



model


(random)




(mixed)



- p. 4/19

Random vs. fixed effects

s In ANOVA examples we have seen so far, the categoricalvariables are well-defi ned categories: below average fi tness,long duration, etc.

s In some designs, the categorical variable is subject.

s Simplest example: repeated measures, where more thanone (identical) measurement is taken on the same individual.

s In this case, the group effect i is best thought of asrandom because we only sample a subset of the entirepopulation of subjects.


5/19

qTodays class

qTwo-way ANOVA




beer


model



table

q Inference for



model


(random)




(mixed)



- p. 5/19

When to use random effects?

s A group effect is random if we can think of the levels weobserve in that group to be samples from a larger population.

s Example: if collecting data from different medical centers,

center might be thought of as random.

s Example: if surveying students on different campuses,campus may be a random effect.


6/19

qTodays class

qTwo-way ANOVA




beer


model



table

q Inference for



model


(random)




(mixed)



- p. 6/19

Example: sodium content in beer

s How much sodium is there in North American beer? Howmuch does this vary by brand?

s Observations: for 6 brands of beer, researchers recorded the

sodium content of 8 12 ounce bottles.

s Questions of interest: what is the grand mean sodiumcontent? How much variability is there from brand to brand?

s Individuals in this case are brands, repeated measures arethe 8 bottles.


7/19

qTodays class

qTwo-way ANOVA




beer


model



table

q Inference for



model


(random)




(mixed)



- p. 7/19

One-way random effects model

s Suppose we take n identical measurements from r subjects.

s Yij + i + ij , 1 i r, 1 j n

s

ij

N(0,

2

), 1

i

r, 1

j

ns i N(0,

2), 1 i r.

s We might be interested in the population mean, : CIs, is itzero? etc.

s Alternatively, we might be interested in the variability acrosssubjects, 2: CIs, is it zero?


8/19

qTodays class

qTwo-way ANOVA




beer


model



table

q Inference for



model


(random)




(mixed)



- p. 8/19

Implications for model

s In random effects model, the observations are no longerindependent (even if s are independent). In fact

Cov(Yij , Yi

j

) = 2

i,i

+ 2

j,j

.s In more complicated mixed effects models, this makes MLE

more complicated: not only are there parameters in themean, but in the covariance as well.

s In ordinary least squares regression, the only parameter toestimate is 2 because the covariance matrix is 2I.


9/19

qTodays class

qTwo-way ANOVA




beer


model



table

q Inference for



model


(random)




(mixed)

q

Confidence intervals forvariances


- p. 9/19

One-way random ANOVA table

Source SS df E(M S)

Treatments SSTR =Pr

i=1 n

Yi Y

2r 1 2 + n2

Error SSE =Pr

i=1Pn

j=1(Yij Yi)2 (n 1)r 2

s Only change here is the expectation of SSTR which reflectsrandomness of is.

s ANOVA table is still useful to setup tests: the same Fstatistics for fi xed or random will work here.

s

Under H0 : 2

= 0, it is easy to see thatM S T R

M SE Fr1,(n1)r.


10/19

qTodays class

qTwo-way ANOVA




beer


model



table

q Inference for



model


(random)




(mixed)

qConfidence intervals for

variances


- p. 10/19

Inference for

s We know that E(Y) = , and can show that

Var(Y) =n2 +

2

rn.

s Therefore,

Y

SSTR(r1)rn tr1

s Why r 1 degrees of freedom? Imagine we could record aninfi nite number of observations for each individual, so thatYi i.

s To learn anything about we still only have r observations(1, . . . , r).

s Sampling more within an individual cannot narrow the CI for.


11/19

qTodays class

qTwo-way ANOVA




beer

qOne-way random effectsmodel



table

q Inference for



model


(random)




(mixed)


variances


- p. 11/19

Estimating 2

s From the ANOVA table

2 =E(SSTR/(r 1))E(SSE/((n 1)r))

n

.

s Natural estimate:

S2 =SSTR/(r 1) SSE/((n 1)r)

n

s Problem: this estimate can be negative! One of thediffi culties in random effects model.


12/19

qTodays class

qTwo-way ANOVA




beer




table

q Inference for



model


(random)




(mixed)


variances


- p. 12/19

Example: productivity study

s Imagine a study on the productivity of employees in a largemanufacturing company.

s Company wants to get an idea of daily productivity, and how

it depends on which machine an employee uses.s Study: take m employees and r machines, having each

employee work on each machine for a total of n days.

s As these employees are not allemployees, and thesemachines are not allmachines it makes sense to think ofboth the effects of machine and employees (andinteractions) as random.
http://www-stat.stanford.edu/~jtaylo/courses/stats203/Rhttp://www-stat.stanford.edu/~jtaylo/courses/stats203/R


13/19

qTodays class

qTwo-way ANOVA




beer




table

q Inference for


qTwo-way random effectsmodel


(random)




(mixed)


variances


- p. 13/19

Two-way random effects model

s Yijk + i + j + ()ij + ij , 1 i r, 1 j m, 1 k n

s ijk N(0, 2), 1 i r, 1 j m, 1 k n

s i N(0, 2), 1 i r.

s j N(0, 2), 1 j m.

s ()ij N(0, 2), 1 j m, 1 i r.

s Cov(Yijk, Yijk) = ii2+ jj2 +iijj2 +iijjkk2.


14/19

qTodays class

qTwo-way ANOVA




beer




table

q Inference for




(random)




(mixed)


variances


- p. 14/19

ANOVA tables: Two-way (random)

SS df E(SS)

SSA = nmPr

i=1

Yi Y

2r 1 2 + nm2 + n

2

SSB = nrPm

j=1

Yj Y

2m 1 2 + nr2

+ n2

SSAB = nP

ri=1P

mj=1

Yij Yi Yj + Y

2(m 1)(r 1) 2 + n2

SSE =P

ri=1

Pmj=1

Pnk=1(Yijk Yij)

2 (n 1)ab 2

s To test H0 : 2 = 0 use SSA and SSAB.

s To test H0 : 2 = 0 use SSAB and SSE.


15/19

qTodays class

qTwo-way ANOVA




beer




table

q Inference for




(random)




(mixed)


variances


- p. 15/19

Mixed effects model

s In some studies, some factors can be thought of as fi xed,others random.

s For instance, we might have a study of the effect of a

standard part of the brewing process on sodium levels in thebeer example.

s Then, we might think of a model in which we have a fi xedeffect for brewing technique and a random effect for beer.


16/19

qTodays class

qTwo-way ANOVA




beer




table

q Inference for




(random)




(mixed)


variances


- p. 16/19

Two-way mixed effects model

s Yijk + i + j + ()ij + ij , 1 i r, 1 j m, 1 k n

s ijk N(0, 2), 1 i r, 1 j m, 1 k n

s i N(0, 2), 1 i r.

s j , 1 j m are constants.

s ()ij N(0, (m 1)2/m), 1 j m, 1 i r.

s Constraints:x

mj=1 j = 0

x

ri=1()ij = 0, 1 i r.

x Cov (()ij , ()ij) = 2/m

s Cov(Yijk, Yijk) =

jj

2 + iim1m

2 (1 ii)

1m

2 + iikk

2


17/19

qTodays class

qTwo-way ANOVA




beer




table

q Inference for




(random)




(mixed)


variances


- p. 17/19

ANOVA tables: Two-way (mixed)

SS df E(M S)

SSA r 1 2 + nm2

SSB m 1 2 + nr

Pmj=1

2i

m1+ n2

SSAB (m 1)(r 1) 2 + n2

SSE =Pr

i=1Pm

j=1Pn

k=1(Yijk Yij)2 (n 1)ab 2

s To test H0 : 2 = 0 use SSA and SSE.

s To test H0 : 1 = = m = 0 use SSB and SSAB.

s To test H0

: 2

use SSAB and SSE.


18/19

S h i d


19/19

qTodays class

qTwo-way ANOVA




beer




table

q Inference for




(random)




(mixed)


variances


- p. 19/19

Sattherwaites procedure

s Given k independent M Ss

L k

i=1 ciM Sis Then

dfT

L

E(L) 2dfT .

where

dfT =

ki=1 ciM Si

2

ki=1 c2i M S2i /dfiwhere dfi are the degrees of freedom of the i-th M S.

s (1 ) 100% CI for E(L):LL =

dfT L2dfT;1/2

, LU =dfT L2dfT;/2

fixed+random

Documents