3. the f test for comparing reduced vs. full models · now back to determining the distribution of...
TRANSCRIPT
![Page 1: 3. The F Test for Comparing Reduced vs. Full Models · Now back to determining the distribution of F = y0(P X P X 0)y=[rank(X) rank(X 0)] y0(I P X)y=[n rank(X)]: An important first](https://reader034.vdocuments.net/reader034/viewer/2022052608/5ae459447f8b9a7b218e4bb3/html5/thumbnails/1.jpg)
3. The F Test for ComparingReduced vs. Full Models
Copyright c©2018 Dan Nettleton (Iowa State University) 3. Statistics 510 1 / 43
![Page 2: 3. The F Test for Comparing Reduced vs. Full Models · Now back to determining the distribution of F = y0(P X P X 0)y=[rank(X) rank(X 0)] y0(I P X)y=[n rank(X)]: An important first](https://reader034.vdocuments.net/reader034/viewer/2022052608/5ae459447f8b9a7b218e4bb3/html5/thumbnails/2.jpg)
Assume the Gauss-Markov Model with normal errors:
y = Xβ + ε, ε ∼ N(0, σ2I).
Suppose C(X0) ⊂ C(X) and we wish to test
H0 : E(y) ∈ C(X0) vs. HA : E(y) ∈ C(X) \ C(X0).
Copyright c©2018 Dan Nettleton (Iowa State University) 3. Statistics 510 2 / 43
![Page 3: 3. The F Test for Comparing Reduced vs. Full Models · Now back to determining the distribution of F = y0(P X P X 0)y=[rank(X) rank(X 0)] y0(I P X)y=[n rank(X)]: An important first](https://reader034.vdocuments.net/reader034/viewer/2022052608/5ae459447f8b9a7b218e4bb3/html5/thumbnails/3.jpg)
The “reduced” model corresponds to the null hypothesis andsays that E(y) ∈ C(X0), a specified subspace of C(X).
The “full” model says that E(y) can be anywhere in C(X).
For example, suppose
X0 =
111111
and X =
1 0 01 0 00 1 00 1 00 0 10 0 1
.
Copyright c©2018 Dan Nettleton (Iowa State University) 3. Statistics 510 3 / 43
![Page 4: 3. The F Test for Comparing Reduced vs. Full Models · Now back to determining the distribution of F = y0(P X P X 0)y=[rank(X) rank(X 0)] y0(I P X)y=[n rank(X)]: An important first](https://reader034.vdocuments.net/reader034/viewer/2022052608/5ae459447f8b9a7b218e4bb3/html5/thumbnails/4.jpg)
In this case, the reduced model says that all 6 observations havethe same mean.
The full model says that there are three groups of twoobservations. Within each group, observations have the samemean. The three group means may be different from oneanother.
Copyright c©2018 Dan Nettleton (Iowa State University) 3. Statistics 510 4 / 43
![Page 5: 3. The F Test for Comparing Reduced vs. Full Models · Now back to determining the distribution of F = y0(P X P X 0)y=[rank(X) rank(X 0)] y0(I P X)y=[n rank(X)]: An important first](https://reader034.vdocuments.net/reader034/viewer/2022052608/5ae459447f8b9a7b218e4bb3/html5/thumbnails/5.jpg)
For this example, let µ1, µ2, and µ3 be the elements of β in thefull model, i.e., β = [µ1, µ2, µ3]
′. Then, for the full model,
E(y) = Xβ =
1 0 01 0 00 1 00 1 00 0 10 0 1
µ1
µ2
µ3
=
µ1
µ1
µ2
µ2
µ3
µ3
, and
H0 : E(y) ∈ C(X0) vs. HA : E(y) ∈ C(X) \ C(X0)
is equivalent to
H0 : µ1 = µ2 = µ3 vs. HA : µi 6= µj, for some i 6= j.
Copyright c©2018 Dan Nettleton (Iowa State University) 3. Statistics 510 5 / 43
![Page 6: 3. The F Test for Comparing Reduced vs. Full Models · Now back to determining the distribution of F = y0(P X P X 0)y=[rank(X) rank(X 0)] y0(I P X)y=[n rank(X)]: An important first](https://reader034.vdocuments.net/reader034/viewer/2022052608/5ae459447f8b9a7b218e4bb3/html5/thumbnails/6.jpg)
For the general case, consider the test statistic
F =y′(PX − PX0)y/ [rank(X)− rank(X0)]
y′(I − PX)y/ [n− rank(X)].
To show that this statistic has an F distribution, we will use thefollowing fact:
PX0PX = PXPX0 = PX0 .
Copyright c©2018 Dan Nettleton (Iowa State University) 3. Statistics 510 6 / 43
![Page 7: 3. The F Test for Comparing Reduced vs. Full Models · Now back to determining the distribution of F = y0(P X P X 0)y=[rank(X) rank(X 0)] y0(I P X)y=[n rank(X)]: An important first](https://reader034.vdocuments.net/reader034/viewer/2022052608/5ae459447f8b9a7b218e4bb3/html5/thumbnails/7.jpg)
There are many ways to see that this fact is true. First,
C(X0) ⊂ C(X) =⇒ Each column of X0 ∈ C(X)
=⇒ PXX0 = X0.
Thus,
PXPX0 = PXX0(X′0X0)−X′0 = X0(X′0X0)
−X′0= PX0 .
This implies that
(PXPX0)′ = P′X0
=⇒ P′X0P′X = P′X0
=⇒ PX0PX = PX0 . 2
Copyright c©2018 Dan Nettleton (Iowa State University) 3. Statistics 510 7 / 43
![Page 8: 3. The F Test for Comparing Reduced vs. Full Models · Now back to determining the distribution of F = y0(P X P X 0)y=[rank(X) rank(X 0)] y0(I P X)y=[n rank(X)]: An important first](https://reader034.vdocuments.net/reader034/viewer/2022052608/5ae459447f8b9a7b218e4bb3/html5/thumbnails/8.jpg)
Alternatively,
∀ a ∈ Rn, PX0a ∈ C(X0) ⊂ C(X).
Thus, ∀ a ∈ Rn, PXPX0a = PX0a.
This implies PXPX0 = PX0.
Transposing both sides of this equality and using symmetry ofprojection matrices yields
PX0PX = PX0 . 2
Copyright c©2018 Dan Nettleton (Iowa State University) 3. Statistics 510 8 / 43
![Page 9: 3. The F Test for Comparing Reduced vs. Full Models · Now back to determining the distribution of F = y0(P X P X 0)y=[rank(X) rank(X 0)] y0(I P X)y=[n rank(X)]: An important first](https://reader034.vdocuments.net/reader034/viewer/2022052608/5ae459447f8b9a7b218e4bb3/html5/thumbnails/9.jpg)
Alternatively, C(X0) ⊂ C(X) =⇒ XB = X0 for some B becauseevery column of X0 must be in C(X).
Thus,
PX0PX = X0(X′0X0)−X′0PX = X0(X′0X0)
−(XB)′PX
= X0(X′0X0)−B′X′PX = X0(X′0X0)
−B′X′
= X0(X′0X0)−(XB)′ = X0(X′0X0)
−X′0 = PX0 .
PXPX0 = PXX0(X′0X0)−X′0 = PXXB(X′0X0)
−X′0
= XB(X′0X0)−X′0 = X0(X′0X0)
−X′0 = PX0 .
2Copyright c©2018 Dan Nettleton (Iowa State University) 3. Statistics 510 9 / 43
![Page 10: 3. The F Test for Comparing Reduced vs. Full Models · Now back to determining the distribution of F = y0(P X P X 0)y=[rank(X) rank(X 0)] y0(I P X)y=[n rank(X)]: An important first](https://reader034.vdocuments.net/reader034/viewer/2022052608/5ae459447f8b9a7b218e4bb3/html5/thumbnails/10.jpg)
Note that PX − PX0 is a symmetric and idempotent matrix:
(PX − PX0)′ = P′X − P′X0
= PX − PX0 .
(PX − PX0)(PX − PX0) = PXPX − PXPX0 − PX0PX + PX0PX0
= PX − PX0 − PX0 + PX0
= PX − PX0 .
Copyright c©2018 Dan Nettleton (Iowa State University) 3. Statistics 510 10 / 43
![Page 11: 3. The F Test for Comparing Reduced vs. Full Models · Now back to determining the distribution of F = y0(P X P X 0)y=[rank(X) rank(X 0)] y0(I P X)y=[n rank(X)]: An important first](https://reader034.vdocuments.net/reader034/viewer/2022052608/5ae459447f8b9a7b218e4bb3/html5/thumbnails/11.jpg)
Now back to determining the distribution of
F =y′(PX − PX0)y/ [rank(X)− rank(X0)]
y′(I − PX)y/ [n− rank(X)].
An important first step is to note that
F =y′(
PX−PX0σ2
)y/ [rank(X)− rank(X0)]
y′( I−PX
σ2
)y/ [n− rank(X)]
.
Now we can show that the numerator is a chi-square randomvariable divided by its degrees of freedom, independent of thedenominator, which is a central chi-square random variabledivided by its degrees of freedom. Once we show all thesethings, we will have established that the statistic F has an Fdistribution (see Slide Set 1 Slide 35).
Copyright c©2018 Dan Nettleton (Iowa State University) 3. Statistics 510 11 / 43
![Page 12: 3. The F Test for Comparing Reduced vs. Full Models · Now back to determining the distribution of F = y0(P X P X 0)y=[rank(X) rank(X 0)] y0(I P X)y=[n rank(X)]: An important first](https://reader034.vdocuments.net/reader034/viewer/2022052608/5ae459447f8b9a7b218e4bb3/html5/thumbnails/12.jpg)
We will use the following result from Slide Set 1.
Suppose Σ is an n× n positive definite matrix.
Suppose A is an n× n symmetric matrix of rank m such thatAΣ is idempotent (i.e., AΣAΣ = AΣ).
Then y ∼ N(µ,Σ) =⇒ y′Ay ∼ χ2m(µ
′Aµ/2).
For the numerator of our F statistic, we have
µ = Xβ, Σ = σ2I, A =
(PX − PX0
σ2
), and
Copyright c©2018 Dan Nettleton (Iowa State University) 3. Statistics 510 12 / 43
![Page 13: 3. The F Test for Comparing Reduced vs. Full Models · Now back to determining the distribution of F = y0(P X P X 0)y=[rank(X) rank(X 0)] y0(I P X)y=[n rank(X)]: An important first](https://reader034.vdocuments.net/reader034/viewer/2022052608/5ae459447f8b9a7b218e4bb3/html5/thumbnails/13.jpg)
m = rank(A) = rank(
PX − PX0
σ2
)= rank(PX − PX0)
= tr(PX − PX0) = tr(PX)− tr(PX0)
= rank(PX)− rank(PX0) = rank(X)− rank(X0).
(Multiplying by a nonzero constant does not affect the rank of amatrix. Rank is the same as trace for idempotent matrices.Trace of a difference is the same as the difference of traces. Therank of a projection matrix is equal to the rank of the matrixwhose column space is projected onto.)
Copyright c©2018 Dan Nettleton (Iowa State University) 3. Statistics 510 13 / 43
![Page 14: 3. The F Test for Comparing Reduced vs. Full Models · Now back to determining the distribution of F = y0(P X P X 0)y=[rank(X) rank(X 0)] y0(I P X)y=[n rank(X)]: An important first](https://reader034.vdocuments.net/reader034/viewer/2022052608/5ae459447f8b9a7b218e4bb3/html5/thumbnails/14.jpg)
To verify that Σ is positive definite, note that for any a ∈ Rn \ {0},
a′Σa = a′(σ2I)
a = σ2a′a = σ2n∑
i=1
a2i > 0.
To verify that AΣ is idempotent, we have
AΣ =
(PX − PX0
σ2
)(σ2I) = PX − PX0 .
Thus, we have
y′(
PX − PX0
σ2
)y ∼ χ2
rank(X)−rank(X0)
(12β′X′
(PX − PX0
σ2
)Xβ
).
Copyright c©2018 Dan Nettleton (Iowa State University) 3. Statistics 510 14 / 43
![Page 15: 3. The F Test for Comparing Reduced vs. Full Models · Now back to determining the distribution of F = y0(P X P X 0)y=[rank(X) rank(X 0)] y0(I P X)y=[n rank(X)]: An important first](https://reader034.vdocuments.net/reader034/viewer/2022052608/5ae459447f8b9a7b218e4bb3/html5/thumbnails/15.jpg)
Denominator Distribution
From the result on Slide Set 2 Slide 19, we have
y′(
I − PX
σ2
)y ∼ χ2
n−rank(X).
(This fact can be proved using the result on Slide Set 1 Slide 31,following the same type of argument used to show thedistribution of the numerator.)
Copyright c©2018 Dan Nettleton (Iowa State University) 3. Statistics 510 15 / 43
![Page 16: 3. The F Test for Comparing Reduced vs. Full Models · Now back to determining the distribution of F = y0(P X P X 0)y=[rank(X) rank(X 0)] y0(I P X)y=[n rank(X)]: An important first](https://reader034.vdocuments.net/reader034/viewer/2022052608/5ae459447f8b9a7b218e4bb3/html5/thumbnails/16.jpg)
Independence of Numerator and Denominator
By Slide Set 1 Slide 38,
y′(
PX−PX0σ2
)y is independent of y′
( I−PXσ2
)y because
(PX − PX0
σ2
)(σ2I)
(I − PX
σ2
)=
1σ2 (PX − PXPX − PX0 + PX0PX)
=1σ2 (PX − PX − PX0 + PX0) = 0.
Copyright c©2018 Dan Nettleton (Iowa State University) 3. Statistics 510 16 / 43
![Page 17: 3. The F Test for Comparing Reduced vs. Full Models · Now back to determining the distribution of F = y0(P X P X 0)y=[rank(X) rank(X 0)] y0(I P X)y=[n rank(X)]: An important first](https://reader034.vdocuments.net/reader034/viewer/2022052608/5ae459447f8b9a7b218e4bb3/html5/thumbnails/17.jpg)
Thus, it follows that
F =y′(PX − PX0)y/ [rank(X)− rank(X0)]
y′(I − PX)y/ [n− rank(X)]
∼ Frank(X)−rank(X0),n−rank(X)
(β′X′(PX − PX0)Xβ
2σ2
).
If H0 is true, i.e. if E(y) = Xβ ∈ C(X0), then the noncentralityparameter is 0 because
(PX − PX0)Xβ = PXXβ − PX0Xβ
= Xβ − Xβ = 0.
Copyright c©2018 Dan Nettleton (Iowa State University) 3. Statistics 510 17 / 43
![Page 18: 3. The F Test for Comparing Reduced vs. Full Models · Now back to determining the distribution of F = y0(P X P X 0)y=[rank(X) rank(X 0)] y0(I P X)y=[n rank(X)]: An important first](https://reader034.vdocuments.net/reader034/viewer/2022052608/5ae459447f8b9a7b218e4bb3/html5/thumbnails/18.jpg)
In general, the noncentrality parameter quantifies how far themean of y is from C(X0) because
β′X′(PX − PX0)Xβ
= β′X′(PX − PX0)′(PX − PX0)Xβ
= || (PX − PX0)Xβ ||2= || PXXβ − PX0Xβ ||2
= || Xβ − PX0Xβ ||2= || E(y)− PX0E(y) ||2 .
Copyright c©2018 Dan Nettleton (Iowa State University) 3. Statistics 510 18 / 43
![Page 19: 3. The F Test for Comparing Reduced vs. Full Models · Now back to determining the distribution of F = y0(P X P X 0)y=[rank(X) rank(X 0)] y0(I P X)y=[n rank(X)]: An important first](https://reader034.vdocuments.net/reader034/viewer/2022052608/5ae459447f8b9a7b218e4bb3/html5/thumbnails/19.jpg)
Note that
y′(PX − PX0)y = y′ [(I − PX0)− (I − PX)] y
= y′(I − PX0)y− y′(I − PX)y
= SSEREDUCED − SSEFULL.
Also rank(X)−rank(X0)
= [n− rank(X0)]− [n− rank(X)]
= DFEREDUCED − DFEFULL,
where DFE = Degrees of Freedom for Error.
Copyright c©2018 Dan Nettleton (Iowa State University) 3. Statistics 510 19 / 43
![Page 20: 3. The F Test for Comparing Reduced vs. Full Models · Now back to determining the distribution of F = y0(P X P X 0)y=[rank(X) rank(X 0)] y0(I P X)y=[n rank(X)]: An important first](https://reader034.vdocuments.net/reader034/viewer/2022052608/5ae459447f8b9a7b218e4bb3/html5/thumbnails/20.jpg)
Thus, the F statistic has the familiar form
(SSEREDUCED − SSEFULL)/(DFEREDUCED − DFEFULL)
SSEFULL/DFEFULL.
Copyright c©2018 Dan Nettleton (Iowa State University) 3. Statistics 510 20 / 43
![Page 21: 3. The F Test for Comparing Reduced vs. Full Models · Now back to determining the distribution of F = y0(P X P X 0)y=[rank(X) rank(X 0)] y0(I P X)y=[n rank(X)]: An important first](https://reader034.vdocuments.net/reader034/viewer/2022052608/5ae459447f8b9a7b218e4bb3/html5/thumbnails/21.jpg)
Equivalence of F Tests
It turns out that this reduced vs. full model F test is equivalent tothe F test for testing
H0 : Cβ = d vs. HA : Cβ 6= d
with an appropriately chosen C and d.
The equivalence of these tests is proved in STAT 611.
Copyright c©2018 Dan Nettleton (Iowa State University) 3. Statistics 510 21 / 43
![Page 22: 3. The F Test for Comparing Reduced vs. Full Models · Now back to determining the distribution of F = y0(P X P X 0)y=[rank(X) rank(X 0)] y0(I P X)y=[n rank(X)]: An important first](https://reader034.vdocuments.net/reader034/viewer/2022052608/5ae459447f8b9a7b218e4bb3/html5/thumbnails/22.jpg)
Example: F Test for Lack of Linear Fit
Suppose a balanced, completely randomized design is used toassign 1, 2, or 3 units of fertilizer to a total of 9 plots of land.
The yield harvested from each plot is recorded as the response.
Copyright c©2018 Dan Nettleton (Iowa State University) 3. Statistics 510 22 / 43
![Page 23: 3. The F Test for Comparing Reduced vs. Full Models · Now back to determining the distribution of F = y0(P X P X 0)y=[rank(X) rank(X 0)] y0(I P X)y=[n rank(X)]: An important first](https://reader034.vdocuments.net/reader034/viewer/2022052608/5ae459447f8b9a7b218e4bb3/html5/thumbnails/23.jpg)
Let yij denote the yield from the jth plot that received i units offertilizer (i, j = 1, 2, 3).
Suppose all yields are independent and yij ∼ N(µi, σ2) for all
i, j = 1, 2, 3.
If y =
y11
y12
y13
y21
y22
y23
y31
y32
y33
, then E(y) =
µ1
µ1
µ1
µ2
µ2
µ2
µ3
µ3
µ3
.
Copyright c©2018 Dan Nettleton (Iowa State University) 3. Statistics 510 23 / 43
![Page 24: 3. The F Test for Comparing Reduced vs. Full Models · Now back to determining the distribution of F = y0(P X P X 0)y=[rank(X) rank(X 0)] y0(I P X)y=[n rank(X)]: An important first](https://reader034.vdocuments.net/reader034/viewer/2022052608/5ae459447f8b9a7b218e4bb3/html5/thumbnails/24.jpg)
Suppose we wish to determine whether there is a linearrelationship between the amount of fertilizer applied to a plot andthe expected value of a plot’s yield.
In other words, suppose we wish to know if there exists realnumbers β1 and β2 such that
µi = β1 + β2(i) for all i = 1, 2, 3.
Copyright c©2018 Dan Nettleton (Iowa State University) 3. Statistics 510 24 / 43
![Page 25: 3. The F Test for Comparing Reduced vs. Full Models · Now back to determining the distribution of F = y0(P X P X 0)y=[rank(X) rank(X 0)] y0(I P X)y=[n rank(X)]: An important first](https://reader034.vdocuments.net/reader034/viewer/2022052608/5ae459447f8b9a7b218e4bb3/html5/thumbnails/25.jpg)
Consider testing H0 : E(y) ∈ C(X0) vs. HA : E(y) ∈ C(X) \ C(X0),where
X0 =
1 11 11 11 21 21 21 31 31 3
and X =
1 0 01 0 01 0 00 1 00 1 00 1 00 0 10 0 10 0 1
.
Copyright c©2018 Dan Nettleton (Iowa State University) 3. Statistics 510 25 / 43
![Page 26: 3. The F Test for Comparing Reduced vs. Full Models · Now back to determining the distribution of F = y0(P X P X 0)y=[rank(X) rank(X 0)] y0(I P X)y=[n rank(X)]: An important first](https://reader034.vdocuments.net/reader034/viewer/2022052608/5ae459447f8b9a7b218e4bb3/html5/thumbnails/26.jpg)
Note H0 : E(y) ∈ C(X0)⇐⇒ ∃ β ∈ R2 3 E(y) = X0β
⇐⇒ ∃[β1
β2
]∈ R2 3
µ1
µ1
µ1
µ2
µ2
µ2
µ3
µ3
µ3
=
1 11 11 11 21 21 21 31 31 3
[β1
β2
]=
β1 + β2(1)β1 + β2(1)β1 + β2(1)β1 + β2(2)β1 + β2(2)β1 + β2(2)β1 + β2(3)β1 + β2(3)β1 + β2(3)
⇐⇒ µi = β1 + β2(i) for all i = 1, 2, 3.
Copyright c©2018 Dan Nettleton (Iowa State University) 3. Statistics 510 26 / 43
![Page 27: 3. The F Test for Comparing Reduced vs. Full Models · Now back to determining the distribution of F = y0(P X P X 0)y=[rank(X) rank(X 0)] y0(I P X)y=[n rank(X)]: An important first](https://reader034.vdocuments.net/reader034/viewer/2022052608/5ae459447f8b9a7b218e4bb3/html5/thumbnails/27.jpg)
Note E(y) ∈ C(X)⇐⇒ ∃ β ∈ R3 3 E(y) = Xβ
⇐⇒ ∃
β1
β2
β3
∈ R3 3
µ1
µ1
µ1
µ2
µ2
µ2
µ3
µ3
µ3
=
1 0 01 0 01 0 00 1 00 1 00 1 00 0 10 0 10 0 1
β1
β2
β3
=
β1
β1
β1
β2
β2
β2
β3
β3
β3
.
This condition clearly holds with βi = µi for all i = 1, 2, 3.
Copyright c©2018 Dan Nettleton (Iowa State University) 3. Statistics 510 27 / 43
![Page 28: 3. The F Test for Comparing Reduced vs. Full Models · Now back to determining the distribution of F = y0(P X P X 0)y=[rank(X) rank(X 0)] y0(I P X)y=[n rank(X)]: An important first](https://reader034.vdocuments.net/reader034/viewer/2022052608/5ae459447f8b9a7b218e4bb3/html5/thumbnails/28.jpg)
The alternative hypothesis
HA : E(y) ∈ C(X) \ C(X0)
is equivalent to
HA : There do not exist β1, β2 ∈ R such that
µi = β1 + β2(i) ∀ i = 1, 2, 3.
Copyright c©2018 Dan Nettleton (Iowa State University) 3. Statistics 510 28 / 43
![Page 29: 3. The F Test for Comparing Reduced vs. Full Models · Now back to determining the distribution of F = y0(P X P X 0)y=[rank(X) rank(X 0)] y0(I P X)y=[n rank(X)]: An important first](https://reader034.vdocuments.net/reader034/viewer/2022052608/5ae459447f8b9a7b218e4bb3/html5/thumbnails/29.jpg)
Because the lack of fit test is a reduced vs. full model F test, wecan also obtain this test by testing
H0 : Cβ = d vs. HA : Cβ 6= d
for appropriate C and d.
β =
µ1
µ2
µ3
C =? d =?
Copyright c©2018 Dan Nettleton (Iowa State University) 3. Statistics 510 29 / 43
![Page 30: 3. The F Test for Comparing Reduced vs. Full Models · Now back to determining the distribution of F = y0(P X P X 0)y=[rank(X) rank(X 0)] y0(I P X)y=[n rank(X)]: An important first](https://reader034.vdocuments.net/reader034/viewer/2022052608/5ae459447f8b9a7b218e4bb3/html5/thumbnails/30.jpg)
R Code and Output
> x=rep(1:3,each=3)> x[1] 1 1 1 2 2 2 3 3 3>> y=c(11,13,9,18,22,23,19,24,22)>> plot(x,y,pch=16,col=4,xlim=c(.5,3.5),+ xlab="Fertilizer Amount",+ ylab="Yield",axes=F,cex.lab=1.5)> axis(1,labels=1:3,at=1:3)> axis(2)> box()
Copyright c©2018 Dan Nettleton (Iowa State University) 3. Statistics 510 30 / 43
![Page 31: 3. The F Test for Comparing Reduced vs. Full Models · Now back to determining the distribution of F = y0(P X P X 0)y=[rank(X) rank(X 0)] y0(I P X)y=[n rank(X)]: An important first](https://reader034.vdocuments.net/reader034/viewer/2022052608/5ae459447f8b9a7b218e4bb3/html5/thumbnails/31.jpg)
●
●
●
●
●
●
●
●
●
Fertilizer Amount
Yie
ld
1 2 3
1015
20
Copyright c©2018 Dan Nettleton (Iowa State University) 3. Statistics 510 31 / 43
![Page 32: 3. The F Test for Comparing Reduced vs. Full Models · Now back to determining the distribution of F = y0(P X P X 0)y=[rank(X) rank(X 0)] y0(I P X)y=[n rank(X)]: An important first](https://reader034.vdocuments.net/reader034/viewer/2022052608/5ae459447f8b9a7b218e4bb3/html5/thumbnails/32.jpg)
> X0=model.matrix(˜x)> X0
(Intercept) x1 1 12 1 13 1 14 1 25 1 26 1 27 1 38 1 39 1 3
Copyright c©2018 Dan Nettleton (Iowa State University) 3. Statistics 510 32 / 43
![Page 33: 3. The F Test for Comparing Reduced vs. Full Models · Now back to determining the distribution of F = y0(P X P X 0)y=[rank(X) rank(X 0)] y0(I P X)y=[n rank(X)]: An important first](https://reader034.vdocuments.net/reader034/viewer/2022052608/5ae459447f8b9a7b218e4bb3/html5/thumbnails/33.jpg)
> X=model.matrix(˜0+factor(x))> X
factor(x)1 factor(x)2 factor(x)31 1 0 02 1 0 03 1 0 04 0 1 05 0 1 06 0 1 07 0 0 18 0 0 19 0 0 1
Copyright c©2018 Dan Nettleton (Iowa State University) 3. Statistics 510 33 / 43
![Page 34: 3. The F Test for Comparing Reduced vs. Full Models · Now back to determining the distribution of F = y0(P X P X 0)y=[rank(X) rank(X 0)] y0(I P X)y=[n rank(X)]: An important first](https://reader034.vdocuments.net/reader034/viewer/2022052608/5ae459447f8b9a7b218e4bb3/html5/thumbnails/34.jpg)
> proj=function(x){+ x%*%ginv(t(x)%*%x)%*%t(x)+ }>> library(MASS)> PX0=proj(X0)> PX=proj(X)
Copyright c©2018 Dan Nettleton (Iowa State University) 3. Statistics 510 34 / 43
![Page 35: 3. The F Test for Comparing Reduced vs. Full Models · Now back to determining the distribution of F = y0(P X P X 0)y=[rank(X) rank(X 0)] y0(I P X)y=[n rank(X)]: An important first](https://reader034.vdocuments.net/reader034/viewer/2022052608/5ae459447f8b9a7b218e4bb3/html5/thumbnails/35.jpg)
> Fstat=(t(y)%*%(PX-PX0)%*%y/1)/+ (t(y)%*%(diag(rep(1,9))-PX)%*%y/(9-3))> Fstat
[,1][1,] 7.538462>> pvalue=1-pf(Fstat,1,6)> pvalue
[,1][1,] 0.03348515
Copyright c©2018 Dan Nettleton (Iowa State University) 3. Statistics 510 35 / 43
![Page 36: 3. The F Test for Comparing Reduced vs. Full Models · Now back to determining the distribution of F = y0(P X P X 0)y=[rank(X) rank(X 0)] y0(I P X)y=[n rank(X)]: An important first](https://reader034.vdocuments.net/reader034/viewer/2022052608/5ae459447f8b9a7b218e4bb3/html5/thumbnails/36.jpg)
> reduced=lm(y˜x)> full=lm(y˜0+factor(x))>> rvsf=function(reduced,full)+ {+ sser=deviance(reduced)+ ssef=deviance(full)+ dfer=reduced$df+ dfef=full$df+ dfn=dfer-dfef+ Fstat=(sser-ssef)/dfn/+ (ssef/dfef)+ pvalue=1-pf(Fstat,dfn,dfef)+ list(Fstat=Fstat,dfn=dfn,dfd=dfef,+ pvalue=pvalue)+ }
Copyright c©2018 Dan Nettleton (Iowa State University) 3. Statistics 510 36 / 43
![Page 37: 3. The F Test for Comparing Reduced vs. Full Models · Now back to determining the distribution of F = y0(P X P X 0)y=[rank(X) rank(X 0)] y0(I P X)y=[n rank(X)]: An important first](https://reader034.vdocuments.net/reader034/viewer/2022052608/5ae459447f8b9a7b218e4bb3/html5/thumbnails/37.jpg)
> rvsf(reduced,full)$Fstat[1] 7.538462
$dfn[1] 1
$dfd[1] 6
$pvalue[1] 0.03348515
Copyright c©2018 Dan Nettleton (Iowa State University) 3. Statistics 510 37 / 43
![Page 38: 3. The F Test for Comparing Reduced vs. Full Models · Now back to determining the distribution of F = y0(P X P X 0)y=[rank(X) rank(X 0)] y0(I P X)y=[n rank(X)]: An important first](https://reader034.vdocuments.net/reader034/viewer/2022052608/5ae459447f8b9a7b218e4bb3/html5/thumbnails/38.jpg)
> anova(reduced,full)Analysis of Variance Table
Model 1: y ˜ xModel 2: y ˜ 0 + factor(x)
Res.Df RSS Df Sum of Sq F Pr(>F)1 7 78.2222 6 34.667 1 43.556 7.5385 0.03349 *---Signif. codes: 0 *** 0.001 ** 0.01 * 0.05 . 0.1 1
Copyright c©2018 Dan Nettleton (Iowa State University) 3. Statistics 510 38 / 43
![Page 39: 3. The F Test for Comparing Reduced vs. Full Models · Now back to determining the distribution of F = y0(P X P X 0)y=[rank(X) rank(X 0)] y0(I P X)y=[n rank(X)]: An important first](https://reader034.vdocuments.net/reader034/viewer/2022052608/5ae459447f8b9a7b218e4bb3/html5/thumbnails/39.jpg)
> test=function(lmout,C,d=0){+ b=coef(lmout)+ V=vcov(lmout)+ dfn=nrow(C)+ dfd=lmout$df+ Cb.d=C%*%b-d+ Fstat=drop(
t(Cb.d)%*%solve(C%*%V%*%t(C))%*%Cb.d/dfn)+ pvalue=1-pf(Fstat,dfn,dfd)+ list(Fstat=Fstat,pvalue=pvalue)+ }> test(full,matrix(c(1,-2,1),nrow=1))$Fstat[1] 7.538462$pvalue[1] 0.03348515
Copyright c©2018 Dan Nettleton (Iowa State University) 3. Statistics 510 39 / 43
![Page 40: 3. The F Test for Comparing Reduced vs. Full Models · Now back to determining the distribution of F = y0(P X P X 0)y=[rank(X) rank(X 0)] y0(I P X)y=[n rank(X)]: An important first](https://reader034.vdocuments.net/reader034/viewer/2022052608/5ae459447f8b9a7b218e4bb3/html5/thumbnails/40.jpg)
SAS Code and Output
data d;input x y;cards;
1 111 131 92 182 222 233 193 243 22;run;
Copyright c©2018 Dan Nettleton (Iowa State University) 3. Statistics 510 40 / 43
![Page 41: 3. The F Test for Comparing Reduced vs. Full Models · Now back to determining the distribution of F = y0(P X P X 0)y=[rank(X) rank(X 0)] y0(I P X)y=[n rank(X)]: An important first](https://reader034.vdocuments.net/reader034/viewer/2022052608/5ae459447f8b9a7b218e4bb3/html5/thumbnails/41.jpg)
proc glm;class x;model y=x;contrast ’Lack of Linear Fit’ x 1 -2 1;
run;
Copyright c©2018 Dan Nettleton (Iowa State University) 3. Statistics 510 41 / 43
![Page 42: 3. The F Test for Comparing Reduced vs. Full Models · Now back to determining the distribution of F = y0(P X P X 0)y=[rank(X) rank(X 0)] y0(I P X)y=[n rank(X)]: An important first](https://reader034.vdocuments.net/reader034/viewer/2022052608/5ae459447f8b9a7b218e4bb3/html5/thumbnails/42.jpg)
The SAS System
The GLM Procedure
Dependent Variable: y
Sum ofSource DF Squares Mean Square F Value Pr > F
Model 2 214.2222222 107.1111111 18.54 0.0027
Error 6 34.6666667 5.7777778
Corrected Total 8 248.8888889
R-Square Coeff Var Root MSE y Mean
0.860714 13.43684 2.403701 17.88889
Source DF Type I SS Mean Square F Value Pr > F
x 2 214.2222222 107.1111111 18.54 0.0027
Source DF Type III SS Mean Square F Value Pr > F
x 2 214.2222222 107.1111111 18.54 0.0027
Copyright c©2018 Dan Nettleton (Iowa State University) 3. Statistics 510 42 / 43
![Page 43: 3. The F Test for Comparing Reduced vs. Full Models · Now back to determining the distribution of F = y0(P X P X 0)y=[rank(X) rank(X 0)] y0(I P X)y=[n rank(X)]: An important first](https://reader034.vdocuments.net/reader034/viewer/2022052608/5ae459447f8b9a7b218e4bb3/html5/thumbnails/43.jpg)
Contrast DF Contrast SS Mean Square F Value Pr > F
Lack of Linear Fit 1 43.55555556 43.55555556 7.54 0.0335
Copyright c©2018 Dan Nettleton (Iowa State University) 3. Statistics 510 43 / 43