1

Stat 5100 Handout #8 – Notes: Simple Inference

After verifying assumptions, look at “inference”

Hypothesis testing – basic steps

1. Null Hypothesis (“no effect”)

Alternative Hypothesis (“some effect”)

2. Test statistic

-- depends on model and H0

3. Determine “sampling distribution”

If H0 is true, and we “drew” many samples of size n from this population, calculating t

for each sample, what would be distribution of these t values?

When model assumptions are true and H0 is true, statistical theory says:

Then t =

4. Get P-value

2

P-value is probability of observing a difference (t) at least as extreme as what was seen,

just by chance, when H0 is true.

5. Make conclusion in context

[Historical note regarding .05 threshold] To Ronald Fisher, the significance test only made sense

in the context of a sequence of experiments, all aimed at clarifying the same effect. The closest

he ever came to defining a specific p-value cut-off was in a 1929 article to the Society for

Psychical Research:

“An observation is judged significant, if it would rarely have been produced, in the

absence of a real cause of the kind we are seeking.”

“It is a common practice to judge a result significant, if it is of such a magnitude that it

would have been produced by chance not more frequently than once in twenty trials.

This is an arbitrary, but convenient, level of significance for the practical investigator ...”

“He should only claim that a phenomenon is experimentally demonstrable when he

knows how to design an experiment so that it will rarely fail to give a significant result.

Consequently, isolated significant results which he does not know how to reproduce are

left in suspense pending further investigation.”

(As on p. 99 of ``The Lady Tasting Tea'' (2001) by David Salsburg; similar discussion in “Truth,

Damn Truth, and Statistics”, by Paul F. Velleman in July 2008 Journal of Statistics Education:

www.amstat.org/publications/jse/v16n2/velleman.pdf)

Confidence Interval (equivalent to hypothesis testing)

General form for some parameter:

For β1:

Interpretation:

o We are 95% confident that the true value of β1 is in this interval.

o With many repeated samples, expect 95% of resulting intervals to contain true β1.

3

Testing H0: β1=0 at level α is the same as checking whether 0 is inside the (1- α)100% CI for β1.

Note: inference (hypothesis testing or confidence interval) for β0 similar to β1 method, but

usually less interesting.

Inference has to do with usefulness of model – is X useful in predicting Y ? We can look at

“model usefulness” a little more generally – the ANOVA table, with components:

Sum of Squares

o SStotal =

o SSerror =

o SSmodel =

Mean Square

F =

R2 =

MSE = Mean Square Error =

4

Two other ways to look at H0: β1=0:

1. How much worse would model fit be if we dropped the β1 term?

Full model:

Reduced model:

F-statistic looks at change in SSerror between these two models

2. Let ρ = Corr(X,Y)

Usually estimate this with:

Text section 2.11 shows how:

What about inference on response (Y)? Consider interval estimation for response:

Two different ways to get SE[�̂�], depending on type of interval desired:

1. Interval estimate of mean (or expected) Y for population of all X=Xh

2. Interval estimate of predicted Y for a single [new] observation at X=Xh

5

Toluca example (with Xh=10):

If we were to go run a new lot with lotsize=10, we are 95% confident that its workhours

would be between:

If we could get all lots with lotsize=10 (in all possible lots, not just in our sample), we are

95% confident that their mean workhours would be between:

Note: Confidence intervals are narrower than prediction intervals

Note: In these models (so far), dfE = n–2 because:

We will make this generalizable by notation:

o n =

o p =

o dfE =