Section 10.3Making Sense of Statistical Significance

February 15th 2013

Choosing a level of significance(alpha level)

How plausible is H0?Depending on H0 plausibility, you may choose

a smaller alpha. If H0 is very plausible, you will need to have

collect “more” evidence to reject it.

What are the consequences of rejecting H0?

If rejecting H0 would costs lots of moneypossibly cost livescosts jobs

then alpha is usually very small

Fishing for significance

Let’s say were trying to find a connection between eating habits and intelligence.

Choose 40 foods, and assign people to increase the amount of the foods they eat, and see if there are any foods that make people smarter.

Of the 40 foods, we find that peeps and green beans make you smarter with alpha=.05. Is this a problem?

Section 10.4.1Inference as Decision

February 15th 2013

When inference is used to make a decision… Either you reject H0 or you fail to reject H0. You can reject H0 correctly You can fail to reject H0 correctly You reject H0 incorrectly

(Type I error) You can fail to reject H0 incorrectly

(Type II error)

Fail to Reject H0

Reject H0

H0 is true Correct Type I error

H0 is false Type II error

Correct

H0: No Wolf

Ha: WolfPeople don’t

believePeople do

believe

No Wolf Correct Type I error

Wolf Type II error Correct

Accept H0 Accept Ha

H0 is true

Ha is true

Potato Chips Example

The salt content of the chips should have a mean of 2 mg with a standard deviation of .1 mg.

When deciding whether to accept or reject a batch of potato chips, a company looks at the salt content of 50 chips.

If the salt content is too far away from the mean, it will reject the batch.

What range values are acceptable?

The company will check a 50 chip sample. If our alpha is .05, the acceptable range is

the same as the 95% confidence interval:

*

* 1.96

zn

z

0.12 1.96

502 .0277,2 .0277

1.9723,2.0277

Accept or reject?

We understand with normal variation and everything working normally, we will get a sodium value between 1.9723 mg and 2.0277 mg 95% of the time.

0.12 1.96

502 .0277,2 .0277

1.9723,2.0277

Accept or reject?

We understand with normal variation and everything working normally, we will get a sodium value between 1.9723 mg and 2.0277 mg 95% of the time.

This means the 5% of the time you will reject a batch of chips that are fine.

When we reject the batch (and H0) incorrectly we have committed a Type I error.

95% Confidence Interval

2.02771.9723

Accept H0

Reject H0Reject H0

Significance and Type I Error

The significance level α of any fixed level test is the probability of a Type I error. That is, α is the probability that the test will reject the null hypothesis H0 when H0 is in fact true.

Probability of the Type II error

What if there really is a difference in the overall saltiness of the potato chips, would always we see a “significant” result?

95% Confidence Interval

2.02771.9723

Accept H0 Reject H0

2.05

What is the probability of having a

saltiness score of less than 2.0277 mg

when the distribution has a mean

0.1of 2.05 and a standard deviation of ?

502.0277 2.05

1.57680.1 50

1.5768 .05742078

z

P z

Section 10.4.2Power

What is Power?

Power is a test of sensitivity. Your statistical test may be able to detect

differences, but how well does it detect difference of a pre-determined nature?

The Power procedure allows to state the probability of our procedure to catch the differences.

Power Procedure

Begin by stating your H0 and Ha as usual.

Find the z* or t* that would allow you to reject H0. Find the x-bar that matches up with the z* or t*. Assuming that you have a particular true mean,

what is the probability that you would be to still reject the H0?

Power Example: Example 10.23

Can a 6-hour study program increase your score on SAT? A team of researchers is planning as study to examine this question. Based on the result of a previous study, they are willing to assume that the change has σ=50. Research would like significance at the .05 level.

Power Example: Example 10.23

A change of 50 points would be considered important, and the researchers would like to have a reasonable chance of detecting a change is this large or larger. Is 25 subjects a large enough sample for this project?

Step 1: State your hypothesis

H0: µ=0 Ha: µ>0 Where µ represents the change is in the

SAT score.

Step 2: Find the z* value and find the data value We'll set α=.05,

invNorm(.95) gives us a z*=1.645.

What is the lowest x-bar would show significance?

Summary: If we had a study with n=25 and x-bar>16.45, we would have significance.

*/

01.645

50 / 25

1.645 50 / 25

16.45

xz

nx

x

x

Step 3: Chance at importance

We stated that gains of 50 points would be considered "important". We state this as the alternative µ=50.

The power against the alternative µ=50 increase is the probability that H0 is rejected when µ=50.

Restated: What the area from 16.45 to ∞ under a normal curve centered at µ=50.

Step 3

normalcdf(16.45,1E99,50,50/√(25))=.9996 Summary: because the power is so high,

there is a great chance of finding a significance when the real increase is 50.

Increase Power by…

increase alpha increase sample size

Exercises

10.71-10.77 odd, 10.79-10.89