significance toolbox 1) identify the population of interest (what is the topic of discussion?) and...

Significance Toolbox1) Identify the population of interest (What is the topic of

discussion?) and parameter (mean, standard deviation,

probability) you want to draw conclusions about. State the null and alternative hypotheses.

2) Choose the appropriate inference procedure (type of

test) and verify conditions (what kind of information is given about population/sample. Is there an SRS? If not, we may be able to perform the test because of the finite number of observations – central limit theorem but generalizations may not be necessarily true especially if the distribution is severely nonnormal).

3) If the conditions are met, carry out the inference procedure (find the mean, deviation, and P-value).

4) Interpret your results (Is the information statistically significant?).

One-sample z statistic

H0: µ = µ0 The test uses

Ha: µ > µ0 is P(Z ≥ z) Ha: µ < µ0 is P(Z ≤ z) Ha: µ ≠ µ0 is 2P(Z ≥ |z|)

x

x

n

xz

00

View graphs on page573.

Fixed Significant Level for Z tests

for Population Mean

The outcome of a test is significant at level alpha if P-value ≤ .

Once we have computed the z test statistic, reject H0 at significant level against a one sided alternative when Ha: µ>µ0 if z ≥ z* and Ha: µ < µ0 if z ≤ - z*

Reject H0 at significant level alpha against a two-sided alternative Ha: µ≠µ0 if |z| ≥ z*

10.3 Making Sense of Statistical

Significance

Choosing a level, Standard: the level of significance gives

a clear statement of the degree of evidence provided by the sample against the null hypothesis.

Best practice: Decide on a significance level prior to testing. If the result satisfies the level, reject the null. If the result fails the level, find the null acceptable (fail to reject).

If we have a fixed significance level, we should ask how much evidence is required to reject H0. If H0 represents an assumption people have believed for years, strong evidence (small ) is needed. If rejecting H0 for Ha means making expensive changeover (products), strong evidence must show sales will soar.

Significant vs Insignificant There is no sharp border between

significant and insignificant only increasingly strong evidence as the P-value decreases.

When a null hypothesis can be rejected (5% or 1% level), there is good evidence that an effect is present.

To keep statistical significance in its place, pay close attention to the actual data and the P-value.

Statistical inference is not always valid

Surveys and experiments that are designed badly will produce invalid results.

Outliers in the data and testing a hypothesis on the same data that suggested the hypothesis invalidates the test.

Since tests of significance and confidence intervals are based on the laws of probability, randomization in sampling or experimentation ensures these laws apply.

Assignment

Exercises 10.44, 10.57, 10.58, 10.62, 10.64

10.4 Inference as Decision Reminders:

Tests of significances assess the strength of evidence ______ (for/against) the null hypothesis.

Measurement: P-value which is the probability computed under the assumption that null hypothesis is ______ (true/false).

The alternative hypothesis helps us to see what outcomes count ______ (for/against) the null hypothesis.

Strength Decision A significance level chosen in advance

points to the outcome of the test as a decision. If the result is significant, we reject the null

hypothesis in favor of the alternate. If the result is not significant, we fail to reject

the null (null hypothesis is acceptable). Making the decision to either fail to reject

(acceptable) or reject results should be left to the user but at times the final decision is stated during the interpretation.

Acceptance Sampling

A decision or action must be made as an end result of inference.

Failing to reject (Acceptable) or rejecting the end product.

Type I and II Errors- In tests of significance

-H0 - the null hypothesis -Ha - the alternative hypothesis

- However, when dealing with Type I and Type II errors, these hypotheses will represent accepting one decision and rejecting the other.

- Now …-H0 should be considered the initial hypothesis -Ha the secondary hypothesis

Type I Error

We have been calculating this type of error all along.

If we reject H0 (acceptable Ha) when in fact H0 is true.

Type II Error

If we find that the H0 is acceptable (reject Ha) when in fact Ha is true.

Quick Comparison

H0 True Ha True

Reject H0 Type 1 Error CorrectDecision

Fail to reject H0

(acceptable)

Correct Decision

Type 2 Error

Truth about the Population

Deci

sion b

ase

d o

n s

am

ple

Cancer Scenario

Ho: “We suspect that you have cancer”

Ho is True Ha is True

Reject the null: “You don’t have cancer!”

Diagnosis: Cancer

Type I Error

Diagnosis: No Cancer

Correct Decision

Fail to reject the null:“You have cancer!”

Diagnosis: Cancer

Correct Decision

Diagnosis: No Cancer

Type II Error

Significance and Type I Error The significance level of any fixed

level test is the probability of a Type I error. is the probability that the test will reject the null hypothesis H0 when H0 is in fact true.

Example 10.68, Page 598

Example 10.68

You have an SRS of size n = 9 from a normal distribution with = 1. You wish to test H0: µ = 0

Ha: µ > 0.

You decide to reject H0 if x > 0 and to accept H0 otherwise.

Power A significance test measures the ability to

detect an alternative hypothesis. The power against a specific alternative is the

probability that the test will reject H0 when the alternative is true.

Calculate the power of a specific alternative: subtract the probability of the Type II error for the alternative from 1.

Class example 10.68. Accept that the mean, H0, will be less than or equal to 0 18.4% of the time; however, the mean should be greater than 0 81.6% of the time (100% - 18.4%).

Power continued

Power works best for fixed significance levels.

Larger sample sizes will increase the power for a fixed significant level.

Increase the Power

If the strength of evidence required for rejection is too low, increase the significance level.

Consider an alternative farther away from µ0.

Increase the sample size. Decrease the standard deviation, .

Assignment (Work due on Monday, 3/28)

Exercises 10.67, 10.69, 10.71 and 10.81

ScenariosOJ Simpson – Guilty man goes free.Ho: OJ is innocentHa: OJ is guiltyFound not guilty – Type 2

Guilty man goes free.Movie: A Time to Kill – Ho: Father is innocentHa: Father is guiltyFound to be innocent –Type 2

To Kill a Mockingbird – Send an innocent man to jailHo: Tom Robinson is innocentHa: Tom Robinson is guiltyFound guilty – Type 1

The Green Mile