Statistics Notes Chapter 9: Testing a Claim Section 9.1 Significance Tests: The Basics

Introduction Activity Jack is a candidate for mayor and must gain at least 50% of the votes to be elected. Based on a poll of voters, can Jack be confident of victory? Step 1: You will sample voters by drawing them out of a hat randomly without replacement. Y – a yes vote for Jack N – a no vote for Jack Draw a person and record their vote and mark the graph to show the running total until 20 voters are sampled: Person Y / N Running %

of Yes Votes

Label the left side from 0% to 100% Label the bottom with the numbers 0 to 20

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 Construct a 90% Confidence interval for the 20 voters: Based on your sample, can you say that Jack will win the election? Jack’s opponent claims that a survey showed that Jack will only receive 40% of the vote. Can you argue that this claim is wrong?

Significance Testing (Hypothesis Testing) While confidence intervals are a common type of statistical inference – used when the goal is to estimate a population parameter using sample data – another form of statistical inference is that of significance testing (A.K.A. Hypothesis Test). Significance tests are used when one wants to assess whether the data provides enough evidence of some claim about the population. The Key Concept: The claim made is the Null Hypothesis. We know need evidence to prove that the claim is not true, so a sample is taken. If we took many samples and the claim was true, then what is the probability of getting the resulting statistic from the sample? This probability is called the p-value. If the probability is low enough, we say the claim is rejected. Example: Mr. E claims that teachers at NWHS tend to inflate their grades in order to keep students from complaining. The class disagrees and thinks that grades are not inflated and state that the average GPA in the school is only 3.0. In order to test Mr. E’s claim of grade inflation, an SRS of 40 students was taken and the average GPA from that sample was found to be 3.3=x (GPA’s at NWHS have a standard deviation 9.0=σ ). Is this enough evidence to suggest that the average NWHS grade is in fact above 3.0 (thus constituting grade inflation)? Solution: Keep in mind, that our sample is only one of many possible samples. So if we were to take another sample of the same size our x might be different. The sampling distribution of all sample means will be approximately

normal (due to CLT) with a mean of µµ =x and a standard deviation of 409.0

==nxσ

σ .

If, in fact, the mean GPA at NWHS is 3.0 (as the student’s claim) and the standard deviation is 0.9, how likely are we to get an x that is 3.3? Is the likelihood so small that we can’t attribute it to chance variation? Or is 3.3 a “reasonable” sample mean based on our data? We can answer this question by sketching the distribution of sample means below and finding the shaded area. This shaded area represents the probability that we would get a mean as large as 3.3 (or larger):

The p-value is .0175. If the true mean is 3.0, then the probability of getting a sample mean of 3.3 from a sample of 40 is .0175. This means that there is less than a one in 50 chance that our x was 3.3 by chance. Because this probability is so small it suggests that the true population mean GPA is in fact not 3.0, but some higher number. It appears that Mr. E is right after all.

How do we know when we have enough evidence to reject the claim? The p-value measures the probability of getting the test statistic if the null hypothesis is true. Most of the time, there is a significance level (called the α-level) set before sample is collected. If the p-value is below the significance level:

• Then we reject the null hypothesis H0 in favor of the alternate hypothesis Ha • We say the results are statistically significant

If the p-value is above the significance level:

• Then we fail to reject the null hypothesis H0 and reject the alternate hypothesis Ha • We say the results are NOT statistically significant

Most α levels are set at 0.05, which means that there would less than a 5% probability that the sample result would have occurred by chance.

From the original example: • Using a α levels of 0.05, we would reject the null hypothesis since our p-value = 0.0175 is less than the

α level. • Using a α levels of 0.01, we would fail to reject the null hypothesis since our p-value = 0.0175 is greater

than the α level. Setting up a formal hypothesis test: Step 1: Set up your hypothesis based on what you think about the population and identify the parameter of interest. µ = average GPA of NWHS students

0.3:0.3:

>

=

µ

µ

a

o

HH

NOTE: The null hypothesis is always stated in terms of oµµ = and the alternative hypothesis will be stated in one of three ways:

oµµ > or oµµ < or oµµ ≠ .

One Sided: We suspect the true mean is higher than

the claim

One Sided: We suspect the true mean is lower than

the claim

Two Sided: We suspect the true

mean is different than the claim

The Null Hypothesis states a claim about the value of a parameter. The previous example made a claim that the GPA was 3 so 0.3=µ There must be evidence to reject this claim.

The Alternate Hypothesis states our suspicion about the population. Our suspicion is that there is grade inflation which would imply that 0.3>µ

Step 2: Make sure the conditions are met. 1.) SRS – Given in the problem 2.) Normality - the sample size is greater than 30, CLT 3.) Independence – the population is more than 10 times larger than the sample size. Step 3: Calculate the test statistic

The test statistic is the statistic from the sample that estimates the parameter in the hypothesis. • When testing a mean, the test statistic is x • When testing a proportion, the test statistic is p̂

Step 4: Calculate the p-value

Find the probability associated with our alternate hypothesis using a Z-Test or T-Test: Step 5: Compare the p-value to the significance level (called an −α level)

If the p-value is below the significance level: • Then we reject the null hypothesis H0 in favor of the alternate hypothesis Ha • We say the results are statistically significant

If the p-value is above the significance level:

• Then we fail to reject the null hypothesis H0 and reject the alternate hypothesis Ha • We say the results are NOT statistically significant

Step 6: State your conclusion using AP Language.

We are NOT concluding that the null hypothesis or the alternate hypothesis are true. We are concluding that either have enough evidence to reject the null or we do NOT have enough evidence to reject the null.

AP Language for a proportion test:

If the proportion is really ______________ for ______________, there is a ____________ chance of finding a (state the null) (state the claim) (state the p-value)

sample of ___________ people with a p̂ value of ______________. We conclude that at the ____________ (state sample size, n) (state the test statistic) (state the α-level )

significance level the claim [is rejected] or [fails to be rejected]. AP Language for a mean test:

If the mean is really ______________ for ______________, there is a ____________ chance of finding a (state the null) (state the claim) (state the p-value)

sample of ___________ people with an x value of ______________. We conclude that at the ____________ (state sample size, n) (state the test statistic) (state the α-level )

(state sample size, n)

significance level the claim [is rejected] or [fails to be rejected].

0175.)108.2()3.3( =≥=≥ ZPXP

p-value: The probability of getting a result at least as far out as our result.

108.240/9.00.33.3

3.3 =−

=Z

Example Consider the election example on page 1. a. Describe the parameter of interest b. State the appropriate hypotheses for performing a significance test c. If the p-value for the hypotheses in part b is .024. What does this mean? d. If the significance level is α = .05. What is the conclusion? Example A golfer wants to improve his play. A friend suggests getting new clubs and lets him try out his 7-iron. Based on years of experience, he has established that the mean distance that balls travel with his old 7-iron mean = 175 yards with standard deviation 15 yards. He is hoping that this new club will make his shots with a 7-iron more consistent (less variable), so he goes to the driving range and hits 50 shots. a. Describe the parameter of interest b. State the appropriate hypotheses for performing a significance test c. If the p-value for the hypotheses in part b is .035. What does this mean? d. If the significance level is α = .01. What is the conclusion? Example Zeb performs a study to see if students prefer name brand chips or generic chips. He randomly selects 50 students. Overall, 34 of the 50 students preferred the name brand chips. Zeb performed a significance test using the hypotheses H0: p = 0.5 Ha: p > 0.5 where p = the true proportion of students at his school who prefer name-brand chips. The p-value was 0.0055. What conclusion would you make at each of the following significance levels? α = .05 α = .01 HW A: 1,3,5,7,9,11,13

Day 2 Comparison to the American Judicial System In the American judicial system we must first assume innocence and then must try to prove guilt. In court, we cannot prove innocence, we either have evidence "beyond a reasonable doubt" to 'prove' guilt, or we declare the person "not-guilty". The same is true in statistics. With a hypothesis test, either one or two tailed, the null hypothesis is our assumption of innocence. If we wish to determine ‘guilt’ (the alternative hypothesis), we must first assume innocence (the null hypothesis). Either we have enough evidence to reject the null hypothesis (reject innocence) using a level of significance or we do not have enough evidence (fail to reject innocence). We can never “accept” the null hypothesis since that is not what we set out to prove. Just like in court, when a jury fails to reject innocence (the null hypothesis) it doesn’t say the person is found innocent, it says the person is found “not guilty”. Example Julie was accused of shoplifting by the store manager. State the hypotheses: Errors in Significance Testing Type I Error: The null hypotheses is true, but is mistakenly rejected. Type II Error: The null hypothesis is false, but is mistakenly NOT rejected.

Conclusion

Population truth

Null is true Null is false

Reject null Type I error

Probability of type I error = α

Correct decision

Fail to reject null

Correct decision

Type II error

Probability of type II error = β

Example Explain a type I error for the shoplifting example Explain a type II error for the shoplifting example Which error type would be worse? Explain.

Power in Significance Testing • Power is the probability of reaching the right conclusion when the alternative hypothesis is true. • Power is the probability that the test will reject the null hypothesis when a specific alternate hypothesis

is true. • Power is the probability of avoiding a type II error (The null hypothesis is false, but is mistakenly NOT

rejected) • Power is a function in which the input is a specific alternate hypothesis and the output is a probability of

rejecting the null hypothesis. The power changes as the alternate changes. • The distance between the null hypothesis and the true parameter (p or µ) is sometimes called the effect.

The greater the effect, the lower the chance of a type II error and the greater the power. • Power = 1 - β where β = Probability of type II error • Power = 0 means there is no chance of detecting the null hypothesis is false • Power = 1 means the test is certain to detect the null hypotheses is false • As the significance level, α-level, becomes smaller, it is tougher to reject the null hypothesis. This

causes the probability of a type II error to increase and the power will decrease. • As sample size increases, power increases because the is more evidence that the null hypothesis should

be rejected. Example The manager of a fast-food restaurant wants to reduce the proportion of drive-through customers who have to wait more than two minutes. The proportion of customers who had to wait at least tow minutes was p = 0.63. To reduce this proportion, the manager assigns an additional employee to assist with the drive-through orders. During the next month, the manager will collect a random sample of drive-through times to test the following hypotheses:

H0: p = 0.63 Ha: p < 0.63 where p = the true proportion of drive-through customers who have to wait more than two minutes.

Describe a Type I error: Describe a Type II error: Which error type would be more serious?

Suppose that the manager decided to carry out this test using a random sample of 250 orders and a significance level α = .10. What is the probability of a type I error?

If the probability of a type II error is 0.38 then find the power. How would the following affect the power?

1. Reducing the significance level to 0.01

2. Reducing the sample size to 100 instead of 250 HW B: 15,19,21,23,25

Section 9-2: Significance Tests on a Proportion Reminders: Define the null hypothesis and alternate hypothesis

Conditions:

Randomly selected sample – Look for the term SRS. Without random selection, we lose the ability to make inferences about the population.

Normal Distribution – We will be using Normal Curves to find probabilities, so we need the distribution to be Normal.

For Proportions: np ≥ 10 and n(1-p) ≥ 10 Note: p is the null proportion, notation po

Independent Observations – In order to use our standard deviation formula, we need the sample size to be less than 10% of the population size, so 10n < N. This is especially true when sampling without replacement.

Standard Deviation of the sample proportion: 𝜎! = npp )1( − Note: p is the null proportion

Calculate the z-value using 𝑝    and 𝜎! Calculate the p-value using the z-value on the normal curve (need tail probability) Compare the p-value to the α-level and make the conclusion about the hypotheses. Example: Better to be last?

HW C: 27-30,41,43,45

Section 9-2 day 2 Starter Task: Complete this Investigative task in groups (20 minutes)

What do significance tests tell us? What do confidence intervals tell us? Which is more valuable? Confidence intervals provide additional information that significance tests do not – a range of plausible values for the true population parameter. Significance tests describe if evidence exists to reject a hypothesis (usually to one side). Confidence Intervals [in the graphs below, the standard deviations are marked on the x-axis] In the graph at the right:

• The confidence level is 95%. • The left tail starts at Z-value -2 and has 2.5% of the area • The right tail starts at Z-value 2 and has 2.5% of the area • The inside area is 95% and the edges are the confidence

interval. • The total area in the tails is 5% • For proportions, this curve is centered at 𝑝              (sample) • For means, this curve is centered at x (sample)

Hypothesis Test [in the graphs below, the standard deviations are marked on the x-axis]

One Sided Test to the left

• The shaded portion starts at Z-value -1.64

• The area of the shaded portion represents 5% of the curve.

• This is a significance level of α=.05

• The unshaded area is 95% of the total area.

One Sided Test to the right • The shaded portion starts at

Z-value -1.64 • The area of the shaded

portion represents 5% of the curve.

• This is a significance level of α=.05

• The unshaded area is 95% of the total area.

One Sided Test to the right • The shaded portions start at

Z-value -2 and 2. • Each tail has area 2.5% • The total area of the shaded

portion represents 5% • This is a significance level

of α=.05 • The unshaded area is 95%

of the total area. • For proportions, this curve

is centered at 𝑝            (null) • For means, this curve is

centered at µ (null)

Examples

HW D: 47,49,51,53,55

Section 9-3: Significance Tests on a Mean Reminders: Define the null hypothesis and alternate hypothesis

Conditions:

Randomly selected sample – Look for the term SRS. Without random selection, we lose the ability to make inferences about the population.

Normal Distribution – We will be using Normal Curves to find probabilities, so we need the distribution to be Normal.

For Means: either the population has to be normal, or the sample size n must be at least 30 CLT.

Independent Observations – In order to use our standard deviation formula, we need the sample size to be less than 10% of the population size, so 10n < N. This is especially true when sampling without replacement.

Z-Test T-Test Standard Deviation

The population SD is known

nxσ

σ =

The population SD is NOT known

nsx

x =σ

Find the P-Value

Find the Z-value

One sided test: Probability in the tail outside the z-value on the normal curve is the p-value

Two sided test: Probability in the tail outside the z-value on the normal curve will be doubled to find the p-value.

Find the T-Value with n-1 Deg. of Freedom

One sided test: Probability in the tail outside the T-value on the T-distribution curve is the p-value

Two sided test: Probability in the tail outside the T-value on the T-distribution curve will be doubled to find the p-value.

Compare the p-value to the α-level and make the conclusion about the hypotheses.

Examples

Describe the hypotheses, compute the test statistic and P-value. What is the conclusion.

HW E: 57-60,71,73

Section 9-3 Day 2 Starter Task: Complete this Investigative task in groups (20 minutes)

Matched Pair Designs (same notes as CH 4) These are experimental designs in which either the same individual or two matched individuals are assigned to receive the treatment and the control. In the case where an individual receives both the treatment and the control, the order in which this happens should be random. And the experiment should be conducted as a Double Blind experiment. Example: A medical researcher is interested in testing a new medication for poison ivy. He decides to conduct a clinical trial on 250 volunteers who are allergic to poison ivy. He purposefully rubs poison ivy on their calf, then after the rash appears, he gives half of the volunteer’s calamine lotion, and the other half he gives his new medication. How can this experiment be conducted in a matched pair design?

F: 75,77,89, 94-97,99-104

Extra Problems Example 2: Worried about his prospects for the prom, Malcolm claims that girls at NWHS are a bit snobby and that the average number of girls that a guy must ask to the prom before getting a “yes” is 4. Doug disagrees. He doesn’t think the girls are that snobby and that the average number of girls that a guy must ask out before getting a positive response is less than 4. An SRS from last year of 50 junior and senior guys found that the average number of girls that a guy asked out to the prom was 3.4. Assuming the standard deviation from the entire population is 2=σ , is there enough evidence to support Alex’s claim (at the level )05.=α ? Example 3: Frank has been sensing that his car is not driving right. He takes his car to the mechanic who does some testing on the ignition timing. In order for Frank’s car to run at optimum efficiency, the spark plugs need to ignite and spark, on the average every 1.3 seconds. Assume that the standard deviation of all spark plug firings is known to be 5.0=σ seconds. The mechanic suspects that Frank’s car spark plugs are not firing at this optimal interval. The mechanic took a random sample of 30 spark plug ignition firings from Frank’s car and got the following data: 1.0 1.1 0.8 1.7 0.9 1.3 1.2 1.5 1.3 0.8 0.6 1.3 1.1 1.2 0.7 1.9 2.0 1.1 1.3 1.4 1.0 1.2 0.9 0.4 1.3 1.2 1.4 1.0 1.3 1.3 Based on this data, can we say that Frank’s car problems stem from spark plug timing? (at the level )05.=α ?