inferential statistics part 2: hypothesis testing chapter 9 p. 280 - 306
TRANSCRIPT
Inferential Statistics Part 2:Hypothesis Testing
Chapter 9p. 280 - 306
IntroductionHypothesis testing is closely related to estimation (i.e., what we studied at last week)
The difference is that now we are posing a hypothesis that we want to test
For example, rather than just estimating a population parameter using a sample, we may hypothesize that a sample is different than the population in some way
Bases on a sample statistic we can either accept or reject the hypothesis
Steps in Classical Hypothesis Testing1: Formulate a hypothesis
2: Specify the sampling statistic and its distribution
3: Select a level of significance
4: Construct a decision rule
5: Compute a value of the test statistic
6: Decide to accept or reject the hypothesis
Formulate a hypothesis
Null Hypothesis (H0) – when the sample statistic follows the population parameter (e.g., when characteristics from a sample more or less match those from the population)
Alternative Hypothesis (HA) – When the sample statistic does not follow the population parameter
Possible statements:
ˆ:
ˆ:0
AH
H
ˆ:
ˆ:0
AH
H
ˆ:
ˆ:0
AH
H
Formulate a hypothesis
Which type of hypothesis (null or alternative) are we typically concerned with?
How do “tails” of a distribution fit the statements?
What does it mean to say these hypotheses are mutually exclusive & exhaustive?
Formulate a hypothesis
Remember that the hypotheses are being tested using sample data that may contain sampling error
This is why hypothesis testing falls under the category of inferential statistics
We have to infer results based on a sample We can’t be completely certain of the results, so there is a degree of uncertainty associated with our answersTo estimate this uncertainty we rely upon probability
Types of errorType 1 Error: when we falsely reject a null hypothesis, the probability of doing so is labeled α (i.e., α = P(type 1 error)
Type 2 Error: when we falsely accept a null hypothesis, the probability of doing so is labeled β (i.e., β = P(type 2 error)
H0 is true H0 is false
Reject H0 Type 1 Error No Error
Accept H0 No Error Type 2 Error
Specify the sampling statistic and its distribution
What sampling statistic should you choose for μ, σ, and pi respectively?
What distributions will the sampling statistics have and how do we know?
FYI, when used to test a hypothesis, sampling statistics are also called test statistics
Select a level of significance
In classical hypothesis testing we are only concerned with type 1 error (α)
For example: alpha of 0.1, 0.05, or 0.01The value for alpha is called the significance level
This means that if we reject H0 we will be very confident that it is false
How confident depends on the significance level
The flip-side of this approach is that we are more likely to not reject a null hypothesis that is false
Select a level of significanceHow does this fit with the idea that we are typically concerned with HA rather than H0?
Answer: since the significance is tied to rejecting H0 it is also linked with accepting HA
This means that the hypotheses we make should be structured so that we are testing HA (i.e., rejecting H0 should be scientifically interesting)
To make this more clear, think about the opposite case: if we were really interested in accepting H0 we would have no idea about the significance because we are ignoring type 2 error
Select a level of significance
Whenever we report a decision about the null hypothesis (to reject it or not) we also report the statistical significance
Example:The null hypothesis is rejected at the 0.05 significance level
Select a level of significance
Which significance level we actually choose depends on the application
When might we want a very small α?
In geography 0.1, 0.05., 0.01 are pretty typical
It is also common to see results reported for multiple alphas
Construct a decision rule
For this step we take the hypothesis we’ve defined and the significance level we’ve selected and determine the critical region and the critical values
In other words, we take our values, and determine the thresholds for accepting or rejecting H0
Construct a decision rule
Critical Regions: if the sample statistic falls within these area(s) we will reject H0
Critical Values: the thresholds that divide the critical region(s) from the non-critical region
Construct a decision ruleFor a test statistic with a normal distribution (e.g., x and p) we make our decision rule using:
For p, the equation is:
For x the equation is:
Key things to remember
How to calculate σThe number of tails
p
Criticalz
0
X
Criticalz
0
statistictest
HinValueCriticalz
_
0__
Compute a value of the test statistic
Here we just compute the values using equations we’re familiar with (e.g., x and p)
Note that constructing a decision rule and computing the values of a test statistic can also be done using z-values for the critical values and for the test statistic (see p. 289 for details)
Decide to accept or reject the hypothesis
Now we just compare the test statistic with the critical values and make our decision to reject H0 or not
Classical Hypothesis Testing Example
Has the mean temperature of Charlotte increased over the last 30 years?
This is an example for μ
Example DataSuppose Charlotte’s annual mean temp for the last:
150 years is 50o F.30 years is 53o F.
Suppose the population variance, σ 2, for these 150 years is 9 (so σ = 3)
Assumptions: Each year is independent of other yearsThe last 30 years act as a sample of the population of years since greenhouse gases have been emitted into the Earth’s atmosphere. (These 30 are all we have access to). These 30 years come from the same distribution.
Steps in Classical Hypothesis Testing1: Formulate a hypothesis
2: Specify the sampling statistic and its distribution
3: Select a level of significance
4: Construct a decision rule
5: Compute a value of the test statistic
6: Decide to accept or reject the hypothesis
Step 1: Formulate a hypothesisScientifically, we say our hypothesis is: the mean temperature of Charlotte has increased over the last 30 years
Statistically, we developNull hypothesis H0: Θ ≤ Θ0
Alternative hypothesis HA: Θ > Θ0
When we apply the data:Null hypothesis H0: x ≤ 50o FAlternative hypothesis HA: x > 50o F
This is a 1-sided test
Step 2: Specify the sampling statistic and its distribution
What sampling statistic should we use?
What distribution with it have?
Answers: The sample mean (in this case 53o F)A normal distribution
• Our sample size is 30, which is just large enough to use the z rather than the t distribution
• This is an application of the central limit theorem
The sample statistic & the hypothesis
If x is below or near 50, we do not reject the null hypothesis:
H0: x ≤ 50o F.
If x is far greater than 50, we reject the null hypothesis in favor of the alternative hypothesis:
HA: x > 50o F.
Why isn’t this simple comparison sufficient?Answer: because x is just a sample and may have error
We set a cutoff point for x, above which we reject our null hypothesis.This cutoff is set at a point where, if the null hypothesis were true, a value of x this large or larger would be very unlikely (due to sampling variation alone).
Step 3: Select a level of significance
This step is always somewhat arbitrary, but we’ll just use 0.05
This means that we’re willing to accept a 5% chance of having a type 1 error (i.e., rejecting H0 when we should not)
Step 4: Construct a decision rule
9037.50505477.0*1.65
1.655477.0
50
5477.0477.5
3
30
3
05.0
0
Critical
Criticalz
n
Criticalz
X
X
Step 4: Construct a decision rule
So we say that we will reject H0 if x is > 50.9037 with a significance level of 0.05
Steps 5 & 6Step 5: Compute a value of the test statistic
In this case we already have the test statistic (x = 53)
Step 6: Decide to reject the null hypothesis (or not)
Now we just compare our test statistic with the critical valueSince 53 is > 50.9037 we will reject the null hypothesis and accept the alternative hypothesis
Shortcomings of the classical approach
The decision to reject the null hypothesis is binary
No detail is given for how far the test statistic is from the critical value (e.g., is it just above it, or way above it)
Different α value might read to different decisions
The PROB-VALUE approachThis approach fixes the shortcomings of the classical approach
Basically it involves using the same equations, but flipping them around so that we solve for α
In other words:At what level is the test statistic significantWhat is the α (i.e., the probability of making a type 1 error)Should we reject H0 how likely are we to be wrong
The PROB-VALUE approach
This is based on the equation:
The difference from the classical approach is that now we look up the z-value to tell us the alpha (α)
ˆ
0ˆ
Z
PROB-VALUE exampleCharlotte Example
Using a z-table, what alpha is associated with this z?
Answer: α = 0.000000021602This value is actually from Excel, the z-table in the book does not go up to 5.477
In other words, there is a 2.16 in 100 million chance of the null being falsely rejected
477.55477.0
5053
ˆ
ˆ
0
z
z
PROB-VALUE & alphaRemember that the PROB-VALUE is equivalent to finding the alpha associated with a z-value
Therefore we can also use the PROB-VALUE to reject a H0 (or not)
Example:If our selected significance level is 0.05And our PROB-VALUE is 0.00001We’d reject the null hypothesis since 0.00001 < 0.05
Additional things to considerAs with confidence intervals, when conducting a hypothesis using μ we should use t instead of z when:
n < 30we have s instead of σ (with an n > 30 either is ok)
As with confidence intervals, when conducting a hypothesis test using π we should use the binomial distribution instead of z when:
n < 100Example 9-4 in the book solves such a problem
Sample Problems Galore!
We’re going to go through several examples that are reminiscent of problems on your homework and what will be on the exam
Key questions to ask before startingWhat is the test statistic?
x and p have slightly different equations, particularly for their standard deviations
How many tails does the test have?Determines whether we use α or α/2Determines whether we multiply the PROB-VALUE by 2
If we are doing a 1 tailed test, which critical value are we concerned about?
: lower critical value : upper critical value
What distribution should we use (t, z, or binomial)
ˆ:
ˆ:
A
A
H
H
Sample Problem #1A census of UNC students found that students had, on average, 3.4 pets each while growing up with a standard deviation of 1.9 pets.
A single dorm with 220 students had an average of 3.65 pets growing up. Assuming the students are assigned to the dorm at random (i.e., they are statistically independent), does this dorm have a higher than normal “pet history” with a 0.01 significance level?
Sample Problem #1What is the test statistic?
How many tails does the test have?
Which critical value are we concerned about?
Putting these together - what are H0 and HA ?
65.34.3::ˆ:
65.34.3::ˆ:0
xH
xH
A
Sample Problem #1What are n, σ, and α?
n = 220σ = 1.9α = 0.01
What distribution should we use and why?The z-distribution since n > 30
What is the z-value associated with α?Z0.01 = 2.33
What is the standard deviation of x?
128.083.14/9.1220/9.1/ nx
Sample Problem #1
Critical Value
Should we reject the null hypothesis?
698.33.4.1280*2.33
33.2128.0
4.301.0
0
Critical
Criticalz
Criticalz
X
Sample Problem #1
What would happen to the critical value if we changed the significance level to 0.05?
Does this make us more or less likely to reject the null hypothesis?
6112.33.4.1280*1.65
65.1128.0
4.305.0
Critical
Criticalz
Sample Problem #1
PROB-VALUE
What values go in this equation?
What do we do with the resulting z-value?
What is the PROB-VALUE
ˆ
0ˆ
z
953.1128.0
4.365.3
z
0255.0
953.1
VALUEPROB
zz VALUEPROB
Sample Problem #2A census of UNC students found that students had, on average, a 12 minute commute (walking, bicycling, bus, car, etc.) to their first class of the day.
16 randomly sampled students living off campus had an average commute of 17 minutes with a sample standard deviation of 4.5 minutes. Do students living off campus have a longer commute with a 0.05 significance level?
Sample Problem #2What is the test statistic?
How many tails does the test have?
Which critical value are we concerned about?
Putting these together - what are H0 and HA ?
1912::ˆ:
1912::ˆ:0
xH
xH
A
Sample Problem #2What are n, s, and α?
n = 16s = 4.5α = 0.05
What distribution should we use and why?The t-distribution since n < 30 and we have s instead of σ
What is the t-value associated with α?t0.05,15 = 1.75
What is the standard deviation of x?125.14/5.416/5.4/ nsx
Sample Problem #2
Critical Value
Should we reject the null hypothesis?
969.1312.1251*1.75
75.1125.1
1215,05.0
0,
Critical
Criticalt
Criticalt
X
df
Sample Problem #2
PROB-VALUE
What values go in this equation?
What do we do with the resulting z-value?
What is the PROB-VALUE
ˆ
0ˆ
t
444.4125.1
1217
t
000.0
444.4,,
VALUEPROB
tt dfVALUEPROBdf
Sample Problem #3A botanical index states that the average weight of a northern red oak acorn is 6 grams.
A random sample of 101 acorns was collected from the red oaks in the quad and the acorns had an average weight of 5.6 grams and a sample standard deviation of 1.3 grams.
Are the oak trees in the quad atypical from normal trees with a significance of 0.05?
Sample Problem #3What is the test statistic?
How many tails does the test have?
Which critical value are we concerned about?
Putting these together - what are H0 and HA ?
6.56::ˆ:
6.56::ˆ:0
xH
xH
A
Sample Problem #3What are n, s, and α?
n = 101s = 1.3α = 0.05
What distribution should we use and why?Either one would be ok, but since we’re using s we’ll go with t
What is the t-value associated with α/2?t0.025,100 = 1.98Note how close this is to z0.025 = 1.96
What is the standard deviation of x?13.005.10/3.1101/3.1/ nsx
Sample Problem #3
Critical Value
Should we reject the null hypothesis?
2574.6613.0*98.1_
7426.5613.0*1.98_
98.113.0
6100,025.0
0,
CriticalUpper
CriticalLower
Criticalt
Criticalt
X
df
Sample Problem #3
PROB-VALUE
What values go in this equation?
What do we do with the resulting z-value?
What is the PROB-VALUE
ˆ
0ˆ
t
077.313.0
66.5
t
001.0
077.3,,2/
VALUEPROB
tt dfVALUEPROBdf
Sample Problem #4Suppose a census of UNC students found that 8 percent of students bike to class regularly.
A random sample of 160 business majors found that 7 biked regularly.
If would seem that business majors bike less than other students, what significance level does this statement have?
Sample Problem #4What is the test statistic?
How many tails does the test have?
Which critical value are we concerned about?
Putting these together - what are H0 and HA ?
0475.008.0::ˆ:
04375.008.0::ˆ:0
pH
pH
A
Sample Problem #4What are n, π, and p?
n = 160π = 0.08p = 7/160 = 0.04375
What distribution should we use and why?The z-distribution since have probabilities and a large n
What is the standard deviation of p?
02145.00046.0160/92.0*08.0/)1(* np
Sample Problem #4
PROB-VALUE
What values go in this equation?
What do we do with the resulting z-value?
What is the PROB-VALUE
ˆ
0ˆ
z
69.102145.0
08.004375.0
z
046.0
69.1
VALUEPROB
zz VALUEPROB
Sample Problem #4
What does a PROB-VALUE of 0.046 indicate about our statement?
Statistical Significance vs. Practical Significance
What are all these tests really telling us?They tell us about the presence of difference (<, >, =), which can be really scientifically uninteresting
Two approaches for managing this situation
Test only important hypothesesUse confidence intervals rather than hypothesis tests