LISA Short Course:A Tutorial in t-tests and ANOVA using JMP

Laboratory for Interdisciplinary Statistical Analysis

Anne RyanAssistant Professor of Practice

Department of Statistics, VTagryan@vt.edu

Laboratory for Interdisciplinary Statistical

Analysis

Collaboration From our website request a meeting for personalized statistical advice

Great advice right now:Meet with LISA before collecting your data

Short Courses Designed to help graduate students apply statistics in their research

Walk-In Consulting

Available 1-3 PM: Mon—Fri in the GLC Video Conference Room for questions requiring <30 minsSee our website for additional times and locations.

All services are FREE for VT researchers. We assist with research—not class projects or homework.

LISA helps VT researchers benefit

from the use of Statistics

www.lisa.stat.vt.edu

Designing Experiments • Analyzing Data • Interpreting ResultsGrant Proposals • Using Software (R, SAS, JMP, Minitab...)

Hypothesis Testing and Criminal Trial

Analogy

3

Defense:

Prosecution:

What’s the Assumed Conclusion?

Criminal Trial

Represent the accused (defendant)

Hold the “Burden of Proof”—obligation to shift the assumed conclusion from an oppositional opinion to one’s own position through evidence

ANSWER: The accused is innocent until proven guilty.• Prosecution must convince the judge/jury that

the defendant is guilty beyond a reasonable doubt

4

Similarities between Criminal Trials and Hypothesis Testing

Burden of Proof—Obligation to shift the conclusion using evidence

TrialHypothesis Test

Innocent until proven guilty

Accept the status quo (what is

believed before) until the data

suggests otherwise

5

Similarities between Criminal Trials and Hypothesis Testing

Decision Criteria

TrialHypothesis Test

Evidence has to convincing beyond a

reasonable

Occurs by chance less than 100α% of the time (ex:

5%)

6

Inferential Statistics is defined as

… a procedure that allows us to make statements about a general population using the results of a random sample from that population.

• Two Types of Inferential Statistics:• Hypothesis Testing• Estimation

Point estimates Confidence intervals

7

What is hypothesis testing ?Hypothesis testing is a detailed protocol for decision-making concerning a population by examining a sample from that population.

8

1. Test

2. Assumptions

3. Hypotheses

4. Mechanics

5. Conclusion

Steps in a Hypothesis Test

9

One Sample t-TestUsed to test whether the population mean is different from a specified value.

10

Example 1: Motivating Example In a glaucoma study, the following intraocular pressure

(mm Hg) values were recorded from a sample of 21 elderly subjects. Based on this data, can we conclude that the mean intraocular pressure of the population from which the sample was drawn differs from 14 mm Hg?*

Intraocular Pressure

14.5 12.9 14 16.1 12 17.5 14.1

12.9 17.9 12 16.4 24.2 12.2 14.4

17 10 18.5 20.8 16.2 14.9 19.6

*Wayne, D. Biostatistics: A Foundation for Analysis in the Health Sciences. 5th ed. New York: John Wiley & Sons, 1991.

𝑦=15.6238 𝑠=3.383

11

State the name of the testing method to be used

It is important to not be off track in the very beginning

1. TestExample 1:

1. Test: One sample t test for

List all the assumptions required for your test to be valid.

All tests have assumptions

Even if assumptions are not met you should still comment on how this affects your results.

2. AssumptionsExample 1:

2. Assumptions• Simple random sample

(SRS) was used to collect data

• The population distribution from which the sample is drawn is normal or approximately normal.

Claims versus suspicions:

The “null hypothesis” is a statement describing a claim about a population constant. - The null hypothesis is denoted as .

The “alternative hypothesis” is a statement describing the researcher’s suspicions about the claim. Also called “research hypothesis”.- The alternative hypothesis is denoted as .

3. Hypotheses

Examples of possible hypotheses:

For hypothesis testing there are three versions for testing that are determined by the context of the research question.◦ Left Tailed Hypothesis Test (less than)◦ Right Tailed Hypothesis Test (greater than)◦ Two Tailed or Two Sided Hypothesis Test (not

equal to)

3. Hypotheses

Left Tailed Hypothesis Test: Researchers are only interested in whether the

true value is below the hypothesized value. Example— Administrators of a health care

center want to know if the mean time spent by patients in the waiting room is less than 20 minutes.

Right Tailed Hypothesis Test: Researchers are only interested in whether the

True Value is above the hypothesized value. Example— Administrators of a health care

center want to know if the mean time spent by patients in the waiting room is greater than 20 minutes.

3. Hypotheses

Two Tailed or Two Sided Hypothesis Test:• The researcher is interested in looking

above and below their hypothesized value.• Example— Administrators of a health care

center want to know if the mean time spent by patients in the waiting room differs from 20 minutes.

◦ Note: The direction of the alternative hypothesis will be used when determining the p-value at a later step.

3. Hypotheses

3. Hypotheses Example 1:3. Hypotheses

In a glaucoma study, the following intraocular pressure (mm Hg) values were recorded from a sample of 21 elderly subjects. Based on this data, can we conclude that the mean intraocular pressure of the population from which the sample was drawn differs from 14 mm Hg?*

• What are the hypotheses for Example 1?

𝑯𝟎 :𝝁=𝟏𝟒 𝒗𝒔 .𝑯 𝒂 :𝝁≠𝟏𝟒Where is the true intraocular pressure

Computational Part of the Test

Parts of the Mechanics Step◦ Stating the Significance Level◦ Finding the Rejection Rule◦ Computing the Test Statistic◦ Computing the p-value

4. Mechanics

Significance Level: Here we choose a value to use as the significance level, which is the level at which we are willing to start rejecting the null hypothesis.

Denoted by α which corresponds to the Type 1 Error for the test.

Type 1 Error is error committed when the true null hypothesis is rejected. Ex: You reject when is true.

* Default value is α=.05, use α=.05 unless otherwise noted!

4. MechanicsExample 1:

4. Mechanics:Significance Level:

*We use here because the significance level was not given in the problem.*Note: The Type I error would be concluding that the true mean intraocular pressure differs from 14 mm Hg, when in fact the pressure is 14 mm Hg.

Rejection Rule: State our criteria for rejecting the null hypothesis

Reject the null hypothesis () if the p-value

p-value: The chance of observing your sample results or more extreme results assuming that the null hypothesis is true. If this chance is “small” then you may decide the claim in the null hypothesis is false.

4. MechanicsExample 1:

4. Mechanics:Rejection Rule:Reject if

Test Statistic: Compute the test statistic, which is usually a standardization of your point estimate.

Translates your point estimate, a statistic, to follow a known distribution so that is can be used for a test.

A point estimate is a single numerical value used to estimate the corresponding population parameter.

• is the point estimate for

4. Mechanics

Deriving the Test Statistic for Example 1

In many cases, including Example 1, the population standard deviation is unknown because it is a parameter from the population that must be estimated.

The best estimate for is .• Our standardized value becomes

: hypothesized mean: sample mean: sample standard deviation: sample size: observed t test statistic

23

Test statistic for a one sample t-test

This t observed ( test statistic follows a t distribution with degrees of freedom.

4. Mechanics Example 1:4. Mechanics Test Statistic:

*In the example it was given that and .

𝒕𝒐𝒃𝒔=𝒚−𝝁𝟎

𝒔/√𝒏=𝟏𝟓 .𝟔𝟐𝟑𝟖−𝟏𝟒

𝟑 .𝟑𝟖𝟑 /√𝟐𝟏=𝟐 .𝟐𝟎

p-value: After computing the test statistic, now you can compute the p-value.

A p-value is the probability of obtaining a point estimate as “extreme” as the current value where the definition of “extreme” is taken from the alternative hypothesis assuming the null hypothesis is true.

The p-value depends on the alternative hypothesis, so there are three ways to compute p-values.

p-value: The chance of observing your sample results or more extreme results assuming that the null hypothesis is true. If this chance is “small” then you may decide the claim in the null hypothesis is false.

4. MechanicsExample 1:

4. Mechanics:P-value (in words):The probability of observing a sample mean of mm hg or a value more extreme assuming the true mean pressure is 14 mm hg.

The p-value is determined based on the sign of the alternative hypothesis.

1. . If this is the case, then the p-value is the area in both tails of the t distribution.

4. Mechanics

0.4

0.3

0.2

0.1

0.0

Densi

ty

-t_obs

1/2 p-value

t_obs

1/2 p-value

0

The p-value is determined based on the sign of the alternative hypothesis.

2. . If this is the case, then the p-value is the area to the left of the observed test statistic.

4. Mechanics

0.4

0.3

0.2

0.1

0.0

Densi

ty

t_obs

p-value

0

The p-value is determined based on the sign of the alternative hypothesis.

3. . If this is the case, then the p-value is the area to the right of the observed test statistic.

4. Mechanics

0.4

0.3

0.2

0.1

0.0

Densi

ty

t_obs

p-value

0

Example 1:4. Mechanics p-value: *In the example the hypotheses are:

0.4

0.3

0.2

0.1

0.0

t

Densi

ty

-2.2

0.01986

2.2

0.01986

0

Example 1:4. Mechanics p-value:

𝒑−𝒗𝒂𝒍𝒖𝒆=𝟎 .𝟎𝟏𝟗𝟖𝟔+𝟎 .𝟎𝟏𝟗𝟖𝟔=𝟎 .𝟎𝟑𝟗𝟕𝟐

0.4

0.3

0.2

0.1

0.0

t

Densi

ty

-2.2

0.01986

2.2

0.01986

0

Example 1:4. Mechanics p-value:

𝒑−𝒗𝒂𝒍𝒖𝒆=𝟎 .𝟎𝟏𝟗𝟖𝟔+𝟎 .𝟎𝟏𝟗𝟖𝟔=𝟎 .𝟎𝟑𝟗𝟕𝟐Hypothesized Value

Actual Estimate

DF

Std Dev

14

15.6238

20

3.38288

Test Statistic

Prob > |t|

Prob > t

Prob < t

2.1997

t Test

0.0398*

0.0199*

0.9801

Test Mean

JMP will give the 3 p-values and you must select the correct p-value based on your alternative hypothesis

𝐻𝑎 :𝜇≠14𝐻𝑎 :𝜇>14𝐻𝑎 :𝜇<14

Conclusion: Last step of the hypothesis test.

Conclusions should always include:◦ Decision: reject or fail to reject

(not accept ). When conducting hypothesis tests, we

assume that is true, therefore the decision cannot be to accept the null hypothesis.

◦ Context: what your decision means in context of the problem.

5. Conclusion Example 1:

5. Conclusion:With a p-value=0.0398, which is less than 0.05, we reject . There is sufficient sample evidence to conclude that the true mean intraocular pressure differs from 14 mm Hg.

Note: The significance level can be thought of as a tolerance for things happening by chance. If we set α=.05 then we are saying that we are willing to say what we observe may be out of the ordinary, but unless it is something that occurs less that 5% of the time we will attribute it to chance.

Summary of One Sample t-test Possible Hypotheses:

Test Statistic:

Degrees of Freedom:

Assumption: The population from which the sample is drawn is normal or approximately normal.

2-Tailed Test Right-Tailed Left Tailed

Null hypothesis

Alternative hypothesis

33

34

P-values for One Sample t-testLet T be a t random variable with and Left-tailed test

*written as Prob<t in jmp Right-tailed test

*written as Prob>t in jmpTwo-tailed tests

*written as Prob>|t| in jmp

𝜶

𝜶

𝜶𝟐

𝜶𝟐

Solution to Example 1 re-visited: In a glaucoma study, the following intraocular pressure (mm Hg)

values were recorded from a sample of 21 elderly subjects. Based on this data, can we conclude that the mean intraocular pressure of the population from which the sample was drawn differs from 14 mm Hg?*

𝑦=15.6238 𝑠=3.383T: One sample t-test for A: i) SRS was used ii)The population from which the sample is drawn is

normal or approximately normal. H: is the true mean intraocular pressureM: Reject if p-value0.05 p-value= (calculated using JMP: Prob>|t|)

C: With a p-value less than 0.05, we reject . There is sufficient sample evidence to conclude that the true mean intraocular pressure differs from 14 mm Hg.

35

Hypothesis Test for a Single Mean in JMP• JMP Demonstration

• Open Pressure.jmp• AnalyzeDistribution• Complete the dialog box as

shown and select OK.• Select the red arrow next to

“Pressure” and select Test Mean.

• Complete Dialog box as shown and select OK.

• Select the red arrow next to “Pressure” and select Confidence Interval->0.95.

36

JMP Output

The normal quantile plot may also be created in JMP to check the normality assumption. The assumption is met if the points fall close to the red line.

Hypothesized Value

Actual Estimate

DF

Std Dev

14

15.6238

20

3.38288

Test Statistic

Prob > |t|

Prob > t

Prob < t

2.1997

t Test

0.0398*

0.0199*

0.9801

12 13 14 15 16

Test Mean

-1.64-1.28 -0.67 0.0 0.67 1.281.64

0.5 0.8 0.90.20.1 0.95

Normal Quantile Plot

37

Two Sample t-TestTwo sample t-tests are used to determine whether the population mean of one group is equal to, larger than or smaller than the population mean of another group.

38

Two Sample T-Test The major goal is to determine whether a

difference exists between two populations.

Examples:◦ Compare blood pressure for male and females.◦ Compare the proportion of smokers and

nonsmokers with lung cancer.◦ Compare weight before and after treatment.◦ Is the mean cholesterol of people taking drug A

lower than the mean cholesterol of people taking drug B?

39

Step 1: Formulate the Hypotheses The population means of the two groups are not equal.

H0: μ1 = μ2

Ha: μ1 ≠ μ2

The population mean of group 1 is greater than the population mean of group 2.H0: μ1 = μ2

Ha: μ1 > μ2

The population mean of group 1 is less than the population mean of group 2.H0: μ1 = μ2

Ha: μ1 < μ2

40

Step 2: Check the Assumptions The two samples are random and

independent.

The populations from which the samples are drawn are either normal or the sample sizes are large.

The populations have the same standard deviation.

41

Steps 3-5 Step 3: Calculate the test statistic

where

Step 4: Calculate the appropriate p-value. Step 5: Write a Conclusion.

42

𝒔𝒑=√ (𝒏𝟏−𝟏) 𝒔𝟏𝟐+ (𝒏𝟐−𝟏) 𝒔𝟐

𝟐

𝒏𝟏+𝒏𝟐−𝟐

Summary of two sample t-test Possible Hypotheses:

Test Statistic:

43

2-Tailed Test Right-Tailed Left Tailed

Null

Alternative

Assumption: The populations from which both samples are drawn are normal or approximately normal.

Two Sample Example A researcher would like to know whether the

mean sepal width of setosa irises is different from the mean sepal width of versicolor irises.

The researcher randomly selects 50 setosa irises and 50 versicolor irises and measures their sepal widths.

Step 1 Hypotheses:H0: μsetosa = μversicolor

Ha: μsetosa ≠ μversicolorhttp://en.wikipedia.org/wiki/Iris_flower_data_set

http://en.wikipedia.org/wiki/Iris_versicolor

44

JMP Steps 2-4:

JMP Demonstration:Analyze Fit Y By XY, Response: Sepal WidthX, Factor: Species

Means/ANOVA/Pooled t

Normal Quantile Plot Plot Actual by Quantile

45

JMP Output

Step 5 Conclusion: There is strong evidence (p-value < 0.0001) that the mean sepal widths for the two varieties are different.

setosa

versicolor

-2.33 -1.64-1.28 -0.67 0.0 0.67 1.281.64 2.33

0.5

0.8

0.9

0.2

0.1

0.0

2

0.9

8

Normal Quantile

46

Paired t-TestThe paired t-test is used to compare the population means of two groups when the samples are dependent.

47

Paired T-Test The objective of paired comparisons is to minimize

sources of variation that are not of interest in the study by pairing observations with similar characteristics.

Example:A researcher would like to determine if background noise causes people to take longer to complete math problems. The researcher gives 20 subjects two math tests one with complete silence and one with background noise and records the time each subject takes to complete each test.

48

Step 1: Formulate the Hypotheses

The population mean difference is not equal to zero. H0: μdifference = 0

Ha: μdifference ≠ 0 The population mean difference is greater than

zero. H0: μdifference = 0

Ha: μdifference > 0 The population mean difference is less than a zero.

H0: μdifference = 0

Ha: μdifference < 0

49

Step 2: Check the assumptions The sample is random.

The data is matched pairs.

The differences have a normal distribution or the sample size is large.

50

Steps 3-5

Where bar is the mean of the differences and sd is the standard deviations of the differences.

Step 4: Calculate the p-value.

Step 5: Write a conclusion.

Step 3: Calculate the test Statistic:

51

Summary of Paired t-test

Possible Hypotheses:

Test Statistic:

Degrees of Freedom:

2-Tailed Right Tailed Left Tailed

Null

Alternative

52

Assumption: The population of differences is normal or approximately normal.

Paired T-Test Example A researcher would like to determine

whether a fitness program increases flexibility. The researcher measures the flexibility (in inches) of 12 randomly selected participants before and after the fitness program.

Step 1: Formulate a HypothesisH0: μAfter - Before = 0

Ha: μ After - Before > 0

http://office.microsoft.com/en-us/images53

Paired T-Test Example Steps 2-4:

JMP Analysis:Create a new column of After – BeforeAnalyze DistributionY, Columns: After – Before

Normal Quantile Plot

Test MeanSpecify Hypothesized Mean: 0

54

JMP Output

Step 5 Conclusion: There is not evidence that the fitness program increases flexibility.

55

One-Way Analysis of Variance

ANOVA is used to determine whether three or more populations have different distributions.

56

One-Way ANOVA ANOVA is used to determine whether three

or more populations have different distributions.

A B C

Medical Treatment

57

ANOVA Strategy

The first step is to use the ANOVA F test to

determine if there are any significant differences

among the population means.

If the ANOVA F test shows that the population

means are not all the same, then follow up tests

can be performed to see which pairs of population

means differ.

58

One-Way ANOVA Model

i

ij

i

ij

ijiij

nj

ri

N

y

y

,,1

,,1

),0(~

groupith theofmean theis

levelfactor ith on the jth trial theof response theis

Where

2

In other words, for each group the observed value is the group mean plus some random variation.

59

One-Way ANOVA Hypothesis Step 1: We test whether there is a

difference in the population means.

equal. allnot are The :

: 210

ia

r

H

H

60

Step 2: Check ANOVA Assumptions The samples are random and independent of

each other. The populations are normally distributed. The populations all have the same standard

deviations.

The ANOVA F test is robust to the assumptions of normality and equal standard deviations.

61

Step 3: ANOVA F Test

Compare the variation within the samples to the variation between the samples.

A B C A B C

Medical Treatment

62

ANOVA Test Statistic

MSE

MSG

Groupswithin Variation

Groupsbetween Variation F

Variation within groups small compared with variation between groups → Large F

Variation within groups large compared with variation between groups → Small F

63

MSG

1-r

)(n)(n)(n

1 -r

SSGMSG

21r

222

211

yyyyyy

The mean square for groups, MSG, measures

the variability of the sample averages.

SSG stands for sums of squares groups.

64

MSE

1

)(

s

Wherer -n

1)s - (n1)s - (n 1)s - (n

r -n

SSE MSE

1i

2rr

222

211

i

n

jiij

n

yyi

Mean square error, MSE, measures the variability within the groups.

SSE stands for sums of squares error.

65

Steps 4-5 Step 4: Calculate the p-value.

Step 5: Write a conclusion.

66

ANOVA Example A researcher would like to determine if

three drugs provide the same relief from pain.

60 patients are randomly assigned to a treatment (20 people in each treatment).

Step 1: Formulate the HypothesesH0: μDrug A = μDrug B = μDrug C

Ha : The μi are not all equal.

http://office.microsoft.com/en-us/images

67

Steps 2-4

JMP demonstrationAnalyze Fit Y By X Y, Response: Pain

X, Factor: Drug

Normal Quantile Plot Plot Actual by Quantile

Means/ANOVA

68

JMP Output and Conclusion

Step 5 Conclusion: There is strong evidence that the drugs are not all the same.

50

55

60

65

70

75

Pa

in

Drug A Drug B Drug CDrug

Drug ADrug BDrug C

-2.33 -1.64-1.28 -0.67 0.0 0.67 1.281.64 2.33

0.5

0.8

0.9

0.2

0.1

0.0

2

0.9

8

Normal Quantile

69

Follow-Up Test The p-value of the overall F test indicates

that the level of pain is not the same for patients taking drugs A, B and C.

We would like to know which pairs of treatments are different.

One method is to use Tukey’s HSD (honestly significant differences).

70

Tukey Tests Tukey’s test simultaneously tests

JMP demonstrationOneway Analysis of Pain By Drug Compare Means All Pairs, Tukey HSD

'a

'0

:H

:H

ii

ii

for all pairs of factor levels. Tukey’s HSD controls the overall type I error.

71

JMP Output

The JMP output shows that drugs A and C are significantly different.

Drug C

Drug C

Drug B

Level

Drug A

Drug B

Drug A

- Level

5.850000

3.600000

2.250000

Difference

1.677665

1.677665

1.677665

Std Err Dif

1.81283

-0.43717

-1.78717

Lower CL

9.887173

7.637173

6.287173

Upper CL

0.0027*

0.0897

0.3786

p-Value

72

Two-Way Analysis of Variance

73

Two-Way ANOVA We are interested in the effect of two

categorical factors on the response. We are interested in whether either of the

two factors have an effect on the response and whether there is an interaction effect. ◦ An interaction effect means that the effect on the

response of one factor depends on the level of the other factor.

74

Interaction

Low High Dosage

Impr

ovem

ent

No Interaction

Drug A Drug B

Low High Dosage

Impr

ovem

ent

Interaction

Drug A Drug B

75

Two-Way ANOVA Model

ij

ijk

ij

j

i

ijk

ijkijjiijk

nk

bj

ai

N

y

y

,...,1

,,1

,,1

),0(~

Bfactor of leveljth theandA factor of levelith theofeffect n interactio theis )(

Bfactor of leveljth theofeffect main theis

Afactor of levelith theofeffect main theis

mean overall theis

level Bfactor jth theand levelA factor ith on the kth trial theof response theis

Where

)(

2

76

Two-Way ANOVA Example We would like to determine the effect of two

alloys (low, high) and three cooling temperatures (low, medium, high) on the strength of a wire.

JMP demonstrationAnalyze Fit ModelY: StrengthHighlight Alloy and Temp and click Macros Factorial to DegreeRun Model

http://office.microsoft.com/en-us/images

77

JMP Output

Conclusion: There is strong evidence of an interaction between alloy and temperature.

78

Conclusion The one sample t-test allows us to test

whether the population mean of a group is equal to a specified value.

The two-sample t-test and paired t-test allow us to determine if the population means of two groups are different.

ANOVA allows us to determine whether the population means of several groups are different.

79

SAS, SPSS and R For information about using SAS, SPSS and

R to do ANOVA:

http://www.ats.ucla.edu/stat/sas/topics/anova.htm

http://www.ats.ucla.edu/stat/spss/topics/anova.htm

http://www.ats.ucla.edu/stat/r/sk/books_pra.htm

80

References Fisher’s Irises Data (used in one sample and

two sample t-test examples).

Flexibility data (paired t-test example):Michael Sullivan III. Statistics Informed Decisions Using Data. Upper Saddle River, New Jersey: Pearson Education, 2004: 602.

81