1

Type II Error, Power and Sample Size

Calculations

Type II Error, Power and Sample Size

Calculations

Chapter 23 Inference for Means: Part 2

2

Hypothesis Testing for , Type II Error Probabilities (Right-tail example)

• Example– A new billing system for a department store will be cost-

effective only if the mean monthly account is more than $170.

– A sample of 400 accounts has a mean of $174 and s = $65.

– Can we conclude that the new system will be cost effective?

3

• Hypotheses– The population of interest is the credit accounts at

the store.– We want to know whether the mean account for all

customers is greater than $170.

HA : m > 170

– Where m is the mean account value for all customers

Example (cont.)

H0 : m = 170

4

–Test statistic:H0 : m = 170HA : m > 170

174 1701.23

65 400

xts n

Example (cont.)

174, 65x s

5

399-value ( 1.23) .1097P P t

01.23t

P-value: The probability of observing a value of the test statistic as extreme or more extreme then t = 1.23, given that m = 170 is…

Example (cont.)

t399

Since the P-value > .05, we conclude that there is not sufficient evidence to reject H0 : =170.

Type II error is possible

6

Calculating , the Probability of aType II Error

• Calculating for the t test is not at all straightforward and is beyond the level of this course– The distribution of the test statistic t is quite

complicated when H0 is false and HA is true– However, we can obtain very good approximate

values for using z (the standard normal) in place of t.

7

Calculating , the Probability of aType II Error (cont.)

• We need to1. specify an appropriate significance level ;2. Determine the rejection region in terms of z3. Then calculate the probability of not being in the

rejection when = 1, where 1 is a value of that makes HA true.

8

–Test statistic:H0 : m = 170HA : m > 170

Choose = .05Rejection region in terms of z: z > z.05 = 1.645

Example (cont.) calculating

rejection region in terms of :

1701.645

65

40065

170 1.645 175.34.400

x

xz

x

a= 0.05

175.34170

9

Express the rejection region directly, not in standardized terms

175.34

a=.05

m= 170

Example (cont.) calculating

– The rejection region with a = .05.

34.175x

Do not reject H0

=m 180

HA: m = 180

H0: m = 170

Specify the alternative value

under HA.

– Let the alternative value be m = 180 (rather than just m>170)

10

175.34

a=.05

m= 170

Example (cont.) calculating

34.175x =m 180

H1: m = 180

H0: m = 170

–A Type II error occurs when a false H0 is not rejected. Suppose =180, that is, H0 is false.

A false H0……is not rejected

11

175.34m= 170

Example (cont.) calculating

=m 180

H1: m = 180

H0: m = 170

0(180) ( 175.34 )

( 175.34 180)

P x given that H is false

P x given that

0764.)40065

18034.175z(P

Power when =180 =

1-(180)=.9236

12

• Increasing the significance level ,a decreases the value of , b and vice versa.

Effects on b of changing a

m= 170 =m 180

a1b1a2 >b2 <

13

• A hypothesis test is effectively defined by the significance level a and by the sample size n.

• If the probability of a Type II error b is judged to be too large, we can reduce it by– increasing a, and/or– increasing the sample size.

Judging the Test

14

• Increasing the sample size reduces b

Judging the Test

By increasing the sample size the standard deviation of the sampling distribution of the mean decreases. Thus, the cutoff value of for the rejection region decreases.

Recall : ,x s

RR z z or x zs n n

15Lx =m 180m= 170

Judging the Test

Lx

Note what happens when n increases:

Lx LxLx Lx

a does not change,but b becomes smaller

• Increasing the sample size reduces b

Recall : ,x s

RR z z or x zs n n

16

• Increasing the sample size reduces b• In the example, suppose n increases from 400 to

1000.65

170 1.645 173.381000

173.38 180( ) ( 3.22) 0

65 1000

sx z

n

P Z P Z

Judging the Test

• a remains 5%, but the probability of a Type II error decreases dramatically.

17

A Left - Tail Test• Self-Addressed Stamped Envelopes.

– The chief financial officer in FedEx believes that including a stamped self-addressed (SSA) envelope in the monthly invoice sent to customers will decrease the amount of time it take for customers to pay their monthly bills.

– Currently, customers return their payments in 24 days on the average, with a standard deviation of 6 days.

– Stamped self-addressed envelopes are included with the bills for 75 randomly selected customers. The number of days until they return their payment is recorded.

18

• The parameter tested is the population mean payment period (m) for customers who receive self-addressed stamped envelopes with their bill.

• The hypotheses are:H0: m = 24H1: m < 24

• Use = .05; n = 75.

A Left - Tail Test: Hypotheses

19

• The rejection region: • t < t.05,74 = 1.666• Results from the 75 randomly selected

customers:

A Left - Tail Test: Rejection Region

22.95 days, 6 daysx s

20

• The test statistic is:

A Left -Tail Test: Test Statistic

.05 1.666t t t

22.95 241.52

6 75

xts n

We do not reject the null hypothesis. Note that the P-value = P(t74 < -1.52) = .066.

Since our decision is to not reject the null hypothesis,A Type II error is possible.

Since the rejection region is

21

• The CFO thinks that a decrease of one day in the average payment return time will cover the costs of the envelopes since customer checks can be deposited earlier.

• What is (23), the probability of a Type II error when the true mean payment return time is 23 days?

Left-Tail Test: Calculating , the Probability of a Type II Error

22

–Test statistic:H0 : m = 24HA : m < 24

Choose = .05Rejection region in terms of z: z < -z.05 = -1.645

Left-tail test: calculating (cont.)

rejection region in terms of :

241.645

6

756

24 1.645 22.86.75

x

xz

x

a = 0.05

22.86 24

23

Express the rejection region directly, not in standardized terms

a=.05

Left-tail test: calculating (cont.)

– The rejection region with a = .05.

22.86x

Do not reject H0

22.86 =m 24

H0: m = 24

m= 23

HA: m = 23Specify the alternative value

under HA.

– Let the alternative value be m = 23 (rather than just m < 24)

24

22.86 m= 23

Left-tail test: calculating (cont.)

=m 24

HA: m = 23

H0: m = 24

0(23) (fail to reject : 24 when 23)

( 22.86 when 23)

P H

P x

22.86 23.58

6 75P z

Power when =23 is 1-(23)=1-.58=.42

a=.05

25

A Two - Tail Test for • The Federal Communications Commission

(FCC) wants competition between phone companies. The FCC wants to investigate if AT&T rates differ from their competitor’s rates.

• According to data from the (FCC) the mean monthly long-distance bills for all AT&T residential customers is $17.09.

26

A Two - Tail Test (cont.)• A random sample of 100 AT&T customers is

selected and their bills are recalculated using a leading competitor’s rates.

• The mean and standard deviation of the bills using the competitor’s rates are

• Can we infer that there is a difference between AT&T’s bills and the competitor’s bills (on the average)?

$17.55, $3.87x s

27

• Is the mean different from 17.09?

• n = 100; use = .05

H0: m = 17.09

: 17.09AH

A Two - Tail Test (cont.)

28

0

/a 2 = 0.025 /a 2 = 0.025

17.55 17.09

3.87.1

1001 9

xts n

-t /2a = -1.9842 t /2a = 1.9842

Rejection region

Rejection region

A Two – Tail Test (cont.)

.025,99 .025,99

1.9842 1.9842

t t or t t

t or t

t99

29

/a 2 = 0.025 /a 2 = 0.02517.55 17.09

1.193.87 100

xts n

-t /2a = -1.9842 t /2a = 1.9842

There is insufficient evidence to conclude that there is a difference between the bills of AT&T and the competitor.

-1.19

Also, by the P-value approach:The P-value = P(t < -1.19) + P(t > 1.19) = 2(.1184) = .2368 > .05

1.190

A Two – Tail Test: Conclusion

A Type II error is possible

30

• The FCC would like to detect a decrease of $1.50 in the average competitor’s bill. (17.09-1.50=15.59)

• What is (15.59), the probability of a Type II error when the true mean competitor’s bill is $15.59?

Two-Tail Test: Calculating , the Probability of a Type II Error

31

/a 2 = 0.025

17.0916.33 17.85

/a 2 = 0.025

Rejection regionTwo – Tail Test: Calculating (cont.)

.025 .025

1.96 1.96

z z or z z

z or z

rejection region in terms of :

17.091.96

3.87100

3.8717.09 1.96

10016.33

17.091.96

3.87100

3.8717.09 1.96

10017.85

x

xz

x

x

xz

x

x

Do not reject H0

Reject H0

3216.63

m= 15.59

Two – Tail Test: Calculating (cont.)

=m 17.09

HA: m = 15.59

H0: m = 17.09

(15.59) (16.33 17.85 given that 15.59)P x 16.33 15.59 15.59 17.85 15.59

3.87 100 3.87 100 3.87 100

(1.912 5.84) .028

xP

P z

Power when =15.59 is 1-(15.59)=.972

17.85

General formula: Type II Error Probability (A) for a Level Test

33

00

00

0 00 /2 /2

:

: 1

:

AA

AA

A AA

H P z zn

H P z zn

H P z z P z zn n

Sample Size n for which a level test also has (A) =

34

2

0

2

/2

0

( )for a 1-tailed (right or left) test

( ) for a 2-tailed test (approx. solution)

A

A

z z

nz z