week 8 - effect size

7/27/2019 Week 8 - Effect Size

1/16

Effect SizeCohens d & r2


2/16

Statistical versus

Practical Significance With Z-tests and t-tests, we are evaluating the

statistical significance of our result. Can we

reject the null hypothesis? Practical significance addresses a different

question: Assuming the result is statistically

significant, is it practically significant?

Is the effect big enough to matter?

Effect Size is a measure of the magnitude of the

effect. How big an effect was it?


3/16

Measures of Effect Size:

Cohens d, and r2

Cohen's d = standardized

mean difference betweengroups

One sample t-test:

OR

Two-sample t-test

Cohens d: Small effect = .20

Med. effect = .50

Large effect = .80

r2 = proportion of variancein the DV that is

accounted for by the IV

r2 = t2 / (t2 + df)

r2 : Small effect = .01

Med. effect = .09

Large effect = .25

sXd /)( 1

d (X1 X2) /s2p


4/16

Subjective Life

Expectancy Accuracy of Subj Life Expectancies: A subjective

life expectancy is an individuals estimate of what

age he or she expects to live to. One interestingquestion is how subjective life expectancies

compare with actual life expectancies as

represented by actuarial predictions. Financial

experts worry that many people underestimate

how long they will live and in doing so fail to plan

or save enough money to carry them throughout

their lives.


5/16

Subjective Life

Expectancy In a study of this issue, Robbins, (1988) asked 49

women to indicate their expected age in response

to the question: Approximately how long do youexpect to live? These estimates were then

compared with actuarial predictions from the

National Center for Health Statistics. Results

showed that mean subjective life expectancy for

females in the samples was 77.2 years with a

standard deviation of 3.4 years. The actuarial

prediction was a mean of 78.2 years. Do females

significantly underestimate their lifespans?


6/16

Descriptive Statistics &

Hypotheses H0: SubLE = 78.2

H1: SubLE < 78.2

Sample N = 49

= 78.2 years

Sample Mean = 77.2

years

s = 3.4 years


7/16

Expectations Sample distribution of the Mean if we did this experiment over and

over again, each time drawing 49 females, what values for mean lifeexpectancy would I expect?

Mean of means = = 78.2

SE = 3.4/(49) = 0.49 Approx normal (N > 30)

Decision rule: p = .05 but now fatness of tails change w/ samplesize

Crit t (48) estimated by being more conservative

Crit t (40) = 2.021

Calculate obtained and find critical test stats:

Obt t (48) = (77.2 78.2) / 0.49 = -2.04


8/16

Substantive Conclusion On average, women significantly underestimate their life

expectancy compared to actuarial predictions. Thus, the financial

experts may be correct they may not be planning ahead

sufficiently


9/16

Cohens d Cohen's d is commonly used measure of effect size

Used to indicate the standardized difference between 2 means.

The difference between two means divided by a standard deviationfor the data

Cohens d: Small effect = .20

mid effect = .50

large effect = .80

sXd /


10/16

How Cohens d Relates to the

Example

d = | | / s = |77.2 78.2| / 3.4 = .29

Here, we rejected H0 and said women significantly underestimate

their life spans. But the effect size is relatively small, so we might

have profited from a relatively large sample of 49.

So, it does have an effect, but how much does it actually tell us about

how much life span estimates are affected?

Effect size not influenced by n

Used before conducting research to see how much power (i.e., n) we

need, based on similar research effect size(s)

X


11/16

Graduates Optimism

Example A psychologist has prepared an Optimism Test that is

administered yearly to graduating college seniors. The test

measures how each graduating class feels about its future thehigher the score, the more optimistic the class. Administratorsare concerned that this years class might be significantly lessoptimistic about the job market than the class 5 years ago due to

the economy. Five years ago, the graduating class had a mean

score of 12.3. A randomly-selected sample of 9 seniors from this

years class was selected and tested. Their scores are:


12/16

XX

10X


13/16

Graduates Optimism

Example IV & DV

H0 & H1

What do you expect under null

Stats

Conclusion


14/16

Graduates Optimism

Example DV = optimism scores IV = surveyed before or now

H0: Now = 12.3 (Before) H1: Now < 12.3

Expect under null

Mean of means = = 12.3

SE = 1.14

Approaches normal (above N=30)

Decision rule based on =.05 critical t(8) = 2.306

Stats obtained t(8) = (10-12.3)/1.14 = -2.02

Conclusion

Substantive: Based on this sample, there is insufficient evidence tosuggest this years class is any more or less optimistic than the class from5 years ago.


15/16

Effect size in t-test Cohens d

d = |mean diff|/s = |10-12.3|/3.43 = 0.67

Thus, even though unable to reject H0, effect size suggests that

there is something going on. That there has been a decrease in

optimism in graduating seniors compared to 5 years ago. We

just had very low power to detect the effect


16/16

Power Is the probability that the test will reject a false null hypothesis

(that it will not make a type II error).

As power increases, the chances of a Type II error decrease.

Increases chance of Type I error.

It can also be used to calculate the minimum

effect size that is likely to be detected in a study using a given sample

size. sample size required to accept the outcome of a statistical test with a

particular level of confidence.

week 8 - effect size

Documents