statistical power 1. first: effect size the size of the distance between two means in standardized...
TRANSCRIPT
![Page 1: Statistical Power 1. First: Effect Size The size of the distance between two means in standardized units (not inferential). A measure of the impact of](https://reader035.vdocuments.net/reader035/viewer/2022070409/56649ea05503460f94ba3533/html5/thumbnails/1.jpg)
1
Statistical Power
![Page 2: Statistical Power 1. First: Effect Size The size of the distance between two means in standardized units (not inferential). A measure of the impact of](https://reader035.vdocuments.net/reader035/viewer/2022070409/56649ea05503460f94ba3533/html5/thumbnails/2.jpg)
2
First:Effect Size
• The size of the distance between two means in standardized units (not inferential).
• A measure of the impact of an intervention based on the distributions of the samples in your study.
• We use two measures of effect size– Cohen’s d– Eta Squared (η2)
![Page 3: Statistical Power 1. First: Effect Size The size of the distance between two means in standardized units (not inferential). A measure of the impact of](https://reader035.vdocuments.net/reader035/viewer/2022070409/56649ea05503460f94ba3533/html5/thumbnails/3.jpg)
3
Effect Size: Cohen’s d• Cohen’s d equals the difference in the means divided by the
average of the standard deviations.• It describes the distance between the means in units of pooled
standard deviation (remember z scores?).• It is a standardized measure of the impact of a statistically
significant intervention (independent variable).
X2X1
![Page 4: Statistical Power 1. First: Effect Size The size of the distance between two means in standardized units (not inferential). A measure of the impact of](https://reader035.vdocuments.net/reader035/viewer/2022070409/56649ea05503460f94ba3533/html5/thumbnails/4.jpg)
4
Effect Size: Eta Squared (η2)• Eta squared is the ratio of between group variance (impact of the
intervention) to total variance. • As the between group variance becomes a larger portion of the total
variance (more intervention impact), eta squared gets closer to 1.• Eta squared is a standardized measure of the explanatory power of
the independent variable.
Between group variance
Total variance
Within group variance sum of squares betweensum of squares totalη2 =
![Page 5: Statistical Power 1. First: Effect Size The size of the distance between two means in standardized units (not inferential). A measure of the impact of](https://reader035.vdocuments.net/reader035/viewer/2022070409/56649ea05503460f94ba3533/html5/thumbnails/5.jpg)
5
Effect Size
Label d r r2 or η2
Extremely large effect 2.0 .707 .500
Very large effect 1.5 .600 .360
Large effect 0.8 .371 .138
Medium effect 0.5 .243 .059
Small effect 0.2 .100 .010
• Although all the measures of effect size represent the impact of the independent variable in somewhat different ways, they are equivalent.
• For an expanded version see the Effect Size table on the website
![Page 6: Statistical Power 1. First: Effect Size The size of the distance between two means in standardized units (not inferential). A measure of the impact of](https://reader035.vdocuments.net/reader035/viewer/2022070409/56649ea05503460f94ba3533/html5/thumbnails/6.jpg)
6
Effect Size
• In the real world how much difference did the independent variable make?
• Effect size is not based on inference. It is based on observed measures.
X2X1
![Page 7: Statistical Power 1. First: Effect Size The size of the distance between two means in standardized units (not inferential). A measure of the impact of](https://reader035.vdocuments.net/reader035/viewer/2022070409/56649ea05503460f94ba3533/html5/thumbnails/7.jpg)
Sample Mean
Sample Distribution
Sampling Distribution Mean for a Given Group Size
Sampling Distribution for a Given Group Size
Now:Inferential Mistakes
![Page 8: Statistical Power 1. First: Effect Size The size of the distance between two means in standardized units (not inferential). A measure of the impact of](https://reader035.vdocuments.net/reader035/viewer/2022070409/56649ea05503460f94ba3533/html5/thumbnails/8.jpg)
Avoiding Type II Errors
.05
Sampling Distribution of the Mean Theoretical distribution based on randomly selected groups of a given size. Means in this area
would appear randomly less than 5% of the time.
![Page 9: Statistical Power 1. First: Effect Size The size of the distance between two means in standardized units (not inferential). A measure of the impact of](https://reader035.vdocuments.net/reader035/viewer/2022070409/56649ea05503460f94ba3533/html5/thumbnails/9.jpg)
Avoiding Type II Errors
.05
Sometimes a mean score would not be identified as significant because it is likely to appear randomly more than 5% of the time.
Sometimes that mean score represents a real world change in what is being measured but the difference isn’t enough to be significant.
Type II Error
![Page 10: Statistical Power 1. First: Effect Size The size of the distance between two means in standardized units (not inferential). A measure of the impact of](https://reader035.vdocuments.net/reader035/viewer/2022070409/56649ea05503460f94ba3533/html5/thumbnails/10.jpg)
Avoiding Type II Errors
.05
Now a larger group size moves the point at which the alpha level appears and something that wasn’t significant becomes so.
Using larger groups reduces type II errors. .05
![Page 11: Statistical Power 1. First: Effect Size The size of the distance between two means in standardized units (not inferential). A measure of the impact of](https://reader035.vdocuments.net/reader035/viewer/2022070409/56649ea05503460f94ba3533/html5/thumbnails/11.jpg)
Avoiding Type II Errors
.05
Great. Using larger group sizes helps reduce type II errors.
But, the cost is that developing large samples is difficult.
We need to figure out how big the sample size needs to be to reasonably reduce type II errors but still keep the group as small as possible.
![Page 12: Statistical Power 1. First: Effect Size The size of the distance between two means in standardized units (not inferential). A measure of the impact of](https://reader035.vdocuments.net/reader035/viewer/2022070409/56649ea05503460f94ba3533/html5/thumbnails/12.jpg)
12
Power
• Power is defined as the probability of finding significance if it exists (avoiding type II errors).
• Eighty percent (.80) is accepted as a reasonable target power.
• If non-random change occurs it has an 80% probability of being observed.
![Page 13: Statistical Power 1. First: Effect Size The size of the distance between two means in standardized units (not inferential). A measure of the impact of](https://reader035.vdocuments.net/reader035/viewer/2022070409/56649ea05503460f94ba3533/html5/thumbnails/13.jpg)
13
Power
• There are 2 ways to use power calculations.• First, they can be used to figure out
appropriate sample sizes for a study.• Second, they can be used to evaluate the use of
a specific sample size after a study has been completed.
![Page 14: Statistical Power 1. First: Effect Size The size of the distance between two means in standardized units (not inferential). A measure of the impact of](https://reader035.vdocuments.net/reader035/viewer/2022070409/56649ea05503460f94ba3533/html5/thumbnails/14.jpg)
14
Power and Effect Size
• If a study shows larger effect sizes, smaller sample sizes will still be expected to show significance.
• Conversely, smaller effect sizes would require larger sample sizes.
• Fortunately, all of this can be read offof a table.
X2X1
![Page 15: Statistical Power 1. First: Effect Size The size of the distance between two means in standardized units (not inferential). A measure of the impact of](https://reader035.vdocuments.net/reader035/viewer/2022070409/56649ea05503460f94ba3533/html5/thumbnails/15.jpg)
15
Power and Effect SizeChoosing Sample Sizes When Designing a StudyUsing Power to Explain Results
![Page 16: Statistical Power 1. First: Effect Size The size of the distance between two means in standardized units (not inferential). A measure of the impact of](https://reader035.vdocuments.net/reader035/viewer/2022070409/56649ea05503460f94ba3533/html5/thumbnails/16.jpg)
16
While we are here …
• Remember we have talked about inferential errors when something appears significant but it really wasn’t?
• Type I errors
![Page 17: Statistical Power 1. First: Effect Size The size of the distance between two means in standardized units (not inferential). A measure of the impact of](https://reader035.vdocuments.net/reader035/viewer/2022070409/56649ea05503460f94ba3533/html5/thumbnails/17.jpg)
Avoiding Type I Errors
.05
Every 100 times a mean appear in the .05 area, 5 of them would have occurred randomly.
That means we would identify something as not random (significant) 5 times out of 100 and be wrong.
Type I Error
![Page 18: Statistical Power 1. First: Effect Size The size of the distance between two means in standardized units (not inferential). A measure of the impact of](https://reader035.vdocuments.net/reader035/viewer/2022070409/56649ea05503460f94ba3533/html5/thumbnails/18.jpg)
Avoiding Type I Errors
.01
Solution:Move the alpha level to .01
That means we would identify something as not random (significant) 1 time out of 100 and be wrong.
Less chance of a Type I Error
![Page 19: Statistical Power 1. First: Effect Size The size of the distance between two means in standardized units (not inferential). A measure of the impact of](https://reader035.vdocuments.net/reader035/viewer/2022070409/56649ea05503460f94ba3533/html5/thumbnails/19.jpg)
Avoiding Type I Errors
.05
But this is social science and there is no good reason to make it this difficult to demonstrate significance. .05 is a reasonable alpha level.
![Page 20: Statistical Power 1. First: Effect Size The size of the distance between two means in standardized units (not inferential). A measure of the impact of](https://reader035.vdocuments.net/reader035/viewer/2022070409/56649ea05503460f94ba3533/html5/thumbnails/20.jpg)
Avoiding Type I Errors
.05
With larger group sizes a given point moves to a smaller probability of appearing.
Using larger groups reduces type I errors.
.05
![Page 21: Statistical Power 1. First: Effect Size The size of the distance between two means in standardized units (not inferential). A measure of the impact of](https://reader035.vdocuments.net/reader035/viewer/2022070409/56649ea05503460f94ba3533/html5/thumbnails/21.jpg)
21
Sample Sizes
• We know that using larger sample sizes is statistically powerful but life isn’t that simple.
• Whenever possible use Power Analysis to help you be more confident you will find something if it is there.
• When things don’t appear to be significant at least now Power Analysis gives you something else to talk about to suggest what might be done to improve the quality of your data.
![Page 22: Statistical Power 1. First: Effect Size The size of the distance between two means in standardized units (not inferential). A measure of the impact of](https://reader035.vdocuments.net/reader035/viewer/2022070409/56649ea05503460f94ba3533/html5/thumbnails/22.jpg)
22
Examples1. Most of the studies in your lit review are
showing medium effects around 0.4 Cohen’s d. You want to be 90% sure you find non-random effects if they are there. Approximately how big does your sample need to be?
2. In your study you showed mean differences of 0.4 Cohen’s d but groups were not significantly different. Your sample size was 30. What was the probability of finding significant differences if they were there?
![Page 23: Statistical Power 1. First: Effect Size The size of the distance between two means in standardized units (not inferential). A measure of the impact of](https://reader035.vdocuments.net/reader035/viewer/2022070409/56649ea05503460f94ba3533/html5/thumbnails/23.jpg)
Analysis
![Page 24: Statistical Power 1. First: Effect Size The size of the distance between two means in standardized units (not inferential). A measure of the impact of](https://reader035.vdocuments.net/reader035/viewer/2022070409/56649ea05503460f94ba3533/html5/thumbnails/24.jpg)
Inferential Statistics
• Assumptions– Dependent variable is an interval measure of one
characteristic of a group.– Tests are based on knowing or assuming the
distribution of a population.– Statistics demonstrate if comparison samples are
from the same population.
![Page 25: Statistical Power 1. First: Effect Size The size of the distance between two means in standardized units (not inferential). A measure of the impact of](https://reader035.vdocuments.net/reader035/viewer/2022070409/56649ea05503460f94ba3533/html5/thumbnails/25.jpg)
Testing Group DifferencesIndependent Dependent Test
1 (1 group) 1 measure 2 times (intervention in between) Paired t-test
1 (2 groups) 1 measure (usually after the intervention) Independent samples t-test
1 (1 group) 1 measure 3 or more times (usually two after the intervention)
Repeated measures ANOVA
1 (2 or more groups) 1 measure (usually after the intervention) Single factor
ANOVA
Non-Parametric
1 group 2 non-interval measures Chi-square
EZA
EZA
EZA
EZA
EZA
![Page 26: Statistical Power 1. First: Effect Size The size of the distance between two means in standardized units (not inferential). A measure of the impact of](https://reader035.vdocuments.net/reader035/viewer/2022070409/56649ea05503460f94ba3533/html5/thumbnails/26.jpg)
Post Hoc TestsTest
ANOVA Similar group sizes Tukey’s
ANOVA Dissimilar group sizes Scheffe’s
EZA
![Page 27: Statistical Power 1. First: Effect Size The size of the distance between two means in standardized units (not inferential). A measure of the impact of](https://reader035.vdocuments.net/reader035/viewer/2022070409/56649ea05503460f94ba3533/html5/thumbnails/27.jpg)
Practical Significance(not inferential)
Test
Pooled Standard Deviation Cohen’s d
Ratio of variances Eta Squared EZA
![Page 28: Statistical Power 1. First: Effect Size The size of the distance between two means in standardized units (not inferential). A measure of the impact of](https://reader035.vdocuments.net/reader035/viewer/2022070409/56649ea05503460f94ba3533/html5/thumbnails/28.jpg)
Testing Group Differences(Things We Haven’t Done)
Independent Dependent Test
2 (2 or more groups) 1 measure Factorial (2-way ANOVA) Shows interaction
between groups2 (2 or more groups)
1 measure 2 or more times
ANCOVA (Analysis of Co-Variance) Allows for control of instance of dependent measure
2 or more variables
2 or more measures MANOVA (Multiple Analysis of Variance)
2 or more variables
2 or more measures 2 or more times
MANCOVA (Multiple Analysis of Co-Variance) Allows for control of instance of dependent measure.
Non-Parametric2 (2 or more groups) Non-interval Kruskal-Wallis (ranked sum)
EZA?
![Page 29: Statistical Power 1. First: Effect Size The size of the distance between two means in standardized units (not inferential). A measure of the impact of](https://reader035.vdocuments.net/reader035/viewer/2022070409/56649ea05503460f94ba3533/html5/thumbnails/29.jpg)
Correlational Statistics
• Assumptions– The relationship among the measures of two
characteristics is linear.– Compared measures come from individuals in the
same population– Correlations are not causal
![Page 30: Statistical Power 1. First: Effect Size The size of the distance between two means in standardized units (not inferential). A measure of the impact of](https://reader035.vdocuments.net/reader035/viewer/2022070409/56649ea05503460f94ba3533/html5/thumbnails/30.jpg)
How Do Variables Relate?Comparison Test
Association of 2 or more interval measures Pearson’s r
Association of 2 measures at least one of which is not interval (ranked comparisons)
Spearman’s ρ (rho)
Measure of internal consistency Cronbach’s alpha
Prediction based on association of 2 measures Linear Regression
Things We Haven’t DoneAssociation of 3 or more measures at least one of which is not interval (ranked comparisons)
Kendall’s τ (tau)
Prediction based on 3 or more associations Multiple Regression
Finding relationships among items in a set of items (data reduction)
Factor Analysis
EZA
EZA?
EZA