analyzing significant differences between means dr. k. a. korb university of jos

Analyzing Significant Differences between Means

Dr. K. A. KorbUniversity of Jos

Outline

Types of Statistics Purpose of Inferential Statistics Null hypotheses Interpreting Inferential Statistics Reporting Inferential Statistics


Types of Statistics

Chi-Square: The purpose of a chi-square is to test whether frequency counts are distributed differently for different samples ONLY use the Chi-Square if you have

categorical data (i.e. teaching at a private or governmental school)

Do NOT use if you have continuous scores (i.e. Likert Scale, grades, attendance)


Types of Statistics

ANOVA, ANCOVA, t-test: Determine whether average scores differ significantly Use t-test if you only have 2 groups Use an ANOVA if you have more than 2

groups. If you get a significant result, follow up with Tukey’s HSD to determine whether groups are significantly different from each other.

Use an ANCOVA if you have pre- and post-test data

These three statistics are also called Inferential Statistics.


Types of Statistics

Correlation, Regression: Determines the strength and direction of a relationship between two variables Use if you want to compare two

continuous variables (i.e., grades and classroom attendance)


ANCOVA: Analysis of Covariance

ANOVA: Analysis of Variance

t-test CorrelationRegression

Yes

No

No

Yes

Yes

No

NoYes

Do you have more than 2 variables?

Do you have more than 2 independent variables?

Do you have pre- and post-

tests?

Do you have categorical data only?

Chi-Square(χ2)

Yes

No

Do you have independent

and dependent variables?


Purpose of Inferential Statistics

In educational research, we can never sample the entire population that we want to generalize our results to. Instead, we choose a sample of the population Then we want to make inferences about the

population based on the results of our study based on the sample.

The purpose of inferential statistics is to determine whether the findings from our sample can generalize to the entire population or that our findings were simply the result of chance.



Imagine a room full of socks. You want to determine whether there are more white socks than green socks in the room. However, there are too many socks to count,

so you want to take a sample of socks and draw a conclusion about whether there are more white socks based on your sample.

The purpose of inferential statistics is to determine whether the colors chosen in your sample likely reflects the entire room or if your results were due to chance.



What factors will determine whether the sample of socks adequately represents the entire room? First, the size of our sample.

If we only pick two socks, they would very likely not represent the entire room.

The larger our sample is, the more representative our sample will be of the entire room and the more accurately our conclusions will be for the entire room.

This is why when conducting experiments, the larger the sample is, the better.

With large samples, our results will more likely reflect the entire population.



What factors will determine whether the sample of socks adequately represents the entire room? Second, the size of the difference between white

and green socks in the entire room will affect our results.

If there are only two more white socks in the entire room, then we likely will not be able to determine this difference in our sample.

If there are thousands more white socks in the entire room, we should find this in the sample.

Therefore, when conducting studies, try to make your treatment very effective.

Very effective treatments result in a large change in your dependent variable and enable you to find a significant difference in your sample.



To summarize, you will most likely find significant results in your study if you:

1. Have a large sample size2. Have an effective treatment that

results in a large change in your dependent variable



For an education example, you want to determine if there is a difference between boys and girls in Nigeria on their science ability. You cannot sample all of the boys and girls in

Nigeria. Instead, draw a sample of boys and girls and

test their science ability. Then you will use inferential statistics to

determine whether any mean difference between boys’ and girls’ science ability scores can generalize to all Nigerian students or if your results are due to chance.



In this study, you will most likely find significant results if: You have a large sample of boys and

girls A large difference in science ability

truly exists between boys and girls


Null hypotheses

Null hypotheses: Predicting that no average difference between the groups will be found You actually want to prove that differences do

exist between groups. Because you do not want to say that

differences exist when they really do not, you begin with the assumption that your results ARE due to chance.

Then if your results are truly significant – there is little likelihood that your results are due to chance – you can confidently say that differences do exist between the groups.


Null hypotheses

When writing your null hypotheses, be sure to include: Significant differences in what?

This will be your dependent variable Significant differences between what two

groups? This will be your independent variable

There are NO significant differences in science ability (in what) between boys and girls (what two groups).


Interpreting Inferential Statistics

t-tests: Use if you are comparing two groups One treatment and one control group Boys and girls Pre- and Post-test for one group of people

ANOVA: Use if you are comparing more than two groups Two treatments and one control group Children in Primary 2, 3, and 4 High, medium, and low socioeconomic students

ANCOVA: Use if you are comparing pre- and post-tests for more than one group Pre- and Post-tests for treatment and control groups Pre- and Post-tests for boys and girls



When you calculate inferential statistics, you will get three important numbers Either t (for a t-test) or F (for ANOVA and

ANCOVA): A number that tells you how large the difference is between your two groups

Degrees of freedom (df): One number (for a t-test) or two numbers (for ANOVA or ANCOVA) that depends on your sample size.

Using tables, the t or F is then compared to the df to determine a probability

Probability (p): This tells you whether your results are due to chance or not.



In general, statisticians recommend that if the p value is less than .05, your results are significant. The significance level that you set prior to

conducting your statistics, p < .05, is the alpha (α)

If your calculated p value is less than your alpha, then your null hypothesis is rejected.

Significant p-values indicate that your results are NOT due to chance and thus represent a meaningful difference in the population.

You can therefore conclude that a difference does exist between your two groups in the population.


Reporting Inferential Statistics

For a t-test: Girls performed significantly better

than boys in their science ability tests (t(43) = 4.61, p <.001).

43 = degrees of freedom 4.61 = t value .001 = p value


Reporting Inferential StatisticsScience Ability

Analyzed using t-test

0

10

20

30

40

50

60

Boys Girls

Ave

rage

on E

xam

.

This chart of average science ability scores clearly shows that girls have more science ability than boys.

The chart can easily be constructed in Excel.


Reporting Inferential Statistics

For either ANOVA or ANCOVA: Significant differences were found in

science achievement between high, medium, and low socioeconomic status students (F(2,136) = 31.86, p < .001).

2, 136 = Degrees of Freedom. ANOVAs and ANCOVAs will always have 2

numbers in the degrees of freedom. In this example, 2 is a function of the number of groups you are comparing and 136 is a function of the sample size.

31.86 = F value .001 = p value



Analyzed using One-Way Anova

0

10

20

30

40

50

60

Low Medium High

Socioeconomic Status

Ave

rage

Sco

re o

n E

xam

This chart shows that science ability increases as SES increases.Dr. K. A. Korb

University of Jos


Analyzed using Factorial Anova (3x2)

0

10

20

30

40

50

60

Low Medium High

Socioeconomic Status

Ave

rage

Sco

re o

n E

xam

Boys

Girls

This chart that science ability increases as SES increases and also that girls tend to do better than girls.Dr. K. A. Korb

University of Jos

analyzing significant differences between means dr. k. a. korb university of jos

Documents

sample of socks

korb university of jos

entire room

large sample of boys

large sample size

types of statistics

types of statistics

white socks