topic 5: parametric analyses - uppsala university
TRANSCRIPT
QM STEM Ed 20181
Quantitative Methods in STEM
Education Research
Topic 5: Parametric analyses
Judy Sheard
Faculty of Information Technology
Monash University, Australia
QM STEM Ed 20182
Overview of topic 5
Parametric tests
Assumptions
t-distribution
Degrees of freedom
t-test
One and two tailed tests
ANOVA
Post hoc analysis
QM STEM Ed 20183
Common parametric statistical
tests
Statistical test Hypothesis tested
t-test
(independent-
samples)
There is no difference in the mean scores from two
samples.
t-test (paired-
samples)
There is no difference in the means of two related
measures on a sample or a sample of matched pairs
of subjects.
ANOVA (one-
way)
There is no difference in the means of scores of two
or more samples. Single independent variable.
ANOVA (two-
way)
There is no difference in the means of scores of two
or more samples. Two independent variables are
included and a hypothesis for their interaction
QM STEM Ed 20184
Parametric analysis - assumptions
Measurement of the data on the interval or ratio scale.
Scores are ‘independent’.
Scores are selected from a normally distributed population. (If the sample is large this is not important.)
Homogeneity of variance – if two or more groups are studied, they must come from populations with similar dispersions in their distribution.
QM STEM Ed 20185
Degrees of freedom
The degrees of freedom (df) is defined as the
number of ways in which the data is free to vary.
This is determined by subtracting the number of
restrictions placed on the data from the number of
scores.
For example, a mean is computed by summing n
scores, when n-1 scores have been totaled the nth
score is uniquely determined as it provides the
remainder for the sum. In this case the df is n-1
QM STEM Ed 20186
Another distribution
The central limit theorem states that the sampling distribution of a statistic (e.g. a sample mean) will form a normal distribution, for a sufficiently large sample size.
If we know the standard deviation of the population, we can compute a z-score, and use the normal distribution to evaluate probabilities with the sample mean.
However, for small sample sizes or if we do not know the standard deviation of the population, the distribution of the t statistic (also known as the t-score) is used:
The distribution of the t statistic is called the t distribution or the Student t distribution.
xt
s n
where x bar is the sample mean, μ is the population mean, s is
the standard deviation of the sample, and n is the sample size
(this is for a one sample t-test)
QM STEM Ed 20187
The t distribution
The t-distribution is actually a family of distributions, with a different t distribution for each degree of freedom.
The smaller the sample, and the fewer the degrees of freedom, the flatter (more spread) the t-distribution.
For large sample sizes (around 100), the normal and t-distributions are almost identical.
The t-distribution rather than the normal distribution is more appropriate to use when sample sizes are small and the sample standard deviation is estimated from the population standard deviation.
QM STEM Ed 20188
William Sealy Gosset, who
developed the "t-statistic" and
published it in 1908 under the
pseudonym of "Student“
(https://en.wikipedia.org/)
QM STEM Ed 20189
The t and normal distributions
Comparison of the
t-distribution with 4
degrees of freedom to
the normal distribution.
QM STEM Ed 201810
The t and normal distributions
4 2 2 4
0.1
0.2
0.3
0.4
t distribution with 1 degrees of freedom .
4 2 2 4
0.1
0.2
0.3
0.4
t distribution with 5 degrees of freedom .
4 2 2 4
0.1
0.2
0.3
0.4
t distribution with 16 degrees of freedom .
4 2 2 4
0.1
0.2
0.3
0.4
t distribution with 30 degrees of freedom .
The red curve is the standard normal curve
QM STEM Ed 201811
t-test
One of the most commonly used statistical
tests.
Used to determine whether the means of two
groups differ to a statistically significant
degree.
There are different versions of the t-test.
QM STEM Ed 201812
Which t-test to use?
Independent samples t-test – the mean score from a group is tested against the mean score from another group. The hypothesis tested is:
H0 : μ1 = μ2
HA : μ1 ≠ μ2
Paired or dependent samples t-test – the mean score of a group on one measure is tested against the mean score on another measure.
H0 : μd = 0
HA : μd ≠ 0
One sample t-test – a mean score is tested against a particular value.H0 : μ1 = value
HA : μ1 ≠ value
QM STEM Ed 201813
Assumptions: interval scale
Dependent variables should be on interval or
ratio scale.
There is some debate as to whether ordinal
scale variables should be used for parametric
tests.
Often in educational research (and other
research) parametric analysis is used on
Likert scale variables.
QM STEM Ed 201814
Assumptions: independence
The value of one observation should not
influence the value of another observation –
except in paired samples tests.
For tests between groups – subjects should
appear in only one group.
Examples of violations:
Subject is unwittingly tested twice
Subjects are not randomly assigned to groups.
QM STEM Ed 201815
Assumptions: normality
t-tests assume that the dependent variable is
normally distributed in each population. In practice
we test the distribution of scores of the sample.
Violations do not greatly influence the test,
particularly when:
The sample size is large
The test is two-tailed
The distribution is not especially skewed
Normality of distributions can be tested in SPSS.
QM STEM Ed 201816
Assumptions: homogeneity of
variance
t-tests assume equal variances.
An adjustment can be made to the t-value
when variances are not homogenous.
Levene’s test is used to test for homogeneity
of variances.
SPSS provides a test for homogeneity of
variance and a adjusted t-value.
QM STEM Ed 201817
Anxiety and Exams study
QM STEM Ed 201818
Independent-samples t-test example
Anxiety and Exams study
We would like to see if there are any differences between the male and female students for: assignment marks
exam marks
number of hours spent on exam preparation.
To do this we will compare the mean values of these variables for the males and females using independent-samples t-tests. In each case we are testing the null hypothesis of no difference between the groups.
H0 : μ1 = μ2
HA : μ1 ≠ μ2
QM STEM Ed 201819
Independent samples t-test
We will go through these steps:
Check assumptions
Perform tests
Examine output
Check more assumptions
Report our findings
QM STEM Ed 201820
Assumptions
Assumptions for an independent samples t-test:
Interval scale. √
Independence of observations √
and groups. √
Normality – we will test this in SPSS.
Homogeneity of variance – we will test this in
SPSS.
QM STEM Ed 201821
Investigate descriptives
1. Select the Analyze menu.
2. Click on Descriptive Statistics.
3. Select the variables assignment, exam and
hours and move into the Variable(s) box.
4. Click on Options… to open the Options dialogue box.
5. Click on the Kurtosis and Skewness check boxes.
6. Click on Continue, then OK.
QM STEM Ed 201822
Examine descriptives
Descriptive Statistics
25 1 10 6.32 2.688 -.301 .464 -.801 .902
25 1 9 5.48 2.330 -.574 .464 -.517 .902
25 2 19 9.68 4.404 -.025 .464 -.342 .902
25
Assignment Mark
Exam Mark
Hours of Exam
Preparat ion
Valid N (listwise)
Stat ist ic Stat ist ic Stat ist ic Stat ist ic Stat ist ic Stat ist ic Std. Error Stat ist ic Std. Error
N Minimum Maximum Mean Std.
DeviationSkewness Kurtosis
QM STEM Ed 201823
Examine descriptives
Descriptive Statistics
25 1 10 6.32 2.688 -.301 .464 -.801 .902
25 1 9 5.48 2.330 -.574 .464 -.517 .902
25 2 19 9.68 4.404 -.025 .464 -.342 .902
25
Assignment Mark
Exam Mark
Hours of Exam
Preparat ion
Valid N (listwise)
Stat ist ic Stat ist ic Stat ist ic Stat ist ic Stat ist ic Stat ist ic Std. Error Stat ist ic Std. Error
N Minimum Maximum Mean Std.
DeviationSkewness Kurtosis
The skewness is
low for all three
distributions
The kurtosis is
low for all three
distributions
QM STEM Ed 201824
Kolmogorov-Smirnov test
The Kolmogorov-
Smirnov test is used to
test that the data comes
from a particular
distribution – in this
case we will test for a
normal distribution.
1. Select Analyze Nonparametric Tests 1-Sample K-S…
2. Select the variables assignment, exam and hours and move into the
Test Variable List box.
3. Check that Normal is selected. This is usually the default.
QM STEM Ed 201825
Kolmogorov-Smirnov test
One-Sample Kolmogorov-Smirnov Test
25 25 25
6.32 5.48 9.68
2.688 2.330 4.404
.120 .228 .169
.088 .097 .136
-.120 -.228 -.169
.599 1.142 .845
.865 .148 .473
N
Mean
Std. Dev iat ion
Normal Parametersa,b
Absolute
Positive
Negativ e
Most Extreme
Dif f erences
Kolmogorov-Smirnov Z
Asy mp. Sig. (2-tailed)
Assignment
Mark Exam Mark
Hours of
Exam
Preparat ion
Test distribution is Normal.a.
Calculated f rom data.b. Since these p-values are all greater than 0.05 we can
conclude that the data has a normal distribution
QM STEM Ed 201826
Independent-samples t-test
1. Select the Analyze Compare Means Independent-Samples T-Test…
2. Select the variables assignment, exam and hours and move into the Test
Variable(s) list box.
3. Select the variable success and move into the Grouping Variable box.
4. Click on Define Groups…
QM STEM Ed 201827
Independent-samples t-test
1. Enter the values for the grouping variable. In this case it will be 1
for male and 2 for female
2. Click on Continue, then OK.
3. Another option in the Independent-samples T-Test dialogue
box is to click on Paste and save the commands you have
specified to a script file.
QM STEM Ed 201828
Script file in SPSS
You may work directly from the SPSS Syntax Editor rather than
the GUI interface. You can specify and run all tests from here.
QM STEM Ed 201829
Independent-samples t-test
– descriptives
Group Statistics
16 6.13 2.964 .741
9 6.67 2.236 .745
16 4.63 2.446 .612
9 7.00 1.000 .333
16 8.50 4.662 1.165
9 11.78 3.114 1.038
Gendermale
f emale
male
f emale
male
f emale
Assignment Mark
Exam Mark
Hours of Exam
Preparat ion
N Mean Std. Dev iat ion
Std. Error
Mean
QM STEM Ed 201830
Independent-samples t-test
– test output
Independent Samples Test
1.258 .274 -.476 23 .639 -.542 1.139 -2.897 1.814
-.515 20.793 .612 -.542 1.051 -2.729 1.645
9.447 .005 -2.765 23 .011 -2.375 .859 -4.152 -.598
-3.410 21.654 .003 -2.375 .696 -3.821 -.929
4.798 .039 -1.878 23 .073 -3.278 1.745 -6.888 .333
-2.100 22.130 .047 -3.278 1.561 -6.513 -.042
Equal variances
assumed
Equal variances
not assumed
Equal variances
assumed
Equal variances
not assumed
Equal variances
assumed
Equal variances
not assumed
Assignment Mark
Exam Mark
Hours of Exam
Preparat ion
F Sig.
Levene's Test f or
Equality of Variances
t df Sig. (2-tailed)
Mean
Dif f erence
Std. Error
Dif f erence Lower Upper
95% Conf idence
Interv al of the
Dif f erence
t-test for Equality of Means
QM STEM Ed 201831
One-tailed or two-tailed?
Hypotheses may be stated in directional or non-
directional form.
Directional means that you predict the direction of
difference before analysis. E.g.
H0 : μ1 > μ2 or HA : μ1 < μ2
It is easier to reject the null hypothesis in the chosen
direction (larger p-value) – but you then must ignore
differences in the other direction.
When in doubt about the direction of change, use a
two-tailed test.
QM STEM Ed 201832
Independent-samples t-test
– test output
Independent Samples Test
1.258 .274 -.476 23 .639 -.542 1.139 -2.897 1.814
-.515 20.793 .612 -.542 1.051 -2.729 1.645
9.447 .005 -2.765 23 .011 -2.375 .859 -4.152 -.598
-3.410 21.654 .003 -2.375 .696 -3.821 -.929
4.798 .039 -1.878 23 .073 -3.278 1.745 -6.888 .333
-2.100 22.130 .047 -3.278 1.561 -6.513 -.042
Equal variances
assumed
Equal variances
not assumed
Equal variances
assumed
Equal variances
not assumed
Equal variances
assumed
Equal variances
not assumed
Assignment Mark
Exam Mark
Hours of Exam
Preparat ion
F Sig.
Levene's Test f or
Equality of Variances
t df Sig. (2-tailed)
Mean
Dif f erence
Std. Error
Dif f erence Lower Upper
95% Conf idence
Interv al of the
Dif f erence
t-test for Equality of Means
We are 95% confident that the mean
difference for each population falls between
the upper and lower confidence limits
QM STEM Ed 201833
Homogeneity of variance –
Levene’s test
Levene’s test is used to determine whether the variances are similar (nearly equal). We do this by looking at the p-values.
In our tests the p-value is > 0.05 for the first test and therefore we can conclude that there is no difference in variances between the assignment marks for the male and female groups.
However, for the exam marks and the number of hours in exam preparation, the p-values are far less than 0.05 indicating differences in variances.
This affects the selection of the t-value, df and significance level when reporting results.
QM STEM Ed 201834
Independent-samples t-test results
We now report the results of our tests:
Independent-samples t-tests (n=25) were conducted to evaluate the hypotheses that males and females differ in their marks for assignments and exams and in their preparation time for exams.
No difference was found in the assignment marks between the male and female students. However, the female students scored higher marks (M=7.0, SD=1.0) than males (M=4.6,SD=2.5) in the exam (t(21.7)= -3.41, p<0.05) and the female students spent longer hours (M=11.8,SD=3.1) than the males (M=8.5,SD=4.7) on their work (t(22.1) = -2.10, p<0.05). These differences were significant.
QM STEM Ed 201835
Paired samples t-test example
We would like to see if there is any difference between the assignment and exam marks for the students.
As these are related measures, we will compare the mean values for the assignment and exam marks using paired samples t-tests. In this case we are testing the null hypothesis of no difference between the measures.
H0 : μd = 0
HA : μd ≠ 0
QM STEM Ed 201836
Assumptions
Assumptions for an independent samples t-test:
Interval scale √
Independence of observations √
Normality of population difference scores –
the difference between the scores for each
subject should be normally distributed. Not
really important for samples greater than 30,
but as our sample is 25 we will check this.
QM STEM Ed 201837
Checking normality of differences
One-Sample Kolmogorov-Smirnov Test
25
-.8400
2.86764
.198
.161
-.198
.989
.282
N
Mean
Std. Dev iation
Normal Parameters a,b
Absolute
Positive
Negative
Most Extreme
Dif f erences
Kolmogorov-Smirnov Z
Asy mp. Sig. (2-tailed)
Dif f between
exam and
assignment
Test distribution is Normal.a.
Calculated f rom data.b. Since this p-values is greater than 0.05 we can
conclude that the data has a normal distribution
QM STEM Ed 201838
Paired samples t-test
1. Select Analyze Compare Means Paired-Samples T-Test…
2. Select assignment and exam and move into the Paired Variables list box.
3. Select OK.
QM STEM Ed 201839
Paired-samples t-test
– descriptives
Paired Samples Statistics
6.32 25 2.688 .538
5.48 25 2.330 .466
Assignment Mark
Exam Mark
Pair
1
Mean N Std. Dev iation
Std. Error
Mean
Paired Samples Correlations
25 .354 .083Assignment Mark
& Exam Mark
Pair
1
N Correlation Sig.
SPSS also does a correlation
with a Paired samples t-test
QM STEM Ed 201840
Paired-samples t-test
– test output
Paired Samples Test
.840 2.868 .574 -.344 2.024 1.465 24 .156Assignment Mark
- Exam Mark
Pair
1
Mean Std. Dev iat ion
Std. Error
Mean Lower Upper
95% Conf idence
Interv al of the
Dif f erence
Paired Dif f erences
t df Sig. (2-tailed)
QM STEM Ed 201841
Paired samples t-test results
We now report the results of our test.
A paired-samples t-test (n=25) was conducted to
evaluate the hypotheses that the students’
marks for assignments and exams were
different.
No difference was found between the
assignment marks and exam marks for the
students in this study.
QM STEM Ed 201842
One-sample t-test
We would like to see if there is any difference between the final results achieved by this group of students and the results from other years. The mean result was 48 for the previous five years.
In this case, we have one set of data that we wish to test against a ‘population’ mean, so a one-sample t-test is appropriate to use. We are testing the null hypothesis of no difference between the groups and a particular value.
H0 : μ1 = 48
HA : μ1 ≠ 48
QM STEM Ed 201843
One-sample t-test
1. Select Analyze Compare Means One-Sample T-Test…
2. Select final and move into the Test Variable(s) list box.
3. Enter 48 into the Test Value box.
4. Select OK.
QM STEM Ed 201844
One-sample t-test – output
One-Sample Statistics
25 57.32 20.591 4.118Final Mark
N Mean Std. Dev iat ion
Std. Error
Mean
One-Sample Test
2.263 24 .033 9.320 .82 17.82Final Mark
t df Sig. (2-tailed)
Mean
Dif f erence Lower Upper
95% Conf idence
Interv al of the
Dif f erence
Test Value = 48
QM STEM Ed 201845
One-sample t-test results
We now report the results of our test.
A one-sample t-test (n=25) was conducted to evaluate the hypotheses that the final marks of this group of students is different to the mean mark obtained over the previous five years.
The final result of the students (M=57.3, SD=20.6) was higher than the mean of 48 recorded for the previous five years and this difference was significant.
QM STEM Ed 201846
T-106.6500 Quantitative Methods in
Computing/Engineering Education
Research
Topic 5 (part 2): Parametric analyses
Judy Sheard
Faculty of Information Technology
Monash University, Australia
QM STEM Ed 201847
ANOVA
If we wish to make comparisons between more than two
groups, then the appropriate test to use is an analysis
of variance (ANOVA).
There are a number of different versions of ANOVA
depending on:
the number of factors (independent variables);
the number of levels of each independent variable; and
whether the independent variables are unrelated (each group
comprises different subjects) or related (same subjects are
used in each group).
QM STEM Ed 201848
ANOVA varieties
The common term for an ANOVA with 1 factor is one-way ANOVA, with 2 factors we use two-way ANOVA, 3 factors we use three-way ANOVA, and so on...
If we want to also specify the levels within each factor, then we use the following:
An ANOVA with one factor with 3 levels and another factor with 5 levels is called a 3 x 5 ANOVA.
An ANOVA with three factors each with 4 levels is called a 4 x 4 x 4 ANOVA.
QM STEM Ed 201849
ANOVA
One-way ANOVA is used to test that the means scores of two or
more groups are not significantly different.
H0 : μ1 = μ2 = μ3 = … = μk
HA : μ1 ≠ μ2 for at least one pair.
The null hypothesis in ANOVA is tested by comparing two estimates
of variance called mean squares. It compares the variance
between groups to the variance within groups.
The ratio of these variances has as its sampling distribution the F-
distribution, determined by two degrees of freedom values. These
are a between groups df and a within groups df.
F-distributions for different degrees of
freedom
QM STEM Ed 201850
http://www.vosesoftware.com/ModelRiskHe
lp/index.htm
F-distribution
QM STEM Ed 201851
QM STEM Ed 201852
One-way ANOVAs
One-way ANOVAs have one factor (independent variable). There are two main types:
One-way between groups – each group comprises different subjects.
One-way repeated measures – the same subjects are used in each group.
For example: You plan a study of the effect of tutorial tasks on final grades in an introductory programming course.
A study that compared performance of 4 tutorial groups in the final exam would be one-way between groups ANOVA.
A study that compared tutorial performance on 3 successive tests would be a one-way repeated measures ANOVA.
QM STEM Ed 201853
One-way ANOVA example
Performance studyAs part of a study of the first year of a new multi-campus IT degree,
data of student performance was collected. We wish to investigate
whether there is any difference in the performance of students in the
computer systems and programming courses across the five
campuses of the university.
In each case we wish to compare the mean results of five campuses. Since have one factor with five levels, will use a one-way between groups AVOVA.
H0 : μ1 = μ2 = μ3 = … = μk
HA : μ1 ≠ μ2 for at least one pair.
QM STEM Ed 201854
Assumptions
The assumptions for an ANOVA are similar to
those for t-tests.
Interval scale √
Independence of observations √
Normality of populations – we will test this in
SPSS.
Homogeneity of variance – we will test this in
SPSS.
QM STEM Ed 201855
Descriptives
Descriptive Statistics
119 7.00 97.00 63.1681 18.74283 -1.240 .222 1.569 .440
119 16.00 100.00 69.4118 21.30585 -.589 .222 -.347 .440
119
Computer Sy stems
Programming
Valid N (listwise)
Stat istic Stat istic Stat istic Stat istic Stat istic Stat istic Std. Error Stat istic Std. Error
N Minimum Maximum Mean Std.
DeviationSkewness Kurtosis
Note the large skewness and
kurtosis value for the Computer
Systems variable.
QM STEM Ed 201856
Normality check
One-Sample Kolmogorov-Smirnov Test
119 119
63.1681 69.4118
18.74283 21.30585
.182 .086
.072 .078
-.182 -.086
1.981 .938
.001 .342
N
Mean
Std. Dev iat ion
Normal Parametersa,b
Absolute
Positive
Negative
Most Extreme
Dif f erences
Kolmogorov-Smirnov Z
Asy mp. Sig. (2-tailed)
Computer
Sy stems Programming
Test distribution is Normal.a.
Calculated f rom data.b.
The low p-value for computer systems
(less than 0.05) indicates that the data
does not have a normal distribution.
QM STEM Ed 201857
Distribution of results for
Computer Systems
QM STEM Ed 201858
Distribution of results for
Programming
QM STEM Ed 201859
ANOVA
1. Select Analyze Compare Means One-Way ANOVA…
2. Select the variable programming and move into the Dependent List box.
3. Select campus and enter into the Factor box.
4. Click on Options… to open the Options box.
5. Click on Descriptive and Homogeneity of variance test
6. Select Continue and then OK or Paste.
QM STEM Ed 201860
ANOVA – descriptives
Descriptives
Programming
39 71.0256 17.99048 2.88078 65.1938 76.8575 30.00 98.00
13 62.8462 19.35565 5.36829 51.1497 74.5427 23.00 94.00
18 60.6667 11.84706 2.79238 54.7753 66.5581 33.00 75.00
8 46.0000 22.18751 7.84447 27.4508 64.5492 25.00 87.00
41 78.3659 23.03666 3.59772 71.0946 85.6371 16.00 100.00
119 69.4118 21.30585 1.95310 65.5441 73.2794 16.00 100.00
Inner city
Southern metropolitan
Eastern metropolitan
Country
Outer city
Total
N Mean Std. Dev iation Std. Error Lower Bound Upper Bound
95% Conf idence Interv al for
Mean
Minimum Maximum
Note that this table will only
appear if you request
Descriptive statistics in the
Options box.
QM STEM Ed 201861
ANOVA – checking homogeneity
Test of Homogeneity of Variances
Programming
1.995 4 114 .100
Levene
Stat istic df1 df2 Sig.
The p-value greater than 0.05
indicates no difference in the
population variances for each group.
QM STEM Ed 201862
ANOVA – test output
ANOVA
Programming
9710.645 4 2427.661 6.311 .000
43854.179 114 384.686
53564.824 118
Between Groups
Within Groups
Total
Sum of
Squares df Mean Square F Sig.
The p-values of the F test is less than O.05. We
therefore reject the null hypothesis that there is
no difference in the results for the programming
course across the five campuses.
The degrees of freedom of values
for each source of variance.
QM STEM Ed 201863
Means plot
The means plot helps
show the structure of
the data and where the
difference may occur.
QM STEM Ed 201864
ANOVA results
We now report the results of our test.
To investigate whether there was any difference in the
performance of students in the programming unit
across the five campuses, a one-way ANOVA was
performed.
This showed that the programming results differed significantly across the five campuses (F (4,114) = 6.31, p < 0.05).
However, where did these differences occur?
QM STEM Ed 201865
Post Hoc analysis
The ANOVA indicates that there was a significant difference in results across the campus – but it does not show which campuses were different – many comparisons were made.
Two approaches may be used:
Planned comparison where each comparison is investigated to assess a particular hypothesis.
Post hoc analysis involves finding where the differences occurred – all differences are investigated.
Post hoc tests help guard against Type 1 error.
QM STEM Ed 201866
Post Hoc analysis
Used for ANOVAs where the factors have
more than 2 levels – otherwise main effect is
sufficient.
Used for between groups comparisons.
There are difference tests available. We will
use Tukey HSD and Scheffe.
QM STEM Ed 201867
Post Hoc analysis
Select Analyze Compare Means One-Way ANOVA…
Click on Post Hoc… to open the Post Hoc box.
Click on Scheffe and Tukey check boxes
QM STEM Ed 201868
Post Hoc analysis – Tukey HSD
Multiple Comparisons
Dependent Variable: Programming
Tukey HSD
8.17949 6.28132 .690 -9.2318 25.5908
10.35897 5.58884 .348 -5.1329 25.8508
25.02564* 7.61245 .012 3.9245 46.1268
-7.34021 4.38706 .455 -19.5008 4.8204
-8.17949 6.28132 .690 -25.5908 9.2318
2.17949 7.13881 .998 -17.6088 21.9677
16.84615 8.81345 .317 -7.5841 41.2764
-15.51970 6.24290 .101 -32.8245 1.7851
-10.35897 5.58884 .348 -25.8508 5.1329
-2.17949 7.13881 .998 -21.9677 17.6088
14.66667 8.33410 .402 -8.4348 37.7681
-17.69919* 5.54563 .015 -33.0712 -2.3271
-25.02564* 7.61245 .012 -46.1268 -3.9245
-16.84615 8.81345 .317 -41.2764 7.5841
-14.66667 8.33410 .402 -37.7681 8.4348
-32.36585* 7.58079 .000 -53.3792 -11.3525
7.34021 4.38706 .455 -4.8204 19.5008
15.51970 6.24290 .101 -1.7851 32.8245
17.69919* 5.54563 .015 2.3271 33.0712
32.36585* 7.58079 .000 11.3525 53.3792
(J) Campus
Southern metropolitan
Eastern metropolitan
Country
Outer city
Inner city
Eastern metropolitan
Country
Outer city
Inner city
Southern metropolitan
Country
Outer city
Inner city
Southern metropolitan
Eastern metropolitan
Outer city
Inner city
Southern metropolitan
Eastern metropolitan
Country
(I) Campus
Inner city
Southern metropolitan
Eastern metropolitan
Country
Outer city
Mean
Dif f erence
(I-J) Std. Error Sig. Lower Bound Upper Bound
95% Conf idence Interv al
The mean dif f erence is signif icant at the .05 level.*.
SPSS prints out a
complete matrix –
you need to ignore
repetitions.
As our factor had
5 levels there are
10 possible
comparisons.
QM STEM Ed 201869
Post Hoc analysis
Programming
8 46.0000
18 60.6667 60.6667
13 62.8462 62.8462
39 71.0256
41 78.3659
.110 .082
8 46.0000
18 60.6667 60.6667
13 62.8462 62.8462
39 71.0256
41 78.3659
.207 .165
Campus
Country
Eastern metropolitan
Southern metropolitan
Inner city
Outer city
Sig.
Country
Eastern metropolitan
Southern metropolitan
Inner city
Outer city
Sig.
Tukey HSDa,b
Schef fea,b
N 1 2
Subset f or alpha = .05
Means for groups in homogeneous subsets are display ed.
Uses Harmonic Mean Sample Size = 16.260.a.
The group sizes are unequal. The harmonic mean of the group sizes
is used. Type I error levels are not guaranteed.
b.
These show the
homogenous subsets.
Note that Scheffe and
Tukey HSD tests
produce the same
subsets but the p-
values are different.
QM STEM Ed 201870
One-way ANOVA with post hoc
analysis results
We now report the results of our test.
To investigate whether there was any difference in the
performance of students in the programming unit
across the five campuses, a one-way ANOVA was
performed.
This showed that the programming results differed significantly across the five campuses (F (4,114) = 6.31, p < 0.05).
Post Hoc analysis using Tukey’s HSD showed that there were two subsets of campuses whose means were not significantly different.
QM STEM Ed 201871
Two-way ANOVAs
Two-way ANOVAs have two factors (independent
variables). This requires:
Null hypothesis for each factor – main effects.
Also a possibility of interaction between the two
factors – interaction effect.
A significant interaction effect means that the
response at a particular level of one factor depends on
the level of the other factor.
In this situation, the main effects are ambiguous.
QM STEM Ed 201872
Two-way ANOVAs
For example, in a study of student exam performance in an introductory programming course you may want to investigate the effect of teaching approach (objects-first, objects-later) and learning approaches (deep, surface , achieving) on final grades.
To investigate this, a two-way between groups ANOVA may be used to compare:
exam performance between objects-first and objects-later students.
exam performance between deep, surface , achieving students.
the interaction between teaching approach and learning approach.
This is called a 2 x 3 ANOVA, That is, two independent variables – one with 2 levels and the other with 3 levels.
(In two-way ANOVAs one of the groups can be a repeated measure.)
QM STEM Ed 201873
Two-way ANOVAs
Consider the following result of the ANOVA:
exam performance is significantly influenced by teaching approach – main effect.
exam performance is significantly influenced by learning approach – main effect.
the interaction between teaching approach and learning approach is significant – interaction effect.
Therefore some combinations of teaching approaches and learning approaches may be more effective than others. In fact, teaching approach may not influence performance for all learning approaches – or vice versa.
n-way ANOVA
In an n-way ANOVA there are 2 or more factors each
containing two or more levels. An n-way ANOVA allows
us to analyse the main effects of all factors
simultaneously as well as any interactions between
them.
If two factors interact then the response at a particular level
of one factor depends on the level of one other factor.
If three factors interact then the response at a particular
level of one factor depends on the level of two other
factors. … and so on.
It is possible to have n-way ANOVAs; however, more than
3 or 4 is not common in education research.QM STEM Ed 2018
74
QM STEM Ed 201875