factorial designs & managing violated statistical assumptions dr james betts developing study...

Factorial Designs & Managing Violated Statistical Assumptions

Dr James Betts

Developing Study Skills and Research Methods (HL20107)

Lecture Outline:

•Factorial Research Designs Revisited

•Mixed Model 2-way ANOVA

•Fully independent/repeated measures 2-way ANOVA

•Statistical Assumptions of ANOVA.

Last Week Recap• In last week’s lecture we saw two worked

examples of 1-way analyses of variance

• However, many experimental designs have more than one independent variable (i.e. factorial design)

Factorial Designs: Technical Terms• Factor

• Levels

• Main Effect

• Interaction Effect

Factorial Designs: Multiple IV’s• Hypothesis:

– The HR response to exercise is mediated by gender

• We now have three questions to answer:

Post Run 1 Pre Run 2 Post Run 2

CHO CHO-PRO

Factorial Designs: Interpretation210

Resting Exercise

Main Effect of Exercise

Not significant

Main Effect of Gender

Not significant

Exercise*Gender Interaction

Not significant

CHO CHO-PRO

Resting Exercise

Significant

Not significant

CHO CHO-PRO

Resting Exercise

Not significant

Significant

Not significant

CHO CHO-PRO

Resting Exercise

Not significant

Not Significant

Significant

CHO CHO-PRO

Resting Exercise

CHO CHO-PRO

Resting Exercise

SystematicVariance

(resting vs exercise)

ErrorVariance

(between subjects)

Systematic Variance

(male vs female)

Systematic Variance

(Interaction)

2-way mixed model ANOVA: Partitioning

= variance between means due to exercise

= variance between means due to gender

= variance between means due IV interaction

= uncontrolled factors and within group differences for males vs females.

ErrorVariance

(within subjects)= uncontrolled factors plus random changes within individuals for rest vs exercise

Procedure for computing 2-way mixed model ANOVA

• Step 1: Complete the table

i.e.-square each raw score

-total the raw scores for each subject (e.g. XT)

-square the total score for each subject (e.g. (XT)2)

-Total both columns for each group

-Total all raw scores and squared scores (e.g. X & X2).

• Step 2: Calculate the Grand Total correction factor

• Step 3: Compute total Sum of Squares

SStotal= X2 - GT

• Step 4: Compute Exercise Effect Sum of Squares

SSex= - GT

= + - GT

(Xex)2

(XRmale+XRfemale)2

nRmale+fem

(XEmale+XEfemale)2

nEmale+fem

• Step 5: Compute Gender Effect Sum of Squares

SSgen= - GT

= + - GT

(Xgen)2

(XRmale+XEmale)2

nmaleR+E

(XRfem+XEfem)2

nfemR+E

• Step 6: Compute Interaction Effect Sum of Squares

SSint= - GT

= + + + - (SSex+SSgen) - GT

(Xex+gen)2

nex+gen

nRmale

nRfem nEmale nEfem

(XRmale)2 (XRfem)2 (XEmale)2 (XEfem)2

• Step 7: Compute between subjects Sum of Squares

SSbet= -SSgen- GT

= -SSgen- GT

(XT)2+(XD)2+(XH)2+(XJ)2+(XK)2+(XA)2+(XS)2+(XL)2

• Step 8: Compute within subjects Sum of Squares

SSwit= SStotal - (SSex+SSgen+SSint+SSbet)

• Step 9: Determine the d.f. for each sum of squares

dftotal = (N - 1)

dfex = (k - 1)

dfgen = (r - 1)

dfint = (k - 1)(r - 1)

dfbet = r(n - 1)

dfwit = r(n - 1)(k - 1)

• Step 10: Estimate the Variances Procedure for computing 2-way mixed model ANOVA

SystematicVariance

ErrorVariance

(between subjects)

Systematic Variance

(male vs female)Systematic

Variance(Interaction)

ErrorVariance

(within subjects)

• Step 11: Compute F values Procedure for computing 2-way mixed model ANOVA

SystematicVariance

ErrorVariance

(between subjects)

Systematic Variance

ErrorVariance

(within subjects)

• Step 12: Consult F distribution table as before Exercise

Gender

Interaction

Tests of Within-Subjects Effects

Measure: MEASURE_1

61504.000 1 61504.000 2396.260 .000

61504.000 1.000 61504.000 2396.260 .000

121.000 1 121.000 4.714 .073

121.000 1.000 121.000 4.714 .073

154.000 6 25.667

154.000 6.000 25.667

Sphericity Assumed

Greenhouse-Geisser

Huynh-Feldt

Lower-bound

Sphericity Assumed

Greenhouse-Geisser

Huynh-Feldt

Lower-bound

Sphericity Assumed

Greenhouse-Geisser

Huynh-Feldt

Lower-bound

Sourceexercise

exercise * Gender

Error(exercise)

Type III Sumof Squares df Mean Square F Sig.

2-way mixed model ANOVA: SPSS OutputCalculated FexSSex dfex

Systematic Varianceex

SSwit dfwitError

Variancewit

Tests of Between-Subjects Effects

Measure: MEASURE_1

Transformed Variable: Average

227529.000 1 227529.000 17064.675 .000

100.000 1 100.000 7.500 .034

80.000 6 13.333

SourceIntercept

Gender

2-way mixed model ANOVA: SPSS Output

Calculated FgenSSgen

Error Variancebet

Systematic Variancegen

SystematicVariance

ErrorVariance

(between subjects)

Systematic Variance

ErrorVariance

(within subjects)

2-way mixed model ANOVA

The previous calculation and associated partitioning is an example of a 2-way mixed model ANOVA– i.e. exercise = repeated measures

gender = independent measures

SystematicVariance

Systematic Variance

ErrorVariance

(within subjects)

2-way Independent Measures ANOVA

So for a fully unpaired design

– e.g. males vs females

rest group vs exercise group

ErrorVariance

(between subjects)

Tests of Between-Subjects Effects

Dependent Variable: HeartRate

58386.000a 3 19462.000 2994.154 .000

234256.000 1 234256.000 36039.385 .000

16.000 1 16.000 2.462 .143

58081.000 1 58081.000 8935.538 .000

289.000 1 289.000 44.462 .000

78.000 12 6.500

292720.000 16

58464.000 15

SourceCorrected Model

Intercept

Gender

CoachCyclist

Gender * CoachCyclist

Corrected Total

R Squared = .999 (Adjusted R Squared = .998)a.

SystematicVariance

Systematic Variance

ErrorVariance

(within subjects)

2-way Independent Measures ANOVA

ErrorVariance

(between subjects)

SystematicVariance

Systematic Variance

(am vs pm)Systematic

Error Variance

(within subjectsexercise)

2-way Repeated Measures ANOVA

…but for a fully paired design

– e.g. morning vs evening

rest vs exercise

Error Variance

(within subjectstime)Error Variance

(within subjectsinteract)

Measure: MEASURE_1

.000 1 .000 .000 1.000

.000 1.000 .000 .000 1.000

1.000 3 .333

1.000 3.000 .333

67081.000 1 67081.000 10062.150 .000

67081.000 1.000 67081.000 10062.150 .000

20.000 3 6.667

20.000 3.000 6.667

.000 1 .000 .000 1.000

.000 1.000 .000 .000 1.000

3.000 3 1.000

3.000 3.000 1.000

Sphericity Assumed

Greenhouse-Geisser

Huynh-Feldt

Lower-bound

Sphericity Assumed

Greenhouse-Geisser

Huynh-Feldt

Lower-bound

Sphericity Assumed

Greenhouse-Geisser

Huynh-Feldt

Lower-bound

Sphericity Assumed

Greenhouse-Geisser

Huynh-Feldt

Lower-bound

Sphericity Assumed

Greenhouse-Geisser

Huynh-Feldt

Lower-bound

Sphericity Assumed

Greenhouse-Geisser

Huynh-Feldt

Lower-bound

SourceTimeofDay

Error(TimeofDay)

Exercise

Error(Exercise)

TimeofDay * Exercise

Error(TimeofDay*Exercise)

SystematicVariance

Systematic Variance

(am vs pm)Systematic

Error Variance

(within subjectsexercise)

2-way Repeated Measures ANOVA

Error Variance

(within subjectstime)Error Variance

(within subjectsinteract)

Summary: 2-way ANOVA• 2-way (factorial) ANOVA may be appropriate

whenever there are multiple IV’s to compare

• We have worked through a mixed model but you should familiarise yourself with paired/unpaired procedures

• You should also ensure you are aware what these effects actually look like graphically.

Statistical Assumptions• As with other parametric tests, ANOVA is

associated with a number of statistical assumptions

• When these assumptions are violated we often find that an inferential test performs poorly

• We therefore need to determine not only whether an assumption has been violated but also whether that violation is sufficient to produce statistical errors.

Energy Intake (calories per day)

1500 2500 3500 4500 5500

16 17 18 19 20

Sustained Isometric Torque (seconds)

e.g. ND assumption from last year

Independent Samples Test

7.842 .012 -2.333 18 .031 -1.69600 .72710 -3.22358 -.16842

-2.333 15.447 .034 -1.69600 .72710 -3.24188 -.15012

Equal variancesassumed

Equal variancesnot assumed

SwimTime50mF Sig.

Levene's Test forEquality of Variances

t df Sig. (2-tailed)Mean

DifferenceStd. ErrorDifference Lower Upper

95% ConfidenceInterval of the

Difference

t-test for Equality of Means

Tests of Normality

.161 10 .200* .968 10 .872

.350 10 .001 .747 10 .003

GroupControl

Visualisation

SwimTime50mStatistic df Sig. Statistic df Sig.

Kolmogorov-Smirnova

Shapiro-Wilk

This is a lower bound of the true significance.*.

Lilliefors Significance Correctiona.

e.g. ND assumption from last year

Assumptions of ANOVA• N acquired through random sampling • Data must be of at least the interval LOM (continuous)

• Independence of observations

• Homogeneity of variance

• All data is normally distributed

“ANOVA is generally robust to violations of the normality assumption, in that even when the data

are non-normal, the actual Type I error rate is usually close to the nominal (i.e., desired) value.”

Maxwell & Delaney (1990) Designing Experiments & Analyzing Data: A Model Comparison Perspective, p. 109

“If the data analysis produces a statistically significant finding when no test of sphericity is

conducted…you should disregard the inferential claims made by the researcher.”

Huck & Cormier (1996) Reading Statistics & Research, p. 432

Group BGroup C

Group ASupplement 1Supplement 2

Placebo

Plac. Supp. 1 Supp. 2 Plac.-Supp. 1 Supp. 1-Supp. 2 Plac.-Supp. 2

Tom 2.4 3.0 3.3 -0.6 -0.3 -0.9

Dick 2.2 2.5 2.4 -0.3 0.1 -0.2

Harry 1.8 1.9 2.2 -0.1 -0.3 -0.4

James 1.6 1.1 1.2 0.5 -0.1 0.4

Mean 2.0 2.1 2.3 -0.1 -0.2 -0.3

SD2 0.1 0.7 0.7 0.2 0.04 0.3

Mauchly's Test of Sphericityb

Measure: MEASURE_1

.310 2.343 2 .310 .592 .752 .500Within Subjects EffectTrial

Mauchly's WApprox.

Chi-Square df Sig.Greenhouse-Geisser Huynh-Feldt Lower-bound

Epsilona

Tests the null hypothesis that the error covariance matrix of the orthonormalized transformed dependent variables isproportional to an identity matrix.

May be used to adjust the degrees of freedom for the averaged tests of significance. Corrected tests are displayed inthe Tests of Within-Subjects Effects table.

Design: Intercept Within Subjects Design: Trial

• Many texts recommend ‘Mauchley’s Test of Sphericity’– A χ2 test which, if significant, indicates a violation to sphericity

• However, this is not advisable on four counts:1.)

• How should we analyse aspherical data?

• Option 1

• Option 2

• Option 3

Managing Violations to Sphericity

Paired 1-way MANOVA: SPSS Output

Multivariate Testsb

.455 .834a 2.000 2.000 .545

.545 .834a 2.000 2.000 .545

.834 .834a 2.000 2.000 .545

Pillai's Trace

Wilks' Lambda

Hotelling's Trace

Roy's Largest Root

EffectTrial

Value F Hypothesis df Error df Sig.

Exact statistica.

Mauchly's Test of Sphericityb

Measure: MEASURE_1

.310 2.343 2 .310 .592 .752 .500Within Subjects EffectTrial

Mauchly's WApprox.

Chi-Square df Sig.Greenhouse-Geisser Huynh-Feldt Lower-bound

Epsilona

Tests the null hypothesis that the error covariance matrix of the orthonormalized transformed dependent variables isproportional to an identity matrix.

May be used to adjust the degrees of freedom for the averaged tests of significance. Corrected tests are displayed inthe Tests of Within-Subjects Effects table.

Paired 1-way ANOVA: SPSS Output

Measure: MEASURE_1

.152 2 .076 .840 .477

.152 1.183 .128 .840 .439

.152 1.505 .101 .840 .457

.152 1.000 .152 .840 .427

.542 6 .090

.542 3.550 .153

.542 4.514 .120

.542 3.000 .181

Sphericity Assumed

Greenhouse-Geisser

Huynh-Feldt

Lower-bound

Sphericity Assumed

Greenhouse-Geisser

Huynh-Feldt

Lower-bound

SourceTrial

Error(Trial)

Summary ANOVA and Sphericity• ANOVA is a generally robust inferential test

• Unpaired data are susceptible to heterogeneity of variance only if group sizes are unequal

• Paired data are susceptible to asphericity only if multiple comparisons are made

• Suggested solutions for the latter include either MANOVA or epsilon corrected df depending on sample size relative to number of levels.

factorial designs & managing violated statistical assumptions dr james betts developing study...

ss total ss ex

ss gen gt x ex

way mixed model anova

gt x gen

way mixed model anova

way mixed model anova

way mixed model anova

ss gen gt x s

Documents

catalog thomas betts

thomas & betts linkedin powerpoint

analysis of variance (anova) developing study skills and...

aimee betts biography - ncbpt€¦ · aimee betts aimee...

alsua-betts v ca

statistical power: sample & effect size dr james betts...

dayton parts and betts truck parts - truecommerce...dayton...

exploring relationships: correlations & multiple linear...

commonly-violated special laws2

by betts nets - the finest in fishing tackle27 hi tider by...

remediating violated customers

richard k. betts a review essay - stanford...

betts and coorg ornithology

developing study skills and research methods unit leader: dr...

betts medal

floats - betts tackle

drs betts betts fort worth grocer codrs betts betts...

thomas betts

betts of brandon - product brochure

mud flap hangers - betts