psyc3010 lecture 7 - psy.uq.edu.auuqwloui1/stats/3010 for... · psyc3010 lecture 7 analysis of...

8/04/2011

1

11

psyc3010 lecture 7psyc3010 lecture 7

analysis of covariance (ANCOVA)analysis of covariance (ANCOVA)

last lecture: correlation and regressionnext lecture: standard MR & hierarchical MR

(MR = multiple regression)

22

announcementsannouncementsquiz 2 quiz 2 –– correlation and regressioncorrelation and regression–– to be completed online May 15 and 16to be completed online May 15 and 16–– assesses material taught in Lectures 6, 7, 8, and 9assesses material taught in Lectures 6, 7, 8, and 9–– Lecture 10 will be a review of regression topics Lecture 10 will be a review of regression topics plus plus fun fun

material on mediation and indirect effectsmaterial on mediation and indirect effects–– practice questions and quiz will be posted on practice questions and quiz will be posted on

BlackboardBlackboard

assignment 2 assignment 2 –– Multiple RegressionMultiple Regression–– due on May 23 (Week 12)due on May 23 (Week 12)–– will learn all skills and concepts by Week 9will learn all skills and concepts by Week 9–– all files on Blackboard next week (Week 8)all files on Blackboard next week (Week 8)

8/04/2011

2

FeedbackFeedbackProsPros

Repetition, revision, elaboration Repetition, revision, elaboration helpful ||||| ||||| ||helpful ||||| ||||| ||Clear, Made it easy ||||| ||||Clear, Made it easy ||||| ||||Funny, Fun ||||| ||||Funny, Fun ||||| ||||Interactive, Engaging, ? breaks Interactive, Engaging, ? breaks good ||||| ||good ||||| ||Slides great, prepared ||||| |Slides great, prepared ||||| |Good pace |||| Good pace |||| Thorough, detailed ||||Thorough, detailed ||||Enthusiasm |||Enthusiasm |||Detailed summaries helpful | ; Detailed summaries helpful | ; Flow charts great | ; Great Flow charts great | ; Great examples | ; examples | ; LectopiaLectopia great |great |Lecture slides up early |; Caring | Lecture slides up early |; Caring | Lecture ended early |Lecture ended early |

ImproveImproveGo slower ||||| ||Go slower ||||| ||More review, repetition needed ||||| More review, repetition needed ||||| Make week 1 lecture slides Make week 1 lecture slides available early |||available early |||Trouble printing ||Trouble printing ||Explanations unclear ||Explanations unclear ||More entertainment ||More entertainment ||Jargon frustrating, boring ||Jargon frustrating, boring ||More practice questions ||More practice questions ||Changing terms/notation Changing terms/notation frustrating |frustrating |Assumed knowledge |Assumed knowledge |More real life examples |More real life examples |Key concepts slide at start |Key concepts slide at start |End early |End early |Spread slides out |Spread slides out |Use harder questions |Use harder questions |Simple Simple simplesimple comparisons / comparisons / effects exasperating |effects exasperating |

33

44

last lectures last lectures this lecturethis lecture2 lectures ago:2 lectures ago:–– importance of maximising power in research importance of maximising power in research

(maximising likelihood of correctly detecting effects that (maximising likelihood of correctly detecting effects that exist in the population, and rejecting Hexist in the population, and rejecting H00))

–– how blocking designs can increase powerhow blocking designs can increase power

last lecture:last lecture:–– correlation (association between two variables) and correlation (association between two variables) and

regression (association + prediction)regression (association + prediction)

this lecture:this lecture:–– an additional strategy to maximise power: an additional strategy to maximise power:

Analysis of Covariance (ANCOVA)Analysis of Covariance (ANCOVA)

8/04/2011

3

55

topics covered in this lecture topics covered in this lecture

review of blocking designs and introduction toanalysis of covariance (ANCOVA)

1st use of ANCOVA: reduce error variance

2nd use of ANCOVA: adjust treatment means

structural model & assumptions of ANCOVA

why ANCOVA is controversial

66

review of blocking review of blocking designs and designs and introduction to ANCOVAintroduction to ANCOVA

8/04/2011

4

77

how blocking designs can helphow blocking designs can helpproblemproblem = there is a lot of unexplained variance (“error = there is a lot of unexplained variance (“error variance”) in your 1variance”) in your 1--way experiment, and so the effect of way experiment, and so the effect of your focal IV is not statistically significantyour focal IV is not statistically significantsolutionsolution = add a 2= add a 2ndnd IV that you know will explain additional IV that you know will explain additional variance (based on previous research)variance (based on previous research)adding this adding this controlcontrol or or concomitantconcomitant variable changes the variable changes the design of your study (now a 2design of your study (now a 2--way factorial)way factorial)22ndnd IV explains additional systematic variance, and so now IV explains additional systematic variance, and so now there is less unexplained / residual / error variancethere is less unexplained / residual / error varianceincreased chance that your focal IV has a significant effect: increased chance that your focal IV has a significant effect:

error

treat

MSMSF =

ideally the same as in your 1-way design

ideally smaller than in your 1-way design

88

ancova ancova –– analysis of covarianceanalysis of covariancehas the same goal as blocking but works has the same goal as blocking but works differently:differently:–– blocking works at the blocking works at the level of designlevel of design –– the the

reduction in the size of the error term is a reduction in the size of the error term is a consequence of including a factor that explains a consequence of including a factor that explains a good proportion of variance in the DVgood proportion of variance in the DV

–– With ancova the error term is adjusted With ancova the error term is adjusted statisticallystatistically

8/04/2011

5

99

“remind me what is covariance?”“remind me what is covariance?”

variancevariance is the tendency for scoresis the tendency for scoresto vary around some mean value to vary around some mean value coco--variancevariance is the tendency for two is the tendency for two scores to vary scores to vary togethertogether–– If a participant’s score on one variable deviates from the mean, If a participant’s score on one variable deviates from the mean,

the score on the other (covarying) variable also deviates the score on the other (covarying) variable also deviates –– positive covariance = both deviate in the same direction positive covariance = both deviate in the same direction [ Review [ Review

pp. 236pp. 236--8 (Howell 68 (Howell 6thth ed.)ed.) if nec]if nec]

a a covariatecovariate is like the control variable used for is like the control variable used for blocking, with a couple of differences:blocking, with a couple of differences:–– the covariate is a the covariate is a continuouscontinuous variable and treated as such (i.e., variable and treated as such (i.e.,

participants are not matched at discrete levels)participants are not matched at discrete levels)–– in ancova, the covariate is used to remove error from both the in ancova, the covariate is used to remove error from both the

error term error term andand treatment effecttreatment effect

( ) 1/2

−−∑ XXij

( )( )1−

−−∑ ZZXX

1010

Analysis of CovarianceAnalysis of Covariance——ANCOVAANCOVA

Originally a technique for analysing Originally a technique for analysing experiments and removing nuisance experiments and removing nuisance variationvariationAttempt to Attempt to reduce error termreduce error term by measuring by measuring another variable and estimating its another variable and estimating its parametersparameters–– if the variable affects the DV and it is not part if the variable affects the DV and it is not part

of the statistical model for the ANCOVA, the of the statistical model for the ANCOVA, the variable is in the unmeasured ‘error’variable is in the unmeasured ‘error’

8/04/2011

6

1111

Ho H1

μo μ1 α

power

β

1212

Ho H1

μo μ1 α

power

β

Use ANCOVA to reduce error (just like with

blocking)

8/04/2011

7

1313

ANCOVAANCOVAAll forms of ANOVA can be performed with a All forms of ANOVA can be performed with a covariate (or several)covariate (or several)A covariate is another IV/predictor in the modelA covariate is another IV/predictor in the model–– but continuous (ordered scores, not discrete groups)but continuous (ordered scores, not discrete groups)

Can reduce Can reduce errorerror termterm——ifif it is related to the DVit is related to the DVif unrelated you lose DF (lose power) without if unrelated you lose DF (lose power) without compensatory reduction in error (i.e., bad tradecompensatory reduction in error (i.e., bad trade--off) off)

1414

Uses of ANCOVAUses of ANCOVA

1.1. To control unwanted variation that would To control unwanted variation that would otherwise inflate the error with which we otherwise inflate the error with which we test our models (classical usage)test our models (classical usage)

2.2. To control for group differences, esp. in To control for group differences, esp. in the analysis of clinical trials or other the analysis of clinical trials or other pre/post designs (controversial, see pre/post designs (controversial, see Howell 16.5)Howell 16.5)

8/04/2011

8

1515

OnewayOneway ANCOVA structural modelANCOVA structural model

Covariate is just another source of varianceCovariate is just another source of varianceUse the term Use the term βΖβΖijij because of continuous because of continuous nature;nature;Score on DV goes up or down depending on Score on DV goes up or down depending on score on Zscore on ZImplicitly, we have specified Implicitly, we have specified no interactionno interactionbetween covariate and the IVbetween covariate and the IV–– the presence of such an interaction is a violation of the presence of such an interaction is a violation of

ANCOVA assumptionsANCOVA assumptions•• stats software normally provides output to check as a defaultstats software normally provides output to check as a default•• Howell includes interaction in the modelHowell includes interaction in the model

XXijij = = μ + μ + ααjj + + βΖβΖijij + + εεijij Error

X (the DV) for participant I in Jth group

Grand mean 1st IV – factor A – group j

2nd IV – score on variable Z multiplied by a fixed weight (beta)

1616

XXijkijk = = μμ + + ααj j + + ββk k + + αβαβjkjk + + eeijkijk

the structural modelsthe structural models

XXijij = = μμ + + ααj j + + ββZZij ij + e+ eijkijk

Factorial ANOVA model

One-way ANCOVA model

XXijij = = μμ + + ττjj + e+ eijijOne-way ANOVA model

βs are not the same!!

8/04/2011

9

1717

how ANCOVA reduces error variancehow ANCOVA reduces error variancecovariatecovariate = another IV or predictor in the model= another IV or predictor in the model–– but continuous (ordered scores, not discrete groups)but continuous (ordered scores, not discrete groups)

if the covariate is associated with the DV:if the covariate is associated with the DV:–– this relationship accounts for some systematic this relationship accounts for some systematic

variance unexplained by the focal IV variance unexplained by the focal IV –– accounting for this systematic variance reduces the accounting for this systematic variance reduces the

amount of unexplained variance in the designamount of unexplained variance in the design

–– A smaller error term because we’ve A smaller error term because we’ve partionedpartioned out out the variance due to the covariate means an increase the variance due to the covariate means an increase in statistical power in testing the effect of the focal in statistical power in testing the effect of the focal IV (just as when using blocking designs)IV (just as when using blocking designs)

1818

an example research studyan example research study

comparison of driving performance with three different car sizes: are smaller cars easier to handle?

easily addressed using 1-way ANOVA:

– DV: handling rating after 10 laps on set course– 3 performance cars are compared:

• BMW Z3 (small)• Subaru WRX (medium)• Ford GTP (large)

different groups of drivers used for each condition

8/04/2011

10

1919

μ1 μ2μo

• results show lots of overlapping variance

• this indicates a large error term

• this results in low power

to reduce the variance, we could identify a a covariate –which past research tells us is related to the DV

in this case:driving experience

handling

2020

driving experience

hand

ling

8/04/2011

11

2121

driving experience

hand

ling

2222

handling

mean handling scores

for each car

8/04/2011

12

2323

driving experience

hand

ling

mean handling scores for each car

juxtaposed with actual scores

on handling and driving experience

2424

2jijerror )XX(SS ∑ −=

in a regular ANOVA, SSerror = the sumof the squared

deviations of scores around their group

meansdriving experience

hand

ling

8/04/2011

13

2525

a lot of the variance is due to the relationship between experience and handling –we can remove this from the error term first

driving experience

hand

ling

2626

reduce error by computing pooled error term based on deviations around each group’s regression slope

driving experience

hand

ling

slope of regression line describes avg. covariance between the two variables

8/04/2011

14

2727

2)(_ ∑ −= jijerror XXSSUnadjusted

driving experience

hand

ling

the DV scores are not clustered around the mean based on random chance alone

they vary systematically (based on relationship with covariate)

unadjusted error includes

the chance variance + covariance

2828

adjusted error

driving experience

hand

ling

Estimate covariate’s effects with a regression line Calculate error as deviation from the Yhat instead of

the mean YIf the covariate is related to the DV, the regression

line is a better “anchor” around which scores cluster (smaller error)

8/04/2011

15

2929

As an adolescent I aspired to lasting fame, I As an adolescent I aspired to lasting fame, I craved factual certainty, and I thirsted for a craved factual certainty, and I thirsted for a meaningful vision of human life meaningful vision of human life -- so I so I became a scientist. became a scientist. This is like becoming This is like becoming an archbishop so you can meet girls. an archbishop so you can meet girls. ---- Matt Cartmill, anthropology professor Matt Cartmill, anthropology professor and author (1943and author (1943-- ))

3030

how does that do anything how does that do anything different different from blockingfrom blocking??

at this stage it does notat this stage it does not–– the effects of the covariate are subtracted from the the effects of the covariate are subtracted from the

error term, making it smaller error term, making it smaller –– The covariate is a more powerful way to do this if the The covariate is a more powerful way to do this if the

control variable is continuous, but it’s conceptually control variable is continuous, but it’s conceptually the samethe same

the next thing the next thing ancovaancova does is quite different does is quite different –– treatment means treatment means are adjusted to account for differences on are adjusted to account for differences on

the covariate the covariate –– random random assignment to IV conditions normally prevent assignment to IV conditions normally prevent

differences in covariate means (confounds should be designed differences in covariate means (confounds should be designed out)out)

–– But in case covariate does differ across groups, ANCOVA But in case covariate does differ across groups, ANCOVA effectively effectively partials outpartials out the effects of the covariate from the the effects of the covariate from the focal IV as well as the error termfocal IV as well as the error term

8/04/2011

16

3131

ANCOVA adjusts treatment means (DV)ANCOVA adjusts treatment means (DV)

why we would do this

if focal IV affects DV scores there is a significant difference among treatment means between the levels of the IV ☺if covariate also differs between levels of focal IV which variable explains difference in DV treatment means? confound!we care about the effect of the focal IV, not the effect of the covariateANCOVA teases apart the effects of the covariate ANCOVA teases apart the effects of the covariate and the IV by asking the question:and the IV by asking the question:“would the focal IV have an effect on the DV “would the focal IV have an effect on the DV if all participants were equivalent on the if all participants were equivalent on the covariate?”covariate?”

3232

How ANCOVA adjusts treatment means How ANCOVA adjusts treatment means on the DVon the DV

problem: participants in each level of focal IV also differ in their scores on the covariate variable

solution: Calculate the overall covariate mean. We assume this is the population mean. In an unconfounded population, all groups of the focal IV are assumed have this covariate mean.For your sample, if a group’s mean is different on the covariate than the overall covariate mean, that is a confound.Adjust the group’s mean on the DV to be what it would be if the group’s covariate mean were the overall covariate mean, by using the regression lineusing the regression line

8/04/2011

17

3333

driving experience

hand

ling

3434

in this case, there are no differences between the groups on the covariate, as you would expect, given random assignmentdriving experience

hand

ling

8/04/2011

18

3535

in this case, the 3 conditions have different covariate means

confound:what’s causing the difference in DV scores?

driving experience

hand

ling

mean scores meet on regression line

3636

1. calculate overall covariate mean

2. adjust DV scores according to regression line

3. test group main effect using adjusted meansdriving experience

hand

ling

shift DV scores to new point on regression line

8/04/2011

19

3737

here, observe a larger effect for car if adjust means so each group has average driving experience

driving experience

hand

ling

3838

logic of ANCOVAlogic of ANCOVAadjusted treatment means assume that covariate means are the same at each level of the focal IVthus, any differences in the adjusted treatment means can be attributed to the focal IV only

“would groups differ on the DV “would groups differ on the DV ifif they were they were equivalent on the covariate?”equivalent on the covariate?”refines error termrefines error term by subtracting variation that is by subtracting variation that is predictable from covariatepredictable from covariate–– larger adjustment when covariatelarger adjustment when covariate--DV relationship DV relationship

is strongis strong

refines treatment effectrefines treatment effect to adjust for any to adjust for any systematic group differences on covariate that systematic group differences on covariate that existed before experimental treatmentexisted before experimental treatment

8/04/2011

20

3939

comparison of results using 1-way ANOVA, blocking, & 1-way ANCOVA

Tests of Between-Subjects Effects

Dependent Variable: ATTRACT

231.780 2 106.890 .710 .5062263.330 15 150.8902477.110 17

SourceCarErrorTotal

Type III Sumof Squares df Mean Square F Sig.

DV = handling

effect is not significant1-way ANOVA

4040



213.780 2 106.890 4.180 .0521933.780 2 966.890 37.830 .000

99.550 4 24.890 .970 .469230.000 9 25.560

2477.110 17

SourceCarExperienceCar x ExperienceErrorTotal


reduction of error term from 150.89

to 25.56

blocking, using factorial ANOVA

DV = handling (block on experience – e.g., no training, some training, professional)


effect is marginally significant

8/04/2011

21

4141



252.040 2 126.020 8.697 .0031833.780 1 1833.780 126.555 .000202.880 14 14.490

2477.110 17

SourceCarRegressionErrorTotal


1-way ANCOVA

DV = handling (experience as a continuous scale, included as a covariate)

reduction of error

term from 150.89 to

14.49

increase in treatment effect from 106.89 to 126.02

effect is now significant!


4242

structural model and structural model and assumptions of ANCOVAassumptions of ANCOVA

8/04/2011

22

4343

ANCOVA vs blocking

blockingblocking–– conceptually simplerconceptually simpler–– requires fewer assumptionsrequires fewer assumptions

ANCOVAANCOVA–– easier to administereasier to administer–– can use continuous covariatecan use continuous covariate–– removes effect from error term removes effect from error term andand DVDV

•• usefuluseful in two situations:in two situations:–– covariate related to IV covariate related to IV andand DV (confound)DV (confound)–– covariate related to DV covariate related to DV onlyonly

does does require specific assumptionsrequire specific assumptions

4444

assumptions of ANCOVAassumptions of ANCOVAall the regular ANOVA assumptions:all the regular ANOVA assumptions:–– homogeneous variancehomogeneous variance–– normal distributionnormal distribution–– independence of errorsindependence of errors

plus:plus:–– relationship between covariate and DV is relationship between covariate and DV is linearlinear–– relationship between covariate and DV is linear relationship between covariate and DV is linear

within each groupwithin each group–– relationship between DV and covariate is equal relationship between DV and covariate is equal

across treatment groups across treatment groups -- homogeneity of homogeneity of regression slopesregression slopes

see Lecture 2 and 2nd year stats notes

8/04/2011

23

4545

re: assumption 1re: assumption 1

covariate

DV

Linear relationships

☺

Non-linear relationships

Non-linear relationships generally cannot be detected with ANCOVA – degrades power.

covariate

DV

4646

re: assumption 3re: assumption 3

covariate

DV

covariate

DV

homogeneity of regression slopes

☺

heterogeneity of regression slopes

homogeneity of regression slopes is important because adjustments to treatment means are based upon an average within-cell regression coefficient

8/04/2011

24

4747

4848

adjusting treatment effects: the fine print

the process is still considered questionable

some people object to the idea of comparing some people object to the idea of comparing adjusted treatment means at all adjusted treatment means at all –– “real” observed means are not compared“real” observed means are not compared–– comparison means are estimated using regression slope, comparison means are estimated using regression slope,

which may not be reliablewhich may not be reliable–– if treatment group does affect the covariate as well as the if treatment group does affect the covariate as well as the

DV, what does the adjusted DV mean really tell you?DV, what does the adjusted DV mean really tell you?

some people don’t mind adjusted means when the some people don’t mind adjusted means when the adjustment makes the treatment effect largeradjustment makes the treatment effect larger–– but it doesn’t always make the treatment effect larger, so but it doesn’t always make the treatment effect larger, so

it doesn’t always work in your favor!it doesn’t always work in your favor!

8/04/2011

25

4949driving experience

hand

ling

adjustment has no effecton mean differences

example A


hand

ling

example B

8/04/2011

26


hand

ling

adjustment increasesmean differences

example B


hand

ling

example C

8/04/2011

27


hand

ling

adjustment decreasesmean differences

example C

5454

A final comparisonA final comparisonThe strength of ANCOVA is The strength of ANCOVA is the ability to handle the ability to handle continuous datacontinuous data–– most psychological variables are continuously most psychological variables are continuously

distributed, distributed, –– splitting splitting people into groups is inefficient (lose info) people into groups is inefficient (lose info)

and error prone (and error prone (mismis--categorisation at group categorisation at group boundaries magnifies error)boundaries magnifies error)

–– if your data is continuous, it is best to analyse it using if your data is continuous, it is best to analyse it using a method which can deal with such data (ANCOVA is a method which can deal with such data (ANCOVA is more powerful than Blocking)more powerful than Blocking)

If adjusted and observed means are very If adjusted and observed means are very different, concerns re interpretation arisedifferent, concerns re interpretation arise

8/04/2011

28

5555

readingsreadings

analysis of covariance (this lecture)Field (3rd ed): Chapter 11Field (2nd ed): Chapter 9Howell (all eds): Chapter 16

standard & hierarchical multiple regression(next lecture)

Field (3rd ed): Chapter 7Field (2nd ed): Chapter 5Howell (all eds): Chapter 15

psyc3010 lecture 7 - psy.uq.edu.auuqwloui1/stats/3010 for... · psyc3010 lecture 7 analysis of...

Documents