Sociology 680

Multivariate Analysis:

Analysis of Variance

A Typology of Models

IV

DVCategory Quantity

Quantity

Category

Linear Models

Category Models

1) Analysis of Variance

Models(ANOVA)

2) Structural Equation Models(SEM)

3) LogLinear Models(LLM)

4) Logistic Regression

Models(LRM)

Examples of the Four Types

1. The effects of sex and race on Income

2. The effects of age and education on income

3. The effects of sex and race on union membership

4. The effects of age and income on union membership

The General Linear Model

Recall that the bi-variate Linear Regression model focuses on the prediction of a dependent variable value (Y), given an imputed value on a continuous independent variable (X).

The variation around the

mean of Y less the variation around the regression line (Y’) is our measure of r2

Y (Weight)

.. .… .. . . …. . … . … . . .. .. . .. . …. . … . …. ... … . ... ...

X (Height)

Y’

Y

The General Linear Model (cont.)

Fixing a value of (X) and predicting a value of (Y) allows us to use the layout of points, under an assumption of linearity, to determine the effect of the IV on the DV. We do this by calculating the Y’ value in conjunction with the standard error of that value (Sy’) Where:

and

Y (Weight)

.. .… .. . . …. . … . … . . .. .. . .. . …. . … . …. ... … . ... ...

X (Height)

Y’

Y

Y’ {)(' XX

Sx

SyrYY i

2' 1 rSS yy

An Example of Simple Regression

)(' XXSx

SyrYY i

Given the following information, what would you expect a student’s score to be on the final examination, if his score on the midterm were 62? Within what interval could you be 95% confident the actual score on the Final would fall (i.e. what is the standard error)?

Midterm (X)   Final (Y)

= 70 = 75

Sx = 4 Sy = 8  

r = 0.60

X Y

2' 1 rSS yy

Y’ = 75 + (0.6)(8/4)(62-70) = 65.4

= 8 (.8) = 6.4

The Test of Differences

But now assume that the goal is not prediction, but a test of the difference in two predictions (e.g. “are people who are 5’8” significantly heavier than those who are 5’4”). That difference hypothesis could just as easily be recast as “Are taller people significantly heavier than shorter people, where taller and shorter connote categories.

Y (Weight)

.. .… .. . . …. . … . … . . .. .. . .. . …. . … . …. ... … . ... ...

X (Height)

Y’

Y

Y’1

Y’2

The t-test

If there are simply two categories, we would be doing an ordinary t-test for the difference of means where:

Y (Weight)

. ..

... …. ... . .. .. . ... …. … .. . Shorter Taller

| | X (Height)

Y’

Y

Y’1

Y’2

2

22

1

12

21

NS

NS

XXt

xx

Analysis of Variance

If we were to have three categories, the test of significance becomes a simple one-way analysis of variance (ANOVA) where we are assessing the variance between means (Y’s) of the categories in relation to the variation within those categories, or:

Variance Between Categories

Variance Within Categories

Y (Weight) . ..

... …. ... . .. .. . ... …. ... . .. .. . ... .... ... .. .Short Med Tall | | |

X (Height)

Y’

Y

Y’1

Y’2

Y’2

Three Types of Analysis of Variance

One Way Analysis of Variance - ANOVA (Factorial ANOVA if two or more - IVs)

Analysis of Covariance - ANCOVA (Factorial ANCOVA if two or more - IVs)

Multiple Analysis of Variance (MANOVA) (Factorial MANOVA if two or more 2IVs)

Simple One Way ANOVA

Concept: When two or more categories of a non-quantitative IV are tested to see if a significant difference exists between those category means on some quantitative DV, we use the simple ANOVA where we are essentially looking at the ratio of the variance between means / variance within categories. As an F-ratio:

i jJij

gji

kNXX

kXXnF

/)(

1/)(2

2

F-ratio = Bet SS/df divided by Within SS/df.

As a formula it is:

Example of a simple ANOVA

Suppose an instructor divides his class into three sub-groups, each receiving a different teaching strategies (experimental condition). If the following results of test scores were generated, could you assume that teaching strategy affects test results?

In Class

At Home

Both C+H

115 125 135

135 145 155

140 150 160

145 155 165

165 175 185

Grand Mean = 150

140 150 160

Step 1: State hypotheses: Ho: 1 = 2 = 3;

Step 2: Specify the distribution: (F-distribution)

Step 3: Set alpha (say .05; therefore F = 3.68)

Step 4: Calculate the outcome:

Step 5: Draw the conclusion: Retain or Reject Ho:

Type of instruction does or does not influence test scores.

Example of a simple ANOVA (cont.)

Example of a simple ANOVA (cont.)

Source SS df MS F

Bet 1000 2 500 1.54

Within 3900 12 325

i jJij

gji

kNXX

kXXn

/)(

1/)(2

2In

ClassAt

HomeBoth C+H

115 125 135

135 145 155

140 150 160

145 155 165

165 175 185

Bet SS = ((5(140-150)2 + 5(150-150)2 +5 (160-150)2)) = 1000

Bet df = 3-1 = 2

W/in SS = (115-140)2 + (135-140)2 + (140-140)2 + (145-140)2+ (165-140)2 + (125-150)2 + (145-150)2 + (150-150)2 + (155-150)2 + (175-150)2 + (135-160)2 + (155-160)2 + (160-160)2 + 165-160)2 + (185-160)2 = 3900

W/in df = 15 – 3 = 12

SPSS Input for One-way ANOVA

SPSS Output from a simple ANOVA

Two Way or Factorial ANOVA

Concept: When we have two or more non-quantitative or categorical independent variables, and their effect on a quantitative dependent variable, we need to look at both the main effects of the row and column variable, but more importantly, the interaction effects.

Example of a Factorial ANOVA

In Class At Home Both C+H

115 125 135

135 145 155

140 150 160

145 155 165

165 175 185

Not Working

Working

Means 140 150 160 150

135

160

SPSS Input for 2x3 Factorial ANOVA

SPSS Output from a 2x3 ANOVA

Analysis of Covariance (ANCOVA)

Concept: Not unlike a 2-way ANOVA, ANCOVA introduces a second independent variable. However, it is not always subject to experimental control (as would be the case in a 2-way ANOVA) and is typically quantitative in nature. Therefore we treat the second IV as a “covariate” of the DV.

Example: to study the effect of race and education (IVs) on income (DV), we would adjust the racial differences by the correlation between education (the covariate) and income. This reduces the residual / error variance (which is the denominator in the F-ratio for the main effect of racial differences).

ANCOVA (cont.)

Income Education Income Education Income Education5 12 1 7 7 94 13 3 11 7 125 11 5 13 9 115 14 4 13 10 146 16 5 16 9 178 15 6 15 12 169 17 6 16 11 1710 18 7 17 12 1811 19 8 18 13 217 15 5 14 10 15

White Black Other

In this example, we are essentially subtracting the covariance of X&Y from both the Bet SS and Within SS of the racial categories:

SPSS Input from an ANCOVA

SPSS Output from an ANCOVA

SPSS Input for Factorial ANCOVA

SPSS Output for Factorial ANCOVA