today: dummy variables. dummy variables in a multiple...

Today: Dummy variables.

Dummy variables in a multiple regression, regression wrap up.

Looking back in regression, we’ve looked at how an interval

data response y changes as an interval data explanatory

variable x. Changes.

Example: Number of books read (y) as a function of television

watched (x).

Y = a + bX + e

Last time, we expanded this idea to consider more than one

explanatory / independent variable at the same time, where all

the variables were interval data.

This is called multiple regression.

Example: Wins as a function of goals for and goals against.

Y = a + b1X1 + b2X2 + e

This time, we’re going to drop the requirement for the

independent variables to be interval data. (Typo fixed)

We’re going to look at nominal data as independent data.

Recall: Nominal means name. It’s data in categories with

no natural order.

Example: Type of Fruit --- Kumquat, Coconut, Tomato,

Dragonfruit.

How do you put a type of fruit into a formula like this:

= a + bX

With a dummy variable.

“Dummy” in this case just means a simple number variable (0

or 1) that we use in the place of nominal, and sometimes

ordinal, data.

We’ve already used dummy variables.

Bearded dragon gender: 0 = Male, 1 = Female

Bearded dragon colour: 0 = Green, 1 = Fancy

Other possibilities:

0 = Non-Smoker, 1 = Smoker

0 = Domestic Student, 1 = International Student

0 = Eastern, 1 = Western

Nominal data can have more than two categories, but we can’t

do this:

Favourite colour:

0 = Blue, 1 = Green, 2 = Red

This would imply an order, and that having a favourite colour

of green is somehow the middle ground between favouring

blue and favouring red.*

*If we cared about wavelength of favourite perhaps, but usually not

Ordinal data can made into a 0,1,2,… scale, as long as we

assume the differences between each category and the next

one are about the same.

0 = Against, 1 = Neutral, 2 = For

Or

-1 = Against, 0 = Neutral, 1 = For

Then we’re treating the ordinal data like interval data.

Handling more than two categories is a for-interest topic, at

the end of the lecture if time permits.

It’s all just words until we get up and do something about it.

Dummy variables in regression:

Consider the NHL data set. Let’s see the difference in

defensive skill between the Eastern and Western conferences,

and by how much.

Dependent variable: Goals against. (More goals against means

weaker defence)

Independent variable: Conference. (East or West)

In our data set, we have conference listed in two different

ways. ConfName: E or W. Conf: 0 or 1.

0 = Eastern Conference, 1 = Western Conference.

ConfName is for when we need conference as nominal.

Conf is our dummy variable for when we need interval data.

We can do a regression by using Conf as our independent.

(SPSS won’t even let you put Confname in)

(Done under Analyze Regression Linear)

We get this model summary.

The conference alone explains .122 of the variance in goals

against.

There’s a lot to goals against that isn’t explained simply by

whether you are in the Eastern or Western Conference.

We get these coefficients.

The prediction formula is:

(Goals against) = 232.867 – 17.333(Conference)

The intercept is the response (Goals against) when the

explanatory variable x = 0.

Here, x=0 means Eastern Conference.

The intercept is the average Goals Against of teams in the

Eastern Conference.

The slope is the amount that (Goals Against) changes when

(Conference) increases by 1.

Changing x=0 to x=1 means switching for the Eastern to the

Western Conference.

So the slope b is the difference in mean goals against between

the conferences.

Here, Western Conference teams let in 17.333 fewer goals.

Plugging in x=0 or 1…

232.867 – 17.333(0) = 232.867 goals against if East

232.867 – 17.333(1) = 215.534 goals against if West

Since there’s only one independent variable, and it’s nominal,

so we COULD do this with a two-tailed independent t-test.

Analyze Compare Means Independent-Sample T Test

ConfName would be the grouping variable.

We would get the same results:

A difference of 17.333 and a 2-tailed p-value of 0.059.

So why do we bother with regression and dummy variables at

all?

Greenland has the fastest moving glaciers in the world.

Multiple regression using a dummy variable.

Let’s go back to predicting wins.

Before, we modelled wins using goals for (GF) and goals

against (GA). Now we can consider conference alongside

everything else.

Your conference (East or West) is part of what determines the

teams you play against. Teams that play against weak

opponents tend to win more.

Will conference explain anything about wins that Goals For and

Goals Against can’t?

In an SPSS multiple regression, we just include the dummy

variable in the list of independents like everything else.

First, the model summary.

Considering goals for, goals against AND conference.

82.9% of the variance in the number of wins can be explained

by these three things together.

Going back to last day, considering only Goals For and Goals

Against, we also got an R square of 0.829.

In other words, adding conference into our model told us

nothing more about wins than goals weren’t already

covering.

The R square of the model is the same with or without

conference.

That means just as much variance is explained by considering

only goals for/against as by considering both goals for/against

and the conference of the team.

Conference contributes nothing extra.

This is probably because the strength of your opponents is

already reflected in the goals for / goals against record. It’s not

like goals against weak teams count for more.

The coefficient table for Wins as a function of Goals

For/Against and Conference:

The fact that conference isn’t improving the model any is

reflected in its significance.

If it’s slope were really zero, we’d still a sample like this .952 of

the time. (p-value = .952)

The regression equation is:

(Estimated Wins) =

37.637 + 0.178(GF) – 0.167(GA) + 0.082(West Conf.)

Meaning being in the west meant winning 0.082 more games.

But

(Estimated Wins) =

37.637 + 0.178(GF) – 0.167(GA) + 0.082(West Conf.)

…is more complicated than

(Estimated Wins) =

37.950 + 0.177(GF) – 0.163(GA)

… which is the model from last day that ignored conference.

But knowing the conference doesn’t change anything.

- The r2 was .829 whether we included conference or not.

- We failed to reject the null that the effect of conference

was zero (controlling for Goals For/Against ).

In that case, we can use the simpler model that only uses goals

and not lose anything. We should always opt for a simpler

model when nothing is lost in doing so.

This is called the principle of parsimony.

"Make everything as simple as possible, but not simpler."

- Nikola Tesla Albert Einstein

Comments about r2 in multiple regression.

Like with single variable regression, r2 must be between 0 and

1.

0 is none of the variance is explained.

1 is all of it is explained.

If you add more and more variables into your model, you will

eventually reach r2 = 1, where you have enough data to model

and predict the response perfectly.

But each variable uses up a degree of freedom and makes the

results harder to interpret.

Just because you can include a variable doesn’t mean you

should.

(Resting heart rate) = a + b1(Age) + b2(Body Mass Index) + b3(L

of Oxygen per Minute) + b4(Height) + b5(Number of Freckles) +

b6(Enjoyment of Sushi) + b7(Kitchen Sinks Owned)

Again, this violates the principle of parsimony.

More regression practice.

From dragons.sav, we have the weight of bearded dragons as a

function of their age, length, and sex.

What is the intercept?

Weight of beardies as a function of age, length, and sex.

What is the intercept?

-551.125

What does it mean?

Weight of beardies as a function of age, length, and sex.

What is the intercept?

-551.125

What does it mean?

A male bearded dragon with 0 years, 0 length, weighs

negative 551 grams. (not real-world useful)

How much heavier is a bearded dragon if it ages two years and

doesn’t get any longer or change sex? (On average)

The slope for age is 17.191, so a dragon would get

2 * 17.191 = 34.382 grams heavier with 2 extra years

(controlling for length and sex)

Is there a significant difference in weight between male and

female dragons of the same age and size?

Is there a significant difference in weight between male and

female dragons of the same age and size?

No. The p-value against there being no difference is .441, so

we fail to reject that null.

What does the regression equation look like?

What does the regression equation look like?

(Esimated Weight) =

-551.1 + 17.1(Age) + 34.3(Length) + 4.9(Female)

How much does the average bearded dragon weight if he’s..

- Male

- 3 Years Old

- 24 cm long

How much does the average bearded dragon weight if he’s..

- Male

- 3 Years Old

- 24 cm long

(Esimated Weight) =

-551.1 + 17.1( 3 ) + 34.3( 24 ) + 4.9( 0 )

= 323.4 grams

Is there a model that likely works just as well but is simpler?

Yes. It’s likely that a model without considering sex would

explain nearly as much of the variance.

From model summaries:

Model with Age, Length, Sex: r2 = .912

Model with Age, Length: r2 = .912 (Not always so exact)

For interest: Nominal data of 3+ categories.

Dummy variables HAVE to be 0 or 1. If not, you’re treating

nominal categories as if they have some sort of order.

If you have 3 categories, you need 2 dummy variables.

Each of the dummy variables is 1 only when a particular

category comes up, and 0 all the other times.

One of the categories is considered a baseline, or starting

point. All of the dummy variables will be 0 for that category.

(Here: Blue is the baseline, all the dummy variables are 0 for it)

Since a colour can’t be red and green at the same time, only

one of the dummy variables will ever be 1 for a particular case.

Doing a linear model with just these two dummy variables

would look like:

= a + b1(Red) + b2(Green) Which would be

= a for blue cases.

= a + b1 for red cases.

= a + b2 for green cases.

= a + b1(Red) + b2(Green)

a , the intercept, the value when Red=0 and Green=0

is the average response for blue cases.

b1 is the average increase/decrease in the response when

the case is green instead of blue.

b2 is the average increase/decrease in the response when

the case is red instead of blue.

Next time: Midterm 2 post-mortem.

Reintroduction to contingency, Odds and Odds Ratios.