1

Day 4Correlation and Regression

by Binam Ghimire

Example DataDay Output (TONS) Cost £000

1 23 582 17 503 24 544 35 645 10 406 16 437 15 428 24 509 18 5310 30 62

The scatter diagram of the data would appear as below:

5 10 15 20 25 30 35 4040

45

50

55

60

65

70

Alternatively a negative correlation would appear as below:

5 10 15 20 25 30 35 400

10

20

30

40

50

Alternatively data with no correlation may appear as below:

0 5 10 15 20 25 30 35 400

10

20

30

40

50

60

Correlation Scale

-1 0 +1

Perfect negative No correlation Perfect positive correlation correlation

Pearson’s product moment correlation coefficient (r)

r = n ∑ xy - ∑x ∑y

√ [n ∑x2 - (∑x)2] [n ∑y2 - (∑y)2]

x y xy x2 y2

23 58 1334 529 3364 17 50 850 289 2500 24 54 1296 576 2916∑ 212 516 11452 5000 27242

Pearson’s product moment correlation coefficient (r) or the formula may be

Pearson’s product moment correlation coefficient (r) (2)

r = 10 x 11452 – 212 * 516

√ [10 x 5000 – (212)] [10 x 27242 – (516)]

= 5128

√ 5056 x 6164

= 0.9186

Linear Regression

Need to establish a ‘line of best fit’ The ‘freehand method’ has many drawbacks.

In some sense we need the ‘best fit’ to the data. To obtain this we do not use crude graphical techniques. We identify the ‘line of best fit’ or ‘least squares line.’

Linear Regression (2)

The equation for this line is Y = 30.10 + 1.014X

Linear Regression (3)

The equation of this line is Y =30.10 +1.014XBut how is this obtained?

The scattered points illustrate the actual data, while the least squares line is an estimate of Y for a given value of X. Notice the distance between the scattered points and the line; this will give you some idea of how good a fit the line is.

Linear Regression (4)

How do we determine the least squares line?

Simply we need to determine the intercept (a) and the (b) gradient.

The formula is therefore Y = a + bx

You need to apply a little calculus (we will omit that process here) to develop standard equations.

Linear Regression Equations b = n ∑ xy - ∑ x ∑ y

n ∑X2 – (∑X)2

b = 10 x 11452 – 212 x 516 10 x 5000 – 44944

b = 1.0142405 Or

2ˆ

ˆˆ

XXXXYY

i

ii

Linear Regression Equations (2)And

a = Y – b. X or

a = 51.6 – 1.0142405 x 21.2

a = 30.098101

Rounding these values a little:Y = 30.10 + 1.014X

XxYi

Coefficient of Determination The coefficient of determination measures the

proportion of the variation in the dependent variable (y) explained by the variation in the independent variable (x).

It is reported as r2 - the square of the product moment correlation coefficient.

Does not explain causation

Coefficient of Determination (2) For our previous example:

r2 = (0.9186)2 = 0.844

This means that 84.4% of the variation in cost is dependent upon output volume. Alternatively, 15.6% of variation is not explained.

So far Correlation is measured on a scale from -1 to +1

using Pearson’s product moment correlation coefficient (r).

Linear regression identifies the line of ‘best fit’ using the formula Y = a + bx

The coefficient of determination (r2) measures the extent to which the dependent variable is explained by the independent variable.

Question to learn the terminologies The data below shows annual company income (£m) against year of

trading.

Year Income (£m)

1 202 233 264 285 35

A regression of income on year gives the following results:

r = 0.974, r squared = 0.948, intercept = 11.4, slope = 3.5  Can we explain each of the results above. Use the results above to make a forecast for company income for year

6. What assumption is made in making this forecast?

Coefficient of Determination (3) Relationship among SST, SSR, SSE:

SST = SSR + SSE

Where,SST = total sum of squares given by

SSR = sum of squares due to regression

SSE = sum of squares due to error

Coefficient of Determination is : r2 = SSR/ SST

Coefficient of Determination (4) Relationship among SST, SSR, SSE:

SST = SSR + SSETotal Variation= Explained Variation + Unexplained

Variation

Coefficient of Determination is : r2 = SSR/ SST

VariationTotalVariationExplained

SSTSSE1

SSTSSESST

SSTSSRr2

Calculating the Standard Error of the Estimate (SEE)

SEE measures the accuracy of the prediction from a regression equation It is the standard dev. of the error term The lower the SEE, the greater the accuracy

Where SSE = sum of Squared Errors

2nSSESEE

Covariance of Rates of Return

n

t,1 1 t,2 2t 1

1,2

R R R Rcov

n 1Example: Calculate the covariance between the returns on the two stocks indicated below:

Covariance Using Historical Data

R2 = 0.07

Σ = 0.0154

Cov = 0.0154 / 2 = 0.0077 R1 = 0.05

Sample Correlation CoefficientCorrelation, ρ, is a standardized measure of covariance and is bounded by +1 and –1

1,21,2

1 2

Cov

1,2

0.0051 0.6620.07 0.11

Example: The covariance of returns on two assets is 0.0051 and σ1= 7% and σ2= 11%. Calculate ρ1,2.

Testing H0: Correlation = 0The test of whether the true correlation between two random variables is zero (i.e., there is no correlation) is a t-test based on the sample correlation coefficient, r. With n (pairs of) observations the test statistic is: Degrees of freedom

is n – 2

ExampleData:n = 10r = 0.475 Determine if the sample correlation is significant at the

5% level of significance.t = 0.475 (8)0.5 / [1 – (0.475)2] 0.5

= 1.3435 / 0.88 = 1.5267 The two-tailed critical t – values at a 5% level of

significance with df = 8 (n-2) are found to be +/- 2.306.

Since -2.306≤ 1.5267≤ 2.306, the null hypothesis cannot be rejected, i.e. correlation between variables X and Y is not significantly different from zero at a 5% significance level.