Andrew Smith

Denoising UK house prices14th April 2010

2 Regression on a graph

• Denoising UK house prices

• Discrete spatial processes

• Longitudinal data analysis

• Scatterplot smoothing

• Image analysis

3 Outline

• UK house price data

The need for regression

• Graph structure

Regression on a graph

• Penalised regression

Results and uses of algorithm

4

Video available at

http://www.maths.bris.ac.uk/~as1637/research/warwick1.wmv

5 Regression

• Seek a (simpler) pattern that explains observed data

• Model:

Data = Signal + Noise

Priceyear,town = True valueyear,town + σyear Zyear,town

6

• No covariate values

• Euclidean distance

inappropriate

• Missing values

Challenges for existing methods

7 Regression on a graph

• Consider observations Priceyear,town to be taken at the

vertices of a graph

• Edges of the graph give an idea of closeness

1999 2000 2001

8

• Consider observations

Priceyear,town

to be taken at the

vertices of a graph

• Edges of the graph

(neighbouring towns)

give an idea of closeness

Regression on a graph

9 Penalised regression on a graph

• Need to penalise

distance from data

• Penalty term:

Sum (over all vertices)

of squares of residuals

10 Penalised regression on a graph

• Need to penalise

roughness

• Penalty term:

Sum (over all edges)

of absolute differences

11 Penalised regression on a graph

• Minimise:

Distance from data

+ λ Roughness

• Computationally intensive,

so use new algorithm

12

Video available at

http://www.maths.bris.ac.uk/~as1637/research/warwick2.wmv

13 Regional analysis

• The estimate identifies

regions of constant value

• These regions change in a

similar way through time

14 Regional analysis

• The estimate identifies

regions of constant value

• These regions change in a

similar way through time

• They are not the same as

government office regions

15 Summary

• Detecting a smooth national trend in noisy UK

house price data

• Use regression on a graph

• Penalise distance from data (vertices) and

roughness (edges)

• arxiv.org/abs/0911.1928 (Kovac & Smith 2009)

• All house price data courtesy of Halifax