university of oxford national data – local knowledge using administrative data david mclennan...

University of Oxford

National data – local knowledge

Using administrative data

David McLennan & Kate Wilkinson

Social Disadvantage Research CentreDepartment of Social Policy and Social Work



Administrative data: a targeting, monitoring and evaluation resource


National Strategy for Neighbourhood Renewal

Aims to ‘narrow the gap’ between the most deprived neighbourhoods and the rest of the country of a range of key outcomes

Five priority themes identified:

Lower Worklessness

Lower Crime

Better Health

Better Skills

Better Housing the Physical Environment


Measuring Outcomes

Establish baseline

Pre- and Post-intervention

NDC v LA v Region v England

Time series

What is success?

Sustainability


Evaluation data sources

Census+ near 100% coverage of entire population+ results reliable at small area level- only every ten years- few suitable indicators of social deprivation- extremely expensive

Surveys+ clear research focus

+ many valuable indicators of social deprivation

- sampling error

- results often not reliable at small area level

- very expensive

Administrative Data+ near 100% coverage of population of

interest+ constantly updated+ results reliable at small area level+ already collected for operational purpose- some indicators are proxies- dependent upon support of data providers- data protection


Worklessness and Low Income


Worklessness & Low Income

Worklessness

Worklessness: Unemployment + Work-Limiting Illness

Unemployment: numbers and proportions of people aged 16-59 receiving Job Seekers Allowance

Work-Limiting Illness: numbers and proportions of people aged 16-59 receiving Incapacity Benefit or Severe Disablement Allowance

‘Exit Rates’ from unemployment, illness and overall worklessness

Low Income

Proportion of adults and dependent children (aged 0-59) living in households receiving means-tested out-of-work benefits (Income Support + income-based Job Seekers Allowance)


Work and Pensions Longitudinal Study

Database of all spells of benefit receipt (DWP) and all tax records (HMRC) from June 1999 onwards.

Spells linked together using individual person unique reference number.

Includes details of person’s age, gender, home postcode, number of children, age of youngest child, spell type, spell start and end etc.

Over 164 million records and growing…

DWP’s primary research tool and the source for their neighbourhood statistics data


Worklessness example

An NDC area sees its unemployment rate change as follows: 12% in 2001 10% in 2003 8% in 2005while the local authority unemployment rate stays the same.

Success?

Change in unemployment rate could be due to:a) unemployed people in the NDC area moving into jobsb) unemployed people in the NDC becoming unable to work due to illnessc) new people moving into the NDC area who are not unemployed


Health


Health Indicators

Standardised Mortality Ratio

A measure of the number of deaths in the NDC area compared to the expected level given the area’s age and gender structure

Standardised Illness Ratio

A measure of the prevalence of illness in the NDC area compared to the expected level given the area’s age and gender structure

Mental Illness Rate

proportion of adults under 60 suffering from mood or anxiety disorders in each area

Low Birth Weight

Percentage of single live births classed as low birth weight in a 5 year time period


Standardised Mortality Ratio


Crime


Crime Indicators

Violence Rate

Number of violent crimes per 1000 ‘at-risk’ population

Burglary Rate

Number of burglaries per 1000 ‘at-risk’ properties

Theft Rate

Number of thefts per 1000 ‘at-risk’ population

Criminal Damage Rate

Number of criminal damage crimes per 1000 ‘at-risk’ population

‘Total Crime’ Rate

Number of violence, burglary, theft and criminal damage crimes per 1000 ‘at-risk’ population


Crime Rate Numerators

Individual level recorded crime from all 39 police forces

Crime type, date/time occurrence, date recorded, grid reference and/or postcode of occurrence

33 different crime types under the broad headings of: Violence crime

Burglary

Theft

Criminal Damage


½ a crime to non-NDC area

½ a crime to NDC area


Estimates of properties / population ‘at-risk’

At risk properties =Residential properties+ Commercial/industrial properties

At risk population = Resident population (minus prison)+ Workers+ Students+ Shoppers / recreational users + Passers by+ etc…



At risk properties =Residential properties Total Dwellings (Census)+ Commercial/industrial properties + OS Address Point

At risk population = Resident population (minus prison) Resident

Population (estimates)+ Workers - Prison Pop+ Students + Workplace Pop (Census)+ Shoppers / recreational users + Passers by+ etc…



At risk properties =Residential properties Total Dwellings (Census)+ Commercial/industrial properties + OS Address Point

At risk population = Resident population (minus prison) Resident

Population (estimates)+ Workers - Prison Pop+ Students+ Students + Workplace Pop (Census)+ Shoppers / recreational users+ Shoppers / recreational users + Passers by+ Passers by+ etc…+ etc…


Education


Indicators & data sources

INDICATORS DATA SOURCES

Pupil attainment at Key Stage 2 (age 11) - % achieving level 4 in English, maths, science

Pupil Level Annual Schools Census (PLASC) – DfES, collected annually, pupil information including home postcode

National Pupil Database (NPD) – DfES, collected annually, pupil test scores

Pupil attainment at Key Stage 3 (age 14), % achieving level 5 in English, maths, science

Pupil attainment at Key Stage 4 – GCSE (age 16), % achieving 5 or more A*-C grades

% pupils staying in full-time education post 16

Child Benefit – HMRC, annual snapshot, counts of children by age, area of residence and gender

% 18-20 year olds accepted to higher education

Universities and Colleges Admissions Service (UCAS) and Higher Education Statistics Agency (HESA) – collected annually, includes age, outcome of application and postcode


Using the education data

A picture of educational performance and attainment from age 11-20 to allow comparison: over time; with Government targets; with district, regional and national figures

Cohort tracking – track performance of 2002 KS2 cohort to 2005 KS3

Example – KS4 performance 2002-2005, NDC, LA and Region

30%

35%

40%

45%

50%

55%

60%

2002 2003 2004 2005

% 5

or

mo

re A

*-C

at

GC

SE

NDC

Local authority / Unitary authority

Region


Data limitations – some examples

Staying on rate and entry to higher education indicators are proxies not actual measurements

Pupil cohorts can have different characteristics across years

Education indicators may not be comparable with locally sourced statistics or national statistics from different providers:

GCSE indicator dependent on whether pupils are included who left school at 16 and were not entered for any exams

KS2 and KS3 also depend on whether or not pupils who are absent for the test are included in the denominator( for example)

DfES generally produces data at school level rather than area level – all NDC pupils do not go to the same schools…


Coping with data limitations

Look at outcomes across the age range – performance across the range of indicators often varies by NDC

Consider change across a long time-series – data is available from 2002-2005, looking at a longer time period is a better indicator of long-term trends

Use survey data to supplement administrative indicators Remember that indicator definitions are consistent between areas and

over time BUT other factors (i.e. pupil characteristics) can influence performance and these vary over time and between areas

In Summary… Performance across the range of education indicators varies within an

NDC Changes over time are important as well as relative performance – all

areas start from a different baseline


NDC variation across education indicators

1 6 11 16 21 26 31 36

Range of ranks

London

Midlands

North

South


Housing


Indicators & data sources

Mean price of houses sold by type: flats, terraced, semi-detached, detached, Source: Land Registry

Number of houses sold by type: flats, terraced, semi-detached, detached, Source: Land Registry

Uses & limitations

Comparing change in house prices over time and relative to district, region and England

However…. No information about type of house i.e. no. of bedrooms so difficult to make

accurate comparisons No information about turnover i.e. rate of house sales – will be available in the

future From current data difficult to draw conclusions about area desirability


Population Estimates


Methodology & data sources

Population counts by age and gender within 5 year age groupings from 1999-2005

Population counts from various administrative data sources: Child Benefit (0-14) Patient Registration (0-90+) Super Older Persons Database (65+)

Only Patient Registration data covers the entire age range and all data sets known to have weaknesses in particular areas and for particular age ranges

Methodology developed to test data reliability and make population estimates based on relative accuracy of each data source

ONS work on producing small area estimates used to supplement and improve our methodology


Uses & limitations

Population change can be an indicator of area desirability….. BUT Estimates are estimates and may be inaccurate – they rely on the quality of the

administrative data Local authority mid-year estimates have been revised since 2001 Populations may change as a result of housing regeneration

-8%

-6%

-4%

-2%

0%

2%

4%

6%

8%

% t

ota

l p

op

ula

tio

n c

han

ge

2001

-200

5

LondonMidlandsNorthSouth


Summary


Administrative data indicators

Useful tool for measuring performance at NDC level because:

Data is collected routinely for operational purposes so enables a consistent time-series to be built up

They enable measurement across a variety of themes so can be linked to specific interventions

They do not suffer from sampling error so reliable at small area level

They can be easily and consistently compared over time and between areas

but there are limitations….


Administrative data limitations

There may be differences in definitions, data sources and time collection points between local and national data

National data available from different providers and may use different definitions

Sometimes indicators are proxies or estimates of events or outcomes

Care needed attributing indicator changes to programme interventions

Suggestions or comments on indicator packages and data provision are welcome and are best directed through the NDC Reference Group.

university of oxford national data – local knowledge using administrative data david mclennan...

Documents

unemployed slide

ndc area

low income slide

sustainability slide

unemployment work

illness unemployment

evaluation resource

expensive administrative