Introduction to the key large-scale government

surveys

Jo Wathan, Paul Norman & Angela Dale ESDS Government

Centre for Census and Survey Research (CCSR)

University of Manchester

Our mission…

• Intro to Economic and Social Data Service (ESDS) and how we can help you.

• What data is available, how it can be used in research

• Indication of good practice• Registration, access, support

services/resources• Short hands-on session with real data

ESDS Government

• One of four specialist services of ESDS. ESDS is a new national data service (since Jan 03)– ESDS Government– ESDS Longitudinal– ESDS Qualidata– ESDS International

• ESDS Government provides access and user support for key large-scale government surveys such as Labour Force Survey and General Household Survey

• Access remains via the UKDA

                                                                 

ESDS Government - some of the things we do!

• Helpdesk

• Survey pages incl. how to get started

• Online guides – SPSS, STATA, Weighting, Employment Research, Health Research

• User Group seminars (data users and data creators)

• Publications Database

• Derived variables - consistent over time- consistent with Census

• Teaching datasets

• Training

http://www.esds.ac.uk/government

Which surveys?

UK or GB surveys • General Household Survey• Labour Force Survey• Family Resources Survey • Expenditure and Food Survey (previously the National

Food Survey and Family Expenditure Survey)• ONS Omnibus Survey • National Travel Survey • Time Use Survey • Life & Times/Young People’s Social Attitudes

British Crime Survey /Scottish Crime Survey• British Social Attitudes /Scottish Social

Attitudes/ Northern Ireland

• Health Survey for England/Wales/Scotland• Survey of English Housing (England only)

Microdata

QUALITY OF DATA (1)

• Two main data collectors:– Office for National Statistics (ONS) – National Centre for Social Research (NatCen)

• Both have considerable experience– ONS Social Surveys started in 1941– Natcen founded in 1969 (as SCPR)

• Permanent panels of highly trained field interviewers

• Management and Quality Checking• (Relatively) high response rates – but falling• Widespread use by secondary analysts

QUALITY OF DATA (2)Example of GHS data collection

STAGE/ PROCESSCommisssionSampling/allocationQ'r developmentSurvey documentationPilot studyInterviewing training FieldworkData processingAnalysing and reportingPublicationArchiving/review etc.

PRE-FIELDWORK

POST FIELDWORKFIELDWORK

What would you use the data for?

• Straightforward secondary analysis– To assess theoretical accounts– To quantify characteristics or behaviours– To challenge official views– To apply alternative definitions

• Context to your own primary research – Your research could be quantitative or

qualitative– To assess the national context of an area study– To assess whether your sample is typical– To assess the scale of behaviours

Practical research uses of the data

• Looking at change over time

• Look at sub-populations

• Using the flexibility of the data to look at alternative definitions

• Looking within households

Change over time

Secondary analysis:change over time among

sub-populations

Marmot, M (2003)

SMOKING AND SOCIAL CLASS - MEN

0

510

15

2025

30

3540

45

1994 1995 1996 1997 1998 1999 2000 2001

year

%

all sc I&II sc IV&VSource:HSE

Using successive cross-sectional data over time

Pros…• Reasonable amount of

comparability• Can pool

years/quarters• Data is representative

at each time point• Good at looking at

impacts on groups

Cons…• Limits to continuity in

the data (e.g. ethnic)• Cannot establish

individual change

Looking at small populations

• Only the Samples of Anonymised Records have larger sample sizes

• Many surveys with 10+k respondents– Permits minority groups to be represented– Rare subpopulations sample size may be too

small… can consider combining years if appropriate

Survey data is subject to sampling error!

Example: Pregnancy and Employment

•Using 1998-99 General Household Survey data alone there are only 168 pregnant women aged 16-49

•95% Confidence interval for % pregnant women economically inactive 34.2 – 49.1%

•Combined 3 years’ data to obtain sample of 465 pregnant women

•Confidence interval using 3 years’ data: 34.9 – 43.9%

Combining datasets to increase sample size

Use the hierarchy to…

• Better describe the household• Describe the household context of an

individual• Look at intra-household differences

(& sameness)

Workless households (source FES, various years 1968-1996)

0

5

10

15

20

25

68 70 72 74 76 78 80 82 84 86 88 90 92 94 96

Year

Pe

rce

nta

ge

(o

f p

res

en

t w

ork

ing

ag

e h

oh

)

workless households

children in worklesshouseholds

Source: Richard Dickens, Paul Gregg and Jonathan Wadsworth(2000) ‘New Labour and the Labour Market, CMPO Working Paper Series00/19 Table 5

Using the flexibility of the data to look at alternative

definitionsWhat are ‘hours worked’?

• Is it just paid work? Or unpaid as well?• Hours usually worked, or actually worked last week?• In main job, or in any job? • What about students?• Overtime – paid?• Overtime – unpaid?• Lunch hours?• Do non-workers work zero hours or should they be

excluded?

Choosing a survey for research

• Which surveys cover your main topic?

• Which other topics are you interested in?

• Measurement over time

• Geography

• Respondents – whole household, children?

• Sample size

Using the data in teaching• Methods courses

– Using the data in a hands on manner– Using substantive exemplars to demonstrate a

methodological point– Using the surveys as methodological exemplars

• Substantive courses– Making your point using data– Integrating methods into substantive courses

• Teaching datasets– General Household Survey– Labour Force Survey– British Crime Survey– Health Survey for England

Documentation you’ll need:• Questionnaire

• Code book of Variables

• Description of Derived Variables

• Definitions

• Methodology including•Sampling method

•Response achieved

•Population base

• Published reports

Unless you can track the variable back to the question asked, and work out who it was asked of, you don’t understand the data!

Published report (1)

Documentation - GHS Questionnaire

Documentation - GHS Questionnaire

Survey year : 2000/05Variable Name : ECSTILOVariable Label : ECON STAT LST WK -

ILO UNEMPLOYMENT DEFTopic : EmploymentPopulation : AdultsStandard/trailer : StandardHhld/indiv.level : IndividualRange : 1 to 10Missing values : -6, -8Priority coded : YProgram : SDate written : 12.04.96Date last amended : 30.11.98VALUE LABELS ECSTILO( -8 ) NA, ECSTA NOT KNOWN( -6 ) UNDER 16, MS( 1 ) WORKING (INCL UNPAID FW)( 2 ) GOVT SCH WTH EMP( 3 ) GOVT SCH AT COLL( 4 ) UNEMP (ILO DEF)( 5 ) OTHER UNEMPLOYED( 6 ) PERM UNABLE WORK( 7 ) RETIRED( 8 ) KEEPING HOUSE( 9 ) STUDENT( 10 ) OTHER ECON INACT

derivation :IF (SCHEDTYP EQ 3 OR AGE LT 16) = -6ELSEIF (TRNCHKA EQ 3) = 1ELSEIF (TRNCHKA EQ 1) = 2ELSEIF (TRNCHKA EQ 2) = 3ELSEIF (UNPDWKR EQ 1) = 1ELSEIF (TRNCHKA EQ 4) = 4ELSEIF (TRNCHKA EQ 5)IF (ABLESTRT EQ 1) = 4ELSEIF (ABLESTRT EQ 2) = 5ELSE = -8ELSEIF (TRNCHKA EQ 6)IF (LOOKWORK EQ -8 OR ABLSTRT EQ -8) = -8ELSEIF (LOOKWORK EQ 1 AND ABLESTRT EQ 1) = 4ELSE = 5ELSEIF (TRNCHKA EQ 7)IF (ACTIVITY EQ 2) = 6ELSEIF (WANTAJOB EQ -8 OR NABLSTRT EQ -8 OR NLOOKWRK EQ -8) = -8ELSEIF (WANTAJOB EQ -9) = 7ELSEIF (NABLSTRT EQ 1 AND NLOOKWRK EQ 1) = 4ELSEIF (ACTIVITY EQ 3) = 7ELSEIF (ACTIVITY EQ 4) = 8ELSEIF (ACTIVITY EQ 5) = 10ELSEIF (ACTIVITY EQ 1) = 9ELSEIF (TRNCHKA EQ -9) = -8

Derived variables

Some other useful resources

• Question Bank– Linked to from ESDS– Contains searchable questionnaires for major

surveys– Contains articles about major topics/concepts– Runs the survey link scheme– http://qb.soc.surrey.ac.uk

• PEAS (Practical Exemplars on the Analysis of Surveys)– Contains learning materials on the impact of

survey design on analysis (e.g. weighting and complex sample designs)

– Uses the government surveys as exemplars– http://www.napier.ac.uk/depts/fhls/peas

• ONS website contains published reports, methodology etc.– http://www.statistics.gov.uk

To find out more about other sources…

• Census data: http://www.census.ac.uk• Economic and Social Data Service has 3

other specialist services – Longitudinal– International – Qualidata– Plus ‘core’ function for other datasets held at the

UK Data Archive

http://www.esds.ac.uk

Accessing the data• Need to be registered with ESDS

– Same system as the Census Registration System

• Easiest and free for academic users– Register online using your ATHENS username and

password– Simple online form, takes about 10 minutes– No charge to download via the web

• Paper form can be used for teaching– Lecturer takes responsibility for the class– Students simply fill their details in on a paper form– Return form to the UK Data Archive

• Non-academic users – Will need to apply for an Athens account first– £500 Charge for commercial use per study

The licence• This is not public data!• We need to know who you are, how to contact

you and what you are using the data for• Agree not to attempt to identify individuals• Only use the data for your stated purpose (you can

re-register if you want to use the data for another use)

• Do not pass the data to unregistered parties (that includes deleting the data before passing on PCs!)

• Tell us if you publish using the data• Some more sensitive or detailed datasets require

more stringent licensing procedures– Special conditions: British Crime Survey– Special licence: Annual Population Survey

Obtaining data once registered

• Choice of downloading entire files:– SPSS, Stata, tab delimited formats for most files– Comes zipped up with documentation

• Or explore the data and download subsets in Nesstar (c. 200 files in this format – including most of the Government surveys)– Allows you to explore metadata without being

registered– Can do basic exploratory anlyses (including OLS)

online without downloading files– Can define subsets to download in a wider range

of formats including SAS

Good practice:Some pointers

• Irresponsible use can threaten future access• Keep data secure • You won’t understand the data unless you consult

the documentation!• Consider the impact of the sample design

– Sample data has sample error; small cell counts are likely to be unreliable

– Will the sampling design affect the standard errors?– Does weighting need to be applied to avoid bias

• Getting started guides, stats guides and topic guides are available on the ESDS site if you don’t know where to start

• Contact helpdesk if you have any problems or find errors in the data

• User groups and news lists can help keep you informed

The last word:

• ESDS Helpdesk– govsurveys@esds.ac.uk– (0161) 275 1980 (09:00-17:00)

• http://www.esds.ac.uk/government