Sampling designs using the National Pupil Database

Some issues for discussionby

Harvey Goldstein (University of Bristol)&

Tony Fielding (University of Birmingham)

Size of data set

• The data set already contains some 3000k longitudinal records and increases by 600k a year.

• To carry out reasonably complex analyses, e.g. value added multilevel models, is already time consuming.

• Worth investigating the efficiency of sampling the database – either as a whole or for specific subpopulations such as LEAs.

• Traditional sampling theory can be used for simple statistics such as means or regression coefficients, and there is a literature for ‘power calculations’ for multilevel models (see ESRC research project by Browne at Nottingham)

Special features of the NPD

• The ‘population’ characteristics are known and can be used for drawing efficient samples.

• The possibility of an adaptive design exists, e.g.:– Select a random subsample to determine relationships of

interest (equivalent of a pilot study)– Fit a suitable model to estimate parameter values– Choose parameters of interest together with their confidence

intervals – Increase sample size to establish relationship between CI and

sample size and extrapolate to sample size needed to achieve required interval size.

– Any statistic of interest (in additon to CI) can be chosen.

Complex designs and replication

• For multilevel models and designs where interest focuses on special groups (e.g. low achievers) we need good choices of numbers of higher level units (schools) and numbers in the groups.

• A similar adaptive approach can be used, evaluating CIs or significance levels as design parameters are altered.

• We also have the opportunity of replicating an analysis by selecting an independent sample from the database.

Using all the data

• When analysing a given sample we will also generally have available data related to the sample members, e.g.:– School level averages for each pupil in a study– School level data for previous schools attended– School level data for previous years– LEA data for previous years– School data for neighbouring schools,

• All such data can be incorporated into a model, increasing the number of variables but not the sample size.

Other possibilities

• Poststratification: using population distributions to re-weight statistics or to incorporate weights in model estimation.

• Setting up an archive of results that may be useful for designing samples

• Using PLASC to select a research sample – subject to appropriate permissions.