Challenges in measurement in longitudinal studies

Emla FitzsimonsInstitute of Education and

Institute for Fiscal Studies, London

Florence, October 2014

Motivation

How to collect good quality data ‘quickly,

accurately and at low cost’?

Key is what variables to collect, what to measure

Highlights importance of measurement agendafor longitudinal studies

Main message

Faced with logistical, monetary, time, expertiseconstraints

there is an urgent need for short batteries ofquestions

- easy and relatively cheap to administer

- validated against the ‘gold standard’

- easy to standardize/compare across contexts

An early childhood RCT in Colombia

Large scale RCT of a parenting programme in Colombia- Weekly home visits- Targeted at children 12-24 months old- 18 months duration

Attanasio et al, BMJ 2014

Aims1. Evaluate effects on children’s development

Measure children’s development

2. Understand mechanisms behind these effectsMeasure home-based stimulation practices

Aim 1: Measuring children’s development

Tools used:

- ‘Gold standard’ measures – Bayley Scales ofInfant Development (BSID) – at baseline andfollow-ups

- Some maternal-reports/interviewer-administered tests

Aim 1: Measuring children’s development

Logistics/costs

- Test centres in 96 villages

- Trained psychologists to administer/observe children for ~1.5 hours- ‘complex manipulation of a complicated flow of

testing materials and instructions, rules for which items to administer based on the child’s age and performance, and judgment calls about whether credit should be given or not’

- Cost $260 per child for baseline & follow-up

(vs ~$300 per child per year to run the programme)

Aim 1: Measuring children’s development

BSID did not correlate highly with some shorter maternal-reports/interviewer administered tests

Urgent need for methodological work to find short batteries, easy/quick to implement, that correlate highly with ‘gold standard’ measures

Aim 2: Measuring family care practices

The HOME is a good measure of the home environment (Caldwell and Bradley 1984; 2003)

But not suitable for large-scale population surveys:

• TIME: 45–60 minutes to administer • TRAINING: Requires skilled, well-trained

interviewers and considerable adaptation for use in developing countries

• STANDARDIZATION: Involves observations => more difficult to standardize

Aim 2: Measuring family care practices

Clear need for indicators that are simple, easy to administer, and applicable across different cultures for use in large population surveys

• UNICEF developed the family care indicators (FCIs) questionnaire to measure the home environment of young children in large population surveys

• Items adapted from several sources, including the HOME

• Implemented across different contexts

Aim 2: Measuring family care practices

Evidence across a range of studies that:

- FCI easy to administer

- mothers readily understand it

- acceptable test-retest reliability

- several of its subscales predictive of child development

=> strong platform for developing survey-based indicators of the family environment that have relevance for child development in developing countries (Hamadani et al, 2010, J Health Popul Nutr.)

Another example UK Millennium Cohort Study

Overweight/obesity a major issue for this cohort

Challenge - how to capture diet in one simple, easy-to-administer,quick measure?

Eating choices index (ECI): 4 item index to measure healthiness of diet(i) consumption of breakfast(ii) consumption of 2 portions of fruit per day(iii) type of milk consumed(iv) type of bread consumed

- each providing a score from 1 to 5

Another example

In analysis of very detailed 5-day ‘gold standard’food diaries, ECI scores correlated very well withnutrient profiles consistent with a healthy diet(Pot et al, Public Health Nutrition 2014)

- evidence that it provides a simple, quick, cost-effective method to rank diet healthiness inlarge observational studies

- ongoing validation of this index

Conclusion

There is a real need for short batteries ofquestions, across many different domains, toincrease utility and impact of longitudinalstudies

- easy and relatively cheap to administer

- validated against the ‘gold standard’

- easy to standardize/ compare across contexts