sources of measurement errors in business surveys mojca bavdaz
TRANSCRIPT
Sources of Measurement Errors in Business Surveys
Mojca Bavdaz
Outline
• Measurement errors & their sources.
• Empirical study.
• Sources of measurement errors redefined.
• Implications.
Measurement errors
• ‘Theoretical’ definition of a measurement error:– ‘deviation’ of the observed survey value from the
true value (Groves, 1991);– ‘difference’ or ‘discrepancy’ between the observed
survey value and the true value (Hansen et al., 1951; Sukhatme and Sukhatme, 1970).
• ‘Practical’ definition of a measurement error:– observational gap between the ideal measurement
and the response obtained (Groves et al., 2004);– error which occurs at the time of data collection
(Biemer and Lyberg, 2003).
Sources of measurement errors (1)
• In household surveys (e.g. Groves, 1989):– respondent;– interviewer;– instrument (the survey questionnaire);– mode of data collection.
Sources of measurement errors (2)
• In business surveys, the addition of the fifth source:– records (Ponikowski, Meily, 1989);– information system
(Biemer, Fecso, 1995; Biemer, Lyberg, 2003); – organization (O'Brien, 2000).
Empirical study
• Business survey under study:– Quarterly Business Survey on Trade (SORS).
• Aim:– examination of actual response process.
• Research methods:– in-depth interview;– observation.
• Sample size (n=28).• Challenges of implementation.
Three-dimensional model of business survey response
Sources of measurement error redefined
Existing sources Identified sources
Respondent → + Business participants
Interviewers → + Survey staff
Survey questionnaire (Survey instrument) → Survey instrument
Mode of data collection → + Survey characteristics
Records/Information system/Organization → Organizational context
INTERACTIONS!
Implications
• Improved tools:
– Three-dimensional model for comprehensive & systematic analysis of response process in business surveys.
– Redefined sources of measurement errors for development of approaches to measurement error prevention & reduction.
Implications: Example 1
• Self-administered business survey.• No interviewer.• But really no survey staff involved?• One-time & rare contacts make huge
impact.• Training of (all) survey staff.
Implications: Example 2
• Recurring business survey.• No source of measurement error.• But really no impact of recurrence?• Learning curve effect & routine.• Support & verification of first-time
questionnaire completion (at business & respondent level).
Future research
• More research on specific aspects:– comprehension of economic concepts;– role of business participants & survey staff;– circulation of survey instrument;– impact of ‘soft’ issues;– ...
• Quantification through experiments.• Generalization to non-business
organizational surveys.
