assuring good field work juan muñoz. what happens when fieldwork is poor? a long and frustrating...
TRANSCRIPT
Assuring good field workJuan Muñoz
What happens when fieldwork is poor?
• A long and frustrating process of “data cleaning” becomes unavoidable
The data loose their policy-making relevance
• Data quality is not guaranteed
The process converges (at best) to databases that are internally consistent
• The process entails a myriad of decisions, generally undocumented
Users mistrust the data
Key factors
• Manage the survey as an integrated project
• Implement the team concept in the organization of field operations
• Integrate computer-based quality controls to field operations
• Establish strong supervision procedures• Ensure sufficient training• Work with a reduced staff over an
extended period of data collection
Management levels
• Core staff– Survey manager– Field operations manager– Data manager
• Tactical options for the organization of field teams– Mobile teams with fixed data entry– Mobile teams with integrated data entry– Sometime in the future: the paperless
interview
Mobile teams with fixed data entry
• Cote d’Ivoire (1984)• Peru (1985)• Ghana• Pakistan• Guinea-Conakry• Mozambique• Iraq (2006)
Composition of a field team
Supervisor Interviewers Anthropo-
metrist
Data entry
operator
The team and its tools
Supervisor Interviewers Data entryoperator
Anthropo-
metrist
Two PSUs visited in a four-week period
Alama Bamako
Regional Office
First weekAlama Bamako
Regional Office
Operator remains in Regional
Office
Rest of the team
travels to Alama
They complete
first half of questionnair
es in all selected
households
Second weekAlama Bamako
Regional Office
Operator enters first
week data from Alama
Rest of the team
travels to Bamako
They complete
first half of questionnair
es in all selected
households
Second weekAlama Bamako
Regional Office
Supervisor gives
Bamako questionnaires to DEO. DEO gives
back Alama questionnair
es with flagged
inconsistencies
Rest of the team
travels to Bamako and back
Third weekAlama Bamako
Regional Office
Operator enters first week data from Bamako
Team completes second half of
questionnaires. They correct
inconsistencies from first half
Fourth weekAlama Bamako
Regional Office
Operator enters second week data
from Alama. Corrects
inconsistencies from first round
Team completes second half of
questionnaires. They correct
inconsistencies from first half
Fourth week
Regional Office
The result is a clean data set on
diskette, ready for analysis
immediately after data collection
Mobile teams with integrated data entry
• Nepal (1992 and 2001)• Argentina• Paraguay• Bangladesh (2000)
Mobile teams with integrated data entry
Regional Office
Alama
Bamako
Cocody
Team works with portable
computers and printers
Mobile teams with integrated data entry
Regional Office
Alama
Bamako
Cocody
Operator travels with the rest of the field team
Mobile teams with integrated data entry
Regional Office
Alama
Bamako
Cocody
Data entry and validation almost
immediate
Mobile teams with integrated data entry
Regional Office
Alama
Bamako
Cocody
Reduced trips to and from
Regional Office to selected PSUs
Mobile teams with integrated data entry
Regional Office
Alama
Bamako
Cocody
Benefits of integration
• Provides reliable and timely databases• Provides immediate feedback on the
performance of the field staff, allowing early detection of inadequate behaviors
• Ensures that all field staff applies uniform criteria throughout the full period of data collection
• Solves inconsistencies through direct verification of households reality, rather that through office guessing
• Is consistent with the total quality culture
Selecting and training field staff
• Why is it important• How long does it take• How is it organized
Example: Day 2 of interviewer training
• Definition of household (and dwelling, family, etc.)
• Pictorial of a sample household• Slide with an empty roster (explain case
conventions, encodings, skip patterns, etc.)• Fill the roster for the sample household (need for
legible handwriting, recording of ages, use of a calendar of events, etc.)
• Role playing (trainer as a respondent, simulating borderline cases)
• Role playing (trainees interview each other)
The role of the team supervisor
• Manager/administrator ("traditional" role)– Monitor completion of work– Collecting and accounting for all of the
questionnaires– Paying interviewers/managing the fuel budget– Administrative functions– Sometime interviewer
• Quality control– Continuous training of interviewers– Random quality checks in the field
Supervision tasks
• Verification of questionnaires for completeness– Completion of household roster & ID of
members– Completion of all sections for all individuals– Limited internal consistency
• Random re-interviews of households• Observation of interviews• Observation of anthropometrics• Supervision of data entry
The paperless interview (CAPI)• The option of the future• Is used successfully by some statistical agencies for
simple surveys (LFS and CPI price collection)• Recent experiments have shown that
– Technology is already available(Lightweight notebooks and software development platform –both Windows based)
– Can be cost-effective– No negative serious externalities
• We still need to solve:– Questionnaire design– Ergonomic aspects of the interview– Interviewer training– Development of supervision procedures adapted to the
new technology (voice recording, use of GPS’s, etc.)
• Each cluster (6 households) visited by one interviewer in a 20-day period (a wave)
• Each household records food expenses in a diary for 10 days• The interviewer visits each household seven times, before
during and after the 10-day diary recording period• During these visits, the interviewer
– Helps with diary recording– Asks different questionnaire modules (education, heath, labor, etc.)– Checks for inconsistencies in the data collected in previous visits
Case study: The IHSESIraq Household Socio-Economic Survey
Presenter: Shwan J Fatah – Sulaimania, KRG Stat Office
• This was possible by organizing the field workers into teams, composed of– One supervisor– Three interviewers– One data entry operator
• Data was entered and checked in between interviewer visits
• Fieldwork concluded in January 2008• A database is already available• Preliminary outputs expected in March 2008
Case study: The IHSESIraq Household Socio-Economic Survey
(continued)