Survey MethodologySurvey Instruments (1)

EPID 626

Lecture 7

What is a survey instrument?

• For interviews: A script for interviewers, including introductions, instructions, and questions

• For self-administered surveys: A questionnaire

Essential components for developing a survey instrument

• Statement of purposes– What do you want to accomplish with the

survey?

• List of the variables to be measured– Group into logical categories

• Draft analysis plan– What are your dependent, independent,

check variables?

Preliminary question design steps

• Focus groups

• Draft questions

• Cognitive laboratory interviews

Focus groups

• Objective: To compare the reality about which respondents will be answering questions with the abstract concepts embedded in the study objectives (Fowler, 1995)

• Usually 6-8 people per group; have a couple of groups

Focus groups(2)

• Try to get participants who are similar to your survey population

• Talk about perceptions, experiences, and feelings related to survey subject

Drafting questions

• Look for standard questions from other surveys– Benefit from their experience– Enhance comparability of your survey with

previous surveys

• Write good questions (see next lecture)

Cognitive laboratory interviews

• Respondents are brought into a laboratory setting– May be videotaped– Interviews conducted by cognitive

psychologist or experienced investigator

• Goal: To get information about how the respondent understood the questions and about the way they answered them

Layout of survey instrument

• Make it clear and simple

• Usually begin with straightforward questions, put complicated or sensitive questions in the middle or at the end

Interviews

• Should have everything scripted, including introductions, instructions, transitions, definitions, and explanations

• Be very explicit about skip instructions

• Differentiate instructions and optional wording from questions– Usually done with all caps and

parentheses, respectively

Self-administered

• Questionnaire should be self-explanatory (minimal instructions needed)

• Limit to closed questions

• Use short questions with consistent formats

• Minimize skips; make them very clear

Field pretesting

• Goal: Find out how well the data collection protocols and survey instruments work under realistic conditions

Interviews

• Usually interview 20-50 respondents who are similar to your survey population

• Interviewers note ways to improve survey instruments

Quantitative methods for pretesting

• Ask interviewers to fill out a rating for each question– Is it easy to read as worded?– Do respondents understand the question in

a consistent way?– Can respondents answer the question

accurately?

• Taping and behavior coding– Trained coders review the recordings and

evaluate the question and answer process in a consistent way

• Do the interviewers read the question as worded?

• Do the respondents ask for clarification?• Do the respondents initially give an inadequate

answer that requires interviewer probing?

Quantitative methods for pretesting(2)

Self-administered

• Have respondents (who are similar to your survey population) fill out the survey and then discuss– Were the instructions clear?– Were the questions clear?– Were there any problems understanding

what kind of answers were expected?