Chapter 9 Survey Research

SUMMARY

Survey research is the most frequently used mode of gathering data in the social sciences. It consistsofSelecting a sample of respondents and administering a questionnaire or an interview to them.Surveys may be used for descriptive, explanatory, and exploratory purposes. The unit of analysis isusually the individual, but surveys can be used for other units of analysis as well. Survey researchis particularly well suited for studying attitudes and orientations in a l~ge population. Surveys areincreasingly used for less legitimate purposes than social research, including marketing surveys andpolitical surveys such as "push polls."

Most concepts are measured thro~gh questions and statements. Questions may take one of two­forms. Closed-ended questions ~re easily process~d, but sometimes overlook alternative ariS"wercategories. Open-ended questions afford a wider range of responses, but they are more difficult toprocess and may yield irrelevant answers.

Questions should also be relevant to the respondent, very clear, and as short as possible. Respondentsshould be capable of answ~the questions and should be willing toanswer the questions. It is bestto avoid negative items as well as biased items and terms. Combining two (or more) questions into

one is known as a double-barreled question and produces both confusion for the responden~ndambiguity for the researcher. The researcher should also ~void using Questions that p...romotesocial~esirability-answers that make respondents look gQQQ.. In the final analysis, the overriding guidelineis that questions should reflect the purpose of the study.

Questionnaire format and appearance are critical. Questionnaires that are well organized, uncluttered,and attractive reduce the likelihood that respondents will overlook or ignore items or dispose of the

questionnaire. The best format for questions is the use of boxes adequately sRaced. C~ntingencyquestions can helpjJrevent respondents from being confused and annoyed-questions that are relevanttoonlv some of the respondents are identified. A series of questions with the same set o{~rcategories can be effectively presented through a matrix format, which saves space and time. Butoccasionally respondents begin marking the same answers without carefully reading the items, aproblem known as a response set.

The order of question presentation is also critical because ~swers to one question can affect answersto later questions. Also, it is usually best to begin questionnaires with the most interesting set of

questions to generate interest, although it is generally best to begin interviews with demographicquestions to enhance rapport. Clear instructions at the beginning of a questionnaire, as well as at thebeginning of each section, are mandatory. Special situations will require special instructions. Finally,it is important to pretest your Questionnaire to help reduce error. Although it is not essential that

pretest participants reflect a representative sample, they should use people for whom the question-naire is at least relevant. ~ ..

~lf-ad~ered questi()~airesa.re generally executed by mail, although sometimes they areadministered to a group of respondents, and sometimes they are delivered and picked up at a latertime. Self-mai~g questionnaires require no return envelops making it easier to return the instru-

168

w:;-<.Zr 9 Survey Research

~uestionnaires can be sent via first-class postage or bulk rate and returned via postage stamps~~ess-reply permits. When each option would be used is determined largely by the budget and~cted return rate.

=.oring the returns of questionnaires is an important part of the study and can be accomplished--:::..:.return rate graph plotting the number of questionnaires returned each day. Such a graph can

-:;:e clues about when follow-up mailings should occur, and it may be ~seful in estimating"="=Sponsebias. Follow-up mailings may contain only a letter of additional encouragement or may~-"?1n a new copy of the questionnaire as well. F_ollow-up mailings significantly increase return""'--:two follow-ups mailed two or three weeks apart are best.

- ~ are no strict standards for getermining ap acceptable response rate. Iiigh response rates=~e the probability that the respondents accurately represent the samE!e, thus reducing the

::z:.:e of response bias. But a demQllStrat~plack of response bias is more crucial than the re~I:>onse......;;itself. A response rate of at least 50 percent is adequate, qQ..Rercentis considered good, and 70~nt is considered very good.

~"~views provide a number of advantages over the self-administered questionnaire. They attain£;:.er response rates, decrease the number of "don't know" and "no answer" responses, h~ correct::-...J.i.!singite~, and provide the opportunity to observe the social situation as well as to ask

.':..~llOns.

:s critical that the interviewer serve as a neutral conduit through which questions and answers are=:1Smitted. Interviewers should have a pleasant demeanor and dress in a fashion similar to the:eople being interviewed. Familiarity with the instrument will lessen the time required and miniilliZe:rJolems with individual items. Interviewers should follow the question wording exactly and record--:"sponsesexactly. One particular skill required is.J2robing, whereby the interviewer redirects inap­

=8priate answers while maintaining the neutral role. When multiJ2leinterviewers are employed, the-:~archer should thoroughly train and supervise the interviewers. Researchers should also prepare~ifications, which are explanatory and clarifYing comments about handling difficult or confusing;:--:uationsthat may arise with a specific item.

-=- elephone surveys save money and time, sometimes enhance willingness to give socially disap­7="Ovedanswers, and yiEld greater control when several interviewers are em2!Qyed. They also

.:..:J~e personal safety concerns. But they come with several problems. They have a ~ad re]2!!tatiQ,n,:or example, and people can hang up easily before the interview is completed. Computer-assisted~lephone interviewing is frequently used to enhanc~ the reliability and validity of the results.:omputer-assisted survey methods are also increasingly being used for in-person interviews as well::.sfor analyzing data as the data are gathered. Recent innovations in computer technology provide~veral options for computer administered questionnaires. Online polling offers advantages overJther forms but comes with some disadvantages, particularly concerning representativeness.

Self-administered questionnaires and interviews each have advantages and disadvantages. Questio~:laires are generally cheaper and quicker and are more appropriate in dealing with more sensitiveissues. Interviews help reduce incomplete answers, are more effective in dealing with complicated

169

Chapter 9 Survey Research

issues, enable a researcher to conduct a survey based only on a sample of addresses, and provide theopportunity to make important observations apart from the responses to the items. Interviews

ge~ yield higher response rates. -

The survey design itself has strengths and weaknesses compared to other designs. It is particularlyaImffipriate for describing the characteristics of large 1lQI2ulations,thereby making large samplesfeasible. In some respects, surveys are quite flexible, and the use of standardized questions enhances~eliability. But these advantages come at some cost. Standardization often results in overlookingo~her appropriate responses and may generate inflexibility in modifying questions. Surveys seldomanalyze the context of social life. They are frequently labeled as artificial because the topic of studymay not be amenable to measurement through questionnaIres, and the act of studying that topic mayaffect the results. In short, survey research is weak on validity and strong on reliability.

Researchers with a need for survey data cannot always afford to conduct large-scale surveys them­

selves. Hence many researchers employ secondary analysis of data gathered for some other purposeby someone else. Such research has experienced increased popularity due to the establishment ofdata archives, research centers that collect and distribute many different types of data sets. Suchanalysis can be done quite rapidly and inexpensively, particularly with those that are online. The key~dvantages of secondary analysis include the low cost and the ability to use data gathered by well

known researchers and data collection agencies. The k~y problem with secondary analysis is validity;when one researcher collects data for a given purpose, other researchers have no assurance that suchda!a will be appmpriate for their research interests. Comparability of measures is a particularproblem. Regarding ethics, surveys often ask for private information (which should remain confiden­tial), and asking questions may cause psychological discomfort (which should be minimized).

TERMS

1. bias (277) 12. meta-analysis (306)2. closed-ended question (272) 13. open-ended question (272)3. Computer Assisted 14. probe (293)

Personal Interviewing (CAPI) (299) 15. questionnaire (272)4. Computer Assisted 16. respondent (270)

Self Interviewing (CASI) (299) 17. response rate (288)5. Computer Assisted Self- 18. response set (281)

Administered Questionnaire (CSAQ) (299) 19. return rate graph (287)6. Computer Assisted 20. secondary analysis (304)

Telephone Interviewing (CATI) (297) 21. self-administered questionnaire (286)7. contingency question (279) 22. social desirability (277)8. data archives (304) 23. specifications (294)9. double-barreled question (273) 24. telephone surveys (295)

10. interview (291) 25. Touchtone Data Entry (TDE) (299)11. matrix questions (280) 25. Voice Recognition (VR) (299)

170