(Source: Causeweb.org)

Elementary Survey Sampling7th Edition

By Scheaffer, Mendenhall, Ott and Gerow

Be careful what you believe

• “4 out of 5 doctors recommend…”• “22% of Americans doubt that the Holocaust ever

occurred…”(Does it seem possible or does is seem impossible to you that the Nazi extermination of the Jews never happened?)

• “Alcohol is more harmful than heroin” (http://www.significancemagazine.org/details/webexclusive/874549/Alcohol-is--more-harmful-than-heroin_-Really--Sample-before-you-speak.html)

• HOWEVER, there are good sources of information (that attempt to sample appropriately!!), Census Bureau, Bureau of Labor Statistics, Gallup Poll

Chapter 2: Elements of the Sampling Problem

Technical terms• Element – an element is an object on which a

measurement is taken• Population – a population is a collection of

elements about which we wish to make an inference

• Sampling units – sampling units are nonoverlapping collections of elements from the population that cover the entire population

Definitions continued

• Frame – a frame is a list of sampling units• Sample – a sample is a collection of sampling

units drawn from a single frame or from multiple frames

2.3 How to select the sample

• Error of estimation

• Bound is usually selected as 2s(theta-hat) • Probability sampling versus Quota Sampling– SRS– Stratified random sample– Systematic sample

B |ˆ|

2.4: Sources of Errors in Surveys

• Nonobservation (sampled elements make up only part of the target population; examples include sampling, coverage, nonresponse)– Sampling error – variation in sample response because not

from population (can be estimated for probability samples)..can be reduced by survey design and sample size

– Coverage…frames do not include everyone in target population (unlisted numbers in telephone directories)

– Nonresponse (inability to contact sampled elements, inability of person to answer questions correctly, refusal to answer)

Errors of Observation

• Observation (recorded data deviate from the truth; examples include interviewer or data collector, instrument, sampling method)– Interviewer can bias how questions are asked– Respondent errors (recall bias, prestige bias, not

understanding question)– Measurement error (measurements should be

clearly defined…for example, education)

Reducing Errors in Surveys

• Callbacks• Rewards and incentives• Trained interviewers• Data checks• Questionnaire construction

2.5 Designing a Questionnaire

• Question ordering – the ordering of questions can change perspectives Example: A. Do you think the US should let Communist newspaper reporters from other countries come in here and send back to their papers the news as they see it?

B. Do you think a Communist country like Russia should let American newspaper reporters come in and send back to America the news as they see it?

Ordering (A,B) 54.7% answered yes to A, ordering (B,A) 74.6% answered yes to A

Open versus Closed, Response Options and Wording of Questions

• Open is good for qualitative information (and to help design options for closed questions). Closed is good for quantitative information (however, options should be appropriately chosen).

• Response Options – Should there be a “Don’t know” or “Not enough information to say” option

• Wording of questions…should try to ensure it is not biasedFor example: Do you favor the use of capital punishment?Do you favor or oppose the use of capital punishment?

2.6 Planning a Survey1. Statement of Objectives2. Target population3. The frame4. Sample design5. Method of measurement (usually personal interviews, telephone

interviews, mailed questionnaire, or direct observation)6. Measurement instrument (if questionnaire, carefully create questions)7. Train fieldworkers (if necessary)8. The pretest9. Organization of fieldwork (if necessary)10. Organization of data management11. Data analysis