Slide 1

Survey Research Slides Prepared by Alison L. OMalley Passer Chapter 7

Slide 2

Surveys: An Introduction Surveys rely on interviews and questionnaires to gather info about people What is the last survey you took? Take Our Survey!

Slide 3

Populations and Samples Operationally defined by sampling frame Population Sampl e Whats the difference between representative and nonrepresentative samples? Representative or biased?

Slide 4

Surveys Can we trust the data? Participants responses may be distorted by social desirability bias, or perhaps theyre misremembering Response Truth ? Fret not. High quality survey research is doable!

Slide 5

Selecting the Sample Probability sampling Every member of the population has chance of being sampled Probability of selection can be specified Nonprobability sampling Probability sampling conditions do not apply

Slide 6

Probability Sampling Simple random sampling Build a sampling frame containing all population members Stratified random sampling Sampling frame divided into groups (based on demographic characteristics) Random sampling applied to each group

Slide 7

Probability Sampling Cluster sampling Units (e.g., schools) containing population members are identified Essentially, this step creates the sampling frame These clusters are then randomly sampled May not represent the entire population What if theres no sampling frame?

Slide 8

Nonprobability Sampling Convenience sampling Grab whomever you can Likely to generate a nonrepresentative sample Quota sampling Sample designed to mirror population characteristics (e.g., % of females) Uses convenience sampling to create sample within each quota group (e.g., males and females)

Slide 9

Nonprobability Sampling Self-selected samples Participants elect to participate (as opposed to being sought out by researcher) A form of convenience sampling Likely to generate a large sample size, but keep in mind that representativeness matters more than sample size!

Slide 10

Nonprobability Sampling Purposive sampling Sample created in line with study goals (e.g., focus only on students in Top 10 graduate programs in research on the work habits of successful graduate students) Two common strategies Expert sampling Snowball sampling participants recruit others to participate

Slide 11

Sampling: Check Your Understanding What is the difference between quota sampling and stratified random sampling?

Slide 12

Margin of Sampling Error Sample results are estimates of the true population value Sampling variability captures how sample characteristics fluctuate If you roll two dice 100 times and encounter this 5 and 2 pattern 13 times, its extremely unlikely youll see this pattern 13 more times in the next 100 rolls

Slide 13

Margin of Sampling Error Thus, sampling error acknowledges that our population estimates vary depending on the sample Survey data are then accompanied by a margin of sampling error, a range of values within which the true population value falls Keeping in mind that we can never be 100% certain in our results, we also report confidence levels (typically 95%)

Slide 14

Sample Reporting For results based on the total sample of national adults, one can say with 95% confidence that the margin of sampling error is 2 percentage points. Gallup Economy Poll - May 28, 2013 http://www.gallup.com/poll/162797/economic-confidence-holds-steady-high- level.aspx

Slide 15

Taking the following data into account, why dont researchers aim for more precise estimates?

Slide 16

Constructing Questionnaires Write a survey item asking participants about their exercise habits Now, ask a classmate for feedback on your item Developing a questionnaire takes a significant amount of time and effort, and typically several versions are piloted before the final version is rolled out.

Slide 17

Types of Questions Closed-ended questions provide specific response options, whereas open-ended questions do not, allowing participants to answer in whatever form they choose Was your item about exercise behavior open- or closed-ended? Open-ended questions generally are more difficult to work with. Why? Open-Ended and Closed-Ended

Slide 18

Multiple choice Ranking scales Closed-Ended Question Types

Slide 19

Forced choice Rating scales Closed-Ended Question Types Describe the signature features of Likert response formats.

Slide 20

Question Wording Leading questions Loaded questions Double-barreled questions Double negatives Common Pitfalls to Avoid Are you or are you not in favor of terrorists in our country corrupting our young people and threatening our core values? Help! Identify whats wrong with this question and rewrite it.

Slide 21

Putting the Survey Together Group related questions together Place open-ended questions before closed-ended questions Move from more general to more specific questions Place personally sensitive questions at or near the end General Rules of Thumb

Slide 22

Administering Surveys Face-to-face (in-person) interviews Achieve higher response rates Facilitate establishment of rapport Enable standardized approach Interviewer can clarify any participant confusion But, theyre pricey! Supplements to ongoing behavioral observations

Slide 23

Administering Surveys What can interviewers do to aid each of the following? 1.Limitations in participant memory 2.Response distortion due to interviewer bias or other interviewer effects

Slide 24

Other Ways to Administer Surveys Telephone Mail Online Discuss the strengths and limitations of each mode of data collection.

Slide 25

Additional Survey Considerations Nonresponse bias occurs when participants who declined to participate would have responded differently than participants did Introduces more error into population estimates Although lower response rates do not appear to drive nonresponse bias, declining participation rates are of concern Many researchers offer incentives to encourage participation

Slide 26

What survey design features enable you to have greater confidence in the results? Be on the lookout for bogus surveys! Be a Smart Survey Consumer Think Critically