JAMM 444: Public Opinion

Survey methodology

Comparing survey methods

Planning your surveys

Tuesday, Sept. 19

Schedule update Survey methods Sources of error Meet in small groups: team project

Schedule update

TODAY: Chapter 3, Survey methods Thursday: Chapter 9, Using public

opinion data Sept. 26: Barbara Foltz & Stephanie

Kane, UI Social Science Research Unit Sept. 28: Valerie Steffen, president,

Strategic Intelligence (Boise)

Survey methodology

Why do a survey? Measure opinion of a population too large

to contact individually Save time, save money

“Surveys can be used in a scientific way to realize the great benefits of interviewing a representative sample instead of the whole population.” Salant & Dillman, 1994

Surveys work, if...

Sample is large enough to yield desired precision.

Everyone in population has equal chance of being selected.

Questions enable respondents to give accurate answers.

Sampled respondents have similar characteristics to non-respondents.

5 key elements of a survey

1. Population

2. Type of sample

3. Method(s) for gathering data

4. Questionnaire

5. Analyses and inferences

5 steps for a successful survey

1. Be specific about what new information you need and why.

2. Understand, avoid 4 kinds of error.

3. Choose the best survey method.

4. Write good questions.

5. Test questionnaire before beginning real survey.

Choosing a sample

Identify population of interest: University of Idaho students Moscow residents Idaho voters ‘The American public’

Choosing a sample

Non-probability sampling 1936, Literary Digest

Probability sampling Simple random sampling (rarely used) Systematic random samples (every 30th

name in student directory) Stratified sample: divided into groups (strata) Cluster sampling (spread over a wide area)

Cluster sampling

Used for many national surveys: National Election Survey (NES) General Social Survey (GSS) Gallup Organization

350 geographical segments Sample to reflect U.S. as a whole

(geographic area, size of community) Random telephone numbers in each area

How big is a typical sample?

UI or Moscow: 200 to 500 Idaho: 500 to 800 National: 800 to 1,200

Four sources of error

1. Coverage error: Not allowing every person in population equal chance of being sampled.

2. Sampling error: Only some members of study population are asked to respond. (Range of possible results)

Four sources of error, cont.

3. Measurement error: inaccurate answers due to improper question wording

4. Non-response: Some people in sample don’t respond; different from respondents

Sampling error

The probable difference in results between interviewing everyone in the population vs. a scientific sample taken from the population.

Expressed as “plus or minus X percentage points”

http://www.ncpp.org/qajsa.htm#11

Sampling error

See explanation, note, p. 79, PO Rules of thumb:

The closer the margin, the larger the sample that is needed to reduce error.

For very large populations, a relatively small sample can produce reasonably accurate results. (See overhead)

How large a sample do you need?

The answer depends on: How much sampling error you can

tolerate The size of your population, if the

population is small How varied the population is The smallest sub-group that you are

likely to analyze

Survey methods

1. Face to face 2. Telephone 3. Mail 4. Internet What are advantages & disadvantages

of each method? Consider: cost, time, response rate, amount of information

For Thursday

Read: PO9: Public Opinion & Policy Making

Small Groups

Team 1: UI issue or controversy Team 2: Moscow or Latah County issue or

controversy Team 3: Political knowledge and/or civic

engagement by UI students Team 4: Idaho state candidates or U.S.

House of Representatives races Team 5: Idaho state ballot issues

Small Groups

1. Meet in groups for at least 15 minutes

2. Refine your topic

3. Brainstorm target population, survey methodology, timetable

4. Choose time and place to meet again before Sept. 26

Small Groups

ONE-PAGE MEMO due Sept. 26: Choose a name for your group (e.g.,

Bird’s Eye Consulting) List names of all participants. Rationale for survey (why?) & target

population (who?) Topics or candidates (what? who?) Methods (how?)