Epistemology and Methods

Survey Research & Interview Techniques

May 26 2009

Q&A RD paper

• Research theme, motivation, research question

• Literature review (theoretical discussions / competing explanations) and your contribution (where does your project fit in)

• Model building – your argument (conceptualization, hypotheses to be tested, key variables, projected causal story)

• Model testing (how do you select, collect and analyze data), critical discussion on the methods used…

Survey research

• No other method for understanding politics is used more often

• Long tradition

• Used when the unit of analysis is the individual (e.g. election studies, Eurobarometer)

• Comparison across societies and trends over time

• Surveys have the great virtues to ask the questions you want to ask, when and where you want!...

• Drawback: costs

Survey research

• Survey research is a method of data collection in which information is obtained directly from individual persons who are selected so at to provide a basis for making inferences about some larger population

• Sample surveys analyzed with the help of statistical techniques

Survey research

Form

• Direct questioning (face-to-face or telephone interviews)

• Questionnaires Mail (regular or electronic) surveys; internet-based surveys:

Types of information:

• facts (personal data)

• perceptions (what respondents know (or think they know))

• opinions (preferences or judgments about events)

• attitudes (stable orientations)

• behavioral reports (did you go voting?)

Survey research

Stages of survey research preparation

• Conceptualization (objective, theory, specific set of questions)

• Survey design (explorative, descriptive, explanatory)

• Linking objective with data-collecting method

• Cross-sectional or longitudinal design (trend and cohort studies, panel studies)– Panel studies – watch out for: costly, keep track of sample, moving

towards a biased sample? (respondent changes due to interviews, drop-outs)

Survey research

Instrumentation (operationalization): define content, form, format, wording and order of questions

• Content: Limit the number of questions and time, keep number of hypotheses to be tested small (degree of data!)

• Form: Open-ended vs. closed-ended questions (choice of options influences responses)

• Format: Techniques how questions are presented (e.g. visual aids…)

Survey research

Wording and order of questions

• compare with prior research

• are respondents competent to answer?, use contingency questions to find out (to assess the level of knowledge)

• statements are more useful than questions (e.g. to measure intensity of opinion, respondents use the same frame of reference)

• but tendency of a response set (tendency to agree with statements), items should be mixed

Survey research

Generally 4 steps: explanation, warm-up questions, substantive questions, demographic questions

• Explanation: should not reveal study information that would bias responses (eliminating fear of study, importance to warrant time and attention)

• Warm-up: establish a good relationship with the respondents

• Substantive questions: check question ordering, pre-test will give some guidance (experiment with different orderings)

• Demographic items: at the end, personal information, prevent respondents’ being ill at ease…

• Do not crowd items

Survey research

• New survey methods involve “experiments embedded in surveys”: modify questions wordings to determine whether counter-arguments, subtle cues, rhetorical, emotional or cognitive factors can change opinion or behaviors (how stable are opinions and how do respondents threat information)

• Work with large n, control and treatment groups

• Quasi-experiments have made surveys even stronger methods for testing theories, they potentially score high on conceptual richness and policy relevance

• Examples: The Use of Competing Frames or Priming

Designing an experiment-survey on

Measuring

a) the attitude of citizen vis-à-vis free trade

• Questions / statements

• Stability of attitudes (framing, priming)

b) the attitude of citizen vis-à-vis international law

• Questions / statements

• Stability of attitudes (framing, priming)

Interview techniques

• “what you already know is as important as what you want to know”

• “what you know determines which questions you ask”

• “what you already know determines how you ask them”

• “between a journalistic style and ethnographic styles of interviewing”

Interview techniques

Measurement issues:

• Reliability

• Validity

“We have a purpose in requesting an interview but ignore the reality that subjects have a purpose in the interview too: they have something they want to say. They thought about what they want to say in the period between the request and the actual interview. They are talking about their work and, as such, justifying what they do. That’s no small matter”

Interview techniques

Types of interviews

• Structured Questions

• Semi-structure questions

• Open questions

Interview techniques

To get started:

• Sampling

• Getting in the Door

• Being flexible (re-scheduling, multiple calls)

• Contacts (letter, cell phone, snow-balling)

Interview techniques

Getting Rapport

• Putting respondents at ease

• On the record, off the record, not for attribution, on background

• Signal knowledge but do not “teach” the other

• Are you listening?

• Question order?

• Tricky questions (when did you stop beating your wife)

• Enough is enough

Interview techniques

Conducting the interview (Checklist)

Types of Questions:

Grand Tour questions

Example questions

Prompts (to keep people talking and rescue when responses turn to mush)

Specific questions

Questions for clarification

“Did we not discuss something you find important”

Interview techniques

After the interview:

• Immediate transcription

• Thanking the interview partner

• Sending your “paper” once its done

Interview techniques

• Ethical issues

• Differences across countries (interviewing national officials of EU countries)

• Differences across elites

• Differences across stakeholders (e.g. NGOs vs. business)

• “Learning by doing”