1

Chapter 5Survey research

Learning Objectives• Identify the information required to be

collected through a survey• Describe and justify the appropriate

respondents for a survey• Summarise the various communication

techniques, including their strengths and limitations

• Explain the factors that will influence the selection of a survey method

• Choose an appropriate survey method for a given situation

• Outline the issues involved when planning surveys internationally

Survey Design

Survey Design

• Survey: the overall planning, design and administration process involved in obtaining data using a questionnaire

• Respondents are asked questions regarding their behaviour, intentions, attitudes, awareness, motivations, and demographics and lifestyle

What is the difference between a survey and a questionnaire???

5

Nature of surveys

• Survey: a method of collecting primary data in which information is gathered by communicating with a representative sample of people.

• Sample survey: more formal term for a survey.

• Respondent: person who verbally answers an interviewer’s questions or provides answers to written questions.

6

Survey objectives: type of information gathered

• Attempt to describe what is happening or to learn the reasons for a particular marketing activity.– Identifying characteristics of target markets– Measuring consumer attitudes– Describing consumer purchasing patterns.

• Most often associated with quantitative findings, but can also be qualitative.

7

Advantages of surveys

• Quick• Inexpensive• Efficient• Accurate• Flexible.

Advantages of Surveys

• Standardisation• Ease of administration• Ability to tap the ‘unseen’• Large sample• Low cost• Increases geographic flexibility• Suitable for tabulations and statistical

analysis• Generalisability• Sensitivity to subgroup differences

Group Statistics

236 1283.4237 3502.02542 227.96244

212 1002.7311 1342.04673 92.17215

GenderMale

Female

How much have youspent, in total, onInternet shopping overthe past 12 months?

N Mean Std. DeviationStd. Error

Mean

Disadvantages of Surveys• Difficult developing surveys• Need to develop constructs, scale

measurements, survey design• Respondents may be unable/unwilling to

provide the desired information• Structured and fixed responses• Difficult to probe

Determine the Information Required

• Begin with clear research objectives, information needed, hypotheses and scope of the research

• Use exploratory research to gain insight and clarification

• Experience with similar studies• Good questionnaire design

From Whom• Target population will influence several aspects

of survey design including:– survey communication method– question content– administration issues

• Need to consider:– Who they are?– Where they are located geographically?– Accessibility?

How do you determine how many people are interviewed?

William Burlace, Director, Media Services

Roy Morgan Research

Methods of Data Collection• Personal interviewing• Telephone interviewing• Mail surveys• On-line surveys• Fax surveys• Combinations of survey methods

} self-completion

14

Personal interviews

• Form of direct communication in which an interviewer asks respondents questions face–to–face.

• Versatile and flexible.

15

The advantages of personal interviews

• Opportunity for feedback• Probing complex answers

– Interviewer asks the respondent for clarification of answers.

• Length of interview controlled• Completeness of questionnaire

– Item non–response is least likely to occur.• Props and visual aids• High participation.

16

Disadvantages of personal interviews

• Interviewer influence– Different interviewer characteristics and

techniques• Lack of anonymity of respondent

– Respondent may be reluctant to provide confidential information

• Cost.

Personal interviews: Door-to-Door Interviews

• Interview conducted at the respondent’s home

Advantages• When personal contact is essential • Conducive environment to questioning

process• Ability to explain complicated tasks• In-depth and detailed interviews can take

place

Personal interviews: Executive Interviews

• Interviewing business people at their offices

Disadvantages• Very expensive• Experienced interviewers required

Personal interviews: Shopping Centre Intercept

Advantages• Travel costs are eliminated• Interviewer can interact with respondents

Disadvantages• Non-representative sample• Uncomfortable environment for respondents

Personal interviews: Purchase Intercept technique

• Intercept customers while in the shopping environment

Advantages• Aids buyer recall• Interviewer can interact with respondents

Disadvantages• Access difficulties• Only purchasers are interviewed

Personal interviews: Computer Assisted

• Computer-assisted personal interviewing (CAPI)

• Direct entry of research information into a computerised database

22

Door–to–door interviews and shopping mall intercepts

• Door–to–door interviews are conducted at respondents’ doorsteps.– An effort to increase the participation rate.– Call–backs are attempts to recontact

individuals selected for a sample who were not available initially.

• Mall intercept interviews are conduced in a shopping mall.– Lower costs but higher refusal rates.

23

Telephone interviews • Personal interview conducted by telephone.

• Speedy data collection

• Relatively inexpensive

• Absence of face–to–face contact is more impersonal.

• Cooperation

• Representative samples

• Lack of visual medium.

24

Telephone interviews

• Central location: 1300 or 1800 lines to a central location

• Computer–assisted: answers to telephone interviews entered directly into a computer.

• Computerised voice–activated: done without human interviewers.

• Random digit dialling: use of table of random numbers to contact respondents.

Telephone InterviewingStrengths • Relatively inexpensive• Yields a very high sample• Quick

Weaknesses • Inability of respondents to see questions• Inability to observe respondents• Limitations on information quality and

quantity• Sampling problems• People associate surveys with

telemarketing

Telephone Interviewing cont.Traditional telephone interviews• Paper and pencil.

Disadvantages• Difficult to measure quality control• High potential for errors• Difficult to determine time spent on

surveys

Telephone Interviewing: Computer Assisted

Computer-assisted telephone interviewing (CATI)

• Computerised survey administered to respondents over the telephone

Telephone Interviewing: Computer Assisted cont.

Advantages• Computer dials phone number• Computer skips questions• Can customise questions• No editing required• Analysis can be done at any stage• Cost savings• Quality control• Time savings

Minimises

research errors

Telephone Interviewing: Computer Automated

Computer automated telephone systems (CATS)

• Computer-synthesised voices are used to ask questions over the phone

• Respondents select numbers on the telephone keypad to answer questions

• Voice recognition is likely to be used in the future to record and count responses

Self Administered Surveys: Direct Mail

• Survey is developed and mailed to pre-selected respondents who return the completed surveys by mail

• Mail interview package consists of the outgoing envelope, cover letter, survey, return envelope

Self Administered Surveys: Direct Mail cont.

Advantages• No interviewers to recruit, train, monitor

and compensate• Inexpensive to implement• Can reach many people

Disadvantages• Low response rate• Self-selection bias• Slow form of collection• Possible misunderstanding of skipped

questions

Methods Used to Increase Response Rates

• Preliminary notification• Foot-in-the-door• Personalisation• Anonymity• Response deadline• Appeals• Sponsorship• Incentives• Survey length• survey size, reproduction, and colour• Type of postage (outgoing)• Type of postage (return envelopes)• Follow-ups

Self Administered Surveys: Mail Panels

• Large representative sample of households that have agreed to participate in periodic mail surveys, product tests and telephone survey

Self Administered Surveys: Mail Panels cont.

Advantages• Panel can be tested prior to the survey to

obtain a representative sample• Produces a higher response compared with

direct mail• Allows for longitudinal research

Disadvantage• May not be a representative sample

Example: Roy Morgan Single Source

• Roy Morgan research company surveys over 50, 000 Australians each year on a range of topics which forms their Single Source Data – Lifestyle and attitudes– Media consumption habits – Brand and product usage– Purchase intentions– Service provider preferences– Financial information– Recreation and leisure activities

Source: www.roymorgan.com

Explain the methodological rigor behind the single source

monitor?

William Burlace, Director, Media Services

Roy Morgan Research

37

Response rate

• The number of questionnaires returned or completed divided by the number of eligible people who were asked to participate in the survey.

38

Increasing response rates for mail surveys

• Stamped return envelope

• Attractive questionnaires

• Cover letter

• Monetary incentives or premiums

• Interesting questions

• Follow–ups

• Advance notification

• Survey sponsorship — auspices bias.

39

Self–administered questionnaires that use other forms of distribution

• In–flight and in–store• Drop–off method• Email surveys• Internet surveys

– Speed and cost–effectiveness– Visual appeal and interactivity– Respondent participation and cooperation– Representative samples– Accurate real–time data capture– Personalised and flexible questioning.

Self Administered Surveys: Drop-off Survey

• A representative of the researcher hand-delivers survey forms to respondents; the completed surveys are returned by mail or picked up by the representative

Self Administered Surveys: Drop-off Surveys cont.

Advantages• Availability of a person who can answer

general questions• Screen potential respondents• Spur respondents in completing the survey

Disadvantages• Fairly expensive in comparison to direct mail

surveys

Computer Assisted Surveys: FaxAdvantages• Flexibility of mail• Speed of the telephone• Administration and clerical functions can also

be reduced• Fax implies urgency

Disadvantages• Fax technology may not be used• Cost to respondents could reduce response

rates• Technical problems• Lack of privacy may cause response problems• May lack clarity of image of a printed mail

survey

Computer Assisted Surveys: E-Mail

• A survey using plain text which is e-mailed for the respondent to read, complete and return

Advantages• Fast • Inexpensive

Computer Assisted Surveys: On-line

• Use HTML to write the survey.• Survey can be found on the web or emailed to a

potential respondent

Advantages• Prepared with speed and accuracy of electronics

Disadvantages• Possible non-response bias• Respondents must seek the survey• May not be a representative sample

How have you changed the way you administer surveys at Colmar Brunton?

Peter Kenny, Managing Director

Colmar Brunton

Preparing your own on-line survey

• A number of companies allow you to design your own survey– www.surveymonkey.com – www.zoomerang.com

• Some research companies have their own panel whom they send surveys to if they meet the criteria specified by the researcher.– www.researchnow.com.au –

http://au.acnielsen.com/industry/online.shtml

– www.theoru.com.au

A Comparative Evaluation of Survey Methods

Combining Methods

• Several alternatives are available• Allows researcher to capitalise on

strengths and minimise limitations of methods involved

• Overall goal is to improved response rates

Factors Affecting Choice of Survey Method

• Sampling• Type of population• Question form• Question content• Response rates• Costs• Available staff• Time

Length of Questionnaire

• Influenced by:– information required– characteristics of respondents– communication method

Example: Shopping centre intercepts are typically 5 minutes or less so pointless drafting lengthy survey

51

Errors in survey research

52

Random sampling error

• A statistical fluctuation that occurs because of change variation in the elements selected for the sample.

53

Systematic error

• Systematic error results from some imperfect aspect of the research design or from a mistake in the execution of the research.

• Sample bias exists when the results of a sample show a persistent tendency to deviate in one direction from the true value of the population parameter.

54

Respondent error

• A classification of sample bias resulting from some respondent action or inaction.– Non–response error– Response bias.

55

Non–response error

• Non–response error: the statistical differences between a survey that includes only those who responded and a perfect survey that would also include those who failed to respond.

• Non–respondent: a person who is not contacted or who refuses to cooperate in the research.

• No contact: a person who is not at home.

56

Non–response error

• Refusal: a person who is unwilling to participate in a research project.

• Self–selection bias: a bias that occurs because people who feel strongly about a subject are more likely to respond to survey questions than people who feel indifferent about it.– Over–represent extreme positions– Under–represent indifference.

57

Response bias

• A bias that occurs when respondents tend to answer questions with a certain slant that consciously or unconsciously misrepresents the truth.

• Deliberate falsification: occasionally people deliberately give false answers.

• Unconscious misrepresentation: response bias arising from question format or content, even when respondent is trying to be truthful.

58

Types of response bias

• Acquiescence bias results because some individuals tend to agree with all questions or to concur with a particular position.

• Extremity bias results because response styles vary from person to person; some individuals tend to use extremes when responding to questions.

59

Types of response bias

• Interviewer bias occurs because the presence of the interviewer influences answers.

• Auspices bias is caused by the respondents being influenced by the organisation conducting the study.

• Social desirability bias is caused by respondents’ desire, either conscious or unconscious, to gain prestige or appear in a different social role.

60

Administrative error

• Data processing error: incorrect data entry, computer programming, or other procedural errors during the analysis stage.

• Sample selection error: improper sample design or sampling procedure execution.

• Interviewer error: field mistakes• Interviewer cheating: filling in fake answers

or falsifying interviewers.

61

Classifying survey research methods

• Surveys may be classified based on the method of communication, the degrees of structure and disguise in the questionnaire, and the time frame in which the data are gathered.

62

Structure and disguised questions

• A structured question imposes a limit on the number of allowable responses.

• An unstructured question does not restrict the respondents’ answers.

• An undisguised question is straightforward and assumes the respondent is willing to answer.

• A disguised question assumes the purpose of the study must be hidden from the respondent.

63

Temporal classification

• Cross–sectional study: various segments of a population are sampled and data are collected at a single moment in time.

• Longitudinal study: respondents are surveyed at different times, thus allowing analysis of continuity and changes over time.– Tracking study uses successive samples to

compare trends and identify changes.

64

Temporal classification

• Consumer panel is a longitudinal study of the same sample of individuals or households to record their attitudes, behaviour, or purchasing habits over time.

65

Mail questionnaires• Self–administered questionnaire sent to

respondents through the mail.• Geographic flexibility• Relatively inexpensive• More convenient for respondents• Anonymity of respondent• Absence of interviewer• Highly standardised questions• Time consuming.

66

Selecting the appropriate survey research design

• There is no best form of survey; each has advantages and disadvantages.– For example, a researcher who must ask

highly confidential questions may use a mail survey, thus sacrificing speed of data collection to avoid interviewer bias.

• Table 5.2 is a summary of the major advantages and disadvantages of typical survey methods.

67

68

Pre–testing

• Pre–testing involves a trial run with a group of respondents to iron out fundamental problems in the instructions or design of a questionnaire.

• Avoids problems of having respondents misunderstand a particular question, skip a series of questions or misinterpret instructions.