5.4 Creating Survey Questions

Surveys

• Very common for data collection

• Types:– Face-to-face interviews– Telephone surveys– Internet surveys– Mail-in surveys

• All have questions to answer

Creating Survey Questions

• Difficult!• Must pilot/test draft questionnaire

– with people who will not form part of survey sample

• Primary data:– Collected by researcher

• Secondary data:– Collected from other sources (e.g. internet)

Creating Survey Questions

• Should be– Relevant to the study– Readable (use familiar words)– Specific– Simple

Things to Avoid

• Jargon (slang)

• Abbreviations or acronyms – Abbreviation, not abbrev.– Jarvis Collegiate Institute, not J.C.I.

• Negatives– “Do you think we should we do this”, not “Do

you think we should not do this?”

Things to Avoid

• Leading respondents– “Should the office continue its heinous practice of

forcing students to come to detentions?”

– “Given the amount of CO2 they produce, rate your opinion of SUVs on a scale from 1 to 10.”

• Double-barrelled questions (asking two things at once)– “Do you walk or take the bus?”

• Insensitivity: belittling or insulting

Question Styles• Open questions

– Respondents answer in their own words

– E.g.: “What changes should be made in the school to ensure better student achievement?”

• Closed questions– Respondents given limited number of responses to

chose from

– Response categories should be • Mutually exclusive: should not be able to choose more than

one unless you can “choose as many as apply”

• Exhaustive: all possibilities listed

Open vs. Closed

• Open Pros– Respondent can interpret

and answer question any way they choose

• Open Cons:– Not necessarily consistent– Difficult to

analyse/interpret answers– Wide variety of

possibilities

• Closed Pros– Faster and easier to

answer– Faster and easier to code

and analyse– Provides consistency

• Closed Cons– Restricts answers to

choices given– Options may bias choice

Ex: Inconsistent Answers

• One orange & a little water and sugar

• 25 % orange juice, 75% carbonated water

• Juice of ½ dozen oranges• 3 oz• Full strength• ¼ c. orange juice

• None• Not much• Don’t know• A pint• Most of it• About 1 ½ glass

Difficulties with open questions:

“How much orange juice do you think this bottle contains?”

Ex: Inconsistent Answers

For a uniform response, sometimes you need to state what kind of response is needed.

“This bottle contains 250 mL orange drink. How many mL of this drink would you say are orange juice?”

millilitres

millilitres

Question Styles

• 4 main types:– Information– Checklist– Ranking– Rating

Information Questions

• Solicit information

• Usually fill-in-the-blank & open

What kind of car do you own?

Model _________________

Make __________________ Year _____

Checklist Questions

• Closed

• Exhaustive

Note that these are mutually exclusive response categories.

What level of license do you hold?

G2 □ G1 □ G □ None

Another Checklist Question

Problems?• Unclear if should only choose one• What if your favourite is not up there or you

don’t have a favourite (not exhaustive)

Other (please state): ________________ None of the above/I don’t have a favourite

(Check only one)(Check all that apply)Which of the following brand names is your favourite? Honda □ Toyota □ Nissan □ GM

Ranking Questions

• Orders preferences

Rank the following car concerns in order of importance from 1 to 4:

____ price ____ appearance

____ maintenance ____ gas costs consumption

Is 1 high or low?

(1 is low, 4 is high)

Rating QuestionsHow satisfied are you with your car?

Very satisfied Satisfied

Dissatisfied Very Dissatisfied

• Can also assign a numerical value

On a scale from 0 to 10, 0 being very dissatisfied and 10 being very satisfied, how satisfied are you with your car? _____

• Could use a line: 0 1 2 3 4 5 6 7 8 9 10

Creating a Questionnaire

• Introduction– Informative– Stimulates respondent interest– Should answer: “Why should I answer this?”– State what will be done with the results

• Begin with easy and interesting questions

• Questions should flow and read well

Testing a Questionnaire

• Fantastically important! Do not skip!

• Identifies problems in survey

• Discovers poor wording/ordering– People might be unwilling/unable to answer a

question

• Identifies errors in layout and instructions

Testing a Questionnaire

• Suggests additional response categories that can be pre-coded– If 10 people suggest that Ford should be added to

favourite car…

• Provides preliminary idea of length of interview• Provides preliminary idea of refusal problems

– Should we adjust survey size to account for non-response?