Questionnaire Issues --Asking the Right Questions

Requirements for a Good Question

1. You must ask the right questions.

2. Respondents must properly understand your questions.

3. Respondents must know the answers.

4. Respondents must be willing and able to tell you those answers.

Ask the Right Questions!

• Information Goals should guide Questionnaire construction

• Make a list “What should I know?”• Remember the purpose of your research

is to assist in marketing decisions.• Prepare a detailed plan of how the data

will be analyzed, including dummy tables and graphs.

Ask the Right Questions!

• Why are you asking the question?

• Ask yourself whether the question fits the way the market works.

• Ways to Measure Awareness of a Service

• Recognition measures– I have a list of restaurants. As I read each one, please tell

me whether or not you have heard of that restaurant. How about . . .

• Recall measures- What are all the names of drycleaners that you can think of?

Do Respondents Understand the Words in the Question?

• Q: Does your home have power conditioning equipment to protect your critical equipment against power fluctuations?

Do Respondents Understand the Question?

• Q : I am going to read you a short list of words that could be used to describe TV programs, movies, products, or almost anything else. If "so-so" is worth 50 points, please tell me what number you would assign to each of these other words. You can give each word any number above zero, no matter how large it is. Let's start with bad. If so-so is worth 50 points, what number would you assign to bad?

Do All Respondents Understand it in the Same Way?

• Q: What is your income?

Do Respondents Understand in the Way You Intended?

• Q: What are all the reasons why you bought your groceries at Publix rather than some other store?

Minimizing Problems of Understanding

• #1. Be specific!

• Q: In the past six months, has your household purchased any major appliances new from the store?

Minimizing Problems of Understanding

• # 2. Specify who, what, when , where and how

• Q: What is your income?

• In 2001, about what was your total family income before taxes? Please count income from all members of your household, and from all sources including sources such as interest and dividends.

Minimizing Problems of Understanding

• # 3. Specify how the answer should be given. Use numbers.

• Q: Overall, how satisfied were you with the care you received at our hospital? – very satisfied, moderately satisfied,

slightly satisfied, or not at all satisfied?• # 4. Use simple language. Use words

with one meaning.

Minimizing Problems of Understanding

• # 5. Ask questions one at a time.

• Q: In the past six months, have you bought a TV or a VCR?

• Pretest the questionnaire.

Do they know the answer?

• Were they there?– Proxy respondents.

• Can they remember?– Memory effects– Overestimation and underestimation issues.– Advertising and recall.

• Q: In the past two months, how many times have you dined at fast food restaurants?

Do they know the answer?• Do they have opinions?• Q: I have a list of hospitals in our area. As I read each one,

please tell me how you would rate that hospital as a place to receive care: excellent, good, fair, or poor. How about . . .

 • Excellent Good Fair Poor • Eastside Hospital? . . . 4 . . . . 3 . . . . 2 . . . . 1•  • Lakeview Hospital? . 4 . . . . 3 . . . . 2 . . . . 1•  • Presbyterian Hospital? . 4 . . . 3 … . . 2 . . . 1•  • St. Peter's Hospital? . 4 . . . . 3 . . . . 2 . . . . 1

Are Intentions Meaningful?

• Intention data have problems– People may not know.– Lead time issues– Demand effects

• Q: If there was a local magazine like Consumer Reports that evaluated restaurants, auto repair shops, plumbers, stores, and other local businesses, what is the likelihood that your family would subscribe to this magazine? Would you say your family definitely would subscribe, probably would subscribe, might or might not subscribe, probably would not subscribe, or definitely would not subscribe?

How do you minimize problems from respondents not knowing the

answers?1. Qualify respondents for knowledge2. Use the right time frame. Rare events can

be recalled after several years.3. Consider providing cues to aid memory. Q: People drink beer at many places – at home,

at restaurants, at bars, at sporting events, at friends’ homes, etc.,

• During the past month, since (DATE), did you drink beer even once?

Willingness to Respond!

• The problem of social desirability

• Q: During the past month, have you read any books other than for work or school?

Minimizing the Problem of Social Loading

• Check questions for social loading– Avoid “angel” or “devil” words– Do you support union czars in their efforts

to force people to join unions?– Avoid extreme words such as “never” or

“always”– Do you think the oil industry is doing

everything possible to…..?

Minimizing the Problem of Social Loading

– Avoid appeals to the norm.– Do you feel, as most normal people do,

that air pollution is a serious threat to the future?

– Don’t ask subjective questions with yes-no answer.

– Do you think Rollins College offers good education?

Minimizing the Problem of Social Loading

• Balance alternatives when contrasting alternatives are given. – Do you think that Pepsi is better or worse

than Coke?– Would you say that Pepsi is better than,

about the same or not as good as Coke?

Minimizing Problems of Knowing

• Do not reveal the identity of the sponsoring company?

• Use longer questions to reduce social stigma.– Many people feel that they just don’t have the

time for reading anymore. In the past month, have you read any books other than for work or for school?

How do you minimize problems from respondents not knowing the

answers

1. Qualify respondents for knowledge

2. Use the right time frame

3. Consider providing cues to aid memory

4. BOUNDED RECALL

5. Use multiple measures for imprecise concepts

Other Issues

• Minimizing Order Effects.• When changing topics, use some transitional

phrase to make it easier for respondents to switch their trains of thought.

• If you ask questions about behaviors over some time period (for example, grocery shopping during the past month) follow chronological order backward in time; i.e., the most recent purchase, the time before that, etc.

Other Issues!

• Start with non threatening questions• No demographic questions unless it is a

screener question.– "What are all the brands of hair shampoo you can

think of?" – Have you bought shampoo in the past 30 days?

• Open and Closed questions.• Number of response categories.• Use vertical formats for closed questions.