Asking the Right Questions

Part I. Five tips for creating effective quantitative and qualitative questions.

Part II. Optimize the survey to collect the best data for each question type.

Part I. Five tips for creating effective quantitative and qualitative

questions.

1. Create a project goal…

and stick to it.

Define the objectives before you begin.

• Prepare survey with an end state vision.–What do you want to learn?–What type of actions are you going to

take?–What type of charts and graphs will you

share?–What will be included in the final report?

Who will take your survey?

• Define your audience.• Ask questions appropriate for that

audience.

Plan and design survey layout.

• Organize the questions in a logical format.

• Surveys are conversations.– Group questions by topic.– Unfold in a logical order

Choose the appropriate time period.

• Memories of a past event change based on how long it has been since the event occurred.

• Avoid long reaching time periods – “During the past year…”

• Choose a recent time frame for reference– “In the last 3 months…”

2. Prepare a survey that is short and sweet…

and to the point!

“Would you like to take a survey?”

• < 5 minutes– A respondent can answer • 5 quantitative questions/minute• 2 qualitative (open ended)

questions/minute

• > 5 minutes – Redesign– Add clear messaging– Compensate

Keep titles and answers short.

• Respondents will not answer lengthy questions.– A well designed table can help here

3. Write straightforward questions.

Concise, and defined, statements.

• The question title/statement should be meaningful on its own.

• Present a clearly defined question.– Poor Ex: “Which of the following

statements is true?”– Better Ex: “Which of the following

statements best reflects your most recent interaction?”

• Avoid irrelevant material/info in the title.

Avoid Conjunctions

• And, but, or, nor, for, yet, so

Positive and negative wording

• Mix positive and negative wording in questions and answer options.– Triggers the respondent to pay attention

to the context of the question.“I spent a minimal amount of time in line.”“I spent too much time in line.”

Avoid HTML

• Avoid HTML in question titles/answer options– Do not • Italicize• Underline• Bold• Change color

• Use in Text elements, section headers, descriptive text.

• Use in titles of Books, Videos, Journals, etc.

Avoid common mistakes

• Avoid acquiescence bias.– Offer two opposing statements instead of

an agreement scale.

• Avoid leading questions. (Do you like/agree)

• Avoid hypothetical questions.• Avoid difficult concepts.

4. Keep the language simple unless your audience

is technical.

Know your audience .

• Use language your audience will understand.

• Avoid abbreviations.– If you need to define it, spell it out.

• Avoid jargon. Use plain language.

Grammar guidelines

• Check subject-verb agreement.– Ex. How important are the following

features?

• Avoid negative statements.– Avoid using “not”– Avoid double negatives.

5. Vary the question type.

Vary the question type

• Use a mix of quantitative questions.• Use open-ended questions as needed.

Part II. Optimize the survey to

collect the best data for each question type.

1. Categorical Choice 2. Scale

Questions 3. Grid Questions

4. Open-Text Questions

Categorical Choice

Categorical Choice

A quantitative question with a defined set of answer options each representing a separate category.

• Radio Button• Dropdown Menu• Dropdown Menu

List• Image Choice (Select

One)

• Checkboxes• Image Choice (Multi

Select)

Categorical choice: orientation matters

Avoid overlapping answer options

Avoid ambiguous words

• could, would, should, might

• always, never• all, none• usually• may, • sometimes, occasionally

Scale Questions

Scale Questions

A quantitative question with a single select response where the answer options represent a range, scale or continuum.

• Likert Scale• Slider• Slider List• Net Promoter Score

Scale Factors to Measure

• Acceptability• Agreement• Awareness• Concern• Familiarity• Importance• Likelihood• Quality• Satisfaction

2

3

4

5

6

7

8

9

Scale: Keep scales consistent

Grid Questions

Grid Questions

A series of two or more questions where each respondent’s answers are recorded using the same metric.

• Radio Button Grid• Checkbox Grid• Textbox Grid• Dropdown Menu

Grid• Star Rating Grid• Ranking Grid• Custom Table

Grid Question Layout

Open-Text Questions

Open-Text Questions

A qualitative question without a defined list of answer options, allowing the respondent to answer in their own words.

• Textbox• Number• Email• Percent

• Date• Textbox List• Essay• Continuous Sum

Open-Text Questions: Use sparingly

Open-Text: Pilot Survey Example

Open-Text: Pilot Study Example

Asking the Right Questions