chapter 5: constructing open- and closed-ended questions damon burton university of idaho university...
TRANSCRIPT
![Page 1: CHAPTER 5: CONSTRUCTING OPEN- AND CLOSED-ENDED QUESTIONS Damon Burton University of Idaho University of Idaho](https://reader036.vdocuments.net/reader036/viewer/2022062305/56649dce5503460f94ac2bda/html5/thumbnails/1.jpg)
CHAPTER 5: CONSTRUCTING OPEN- AND CLOSED-ENDED
QUESTIONS
CHAPTER 5: CONSTRUCTING OPEN- AND CLOSED-ENDED
QUESTIONSDamon BurtonDamon Burton
University of IdahoUniversity of Idaho
![Page 2: CHAPTER 5: CONSTRUCTING OPEN- AND CLOSED-ENDED QUESTIONS Damon Burton University of Idaho University of Idaho](https://reader036.vdocuments.net/reader036/viewer/2022062305/56649dce5503460f94ac2bda/html5/thumbnails/2.jpg)
What is the difference between open- and closed-ended questions?
![Page 3: CHAPTER 5: CONSTRUCTING OPEN- AND CLOSED-ENDED QUESTIONS Damon Burton University of Idaho University of Idaho](https://reader036.vdocuments.net/reader036/viewer/2022062305/56649dce5503460f94ac2bda/html5/thumbnails/3.jpg)
QUESTION-TYPE DEFINITIONS
•Open-Ended Questions – are questions in which the choice of responses are not specified and respondents must generate their own responses.
•Close-Ended Questions – are items in which responses choices are specified and respondents must choose one of
the available alternatives.
![Page 4: CHAPTER 5: CONSTRUCTING OPEN- AND CLOSED-ENDED QUESTIONS Damon Burton University of Idaho University of Idaho](https://reader036.vdocuments.net/reader036/viewer/2022062305/56649dce5503460f94ac2bda/html5/thumbnails/4.jpg)
TYPES OF OPEN-ENDED QUESTIONS
• Descriptive Questions – in which respondents are asked to provide in-depth information on the topic of the question.
• Numerical Response Questions – respondents must provide numerical data such as dates, frequency, monetary value, count, amount or scalar value.
• List-of-Items Questions – respondents must provide a list such as grocery stores they frequent, brands of clothing, or major purchases they’d like to make.
![Page 5: CHAPTER 5: CONSTRUCTING OPEN- AND CLOSED-ENDED QUESTIONS Damon Burton University of Idaho University of Idaho](https://reader036.vdocuments.net/reader036/viewer/2022062305/56649dce5503460f94ac2bda/html5/thumbnails/5.jpg)
We’re going to discuss a number of different question types and show you how to best write each type of question.
![Page 6: CHAPTER 5: CONSTRUCTING OPEN- AND CLOSED-ENDED QUESTIONS Damon Burton University of Idaho University of Idaho](https://reader036.vdocuments.net/reader036/viewer/2022062305/56649dce5503460f94ac2bda/html5/thumbnails/6.jpg)
NUMERICAL RESPONSE GUIDELINES
1.Ask for the specific unit desired in the question stem (5.1).
2.Provide answer spaces that are sized appropriately for the response task (5.2).
3.Provide unit labels with the answer spaces (5.1).
4.Specificity enhances response accuracy.
![Page 7: CHAPTER 5: CONSTRUCTING OPEN- AND CLOSED-ENDED QUESTIONS Damon Burton University of Idaho University of Idaho](https://reader036.vdocuments.net/reader036/viewer/2022062305/56649dce5503460f94ac2bda/html5/thumbnails/7.jpg)
5.1 NUMBER BOX QUESTIONS
![Page 8: CHAPTER 5: CONSTRUCTING OPEN- AND CLOSED-ENDED QUESTIONS Damon Burton University of Idaho University of Idaho](https://reader036.vdocuments.net/reader036/viewer/2022062305/56649dce5503460f94ac2bda/html5/thumbnails/8.jpg)
5.2 IMPACT OF ANSWER SPACES ON RESPONSES
![Page 9: CHAPTER 5: CONSTRUCTING OPEN- AND CLOSED-ENDED QUESTIONS Damon Burton University of Idaho University of Idaho](https://reader036.vdocuments.net/reader036/viewer/2022062305/56649dce5503460f94ac2bda/html5/thumbnails/9.jpg)
DESCRIPTION & ELABORATION GUIDELINES
1.Provide extra motivation to respond (5.3).
2.Provide adequate space for respondents to completely answer the question (5.3).
3.Use scrollable boxes on web surveys (5.3).
4.Consider programming probes to open-ended responses in internet surveys (5.4).
![Page 10: CHAPTER 5: CONSTRUCTING OPEN- AND CLOSED-ENDED QUESTIONS Damon Burton University of Idaho University of Idaho](https://reader036.vdocuments.net/reader036/viewer/2022062305/56649dce5503460f94ac2bda/html5/thumbnails/10.jpg)
5.3 DESCRIPTION & ELABORATION QUESTIONS
= 4/DAY
![Page 11: CHAPTER 5: CONSTRUCTING OPEN- AND CLOSED-ENDED QUESTIONS Damon Burton University of Idaho University of Idaho](https://reader036.vdocuments.net/reader036/viewer/2022062305/56649dce5503460f94ac2bda/html5/thumbnails/11.jpg)
5.4 PROBING OPEN-ENDED QUESTIONS
![Page 12: CHAPTER 5: CONSTRUCTING OPEN- AND CLOSED-ENDED QUESTIONS Damon Burton University of Idaho University of Idaho](https://reader036.vdocuments.net/reader036/viewer/2022062305/56649dce5503460f94ac2bda/html5/thumbnails/12.jpg)
You’ll probably use more closed- than open-ended questions, so here are guidelines for how to write good items.
![Page 13: CHAPTER 5: CONSTRUCTING OPEN- AND CLOSED-ENDED QUESTIONS Damon Burton University of Idaho University of Idaho](https://reader036.vdocuments.net/reader036/viewer/2022062305/56649dce5503460f94ac2bda/html5/thumbnails/13.jpg)
CLOSED-ENDED GUIDELINES
1.State both positive and negative sides in the question stem when asking either/or questions (5.5).
2.Develop lists of answer categories that include all reasonable possible answers (5.6).
3.Develop lists of answer categories that are mutually
exclusive questions (5.7).
![Page 14: CHAPTER 5: CONSTRUCTING OPEN- AND CLOSED-ENDED QUESTIONS Damon Burton University of Idaho University of Idaho](https://reader036.vdocuments.net/reader036/viewer/2022062305/56649dce5503460f94ac2bda/html5/thumbnails/14.jpg)
5.5 INCLUDE POSITIVE & NEGATIVE SIDES
= 4/DAY
![Page 15: CHAPTER 5: CONSTRUCTING OPEN- AND CLOSED-ENDED QUESTIONS Damon Burton University of Idaho University of Idaho](https://reader036.vdocuments.net/reader036/viewer/2022062305/56649dce5503460f94ac2bda/html5/thumbnails/15.jpg)
5.6 MUTUALLY EXCLUSIVE
CATEGORIES
= 4/DAY
![Page 16: CHAPTER 5: CONSTRUCTING OPEN- AND CLOSED-ENDED QUESTIONS Damon Burton University of Idaho University of Idaho](https://reader036.vdocuments.net/reader036/viewer/2022062305/56649dce5503460f94ac2bda/html5/thumbnails/16.jpg)
5.7 ORDERING OF RESPONSES
= 4/DAY
![Page 17: CHAPTER 5: CONSTRUCTING OPEN- AND CLOSED-ENDED QUESTIONS Damon Burton University of Idaho University of Idaho](https://reader036.vdocuments.net/reader036/viewer/2022062305/56649dce5503460f94ac2bda/html5/thumbnails/17.jpg)
CLOSED-ENDED GUIDELINES
4. Maintain spacing between answer categories that is consistent with measurement intent (Figure 5.8).
5. Ask respondents to rank only a few items at once rather than a long list (Figure 5.9).
• Ranking questions are difficult to understand and complete correctly.
• Only have respondents rank top 3-4.
• Pair all possible options and calculate overall rankings during analysis.
![Page 18: CHAPTER 5: CONSTRUCTING OPEN- AND CLOSED-ENDED QUESTIONS Damon Burton University of Idaho University of Idaho](https://reader036.vdocuments.net/reader036/viewer/2022062305/56649dce5503460f94ac2bda/html5/thumbnails/18.jpg)
5.8 SPACING RESPONSE OPTIONS
EVENLY
= 4/DAY
![Page 19: CHAPTER 5: CONSTRUCTING OPEN- AND CLOSED-ENDED QUESTIONS Damon Burton University of Idaho University of Idaho](https://reader036.vdocuments.net/reader036/viewer/2022062305/56649dce5503460f94ac2bda/html5/thumbnails/19.jpg)
5.9 CLOSED-ENDED WITH UNORDERED
RESPONSES
= 4/DAY
![Page 20: CHAPTER 5: CONSTRUCTING OPEN- AND CLOSED-ENDED QUESTIONS Damon Burton University of Idaho University of Idaho](https://reader036.vdocuments.net/reader036/viewer/2022062305/56649dce5503460f94ac2bda/html5/thumbnails/20.jpg)
CLOSED-ENDED GUIDELINES
6. Avoid bias from unequal comparisons (Figure 5.10).
7. Randomize response options if there is concern about order effects (Figure 5.11).
8.Use forced-choice questions instead of check-all-that-apply questions (Figure 5.11).
![Page 21: CHAPTER 5: CONSTRUCTING OPEN- AND CLOSED-ENDED QUESTIONS Damon Burton University of Idaho University of Idaho](https://reader036.vdocuments.net/reader036/viewer/2022062305/56649dce5503460f94ac2bda/html5/thumbnails/21.jpg)
5.10 AVOIDING UNEQUAL COMPARISON
BIAS
= 4/DAY
![Page 22: CHAPTER 5: CONSTRUCTING OPEN- AND CLOSED-ENDED QUESTIONS Damon Burton University of Idaho University of Idaho](https://reader036.vdocuments.net/reader036/viewer/2022062305/56649dce5503460f94ac2bda/html5/thumbnails/22.jpg)
5.11 UNORDERED RESPONSE OPTIONS
= 4/DAY
![Page 23: CHAPTER 5: CONSTRUCTING OPEN- AND CLOSED-ENDED QUESTIONS Damon Burton University of Idaho University of Idaho](https://reader036.vdocuments.net/reader036/viewer/2022062305/56649dce5503460f94ac2bda/html5/thumbnails/23.jpg)
5.12 COMPARISON OF RESPONSE RATES
= 4/DAY
![Page 24: CHAPTER 5: CONSTRUCTING OPEN- AND CLOSED-ENDED QUESTIONS Damon Burton University of Idaho University of Idaho](https://reader036.vdocuments.net/reader036/viewer/2022062305/56649dce5503460f94ac2bda/html5/thumbnails/24.jpg)
CLOSED-ENDED GUIDELINES
9. Use differently shaped answer spaces (circles vs squares) to help respondents distinguish between single- and multiple-answer questions (Figure 5.13).
![Page 25: CHAPTER 5: CONSTRUCTING OPEN- AND CLOSED-ENDED QUESTIONS Damon Burton University of Idaho University of Idaho](https://reader036.vdocuments.net/reader036/viewer/2022062305/56649dce5503460f94ac2bda/html5/thumbnails/25.jpg)
5.13 QUESTION TYPE & SHAPES OF ANSWER
SPACES
= 4/DAY
![Page 26: CHAPTER 5: CONSTRUCTING OPEN- AND CLOSED-ENDED QUESTIONS Damon Burton University of Idaho University of Idaho](https://reader036.vdocuments.net/reader036/viewer/2022062305/56649dce5503460f94ac2bda/html5/thumbnails/26.jpg)
How do the type of analyses you plan to do influence your response choices?
![Page 27: CHAPTER 5: CONSTRUCTING OPEN- AND CLOSED-ENDED QUESTIONS Damon Burton University of Idaho University of Idaho](https://reader036.vdocuments.net/reader036/viewer/2022062305/56649dce5503460f94ac2bda/html5/thumbnails/27.jpg)
ORDINAL CLOSED-ENDED GUIDELINES
10. Choose appropriate scale length of 4-5 categories (Figure 5.14).
• Typically you must have a minimum of 5 choices on a Likert scale to assume interval data and utilize inferential statistics.
• Even versus odd-number scales are based on whether you want to allow respondents
to choose a neutral response.
![Page 28: CHAPTER 5: CONSTRUCTING OPEN- AND CLOSED-ENDED QUESTIONS Damon Burton University of Idaho University of Idaho](https://reader036.vdocuments.net/reader036/viewer/2022062305/56649dce5503460f94ac2bda/html5/thumbnails/28.jpg)
5.14 SCALER QUESTIONS AND RESPONSE
OPTIONS
= 4/DAY
![Page 29: CHAPTER 5: CONSTRUCTING OPEN- AND CLOSED-ENDED QUESTIONS Damon Burton University of Idaho University of Idaho](https://reader036.vdocuments.net/reader036/viewer/2022062305/56649dce5503460f94ac2bda/html5/thumbnails/29.jpg)
ORDINAL CLOSED-ENDED GUIDELINES
11. Choose direct or construct-specific labels to improve understanding (Figure 5.15).
12. Provide scales that approximate the actual distribution of the characteristic in the population. • Typically respondents assume that the
midpoint response category represents the middle of the distribution in the population.
![Page 30: CHAPTER 5: CONSTRUCTING OPEN- AND CLOSED-ENDED QUESTIONS Damon Burton University of Idaho University of Idaho](https://reader036.vdocuments.net/reader036/viewer/2022062305/56649dce5503460f94ac2bda/html5/thumbnails/30.jpg)
How does the reason for conducting the survey influence response options (e.g., academic research versus consumer satisfaction)?
![Page 31: CHAPTER 5: CONSTRUCTING OPEN- AND CLOSED-ENDED QUESTIONS Damon Burton University of Idaho University of Idaho](https://reader036.vdocuments.net/reader036/viewer/2022062305/56649dce5503460f94ac2bda/html5/thumbnails/31.jpg)
ORDINAL CLOSED-ENDED GUIDELINES
13. Provide balanced scales where categories are relatively equal distance apart conceptually (Figure 5.16).
14. Consider how verbally labeling and visually displaying all response categories may influence answers (Figure 5.17). • Dillman suggests that most surveyors prefer fully
labeled scales that range from 2-5 choices.
• Conversely, researchers tend to prefer numerical scales of 5-9-point Likert scales (see examples).
![Page 32: CHAPTER 5: CONSTRUCTING OPEN- AND CLOSED-ENDED QUESTIONS Damon Burton University of Idaho University of Idaho](https://reader036.vdocuments.net/reader036/viewer/2022062305/56649dce5503460f94ac2bda/html5/thumbnails/32.jpg)
5.16 EVENLY-SPACE CONCEPTUAL CATEGORIES
= 4/DAY
![Page 33: CHAPTER 5: CONSTRUCTING OPEN- AND CLOSED-ENDED QUESTIONS Damon Burton University of Idaho University of Idaho](https://reader036.vdocuments.net/reader036/viewer/2022062305/56649dce5503460f94ac2bda/html5/thumbnails/33.jpg)
5.17 VERBAL VERSUS NUMERICAL LABELS
= 4/DAY
![Page 34: CHAPTER 5: CONSTRUCTING OPEN- AND CLOSED-ENDED QUESTIONS Damon Burton University of Idaho University of Idaho](https://reader036.vdocuments.net/reader036/viewer/2022062305/56649dce5503460f94ac2bda/html5/thumbnails/34.jpg)
ORDINAL CLOSED-ENDED GUIDELINES
15. Carefully evaluate the use of numeric labels and their impact on measurement (see examples).
• Numbers reinforce the interval nature of data.
• Dillman contends that numbers add to processing time, but this depends on whether questions use same response categories or change categories from question to question.
• Number consistently across questions.
![Page 35: CHAPTER 5: CONSTRUCTING OPEN- AND CLOSED-ENDED QUESTIONS Damon Burton University of Idaho University of Idaho](https://reader036.vdocuments.net/reader036/viewer/2022062305/56649dce5503460f94ac2bda/html5/thumbnails/35.jpg)
ORDINAL CLOSED-ENDED GUIDELINES
16. Align response options vertically in one column or horizontally in one row and strive for equal distance between
categories (Figure 15.18).17. Place nonsubstantive options at the
end of the scale and separate from substantitive options (Figure 15.19).
• Be very careful about using “don’t know” or “no opinion” categories.
![Page 36: CHAPTER 5: CONSTRUCTING OPEN- AND CLOSED-ENDED QUESTIONS Damon Burton University of Idaho University of Idaho](https://reader036.vdocuments.net/reader036/viewer/2022062305/56649dce5503460f94ac2bda/html5/thumbnails/36.jpg)