Survey Design 102Getting Started

FIE-SDFSFIE-SDFS

Research & EvaluationResearch & Evaluation

20032003

Table Of Contents

Golden Rules Plan For Success

1. Survey Contents

2. Item Development

3. Response Formats Summary

Golden Rules

1. Understand the goals of the project.

2. Keep the survey simple.

3. Field/Pilot-test the survey.

4. Consider the demographic characteristics of the respondents.

5. Visual appeal.

Golden Rule #1Understand the Project Goals

A comprehensive understanding of the project goals

paves the way for successful surveying. Without

that understanding, it is impossible to appropriately

design and to maximize the power of the survey.

Golden Rule #2Keep It Simple

Avoid the temptation to add a couple of questions just in case you did not cover all the bases. Only ask questions that fit the scheme of your project. In this case, oftentimes, less is more. The design focus should be quality not quantity.

Golden Rule #3Field/Pilot Test

Pre-testing the survey enhances clarity, removes word

meaning obscurity, establishes approximate time to

completion, proper sequencing, question transitioning,

and helps assure that questions are not too difficult.

Golden Rule #4Respondent Demographic Characteristics

It is important to know what and how demographic

characteristics may impact the interpretation and

meaning of certain questions. Variations truly exist

from one setting to the next.

Golden Rule #5Visual Appeal

Appearance means everything. If the survey appears

complex and difficult to follow with the eyes,

respondents will be less likely to do a good job. It

should appear to be short, organized, and easy to

answer.

Plan For Success

Survey Content

Writing the survey requires knowing what questions need to be answered, how the results be used, reporting requirements, what types of data analyses will be conducted, construct development, demographic characteristics, and response formats etc. Directions should be clear and terse. Avoid the temptation to overstate the survey intent in the directions.

Item Development

Listed below are five key components to consider:

1. Meaning—whether the majority of respondents perceive the meaning intended by the survey developers.

2. Item difficulty—a question can be hard to answer even though the meaning is entirely clear, if the respondent does not have prior knowledge about the specific item construct.

Item Development

3. Respondent interest and attention. Assure the survey design and format is appealing as to pare down, brief instances of inattention by the respondents.

4. Flow, naturalness, and sequencing. Assure a natural flow and transition from one survey item to the next.

5. Determine if the survey creates a positive impression that allows candor from one question to the next.

6. If the question is not necessary, do not include it.

Item Development7. Avoid loaded questions or the type that may put

respondents on edge. Such as emotionally charged words.

8. Avoid putting two questions in one. Compound questions may lead to ambiguity in the data analysis phase.

9. Be specific. Are you seeking opinions, facts, or a first witness to an account?

Item Development10. Avoid the use of jargon. Your respondents should

not have to infer meaning.

11. Questions should not be leading. For example, smoking cigarettes cause cancer--

Response Formats

Survey developers use several different types of response formats. Listed below are several of the most common response formats:1) Multiple choice2) Open ended3) Rank order4) Rating Scale--Agreement5) To what extent6) Forced choice7) Closed ended

Response FormatsWhen deciding which response type to use consider the following: a. Response type should be appropriate to the item. b. Allow a don’t know response. c. Allow an other response. d. Allow a none response. e. Allow an undecided response. f. All possibilities should be included. g. Response formats should be mutually exclusive. h. Avoid using acronyms unless they are

commonplace.

Summary

This presentation is one in a series of

presentations designed principally for the

novice survey developer. Conversely,

the series may serve as a resource tool

for experience developers. If you have

questions or comments please call

Dr. William Respress at 850-413-7650

respress_w@firn.edu