Questionnaires

Coombes, 2001, Research Using IT

Stages of Questionnaires

1. Selecting the population2. Designing the questionnaire

a. Preparatory workb. Question contentc. Questionnaire formatd. Pre-testing

3. Training the interviewers4. Coding the data5. Analyzing the data

Questionnaire should be…

• Objectives questions• Completion easy and simple• Answers data needed & data analysis• Good Questions

Clear Explicit No ambiguity

• Wrong questions Confusing Wrong Not crucial

Open vs. Closed Questions

• Number of subjects• Data analysis• Advantages &

disadvantages (p. 126)

Open-Ended Questions

Advantages• More freedom to

express opinion• Bias reduced (not

limited to certain answers)

• Can explain answers• More questions to

clarify• Not influencing the

answers

Disadvantages • Analysis is time

consuming• Longer time to

carry out• Trainings for

researcher• Answer may not be

full for busy participants

Closed Questions

Advantages

• Quick to administer

• Easier to code and analyze

• Participants with more info can’t express themselves more

Disadvantages• Misleading

conclusions because options are limited

• Not able to express personal opinions (choosing the nearest answers)

Problem Questions

•Double-barred questions Two in one questions “and”? Check!

• Leading questions Emotive No “do you think…”, “do you

agree…”

Problem Questions (con’t)

•Hypothetical questions Imaginary situation No “if”

•Memory questionsRecalling events, information, or

datesCan you answer the question? If

not, revise!Use “can you list…” or “what

were your subjects in college..”

Problem Questions (con’t)

•Sensitive questionsCulturally not OK questionsPut at the end of the

questionnaireUse a show card (a list of

banded answers)

•Long questionsToo long too complicated

to understandUse a show card (p. 129)Shorten/divide the

question

Problem Questions (con’t)

• Questions requiring prior knowledge Takes time to answer If you can’t instantly

answer it, remove it or restate it

• Confusing questions Ambiguous/

overlapping categories (p. 131)

Piloting the questionnaire

Problem Questions (con’t)

• Specific questions Comparative words (p. 132) Make it specific/countable

Presentation of Questionnaire

• See p. 133• Package, package, package!• Clearly printed• Consistent in display• Logically presented• Plenty of space for reply• Clear and unambiguous

instructions

Coding the questionnaire

• Summary sheet (for questionnaire in p. 134)

Q #

Question 1 Question 2 Question 3 Question 4 Question 5

1.1

1.2 1.3

Other

2.1

2.2

2.3

2.4

Other

3.1

3.2

3.3

4.1

4.2

4.3

5.1

5.2

5.3

Other

1

2

3 Gate-crashed

4 free

Coding Data for Open-Ended Answers

• Simple coding Create a categorization Each time the answer refers to one of the

category, write a number beside it Move the coded answer to the summary sheets

• Individual topic coding numbers Long, in-depth open-ended answers Important vs. unimportant answers Worth quoting? Steps:

o Different color for different subjectso Papers with different headings (topics)o Cut and paste the same topic answers to the

heading papero Code the answers in the heading paper +

reference to the page number

Coding the questionnaire (con’t)

• Use commercially available software• Excel or SPSS• Easy to do statistical

analysis

Piloting the questionnaire

• Try it on!• Pilot subjects should be

similar to the subjects• Ask questions (see p. 137)

Length of completion? Clear questions? Ambiguous questions? Objection to answer? Layout/presentation: easy to follow? First impression? Any other comments?

Distributing the finished questionnaire

• Objectives explanation Separate letter Instructions on top

• Return date 2 weeks or less

• Return arrangement• Methods of distribution• Pre-coding questionnaires• Analysis of data obtained