designing instrument for data collection

8/4/2019 Designing Instrument for Data Collection

http://slidepdf.com/reader/full/designing-instrument-for-data-collection 1/35

Questionnaire & FormDesign



Questionnaire Design – lookseasy….but…..

• We all think we are experts at asking questions

• Mistakes often do not lead to obvious errors

– Interviewers and responders do their best

• Consequences of mistakes are not immediate

• Disparate interests involved

– Client, interviewer, respondent, data entry persons



About 26 steps involved!

• Mistakes could occur in any one of these

– Questionnaires are part of a lengthycommunication process



“Don’t shoot me, I’m just the statistical

analyst”

• Extreme phrasing of question or combination ofquestions is the most frequent source ofdistortion in survey results

• So let’s see how we can design goodquestionnaires…



Questionnaire Definition

• A questionnaire is a formalized set of questions forobtaining information from respondents.



Types of Questions



Choosing Question Structure Unstructured Questions

• Unstructured questions are open-ended questions thatrespondents answer in their own words.

How do you feel?

__________________________________



Choosing Question Structure Structured Questions

• Structured questions specify the set of responsealternatives and the response format. A structuredquestion may be multiple-choice, dichotomous, or ascale.



Choosing Question Structure Dichotomous Questions

• A dichotomous question has only two responsealternatives: yes or no, agree or disagree, and so on.

• Often, the two alternatives of interest are supplemented

by a neutral alternative, such as “no opinion,” “don'tknow,” “both,” or “none.”

Do you intend to buy a new car within the next sixmonths?

_____ Yes _____ No _____ Don't know



Layout Considerations

Make your questionnaire…

• Short

• Simple

• Sweet



For interviewers

–Make it short • 15 minutes by phone

• 30 to 40 minutes in-home

– use show cards and shuffle packs

– Make instructions explicit

• Put them in capital letters



For Respondents

• Make it simple

• Be respectful

– Address concerns up front

– “Thank you” at the end

• Make it flow – general > interesting > difficult or tedious > interesting

> sensitive > classification



For data entry persons

• Make it sweet (looking) – Use pre-codes

– Lay-out matters

• Questions on left, responses to right

• Include a space for survey/project number



Questionnaire

Design

Process



Questionnaire design issues

• Factual data

• Problem of understanding

• Problem of willingness to answer

• Problem of ability to answer



Are Several Questions Needed Insteadof One?

• Sometimes, several questions are needed to obtain therequired information in a clear manner. Consider thequestion,

“Do you think Coca-Cola is a tasty and refreshing soft drink?” (Incorrect)

• Such a question is called a double-barreled question,because two or more questions are combined into one.

To obtain the required information, two distinctquestions should be asked:

“Do you think Coca-Cola is a tasty soft drink?” and

“Do you think Coca-Cola is a refreshing soft drink?” (Correct)



Overcoming Inability To Answer Is the Respondent Informed?

• In situations where not all respondents are likely to beinformed about the topic of interest, filter questions thatmeasure familiarity and past experience should be asked

before questions about the topics themselves.

• A “don't know” option appears to reduce responses

without reducing the response rate.



Overcoming Unwillingness To Answer Effort Required of the Respondents

• Most respondents are unwilling to devote a lot of effort toprovide information.



Overcoming Unwillingness To Answer

Please list all the departments from which you purchased

merchandise on your most recent shopping trip to a department

store. (Incorrect)

In the list that follows, please check all the departments from

which you purchased merchandise on your most recent shopping

trip to a department store.

1. Women's dresses ____ 2. Men's apparel ____ 3. Children's apparel ____

4. Cosmetics ____ ...16. Jewelry ____ 17. Other (please specify) ____ (Correct)



Choosing Question Wording Define the Issue

• Define the issue in terms of who, what, when, where,why, how?

• Who, what, when, and where are particularly important.

Which brand of shampoo do you use? (Incorrect)

Which brand or brands of shampoo have you

personally used at home during the last month?In case of more than one brand, please

list all the brands that apply. (Correct)



Choosing Question Wording Use Unambiguous Words

In a typical month, how often do you shop in departmentstores?

_____ Never _____ Occasionally

_____ Sometimes _____ Often _____ Regularly (Incorrect)

In a typical month, how often do you shop in departmentstores?

_____ Less than once _____ 1 or 2 times _____ 3 or 4 times

_____ More than 4 times (Correct)



Choosing Question Wording Avoid Leading or Biasing Questions

• A leading question is one that clues the respondent to what theanswer should be, as in the following:

Do you think that patriotic Americans should buy importedautomobiles when that would put American labor out of work?

_____ Yes _____ No _____ Don't know

(Incorrect)

Do you think that Americans should buy imported automobiles? _____ Yes _____ No _____ Don't know

(Correct)



Choosing Question Wording Avoid Implicit Assumptions

• Questions should not be worded so that the answer isdependent upon implicit assumptions about what willhappen as a consequence.

1. Are you in favor of a balanced budget?(Incorrect)

2. Are you in favor of a balanced budget if it wouldresult in an increase in the personal income tax?

(Correct)



Determining the Order ofQuestions

Opening Questions

• The opening questions should be interesting, simple,and non-threatening.

Type of Information • As a general guideline, basic information should be

obtained first, followed by classification, and, finally,identification information.

Difficult Questions

• Difficult questions or questions which are sensitive,embarrassing, complex, or dull, should be placed late inthe sequence.

D t i i th O d



Determining the Order

of Questions

Effect on Subsequent Questions • General questions should precede the

specific questions (funnel approach).

Q1: “What considerations are important toyou in selecting a department store?”

Q2: “In selecting a department store, how

important is convenience of location?”

(Correct)



Determining the Order ofQuestionsLogical Order

The following guidelines should be followed for branchingquestions:

• The question being branched (the one to which therespondent is being directed) should be placed as closeas possible to the question causing the branching.

• The branching questions should be ordered so that therespondents cannot anticipate what additional informationwill be required.



Some thumb-rules for Questionnaire Design

• 1. Don’t crowd the questions

– makes the questionnaire look difficult

• 2. Don’t split questions across pages.

• 3. Put special instructions on the questionnaire• use a different typeface (bold, CAPITALS, or italics).

• put “Don’t read” above the question.

• put “Skip” immediately after trigger answers.

• 4. Precode all closed questions.



• 5. Use vertical formats for closed questions

– NO! What is your favorite brand of cereal?

• A ___ B___ C____ Other_____

– Yes! What is your favorite brand of cereal?

• 1……….BRAND A

• 2……...BRAND B • 3……...BRAND C

• 4……...OTHER




• 6. Record the interview start and stop times.

• 7. Provide a beginning (a greeting ) and an endto the interview (A thank you ). – “Hello, my name is (your name), and I’m calling from

Harmon Research, a marketing research company. I needto speak with the person who is most responsible for healthcare decisions in your household. Is that you?”




Less than 30 minutes....................-1

30 to 59 minutes............................-2

1 hour or more……………………..-3

The American LawyerA Confidential Survey of Our Subscribers

(Please ignore the numbers alongside the answers. They are only to help us

in data processing.)

1. Considering all the times you pick it up, about how much time, in total, do

you spend reading or looking through a typical issue of THE AMERICAN

LAWYER?

Example of a Precoded Questionnaire



Pretesting

Pretesting refers to the testing of the questionnaire on asmall sample of respondents to identify and eliminatepotential problems.

• A questionnaire should not be used in the field surveywithout adequate pretesting.

• The pretest sample size varies from 15 to 30

respondents for each wave.



Pre-test, pre-test, pre-test!

• Discuss with client, data entry persons, and respondents

• The career you save may be your own!



Types of errors

Random sampling errors: • sample too small• sample not representative• inappropriate sampling method used

Research design errors: • bias introduced• measurement error• data analysis error• population definition error• scaling error• question construction error



Interviewer errors: • recording errors• cheating errors

• questioning errors• respondent selection errorRespondent errors: • non-response error

• inability error• falsification errorHypothesis errors: • type I error (also called alpha error)

– the study results lead to the rejection of the null hypothesis even

though it is actually true• type II error (also called beta error)

– the study results lead to the acceptance (non-rejection) of thenull hypothesis even though it is actually false

http://en.wikipedia.org/wiki/Type_I_and_type_II_errors




designing instrument for data collection

Documents