methods of data collection - johns hopkins bloomberg school of
TRANSCRIPT
![Page 1: Methods of Data collection - Johns Hopkins Bloomberg School of](https://reader031.vdocuments.net/reader031/viewer/2022020913/6203496224f6b61e9c663da6/html5/thumbnails/1.jpg)
Copyright 2007, The Johns Hopkins University and Ann Skinner. All rights reserved. Use of these materials permitted only in accordance with license rights granted. Materials provided “AS IS”; no representations or warranties provided. User assumes all responsibility for use, and all liability related thereto, and must independently review all materials for accuracy and efficacy. May contain materials owned by others. User is responsible for obtaining permissions for use from third parties as needed.
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike License. Your use of this material constitutes acceptance of that license and the conditions of use of materials on this site.
![Page 2: Methods of Data collection - Johns Hopkins Bloomberg School of](https://reader031.vdocuments.net/reader031/viewer/2022020913/6203496224f6b61e9c663da6/html5/thumbnails/2.jpg)
Survey Research: Designing an Instrument
Ann Skinner, MSWJohns Hopkins University
![Page 3: Methods of Data collection - Johns Hopkins Bloomberg School of](https://reader031.vdocuments.net/reader031/viewer/2022020913/6203496224f6b61e9c663da6/html5/thumbnails/3.jpg)
Section A
Choice of Method
![Page 4: Methods of Data collection - Johns Hopkins Bloomberg School of](https://reader031.vdocuments.net/reader031/viewer/2022020913/6203496224f6b61e9c663da6/html5/thumbnails/4.jpg)
4
Characteristics of a Survey
It applies systematic methodologyIt involves obtaining information directly from individualsIt involves selecting a subgroup from a larger groupIt is done for the purpose of . . .−
Description
−
Exploration−
Explanation
![Page 5: Methods of Data collection - Johns Hopkins Bloomberg School of](https://reader031.vdocuments.net/reader031/viewer/2022020913/6203496224f6b61e9c663da6/html5/thumbnails/5.jpg)
5
Tailored Survey Design
Sampling IssuesWhether to use probability sampling−
The sampling frame
−
Size of the sample−
Sampling design/strategy
−
Expected response rateSampling Issues
Question design issues−
Reliability and validity of items
−
Pre-testing or pilot workInterviewing issues−
Selection of interviewers
−
Supervision
![Page 6: Methods of Data collection - Johns Hopkins Bloomberg School of](https://reader031.vdocuments.net/reader031/viewer/2022020913/6203496224f6b61e9c663da6/html5/thumbnails/6.jpg)
6
Questionnaire Design
Quantitative surveys in health services research
![Page 7: Methods of Data collection - Johns Hopkins Bloomberg School of](https://reader031.vdocuments.net/reader031/viewer/2022020913/6203496224f6b61e9c663da6/html5/thumbnails/7.jpg)
7
Traditional Survey Model
Interviewer
A carefully standardizedphysical stimulus
(i.e., question)
Respondent
A response
(i.e., answer)expressed in terms of a standardized
format provided by the researcher
![Page 8: Methods of Data collection - Johns Hopkins Bloomberg School of](https://reader031.vdocuments.net/reader031/viewer/2022020913/6203496224f6b61e9c663da6/html5/thumbnails/8.jpg)
8
Symbolic Interactionist View
InterviewerEncodes question
Own purposesPerceptions of
respondent
RespondentDecodes question
Own purposesPerceptions of
Interviewer
InterviewerDecodes answer
Own presumptionsPerceptions of
respondent
RespondentEncodes answer
Own presumptionsPerceptions of
interviewer
![Page 9: Methods of Data collection - Johns Hopkins Bloomberg School of](https://reader031.vdocuments.net/reader031/viewer/2022020913/6203496224f6b61e9c663da6/html5/thumbnails/9.jpg)
9
Methods of Data Collection
Self-administered−
Individually
−
By mail−
In groups
−
Internet or emailInterviewer-administered−
By telephone
−
In person
![Page 10: Methods of Data collection - Johns Hopkins Bloomberg School of](https://reader031.vdocuments.net/reader031/viewer/2022020913/6203496224f6b61e9c663da6/html5/thumbnails/10.jpg)
10
Methods of Data Collection
CombinationsSelf-administered with interviewer instructionsMail with telephone follow-upInterviewer-administered with embedded self-administered section
![Page 11: Methods of Data collection - Johns Hopkins Bloomberg School of](https://reader031.vdocuments.net/reader031/viewer/2022020913/6203496224f6b61e9c663da6/html5/thumbnails/11.jpg)
11
Factors in Choice of Method
Characteristics of study population−
Literacy
−
Physical and mental abilities−
Motivation
![Page 12: Methods of Data collection - Johns Hopkins Bloomberg School of](https://reader031.vdocuments.net/reader031/viewer/2022020913/6203496224f6b61e9c663da6/html5/thumbnails/12.jpg)
12
Factors in Choice of Method
Access to sample−
Location
−
Time available for data collection−
Infrastructure available (telephones, mail service, internet access)
![Page 13: Methods of Data collection - Johns Hopkins Bloomberg School of](https://reader031.vdocuments.net/reader031/viewer/2022020913/6203496224f6b61e9c663da6/html5/thumbnails/13.jpg)
13
Factors in Choice of Method
Availability of information about study population−
Telephone numbers
−
Completeness of addresses−
Tracing information
![Page 14: Methods of Data collection - Johns Hopkins Bloomberg School of](https://reader031.vdocuments.net/reader031/viewer/2022020913/6203496224f6b61e9c663da6/html5/thumbnails/14.jpg)
14
Factors in Choice of Method
Survey objectives−
Complexity of questioning
−
Difficulty of reporting task−
Topic sensitivity
![Page 15: Methods of Data collection - Johns Hopkins Bloomberg School of](https://reader031.vdocuments.net/reader031/viewer/2022020913/6203496224f6b61e9c663da6/html5/thumbnails/15.jpg)
15
Factors in Choice of Method
Question forms to be used−
Open-ended
Fill in a numberWrite in text
−
Closed-endedMultiple choiceScalar (likert-type)Dichotomous (yes/no)
![Page 16: Methods of Data collection - Johns Hopkins Bloomberg School of](https://reader031.vdocuments.net/reader031/viewer/2022020913/6203496224f6b61e9c663da6/html5/thumbnails/16.jpg)
16
Factors in Choice of Method
Expected response ratesExample (assume a group with interest):−
Mail survey with no follow-up—30%
−
Mail survey with mail follow-up—50%−
Mail survey with telephone follow-up—60% to 80%
![Page 17: Methods of Data collection - Johns Hopkins Bloomberg School of](https://reader031.vdocuments.net/reader031/viewer/2022020913/6203496224f6b61e9c663da6/html5/thumbnails/17.jpg)
Section B
Guidelines for Writing Questions
![Page 18: Methods of Data collection - Johns Hopkins Bloomberg School of](https://reader031.vdocuments.net/reader031/viewer/2022020913/6203496224f6b61e9c663da6/html5/thumbnails/18.jpg)
18
Guidelines for Writing Questions
Restrain the impulse to write specific questions until you have thought through your research questionsWrite down your research questions and keep them handy when you are working on the questionnaireEvery time you write a question, ask yourself . . .−
“Why do I want to know this?”
−
“How will it help answer a research question?”
Notes Available
![Page 19: Methods of Data collection - Johns Hopkins Bloomberg School of](https://reader031.vdocuments.net/reader031/viewer/2022020913/6203496224f6b61e9c663da6/html5/thumbnails/19.jpg)
19
Open-Ended or Closed-Ended Questions?
Type of information sought−
Facts
−
Opinions or attitudes−
Exploratory
Complexity of the information or difficulty of the reporting task
![Page 20: Methods of Data collection - Johns Hopkins Bloomberg School of](https://reader031.vdocuments.net/reader031/viewer/2022020913/6203496224f6b61e9c663da6/html5/thumbnails/20.jpg)
20
Open-Ended or Closed-Ended Questions?
Feasibility−
Range of possible answers
−
Coding capabilities−
Sample size
−
Data collection method
![Page 21: Methods of Data collection - Johns Hopkins Bloomberg School of](https://reader031.vdocuments.net/reader031/viewer/2022020913/6203496224f6b61e9c663da6/html5/thumbnails/21.jpg)
21
Advantages of Open-Ended Items
Can get unanticipated answersMay describe the respondent’s real views betterRespondents can answer in their own wordsAppropriate when the range of possible answers is long
![Page 22: Methods of Data collection - Johns Hopkins Bloomberg School of](https://reader031.vdocuments.net/reader031/viewer/2022020913/6203496224f6b61e9c663da6/html5/thumbnails/22.jpg)
22
Advantages of Closed-Ended Items
Task of answering is easierInterpretation of the answer is easierAvoids rare answers
![Page 23: Methods of Data collection - Johns Hopkins Bloomberg School of](https://reader031.vdocuments.net/reader031/viewer/2022020913/6203496224f6b61e9c663da6/html5/thumbnails/23.jpg)
23
Scales, Indices, and Questionnaires
Seek out appropriate scales, indices, questionnaires that have been used and tested by othersExamples−
Health status scales
−
Quality of life−
Mental health status
−
Health services utilization−
Satisfaction ratings
![Page 24: Methods of Data collection - Johns Hopkins Bloomberg School of](https://reader031.vdocuments.net/reader031/viewer/2022020913/6203496224f6b61e9c663da6/html5/thumbnails/24.jpg)
24
Scales, Indices, and Questionnaires
Can combine several in one questionnaireBe careful about copyrighted instruments
![Page 25: Methods of Data collection - Johns Hopkins Bloomberg School of](https://reader031.vdocuments.net/reader031/viewer/2022020913/6203496224f6b61e9c663da6/html5/thumbnails/25.jpg)
25
Writing Questions
Fully scripted, so that as written, the question prepares the respondent to answerThe question means the same thing to every respondentThe respondent understands what an appropriate answer should be
![Page 26: Methods of Data collection - Johns Hopkins Bloomberg School of](https://reader031.vdocuments.net/reader031/viewer/2022020913/6203496224f6b61e9c663da6/html5/thumbnails/26.jpg)
26
Common Pitfalls
Fuzzy WordingVague and general questions produce vague and general answersBad—What do you like best about this neighborhood? We’re interested in anything, like houses, the people, the parks, or whatever
![Page 27: Methods of Data collection - Johns Hopkins Bloomberg School of](https://reader031.vdocuments.net/reader031/viewer/2022020913/6203496224f6b61e9c663da6/html5/thumbnails/27.jpg)
27
Poor Question Organization
Bad“I would like you to rate different features of your neighborhood as very good, good, fair, or poor. Please think carefully about each item as I read it. Public schools. Parks. Other.”
![Page 28: Methods of Data collection - Johns Hopkins Bloomberg School of](https://reader031.vdocuments.net/reader031/viewer/2022020913/6203496224f6b61e9c663da6/html5/thumbnails/28.jpg)
28
Poor Question Organization
Better“I am going to ask you to rate different features of your neighborhood. I want you to think carefully about your answers. How would you rate the public schools—would you say very good, good, fair, or poor? How would you rate the parks, would you say very good, good, fair or poor?”
![Page 29: Methods of Data collection - Johns Hopkins Bloomberg School of](https://reader031.vdocuments.net/reader031/viewer/2022020913/6203496224f6b61e9c663da6/html5/thumbnails/29.jpg)
29
Difficult Words
Risk miscommunication with respondentExample:−
“Do you think TV programs are
impartial about politics?”
![Page 30: Methods of Data collection - Johns Hopkins Bloomberg School of](https://reader031.vdocuments.net/reader031/viewer/2022020913/6203496224f6b61e9c663da6/html5/thumbnails/30.jpg)
30
Of 56 Respondents . . .
26—interpreted “impartial” correctly10—overlooked the word altogether9—thought it meant “tending to spend too much time on politics”5—thought it meant unfair or biased2—thought it meant giving too little time to politics7—had no idea
![Page 31: Methods of Data collection - Johns Hopkins Bloomberg School of](https://reader031.vdocuments.net/reader031/viewer/2022020913/6203496224f6b61e9c663da6/html5/thumbnails/31.jpg)
31
Negatives and Sneaky Double Negatives
Bad“What is your view about the statement that conservationists should not be so uncooperative with the government?”
![Page 32: Methods of Data collection - Johns Hopkins Bloomberg School of](https://reader031.vdocuments.net/reader031/viewer/2022020913/6203496224f6b61e9c663da6/html5/thumbnails/32.jpg)
32
Negatives and Sneaky Double Negatives
Better“What is your view about the statement that conservationists should be cooperative with the government?”
![Page 33: Methods of Data collection - Johns Hopkins Bloomberg School of](https://reader031.vdocuments.net/reader031/viewer/2022020913/6203496224f6b61e9c663da6/html5/thumbnails/33.jpg)
33
Asking Two or More Questions at Once
Bad”When riding in the back seat of a car, do you wear a seat belt all of the time, most of the time, some of the time, once in a while, or never?”
![Page 34: Methods of Data collection - Johns Hopkins Bloomberg School of](https://reader031.vdocuments.net/reader031/viewer/2022020913/6203496224f6b61e9c663da6/html5/thumbnails/34.jpg)
34
Backseat Belt Question
![Page 35: Methods of Data collection - Johns Hopkins Bloomberg School of](https://reader031.vdocuments.net/reader031/viewer/2022020913/6203496224f6b61e9c663da6/html5/thumbnails/35.jpg)
35
Asking Two or More Questions at Once
Better“In the past year, have you ridden in the back seat of a car?”“When you are riding in the back seat of a car, do you wear a seat belt all of the time, most of the time, some of the time, once in a while, or not at all?”
![Page 36: Methods of Data collection - Johns Hopkins Bloomberg School of](https://reader031.vdocuments.net/reader031/viewer/2022020913/6203496224f6b61e9c663da6/html5/thumbnails/36.jpg)
36
Backseat Belt Question
![Page 37: Methods of Data collection - Johns Hopkins Bloomberg School of](https://reader031.vdocuments.net/reader031/viewer/2022020913/6203496224f6b61e9c663da6/html5/thumbnails/37.jpg)
Section C
Questionnaire Construction and Critique
![Page 38: Methods of Data collection - Johns Hopkins Bloomberg School of](https://reader031.vdocuments.net/reader031/viewer/2022020913/6203496224f6b61e9c663da6/html5/thumbnails/38.jpg)
38
Putting the Questionnaire Together
Item OrderSelf-administration—capture respondent’s interestInterviewer—allow for practice with question/response format (information interviewer needs to know)
![Page 39: Methods of Data collection - Johns Hopkins Bloomberg School of](https://reader031.vdocuments.net/reader031/viewer/2022020913/6203496224f6b61e9c663da6/html5/thumbnails/39.jpg)
39
Putting the Questionnaire Together
Ideal LengthSelf-administration—15–20 minutesInterviewer—20–30 minutes
TestingRead it aloud to yourselfRead it aloud to co-workers, friendsAsk two to three others to fill it out themselves
![Page 40: Methods of Data collection - Johns Hopkins Bloomberg School of](https://reader031.vdocuments.net/reader031/viewer/2022020913/6203496224f6b61e9c663da6/html5/thumbnails/40.jpg)
40
Systematic Pilot Testing
Small sample (10–15)Similar to study populationUse all study proceduresDiscussion of problem questions
![Page 41: Methods of Data collection - Johns Hopkins Bloomberg School of](https://reader031.vdocuments.net/reader031/viewer/2022020913/6203496224f6b61e9c663da6/html5/thumbnails/41.jpg)
41
Standard 10-Step Critique
Use simple unambiguous language that can be understood in the same way by all respondentsAvoid long and complex sentencesAvoid hypothetical questionsAvoid “double-barreled” questions (asking two questions at once and questions that include assumptions)
Notes Available
![Page 42: Methods of Data collection - Johns Hopkins Bloomberg School of](https://reader031.vdocuments.net/reader031/viewer/2022020913/6203496224f6b61e9c663da6/html5/thumbnails/42.jpg)
42
Standard 10-Step Critique
Do not ask questions that ask respondents for information they do not haveAvoid questions that ask about causalityThe time frame referred to in the question should be unambiguous and explicit
![Page 43: Methods of Data collection - Johns Hopkins Bloomberg School of](https://reader031.vdocuments.net/reader031/viewer/2022020913/6203496224f6b61e9c663da6/html5/thumbnails/43.jpg)
43
Standard 10-Step Critique
For fixed response questions, response categories must be exhaustive and mutually exclusiveMake sure the context of the question does not inappropriately affect its meaningDefine terms as needed (complex definitions and instructions should be given in a preamble or introduction, not in the question itself)