survey research uapp702: research design for urban & public policy class notes
Post on 23-Feb-2016
46 Views
Preview:
DESCRIPTION
TRANSCRIPT
1
Survey ResearchUAPP702: Research Design for Urban & Public PolicyClass Notes
Based on Earl Babbie, The Practice of Social Research
Danilo YanichCenter for Community Research & ServiceSchool of Public Policy & AdministrationUniversity of Delaware
2
Topics appropriate for survey research
Used for exploratory, descriptive & explanatory purposes
Probably best method to collect original data for describing a population too large to observe directly
3
Guidelines for Asking Questions Choose appropriate question form: two options
Open-ended--R is asked to provide own answer to question
Closed-ended--R is asked to select an answer from among a list provided by researcher
Closed-ended questions require the categories of answers to be exhaustive and mutually exclusive
4
Guidelines for Asking Questions, p.2
Make items clear
Babbie uses examples of "proposed peace plan"...Which peace plan?
And, employment "last week"...Monday through Friday or Sunday through Saturday as the Census Bureau defines week?
5
Guidelines for Asking Questions, p.3
Avoid double-barreled questions
Example: "Did you walk to work or carry your lunch?" …..Yes or No
Regardless of the answer you get from the respondent, you don’t know what information you have.
6
Double-barreled to the max Question on 1986 Harris poll re: American
public opinion about Libya
If Libya now increases terrorist acts against the U.S. and we keep inflicting more damage on Libya, then inevitably it will all end in the U.S. going to war and finally invading that country which would be wrong.
Respondents were given opportunity to answer “Agree”, “Disagree” or “Not sure”
7
Double-barreled to the max, p.2
Notice the elements contained in the complex statement
Will Libya increase its terrorist acts against the U.S.? Will the U.S. inflict more damage on Libya? Will the U.S. inevitably or otherwise go to war with Libya? Would the U.S. invade Libya? Would that be right or wrong?
These elements offer the possibility of numerous points of view—far more than the three alternatives offered to the R’s
8
Guidelines for Asking Questions, p.4
Respondents must be competent to answer Babbie uses example of asking students to indicate the percentage of fees to be
used for a long list of activities, about which the respondents have little knowledge
Respondents must be willing to answer There may be a risk to the respondent in answering the question
Reluctance may be due to nature of the information sought, but good science and human subjects requirements dictate caution.
Example: asking a staff member of a non-profit agency her/his opinion about the leadership qualities of the administrator
Using a technique that guarantees anonymity or confidentiality greatly increases R’s willingness to answer questions
9
Guidelines for Asking Questions, p.5 Questions should be relevant to the Respondent
Babbie uses example of fictitious person, Tom Sakumoto
9% of R's said they were familiar with him. Shows R’s tendency to be helpful to researcher
Short items are best
Avoid negative items Possibility of misinterpretation is great
Babbie uses example of asking R to say if they agree/disagree with statement that "The U.S. should not recognize Cuba".
Often, R’s will “read over” the word “NOT”
10
Guidelines for Asking Questions, p.6
Avoid biased items and terms
Questions should not encourage a particular response or discourage another.
Be wary of "social desirability" of answers
Attitudes re: government spending for programs in the General Social Survey, 1989
More support Less support
Assistance to the poor Welfare reform
Halting rising crime rate Law enforcement
Dealing with drug addiction Drug rehabilitation
Solving problems of big cities Assistance to big cities
Improving conditions of blacks Assistance to blacks
Protecting social security Social security
Surveys “gone wild” National Park Survey
11
NRA Survey 1
NRA Survey 2
NRA Survey 3
12
Questionnaire Construction General questionnaire format should be
uncluttered
Squeezed-together questionnaires are disastrous
Formats for respondents should be clean and clear
13
Questionnaire Construction, p.2
Contingency question
A question that only applies to the R’s who answered a previous question in a particular “qualifying” manner
Make sure that the directions for answering the subsequent questions are clear for the respondent
14
Contingency question, example
Have you ever been abducted by aliens? [ ] Yes [ ] No (go to ques. # 4)
If yes: Did they let you steer the ship?[ ] Yes[ ] No
If yes: How fast did you go?[ ] Warp speed[ ] Weenie speed
15
Questionnaire Construction, p.3
Matrix questions
Advantages Uses space efficiently, R's will probably complete it faster
Format may increase comparability of responses to different questions
Disadvantages May encourage researcher to use matrix format when
another form would be better
Can foster a response-set among R's in which they may develop a pattern
16
Matrix question, example Contact with criminal justice system
Have you ever…Yes No Ref/DK
Been a defendant in a criminal case 1 2 7
Been a witness in a criminal case 1 2 7
Reported a crime to the police 1 2 7
Been the victim of a crime 1 2 7
17
Matrix question, example 2
Using the A, B, C, D, F grading system that we learned in school, please rate the importance of the following services in this hotel.
Hotel service A B C D F Ref/DK
Dining services 5 4 3 2 1 7
Laundry service 5 4 3 2 1 7
Room service 5 4 3 2 1 7
18
Questionnaire Construction, p.4
Ordering questions in a questionnaire
The appearance of one question can affect answers given to later ones
As remedy, do not make the questions random, that will look chaotic to the R and confuse the issue
Safest solution is sensitivity to the problem
19
Questionnaire Construction, p.5
Questionnaire Instructions
Must be clear, concise and COMPLETE
Pre-testing the questionnaire
ALWAYS PRE-TEST!!!!
20
Comparing different survey methodsChoosing among mail questionnaire, personal interview and telephone survey
Factors influencing coverage and information gathered
Mailed questionnaire
Personal interview
Telephone survey
Lowest relative cost 1 3 2
Highest % return 3 1 2
Highest accuracy of information 2 1 3
Largest sample coverage 3 1 3
Completeness, including sensitive material 3 1 2
Overall reliability & validity 2 1 3
Time required to gather information 3 2 1
Ease of gathering information 1 3 2
Total number of rankings—1,2,3 2,2,4 5,1,2 1,5,1
Note: 1=most favorable ranking, 2=intermediate ranking, 3=least favorable rankingSource: Delbert Miller. Handbook of Social Research Design and Measurement, 5th Edition,1991, p. 168
21
Strengths of survey research Useful in describing large populations
Are flexible in that you can ask many questions regarding your topic
Standardized questionnaires have strength regarding measurement generally
Survey researcher must ask the same question of all subjects and…
Impute the same intent to all respondents giving a particular response
22
Weaknesses of survey research Standardized questionnaire items often
represent the least common denominator in assessing people’s attitudes, etc.
Does not deal well with the context of social life
Can be inflexible…cannot change the survey instrument if field conditions change
23
Weaknesses of survey research, p.2
Subject to artificiality
A person giving a conservative answer to a questionnaire does not necessarily mean the person is conservative
Artificiality has two aspects
Topic of study may not be amenable to measurement through a questionnaire
Studying the topic may affect it…
Asking someone whether they think the governor should be impeached when R may have given no thought to it until asked for the opinion
24
Ch 14: Quantifying data , Coding System to translate the responses to questions to
numeric expressions that the computer can process
Two basic approaches to coding process
Begin with developed coding scheme Babbie uses occupation as example...but must use scheme
that is appropriate for research question.
Generate codes from data that you collected.
25
Using a developed coding scheme, example
Occupation
The question to the R would be: What is your occupation?
The R’s response is then placed in one of several categories that you have already identified
26
Using a developed coding scheme, example, p.2
What is your occupation?
Let’s say that the answer was “nurse”
Scheme 1: by type Professional, managerial, clerical, semi-skilled, etc.
In this scheme, the R’s occupation would be placed in the “professional” category
Scheme 2: by sector of the economy Manufacturing, health, education, commerce, etc.
In this scheme, the R’s occupation would be placed in the “health” category
27
Using a developed coding scheme, example, p.3
Note: record the response of the R verbatim
Place the response into one of your pre-determined categories during the data manipulation phase
Use “recode” facility in SPSS
Remember, you can always “aggregate” data
Cannot “disaggregate” data
28
Using a developed coding scheme, example, p.4
What is your occupation?
Verbatim responses
R1: “nurse” R2: “sell shoes” R3: “build cars” R4: “manager at Wendy’s” R5: “physician” R6: “computer programmer” R7: “retired” R8: “soldier”
Type Sector
professional health
sales commercial
semi-skilled labor commercial
managerial commercial
professional health
professional information tech
retired retired
military military
29
Generating codes from data collected, example
Student responses to “biggest problem facing college today”
Responses can be coded as “academic” or “non-academic”Academic Non-academic
Tuition is too high x
Not enough parking spaces x
Faculty don’t know what they’re doing x
Advisors are never available x
Not enough classes offered x
Cockroaches in the dorms x
Too many requirements x
Cafeteria food is infected x
Books cost too much x
Not enough financial aid x
30
Codebook construction End product of the coding process is the conversion of data
items into numerical codes
These codes represent attributes composing variables which....
Are assigned locations within a data file
Location means the specific column of the data file where, for example, the responses for “gender” would occur
A codebook is a document that describes the locations and lists the assignment of codes to the attributes composing those variables
It is the fundamental document of the research process
31
Codebook serves two functions
Primary guide for the coding process
Guide to locating variables in the data file during analysis
32
Example of Coding InstructionsPublic Attitudes: Crime, Drugs & Public ServicesEnterprise Community/Wilmington/Statewide Survey
Variable Name (column location) Value label
1. ID# (1-3) Continuous (001 through n)
2. Area (4)1= Westside, 2= West Center, 3= Delaware Avenue4= Southwest, 5= Eastside, 6= Northeast7= Northwest, 8= New Castle, 9= Kent, 10= Sussex
3. Defendant in criminal case (5) 1=Yes, 2=No, 7=Ref/DK, 8=NA
4. Witness in a criminal case (6) 1=Yes, 2=No, 7=Ref/DK, 8=NA
5. Report crime to police (7) 1=Yes, 2=No, 7=Ref/DK, 8=NA
6. Victim of crime (8) 1=Yes, 2=No, 7=Ref/DK, 8=NA
33
Data Cleaning...a fundamental activity The process of detecting and correcting coding errors
Two types
Possible-code cleaning--for any given variable there are only a specified set of codes possible
Example, gender--computer program would "beep" when an erroneous code is entered and refuse the code
Family Court experience with gender codes
Contingency Cleaning--process of checking only those cases that should have data on a particular variable do in fact have such data
top related