bua 500: introduction to research methods research design and methodology
TRANSCRIPT
BUA 500: Introduction to Research Methods
Research design and methodology
Class will cover…
Aspects and issues in designing a research project
Issues and techniques in library research
Introductory issues in statistics and statistical methodologies
Survey Process
Information gathering and needs assessment
Sampling design
Instrumentation
Data Collection
Data Processing
Report Generation
Information gathering and needs assessment
Specify the information needsEstimate the information valueAssess internal resource availabilityChoose the data collection methodSelect the data processing methodMeasure the scope of the response taskDetermine a sample size and designSpecify the instrumentation and costsDescribe the type of reports requiredSummarize final costs and the timetable
Alternative Data Collection Methods
Mail Survey
Telephone Survey
Personal Interview
Experiments
Comparison of Data Collection Methods
Personal Telephone Mail
Data collection costs: High Medium Low
Time required: Medium Low High
Sample size needed: Small Medium Large
Data quantity: High Medium Low
Dispersed sample: No Maybe Yes
Reaches special locations: Yes Maybe No
Interactions with respondents: Yes Yes No
Degree of interview bias: High Medium None
Severity of non-response bias: Low Low High
Presentation of visual stimuli: Yes No Maybe
Field worker training required: Yes Yes No
Interviews versus Mail surveys
Does the task require interaction with the respondents?Is there a possibility of interaction respondent bias?Are there accurate mailing databases available?Must the data be collected at a specific location?How involved is the data recording process?Large amount of data from limited subjects or small amount of data from large pool of subjects?How widely dispersed are the subjects?Would there be embarrassment or threatening issues in face to face communication?
Telephone versus In-Person
Specific location required?Is there a “show and tell” component to the interviewing process?Are you using some type of quota based survey? E.g. is there some type of screening process required?Would appearances self-select some types of respondents undesirably?Is intimidation of any issue?Will the interview take a long time?Are there substitutive database to pull from?How rapid of a turnaround time is needed?
Factors determining sampling strategy—indications of large sample, small response task
Very precise numerical estimates are neededRange of volume of information is limitedLeaning toward a mail data collection approachOverall survey of information is of more interest than in-depth patterns by respondents
Controlling sampling error and bias
Any sample less than the population will have sampling errorSampling error is random error affecting both validity and reliabilityThe greater the sampling error, the lower the reliability and validityThe less the variation among individual members of the population, the small the sampling errorSurvey data cannot be more valid than it is reliable, because reliability limits validitySurvey data can be high reliable, but still be low in validity because of systematic biasLike sampling error, systematic bias can never be eliminated completelySystematic bias can best be controlled by identifying the things that are most likely to exert bias and eliminating them or protecting from them in the sampling design
Instrumentation bias issues
Does the question state the criterion for answering?Is the question applicable to all respondents?Does the item contain an example that is also a possible answer?Does the question require respondents to remember too much detail or recall distant events?Is the question as specific as it can reasonably be?Does the question overemphasize some condition?Are some of the words in the item ambiguous?Is the question as free from threat to respondents as possible?IMPORTANT: Does the question only include one issue?Will yea or nay sayers always chose one answer?Does the question lead respondents toward a particular answer?Is the question “loaded” with a reason for responding in a particular way?
Sources of response bias
Social desirability
Acquiescence
Yea or nay-saying tendency
Prestige
Threat
Hostility
Mental sets
Order
Avoiding sample selection bias
Are some sampling units more visible or accessible than others?Is there any systematic order to the sampling units?Are units clustered together?Will field workers have more affinity for some units over others?Is there a self-selection component to be concerned with?Will there be a large number of non-respondents?
Factors indicating a large sample
Decision is very serious and/or costlyVery high level of confidence neededHigh level of variance within the populationSmall sub samples are needed during analysis and interpretationCosts vary only slightly to sample size selectionTime and resources are readily available
Special sampling designs
Stratified or proportional sampling Divides population into two or more segments and
sample a different proportion of each E.g. opinions on a topic with respect to gender, race,
religious beliefs, party affiliation, etc
Cluster sampling Sample cluster of respondents, e.g. random city blocks
Quota sampling Sample a population and meet quota’s based on some
strata or delineation condition
Guide list for measuring Attitudes
Be sure to include all three components of the attitude: knowledge, feelings, and action tendencies, in that orderUse unaided recall to measure awareness, if possible, to avoid false reports of recognitionUse ratings scales to measure feelings, so that both direction and distance from neutral are revealedDo not ignore the intensity of feelings or assume intensity is the same as distance from neutral
Focus issues on question design
Wrong: Which brand do you like the best?Right: Which of these brands are you most likely to purchase?Wrong: When do you usually go to work?Right: What time do you ordinarily leave home for work?Wrong: Are you going to vote Democrat or Republican?Right: Which candidate will you vote for on election day?
Brevity of the question
Wrong: Can you tell me how many children you have, whether they are girls or boys, and how old they are?Right: What is the age and sex of each of your children?Wrong: If you own one or more automobiles, please list the year and the make of each one, starting with the newest one?Right: Please list the year and make or each car you own?Wrong: When was the last time that you went to the doctor for a physical examination on your own or because you had to?Right: How many months ago was your last physical examination?
Clarity of the question
Wrong: What do you have to say about the charities that your church contributes to?Right: How much influence do you, yourself have on which charities your church contributes to?Wrong: About how much of the storage space in your home do you and your spouse use?Right: What proportion of the storage space in your home is used for your things and what is used for your spouses?Wrong: Ordinarily, do you take aspirin when you feel some discomfort or when you feel actual pain?Right: Do you usually take aspirin as soon as you feel some discomfort, or only when you feel actual pain?
Question construction guidelines
Does the question focus directly on the issue or topic to be measured?
Is the question stated as briefly as it can be?
Is the question expressed as clearly and simply as it can be?
Use of vocabulary
Wrong: Are you cognizant of all the concepts to be elucidated?Right: Do you know about all the ideas that will be explained?Wrong: With what frequency have you experienced the of late?Right: How many times have you had this happen lately?Wrong: What emotions were evoked by perceiving the spectacle?Right: What kinds of feelings did you have when you saw it?
Use of compound sentences
Wrong: What would you do when you had only a few things to buy and there were a lot of people in the checkout line?Right: Suppose you have only a few things to buy. There are a lot of people in the checkout line. What would you do?Wrong: How do you work it out when you want on thing and your spouse wants another and you both feel very strongly about it?Right: How do you settle disagreements with your spouse when you both have strong feelings about it?Wrong: If you didn’t have a reservation ahead of time and you found out that the only seats that you could get were at the very top of the upper balcony, what would you do?Right: What would you do if the only seats available at show time were on the top of the upper balcony?
Expressing questions correctly
Use only core vocabulary; the words and phrases that one would use in casual speech.
Limit the vocabulary so that the least sophisticated respondent would be familiar with the words.
Use simple sentences where possible, and complex sentences only when they are actually required.
Use two or more short, simple sentences rather than one compound or compound-complex sentence.
Change long, potentially confusing clauses in sentences to words or short phrases where possible.
Unstated criteria issues
Wrong: How important is it for stores to carry a large number of different brands of this product?
Right: How important is it to you that the store you shop at carries a large number of different brands?
Inapplicable question
Wrong: how long does it take for you to find a parking place after you arrive at the plant?
Right: If you drive to work, how long does it take you to find a parking place after you arrive at the plant?
Examples in questions
Wrong: What small appliances, such as counter-top appliances, have you purchased in the past month?
Right: Aside from major appliances, what other smaller appliances have you bought in the past month?
Overdemanding recall
Wrong: How many times did you go out on a date with your spouse before you were married?
Right: How many months were you “dating” your spouse before you were married?
Overgeneralizations
Wrong: When you buy “fast food”, what percentage of the time do you order each of the following types of foods?
Right: Of the last 10 times you bought “fast food”, how many times did you each each type of food?
Over specificity
Wrong: When you visited the museum, how many times did you read the plaques that explain what the exhibit contains?
Right: When you visited the museum, how often did you read the plaques that explain what the exhibit contained? Would you say often, sometimes, rarely, or never?
Overemphasis
Wrong: Would you favor increasing taxes to cope with the current fiscal crisis?
Right: Would you favor increasing taxes to cope with the current fiscal problem?
Ambiguous wording
Wrong: About what time do you ordinarily eat dinner?
Right: About what time do you ordinarily dine in the evening?
Double-barreled questions
Wrong: Do you regularly take vitamins to avoid getting sick?
Right: Do you regularly take vitamins? Why are why not?
Leading questions
Wrong: Don’t you see some danger in the new policy?
Right: Do you see any danger in the new policy?
Loaded questions
Wrong: Do you advocate a lower speed limit to save human lives?
Right: Does traffic safety require a lower speed limit?
Types of question scales
Likert scale
Should be used to decipher differences on a number of itemsShould be able to compose statements that are opinionsMust be reasonably certain that the majority of respondents will not pick only the neutral categoryIf a summated score is to be computed, about half the items should be inclined to the pro side and half to the negative side to avoid yea and nay sayer bias.
Please pick a number from the scale to show how much you agree or disagree with each statement and jot it to the right of the item.
Scale 1=Strongly agree 2= Agree 3 = Neutral 4 = Disagree 5 = Strongly disagree
A man should never cry in public………………………………____
Higher education is more important for women than men……………………….______
Women should receive equal pay for equal work…………………………….._____
A man should not resent a women supervising his work…………………_____
A women’s place is in the home?…………………………..____
The husband should make the majority of the family decisions?………………______
Women should never put career before family?………………………….____
A man should help and protect a women in public……………………._____
Verbal frequency scale
Used for several items, not just one or twoCompose items that assume the opportunity to perform the actionShould be used only when approximate percentages are requiredMost appropriate when respondents are unable or unwilling to compute exact percentages
Please pick a number from the scale to show how often you do each of the things listed below and jot in the space to the right of the item.
Scale 1=Always 2=Often 3=Sometimes 4=Seldom 5=Never
Seek out information about candidates and issues……………………_____Actually vote during strictly local elections………………………_____Actually vote during a state or national election……………._____Vote along strict party lines………………………._____
Ordinal scale
Used when a direct question concerning a quantity or value would not be sufficient
The researcher should be aware of the statistical limitations of ordinal versus interval or ratio data
Response alternative must be positioned in a meaningful sequence
Important: the interval between categories must be of little or no interest
Ordinarily, when do you or someone in your family first turn on a television set in your home on a weekday? (Please check one)
___The first thing in the morning___ A little while after awakening___ Mid-morning___ Just before lunch___ Right after lunch___ Mid-afternoon___ Early evening, before dinner___ Right after dinner___ Late evening___ Usually don’t turn it on
Forced ranking scale
Number of things to be ranked should be less than 10The major focus should be on the relative standing of the itemsThe researcher must be willing to forego measurement of the distance between ranksA single judgment criterion must be clearly statedAs with other scales yielding ordinal data, analysis is confined to a limited set of statistical procedures that do not require equal intervals
Please rank the colas listed below in their order of preference. Jot the number 1 next to the one you prefer the most, number 2 by your second choice, and so forth.
___ Pepsi-Cola
___ Coca-Cola
___ Royal Crown Cola
___ Diet Rite Cola
Paired comparison scale
Number of things to compare should be < 10
The method is most effective when actual choices in the real situation are always between two things
The researcher must be willing to forego measurement of the ‘distance’ between items in each pair
A single judgment criterion must be clearly stated
For each pair of soft drinks listed below, please put a check mark by the one you most prefer, if you had to choose between the two
1 ___ Pepsi-Cola ___ Coca-Cola
2 ___ Royal Crown Cola ___ Pepsi-Cola
3 ___ Royal Crown Cola ___ Diet Rite Cola
4 ___ Royal Crown Cola ___ Coca-Cola
Comparative scale
The major emphasis should be the comparison of a single, standard entity with one or more othersThe actual rating scale should have an even number of alternatives, if ‘fence-riding’ is likelyThe researcher must be sure the respondent is very clear about which is the standard and which is to be ratedThis method is advisable only if all or nearly all respondents are very familiar with the standardThe methods is particularly applicable when interval data are desired, but a relative measure is required. BOTH RANKING and DISTANCE between entities are generated.
Compared to the previous management group, the new one is … (check one space)
Very Superior About the Same Very Inferior
_____ _____ ______ _____ _____
1 2 3 4 5
Horizontal numeric scale
The method is most applicable when evaluative responses are to be arrayed on a single dimensionThe scale is most economical where several items are all to be rated on the same dimensionScale extremes should be labeled, “Extremely _____” to define the dimension and the words used must be bipolar oppositesIn the vast majority of cases, the intermediate scale values should not be labeled with words, and only numbers should be used, spaced at equal intervals
How important to you is each of the public issues listed below
Extremely Extremely
Unimportant Important
1 2 3 4 5
The protection of endangered species of animals………………………………..___
The improvement of the quality of the air…………………………………………..___
The discovery of additional petroleum reserves…………………………………..___
The development of “renewable” sources of energy…………………………….___
The reduction or elimination of water pollution……………………………………___
The development of additional nuclear power…………………………………….___
The provision of social services to those in need…………………………………___
Semantic differential scale
This method is most effective for measuring image profile, people, things, organizations, or conceptsAdjectives must define a single dimension and each pair must be bi-polar opposites labeling the extremesPrecisely what the respondent is to rate must be clearly stated in the introductory instructionsNo more than about 20 items should be used, and about half should begin with a positive word
Please put a check mark in the space on each line below to show your opinion of the pizza served here
Hot ___ : ___ : ___ : ___ : ___ : ___ : ___ Cold
1 2 3 4 5 6 7
Bland ___ : ___ : ___ : ___ : ___ : ___ : ___ Spicy
1 2 3 4 5 6 7
Expensive ___ : ___ : ___ : ___ : ___ : ___ : ___ Inexpensive
1 2 3 4 5 6 7
Small ___ : ___ : ___ : ___ : ___ : ___ : ___ Large
1 2 3 4 5 6 7
Good ___ : ___ : ___ : ___ : ___ : ___ : ___ Bad
1 2 3 4 5 6 7
Adjective check list
Method is appropriate for measuring imagesInstructions and response task are quick and simple, and a large number of adjectives could be applicablePrecisely what the respondent is to rate or judge must be stated very clearly in the insturctionsThe scale yields a profile, but only in terms of discrete, nominal data indicating a dichotomous choice
Please put a check mark in the space in front of any word or phrase that describes your job
___ Easy ___ Safe
___ Technical ___ Exhausting
___ Boring ___ Difficult
___ Interesting ___ Rewarding
___ Low-Paying ___ Secure
___ Important ___ Degrading
Stapel scale
This method is appropriate for measuring images
A relatively large number of words or phrases can be inlcuded
Precisely what the respondent is to judge and rate must be stated very clearly in the instructions
The scale yields a profile indicating “how much” each phrase describes the topic, with continuous data
Please pick a number from the scale to show how well each word or phrase below describes your job and jot it in the space in from of each item
Not all all 1 2 3 4 5 6 7 Prefectly
___ Easy ___ Safe
___ Technical ___ Exhausting
___ Boring ___ Difficult
___ Interesting ___ Rewarding
___ Low-Paying ___ Secure
___ Important ___ Degrading
Fixed sum scale
The scale is used to measure proportions, rather than absolute valuesThe occurrences must be apportioned into not more than about ten categoriesThe instructions must state very clearly the value to which the responses must totalThe scale provides continuous data so that proportions can easily be compared among alternativesThe data can be converted to percentages by dividing each value by the actual total, so every case will total a hundred percent and be comparable
Of the last 10 times that you ate lunch or dinner at a casual or fast food restaurant, how many times did you have each of the things listed below? (please be sure to make the total 10)
___ Hamburgers___ Hot dogs or sausage___ Chicken___ Pizza___ Chinese food___ Fish or seafood___ Deli sandwiches___ Hot sandwiches___ Mexican food___ other (What?___________________)Total = 10
Scale Creation Criteria
Keep it simpleRespect the respondentDimension the responsePick the denominationsDecide on the rangeGroup only when requiredHandle neutrality carefullyState instructions carefullyAlways be flexiblePilot test the scales
Ethical issues in research
Honesty
Don’t over consume one’s time
Respect confidentiality
Integrity
Be aware to possible ramifications
Basic person-to-person consideration