Sampling

Polls predicting 1992 U.S. Polls predicting 1992 U.S. presidential election outcomespresidential election outcomesDate Agency Clinton Bush Perot

10/25 Gallup/CNN 44 38 1810/26 ABC 43 36 2110/27 NBC/Wall St. Journal 46 38 1610/28 Gallup/CNN 43 40 1710/31 CBS/New York Times 45 37 1811/1 Gallup/CNN 47 38 1511/2 CBS/New York Times 46 38 1511/2 NBC/Wall St. Journal 46 38 1611/2 ABC 45 38 1611/3 Election Results 43 38 19

Polls predicting 1996 U.S. Polls predicting 1996 U.S. presidential election outcomespresidential election outcomes

Dates Agency Clinton Dole Perot Other

10/28-31 Hotline 49 40 9 210/30-11/2 CBS/New York Times 54 35 9 210/31-11/3 Pew Research Center 52 38 8 2

11/1-3 Euter/Zogby 49 41 8 211/1-3 Harris 51 39 9 111/2-3 ABC 52 39 7 211/2-3 NBC/Wall St. Journal 51 38 9 211/3-4 Gallup/CNN/USA Today 51 38 9 2

Election Results 49 41 9 2

How many interviews it took to How many interviews it took to estimate the behaviors of 90 estimate the behaviors of 90

million voters? million voters?

• Less than 2,000

The History of Sampling The History of Sampling • In 1920, Literary Digest mailed postcards

to people in 6 states, asking whom they were planning to vote for in the presidential campaign.

• The Digest correctly predicted that Harding would be elected.

• In the elections that followed, the Literary Digest expanded the size of its poll and made correct predictions in 1924, 1928, 1932.

The History of Sampling The History of Sampling • In 1936, Literary Digest conducted its most

ambitious poll: 10 million ballots were sent to people listed in the telephone directories and on lists of automobile owners.

• Over 2 million responded, given the Republican contender Alf London, a 57 to 43 percent landslide over the incumbent, president Roosevelt.

• Election results: Roosevelt won 61% of the votes.

The History of Sampling The History of Sampling • Problem: 22% return rate. • A part of the answer to these questions lay in the

sampling frame used by the Digest: telephone subscribers and automobile owners.

• Such a design selected a disproportionately wealthy sample.

• The sample effectively excluded poor people, and the poor people predominantly voted for Roosevelt’s New Deal recovery program during the depression period.

The History of Sampling The History of Sampling • In the same year (1936), George Gallup correctly

predicted that Roosevelt would beat London. • Gallup’s success in 1936 hinged on his use of quota

sampling, which is based on a knowledge of the characteristics of the population being sampled. People are selected to match the population characteristics.

• Using quota sampling, Gallup successfully predicting the presidential winner in 1940 and 1944.

The History of Sampling The History of Sampling • In 1948, Gallup mistakenly picked Thomas Dewey ov

er incumbent president Harry Truman. • Factors accounted for 1948’s failure: • 1). Most of the pollsters stopped polling in early Oct d

espite a steady trend toward Truman during the campaign.

• 2). “Undecided” voters went disproportionately for Truman.

• 3). Unrepresentativeness of the sample (resulting from quota sampling).

The History of Sampling The History of Sampling • Quota sampling technique requires that the

researcher know something about the total population.

• For national political polls, such information came primarily from census data.

• By 1948, however, WWII had produced a massive movement from country to city, radically changing the character of the U.S. population, and Gallup relied on 1940 census data (City dwellers tended to vote Democratic; hence the over-representation of rural voters also underestimated the number of Democratic votes).

Population and Sample element Population and Sample element • Element: An element is that unit about which

information is collected and that provides the basis of analysis. – People, families, corporations – usually the same as unit of analysis

• Population: The entire group of individuals that we want information about is called the population. A population is the theoretically specified aggregation of study elements.

• A sample is a part of the population that we actually examine in order to gather information.

Defining the target population目標母體

It is vitally important to carefully define the target population so the proper source from which the data are to be collected can be identified.

Question: "To whom do we want to talk?" What or who will be observed?---answer the questions about the tangible characteristics of the population (1) definition of the element (2) time referent for the study.

• EX> “ 有生育能力的婦女” Or “female between age 12-50”?.

• EX> 台灣地區成年人口• EX> 大學生：二專、三專﹖夜間部﹖空中大學﹖研究生

Defining the study population研究母體

• Study Population: A study population is that aggregation of elements from which the sample is actually selected.

• Lists of elements are usually somewhat incomplete• 全國性的調查常忽略「金門」「馬祖」• 「社會學教授」 = 在社會系教書的教授

Sampling units 抽樣單位• A sampling unit is that element or set of elements consid

ered for selection in some stage of sampling.• In a simple single-stage sample, the sampling units are th

e same as the elements and are probably the units of analysis.

• EX> passengers on a passengers list sampling unit = elements

• In a multi-stage sample: • EX> the airlines could first select flights as the sampling

unit, then select certain passengers on the previously select flights.

• PSU (primary sampling units) = flights• Secondary sampling units = passengers

Observation unit

• An observation unit, or unit of data collection, is an element or aggregation of elements from which information is collected.

• EX) Researcher may interview heads of households (the observation units) to collect information about all members of the households (the units of analysis).

Sampling DesignSample designs

Nonprobability samplesVoluntary Response SampleConvenience JudgmentQuotaSnowball

Probability samplesSimple randomSystematicStratified ProportionateDisproportionateClusterMultistage

There are no appropriate statistical techniques for measuring random sampling error from a non-probability sample. Thus projecting the data beyond the sample is statistical inappropriate.

There are no appropriate statistical techniques for measuring random sampling error from a non-probability sample. Thus projecting the data beyond the sample is statistical inappropriate.

Nonprobability SamplingNonprobability Sampling

• Social research is often conducted in situations where you can't select the kinds of probability samples used in large-scale social surveys.

• Lack of population list: Suppose you wanted to study homelessness: There is no list of all homeless individuals, nor are you likely to create such a list.

Voluntary Response Sample

• A voluntary response sample consists of people who choose themselves by responding to a general appeal. Voluntary response samples are biased because people with strong opinions, especially negative opinions, are most likely to respond.

• radio station call in to reflect public opinions.

Convenience SamplingConvenience Sampling• 便利抽樣 (haphazard or accidental sampling), re

lying on available subjects

• EX> man-on-the-street interviews, talk to friend about their political sentiment

• EX> professor uses students as sample• EX> every tenth student entering the universit

y library.• EX> Survey “over sea Chinese for internation

al marketing?”

Convenience SamplingConvenience Sampling

• Advantages: Very low cost, extensively used, No need for list of population.

• It is justified only if the researcher wants to study the characteristics of people passing the sampling point at specified times or if less risky sampling methods are not feasible.

Convenience SamplingConvenience Sampling

• Problems: • (1) no way of knowing if those included are

representative. • (2) Variability and bias of estimates cannot be

measured or controlled. • (3) Projecting the results beyond the specific sample is

inappropriate. • Should be use only for exploratory design to generate

ideas and insights. • you should alert readers to the risks associated with

this method.

Judgment Samples (Purposive Samples) 判斷抽樣• hand-picked sample elements, believed to be represe

ntative of the population of interest• EX> a fashion manufacturer regularly selects a samp

le of key accounts that it believes are capable of providing the information to predict what will sell in the fall.

• EX> Dow Jones industrial average: select 30 blue-chip stocks out of 1,800 stocks. Highly correlated with other NYSE indicators on the daily percentages of price changes

• EX>Representative communities in U.S. presidential election.

• EX> CPI— 產品項目的選擇。

Snowball sampleSnowball sample 雪球抽樣雪球抽樣

• Locate an initial set of respondents. These individual are then used as informants to identify others with the desired characteristics.

• Appropriate when the members of a special population are difficult to locate. 為估計很難尋找或十分稀少的特性。

Snowball sampleSnowball sample 雪球抽樣雪球抽樣

• EX> survey users of an unusual product: a study among deaf for product that would allow deaf people to communicate over telephone.

• EX> 特殊生活習慣 ( 同志調查 ) ， homeless, gangsters, migrant workers, undocumented immigrants.

• EX> network study ，特殊疾病 (HIV)• Bias: a person who is known to someone has a higher

probability of being similar to the first person.

Quota samples 配額抽樣• by selecting sample elements in such a way that the

proportion of the sample elements possessing a certain characteristics is approximately the same as the proportion with the characteristics in the population.

Establishing a characteristics matrix: What proportion of the target population is male and female? what proportions of each gender fall various age categories, educational level, ethnic groups,…etc.

Once such a matrix has been created and a relative proportion assigned to each cell in the matrix, you collect data from people having all the characteristics of a given cell.

All the persons in a given cell are then assigned a weight appropriate to their portion of the total population.

Quota samples 配額抽樣

• Problems: The sample could be far off with respect

to other important characteristics. The quota frame must be accurate, and

it is often difficult to get up-to-date information for this purpose.

Quota samples 配額抽樣

Biases may exist in the selection of sample elements within a given cell. The interviewer has a quota to achieve. The actual choice of elements left to the discretion of the individual field worker. Interviewers are prone to follow certain practices

Quota samples 配額抽樣

those who are similar to the interviewers are more likely to be interviewed,

toward the accessible (first floor, airline terminals, business district, college campus),

toward household with children, exclude working people, against workers in manufacturing (service and

administrative), against extreme of income (EX> "mansions" were

skipped because the interviewer did not feel comfortable knocking on doors that were answered by servants. ),

against the less educated, against low-status individuals

Probability sampleProbability sample

• A probability sample is a sample chosen by chance. We must know what samples are possible and what chance or probability, each possible sample has.

Probability sampling offers two advantages: Probability sampling offers two advantages:

• First, probability samples, although never perfectly representative, are typically more representative than other types of samples because the biases previously discussed are avoided.

• Second, and more important, probability theory permits us to estimate the accuracy or representativeness of the sample.

Types of Sampling Designs

• Simple Random Sampling

• Systematic Sampling

• Stratified Sampling

• Cluster Sampling

Simple random sample

• A simple random sample (SRS) of size n consists of n individuals from the population chosen in such a way that every set of n individuals has an equal chance to be the sample actually selected.

Simple Random Sampling 簡單隨機抽樣

• Simple random sampling is the basic sampling method assumed in the statistical computations of social research.– Establish a sampling frame – Assigns a single number to each element in the

list, not skipping any number in the process.– generates series of random numbers to select the

elements• Simple random sampling is seldom used in

practice

Systematic Sampling 系統抽樣

• A systematic sample with a random start--a procedure in which an initial starting point is selected by a random process, and then every kth number on the list is selected.

• Sampling interval: the number of population elements between the units selected for the sample.

• Sampling interval = population size / sample size• Sampling ratio = sample size / population size• Systematic sampling is virtually identical to simple random sampl

e. If the list of elements is indeed randomized before sampling, one might argue that a systematic sample drawn from that list is in fact a simple random sample.

• Systematic sampling is much easier to conduct.

Problem of periodicity

• The arrangement of elements in the list can make systematic sampling unwise.

• EX> collecting retail sales information every seventh day (Monday)

• EX> apartment number

Stratified Random SamplingStratified Random Sampling 分層隨機分層隨機

The parent population is divided into mutually exclusive and exhaustive subsets.

A simple random sample of elements is chosen independently from each group or subset.

• To organize the population into homogeneous subsets and to select the appropriate number of elements from each. 先將母體劃分成數層 (strata) ，在每一層分別運用隨機抽樣方法抽取部份子樣本。

Stratified Random SamplingStratified Random Sampling 分層隨機分層隨機• Sampling error can be reduced by • (1) increase sampling size• (2) a homogeneous population produces samples

with smaller sampling errors than does a heterogeneous population.

• The logic of stratified sampling: rather than selecting your sample from the total population at large, you ensure that appropriate numbers of elements are drawn from homogeneous subsets of that population.

Stratified Random SamplingStratified Random Sampling 分層隨機分層隨機

• EX> urban and rural groups differ widely on attitudes toward energy conservation, members within each group hold very similar attitudes.

• EX> divide the university by college class (freshmen, sophomores, juniors, seniors)

• In selecting stratification variables, you should be concerned primarily with those that are presumably related to variables that you want to represent accurately. Such as sex, education, geographic location,…etc.

• EX> estimate income stratified by educational level.

Example 3.17

• 美國的音作曲家協會 (ASCAP) 對於廣播電台的所播放的流行音樂收取版權費。 ASCAP 包含四百萬首歌取，每年向廣播公司收取 4 億三千五百萬美金的收入。如何將這比錢發給作曲家？

• 透過每年 60,000 小時的抽樣錄音的分析來決訂版權費的分配。

• Radio stations are stratified by type of community (metropolitan, rural), geographic location, and the size of the license fee paid to ASCAP, which reflect the size of the audience.

Stratified Random SamplingStratified Random Sampling 分層隨機分層隨機• 層間元素差異性大而層內元素差異性小時 (homoge

neous within strata) ，分層抽樣所得結果較佳 (sampling error is smaller) 。

• The investigator should divide the population into strata so that the elements within any given stratum are as similar in value as possible and the values between any two strata are as disparate as possible.

• In the limit, if the investigator is successful in partitioning the population so that the elements in each stratum are exactly equal, there will be no error associated with the estimate of the population parameters.

Increased precision of stratified samplesIncreased precision of stratified samples

• EX> N=1,000

• Mean = 5 (.2) + 10 (.3) + 20 (.5)= 14, variance = 39

• Suppose that a researcher was able to partitioning the total population so that all the elements with a value of 5 in one stratum, those with value of 10 were in the second, and those with the value of 20 were in the third.

• Take a proportionate stratified sample of n=10.

• Or select a sample of n=3, and calculate the weighted average.

x f(x)

5 20010 30020 500

1000

Proportional stratified sample

• Proportional stratified sample: the number of sampling units drawn from each stratum is in proportion to the relative population size of that stratum.

• (1) Sort the population into discrete groups (2) On the basis of relative proportion of the population represented by a given group, select several elements from tat group constituting the same proportion of your desired sample size.

• (1) Group elements and then put groups together in a continuous list (an ordered list, if no periodicity, is sometime better than randomized list--implicit stratification in systematic sampling). (2) Select a systematic from the entire list.

Disproportionate stratified samplingDisproportionate stratified sampling

• Balancing the two criteria of strata size and strata variability. Strata exhibiting more variability are sampled more than proportionately to their relative size; those strata that are very homogeneous are sampled less than proportionately.

Multistage cluster sampling 群集抽樣• Used when it is either impossible or impractical to compile an e

xhaustive list of the elements composing the target population.• 先將母體分群 (cluster) ，將群 (cluster) 視為母體的抽樣單

位進行抽樣。• EX) 以都市化的程度將台灣分層，每一個都市層內以村里

為群進行第一階段隨機抽樣，抽取樣本村里。於樣本村里中進行第二階段的隨機抽樣，抽取樣本戶。

• EX) census blocks---sampled blocked sample household sample individual

• EX> sampling high school students in Taiwan requires the entire student list. Cluster sampling: no initial listing is required.

Multistage cluster sampling 群集抽樣

分層抽樣 :層數少，層內單位較多。所有層中至少有一單位被選入樣本。只有在每層中選部份單位做為樣本。群集抽樣 :群數多，群內單位較少。只有部份群集被選為樣本。群集抽樣則在被抽選之群集中進行普查或進行再次抽查。

Multistage cluster sampling 群集抽樣Price of the efficiency less accurate sample:

A simple random sample drawn from a population list is subject to a single sampling error, but a two-stage cluster sample is subject to two sampling errors. (ex> selecting a sample of disproportionately wealthy city blocks, plus a sample of disproportionately wealthy households within those blocks.)

Comparisons of sampling techniquesDescription

Simple Random Researcher assings each member of thesampling frame a number, then selectssample units by a random method.

Systematic Researcher uses natural ordering or orderof sampling frame, selects an arbitrarystrating point, then selects items at apreselected interval

Stratified researcher divides the population intogroups and randomly selects subsamplesfrom each group.

Cluster Researcher selects sampling units atrandom, then does complete observationof all units in the group

Multistage Progressively smaller areas are selectedin each stage. Researcher performs somecombination of the first four techniques

Comparisons of sampling techniques

Cost and Degree of UseSimple Random High cost, not frequently used in practice

(except random-digit dialing)Systematic Moderate cost, moderately usedStratified Hihg cost, moderately usedCluster Low cost, frequently usedMultistage Hihg cost, frequently used, especially in

nationwide surveys

Comparisons of sampling techniques

AdvantagesSimple Random Only minimal advance knowledge of

population needed; easy to analyze dataand compute error

Systematic Simple to draw sample; easy to checkStratified Assures representation of all groups in

sample; characteristics of each stratumcacn be estimated and comparisons made;reduces variability for same sample size

Cluster If clusters geographically definded, yieldslowest field cost; requries listing of allclusters but of individuals only withinclusters; can estimate characteristics ofclusters as well as of population

Multistage Depends on the techinques comined

Comparisons of sampling techniquesDisadvantages

Simple Random Requries sample frame to work from;does not use knowledge of population thatresearcher may have; larger errors frosame sample size than startified sampling; respondents may be widely dispersed,hence higher cost

Systematic If sampling interval is related to aperiodic ordering of the popuation, mayintroduce increased variability

Stratified Requires accurate information onproporiton in each stratum; if stratifiedlists are not already available, they can becostly to prepare

Cluster Larger error for comparable size thanother probability samples; researchermust be able to assing populationmembers to unique cluster, or duplicaitonor ommision of individuals results

Multistage Depends on the techniques combined

Sampling Bias

• A sample is biased if it is obtained by a method that favors the selection of elementary units having particular characteristics.

Sampling Error or Error of Estimation

-ˆ toequal is

,estmiation oferror or error, sampling The

,parameter population

some of estimate samplea be ˆLet

Respondent errorRespondent error

Systematic (nonsampling) error

Error in survey researchError in survey researchError in survey researchError in survey research

Random sampling error

Administrative error

Nonresponse error

Nonresponse error

Response biasResponse bias

Deliberate falsification Unconscious misrepresentation

Deliberate falsification Unconscious misrepresentation

Extremity bias

Interviewer bias

Auspices bias

Social desirability bias

Data processing error

Sample selection error

Interviewer error

Interviewer cheating

Acquiescence bias

Self-selection biasSelf-selection bias

Contamination by others

Random Sampling Error

A statistical fluctuation that occurs because of chance variation in the elements selected for a sample.

Can be estimated.Can be reduce through increasing sample

size.

Systematic Error 系統誤差 — nonsampling errors

測量工具不精準 imperfect aspect of the research design

測量執行時的錯誤 mistake in the execution of the research

A sample bias exists when the results of a sample show a consistent tendency to deviate in one direction from the true value of the population parameter.

Two general categories: – ) Respondent error –Nonresponse error + Response bias

– ) Administrative error

Non-response error

• The statistical difference between a survey that includes only those who responded and a survey that also includes those who failed to respond.

• Non-respondent—a person who is not contacted or who refuses to cooperate

• 1. not-at-home—married women • 2. refusal – a person who is unwilling to

participate.

Non-response error• To identify the extent of nonresponse error, business resear

cher often select a sample of nonrespondents who are then recontacted. 抽取一小部份不回答者做再造訪 (call back or follow-up), 比較次樣本與受訪樣本之差異。

• Comparing the demographics of the sample with the demographics of the target population is one mean of inspecting for possible bias. 與母體統計相比。

EX) 500 人以上之公司的百分比。老年人人口比率。 EX> sample from the educational or personnel records

Self-selection bias• (EX) who are more likely to respond to customer s

atisfaction survey on the dining table? • (EX) PC software--expert views on degree of "use

r friendly", might be more critical. • Self-selection biases the survey because it allows e

xtreme positions to be over-represented while those who are indifferent are under-represented 極端意見 .

Deliberate falsification Appear to be intelligent—EX> price of a good,

reluctant to say "can't remember". Conceal personal information—EX>income,

political attitude To avoid embarrassment—EX>sexual behaviors,

smoking/drinking Become bored—to get rid of the interviewer Reluctant to give negative feeling—EX> in

employee survey; to safeguard their job To please interviewer. ”Average man" hypothesis—to conform to their

perception of the average person. EX> number of hour worked.

Unconscious Misrepresentation• in the absence of strong preference, respondents

will choose answers to justify their behavior—– (EX) which PC is better? In-flights survey concerni

ng aircraft preference

• Misunderstand the question—– EX> 請問你的職位是？

• Never thought about the question—– EX>buying intention, quitting intention

• Forgot the exact details—– EX>when was last time you…? How many times di

d you…?

Acquiescence bias 默許偏誤

A tendency to agree with all questions or to indicate a positive connotation. “yea (no) sayers”

EX> Japanese do not wish to contradict others

particularly prominent in ideas previously unfamiliar to the respondents

Extremity bias (or avoid extreme position)

Consistently low or high scores were given to every question.

EX) student evaluation of the class.

Interviewer bias Bias due to the influence of the interviewer (mere presence) Provide the “right” answer to please interviewer; Appear intelligent and wealthy to “save face”. Interviewer’s age, sex, tone of voice, facial expressions, or other

noverbal characteristics. Will interviewer’s gender make a difference when asking the fol

lowing questions? ＥＸ）「威而剛」已經獲准上市，你會不會有興趣試一試？ ＥＸ）你平常花最多時間閱讀的報紙版面依序為… ? Interviewer shorten or rephrase question

Auspices bias贊助單位偏誤• —bias in the responses of subjects caused by the res

pondents being influenced by the organization conducting the study.

EX) 中研院進行政黨傾向的調查EX) 婦女新知針對「新女性移民」的態度來進行調查

Social desirability bias—bias in the responses of subjects caused by

respondent's desire, either consciously or unconsciously, to gain prestige or to appear in a different social role.

inflated income “have you ever been fired from a job?” “Do you have roaches in your home?” “how many times you brash your teeth per day”• Likelihood for social desirability bias: face-to-face > telephone > mail

Contamination by others• EX> complete a question on the satisfaction with

family (marital) relationship (Under the presence of a spouse).

Administrative error

Data processing errorSample selection error—unlisted telephone respondent,

stopping respondents during daytime hours in shopping center exclude working women, wrong household member answer the phone…etc.

Interviewer error—check the wrong response, can't write fast enough to record answers, selective perception (take liberty in interpreting questions, specific words may unconsciously be emphasized).

Interviewer cheating (deliberate subversion)—• fills in the answers to certain questions, skip questions, in order to finish

the question as soon as possible. • remedy–mini-re-interviews: a percentage of respondent will be call upon

to verify the data.

What can be done to reduce error:

• Questionnaire design—to reduce response bias• Sampling—to control random sampling error• Interviewer training• Use rule-of-thumb estimates for systematic error based on the result of other studies (areas), create benchmark

figures or standards of comparison— EX>½ of those who say they “will definitely buy “ within the

next three months actually do make a purchase. For durable—1/3. "will probably buy" durable = no actual buy