sampling design. how do we gather data? surveys surveys opinion polls opinion polls interviews...

Sampling Sampling DesignDesign

How do we gather data?How do we gather data?

Surveys Surveys Opinion pollsOpinion polls InterviewsInterviews StudiesStudies

ObservationalObservational Retrospective (past)Retrospective (past) Prospective (future)Prospective (future)

ExperimentsExperiments

PopulationPopulationthe entire group of the entire group of individuals that we individuals that we want information want information aboutabout

CensusCensusa complete count of a complete count of the populationthe population

SampleSampleA part of the population A part of the population that we actually examine in that we actually examine in order to gather informationorder to gather information

Use sample to generalize to Use sample to generalize to populationpopulation

Sampling Sampling designdesign

refers to the refers to the methodmethod used to used to choose the sample choose the sample from the populationfrom the population

Sampling Sampling frameframe

a list of a list of everyevery individual in the individual in the populationpopulation

consist of consist of nn individuals from the individuals from the population chosen in such a way population chosen in such a way thatthatevery individual has an equal every individual has an equal chance of being selectedchance of being selected

every set of every set of nn individuals has an individuals has an equal chance of being selectedequal chance of being selected

Simple Random Simple Random Sample (SRS)Sample (SRS)Suppose we were to take an

SRS of 100 SHS students – put each students’ name in a hat.

Then randomly select 100 names from the hat. Each

student has the same chance to be selected!

Not only does each student has the same chance to be selected – but every possible group of 100 students has the same chance to be selected! Therefore, it has to be possible for all 100 students

to be seniors in order for it to be an SRS!

Stratified Stratified random samplerandom samplepopulation is divided population is divided into homogeneous into homogeneous groups called stratagroups called strata

SRS’s are pulled from SRS’s are pulled from each stratumeach stratum

Homogeneous groups are groups that are alike based upon some

characteristic of the group members.

Suppose we were to take a stratified random sample of 100

SHS students. Since students are already divided by grade level, grade level can be our strata.

Then randomly select 50 seniors and randomly select 50 juniors.

Cluster Cluster SampleSample

based upon locationbased upon locationrandomly pick a randomly pick a location & sample location & sample allall there there

Suppose we want to do a cluster sample of SHS students. One way to do this would be to randomly select 10 classrooms during 2nd period. Sample all students in

those rooms!

Systematic Systematic random samplerandom sampleselect sample by select sample by following a following a systematic approachsystematic approach

randomly select randomly select where to beginwhere to begin

Suppose we want to do a systematic random sample of SHS students -

number a list of students(There are approximately 2000 students – if we want a sample of 100, 2000/100 =

20)Select a number between 1 and 20 at

random. That student will be the first student chosen, then choose

every 20th student from there.

Multistage Multistage samplesample

select successively select successively smaller groups within smaller groups within the population in stagesthe population in stages

SRS used at each stageSRS used at each stage

To use a multistage approach to sampling SHS students, we could first divide 2nd period classes by

level (AP, Honors, Regular, etc.) and randomly select 4 second period

classes from each group. Then we could randomly select 5 students from each of those classes. The

selection process is done in stages!

Identify the sampling Identify the sampling designdesign

1)The Educational Testing Service 1)The Educational Testing Service (ETS) needed a sample of colleges. (ETS) needed a sample of colleges. ETS first divided all colleges into ETS first divided all colleges into groups of similar types (small groups of similar types (small public, small private, etc.) Then public, small private, etc.) Then they randomly selected 3 colleges they randomly selected 3 colleges from each group.from each group.

Stratified random Stratified random samplesample

2) A county commissioner wants to 2) A county commissioner wants to survey people in her district to survey people in her district to determine their opinions on a determine their opinions on a particular law up for adoption. She particular law up for adoption. She decides to randomly select blocks decides to randomly select blocks in her district and then survey all in her district and then survey all who live on those blocks.who live on those blocks.


Cluster samplingCluster sampling

3) A local restaurant manager wants 3) A local restaurant manager wants to survey customers about the to survey customers about the service they receive. Each night service they receive. Each night the manager randomly chooses a the manager randomly chooses a number between 1 & 10. He then number between 1 & 10. He then gives a survey to that customer, gives a survey to that customer, and to every 10and to every 10thth customer after customer after them, to fill it out before they leave.them, to fill it out before they leave.


Systematic random samplingSystematic random sampling

Random digit Random digit tabletable

each entry is equally each entry is equally likely to be any of the likely to be any of the 10 digits10 digits

digits are independent digits are independent of each otherof each other

The following is part of the random digit table found on page 847 of your textbook:

Row

1 4 5 1 8 5 0 3 3 7 1

2 4 2 5 5 8 0 4 5 7 0

3 8 9 9 3 4 3 5 0 6 3

Numbers can be read across.

Numbers can be read vertically.

Numbers can be read diagonally.

Suppose your population consisted of these 20 Suppose your population consisted of these 20 people:people:

1) Aidan6) Fred 11) Kathy 16) Paul2) Bob 7) Gloria 12) Lori 17) Shawnie3) Chico 8) Hannah 13) Matthew 18) Tracy4) Doug 9) Israel 14) Nan 19) Uncle Sam5) Edward 10) Jung 15) Opus 20) Vernon

Use the following random digits to select a sample of five from these people.

We will need to use double digit random

numbers, ignoring any number greater than 20. Start with Row 1

and read across.

Row1 4 5 1 8 0 5 1 3 7 12 0 1 5 5 8 0 1 5 7 03 8 9 9 3 4 3 5 0 6 3

Ignore.

18) Tracy

5) Edward

13) Matthew

1) Aidan

15) Opus

Ignore.Ignore.Ignore.

Stop when five people are selected. So my sample would

consist of :

Aidan, Edward, Matthew, Opus, and Tracy

BiasBiasA systematic error A systematic error in in measuringmeasuring the the estimateestimate

favors certain favors certain outcomesoutcomes

Anything that causes the data to be wrong!

It might be attributed to the researchers, the

respondent, or to the sampling method!

Sources of Sources of BiasBias

things that things that can can causecause bias in your bias in your samplesample

cannot do anything cannot do anything with bad datawith bad data

UndercoveragUndercoveragee

some groups of some groups of population are left population are left out of the selection out of the selection processprocess

Suppose you take a sample by

randomly selecting names from the phone

book – some groups will not

have the opportunity of being selected!

People with unlisted phone numbers – usually high-income families

People without phone numbers –usually low-income families

People with ONLY cell phones – usually young adults

Voluntary Voluntary responseresponse

People chose to People chose to respond respond

Usually only people Usually only people with very strong with very strong opinions respondopinions respond

An example would be the surveys in magazines that ask readers to mail in the survey.

Other examples are call-in shows, American Idol, etc.

Remember, the respondent selects themselves to

participate in the survey!

Remember – the way to determine

voluntary response is:

Self-selection!!

NonresponseNonresponseoccurs when an individual occurs when an individual chosen for the sample can’t chosen for the sample can’t be contacted or refuses to be contacted or refuses to cooperatecooperate

telephone surveys 70% telephone surveys 70% nonresponsenonresponse

People are chosen by the researchers, BUT refuse to

participate.

NOTNOT self-selected!

This is often confused with voluntary response!

Because of huge telemarketing efforts in the past few years,

telephone surveys have a MAJOR problem with

nonresponse! One way to help with the

problem of nonresponse is to make follow contact with

the people who are not home when you first contact

them.

Response biasResponse biasoccurs when the occurs when the behavior of respondent behavior of respondent or interviewer causes or interviewer causes bias in the samplebias in the sample

wrong answerswrong answers

Suppose we wanted to survey high school students on drug

abuse and we used a uniformed police officer to

interview each student in our sample – would we get honest

answers?

Response bias occurs when for some reason (interviewer’s or

respondent’s fault) you get incorrect answers.

Convenience Convenience samplingsampling

Ask people who Ask people who are easy to askare easy to ask

Produces bias Produces bias resultsresults

An example would be stopping friendly-looking people in the

mall to survey. Another example is the surveys left on

tables at restaurants - a convenient method!

The data obtained by a convenience sample will be

biased – however this method is often used for surveys &

results reported in newspapers and magazines!

Wording of the Wording of the QuestionsQuestions

wording can influence the wording can influence the answers that are givenanswers that are given

connotation of wordsconnotation of wordsuse of “big” words or use of “big” words or technical wordstechnical words

Questions must be worded as neutral as possible to

avoid influencing the response.

The level of vocabulary should be appropriate for

the population you are surveying

– if surveying Podunk, AR, then you should

avoid complex vocabulary.

– if surveying doctors, then use more

complex, technical wording.

1. A uniformed policeman interviews a group of 50 college freshmen. He asks each one his or her name and then if he or she as used an illegal drug in the last month.

A. Selection bias

B. Measurement or Response bias

C. Nonresponse bias

D. Systematic rejection bias

2. A survey about the food in the school cafeteria was conducted by passing out questionnaires to students as they entered the cafeteria. A drop box for completed forms was on a table by the cash register.

A. Selection bias


C. Nonresponse bias


3. The magazine Harley Davidson Today sent a survey to its subscribers asking whom they admire most in America.

A. Selection bias


C. Nonresponse bias


4. A poll of parents in Texas found that 90% of parents say they have spoken to their teenagers about the dangers of drinking and driving, while only 45% of those teens say they recall such a discussion.

A. Selection bias


C. Nonresponse bias


5. In a census in Russia, 1.8 million more women than men reported that they were married.

A. Selection bias


C. Nonresponse bias


6. One year after the Detroit race riots of 1967, interviewers asked a sample of black residents in Detroit if they felt they could trust most white people, some white people, or none at all. When the interviewer was white, 35% answered "most"; when the interviewer was black, 7% answered "most".

A. Selection bias


C. Nonresponse bias


7. A political party mailed questionnaires to all registered voters in Texas, asking whether or not the party should support the death penalty. The voters mailed the completed questionnaires back in an envelope provided.

A. Selection bias


C. Nonresponse bias


8. The Nielson rating service estimates the popularity of television stations in the Dallas area. Suppose that four times a year, Nielson takes a random sample of about 5000 viewers. Every member of the household over age 12 is asked to fill out a diary, showing what he or she watches every quarter hour from 6:00 am to midnight. Each diarist receives $5 for his or her trouble. At the end of 12 weeks, Nielson tallies the results from the usable diaries - usually between 33% and 50% of the 5000 sent out.

A. Selection bias


C. Nonresponse bias


9. In the 1936 presidential election, Franklin D. Roosevelt ran for reelection against Alfred Landon. As it had done since 1916, the Literary Digest, a popular magazine, ran a preelection poll. To obtain its sample, the magazine compiled a list of about 10 million names from sources such as telephone books, lists of automobile owners, club membership lists, and its own subscription lists. All 10 million people received questionnaires, about 2.4 million returned them; these people made up the sample. Literary Digest had correctly predicted the winner in all presidential races since 1916. Then in 1936, based on sample responses, the magazine predicted that Landon would win, 57% to 43%. In fact, Roosevelt won, 62% to 38%.

A. Selection bias


C. Nonresponse bias


Which of the following sampling methods produces a simple random

sample?10. From a class of 25 students, the

teacher selects the last 5 to enter the room to be in the sample.

A)Is a simple random sample

B) Is not a simple random sample

11. From a group of 100 employees, the manager selects those whose phone numbers end in 7.

A) Is a simple random sample


12.A large elementary school has 15 classes with 24 children in each classroom. A sample of 30 is chosen by the following procedure: Each of the 15 teachers selects 2 children from his or her classroom to be in the sample by numbering the children from 1 to 24, then using a random digit table to select two different numbers between 01 and 24. The two children with those numbers are in the sample.



13. Suppose that in a class of 24 there are 12 boys and 12 girls. The teacher selects 6 students for a sample by numbering the boys from 1 to 12 and the girls from 1 to 12. Then using a random digit table, the first number between 01 and 12 is a boy, the next number between 01 and 12 is a girl and so on until the 6 students are selected.



14. Suppose that in a class of 24 there are 12 boys and 12 girls. The teacher selects 6 students for a sample by numbering the boys from 1 to 12 and the girls from 13 to 24. Then she uses a random number table to select 6 two-digit numbers between 01 and 24.



Definitions:Definitions:

1) Observational study - 1) Observational study - observe outcomes without observe outcomes without imposing any treatmentimposing any treatment

2) Experiment - actively impose some treatment in order to observe the response

3)Experimental unit – the single individual (person, animal, plant, etc.) to which the different treatments are assigned

4) Factor – is the explanatory variable – it’s what we test5) Level – a specific value for the factor

6) Response variable – what 6) Response variable – what you measureyou measure

7) Treatment – a specific experimental condition applied to the units

8) Control group – a group that is used to compare the factor against; can be a placebo or the “old” or current item9) Placebo – a “dummy” treatment that can have no physical effect

10) blinding - method used so 10) blinding - method used so that units do not know which that units do not know which treatment they are gettingtreatment they are getting

11) double blind - neither the units nor the evaluator know which treatment a subject received

Principles of Experimental Principles of Experimental DesignDesign

Control Control of effects of extraneous of effects of extraneous variables on the response – by variables on the response – by comparing treatment groups to a comparing treatment groups to a control group (placebo or “old”)control group (placebo or “old”)

ReplicationReplication of the experiment on of the experiment on many subjects to quantify the many subjects to quantify the natural variation in the experimentnatural variation in the experiment

Randomization –Randomization – the use of chance the use of chance to assign subjects to treatmentsto assign subjects to treatments

The ONLY way to show cause & effect is with a well-designed, well-controlled experiment!!!

Example 1: A farm-product manufacturer wants to determine if the yield of a crop is different when the soil is treated with three different types of fertilizers. Fifteen similar plots of land are planted with the same type of seed but are fertilized differently. At the end of the growing season, the mean yield from the sample plots is compared.Experimental units?

Factors?

Levels?

Response variable?

How many treatments?

Plots of land

Type of fertilizer

Fertilizer types A, B, & CYield of crop

3

Example 2: A consumer group wants to test cake pans to see which works the best (bakes evenly). It will test aluminum, glass, and plastic pans in both gas and electric ovens.

Experiment units?

Factors?

Levels?

Response variable?

Number of treatments?

Two factors - type of pan & type of oven

Type of pan has 3 levels (aluminum, glass, & plastic & type of oven has 2 levels (electric & gas)

How evenly the cake bakes

6

Cake batter

Experiment Experiment DesignsDesigns

Completely randomized – all Completely randomized – all experimental units are experimental units are allocated at random among allocated at random among all treatmentsall treatments

Treatment group 1

Treatment group 2

Treatment group 3

explanatory

variable

response

variable

Ra n

dom

ass

ignm

e nt

Randomized block – units are Randomized block – units are blocked into groups blocked into groups (homogeneous) and then (homogeneous) and then randomly assigned to randomly assigned to treatmentstreatments

Ran

dom

ass

ignm

entGroup1

Group2

Treatment 1Treatment 2Treatment 3

Treatment 3Treatment 2Treatment 1

explanatory response

varaible varaibleUnits should be blocked on a variable that effects the response!!!

match up experimental units match up experimental units according to similar according to similar characteristics & randomly characteristics & randomly assign on to one treatment & assign on to one treatment & the other automatically gets the other automatically gets the 2nd treatmentthe 2nd treatment

have each unit do both have each unit do both treatments in random ordertreatments in random order

the assignment of treatments the assignment of treatments is is dependentdependent

•Matched pairs - a Matched pairs - a special type of block special type of block designdesign

12) Confounding variable – the 12) Confounding variable – the effect of the confounding effect of the confounding variable on the response cannot variable on the response cannot be separated from the effects of be separated from the effects of the explanatory variable (factor)the explanatory variable (factor)

Example 5: Four new word-processing programs are to be compared by measuring the speed with which standard tasks can be completed. One hundred volunteers are randomly assigned to one of the four programs and their speeds are measured.

Is this an experiment? Why or why not?What type of design is this?

Factors? Levels?

Response variable?

Yes, a treatment is imposed.

Completely randomizedone factor: word-processing program with 4 levels speed

Example 5: Four new word-processing programs are to be compared by measuring the speed with which standard tasks can be completed. One hundred volunteers are randomly designed to one of the four programs and their speeds are measured.

Is there a potential confounding variable?

Can this design be improved?

Explain.

NO, completely randomized designs have no confounding

You could do a block design where each person uses each

program in random order.

Randomization reduces bias by spreading any uncontrolled confounding variables evenly throughout the treatment groups.

Variability is controlled by sample size. Larger samples produce statistics with less variability.

Blocking also helps reduce variability.

Is there another way to reduce

variability?

Bias is a systematic error in measuring the estimate

High bias & low variability

Low bias & low variability

Low bias & high variability

High bias & high variability

sampling design. how do we gather data? surveys surveys opinion polls opinion polls interviews...

Documents