survey design. what is a survey?? asking questions – questionnaires finding out things about...
TRANSCRIPT
Survey design
What is a survey??
•Asking questions – questionnaires•Finding out things about people•Simple things – lots of people•What things?•What people?
Why do a survey?
•Cheap (!)•Quick (!)•Need data on a ‘population’
Surveys•What people think, feel and do and are.…
▫Self-reported beliefs, knowledge, opinions (questionnaire)
▫What they are like (field tests)•Quantitative methodology
▫numbers / statistics•Cross-sectional (a ‘snapshot’)
▫differences and relationships▫NOT causes
Surveys can be:-
•Self-report questionnaires•Interviewer-administered questionnaires•Field tests
▫e.g. physical, psychological, behavioural
Main requirements of surveys
•Representative sample ▫generalisability (to the population)▫sub-group analyses▫reduce sampling bias
•Meaningful results▫validity of questions, scales, tests▫accurate descriptions, differences and
relationships•High response rate
▫reduce response bias
Surveys
•Find a few people…….•Ask a few questions…….•Get my tutor to do the statistics…..
NO!
So you want to do a survey…?
•PROS▫quick results▫relatively cheap▫can use relatively simple measures▫high levels of statistical power▫can involve simple statistics
So you want to do a survey…?
•CONS▫substantial planning needed▫sampling can get complicated▫time consuming data cleaning / entry▫need large numbers▫cannot identify cause and effect▫can involve complicated statistics▫complicated logistics
Value for money
RigourRight research question(s)Right sizeInternal validityExternal validityReliabilityAppropriate design Competent team
FeasibilityTimePersonnelAccessEthics
Not too bigNot too smallData is NEEDEDCosts are justified
So, let’s design a survey
Survey planning
•State the research questions•Define your population•Define your main dependent variables (DV)•Define your independent variables (IV)•Choose questions, scales, tests and analyses •What will be your sample?•Design sampling methodology•Estimate sample size
Research questions (permissible)
•What is the prevalence of….?•How tall / overweight / active / fit are..?•Is there a difference between….?•What factors are related to….?•What factors predict…?
Research questions (not permissible)•What causes x…..?•What effect does x have…..?
Establish your aimEstablish your research question(s)
Data collection•Use questionnaire / data sheets that make
coding and data collection easy•Use simple questions•For field tests:-
▫Ensure test accuracy▫Calibrate equipment▫Standardise test administration
•Record data systematically and safely
For each research question:What variables must you measure?How will you measure them? Hw will you analyze the data?
Populations
•Do you need ‘population’ data•What is a ‘population’?
▫A group of people▫Carefully defined▫Inclusion / exclusion criteria
•What is your population?
Considerations
•Access to population•Nature of the population (ethical
implications)•Existence of sampling frame
▫Lists, directories, addresses, tel. numbers•Access to sampling frame•Access to sample
Define your population
Sampling........
•A way of representing a large population through the study of a smaller number selected from that population
•Saves time and money ▫ BUT - introduces error
•The sample must REPRESENT the population as closely as possible....…▫E.g. Age, sex, socio-demographic profile,
ethnicity, environment, ▫Need proportionate numbers of all sub-
groups
Sampling
•Type of people?•Number of people?•Number of people within each type?
Individuals
Population country, city, university, school
•Simple random sampling▫Everyone in the target population has an
equal chance of being selected
Population - country, city, university, school
Do you have a SAMPLING FRAME?
StrataagegenderSEP
Population country, city, university, school
Clusterscities,suburbsdepartments,classes
Population country, city, university, school
Clusterscitiessuburbsdepartmentsclasses
StrataagegenderSEP
Gets complicated…!!
Sources of error
•Systematic error (bias)▫e.g. high non-response, interviewer or
instrument variation, incomplete sampling frame
•Response bias▫Subjects giving socially-desirable answers
(lying???!!!)•Random error
▫Because your tests are carried out on a sample, not the whole population
Major tasks•Identify relevant ‘strata’ within the
population▫do they influence the DV?
•Locate the sampling frame•Does it contain strata information? •Calculate proportionate numbers to sample
within each stratum•Select the sample
How will you choose your sample?
How many subjects do I need...??
No. of subjects VARIES with:
No. of sub-groupsLevel of sampling errorVariability of outcome measuresTarget differences / relationshipsStatistical power requiredSignificance level
You need to also consider:
Multiple outcome measuresLogisticsCostUse of results
No. of subjects DOES NOT vary with: Size of population
How many participants will you need?[Or – what is the largest number of participants you can cope with?]
• Response rate▫No. of respondents divided by number asked
to respond• Non-response bias
▫Sample potentially being biased because non-respondents are fundamentally different to respondents
• Non-participation study▫Finding out whether non-respondents ARE
fundamentally different from respondents