SII: Quantitative Methods

and Surveys

Tuesday 20th January

Outline• Introduction to Quantitative

Methods• The Hypodeductive Method• Units of Analysis• Sampling• Variables (and questions)• Analysis:

– Univariate– Bivariate (and multivariate)

• Strengths of Survey Research• Weaknesses of Survey

Research

Defining Terms

• Quantitative MethodsAre used to answer any counting related question: How many? What proportion?

• Survey ResearchInvolves analysis of responses to a standardized questionnaire, containing a battery of questions.

• Primary (Data) AnalysisIs analysis conducted by the investigator(s) or institution that collected the data.

• Secondary (Data) AnalysisIs any further analysis of an existing dataset that produces results or conclusions other than those produced as a result of the first report on the inquiry. Often carried out by different people than those that collected the data.

Where quantitative data comes from

• Primary Sources:– Surveys– Content analysis– Observation studies– Archival studies– Other…

• Secondary Sources:– Official statistics (surveys and other material)– Archived academic surveys– Other archived datasets

A lot of secondary datasets are now available on the web and access is free for those in academic environments.

You can browse some data at: UK Data Archive www.data-archive.ac.uk/ Economic and Social Data www.esds.ac.uk/Office for National Statistics

www.statistics.gov.uk/

The ‘typical’ hypodeductive structure of quantitative

research

TheoryDeduction

HypothesisOperationalization(research design)

Data CollectionData-organization

Data Analysis Interpretation

ResultsInduction

From Corbetta (2003: 59)

PHASES

PROCESSES

From Theory to Hypotheses• Theory involves wide-ranging statements about the world.

These are located at a high level of abstraction and generalization. They are often derived from empirical patterns and give rise to empirical forecasts.

• Quantitative analysis is usually involved in empirically testing particular hypotheses that are derived from theory.

• Hypotheses are general statements at a lower level of abstraction. They involve particular relationships (and directionality) between two (or more) concepts.

• Sometimes different theories will give rise to competing hypotheses. These can be empirically arbitrated. For example:1. From gender role theory: When married women earn

more than their husbands they violate normal gender roles. They are therefore likely to increase the amount of housework that they do in order to compensate.

2. From dependency theory: The person who earnsthe most will have the most power. Therefore when women earn more than their husbands they have more power. And they will reduce the amount of housework that they do.

Unit of Analysis• In order for any hypothesis to be tested empirically it

needs to be located.• Units of analysis are the things that are to be compared

or analysed. • Example: If I want to investigate revolution I may use as

my unit of analysis:– Individuals: investigating who is more or less likely to

become involved in revolutionary ferment– Countries: investigating what sort of society is more/less

likely to become revolutionary– Rebellions: investigating which rebellions are more/less

likely to develop into fully fledged revolutions– Literature: investigating which books are more/less likely to

be published in revolutionary epochs…etc.

• Corbetta (2003) lists the following types of unit of analysis: the individual; the aggregate of individuals; the group-organization-institution; the event; and the cultural product.

A question of sampling• Sometimes we can study every example of the

thing that we are interested in: the whole population.

• But most of the time this would be too difficult or time consuming.

• So we usually study just a sample of the cases that we are interested in.

• What is most important in selecting a sample is that it is representative of the population.

• When a sample is representative we can make inferences about the population based on the sample.

What is a Representative Sample?

• To be representative the sample should accurately reflect the range of possible responses/attitudes/behaviours of the whole population (n.b. this is not the population of the country, but the population of sociological interest).

• Since we may not know what that range is, we cannot know how to select a sample that is representative.

• Therefore the best that we can do is to ensure that every case (and so every attitude/behaviour…) has an equal chance of being included into the sample.

• This is the Equal Probability of Selection Method (EPSM). And the sample that results is known as a Probability Sample.

• The central principle in a Probability Sample is random selection.

A Simple Random Sample

What is a Representative Sample?• Some random samples are more complex than this –

involving ‘clustering’ or ‘stratifying.’ However these are still based on probabilities and so we can mathematically estimate the probability that any one case/person be selected.

• Not all samples are probability samples.

• Types of Non-Probability Sample:

– Reliance on available subjects

– Purposive or judgemental sampling

– Snowball sampling

– Quota sampling

• Some of these samples may be relatively representative. However they are less likely to be representative of attitudes/characteristics that we did not foresee.

• Additionally, since non-probability samples do not involve an EPSM findings cannot be used to make inferences about the whole population.

Variables• Quantitative Analysis involves the study

of variables.

• Variables are attributes that vary across cases, and/or within a case over time.

• For example, gender, age, happiness, political association, occupation, number of students on a course…

• The process of going from a concept to a variable involves operationalisation of the research question.

• When variables are produced in surveys they are often the product of ‘closed questions’.

• Closed questions are questions in which possible answers are given and the respondent selects from these: all of the questions in last-week’s survey were closed as you did not have the option to write your own answers.

Types of ‘variable’

1. Nominal (or categorical) i.e. ethnicity,

religion, favourite colour…, or:

What is your gender? Male Female

Types of ‘variable’

2. Ordinal(categories are in order) i.e. social class, status, agreement/disagreement scale…, or:

Thinking back over your first term at Warwick (from the time you arrived here until Christmas break), what do you think was the longest period that you went WITHOUT having an alcoholic drink? Less than a day (i.e. you drank every day) One or two days Three to six days Between one and two weeks Between two weeks and a month More than a month, but less than the full term The whole term (i.e. you did not have a single drink)

Types of ‘variable’

3. Interval/Ratiomathematical operations possible. i.e. age, income, hours of work…, or:

How many alcoholic drinks have you consumed in the last 7 days?

____________

Problems with survey questions• Survey questions need to be:

Exhaustive – that everyone fits into one category

Exclusive – so that everyone fits into only one category (unless specifically required to ‘tick as many as apply).

Unambiguous – so that they mean the same to everyone and all responses are comparable.

• When one or more of these is violated, even if researchers are trying hard to be ‘unbiased,’ the survey data will ‘reflect’ not reality but the specific interpretations of each respondent, and since the researcher has no way of knowing what these are, she will have no way of knowing what she is analysing.

• For example: How did you calculate your answer to the following: How many alcoholic drinks have you consumed in the last 7 days? How did you work out what counted as “one drink”? Could you remember how many you’d had?

Statistical AnalysisAnalysis can be• Univariate – involving just one variable at a time• Bivariate – involving two variables• Multivariate – involving three or more variables.Statistical Analysis can aim to:• Describe – called ‘descriptive statistics’• Make inferences from a sample to the

population – called ‘inferential statistics’• Analyse relationships between variables – called

‘analytic statistics’

The choice of statistical technique depends on both the aims of the researcher and the types of variable to be analysed.

Univariate Analysis• We are interested in the form taken by the

distribution of cases. • This analysis is usually descriptive (although

sometimes it involves inference).• Where variables are categorical – i.e. nominal or

ordinal: we use the mode and pictorial representations (such as pie-charts and bar-charts). We also give percentages of cases falling into each category.

• With interval level data we can go beyond the pictorial (although we will often start by looking at a chart called a histogram to get a feel of the data).

• We want to be able to summarise data as efficiently as possible – so that we can see the wood for the trees.

Nominal or Ordinal variables can be represented with pie-charts

The modal (most common) response was that students went a maximum of 3 to 6 days without a drink in their first term.

Because this is an ordinal variable we can add together the red and light blue pieces of the pie and say that about 20% of

you managed to go “more than a whole month.”

Responses to question: “Longest period without a drink in first term at Warwick”

2008/9

2007/8 2006/7

Alcohol 98% 94% 98%

Cigarettes 57% 59% 65%

Marijuana/Cannabis/Hash 41% 40% 51%

Cocaine or Crack 16% 6% 13%

Ecstasy 11% 7% 6%

Speed/Amphetamine or Crystal Meth/Crank

5% 1% 4%

Heroin 2% 0% 2%

Poppers/Amyl Nitrite 17% 13% 15%

Acid/LSD 6% 0% 4%

Magic Mushrooms 8% 2% 6%

Prescription drugs (for recreational purposes)

12% 4% 11%

Total Responses 98 86 82

Categorical Variables: Dichotomous Variables.

When we are studying whether or not people have done something there are only two possible answers – people have or have not done this thing – these are examples of dichotomous variables.

Describing dichotomous variables usually takes the form of saying what proportion of people fall into one of the two categories – in this case, those who have done the thing. (Note: comparisons across years involve Bivariate Analysis – more later).1st Year Sociology students (2008/9, 2007/8 & 2006/7) who have ever consumed:

A biased sample?• To be representative the sample should truly reflect the

range of possible responses/attitudes/behaviours of the whole population. So…

• Question: Since not every Warwick sociology student filled out a survey last week (as some were not in lecture), how representative do you think that the Survey of Warwick Sociology students was?

– Specifically, given that it asked questions about drink and drugs, do you think that the people who were in lecture last Tuesday were representative of those who were not here as well?

– One way of thinking about this is to ask whether there is likely to be differences between those people who do and who do not come to lecture in terms of how they’d have answered the questions.

It may be that there is a correlation between taking drugs/drinking more and non-attendance at lecture. Therefore the sample of students may be biased, especially in relation to this topic.

Interval-ratio variables have meaningful response-categories.

Their central tendency can be described with a mean. And the amount of variation from (or spread around) the mean, can be described with the standard deviation.

Interval-ratio variables can be graphed with a histogram.

Describing Interval-Ratio Variables

Univariate Analysis – summarising dataWhen we summarise data we look at:• Measures of location (or central tendency)

– Mean – what people refer to as the arithmetical ‘average’

– Mode – the most common value (or peak)

– Median – if we place values in order, the middle one.

• Measures of dispersion– The standard deviation – based on the difference between (individual)

data points and the (arithmetic) mean (actually the square root of the average of these).

– The range – as in the everyday sense – the largest value minus the smallest (i.e. height of tallest person minus height of shortest)

– The inter-quartile range – one cuts off the highest and lowest 25 percent of data values and calculates difference between the new ‘extremes’ (upper quartile minus lower quartile).

Note: The standard deviation is the most commonly used measure of dispersion because it provides part of the solution to assessing sampling error in random sample designs – i.e. helps us to judge how close our results (from a sample) are likely to be to the underlying population characteristics – this is the essence of statistical inference.

Bivariate and Multivariate Analysis

• …is used to examine and specify relationships between two or more different variables.

• The relationships that are specified are probabilistic – it is not that every man will do x and every women won’t – rather that men are more likely to do x.

• Furthermore quantitative analysis cannot explain why relationships exist, it can just show that they appear to and help to specify the associated social factors.

• Only things that can be measured and have been included in a statistical ‘model’ can be ‘found’ to be associated with one another.

Example of Bivariate Analysis: Comparing Groups

How many alcoholic drinks have youconsumed in the last 7 days?

18

2

19.39

19.011

66

10

10.97

11.239

N

Mean

Std. Deviation

N

Mean

Std. Deviation

Male

Female

You can also look at differences in the shape of the histograms – here you can see that the female histogram goes down much more steeply.

Histograms showing responses to the question: ‘How many alcoholic drinks have you consumed in the last 7 days?’ by Gender

Male Female When you are comparing an interval-ratio variable across groups you can compare their means and medians

0 25 50 750

5

10

15

20

0 25 50 75

Example of Bivariate Analysis: Comparing GroupsBar chart showing EXPERIENCES AFTER DRINKING of First Year Sociology

students at Warwick, by Gender (only students who’ve drunk alcohol since at Warwick are included)

Where the two variables are both nominal we can compare the proportion of cases (or people) who fall into different groups.

However just because groups seem to differ, does not mean that there is a relationship that would persist in the population at large or one that is meaningful. More statistical tests would be required to evaluate this.

Example of Bivariate Analysis: Comparing GroupsComparison #2: Students from 2006/7, 2007/8 and 2008/9

cohorts.

Do you think that results for the three years are similar?

Does this give you more or less confidence that the results for this year are a ‘true’ representation of student life?

Juggling the numbers or doing research?

Researchers who conduct quantitative analysis are responsible for making clear to readers the basis on which they make any claims. This requires specifying, among other things: – The number of cases in any

category, or in any analysis– Their sampling process– The way in which questions

and categories were constructed.

Strengths of Survey Research

• Useful in describing the characteristics of a large population.

• Make large samples feasible – often relatively quick (and telephone/postal surveys can be conducted at a distance).

• Flexible – many questions can be asked on a given topic.

• Relatively impersonal form of research – can be good for asking sensitive questions that people are uncomfortable talking about.

• Easy to standardize interactions.• Reliable (and replicable). • Therefore relatively transparent methodology.

Weaknesses of Survey Research

• Can seldom deal with the context of social life.• Inflexible - in that it requires that the researcher

knows what to ask about before starting (and therefore poor for exploratory research).

• Subject to artificiality – the product of respondents’ consciousness that they are being studied. This can be exacerbated where there is a power-relationship between the person studying and the person being studied.

• Weak on validity.• Poor at answering questions where individual is not

the unit of analysis.• Usually inappropriate for historical research.• Particularly weak at gathering at certain sorts of

information:– Highly complex or ‘expert’ knowledge– People’s past attitudes or behaviour– Subconscious (especially macro-social) influences– Attitudes (or at least embodied attitudes)– Shameful or stigmatized behavior or attitudes

(especially in face-to-face interview – better in self-completion surveys)

• We will develop these criticisms more next week.