![Page 1: 1 MSc ASR, SR06 Session 9 Quantitative methods of social research for cross-national comparisons Paul Lambert, 5.2.02](https://reader035.vdocuments.net/reader035/viewer/2022062401/5a4d1aee7f8b9ab05997c5bc/html5/thumbnails/1.jpg)
1
MSc ASR, SR06 Session 9
Quantitative methods of social research for cross-national
comparisons
Paul Lambert, 5.2.02
http://staff.stir.ac.uk/paul.lambert/teaching.htm
![Page 2: 1 MSc ASR, SR06 Session 9 Quantitative methods of social research for cross-national comparisons Paul Lambert, 5.2.02](https://reader035.vdocuments.net/reader035/viewer/2022062401/5a4d1aee7f8b9ab05997c5bc/html5/thumbnails/2.jpg)
2
Quantitative cross-national social research
1) Introduction
2) Three traditions in Qn cross-national research
3) Seven themes in Qn cross-national research
Case study 1: Secondary analysis of cross-national surveys
![Page 3: 1 MSc ASR, SR06 Session 9 Quantitative methods of social research for cross-national comparisons Paul Lambert, 5.2.02](https://reader035.vdocuments.net/reader035/viewer/2022062401/5a4d1aee7f8b9ab05997c5bc/html5/thumbnails/3.jpg)
3
Introduction: Formats of Quantitative Cross-National
research
Aside: cross-national between country cross-national comparative But in quantitative methods, ‘XN’ &
‘comparative’ often used interchangeably
![Page 4: 1 MSc ASR, SR06 Session 9 Quantitative methods of social research for cross-national comparisons Paul Lambert, 5.2.02](https://reader035.vdocuments.net/reader035/viewer/2022062401/5a4d1aee7f8b9ab05997c5bc/html5/thumbnails/4.jpg)
4
QDA: Analysis of patterns of relationships between variables in the variable-by-case matrix
[Low # of vars; stats / graphical summaries]
Cases Variables 1 1 17 1.73 A . . . .2 1 18 1.85 B . . . .3 2 17 1.60 C . . . .4 2 18 1.69 A . . . .. . . . . . . . .
N
![Page 5: 1 MSc ASR, SR06 Session 9 Quantitative methods of social research for cross-national comparisons Paul Lambert, 5.2.02](https://reader035.vdocuments.net/reader035/viewer/2022062401/5a4d1aee7f8b9ab05997c5bc/html5/thumbnails/5.jpg)
5
A convenient distinction
Micro-macro distinction isn’t always important (& can be confusing). But is widely used, & tends to be associated with different research fields.
Macro-social data Micro-social data
Work and/or report at level of
aggregated unit (country)
Work and/or report at level of
constituent unit (eg individuals)
![Page 6: 1 MSc ASR, SR06 Session 9 Quantitative methods of social research for cross-national comparisons Paul Lambert, 5.2.02](https://reader035.vdocuments.net/reader035/viewer/2022062401/5a4d1aee7f8b9ab05997c5bc/html5/thumbnails/6.jpg)
6
a) Macro-Social QnXR
Each case represents country, & aggregate statistics are compared
Ideal family size1979 1989 Religiosity ‘81
Denmark 2.31 2.13 2.06Ireland 3.62 2.79 3.42
Italy 2.11 2.20 2.90Portugal 2.29 2.23 2.66
UK 2.29 2.14 2.33(eg from Coleman 1996:39)
![Page 7: 1 MSc ASR, SR06 Session 9 Quantitative methods of social research for cross-national comparisons Paul Lambert, 5.2.02](https://reader035.vdocuments.net/reader035/viewer/2022062401/5a4d1aee7f8b9ab05997c5bc/html5/thumbnails/7.jpg)
7
b) Micro-social QnXR
Cases (eg people) are grouped by country
Case id Country Indv. vars Natl. var1 1 17 1.73 A 56.22 1 18 1.85 B 56.23 1 17 1.60 C 56.24 2 18 1.69 A 50.85 2 18 1.65 C 50.86 3 19 1.84 B 260.3. . . . . .
![Page 8: 1 MSc ASR, SR06 Session 9 Quantitative methods of social research for cross-national comparisons Paul Lambert, 5.2.02](https://reader035.vdocuments.net/reader035/viewer/2022062401/5a4d1aee7f8b9ab05997c5bc/html5/thumbnails/8.jpg)
8
Data Analytical techniques
Same core data analysis techniques as for other social science applications, eg :
Critical issue is ‘level of measurement’Univariate, bivariate, multivariate Description v’s inferenceSurvey methodology issuesA few advanced extensions, eg ‘mixed
models’ to cater for hierarchical effects.
![Page 9: 1 MSc ASR, SR06 Session 9 Quantitative methods of social research for cross-national comparisons Paul Lambert, 5.2.02](https://reader035.vdocuments.net/reader035/viewer/2022062401/5a4d1aee7f8b9ab05997c5bc/html5/thumbnails/9.jpg)
9
Key feature of QnXR: Country as a categorical factor
Analyse within countries then compare outcomes (‘case oriented’)
V’sAnalyse data pooled between countries, use
countries / country level factors as explanations (‘variable oriented’)
![Page 10: 1 MSc ASR, SR06 Session 9 Quantitative methods of social research for cross-national comparisons Paul Lambert, 5.2.02](https://reader035.vdocuments.net/reader035/viewer/2022062401/5a4d1aee7f8b9ab05997c5bc/html5/thumbnails/10.jpg)
10
Country as a categorical factor
Often criticised: • Appears to be overly simplisticHowever • Same as other QDA factors, eg gender,.. • Critics forget qualified interpretations that good
QDA makes: [these patterns] are associated with categories, all other things being equal.
• Bad QDA: forget controls for relevant other things
![Page 11: 1 MSc ASR, SR06 Session 9 Quantitative methods of social research for cross-national comparisons Paul Lambert, 5.2.02](https://reader035.vdocuments.net/reader035/viewer/2022062401/5a4d1aee7f8b9ab05997c5bc/html5/thumbnails/11.jpg)
11
Quantitative cross-national social research
1) Introduction
2) Three traditions in Qn cross-national research
3) Seven themes in Qn cross-national research
Case study 1: Secondary analysis of cross-national surveys
![Page 12: 1 MSc ASR, SR06 Session 9 Quantitative methods of social research for cross-national comparisons Paul Lambert, 5.2.02](https://reader035.vdocuments.net/reader035/viewer/2022062401/5a4d1aee7f8b9ab05997c5bc/html5/thumbnails/12.jpg)
12
A typology of quantitative cross-national research designs?
• Bryman 2001(p53): 4 types of cross-cultural research
• Ragin 1987: 2 analytical orientations, one mainly Qn, the other mainly Ql; proposed resolution with Qn-style summaries of Ql research
No typology is perfect – there is much overlap and ambiguity in methods – but it can be useful to classify patterns of modern social research…
![Page 13: 1 MSc ASR, SR06 Session 9 Quantitative methods of social research for cross-national comparisons Paul Lambert, 5.2.02](https://reader035.vdocuments.net/reader035/viewer/2022062401/5a4d1aee7f8b9ab05997c5bc/html5/thumbnails/13.jpg)
13
A popular two-stage story:
Eg: Hantrais and Mangen 96: moves to interpretive methods;Ragin 87: variable v’s case oriented approaches
Early quantitative researchers naively attempted to measure national differences as single variables. They badly misclassified or ignored important national level differences.
Much more thoughtful considerations of complex national contexts are needed, & often
these are more suited to qualitative research methods.
![Page 14: 1 MSc ASR, SR06 Session 9 Quantitative methods of social research for cross-national comparisons Paul Lambert, 5.2.02](https://reader035.vdocuments.net/reader035/viewer/2022062401/5a4d1aee7f8b9ab05997c5bc/html5/thumbnails/14.jpg)
14
This inaccurate simplification implies a false Qn/Ql division:
• Doesn’t reflect variety of current practice in QnXR (& indeed past practice)
• Doesn’t acknowledge multivariate QnXR• Doesn’t do justice to many carefully
conducted / reported QnXR projects• Tends to over-estimate QlXR capactity
![Page 15: 1 MSc ASR, SR06 Session 9 Quantitative methods of social research for cross-national comparisons Paul Lambert, 5.2.02](https://reader035.vdocuments.net/reader035/viewer/2022062401/5a4d1aee7f8b9ab05997c5bc/html5/thumbnails/15.jpg)
15
A picture of QnXR under this typology:
Crude variable oriented
Case oriented
![Page 16: 1 MSc ASR, SR06 Session 9 Quantitative methods of social research for cross-national comparisons Paul Lambert, 5.2.02](https://reader035.vdocuments.net/reader035/viewer/2022062401/5a4d1aee7f8b9ab05997c5bc/html5/thumbnails/16.jpg)
16
Multitude of contemporary social research examples don’t fit this
• There are a great many quantitative case-oriented designs
• It is unfair to describe all variable-oriented designs as inadequate
• ..though to be fair, many variable-oriented projects are genuinely weak!
![Page 17: 1 MSc ASR, SR06 Session 9 Quantitative methods of social research for cross-national comparisons Paul Lambert, 5.2.02](https://reader035.vdocuments.net/reader035/viewer/2022062401/5a4d1aee7f8b9ab05997c5bc/html5/thumbnails/17.jpg)
17
A fairer typology of QnXR
Crude variable-oriented
Sophisticated variable oriented
Case oriented
![Page 18: 1 MSc ASR, SR06 Session 9 Quantitative methods of social research for cross-national comparisons Paul Lambert, 5.2.02](https://reader035.vdocuments.net/reader035/viewer/2022062401/5a4d1aee7f8b9ab05997c5bc/html5/thumbnails/18.jpg)
18
Crude variable orientedEarly or recent, micro- and macro- research making
claims over country level differences, with: • Insufficient exploration of relevant explanatory
factors• Limited or poor quality variable
operationalisations & discussions• Relevant national contexts not appreciated • False assumptions of good harmonisation
Example: see the illustrated analysis using the ESS
![Page 19: 1 MSc ASR, SR06 Session 9 Quantitative methods of social research for cross-national comparisons Paul Lambert, 5.2.02](https://reader035.vdocuments.net/reader035/viewer/2022062401/5a4d1aee7f8b9ab05997c5bc/html5/thumbnails/19.jpg)
19
Sophisticated variable orientedEarly or recent micro- and macro- research making
claims over country level differences, with: • Sufficient exploration of relevant explanatory
factors• Good quality variable operationalisations and
discussions• Relevant national contexts suitably described• Accurate assumptions of good harmonisation
Example: more applications than is often realised…
![Page 20: 1 MSc ASR, SR06 Session 9 Quantitative methods of social research for cross-national comparisons Paul Lambert, 5.2.02](https://reader035.vdocuments.net/reader035/viewer/2022062401/5a4d1aee7f8b9ab05997c5bc/html5/thumbnails/20.jpg)
20
Case orientedQn analyses within countries, then outcomes
evaluated between countries by authors / readers
• Doesn’t require strong assumptions of data harmonisation
• Expertise of report writer covers national context
Examples: Edited books; centrally coordinated projects; end user reviews; …
![Page 21: 1 MSc ASR, SR06 Session 9 Quantitative methods of social research for cross-national comparisons Paul Lambert, 5.2.02](https://reader035.vdocuments.net/reader035/viewer/2022062401/5a4d1aee7f8b9ab05997c5bc/html5/thumbnails/21.jpg)
21
Sophisticated variable oriented
• Attractive method: – offers parsimony of XN summary– uses large scale resources
• Methodology for good conduct necessary– Reliability, validity, implementation, translation– Sample design– Reporting strategy and claims
• Boundary to crude research subjective / contested• Existence often denied by anti-Qn sociologists…
![Page 22: 1 MSc ASR, SR06 Session 9 Quantitative methods of social research for cross-national comparisons Paul Lambert, 5.2.02](https://reader035.vdocuments.net/reader035/viewer/2022062401/5a4d1aee7f8b9ab05997c5bc/html5/thumbnails/22.jpg)
22
Why not be over-cautious?
• Case oriented QnXR seems a safe bet?Doesn’t make claims not justifiedBut doesn’t make much impact either
• Remains need for good variable oriented: Offers a parsimonious summary of
national differencesGovt / media with utilise regardless
![Page 23: 1 MSc ASR, SR06 Session 9 Quantitative methods of social research for cross-national comparisons Paul Lambert, 5.2.02](https://reader035.vdocuments.net/reader035/viewer/2022062401/5a4d1aee7f8b9ab05997c5bc/html5/thumbnails/23.jpg)
23
Quantitative cross-national social research
1) Introduction
2) Three traditions in Qn cross-national research
3) Seven themes in Qn cross-national research
Case study 1: Secondary analysis of cross-national surveys
![Page 24: 1 MSc ASR, SR06 Session 9 Quantitative methods of social research for cross-national comparisons Paul Lambert, 5.2.02](https://reader035.vdocuments.net/reader035/viewer/2022062401/5a4d1aee7f8b9ab05997c5bc/html5/thumbnails/24.jpg)
24
3.1) Data availability
• Massive increases in data resources accessible to social researchers
– Secondary survey datasets– Official statistics resources– Internet provision / communications
• Many data resources under-exploited• Most data originates from survey sources
- but some exceptions
![Page 25: 1 MSc ASR, SR06 Session 9 Quantitative methods of social research for cross-national comparisons Paul Lambert, 5.2.02](https://reader035.vdocuments.net/reader035/viewer/2022062401/5a4d1aee7f8b9ab05997c5bc/html5/thumbnails/25.jpg)
25
3.2) Dataset complexity
• Secondary surveys tend to feature– Many variables and cases– Complex variable operationalisation choices– Complex structuring (eg multiple hierarchies)– Complex weighting / sampling information– Data analysis & management software needs
• Aggregate statistics’ features– Difficulty understanding source derivation
![Page 26: 1 MSc ASR, SR06 Session 9 Quantitative methods of social research for cross-national comparisons Paul Lambert, 5.2.02](https://reader035.vdocuments.net/reader035/viewer/2022062401/5a4d1aee7f8b9ab05997c5bc/html5/thumbnails/26.jpg)
26
3.3) Variable operationalisation
• Single biggest issue in most QnXR conduct – Survey design– Dataset analysis – Result reporting
• Models of comparability– Exact equivalence of measures– Relativistic equivalence of meanings– Wide literature on ‘reliability’, ‘validity’ of X-N
variable measures and aggregate statistics
![Page 27: 1 MSc ASR, SR06 Session 9 Quantitative methods of social research for cross-national comparisons Paul Lambert, 5.2.02](https://reader035.vdocuments.net/reader035/viewer/2022062401/5a4d1aee7f8b9ab05997c5bc/html5/thumbnails/27.jpg)
27
Variable harmonisation ctd
• Choices over key variables allow use of previous literatures (eg H-Z & Wolf 2003).
Eg measures of income; occupation; ethnic group; education; region; crime; health; ..
• Choices over specific analytical variables require new efforts
Eg, attitude harmonisations of Inglehart.
![Page 28: 1 MSc ASR, SR06 Session 9 Quantitative methods of social research for cross-national comparisons Paul Lambert, 5.2.02](https://reader035.vdocuments.net/reader035/viewer/2022062401/5a4d1aee7f8b9ab05997c5bc/html5/thumbnails/28.jpg)
28
3.4) Survey design
Harkness et al 2003:
Ex post facto harmonisation (more widespread, eg Eurostat, IPUMS, LIS)
v’s Coordinated design, sampling, & implementation
(big money projects, eg ESS, ISSP)Latter as preferable – but whilst many projects
attempt this model, far fewer succeed...
![Page 29: 1 MSc ASR, SR06 Session 9 Quantitative methods of social research for cross-national comparisons Paul Lambert, 5.2.02](https://reader035.vdocuments.net/reader035/viewer/2022062401/5a4d1aee7f8b9ab05997c5bc/html5/thumbnails/29.jpg)
29
3.5) Conduct and logistics• High costs of coordinated surveys• Considerable efforts, and many errors, in ex
post facto harmonisation • Issues of cooperating with colleagues /
diverging academic traditions, eg – different views data access / confidentiality– Technical / software compatibility – different organisations involved in survey production
QnXR can be very slow process
![Page 30: 1 MSc ASR, SR06 Session 9 Quantitative methods of social research for cross-national comparisons Paul Lambert, 5.2.02](https://reader035.vdocuments.net/reader035/viewer/2022062401/5a4d1aee7f8b9ab05997c5bc/html5/thumbnails/30.jpg)
30
3.6) Temptation
Cross-national datasets nearly always look simpler than they really are
dangerous temptation to rush into uncritical variable analysis
![Page 31: 1 MSc ASR, SR06 Session 9 Quantitative methods of social research for cross-national comparisons Paul Lambert, 5.2.02](https://reader035.vdocuments.net/reader035/viewer/2022062401/5a4d1aee7f8b9ab05997c5bc/html5/thumbnails/31.jpg)
31
3.7) Prejudice
• Prejudices against quantitative methods pronounced in European sociology, especially wrt cross-national comparisons
– QnXR evidence often ignored– QnXR researchers portrayed as simplistic
• Prejudices favouring quantitative methods often seen in governmental and media organisations
– Mainly: uncritical acceptance of harmonisations
![Page 32: 1 MSc ASR, SR06 Session 9 Quantitative methods of social research for cross-national comparisons Paul Lambert, 5.2.02](https://reader035.vdocuments.net/reader035/viewer/2022062401/5a4d1aee7f8b9ab05997c5bc/html5/thumbnails/32.jpg)
32
Quantitative cross-national social research
1) Introduction
2) Three traditions in Qn cross-national research
3) Seven themes in Qn cross-national research
Case study 1: Secondary analysis of cross-national surveys
![Page 33: 1 MSc ASR, SR06 Session 9 Quantitative methods of social research for cross-national comparisons Paul Lambert, 5.2.02](https://reader035.vdocuments.net/reader035/viewer/2022062401/5a4d1aee7f8b9ab05997c5bc/html5/thumbnails/33.jpg)
33
Some leading secondary surveys:(see handout for internet links)
ESS ISSP
IPUMS LIS / LES / LWS
ECHP / CHER / PACO WVS / EVS
Eurobarometer Education: PISA / TIMSS
Social Stratification: CASMIN / CCAP / …
Health /Welfare: eg SVH
![Page 34: 1 MSc ASR, SR06 Session 9 Quantitative methods of social research for cross-national comparisons Paul Lambert, 5.2.02](https://reader035.vdocuments.net/reader035/viewer/2022062401/5a4d1aee7f8b9ab05997c5bc/html5/thumbnails/34.jpg)
34
European Social Survey
• New annual attitudes / values / social circumstances cross-sections, 2002
• Equivalence of design and survey implementation between countries
• Extensive methodological resources• Free access to data
![Page 35: 1 MSc ASR, SR06 Session 9 Quantitative methods of social research for cross-national comparisons Paul Lambert, 5.2.02](https://reader035.vdocuments.net/reader035/viewer/2022062401/5a4d1aee7f8b9ab05997c5bc/html5/thumbnails/35.jpg)
35
Analysis (see SPSS syntax eg)
• Opens harmonised files from 15 countries in 2002• Select variables measuring attitudes, age, gender
and educational levels• Generate tables of patterns split by countries• Use regression models to evaluate contribution of
mulitiple explanatory factors:– Country specific ‘structural breaks’– Country effects as variables / interactions
![Page 36: 1 MSc ASR, SR06 Session 9 Quantitative methods of social research for cross-national comparisons Paul Lambert, 5.2.02](https://reader035.vdocuments.net/reader035/viewer/2022062401/5a4d1aee7f8b9ab05997c5bc/html5/thumbnails/36.jpg)
36
Liberal attitudes to homosexuality and their associations with educational level
(national average and Cramer’s V to educ)
% CV % CV
Switzerland 81 10 Israel 59 20Czech Rep 58 11 Netherlands 88 6Spain 70 20 Norway 77 13Finland 62 14 Poland 46 16UK 75 7 Portugal 71 15Greece 51 23 Sweden 82 12Hungary 48 5 Slovenia 52 18Ireland 82 8
![Page 37: 1 MSc ASR, SR06 Session 9 Quantitative methods of social research for cross-national comparisons Paul Lambert, 5.2.02](https://reader035.vdocuments.net/reader035/viewer/2022062401/5a4d1aee7f8b9ab05997c5bc/html5/thumbnails/37.jpg)
37
Log-regression prediction of liberalism to homosexuality for ESS adults
(value & significance of coefficient estimate) Age-squared -1.72** Interactions:
Low educ -0.31** Low educ*NW 0.19**
High educ 0.35** Female*NW 0.35**
Female 0.21** Female*South -0.15*
North West 1.07** Contrast: medium education male from eastern European country. Southern 0.56**
![Page 38: 1 MSc ASR, SR06 Session 9 Quantitative methods of social research for cross-national comparisons Paul Lambert, 5.2.02](https://reader035.vdocuments.net/reader035/viewer/2022062401/5a4d1aee7f8b9ab05997c5bc/html5/thumbnails/38.jpg)
38
This is ‘crude’ variable oriented
• Didn’t try out sufficient relevant explanatory factors
• Didn’t check variable choices extensively• Merged variable categories for convenience• Didn’t use survey weights• Didn’t contextualise reporting with sufficient
substantive national background and cross-examinations of data sources and measures
![Page 39: 1 MSc ASR, SR06 Session 9 Quantitative methods of social research for cross-national comparisons Paul Lambert, 5.2.02](https://reader035.vdocuments.net/reader035/viewer/2022062401/5a4d1aee7f8b9ab05997c5bc/html5/thumbnails/39.jpg)
39
..but it could have been sophisticated variable oriented
• Could have evaluated variable meanings• Could have studied backgrounds• Could have added more explanatory factors• Could have reported more carefully
• .. Research consumption = understanding how well the results were prepared
![Page 40: 1 MSc ASR, SR06 Session 9 Quantitative methods of social research for cross-national comparisons Paul Lambert, 5.2.02](https://reader035.vdocuments.net/reader035/viewer/2022062401/5a4d1aee7f8b9ab05997c5bc/html5/thumbnails/40.jpg)
40
Summary on Quantitative cross-national research
Quant methods contribute to both ‘variable’ & ‘case’ oriented comparisons
Crude variable oriented widely criticised, and many bad examples persist
Sophisticated variable oriented research can be found, and represents most attractive format of QnXR