Research design: comparative and case studies

HEM 4112 – Research methods I

Martina Vukasovic

Overview of session

• Case studies

• Comparative studies

• Causal effect and causal mechanism

• Methods of comparison

• Indeterminate research design

• Literature: Bryman, chapter 2; notes from the lecture; suggested advanced literature

Case study (1)• Intensive study of one single community, group, individual,

organization, person, event... – the context is important– the specific case should not be considered as a sample where N=1

• Nomothetic/theory-centred (a) vs. idiographic/case-centred (b):

• are you interested in something that somehow corresponds to a wider set of phenomena (a)

• for example: a case study of organizational change (in three flagship universities in the former Yugoslavia)

– or are you interested in what is unique about that particular case (b)

(this classification departs a bit from Bryman!)

Case study (2)• Case study – research design, not research

methodology• Both qualitative and quantitative (and mixed)

methods can be used in a case study – depends on the research questions asked and on the

goal of the case study

• But – more case studies employ qualitative methods, since

the study is supposed to be intensive and the context matters

• Focus on how and why, not just what

Case study (3)Types of case studies (1)

• Bryman: – critical – unique – revelatory– exemplifying

• Nomothetic vs. idiographic– if nomothetic: most-likely

or least-likely cases of...

• Lijphart (1971)*: – atheoretical

– interpretative

– hypothesis-generating

– theory-confirming or theory-infirming

– deviant

* Lijphart, A. (1971) Comparative Politics and the Comparative Method. In The American Political Science Review, vol. 65, n. 3, pp. 682-693

Case study (4)

• Atheoretical – no theory used, the goal is to provide a thick description• Interpretative – the goal is also to describe, but using theory• Hypothesis-generating – starts with a very vague idea about

hypothesis and ends up with one or more hypotheses that should be tested in future research; similar to revelatory

• Theory-confirming (a) or theory-infirming (b) – tries to see if an already established theory “works” on a previously not analysed case; can result in a confirmation of a theory (a) or its modification (b)

• Deviant – a case that “is not behaving” as it is supposed to, the theory does not “work” and the modifications necessary to make it work are too big, can result in rejection of old and building of new theory, similar to critical case

Types of case studies (2)

Case study (5)

• Critical – a case in which a hypothesis does not hold, similar to deviant cases

• Unique – interesting because of peculiarities of the case/context

• Revelatory – a context/phenomena which was not accessible for studying before, could be used for hypothesis building

• Exemplifying – similar to a nomothetic case study, interesting because it sheds more light on some phenomena

Types of case studies (3)

Case study (6)

• Most likely vs. least likely– important for assessing whether or not (and how much) you can

generalize from a case study

• Most likely case of...– A case that is most likely to be in line with a theoretical perspective– If it does ... So what?– If it does not ..... Yippieeee!

• Least likely case of...– A case that is least likely to be in line with a theoretical perspective– If it does not ... So what?– If it does ..... Yippieeee!

Types of case studies (4)

Case study (7)

• Most likely vs. least likely, an example: – “Assumption”: flagship universities as least likely cases of

organizational change in higher education institutions– Hypothesis: change in universities in the post-Communist

countries is determined by the pressures coming from the European level (e.g. Bologna Process)

– Assumption + hypothesis if hypothesis is confirmed for flagship universities – if flagship universities change due to the pressure coming from the European level – there is sufficient ground to claim that it will be confirmed for other higher education institutions

Types of case studies (5)

Case study (8)

• How to determine what type of case study are you doing? Should you? Is this essential?

• What are the previous classifications good for?– The types are ideal types – a particular study can possibly fit in more

than one type– The types are more useful for understanding and explaining to others

the different aspects of your study– Some choices imply a particular ontological or epistemological

position

• Back to research problem and research questions– Ask yourself: how much do you use/rely on theory, why is that

particular case interesting for you, what are your aims ... ?

Types of case studies (6)

Comparative study (1)• A comparison of two or more instances of (the same)

phenomena– “two or more instances” similar to “two or more cases” or

“two or more contexts”

• Cases need to be comparable– you need to be able to argue convincingly that the

phenomena in all instances is the same or at least very similar

– you need to demonstrate that you used the same research approach in terms of type of data collected and methods of analysis used

• Strongly linked to theory

Comparative study (2) - comparability

• Concepts - definitions and operationalisations– participation rate in HE may be defined as gross-enrolment ratio

(GER), as net-enrolment ratio (NER) or in some other way

• Data – what is measured/observed and how?– in country A GER is calculated only for universities, while in

country B GER includes also students studying in the non-university sector

• Context - are the phenomena/process that you want to study really the same in all contexts?– in country A governance of HE includes a federal, state and local

level of governance (e.g. USA), while in country B it includes a supranational, national and regional level of HE (e.g. Spain)

Questions to ponder

• Recall the distinction between objectivist and constructivist perspective as well as insider and outside perspectives (day 1)

• Where would you put comparative studies?

• Why?

Causal effect and causal mechanism (1)

• Comparative studies and some case studies may be constructed in terms “suspected” causes (Xs) and effects (Ys)– Quantitative approach: X – independent variable, Y –

dependent variable

• Causality – Cause precedes effect– There is a regular connection between cause and effect– A process that embodies this connection can be

identified

Causal effect and causal mechanism (2)

• The use of “cause” and “effect” during analysis is, strictly speaking, premature– Hence “suspected” cause and sometimes “expected”

effect

• Essentially two steps in the analysis:1.compare across cases to see the patterns in Xs and

Y causal effect of X and Y

2.analyse within cases to see what steps (events, processes) link X and Y a causal mechanism

Causal effect and causal mechanism (3)

• Causal mechanism– What links X and Y? How does X lead to Y? Why?

– A series of intermediary “events” (Is):

X I1 I2 I3 .... I10 .... Y

– Usually done through a technique called process-tracing • A number of sources of information can be used in process-tracing

(interviews, documents, statistical data, observation...)

– An X and a Y can be linked with different causal mechanisms in different cases, i.e. the causal mechanism may depend on the context, BUT that does not change the causal effect

Causal effect and causal mechanism (4)

Case

X1 – European pressures for change

X2 – national

pressures for change

X3 – internal/

institutional pressures for

change

Y – organisational

change

University A

University B

University C

Causal effect and causal mechanism (4)

Case

X1 – European pressures for change

X2 – national

pressures for change

X3 – internal/

institutional pressures for

change

Y – organisational

change

University A +University B +University C -

Causal effect and causal mechanism (4)

Case

X1 – European pressures for change

X2 – national

pressures for change

X3 – internal/

institutional pressures for

change

Y – organisational

change

University A + +University B + +University C + -

Causal effect and causal mechanism (4)

Case

X1 – European pressures for change

X2 – national

pressures for change

X3 – internal/

institutional pressures for

change

Y – organisational

change

University A + + +University B + + +University C - + -

Causal effect and causal mechanism (4)

Case

X1 – European pressures for change

X2 – national

pressures for change

X3 – internal/

institutional pressures for

change

Y – organisational

change

University A + + - +University B + + - +University C - + - -

Causal effect and causal mechanism (5)

• Words of caution (1):– The table presents a simplified perspective, it

should not be used as the only tool• But can point towards interesting patterns

– Often not possible to provide a clear + or –• Sometimes a better description would be e.g.

strong, intermediate, weak• Lead to development of QCA and fuzzy sets

analysis**Ragin, C.C. (1987) The Comparative Methods: Moving beyond

Qualitative and Quantitative Strategies. Berkeley: University of California Press

Causal effect and causal mechanism (6)

• Words of caution (2)– Some “suspected” causes may be overlooked when

designing research – Some causes may “act together” – consider whether

there is some theoretical grounding in “merging” these causes together

– A particular research design may not give clear cut indications of what are the suspects for causes

Task to work in groups• Compare across cases in the examples for

regularities between Xs and Y• Which X you think is a good suspect for a cause

of the effect Y?

Case X1 X2 X3 Y

A + + - +

B + + - +

C - + - -

Comparison methods

• Classical methods: MoA and MoD– Comparisons of pairs of cases

• Method of Agreement (MoA)– Two cases have the same “score” (+ or -) on the effect (Y) and

different “scores” for all causes (Xs) except one

• Method of Difference (MoD)– Two cases have different “scores” on the effect (Y) and similar

“scores” for all the causes (Xs) except for one

• Indirect method of difference (Ragin 1987)– Combination of MoA and MoD

Task to work in groups

• Which method(s) did you use for Task 1?

• Use MoA, MoD or a combination for the examples in the handout

• Solve the other 4 problems

Indeterminate research design for comparative studies (1)

• Suspects for causes: – lack of X2 alone OR– presence of X3 alone OR– lack of X2 combined with presence of X3?

Case X1 X2 X3 Y

A + - + +

B - - + +

C + + - -

D - + - -

Indeterminate research design for comparative studies (2)

• How to deal with them?– Add more cases

• But that complicates the study

– Add more “suspects” for causes• But that also complicates the study

– Check what the theory suggests– Dive into within-case analysis, i.e. analyse the causal

mechanismsor

– Conclude that the design was indeterminate and write that this is a good topic for further research

Concluding remarks...

• Do not fill intimidated by –isms, -ologies etc.

• Research primarily requires:– Discipline and systematic approach to data

collection and data analysis– Logical reasoning when developing

inferences (think as a detective!)– Strong argumentation of inferences– Good writing skills