process tracing methods – an introduction · 1. what is process tracing? ‘the cause‐effect...
TRANSCRIPT
AARHUS UNIVERSITY
DEPARTMENT OF POLITICAL SCIENCE
Process Tracing methods – an introduction Ph.D. workshop
University of Konstanz, Germany
March 16, 2012
Derek Beach, PhD Associate Professor Department of Political Science University of Aarhus, Denmark
Email: [email protected]
AARHUS UNIVERSITY
DEPARTMENT OF POLITICAL SCIENCE
Outline
1. WhatisProcessTracing?
2. Whatarecausalmechanisms?
3. ThreevariantsofPT
4. CausalinferenceinPT
5. Studyingcausalmechanisms?
6. WhencanPTbeused,andnotused?
2
AARHUS UNIVERSITY
DEPARTMENT OF POLITICAL SCIENCE
1.WhatisProcesstracing?
Single case researchmethod that canbeused tomakewithin‐case inferencesabout
presence/absenceofcausalmechanisms
3
AARHUS UNIVERSITY
DEPARTMENT OF POLITICAL SCIENCE
1.WhatisProcesstracing?
‘thecause‐effect linkthatconnects independentvariableandoutcomeisunwrapped
anddividedintosmallersteps;thentheinvestigatorlooksforobservableevidence
ofeachstep.’(VanEvera1997:64).
‐focusisonstudyingcausalmechanismsusingindepthsinglecasestudy
4
AARHUS UNIVERSITY
DEPARTMENT OF POLITICAL SCIENCE 5
AARHUS UNIVERSITY
DEPARTMENT OF POLITICAL SCIENCE
1.WhatisProcesstracing?
KKV,Gerring–casestudymethodsmoreanalogoustomedicalexperiment
‐inperfectworldmeasureeffectoftandconsameunit(UtandUc)
‐analyzemeancausaleffects
PT–closertocriminaltrial
‐ evidence assessed for each part of explanation (mechanism) to detect whether it can be concluded
beyondreasonabledoubtthatmechanismexisted
6
AARHUS UNIVERSITY
DEPARTMENT OF POLITICAL SCIENCE
2.Whatarecausalmechanisms?
:atheoryofasystemofinterlockingpartsthattransmitscausalforcesfromXtoY
(Glennan,1996,2002;Bunge,1997,2004;Bhaskar,1979).
7
AARHUS UNIVERSITY
DEPARTMENT OF POLITICAL SCIENCE 8
AARHUS UNIVERSITY
DEPARTMENT OF POLITICAL SCIENCE 9
USgovernmentworksforensuringan‘OpenDoor’,de^inedasaninternationalpoliticalsystem
conducivetotradeand
investment,inWesternEuropeUSstrivesfor
singlemarket
USdecisionmakersbelievethatprosperityisthekeytoUSsecurity
USdecisionmakersbelievethatUSprosperitydependson
foreignmarkets,inparticularontheeconomicrevivalof
WesternEuropeafterWWII
(withclosure,thefearisthatUSwouldneedaregimented,state‐plannedeconomy)
USgovernmentusestoolsavailabletopressureWesternEuropetoadopt
economicopenness(e.g.usingtheMarshall
Plan)
USgrandstrategy=
extraregionalhegemony‐USactsasregional
stabilizerinWesternEurope‐USensuresthatcountriesaregovernedby‘rightkind’ofgovernment
RelativepowerofUSvisavisothergreatpowers
X Causalmechanism(OpenDoor) outcome
Layne’s case-specific Open Door mechanism
AARHUS UNIVERSITY
DEPARTMENT OF POLITICAL SCIENCE
Regularityunderstandingofcausality
‘…thedifferencebetweenthesystematiccomponentofobservationsmadewhenthe
explanatoryvariabletakesonevalueandthesystematiccomponentofcomparable
observationswhentheexplanatoryvariablestakesonanothervalue.’
(King,KeohaneandVerba,1994:81‐82,italicsadded).
10
2.Whatarecausalmechanisms?
AARHUS UNIVERSITY
DEPARTMENT OF POLITICAL SCIENCE
Mechanismicunderstandingofcausality
‐ Openup‘blackbox’betweenXandY
‐the dynamic, interactive in^luence of causes upon outcomes, and in particular how
causal forces are transmitted through a series of interlocking parts of a causal
mechanismtocontributetoproduceanoutcome.
11
2.Whatarecausalmechanisms?
AARHUS UNIVERSITY
DEPARTMENT OF POLITICAL SCIENCE
2.Whatarecausalmechanisms?
‘…Amechanismisasetof interactingparts–anassemblyofelementsproducingan
effectnotinherentinanyoneofthem.Amechanismisnotsomuchabout‘nutsand
bolts’ asabout ‘cogsandwheels’ – thewheelworkoragencybywhichaneffect is
produced.’(Hernes,1998:78,italicsadded)
12
AARHUS UNIVERSITY
DEPARTMENT OF POLITICAL SCIENCE
2.Whatarecausalmechanisms?
Parts = factors that are individually necessary parts of mechanism, composed of
entitiesthatengageinactivities(notinterveningvariables!)
Entities=objectengaginginactivities(noun)
Activities=producersofchangeorwhattransmitscausalforcesthroughCM(verbs)
13
AARHUS UNIVERSITY
DEPARTMENT OF POLITICAL SCIENCE
2.Whatarecausalmechanisms?
14
X Y
Scope conditions
*
causal mechanism
activities
entities
part 1 part 2
noun
verb
noun
verb
AARHUS UNIVERSITY
DEPARTMENT OF POLITICAL SCIENCE
2.Whatarecausalmechanisms?
‐ MechanismsareNOTaseriesofinterveningvariables
‐ (examplefromRosato,2003:585)
15
Democracy accountability groupconstraint Peace
Independentvariable
Dependentvariable
Causalmechanism
Interveningvariable1
Interveningvariable2
AARHUS UNIVERSITY
DEPARTMENT OF POLITICAL SCIENCE
2.Whatarecausalmechanisms?
16
AARHUS UNIVERSITY
DEPARTMENT OF POLITICAL SCIENCE
Discussion
1. Developaplausiblecausalmechanismthatcanexplainwhyeconomic
development(X)contributestoproducedemocratization(Y)throughthecreation
ofaneducatedmiddleclass.
17
AARHUS UNIVERSITY
DEPARTMENT OF POLITICAL SCIENCE
3.ThreevariantsofProcessTracing
1. Theory‐testing
2. Theory‐building
3. Explainingoutcome
18
AARHUS UNIVERSITY
DEPARTMENT OF POLITICAL SCIENCE
3.ThreevariantsofProcessTracing
19
AARHUS UNIVERSITY
DEPARTMENT OF POLITICAL SCIENCE 20
Theory‐testing
AARHUS UNIVERSITY
DEPARTMENT OF POLITICAL SCIENCE 21
Theory‐building
AARHUS UNIVERSITY
DEPARTMENT OF POLITICAL SCIENCE 22
Explainingoutcome
Empirical,casespeci.ic
level
’Facts’ofthecase(egasempiricalnarrative)
Inductivepath
Suf^icientexplanationofoutcome?
Deductivepath
either
Continueuntil
suf^icientexplanation
Theoreticallevel
Causalmechanisms=>systematicCM,case‐speci^ic(non‐systematic)CM,case‐speci^iccombinationofsystematicCM(eclectictheorization)
1
3
1
2
3
AARHUS UNIVERSITY
DEPARTMENT OF POLITICAL SCIENCE
4.CausalinferenceinPT
‐ KKV,Gerringsuggestthatthereisonelogicofinferenceinallpoliticalscience
‘thedifferencesbetweenthequantitativeandqualitativetraditionsareonlystylisticand
are methodologically and substantively unimportant. All good research can be
understood–indeed,isbestunderstood–toderivefromthesameunderlyinglogicof
inference.’(King,KeohaneandVerba,1994:4).
AARHUS UNIVERSITY
DEPARTMENT OF POLITICAL SCIENCE 24
X n1 n2 n3 Y
observable implications of
each part
AARHUS UNIVERSITY
DEPARTMENT OF POLITICAL SCIENCE
4.CausalinferenceinPT
AARHUS UNIVERSITY
DEPARTMENT OF POLITICAL SCIENCE
4.CausalinferenceinPT
‐ Bayesianlogicofinference=analystgivesgreaterweighttoevidencethatisexpected
aprioritobelessprobablebaseduponourpreviousknowledgeofphenomenon.
‐ ‘What is important isnotthenumberofpiecesofevidencewithinacasethat ^itone
explanationoranother,butthelikelihoodof^indingcertainevidenceifatheoryistrue
versus the likelihood of ^inding this evidence if the alternative explanation is
true.’(Bennett2006:341).
AARHUS UNIVERSITY
DEPARTMENT OF POLITICAL SCIENCE
4.CausalinferenceinPT
Bayes’formula
posteriorprobability=priorprobabilityxlikelihoodratio
27
AARHUS UNIVERSITY
DEPARTMENT OF POLITICAL SCIENCE
4.CausalinferenceinPT
posteriorprobability=theposteriorprobabilityofthedegreeofcon^idencewehave
inthevalidityofahypothesis(h)abouttheexistenceofapartofacausalmechanism
aftercollectingevidence(e).
p(h│e)
28
AARHUS UNIVERSITY
DEPARTMENT OF POLITICAL SCIENCE
4.CausalinferenceinPT
Prior=degreeofcon^idencethattheresearcherhasinthevalidityofahypothesis
priortogatheringevidence,baseduponexistingtheorization,empiricalstudiesand
otherformsofexpertknowledge.
p(h)
29
AARHUS UNIVERSITY
DEPARTMENT OF POLITICAL SCIENCE
4.CausalinferenceinPT
Likelihood ratio = expected probability of ^inding evidence supporting a hypothesis
basedupontheresearcher’sinterpretationoftheprobabilityof^indingitinrelation
to the hypothesis and background knowledge informed by previous studies
(p(e│h), compared with the expected probability of ^inding the evidence if the
hypothesisisnottrue(p(e│~h).
p(e│~h)/p(e│h)
30
AARHUS UNIVERSITY
DEPARTMENT OF POLITICAL SCIENCE
4.CausalinferenceinPT
Bayes’formula
31
p(h|e)= p(h)
p(h)+p(e|~h)*p(~h)p(e|h)
AARHUS UNIVERSITY
DEPARTMENT OF POLITICAL SCIENCE
4.CausalinferenceinPT
SilverBlazeexample–testingpartofmechanism(whetherhorseabductedbyinsider)
‐Prior=low(whywouldinsiderkidnapownhorse!)=20%(p(~h)=80%)
‐Likelihoodoftest=p(e|h)=90%,p(e|~h)=10%
32
0.692= 0.2
0.2+(0.1/0.9)*0.8
AARHUS UNIVERSITY
DEPARTMENT OF POLITICAL SCIENCE
4.CausalinferenceinPT
Whatif50‐50test?
‐Prior=low=20%(p(~h)=80%)
‐ Likelihoodoftest=p(e|h)=50%,p(e|~h)=50%
33
p(h|e)= p(h)
p(h)+p(e|~h)*p(~h)p(e|h)
AARHUS UNIVERSITY
DEPARTMENT OF POLITICAL SCIENCE
4.CausalinferenceinPT
Whatifhighcon^idenceinprior?
‐Prior=low=70%(p(~h)=30%)
‐ Likelihoodoftest=p(e|h)=80%,p(e|~h)=20%
34
p(h|e)= p(h)
p(h)+p(e|~h)*p(~h)p(e|h)
AARHUS UNIVERSITY
DEPARTMENT OF POLITICAL SCIENCE
5.Studyingcausalmechanisms
‐ developstrongempiricaltestsforwhetherallpartsofcausalmechanismarepresent
ornot
‐ logicofempiricaltestinginprocesstracing=>ifweexpectedXtocauseY,eachpart
of the mechanism between X and Y should leave the predicted empirical
manifestationswhichcanbeobservedintheempiricalmaterial.
35
AARHUS UNIVERSITY
DEPARTMENT OF POLITICAL SCIENCE
5.Studyingcausalmechanisms
‐Detectingthesemanifestations=>developmentofcarefullyformulatedpredictions
ofwhatevidenceweshouldexpecttoseeifthehypothesizedpartofthemechanism
exists
‐PredictionstranslatetheoreticalconceptsofthecausalmechanismintocasespeciFic
observablemanifestations(expectedevidence).
36
AARHUS UNIVERSITY
DEPARTMENT OF POLITICAL SCIENCE
5.Studyingcausalmechanisms
Empiricalpredictions‐4differenttypesofevidence
1. Patternevidence=statisticalpatternsintheevidence.
2. Sequenceevidence=temporalandspatialchronologyofevents
3. Traceevidence=mereexistenceprovidesproof
4. Accountevidence=contentofempiricalmaterial
37
AARHUS UNIVERSITY
DEPARTMENT OF POLITICAL SCIENCE
5.Studyingcausalmechanisms
‐ uniquepredictions=>empiricalpredictionsthatdonotoverlapwiththoseofother
theories=>con^irmatorypowerifefound
‐ Uniqueness corresponds to the likelihood ratio, where predictions are developed
thatmaximizethevalueofp(e|h)inrelationtop(e|~h).
38
AARHUS UNIVERSITY
DEPARTMENT OF POLITICAL SCIENCE
5.Studyingcausalmechanisms
‐ certain prediction => prediction is unequivocal and the prediction (e) must be
observedorelsethetheoryfailstheempiricaltest=>discon^irmatorypowerifenot
found
39
AARHUS UNIVERSITY
DEPARTMENT OF POLITICAL SCIENCE 40
Certainty(ifenotfound~discon>irmatorypower)
Uniqueness(ifefound–con>irmatory
power)
High
High
Low
Low
‘Hoop’tests‘Doubly‐decisive’
tests
‘Smoking‐gun’tests‘Straw‐in‐the‐wind’
tests
AARHUS UNIVERSITY
DEPARTMENT OF POLITICAL SCIENCE
5.Studyingcausalmechanisms
strawinthewindtest=empiricalpredictionsthathavealowlevelofuniquenessand
alowlevelofcertainty(lowcon^irmatoryanddiscon^irmatorypower)
‐dolittletoupdateourcon^idenceinahypothesisirrespectiveofwhetherwe^indeor
~e,asbothpassedandfailedtestsareoflittleifanyinferentialrelevanceforus.
41
AARHUS UNIVERSITY
DEPARTMENT OF POLITICAL SCIENCE
5.Studyingcausalmechanisms
Hoop tests=predictions thatarecertainbutnotunique (lowcon^irmatoryandhigh
discon^irmatorypower)
‐ failure of test (^inding~e) reduces our con^idence in thehypothesis but ^indinge
doesnotenableupdating.
42
AARHUS UNIVERSITY
DEPARTMENT OF POLITICAL SCIENCE
5.Studyingcausalmechanisms
Smokinggun tests =highlyuniquebuthave loworno certainty in theirpredictions
(highcon^irmatoryandlowdiscon^irmatorypower)
‐ Likelihood ratio is small (^inding e given h highly probable whereas ~h is highly
improbable),therebygreatlyincreasingourcon^idenceinthevalidityofhifwe^ind
e.Ifnot^inde=>noupdating.
43
AARHUS UNIVERSITY
DEPARTMENT OF POLITICAL SCIENCE
5.Studyingcausalmechanisms
Doubly decisive tests => both certainty and unique (high con^irmatory and
discon^irmatorypower)
‐ evidence has to be found or our con^idence in the validity of the hypothesis is
reduced(updatingwhen~e)
‐ at the same time the test is able to discriminate strongly between evidence that
supportsthehypothesisandalternatives(smalllikelihoodratio),enablingupdating
whenwe^inde.44
AARHUS UNIVERSITY
DEPARTMENT OF POLITICAL SCIENCE
5.Studyingcausalmechanisms
45
Supranationalactorsenjoy
privilegedaccesstoorabilityto
processinformationand
ideas
Nationalgovernmentsunableorunwillingtoaccessand
processcriticalinformationandideas
Informationalasymmetries
inducebottlenecksinperformingthreekeytasks:policy
initiation,mediationand
socialmobilization
Supranationalactorscanmosteffectivelyinitiate,
mediateandmobilize
In^luenceofsupranationalactorsoninterstatebargaining
outcomesinEUnegotiations
Activitiesof
supranationalactors
X Causalmechanism(supranationalentrepreneurship) Y
AARHUS UNIVERSITY
DEPARTMENT OF POLITICAL SCIENCE
5.Studyingcausalmechanisms
Moravcsikexample:testof‘theCommissionhasprivilegedaccesstoinformation’.
‐ strawinthewind=‘expecttoseethattheCommissionhasmanycivilservants’
‐ strongertest=‘expecttoseethattheCommissioninthemostsensitiveareasof
negotiationswasmuchbetterinformedaboutthecontentandstate‐of‐playofthe
negotiationsthangovernments,possessingmoredetailedsubstantiveissuebriefsand
moreaccurateandupdatedinformationonthestateofplay’46
AARHUS UNIVERSITY
DEPARTMENT OF POLITICAL SCIENCE
Discussion
1. Operationalize an empirical test drawn from your own research,
describingtheuniquenessandcertainty.
47
Case study methodology – small-n research designs
Derek Beach, PhD
AARHUS UNIVERSITY
DEPARTMENT OF POLITICAL SCIENCE
6.TheusesofPT
48
AARHUS UNIVERSITY
DEPARTMENT OF POLITICAL SCIENCE
6.TheusesofPT–nesting?
• Systematicfactorsonlyincross‐case(Rohl^ing)• Deterministictheory
• LNAwhentraditionalstatisticalanalysis=probabilistic(meancausaleffectsacrosspopulation)• SNA(PT)=deterministicontology
• DivorcingXfromX+CM• canXbemeaningfullydivorcedfromCMifwePTstudiesaretocommunicatewithothermethods?• Arewestudyingtwodifferentthings:LNA=X:Y/PT=X+CM=>Y• Onesolution=usecon^igurationaltheories
• FxX=liberalideasorX1(liberalideas)+X2(liberalgroups)+X3(responsivegov)
49
AARHUS UNIVERSITY
DEPARTMENT OF POLITICAL SCIENCE
6.TheusesofPT–nesting?
• ExplainingoutcomePTcannotbenestedforseveralreasons:
1. Useofnon‐systematicfactorsinaccountingforY(minimalsuf^iciency)2. Eclectic,non‐systematic(case‐speci^iic)combinationoftheories,withtheoriesused
inpragmaticfashionasheuristictoolstoaccountforoutcome(moreidiographic
focus)
**deeplyinterestedinthecase
**howeverEOPTcanhavesomeexportable^indings–‘lessons’
50
AARHUS UNIVERSITY
DEPARTMENT OF POLITICAL SCIENCE
6.TheusesofPT–nesting?
• Theory‐testingstudiescanbenestedintwosituations1)havestrongX:YcorrelationfromLNAresearch
• DoesXcauseYinmannerpredictedbytheory?(Owen)
• Isthereacausalrelationship,orisitspurious?
2)well‐developedtheorybutisthereempiricalsupport(whensmallscopeofN)
**problemwithprobabilistic/deterministictheorization+whatwearestudying…
51
AARHUS UNIVERSITY
DEPARTMENT OF POLITICAL SCIENCE
6.TheusesofPT–nesting?
• Theory‐buildingstudiescanbenestedintwosituations1)havestrongX:YcorrelationfrompriorresearchbutnoideahowXcausedY
2)KnowYbutunclearaboutwhatcausedit(whatisX?)
**challengeofidentifyingnon‐systematicfactorsinsinglecasestudy
52
AARHUS UNIVERSITY
DEPARTMENT OF POLITICAL SCIENCE
Whatcasesarerelevantfor:
‐ Theory‐testingofeconomicdevelopment‐>democracy
‐ Theory‐buildingexplainingwhylowincomecountriescanbecomedemocratic
Case study methodology – small-n research designs
AARHUS UNIVERSITY
DEPARTMENT OF POLITICAL SCIENCE
Case study methodology – small-n research designs
AARHUS UNIVERSITY
DEPARTMENT OF POLITICAL SCIENCE 55
AARHUS UNIVERSITY
DEPARTMENT OF POLITICAL SCIENCE 56
AARHUS UNIVERSITY
DEPARTMENT OF POLITICAL SCIENCE
6.TheusesofPT
‐ Strongwithincaseinferencescanbemadeusingin‐depthsinglecasestudy
‐ NocrosscaseinferencescanbemadewithPT
‐ WhetherPTcanbeusedinconjunctionwithothermethodsdependsuponthe
variantofPT(yesfortheory‐testingandbuilding,noforexplainingoutcome)
57