fuzzy logic or fuzzy application? - ecpr.eu · 1 fuzzy logic or fuzzy application? a replic to...
TRANSCRIPT
1
Fuzzy Logic or Fuzzy Application?
A Replic to Stockemer’s “Fuzzy Set or Fuzzy Logic? Comparing the Value of
Qualitative Comparative Analysis (fsQCA) versus Regression Analysis for the
Study of Women’s Legislative Representation”
Paper to presented at the 4th
European Conference on Politics and Gender (ECPG)
June 11th
– 13th
2015
Uppsala (Sweden)
Antje Buche (University Erlangen-Nuremberg; [email protected])
Jonas Buche (Goethe University Frankfurt; [email protected])
Markus B. Siewert (Goethe University Frankfurt; [email protected])
**First draft – please, do not quote or cite without permission by the authors. Comments and
remarks are very welcome.**
Abstract (120 words)
In a recent article in European Political Science (EPS), Daniel Stockemer characterizes fuzzy-
set Qualitative Comparative Analysis (fsQCA) as a “poor methodological choice” because of
its “suboptimal nature” for the study of descriptive women representation in national
assemblies across the globe. However, we demonstrate that his judgments are based on two
"mis-es”: first, a misunderstanding of set-theoretical thinking in general, and specifically
QCA; and second, a misinformed application through-out various steps of the fsQCA, e.g. the
calibration process, the analysis of necessary and sufficient conditions, and the interpretation
of the results. We therefore argue – in contrast to Stockemer – that fsQCA is a valuable tool
for the comparative study of social phenomena providing a fundamentally different analytical
perspective from standard quantitative techniques.
2
1. Introduction
The descriptive representation of women in legislatures is at the core of today’s comparative
political science research (for an overview see e.g. Paxton and Hughes 2014; Krook and
Childs 2010; Wängnerud 2009). While women constitute even more than half of the worlds’
population the global average of female parliamentarians is only around 22.1% (date: April 1st
2015; IPU 2015). A closer look reveals two further aspects: First, the variation across the
globe is immense. While some countries display (nearly) equal gender representation in their
national legislatures, others have only a few or even none female representatives. Second,
parliaments with a high share of women representation are not restricted to countries which
are classically associated with gender equality like the Nordic countries; others as diverse as
Ruanda, Bolivia, Cuba, Seychelles, Ecuador, and South Africa have joined Sweden or Finland
at the top of the rankings. On the other hand, there are several prominent OECD and EU
countries like the United States of America, Japan, Ireland, Malta, or Chile that exhibit a low
share of women in their national parliaments (IPU 2015).
Explicating this cross-national variation in women representation existing scholarship has
identified mainly three sets of factors having a positive or negative effect on the share of
women in legislatures: First, institutional approaches focus on the design of the electoral
system. According to some studies proportional systems and multi-member districts further
equal representation of women while majoritarian electoral systems and single-member
districts seem to have a negative effect (e.g. Rosen 2012; McAllister and Studlar 2002; in
opposition Roberts et al. 2013). Additionally, statutory or party-based gender quotas have
been identified as effective drivers of higher women representation (e.g. Krook 2009; Tripp
and Kang 2008; Dahlerup 2006). A second strand of research emphasizes socio-economic
conditions; from this perspective higher proportions of women in parliaments are strongly
interlinked with equal access to education systems and emancipation at the workplace (e.g.
Viterna et al. 2008; Rosenbluth et al. 2006; Matland 1998). Third, cultural approaches refer to
the importance of models of femininity, gender roles and religious norms within societies as
explanations for descriptive political representation (e.g. Paxton and Kunovich 2003;
Inglehart and Norris 2001). While these bundles of explanatory factors affecting the share of
women in national parliaments are well defined their effects are still largely contested.
Therefore, newer studies have concentrated on the interplay of these factors and varying
patterns across different contexts such as status of development or geographical location (e.g.
Stockemer 2014, 2013; Rosen 2012; Ruedin 2012; Krook 2010).
3
This newer line of research has also led to methodical innovations: For instance, Mona Lena
Krook’s study (2010) of women representation in sub-Saharan Africa and Western Europe
introduced Qualitative Comparative Analysis (QCA; see Ragin 2008; Rihoux and Ragin
2009; Schneider and Wagemann 2012) to the field of gender politics in general, and women
representation specifically. At first sight, QCA seems to fit nicely with the above mentioned
trends in research on descriptive women representation for several reasons: It strives for a
more case-oriented, and therefore, context-specific assessment; it looks for combinations of
conditions and not so much on the average effects of single variables; it focuses on equifinal
configurations that explain various sets of cases; and it introduces the notion of asymmetry
which opens the view for different explanations for high and low shares of women
representation.
In stark contrast to these arguments, Daniel Stockemer (2013) in a recent article comes to the
conclusion “that QCA as a method is a poor methodological choice for explaining the factors,
or combinations of variables, that lead to high or low women’s representation in national
assemblies” (Stockemer 2013: 95) which is unable “to provide any additional insights”
(ibid: 96). He arrives at this statement through the comparison of a fuzzy-set QCA (fsQCA)
and a standard OLS regression model to the same data set of 54 Latin American and Asian-
Pacific countries. In his study, Stockemer applies the two methods separately and sequentially
in order to reveal “whether they yield different results and, if so, which method is more
appropriate for the study of women’s parliamentary representation?” (ibid: 87). In doing so a
range of seven institutional and socio-economic explanatory factors – democratic or
communist status, electoral systems, gender quotas and women suffrage, female economic
activity rate and country’s level of education – are included in the analysis of high and low
levels of women representation in national legislatures.
Somewhat puzzled by Stockemer’s harsh and absolutizing conclusions we argue that his
outright dismissal of fsQCA as a valuable tool for the study of women representation is
largely based on two “mis-es”: a misunderstanding of the Boolean and set-theoretic roots of
QCA and a misinformed application of fsQCA. We tackle both “mis-es” building our
argumentation on two pillars: on the one hand, we engage in a methodological discussion
explicating the core differences between QCA having its origins in a qualitative-empirical,
case-oriented paradigm and standard regression analysis stemming from a quantitative,
variable-oriented research tradition (Ragin 1987, 2000, 2008; Schneider and Wagemann
2012; Goertz and Mahoney 2012; Thiem et al. 2015). On the other hand, we highlight in three
4
steps some major shortcomings of Stockemer’s statements by debating specific elements of
his research design like case selection and operationalization, via replicating his fsQCA and
OLS regression, and through assessing his interpretation of the research findings.
The paper is structured along these two “mis-es”: In the first section we elaborate on the core
differences between the OLS and QCA arguing that both methods are endemic to two
fundamentally different research cultures which cannot – and should not – be evaluated
against each other. The second section discusses the research design and application of
Stockemer’s study outlining problems and alternatives to avoid some of pitfalls related to both
methods. The paper concludes with a short summary of the main arguments.
2. Misunderstanding QCA
The central aim of Stockemer’s study is to assess the value of fsQCA and standard OLS
regression for the question of descriptive women representation. Acknowledging that these
two approaches are based on completely different logics he asks “whether they yield different
results and, if so, which method is more appropriate for the study of women’s parliamentary
representation?” (Stockemer 2013: 87). For the comparison, Stockemer formulates as his
yardstick for evaluating the analytical power of QCA that “[i]f fsQCA helps identify
parsimonious combinations of factors that explain high women’s representation, then this
method is a powerful tool. In contrast, if we find few unique combinations of indicators and
little empirical coverage of these factor combinations, then this method has some serious
limitations when applied to the study of women’s representation” (ibid: 87f).
From our perspective, both the overall goal and the benchmark of Stockemer’s study are
based on two major misunderstandings and therefore are to no avail. On the one hand, the
rationales and technicalities of QCA and standard regressions are so fundamentally different
that it is neither valuable nor feasible to compare results produced by the two approaches
against each other. On the other hand, parsimony is not nor should be central to any QCA.
Quite contrary researchers applying QCA are usually interested in complex and multiple
configurations of explanatory factors.1
The logic behind QCA is deeply linked to Boolean algebra and set-theoretical thinking (Ragin
1987, 2000, 2008; Schneider and Wagemann 2012; Goertz and Mahoney 2012). From these
1 Nevertheless Stockemer has a point here, in the sense that QCA should strive for some form of generalization
across cases through minimization. If single solution terms are so complex that they still include all
explanatory factors this is not achieved. However, we will see below there are many reasons for this like
quality, calibration, or skewedness of the data, and is clearly not per se a problem inherent to QCA.
5
basic principles follow some major ramifications which distinguish QCA from standard
quantitative techniques like regressions:
First of all the central units of QCA and regression analysis are different. While the former
makes use of variables, the later utilizes sets as building blocks for the analysis. This
difference is far from being superficial: A variable stands for latent concept which is
measured directly by one or several indicators. Here the underlying assumption typically is
that the value of the indicator reflects the latent concept – aka the variable – in a linear way.
On the other side, the linkage between sets and concepts is more about assessing the meaning
of a concept. This regularly goes hand in hand with an asymmetric transformation of the data
in order to capture the semantic meaning of the concept – which is called set calibration. Put
differently, through the construction of sets the researcher interprets the data, truncates
unnecessary variation across an indicator, and through the process of assigning set
memberships attaches conceptual meaning to the set value. In the case of women
representation, variable capture the relative share of female parliamentarians while sets are
calibrated in a way to assess high, low, or medium levels of women representation (Ragin
2008: 71-84; Goertz and Mahoney 2012: 139-149; Schneider and Wagemann 2012: 24-31;
Thiem et al. 2015: 8).
Secondly, once sets are calibrated QCA enables the researcher to examine subset-superset-
relations, or in other words, checking for necessity and sufficiency. Looking for necessity
involves the analysis of shared antecedents of a given outcome. In order to be necessary a
condition has to be present if the outcome is present; thus the outcome cannot occur without
the antecedent.2 Sufficiency, on the other side, aims at the analysis of shared outcomes. So the
focus is on the question is the outcome present if a certain (combination of) condition(s) is
present? From this short depiction it should become clear that hypotheses based on necessary
and sufficient conditions are different from those based on standard regressions.3 While the
latter aim at correlational relationships which are symmetric – e.g. the higher or lower the
share of female workforce the higher or lower the proportion of women in legislatures – set
relations, by nature, are asymmetric (Schneider and Wagemann 2012: 83ff; Goertz and
Mahoney 2012: 16-38; Thiem et al. 2015: 11ff).
2 Stockemer defines a necessary condition as “one in which an outcome must be present whenever the initial
condition is present” (2013: 88). This is plainly wrong. It is rather an almost sufficient definition for a
sufficient condition. 3 However, there are approaches to mimic necessity and sufficiency analysis with regressions; see e.g.
Braumoeller and Goertz 2000, 2003; Clark et al 2006.
6
Thirdly, researchers trained in set-theory think in terms of configurations, equifinality, and
asymmetry. Again, this offers a completely different perspective from standard regression
analysis of social science phenomena. In contrast to quantitative techniques QCA does not
focus on the strength and significance of net-effects of single variables. Cases are rather
perceived as configurations of conditions. For instance, a high share of women representation
is explained by the simultaneous presence or absence of certain conditions. Thus, one is
interested in the concurrence of several explanantes that are associated with a given
explanandum.4 Contrary to standard regressions which usually focus on the one model that
fits best, QCA aims at detecting equifinal patterns uncovering multiple configurations of
conditions that explain an outcome like high share of female parliamentarians. Asymmetry
refers to the fact that configurations of conditions contributing to a high proportion of female
representatives can be different from such configurations that explain a low share of women
in legislatures. Again, this is dissimilar to standard regression approaches which assume a
symmetric relationship in the sense if a higher share of women with a high educational degree
leads to a higher share of women in legislatures then a lower level of women with high
education should lead to a lower proportion of female parliamentarians (Schneider and
Wagemann 2012: 5ff, 76-90).
This abbreviated juxtaposition shows that set-theoretic approaches like QCA and standard
regressions like OLS start out from dissimilar assumptions, are embedded in opposite logics,
and apply different procedures. If we accept this the next questions are how is it possible to
compare the results of these two approaches, and how should we assess the value of QCA and
regressions for the study of women representation? The short answer to these questions: we
cannot compare them and we cannot assess them against each other. If two methods use
different building blocks in their analysis, aim at answering different hypotheses, and rivet on
different aspects of the social world, it is not at all surprising that their results also differ. It is
rather the task of the researcher to use the different insights gained through different
analytical perspectives and methodological choices in a constructive way to shed light on
social science problems.
4 It is important to stress that configurations of conditions are not the same as interaction terms. Interactions in
regressions follow an additive or multiplicative logic based on linear algebra. Configurations in QCA, on the
other side, are based on Boolean algebra and therefore are formed applying minimum or maximum rules. See
Schneider and Wagemann 2012: 42-56; 87f; Goertz and Mahoney 2012: 54ff; Thiem et al. 2015: 16ff.
7
3. Misinformed Application
3.1. Debating Case Selection and Operationalization
We start our discussion of Stockemer’s application with his case selection. In his study,
Stockemer analyses 54 Latin American and Asian-Pacific countries. Despite his own
literature review pointing to the fact that “indicators driving women’s representation are often
embedded in a specific regional, social, and/or economic context” (ibid.: 87), he rates the
selection of two different world regions as a “a propitious laboratory to help determine the
usefulness of fsQCA and/or regression analysis” (ibid.: 90). He bases this assertion, for
instance, on a roughly similar level of development of the regions (measured by the region-
wide average GDP), similar “other key explanatory variables in the data set” (ibid.), and the,
on average, comparable number of female parliamentarians. However, cultural differences of
the regions are left out completely. Moreover, the difficulties in the comparison of micro-
states such as Bhutan or Papua-New Guinea with countries likes Australia, Brazil or China are
not discussed at all. While this might not necessarily be problematic, this decision has to have
an influence on the interpretation of the results. One way to avoid problems that stem from
case selection is to run separate analyses separate analyses with more context sensitive
measures, with different calibration thresholds, and probably even partially different
conditions (see e.g. Krook 2010)
However, to put more emphasis on region specific differences comes at a price. If on the one
hand two QCAs and two regressions are conducted, the number of cases decreases to 32 Latin
American or 22 Asian countries, respectively. Thus, to apply an OLS regression for each sub
sample would largely scratch the limits of statistical power (Cohen 1998), while for a QCA
these numbers of cases are not problematic per se (Schneider and Wagemann 2012: 317). One
the other hand, one could include a region specific dummy – or crisp set – to control for the
basic cultural differences. While keeping the number of cases constant, this would lead to a
higher number of explanatory factors. Hence, both approaches might not be a convenient
solution for the problem. This is especially true as the original analysis with 54 cases and 7
explanatory factors is already tangent to the “many variables, small N problem” (Lijphart
1971: 686).
According to this problem, a certain number of cases in an analysis might not be able to
display the diversity that is conceptually included by the number of explanatory factors. With
seven conditions included to the analysis 128 combinations of these conditions are logically
8
possible which outnumber the cases by far. Every combination of conditions that lacks
empirical evidence by a case is a so-called logical remainder (Ragin 2008; Schneider and
Wagemann 2012: 151ff.). For OLS regression the problem of limited empirical diversity
holds too because sufficient variance on the explanatory factors and the dependent variable
cannot be guaranteed. Additionally, the problem of influential observations (outliers) appears
in this context. This means that some observations have a “demonstrably larger impact on the
calculated values of the various estimates (coefficients, standard errors, t-values, etc.) than is
the case for most of the other observations” (Belsley et al. 1980: 11). Given the relatively
small number of 54 cases this problem does occur and weakens the statistical power of the
model, too.
Furthermore, not just the number of cases and explanatory factors but also the
operationalization of the latter is a crucial factor in both QCA and OLS regression analysis;
despite the differences between the operationalization of variables and sets. We will
exemplify some problem associated with operationalization and calibration by focusing on the
two ordinal variables “quotas” and “electoral systems” with three categories each.
Proceeding with OLS, taking these variables into regression contradicts the logic of linear
regression techniques, because dependent and independent variables have to be continuous or,
in the case of the independent variable, dummy variables. Using other types of variables for
linear regression is exclusively possible if some important conditions are fulfilled (Urban and
Mayerl 2011):
(1) The variables must at least have five categories5,
(2) the categories of variables must be in a ranked order,
(3) the distances between the categories have to be equal (both in semantic meaning and
numeric value), and
(4) the categories can be interpreted as intervals for the continuous latent variable.
But, firstly, there is ‘quotas’ (quota clauses in parties), which is coded into zero, 0.5 and one.
Whereas zero means no quota clauses in all parties, 0.5 stands for at least one party that has
quotas. If all parties have quotas (constitutionally implemented quota clauses), the variable
becomes one. Secondly, the ‘electoral system’ is coded one for proportional systems, zero for
majoritarian systems and 0.5 for all others. The two variables violate both the first and the
third condition. They have just three categories and the distance between them is neither
5 Others, e.g. Johnson 2009, even say it has to be seven or more.
9
semantically nor numerically the same. Here, Stockemer should have used categorical
dummies6 or, as a more parsimonious way, recode the data to get at least two dummy
variables. However, this would cause a loss of information. Taking this problem serious, we
use categorical dummies for the replication.
A similar problem with the same variables occurs under the calibration procedure in QCA. In
fsQCA, set memberships vary between full membership (set membership score equals 1) and
non-membership (membership score equals 0). Stockemer states that as the variables
‘electoral system’ and ‘quotas’ already have “values between 0 and 1 [… they] consequently
do not need to be transformed any more (Stockemer 2013: 93). This means that every country
with a mixed party system is calibrated as a 0.5 member in the set of ‘electoral system type’.
The same is true for every country with not all but at least one party with quota applied. But
in QCA the 0.5 anchor defines the transition point between membership and non-membership,
i.e. the “point of maximum ambiguity” (Ragin 2008: 30). Assigning this value to cases
conceptually means that we cannot say anything about the set membership of these cases;
they are neither members nor non-members of the respective set. Technically, these cases
cannot be assigned to only one truth table row but to at least two. These cases display multiple
combinations of conditions ideal typically. Thus, computer software will, first, ‘hide’ these
cases from the truth table which may cause false logical remainders.7 Second, if these cases
are covered by an explanatory conjunction that encompasses the condition with the 0.5
calibration, the software cannot display this since the cases’ membership score in the
conjunction is 0.5, too. Thus, just if the problematically calibrated condition is dropped within
the minimization process, the case can be explained at all. To be clear, none of the cases with
a 0.5 membership in at least one of the conditions is excluded from the analysis by default.
But due to this easily avoidable calibration issue, Stockemer risks to lose the explanatory
power of his model for those 17(!) cases he assigned a 0.5 set membership to in either
‘electoral system type’, ‘quota’ or both. This equals 1/3 of the cases.
Taking this points serious, in the replication we change the calibration for the two most
problematic conditions, i.e. ‘electoral system type’ and ‘quota’ without the allocation of the
0.5 anchor. In order to avoid this membership score we have to decide whether the conceptual
meaning of a 0.5 corresponds more to a (partial) membership or a (partial) non-membership in
6 That means to create as much dummy variables as categories of the former variable. The dummies get the
value one if they apply to the right category and zero otherwise. 7 A false logical remainder is a truth table row that is empty although there are cases representing the given
combination of conditions. This happens only if a set membership scores of 0.5 are assigned.
10
the particular set. For the set ‘quota’ we proceed the following way. As a case was coded 0.5
“in which at least one party represented in the national assembly […] has a party quota”
(Stockemer 2013: 91), the underlying concept of the set is (partially) represented by those
cases. Thus, we recode all 0.5 set memberships in the set ‘quota’ into a 0.668. In comparison,
the coding 0.5 for the variable ‘electoral system type’ includes any “semi-proportional or
mixed system” (ibid.). Here, without case knowledge we cannot allocate the same set
membership score for all of the countries under scrutiny. Thus, we allocate a 0.66
membership in the set ‘proportional electoral system type’ to all countries with a “Mixed
Member Proportional System (MMP)” (Idea 2015) as in this type of mixed system the “List
PR system compensates for the disproportionality in the results from the plurality/majority
system” (Idea 2015). In our understanding, this represents the concept of a proportional
electoral system rather good; i.e. such a case is more in the set than out of the set. In contrast,
a “Parallel System is a mixed system […] where no account is taken of the seats allocated
under the first system in calculating the results in the second system” (Idea 2015).
Consequently, cases with a parallel system of proportional list and majority system are treated
as more out of the set than in the set; i.e. allocated a 0.33 membership.
3.2. Replicating the Analysis
When replicating Stockemer’s analysis with categorical dummies, the results look very
similar to the original analysis (for a comparison see Stockemer’s results in appendix 1).
There are no significant effects for the electoral system and, ceteris paribus, a significant,
approximately ten percentage points higher women’s representation for countries with
implemented quotas (D_Quotas3). But there is one elementary difference between the results.
Stokemer treats the variables as continuous ones. Due to linearity assumption, Stockemer
(2013) has to interpret the coefficient in a way that every change in the independent variable
“quotas” causes a change in the dependent variable “women’s representation”. In Stokemer’s
model, ceteris paribus, countries with quotas for at least one party but not for all (quotas has
the value 0.5), the women’s representation gets approximately five percentage points higher
than in countries without quotas (D_Quotas2). But that is not true. As one can see in table 1,
8 It goes without saying that this decision is highly debatable since a) the memberships scores of particular
country should be different according to the number or importance of parties having the quota, b) the
membership score of 0.66 could be replaced by any other value indicating a partial membership (such as 0.75
or 0.8), and c) the underlying concept could have the (hidden) label ‘all-encompassing quota’ suggesting that
a single party without quota is sufficient for a (partial) non-membership of that particular case in the set
‘quota’.
11
there is no significant difference between these countries and countries without quotas9. Thus,
the effect does just occur between no quota and quotas at all parties proving the weakness of
the ordinal variable for the OLS.
Table 1: Results of OLS regression with categorical dummy variables
B SE Sig
D_Quotas2 3.98 3.01 0.202
D_Quotas3 10.35 2.38 0.000
D_PR2 -2.15 2.90 0.46
D_PR3 -0.38 2.43 0.876
Education 14.79 9.57 0.129
Year of Women Suffrage 0.06 0.09 0.535
Female Activity Rate -0.04 0.08 0.605
Degree of Democracy 0.09 0.05 0.060
D_Communist State 15.51 3.41 0.000
Constant -1.91 7.00 0.787
Furthermore, in his analysis Stockemer argues about the impact of the degree of democracy.
While it is not statistically significant at the 0.05 level, it is at the 0.1 level. This problem is
caused because of the absence of diagnostic tests to confirm the results of the analysis. As
mentioned above regression analyses with small N are very sensitive and one has to account
for influential observations (Jann 2009). A look on the data reveals that there are some
countries with a great influence. This holds especially true for the variable ‘degrees of
democracy’. Here, the effect is mainly driven by New Zealand and Australia which have a
long democratic tradition (129, respectively 106 years of democracy) and a rather high
percentage of women’s representation (33.1%, respectively 26.7%). Figure 1 shows that
without these two countries there is no significant effect for ‘degrees of democracy’ at all10
.
Even if one does not eliminate the two (or at least New Zealand) countries for the analysis,
this figure shows the sensitivity of the regression results. So here might be the overall
question whether OLS regression is an adequate method for these data.
9 When using categorical dummies in a regression analysis one category is used as reference. In this case “no
quota clauses” and majoritarian systems are taken as reference categories. 10
This is also true if only New Zealand is dropped out. Note that the education level becomes now significant
at the 0.1 level.
12
Figure 1: Added variable plot11
for years of democracy
With New Zealand & Australia Without New Zealand & Australia
Regarding the replication of the QCA, one should start with the analysis of necessary
conditions missing so far. As can be seen in table 2, no single condition is necessary due to
the non-perfect subset relations indicated by the low consistency values.
Table 2: Analysis of necessity of single conditions
Condition Consistency
necessity
Coverage
necessity Condition
Consistency
necessity
Coverage
necessity
Education 0.86 0.49 ~12
Education 0.75 0.63
Consolidation
of Democracy 0.27 0.54
~ Consolidation
of Democracy 0.21 0.62
Econ. female
activity rate 0.28 0.48
~ Econ. female
activity rate 0.34 0.86
Women’s
suffrage 0.56 0.54
~ Women’s
suffrage 0.51 0.73
Electoral
system 0.52 0.45
~ Electoral
system 0.49 0.63
Communist
country 0.21 0.78
~ Communist
country 0.04 0.22
Quotas 0.58 0.57 ~ Quotas 0.35 0.51
The analysis of sufficient combinations of conditions with the newly calibrated data does
reveal a solution term with three equifinal combinations consisting of six conditions each (see
11
Added variable plots show the effect of adding another variable to the model. For an introduction see Cook
and Weisberg 1982. 12
The tilde indicates the absence of the respective condition.
Kyrgyzstan
TurkmenistanKazakhstan
Tajikistan
Singapore
Philippines
Thailand
Mongolia
Sri LankaUzbekistan
MalaysiaBolivia
Cuba
EcuadorBhutan
GuatemalaVietnam
Brazil
Mexico
Saint Lucia
Indonesia
Haiti
Pakistan
Nicaragua
Colombia
Chile
Korea. South
Domincan R.
Honduras
Venezuela
Guyana
Cambodia
Papua New GuineaBelize
Bangladesh
El Salvador
China
Nepal
Peru
Panama
Argentina
Uruguay
Paraguay
Laos
Trinidad
Barbados
Suriname
JamaicaBahamas
Japan
India
Costa Rica
Australia
New Zealand
-10
010
20
e(
wo
men
sre
p | X
)
-50 0 50 100e( yearsdemocracy | X )
coef = .0897167, se = .04462221, t = 2.01
Malaysia
Singapore
Philippines
Uzbekistan
TurkmenistanKazakhstan
ColombiaBangladesh
Nicaragua
Tajikistan
Paraguay
Thailand
Bolivia
Kyrgyzstan
Papua New Guinea
Mexico
BhutanHonduras
Sri Lanka
Vietnam
Cuba
Peru
Haiti
Cambodia
Indonesia
Guatemala
Korea. South
Mongolia
Belize
Laos
Venezuela
Ecuador
Chile
Guyana
Argentina
China
Pakistan
BrazilDomincan R.
Nepal
Panama
Bahamas
El Salvador
Trinidad
Saint Lucia
Uruguay
Suriname
Barbados
Jamaica
Japan
Costa Rica
India
-10
010
20
e(
wo
men
sre
p | X
)-40 -20 0 20 40
e( yearsdemocracy | X )
coef = .04526299, se = .06816439, t = .66
13
table 3). While this is somewhat more parsimonious than the result by Stockemer (2013: 101)
which consists of four complex solution paths, the problems of low coverage values and
rather complex conjunctions remain.
Table 3: Analysis of sufficient (combinations of) conditions, conservative solution
Parameters of Fit edu*~dem*~eco*
~pr*com*~q
edu*dem*~eco*
~pr*~com*q
~edu*~dem*~eco
~ws*~pr*com
Consistency 1.00 0.98 1.00
Raw Coverage 0.10 0.08 0.10
Unique Coverage 0.07 0.08 0.06
Cases Cuba, Vietnam Costa Rica, Australia Nepal, Laos
Solution
consistency 0.99
Solution
coverage 0.24
Notes: edu = education, dem = consolidation/longevity of democracy, eco = female economic
activity rate, pr = proportional electoral system, q = quota provisions, com =
communist regime type, ws = women’s suffrage
In order to deal with these problems, the logical remainders can be assessed. The high number
of conditions combined with a rather medium number of cases result in 128 logically possible
combinations of the seven conditions, i.e. truth table rows (27). But, just 29 of them are
covered by empirical cases. Thus, another 99 combinations cannot be included in the analysis
in order to reduce the model complexity. Next to the so called conservative strategy of
excluding all logical remainders, two more solution strategies can be applied. First, the most
parsimonious solution term includes all those logical remainders into the minimization
process that reduce the model complexity (Schneider and Wagemann 2012: 169ff). With 99
possible logical remainders to include, the most parsimonious solution is way less complex in
comparison to the conservative one (see table 4).
Table 4: Analysis of sufficient (combinations of) conditions, most parsimonious solution
Parameters of Fit ~pr*com dem*q
Consistency 0.92 0.94
Raw Coverage 0.18 0.12
Unique Coverage 0.18 0.12
Cases Cuba, Vietnam, Nepal, Vietnam Costa Rica, Australia
Solution
consistency 0.93
Solution
coverage 0.30
Notes: dem = consolidation/longevity of democracy, pr = proportional electoral system, q =
quota provisions, com = communist regime type
14
While the results are clearly more parsimonious, they might not be as trustworthy as before
due to the ‘computerized strategy’ of selecting logical remainders for the minimization
process. However, there is a third solution term in QCA offering the opportunity to select
logical remainders on a theoretical basis, the so-called intermediate solution term (Schneider
and Wagemann 2012: 169f). In contrast to the conservative strategy it allows for the inclusion
of logical remainders, in general. But, contrary to the most parsimonious solution just those
remainders are included that are in line with the directional expectations implied by the
researcher on a theoretical basis.13
Applying this accepted strategy to the data at hand, the
complexity of the initial solution by Stockemer is resolved into two combinations of
conditions (see table 5).
Table 5: Analysis of sufficient (combinations of) conditions, intermediate solution
Parameters of Fit ~pr*com edu*dem*q
Consistency 0.92 0.94
Raw Coverage 0.18 0.12
Unique Coverage 0.18 0.12
Cases Cuba, Vietnam, Nepal, Vietnam Costa Rica, Australia
Solution
consistency 0.93
Solution
coverage 0.30
Notes: edu = education, dem = consolidation/longevity of democracy, pr = proportional
electoral system, q = quota provisions, com = communist regime type
To summarize the QCA replication: We changed the calibration for two out of seven
conditions in order to avoid allocating the 0.5 anchor. The results for the conservative strategy
are quite similar to the one done by Stockemer. Although we are able to reduce the
complexity to some degree the single solution paths still comprise rather complex
configurations. However, by applying the intermediate strategy quite parsimonious and
interpretable results can be easily obtained. What remains is the problem of low coverage
which we discuss in the following section.
3.3. Assessing the Interpretation
Stockemer’s interpretation of the effects resulting from the OLS is correct and clearly
formulated – besides the question if using quotas and electoral system as continuous
variables. One minor issue is the use of the R2 in order to interpret the quality of the analysis.
13
These criteria are so called directional expectations for the theoretically expected contribution of single
conditions to the presence of the outcome (Ragin 2008; Schneider and Wagemann 2012: 168f).
15
Based on the R2 Stockemer concludes “the current model explains 50 per cent of the variance
in the dependent variable” (2013: 96). However, due to "Occam’s Razor“ it would be better to
interpret the adjusted R2 which accounts for the number of independent variables. Even when
the interpretation of the adjusted R2 is not as intuitive as the R
2, it works better by evaluating
the accuracy of fit between data and model. In Stockemer’s analysis the adjusted R2 is 0.42
which still is considerably high..
Shifting our focus towards the fsQCA, the interpretation of the conservative solution paths is
a complex task. As stated above, one factor to explain this is the combination of few cases and
many conditions which subsequently result in a high number of logical remainders. Thereby,
the opportunity for minimization is potentially reduced. Thus, either we reduce the number of
explanatory factors in order to match it with the number of cases or we make use of logical
remainders. Applying the intermediate solution on a solid theoretical foundation is not only a
widely accepted strategy to deal with the almost inevitable problem of limited empirical
diversity in the social science (Schneider and Wagemann 2012: 152f; Schneider and
Wagemann 2013). Moreover, it reduces the complexity of the explanatory model and makes
the results more interpretable without the problem of untenable assumptions one has to face
using the most parsimonious solution. According to these results, either the combination of a
communist state with no proportional electoral system or the combination of quota provisions,
a consolidated democracy and a high level of education is sufficient for a high share of female
deputies.
Figure 2: XY-Plot for the analysis of sufficiency, intermediate solution
16
However, as we already outlined above, the coverage of the intermediate solution is still
rather low. Although being highly consistent with a consistency score of 0.95 only
approximately one third (coverage of 0.3) of the outcome set high women representation can
be explained.14
This is nicely visualized by a XY-plot from which we can deduce three things
(see figure 2).
First, we can see that we do not have cases which strongly contradict the statement of
sufficiency showing the combination of conditions but not the outcome. But the consistency is
not perfect because five countries (Laos, Vietnam, Nepal, South Korea, and Uruguay) have a
slightly greater membership in the configuration of the intermediate solution than in the
outcome set.
Second, in the end only six countries out of twenty being in the set high levels of women
representation are covered by the overall solution (Laos, Vietnam, Nepal, Cuba, Australia,
and Costa Rica). While Stockemer is right that this is not a satisfying result for a case-oriented
method which is interested in explaining as many of its cases as possible, he draws the wrong
conclusions from that: The low coverage does not mean that QCA is a poor methodological
tool for the analysis of women representation in national assemblies. The clustering of cases
in the upper left quadrant of the XY-plot – what we call “equifinality-corner” – rather
indicates that not the most relevant factors (or indicators) have been selected for the analysis
and that others – equifinal ones – have to be found to sufficiently explain high women
representation in Latin American and Asian-Pacific countries.
Closely related to that, third, data allocation is highly skewed. Just ten out of 54 cases are
located outside the left triangle of the XY plot. By that, all other cases do not contradict both
the sufficiency of the intermediate solution for the outcome and for the non-outcome. This
indication of simultaneous subset relations raises the questions whether skewedness stems
from single variables (e.g. communism) and subsequently, whether the explanatory factors
have been selected and calibrated in a proper way (Schneider and Wagemann 2012: 168ff.).
4. Concluding Remarks
The starting point of our discussion was the devastating critique by Daniel Stockemer
dismissing QCA due to its “suboptimal nature” (Stockemer 2013: 88) calling it a “poor
methodological choice” (ibid: 94) to study women representation in national assemblies 14
Coverage of 0.3 does not mean that 30% of the cases are explained by the solution as Stockemer suggests but
that the solution set covers 30% of the outcome set. Instead, explaining 6 of 54 cases equals just 11%.
17
which does not “provide any additional insight” (ibid: 96). However, from our perspective
these judgments stem from a misunderstanding of central ideas and features of QCA, and its
misinformed application. In our opinion, there is nothing “inherently problematic”
(Stockemer 2013: 97) with neither fsQCA nor OLS regressions. Both approaches are rooted
in fundamentally different logics, apply completely diverse techniques, and have their distinct
pitfalls one has to cope with.
If we take these points seriously for the study of descriptive women representation in national
legislatures across the globe we cannot subscribe to Stockemer’s observations. Set-theoretical
thinking differs from standard quantitative approaches in so many ways that it is not possible
to contrast their findings against each other. Thus, questions like “which method is more
appropriate” (ibid: 87) or which one has “greater leverage” over the other (Krook 2010: 887)
are not very expedient and fruitful. It is not about more or less leverage but about different
leverages. In this sense, it is like in the simile of the blind men touching and feeling different
parts of an elephant coming to completely different conclusions about what kind of animal
they have in front of them. Goertz and Mahoney (2012, 2013) have called this the “Rorschach
principle”: Social scientists perceive social phenomena differently if they look at them
through set-theoretic or quantitative lenses. Therefore, the real challenge is to reconcile the
different perspectives and to strive for complementary conclusions.
References
Belsley, David A., Edwin Kuh, Roy E. Welsch. 1980. Regression Diagnostics: Identifying
Influential Data and Sources of Collinearity. Massachusetts: Wiley-Interscience.
Braumoeller, Bear F. and Gary Goertz. 2000. “The Methodology of Necessary Conditions”.
In American Journal of Political Science, Vol. 44, 844-858.
Braumoeller, Bear F. and Gary Goertz. 2003. “The Statistical Methodology of Necessary
Conditions”. In Gary Goertz and Harvey Starr (eds.). Necessary Conditions: Theory,
Methodology, and Applications. Lanham: Rowman & Littlefield, 197-223.
Clark, William R., Michael J. Gilligan, and Matt Golder. 2006. “A Simple Multivariate Test
for Asymmetric Hypotheses”. In Political Analysis, Vol. 14, 311-331.
Cohen, Jacob 1988. “Statistical power analysis for the behavioral sciences”. Hillsdale, NJ:
Erlbaum.
Cook, Dennis R. and Sanford Weisberg .1982. Residuals and Influence in Regression. New
York: Chapman and Hall.
Dahlerup, Drude. (ed.). 2006. Women, Quotas, and Politics. New York: Routledge.
Goertz, Gary and James Mahoney, 2012. A Tale of Two Cultures: Qualitative and
Quantitative Research in the Social Sciences. Princeton: Princeton University Press.
18
Goertz, Gary and James Mahoney. 2013. “Methodological Rorschach Tests: Contrasting
Interpretations in Qualitative and Quantitative Research”. In Comparative Political Studies,
Vol. 46(2), 236-251.
International Institute for Democracy and Electoral Assistance. 2015. Glossary of Terms.
http://www.idea.int/esd/glossary.cfm (last access: May 22nd
2015)
Inglehart, Ronald and Pippa Norris. 2001. Rising Tide: Gender Equality and Cultural Change
Around the World. Cambridge: Cambridge University Press.
Inter-Parliamentary Union. 2015. Women in National Parliaments. http://www.ipu.org/wmn-
e/classif.htm (last access: May 12th
2015).
Jann, Ben. 2009. “Diagnostik von Regressionsschätzungen bei kleinen Stichproben (mit
einem Exkurs zu logistischer Regression)”.In Peter Kriwy and Christiane Gross (Eds.).
Klein aber fein! Quantitative empirische Sozialforschung mit kleinen Fallzahlen.
Wiesbaden: VS-Verlag, 93–125 .
Johnson, Paul E. 2009. Working with Ordinal Predictors. Paper presented at the annual
meeting of the Midwest Political Science Association, Chicago, Illinois.
Krook, Mona Lisa. 2009. Quotas for Women in Politics: Gender and Candidate Selection
Reform Worldwide. New York: Oxford University Press.
Krook, Mona Lisa. 2010. “Women’s Representation in Parliament: A Qualitative
Comparative Analysis”. In Political Studies, Vol.58(5), 886-908.
Krook, Mona Lisa and Sara Childs. 2010. Women, Gender, Politics. A Reader. Oxford:
Oxford University Press.
Lijphart, Arendt. 1971. Comparative Politics and Comparative Method. In American Political
Science Review, Vol. 65, 682-93
Matland, Richard E. 1998. “Women’s Representation in National Legislatures: Developed
and Developing Countries”. In Legislative Studies Quarterly, Vol. 23(1), 109-25.
McAllister, Ian and Donley T. Studlar. 2002. “Electoral Systems and Women’s
Representation: A Long-Term Perspective.” In Representation 39 (1), 227-249.
Paxton, Pamela M. and Melanie M. Hughes. 2014. Women, Politics, and Power: A Global
Perspective. Thousand Oaks, CA: Pine Forge Press.
Paxton, Pamela M. and Sheri Kunovich. 2003. “Women’s Political Representation: The
Importance of Ideology”. In Social Forces, Vol. 82(1), 87-113.
Ragin, Charles C., 1987. The Comparative Method. Moving Beyond Qualitative and
Quantitative Strategies. Berkley: University of California Press.
Ragin, Charles C., 2000. Fuzzy-Set Social Science. Chicago: University of Chicago Press.
Ragin, Charles C., 2008. Redesigning Social Inquiry. Fuzzy Sets and Beyond. Chicago:
University of Chicago Press.
Rihoux, Benoît and Charles C. Ragin (eds.), 2009. Configurational Comparative Methods.
Qualitative Comparative Analysis (QCA) and Related Techniques. Thousand Oaks: Sage.
Roberts, Andrew, Jason Seawright and Jennifer Cyr. 2013. “Do Electoral Laws Affect
Women’s Representation?” In Comparative Political Studies, Vol. 46(12), 1555-1581.
19
Rosen, Jennifer. 2012. “The Effects of Political Institutions on Women’s Political
Representation. A Comparative Analysis of 168 Countries from 1992-2010”. In Political
Research Quarterly, Vol. 66(2), 306-321.
Rosenbluth, Frances M., Robert Salmond and Michael F. Thies. 2006. “Welfare Works:
Explaining Female Legislative Representation”. In Politics and Gender, Vol. 2(2), 165-192.
Ruedin, Didier. 2012. “The Representation of Women in National Parliaments. A Cross-
National Comparison”. In European Sociological Review, Vol. 28(1), 96-109.
Schneider, Carsten Q. and Claudius Wagemann. 2012. Set-Theoretic Methods for the Social
Sciences. A Guide to Qualitative Comparative Analysis. Cambridge: Cambridge University
Press.
Schneider, Carsten Q. and Claudius Wagemann. 2013. “Doing Justice to Logical Remainders
in QCA. Moving Beyond the Standard Analysis”. In Political Research Quarterly,
Vol. 66(1), 211-220.
Stockemer, Daniel. 2013. “Fuzzy Set or Fuzzy Logic? Comparing the Value of Qualitative
Comparative Analysis (fsQCA) versus Regression Analysis for the Study of Women’s
Legislative Representation”. In European Political Science, Vol. 12, 86-101.
Stockemer, Daniel. 2014, “Women’s Descriptive Representation in Developed and
Developing Countries”. In International Political Science Review, 1-16.
Thiem, Alrik, Michael Baumgartner and Damien Bol. 2015. „Still Lost in Translation! A
Correction of Three Misunderstandings Between Configurational Comparativists and
Regression Analysts”. In Comparative Political Studies, online first.
Tripp, Aili M. and Alice Kang. 2008. “The Global Impact of Quotas: The Fast Track to
Female Representation”. In Comparative Political Studies, Vol. 41(3), 338-361.
Urban, Dieter, Jochen Mayerl. 2011. Regressionsanalyse: Theorie, Technik und Anwendung.
Wiesbaden: VS Verlag.
Viterna, Jocelyn, Kathleen M. Fallon and Jason Beckfield. 2008. “How Development Matters:
A Research Note on the Relationship Between Development, Democracy and Women’s
Political Representation”. In International Journal of Comparative Sociology, Vol. 49(6),
455-77.
Wängnerud, Lena. 2009. “Women in Parliaments: Descriptive and Substantive
Representation”. In Annual Review of Political Science, Vol. 12, 51-69.
20
Appendix
Table A1: Table 1: “Results the regression equation” by Stockemer (2013: 95)