measurement equivalence of nationalism and patriotism in 34 countries in a comparative perspective...

MEASUREMENT EQUIVALENCE OF NATIONALISM AND

PATRIOTISM IN 34 COUNTRIES IN A COMPARATIVE PERSPECTIVE

Eldad Davidov

GESIS-Central Archive for Empirical Social Research, University of Cologne

Outline

1) What is measurement equivalence

2) Some theoretical background

3) Data and measurements: nationalism and patriotism

4) Testing for measurement invariance using the ISSP 2003 in 34 countries

5) Discussion, concluding points

1) Invariance

• We often have several groups in our analyses: different cultures, regions or countries.

• In order to compare relationships between constructs or means across groups, we need certain level of invariance of the constructs across those groups.

• The meaning of invariance is “whether or not, under different conditions of observing and studying phenomena, measurement operations yield measures of the same attribute” (Horn and McArdle 1992, 117).

• In order to compare effect parameters across countries at least measurement (metric) equivalence or invariance of the factor loadings between items and theoretical constructs is needed.

• A higher level of invariance, scalar invariance, is needed for comparing means across countries.

Techniques to test invariance

• Various techniques have been developed to test measurement invariance (De Beuckelaer, 2005).

• Multiple- group confirmatory factor analysis (MGCFA: Jöreskog 1971) is among the most powerful.

Configural Invariance (1)

• The lowest level of invariance is ‘configural’ invariance.

• Configural invariance requires that the items in the measuring instrument exhibit the same configuration of loadings in each of the different countries.

• That is, the confirmatory factor analysis thus confirms that the same items measure each construct in all countries in the cross-national study (or cross-group).

Configural Invariance (2)

Configural invariance is supported if

(a) a single model specifying which items measure each construct fits the data well,

(b) all item loadings are substantial and significant,

(c) there are no large modification indices, and

(d) the correlations between the factors are less than one. The latter requirement guarantees discriminant validity between the factors (Steenkamp and Baumgartner 1998).

Measurement invariance (1)

• Configural invariance does not ensure that the people in different nations understand the items in the same way.

• The factor loadings may still be different across countries.

• The test of the next higher level of invariance, ‘measurement’ or ‘metric’ invariance, requires that the factor loadings between items and constructs are invariant across nations

Measurement invariance (2)

• It is tested by constraining the factor loading of each item on its corresponding construct to be the same across groups.

• Measurement invariance is supported if the model cannot be significantly improved by releasing some of the constraints.

Partial measurement invariance (1)

• However, for cross-cultural comparison to be allowed, it is not necessary that all factor loadings are equal.

• Several scholars have suggested that it is enough to have two equal factor loadings per construct across countries to allow comparison of effects.

• They termed it partial measurement (metric) invariance (Byrne, Shavelson, and Muthen 1989; Steenkamp and Baumgartner 1998).

Scalar invariance (1)

• A third level of invariance is necessary to allow mean comparison of the underlying constructs across countries.

• This is often a central goal of cross-national research.

• Such comparisons are meaningful only if ‘scalar’ invariance of the items is ensured.

• Scalar invariance guarantees that cross-country differences in the means of the observed items are a result of differences in the means of their corresponding constructs.

Scalar invariance (2)

• To assess scalar invariance, one constrains the intercepts of the underlying items to be equal across countries.

• It is supported if the model fit to the data is good and if it cannot be improved by releasing some of the equality constraints.

Invariance - summary

• Meaningful comparison of construct means across countries requires three levels of invariance, configural, metric, and scalar.

• Meaningful comparison of relationships between constructs requires two levels of invariance, configural and metric.

• Only if all these types of invariance are supported can we confidently carry out comparisons.

2

=1

=1

1

Item a

Item b

Item c

Item d

Item e

Item f

Measurement Invariance:

Equal factor loadings across groups

2

=1

=1

1

Item a

Item b

Item c

Item d

Item e

Item f

Group A Group B

Outline






2) Theoretical background

• National identity is considered a central concept of group attachment in the modern world.

• Attachment of group members toward their country is expressed by a sense of belonging, love, loyalty, pride, and care toward the group and land (Bar-Tal 1997, 246).

• However, the concept of national identity still lacks a distinct and uncontroversial definition. This makes comparative research on national identity problematic.

Nationalism and patriotism

• Schatz, Staub, and Lavine (1999) differentiate between blind and constructive patriotism.

• They describe blind patriotism as "a rigid and inflexible attachment to country, characterized by unquestioning positive evaluation" (p. 153).

• In contrast, they define constructive patriotism as "an attachment to country characterized by critical loyalty“.

• The two orientations are indeed patriotic in the sense of positive national identification.

• However, the blind patriot considers

criticism of the state as disloyal, whereas constructive patriots may even criticize the state themselves, if they feel that the state violates their ideology or if they believe the state is mistaken.

• Blank and Schmidt (2003) describe nationalism and patriotism as more

specific expressions of national identity whereas national identity is the

more general concept.

• Nationalism is characterized by idealization of the nation, a feeling of

national superiority, an uncritical acceptance of national, state, and

political authorities, a suppression of ambivalent attitudes toward the

nation, an inclination to define one’s own group by criteria of descent,

race, or cultural affiliation, and derogation of groups not considered to be

part of the nation.

• Patriotism is viewed as having the following aspects: the nation is not

idealized, but critically evaluated; support for the system as long as the

nation’s aims are in accord with humanistic values; support for

democratic principles and an advanced social system; rejection of an

uncritical acceptance of state authorities; and acceptance of negative

nation-related emotions.

• However, using the 1995 ISSP data, they evidenced some validity problems in the operationalization of the two concepts (some loadings below 0.4).

• Their conceptualization was criticized by

Cohrs (2005) who argued that the criterion-related validity of these constructs is sometimes not supported by the data.

Nationalism and patriotism – summary and goals

• There is no full agreement in the literature on the conceptualization and the operationalization of national identity in general, and of nationalism and patriotism in particular.

• In this study we are not going to solve this problem.

• We propose a possible and reasonable set of items from the ISSP 2003 National Identity Module to operationalize the concepts nationalism and patriotism.

• This proposal is especially in line with Blank and Schmidt's (2003) conceptions, but includes a shortened version of it.

• We evaluate their measurement characteristics cross-nationally, using a large set of countries.

Outline






3) Data and measurements

• The last release of the ISSP (International Social Survey Program) National Identity Module was was collected in 2003.

• 34 countries

• For further details see http://www.gesis.org/en/data_service/issp/data/2003_National_Identity_II.htm

• Total number of respondents: 44,170

http://www.gesis.org/en/data_service/issp/data/2003_National_Identity_II.htm

http://www.gesis.org/en/data_service/issp/data/2003_National_Identity_II.htm

Measurement - nationalism

Two questions in the ISSP regarding nationalism are used. These question items refer to the superiority of one’s own country and its residents:

(1) The world would be a better place if people from other countries were more like the [Country Nationality]; and

(2) Generally speaking, [Country] is a better country than most other countries.

Measurement - patriotism

Patriotism was measured in the ISSP based on responses to the questions related to civic or political pride:

How proud are you of [country] in each of the following:

(a) the way democracy works(b) its social security system; and(c) its fair and equal treatment of all groups

in society.

The responses were measured on a five-point scale ranging from 1 (strongly disagree) to 5 (strongly agree).

Outline






Testing for measurement invariance using the ISSP in 34 countries

2 strategies:(1) A ‚top-down‘ strategy(2) A ‚bottom-up‘ strategy

We apply a ‚bottom-up‘ strategy:

- We start with the weakest level of invariance and identify countries which guarantee configural invariance.

- Then we test measurement and scalar invariance.

- We do this because we wish to establish first whether even weak forms of invariance are empirically incorrect.

Analysis steps

(1) We start with 34 separate confirmatory factor analyses (CFAs) for each country.

(2) After identifying the countries with similar model configuration, we block them into separate groups and conduct configural, metric, and scalar invariance tests sequentially in those groups.

Nationalism

Patriotism

v22e21

1

v21e11

v35e5

11

v29e41

v26e31

Step 1 (1)

• None of the single-country models can be rejected on the basis of the global fit measures.

• All items have moderate to high factor loadings. Most of the standardized loadings are higher than 0.6.

• For 22 countries no modification is needed.

• This implies that the measurement of nationalism and patriotism produces an acceptable fit to the data in these countries.

Step 1 (2)

• A few modifications are needed to fit the models of 12 countries to the data better.

• Some of the modifications include error correlations and others include cross loadings.

Country ModificationB Pclose RMSEA Covariance Nationalism <> patriotism

Correlation Nationalism <> patriotism

1. Australia Nv35 0.83 0.035 .077* (.012) (S.E.) .324

2. Austria Nv35 0.69 0.035 .039 (.021) .102

8. Finland Dropping v21; Pv22;E2<->E4

0.89 0.000 Cov(e2,e4)= .067* (.021)

Cor(e2,e4)= .103

11. Great Britain Nv29 0.96 0.000 .198* (.028) .381

12. Hungary Nv29 0.73 0.032 .173* (.023) .495

13. Ireland Nv35 0.65 0.039 .115* (.023) .494

15. Japan Nv29 0.66 0.037 .271* (.027) .877

16. Latvia Nv26; E2<->E4;E2<->E5

0.77 0.000 .243* (.028)Cov(e2,e4)=-.046* (.020); Cov(e2,e5)=-.078* (.021)

.551Cor(e2,e4)= -.147Cor(e2,e5)=-.269

19. Norway E1<->E5 0.98 0.007 .124* (.015)Cov(e1,e5)=.091* (.017)

.300Cor(e1,e5)= .155

24. Slovakia Nv35 0.89 0.020 .116* (.018) .351

27. South Korea Nv26 1.00 0.000 .190* (.019) .521

29. Sweden E1<->E5; Pv21 0.59 0.042 .163* (.023)Cov(e1,e5)= .070* (.021)

.281Cor(e1,e5)= .121

Step 2 (1)

• It is very probable that in the multiple-group confirmatory factor analysis we will also find no evidence for configural invariance across these 12 countries with the other 22.

• Therefore, we grouped the countries into blocks of nations with identical model configurations found in the previous section.

Step 2 (2)

• The first group (a) comprised of 22 countries where no modifications in the model were conducted.

• The second block (b) included 4 countries with a single identical modification, an additional path from nationalism to the item pride of fair and equal treatment v35.

• The third block (c) included three countries with an additional path from nationalism to the item measuring pride of the social security system v29.

• The remaining 5 countries had different model specifications due to modifications, and we excluded them from further tests (since configural invariance is a precondition for measurement and scalar invariance).

Step 2 (3)

• This resulted in three multiple-group models, one comprised of 22 countries, the second of four, and the third of 3 countries.

• In each set of models, to test for different levels of invariance we inspected the chi-square differences between the models and other global fit measures.

First set - 22 countries (1)

Model Chi-square Degrees of freedom

Pclose RMSEA

1. Configural invariance

185 88 1.00 .006

2. Full measurement invariance

650 151 1.00 .011

3. Partial measurement invariance

489 144 1.00 .009

3a. Model 3 without France

399 137 1.00 .008

4. Scalar invariance test without France

15522 237 1.00 .049

First set - 22 countries (2)• We cannot reject the configural invariance model

(model 1).

• Furthermore, the measurement invariance model (model 2) also cannot be rejected based on several indices (RMSEA = 0.011, Pclose = 1.00).

• However, the chi-square difference implies a significant increase in the restrictive measurement invariance model (P < 0.05).

• In order to improve it, one equality constraint was released (Pat.v26) in 7 countries (model 3).

• Dropping France (many modification indices) further improved the model.

First set - 22 countries (3)

• One should keep in mind that the proposed changes leading to partial measurement invariance were conducted to improve the model fit, but even the full measurement invariance model (2) was quite good itself.

• Finally, the increase in the chi-square value in the scalar invariance model is very large and significant compared to the partial measurement invariance model (p < 0.05). Therefore, we reject it.

Second set - 4 countries (1)MGCFA: Fit Measures of the Invariance Test (4 Countries, Block B)


Pclose RMSEA

1. Configural invariance 29 12 1.00 .0172. Full measurement invariance

78 24 1.00 .021

3. Modified measurement invariance model

61 23 1.00 .018

4. Scalar invariance test (includes the modification in Model 3)

2922 38 .00 .122

Second set - 4 countries (2)

• Although the measurement invariance model (model 2) fails according to the chi-square difference test compared to the configural invariance model (model 1), it is rather acceptable.

• To improve its fit, we released one constraint in model 3, that is, the equality constraint of the factor loading between nationalism and the pride of fair treatment of groups in society item v35 in Australia.

Second set - 4 countries (3)

• This change resulted in good and improved global fit measures.

• By contrast, the global fit measures of the scalar invariance test (model 4) and the high increase in chi-square compared to model 3 are not acceptable, and we reject this model.

Third set - 3 countries (1)MGCFA: Fit Measures of the Invariance Test (3 Countries, Block C)


Pclose RMSEA

1. Configural invariance

107 12 .17 .055

2. Full measurement invariance

121 20 .88 .044

3. Scalar invariance test

951 30 .00 .109

Third set - 3 countries (2)

• The full measurement invariance model (model 2) is not rejected by the chi-square difference test compared with model 1.

• It guarantees equality of meaning of the constructs nationalism and patriotism in the three countries, and allows the comparison of regression coefficients between those countries.

• The scalar invariance test in model 3 fails.

What about other country sets?

• May associations or means be compared across countries from different blocks?

• The answer is not necessarily negative – each case should be tested.

• Example: Bulgaria (block a) and Hungary (block c).

• After allowing a path from patriotism to the item which measures the belief that the world would be a better place if people from other countries were more like those from one’s own country v21, and

• another path from nationalism to the item which measures pride of the social security system v29,

• we are able to establish configural and metric invariance between Bulgaria and Hungary.

• Finding out whether all possible pairs or triads of countries are comparable includes an enormous amount of comparisons which is beyond the scope of this study.

• Researchers interested in studying certain countries should follow similar procedures of consecutively testing for configural, metric, and scalar invariance in these countries prior to comparing constructs and their associates across them.

Outline






5) Discussion, concluding remarks (1)

• Under certain constraints on several countries, full (in block c) or partial (in blocks a and b) measurement invariance may be supported.

• Scalar invariance is not supported in any of these groups of countries.

• One can conclude that comparing means of nationalism and patriotism across these groups of countries is problematic.

• Comparing relationships is allowed within those groups (a, b or c) but not only.

Discussion, concluding remarks (2)

• It is also possible to test invariance only for one construct separately-depending on the theoretical question.

Implications (1)

• For example, one can compare the relations of national attachment to sociodemographic characteristics, behavior, and attitudes.

• If differences are found in the relationship between national attachment and feelings of threat due to immigration, evidence of measurement invariance make it legitimate to try to interpret these differences meaningfully.

• Nationalism and patriotism may also mediate the effect of sociodemographic variables on attitudes or feelings of threat from immigrants, and differences or similarities in the mediation process may be meaningfully interpreted.

Implications (2)

By using the ISSP 1995 National Identity Module one could investigate changes and trends in national attachment in the same country if scalar invariance over time in this country is guaranteed –

this way one can study changes in national identity in response to external developments such as crimes, political and cultural events, or changing economic conditions.

Final word

• Recent studies suggest that when full or partial measurement invariance is not guaranteed, it may still be the case that constructs are equivalent.

• Saris and Gallhofer (2007, chapter 16) indicate that the test of measurement invariance is too strict and may fail although cognitive equivalence still holds.

• Inspecting it may be done by using more than one question item per indicator.

• Repeated questions to measure the same indicator are seldom used.

• An alternative procedure would be to conduct cognitive pretests in the different cultures.

• When these procedures are not feasible, testing for invariance should be done.

www.Thank you very much for your attention!!!!.de