differential item functioning analysis of the copsoq ...€¦ · questionnaires jcq eri copsoq dif...
TRANSCRIPT
Differential item functioning analysis of
the COPSOQ across ten countries:
the NEXT Study
Jian LI, MD, MPH, PhD, Dr.rer.sec.
and the NEXT Study Group
Institute of Occupational and Social Medicine
University of Düsseldorf, Germany
The 4th International Workshop on COPSOQ
Gent, Belgium, 19-20 September 2013
World Happiness Report 2013
http://unsdsn.org/files/2013/09/WorldHappinessReport2013_online.pdf
Question:
Can the scores be used to
compare across countries?
Whether questionnaire scales capture the same
concepts across countries?
Whether respondents interpret and respond to
items in the same way?
Differential Item Functioning (DIF) analysis is
traditionally used in Psychology, Education, and
Business/Management disciplines. Recently, it
has been applied to mental health/quality of life
measurement.
International Comparison based on questionnaire scores
Translation inequivalence
Confounding effect
Cultural differences
Main sources of DIF:
Orhede E, et al. Item bias in indices measuring psychosocial work
environment and health. Scand J Work Environ Health, 2000, 26 (3): 263-
272.
Pelfrene E,et al. Scale reliability and validity of the Karasek 'Job Demand-
Control-Support' model in the Belstress study. Work & Stress, 2001, 15 (4):
297-313.
Kristensen TS, et al. The distinction between work pace and working hours
in the measurement of quantitative demands at work. Work & Stress, 2004,
18 (4): 305-322.
Pejtersen JH, et al. The second version of the Copenhagen Psychosocial
Questionnaire. Scand J Public Health, 2010, 38 (3 Suppl): 8-24.
DIF analysis in research of work stress
questionnaire within one country
Questionnaires
JCQ
ERI
COPSOQ
DIF methods
• Three-way contingency tables
• Item response theory
• Logistic regression
Choi B, et al. A cross-national study on the multidimensional characteristics of the
five-item psychological demands scale of the Job Content Questionnaire. Int J Behav
Med, 2008, 15 (2): 120-132.
Tsutsumi A, et al. Application of item response theory to achieve cross-cultural
comparability of occupational stress measurement. Int J Methods Psychiatr Res,
2009, 18 (1): 58-67.
DIF analysis in research of work stress
questionnaire across countries
10 participating EU countries
United Kingdom
Finland
Norway
The NEXT Study
Germany
Italy
France
The Netherlands
Belgium
Poland
Slovakia
1 participating Asian country
China
COPSOQ used in the NEXT Study
Quantitative Demands (QD)
QD1: Do you have to work very fast?
QD2: Is your workload unevenly distributed so it piles up?
QD3: How often do you not have time to complete all your work tasks?
Hardly/ever (1), Seldom (2), Sometimes (3), Often (4), Always (5)
Possibilities for Development (PD)
Quality of Leadership (QL)
Meaning of Work (MW)
Insecurity at Work (IW)
Independent judgmental review on the
translation comparability of the COPSOQ items
England
English
Germany
German
Italy
Italian
France
French
Netherlan
ds
Dutch
Belgium
French/Flemi
sh
Finland
Finnish
Poland
Polish
Slovakia
Slovak
China
Chinese
QD
QD1 Reference A A A A A A A A A
QD2 Reference A A B A B B A A A
QD3 Reference A A A A A A A A A
Translation equivalence:
A: no difference;
B: negligible difference (no conceptual difference, but some wording is not perfect);
C: moderate difference;
D: large difference.
Characteristics of the NEXT Study subjects
Note: * p < 0.05, difference among countries by Analysis of Variance or Chi-Square Test
England
(N=1694)
Belgium
(N=1954)
Germany
(N=2145)
Finland
(N=1869)
France
(N=3657)
Italy
(N=3452)
Netherlan
ds
(N=2186)
Poland
(N=3171)
Slovakia
(N=2725)
China
(N=3088)
Age
(mean ± SD) *
39.70 ±
9.69
37.48 ±
9.21
37.21 ±
9.28
41.18 ±
9.97
38.81 ±
9.36
37.18 ±
7.50
36.98 ±
9.48
37.63 ±
7.22
38.40 ±
8.65
30.94 ±
8.29
Marital status (N, (%)) *
Single 198
(11.69)
288
(14.74)
512
(23.87)
254
(13.59)
558
(15.26)
447
(12.95)
318
(14.55)
156
(4.92)
165
(6.06)
1193
(38.63)
Cohabitated 1496
(88.31)
1666
(85.26)
1633
(76.13)
1615
(86.41)
3099
(84.74)
3005
(87.05)
1868
(85.45)
3015
(95.08)
2560
(93.94)
1895
(61.37)
Position rank (N, (%)) *
Low 1176
(69.42)
1750
(89.56)
1769
(82.47)
1624
(86.89)
3460
(94.61)
3118
(90.32)
2172
(99.36)
2829
(89.21)
2441
(89.58)
2446
(79.21)
High 518
(30.58)
204
(10.44)
376
(17.53)
245
(13.11)
197
(5.39)
334
(9.68) 14 (0.64)
342
(10.79)
284
(10.42)
642
(20.79)
Shift work (N, (%)) *
Day shift 919
(54.25)
868
(44.42)
835
(38.93)
623
(33.33)
2416
(66.07)
1355
(39.25)
600
(27.45)
614
(19.36)
929
(34.09)
1036
(33.55)
Rotating shift 775
(45.75)
1086
(55.58)
1310
(61.07)
1246
(66.67)
1241
(33.93)
2097
(60.75)
1586
(72.55)
2557
(80.64)
1796
(65.91)
2052
(66.45)
DIF analysis of COPSOQ
Belgium Germany Finland France Italy Netherlan
ds Poland Slovakia China
% of
sample
s with
DIF
Quantitative Demands
QD1 0 0 0 0 0 0 0 0 + 11%
QD2 0 0 + 0 0 0 + - 0 33%
QD3 0 0 - 0 - 0 - + - 56%
Note: England as reference (adjusting for age, marital status, position rank, and shift work)
0: non-DIF item
+: DIF item with higher item score for country than for England at a given score
-: DIF item with lower item score for country than for England at a given score
Measurement of quantitative demands
QD1: Do you have to work very fast?
QD2: Is your workload unevenly distributed so it piles up?
QD3: How often do you not have time to complete all your work tasks?
Kristensen TS, et al. The distinction between work pace and
working hours in the measurement of quantitative demands
at work. Work & Stress, 2004, 18 (4): 305-322.
Work pace
Work time
Means of Quantitative Demands
Note: ANCOVA adjusting for age, marital status, position rank, and shift work
Means of Quantitative Demands
Note: ANCOVA adjusting for age, marital status, position rank, and shift work
Attention should be paid for the international comparison of psychosocial work characteristics.
DIF analysis is useful for such comparison study: more sensitive than routine methods.
Conclusion
Thank you
for your attention!