2. Title Page

Title: Assessing measurement error in surveys by latent class analysis: application on self-

reported illicit drug use in Iranian Mental Health Survey data

Type of manuscript: Special Articles (Methods)

Author names and affiliations:

Kazem Khalagi1, Mohammad Ali Mansournia

1, Afarin Rahimi-Movaghar

2, Keramat

Nourijelyani1, Masoumeh Amin-Esmaeili

2, Ahmad Hajebi

3,4, Vandad Sharifi

5,6, Reza

Radgoodarzi7, Mitra Hefazi

7, Seyed-Abbas Motevalian

3,8

1 Department of Epidemiology and Biostatistics, School of Public Health, Tehran University of

Medical Sciences, Tehran, Iran. 2

Iranian National Center for Addiction Studies (INCAS), Tehran University of Medical Sciences,

Tehran, Iran. 3

Addiction and High Risk Behavior Research Center, Iran University of Medical Sciences,

Tehran, Iran. 4 Department of Psychiatry, Iran University of Medical Sciences, Tehran, Iran.

5 Psychiatry and Psychology Research Center, Tehran University of Medical Sciences, Tehran,

Iran. 6 Department of Psychiatry, Tehran University of Medical Sciences, Tehran, Iran.

7 Department for Mental Health and Substance Use, Iranian Research Center for HIV/AIDS

(IRCHA), Iranian Institute for Reduction of High-risk Behaviors, Tehran University of Medical

Sciences, Tehran, Iran. 8

Department of Epidemiology, School of Public Health, Iran University of Medical Sciences,

Tehran, Iran.

Corresponding author: Seyed-Abbas Motevalian,

Postal address: Department of Epidemiology, School of Public Health (P.O BOX: 1449614535),

Iran University of Medical Sciences, Hemmat Highway, Tehran, Iran.

E-mail address: amotevalian@yahoo.com

Telephone number: +98 (21) 86704704

Fax number: +98 (21) 88622703

Running title: Assessing measurement error of binary outcomes in surveys

Funding source: Tehran University of Medical Sciences

Conflicts of interest of authors: About all authors, there aren’t any conflicts of interest.

1

3. Abstract

Objectives: Latent class analysis (LCA) is a solution for assessing and correcting measurement error in

surveys. “Local independence” assumption in LCA expresses that indicators are independent from each

other condition on the latent variable. Violation from this assumption will be unreliable results seriously.

We focus on this issue while we used LCA for estimating prevalence of any illicit drug use in “Iranian

Mental Health Survey” as an example.

Methods: The three indicators included in LCA models were: indicator A (≥ 5 times use of any illicit

drug in the past 12 months), indicator B (any use of any illicit drug in the past 12 months) and indicator D

(need for using treatment services or using them for substance use disorders in the past 12 months).

Gender is also used in all LCA models as a grouping variable. One LCA model using two A and B

indicators, and ten different LCA models using three A, B and D indicators were fitted to the data.

Results: The three models which have the best fitness to the data were those including the following

correlations between indicators: (AD and AB), (AD), and (AD, BD and AB). The prevalence estimate of

any illicit drug use based on these three models were 28.9%, 6.2% and 42.2%, respectively. None of the

models have controled violation from local independence assumption completely.

Conclusions: To achieve the unbiased estimations using LCA approach, the three factors violating local

independence assumption should be taken into account completely in models through well-known

methods.

4. Key words

Surveys and Questionnaires, Bias, Mesurement error, Latent class analysis, Self Report, Substance-

Related Disorders

2

5. Manuscript body

Introduction

Measurement error is one of the major sources of systematic errors when estimating different

variables, especially the outcomes related to sensitive topics in surveys. This error refers to the

difference between the true value of an outcome and what obtains from a measuring tool [1, 2].

Such an error causes misclassification in categorical outcomes. A measuring tool, interviewers,

respondents, and data collection styles (in person, telephone, web, etc.) are among the major

sources of this type of error in surveys [1-3]. This error in categorical data causes bias and

reduces precision of estimation [4, 5]. In spite of different measures taken in designing and

executing surveys, such errors are inevitable, especially in sensitive outcomes such as risky

behaviors like sexual relations and use of illicit drugs. In such situations, assessment and

correction of measurement error become more important.

If there are at least two measurements of a categorical outcome in a survey and one of them is a

gold standard (i.e. a measurement with negligible error), the classification error of another tool

can be estimated and corrected easily. Such an assessment method of classification error is

known as “finite fixture” approach [1]. However, assessment and correction of a classification

error by this approach will not be possible in most cases due to lack of a gold standard or its

costliness. Latent class analysis (LCA) will be a suitable solution for assessing and correcting a

classification error of any measurement in a survey study with different wordings of repeated

measurements of an outcome when none of the measurements is a gold standard [1, 3]. LCA

was first introduced by Lazarsfeld and Henry for classification error assessment at 1968 [6].

Major experiences have been obtained on its advantages and limitations over the past few years.

The use of LCA method can be highly suitable in the quantitative analysis of a classification

error in survey data if the model assumptions are logically established. Otherwise, its incorrect

use may lead to invalid results, similar to any other modeling approach [3].

This paper aims at introducing LCA approach for assessing and correcting classification error

when estimating categorical outcomes in surveys through focusing on its required assumptions,

which should be considered while using it correctly. As an example, we used LCA approach for

3

estimating prevalence of any illicit drug use in the past 12 months on “Iranian Mental Health

Survey” (IranMHS) data and discussed the essential assumptions.

LCA and assessing measurement error in survey data

Several indicator (manifest) variables are used in LCA approach for estimating a latent

variable [1]. This method assumes a relationship between classifications of indicator variables

and unobserved latent classes of an outcome variable [7, 8].

To use an LCA model for assessing and correcting measurement error of a categorical outcome

in survey data, there should be two or more of its indicators which measure it, and none of them

should necessarily be a gold standard [9]. The other indicators except first one could be

provided by re-measurement of outcome using another survey a few weeks later, or repetition of

measuring outcome in a survey using different wordings. In the first method, as far as execution

is concerned, this method is difficult and costly and the short interval between the two

measurements will lead to interrelated responses under the influence of respondent’s memories.

Limitations of the second one include possibility of lack of cooperation of respondents to answer

similar questions, which seem to be redundant, and lack of measurement of necessarily a

common latent variable by different questions [3].

In standard LCA models, the “local independence” assumption is used for providing the degrees

of freedom required for estimating model’s parameters [7, 10]. The assumption expresses that

indicators are independent from each other condition on the latent variable and 100% of

correlation among indicators are justified by the latent variable (Figure 1) [3, 7, 10]. In the

casual diagram of Figure (1), if a condition is placed on the latent variable, the connecting path

between the indicators A and B will be closed completely. Most statistical software and packages

prepared for LCA analysis have been designed based on this assumption.

Figure1 here

4

Local independence assumption may not be established while using LCA models for assessing

measurement error of survey data [3, 7]. Violation of the assumption is due to: (i) “behaviorally

correlated error” that occurs when a respondent, for instance, answer A and B indicators in a

survey wrongly in an intentional manner or when the answers to both indicators are incorrect

because the respondent has not understood the concept of indicators A and B. Therefore, A and

B indicators are dependent to each other condition on the latent variable as shown by Figure

(2.a). The first mode is probable in repeating a measurement in a survey and the second mode is

probable both in repeating measurement in a survey and in a re-measurement mode using another

survey [3, 9]. (ii) “bivocality” that occurs when indicators do not necessarily measure a common

latent variable. For example, indicator A measures latent variable X while indicator B measures

latent variable Y, and latent variables X and Y may be correlated, but not be identical. In this

mode, indicators A and B are also correlated condition on the latent variable (Figure 2.b). This

situation is probable following repeated measurements in a survey [3, 9]. (iii) “latent

heterogeneity” that occurs when the classification errors of indicators changes at the different

levels of an unknown grouping variable in a population. For instance, it is completely evident

that the probability of questions misunderstanding by people with low literacy is higher than the

ones with high literacy (Figure 2.c) [3, 9].

Figure 2 here

If non-establishment of local independent assumption isn’t considered in LCA models, the

measurement bias in estimating the parameters such as classification error rate of indicators or

prevalence of the outcome variable would not be corrected completely. For instance, if the

correlation between indicators’ classification errors is positive, estimation of their values for

different indicators by LCA model with local independent assumption will be

underestimated [11, 12].

The use of latent class log-linear (LCLL) models provides the flexibility required for data

analysis when local independent assumption is not established [1]. Non-establishment of local

independence assumption can be taken into account in LCLL models by inserting interaction

5

terms between the indicators, which are correlated due to “bivocality” or “behaviorally

correlated error” [3]. It is possible to provide sufficient degree of freedom for LCLL models by

inserting grouping variables to estimate correlation parameters between indicators [7]. Of

course, it is better to select the grouping variables among the variables, which have roles in latent

heterogeneity [3].

Application of LCA to IranMHS data on illicit drug use

IranMHS was a three-stage national household survey conducted from January to June 2011. The

primary goal was to estimate a 12-month prevalence and severity of psychiatric disorders among

15 to 64 years Iranian population. The validated Persian version of the paper and pencil

interview form (PAPI) of Composite International Diagnosis Interview version 2.1 (CIDI 2.1)

was used as the main tool for diagnosis of psychiatric disorders in this study. CIDI 2.1 includes

questions regarding drug and alcohol use disorders. The study design and field procedures have

been published elsewhere [13].

We intend to assess the measurement errors in estimating the prevalence of any illicit drugs use

in the past 12 months in IranMHS data by LCA approach. Various forms of cannabis;

amphetamine-type stimulants; opioids including opium and heroin/crack of heroin [14],; ecstasy

and LSD from the group of hallucinogens; volatile solvents; and other illicit substances were in

the list of illicit drugs in IranMHS. This list did not include sedative-hypnotics and codeine-

containing preparations that could be purchased over-the-counter.

The LCA approach using the latent class log-linear (LCLL) models were used to fit the IranMHS

data. A LCLL model was fit to this data using two indicators for any illicit drug use in the past

12 months according to the IranMHS questionnaires, and ten different LCLL models were also

fit to the data using three indicators. Moreover, a gender grouping variable was defined in order

to account for latent heterogeneity and to increase the degrees of freedom of the models.

The used indicators include: (i) Indicator A, which was based on answers to questions about

using any kind of illicit drugs without prescription for more than 5 times in the past 12 months

during a face to face interview, while presenting a list of those drugs to the interviewee. (ii)

Indicator B, which was based on answers to ten questions about using any kind of illicit drugs

without prescription for at least one time in the past 12 months using a self-administrated

6

questionnaire. (iii) Indicator D which asked the respondent’s needs to receive treatment for drug

use and dependency in the past 12 months, or using of any outpatient, inpatient, short-term

residential or traditional services because of drugs use and dependency in the past 12 months

during a face to face interview. Receiving agonist maintenance treatment and membership in NA

groups were excluded from indicator D, because many people in these long-term treatments

might be abstinent from their primary drug for long time. Definitions of these three indicators

are presented in Appendix A based on the specific terms used in IranMHS questionnaires.

Table (2) describes characteristics of 11 different LCLL models applied to the IranMHS data. In

these models, Y denotes the real drugs consumption status in the absence of classification errors

which is latent. Mode1 (0) is a two indicators model and models (1) to (10) are the three

indicators models. In the all models, the diversity of amount of the classification errors among

indicators is imposed to the data which is reflected through the interaction terms (AY and BY in

Model (0), and AY, BY and DY in Models (1) to (10)). In the all models, GY indicates different

amounts of the prevalence of any illicit drug use between genders after adjusting for the

classification errors. In Models (0), (1) and (9), it is assumed that there is not any correlation

between the indicators condition on the latent status, that is known as local independence

assumption [7, 10]. In the all models except Models (9) and (10), it is assumed that amount of

the classification errors for each indicator is the same in both genders. In Models (2)-(8) and

(10), different states where the local independence assumption does not hold are imposed to the

data. The interaction term between two indicators shows that there is correlation between the two

indicators condition on the latent Y variable. Model (9) was obtained by adding the interaction

terms AG, BG, and DG to Model (1). These terms impose to the data a specific kind of

variability in amount of the classification errors for each indicator between two genders. For

example, the BG term next to the BY term in Model (9) of Table (2) implies that the

multiplication product of the odds of false positive error in indicator B by the odds of false

negative error in indicator B is always constant for both genders. To establish this constancy, it is

necessary (as an example) for men who have responded to indicator B with a higher false

positive error probability than women to respond also to the indicator with a lower false negative

error probability. Thus, the above product of the odds shall remain constant for both genders.

This is discussed in more detail in Biemer and Wiesen (2002) [7].

7

By adding interaction terms (AD, AB) to Model (9), Model (10) is obtained which also imposes

the lack of local independence assumption on the data. There are two differences between Model

(10) and Model (1): (i) in Model (10), it is assumed that amount of the classification errors for

each indicator vary with gender, (ii) in Model (10), the local independence assumption no longer

holds.

To select the best model among the models with the three indicators listed in Table (2), the Lin

and Dayton criteria (1997) were used [15]. Based on these criteria, all of the three indicators

models in Table (2) were identifiable since the numbers of their parameters were smaller than

degrees of freedom (i.e. the cells number in the ABDG cross table) [16, 17], but Model (7) has

the lowest BIC as compared to the other models with three indicators. So it was selected as the

best model among the models with three indicators and was then used to estimate the prevalence

of any illicit drug use after adjusting for the classification error and the sensitivity and specificity

of each of the indicators.

The Expectation Maximization algorithm is one of the methods for estimating parameters of the

models [10, 17, 18]. So we used it as parameters estimator. The non-parametric bootstrap 95%

confidence intervals (CIs) of the estimates of two indicators and the best three indicators LCLL

models were constructed through independent resampling of the province strata in the main data.

This is from to the fact that the first stage of the sampling for IranMHS was carried out using

stratified sampling of provinces. Multi-stage sampling design of IranMHS was not taken into

account in the estimations.

All statistical analyses were done in R software version 3.2.0 [19]. The “emgllm” function of the

“gllm” package [20] was used for fitting the LCLL models. The sample codes of statistical

analysis in R software were presented in Appendix B.

Results of applying LCA on IranMHS data

The questionnaires, based on which the A and D indicators for any illicit drug use in IranMHS

were developed, were completed through face-to-face interviews with the participants. However,

the indicator B questionnaire was completed by the participants themselves and was put in a

closed ballot to protect their confidentiality. Table (1) presents the degree of disagreement

among indicators A, B and D for the use of any illicit drug in IranMHS. Indicator B shows a

higher degree of inconsistency with the indicators D and A, respectively.

8

Table 1 here

In Table (2), among the models with three indicators, Model (7) was selected as the best model.

This model takes into account correlation between indicators A, D and A, B in estimation of

prevalence of any illicit drug use in the past 12 months. From expert’s point of view, the

estimated prevalence of any illicit drug use by this model is logically high.

Table 2 here

Table (3) presents estimates of sensitivity and specificity (%) of indicators after application of

model with two A and B indicators (Model (0)) and selected model among models with three A,

B and D indicators (Model (7)) to the IranMHS data. The estimated specificity of all the

indicators by models (0) and (7) are extremely high because the likelihood of the false positive

response about the use of illicit drugs is very low logically. On the other hand, the estimated

sensitivity of the indicators by Model (7) is very lower than it by Model (0), because in Model

(0) correlation between the indicators was not imposed to the data, so amount of the false

negative error of the indicators was underestimated in this model.

The sensitivity of indicator B is higher than that of indicator A in Model (0) and two other

indicators in Model (7), and the sensitivity of indicator A is higher than that of indicator D in

Model (7). Since, indicator B directly questions the use of any illicit drugs at least one time in the

past 12 month using a self-administrated questionnaire; it has better sensitivity than the other

indicators. On the other hand, since the indicator D asks about feeling the need or service using

for treating the disorders resulted from any illicit drug use, and so ignored non-problematic drug

users, it has the lowest sensitivity among the indicators in Model (7).

Table 3 here

In Table (4) estimates of prevalence of any illicit drug use employing the LCLL models with two

and three indicators (adjusted for the classification errors) and those of the indicators A, B and D

9

(unadjusted) were presented by gender. After adjusting for the classification errors in the self-

report of illicit drug use, estimation of the prevalence of drug use increased. This increasing by

the LCLL model with two indicators is lower than that by the model with three indicators. The

reason is that measurement error correction by the LCLL model with two indicators is

incomplete. Because, this model did not take into account the correlation between the indicators.

The findings of Table (4) indicated also that the measurement error in self-report of illicit drug

use is an important factor in underestimating the prevalence of these drugs use. Although

adoption of techniques, such as the self-administrated questionnaire leads to a reduction in such

frequent errors in the course of research [21] , it does not fully prevent these errors.

Table 4 here

Discussion

In IranMHS data, the indicators used in LCA models are bivocal because they do not measure a

common latent variable. Indicator A measures the latent variable of “more than 5 times uses of

any illicit drug in the past 12 months”. Indicator B measures the latent variable of “at least one

time use of any illicit drug in the past 12 months”. Indicator D measures the latent variable of

“need for using services or using them due to drug abuse and addiction in the past 12 months”.

The latent variable of indicator D is nested in the latent variable of indicator A. The latent

variable of indicator A is nested in the latent variable of indicator B. The correlation between the

latent variables of indicators A and B is high; however, the correlation between the latent

variable of indicator D and the latent variables of other two indicators is weaker. Therefore,

indicator D is considered as a weak indicator.

On the other hand, a “behaviorally correlated error” is expected to occur among the three

indicators measured in the survey due to the question nature, i.e. illicit drug use. That is, if a

respondent denies illicit drug use in answering to indicator A intentionally, he will deny it

deliberately in answering B and D indicators. Of course, it may occur less for indicator D due to

its indirect questions.

10

Incidence of bivocality and behaviorally correlated error phenomena among IranMHS indicators

indicate non-establishment of local independence assumption when using the LCA approach on

the data of this survey. As correlation among indicators in this survey is positive, it is expected

that lack of considering violation of this assumption in LCA models would lead to bias in

estimating model parameters as underestimation of classification error of the indicators and

consequently incomplete correction of prevalence estimation of drug use. In Model (0), which

only used indicators A and B along with gender grouping variable for assessing and correcting

classification error of prevalence of illicit drugs use (this method is known as Hui-Walter

method [22]), local independence assumption was imposed to the data. Therefore, it is expected

that the classification error rate of these two indicators would be underestimated and estimation

of drug use prevalence would be lower than the real rate due to incomplete correction. Of course,

there is only 8 degrees of freedom in the two-indicator model along with one binary grouping

variable, which is used for estimating its eight parameters, and it will not be possible to consider

a correlation between A and B indicators through inserting an interaction term between them to

the model due to shortage of degrees of freedom.

As there are sufficient degrees of freedom for estimating parameters of correlation between

indicators in the three-indicator models with one gender grouping variable (Models 1 to 10),

non-establishment of local independence assumption can be imposed on the data in these models.

Among the three-indicator models of Table 2, local independence assumption is established in

Models 1 and 9. Therefore, these two models, like the two-indicator model (Model (0)), are not

suitable for IranMHS data. Among other three-indicator models, different modes of non-

establishment of local independence assumption were imposed on the data. It is noticed that

there is variation in the estimated value of prevalence of drug use among the models with

different modes of correlation structure among indicators. On the Other hand, experts believe

that the prevalence estimation obtained from selected three- indicator model i.e. model with

minimum BIC (Model 7), does not seem to be logical.

Indicators A and D in Iran MHS data showed the highest marginal correlation and the correlation

strength between indicators A and B is on the second order. It is noticed that BIC of the model

with interaction term AD is the lowest and BIC of the model with interaction term BD is the

highest among Models 2, 3, and 4, which only added one interaction term to the LCA models.

BIC of the model with AD and AB interaction terms is the lowest and BIC of the model with AB

11

and BD interaction terms is the highest among Models 5, 6, and 7, which added two interaction

terms to the LCA models. It means that a lower BIC was obtained naturally in the models in

which the correlation structure between indicators has further conformity with the correlation

structure between them in the data.

Both bivocality and behaviorally correlated error phenomena occurred in IranMHS data;

however, the correlation structure, which was created between indicators by these two

phenomena, does not correspond. While bivocality between AD and BD indicators is strong and

between AB indicators is weak, behaviorally corrected error between AD and AB indicators is

strong and between BD indicators is weak. The correlation structure caused by behaviorally

correlated error in the data exceeds the one of bivocality phenomenon. Therefore, some

bivocality remains in the models with further fitness with this correlation structure in the data

(such as Model 7 that have lowest BIC), which has not been considered, and it will lead to

obtaining some estimations, which are not completely unbiased.

On the other hand, only gender grouping variable was used in the LCA models for IranMHS

data. Statistically, this variable does not play a crucial role in creating latent heterogeneity (with

respect to BIC of Models 9 and 10). One of the other factors, which probably involves in

illogicality of estimations of the three-indicator model with lowest BIC (Model 7), is the

probability of a latent heterogeneity by the unknown variables that were not considered in

analyzing IranMHS data.

To achieve the unbiased estimations while assessing and correcting classification error of

categorical outcomes estimate in surveys using LCA approach, the factors violating local

independence, i.e. bivocality, behaviorally corrected error, and latent heterogeneity, should be

paid attention to and considered in models. It is proposed to define at least three strength

indicators for a latent variable while designing survey studies, which have univocality or at least

have minimum bivocality. Moreover, all the variables in which the classification error of

indicators at their levels is supposed to be variable and consequently lead to latent heterogeneity

should be identified and measured. Finally, well-known methods should be used for considering

any factor violating local independence assumption in the models while analyzing survey data by

LCA.

12

6. Aknowledgements

This paper was extracted from an epidemiology Ph.D. thesis in the School of Public Health of

Tehran University of Medical Sciences.

13

7. Ethics Statement

The study protocol was approved by the institutional review board of Tehran University of

Medical Sciences.

14

8. References

1. Berzofsky M, Biemer P, Kalsbeek W. A brief history of classification error models. Section on Survey Research Methodes-JSM 2008:3667-3673. 2. Kasprzyk D. Measurement error in household surveys: sources and measurement. Household Sample Surveys in Developing and Transition Countries. New York: UNITED NATIONS PUBLICATION 2005, p. 171-198. 3. Berzofsky ME, Biemer PP, Kalsbeek WD. Local dependence in latent class analysis of rare and sensitive events. Sociological Methods & Research 2014;43:137-170. 4. Biemer PP, Trewin D. A review of measurement error effects on the analysis of survey data. Survey measurement and process quality 1997:601-632. 5. Gustafson P, Greenland S. Misclassification. Handbook of Epidemiology: Springer; 2014, p. 639-658. 6. Lazarsfeld PF ,Henry NW, Anderson TW. Latent structure analysis: Houghton Mifflin Boston; 1968 7. Biemer PP, Wiesen C. Measurement error evaluation of self-reported drug use: a latent class analysis of the US National Household Survey on Drug Abuse. J. R. Statist. Soc .A 2002;165:97–119. 8. Spencer BD. When Do Latent Class Models Overstate Accuracy for Binary Classifiers?: With Applications to Jury Accuracy, Survey Response Error, and Diagnostic Error. In: Institute for Policy Research NU, editor. 2009. 9. Biemer PP. Latent class analysis of survey error: John Wiley & Sons; 2011 10. Agresti A. Categorical Data Analysis. Second Edition ed. Hoboken, New Jersey: John Wiley & Sons, Inc; 2002, p. 721. 11. Brenner H. Use and Limitations of Dual Measurements in Correcting for Nondifferential Exposure misclassification. Epidemiology 1992;3:216-222. 12. Walter SD, Irwig LM. Estimation of test error rates, disease prevalence and relative risk from misclassified data: a review. Journal of Clinical Epidemiology 1988;41:923-937. 13. Rahimi-Movaghar A, Sharifi V, Motevalian S-A, Amin-Esmaeili M, Hajebi A, Radgoodarzi R, et al. Iran Mental Health Survey (1389-90) 1ed. Tehran: Mehrsa; 1393.

14. Farhoudian A, Sadeghi M, Vishteh HRK, Moazen B, Fekri M, Movaghar AR. Component analysis of Iranian crack; a newly abused narcotic substance in Iran. Iranian journal of pharmaceutical research: IJPR 2014;13:337.

15. Lin TH, Dayton CM. Model selection information criteria for non-nested latent class models. Journal of Educational and Behavioral Statistics 1997;22:249-264.

16. Dayton CM, Macready GB. A scaling model with response errors and intrinsically unscalable respondents. Psychometrika 1980;45:343-356. 17. Goodman LA. Exploratory latent structure analysis using both identifiable and unidentifiable models. Biometrika 1974;61:215-231. 18. Vermunt J. Log-linear models for event histories, advanced quantitative techniques in the social sciences series. Thousand Oaks, CA: SAGE Publications, Inc 1997 19. R Core Team. R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing ;2015. URL https://www.R-project.org/. 20. Duffy D. Package ‘gllm’: Generalised log-linear model. 0.35 ed: CRAN; 2014. 21. Tourangeau R, Yan T. Sensitive questions in surveys. Psychological bulletin 2007;133:859. 22. Hui SL, Walter SD. Estimating the Error Rates of Diagnostic Tests. Biometrics 1980;36:167-171.

23. Nelder JA, Baker R. Generalized linear models: Wiley Online Library; 1972.

15

24. Schwarz G. Estimating the dimension of a model. The annals of statistics. 1978;6(2):461-464. 25. Wit E, Heuvel Evd, Romeijn JW. ‘All models are wrong...’: an introduction to model uncertainty. Statistica Neerlandica. 2012;66(3):217-236.

1

Tables and Figures

Table 1: Cross tabulation among the three indicators of any illicit drug use in the past 12

months

Indicator A Indicator D

Total Yes No

Yes Indicator B

Yes 44 (37.6) 64 (54.7) 108 (92.3)

No 1 (0.9) 8 (6.8) 9 (7.7)

Total 45 (38.5) 72 (61.5) 117 (100)

No Indicator B

Yes 3 (0.1) 114 (3.6) 117 (3.7)

No 18 (0.6) 3018 (95.7) 3036 (96.3)

Total 21 (0.7) 3132 (99.3) 3153 (100)

Total Indicator B

Yes 47 (1.4) 178 (5.4) 225 (6.9)

No 19 (0.6) 3026 (92.5) 3045 (93.1)

Total 66 (2.0) 3204 (98.0) 3270 (100)

2

Table 2: Characteristics of alternative latent class log-linear models applied on IranMHS

data for any illicit drug use

Model Degrees

of

freedom

Number of

parameters

Deviance1 P-value of

Likelihood

Ratio Test

BIC + [2*

ln(likelihood

of saturated

model)]2

Prevalence

(adjusted for

classification

error) 3

Model 0: (AY, BY,

GY)

0 8 1.70 0.99 66.44 6.14

Model 1: (AY, BY,

DY, GY)

6 10 50.45 < 0.001 131.38 5.56

Model 2: same as

model 1 + (AB)

5 11 40.92 0.006 129.94 7.31

Model 3: same as

model 1 + (BD)

5 11 50.37 < 0.001 139.39 5.52

Model 4: same as

model 1 + (AD)

5 11 19.90 0.53 108.92 6.18

Model 5: same as

model 1 + (AD, BD)

4 12 15.77 0.73 112.88 6.23

Model 6: same as

model 1 + (AB, BD)

4 12 38.84 0.007 135.95 7.98

Model 7: same as

model 1 + (AD, AB)

4 12 7.43 0.995 104.54 28.87

Model 8: same as

model 1 + (AD, BD,

AB)

3 13 3.77 0.9999 108.98 42.17

Model 9: same as

model 1 + (AG, BG,

DG)

3 13 9.22 0.97 114.42 3.55

Model 10: same as

model 9 + (AD, AB)

1 15 20.37 0.25 141.76 49.99

1 - 2 * [ln(likelihood of current model) – ln(likelihood of saturated model)]

2 The reason for replacing BIC with "BIC + [2* ln(likelihood of saturated model)]" has been explained in Appendix

C. 3 %

3

Table 3: Estimated sensitivity and specificity (%) of indicators for any illicit drug use in the

past 12 months

Indicator

By LCLL model with 2 indicators (Model (0)) By LCLL model with 3 indicators (Model (7))

Sensitivity (95% CI1) Specificity (95% CI

1) Sensitivity (95% CI

1) Specificity (95% CI

1)

A 58.29 (49.63 , 67.53) 99.999 (99.999 , 99.999) 12.39 (7.65 , 34.45) 99.999 (99.994 , 99.999)

B 92.32 (87.13 , 96.82) 98.705 (98.008 , 99.380) 20.84 (13.57 , 54.89) 98.786 (98.135 , 99.397)

D ____ ____ 6.77 (4.03 , 18.43) 99.909 (99.667 , 99.999)

1 Calculated using nonparametric bootstrapping with 500 iterations.

4

Table 4: Comparison of prevalence estimates by indicators A, B, D and LCLL models with

2 and 3 indicators for use of any illicit drug in the past 12 months

Gender Prevalence 1 (95% CI)

Indicator A Indicator B Indicator D LCLL (2 indicator)2 LCLL (3 indicator)

2

Males n=3366

7.22 (6.39 , 8.14)

n=1447

12.78 (11.16 , 14.61)

n= 3369

4.45 (3.80 , 5.20)

n = 1446

12.69 (10.81 , 14.67)

n = 1446

40.69 (10.04 , 78.86)

Females n=4475

0.54 (0.36 , 0.80)

n=1825

2.19 (1.61 , 2.97)

n= 4475

0.49 (0.32 , 0.75)

n = 1824

0.94 (0.43 , 1.61)

n = 1824

4.73 (1.30 , 9.11)

Total n=7841

3.40 (3.03 , 3.83)

n = 3272

6.88 (6.06 , 7.80)

n= 7844

2.19 (1.89 , 2.54)

n = 3270

6.14 (5.17 , 7.14)

n = 3270

28.87 (10.45 , 43.74)

1 %

2 95% CIs calculated using nonparametric bootstrapping with 500 iterations

5

Figure1: Local Independence assumption (Ɛ is measurement error)

Latent Variable

Indicator B

Indicator A

ƐB

ƐA

6

Figure 2: Violation of the local independence assumption (Ɛ is measurement error)

A) Behaviorally correlated error:

B) Bivocality:

C) Latent heterogeneity: