Factor Structure of the Difficulties in Emotion Regulation Scalein Treatment Seeking Adults with Eating Disorders

Line Nordgren1& Elin Monell2 & Andreas Birgegård2

& Johan Bjureberg2,3& Hugo Hesser1

Published online: 4 December 2019

AbstractThe Difficulties in Emotion Regulation Scale (DERS) is extensively used as a measure of emotion (dys-)regulation ability in bothclinical and nonclinical populations. This is the first study to examine the factor structure of both the original 36-item and short16-item version of the DERS in adults with eating disorders and to test measurement invariance across diagnostic subgroups. Thefactor structure of the scale was examined using confirmatory factor analysis in a psychiatric sample of adults with eatingdisorders (N = 857). Four primary factor structures were fitted to the data: (1) a unidimensional model, (2) a six-factor correlat-ed-traits model, (3) a higher-order factor solution, and (4) a bifactor model. Measurement invariance was tested for diagnosticsubgroups of anorexia nervosa and bulimia nervosa and associations between factors and eating pathology were examined ineach diagnostic group. Results indicated that a modified bifactor solution fitted the data adequately for both the 36-item and 16-item version of the DERS. A general factor explained most of the variance (86%) and reliability was high for the general factor ofDERS (total) but lower for the subscales. Measurement invariance of the bifactor model was supported across diagnosticsubgroups and test of factor means reveled that bulimia nervosa had a higher factor mean than anorexia nervosa on the generalfactor. The general factor accounted for a significant proportion of variance in eating pathology. Our results support the use of thetotal scale of both the 36-item and 16-item version among adults with eating disorders.

Keywords Difficulties in emotion regulation scale . Emotion . Confirmatory factor analysis . Measurement invariance . Eatingdisorders

The data reported in this manuscript were obtained from SwedishStepwise clinical database [STEPWISE/KÄTS]. A bibliography ofjournal articles using STEPWISE is available at [http://www.atstorning.se/forskning-utbildning-2/forskning-inom-kats/kats-senaste-publicerade-forskning-om-atstorningar/]. The main variable, Difficulties in EmotionRegulation Scale (DERS), examined in the present article has been usedin one previous study (cited in the current manuscript). To the best of ourknowledge the DERS has never been analyzed with the same statisticalmethods or with the same study aim as in the present study.

Electronic supplementary material The online version of this article(https://doi.org/10.1007/s10862-019-09765-8) contains supplementarymaterial, which is available to authorized users.

Journal of Psychopathology and Behavioral Assessment (2020) 42:111–126https://doi.org/10.1007/s10862-019-09765-8

# The Author(s) 2019

* Hugo Hesserhugo.hesser@liu.se

Line Nordgrenline.nordgren@liu.se

Elin Monellelin.monell@ki.se

Andreas BirgegårdAndreas.Birgegard@ki.se

Johan Bjurebergjohan.bjureberg@ki.se

1 Department of Behavioural Sciences and Learning, LinköpingUniversity, SE-581 83 Linköping, Sweden

2 Centre for Psychiatry Research, Department of ClinicalNeuroscience, Karolinska Institutet, Norra Stationsgatan 69, SE-11364 Stockholm, Sweden

3 Stockholm Health Care Services, Stockholm County Council,Stockholm, Sweden

Eating disorders can be described as a prolonged disturbanceof eating or related behavior such as selective or restrictiveeating, compulsory eating behavior, binge eating episodesand/or compensatory behavior such as fasting, frequentvomiting, use of laxatives or diuretics as well as excessiveand compulsory exercise (American Psychiatric Association2013). Eating disorders are divided into sub-diagnoses an-orexia nervosa, bulimia nervosa, binge eating disorder andother specified feeding or eating disorders, the latter includingatypical anorexia nervosa, bulimia nervosa and binge eatingdisorder of low frequency and/or limited duration, ruminationdisorder, and purging disorder (American PsychiatricAssociation 2013). Overall eating disorder point prevalencehas been reported between 0.5% and 5.3% for females andfrom 0.62% to 0.64% in males (Lindvall Dahlgren andWisting 2016). In eating disorders, emotion regulation seemsto play an important role. Previous research, for example,indicates that the risk of suicide attempts is highly elevatedin individuals with eating disorders (Pisetsky et al. 2013) and alink between suicide attempts, non-suicidal self-injury andemotion dysregulation in individuals with eating disordershas been suggested (Gómez-Expósito et al. 2016; Vieiraet al. 2016).

Emotion regulation has emerged as a central component ofboth theories of psychopathology and psychological interven-tions (Gross 2015). Emotion regulation can be conceptualizedas the awareness, understanding and acceptance of one’s emo-tions, the ability to inhibit inadequate behaviors when emo-tionally aroused, and the ability to use adaptive regulationstrategies in order to reach one’s goals (Gratz and Roemer2004). Individuals with eating disorders have been found todisplay higher levels of emotion dysregulation compared tohealthy controls, lower levels of emotional awareness, clarity,and recognition as well as problems regarding emotional in-hibition and access to healthy emotion regulation strategies(e.g., Lavender et al. 2015; Monell et al. 2018). Regardingdifferent eating disorder diagnostic subgroups, Lavenderet al. (2015) concludes in a review that global emotion regu-lation difficulties seems to be a transdiagnostic trait across theeating disorder spectrum, but that there might be some distinctpatterns that potentially distinguishes the different eating dis-orders. Although numerous studies have reported relation-ships between eating disorders and emotion dysregulation(e.g., Brockmeyer et al. 2014; Lavender et al. 2015), a consis-tent conceptualization of emotion regulation seems to be lack-ing within the field of eating disorders. This is evidenced bythe use of a wide range of different measures such as thesubscale interoceptive awareness from the Eating DisorderInventory (Garner et al. 1983), the Emotional AwarenessQuestionnaire (Rieffe et al. 2007), the Toronto AlexithymiaScale (Bagby et al. 1994), the Emotion RegulationQuestionnaire (Gross and John 2003) and the Difficulties inEmotion Regulation Scale (DERS; Gratz and Roemer 2004).

Given the growing interest in emotion regulation in eatingdisorders, there is a need for valid and reliable measures. Onewidely used measure in both research and clinical practice isthe aforementioned DERS (Gratz and Roemer 2004). Unlikemost other measures that mainly focus on specific aspects ofemotion regulation, the DERS was designed to comprehen-sively measure multiple dimensions of emotion regulationability. The original version of the DERS includes 36 itemsformulated as assertions scored 1–5 yielding a total score aswell as scores on six subscales (henceforth referred to asDERS-36). In the original study by Gratz and Roemer(2004), exploratory factor analysis (EFA) revealed a correlat-ed trait lower-order six-factor solution. The six factors or sub-scales were presented as distinct but related dimensions withadequate internal consistency. All items in the final explorato-ry factor solution had factor loadings of .40 or higher on thecorresponding subscale and none of the items in the finalversion had significant loadings (above .40) on more thanone factor (Gratz and Roemer 2004).

The six subscales were named Nonacceptance, Goals,Impulse , Awareness , Strategies, and Clarity. TheNonacceptance subscale (nonacceptance of emotional re-sponses) is composed of items reflecting a tendency to expe-rience negative secondary emotional responses or anonaccepting reaction to distress, mainly shame, guilt, orself-blame regarding one’s own (negative) emotions. Itemsfrom the Goals subscale (difficulties engaging in goal directedbehavior) reflect difficulties regarding concentration oraccomplishing tasks when upset. The Impulse subscale (im-pulse control difficulties) reflect difficulties remaining in con-trol of one’s behavior when feeling distress and also includesitems reflecting the perception of emotions as overwhelming.The fourth subscale Awareness (lack of emotional awareness)comprises reverse scored items reflecting the tendency to payattention to and acknowledge emotions. The subscaleStrategies (limited access to emotion regulation strategies)consists of items reflecting a belief that there is nothing thatcan help regulate negative emotions, suggesting a kind ofhopelessness when confronted with one’s feelings. The finalsubscale Clarity (lack of emotional clarity) addresses the abil-ity to understand emotions; a high score indicates a high de-gree of confusion regarding emotions.

Previous research has demonstrated support for the validityof the DERS. The DERS and its subscales has shown corre-lations with other scales measuring related aspects of emotionregulation, such as alexithymia, levels of positive and negativeaffect, experiential avoidance and suppression, in other popu-lations including individuals with alcohol dependence, chron-ic pain, and aggression as well in psychiatric inpatient adoles-cents (e.g., Ghorbani et al. 2017; Kökönyei et al. 2014; Velottiet al. 2016; Venta et al. 2012). Levels of emotion regulationdifficulties as measured by the DERS have also been found tobe associated with different kinds of psychopathology such as

112 J Psychopathol Behav Assess (2020) 42:111–126

personality disorders, posttraumatic stress disorders, depres-sion and anxiety (e.g., Gratz et al. 2006; Sloan et al. 2017; Tulland Roemer 2007; Villalta et al. 2018; Visted et al. 2018).Regarding eating disorders, previous research has found asso-ciations between the DERS (total scale and all subscales) andlevels of eating disorder pathology (e.g., Brockmeyer et al.2014; Burns et al. 2012; Cooper et al. 2014; Haynos et al.2015; Monell et al. 2015; Racine and Wildes 2015), and thereis also evidence of relations to specific eating disorder behav-iors (e.g., Burns et al. 2012; Monell et al. 2018). Furthermore,studies incorporating other emotion regulation measures inaddition to DERS show that the relationships between theDERS and its subscales and levels of eating disorder pathol-ogy are fairly consistent with associations that were observedwhen using other measures that tap into similar constructs(e.g., Svaldi et al. 2012). Levels of emotion regulation diffi-culties in individuals with eating disorders, as measured by theDERS, have also been found to decrease following treatment(e.g., Mallorquí-Bagué et al. 2018; Sloan et al. 2017).

Since the original factor analytic study, a number of studieshave examined the psychometric properties of the DERS-36.The majority of these studies have, however, been conductedin non-clinical samples. Several of the previous studies reportacceptable or good fit for the original correlated traits six-factor solution in non-clinical adults (e.g., Bostan andZaharia 2016; Ritschel et al. 2015) and adolescents (e.g.,Neumann et al. 2010; Sarıtaş-Atalar et al. 2015). However, afew studies investigating the factor structure of the DERS-36in non-clinical samples report trouble with the original six-factor solution, primarily relating to problems with theAwareness subscale (e.g., Bardeen et al. 2012; Lee et al.2016). For example, Bardeen et al. (2012) found support fora revised five-factor higher order solution and Lee et al. (2016)suggested a five-factor lower order model – both excluded thesix items from the Awareness subscale.

While the factor structure of the DERS-36 has been exam-ined in non-clinical samples, only a handful of studies haveexamined the factor structure in clinical populations (Fowleret al. 2014; Hallion et al. 2018; Osborne et al. 2017; Perezet al. 2012; Wolz et al. 2015). Perez et al. (2012) investigatedthe factor structure of DERS-36 in a sample of 218 adoles-cents with nonsuicidal self-injury. Confirmatory factor analy-sis (CFA) suggested acceptable fit for the six-factor originalcorrelated traits solution. Fowler et al. (2014) examined asample of 592 adult in-patients with severe mental illnessand reported acceptable and equivalent fit with the originalcorrelated traits six-factor solution. In the recent study byOsborne et al. (2017), the factor structure of the DERS-36was examined in an adult population of 344 outpatients re-ceiving Dialectical Behavior Therapy (DBT). Seven differentmodels were tested, including two unidimensional models,three correlated traits models, one higher-order model andone bifactor model. Results showed good fit for the bifactor

model including the original six factors, with a modificationthat prohibited items from the Awareness subscale from load-ing on the general factor but allowing the factor Awareness tobe correlated with the Clarity subscale (Osborne et al. 2017).Support for a bifactor solution was replicated by Hallion et al.(2018) using CFA in a study of 427 adults with emotionaldisorders. However, unlike the study by Osborne et al.(2017), Hallion et al. (2018) found support for a five-factorbifactor solution that excluded the subscale Awarenessaltogether.

The only study to our knowledge to examine the factorstructure of the DERS-36 in patients with eating disorders, isthe study byWolz et al. (2015). The study involved 74 healthycontrols and 134 adult patients with a DSM-IVeating disorder(including anorexia nervosa, bulimia nervosa, binge eatingdisorder, and other specified feeding or eating disorders).The sample was analyzed with both EFA and CFA. EFA re-sults suggested a one-dimension solution as well as a six-factor solution with close resemblance to the original factorstructure. CFA results showed acceptable fit for a model com-parable to the original six-factor correlated traits solution(Wolz et al. 2015).

In addition to the original 36-item version of the DERS,Bjureberg et al. (2016) developed and evaluated a brief ver-sion of the DERS consisting of 16 items that generates a singleglobal score for emotion regulation difficulties (henceforthreferred to as the DERS-16). The DERS-16 showed goodconvergent and discriminant validity, excellent internal con-sistency and good test-retest reliability and was concluded tooffer a valid and brief method for assessing emotion regulationdifficulties (Bjureberg et al. 2016). The factor structure of theDERS-16 has been investigated in five studies. Miguel et al.(2017), Shahabi et al. (2018), Westerlund and Santtila (2018),and Yiugit and Guzey Yiugit (2017) all found support for thefive-factor solution proposed by Bjureberg et al. (2016) innon-clinical populations. Hallion et al. (2018) examined thefactor structure of the DERS-16 in a clinical sample consistingof 427 adults with emotional disorders. Results indicated goodfit for a bifactor model.

In summary, the factor structure of DERS-36 and DERS-16has been investigated in several studies. However, most factoranalytic studies are based on non-clinical samples and there islimited knowledge of the factor structure of both versions inclinical samples. To our knowledge, only one study has ex-amined the DERS-36 in an eating disorder population (Wolzet al. 2015). That study, however, was marked with somelimitations, most significantly the relatively small sample sizefor the analyses that were conducted in the study (i.e., CFA).The small sample size also precluded a careful examination ofthe factor structure in different subpopulations of eating dis-orders. Further, only one model was investigated using CFAmeaning that there were no direct comparisons between dif-ferent factor solutions. Also, given that both EFA and CFA

J Psychopathol Behav Assess (2020) 42:111–126 113

was conducted using the same sample in the study, there is aclear need to confirm the factor structure using CFA in anindependent larger sample of individuals with eatingdisorders.

The present study is the first to investigate the factor struc-ture of both the 36-item and 16-item DERS in a large eatingdisorder sample. Based on previous examinations of the factorstructure of the 36-item DERS in clinical and non-clinicalsamples, a series of CFA models were fitted and comparedin the present study. Given previously identified problemswith the Awareness subscale of the DERS-36, a priori modi-fications were made to each model that was tested.Specifically, four primary structures were fitted to the dataand compared in terms of model fit: (1) a unidimensionalmodel, (2) a six-factor correlated-traits model, (3) a higher-order factor model, and (4) a bifactor model. We were primar-ily interested in the bifactor model that has to date not beenexamined among individuals with eating disorder but has pre-viously been successfully tested on the DERS-36 in two psy-chiatric samples. The bifactor model also has a clear advan-tage by examining the specific variance accounted for by thesubfactors of DERS-36 over and above the varianceaccounted for by the general factor. Thus, information gainedfrom the bifactor model can, for example, be used to deter-mine the adequacy of the total score of the original DERS-36and the DERS-16, as well as the additional value of scoringsubscales as the correlations among all the items are accountedfor by both a general factor and set of subfactors in the model.Finally, the present study is, to our knowledge, the first to testfor measurement and structural invariance across subgroupsof different eating disorders, that is, anorexia nervosa andbulimia nervosa.

Methods

Participants

The sample was drawn from the Swedish Stepwise clinicaldatabase (Birgegård et al. 2010) with nationwide data on treat-ment seeking individuals of all ages and both genders witheating disorder; however, in the present study only adult pa-tients (≥18 years) were included. Data was extracted inOctober 2017. Stepwise is a longitudinal internet-based qual-ity assurance register developed in 2005. Inclusion criteria forentry in the Stepwise database are a DSM-IV eating disorder(American Psychiatric Association 2000), medical or self-referral to a participating treatment unit, and intention to treatat the unit. All measures are collected during an initial assess-ment within the first three visits at the clinical unit.Assessment was carried out by eating disorder professionalswith special training (a 2-day mandatory course). TheStepwise database includes structured diagnostic interviews

for eating disorders and other DSM-IVAxis I disorders andcontains a variety of both mandatory and optional assessmentinstruments (i.e. units can decide which of the optional instru-ments to include in their test battery). For a full description ofStepwise, see Birgegård et al. (2010).

The DERS-36 was added as an optional assessment forunits on 7th April 2014. Registrations prior to this date werethus excluded, leaving N = 2625 patients from 30 units. Thefollowing exclusions were then made: no DERS ratings (n =1674), ratings from patients at units with <20 DERS-36 ad-ministrations (n = 94), leaving a final simple size of 857 pa-tients from 14 units. A previous study using the DERS-36from the Stepwise register, largely sharing the present sample(N = 999, only women), could not find any differences be-tween patients with or without DERS-36 registration regard-ing variables such as age, body mass index, eating disordercharacteristics, and psychiatric comorbidity (see Monell et al.2018).

The present sample consisted of 820 women and 37 menwith a mean age of 26. As is typical in the eating disorderpopulation, mean age of eating disorder onset was in adoles-cence. The majority of the participants had graduated fromupper secondary school and had an employment of at leasthalf-time. Bulimia nervosa was the most common diagnosisin the sample (n = 319), followed by purging disorder (n =189), anorexia nervosa (n = 141; whereof n = 90 for restrictingsubtype, n = 32 for binge/purging subtype, and n = 19 for an-orexia nervosa except no amenorrhea), atypical anorexianervosa (n = 131), binge eating disorder (n = 41), unspecifiedfeeding or eating disorder (n = 23) and rumination disorder(n = 13). Demographic information for the final sample is pre-sented in Table 1. Eating disorder diagnoses wererecategorized to correspond to the diagnostic criteria inDSM-5 (American Psychiatric Association 2013); eating dis-orders not otherwise specified type 1 (anorexia nervosa exceptno amenorrhea) were recategorized into anorexia nervosa andeating disorders not otherwise specified type 3 (bulimianervosa except binge/purge frequency lower than the DSM-IV threshold) into bulimia nervosa.

Measures

DERS DERS is a self-assessment scale measuring emotiondysregulation. The original DERS includes 36 items scored1–5 where 1 is almost never (0–10%), 2 is sometimes (11–35%), 3 is about half the time (36–65%), 4 is most of the time(66–90%), and 5 is almost always (91–100%). Of the 36items, 11 are reverse scored. The DERS-36 yields a total scoreas well as six subscales where higher scores indicate moredifficulties. The DERS-36 has demonstrated adequate con-struct and predictive validity as well as good test-retest reli-ability (Gratz and Roemer 2004). The DERS-16 is a shortversion developed by Bjureberg et al. (2016). In this study,

114 J Psychopathol Behav Assess (2020) 42:111–126

the DERS-36 was administrated in Swedish to all participantsand the items composing the DERS-16 were extracted fromparticipants’ scores on the original 36-item version. TheSwedish DERS-36 and DERS-16 have been used in severalprevious studies with retained reliability and validity (e.g.Bjureberg et al. 2016; Garke et al. 2019; Monell et al. 2018;Monell et al. 2015). In the selected sample of participants,there were no missing data on the DERS.

Structured Eating Disorder Interview (SEDI) SEDI is a semi-structured clinical interview assessing eating disorder symp-toms and establishes a preliminary DSM-IV eating disorderdiagnosis which is then confirmed by an expert clinician. Theinterview includes 20–25 questions and is based on SCID-I(First et al. 2002). Preliminary validation against EatingDisorder Examination Interview (Cooper and Fairburn 1987)has shown concordance of 81% for specific eating disorderdiagnosis and subtype (de Man Lapidoth and Birgegård2010).

Eating Disorder Examination Questionnaire (EDE-Q Version4.0) EDE-Q is a self-report measure assessing eating disordersymptomatology with 36 items scored 0 to 6, where 22 of the36 items yields four subscales where higher scores indicatemore pathology: Restraint, Eating concern, Shape concern,and Weight concern, as well as a mean Global score(Fairburn and Bèglin 1994). The EDE-Q is frequently usedin eating disorder treatment and research and it has establishedreliability (Luce and Crowther 1999) and validity (Mond et al.2004).

Data Analytic Approach and Statistical Analysis

Confirmatory Factor Analysis (CFA)To examine the underlyingfactor-structure and to identify the best-fitted factor model, aseries of CFA models were fitted based on previous

examinations of the factor structure of the 36-item DERS inclinical and non-clinical samples. Models were fitted with thenon-normality robust maximum likelihood estimator (MLR)using Mplus vs. 7.4 (Muthén 2015).1 The primary structuresthat were examined were (a) a unidimensional model (Model1), (b) the original posited six-factor correlated-traits model(Model 2), (c) a second-order model with uncorrelated six-factors and a higher-order factor representing general emotionregulation ability (Model 3), and (d) a bifactor model in whichitems loaded directly on both the general factor and their cor-responding subscale factor (Model 4). Given previously iden-tified problems with the Awareness subscale of the DERS-36,one a priori modification was made to each one of these pri-mary CFA structures (Model 1–4). The six Awareness itemswere either excluded from the analysis (model 1.2 and 2.2) orthe Awareness factor was allowed to be correlated with theClarity factor in models where covariance between factorswere constrained to zero as default (model 3.2 and model4.2). In total, eight models were fitted and compared to iden-tify the best-fitted model. Once the best-fitted model was iden-tified, the model was subsequently used to test for measure-ment and structural invariance across subgroups of partici-pants with different eating diagnoses, for scale reliability anal-yses, and for test of the factor structure of the DERS-16.

To evaluate and compare model fit, the Root Mean SquareError of Approximation (RMSEA) were used with values

1 We also reran the primary set of models in which we controlled for depen-dence in the data due to the nesting of individuals within clinics (using theCluster option in Mplus) and with a different estimator (robust weighted leastsquares means and variance adjusted estimator [WLSMV]) that treated theindicators as ordered categorical. Although this improved model fit overall,these analyses did not alter the overall conclusions in terms of the best-fittedmodel among the set of estimated models. For this reason and given that theWLSMVestimator cannot produce estimates of error variance that is requiredfor the computation of scales reliability statistics in the form of Omega andOmegaH, we present the results from analyses withMLR estimator that treatedindicators as non-normal continuous.

Table 1 Descriptive and clinical characteristics of the sample

Total (N = 857) Anorexia Nervosa (n = 272) Bulimia Nervosa (n = 319)

Characteristics M SD % (n) M SD % (n) M SD % (n)

Age 26.37 8.23 24.24 7.38 27.09 8.19

Age of ED onset 16.45 5.60 16.83 5.56 15.65 4.05

BMI 22.94 6.16 18.75 3.83 25.32 5.57

EDE-Q 3.86 1.19 3.66 1.36 4.15 0.96

Gender 96 (820) 94 (255) 97 (308)

Education 80 (688) 22 (60) 31 (100)

Employment 79 (673) 76 (207) 83 (264)

Living alone 39 (337) 34 (91) 42 (134)

The subgroup Anorexia Nervosa in this table includes a combination of anorexia nervosa (n = 141) and atypical anorexia nervosa (n = 131); BMI, BodyMass Index; EDE-Q, Eating Disorder Examination Questionnaire version 4.0, total score; Gender, female; Education, upper secondary graduate;Employment, employment above 50%; ED, eating disorder

J Psychopathol Behav Assess (2020) 42:111–126 115

<.08 and < .06 as benchmarks for adequate and good fit (Huand Bentler 1999; Wang and Wang 2012). In addition, weprovided the 90% confidence interval (CI) around theRMSEA value and only considered a model to have a goodfit if the upper limit was below .08 (Wang and Wang 2012).Simulation studies have shown that RMSEA performs betterthan other fit indices in certain applications (see Wang andWang 2012), and we thus relied on it as the primary measureof fit. We also employed the Comparative Fit Index (CFI)(Bentler 1990), with values >.90 and > .95 as benchmarksfor adequate and good model fit (Bentler 1990; Hu andBentler 1999), and the Standardized Root Mean SquareResidual (SRMR) with values <.08 as a benchmark for a goodfitted model (Hu and Bentler 1999). When comparing nestedmodels, a scaled chi-square difference test was used for modelcomparisons (Muthén 2015; Satorra and Bentler 2001). Amore constrained model (null model) was deemed to fit worsethan an alternative less restrictive model if the increase in chi-square statistics was statistically significant at the level ofp < .05, with degrees of freedom equal to the difference in freeparameters between models. In addition, following recent rec-ommendations for testing measurement invariance (Cheungand Rensvold 2002; Wang and Wang 2012), we also exam-ined a drop in CFI between models with a delta CFI less thanor equal to .01 as indicative of no significant difference be-tween models.

Scale Reliability and Explained Common Variance (ECV)Reliability measures for total scales and subscales were basedon estimated loadings and measurement errors. Specifically,for both total scales and subscales, we calculated Omega(Raykov 1997) that uses all sources of reliable variances inthe calculation of estimates of reliability (and is comparable toa weighted alpha coefficient; Reise 2012; Rodriguez et al.2016a, b), and Hierarchical Omega (OmegaH; Rodriguezet al. 2016b; Zinbarg et al. 2005) that uses only the reliablevariance that is specifically accounted for by the particularscale under examination (disregarding other reliable variancethat was not accounted for by the scale). Similar to coefficientalpha, values range from 0 (no reliability) to 1 (perfect reli-ability). These “model-based” indices of reliability are gener-ally considered to yield more accurate estimates of reliabilitythan simple alpha coefficients mainly because they do notassume that each item is measured with the same degree ofprecision and measurement error (as assumed with coefficientalpha; Graham 2006; Raykov 1997). In addition, OmegaH hasbeen suggested to be particularly useful for models that pro-vide estimates of variance accounted for by both a generalfactor and subscale factor (e.g., bifactor or higher-order factormodels) because these indices separate reliable variance ex-plained by the general and subfactors and hence do not con-flate variance (Rodriguez et al. 2016b). Specifically, OmegaHfor the total scale provides an estimate of the reliable variance

explained by all items loading on the general factor whilstpartialling out reliable variance accounted for by the loadingson the subscales, whereas OmegaH for the subscale providesan estimate of the reliable variance explained by the items forthe subscale whilst partialling out reliable variance accountedfor by these items’ loadings on the general factor.

Based on recommendations for evaluating bifactor models,we also calculated ECV as a measure of how much of thecommon variance was explained by the general factor(Rodriguez et al. 2016b). High ECVindicates that the majorityof the common variance is accounted for by the general factor,whereas low ECV indicates that most of the explained vari-ance is accounted for by the subscales. ECV can hence bethought of as an indicator of the degree to which the constructis unidimensional (Rodriguez et al. 2016b).

Measurement Invariance Once a global structure had beenidentified in the full sample, we aimed to test for measurementinvariance across groups with different eating diagnoses fol-lowing recommendations provided by Wang and Wang(2012). Specifically, we tested measurement invariance ofthe best-fitted model across individuals with bulimia nervosa(n = 319) and individuals with anorexia nervosa (n = 272).Data from participants with atypical anorexia nervosa diagno-sis and anorexia nervosa diagnosis were combined to form theanorexia nervosa group. This was done for three reasons.First, these diagnoses only differ in terms of the weight criteria(for atypical anorexia nervosa the individual has lost a signif-icant amount of weigh although still not considered signifi-cantly underweight) and thus we considered the groups to besimilar in terms of diagnostic criteria. Second, this increasedsample size to an acceptable level for these complex SEMmodels. Third, as reported in previous studies (e.g.,Moskowitz and Weiselberg 2017; Sawyer et al. 2016), indi-viduals with anorexia nervosa and atypical anorexia nervosashow more similarities than differences in levels of malnutri-tion and dietary restrictions, type of medical complicationsand psychological morbidities.

To establish factorial structure invariance, the best-fittedmodel identified in the full sample was first fitted in eachsubsample to examine whether the overall factor structurewas similar across groups. The model fit indices used to eval-uate model fit in the full sample were used to determinewhether the model fit adequately in each subsample. Themodels were then combined into a multigroup CFA to testfor both configural invariance and strong measurement invari-ance (i.e., invariant loadings and intercepts; also known asScalar invariance; Meredith 1993). The configural modelwas also fitted as a baseline model in which subsequent morerestrictedmodels could be compared. The baselinemodel (i.e.,configural model) included the same factor structure with thesame pattern of fixed and free loadings in each group butwithout posing any equality restrictions on any measurement

116 J Psychopathol Behav Assess (2020) 42:111–126

(intercepts, loadings, and residuals) and structural parameters(factor variance) across groups. To formally test for strongmeasurement invariance (Meredith 1993), we then fitted amore restricted model in which both factor loadings and itemintercepts were constrained to be invariant across groups andtested whether these constraints significantly degraded modelfit. If model fit was not significantly worse in the constrainedmodel as compared to the baseline model, strong measure-ment invariance was presumed to be established. Given thatthe measurement invariant model was nested within theconfigural model (i.e., baseline model), the scaled chi-squaredifference test and delta CFI were used to determine whetherconstraints significantly worsened model fit.

Structural Invariance Once measurement invariance wasestablished, we proceeded to test the structural parameters ofthe best-fitted CFA model using multiple-group CFA analy-ses. Specifically, we first tested the factor variance and thenproceeded to test the factor means (usingmodels with equalityconstraints on loadings and intercepts in accordance with thestrong measurement invariance assumption). Factor variancewas set to be invariant in the model and this model was com-pared with the same model but where factor variances werefree parameters in all groups. The scaled chi-square differencetest and the decrement in CFI were used to examine whetherequality constraints on factor variance significantly worsenedmodel fit (Wang and Wang 2012).

Following the test of factor variance, we proceeded andtested whether the factor means for the factor of the subscalesand the total scale differed as a function of eating disordergroup (using the anorexia nervosa group as the reference cat-egory). In order not to inflate alfa for multiple comparisons,we used the scaled chi-square difference test (similar toANOVA) to examine an overall difference between a modelthat constrained all factor means to be equal across groupswith a model in which factor means differed as a function ofeating disorder group (using one group as a reference categoryfor identification purposes). If the ratio test was found to bestatistically significant, we proceeded to test each factor meandifference with normal theory tests (i.e., estimate/standarderror).

Criterion Validity In the best-fitted CFA model, we examinedthe contribution of each factor dimension in accounting forvariability in eating pathology as measured with EDE-Q.Our aim was to explore whether each latent factor could ac-count for unique variance in the external criterion by incorpo-rating EDE-Q as a dependent variable in the structural part ofthe CFA model (similar to regression analysis). By using thelatent factors, as compared to the observed scores of thescales, we could eliminate regression bias due to measurementerror and explicitly model the true score variance of eachfactor and their diverging associations with the external

criterion. Models were run separately for individuals with bu-limia nervosa (n = 319) and individuals with anorexia nervosa(n = 272).

Results

Factor Structure of the DERS-36

Descriptive statistics for each item of the DERS-36 are pre-sented in supplementary Table S1 and descriptives for the totalscale of DERS and subscales are presented in Table 2.Measures of skewness and kurtosis for both items and scaleswere generally low. In total, 8 models were estimated usingCFA. Table 3 provides model fit indices for all estimatedmodels in the full sample (N = 857).

Unidimensional Models The unidimensional model (model1.1) using all 36 items of the DERS-36 provided a poor fitto the data (see Table 3). The same was true for the secondunidimensional model (model 1.2) that excluded the sixAwareness items.

Traits Models The six-factor correlated traits model based onall 36 items of the DERS showed adequate fit according toRMSEA but only approached adequate CFI and SRMR (mod-el 2.1). Factor loadings were all significant in the model.Correlations between latent factors were in the range of .20to .75. Noteworthy is that the Awareness factor had relativelylow correlations with all other factors (all <.30) with the onlyexception for the correlation with the Clarity factor (.75). Thefive-factor correlated traits model that excluded the Awarenessitems evidenced acceptable model fit (model 2.2). All itemshad significant factor loadings and correlations between latentfactors ranged from .45 to .78.

Higher-Order Models The first higher-order factor model thatallowed the six factors to load on a higher-order factor evi-denced acceptable fit according to RMSEA but onlyapproached an acceptable CFI and SRMR (model 3.1). Themodified version of the higher-order model (model 3.2),which allowed the Awareness and Clarity factor to be corre-lated, did improve model fit significantly, Δχ2(1) = 209.362,p < .001. However, examining model fit indices indicated nosubstantial improvement in fit (see Table 3). Factor loadingswere all significant in the model and the model estimatedcorrelation between the Awareness and Clarity factor was .75.

Bifactor Models The first unmodified bifactor model thatallowed items to load directly on both the general factor andeach subscale factor (model 4.1) evidenced an acceptable fitaccording to RMSEA but only approached acceptable CFI

J Psychopathol Behav Assess (2020) 42:111–126 117

and SRMR. However, the modified version of the bifactormodel (model 4.2), which also estimated the correlation be-tween the Awareness and Clarity factors, improved model fit,Δχ2(1) = 180.284, p < .001.2 The model evidenced a good fitaccording to RMSEA and SRMR, and an acceptable fit ac-cording to CFI.

Model Comparisons Examination of fit indices provided thestrongest support for the modified bifactor model (model 4.2).Specifically, whereas the modified correlated traits model wasacceptable, the final bifactor model was the only model thatevidenced a good fit for certain fit indices, most importantlyRMSEA. Given these results, and the fact that the bifactormodel allowed us to directly evaluate the usefulness of thesubscales and the total scale in one combined model, thebifactor model was selected for further analyses. In addition,the modified correlated traits model that demonstrated an ad-equate fit excluded the Awareness items and the model thusprovided no information regarding the behavior of theAwareness items in relation to the other 30 items of the DERS.

Reliability and Explained Common Variance

Omega and OmegaH values obtained from the estimated finalselected bifactor model are presented in Table 4. The overallreliability values (Omega), which can be thought of as weight-ed coefficient alpha, of the total and subscales were all high(range .796–.963). The OmegaH value for the total scale waslikewise high, indicating that almost all of the reliable variancein total scores (.852/.962 = .885) could be attributed to thegeneral factor, assumed to measure general emotion dysregu-lation. Despite high omega values for all subscales, low

OmegaH values for the Goals, Impulse and Strategies sub-scales, however, indicate that most of the reliable varianceon these subscales was accounted for by the reliable variancedue to the general factor. The opposite was true for theAwareness and Clarity subscales that both had relatively highOmegaH and subsequently explained more of the reliable var-iance relative to the general factor (proportion = .94 and .70respectively).

The estimate of ECV statistics was .86. This suggested thatmost of the common variance was accounted for by the gen-eral factor and thus only a small proportion of the variance(1–.86 = .14) appeared to be explained by the subscale factorsbeyond the general factor.

Measurement and Structural Invarianceacross Anorexia Nervosa and Bulimia Nervosa

Measurement Invariance The bifactor solution fitted ade-quately in both the anorexia nervosa and bulimia nervosa sub-samples (see Table 3). The combined multigroup configuralmodel that posed no equality restrictions on intercepts andloadings across groups also fitted adequately (see Table 3).Multigroup CFA analyses revealed that a model thatconstrained intercepts and loadings (i.e., strong measurementinvariance) to be equal across eating disorder groups did notsignificantly degrade model fit relative to the configural mod-el, Δχ2(94) = 101.216, p = .287 and ΔCFI <.001. This sug-gested that strong measurement invariance assumption wasmet, and we could proceed and test for differences in structuralparameters across groups (i.e., factor variance and factormeans).

Structural InvarianceMultigroup CFA analyses revealed that amodel that constrained factor variance to be equal acrossgroups did not significantly degrade model fit relative to amodel in which factor variance were set as free parametersin each group (assuming strong measurement invariance inline with previous findings), Δχ2(8) = 6.382, p = .604 and

2 Given that the scaled chi square test for MLR produced a negative value dueto a negative correction factor and thus could not be used, we instead used thez-value for the specific test of the covariance between Awareness and Clarityfactor to obtain the chi-two value for model comparison with one degree offreedom (as two times the z-value approximately follow the chi-squaredistribution).

Table 2 Descriptive statistics for total and subscales of Difficulties in Emotion Regulation Scale (36 items) as a function of diagnostic groups

Subscale Total (N = 857) Anorexia Nervosa (n = 272) Bulimia Nervosa (n = 319)

Sum SD M SD Skew Kurt Sum SD M SD Skew Kurt Sum SD M SD Skew Kurt

Total 100.12 26.43 2.80 0.73 0.10 0.17 99.31 27.38 2.78 0.76 0.16 −0.32 102.32 26.62 2.86 0.73 −0.00 −0.53Non-accept 16.15 6.22 2.69 1.04 0.37 −0.72 16.15 6.10 2.69 1.02 0.30 −0.69 16.17 6.34 2.69 1.06 0.39 −0.84Goals 16.30 5.30 3.26 1.06 −0.20 −0.93 15.93 5.41 3.19 1.08 −0.08 −1.13 16.87 5.16 3.37 1.03 −0.39 −0.75Impulse 14.39 6.17 2.40 1.03 0.61 −0.48 13.85 6.21 2.31 1.03 0.74 −0.30 15.11 6.15 2.52 1.02 0.46 −0.65Awareness 18.23 5.15 3.04 0.86 −0.15 −0.55 18.24 5.15 3.04 0.86 0.02 −0.52 18.40 5.14 3.07 0.86 −0.26 −0.48Strategies 21.10 7.75 2.64 0.97 0.29 −0.76 20.72 8.11 2.59 1.01 0.40 −0.70 21.76 7.64 2.72 0.95 0.18 −0.82Clarity 13.95 4.59 2.79 0.92 0.15 −0.58 14.40 4.67 2.88 0.93 0.03 −0.65 14.01 4.61 2.80 0.92 0.16 −0.51

Skew, Skewness;Kurt, Kurtosis. The scores from DERS total scale and subscales are presented as sums as well as means, with the purpose to enablestandardized comparisons between subscales due to the fact that the DERS subscales consist of different number of items (5–8 items per subscale)

118 J Psychopathol Behav Assess (2020) 42:111–126

ΔCFI <.001. This indicated that dispersion in factor scoreswere similar across groups.

Multigroup CFA analysis revealed that the bifactor modelwhich held intercepts, loadings, factor variance and factormeans equal across groups, fitted worse than an identical modelbut in which factor means were free parameters in bulimianervous subsamples (using the anorexia nervosa diagnosis asthe reference group with a mean of zero), Δχ2(7) = 22.284,p = .002. When inspecting specific mean differences, a statisti-cally significant difference was detected for the factor mean ofthe general factor (z = 2.163, p = .031, d = .214) with partici-pants with bulimia nervosa having a higher factor mean thanthose with anorexia nervosa (with a factor mean difference of.099 points). There was also statistically significant differenceon the Clarity factor with participants with bulimia nervosahaving a lower factor mean (mean difference = .136) than thosewith anorexia nervosa (z = 2.298, p = .022, d = .237). There wasalso a tendency that participants with bulimia nervosa had alower mean on the Strategies factor (mean difference = .069)and on the Nonacceptance factor (mean difference = .155) thanthose with anorexia nervosa, but these differences onlyapproached and did not reach conventional level of statisticalsignificance (z = 1.918, p = 0.055, d = .277, z = 1.703, p = .089,d = .167, respectively). None of the other comparisons werestatistically significant (all p’s > .6 and d’s < .05).

Factor Structure of the DERS-16

The bifactor model that was established as the best-fitted mod-el for all the 36 items was used to evaluate the factor structureof the 16-item version of the DERS in the full sample. Giventhat this shortened version did not include the Awareness

scale, no covariance could subsequently be added betweenthis subfactor and the Clarity factor (as model 4.2 estimatedin the full sample). In addition, due to the fact that the Claritysubscale only had two indicators, loadings were constrained to1 for these items to make the model identified globally. Thebifactor model provided a good fit to the data (χ2 (89) =379.985, p < .001; RMSEA = .062 [95% CI .055, .068];CFI = .954; SRMR= .04).

Criterion Validity and Diverging Associationswith Eating Pathology

To examine the associations between dimensions of emotiondysregulation (i.e., latent factors) and eating pathology andwhether the subfactors could account for unique variance overand above the general factor, we incorporated EDE-Q (as ameasure of eating pathology) as an observed dependent vari-able into the best-fitted bifactor model. First, a model was run inwhich the EDE-Qwas regressed on the general factor only (andall the subfactors effects on EDE-Q were constrained to zero.)Second, a model was run in which EDE-Q was regressed on allthe subfactors and the general factor. The contribution of allsubfactors ability to account for unique variance in the outcomeabove the contribution of the general factor was examined byimprovement in global model fit between models using thescaled chi-square difference test. This analytic procedure wasrepeated for individuals with bulimia nervosa and individualswith anorexia nervosa.

For the anorexia nervosa subsample, the general factor wasstatistically significantly associated with scores on the EDE-Q(z = 5.099, p < .001) and accounted for approximately 15% ofthe variance (R2 = .154). The subfactors accounted for an

Table 3 Estimated confirmatory factor analysis models for DERS-36 using the MLR estimator

Model Chi-2 df CFI SRMR RMSEA [90% CI]

Unidimensional Model 1.1 7320.013 594 0.592 0.111 0.115 [0.113, 0.117]

Model 1.2 5164.097 405 0.663 0.092 0.117 [0.114, 0.120]

Trait Model 2.1 2440.130 579 0.887 0.065 0.061 [0.059, 0.064]

Model 2.2 1729.675 395 0.905 0.058 0.063 [0.060, 0.066]

Higher-order Model 3.1 2756.743 588 0.869 0.082 0.066 [0.063, 0.068]

Model 3.2 2472.762 587 0.886 0.067 0.061 [0.059, 0.064]

Bivariate Model 4.1 2382.028 558 0.889 0.073 0.062 [0.059, 0.064]

Model 4.2 2010.668 557 0.912 0.049 0.055 [0.053, 0.058]

Measurement invariance

Bifactor AN 1047.596 557 0.914 0.059 0.057 [0.052, 0.062]

BN 1153.135 557 0.906 0.051 0.058 [0.053, 0.063]

Configural 2199.933 1114 0.910 0.055 0.057 [0.054, 0.061]

Scalar 2293.930 1208 0.910 0.060 0.055 [0.052, 0.059]

Best fitted model in bold

CFI, Comparative Fit Index; SRMR, Standardized Root Mean Residual; RMSEA, Root Mean-square error of approximation; AN, Anorexia Nervosa;BN, Bulimia Nervosa; CI, Confidence interval

J Psychopathol Behav Assess (2020) 42:111–126 119

additional 2% of the variance, but the overall contribution wasnot statistically significant, Δχ2(6) = 9.509, p = .147. For thebulimia nervosa subsample, the general factor was statisticallysignificantly associated with scores on the EDE-Q (z = 5.245,p < .001) and accounted for approximately 14% of the variance(R2 = .144). The subfactors accounted for an additional 5.7% ofthe variance, a contribution that was statistically significant,Δχ2(6) = 13.055, p = .042. However, despite accounting for asignificant contribution together, none of the estimates of theeffects of subfactors on EDE-Q had a statistically significantindividual contribution in the model (all p’s > .06; all standard-ized beta’s < .17).

Discussion

Factor Structure of DERS

The factor structure of the DERS-36 in a large eating disordersample was examined using CFAwith eight different models.Results indicated that a modified bifactor model that allowedthe subscale Awareness to correlate with the subscale Clarity(model 4.2) provided best fit. This model was the only modelto reach acceptable or good fit on all indices. Further, it wasthe only model to reach good fit according to our primarymeasure of fit (i.e., RMSEA). A similar bifactor solutionwas also found to be a good fit for the DERS-16. The bifactormodel suggest that the composite score has more than onesource of variance, meaning that the scores of all items onthe DERS-36 and DERS-16 are influenced by more thanone distinct underlying construct (i.e. subscales) beyond thecontributions from the general factor. A bifactor model of theDERS-36 and DERS-16 has previously been suggested in twostudies (Hallion et al. 2018; Osborne et al. 2017). Similar toour model, both of these studies made modifications to themodel due to identified problems with the Awareness sub-scale. Osborne et al. (2017) tested a bifactor model that ex-cluded items of Awareness scale from loading onto the totalscale but permitted a correlation between this subfactor andthe Clarity factor. Hallion et al. (2018) tested a bifactor modelwith all items from the Awareness subscale excluded. Thus,while the overall structure differed slightly with previous test-ed bifactor models, our result aligns with previous examina-tions of the factor structure of DERS in other clinicalpopulations.

In comparisonwith the other tested bifactor models in otherclinical samples, our bifactor model also allowed us to explic-itly test the contribution of the Awareness subscale in account-ing for unique and shared variance. The Awareness concept orsimilar constructs are common features in several emotionregulation models as well as in several models of psychiatricpathology (D’Agostino et al. 2017) and specific models ofeating disorder pathology (Lavender et al. 2015). Keepingthe Awareness subscale is also in line with recommendations

from the study by Fowler et al. (2014). While their study didnot test a bifactor solution, the authors concluded that removalof the Awareness subscale did nothing to improve fit in theirsample of 592 adult patients with severe mental illness. Theonly previous study of the factor structure of the DERS-36 inan eating disorder sample also found acceptable fit for a(correlated-traits) solution that included Awareness (Wolzet al. 2015). As for the DERS-16, the results from the presentstudy indicated a good fit for a bifactor model, a finding that isin line with the results from Hallion et al. (2018) that is theonly previous study to investigate the factor structure of theDERS-16 in a psychiatric clinical sample. While general rec-ommendations on the factor structure are difficult to providegiven different populations and methods in these studies, thepresent study adds additional support to this factor structure inanother clinical population.

Reliability and Explained Common Variance

Reliability of DERS-36 for individuals with eating disorderwas examined in terms of Omega (comparable to a weightedcoefficient alpha), OmegaH (reliable variance accounted forby the particular scale excluding reliable variance accountedfor by the general factor), and ECV (the degree of commonvariance explained by the general factor). Results showed thatOmegaH and the explained proportion of variance were highfor DERS total, indicating that most of the explained commonvariance can be attributed to the total scale. Regarding thesubscales, results varied. The results from the present studyas well as the study by Osborne et al. (2017) are in line withearlier findings suggesting that the large majority of psycho-logical multidimensional measures show similar tendencieswith high OmegaH and ECV for the measures total scaleand corresponding low OmegaH for subscales (Rodriguezet al. 2016a). There is no consensus in the field regardingspecific benchmarks for evaluatingOmegaH, but values great-er than .50 or .75 have been suggested (Reise et al. 2013).Comparing with these cutoffs, our results indicate that halfof the DERS subscales (Awareness, Clari ty, andNonAcceptance) reach sufficient or close to sufficientOmegaH values and the other half present lower values. Inthis context it is important to note that all Omega values (com-parable to a weighted coefficient alpha) were high for allsubscales.

The subscales Goals, Impulse, and Strategies all exhibitedlow OmegaH, indicating that these subscales contribute withlittle unique explanatory value beyond the contributions of thetotal scale. The Awareness, Clarity and Nonacceptance sub-scales showed medium to relatively high OmegaH-values.Only one previous study by Osborne et al. (2017) has present-ed OmegaH for the DERS. Results were similar with lowOmegaH for Goals, Impulse and Strategies as well as highOmegaH for the subscale Clarity. Osborne et al. (2017) did

120 J Psychopathol Behav Assess (2020) 42:111–126

not provide OmegaH for the Awareness subscale as their mod-ified bifactor model prevented Awareness items from loadingonto the total scale. OmegaH for DERS total were similar toours.

There is no consensus in the literature regarding the use of theDERS total score or the use of subscale scores in clinical popu-lations. For example, Fowler et al. (2014) cautions against the useof DERS total, since the results from the higher-order models intheir study showed a weak fit, suggesting that future use shouldfocus primarily on the six subscales, whereas Osborne et al.(2017) recommended the use of a general or total score fromthe modified DERS with only five subscales (excluding theAwareness subscale). Taken together, the results from the presentstudy indicate that there may be reasons for being cautious wheninterpreting scores of some of the DERS subscales in an eatingdisorder population as most of the reliable variance might beaccounted for by a general factor representing shared varianceamong all the items. This is especially true for Strategies.Regarding Awareness, the high OmegaH values indicate thatthe subscale contributes a high proportion of unique varianceover and above the variance explained by the general factor.This could lead to questions regarding the underlying latent con-struct, as it is possible that Awareness measures some differentaspect of emotion regulation than the rest of the items of theDERS (Bardeen et al. 2012; Lee et al. 2016). The present find-ings dovetail with previous studies that have found small inter-correlations between the Awareness factor and the other factors(Fowler et al. 2014; Osborne et al. 2017; Perez et al. 2012; Wolzet al. 2015). On the other hand, the Awareness subscale is theonly one to meet the preferred benchmark of .75 or higherOmegaH suggested by Reise et al. (2013). It might thus be pos-sible to use this scale on its own, but more difficult to knowwhether the scale actually taps into to a more general emotionregulation ability as measured by the DERS.

Measurement and Structural Invarianceacross Anorexia Nervosa and Bulimia Nervosa

The present study is the first to test for measurement andstructural invariance for the DERS-36 across the eating

disorder subgroups anorexia nervosa and bulimia nervosa.Results showed that strong measurement invariance assump-tion was met. This suggests that the DERS-36 measures thesame underlying latent construct in both groups, meaning thatit is possible to make relevant comparisons between patientswith anorexia nervosa and bulimia nervosa in terms of emo-tion regulation difficulties as measured by the DERS-36. Thisresult is of importance in the field as comparisons and corre-lations between emotion regulation difficulties measured bythe DERS in different eating disorder subgroups is a commonstudy aim in eating disorder research (e.g. Brockmeyer et al.2014; Lavender et al. 2015; Segal and Golan 2016; Svaldiet al. 2012). Further analyses showed that the dispersion infactor scores was similar across groups. Regarding mean fac-tor score differences between anorexia nervosa and bulimianervosa the results indicated small but statistically significantdifferences for DERS total as well as Clarity, with higher totalmean and lower mean on Clarity in the bulimia nervosa group.No other comparisons were statistically significant. Taken to-gether, patients with anorexia nervosa and bulimia nervosaseem to have fairly similar emotion regulation difficulties asmeasured by the DERS-36, a finding that is consistent withprevious research (e.g., Brockmeyer et al. 2014; Lavenderet al. 2015; Monell et al. 2018; Svaldi et al. 2012).

Criterion Validity

Results from bifactor models revealed that the general factorof emotion dysregulation as measured with the DERS in thebest-fitted bifactor model was strongly associated with eatingpsychopathology in both anorexia nervosa and bulimianervosa subsamples (15% and 14% respectively). These find-ings are in line with previous research that has shown thatemotion dysregulation is an important feature of eating pathol-ogy across eating disorders (e.g., Brockmeyer et al. 2014;Lavender et al. 2015). Interestingly, once the variance by thegeneral factor was accounted for in the model, the overallcontribution of the subfactors was relatively small in bothdiagnostic groups (2% and 5%) and was not statistically sig-nificant in the anorexia nervosa subsample. None of the

Table 4 Omega and Omega Hierarchical for total and subscales of the DERS-36

Scale Omega OmegaH Proportion of reliable variance (OmegaH/Omega)

Total 0.963 0.852 0.885

Non-Accept 0.897 0.479 0.534

Goals 0.895 0.249 0.278

Impulse 0.918 0.261 0.284

Awareness 0.796 0.749 0.941

Strategies 0.904 0.098 0.108

Clarity 0.842 0.589 0.700

OmegaH, Omega Hierarchical

J Psychopathol Behav Assess (2020) 42:111–126 121

subfactors had a statistically significant contribution on theirown in accounting for eating pathology. By incorporating anexternal criterion of relevance for eating disorders (eating dis-order psychopathology), these findings corroborate our otherresults showing that most of the reliable variance wasaccounted for by the general factor. Of course, one mightargue that it is difficult to know what it actually means to behigh (or low) on a subscale that is supposed to tap into aspecific emotion regulation skill when partielling out generalemotion ability. Indeed, concerns in terms of the interpretationof a bifactor model have been raised (Bonifay et al. 2017;Murray and Johnson 2013). On the other hand, if most ofthe reliable variance is indeed accounted for by a generalfactor, and in the absence of evidence for reliability of thesubscale scores, it is difficult to provide a meaningful inter-pretation of the associations between observed subscalesscores of the DERS and other important variables. At this timepoint, it is difficult to make definitive conclusions as moreresearch on the validity of the DERS in individuals with eatingdisorders is warranted. Still, in this subsample of individualswith eating disorders, our results highlight that the subfactorscannot be considered to reflect broad, independent abilitiesbecause they include a strong contribution of the generalfactor.

Strengths and Limitations

The findings of this study should be interpreted consideringsome strengths and limitations. First, a major strength is thelarge psychiatric eating disorder sample that provides an eco-logically valid context for assessment. The DERS was devel-oped for populations with clinical problems regarding emotionregulation and different kinds of psychopathology (Gratz andRoemer 2004). The present study makes important contribu-tions regarding the psychometric properties and use of theDERS-36 and DERS-16 in the field of clinical psychiatric eat-ing disorders. Second, the present study investigates the latentstructure of the DERS with several models, including a bifactormodel that has previously been suggested to be suitable formeasures that are assumed to have a multidimensional structure(such as the DERS). The use of a bifactor model also allowedfor investigations of reliability in terms of Omega, OmegaH,and explained common variance, giving important informationregarding the previously identified problems with certain sub-scales. To our knowledge, there is only one previous study thathas reported OmegaH for the DERS and the present study is thefirst to present OmegaH for all six subscales. Lastly, the test formeasurement invariance is an important strength as this pro-vides essential information regarding comparisons of emotionregulation across eating disorder diagnostic subgroups.

There are also a few limitations with the current study. TheDERS was added as an optional measure in the Stepwiseregister and criteria for clinics’ decisions in inclusion are not

recorded. A previous study using the DERS from the Stepwiseregister (N = 999, only women) could not find any differencesbetween patients with or without DERS registration regardingvariables such as age, body mass index, eating disorder char-acteristics and psychiatric comorbidity (see Monell et al.2018). The fact that the present study largely shares this sam-ple makes it plausible that this should be true for this study aswell. Another related aspect concerns the nesting of individ-uals within clinics that could potentially have influenced theresults. It is however important to note that sensitivity analyseswere run that controlled for the nesting of observations withinunits and results were not altered materially from thosepresented.

Second, although the bifactor model comes with certainadvantages, some concerns have also been raised. The bifactormodel has shown tendencies to outperform other models interms of fit statistics, possibly due to statistical bias, makingmodel comparisons and interpretation of fit indices more chal-lenging (Bonifay et al. 2017; Murray and Johnson 2013). It isalso important to note that the differences in fit between someof the factor models were not substantial and even though thebifactor model provided an adequate fit to the data in thecurrent sample, we cannot know for sure if this is a validrepresentation of the structure in the population.Additionally, conceptual concerns have also been raised re-garding fitting bifactor models to psychological constructs, forexample related to interpretation of the meaning of a bifactorstructure (Bonifay et al. 2017; Murray and Johnson 2013).Notwithstanding these concerns, as pointed out by Bonifayet al. (2017) and others, the bifactor model still might be ofimportance for the development and evaluation of measure-ments given that it offers a unique opportunity to calculatevarious useful indices (e.g., OmegaH, EVC) that can assistin determining whether a measure has an acceptable true scorevariance and the extent to which subscales scores are reliableafter accounting for the general factor.

Our study did not examine validity by comparing theDERS with other established emotion regulation measures.It should, however, be noted that the validity of the DERShas previously been examined by associations with other rel-evant self-assessed variables in both nonclinical and clinicalpopulations, including eating disorders (e.g., Brockmeyeret al. 2014; Monell et al. 2015; Svaldi et al. 2012). It wouldbe most informative if future research also makes efforts toestablish the validity of the DERS in other ways than merelystudying correlations among self-report assessments, for ex-ample, using data from multimethod and longitudinal studiesor experiments to assess the evidence of criterion relevance,discriminative validity and responsiveness to treatment amongindividuals with different eating disorders.

The sample in the current study was limited to treatmentseeking adults (mostly females) with eating disorder withoutany reference group (e.g., normal controls). This, naturally,

122 J Psychopathol Behav Assess (2020) 42:111–126

makes it difficult to determine the extent to which these resultscan be generalized to other clinical and nonclinical popula-tions. Also, regarding measurement and structural invariancefor diagnostic subgroups, the present study is limited to com-parisons between individuals with anorexia nervosa and bu-limia nervosa. For future research, the final bifactor solutionneeds replication as this is the first study to test that specificmodel in a clinical sample of eating disorder. There is also aneed to broaden the investigation between subgroups to dif-ferent kinds of eating disorders as well as comparisons be-tween subgroups defined in other ways than by the diagnosticmanual. The eating disorder diagnostic criteria have been crit-icized (e.g., Forbush and Wildes 2017) due to the overlap ofbehavioral symptoms and the fact that individuals with eatingdisorders tend to move between different sub-diagnoses overtime. The findings of this study also warrant replication in asample of younger patients.

Conclusions

The present study is the first to examine the factor structure ofthe DERS-36 and DERS-16 in a large clinical eating disordersample, and to test a bifactor model as well as measurementinvariance across diagnostic subgroups in this population.Results indicated good fit for a bifactor model with permittedcorrelations between the subscales Awareness and Clarity.Analyses of reliability suggest that the DERS-36 and theDERS-16 total scale are reliable, while results from the reli-ability analyses of the subscales varied. The interpretation ofsome specific subscales is less clear as most of the reliablevariance accounted for by these subscales overlapped with thevariance accounted for by the general factor. The general fac-tor also accounted for most of the variance in eating pathologyand none of the subfactors had a statistically significant con-tribution on their own. Finally, results also showed that criteriafor strong measurement invariance were met, indicating thatmeaningful comparisons of the eating disorder subgroups(i.e., anorexia nervosa and bulimia nervosa) using the DERSare possible. Taken together, findings indicate that the use ofthe total scale can be recommended when administered in aneating disorder population.

Acknowledgements Open access funding provided by LinköpingUniversity.

Compliance with Ethical Standards

Conflict of Interest Line Nordgren, Elin Monell, Andreas Birgegård,Johan Bjureberg and Hugo Hesser declare that they have no conflict ofinterest.

Experiment Participants All procedures performed in studies involvinghuman participants were in accordance with the ethical standards of theinstitutional and/or national research committee and with the 1964

Helsinki declaration and its later amendments or comparable ethicalstandards.

Informed Consent Informed consent was obtained from all individualparticipants included in the study.

J Psychopathol Behav Assess (2020) 42:111–126 123

Open Access This article is distributed under the terms of the CreativeCommons At t r ibut ion 4 .0 In te rna t ional License (h t tp : / /creativecommons.org/licenses/by/4.0/), which permits unrestricted use,distribution, and reproduction in any medium, provided you giveappropriate credit to the original author(s) and the source, provide a linkto the Creative Commons license, and indicate if changes were made.

References

American Psychiatric Association. (2000). Diagnostic and statisticalmanual of mental disorders: DSM-IV-TR. Washington, DC:American Psychiatric Association.

American Psychiatric Association. (2013). Diagnostic and statisticalmanual of mental disorders: DSM-5™ (5th ed.). Arlington, VA,US: American Psychiatric Publishing, Inc..

Bagby, R. M., Taylor, G. J., & Parker, J. D. A. (1994). The twenty-itemToronto alexithymia scale: II. Convergent, discriminant, and concur-rent validity. Journal of Psychosomatic Research, 38(1), 33–40.https://doi.org/10.1016/0022-3999(94)90006-X.

Bardeen, J. R., Fergus, T. A., & Orcutt, H. K. (2012). An examination ofthe latent structure of the difficulties in emotion regulation scale.Journal of Psychopathology and Behavioral Assessment, 34(3),382–392. https://doi.org/10.1007/s10862-012-9280-y.

Bentler, P. M. (1990). Comparative fit indexes in structural models.Psychological Bulletin, 107(2), 238–246. https://doi.org/10.1037/0033-2909.107.2.238.

Birgegård, A., Björck, C., & Clinton, D. (2010). Quality assurance ofspecialised treatment of eating disorders using large-scale internet-based collection systems: Methods, results and lessons learned fromdesigning the Stepwise database. European Eating DisordersReview, 18(4), 251–259. https://doi.org/10.1002/erv.1003.

Bjureberg, J., Ljótsson, B., Tull, M. T., Hedman, E., Sahlin, H., Lundh, L.G., … Gratz, K. L. (2016). Development and validation of a briefversion of the difficulties in emotion regulation scale: The DERS-16. Journal of Psychopathology and Behavioral Assessment, 38(2),284–296. https://doi.org/10.1007/s10862-015-9514-x.

Bonifay, W., Lane, S. P., & Reise, S. P. (2017). Three concerns withapplying a bifactor model as a structure of psychopathology.Clinical Psychological Science, 5(1), 184–186. https://doi.org/10.1177/2167702616657069.

Bostan, C. M., & Zaharia, D. V. (2016). Emotional dysregulation - factorstructure and consistency in the Romanian version of the difficultiesin emotion regulation scale (DERS). Annals of the Al. I. CuzaUniversity, Psychology Series, 25(2), 57–77.

Brockmeyer, T., Skunde, M., Wu, M., Bresslein, E., Rudofsky, G.,Herzog, W., & Friederich, H. C. (2014). Difficulties in emotionregulation across the spectrum of eating disorders. ComprehensivePsychiatry, 55(3), 565–571. https://doi.org/10.1016/j.comppsych.2013.12.001.

Burns, E. E., Fischer, S., Jackson, J. L., & Harding, H. G. (2012). Deficitsin emotion regulation mediate the relationship between childhoodabuse and later eating disorder symptoms.Child Abuse and Neglect,36(1), 32–39. https://doi.org/10.1016/j.chiabu.2011.08.005.

Cheung, G. W., & Rensvold, R. B. (2002). Evaluating goodness-of-fitindexes for testing measurement invariance. Structural Equation

Mode l ing , 9 ( 2 ) , 233–255 . h t t p s : / / do i . o rg /10 .1207 /S15328007SEM0902_5.

Cooper, Z., & Fairburn, C. (1987). The eating disorder examina-tion: A semi-structured interview for the assessment of thespecific psychopathology of eating disorders. InternationalJournal of Eating Disorders, 6(1), 1–8. https://doi.org/10.1002/1098-108X(198701)6:1<1::AID-EAT2260060102>3.0.CO;2-9.

Cooper, J. L., O’Shea, A. E., Atkinson, M. J., & Wade, T. D. (2014).Examination of the difficulties in emotion regulation scale and itsrelation to disordered eating in a young female sample. InternationalJournal of Eating Disorders, 47(6), 630–639. https://doi.org/10.1002/eat.22278.

D’Agostino, A., Covanti, S., Rossi Monti, M., & Starcevic, V.(2017). Reconsidering emotion dysregulation. PsychiatricQuarterly, 88(4), 807–825. https://doi.org/10.1007/s11126-017-9499-6.

deMan Lapidoth, J., & Birgegård, A. (2010). Validation of the structuredeating disorder interview (SEDI) against the eating disorder exam-ination (EDE). Stockholm: Karolinska institutet.

Fairburn, C. G., & Bèglin, S. J. (1994). Assessment of eating disorders:Interview or self-report questionnaire? International Journal ofEating Disorders, 16(4), 363–370.

First, M., Spitzer, R. L., Gibbon, M. L., &Williams, J. (2002). Structuredclinical interview for DSM-IV-TR Axis I disorders, research ver-sion, non-patient edition. (SCID-I/P). New York: Biometrics re-search, New York state psychiatric institute.

Forbush, K. T., &Wildes, J. E. (2017). Application of structural equationmixture modeling to characterize the latent structure of eating pa-thology. International Journal of Eating Disorders, 50(5), 542–550.https://doi.org/10.1002/eat.22634.

Fowler, J. C., Charak, R., Elhai, J. D., Allen, J. G., Frueh, B. C.,& Oldham, J. M. (2014). Construct validity and factor struc-ture of the difficulties in emotion regulation scale amongadults with severe mental illness. Journal of PsychiatricResearch, 58, 175–180. https://doi.org/10.1016/j.jpsychires.2014.07.029.

Garke, M., Sörman, K., Jayaram-Lindström, N., Hellner, C., &Birgegård, A. (2019). Symptom shifting and associations with men-tal illness : A transdiagnostic approach applied to eating disorders.Journal of Abnormal Psychology, 128(6), 585–595. https://doi.org/10.1037/abn0000425.

Garner, D . M. , Olms tead , M. P. , & Pol ivy, J . (1983) .Multidimensional eating disorder inventory for anorexianervosa and bulimia. International Journal of EatingDisorders, 2(2), 15–34.

Ghorbani, F., Khosravani, V., Shari, F., & Jamaati, R. (2017). Thealexithymia, emotion regulation, emotion regulation difficulties,positive and negative affects, and suicidal risk in alcohol-dependent outpatients. Psychiatry Research, 252, 223–230. https://doi.org/10.1016/j.psychres.2017.03.005.

Gómez-Expósito, A., Wolz, I., Fagundo, A. B., Granero, R., Steward, T.,Jiménez-Murcia, S., et al. (2016). Correlates of non-suicidal self-injury and suicide attempts in bulimic spectrum disorders.Frontiers in Psychology, 7, 1244. https://doi.org/10.3389/fpsyg.2016.01244.

Graham, J. M. (2006). Congeneric and (essentially) tau-equivalentestimates of score reliability: What they are and how to usethem. Educational and Psychological Measurement, 66(6),930–944. https://doi.org/10.1177/0013164406288165.

Gratz, K. L., & Roemer, L. (2004). Multidimensional assessment of emo-tion regulation and dysregulation. Journal of Psychopathology andBehavioral Assessment, 26(1), 41–54. https://doi.org/10.1023/B:JOBA.0000007455.08539.94.

Gratz, K. L., Rosenthal, M. Z., Tull, M. T., Lejuez, C. W., & Gunderson,J. G. (2006). An experimental investigation of emotion

dysregulation in borderline personality disorder. Journal ofAbnormal Psychology, 115(4), 850–855. https://doi.org/10.1037/0021-843X.115.4.850.

Gross, J. J. (2015). Emotion regulation: Current status and future pros-pects. Psychological Inquiry, 26(1), 1–26. https://doi.org/10.1080/1047840X.2014.940781.

Gross, J. J., & John, O. P. (2003). Individual differences in two emotionregulation processes: Implications for affect, relationships, and well-being. Journal of Personality and Social Psychology, 85(2), 348–362. https://doi.org/10.1037/0022-3514.85.2.348.

Hallion, L. S., Steinman, S. A., Tolin, D. F., & Diefenbach, G. J. (2018).Psychometric properties of the difficulties in emotion regulationscale (DERS) and its short forms in adults with emotional disorders.Frontiers in Psychology, 9, 539. https://doi.org/10.3389/fpsyg.2018.00539.

Haynos, A. F., Roberto, C. A., & Attia, E. (2015). Examining the associ-ations between emotion regulation difficulties, anxiety, and eatingdisorder severity among inpatients with anorexia nervosa.Comprehensive Psychiatry, 60, 93–98. https://doi.org/10.1016/j.comppsych.2015.03.004.

Hu, L. T., & Bentler, P. M. (1999). Cutoff criteria for fit indexes incovariance structure analysis: Conventional criteria versus new al-ternatives. Structural Equation Modeling, 6(1), 1–55. https://doi.org/10.1080/10705519909540118.

Kökönyei, G., Urbán, R., Reinhardt, M., Anna, J., & Demetrovics, Z.(2014). The difficulties in emotion regulation scale: Factor structurein chronic pain patients. Journal of Clinical Psychology, 70(6), 589–600. https://doi.org/10.1002/jclp.22036.

Lavender, J. M., Wonderlich, S. A., Engel, S. G., Gordon, K. H., Kaye,W. H., & Mitchell, J. E. (2015). Dimensions of emotion dysregula-tion in anorexia nervosa and bulimia nervosa: A conceptual reviewof the empirical literature.Clinical Psychology Review, 40, 111–122.https://doi.org/10.1016/j.cpr.2015.05.010.

Lee, D. J., Witte, T. K., Bardeen, J. R., Davis, M. T., & Weathers, F. W.(2016). A factor analytic evaluation of the difficulties in emotionregulation scale. Journal of Clinical Psychology, 72(9), 933–946.https://doi.org/10.1002/jclp.22297.

Lindvall Dahlgren, C., &Wisting, L. (2016). Transitioning fromDSM-IVto DSM-5: A systematic review of eating disorder prevalence as-sessment. International Journal of Eating Disorders, 49(11), 975–997. https://doi.org/10.1002/eat.22596.

Luce, K. H., & Crowther, J. H. (1999). The reliability of the eating dis-order examination—Self-report questionnaire version (EDE-Q).International Journal of Eating Disorders, 25(3), 349–351. https://doi.org/10.1002/(SICI)1098-108X(199904)25:3<349::AID-EAT15>3.0.CO;2-M.

Mallorquí-Bagué, N., Vintró-Alcaraz, C., Sánchez, I., Riesco, N., Agüera,Z., Granero, R., … Fernández-Aranda, F. (2018). Emotion regula-tion as a transdiagnostic feature among eating disorders: Cross-sectional and longitudinal approach. European Eating DisordersReview, 26(1), 53–61. https://doi.org/10.1002/erv.2570.

Meredith, W. (1993). Measurement invariance, factor analysis and facto-rial invariance. Psychometrika, 58(4), 525–543. https://doi.org/10.1007/BF02294825.

Miguel, F. K., Giromini, L., Colombarolli, M. S., Zuanazzi, A. C., &Zennaro, A. (2017). A Brazilian investigation of the 36- and 16-item difficulties in emotion regulation scales. Journal of ClinicalPsychology, 73(9), 1146–1159. https://doi.org/10.1002/jclp.22404.

Mond, J. M., Hay, P. J., Rodgers, B., Owen, C., & Beumont, P. J. V.(2004). Validity of the eating disorder examination questionnaire(EDE-Q) in screening for eating disorders in community samples.Behaviour Research and Therapy, 42(5), 551–567. https://doi.org/10.1016/S0005-7967(03)00161-X.

Monell, E., Högdahl, L.,Mantilla, E. F., & Birgegård, A. (2015). Emotiondysregulation, self-image and eating disorder symptoms in

124 J Psychopathol Behav Assess (2020) 42:111–126

university women. Journal of Eating Disorders, 3, 44. https://doi.org/10.1186/s40337-015-0083-x.

Monell, E., Clinton, D., & Birgegård, A. (2018). Emotion dysregulationand eating disorders-associations with diagnostic presentation andkey symptoms. International Journal of Eating Disorders, 51, 921–930. https://doi.org/10.1002/eat.22925.

Moskowitz, L., & Weiselberg, E. (2017). Anorexia Nervosa/AtypicalAnorexia Nervosa. Current Problems in Pediatric and AdolescentHealth Care, 47(4), 70–84. https://doi.org/10.1016/j.cppeds.2017.02.003.

Murray, A. L., & Johnson, W. (2013). The limitations of model fit incomparing the bi-factor versus higher-order models of human cog-nitive ability structure. Intelligence, 41(5), 407–422. https://doi.org/10.1016/j.intell.2013.06.004.

Muthén, B. O. (2015). Mplus user’s guide (Seventh ed.). Los Angeles:Muthen & Muthen.

Neumann, A., van Lier, P., Gratz, K. L., & Koot, H. (2010).Multidimensional assessment of emotion regulation difficulties inadolescents using the difficulties in emotion regulation scale.Assessment, 17(1), 138–149.

Osborne, T. L., Michonski, J., Sayrs, J., Welch, S. S., & Anderson, L. K.(2017). Factor structure of the difficulties in emotion regulationscale (DERS) in adult outpatients receiving dialectical behavior ther-apy (DBT). Journal of Psychopathology and BehavioralAssessment, 39(2), 355–371. https://doi.org/10.1007/s10862-017-9586-x.

Perez, J., Venta, A., Garnaat, S., & Sharp, C. (2012). The difficulties inemotion regulation scale: Factor structure and association withnonsuicidal self-injury in adolescent inpatients. Journal ofPsychopathology and Behavioral Assessment, 34(3), 393–404.https://doi.org/10.1007/s10862-012-9292-7.

Pisetsky, E. M., Thornton, L. M., Lichtenstein, P., Pedersen, N. L., &Bulik, C. M. (2013). Suicide attempts in women with eating disor-ders. Journal of Abnormal Psychology, 122(4), 1042–1056. https://doi.org/10.1037/a0034902.

Racine, S. E., & Wildes, J. E. (2015). Dynamic longitudinal relationsbetween emotion regulation difficulties and anorexia nervosa symp-toms over the year following intensive treatment. Journal ofConsulting and Clinical Psychology, 83(4), 785–795. https://doi.org/10.1037/ccp0000011.

Raykov, T. (1997). Scale reliability, Cronbach’s coefficient alpha, andviolations of essential tau-equivalence with fixed congeneric com-ponents.Multivariate Behavioral Research, 32(4), 329–353. https://doi.org/10.1207/s15327906mbr3204_2.

Reise, S. P. (2012). The rediscovery of Bifactor measurement models.Multivariate Behavioral Research, 47(5), 667–696. https://doi.org/10.1080/00273171.2012.715555.

Reise, S. P., Bonifay, W. E., & Haviland, M. G. (2013). Scoring andmodeling psychological measures in the presence of multidimen-sionality. Journal of Personality Assessment, 95(2), 129–140.https://doi.org/10.1080/00223891.2012.725437.

Rieffe, C., Terwogt, M. M., Petrides, K. V., Cowan, R., Miers, A. C., &Tolland, A. (2007). Psychometric properties of the emotion aware-ness questionnaire for children. Personality and IndividualDifferences, 43, 95–105.

Ritschel, L. A., Tone, E. B., Schoemann, A. M., & Lim, N. E. (2015).Psychometric properties of the difficulties in emotion regulationscale across demographic groups. Psychological Assessment,27(3), 944–954. https://doi.org/10.1037/pas0000099.

Rodriguez, A., Reise, S. P., &Haviland,M.G. (2016a). Applying bifactorstatistical indices in the evaluation of psychological measures.Journal of Personality Assessment, 98(3), 223–237. https://doi.org/10.1080/00223891.2015.1089249.

Rodriguez, A., Reise, S. P., & Haviland, M. G. (2016b). Evaluatingbifactor models: Calculating and interpreting statistical indices.

Psychological Methods, 21(2), 137–150. https://doi.org/10.1037/met0000045.

Sarıtaş-Atalar, D., Gençöz, T., & Özen, A. (2015). Confirmatory factoranalyses of the difficulties in emotion regulation scale (DERS) in aTurkish adolescent sample. European Journal of PsychologicalAssessment, 31(1), 12–19. https://doi.org/10.1027/1015-5759/a000199.

Satorra, A., & Bentler, P. M. (2001). A scaled difference chi-square teststatistic for moment structure analysis. Psychometrika, 66(4), 507–514. https://doi.org/10.1007/BF02296192.

Sawyer, S. M., Whitelaw, M., Le Grange, D., Yeo, M., & Hughes, E. K.(2016). Physical and psychological Mmorbidity in adolescents withatypical anorexia nervosa. Pediatrics, 137(4), e20154080. https://doi.org/10.1542/peds.2015-4080.

Segal, A., & Golan, M. (2016). Differences in emotion regulation alongthe eating disorder spectrum: Cross sectional study in adolescentsout patient care. Journal of Psychology & Clinical Psychiatry, 6(1),00314. https://doi.org/10.15406/jpcpy.2016.06.00314.

Shahabi, M., Hasani, J., & Bjureberg, J. (2018). Psychometric propertiesof the brief Persian version of the difficulties in emotion regulationscale (the DERS-16). Assessment for Effective Intervention. https://doi.org/10.1177/1534508418800210.

Sloan, E., Hall, K., Moulding, R., Bryce, S., Mildred, H., & Staiger, P. K.(2017). Emotion regulation as a transdiagnostic treatment constructacross anxiety, depression, substance, eating and borderline person-ality disorders: A systematic review. Clinical Psychology Review,57, 141–163. https://doi.org/10.1016/j.cpr.2017.09.002.

Svaldi, J., Griepenstroh, J., Tuschen-Caffier, B., & Ehring, T. (2012).Emotion regulation deficits in eating disorders: A marker of eatingpathology or general psychopathology? Psychiatry Research,197(1–2), 103–111. https://doi.org/10.1016/j.psychres.2011.11.009.

Tull, M. T., & Roemer, L. (2007). Emotion regulation difficulties associ-ated with the experience of uncued panic attacks: Evidence of expe-riential avoidance, emotional nonacceptance, and decreased emo-tional clarity. Behavior Therapy, 38(4), 378–391. https://doi.org/10.1016/j.beth.2006.10.006.

Velotti, P., Garofalo, C., Petrocchi, C., Cavallo, F., Popolo, R., &Dimaggio, G. (2016). Alexithymia , emotion dysregulation , impul-sivity and aggression : A multiple mediation model. PsychiatryResearch, 237, 296–303. https://doi.org/10.1016/j.psychres.2016.01.025.

Venta, A., Hart, J., & Sharp, C. (2012). The relation between experientialavoidance , alexithymia and emotion regulation in inpatient adoles-cents. Clinical Child Psychology and Psychiatry, 18(3), 389–410.https://doi.org/10.1177/1359104512455815.

Vieira, A. I., Ramalho, S., Brandão, I., Saraiva, J., & Gonçalves, S.(2016). Adversity, emotion regulation, and non-suicidal self-injuryin eating disorders. Eating Disorders, 24(5), 440–452.

Villalta, L., Smith, P., Hickin, N., & Stringaris, A. (2018). Emotion reg-ulation difficulties in traumatized youth : A meta-analysis and con-ceptual review. European Child & Adolescent Psychiatry, 27, 527–544.

Visted, E., Vøllestad, J., Nielsen, M. B., & Schanche, E. (2018). Emotionregulation in current and remitted depression : A systematic reviewand meta-analysis. Frontiers in Psychology, 9(756). https://doi.org/10.3389/fpsyg.2018.00756.

Wang, J., & Wang, X. (2012). Structural equation modeling:Applications using Mplus. Chichester: John Wiley & Sons.

Westerlund, M., & Santtila, P. (2018). A Finnish adaptation of the emo-tion regulation questionnaire (ERQ) and the difficulties in emotionregulation scale (DERS-16). Nordic Psychology, 70(4), 304–323.https://doi.org/10.1080/19012276.2018.1443279.

Wolz, I., Agüera, Z., Granero, R., Jiménez-Murcia, S., Gratz, K. L.,Menchón, J. M., & Fernández-Aranda, F. (2015). Emotion regula-tion in disordered eating: Psychometric properties of the difficultiesin emotion regulation scale among spanish adults and its

J Psychopathol Behav Assess (2020) 42:111–126 125

interrelations with personality and clinical severity. Frontiers inPsychology, 6, 907. https://doi.org/10.3389/fpsyg.2015.00907.

Yiugit, I., &GuzeyYiugit, M. (2017). Psychometric properties of Turkishversion of difficulties in emotion rRgulation scale-brief form(DERS-16). Current Psychology, 7, 1–9. https://doi.org/10.1007/s12144-017-9712-7.

Zinbarg, R. E., Revelle, W., Yovel, I., & Li, W. (2005). Cronbach’s, αRevelle’s β and McDonald’s ωH: Their relations with each other

and two alternative conceptualizations of reliability. Psychometrika,70(1), 123–133. https://doi.org/10.1007/s11336-003-0974-7.

Publisher’s Note Springer Nature remains neutral with regard to juris-dictional claims in published maps and institutional affiliations.

126 J Psychopathol Behav Assess (2020) 42:111–126