Journal of Affective Disorders 131 (2011) 412–416

Contents lists available at ScienceDirect

Journal of Affective Disorders

j ourna l homepage: www.e lsev ie r.com/ locate / j ad

Brief report

Using the distress thermometer and hospital anxiety and depression scale toscreen for psychosocial morbidity in patients diagnosedwith colorectal cancer

Deepa Patel a,⁎, Louise Sharpe a, Belinda Thewes a, Melanie L. Bell a,c, Stephen Clarke b

a School of Psychology, University of Sydney, NSW, Australiab Concord Clinical School, University of Sydney, NSW, Australiac Psycho-Oncology Co-Operative Research Group, University of Sydney, Australia

a r t i c l e i n f o

⁎ Corresponding author. School of Psychology (A18)NSW 2006, Australia. Tel.: +61 293514558; fax: +61

E-mail addresses: louises@psych.usyd.edu.au, louise.(L. Sharpe).

0165-0327/$ – see front matter © 2010 Elsevier B.V.doi:10.1016/j.jad.2010.11.014

a b s t r a c t

Article history:Received 19 July 2010Received in revised form 11 November 2010Accepted 11 November 2010Available online 3 December 2010

Background: The Distress Thermometer (DT) and Hospital Anxiety and Depression Scale(HADS) are commonly used within oncology settings. However there is a paucity of researchcomparing these measures to gold standard structured clinical interviews assessing for clinicaldisorders. The aim of this study is to establish the sensitivity, specificity and optimal cut-offscores on these measures when compared to a clinical interview.Method: Ninety-nine patients with colorectal cancer completed the DT and HADS and apsychologist-administered gold standard structured clinical interview (the CompositeInternational Diagnostic Interview-CIDI). Receiver Operator Characteristic analyses (ROC)were conducted to establish the optimal cut-off score on the DT and HADS to identify clinicaldisorders based on the CIDI.Results: Seventeen patientsmet criteria for a clinical disorder in the sample. A cut-off score of 4 onthe DT indicated acceptable sensitivity (60%) and specificity (67%) to detect a current clinicaldisorder, while the optimal cut-off for the HADSwas 10 (sensitivity=73%, specificity=72%). Thearea under the ROC valueswere 0.66 for theDT (95% CI: 0.51, 0.82) and 0.78 for theHADS (95%CI:0.67, 0.90). The difference in AUC between the two measures was not statistically significant.Limitations: The limitations to the design and methodology of the study are discussed.Conclusion: The single item DT performed fairly, however not as well as the longer HADS inidentifying clinical disorders amongst oncology patients, particularly anxiety disorders.

© 2010 Elsevier B.V. All rights reserved.

Keywords:CancerDepressionAnxietyScreeningDistress thermometerPsychological disorder

1. Introduction

The National Comprehensive Cancer Network (NCCN) hasrecommended that oncology patients be routinely assessed fordistress (NCCN, 2003, 2007). Short screening measures havegained substantial support, in particular the single itemmeasureof distress, the Distress Thermometer (DT) (Mitchell, 2007).

There is a growing literature investigating levels of distressin cancer patients using the DT (e.g. Dabrowski et al., 2007;Hegel et al., 2006; Trask et al., 2002; Graves et al., 2007) and

, University of Sydney293517328.sharpe@sydney.edu.au

All rights reserved.

,

studies assessing its validity in predicting clinically significantlevels of psychopathology (Roth et al., 1998, Trask et al., 2002,Gessler et al., 2008, Tuinman et al., 2008). Despite somemixedresults, the majority of studies favour a cut-off score of 4-5 ashaving optimal sensitivity to detect clinically significant levelsof psychopathology (Dolbeault et al., 2008,Hoffmanet al., 2004,Jacobsen et al., 2005, Gil et al., 2005, Ransom et al., 2006,Tuinman et al., 2008, Gessler et al., 2008). However, there areonly four studies that have investigated the validity of the DTagainst a diagnostic interview (Roth et al., 1998; Akizuki et al.,2003; Grassi et al., 2009; Thekkumpurath et al., 2009).

As Table 1 indicates, sensitivity has been found to varybetween 77% (Thekkumpurath et al., 2009) and 84% (Akizukiet al., 2003), and specificity between 61% (Thekkumpurath et al.,2009) and 84% (Akizuki et al., 2003). Two recent reviews have

413D. Patel et al. / Journal of Affective Disorders 131 (2011) 412–416

addressed the issue of screening for disorders in oncologysettings. Mitchell (2010a) examined the relationship of HADStotal and its sub-scales in predicting depression, anxiety and anydiagnosis. Theirmeta-analysis concluded that therewasevidenceto support the HADS as a screening measure. However, a reviewof all available brief screening instruments, also concluded thatthere was no evidence that brief screeners, including the DT,performed less well than the HADS (Mitchell, 2010b). Nonethe-less, the review clearly highlighted the dearth of available studiesand concluded that data on their validity in clinical settings is“currently incomplete” (Mitchell, 2010b, p. 487).

Further, with the exception of Roth et al. (1998), whosemethodology did not allow for calculation of sensitivity andspecificity, all studies that have compared the DT to clinicalinterview have used heterogeneous oncology patients overdifferent time points in the illness. Therefore, how the DTfunctions near diagnosis where one might expect higher levelsof transient distress and when psychosocial screening is mostlikely, remains unknown. The aim of this study is to investigateand compare the ability of the DT and the HADS to identifycolorectal cancer patients with clinical disorders in the periodshortly after diagnosis and initial surgery.

2. Method

2.1. Participants

BetweenMarch 2005 and September 2006, participantswererecruited from 5 public (one rural) and 2 private hospitals.Participants were eligible if they were over 18 years and had aprimary diagnosis of stage A toD colorectal cancer. One-hundredand thirty-seven patients were approached and 114 agreed toparticipate (83%). Fifteen participants failed to return baselineassessments. Therefore 99 patients diagnosed with colorectalcancer participated in the study (recruitment rate=72%).Ninety-seven participants completed the clinical interview and95 participants completed the questionnaire measures.

2.2. Measurements

2.2.1. Demographic and medical variablesDemographic informationon age, gender,marital status, level

of education and employment status was collected. Medicalvariables collected included information on the disease stage,treatment status (chemotherapy, radiotherapy, no currenttreatment, or neoadjuvant) and date and type of surgery.

Table 1Outcomes of the four studies that investigated the validity of the DT in comparison

Study N Sample Sensitivity

(Roth et al., 1998) 93 Prostate n/a

(Akizuki et al., 2003) 275 Mixed 84%

(Grassi et al., 2009) 109 Mixed 79.5%(Thekkumpurath et al., 2009) 150 Mixed palliative care 77–78%

2.2.2. Clinical interviewThe Composite International Diagnostic Interview (CIDI) is a

gold standard structuredclinical interviewthatusesDSM-IVand/or ICD-10diagnostic criteria todiagnosemental disorders (WorldHealth Organisation, 1997, Andrews and Peters, 1998). Theinter-rater reliability is excellent and test-retest reliabilityand thevalidity are good (Wittchen et al., 1991, Andrews and Peters,1998). For the current study, the depressive disorders (excludingadjustment disorders) and anxiety disorders (excluding PTSD)modules were used. It was determined whether patients metcriteria for depressive, anxiety or any clinical disorder.

2.2.3. Self report measures1) The Distress Thermometer (DT) is a single item visual

analogue scale which is shaped like a thermometer, with anumerical scale between 0 to 10, with 0 indicating nodistress and 10 indicating extreme distress (NCCN, 2007).Patients are asked to circle the number that best describestheir level of distress over the last week.

2) Hospital anxiety and depression scale (HADS) is a measureof anxiety anddepression for patientswho are physically ill(Zigmond and Snaith, 1983). It contains 14 items, half thatassess anxiety and the remainder depression. The HADShas been used widely in oncology and has good reliabilityand validity (Moorey et al., 1991, Ibbotson et al., 1994,Walker et al., 2007).

2.3. Procedure

The research was approved by the ethics committees of allparticipating institutions. Participants were identified by colo-rectal care coordinators, but recruited by a member of theresearch team (DP). Those who agreed to take part were given aquestionnaire package including the DT and HADS to complete.DP contacted participants by telephone within one week ofrecruitment and administered the clinical interview (CIDI) overthe telephone. Because of the different treatment possibilities,patients were recruited either after surgery during theiradmission (n=60), at their first oncology post-surgery appoint-ment (n=29) or prior to surgery at the first consultationappointment before undergoing neo-adjuvant treatment(n=10). Patients were recruited within 9 weeks of receivingtheir diagnosis.

2.4. Analysis

For the medical and demographic data, descriptive analyseswere calculated. We performed Receiver Operator Characteristic

to interview.

Specificity Limitations/comments

n/a Only those who met criteria of being in the clinical range oneither the HADS or DT were assessed by interview. Of 26interviewed, 7 were missed, 5 refused, 8 met criteria for apsychiatric disorder and 9 did not.

61% Unstructured diagnostic interviews, based on DSM-IV criteriafor major depressive disorder and adjustment disorder

75.4% HADS performed better than DT.59–62% DT was compared to BSI and GHQ and. DT was comparable

to the other measures.

414 D. Patel et al. / Journal of Affective Disorders 131 (2011) 412–416

(ROC) analyses to find an optimal cut-off score for the DT andHADS against any disorder on the CIDI. Positive and negativepredictive values and likelihood ratios (sensitivity/(1-specifici-ty)) were calculated for all possible cut-off scores. Sensitivity,specificity, negative and positive predictive power are reportedfor the optimal cut-off points for DT and HADS against anydisorder, anxiety disorder and depressive disorder separately. Allstatistical analyses used SAS v9.2 (SAS, 2008).

3. Results

3.1. Descriptive statistics

The average age of participants was 65.7 years, nearly twothirdweremales (n=61) and themajority had 12 or less yearsof education and or trade/technical qualifications (n=80)(Table 2). Approximately half were in a relationship (n=56),and retired (n=46). Patients were almost equally distributedin being diagnosed with early stage (A (n=16) or B (n=30))and more advanced stage cancer (C (n=33) or D (n=14)).Forty-three patients had no adjuvant therapy post-surgery, 43had chemo-therapy, 3 had radiation and 10 had neoadjuvanttherapy.

Seventeen patients were diagnosed with a current mood oranxiety disorder, 11 met criteria for a depressive disorder(major depressive disorder (n=9) and dysthymic disorder(n=2)) and 7 with a primary anxiety disorder (Social anxiety(n=1), generalized anxiety disorder (GAD) (n=2) and panicdisorder (n=1) and specific phobia (n=3)), one patient had asecondary diagnosis of GAD. The means on the DT (2.79;SD=2.74) and HADS (8.27; SD=6.20) were within thenon-clinical range.

3.2. ROC analyses

A cut-off score of 4 on the DT indicated acceptablesensitivity (60%; 95% CI: 50%, 70%) and specificity (67%; 95%CI: 57%, 76%) to detect a current clinical disorder. A cut-off of 10on theHADS also performed acceptably (sensitivity=73%; 95%CI: 63%, 81%, specificity=72%; 95% CI: 62%, 80%). Thecomparison between the 2 measures is outlined in the ROCcurves in Fig. 1. The area under the curve (AUC) values were0.66 for the DT (95% CI: 0.51, 0.82) and 0.78 for the HADS (95%CI: 0.67, 0.90),whichwasnot statistically significantly different,p=0.09.

As can be seen from Table 2, the sensitivity (88%) andspecificity (69%) for the HADS remained strong in theprediction of depression and acceptable for anxiety (sensitiv-ity=67%; specificity=67%). The sensitivity dropped to 50% for

Table 2Sensitivity, specificity, positive and negative predictive values for the HADS and DTdisorder or a depressive disorder according to the CIDI.

Disorder Screener Sensitivity Specificity

Any disorder DT 0.60 0.67HADS 0.73 0.72

Depression DT 0.63 0.65HADS 0.88 0.69

Anxiety DT 0.5 0.64HADS 0.67 0.67

the DT in predicting anxiety disorders, with a specificity of 64%.However, in the prediction of depression, psychometricsremained acceptable (sensitivity=63%; specificity=65%).

4. Discussion

The aim of this studywas to investigate the ability of the DTand HADS to identify patients with clinical disorders afterdiagnosis. Results indicated that at a cut-off score of 4, the DTperformed fairly in identifying patients with clinical disorders;while the HADS at a cut-off of 10, performed well. Thedifference between the area under the ROC curves of the DTand HADS was not statistically significant.

The results of this study are consistent with the literature.That is, both the HADS and DT performed comparably in theprediction of any clinical disorder (Mitchell, 2010a). Thissuggests that if the primary aim of a screener is to identifyindividuals with an anxiety or depressive disorder, that eitherthe HADS or DT would suffice. However, the low levels ofpositive predictive power indicate that both measures areuseful for screening, but should not substitute for clinicalassessment (Mitchell, 2010a, 2010b).

We also calculated specificity and sensitivity for theprediction of anxiety and depressive disorders. Due to the lowbase rate of disorder, these have considerable margin for error.BothDT andHADs held upwell for the prediction of depression,although for anxiety, the sensitivity of the DT fell belowacceptable levels. Despite concerns about the low base rate, wenote that there are currently only two studies that examine theHADS against a diagnosis of anxiety (Mitchell, 2010a) and nonefor the DT. Hence, it seems important to include these results,that suggest that caution is warranted in using the DT if themain aimof screening is to identify anxiety disorders. However,both appear more suited to screening for depression.

The low rate of clinical disorder in this study is a limitation.Adjustment disorders were purposefully not included due tothe time point at which patients were recruited. Distress iscommon after diagnosis, and including adjustment disordersduring this time would have artificially inflated the prevalenceof disorder in the sample. Nonetheless, adjustment disorderswere the most common diagnoses in previous studies andexcluding them resulted in a lower base rate of disorders in thissample (Akizuki et al., 2003). Another limitation is the samplesize, a larger sample would have increased power, and possiblywould have found a statistically significant difference betweenthe area under the ROC curves HADS and the DT. However, thesample is similar in size to other samples (e.g. Roth et al., 1998,Grassi et al., 2009). Further, the use of telephone-administeredCIDI could have reduced the reliability of the interviews.However, research shows good concordance between tele-

in predicting the presence of an anxiety or depressive disorder, and anxiety

Positive predictive value Negative predictive value

0.26 0.90.33 0.930.14 0.940.23 0.980.09 0.960.13 0.96

0.00

0.25

0.50

Sen

sitiv

ity0.

751.

00

0.00 0.25 0.50 0.75 1.001-Specificity

HADS area: 0.78 DT area: 0.66

Reference

ROC Curves

Fig. 1. ROC analysis of performance of DT and HADS against current disorderfrom the CIDI.

415D. Patel et al. / Journal of Affective Disorders 131 (2011) 412–416

phone and face-to-face interviews, using the CIDI, both inpsychiatric (Allen et al., 2003) andmedical out-patient settings(Rohde et al., 1997).

These methodological issues, notwithstanding, results of thecurrent study indicated that the performance of the 14-itemHADS is likely to be better than the single item DT. Sensitivity,specificity, and ROC AUC were higher, though not statisticallysignificant, in predicting clinical disorders, and particularly foranxiety disorders. Both measures had low positive predictivepower underscoring their role as screening instruments, and notdiagnostic tools. However, within clinical practice brevity of ameasure is important, hence, if the efficacy of shorter screeningtool (i.e. DT) is similar to longer measures (i.e. HADS), then itfollows that in busy oncology clinics it is more useful to use thesingle item measure. However, at the optimal cut-off score of 4,the sensitivity of the DT was 60%, which though fair, is notsufficientlyhigh tobeclinically stringent. Importantly, thismeansthat a proportion of patients who have a disorder are notidentified by the DT, particularly with anxiety disorders. So,although it may be viable to use the DT or HADS as a first stagescreen if theprimary target is depressionor thepresenceof eitherdisorder, additional methods of assessing for clinical disordersare required. Hence, the use of these tools should be supple-mented by attempts to improve liaison between cancer andpsycho-oncology services to provide formal referral and assess-ment for those patients at risk.

Role of Funding SourceThis research was supported by the University of Sydney Cancer Research

Fund, and Deepa Patel was supported by an Australian Rotary Health ResearchFund - PhD Scholarship. Belinda Thewes was supported by a National BreastCancer Foundation Post Doctoral Fellowship. None of these funding bodies hadany further role in study design; in the collection, analysis and interpretation ofdata; in the writing of the report; and in the decision to submit the paper forpublication.

Conflict of InterestNone of the authors have any conflict of interest to declare.

Acknowledgements

DP would like to thank Professor Phyllis Butow whoprovided her with guidance and mentorship during thewrite-up of this manuscript. We would like to thank thecolorectal care coordinators at the participating hospital siteswho assisted DP with recruitment of participants. We wouldalso like to thank all participants who generously gave up theirtime to take part in this study.

References

Akizuki, N., Akechi, T., Nakanishi, T., Yoshikawa, E., Okamura, M., Nakano, T.,Murakami, Y., Uchitomi, Y., 2003. Development of a brief screeninginterview for adjustment disorders and major depression in patients withcancer. Cancer 97, 2605–2613.

Allen, K., Cull, A., Sharpe, M., 2003. Diagnosing major depression in medicaloutpatients: acceptability of telephone interviews. Journal of PsychosomaticResearch 55, 385–387.

Andrews, G., Peters, L., 1998. The psychometric properties of the CompositeInternational Diagnostic Interview. Social Psychiatry and PsychiatricEpidemiology 33, 80–88.

Dabrowski, M., Boucher, K., Ward, J.H., Lovell, M.M., Sandre, A., Bloch, J.,Carlquist, L., Porter, M., Norman, L., Buys, S.S., 2007. Clinical experiencewith the NCCN distress thermometer in breast cancer patients. Journal ofthe National Comprehensive Cancer Network 5, 104–111.

Dolbeault, S., Bredart, A., Mignot, V., Hardy, P., Gauvain-Piquard, A.,Mandereau, L., Asselain, B., Medioni, J., 2008. Screening for psychologicaldistress in two French cancer centers: feasibility and performance of theadapted distress thermometer. Palliative & Supportive Care 6, 107–117.

Gessler, S., Low, J., Daniells, E., Williams, R., Brough, V., Tookman, A., Jones, L.,2008. Screening for distress in cancer patients: is the distress thermometera valid measure in the UK and does it measure change over time? Aprospective validation study. Psycho-Oncology 17, 538–547.

Gil, F., Grassi, L., Travado, L., Tomamichel, M., Gonzalez, J.R., SOUTHERNEUROPEAN PSYCHO-ONCOLOGY STUDY, G, 2005. Use of distress anddepression thermometers to measure psychosocial morbidity amongsouthern European cancer patients. SupportiveCare inCancer 13, 600–606.

Grassi, L., Sabato, S., Rossi, E., Marmai, L., Biancosino, B., 2009. Affectivesyndromes and their screening in cancer patients with early and stabledisease: Italian ICD-10 data and performance of the Distress Thermometerfrom the Southern European Psycho-Oncology Study (SEPOS). Journal ofAffective Disorders 114, 193–199.

Graves, K.D., Arnold, S.M., Love, C.L., Kirsh, K.L., Moore, P.G., Passik, S.D., 2007.Distress screening in a multidisciplinary lung cancer clinic: prevalenceand predictors of clinically significant distress. Lung Cancer 55, 215–224.

Hegel, M.T., Moore, C.P., Collins, E.D., Kearing, S., Gillock, K.L., Riggs, R.L., Clay,K.F., Ahles, T.A., 2006. Distress, psychiatric syndromes, and impairmentof function in women with newly diagnosed breast cancer. Cancer 107,2924–2931.

Hoffman, B.M., Zevon, M.A., D'Arrigo, M.C., Cecchini, T.B., 2004. Screening fordistress in cancer patients: the NCCN rapid-screening measure. Psycho-Oncology 13, 792–799 [erratum appears in Psychooncology. 2004 Nov;13(11):831].

Ibbotson, T., Maguire, P., Selby, P., Priestman, T., Wallace, L., 1994. Screeningfor anxiety and depression in cancer patients: the effects of disease andtreatment. European Journal of Cancer 30(A, 37–40.

Jacobsen, P.B., Donovan, K.A., Trask, P.C., Fleishman, S.B., Zabora, J., Baker, F.,Holland, J.C., 2005. Screening for psychologic distress in ambulatorycancer patients. Cancer 103, 1494–1502.

Mitchell, A.J., 2007. Pooled results from 38 analyses of the accuracy of distressthermometer and other ultra-short methods of detecting cancer-relatedmood disorders. Journal of Clinical Oncology 25, 4670–4681.

Mitchell, A.J., 2010a. Diagnostic validity of the Hospital Anxiety andDepression Scales in cancer and palliative settings: a meta-analysis.Journal of Affective Disorders 216, 335–348.

Mitchell, A.J., 2010b. Short screening tools for cancer-relateddistress: a reviewanddiagnostic validity meta-analysis. Journal of the National ComprehensiveCancer Network 8, 487–494.

Moorey, S., Greer, S., Watson, M., Gorman, C., Rowden, L., Tunmore, R.,Robertson, B., Judith, B., 1991. The factor structure and factor stability ofthe Hospital Anxiety and Depression Scale in Patients with Cancer. TheBritish Journal of Psychiatry 158, 255–259.

NCCN. 2003. Distress management. Clinical practice guidelines. Journal of theNational Comprehensive Cancer Network 1, 344–374.

416 D. Patel et al. / Journal of Affective Disorders 131 (2011) 412–416

NCCN. 2007. The National Comprehensive Cancer Network Distress ManagementClinical Practice Guidelines in Oncology (Version.1.2007) [Online]. 2006National Comprehensive Cancer Network, Inc. Available at: http://www.nccn.org. Available: www.nccn.org [Accessed 22 March 2007 2007].

Ransom, S., Jacobsen, P.B., Booth-Jones, M., 2006. Validation of the distressthermometer with bonemarrow transplant patients. Psycho-Oncology 15,604–612.

Rohde, P., Lewinsohn, P.M., Seeley, J.R., 1997. Comparability of telephone andface-to-face interviews in assessing axis I and II disorders. The AmericanJournal of Psychiatry 154, 1593–1598.

Roth, A.J., Kornblith, A.B., Batel-Copel, L., Peabody, E., Scher, H.I., Holland, J.C.,1998. Rapid screening for psychologic distress in men with prostatecarcinoma: a pilot study. Cancer 82, 1904–1908.

Sas, 2008. SAS v9.2 SAS 9.2 Online Documentation. SASINstitute Inc, Cary,NC.Thekkumpurath, P., Venkateswaran, C., Kumar, M., Newsham, A., Bennett, M.I.,

2009. Screening for psychological distress in palliative care: performanceoftouch screen questionnaires compared with semistructured psychiatricinterview. Journal of Pain and SymptomManagement 38, 597–605.

Trask, P.C., Paterson, A., Riba,M., Brines, B., Griffith, K., Parker, P.,Weick, J., Steele, P.,Kyro, K., Ferrara, J., 2002. Assessment of psychological distress in prospectivebonemarrow transplantpatients. BoneMarrowTransplantation29, 917–925.

Tuinman, M.A., Gazendam-Donofrio, S.M., Hoekstra-Weebers, J.E., 2008.Screening and referral for psychosocial distress in oncologic practice: useof the Distress Thermometer. Cancer 113, 870–878.

Walker, J., Postma, K., Mchugh, G.S., Rush, R., Coyle, B., Strong, V., Sharpe, M.,2007. Performance of the Hospital Anxiety and Depression Scale as ascreening tool for major depressive disorder in cancer patients. Journal ofPsychosomatic Research 63, 83–91.

Wittchen, H.U., Robbins, L.N., Cottler, L.B., Sartorius, N., Burke, J.D., Regier, D.,1991. Cross-cultural feasibility, reliability and sources of variance of thecomposite international diagnostic interview (CIDI). The British Journal ofPsychiatry 159, 645–653.

World Health Organisation, 1997. Composite International DiagnosticInterview, 2.1 ed. World Health Organisation, Geneva.

Zigmond, A., Snaith, R., 1983. The hospital anxiety and depression scale.Psychiatrica Scandinavia 67, 361.