Bias in computerized neuropsychologicalassessment of depressive disorders causedby computer attitude

Introduction

Computerized examinations are increasinglyapplied in health care. Even psychiatric patientsare confronted to computerized psychological andpsychopathological assessment (1±7) and clinicalresearch even begins dealingwith `telepsychiatry' (8).Computerization has important advantages (9): Itis less time consuming and thus potentially morecost-e�ective, it assures standardization and reli-ability (10), it enables a precise time registration,it facilitates quality monitoring and statisticalanalysis, and recent publications indicate evenan increased reporting of deviating behaviour incomputerized interviews (11±13).

Nevertheless, acceptance and validity have notyet been su�ciently investigated. From a meth-odological point of view acceptance of computer-ized examination is to be regarded as an important

motivational factor (14, 15) where limited accept-ance might endanger validity and reliability ofcomputerized assessment.In particular, the attitude towards computers

was reported to a�ect acceptance of computerizedassessment in psychiatric patients. Spinhovenet al. (16) have investigated the feasibility ofcomputerized psychological examination in psy-chiatric out-patients. Fifty-four per cent of theinitially selected patients refused the examination.Educational level, previous experience with com-puters and attitude towards computers wererelated to successful patient±computer interac-tion.Aim of the present study was to evaluate

computer attitude as well as acceptance of com-puterized assessment and their possible correlationto neuropsychological test performance in psychi-atric in-patients.

Weber B, Fritze J, Schneider B, KuÈ hner T, Maurer K. Bias incomputerized neuropsychological assessment of depressive disorderscaused by computer attitude.Acta Psychiatr Scand 2002: 105: 126±130. ã Munksgaard 2002.

Objective: Psychiatric patients are increasingly confronted tocomputerized psychological and psychopathological assessment.Patients' attitude to computers was reported to a�ect acceptance ofcomputerized assessment.Method: In 78 psychiatric in-patients neuropsychological impairmentwas examined following admission on an open ward by conventionalas well as computerized memory and attention tasks. Besidespsychopathological assessment, self ratings of computer attitude andacceptance of the computerized assessment were completed.Results: A more negative attitude to computers was found to besigni®cantly correlated to higher nervousness in patients' self report(R � 0.38, P � 0.0005) as well as to poorer results of computerizedattention tasks (R � 0.39, P � 0.0007). Particularly in patients withdepressive disorders computer attitude could be shown to explain 39%of the variance of attention performance.Conclusion: Results indicate a signi®cant e�ect of negative computerattitude on acceptance and thus reliability of computerizedexamination, resulting in a bias in computerized attention-relatedassessment in patients with depressive disorders.

B. Weber, J. Fritze, B. Schneider,T. Kühner, K. MaurerDepartment of Psychiatry and Psychotherapy I,J. W. Goethe University, Frankfurt/Main, Germany

Key words: psychometrics; neuropsychology;depression; computer; attitude to computers;acceptance

Bernhard Weber, Department of Psychiatry andPsychotherapy I, J. W. Goethe University, Heinrich-Hoffmann-Str. 10, D-60528 Frankfurt/Main, GermanyE-mail: b.weber@em.uni-frankfurt.de

Accepted for publication 4 September, 2001

Acta Psychiatr Scand 2002: 105: 126±130Printed in UK. All rights reserved

Copyright ã Munksgaard 2002

ACTA PSYCHIATRICASCANDINAVICAISSN 0001-690X

126

Material and methods

In the present study conventional and computer-ized neuropsychological assessment was performedin 78 psychiatric in-patients following admissionto an open psychiatric ward. Computer attitude,acceptance of the computerized assessment andpsychopathological symptoms were examined inaddition.Patients with organic brain diseases or patients

insu�ciently familiar with the German languagewere excluded. Less than 10% of the patientsrefused the procedure because they felt to bestrained too much with it. Less than 5% stoppedduring the course of computerized assessmentbecause of a loss of concentration or compliance.Seventy-eight patients completed the examinationwithout experiencing major problems. The age ofthese patients was m=43.9 � sd=15.2 (range=18.8±76.5) years. 47 patients were female [47.3 �15.5 (25.3±76.5) years] and 31 male [38.8 � 13.4(18.8±70.6) years]. Principal DSM IV (17) diag-noses are shown in Table 1.A computerized version of a German translation

of theGroningenComputerAttitude Scale (GCAS),developed by Bouman et al. (18), was used for theassessment of the general attitude towards comput-ers. The Lickert-type scale consists of 16 statementswith ®ve possible answers varying between `totallyagree' and `totally disagree'. Items originally wereselected from the `Attitudes Towards ComputersScale' (ATCS) (19), the `Attitudes Towards Com-puters Usage Scale' (ATCUS) (20), the `ComputerAttitude Scale' (CAS) (21) and in accordance withface validity. The minimum score of the scale is 16and the maximum 80; a score higher than 48 isconsidered to represent a positive attitude towardscomputers (18). In the sample of this study Cron-bach's a was calculated with 0.80 and split halfreliability with 0.79.During the computerized assessment patients

had to answer an `Operation and Preference Ques-tionnaire' (OPQ) asking for experiences andimpressions particularly concerning the `situation'of computerized assessment and indicating ®nalacceptance of the computerized examination. Theself developed scale consists of 13 statementsselected in accordance with face validity (7). Foreach of these statements a score is calculated withhigh values for good and low values for pooracceptance of computerized assessment. Threeadditional questions evaluate former assessmentexperience and educational level. The theoreticalminimum score is 10.5 and the maximum 39. In thisstudy sample Cronbach's a was calculated with0.80 and split half reliability with 0.89. A score

higher than 26 points indicates positive judgementson average and for that reason it is considered torepresent good acceptance and successful `patient±computer interaction'. The sum score of items 7, 10and 13 was used as `nervousness subscale'.Neuropsychological assessment was performed

by the Mini Mental State Examination [MMSE(22)], a conventional and a computerized memorytask [ADAS (23) word recognition, form A and B],a conventional [d2 (24)] and a computerizedattention task [ERTS (25)] and a visual pursuittracking task (ERTS).The psychopathological assessment included ±

depending on diagnoses ± the Hamilton Depres-sion Rating Scale [HDRS, 21 items (26, 27)], theBeck Depression Inventory [BDI (28)], the Pro®leof Mood States [POMS (29)], a Visual AnalogueMood-Scale [VAS (30)], the Brief PsychiatricRating Scale [BPRS (31)], the Scale for theAssessment of Positive Symptoms [SAPS (32)],the Scale for the Assessment of Negative Symp-toms [SANS (33)] and the Clinical Global Impres-sions subscale severity of illness [CGI (34)].Dosagesofneurolepticswere tranformed tochlor-

promazine (CPZ) equivalents and dosages of ben-zodiazepines to diazepam (DZP) equivalents (35).Statistical analysis of the relation between ordi-

nal and interval variables was performed by theSpearman rank order correlation. The Mann±Whitney U-test was used for gender related di�er-ences and the Kruskal±Wallis ANOVA for analysisof di�erences between diagnostic subgroups.Additionally, a (forward stepwise) multiple regres-sion analysis for ERTS attention scores wasperformed.

Results

The mean score of the GCAS was 57.7 � 10.6 (35±78) points in the 78 patients. A positive attitude

Table 1. Diagnostic subgroups of the 78 psychiatric in-patients examined

Main diagnoses (DSM IV) n

Mood (depressive) disorders (n � 35)Depressive episode 32Dysthymic disorder 3

Psychotic disorders (n � 28)Schizophrenia 16Schizoaffective disorder 9Brief psychotic disorder 2Substance induced psychotic disorder 1

Other disorders (n � 15)Neurotic or personality disorder 10Substance related disorder 4Post-traumatic stress disorder 1

Total 78

Computerized neuropsychological assessment

127

towards computers (GCAS score > 48) was foundin 65 patients (83.3%) whereas 13 (16.7%) showeda negative attitude. The mean score of OPQ was30.3 � 4.92 points. A good acceptance of thecomputerized examination (OPQ score > 26) wasfound in 63 patients (80.8%). No signi®cantdi�erences were found between diagnosticsubgroups with regard to GCAS and OPQ scores.

A more negative attitude towards computers(GCAS score) was found to be signi®cantlycorrelated to a poorer acceptance (OPQ: R � 0.41,P � 0.0002), higher self reported nervousness (OPQitems 7, 10, 13: R � 0.38, P � 0.0005) as well as topoorer results in the computerized attention(R � 0.39, P � 0.0007) and pursuit tracking(R � ±0.33, P � 0.003) tasks (Table 2). Thesecorrelations were found to partly di�er betweendiagnostic subgroups (Table 3).

Experimental run time system attention taskscores correlated signi®cantly to GCAS scores(R � 0.62, P � 0.0002) and education level(R � 0.42, P � 0.02) in depressive disorders, toconventional attention scores (R � 0.62,P � 0.0008) in psychotic disorders and to age inthe subgroup of other disorders (R � ±0.54,P � 0.48). In patients with depressive disorders,no signi®cant correlations were found betweencomputer attitude (GCAS) and psychopathologi-cal ratings (HDRS, BDI, POMS, VAS). However,a signi®cant correlation was found between BDIand ERTS attention scores (R � ±0.36, P � 0.48).

A stepwise forward multiple regression analysiswith ERTS attention scores as dependent variableand age, gender, diagnostic subgroup, educationlevel, CPZ and DZP equivalent doses, conven-tionally measured attention and memory scores,MMSE, BPRS, GCAS, and OPQ scores as inde-pendent variables showed 5% of the ERTS

attention score variance to be explained bycomputer attitude (GCAS scores) in the completesample and 39% in the subgroup of depressivepatients.

Discussion

The results indicate a signi®cant e�ect of negativecomputer attitude on acceptance of computerizedexamination and nervousness during computerizedassessment. This corresponds to the ®ndings ofSpinhoven et al. (16) who saw a signi®cant cor-relation between computer attitude and `relaxationduring computerized assessment'. Less positivecomputer attitude seems to result in biased ®ndingsin computerized assessment of attention, and

Table 2. Crosscorrelations between neuropsychological measurements, GCAS and OPQ scores and possible covariables for all patients (n = 78)

CPZequivalent

DZPequivalent

Memory(convent.)a

Memory(PC)a MMSE

Attention(convent.)

Trackingtask (PC)a

Reactiontime (PC)a

Attention(PC)

GCAStotal score

Nervousn.(OPQ)a

OPQtotal score

Age )0.27* )0.01 )0.22 )0.17 )0.07 )0.25* )0.57**** )0.35** )0.24* )0.27* 0.04 )0.11CPZ equivalent 0.10 )0.04 0.07 )0.25 )0.36** )0.08 0.00 )0.06 )0.01 )0.07 )0.02DZP equivalent )0.01 )0.02 0.05 )0.10 0.03 )0.11 0.02 0.04 0.06 )0.08Memory (convent.)a 0.42*** 0.04 0.14 0.05 0.12 0.15 0.22 )0.23* 0.13Memory (PC)a 0.18 0.11 0.17 )0.03 0.21 0.17 0.04 0.03MMSE 0.44**** 0.26* 0.26* 0.19 0.14 )0.03 0.10Attention (convent.) 0.30** 0.33** 0.30* 0.06 0.09 0.00Tracking task (PC)a 0.33** 0.36** 0.33** 0.01 0.09Reaction time (PC)a 0.43*** 0.13 )0.03 0.00Attention (PC) 0.39*** 0.02 )0.03GCAS total score )0.38*** 0.41***Nervousness (OPQ) )0.74****

* P < 0.01, **P < 0.001, ***P < 0.0001, ****P < 0.00001.a Signs of correlations exchanged.

Table 3. Differences between diagnostic subgroups with regard to correlationsbetween computer attitude (GCAS) and acceptance of computerized assessment(OPQ), computerized and conventional neuropsychological measurements, medi-cation, age and education level

Correlations to GCAS total score

Mood disorders(n � 35)

Psychotic disorders(n � 28)

Other disorders(n � 15)

Age )0.44** )0.06 )0.05CPZ equivalent )0.27 0.18 )0.22DZP equivalent )0.03 0.38 )0.10Memory (convent.)a 0.04 0.39* 0.06Memory (PC)a 0.16 0.33 0.14MMSE 0.24 0.09 0.08Attention (convent.) 0.34* 0.19 )0.25Tracking task (PC)a 0.50** 0.09 0.11Reaction time (PC)a 0.33* )0.05 0.02Attention (PC) 0.62**** )0.01 0.52Nervousness (OPQ) )0.38* )0.29 )0.47OPQ total score 0.32 0.36 0.56*Education level 0.57*** 0.36 )0.06

*P < 0.5, **P < 0.01, ***P < 0.001, ****P < 0.0001.a Signs of correlations exchanged.

Weber et al.

128

particularly so in patients with depressive disor-ders.The promising supposition of a direct impact of

depressive psychopathology on test performancecould not be really supported: only self-ratedextent of BDI seemed to have some relevance forcomputerized attention tasks in depressed patients.Although even computer attitude was not found tobe signi®cantly correlated to depressive psycho-pathology, the bias found in test performance ofdepressive patients might be mediated by patients'susceptibility to negative dysfunctional cognitionsand anxiety.These ®ndings are limited because the study had

an explorative design and was performed in adiagnostically heterogeneous group of psychiatricin-patients using a speci®c assessment software.Computer attitude (GCAS) and acceptance ofcomputerized assessment (OPQ) could not befound to di�er signi®cantly between diagnosticsubgroups. But the number of patients in subsam-ples was very small compared with the manyremaining intervening variables involved and restrictsthe interpretation of subgroup di�erences in parti-cular.Nevertheless, results suggest a very careful inter-

pretation of results obtained by computerizedassessment of attention-related tasks, particularlyin patients with depressive disorders. Individualcomputer attitude and acceptance ± besides theobservation of behaviour ± should be systematicallymonitored as possibly confounding covariables.Computerized assessment is not simply a support

by a machine but a particular mode of informationprocessing where the computer has impacts on, e.g.individual strategies (14). There remains furtherneed for research on the complex factors whichgovern the human±computer interaction (36) inorder to optimize the patient±computer interplayand particularly to reduce anxiety provokingstimuli. Apart from psychiatric problems relatedto excessive computer use (37) several studies on`computer hassles' (38, 39) and `computerphobia'(40±43) in healthy subjects have been publishedrecently. The results of the present study indicatethat this is of importance in psychiatry, too.In this context the possibility of improved

computer attitude and acceptance simply by habi-tuation should be examined. Computer traininghas been reported to improve attitude towardscomputers in healthy adults (44). Accordingly,computerized training options such as that ofcognitive functions, computer painting or compu-ter games (45) accompanied by positive feedback(46) open new perspectives for familiarity withcomputers.

Acknowledgements

This work was supported by grant We2263/1-1 from DeutscheForschungsgemeinschaft (DGF).

References

1. FREUDENMANN RW, SPITZER M. Computer assistedquestioning as a basis for a modern quality assurance inpsychiatry. Results from pilot studies. [ComputergestuÈ tztePatientenbefragungen als Grundlage fuÈ r eine moderneQualitaÈ tssicherung in der Psychiatrie. Ergebnisse derPilotstudien]. Nervenarzt 2001;72:40±51.

2. HEDLUND JL, VIEWEG BW, CHO DW. Mental healthcomputing in the 1980s. II. Clinical applications. ComputHum Serv 1985;1:1±31.

3. NEAL LA, BUSUTTIL W, HERAPATH R, STRIKE PW.Development and validation of the computerized clinicianadministered post-traumatic stress disorder scale-1-revised.Psychol Med 1994;24:701±706.

4. PELISSOLO A, VEYSSEYRE O, LEPINE JP. Validation of acomputerized version of the temperament and characterinventory (TCI) in psychiatric inpatients. Psychiatry Res1997;72:195±199.

5. SHAKESHAFT AP, BOWMAN JA, SANSON-FISHER RW. Com-puters in community-based drug and alcohol clinical set-tings: are they acceptable to respondents? Drug AlcoholDepend 1998;50:177±180.

6. STEVENSON JF, BEATTIE MC, ALVES RR, LONGABAUGH R,AYERS T. An outcome monitoring system for psychiatricinpatient care. QRB Qual Rev Bull 1988;14:326±331.

7. WEBERB,FRITZE J, SCHNEIDERB, SIMMINGERD,MAURERK.Computerized self-assessment in psychiatric in-patients.Acceptability, feasibility and in¯uence of computer atti-tude. Acta Psychiatr Scand 1998;98:140±145.

8. YOSHINO A, SHIGEMURA J, KOBAYASHI Y et al. Tele-psychiatry: assessment of televideo psychiatric interviewreliabilibilty with present- and next-generation Internetinfrastructures. Acta Psychiatr Scand 2001;1043 :223±226.

9. WEBER B, FRITZE J, SCHNEIDER B et al. Acceptability ofcomputerized self-assessment in psychiatric inpatients.European Psychiatry 1998;13(Suppl. 4):322s.

10. CANFIELD ML. Computerized rating of the psychother-apeutic process. Psychotherapy 1991;28:304±316.

11. JARLAIS DC, PAONE D, MILLIKEN J et al. Audio-computerinterviewing to measure risk behaviour for HIV amonginjecting drug users: a quasi-randomised trial. Lancet1999;353:1657±1661.

12. TURNER CF, KU L, ROGERS SM, LINDBERG LD, PLECK

JH, SONENSTEIN FL. Adolescent sexual behavior, drug use,and violence: increased reporting with computer surveytechnology. Science 1998;280:867±873.

13. LUCAS RW, MULLIN PJ, LUNA CB, MCINROY DC. Psy-chiatrists and a computer as interrogators of patients withalcohol-related illnesses: a comparison. Br J Psychiatry1977;131:160±167.

14. JAÈ GER RS, KRIEGER W. Future prospects of computer-based assessment, exempli®ed on the basis of treatment-oriented assessment. Diagnostica 1994;40:217±243.

15. LIENERT GA, RAATZ U. Testaufbau und Testanalyse.Weinheim: Psychologie Verlags Union, 1994.

16. SPINHOVEN P, LABBE MR, ROMBOUTS R. Feasibility ofcomputerized psychological testing with psychiatric out-patients. J Clin Psychol 1993;49:440±447.

17. AMERICAN PSYCHIATRIC ASSOCIATION. Diagnostic andstatistical manual of mental disorders: DSM IV, 4th edn.Washington, DC: American Psychiatric Association, 1994.4

Computerized neuropsychological assessment

129

18. BOUMAN TK, WOLTERS FJM, WOLTERS-HOFF GH. Eenschaal om attitudes tegenover computers te meten.Nederlands Tijdschrift Voor Psychologie 1989;44:288±292.

19. ROSEN LD, SEARS DC, WEI MM. Computerphobia mea-surement. Amanual for administration and scoring of threeinstruments: Computer Anxiety Rating Scale (CARS),Attitudes Towards Computer Scale (ATCS), ComputerThoughts Survey (CTS).5 California State University:Dominiquez Hills, Carson, CA, 1987.

20. POPOVICH PM, HYDE KR, ZAKRAJSEK T, BLUMER C. Thedevelopment of the attitudes toward computer usage scale.Educational Psychol Measurement 1987;47:261±270.

21. LOYD BH, GRESSARD CP. Reliability and factorial validityof computer attitude scales. Educ Psychol Measurement1984;44:501±505.

22. FOLSTEIN MF, FOLSTEIN SE, MCHUGH PR. `Mini-mentalstate'. A practical method for grading the cognitive stateof patients for the clinician. J Psychiatr Res 1975;12:189±198.

23. IHL R, WEYER G. Alzheimer's Disease Assessment Scale(ADAS) ± German Version. GoÈ ttingen: Beltz, 1994.

24. BRICKENKAMP R. Test d2 ± Aufmerksamkeits±Belastungs-test. GoÈ ttingen: Hogrefe, 1994.

25. BERINGER J. Experimental Run Time System (ERTS).Frankfurt/Main: BeriSoft Cooperation, 1993.

26. HAMILTON M. Development of a rating scale for primarydepressive illness. Br J Social Clin Psychol 1967;6:278±296.

27. HAMILTON M. Hamilton Depression Scale. In: Collegiuminternationale psychiatriae scalarum, ed. InternationaleSkalen fuÈ r Psychiatrie. GoÈ ttingen: Beltz, 1996:93±96.

28. BECK AT. Cognitive therapy of depression. New York:The Guilford Press, 1979.

29. MCNAIR DM, LORR M, DOPPLEMAN LF. The Pro®le ofMood States (POMS). San Diego, CA: Educational andIndustrial Testing Service,6 1971.

30. FOLSTEIN MF, LURIA R. Reliability, validity, and clinicalapplication of the Visual Analogue Mood Scale. PsycholMed 1973;3:479±486.

31. OVERALL JE, GORHAM DR. Brief Psychiatric Rating Scale.In: Collegium internationale psychiatriae scalarum, ed.Internationale Skalen fuÈ r Psychiatrie. GoÈ ttingen: Beltz,1996:105±107.

32. ANDREASEN NC. The scale for the assessment of positivesymptoms (SAPS).7 University of Iowa, Iowa City, Iowa,1984.

33. ANDREASEN NC. The Scale for the Assessment of NegativeSymptoms (SANS): conceptual and theoretical founda-tions. Br J Psychiatry 1989;155(Suppl. 7):49±58.

34. NATIONAL INSTITUTE OF MENTAL HEALTH. Clinical globalimpressions. In: Collegium internationale psychiatriaescalarum, ed. Internationale Skalen fuÈ r Psychiatrie.GoÈ ttingen: Beltz, 1996:147±149.

35. DIETMAIERO, LAUXG.UÈ bersichtstabellen. In: RIEDERER P,LAUX G, POÈ LDINGER W, eds. Neuro-Psychopharmaka.Wien, New York: Springer, 1992.

36. FRENCH CC, BEAUMONT JG. The reaction of psychiatricpatients to computerized assessment. Br J Clin Psychol1987;26:267±278.

37. YANG C-K. Sociopsychiatric characteristics of adolescentswho use computer to excess. Acta Psychiatr Scand2001;1048 :217±222.

38. HUDIBURG RA. Relationship of computer hassles, somaticcomplaints, and daily hassles. Psychol Rep 1991;69:1119±1122.

39. HUDIBURG RA, AHRENS PK, JONES TM. Psychology ofcomputer use. 31. Relating computer users stress, dailyhassles, somatic complaints, and anxiety. Psychol Rep1994;75:1183±1186.

40. ROSEN LD, SEARS DC, WEIL MM. Computerphobia.Behavior research methods. Instrument Comput1987;19:167±179.

41. ROSEN LD, MAGUIRE PD Myths and realities of computer-phobia: a meta-analysis. Anxiety Res 1990;3:175±191.

42. TSENG HM, TIPLADY B, MACLEOD HA, WRIGHT P.Computer anxiety: a comparison of pen-based personaldigital assistants, conventional computer and paperassessment of mood and performance. Br J Psychol1998;89:599±610.

43. WEIL MM, ROSEN LD, SHAW S. Computerphobia reduc-tion program: clinical resource manual. Carson, CA:California State University Dominguez Hills, 1988.

44. JAY GM, WILLIS SL. In¯uence of direct computer experi-ence on older adults' attitudes toward computers. J Ger-ontol 1992;47:250±257.

45. SAMOILOVICH S, RICCITELLI C, SCHIEL A, SIEDI A. Attitudeof schizophrenics to computer videogames. Psycho-pathology 1992;25:117±119.

46. MORRISON GR, ROSS SM, GOPALAKRISHNAN M, CASEY J.The e�ects of feedback and incentives on achievement incomputer based instruction. Contemporary Educat Psychol1995;20:32±50.

Weber et al.

130