emotion processing: relevance to psychotic-like symptoms

Emotion processing: Relevance to psychotic-like symptoms

Devon Spaapen

BPsych

This thesis is presented for the degree of Master of Science of the University of Western

Australia.

School of Psychiatry and Clinical Neurosciences

2015

ii

ABSTRACT

Rationale: Studies show that negative emotions are highly prevalent in psychosis, and that

they play a key role in the onset and maintenance of psychotic symptoms. Emotions and

emotion processing, however, have not received much attention in the context of psychotic-

like experiences (PLE). PLE resemble the positive symptoms of psychosis, and are common

experiences in the general population, although they do not cause high distress or loss of

functioning. A better understanding of emotional vulnerabilities in individuals with PLE may

be used to put in place interventions aimed at reducing the risk for transition into psychosis.

Research aims: This thesis aimed to explore different aspects of emotion processing

(emotion perception, regulation, and negative affect) in people with and without PLE. Before

this relationship could be explored however, the psychometric properties of emotion

processing scales were investigated to determine their reliability and suitability so that they

could be used with confidence. Methods: Two large community samples from Australia (N =

575) and the United Kingdom (N = 597) completed a set of questionnaires. The

questionnaires included: the Emotion Regulation Questionnaire (ERQ; Gross & John, 2003),

Emotion Processing Scale (EPS; Baker, 2009), and the Depression, Anxiety and Stress Scale

(DASS; Lovibond & Lovibond, 1995). PLE were assessed using the Psychosis Screening

Questionnaire (PSQ; Bebbington & Nayani, 1995). First, the psychometric properties of the

ERQ and EPS were explored using Confirmatory Factor Analysis (CFA). Then the

relationships between the measures of emotion processing and PLE were investigated with

SEM analysis. Results: Adequate model fit was not established for the original ERQ (10

items), but with a minor adjustment, a revised version of the scale (ERQ-9) presented with

strong model fit. In contrast, the original EPS factor structure was not supported and attempts

to refine the factor structure were unsuccessful. In the final analyses, the relationship between

PLE, ERQ-9 and DASS was examined. PLE were linked to negative affect assessed with

iii

DASS (only through the shared variance between depression, anxiety and stress), but not to

poorer emotion regulation (ERQ-9). Conclusion: PLE are linked to negative affect.

However, PLE were not linked to poorer emotion regulation as assessed with the current

measures. It is recommended that future studies exploring emotion processing and PLE

include a wider range of validated emotion processing/regulation tasks, and analyse the

extent to which unique and shared variance of depression, anxiety and stress explain the

relationship between emotion processing and PLE.

iv

TABLE OF CONTENTS

Abstract……………………………………………………………………………….. ii

Statement of Original Contribution………………………………………………… ix

Manuscripts and publications generated from this thesis…………………………. xi

Acknowledgements…………………………………………………………………… xii

Preamble………………………………………………………………………………. xiii

CHAPTER 1: Introduction. ………………………………………………………… 1

1.1 Schizophrenia and psychosis……………………………………………………… 1

1.2 Psychotic like experiences (PLE)………………………………………………… 2

1.3 What is emotion processing……………………………………………………… 7

1.4 Neurobiological findings………………………………………………………… 8

1.5 Emotion processing and negative affect………………………………………… 9

1.5.1 Link between emotion perception and negative affect………………… 9

1.5.2 Link between emotion regulation and negative affect………………… 10

1.5.3 Link between emotion perception, regulation and negative affect…….. 11

1.6 Schizophrenia and emotion processing…………………………………………… 11

1.6.1 Negative affect in schizophrenia………………………………………. 11

1.6.2 Emotion perception in schizophrenia…………………………………… 12

1.6.3 Emotion regulation in schizophrenia…………………………………… 13

1.7 PLE and emotion processing…………………………………………………… 14

1.7.1 PLE and negative affect………………………………………………. 14

1.7.2 PLE and emotion perception………………………………………….. 14

1.7.3 PLE and emotion regulation………………………………………….. 15

v

1.8 Summary of PLE and emotion processing literature…………………………… 16

1.9 Aims of the thesis……………………………………………………………….. 17

1.10 References……………………………………………………………………… 18

1.11 Foreword to Chapters 2 and 3…………………………………………………… 34

Chapter 2: The Emotion Regulation Questionnaire: validation of the ERQ-9

in two community samples….……………………………………………………… 35

Abstract……………………………………………………………………………… 36

2.1 Method…………………………………………………………………………… 40

2.1.1 Participants…………………………………………………………….. 40

2.1.2 Materials………………………………………………………………. 42

2.1.3 Data analysis plan…………………………………………………….. 42

2.2 Results…………………………………………………………………………….. 44

2.2.1 Factor structure………………………………………………………… 44

2.2.2 Between samples invariance testing…………………………………… 45

2.2.3 Demographic invariance testing………………………………………. 46

2.2.4 The 9-item ERQ, demographics, and negative affect…………………. 48

2.3 Discussion……………………………………………………………………… 52

2.3.1 CFA properties ……………………………………………………… 52

2.3.2 Age differences………………………………………………………. 53

2.3.3 Education differences………………………………………………… 53

2.3.4 Gender differences…………………………………………………… 54

2.2.5 Negative affect………………………………………………………. 54

2.4 References……………………………………………………………………… 56

2.5 Acknowledgements……………………………………………………………… 62

vi

Chapter 3: Assessment of emotion processing: Analysing the psychometric

properties of the EPS-25.………………………………………………………….. 63

Abstract…………………………………………………………………………....... 63

3.1 Method…………………………………………………………………………… 66

3.1.1 Participants…………………………………………………………. 66

3.1.2 Materials…………………………………………………………… 68

3.2 Results…………………………………………………………………………… 68

3.2.1 Original confirmatory factor analysis…………………………………. 79

3.2.2 Exploratory factor analysis…………………...……………………….. 72

3.2.3 Revised 2-factor confirmatory factor analysis……………………….... 74

3.3 Discussion………………………………………………………………………… 74

3.4 References………………………………………………………………………… 78

3.5 Acknowledgements………………………………………………………………. 84

3.6 Forward to Chapter 4…………………………………………………………….. 85

Chapter 4: The relationship between emotion processing and psychotic-like

experiences………………………………………………………………………… 86

Abstract……………………………………………………………………………… 86

4.1 Psychotic-like experiences……………………………………………………… 87

4.2 Emotion regulation and negative affect………………………………………… 88

4.3 Negative affect influencing psychosis and PLE………………………………… 89

4.4 Emotion regulation and PLE………………….………………………………… 92

4.5 Method…………………………………………………………………………… 93

4.5.1 Participants…………………………………………………………… 93

4.5.2 Materials……………………………………………………………… 93

vii

4.6 Results…………………………………………………………………………. 96

4.6.1 PSQ (PLE prevalence)………………………………………………. 96

4.6.2 Demographics and ER, negative affect, and PLE…………………… 98

4.6.3 Factor analysis………………………………………………………. 100

4.6.4 Establishing the measurement models using confirmatory factor

Analysis…………………………………………………………………… 101

4.6.4.1 Full measurement model……………………………………. 101

4.6.5 Structural Equation Modelling Predicting PSQ with the quadripartite

model of the DASS………………………………………………………….. 103

4.6.6 ERQ PLE model (direct effects)……………………………………… 104

4.6.7 Full ERQ, DASS quadripartite, PLE model………………………….. 104

4.7 Discussion……………………………………………………………………… 106

4.7.1 Negative affect and PLE………………………………………………. 106

4.7.2 ER and negative affect………………………………………………… 108

4.7.3 Emotion regulation and PSQ…………………………………………. 109

4.8 References………………………………………………………………………. 113

Chapter 5: General Discussion…………………………………………………… 128

5.1 Chapter 2………………………………………………………………………… 128

5.2 Chapter 3………………………………………………………………………… 129

5.3 Chapter 4………………………………………………………………………… 130

5.4 Negative affect, emotion processing problems: relevance to PLE?……………... 131

5.5 Limitations……………………………………………………………………… 132

5.6 Areas for future research………………………………………………………… 133

5.7 References……………………………………………………………………….. 135

5.8 Appendices and supplementary materials………………………………………... 138

viii

5.8.1 Chapter 2……………………………………………………………….. 138

5.8.2 Chapter 3……………………………………………………………….. 152

5.8.3 Chapter 4……………………………………………………………….. 158

ix

Statement of Original Contribution

The work contained in this thesis has not previously been submitted for a degree or diploma

at any other higher education institution. This thesis is entirely my own work and has been

accomplished during enrolment in the degree of Masters of Science (Psychiatry). All external

sources have been acknowledged.

Data collection was conducted in collaboration with other researchers or institutions, as part

of a larger study titled "Emotions across the Lifespan", directed by Professor Romola Bucks

(RSB) as outlined in the Acknowledgements section. However, the design of the current

project, data analysis, and the preparation of this thesis have been completed by the candidate

alone with guidance from his supervisors.

The articles that were published or submitted as a result of the work undertaken for this thesis

are included in chapters 2 (published), 3 (submitted). Chapter 4 will be submitted as soon as

the thesis has been marked.

The student completed all analyses, formulated and wrote the papers. The other authors on

the papers provided intellectual input, data collection, advised on the analyses and

interpretation, and assisted with formulation and editing of the papers. More specifically, in

chapter 2, the authors A/Prof Flavie Waters (FW) and RSB provided intellectual input into

the conceptualisation of the concept, conducted quality assessment, statistical advice, and

edited the manuscript. Additional statistical guidance, and advice regarding analysis and

interpretation was provided by Patrick Dunlop (PD) and Mark Griffin (MG). Data was

collected by RSB, Lusia Stopa (LS) and Laura Brummer (LB). LS and LB also reviewed the

chapter. In chapters 3, the authors RSB, FW and PD provided intellectual input, advised on

the analyses and interpretation, and assisted with editing the manuscript. Additional statistical

x

guidance was provided by MG. Data was collected by RSB, LS and LB. LS and LB also

reviewed the chapter.

In chapter 4, the authors FW, RSB and DP provided intellectual input, advised on the

analyses and interpretation, and assisted with editing the manuscript. Data was collected by

RSB, LS and LB.

xi

Manuscripts and publications generated from this thesis.

Published

Spaapen, D. L., Waters, F., Brummer, L., Stopa, L., & Bucks, R. S. (2014) The Emotion

Regulation Questionnaire: validation of the ERQ-9 in two community samples.

Psychological Assessment. 26. 46-54. (Chapter 2).

Under review

Spaapen, D. L., Dunlop, P. D., Waters, F., Brummer, L., Stopa, L., & Bucks, R. S. (under

review). Assessment of emotion processing: Analysing the psychometric properties of

the EPS-25. Journal of Psychosomatic Research. (Chapter 3).

Conference Presentations

Spaapen, D. L., Waters, F., Brummer, L., Stopa, L., & Bucks, R. S. (2013). The Emotion

Regulation Questionnaire: Development and validation of the ERQ-9 in two

community samples. The Australasian Society for Psychiatric Research (ASPR)

Conference 2012 – Fremantle, Western Australia.

Spaapen, D. L., Waters, F., Brummer, L., Stopa, L., & Bucks, R. S. (2013). Assessment of

emotion processing: Analysing the psychometric properties of the EPS-25. The

Australasian Society for Psychiatric Research (ASPR) Conference 2012 – Fremantle,

Western Australia.

xii

Acknowledgements

I have been extremely fortunate to have two great supervisors, Flavie Waters and

Romola Bucks. From initially seeing my potential through volunteering at the CRC and

giving me the chance to pursue my psychology aspirations, to providing all forms of

guidance on manuscripts and becoming a researcher, I truly owe all of my future psychology

career, and much of my personal growth to both of you. The maturity, professionalism, and

friendly collaboration between you and other co-authors has provided me a solid grounding

and high hopes for what is possible in future collaborations. I can’t thank you enough for the

lovely meetings and the necessary reality checks.

Without the guidance and large contributions of Patrick Dunlop, particularly in

relation to statistical analysis, I’m not sure what direction the thesis or the three papers for

publication would have taken. The in-depth guidance through my exploration of structural

equation modelling, exploratory and confirmatory factor analysis has been invaluable. Whilst

learning CFA and SEM in AMOS and MPlus was testing at times, I feel like this exploration

of statistical anaylses has left me with a far better understanding of the fundamentals of

research, and a wealth of skills for the future that I can’t imagine being without. I’m very

grateful to have you in my life. As one of my closest friends, you have always been an

inspiration and role model to me.

I would also like to thank Mark Griffin for statistical guidance with the ERQ and EPS

statistics. At a time when I was lost and how to progress, your help was very timely and

provided direction at a critical moment.

xiii

Preamble

This thesis sought to evaluate the presence of emotional processing difficulties in people with

psychotic-like experiences (PLE) without the need for care. Because individuals with PLE

are thought to be at ‘high risk’ to make a transition to psychosis, the identification of the

specific type of emotion processing dysfunctions in PLE is likely to be informative when

putting in place intervention strategies aimed at reducing the vulnerability of the individuals

in developing psychosis.

The methods for this thesis include a set of questionnaires. The choice of measures was

driven by a theoretical framework which suggests that emotion processing is better

understood as a family of processes, comprising emotion perception, regulation and negative

emotions. Tasks were chosen because they were well known, frequently used, and generally

accepted in the emotion literature. With the exception of measures of negative mood (such as

the DASS), however, many measures of emotion perception and regulations have not been

fully validated. This is the case for two of the measures included in this thesis: the Emotion

Regulation Questionnaire (ERQ) and the Emotion Processing Scale (EPS).

Establishing the psychometric properties of an instrument is an important step in assessing

the responsiveness of measurement instruments (Nunnally, 1967). For this reason, my first

task was to begin this investigation by assessing the psychometric properties of the ERQ and

EPS. By validating these tasks, I hoped to use the best tasks possible to examine the

relationship between PLE and emotion processing. This process allowed a refinement of the

ERQ (Chapter 2); Unfortunately, I was not able to find any satisfactory fit for the EPS

(Chapter 3), despite best efforts to optimize its factor structure. Therefore, the measure could

not be used for the further analyses of emotion processing difficulties in PLE. In the final

chapter (chapter 4), the only suitable emotional processing tasks were the ERQ and DASS.

xiv

This narrowed the scope of examination of emotional processing in PLE, but is still able to

provide useful information regarding the relationship between emotion processing and

negative mood in the context of PLE.

This thesis is presented as a collection of papers in a format suitable for publication. At the

time of submission, Chapter 2 has been published, and Chapter 3 has been submitted for

publication (revise and resubmit). Chapter 4 will be submitted after this thesis has been

examined.

References

Nunnally, J. C. (1967). Psychometric Theory. New York: McGraw-Hill.

1

CHAPTER 1: Introduction

1.1 Schizophrenia and psychosis

The prototype for psychosis as an illness has often been schizophrenia. Schizophrenia

is a chronic mental illness with severe social, economic and health consequences. Along with

other psychotic disorders, schizophrenia is associated with poor self-care, low employment

rates (Marwaha & Johnson, 2004; Thornicroft et al., 2004), increased work absenteeism,

diminished social relations (including social mistreatment and isolation) trouble with the law,

and higher mortality rates (mostly through suicide and accidents; Access Economics, 2002;

Harris & Barraclough, 1998; Jablensky et al., 1999). Schizophrenia also has high comorbidity

with other mental disorders, including mood (Birchwood, Zaffer, Chadwick, & Trower,

2000), anxiety (Berman, Kalinowski, Berman, Lengua, & Green, 1995; Mueser et al., 1998;

Turnbull & Bebbington, 2001), substance abuse (Jablensky et al., 2000), and personality

disorders (Keown, Frank, & Kuipers, 2002; Keown, Holloway, & Kuipers, 2005), as well as

some physical disorders, including HIV, cancer, and hepatitis (Lawrence, Holman, &

Jablensky, 2001), increasing the likelihood of negative outcomes (Drake, Mueser, Clark, &

Wallach, 1996). The direct health cost of schizophrenia in Australia was approximately $660

million in 2001, and is projected to be over one billion in 2011 (Access Economics, 2002). In

addition, the indirect costs associated with schizophrenia are thought to be about double the

direct costs when including productivity. It is clear why it is considered to be a global leading

cause of disability and the overall disease burden (World Health Organisation, 2001).

Schizophrenia is a type of psychotic disorder. ‘Psychosis’ is characterised by a set of

symptoms (particularly positive symptoms like hallucinations, delusions, and

disorganisation), that are common features of schizophrenia (American Psychiatric

Association, 2000). Depending on the severity, these symptoms can be accompanied by

abnormal behaviour and impairment in social interaction and daily functioning. The lifetime

2

prevalence estimates of all psychotic disorders is approximately 3% (Perala et al., 2007), with

schizophrenia accounting for about a third of cases (American Psychiatric Association,

2000). Variations in the diagnostic instruments used, and methods of case-finding are

considered the reasons for varying prevalence estimates (Perala et al., 2007).

Unfortunately, research examining possible mechanisms that underlie psychotic

symptoms is challenging due to the confounds associated medication, comorbidity with other

mental illnesses, heterogeneous clinical symptom presentation, the challenges of working

with individuals who are often acutely mentally ill (Modinos, Renken, Ormel, & Aleman,

2011).

To discount these extraneous features of mental illness, a frequently adopted method

is the examination of people who show similar symptoms, but without the confounds

associated with clinical populations. These methods allow investigations into the mechanisms

of psychosis without confounds such as anti-psychotic medications, chronicity, multiple

symptoms (comorbidity), and lack of insight (McCreery & Claridge, 1996). In regards to

schizophrenia, this involves the examination of people in the general community who have

Psychotic-Like Experiences (PLE).

1.2 Psychosis-like experiences (PLE)

A detailed review of PLE is provided in chapter 4 so it is not repeated here. Briefly,

PLE refer to group of psychotic experiences that are found in the general population, which

occur below the clinical threshold, and that are not associated with high levels of distress or

functional disability. PLE are less frequent, intense and intrusive than psychotic symptoms,

and are typically not associated with strong emotional reactions. PLE are sometimes referred

to as ‘schizotypy’ or ‘anomalous perceptual experiences’ although these have their own

3

conceptual tradition which means that the overlap is not complete (Laroi, Raballo, & Bell, in

press).

PLE include a range of experiences, including: hearing a few words, or one’s name

when the person is alone, seeing shapes, animals or faces when there is nothing in the

environment to account for it, the experience of paranoia and that people wish them harm, the

belief in magical powers or that they can read other people’s mind or move objects with their

minds. These experiences are relatively common, with estimates ranging between 8% to

43%, depending on the population sample, measure, and symptom criteria (Loewy, Johnson,

& Cannon, 2007; van Os, Linscott, Myin-Germeys, Delespaul, & Krabbendam, 2009; Wiles

et al., 2006). In a recent meta-analysis, and using strict symptom definitions, van Os and

colleagues (2009) made a distinction between psychotic symptoms with clinical relevance

(i.e. associated with distress and help-seeking behaviour) and psychotic experiences that are

associated with neither distress nor help-seeking behavior. The prevalence rates were 4%

versus 8% respectively (see Figure 1).

Findings that these experiences differ as a function of degree across both healthy and

psychiatric ill populations has led to a ‘psychosis continuum’ view. This dimensional view

proposes psychopathological symptoms of psychosis (psychotic phenotype) occur on a

continuum of severity, ranging from psychotic-like experiences in the general community to

full-blown clinical psychosis (van Os, Hanssen, Bijl, & Ravelli, 2000; van Os et al., 2009),

with people falling somewhere along this continuum of psychosis expression (Allardyce,

Suppes, & Van Os, 2007). According to this conceptualization, symptoms may change from

sub-clinical symptoms to its clinical manifestation in the form of psychosis, with a

corresponding increase in symptom intensity, frequency, persistence, and functional

impairments.

4

Figure 1. Psychosis: variation along a continuum. Reprinted from “A systematic review and

meta-analysis of the psychosis continuum: evidence for a psychosis-proneness-persistence-

impairment model of psychotic disorder,” by J. van Os et al., 2009, Psychological Medicine,

39, p. 184. Copyright 2003 by Elsevier B. V.

These distribution findings have been proposed as evidence that PLE are part of a

general vulnerability for schizophrenia. Other evidence tends to support this proposal. First,

longitudinal studies show that PLE may predict the future transition to psychosis. Hanssen,

Bak, Bijl, Vollebergh, and van Os (2005), for example, reported that 8% of individuals with

PLE had developed psychosis after a 2-year follow up, and this figure increases to 25% after

15 years (Poulton et al., 2000). Second, the relationship between psychiatric disorders and

demographic characteristics (e.g. male gender, ethnic minority, being unemployed and

unmarried) is also observed in people with PLE (McGrath et al., 2004; van Os et al.,

2009).Third, there is biological overlap, as shown in co-clustering of clinical and subclinical

psychosis phenotypes in families (Kendler, Mcguire, Gruenberg, O'hare, & Walsh, 1993; van

Os et al., 2009). In sum, many now view this phenotype as an underlying liability for

psychosis (Rössler, Hengartner, Ajdacic-Gross, Haker, & Angst, 2013).

5

A different view suggests a more careful examination of the evidence is warranted

(De Leede-Smith & Barkus, 2013). For instance, Larøi (2012) proposed that the continuum of

(symptom) experiences must be distinguished from that of a continuum of risk. Specifically,

there is strong evidence that symptom experiences vary on a continuum: for example,

auditory hallucinations vary from indistinct sounds (scratching, buzzing) to increasingly

audible and complex voices. Similarly, visual hallucinations start as shadows in the corner of

the visual field, or a sensation that objects in the environment look “different” or changed, all

the way through to fully formed and complex visions. Finally, delusions vary on a continuum

of bizarreness and strength of conviction. By contrast, a continuum at the level of risk has

been proposed, albeit with less consensus. Accordingly, Larøi argued that the symptom-

continuum may not necessarily dovetail with risk mechanisms.

Regardless of this dissociation, the idea of a psychosis continuum implies that any

underlying mechanisms associated with schizophrenia should also be detectable at the

subclinical level. There is some evidence supporting this view, although some contradictory

findings exists in cognitive and psychological mechanisms (Johns et al., 2014). There are also

some differences in phenomenological characteristics, with differences in negative content

and persistence (Larøi et al., 2012).

Altogether, the continuum view of psychosis has not altogether reached consensus.

Nonetheless, research into the extended psychosis phenotype is important. First, it allows a

better understanding of this subclinical group who may, potentially, experience general

vulnerability for psychosis and schizophrenia. Secondly, it allows the identification of risk,

and protective factors that determine whether this group might develop a diagnosable

psychotic illness.

In recent years, emotion processing has emerged as an important area of research

in schizophrenia. There is now recognition that dysfunctional emotion processing (including

6

emotion perception, regulation and negative affect) is a core component of schizophrenia,

directly influencing symptom frequency, content, and severity (Badcock, Paulik, & Maybery,

2011; Serper & Berenbaum, 2008; Tien & Eaton, 1992). It also significantly impacts

individuals’ quality of life (Baslet, Termini, & Herbener, 2009), as well as social and

functional outcomes, including work functioning and living independently (Henry, Rendell,

Green, McDonald, & O'Donnell, 2008; Kee, Green, Mintz, & Brekke, 2003; Kimhy et al.,

2012). However, substantially less research has been conducted in individuals with PLE.

Compared to prenatal or early aetiological factors that contribute to schizophrenia onset and

maintenance (Hanssen, Krabbendam, Vollema, Delespaul, & Van Os, 2006; Linney et al.,

2003; Stefanis et al., 2004), emotion processing is an avenue for experimentation and

intervention that is more amenable to change. In the pursuit of factors that place people at

risk of developing schizophrenia, potentially preventing symptom onset, and mitigating the

personal impact of schizophrenia symptoms, emotion processing may be an understudied, but

valuable area of research. The link between emotion processing and schizophrenia is

discussed further below. Its relationship to PLE symptoms is discussed in depth in Chapter 4.

1.3 What is emotion processing

The way we arrive at emotions, how they affect us, and how they are managed is

determined by a set of interrelated processes (Koole, 2009). The application of these

processes, that determine our experience of emotion, is commonly referred to as ‘Emotion

Processing’. Whilst there has been considerable investment in emotion processing research in

the last decade, these processes are still not fully understood. To refine the current

7

understanding of how aspects of emotion processing work, studies have conceptualised these

as how a person’s emotional response develops over time (Baumann, Kaschel, & Kuhl, 2007;

Davidson, Jackson, & Kalin, 2000; Gross, 1998b; Skinner & Zimmer-Gembeck, 2007). In

particular, studies tend to differentiate between a person’s primary emotional response

(emotion perception) and their secondary response (emotion regulation). This distinction has

been adopted due the apparent functional differences between them.

In response to an emotionally salient stimulus, emotion perception serves to identify

the emotional significance of a stimulus and provide input for the generation of emotional

states and emotion regulation (Koole, 2009; M. L. Phillips, W. C. Drevets, S. L. Rauch, & R.

Lane, 2003b). Furthermore, emotion perception is thought to be largely automatic,

uncontrolled and unregulated. The generation/expression of affect states is also considered to

be part of emotion perception, but will be discussed separately in the following sections

better to articulate the relationship between emotion processing (including negative affect),

and psychosis. Emotion regulation (ER) is typically considered to be a controlled, deliberate

process that seeks to modify or override a person’s spontaneous emotion response (Koole,

2009). Understandably, the successful regulation of emotions is dependent on the initial

perception of an emotional stimulus (Füstös, Gramann, Herbert, & Pollatos, 2012).

Therefore, difficulty in identifying and expressing one’s emotions is thought to impede the

ability to regulate emotions (Gross, 1998b; Lane, Sechrest, Riedel, Shapiro, & Kaszniak,

2000; Taylor, Bagby, & Parker, 1997).

Not all ER strategies are adaptive (e.g. reappraisal; reframing how one thinks about an

event to be less negative or more positive), the excessive or dysfunctional use of some

strategies (e.g. expressive suppression; masking behaviour which expresses emotion) is

associated with individuals experiencing negative outcomes including negative affect

(Aldao, Nolen-Hoeksema, & Schweizer, 2010; Beevers & Meyer, 2004; Blalock & Joiner,

8

2000). Negative affect refers to the experience of negative emotional states, for example,

depression, anxiety, and distress. Studies have found that dysfunctional or maladaptive ER

results in increased depression, anxiety and stress (Gross, 2002).

1.4 Neurobiological findings

In support of these psychological constructs, neurobiological studies have found

structural and functional links in related brain networks. Aspects of emotion processing

(specifically, emotion perception and affect) have been linked to activation in the amygdala,

insula, ventral striatum, thalamus, anterior cingulate gyrus, and areas of the prefrontal cortex

(M. L. Phillips & Young, 1997; Reiman et al., 1997; Shin et al., 2000). The use of ER

strategies has been linked to activation in the prefrontal cortex (PFC), distal and subgenual

anterior cingulate cortex (ACC), amygdala, insula, and hippocampus (Campbell-Sills et al.,

2011; Goldin, McRae, Ramel, & Gross, 2008; Phan et al., 2005; Pitskel, Bolling, Kaiser,

Crowley, & Pelphrey, 2011). Many of these areas of activation are similar in reappraisal and

suppression, however, during reappraisal, activation of the PFC down-regulates the neural

substrates in primary emotion processing areas (e.g. amygdala and insula), reducing the

experience of negative emotion (Beauregard, Levesque, & Bourgouin, 2001; Goldin et al.,

2008; Levesque et al., 2003; Ochsner, Bunge, Gross, & Gabrieli, 2002; Ochsner et al., 2004;

Phan et al., 2005). In contrast, activation during suppression does not result in reductions of

primary emotion processing areas (Goldin et al., 2008), and also increases sympathetic

nervous system activity (Demaree et al., 2006; Gross & Levenson, 1993; Roberts, Levenson,

& Gross, 2008). Whilst the brain areas involved in emotion processing and regulation appear

largely to overlap, there is significant differentiation (M. L. Phillips et al., 2003b). There is

evidence that the neural systems involved in emotion perception and affect (ventral system),

ER (dorsal system for effortful ER) have a reciprocal functional relationship, suggesting that

9

deficits in functioning in a single emotion processing area may be associated with problems

in other aspects of emotion processing (M. L. Phillips et al., 2003b). Several studies have

found structural and functional abnormalities, in the neural areas required in emotion

processing, in patients with schizophrenia (Hirsch & Weinberger, 2003; see Phillips, W. C.

Drevets, S L. Rauch, & R. Lane, 2003a for review) . These abnormalities have been

associated with specific symptoms of schizophrenia, including hallucinations and persecutory

delusions (M. L. Phillips et al., 2003a).

1.5 Emotion processing and negative affect

Several studies indicate that dysfunction in emotion perception/generation, and

emotion regulation (ER), are related to negative affect (e.g. depression, anxiety). Research

examining the relationship between negative affect and each of these aspects of emotion

processing is described below.

1.5.1 Link between emotion perception and negative affect. The capacity to

identify and express emotions (which are aspects of alexithymia) is an integral part of

emotion perception. Previous research suggests that alexithymia is associated with anxiety

(Karukivi et al., 2010), depression, decreased social functioning (Spitzer, Siebel-Jurges,

Barnow, Grabe, & Freyberger, 2005; Vanheule, Desmet, Meganck, & Bogaerts, 2007), and

reduced life satisfaction (Bamonti et al., 2010; Honkalampi, Hintikka, Tanskanen, Lehtonen,

& Viinamaki, 2000; Mattila, Poutanen, Koivisto, Salokangas, & Joukamaa, 2007; Valkamo et

al., 2001).

10

1.5.2 Link between emotion regulation (ER) and negative affect. Evidence

suggests that some types of ER are associated with increased psychopathology, negative

affect, and decreased quality of life (Berenbaum, Raghavan, Le, Vernon, & Gomez, 2003;

Gross, 2002; Gross & John, 2003). For example, greater use of experiential suppression

(suppression of thoughts/feelings) was associated with increases in depressive symptoms in

college students (Beevers & Meyer, 2004; Wenzlaff & Luxton, 2003). Similar findings have

been observed for studies of expressive suppression (inhibiting the expression of emotion;

e.g. poker face), which suggest that individuals who predominantly use this regulation

strategy are more susceptible to depression (Matheson & Anisman, 2003), have lower self-

esteem, and are less satisfied with themselves and their relationships (Gross & John, 2003).

Expressive suppression is also associated with greater levels of anxiety (Badcock et al., 2011)

and decreased positive experiences (Kashdan & Breen, 2008). Furthermore, experimental and

longitudinal studies of expressive and thought suppression suggest that maladaptive ER

strategies result in increased negative affect (e.g. depression, anxiety, stress; Beevers &

Meyer, 2004; Brans, Koval, Verduyn, Lim, & Kuppens, 2013; Gross, 2002). A meta-analytic

review of ER research found that reappraisal was negatively associated with depression and

marginally negatively associated with anxiety (Aldao et al., 2010).

Blalock and Joiner (2000) found that cognitive avoidance predicted increases in

depression and anxiety symptoms over three weeks in college students, particularly in men.

No significant effects were observed for behavioural avoidance. Finally, Holahan, Moos,

Holahan, Brennan, and Schutte (2005) found that a combination of cognitive and behavioural

avoidance predicted increases in depressive symptoms over 10 years in middle to older aged

adults. This is important because it shows that dysfunctional emotional processing/regulation

can have an enduring negative impact on mental health.

11

1.5.3 Link between emotion perception, regulation and negative affect. Despite

the existing research on the relation between emotion perception, regulation, and negative

affect, the exact nature of these relationships is still unclear. For example, it is possible that

the relationship between perception and depression is moderated by ER. Specifically,

difficulties in identifying and expressing one’s emotions (required for reappraisal) is thought

to impede one’s ability to regulate emotions (Gross, 1998b; Taylor et al., 1997). In addition,

it is possible that there is a feedback loop between negative affect and processing. For

example, findings from Honkalampi, Hintikka, Laukkanen, Lehtonen, and Viinamaki (2001)

suggest that depression increases alexithymic traits. There is, as yet, no consensus in the

literature about these issues. Negative affect may also influence cognitive biases (such as

negative self and interpersonal schemata and attentional biases; Foland-ross & Gotlib, 2012;

Mogg & Bradley, 2005) and regulation (M. L. Phillips et al., 2003b).

In summary, dysfunctional emotion perception and regulation are associated with

negative affect. This is highly pertinent for psychosis given that negative affect is

prominently featured in individuals with the disorder. This points to the possibility that

emotion processing difficulties may be an antecedent to the onset of psychosis, representing a

vulnerability, and possibly a maintaining factor for psychosis.

1.6 Schizophrenia and emotion processing

Studies have shown a strong relationship between schizophrenia and negative affect,

emotion perception, and ER.

1.6.1 Negative affect in schizophrenia. Studies have consistently found that people

with schizophrenia have increased levels of depression, anxiety, and stress. Associations have

also been shown between increased negative affect (depression, anxiety, stress) and psychotic

12

symptoms (Birchwood et al., 2000; Garety, Kuipers, Fowler, Freeman, & Bebbington, 2001;

Turnbull & Bebbington, 2001). Results from Smith et al. (2006) suggest that individuals who

are more depressed, have lower self-esteem, and more negative self-beliefs, and tend to

experience more frequent and intense auditory hallucinations with negative content.

Furthermore, individuals who experience psychosis appear to experience negative emotions

more frequently than healthy controls, irrespective of whether they are diagnosed with or

without affective symptoms (Suslow, Roestel, Ohrmann, & Arolt, 2003).

The role of negative affect has also been cited as a key factor in the transition to

psychosis. Large-scale studies focusing on negative affect well before symptom development

have consistently shown that individuals with poor social adjustment (including greater social

anxiety) in adolescence were significantly more likely to develop schizophrenia in later years

(Jones, Murray, Jones, Rodgers, & Marmot, 1994; Kugelmass et al., 1995; Malmberg, Lewis,

David, & Allebeck, 1998). An early study by Tien and Eaton (1992) also found that anxiety

was associated with increased likelihood of positive symptoms a year later. In addition,

studies of the prodrome phase of psychosis reveal that depression, anxiety and irritability

precede symptom onset in 60-80% of cases (see reviews by Docherty, Van Kammen, Siris, &

Marder, 1978; Yung & McGorry, 1996). Altogether, negative affect is often considered to be

a key factor in the development and maintenance of symptoms, albeit it may not be sufficient

as an initiating factor (Freeman & Garety, 2003).

1.6.2 Emotion perception in schizophrenia. Deficits in emotion perception have

been found in studies of psychosis and schizophrenia. In particular, problems with emotional

awareness, recognition, and expression have been associated with positive and negative

symptoms of psychosis/schizophrenia in several studies (see Tremeau, 2006 for review). A

study by Cedro, Kokoszka, Popiel, and Narkiewicz-Jodko (2001) found that outpatients with

13

schizophrenia reported greater difficulty in identifying their emotions (emotion perception),

and exhibited a more externally oriented thinking style (suggesting less attention to their

emotions), when compared to a control group. Serper and Berenbaum’s (2008) study suggests

that emotional awareness is associated with delusions and hallucinations in psychosis. For the

schizophrenia spectrum disorder group, lower levels of attention to emotion were associated

with more severe hallucination ratings. In the other psychosis group (mood and substance

abuse), higher levels of attention to emotion were related to higher ratings of delusion. Baslet

et al. (2009) also found that schizophrenia-spectrum patients exhibited a less complex

understanding of emotions in other people, when compared to a healthy control group (even

after accounting for premorbid intelligence). Overall, this suggests that individuals with

psychosis and schizophrenia have difficulties in emotion perception, particularly with

recognising and understanding emotions in themselves and others.

1.6.3 Emotion regulation in schizophrenia. Deficits in ER have also been found in

studies of psychosis and schizophrenia. Recent studies indicate that individuals with

schizophrenia are less likely to use cognitive reappraisal and more likely to use expressive

suppression than non-patient control groups (Livingstone, Harper, & Gillanders, 2009; van

der Meer, van 't Wout, & Aleman, 2009). However, this finding is not unanimous, with others

not observing a significant difference in reappraisal and expressive suppression between

schizophrenia patients and a healthy control group (Henry et al., 2008). Nevertheless, the

latter authors did find that problems with up-regulating emotion expression were related to

blunted affect. Badcock et al. (2011) reported that greater use of expressive suppression was

related to increased severity of hallucinations in people with schizophrenia (who are currently

experiencing hallucinations).

14

1.7 PLE and emotion processing

To date, there are limited studies exploring differences in emotion perception,

emotion regulation (ER), and negative affect based on levels of psychosis-proneness.

However, the studies so far suggest there may be a relationship. These findings will be

examined by (i) negative affect, emotion perception/generation, and ER.

1.7.1 PLE and negative affect. A relationship appears to exist between PLE and

negative affect. For example, studies by Fowler et al. (2006) and Martin and Penn (2001)

suggest that sub-clinical symptoms (including hallucination experiences and paranoid

ideation) are linked to feelings of anxiety. Furthermore, results from a study byMackie,

Castellanos-Ryan, and Conrod (2011) suggest that adolescents with persistent PLE have

higher levels of depression and anxiety, when compared to individuals with no psychotic-like

experiences. Similar studies have found a strong association between the presence (and

number of) PLE and distress and depressive symptoms, in secondary and tertiary students

(Armando et al., 2010; Yung et al., 2009).

1.7.2 PLE and emotion perception. Tentative evidence exists for a link between PLE

and emotion perception changes. A review of emotion processing deficits in individuals ‘at

risk’ for psychosis (often identified by genetic risk and PLE with distress) suggests there are

deficits in emotion perception in these individuals, however that such impairments varied

depending on the severity subtype of clinical symptoms, with positive symptoms being linked

to greater perception problems than negative symptoms, e.g. positive or negative; (L. K.

Phillips & Seidman, 2008). Research by van Rijn et al. (2011) suggests that individuals with

PLE tend to have specific emotion perception and social functioning deficits, particularly in

identifying and expressing emotions, as assessed using the Bermond-Vorst Alexithymia

15

Questionnaire (BVAQ), when compared to individuals without PLE. These emotion

perception difficulties were still significant after accounting for IQ. These findings are similar

to those seen in clinical populations. It is worth noting, however, that van Rijn et al. (2011)

used a small, adolescent only, sample (N = 57) comprised of two groups (a no-PLE control

group and an ‘ultra-high risk’ group) representing the extreme ends of the sub-clinical

spectrum). Furthermore, using the BVAQ, a previous study by van 't Wout, Aleman, Kessels,

Larøi, and Kahn (2004) found individuals with high psychosis-proneness were more sensitive

to emotional arousal, compared to low psychosis-prone individuals. Additional studies are

required to confirm the reliability and generalizability of emotion perception changes in PLE.

1.7.3 PLE and emotion regulation (ER). There are mixed and hard-to-interpret

findings regarding the relationship between ER and PLE. There appears to be a trend that

ineffective ER is associated with PLE (Modinos, Ormel, & Aleman, 2010; Westermann,

Kesting, & Lincoln, 2012; Westermann & Lincoln, 2011), however – unexpectedly - these

findings are not present for maladaptive ER strategies (e.g. suppression Henry et al., 2009;

Westermann et al., 2012). For example, a recent experimental study by Modinos et al. (2010)

found that reappraisal in high PLE individuals required more activation in pre-frontal areas,

and was less successful, compared to individuals with low PLE, pointing to poor ER.

Westermann et al. (2012) expanded on these results, showing that, in stressful social

situations, reappraisal predicted paranoia expression in individuals with high paranoia

proneness. This is unexpected given reappraisal is generally an adaptive ER strategy (Aldao

et al., 2010). One explanation is that successful reappraisal makes situations less stressful,

however, in high PLE individuals (particularly with high distress), cognitive biases including

(e.g. jumping to conclusions; Fine, Gardner, Craigie, & Gold, 2007; Lincoln, Lange, Burau,

Exner, & Moritz, 2010), pre-existing delusional beliefs (Freeman, Garety, Kuipers, Fowler, &

16

Bebbington, 2002; Garety et al., 2001), negative interpersonal schemata (Lincoln, Mehl,

Kesting, & Rief, 2011), and aberrant salience (a hyperdopaminergic state leading to the

assignment of inappropriate importance and motivational significance to stimuli; Kapur,

2003) disrupt normal reappraisal, resulting in ineffective ER and greater distress (see

Westermann et al., 2012). Overall, the mechanistic processes between ER and PLE maybe

complex and are not well understood.

In contrast to (mixed) findings for reappraisal, no relationship has been observed

between expressive suppression (generally considered a maladaptive strategy) and PLE

(Henry et al., 2009; Westermann et al., 2012). Another study by Westermann and Lincoln

(2011), using the CERQ, found difficulties in impulse control to be related to paranoia

(persecutory delusions), however, this measure does not measure specific ER strategies.

Overall, it is difficult to synthesize the literature in this area because the studies vary

considerably in symptom focus (e.g. Westerman, Kesting & Lincoln, 2012, measured

paranoia only, rather than general PLE), and the nature of the study samples investigated

(some particularly small N = 34, Modinos, 2010; as well as undergraduates; Modinos, 2010;

Henry et al., 2009, rather than community samples; and the selection of participants

presenting high-low extreme scores; Modinos, 2010, Henry, et al., 2009). Given the limited

number of studies examining ER and PLE, with only a few types of ER strategies addressed,

additional studies are required to explain the relationship between ER and PLE.

1.8 Summary of PLE and emotion processing literature

Overall, emotional processing deficits (emotion perception, regulation, and negative

affect) are common features of psychosis. By contrast, evidence of the relationship between

some aspects of emotion processing and PLE is more inconsistent. Specifically, there is a link

between PLE and negative affect (depression, anxiety and stress), although limited evidence

17

exists for dysfunction in emotion perception and regulation. In addition, there are few

systematic and rigorous studies of PLE. Furthermore, the findings are mixed and at times

contradictory, pointing to the need for a re-examination of this topic.

According to models of emotion processing a relationship should exists between

emotion perception, regulation and affect. On the basis of negative affect in PLE, one might

expect emotion perception and regulation deficits. This type of information would be key in

developing targeted interventions in social and cognition therapies. In addition, the finding

of emotional regulation and emotion perception deficits in PLE would support the view that

negative affect predates psychosis, and may be key to the transition to clinical status.

1.9 Aims of this thesis

The main objective of this thesis was to investigate the relationship between PLE in

the general community, and emotion processing, using a set of questionnaires. The measures

chosen had some preliminary validity and reliability and were frequently used in the emotion

literature. However, some of the measures designed to assess aspects of emotion processing,

specifically the Emotion Regulation Questionnaire (ERQ) and the Emotion Processing Scale

(EPS), were not fully validated. Given that establishing the psychometric properties of an

instrument is an important step in assessing the responsiveness of measurement instruments,

this thesis began by assessing the psychometric properties of the ERQ and EPS. By

validating the measures, I endeavoured to use the best tasks possible to examine the

relationship between PLE and emotion processing.

In Chapter 2, the psychometric properties of the ERQ were examined, and in Chapter

3, the EPS. In Chapter 4, I examine emotional regulation and negative affect in PLE, and the

relationship between emotional regulation and negative affect.

18

1.10 References

Access Economics. (2002). Schizophrenia: costs. An analysis of the burden of schizophrenia

and related suicide in Australia. Melbourne: SANE Australia.

Aldao, A., Nolen-Hoeksema, S., & Schweizer, S. (2010). Emotion-regulation strategies

across psychopathology: A meta-analytic review. Clinical Psycholology Review,

30(2), 217-237. doi: 10.1016/j.cpr.2009.11.004

Allardyce, Judith, Suppes, Trisha, & Van Os, Jim. (2007). Dimensions and the psychosis

phenotype. International Journal of Methods in Psychiatric Research, 16, 34-40.

American Psychiatric Association. (2000). Diagnostic and statistical manual of mental

disorders : DSM-IV-TR. Washington, DC: American Psychiatric Association.

Armando, Marco, Nelson, Barnaby, Yung, Alison R., Ross, Margaret, Birchwood,

Maximilian, Girardi, Paolo, & Nastro, Paolo Fiori. (2010). Psychotic-like experiences

and correlation with distress and depressive symptoms in a community sample of

adolescents and young adults. Schizophrenia Research, 119(1), 258-265.

Badcock, J. C., Paulik, G., & Maybery, M. T. (2011). The role of emotion regulation in

auditory hallucinations. Psychiatry Research, 185(3), 303-308.

Bamonti, Patricia M., Heisel, Marnin J., Topciu, Raluca A., Franus, Nathan, Talbot, Nancy

L., & Duberstein, Paul R. (2010). Association of alexithymia and depression symptom

severity in adults aged 50 years and older. American Journal of Geriatric Psychiatry,

18(1), 51-56.

Baslet, G., Termini, L., & Herbener, E. (2009). Deficits in emotional awareness in

schizophrenia and their relationship with other measures of functioning. The Journal

of Nervous and Mental Disease, 197(9), 655-660. doi:

10.1097/NMD.0b013e3181b3b20f

19

Baumann, N., Kaschel, R., & Kuhl, J. (2007). Affect sensitivity and affect regulation in

dealing with positive and negative affect☆. Journal of Research in Personality,

41(1), 239-248. doi: 10.1016/j.jrp.2006.05.002

Beauregard, M, Levesque, J, & Bourgouin, P. (2001). Neural correlates of conscious self-

regulation of emotion. The Journal of Neuroscience, 21, 165-171.

Beevers, C. G. , & Meyer, B. (2004). Thought suppression and depression risk. Cognition &

Emotion, 18(6), 859-867. doi: 10.1080/02699930341000220

Berenbaum, Howard, Raghavan, Chitra, Le, Huynh-Nhu, Vernon, Laura L., & Gomez, Jose

J. (2003). A Taxonomy of Emotional Disturbances. Clinical Psychology: Science and

Practice, 10(2), 206-226. doi: 10.1093/clipsy.bpg011

Berman, Ileana, Kalinowski, Anthony, Berman, Stuart M., Lengua, JoAnn, & Green, Alan I.

(1995). Obsessive and compulsive symptoms in chronic schizophrenia.

Comprehensive Psychiatry, 36(1), 6-10. doi: 10.1016/0010-440x(95)90092-a

Birchwood, Maximilian, Zaffer, Iqbal, Chadwick, P, & Trower, P. (2000). Cognitive

approach to depression and suicidal thinking in psychosis: I. Ontogeny of post-

psychotic depression. British Journal of Psychiatry, 177, 516-521.

Blalock, Janice A., & Joiner, Thomas E. (2000). Interaction of Cognitive Avoidance Coping

and Stress in Predicting Depression/Anxiety. Cognitive Therapy and Research, 24(1),

47-65. doi: 10.1023/a:1005450908245

Brans, K., Koval, P., Verduyn, P., Lim, Y L., & Kuppens, P. (2013). The regulation of

negative and positive affect in daily life. Emotion, 13, 926-939. doi:

10.1037/a0032400

Campbell-Sills, L., Simmons, A. N., Lovero, K. L., Rochlin, A. A., Paulus, M. P., & Stein,

M. B. (2011). Functioning of neural systems supporting emotion regulation in

20

anxiety-prone individuals. Neuroimage, 54(1), 689-696. doi:

10.1016/j.neuroimage.2010.07.041

Cedro, Artur , Kokoszka, Andrzej , Popiel, Agnieszka , & Narkiewicz-Jodko, Wladyslaw.

(2001). Alexithymia in schizophrenia: an exploratory study. Psychological Reports,

89(1), 95-98.

Davidson, Richard J., Jackson, Daren C., & Kalin, Ned H. (2000). Emotion, Plasticity,

Context, and Regulation: Perspectives From Affective Neuroscience. Psychological

Bulletin, 126(6), 890-909.

De Leede-Smith, S., & Barkus, E. (2013). A comprehensive review of auditory verbal

hallucinations: lifetime prevalence, correlates and mechanisms in healthy and clinical

individuals. Frontiers in Human Neuroscience., 7, 367. doi: doi:

10.3389/fnhum.2013.00367

Demaree, H. A., Schmeichel, B. J., Robinson, J. L., Pu, J., Everhart, D. E., & Berntson, G. G.

(2006). Up- and down-regulating facial disgust: Affective, vagal, sympathetic, and

respiratory consequences. Biological Psychology, 71(1), 90-99. doi:

10.1016/j.biopsycho.2005.02.006

Docherty, J. P., Van Kammen, D. P., Siris, S. G., & Marder, S. R. (1978). Stages of onset of

schizophrenia psychosis. American Journal of Psychiatry, 135, 420-426.

Drake, Robert E., Mueser, Kim T., Clark, Robin E., & Wallach, Michael A. (1996). The

course, treatment, and outcome of substance disorder in persons with severe mental

illness. American Journal of Orthopsychiatry, 66(1), 42-51. doi: 10.1037/h0080153

Fine, C., Gardner, M., Craigie, J., & Gold, I. (2007). Hopping, skipping or jumping to

conclusions? clarifying the role of JTC bias in delusions. Cognitive Neuropsychiatry,

12, 46-77. doi: 10.1080/13546800600750597

21

Foland-ross, L. C., & Gotlib, I. H. (2012). Cognitive and neural aspects of information

processing in major depressive disorder: An intergrative perspective. Frontiers in

Psychology, 3, 1-17. doi: 10.3389/fpsyg.2012.00489

Fowler, D., Freeman, D., Smith, B., Kuipers, E., Bebbington, P., Bashforth, H., . . . Garety, P.

(2006). The Brief Core Schema Scales (BCSS): psychometric properties and

associations with paranoia and grandiosity in non-clinical and psychosis samples.

Psychological Medicine, 36, 749-759.

Freeman, D., & Garety, P. A. (2003). Connecting neurosis and psychosis: the direct influence

of emotion on delusions and hallucinations. Behaviour Research and Therapy, 41(8),

923-947. doi: 10.1016/s0005-7967(02)00104-3

Freeman, D., Garety, P. A., Kuipers, E., Fowler, D., & Bebbington, P. E. (2002). A cognitive

model of persecutory delusions. The British Psychological Society, 41, 331-347. doi:

10.1348/014466502760387461

Füstös, J, Gramann, K, Herbert, B. M , & Pollatos, O. (2012). On the embodiment of emotion

regulation: interoceptive awareness facilitates appraisal. Social Cognitive and

Affective Neuroscience, 8, 911-917. doi: 10.1093/scan/nss089

Garety, P. A., Kuipers, E., Fowler, D., Freeman, D., & Bebbington, P. E. (2001). A cognitive

model of the positive symptoms of psychosis. Psychological Medicine, 31(2), 189-

195.

Goldin, P. R., McRae, K., Ramel, W., & Gross, J. J. (2008). The neural bases of emotion

regulation: reappraisal and suppression of negative emotion. Biological Psychiatry,

63(6), 577-586. doi: 10.1016/j.biopsych.2007.05.031

Gross, J. J. (1998). The Emerging Field of Emotion Regulation: An Integrative Review.

Review of General Psychology, 2(3), 271-299. doi: 10.1037/1089-2680.2.3.271

22

Gross, J. J. (2002). Emotion regulation: affective, cognitive, and social consequences.

Psychophysiology, 39(3), 281-291. doi: 10.1017/S0048577201393198

Gross, J. J., & John, O. P. (2003). Individual differences in two emotion regulation processes:

Implications for affect, relationships, and well-being. Journal of Personality and

Social Psychology, 85(2), 348-362. doi: 10.1037/0022-3514.85.2.348

Gross, J. J., & Levenson, Robert W. (1993). Emotional Suppression: Physiology, Self-Report,

and Expressive Behavior. Journal of Personality and Social Psychology, 64(6), 970-

986.

Hanssen, M., Bak, M., Bijl, R., Vollebergh, W., & van Os, J. (2005). The incidence and

outcome of subclinical psychotic experiences in the general population. British

Jouranl of Clinical Psychology, 44(Pt 2), 181-191. doi: 10.1348/014466505X29611

Hanssen, M., Krabbendam, L., Vollema, M., Delespaul, P., & Van Os, J. (2006). Evidence

for instrument and family-specific variation of subclinical psychosis dimensions in the

general population. Journal of Abnormal Psychology, 115(1), 5-14.

Harris, E. C, & Barraclough, B. (1998). Excess mortality of mental disorder. The British

Journal of Psychiatry, 173, 11-53.

Henry, J. D., Green, M. J., Restuccia, C., de Lucia, A., Rendell, P. G., McDonald, S., &

Grisham, J. R. (2009). Emotion dysregulation and schizotypy. Psychiatry Research,

166(2-3), 116-124. doi: 10.1016/j.psychres.2008.01.007

Henry, J. D., Rendell, P. G., Green, M. J., McDonald, S., & O'Donnell, M. (2008). Emotion

regulation in schizophrenia: affective, social, and clinical correlates of suppression

and reappraisal. Journal of Abnormal Psychology, 117(2), 473-478. doi:

10.1037/0021-843X.117.2.473

Hirsch, S. R, & Weinberger, D. R. (2003). Schizophrenia (2nd ed.). Oxford: Blackwell

Publishing.

23

Holahan, C. J., Moos, R. H., Holahan, C. K., Brennan, P. L., & Schutte, K. K. (2005). Stress

generation, avoidance coping, and depressive symptoms: a 10-year model. Journal of

Consulting and Clinical Psychology, 73(4), 658-666. doi: 10.1037/0022-

006X.73.4.658

Honkalampi, K., Hintikka, J., Laukkanen, E., Lehtonen, J., & Viinamaki, H. (2001).

Alexithymia and depression: a prospective study of patients with major depressive

disorder. Psychosomatics, 42(3), 229-234.

Honkalampi, K., Hintikka, J., Tanskanen, A., Lehtonen, J., & Viinamaki, H. (2000).

Depression is strongly associated with alexithymia in the general population. Journal

of Psychosomatic Research, 48(1), 99-104. doi: 10.1016/S0022-3999(99)00083-5

Jablensky, A., McGrath, J., Herrman, H., Castle, D., Gureje, O., Morgan, V., & Korten, A.

(1999). People Living with Psychotic Illness: An Australian Study 1997-98. National

Mental Health and Wellbeing Report 4.

Jablensky, A., McGrath, John, Herrman, Helen, Castle, David, Gureje, Oye, Evans, Mandy, .

. . Harvey, Carol. (2000). Psychotic disorders in urban areas: an overview of the Study

on Low Prevalence Disorders. Australian & New Zealand Journal of Psychiatry,

34(2), 221-236. doi: 10.1046/j.1440-1614.2000.00728.x

Johns, L C., Kompus, K., Connell, M., Humpston, C., Lincoln, T M. , Longden, E, . . . Larøi,

F. . (2014). Auditory verbal hallucinations in persons with and without a need for

care. Schizophrenia Bulletin, 40, 255-264. doi: 10.1093/schbul/sbu005

Jones, P., Murray, R., Jones, P., Rodgers, B., & Marmot, M. (1994). Child developmental

risk factors for adult schizophrenia in the British 1946 birth cohort. The Lancet,

344(8934), 1398-1402. doi: 10.1016/S0140-6736(94)90569-X

24

Kapur, Shitij. (2003). Psychosis as a state of aberrant salience: A framework linking biology,

phenomenology, and pharmacology in schizophrenia. The American Journal of

Psychiatry, 160(1), 13-23.

Karukivi, M., Hautala, L., Kaleva, O., Haapasalo-Pesu, K. M., Liuksila, P. R., Joukamaa, M.,

& Saarijarvi, S. (2010). Alexithymia is associated with anxiety among adolescents.

Journal of Affective Disorders, 125(1-3), 383-387. doi: 10.1016/j.jad.2010.02.126

Kashdan, T. B., & Breen, W. E. (2008). Social anxiety and positive emotions: a prospective

examination of a self-regulatory model with tendencies to suppress or express

emotions as a moderating variable. Behaviour Therapy, 39(1), 1-12. doi:

10.1016/j.beth.2007.02.003

Kee, K. S., Green, M. J., Mintz, J., & Brekke, J. S. (2003). Is emotion processing a predictor

of functional outcome in schizophrenia? Schizophrenia Bullettin, 29, 487-497.

Kendler, K. S., Mcguire, M., Gruenberg, A., O'hare, A. M. Spellman, M., & Walsh, D.

(1993). The Roscommon Family Study: III. Schizophrenia-related personality

disorder in relatives. Archives of General Psychiatry, 50, 781-788.

Keown, Patrick, Frank, Holloway, & Kuipers, E. (2002). The prevalence of personality

disorders, psychotic disorders and affective disorders amongst the patients seen by a

community mental health team in London. Social Psychiatry & Psychiatric

Epidemiology, 37(5), 225.

Keown, Patrick, Holloway, Frank, & Kuipers, E. (2005). The impact of severe mental illness,

co-morbid personality disorders and demographic factors on psychiatric bed use.

Social Psychiatry & Psychiatric Epidemiology, 40(1), 42-49. doi: 10.1007/s00127-

005-0842-0

Kimhy, David, Vakhrusheva, Julia, Jobson-Ahmed, Lauren, Tarrier, Nicholas, Malaspina,

Dolores, & Gross, James J. (2012). Emotion awareness and regulation in individuals

25

with schizophrenia: Implications for social functioning. Psychiatry Research, 200(2–

3), 193-201. doi: 10.1016/j.psychres.2012.05.029

Koole, S. L. (2009). The psychology of emotion regulation: An integrative review. Cognition

& Emotion, 23(1), 4-41. doi: 10.1080/02699930802619031

Kugelmass, S., Faber, N. Ingraham, L. J., Frenkel, E., Nathan, M., Mirsky, A. F., & Shakhar,

G. B. (1995). Reanalysis of SCOR and anxiety measures in the Israeli High-Risk

Study. Schizophrenia Bulletin, 21.

Lane, R, Sechrest, L, Riedel, R, Shapiro, D. E, & Kaszniak, A. W. (2000). Pervasive emotion

recognition deficit common to alexithymia and the repressive coping style.

Psychosomatic Medicine, 62, 492-501.

Larøi, F. (2012). How do auditory verbal hallucinations in patients differ from those in non-

patients. Frontiers in Human Neuroscience., 6, 1-9. doi: 10.3389/fnhum.2012.00025

Larøi, F., E., Sommer. I, Blom, J D. , C, Fernyhough, Ffytche, D. H. , Hugdahl, K., . . .

Waters, F. (2012). The Characteristic Features of Auditory Verbal Hallucinations in

Clinical and Nonclinical Groups: State-of-the-Art Overview and Future Directions.

Schizophrenia Bulletin, 38, 724-733. doi: 10.1093/schbul/sbs061

Lawrence, D., Holman, D., & Jablensky, A. (2001). Preventable phusical illness in people

with mental illness. Crawley, Australia: Centre for Health Services Research.

Levesque, J, Eugene, F, Joanette, Y, Paquette, V, Mensour, B, Beaudoin, G, . . . Beauregard,

M (2003). Neural circuitry underlying voluntary suppression of sadness. Biological

Psychiatry, 53(6), 502-510.

Lincoln, T M., Lange, J., Burau, J., Exner, C., & Moritz, S. (2010). The effect of state anxiety

on paranoid ideation and jumping to conclusions. An experimental investigation.

Schizophrenia Bulletin, 36, 1140-1148. doi: 10.1093/schbul/sbp029

26

Lincoln, T M., Mehl, S., Kesting, M., & Rief, W. (2011). Negative symptoms and social

cognition: Identifying targets for psychological interventions. Schizophrenia Bullettin,

37, 23-32. doi: doi:10.1093/schbul/sbr066

Linney, Y., Murray, R., Peters, E., MacDonald, A., Rijsdijk, F., & Sham, P. (2003). A

quantitative genetic analysis of schizotypal personality traits. Psychological Medicine,

33(5), 803.

Livingstone, K., Harper, S., & Gillanders, D. (2009). An exploration of emotion regulation in

psychosis. Clinical Psychology & Psychotherapy, 16(5), 418-430. doi:

10.1002/cpp.635

Loewy, Rachel L., Johnson, Jennifer K., & Cannon, Tyrone D. (2007). Self-report of

attenuated psychotic experiences in a college population. Schizophrenia Research,

93(1-3), 144-151.

Mackie, C. J., Castellanos-Ryan, N., & Conrod, P. J. (2011). Developmental trajectories of

psychotic-like experiences across adolescence: impact of victimization and substance

use. Psychological Medicine, 41(1), 47-58.

Malmberg, A., Lewis, G., David, A., & Allebeck, P. (1998). 1998. The British Journal of

Psychiatry, 172, 308-313.

Martin, James A. , & Penn, David L. (2001). Social cognition and subclinical paranoid

ideation. British Journal of Clinical Psychology, 40(3), 261.

Marwaha, Steven, & Johnson, Sonia. (2004). Schizophrenia and employment - a review.

Social Psychiatry & Psychiatric Epidemiology, 39(5), 337-349.

Matheson, K., & Anisman, H. (2003). Systems of coping associated with dysphoria, anxiety

and depressive illness: a multivariate profile perspective. Stress, 6(3), 223-234. doi:

10.1080/10253890310001594487

27

Mattila, Aino K., Poutanen, Outi, Koivisto, Anna-Maija, Salokangas, Raimo K. R., &

Joukamaa, Matti. (2007). Alexithymia and life satisfaction in primary healthcare

patients. Psychosomatics, 48(6), 523-529.

McCreery, Charles, & Claridge, Gordon. (1996). A study of hallucination in normal

subjects—I. Self-report data. Personality and Individual Differences, 21(5), 739-747.

doi: 10.1016/0191-8869(96)00115-8

McGrath, J., Saha, S., Welham, J., El Saadi, O., Maccauley, C., & Chant, D. (2004). A

systematic review of the incidence of schizophrenia: the distribution of rates and the

influence of sex, urbanicity, migrant status and methodology. BMC medicine, 2, 13.

doi: 10.1186/1741-7015-2-13

Modinos, Gemma, Ormel, Johan, & Aleman, A. (2010). Altered activation and functional

connectivity of neural symptoms supporting cognitive control of emotion in psychosis

proneness. Schizophrenia Research, 118, 88-97. doi: 10.1016/j.schres.2010.01.030

Modinos, Gemma, Renken, Remco, Ormel, Johan, & Aleman, André. (2011). Self-Reflection

and the Psychosis-Prone Brain: An fMRI Study. Neuropsychology, 25(3), 295-305.

doi: 10.1037/a0021747

Mogg, K., & Bradley, B. (2005). Attentional bias in generalised anxiety disorder versus

depressive disorder. Cognitive Therapy & Research, 29, 29-45. doi: 10.1007/s10608-

005-1646-y

Mueser, Kim T., Goodman, Lisa B., Trumbetta, Susan L., Rosenberg, Stanley D., Osher, Fred

C., Vidaver, Robert, . . . Foy, David W. (1998). Trauma and Posttraumatic Stress

Disorder in Severe Mental Illness. Journal of Consulting and Clinical Psychology,

66(3), 493-499.


28

Ochsner, K. N, Bunge, S. A, Gross, J. J., & Gabrieli, J. D. E. (2002). Rethinking Feelings: An

fMRI Study of the Cognitive Regulation of Emotion. Journal of Cognitive

Neuroscience, 14(8), 1215-1229. doi: 10.1162/089892902760807212

Ochsner, K. N, Ray, Rebecca D., Cooper, Jeffrey C., Robertson, Elaine R., Chopra, Sita,

Gabrieli, J. D. E., & Gross, J. J. (2004). For better or for worse: neural systems

supporting the cognitive down- and up-regulation of negative emotion. Neuroimage,

23(2), 483-499. doi: 10.1016/j.neuroimage.2004.06.030

Perala, Jonna, Suvisaari, Jaana, Saarni, Samuli I., Kuoppasalmi, Kimmo, Isometsa, Erkki,

Pirkola, Sami, . . . Lonnqvist, Jouko. (2007). Lifetime prevalence of psychotic and

bipolar I disorders in a general population. Archives of General Psychiatry, 64(1), 19-

28.

Phan, K. Luan, Fitzgerald, Daniel A., Nathan, Pradeep J., Moore, Gregory J., Uhde, Thomas

W., & Tancer, Manuel E. (2005). Neural substrates for voluntary suppression of

negative affect: A functional magnetic resonance imaging study. Biological

Psychiatry, 57(3), 210-219. doi: 10.1016/j.biopsych.2004.10.030

Phillips, L. K., & Seidman, L. J. (2008). Emotion processing in persons at risk for

schizophrenia. Schizophrenia Bulletin, 34(5), 888-903. doi: 10.1093/schbul/sbn085

Phillips, M. L., Drevets, W. C., Rauch, S L., & Lane, R. (2003a). Neurobiology of emotion

perception II: implications for major psychiatric disorders. Biological Psychiatry,

54(5), 515-528. doi: 10.1016/s0006-3223(03)00171-9

Phillips, M. L., Drevets, W. C., Rauch, S. L., & Lane, R. (2003b). Neurobiology of emotion

perception I: the neural basis of normal emotion perception. Biological Psychiatry,

54(5), 504-514. doi: 10.1016/s0006-3223(03)00168-9

Phillips, M. L., & Young, A. W. (1997). A specific neural substrate for perceiving facial

expressions of disgust. Nature, 389, 495-498.

29

Pitskel, N. B., Bolling, D. Z., Kaiser, M. D., Crowley, M. J., & Pelphrey, K. A. (2011). How

grossed out are you? The neural bases of emotion regulation from childhood to

adolescence. Developmental Cognitive Neuroscience, 1(3), 324-337. doi:

10.1016/j.dcn.2011.03.004

Poulton, R., Caspi, A., Moffitt, T. E., Cannon, M., Murray, R., & Harrington, H. (2000).

Children's self-reported psychotic symptoms and adult schizophreniform disorder: a

15-year longitudinal study. Archives of General Psychiatry, 57(11), 1053-1058.

Reiman, E. M, Lane, R. D, Ahern, G. L, Schwartz, G. E, Davidson, Richard J., Friston, K. J, .

. . Chen, K. (1997). Neuroanatomical correlates of externally and internally generated

human emotions. The American Journal of Psychiatry, 154(7), 918-925.

Roberts, Nicole A., Levenson, Robert W., & Gross, J. J. (2008). Cardiovascular costs of

emotion suppression cross ethnic lines. International Journal of Psychophysiology,

70(1), 82-87. doi: 10.1016/j.ijpsycho.2008.06.003

Rössler, W., Hengartner, M. P., Ajdacic-Gross, V., Haker, H., & Angst, J. (2013).

Deconstructing sub-clinical psychosis into latent-state and trait variables over a 30-

year time span. Schizophrenia Research, 150, 197-204. doi:

10.1016/j.schres.2013.07.042

Serper, M., & Berenbaum, H. (2008). The relation between emotional awareness and

hallucinations and delusions in acute psychiatric inpatients. Schizophrenia Research,

101(1-3), 195-200. doi: 10.1016/j.schres.2008.01.012

Shin, Lisa M., Dougherty, Darin D., Orr, Scott P., Pitman, Roger K., Lasko, Mark, Macklin,

Michael L., . . . Rauch, Scott L. (2000). Activation of anterior paralimbic structures

during guilt-related script-driven imagery. Biological Psychiatry, 48(1), 43-50. doi:

10.1016/s0006-3223(00)00251-1

30

Skinner, E. A., & Zimmer-Gembeck, M. J. (2007). The development of coping. Annual

Review of Psychology, 58, 119-144. doi: 10.1146/annurev.psych.58.110405.085705

Smith, B., Fowler, D. G., Freeman, D., Bebbington, P., Bashforth, H., Garety, P., . . .

Kuipers, E. (2006). Emotion and psychosis: links between depression, self-esteem,

negative schematic beliefs and delusions and hallucinations. Schizophrenia Research,

86(1-3), 181-188. doi: 10.1016/j.schres.2006.06.018

Spitzer, C., Siebel-Jurges, U., Barnow, S., Grabe, H. J., & Freyberger, H. J. (2005).

Alexithymia and interpersonal problems. Psychotherapy and Psychosomatics, 74(4),

240-246. doi: 10.1159/000085148

Stefanis, Nicholas C., Os, J. V., Avramopoulos, Dimitrios, Smyrnis, Nikolaos, Evdokimidis,

Ioannis, Hantoumi, Ioanna, & Stefanis, Costas N. (2004). Variation in catechol-o-

methyltransferase val158 met genotype associated with schizotypy but not cognition:

A population study in 543 young men. Biological Psychiatry, 56(7), 510-515. doi:

10.1016/j.biopsych.2004.06.038

Suslow, Thomas, Roestel, Cornelia, Ohrmann, Patricia, & Arolt, Volker. (2003). The

experience of basic emotions in schizophrenia with and without affective negative

symptoms. Comprehensive Psychiatry, 44(4), 303-310. doi: 10.1016/s0010-

440x(03)00085-3

Taylor, G. J, Bagby, R. M, & Parker, J. D. A. (1997). Disorders of Affect Regulation:

Alexithymia in Medical and Psychiatric Illness Cambridge, UK: Cambridge

University Press.

Thornicroft, Graham, Tansella, Michele, Becker, Thomas, Knapp, Martin, Leese, Morven,

Schene, Aart, . . . Group, Epsilon Study. (2004). The personal impact of schizophrenia

in Europe. Schizophrenia Research, 69(2-3), 125-132.

31

Tien, A. Y., & Eaton, W. W. (1992). Psychopathologic precursors and sociodemographic risk

factors for the schizophrenia syndrome. Archives of General Psychiatry, 49, 37-46.

Tremeau, Fabien. (2006). A review of emotion deficits in schizophrenia. Dialogues in

Clinical Neuroscience, 8(1), 59-70.

Turnbull, G, & Bebbington, P. (2001). Anxiety and the schizophrenic process: clinical and

epidemiological evidence. Social Psychiatry and Psychiatric Epidemiology, 36(5),

235-243.

Valkamo, M., Hintikka, J., Honkalampi, K., Niskanen, L., Koivumaa-Honkanen, H., &

Viinamaki, H. (2001). Alexithymia in patients with coronary heart disease. Journal of

Psychosomatic Research, 50(3), 125-130.

van der Meer, L., van't Wout, M., & Aleman, A. (2009). Emotion regulation strategies in

patients with schizophrenia. Psychiatry Res, 170(2-3), 108-113. doi:

10.1016/j.psychres.2009.07.010

van Os, J., Hanssen, M., Bijl, R. V., & Ravelli, A. (2000). Strauss (1969) revisited: a

psychosis continuum in the general population? Schizophrenia Research, 45(1–2), 11-

20. doi: 10.1016/S0920-9964(99)00224-8

van Os, J., Linscott, R. J., Myin-Germeys, I., Delespaul, P., & Krabbendam, L. (2009). A

systematic review and meta-analysis of the psychosis continuum: evidence for a

psychosis proneness-persistence-impairment model of psychotic disorder.

Psychological Medicine, 39(2), 179-195.

van Rijn, S., Schothorst, P., Wout, M., Sprong, M., Ziermans, T., van Engeland, H., . . .

Swaab, H. (2011). Affective dysfunctions in adolescents at risk for psychosis:

emotion awareness and social functioning. Psychiatry Research, 187(1-2), 100-105.

doi: 10.1016/j.psychres.2010.10.007

32

Vanheule, S., Desmet, M., Meganck, R., & Bogaerts, S. (2007). Alexithymia and

interpersonal problems. Journal of Clinical Psychology, 63(1), 109-117. doi:

10.1002/jclp.20324

Wenzlaff, Richard. M, & Luxton, David. D. (2003). The Role of Thought Suppression in

Depressive Rumination. Cognitive Therapy & Research, 27(3), 293-308. doi:

10.1023/A:1023966400540

Westermann, S., Kesting, M., & Lincoln, T. M. (2012). Being deluded after being excluded?

How emotion regulation deficits in paranoia-prone individuals affect state paranoia

during experimentally induced social stress. Behaviour Therapy, 43, 329-340. doi:

10.1016/j.beth.2011.07.005

Westermann, S., & Lincoln, T M. (2011). Emotion regulation difficulties are relevant to

persecutory delusion. Psychology and Psychotherapy: Theory, Research and

Practice., 84, 273-287. doi: 10.1348/147608310X523019

Wiles, Nicola J., Zammit, Stanley, Bebbington, Paul, Singleton, Nicola, Meltzer, Howard, &

Lewis, Glyn. (2006). Self-reported psychotic symptoms in the general population:

results from the longitudinal study of the British National Psychiatric Morbidity

Survey. British Journal of Psychiatry, 188, 519-526.

World Health Organisation. (2001). The World Health Report 2001 : Mental health: new

understanding, new hope. Geneva: World Health Organisation.

Yung, A. R., & McGorry, P. D. (1996). The prodromal phase of first-episode psychosis: past

and current conceptualizations. Schizophrenia Bulletin, 22, 353-370.

Yung, A. R., Nelson, B., Baker, K., Buckby, J. A., Baksheev, G., & Cosgrave, E. M. (2009).

Psychotic-like experiences in a community sample of adolescents: implications for the

continuum model of psychosis and prediction of schizophrenia. The Australian and

33

New Zealand Journal of Psychiatry, 43(2), 118-128. doi:

10.1080/00048670802607188

34

1.11 Foreword to Chapters 2 and 3.

Before evaluating the presence of emotion processing difficulties in people with

psychotic-like experiences (PLE), it was important to validate the emotion processing scales

in this study that have not been extensively validated (the Emotion Regulation Questionnaire;

ERQ, and the Emotion Processing Scale; EPS). Establishing the psychometric properties of

measures is essential before we can have any confidence in inferences based on a scale’s

scores (Nunnally, 1967). For this reason, Chapter 2 is dedicated to assessing the psychometric

properties of the ERQ, whilst Chapter 3 focusses on the assessment of the EPS.

35

Chapter 2: The Emotion Regulation Questionnaire: validation of the ERQ-9 in two

community samples

Abstract

The 10-item Emotion Regulation Questionnaire (ERQ) was developed by Gross and John

(2003) to measure the habitual use of two emotion regulation strategies –reappraisal and

suppression. Several studies using student samples have provided validation for the ERQ,

although the only paper (Wiltink et al. 2011) that evaluated the ERQ in a community sample

was unable to replicate the original factor structure. Before using the ERQ in non-student

samples it is important to validate the scale in a sample broadly representative of the adult

population and determine the influence of demographic variables. The current study

examined the psychometric properties of the ERQ in two community samples (Australia, N =

550; United Kingdom, N = 483, aged 17-95) using confirmatory analysis. The original ERQ

factor structure was not supported by either the Australian or UK samples. However, with the

removal of one item, a strong model fit was obtained for both samples (9-item ERQ). Using

measurement invariance tests, the revised 9-item ERQ was found to be equivalent across the

samples and demographics (age, gender, education). Gender, depression, anxiety and stress

(DASS) were the only factors which were significantly associated with reappraisal and

suppression use. Overall, the ERQ-9 provides better fit of the data than the ERQ-10. The

utility of this measure is enhanced by the provision of normative data for males and females.

Keywords: Emotion Regulation, Questionnaire, Processing, Validation, Psychometric

Properties

36

Spaapen, D. L., Waters, F., Brummer, L., Stopa, L., & Bucks, R. S. (2014) The Emotion

Regulation Questionnaire: validation of the ERQ-9 in two community samples.

Psychological Assessment. 26. 46-54. (Chapter 2).

Emotion regulation is a key mechanism for our survival, and is at the core of

successful social interactions. Emotion regulation is a controlled process that is used to

change a person’s spontaneous emotional response (Koole, 2009). Indeed, it is required to

attain desired affective states and intentional goals (Gross, 2002). Dysfunctional cognitive

emotion regulation is linked to increased psychopathology and negative affect (Aldao et al.,

2010; Beevers & Meyer, 2004; Matheson & Anisman, 2003; Wenzlaff & Luxton, 2003),

decreased quality of life and well-being (Gross & John, 2003), impaired memory (Nielson &

Lorber, 2009; Richards & Gross, 2000), and increased sympathetic nervous system activity

(Demaree et al., 2006; Gross, 1998a).

There are several ways people may modulate their emotions, commonly referred to as

emotion regulation strategies (Gross & John, 2003). Contemporary models propose that, at

the broadest level, emotion regulation strategies are distinguished by whether they are

‘antecedent’ or ‘response-focused’ (Gross & John, 2003), referring to when these cognitive

events occur along the timeline of information processing. The use of antecedent and

response-focussed strategies are considered to differ in their consequences to health and well-

being (Gross, 1998a). To examine these differences, studies commonly compare two types of

emotion regulation strategies, cognitive reappraisal (antecedent) and expressive suppression

(response-focussed) (Gross & John, 2003).

The Emotion Regulation Questionnaire (ERQ; Gross & John, 2003) was originally

designed to measure the habitual use of reappraisal and suppression strategies. It is now a

widely accepted and commonly used measure of emotion regulation. ERQ studies have

37

consistently found that habitual suppression is associated with increased depression, anxiety,

and stress symptoms (negative affect), while habitual reappraisal is associated with greater

positive emotion and less negative emotion (Gross & John, 2003; Joormann & Gotlib, 2010;

Moore, Zoellner, & Mollenholt, 2008; Wiltink et al., 2011). Compared to the use of

suppression, reappraisal is also associated with better interpersonal functioning, positive well-

being (Gross & John, 2003), better social adjustment and decision making (Heilman, Crisan,

& Houser, 2010; Magar, Phillips, & Hosie, 2008). The ERQ is therefore useful in identifying

groups vulnerable to negative physical and psychological outcomes, as well as factors with an

important influence on emotional wellbeing.

When Gross and John (2003) first developed the 10-item ERQ scale, a principal-

components analysis (PCA) on the scores of 1483 students (across four samples) revealed a

2-factor solution. Together, the two factors (represented by reappraisal and suppression)

accounted for over 50% of the variance in all four samples. Further studies in student samples

have replicated this 2-factor solution using exploratory (Chen, 2010; D'Argembeau & Van

der Linden, 2006) and confirmatory factor analyses (Balzarotti, Gross, & John, 2010;

Matsumoto, Yoo, Nakagawa, & Altarriba, 2008; Melka, Lancaster, Bryant, & Rodriguez,

2011). Few studies, however, have examined the psychometric properties of the ERQ outside

student samples. Two exceptions include Moore et al. (2008), who, in addition to a student

sample (N = 289) recruited a small (N = 67) all-female, trauma-exposed group and Wiltink et

al. (2011) who tested a German sample using a German translation of the ERQ (Abler &

Kessler, 2009) (N = 2524, 55.5% female).

Moore et al. (2008) found an overall good fit using the original ERQ factor structure

when combining their student and community samples. Their results however, may have been

heavily skewed by the inclusion of both a student sample (N = 289), and a clinical group, in

the analysis. In contrast, Wiltink et al. (2011) failed to replicate the original ERQ factor

38

structure. Instead, they proposed a modification of the scale in which item 8 (I control my

emotions by changing the way I think about the situation I’m in) was allowed to load onto

both reappraisal and suppression factors equally, and was freed from equality constraint with

other item factor loadings. This raises the question whether this deviation in factor structure

stems from cultural differences, the use of a German translation of the original scale, or

demographic differences between student and community samples (e.g. age, education).

These questions are important, because the ERQ is increasingly used in research with non-

undergraduate samples (Henry et al., 2008; Perry, Henry, & Grisham, 2011), when it has not

yet been validated for such use. Establishing the psychometric properties of an instrument is

an important step in assessing the responsiveness of measurement instruments, particularly

for measures that are frequently used.

The primary objectives of the current study were to test the original two factor

structure fit of the ERQ (Gross & John, 2003) on two large community samples (Australian

and United Kingdom), and to determine the influence of demographic variables. The focus

was on variables which are generally homogeneous in student samples – age and education,

as well as other variables such as gender, depression, anxiety and stress.

There are mixed findings in the literature about the relationship between emotion

regulation strategies and demographic variables. With regard to age, some studies show a link

between ageing and a decrease in negative emotions and an increase in positive emotions

(Gross et al., 1997; Helson & Klohnen, 1998). According to some theories (including the

Socioemotional Selectivity Theory, SST; Charles & Carstensen, 2007), this may be due to the

increased use of antecedent strategy in older adults, specifically selective attention, situation

selection and situation modification to modulate emotion. In conjunction with this theory,

Gross (1998b) suggests this increased use of antecedent strategies also includes cognitive

reappraisal. This is consistent with results from Folkman, Lazarus, Pimley & Novacek’s

39

(1987) study, which found that older participants reported greater positive reappraisal and

distancing, and less confrontational coping, compared to younger participants. These results,

however, stand in contrast to Wiltink et al.’s (2011) German study, in which age was not a

significant predictor of reappraisal or suppression use, after controlling for other demographic

and clinical variables (e.g. gender, depression, and anxiety).

There is limited research into how education is related to emotion regulation strategy

use. Wiltink et al. (2011) found that individuals with a tertiary degree reported lower

suppression scores (but not reappraisal) compared to those without. In contrast, Moore et al.

(2008) did not find any relationship between education and reappraisal or suppression.

However Moore et al.’s sample was largely homogenous with respect to education, possibly

explaining the lack of significant association. Further research using an independent

community sample is needed to clarify this relationship.

The association between the 10-item ERQ, gender differences, and clinical symptoms

has been examined previously. Studies show that males report a greater use of expressive

suppression compared to females (Balzarotti et al., 2010; Chen, 2010; Gross & John, 2003;

Melka et al., 2011; Wiltink et al., 2011). For reappraisal, no gender differences have been

observed in any ERQ validation studies, with the exception of Chen (2010), who found that

female students use reappraisal more often than males.

As mentioned above, with regards to psychological symptoms, individuals who

habitually use reappraisal tend to have lower levels of depression and anxiety (with greater

positive functioning e.g. life satisfaction), whereas individuals who habitually use

suppression show the reverse pattern with decreased positive functioning (Gross & John,

2003; Wiltink et al., 2011).

In summary, this study a) examined the psychometric properties of the ERQ in two

separate community samples (Australia, N = 550; United Kingdom, N = 483, age 17-95), b)

40

conducted invariance tests by demographic variables, and c) examined the relationships

between the ERQ scales, demographics and clinical variables. We hypothesised that habitual

reappraisal would be associated with reduced psychological symptoms, and be more

prevalent in older individuals. In contrast, we predicted that suppression would be associated

with increased psychological symptoms, and show greater prevalence in males, younger, and

less (formally) educated individuals.

2.1 Method

2.1.1 Participants

Participants were drawn from Australian (AUS) and United Kingdom (UK)

community samples (aged 17-95). Sample characteristics are presented in Table 1.

Questionnaires were distributed via broad community advertising which asked for volunteers

to take part in an “Emotions across the Lifespan” study. Volunteers were given the choice to

complete the questionnaires online, or on hardcopy mailed out with a reply-paid envelope.

This approach was considered acceptable due to evidence that online and printed versions of

questionnaires give similar results (Ritter, Lorig, Laurent, & Matthews, 2004). Participation

was voluntary and completion of the questionnaires was taken to indicate consent. This

project was granted approval from the UWA Human Research Ethics Committee (Project No

RA/4/3/1247) and from the University of Southampton School of Psychology-Research

Ethics Committee (Project No ST/03/90).

Table 1.

Sample Characteristics

Items AUS UK Total

41

M SD Range M SD Range M SD Range

Sample

Size 550 483 1033

Gender (N,

%)

F: 351,

65.7%

M: 183,

34.3%

F: 380,

79.8% M: 96, 20.2%

F: 731,

72.4%

M: 279,

27.6%

Age (years) 39.1 20.4 17 - 95 39.2 20.7 18 - 91 39.1 20.5 17-95

Education

(years) 13.5 3.1 4 - 27.2 15 3.1 6 - 25 14.2 3.2

4 -

27.2

DASS -

Depression 8.8 8.7 0 – 40 8.7 9.0 0 – 42 8.8 8.8 0 – 42

DASS –

Anxiety 6.7 7.6 0 – 42 6.1 7.4 0 – 42 6.4 7.5 0 – 42

DASS -

Stress 13.3 9.2 0 - 42 13.1 9.2 0 – 42 13.2 9.2 0 – 42

Ethnicity

Australian

Aboriginal/Torres

Strait Islander

15

(2.7%) White British

419

(86.8%)

Western

European

219

(39.8%) Irish

35

(7.2%)

Southern or

Eastern European

34

(6.2%)

Indian/Indian

sub-

continent/Mixed

9

(1.9%)

Asian 74

(13.5%)

Black

Carribean/Mixed

4

(0.8%)

African 8

(1.5%) Other

11

(2.3%)

Other 164

(29.8%) Declined to say

5

(1.0%)

Declined to say 36

(6.5%)

42

Note. DASS (Lovibond & Lovibond, 1995)

2.1.2 Materials

The Emotion Regulation Questionnaire (ERQ; Gross & John, 2003) is a 10-item,

self-report measure of habitual expressive suppression (4 items) and reappraisal (6 items).

The ERQ uses a 7-point Likert scale ranging from 1 (strongly disagree) to 7 (strongly agree).

Example questions include “I control my emotions by changing the way I think about the

situation I’m in” (reappraisal) and “I control my emotions by not expressing them”

(suppression). Internal consistency has been adequate and consistent across studies

(Balzarotti et al., 2010; Gross & John, 2003; Melka et al., 2011; Wiltink et al., 2011).

The Depression, Anxiety and Stress Scale 21 (DASS-21) is a shortened version of

Lovibond and Lovibond’s (1995) 42-item self-report measure of depression, anxiety, and

stress (DASS). Previous studies have found the DASS-21 yields satisfactory reliability

estimates (Antony, Bieling, Cox, Enns, & Swinson, 1998; Clara, Cox, & Enns, 2001; Henry

& Crawford, 2005).

2.1.3 Data analysis plan

AMOS 20 (Arbuckle, 2011) was used to conduct the CFA. The current study’s

multivariate critical ratio (22.97), which calculates overall non-normality across all items,

exceeded Bentler’s (2005) recommended critical value of 5, suggesting that the data were not

normally distributed. To address this issue, asymptotic distribution free (ADF) estimation

was adopted for single group CFAs. The ERQ’s factor structure was examined using chi-

square, the Standardised Root Mean-square Residual (SRMR; Jöreskog & Sörbom, 1993) the

comparative fit index (CFI; Bentler, 1990), Tucker-Lewis Index (TLI; Tucker & Lewis,

1973), and root mean square error of approximation (RMSEA; Steiger & Lind, 1980). Chi-

43

square non-significance, SRMR values below .08, TLI and CFA values above .95 and

RMSEA values below .06 are recommended (Hu & Bentler, 1999). In addition, modification

indices (M.I) and parameter estimates were examined as they provide more specific

information about model misfit than the standard measure of overall goodness of fit (Brown,

2006). As recommended by Jöreskog and Sörbom (1993), we examined the items with the

highest M.I and, if there was a theoretical, conceptual or practical reason for doing so, we

removed that item (see Brown, 2006).

For demographic invariance testing, calculations of effects size were used instead of

measures of statistical significance. This was chosen because when dealing with such large

sample sizes (1000+), the chance of finding non-equivalence even when there is equivalence

(Type-I error), is almost a certainty when using statistical significance tests. In these

circumstances of extreme statistical power, it is more appropriate to assess the practical

significance (effect size) of the non-equivalence tests. The calculations of effect size (by

item) were generated by Mean and Covariance Structure Analysis (MACS) using a program

(dMACS) freely available online - see Nye and Drasgow (2011). Measurement invariance

(by statistical significance) was assumed if the change in CFI (∆CFI < .002) was below the

cut-off (Meade, Johnson, & Braddy, 2008). Effect size was compared to Cohen’s (1988)

guidelines (.20 = small, .50 = medium, .80 = large). For analyses of the relationship between

ERQ subscales and gender, a multiple analysis of variance (MANOVA) was used.

Correlation analysis was used to examine the relationships between demographic variables

(age and education), clinical variables (depression, anxiety and stress) and the ERQ

subscales. Missing data by variable ranged from .2 to 3.7%. Little's MCAR revealed that the

data were missing completely at random, χ²(346, N = 1033) = 379.19, p = .106. Missing data

were handled using pair-wise deletion.

44

2.2 Results

2.2.1 Factor Structure

Results of the AUS and UK single-group CFAs suggested barely adequate fit (see

Table 2). A significant chi-square, coupled with χ²/df, TLI and CFI statistics beyond the

recommended cut-off values, suggests that the original two factor structure of the ERQ was

not an adequate representation of the response behaviour observed either in the Australian or

the United Kingdom community samples.

The M.I revealed high covariance between Items 1 and 3 in both samples (AUS: M.I

= 19.294, Parameter Change = .374; UK: M.I = 9.272, Parameter Change = .225). Arguably,

both items tap very similar aspects of reappraisal. They both refer to changing one’s thoughts

to increase positive (Item 1 – When I want to feel more positive emotion (such as joy or

amusement), I change what I’m thinking about) or decrease negative (Item 3 – When I want

to feel less negative emotion (such as sadness or anger), I change what I’m thinking about)

emotion experience. Item 3 was dropped due to a lower factor loading, and the finding that

negatively worded items can introduce method effects (DiStefano & Motl, 2006).

Computation after removing Item 3 (revised 9-item ERQ) produced strong model fit for both

the Australian and United Kingdom samples. All indicator values were within the acceptable

range, with the exception of the chi-square statistic. However, the chi-square statistic is

known to be inflated by large samples and skewed distributions, increasing the possibility of

rejecting an otherwise good fitting model. Based on all other fit statistics, the revised 9-item

ERQ measure is a strong model for both samples.

Table 2.

Single group CFA – Australian and United Kingdom samples

Model Df χ² Sig. χ²/df SRMR TLI CFI RMSEA LO 90 HI 90

45

Model Df χ² Sig. χ²/df SRMR TLI CFI RMSEA LO 90 HI 90

Single group solutions

Australia (10-item) 34 79.396 <.001* 2.335 .060 .890 .917 .049 .035 .064

United Kingdom (10-

item)

34

75.023

<.001*

2.207

.062 .850 .887 .050 .035 .065

Australia (9-item) 26 52.152 .213 1.208 .033 .984 .989 .019 0 .041

United Kingdom (9-

item)

26

41.712

.026*

1.604

.048 .936 .954 .035 .012 .055

Note. SRMR = standardised root mean-square residual, RMSEA = root mean square error of

approximation, CFI = comparative fit index, TLI = Tucker-Lewis Index, LO 90 and HI 90 =

lower and upper boundary of 90% confidence interval for RMSEA. Asymptoticallly

distribution-free estimation (ADF) was used.

Coefficients for reappraisal, before (AUS: Cronbach’s α = .79, UK: α = .82) and after

(AUS: Cronbach’s α = .76, UK: α = .80) Item-3 removal, and suppression (AUS: α = .78,

UK: α = .74) indicate adequate internal consistency of test scores for both subscales.

2.2.2 Between Samples Invariance Testing

Measurement invariance (MI) analysis examines the extent to which a scale measures

the same construct/s across groups. The 9-item ERQ was found to be equivalent across the

AUS and UK samples when the factor loadings, mean and intercepts were constrained

(metric and scalar invariance). Results are presented in Table 3. As strong, between-group

invariance was established between the samples, it was considered viable to combine the

samples for subsequent analyses.

Table 3.

Multi-group analysis (9-item) - invariance test between Australian and United Kingdom

community samples.

46

Model df χ² Sig. χ²/df SRMR TLI CFI RMSEA LO 90 HI 90

Unconstrained 56 107.308 <.001* 2.064 .030 .973 .980 .032 .023 .041

Equal factor loading 49 113.897 <.001* 1.930 .031 .976 .980 .030 .022 .038

Equal

means/intercepts

40

128.041

<.001* 1.883

.031 .977 .979 .029 .021 .037

Equal covariance 37 134.961 <.001* 1.901 .037 .977 .977** .030 .022 .037

Equal Error 28 174.793 <.001* 2.185 .038 .970 .966** .034 .027 .041

Note. * = p < .05, ** = ∆CFI > .002. Models: Unconstrained = the standard 9-item model,

SRMR = standardised root mean-square residual, RMSEA = root mean square error of


lower and upper boundary of 90% confidence interval for RMSEA. Maximum Likelihood

(ML) estimation was used because it is a more thorough method to examine the extent of

invariance (compared to Asymptotic Distribution Free - ADF).

In summary, data from our AUS and UK community samples did not adequately fit

the original 10-item, two-factor structure, ERQ (Gross & John, 2003). However, a revised, 9-

item ERQ was a good fit for both samples. These findings raise the question, why do the

current community samples differ from the many student samples previously used

successfully to validate the original 10-item ERQ? One hypothesis is that variations in

demographic variables (e.g. age, education and gender) between student and community

samples influence the model.

2.2.3 Demographic Invariance Testing

To examine the effect of demographics on the ERQ-9 model fit, age and education

were divided into two groups (both by median split)1. Age was divided into younger (17 – 29

1In response to a reviewer’s comments regarding median-split analyses, we re-analysed the demographic invariance tests for age and education, increasing the number of groups (Age – quintile split – groups: 17 – 20, 21 – 25, 26 – 41, 42 – 61, 62 – 95 years; Education - high school (Years 4 – 12), undergraduate/tertiary (Years

47

years) and older (30 – 95 years) age groups: Education into lower (4 – 13.5) and higher (14 –

27.2) years of education. Separate invariance tests were conducted comparing age, education

and gender (see Table 4). For all invariance tests, all indices of fit exceeded critical values

(statistically significant). All measures of effect size (by item) for age (.09 - .49), education

(.06 - .39) and gender (.05 - .38) were small to medium, however the factor-level effect size

due to differential item functioning (DIF) for demographics (.04 to .17) was relatively small.

Given the general purpose of the ERQ as a research tool, this level of invariance is

considered acceptable. Therefore, the demographic variables are considered to have

equivalence on the ERQ, allowing for subsequent analysis.

Table 4.

Multi-group analysis (9-item ERQ) - invariance test for Age, Education and Gender with the

combined Australian and United Kingdom community sample.


Age

Unconstrained 52 121.913 <.001* 2.344 .032 .965 .975 .036 .028 .045

Equal factor

loadings 59 133.145

<.001* 2.257

.037 .968 .973 .035 .027 .043

Equal

means/intercepts

68

157.736

<.001* 2.320

.036 .966 .968 .036 .029 .043

Equal covariance 71 160.223 <.001* 2.257 .040 .968 .968 .035 .028 .042

Equal Error 80 174.617 <.001* 2.183 .039 .970 .966 .034 .027 .041

Education

Unconstrained 52 122.978 <.001* 2.365 .038 .965 .974 .037 .029 .045

Equal factor

loadings 59 139.7314

<.001* 2.361

.042 .965 .971 .037 .029 .045

Equal 68 168.728 <.001* 2.481 .042 .962 .964 .038 .031 .046

12.5 – 16), and postgraduate (Years 16.5 – 27.25). Findings were comparable to the median-split findings. Details of these additional analyses are available upon request.

48


means/intercepts


Equal Error 80 210.497 <.001* 2.631 .044 .958 .953 .040 .034 .047

Gender

Unconstrained 52 121.109 <.001* 2.329 .049 .965 .974 .036 .028 .045

Equal factor

loadings 59 130.884

<.001* 2.218

.053 .968 .973 .035 .027 .043

Equal

means/intercepts

68

165.679

<.001* 2.436

.053 .962 .964 .038 .030 .045


Equal Error 80 201.935 <.001* 2.524 .057 .959 .955 .039 .032 .046

Note. SRMR = standardised root mean-square residual, RMSEA = root mean square error of



(ML) estimation was used. Maximum Likelihood (ML) estimation was used because it is a

more thorough method to examine the extent of invariance (compared to asymptotic

distribution free - ADF).

2.2.4 The 9-item ERQ, Demographics, and Negative Affect

Spearman’s Rho correlations were conducted to analyse the relationships between

age, education, the ERQ scales, and negative affect (see Table 5). Age was not significantly

associated with either ERQ scale. However, there was a weak, negative correlation between

age and all measures of negative affect. A very weak, negative association was observed

between education and suppression. In contrast, education was not related to measures of

negative affect. Very weak, negative relationships were observed between reappraisal and

depression, anxiety and stress. In contrast, very small to small positive relationships were

observed between suppression and all measures of negative affect

49

Table 5.

Correlations between ERQ, Demographics, and Negative Affect – Combined Sample

Note. * p < .05** p < .01 (2-tailed). Spearman’s rho correlation coefficient. Education =

total years of education; Depression, Anxiety and Stress - DASS (21 item) short version

(Lovibond & Lovibond, 1995); Reappraisal (5 items) and Suppression (4 items) - ERQ (9 –

item) (Gross & John, 2003).

A MANOVA was conducted using reappraisal and suppression as dependent

variables, and gender as the independent variable. A significant omnibus effect was observed

for gender, F (2, 1007) = 10.00, p < .001, partial η2 = .02. Inspection of the univariate tests by

gender revealed a significant effect for reappraisal, F (1, 1008) = 4.534, p <.05, partial η2 <

.01 and suppression, F (1, 1008) = 16.50, p < .001, partial η2 = .02. Males (M = 14.2, SD =

5.1) reported using expressive suppression more than females (M = 12.7, SD = 5.0).

Conversely, females (M = 25.0, SD = 5.2) reported higher reappraisal scores than males (M =

24.3, SD = 5.6).

Population based norms for the 9-item ERQ are provided in Table 6. Norms were

calculated using percentiles (cumulative percentages), and presented separately for gender

because it was the only demographic variable significantly associated with both ERQ scales.

Measure

Education

Reappraisal (5-item)

Reappraisal

Suppression

Depression

Anxiety

Stress

Age -.149** .054 .043 -.014 -.223** -.260** -.241**

Education - .023 .021 -.109** -.061 -.053 .022

Reappraisal (5-item) - .973** -.098** -.136** -.087** -.086**

Reappraisal - -.091** -.126** -.069* -.065*

Suppression - .256** .157** .118**

Depression - .538** .611**

Anxiety - .617**

50

Table 6.

Population based norms of the 9-item ERQ by Gender (N = 1010) from Australian and

United Kingdom samples.

Score Reappraisal

(cumulative %)

Suppression

(cumulative %)

Female Male Female Male

4 2.5 2.9

5 0.1 0.4 5.2 5.4

6 10.0 6.8

7 0.7 14.6 10.8

8 21.3 15.8

9 0.8 1.1 28.0 20.1

10 1.1 2.2 36.3 28.0

11 1.6 3.2 44.6 32.6

12 2.1 3.9 51.6 36.9

13 2.9 5.4 58.4 43.4

14 4.5 6.5 66.1 50.2

15 5.2 8.6 71.7 57.7

16 6.7 9.0 76.6 63.8

17 8.1 10.4 82.6 71.3

18 10.3 12.5 86.7 78.1

19 13.0 15.1 90.2 83.5

20 17.4 22.2 92.2 89.2

51

21 21.2 29.0 94.9 94.3

22 26.7 34.8 96.7 95.7

23 32.7 38.4 97.5 97.1

24 41.7 45.2 98.4 98.9

25 51.4 55.2 99.2

26 59.8 65.2 99.3 100

27 66.6 73.8 99.5

28 72.4 79.6 100

29 79.8 84.2

30 87.7 88.2

31 92.2 91.0

32 94.9 93.5

33 96.3 95.7

34 97.5 98.2

35 100 100

52

2.3 Discussion

2.3.1 CFA properties

The present study presents, for the first time, an examination of the psychometric

properties of the original 10-item ERQ in samples broadly representative of the adult

community population. Another strength of our study included independent validation of the

ERQ in two English-speaking community samples (AUS and UK). The results showed that

the original model obtained by Gross and John (2003) did not present an adequate fit for

either community sample. The inadequate model fit may be due to minor problems with the

original factor structure. Given the high covariance between Items 1 and 3 in both AUS and

UK samples, one (Item 3) was removed as there appeared to be redundancy. This produced a

substantial improvement in model fit, resulting in a revised 9-item ERQ.

It is noteworthy that the other community study using a German translation of the

scale (Wiltink et al., 2011) also reported inadequate model fit compared to Abler & Kessler’s

(2009) student study. In contrast to the current findings, however, the revised ERQ factor

solution by Wiltink et al. (2011) involved modifications to Item 8, requiring it to cross-load

equally on the reappraisal and suppression factors, where cross-loading is not to be

recommended (Brown, 2006).

To validate the revised 9-item model, an invariance test comparing both AUS and UK

samples was conducted. Findings suggest the revised 9-item model is valid and equivalent

across countries. Matsumoto et al. (2008) reported a 23 country-level CFA analysis and also

found equivalence. Whilst ethnically diverse, the current samples were insufficient in size (by

ethnicity groups) for invariance testing to be conducted. However, Melka et al. (2011)

examined invariance by ethnicity in a US sample and showed equivalence. Given the failure

to find either country-level differences in both this study and Matsumoto et al. (2008), and

53

the evidence of a lack of ethnicity differences in Melka et al. (2011), this suggests to us that

Wiltink et al.’s (2011) German study may have suffered from translation effects.

To further examine the external validity of the 9-item ERQ, we explored whether

differences in demographics could be responsible for influencing model fit. Item-level

measures of effect size for non-equivalence for all items by age, education and gender were

small to medium, but at the scale-level the influence of DIF was small. That is, the effect of

this non-equivalence was minimal. Thus, the 9-item ERQ was considered to be stable across

age, education and gender. This allowed us to explore whether age, education and gender

differences were present in reappraisal and suppression assessed with the 9-item ERQ.

2.3.2 Age differences

Contrary to expectation, no significant age differences were observed for reappraisal

or suppression strategy use. This stands in contrast to the literature that suggests that older

people shift towards using more antecedent strategies (including reappraisal) than their

younger counterparts (Charles & Carstensen, 2007; Folkman, Lazarus, Pimley, & Novacek,

1987; Gross et al., 1997). Differences between studies may be due to differences in

methodologies, sample characteristics, or methods of analysis. For example, the differences

between the current study and that of Folkman et al. (1987) could be due to a different

measure of reappraisal (a 1-item measure in Folkman et al.’s study) or sample characteristics

(only married couples of middle to older age). The current results, however, are consistent

with Wiltink et al. (2011) who also found no age differences.

2.3.3 Education differences

Similarly to age, and again not as predicted, there were no differences in reappraisal

and suppression use based on level of education. These findings are consistent with Moore et

54

al. (2008), in a student sample using the ERQ-10 but divergent from Wiltink et al.’s (2011)

community study which indicated that people with a college/university (tertiary) degree

reported lower suppression scores (but not reappraisal) compared to those without. Again,

translation effects may account for this.

2.3.4 Gender differences

Males reported greater use of suppression compared to females: a finding consistent

with previous studies (Balzarotti et al., 2010; Gross & John, 2003; Matsumoto et al., 2008;

Melka et al., 2011; Wiltink et al., 2011). The current study also found that women reported

higher reappraisal scores compared to males. This finding is in contrast to most previous

ERQ studies – which failed to find gender effects in reappraisal (only Chen, 2010 found

higher reappraisal in females). Population norms using our community sample were thus

created for the 9-item ERQ based on gender alone (Table 6), as no other demographics were

related to ERQ scale scores. It is important to note that until these findings are replicated in

other countries (e.g. US samples), these norms may only be suitable for Australian and

United Kingdom samples.

2.3.5 Negative affect

Depression, anxiety and stress were all positively associated with suppression. This

finding is consistent with previous community (Wiltink et al., 2011) and student (Abler &

Kessler, 2009; Gross & John, 2003) studies, and other studies using comparable constructs

(emotional containment) (e.g. Matheson & Anisman, 2003). For reappraisal, there was a

significant negative association with all measures of negative affect. This finding underscores

that of Wiltink et al. (2011), who found a negative association between reappraisal and

negative affect (depression and anxiety). It also shadows previous research showing a link

55

between suppression and negative affect, and reappraisal with greater positive affect (Gross

& John, 2003; Wiltink et al., 2011).

In summary, the original 10-item ERQ did not provide good fit for Australian and

United Kingdom community samples, which suggests that the scale is not ideal in its original

form. However, the minor modification of removing one item produced a significant

improvement and strong model fit. This revised 9-item ERQ was equivalent across countries

(Australia and UK), across age, education and gender, suggesting that the revised measure is

preferable when measuring reappraisal and suppression in community samples. Exploration

of ethnicity effects is warranted, as is establishing test-retest reliability, predictive and

concurrent validity of the ERQ-9.

56

2.4 References

Abler, B, & Kessler, H. (2009). Emotion regulation questionnaire - Eine deutsche version des

ERQ von gross & john. Diagnostica, 55, 144-152. doi: DOI: 10.1026/0012-

1924.55.3.144



30(2), 217-237. doi: 10.1016/j.cpr.2009.11.004

Antony, M. M., Bieling, P. J., Cox, B. J., Enns, M. W., & Swinson, R. P. (1998).

Psychometric Properties of the 42-Item and 21-Item Versions of the Depression

Anxiety Stress Scales in Clinical Groups and a Community Sample. Psychological

Assessment, 10(2), 176-181. doi: 10.1037/1040-3590.10.2.176

Arbuckle, J. L. (2011). IBM SPSS Amos 20 Users' Guide. Chicago, IL: Smallwaters

Corporation.

Balzarotti, S., Gross, J. J., & John, O. P. (2010). An Italian Adaptation of the Emotion

Regulation Questionnaire. European Journal of Psychological Assessment, 26(1), 61-

67. doi: 10.1027/1015-5759/a000009


Emotion, 18(6), 859-867. doi: 10.1080/02699930341000220

Bentler, P. M. (1990). Comparative fit indexes in structural models. Psychological Bulletin,

107, 238-246. doi: 10.1037/0033-2909.107.2.238

Brown, T. A. (2006). Confirmatory factor analysis for applied research. New York: Guilford

Publications, Inc.

Charles, S. T., & Carstensen, L. L. (2007). Emotion regulation and aging. In J. J. Gross (Ed.),

Handbook of Emotion Regulation (pp. 307-330). New York: Guilford Press.

57

Chen, Z. (2010). Assessing two emotion regulation processes in Chinese adolescents. Paper

presented at the IEEE International Conference on Advanced Management Science

(ICAMS)

Clara, I. P., Cox, I. P., & Enns, Murray W. (2001). Confirmatory factor analysis of the

depression anxiety stress scales in depressed and anxious patients. Journal of

Psychopathology and Behavioral Assessment, 23, 61-67. doi:

10.1023/A:1011095624717

Cohen, J. (1988). Statistical Power Analysis for the Behavioural Sciences (2nd ed ed.).

Hillsdale, NJ: Erlbaum.

D'Argembeau, A., & Van der Linden, M. (2006). Individual differences in the

phenomenology of mental time travel: The effect of vivid visual imagery and emotion

regulation strategies. Conscious Cogn, 15(2), 342-350. doi:

10.1016/j.concog.2005.09.001

Demaree, H. A., Schmeichel, B. J., Robinson, J. L., Pu, J., Everhart, D. E., & Berntson, G. G.

(2006). Up- and down-regulating facial disgust: Affective, vagal, sympathetic, and

respiratory consequences. Biological Psychology, 71(1), 90-99. doi:

10.1016/j.biopsycho.2005.02.006

DiStefano, C., & Motl, W. R. (2006). Further investigating method effects associated with

negatively worded items on self-report surveys. Structural Equation Modeling: A

Multidisciplinary Journal, 13(3), 440-464. doi: 10.1207/s15328007sem1303_6

Folkman, S., Lazarus, R. S., Pimley, S., & Novacek, J. (1987). Age differences in stress and

coping processes. Psychology and Aging, 2(2), 171-184. doi: 10.1037/0882-

7974.2.2.171

58

Gross, J. J. (1998a). Antecedent- and response-focussed emotion regulation: divergent

consequences for experience, expression, and physiology. Journal of Personality and


Gross, J. J. (1998b). The Emerging Field of Emotion Regulation: An Integrative Review.

Review of General Psychology, 2(3), 271-299. doi: 10.1037/1089-2680.2.3.271



Gross, J. J., Carstensen, L. L., Pasupathi, M., Skorpen, C. G., Tsai, J., & Hsu, A. Y. (1997).

Emotion and aging: experience, expression and control. Psychology and Aging, 12(4),

590 - 599. doi: 10.1037/0882-7974.12.4.590




Heilman, R. M., Crisan, L. G., & Houser, D. (2010). Emotion Regulation and Decision

Making Under Risk and Uncertainty. Emotion, 10(2), 257-265. doi:

10.1037/a0018489

Helson, R., & Klohnen, E. C. (1998). Affective Coloring of Personality from Young

Adulthood to Midlife. Personality and Social Psychology Bulletin, 24(3), 241-252.

doi: 10.1177/0146167298243002

Henry, J. D., & Crawford, J. R. (2005). The short-form version of the Depression Anxiety

Stress Scales (DASS-21): Construct validity and normative data in a large non-

clinical sample. The British Journal of Clinical Psychology, 44(Pt 2), 227-239. doi:

10.1348/014466505X29657



59


10.1037/0021-843X.117.2.473

Hu, L., & Bentler, P. M. (1999). Cuttoff criteria for fit indexes in covariance structure

analysis: conventional criteria versus new alternatives. Structural Equation Modeling:

A Multidisciplinary Journal, 6, 1-55. doi: 10.1080/10705519909540118

Joormann, J., & Gotlib, I. H. (2010). Emotion regulation in depression: Relation to cognitive

inhibition. Cognition & Emotion, 24(2), 281-298. doi: 10.1080/02699930903407948

Jöreskog, K., & Sörbom, D. (1993). LISREL 8: Structural equation modeling with the

SIMPLIS command language. Chicago, IL: Scientific Software International Inc.


& Emotion, 23(1), 4-41. doi: 10.1080/02699930802619031

Lovibond, P. F., & Lovibond, S. H. (1995). The structure of negative emotional states:

comparison of the Depression Anxiety Stress Scales (DASS) with the Beck

Depression and Anxiety Inventories. Behaviour Research & Therapy, 33(3), 335-343.

doi: 10.1016/0005-7967(94)00075-U

Magar, E. C., Phillips, L. H., & Hosie, J. A. (2008). Self-regulation and risk-taking.

Personality and Individual Differences, 45(2), 153-159. doi:

10.1016/j.paid.2008.03.014



10.1080/10253890310001594487

Matsumoto, D., Yoo, S. H., Nakagawa, S. Alexandre, J., & Altarriba, J. Anguas-Wong, A. M.

(2008). Culture, Emotion Regulation, and Adjustment. Journal of Personality and


60

Meade, A. W., Johnson, E. C., & Braddy, P. W. (2008). Power and sensitivity of alternative

fit indices in tests of measurement invariance. Journal of Applied Psychology, 93(3),

568-592. doi: 10.1037/0021-9010.93.3.568

Melka, S. E., Lancaster, S. L., Bryant, A. R., & Rodriguez, B. F. (2011). Confirmatory factor

and measurement invariance analyses of the emotion regulation questionnaire.

Journal of Clinical Psychology, 67(12), 1-11. doi: 10.1002/jclp.20836

Moore, S. A., Zoellner, L. A., & Mollenholt, N. (2008). Are expressive suppression and

cognitive reappraisal associated with stress-related symptoms? Behaviour Research

and Therapy, 46(9), 993-1000. doi: 10.1016/j.brat.2008.05.001

Nielson, K. A., & Lorber, W. (2009). Enhanced post-learning memory consolidation is

influenced by arousal predisposition and emotion regulation but not by stimulus

valence or arousal. Neurobiology of Learning and Memory, 92(1), 70-79. doi:

10.1016/j.nlm.2009.03.002


Nye, C. D., & Drasgow, F. (2011). Effect size indices for analysis of measurement

equivalence: understanding the practical improtance of differences between groups.

The Journal of Applied Psychology, 96(5), 966-980. doi: 10.1037/a0022955

Perry, Y., Henry, J. D., & Grisham, J. R. (2011). The habitual use of emotion regulation

strategies in schizophrenia. British Journal of Clinical Psychology, 50(2), 217-222.

doi: 10.1111/j.2044-8260.2010.02001.x

Richards, J. M., & Gross, J. J. (2000). The cognitive costs of keeping one's cool. Journal of

Personality and Social Psychology, 79(3), 410-424. doi: 10.1037//0022-

3514.79.3.410

61

Ritter, P., Lorig, K., Laurent, D., & Matthews, K. (2004). Internet versus mailed

questionnaires: A randomised comparison. Journal of Medical Internet Research,

6(3), 19-25. doi: 10.2196/jmir.6.3.e29

Steiger, J. H., & Lind, J. C. (1980). Statistically based tests for the number of common

factors. Paper presented at the annual spring meeting of the Psychometric Society,

Iowa City, IA.

Tucker, J. S, & Lewis, C. (1973). The reliability coefficient for maximum likelihood factor

analysis. Psychometrika, 38, 1-10. doi: 10.1007/BF02291170

Wenzlaff, Richard. M, & Luxton, David. D. (2003). The Role of Thought Suppression in

Depressive Rumination. Cognitive Therapy & Research, 27(3), 293-308. doi:

10.1023/A:1023966400540

Wiltink, J., Glaesmer, H., Canterino, M., Wolfling, K., Knebel, A., Kessler, H., . . . Buetel,

M. E. (2011). Regulation of emotions in the community: suppression and reappraisal

strategies and its psychometric properties. GMS Psycho-Social-Medicine, 8, 1-12. doi:

10.1055/s-0031-1272451

62

2.5 Acknowledgements

We thank Mark A. Griffin and Patrick D. Dunlop for statistical analysis guidance.

63

Chapter 3: Assessment of emotion processing: Analysing the psychometric

properties of the EPS-25

Abstract

Objective: The Emotion Processing Scale (EPS-25) was developed by Baker et al. (2010) to

identify difficulties in the processing of emotions. It is currently being used as a clinical and

research tool, yet no known studies to date have independently evaluated its psychometric

properties since its publication. The aim of this study was to examine the psychometric

properties of the EPS-25. Methods: The EPS-25 was assessed across two community

samples (Australia, N = 575; United Kingdom, N = 597, age 17-95) using confirmatory factor

analysis (CFA). Results: The original EPS factor structure was not supported either in the

Australian or United Kingdom samples, based on overall model fit indices. Efforts to refine

the scale by removing problematic items by analysing localised misfit, while retaining the

original essence of the 5-factor model, were not viable due to widespread significant

standardised residual covariances. A re-specification of an optimal measurement model was

then attempted using an exploratory factor analysis (EFA) on the UK sample, resulting in a

22-item, two factor EPS. However, this revised model was also not supported in either

sample when tested using CFA. Conclusion: In summary, acceptable levels of model fit

could not be found for either the original 5-factor model by Baker et al. (2010), or the EFA

model optimised for the current samples. It is suggested that the EPS-25, in its current form,

is not suitable for use in healthy (i.e. non-clinical) individuals until further scale refinement or

redevelopment with subsequent validation is achieved.

Keywords: Emotion Processing, Scale, Psychometric Properties.

64

Spaapen, D. L., Dunlop, P. D., Waters, F., Brummer, L., Stopa, L., & Bucks, R. S. (under

review). Assessment of emotion processing: Analysing the psychometric properties of

the EPS-25. Journal of Psychosomatic Research. (under review).

Emotions refer to the complex set of physiological, cognitive and behavioural

reactions that occur in response to the appraisal of a situation with personal significance.

Required for functional and adaptive social interactions, emotions are a key mechanism for

our survival. Briefly, the term ‘emotion processing’ refers to a set of interrelated processes

that include emotion perception (how we perceive emotions and identify the emotional

significance of stimuli), emotion regulation (how they are managed), and emotion

experiences (how they affect us) (Koole, 2009; Phillips, Drevets, Rauch, & Lane, 2003).

Together, these processes help the brain and body to manage the smooth transition and

change of emotions, so that other mental experiences and behaviours can remain relatively

unaffected by rapid changes in emotions (Rachman, 1980).

According to Rachman (1980), failures at any stage of emotion processing can result

in faulty cognitive processing which are subsequently experienced as intrusive thoughts,

rumination, obsessions and fears, and, over time, the emergence and maintenance of

psychological disorders. In support, dysfunctions in emotion perception, regulation, and

experience have been linked a wide range of clinical disorders including, panic disorder

(Baker, Holloway, Thomas, Thomas, & Owens, 2004), post-traumatic stress disorder

(Kumpula, Orcutt, Bardeen, & Varkovitzky, 2011; Rachman, 2001), obsessive-compulsive

disorder (Kang, Namkoong, Yoo, Jhung, & Kim, 2012), eating disorders (Bydlowski et al.,

2005), psychosis (Baslet, Termini, & Herbener, 2009; Serper & Berenbaum, 2008),

borderline personality disorder (Beblo et al., 2010), and depression (Honkalampi, Hintikka,

Tanskanen, Lehtonen, & Viinamaki, 2000; Joormann & Gotlib, 2010).

65

Whilst there are many scales that examine aspects of emotion processing, e.g.

perception (Toronto Alexithymia Scale-20 - TAS-20; Bagby, Parker, & Taylor, 1994; Levels

of Emotional Awareness Scale – LEAS; Lane, Quinlan, Schwartz, Walker, & Zeitlin, 1990),

regulation (Emotion Regulation Questionnaire – ERQ; Gross & John, 2003) and experience

(Depression Anxiety and Stress Scale-21 - DASS-21; Lovibond & Lovibond, 1995);

(Hospital Anxiety and Depression Scale – HADS; Zigmond & Snaith, 1983), there are few

scales which draw together the multiple dimensions of emotion processing. To address this,

the Emotion Processing Scale (EPS) was developed by Baker, Thomas, Thomas, and Owens

(2007) as a measure of emotion processing dysfunction that incorporates aspects of

perception, regulation and experience.

The EPS was originally developed as a 38-item 8 factor measure (EPS-38; Baker et

al., 2007) using exploratory factor analysis (maximum likelihood) on a sample of individuals

with psychological problems, psychosomatic disorders, physical disease, and healthy

individuals (n = 460). The scale was later refined into a 25-item (EPS-25; Baker et al., 2010),

five factor measure using exploratory factor analysis (59.4% of variance explained - n = 690,

comprising mental health patients, healthy controls, pain patients and general medical

practice attendees).

Though the EPS-25 is increasingly used as a clinical and research tool (Elbeze

Rimasson & Gay, 2012; Johansson et al., 2013; Kealy, Ogrodniczuk, & Howell-Jones, 2011;

Wilkins, 2012), to the best of our knowledge, no studies have formally validated its

psychometric properties beyond Baker et al.’s (2010) original investigation. Establishing the

psychometric properties is an important step towards assessing the responsiveness of

measurement instruments, particularly for measures that are frequently used (Nunnally,

1967). Further, much of the published empirical literature on the EPS-25 has been based on

clinical samples, and as such it remains uncertain if the EPS-25 functions in an equivalent

66

manner when administered to non-clinical samples. This may be problematic if the scale is

used to make inferences about emotion processing, because if the EPS functions differently

when used in non-clinical populations, then judgements based on scale scores may be

misguided.

The aim of this study was therefore to psychometrically evaluate the EPS-25 using

two large community samples from Australian and the United Kingdom. We do so by using

confirmatory factor analyses to examine the factor structure of the EPS-25.

3.1 Method

3.1.1 Participants

Participants were drawn from Australian (AUS) and United Kingdom (UK)

community samples (aged 17-97). Sample characteristics are presented in Table 1.

Questionnaires were distributed via broad community advertising which asked for volunteers

to take part in an “Emotions across the Lifespan” study. Volunteers were given the choice to

complete the questionnaires online, or in a hardcopy form, mailed out with a reply-paid

envelope. This approach was considered acceptable due to evidence that online and printed

versions of questionnaires give similar results (Ritter, Lorig, Laurent, & Matthews, 2004).

Participation was voluntary and completion of the questionnaires was interpreted to indicate

consent. This project was granted approval from the UWA Human Research Ethics

Committee (Project No RA/4/3/1247) and from the University of Southampton School of

Psychology-Research Ethics Committee (Project No ST/03/90).

Table 1.


Items AUS UK Total

67

M SD Rang

e

M SD Range M SD Rang

e

Sample

Size 575 597 1172

Gender

(N, %)

F: 369,

66.2%

M: 188,

33.8%

F: 481,

81.8%

M: 107,

18.2%

F: 850,

74.2%

M: 295,

25.8%

Age

(years) 39.8 20.9

17 -

97 36.7 20.3 18 - 91

38.

3 20.6 17-97

Education

(years) 13.4 3.1

4 -

27.2 14.1 5.5 8 - 21

13.

7 2.8

4 -

27.2

DASS -

Depressio

n

8.8 8.9 0 – 42 8.8 9.2 0 – 42 8.8 9.1 0 – 42

DASS –

Anxiety 6.7 7.7 0 – 42 6.3 7.7 0 – 42 6.5 7.7 0 – 42

DASS -

Stress 13.2 9.2 0 - 42 13.4 9.4 0 – 42

13.

3 9.3 0 – 42

Ethnicity

Australian

Aboriginal/Torre

s Strait Islander

15

(2.6%) White British

520

(87.1%

)

Western

European

228

(39.7%) Irish

41

(6.9%)

Southern or

Eastern European

34

(5.9%)

Indian/Indian

sub-

continent/Mixed

11

(1.8%)

Asian 75

(13.0%)

Black

Carribean/Mixe

d

6

(1.0%)

African 8 (1.4%) Other 12

(2.0%)

Other 167 Declined to say 7

68

(29.0%) (1.2%)

Declined to say 48

(8.4%)

Note. DASS (Lovibond & Lovibond, 1995) – DASS scores are doubled to replicate DASS-42

scoring standard.

3.1.2 Materials

The Emotion Processing Scale (EPS-25; Baker et al., 2010) is a 25-item, self-report

measure of dysfunctional emotion processing that comprises five 5-item subscales. Two

subscales relate to dysfunctional emotion regulation (Avoidance and Suppression). The three

other subscales, Unregulated Emotion, Signs of Unprocessed Emotion and Impoverished

Emotion Experience measure emotion experience. However, the Impoverished Emotion

Experience subscale also measures elements of perception, in that people who have trouble

identifying their feelings as emotions also exhibit a deficit in perception. Participants respond

to the 25 items on a 9-point Likert scale ranging from 1 (completely disagree) to 9

(completely agree). Internal consistency of the EPS-25 was adequate (three subscales α > .80,

two subscales α > .70) in the original paper by Baker et al. (2010).

3.2 Results

Before factor analysis, patterns of missingness, distribution and outliers of the data

were examined to determine its suitability using SPSS version 21. Missing data by variable

ranged from 0.5 to 2.3%. Little’s MCAR test revealed the data were missing completely at

random. Missing data was handled using pair-wise deletion. Multivariate normality was

assessed using DeCarlo’s (DeCarlo, 1997) SPSS macro. The omnibus test of multivariate

normality, based on Small’s (Small, 1980) statistics (see Looney, 1995, for add-on

information) was significant, suggesting the data departed substantially from multivariate

69

normality, χ²(26, N = 1061) = 8698.26, p < .001. Internal consistency of test scores for the

EPS-25 factors (combined AUS and UK sample) were was .87 for Suppression, .84 for

Unprocessed, .73 for Unregulated, .68 for Avoidance, and .76 for Impoverished.

3.2.1 Original confirmatory factor analysis

Initially, a five factor confirmatory factor analytic model was specified, with each

factor representing the EPS-25 subscales proposed by Baker et al. (2010). Mplus version 6.1

was used to conduct the factor analyses, using Maximum Likelihood Robust (MLR)

estimation to account for the non-normality of the data (Muthén & Muthén, 2010). The EPS’s

factor structure was evaluated using the Standardised Root Mean-square Residual (SRMR;

Jöreskog & Sörbom, 1993) the comparative fit index (CFI; Bentler, 1990), Tucker-Lewis

Index (TLI; Tucker & Lewis, 1973), and root mean square error of approximation (RMSEA;

Steiger & Lind, 1980). Chi-square non-significance, TLI and CFA values above 0.95, SRMR

below .08, and RMSEA values below .06 are commonly recommended criteria for good fit

(Hu & Bentler, 1999).

To first identify the model, a five-factor CFA using both samples was run several

times, each time with a different allocated indicator variable for each factor to assess which

indicator factor loading differed the least across the samples (see Brown, 2006). The item

within each factor that exhibited the most equivalent factor loading across the two samples

was then chosen to be the marker indicators (marker items were, 16, 2, 8, 9, 25); that is the

indicators with factor loadings to be fixed at 1.00 in subsequent analyses. Selecting the most

equivalent items as marker indicators is particularly important when examining if a scale

functions in the same way across samples (multi-group invariance analysis) (Brown, 2006). If

the marker items’ factor loadings are not equivalent between the samples, the scale may

function differently between samples and go undetected because the differences in the

70

indicator items is being masked by their unstandardized factor loadings being fixed to 1.

Additionally, it is possible that observed differences in scale scores between samples may be

an artefact of scaling the latent variable with an indicator which has a different relationship to

the factor based on the sample (Brown, 2006).

Following the selection of marker indicators, the five-factor model was then specified

separately for the two samples. Table 2 shows the fit statistics of these analyses, and fit

indices were generally indicative of poor model fit. The RMSEA (UK & AUS = .062) values

were moderate but generally within an acceptable range, and SRMR values (UK = .060, AUS

UK = .056) were within the recommended cut-offs by Hu and Bentler (1999). In contrast,

however, Chi-square, PCLOSE, CFI and TLI values all suggest poor model fit. With the

majority of fit indices indicating poor model fit, it was concluded that the original five factor

structure of the EPS-25 was not considered an adequate representation of the response

behaviour in the Australian and United Kingdom samples.

Lack of fit could be attributed to items loading onto multiple factors or items of a

common factor being more or less related to some items than others. Indeed, it may be that

the EPS-25 is a generally well-fitting model in this case, but that the fit diagnostics are being

unduly influenced by a small number of problematic items. If so, it is possible that the factor

structure might be preserved by trimming statistically problematic items. To assess the degree

of localised (item) misfit, the standardised residual covariances for both the AUS and UK

CFAs were examined (Brown, 2006). Based on J reskog & S r bom's (1993) recommended

cut-off value of 2.58 for standardised residual covariances (which corresponds to a

statistically significant score at a p-value of < .01)2, both samples presented several

2 Whilst standardised residual scores are inflated with large samples (Brown, 2006), the large number of significant residual covariances whilst using a conservative cut-off value is considered strong evidence for widespread model misfit.

71

statistically significant residual covariances for every item. The pattern of high covariances

by item was also not consistent in both samples (number of problematic covariances unique

to AUS sample = 36, number unique to UK = 40, number common to both samples = 33).

Table 2.

Single group confirmatory factor analyses (25 and 22-item) with Australian and United

Kingdom community samples.

Model χ² df p SRMR TLI CFI RMSEA LO

90

HI 90 P-CLOSE

UK

Five-Factor

model (25-item)

867.278

265

<.001* .060 0.865 0.881 .062 .057 .066 <.001*

AUS

Five-Factor

model (25-item)

843.910

265

<.001* .056 0.858 0.875 .062 .057 .066 <.001*

UK

Two-Factor

Model (22-item)

821.367

208

<.001* .061 0.848 0.863 .070 .065 .075 <.001*

AUS

Two-Factor

Model (22-item)

750.562

208

<.001* .058 0.855 0.869 .067 .062 .073 <.001*

Note. * = p < .05, Models: SRMR = standardised root mean-square residual, RMSEA = root

mean square error of approximation, CFI = comparative fit index, TLI = Tucker-Lewis Index,

LO 90 and HI 90 = lower and upper boundary of 90% confidence interval for RMSEA.

Maximum Likelihood Robust (MLR) estimation was used.

After respecifying the model, removing the most problematic items (to a maximum of

one item per factor), we still did not obtain adequate model fit.

Though it might be possible to establish good fit with the further removal of items, we

elected to stop at this point as we felt that the original essence of the five factor model

72

proposed by Baker et al. (2010) would be unrecognisable with the continued removal of

items. As such, we concluded that the EPS-25 did not exhibit a recoverable five factor

structure. The above conclusion raised the question of what factor structure exists within the

EPS-25.

3.2.2 Exploratory factor analysis

To investigate this, we turned to exploratory factor analytical techniques. Initially, the

EFA was conducted with the UK sample alone to identify items that may be causing misfit.

The UK sample was selected as a starting point because this was the country where the

measure was originally developed. We conducted an EFA using MLR extraction and geomin

rotation using Mplus version 6.1. MLR extraction was chosen because it is appropriate for

ordinal variables, and provides standard errors appropriate for a more comprehensive

examination of factor structure and individual parameters (Schmitt, 2011). Obtained

eigenvalues for the current sample3 and are presented in Table 3. The eigenvalues obtained

by Baker et al. (2010) in their original maximum likelihood analysis are also shown in Table

3 for comparison purposes.

The decision around how many factors to extract was based on Horn’s (Horn, 1965)

Parallel Analysis, a method which repeatedly undertakes EFAs of randomly generated data of

the same size as the study data to set as a benchmark (using ViSta-PARAN software;

Ledesma & Valero-Mora, 2007). If a factor eigenvalue from an EFA is greater than the 95%

probability cut-off of factors obtained from randomly generated data (i.e. less than 5% of

being obtained from randomly-generated data) then it is considered suitable for extraction.

The eigenvalues generated from 500 randomly generated datasets are therefore also shown in

3 We also undertook a maximum-likelihood EFA, as per Baker et al.’s (Baker et al., 2010) original approach, and the eigenvalues were very similar to those obtained when MLR was used - 8.65, 2.20, 1.42, 1.33, 1.11.

73

Table 3. Based on the Parallel Analysis, the current UK sample presents two factors that

exceeded the benchmark (at 5% probability). Clearly, based on this factor retention method,

the present data do not support a five factor structure for the UK sample. Indeed, if the same

technique were to be applied to Baker et al. (2010) data, two factors would have been

extracted. We therefore suggest here that Baker et al. (2010) original results also do not

support a five factor structure, and suspect that the problems we encountered in the CFA

might be due to the five factor model being inappropriate.

Table 3.

Comparison of EFA Eigenvalues by Factor for the UK sample, Baker et al.’s (2010) sample,

and Parallel Analysis.

Factor 1 2 3 4 5

UK 8.81 2.23 1.41 1.33 1.06

Baker et al. (2010) 8.70 2.30 1.40 1.29 1.10

Parallel Analysis 1.57 (1.66) 1.48 (1.55) 1.41 (1.46) 1.35 (1.40) 1.30 (1.34)

Note. Extraction Method for UK was Maximum Likelihood Robust (MLR) with direct

oblimin rotation (results were comparable when using Maximum Likelihood). Extraction

Method for Baker and Parallel Analysis samples was Maximum Likelihood. Parallel analysis

was based on a randomised normal distribution of 500 samples. All EFA and simulation data

was conducted with 25 items. For the Parallel Analysis, unbracketed values represent the

mean eigenvalue for a given factor. Bracketed values represent an upper-bound eigenvalue

that there is only a 5% chance of obtaining, given a random sample.

Given the above, we decided that two factors would be extracted from the UK sample

for further analysis. The first factor appeared to capture similar content to the original five-

item Suppression factor (i.e. the same five items loaded onto this factor, but not the other).

The second factor captured almost all of the remaining items though two items, 4 and 14, did

not load greater than .30 onto either factor, and item 5 loaded on both factors (factor 1 = .33,

74

factor 2 = .38). These three items were dropped and this resulted in a new proposed 2 factor,

22 item (5 and 17 items) model.

3.2.3 Revised 2-factor confirmatory factor analysis

The new two-factor model, proposed on the basis of the EFA results above, was

specified for CFA analysis on the UK and AUS samples. Overall goodness of fit measures for

this revised 22-item EPS did not indicate acceptable model fit (see Table 2) in the UK or

AUS samples. Once again, we examined localised model fit to examine whether any items

were disproportionately affecting what might otherwise be a good fitting model. Examination

of the standardised residual covariances revealed widespread residual covariances between

items in both samples (number of problematic covariances unique to AUS sample = 24,

number unique to UK = 37, number common to both samples = 27), suggesting the poor

model fit could not be explained just by the presence of a few problematic items as opposed

to poor overall specification. Therefore, we concluded that the 2 factor model which appeared

to represent suppression and an overarching measure of unprocessed emotions was also not

supported by the data.

3.3 Discussion

The aim of this study was to psychometrically evaluate the EPS-25 (Baker et al.,

2010) using two large community samples from Australian and the United Kingdom. The

original EPS factor structure did not exhibit adequate model fit for either the Australian or

United Kingdom community sample, and thus the results observed here are inconsistent with

the model proposed by Baker et al. (2010).

Initial efforts to revise the 25-item scale whilst generally retaining the essence of the

original factor structure (by examining localised misfit) were unsuccessful due to many

unusually high standardised residual covariances in both samples (with different item

75

covariance across the two samples). Final efforts to revise the 25 item scale via EFA methods

further called into question the robustness of the 5-factor solution. Instead, a 22-item, two

factor solution (suppression and a generalised dysfunctional emotion processing factor) was

extracted. This 2-factor revised model also failed to exhibit sound fit when subjected to CFA.

In summary, the EPS-25 items, a working model of emotion processing dysfunction, appears

not to function psychometrically as first believed, when examined with two large community

samples.

While the present results do call into question the general utility of the EPS-25 as a measure

of emotion processing, there are a number of potential explanations that should be

considered.

Firstly, the factor extraction method of the current study is different from Baker et

al.’s (2010) study. Whilst Baker et al. (2010) did not explicitly state which extraction method

they used, a reasonable guess would be that their selection criteria were similar to those

employed in the previous EPS development paper (Baker et al., 2007), which involved a)

Kaiser’s (1960) eigenvalue-greater-than-1 (K1) criterion, b) Cattell’s (1966) scree test, and c)

subjective assessment of the interpretability of factor structures. Indeed, after analysing the

results of both EPS development papers, it appears the only established extraction method

which explains the extraction of five factors is the K1 criterion. This is concerning because,

whilst Kaiser’s (1960) K1 criterion is still widely employed by researchers (probably because

it has been the default extraction method in many statistical packages), it is generally

recognised as being highly inaccurate, and consistently overestimates the number of factors,

often by a severe margin (see Lance, Butts, & Michels, 2006 for comprehensive examination

of EFA extraction methods and the shortcomings of the K1 criterion; Velicer, Eaton, & Fava,

2000; Zwick & Velicer, 1986). Whilst the extraction method used by Baker et al. (2010) is

unknown, after considering the current EFA results and those of Baker et al.’s (2010) study,

76

and in light of contemporary techniques (parallel analysis), we suspect that the initial decision

to extract five factors led to the misspecification of the EPS-25 model.

A second possibility is that our UK and AUS samples comprised random individuals

from the community. In contrast, the EPS-25 was developed using samples which included

participants with physical and psychological illness, as well as healthy samples. Differences

in the results may have been due to differences in our samples which might have led to

statistical differences in the manner in which the items behave in community and clinical

samples. It is possible that the EPS-25 has adequate model fit for clinical samples (though

Baker et al., 2010, did not report a CFA), did not report a CFA), but not in community adults.

Nonetheless, we suspect this explanation is unlikely, particularly given the fact that the

Eigenvalues observed in this study were very similar to those observed by Baker et al.

(2010). If the factor structure of the EPS-25 is indeed unstable across clinical and community

samples, it calls into question its general utility as a research tool.

There are important clinical implications to the current findings: firstly, the EPS is

currently used in research settings in healthy samples when the current results do not support

its continuing use in these settings. Therefore, any inferences based on scale scores in healthy

populations might be misguided. Secondly, for similar reasons, it is recommended that the

psychometric properties of the EPS-25 scale are examined in clinical populations before it is

used as a research tool in clinical samples. If necessary, the scale could be further developed

or expanded perhaps to include other existing measures. For example, ‘avoidance’ is

successfully measured with the Cognitive Avoidance Questionnaire (CAQ; Sexton & Dugas,

2008) and with the Acceptance and Action Questionnaire (AAQ-II; Bond et al., 2011);

‘Expressive and thought suppression’ are assessed with the Emotion Regulation

Questionnaire (ERQ; Gross & John, 2003) and White Bear Suppression Inventory (WBSI;

Wegner & Zanakos, 1994); ‘impoverished emotional experience’ has a large overlap with

77

measures of alexithymia such as the 20 item Toronto Alexithymia Questionnaire (TAS-20;

Bagby et al., 1994) and the Bermond-Vorst Alexithymia Questionnaire (BVAQ; Vorst &

Bermond, 2001). It is possible that the identification of dysfunctional emotion processing can

be achieved by examining these scales, however, for the EPS-25, it appears that the items

chosen to capture the factors have not been successful. The use of alternative items or scales

might have helped in the identification of factors consistent with Baker et al.’s (2010) model.

We suggest that subsequent EPS development or validation studies consider analysing the

concurrent validity of subscales with existing comparable scales.

In summary, the EPS-25 factor structure could not be recovered in large Australian or

United Kingdom community samples, and arguably was not supported in Baker et al.’s

(2010) study, based on EFA results. We suggest that the EPS is not suitable for use in healthy

samples until further scale refinement or redevelopment and subsequent validation is

completed. For the purposes of constructing a working scale of dysfunctional emotion

processing, the use of alternative, validated scales to replace the scale items established by

Baker et al. (2010) study may be successful in subsequent validation studies.

78

3.4 References

Bagby, R. M., Parker, J. D. A., & Taylor, G. J. (1994). The twenty-item Toronto Alexithymia

Scale—I. Item selection and cross-validation of the factor structure. Journal of

Psychosomatic Research, 38(1), 23-32. doi: 10.1016/0022-3999(94)90005-1

Baker, R., Holloway, J., Thomas, P. W., Thomas, S., & Owens, M. (2004). Emotion

processing and panic. Behaviour Research & Therapy, 42, 1271-1287. doi:

10.1016/j.brat.2003.09.002

Baker, R., Thomas, S., Thomas, P. W., Gower, P., Santonastaso, M., & Whittlesea, A.

(2010). The Emotional Processing Scale: scale refinement and abridgement (EPS-25).

Journal of Psychosomatic Research, 68(1), 83-88. doi:

10.1016/j.jpsychores.2009.07.007

Baker, R., Thomas, S., Thomas, P. W., & Owens, M. (2007). Development of an emotional

processing scale. Journal of Psychosomatic Research, 62(2), 167-178. doi:

10.1016/j.jpsychores.2006.09.005




10.1097/NMD.0b013e3181b3b20f

Beblo, T., Pastuszak, A., Griepenstroh, J., Fernando, S., Driessen, M., Schütz, A., . . .

Schlosser, N. (2010). Self-reported emotional dysregulation but no impairment of

emotional intelligence in borderline personality disorder. Journal of Nervous and

Mental Disease, 198, 385-388. doi: 10.1097/NMD.0b013e3181da4b4f


107, 238-246. doi: 10.1037/0033-2909.107.2.238

79

Bond, F. W., Hayes, S. C., Baer, R. A., Carpenter, K M., Guenole, N., Orcutt, H. K., . . .

Zettle, R. D. (2011). Preliminary Psychometric Properties of the Acceptance and

Action Questionnaire–II: A Revised Measure of Psychological Inflexibility and

Experiential Avoidance. Behavior Therapy, 42(4), 676-688. doi:

10.1016/j.beth.2011.03.007


Publications, Inc.

Bydlowski, S., Corcos, M., Jeammet, P., Paterniti, S, Berthoz, S., Laurier, C., . . . Consoli, S.

M. (2005). Emotion-processing deficits in eating disorders. International Journal of

Eating Disorders, 37, 321-329. doi: 10.1002/eat.20132

Cattell, R. B. (1966). The scree test for the number of factors. Multivariate Behavioural

Research, 1, 245-276. doi: 10.1207/s15327906mbr0102_10

DeCarlo, L. T. (1997). On the meaning and use of kurtosis. Psychological Methods, 2, 292-

307. doi: 10.1037/1082-989X.2.3.292

Elbeze Rimasson, D., & Gay, M. C. (2012). Le fonctionnement émotionnel lors de la douleur

chronique : état de la question [Emotional processing in chronic pain: The state of the

art]. Annales Médico-psychologiques, revue psychiatrique, 170, 163-168. doi:

10.1016/j.amp.2011.03.015







80

Horn, J. L. (1965). A rationale and test for the number of factors in factor analysis.

Psychometrika, 30, 179-185. doi: 10.1007/BF02289447




Johansson, R., Björklund, M., Hornborg, C., Karlsson, S., Hesser, H., Ljótsson, B., . . .

Andersson, G. (2013). Affect-focused psychodynamic psychotherapy for depression

and anxiety through the Internet: a randomized controlled trial. PeerJ, 1. doi:

10.7717/peerj.102





Kaiser, H. F. (1960). The application of electronic computers to factor analysis. Educational

and Psychological Measurement, 20, 141-151. doi: 10.1177/001316446002000116

Kang, J. I., Namkoong, K., Yoo, S. W., Jhung, K., & Kim, S. J. (2012). Abnormalities of

emotional awareness and perception in patients with obsessive–compulsive disorder.

Journal of Affective Disorders, 141, 286-293. doi: 10.1016/j.jad.2012.04.001

Kealy, D., Ogrodniczuk, J. S., & Howell-Jones, G. (2011). Object relations and emotional

processing deficits among psychiatric outpatients. Journal of nervous and Mental

Disease, 199(2), 132-135. doi: 10.1097/NMD.0b013e3182083162


& Emotion, 23(1), 4-41. doi: 10.1080/02699930802619031

81

Kumpula, M. J., Orcutt, H. K., Bardeen, J. R., & Varkovitzky, R. L. (2011). Peritraumatic

dissociation and experiential avoidance as prospective predictors of posttraumatic

stress symptoms., 120, 617-627. doi: 10.1037/a0023927

Lance, C. E., Butts, M. M., & Michels, L. C. (2006). The Sources of Four Commonly

Reported Cutoff Criteria: What Did They Really Say? Organizational Research

Methods, 9(2), 202-220. doi: 10.1177/1094428105284919

Lane, R. D., Quinlan, D. M., Schwartz, G. E., Walker, P. A., & Zeitlin, S. B. (1990). The

Levels of Emotional Awareness Scale: A Cognitive-Developmental Measure of

Emotion. Journal of Personality Assessment, 55, 124-134. doi:

10.1097/NMD.0b013e3182083162

Ledesma, R. D., & Valero-Mora, P. (2007). Determining the number of factors to retain in

EFA: an easy-to-use computer program for carrying out paralell analysis. Practical

Assessment, Research & Evaluation, 12, 1-11. doi: Available online:

http://pareonline.net/getvn.asp?v=12&n=2

Looney, S. W. (1995). How to use tests for univariate normality to assess multivariate

normality. The American Statistician, 49, 64-70. doi:

10.1080/00031305.1995.10476117




doi: 10.1016/0005-7967(94)00075-U

Muthén, L. K, & Muthén, B. O. (2010). Mplus User's Guide (6th ed.). Los Angeles, CA:

Muthén & Muthén.



82

Phillips, M. L., Drevets, W. C., Rauch, S. L., & Lane, R. (2003). Neurobiology of emotion


54(5), 504-514. doi: 10.1016/s0006-3223(03)00168-9

Rachman, S. (1980). Emotion processing. Behaviour Research & Therapy, 18, 51-60. doi:

10.1016/0005-7967(80)90069-8

Rachman, S. (2001). Emotion processing, with special reference to post-traumatic stress

disorder. International Review of Psychiatry, 13, 164-171. doi:

10.1080/09540260120074028



6(3), 19-25. doi: 10.2196/jmir.6.3.e29

Schmitt, T. A. (2011). Current methodological considerations in exploratory and

confirmatory factor analysis. Journal of Psychoeducational Assessment, 29, 304-321.

doi: 10.1177/0734282911406653



101(1-3), 195-200. doi: 10.1016/j.schres.2008.01.012

Sexton, K. A., & Dugas, M. J. (2008). The Cognitive Avoidance Questionnaire: Validation of

the English translation. Journal of Anxiety Disorders, 22(3), 355-370. doi:

10.1016/j.janxdis.2007.04.005

Small, N. J. H. (1980). Marginal skewness and kurtosis in testing multivariate normality.

Journal of Royal Statistical Society, Series C (Applied Statistics), 29, 85-87.



Iowa City, IA.

83



Velicer, W. F, Eaton, C. A, & Fava, J. L. (2000). Construct explication through factor or

component analysis: A review and evaluation of alternative procedures for

determining the number of factors or components. In R. D. Goffin & E. Helmes

(Eds.), Problems and solutions in human assessment: Honoring Douglas N. Jackson

at seventy (pp. 41-71). Boston: Kluwer Academic.

Vorst, H. C. M., & Bermond, B. (2001). Validity and reliability of the Bermond–Vorst

Alexithymia Questionnaire. Personality and Individual Differences, 30(3), 413-434.

doi: 10.1016/S0191-8869(00)00033-7

Wegner, D. M., & Zanakos, S. (1994). Chronic Thought Suppression. Journal of Personality,

62(4), 615-640. doi: 10.1111/j.1467-6494.1994.tb00311.x

Wilkins, C. (2012). A longitudinal study of women's management of emotions during

pregnancy and the association with postnatal depression [PhD thesis]. (PhD thesis),

Bournemouth University, Dorset, United Kingdom.

Zigmond, A. S., & Snaith, R. P. (1983). The Hospital Anxiety and Depression Scale. Acta

Psychiatrica Scandinavica, 67(6), 361-370. doi: 10.1111/j.1600-0447.1983.tb09716.x

Zwick, W. R., & Velicer, W. F. (1986). Comparison of five rules for determining the number

of components to retain. Psychological Bulletin, 99, 432-442. doi: 10.1037/0033-

2909.99.3.432

84

3.5 Acknowledgements

We thank Mark A. Griffin for statistical guidance.

85

3.6 Foreword to chapter 4.

By validating the Emotion Regulation Questionnaire (ERQ) and Emotion Processing

Scale (EPS), I hoped to use the tasks that were robust and generalisable to a community

sample, to examine the relationship between PLE and emotion processing. This process

resulted in a refinement of the ERQ (Chapter 2). Inadequate model fit was obtained with the

original scale, but with the removal of one item provided a substantial improvement to model

fit, resulting in a strong, revised ERQ factor structure (ERQ-9). Unfortunately, I was not able

to find any satisfactory fit for the EPS (Chapter 3), despite efforts to optimize its factor

structure. Therefore, the measure could not be used for further analyses. In the final chapter

(chapter 4), the only suitable emotional processing tasks were the ERQ and DASS. This

narrowed the scope of examination of emotional processing in PLE, but is still able to

provide useful information.

86

Chapter 4: The relationship between emotion processing and psychotic-like experiences

Abstract

A large body of evidence reveals that levels of psychosis are present in the general

population, ranging from nonclinical (or subclinical) to full-blown clinical psychotic

symptoms (see Van os et al., 2009, for meta-analysis). Psychotic-Like Experiences (PLE)

refer to experiences that resemble the positive symptoms of psychosis, but which do not

cause the levels of distress or impairment that would lead to clinical treatment, disability or

loss of functioning (Laroi, 2012). Despite studies finding associations between psychosis and

negative affect and maladaptive emotion regulation (ER), emotion processing has not

received much attention in the context of PLE. The current study investigated the relationship

between ER, assessed with the Emotion Regulation Questionnaire (Gross & John, 2003),

negative affect (Depression, Anxiety and Stress Scales (Lovibond & Lovibond, 1995), and

PLE (Psychosis Screening Questionnaire) using a large Australian community sample (N =

575). Specifically, the unique and shared variance of the DASS negative affect constructs

was assessed. It was hypothesised that ER would predict negative affect, which in turn,

would predict PLE. Consistent with previous research, the results showed that more

maladaptive ER (suppression) was related to negative affect, and that negative affect was

positively related to PLE. In contrast, no direct or indirect relationship was observed between

ER and PLE after accounting for demographics and negative affect. The current findings

provide support for the hypothesis that negative affect is associated with PLE. It is

recommended that future studies exploring ER, negative affect and PLE, examine a wider

range of ER strategies, and analyse the extent to which unique and shared variance of

depression, anxiety and stress (by Structural Equation Modelling) is related to ER and PLE.

87

Hallucinations and delusions are classic symptoms of schizophrenia and psychosis,

but these (or similar) symptoms also occur relatively frequently in the general population.

Further, there is evidence suggesting that mechanisms underlying these symptoms are

common to both the general and psychiatric populations. Psychotic-like experiences (PLE) in

people without a clinical diagnosis have increasingly assumed centre stage in schizophrenia

research because they can provide a unique context for potential early intervention.

Consequently, studies are conducted in the hope that vulnerable individuals are identified

early, and guided towards simple and well-timed intervention before symptoms become

distressing and disabling. In the present study, we consider emotion regulation (ER) deficits

and negative affect as potential mechanisms towards PLE in non-clinical populations.

4.1 Psychotic-like Experiences

PLE resemble the positive symptoms of psychosis but do not cause high levels of

distress or impairment (Larøi, 2012). The 1996 US National Comorbidity Study (n = 5,877),

found that approximately of one-quarter (28.4%) of all individuals from the general

population report one or more PLE (Kendler & Kessler, 1996). These findings are consistent

with other studies that have found the prevalence to range between 10% and 50% of

individuals in the community (Loewy, Johnson, & Cannon, 2007; Wiles et al., 2006), with

estimates differing according to the population characteristics and inclusion criteria (e.g. help

seeking behaviour). According to the ‘continuum hypothesis’ of psychosis (van Os, Linscott,

Myin-Germeys, Delespaul, & Krabbendam, 2009), psychosis exists in nature as a

distribution, ranging from nonclinical (or subclinical) to full blown clinical psychotic

symptoms. According to this view, PLE are merely quantitatively different from acute

psychotic symptoms, and are at the lower end of the spectrum with regards to distress and

clinical need for care (van Os et al., 2009). Thus, PLE are a valuable focus of research

88

because they are thought to reflect a general vulnerability for schizophrenia. For example,

several longitudinal studies show that the presence of PLE predicts future transition to

psychosis. Hanssen, Bak, Bijl, Vollebergh, and van Os (2005) showed that 8% of individuals

with PLE develop psychosis within 2 years, and others have shown that figure increases to

25% after 15 years (Poulton et al., 2000). There is also evidence of common mechanisms

giving rise to PLE in the general populations and in psychosis. Some are related to prenatal or

early aetiological factors, such as genetics, family history, childhood experiences and family

function (Hanssen, Krabbendam, Vollema, Delespaul, & Van Os, 2006; Linney et al., 2003;

Stefanis et al., 2004). Others refer to behavioural, cognitive and psychological factors, such

as coping skills (Janssen et al., 2004; Lataster et al., 2006; Spauwen, Krabbendam, Lieb,

Wittchen, & van Os, 2006), drug use (Smith, Thirthalli, Abdallah, Murray, & Cottler, 2009;

Tien & Anthony, 1990), and cognitive biases (Barkus, Stirling, Hopkins, McKie, & Lewis,

2007). This latter set of factors can arguably be better controlled than the former set,

therefore are of most immediate clinical interest.

4.2 Emotion regulation and negative affect

ER and negative affect have received little research interest in relation to PLE. ER

involves the control of a person’s initial emotional experience, to a more socially and

personally adaptive form. However, all types of ER are not always adaptive or successful.

When the employment of ER strategies is ineffective, individuals often experience negative

emotional states such as depression, anxiety and stress (negative affect). Experimental and

longitudinal studies of emotion regulation suggest that maladaptive ER strategies (e.g.

expressive suppression; masking behaviour which expresses emotion) result in increased

negative affect (e.g. depression, anxiety, stress; Beevers & Meyer, 2004; Brans, Koval,

Verduyn, Lim, & Kuppens, 2013; Gross, 2002). In contrast, adaptive emotion regulation

89

strategies (e.g. reappraisal; interpreting events in a more positive or productive way) are often

associated with reduced depression, anxiety, stress and greater personal well-being (Brans et

al., 2013; Gross & John, 2003; Kemeny et al., 2012; Spaapen, Waters, & Stopa, 2014).

There is already much evidence showing that people with psychosis have

dysfunctional emotion regulation (Henry et al., 2007; Livingstone, Harper, & Gillanders,

2009; van der Meer, van 't Wout, & Aleman, 2009) and dysfunctional emotional experiences

(Tremeau, 2006; although see Henry, Rendell, Green, McDonald, & O’Donnell, 2008). This

is relevant because negative affect is not only one of the key features of psychosis, but it also

thought to influence the content and frequency of psychotic symptoms (Freeman & Garety,

2003; Garety, Kuipers, Fowler, Freeman, & Bebbington, 2001).

By contrast, there is very limited research regarding ER and negative affect in PLE,

and questions remain about how, and exactly which types of, ER strategies and negative

affect contribute to PLE expression. This information is important as a better understanding

of the mechanistic pathways between ER and negative affect vulnerabilities may partially

explain the link between PLE and psychotic symptoms, and provide an avenue for early

intervention.

4.3 Negative affect influencing psychosis and PLE

Studies point to negative affect being present in individuals who are vulnerable to

psychosis. For example, studies of individuals in the prodromal phase of psychosis show that

depression, anxiety and irritability precede symptom onset in 60-80% of cases (Docherty,

Van Kammen, Siris, & Marder, 1978; Yung & McGorry, 1996). Studies of children and

adolescents have also demonstrated negative affect (particularly social anxiety) arises several

years before schizophrenia onset (Jones, Murray, Jones, Rodgers, & Marmot, 1994;

Kugelmass et al., 1995; Malmberg, Lewis, David, & Allebeck, 1998). Finally, the presence of

90

trauma and distress is also associated with imminent risk of psychosis (Krabbendam et al.,

2002; Tien & Eaton, 1992). Overall, negative affect is considered to be a key factor in the

development of psychosis, and maintenance of symptoms, albeit they may not be sufficient

alone as an initiating factor (Freeman & Garety, 2003).

Several studies have shown a link between PLE and negative affect, with virtually all

studies finding bivariate correlations between depression, anxiety, stress, and PLE (Armando

et al., 2010; Cella, Cooper, Dymond, & Reed, 2008; Mackie, Castellanos-Ryan, & Conrod,

2011; Paulik, Badcock, & Maybery, 2006). However, when the shared variance between

depression, anxiety and stress was accounted for, the majority of studies have found anxiety,

but not depression or stress, predicted PLE (Allen, Freeman, McGuire, Garety, & et al., 2005;

Fowler et al., 2006; Paulik et al., 2006).

In contrast, Cella et al. (2008) found depression, but not anxiety, was a predictor of

PLE. However, they measured both state and trait anxiety, and accounted for the shared

variance between both measures of anxiety (as well as depression) when predicting PLE.

Given the high correlation between state and trait anxiety (70% in Cella et al., 2008),

arguably the majority of variance explained by the anxiety was lost when the common

variance was excluded. Therefore it is not surprising that anxiety was not a significant

predictor of PLE. Nevertheless, Cella et al. (2008) did find an interaction effect of depression

and anxiety predicting delusion, but not hallucination, PLE. Additionally, Martin and Penn

(2001) found depression to be a better predictor of paranoia PLE than anxiety, however, they

used different measures of anxiety (both examining social anxiety).

These findings are largely consistent with contemporary models of how negative

affect relates to psychotic symptoms (see Freeman, Garety, Kuipers, Fowler, & Bebbington,

2002). Specifically, that anxiety is thought to play a key role in hallucinations and paranoia,

particularly in persecutory delusions, as anxiety disorders and delusions are thought to share

91

the same maintaining factors (e.g. threat beliefs; Clark, 1999; Freeman & Garety, 1999).

Depression and stress are also considered to play a role in hallucination and paranoia

precipitation and maintenance, but to a lesser extent, whilst one’s emotional state (including

negative affect) influences positive symptom content (Freeman et al., 2002; Trower &

Chadwick, 1995)).

A notable limitation of the above studies is the failure to separately account and

control for the unique and shared variance of negative affect constructs such as depression,

anxiety and stress. While phenomenologically distinct, these constructs are also highly

correlated. Evidence exists that each of these constructs provides unique contribution to

negative affect, but also that they share a common emotional construct which Henry and

Crawford (2005) termed ‘psychological distress’. Therefore, failing to fully account for the

unique variances of the depression, anxiety and stress, and the shared contribution of the

general psychological distress factor, can lead to over-inflation of what appears to be the

unique influence of one construct, masking of the role of the shared construct, and overall

reduced specificity of findings to PLE. This is relevant because the optimal screening

measures, preventative methods, and how it relates to PLE onset and maintenance may differ

based on negative affect type (e.g. anxiety, depression). As an example, if anxiety is the

prevailing factor, it may indirectly result from inadequate emotion perception and regulation

(Rachman, 1980), and share common maintaining factors as anxiety disorders (Freeman &

Garety, 1999).

In the current study we directly address this concern by analysing the unique and

shared contributions of depression, anxiety and stress in the prediction of PLE. Based on the

difference in findings in previous studies and the high covariance of depression, anxiety and

stress, it is hypothesised that the generalised psychological distress factor will be the only

significant predictor of PLE.

92

4.4 Emotion regulation and PLE

While the positive association between of negative affect and PLE has been reported,

only a handful of studies have explored the relationship between ER and PLE. In addition,

findings are generally mixed. Some negative findings exist (Henry et al., 2008), while others

show tentative evidence that that ineffective ER is positively associated with PLE (Modinos,

Ormel, & Aleman, 2010, n = 34; Westermann & Lincoln, 2011, n = 151 but only weakly;

Westermann, Kesting, and Lincoln, 2012).

Regarding reappraisal, in an experimental fMRI study, Modinos et al. (2010) found

reappraisal to be less effective in high-PLE individuals. Furthermore, another experimental

study by Westermann et al. (2012) found reappraisal predicts paranoia expression (state

paranoia), but only in stressful situations and individuals who score high in paranoia-

proneness (trait paranoia). This is very surprising considering reappraisal is commonly found

to be an adaptive strategy (Aldao, Nolen-Hoeksema, & Schweizer, 2010; Spaapen et al.,

2014). One explanation proposed by Westermann et al. (2012) is that in healthy individuals,

reappraisal lowers stress, however, in individuals with who experience PLE and significant

distress, cognitive factors including jumping to conclusions, theory of mind deficits , pre-

existing delusional beliefs, negative interpersonal schemata, and aberrant salience (a

hyperdopaminergic state leading to the assignment of exaggerated importance and

motivational significance to stimuli) may negatively bias their interpretation of an event,

resulting in ineffective or maladaptive reappraisal, which may give rise to paranoid thoughts.

However, this speculative hypothesis remains untested. Westermann et al. (2012) only

measured adaptive reappraisal (and not whether it was successful). While tentative evidence

of a link between reappraisal and PLE exists, no relationship has been found between

expressive suppression and PLE (Henry et al., 2009; Westermann et al., 2012)). In 2011,

93

Westermann and Lincoln, using the DERS, only found differences in impulse control to be

related to paranoia, although this measure does not measure specific ER strategies.

Considerable variations exists in the literature in methods for assessing ER (and PLE), which

makes direct comparisons difficult.

In summary, the current study aimed to examine how specific types of emotion

regulation and negative affect is associated with PLE. Whilst there are limited and

inconclusive findings in regards to how ER directly relates to PLE, there are consistent

findings that ER predicts negative affect, and negative affect predicts PLE. Therefore, it is

hypothesised that ER will predict PLE indirectly through negative affect. Although no

explicit hypothesis is set, the exploration of direct effects of ER on PLE will be examined.

4.5 Method

4.5.1 Participants

Participants were drawn from an Australian community sample (aged 17-95). Sample

characteristics are presented in Table 1. Questionnaires were distributed via broad community

advertising which asked for volunteers to take part in an “Emotions across the Lifespan”

study. Volunteers were given the choice to complete the questionnaires online, or in hardcopy

mailed out with a reply-paid envelope, as there is evidence that online and printed versions of

questionnaires give similar results (Ritter et al., 2004). Participation was voluntary and

completion of the questionnaires was taken to indicate consent. This project was granted

approval from the UWA Human Research Ethics Committee (Project No RA/4/3/1247)

4.5.2 Materials

The Psychosis Screening Questionnaire (PSQ; Bebbington & Nayani, 1995) is a

self-report measure used to screen for psychosis in the past year. Because of the need for

94

brevity in assessment battery, an abbreviated version was used comprising 6 items assessed:

strange experiences (“Have there been times when you felt something strange was going

on?”), delusions of control (“Have you ever felt that your thoughts were directly interfered

with or controlled by some outside force or person?”) delusions (“Have there been times

when you felt that people were against you”; “Have people deliberately acted to harm or plot

against you?”), hallucinosis (“Have there been times when you heard or saw things that other

people couldn’t?”; “Did you at any time hear voices saying quite a few words or sentences

when there was no-one around that might account for it?”) Questions were answered with yes

or no, and the final score was the number of yes responses.

The Emotion Regulation Questionnaire – Revised (ERQ-9; Spaapen et al., 2014) is

a 9-item, self-report measure of habitual expressive suppression (4 items) and reappraisal (5

items), based on the original 10-item ERQ by Gross and John (2003), with the exclusion of

Item 3 (“When I want to feel less negative emotion [such as sadness or anger], I change what

I’m thinking about”). The ERQ-9 uses a 7-point Likert scale ranging from 1 (strongly

disagree) to 7 (strongly agree). Example questions include “I control my emotions by

changing the way I think about the situation I’m in” (reappraisal) and “I control my emotions

by not expressing them” (suppression). Internal consistency was adequate in the community

validation study (Spaapen et al., 2014).

Table 1.


M SD Range

Sample Size 623

Gender (N, %) F: 402, 66.3% M: 204, 33.7%

Age (years) 39.2 20.6 17 - 95

95

Education (years) 13.4 3.1 4 - 27.2

ERQ-9 - Suppression 25.2 5.3 5 – 35

ERQ-9 – Reappraisal 28.0 13.6 4 – 28

DASS - Depression 8.8 8.9 0 – 42

DASS – Anxiety 6.7 7.7 0 – 42

DASS - Stress 13.2 9.2 0 - 42

PSQ 1.7 1.6 0 – 6

Ethnicity

Australian Aboriginal/Torres Strait

Islander

17 (2.7%)

Western European 252 (40.5%)

Southern or Eastern European 37 (5.9%)

Asian 88 (14.1%)

African 8 (1.3%)

Other 171 (27.5%)

Declined to say 50 (8.0%)

Note. ERQ-9 (Spaapen, Waters, & Stopa, 2014) = Emotion Regulation Questionnaire,

DASS-21 (Lovibond & Lovibond, 1995) = Depression Anxiety and Stress Scale– DASS

scores are doubled to replicate DASS-42 scoring standard. PSQ (Bebbington & Nayani,

1995) = Psychosis Screening Questionnaire.

The Depression, Anxiety and Stress Scale 21 (DASS-21) is a shortened version of

Lovibond and Lovibond’s (1995) 42-item self-report measure of depression, anxiety, and

stress (DASS). Previous studies have found the DASS-21 yields satisfactory reliability

estimates (Antony, Bieling, Cox, Enns, & Swinson, 1998; Clara, Cox, & Enns, 2001; Henry

& Crawford, 2005). The DASS-21 uses a 4 point severity/frequency Likert scale, with scores

96

doubled to give comparable results to the original 42-item DASS. It can be scored as separate

sub scales or as a single, generalised psychological distress factor (Henry & Crawford, 2005).

4.6 Results

Before analysis, patterns of missingness, distribution and outliers were examined to

determine its suitability using SPSS version 21. Missing data by variable ranged from 0.5%

to 4.1%. Little’s MCAR test revealed the data were missing completely at random. However,

normality assumption testing based on Tabachnick and Fidell’s (2013) suggestion to examine

the skewness and kurtosis z-scores over their standard errors, indicated both (Skewness =

4.97; Kurtosis = 2.70) were above the 1.96 critical score (which corresponds to a statistically

significant score at α of .05). Therefore, the univariate normality assumption was not

achieved. Multivariate normality was assessed using DeCarlo’s (1997) SPSS macro. The

omnibus test of multivariate normality, based on Small’s (1980) statistics (see Looney, 1995,

for add-on information) was significant, suggesting the data departed substantially from

multivariate normality, χ²(16, N = 356) = 478.92, p < .001. These departures from normality

are perhaps to be expected due to the clinical nature of the scales.

4.6.1 PSQ (PLE prevalence)

PSQ data by frequency are presented in Table 2. In descending order of prevalence,

the most commonly reported PSQ items are ‘Have there been times when you felt that people

were against you’ (53.7%), ‘Have there been times when you felt something strange was

going on?’ (42.4%), ‘Have you ever felt that your thoughts were directly interfered with or

controlled by some outside force or person?’ (33.4%), ‘Have people deliberately acted to

harm or plot against you’?’ (20.7%), ‘Have there been times when you heard or saw things

that other people couldn’t?’ (16.2%), and ‘Did you at any time hear voices saying quite a few

97

words or sentences when there was no-one around that might account for it?’ (6.1%).

Approximately 30% of participants reported no symptoms, 70% reported at least one

symptom, and 50% reported two or more symptoms.

Table 2.

Frequency of Psychosis Screening Questionnaire symptoms (6-item PSQ)

PSQ-symptoms Frequency Percent

0 137 30.1

1 90 19.8

2 82 18.0

3 79 17.4

4 44 9.7

5 19 4.2

6 4 .9

Total 455 100

Note: Frequency refers to the number of PLE items endorsed, regardless of which items

apply.

To determine if the PSQ items could be separated into factors (e.g. hallucinosis and

paranoia), an exploratory factor analysis (EFA) was conducted using MLR extraction and

geomin rotation using Mplus version 7.2. MLR extraction was chosen because it is

appropriate for ordinal variables, and provides standard errors appropriate for a more

comprehensive examination of factor structure and individual parameters (Schmitt, 2011).

Obtained eigenvalues for the current sample are 2.32, 1.17, 0.78, 0.65, 0.60, 0.48. The

decision around how many factors to extract was based on Horn’s (1965) Parallel Analysis, a

method which repeatedly undertakes EFAs of randomly generated data of the same size as

98

the study data to set as a benchmark (using ViSta-PARAN software; Ledesma, 2007). If a

factor eigenvalue from an EFA is greater than the 95% probability cut-off of factors obtained

from randomly generated data (i.e. less than 5% of being obtained from randomly-generated

data) then it is considered suitable for extraction. The PA eigenvalues (95% cut-off values)

generated from 500 randomly generated datasets were 1.18, 1.11, 1.05, 1.00, 0.93, 0.91.

Based on the Parallel Analysis, only one factor could be extracted. Therefore, no analysis by

PLE clusters could be explored in this sample.

4.6.2 Demographics and ER, negative affect, and PLE

PSQ scores were negatively correlated with age, with younger participants reporting

higher PSQ scores (see Table 3 for correlation matrix). There were no Gender (F < 1) or

Education differences for PSQ (see Table 4 comparisons by gender). Older participants

reported fewer depressive, anxious and stress symptoms (p < .01). No significant

relationships were found between any of the DASS subtests and Education. Overall, there

was a main effect of gender for DASS subscale scores (Depression, Anxiety and Stress), F (3,

533) = 4.23, p = .006, partial η2 = .023. Inspection of the Univariate tests revealed that this

omnibus effect was driven by significant gender differences in Depression and Stress, but not

in Anxiety.

Table 3.

Correlations Between Measures

Measure Education PSQ DASS-D DASS-A DASS-S ERQ-Rea ERQ-Supp

Age -.18** -.31** -.27** -.32** -.27** .08 -.02

Education .03 -.04 -.02 .03 -.01 -.09*

PSQ - .36** .43** .41** -.04 .04

DASS-D - - .55** .59** -.15** .26**

DASS-A - - - .64** -.10** .15**

99

DASS-S . - - - -.10* .11**

ERQ-Rea - - - - - -.12**

ERQ-Supp - - - - - -

Note. * p < .05** p < .01 (1-tailed). Statistics are calculated using Spearman’s rho

correlation coefficient. Education = total years of education; PSQ = Psychosis Screening

Questionnaire; DASS = Depression, Anxiety and Stress Scale (21 item) short version, DASS-

D = Depression subscale (7 items), DASS-A = Anxiety subscale (7 items), DASS-S = Stress

Subscale (7 items); ERQ = Emotion Regulation Questionnaire (9 item), ERQ-Reap =

Reappraisal (5-item), ERQ-Supp = Suppression (4-item).

Table 4.

Group Comparisons by Gender

Measure N Male Female Sig. (η2)

M (SD) range M (SD) range

PSQ 428 1.60 (1.50) 0 - 5 1.71 (1.58) 0 - 6 .526 (0)

DASS-D 537 7.03 (7.00) 0 - 34 9.56 (9.59) 0 - 42 .002 (.02)

DASS-A 537 5.91 (6.84) 0 - 36 6.87 (7.85) 0 - 42 .163 (0)

DASS-S 537 11.54 (8.77) 0 - 42 13.87 (9.27) 0 – 40 .005 (.02)

ERQ-Reap 534 29.67 (6.28) 12 - 42 30.30 (6.29) 6 - 42 .304 (0)

ERQ-Supp 534 14.40 (5.28) 4 - 26 13.02 (5.23) 4 - 28 .004 (.01)

Note. PSQ = Psychosis Screening Questionnaire; DASS = Depression, Anxiety and Stress

Scale (21 item) short version, DASS-D = Depression subscale (7 items), DASS-A = Anxiety

subscale (7 items), DASS-S = Stress Subscale (7 items), DASS-T = Total score (21 items);

ERQ = Emotion Regulation Questionnaire (10 item), ERQ-Reap = Reappraisal (5-item),

ERQ-Supp = Suppression (5-item).

No significant correlations were found between ERQ Reappraisal and Age or

Education. ERQ Suppression was significantly correlated (negative and very weak) with

education (see Table 5 for correlations). As level of education increased, participants’ scores

on ERQ Suppression slightly decreased. No relationship was observed between ERQ

Suppression and age. There was a significant gender difference for the combined ERQ

100

subscales, F (2, 531) = 4.429, p = .012, partial η2 = .016. Analysis of the ERQ subscales

individually revealed a significant gender difference for ERQ Suppression, F (1, 532) =

8.244, p = .004, partial η2 = .015. No gender difference was observed for ERQ Reappraisal.

Significant correlations were found between all DASS subscales and ERQ

Suppression scores, which were positive and weak. Significant correlations were obtained

between ERQ Reappraisal and DASS Depression scores. The relationship was negative and

weak, indicating that participants who reported higher ERQ Reappraisal scores had slightly

lower DASS Depression scores. No significant relationship was found between ERQ

Reappraisal and DASS Anxiety or Stress subscales.

4.6.3 Factor analysis

To examine the relationship between ER, negative affect, and PLE, accounting for the

proportion of unique and shared variance within the depression, anxiety and stress factors, a

Confirmatory Factor Analyses (CFA) was undertaken, so as to establish an appropriate

measurement model for the DASS, given the strong intercorrelations that are typically

observed amongst the three components. As such, we followed the method described by

Henry and Crawford (2005; see below). Following the CFA, full Structural Equation Models

(SEM) were specified to test the hypotheses. The CFA and SEM analyses were all

undertaken using Mplus version 7.2 with Maximum Likelihood Robust (MLR) estimation,

which calculates parameter standard errors and a χ2 statistic that is robust to the non-

normality of the data (Muthén & Muthén, 2010). All models were evaluated using the a

combination of fit indices including χ2, the Standardised Root Mean-square Residual (SRMR;

Jöreskog & Sörbom, 1993) the comparative fit index (CFI; Bentler, 1990), the Tucker-Lewis

Index (TLI; Tucker & Lewis, 1973), and Root Mean Square Error of Approximation

(RMSEA; Steiger & Lind, 1980). A non-significant χ2 , TLI and CFI values above 0.95,

101

SRMR below .08, and RMSEA values below .06 are commonly recommended criteria for

good fit (Hu & Bentler, 1999). To identify the models, the first item of each factor were

selected to be the marker indicators with factor loadings fixed at 1.00.

4.6.4 Establishing the measurement models using confirmatory factor analysis

As discussed in the introduction, Henry and Crawford (2005) raised the importance of

the distinction between the three specific factors within the DASS, namely Depression,

Anxiety, and Stress, and the general psychological distress factor. As such, here we

replicated the quadripartite CFA that they undertook. To do so, we specified a model with

three ‘unique’ Depression, Anxiety and Stress factors, on which their respective DASS items

loaded. We also specified a fourth general ‘psychological distress’ factor on which all DASS

items were loaded (see Figure 1 for the quadripartite structure with standardised factor

loadings).4 Within this model, the generalised distress factor is therefore a measure of the

shared variance between the DASS items, and the depression, anxiety and stress factors only

represent their unique variance. With the exception of the χ2 statistic, which is known to be

inflated in large samples, the fit indices for the quadripartite factor structure were close to the

recommended cut-offs (see Table 5 for fit indices), and this model was used for the remainder

of the analyses involving the DASS.

4.6.4.1 Full measurement model. All factors were then entered into a measurement

model to assess construct validity. All factors were set to covary, with the exception of the

DASS quadripartite factors, which were not set to internally covary because the generalised

psychological distress factor already measures the covariance between the depression,

4 “Due to the highly interrelated nature of the DASS subscales, caution is advised when interpreting the association of variables with a single quadripartite sub-factor in isolation.”

102

anxiety and stress factors. Once again, the fit indices were close to recommended cut-offs,

suggesting the scales are measuring distinct constructs.

Figure 1. DASS Quadripartite Model Confirmatory Factor Analysis model. Dep = DASS

Depression, Anx = DASS Anxiety, Stress = DASS Stress, gd = generalised psychological

distress factor.

103

Table 5.

Confirmatory Factor Analyses and Structural Equation Models

Model χ² df p SRMR TLI CFI RMSEA LO 90 HI 90 P-CLOSE

Measurement

models

Quadripartite 473.25 168 <.001* .035 .930 .944 .057 .051 .063 .03

7 factor 946.114 558 <.001* .047 .930 .938 .033 .030 0.37 1.00

Structural models

Quad + PLE +age 640.067 321 <.001* .041 .921 .933 .040 .036 .045 1.00

Full structural model

+age

1054.489

587

<.001* .048 .919 .928 .036 .032 .039 1.00

Note. For the fit statistics, the following is regarded as indicating good fit; CFI, TLI > .95;

SRMR < .08; RMSEA < .06. SRMR = standardised root mean-square residual, TLI =

Tucker-Lewis Index, RCFI = Robust comparative fit index, CFI = comparative fit index,

RMSEA = root mean square error of approximation, LO 90 and HI 90 = lower and upper

boundary of 90% confidence interval for RMSEA. Maximum Likelihood Robust (MLR)

estimation was used. Quad = quadripartite model (see Figure 1).

4.6.5 Structural Equation Modelling Predicting PSQ with the quadripartite model of

the DASS

It was hypothesised that generalised psychological distress, and not the individual

contributions of depression, anxiety and stress, would predict PLE. To test this, a SEM was

specified with the PSQ being indicated with its six items, and then regressed on the four

DASS factors (i.e. Depression, Anxiety, Stress, and general). All factors were also regressed

on age, because both negative affect and PLE are considered to be influenced by age (see

Table 3 for correlations; Charles, Reynolds, & Gatz, 2001; Scott et al., 2008).

In support of the current hypothesis, the generalised psychological distress factor

(shared variance) was the only significant DASS predictor of PSQ scores (β = .55, p <.001),

104

accounting for 30% of the variance in PSQ (depression, anxiety, stress p > .05). Not

surprisingly, age also predicted PSQ scores (β = .26, p <.001), accounting for 6.8% of the

variance in PSQ scores. Model fit was close to recommended cut-off values and considered

adequate (see Table 5 for fit indices).

4.6.6 ERQ PLE model (direct effects)

Attention then turned to an exploratory analysis of possible direct effects of emotion

regulation on PLE. To explore this, a SEM was specified with PSQ regressed on suppression,

reappraisal and age (ER was not regressed on age because no bivariate relationship was

observed, and there is no consistent evidence suggests reappraisal or suppression is

influenced by age). After accounting for age, no direct effects of reappraisal (β = -.05, p =

.443) or suppression (β = .03, p = .625) were observed. We concluded from this that there are

no mediating effects of DASS between the ERQ and PSQ scores5.

4.6.7 Full ERQ, DASS quadripartite, PLE model

It was hypothesised, however, that ER indirectly influences PLE through negative

affect. To determine whether there are indirect effects of ER on PSQ through DASS a full

model was then specified with all seven factors (see Figure 2 for simplified path model, and

Chapter 4 appendix for full model). This model was identical to the previous model, except

the DASS quadripartite factors were added as second-order predictors. That is, PLE was

regressed on the four DASS factors, which were regressed on ERQ-S and ERQ-R.

The total indirect effect of suppression (β = .09, p = .018) on PSQ, through the DASS

quadripartite factors was significant. The specific indirect effects for suppression on PLE

were through the generalised distress factor only (Suppression – Generalised Distress – PSQ:

5 A moderator analysis was also conducted to see if DASS subscales moderated the possible relationship between the emotion regulation and PLE. DASS did not moderate this relationship.

105

β = .08, p = .022, R2 = .01). However, once again, this indirect effect only accounted for less

than 1% of the variance in PLE, representing a practically insignificant effect.

Figure 2. Full Structural Equation Model. Statistics are: Beta weights (standard error). * = p <

.05. Paths that are grey represent associations with negligible to no practical significance. A

full SEM model with indicators is presented in the Appendix A.

Attention then turned towards the relationships between ER and negative affect. It

was hypothesised that ER would predict negative affect. Reappraisal significantly and

inversely predicted depression (β = -.12, p < .05) only, however this relationship was

negligible in magnitude (only accounted for 1% of the variance depression). By contrast,

suppression positively predicted depression (β = .33, p < .01) and generalised psychological

distress (β = .14, p < .05). In the case of generalised psychological distress, however,

106

suppression only explained 2% of the variance, so this relationship is arguably practically

insignificant. The degree of variance explained was determined by running the analysis three

times, once with both factors (reappraisal and suppression), and two with only suppression or

reappraisal being predictors of the generalised psychological distress factor. In summary, the

only practically significant relationship observed between ER and negative affect was

suppression predicting depression.

Once again, of the DASS factors, the generalised psychological distress factor was the

only significant predictor of PSQ (β = .54, p < .01; depression, anxiety, stress p > .05). Thus

in sum, suppression was positively associated with depression, and the generalised distress

factor was positively associated with PLE, but the remaining direct paths were either weak

but significant, or simply non-significant.

4.7 Discussion

The aim of the current study was explore the extent to which ER strategies and

negative emotion are associated with PLE. In the current sample, between 40-54% of the

population reported paranoid thoughts, 33% passivity thoughts, and 6-16% hallucinosis.

These figures are a little higher than most other studies of PLE, although much variation

exist depending on scale used, the symptom criteria (strict, general) and population group in

the population (PSQ; 66%; Johns et al., 2004; CIDI; 28.4%; Kendler & Kessler, 1996; PQ;

43%; Loewy et al., 2007; 8%; van Os et al., 2009).

4.7.1 Negative affect and PLE

Consistent with previous studies, increased depression, anxiety and stress scores were

associated with the presence of PLE (Armando et al., 2010; Cella et al., 2008; Mackie et al.,

2011; Paulik et al., 2006). To ascertain the unique and shared contribution of depression,

107

anxiety and stress in predicting PLE, an analysis using SEM was performed (quadripartite

model). The findings supported our hypothesis, with the generalised psychological distress

factor being the only significant predictor of PLE. This finding may explain the inconsistent

results of previous studies which have found that a single negative affect factor (e.g.

depression, or anxiety) predicted PLE (Allen et al., 2005; Cella et al., 2008; Paulik et al.,

2006). It is possible that these single (unique) factor results are an artefact of using common

statistical analyses, with standard regression usually emphasising the unique contributions of

predictors over the predictive utility of shared variance. Ultimately, it is difficult to compare

the current results to previous studies which did not account for shared variance when

examining how negative affect predicts PLE (Mackie et al., 2011; Martin & Penn, 2001).

One possibility exists that these specific negative affects domains may be

differentially linked to PLE symptom clusters (e.g. hallucinations or paranoia), or even

symptom subtype. For example, anxiety is considered central in the formation in persecutory

delusions (Freeman et al., 2002), whilst depression may contribute to other subtypes of

delusions (i.e. paranoia content relating to self-punishment for intrinsic badness; Trower &

Chadwick, 1995), whilst stress is not as relevant for grandiose delusions (Freeman et al.,

2002). Also, other research suggests the content of hallucinations can reflect negative affect

(depression and anxiety; Garety et al., 2001) and maintain symptoms (Chadwick &

Birchwood, 1994), but not are not a critical hallucination onset. Unfortunately, PLE could not

be fully functionally separated into symptom clusters as symptom subtype information was

not available. In addition, all individuals with hallucinations also had high levels of paranoid

thoughts, so no differentiation between depression and paranoia could be undertaken.

Nevertheless, it is possible that negative affects domains are specifically linked to PLE

symptom clusters, although this hypothesis could not be explored in this population group.

108

4.7.2 ER and negative affect

An examination of bivariate correlations between ER strategy use and negative affect

shows that greater suppression and lower reappraisal were related to increased depression,

anxiety and stress. This is finding is consistent across studies of ER and negative emotion

(Aldao et al., 2010; Matheson & Anisman, 2003; Spaapen et al., 2014). Subsequent

examination of the influence of ER on the unique and shared variance of negative affect,

showed that suppression predicted depression only. No other practically significant results

were observed between ER and negative affect.

These findings are somewhat consistent with Wiltink et al. (2011) who found

suppression predicted depression, male gender, lower education level, and lower household

income, but not anxiety. In contrast, Wiltink et al. (2011) did find reappraisal predicted

depression, but not anxiety. Other studies have found contrary results. For example, Dennis

(2007) found that suppression was related to state anxiety only, and reappraisal was

associated with depressed mood only, depending on affect style. Another study using a

different, wider set of emotion regulation strategies (SCOPE abbreviated; Matheson &

Anisman, 2003) found that ER strategies were related to differences in individuals’ negative

affect profile (i.e. control, anxious only, dysphoric only, both). Most maladaptive ER

strategies (e.g. rumination and emotional containment.) were related to anxiety and

dysphoria. However, there was a trend for these ER strategies to result in higher dysphoria

scores, compared to anxiety (with the exception of emotional expression, with higher anxiety

scores). The adaptive ER strategies were endorsed less by the dysphoric only group,

compared to the anxious only group. Additionally, the most maladaptive, and least adaptive

ER endorsement was found in the comorbid (both anxious and dysphoric) group. It is worth

noting that Matheson and Anisman (2003) also found substantial overlap in almost every

association between ER strategies, dysphoria and anxiety. It is well established that

109

depression and anxiety are highly correlated, so is this finding is not overly surprising. Based

on these findings, it appears that the degree to which ER is associated with depression and

anxiety, is dependent on the specific emotion regulation strategies used (with some degree of

overlap), with depression appearing to be associated with adopting more maladaptive, and

less adaptive ER strategies. This area of ER and negative affect is still not well explored. It is

suggested that additional studies explored the relationship of ER with a comprehensive list of

ER strategies.

The discrepancy in findings between the current study and Wiltink et al. (2011), and

that of Dennis (2007), may be explained by differing methodology. Indeed, Dennis (2007)

used a smaller (N = 67), less representative sample (all females and lower age range), as well

as different measures of depression and anxiety. Whilst ER strategies are consistently related

to age and gender, this is not expected to account for these findings. More likely, it is related

to the different measures of negative affect or sampling error (more likely in smaller samples)

influencing results.

4.7.3 Emotion regulation and PSQ

An exploratory SEM analysis examining the possible direct effect of ER on PLE was

undertaken, however, no significant relationships were found. These results are divergent

from the existing literature, with some studies finding reappraisal (generally considered

adaptive), but not suppression, was associated with paranoia expression (Taylor, Graves, &

Stopa, 2009; Westermann et al., 2012). In contrast, Westermann, Boden, Gross, and Lincoln

(2013) found only maladaptive strategies (not reappraisal) were related to paranoia

expression. It is worth noting that these studies did not account for the potential mediating

relationship of negative emotion. However, as the few previous studies have used different

measures or procedures, and have returned largely different results, it is difficult to compare

110

the findings. Nevertheless, the current study does differ from other studies in that it found no

direct relationship between adaptive or maladaptive ER and PLE.

The relationship between ER strategies and negative emotion, and negative emotion

and PLE is well established (Brans et al., 2013; Cella et al., 2008; Gross & John, 2003; Paulik

et al., 2006). Therefore, it was hypothesised that maladaptive and adaptive ER strategies use

would lead to increased or decreased negative emotion, respectively, and in result, would

help influence PLE (indirect relationship). The current study found a relationship between

suppression and PLE through the general psychological distress factor, however this finding

only accounted for less than 1% of the variance in PLE scores. No other indirect relationships

were observed in the current sample. This suggests that the habitual use of ER strategies

assessed by the ERQ is not related to PLE.

There are several explanations for the findings. There may be a relationship between

ER and PLE, but limitations related to measurement or sampling have influenced the

findings.

Whilst there are many ER strategies, the current study only examined reappraisal and

expressive suppression. It is possible that including a more comprehensive set of ER

strategies, particularly those associated with psychopathology (e.g. rumination, avoidance,

thought suppression; Aldao et al., 2010) may return different results. Also, it may be self-

report measures of ER strategy use are less accurate due to inherent drawback of not knowing

if reported self-reflection truly reflects habitual cognition/behaviour. Nevertheless, self -

report measures are still considered the most viable method of examining the unique and

shared variance between negative emotion, ER strategy use, and PLE. The abbreviated PSQ

measure adopted for this study may also have skewed the results. For example, it is possible

the current PSQ version measures more common, less distressing PLE. Alternatively, the

sample composition may have also influenced the results. In the case of a sample being

111

relatively homogenous for PLE, finding no association between ER strategies and PLE could

be foreseeable. However, the current sample distribution (positively skew) is expected in sub-

clinical presentations of mental illnesses with multi-factorial causes (e.g. psychosis; van Os et

al., 2009). This coupled with the large sample size (leading to sufficient numbers of extreme

cases), leads us to believe sample composition was not a significant influence on the results.

In addition, some of our results above replicated existing findings, suggesting that our sample

was not unusual and that the analyses were adequate.

As mentioned previously, it is recommended that future studies examine the

relationship between a wide range of ER strategies, the shared and unique variance within

negative emotion, and their influence on hallucination and paranoia PLE independently. It is

suggested that structural equation modelling (SEM) is used to assess these relationships, due

to several advantages compared to conventional regression analyses. Firstly, it will allow for

indirect effects, to assess the influence of negative emotion on the relationship between ER

strategies and PLE, which is not appropriate in standard regression models due to the

covariance of negative emotion with both the predictor and criterion. Secondly, SEM

simultaneously gives access to examine the unique and shared variance between measures of

negative affect (quadripartite model). This way, the extent to which generalised distress (i.e.

underlying shared variance between depression, anxiety and stress) and variance unique to

specific negative affect factors are related to hallucination and paranoia PLE can be

examined. Thirdly, SEM is a more comprehensive assessment technique of the relationships

between aspects of negative emotion, ER and PLE because it can account for the random

error present in all measurement that is not directly related to relevant variables.

In summary, our results support existing research suggesting ER influences negative

affect, and negative affect predicts PLE. However, the relationship between ER and PLE is

less clear, with limited and inconsistent results in the literature. Due to the limited research in

112

the area of emotion and PLE, and the complex nature of PLE expression, the exact nature of

these relationships needs to be explored further.

113

4.8 References



30(2), 217-237. doi: 10.1016/j.cpr.2009.11.004

Allen, P., Freeman, D., McGuire, P., Garety, P., & et al. (2005). The prediction of

hallucinatory predisposition in non-clinical individuals: Examining the contribution of

emotion and reasoning. The British Journal of Clinical Psychology, 44, 127-132. doi:

10.1348/014466504X20044

Antony, M. M., Bieling, P. J., Cox, B. J., Enns, M. W., & Swinson, R. P. (1998).

Psychometric Properties of the 42-Item and 21-Item Versions of the Depression

Anxiety Stress Scales in Clinical Groups and a Community Sample. Psychological

Assessment, 10(2), 176-181. doi: 10.1037/1040-3590.10.2.176

Armando, Marco, Nelson, Barnaby, Yung, Alison R., Ross, Margaret, Birchwood,

Maximilian, Girardi, Paolo, & Nastro, Paolo Fiori. (2010). Psychotic-like experiences

and correlation with distress and depressive symptoms in a community sample of

adolescents and young adults. Schizophrenia Research, 119(1), 258-265.

Bagby, R. M., Parker, J. D. A., & Taylor, G. J. (1994). The twenty-item Toronto Alexithymia

Scale—I. Item selection and cross-validation of the factor structure. Journal of

Psychosomatic Research, 38(1), 23-32. doi: 10.1016/0022-3999(94)90005-1

Baker, R., Holloway, J., Thomas, P. W., Thomas, S., & Owens, M. (2004). Emotion

processing and panic. Behaviour Research & Therapy, 42, 1271-1287. doi:

10.1016/j.brat.2003.09.002



114


10.1016/j.jpsychores.2009.07.007

Baker, R., Thomas, S., Thomas, P. W., & Owens, M. (2007). Development of an emotional

processing scale. Journal of Psychosomatic Research, 62(2), 167-178. doi:

10.1016/j.jpsychores.2006.09.005

Barkus, Emma., Stirling, John., Hopkins, Richard., McKie, Shane., & Lewis, Shon. . (2007).

Cognitive and neural processes in non-clinical auditory hallucinations. The British

Journal of Psychiatry, 191, 76-81. doi: 10.1192/bjp.191.51.s76




10.1097/NMD.0b013e3181b3b20f

Bebbington, Paul, & Nayani, Tony. (1995). The Psychosis Screening Questionnaire.

International Journal of Methods in Psychiatric Research, 5(1), 11-19.

Beblo, T., Pastuszak, A., Griepenstroh, J., Fernando, S., Driessen, M., Schütz, A., . . .

Schlosser, N. (2010). Self-reported emotional dysregulation but no impairment of

emotional intelligence in borderline personality disorder. Journal of Nervous and

Mental Disease, 198, 385-388. doi: 10.1097/NMD.0b013e3181da4b4f


Emotion, 18(6), 859-867. doi: 10.1080/02699930341000220


107, 238-246. doi: 10.1037/0033-2909.107.2.238

Bond, F. W., Hayes, S. C., Baer, R. A., Carpenter, K M., Guenole, N., Orcutt, H. K., . . .

Zettle, R. D. (2011). Preliminary Psychometric Properties of the Acceptance and

Action Questionnaire–II: A Revised Measure of Psychological Inflexibility and

115

Experiential Avoidance. Behavior Therapy, 42(4), 676-688. doi:

10.1016/j.beth.2011.03.007

Brans, K., Koval, P., Verduyn, P., Lim, Y L., & Kuppens, P. (2013). The regulation of

negative and positive affect in daily life. Emotion, 13, 926-939. doi:

10.1037/a0032400


Publications, Inc.

Bydlowski, S., Corcos, M., Jeammet, P., Paterniti, S, Berthoz, S., Laurier, C., . . . Consoli, S.

M. (2005). Emotion-processing deficits in eating disorders. International Journal of

Eating Disorders, 37, 321-329. doi: 10.1002/eat.20132

Cattell, R. B. (1966). The scree test for the number of factors. Multivariate Behavioural

Research, 1, 245-276. doi: 10.1207/s15327906mbr0102_10

Cella, M., Cooper, A., Dymond, S., & Reed, P. (2008). The relationship between dysphoria

and proneness to hallucination and delusions among young adults. Comprehensive

Psychiatry, 49, 544-550. doi: 10.1016/j.comppsych.2008.02.011

Chadwick, P, & Birchwood, M. (1994). The omnipotence of voices: a cognitive approach to

hallucinations. British Journal of Psychiatry, 164, 190-201.

Charles, S. T., Reynolds, C. A., & Gatz, M. (2001). Age-related differences and changes in

positive and negative affect over 23 years. Journal of Personality and Social

Psychology, 80, 136-151.

Clara, I. P., Cox, I. P., & Enns, Murray W. (2001). Confirmatory factor analysis of the

depression anxiety stress scales in depressed and anxious patients. Journal of

Psychopathology and Behavioral Assessment, 23, 61-67. doi:

10.1023/A:1011095624717

116

Clark, D. M. (1999). Anxiety disorders: why they persist and how to treat them. Behaviour

Research and Therapy, 37, 5-27. doi: 10.1016/S0005-7967(99)00048-0

DeCarlo, L. T. (1997). On the meaning and use of kurtosis. Psychological Methods, 2, 292-

307. doi: 10.1037/1082-989X.2.3.292

Dennis, T. A. (2007). Interactions between emotion regulation strategies and affective style:

Implications for trait anxiety versus depressed mood. Motivation & Emotion, 31(3),

200-207. doi: 10.1007/s11031-007-9069-6

Docherty, J. P., Van Kammen, D. P., Siris, S. G., & Marder, S. R. (1978). Stages of onset of

schizophrenia psychosis. American Journal of Psychiatry, 135, 420-426.

Elbeze Rimasson, D., & Gay, M. C. (2012). Le fonctionnement émotionnel lors de la douleur

chronique : état de la question [Emotional processing in chronic pain: The state of the

art]. Annales Médico-psychologiques, revue psychiatrique, 170, 163-168. doi:

10.1016/j.amp.2011.03.015

Fowler, D., Freeman, D., Smith, B., Kuipers, E., Bebbington, P., Bashforth, H., . . . Garety, P.

(2006). The Brief Core Schema Scales (BCSS): psychometric properties and

associations with paranoia and grandiosity in non-clinical and psychosis samples.

Psychological Medicine, 36, 749-759.

Freeman, D., & Garety, P. (1999). Worry, worry processes and dimensions of delusions: An

exploratory investigation of a role for anxiety processed in the maintenance of

delusional distress. Cambridge University Press, 27, 47-52.

Freeman, D., & Garety, P. A. (2003). Connecting neurosis and psychosis: the direct influence

of emotion on delusions and hallucinations. Behaviour Research and Therapy, 41(8),

923-947. doi: 10.1016/s0005-7967(02)00104-3

117

Freeman, D., Garety, P. A., Kuipers, E., Fowler, D., & Bebbington, P. E. (2002). A cognitive

model of persecutory delusions. The British Psychological Society, 41, 331-347. doi:

10.1348/014466502760387461

Garety, P. A., Kuipers, E., Fowler, D., Freeman, D., & Bebbington, P. E. (2001). A cognitive

model of the positive symptoms of psychosis. Psychological Medicine, 31(2), 189-

195.






Hanssen, M., Bak, M., Bijl, R., Vollebergh, W., & van Os, J. (2005). The incidence and

outcome of subclinical psychotic experiences in the general population. British

Jouranl of Clinical Psychology, 44(Pt 2), 181-191. doi: 10.1348/014466505X29611

Hanssen, M., Krabbendam, L., Vollema, M., Delespaul, P., & Van Os, J. (2006). Evidence

for instrument and family-specific variation of subclinical psychosis dimensions in the

general population. Journal of Abnormal Psychology, 115(1), 5-14.

Henry, J. D., & Crawford, J. R. (2005). The short-form version of the Depression Anxiety

Stress Scales (DASS-21): Construct validity and normative data in a large non-

clinical sample. The British Journal of Clinical Psychology, 44(Pt 2), 227-239. doi:

10.1348/014466505X29657

Henry, J. D., Green, M. J., de Lucia, A., Restuccia, C., McDonald, S., & O'Donnell, M.

(2007). Emotion dysregulation in schizophrenia: Reduced amplification of emotional

expression is associated with emotional blunting. Schizophrenia Research, 95(1–3),

197-204. doi: 10.1016/j.schres.2007.06.002

118

Henry, J. D., Green, M. J., Restuccia, C., de Lucia, A., Rendell, P. G., McDonald, S., &

Grisham, J. R. (2009). Emotion dysregulation and schizotypy. Psychiatry Research,

166(2-3), 116-124. doi: 10.1016/j.psychres.2008.01.007




10.1037/0021-843X.117.2.473




Horn, J. L. (1965). A rationale and test for the number of factors in factor analysis.

Psychometrika, 30, 179-185. doi: 10.1007/BF02289447




Janssen, I., Krabbendam, L., Bak, M., Hanssen, M., Vollebergh, W., de Graaf, R., & van Os,

J. (2004). Childhood abuse as a risk factor for psychotic experiences. Acta

Psychiatrica Scandinavica, 109(1), 38-45. doi: 10.1046/j.0001-690X.2003.00217.x

Johansson, R., Björklund, M., Hornborg, C., Karlsson, S., Hesser, H., Ljótsson, B., . . .

Andersson, G. (2013). Affect-focused psychodynamic psychotherapy for depression

and anxiety through the Internet: a randomized controlled trial. PeerJ, 1. doi:

10.7717/peerj.102

Johns, L C., Cannon, M., Singleton, N., Murray, R M., Farrell, M., Brugha, T., . . . Meltzer,

H. (2004). Prevalence and correlates of self-reported psychotic symptoms in the

119

British population. British Journal of Psychiatry, 185, 298-305. doi:

10.1192/bjp.185.4.298



344(8934), 1398-1402. doi: 10.1016/S0140-6736(94)90569-X





Kaiser, H. F. (1960). The application of electronic computers to factor analysis. Educational

and Psychological Measurement, 20, 141-151. doi: 10.1177/001316446002000116

Kang, J. I., Namkoong, K., Yoo, S. W., Jhung, K., & Kim, S. J. (2012). Abnormalities of

emotional awareness and perception in patients with obsessive–compulsive disorder.

Journal of Affective Disorders, 141, 286-293. doi: 10.1016/j.jad.2012.04.001

Kealy, D., Ogrodniczuk, J. S., & Howell-Jones, G. (2011). Object relations and emotional

processing deficits among psychiatric outpatients. Journal of nervous and Mental

Disease, 199(2), 132-135. doi: 10.1097/NMD.0b013e3182083162

Kemeny, M, Foltz, C, Cavanagh, J. F., Cullen, M, Giese-Davis, J, Jennings, P, . . .

AlanEkman, P. (2012). Contemplative/emotion training reduces negative emotional

behaviour and propotes prosocial responses. Emotion, 12, 338-350. doi:

10.1037/a0026118

Kendler, K. S. Gallagher, T. J. Abelson, J. M., & Kessler, R. C. (1996). Lifetime prevalence,

demograpic risk factors, and diagnistic validity of nonaffective psychosis as assessed

in a US community sample: The National Comorbidity Survey. Archives of General

Psychiatry, 53(11), 1022-1031.

120


& Emotion, 23(1), 4-41. doi: 10.1080/02699930802619031

Krabbendam, Lydia, Janssen, Ilse, Bak, Maarten, Bijl, Rob V., de Graaf, Ron, & Jim van, Os.

(2002). Neuroticism and low self-esteem as risk factors for psychosis. Social

Psychiatry and Psychiatric Epidemiology, 37(1), 1-6. doi: 10.1007/s127-002-8207-y




Kumpula, M. J., Orcutt, H. K., Bardeen, J. R., & Varkovitzky, R. L. (2011). Peritraumatic

dissociation and experiential avoidance as prospective predictors of posttraumatic

stress symptoms., 120, 617-627. doi: 10.1037/a0023927

Lance, C. E., Butts, M. M., & Michels, L. C. (2006). The Sources of Four Commonly

Reported Cutoff Criteria: What Did They Really Say? Organizational Research

Methods, 9(2), 202-220. doi: 10.1177/1094428105284919

Lane, R. D., Quinlan, D. M., Schwartz, G. E., Walker, P. A., & Zeitlin, S. B. (1990). The

Levels of Emotional Awareness Scale: A Cognitive-Developmental Measure of

Emotion. Journal of Personality Assessment, 55, 124-134. doi:

10.1097/NMD.0b013e3182083162

Larøi, F. (2012). How do auditory verbal hallucinations in patients differ from those in non-

patients. Frontiers in Human Neuroscience., 6, 1-9. doi: 10.3389/fnhum.2012.00025

Lataster, Tineke, van Os, J., Drukker, Marjan, Henquet, Cécile, Feron, Frans, Gunther,

Nicole, & Myin-Germeys, I. (2006). Childhood victimisation and developmental

expression of non-clinical delusional ideation and hallucinatory experiences. Social

Psychiatry and Psychiatric Epidemiology, 41(6), 423.

121

Ledesma, R. D., & Valero-Mora, P. (2007). Determining the number of factors to retain in

EFA: an easy-to-use computer program for carrying out paralell analysis. Practical

Assessment, Research & Evaluation, 12, 1-11. doi: Available online:


Linney, Y., Murray, R., Peters, E., MacDonald, A., Rijsdijk, F., & Sham, P. (2003). A

quantitative genetic analysis of schizotypal personality traits. Psychological Medicine,

33(5), 803.

Livingstone, K., Harper, S., & Gillanders, D. (2009). An exploration of emotion regulation in

psychosis. Clinical Psychology & Psychotherapy, 16(5), 418-430. doi:

10.1002/cpp.635

Loewy, Rachel L., Johnson, Jennifer K., & Cannon, Tyrone D. (2007). Self-report of

attenuated psychotic experiences in a college population. Schizophrenia Research,

93(1-3), 144-151.

Looney, S. W. (1995). How to use tests for univariate normality to assess multivariate

normality. The American Statistician, 49, 64-70. doi:

10.1080/00031305.1995.10476117




doi: 10.1016/0005-7967(94)00075-U

Mackie, C. J., Castellanos-Ryan, N., & Conrod, P. J. (2011). Developmental trajectories of

psychotic-like experiences across adolescence: impact of victimization and substance

use. Psychological Medicine, 41(1), 47-58.

Malmberg, A., Lewis, G., David, A., & Allebeck, P. (1998). 1998. The British Journal of

Psychiatry, 172, 308-313.


122

Martin, James A. , & Penn, David L. (2001). Social cognition and subclinical paranoid

ideation. British Journal of Clinical Psychology, 40(3), 261.



10.1080/10253890310001594487

Modinos, Gemma, Ormel, Johan, & Aleman, A. (2010). Altered activation and functional

connectivity of neural symptoms supporting cognitive control of emotion in psychosis

proneness. Schizophrenia Research, 118, 88-97. doi: 10.1016/j.schres.2010.01.030

Muthén, L. K, & Muthén, B. O. (2010). Mplus User's Guide (6th ed.). Los Angeles, CA:

Muthén & Muthén.


Paulik, G., Badcock, J. C., & Maybery, M. T. (2006). The multifactorial structure of the

predisposition to hallucinate and associations with anxiety, depression and stress.


10.1016/j.paid.2006.04.012



54(5), 504-514. doi: 10.1016/s0006-3223(03)00168-9

Poulton, R., Caspi, A., Moffitt, T. E., Cannon, M., Murray, R., & Harrington, H. (2000).

Children's self-reported psychotic symptoms and adult schizophreniform disorder: a

15-year longitudinal study. Archives of General Psychiatry, 57(11), 1053-1058.

Rachman, S. (1980). Emotion processing. Behaviour Research & Therapy, 18, 51-60. doi:

10.1016/0005-7967(80)90069-8

123

Rachman, S. (2001). Emotion processing, with special reference to post-traumatic stress

disorder. International Review of Psychiatry, 13, 164-171. doi:

10.1080/09540260120074028



6(3), 19-25. doi: 10.2196/jmir.6.3.e29

Schmitt, T. A. (2011). Current methodological considerations in exploratory and

confirmatory factor analysis. Journal of Psychoeducational Assessment, 29, 304-321.

doi: 10.1177/0734282911406653

Scott, J., Welham, J., Martin, G., Bor, W., Najman, K., O'callaghan, M., . . . McGrath, J.

(2008). Demographic correlates of psychotic-like experiences in young Australian

adults. Acta Psychiatrica Scandinavica, 118, 230-237. doi:

10.1111/j.1600‐0447.2008.01214.x



101(1-3), 195-200. doi: 10.1016/j.schres.2008.01.012

Sexton, K. A., & Dugas, M. J. (2008). The Cognitive Avoidance Questionnaire: Validation of

the English translation. Journal of Anxiety Disorders, 22(3), 355-370. doi:

10.1016/j.janxdis.2007.04.005

Small, N. J. H. (1980). Marginal skewness and kurtosis in testing multivariate normality.

Journal of Royal Statistical Society, Series C (Applied Statistics), 29, 85-87.

Smith, Matthew J., Thirthalli, Jagadisha, Abdallah, Arbi Ben, Murray, Robin M., & Cottler,

Linda B. (2009). Prevalence of psychotic symptoms in substance users: a comparison

across substances. Comprehensive Psychiatry, 50(3), 245-250. doi:

10.1016/j.comppsych.2008.07.009

124

Spaapen, D. L., Waters, F. Brummer, L., & Stopa, L., Bucks, R. S. (2014). The Emotion

Regulation Questionnaire: Validation of the ERQ-9 in Two Community Samples.

Psychological Assessment, 26, 46-54. doi: 10.1037/a0034474

Spauwen, J., Krabbendam, L., Lieb, R., Wittchen, H. U., & van Os, J. (2006). Impact of

psychological trauma on the development of psychotic symptoms: relationship with

psychosis proneness. British Journal of Psychiatry, 188, 527-533.

Stefanis, Nicholas C., Os, J. V., Avramopoulos, Dimitrios, Smyrnis, Nikolaos, Evdokimidis,

Ioannis, Hantoumi, Ioanna, & Stefanis, Costas N. (2004). Variation in catechol-o-

methyltransferase val158 met genotype associated with schizotypy but not cognition:

A population study in 543 young men. Biological Psychiatry, 56(7), 510-515. doi:

10.1016/j.biopsych.2004.06.038



Iowa City, IA.

Tabachnik, B.G., & Fidell, L S. (2013). Using multivariate statistics (6th Ed ed.). Boston:

Pearson Education.

Taylor, K. N., Graves, A., & Stopa, L. (2009). Strategic Cognition in Paranoia: The Use of

Thought Control Strategies in a Non-Clinical Population. Behavioural and Cognitive

Psychotherapy, 37(01), 25-38. doi: doi:10.1017/S1352465808005006

Tien, A. Y., & Anthony, J. C. (1990). Epidemiological analysis of alcohol and drug use as

risk factors for psychotic experiences. Journal of Nervous and Mental Disease, 178,

473-480.

Tien, A. Y., & Eaton, W. W. (1992). Psychopathologic precursors and sociodemographic risk

factors for the schizophrenia syndrome. Archives of General Psychiatry, 49, 37-46.

125

Tremeau, Fabien. (2006). A review of emotion deficits in schizophrenia. Dialogues in

Clinical Neuroscience, 8(1), 59-70.

Trower, Peter, & Chadwick, Paul. (1995). Pathways to Defense of the Self: A Theory of Two

Types of Paranoia. Clinical Psychology: Science and Practice, 2(3), 263-278. doi:

10.1111/j.1468-2850.1995.tb00044.x



van der Meer, L., van 't Wout, M., & Aleman, A. (2009). Emotion regulation strategies in

patients with schizophrenia. Psychiatry Res, 170(2-3), 108-113. doi:

10.1016/j.psychres.2009.07.010

van Os, J., Linscott, R. J., Myin-Germeys, I., Delespaul, P., & Krabbendam, L. (2009). A

systematic review and meta-analysis of the psychosis continuum: evidence for a

psychosis proneness-persistence-impairment model of psychotic disorder.

Psychological Medicine, 39(2), 179-195.

Velicer, W. F, Eaton, C. A, & Fava, J. L. (2000). Construct explication through factor or

component analysis: A review and evaluation of alternative procedures for

determining the number of factors or components. In R. D. Goffin & E. Helmes

(Eds.), Problems and solutions in human assessment: Honoring Douglas N. Jackson

at seventy (pp. 41-71). Boston: Kluwer Academic.

Vorst, H. C. M., & Bermond, B. (2001). Validity and reliability of the Bermond–Vorst

Alexithymia Questionnaire. Personality and Individual Differences, 30(3), 413-434.

doi: 10.1016/S0191-8869(00)00033-7

Wegner, D. M., & Zanakos, S. (1994). Chronic Thought Suppression. Journal of Personality,

62(4), 615-640. doi: 10.1111/j.1467-6494.1994.tb00311.x

126

Westermann, S., Boden, M T., Gross, J J., & Lincoln, T M. (2013). Maladaptive cognitive

emotion regulation prospectively predicts subclinical paranoia. Cognitive Therapy &

Research, 2013, 881-885. doi: 10.1007/s10608-013-9523-6




10.1016/j.beth.2011.07.005

Westermann, S., & Lincoln, T M. (2011). Emotion regulation difficulties are relevant to

persecutory delusion. Psychology and Psychotherapy: Theory, Research and

Practice., 84, 273-287. doi: 10.1348/147608310X523019

Wiles, Nicola J., Zammit, Stanley, Bebbington, Paul, Singleton, Nicola, Meltzer, Howard, &

Lewis, Glyn. (2006). Self-reported psychotic symptoms in the general population:

results from the longitudinal study of the British National Psychiatric Morbidity

Survey. British Journal of Psychiatry, 188, 519-526.

Wilkins, C. (2012). A longitudinal study of women's management of emotions during

pregnancy and the association with postnatal depression [PhD thesis]. (PhD thesis),

Bournemouth University, Dorset, United Kingdom.




10.1055/s-0031-1272451

Yung, A. R., & McGorry, P. D. (1996). The prodromal phase of first-episode psychosis: past

and current conceptualizations. Schizophrenia Bulletin, 22, 353-370.

Zigmond, A. S., & Snaith, R. P. (1983). The Hospital Anxiety and Depression Scale. Acta

Psychiatrica Scandinavica, 67(6), 361-370. doi: 10.1111/j.1600-0447.1983.tb09716.x

127

Zwick, W. R., & Velicer, W. F. (1986). Comparison of five rules for determining the number

of components to retain. Psychological Bulletin, 99, 432-442. doi: 10.1037/0033-

2909.99.3.432

128

Chapter 5: General discussion

The aim of the thesis was to explore emotion processing, and its association with

PLE. This is important because individuals with PLE frequently have negative affect, and

studies have demonstrated a relationship between poor emotion processing and negative

affect. The broad objective of my thesis was therefore to examine whether PLE in the general

community were linked to negative affect and difficulties with emotional processing. This is

significant because individuals with PLE may be at ‘high risk’ to make a transition to

psychosis. The identification of the specific type of emotion processing dysfunctions in PLE

is therefore likely to be informative when putting in place intervention strategies aimed at

reducing the vulnerability of these individuals in developing psychosis.

The tasks chosen to assess emotion processing included the Emotion Regulation

Questionnaire (ERQ) and the Emotion Processing Scale (EPS), which are frequently used and

generally well accepted in the emotion processing literature. However, given that the ERQ

and EPS have not yet been well validated, my first task was to assess their psychometric

properties in community samples. Without this assessment, inferences made based on scale

scores may be inaccurate and misguided. By achieving validation of the tasks, I hoped to use

the best tasks possible to examine the relationship between PLE and emotion processing.

Therefore, the psychometric assessment of the ERQ is covered in Chapter 2, and the EPS is

assessed in Chapter 3. In chapter 4, I specifically assessed the relationships between emotion

regulation, negative affect, and PLE.

5.1 Chapter 2

The 10-item Emotion Regulation Questionnaire (ERQ) was developed by Gross and

John (2003) to measure the habitual use of two emotion regulation strategies –reappraisal and

129

suppression. As a preliminary study to our investigations of emotion processing in PLE, a

confirmatory factor analysis (CFA) on the ERQ was carried out on two large community

samples (Australia, N = 550; United Kingdom, N = 483; 17–95 years of age). The original

factor ERQ structure was not supported by either sample. However, with the removal of one

item, a strong model fit was obtained for both samples (revised ERQ-9). Using measurement

invariance tests, the revised ERQ-9 was found to be equivalent across the samples and

demographics (age, gender and education). In other words, people who varied by country and

demographics did not differ in how they interpreted and scored themselves on the ERQ-9.

This lends support to the external validity of the ERQ-9. Furthermore, there was an

association between gender, depression, anxiety and stress (DASS), whereby males and

higher DASS scores were associated with increased suppression use, and females and lower

DASS scores were associated with greater reappraisal scores. Whilst it was recommended

that these findings are replicated in other community samples, the results suggest that the

revised ERQ-9 is a robust measure of reappraisal and suppression in community samples.

5.2 Chapter 3

The Emotion Processing Scale (EPS-25) was developed by Baker et al. (2010) to

identify difficulties in the processing of emotions Similar to Chapter 2, a CFA was conducted

on the EPS, using the same two community samples (Australia, N = 575; United Kingdom,

N = 597, age 17-95). In contrast to the previous chapter, the original factor structure of the

EPS could not be replicated in either the Australian or United Kingdom samples, and

attempts to revise the scale by re-specifying an optimal measurement model for the samples

were not successful. Despite all attempts to find a working model of the scale, problems with

significant widespread standardised residual covariances in the original and optimised model

could not be overcome.

130

This was a significant blow to this study, as the overarching conclusion of these

analyses was that the EPS could not be validated as a measure of emotion processing in

healthy populations, and that it was unstable and not suitable for use in community samples

until substantial scale refinement or redevelopment with subsequent validation is achieved.

Therefore, the EPS could not be used for subsequent analyses.

5.3 Chapter 4

After examining the psychometric properties of the ERQ and EPS, attention turned

towards the examination of whether emotion regulation and negative affect predicted PLE.

Given that the EPS could not be used, the focus was on the ERQ, and the measure of

depression anxiety and stress (DASS), in relation to PSQ.

The specific questions were: does negative affect predict PLE, does emotion

regulation predict negative affect, and does emotion regulation predict PLE (directly, and

indirectly.

This was achieved by through several structural equation models, with the final model

specifying all scales, with emotion regulation factors assigned as first-order predictors, and

the negative affect factors set as second-order predictors of PLE. Surprisingly, the only

significant predictor of PLE was the negative affect factor, generalised psychological distress,

which measured the common variance from depression, anxiety, and stress. Despite previous

research suggesting a relationship between emotion regulation (including reappraisal) and

PLE (Taylor, Graves, & Stopa, 2009; Westermann, Boden, Gross, & Lincoln, 2013;

Westermann, Kesting, & Lincoln, 2012), no direct or indirect association was observed. The

only relationship found between ER and negative affect was suppression predicting

depression. Other statistically significant results were observed, however the effect of these

results were practically negligible. It was suggested that future research examine a wider

131

range of ER strategies, especially those which are rarely adaptive, and related to

psychopathology (e.g. thought suppression).

5.4 Negative affect, emotion processing problems: relevance to PLE?

In summary, the findings of my thesis in regards to PLE and emotion processing were

as follows: First, my thesis showed a link between PLE and negative affect. This finding is

largely consistent with studies finding higher levels of depression, anxiety and stress predict

PLE (Cella, Cooper, Dymond, & Reed, 2008; Jones, Murray, Jones, Rodgers, & Marmot,

1994; Kugelmass et al., 1995; Paulik, Badcock, & Maybery, 2006). Specifically novel to the

current study, we also explored the contribution of unique and shared variance within

depression, anxiety and stress, as predictors of PLE. The results showed the generalised

distress factor (shared variance) to be the only predictor of PLE. This suggests that negative

affect predicting PLE may be best explained by a general predisposition to distress, rather

than specifically to negative symptoms such as depression, anxiety and stress.

Second, the only significant relationship between emotion regulation and negative

affect in the current study was a positive association between emotion suppression and

depression. This underscores a general trend in the literature that suppression is a better

predictor of depression, than anxiety or stress (Matheson & Anisman, 2003; Wiltink et al.,

2011). In contrast, previous studies have found reappraisal to be associated with decreased

negative affect (Gross & John, 2003; Wiltink et al., 2011). The difference in findings may be

due to studies using different negative affect measures (PANAS - mood, r = -.51, Gross &

John, 2003; HADS - depression, r = -.16, Wiltink et al., 2011; DASS – depression, r = -.12,

Chapter 4).

Third, the results showed that there was no direct, or indirect (via DASS), relationship

between emotion regulation and PLE. This is somewhat at odds with a previous report, which

132

reported that (contrary to expectations) adaptive ER strategies (e.g. reappraisal), but not

maladaptive strategies (e.g. reappraisal) were associated with PLE (Westermann et al., 2012),

and a later (contradictory) finding by the same author- that maladaptive, but not adaptive ER,

predicted PLE (Westermann et al., 2013). Generally, it is difficult to extrapolate any trends

from the literature, because existing studies varying considerably in how they capture PLE

and ER, and the nature of the samples investigated.

Overall, on the basis of the current results, and the tasks used in this study, I can

conclude that emotion processing difficulties are not directly related to PLE as assessed using

our measures. However, there may be alternative explanations for this finding, including

limitations regarding measurement and sampling methods. As we discussed in chapter 4, the

current measure of emotion regulation (ERQ) is very limited in content. Including a more

comprehensive set of ER strategies, particularly those associated with psychopathology (e.g.

rumination, avoidance, and thought suppression) may return different results. In addition, the

use of other emotion processing scales would be worthwhile. The study sample also was

relatively homogenous for PLE, however, the sample distribution was expected for

measuring sub-clinical presentations of mental illness. Given the large sample size, we

considered sample composition is unlikely to significantly influence the results.

5.5 Limitations

In addition to the limitations discussed above, other limitations of this research

included: cross-sectional data, scale timeframes, and no measure of emotion

perception/awareness. The cross-sectional nature of the data also makes it difficult to argue

the causality of associations. The current study has drawn on previous experimental and

prospective research which found maladaptive ER increases and negative affect, and

persistent negative affect increases PLE, to argue the direction of relationships. However,

133

there are also findings that suggest psychotic symptoms increase negative affect (Kuipers et

al., 2006; Smith et al., 2006), and negative affect influences the effectiveness and choice of

emotion processing (Joormann & Gotlib, 2010; Phillips, Drevets, Rauch, & Lane, 2003) . To

our knowledge, no experimental or prospective studies have incorporated emotion

processing, negative affect, and PLE simultaneously. Given the apparent reciprocal

relationship between aspects of emotion processing, negative affect and PLE, experimental

and prospective studies encompassing all three factors is recommended.

The measurement time frame of the scales may also have influenced results. The PSQ

measures PLE in the last previous year, the DASS measures negative affect in the past week,

whilst the ERQ has no set time frame. Indeed, the larger the discrepancy in the time frame

between the traits captured, the more likely any relationship between them is spurious. For

example, it is possible that high negative affect scores reflect a transitional stressful event,

with the individual otherwise being low on negative affect. It is expected that longer term or

trait negative affect is more likely to influence PLE (Docherty, St-Hilaire, Aakre, & Seghers,

2008; Rössler, Hengartner, Ajdacic-Gross, Haker, & Angst, 2013), so in this example,

transitional scores may suppress what would otherwise be a stronger relationship between

negative affect and PLE.

The study was also limited by the lack of validation of the EPS-25. As a measure

which tapped emotion perception, regulation and experience, it would have allowed for a

more comprehensive assessment of how emotion processing relates to PLE (specifically

emotion perception). However, it was fortuitous that we conducted a validation study before

exploring any associations, otherwise any inferences may have been unsound.

5.6 Areas for future research

134

It is recommended that future studies of emotion processing and PLE include

measures of dysfunctional ER (e.g. intrusion of thoughts) and a more comprehensive set of

emotion regulation strategies, particular those associated with psychopathology (e.g.

rumination and thought suppression). Accounting for the unique and shared variance between

negative affect scales may also elucidate to what extent negative affect types influence PLE

onset and maintenance. This may help with the development of optimal screening and

preventative measures. Furthermore, future studies may benefit from obtaining larger samples

sizes, or sampling groups with more distinct PLE clusters (e.g. paranoia and hallucinations),

and the use of a PLE scale with stricter symptom inclusion. This would allow for the

exploration of whether aspects of emotion processing differentially influence PLE clusters.

135

5.7 References




10.1016/j.jpsychores.2009.07.007

Bebbington, Paul, & Nayani, Tony. (1995). The Psychosis Screening Questionnaire.

International Journal of Methods in Psychiatric Research, 5(1), 11-19.

Cella, M., Cooper, A., Dymond, S., & Reed, P. (2008). The relationship between dysphoria

and proneness to hallucination and delusions among young adults. Comprehensive

Psychiatry, 49, 544-550. doi: 10.1016/j.comppsych.2008.02.011

Docherty, N. M., St-Hilaire, A., Aakre, J. M., & Seghers, J. P. (2008). Life events and high-

trait reactivity together predict psychotic symptom increases in schizophrenia.

Schizophrenia Bulletin, 35, 638-645. doi: 10.1093/schbul/sbn002






344(8934), 1398-1402. doi: 10.1016/S0140-6736(94)90569-X






136

Kuipers, E., Garety, P., Fowler, D., Freeman, D., Dunn, G., & Bebbington, P. (2006).

Cognitive, emotional, and social processes in psychosis: refining cognitive behavioral

therapy for persistent positive symptoms. Schizophrenia Bulletin, 32, 24-31. doi:

10.1093/schbul/sbl014




doi: 10.1016/0005-7967(94)00075-U



10.1080/10253890310001594487

Paulik, G., Badcock, J. C., & Maybery, M. T. (2006). The multifactorial structure of the

predisposition to hallucinate and associations with anxiety, depression and stress.


10.1016/j.paid.2006.04.012



54(5), 504-514. doi: 10.1016/s0006-3223(03)00168-9

Rössler, W., Hengartner, M. P., Ajdacic-Gross, V., Haker, H., & Angst, J. (2013).

Deconstructing sub-clinical psychosis into latent-state and trait variables over a 30-

year time span. Schizophrenia Research, 150, 197-204. doi:

10.1016/j.schres.2013.07.042

Smith, B., Fowler, D. G., Freeman, D., Bebbington, P., Bashforth, H., Garety, P., . . .

Kuipers, E. (2006). Emotion and psychosis: links between depression, self-esteem,

137

negative schematic beliefs and delusions and hallucinations. Schizophrenia Research,

86(1-3), 181-188. doi: 10.1016/j.schres.2006.06.018

Spaapen, D. L., Waters, F. Brummer, L., & Stopa, L., Bucks, R. S. (2014). The Emotion

Regulation Questionnaire: Validation of the ERQ-9 in Two Community Samples.

Psychological Assessment, 26, 46-54. doi: 10.1037/a0034474

Taylor, K. N., Graves, A., & Stopa, L. (2009). Strategic Cognition in Paranoia: The Use of

Thought Control Strategies in a Non-Clinical Population. Behavioural and Cognitive

Psychotherapy, 37(01), 25-38. doi: doi:10.1017/S1352465808005006

Westermann, S., Boden, M T., Gross, J J., & Lincoln, T M. (2013). Maladaptive cognitive

emotion regulation prospectively predicts subclinical paranoia. Cognitive Therapy &

Research, 2013, 881-885. doi: 10.1007/s10608-013-9523-6




10.1016/j.beth.2011.07.005




10.1055/s-0031-1272451

138

5.8 Appendices and supplementary materials 5.8.1 Chapter 2

Appendix A

The following tables and figures were provided to the reviewers, and are available as an

online resource at

http://supp.apa.org/psycarticles/supplemental/a0034474/a0034474_supp.html

Figure 1. AUS sample: Standardized factor loadings for CFA.

Appendix B

http://supp.apa.org/psycarticles/supplemental/a0034474/a0034474_supp.html

139

Figure 2. UK sample: Standardized factor loadings for CFA.

140

Appendix C

Figure 3. AUS sample: Modified Standardized factor loadings for CFA with item-3 deletion.

141

Appendix D

Figure 4. UK sample: Modified standardized factor loadings for CFA with Item-3 deletion.

142

Appendix E

Revised 9-item Emotion Regulation Questionnaire (9-item ERQ)

Scale Items

Reappraisal 1 - When I want to feel more positive (such as joy or amusement), I change

what I’m thinking about.

5 – When I’m faced with a stressful situation, I make myself think about it

in a way that helps me calm down.

7 – When I want to feel more positive emotion, I change the way I’m

thinking about the situation.

8 – I control my emotions by changing the way I think about the situation

I’m in.

10 – When I want to feel less negative emotion, I change the way I’m

thinking about the situation.

Suppression 2 - I keep my emotions to myself

4 – When I am feeling positive emotions, I am careful not to express them.

6 – I control my emotions by not expressing them

9 – When I am feeling negative emotions, I make sure not to express them.

143

Appendix F

Nested Model Comparisons for invariance test between Australian and United Kingdom

community samples.

Model ∆df ∆ χ² Sig. ∆TLI ∆CFI ∆RMSEA

Equal factor loadings 7 6.589 .473 .004 0 .002

Equal means/intercepts 9 14.144 .117 0 .001 .001

Equal covariance 3 6.920 .074 0 .002** .001

Equal Error 9 39.832 .001* .007 .011** .004

Note. * = p < .05; ** = ∆CFI > .002; Each row of this table is assuming that the previous

row is invariant. E.g. the Equal FL statistics are assuming the unconstrained model is

invariant. TLI = Tucker-Lewis Index, CFI = comparative fit index, RMSEA = root mean

square error of approximation. Maximum Likelihood (ML) estimation was used. Maximum

Likelihood (ML) estimation was used because it is a more thorough method to examine the

extent of invariance (compared to asymptotic distribution free - ADF).

144

Appendix G

Nested Model Comparisons for invariance tests for Age, Education and Gender in the


Model ∆df ∆ χ² Sig. ∆TLI ∆CFI ∆RMSEA

Age

Equal factor loadings 7 11.232 .129 .003 .002** .002

Equal means/intercepts 9 24.591 .003* .002 .005** .001

Equal covariance 3 2.497 .476 .002 <.001** <.001

Equal Error 9 14.385 .109 .002 .002** .001

Education

Equal factor loadings 7 16.335 .022* <.001 .003** <.001

Equal means/intercepts 9 29.414 .001* .003 .007** .001

Equal covariance 3 14.173 .003* .003 .004** .002

Equal Error 9 27.596 .001* .001 .007** <.001

Gender

Equal factor loadings 7 9.775 .202 .003 .001 .001

Equal means/intercepts 9 34.794 <.001* .006 .009** .003

Equal covariance 3 4.229 .238 .001 .001 .001

Equal Error 9 32.027 <.001* .004 .008** .002

Note. * = p < .05; ** = ∆CFI > .002; Each row of this table is assuming that the previous

row is invariant. E.g. the Equal FL statistics are assuming the unconstrained model is

invariant. TLI = Tucker-Lewis Index, CFI = comparative fit index, RMSEA = root mean

square error of approximation. Maximum Likelihood (ML) estimation was used. Maximum

145

Likelihood (ML) estimation was used because it is a more thorough method to examine the

extent of invariance (compared to asymptotic distribution free - ADF).

146

Appendix H

Effects of Non-Equivalence on Scale-Level Properties

Scale ∆ mean Observed

mean

difference

∆ Var Observed

variance

difference

dMACS

min - max

dMACS

Factor-

level

Age

Reappraisal -0.81 -0.67 9.95 -1.26 .11 - .49 .17

Suppression 0.70 0.03 -18.93 -3.25 .09 - .44 .15

Education

Reappraisal -0.49 -0.37 6.11 8.04 .02 - .39 .10

Suppression -0.17 0.93 10.79 5.68 .14 - .30 .04

Gender

Reappraisal 0.18 -0.88 -16.82 3.73 .05 - .38 .04

Suppression 0.49 1.43 4.49 1.60 .09 - .22 .11

Note. ∆ mean = the difference in means that can be attributed to items functioning differently

between the analysed groups due to non-equivalence. ∆ Var = the difference in variance that

can be attributed to items functioning differently between the analysed groups due to non-

equivalence. The observed mean and variance differences are the true differences between

the groups. Positive values refer to the reference group having higher mean/variance than the

focal group. Negative values imply that the focal has higher mean/variance than the reference

group. Reference groups: Age, Younger; Education, Lower; Gender, Male. Focal groups:

Age, Older; Education, Higher; Gender, Female. dMACS = effect size of mean and

147

covariance structure. Factor-level dMACS is calculated by dividing the ∆ mean by the pooled

standard deviation.

148

Appendix I

Revised multi-group analysis (9-item ERQ) - invariance test for age and education with the



Age

Unconstrained 130 219.805 <.001* 1.579 .036 .962 .973 .024 .017 .030

Equal factor

loadings 151 255.428

<.001* 1.492

.045 .968 .973 .022 .016 .028

Equal

means/intercepts

178

324.264

<.001* 1.554

.046 .964 .964 .023 .018 .029


Equal Error 214 372.002 <.001* 1.528 .047 .966 .959 .023 .018 .028

Education

Unconstrained 78 155.364 <.001* 1.992 .038 .961 .972 .031 .024 .039

Equal factor

loadings 92 176.154

<.001* 1.915

.044 .964 .970 .030 .023 .037

Equal

means/intercepts

110

223.269

<.001* 2.030

.043 .960 .959 .032 .026 .038


Equal Error 134 285.215 <.001* 2.128 .049 .956 .945 .034 .028 .039

Note. Age = quintile split – age groups in years (17 – 20, 21 – 25, 26 – 41, 42 – 61, 62 - 95);

Education = 3 groups – designed to represent up to highschool (4 – 12),

undergraduate/tertiary (12.5 – 16), and postgraduate (16.5 – 27.25) years of education.

149

SRMR = standardised root mean-square residual, RMSEA = root mean square error of



(ML) estimation was used. Maximum Likelihood (ML) estimation was used because it is a

more thorough method to examine the extent of invariance (compared to asymptotic

distribution free - ADF).

150

Appendix J

Revised multiple (two-group) invariance testing by effect size

Effect size (min – max)

1 2 3 4 5

Age groups

Reappraisal

1

2

-

-

0.07 – 0.17

-

0.13 – 0.24

0.03 - 0.21

0.16 – 0.46

0.06 - 0.33

0.04 – 0.21

0.03 – 0.13

3 - - - 0.05 - 0.30 0.08 – 0.22

4 - - - - 0.04 – 0.28

5 - - - - -

Suppression

1

2

-

-

0.22 – 0.36

-

0.21 – 0.29

0.03 - 0.11

0.28 – 0.46

0.02 - 0.17

0.10 – 0.27

0.16 – 0.48

3

4

5

Education groups

Reappraisal

1

2

3

Suppression

-

-

-

-

-

-

-

-

-

0.01 - 0.08

-

-

-

-

-

0.03 - 0.17

0.04 - 0.21

-

0.02 - 0.23

-

-

-

-

-

0.14 – 0.52

0.28 – 0.33

-

-

-

-

151

1

2

3

-

-

-

0.19 - 0.26

-

-

0.20 - 0.39

0.06 – 0.19

-

-

-

-

-

-

-

Note. Age groups (in years): 1) 17 – 20, 2) 21 – 25, 3) 26 – 41, 4) 42 – 61, 5) 62 – 95. Age

groups were determined by a quintile split. Education groups (in years): 1) 4 – 12, 2) 12.5 –

16, 3) 16.5 – 27.25. The three groups for the revised education-based invariance testing

represented up to high school (group 1), undergraduate/tertiary (group 2), and postgraduate

(group 3) education levels.

152

5.8.2 Chapter 3 Appendix A

EFA item loadings for UK, 5 factors specified.

Factor

Item 1 2 3 4 5

EPS1 .72

EPS6 .59

EPS11 .86

EPS16 .78

EPS21 .69

EPS2 .52

EPS7 .70

EPS12 .65

EPS17 .60

EPS22 .81

EPS3 .69

EPS8 .54

EPS13 .41

153

EPS18 .58

EPS23 .55

EPS4 .51

EPS9

EPS14 .44

ESP19 .30

EPS24 .74

EPS5 .65

EPS10 .5

EPS15

EPS20 .86

EPS25 .36

Note: Geomin rotation. Item loadings less than .30 were omitted.

154

Appendix B

EFA item loadings for UK. 2 factors specified.

Factor

Item 1 2

EPS1 .70

EPS6 .58

EPS11 .87

EPS16 .71

EPS21 .68

EPS2 .59

EPS7 .61

EPS12 .61

EPS17 .83

EPS22 .71

EPS3 .63

EPS8 .64

EPS13 .46

EPS18 .51

EPS23 .70

EPS4 - -

EPS9 .47

EPS14 - -

ESP19 .73

155

EPS24 .48

EPS5 .33 .38

EPS10 .54

EPS15 .57

EPS20 .44

EPS25 .43

Note: Geomin rotation. Item loadings less than .30 were omitted.

156

Appendix C.

Emotion Processing Scale (25-item EPS)

Scale Items

Suppression 1 – I smothered my feelings

6 – I could not express feelings

11 – I kept quiet about my feelings

16 – I bottled up my emotions

21 – I tried not to show my feelings to others

Unprocessed 2 – Unwanted feelings kept intruding

7 – My emotional reactions lasted for more than a day

12 – I tended to repeatedly experience the same emotion

17 – I felt overwhelmed by my emotions.

22 – I kept thinking about the same emotional situation again and again

Unregulated 3 – When upset or angry it was difficult to control what I said.

8 – I reacted too much to what people said or did

13 – I wanted to get my own back on someone

18 – I felt the urge to smash something

23 – It was hard for me to wind down

Avoidance 4 – I avoided looking at unpleasant things (e.g. on TV/in magazine)

9 – Talking about negative feelings seemed to make them worse

14 – I tried to talked only about pleasant feelings

19 – I could not tolerate unpleasant feelings

24 – I tried very hard to avoid things that might make me upset

Impoverished 5 – My emotions felt blunt/dull

157

10 – My feelings did not seem to belong to me

15 – It was hard to work out if I felt ill or emotional

20 – There seemed to be a big blank in my feelings

25 – Sometimes I got strong feelings but I’m not sure if their emotions

158

5.8.3 Chapter 4

Appendix A

Figure 2. Full Structural Equation Model. Dep = DASS Depression, Anx = DASS Anxiety,

Stress = DASS Stress, gd = generalised distress factor, PLE = PSQ.

emotion processing: relevance to psychotic-like symptoms

Documents