applying artificial neural networks to predict ... · original research: empirical research –...

ORIGINAL RESEARCH: EMPIRICAL RESEARCH –QUANTITATIVE

Applying artificial neural networks to predict communication risks in

the emergency department

Annamaria Bagnasco, Anna Siri, Giuseppe Aleo, Gennaro Rocco & Loredana Sasso

Accepted for publication 13 April 2015

Correspondence to A. Bagnasco:

e-mail: [email protected]

Annamaria Bagnasco MSC PhD

Nursing Researcher

Department of Health Sciences, University

of Genoa, Italy

Anna Siri PhD

Researcher

School of Medical and Pharmaceutical

Sciences, University of Genoa, Italy

Giuseppe Aleo MA

PhD Student in Epidemiology


of Genoa, Italy

Gennaro Rocco MSC RN NP

Director of the Centre of Excellence for

Nursing Scholarship

Centre of Excellence for Nursing

Scholarship, Rome, Italy

Loredana Sasso MA MSC RN

Associate Professor of Nursing


of Genoa, Italy

BAGNASCO A . , S IR I A . , ALEO G . , ROCCO G . & SAS SO L . ( 2 0 1 5 ) Applying arti-

ficial neural networks to predict communication risks in the emergency depart-

ment. Journal of Advanced Nursing 00(0), 000–000. doi: 10.1111/jan.12691

AbstractAims. To describe the utility of artificial neural networks in predicting

communication risks.

Background. In health care, effective communication reduces the risk of error.

Therefore, it is important to identify the predictive factors of effective

communication. Non-technical skills are needed to achieve effective

communication. This study explores how artificial neural networks can be applied

to predict the risk of communication failures in emergency departments.

Design. A multicentre observational study.

Methods. Data were collected between March–May 2011 by observing the

communication interactions of 840 nurses with their patients during their routine

activities in emergency departments. The tools used for our observation were a

questionnaire to collect personal and descriptive data, level of training and experience

and Guilbert’s observation grid, applying the Situation-Background-Assessment-

Recommendation technique to communication in emergency departments.

Results. A total of 840 observations were made on the nurses working in the

emergency departments. Based on Guilbert’s observation grid, the output

variables is likely to influence the risk of communication failure were

‘terminology’; ‘listening’; ‘attention’ and ‘clarity’, whereas nurses’ personal

characteristics were used as input variables in the artificial neural network model.

A model based on the multilayer perceptron topology was developed and trained.

The receiver operator characteristic analysis confirmed that the artificial neural

network model correctly predicted the performance of more than 80% of the

communication failures.

Conclusion. The application of the artificial neural network model could offer a

valid tool to forecast and prevent harmful communication errors in the

emergency department.

Keywords: artificial neural network, communication skills, emergency depart-

ment, nursing, prediction, risk

© 2015 John Wiley & Sons Ltd 1

Introduction

Several studies on communication relations in Emergency

Departments report that approximately 89% of the profes-

sional activities require good communication skills (Spencer et al.

2002) and that the use of standardized communication tech-

niques, such as the Situation-Background-Assessment-Recom-

mendation (SBAR) technique (Haig et al. 2006), in clinical

situations requiring immediate interventions, is mandatory for

patient safety (Meginniss et al. 2012). Clear and complete com-

munication and the constant check that communication is prop-

erly understood are instrumental to ensure patient safety and

the quality of care (Spencer et al. 2002, Jones et al. 2013).

A study, previously conducted by our research group, per-

mitted to identify and categorize the communication failures

detected in the nurse–patient communication interactions in

the Emergency Department and their coding according to the

Joint Commission International (JCI) standards (Bagnasco

et al. 2013). We used the failure mode and effects analysis

(FMEA), because it effectively assesses and prioritizes the

areas of risk in clinical practice (Day et al. 2006, Lago et al.

2012). The preliminary phase of that study, after a review of

the literature, led to the identification of an observation grid

on ‘Communication Performance’ by Jean-Jacques Guilbert

(Guilbert 1990, 2002) (Figure 1), a validated tool used for

the observation of health workers’ communication perfor-

mance based on ‘Terminology’ (i.e. whether words used were

precise and adequate), ‘Listening’ (i.e. listening and correctly

understanding a message), ‘Attention’ (i.e. paying verbal and

behavioural attention by providing feedback) and Clarity

(i.e. communicating in a clear and precise manner so that

information is correctly understood. With this observation

grid, we observed the nursing staff working in the Emergency

Departments and measured the quali-quantitative require-

ments that needed to be implemented to improve their non-

technical skills (Flin et al. 2003) and in particular internal

and external communication (Kilner & Sheppard 2010).

‘Communication Skills Training’ can improve emergency

nurses’ communication and empathy skills associated with

an increase inpatient satisfaction and a reduction in unde-

sirable events and complaints during nurse–patient interac-

tions (Ak et al. 2011). Therefore, in 2013 our team

conducted a study on communication skills training in an

advanced simulation centre (Bagnasco et al. 2014) to cor-

rect communication failures. In this case Guilbert’s observa-

tion grid was used for educational purposes.

Instead, in this study, Guilbert’s observation grid (Fig-

ure 1) was used with the artificial neural networks (ANNs)

to predict nurse–patient communication risks from data col-

lected from three Adult Emergency Departments in Central

Italy between March 2011–May 2011.

What are artificial neural networks?

ANNs are mathematical models that try to imitate the func-

tional processes of the human brain. ANNs involve non-

linear relationships among different data sets that cannot

always be fully identified through the use of conventional

linear analyses. ANNs are extensively used in technology

and science with applications in chemistry, physics, biology

and medicine. As reported by Amato et al. (2013), ANNs

manage several types of input data to produce a clinically

relevant output, for example, the probability of a certain

pathology or classification of biomedical objects. These data

are processed in the ‘setting’ of previous training history on

a defined sample database.

ANNs are known to be a potent tool to simulate several

non-linear systems and have been applied to numerous

Why is this research or review needed?

� Communication failures in the emergency department con-

tinue to raise serious safety issues. Artificial neural net-

works can be successfully applied to predict and avoid

communication failures.

� Artificial neural networks are best suited for cases, where

there is a known relationship between inputs and outputs,

but their precise nature is not clear.

What are the key findings?

� Artificial neural networks can extract patterns and trends

from non-linear data sets that are too difficult for conven-

tional computers to calculate.

� Artificial neural networks are able to include uncertainty

in data sets.

� Artificial neural networks can be ‘trained’ using fewer

points than any other statistical model.

How should the findings be used to influence policy/practice/research/education?

� Artificial neural networks are demonstrated to be an effec-

tive tool for predicting the risk of communication failures,

but further research is needed.

� Artificial neural networks can be used in general as a risk

prediction tool.

� Early prediction of risk enables to arrange targeted educa-

tional interventions before harmful situations actually

occur.

2 © 2015 John Wiley & Sons Ltd

A. Bagnasco et al.

problems of statistically significant complexity in many

fields, including engineering, agriculture (Rodriguez Galdon

et al. 2010), education (Siri 2014), medicinal chemistry

(Pandini et al. 2013), diagnostics (Qeethara Kadhim 2011,

Irfan Khan et al. 2013) and pharmaceutical research (Weso-

lowski & Suchacz 2012). In medicine, ANNs have been

successfully applied to various areas, such as diagnostic sys-

tems (Dey et al. 2011, Hu et al. 2013, Liu & Jiang 2013),

biochemical analysis (Catalogna et al. 2012, Fernandez de

Canete et al. 2012), image analysis (Barbosa et al. 2012,

Saghiri et al. 2012) and drug development (Li et al. 2005,

Patel et al. 2012). In diagnostic systems, ANNs are usually

exploited to identify cancer and heart diseases. In biochemi-

cal analysis, ANNs have been used to examine glucose lev-

els in people affected by pathological conditions such as

tuberculosis and diabetes or in the analysis of medical

images.

In the emergency department, ANNs have predominantly

been used to quantitatively forecast patient arrivals and

admission rates (Champion et al. 2007, Jones et al. 2008,

Schweigler et al. 2009, Xu et al. 2013), or to obtain feed-

back to support clinical decisions (Hollander et al. 2004).

To our knowledge, ANNs have never been used to predict

and therefore prevent potentially harmful nurse–patient

communication interactions to increase patient safety,

which is the purpose of the present article.

Why use artificial neural networks?

ANNs are best suited for cases, where there is a known

relationship between the variable inputs and outputs, but

the particular nature of the connection is not clear. ANNs

are helpful when the link between several different variables

needs an accurate mathematical model that still has not

been developed.

The added value of ANNs is that they extract patterns

and trends from data sets that are too complex to build

with conventional computers. Another advantage of ANNs

is their ability to include uncertainty or ‘noise’ in a given

data set. Furthermore, ANNs make no assumptions about

the statistical nature of the data and can integrate nominal

and ordinal data. ANNs can be trained using comparatively

fewer points than any other statistical model and it is not

necessary to select a data distribution model.

What do we mean by ‘training’ a Neural Network?

Biologically, neural networks are made up of large numbers

of neuron cells deeply interconnected with one another. In

fact, each neuron is a specialized component that sends out

specific electrochemical signals and the axons of each neu-

ron are linked to the dendrites of other neurons through

synapses. The signals are sent through this intricate mecha-

Terminology Listening Attention Clarity

–2 Too detailed – not

adequate

Hears but not listens Not pay attention

neither verbally nor

through behaviour

Communication not

clear and information

not precise

–1

0

+1

Too detailed –

adequate

Listens but not repeats Verbal attention is

inconsistent with

behavioural one

Communication not

clear and information

is partially precise

Adequate – Not precise Listens but not always

repeats correctly

Pays verbal attention Communication not

always clear and

information not always

precise

Precise terminology

but not replies

immediately to

questions

Listens and repeats

correctly

Pays verbal and

behavioural attention

Communication not

always clear but

information is precise

+2 Replies immediately to

questions

Checks understanding Feedback to

interlocutor

Information is correctly

understood

Figure 1 Guilbert’s observation grid on communication performance. The scores in first column on the left measure the level of risk.

Source: J.-J. Guilbert. Guide p�edagogique pour les personnels de sant�e. Geneva, World Health Organization, 1990.


JAN: ORIGINAL RESEARCH: EMPIRICAL RESEARCH – QUANTITATIVE Risk prediction in ED using ANNs

nism, which is fundamentally regulated by synapses. In his

study, Hebb (1949) showed that the ‘learning process’

mainly consists in changing the strength of several synaptic

connections. The signals that pass produce a response and

the synapses carefully regulate the way it is transferred.

Artificial neural networks (ANNs) try to imitate the

behaviour of the brain. Similar to the biological nervous

system, also ANNs contain layers of simple processors (or

nodes) of data that interact through weighted connection

lines and generate an outcome. The weight-balance of these

lines is done through a ‘training session’ of input data that

the network uses to adapt its links. In short, ANNs during

training attempt to extract patterns, trends and relation-

ships from data sets starting from various input and output

variables. Obviously, the structure of an ANN is much sim-

pler than that of a biological nervous network, with many

fewer nodes or interconnections.

Researchers have explored artificial intelligence over the

last 70 years. McCulloch and Pitts carried out the first stud-

ies on ANNs in 1943. In 1959, Rosenblatt formulated the

first learning algorithm, known as the perceptron that was

only a solution to simple linear problems. In 1974, Werbos

reported the first non-linear processing capabilities of

ANNs. Thanks to the growing progresses in computer tech-

nology, the scientific community discovered the back propa-

gation algorithm – which is a network training or learning

algorithm – and its advantages in computational power.

Background

Our research team started investigating the important role

of communication skills for ensuring safety in the Paediatric

Emergency Department setting in 2008 (Bagnasco et al.

2013). During that study, many nurses were found to have

poor communication skills, therefore, our research team

conducted another study to improve communication skills

through educational interventions in an Advanced Simula-

tion Centre (Bagnasco et al. 2014).

However, educational interventions on their own are not

enough to ensure safe communication in the Emergency

Department, but needed to find a tool that could predict

communications risks before they actually occurred. There-

fore, ANNs were identified as a possible tool that could

reliably achieve this purpose.

The Artificial Neuron

As described by Bardan (2014), ANNs are made up of neu-

rons and each neuron consists of a ‘summation block’ and an

‘activating block’. The input variables are multiplied by the

optimized weight parameters (w) and their products are

summed. The neuron output is attained through the transfor-

mation of the sum using an activation function. Perceptron

networks commonly use continuous sigmoidal activation

functions that convert the output value into a range between

0-1. From this perspective, the artificial neuron is similar to

regression models. In fact, a bias weight (‘constant’ in statisti-

cal terminology) is used to optimize the final prediction, simi-

lar to regression models. However, regression models have

an important weakness that consists in the inability to reveal

intricate relationships in given data sets. Moreover, the corre-

lation type between inputs and outputs has to be chosen

before starting the analysis (Bardan 2014) (Figure 2).

In 2000, Zhang and Gupta described that the most com-

mon network structure is the multilayer perceptron (MLP),

where the neurons are grouped into layers. No layers

Inputs

Threshold (bias)

Weights

Sum-of-Product

Outputγ

χ

Θ

∑ ƒ(χ)

Summationfunction

Activationfunction

A1

W1

W2

W3

WN

1

A2

A3

AN

Figure 2 Visual display of an artificial neuron. The inputs are multiplied with the optimized weights and thereafter summed. The sum is

transformed by the neuron’s activation function into the output value. Source: Own elaboration.


A. Bagnasco et al.

(levels) of neurons communicate with each other. The first

and last layers are respectively named ‘Input’ and ‘output’,

because they represent inputs and outputs of the whole net-

work. The ‘Hidden layers’ are the remaining layers that are

situated inside the network, between the input and the out-

put layers. Usually, an MLP neural network contains an

input layer, one or more hidden layers and an output layer

(Figure 3).

Statistically significant parameters for an accurate predic-

tion are the number of hidden layers and the number of neu-

rons on each level. The output layer generates the network

response that commonly consists of one or two neurons.

The study

This study consists of four phases. In Phase 1, we collected

the data through the observations of the nurse–patient com-

munication interactions and measured communication risk

with Guilbert’s observation grid (Figure 1). In Phase 2, we

trained the ANNs using the input (Table 1) and output

variables (Table 2). In Phase 3, we tested the predictive per-

formance of the ANNs (Figure 5). In Phase 4, we used the

ROC analysis (Figure 4) to understand whether the predic-

tive performance of the ANNs was correct.

Aim

The aim of this paper was to study the usefulness of ANNs

in predicting the risk of communication failures in EDs

through selected variables.

Rationale

Based on our previous studies (Bagnasco et al. 2013, 2014),

two tools were identified to collect data: (1) A question-

naire to collect nurses’ personal characteristics; (2) the Situ-

ation-Background-Assessment-Recommendation (SBAR)

technique applied to communication in the Emergency

Department, which investigated the communication behav-

iours of the nurses with patients (and their family members)

using the four dimensions of Guilbert’s observation grid

(i.e. terminology, listening, attention and clarity).

Therefore, to test ANNs as a risk prediction tool for

communication failures in the ED we asked two research

questions, one for each tool:

1 How accurately do nurses’ personal characteristics predict

the risk of communication failures?

2 Which of the four communication dimensions weigh most

in predicting the risk of communication failures?

Design

A multicentre observational study.

Sample

We selected a convenience sample of 840 nurses working

in the emergency departments (EDs) from three different

hospitals in Central Italy. After contacting the Nursing

Inputlayer

Hiddenlayer

Outputlayer

Synapses

Figure 3 The layers of an artificial neural network. Source: Own

elaboration.

Table 1 Input variables and respective description. The nominal

variables are qualitative characteristics that cannot be numerically

ordered. Instead, ordinal variable are characteristics that can be

numerically ordered, such as level of education and school marks.

Input variables Description of the input variables Type

M/F Gender Nominal

Age Age bracket Ordinal

Seniority Years of service in current Emergency

Department

Ordinal

Professional

Qualification

Title of professional qualification Nominal

Postgraduate

qualification

Titles of Postgraduate qualification

(masters, specialization, etc.)

Nominal

Structure Hospital where nurses worked Nominal

Source: Own elaboration.

Table 2 Output variables.

Output variables Type

Terminology Ordinal

Listening Ordinal

Attention Ordinal

Clarity Ordinal




Directors of the three hospitals and obtaining their

approval for conducting this observational study involving

the nurse–patient communication interactions, our research

team illustrated the project to the head nurse of each emer-

gency department (ED). The head nurses then asked if any

nurses working in the ED on a full time basis volunteered

to take part in this study. The members of our research

team provided preliminary training both to each head nurse

and to their group of nurses who had volunteered to take

part in this study. Preliminary training was conducted in

small groups and consisted of a four-hour theoretical and

practical training session to make sure that the tools (the

personal characteristics questionnaire and Guilbert’s obser-

vation grid) were used correctly and that observations were

conducted in a rigorous and objective manner. This preli-

minary training session was essential both to produce com-

parable data and ensure good inter-rater reliability.

Data collection

A multicentre observational study database was used for this

research. Data were collected by observing nurse–patient

1·0Terminology Listening

Attention Clarity

·8

·6

·4

·2

·2 ·4 ·6 ·8 1·01 - Specificity

Area under curve1 ·910

·917·927

23


·915·952

23


·866·915

23


·886·924

23

Sen

sitiv

ity

·0·0

1·0

·8

·6

·4

·2

·2 ·4 ·6 ·8 1·01 - Specificity

Sen

sitiv

ity

·0·0

1·0

·8

·6

·4

·2

·2 ·4 ·6 ·8 1·01 - Specificity

Sen

sitiv

ity

·0·0

1·0

·8

·6

·4

·2

·2 ·4 ·6 ·8 1·01 - Specificity

Sen

sitiv

ity

·0·0

123

123

123

123

Figure 4 ROC curve of the four dimensions of communication. Source: Own elaboration.


A. Bagnasco et al.

communication interactions in 840 nurses while carrying

out their routine activities in the Emergency Department.

Observations were made over a period of 3 months

(March–May 2011), until at least 200 communication inter-

actions were collected in each hospital. Therefore, the

groups of appropriately trained nurses observed the com-

munication behaviours of their nurse colleagues with

patients during their routine work activities in the ED. This

strategy to train nurses already working in the ED instead

of having a person from outside the hospital (i.e. a member

of our research team), to minimize the Hawthorne effect

(Landsberger 1958).

The tool used for observation consisted of two parts:

1 A questionnaire to collect data on the facility, field of

investigation and characteristics of the nurses included in

the sample.

2 Guilbert’s observation grid (Guilbert 1990) based on the

four communication dimensions.

The purpose of the first part of the tool was to collect the

ED staff nurses’ personal and descriptive data and their levels

of training and experience (i.e. age, gender, years of service in

the current workplace, professional qualification, postgradu-

ate education and type of structure). The second part of the

tool investigated the communication behaviours of the ED

staff nurses with patients in relation to four dimensions:

terminology, listening, attention and clarity (Figure 1). For

each of these dimensions, a rating scale (ranging from �2-+2)

measured the level of communication risk.

Ethical considerations

This study received Research Ethics Committee approval

from the University of Genoa. All participants took part

on a voluntary basis and were appropriately informed

before being included in the study. In addition, all partici-

pants spontaneously accepted to reciprocally observe one

another.

Data analysis

The database of 840 nurses working in three Accident and

Emergency Departments in Rome consisted of a total of 10

variables: the four dimensions of the observation grid (i.e.

terminology, listening, attention and clarity; these were the

output variables) and six regarding age, gender, working

seniority, level of education, professional qualifications and

the structure (place), where they worked (these were the

input variables). Not all the variables included in the model

were suitable at the first attempt to train the ANN.

The six main input variables included in this study are

shown in Table 1. The output variables were the four

dimensions included in Guilbert’s observation grid related

to communication in the ED: terminology, listening, atten-

tion and clarity (Table 2). The output layer consisted of a

variable that indicated communication risk, which in the

ANNs was summarized into three units (1 = real risk;

2 = moderate risk; 3 = no risk).

To increase ANN predictive performance, a correlation

analyses was conducted between all the variables to identify

any strong correlations between the different variables

included and remove redundant information from the data-

base. This analysis showed that there were strong obvious

correlations with age and seniority and weak correlations

with gender, education, professional qualifications and the

structure where they worked. Microsoft Office Excel, Ver-

sion 2007 and IBM SPSS Statistics for Windows, Version

22�0 (IBM Corp. Released 2013, Armonk, NY, USA) were

used for data processing and running the ANNs.

ANN Topology selection and training

The identification of an appropriate topology (i.e. the num-

ber of hidden layers and the number of nodes per hidden

layer) is one of the most crucial aspects of the application

of neural networks. The neural network architecture used

in this study is the multi-layered feed-forward network

model that is widely and successfully applied for classifica-

tion, forecasting and problem-solving (Irfan Khan et al. 2013).

In general, a network topology is often decided only after

a lengthy process of trial-and-error. The approach we

adopted considered the training of many different networks

and the subsequent selection of the ‘best’ one. All neural

networks are trained until the error (sum of squared error)

for each training session reached the minimum, before it

started to rise again.

After topology determination, the data were normalized

finding a mean and a standard deviation of each feature

and for every cell of each row to subtract its mean and

divide it by the standard deviation of the right feature. The

resulting data have a zero mean and a standard deviation

equal to one.

Therefore, the data are separated into training and vali-

dation subsets. The ANN is trained with a standard back-

propagation learning algorithm. The activation function

used for each node in the network is the binary sigmoidal

function demarcated (with r = 1) as output = 1/(1 + e�x),

where x is the sum of the weighted inputs to that specific

node. This function restricts the output of all nodes in the

network to the range 0-1. Between each training epoch, the

error rate is computed for data validation.



Due to the risk of overfitting the neural network too eas-

ily, producing an error rate on validation data larger than

the error rate on the training data, it is crucial not to over-

train the data. Overtraining is when you train the network

for so long that it starts to memorize the training set. The

validation error reduces in the early epochs but after some

time, it begins to rise. The point of minimum validation

error is a valid indicator of the best number of epochs for

training. Our results indicated that the minimum error of

the validation set could be obtained in approximately 500

epochs. The training results are certainly dependent on the

initial values of the weights that are generally random.

We examined the performance of the test across the

entire range of possible values using the curves of the recei-

ver operator characteristic (ROC) analysis. The ROC curve

is a plot of the true positive rate (sensitivity) vs. the false

positive rate (1-specificity). A perfect test would show

points in the upper-left-corner, with 100% sensitivity and

100% specificity (Woods & Bower 1997). An area under a

curve (AUC) of 1 means that the test is perfect, whereas an

AUC of 0�5 means that the test has a probability of 50% to

classify (similar to the toss of a coin). A diagnostic test is

considered adequate with an area under the curve >80%.

Validity and reliability

The validity and reliability of the data sets collected during

the communication observations made at the beginning of

this was ensured through the observers’ use of Guilbert’s

observation grid, by attributing a score between �2-+2 to

the four dimensions of communication: terminology; listen-

ing; attention and clarity (Figure 1). For this purpose, a 4-

hour preliminary training session was provided to the ED

nurses who were going to observe the nurse–patient com-

munication interactions of their peers.

Results

A total of 840 observations were made on the nurses work-

ing in the EDs, with a good distribution between males and

females. Most of the observations involved nurses aged

between 31-40 years. With regard to seniority (years of ser-

vice in the current ED) most of the nurses had worked

between 6-10 years (Table 3). The other three variables

(education, professional qualification and structure) are not

included because we found that they did not have an

impact on the prediction of communication risk.

The ANN’s learning ability was checked through the cal-

culation of the mean-squared error, as error measure. We

used a learning algorithm called ‘Back Propagation’, which

modified the synaptic weights to minimize global error

made by the ANN, measured with the mean squared error

(MSE) between the output of the network and the correct

results. The network was trained on 80% of the population

included in the study; 20% of the population was used as a

pilot test.

The network considered to have the highest level of pre-

diction is the multi-perceptron network with 35 units, one

hidden layer and 10 units in the hidden layer. The four

dimensions of communication (clarity, attention, listening

and terminology) were analysed separately to determine the

importance of the independent variables for each one and

compare them with one another. We examined the perfor-

mance of the test across the entire range of possible values

using the ROC curve analysis (Figure 3). Regarding the

clarity of communication, training was performed on 684

cases, causally chosen (81�4% of the cases) and the test on

the remaining 156 (18�6%).

In our case, the values using the ROC curve analysis

were higher than 90%. Therefore, the test was very accu-

rate for the risk levels 1 and 3 and moderately accurate for

risk level 2. During training, the neural network correctly

predicted 82�5% of the cases and 86�5% during the test.

Concerning the attention to communication, the training

was performed on 669 cases, chosen on a casual basis

(79�6% of the cases) and the test was conducted on the

remaining 20%, equal to 171 (20�4%). The values using

the ROC curve analysis were >90% and, therefore, corre-

sponded to a probability that was 90% higher than a cor-

rect classification. Therefore, the test was very accurate for

the risk levels 1 and 3 and moderately accurate for risk

level 2. The neural network predicted correctly during

training in 81% of the cases and 83�6% during the test.

Pertaining to the terminology dimension, the training was

performed on 689 cases, causally chosen (82% of the cases)

Table 3 The most significant results of the observations made on

the communication interactions of 840 nurses in three different

Emergency Departments with their patients between March–May

2011.

Gender Age

Seniority

(years of service)

Males = 40�6%Females = 59�4%

31-40 years = 41�3%41-50 years = 32�9%51-60 years = 7�5%>61 years = 0�8%

0-5 years = 26�2%6-10 years = 37%

11-15 years = 22%

16-20 years = 8�2%>20 years = 6�05%

The other three variables (education, professional qualifications

and place of work) were excluded because they were found to have

no impact on the prediction of communication risk.


A. Bagnasco et al.

and the test on the remaining 151 (18%). Also in this case,

the test was very accurate. The neural network predicted

correctly during training in 81�9% of the cases and 81�5%during the test.

Concerning the listening dimension, the training was per-

formed on 675 cases, causally chosen (80�4% of the cases)

and the test on the remaining 19�6% was equal to 165. The

values were >90% and, therefore, the test was very accu-

rate. During training, the neural network correctly pre-

dicted 83% of the cases and 81�8% during the test.

To determine the optimal series of variables to predict

the expected result among the ones used in the input, a net-

work sensitivity analysis was conducted. This analysis pro-

vided the relative importance of the variables included to

determine the output, where 0 indicated a variable with no

predictive validity and 1 indicated a variable with the maxi-

mum predictive validity.

If we consider the importance of the independent vari-

ables of the four dimensions (clarity, attention, terminology,

listening), we can observe the variables that played the most

important role in the four dimensions (Figure 5). The age

variable played an important role in all of the four dimen-

sions: the ability to listen, pay attention, use precise termi-

nology and clarity increased with age. Then followed

seniority, which is also related to age. Gender and level of

education did not have a statistically significant impact on

the effectiveness of communication. This was because

the data on the level of education did not allow its role in

communication to be checked across the years in each

professional. In fact, only education focusing on the com-

munication between health professionals and patients could

have had a strong impact on the SBAR scores.

Discussion

To our knowledge, ANNs have never been used to predict

and prevent harmful nurse–patient communication interac-

tions to increase patient safety. We found that ANNs were

able to correctly predict more than 80% of the communica-

tion characteristics including the communication risks.

Answering to our two research questions, through ANNs

we found that the nurses’ personal characteristics that

enable to predict the risk of communication failures were

age and years of working seniority and experience in the

Emergency Department. Gender and education did not have

a significant impact on communication. In the case of edu-

cation, this would have had a significant impact on commu-

nication between nurses and patients only if education

focused specifically on communication. With regard to the

four communication dimensions, clarity and attention

weighed most in predicting the risk of communication fail-

ures.

In summary, all this means that a correct prediction of

communication failures needs to be followed by appropriate

educational interventions. Notwithstanding, the positive

results that emerged from this study, there is still a long

way to go in this direction because of the many limits.

Limitations

Some of the variables we used were not appropriate to be

specifically processed with the neural networks and there-

fore did not produce any information, so we had to collect

other data, to limit subjective evaluations during observa-

tions. More specifically, with regard to the observers, we

identified the absence of a variable that enabled to under-

stand whether the observer had accessed the personal file of

the person observed, and the opportunity to create the

observer’s profile (personal data, education, scale rating

objectiveness/trustworthiness/level of discrimination or

something similar) to cross-check the data collected by the

observers.

With regard to the population observed, the personal

data needed to be more detailed. For instance, the ‘age of

the nurse observed’ was reported by the observer using a

very wide age bracket. For instance, data on the nurses’

marital status were missing, and the years of education and

incentives for continued education in the field of communi-

cation, including university courses.

Structure30%

25%

20%

15%

10%

5%

0%

Prof. Qualif.

Education

Seniority

Clarity Attention Terminology Listening

Gender

Age

Figure 5 The predictive performance of the independent variable.




The variable ‘years of service’ did not include how many

years of work were spent on a given facility or in general

as a nurse. It would be better to state the number of years

of nursing practice, the number of years worked in a given

clinical facility and in the emergency department where the

survey was conducted.

The variable ‘context’ could not be included, because it

was not stated in all of the observations. We also found

that it was important to differentiate the observations made

on admission or triage from those made on discharge.

Another variable that was missing was the one indicating

the nurses’ level of burnout, because it could be important

to reduce the level of subjectivity of the survey on the effi-

cacy of communication in the Emergency Department. We

also found that it would be important to follow the same

patient from admission to discharge, to learn more about

the opinions of the patients and of those take care of them,

because this would allow to control the variables related to

aspects such as their emotional status and concerns.

With regard to ANNS, for a specified data set there may

be an infinite number of relevant network structures that

can learn the characteristics of the data. The question

remains: what size of network will be best for a specific

data set. Unfortunately, it is not easy to answer this ques-

tion. The neural network structure that offers the best

result for a specific problem needs to be experimentally

decided. The size, the characteristics of the training data set

and the number of iterations are the other factors that

affect the generalization (and therefore predictive) capabili-

ties of a neural network. Generally, the network size influ-

ences its complexity and ‘learning time’ but, most

importantly, it affects the generalization capabilities of the

network. Generalization is the ability of the neural network

to interpolate and extrapolate data that it has not seen

before. The predictive power of an ANN depends on how

well it generalizes from the training data.

Conclusion

Nurses can play a very important role in healthcare organi-

zations as safety nets, but very often the environment where

they work harbours many latent risks, such as communica-

tion failures. The proactive analysis of the factors that con-

tribute to communication failures not only between health

professionals and patients but also among health profes-

sionals, could be described by the type of organizational

culture and in addition stimulates reflection about the

health professionals’ communication skills and about the

factors linked to the working environments that have an

influence on communication errors.

This study has shown the potential of the artificial neural

network in predicting the risk of communication failures in

Emergency Departments. The model was developed on

some selected input variables from the SBAR data. It

achieved an accuracy of over 80%, which shows the poten-

tial efficacy of an Artificial Neural Network as a prediction

tool. A ‘spin-off’ result of this study was the validation of

Guilbert’s observation grid as a tool to predict communica-

tion failures. Early detection of communication failures in

health professionals who have specific social and cultural

characteristics would allow health organizations to arrange

specific courses only or prevalently for those who have

weak communication skills.

Funding

This research received no specific grant from any funding

agency in the public, commercial or not-for-profit sectors.

Conflict of interest

No conflict of interest was declared by the authors in rela-

tion to the study itself. Note that Editor-in-Chief Roger

Watson is a visiting professor at the University of Genoa

and has been consulted about this paper but, in line with

usual practice, this article was subjected to double-blind

peer review and was edited by another editor.

Author contributions

All authors have agreed on the final version and meet at

least one of the following criteria [recommended by the IC-

MJE (http://www.icmje.org/recommendations/)]:

• substantial contributions to conception and design,

acquisition of data or analysis and interpretation of

data;

• drafting the article or revising it critically for important

intellectual content.

References

Ak M., Cinar O., Sutcigil L., Congologlu E.D., Haciomeroglu B.,

Canbaz H., Yaprak H., Jay L. & Ozmenler K.N. (2011)

Communication skills training for emergency nurses.

International Journal of Medical Sciences 8(5), 397–401.

Amato F., L�opez A., Pe~na-M�endez E.M., Va�nhara P., Hampl A. &

Havel J. (2013) Artificial neural networks in medical diagnosis.

Journal of Applied Biomedicine 11, 47–58. doi:10.2478/v10136-

012-0031-x.

Bagnasco A., Tubino B., Piccotti E., Rosa F., Aleo G., Di Pietro

P. & Sasso L. (2013) Identifying and correcting


A. Bagnasco et al.

http://www.icmje.org/recommendations/

info:doi/10.2478/v10136-012-0031-x

info:doi/10.2478/v10136-012-0031-x

communication failures among health professionals working in

the Emergency Department. International Emergency Nursing

21, 168–172.

Bagnasco A., Pagnucci N., Tolotti A., Rosa F., Torre G. & Sasso

L. (2014) The role of simulation in developing communication

and gestural skills in medical students. BMC Medical Education

14, 106. doi:10.1186/1472-6920-14-106.

Bardan R.T. (2014) Artificial neural networks. Applications in

urology. Timisoara Medical Journal 54, 289–296.

Barbosa D., Roupar D., Ramos J., Tavares A. & Lima C. (2012)

Automatic small bowel tumor diagnosis by using multi-scale

wavelet-based analysis in wireless capsule endoscopy images.

Biomedical Engineering Online 11, 3. doi:10.1186/1475-925X-

11-3.

Catalogna M., Cohen E., Fishman S., Halpern Z., Nevo U. & Ben-

Jacob E. (2012) Artificial neural networks based controller for

glucose monitoring during clamp test. PLoS ONE 7(8), e44587.

doi:10.1371/journal.pone.0044587.

Champion R., Kinsman L.D., Lee G.A., Masman K.A., May E.A.,

Taylor M.D., Thomas P.R. & Williams R.J. (2007) Forecasting

emergency department presentations. Austrian Health Review

31, 83–90.

Day S., Dalto J., Fox J. & Turpin M. (2006) Failure mode and

effects analysis as a performance improvement tool in trauma.

Journal of Trauma Nursing 13(3), 111–117.

Dey P., Lamba A., Kumari S. & Marwaha N. (2011) Application

of an artificial neural network in the prognosis of chronic

myeloid leukemia. Analytical and Quantitative Cytology and

Histology 33(6), 335–339.

Fernandez de Canete J., Gonzalez-Perez S. & Ramos-Diaz J.C.

(2012) Artificial neural networks for closed loop control of in

silico and ad hoc type 1 diabetes. Computer Methods and

Programs in Biomedicine 106, 55–66.

Flin R., Martin L., Goeters K., Hoermann J., Amalberti R., Valot

C. & Nijhuis H. (2003) The development of the NOTECHS

system for evaluating pilots’ CRM skills. Human Factors and

Aerospace Safety 3, 95–117.

Guilbert J.-J. (1990) Guide p�edagogique pour les personnels de

sant�e. World Health Organization, Geneva, Switzerland.

Guilbert J.-J. (2002) The ambiguous and bewitching power of

knowledge, skills and attitudes leads to confusing statements of

learning objectives. Education for Health 15(3), 362–369.

Haig K.M., Sutton S. & Whittington J. (2006) SBAR: a shared

mental model for improving communication between clinicians.

The Joint Commission Journal on Quality and Patient Safety 32,

167–175.

HebbD.O. (1949)TheOrganization of Behavior. Wiley, NewYork.

Hollander J.E., Blomkalns A.L., Brogan G.X., Diercks D.B., Field

J.M., Garvey J.L., Gibler W.B., Henry T.D., Hoekstra J.W.,

Holroyd B.R., Hong Y., Kirk J.D., O’Neil B.J., Jackson R.E.,

Aufderheide T., Blomkalns A.L., Brogan G.X., Christenson J.,

Collins S., Diercks D.B., Fesmire F.M., Garvey J.L., Green G.B.,

Lindsell C.J., Peacock W.F., Pollack C.V., Zalenski R.;

Multidisciplinary Standardized Reporting Criteria Task Force;

Standardized Reporting Criteria Working Group of Emergency

Medicine Cardiac Research and Education Group-International

(2004) Standardized reporting guidelines for studies evaluating

risk stratification of emergency department patients with

potential acute coronary syndromes. Annals of Emergency

Medicine 44(6), 589–598.

Hu X., Cammann H., Meyer H.A., Miller K., Jung K. & Stephan

C. (2013) Artificial neural networks and prostate cancer-tools for

diagnosis and management. Nature Reviews Urology 10(3), 174–

182. doi:10.1038/nrurol.2013.9.

Irfan Khan Y., Zope P.H. & Suralkar S.R. (2013) Importance of

Artificial Neural Network in Medical Diagnosis disease like

acute nephritis disease and heart disease. International Journal of

Engineering, Science and Innovative Technology 2(2), 210–217.

Jones F., Podila P. & Powers C. (2013) Creating a culture of safety

in the emergency department: the value of teamworking training.

Journal of Nursing Administration 43(4), 194–200.

Jones S.S., Thomas A., Evans R.S., Welch S.J., Haug P.J. & Snow

G.L. (2008) Forecasting daily patient volumes in the emergency

department. Academic Emergency Medicine 15, 159–170.

Kilner E. & Sheppard L.A. (2010) The role of team work and

communication in the emergency department: a systematic

review. International Emergency Nursing 18(3), 127–137.

Lago P., Bizzarri G., Scalzotto F., Parpaiola A., Amigoni A., Putoto

G. & Perilongo G. (2012) Use of FMEA analysis to reduce risk

of errors in prescribing and administering drugs in paediatric

wards: a quality improvement report. BMJ Open 2(6), pii:

e001249. doi: 10.1136/bmjopen-2012-001249.

Landsberger H.A. (1958) Hawthorne Revisited. Cornell University

Press, Ithaca.

Li Y., Rauth A.M. & Wu X.Y. (2005) Prediction of kinetics of

doxorubicin release from sulfopropyl dextran ion-exchange

microspheres using artificial neural networks. European Journal

of Pharmaceutical Sciences 24, 401–410.

Liu B. & Jiang Y. (2013) A multi-target training method for

artificial neural network with application to computer-aided

diagnosis. Medical Physics 40, 011908. doi:10.1118/1.4772021.

McCulloch W.S. & Pitts W.H. (1943) A logical calculus of the

ideas imminent in nervous activity. The Bulletin of Mathematical

Biophysics 5, 115–133.

Meginniss A., Damian F. & Falvo F. (2012) Time out for patient

safety. Journal of Emergency Nursing 38(1), 51–53.

Pandini A., Fraccalvieri D. & Bonati L. (2013) Artificial neural

networks for efficient clustering of conformational ensembles and

their potential for medicinal chemistry. Current Topics in

Medical Chemistry 13(5), 642–651.

Patel A., Mehta T., Patel M., Patel K. & Patel N. (2012) Design

porosity osmotic tablet for delivering low and pH-dependent

soluble drug using an Artificial Neural Network. Current Drug

Delivery 9, 459–467.

Qeethara Kadhim A-S. (2011) Artificial neural network in medical

diagnosis. International Journal of Computer Science 8(2), 150–

153.

Rodriguez Galdon B., Penna-Mendez E., Havel J., Rodriguez E. &

Diaz Romero C. (2010) Cluster analysis and artificial neural

networks multivariate classification of onion varieties. Journal of

Agricultural Food Chemistry 58, 11435–11440.

Rosenblatt F. (1958) The perceptron: a probabilistic model for

information storage and organization in the brain. Psychological

Review 65, 386–408.

Saghiri M., Asgar K., Boukani K., Lotfi M., Aghili H., Delvarani

A., Karamifar K., Saghiri A., Mehrvarzfar P. & Garcia-Godoy F.



info:doi/10.1186/1472-6920-14-106

info:doi/10.1186/1475-925X-11-3

info:doi/10.1186/1475-925X-11-3

info:doi/10.1371/journal.pone.0044587

info:doi/10.1038/nrurol.2013.9

info:doi/10.1136/bmjopen-2012-001249

info:doi/10.1118/1.4772021

(2012) A new approach for locating the minor apical foramen

using an artificial neural network. International Endodontic

Journal 45, 257–265.

Siri A. (2014) Predicting Students’ Academic Dropout Using

Artificial Neural Network. Nova Science Publishers, Inc. U.S.,

Commack, NY.

Spencer R., Logan P. & Coiera E. (2002) Supporting

Communication in the Emergency Department. Centre for

Health Informatics. University of New South Wales, Sidney,

NSW.

Schweigler L.M., Desmond J.S., McCarthy M.L., Bukowski K.J.,

Ionides E.L. & Younger J.G. (2009) Forecasting models of

emergency department crowding. Academic Emergency Medicine

16, 301–308.

Werbos P.J. (1974) Beyond Regression: New Tools for Prediction

and Analysis in the Behavioral Sciences. PhD thesis, Harvard

University, Published in the The roots of backpropagation: from

ordered derivatives to neural networks and political forecasting

(Vol. 1) Werbos, P. J. (1994) John Wiley & Sons. New York.

Wesolowski M. & Suchacz B. (2012) Artificial neural networks:

theoretical background and pharmaceutical applications: a

review. Journal of AOAC International 95, 652–668.

Woods K. & Bower K.W. (1997) Generating ROC curves for

artificial neural networks. IEEE Transactions on Medical

Imaging 16, 329–337.

Xu M., Wong T.C. & Chin K.S. (2013) Modeling daily patient

arrivals at Emergency Department and quantifying the relative

importance of contributing variables using artificial neural

network. Decision Support Systems 54, 1488–1498.

Zhang Q.-J. & Gupta K.C. (2000) Neural Networks for RF and

Microwave Design, Chapter 3. Artech House Inc., Boston, MA,

pp. 61–103.

The Journal of Advanced Nursing (JAN) is an international, peer-reviewed, scientific journal. JAN contributes to the advancement of

evidence-based nursing, midwifery and health care by disseminating high quality research and scholarship of contemporary relevance

and with potential to advance knowledge for practice, education, management or policy. JAN publishes research reviews, original

research reports and methodological and theoretical papers.

For further information, please visit JAN on the Wiley Online Library website: www.wileyonlinelibrary.com/journal/jan

Reasons to publish your work in JAN:

• High-impact forum: the world’s most cited nursing journal, with an Impact Factor of 1·527 – ranked 14/101 in the 2012 ISI Jour-

nal Citation Reports © (Nursing (Social Science)).

• Most read nursing journal in the world: over 3 million articles downloaded online per year and accessible in over 10,000 libraries

worldwide (including over 3,500 in developing countries with free or low cost access).

• Fast and easy online submission: online submission at http://mc.manuscriptcentral.com/jan.

• Positive publishing experience: rapid double-blind peer review with constructive feedback.

• Rapid online publication in five weeks: average time from final manuscript arriving in production to online publication.

• Online Open: the option to pay to make your article freely and openly accessible to non-subscribers upon publication on Wiley

Online Library, as well as the option to deposit the article in your own or your funding agency’s preferred archive (e.g. PubMed).


A. Bagnasco et al.

http://www.wileyonlinelibrary.com/journal/jan

http://mc.manuscriptcentral.com/jan

applying artificial neural networks to predict ... · original research: empirical research –...

Documents