a fuzzy back propagation algorithm
TRANSCRIPT
-
8/3/2019 A Fuzzy Back Propagation Algorithm
1/13
Fuzzy Sets and Systems 112 (2000) 2739
www.elsevier.com/locate/fss
A fuzzy backpropagation algorithm
Stefka Stoevaa, Alexander Nikovb;
a Bulgarian National Library, V. Levski 88, BG-1504 Soa, BulgariabTechnical University of Soa, P.O. Box 41, BG-1612 Soa, Bulgaria
Received November 1994; revised March 1998
Abstract
This paper presents an extension of the standard backpropagation algorithm (SBP). The proposed learning algorithm is
based on the fuzzy integral of Sugeno and thus called fuzzy backpropagation (FBP) algorithm. Necessary and sucient
conditions for convergence of FBP algorithm for single-output networks in case of single- and multiple-training patterns are
proved. A computer simulation illustrates and conrms the theoretical results. FBP algorithm shows considerably greater
convergence rate compared to SBP algorithm. Other advantages of FBP algorithm are that it reaches forward to the target value
without oscillations, requires no assumptions about probability distribution and independence of input data. The convergence
conditions enable training by automation of weights tuning process (quasi-unsupervised learning) pointing out the interval
where the target value belongs to. This supports acquisition of implicit knowledge and ensures wide application, e.g. forcreation of adaptable user interfaces, assessment of products, intelligent data analysis, etc. c 2000 Elsevier Science B.V.
All rights reserved.
Keywords: Neural networks; Learning algorithm; Fuzzy logic; Multicriteria analysis
1. Introduction
Recently, the interest in neural networks has grown
dramatically: it is expected that neural network mod-els will be useful both as models of real brain func-
tions and as computational devices. One of the most
popular neural networks is the layered feedforward
neural network with a backpropagation (BP) least-
mean-square learning algorithm [17]. Its topology is
shown in Fig. 1. The network edges connect the pro-
cessing units called neurons. With each neuron input
there is associated a weight, representing its relative
Corresponding author.E-mail address: [email protected] (A. Nikov)
importance in the set of the neurons inputs. The in-
puts values to each neuron are accumulated through
the net function to yield the net value: the net value is
a weighted linear combination of the neurons inputsvalues.
For the purpose of multicriteria analysis [13,20,21]
a hierarchy of criteria is used to determine an over-
all pattern evaluation. The hierarchy can be encoded
into a hierarchical neural network where each neuron
corresponds to a criterion. The input neurons of the
network correspond to single criterion. The hidden
and output neurons correspond to complex criteria. As
evaluation function it can be used as the net function
of the neurons. However, the criteria can be combined
linearly when it is assumed that they are independent.
0165-0114/00/$- see front matter c 2000 Elsevier Science B.V. All rights reserved.
PII: S 0 1 6 5 - 0 1 1 4 ( 9 8 ) 0 0 0 7 9 - 7
-
8/3/2019 A Fuzzy Back Propagation Algorithm
2/13
28 S. Stoeva, A. Nikov / Fuzzy Sets and Systems 112 (2000) 2739
Fig. 1. Example neural network with single-output neuron and a hidden layer.
But in practice the criteria are correlated to some
degree. The linear evaluation function is unable to
capture relationship between the criteria. In order to
overcome this drawback of standard backpropagation
(SBP) algorithm we propose a fuzzy extension called
fuzzy backpropagation (FBP) algorithm. It determinesthe net value through the fuzzy integral of Sugeno [3]
and thus does not assume independence between the
criteria. Another advantage of FBP algorithm is that
it reaches always forward to the target value without
oscillations and there is no possibility to fall into lo-
cal minimum. Necessary and sucient conditions for
convergence of FBP algorithm for single-output net-
works in case of single- and multiple-training patterns
are proved. The results of computer simulation are
reported and analysed: FBP algorithm shows consid-erably greater convergence rate compared to SBP
algorithm.
2. Description of the algorithm
2.1. Standard backpropagation algorithm
(SBP algorithm)
First we describe the standard backpropagation al-
gorithm [17,18] because FBP algorithm proposed by
us can be viewed as fuzzy-logic-based extension of the
standard one. SBP algorithm is an integrative gradient
algorithm designed to minimise the mean-squared er-
ror between the actual output and the desired output by
modifying network weights. In the following we use
for simplicity a network with one hidden layer but theresults are valid also for networks with more than one
hidden layers. Let us consider a layered feedforward
neural network with n0 inputs, n1 hidden neurons and
single-output neuron (cf. Fig. 1). The inputoutput re-
lation of each neuron of the neural network is dened
as follows:
Hidden neurons
net(1)i =
n0
j=1
w(1)ij xj ;
a(1)i = f(net
(1)i ); i = 1; 2; : : : ; n1:
Output neurons
net(2) =
n1i=1
w(2)1i a
(1)i ;
a(2) = f(net(2));
where X = (x1
; x2
; : : : ; xn0
) is the vector of patterns
inputs.
-
8/3/2019 A Fuzzy Back Propagation Algorithm
3/13
S. Stoeva, A. Nikov / Fuzzy Sets and Systems 112 (2000) 2739 29
The activation functions f(net) can be linear ones
or Fermi functions of type
f(net) =1
1 + e4(net):
Analogous to the writing and reading phases, there are
also two phases in the supervised learning BP network.
There is a learning (training) phase when a training
data set is used to determine the weights that dene the
neural model. So the task of the BP algorithm is to nd
the optimal weights to minimise the error between the
target value and the actual response. Then the trained
neural model will be used later in the retrieving phase
to process and evaluate real patterns.
Let the patterns output corresponding to the vector
of patterns inputs X be called the target value t.
Then the learning of the neural network for train-
ing pattern (X;t) is performed in order to minimise
the squared-error between the target and the actual
response
E = 12
(t a(2))2:
The weights are changed according to the following
formula:
wnewij = woldij + wij;
where
wij = (l)i a
(l)j :
[0; 1] denotes the learning rate, (l)i is the error
signal relevant to the ith neuron, and a(l)
j the signal at
jth neuron input.
The error signal is obtained recursively by back-
propagation.
Output layer
(2) = f(net(2))(t a(2)); (1)
where the derivation of the activation function
f(net) = 4net(1 net):
Hidden layer
(1)i = f
(net(1)i )
(2)w(2)1i ;
where (2) is determined by formula (1).
When there are more than one training pattern
(Xm; tm); m = 1; 2; : : : ; M , the sum-squared error is
dened as
E=
1
2
M
m=1
(tm
a(2)m )
2
:
2.2. Fuzzy backpropagation algorithm
(FBP algorithm)
Recently, many neuro-fuzzy models have been in-
troduced [2,6,8,10,11]. The following extension of
the standard BP algorithm to fuzzy BP algorithm
is proposed. For yielding the net value neti the in-
puts values of the ith neuron are aggregated. The
mapping is mathematically described by the fuzzy
integral of Sugeno that relies on psychological back-
ground.
Let xj [0; 1] R; j = 1; : : : ; n0, and all network
weights w(l)ij [0; 1] R, where l = 1; 2. Let P(J) be
the power set of the set J of the network inputs (in-
dices), where Jij is the jth input of the ith neuron.
Further on for simplicity, network inputs with their in-
dices are identied. Let g :P(J) [0; 1] be a functiondened as follows:
g() = 0;
...
g({Jij}) = w(1)ij ; 16j6 n0;
...
g({Jij1 ; Jij2 ; : : : ; J ijr}) = w(1)ij1
w(1)ij2 w(1)ijr
;
where {j1; j2; : : : ; jr} {1; 2; : : : ; n0};
...
g({Ji1; Ji2; : : : J ij; : : : ; J in0 }) = 1;
where the notations and stand for the operationsMAX and MIN, respectively, in the unit real inter-
val [0; 1] R. It is proved [20] that the function g isa fuzzy measure on P(J). Therefore, the functional
assumes the form of the Sugeno integral over the -
nite reference set J.
Let h :P(I) [0; 1] be a function, dened in a sim-ilar way.
-
8/3/2019 A Fuzzy Back Propagation Algorithm
4/13
30 S. Stoeva, A. Nikov / Fuzzy Sets and Systems 112 (2000) 2739
Hidden neurons
net(1)i =
GP(J)
pG
xp
g(G)
;
a(1)i = f(net
(1)i ); i = 1; 2; : : : ; n1:
Output neurons
net(2) =
GP(I)
pG
a(1)p
h(G)
;
a(2) = f(net(2)):
The activation functions f(net) are linear ones or
equal to
f(net) =1
1 + e4(net0:5);
in order to obtain f(net) [0; 1].Therefore, the activation values a
(l)i [0; 1], where
l = 1; 2.
The error signal is obtained by back propagation.
Output layer
(2) = |t a(2)|: (2)
Hidden layer
(1)i = |t a
(1)i |; i = 1; : : : ; n1: (3)
In order to obtain wnewij [0; 1] the weights are handledas follows:
Case t= a(2): No adjustment of the weights is
necessary.
Case ta(2):
If woldij t then wnewij = 1 (w
oldij + wij).
If woldij t then wnewij = w
oldij .
Case ta(2):
If woldij t then wnewij = 0 (w
oldij wij).
If woldij 6 t then wnewij = w
oldij ,
where wij = (l)i ;
(l)i is determined by formula (2)
resp. (3).
In the following let us denote the iteration steps
of the proposed algorithm by s, the corresponding
weights by w(l)ij (s) and the activation values by a(l)i (s).
2.3. Necessary and sucient conditions for
convergence of FBP algorithm for single-output
networks
The following nontrivial necessary and sucientconditions for convergence of FBP algorithm in case
of single-output neural networks will be formulated
and proved: the interval between the minimum and
maximum of the patterns inputs shall include the pat-
terns output, i.e. the target value.
2.3.1. Convergence condition in case of single
training pattern
Denition 1. FBP algorithm is convergent to the tar-
get value t if there exists a number s0 N+ such thatthe following equalities hold:
t= a(2)(s0) = a(2)(s0 + 1) = a
(2)(s0 + 2) = ;
w(l)ij (s0) = w
(l)ij (s0 + 1 ) = w
(l)ij (s0 + 2 ) = ; l = 1; 2:
The following lemma concerns the outputs of the
neurons at the hidden layer.
Lemma 1.
(i) Ifa
(1)
i (s)t then a
(1)
i (s + 1)
t.(ii) Ifa(1)i (s)t then a
(1)i (s + 1)6 t.
The proof of this lemma is similar to that one of the
lemma in [19].
The following lemma concerns the output of the
output neuron:
Lemma 2.
(i) Ifa(2)(s)t then a(2)(s + 1) t;
(ii) Ifa(2)(s)t then a(2)(s + 1)6 t.
Proof. In the sequel for each set G P(I),
hs(G) =
pG
w(2)1p (s); vs(G) =
pG
a(1)p (s);
a2(s) =
GP(I)
[vs(G) hs(G)]:
(i) Since a(2)(s)t, and P(I) is a nite set, there
exists a set Gs P(I) such that
vs(Gs) hs(Gs) = a(2)(s)t;
-
8/3/2019 A Fuzzy Back Propagation Algorithm
5/13
S. Stoeva, A. Nikov / Fuzzy Sets and Systems 112 (2000) 2739 31
i.e.
vs(Gs) a(2)(s)t and hs(Gs) a
(2)(s)t:
By Lemma 1 it follows that vs+1(Gs) t. Since Gs
is a nite set, there exists an input ps Gs, such thatw
(2)1ps
(s) = hs(Gs). Therefore, the weight w(2)1ps
(s + 1) is
produced through the formula
w(2)1ps
(s + 1) = 0 (w(2)1ps (s) |t a(2)(s)|)
0 (w(2)1ps (s) |t a(2)(s)|)
= 0 (w(2)1ps (s) a(2)(s) + t)
0 (a(2)(s) a(2)(s) + t)
= 0 t
= t:
So, vs+1(Gs) w(2)1ps
(s + 1) t. Let
a(2)(s + 1) = vs+1(Gs+1) hs+1(Gs+1)
for some set Gs+1 P(I). Then
a(2)(s + 1) = vs+1(Gs+1) hs+1(Gs+1)
vs+1(Gs) w(2)1ps (s + 1) t:
Thus, a(2)(s + 1) t.
(ii) Since a(2)(s)t, and P(I) is a nite set, there
exists a set Gs P(I) such that
vs(Gs) hs(Gs) = a(2)(s)t:
Let
a(2)(s + 1) = vs+1(Gs+1) hs+1(Gs+1)
for some set Gs+1 P(I). Since Gs+1 is a nite set,there exists an input ps+1 Gs+1, such that
w(2)1ps+1
(s + 1) = hs+1(Gs+1):
Then it holds:
ta(2)(s) = vs(Gs) hs(Gs)
vs(Gs+1) w(2)1ps+1
(s):
If vs
(Gs+1
)6 a(2)(s)t, then by Lemma 1 it follows
that vs+1(Gs+1)6 t, and thus a(2)(s + 1)t.
If w(2)1ps+1
(s)6 a(2)(s)t, the weight w(2)1ps+1
(s + 1)
is produced through the formula
w(2)1ps+1
(s + 1) = 1 (w(2)1ps+1 (s) + |t a(2)(s)|)
6 1 (w(2)1ps+1 (s) + |t a(2)(s)|)
= 1 (w(2)1ps+1 (s) + t a(2)(s))
6 1 (a(2)(s) + t a(2)(s))
= 1 t
= t:
Hence a(2)(s + 1)6 t.
Theorem 1. FBP algorithm is convergent to the tar-
get value t i for the neurons at the input layer thefollowing condition holds:
jj(a(0)j t & a(0)
j 6 t): (4)
Proof. The if part of Theorem 1 is proved by as-
suming that condition (4) is not satised.
Suppose there is no input j J such that a(0)j t.
Then for each input j J, 16j6 n0, it holds thata
(0)j t. Hence, for each set G P(J) it holds that
v(G) =
pG
a(0)p t:
So, at each iteration step s it holds that
a(1)i (s) =
GP(J)
[v(G) gis(G)]
t
GP(J)
gis(G)
= t 1 = tfor each i, 16 i6 n1, where gis(G) =
pG w
(1)ip (s).
Therefore, at each iteration step s it holds that
a(2)(s) =
GP(I)
[vs(G) hs(G)]
=
GP(I)
pG
a(1)p (s) hs(G)
GP(I)[t hs(G)]
-
8/3/2019 A Fuzzy Back Propagation Algorithm
6/13
32 S. Stoeva, A. Nikov / Fuzzy Sets and Systems 112 (2000) 2739
= t
GP(I)
hs(G)
= t 1 = t:
Thus, in such case FBP algorithm is not convergent
to the target value t.
The assumption that there is no input j J, suchthat a
(0)j 6 t, implies in a similar way that at each
iteration step s, a(2)(s)t holds. Thus, in this case
FBP algorithm is not convergent to the target value t,
either.
The only if part of Theorem 1 is proved by as-
suming that FBP algorithm is not convergent to the
target value t. Hence, there exists no iteration step
s0 N+, such that a(2)(s0) = t, i.e. at each iteration
step s it holds either a(2)(s)t or a(2)(s)t.If a(2)(1)t is fullled, then at each iteration step
s, a(2)(s)t is fullled, too. The proof is performed
through mathematical induction. Let a(2)(1)tbe ful-
lled indeed. Since FBP algorithm is not convergent
by Lemma 2, it follows that a(2)(2)t. Let now at
the sth iteration step a(2)(s)tbe fullled. Since FBP
algorithm is not convergent by Lemma 2, it follows
that a(2)(s + 1)t. According to the mathematical in-
duction axiom at each iteration step s, a(2)(s)t is
satised.Since I is a nite set, at each iteration step s there
exists a set Gs P(I) such that
vs(Gs) hs(Gs) = a(2)(s)t;
i.e. vs(Gs)t and hs(Gs)t.
Let s0 N+ be the iteration step, at which the pro-cess of adjusting all the weights of the network comes
to an end. Then the above inequalities can be fullled
only ifhs0 (Gs0 ) = 1, i.e. if set Gs0 is equal to I, Gs0 I,
and for each input i, 16 i6 n1
, a(1)
i(s
0)t holds.
Since J is a nite set, for each neuron i, 16 i6 n1,
at the hidden layer there exists a set Gis0 P(J), suchthat
v(Gis0 ) gis0 (Gis0 ) = a(1)i (s0)t;
i.e. v(Gis0 )t and gis0 (Gis0 )t. Since the pro-
cess of adjusting all the weights has already been
stopped, the above inequalities can be fullled only if
gis0 (Gis0 ) = 1, i.e. if set Gis0 is equal to J, Gis0 J, and
for each input j, 16j6 n0
, a(0)
jt holds. Therefore,
condition (4) is not satised.
The assumption that a(2)(s)t holds, implies in a
similar way that condition (4) is not satised in that
case, either.
2.3.2. Convergence condition in case of
multiple-training patterns
Denition 2. In case of multiple-training patterns
(Xm; t); m = 1; : : : ; M , FBP algorithm is convergent to
the target value t if there exists a number of s0 N+
such that the following equalities hold:
t= a(2)m (s0) = a(2)m (s0 + 1) = a
(2)m (s0 + 2) = ;
m = 1; : : : ; M ;
w(l)ij (s0) = w
(l)ij (s0 + 1) = w
(l)ij (s0 + 2) = ;
l = 1; 2:
Theorem 2. Let more than one training pattern
(Xm; t); m = 1; : : : ; M ; be given. Let a(0)mj ; a
(0)mj be in-
puts of the mth pattern; such that Theorem 1 holds;
i.e. a(0)mj t and a
(0)mj6 t. Let
a
(0)
m
j
=
M
m=1 a
(0)
mj
and a
(0)
m
j
=
M
m=1 a
(0)
mj
:
Then FBP algorithm is convergent to the target value
t i the following condition holds:
a(0)mj t & a
(0)mj6 t: (5)
Proof. Let condition (5) be satised. Since the in-
equalities t6 a(0)mj6 a
(0)mj and t a
(0)mj a
(0)mj hold
for each m; m = 1; : : : ; M , according to Theorem 1.
FBP algorithm is convergent to the target value t for
each m; m = 1; : : : ; M .Let now FBP algorithm be convergent to the tar-
get value t for each m; m = 1; : : : ; M . Then according
to Theorem 1 the inequalities a(0)mj t and a
(0)mj6 t
hold for each m = 1; : : : ; M . Therefore, condition (5)
is satised, too.
3. Simulation results
With the help of a computer simulation we demon-
strate the proposed FBP algorithm using two neural
-
8/3/2019 A Fuzzy Back Propagation Algorithm
7/13
S. Stoeva, A. Nikov / Fuzzy Sets and Systems 112 (2000) 2739 33
Fig. 2. Example neural network with single-output neuron in the case of single- and multiple-training patterns.
networks with single-output neuron in case of single-
training pattern (cf. Fig. 2) and in case of multiple
training patterns (cf. Figs. 2 and 7). The FBP algo-
rithm is compared with SBP algorithm (cf. Fig. 8).
3.1. Single-training pattern
In case of single-training pattern FBP algorithm was
iterated with linear activation function, training ac-
curacy 0.001 (error goal), learning rate = 1:0 and
target value t= 0:8 (cf. Fig. 2). According to The-
orem 1 FBP algorithm is convergent to target value
t belonging to the interval between the minimal and
maximal input values [a(0)
j ; a(0)
j ] = [ 0:1; 0:9]. Fig. 3 il-
lustrates that for target values t= 0:2; 0:5 and 0.8
belonging to the interval [0:1; 0:9] the network was
trained only for 2, respectively, for 3 steps. If the
target value (t= 0:0 and 1.0) is outside the interval
[0:1; 0:9] the network output cannot reach it and the
training process is not convergent. This conrms the
-
8/3/2019 A Fuzzy Back Propagation Algorithm
8/13
34 S. Stoeva, A. Nikov / Fuzzy Sets and Systems 112 (2000) 2739
Fig. 3. Output values in case of single-training pattern for dierent target values.
Fig. 4. Output values in case of multiple-training patterns for dierent target values.
-
8/3/2019 A Fuzzy Back Propagation Algorithm
9/13
S. Stoeva, A. Nikov / Fuzzy Sets and Systems 112 (2000) 2739 35
Fig. 5. Sum-squared error in case of multiple-training patterns for dierent target values.
theoretical results. Here the nal output values are 0:1
and 0:9 for t= 0:0 and for t= 1:0, respectively.
3.2. Multiple training patterns
FBP algorithm was iterated with ve training pat-
terns (cf. Fig. 2), linear activation function, error
goal 0:001 and learning rate = 1:0. The interval
between the maximum of minimal input values and
minimum of maximal input values of the training
patterns is [a(0)mj ; a
(0)mj] = [ 0:4; 0:8]. Thus, according
to Theorem 2 for the target values t= 0:4; 0:6 and 0:8
belonging to the interval [0:4; 0:8] the convergence ofFBP algorithm is guaranteed. As illustrated in Fig. 4,
FBP algorithm trains the network only for 10 steps.
If the target value (t= 0:0 and 1:0) is outside the
interval [0:4; 0:8] the output value oscillates. There is
no convergence outside this interval. This conrms
the theoretical result.
The sum-squared error for the targets t= 0:4; 0:6
and 0:8 reaches the error goal 0:001 only for two
epochs (cf. Fig. 5) where during one epoch all ve
training patterns are learned. For target values outside
the interval [0:4; 0:8] the network cannot be trained
and the nal sum-squared error remains constant equal
to 0:59 for t= 0:0 and 0:26 for t= 1:0.
From the denition of FBP algorithm (cf.Section 2.2) it follows that its convergence speed is
maximal for the learning rate = 1:0. Fig. 6 illus-
trates this theoretical result. In case of single- and
multiple-training patterns only 3 and 10 steps, respec-
tively, are needed for training the network at learning
rate = 1:0. If the learning rate decreases the training
steps needed increase. For example for = 0:01 589
training steps in case of single-training pattern and
870 training steps in case of multiple-training patterns
are needed.
3.3. Comparison of FBP algorithm with SBP
algorithm
The convergence speed of FBP algorithm was
compared with that one of SBP algorithm (cf. Fig. 8)
for a network with four hidden layers (cf. Fig. 7),
100 randomly generated training patterns, randomly
generated initial network weights, error goal 0:02
and target value t= 0:8 belonging to the interval
[a(0)j ; a(0)
j ] = [ 0:4; 0:8]: FBP algorithm is signicantly
-
8/3/2019 A Fuzzy Back Propagation Algorithm
10/13
36 S. Stoeva, A. Nikov / Fuzzy Sets and Systems 112 (2000) 2739
Fig. 6. Dependence between learning rate and training steps needed for single-training pattern (STP) and multiple-training patterns (MTP).
Fig. 7. Example neural network with single-output neuron and four hidden layers.
-
8/3/2019 A Fuzzy Back Propagation Algorithm
11/13
S. Stoeva, A. Nikov / Fuzzy Sets and Systems 112 (2000) 2739 37
Fig. 8. Comparison of FBP and SBP algorithms in terms of steps needed for convergence in case of multiple-training patterns for two
learning rates (LR).
faster (seven epochs) than SBP algorithm (5500epochs) for learning rate = 0:01. SBP algorithm is
not convergent for learning rate 0:01. In Fig. 8
this is shown for = 1:0. For the same learning rate
FBP algorithm is convergent only for two epochs.
Numerous other experiments with dierent neu-
ral networks, learning rates and training patterns
conrmed the greater convergence speed of FBP al-
gorithm over SBP algorithm. Of course, this speed
depends also on the initial network weights.
4. Conclusions
In this paper we propose a fuzzy extension of the
backpropagation algorithm: fuzzy backpropagation al-
gorithm. This learning algorithm uses as net function
the fuzzy integral of Sugeno. Necessary and sucient
conditions for its convergence for single-output net-
works in case of single- and multiple-training patterns
are dened and proved. A computer simulation con-
rmed these theoretical results.
FBP algorithm has a number of advantages com-pared to SBP algorithm:
(1) greater convergence speed implying signicant
reduction in computation time what is important
in case of large sized neural networks,
(2) reaches always forward to the target value without
oscillations and there is no possibility to fall into
local minimum,
(3) requires no assumptions about probability distri-
butions and independence of input data (patterns
inputs),(4) does not require initial weights to be dierent from
each other,
(5) enables the automation of the weights tuning pro-
cess (quasi-unsupervised learning) by pointing
out the interval where the target value belongs to
[14,15]. The target values are determined:
automatically by computer in the interval,specied by input data,
semi-automatically by experts that dene thedirection and degree of target value changes.
Thus the actual target value is determined by
-
8/3/2019 A Fuzzy Back Propagation Algorithm
12/13
38 S. Stoeva, A. Nikov / Fuzzy Sets and Systems 112 (2000) 2739
fuzzy sets relevant to predened linguistic
expressions.
Another advantage of FBP algorithm is that it presents
a generalisation of the fuzzy perceptron algorithm
[19].FBP algorithm can support knowledge acquisition
and presentation especially in cases of uncertain im-
plicit knowledge, e.g. for construction of Common
KADS models in creative design process [5]; for in-
telligent data analysis [1]; for creating adaptive and
adaptable interfaces [4], for ergonomic assessment of
products [9], etc. It can be implemented as a module of
a CAKE-tool like CoMo-Kit [12] or as a stand-alone
tool for knowledge acquisition in systems for analysis
and design like ERGON EXPERT [7] and ISO 9241
evaluator [16].The ability of FBP algorithm for quasi-unsupervised
learning was successfully implemented for the cre-
ation of adaptable user interface of the information
system of Bulgarian parliament [15] and usability
evaluation of hypermedia user interfaces [17].
Finally, we should say that this study is not yet ap-
plied to many complex problems and the results are
not compared with dierent modications of SBP al-
gorithm. However, the current results are encouraging
for continuation of our work. We will have to make
many experiments and practical implementations of
FBP algorithm.
Appendix 1: Notation list
xj the jth patterns input
net(l)i net value of the ith neuron at the lth layer
a(l)i activation value of the ith neuron at the
lth layer
w(l)ij weight of the jth input of the ith neuron
at the lth layer
f(net) activation function
t the patterns output, i.e. target value
(l)i error signal relevant to the ith neuron at
the lth layer
s the sth iteration step
w(l)ij (s) weight w
(l)ij at the sth step
a(l)
i(s) activation value a
(l)
iat the sth step
wij weight change
woldij weight w(l)ij (s)
wnewij weight w(l)ij (s + 1)
the empty set
R the set of real numbers
N+
the set of positive integersJ the set of (indices of) network inputs
I the set of (indices of) neurons at the
hidden layer
P(I) the power set of the set I
max operation in unit interval [0; 1] min operation in unit interval [0; 1]
a(l)m activation value a
(l) relevant to the mth
pattern
tm the patterns output of the mth pattern
g fuzzy measure on the set P(J)
h fuzzy measure on the set P(I)
References
[1] J. Angstenberger, M. Fochem, R. Weber, Intelligent data
analysis methods and applications, in: H.-J. Zimmermann
(Ed.), Proc. 1st Eur. Congr. on Fuzzy and Intelligent
Technologies EUFIT93, Verlag Augustinius Buchhandlung,
Aachen, 1993, pp. 10271030.
[2] M. Brown, C. Harris, Neurofuzzy Adaptive Modelling and
Control, Prentice-Hall, New York, 1994.[3] D. Dubois, H. Prade, Fuzzy Sets and Systems: Theory and
Applications, Academic Press, New York, 1980.
[4] G. Grunst, R. Oppermann, C.G. Thomas, Adaptive and
adaptable systems, in: P. Hoschka (Ed.), A New Generation
of Support Systems, Lawrence Erlbaum Associates, Hillsdale,
1996, pp. 2946.
[5] R. de Hoog et al., The Common KADS model set,
ESPRIT Project P5248 KADS-II, University of Amsterdam,
Amsterdam, 1993.
[6] E. Ikonen, K. Najim, Fuzzy neural networks and application
to the FBC process, IEE Proc. Control Theory Appl. 143 (3)
(1996) 259 269.
[7] M. Jager, K. Hecktor, W. Laurig, Ergon expert a knowledge- based system for the evaluation and design of manual
materials handling, in: Y. Queinnec, F. Daniellou (Eds.),
Proc. 11th Congr. of IEA Designing for Everyone, Taylor &
Francis, London, 1991, pp. 411413.
[8] M. Keller, H. Tahani, Backpropagation neural network for
fuzzy logic, Inform. Sci. 62 (1992) 205221.
[9] J.-H. Kirchner, Ergonomic assessment of products: some
general considerations, in: Proc. 13th Triennial Congr.
International Ergonomics Association, vol. 2, Tampere,
Finland, 1997, pp. 5658.
[10] C.-H. Lin, C.-T. Lin, An ART-based fuzzy adaptive learning
control network, IEEE Trans. Fuzzy Systems 5 (4) (1997)477496.
-
8/3/2019 A Fuzzy Back Propagation Algorithm
13/13
S. Stoeva, A. Nikov / Fuzzy Sets and Systems 112 (2000) 2739 39
[11] D. Nauck, F. Klawonn, R. Kruse, Foundations on Neuro-
Fuzzy Systems, Wiley, Chichester, 1997.
[12] S. Neubert, F. Maurer, A tool for model-based knowledge
engineering, Proc. 3rd KADS Meeting, Munich, 1993,
pp. 97112.
[13] A. Nikov, G. Matarazzo, A methodology for human factorsanalysis of oce automation systems, Technol. Forecasting
Social Change 44 (2) (1993) 187197.
[14] A. Nikov et al., ISSUE: an intelligent software system
for usability evaluation of hypermedia user interfaces, in:
A.F. Ozok, G. Salvendy (Eds.), Advances in Applied
Ergonomics, USA Publishing Co., West Lafayette, 1996,
pp. 10381041.
[15] A. Nikov, S. Delichev, S. Stoeva, A neuro-fuzzy approach
for user interface adaptation implemented in the information
system of Bulgarian parliament, in: Proc. 13th Int.
Ergonomics Congr. IEA97, vol. 5, Tampere, Finland, 1997,
pp. 100112.
[16] R. Oppermann, H. Reiterer, Software evaluation using 9241evaluator, Behaviour Inform. Technol. 16 (4=5) (1997)
232245.
[17] R. Rojas, Neural Networks, A Systematic Introduction,
Springer, Berlin, 1996.
[18] D.E. Rumelhardt, G.E. Hinton, R.J. Williams, Learning
internal representations by error propagation, in: D.E.
Rumelhardt, J.L. McClelland, PDP Research Group (Eds.),
Parallel Distributed Processing. Exploration in the Micro-structure of Cognition, vol. 1, foundations, MIT Press,
Cambridge, MA, 1986, pp. 318362.
[19] S. Stoeva, A weight-learning algorithm for fuzzy production
systems with weighting coecients, Fuzzy Sets and Systems
48 (1) (1992) 8797.
[20] S. Stoeva, A. Nikov, A fuzzy knowledge-based mechanism
for computer-aided ergonomic evaluation of industrial
products, Simulation Control C 25 (3) (1991) 1730.
[21] S. Wang, N.P. Archer, A neural network technique in
modelling multiple criteria multiple person decision making,
Comput. Oper. Res. 21 (2) (1994) 127142.