a comparative study of the max and sum machine-learning algorithms using virtual fuzzy sets

Pergamon PII:S0952-1976(96)00045-0

EngngApplic. Artif. Intell. Vol. 9, No. 5, pp. 515-522, 1996 Copyright © 1996 Elsevier Science Ltd

Printed in Great Britain. All rights reserved 0952-1976/96 $15.00+0.00

Contributed Paper

A Comparative Study of the MAX and SUM Machine- Learning Algorithms using Virtual Fuzzy Sets

K. C. C H A N

University of New South Wales, Australia

(Received August 1995; in revised form March 1996)

This paper presents an adaptive fuzzy learning algorithm for non-linear system identification. The algorithm is based on a concept called a "virtual fuzzy set". The concept provides a new representation for the consequent part of a fuzzy production rule. A virtual fuzzy set, which consists of two consecutive fuzzy sets with different degrees of membership, is an imaginary fuzzy set located at a location most appropriate for a numerical training data point. The aim of virtual fuzzy set is to increase the system accuracy, at minimum expense in terms of computer resources. The concept is incorporated into a one-pass build-up adaptive algorithm for non-linear system identification. The results show that better performance can be achieved, compared with traditional fuzzy sets. Two variations of the learning algorithm, using the MAX or SUM of rule degrees, are proposed. The performance of these two approaches in identifying processes with different noise levels is evaluated. Copyright © 1996 Elsevier Science Ltd

Keywords: Fuzzy systems, adaptive clustering, adaptive control, adaptive prediction, fuzzy control, system identification, system modeling.

1. INTRODUCTION

There has been significant research interest in the area of fuzzy identification since the late 1970s. A number of algorithms have been proposed to obtain fuzzy rules from learning data. Takagi and Sugeno proposed an off-line algorithm to model a non-linear static system using linearised input-output data pairs, t A repeated identification process is employed to continuously tune the rule base until a performance index is satisfied. More recently, Sugeno and Yasukawa proposed a more general qualitative modeling approach, in which fuzzy identification is considered as only one (but the most important) of the tasks. 2 Xu and Lu developed a self-learning fuzzy model-identification algorithm for multi-input/multi-output dynamic systems) Liu and Yu proposed an on-line approximator for a non-linear dynamic system using fuzzy logic. However, the requirement for the measurements of full state variables limits this method only to some special c a s e s . 4 Later on, Liu and Yu developed an on-line modeling method which only requires input-output information. 5 The on-line approximator con-

Correspondence should be sent to: Dr K. C. Chan, School of Mechanical and Manufacturing Engineering, University of New South Wales, Sydney 2052, Australia. E-maih [email protected].

sists of a number of well-defined continuous fuzzy variables and a small rule base which contains unknown parameters. The parameters are tuned according to the input-output data pairs. Wang and Mendel proposed a general method for combining both numerical and linguistic information into a fuzzy associative memory bank. 6 Batur and Srinivasan proposed an integrated approach 7"s which uses the quantisa- tion scheme suggested by Wang and Mendel, and then the quantised data is sent to an inductive learning system, the ID3 decision tree generator, 9 for automatic generation of production rules. Pfeiffer suggested an identification algorithm based on a CoS (Centre of Singletons) algorithm, j°'" The algorithm is formulated as a linear least-squares optimisation problem.

This paper proposes an on-line adaptive learning algorithm based on the concept of virtual fuzzy sets. The concept of virtual fuzzy set was first proposed and used in a fuzzy rule-generation scheme using a back-propagation neural network. 12']3 The neural network was used to learn the input-output numerical relationship, and virtual fuzzy rules were then generated by the neural network. To obtain fuzzy rules directly from numerical samples, a seven-step procedure 14 was developed, based on a previous fuzzy learning algorithm. 6 That procedure is in fact the MAX scheme that will be discussed in this paper. A successful

515

516 K.C. CHAN: MAX AND SUM MACHINE-LEARNING ALGORITHMS

application of the concept of virtual fuzzy sets and the MAX learning algorithm to the tuning of PI and PID feedback controllers has also been reported in the literature. ~5

In the previous work related to the concept of virtual fuzzy sets and the learning algorithm, the noise susceptibil- ity of the system was never investigated, and the performance of the systems is only compared with that of the traditional fuzzy-set approach. In an attempt to study the noise effect on the learning algorithm, an alternative new scheme, the SUM scheme, was developed. This paper will present both the MAX and the SUM learning algorithms, and their similarities and differences. The performance of the two schemes at different noise levels, using both virtual and traditional fuzzy sets, will also be evaluated.

For the MAX scheme proposed in earlier work, the idea is that the data point having the highest rule degree for a particular fuzzy set output will replace previous lower-rule- degree data points. Therefore, the rule degree of a data point is of vital importance, but the frequency of occurrence of a data point is irrelevant to the rule-generation scheme. It is arguable whether the frequency of occurrence should be considered in rule generation. The root of the problem lies in the definition of the truthfulness of a virtual fuzzy rule. It may be said that a rule is "more true" if there exists a data point with a very high degree for that particular rule; or a rule is "more true" if there exist more data points belonging to that particular rule, even though some of them may have a very low degree. It is not the aim of this paper to provide a philosophical answer to this problem but, rather, to

propose an alternative scheme (SUM) to investigate the effect of the frequency of occurrence on the accuracy of a system, and to compare the results with the previous (MAX) scheme. Which rule-generation scheme would be more accurate would greatly depend on the nature of the problem. However, it is envisaged that the usefulness of these two schemes may also depend on the system noise level. Therefore, the effect of noise level on system performance will be studied in this paper.

2. THE CONCEPT OF VIRTUAL FUZZY SET

The concept of virtual fuzzy sets was developed to improve the accuracy of traditional fuzzy inference proce- dures. To illustrate the concept, consider a single-input single-output mapping f:x---*y, where both the input space x and the output space y are quantised using seven identical triangular fuzzy sets (X0, X1 . . . . . X6), and (Y0, YI . . . . . Y6), respectively, as shown in Fig. 1. Numerical-point samples (xi, Yi) are required to generate fuzzy production rules to estimate the mapping. The (xi, Yi) data points are assumed to be error-free. According to the specific example shown in Fig. 1, the training sample (xi, y~) activates the fuzzy-set association (Xi, Y~) to a higher degree of membership than other fuzzy-set associations such as [(X i, Y/-I), (Xi, Yi-2) . . . . . (Xi, Yi+l), (Xi, Yi+2) . . . . ]. Therefore, the most accurate production rules that can be obtained are (X,, Y0 . . . . . (Xs, Ys). The next step is to use this five-rule fuzzy model to estimate the function y=f(x). According to

yory'

i i

i P

0.7 0.3

I¢1

y'"g(x!.. . . . . . . . . . . . . . . . . . . . . . . . . . . ~'~~(x)

(~ i~ .~ (xs ,Ys)

. . . . . . . . . . . . . . . . I . . . . I

XO XI X2 13 X4 15 X6 X

Fig. 1. Concept of a virtual fuzzy set.

K. C. CHAN: MAX AND SUM MACHINE-LEARNING ALGORITHMS 517

these five "most accurate" production rules, the best estimate that can be obtained would be the straight-line function y'=g(x), but not the true function y=f(x). The inaccuracy is mainly due to the locations of the fuzzy membership functions (X0, X1 . . . . . X6) and (Y0, YI . . . . . Y6), initially defined by the user. If there exist some imaginary fuzzy sets (V0, V1 . . . . . V6), which are called virtual fuzzy sets, the training sample (x~, y~) would activate the fuzzy-set association (X~, V;) to a degree of one. Therefore, the rule (X~, V~) would be the best representation for the training sample (x~, y;). The following sections will propose a representation for the virtual fuzzy sets by using the user-defined fuzzy sets, and a production rule generation procedure. There are two variations to the production rule generation procedure: the MAX and the SUM of the rule degrees. Their performance in identifying non-linear systems with differing noise levels will be discussed.

3. THE NEW FUZZY RULE REPRESENTATION SCHEME

In the new fuzzy rule representation scheme, the antecedent part of a production rule remains the same as that of the traditional fuzzy rule representation; only the consequent part is modified. A consequent term is expressed in terms of two consecutive user-defined fuzzy sets, to which degrees are assigned. A virtual fuzzy set is determined from the centroid of the consecutive fuzzy sets. As depicted on the left-hand side of Fig. l, a typical new fuzzy rule representation is:

consists of seven steps. All the steps are the same, with the exception of Step 4.

Step 1: The input and output spaces are quantised using fuzzy sets

Specify the domain intervals in which the values of the input and output variables are likely to lie. Triangular fuzzy membership functions are employed to quantise the input and output spaces.

Step 2: Generate a potential fuzzy rule from a data sample The rule generated in this step is only a potential

candidate, which may or may not be included in the final rule base. If the IF part of the rule is new, the rule is considered as a new rule and will be included in the rule base. Otherwise, it is necessary to determine the virtual fuzzy set representation for the THEN part of the rule, and to resolve any conflict (if there is one). This procedure will be explained in Step 4.

The degrees of membership of a given sample n(x~ n~, x~2% y<n)) in different fuzzy sets are determined first. For example ~"~, in Fig. 2 has degree 0.25 in P1, degree 0.75 in P2, and zero degree in all other fuzzy sets. Similarly, x~ n) has non- zero degree 0.67 in N3, degree 0.33 in N2; and y<n~ has degree 0.9 in CE, degree 0.1 in P1. To assign a single fuzzy set for each variable, the fuzzy set in which the degree of membership is the maximum is chosen. Therefore, for the sample shown in Fig. 2, the potential rule is:

Rule n: IF is P2 and is N3, THEN y is CE.

"IF X is X4, THEN Y is (Y3 to a degree of 0.3 and Y4 to a degree of 0.7)"

or equivalently,

"IF X is X4, THEN Y is V4".

The rule can be alternatively expressed as (X4; Y3/0.7, Y4/0.3). The consequent part of the fuzzy association is a normalised combination of degree values. The centre value of the virtual fuzzy set V4 is determined by the fuzzy centroid defuzzification method:

i ykrnyk(yi)

k=i- I

Vi = ~ mvt (Y i ) , (1 ) k=i- I

where Yk denotes the centre value of fuzzy set Yk, and mvk(y ~) the degree of membership of Yi assigned to the fuzzy set Yk.

Step 3: Assign a degree to the rule The degree of a rule is defined as the product of the

maximum degree of membership of each variable. For the rule "IF x~ is A and x2 is B, THEN y is C", the degree of this rule is:

D(Rule n)=mA(xl) × ms(x2) × mc(y).

As an example, Rule n has degree

D(Rule n)=m~(~n))mN3(x~n))mcE(Y cn~)

= (0.75)(0.67)(0.9) = 0.45.

Step 4: Assign a rule to the rule base Step 4 for the MAX and SUM schemes are different. The

procedure for the MAX scheme is first presented, followed by the SUM scheme.

For the max scheme: There are three possible cases in assigning a rule to the rule base.

4. THE MAX/SUM FUZZY RULE GENERATION PROCEDURE

This section will explain both the MAX and SUM fuzzy rule generation procedure simultaneously. Each scheme

Case 1: The FAM cell is empty. If the FAM cell is empty (i.e., there is no existing rule

with the same antecedent part as the current potential rule), assign the new rule to the rule base with a normalised rule certainty of one. It should be noted that the rule degree must


re(x+ )

1.0 0.75

0 2 5

0.C

N3 N2 N1 CE P1 P2 P3

;; x , . .s xp) x,+ -60 .4O -20 0 20 40 6O

x 1

m(x 2 )

1.0

0.67

0.33

0.0

N6 N5 N4 N3 N2 N1 CE P1 P2 P3 P4 P5 1='6

X~ X~ n) X2- 6.67 X~ -60-50-40 -30-20-10 0 10 20 30 40 50 60

x 2

re(y)

1.0 0.9

0.1 0.0 y

N3 N2 N1 CE P1 P2 1:'3

y- -75 -37.5 yln) y+

-150 -100 -50 0 50 100 150

Fig. 2. Fuzzy membership functions for the input and output spaces.

Table 1. The MAX scheme assignment of fuzzy rules in the FAM bank

Training Maximum Maximum Maximum Rule Rule Consequent Rule expressed as Sample degree degree degree degree operation term of the rule virtual fuzzy sets

for x~ for x2 for y

1 (P2/0.75) (N3/0.67) (CFJ0.90) 0.45 (CE/0.45) (CE; 0.45) (P2,N3;CE/I) 2 (P2/0.60) (N3/0.55) (PI/0.70) 0.23 (CE/0.45) (CE/0.45,PI/0.23) (P2,N3;CFJ0.66,P 1/0.34)

(P1/0.23) 3 (P2/0.60) (N3/0.55) (P2/0.55) 0.18 (CE/0.45) (CE/0.45,PI/0.23) (P2,N3 ;CE/0.66,P 1/0.34)

(P1/0.23) 4 (P2/0.70) (N3/0.62) (P3/0.75) 0.33 (CFJ0.45) (CE/0.45,PI/0.23) (P2,N3;CEIO.66,PI/0.34)

(P1/0.23) 5 (P2/0.90) (N3/0.95) (Ni/0.85) 0.73 (N1/0.73) (NI/0.73,CE/0.45) (P2,N3;N 1/0.62,CE/0.38)

(CE/0.45) 6 (P2/0.95) (N3/0.78) (N1/0.87) 0.64 (N1/0.73) (N 1/0.73,CF--/0.45) (P2,N3;N 1/0.62,CF-./0.38)

(CE/0.45) 7 (1)2/0.85) (N3/0.77) (CFJ0.90) 0.59 (N1/0.73) (NI/0.73,CE/0.59) (P2,N3;N 1/0.55,CE/0.45)

(CFJ0.59)


be stored in the rule base, as it will be required when more samples become available. Table 1 lists a sequence of rule operations and assignments for a set of training samples. As an example for this case, training sample 1 is assigned to an empty FAM cell.

Case 2: The FAM cell is non-empty, and the consequent part of the potential rule suggests a new linguistic label as output.

A conflicting rule occurs in this case. It is necessary to resolve the conflict by choosing the rule with the highest confidence. For this case, the rule degree of the suggested linguistic label is compared with the maximum degree of the two existing adjacent consequent linguistic labels. The one with higher degree will be chosen first. The immediate neighbours (if they exist in the consequent part of the rule) of the chosen linguistic label (the one with the highest degree) are then compared, and the one with the higher degree among the two neighbours will be chosen as the second fuzzy set. The final step is to normalise the degrees and to express the rule as virtual fuzzy sets. Samples 2 to 5 in Table 1 are typical examples of this case.

Case 3: The FAM cell is non-empty, and the consequent part of the potential rule suggests a linguistic label which is the same as one of the output linguistic labels in the FAM cell.

For this case, the degree of the suggested linguistic label is compared with that of the same label in the FAM cell, and the maximum degree is kept. Although the consequent terms remain the same in this case, the degrees need to be normalised again. Samples 6 and 7 in Table 1 are typical examples of this case.

When a new sample is available, steps 2 to 4 are

performed repeatedly, and the FAM bank is modified adaptively.

For the SUM scheme: The SUM scheme is simpler compared to the MAX scheme, but more computer storage is needed to store the sums of output fuzzy sets for each rule. To use the same example as the MAX scheme, the rule assignments are shown in Table 2.

For the SUM scheme, it does not matter whether the FAM cell is empty or not, the rule degree is always added to the current degree of the same output fuzzy set. As in training sample 6 shown in Table 2, a degree of 0.64 is added to the current degree value 0.73 of the N1 fuzzy set. Then the fuzzy set with the highest degree will be chosen first, followed by the immediate neighbour which has the higher degree. The virtual fuzzy rule is finally obtained by normalising the degrees of the chosen fuzzy sets.

In the MAX scheme, only the data points having the highest degrees contribute to the fuzzy rule base, and all other data points having lower degrees are deleted. For this approach the accuracy of the rule base is greatly dependent on the accuracy of the learning data points, especially those having an input to the rule base. The accuracy of the rule base would be greatly affected if there are (inaccurate/ noisy) data points having very high degrees and dominating certain fuzzy rules. In the SUM scheme, all data points make a contribution to the fuzzy rule base. The level of a data point's contribution to a fuzzy rule depends on its rule degree. Therefore, the SUM scheme would be more accurate than the MAX scheme when the learning data points are noisy.

In terms of computer resource requirements, the MAX scheme is more efficient, since only two consequent terms (the one with the highest degree and its immediate

Table 2. The SUM scheme assignment of fuzzy rules in the FAM bank

Training Maximum Maximum Maximum Rule Rule Consequent Rule expressed as Sample degree degree degree d eg ree operation term of the rule virtual fuzzy sets

for x~ for x2 for y

I (P2/0.75) (N3/0.67) (CE/0.90) 0 .45 (CE/0.45) (CE; 0.45) (P2,N3;CE/I) 2 (P2/0.60) (N3/0.55) (PI/0.70) 0 .23 (CE/0.45) (CE/0.45,PI/0.23) (P2,N3;CE/0.66,P I/0.34)

(P1/0.23) 3 (P2/0.60) (N3/0.55) (P2/0.55) 0 .18 (CE/0.45) (CFd0.45,PI/0.23) (P2,N3;CE/0.66,P 1/0.34)

(P!/0.23) (P2/0.18)

4 (P2/0.70) (N3/0.62) (P3/0.75) 0 .33 (CE/0.45) (CE/0.45,Pl/0.23) (P2,N3;CE/0.66,P 1/0.34) (P1/0.23) (P2/0.18) (P3/0.33)

5 (P2/0.90) (N3/0.95) (N1/0.85) 0 .73 (N1/0.73) (N 1/0.73,CE/0.45) (P2,N3;N 1/0.62,CE/0.38) (CE/0.45) (P1/0.23) (P2/0.18) (P3/0.33)

6 (P2/0.95) (N3/0.78) (NI/0.87) 0.64 (Nl/0.73+0.64) (NI/1.37,CE/0.45) (P2,N3;N 1/0.75.CE/0.25) (CE/0.45) (Pl/0.23) (P2/0.18) (P3/0.33)

7 (P2/0.85) (N3/0.77) (CE/0.90) 0 .59 (NI/1.37) (NI/1.37,CE/l.04) (P2,N3;N 1/0.57,CE/0.43) (CE/0.45 +0.59)

(P1/0.23) (P2/0.18) (P3/0.33)


neighbour) and their degrees are stored for each rule. The SUM scheme requires more on-line computer storage, since the accumulated degrees for all output fuzzy sets need to be stored for each rule. However, both learning schemes are very efficient when implemented as real-time adaptive algorithms.

Step 5: Combine expert linguistic rules with fuzzy rules generated from numerical data (optional)

This step is necessary only if linguistic information is available from a human expert. A linguistic rule can be entered into the FAM bank directly if it is a new rule. But if there exists a conflicting rule in the FAM bank, the conflict has to be resolved by the human expert.

Step 6: Assign weights to all the rules (optional) This step is optional. Weight assignment is a mechanism

to provide a final human expert adjustment to the rule base that is either automatically generated from sample data or assigned by a human expert. This step may be performed when some a priori information about the data set (such as accuracy, reliability and usefulness) is available. The weight actually represents the belief in the usefulness of a fuzzy production rule.

The weight of an automatically generated rule is equal to unity. When a weight is assigned to a rule, the centroid of the virtual fuzzy set does not change, but the height of the virtual fuzzy set is changed to a value equal to the weight. Consider the last sample listed in Table 1, (P2, N3; Nl/0.55, CE/0.45). If a weight of 0.6 is assigned to the rule, the rule representation becomes (P2, N3; N1/0.55, CE/0.45; 0.6), or equivalently, (P2, N3; N1/0.33, CE/0.27).

Step 7: Fuzzy inference and defuzzification After the rule base has been obtained from the given data,

the fuzzy system is ready to determine a mapping from the inputs (x~, x2) to the output y. The correlation-product fuzzy inference ~6 is used to compute the members of the output fuzzy set, weights them with the user-assigned weights (if any), and sums them to produce the output fuzzy set. The output fuzzy set is then defuzzified to obtain a numerical output by using the fuzzy centroid defuzzification scheme.

To illustrate the fuzzy inference and defuzzification procedure, the following standard rule representation is used:

ai=rrfin(mi, lj(xj), mi,2,(x2)). (3)

The virtual fuzzy set for Rule i is determined by combining the consequent fuzzy setsf~ and s~ as:

m,,,(y) =d, my,(y) + (1 - d,)ms,(y), (4)

or equivalently,

v,=d,F,+ (1 - d,)S~. (5)

The centre value of V~, denoted as vi, can be determined as the centroid of the fuzzy sets diF~ and (1 - d~)S~:

~,=d~f,+(1 - d~)~, (6)

where f, and ~i are the centre values of fuzzy sets F~ and Si, respectively. The input pair (x~, x2) activates the consequent part of each FAM rule to a different degree a~. With correlation-product encoding inference, the ith rule yields the weighted output fuzzy set O~:

or equivalently,

moi(Y)=alwimvi(y), (7)

Oi=aiwiVi. (8)

The fuzzy system then sums the output fuzzy set O i from each FAM rule to form the combined output fuzzy set O:

N

mo(Y)= E mo,(y), (9) i = l

or equivalently,

N

O= ~ O~, (10) i=1

where N is the number of fuzzy rules in the FAM bank.The control output u is equal to the fuzzy centroid of O:

~ymo(Y) dy u = ~mo(y ) dy

( l l )

To reduce computation, the following formula can be used to determine the output u for triangular membership functions:

Rule i: (inl~, in2i;f/di; s/(1 -d~); wi), (2)

where inli and in2i are the antecedent fuzzy sets, a degree of membership di is assigned to the first consequent fuzzy set f , and a degree of membership (1 -d i ) is assigned to the second consequent fuzzy set si. The input pair (Xl, x2) activates the antecedent fuzzy sets inli and in2~ to degrees ml, l,(xl) and mi,2,(xz), respectively.

The activation value of the consequent part of Rule i is equal to the minimum of the antecedent conjuncts' values. In this example, the activation value ai is determined as follows:

N

VtaiWi u= i=] (12)

N

a i w i i=l

5. APPLICATION TO NON-LINEAR SYSTEM IDENTIFICATION

The proposed MAX and SUM schemes have been applied to non-linear system identification. The system identified has been used by other researchers in fuzzy system modelling. 7'sJ2-14J7 The discrete form of the system


is given as:

y(t) =(0.8 - 0.5e _y2(,_ O)y(t _ 1) - (0.3 +0.9e-g('-I))y(t - 2)

+ u ( t - 1)+0.2u( t -2)+0.1u( t - l )u(t-2)+e(t) , (13)

where the system input u(0 is an independent sequence of uniform distribution with mean zero and variance 1.0; and the system noise e(t) is an independent Gaussian sequence with mean zero and different levels of standard deviation ranging from 0 (i.e., no noise) to 0.18. The input order of the model is four (two past inputs and two past outputs). Two sets of independent data, each containing 2000 points, are generated for learning and testing, respectively. As a first step, the fuzzy membership functions are defined for u(t) and y(t). The range of u(t) is defined as [ - 1.8, 1.8]; while the range of y(t) is defined as [ - 6, 6]. These ranges are divided into nine evenly spaced fuzzy sets for both the u(t) and y(t) during the simulation.

To evaluate the performance of the MAX and SUM schemes in modelling the system, the widely used Euclid- ean distance square between the actual and predicted output is used as the performance measure:

1 ~ [y(i)_~(i)]2, (14) 12~--- - -

12 i=l

The training data was used to generate fuzzy rules using traditional and virtual fuzzy sets, with both the MAX and SUM schemes, at different noise levels. The fuzzy models were then applied to predict the output of an independent testing data set. An example of the prediction is given in Fig. 3. The performance measure of the tests is summarized in Fig. 4.

For the same scheme (either MAX or SUM), the virtual fuzzy-set approach (the two bottom lines in Fig. 4) performs consistently better than the traditional fuzzy sets (the two top lines in Fig. 4) at all noise levels. For the same fuzzy-set approach (either traditional or virtual), the MAX scheme performs better than the SUM scheme at lower noise levels, while the SUM scheme performs better than the MAX scheme at higher noise levels in general. This can be explained as follows. When the noise level is low, a higher rule degree means a certain data point is more accurate, and thus, the MAX scheme is more suitable and would give more accurate prediction. When the noise level is high, the confidence that the rule degree accurately reflects the truthfulness of a certain data point will decrease. When the SUM scheme is used, the frequency of occurrence is taken into account, and this helps to average out the noise effects; thus, more accurate prediction can be achieved.

where y(t) is the actual process output, and ~(t) is the prediction of the process output obtained from the fuzzy system.

6. CONCLUSION

The development of fuzzy systems for non-linear system identification based on the virtual fuzzy set concept has

...... TradiUoul-MAX

.... Virtual-MAX

.... Traditiomd--SUM

3 d vi~d-SvM

2

-3

I !

..4

FI$. 3. Prediction for system wRh noise ievel,,O.15 sUmdemt deviation.

522 K. C. C H A N : M A X A N D S U M M A C H I N E - L E A R N I N G A L G O R I T H M S

Performance measure 0.19 --

0.18 o Traditiaual fu-zy set, MAX scheme /e

0.17 - 0 Virtual fuzzy set, MAX scheme J / ¢ ~ @ Traditional fuzzy set, SUM scheme ~ ' /

0.16 - o Virtual fuzzy set, SUM scheme / / " Q

0.13,

0.12 i

0.11 i

o.1o I I I I 1 I o 0.03 0.06 0.09 o.t2 0.~5 o.ts

Noise leve l (S .D.)

Fig. 4. Performance measure at different noise levels.

been presented in this paper. Virtual fuzzy sets can improve fuzzy-system accuracy. Two fuzzy rule generation schemes, the MAX and SUM schemes, were proposed. A comparative study of using these two schemes in identifying a non-linear system at various noise levels was also carried out. The results demonstrated that the MAX scheme is more accurate at lower noise levels, while the SUM scheme is more accurate at higher noise levels. It was also shown that the virtual fuzzy-set approach is more accurate than the traditional fuzzy-set approach, for both the MAX and the SUM schemes. The on-line computer storage requirement for the SUM scheme is higher, due to the fact that the accumulative degrees for all output fuzzy sets need to be stored for each rule, while the MAX scheme only stores two output fuzzy sets and two degree values for each rule. Further research is currently being undertaken to develop a new hybrid scheme, based on the MAX and SUM schemes, which can be adjusted to achieve the best results according to noise levels.

R E F E R E N C E S

1. Takagi, T. and Sugeno, M., Fuzzy identification of systems and its applications to modeling and control. IEEE Transactions on Systems, Man, and Cybernetics, 1985, 15, 116-132.

2. Sugeno, M. and Yasukawa, T., A fuzzy-logic-based approach to qualitative modeling. IEEE Transactions on Fuzzy Systems, 1993, 1(1), 7-31.

3. Xu, C. W. and Lu, Y. Z., Fuzzy model identification and self-learning for dynamic systems. IEEE Transactions on Systems, Man, and Cybernetics, 1987, SMC-17(4), 683-689.

4. Liu, K. and Yu, I. H., Adaptive fuzzy logic approximation of unknown

nonlinear systems: state-variable feedback tuning. Proceedings, Amer- ican Control Conference, Baltimore, ML, 1994, pp. 190-194.

5. Liu, K., Lewis, E L. and Yu, I. H., Fuzzy logic modeling for unknown nonlinear systems by input-output measurements. Proceedings, EXPERSYS-94, Expert Systems Applications Artificial Intelligence, Houston, TX, 1994, pp. 505-510.

6. Wang, L. X. and Mendel, J. M., Generating fuzzy rules by learning from examples. Proceedings of the 1991 IEEE Int. Symposium on Intelligent Control, Arlington, VA, 1991, pp. 263-268.

7. Batur, C., Srinivasan, A. and Chan, C. C., Automated rule based model generation for uncertain complex dynamic systems. Proceedings of the 1991 1EEE Int. Symposium on Intelligent Control, Arlington, VA, 1991, pp. 275-279.

8. Srinivasan, A., Batur, C. and Chart, C. C., Using inductive learning to determine fuzzy rules for dynamic systems. Engng Applic. Artif. lntell., 1993, 6(3), 257-264.

9. Quinlan, J. R., Induction of decision trees. Machine Learning, 1986, 1, 81-106.

I0. Pfeiffer, B.-M. and Isermann, R., Symbolic analysis and synthesis of linguistic parametrization in fuzzy controllers. Proceedings of the 3rd IEEE Conf. on Fuzzy Systems, IEEE World Congress on Computa- tional Intelligence, Vol. 1, 1994, pp. 555-560.

11. Pfeiffer, B.-M., Identification of fuzzy rules from learning data. Proceedings, IFAC Symposium, Artificial Intelligence in Real Time Control, Valencia, Spain, 1994, pp. 49-54.

12. Chan, K. C., Intelligent control techniques for feedback controller tuning applications. Ph.D. thesis, Vol. 1 and Vol. 2, University of New South Wales, Australia, 1994.

13. Chan, K. C. and Leong, S. S., Neural-virtual fuzzy modelling of non- linear systems. Proceedings, 1994 Int. Symposium on Artificial Neural Networks, ISANN-94, Taiwan, 1994, pp. 234-240.

14. Chan, K. C., Lin, G. C. I. and Leong, S. S., A more accurate adaptive fuzzy inference system. Computers in Industry, 1995, 26(1), 61-73.

15. Chart, K. C., Development of a feedback controller tuner using virtual fuzzy set. Computers in Industry, 1996, 28, 219-232.

16. Kosko, B., Neural Networks and Fuzzy Systems. Prentice Hall, Englewood Cliffs, NJ, 1992.

17. Chen, S., Billings, S. A. and Grant, P. M., Non-linear system identification using neural networks. Int. J. Control, 1990, 51(6), 1191-1214.

a comparative study of the max and sum machine-learning algorithms using virtual fuzzy sets

Documents