fuzzy interpretation of discretized intervals dr. xindong wu andrea porter april 11, 2002
Post on 21-Dec-2015
219 views
TRANSCRIPT
Fuzzy Interpretation of Discretized Intervals
Dr. Xindong Wu
Andrea Porter
April 11, 2002
Plan For Presentation
Introduction to Problem, HCVDiscretization Techniques/Fuzzy
BordersA Hybrid Solution for HCVExperiments and ResultsConclusion
Introduction
Real-world data contains both numerical and nominal data, must be able to deal with different types of data.
Existing systems discretize numerical domains into intervals and treat intervals as nominal values during induction.
Problems occur if test examples are not covered in training data (no-match, multiple match)
The solution is a hybrid approach using fuzzy intervals for no-match problem.
HCV
Attribute based rule induction algorithm, extension matrix approach Divide positive examples into intersecting groups Find a heuristic conjunctive rule in each group that
covers all PE and no NE
HCV can find a rule in the form of variable-valued logic
More compact than the decision trees/rules of ID3 and C4.5
Variable Valued Logic and Selectors
Represents decisions where variables can take a range
Selector:
[ X # R ]
X = attribute
# = relational operator ( = , <, >, . . . )
R = Reference, list of 1 or more values
e.g [ Windy = true] [Temp > 90]
, , ,...
HCV Software
C++ implementationCan work with noisy and real-valued
domains as well as nominal and noise-free databases
Provides a set of deduction facilities for the user to test the accuracy of the produced rules on test examples
Example DB ORDER X1 X2 X3 X4 CLASS
1 1 a a 1 F
2 1 a b 1 F
3 1 a c 1 F
4 1 a a 0 F
5 1 b c 1 T
6 0 b b 0 T
7 0 a c 1 T
8 1 b a 0 T
9 1 b a 1 T
10 1 c c 0 F
11 1 c b 1 F
12 0 c b 0 T
13 0 a a 0 T
14 0 c c 1 F
15 0 c a 0 T
16 1 a b 0 F
17 0 a a 1 T
18 0 b a 1 T
C4.5 Results vs. HCV
C4.5:The T classX2 = b
X1 = 0 & X3 = a X1 = 0 & X3 = b X1 = 0 & X2 = a
HCV:The T classX2 = b
X1 = 0 & X2 = a
X1 = 0 & X4 = 0
C4.5:The F classX1 = 1 & X2 = a
X1 = 1 & X2 = c X2 = c & X3 = c
Deduction of Induction Results
Induction generates knowledge from existing data Deduction applies induction results to interpret
new data. With real-world data, induction can not be
assumed to be perfect Three cases:
1) no-match (measure of fit)2) single-match3) multiple-match (estimate of probability)
Discretization
Occurs during rule induction Discretize numerical domains into intervals and
treat similar to nominal values. The challenge is to find the right borders for the
intervals Possible Methods:
1) Simplest Class-Separating Method
2) Information Gain Heuristic (implemented in HCV)
Simplest Class- Separating Method:
Interval Borders are places between each adjacent pair of examples which have different classes.
If attribute is very informative - method is efficient and useful.
If attribute is not informative - method produces too many intervals
Information Gain Heuristic
Use IGH to find more informative border.
x = (xi + xi+1)/2 for (i = 1, …, n-1) x is a possible cut point if xi and xi+1 are of different
classes. Use IGH to find best x Recursively split on left and right To stop recursive splitting:
1) stop if IGH is same on all possible cut points.2) stop if # of examples to split is less than a predefined number3) limit the number of intervals
Fuzzy Borders
Discretization of continuous domains does not always fit accurate interpretation.
Instead of using sharp borders, use a membership function, measures the degree of membership.
A value can be classified into a few different intervals at the same time (e.g. single to multiple match)
Fuzzy Borders (2)
Fuzzy matching - deduction with fuzzy borders of discretized intervals.
Take the interval with the greatest degree as the value’s discrete value.
3 functions to fuzzify borders:
1) linear2) polynomial3) arctan
Definitionss = spread parameter l = length of original
xleft, xright = left/right sharp borders
xleft xright
l
a = -kxleft + 1/2 b = kxright + 1/2
linleft(x) = kx + a
lin right(x) = -kx + b
lin(x) = MAX(0, MIN(1,linleft(x),linright(x)))
Linear Membership Function
k = 1/2sl
xleft xright
l
sl
Arctan Membership Function
*Polynomial Membership Function
polyside(x) = asidex3 + bsidex2 + csidex + dside
aside = 1/(4(ls)3)
bside = -3asidexside side {left,right}
cside = 3aside(xside2 - (ls)2)
dside = -a(xside3 -3xside(ls)2 + 2(ls)3)
polyleft(x), if xleft -ls x xleft + ls
poly(x) = polyright(x), if xright -ls x xright +ls
1, if xleft +ls x xright -ls
0, otherwise
Match Degree
Selector method - take the max membership degree of the value in all the intervals involved. If 2 adjacent intervals have the same class, values close to the border will have low membership.
Conjunction method - adds with fuzzy plus
ab=a + b - ab
No-Match Resolution
Largest Class
Assign all no match examples to the largest class, the default class.
Works well, if the number of classes in a training set is small and one class is clearly larger.
Deteriorates if there is a larger number of classes and the examples are evenly distributed
No-Match Resolution
Measure of FitCalculate the measure of fit for each class:
1) calculate MF for each selector (sel)MF(sel, e) = 1, if sel is satisfied by e
n/|x|, otherwise2) calculate MF for each conjunctive rule(conj)MF(conj, e) = MF(sel, e) * n(conj)/N
No-Match Resolution
Measure of Fit (2)3) calculate MF for each class c
MF(c, e) = MF(conj1, e) + MF(conj2, e) - MF(conj1,e)MF(conj2,e)
* For more than two rules, apply formula recursively.
* Find maximum MF - determines which class is closest to the example
Multiple-Match
Caused by over-generalization of the training examples at induction time
Example (X1 = a, X2 = 1)
All PE cover X1 = a
All NE cover X2 = 1 Multiple Match
Multiple-Match Resolution
First Hit Use first rule which classifies the example Produces reasonable results if the rules from
induction have been ordered according to a measure of reliability
Advantages - straightforward, efficient Disadvantages - have to sort rules at
induction time
Multiple-Match Resolution
Largest Rule
Similar to largest class method from no-match resolution
Choose conjunctive rule that covers the most examples in the training set.
Multiple-Match Resolution
Estimation of Probability Assign EP value to each class based on the
size of the satisfied conjunctive rules.
1) Find EP for each conjunctive rule (conj):EP(conj, e)= { n(conj)/N, if conj is satisfied
by e0, otherwise
n(conj) = number of examples covered by conjN = number of total examples
Multiple-Match Resolution
Estimation of Probability (2)
2) Find EP value for each class:
EP(c, e) = EP(conj1, e) + EP(conj2, e) - EP(conj1,e)EP(conj2,e).
* For more rules, apply formula recursively
* Choose class with highest EP value
Hybrid Interpretation
Used because fuzzy borders only add conflicts because they don’t reduce the number rules that are applicable
HCV - use sharp borders during induction and use fuzzy borders only during deduction
Algorithm:* Single match - use class indicated by rules* Multiple match - use estimation probability
(EP) with sharp borders* No match - use fuzzy borders with polynomial
membership function to find closest rule
The Data
Used 17 databases from the Machine Learning Database Repository, U. of California, Irvine.
Databases selected because:
1) All include numerical data
2) All lead to situations where no rules clearly apply.
Results – Predictive Accuracy
Results (cont.)
The results shown for C4.5 and NewID are the pruned ones These were usually better than the
unpruned ones in this experimentHCV did not fine tune different
parameters because this would be loss of generality and applicability of the conclusions
Accuracy Results
HCV(hybrid) - 9 databasesC4.5 (R 8) - 7 databasesC4.5 (R 5) - 6 databasesHVC (large) - 3 databasesHCV (fuzzy) - 2 databases
HCV Comparison
HCV (fuzzy) generally performs better than the simple largest class method
HCV’s performance improves significantly when the fuzzy borders (for no match) are combined with probability estimation (for multiple match) in HCV (hybrid)
Conclusions
Fuzzy borders are constructed and used at deduction time only when a no match case occurs.
This hybrid method performs more accurately than several other current deduction programs.
Fuzziness is strongly domain dependent, HCV allows the user to specify their own intervals and fuzzy functions.