introduction to computational intelligence · introduction to computational intelligence, nanjing...
TRANSCRIPT
INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University Spring 2016
INTRODUCTION TOCOMPUTATIONAL INTELLIGENCE
Lin ShangDept. of Computer Science and Technology
Nanjing University
INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University Spring 2016
Fuzzy Set and Rough Set
n Introductionn History and definition
n Fuzzy Setsn Membership functionn Fuzzy set operations
n Rough Setsn Approximationn Reduction
INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University Spring 2016
“Fuzzification isakindofscientificpermisiveness; ittendstoresultinsociallyappealingslogansunaccompaniedbythedisciplineofhardwork.”
R.E.Kalman,1972
INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University Spring 2016
Set
Fuzzy Set
Rough Set
(collections) of various objects of interest
“Number of things of the same kind, that belong together because they are similar or complementary to each other.”The Oxford English Dictionary
Set Theory: George Cantor (1893)
an element can belong to a set to a degree k (0 ≤ k ≤ 1)
completely new, elegant approach to vagueness
Fuzzy Set theory: Lotfi Zadeh(1965)
imprecision is expressed by a boundary region of a set
another approach to vagueness
Rough Set Theory: Zdzisaw Pawlak(1982)
Introduction
INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University Spring 2016
Introduction
n Early computer science• Not good at solving real problems• The computer was unable to make accurate inferences• Could not tell what would happen, give some
preconditions• Computer always seemed to need more information
INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University Spring 2016
Lotfi Zadeh
n “Fuzzy Sets” paper published in 1965n Comprehensive - contains everything needed to implement
FLn Key concept is that of membership values:extent to which an object meets vague or imprecise propertiesn Membership function: membership values over domain of
interestn Fuzzy set operationsn Awarded the IEEE Medal of Honor in 1995
INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University Spring 2016
History for fuzzy sets and system
n First fuzzy control system, work done in 1973 with Assilian(1975)
n Developed for boiler-engine steam plant
n 24 fuzzy rules
n Developed in a few days
n Laboratory-based
n Served as proof-of-concept
INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University Spring 2016
Early European Researchers
Hans Zimmerman, Univ. of Aachen•Founded first European FL working group in 1975•First Editor of Fuzzy Sets and Systems•First President of Int’l. Fuzzy Systems Association
Didier Dubois and Henri Prade in France•Charter members of European working group•Developed families of operators•Co-authored a textbook (1980)
INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University Spring 2016
Early U. S. Researchers
K. S. Fu (Purdue) and Azriel Rosenfeld (U. Md.) (1965-75)
Enrique Ruspini at SRI•Theoretical FL foundations•Developed fuzzy clustering
James Bezdek, Univ. of West Florida•Developed fuzzy pattern recognition algorithms•Proved fuzzy c-means clustering algorithm•Combined fuzzy logic and neural networks•Chaired 1st Fuzz/IEEE Conf. in 1992 and others•President of IEEE NNC 1997-1999
INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University Spring 2016
The Dark Age
•Lasted most of 1980s
•Funding dried up, in US especially
“...Fuzzy logic is based on fuzzy thinking. It fails to distinguish between the issues specifically addressed by the traditional methods of logic, definition and statistical decision-making...”
- J. Konieki (1991) in AI Expert
•Symbolics ruled: “fuzzy” label amounted to the ‘kiss of death’
INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University Spring 2016
Michio Sugeno
•Secretary of Terano’s FL working group, est. in 1972
•1974 Ph.D. dissertation: fuzzy measures theory
•Worked in UK
•First commercial application of FL in Japan: control system for water purification plant (1983)
INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University Spring 2016
Other Japanese Developementsn 1st consumer product: shower head using FL circuitry to
control temperature (1987)
n Fuzzy control system for Sendai subway (1987)
n 2d annual IFSA conference in Tokyo was turning point for FL (1987)
n Laboratory for Int’l. Fuzzy Engineering Research (LIFE) founded in Yokohama with Terano as Director, Sugeno as Leading Advisor in 1989.
INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University Spring 2016
Fuzzy Systems Theory and Paradigms
n Variation on 2-valued logic that makes analysis and control of real (non-linear) systems possible
n Crisp “first order” logic is insufficient for many applications because almost all human reasoning is imprecise
n fuzzy sets, approximate reasoning, and fuzzy logic issues and applications
INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University Spring 2016
Fuzziness is not probability
• Probability is used, for example, in weather forecasting
• Probability is a number between 0 and 1 that is the
certainty that anevent will occur
• The event occurrence is usually 0 or 1 in crisp logic, but
fuzziness says that it happens to some degree
• Fuzziness is more than probability; probability is a subset
of fuzziness
• Probability is only valid for future/unknown events
• Fuzzy set membership continues after the event
INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University Spring 2016
Probability
• Probability is based on a closed world model in which it isassumed that everything is known
• Probability is based on frequency; Bayesian on subjectivity
• Probability requires independence of variables
• In probability, absence of a fact implies knowledge
INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University Spring 2016
Fuzzy Set Membership
•In fuzzy logic, set membership occurs by degree•Set membership values are between 0 and 1•We can now reason by degree, and apply logical operations to fuzzy sets
We usually write
or, the membership value of x in the fuzzy set A is m, where
mxA =)(µ
10 ≤≤ m
INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University Spring 2016
Fuzzy Set Membership Functions
• Fuzzy sets have “shapes”: the membership values plotted versusthe variable
• Fuzzy membership function: the shape of the fuzzy set over therange of the numeric variableCan be any shape, including arbitrary or irregularIs normalized to values between 0 and 1Often uses triangular approximations to save computation time
INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University Spring 2016
Fuzzy Sets Are Membership Functions
fromBezdek
INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University Spring 2016
Representations of Membership Functions
⎭⎬⎫
⎩⎨⎧ ++++=
⎭⎬⎫
⎩⎨⎧ ++=
900
805.
701
605.
500
15.21
95.150.
75.10
Warm
TAMPBP
( ) ( ) 50/80_
2−−= pPRICEFAIR epµ
INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University Spring 2016
Two Types of Fuzzy Membership Function
INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University Spring 2016
Equality of Fuzzy Sets
• In traditional logic, sets containing the same members are equal:{A,B,C} = {A,B,C}
• In fuzzy logic, however, two sets are equal if and only if allelements have identical membership values:
{.1/A,.6/B,.8C} = {.1/A,.6/B,.8/C}
INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University Spring 2016
Fuzzy Union
• In traditional logic, all elements in either (or both) set(s)are included
• In fuzzy logic, union is the maximum set membership value
( ) ( ) ( )If m x and m x then m xA B A B= = =∪07 09 09. . .
INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University Spring 2016
Fuzzy Relations and Operators
INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University Spring 2016
Summary: FUZZY SETSMembership function and Fuzzy set operations
INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University Spring 2016
Tom is rather tall, but Judy is short.
If you are tall, than you are quite likely heavy.
Examples on fuzzy concepts
INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University Spring 2016
• The description of a human characteristic such as
healthy.
• The classification of patients as depressed.
• The classification of certain objects as large.
• The classification of people by age such as old.
• A rule for driving such as “if an obstacle is close, then
brake immediately”.
Examples on fuzzy concepts
INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University Spring 2016
Concept and setintension (内涵):attributes of the object
concept
extension (外延):all of the objects defined by
the concept(set)
G. Cantor (1887)
{ | ( )}A a P a=
INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University Spring 2016
If a fuzzy concept can be rigidly described by Cantor’s
notion of sets or the bivalent (true/false or two-valued)…
INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University Spring 2016
模糊概念能否用Cantor集合来刻画?秃头悖论一位已经谢顶的老教授与他的学生争论他是否为秃头问题。
教授:我是秃头吗?
学生:您的头顶上已经没有多少头发,确实应该说是。
教授:你秀发稠密,绝对不算秃头,问你,如果你头上脱落了一根头发之后,
能说变成了秃头了吗?
学生:我减少一根头发之后,当然不会变成秃头。
教授:好了,总结我们的讨论,得出下面的命题:‘如果一个人不是秃头,那么他减少一根头发仍不是秃头’,你说对吗?
学生:对!
教授:我年轻时代也和你一样一头秀发,当时没有人说我秃头,后来随着年龄
的增高,头发一根根减少到今天的样子。但每掉一根头发,根据我们刚
才的命题,我都不应该称为秃头,这样经有限次头发的减少,用这一命
题有限次,结论是:‘我今天仍不是秃头’。
INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University Spring 2016
Postulate: If a man with n (a nature number) hairs is
baldheaded, then so is a man with n+1 hairs.
Baldhead Paradox:Every man is baldheaded.
Cause: due to the use of bivalent logic for inference, whereas
in fact, bivalent logic does not apply in this case。
r
INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University Spring 2016
Fuzzy Sets:Membership Functions
fromBezdek
Fuzzy membership function: the shape of the fuzzy set over the range of the numeric variable
> Can be any shape, including arbitrary or irregular
> Is normalized to values between 0 and 1> Often uses triangular approximations to save
computation time
INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University Spring 2016
Crisp sets VS Fuzzy Sets
C={Lineslongerthan4cm} C={Longlines}
INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University Spring 2016
For contiguous data:
C={MENOLDERTHAN50YEARSOLD} C={OLDMEN}
INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University Spring 2016
Example of Fuzzification
Assume inside temperature is 67.5 F, change in temperature last five minutes is -1.6 F, and outdoor temperature is 52 F.
Now find fuzzy values needed for our four example rules:
For InTemp,
0.0)5.67( and ,75.0)5.67(,25.0)5.67( _ === warmtooecomfortablcool µµµ
.For DeltaInTemp,
0.0)6.1( and ,2.0)6.1( ,8.0)6.1( _arg__ =−=−=− positiveelzeronearnegativesmall µµµ
For OutTemp, 9.0)52( =chillyµ
INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University Spring 2016
Fuzzy …… Crisp
•Fuzzy logic comprises fuzzy sets and approximate reasoning
• A fuzzy “fact” is any assertion or piece of information, and can have a “degree of truth”, usually a value between 0 and 1
• Fuzziness: “A type of imprecision which is associated with ... Classes in which there is no sharp transition from membership to non-membership” - Zadeh (1970)
INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University Spring 2016
Fuzziness ……probability• Probability is used, for example, in weather forecasting• Probability is a number between 0 and 1 that is thecertainty that an event will occur• The event occurrence is usually 0 or 1 in crisp logic, but fuzziness says that it happens to some degree• Fuzziness is more than probability; probability is a subset of fuzziness• Probability is only valid for future/unknown events• Fuzzy set membership continues after the event
INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University Spring 2016
Fuzzy relations and operations Realtions:EqualityandContainment
INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University Spring 2016
Equality of Fuzzy Sets
* In traditional logic, sets containing the same members are equal:{A,B,C} = {A,B,C}
* In fuzzy logic, however, two sets are equal if and only if allelements have identical membership values:
{.1/A,.6/B,.8C} = {.1/A,.6/B,.8/C}
INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University Spring 2016
Fuzzy Containment
• In traditional logic, A B⊂if and only if all elements in A are also in B.
• In fuzzy logic, containment means that the membership valuesfor each element in a subset is less than or equal to themembership value of the corresponding element in thesuperset.
• Adding a hedge can create a subset or superset.
INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University Spring 2016
Fuzzy Intersection
* In standard logic, the intersection of two sets contains those elements in both sets.
* In fuzzy logic, the weakest element determines the degreeof membership in the intersection
( ) ( ) ( )If m x and m x then m xA B A B= = ≡∩05 03 03. . .
INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University Spring 2016
Fuzzy Union
• In traditional logic, all elements in either (or both) set(s)are included
* In fuzzy logic, union is the maximum set membership value
( ) ( ) ( )If m x and m x then m xA B A B= = =∪07 09 09. . .
INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University Spring 2016
Fuzzy Complement
• Intraditionallogic,thecomplementofasetisallofthe elementsnot intheset.
• Infuzzylogic,thevalueofthecomplementofamembership is(1- membership_value)
INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University Spring 2016
Examples:Intersection,union, complement
INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University Spring 2016
U={u1,u2,u3,u4,u5}A=0.2/u1+0.7/u2+1/u3+0.5/u5B=0.5/u1+0.3/u2+0.1/u4+0.7/u5
A…?...B=
B=
INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University Spring 2016
Rough SetsRough Sets: Background• vagueness
• boundary region approach(Gottlob Frege )
• existing of objects which cannot be uniquely classified to the set or its complement
• another approach to vagueness
•imprecision in the approach is expressed by a boundary region of a set
• defined quite generally by means of topological operations, interior and closure, called approximations
lowerapproximationupperapproximation
INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University Spring 2016
• Human knowledge about a domain is expressed by classification• Rough set theory treats knowledge as an ability to classifyperceived objects into categories• Objects belonging to the same category are considered to be indistinguishable to each other. • The primary notions of rough set theory are the approximation space: lower and upper approximations of an object set• The lower approximation of an object set (S) is a set of objects surely belonging to S, while its upper approximation is a set of objects surely or possibly belonging to it • An object set defined through its lower and upper approximations is called a rough set
Rough Sets: Introduction
INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University Spring 2016
Introduction
•Research on rough set theory and applications in China began in the middle 1990s.
•Chinese researchers achieved many significant results on rough set theory and applications.
•both the quality and quantity of Chinese research papers are growing very quickly
•many topics being investigated by Chinese researchers: fundamental of rough set, knowledge acquisition, granular computing based on rough set,extended rough set models, rough logic, applications of rough set, et al.
INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University Spring 2016
Basic Concepts
• Knowledge
• Indiscernibility Relation
• lower and upper approximations
1. preliminary
2. secondary
• Reduct
• Indiscernibility Matrix
• Attributes Significance
INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University Spring 2016
Basic Concepts
PART I: preliminary
knowledge
approximatespace:
K=(U,R)
INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University Spring 2016
Basic ConceptsPART I: preliminary
patients
Patient Headache Muscle-pain Temperature Flu
p1 yes yes normal nop2 yes yes high yesp3 yes yes very high yesp4 no yes normal nop5 no no high nop6 no yes very high yes
IS(Information System/Tables)
Attributes Decision Attribute
Condition Attribute
INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University Spring 2016
Basic concepts of rough set theory :• lower approximation of a set X with respect to R :is the set of all objects, which can be for certain classified as X with respect to R (are certainly X with respect to R).• upper approximation of a set X with respect to R:is the set of all objects which can be possibly classified as Xwith respect to R (are possibly X in view of R).• boundary region of a set X with respect to R :is the set of all objects, which can be classified neither as Xnor as not-X with respect to R.
Basic ConceptsPART I: preliminary
INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University Spring 2016
X1 = {u | Flu(u) = yes}
= {u2, u3, u6, u7}RX1 = {u2, u3}
= {u2, u3, u6, u7, u8, u5}
X2 = {u | Flu(u) = no}
= {u1, u4, u5, u8}
RX2 = {u1, u4}= {u1, u4, u5, u8, u7, u6}X1R X2R
U Headache Temp. Flu U1 Yes Normal No U2 Yes High Yes U3 Yes Very-high Yes U4 No Normal No U5 NNNooo HHHiiiggghhh NNNooo U6 No Very-high Yes U7 NNNooo HHHiiiggghhh YYYeeesss U8 No Very-high No
The indiscernibility classes defined by R = {Headache, Temp.} are {u1}, {u2}, {u3}, {u4}, {u5, u7}, {u6, u8}.
Basic ConceptsPART I: preliminary
INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University Spring 2016
RX1 = {u2, u3}= {u2, u3, u6, u7, u8, u5}
Lower & Upper Approximations (4)
R = {Headache, Temp.}U/R = { {u1}, {u2}, {u3}, {u4}, {u5, u7}, {u6, u8}}
X1 = {u | Flu(u) = yes} = {u2,u3,u6,u7}X2 = {u | Flu(u) = no} = {u1,u4,u5,u8}
RX2 = {u1, u4}
= {u1, u4, u5, u8, u7, u6}
X1R
X2R
u1
u4u3
X1 X2
u5u7u2
u6 u8
Basic ConceptsPART I: preliminary
INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University Spring 2016
| ( ) |( )| ( ) |BB XXB X
α −−
=• accuracy of approximation:
Basic ConceptsPART I: preliminary
where |X| denotes the cardinality of
Obviously
If X is crisp with respect to B.
If X is rough with respect to B.
.φ≠X.10 ≤≤ Bα
,1)( =XBα,1)( <XBα
INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University Spring 2016
Basic ConceptsPART II: secondary
B AÃ
( ) ( )IND B IND A=
is a reduct of information system if
and no proper subset of B has this property
ReductPatient Headache Muscle-pain Temperature Flu
p1 yes yes normal nop2 yes yes high yesp3 yes yes very high yesp4 no yes normal nop5 no no high nop6 no yes very high yes
Reducts: {Headache, Temperature}
or {Muscle-pain, Temperature}
Core: CORE(P)=∩RED(P)
INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University Spring 2016
Attributes Reduct
,if ,then S is the Reduct of D。
其中, , X∈U/DS P⊂ S PPOS (D)=POS (D)
PPOS (D) P_(X)= U
BasicConceptsPART II: secondary
Patient Headache Muscle-pain Temperature Flup1 yes yes normal nop2 yes yes high yesp3 yes yes very high yesp4 no yes normal nop5 no no high nop6 no yes very high yes
Positive region
POS{M,T}={p1,p2,p3,p4,p5,p6}
INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University Spring 2016
Patient
Headache Temperature Flu
p1 no high yesp2 yes high yesp3 yes very high yesp4 no normal nop5 yes high nop6 no very high yes
Patient Headache Muscle-pain Temperature Flu
p1 yes yes normal nop2 yes yes high yesp3 yes yes very high yesp4 no yes normal nop5 no no high nop6 no yes very high yes
Patient
Muscle-pain
Temperature
Flu
p1 yes high yesp2 no high yesp3 yes very high yesp4 yes normal nop5 no high nop6 yes very high yes
Basic ConceptsPART II: secondary
INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University Spring 2016
A.Skowron:Indiscernibility Matrix
Basic ConceptsPART II: secondary
p1 p2 p3 p4 p5 p6p1 % T T H T H,T
p2 %p3 % H,T H,M,T %
p4 % % T
p5 % M,T
p6 %
M(S)=[cij]n×n,cij={a∈A:a(xi)≠a(xj),i,j=1,2,…,n}
Patient Headache Muscle-pain Temperature Flup1 yes yes normal nop2 yes yes high yesp3 yes yes very high yesp4 no yes normal nop5 no no high nop6 no yes very high yes
INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University Spring 2016
BasicConceptsPART II: secondary
Patient Headache Muscle-pain Temperature Flup1 yes yes normal nop2 yes yes high yesp3 yes yes very high yesp4 no yes normal nop5 no no high nop6 no yes very high yes
headache muscle-pain temperature flu
Which is more important?
(C,D)
( γ (C,D)-γ (C- {a} ,D) ) γ (C- {a} ,D)σ (a) = =1-
γ (C,D) γ (C,D)
Definition: σ (Headache) = 0,
σ (Muscle-pain) = 0,
σ (Temperature) = 0.75
INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University Spring 2016
Theory and Applications
Theory in the view of algebra
in the view of information theory
in the view of logic
Applications
medical data analysis
finance
voice recognition
image processing
…
INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University Spring 2016
• Good at…
discrete values
uncertainty
Advantages and Disadvantages
Disadvantages:
discrete valuessensitive
INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University Spring 2016
• Models
• Data
• Algorithms
• Application
Trends and Challenges