witold pedrycz department of electrical & computer engineering university of alberta, edmonton,...
TRANSCRIPT
Witold PedryczDepartment of Electrical & Computer Engineering University of Alberta, Edmonton, CanadaandSystems Research Institute, Polish Academy of SciencesWarsaw, Poland
Agenda
Introduction: human-centricity of intelligent systems and information granules
Conceptualization and realization of information granules
Information content and its characterization
Context-based information granules
Successive refinements of information granules
Information granules –based architectures
Conclusions
Human-system interactionand system modeling
Perception and processing processing realized at certain level of abstraction
Acceptance of granular (non-numeric) data
Effective two-way communication with the user at the level of information granules
Adjustment of level of detail (abstraction) dependent upon the needs of the individual user (personalization); avoidance of unnecessary details and focus on essentials
Clustering and fuzzy clustering
Discovery of structures and relationships in data
Data analysis
Construction of fuzzy sets
Clustering in fuzzy modeling and modeling…
Information granules: from conceptualization to realization
Implicit information granulesImplicit information granulesExplicit (operational)information granules
Humans Computer realizations
Various points of view (models)Various points of view (models) Fuzzy setsRough setsIntervals (sets)Shadowed setsProbability functions
Information granulesInformation granules
Development of information granules
Usage of all available experimental evidence (numeric and non-numeric;knowledge hints)
Information granules capturing existing domain knowledge (especiallyguidance –knowledge hints provided by the designer/user)
Results of information granulation dependent upon the underlying formalism of Granular Computing
Information granules are context-dependent; the ensuing design framework should incorporate this aspect in an explicit way
Information granules: from conceptualization to realization
Data Information granules Construction of Intelligent systems
auxiliary guidance mechanisms
Content of information granules
Formalisms of sets, fuzzy sets, rough sets
Clustering and fuzzy clustering
Clustering 2,670,000
Fuzzy clustering 443,000
Rough clustering 268,000
Information granulation/granules 158,000
Granular Computing 104,000
Google Scholar, October 29, 2014
Objective function-based clustering
{x1, x2,…, xN}
Objective function Minimize w.r.t. structure
information granules G1, G2 , …,Gc Prototypes, medoidsv1, v2,…, vc
Partition matrix U
Fuzzy C-Means (FCM) as an exampleof fuzzy clustering
vi – prototypes
U- partition matrix
FCM – representation of information granules (granules)
Partition matrix U
prototypes v1, v2, …, vc
c 1,2,...,i ,Nu0N
1kik
Fuzzy Clustering: Fuzzy C-Means (FCM)
Given data x1, x2, …, xN, determine its structure byforming a collection of information granules – fuzzy sets
Objective function
2ik
N
1k
mik
c
1i||||uQ vx
Minimize Q; structure in data (partition matrix and prototypes)
FCM – flow of optimization
2ik
N
1k
mik
c
1i||||uQ vx
Minimize
subject to
(a) prototypes
(b) partition matrix
Construction of information granules: Fuzzy C-Means (FCM)
Data {x1, x2, …, xN} xk in Rn.
Performance index (objective function)
Construct information granules (clusters) - fuzzy sets A1, A2, …, Ac organized aspartition matrix U
Partition matrix
Prototype vi
Quality of clustering
Cluster validity indexes
Cluster content
Granulation-degranulation: reconstruction criterion
Information content of clusters
Description of information content:
•Variability of data
•Classification content
•Variability with regard to auxiliary (output) data
Variability of data
Description of data residing within a given ith cluster
Variability of data around the prototype vi
Variability in terms of membership grades of data
Classification content of clusters
Applied to classification problems.
Dominant class present in ith cluster
Classification content:
count index
cumulative membership grades of classes
Granular mapping: an architecture
Aggregation of contents of information granules and their activation levels
Auxiliary variable content
Problems in which occur some additional variables (say output variable, y) whose values determine the content of the cluster.
Clustering realized for data in the multivariable input space
Information granulation and degranulation: reconstruction criterion
v1 v2 vc
granulation
u1, u2, …, uc
v1 v2 vc
degranulation
Results of degranulation made more abstract (in the form of information granules):granular clustering
Key challenges of clustering
Selection of distance function (geometry of clusters)
Number of clusters
Quality of clustering results
Landscape of clustering
Graph-oriented and hierarchical (single linkage, complete linkage, average linkage..)
Objective function-based clustering
Variety of formalisms and optimization tools(e.g., methods of Evolutionary Computing)
Diversity
CommonalityData-driven methods
The dichotomy and a paradigm shift
Human-centricityGuidance mechanisms
Knowledge –based clustering
dataknowledge
Partial supervision
Context-based guidance
Proximity –based
Viewpoints
Domain Knowledge:categories of knowledge-oriented
guidance
Context-based guidance: clustering realized in a certain contextspecified with regard to some attribute
Viewpoints: some structural information is provided
Partially labeled data: some data are provided with labels (classes)
Proximity knowledge: some pairs of data are quantified interms of their proximity (resemblance, closeness)
Context-based clustering
Clustering : construct clusters in input space X
Context-based Clustering : construct clusters in input space X given some context expressed in output space Y
Active role of the designer [customization of processing]
Context-based clustering:Conmputational considerations
•computationally more efficient,•well-focused, •designer-guided clustering process
Data
structure
Data
structure
context
Context-based clustering:focus mechanism
Determine structure in input space given the output is high
Determine structure in input space given the output is medium
Determine structure in input space given the output is low
Input space (data)
Context-based clustering:examples
Find a structure of customer data [clustering]
Find a structure of customer data considering customers making weekly purchases in the range [$1,000 $3,000]
Find a structure of customer data considering customers making weekly purchases at the level of
around $ 2,500
Find a structure of customer data considering customers making significant weekly purchases who
are young
no context
context
context
context(compound)
Context-based Fuzzy C-Means
data(xk, yk), k=1,2,…,N
contexts: fuzzy sets W1, W2, …, Wp defined in the output space
wjk = Wj(yk)
c
1i
N
1kikjkikikj iNu0andk wu|0,1u)(WU
Context-drivenpartition matrix
Context-based clustering:the use of context
xk
Context Wj
yk
Wj(yk)
xkContext-based fuzzy clustering
Context-oriented FCM:Optimization flow
Objective function
Iterative adjustment of partition matrix and prototypes
2ik
c
1i
N
1k
mik ||||uQ vx
c
1j
1m
2
jk
ik
jkik
wu
vx
vx
N
1k
mik
N
1kk
mik
i
u
u xv
Subject to constraint U in U(Wj)
Successive refinements of information granules
Information granules constructed in a successive manner forming a hierarchy of refined constructs of higher specificity
The refinement applied to information granules based on their information content
Successive usage of context-based fuzzy clustering
Successive refinements of information granules
information granuleto be refined
membership function Ai [1] used as a contextrefinement process
membership function Aj [2] used as a context
Expansion formulas:Context-based FCM
information granuleto be refined
membership function Ai [1] used as contextrefinement process
property of fuzzy partition
membership function Aj [2] used as context
Expansion formulas:Context-based FCM
information granuleto be refined
membership function Ai [1] used as context
membership function Aj [2] used as context
Successive fuzzy partitions
Fuzzy clustering with viewpoints
Viewpoints: definitionDescription of entity (concept) which is deemed essential in describing phenomenon (system) and helpful in castingan overall analysis in a required setting
“external” , “reinforced” clusters
Viewpoints: examples
-150
-100
-50
0
50
100
150
200
0 100 200 300 400 500
x1
x2
a
b
x1
x2
a
viewpoint (a,b) viewpoint (a,?)
Viewpoints in fuzzy clustering
x1
x2
a
b
otherwise 0,
viewpointby the determined is B of rowth -i theof featureth -j theif 1,b ij
0
0
1
0
0
1
B
0
0
b
0
0
a
F
B- Boolean matrix characterizing structure: viewpoints prototypes (induced by data)
Viewpoints in fuzzy clustering
Q = 2ijkj
n
1:bji,1j
mik
c
1i
N
1k
2ijkj
n
0:bji,1j
mik
c
1i
N
1k
)f(xu)v(xu
ijij
1b if f
0bif vg
ijij
ijijij
2ijkj
n
1j
mik
c
1i
N
1k
)g(xuQ
Fuzzy clustering with proximity guidelines
Proximity hints
Characterization in terms of proximity degrees:
Prox(k, l), k, l=1,2, …., N
and supervision indicator matrix B = [bkl], k, l=1,2,…, N
Prox(k,l)
Prox(s,t)
Proximity measureProperties of proximity:
(a)Prox(k, k) =1
(b)Prox(k,l) = Prox(l,k)
Proximity induced by partition matrix U
Linkages with kernel functions K(xk, xl)
Augmented objective function
> 0
Granular fuzzy clustering
Granular prototypes and reconstruction criterion
prototypes granular prototypes
Selection/construction of prototypes
Forming granular prototypes to capture existing structural variability and satisfyingdegranulation criterion
(a) information granules of prototypes built around prototypes(b) optimization of allocation of granularity by minimizing reconstructioncriterion
Formation of granular (interval) membership grades – details
xVi
ui-(x)=min(w1(x), w2(x))
ui+(x)=max(w1(x), w2(x))
Overview
Information granules Blueprint of model
content
Model development,refinements, augmentations
Conclusions
Fuzzy clustering as a conceptual and algorithmic backbone ofdesign of information granules
Human-centric (knowledge-oriented) design of information granules
Emergence of higher type granular constructs
Needs for further advancements in optimization frameworks of fuzzy clustering