2011 - current research
Post on 28-Nov-2014
596 Views
Preview:
DESCRIPTION
TRANSCRIPT
Outline MD Classification MDSSL Application to SA Conclusions Current Work
Semi-supervised Learning of Multi-dimensionalClass Bayesian Network Classifiers
Jonathan Ortigosa-Hernandez
January 28th, 2011
Jonathan Ortigosa-Hernandez
Semi-supervised Learning of Multi-dimensional Class Bayesian Network Classifiers
Outline MD Classification MDSSL Application to SA Conclusions Current Work
Multi-dimensional Supervised Classification
Multi-dimensional Semi-supervised Learning
Application to Sentiment Analysis
Conclusions
Current Topics of Research
Jonathan Ortigosa-Hernandez
Semi-supervised Learning of Multi-dimensional Class Bayesian Network Classifiers
Outline MD Classification MDSSL Application to SA Conclusions Current Work
Supervised Classification
I It consists of building a classifier Ψ from a given labelledtraining dataset D, by using an induction algorithm A(A(D) = Ψ),
X1 X2 ... Xn C
x(1)1 x
(1)2 ... x
(1)n c (1)
x(2)1 x
(2)2 ... x
(2)n c (2)
... ... ... ... ...
x(N)1 x
(N)2 ... x
(N)n c (N)
I in order to predict the value of a class variable C for any newunlabelled instance x (Ψ(x) = c).
Jonathan Ortigosa-Hernandez
Semi-supervised Learning of Multi-dimensional Class Bayesian Network Classifiers
Outline MD Classification MDSSL Application to SA Conclusions Current Work
Uni-dimensional and Multi-dimensional Classification
I Uni-dimensional classification tries to predict a single classvariable based on a dataset composed of a set of labelledexamples.
I (Uni-dimensional Class) Bayesian Network Classifiers(Larranaga et al, 2005).
I Multi-dimensional classification is the generalisation of thesingle-class classification task to the simultaneous predictionof a set of class variables.
I Multi-dimensional Class Bayesian Network Classifiers (v.d.Gaag and d. Waal, 2006).
I Do not confuse with multi-class and multi-label classification.
Jonathan Ortigosa-Hernandez
Semi-supervised Learning of Multi-dimensional Class Bayesian Network Classifiers
Outline MD Classification MDSSL Application to SA Conclusions Current Work
Multi-dimensional Supervised Learning
A typical supervised training dataset
X1 X2 ... Xn C1 C2 ... Cm
x(1)1 x
(1)2 ... x
(1)n c
(1)1 c
(1)2 ... c
(1)m
x(2)1 x
(2)2 ... x
(2)n c
(2)1 c
(2)2 ... c
(2)m
... ... ... ... ... ... ... ...
x(N)1 x
(N)2 ... x
(N)n c
(N)1 c
(N)2 ... c
(N)m
Each instance of the dataset contains both the values of theattributes and m labels which characterise the attributes.
Jonathan Ortigosa-Hernandez
Semi-supervised Learning of Multi-dimensional Class Bayesian Network Classifiers
Outline MD Classification MDSSL Application to SA Conclusions Current Work
Bayesian Network Classifiers
X1 X2 X3 X4 X5 X6
C
Figure: A (uni-dimensional) naive Bayes structure.
Jonathan Ortigosa-Hernandez
Semi-supervised Learning of Multi-dimensional Class Bayesian Network Classifiers
Outline MD Classification MDSSL Application to SA Conclusions Current Work
Multi-dimensional Class Bayesian Network Classifiers(MDBNC)
X1 X2 X3 X4 X5 X6
C1 C2 C3
Figure: A multi-dimensional naive Bayes structure.
Jonathan Ortigosa-Hernandez
Semi-supervised Learning of Multi-dimensional Class Bayesian Network Classifiers
Outline MD Classification MDSSL Application to SA Conclusions Current Work
MDBNC Structure
X1 X2 X3 X4 X5
C1 C2 C3
X1 X2 X3 X4 X5
C1 C2 C3
X1 X2 X3 X4 X5
(a) Complete graph
(c) Class subgraph
(b) Feature selection subgraph
(d) Feature subgraph
C1 C2 C3
Figure: A MDNBC structure and its division.
Jonathan Ortigosa-Hernandez
Semi-supervised Learning of Multi-dimensional Class Bayesian Network Classifiers
Outline MD Classification MDSSL Application to SA Conclusions Current Work
Sub-families of MDBNC
(a) Multi-dimensional naive Bayes
(c) Multi-dimensional J/K dependence Bayesian (2/3)
(b) Multi-dimensional tree-augmented network
!"
!#
!$
!%
!& !'
!(
!)
*# *$ *%
!+ !",
*"
!" !# !$ !% !& !' !( !) !* !"+
," ,# ,$
!" !# !$ !% !&
'" '#
Figure: Different subfamilies of MDBNC.
Jonathan Ortigosa-Hernandez
Semi-supervised Learning of Multi-dimensional Class Bayesian Network Classifiers
Outline MD Classification MDSSL Application to SA Conclusions Current Work
Sub-families of MDBNC - MDnB
Multi-dimensional naive Bayes (MDnB)
!" !# !$ !% !& !' !( !) !* !"+
," ,# ,$
The class and featuresubgraphs are empty.
Each class variable isparent of all thefeatures.
Jonathan Ortigosa-Hernandez
Semi-supervised Learning of Multi-dimensional Class Bayesian Network Classifiers
Outline MD Classification MDSSL Application to SA Conclusions Current Work
Sub-families of MDBNC - MDnB
Multi-dimensional naive Bayes (MDnB)
!" !# !$ !% !& !' !( !) !* !"+
," ,# ,$
It has a fixed structure.
Thus, it has nostructural learning (v.d.Gaag and d. Waal,2006).
Jonathan Ortigosa-Hernandez
Semi-supervised Learning of Multi-dimensional Class Bayesian Network Classifiers
Outline MD Classification MDSSL Application to SA Conclusions Current Work
Sub-families of MDBNC - MDnB
Multi-dimensional tree-augmented network classifier (MDTAN)
!" !# !$ !% !&
'" '#
The class and featuresubgraphs are trees.
Jonathan Ortigosa-Hernandez
Semi-supervised Learning of Multi-dimensional Class Bayesian Network Classifiers
Outline MD Classification MDSSL Application to SA Conclusions Current Work
Sub-families of MDBNC - MDnB
Multi-dimensional tree-augmented network classifier (MDTAN)
!" !# !$ !% !&
'" '#
A wrapper structurallearning algorithm isproposed in (v.d. Gaagand d. Waal, 2006).
[NEW] We have recentlyproposed a filterapproach to learnMDTAN structures.
Jonathan Ortigosa-Hernandez
Semi-supervised Learning of Multi-dimensional Class Bayesian Network Classifiers
Outline MD Classification MDSSL Application to SA Conclusions Current Work
Sub-families of MDBNC - MDnB
Multi-dimensional J/K dependence Bayesian classifier (MD J/K )
!"
!#
!$
!%
!& !'
!(
!)
*# *$ *%
!+ !",
*"
The class subgraph is aJ-dependence graph.
The feature subgraph isa K -dependence graph.
Jonathan Ortigosa-Hernandez
Semi-supervised Learning of Multi-dimensional Class Bayesian Network Classifiers
Outline MD Classification MDSSL Application to SA Conclusions Current Work
Sub-families of MDBNC - MDnB
Multi-dimensional J/K dependence Bayesian classifier (MD J/K )
!"
!#
!$
!%
!& !'
!(
!)
*# *$ *%
!+ !",
*"
There was not a specificstructural learningalgorithm.
So, we proposed alearning algorithm in(Ortigosa-Hernandez etal, 2010).
Jonathan Ortigosa-Hernandez
Semi-supervised Learning of Multi-dimensional Class Bayesian Network Classifiers
Outline MD Classification MDSSL Application to SA Conclusions Current Work
A supervised method to learn a MD J/K structure(Ortigosa-Hernandez et al, 2010)
Step 0 - Initialisation
Establish the maximumnumber of parents inboth class and featuresubgraphs, i.e. J = 2and K = 2.
X1 X2 X3 X4 X5 X6 X7 X8
C1 C2 C3 C4
Jonathan Ortigosa-Hernandez
Semi-supervised Learning of Multi-dimensional Class Bayesian Network Classifiers
Outline MD Classification MDSSL Application to SA Conclusions Current Work
A supervised method to learn a MD J/K structure(Ortigosa-Hernandez et al, 2010)
Step 1 - Learn the structure between the class variables (Ac)
Calculate the mutualinformation MI (Ci ,Cj)for each pair of classvariables.
X1 X2 X3 X4 X5 X6 X7 X8
C1 C2 C3 C4
Jonathan Ortigosa-Hernandez
Semi-supervised Learning of Multi-dimensional Class Bayesian Network Classifiers
Outline MD Classification MDSSL Application to SA Conclusions Current Work
A supervised method to learn a MD J/K structure(Ortigosa-Hernandez et al, 2010)
Step 1 - Learn the structure between the class variables (Ac)
Calculate the p-values(significance of eachmutual information)using independence test.
C1 C2 C3
C4 0.36 0.57 0.01C3 0.27 0.63C2 0.06
X1 X2 X3 X4 X5 X6 X7 X8
C1 C2 C3 C4
Jonathan Ortigosa-Hernandez
Semi-supervised Learning of Multi-dimensional Class Bayesian Network Classifiers
Outline MD Classification MDSSL Application to SA Conclusions Current Work
A supervised method to learn a MD J/K structure(Ortigosa-Hernandez et al, 2010)
Step 1 - Learn the structure between the class variables (Ac)
Remove the p-valuesgreater than thethreshold α = 0.1.
C1 C2 C3
C4 0.36 0.57 0.01C3 0.27 0.63C2 0.06
X1 X2 X3 X4 X5 X6 X7 X8
C1 C2 C3 C4
Jonathan Ortigosa-Hernandez
Semi-supervised Learning of Multi-dimensional Class Bayesian Network Classifiers
Outline MD Classification MDSSL Application to SA Conclusions Current Work
A supervised method to learn a MD J/K structure(Ortigosa-Hernandez et al, 2010)
Step 1 - Learn the structure between the class variables (Ac)
From the lowest value,add arcs to the graphfulfilling the conditionsof no cycles and no morethan J-parents per classvariable.
C1 C2 C3
C4 x x 0.01C3 x xC2 0.06
X1 X2 X3 X4 X5 X6 X7 X8
C1 C2 C3 C4
Jonathan Ortigosa-Hernandez
Semi-supervised Learning of Multi-dimensional Class Bayesian Network Classifiers
Outline MD Classification MDSSL Application to SA Conclusions Current Work
A supervised method to learn a MD J/K structure(Ortigosa-Hernandez et al, 2010)
Step 1 - Learn the structure between the class variables (Ac)
From the lowest value,add arcs to the graphfulfilling the conditionsof no cycles and no morethan J-parents per classvariable.
C1 C2 C3
C4 x x xC3 x xC2 0.06
X1 X2 X3 X4 X5 X6 X7 X8
C1 C2 C3 C4
Jonathan Ortigosa-Hernandez
Semi-supervised Learning of Multi-dimensional Class Bayesian Network Classifiers
Outline MD Classification MDSSL Application to SA Conclusions Current Work
A supervised method to learn a MD J/K structure(Ortigosa-Hernandez et al, 2010)
Step 2 - Learn the structure between the class variables and thefeatures (ACF )
Calculate the mutualinformation MI (Ci ,Xj)for each pair Ci and Xj .
X1 X2 X3 X4 X5 X6 X7 X8
C1 C2 C3 C4
Jonathan Ortigosa-Hernandez
Semi-supervised Learning of Multi-dimensional Class Bayesian Network Classifiers
Outline MD Classification MDSSL Application to SA Conclusions Current Work
A supervised method to learn a MD J/K structure(Ortigosa-Hernandez et al, 2010)
Step 2 - Learn the structure between the class variables and thefeatures (ACF )
Calculate the p-value ofthe mutual informations.
C1 C2 C3 C4
X1 0.64 0.00 0.77 0.98X2 0.82 0.03 0.11 0.37X3 0.00 0.06 0.00 0.01X4 0.68 0.09 0.00 0.55X5 0.81 0.12 0.81 0.65X6 0.57 0.24 0.00 0.00X7 0.25 0.26 0.00 0.00X8 0.32 0.15 0.00 0.44
X1 X2 X3 X4 X5 X6 X7 X8
C1 C2 C3 C4
Jonathan Ortigosa-Hernandez
Semi-supervised Learning of Multi-dimensional Class Bayesian Network Classifiers
Outline MD Classification MDSSL Application to SA Conclusions Current Work
A supervised method to learn a MD J/K structure(Ortigosa-Hernandez et al, 2010)
Step 2 - Learn the structure between the class variables and thefeatures (ACF )
Remove the p-valuesgreater than α = 0.1.
C1 C2 C3 C4
X1 0.64 0.00 0.77 0.98X2 0.82 0.03 0.11 0.37X3 0.00 0.06 0.00 0.01X4 0.68 0.09 0.00 0.55X5 0.81 0.12 0.81 0.65X6 0.57 0.24 0.00 0.00X7 0.25 0.26 0.00 0.00X8 0.32 0.15 0.00 0.44
X1 X2 X3 X4 X5 X6 X7 X8
C1 C2 C3 C4
Jonathan Ortigosa-Hernandez
Semi-supervised Learning of Multi-dimensional Class Bayesian Network Classifiers
Outline MD Classification MDSSL Application to SA Conclusions Current Work
A supervised method to learn a MD J/K structure(Ortigosa-Hernandez et al, 2010)
Step 2 - Learn the structure between the class variables and thefeatures (ACF )
Add all the arcs to thestructure.
C1 C2 C3 C4
X1 x 0.00 x xX2 x 0.03 x xX3 0.00 0.06 0.00 0.01X4 x 0.09 0.00 xX5 x x x xX6 x x 0.00 0.00X7 x x 0.00 0.00X8 x x 0.00 x
X1 X2 X3 X4 X5 X6 X7 X8
C1 C2 C3 C4
Jonathan Ortigosa-Hernandez
Semi-supervised Learning of Multi-dimensional Class Bayesian Network Classifiers
Outline MD Classification MDSSL Application to SA Conclusions Current Work
A supervised method to learn a MD J/K structure(Ortigosa-Hernandez et al, 2010)
Step 3 - Learn the structure between the features(AF )
I Calculate theconditional mutualinformationMI (Xi ,Xj ||Pac(Xj)).
I Calculate thep-values.
I Remove thep-values greaterthan the thresholdα = 0.1.
X1 X2 X3 X4 X5 X6 X7 X8
C1 C2 C3 C4
Jonathan Ortigosa-Hernandez
Semi-supervised Learning of Multi-dimensional Class Bayesian Network Classifiers
Outline MD Classification MDSSL Application to SA Conclusions Current Work
A supervised method to learn a MD J/K structure(Ortigosa-Hernandez et al, 2010)
Step 3 - Learn the structure between the features (AF )
Add arcs between thefeatures fulfilling theconditions of no cyclesbetween the featuresand no more thanK -parents per feature.
X1 X2 X3 X4 X5 X6 X7 X8
C1 C2 C3 C4
Jonathan Ortigosa-Hernandez
Semi-supervised Learning of Multi-dimensional Class Bayesian Network Classifiers
Outline MD Classification MDSSL Application to SA Conclusions Current Work
A supervised method to learn a MDTAN structure(MDTANfi)
It is similar to the method to learn MD J/K structures, but treesare learnt in the AC and AF by means of a maximum spanning treealgorithm
Jonathan Ortigosa-Hernandez
Semi-supervised Learning of Multi-dimensional Class Bayesian Network Classifiers
Outline MD Classification MDSSL Application to SA Conclusions Current Work
Major Problem of Supervised Learning
I However, in many real world problems, obtaining data isrelatively easy, while labelling is difficult, expensive or laborintensive (usually done by an external mechanism, e.g. humanbeings).
I This problem is accentuated when using multiple targetvariables.
I DESIRE: Learning algorithms able to incorporate a largenumber of unlabelled data with a small number of labeleddata when learning competitive classifiers.
Jonathan Ortigosa-Hernandez
Semi-supervised Learning of Multi-dimensional Class Bayesian Network Classifiers
Outline MD Classification MDSSL Application to SA Conclusions Current Work
Multi-dimensional Semi-supervised Learning
A typical semi-supervised training dataset
X1 X2 ... Xn C1 C2 ... Cm
x(1)1 x
(1)2 ... x
(1)n c
(1)1 c
(1)2 ... c
(1)m
x(2)1 x
(2)2 ... x
(2)n c
(2)1 c
(2)2 ... c
(2)m
... ... ... ... ... ... ... ...
x(L)1 x
(L)2 ... x
(L)n c
(L)1 c
(L)2 ... c
(L)m
x(L+1)1 x
(L+1)2 ... x
(L+1)n ? ? ... ?
x(L+2)1 x
(L+2)2 ... x
(L+2)n ? ? ... ?
... ... ... ... ... ... ... ...
x(N)1 x
(N)2 ... x
(N)n ? ? ... ?
Semi-supervised Learning fulfils this desire.
Jonathan Ortigosa-Hernandez
Semi-supervised Learning of Multi-dimensional Class Bayesian Network Classifiers
Outline MD Classification MDSSL Application to SA Conclusions Current Work
The Expectation-Maximisation Algorithm
The EM algorithm (Dempster et al, 1977)
Learn an initial model.Repeat until convergence:(a) Expectation step: Using the current model, estimate themissing values of the data.(b) Maximisation step: Using the whole data and the previousestimations, learn a new current model.
Any MDBNC learning algorithm can be used as model in thisalgorithm.
Jonathan Ortigosa-Hernandez
Semi-supervised Learning of Multi-dimensional Class Bayesian Network Classifiers
Outline MD Classification MDSSL Application to SA Conclusions Current Work
Artificial Experimentation
Study the behaviour of the proposed algorithms along several axesof variability:
1. Complexity of the problem (generative structure)√
2. Number of variables (features and class variables)
3. Balance of the labels in the generative structure (values of thehyperparamenters for the Dirichlet)
4. Size of the labelled sample
5. Ratio of labelled-unlabelled data
Jonathan Ortigosa-Hernandez
Semi-supervised Learning of Multi-dimensional Class Bayesian Network Classifiers
Outline MD Classification MDSSL Application to SA Conclusions Current Work
Artificial Experimentation
A preliminary experimentation onthe complexity of the problem canbe found in:
http://www.sc.ehu.es/ccwbayes/members/
jonathan/home/News_and_Notables/Entries/
2010/11/30_IMACS_2011.html
2DB
3DB
MDnB
123456789
nB
MDTAN
TAN
MD 1/1
MD 2/2
MD 2/3
Figure: Accuracy ranking fordifferent algorithms on 20artificial datasets, α = 0.05.
Jonathan Ortigosa-Hernandez
Semi-supervised Learning of Multi-dimensional Class Bayesian Network Classifiers
Outline MD Classification MDSSL Application to SA Conclusions Current Work
Application to Sentiment Analysis
I Sentiment Analysis (AKA Opinion Mining) is thecomputational study of opinions, sentiments and emotionsexpressed in text (Liu, 2010).
I When treating Sentiment Analysis as a classification problem,several different (but related) problems appear. For example:
1. Subjectivity Classification. Its aim is to classify a text assubjective or objective.
2. Sentiment Classification. It classifies an opinionated text asexpressing a positive, neutral, or negative opinion.
Jonathan Ortigosa-Hernandez
Semi-supervised Learning of Multi-dimensional Class Bayesian Network Classifiers
Outline MD Classification MDSSL Application to SA Conclusions Current Work
Motivation for Using Semi-supervised Learning ofMulti-dimensional Classifiers
1. Up to now, these subproblems have been studied in isolationdespite of being closely related. So, probably it would behelpful to use multi-dimensional classifiers.
2. Obtaining enough labeled examples for a classifier may becostly and time consuming. This motivates us to deal withunlabelled examples in a semi-supervised framework.
Jonathan Ortigosa-Hernandez
Semi-supervised Learning of Multi-dimensional Class Bayesian Network Classifiers
Outline MD Classification MDSSL Application to SA Conclusions Current Work
Hypothesis Formulated
First Hypothesis
The explicit use of the relationships between different classvariables can be beneficial to improve their recognition rates.
Second Hypothesis
Multi-dimensional techniques can work with unlabelled data inorder to improve the classification rates in Sentiment Analysis.
Jonathan Ortigosa-Hernandez
Semi-supervised Learning of Multi-dimensional Class Bayesian Network Classifiers
Outline MD Classification MDSSL Application to SA Conclusions Current Work
Properties of the Dataset
I Collected by Socialware Company S.A., from the ASOMOservice of mobilised opinion analysis.
I It consists of 2, 542 Spanish reviews extracted from a blog:I 150 documents have been labeled in isolation by an expert.I 2, 392 posts are left unlabelled.
Figure: The ASOMO corpus.Jonathan Ortigosa-Hernandez
Semi-supervised Learning of Multi-dimensional Class Bayesian Network Classifiers
Outline MD Classification MDSSL Application to SA Conclusions Current Work
Properties of Each Document I
Each document is represented as 14 features:I Obtained by using an open source morphological analyser
(Carreras et al, 2006).I Each feature provide different information related to
part-of-speech (PoS).
Feature Description Example1 First Persons Number of verbs in the fist person. Contrate ... .2 Second Persons Number of verbs in the second per-
son.Tienes ...
3 Third Persons Number of verbs in the third per-son.
Sabe ... .
4 Relational Forms Number of phatic expressions, i.e.expressions whose only function isto perform a social task.
(1) Hola.(2) Gracias de antemano.
5 Agreement Expres-sions
Number of expressions that showagreement or disagreement.
(1) Estoy de acuerdo contigo.(2) No tienes razon.
6 Request Number of sentences that expressa certain degree of request.
(1) Me gustarıa saber ...(2) Alguien podrıa ...
Table: Subset of features related to the implication of the author withother customers.
Jonathan Ortigosa-Hernandez
Semi-supervised Learning of Multi-dimensional Class Bayesian Network Classifiers
Outline MD Classification MDSSL Application to SA Conclusions Current Work
Properties of Each Document II
Each document has 3 class variables:
I Will to Influence: {declarative sentence, soft WI, medium WI,strong WI}
I Sentiment: {very negative, negative, neutral, positive, verypositive}
I Subjectivity: {Yes (subjective), No (objective)}
Jonathan Ortigosa-Hernandez
Semi-supervised Learning of Multi-dimensional Class Bayesian Network Classifiers
Outline MD Classification MDSSL Application to SA Conclusions Current Work
Experiment 1 - Set Up I
First Hypothesis
The explicit use of the relationships between different classvariables can be beneficial to improve their recognition rates.
I The ASOMO corpus has been used to learn:I 3 (uni-dimensional) naive Bayes classifiers, one per each class
variable.I A (uni-dimensional) naive Bayes classifier with a compound
class variable.I A multi-dimensional naive Bayes classifier.
Jonathan Ortigosa-Hernandez
Semi-supervised Learning of Multi-dimensional Class Bayesian Network Classifiers
Outline MD Classification MDSSL Application to SA Conclusions Current Work
Experiment 1 - Set Up II
First Hypothesis
The explicit use of the relationships between different classvariables can be beneficial to improve their recognition rates.
I Features from ASOMO dataset are discretised into 3 valuesusing equal frequency.
I In addition to the ASOMO feature set, three state-of-the-artfeature sets are used:
I UnigramsI Unigrams + BigramsI PoS tagging
I Results averaged over 20 × 5 fold cross validation.
Jonathan Ortigosa-Hernandez
Semi-supervised Learning of Multi-dimensional Class Bayesian Network Classifiers
Outline MD Classification MDSSL Application to SA Conclusions Current Work
Experiment 1- JOINT Accuracies
Figure: JOINT accuracies on ASOMO corpus using three differentfeature sets in both uni and multi-dimensional scenarios (20× 5cv)
Jonathan Ortigosa-Hernandez
Semi-supervised Learning of Multi-dimensional Class Bayesian Network Classifiers
Outline MD Classification MDSSL Application to SA Conclusions Current Work
Experiment 2 - Set Up
Second Hypothesis
Multi-dimensional techniques can work with unlabelled data inorder to improve the classification rates in Sentiment Analysis.
I The ASOMO dataset has been used to learn:I 3 (uni-dimensional) Bayesian network classifiers: nB, TAN and
2DB.I 5 MDBNC: MDnB, MDTAN, MD 2/2, MD 2/3 and MD 2/4.
I In both Supervised and Semi-supervised (EM algorithm)learning frameworks.
I Features from ASOMO dataset are discretised into 3 valuesusing equal frequency.
I Results averaged over 20 × 5 fold cross validation.
Jonathan Ortigosa-Hernandez
Semi-supervised Learning of Multi-dimensional Class Bayesian Network Classifiers
Outline MD Classification MDSSL Application to SA Conclusions Current Work
Experiment 2 - JOINT Accuracy
Figure: JOINT accuracies on ASOMO dataset in the supervised andsemi-supervised learning frameworks.
Jonathan Ortigosa-Hernandez
Semi-supervised Learning of Multi-dimensional Class Bayesian Network Classifiers
Outline MD Classification MDSSL Application to SA Conclusions Current Work
Experiment 2 - Will to Influence
Figure: Accuracies for the Will to Influence class variable on ASOMOdataset in the supervised and semi-supervised learning frameworks.
Jonathan Ortigosa-Hernandez
Semi-supervised Learning of Multi-dimensional Class Bayesian Network Classifiers
Outline MD Classification MDSSL Application to SA Conclusions Current Work
Experiment 2 - Sentiment Polarity
Figure: Accuracies for the Sentiment Polarity class variable on ASOMOdataset in the supervised and semi-supervised learning frameworks.
Jonathan Ortigosa-Hernandez
Semi-supervised Learning of Multi-dimensional Class Bayesian Network Classifiers
Outline MD Classification MDSSL Application to SA Conclusions Current Work
Experiment 2 - Subjectivity
Figure: Accuracies for the Subjectivity class variable on ASOMO datasetin the supervised and semi-supervised learning frameworks.
Jonathan Ortigosa-Hernandez
Semi-supervised Learning of Multi-dimensional Class Bayesian Network Classifiers
Outline MD Classification MDSSL Application to SA Conclusions Current Work
Conclusions I - Methodology
I Multi-dimensional classification and semi-supervised learningare two different branches of machine learning.
I With this research, we have established a bridge betweenthem showing that:
I Uni-dimensional approaches cannot capture the real nature ofmulti-dimensional problems.
I More accurate classifiers can be found using themulti-dimensional learning approaches.
I The use of large amounts of unlabelled data can be beneficialto improve recognition rates.
Jonathan Ortigosa-Hernandez
Semi-supervised Learning of Multi-dimensional Class Bayesian Network Classifiers
Outline MD Classification MDSSL Application to SA Conclusions Current Work
Conclusions II - Application
I With respect to the Sentiment Analysis application, we haveproposed a novel perspective to solve the problem.Experimental results demonstrate that the use ofmulti-dimensional classification, as well as the use ofunlabelled data, can lead us to more accurate classifiers.
Jonathan Ortigosa-Hernandez
Semi-supervised Learning of Multi-dimensional Class Bayesian Network Classifiers
Outline MD Classification MDSSL Application to SA Conclusions Current Work
Feature Selection
Title: Semi-supervised Feature Selection in Multi-dimensionalProblems
Description: Develop a methodology able to identify irrelevant and redundantfeatures in multi-dimensional problems for dimension reduction ina semi-supervised framework.
Motivation
I Feature selection try to avoid problems related to overfitting,computation burden, etc.
I Up to now, there is no feature selection technique which is able to dealwith multiple class variables.
I Few work has been done in semi-supervised feature selection (Cai et al,2011).
Jonathan Ortigosa-Hernandez
Semi-supervised Learning of Multi-dimensional Class Bayesian Network Classifiers
Outline MD Classification MDSSL Application to SA Conclusions Current Work
Semi-supervised Feature Selection
X1 XnC1 Cm
Supervised feature selection Unsupervised feature selection
I Feature relevance is usuallyevaluated by their correlation withthe class label.
I The labelled sample is generallytoo small and insufficient for thispurpose.
I Evaluated by their capability ofkeeping certain properties of thedata, e.g. variance or separability.
I Ignoring label information cancause downgrades in theperformance.
Jonathan Ortigosa-Hernandez
Semi-supervised Learning of Multi-dimensional Class Bayesian Network Classifiers
Outline MD Classification MDSSL Application to SA Conclusions Current Work
Semi-supervised Feature Selection via Spectral Analysis I
Based on the cluster assumption - Unsupervised feature selection(Zhao et al, 2007)
f f'
I Unsupervised perspective:Both solutions are OK.
I Supervised point of view: fis better than f ′.
Jonathan Ortigosa-Hernandez
Semi-supervised Learning of Multi-dimensional Class Bayesian Network Classifiers
Outline MD Classification MDSSL Application to SA Conclusions Current Work
Semi-supervised Feature Selection via Spectral Analysis II
Proposal: Use the clustering assumption to identify the relevantfeatures, but giving more relevance to the features which clearlyseparate the labels.
I Drawback: This algorithm does not take into account theredundant detection.
Jonathan Ortigosa-Hernandez
Semi-supervised Learning of Multi-dimensional Class Bayesian Network Classifiers
Outline MD Classification MDSSL Application to SA Conclusions Current Work
Semi-supervised Feature Selection via BASSUM
Calculate the Markov blanket of a class variable by using G 2
conditional independence tests with both labelled and unlabelleddata. It detects redundant features (Cai et al, 2011).
Markov blanket of a classvariable A is the set of all parents,children and spouses of A in theBayesian network.
Jonathan Ortigosa-Hernandez
Semi-supervised Learning of Multi-dimensional Class Bayesian Network Classifiers
Outline MD Classification MDSSL Application to SA Conclusions Current Work
BASSUM Example
F3
F1
F2
I F3 is the class variable
I F2 is in the Markov Blanket S(F3), i.e.S(F3) = {F2}
I We are checking if F1 is also in S(F3)
I So, want to determineF1 ⊥ F3|S(F3) = F1 ⊥ F3|F2
F1 F2 F3
DEFINITIONS
cijk ≡ number ofinstances that satisfyF1 = f i
1 ,F2 = f j2 ,F3 =
f k3
Marginal sumsc+jk =
Pi cijk ,
ci+k =P
j cijk ,cij+ =
Pk cijk .
G 2 conditional independence test
G 2 = 2Xijk
cijk lncijkc++k
ci+kc+jk∼ χ2
df = (|F1| − 1)(|F2| − 1)|F3|
Labelled data Unlabelled data
Jonathan Ortigosa-Hernandez
Semi-supervised Learning of Multi-dimensional Class Bayesian Network Classifiers
Outline MD Classification MDSSL Application to SA Conclusions Current Work
Important Ideas
I Redundancy and Relevancy
I Spectral Analysis → A definition of the Markov Blankets inmulti-dimensional Bayesian networks is needed.
C*
X2X4
X1X6
X5 X3
X2X4
X1X6
X5 X3
C1
C2 C3
I BASSUM approach → Modify a classical feature selectiontechnique (Saeys et al, 2007) to be able to deal withmulti-dimensional problems in a semi-supervised framework.
Jonathan Ortigosa-Hernandez
Semi-supervised Learning of Multi-dimensional Class Bayesian Network Classifiers
Outline MD Classification MDSSL Application to SA Conclusions Current Work
Affect Analysis
Title: Application to Affect AnalysisDescription: Use the methodology proposed in this presentation to dealwith the problem of Affect Analysis.Collaboration: Socialware S.A.
Motivation (Abbasi et al., 2008)
I Affect Analysis is concerned with the analysis of text containing emotionsand it tries to extract a large number of potential emotions, e.g.happiness, sadness, anger, hate, violence, excitement, etc.
Jonathan Ortigosa-Hernandez
Semi-supervised Learning of Multi-dimensional Class Bayesian Network Classifiers
Outline MD Classification MDSSL Application to SA Conclusions Current Work
Plutchik’s Affect Model
We want to take a step forward in this problem taking advantageof the potential possibilities of the MDBNC to model complexrelationships between the class variables.
4 class variables with threepossible values: {−1, 0, 1}.
I LOVE-REMORSE(Aceptacion-Disgusto)
I CONTEMPT-SUBMISSION(Anticipacion-Sorpresa)
I AGGRESSIVENESS-AWE(Ira-Miedo)
I OPTIMISM-DISAPPROVAL(Alegrıa-Tristeza)
Jonathan Ortigosa-Hernandez
Semi-supervised Learning of Multi-dimensional Class Bayesian Network Classifiers
Outline MD Classification MDSSL Application to SA Conclusions Current Work
Questions
THANKS!jonathan.ortigosa@ehu.es
Jonathan Ortigosa-Hernandez
Semi-supervised Learning of Multi-dimensional Class Bayesian Network Classifiers
top related