fypca4
TRANSCRIPT
Mining User’s
Opinions in
Hotel
National University Of Singapore
TEY JUN HONG
U095074X
1. Background
2. Formulating the problem
3. Data Mining Process
4. Techniques
5. Analysis
Content
01
• Extraction of meaningful / useful / Interesting
patterns from a large volume of data sources
• In this project, the source will be large
volume of WEB HOTEL REVIEWS data
• Data mining is one of the top ten emerging
technology
What is Data
Mining?
MIT’s TECHNOLOGY REVIEW 2004
• Process of exploration and analysis
• By automatic / semi automatic means
• With little or no human interactions
• To discover meaningful patterns and rules
What is Data
Mining?
MASTERING DATA MINING BY BERRY AND LINOFF, 2000
• Increase in social media and web
user
• Increase in valuable opinion
oriented data in Hotel due to web
expansion
• Identify potential hotel to stay by
looking at the aspects
• Overall Sentiments on hotel are
greatly sought on the web for
Sentiment Analysis
User’s Opinions in Hotel
• Identify best prospects
(ASPECTS), and retain customers
• Predict what ASPECTS
customers like and promote
accordingly
• Learn parameters influencing
trends in sales and margins
• Identification of opinions for
customers
Sentiment Analysis !!!
What can Data Mining do?
• Exponential growth of user’s
opinions
• Limitations of human analysis
• Accuracy of human analysis
Machines can be trained to take
over human analysis with advanced
computer technology and it is done
with LOW COST
What are the problems?
• Unable to read like a human
• No emotions
• Cannot detect sarcasm
• Expression of sentiments in
different topic and domain
• Polarity analysis
• Facts Vs Opinion
Some Limitations of machines
• “The service is as good as none”. Negation not obvious to machine
• “Swimming pool is big enough to swim with comfort” , “There is a big crowd at the counter complaining”. Polarity might change with context.
• “The room is warmer than the lobby”. Comparisons are hard to classify
Some machine limitation
examples
• Machine learning
• Pattern recognition
• Statistics
• Databases
Sentiment
Analysis
• A tool for data mining and intelligent decision
support
• Application of computer algorithms that
improve automatically through experience
Machine Learning
MASTERING DATA MINING BY BERRY AND LINOFF, 2000
• Supervised Learning
• A training set is provided (data
with correct answers) which is
used to mine for known pattern
• Unsupervised Learning
• Data are provided with no prior
knowledge of the hidden
patterns that they contain.
• Semi Supervised Learning
Types of Machine learning
• Rule Mining and Rule learning
• Bayesian Networks
• Support Vector Machine
Supervised Learning techniques
• Prediction of sentence polarity
• Classification of polarity for sentiment
lexicon
• Detection of relations
Project Objective
• Large data set
• Relevant Prior Knowledge to
domain, in our case the hotel
domain
• Eg. Rating
• Sentiment lexicon for sentiment
analysis
• Data selection for reliability and
standards
Pre-requisite
Data Mining Process
• Frequent problem : Data inconsistencies
• Duplicate data
• Spelling Errors != Trim from data
• Foreign accent and characters
• Singular / Plural conversion
• Punctuations removal / replacement
• Noise and incomplete data
• Naming convention misused, same name but
different meaning
Cleaning the “Dirty” Data (60% of
effort)
• Part of Speech Tagging (POS) using Brill
Tagger
• Polarity tagging using sentiment lexicon
Data Preprocessing (Laundering)
• Part of Speech Tagging (POS) using Brill
Tagger - NO PROBLEM
-95% accuracy POS tagging words after data
cleaning
Findings
• Polarity tagging using sentiment lexicon –
BIG PROBLEM
-40% sentiment words not found in sentiment
lexicon
-10% sentiment words with a positive or
negative polarity found are in the neutral section
of sentiment lexicon
Findings
• Sentiment lexicon not comprehensive to fulfill
machine learning technique adopted
• Polarity of sentiment words who are domain
dependent are founded in neutral section of
sentiment lexicon
• Polarity of sentiment words can also change
within the domain even though they are
domain dependent
EXPANSION OF LEXICON !!!
Problems
• Classify the polarity of unlabeled sentiment
word using rule based mining
• Classify domain dependent sentiment words
• Establish word relations between labeled and
unlabeled sentiment words
Solution
• Rule based mining using conjunction and
punctuation
Data Processing
Polarity Assignment Rules
Same Adj – AND/OR - Adj
Opposite Neg - Adj – AND/OR - Adj / Adj – AND/OR - Neg- Adj
Same Neg - Adj – AND/OR - Neg- Adj
Opposite Adj – BUT/NOR – Adj
Same Neg - Adj – BUT/NOR - Adj / Adj – BUT/NOR - Neg- Adj
Opposite Neg - Adj – BUT/NOR - Neg- Adj
Same Adj , Adj
• Relation Network – Aspect – Sentiment word
pair
Data Processing
• Relation Network – Aspect – Sentiment word
pair
Data Processing
• Using the expanded sentiment lexicon, we
analyze the polarity sentiment by doing a
sentiment lookup using Bayesian Network
Analysis
• To determine polarity of sentiments
P(X | Y) = P(X) P(Y | X) / P(Y)
• Probability that a sentiments is positive or
negative, given it's contents
• Assumptions: There is no link between words
• P(sentiment | sentence) =
P(sentiment)P(sentence | sentiment) /
P(sentence)
Bayesian
• Precision = N (agree & found) / N (found)
• High precision means most of the correct
sentiment words are found by the system
• Recall = N (agree & found) / N (agree)
• High recall means most of found sentiment
words are correctly labeled by the system
Validation
• It is found that out of the 350 aspect-
unlabelled sentiment word pairs,
• Only 194 are founded by the methods.
Thus, the precision is about 57%.
• The recall is also not very high; only 126
words are corrected labelled by the
system, which is about 63%.
Validation Results
• The results will improve if more rules are
applied such the inclusion of more adverbs
such as “excessively” as negation words.
• There might not be enough dataset for the
system to work on. There are only 350 aspect-
unlabelled sentiment word pairs for the
application to work with.
• This, however requires more human judges to
validate the data
Discussion
• Comprehensive Sentiment Lexicon is a
simple yet effective solution to sentiment
analysis as it does not requires prior training
• Current sentiment lexicon does not capture
such domain and context sensitivities of
sentiment expressions
Conclusion
• This leads to poor coverage
• Thus, expanding general sentiment lexicon to
capture domain and context sensitivities of
sentiment expressions are advocated
Conclusion
01DEMO
Questions?