analysis, design and implementation of a multi-criteria recommender system based on aspect...
Post on 19-Jul-2015
105 Views
Preview:
TRANSCRIPT
Analysis, design and implementation of a Multi-
Criteria Recommender System based on Aspect
Extraction and Sentiment Analysis techniques
Instructors: Student:
Prof. Giovanni Semeraro Davide GIANNICO
Dott. Marco de Gemmis
Department of Computer ScienceMaster of Computer Science (Msc)
3
Need: necessity of the user to be supported during
complex decisional processes
Tendency: development of on-line platforms which provide
recommendations to the user and where the community
expresses the own opinion for a item class
Some stats 315M unique views per month & 200M reviews(TripAdvisor, 2014);
168M unique views per month & 35M reviews (Amazon, 2013);
139M unique views per month & 67M reviews (Yelp, 2014).
Scenery
Opportunity
4
Take advantage of the reviews informative power
incorporating such information in the recommendation
process.
Concept: make «value» from raw data.
Recommender Systems
5
Recommender Systems (RecSys) are decision support and
information filtering tools
The main goal is helping users which access to a data
source for discovering information or items that could be
interesting for them
Recently, RecSys area has focused on Multi-Criteria
RecSys[AMK11]
[AMK11] G. Adomavicius, N. Manouselis, Y. Kwon. ‘Multi-Criteria Recommender Systems’. In:
Recommender Systems Handbook, pp 769-803, Springer US, 2011
Multi-Criteria Recommender Systems
6
Techniques which provide recommendations to the user,
modeling the utility concept espressed by the user for an
item as a ratings vectore on several criteria
Focus: multi-criteria item-based collaborative filtering
Item-to-item similarity matrix
1.Item-to-item matrix calculus
2. Rating prediction
multi-static-criteria ratings
Input:
multi-criteria ratings
Output: recommendations
𝑠𝑖𝑚 Itemi, Itemj= …
…
IB Collaborative Filtering RecSys
Discussed Problem
7
State-of-the-art Approach: default criteria taxonomies on which users can rate
Limits and challenges:
Proposed criteria are not exhaustive respect to userpreferences need to «customize» the criteria
Vagueness of the criteria need of fine-grained criteria
Proposed Approach
8
Automatic identification of criteria from the reviews
using Aspect Extraction techniques
Sentiment Analysis for associating a preference level to
the extracted criteria (implicit rating)
Extension of multi-criteria item-based recommendation
algorithms exploiting the information extracted
Aspect Extraction e Sentiment Analysis
Step[V11]
9
Aspect
Extraction
Sentiment
Analysis
review<Main Aspect,Sub-Aspect,Rilevance,
Sentiment Score,holder>
<location,station, 0.8, 0.9, >
<staff, courtesy, 0.9, -0.6, >
<…, …, …, …, >
Opinion Retrieval
Engine (ORE) system
[V11] V. Giuliani. ‘Analisi, progettazione e sviluppo di un framework per l'opinion retrieval basato su tecniche
automatiche di aspect extraction’. In Tesi di Laurea in Informatica Magistrale, 2011
(multi-ORE-criteria ratings)
Algorithm A: #multi-ORE-criteria (1/2)
10
review dataset
multi-ORE-criteria ratings
Aspect
Extraction
Sentiment
Analysis
ORE System
Multi-Criteria Item-Based Collaborative Filtering RecSys which
uses the extracted ratings from the reviews for calculating the
item-to-item similarity matrix
Algorithm A: #multi-ORE-criteria (2/2)
11
multi-ORE-ratings Item-item similarity
matrix
(Multi-dimensional Pearson
coefficient)
𝑠𝑖𝑚 𝐼𝑡𝑒𝑚𝑖, Item𝑗 =
𝑢∈U,c=1..k 𝑅𝑢,𝑐,𝑖 − 𝑅 𝑖)(𝑅𝑢,𝑐,𝑖 − 𝑅 𝑗
𝑢∈U,c=1…k 𝑅𝑢,𝑐,𝑖 − 𝑅 𝑖
2𝑅𝑢,𝑐,𝑖 − 𝑅 𝑗
2
(Rating Prediction)
𝑃𝑢,𝑖 = 𝑎𝑙𝑙 𝑠𝑖𝑚𝑖𝑙𝑎𝑟 𝑖𝑡𝑒𝑚𝑠 𝑗(𝑠𝑗 ∗ 𝑅𝑢)
𝑛( , , 0.8 )
( , , 0.6 )
predicted ratings
Algorithm B: Multi-criteria Aspect-based Recommender
system based on sentimenT Analysis (#MARTA) (1/2)
12
Item #1Aspect
Extraction
Sentiment
Analysis
Position
Food
Staff-courtesy
Breakfast-buffet
<position, *,0.9, ..>(review#1)
<staff,courtesy, -0.6, ..>(review#2)<position, *,0.7, ..>(review#2)
(Item#1 summary)
If <aspect> 𝜖 summary
add <aspect> to the running average
else
add <aspect> to summary
Item#1 review set
(Item#1 multi-ORE-ratings)
…
…
Multi-criteria CF RecSys in which the item-to-item similarity matrix
is calculated considering each item description (summary)
13
Position
Food
Staff-courtesy
Breakfast-buffet
…
Food
Room-clean
Pool
Item-item similarity matrix
Item #1 summary
Item #2 summary
(Multi-dimensional
similarity measure)
(Rating Prediction)
( , , 0.9 )
( , , 0.7 )
predicted ratings
Algorithm B: Multi-criteria Aspect-based Recommender
system based on sentimenT Analysis (#MARTA) (2/2)
Room-view
…
𝑠𝑖𝑚 Itemi, Itemj= …
Design
17
Research questions
Q1) #multi-ORE-criteria algorithm overtakes the single-criteria(#single-criteria) and multi-criteria techniques (#multi-static-criteria), which use default criteria, results ?
Q2) Multi-criteria Aspect-based Recommender system based on sentimenT Analysis(#MARTA) algorithm overtakes the state-of-the-art approaches results?
10-Fold Cross Validation. For each run the training set isused for calculating the similarity matrix
Evaluation of the approaches on the respective test set
Evaluation metrics: MAE, RMSE, Precision, Recall and F-Measure
Statistical validation using the Wilcoxon test
Dataset
18
Amazon
TripAdvisor
Yelp
#ratings #users #items #ratings/
user
#ratings/
item
sparsity
355.949 2.850 2.820 124,73 126,06 0,96
#ratings #users #items #ratings/
user
#ratings/
item
sparsity
229.905 45.981 11.537 5 19,9 0,99
#ratings #users #items #ratings/
user
#ratings/
item
sparsity
208.135 1.386 1.580 150,16 131,73 0,90
Q1 Results (#multi-ORE-criteria vs #single-
criteria and #multi-static-criteria)
19
Dataset TripAdvisor
MAE
SVD 0,87
#single-criteria 1,32
#multi-static-criteria 1,10
#Multi-ORE-criteria 0,96
0,87
1,32
1,10
0,96
0,00
0,20
0,40
0,60
0,80
1,00
1,20
1,40
MA
E
F-Measure
SVD 0,59
#single-criteria 0,52
#multi-static-criteria 0,53
#multi-ORE-criteria 0,71
0,59
0,52 0,53
0,71
0,00
0,10
0,20
0,30
0,40
0,50
0,60
0,70
0,80
F-M
easu
re
Q2 Results (#MARTA)
20
DatasetYelp (sparsity: 0,99)
MAE
SVD 0,91
#single-criteria 1,18
#multi-ORE-criteria 1,13
#MARTA 0,85
0,91
1,181,13
0,85
0,00
0,20
0,40
0,60
0,80
1,00
1,20
1,40
MA
E
F-Measure
SVD 0,58
#single-criteria 0,49
#multi-ORE-criteria 0,55
#MARTA 0,63
0,58
0,49
0,55
0,63
0,00
0,10
0,20
0,30
0,40
0,50
0,60
0,70
F-M
easu
re
Q2 Results (#MARTA)
21
Dataset TripAdvisor (sparsity: 0,96)
MAE
SVD 0,87
#single-criteria 1,32
#multi-ORE-criteria 0,96
#MARTA 0,87
0,87
1,32
0,960,87
0,00
0,20
0,40
0,60
0,80
1,00
1,20
1,40
MA
E
F-Measure
SVD 0,59
#single-criteria 0,51
#multi-ORE-criteria 0,71
#MARTA 0,58
0,59
0,51
0,71
0,58
0,00
0,10
0,20
0,30
0,40
0,50
0,60
0,70
0,80
F-M
easu
re
Q2 Results (#MARTA)
22
Dataset Amazon (sparsity: 0,90)
MAE
SVD 0,71
#single-criteria 0,65
#multi-ORE-criteria 0,55
#MARTA 0,71
0,710,65
0,55
0,71
0
0,1
0,2
0,3
0,4
0,5
0,6
0,7
0,8
MA
E
F-Measure
#SVD 0,62
#single-criteria 0,65
#multi-ORE-criteria 0,74
#MARTA 0,63
0,62
0,65
0,74
0,63
0,54
0,56
0,58
0,60
0,62
0,64
0,66
0,68
0,70
0,72
0,74
0,76
F-M
easu
re
Analysis of the results
23
Q1) #multi-ORE-criteria algorithm overtakes the single-
criteria(#single-criteria) and multi-criteria techniques (#multi-
static-criteria), which use default criteria, results ?
#multi-ORE-criteria showed better results than #multi-
static-criteria and #single-criteria (MAE and F1)
Q2) Multi-criteria Aspect-based Recommender system based on
sentimenT Analysis(#MARTA) algorithm overtakes the state-of-
the-art approaches results?
#MARTA showed good results especially on sparse
dataset
Conclusions
24
Indipendence from the domain
Influenced by the review quality
Good performance especially on the item classification
(relevant or not)
top related