an architecture for a general purpose multi-algorithm recommender system
DESCRIPTION
Our presentation at PRSAT Workshop (Practical use of Recommender Systems, Algorithms and Technology), at RecSys 2010.TRANSCRIPT
PRSAT 2010: Practical Use of Recommender Systems, Algorithms and Technologies
An Architecture for a General Purpose Multi-Algorithm Recommender System
José Carlos Cortizo, Francisco Manuel Carrero, Borja Monsalve
1
PRSAT 2010: Practical Use of Recommender Systems, Algorithms and Technologies
Index
2
Our starting point
From specific to general
Main challenges
General architecture
API definition
Current state and future work
PRSAT 2010: Practical Use of Recommender Systems, Algorithms and Technologies
Our starting point
Wipley: a social network for videogamers
Integrated our first recommender
Recommended games (CF)
Related games (content)
Gamers to follow (social)
Good performance
3
PRSAT 2010: Practical Use of Recommender Systems, Algorithms and Technologies
From specific to general
Wipley served us as a test case
We wanted to offer recommendations in other products
FlickrBabel (a multilingual multimedia search engine) -> Babxel
“Sell” recommendations to other web application developers
Use of Software as a Service (SaaS) as a way to decouple the recommender system from the specific applications
4
PRSAT 2010: Practical Use of Recommender Systems, Algorithms and Technologies
From specific to general
Several commercial services: Strands, DirectedEdge, RichRelevance, etc.
But almost no literature on how to develop an architecture for a commercial general recommender system
Needed to realize about the hardest challenges, and figure it out how to solve them
5
PRSAT 2010: Practical Use of Recommender Systems, Algorithms and Technologies
Main challenges
Interoperability
Accessing sources with different data structures
Not all recommender algorithms need the same information
Configurability and ease of use
Configure the algorithms to be used and the parameters and data to use
Usability may become a problem difficult to solve
6
PRSAT 2010: Practical Use of Recommender Systems, Algorithms and Technologies
Main challenges
Performance
Recommendations must be served in real time
But not everything can be precomputed
Disk usage
As we have to manage several recommenders, the data representation cannot be optimized
Scalability
Growing in users and items implies growing exponentialy the complexity of some kind of recommenders (ie CF based)
7
PRSAT 2010: Practical Use of Recommender Systems, Algorithms and Technologies
General architecture
We used an API REST to make it easier to integrate the recommender engine within any application
The backend architecture is based on several modules, making it easier to configure the system for each application domain
The REST API allows us to introduce several configuration elements as optional parameters in the requests
8
PRSAT 2010: Practical Use of Recommender Systems, Algorithms and Technologies
General architecture
9
ClientRequest
Processor
UpdateProfile
Recommendation
PRSAT 2010: Practical Use of Recommender Systems, Algorithms and Technologies
General architecture
10
PRSAT 2010: Practical Use of Recommender Systems, Algorithms and Technologies
General architecture
11
PRSAT 2010: Practical Use of Recommender Systems, Algorithms and Technologies
API definition
12
PRSAT 2010: Practical Use of Recommender Systems, Algorithms and Technologies
API definition
13
PRSAT 2010: Practical Use of Recommender Systems, Algorithms and Technologies
API definition
14
PRSAT 2010: Practical Use of Recommender Systems, Algorithms and Technologies
API definition
15
PRSAT 2010: Practical Use of Recommender Systems, Algorithms and Technologies
API definition
To make a request for a recommendation
http://webservices.brainsins.com/api/recomm/player/has/videogame/2.xml
http://webservices.brainsins.com/api/recomm/player/has/videogame/2/435.xml
The system returns a XML/JSON file containing the recommendations
16
PRSAT 2010: Practical Use of Recommender Systems, Algorithms and Technologies
Integration
17
PRSAT 2010: Practical Use of Recommender Systems, Algorithms and Technologies
Current state and future work
REST API solves the integration and interoperability problems
And helps with configuration
Need to work on performance and scalability
Studying to re-code the recommenders using map-reduce
Experimental platform
Developing new content based and CF based algorithms
Now we can make exhaustive studies
18
PRSAT 2010: Practical Use of Recommender Systems, Algorithms and Technologies
Questions?
19