Self-Adaptation of �Online Recommender Systems �via Feed-Forward Controllers�

Licia Capra �University College London �

Workshop on Self-Awareness in Computing �June 27th, 2010 �

RECOMMENDER SYSTEMS�

SCENARIO�

Web 2.0 Recommender Systems (CiteULike)�

RECOMMENDER SYSTEMS�

items�

users�

ratings �

RECOMMENDER SYSTEMS�

items�

users�

ratings �

WEB 2.0 RECOMMENDER SYSTEMS�

RECOMMENDER SYSTEMS (Web 2.0)�

items�

users�

tags�

RECOMMENDER SYSTEMS�

User x Item�

a�b �c�d�

i j k l m �

PROBLEM�

DOMAIN PROBLEM ANALYSIS�

  Dataset: Bibsonomy�•  1.3k users; 24k items; 12k tags; 73k bookmarks�

!"

#!!!!"

$!!!!"

%!!!!"

&!!!!"

'!!!!"

(!!!!"

)!!!!"

#" (" ##" #(" $#" $(" %#" %(" &#" &(" '#"

*++,-./,0"

123-0"

4.50"

603/0"

4.50"5/+7"8.023/"

123-0"5/+7"8.023/"

9-3":-+;2<0="

5/+72<"

DOMAIN PROBLEM ANALYSIS�

 Method of Assessment �Time    Period  

User   Tag   Item  

T1   6%   17%   2%  

T2   6%   1%   11%  

T3   6%   2%   3%  

Ti  

Training  Ti(1)  

Training  Ti(2)  

1500  tests  

DOMAIN PROBLEM ANALYSIS�

 Method of Assessment �Time    Period  

User   Tag   Item  

T1   6%   17%   2%  

T2   6%   1%   11%  

T3   6%   2%   3%  

Ti  

Training  Ti(1)  

Training  Ti(2)  

1500  tests  

Accuracy  Loss  (25-­‐75)  

Accuracy  Loss  (50-­‐50)  

Accuracy  Loss  (75-­‐25)  

24%   32%   45%  

12%   20%   23%  

3%   10%   14%  

DOMAIN PROBLEM ANALYSIS�

 Method of Assessment �Time    Period  

User   Tag   Item  

T1   6%   17%   2%  

T2   6%   1%   11%  

T3   6%   2%   3%  

Ti  

Training  Ti(1)  

Training  Ti(2)  

1500  tests  

Accuracy  Loss  (25-­‐75)  

Accuracy  Loss  (50-­‐50)  

Accuracy  Loss  (75-­‐25)  

24%   32%   45%  

12%   20%   23%  

3%   10%   14%  

IDEA �

DYNAMIC UPDATE METHODOLOGY�

  Recommender Systems as Self-Adaptive Systems�

Feed-­‐Back  

[Recommender  ]System  

u  [update  frequency]  

y  [recommendaFon    

list]  

x  [users,items,tags]  

DYNAMIC UPDATE METHODOLOGY�

  Feed-Forward Controller for Dynamic Updating of Recommender Systems�

Feed-­‐Forward  

[Recommender  ]System  u  

[update  frequency]  

x  [users,items,tags]   y  

[recommendaFon    list]  

DOES IT WORK?�

EVALUATION �

  Empirical Estimate of Performance Loss�

!"#

$"#

%"#

&"#

'"#

(!"#

($"#

(%"#

(&"#

("# $"# )"# %"# *"# &"# +"# '"# ,"# (!"#

+*"-$*"#

*!"-*!"#

$*"-+*"#

./01232456/01788#8433#

9/4:;<#/7;0#

EVALUATION �

  Cumulative Error (50-50 test set)�

!"

#"

$"

%"

&"

'!"

'#"

'" %" ''" '%" #'" #%" ('" (%" $'" $%" )'" )%" %'" %%" *'" *%" &'"

+,-./01"

2345678")9"

2345678"#9"

:88;01"

EVALUATION �

  Cumulative Error & N. of Updates�

!"

#"

$"

%"

&"

'!"

'#"

'" %" ''" '%" #'" #%" ('" (%" $'" $%" )'" )%" %'" %%" *'" *%" &'"

+,-./01"

2345678")9"

2345678"#9"

:88;01"

Technique� N. Of Updates�

Weekly� 80 �

Adaptive 2% � 29 �

Adaptive 5% � 13 �

Monthly � 18 �

EVALUATION �

  Cumulative Error & N. of Updates�

!"

#"

$"

%"

&"

'!"

'#"

'" %" ''" '%" #'" #%" ('" (%" $'" $%" )'" )%" %'" %%" *'" *%" &'"

+,-./01"

2345678")9"

2345678"#9"

:88;01"

Technique� N. Of Updates�

Weekly� 80 �

Adaptive 2% � 29 �

Adaptive 5% � 13 �

Monthly � 18 �

CONCLUSIONS�

  Accuracy vs Cost Tradeoff may lead to suboptimal choices�

  Recommender Systems as Self-Adaptive Systems�

  Feed-Forward Control Theory for Unobservable Outputs�

… & FUTURE WORK�

  Automation of Empirical Evaluation �

  Beyond Accuracy and Cost (diversity, surprise, serendipity)�

  On self-adaptation �•  B.H. Cheng, et al. Software Engineering for Self-Adaptive Systems: A Research Roadmap.

In Software Engineering for Self-Adaptive Systems, pages 1-26, 2009. Springer-Verlag �•  Y. Brun, et al. Engineering Self-Adaptive Systems through Feedback Loops. In Software

Engineering for Self-Adaptive Systems, pages 48-70, 2009. Springer-Verlag.�

  On recommender-systems�•  J. Herlocker, et al. An Algorithmic Framework for Performing Collaborative Filtering. In

Proc. of the 22nd Annual International Conference on Research and Development in Information Retrieval, pages 230-237, New York, NY, USA, 1999. ACM.�

•  G. Adomavicius and A. Tuzhilin. Context-Aware Recommender Systems. In Proc. of the ACM Conference on Recommender Systems, 2008.�

  From my group �•  V. Zanardi and L. Capra. Social Ranking: Uncovering Relevant Content using Tag-based

Recommender Systems. In Proc. of the Conference on Recommender Systems, pages 51-58, 2008. ACM.�

•  V. Zanardi and L. Capra. "Dynamic Updating of Online Recommender Systems via Feed-Forward Controllers". In 6th Intl. Symposium on Software Engineering for Adaptive and Self-Managing Systems (SEAMS 2011). Waikiki, Honolulu, Hawaii, USA. May 2011 �

THANK YOU! �