assessing the value of contributions in tagging systems
DESCRIPTION
Assessing the Value of Contributions in Tagging Systems. Elizeu Santos- Neto , Flavio Figueiredo Jussara Almeida, Miranda Mowbray Marcos Gon çalves, Matei Ripeanu. The 2 nd IEEE SocialCom /SIN -- August 2010. Commons-based Peer Production Systems. - PowerPoint PPT PresentationTRANSCRIPT
Elizeu Santos-Neto, Flavio FigueiredoJussara Almeida, Miranda MowbrayMarcos Gonçalves, Matei Ripeanu
Assessing the Value of Contributions in Tagging Systems
The 2nd IEEE SocialCom/SIN -- August 2010
Decentralized, Collaborative & Non-proprietary [1]2[1] Y. Benkler. “The Wealth of Networks”. Yale University Press (2006).
Commons-based Peer Production Systems
Offline Online
Car pooling Information Sharing
Resource Sharing
Volunteer Firefighters
BitTorrent
Wikis
TaggingQ&A Portals
Online Peer Production Systems
3
Tagging Systems
BandwidthTimeExpertise CPU
Annotation = Tags + Items
Tags are free-form words
Items can be virtually anything URLs, photos, videos,
citation records, etc…Photos Annotations
Why is it important to study tagging systems?
Increasingly popularMillions of users [2]GBytes of content and annotations daily [2]
User-generated metadata = new opportunities To improve existing systems (e.g., social search)To create new mechanisms (e.g., reputation systems)
Open problem: how to quantify the value of user contributions in these systems?
4[2] R. Ramakrishnan, A. Tomkins. “Toward a PeopleWeb”. IEEE Computer 40(8): 63-72 (2007)
5
To define a method that quantifies the value of users’ contribution in tagging systems that is
accurate, feasible and robust.
What is the tolerance to
malicious users?
What is the computational complexity?
How close is it to the true value of user contribution?
Long Term Goal
Problem formalization
A solution framework
A method to quantify the value of tagsEntropy-based metricAlgorithms to compute the metric
6
Where Are We? (Contributions to Date)
Solution FrameworkIntroduction& Motivation
Solution Framework
Value of Tags Evaluation
Conclusions &
Future Work
Value of User Contributions inTagging Systems
Contribution = Tags + Items
Value of tags Context: navigation/search Intuition: value ≈ improvement on navigation/search
Value of an itemContext: item usage Intuition: value ≈ usefulness to a user
8
Content Annotations
9
Info Seeker
Relevant Item Set Finder
Contribution Value
Item Usage Monitor
Tag Value Calculator
Info Producer
Item Value Calculator
Tag Value Aggregator
Item Value Aggregator
Information Needs
Past activity
Tags
Items
Items
UsageValues
Values
The Value of User ContributionInformation flow
Value of TagsIntroduction& Motivation
Solution Framework
Value of Tags Evaluation
Conclusions &
Future Work
Intuition: tags are valuable if they narrow the scope of navigation, while retrieving relevant items.
11
Value of Tags
Items in the system
Relevant items to an info seeker
Items retrieved by a set of tags
Value of tags is proportional to this
intersection
Tags published by an info producer
Evaluation CriteriaIntroducti
on&
Motivation
Solution Framew
orkValue of
TagsEvaluati
on
Conclusions & Future Work
Feasibility: is the method efficient?
Accuracy: is the estimation close to the real value?
Robustness: can users boost their contributions maliciously?
13
Evaluation Criteria
14
Preliminary ResultsFeasibility – Part 1
Only 4% of users have more than 100 unique tag
assignments.
15
Preliminary ResultsFeasibility – Part 2
80% of users have NOT produced tags/items in 1
moth or more.
Conclusions & Future WorkIntroduction& Motivation
Solution Framework
Value of Tags Evaluation
Conclusions &
Future Work
ConclusionsAssessing the value of user contributions in social
tagging systems is a relevant and challenging problem
This work ……proposes a solution framework…provides preliminary results on feasibility
Current efforts…Evaluating techniques that estimate relevant itemsDesigning algorithms to calculate and aggregate value
17
Future WorkAlgorithms
Exploit user activity similarity Aggregation method that exploits implicit social relations
Evaluation Accuracy – does the estimated value match a ground truth? Robustness – what about spammers?
Value of items
Build mechanisms that harness the value of contributions Spam detection Social search 18
http://www.ece.ubc.ca/~elizeus
Comments? Questions?
Assessing the Value of Contributions in Tagging Systems