slua: towards semantic linking of users with actions in crowdsourcing
DESCRIPTION
Recent advances in web technologies allow people to help solve complex problems by performing online tasks in return for money, learning, or fun. At present, human contribution is limited to the tasks defined on individual crowdsourcing platforms. Furthermore, there is a lack of tools and technologies that support matching of tasks with appropriate users, across multiple systems. A more explicit capture of the semantics of crowdsourcing tasks could enable the design and development of matchmaking services between users and tasks. The paper presents the SLUA ontology that aims to model users and tasks in crowdsourcing systems in terms of the relevant actions, capabilities, and rewards. This model describes different types of human tasks that help in solving complex problems using crowds. The paper provides examples of describing users and tasks in some real world systems, with SLUA ontology.TRANSCRIPT
www.insight-‐centre.org
SLUA: TOWARDS SEMANTIC LINKING OF USERS WITH ACTIONS IN CROWDSOURCING
Umair ul Hassan, Sean O’Riain, Edward Curry INSIGHT Centre for Data Analy4cs
Na4onal University of Ireland, Galway
1st International Workshop on Crowdsourcing the Semantic Web, CrowdSem’13, Sydney, Australia
www.insight-‐centre.org
Paper Overview
• MoJvaJon – MulJple crowdsourcing plaBorms – Lack of tools for finding tasks – Need to query across plaPorms for skills and knowledge of workers
• Problem – Enabling interoperability across crowdsourcing plaBorms – Suppor4ng users in their search for tasks – Enable task and user matching services
• ContribuJon – An ini4al lightweight ontology for describing crowd sourced tasks and
users with regard to human capabili4es, ac4ons, and rewards
2
www.insight-‐centre.org
Agenda
• Mo4va4on – Crowdsourcing Landscape – Seman4c Heterogeneity
• Challenges • SLUA Ontology • Examples • Summary
3
www.insight-‐centre.org
4
Crowdsourcing Landscape
www.insight-‐centre.org
Crowdsourcing Landscape
5
www.insight-‐centre.org
Challenges
• Difficult to interoperate across crowdsourcing systems and plaPorms – e.g. searching for appropriate workers on StackExchange for Wiki edi4ng tasks
• VariaJons of data semanJcs across systems and plaPorms – Different APIs used by current marketplaces
• ExisJng Taxonomies plaPorm-‐centric – Categorize crowdsourcing plaBorms instead of tasks – LiPkle considera4on human of factors such as ac4ons, capabili4es, mo4va4on
6
www.insight-‐centre.org
Heterogeneous Crowds
• Mul4ple requesters, tasks, workers, plaBorm
7
Collaborative Data Curation
Tasks Workers
Cyber Physical Social System
Platforms
www.insight-‐centre.org
PlaBorm Heterogeneity
Tasks Human AcJons Required CapabiliJes Rewards
Wikipedia Create Content Edit Content Moderate Content
Write text Include references Highlight mistakes
Domain Knowledge Wri4ng Research
Social Good
Quora Ask Ques4ons Answer Ques4ons
Write text Domain Knowledge
Reputa4on
Amazon Mechanical Turk
Micro Tasks Transcribe, Translate, Categorize, etc.
Various Capabili4es Money
TaskRabbit Physical Tasks Collect Item Deliver Item Shop Item, etc.
Various Capabili4es Money
Microtask Form Filling Scan Correc4on Data Verifica4on
Play games Online gaming Fun
8
www.insight-‐centre.org
Proposed Solu4on • A common model for describing tasks in crowdsourcing
• Seman4cally Linked Users and Ac4ons (SLUA)
9
– Lightweight semanJc descripJon of crowdsourcing tasks in terms of human capabiliJes, acJons, and rewards
• SLUA – Gaelic word for… …“Crowd”
www.insight-‐centre.org
SLUA Design Medthdology
1. Enumerate similar terms on crowdsourcing plaBorms
2. Define the main concept in each group of terms
3. Compare with exis4ng ontologies 4. Define core classes and their rela4onships 5. Extend core classes with subclasses 6. Create example instances
10
www.insight-‐centre.org
• Terms used for similar concepts in crowdsourcing plaBorms
Crowdsourcing Terminology
11
Amazon Mechanical
Turk Mobileworks ShorXask CrowdFlower
Task HIT Task ShortTask Microtask
User Worker Requester
Worker Developer
Solver Seeker
Contributor Customer
Capability Qualifica4on Filter
Reward Payment Payment Reward Payment
www.insight-‐centre.org
Conceptual Requirements
• Task: A unit of work to be performed by people in the crowd
• AcJon: The cogni4ve or psychomotor ac4vity that leads towards the comple4on of a task
• User: The human par4cipant, commonly described as “worker” in crowdsourcing marketplaces.
• Capability: The human ability, knowledge, or skill that allows a user to perform the necessary ac4ons for task comple4on.
• Reward: A core concept to the mo4va4on of people in the crowd
12
www.insight-‐centre.org
Exis4ng Ontologies Concept PIMO TMO HRM-‐O FOAF SIOC
Task Task Task
AcJon
User Person Job Seeker Person UserAccount
Reward Compensa4on
Reputa4on
Money Salary
Fun
Altruism
Learning
Capability
Loca4on Loca4on
Skill Skill
Knowledge
Ability Ability
Availability Interval
13
Personal Information Management Ontology (PIMO) Task Management Ontology (TMO) Human Resource Management Ontology (HRM-O) Friend of a Friend (FOAF) Semantically Interlinked Online Communities (SIOC)
www.insight-‐centre.org
SLUA Core Classes and Proper4es
14
Reward
Action
Capability
User Task
offers earns
includes performs
requires possesses
www.insight-‐centre.org
SLUA Sub-‐classes
• Capability – The ability of people to do things -‐ both the capacity and the opportuni4es to do things.
– Main capabili4es in literature • Knowledge, Skill, Ability, and Others (e.g. Loca4on, Availability)
• Reward – The benefit generated from the use of capability in both labour market and non-‐labour market ac4vi4es.
– Main rewards in literature • Reputa4on, Money, Fun, Learning, Altruism or Social Good
15
www.insight-‐centre.org
SLUA Ontology
16
Reward
Action
Capability
User Task
offers earns
includes performs
requires possesses
Location Skill Knowledge Ability Availability
Reputation Money Fun Altruism Learning
subClassOf
subClassOf
www.insight-‐centre.org
Example Task
17
hPp://www.wikipedia.org/wiki/A3_road/tasks/1
slua:Task Please consider adding full cita4ons to
the Wikipedia ar4cle
:loc1
slua:Loca4on
hPp://live.dbpedia.org/resource/London
:knw1
slua:Knowledge
hPp://live.dbpedia.org/resource/London
:rw1
slua:Reward
slua:Reputa4on
:ac1
slua:Ac4on Wiki page edit
rdfs:label
rdfs:label
a
a
a
a
a
a
slua:offers slua:includes slua:requires
slua:requires
www.insight-‐centre.org
SLUA in Ac4on
SLUA Mediated Task Routing
18
Worker Profiling
Task Modelling
Task Routing
Matching
Cold Start
Ordering
SLUA Mediated Infrastructure Services Application Interface, User Interface, Identity Management,
Notification Services
Task Model
Capability Model
Capability Models Worker
Profiless
Worker Profiles
Worker Profiles Collaborative
Data Curation
Tasks
Cyber Physical Social System
Workers
www.insight-‐centre.org
Collabora4ve Data Cura4on
19
DQ Rules & Algorithms
Entity Linking Data Fusion
Relation Extraction
Human Computation
Relevance Judgment
Data Verification Disambiguation
Clean Data Internal Community - Domain Knowledge - High Quality Responses - Trustable
Web of Data
Databases
Textual Content
Programmers Managers
External Crowd - High Availability - Large Scale - Expertise Variety
www.insight-‐centre.org
Monitored Physical Environment
Cyber-‐Physical Social Systems
20
Sensor Data Collection
Human Actuation
Energy Decision Models
Temperature of Room 202e
Close Window Room 202e
Located near Room 202e Located near Room 202e
www.insight-‐centre.org
Summary & Future Work
• SLUA is an ini4al step towards defining a light-‐weight ontology for describing tasks, acJons, users, rewards, and capabiliJes in crowdsourcing plaBorms
• Future Plans – Enumerate design with addi4onal crowdsourcing plaBorms
– Prototype SLUA-‐based for cross plaBorm query – Task rou4ng system used match between tasks and users with SLUA descriptors
21
www.insight-‐centre.org
Further Reading
Ontology Available at: hXp://vocab.deri.ie/slua U. Ul Hassan, S. O’Riain, and E. Curry, “SLUA: Towards SemanJc Linking of Users with AcJons in Crowdsourcing,” in 1st InternaJonal Workshop on Crowdsourcing the SemanJc Web, 2013. hXp://deri.ie/users/umair-‐ul-‐hassan
22
International Workshop on “Crowdsourcing the Semantic Web” Sydney, 21 October 2013