insemtives semtech2010-20100622
DESCRIPTION
TRANSCRIPT
![Page 1: Insemtives semtech2010-20100622](https://reader034.vdocuments.net/reader034/viewer/2022051816/546f43ccb4af9f220c8b4655/html5/thumbnails/1.jpg)
Reputation, community, Web 2.0, and games: incentives for
semantics – about getting people involved
www.insemtives.eu 1
Katharina Siorpaes and Elena SimperlSemTech 2010
![Page 2: Insemtives semtech2010-20100622](https://reader034.vdocuments.net/reader034/viewer/2022051816/546f43ccb4af9f220c8b4655/html5/thumbnails/2.jpg)
Executive summary• Many tasks related to semantic content authoring cannot be undertaken
without human contribution.
• User motivation is essential for semantic applications to achieve critical mass and ensure sustainable growth.
• How to encourage user participation– Incentives– Technology design– Usability engineering– Games with a purpose
www.insemtives.eu 2
![Page 3: Insemtives semtech2010-20100622](https://reader034.vdocuments.net/reader034/viewer/2022051816/546f43ccb4af9f220c8b4655/html5/thumbnails/3.jpg)
About the speakers
www.insemtives.eu 3
Katharina Siorpaes Elena Simperl
![Page 4: Insemtives semtech2010-20100622](https://reader034.vdocuments.net/reader034/viewer/2022051816/546f43ccb4af9f220c8b4655/html5/thumbnails/4.jpg)
Agenda
• Human-driven semantic content authoring• Incentives and participatory design• Example: Casual games• Example: Virtual worlds• Conclusion
www.insemtives.eu 4
![Page 5: Insemtives semtech2010-20100622](https://reader034.vdocuments.net/reader034/viewer/2022051816/546f43ccb4af9f220c8b4655/html5/thumbnails/5.jpg)
HUMAN-DRIVEN SEMANTICCONTENT AUTHORING
Human-driven semantic content authoringIncentives and participatory design
Example: Casual gamesExample: Virtual worlds
Conclusions
www.insemtives.eu 5
![Page 6: Insemtives semtech2010-20100622](https://reader034.vdocuments.net/reader034/viewer/2022051816/546f43ccb4af9f220c8b4655/html5/thumbnails/6.jpg)
Human vs computational intelligence• Ordered sequence of tasks into which the
authoring exercise can be divided• Required skills and expertise
www.insemtives.eu 6
![Page 7: Insemtives semtech2010-20100622](https://reader034.vdocuments.net/reader034/viewer/2022051816/546f43ccb4af9f220c8b4655/html5/thumbnails/7.jpg)
Example: semantic annotation
www.insemtives.eu 7
![Page 8: Insemtives semtech2010-20100622](https://reader034.vdocuments.net/reader034/viewer/2022051816/546f43ccb4af9f220c8b4655/html5/thumbnails/8.jpg)
Example: ontology evaluation
www.insemtives.eu 8
![Page 9: Insemtives semtech2010-20100622](https://reader034.vdocuments.net/reader034/viewer/2022051816/546f43ccb4af9f220c8b4655/html5/thumbnails/9.jpg)
Example: ontology alignment
www.insemtives.eu 9
![Page 10: Insemtives semtech2010-20100622](https://reader034.vdocuments.net/reader034/viewer/2022051816/546f43ccb4af9f220c8b4655/html5/thumbnails/10.jpg)
MILLION DOLLAR QUESTION
HOW CAN WE MOTIVATE PEOPLE TO DEDICATE THEIR VALUABLE TIME
TO ANNOTATE THEIR/OUR DATA?
www.insemtives.eu 10
![Page 11: Insemtives semtech2010-20100622](https://reader034.vdocuments.net/reader034/viewer/2022051816/546f43ccb4af9f220c8b4655/html5/thumbnails/11.jpg)
INCENTIVES AND PARTICIPATORY DESIGN
Human-driven semantic content authoringIncentives and participatory design
Example: Casual gamesExample: Virtual worlds
Conclusions
www.insemtives.eu 11
![Page 12: Insemtives semtech2010-20100622](https://reader034.vdocuments.net/reader034/viewer/2022051816/546f43ccb4af9f220c8b4655/html5/thumbnails/12.jpg)
www.insemtives.eu 12
Motivation vs incentives
• Incentives are ‘rewards’ assigned by an external‘judge’ to a performer for undertaking a specific task.
• Common belief (among economists): incentives can be translated into a sum of money for all practical purposes.
• Incentives can be related to both extrinsic and intrinsic motivations.
![Page 13: Insemtives semtech2010-20100622](https://reader034.vdocuments.net/reader034/viewer/2022051816/546f43ccb4af9f220c8b4655/html5/thumbnails/13.jpg)
www.insemtives.eu 13
Extrinsic vs intrinsic motivation
• Extrinsic motivation if task is considered– Boring, dangerous, useless, socially undesirable,
dislikable by the performer.• Intrinsic motivation if
– The performer likes what he/she is doing– The act is satisfying in itself (it can happen for
many different reasons).
![Page 14: Insemtives semtech2010-20100622](https://reader034.vdocuments.net/reader034/viewer/2022051816/546f43ccb4af9f220c8b4655/html5/thumbnails/14.jpg)
Web 2.0 is hot, Semantic Web isnot?
www.insemtives.eu 14
![Page 15: Insemtives semtech2010-20100622](https://reader034.vdocuments.net/reader034/viewer/2022051816/546f43ccb4af9f220c8b4655/html5/thumbnails/15.jpg)
Web 2.0 motivators
www.insemtives.eu 15
![Page 16: Insemtives semtech2010-20100622](https://reader034.vdocuments.net/reader034/viewer/2022051816/546f43ccb4af9f220c8b4655/html5/thumbnails/16.jpg)
User-empowered Web
• Comprehensive incentives studies are available (Kuznetsov, 2004; Marlow et al.,2006; Wikipedia, 2009).
www.insemtives.eu 1616
![Page 17: Insemtives semtech2010-20100622](https://reader034.vdocuments.net/reader034/viewer/2022051816/546f43ccb4af9f220c8b4655/html5/thumbnails/17.jpg)
Example: Wikipedians
• Reciprocity: Altruistic contributors receive a benefit in return.
• Community: “Wikipedians […] feel needed”, there is “a sense of common purpose and belonging “.
• Reputation: Contributors “develop identities in order to be respected, trusted, and appreciated by peers”.
• Autonomy: Contributors enjoy “the freedom of independent decision”.
Kuznetsov, 2004
www.insemtives.eu 17
![Page 18: Insemtives semtech2010-20100622](https://reader034.vdocuments.net/reader034/viewer/2022051816/546f43ccb4af9f220c8b4655/html5/thumbnails/18.jpg)
Factors influencing incentives design
• Tasks or an ordered collection of tasks into which the authoring exercise can be divided.
• Skills required to undertake the task.
• Goal of the authoring exercise.• Social structure, i.e. social relationships
among the subjects participating in the exercise.
• Nature of the good.www.insemtives.eu 18
![Page 19: Insemtives semtech2010-20100622](https://reader034.vdocuments.net/reader034/viewer/2022051816/546f43ccb4af9f220c8b4655/html5/thumbnails/19.jpg)
Example: image sharing on a social platform
• Task: attaching concepts and relationships to images (iPhoto).
• Skills: identifying objects in images and choosing the right description.
• Goal: sharing and retrieval. • Social structure: Friends (in
social network).• Nature of the good:
annotations accessible to me and my friends.
www.insemtives.eu 19
![Page 20: Insemtives semtech2010-20100622](https://reader034.vdocuments.net/reader034/viewer/2022051816/546f43ccb4af9f220c8b4655/html5/thumbnails/20.jpg)
Example: UNSPSC and eCl@ssalignment
• Task: ontology alignment• Skills: domain knowledge,
modeling skills• Goal: interoperability • Social structure:
enterprise, team• Nature of good: for
benefit of enterprise
www.insemtives.eu 20
![Page 21: Insemtives semtech2010-20100622](https://reader034.vdocuments.net/reader034/viewer/2022051816/546f43ccb4af9f220c8b4655/html5/thumbnails/21.jpg)
Technology and application design
• Involve users in the design process. – Design workshops, interviews etc.
• Take usability serious. – Design for efficiency, effectiveness and user
satisfaction.
• Design for sociability. – Support sharing, contributing, collaborating,
identity-building.
www.insemtives.eu 21
![Page 22: Insemtives semtech2010-20100622](https://reader034.vdocuments.net/reader034/viewer/2022051816/546f43ccb4af9f220c8b4655/html5/thumbnails/22.jpg)
Technology and application design (ii)
• Respect trust/security and safety/privacy issues. – Support visibility and awareness.
• Design for fun. • Follow an user experience design approach.• Support a user’s individual identity in a (and
contribution to a) community with collaborative activities.
www.insemtives.eu 22
![Page 23: Insemtives semtech2010-20100622](https://reader034.vdocuments.net/reader034/viewer/2022051816/546f43ccb4af9f220c8b4655/html5/thumbnails/23.jpg)
Example: image annotation in OKEnterprise (i)
• Involvement of users in design process– Interviews and user
workshops• Usability
– Interviews, usability evaluations
• Design for sociability– User interaction, visibility of
contributions, registered users, collaborative working
• Trust and privacy– Trust in enterprise portal,
rating– Privacy: users have control
over their data
www.insemtives.eu 23
![Page 24: Insemtives semtech2010-20100622](https://reader034.vdocuments.net/reader034/viewer/2022051816/546f43ccb4af9f220c8b4655/html5/thumbnails/24.jpg)
Example: image annotation in OKEnterprise (ii)
• Design for fun– Annotator of the week,
rankings– Casual games
• User-experience design approach– Evaluate and revise
depending on user experience
• Support user’s individual identity– Registration of users,
individual user pages
www.insemtives.eu 24
![Page 25: Insemtives semtech2010-20100622](https://reader034.vdocuments.net/reader034/viewer/2022051816/546f43ccb4af9f220c8b4655/html5/thumbnails/25.jpg)
NOW: FUN AND COMPETITION: GAMES FOR SEMANTICS!
Human-driven semantic content authoringIncentives and participatory design
Example: Casual gamesExample: Virtual worlds
Conclusions
www.insemtives.eu 25
![Page 26: Insemtives semtech2010-20100622](https://reader034.vdocuments.net/reader034/viewer/2022051816/546f43ccb4af9f220c8b4655/html5/thumbnails/26.jpg)
8/10/2011ESWC 2008 26
OntoPronto: Step 1
![Page 27: Insemtives semtech2010-20100622](https://reader034.vdocuments.net/reader034/viewer/2022051816/546f43ccb4af9f220c8b4655/html5/thumbnails/27.jpg)
OntoPronto: Step 2
![Page 28: Insemtives semtech2010-20100622](https://reader034.vdocuments.net/reader034/viewer/2022051816/546f43ccb4af9f220c8b4655/html5/thumbnails/28.jpg)
8/10/2011ESWC 2008 28
OntoTube
![Page 29: Insemtives semtech2010-20100622](https://reader034.vdocuments.net/reader034/viewer/2022051816/546f43ccb4af9f220c8b4655/html5/thumbnails/29.jpg)
OntoTube
![Page 30: Insemtives semtech2010-20100622](https://reader034.vdocuments.net/reader034/viewer/2022051816/546f43ccb4af9f220c8b4655/html5/thumbnails/30.jpg)
Results
n=2905 rounds
n=2234
12.9%
10.2%
76.9%
Either skipped or noconsensus found inthe first task
Challenges in whichonly the first task wascompletedconsensually
Challenges in whichboth tasks werecompletedconsensually
97.6%
0.5%
1.9%
Challenges in whichat least the first taskwas completedconsensuallyWrong judgements ofontological nature
Wrong abstractions
![Page 31: Insemtives semtech2010-20100622](https://reader034.vdocuments.net/reader034/viewer/2022051816/546f43ccb4af9f220c8b4655/html5/thumbnails/31.jpg)
MASSACRE – MASive Semantic Annotation Creation Game
www.insemtives.eu 31
![Page 32: Insemtives semtech2010-20100622](https://reader034.vdocuments.net/reader034/viewer/2022051816/546f43ccb4af9f220c8b4655/html5/thumbnails/32.jpg)
Phratris – Phrase Analysis TETRIS
www.insemtives.eu 32
![Page 33: Insemtives semtech2010-20100622](https://reader034.vdocuments.net/reader034/viewer/2022051816/546f43ccb4af9f220c8b4655/html5/thumbnails/33.jpg)
playence’ Uhani (i)
www.insemtives.eu 33
Check www.playence.com!
![Page 34: Insemtives semtech2010-20100622](https://reader034.vdocuments.net/reader034/viewer/2022051816/546f43ccb4af9f220c8b4655/html5/thumbnails/34.jpg)
playence’ Uhani (ii)
www.insemtives.eu 34
![Page 35: Insemtives semtech2010-20100622](https://reader034.vdocuments.net/reader034/viewer/2022051816/546f43ccb4af9f220c8b4655/html5/thumbnails/35.jpg)
Casual games
1. Steep learning curve2. Fast game play: little time effort required 3. Simple implementation (simple interface and
graphics)4. Low hardware efforts (usually browser or
mobile app)5. Low bandwidth requirements6. Mass audience
www.insemtives.eu 35
![Page 36: Insemtives semtech2010-20100622](https://reader034.vdocuments.net/reader034/viewer/2022051816/546f43ccb4af9f220c8b4655/html5/thumbnails/36.jpg)
How to design your own game
1. Specify output2. Identify input3. Choose type of game and define game play4. Based on previous decisions, define game
play and adapt underlying game ontology5. Adapt or define export algorithm6. Evaluate output
www.insemtives.eu 36
![Page 37: Insemtives semtech2010-20100622](https://reader034.vdocuments.net/reader034/viewer/2022051816/546f43ccb4af9f220c8b4655/html5/thumbnails/37.jpg)
Guidelines
1. Timed response2. Score keeping3. Player skill level4. High score lists5. Randomness
5. Random player pairing6. Player testing7. Repetition8. Taboo outputs
www.insemtives.eu 37
Luis von Ahn. Games With A Purpose. IEEE Computer Magazine, June 2006. pp 96-98.
![Page 38: Insemtives semtech2010-20100622](https://reader034.vdocuments.net/reader034/viewer/2022051816/546f43ccb4af9f220c8b4655/html5/thumbnails/38.jpg)
Challenges• Identifying suitable tasks in semantic content creation.• Designing games. • Designing a usable, attractive interface. • Identifying suitable knowledge corpora. • Preventing cheating. • Defusing typical pitfalls of conceptual modeling.• Distribution of labor. • Fostering user participation. • Deriving formal representations. • Scalability and performance.
www.insemtives.eu 38
![Page 39: Insemtives semtech2010-20100622](https://reader034.vdocuments.net/reader034/viewer/2022051816/546f43ccb4af9f220c8b4655/html5/thumbnails/39.jpg)
ANNOTATION IN VIRTUAL WORLDS
Human-driven semantic content authoringIncentives and participatory design
Example: Casual gamesExample: Virtual worlds
Conclusions
www.insemtives.eu 39
![Page 40: Insemtives semtech2010-20100622](https://reader034.vdocuments.net/reader034/viewer/2022051816/546f43ccb4af9f220c8b4655/html5/thumbnails/40.jpg)
Tiny Planets
• Kids TV property licensed in over 100 countries worldwide
• Education• Diverse broadcaster base – from
Al Jazeera to American Armed Forces network
• Unusual age profile – kids 4 -11• Brand extensions into web and
virtual world
www.insemtives.eu 40
![Page 41: Insemtives semtech2010-20100622](https://reader034.vdocuments.net/reader034/viewer/2022051816/546f43ccb4af9f220c8b4655/html5/thumbnails/41.jpg)
Tiny Planets (ii)
• Tiny Planets website – 40,000 unique visitors per month
• 100,000 page impressions• Average stay 8 minutes• MTP virtual world – active
47,000 user accounts• Books, Fun, TV, Learning, Labs
sites as well as virtual world• Real and virtual currencies• Cross-site account creation
www.insemtives.eu 41
![Page 42: Insemtives semtech2010-20100622](https://reader034.vdocuments.net/reader034/viewer/2022051816/546f43ccb4af9f220c8b4655/html5/thumbnails/42.jpg)
• Engage kids with annotation games as a means of expression
• Combination of ‘rating’, ‘tagging’, and pre-scripted comments
• Locate content within virtual world to inform friends, give clues in games, give an ‘emotional’ context to items by expressing opinions. Search for clues and comments left by friends
• Reward annotations with virtual currency
www.insemtives.eu 42
Virtual world
![Page 43: Insemtives semtech2010-20100622](https://reader034.vdocuments.net/reader034/viewer/2022051816/546f43ccb4af9f220c8b4655/html5/thumbnails/43.jpg)
Real world• Real-world science experiment
based on Galaxy Zoo format• Crater marking using data from
NASA LRO• Use My Tiny Planets account
for sign in and performance tracking
• Reward annotations with achievements and virtual currency
• And kudos; kids get to know they’re doing science for real, and its going to be used.
www.insemtives.eu 43
![Page 44: Insemtives semtech2010-20100622](https://reader034.vdocuments.net/reader034/viewer/2022051816/546f43ccb4af9f220c8b4655/html5/thumbnails/44.jpg)
![Page 45: Insemtives semtech2010-20100622](https://reader034.vdocuments.net/reader034/viewer/2022051816/546f43ccb4af9f220c8b4655/html5/thumbnails/45.jpg)
Incentives• Earn Stars to spend on Virtual
Goods, games or videos• Earn awards that can be seen by
other players in your passport• Rise in Rank as you perform
targeted activities • Join the Pro circuit so that your
top scores can be seen by other players
• Buy Keys, which unlock new areas and new activities in which to spend your Stars.
www.insemtives.eu 45
![Page 46: Insemtives semtech2010-20100622](https://reader034.vdocuments.net/reader034/viewer/2022051816/546f43ccb4af9f220c8b4655/html5/thumbnails/46.jpg)
![Page 47: Insemtives semtech2010-20100622](https://reader034.vdocuments.net/reader034/viewer/2022051816/546f43ccb4af9f220c8b4655/html5/thumbnails/47.jpg)
![Page 48: Insemtives semtech2010-20100622](https://reader034.vdocuments.net/reader034/viewer/2022051816/546f43ccb4af9f220c8b4655/html5/thumbnails/48.jpg)
![Page 49: Insemtives semtech2010-20100622](https://reader034.vdocuments.net/reader034/viewer/2022051816/546f43ccb4af9f220c8b4655/html5/thumbnails/49.jpg)
![Page 50: Insemtives semtech2010-20100622](https://reader034.vdocuments.net/reader034/viewer/2022051816/546f43ccb4af9f220c8b4655/html5/thumbnails/50.jpg)
CONCLUSIONS
Human-driven semantic content authoringIncentives and participatory design
Example: Casual gamesExample: Virtual worlds
Conclusions
www.insemtives.eu 50
![Page 51: Insemtives semtech2010-20100622](https://reader034.vdocuments.net/reader034/viewer/2022051816/546f43ccb4af9f220c8b4655/html5/thumbnails/51.jpg)
Summary• Human contribution is needed for selected tasks in
semantic content authoring. • Web 2.0 has impressively demonstrated how strong
incentives can foster user participation and create successful applications.
• Turning semantic content creation tasks into games is an especially intriguing idea because of the potentially large amounts of human-produced data that can be created.
• Challenges are manifold: knowledge corpora, interesting game design, ensuring uptake, difficulty of tasks, etc.
www.insemtives.eu 51
![Page 52: Insemtives semtech2010-20100622](https://reader034.vdocuments.net/reader034/viewer/2022051816/546f43ccb4af9f220c8b4655/html5/thumbnails/52.jpg)
8/10/2011 www.insemtives.eu 52
Realizing the Semantic Web by encouraging millions of end-users
to create semantic content.