nicole nogoy: community annotation in the gigacuration challenge
TRANSCRIPT
Community Annotation in the Giga-Curation
ChallengeAnnotating All Knowledge Coalition Face-to-Face Kick Off
MeetingApril 17, 2016
Nicole NogoyCommissioning Editor, GigaScience
Why Community Curation?• Curation and annotation – under appreciated parts of scholarship• Extra information for efficient and easy re-use
• Improve curation quantity and utility by accurate addition of essential metadata• Impossible task for one person or group• A need for easy and efficient curation:• Tons of data across multiple repositories and need to link, collate and share
peoples interpretations via• All talk and no action
The Giga-Curation Challenge 2016
• Use case – curator wanted to study how many people community curation can reach out to via Hypothes.is and iCLiKVAL and the outcome• Challenge held at ISB BioCuration2016, April 10-14th
• Attendees encouraged to annotate data in GigaDB (and beyond) via Hypothes.is and iCLiKVAL general purpose annotation tools• Monitored by the Annotometer – with live leaderboard
Code is in GitHub: https://github.com/gigascience/annotometer
Lessons Learned• Slow to get started – so encourage pre-registration next time
Lessons Learned• Slow to get started – so encourage pre-registration next time• Each resource needs an introduction • People seem shy about sharing annotations publicly • There is a need for a dedicated time slot to give everyone a chance to
use the tools.
Feedback• People did not understand what was expected of them• Next time- get a bigger time slot for introductions
• The goal was too vague• Next time- give specific tasks to get them started• And provide a better target (e.g. make X annotations and get a T-shirt)
• A barrier to joining for some was the fact they had to register for 3 separate accounts• Next time- try to make it a single registration to create all accounts
• Hold shorter competitions within the main one (one day competitions)
Results• Number of Annotometer registered users = 21 (~10% of attendees) • iCLikVAL = 8 reg. users made annotations• Hypothes.is = 10 reg. users made annotations
• Total number of annotations made = 1201
• Cost of annotations = cost of prize/ No annotations = $0.41 USD per annotation
And the winner is…..
Show of Hands• If we run it again next year (with actions from all the lessons we’ve
learned this year), who would join in?
• If any other tools want to join in (PhenPackets?), get in touch.
• If you want to run your own challenge using the Annotometer, get in touch. (its all open-source:https://github.com/gigascience/annotometer)
For feedback - please email, Chris: [email protected]
Thanks – to everyone that took part and HUGE thank you to the conference organisers
the Annotometer Hypothes.is iCLiKVAL
Jesse Xiao Maryann Martone
And thanks to
Todd TaylorNaveen Kumar