nicole nogoy: community annotation in the gigacuration challenge

Community Annotation in the Giga-Curation

ChallengeAnnotating All Knowledge Coalition Face-to-Face Kick Off

MeetingApril 17, 2016

Nicole NogoyCommissioning Editor, GigaScience

Nicole@gigasciencejournal.com

Why Community Curation?• Curation and annotation – under appreciated parts of scholarship• Extra information for efficient and easy re-use

• Improve curation quantity and utility by accurate addition of essential metadata• Impossible task for one person or group• A need for easy and efficient curation:• Tons of data across multiple repositories and need to link, collate and share

peoples interpretations via• All talk and no action

The Giga-Curation Challenge 2016

• Use case – curator wanted to study how many people community curation can reach out to via Hypothes.is and iCLiKVAL and the outcome• Challenge held at ISB BioCuration2016, April 10-14th

• Attendees encouraged to annotate data in GigaDB (and beyond) via Hypothes.is and iCLiKVAL general purpose annotation tools• Monitored by the Annotometer – with live leaderboard

Code is in GitHub: https://github.com/gigascience/annotometer

Lessons Learned• Slow to get started – so encourage pre-registration next time

Lessons Learned• Slow to get started – so encourage pre-registration next time• Each resource needs an introduction • People seem shy about sharing annotations publicly • There is a need for a dedicated time slot to give everyone a chance to

use the tools.

Feedback• People did not understand what was expected of them• Next time- get a bigger time slot for introductions

• The goal was too vague• Next time- give specific tasks to get them started• And provide a better target (e.g. make X annotations and get a T-shirt)

• A barrier to joining for some was the fact they had to register for 3 separate accounts• Next time- try to make it a single registration to create all accounts

• Hold shorter competitions within the main one (one day competitions)

Results• Number of Annotometer registered users = 21 (~10% of attendees) • iCLikVAL = 8 reg. users made annotations• Hypothes.is = 10 reg. users made annotations

• Total number of annotations made = 1201

• Cost of annotations = cost of prize/ No annotations = $0.41 USD per annotation

And the winner is…..

Show of Hands• If we run it again next year (with actions from all the lessons we’ve

learned this year), who would join in?

• If any other tools want to join in (PhenPackets?), get in touch.

• If you want to run your own challenge using the Annotometer, get in touch. (its all open-source:https://github.com/gigascience/annotometer)

For feedback - please email, Chris: chris@gigasciencejournal.com

Thanks – to everyone that took part and HUGE thank you to the conference organisers

the Annotometer Hypothes.is iCLiKVAL

Jesse Xiao Maryann Martone

And thanks to

Todd TaylorNaveen Kumar