improving research indicators using crowdsourcing ivan nevolin, cemi ras 11 may 2015
TRANSCRIPT
![Page 1: Improving Research Indicators using Crowdsourcing Ivan Nevolin, CEMI RAS 11 May 2015](https://reader035.vdocuments.net/reader035/viewer/2022062718/56649e7c5503460f94b7ebe4/html5/thumbnails/1.jpg)
Improving Research Indicators using Crowdsourcing
Ivan Nevolin, CEMI RAS11 May 2015
![Page 2: Improving Research Indicators using Crowdsourcing Ivan Nevolin, CEMI RAS 11 May 2015](https://reader035.vdocuments.net/reader035/viewer/2022062718/56649e7c5503460f94b7ebe4/html5/thumbnails/2.jpg)
Sources of information in Russia
Report
Scopus
Web of Science
RSCI
![Page 3: Improving Research Indicators using Crowdsourcing Ivan Nevolin, CEMI RAS 11 May 2015](https://reader035.vdocuments.net/reader035/viewer/2022062718/56649e7c5503460f94b7ebe4/html5/thumbnails/3.jpg)
eLIBRARY – Platform for RSCI
![Page 4: Improving Research Indicators using Crowdsourcing Ivan Nevolin, CEMI RAS 11 May 2015](https://reader035.vdocuments.net/reader035/viewer/2022062718/56649e7c5503460f94b7ebe4/html5/thumbnails/4.jpg)
Indicators from eLIBRARY
• Researcher– Number of publications– Number of citations– Collaboration and self-
citation– h-index– Types of publications– Types of citation– Research fields
• Organization– Number of publications– Number of citations– Number of authors– h-index– g-index– i-index– Types of publications– Indicators per author or
per publication
![Page 5: Improving Research Indicators using Crowdsourcing Ivan Nevolin, CEMI RAS 11 May 2015](https://reader035.vdocuments.net/reader035/viewer/2022062718/56649e7c5503460f94b7ebe4/html5/thumbnails/5.jpg)
Misprints and Errors
• Misprints in title and abstract• Incorrect citation• Different spelling of names and
organizations
![Page 6: Improving Research Indicators using Crowdsourcing Ivan Nevolin, CEMI RAS 11 May 2015](https://reader035.vdocuments.net/reader035/viewer/2022062718/56649e7c5503460f94b7ebe4/html5/thumbnails/6.jpg)
Author manages his Profile
![Page 7: Improving Research Indicators using Crowdsourcing Ivan Nevolin, CEMI RAS 11 May 2015](https://reader035.vdocuments.net/reader035/viewer/2022062718/56649e7c5503460f94b7ebe4/html5/thumbnails/7.jpg)
Organization manages its Profile
![Page 8: Improving Research Indicators using Crowdsourcing Ivan Nevolin, CEMI RAS 11 May 2015](https://reader035.vdocuments.net/reader035/viewer/2022062718/56649e7c5503460f94b7ebe4/html5/thumbnails/8.jpg)
Manual Review is a Problem
Submit Moderate Update
800 organizations and 93 000 authors edit the content
![Page 9: Improving Research Indicators using Crowdsourcing Ivan Nevolin, CEMI RAS 11 May 2015](https://reader035.vdocuments.net/reader035/viewer/2022062718/56649e7c5503460f94b7ebe4/html5/thumbnails/9.jpg)
Crowdsourcing as a solution
• Involve - select job easy for people, hard for computer.• Make rules for job to be approved.
Image source: Hirth et al (2011)
![Page 10: Improving Research Indicators using Crowdsourcing Ivan Nevolin, CEMI RAS 11 May 2015](https://reader035.vdocuments.net/reader035/viewer/2022062718/56649e7c5503460f94b7ebe4/html5/thumbnails/10.jpg)
Motivation
Get article for 6 Euros
Suggest scores for the access to the paid services
![Page 11: Improving Research Indicators using Crowdsourcing Ivan Nevolin, CEMI RAS 11 May 2015](https://reader035.vdocuments.net/reader035/viewer/2022062718/56649e7c5503460f94b7ebe4/html5/thumbnails/11.jpg)
Quality Control
• Majority decision
• Golden standard
Image source: Hirth et al (2011)
![Page 12: Improving Research Indicators using Crowdsourcing Ivan Nevolin, CEMI RAS 11 May 2015](https://reader035.vdocuments.net/reader035/viewer/2022062718/56649e7c5503460f94b7ebe4/html5/thumbnails/12.jpg)
Conclusion
• Users support valuation process and culture of e-science
• Provide clear rules and manage the activity• All the instruments are at hand• Let’s level-up the users’ experience from our
systems!
![Page 13: Improving Research Indicators using Crowdsourcing Ivan Nevolin, CEMI RAS 11 May 2015](https://reader035.vdocuments.net/reader035/viewer/2022062718/56649e7c5503460f94b7ebe4/html5/thumbnails/13.jpg)
Ivan NevolinResearcher at CEMI [email protected]