peer review & quality assurance 2.0

Peer review & Quality Assurance 2.0

Public Post Publication Peer Review

2

The beloved Myths of Peer Review

[ Pre Publication ] Peer Review:

� is all about Quality Assurance

� is the Researchers Obligation to the Research

Community

� is impartial

� needs to be anonymous to be fair & good

P4R & Quality Assurance 2.0

3

The brutal Facts of Peer Review

[ Pre Publication ] Peer Review:

� is all about Selection

� is free labor for the publisher

� increases cost of researchby slowing down publishing

� filters availability

Is a “sacred cow”


Paul Jump „Slay peer review ‘sacred cow’,

says former BMJ chief” Times Higher Ed. Apr

21, 2015; Reporting from Royal Society’s

Future of Scholarly Scientific

Communication conference

4

At the Core of it:

Selection is the publishing paradigm of the past:

� Journal based (cascading selection)

� Subscription based ( branding )

� Neglects the incrementalcharacter of research

� Impedes communication of research results (e.g.: negative results, …)

Based on “free” labor by the research community, as the benefitsare seized by the publishing industry


5

ScienceOpen promotes:

- Publishing must become:

- fast

- more complete

- access has to be free

- Reviews are about scientific discourse

- Research Publications have to serve a

global community


6

Research evaluation by researchers

ScienceOpen offers 2 kinds of peer review:

� Pre-publication peer review by endorsement

� Public post-publication peer review

Image Credit: Bryan Jones, Flickr, CC BY NC SA

7

Pre-Publication author-led Peer Review

� Concept developed by Jan Velterop,

publisher at Elsevier, Academic Press,

Nature and BioMedCentral.

Participated in the first Budapest Open

Access Initiative to define Open Access.

� gives researcher opportunity to

consult peers

� In roll out since fall 2015

8

Public Post-Publication Peer Review

� Editorial Check: Plagerism, basic scientific

principles, basic readability, researcher check

� Immediate publication as PDF

� After typesetting open for peer review

� Peer Reviewers must have published 5 articles

(ORCID verification)

� Anyone in the network can invite a reviewer

� Trackable CrossRef DOI for peer review reports

9

Open reports

� Peer

Reviewer

Name

� Full

comments

10

Rating

Level of importance: Is the publication of relevance for the academic community and does it provide important insights? Does the work represent a new approach or new findings in comparison with other publications in the field?

Level of validity: Is the hypothesis clearly formulated? Is the argumentation stringent? Are the data sound, well-controlled and statistically significant? Is the interpretation balanced and supported by the data? Are appropriate and state-of-the-art methods used?

Level of completeness: Do the authors reference the appropriate scholarly context? Do the authors provide or cite all information to follow their findings or argumentation? Do they cite the all relevant publications in the field?

Level of comprehensibility: Is the language correct and easy to understand for an academic in the field? Are the figures well displayed and captions properly described? Is the article systematically and logically organized?

11

Reproducibility & open ended PeerRev

Image © Fotolia Gino Santa Maria

� Peer reviewers can check data andMaterials & Methods sections, but the real test of a research papercomes when the scientificcommunity tries to build on results.

� ScienceOpen: Open-ended Peer Review – interesting commentsand real critique may come onlylater

12

... and then there was Selection ...

What aspects of scholarly journals are

most important to users?

� Topic-specific bundling

� Editorial selection

� Quality assurance

� Trust and reliability

ScienceOpen Collections provide

these functions beyond individual

publishers or journals.

13

Evaluation by selection

Image © Fotolia

ScienceOpen Collections: A new kind of editorial selection

14

Researcher-driven collections

15

Best of OA promotion: IUCr

16

In summary…

� Science needs even morepublications: negative results, all clinical trials, protocols, data papers, observations. But how to

evaluate more?

� ScienceOpen is concernedabout providing methodsfor sustainable evaluationof scientific results by andfor a global scientificcommunity.

Image credit: Kay Gaensler, Flickr, CC-BY-NC-SA

17

ScienceOpen - Vision

Aggregated Science (Publisher & Journal Independent):

� Subject based access to content

� Collections representing BEST OF

� Basic Research

� State of the Art

� Pushing the Envelope

� Completeness: Negative Results, +++

� Reviews / Versions

� Public Discourse

� Featuring: Research, Authors, Peers

& NOT Publishers, Journals, Vanity

� Providing public citation & social communication index

18

ScienceOpen as of last night

Image Credit: Bryan Jones, Flickr, CC BY NC SA

Aggregated Articles: 10,864,144

Authors (of these): 6,071,960

Collections: 23

19

Thank You !

Tibor TschekeChief Strategy Officer

ScienceOpen, Inc.Burlington, MA 01803

twitter: @tigracc

skype: tibor_tscheke

tel.: +1-781-222-5200

[email protected]