SISOB CONCEPTUAL MODEL

Richard WalkerMay 30, 2011

Overview

• Goals• General approach• Entities• Operationalizing model• D2.2 Table of Contents

Goals

• Common vocabulary and approach• Homogeneous approach to case studies

General approach

• Inspired by computer science• Abstract classes for networks, actors and

outcomes• Each class has attributes• Real actors in case studies are instances of

abstract classes• Measurements give value to attributes

Entities

• Production– Actors– Networks– Context

• Distribution– Actors– Networks– Context

• Consumption– Actors– Networks– Context

• Outcomes– Scientific– Economic– Social

Operationalizing model• What are the questions I want to ask?• What are the operational entities relevant to my model?

– Example• Who are the production actors?• What are the production networks?

• What are my data sources?– Do I already have access to the data I need?– Do I need crawling / data from other partners

• How do I characterize my entities using my data sources?– Example

• What measurements do I use to characterize networks

D2.2 Table of Contents• 1 Objectives and structure of this document• 2 The Impact of science on society (Frontiers with contributions from all partners)• 3 The SISOB conceptual mode• 3.1 Goals of the model (Frontiers with input from all partner)• 3.2 Overview• 3.3 Entities in the model (Frontiers UDE and ELTE for measurements and UM for tools, all partners)• 4 Operationalizing the model – Researcher Mobility (Unito)• 4.1 Background• 4.2 Goals and hypotheses of the case study• 4.3 The model – an overview• 4.4 Model entities• 5 Operationalizing the model – Knowledge Sharing (UDE)• 5.1 Background• 5.2 Goals and hypotheses of the case study• 5.3 The model – an overview• 5.4 Model entities• 6 Operationalizing the model – Literature review (Frontiers)• 6.1 Background• 6.2 Goals and hypotheses of the case study• 6.3 The model – an overview• 6.4 Model entities• 6.4.1 Appendix 1: Requirements on SISOB tools• Summary of required measurements• Summary of required measurement tools• REFERENCES• Appendix A: Common Network Indicators

PEER REVIEW – DATA REQUIREMENTS

Richard Walker

Company Logo

Overview

• Scientific questions• Operationalization of conceptual model• Sample hypotheses• Data sources required

Scientific questions• How does peer review affect impact of science• “Traditional” issues

– Cognitive biases– Traditional cronyism– “Cognitive cronyism”

• “Social” issues– How do relationships among reviewers affect review process?– How do relationships among reviewers and authors affect the

review process?• “New” issue

– How do new models of review affect review process?

Operationalizing the conceptual modelConceptual model Operational model

Production actors Authors

Production network Coauthoring networkShared institution network

Distribution actors Reviewers

Distribution network Co-reviewer networkReviewer-editor networksAuthor-reviewer-editor networks

Artifacts Publications

Outcomes Acceptance of publication

Post-publication review by readers

Academic citations

Non-academic citations

Productivity (individual, community)

Sample hypotheses

• Traditional– Papers with a woman as a first author are more likely to

be accepted review committee includes a woman• Social– Papers are more likely to be accepted if authors are

“close” to reviewers in author-reviewer network (cognitive cronyism)

• Different techniques of reviewing– Open reviewing (Frontiers) is less affected by bias x than

traditional reviewing

Data sets required/1• Frontiers papers

– Attributes of papers, authors, reviewers• Source: Frontiers• Use: reconstruct author and reviewer networks

– All papers by Frontiers authors and reviewers (last 10 years) • Source: crawling• Use: enhance author and reviewer networks

– All citations of papers in Frontiers data set• Source: crawling• Use: outcome measurement

– Productivity of authors and reviewers• Measirement: number, citations, outside references• Source: crawling• Use: outcome measurement

– Non-academic citations of papers in Frontiers data set• Source: crawling• Use: outcome measurement

Data sets required• Conference papers (UMA)

– Attributes of papers, authors, reviewers• Source: ??• Use: reconstruct author and reviewer networks

– All papers by authors and reviewers (last 10 years) • Source: crawling• Use: enhance author and reviewer networks

– All citations of papers in data set• Source: crawling• Use: outcome measurement

– Productivity of authors and reviewers• Measirement: number, citations, outside references• Source: crawling• Use: outcome measurement

– Non-academic citations of papers in data set• Source: crawling• Use: outcome measurement

Open issues

• Availability of data sources• Network indicators• BIG ISSUE 1 – how do we make this useful for

policy makers?• BIG ISSUE 2 – how do look this in a

community perspective