interdisciplinary research collaborations: evaluation of a funding program
DESCRIPTION
Rons, N., PRESENTATION at the Sixth International Conference on Webometrics, Informetrics and Scientometrics (WIS) and Eleventh COLLNET Meeting. University of Mysore, Mysore, India, 19-22 October, 2010TRANSCRIPT
17/05/2006 1 Herhaling titel van presentatie
Interdisciplinary Research Collaborations: Evaluation of a Funding Program
Nadine Rons Vrije Universiteit Brussel (VUB)
Sixth International Conference on Webometrics, Informetrics and Scientometrics (WIS) & Eleventh COLLNET Meeting, 19-22 October 2010, Mysore, India Version 13.10.2010
October 2010 - Slide 2 6th WIS & 11th COLLNET
Structure
o Place in the broader context of INTERDISCIPLINARITY o Origins & scope o Interdisciplinarity in research, studied bibliometrically o Focus: synergies between researchers from different disciplines o Research policy: search for best practices
o Experiences from a FUNDING PROGRAM o Program presentation: procedures & criteria o Program evaluation: network composition, output & impact, feedback o Conclusions
October 2010 - Slide 3 6th WIS & 11th COLLNET
Origins of interdisciplinarity
o Human KNOWLEDGE divided in disciplines o categorization of scientific knowledge, subject to change o division in more specialized communities, focusing efforts in a
growing context of information produced and communicated o if institutionalized: shaping knowledge production & distribution o in their present form: fairly recent (±200 years), relatively stable,
instrument for science studies
o Need for CROSSING THE BOUNDARIES o the ‘interdisciplinary’ nature of reality o political & socio-economic priority areas (nanotechnology, climate
research), ‘transdisciplinary’ step towards application o opportunities for progress through cross fertilization
October 2010 - Slide 4 6th WIS & 11th COLLNET
Kinds of interdisciplinarity
October 2010 - Slide 5 6th WIS & 11th COLLNET
Bibliometric studies of interdisciplinarity
o Research publication DATABASES o Web of Science: created to “bring together material that would never
be collated by the usual subject indexing” in “an association-of-ideas index” (Garfield, Science 122, 1955)
o Scopus
o Examples of empirical STUDIES o Cross-disciplinary citations (Porter and Chubin, Scientometrics 8, 1985)
o Larger time lag (Rinia et al., Scientometrics 51, 2001)
o ‘Natural’ shares of cross-disciplinary activity (Rinia et al., Scientometrics 54, 2002)
o Evolution over time (Porter and Rafols, Scientometrics 81, 2009)
o Measures for integration and specialization (Porter et al., Scientometrics 72, 2007)
Bordons et al. Analysis of cross-disciplinary research through bibliometric tools. In: Moed et al. (Eds.), Handbook of quantitative science and technology research, pp. 437-456, 2004
October 2010 - Slide 6 6th WIS & 11th COLLNET
Focus in current research policy
o FUNDING programs / policies for interdisciplinary research o ‘Top down’, thematic o Open call, reserved fraction
o ASSESSMENT of interdisciplinary research o Dedicated issue of Research Evaluation (Research Evaluation 15, 2006)
o Heterogeneity
o No generally accepted methodology yet o Search for best practices o Crucial: composition and functioning of the evaluation committee
o VUB: HORIZONTAL RESEARCH ACTIONS (HOA) program o Set up in 2002, evaluated in 2009
October 2010 - Slide 7 6th WIS & 11th COLLNET
HOA program focus
o INTERDISCIPLINARY RESEARCH
o Integrating expertise from different disciplines o Offering an all-encompassing expertise in a theme proposed
‘bottom up’
o COLLABORATIONS
o Between researchers from different departments / faculties o In the framework of a research project o Stimulated / supported by a dedicated funding program
October 2010 - Slide 8 6th WIS & 11th COLLNET
HOA call & selection phases
o CALL for applications o Applications incl. theme & its relevance, applicants & teams, project
description & budget, external collaborations
o Selection COMMITTEE o Board of the Research Council o Broad scope, open attitude to different standards, coherence, good
overview of expertise and performances, views expected to surpass disciplinary perspectives
o PRE-SELECTION and SELECTION o Pre-selection: advice from two experienced reviewers per
application (of which 1 member of the Board) o Final selection: after oral presentation before the Board
October 2010 - Slide 9 6th WIS & 11th COLLNET
HOA criteria
o TOPIC o Interdisciplinary and innovative character, added value, importance for
science and society
o PARTNERS o Separate strengths in disciplinary performance o All-encompassing expertise in the proposed theme
o PROJECT and PLANNING o Planned function of the researcher(s) financed, embodying the
integration of expertise o Planned international network participations
≈ How panel members define a good proposal (Lamont et al., Research Evaluation 15, 2006)
≈ Grounds for evaluation suggested by researchers (Feller, Research Evaluation 15, 2006)
October 2010 - Slide 10 6th WIS & 11th COLLNET
Program evaluation
o Analysis of APPLICATION DATA o 7 generations: (Co)applicants, affiliations to teams & faculties o -> Degree of interdisciplinarity, faculty participation
o BIBLIOMETRIC study o First 4 generations: Co-publications & co-citations in Web of Science o -> Measures for output & impact and evolution over time *
o SURVEY o Funded applications followed for 4 full years (full project length) o Validity of data & indicators used, opinions
* Not used in this context: interdisciplinarity of references, breath of journal subject categories
October 2010 - Slide 11 6th WIS & 11th COLLNET
Faculty participation
o ALL FACULTIES are represented among the applicants
Faculties: MEdicine and pharmacy, SCience and bio-engineering sciences, ENgineering sciences, ARts and philosophy, EConomics, social and political sciences, LAw and criminology, PHysical education and physiotherapy, PSychology and educational science; CA: Central academic; linear correlation coefficient r=0,910 for N=9 (i.e. 8 faculties + central academic staff).
October 2010 - Slide 12 6th WIS & 11th COLLNET
Degree of interdisciplinarity
o A majority of BROAD INTERDISCIPLINARITY o The total sample of applications links each of the 8 faculties to at
least 5 others. o Degree of interdisciplinarity: joining applicants from different faculty
clusters* (broad), from different faculties within one cluster (medium), and from different departments within one faculty (narrow). [* ‘Social Sciences and Humanities’, ‘Basic, Natural and Applied Sciences’ or ‘Biomedical Sciences’]
October 2010 - Slide 13 6th WIS & 11th COLLNET
Output & impact
o BIBLIOMETRIC interdisciplinary OUTPUT & IMPACT o Co-publications: joint publications by applicants from different
departments o Co-citations: publications citing applicants from different departments
A: "Newly activated", B: Previously active", C: "Output only", D: "Citation impact only", E: "Not visible".
See also: Added value of funding visible after ±3 years (Rons, Proceedings 11th Conference on STI, 2010)
October 2010 - Slide 14 6th WIS & 11th COLLNET
Feedback & opinions
o Confirmed interdisciplinary SYNERGIES o The large majority (82%) of co-publications concern a synergy (as
opposed to an application of expertise from one field in another)
o Confirmed OUTPUT & IMPACT representation o A majority of co-publications (89%) and co-citations (64%) is related to
the network topic and contain the majority of related output and impact of various forms (next to 30% other output items and 11% other impact items reported: contributions in proceedings, Dutch language journal articles and books for professional audience, PhD’s obtained and in preparation, obtained research funding)
o Confirmed SPECIFIC NEEDS o Stimulating and necessary funding channel o Longer startup periods due to multiple differences in traditions
October 2010 - Slide 15 6th WIS & 11th COLLNET
Program Evaluation: conclusions
o DEGREE OF INTERDISCIPLINARITY and FACULTY PARTICIPATION o The program attracts a potential present in all faculties for -
mainly ‘broad’ - interdisciplinary collaborations
o OUTPUT & IMPACT o The program successfully selects & supports interdisciplinary
research synergies, and activates funded as well as unfunded networks
o FEEDBACK & OPINIONS o The program stimulates interdisciplinary collaborations and responds
to a need for a dedicated funding channel
Presentation of the REPORT to the Research Council -> Program continued
October 2010 - Slide 16 6th WIS & 11th COLLNET
Possibilities for future studies
Bibliometric characteristics of interdisciplinary research synergies
Feasibility of normalization in interdisciplinary domains
Indicators for ex-post evaluation of interdisciplinary research
October 2010 - Slide 17 6th WIS & 11th COLLNET
THANK YOU FOR YOUR ATTENTION!
Nadine Rons
Vrije Universiteit Brussel (VUB) VUB R&D Dept, Research Coordination Unit Pleinlaan 2, B-1050 Brussels, Belgium
[email protected] http://rd-ir.vub.ac.be/en_GB/people/show/id/554
http://www.vub.ac.be/studyinginbrussels/index.html