Analysis of

Bibliometric indicators for European policies

2000–2013

Research and Innovation

EUROPEAN COMMISSION

Directorate-General for Research and InnovationDirectorate A — Policy Development and Coordination Unit A.4 — Analysis and monitoring of national research policies

Contact: Sylviane Troger

E-mail: Sylviane.troger@ec.europa.eu RTD-PUBLICATIONS@ec.europa.eu

European CommissionB-1049 Brussels

EUROPEAN COMMISSION

Directorate-General for Research and Innovation2015

2000–2013

This study was financed under FP7 (Capacities Programme – Support for the coherent development of research policies), tender N° 2009/S 158-229751

(Study for the analysis and regular update of bibliometric indicators)

Edited by Science-MetrixDavid Campbell, Guillaume Roberge, Andréa Ventimiglia, Isabelle Labrosse,

Christian Lefebvre, Michelle Picard-Aitken, Stéphanie Haustein, Grégoire Côté, & Éric Archambault

Analysis of bibliometric indicators for European policies

Brussels | Montreal | Washingtoninfo@science-metrix.com | www.science-metrix.com

Europe Direct is a service to help you find answers to your questions about the European Union

Freephone number (*):00 800 6 7 8 9 10 11

(*) The information given is free, as are most calls (though some operators, phone boxes or hotels may charge you).

LEGAL NOTICE

This document has been prepared by Science-Metrix for the European Commission. It reflects the views only of the authors, and the Commission cannot be held responsible for any use which may be made of the information contained therein.

More information on the European Union is available on the Internet (http://europa.eu).

Luxembourg: Publications Office of the European Union, 2015

ISBN 978-92-79-47690-7doi: 10.2777/194026

© European Union, 2015Reproduction is authorised provided the source is acknowledged.

Image © pashabo, #80541196, 2015. Source: Fotolia.com

CONTENTS

4 INTRODUCTION

5 I—CURRENTSTATEOFEUROPEANRESEARCH

5 Trends in the size and scientific impact of European Research7 Positional analysis of European Research within FP7 thematic priorities9 Scientific performance and collaboration profiles of countries, regions and organisations in the ERA11 National and regional specialisation patterns within the ERA

12 II—SCIENTIFICCOOPERATIONASAMEANSTOWARDSCREATINGTHEERA,ACHIEVINGGREATER RESEARCHIMPACTASWELLASBOOSTINGINNOVATION

12 Role of the networking approach promoted under FP7 towards realising the ERA14 Achieving greater research impact through scientific cooperation16 Boosting innovation through interdisciplinary collaboration

Introduction

From September 2010 to March 2015, Science-Metrix produced, analysed and updated a broad range of bibliometric indicators for the European Commission.

These indicators are designed to take into account national and sector specificities, as well as to allow for a comprehensive analysis of the evolution, inter-connectivity, performance and impact of national research and innovation systems in Europe. They also provide an overall view of Europe’s strengths and challenges in knowledge production across fields of science. These indicators are defined in the adjacent box.

At the start of this study, the Seventh Framework Programme for RTD (FP7) was one of the main policy instruments towards realising the European Research Area (ERA) — an open space for the free circulation of knowledge and growth. The main goals of the data provided in this study were supporting the development of research policies for the ERA and informing the development of Horizon 2020 (H2020, the successor to FP7).

The ‘Europe 2020 Strategy’ was launched in 2010 with the aim to achieve smart, sustainable and inclusive growth for the coming decade. This should be seen in the economic context, where increasing global competition is forcing the European Union Member States to improve their innovation perfor-mance. H2020 aims to provide a comprehensive set of actions for stepping up research and innovation and focusing on three priorities: excellence in science, industrial leadership and societal challenges.

More specifically, the analyses provided by Science-Metrix focused on the scientific performance and collaboration patterns of countries, regions1 and organisations (i.e. universities, Research Public Organisations [RPOs] and companies) within the ERA and beyond, as measured through an analysis of peer-reviewed scientific publications indexed in Elsevier’s Scopus database.

The geographic coverage of the ERA includes EU28 Member States (MS) as well as 13 Associated Countries (AC) (i.e. Albania, Bosnia and Herzegovina, Faroe Islands, Iceland, Israel, Liechtenstein, the former Yugoslav Republic of Macedonia, Moldova, Montenegro, Norway, Serbia, Switzerland and Turkey). International comparators were also included — namely, the United States, China, India, Japan, the Republic of Korea, Brazil and Russia.

Definitions of indicators used in the study

Average of Relative Citations (ARC): Measures the scientific impact of an entity (country, university, etc.) based on the amount of citations received by its scientific publications. A score above one indicates that the scientific ‘impact’ of an entity is above the world average. A score below one means the opposite.Average of Relative Impact Factors (ARIF): A measure of the expected scientific impact of publications produced by a given entity based on the impact factors of the journals in which they were published (measured by the journals’ citations). A score above one indicates that the research ‘quality’ of an entity is above the world average. A score below one means the opposite.Collaboration Index (CI): A measure of scientific collaboration comparing the observed number of co-publications of an entity to that expected given its total number of publications. A score above one indicates that the entity collaborates more than expected. A score below one means the opposite.Collaboration rate: The number of co-publications of an entity, divided by the entity’s total number of publications.Co-publications: The number of co-publications of an entity (at international, national or intra-university level, for example).Growth Ratio (GR): Measures the increase in the number of publications between two periods. A value above one indicates an increase, and a value below one a decrease. For the purposes of this brochure, the GR has been calculated comparing the output of the 2010 to 2013 period to that of the 2004 to 2007 period.Highly cited publications: This represents the percentage of an entity’s papers falling in the 10% most cited papers in the reference database. A score above 10% means that the entity is contributing to highly cited papers beyond what would be expected, somehow reflecting research ‘excellence’.Publications: The number of papers produced by an entity (country, institution, etc.). It is presented both in full [FULL] and fractional [FRAC] counting. FULL refers to the method of assigning one publication count to a given entity each time this entity appears on a publication, and FRAC refers to the method of assigning only a fraction of a paper to a given entity when this paper involves more than one author.Specialisation Index (SI): Measures the research intensity of an entity in a given research area, relative to the world. A score above one indicates specialisation in this area while a score below one means the opposite.Note on colour coding: Most indicators are presented with a colour code according to the position of the score relative to the world level, as shown in the example below:

Limitations for Social Sciences and Humanities

Country biases: Most bibliographic databases present a bias in favour of scientific literature published in English. In the Natural Sciences and Engineering (NSE) and Health Sciences (HS), most publications are in English because the research issues mostly have an international orientation. However, in the Social Sciences and Humanities (SSH), researchers publish their work more frequently in journals of local in-terest, and thus in their mother tongue. This leads to stronger country biases in the SSH. This also leads to bias in the scientific impact for some countries. As authors often cite papers from peers in their own country, these references will be indexed more rarely if they are not in English, which reduces the scientific impact (as measured by the number of citations). Field biases: Researchers in the SSH publish their work in books more frequently than those in the NSE and HS. As such, SSH output is usually underestimated, as the databases used for the study only cover articles.

4 5

Detailed accounts of the analyses that have been performed within the context of this study are available in six published reports, appearing under the Analysis and Regular Update of Bibliometrics Indicators header on the European Commission’s portal:2

• Country and Regional Scientific Production Profiles

• Cross-Cutting Analysis of Scientific Publications versus Other Science, Technology and Innovation Indicators

• Scientific Output and Collaboration of European Universities

• Scientific Output and Collaboration of Research Public Organisations

• Scientific Output and Collaboration of Companies Publishing the Most in the ERA

• Intra-European Cooperation Compared to the In-ternational Collaboration of the ERA Countries

Using the data presented in the above reports, this brochure first provides a brief overview of the current state of European Research and of national and regional specialisation patterns within the ERA (Section I). Subsequently, it presents results inves-tigating scientific cooperation as a means towards creating the ERA, achieving greater research impact as well as boosting innovation (Section II). Highlights of the study’s key findings are provided throughout the brochure in boxes.

I — Current state of European Research

This section examines trends in the size and scien-tific impact of European Research compared to a selected set of international comparators. It identi-fies some of the FP7 thematic priorities in which the ERA appears to have a competitive advantage

relative to these comparators. It also presents examples of the bibliometric analyses that have been performed in characterising the scientific performance and collaboration profile of countries, regions and organisations within the ERA. Finally, it presents an analysis of the specialisation patterns of ERA countries and regions.

Trendsinthesizeandscientificimpact of European Research

European Research, whose boundaries are defined by the 41 countries of the ERA, has been com-pared to three international comparators: the United States, China and Japan. This analysis was performed for all scientific subfields combined and was based on cumulative data taken from the 2000 to 2013 period.

Worldwide, China is the key emerging player, com-bining a large scientific production with strong growth. China’s yearly production experienced a seven-fold increase from 2004 to 2013, thereby closing the gap with the United States, whose production increased at a slower pace (GR of 1.13 versus 2.10 for China). In fact, China published slightly more papers than the United States in 2013 (439,854 versus 430,726).

The ERA comes second in growth of production (GR of 1.30), while Japan showed practically no increase in its output during the 2004 to 2013 period (GR of 1.01). However, the ERA score is still slightly smaller than that of the world (GR of 1.42), which is heavily influenced by a number of emerging and fast-growing countries. Aside from China, these countries include India, Brazil and the Republic of Korea (data not shown here due to space constraints). Within ERA countries, those who stand out for having very strong growth include Albania, Montenegro, Serbia, and Bosnia and Herzegovina among the ERA AC, and Luxembourg, Romania, Latvia, Malta, and Cyprus among the EU28 MS.

In 2013, China closed the gap with the US for the size of its scientific production, publishing slightly more papers than the latter.

Altogether, ERA countries are performing better than the US and Japan in terms of growth. Also, ERA countries collectively contribute to a greater share of the world’s production than either China or the US.

The largest producers within the ERA include the following EU28 MS: the UK, Germany, France, Italy and Spain. Those with the strongest growth are found among both the AC and EU28 MS. They include many countries in the Balkans, as well as Luxembourg, Latvia, Malta and Cyprus.

ERA countries, as a group, reduced the gap with the US in terms of scientific impact, quality and excellence from the early 2000s to 2013.

The strongest players within the ERA in terms of scientific impact, quality and excellence are Switzerland, Iceland, Denmark, the Netherlands and Sweden.

Below world level World level Above world level

1 Regions come from the NUTS classification (Nomenclature of

territorial units for statistics) of the EU. The study focused on

NUTS 2 regions, which are the basic regions for the application

of regional policies.2 http://ec.europa.eu/research/innovation-union/index_en.cfm?pg=other-studies

Impact trendsPublication trends

ERA countries are collectively contributing a greater share of the world’s output than either China or the United States, accounting for nearly one third of all papers in Scopus compared to a fifth for each of those two comparators. The largest producers within the ERA are the United Kingdom, Germany, France, Italy and Spain, all of them being part of the EU28 MS.

The general scientific impact (i.e. considering all subfields combined) of both the ERA and China has improved from 2000 to 2010. While China’s prog-ress in this regard is more pronounced, with a gain of 19 percentage points compared to 11 points for the ERA, the average ERA paper published in 2010 generally received 16% more citations than the average world paper in Scopus, compared to 22% fewer citations for the average Chinese paper.

On the contrary, the impact of the United States and Japan remained roughly stable over the entire period, with the United States scoring well above world level and the ERA. In 2010, US papers were cited about 41% more frequently than the average world paper, while Japan scored below world level with its papers receiving about 13% fewer citations.

Thus, the ERA progressed well in this respect, as well as in terms of scientific ‘quality’ (ARIF) and research ‘excellence’ (% of highly cited papers); due to space constraints, data for these latter two indi-cators are not shown here as they correlate very well with scientific impact. Indeed, the ERA reduced the gap with the United States in terms of scientific impact, going from a gap of 33 percentage points in 2000 to 25 points in 2010.

6 7

From the pre-FP7 to the core FP7 period, the scientific performance of ERA countries improved in most thematic priorities.

Areas of strength for the ERA include: Health; Food, Agriculture & Fisheries; Environment; Socio-Economic Sciences; Construction & Construction Technologies; and ICT.

As explained in the limitations, the impact of some countries is underestimated due to an unbalanced coverage of the literature published in English relative to that published in other languages. This underestima-tion is likely to be particularly strong for Asian countries (i.e. China and Japan in the table on page 6).

Within the ERA, the greater average impact of ERA AC compared to EU28 MS is due to the large share, among all papers by ERA AC, of papers from Israel and EFTA countries (i.e. Iceland, Liechtenstein, Norway and Switzerland), which all have high impact well above world level. In fact, in 2010, the scientific impact of EFTA countries stood at 1.62, which means they had 62% more citations than the average world paper. This is the highest score among the ERA’s subgroups (i.e. Candidate countries, EFTA countries, EU28 MS and others). However, ERA AC barely have an effect on the average impact of the ERA’s total production since they contribute a relatively small share of its total output. Indeed, the ERA’s impact is mostly determined by the group of EU28 MS, whose impact trend almost perfectly overlays that for the ERA as a whole.

Among EU28 MS, countries that stand out in terms of scientific impact include Denmark, the Netherlands and Sweden, all with scores that markedly exceed the score of the United States. Among ERA AC, Iceland and Switzerland stand out, though the production size for Iceland is relatively small.

PositionalanalysisofEuropeanResearchwithinFP7thematicpriorities

The ERA’s areas of strength are identified through a positional analysis accounting for the size, specialisation and scientific impact of the ERA by research area, which were defined on the basis of the thematic priorities of the Cooperation specific programme under FP7. The analysis high-lights changes in the ERA’s scientific performance trajectories that could potentially be attributed to FP7 by comparing the position of the ERA in each thematic priority between the period 2004–2007, prior to FP7, and 2010–2013, which broadly cor-responds to the FP7 period, allowing for a two-year time lag from 2008 when the first sizeable batch of projects started. Note that these changes cannot be attributed to FP7 with certainty due to the lack of a control group. Additionally, FP7-supported papers should continue to be published in the years to come (i.e. beyond 2015) since the last sizeable batch of projects started in 2013.

From the pre-FP7 to the core FP7 period, the scientific performance of ERA countries in terms of both scientific specialisation and impact gener-ally progressed well. An increase or stability in the former indicator combined with an increase in the latter was observed in slightly more than

Scientific performance of ERA countries in Scopus (2000–2013)

Trends in the scientific performance of the ERA in FP7 thematic priorities (2004–2007 vs. 2010–2013)

half of the thematic priorities. For another 40% of the thematic priorities, the ERA either improved its impact (29%) or its specialisation (12%) with a concomitant decrease in the other dimension. Only one thematic priority experienced a decrease in both specialisation and impact: Nanosciences & Nanotechnologies. This decrease began in 2008 after a high in 2007; however, note that this is one of two areas in which the scientific impact (ARC) of publications shows an opposite trend to that based on scientific quality (ARIF).

The growth in the size of ERA output also gener-ally kept pace with the world (GRs slightly above or below world level) with the exception of five areas: Nanosciences & Nanotechnologies, Materials (exclud-ing Nanotech), Aeronautics & Space, Automobiles, and Other Transport Technologies.

The areas in which the scientific performance of the ERA increased with respect to both spe-

cialisation and impact include Information & Communication Technologies (ICT), Construction & Construction Technologies, Environment (including Climate Change), the Socio-Economic Sciences, the Humanities, and New Production Technologies.

Areas of strength clearly include Health, Food, Agriculture & Fisheries, Environment, and Socio-Economic Sciences, in which the ERA has a sizeable production, as well as being special-ised with an impact above world level (top-right quadrant in the previous figure). The areas of Construction & Construction Technologies and ICT are also worthy of mention. In the former area, the ERA is specialised with an impact above world level despite a smaller production. In the latter area, it is approaching the research intensity (specialisation) generally observed at world level, combined with an appreciable out-put size and high impact.

Here is a brief comparison of the ERA against the United States, China and Japan in FP7 thematic priorities (data not shown due to space constraints):

• The ERA scores above the United States in impact in Aeronautics & Space, Other Transport Tech-nologies, Energy, and Construction & Construction Technologies.

• The ERA always outperforms China and Japan in impact, with the exception of China in the Humanities and Security. However, the scores of China and Japan are likely underestimated.

• A challenging area for the ERA is Nanosciences & Nanotechnologies. Indeed, although this is one of the thematic priorities in which the production of the ERA is growing fastest, it is not keeping pace with world output due to the strong growth of emerging countries in this area. Additionally, although its impact remained above world level in 2010, it is decreasing compared to the pre-FP7 period.

Scientificperformanceand collaborationprofilesof countries,regionsand organisationsintheERA

Bibliometric indicators were computed at the national, regional (NUTS 2 regions) and organisational (RPOs, universities and companies) levels to monitor their scientific performance and collaboration profile. They were also computed for all S&T fields (including FP7 thematic priorities) and for economic sectors to iden-tify areas of strength or challenges. The indicators were also computed on a yearly basis to measure evolution in time.

Given the large amount of data produced at these levels, this brochure simply reports on some examples for the organisational level. For a thorough examina-tion of national, regional and organisational data, the reader is referred to the study’s six reports.3

Bibliometric data were collected to benchmark organisations by area of research. The table on page 8 provides an example of the 25 universities

8 9

Example of the scientific performance of universities in Food, Agriculture & Fisheries• The Wageningen

University and Research Centre in the Netherlands and the Swedish University of Agricultural Sciences stand out for their scientific performance.

• Two universities from the Czech Republic stand out for their sizeable output, strong growth and specialisation. Within the context of a smart specialisation strategy, efforts could be made to build up research impact through strategic partnerships with Western European institutions leading in this respect.

3 http://ec.europa.eu/research/innovation-union/index_en.cfm?pg=other-studies

Scientific performance of the selected 25 ERA universities in Food, Agriculture & Fisheries (2007–2013)

Co-publication patterns of the most-publishing ERA RPOs by RPO level in Scopus (2007–2013)

publishing the most in the FP7 thematic priority of Food, Agriculture & Fisheries. These have been taken from among the 303 ERA universities selected to provide a representative coverage of the most-publishing universities in each ERA country.4 Two universities stand out: the Wageningen University and Research Centre in the Netherlands and the Swedish University of Agricultural Sciences. They are respectively first and second in terms of output size; they respectively devote about 8 and 12 times as much of their production, proportionately, to this area relative to the world, and their scientific impact is considerably higher than the world level. Similar findings hold true for research quality and excellence (data not shown due to space constraints).

In terms of growth, two universities stand out for a GR well above world level (comparing 2011–2013 to 2007–2009). Both universities are located in the Czech Republic and are highly specialised in this thematic priority. However, their respective scientific impact is well below world level. Within the context of a smart specialisation strategy building on this strength, efforts could be made to build up research impact through strategic partnerships with Western European institutions that are leading in this respect.

Bibliometric data were also used to study the co-publication patterns of organisations. The table on page 9 provides an example of the most-publishing RPOs in Scopus.

Note that three levels of RPOs are distinguished based on their size, multidisciplinarity and geo-graphic dispersion:

• Level-1 RPOs are multidisciplinary in scope and often comprise several small to medium research institutions and centres across a country;

• Level-2 RPOs perform research in a specific sci-entific domain, are smaller than the level-1 RPOs and may also have small centres and/or institu-tions across a country; and

• Level-3 RPOs conduct research internationally or across the ERA.

Using these data, it can be seen that RPOs devoted to applied research (e.g. IMEC, Technical Research Centre of Finland, Netherlands Organisation for Applied Scientific Research and Paul Scherrer Institute) present the strongest co-publication rates with the private sector and that international RPOs (level 3) exhibit some of the strongest co-publication rates with foreign partners.

Bibliometric data were also used to analyse co-publication networks. Below is an example for the companies publishing the most within the ERA. Seven coherent sub-networks (or communities) were identified, each corresponding to organisations active in similar industrial sectors/markets. These clusters represent research-intensive industrial sectors in Europe and appear to cover 9 of the 17 thematic priorities of the European Commission under FP7. Companies in the Health as well as in the Basic Materials & Food Producers sectors are co-publishing the most with external partners in the academic/RPO sectors.

On page 11 is a graphic presentation of the col-laboration network of these companies.

NationalandregionalspecialisationpatternswithintheERA

The following figure (page 12) provides a brief over-view of the clustering of ERA countries based on their specialisation patterns across FP7 thematic priorities. Generally, EU28 countries are specialised in the the-matic priorities of Health and Environment, whereas Eastern European countries are usually not. Within the cluster composed mostly of EU28 countries, sub-clusters are strongly differentiated on the basis of whether or not they are specialised in Security. Within

the cluster composed mostly of Eastern European countries, sub-clusters are strongly differentiated on the basis of whether or not they are specialised in Food, Agriculture & Fisheries. Almost none of the ERA countries are specialised in applied thematic priori-ties (i.e. Nanosciences & Nanotechnologies, Materials, Aeronautics/Space, Automobiles and Other Transport Technologies) with the exception of ICT and New Production Technologies.

As can be observed, there is not a high level of heterogeneity in the specialisation patterns of ERA

10 11

Example of the co-publication patterns of the most-publishing RPOs• RPOs devoted to

applied research (e.g. IMEC, Technical Research Centre of Finland, Netherlands Organisation for Applied Scientific Research and Paul Scherrer Institute) present the strongest co-publication rates with the private sector.

Example of the co-publication patterns for the most-publishing companies • Companies tend to

aggregate in co-publication networks according to their main industrial sector/market.

Co-publication patterns of the 100 companies publishing the most within the ERA by grouping of industrial sectors (2007–2012)

4 Some of the latest AC of the ERA were not covered at the organisational level.

countries when looking at FP7 thematic priori-ties. This is not so surprising since these priorities represent very broad areas that generally match to scientific disciplines at a fairly high aggregation level — namely, at the field level of Science-Metrix’s journal-based classification, which was developed as part of this study to fulfil the needs of the Commission towards presenting data by main S&T field and FP7 thematic priorities. This classification consists of a three-level hierarchical tree made up of 6 domains, 22 fields and 176 subfields. Data at lower aggregation levels (i.e. by subfields and regions) might provide more meaningful informa-tion towards supporting the development of smart specialisation strategies. The table on page 13 shows the potential added-value of including such information to support the development of smart specialisation strategies on a regional level.

As illustrated, the somewhat homogenous patterns observed at the national level in FP7 thematic priorities may hide a substantial amount of hetero-geneity at the regional level, as well as within the underlying subfields of the FP7 thematic priorities. For instance, while the United Kingdom is not spe-cialised in Other Transport Technologies or in any of its underlying subfields, some UK regions clearly have a niche in some of those subfields, especially in Logistics & Transportation, highlighting the wide variation that might prevail in the relative empha-sis regions place on various subjects. Combined with data on the sheer size and scientific impact of regions by subfield, these data could help target the respective strengths of regions in reinforcing the research capacity of the ERA in strategic areas.

II—ScientificcooperationasameanstowardscreatingtheERA,achievinggreaterresearchimpactaswellasboostinginnovation

Framework Programmes for RTD (FPs) are the cornerstone of the EU’s strategy to increase Europe’s competitiveness through efforts towards closing the ‘technology gap’ with the US. From FP1 (launched in 1984) to H2020 (2014), inter-organisational, inter-regional and international cooperation have become key instruments of policy intervention to realise this objective and create the ERA. For example, networking is often viewed as an effective approach towards sharing knowledge and resources to avoid duplication; towards boosting the research capacities of partnering entities as well as the visibility of their work; and towards complementing competencies to address complex problems that can only be tackled with difficulty by individual teams and/or disciplines. The following sections present find-ings on the potential usefulness of the coopera-tive approach implemented in the FPs.

RoleofthenetworkingapproachpromotedunderFP7towards realisingtheERA

Realising the ERA passes, at least partly, through an increased integration of EU28 MS and ERA Associated Countries. Using bibliometric indicators,

a number of analyses have been produced to determine whether or not the cooperation actions implemented under FP7 have led to an increased integration of ERA countries.

Briefly, analyses studying trends from the pre-FP7 to FP7 period were performed on a number of networking dimensions for ERA countries within and beyond the ERA in an attempt to assess the likelihood that the observed patterns of change could have resulted from FP7. Generally, an increase in the collaboration intensity of ERA countries was observed within and outside the ERA across many FP7 thematic priorities between the pre-FP7 and FP7 periods. This is especially true in Environment (including Climate Change) where the extent of increase is more pronounced within than outside the ERA.5

Unfortunately, the data requested as part of this study did not include a control group such that it was not feasible to attribute observed patterns of change to FP7 with high certainty. A control group would have allowed ruling out other fac-tors that could explain the observed patterns, such as the worldwide increase in the extent of international co-publishing, for example. Below is a new set of results, produced by Science-Metrix through another commitment for the European Commission, examining the effect of FP7 on the international co-publishing rates of ERA countries within and beyond the ERA using a control group.

These data show that the propensity of ERA countries to co-publish with other ERA countries increased more markedly among FP7 participants than is generally observed for non-participants within the ERA. Regardless of the specific pro-gramme under FP7, the observed change is at least five orders of magnitude stronger for FP7 participants than the general population of researchers within the ERA. Only for the Ideas spe-cific programme is the extent of change smaller, which is not surprising since this FP7 component did not aim to directly foster cooperation. Finally, although similar patterns are observed when examining the co-publication propensity of ERA countries with foreign partners located outside the ERA, differences in the magnitude of change between FP7 participants and non-participants

12 13

The somewhat homogenous

patterns observed at the national level

in FP7 thematic priorities hide a

substantial amount of heterogeneity at

the regional level, as well as within the underlying subfields of the FP7 thematic

priorities.

Upon its renewal, the coverage of this study could be expanded to

incorporate data on regional specialisation by subfield. Combined

with data on the sheer size and

scientific impact of regions by subfield,

these data could help target the respective strengths of regions

in reinforcing the research capacity of

the ERA in specific areas.

Two main clusters of countries can be

differentiated within the ERA on the basis of their specialisation

pattern in FP7 thematic priorities,

namely EU28 MS and Eastern European

countries. The former group is generally

specialised in Health and Environment,

whereas the latter is not.

5 For more details, see the study’s report entitled Intra-European Cooperation Compared to International Collaboration of the

ERA Countries.

The results presented here show that the inclusion of a control group in producing bibliometric data for a funding programme such as FP7 can help attribute observed patterns of change to its effects. It was shown that FP7 contributed to increasing the integration of the ERA.

Future large-scale bibliometric studies, such as the one reported here for the European Commission, could therefore include such a control group more systematically (i.e. non-FP7 participants within the ERA). This will work towards addressing many questions pertaining to the success of FPs in achieving their full range of goals, especially in the H2020 context.

Current alignment of ERA countries with FP7 thematic priorities based on their specialisation patterns (2013)

Specialisation by region in the UK and by subfield in the FP7 thematic priority of

Other Transport Technologies (2010–2013)

International collaboration rates of ERA and of FP7 participants prior to and while supported

are much less pronounced; for Ideas, no positive change is observed in this case. Altogether, these results suggest that FP7 did play a positive role in creating the ERA by increasing the integration of European countries through more frequent part-nerships in the form of co-publications.

Achievinggreaterresearchimpactthroughscientificcooperation

There is ample evidence that the knowledge dis-seminated in co-authored scientific publications, especially those involving transnational partnerships, is taken up more broadly by the scientific community than that of single author publications, as measured through citations. It is therefore not surprising that

this phenomenon has attracted much attention from policymakers as a means to foster research excel-lence, and the FPs are no exception in this regard.

In investigating the mechanisms underlying the increased citation impact of co-publications, Science-Metrix previously showed that author self-citations account for only 12% of the difference in the ARC scores of international co-publications versus single author publications. Additionally, it showed that although the ARC scores of various types of co-publications increase as the number of authors increases (with or without self-citations), the actual number of authors is a bad predictor of citation impact at the paper level. In other words, co-publications with multiple authors are not a guarantee of an increased citation impact of individual papers, but they increase the likelihood

of producing high-impact papers. Science-Metrix’s analysis also showed that the number of countries in which the authors of a publication are located matters.6 Because scientific collaboration networks are often characterised by a strong geographic clustering of researchers, increasing the diversity of countries involved in a scientific publication is likely to increase its impact by tapping into disconnected networks; there is evidence in the scientific literature suggesting that impact increases as the geographical distances between countries increases.7

The above findings from the literature are coher-ent with the citation impacts of various types of co-publications that have been computed for ERA countries within the context of this study (see table on page 14).

The publication types associated with the highest ARC scores are, in descending order:

• EU28 & Non-EU28 CP: international co-publications involving at least one author in the given country, at least one author from an EU28 country and at least one author from a non-EU28 country;

• ERA CP: international co-publications involving at least one author in the given country and at least one author from EU28, EFTA or candidate countries;

• ICP: international co-publications involving the given country and at least one author from an-other country;

• Non-EU28 CP: international co-publications involving the given country and one or more non-EU28 only author(s);

• EU28 CP: international co-publications involving the given country and one or more EU28 only author(s);

• SCCP: domestic only co-publications (i.e. single country co-publications); and

• SAP: single author publications.

Thus, it appears that the cooperation actions imple-mented under the FPs have the potential to benefit the uptake/influence of the research performed by ERA countries. However, it must be pointed out that the types of partnerships created within the context of the FPs often imply mixing researchers from multiple sectors (for example, academic–private partnerships), which might not be as conducive to high impact as partnerships within academia. For instance, if the increased citation impact of co-publications comes from the dissemination of the knowledge within each of the authors’ usual citation networks, then the notoriety of the co-authors certainly matters in

14 15

The cooperation actions implemented under the FPs have the potential to benefit the uptake/influence of the research performed by ERA countries. For instance, it was shown that co-publications involving a larger number of authors and countries, especially those involving both EU28 MS and non-EU28 countries, generate more impact within the ERA than any other type of publication analysed in this study.

Additional data comparing the impact of other types of co-publications (i.e. intersectoral and interdisciplinary instead of those based on geography) not considered here are required to gain a better understanding of the potential effects of the FPs on the research impact and excellence of ERA countries.

6 For more details, see Bibliometric Study in Support of Norway’s Strategy for International Research Collaboration.7 Nomaler, O., Frenken, K., and Heimeriks, G. (2013). Do more distant collaborations have more citation impact? Journal of

Informetrics, 7(4) 966–971.

Scientific impact (ARC) of various types of publications for ERA countries and selected comparators (2000–2010)

Interdisciplinarity by NUTS 2 region within the ERA (2010–2014)

addition to the extent they are disconnected (i.e. the geographic distance that separates them). Since academic researchers usually publish to a far greater extent than those in the private sector, their citation networks are also likely to be much more developed than those of the latter.

Consequently, the impact of academic–private partnerships might generally not be as high as those of academic–academic partnerships. Additionally, although it is often recognised that the citation impact of a researcher reflects, at least to some degree, the excellence of his or her work, one might question to which extent this is the case for the different types of co-publications. Indeed, the increased impact of co-publications might not always result from the greater novelty of the experiments and/or findings stemming from them, but rather from the mere additive effect of disseminating the findings in a greater number of citation networks (i.e. the usual citation network of each of the co-authors); in other words, the gain in impact originates from the respective characteristics of the authors before the actual collaboration takes place.

Thus, additional data comparing the impact of additional types of co-publications are required to gain a better understanding of the potential effects of the FPs on the research impact and excellence of ERA countries. Such analyses would be particularly useful if they included datasets for FP7 participants and non-participants within the ERA.

Boostinginnovationthrough interdisciplinarycollaboration

Interdisciplinary collaboration is now broadly recognised as having the potential to fuel revolutionary/transformative research whose outcomes extend beyond what could have been achieved had each discipline evolved separately. As such, governments are increasingly funding programmes to promote interdisciplinary research networks, with the ultimate aim of spurring greater

innovation. For example, as part of H2020, the Commission recently introduced a theme dealing specifically with high-risk interdisciplinary research aimed at generating tomorrow’s most innovative technologies.8

Given the emphasis placed on this dimension within H2020, this study, once renewed, should include novel metrics aimed at capturing trends in the extent of disciplinary mixture within the peer-reviewed scientific literature published by ERA countries, as well as beyond. For instance, Commission officials identified such metrics as being essential to monitoring the effects of interdisciplinary collaboration in the context of H2020 on the occasion of the ‘Use of indicators for STI Policy Workshop’.9

The previous map (page 15) shows the extent of variation observed in the propensity of European regions to perform research that crosses disciplinary boundaries.

Using this indicator, Science-Metrix has shown that research in applied fields is generally more inter-disciplinary (e.g. findings submitted as a conference papers). In line with this finding, Science-Metrix also found that technical universities, which are closer to technological innovation than other universities, are over-represented among universities having a higher-than-expected proportion of highly interdisci-plinary papers. This suggests that interdisciplinarity can likely spur the emergence of new technologies, as applied research is closer to innovation than the more fundamental research performed in the natural or health sciences.

Since it was also found that the 100 most-publishing ERA universities are generally less interdisciplinary than the 100 most-publishing non-ERA universities, the emphasis placed on interdisciplinary collaboration in the context of H2020 appears as a potentially promising avenue to boost Europe’s innovation capabilities. Given the strong limitations faced in developing this metric, further research is required to refine the analysis of interdisciplinarity.

In light of the efforts the European

Commission is currently devoting to spur innovation

through the promotion of

interdisciplinary collaboration within

the context of H2020, new indicators are

required to monitor trends in the share of

this mode of research, as well as to assess

its impact on the innovation capabilities of ERA countries and,

more generally, on the competitiveness

of the European economy.

8 http://exquisitelife.researchresearch.com/exquisite_life/horizon-2020-work-programme-summaries.html9 This workshop was held in Brussels, Belgium, at the Commission premises of DG RTD on 21 January 2015.

HOW TO OBTAIN EU PUBLICATIONS

Free publications:• one copy:

via EU Bookshop (http://bookshop.europa.eu);• more than one copy or posters/maps:

from the European Union’s representations (http://ec.europa.eu/represent_en.htm); from the delegations in non-EU countries (http://eeas.europa.eu/delegations/index_en.htm); by contacting the Europe Direct service (http://europa.eu/europedirect/index_en.htm) or calling 00 800 6 7 8 9 10 11 (freephone number from anywhere in the EU) (*). (*) The information given is free, as are most calls (though some operators, phone boxes or hotels may charge you).

Priced publications:• via EU Bookshop (http://bookshop.europa.eu).

16

This study presents an analysis of bibliometric indicators measuring the quantity and impact of scientific publications produced from 2000 to 2013 in Europe and other parts of the world. The main goal is to support the development of European research and innovation policies. These indicators allow for a comprehensive analysis of the evolution, inter connectivity, performance and impact of national research and innovation systems in Europe. They also provide an overall view of Europe’s strengths and challenges in knowledge production across fields of science.

The study shows how the EU’s 7th Framework programme for research (FP7, 2007-2013) contributed to the progress of the European Research Area (ERA) initiated in 2000 to ensure that scientific knowledge, technology and researchers circulate freely in the European Union.

Studies and reports

doi: 10.2777/194026

KI-04-15-284-EN-N