Vienna, Austria, 15-19 July 2013, http://www.issi2013.org/

Introduction Groups of highly cited publications may change in content of individual

publications with extending citation windows, while remaining stable in size.

Such content changes can be relevant in particular when highly cited publications are used to assess the level of excellence for publication records of limited size,

e.g. of individual scientists.

This poster paper shows content changes between groups of highly cited articles identified in different citation window lengths, in 2 domains with different

citation characteristics: a Physics sub-domain (faster) and Mathematics (slower).

Groups of Highly Cited Publications: Stability in Content with Citation Window Length

Nadine Rons, Nadine.Rons@vub.ac.be, http://rd-ir.vub.ac.be/en_GB/people/show/id/554 Research Coordination Unit, Vrije Universiteit Brussel (VUB), Pleinlaan 2, B-1050 Brussels, Belgium

Specifications and notations

Data: Data sourced from Thomson Reuters Web of Knowledge (formerly referred to as ISI Web of Science - WoS). WoS accessed online, 01-02.2013. Document type: ‘article’ only, Publication year: 2004. Citation windows: up to 8 years, including publication year. Two domains, each consisting of three partition cells containing all journals assigned to a same combination of subject categories[a]: Physics sub-domain: Cells consisting of the journals assigned to subject category ‘Astronomy Astrophysics’ only (A, 8047 articles), to ‘Physics Particles Fields’ only (P, 1977 articles), and to both (A&P, 2440 articles). Mathematics: Cells consisting of the journals assigned to subject category ‘Mathematics’ only (M, 10022 articles), to ‘Mathematics Applied’ only (MA, 3938 articles), and to both (M&MA, 3286 articles). Highly cited publications: The frame below shows details for ‘outstandingly’ cited articles identified in the third iteration of the parameter free Characteristic Scores and Scales (CSS) technique[b]. The poster paper in the proceedings also discusses, in less detail, similar results obtained for ‘remarkably cited’ publications, and with pre-set percentiles of 1% and 5%.

References [a] Rons, N. (2012). Partition-based field normalization: An approach to highly specialized publication records. Journal of Informetrics, 6(1), 1-10.

[b] Glänzel, W. & Schubert, A. (1988). Characteristic scores and scales in assessing citation impact. Journal of Information Science, 14(2), 123-127. [c] Glänzel, W. (2007). Characteristic scores and scales: A bibliometric analysis of subject characteristics based on long-term citation observation. Journal of Informetrics, 1(1), 92-102.

Changing CONTENT of groups of outstandingly cited articles with citation window length

Summary and conclusions

A group of highly cited publications does not contain exactly the same individual publications calculated in citation windows of different length. As citation window length is gradually incremented per year, successive groups of highly cited publications more strongly resemble in content [1.1], evolving towards a phase of higher ‘content stability’ that is still subject to changes from year to year [1.2].

Characteristics of publications entering and leaving the group of highly cited publications with increasing citation window length vary with domain, regarding their relative amounts [2.1] from year to year [1.3], and citation levels before entering [2.3] or after leaving [2.2] the group of highly cited articles.

The changing content of a group of highly cited publications with citation window length is an important issue in particular in methodologies for the assessment of publication records of limited size, such as those of individual scientists.

For assessments of small subjects, effects of choices regarding citation window length deserve particular attention in domains with ‘slow’ citation characteristics, and in comparisons between domains with different citation characteristics.

[2] Shifts of individual articles Change over 5 years, between citation windows of 3 and 8 years

Between a small 3-year citation window and a large 8-year citation window, change in content can be substantial

[Vertical axis: logarithmic scale, value 1 represents 1 or 0 accumulated citations]

DOMAIN DEPENDENCE: With increasing citation window length, articles partly shift to other citation levels, more so in Mathematics than in Physics, in relative amount and in range of citation levels: (illustrated left for 1 cell per domain; numbers below for all 3 cells per domain)

[2.1] The relative amount of articles entering the group of outstandingly cited articles later than the first 3 years (black dashed lines versus colored full lines) is lower in Physics (0,3 to 0,4) than in Mathematics (0,7 to 0,8).

[2.2] Most articles leaving the group of outstandingly cited articles between 3 and 8 years (grey dashed lines) are still remarkably cited in the 8-year window, more often in Physics (80-100%) than in Mathematics (65-75%).

[2.3] Most articles entering the group of outstandingly cited articles between 3 and 8 years (black dashed lines) already are remarkably cited in the 3-year window, more often in Physics (79-94%) than in Mathematics (58-78%).

[2.4] The most cited articles in the 3-year and 8-year citation windows are the same in Physics, and are different in Mathematics.

Occurrence of articles changing outstandingly cited status in publication records of individual scientists: (preliminary observations, not illustrated)

In the observed cells, a number of individual scientists published more than one article in 2004 that gained outstandingly cited status later than the first 3 years:   2 articles for 19 scientists in Physics, representing 7% up to all of the scientist’s articles from 2004 in this domain;   2 to 4 articles for 16 scientists in Mathematics, representing 11% up to all of the scientist’s articles from 2004 in this domain.

[1] Overall content resemblance Changes between consecutive citation windows, incremented by one year

Between two citation windows, one year different in length, changes in content become smaller between larger windows

[1.1] Groups of outstandingly cited articles identified in two consecutive citation windows have an increasing part of content in common for increasing citation windows. A similar convergence in content is observed in each cell of the two domains. (illustration above: larger intersections for pairs of consecutive citation windows of higher length)

[1.2] In sufficiently large citation windows, a later more stable phase with smaller remaining changes in content from year to year is reached.

[1.3] DOMAIN DEPENDENCE: A same degree of ‘content stability’ between groups of outstandingly cited articles identified in consecutively incremented citation windows is reached sooner in Physics than in Mathematics.

While their content changes, the groups of outstandingly cited articles remain stable in size with extending citation windows beyond an initial citation period of about 3 years[c] [frame below].

Stable SIZE of groups of highly cited articles

(illustrated below for Characteristic Scores and Scales (CSS): stability beyond an initial citation period of about 3 years, see also [c]; obvious for pre-set percentiles)