This article was downloaded by: [York University Libraries]On: 12 August 2014, At: 08:05Publisher: RoutledgeInforma Ltd Registered in England and Wales Registered Number: 1072954Registered office: Mortimer House, 37-41 Mortimer Street, London W1T 3JH,UK

Measurement: InterdisciplinaryResearch and PerspectivesPublication details, including instructions forauthors and subscription information:http://www.tandfonline.com/loi/hmes20

Facing Diversity of Science:A Challenge for BibliometricIndicatorsMichel ZittPublished online: 17 Nov 2009.

To cite this article: Michel Zitt (2005) Facing Diversity of Science: A Challenge forBibliometric Indicators, Measurement: Interdisciplinary Research and Perspectives,3:1, 38-49, DOI: 10.1207/s15366359mea0301_6

To link to this article: http://dx.doi.org/10.1207/s15366359mea0301_6

PLEASE SCROLL DOWN FOR ARTICLE

Taylor & Francis makes every effort to ensure the accuracy of all theinformation (the “Content”) contained in the publications on our platform.However, Taylor & Francis, our agents, and our licensors make norepresentations or warranties whatsoever as to the accuracy, completeness,or suitability for any purpose of the Content. Any opinions and viewsexpressed in this publication are the opinions and views of the authors, andare not the views of or endorsed by Taylor & Francis. The accuracy of theContent should not be relied upon and should be independently verified withprimary sources of information. Taylor and Francis shall not be liable for anylosses, actions, claims, proceedings, demands, costs, expenses, damages,and other liabilities whatsoever or howsoever caused arising directly orindirectly in connection with, in relation to or arising out of the use of theContent.

This article may be used for research, teaching, and private study purposes.Any substantial or systematic reproduction, redistribution, reselling, loan,sub-licensing, systematic supply, or distribution in any form to anyone is

expressly forbidden. Terms & Conditions of access and use can be found athttp://www.tandfonline.com/page/terms-and-conditions

Dow

nloa

ded

by [

Yor

k U

nive

rsity

Lib

rari

es]

at 0

8:05

12

Aug

ust 2

014

Facing Diversity of Science: A Challengefor Bibliometric Indicators

Michel ZittLereco, Institut National de la Recherche Agronomique

Nantes, FranceObservatoire des Sciences et des Techniques

Paris

Bibliometrics has come of age and is generally accepted, together with peer re-view, as one way to describe the activity of players in science. Van Raan (this is-sue), with his team a prominent contributor to this specialty, describes in the fo-cus article a variety of state-of-the-art indicators that show the concern ofbibliometricians for improving measures used in sensitive contexts such as re-search assessment. I shall not try to comment on all aspects of this substantialcontribution. The sections of the focus article (performance, interdisciplinarity,structure) deal in various ways with the central question of the diversity of sci-ence as either an object or a framework for performance indicators. I limit mydiscussion here to a few practical issues stemming from this diversity: How doyou define or structure a scientific field? Is normalization straightforward?Which measure can be used for measuring knowledge flows and relationsamong fields?

As the present challenges of science and technology indicators can hardly bedisconnected from a historical perspective of scientometrics, a bit of hindsight isperhaps necessary. There is a lot of evidence since the work of Price (1963),Nalimov & Mulczenko (1969), and Garfield & Cawkell (1970) that many facets ofscientific activity are amenable to measurement and modeling. Scientometricshandles as typical objects the networks generated by scientific activity, especiallythose implicitly carried by published outputs (networks of authors–institutions, ofdocuments, of lexical contents, etc.). A typical one is the network of citation link-

Requests for reprints should be sent to Michel Zitt, Lereco, INRA Nantes, Rue de la Geraudiere BP71627, F-44316 Nantes Cedex 03 France. E-mail: zitt@nantes.inra.fr

MEASUREMENT, 3(1), 38–49Copyright © 2005, Lawrence Erlbaum Associates, Inc.

Dow

nloa

ded

by [

Yor

k U

nive

rsity

Lib

rari

es]

at 0

8:05

12

Aug

ust 2

014

ages. Scientometrics mobilizes a variety of statistical and data analysis tools to in-vestigate these networks and the associated distributions.

Certain forms of statistical distributions depicting the concentration of infor-mation are pervasive in science studies. An example is the power-law form ofthe “bibliometric trilogy” (Bradford, 1934; Lotka, 1926; Zipf, 1949; for a syn-thesis, see Rousseau, 1990, and Egghe, 1991), which proves resilient and com-patible with a variety of static or dynamic models (Bookstein, 1990)—as inphysics and economics. In particular, concentrated laws were soon associatedwith mechanisms of positive feedback on reputation and scientific resources (thesocial process of cumulative advantage was studied by Merton, 1968, whocoined the term “Matthew effect” after a passage in the Gospel of Matthew). In aclassical interpretation of power laws in terms of self-similarity, the degree ofconcentration is also inversely related to complexity and diversity. Thepower-law approximation is present in other bibliometric indicators forms (Katz,1999; Zitt, Ramanana-Rahary, & Bassecoulard, 2003b) or their Web-basedanalogs (Ingwersen, 1998; Rousseau, 1997). Different forms of concentrateddistributions have been investigated, for example, quasi power law (van Raan,2001, in his competition model); the Zipf variant in a network interaction model(Ruiz-Banos, Bailon-Moreno, Jimenez-Contreras, & Courtial, 1999); Weibullfor empirical representation combining citation reception and emission (Zitt,Ramanana-Rahary, & Bassecoulard, 2003a); and so on. For the construction ofindicators, the prevalence of very concentrated distributions has rather importantconsequences: among its advantages, the efficiency of the principle of selectionof sources, items, and so on; among its shortcomings, the atypical character ofusual moments or order statistics, especially for extreme concentration (Haitun,1982). The Matthew effect also suggests avoiding overinterpretation of absolutediscrepancies in citation figures.

Beyond aggregated distributions, close-up analyses of networks have been pi-oneered on the citation side by Crane (1972), Garfield (1970), Kessler (1963),and Small and Griffith (1974), as well as on the lexical side, with the cowordanalysis promoted by the Anglo–French sociology of innovation (Callon,Courtial, Turner, & Bauin, 1983). They are now being renewed by social net-work analysis, which helps to characterize both local and global properties ofnetworks—for example, the small world (Watts & Strogatz, 1998) structure.Self-similarity, found in most scientific networks and already suggested by ag-gregate distributions of nodes degrees, is often put down to self-organization ofscientific communities (see, e.g., Katz, 1999).

A striking point is that, within the framework of general laws, a large varia-tion of the parameters among local subnetworks of science takes place, express-ing the specificity of behavior, organization, and diversity within each field andtype of research. Besides the theoretical challenges of modeling self-organizedscientific activity, which takes the generation of diversity and complementarity

COMMENTARIES 39

Dow

nloa

ded

by [

Yor

k U

nive

rsity

Lib

rari

es]

at 0

8:05

12

Aug

ust 2

014

as an object, a practical challenge in evaluation-oriented scientometrics is tocope with the consequences of this diversity—namely, the heterogeneity of ar-eas—and to find appropriate reference sets for indicators to avoid mixing applesand oranges. Besides, beyond traditional specialization issues, indicators of di-versity are also increasingly used in evaluation studies (see, e.g., Adams &Smith, 2003).

HOW TO DEFINE AND STRUCTUREA SCIENTIFIC FIELD

There is no universally accepted definition of scientific fields. Depending on one’spoint of view, many criteria are put forth: institutional setting, shared topics andpossibly shared journals, shared terminology, close connections of collaborationor citation. Scientometrics can operationalize many theoretical or pragmatic defi-nitions of that kind, provided that they can be translated in terms of scientific itemsand networks. Information may be extracted from published output or externalsources—for example, lists of researchers or institutional sites—built ex ante innomenclatures or ex post by statistical classifications, vary in level of aggregation,can be general or specific to a player, or stem from coarse-grained or fine-grainedobservation.

A typical coarse-grained approach takes all articles in a set of complete scien-tific journals. The set of journals defined by each “Institute of Scientific Informa-tion (ISI) category class” is a common coarse-grained reference often used for in-stitutional spectroscopy, although the composition of categories is oftendisputable. The set of journals in which the player under scrutiny publishes—anexample given by van Raan (this issue)—is a classical solution with the merit ofsimplicity with respect to some level and time thresholds. It can be advisable to ei-ther discard journals where the player is marginally active in a rationale of selec-tion à la Bradford (1934) or, in particular cases, to enlarge the set with additionaljournals close to the publication journal in some respect to figure a set where theplayer could publish—for example, in journals that are close in terms of citationflows or journals where the competitors publish. Ad hoc modifications of ISI cate-gories, and in some cases entirely expert-built lists of journals, are other commonpractices. Such coarse-grained approaches are relatively inexpensive and quitesuitable for a “macro” positioning of players, provided that multidisciplinary jour-nals are dealt with separately.

If a precise delineation of fields is required, coarse-grained approaches usuallyfail to reach an acceptable trade-off in terms of recall-precision: Recalling morejournals for inclusion of relevant articles is paid for by an excessive number of ir-relevant ones. The alternative is a fine-grained delineation at the level of individualarticles. Producers of indicators often rely on classical information retrieval pro-

40 COMMENTARIES

Dow

nloa

ded

by [

Yor

k U

nive

rsity

Lib

rari

es]

at 0

8:05

12

Aug

ust 2

014

cesses based on lexical queries on titles, abstracts, or full texts. Recent advances innatural language processing techniques boost the potential of lexical methods.However, bibliometrics can also rely on kindred networks linked to the self-repre-sentation of scientific communities, especially citation relations pioneered at ISIand Drexel University in the 1960s and 1970s (Garfield, 1970; Small & Griffith,1974), or institutional arrangements. Citation relations in many forms(transactional, cocitation, bibliographic coupling, etc.) are particularly powerfultechniques. In tricky emerging areas, citation may perform well, whereas lexicalmethods get trapped, but the reverse may happen. Each network conveys a particu-lar rationale, and, of course, the outcome is also sensitive to the statistical methods,choice of metric, and so on. How to build on the complementarity of techniques,together with expert or peer validation protocols, is a challenge for scientometrics.

The definition of fields is critical for a wide range of scientometric assessments:Specialization studies (actors’ spectrum) obviously depend on the relevance ofbreakdowns; studies of multidisciplinarity are closely connected to the definitionof disciplines or fields (see Multidisciplinarity and Knowledge Flows); normaliza-tion of performance indicators also demands a relevant field definition and struc-turing.

Related issues are found at various scales:

• The coverage of databases, which constrains all further delineations, hasraised many controversies, especially for the standard database, the ScienceCitation Index (Moravcsik, 1988; Sivertsen, 1992). Often considered too re-strictive, it also appears too large in other respects for internationalbenchmarking purposes (van Leeuwen, et al., 2001; Zitt et al., 2003b). A re-current issue, addressed in the focus article, is the ISI coverage of social andhuman sciences. The variety of publication modes, recently reviewed byHickes (2004), is a well-known limitation. Also, the balkanization of na-tional traditions and research objects, especially for large European(non-English-speaking) countries, jeopardizes indicators based on ISI datain a way that is balanced for the degree of internationalization of these disci-plines The transition toward the transnational model, almost completed in thenatural sciences, has hardly begun in some human sciences. Examples fromthe hardest soft sciences (linguistics, psychology, to a large extenteconomics1) cannot be extrapolated to other fields, with each discipline call-ing for a particular assessment. Van Raan’s (this issue) optimism seems a bitexcessive in this respect. It is possible that new databases will in the futurebring alternative solutions.

COMMENTARIES 41

1On a much more selective basis than the specialized database JEL-ECONLIT, (Journal of Eco-nomic Literature), however. Law and history, for example, can be overbiased. Some empirical aspectsof transition in SCI were studied by Zitt, Perrot, and Barré (1998).

Dow

nloa

ded

by [

Yor

k U

nive

rsity

Lib

rari

es]

at 0

8:05

12

Aug

ust 2

014

• For “large-scale” mapping (the “microapproach”), the fine-grained delinea-tion of subfields or themes within a given field is a quite similar issue. Forcosts and consistency constraints, a monocriterion method is generally usedfor calculating similarity. Discontinuous (clustering) methods lead to either asegmentation or a hierarchy of clusters, overlapping or not, that represent lo-cal agglomerations in the network. Continuous methods reveal latent dimen-sions (principal components, correspondence, latent semantic indexing, etc.)or optimize mapping by multidimensional scaling (MDS). Within this vari-ety of techniques, currently boosted by data-mining and Web applications,van Raan (this issue) gives an example of a mapping technique based on lexi-cal profiles. Since the pioneering text analyses techniques in the 1980s byBenzecri (1981) on the one hand, and the “sociology of innovation” on theother hand with “Leximappe” coword techniques (Callon et al., 1983), sig-nificant progress has been brought about by natural language processingtechniques such as extraction of multiterms or noun phrases, and languagedisambiguation.

More generally, the position of bibliometrics at a crossroad of quantitative dis-ciplines raises the question of benchmarking of methods for specific applicationsin this area. A challenge for scientometric teams, which sometimes stick to theirown learning curves, is to keep pace with evolving methods in nearby disciplines,especially computational linguistics, and to benchmark the various techniquesavailable for either data analysis or linguistic treatment.

WHICH SET SHOULD BE USED FOR NORMALIZATION?

Whether delineated at a coarse or fine-grained level, and with respect to the par-ticular interpretation of the structuring relations used, fields or themes are natu-ral candidates for reference sets in which active players can be compared to theset average or benchmarked against each other. Take the example, discussed byvan Raan (this issue), of citation performance suitable to measure internationalvisibility (rather than quality). Bibliometricians should pay tribute to sociolo-gists who enlightened the complex social nature of citation and navigate be-tween the “rosy” version of the Mertonian school (in short, citation as one wayto recognize an intellectual debt, however blurred by the Matthew effect; seeMerton, 1969, and Small, 2004) and the “dark” version of the Latourian school(citation as a form of enrollment and immunity; Latour, 1987). Self-citationraises similar questions and exists at all levels—the laboratory, the institution,and the nation. Neutralization at one level, mentioned by van Raan, is a partialsolution. Self-citation seems a more serious issue in performance rating applica-tions than in structuring and mapping applications.

42 COMMENTARIES

Dow

nloa

ded

by [

Yor

k U

nive

rsity

Lib

rari

es]

at 0

8:05

12

Aug

ust 2

014

A major question is the heterogeneity of citation networks and differences inaverage levels among fields. Related issues for performance measures were recog-nized early in pioneer quantitative science studies (Murugesan & Moravcsik,1978). Time features add to the difficulty (Glanzel & Schoepflin, 1992). A prag-matic solution for interfields comparison emerged with various kinds of normal-ization, starting with field-independent “relative indicators.” The CHI Researchteam in the United States and the Information Science and Scientometric ResearchUnit team at the Academy of Sciences in Budapest (Schubert, Glanzel, & Braun,1988) in Europe popularized the central concepts in the 1980s. Scientometricteams and observatories later developed on this basis various practical responses tothe normalization issue, and among them the Leiden group has cumulated a richexperience of such indicators in an evaluation context in addition to penetratingworks on the limits of classical indicators such as ISI’s “impact factor” (Moed &van Leeuwen, 1995). Normalization is easily extended to other sources of discrep-ancies like the type of document or language, but other possibly interestingnormalizations are more difficult to implement, such as the type of research (say,theoretical, methodological, or applied), especially when articles of all kinds coex-ist in the same journal. The effectiveness of practical solutions cannot hide the lim-its of the exercise.

Which level of aggregation, in the range from large disciplines to small themes,should be used? In contrast with other forms of proximity (geographical or institu-tional), no particular level can be deemed relevant. Self-similarity makes doubtfulthe idea of an optimum level where science networks should be observed.2

Fundamentally, scientific activity can be observed at any scale as far as themes areconcerned with respect to some local optima that may be detected by varioustechniques.

Proximity of articles in terms of themes is correlated to their proximity in termsof citation score. The correlation is not very high but is sufficient to produce bothacross-fields and across-scale instability: In a realistic embedded classification,the rating of an article is highly sensitive to changes in the level of observation(Zitt, Ramanana-Rahary, & Bassecoulard, 2004). One consequence is that theusual “excellence” measures based on top-cited fractions of articles prove to bevery fragile. The level of aggregation matters for the rating of articles and actors.

A challenge is to find a trade-off between the argument of relevance, favorableto the choice of small clusters for normalization (Kostoff, 2002) with the monop-oly as a boundary,3 and the necessity of enlarging the reference set for statistical re-liability, with the limits due to statistical distributions stated previously. Another

COMMENTARIES 43

2A similar argument applied to discoveries scale concludes van Raan’s (2000) article on the fractalmodel of science.

3Expressed, for example, by Fitoussi and Wasmer (Wasmer, 2001): In the extreme, “any researcheris in a monopoly situation and can by definition not be evaluated.”

Dow

nloa

ded

by [

Yor

k U

nive

rsity

Lib

rari

es]

at 0

8:05

12

Aug

ust 2

014

somewhat paradoxical issue is the fact that bibliometric methods are able to findclusters of multidisciplinary articles adequate for their mutual benchmarking, butusing these clusters as references for normalization will tend to underratemultidisciplinary articles, often highly cited, whereas a quick-and-dirty normal-ization using their main disciplinary assignment, or some all-science mix, wouldperform better. More generally, overnormalization unduly equalizes all fields orthemes in science. A variety of points of view and of levels of normalization shouldbe accepted.

We have focused on normalization, but several classical indicators have seriouslimits. The concentrated distributions and underlying feedback, previously men-tioned and stressed by van Raan (this issue), suggest that in many cases rank-basedindicators are preferable. They are widely used by bibliometric bureaus, enablingthem to communicate on various aspects of the distribution of citations, including“excellence” measures. The Observatoire des Sciences et Techniques (Paris), forexample, has been using a variety of these rank indicators, flexible in changes ofscale, for interinstitutional benchmarking.

MULTIDISCIPLINARITY AND KNOWLEDGE FLOWS

Multidisciplinarity is a crucial phenomenon in the evolution of science, with manyinterpretations along with its variants, interdisciplinarity and transdisciplinarity. Itcan be seen as both the counterpart of diversity—maintaining connections be-tween established domains—and the source of further diversity—creating newbuds by recombination. Usual quantitative analysis of multidisciplinarity sup-poses, first, an operational definition of disciplines (fields, themes, etc.) and, sec-ond, the capability to study overlaps, proximity, or transactions between these dis-ciplines. As mentioned previously, bibliometric networks analysis is a tool, amongothers, used to define fields on a particular criterion (institutional, conceptual,knowledge-flow, etc.). Then, once fields are defined, their relations on any of thesecriteria can be studied:

• Structural relations: sharing players from various origins (authors or institu-tions), concepts markers (vocabulary), a knowledge base (cited articles), orknowledge users (citing articles).

• Transaction relations: exchanges of staff (thematic migration of players) orflows of knowledge supposed measurable by citations in either direction.Van Raan (this issue) provides an example of the latter to describe the area ofinfluence of a particular actor.

The most direct type of analysis uses the same criterion to structure fields andstudy their relations. Van Raan (this issue) shows an example of such a map

44 COMMENTARIES

Dow

nloa

ded

by [

Yor

k U

nive

rsity

Lib

rari

es]

at 0

8:05

12

Aug

ust 2

014

based on lexicology4 and mentions the other classical type, that based oncocitations. The first measure of multidisciplinarity is monocriterion and builtin: For example, the lexical similarity of Clusters 3 and 14, identified as PhysicalChemistry and Materials, is a possible static measure of their interdisciplinaryrelation. A related dynamic measure of interdisciplinarity would be based on thespeed of reconfiguration of the lexical network and related clusters. The firststudy of this kind was perhaps the “comparative static” of cocitation clustersstudied by ISI in the 1970s (Small, 1976). Each method goes along with hypoth-eses or points of view on the evolution of science. For example, cocitation mapsconvey the perspective that the current state of knowledge is based on a cumula-tive process with creative rearrangements of advances embodied in articles.Above all, such a bibliometric construction of fields, flexible and built ex post,may strongly contrast with the ex ante institutional definitions, an oppositionthat scientometricians have to manage in practical contexts when dealing withusers and experts.

Among transaction measures, citation flows need perhaps particular attentioninsofar as, in a Mertonian perspective mentioned earlier, they are interpretable as acounterpart of knowledge flows and hence of scientific influence. Bibliome-tricians have used them for decades—for example, for building classificationsbased on interjournal transactions (Bassecoulard & Zitt, 1999; Carpenter & Narin,1973; Leydesdorff & Cozzens, 1993). A new impetus has come from the study ofthe science base of patents (Narin & Olivastro, 1992) or among patents exchanges(Jaffe & Trajtenberg, 1999). Van Raan (this issue) gives an illustration of interfieldmeasure of dependence in his Figure 2.

It should be stressed that measures of interfield connections are highly sensi-tive to methodology. One example: Discrepancies in citation behavior concernthe emission side, whereas normalization usually bears on the reception side.Whereas the intellectual base of a field (or an institution)—that is, the spectrumof its emitted citations—can be straightforwardly studied on the hypothesis of itsrelative homogeneity of behavior (a sometimes heroic hypothesis; see thecross-scale normalization issue previously mentioned), the characterization ofthe spectrum of citations received by a field is likely to be biased by the respec-tive features of the emitting fields, and this issue can be extended to several clas-sical measures of connections. A normalization of citations at the emission,

COMMENTARIES 45

4Van Raan’s Figure 3 combines a positional and a relational display. In positional methods, theproximity of items on the map is meant to express their multidimensional similarity. MDS allows an op-timal representation in n dimensions, especially two, but the projection stress often makes the interpre-tation difficult for final users. This is also true for factor techniques, which have the advantage of reveal-ing latent dimensions. In relational methods, proximity is figured by linkages in a network (classicalrepresentation of coword or cocitation technique). The interpretation is easier if the display is legible.Various algorithms, including MDS, can be used in an attempt to optimize the distribution of items onthe map. Mapping techniques benefit from the current needs for mapping information from the Web.

Dow

nloa

ded

by [

Yor

k U

nive

rsity

Lib

rari

es]

at 0

8:05

12

Aug

ust 2

014

rather than at the target, is worth examining at either the article level or the fieldlevel (Zitt et al., 2005). At the article level, some fractional counting of emittedcitations may be used. This practice has been used in a cocitation context: Small& Sweeney (1985) made a historical point; Zitt & Bassecoulard (1994) obtainedsatisfactory results using a form of fractionation. The correction is quite strong iffields with contrasted behavior (say, fundamental biology and mathematics) areconcerned and may alleviate the risk of overnormalization for articles inmultidisciplinary categories. This is, of course, quite different from another in-teresting form of weighting, also on the citing side, that gives more importanceto citations emitted by visible articles or authors. This weighting, mentioned byvan Raan (this issue), is a variant of the “influence measures” advocated by theCHI team in the 1970s at the journal level (Pinski & Narin, 1976). At the articlelevel, this weighting may still reinforce the Matthew effect in the results.

Scientometrics offers some powerful means to help with the understanding ofmultidisciplinarity, a quite difficult subject touching on the very structure and dy-namics of science. Some modest achievements have already been shown byscientometricians in various contexts, but, clearly, developing measures adapted toemerging research questions is still a challenge for the discipline.

CONCLUSION

There are appealing issues in bibliometrics, among them the characterization of agrowth regime of research systems (e.g., Bonaccorsi, 2002; Leydesdorff, 2000),including the trade-offs of production–visibility (e.g., Butler, 2003; Zitt et al.,2003b); the measure of science–technology linkage and coevolution; the measureof the societal effects of science (Arunachalam, 1997; Lewison, 2002); new mod-els of collaboration applied to networks dynamics; the multicriterion evaluation ofactors, especially universities; and so on. Classical, more focused technicalitiescall for new developments, however: measures of productivity, indicators fullyconsistent with bibliometric distributions, the question of standardization, and soon. Indisputably, however, the three issues addressed by the focus article are partic-ularly crucial for evaluative bibliometrics. Moreover, they are deeply related toeach other—for example, performance measures are meaningful only if they com-ply with the reality of science in its diversity and connectivity. We have mentioneda few challenges for scientometrics related to these issues. Also recall thatbibliometric tools are dependent on advances in linguistics, data analysis and min-ing, and network analysis techniques, with a real need for benchmarking thesetechniques in a scientometric context.

Bibliometricians are immersed in decision-support processes. They haveproven that effective and efficient indicators could be proposed to decision makersand stake-holders, and much investment has been made, especially by the Leiden

46 COMMENTARIES

Dow

nloa

ded

by [

Yor

k U

nive

rsity

Lib

rari

es]

at 0

8:05

12

Aug

ust 2

014

group, in conceiving user-friendly bibliometric applications (Noyons, 2001). Atthe same time, more precautions are needed, as “desktop” bibliometrics may rein-force a danger already present in less reactive scientometrics: the contrast betweenthe highly sensitive nature of evaluation issues and the eagerness of users or clientsto elicit a figure or a display and forget crucial warnings about statistical distribu-tions and methodology artifacts. If scientometrics is a mirror of science in action,then scientometricians’ particular responsibility is to both polish the mirror andwarn against optical illusions.

Scientometrics and bibliometrics have a particular role not only toward sciencepolicy and monitoring, but also toward the research questions of nearby disci-plines. They have been largely built over foundations laid by information scientistsand sociologists of various allegiances and reciprocally can feed social scienceswith creative methodology. The relation to the economics of science will certainlybe tighter and tighter: Scientific knowledge embodied in articles is now recognizedas a legitimate object by economists, and a great interest arises for measures of sci-entific activity, knowledge flows, scientific collaboration networks, and sciencegrowth regimes in the wake of Price’s works.

ACKNOWLEDGMENTS

The author thanks E. Bassecoulard, INRA, and G. Filliatreau, OST, for helpfulcomments.

REFERENCES

Adams, J., & Smith, D. (2003). Funding research diversity. London: Universities UK.Arunachalam, S. (1997). How relevant is medical research done in India? A study based on Medline.

Current Science, 72, 912–922.Bassecoulard, E., & Zitt, M. (1999). Indicators in a research institute: A multi-level classification of

scientific journals. Scientometrics, 44, 323–345.Benzecri, J. P. (Ed.). (1981). Pratique de l’analyse des données: Linguistique et lexicologie. Paris:

Dunod.Bonaccorsi, A. (2002, January). Matching properties: Research regimes and institutional systems in

science. Paper presented at conference on Science as an Institution and the Institution of Science,Siena, Italy.

Bookstein, A. (1990). Informetric distributions, Part I: Unified overview. Journal of the American Soci-ety for Information Science, 41, 368–375.

Bradford, S. C. (1934). Sources of information on specific subjects, Engineering 137, 85–86.Butler, L. (2003). Explaining Australia’s increased share of ISI publications—the effects of a funding

formula based on publication counts. Research Policy, 32, 143–155.Callon, M., Courtial, J. P., Turner, W. A., & Bauin, S. (1983). From translations to problematic net-

works: An introduction to co-word analysis. Social Science Information, 22, 191–235.

COMMENTARIES 47

Dow

nloa

ded

by [

Yor

k U

nive

rsity

Lib

rari

es]

at 0

8:05

12

Aug

ust 2

014

Carpenter, M. P., & Narin, F. (1973). Clustering of scientific journals. Journal of the American Societyfor Information Science, 24, 425–436.

Crane, D. (1972).The social organization of research areas. In Invisible colleges: Diffusion of knowl-edge in scientific communities (pp. 44–46). Chicago: University of Chicago Press.

Egghe, L. (1991). The exact place of Zipf’s law and Pareto’s law amongst the classical informetric laws.Scientometrics, 20, 93–106.

Garfield, E. (1970). Citation indexing for studying science. Nature, 227, 669–671.Garfield, E., & Cawkell, A. E. (1970). Location of milestone papers through citation networks. Journal

Library History, 5, 184–188.Glanzel, W., & Schoepflin, U. (1992). A bibliometric study on ageing and reception processes of sci-

entific literature. In A. F. J. van Raan, R. E. de Buin, H. F. Moed, A. J. Nederhof, & R. W. J.Tijssen (Eds.), Science and technology in a policy context: Select proceedings of the JointEC–Leiden Conference on Science & Technology Indicators (pp. 121–137). Leiden, The Nether-lands: DSWO Press.

Haitun, S. D. (1982). Stationary scientometric distributions. Part II: Non-gaussian nature of scientificactivities. Scientometrics, 4, 89–104.

Hicks, D. (2004). The four literatures of social science. In H. F. Moed, W. Glanzel, & U. Schmoch(Eds.), Handbook of quantitative science and technology research (pp. 473–493). Dordrecht, TheNetherlands: Kluwer.

Ingwersen, P. (1998). The calculation of Web impact factors. Journal of Documentation, 54, 236–243.Jaffe, A. B., & Trajtenberg, M. (1999). International knowledge flows: Evidence from patent citations.

Economics of Innovation and New Technology, 8, 105–136.Katz, J. S. (1999). The self-similar science system. Research Policy, 28, 501–517.Kessler, M. M. (1963). Bibliographic coupling between scientific papers. American Documentation,

14, 10–25.Kostoff, R. N. (2002). Citation analysis of research performer quality. Scientometrics, 53, 49–71.Latour, B. (1978). Science in action: How to follow scientists and engineers through society. Cam-

bridge, MA: Harvard University Press.Lewison, G. (2002). From biomedical research to health improvement. Scientometrics, 54, 229–240.Leydesdorff, L. (2000). The triple helix: An evolutionary model of innovations. Research Policy, 29,

243–255.Leydesdorff, L., & Cozzens, S. E. (1993). The delineation of specialities in terms of journals using the

dynamic journal set of the SCI. Scientometrics, 26, 135–156.Lotka, A. J. (1926). The frequency distribution of scientific productivity. Journal of the Washington

Academy of Sciences, 16, 317–323.Merton, R. K. (1968). The Matthew effect in science. Science, 159, 56–63.Merton, R. K. (1969). Behavior patterns of scientists. American Scientist, 56–63.Moed, H. F., & van Leeuwen, T. N. (1995). Improving the accuracy of Institute for Scientific Infor-

mation’s journal impact factors. Journal of the American Society for Information Science, 46,461–467.

Moravcsik, M. J. (1988). The coverage of science in the third world. The “Philadelphia program.” In L.Egghe & R. Rousseau (Eds.), Informetrics 87/88 (pp. 147–155). Amsterdam: Elsevier.

Murugesan, P., & Moravcsik, M. J. (1978). Variation of the nature of citation measures with journal andscientific specialties. Journal of the American Society for Information Science, 29, 141–155.

Nalimov, V. V., & Mulczenko, Z. M. (1969). Naoukometria [Scientometrics. The study of science as aninformation process]. Moscow: Nauka.

Narin, F., & Olivastro, D. (1992). Status-report—Linkage between technology and science. ResearchPolicy, 21, 237–249.

Noyons, E. (2001). Bibliometric mapping of science in a science policy context. Scientometrics, 50,83–98.

48 COMMENTARIES

Dow

nloa

ded

by [

Yor

k U

nive

rsity

Lib

rari

es]

at 0

8:05

12

Aug

ust 2

014

Pinski,G.,&Narin,F. (1976).Citation influence for journalaggregatesof scientificpublications:Theory,withapplication to the literatureofphysics. InformationProcessingandManagement,12,297–312.

Price, D. J. S. (1963). Little science, big science. New York: Columbia University Press.Rousseau, R. (1990). Relations between continuous versions of bibliometric laws. Journal of the Amer-

ican Society for Information Science, 41, 197–203.Rousseau, R. (1997). Citations: An explanatory study. Cybermetrics, 1, 7.Ruiz-Banos, R., Bailon-Moreno, R., Jimenez-Contreras, E., & Courtial, J. P. (1999). Structure and dy-

namics of scientific networks. Part II: The new Zipf’s law, the clusters of co-citations and the modelof the descriptor presence. Scientometrics, 44, 235–265.

Schubert, A., Glanzel, W., & Braun, T. (1988). Against absolute methods: Relative scientometric indi-cators and relational charts as evaluation tools. In A. F. J. van Raan (Ed.), Handbook of quantitativestudies of science and technology (pp. 137–169). The Netherlands: Leiden University.

Sivertsen, G. (1992). Should a new bibliometric database for international comparisons be more re-stricted in journal coverage? In A. F. J. van Raan, R. E. de Bruin, H. F. Moed, A. J. Nederkof, and R.W. J. Tijssen (Eds.), CG Leiden Conference on Science and Technology in a Policy Context(pp. 35–50). Leiden, The Netherlands: DSWO Press.

Small, H. (1976). Structural dynamics of scientific literature. International Classification, 3(2), 67–74.Small, H. (2004). On the shoulders of Robert Merton: Towards a normative theory of citation.

Scientometrics, 60(1), 71–79.Small, H., & Griffith, B. C. (1974). The structure of scientific literatures I: Identifying and graphing

specialties. Science Studies, 4, 17–40.Small, H., & Sweeney, E. (1985). Clustering the Science Citation Index using co-citations I—A com-

parison of methods. Scientometrics, 7, 391–409.van Leeuwen, T. N., Moed, H. F., Tijssen, R. J. W., Visser, M. S., & van Raan, A. F. J. (2001). Language

biases in the coverage of the Science Citation Index and its consequences for international compari-sons of national research performance. Scientometrics, 51, 335–346.

van Raan, A. F. J. (2000). On growth, aging, and fractal differentiation of science. Scientometrics, 47,347–362.

van Raan, A. F. J. (2001). Competition amongst scientists for publication status: Toward a model of sci-entific publication and citation distribution. Scientometrics, 51, 347–357.

Wasmer, E. (2001, November). Some political economy of excellence. Paper presented at workshop, InSearch of Scientific Excellence: Research Performance by Discipline, [European Commission pro-ject, Science & Technology Indicators for the European Research Area], Brussels, Belgium.

Watts, D. J., & Strogatz, S. H. (1998). Collective dynamics of “small-world” networks. Nature, 393,440–442.

Zipf, G. K. (1949). Human behavior and the principle of least effort. Cambridge, MA: Addison-Wes-ley.

Zitt, M., & Bassecoulard, E. (1994). Development of a method for detection and trend analysis of re-search fronts built by lexical or cocitation analysis. Scientometrics, 30, 333–351.

Zitt, M., Perrot, F., & Barré, R. (1998). The transition from “national” to “transnational” and relatedmeasures of countries’performance. Journal of the American Society for Information Science, 49(1),30–42.

Zitt, M., Ramanana-Rahary, S., & Bassecoulard, E. (2003a). Bridging citation and reference distribu-tions: Part I—The referencing-structure function and its application to co-citation and co-item stud-ies. Scientometrics, 57, 93–118.

Zitt, M., Ramanana-Rahary, S., & Bassecoulard, E. (2003b). Correcting glasses help fair comparisonsin international science landscape: Country indicators as a function of ISI database delineation.Scientometrics, 56, 259–282.

Zitt, M., Ramanana-Rahary, S., & Bassecoulard, E. (in press). Measuring excellence by top-citation:How performances depend on the zoom setting. Scientometrics.

COMMENTARIES 49

Dow

nloa

ded

by [

Yor

k U

nive

rsity

Lib

rari

es]

at 0

8:05

12

Aug

ust 2

014