an experimental comparison of bibliometric mapping techniques

1111

An Experimental Comparison of Bibliometric Mapping Techniques

Nees Jan van Eck, Ludo Waltman, Rommert Dekker

Erasmus University Rotterdam, The Netherlands{nvaneck,lwaltman,rdekker}@few.eur.nl

Jan van den BergDelft University of Technology, The Netherlands

[email protected]

10th International Conference on Science and Technology Indicators

Vienna, September 18, 2008 1

2

Bibliometric mapping

Similarity measureDirect Indirect

Jaccard Cosine Association strength … Pearson

correlation Cosine …

Unit of analysisAuthors Journals Words/terms Web pages …

Mapping techniqueDistance based Graph based

MDS VxOrd VOS … Pajek Pathfinder networks …

3

Bibliometric mapping







4

Data sets






1. Journals in economics, management, and operations research 2001-2005 ISI categories: business; business, finance; economics;

management; and operations research & management science 376 journals

2. Terms in computational intelligence 2001-2005 337 terms



5

Similarity measures






• Other names for the association strength probabilistic affinity index proximity index


6

Mapping techniques






• Ordinal MDS in SPSS PROXSCAL• VxOrd (DrL)

VxOrd1: top 10 similarities, edge cutting (default settings) VxOrd2: all similarities, no edge cutting VxOrd3: top 10 similarities, no edge cutting

7

Evaluation criteria

1. Quality of the mapsDoes a map provide a satisfactory representation of the data?

2. RobustnessMeasures the similarity between maps that have been constructed based on different samples drawn randomly from the same data set

3. EfficiencyMeasures the similarity between a map that has been constructed based on an entire data set and maps that have been constructed based on samples drawn randomly from that data set

8

MDS – Jaccard index (journals)

9

VxOrd1 – cosine (journals)

10


11


12

VOS – association strength (journals)

1313

Comparison of maps (journals)

• MDS clearly performs worst, with no good separation between journals from different fields

• As is typical for MDS, journals are located more or less uniformly distributed within a circle

• VxOrd3 and VOS perform quite well, with a clear clustering of journals based on their field

1414

Robustness and efficiency (journals)

• MDS and VxOrd1 have a low robustness and efficiency• The other techniques do much better

15

MDS – Jaccard index (terms)

16

VxOrd1 – cosine (terms)

17


18


19

VOS – association strength (terms)

2020

Comparison of maps (terms)

2121

Robustness and efficiency (terms)

2222

Conclusions

• No complete specification of VxOrd is available in the literature

MDS VxOrd VOS

Quality of the maps

locates objects more or less uniformly

distributed within a circle, with no clearly

separated clusters

produces attractive maps with a very clear clustering, but only if

appropriate parameter values are

chosen

produces satisfactory maps without the

need to choose any parameter values

Robustness and

efficiencylow

depends on parameter values and

data sethigh

an experimental comparison of bibliometric mapping techniques

Documents

vxord1 cosine journals

vxord3 cosine journals

vxord2 cosine journals

vxord1 cosine terms

vxord3 cosine terms

vxord2 cosine terms

comparison of maps terms

mds jaccard index journals