mendeley as a source of readership by students and postdocs? evaluating article usage by academic...
DESCRIPTION
Stefanie Haustein & Vincent Larivière (2014). Mendeley as a Source of Readership by Students and Postdocs? Evaluating Article Usage by Academic Status. Presentation at IATUL 2014. http://docs.lib.purdue.edu/cgi/viewcontent.cgi?article=2033&context=iatulTRANSCRIPT
Mendeley as a Source of
Readership by Students and Postdocs?
Evaluating Article Usage by Academic Status
@stefhaustein
Stefanie Haustein
& Vincent Larivière
Introduction
Measuring use of scholarly documents
• reshelving, interlibrary loan
• citation analysis
• electronic full text access
• social reference managers and bookmarking systems
Introduction
Research questions:
• Can Mendeley readership counts be used to monitor use
of scholarly documents?
• Does use differ between scientific fields?
• Can different user sectors and user types be identified
based on the academic status?
• Can the data be used to determine whether specific user
groups predict citation impact?
Introduction
Introduction
• 2.8 million users, 275,860 groups, 535 user documents (02/2014)
• monthly growth rate of 3.7% (documents) and 2.3% (users) 2013
• 68 million unique publications (08/2012; 281 million user
documents)
Mendeley statistics based on monthly user counts from 10/2010 to 02/2014 on the Mendeley website accessed through the Internet Archive
Data sets & methods
• 1,161,145 PubMed papers covered by WoS
• publication years: 2010-2012
• document types: articles & reviews
• NSF disciplines: Biomedical Research, Clinical Medicine, Health, Psychology (journal-based classification)
• open citation window
• Mendeley readership data collected via API
• Levenshtein distance (5%) to account for errors in
metadata
• document title (long titles = 70 characters, 5 words)
• document title and first author name (short titles)
1.7% false positives, 0.7% false negatives
Data sets & methods
• aggregating reader counts of multiple entries
Data sets & methods
• number of readers per academic status
• number of missing readership status per paper
29% 7 PhD students
21% 5 Master students
8% 2 Doctoral students
58% 14 readership status available
42% 10 missing readership status
Data sets & methods
• aggregating academic status information
Results: disciplines
available missing
all disciplines 1,161,145 7.5 65.9% 9.6 8.9 0.512 ** 70.0% 30.0%
Biomedical Research 286,398 10.3 72.4% 14.3 11.8 0.575 ** 69.5% 30.5%
Clinical Medicine 779,707 6.8 62.8% 7.6 8.2 0.492 ** 70.5% 29.5%
Health 59,073 4.4 67.0% 6.5 4.3 0.434 ** 72.8% 27.2%
Psychology 35,967 6.1 81.0% 14.0 6.6 0.545 ** 67.5% 32.5%
ρReadership statusNSF discipline
Papers
PubMed
& WoS
Mean
citation
rate
Papers with readers
%Mean
reader rate
Mean
citation rate
• two-thirds of papers saved at least once on Mendeley
• reader rate comparable to citation rate
• Spearman correlations between citations and reader counts
moderately positive (ρ=0.445** / ρ=0.512**)
• academic status not available for 30% of reader counts
Results: specialtiesDifferences between specialties
%Mean
reader rate
all disciplines 65.9% 9.6 0.512 **
Biomedical Research 72.4% 14.3 0.575 **
Anatomy & Morphology 68.2% 5.5 0.380 **
Biochem & Mol Biol 71.6% 12.4 0.550 **
Biomedical Engineering 74.9% 10.4 0.513 **
Biophysics 78.6% 11.8 0.537 **
Cell Biol, Cytol & Histol 74.7% 14.3 0.584 **
Embryology 79.2% 13.2 0.649 **
Gen Biomed Research 72.5% 35.1 0.689 **
Genetics & Heredity 74.1% 17.3 0.558 **
Microbiology 72.7% 10.4 0.555 **
Microscopy 72.5% 6.7 0.494 **
Misc Biomed Research 74.3% 8.8 0.585 **
Nutrition & Dietetic 66.9% 6.6 0.494 **
Parasitology 66.0% 6.1 0.436 **
Physiology 72.1% 8.0 0.457 **
Virology 68.9% 7.1 0.534 **
Papers with readers
ρNSF discipline or specialty
Size of data points represents mean reader rate.
Anatomy & Morphology
General Biomedical Research
Embryology
Results: specialtiesDifferences between specialties
Size of data points represents mean reader rate.
%Mean
reader rate
all disciplines 65.9% 9.6 0.512 **
Clinical Medicine 62.8% 7.6 0.492 **
Addictive Diseases 68.2% 5.8 0.436 **
Allergy 69.8% 8.3 0.582 **
Anesthesiology 63.0% 6.8 0.497 **
Arthritis & Rheumatology 63.3% 6.3 0.488 **
Cancer 62.8% 7.3 0.550 **
Cardiovascular System 56.6% 7.4 0.555 **
Dentistry 68.5% 5.6 0.398 **
Dermat & Venerial Dis 51.3% 4.2 0.433 **
Endocrinology 64.4% 7.1 0.518 **
Environ & Occupat Health 66.1% 6.9 0.501 **
Fertility 64.4% 4.3 0.417 **
Gastroenterology 58.1% 6.0 0.508 **
Gen & Internal Medicine 51.8% 8.2 0.519 **
Geriatrics 73.5% 7.5 0.494 **
Hematology 59.5% 6.9 0.557 **
Immunology 65.8% 9.1 0.561 **
Misc Clinical Medicine 70.6% 9.1 0.458 **
Papers with readers
ρNSF discipline or specialty
Veterinary Medicine
Psychiatry
Neurology & Neurosurgery
Results: specialtiesDifferences between specialties
Size of data points represents mean reader rate.
%Mean
reader rate
all disciplines 65.9% 9.6 0.512 **
Clinical Medicine 62.8% 7.6 0.492 **
Nephrology 63.9% 5.3 0.458 **
Neurol & Neurosurgery 73.1% 13.6 0.554 **
Obstetrics & Gynecology 60.4% 4.3 0.420 **
Ophthalmology 63.0% 4.4 0.486 **
Orthopedics 66.0% 6.9 0.449 **
Otorhinolaryngology 59.7% 4.1 0.383 **
Pathology 60.1% 5.3 0.503 **
Pediatrics 62.0% 5.8 0.469 **
Pharmacology 63.4% 6.5 0.501 **
Pharmacy 55.9% 4.8 0.405 **
Psychiatry 72.1% 9.2 0.583 **
Radiol & Nucl Medicine 63.9% 6.8 0.467 **
Respiratory System 65.1% 6.8 0.487 **
Surgery 58.0% 4.2 0.420 **
Tropical Medicine 65.4% 5.8 0.478 **
Urology 54.8% 4.1 0.432 **
Veterinary Medicine 66.3% 7.5 0.236 **
Papers with readers
ρNSF discipline or specialty
Veterinary Medicine
Psychiatry
Neurology & Neurosurgery
Results: specialtiesDifferences between specialties
Size of data points represents mean reader rate.
%Mean
reader rate
all disciplines 65.9% 9.6 0.512 **
Health 67.0% 6.5 0.434 **
Geriatrics & Gerontology 69.8% 7.3 0.540 **
Health Policy & Services 66.1% 6.8 0.421 **
Nursing 62.0% 5.1 0.378 **
Public Health 66.0% 6.0 0.439 **
Rehabilitation 73.0% 8.0 0.434 **
Social Sciences, Biomed 76.0% 9.2 0.495 **
Social Studies of Med 49.5% 3.1 0.281 **
Speech-Lang Path & Audio 79.0% 7.7 0.436 **
Psychology 81.0% 14.0 0.545 **
Behav Sci & Compl Psych 83.4% 12.2 0.503 **
Clinical Psychology 80.7% 11.1 0.536 **
Develop & Child Psych 80.3% 13.2 0.531 **
Experimental Psychology 85.6% 19.2 0.582 **
General Psychology 68.5% 9.3 0.493 **
Human Factors 84.2% 9.2 0.434 **
Misc Psychology 79.3% 11.4 0.531 **
Psychoanalysis 39.5% 3.6 0.137
Social Psychology 82.4% 24.8 0.687 **
Papers with readers
ρNSF discipline or specialty
Social Studies of Medicine
Geriatrics & Gerontology
Psychoanalysis
Social Psychology
Results: sectors
Results: sectors
• Biomedical Research papers mostly used by readers from
scientific sector
• more professionals in Clinical Medicine
• more educational and professional users in Health
• more educational, less professional users in Psychology
%Mean
reader rate
Mean
citation rateScientific Educational Professional missing
all disciplines 65.9% 9.6 8.9 0.512 ** 48.5% 15.7% 5.8% 30.0%
Biomedical Research 72.4% 14.3 11.8 0.575 ** 54.9% 12.0% 2.6% 30.5%
Clinical Medicine 62.8% 7.6 8.2 0.492 ** 44.2% 17.6% 8.7% 29.5%
Health 67.0% 6.5 4.3 0.434 ** 38.1% 27.3% 7.4% 27.2%
Psychology 81.0% 14.0 6.6 0.545 ** 46.6% 19.0% 1.8% 32.5%
NSF discipline
Papers with readers Sector type of readership status
ρ
Results: sectors
y = 0.0031x + 0.3823R² = 0.433
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1.0
Ve
teri
nar
y M
edic
ine
De
nti
stry
Mis
c C
linic
al M
edic
ine
Nu
rsin
gSo
cial
Stu
die
s o
f M
edR
eh
abili
tati
on
An
esth
esio
logy
Ob
ste
tric
s &
Gyn
eco
logy
Ort
ho
ped
ics
Uro
logy
De
rmat
& V
ene
rial
Dis
Nu
trit
ion
& D
iete
tic
Surg
ery
Pu
blic
Hea
lth
Tro
pic
al M
edic
ine
Pe
dia
tric
sFe
rtili
tyH
ealt
h P
olic
y &
Ser
vice
sO
ph
thal
mo
logy
Ph
arm
acy
Oto
rhin
ola
ryn
golo
gySp
eec
h-L
ang
Pat
h &
Au
dio
Art
hri
tis
& R
he
um
ato
logy
Ad
dic
tive
Dis
eas
es
Gen
& In
tern
al M
ed
icin
eC
ard
iova
scu
lar
Syst
emP
sych
oan
alys
isR
esp
irat
ory
Sys
tem
Nep
hro
logy
Ger
iatr
ics
Alle
rgy
Envi
ron
& O
ccu
pat
He
alth
Soci
al S
cie
nce
s, B
iom
ed
Pat
ho
logy
Hu
man
Fac
tors
Ph
arm
aco
logy
Gas
tro
ente
rolo
gyP
aras
ito
logy
Gen
eral
Psy
cho
logy
End
ocr
ino
logy
Rad
iol &
Nu
cl M
edic
ine
Mis
c B
iom
ed
Re
sear
chG
eria
tric
s &
Ge
ron
tolo
gyC
linic
al P
sych
olo
gyP
sych
iatr
yA
nat
om
y &
Mo
rph
olo
gyC
ance
rP
hys
iolo
gyB
iom
edic
al E
ngi
ne
erin
gM
isc
Psy
cho
logy
Soci
al P
sych
olo
gyH
emat
olo
gyV
iro
logy
Imm
un
olo
gyB
eh
av S
ci &
Co
mp
l Psy
chD
eve
lop
& C
hild
Psy
chM
icro
bio
logy
Exp
erim
enta
l Psy
cho
logy
Neu
rol &
Ne
uro
surg
ery
Mic
rosc
op
yB
ioch
em
& M
ol B
iol
Bio
ph
ysic
sG
enet
ics
& H
ere
dit
yEm
bry
olo
gyC
ell
Bio
l, C
yto
l & H
isto
lG
en B
iom
ed R
ese
arch
Perc
enta
ge o
f re
ader
s p
er s
ecto
r
Spea
rman
co
rrel
atio
n b
etw
een
cit
atio
ns
and
rea
der
co
un
ts
Professional Educational Scientific missing Spearman's ρ
Results: users
Results: users
Results: users
0.575**
0.559**
0.534**
0.478**
0.435**
0.426**
0.410**
0.396**
0.353**
0.318**
0.224**
0.089**
0.234**
0.135**
0.183**
0.059**
0.071**
0.059**
0.066**
0.074**
0.049**
0.042**
0.040
0.051
all readers
Postdoc
PhD Student
Researcher (Academic)
Student (Postgraduate)
Researcher (Non-Academic)
Professor
Assistant Professor
Student (Bachelor)
Associate Professor
Other Professional
Librarian
Biomedical Research
all documents (n=207,255) 100% available reader status (n=80,858)
All documents
• postdocs and PhD
students most
similar to citations
• librarians least
similar
100% status info
• PhD students and
Postdocs most
similar to citations
• other professionals
and associate
professors least
similar
Results: users
0.492**
0.451**
0.425**
0.410**
0.408**
0.364**
0.361**
0.317**
0.300**
0.183**
0.137**
0.055**
0.238**
0.075**
0.093**
0.174**
0.121**
0.067**
0.059**
0.056**
0.079**
0.050**
0.030**
0.029**
all readers
Researcher (Academic)
Researcher (Non-Academic)
PhD Student
Postdoc
Assistant Professor
Professor
Associate Professor
Other Professional
Student (Postgraduate)
Student (Bachelor)
Librarian
Clinical Medicine
all documents (n=489,597) 100% available reader status (n=258,656)
All documents
• researchers most
similar to citations
• librarians least
similar
100% status info
• PhD students and
Postdocs most
similar to citations
• Bachelor students
and librarians least
similar
Results: users
0.434**
0.340**
0.329**
0.320**
0.307**
0.282**
0.280**
0.276**
0.266**
0.250**
0.214**
0.083**
0.196**
0.127**
0.099**
0.038
0.000
0.093**
0.021
0.004
0.076**
0.044
0.058**
-0.028
all readers
PhD Student
Researcher (Academic)
Researcher (Non-Academic)
Postdoc
Student (Postgraduate)
Professor
Assistant Professor
Other Professional
Associate Professor
Student (Bachelor)
Librarian
Health
all documents (n=39,564) 100% available reader status (n=19,955)
All documents
• low correlations
• PhD students,
researchers and
postdocs most
similar
100% status info
• PhD students and
Postdocs most
similar to citations
• no similarity for
librarians and
postdocs
Results: users
0.545**
0.480**
0.480**
0.425**
0.403**
0.400**
0.368**
0.356**
0.321**
0.299**
0.282**
0.107**
0.189**
0.158**
0.125**
0.082**
0.070
0.052
0.076*
0.048
0.037
0.120**
0.048
-0.069
all readers
PhD Student
Postdoc
Student (Postgraduate)
Professor
Researcher (Academic)
Assistant Professor
Student (Bachelor)
Researcher (Non-Academic)
Other Professional
Associate Professor
Librarian
Psychology
all documents (n=29,121) 100% available reader status (n=7,932)
All documents
• PhD students and
postdocs most
similar to citations
• librarians least
similar
100% status info
• PhD students,
postdocs and other
professionals most
similar to citations
• negative
correlation for
librarians
Conclusions: general results
• Mendeley important source of documents’ usage
• 2.8 million users, 535 million user documents
• 65.9% of sampled documents saved 9.6 times on average
• reader counts reflect similar but broader use of scholarly
documents than citations
• Spearman’s ρ = 0.445**/0.512**
• PhD students, postgraduate students and postdocs largest
user group, librarians the smallest
Conclusions: general results
• differences between disciplines and specialties
• coverage:39.5% (Psychoanalysis) – 85.6% (Experimental Psychology)
• reader rate:3.1 (Social Studies of Medicine) – 35.1 (General Biomedical Research)
• correlation with citations:0.137 (Psychoanalysis) – 0.687** (Social Psychology)
• user sector
• scientific:27.7% (Veterinary Medicine) – 63.2% (Microscopy)
• educational:8.6% (General Biomedical Research) – 32.5% (Dentistry)
• professional:0.7% (Experimental Psychology) – 18.6% (Veterinary Medicine)
Limitations
• metadata quality
• academic status self-reported
need to verify whether accurate and up-to-date
• restriction to top 3
• differences between user groups cannot be determined due
to data restriction
• similarity with citation patterns of different user groups cannot
be accurately determined
• even more problematic for countries and disciplines
complete data needed for detailed and accurate statistics