the secret life of articles: from download metrics to downstream … · 2016-02-17 · the secret...

Center for Information and Communication Studies

The secret life of articles: From download metrics to downstream impact

Carol Tenopir University of Tennessee

[email protected]

Lorraine Estelle Project COUNTER

[email protected]

Wouter Haak Elsevier

[email protected]

CIBER Research Ltd.

mailto:[email protected]




http://ciber-research.eu/CIBER_Research_Ltd.html


Beyond Project COUNTER

• What are download counts missing?

• How much do scholars share and what do they share?

• How might sharing be measured?

• What is a more complete value of articles?


14 focus group participants 15 interviews

Interviews / Focus Groups Demographics


International Survey with 1000 responses


Main takeaways

1. A means to an end 2. Function drives form 3. Email is #1 4. Version matters 5. The library is key


1. A MEANS TO AN END


“If I have got it, I will just share it. It is easier than having them go track it down, even if I have got the citation.”

“It’s not like I’m giving away something that they could not get anyhow. But it never dawned on me that in effect, you know in theory, I am violating copyright.”


When working in a research group …

N=771

Per

cent

share more 67%

sharing habits would not change

30%

share less 3%


2. FUNCTION DRIVES FORM


Interviews / Focus Groups

Two types of saving and sharing 1. Metadata (citation) When other researchers could easily access articles. 2. Saving document (pdf). When access was an issue.

Full-text vs. reference/link sharing


Their most recent link or reference is shared …

Email Learning Management Software an average of 10.5

(8.2 minus outliers) times per download.

an average of 13.9 (5.4 minus outliers) times per download.


Their most recent full-text is shared …

Email Research social networks an average of 10.5 (8.4 minus outliers) times per download.

an average of 10.8 (5.3 minus outliers) times per download.


3. EMAIL IS #1


Top preferred methods of sharing for research

Email ranked 1st with 73.8% respondents listing as most preferred method of sharing.

Cloud services 2nd with 11.6% respondents listing as most preferred method of

sharing.

Internal networks ranked 3rd with 10.7% respondents listing as most preferred method of sharing.


Top preferred methods of sharing for teaching

Email ranked 1st with 82.8% respondents listing as most preferred method of sharing.

Internal networks ranked 3rd with 8% respondents listing as most preferred method of sharing

Learning management systems ranked 2nd with 9.6% respondents listing as most preferred method

of sharing


However, researchers feel more people are reached through …

General social networks: 96-614 people potentially reached

Internal networks: 40-189 people potentially reached

Research social networks: 559-4066 people potentially reached


4. VERSION MATTERS


9.7%

77.3%

9.3% 3.8% 5.5%

83.9%

7.3% 3.3%

Preprint Published Accepted MSS Other

When sharing scholarly articles, are you more likely to share Own Work Other's Work


My feeling is that I’ve written this, this is my work, they’re publishing it. I’m not getting paid for this, they’re benefiting from this. I’m a little fish . . . Let them come for me.

The main point should be that I share a version that is evidentially the peer-reviewed version (with the publisher and the journal stamp on it).


5. THE LIBRARY IS KEY


Average number of downloads for last research project:

65

Average number of downloads for last teaching term:

26


54.1%

9.4% 8.8%

18.9%

7.0% 6.1%

35.4%

5.8% 5.7% 8.5%

3.4% 6.1%

0%

10%

20%

30%

40%

50%

60%

Library Otherdatabases

Institutionalrepositories

Publisherwebsites

Researchsocial

networks

Otherwebsites

What percentage of your downloads were from the following sources?

Research Teaching


But, library download counts may underestimate use, because when returning to an article …

61.3%

11.7%

57.3%

9.4%

Saved copy Download again

Research Teaching


Sharing is a natural behavior

• Sharing supports research & disseminating results

• Convenience is a key concern

• Sharing channels chosen for fit to

work style • New policies and measures must

fit preferences & likely behavior


Calculating sharing metrics: Possible approaches

Lorraine Estelle


Range of Systems: Key findings include: • Lack of global usage data standards makes quantifying sharing

problematic.

• Difficult to obtain data on authors’ sharing via email and cloud services.

• Data obtained is likely to be out of date rather quickly.


Two suggested approaches

1. Data-based approach employing usage data from publishers and citation data – where such data is comprehensive and reliable.

2. Survey-based approach using an online community as a basis for monitoring sharing behavior over time.


A data-based approach to measuring the extent of scholarly materials sharing

1. Data on article sharing by formal and informal methods will be difficult to collect reliably owing to the large number of channels used and the lack of global standards

2. The mix of sharing methods used varies from discipline to discipline and from year to year

3. It would make sense to supplement a data-based approach to measuring post-download usage with regular surveys of the scholarly community


Full-Text Sharing Calculator • Can a calculator be derived from the findings? • If not, can a suggested range of “sharing” be derived from the

findings?


Problems with a calculator…

• Ranges and SE are high • Outliers skew results • Demographics matter a lot


A range of sharing, in lieu of a calculator, may be the answer. This range can come in the form of an overall range (derived from all respondents) or categorized ranges (derived from selected identifying categories).


Demographics sometimes matter…

• Researchers age 39 and younger share less through formal methods

• Researchers age 40-59 rely more on formal sharing methods and post slightly less, but to outlets that reach more people


Demographics sometimes matter…

Social scientists share more via social media.


Alternatives • Any estimate on number of times an article is shared per

download depends on many things.

• An alternative to an exact calculator would be to apply a Confidence Interval to the findings and provide an estimated “range of sharing”, with an lower bound and upper at a 95% level of confidence.

• This range could in-turn be combined with download

numbers to estimate a level of actual, post-download usage.


Collaborative initiatives with

• DOI Event Tracking – Open, common infrastructure to track activities

around DOIs recording user content interactions

• Distributed Usage Logging – Peer-to-peer system for the exchange of

usage data on user content interactions

From download metrics to downstream impact => implications for Elsevier

Oct 2015

Wouter Haak [email protected]


• The graph shows how many users have joined Mendeley groups year on year between 2009-2015 • The results were calculated based on groups with more than 1 member • The 2015 extrapolation assumes that the last 10 months’ growth will continue towards end of the year

Source: Mendeley analytics team

49%

17% -15%

17%

165%

697%

At Elsevier / Mendeley we also see that online sharing and collaborating is increasingly important (1)

• The graph shows how many documents were added to the groups by users year on year between 2009-2015 • The results were calculated based on groups with more than 1 member • The 2015 extrapolation assumes that the last 10 months’ growth will continue towards end of the year Source: Mendeley analytics team

148%

546%

46%

3% 24%

14%

At Elsevier / Mendeley we also see that online sharing and collaborating is increasingly important (2)

1) For the researcher: sharing in

2) For the institution: collaborations in

3) With publishers and platforms: principles for sharing on SCN’s

Actions undertaken and next steps – based on this research

40


41


45


48

• SCNs are platforms that allow researchers to develop and maintain professional relationships

• Typically they enable

- Discussion or research and information exchange - Sharing of research results, articles and data

• SCNs have grown substantially* in recent years, particularly in the “big”

three – Academia.edu, ResearchGate, and Mendeley

• STM working group identified 40+ such networks

*CCC/CLA Report on Scholarly Collaboration Platforms, M. Ware, November 27,2014 (CCC/Outsell)

Social Collaboration Networks (SCNs)

49

STM assembled a working group in September 2014 to examine the current landscape of article sharing via SCNs and publishers’ role

Ten publishers, both for profit and not-for-profit, and some of whom owned or interacted with SCNs, took part

They produced the draft set of ‘voluntary principles for article sharing on scholarly collaboration networks’ and posted it for a two month STM Open Consultation in early 2015, and invited wide participation among all stakeholder groups

STM Consultation on Article Sharing

50

The principles are meant to: • clarify how, where and what content should be shared using

these networks and sites,

• improve the experience for all stakeholders

• encourage publishers and SCN’s to work together to facilitate sharing, which benefits researchers, institutions, and society as a whole

• All comments and a summary posted on the STM website • Key recommendation: Principles should be: clear, simple, uniform

Voluntary Principles for Article Sharing on SCNs

51

• Based on the community’s constructive feedback, the working group posted a revised set of principles August 28.

http://www.stm-assoc.org/stm-consultations/scn-consultation-2015/

• Posting includes an explanation of the changes made to the principles as a result

Response to STM’s Open Consultation

52


• Publishers commit to facilitate the dissemination and discovery of their authors’ scholarly articles.

• Sharing should be allowed within a research collaboration group (to be defined by the group) - Allow article sharing between subscribers and non-subscribers

within the group - Include members of the wider public participating with the group - Include commercial researchers, subject to publisher policy or

appropriate licensing • Publishers and libraries should extend their collective use of standards

such as COUNTER to quantify article use on networks.

• Publishers and standards organizations should continue to work together on tools that facilitate sharing (article versioning and access rights metadata).

• Publisher policies on research collaboration group sharing and public posting of articles should be clear and easily discoverable.

Voluntary principles: key points

53

Publisher posting policy - Several publishers (e.g. NPG, Wiley, Elsevier) have issued policies that address posting via SCNs.

Article tagging project - To support implementation a sub-working group is tasked with demonstrating a technical solution based on tagging xmp metadata in scholarly articles

The SCN working group continues to reach out for wider involvement, including non-publisher owned networks

Comments/updates on progress, email [email protected] Updates will continually be posted on:

www.stm-assoc.org/stm-consultations/scn-consultation-2015/

Next steps

54

http://www.stm-assoc.org/2015_06_08_SCN_Article_Tagging_Project_progress.pdf


Seeking your involvement and/or endorsement http://www.stm-assoc.org/stm-consultations/scn-consultation-2015/

Alternatively: Google for “STM principles” or “STM consultation”

Next steps

55





From download metrics to downstream impact => implications for Elsevier

56

the secret life of articles: from download metrics to downstream … · 2016-02-17 · the secret...

Documents