michael levine-clark - ithaka
TRANSCRIPT
Can Users Find Your Content Anymore? The Impact of Discovery
Service on Journal Use
ITHAKA Sustainable Scholarship
Pre-Conference
New York, October 21, 2013 Michael Levine-Clark, University of Denver
John McDonald, University of Southern California
Jason Price, SCELC
Does implementation of a discovery service impact usage of journal content?
Discovery tools are not
• Federated search tools
• Library catalogs
• Indexing and abstracting services (PsycInfo, Web of Science, etc)
Web-Scale Discovery Tools
• Single source for finding information
– Books
– Articles
– Local content
• Metadata and/or full text
• Content is pre-indexed and/or pre-harvested
• Single fast search
ILS
HathiTrust
MLA Bibliography
Institutional Repository
Publisher Metadata
Discovery Tool
Metadata
• Publisher-provided metadata
• Third-party metadata
– Journal or ebook aggregator
– Indexing service (EBSCO, ProQuest, MLA Bibliography)
• Full text
• MARC
Assumptions
• At any given institution, given a relatively stable user base, the total search effort will remain roughly the same.
– X students will have Y assignments and Z hours per day to search
Discovery tools
– Will take up an increasing amount of a finite time for searching
– Will alter the overall productivity of searches (users will find more or less)
– Will alter the overall efficiency of users (users will find more or less per search)
– Will draw users from other (more or less efficient) search tools
Given those assumptions
• Introducing a discovery tool may:
– Cause users to view more (or fewer) articles per search
– Cause users to view different versions of articles (publisher vs aggregator)
• This study:
– WILL NOT identify which (if any) of these causes is in effect
– WILL provide a rigorous, unbiased test of whether an effect can be detected
Identifying Institutions
• Survey of libraries
– Which discovery tool
– Implementation Date (month/year)
– Search box on library web page
– 149 responses
• 24 libraries
– 6 for each of the 4 major discovery tools
Library Demographics
• 21 US, 1 UK, 1 NZ, 1 Canada
• 10 ARL
• WorldCat book holdings
– Average: 1,114,193
– Median: 1,044,153
– High: 2,665,796
– Low: 298,365
Implementation Dates
• 2010
– 3 libraries (Discovery 2)
• 2011
– 19 libraries
• 2012
– 2 libraries (Discovery 2)
Methodology
• Compare COUNTER JR1 data for 12 months before and 12 months after implementation date. Implementation date counted as month 12 of year 1.
Jun
e 2
01
0
Star
t
Imp
lem
en
tati
on
M
ay 2
01
1
May
20
12
En
d
Year 1 Year 2
Journal Usage
• Title must be available to that library for all 24 months
– Net change in usage from 12 months prior to 12 months after
– All titles added together for each institution
– Titles not counted in analysis unless 12+ uses in both years
Publishers
• 6 publishers
– 56,286 titles (not deduped)
Journal Availability by Institution
• Title counts vary (different institutions have different packages)
– Average: 2,345
– Median: 2,416
– High: 5,531
– Low: 125
Journal Availability by Discovery Tool
• Discovery 1 – Total (combined): 23,717
– Average: 3,952
– Median: 3,862
– High: 5,531
– Low: 2,441
• Discovery 2 – Total (combined): 5,744
– Average: 957
– Median: 449
– High: 2,896
– Low: 161
• Discovery 3 – Total (combined): 10,116
– Average: 1,686
– Median: 1,716
– High: 3,666
– Low: 125
• Discovery 4 – Total (combined): 16,705
– Average: 2,784
– Median: 2,603
– High: 3,702
– Low: 2,194
Net Change
Discovery 4, 20%
Discovery 3, 7%
Discovery 2, 0%
Discovery 1, -4%
-10% -5% 0% 5% 10% 15% 20% 25%
Observations
• Variations by publisher within each discovery tool
• Variations by institution within each discovery tool
• Some publishers are net losers, some net winners
Next Steps
• Charleston Conference (November 2013)
– Additional publishers
– Deeper analysis of existing publisher data
• UKSG (April 2014)
– Control group
– Additional libraries
– Comparison by library type
