From Spreadsheets to SUSHI

Five years of assessing useof e-resources

Leslie Farison: Appalachian State UniversityKristin Calvert: Western Carolina University

In the beginning

Create SpreadsheetPopulate Fields

Acquisitions: Title, order record, publisher platform, costCollections: usage statistics, cost per use, vendor site access information (url; login; password)

Vendor Site Access

URL Log in

Password

Which Reports?

Vendor Reports

Counter Reports

Which Reports?

• COUNTER provides a somewhat comparable model for providing usage data.

• COUNTER reports are the most standardized so use those where available.

What measure is meaningful?

Sessions

Searches

Abstracts

Full Text

Selected Measures

• Sessions and searches not meaningful

• Full text for full text resources

• Abstracts for index/abstract resources

Summer 2009Cancellation Project

• Added fields to spreadsheet:• Selector• One time $• Date acquired• Invoice date• Ebsco SA• Fund code• Notes & cancellation restrictions

ERM

• Added in 2009 • Usage module not ready• Other problems encountered• Discontinued late 2010

Preparing for CY 2013

• Many changes to take into consideration due to COUNTER R4.

• Released April 2012

• Implementation date December 2013

COUNTER Release 4

• A single, integrated Code of Practice covering journals, databases, books, reference works and multimedia content.

• No more sessions; result clicks & record views.

• Increased learning curve.

Reports Types

Release0

5

10

15

20

25

7 8

17

23

OneTwoThreeFour

Glossary Terms

Release0

102030405060708090

100

51 60 7293

OneTwoThreeFour

R4: Journal Reports

COUNTERReports

Description Status

Journal Report 1 Number of Successful Full-Text Article Requests by Month and Journal M

Journal Report 1 GOA Number of Successful Gold Open Access Full-Text Article Requests by Month and Journal

M

Journal Report 1a Number of Successful Full-Text Article Requests from an Archive by Month and Journal

O

Journal Report 2 Access Denied to Full-Text Articles by Month, Journal and Category M

Journal Report 3 Number of Successful Item Requests by Month, Journal and Page-type

O

Journal Report 3 Mobile Number of Successful Item Requests by Month, Journal and Page-type for usage on a mobile device

O

Journal Report 4 Total Searches Run By Month and Collection O

Journal Report 5 Number of Successful Full-Text Article Requests by Year-of-Publication (YOP) and Journal

M

Database & Book Reports

COUNTER Reports

Description Status

Database Report 1 Total Searches, Result Clicks and Record Views by Month and Database

M

Database Report 2 Access Denied by Month, Database and Category M

Platform Report 1 (formerly Database Report 3)

Total Searches, Result Clicks and Record Views by Month and Platform

M

COUNTER Reports

Status

Book Report 1 Number of Successful Title Requests by Month and Title M

Book Report 2 Number of Successful Section Requests by Month and Title M

Book Report 3 Access Denied to Content Items by Month, Title and Category M

Book Report 4 Access Denied to Content items by Month, Platform and Category

M

Book Report 5 Total Searches and by Month and Title M

Other Reports

COUNTER Reports

Status

Multimedia Report 1 Number of Successful Full Multimedia Content Unit Requests by Month and Collection

M

Multimedia Report 2 Number of Successful Full Multimedia Content Unit Requests by Month, Collection and Item Type

O

Title Report 1 (formerly Journal/Book Report 1)

Number of Successful Requests for Journal Full-Text Articles and Book Sections by Month and Title

O

Title Report 1 Mobile Number of Successful Requests for Journal Full-Text Articles and Book Sections by Month and Title (formatted for normal browsers/delivered to mobile devices AND formatted for mobile devices/delivered to mobile devices

O

Title Report 2 Access Denied to Full-Text Items by Month, Title and Category

O

Title Report 3 Number of Successful Item Requests by Month, Title and Page Type

O

Title Report 3 Mobile Number of Successful Item Requests by Month, Title and Page Type (formatted for normal browsers/delivered to mobile devices AND formatted for mobile devices/delivered to mobile devices

O

Workflow

Librarians must download spreadsheets a file at-a-time from each vendor site then load them.

Librarians need a more efficient method for getting the data.

Counter + NISO = SUSHI

Standardized Usage Statistics Harvesting Initiative

SUSHI

• Not a stand alone application• Works with another system• Retrieve COUNTER reports• XML format

Not a panacea

To date, SUSHI has failed to live up to its promise, largely because of the lack of systems available to take advantage of the protocol and the ongoing irregularity of vendors' applications of the COUNTER standard.

Hybrid Models

• Many universities are adding ERMs and/or third party assessment products and are using SUSHI to gather some data, mostly for journals.

• Continuing to use other methods, such a spreadsheets, for others.

• Fry, A. (2013). "A Hybrid Model for Managing Standard Usage Data: Principles for e-Resource Statistics Workflows." Serials Review 39(1): 21-28.

EBSCO Usage Consolidation

• EBSCONET Usage Consolidation Product launched Jan 2012

• Reviewed in April 2012• Revisited June 2013• Added in July

E-resource Statistics at Western Carolina University•Statistics we collected were…

▫Driven by reporting agencies ▫Entirely reactive▫Time consuming▫Lack trending

Goals for Western Carolina Univ•Reduce time spent maintaining database

statistics spreadsheet.

•Make cost-per-use figures more accessible to collection development.

•Be able to use journal statistics holistically and move away from title-by-title requests▫“Do we have use data for Nature?”

EBSCO Usage Consolidation

COUNTER-compliant statistics are loaded from all sources and matched against EBSCO’s AtoZ Knowledgebase

SUSHI enabled for harvesting statistics automatically

Report tools look at use by platform, title, publisher, etc.Quickly identify most- and least-used resources

Data is fed into EBSCOnet Subscription Management to combine use and cost data for incredibly powerful collection development analysis

Configuring Usage ConsolidationSelected vendors where EBSCO loads/manages statistics for you. These vendors cannot be changed later.

Configured remaining e-resource providers, selecting SUSHI harvesting as often as possible. Troubleshooting required.

Manually load statistics for remaining providers.

Load historical statistics.

Demonstration : Load usage

Access to credentials

Manually trigger SUSHI request

Requires staff to touch each exception and ignore a title or find a match.

Process & Clean-up

Check resources have successfully matched against the EBSCO AtoZ knowledgebase.

Requires some staff time to deal with exceptions report, though the size of these reports shrink each month as the system learns from you.

Decide how often to load statistics manually.

Set-up period

*2-3 weeks

Initial clean-ups*8+ hours

Quarterly loads

*4-6 hours

Monthly clean-up

* 1 hour or less

ReportsAvailable Criteria Example

• Report types

• Date range

• Single metric per report

• Limit to one platform

• Limit to top/bottom use

• Limit to use amount

• Estimate missing usage

• Title, Database or COUNTER

• Sessions or FT requests

• IngentaConnect

• 10 titles with most use

• All titles with use <= 5

• Project use for the remainder of the year

Reports, cont.

Exporting Limitations

• Export to Excel, csv, tsv

• Includes:▫ Standard COUNTER

report fields

▫ Title sort field

▫ EBSCO A-Z Holdings Y/N

• Unable to exclude aggregator usage (as a group) from title reports

• Must export reports to include cost information, OR look up title in Subscription Management

Journals in UC• With integration with EBSCO AtoZ, journal

statistics work really well.

• COUNTER reports from publishers can include non-subscribed titles. UC lets you limit your reports to titles in your collection.

• Spreadsheet formatting from publisher can require tweaking before loading in UC.

• Publishers can be slow to respond to requests for SUSHI permissions.

Databases in UC• WCU still using spreadsheets

• Many, many database providers are not COUNTER compliant.

• Usage loading service is great for EBSCOhost databases – there are a LOT of titles in the DB1 and JR1 reports.

• Database title matching is problematic and there is no standard number to use.

▫We cannot load stats for some because UC can’t match the name of the database to the entry in the knowledgebase (bug).

E-Books in UC

•WCU testing this now

•WCU doesn’t use EBSCO AtoZ to manage our ebooks

•No way to integrate cost/order information

▫Also true for databases and journals not acquired through EBSCO

▫Must export report and calculate in Excel

Un

der C

on

stru

ctio

nIf there are some functional issues, and we have not eliminated spreadsheets entirely, and some calculations still need to be done in Excel, why do we want to use Usage Consolidation?

Gain

ing

Effi

cie

ncie

sThe small amount of work it takes to load data for journal subscriptions allows us to spend more time to use the data to do our jobs then to have the data collection become our entire job.

Serials ReviewData for decision-making at Western Carolina University

2011 process versus 2013 process

More than usage data

Usage

• Publisher site

• Databases

Access

• Alternate availability

Cost

• Inflation

• Cost-per-use

CollectionManageme

ntDecisions

2011 = Access Database

3 solid months of work and

knowledge of Access

databases

2013 = Usage ConsolidationEBSCO Usage Consolidation & EBSCO Subscription Management Analytics

Journal Packages, analytics

Journal Packages, analytics

Journal Packages, title list

A Vision for the Future• SUSHI implementation seems to be

better suited for a group of libraries.

• A shared portal for some group of libraries in North Carolina.

• Perhaps the UNC System.• Libraries participating in the

Carolina Consortium.• Samples of portals include:

JISC - JUSP

SUSHI: Delivering Major Benefits to JUSPThe use of SUSHI has demonstrably saved JUSP and the UK HE community hundreds of thousands of pounds of staff costs since its inception; add in an estimated 97%+ of time in data collection and processing every month and the dual benefits are enormous; as more publishers join, this efficiency will continue to increase. In an age of funding cuts and budget restrictions, the combination of JUSP and SUSHI thus affords an economical, high-quality alternative to the previously onerous and unending task of journal statistics gathering and management.

Shared Tools for analysis of e-resources in Higher Education Libraries in Sweden (

NordLIC)

The perfect storm

• Counter R4: December 2013• New data elements• More stringent auditing• Increasing # vendors SUSHI

compliant• New implementation protocol• A single portal