BEYOND COUNTER-COMPLIANTTHE IMPORTANCE OF ASSESSING E-RESOURCES REPORTING TOOLS

University Library

Kelly BlanchatElectronic Resources Support Librarian

Excel Handout: https://tinyurl.com/y7bvlg27

360 COUNTER @ YALE

Out-source usage statistics harvestingTo allocate staff resources elsewhere

Consolidate usage across time and providersTo enhance reporting

NOT to calculate Cost-Per-Use, not using SUSHI

360 COUNTER: HOW IT WORKS

Part 2: INTOTA ASSESSMENT

A. Consolidates usage• Across time • Across providers

B. Assessment (i.e.: CPU)

Part 1: 360 COUNTER

A. Collection of usage statistics• “Data Retrieval Service”• SUSHI

B. Archives usage data

Jan Feb

0 0

1 42

3 7

89 0

Jan Feb

2 4

16 0

0 0

1 0

Jan Feb

2 0

7 6

0 0

26 87

Jan Feb

23 7

0 0

1 1

0 9

Jan Feb

27 11

32 48

4 8

106 96

IMPLEMENTATION TIMELINE

Prior to 2015 Spring / Summer 2015

Administrative credentials added to 360 COUNTER for Data Retrieval Service (DRS)

YUL staff manually uploads historical statistics to 360 COUNTER

YUL staff manually retrieves usage statistics every 6 months

YUL staff uploads statistics to a local website

The DRS team finalizes the first retrieval of Yale’s usage statistics, for January – June 2015

September 2015 November 2015

Collection Development retrieves usage statistics for 2015 OUP BR2 within IA

Oxford English Dictionary is MISSING from consolidated report, as are 342 titles, and 29,170 total uses

October 2015

Begin exploring the consolidated system

Open tickets for data corrections

Pull sample reports from Intota Assessment (IA)

…WHAT DID THAT SAY?

Oxford English Dictionary is MISSINGfrom consolidated report, as are 342 other titles, and 29,170 total uses

November 2015

PHASE 1: TITLE-LEVEL ANALYSIS

Excel V-LOOKUP on ISSN/ISBN and title between COUNTER report and consolidated report

PHASE 1: SAMPLE FINDINGS

360 COUNTER

Missing Titles (20 total titles):• A Dictionary of Geography• The Oxford Dictionary of Plays

Variant Usage (34 total titles):• A Dictionary of Economics• The Oxford Classical Dictionary

Variant ISBN (63 total titles):• Between Two Empires: Race,

History, and Transnationalism in Japanese America

INTOTA ASSESSMENT

Duplicate entries / editions scrubbed

Usage from distinct editions merged

ISBN changed from raw COUNTER

2015 Oxford BR2

REASON

REASON

REASON

PHASE 1: OUTCOME

Title-level example errors submitted to ProQuest for the 2015 Oxford BR2.

After response and fix, only 2 titles and 20 uses remained outstanding.

PHASE 1: SUCCESSFUL, NOT SUSTAINABLE

Favorable results!

Time consuming and really not fun at all

Yale has over 100 providers in 360 COUNTER

PHASE 2: SIMPLIFIED TO COLLECT TOTALS

Collection Date: June 2016

VERY SIMPLE SUBTRACT FORMULA BUILT-IN

PHASE 2: IN PRACTICE

CONSTANT: REPORTSVARIABLE: TIME

RESULT: ACCURACY, OVER TIME

High-level data collection from 8 content providers for JR1 & BR2 submitted to ProQuest’s 360 COUNTER team.

PHASE 2: OUTCOME

360 COUNTER INTOTA ASSESSMENT

HOW

Received granular information on HOW data is consolidated through normalization to the Authority Title.

Jan Feb

0 0

1 42

3 7

89 0

Jan Feb

2 4

16 0

0 0

1 0

Jan Feb

2 0

7 6

0 0

26 87

Jan Feb

23 7

0 0

1 1

0 9

Jan Feb

27 11

32 48

4 8

106 96

CONSOLIDATION = NORMALIZATION

Normalization to the Authority Title will affect overall title count because…

• When duplicates have matching ISSN and title (i.e.: full match), usage is merged onto 1 entry

• When a title has variant data points (DOI, ISSN) over time, titles may display multiple times

• When duplicate titles have the same ISSN but distinct titles, usage is picked from 1 version

EXAMPLE: SAME ISSN, VARIANT NAMES

2015 Springer JR1: When duplicate titles have the same ISSN butdistinct titles, usage is picked from 1 version

EVALUATION

Yale has so much data. High-level analysis has helped us understand WHAT exactly what happening to our data better behind the scenes.

And… it is complex because it is connected to the knowledgebase (and the knowledgebase is complicated).

1) Do the “rules” for title normalization/consolidation make sense?• How does it affect potential CPU reporting?

2) Which results ….• should trigger a bug fix?

• are a result of incorrect COUNTER data?

• are a result of over title normalization?

EVALUATION

EVALUATION: USE MY TEMPLATE!

https://tinyurl.com/y7bvlg27

WHERE WE ARE IN 2017

Prior to 2015 Spring –Summer 2015

Administrative credentials added to 360 COUNTER for Data Retrieval Service (DRS)

YUL staff manually uploads historical statistics to 360 COUNTER

YUL staff manually retrieves usage statistics every 6 months

YUL staff uploads statistics to a local website

The DRS team finalizes the first retrieval of Yale’s usage statistics, for January – June 2015

OED is MISSING from consolidated report, as are 342 titles, and 29,170 total uses

Sept – Nov 2015 April 2016Dec 2015 –March 2016

Phase 1 title-level analysis comparing raw COUNTER reports to consolidated reports in Intota Assessment

Phase 2 begins with high-level analysis, gathering totals between reports

August 2016

Lots of conference calls with 360 COUNTER

Planning for Phase 3 begins, using ARL statistics as a pilot project; transforming COUNTER as data source in Tableau

Lots more conference calls with 360 COUNTER

January 2017

Pilot with ARL stats is complete, internal testing begins in Access, Tableau

March 2017

Add more stats for ARL providers to Tableau

Discuss more robust data solutions (MySQL, Python)

June 2017

NASIG – HELLO!

The Future….

YOU BET THERE’S A PHASE 3!

August 2016

Planning for Phase 3 begins

Transforming COUNTER reports as data source for Tableau

Yay!

• We’re putting ourselves in charge, removing guess-work and assuming “burden” of accepting all data

• Still leveraging the use of 360 COUNTER’s data retrieval – YAY!

• More easily import ILS $$$ data into Tableau to merge with usage

PHASE 3: “LET COUNTER BE COUNTER”

PHASE 3: COUNTER AS DATA SOURCE

STANDARD COUNTER REPORT, JR1

COUNTER REPORT TRANSFORMED AS A DATA SOURCE WITH EXCEL

TABLEAU PLUG-IN

REMEMBER THIS SPRINGER JOURNAL?

PHASE 3: TABLEAU, PROOF OF CONCEPT

2015 Springer JR1: When duplicate titles have the same ISSN butdistinct titles, usage is picked from 1 version – now fixed!

Problem: We collect data on calendar years, but ARL needs fiscal years…(UGH)

PHASE 3: PILOTING

PHASE 3: PILOT WITH ARL STATS

• It’s self service! We can set-up standard views set-up in Tableau for subject librarians to retrieve

• Move from less manual (Excel, Access) to more automated and robust (Python, SQL)

• Data visualization!

• Self-service for renewals

• NO QUESTIONS about data, built on COUNTER standards

NEXT GOALS…

PREPARING FOR WHAT’S AHEAD

Phases 1-3

SUSHI, COUNTER R5