developing a methodology for evaluating the cost-effectiveness of journal packages

ER&L 2010 February 1-3, 2010 Austin, Texas

1

Developing a Methodology in Evaluating Cost Effectiveness of Journal Packages

Nisa Bakkalbasi

Head, Electronic Collections

Yale University Library


2

About today’s talk

Since 1990's many academic libraries signed multi-year contracts to subscribe to electronic journal packages from large publishers.

With recent collection budget cuts, many academic libraries are having second thoughts about journal package arrangements, which force them to spend too much money on journals they don't need and which make it difficult to pay for journals from smaller publishers and scholarly monographs.

In this presentation, I will share a methodology we have developed for evaluating the cost effectiveness of journal packages.


3

Outline of methodology

Identify the issue.

Collect and analyze data for decision making.

Identify and review options: pros and cons.

Discuss decision time-line and implementation.


4

To identify the issue, we need:

An overview of the product

How funding was established?


5

To collect and analyze data, we use:

Data source: COUNTER-compliant usage reports Accessible titles list from consortium or publisher Publisher’s title-by-title list price Subject categories assigned by the publisher or locally

Data analysis technique Exploratory data analysis using quantitative and qualitative

variables

Tool/software Excel PivotTable Report


6

To review options and discuss pros and cons, we need:

Pricing model or the sales model Base-value Subscribed/non-subscribed titles, if relevant Cancellation allowance Price cap DDP rate Perpetual (or post-cancellation) access rights Duration of the contract Transfer titles Third-party titles


7

Let’s get started!

Using a case study, we will walk through a step-by-step evaluation of a journal package!


8

Issue

An Academic Library has been subscribing to the ABC Publishers Complete Journals Collections since 2005. The initial base-value of the journal package arrangement was established based on historical print spend in 2004.

The staff wants to gain an understanding of whether the package is worth retaining when it is up for renewal -- i.e. are we paying less in the package than we would if we broke the package and had to pay title-by-title for the journals we need to retain access to.

Reductions in collection budget and different usage patterns among subjects are reasons to consider dissolving the journal package arrangement.

9

Product Overview

ABC Publishers Complete e-Journal Collection: Publishes academic and research journals. Publishes in partnership with learned societies. Publishes in physical and life sciences, medicine, social

sciences, humanities, law, and mathematics. Publishes 200.



10

Data Collection

Download COUNTER Journal Report 1: Number of Successful Full-Text Article Requests by Month and Journal.

Exclude all titles that are based on separate pricing models. Exclude journal archives usage if backfile purchase was a

separate acquisition. Obtain a list of “Accessible Titles” list, which contains a list

of all titles (subscribed and non-subscribed) included in the package arrangement.

Obtain a list of e-only list prices. Aggregate all data in a single spreadsheet.


11

Producing data

Each record should contain the following variables: Journal title ISSNs Usage count (YTD Total from the COUNTER JR1 report) List price Categories for subscribed and non-subscribed variables Categories for subjects


12

Enhance data to facilitate decisions

Calculate cost-per-use (CPU)

CPU= e-only price for the title /YTD Total (per year)

Using ILL cost as a threshold, create two new categorical variables to filter for possible renewals and cancellations

Let’s take a look at an anonymized data!


13

Analysis: Title count

Subscribed vs. non-subscribed title counts

Subject Cluster NO YES Grand Total

Humanities 9 38 47

Law 4 21 25

Social Sciences 11 26 37

Science & Medicine 32 41 73

Mathematics 10 8 18

Grand Total 66 134 200


14

Analysis: Usage

Subscribed vs. non-subscribed titles usage counts


Humanities 1,057 7,775 8,832

Law 257 1,821 2,078

Social Sciences 1,450 4,314 5,764

Science & Medicine 10,544 60,246 70,790

Mathematics 204 1,384 1,588

Grand Total 13,512 75,540 89,052


15

Analysis: Usage

Cumulative Relative Frequency (%)

10% 20% 50% 80% 90% No use

1% (2) 2% (4) 8% (15) 24% (48) 40% (80) 0

10 % of use comes from two titles, 20 % of use comes from four titles, 50 % of use comes from fifteen titles, and so on.


16

Analysis: Cost

Subscribed vs. non-subscribed titles cost


Humanities $2,074 $10,725 $12,799

Law $1,666 $9,346 $11,012

Social Sciences $4,511 $10,363 $14,874

Science & Medicine $24,764 $42,071 $66,835

Mathematics $6,402 $9,198 $15,600

Grand Total $39,417 $81,703 $121,120


17

Analysis: Candidates for renewal cancellation (based on $10/ILL)

Count of possible candidates for renewal/cancellation

Subject CPU >= $10

CPU <= $10

Grand Total

Humanities 7 40 47

Law 13 12 25

Social Sciences 6 31 37Science & Medicine 8 65 73

Mathematics 16 2 18



18

Analysis: Candidates for renewal & cancellation (based on $25/ILL)

Count of possible candidates for renewal/cancellation

Subject CPU >= $25

CPU <= $25 Grand Total

Humanities 3 44 47

Law 6 19 25

Social Sciences 2 35 37

Science & Medicine 3 70 73

Mathematics 10 8 18



19


Cost of possible candidates for renewal/cancellation

SubjectCPU >= $10

CPU <= $10

Grand Total

Humanities $1,782 $11,017 $12,799

Law $6,738 $4,274 $11,012

Social Sciences $2,581 $12,293 $14,874

Science & Medicine $7,745 $59,090 $66,835

Mathematics $15,176 $424 $15,600

Grand Total $34,022 $87,098 $121,120


20


Cost of possible candidates for renewal/cancellation

Subject CPU>=$25

CPU<=$25 Grand Total

Humanities $588 $12,211 $12,799

Law $3,832 $7,180 $11,012

Social Sciences $649 $14,225 $14,874Science & Medicine $1,329 $65,506 $66,835

Mathematics $9,110 $6,490 $15,600

Grand Total $15,508 $105,612 $121,120


21

Methodology reveals

proof that users are using electronic resources.

value for money.

necessary but insufficient data to make strategic decisions.


22

Methodology does not reveal

The users’ experience or perception of the utility or value of a collection or service.

Low use can occur because the product’s user interface is difficult to use or because users are unaware that the product is available.


23

Questions?


24

Thank you for coming!

developing a methodology for evaluating the cost-effectiveness of journal packages

Technology