cilip arlg '12: measuring impact and quality
TRANSCRIPT
![Page 1: CILIP ARLG '12: Measuring impact and quality](https://reader034.vdocuments.net/reader034/viewer/2022042717/55d5330bbb61eb07558b45e9/html5/thumbnails/1.jpg)
Measuring impact and quality
Workshop: Measuring the use of online resources Tom Edmonds
OpenAthens Product Manager
![Page 2: CILIP ARLG '12: Measuring impact and quality](https://reader034.vdocuments.net/reader034/viewer/2022042717/55d5330bbb61eb07558b45e9/html5/thumbnails/2.jpg)
Maximising the ROI for library budgets
• Lightening demo!
• Where does your library budget go?
• What challenges do you face?
• Some solutions
• YOU
• Case studies
![Page 3: CILIP ARLG '12: Measuring impact and quality](https://reader034.vdocuments.net/reader034/viewer/2022042717/55d5330bbb61eb07558b45e9/html5/thumbnails/3.jpg)
www.eduserv.org.uk
Eduserv - OpenAthens
1994 2000 2008 2009 2010 2011 Today…
• Athens developed & first web service launched
• Athens adopted by NHS England in the UK
• Athens became OpenAthens
• 33% of the Australian healthcare market adopts OpenAthens - SAH - DoH Victoria
• US Department of Veteran Affairs adopts OpenAthens
• US Navy Medicine Information Systems Support Agency adopts OpenAthens
• > 4,000 customers use OpenAthens
• > 4 million users worldwide
[Lightning demo]
![Page 4: CILIP ARLG '12: Measuring impact and quality](https://reader034.vdocuments.net/reader034/viewer/2022042717/55d5330bbb61eb07558b45e9/html5/thumbnails/4.jpg)
Identity & Access Management - markets
Academic
Healthcare
Publishing
Public
Sector
Commercial
[Lightning demo]
![Page 5: CILIP ARLG '12: Measuring impact and quality](https://reader034.vdocuments.net/reader034/viewer/2022042717/55d5330bbb61eb07558b45e9/html5/thumbnails/5.jpg)
www.eduserv.org.uk
Some library stats.
![Page 6: CILIP ARLG '12: Measuring impact and quality](https://reader034.vdocuments.net/reader034/viewer/2022042717/55d5330bbb61eb07558b45e9/html5/thumbnails/6.jpg)
www.eduserv.org.uk
Library developments
Print and online
Online
100%
2007
2009
1999
Source: Ebsco
![Page 7: CILIP ARLG '12: Measuring impact and quality](https://reader034.vdocuments.net/reader034/viewer/2022042717/55d5330bbb61eb07558b45e9/html5/thumbnails/7.jpg)
www.eduserv.org.uk
Where does your library budget go?
Resources
Personnel
Buildings
Other
![Page 8: CILIP ARLG '12: Measuring impact and quality](https://reader034.vdocuments.net/reader034/viewer/2022042717/55d5330bbb61eb07558b45e9/html5/thumbnails/8.jpg)
www.eduserv.org.uk
Where does your library budget go?
39% 25%
13%
23%
Resources
Personnel
Buildings
Other
Source: commercial research company
![Page 9: CILIP ARLG '12: Measuring impact and quality](https://reader034.vdocuments.net/reader034/viewer/2022042717/55d5330bbb61eb07558b45e9/html5/thumbnails/9.jpg)
www.eduserv.org.uk
Where does your resources budget go?
68%
15%
10% 3%
2%
2%
e-Journals
Online databases
Magazines & newspapers
Books
e-Books
Source: commercial research company
![Page 10: CILIP ARLG '12: Measuring impact and quality](https://reader034.vdocuments.net/reader034/viewer/2022042717/55d5330bbb61eb07558b45e9/html5/thumbnails/10.jpg)
www.eduserv.org.uk
Where does your e-resources budget go?
Source: Bath Spa University (2010)
![Page 11: CILIP ARLG '12: Measuring impact and quality](https://reader034.vdocuments.net/reader034/viewer/2022042717/55d5330bbb61eb07558b45e9/html5/thumbnails/11.jpg)
www.eduserv.org.uk
JISC Activity data programme
Looking for the link between library usage and student
attainment
“There is a statistically significant correlation across a number of
universities between library activity data and student attainment”
- thank you to the JISC & Huddersfield University
![Page 12: CILIP ARLG '12: Measuring impact and quality](https://reader034.vdocuments.net/reader034/viewer/2022042717/55d5330bbb61eb07558b45e9/html5/thumbnails/12.jpg)
www.eduserv.org.uk
![Page 13: CILIP ARLG '12: Measuring impact and quality](https://reader034.vdocuments.net/reader034/viewer/2022042717/55d5330bbb61eb07558b45e9/html5/thumbnails/13.jpg)
www.eduserv.org.uk
![Page 14: CILIP ARLG '12: Measuring impact and quality](https://reader034.vdocuments.net/reader034/viewer/2022042717/55d5330bbb61eb07558b45e9/html5/thumbnails/14.jpg)
www.eduserv.org.uk
50.3
56.8
53.1
56.4
59.3
49.5
53.1
49.8 50.9
48.5
39.2
36.1 36.4 37.2
34.3
26.1
22.8 23.0 22.3 21.1
10.0
20.0
30.0
40.0
50.0
60.0
70.0
2005/06 2006/07 2007/08 2008/09 2009/10
1 2:1 2:2 3
![Page 15: CILIP ARLG '12: Measuring impact and quality](https://reader034.vdocuments.net/reader034/viewer/2022042717/55d5330bbb61eb07558b45e9/html5/thumbnails/15.jpg)
www.eduserv.org.uk
![Page 16: CILIP ARLG '12: Measuring impact and quality](https://reader034.vdocuments.net/reader034/viewer/2022042717/55d5330bbb61eb07558b45e9/html5/thumbnails/16.jpg)
www.eduserv.org.uk
What are the challenges?
…for optimizing your library spend
![Page 17: CILIP ARLG '12: Measuring impact and quality](https://reader034.vdocuments.net/reader034/viewer/2022042717/55d5330bbb61eb07558b45e9/html5/thumbnails/17.jpg)
www.eduserv.org.uk
Challenges…
• Budgets
![Page 18: CILIP ARLG '12: Measuring impact and quality](https://reader034.vdocuments.net/reader034/viewer/2022042717/55d5330bbb61eb07558b45e9/html5/thumbnails/18.jpg)
www.eduserv.org.uk
Challenges…
• Budgets
• Suppliers
• Increasing subscription costs
• Lack of flexibility.
![Page 19: CILIP ARLG '12: Measuring impact and quality](https://reader034.vdocuments.net/reader034/viewer/2022042717/55d5330bbb61eb07558b45e9/html5/thumbnails/19.jpg)
www.eduserv.org.uk
Challenges…
• Budgets
• Suppliers
• Technologies • IP address management
• Authentication steps for both the library and publisher
• User ID setups
• A to Z list management
• OpenURL management
• Licence compliance policy management
• Support: troubleshooting, downtime, usage monitoring, training etc.
![Page 20: CILIP ARLG '12: Measuring impact and quality](https://reader034.vdocuments.net/reader034/viewer/2022042717/55d5330bbb61eb07558b45e9/html5/thumbnails/20.jpg)
www.eduserv.org.uk
The challenges…
• Budgets
• Suppliers
• Technologies
• politics
Source: stus.com
![Page 21: CILIP ARLG '12: Measuring impact and quality](https://reader034.vdocuments.net/reader034/viewer/2022042717/55d5330bbb61eb07558b45e9/html5/thumbnails/21.jpg)
www.eduserv.org.uk
The challenges…
• Budgets
• Suppliers
• Technologies
• politics
• Usability
![Page 22: CILIP ARLG '12: Measuring impact and quality](https://reader034.vdocuments.net/reader034/viewer/2022042717/55d5330bbb61eb07558b45e9/html5/thumbnails/22.jpg)
www.eduserv.org.uk
![Page 23: CILIP ARLG '12: Measuring impact and quality](https://reader034.vdocuments.net/reader034/viewer/2022042717/55d5330bbb61eb07558b45e9/html5/thumbnails/23.jpg)
www.eduserv.org.uk
23
![Page 24: CILIP ARLG '12: Measuring impact and quality](https://reader034.vdocuments.net/reader034/viewer/2022042717/55d5330bbb61eb07558b45e9/html5/thumbnails/24.jpg)
www.eduserv.org.uk
24
![Page 25: CILIP ARLG '12: Measuring impact and quality](https://reader034.vdocuments.net/reader034/viewer/2022042717/55d5330bbb61eb07558b45e9/html5/thumbnails/25.jpg)
www.eduserv.org.uk
25
![Page 26: CILIP ARLG '12: Measuring impact and quality](https://reader034.vdocuments.net/reader034/viewer/2022042717/55d5330bbb61eb07558b45e9/html5/thumbnails/26.jpg)
www.eduserv.org.uk
26
![Page 27: CILIP ARLG '12: Measuring impact and quality](https://reader034.vdocuments.net/reader034/viewer/2022042717/55d5330bbb61eb07558b45e9/html5/thumbnails/27.jpg)
www.eduserv.org.uk
27
![Page 28: CILIP ARLG '12: Measuring impact and quality](https://reader034.vdocuments.net/reader034/viewer/2022042717/55d5330bbb61eb07558b45e9/html5/thumbnails/28.jpg)
www.eduserv.org.uk
The challenges…
• Budgets
• Suppliers
• Technologies
• politics
• Usability
• Users.
![Page 29: CILIP ARLG '12: Measuring impact and quality](https://reader034.vdocuments.net/reader034/viewer/2022042717/55d5330bbb61eb07558b45e9/html5/thumbnails/29.jpg)
www.eduserv.org.uk
The “Path of least resistance”
“...an information [seeker] will
tend to use the most convenient
search method, in the least
exacting mode available.
Information seeking behaviour
stops as soon as minimally
acceptable results are found.”
Source: http://en.wikipedia.org/wiki/
![Page 30: CILIP ARLG '12: Measuring impact and quality](https://reader034.vdocuments.net/reader034/viewer/2022042717/55d5330bbb61eb07558b45e9/html5/thumbnails/30.jpg)
www.eduserv.org.uk
“...numerous studies have shown users
are often willing to sacrifice information
quality for accessibility. This fast food
approach to information consumption
drives librarians crazy. ‘Our information
is healthier and tastes better too’ they
shout.”
- Peter Morville (“Ambient Findability”, 2005)
Principle of Least Effort
30
![Page 31: CILIP ARLG '12: Measuring impact and quality](https://reader034.vdocuments.net/reader034/viewer/2022042717/55d5330bbb61eb07558b45e9/html5/thumbnails/31.jpg)
www.eduserv.org.uk
31
![Page 32: CILIP ARLG '12: Measuring impact and quality](https://reader034.vdocuments.net/reader034/viewer/2022042717/55d5330bbb61eb07558b45e9/html5/thumbnails/32.jpg)
www.eduserv.org.uk
Marketing your library
![Page 33: CILIP ARLG '12: Measuring impact and quality](https://reader034.vdocuments.net/reader034/viewer/2022042717/55d5330bbb61eb07558b45e9/html5/thumbnails/33.jpg)
www.eduserv.org.uk
Social media
“I found out all my students were looking
at material in the VLE but going straight
to Facebook to use the discussion tools
and discuss the material and the
lectures. I thought I might as well join
them and ask them questions in their
preferred space.”
- What is Web 2.0? Ideas, technologies and implications
for education. Paul Anderson. A JISC report
![Page 34: CILIP ARLG '12: Measuring impact and quality](https://reader034.vdocuments.net/reader034/viewer/2022042717/55d5330bbb61eb07558b45e9/html5/thumbnails/34.jpg)
www.eduserv.org.uk
Mobilise your library…
![Page 35: CILIP ARLG '12: Measuring impact and quality](https://reader034.vdocuments.net/reader034/viewer/2022042717/55d5330bbb61eb07558b45e9/html5/thumbnails/35.jpg)
www.eduserv.org.uk
Market your resources
Source: Gungahlin Library, Australia
flickr.com/photos/silversprite/5120507383
![Page 36: CILIP ARLG '12: Measuring impact and quality](https://reader034.vdocuments.net/reader034/viewer/2022042717/55d5330bbb61eb07558b45e9/html5/thumbnails/36.jpg)
www.eduserv.org.uk
Library portal
![Page 37: CILIP ARLG '12: Measuring impact and quality](https://reader034.vdocuments.net/reader034/viewer/2022042717/55d5330bbb61eb07558b45e9/html5/thumbnails/37.jpg)
www.eduserv.org.uk
Over to you...
What have you done/are doing to get more from your library budget?
![Page 38: CILIP ARLG '12: Measuring impact and quality](https://reader034.vdocuments.net/reader034/viewer/2022042717/55d5330bbb61eb07558b45e9/html5/thumbnails/38.jpg)
www.eduserv.org.uk
Case Study 1:
![Page 39: CILIP ARLG '12: Measuring impact and quality](https://reader034.vdocuments.net/reader034/viewer/2022042717/55d5330bbb61eb07558b45e9/html5/thumbnails/39.jpg)
www.eduserv.org.uk
Small UK HE institution
to implement Shibboleth
reduce burden of administering access
increase use of electronic resources
no IT resource
implement a local authentication
solution.
Objectives:
Constraints:
Solution:
![Page 40: CILIP ARLG '12: Measuring impact and quality](https://reader034.vdocuments.net/reader034/viewer/2022042717/55d5330bbb61eb07558b45e9/html5/thumbnails/40.jpg)
www.eduserv.org.uk
Project: duration – 9 months…
![Page 41: CILIP ARLG '12: Measuring impact and quality](https://reader034.vdocuments.net/reader034/viewer/2022042717/55d5330bbb61eb07558b45e9/html5/thumbnails/41.jpg)
www.eduserv.org.uk
Small UK HE institution –
• No access queries
• Handful of Athens queries!
• Integration with Google Docs
• Gmail access for post-graduates
• Increased resource usage
• 20,000 more sessions in 2010/11.
one year on…
![Page 42: CILIP ARLG '12: Measuring impact and quality](https://reader034.vdocuments.net/reader034/viewer/2022042717/55d5330bbb61eb07558b45e9/html5/thumbnails/42.jpg)
www.eduserv.org.uk
Small UK HE institution – number of logins
Year/
Month January
No. %
2009 16,460
2010 18,198 11
2011 22,930 39
February
No. %
8,247
11,345 37
27,359 232
March
No. %
13,860
21,276 54
27,446 98
![Page 43: CILIP ARLG '12: Measuring impact and quality](https://reader034.vdocuments.net/reader034/viewer/2022042717/55d5330bbb61eb07558b45e9/html5/thumbnails/43.jpg)
www.eduserv.org.uk
Case Study 2:
![Page 44: CILIP ARLG '12: Measuring impact and quality](https://reader034.vdocuments.net/reader034/viewer/2022042717/55d5330bbb61eb07558b45e9/html5/thumbnails/44.jpg)
www.eduserv.org.uk
Commercial research library
to simplify working in a fluid environment
manage changing need for information
fixed (if not falling) budget
re-negotiate licences
integrate library services.
Objectives:
Constraints:
Solution:
![Page 45: CILIP ARLG '12: Measuring impact and quality](https://reader034.vdocuments.net/reader034/viewer/2022042717/55d5330bbb61eb07558b45e9/html5/thumbnails/45.jpg)
www.eduserv.org.uk
User behaviour:
Copied in the library
Borrowed
Downloaded
![Page 46: CILIP ARLG '12: Measuring impact and quality](https://reader034.vdocuments.net/reader034/viewer/2022042717/55d5330bbb61eb07558b45e9/html5/thumbnails/46.jpg)
www.eduserv.org.uk
What are the consequences?
-
10,000
20,000
30,000
40,000
50,000
60,000
2007 2008 2009 -
1,000
2,000
3,000
4,000
5,000
6,000
2007 2008 2009
Subscription cost Number of downloads
![Page 47: CILIP ARLG '12: Measuring impact and quality](https://reader034.vdocuments.net/reader034/viewer/2022042717/55d5330bbb61eb07558b45e9/html5/thumbnails/47.jpg)
www.eduserv.org.uk
What are the consequences?
0.00
2.00
4.00
6.00
8.00
10.00
12.00
14.00
16.00
18.00
20.00
2007 2008 2009
Cost per download
![Page 48: CILIP ARLG '12: Measuring impact and quality](https://reader034.vdocuments.net/reader034/viewer/2022042717/55d5330bbb61eb07558b45e9/html5/thumbnails/48.jpg)
www.eduserv.org.uk
Three steps to improve ROI
• Re-negotiated subscriptions
• Implemented a local identity and access management
solution (OpenAthens)
• Implemented OpenURL and A to Z list management
using TDNet.
![Page 49: CILIP ARLG '12: Measuring impact and quality](https://reader034.vdocuments.net/reader034/viewer/2022042717/55d5330bbb61eb07558b45e9/html5/thumbnails/49.jpg)
www.eduserv.org.uk
Re-negotiated subscriptions
Publisher 1
• No movement on price or licencing terms
• No alternative arrangements available
• From 2010, the complete collection was replaced by a
collection of the most used titles.
![Page 50: CILIP ARLG '12: Measuring impact and quality](https://reader034.vdocuments.net/reader034/viewer/2022042717/55d5330bbb61eb07558b45e9/html5/thumbnails/50.jpg)
www.eduserv.org.uk
-
1,000
2,000
3,000
4,000
5,000
6,000
2007 2008 2009 2010
-
10,000
20,000
30,000
40,000
50,000
60,000
2007 2008 2009 2010
What were the consequences?
Subscription cost Number of downloads
![Page 51: CILIP ARLG '12: Measuring impact and quality](https://reader034.vdocuments.net/reader034/viewer/2022042717/55d5330bbb61eb07558b45e9/html5/thumbnails/51.jpg)
www.eduserv.org.uk
0.00
2.00
4.00
6.00
8.00
10.00
12.00
14.00
16.00
18.00
20.00
2007 2008 2009 2010
What were the consequences?
Cost per download
![Page 52: CILIP ARLG '12: Measuring impact and quality](https://reader034.vdocuments.net/reader034/viewer/2022042717/55d5330bbb61eb07558b45e9/html5/thumbnails/52.jpg)
www.eduserv.org.uk
The impact: publisher 1
• End users have less access
• Library service is curtailed
• Publisher loses income.
![Page 53: CILIP ARLG '12: Measuring impact and quality](https://reader034.vdocuments.net/reader034/viewer/2022042717/55d5330bbb61eb07558b45e9/html5/thumbnails/53.jpg)
www.eduserv.org.uk
Re-negotiated subscriptions
Publisher 2:
• 2007 ~35 journals, not much used
• 2008 cancelled most of them
• 2010 new arrangement with publisher
o download everything at a fixed price
o subscriptions commence when it is profitable.
![Page 54: CILIP ARLG '12: Measuring impact and quality](https://reader034.vdocuments.net/reader034/viewer/2022042717/55d5330bbb61eb07558b45e9/html5/thumbnails/54.jpg)
www.eduserv.org.uk
What were the consequences?
Cost of subscriptions Number of downloads
-
10,000
20,000
30,000
40,000
50,000
60,000
70,000
80,000
2007 2008 2009 2010
-
1,000
2,000
3,000
4,000
5,000
6,000
2007 2008 2009 2010
![Page 55: CILIP ARLG '12: Measuring impact and quality](https://reader034.vdocuments.net/reader034/viewer/2022042717/55d5330bbb61eb07558b45e9/html5/thumbnails/55.jpg)
www.eduserv.org.uk
What were the consequences?
Cost per download
0.00
5.00
10.00
15.00
20.00
25.00
30.00
35.00
40.00
2007 2008 2009 2010
![Page 56: CILIP ARLG '12: Measuring impact and quality](https://reader034.vdocuments.net/reader034/viewer/2022042717/55d5330bbb61eb07558b45e9/html5/thumbnails/56.jpg)
www.eduserv.org.uk
The impact: publisher 2
• End users have more access than they did 3 years ago
• The library service improves
• Publisher regains income.
![Page 57: CILIP ARLG '12: Measuring impact and quality](https://reader034.vdocuments.net/reader034/viewer/2022042717/55d5330bbb61eb07558b45e9/html5/thumbnails/57.jpg)
www.eduserv.org.uk
Conclusions…
• Negotiate
• Technology solutions
• Environment
• Social media
• Traditional tools
![Page 58: CILIP ARLG '12: Measuring impact and quality](https://reader034.vdocuments.net/reader034/viewer/2022042717/55d5330bbb61eb07558b45e9/html5/thumbnails/58.jpg)
www.eduserv.org.uk
Thank you for listening
Twitter: @edm0nds
Skype: tomedmonds.eduserv
Email: [email protected]