basics terminology nat lib stats lib meter zbw hh workshop
TRANSCRIPT
LibMeterLibMeter
LibMeter - Library eMetrics
Basics & TerminologyWorkshop ZBW-HH Part 1-2
Nutzungsanalyse elektronischer Informationsdienstleistungen für das praktische Bibliotheks-Management
Veranstalter: Berufsverband Information
Bibliothek e. V. (BIB)Landesgruppe Hamburg
6. November 2009
Referent:Dr. Peter Ahrens
Freier Referent
Assistenz:Tanja Haberkorn
Beta 0.8 2009-11-01
© Peter Ahrens, Cologne, 2009
LibMeterLibMeter
Nutzungsanalyse elektronischer Informationsdienstleistungen für
das praktische Bibliotheks-Management
Grundlagen Standortbestimmung & Perspektiven
Vorstellung Programm Überblick 1. Grundlagen und Begriffe 2. DBS & BIX Vorstelllung praktischer Teil – BIX Übungen 3. Nationale eUsage Jahres-Statistiken 4. COUNTER, OpenURL & u.a. Monatsstatistiken 5.Alternativ-Blöcke 6. Diskussion & Feedback
© Peter Ahrens, Cologne, 2009
LibMeterLibMeter
LibMeter Seminar
x
1. Intro
© Peter Ahrens, Cologne, 2009
LibMeterLibMeter
LibMeter Seminar
x
Introduction
© Peter Ahrens, Cologne, 2009
LibMeterLibMeter
Questions to StartAre you familiar with ? EZB / DBIS DBS / BIX Indikatoren-Raster COUNTER / Sushi OpenURL / SFX OPAC 2.0 Funktionen Web Analytics
© Peter Ahrens, Cologne, 2009
LibMeterLibMeter
How Needs, Services, Usage & Community Value are related ?
C O M M U N I T Y with N E E DS E R V I C E S
S C
O P
E
U T
I L
I T Y
USA
BIL
ITY
ACCE
SSIB
ILIT
Y
VISI
BIL
ITY
AW
AR
ENES
S
USAGE ( = ACCEPTANCE)VALUE - Performance
© Peter Ahrens, Cologne, 2009
LibMeterLibMeter
What Performance Indicators to measure ? Input
Resourcen: Finanzen, Räume, Personal, Bestände, PCs
Output Bestände, Angebote Besuche, Ausleihen, konv. Nutzung &
elektron. Outcome / Impact - Wirksamkeit
Z.B. Informationskompetenz, Wissen Z.B. Fähigkeiten / Verhaltensänderungen
© Peter Ahrens, Cologne, 2009
LibMeterLibMeter
How to collect data about Libary‘s eService Usage with eMetrics?4 Dimensions of Assessment Census counts (full):
externally generated, Vendor usage data (eg COUNTER)
Locally/internally generated usage data, (eg OpenURL)
Sample counts (random or non-random): externally generated, web survey data Internally generated, web survey/usage data
© Peter Ahrens, Cologne, 2009
LibMeterLibMeter
eMetrics and broader context of Library assessment not covered Integrated Assessment Instruments
LibQual / ServQual Measure users expectations and subjective
fullfillment of Library Services ISO 2789 und ISO 11620 Management Approaches
Balanced Scorecard Library Controlling Cycles
© Peter Ahrens, Cologne, 2009
LibMeterLibMeter
How to look at any Statistic ? Motivation (Why) Methods / Tools / Technologies (How) Limitations (Scope, Issues, Problems) Observations (Figures / Results) Interpretation (Changes, Trends, Pittfalls) Consequences (Decisions, Actions)
And remember: Don‘t trust any statistic you did not fake yourself !
© Peter Ahrens, Cologne, 2009
LibMeterLibMeter
Questions asked: How are we doing ? How high is a specific usage ?
(absolute/normalized) How does this compare to peer-
institutions? How does this compare to other services? What is the trend ?
Growth/Decline Acceleration, Slowdown Life-Cycle Phase
© Peter Ahrens, Cologne, 2009
LibMeterLibMeter
Usage „Stats“ Terms &
Difinitions
© Peter Ahrens, Cologne, 2009
LibMeterLibMeterBasic Terms I Log (item)
Record of Events Count = Value = Figure = Measure = Parameter
Numerical Quantity Data Variable (numerical) value; Here: Output Parameter!
Indicator Something giving information, indicating something
Statistics = „Stats“ Collection of numerical data Analysis and Interpretation of Statistical Data
© Peter Ahrens, Cologne, 2009
LibMeterLibMeterBasic Terms II Logging
Storing and preserving a File-record with detailed technical Information about all occuring service events (Web)
Usage (figures or statistics) Quantitative parameters/values derived from filtering
and processing Log-Files (aggregating, relating, normalizing)
Usage monitoring = analysing (web) usage figures over time (longitudinal) or between services (transversal)
Usability Degree of suitability of (web) service for purpose
© Peter Ahrens, Cologne, 2009
LibMeterLibMeterBasic Terms III Performance
Degree of Effectiveness (e.g. of work or a service) Ranking
Position in Relation to others / Working out Ranking Order
Rating classification of something on a scale according to
how much or how little of a quality it possesses Benchmarking
Comparing against a standard against which something can be measured
Best Practice
© Peter Ahrens, Cologne, 2009
LibMeterLibMeter
Basic Terms IVDescriptive Statistics Minimum / Maximum
Smallest value / Highest value Average (Artihmetic Mean)
Sum of all values divided by total number Median
Value in the middle of the range
Standard Deviation (Measure for average Distance) Variation Coefficient (Measure for Width of Distribution) Quantil, Percentil, Quartil … (->Indikatoren-Raster) (Skewness, Kurtosis …)
© Peter Ahrens, Cologne, 2009
LibMeterLibMeter
Basic Terms VUsage Counts and Derivatives Value
Example: Downloads (D) Normalized Value or Ratio (per external basis)
Example: Downloads per User (DpU) First Derivative (by time=rate) – per period
Example: Downloads per User per Year (DpUpY) Second Derivative –> Change Rate
Example: Growth/Decline=Change (CoDpUpY) Third Derivative - > Trend
Example: Acceleation/Slowdown (CoCoDpUpY)
© Peter Ahrens, Cologne, 2009
LibMeterLibMeter
LibMeter Seminar
x
2. NatLibStats
© Peter Ahrens, Cologne, 2009
LibMeterLibMeter
Comparing Usage at
different Types of Institutions
© Peter Ahrens, Cologne, 2009
LibMeterLibMeter
2007 EZB Usage per Primary User at Universities versus Polytechs
EZB Statistics Universities (n=57)
Median = 6,2 Mean / Average = 6.9 Standard Deviation = 5.6 Variation Coefficient = 0.8
Polytechs (n=34) Median = 0,42 (sic!) Mean / Average = 2.0 Standard Deviation = 6.1 Variation Coefficient =3.0
Düsseldorf: Mean / Average 7.0 = 0.13 Std. Dev. above
Mean
While you may calculate Mean and St. Dev. for a population, make sure it is normally
distributed before drawing further conclusions (check Median) !
© Peter Ahrens, Cologne, 2009
LibMeterLibMeterLooking annually at
1 Parameter (EZB)
at 1 time (DBS 2007)
for many libraries
© Peter Ahrens, Cologne, 2009
LibMeterLibMeter
Comparing EZB Use among University Libraries in 2007
Compiled from DBS VA: http://www.bibliotheksstatistik.de/eingabe/dynrep/index.php [2009-04-15]
EZB (B+C only) call as per DBS 2007Standardized per Primary User per Year
© Peter Ahrens, Cologne, 2009
LibMeterLibMeter
Sample University Library Profile (DBS)Basics on Library-“Size“, -Input & Output
Actions Raw Numbers
Often very high values Go beyond imagination
Work Basis: User Numbers Different Types Can be combined to
Primary User NumberPrimary User Number Standardization
Key Step: Calculation of „Value Value per User per per User per YearYear“ = Normalization
Benefits Numbers become imaginable Numbers can be memorized Numbers become
comparable
Data downloaded from www.bibliotheksstatistik.de into PC-Spreadsheet Program, [2009-05-03]
© Peter Ahrens, Cologne, 2009
LibMeterLibMeter
BIX Performance-Indicator „Virtual Library Use“ New (per 2008*) in German Library Performance Index
„BIX“ Normalized Combined Parameter: „Virtual Annual Use“ Definition: (OPAC & Homepage) / Number of
Primary Users Simple Interpretation
Year Source OPAC Homepage OPAC + HP Primary Users2007 DBS 684.097 3.477.400 4.161.497 16.991
HDM 1.116.511 2.360.890 3.477.401 16.991
Hits per User 65,7 138,9 204,7 = Virtual Use
© Peter Ahrens, Cologne, 2009
LibMeterLibMeter
How SMART is „Virtual Use“ ?Specific: Quite general Measurable: YES Community Standard with help
of HDM ZählpixelAchievable and Attributable: Library is actively producing Service Today every library can produce #Relevant and Realistic: YES Users frequently go there Closely tied to core services How representative for total
spectrum of electronic Library Services ?
Time-bound, Timely, Trackable, and Targeted:
No log-files publicly available (yet?) Not available for all periods
© Peter Ahrens, Cologne, 2009
LibMeterLibMeterGo to Screens List
Slides on DBS, BibS,BIX &
ARL(Start Present_26 Slide 07)
© Peter Ahrens, Cologne, 2009
LibMeterLibMeter
German Library Statistics 10 Year Overview
2008
1999 Univ Lib Change +/- 0% -24% +35% +50% -7% +10% +29% +31% Price Index DEEducation
1999 to 2008> + 50 %
© Peter Ahrens, Cologne, 2009
LibMeterLibMeter
„DBS Variable Analysis“ (Display!) Selection Dialogue
© Peter Ahrens, Cologne, 2009
LibMeterLibMeter
DBS Variable AnalysisResult Table
© Peter Ahrens, Cologne, 2009
LibMeterLibMeter
DBS – IndikatorenrasterBenchmark for Public Libraries Logical Framework
Public Libraries Only 1 of 5 Community Size
Classes 2007 (only as per 2009-10-10)
Data Shown 21 Indicators calculated
from Primary Measures Grouped in 4 Dimensions Minimum, Median,
Maximum 6 Percentile Ranges Positioning „own“ Library
A 5% B 25% C 50% D 75% E 95% F gültige Werte
Medien / EW 0,69 1,15 1,21 1,66 2,18 3,24 565
Erneuerungsquote (Zugang / Bestand in Prozent) 3,1 6,0 8,1 8,4 10,5 15,0 557
Neuerwerbungen: ME / EW 0,03 0,09 0,10 0,13 0,19 0,30 563
Veranstaltungen / 1000 EW 0,5 1,6 1,8 2,9 5,0 9,0 570
Jahresöffnungsstunden / 1000 EW 33,8 35,0 56,9 73,2 93,3 131,7 566
Publikumsfläche / 1.000 EW (qm) 7,44 14,82 21,87 25,18 32,16 56,55 560
Computerplätze/1000 EW 0,0 0,09 0,1 0,1 0,2 0,5 570
Anteil aktive Entleiher an EW 3,8% 7,6% 10,0% 11,2% 15,2% 26,0% 557
Besuche / EW 0,3 1,0 1,5 2,3 2,5 4,7 404
Umschlag (Entleihungen / ME) 1,0 2,0 2,7 3,7 4,8 5,3 565
Entleihungen / EW 1,0 2,8 4,5 5,5 7,1 11,6 567
Recherchen / EW 0,00 0,06 0,15 0,16 0,30 0,68 149
Mitarbeiter (VZÄ )/ 1000 EW 0,04 0,09 0,14 0,17 0,20 0,33 522
1000 Entleihungen / MA (VZÄ) 12,6 23,6 31,5 33,7 44,6 62,9 515
Jahresöffnungsstunden / MA (VZÄ) 193,4 255,8 392,0 524,5 727,1 1053,4 505
Fortbildungsstunden / MA (VZÄ) 0,00 0,00 7,22 9,88 20,00 46,91 483
Laufende Ausgaben / Besuch (€) 2,34 3,68 5,07 7,66 7,72 14,27 400
Laufende Ausgaben / EW (€) 2,56 5,03 8,45 12,13 17,31 20,07 560
Erwerbungsausgaben / EW (€) 0,24 0,68 1,04 1,14 1,76 2,89 568
Anteil Personalausgaben / Gesamtkosten 1% 43% 58% 66% 74% 83% 555
erwirtsch. Mittel + Fremdmittel / Gesamtausgaben 0,5% 1,0% 5,5% 9,9% 16,0% 42,5% 552
572 Bibliotheken in dieser Größenklasse Copyright © 2009 hbz.
Fina
nzen
Musterbibliothek - INDIKATORENRASTER Größenklasse: 10.000 bis 20.000 EW - Berichtsjahr: 2007
Ser
vice
Nut
zung
Per
sona
l
© Peter Ahrens, Cologne, 2009
LibMeterLibMeter
DBS – Public Libraries Indicatorgrid „Indikatorenraster“
© Peter Ahrens, Cologne, 2009
LibMeterLibMeter
German Library Statistics 2009New Online Exploration Tool - BibS
© Peter Ahrens, Cologne, 2009
LibMeterLibMeter
Basic interactive Comparrison in Time Series
© Peter Ahrens, Cologne, 2009
LibMeterLibMeter
BIX - 1-Level Univ. LibrariesRanking Result Table in Print 2006
© Peter Ahrens, Cologne, 2009
LibMeterLibMeter
(Germany+) BIXLibrary Statistics Index - Overview
© Peter Ahrens, Cologne, 2009
LibMeterLibMeter
(Germany+) BIXLibrary Statistics Index - Measures
© Peter Ahrens, Cologne, 2009
LibMeterLibMeter
(Germany+) BIXMeasures over Years
© Peter Ahrens, Cologne, 2009
LibMeterLibMeter
(Germany+) BIXCompare & Show „Usage“
© Peter Ahrens, Cologne, 2009
LibMeterLibMeter
American Association of Research Libraries (ARL) – Interactive Statistics
© Peter Ahrens, Cologne, 2009
LibMeterLibMeter
ARL – Interactive StatisticsExample: Library Service Hours
© Peter Ahrens, Cologne, 2009
LibMeterLibMeter
ARL – Interactive StatisticsService Hours over time @ 9 Inst.
ARL – Interactive StatisticsExample: Library Service Hours
© Peter Ahrens, Cologne, 2009
LibMeterLibMeter
National Statistics (e.g. BIX/DBS)Benefits & LimitsBenefits Broad Institutions Base Fairly Comprehensive Creates Awareness
Community Public
Online Data Warehouse Transparency Allows Meta Studies
Limits Limited Parameter Set Scope
i.e. more detailled services missing
Data completeness & quality Standardization Issues Time Lag, Annual FrequencyNational Differences International Comparability ? Interactivity/-Download Meta-Analysis Tools, Charts
© Peter Ahrens, Cologne, 2009
LibMeterLibMeterDiskussionsfragenWann erste Ergebnisse ?Wie lange auf welche Aussagen
warten ?
© Peter Ahrens, Cologne, 2009
LibMeterLibMeter
LibMeter Seminar
x
2. bÜbungen
© Peter Ahrens, Cologne, 2009
LibMeterLibMeterGo to Screens List
Übungen hier
© Peter Ahrens, Cologne, 2009
LibMeterLibMeterDiskussionsfragenWelche Komponenten können
Nationalstatistiken haben ?Möglichkeiten und Grenzen ?Wie SMART Virtual Use ?