Dr. Roswitha PollMünster

Benchmarking with performance indicators and Balanced Scorecard

Quality in libraries

User-orientation

Accessibility (building, collection, services)

Accuracy and reliability

Speed and currency

Competence and helpfulness of staff

Effectiveness and efficieny of processes

Quality in libraries: stakeholders

Users (actual and potential)

Funding institutions (university, community)

Policy makers

General public

Library staff

Library managers

Quality assessment in libraries

Performance indicatorsmeasure the effectiveness and cost-efficiency of library services: quantitative, objective

User satisfaction surveysmeasure the perceived quality, the users‘ impression of library services: qualitative, subjective

Outcome assessmenttries to show the benefits, the value for individual users and society

Collections of performance indicators for libraries

ISO 11620, 2nd ed. (2008), Information and documentation – Library performance indicators (45 indicators)

ISO TR 28118 (2008), Information and documentation – Performance indicators for national libraries (30 indicators)

Poll, R. and te Boekhorst, P., 2nd ed. (2007), Measuring quality, performance measurement in libraries, Saur, München (IFLA Publications 127) (40 indicators)

Criteria for performance indicators

• informative = helpful for identifying problems and possible actions to be taken

• reliable = producing the same results when used under the same circumstances

• valid = measuring what they are intended to measure• appropriate = compatible with the library's

procedures and working environment• practical = easy to use and understand, applicable

with a reasonable amount of effort• comparable = allowing comparison of results

between libraries of similar mission, structure and clientele

What indicators measure

Usage of library services

- Library visits per capita - Number of content units downloaded per capita

The quality of the collection- Document use rate (collection use rate)- Availability of required titles

The quality of processes- Shelving accuracy- Speed of interlibrary lending

Cost-efficieny- Cost per database session

- Cost per loan

How to get the data?How to get the data?

Institutional statistics

Library statistics

Automated library system

Manual data collection

Number of visits

Number of user places

Number of students

Number of academic staff

Number of active users

ILL requests

Number of working-places used

User satisfaction survey Use rate of working-places

Usersatisfaction

Market penetration

Indicators for joint projects The set of indicators should…

Cover all traditional and new services

Consider each individual library

Allow comparison

Show the importance and benefit of libraries

Include only a very few indicators

Involve no additional workload (data taken from the normal library statistics)

???

Projects that only recommend indicators

Swedish Quality Handbook (2005) ) all types of librariesall types of libraries3-years project 2001-20043-years project 2001-2004 http://www.biblioteksforeningen.orghttp://www.biblioteksforeningen.org

The Norwegian indicators, 4th revision (2007)academic and public librarieshttp://www.abm-utvikling.no/bibliotek/statistikk-for-bibliotek/indikatorer-for-fag-og-folkebibliotek

Projects that make a set of indicators compulsory

BIX –Library IndexBIX –Library IndexGerman public and academic libraries German public and academic libraries public libraries 1999 ff., academic libraries 2002 ff.public libraries 1999 ff., academic libraries 2002 ff.over 260 libraries (not all continuously)over 260 libraries (not all continuously)http://www.bix-bibliotheksindex.dehttp://www.bix-bibliotheksindex.de//

The Benchmarking system of the Netherlands University Librariesall university libraries and the national library 1999 ff. http://www.ukb.nl/benchmark.htmhttp://www.ukb.nl/benchmark.htm

HELMS (UK Higher Education Library HELMS (UK Higher Education Library Management Statistics)Management Statistics) academic librariesacademic libraries1997/98 ff.1997/98 ff.

Balanced Scorecard

Kaplan, R.S./Norton, D.P.:Kaplan, R.S./Norton, D.P.: The Balanced Scorecard: Translating Strategy into

Action. Boston 1996

Balanced Scorecard: 4 perspectives

Original1. Finances2. Customers3. Internal processes4. Learning and growth

Libraries1. Resources, infrastructure2. Use3. Efficiency4. Potentials and development

(Learning and growth)

Balanced Scorecard: How to proceed

1. Starting with the vision (mission), the strategy is developed: What are the main tasks and goals?

2. The goals are structured in the perspectives

3. Within the four perspectives, “critical success factors” are defined: What is most important for reaching the goals?

4. Based on the critical success factors, “key performance indicators” are selected.

Balanced Scorecard: Example

Vision: The library is the main meeting and communication centre in the community or university

Perspective: Customers (users)

Goal: To attract the population to the library premises Critical success factors: opening times, adequate space and equipment for working in the library

Key performance indicators: - m2 of user area per 1,000 capita- number of workplaces per 1,000 capita- opening times compared to user demand

Balanced Scorecard: The 4th perspective

Potentials and development,Learning and growth

• intangible assets of an organisation• internal skills and capabilities• effective information management• overall climate in an organisation

Balanced Scorecard: Why?

All important perspectives for quality are considered.

Each perspective with its goals is evaluated by adequate performance indicators.

There is a limited number of performance indicators (up to 20).

The indicators are related in a systematic way and influence each other.

The BSC is an instrument for an integrated view of the institution’s quality and for continuous assessment of how far targets have been reached.

Global statistics project:Performance indicators

Global statistics: Only a limited number of data

Socio-demographic data from UIS to be added- size of population- number of students- literacy in a country

76 indicators were calculated in the project

Of these, 23 were defined as core indicators

Can the Balanced Scorecard be Can the Balanced Scorecard be used for the “global statistics”?used for the “global statistics”?

Balanced ScorecardResources and infrastructure

Public libraries1. Average number of public libraries per 1,000 inhabitants2. Weighted average opening hours3. Percentage of libraries offering an internet access for users4. Percentage of libraries offering websites5. Average number of volumes in public libraries per 1,000 inhabitants6. Average number of volumes per 1,000 literate inhabitants7. Average number of employees in public libraries

Academic libraries1. Weighted average opening hours2. Average number of volumes in libraries per student3. Average number of workplaces per 1,000 students 4. Average number of electronic serials (subscriptions)5. Average number of employees in libraries

Balanced ScorecardUsage

Public libraries1. Number of registered users per 1,000 inhabitants2. Number of registered users per 1,000 literate inhabitants3. Average number of loans per 1000 inhabitants4. Average number of loans per 1,000 literate inhabitants5. Number of visits per 1,000 inhabitants6. Number of visits per 1,000 literate inhabitants

Academic libraries1. Number of registered users as a percentage of number of students2. Average number of loans per student3. Number of visits per student4. Number of attendances at training sessions per student5. Average number of content units downloaded per registered user

Balanced ScorecardEfficiency/Finances

Public libraries1. Expenditure on literature and information per capita 2. Ratio of expenditure on literature and information to staff costs3. Cost per visit4. Cost per registered user

Academic libraries1. Expenditure on literature and information per capita 2. Ratio of expenditure on literature and information to staff costs3. Cost per visit4. Cost per registered user

Balanced ScorecardPotentials and development(Learning and growth)

Indicators that have been used

E-services• expenditure for the E-collection• staff in E-services Staff development and motivation• time and money spent on staff training !• availability and fluctuation rate of staff Raising resources• library means received by special grants or income generated • percentage of institutional means allocated to the library

No indicators in “global statistics”

New perspective „Impact on population“?

Average number of volumes per 1,000 literate inhabitants

Number of registered users per 1,000 literate inhabitants

Average number of loans per 1,000 literate inhabitants

Number of visits per 1,000 literate inhabitants

High numbers of public libraries, High numbers of public libraries, collections and services, and high collections and services, and high library use may have influenced the library use may have influenced the percentage of literate inhabitants in a percentage of literate inhabitants in a countrycountry

Keep Keep your your balancebalance