Scottish Library and Information CouncilScottish Library and Information Council

Being the best we can

Developing a self evaluation and improvement model

Scottish Library and Information CouncilScottish Library and Information Council

Background

How we got to the Public Library Quality Improvement Matrix

Legislative context Standards and benchmarks based on inputs and

outputsChanging paradigm - local government re-organisation,

People’s Network, SNP administration Increased scrutiny-Best Value, Efficient Government, Changing national policies on Culture and shared

services Currently working with 28 authorities – 9 reports

published, plus 9 pilot reports

Public Library Quality Improvement Matrix

Scottish Library and Information Council

Developed 2004European Framework of Quality

Management and schools inspection7 Quality Indicators 4 service QIs and 3 management QIsToolkit to support roll outBenchmarks and characteristics more at www.slainte.org.uk

Scottish Library and Information CouncilScottish Library and Information Council

So how did self evaluation work in Scotland in 2008?Moray QI 2 Community and personal participation and QI 6 Organisation and use of resources and space

East Dunbartonshire QI 1 Access to information and QI 5 Ethos and values

Edinburgh – all 7 QIs

Dundee QI 3 Meeting readers’ needs and QI 5 Ethos and values

Shetland – all 7 QIs

Fife – QI 3 Meeting readers’ needs

Falkirk QI 1 Access to information and QI 6 Organisation and use of resources and spaceNorth Ayrshire QI 2 Community and personal participation and QI 3 Meeting readers’ needs

Scottish Library and Information CouncilScottish Library and Information Council

So what were the outcomes of self evaluation in Scotland in 2008? 10 Very good 12 Good 3 Satisfactory Level 5s are predominantly in QI 2 Community and personal

participation. The level 3s are in Organisation and use of resources and space and Leadership.

Scottish Library and Information CouncilScottish Library and Information Council

Some examples of strengths in 2008

Moray Council

Accredited courses. Use of Individual Learning Accounts to fund two posts which improved learning opportunities offered by the library.

East Dunbartonshire

Range of information services available to residents in a small area. Staff are well-trained, knowledgeable and keen to share their enthusiasm for subjects and online sources with the public

Edinburgh Award winning Libraries4YOU initiative, working with young teenagers to combat anti-social behaviour problems

Dundee Staff commitment to needs of vulnerable groups: provision for minority groups, Customer Care Standards, literacies learners and engagement with drug and alcohol recovery groups

Scottish Library and Information CouncilScottish Library and Information Council

Scottish Government Public Libraries Improvement FundThe Scottish Government Public Libraries Improvement

Fund is a £500,000 fund for a 5 year period to encourage service development in authorities who already investing and achieving progress in their public library service delivery.

Authorities who undertake the self-evaluation process and are assessed externally verified as satisfactory or better will be able to submit proposals based on their action plans to the Improvement Fund.

The Public Library Quality Improvement Matrix can be used as a performance management tool within local authorities.

Excellence frameworks

Japanese established the Deming Prize in 1951.

In 1987 Australia established the Australian Quality Award.

In 1988 US established Malcolm Baldrige National Quality Award.

1991 European Foundation for Quality Management (EFQM) established the European Model for TQM.

All models are reviewed regularly and have developed a focus on overall business excellence rather than quality.

Australian Business Excellence Framework

Being the best we can

Developed by the Library Network in VictoriaProvides tools for local and state government to meet

service obligations and objectivesFocus on planning and continuous improvementDemonstrates the continuing relevance, contribution

and impact of library servicesBased on the Scottish Library and Information

Council’s Public Library Quality Improvement MatrixContextualized to reflect Australian policy, the

Australian Business Excellence Framework and Best Value

What is self-evaluation?

3 basic steps

How are we doing?How do we know?What are we going to do now?

What benefits are there?

Enables us to take stock of current performance

Measures services in terms of progress towards objectives

Allows us to identify and understand our key strengths

Allows us to identify areas for improvementSupports us as we prioritise areas for action

Outputs of self evaluation

Improvement plan which helps us use resources and finances effectively

Reporting to stakeholders, so improved understanding of services

Benchmarks for ourselves and othersTraining resourcesCase studies of good practiceUnderpinning a culture of quality

Why is self evaluation effective?

More a process than an eventNeeds to be owned not imposedMust be inclusive not exclusiveMust not depend on measurabilityBuilt in not bolted onSize does NOT matterArt not science

Nine Step Process

Step 1 Orientation Step 2 Training for staffStep 3 Deciding which Key Result Areas to

reviewStep 4 Preparation of the evidence and

identification of key strengths and areas for improvement with completion of template in toolkit for review

Nine Step Process 2

Step 5 Award star scoreStep 6 Visit and assessment with external peer

reviewersStep 7 Feedback from external review team and

a agreement is reached on the star awardStep 8 ReportStep 9 Improvement Plan

Key Result Areas and users

Providing informationBuilding skillsDeveloping social

capital

AdultsChildrenSpecial NeedsGroupsBusinesses

What are we going to evaluate?

Key Result Areas for Library Services Overall evaluation

1. Providing gateways to information

2. Building individual skills, capability and wellbeing

3. Developing social capital

4. Demonstrating leadership and values

5. Designing, managing and improving systems and processes

KRA 1 Providing gateways to information

Scottish Library and Information Council

1. Sufficiency, range and suitability of resourcesWhy is this important?Library services should offer consistent and equitable access to print and digital collections, internet and to a range of other resources for their communities. Provision of a range of resources, fit for purpose and specific need allows individuals to make informed choices. A successful library should have a high percentage of use by its local population. Key question to ask To what extent are the information needs of all met?Things to think aboutWhat range of print and digital collections does the library offer across a range of user groups?How is the selection of information resources made?How well are the information services used by the population?What are the most frequently used services?

Gathering the evidence

Consulting users, non-users and staffObserving activitiesConsidering measurable outcomesExamining documentation and

resources

Evaluating the evidence

Best done as a teamDoes evidence show a service

strength or an area for improvement Is everyday a strength?Some evidence is neither but shows  

effective and necessary processesTemplate in the toolkit

5 Star Awards

Outstanding delivery of core and a range of enhanced services

Excellent

Sector leading and worthy of wider dissemination

Major strengths

High standard of provision

Any weaknesses do not impact on users’ experience

5 Star Awards 2

Important strengths that have a positive impact

Weaknesses do not have substantial adverse effect

Weaknesses are recognised and action is being taken to improve

Satisfactory provision of core services

Provision with a few strengths and examples of good practice

Weaknesses have some adverse effects on some users

5 Star Awards 3

Limited range of services or weaknesses in a core service

Weaknesses have a significant impact on the quality of the users’ experiences

Planned action required for improvement

Illustrative models

These provide an example of what star awards mean in practice. Between the descriptions set out in the five star rating system and these illustration, services are asked to make their own star award. As more services go through the process the Library Network will be able to collect information and benchmarks which will also be helpful.

External review

It is always useful to have a comparative view. Peers will be acting as external reviewers of the self-evaluation process and outcomes.

The Library Network will help co-ordinate peer review and there is support documentation for them.

Being the Best We Can – Pilot

Initial training, pilot library AprilSelf-assessment, pilot library May-JuneVerification & report JulyReview & modify process AugustTraining, second round pilots SeptemberSecond round self-assessments October -

March

Thoughts from working group

The rigour of the processIt’s a journey – it’s constantly informedSelf evaluation which leads to a cycle of improvementBe creative and consistent where you find the evidenceIt’s about telling your own storyIt’s all staff engagementIt’s a learning and knowledge experienceIt builds the evidence base for advocacy – locally,

state-wide and nationally!