what does success look like? evidence–based assessment of the academic library

1GreekAcadLibs17 JED 2008 © J.E. Davies

What does success look like? What does success look like? Evidence–based assessment of the Evidence–based assessment of the

academic libraryacademic library

Dr. J. Eric DaviesHonorary Visiting Research Fellow

Loughborough University

17th Conference Greek Academic Libraries


Outline:Outline:

• Management environmentManagement environment• Success factors, mission and strategySuccess factors, mission and strategy• Performance evaluation and enhancementPerformance evaluation and enhancement• Evidence typesEvidence types• Culture and competencesCulture and competences• Future trends speculations Future trends speculations



Management environment:Management environment:Change –Change –• TechnologyTechnology• Scholarly Communication and InformationScholarly Communication and Information• Attitudes/Expectations/DemandsAttitudes/Expectations/Demands• Services, Information Skills and SupportServices, Information Skills and Support• Finance/Resources/AccountabilityFinance/Resources/Accountability• Innovation/DevelopmentInnovation/Development



Success factors, mission and strategy:Success factors, mission and strategy:• Success Framework/CriteriaSuccess Framework/Criteria• Mission/Vision/ValuesMission/Vision/Values• Aims/Objectives/StrategiesAims/Objectives/Strategies• Service goals/TargetsService goals/Targets• Performance IndicatorsPerformance Indicators• Evidence Based ManagementEvidence Based Management



Framework for Performance Measurement:-Framework for Performance Measurement:-

Integration with institutional goals Integration with institutional goals User satisfaction User satisfaction Effectiveness (delivery) Effectiveness (delivery) Efficiency Efficiency EconomyEconomy

Follett Report – academic libraries –1993Follett Report – academic libraries –1993



Special Libraries Association –Special Libraries Association –

For special librarians, evidence-based practice For special librarians, evidence-based practice refers to consciously and consistently making refers to consciously and consistently making professional-level decisions that are based on professional-level decisions that are based on the strongest evidence of what would work best the strongest evidence of what would work best for our clients.for our clients.



Types of evidence –Types of evidence –

• Performance EvidencePerformance Evidence

• Research EvidenceResearch Evidence



Systematically gathered, structured and Systematically gathered, structured and contextualised information relating to local contextualised information relating to local operations and needsoperations and needs

Performance EvidencePerformance Evidence



Performance Evidence –Performance Evidence – Services Services

ProcessesProcesses

QuantitativeQuantitativeQualitativeQualitative



Inputs Inputs Quantitative DataQuantitative Data

OutputsOutputsQualitative Evidence Qualitative Evidence

OutcomesOutcomesUser ResponsesUser Responses

ImpactsImpacts

Performance



User Responses –User Responses –• SurveysSurveys• Focus groupsFocus groups• Open DaysOpen Days• LibQUALLibQUAL++TMTM

• Rodski/InsyncRodski/Insync



User Surveys –User Surveys –

• User Practice [frequency of visits, types of use]User Practice [frequency of visits, types of use]• User Capabilities [ IT use, vision, mobility]User Capabilities [ IT use, vision, mobility]• User Motivations [reasons for use/non-use]User Motivations [reasons for use/non-use]• User Preferences [service development]User Preferences [service development]• User Needs/Wants [service priorities]User Needs/Wants [service priorities]• User Opinions [perceptions of service]User Opinions [perceptions of service]• User Satisfaction [with various features]User Satisfaction [with various features]



0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Feb-02

Mar-03

Feb-04

Mar-05

Feb-02

Mar-03

Feb-04

Mar-05

Sat

isfa

ctio

n w

ith s

taff

Sat

isfa

ctio

n w

ithse

rvic

es/fa

cilit

ies

Excellent

Good

Fair

Poor

British Library reader satisfaction survey ratings


LibQUAL+LibQUAL+TMTM

Developed by ARL/Texas A&M UnivDeveloped by ARL/Texas A&M Univ

Based on SERVQUAL - Gap Analysis Based on SERVQUAL - Gap Analysis

Minimum, Desired, Perceived user view of quality Minimum, Desired, Perceived user view of quality relating to -relating to -•Access to InformationAccess to Information•Affect of ServiceAffect of Service•Library as PlaceLibrary as Place•Personal ControlPersonal Control



4

5

6

7

8

9AI-1

AI-2

AI-3

AI-4

AI-5

AS-1

AS-2

AS-3

AS-4

AS-5

AS-6

AS-7AS-8AS-9

LP-1

LP-2

LP-3

LP-4

LP-5

PC-1

PC-2

PC-3

PC-4

PC-5

PC-6

Perceived greater than desired Perceived less than desired

Perceived greater than minimum perceived less than minimum

Example of data display of LibQUAL+TM

Results


XXX

4

5

6

7

8

9AI-1

AI-2

AI-3

AI-4

AI-5

AS-1

AS-2

AS-3

AS-4

AS-5

AS-6

AS-7

AS-8

AS-9

LP-1

LP-2

LP-3

LP-4

LP-5

PC-1

PC-2

PC-3

Perceived greater than desired Perceived less than desired

Perceived greater than minimum perceived less than minimum

Example of data display of LibQUAL+TM

Results


Example of Gap Analysis from LibQUAL+TM Results

0

1

2

3

4

5

6

7

8

9

Access toInformation

Affect of Service Library asPlace

Personalcontrol

Range ofminimum todesired

Range ofminimum toperceived


The amount of information and a manager’s ability to use it have The amount of information and a manager’s ability to use it have increased explosively not only for internal processes but also for the increased explosively not only for internal processes but also for the engagement of customers. The more a company knows about them, engagement of customers. The more a company knows about them, the better able it is to create offerings they want, to target them with the better able it is to create offerings they want, to target them with messages that get a response, and to extract the value that an messages that get a response, and to extract the value that an offering gives them. The holy grail of deep customer insight – more offering gives them. The holy grail of deep customer insight – more granular segmentation, low-cost experimentation, and mass granular segmentation, low-cost experimentation, and mass customization – becomes increasingly accessible through customization – becomes increasingly accessible through technological innovations in data collection and processing in technological innovations in data collection and processing in manufacturing.manufacturing. James M. Manyika, Roger P. Roberts and Kara L. Sprague ~ James M. Manyika, Roger P. Roberts and Kara L. Sprague ~

McKinsey Q.(1) 2008. McKinsey Q.(1) 2008.



Service Service DomainDomain Community ProfilingCommunity Profiling

DemographicsDemographicsUser DataUser DataLapsed, Non-Lapsed, Non-

User, Potential User DataUser, Potential User Data

Information EnvironmentInformation Environment

Performance



Comparative Benchmarking –Comparative Benchmarking –

Evaluate the level of performance of Evaluate the level of performance of various services within an institutionvarious services within an institution

Compare performance over timeCompare performance over time Compare with other institutionsCompare with other institutions Compare with recognised standardsCompare with recognised standards




STRATEGIC STRATEGIC macro performance metrics macro performance metrics and qualitative evidenceand qualitative evidence

PROCESS PROCESS detail cost performance datadetail cost performance data




To make changes that lead to quantum and To make changes that lead to quantum and continuous improvements in products, continuous improvements in products, processes and services that result in total processes and services that result in total customer satisfaction and competitive customer satisfaction and competitive advantage.advantage.

[Motorola + D.E.C. + Xerox][Motorola + D.E.C. + Xerox]



Percentage of requests met in 7,15 & 30 days

0%

20%

40%

60%

80%

100%

98/9 99/0 00/1 01/2 02/3 03/4

XXX 30

XXX15

XXX 7

Std 30

Std 15

Std 7


0

1

2

3

4

5

6

7

8

9

10

1997-98 1998-99 1999-00 2000-01 2001-02 2002-03 2003-04

XXXXX

Group

CURL

SCONUL

University Library - FTE students per seat


0

5

10

15

20

25

30

35

40

45

1991-92 1992-93 1993-94 1994-95 1995-96 1996-97 1997-98 1998-99 1999-00 2000-01

£

Library X Comparators New SCONUL

University Library - Book expenditure per FTE student


Average Weekly Opening Hours

0

10

20

30

40

50

60

70

80

90

94-5 95-6 96-7 97-8 98-9 99-0 00-1 01-2 02-3 03-4 04-5

xxxxxxxxx

Benchmark

Old

Old + CURL

SCONUL


Re-shelving and tidying performance

Site A

Site B

Site C

Site D

Site E

Site F

Site G

0%

20%

40%

60%

80%

100%

0 2 4 6 8 10 12 14 16 18 20 22 24

Number of hours


Research based intelligence and macro data Research based intelligence and macro data that informs service design and decisionsthat informs service design and decisions

Research EvidenceResearch Evidence



Performance Evidence Culture –Performance Evidence Culture –

Service design and delivery are informed Service design and delivery are informed by appropriate internal and external by appropriate internal and external evidenceevidence ‘ ‘Think Evidence’ attitude permeates the Think Evidence’ attitude permeates the organisationorganisation Skills, knowledge, attitudes in using Skills, knowledge, attitudes in using evidence are embedded at ALL levels of the evidence are embedded at ALL levels of the organisationorganisation



What will success look like – What will success look like – tomorrow??? tomorrow??? • Optimal Management Optimal Management • Maximised Value AddedMaximised Value Added• Academic Culture/EnvironmentAcademic Culture/Environment



How will we measure success – How will we measure success – tomorrow??? tomorrow??? • Contingent valuation analysisContingent valuation analysis• Economic impact studiesEconomic impact studies• Critical incident studies Critical incident studies • Scenario planning Scenario planning • Lean managementLean management• Six sigmaSix sigma



‘‘Management in particular is not about the preservation Management in particular is not about the preservation of the status quo, it is about maintaining the highest of the status quo, it is about maintaining the highest rate of change that the organization and the people rate of change that the organization and the people within it can stand.’within it can stand.’

‘‘Whether change is comfortable or not, it is inevitable.’Whether change is comfortable or not, it is inevitable.’

[ Harvey-Jones, J. [ Harvey-Jones, J. Making it happen: Reflections on Making it happen: Reflections on leadershipleadership. London, Collins, 1988. ]. London, Collins, 1988. ]



Thank you for your attention.Thank you for your attention.

Questions?Questions?


what does success look like? evidence–based assessment of the academic library

Documents