1
Performance Management Presentation
Provide Library Services
Team Members:
Ted Crump, Rosalie Stroman, Jennifer Vrettos, and Susan Whitmore (Team Leader)
ORS National Institutes of Health
January 21, 2004
2
Table of Contents
Main Presentation
PM Template ………………….…………………………..……….5
Customer Perspective……………………….…………..….…….7
Internal Business Process Perspective……………………………………………..………16
Learning and Growth Perspective…………………………………………….….……22
Financial Perspective…………………………………..…………25
Conclusions and Recommendations……………………………………………..31
3
Table of Contents (cont.)Appendix
Customer PerspectiveC1a: Percent of current print journals in electronic format……………..…38C1b: Percentage of license agreements that cover all NIH……………....39C1c: Percentage of document requests filled in-house…..……………….40C1d: Number of items translated by government staff and contractors…41C2a: Number of web-based instruction modules………………………..…43C2b: Training attendance by type……………………………………………44C2c: Training evaluation scores…………………………………………..…45C3a: Customer Scorecard ratings for translation service and
mediated database search service..………………………………….…..48C3c: Digital reference service evaluation ratings…………………………..54C4: Customer segmentation data……………………………………..……..56
Internal Business Process PerspectiveIB1: Number and type of special projects contracted for…………..…..…63IB2: Number of new IT systems implemented…………………………..…65IB3a: Percent of document requests filled in-house in 3 business days……………………………………………………………………….….69IB3b: Percentage of documents translated by library staff
within 3 business days…………………….…….……………………..…..70IB3c: Percent of tutorial requests responded to within 1
business day………………………………………………….….….71
4
Table of Contents (cont.)
Appendix
Learning and Growth PerspectiveLG1a: Percentage of staff members with IDPs completed by
February 28 each fiscal year……………………………………...........…74LG1b: Percentage of staff completing 80% of activities listed in
IDP.......................................................................................................75LG1c: Percentage of training modules completed on schedule…………76
Financial PerspectiveF1: Change in unit cost of information…………….……………………….79
F2a: Change in unit cost of custom services…………………………..….82 F2b: No. of contracts/IAGs……………………………………………….….82
F3a: Change in unit cost of providing copies of publications…………….82 F3b: No. of document requests filled……………………………………….82
F4a: Change in unit cost of translating services…………………….…….83 F4b: No. of documents translated…………………………………….…….83
F5a: Change in unit cost of providing primary library services……….….83 F5b: No. of units of information provided…………………………….…….83
F6a: Change in unit cost of providing self-service copiers…………...….83 F6b: No. of visits to self-service copy center………………………..…….83
5
Performance Management Plan (PMP)
DS3: Translate documents
DS2: Provide copies of publications
DS1: Provide custom research assistance
Discrete Services
Division Approval/Date: July 30, 2003 Associate Director Approval/Date:
DS5: Provide self service copiers
DS4: Provide primary library services
Provide library services
Service Group
Service Strategy
In support of NIH’s medical and behavioral research mission, the NIH Library provides quick and convenient access to the latest scientific and medical information resources. Providing these resources to the staff contributes to improvements in patient ca
Value Proposition
Team Leader
Excellence: The NIH library is committed to providing excellent information services, products, collections, and people to aid NIH in achieving its mission. Staff demonstrate this commitment by their helpfulness, their respect for the information needs
Strategy Description
Team Members
Susan Whitmore
Ted Crump, Rosalie Stroman, Jennifer Vrettos
Date: 12/15/2003
Operational Excellence
Customer Intimacy
Product Leadership
Growth
Sustain
Harvest
6
Relationship Among Performance Objectives
Inte
rnal
Bus
ines
sLe
arni
ng &
Gro
wth
Cus
tom
er
Financial
C2: Enhance researchersreferencing knowledge and skills
C4: Increase understanding ofcustomer base
IB2: Adopt best practices and infusenew technology
IB3: Maintain or improve turnaround time for filling requests with
accuracy
F1: Maintain or reduce costof information
IB1: Promote value added customservice projects
C3: Increase customer/stakeholdersatisfaction with overall service
LG1: Support continuous, shared stafflearning and improvement
_____ relationship between objectives
most important objectives
C1: Provide integrated transparentaccess to the most relevant
information
7
Customer Perspective
8
Customer Perspective
Objective MeasureFY 03 Target
FY04 Target
FY05 Target
Initiative Owner
C1: Provide integrated transparent acccess to the most relevant information via the most effective information sources for NIH researchers and staff
C1a. Percent of current print journals in electronic format
> FY02 > FY03 > FY04 Convert all journals to e-format.
C1a: RS
9
C1a: Percent of current print journals in electronic format
• Target: Provide 100% of current journals in electronic format
• Eighty-one percent of the journals are now available in electronic format (as compared to 75% in FY02 and 52% in FY01)
1597
1035739
2383
3085
3171
0
500
1000
1500
2000
2500
3000
3500
4000
4500
FY01 FY02 FY03
Fiscal Year
No.
of
Title
s
Print Only Electronic journals
10
Customer Perspective (cont.)
Objective MeasureFY 03
TargetFY04
TargetFY05
TargetInitiative Owner
C3:Increase customer/stakeholder satisfaction with overall service
C3a. Customer Scorecard ratings for mediated database searching
> FY02 > FY03 > FY04 Use scorecard to evaluate provide copies of publications and reference services delivered at the information desk in FY04
C3a: JV
11
C3a: Customer Scorecard ratings for mediated database searching service
• Mediated database searching service was evaluated from mid-October to mid-December of 2003 using paper or electronic survey forms
• Forty surveys were distributed and 14 were returned for a return rate of 35%
• The surveys asked users of the service to rate customer service and search results on a 10 point scale with 1 being unsatisfactory and 10 being outstanding
• Customers were very satisfied with the service
12
C3a: Customer Scorecard ratings for mediated database searching service – Customer Service
9.93 10 10 10 9.93
123456789
10
Understanding ofQuery
Competence Responsiveness Convenience Availability
Criteria
Un
sati
sfac
tory
Ou
tsta
nd
ing
FY03 N = 14
13
C3a: Customer Scorecard ratings for mediated database searching service – Search Results
9.29 9.299.79 9.36
123456789
10
Presentation Relevance Timeliness Quality
Criteria
Uns
atis
fact
ory
O
utst
andi
ng
FY03 N=14
14
Customer Perspective (cont.)
Objective MeasureFY 03
TargetFY04
TargetFY05
TargetInitiative Owner
C4: Increase understanding of customer base
C4. Customer segmentation data C4; RS & TC
15
C4: Customer segmentation data• Customer segmentation data is presented in the Appendix for
four services: • Provide copies of publications• Provide self-service copiers• Translate documents• Provide custom research assistance
• As in past years, customers are spread across all IC’s
• Heavier use of services by larger IC’s and groups most directly engaged in patient care and biomedical research
• The Library serves not only NIH but also FDA/CBER on the Bethesda campus and provides document delivery to several groups including NIST and CDC.
16
Internal Business Perspective
17
Internal Business Perspective
Objective MeasureFY 03
TargetFY04
TargetFY05
TargetInitiative Owner
IB2: Adopt best practices and infuse new technology
IB2: Number of new IT systems implemented.
Plan to implement HHS Consortium AccessPlan to implement SFXPlan to implement RelaisPlan to implement CMS
IB2: SW, SG, JV, RS
18
IB2: Number of new IT systems implemented
• There were four areas of concentration in FY03:• HHS Library Consortium • SFX or Full-Text Plus! • Relais• Content Management Software
• The major areas of concentration for FY04 are:• Continue with the development of the HHS Consortium, Full-
Text Plus!, Relais, and CMS• Implement MetaLib• Enable wireless access in the Library
19
New IT systems implemented in FY03• HHS Library Consortium –In FY03 the Consortium jointly purchased
the Nature journals, Science, and Science Express at a considerable savings to NIH
• Full-Text Plus! – This software, which links various services and resources available from the Library, was implemented for the online journals, the Web of Science and Ovid databases, the new online document request form, and other resources in FY03
• Relais – In FY03, phase II of the Relais system was implemented with the integration of the web-based request form and the Web of Science requests. Through Relais the processing of document requests has been considerably streamlined. A process that used to have eight steps now has only three with a turn around time of 1-3 business days
• Content Management Software has been implemented for the Library’s Intranet and public websites resulting in a diversification of responsibility for the sites. The redesigned public website will be unveiled on February 10, 2004
20
Internal Business Perspective (cont.)
Objective MeasureFY 03
TargetFY04
TargetFY05
TargetInitiative Owner
IB3: Maintain or improve turn around time for filling requests with accuracy
IB3b: Percentage of documents translated by library staff within 3 business days
> FY02 > FY03 > FY04 IB3b: TC
21
IB3b: Percentage of documents translated by library staff within 3 business days• Target: Translate document requests in 3 business days
• Results: In FY03 an average 55% of documents were translated in 3 business days. In FY04 the translation speed was measured for Library translators and contractors separately. In FY04 Library translators translated 55% of documents in 3 days and contractors translated 26% of documents in 3 days.
• Efforts to improve the turn around time for translating documents are complicated by:
• Translation requests are increasing and the current staff are already working as efficiently as possible
• The more translations that are contracted out, the more time the Library translation staff must spend checking contracted translations thereby reducing the time they have to translate
22
Learning and Growth Perspective
23
Learning and Growth Perspective
Objective MeasureFY 03
TargetFY04
TargetFY05
TargetInitiative Owner
LG1: Support continuous, shared staff learning and improvement
LG1a: Percentage of staff members with IDPs completed by February 28 each fiscal year
baseline 66% 100% Develop Individual Development Plan for each staff member
LG1: LT
24
LG1a: Percentage of staff members with IDPs completed by February 28 each fiscal year
• Staff are asked to develop Individual Development Plans (IDPs) in conjunction with their supervisor each year
• IDPs list areas for development and training for each staff member for the coming calendar year
• Target: 33% of staff will have IDPs developed by February 28, 2003
• Results: 39% of staff had IDPs by February 28, 2003
25
Financial Perspective
26
Financial Perspective
ObjectiveLG1c: Percentage of training modules
completed on schedule.FY 03 Target
FY04 Target
FY05 Target
Initiative Owner
F1: Maintain or reduce cost of information provided to the NIH staff
F1: Change in unit cost of information = FY00 = FY00 = FY00 Monitor use of resources and maintain access to the most heavily used
F1: SG
F2: Minimize unit cost for custom research assistance at a defined level of service (DS1)
F2a: Change in unit cost F2b: No. of contracts/IAG's
>FY02 >FY03 >FY04
Promote custom services F2: SG & SW
F3: Minimize unit cost of providing copies of publications at a defined level of service (DS2)
F3a: Change in unit cost F3b: No. of document requests filled
<FY02 <FY03 <FY04
F3: SG & RS
F4: Minimize unit cost for translating documents at a defined level of service (DS3)
F4a: Change in unit cost F4b: No. of documents translated
>FY02 >FY03 >FY04
Utilize contract staff when appropriate
F4: SG & TC
F5: Minimize unit cost for providing primary library services at a defined level of service (DS4)
F5a: Change in unit cost F5b: No. of units of information provided
>FY02 >FY03 >FY04
F5: SG, RS, SW
F6: Minimize unit cost for providing self service copiers at a defined level of service (DS5)
F6: Change in unit cost F6b: No. of visits to self-service photocopy center
<FY02 <FY03 <FY04
Monitor and adjust number of photocopy machines available to users
F6: SG & RS
27
F1: Change in “Unit Cost of Information” – Number of Units Used• Use of information units has
increased 35% in the past four years
• A unit of information may be a search, a document supplied, a customer trained, an article downloaded, or a visit to our web site
• The electronic resources are heavily used and their use increases each year
6.2
7.37.8
9.6
0
2
4
6
8
10
12
FY00 FY01 FY02 FY03
Fiscal Year
Un
its
in M
illio
ns
28
F1: Change in “Unit Cost of Information” – Unit Costs FY00–FY03• Target: Maintain or reduce the
cost of information provided to NIH staff
• The unit cost of information is calculated by dividing the cost of the information into the number of information units used
• The unit cost of information has remained fairly steady over the past 4 years
• E-resources are more expensive than print resources; however, they are used more heavily thereby keeping the unit cost of information steady
$0.91$1.03
$0.91 $0.93
$0.00
$0.50
$1.00
$1.50
FY00 FY01 FY02 FY03
Fiscal Year
Un
it C
os
t in
Do
llars
29
F2-6a: Change in Unit Cost• The unit cost for provide
copies of publications is lower in FY04. In FY03 276,000 copies of publications were supplied. As more documents are delivered as e-documents, costs are reduced.
• The unit cost for self-service copiers rose in FY04. In FY03 there were 24,000 visits to the self-service copiers. Use of copiers has decreased and hence the unit cost for the service has increased.
$0
$1
$2
$3
$4
$5
$6
FY01 FY02 FY03 FY04
Fiscal YearC
ost
in D
olla
rs
Provide Copies of Publications
Provide Self-Service Copiers
30
F2-6a: Change in Unit Cost (cont.)
• The unit cost for translate documents has risen over the past 4 years. The costs are largely based on salaries and contracting costs. 755 items were translated in FY04.
• The unit cost for provide primary library services has risen over the past 4 years. This service is growing. Costs are primarily based on salaries and purchase of print and e-resources. The cost of print and electronic resources rises 10-20% each year, and the Library is increasing the amount spent on e-resources by 20% each year. There were 9.6 million units of information provided in FY04.
• The unit cost for provide custom research assistance has not changed since last year. Seventeen custom projects were undertaken in FY04.
$0
$100
$200
$300
$400
$500
$600
FY01 FY02 FY03 FY04
Fiscal YearC
ost
in D
olla
rs
Provide Custom ResearchAssistanceTranslate Documents
Provide Primary Library Services
31
Conclusions
32
Accomplishments for FY03• Customers are very satisfied with the Library’s resources and services.
• The website has been redesigned in content management software, a new brand has been developed for the Library, and new print materials have been designed. These materials will be introduced to the NIH staff on February 10, 2004.
• Full-Text Plus! software that will enable users to link easily from databases to journal articles to training and more has been implemented.
• Relais II has been implemented, which has streamlined the process used to fill document requests, and a web-based document request form is available to customers.
• The “Unit Cost of Information” remains steady as we add new electronic resources to the collection.
33
Initiatives for FY04• We are in the process of developing a web-based survey, in conjunction with the NLM, to
assess customer satisfaction with the informationist service.
• We will be conducting either the bi-annual user satisfaction telephone survey or a user needs assessment in FY04, and we will use the scorecard to gauge customer satisfaction with provide copies of publications and reference service provided at the information desk.
• The Parklawn Health Library will become a branch of the NIH Library.
• In FY04, MetaLib, software that will enable searching across many e-resources, will be implemented.
• The Web of Science software will be updated to the Web of Knowledge.
• Development of an in-house training curriculum to ensure all staff are knowledgeable about Library services and to retrain staff for new work will continue.
• To maintain the unit cost of information, resources and services will continue to be selected based on their usefulness to NIH staff and cost per use will be a major factor in the selection and retention of electronic resources.
34
AppendixCustomer Perspective
C1a: Percent of current print journals in electronic format……………..…38C1b: Percentage of license agreements that cover all NIH……………....39C1c: Percentage of document requests filled in-house…..……………….40C1d: Number of items translated by government staff and contractors…41C2a: Number of web-based instruction modules………………………..…43C2b: Training attendance by type……………………………………………44C2c: Training evaluation scores…………………………………………..…45C3a: Customer Scorecard ratings for translation service and
mediated database search service..………………………………….…..48C3c: Digital reference service evaluation ratings…………………………..54C4: Customer segmentation data……………………………………..……..56
Internal Business Process PerspectiveIB1: Number and type of special projects contracted for…………..…..…63IB2: Number of new IT systems implemented…………………………..…65IB3a: Percent of document requests filled in-house in 3 business days……………………………………………………………………….….69IB3b: Percentage of documents translated by library staff
within 3 business days…………………….…….……………………..…..70IB3c: Percent of tutorial requests responded to within 1
business day………………………………………………….….….71
35
Appendix (cont.)
Learning and Growth PerspectiveLG1a: Percentage of staff members with IDPs completed by
February 28 each fiscal year……………………………………...........…74LG1b: Percentage of staff completing 80% of activities listed in
IDP.......................................................................................................75LG1c: Percentage of training modules completed on schedule…………76
Financial PerspectiveF1: Change in unit cost of information…………….……………………….79
F2a: Change in unit cost of custom services…………………………..….82 F2b: No. of contracts/IAGs……………………………………………….….82
F3a: Change in unit cost of providing copies of publications…………….82 F3b: No. of document requests filled……………………………………….82
F4a: Change in unit cost of translating services…………………….…….83 F4b: No. of documents translated…………………………………….…….83
F5a: Change in unit cost of providing primary library services……….….83 F5b: No. of units of information provided…………………………….…….83
F6a: Change in unit cost of providing self-service copiers…………...….83 F6b: No. of visits to self-service copy center………………………..…….83
36
Customer Perspective
37
Customer Perspective
Objective MeasureFY 03 Target
FY04 Target
FY05 Target
Initiative Owner
C1: Provide integrated transparent acccess to the most relevant information via themost effective information sources for NIH researchers and staff
C1a. Percent of current print journals in electronic formatC1b. Percentage of license agreements that cover all NIHC1c. Percentage of document requests filled in-houseC1d. Number of items translated (by gov and contr)
> FY02
> FY02
> FY02
> FY03
> FY03
> FY03
> FY04
> FY04
> FY04
Convert all journals to e-format.Negotiate all e-license agreements for access by all of NIH.
C1a: RSC1b: RSC1c: RSC1d: TC
38
C1a: Percent of current print journals in electronic format
• Target: Provide 100% of current journals in electronic format
• Eighty-one percent of the journals are now available in electronic format (as compared to 75% in FY02 and 52% in FY01)
1597
1035739
2383
3085
3171
0
500
1000
1500
2000
2500
3000
3500
4000
4500
FY01 FY02 FY03
Fiscal Year
No.
of
Title
s
Print Only Electronic journals
39
C1b: Percentage of license agreements that cover all NIH
• Target: Negotiate all electronic resource license agreements for access by all of NIH
• Currently 62% of e-license agreements give all NIH staff access (as compared to 53% in FY02 and 45% in FY01) 0%
25%
50%
75%
100%
FY01 FY02 FY03
Fiscal Year
Per
cen
t A
gre
emen
ts
Access NIH-w ide Limited Access
40
C1c: Percentage of document requests filled in-house• Target: Maintain percentage
of document requests filled in-house or at NLM to keep costs down and turn around time fast
• Results: Percentage filled in-house in FY03 was 82% and in FY02 was 84%
• The large number of requests that can be filled in-house reflects the careful tailoring of the collection to the needs of the NIH staff
0
20
40
60
80
100
120
FY02 FY03
Fiscal Year
Pe
rce
nta
ge
Filled at otherlibraries
Filled at NLM
Filled in-house
41
C1d: Number of items translated by government staff and contractors• The total number of items
translated in FY03 was 755 items
• The number of words translated has increased 44% over the past 6 years
• To ensure items are translated in a timely manner increasingly, items are given to contractors to translate
499 465 500 513 510 496421
7476
84121
179267
334
0
100
200
300
400
500
600
700
800
900
FY97FY98
FY99FY00
FY01FY02
FY03
Fiscal Year
No
. of
Req
ues
ts
Contractors
Government Staff
42
Customer Perspective (cont.)
Objective MeasureFY 03
TargetFY04
TargetFY05
TargetInitiative Owner
C2: Enhance researchers referencing knowledge and skills to support their contribution to advancing medical research and patient care
C2a. Number of web-based instruction modules C2b: Training attendance by type (classes, tutorials, etc. versus web-based)C2c: Training evaluation scores
> FY02 = FY03 = FY04 Develop web-based instruction modules for all training classes, if feasible
Instruction Team/SW
43
C2a: Number of web-based instruction modules• Target: Develop web-based
instruction modules for all training classes, if feasible
• We have found that realistically only about half of our classes can be made into web-based modules.
• Library staff currently are busy updating all modules with new NIH Library branding
2024
32 32
13
19 17
0
10
20
30
40
50
60
FY00 FY01 FY02 FY03
Fiscal Year
Nu
mb
er
of
Cla
ss
es
Web-based
Seminars and Tutorials
44
C2b: Training attendance by type• Customers have clearly
indicated their preference for web-based training which is available 24/7
• Nevertheless, attendance at classes remains steady
• Class attendance numbers are used to determine which classes to offer
1152 904 1219 1122525
4031
8094
0
1,000
2,000
3,000
4,000
5,000
6,000
7,000
8,000
9,000
10,000
FY00 FY01 FY02 FY03
Fiscal Year
No
. Sta
ff
Web-basedTraining
Seminars &Tutorials
45
C2c: Training evaluation scores• Classes are provided to teach NIH staff effective use of
electronic resources and bibliographic management software
• In FY03, 75% of attendees (451/599) completed an evaluation form designed to assess satisfaction with the training classes
• Customers were asked to rate their satisfaction on a 5 point scale with 1 being poor and 5 being excellent
• The classroom was upgraded at the beginning of FY03 and the rating for the classroom rose from 4.2 to 4.6
46
C2c: Training evaluation scores
4.34.4 4.5
4.2 4.2 4.24.4 4.5 4.54.3 4.3
4.6
0.0
1.0
2.0
3.0
4.0
5.0
Prese
ntat
ion
Knowled
ge
Respo
nsive
ness
Hando
ut
Obje
ctive
Class
room
Criteria
Ave
rag
e R
esp
on
se (
5 p
t. s
cale
)FY02 FY03 N=451
47
Customer Perspective (cont.)
Objective MeasureFY 03
TargetFY04
TargetFY05
TargetInitiative Owner
•Following are the results of the survey of customers satisfaction with three services:
•Translation Services
•Mediated database searching
•Digital reference service
•Evaluations were transaction based
•All services received high scores
C3:Increase customer/stakeholder satisfaction with overall service
C3a. Customer Scorecard ratings for translation service and mediated database searchingC3b: Bi-annual telephone survey ratings C3c: Digital reference service evaluation ratings
> FY02
> FY02
> FY02
> FY03
> FY03
> FY03
> FY04
> FY04
> FY04
Use scorecard to evaluate provide copies of publications and reference services delivered at the information desk in FY04
C3a: TC & JV C3b: NAC3c: SW
48
C3a: Customer Scorecard ratings for translation service• Evaluation surveys were distributed to customers who used the
translation service in FY02 and FY03
• Customers were asked to rate satisfaction and importance of the service on a 10 point scale with 1 being unsatisfactory and 10 being outstanding
• Customers were highly satisfied with all aspects of the translation service. In addition, customers felt that timeliness was much more important than cost
Survey distribution FY02 FY03
No. of surveys distributed 88 390
No. of respondents 37 70
Response rate 42% 18%
49
C3a: Customer Scorecard ratings for translation service – Satisfaction Ratings
0
2
4
6
8
10
Handl
ing o
f Pro
blem
s
Com
peten
ce
Con
veni
ence
Res
pons
ivene
ss
*Ava
ilabi
lity
Rel
iabilit
y
Tim
eline
ss
Qua
lity
Cost
Criteria
Un
sati
sfac
tory
Ou
tsta
nd
ing FY02 FY03 N=70
* FY03 availability ratings are significantly higher (p< .05) than FY02 ratings
50
C3a: Customer Scorecard ratings for translation service – Importance Ratings
0
2
4
6
8
10
Handli
ng o
f Pro
blems
Com
petenc
e
Con
venie
nce
Res
pons
ivene
ss
Availa
bility
Reli
abilit
y
Tim
elines
s
Quality
Cost
Criteria
Un
sati
sfac
tory
O
uts
tan
din
g FY02 FY03 N=70
51
C3a: Customer Scorecard ratings for mediated database searching service
• Mediated database searching service was evaluated from mid-October to mid-December of 2003 using paper or electronic survey forms
• Forty surveys were distributed and 14 were returned for a return rate of 35%
• The surveys asked users of the service to rate customer service and search results on a 10 point scale with 1 being unsatisfactory and 10 being outstanding
• Customers were very satisfied with the service
52
C3a: Customer Scorecard ratings for mediated database searching service – Customer Service
9.93 10 10 10 9.93
123456789
10
Understanding ofQuery
Competence Responsiveness Convenience Availability
Criteria
Un
sati
sfac
tory
O
uts
tan
din
g
FY03 N = 14
53
C3a: Customer Scorecard ratings for mediated database searching service – Search Results
9.29 9.299.79 9.36
123456789
10
Presentation Relevance Timeliness Quality
Criteria
Uns
atis
fact
ory
O
utst
andi
ng
FY03 N=14
54
C3c: Digital reference service evaluation ratings• The Library has offered a digital reference service for the past
two years
• Digital reference combines chat and the ability to push web pages to customers in response to a reference question
• Customers are given the option of filling out an evaluation form at the end of each transaction
• In FY02 37% of customers responded to the survey and in FY03 the response rate was 39%
• Customers found the service easy to use and would use it again. Survey questions will be revised in FY04 to give a more rounded view of the customer’s satisfaction with the service.
55
C3c: Digital reference service evaluation ratings
8378
68 72
60
8493
89
7682 85
92
0
10
20
30
40
50
60
70
80
90
100
Ease of use Moreconvenientthan email
Moreconvenientthan phone
Questionanswered
Saved me time Will use again
Criteria
Pe
rce
nta
ge
re
sp
on
din
g a
gre
e o
r s
tro
ng
ly
ag
ree
FY02 FY03
56
Customer Perspective (cont.)
Objective MeasureFY 03
TargetFY04
TargetFY05
TargetInitiative Owner
C4: Increase understanding of customer base
C4. Customer segmentation data C4; RS & TC
57
C4: Customer segmentation data• Customer segmentation data is presented for four services:
• Provide copies of publications• Provide self-service copiers• Translate documents• Provide custom research assistance
• As in past years, customers are spread across all IC’s
• Heavier use of services by larger IC’s and groups most directly engaged in patient care and biomedical research
• The Library serves not only NIH but also FDA/CBER on the Bethesda campus and provides document delivery to several groups including NIST and CDC.
58
C4: Customer Segmentation for Provide Copies of Publications
0
10000
20000
30000
40000
50000
60000
70000
80000
90000
NCI
NIMH CC
NICHD
NINDS
NIAID
NIDDK
NHLBI
CBERNID
ANIA
NIDCR
NIAAA
NHGRI
NIDCD
NEIOD
NIAMS
NINR
NIGMS
NLMORS
NCCAM
NCRRFI
CCIT
HHMICSR
NIEHS
NIBIB
NCMHD
Customers
Nu
mb
er
of
Re
qu
es
ts F
ille
d
FY02 FY03
59
C4: Customer Segmentation for Provide Self-Service Copiers
0
1000
2000
3000
4000
5000
6000
7000
Customers
Num
ber
of
Vis
its
FY2002 FY2003
60
C4: Customer Segmentation for Translate Documents
0
20
40
60
80
100
120
140
160
180
NCI
NIAID
NICHD
NIDDK
NIMH
NINDS
NHLBI
NHGRI
NIDCR CC
NIDCD
NEINIA
NLM
CBER OD
NIAMS
FICORS
NIEHS
NIDA
CITNIA
A
Customers
Jo
bs
Co
mp
lete
d
FY 2001 FY 2002 FY 2003
61
C4: Customer Segmentation for Provide Custom Research Assistance
0
1
2
3
4
5
Customers
Nu
mb
er
of
Job
s
FY02 FY03
62
Internal Business Perspective
63
Internal Business Perspective
Objective MeasureFY 03 Target
FY04 Target
FY05 Target
Initiative Owner
IB1: Promote value added custom service projects
IB2. Number and type of special projects contracted for
> FY02 > FY03 > FY04 Hire more informationists and promote informationist service to IC's
IB1: SW
64
IB1: Number and type of special projects contracted for
• Target for FY04 is to hire more informationists and increase the number of informationist assignments to 8
Custom Services FY02 FY03
Collection Related 6 7
Information Related
Bibliography, database, web page 7 5
Informationist assignments 3 5
Totals 16 17
65
Internal Business Perspective (cont.)
Objective MeasureFY 03
TargetFY04
TargetFY05
TargetInitiative Owner
IB2: Adopt best practices and infuse new technology
IB2: Number of new IT systems implemented.
Plan to implement HHS Consortium AccessPlan to implement SFXPlan to implement RelaisPlan to implement CMS
IB2: SW, SG, JV, RS
66
IB2: Number of new IT systems implemented
• There were four areas of concentration in FY03:• HHS Library Consortium • SFX or Full-Text Plus! • Relais• Content Management Software
• The major areas of concentration for FY04 are:• Continue with the development of the HHS Consortium, Full-
Text Plus!, Relais, and CMS• Implement MetaLib• Enable wireless access in the Library
67
New IT systems implemented in FY03• HHS Library Consortium –In FY03 the Consortium jointly purchased
the Nature journals, Science, and Science Express at a considerable savings to NIH
• Full-Text Plus! – This software, which links various services and resources available from the Library, was implemented for the online journals, the Web of Science and Ovid databases, the new online document request form, and other resources in FY03
• Relais – In FY03, phase II of the Relais system was implemented with the integration of the web-based request form and the Web of Science requests. Through Relais the processing of document requests has been considerably streamlined. A process that used to have eight steps now has only three with a turn around time of 1-3 business days
• Content Management Software has been implemented for the Library’s Intranet and public websites resulting in a diversification of responsibility for the sites. The redesigned public website will be unveiled on February 10, 2004
68
Internal Business Perspective (cont.)
Objective MeasureFY 03
TargetFY04
TargetFY05
TargetInitiative Owner
IB3: Maintain or improve turn around time for filling requests with accuracy
IB3a: Percent of document requests filled in-house in 3 business daysIB3b: Percentage of documents translated by library staff within 3 business daysIB3c: Percent of tutorial requests responded to within I business day
> FY02
> FY02
> FY02
> FY03
> FY03
> FY03
> FY04
> FY04
> FY04
IB3a: RSIB3b: TCIB3c: SW
69
IB3a: Percent of document requests filled in–house in 3 business days
• Target: Fill 96% of requests in 3 business days
• Results: Average fill rate for FY03 was 98% and for FY03 was 97%
70
IB3b: Percentage of documents translated by library staff within 3 business days• Target: Translate document requests in 3 business days
• Results: In FY03 an average 55% of documents were translated in 3 business days. In FY04 the translation speed was measured for Library translators and contractors separately. In FY04 Library translators translated 55% of documents in 3 days and contractors translated 26% of documents in 3 days.
• Efforts to improve the turn around time for translating documents are complicated by:
• Translation requests are increasing and the current staff are already working as efficiently as possible
• The more translations that are contracted out, the more time the Library translation staff must spend checking contracted translations thereby reducing the time they have to translate
71
IB3c: Percent of tutorial requests responded to within 1 business day
100% 100%
0%
20%
40%
60%
80%
100%
FY02 FY03
Fiscal Year
Pe
rce
nt
72
Learning and Growth Perspective
73
Learning and Growth Perspective
Objective MeasureFY 03
TargetFY04
TargetFY05
TargetInitiative Owner
LG1: Support continuous, shared staff learning and improvement
LG1a: Percentage of staff members with IDPs completed by February 28 each fiscal yearLG1b: Percentage of staff completing 80% of activities listed in IDPLG1c: Percentage of training modules completed on schedule
baseline
baseline
66%
80%
100%
80%
Develop Individual Development Plan for each staff member Develop in-house training plan
LG1: LT, TC, CCC
74
LG1a: Percentage of staff members with IDPs completed by February 28 each fiscal year
• Staff are asked to develop Individual Development Plans (IDPs) in conjunction with their supervisor each year
• IDPs list areas for development and training for each staff member for the coming calendar year
• Target: 33% of staff will have IDPs developed by February 28, 2003
• Results: 39% of staff had IDPs by February 28, 2003
75
LG1b: Percentage of staff completing 80% of activities listed in IDP
• Target: Eighty percent of staff will have completed 80% of IDP objectives by Dec. 30
• Result: 88% of staff with plans in place in February completed 80% of their objectives
76
LG1c: Percentage of training modules completed on schedule
• Training Project Team established in FY03
• Projects for FY04 are:• Survey Library staff to assess the state of learning
in the NIH Library• Conduct needs assessment• Develop training plan
77
Financial Perspective
78
Financial Perspective
ObjectiveLG1c: Percentage of training modules
completed on schedule.FY 03 Target
FY04 Target
FY05 Target
Initiative Owner
F1: Maintain or reduce cost of information provided to the NIH staff
F1: Change in unit cost of information = FY00 = FY00 = FY00 Monitor use of resources and maintain access to the most heavily used
F1: SG
F2: Minimize unit cost for custom research assistance at a defined level of service (DS1)
F2a: Change in unit cost F2b: No. of contracts/IAG's
>FY02 >FY03 >FY04
Promote custom services F2: SG & SW
F3: Minimize unit cost of providing copies of publications at a defined level of service (DS2)
F3a: Change in unit cost F3b: No. of document requests filled
<FY02 <FY03 <FY04
F3: SG & RS
F4: Minimize unit cost for translating documents at a defined level of service (DS3)
F4a: Change in unit cost F4b: No. of documents translated
>FY02 >FY03 >FY04
Utilize contract staff when appropriate
F4: SG & TC
F5: Minimize unit cost for providing primary library services at a defined level of service (DS4)
F5a: Change in unit cost F5b: No. of units of information provided
>FY02 >FY03 >FY04
F5: SG, RS, SW
F6: Minimize unit cost for providing self service copiers at a defined level of service (DS5)
F6: Change in unit cost F6b: No. of visits to self-service photocopy center
<FY02 <FY03 <FY04
Monitor and adjust number of photocopy machines available to users
F6: SG & RS
79
F1: Change in “Unit Cost of Information” – Number of Units Used• Use of information units has
increased 35% in the past four years
• A unit of information may be a search, a document supplied, a customer trained, an article downloaded, or a visit to our web site
• The electronic resources are heavily used and their use increases each year
6.2
7.37.8
9.6
0
2
4
6
8
10
12
FY00 FY01 FY02 FY03
Fiscal Year
Un
its
in M
illio
ns
80
F1: Change in “Unit Cost of Information” – Unit Cost FY00–FY03• Target: Maintain or reduce the
cost of information provided to NIH staff
• The unit cost of information is calculated by dividing the cost of the information into the number of information units used
• The unit cost of information has remained fairly steady over the past 4 years
• E-resources are more expensive than print resources; however, they are used more heavily thereby keeping the unit cost of information steady
$0.91$1.03
$0.91 $0.93
$0.00
$0.50
$1.00
$1.50
FY00 FY01 FY02 FY03
Fiscal Year
Un
it C
os
t in
Do
llars
81
F1: Calculations for the “Unit Cost of Information”
Units of Information FY00 FY01 FY02 FY03Information requests 44,656 40,558 40,558 30,000Searches performed 3,017 892 849 1,200NIH staff trained 1,335 1,454 5,441 9,300Books circulated/renewed 45,294 40,233 43,966 43,000Documents supplied from collection 327,405 289,863 255,194 324,000Home Page visits 646,550 842,856 1,055,001 1,116,000Web of Science searches 4,495,507 5,036,855 5,137,149 5,368,000Psychological Abstracts searches 9,277 10,020 12,097 12,000Biological Abstracts searches 4,867 5,930 2,637 2,000Ovid Medline searches 8,905 11,772 25,468 30,000Other Ovid database searches n/a n/a 4,201 4,105Articles downloaded from Amer Soc for Microbiology journals n/a n/a 319,614 274,000Articles downloaded from Cell Press journals n/a n/a 105,678 137,000Articles downloaded from JBC 166,476 188,544 225,580 264,000Articles downloaded from Nature Journals n/a n/a 405,658 502,000Articles downloaded from PNAS 94,725 125,380 149,544 185,000Articles downloaded from Science 129,971 152,302 207,012 212,000Articles downloaded from Elsevier/Academic Press 219,743 529,894 604,401 879,000Articles downloaded from misc e-journals n/a n/a n/a 250,000TOTALS 6,197,728 7,276,553 8,600,048 9,642,605
Primary Services Budget FY00* FY01** FY02 FY035,644,550 7,466,190 7,831,358 $8,953,706
Unit cost of information FY00* FY01 FY02 FY030.91$ 1.03$ 0.91$ $0.93
* FY00 budget did not include rent and overhead
** In FY01 NIHLB assumed added e-resources costs for Baltimore , Frederick and RML sites, plus paid rent and overhead
82
F2-6a: Change in Unit Cost• The unit cost for provide
copies of publications is lower in FY04. In FY03 276,000 copies of publications were supplied. As more documents are delivered as e-documents, costs are reduced.
• The unit cost for self-service copiers rose in FY04. In FY03 there were 24,000 visits to the self-service copiers. Use of copiers has decreased and hence the unit cost for the service has increased.
$0
$1
$2
$3
$4
$5
$6
FY01 FY02 FY03 FY04
Fiscal YearC
ost
in D
olla
rs
Provide Copies of Publications
Provide Self-Service Copiers
83
F2-6a: Change in Unit Cost (cont.)
• The unit cost for translate documents has risen over the past 4 years. The costs are largely based on salaries and contracting costs. 755 items were translated in FY04.
• The unit cost for provide primary library services has risen over the past 4 years. This service is growing. Costs are primarily based on salaries and purchase of print and e-resources. The cost of print and electronic resources rises 10-20% each year, and the Library is increasing the amount spent on e-resources by 20% each year. There were 9.6 million units of information provided in FY04.
• The unit cost for provide custom research assistance has not changed since last year. Seventeen custom projects were undertaken in FY04.
$0
$100
$200
$300
$400
$500
$600
FY01 FY02 FY03 FY04
Fiscal YearC
ost
in D
olla
rs
Provide Custom ResearchAssistanceTranslate Documents
Provide Primary Library Services
84
Conclusions
85
Accomplishments for FY03• Customers are very satisfied with the Library’s resources and services.
• The website has been redesigned in content management software, a new brand has been developed for the Library, and new print materials have been designed. These materials will be introduced to the NIH staff on February 10, 2004.
• Full-Text Plus! software that will enable users to link easily from databases to journal articles to training and more has been implemented.
• Relais II has been implemented, which has streamlined the process used to fill document requests, and a web-based document request form is available to customers.
• The “Unit Cost of Information” remains steady as we add new electronic resources to the collection.
86
Initiatives for FY04• We are in the process of developing a web-based survey, in conjunction with the NLM, to
assess customer satisfaction with the informationist service.
• We will be conducting either the bi-annual user satisfaction telephone survey or a user needs assessment in FY04, and we will use the scorecard to gauge customer satisfaction with provide copies of publications and reference service provided at the information desk.
• The Parklawn Health Library will become a branch of the NIH Library.
• In FY04, MetaLib, software that will enable searching across many e-resources, will be implemented.
• The Web of Science software will be updated to the Web of Knowledge.
• Development of an in-house training curriculum to ensure all staff are knowledgeable about Library services and to retrain staff for new work will continue.
• To maintain the unit cost of information, resources and services will continue to be selected based on their usefulness to NIH staff and cost per use will be a major factor in the selection and retention of electronic resources.
87
The End