library assessment
DESCRIPTION
Presentation to Pratt Institute, Information and Library School, Human Information Behavior ClassTRANSCRIPT
![Page 1: LIBRARY ASSESSMENT](https://reader033.vdocuments.net/reader033/viewer/2022051609/547d4668b47959bb508b48d1/html5/thumbnails/1.jpg)
LIBRARY ASSESSMENT
Jennifer Rutner, Senior Analyst, Ithaka S+R Pratt Institute
Human Information Behavior September 22, 2011
![Page 2: LIBRARY ASSESSMENT](https://reader033.vdocuments.net/reader033/viewer/2022051609/547d4668b47959bb508b48d1/html5/thumbnails/2.jpg)
Senior Analyst, Ithaka S+R
School:
BA in Religious Studies, 2002
MLIS from Pratt, 2005
Enrolled in Quantitative Methods for the Social Sciences at Columbia, 2010-?
Work:
Assessment & Planning Librarian, Columbia University Libraries, 2006-2011
Chair, ACRL Assessment Committee, 2011
![Page 3: LIBRARY ASSESSMENT](https://reader033.vdocuments.net/reader033/viewer/2022051609/547d4668b47959bb508b48d1/html5/thumbnails/3.jpg)
ASSESSMENT LIBRARIANSHIP
![Page 4: LIBRARY ASSESSMENT](https://reader033.vdocuments.net/reader033/viewer/2022051609/547d4668b47959bb508b48d1/html5/thumbnails/4.jpg)
Assessment
“To assess, in general, is to determine the importance, size, or value of; to evaluate. Library staff assess operations by collecting, interpreting, and using data to make decisions and improve customer service.”
ARL Spec Kit #303, Library Assessment, December 2007
![Page 5: LIBRARY ASSESSMENT](https://reader033.vdocuments.net/reader033/viewer/2022051609/547d4668b47959bb508b48d1/html5/thumbnails/5.jpg)
What we talk about when we talk about “assessment”
assessment Evaluation
User research
Quality assurance
ROI/Value
Impact
![Page 6: LIBRARY ASSESSMENT](https://reader033.vdocuments.net/reader033/viewer/2022051609/547d4668b47959bb508b48d1/html5/thumbnails/6.jpg)
Culture of Assessment
A Culture of Assessment is an organizational environment in which decisions are based on facts, research and analysis, and where services are planned and delivered in ways that maximize positive outcomes and impacts for library clients.
A Culture of Assessment exists in organizations
where staff care to know what results they produce and how those results relate to customer expectations.
Amos Lakos: www.usc.edu/.../locations/leavey/news/conference/presentations/presentations_9-16/Assessment/UCLA_Lakos.ppt
![Page 7: LIBRARY ASSESSMENT](https://reader033.vdocuments.net/reader033/viewer/2022051609/547d4668b47959bb508b48d1/html5/thumbnails/7.jpg)
Why assess?
![Page 8: LIBRARY ASSESSMENT](https://reader033.vdocuments.net/reader033/viewer/2022051609/547d4668b47959bb508b48d1/html5/thumbnails/8.jpg)
Why assess?
![Page 9: LIBRARY ASSESSMENT](https://reader033.vdocuments.net/reader033/viewer/2022051609/547d4668b47959bb508b48d1/html5/thumbnails/9.jpg)
Why assess?
Stuff I know Stuff I know I don't know
Stuff I don't know I don't
know
![Page 10: LIBRARY ASSESSMENT](https://reader033.vdocuments.net/reader033/viewer/2022051609/547d4668b47959bb508b48d1/html5/thumbnails/10.jpg)
How is data used in libraries?
• Strategic planning + management
• Decision making
• Program evaluation
• Advocacy
• Budgeting
• Regular service improvements
![Page 11: LIBRARY ASSESSMENT](https://reader033.vdocuments.net/reader033/viewer/2022051609/547d4668b47959bb508b48d1/html5/thumbnails/11.jpg)
![Page 12: LIBRARY ASSESSMENT](https://reader033.vdocuments.net/reader033/viewer/2022051609/547d4668b47959bb508b48d1/html5/thumbnails/12.jpg)
Assessment Mission (CUL)
“Serve library users and staff through the gathering, analysis, and application of high-quality, actionable information to guide library decision making.”
![Page 13: LIBRARY ASSESSMENT](https://reader033.vdocuments.net/reader033/viewer/2022051609/547d4668b47959bb508b48d1/html5/thumbnails/13.jpg)
The Research Process
Establish research
questions
Environmental scan
Gather available
information
Information Needs
(unknowns)*
Establish priorities
Assign methodology
Develop tools Test tools
IRB Conduct
assessment Analysis
Decision making
![Page 14: LIBRARY ASSESSMENT](https://reader033.vdocuments.net/reader033/viewer/2022051609/547d4668b47959bb508b48d1/html5/thumbnails/14.jpg)
The Assessment Process
Brainstorming Information Need
Priority Methodology Formal Question
Outcome
“I want to know…”
![Page 15: LIBRARY ASSESSMENT](https://reader033.vdocuments.net/reader033/viewer/2022051609/547d4668b47959bb508b48d1/html5/thumbnails/15.jpg)
RESEARCH METHODS
![Page 16: LIBRARY ASSESSMENT](https://reader033.vdocuments.net/reader033/viewer/2022051609/547d4668b47959bb508b48d1/html5/thumbnails/16.jpg)
Surveys
Use random sampling to generalize to the broader population.
Set questions, with no opportunity for follow-up.
Allows for statistical analysis.
![Page 17: LIBRARY ASSESSMENT](https://reader033.vdocuments.net/reader033/viewer/2022051609/547d4668b47959bb508b48d1/html5/thumbnails/17.jpg)
Questionnaires Evaluation tool: How was it?
Exploratory tool: What do you do?
Preferences tool: What do you want?
Reporting tool: What did you do?
Analysis stops at descriptive statistics.
![Page 18: LIBRARY ASSESSMENT](https://reader033.vdocuments.net/reader033/viewer/2022051609/547d4668b47959bb508b48d1/html5/thumbnails/18.jpg)
Response Rate vs. Representativeness
“It ain’t response rate.”
0.00%
10.00%
20.00%
30.00%
40.00%
50.00%
60.00%
70.00%
% of population
% of respondents
![Page 19: LIBRARY ASSESSMENT](https://reader033.vdocuments.net/reader033/viewer/2022051609/547d4668b47959bb508b48d1/html5/thumbnails/19.jpg)
Focus Groups “Researchers attempt to capture peoples explanations”
• Answering “how?” “why?” and “what?” questions – open ended question
• Look at a topic up-close, rather than get a panoramic view of the entire issue
Focus groups can uncover insights and perspectives that are not retrievable by other methods, from a small group of participants.
![Page 20: LIBRARY ASSESSMENT](https://reader033.vdocuments.net/reader033/viewer/2022051609/547d4668b47959bb508b48d1/html5/thumbnails/20.jpg)
Focus Groups
Standard
– Uses a rigid set of questions
Guided
– Uses a set of topics to explore, but the question wording is flexible
Exploratory
– The most informal, questions arise through the course of the conversation about the topic
![Page 21: LIBRARY ASSESSMENT](https://reader033.vdocuments.net/reader033/viewer/2022051609/547d4668b47959bb508b48d1/html5/thumbnails/21.jpg)
Ethnography
Methodologies
– Work study
– Photo essays
– Dream-catcher workshops
– Mapping diaries
– Pilot projects
– Interviews
![Page 22: LIBRARY ASSESSMENT](https://reader033.vdocuments.net/reader033/viewer/2022051609/547d4668b47959bb508b48d1/html5/thumbnails/22.jpg)
Observation
Wait, watch, write. Head Count+
• Sections of the library
• Type of seating
• Group vs. individual
• Food/drink
• Technology
![Page 23: LIBRARY ASSESSMENT](https://reader033.vdocuments.net/reader033/viewer/2022051609/547d4668b47959bb508b48d1/html5/thumbnails/23.jpg)
Way Finding: 3D Usability
How do users navigate space?
– Provide task
– Record steps to completion
– Record completion rate
– Record challenges
![Page 24: LIBRARY ASSESSMENT](https://reader033.vdocuments.net/reader033/viewer/2022051609/547d4668b47959bb508b48d1/html5/thumbnails/24.jpg)
Learning Outcomes
• Rubrics
• Pre- and post-tests
• Minute Papers
– What’s one useful thing you learned today?
– What’s one thing you’re still confused about?
How does this impact their academic success?
What are they learning?
What are we teaching?
![Page 25: LIBRARY ASSESSMENT](https://reader033.vdocuments.net/reader033/viewer/2022051609/547d4668b47959bb508b48d1/html5/thumbnails/25.jpg)
Usability Studies
Methods
• Card sort
• Heuristiv evaluation
• Paper prototyping
• Personas
• Task analysis
• Work-study
Evaluates
• Ease of use
• Efficiency of use
• Memorability
• Error frequency and severity
• Subjective satisfaction www.usability.gov
![Page 26: LIBRARY ASSESSMENT](https://reader033.vdocuments.net/reader033/viewer/2022051609/547d4668b47959bb508b48d1/html5/thumbnails/26.jpg)
ROI
“For every dollar invested in the library, the college/university/school/community/business received X dollars in return.”
“For every dollar invested in the library, the library produces X services, which can be valued at Y.”
![Page 27: LIBRARY ASSESSMENT](https://reader033.vdocuments.net/reader033/viewer/2022051609/547d4668b47959bb508b48d1/html5/thumbnails/27.jpg)
Institutional Review Board (IRB)
• Protect human subjects.
• Ensure research is ethical.
• Ensure research complies with federal and state laws.
Protocols include:
• Research questions and hypothesis
• Subject profiles
• Study procedures
• Recruitment materials
• Report on findings
![Page 28: LIBRARY ASSESSMENT](https://reader033.vdocuments.net/reader033/viewer/2022051609/547d4668b47959bb508b48d1/html5/thumbnails/28.jpg)
LIBQUAL+
![Page 29: LIBRARY ASSESSMENT](https://reader033.vdocuments.net/reader033/viewer/2022051609/547d4668b47959bb508b48d1/html5/thumbnails/29.jpg)
LibQual+ Survey 2009 www.libqual.org
“22 questions and a box” Affect of Service
Information Control
Library as Place
Comments?
![Page 30: LIBRARY ASSESSMENT](https://reader033.vdocuments.net/reader033/viewer/2022051609/547d4668b47959bb508b48d1/html5/thumbnails/30.jpg)
LibQual+ Items
Affect of Service AS-1 Employees who instill confidence in users AS-2 Giving users individual attention AS-3 Employees who are consistently courteous AS-4 Readiness to respond to users' questions AS-5 Employees who have the knowledge to answer user
questions AS-6 Employees who deal with users in a caring fashion AS-7 Employees who understand the needs of their users AS-8 Willingness to help users AS-9 Dependability in handling users' service problems Library as Place LP-1 Library space that inspires study and learning LP-2 Quiet space for individual activities LP-3 A comfortable and inviting location LP-4 A getaway for study, learning, or research LP-5 Community space for group learning and group
study
Information Control IC-1 Making electronic resources accessible from my
home or office IC-2 A library Web site enabling me to locate information
on my own IC-3 The printed library materials I need for my work IC-4 The electronic information resources I need IC-5 Modern equipment that lets me easily access
needed information IC-6 Easy-to-use access tools that allow me to find things
on my own IC-7 Making information easily accessible for
independent use IC-8 Print and/or electronic journal collections I require
for my work Local Questions Providing help when and where I need it Making me aware of library services Availability of subject assistance Ability to navigate library Web pages Access to archives, special collections
![Page 31: LIBRARY ASSESSMENT](https://reader033.vdocuments.net/reader033/viewer/2022051609/547d4668b47959bb508b48d1/html5/thumbnails/31.jpg)
Response: Representativeness
Response by status across the University matches the population distribution very closely.
Greatest difference: 8%
This is representative data!
http://www.libqual.org/documents/admin/Representativeness.pdf
Status % of responses
% of population
Undergraduates 40.03% 32.38%
Graduates 53.21% 55.21%
Faculty 6.78% 12.40%
![Page 32: LIBRARY ASSESSMENT](https://reader033.vdocuments.net/reader033/viewer/2022051609/547d4668b47959bb508b48d1/html5/thumbnails/32.jpg)
Response: Representativeness
Response by discipline across the University matches the population distribution nearly perfectly.
E.g. We’re not missing anyone!
http://www.libqual.org/documents/admin/Representativeness.pdf
![Page 33: LIBRARY ASSESSMENT](https://reader033.vdocuments.net/reader033/viewer/2022051609/547d4668b47959bb508b48d1/html5/thumbnails/33.jpg)
Reading LibQual+ Charts
Desired
Minimum
Perceived/Reality Zone of Tolerance
Superiority Gap
Adequacy Gap
![Page 34: LIBRARY ASSESSMENT](https://reader033.vdocuments.net/reader033/viewer/2022051609/547d4668b47959bb508b48d1/html5/thumbnails/34.jpg)
Undergraduate Students
4
5
6
7
8
9
AS-1 AS-2 AS-3 AS-4 AS-5 AS-6 AS-7 AS-8 AS-9 IC-1 IC-2 IC-3 IC-4 IC-5 IC-6 IC-7 IC-8 LP-1 LP-2 LP-3 LP-4 LP-5
![Page 35: LIBRARY ASSESSMENT](https://reader033.vdocuments.net/reader033/viewer/2022051609/547d4668b47959bb508b48d1/html5/thumbnails/35.jpg)
Graduate Students
4
5
6
7
8
9
AS-1 AS-2 AS-3 AS-4 AS-5 AS-6 AS-7 AS-8 AS-9 IC-1 IC-2 IC-3 IC-4 IC-5 IC-6 IC-7 IC-8 LP-1 LP-2 LP-3 LP-4 LP-5
![Page 36: LIBRARY ASSESSMENT](https://reader033.vdocuments.net/reader033/viewer/2022051609/547d4668b47959bb508b48d1/html5/thumbnails/36.jpg)
Faculty Responses
4
5
6
7
8
9
AS-1 AS-2 AS-3 AS-4 AS-5 AS-6 AS-7 AS-8 AS-9 IC-1 IC-2 IC-3 IC-4 IC-5 IC-6 IC-7 IC-8 LP-1 LP-2 LP-3 LP-4 LP-5
![Page 37: LIBRARY ASSESSMENT](https://reader033.vdocuments.net/reader033/viewer/2022051609/547d4668b47959bb508b48d1/html5/thumbnails/37.jpg)
Library Staff
4
5
6
7
8
9
AS-1 AS-2 AS-3 AS-4 AS-5 AS-6 AS-7 AS-8 AS-9 IC-1 IC-2 IC-3 IC-4 IC-5 IC-6 IC-7 IC-8 LP-1 LP-2 LP-3 LP-4 LP-5
![Page 38: LIBRARY ASSESSMENT](https://reader033.vdocuments.net/reader033/viewer/2022051609/547d4668b47959bb508b48d1/html5/thumbnails/38.jpg)
FACULTY PERCEPTION OF JOURNAL COLLECTIONS
Following up…
![Page 39: LIBRARY ASSESSMENT](https://reader033.vdocuments.net/reader033/viewer/2022051609/547d4668b47959bb508b48d1/html5/thumbnails/39.jpg)
The Infamous IC-8
“The print and/or electronic journal collections I require for my work.”
![Page 40: LIBRARY ASSESSMENT](https://reader033.vdocuments.net/reader033/viewer/2022051609/547d4668b47959bb508b48d1/html5/thumbnails/40.jpg)
ARL Libraries 2009
4
5
6
7
8
9
10
AS - 1 AS - 2 AS - 3 AS - 4 AS - 5 AS - 6 AS - 7 AS - 8 AS - 9 IC-1 IC-2 IC-3 IC-4 IC-5 IC-6 IC-7 IC-8 LP - 1 LP - 2 LP- 3 LP - 4 LP - 5
Figure 3: LibQUAL+ 2009, ARL Libraries Faculty
![Page 41: LIBRARY ASSESSMENT](https://reader033.vdocuments.net/reader033/viewer/2022051609/547d4668b47959bb508b48d1/html5/thumbnails/41.jpg)
ARL Libraries 2009: IC8
![Page 42: LIBRARY ASSESSMENT](https://reader033.vdocuments.net/reader033/viewer/2022051609/547d4668b47959bb508b48d1/html5/thumbnails/42.jpg)
Have perceptions changed?
4
5
6
7
8
9
2006 2007 2008 2009
Figure 5: LibQUAL+ 2006-09, Faculty Ratings of Journal Collections, ARL Libraries
M=8.58, SD=0.10
M=7.44, SD=0.17
M=6.77, SD=0.40
M=8.61, SD=0.07
M=7.52, SD=0.15
M=7.01, SD=0.25
M=8.57, SD=0.13
M=7.51, SD=0.21
M=7.15, SD=0.39
M=8.56, SD=0.10
M=7.52, SD=0.14
M=6.96, SD=0.35
![Page 43: LIBRARY ASSESSMENT](https://reader033.vdocuments.net/reader033/viewer/2022051609/547d4668b47959bb508b48d1/html5/thumbnails/43.jpg)
Is there a difference in scores from year to year? (ANOVA)
• 2006-2009 adequacy gaps from each ARL institution.
• P-value = 0.119, which is not deemed statistically significant.
Faculty were no more or less dissatisfied with journal collections in 2009.
![Page 44: LIBRARY ASSESSMENT](https://reader033.vdocuments.net/reader033/viewer/2022051609/547d4668b47959bb508b48d1/html5/thumbnails/44.jpg)
Journals and Overall Satisfaction
5
6
7
8
9
-1.2 -1 -0.8 -0.6 -0.4 -0.2 0
Figure 8: LibQUAL+ 2009, Correlation of Faculty Satisfaction with Journal Collections (IC-8) and Overall Library Service, 21 Libraries
r =0.71
![Page 45: LIBRARY ASSESSMENT](https://reader033.vdocuments.net/reader033/viewer/2022051609/547d4668b47959bb508b48d1/html5/thumbnails/45.jpg)
IC Item 2006 2007 2008 2009 Standard
Deviation
(by question)
Mean
(by
question)
IC-1 0.44 0.61 0.44 0.55 0.08 0.51
IC-2 0.71 0.55 0.61 0.42 0.12 0.57
IC-3 0.86 0.42 0.31 0.73 0.26 0.58
IC-4 0.71 0.67 0.4 0.61 0.14 0.60
IC-5 0.72 0.49 0.27 0.45 0.19 0.48
IC-6 0.73 0.58 0.42 0.12 0.26 0.46
IC-7 0.81 0.46 0.40 0.67 0.19 0.59
IC-8 0.80 0.60 0.55 0.71 0.11 0.67
![Page 46: LIBRARY ASSESSMENT](https://reader033.vdocuments.net/reader033/viewer/2022051609/547d4668b47959bb508b48d1/html5/thumbnails/46.jpg)
What else should we be watching?
-0.7
-0.6
-0.5
-0.4
-0.3
-0.2
-0.1
0
0.1
0.2
IC1 IC2 IC3 IC4 IC5 IC6 IC7 IC8 2006 (n=37 )
2007 (n=19)
2008 (n=14)
2009 (n=21)
Figure 6: LibQUAL+ 2006-09, Information Control Adequacy Gaps Over Time
![Page 47: LIBRARY ASSESSMENT](https://reader033.vdocuments.net/reader033/viewer/2022051609/547d4668b47959bb508b48d1/html5/thumbnails/47.jpg)
What do our faculty say?
Support
“What would be great for faculty would be if when things are not available, there was one source in the library, extraordinarily skilled at tracking down items. […] These people would be specialists in working the electronic and journal capabilities.”
![Page 48: LIBRARY ASSESSMENT](https://reader033.vdocuments.net/reader033/viewer/2022051609/547d4668b47959bb508b48d1/html5/thumbnails/48.jpg)
What do our faculty say?
Search and Online Access
“I think just having free text search, like Google book search, would be something that would be very, very useful to have. I still feel like we are living 20 years behind where the rest of the world is in terms of being able to search these databases and large collections of books that we have.”
![Page 49: LIBRARY ASSESSMENT](https://reader033.vdocuments.net/reader033/viewer/2022051609/547d4668b47959bb508b48d1/html5/thumbnails/49.jpg)
What do our faculty say?
Work-Arounds
“I just buy them individually from my research funds, so it’s coming out of my research money.”
![Page 50: LIBRARY ASSESSMENT](https://reader033.vdocuments.net/reader033/viewer/2022051609/547d4668b47959bb508b48d1/html5/thumbnails/50.jpg)
What do our faculty say?
Quick List
“If I was to give a suggestion, maybe to have discipline-specific pointers that could help each discipline find things. […] It’s more of an interface issue than a collections issue.”
![Page 51: LIBRARY ASSESSMENT](https://reader033.vdocuments.net/reader033/viewer/2022051609/547d4668b47959bb508b48d1/html5/thumbnails/51.jpg)
What do our faculty say?
Resources
“The size of the collection is not as important as getting the current collection working as smooth as possible. Before, when we used to go to the library, we got service.”
![Page 52: LIBRARY ASSESSMENT](https://reader033.vdocuments.net/reader033/viewer/2022051609/547d4668b47959bb508b48d1/html5/thumbnails/52.jpg)
What do our faculty say?
Print vs. Electronic
“A few years ago, I wouldn’t have said that. But, I guess things have changed.”
![Page 53: LIBRARY ASSESSMENT](https://reader033.vdocuments.net/reader033/viewer/2022051609/547d4668b47959bb508b48d1/html5/thumbnails/53.jpg)
2CUL HUMANITIES PHD STUDENTS
![Page 54: LIBRARY ASSESSMENT](https://reader033.vdocuments.net/reader033/viewer/2022051609/547d4668b47959bb508b48d1/html5/thumbnails/54.jpg)
The 2CUL Project
“Columbia and Cornell University Libraries are pleased to join forces in a transformative and enduring partnership between our two great library systems that enables us to pool resources to provide content, expertise, and services that are impossible to accomplish acting alone.”
http://2cul.org/
![Page 55: LIBRARY ASSESSMENT](https://reader033.vdocuments.net/reader033/viewer/2022051609/547d4668b47959bb508b48d1/html5/thumbnails/55.jpg)
Research Procedures
Spring 2010 • Ethnographic Training • Focus groups (5 total) Summer + Fall 2010 • Interviews (45 total)
– 90 minutes, individual
• Post-questionnaire (paper) Winter 2010-11 • Analysis and reporting
$
![Page 56: LIBRARY ASSESSMENT](https://reader033.vdocuments.net/reader033/viewer/2022051609/547d4668b47959bb508b48d1/html5/thumbnails/56.jpg)
870 pages of transcripts.
![Page 57: LIBRARY ASSESSMENT](https://reader033.vdocuments.net/reader033/viewer/2022051609/547d4668b47959bb508b48d1/html5/thumbnails/57.jpg)
The Student
Personal Space
Previous Experience
Personal Expectations
Self-determination
Personal Life
The Institution + Department
Institutional Space
Funding
Dept. Requirements + Expectations
Culture + Community
Advising
Teaching
Attrition
Research + Writing
Coursework
Exams + Preparation
Prospectus + Preparation
Discovery
Writing Process + Revision
Defense + Preparation
Information Management
The Library
Librarians
Collections
Services
Library Space
Technology
The Profession
Job Search
Publishing
Challenges
Successes
Opportunities
First
Second
Third
Code Level
Fourth
Code Tree
![Page 58: LIBRARY ASSESSMENT](https://reader033.vdocuments.net/reader033/viewer/2022051609/547d4668b47959bb508b48d1/html5/thumbnails/58.jpg)
Time from BA graduation Through Expected PhD Completion
Graduation from undergrad to start of PhD Start of PhD to candidacy Candidacy to PhD expected graduation
![Page 59: LIBRARY ASSESSMENT](https://reader033.vdocuments.net/reader033/viewer/2022051609/547d4668b47959bb508b48d1/html5/thumbnails/59.jpg)
14
14
10
21
24
3
24
22
8
19
16
6
2
2
2
3
2
8
3
2
1
5 1
2
5
3
2
3
1
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Grad Program
Funding
Advising (Columbia only)
Library Services
Library Collections
Library Spaces (Columbia only)
Humanities PhD Student Study Questionaire Please rate your overall satisfaction with the following at CU:
Very Satisfied
Satisfied
Neutral
Dissatisfied
Very Dissatisfied
n/a
![Page 60: LIBRARY ASSESSMENT](https://reader033.vdocuments.net/reader033/viewer/2022051609/547d4668b47959bb508b48d1/html5/thumbnails/60.jpg)
0
1
2
3
4
5
6
7
8
9
10
1< 1 -2 hrs 2 - 4 hrs 4 - 6 hrs 6 - 8 hrs 8+
No
. of
Stu
de
nts
Time Spent in the Library by Frequency and Duration
Daily Weekly Monthly
![Page 61: LIBRARY ASSESSMENT](https://reader033.vdocuments.net/reader033/viewer/2022051609/547d4668b47959bb508b48d1/html5/thumbnails/61.jpg)
0% 20% 40% 60% 80% 100%
browse
write
read
office hours
research w. library …
research w. non-library …
computers
consult librarian
meet colleagues
other Post-Exam Pre-Exam
Activities in the Library by Status
Percentage of Students
![Page 62: LIBRARY ASSESSMENT](https://reader033.vdocuments.net/reader033/viewer/2022051609/547d4668b47959bb508b48d1/html5/thumbnails/62.jpg)
30
10
5
Humanities PhD Student Study Questionnaire Have you visited any non-CU libraries to use their collections
for dissertation research?
Yes No n/a
![Page 63: LIBRARY ASSESSMENT](https://reader033.vdocuments.net/reader033/viewer/2022051609/547d4668b47959bb508b48d1/html5/thumbnails/63.jpg)
39
2
4
Humanities PhD Student Study Questionnaire Do you receive financial support from CU for the academic
year?
Yes No n/a
![Page 64: LIBRARY ASSESSMENT](https://reader033.vdocuments.net/reader033/viewer/2022051609/547d4668b47959bb508b48d1/html5/thumbnails/64.jpg)
6
34
5
Humanities PhD Student Study Questionnaire Do you have an outside job that provides income?
Yes No n/a
![Page 65: LIBRARY ASSESSMENT](https://reader033.vdocuments.net/reader033/viewer/2022051609/547d4668b47959bb508b48d1/html5/thumbnails/65.jpg)
Findings: Provide Space
"The thing that has been the best for me is having a space to work. I got more done last year when I had my locked carrel than I had gotten done in years before or since, because it was a dedicated space in which I could keep all of my sources [...]."
![Page 66: LIBRARY ASSESSMENT](https://reader033.vdocuments.net/reader033/viewer/2022051609/547d4668b47959bb508b48d1/html5/thumbnails/66.jpg)
Findings: Foster Community
“It’s having community. Belonging to your community, having friends that are doing this and feeling that you have something worthwhile to say that other people are recognizing it.”
![Page 67: LIBRARY ASSESSMENT](https://reader033.vdocuments.net/reader033/viewer/2022051609/547d4668b47959bb508b48d1/html5/thumbnails/67.jpg)
Findings: Provide Access to Deep Research Collections
“I have to say that I have had every resource that I have needed from the library. I really can’t say, ‘Here I am in the sixth year because you didn’t buy that set of resources for me and I don’t have the materials to work with, so how can you expect me produce work?’”
![Page 68: LIBRARY ASSESSMENT](https://reader033.vdocuments.net/reader033/viewer/2022051609/547d4668b47959bb508b48d1/html5/thumbnails/68.jpg)
Findings: Provide Research, Information Management, and Teaching Expertise Assistance
“[…] maybe sitting down with an advanced research reference librarian . . . might be in my best interest as I go into the writing stage of my paper, just so that I can make sure I am not saying something that has already been said or duplicating research, or that I am not missing something that is cutting-edge and that's really important to my argument.”
![Page 69: LIBRARY ASSESSMENT](https://reader033.vdocuments.net/reader033/viewer/2022051609/547d4668b47959bb508b48d1/html5/thumbnails/69.jpg)
Findings: Developing Scholarly Identity
“I had to tell my committee in an email, I plan on having a draft of the first chapter to you by June. If I don't, please get on my case… So, I actually found that I needed to make deadlines for myself and then tell them so that they knew, and even though they wouldn't care, my pride was at stake at that point.”
![Page 70: LIBRARY ASSESSMENT](https://reader033.vdocuments.net/reader033/viewer/2022051609/547d4668b47959bb508b48d1/html5/thumbnails/70.jpg)
ITHAKA S+R
![Page 71: LIBRARY ASSESSMENT](https://reader033.vdocuments.net/reader033/viewer/2022051609/547d4668b47959bb508b48d1/html5/thumbnails/71.jpg)
ITHAKA is a not-for-profit organization that helps the academic community use digital technologies to preserve the scholarly record and to advance research and teaching in sustainable ways.
We pursue this mission by providing innovative services that aid in the adoption of these technologies and that create lasting impact. .
Mission
![Page 72: LIBRARY ASSESSMENT](https://reader033.vdocuments.net/reader033/viewer/2022051609/547d4668b47959bb508b48d1/html5/thumbnails/72.jpg)
• Ithaka S+R is a strategic consulting and research service that focuses on the transformation of scholarship and teaching in an online environment, with the goal of identifying the critical issues facing our community and acting as a catalyst for change.
• JSTOR is a research platform that enables discovery, access, and preservation of scholarly content.
• Portico is a digital preservation service for e-journals, e-books, and other scholarly e-content.
Our Services
![Page 73: LIBRARY ASSESSMENT](https://reader033.vdocuments.net/reader033/viewer/2022051609/547d4668b47959bb508b48d1/html5/thumbnails/73.jpg)
Ithaka S+R Surveys
2009 Faculty Survey
3,025 Faculty responses
8.6% of population
Focus on research institutions Conducted 2000, 2003, 2006,
2009, 2012
2010 Library Director Survey
239 Library Directors responded
13% of population
9 Carnegie Classification Levels:
– 79 responses are doctoral
– 66 are master’s
– 94 are baccalaureate
Ithaka S+R Staff: Roger Schonfeld, Ross Housewright, Matt Long
![Page 74: LIBRARY ASSESSMENT](https://reader033.vdocuments.net/reader033/viewer/2022051609/547d4668b47959bb508b48d1/html5/thumbnails/74.jpg)
0% 20% 40% 60% 80% 100%
Gateway
Archive
Buyer
Research Supporter
Undergraduate Information Literacy Teacher
Teaching Facilitator
How important to you is it that your college or university library provide each of the functions below? (Percentage answering very important.)
Faculty Members Library Directors
The Role of the Library: Comparison with Faculty
Note: Faculty member data are from Ithaka S+R Faculty Survey 2009.
![Page 75: LIBRARY ASSESSMENT](https://reader033.vdocuments.net/reader033/viewer/2022051609/547d4668b47959bb508b48d1/html5/thumbnails/75.jpg)
Digital vs. Print Spending
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Five years from Now
Now
Journals Books All other items
“What percentage of your library’s materials budget is spent on the following items?”
![Page 76: LIBRARY ASSESSMENT](https://reader033.vdocuments.net/reader033/viewer/2022051609/547d4668b47959bb508b48d1/html5/thumbnails/76.jpg)
Digital vs. Print Spending
Journals Books
Directors predict a
106% drop in spending on print journals in the
next 5 years…
…bringing budget shares to:
12% Print
88% Digital / 54%
46% Digital /
Directors predict a
46% drop in spending on print books in the
next 5 years.
…bringing budget shares to:
![Page 77: LIBRARY ASSESSMENT](https://reader033.vdocuments.net/reader033/viewer/2022051609/547d4668b47959bb508b48d1/html5/thumbnails/77.jpg)
Print to Electronic Transition: Publishing
“I am completely comfortable with journals (that I use regularly/ that my library subscribes to) ceasing their print versions and publishing in electronic-only form.”
Faculty Members Library Directors
Agree 70% Neither agree
nor disagree 22% Disagree 8%
39% Agree
30% Neither agree nor disagree
32% Disagree
![Page 78: LIBRARY ASSESSMENT](https://reader033.vdocuments.net/reader033/viewer/2022051609/547d4668b47959bb508b48d1/html5/thumbnails/78.jpg)
Print to Electronic Transition: Existing Collections
“Within the next five years, the use of (online or digitized journals/ electronic versions of scholarly monographs) will be so prevalent among faculty and students that it will not be necessary to maintain library collections of hard-copy…
… Books.” … Journals.”
Agree 54% Disagree 13%
7% Agree
59% Disagree
![Page 79: LIBRARY ASSESSMENT](https://reader033.vdocuments.net/reader033/viewer/2022051609/547d4668b47959bb508b48d1/html5/thumbnails/79.jpg)
Library Spending Priorities
If you received a 10% increase in your library's budget next year in addition to the funds you already expect to receive, in which of the following areas would you allocate the money? (Please check up to three areas in the following list that you would invest in.)
![Page 80: LIBRARY ASSESSMENT](https://reader033.vdocuments.net/reader033/viewer/2022051609/547d4668b47959bb508b48d1/html5/thumbnails/80.jpg)
0% 20% 40% 60% 80% 100%
Staff in management/administration of …
Electronic versions of scholarly …
Other digital resources
Facilities expansions and renovations
Staff for reference and user services/ …
Tools for discovery (OPACs, indices, …
Online or digital journals
Library Spending Priorities
If you received a 10% increase in your library's budget next year in addition to the funds you already expect to receive, in which of the following areas would you allocate the
money?
![Page 81: LIBRARY ASSESSMENT](https://reader033.vdocuments.net/reader033/viewer/2022051609/547d4668b47959bb508b48d1/html5/thumbnails/81.jpg)
0% 20% 40% 60% 80% 100%
Developing and maintaining special …
Supporting the research projects of faculty …
Building or maintaining local discovery resources
Purchasing/ licensing digital resources
Providing reference services
Supporting faculty instruction and student …
Percentage ranking this item as 1
Percentage ranking this item as 2
Library Staffing Priorities
Ideally, how would you prioritize your staff resources in the following areas? Please rank the items by order of importance.
![Page 82: LIBRARY ASSESSMENT](https://reader033.vdocuments.net/reader033/viewer/2022051609/547d4668b47959bb508b48d1/html5/thumbnails/82.jpg)
Discovery: A Declining Role for the Library?
0%
10%
20%
30%
40%
50%
60%
70%
80%
2006 "Now" "5 years from now"
Library Directors
Faculty Members
Percentage answering that it is very important that the library serve as a "gateway"
![Page 83: LIBRARY ASSESSMENT](https://reader033.vdocuments.net/reader033/viewer/2022051609/547d4668b47959bb508b48d1/html5/thumbnails/83.jpg)
User Needs Assessment
94% Informal discussions with faculty and students or emails soliciting feedback
71% Locally developed polls or surveys
49% Focus groups or test sessions
37% Cross-institutional polls or survey (such as Libqual+)
16% Structured Interviews
10% Ethnographic studies
8% With the help of outside consultants
6% Other
In the past 2 years, has your library regularly solicited feedback about services or collections from library users in any of the following ways? (Please check all that apply.)
Only 13% do not have a formal means to assess user needs.
![Page 84: LIBRARY ASSESSMENT](https://reader033.vdocuments.net/reader033/viewer/2022051609/547d4668b47959bb508b48d1/html5/thumbnails/84.jpg)
35% of respondents agreed that “My
library has a well-developed strategy to meet changing user needs and research habits.”
Library Strategies
![Page 85: LIBRARY ASSESSMENT](https://reader033.vdocuments.net/reader033/viewer/2022051609/547d4668b47959bb508b48d1/html5/thumbnails/85.jpg)
CHALLENGES
![Page 86: LIBRARY ASSESSMENT](https://reader033.vdocuments.net/reader033/viewer/2022051609/547d4668b47959bb508b48d1/html5/thumbnails/86.jpg)
challenges • Lack tradition of using data for improvement
• No assessment advocate within organization
• Library staff lack research methodology abilities
• Weak analysis and presentation of data
• Inability to identify actionable data
• Library “culture” is skeptical of data
• Leadership does not view as priority/provide resources
• Library organizational structure is “silo-based”
• Staff do not have sufficient time
Turning Results into Action: Using Assessment Information to Improve Library Performance, Steve Hiller (University of Washington) , Stephanie Wright (University of Washington)
![Page 87: LIBRARY ASSESSMENT](https://reader033.vdocuments.net/reader033/viewer/2022051609/547d4668b47959bb508b48d1/html5/thumbnails/87.jpg)
challenges
![Page 88: LIBRARY ASSESSMENT](https://reader033.vdocuments.net/reader033/viewer/2022051609/547d4668b47959bb508b48d1/html5/thumbnails/88.jpg)
jobs
![Page 89: LIBRARY ASSESSMENT](https://reader033.vdocuments.net/reader033/viewer/2022051609/547d4668b47959bb508b48d1/html5/thumbnails/89.jpg)
THANK YOU [email protected]
![Page 90: LIBRARY ASSESSMENT](https://reader033.vdocuments.net/reader033/viewer/2022051609/547d4668b47959bb508b48d1/html5/thumbnails/90.jpg)
References About Assessment
• “Value of Academic Libraries: A Comprehensive Review and Report,” Megan Oakleaf: http://www.ala.org/ala/mgrps/divs/acrl/issues/value/val_report.pdf
• Focus Groups: A Practical Guide for Applied Research by Richard Krueger and Mary Anne Casey
• ARL SPEC Kit #303 on Library Assessment, December 2007
• Keys to Effective, Sustainable, and Practical Assessment Steve Hiller, Martha Kyrillidou, and Jim Self http://www.arl.org/arldocs/stats/statsevents/laconf/2006/HillerSelf.ppt
• www.libraryassessment.info (blog)
• Columbia Assessment Program: https://www1.columbia.edu/sec/cu/libraries/bts/assessment/index.html
Interesting Studies
• Studying Students, The Undergraduate Research Project at the University of Rochester http://www.ala.org/ala/mgrps/divs/acrl/publications/digital/Foster-Gibbons_cmpd.pdf
• For fun: http://www.coolinfographics.com/
• Cornell University Libraries ROI: http://research.library.cornell.edu/value
• Ellsevier/UIUC ROI: http://www.slidefinder.net/t/library_strategic_investment/illinois_roi_study/1313459
• University of Arizona), Learning in an Online Environment: Assessment of an Online Information Literacy Credit
• Course, Yvonne Mery, Jill Newby, Ke Peng: http://libraryassessment.org/bm~doc/2010_lac_poster.pdf
• University of Chicago: Wayfinding Revisited: Improved Techniques for Solving Usability Problems in Physical Spaces Agnes Tatarka, David K. Larsen
• LibQual+ Survey Literature: www.libqual.org/Publications
• 2CUL Humanities PhD Study:
Conferences
• Library Assessment Conference Proceedings: http://libraryassessment.org
• Northumbria Conference Proceedings: http://www.lib.sun.ac.za/Northumbria7/Programme.htm
Ithaka S+R
• http://www.ithaka.org/ithaka-s-r/research/faculty-surveys-2000-2009
• http://www.ithaka.org/ithaka-s-r/research/ithaka-s-r-library-survey-2010