project outcome overview · project outcome overview welcome! 1 ... june 2015: project outcome ......
TRANSCRIPT
3/21/2016
1
Project Outcome Overview
Welcome!
1
Emily Plagman
Project Manager
Public Library Association
2
Agenda
Project Outcome: Overview
Lessons from the Field
What’s Next?
Q&A
3/21/2016
2
Project Outcome:
Overview
Project Outcome Background
4
Managed by the Public Library Association
Project goal: Help public libraries understand
and share the true impact of essential library
services and programs.
Measures patron’s change in knowledge,
behavior, skills, attitude, & awareness
Does NOT measure outputs
Performance Measurement Task Force
5
July 2013: Task Force (PMTF) initiated
Libraries piloted and tested surveys late 2014
Survey design
o Simple & short
o Easy to administer
Task Force members continue to develop new,
advanced measures
3/21/2016
3
Project Outcome Timeline
6
June 2015: Project Outcome launched
Funded by the Bill & Melinda Gates Foundation
Builds resources & support around PMTF work
Summer 2016: PMTF to release advanced
measures
Post-Grant (2018): PLA management of Project
Outcome continues
Project Outcome Results
7
Over 1200 Project Outcome Users
o 1100 Public Library
o 70 State
o 70+ Other
380 libraries using survey portal
360 surveys scheduled
8000+ patron surveys collected
Project Outcome Testimonial
8
Circulation is not everything. We are getting questions
as to why numbers are going down.
Project Outcome results changed the conversation
from numbers being down to:
What are we accomplishing by being open?
What is happening in the lives of our users?
3/21/2016
4
Project Outcome Website
Registration is free & includes:
Simple surveys across 7 topics
Visually interactive data dashboards
Ready-made reports
Easy-to-use survey portal
Resources: beginning to end
Project Outcome Website 10 www.projectoutcome.org
11 Project Outcome Website
3/21/2016
5
7 Survey Areas
12
Survey Questions
13
Multiple choice questions measure:
Learned something new
Increased confidence
Anticipated change in behavior
Increased awareness
Survey Questions
14
Open-ended questions measure:
General patron feedback
Suggestions for improvement
3/21/2016
6
Survey Example: Early Childhood Literacy
15
Communicate Results & Strategic Planning
16
• Communicate about survey results with community,
library board, and stakeholders
• Use survey results to support:
Advocacy
Fundraising
Developing strategic plans
Measuring against strategic goals
Programming Changes
17
Examples of how other Project Outcome libraries have
used outcome results for programming needs:
To create additional programs
To make changes to a program
To eliminate a program
To create a partnership action plan
To write a grant proposal
3/21/2016
7
Project Outcome:
Website Demo
19 Project Outcome Website
20 Survey Portal
3/21/2016
8
21 Survey Portal
22 Survey Reports & Raw Data
23 Project Outcome Website
3/21/2016
9
24
Overview
Data Dashboard
25
Matrix
Data Dashboard
26
Detail
Data Dashboard
3/21/2016
10
27 Data Dashboard
Library Info
28 Project Outcome Website
Examples of Resources
#HELP
Choosing the Right Survey(s)
How to Talk to Patrons about Surveys
Survey Collection Best Practices
How to Use the Survey Portal
How to Use the Data Dashboard
#PLANNING
Good Practices for Communicating Data
Advocacy Resources for All Audiences
3/21/2016
11
Outcome Measurement Planning Process
Step 1: Set Goals
Step 2: Measure Outcomes
Step 3: Review Results
Step 4: Take Action
Results
31
1712
3270
1714 544
248
285
297
From the Field:
Plano (TX) Library System
3/21/2016
12
Plano Public Library System
Brent Bloechle
Library Manager
Population of Service Area: 269,776
Annual Operating Expenditures: 10,248,196
Number of Libraries: 5
FTE Staff: 150
Surveys Conducted (2015-16)
Education/Lifelong Learning (2015)
• Total Surveys: 140
Early Childhood Literacy (2015)
• Total Surveys: 28
Early Childhood Literacy (2016)
• Total Surveys: 31
Education/Lifelong Learning Survey
MS Word Common Tasks
Word Intro
Beginning Excel
Internet Basics
Excel – Intermediate/Advanced
Word Tares, Spanish
(Common Tasks)
3D Printing Intro to Modeling
3D Printing 123D Design
3D Printing: Preparing
Models Using Blender
3D Printing: Tinkercad
S.T.E.A.M. Saturday
Arduino Basics Workshop
3/21/2016
13
Survey Results: 85 Surveys
Survey Results: Feedback
“Offer 1-2 week classes extended”
“I’d like to go further in learning more”
“New/more software classes”
“I’d love an advanced PP class or graphics class or beginning HTML”
“Have more volunteers to assist so the instructor can keep teaching”
“Keep offering basic & advanced courses for technology like this”
“More science, math, & robotics classes”
What Did We Learn?
Realized we needed to have internal staff
conversations: “Are we offering what we need to?”
Had higher number of patrons participating in high-
level learning programs
Different patron needs than 5 years ago & need to
accommodate those changing needs
3/21/2016
14
What Can We Change?
Add more advanced-level technology & computer programs
Quick Fix: Increase number of volunteers & class size
Advanced Fix: Add more high-level computer and
technology programs or extend beginner/intermediate
classes for more advanced learning
Early Childhood Literacy Survey
Program: App Time!
Measured across 2 branches
More popular vs.
less popular
Results
28 paper surveys
What Did We Learn?
Less popular branch
Lacked traditional
Storytime components
The program was
requested in additional
languages
More popular branch
The parents liked that
there was more than just
working with the iPad
minis
Common comment from parents at both programs:
Parents felt the learning was enhanced by relating
books to technology
3/21/2016
15
What Can We Change?
Increase the budget for the
program to purchase apps
Vary the components of the
program
Insure that there is enough
equipment
Offer the program in
additional languages
Early Childhood Literacy Survey
Program/Service: Science Kits
Circulating Science Kits – 30 kits
Measured at one location
Results
31 paper surveys
What Did We Learn?
“The accessibility of various equipment to the children to learn hands on.”
“My son loves the STEAM kits. He enjoys the tangible materials in the kits that
allow him to visually understand the topic. I love the fact that we both are
learning new things together!”
“Good material that is easy to use and understand for kids of all ages.”
“Great collection of books and items
that explains the topic nicely.”
“You can touch and look at the models”
“ More kits in the future”
3/21/2016
16
What Can We Change?
“Provide kids programs, so kids like to come more often to [the]
library.”
“Maybe conduct some workshop[s] related
to the science kit topics to show
the kids how things work.”
“More STEAM programs & kits availability.”
“Keep mixing books and interactive items”
From the Field:
Farmers Branch Manske
Library
Farmers Branch Manske Library
Population of Service Area: 29,368
Annual Operating Expenditures: 1,618,469
Number of Libraries: 1
FTE Staff: 16.5
Denise Wallace
Library Director
3/21/2016
17
Surveys Conducted (2015)
Civic/Community Engagement
• Total Surveys: 113
Education/Life Long Learning
• Total Surveys: 24
Summer Reading
• Total Surveys: 27
• Digital Inclusion
• Underway
Civic/Community Engagement
Used Survey for a Variety of Programs
• Electric Guitar Ensemble
• Cool After School
• Patron Appreciation Day
• Preschool Story Time
• Santa & Cookies
Survey Results: 113 Surveys (FB)
3/21/2016
18
Sample Survey, Custom Questions
Santa and Cookies
Results: Feedback
“Keep up all the good work at the library” (“Best library in Texas!”)
“7 day access to the library”
“What the library is doing right now is worthwhile and up-to-date”
“This is a wonderful library and it helps the community every day”
“Seeing the varied talents of the local high school students artwork”
“Thank you for the kind librarians and their friendly service”
“Send out more emails and information… advertise more”
“Spent time with my son”
Results
Patron responses (agreed or strongly agreed):
• 57% were more aware of issues in their community
• 78% felt more confident about becoming involved in
their community
• 77% intended to become more engaged in their
community
• 85% were more aware of applicable
resources and services
3/21/2016
19
Survey Results: 24 Surveys (FB)
Survey Results: 11 Surveys (FB)
Overall Results
3/21/2016
20
Overall Results
Lessons Learned
Survey Administration:
• Paper surveys produce highest response rate
• Most patrons glad to help promote the library
Programming:
• In-house staff vs. outside services
• Results influence staff positioning & job responsibilities
• Allows us more time to interact with patrons as they fill out surveys;
communication is more “intimate” with staff
Civic/Community Engagement
Currently Collecting 2016 Data for the following…
• Downton Abbey Trivia and Costume Contest
• Games, Games, Games
• Cathy’s Critters
• Celebrate Texas History Month
• Coloring for Adults
• Spelling Bee for Adults
• Teen Volunteer Orientation
March 7 – May 31: Results not available until end of survey period.)
3/21/2016
21
Opportunities for Change
• Certain programs are well loved, traditional components that
continue to actively engage the public.
• More staff training – Night staff (P/T) not as aware of day-
to-day programming as Day Staff.
• Recognize the need to offer more
job-skill related programs and
library resources available.
What’s Next?
2016 Planning
62
Task Force creates & pilot tests
advanced measures: Underway
Launch advanced measures: ALA
Annual, June
State training: Fall
Free monthly webinars (ongoing)
Conference presentations
Peer share opportunities
(ongoing)
3/21/2016
22
Upcoming Events
TechSoup Webinar
(www.techsoupforlibraries.org)
Topic: Project Outcome for Rural & Small
Libraries
Wednesday, May 4, 11:00 a.m. CST
Questions?
www.projectoutcome.org
twitter.com/ProjectOutcome
www.facebook.com/groups/projectoutcome
Stay Connected!