data, evidence and outcomes
DESCRIPTION
Presentation at the American Library Association Conference on June 28, 2014. Sponsored by counting OpinionsTRANSCRIPT
Data, Evidence and Outcomes
Joe Matthews
American Library Association
Sponsored by Counting Opinions
June 28, 2014
Need for Data
Making management decisions
Increase program & service effectiveness
Justifying budgets
Evaluating performance
Doing advocacy
Types of Data
Quantitative
Counting Surveys
evaluation of services Customer satisfaction
Qualitative
Purpose
The purpose of any data gathering activity
& analysis is to INFORM
Strategies
User versus library’s perspective
Examining trends
Comparing libraries
Social context
Outcomes
User’s Perspective
How well?
How courteous?
How responsive?
How satisfied?
How courteous?
Welcoming
Attentive
How responsive?
Anticipatory
Helpful
Empathetic
How satisfied?
Expectations met Materials obtained Personal interaction Ease of use Equipment used Environment Comfort Willingness to return
Library’s Perspective
How much?
How many?
How prompt?
How economical?
How much?
Magnitude
Percent of change from last year
Percent of overall change
Cost
How many?
Magnitude
Change
How prompt?
Cycle times
Turnaround time
Wait times
Anticipatory
How economical?
Resources used
Units processed
Productivity
Library & Customers Perspective
How valuable?
How reliable?
How accurate?
How well?
How valuable?
Effort expended
Costs
Benefits obtained
How reliable?
Dependability/Consistency
Access
Accuracy
How accurate?
Completeness
Comprehensiveness
Currency
How well?
Accuracy
Promptness
Courtesy
Expertise
Outcomes
Change in a person’s life
Affective – attitudes, confidence, satisfaction
Behavioral
Knowledge-based
Competency-based – apply new skills
Tracking Trends
Year-to-year trends
Comparing trends
Indexing dollars for inflation
Indexing budgets for cost of living
Projecting future needs
Comparing Libraries
Identifying peers
Individual vs. grouped data
Rankings
Comparing Libraries Examples
Comparing Peers – criteria ?
Rankings
Percentages/grouped data
Summary stats
Benchmarking/standards
Libraries in Social Context
Social, political, & economic environment
Focusing on decision-makers’ concerns
Finding the right data
Creating the “hook”
Social Context Examples
PLs rank 2nd behind fire protection among local government services
Visits to PLs outnumber professional football attendance of X annually by a factor of Y
PLs loan 4.5 million items per day – FedEx delivers about 1.75 million packages each day
Evidence How can evidence contribute to the quality
of the decision making process?
What are the risks, costs, and benefits of an evidence-based approach?
Ensure that objective evidence is reflected in the decision making process.
Evidence
Depends on the need
BIG DATA
Combine data
Source of Data
IMLS data
ARL data
PLAmetrics
ACRLMetrics
Edge Initiative
Impact Survey – UW
Evidence-based Library & Information Practice
Collection Insights
collectionHQ – selection, management, promotion
Reports from your ILS
Patron Perspective
Segmentation analytics
Demographics
Lifestyles
Geography
Benefit segmentation
Program Priorities
Track program measures
Provide programs tied to strategic goals –
aligned with community needs
Early childhood education School age education support Economic development Community engagement . . .
Advocacy Hooks
Use data for presentations
Use data to target likely voters
Use data to demonstrate value in the life of your customers
Outcomes
Libraries cannot demonstrate value
until they define outcomes
of institutional relevance
and then measure the degree
to which they attain them
Outcomes
Focus on the end results
Logic Model
Orr’s Input-Process-Output-Outcomes Model
Logic Model
use the if then exercise
If the library provides assistance to students with their
homework then the students will do better
with their schoolwork.
If they do better with their schoolwork
then they will get better grades and attend more regularly.
If they get better grades and attend more regularly
then they are more likely to graduate.
If they graduate then they are more
likely to become employed and have a
higher standard of living.
Orr’s Model
Input Process Output Outcome Impact
Gates Common Impact Measurement System
Digital inclusion
Culture & Leisure
Education
Communication
Health
Economic Development
Academic Library Outcomes
Student Learning
Teaching Effectiveness
Research
Public Library Outcomes PLA Performance Measurement Task Force
Focus is on programs and their outcomes
Summer Reading Programs
Early Childhood Education Programs
Civic Engagement
Digital Access & Learning
Economic Development
Communication
Know your customer needs
What’s going to resonate
Data + stories = success
Focus on value – not how busy you are
Data, Evidence and Outcomes
Joe AT JoeMatthews.Org