measuring engagement presentation for aci
DESCRIPTION
How to measure alumni engagementTRANSCRIPT
MEASURING ENGAGEME
NTReggie Bustinza
Director of Alumni RelationsLewis University
Romeoville, IL Private Catholic institution 6,500 students (4,500 undergrad, 2,000
grad) 37,000 alumni – primarily in Chicago area Alumni Relations staff of 3 Database: Raiser’s Edge (“R/E”)
About Lewis
Why try to measure engagement? What are some methods? The Lewis System
◦ Process◦ Details◦ Results
Now what?
Overview
Metrics can guide decision making◦ Spot trends◦ Identify opportunities◦ Abandon dead ends◦ Quantify program success
More efficiency & efficacy Justify our existence
Why Measure Engagement?
Hope College Bradley University Grand Valley State University Lewis University
Metric Systems
Used regression analysis to identify 5 representative variables that drove engagement.
Each item was given a weight (sum of 10) ◦ Phone number 2.970◦ Volunteerism 2.009◦ Valid email address 1.221◦ E-newsletter subscriber 1.208◦ Event attendance 0.681
Method Overview – Hope College
Courtesy Scott TravisDir. Of Alumni & Parent RelationsHope College
Method Overview – Hope CollegeStrengths Weaknesses
Statistically highly accurate
Formula they can run easily
Few factors make it easy to look for correlations
Scale of 1 to 10
Not intuitive Difficult to set up
(unless you’re a statistician and programmer)
Giving not considered Still room for errors of
human judgment Scale of 1 to 10
Stratified alumni engagement levels
◦ The most engaged activity a person is involved in defines which engagement level they are placed in
Method Overview – Bradley University
Non Interest – Universe of BU alumni
Interested – Passive Activity Engaged – Actions that
result in benefit TO alumnus
Active – Actions that provide benefit TO university
Advocate – Actions result in sphere of influence to benefit BU
Method Overview – Bradley University
Advocate
Active
Engaged
Interested
Non-Interest
Method Overview – Bradley UniversityStrengths Weaknesses
Very simple Gives nice overview Clearly defined
stratum
Very simple No number assigned Requires
understanding of the system
Limited applications Cannot be used in a
query
Cumulative point system
◦ Track things like giving, events, student activities, alumni groups, leadership positions, legacy, volunteerism, social media
◦ Points accumulate over alumnus’s lifetime
Method Overview – Grand Valley State
Courtesy Mindy Kalinowski EarlyAssistant Dir. Of Alumni RelationsGVSU
Method Overview – Grand Valley StateStrengths Weaknesses
Works with existing data
Relatively simple setup
Point inflation No accounting for
timing of involvement
Requirements◦ Work with existing data◦ Searchable◦ Valid as aggregate and/or individual data◦ Easy to understand
Not required, but nice to have…◦ Inexpensive to implement◦ Ability to run ourselves, as frequently as we want
OR dynamic◦ Option to exclude data to look for correlations◦ Simplicity
The Lewis System
Liked the way Hope College utilized their process but we wanted to consider more data points
Liked the ease of the Bradley system, but really wanted numerical output
Liked the breadth of GVSU’s categories, but didn’t like the point inflation
The Lewis System
Process1. Make sure database is capable of outputting
what we desire, and we can import results2. Identify what relevant data we track – “What
information do we have that shows some kind of engagement?”
3. Assign relative values4. Test5. Repeat until values are no longer questioned
The Lewis System
Step 1: Can Database Handle It?
Step 2: What do we Track?
Is it indicative of engagement? Is it accurate? Will we keep tracking it?
Step 2: What do we Track?
• Event Attendance• Giving – how much
and how often• Valid email• Open emails• Social media• Valid address• Valid business info• Active volunteer• Board member
• Award winner• Legacy parent• Campus visits• Interested volunteer
Can we categorize?
Step 2: What do we Track?
EventsEvent Attendance
GivingGiving – how much and how often
CommunicationsValid emailOpen emailsSocial mediaValid addressValid business info
VolunteerismActive volunteerBoard memberInterested volunteer
OtherAward winnerLegacy parentCampus visitsEmployeeAffinity Partners
Step 3: Assign Relative Values
Some categories require point decay◦ Giving consecutive years (most recent 3 years)◦ Event attendance over years
Some attributes require point cutoffs◦ Email open rates◦ Giving consecutive years (beyond 3 years)
Some attributes require point differentiation◦ Where gifts come from (event vs. outright)◦ Volunteer type
Step 3: Assign Relative Values
Challenges◦ Not all board members are equally engaged.◦ How stratified should we make giving levels?◦ Free events vs. Paid events
Step 3: Assign Relative Values
Category: Events
Step 3: Assign Relative Values
Indicator Value
FY – 13 4 / event
FY – 12 2 / event
FY – 11 1 / event
Category: Giving
Step 3: Assign Relative Values
Indicator Value
Gift – FY 13 Event – 5Outright – 10
Gift – FY 12 Event – 2Outright – 5
Gift – FY 11 Event – 1Outright – 2
Planned Gift 7
Presidents Circle – FY 13 5
Presidents Circle – FY 12 2
Consecutive Years 2 pts / year (max 10 years)
Category: Communications
Step 3: Assign Relative Values
Indicator Value
Receive Emails 2
Open Emails 1-3 = 2 ---- 4+ = 7
Click Emails 1-3 = 2 ---- 4+ = 7
Business Info 2
Valid Address 2
Valid Email 2
Valid Phone 2
Net Community 2
Social Media 2
Category: Volunteer
Step 3: Assign Relative Values
Indicator Value
Active 5
Interest 5
Affinity Group 3
Affinity Group Board 5
LUAA Board 5
BoT 10
Committee Chair 2
Class Speaker 5
Organization (mock trial, etc)
7
Other 3
Category: Other
Step 3: Assign Relative Values
Indicator Value
Initial Visit Acceptance (dvlp)
2
Voluntary Campus Visit 2
Legacy Parent 5
Award Winner 2
Employee 5
Liberty Mutual 2
6 scores are actually produced◦ 1 for each category◦ Overall Engagement Score (sum of each category)
Step 3: Assign Relative Values
Step 4: Test
Share values with colleagues for feedback Run the numbers, see what results are Spot check results Pull top 10, top 20, top 50, top 100 alumni
◦ Does it add up?◦ If not, why?
Step 4: Test
Tweak values, repeat test
Step 5: Repeat!
Step 5: Repeat!
Run Quarterly (past 12 months)
Exported each category to Excel where values are assigned and coded
SPSS is used to merge data Import integers back into Raiser’s Edge
Final Process
Results
Example 1: Tyler Durden – top 5%
Results - Individual
Item Points
Given 2 consecutive years
4 points (2 per year)
Given 2 most recent years
15 points (10 this year, 5 last)
Attended 1 event 3 years ago
1 point
Valid address & phone # 4 points (2 for each)
TOTAL 24 points
Example 2: Art Vandelay – top 25%
Results - Individual
Item Points
Given once, 7 years ago 0 points
No events, volunteerism 0 points
Read 6 emails, 1 click-through
9 points (7 for more than 4 emails opened, 2 for 1 clickthrough)
Valid email, address, phone #
6 points (2 for each)
TOTAL 15 points
Category Giving
Events
Comm.
Vol. Other TOTAL
All Alumni 1.3 0.152 8.859 0.042 0.227 10.584
Young Alumni 1.087 0.189 9.48 0.038 0.103 10.898
Athletes 2.78 0.48 9.69 0.06 0.35 13.29
Alumni Board 21.2 19.52 15.2 6.64 2.24 64.8
Board of Trustees
35.5 13.82 12.73 10.4 2.77 75.22
Aviation Alumni 0.794 0.065 8.78 0.0299 0.195 9.865
Results - Aggregate
Results - Aggregate
Top X % Point Cutoff
1% 45
5% 23
10% 17
25% 12
50% 8
Results - Aggregate
Lewis System: Strengths & WeaknessesStrengths Weaknesses
Can run in house Values recent activity
over old activity Results are easy to
understand
Some data can be suspect (eg: acquired mailing lists)
Have to export, use two programs, then import for scores
As data points are added, historical scores are distorted
Now What?
Metrics are half of the battle. The real question is: How will you use this tool?◦ Whittle mailing lists◦ Identify prospects that were under-the-radar◦ Look for correlations between giving and
programming◦ Identify potential affinity groups◦ Snapshot of prospect’s history◦ Evaluate programming
Now What?
Questions