creating a comprehensive early alert program...–kpi • success of program & implementation...
TRANSCRIPT
Creating a Comprehensive
Early Alert Program:
Design, Implementation and
Assessment
Loralyn Taylor, Ph.D.
Kerri Koteskey, MIS
Outline
• Design a comprehensive program
– Common components & problems
– Reactive strategies
• Identifying at-risk students
– Proactive strategies
• Identifying high risk students
• Implementation
– Change Management
• Assessment
– KPI
• Success of program & implementation
“Retention is everyone’s business.”
First Time Freshmen Fall-to-Fall Retention
All Fresh: 60-65%
Bachelors: 64-65%
Associates: 50-60%
Retention Office
Accommodative Services
TRiO Advisor
ARP Advocate
Faculty Advisor
HEOP Advisor
Program Coordinators
Instructors
Academic Support
Student Life
Deans
Student
Student Development
Health Services
Retention Office
Accommodative Services TRiO Advisor
ARP Advocate
Faculty Advisor
HEOP Advisor
Program Coordinators
Instructors
Academic Support
Student Life
Deans
Student
Student Development
Health Services
Success/Early Alert Program
1. Data Collection
2. Analyze, Identify, Prioritize
3. Outreach and Intervention
4. Assessment & Evaluation
Why Retention Programs Fail
1. No faculty buy-in
2. The “risk group of the year” syndrome
3. Information & communication problems
– silos
– coordination
– timeliness
Information flow and communication
problems can sink your retention efforts.
In Search of a Solution
• Technology• Automate--Gather, manage and distribute information
• Appropriate offices
• Coordinate response
• Easy to use
Info/Comm hub
Accommodative Services
TRiO Advisor
ARP Advocate
Faculty Advisor
HEOP Advisor
Program Coordinators
Instructors
Academic Support
Student Life
Deans
Student
Student Development
Health Services
Planning for Success
Planning for Success
Maximize info but minimize data & effort
• Intentionality
– Paralysis by Analysis
– Make a plan
• Simplicity
– Less is more
– Make a simple plan
Planning for Success
• Clarity
– Decrease training, increase compliance
– Make a clear, simple plan
• KISS
– Make a clear, simple plan that can be
understood and followed
Planning for Success
• Information and communications plan
1. Permissions
• Who sees what, when
2. Communications
• Who gets what, when
3. Action hierarchy
• Who does what, when
Planning for Success
• Automate prioritization
• Set appropriate thresholds
• Staffing
• Multiple intervention strategies
• Connections
• Modes
• Times
Planning for Success
Implementation plan– John Kotter’s Change Management Theory
• 8 Stages for Creating Major Change
• Marketing campaign
• Words
• Behaviors
• Actions
Information Management Plan
Three types of Flags
– Informational
• Notify about concern
– Action
• Student in need of intervention
– Urgent
• Critical action needed
• Student in danger of suspension
High Risk vs. At-Risk
Proactive & Reactive Strategies
• Proactive
– Potential High Risk
– Pre-semester
– Pre-enrollment
• Reactive
– Actually At-risk
– During semester
Proactive Strategies
High Risk Students
Prioritize Early Outreach
– Early Intervention
– Refer to services
• No FERPA—can communicate with parents
• Behaviors and habits not set
– Efficient use of resources
• Intervention during slower times
• Preventative, not remediation
Proactive Data
You have it, can you access it?
Matching data to problems
Focus on:
– Attrition
– Causes
– Difficult
– End First Semester GPA
– Credits earned
– Good standing
Finding & Inventorying Proactive Data
• What do you collect?• Admission
• Business processes
• Orientation
• Other tracked student contacts
• Separating the wheat from the chaff• Use what works
• Time consuming, IR
• Start small
• Educated guess
Proactive Strategies—Pre-Enrollment
• High Risk identification
– Demographics
– Underperforming groups
– Pell
– First Gen
– Survey data
– BCSSE
– CSI
– CIRP—Freshman Survey
Proactive Strategies—Pre-Enrollment
• High-Risk Identification
– Majors
– Health Sciences, Engineering, etc.
– Coursework
– Pre-college courses
– Credits Carried
– High Challenge Courses
– High DFW rates
– Lab Sciences
• High Risk identification
– Financial Aid
– Loan size
– Gap
– Parent PLUS loans
– Predictive Modeling
– New student surveys
– BCSSE
– CSI
– etc.
Proactive Strategies—Pre-Enrollment
Proactive Strategies—Returning
• High Risk identification
– Coursework
– Credits carried
– Prior academic performance
– Standing
– GPA drops
– Transcript requests prior to semester
– Transcript reviews
– Loan denials
• High Challenge Courses
– Targeting support
• Early Alert Survey Data
• Grades
– Midterm/final distributions
• DFW rates
– Math
– English
– Critical FY major courses
Proactive Strategies
Reactive Strategies
Reactive Strategies
• At-Risk Identification
– Behaviors/Engagement
– Academic
– Attendance, grades, homework
– Social
– Clubs, sports, gaming
– Disciplinary
– Violations, residence hall behavior
– Wellness
– Health, drugs & alcohol
Finding & Inventorying existing data
• Early Alert Surveys
• MidTerm Grades
• Attendance
• Systems & Business processes data
• Automation is KEY
• Efficient & effective
Data Sources
• Instructors
• LMS automated alerts
• Log in/ Grades/ assignments/ discussions/ Clicker
• Admissions
• Student Surveys
• Appointments
• Tutoring/advising/other services
• Residence Life/Student Life
• Business processes
Matching data to problems
Focus on:
– YOUR key problem(s)
– Paralysis by Analysis
– Risk group of the month trap
– success NOT retention
– Holistic
– Building support
Common Reactive EA Problems
1. Too little data• Can’t prioritize or refer
2. Too few participating• Can’t catch problems
3. Too late• Can’t recover
4. Too much data
Reactive Strategies Tips
• Goal: Timely identification & outreach• 3 Rs: Right student, right time, right intervention
• Get biggest bang for buck• Efficiency & effectiveness
• Clear, simple, intuitive
• Automate EVERYTHING!• Create information & communications hub
• Communicate, communicate, communicate
Reactive Strategies Tips
• Avoid Flag overload• Data without effort
• Collect what you can use, no more
• Manage expectations
• Clearance rates
• Set appropriate intervention thresholds
• Staffing
• Multiple intervention strategies
• automate
Reactive Strategies Tips
• Intervention strategies• Rapid
• Real time
• Automated
• Multiple modes
• Multiple people
• Don’t waste data or effort!
Reactive Strategies
• Faculty resistance• Can be AVOIDED
• Communication
• Goals & language
• Can be overcome
• Time
• Ease of use
• Motivation
• Success
Doing without Data
• The plural of anecdote is not data
but educated guess.
Doing without Data– Know your college & students
– Best Practices
– Institution type
– Student body
Planning for Success
Interventions
• Threshold
• Casting too wide a net
• Minimize false positives
• Type & Level
• Automated email
• Personal outreach
• Parents?
Assessment
Measure what matters,
when it matters.
Key Performance Indicators
Reasons:
• To demonstrate success
• For continuous improvement
• To demonstrate ROI
• To show need for resources
• Different measures for different stakeholders
Short Term Wins• Planned
• Meaningful
• Timely
• Multiple measures
No ROI = No more re$ources
Evaluating your Implementation
• New Behaviors• Log ins
• Account creations/profile creations
• Flags raised
• Office hours created
• Appointments made
• Referrals made
• Kudos raised
• Clearance rates/ completed referrals
Rewarding New Behaviors
• Recognition• Small is ok
• Doctors & Starbucks
• Better to come from on high
• President is best
• Visibility
Evaluating your Interventions
• Outreach• Clearance rates
• Reaching students?
• Timely
• methods
• Interventions• Support Services, Success Counseling,
Accommodative Services, Tutoring
• Dosage effect?
Early Alert Survey Results
2010 2011 2012
Fall Fall Fall
Sections Reporting 98% 84% 99%
Completion Time 1.4 1.7 2.47
# Flags Raised 789 810 941
# Students Flagged 380 387 489
% Students Flagged 38% 36% 46%
Student Success KPI
• Early
– EA Survey results
– Flag rates
– Flag clearance times
– MidTerm and Final Grades
• % with at least one D or F grade
• % D and F grades awarded
– % suspended after probation
Student Success KPI
• Mid
– Year-to-year retention rates
– % with overall gpa >2.00
– % on probation
– % suspended
– % on Dean’s List
Late KPI:Multi-year Retention Rates
Graduation Rates
50.0%
60.0%
70.0%
80.0%
90.0%
100.0%
prior 3 yrave
2010 2011 2012
First Time Students in Good Academic Standing
2.40
2.50
2.60
2.70
2.80
2.90
3.00
3 year priorave
2010 2011 2012
GPA
Term & Overall GPA
Overall
Term
36% Probations
41% Suspensions
Dean’s List: +17%
Adirondack Scholar: + 44%
High and Low Risk Students identified by Predictive Model
Low Risk High Risk % Change
Average GPA 2.87 1.78 -38%
Probation 10% 38% 280%
Withdraws 3% 8% 167%
% in Good Standing 84% 45% -46%
The Intersection of High Risk
CSI Risk Factors
Predictive Model Low Risk High Risk
Low Risk
High Risk
% Achieving Good Academic Standing
(GPA≥2.00)
CSI Risk Factors
Predictive Model Low Risk High Risk
Low Risk 87% 56%
High Risk 55% 34%
Earned Associates in 2 years
+34%
Bachelor’s
in 4 years:
+19%
Retention = Revenue
Increase in Net Student Revenue due to Increased Overall Retention Rate
Increased retention
Total Additional Students
Net Student Tuition
Spring 2010 to Fall 4.40% 36 360,000
Fall 2010 to Spring 0.80% 39 390,000
Spring 2011 to Fall 3.60% 59 590,000
Fall 2011 to Spring 2.00% 70 700,000
Spring 2012 to Fall 5.90% 102 1,020,000
Fall 2012 to Spring 1.20% 99 990,000
Spring 2013 to Fall 4.80% 107 1,070,000
Grand Total $5,120,000
Planning for Success−Maximize info, minimize effort
−Information & Communications
−Clear, simple, automate
−Implementation
−Change Management
Questions?
Contact us!
Loralyn Taylor
@LoralynTaylor
Kerri Koteskey
@MTOutdoor