#pearsoncite

Admission data: What more can it tell us?

How to use data to increase application-to-enrollment conversion rates

Lindsay Baun & Deb Corso-Larson, Pearson Michele Long, University of Denver (Contributor)

The potential in the use of data to inform targeted improvements to retain students throughout the student lifecycle is great. But there can be a lot of confusion about where to begin:

What interventions or teaching and

learning practices work best to retain

students? We have reporting, but

how do we turn that into

action?

What variables have the

largest impact on retention?

How can we assess the

effectiveness of our current

student support practices?

Do we have enough data? Is it the right

data?

How can we effectively

leverage data we already

have?

Institutions are increasingly focused on addressing a variety of retention challenges.

Data

Reporting & Accountability

Melt -- referring to the % of students who are admitted but do not enroll at a given college -- is a vexing problem for many institutions.

The long standing belief at University College of the University of Denver: If you admit them they will come

The landscape of higher education has changed and post-traditional students have options!

University College believed their melt rate had grown to as high as 40%.

University College was struggling to reduce a persistent melt rate.

They knew they had lots of data that could

help them find targeted ways to improve their

melt rate… …but they had limited

internal recourses for data collection, integration, and

analysis.

Pearson Consulting Services and University College partnered to take a data informed approach to addressing their melt challenge.

Pearson Consulting Services North America Higher Ed Services

Lindsay Baun, M.S. Data Analyst

• Biz intelligence & statistical modeling

• Former Data Analyst for Target Corp

Project Consultants:

Deb Corso-Larson, M.A. Academic Consultant

• Research & Evaluation Methods • Data informed approaches to

drive improvements to curriculum & instruction

Project Overview / Goals Learning Analytics Process

Data Model Findings and Insights

Actions Discussion

Michele Long Director of Student Services • 19 years in Continuing Ed • Past Chair UPCEA

Marketing, Enrollment and Student Services Network (MESS)

DU Project Lead: ● Credit Programs:

87% graduate 6% bachelor completion 7% non-degree

● Fall 2014: 71 Lecture courses

and 243 Online courses ● Enrollments: 77% online; 23%

on campus ● Students: 55% in-state; 40%

out-of-state; 5% international

University College of the University of Denver College of Professional and Continuing Education

Desired Project Outcomes What is the best way to reduce melt?

• Understand which admission characteristics matter Establish a benchmark of graduate melt rate and better

understand enrollment predictors

• Pinpoint where to put efforts Strategically utilize limited human resources - admission, student services, academic advisors to impact ‘at-risk’ new admits

• Identify challenges and actions that will positively impact

enrollment

• Be Better! Improve the quality of the experience from inquiry to enrollment

Learning Analytics Process

10

Discovery Analytical Modeling

Strategic Planning

• Articulated the project objectives

• Engaged the right stakeholders

• Defined measurement & success metrics

• Gained permission to share the data

• Conducted a data inventory

• What data do we need?

• Who owns the data? • Integrated and cleaned

multiple data sets • Utilized Decision Tree

Modeling to identify the variables most predictive of melt

• Provided a rigorously grounded analysis of the most important melt drivers

• Implemented an operationalized predictive model to identify at-risk students

• Developed a plan of action to identify at-risk students and start addressing risk factors to reduce melt.

Pearson and University College partnered to engage the right stakeholders on campus, develop a custom analytical/statistical approach, and provide actionable recommendations toward the goal of reducing enrollment melt.

Data Summary

• 55 variables analyzed • Application start, complete, last application requirement submitted, and

admission timing • Financial aid – applied, offered, declined, and received (including dates) • Previous education, distance to DU campus, and demographics • Degree, major, concentration, intended delivery modality, and prior credits

• Data Sources • Banner, Online Application, Financial Aid, Resumes, National Student

Clearinghouse, and Others

*Note: Summer term was excluded from the analysis

Term Student Count

Fall ’13 325

Winter ’14 259

Spring ’14 223

Fall ’14 353

Domestic Graduate Student Admitted in the following terms:

Insights Preview

Factors Discovered to be Responsible for Driving the Majority of Melt at University College: • Financial Aid – have not applied or applied and did not qualify • Distance to DU campus – Current address further than 46

miles from campus • Time between application start & application package

completed – More than a month • Time between admit decision & term start - More than 5

months

Enrollment Melt Benchmark Report:

Fall 2013

Winter 2014

Spring 2014

Fall 2014

Average All Terms

Melt % 15.7% 14.2% 12.7% 20.0% 16.6%

Decision Tree Data Model

• Predicts an applicant’s enrollment

• Sorts applicants from the “root node” of the tree, which

contains all applicants, down to a “leaf node” which

classifies the applicant.

• Uses an algorithm to select the best variables and the

best splits

• Each split is chosen to increase homogeneity within each

node as you travel down the tree.

• Data is split into a test and train dataset to avoid overfitting

• Best model correctly classifies the most applicants in the

test dataset

• Emphasis place on correctly classifying at-risk applicants

Decision Tree Data Model

Applicants in these buckets are predicted to not-enroll (At-Risk)

Applicants in these buckets are predicted to enroll

< 46 miles

> 46 miles

Financial Aid Status at Cut-off Date

Enrolled

Time between Application Start and Application Package Complete

Offered financial aid or applied

and waiting for award

offer

Has not applied for financial aid or applied but did not

qualify

Distance to DU campus

< 34 Days

> 34 Days

Enrolled Not

Enrolled

Not Enrolled

Classification Matrix for Test Dataset Predicted

Actual Not-Enrolled Enrolled Not-Enrolled

81% 36%

Enrolled 19% 64% Overall Misclassification Rate: 33.2%

Building a Replicable Model to Inform Target Actions

Operationalizing the Predictive Model

How do we use the model to identify an at-risk student?

For Fall, Spring, & Winter Terms: 1. Pull the following 4 variables for each

applicant 7 days after their admit decision is made: o Financial Aid Status o Zip Code o Application Start Date o Last Application Requirement

Date

2. Complete the form with the corresponding data to assess risk status

3. Save the student information, data, and risk status to the database table for future tracking

Recommendations

Key Insight Findings

At-Risk Variables: Identification of student’s who are predicted to be at-risk of not enrolling

• Student’s who have not applied or applied and did not qualify for financial aid

• Student’s who live further than 46 miles from DU’s campus (despite the fact that the majority of University College’s enrollments are online - 77%)

• Student’s who take more than a month to complete their application package

• Student’s admitted more than 5 months before term start

Process Change: Limited Financial Aid Advising

• Lack of financial aid is the largest driver of enrollment melt • An analysis of the communication plan revealed not enough is

being done to advise on financial aid options

Process Change: Gap in Communication Plan

• NO communication with applicants after admission letter is sent until registration period begins

• Significant negative impact on applicants who apply early and for applicants who defer to a future term

• Pre-enrollment advisor contact varies by department

New Data Opportunity: Lack of tracking of communications and outreach

• Important pre-enrollment contacts are not currently tracked

Action Plan Going Forward

Key Insight Initiative

At-Risk Variables: Identification of student’s who are predicted to be at-risk of not enrolling

• Create an action plan to implement the predictive model to identify at risk students.

• Define a contact plan with custom interventions around the applicant’s risk variables and execute for each high risk applicant.

Process Change: Limited Financial Aid Advising

• Getting comfortable talking about money and ‘financing your education’

• Staff training with the Financial Aid office • Discussing financial aid intentionally with inquiries and applicants

during initial phone conversation and adding financial aid information to the inquiry communication plan.

Process Change: Gap in Communication Plan

• Make changes to communication plan to fill this gap for students who apply early or defer to a future term

• Fantastic opportunity to build relationship and provide meaningful communications − Develop a Prospective Student Success Communications and

Resource Library

New Data Opportunity: Lack of tracking of communications and outreach

• Scheduled implementation of Technolutions Slate communications tool - fall 2015. This admission CRM will significantly help keep track of pre-enrollment student contacts and pre-advising outreach.

The Ecosystem of a Successful Learning Analytics Initiative

19

Learning analytics can inform improvements across the full circle of components essential to achieving bold, strategic outcomes for student success and academic quality.

Faculty Development

Course & Program Design

Student Support & Success

Instructional Innovation

Outcomes Assessment

• Impact of faculty development

programs

• Targeted curriculum

improvements

• Increased student success and

retention across the student

lifecycle

• Development of personalized

learning and student support

• Assessment of student and

program learning outcomes

Aligning data collection, reporting, and analysis needs with strategic, institutional goals and specific outcomes

Planning

Phase 1

Phase 2

• Create a student success/assessment plan that outlines strategy & desired outcomes

• Identify the people, process & technology needed to achieve outcomes

• Review plan • Collect data • Review/analyze data • Develop improvement

action plan informed by data

• Implement improvement plan

• Collect data • Analyze data • Evaluate improvement plan • Review original student

success/assessment plan

Lindsay Baun Data Analyst Pearson Consulting Services lindsay.baun@pearson.com

Deb Corso-Larson Academic Consultant Pearson Consulting Services debora.corso-larson@pearson.com

#pearsoncite

Michele Long (Contributor) Director of Student Services University College of the University of Denver michele.long@du.edu