2011using data analytics
DESCRIPTION
TRANSCRIPT
Assessment & Analytics
Using Data to Inform & Improve Student Success Strategy
Introductions
Initial Data ReviewFindings
Fall 2005
85% of online students were completing courses.
77% of students who completed, passed course with grade of C or better.
Is this good enough?
What questions should we ask?
Initial Data ReviewFindings
Challenge identified in 2005:
77% of at-risk students completed courses
57% of at-risk students earned C or better
At-risk included:
New to online (40%)
Returning to school after 5 years (35%)
Low High School GPA (15%)
Desire to increase Student Success
By 2007:
82% of at-risk students completed
5% Improvement
66% of at-risk students earned C or better
9% Improvement
Program ImprovementIowa Results
Identify at-risk students early.
“The effectiveness of early (identification) is that it’s early” - Vincent Tinto, NISOD 2011
Intervene and support students in need.
Increase at-risk student success rate.
Goal in 2005Improve At-Risk Student Success
What is At-Risk?Before Course Start
First-time online student
Returning to school after 5+ years
Registered for three or more online classes simultaneously
Failed an online course the previous 2 years
What is At-Risk?After Course Start
Student enrolls after course start
Grade-to-date is below a ‘C’
Stops attending for 5 consecutive days
Logs-in fewer than 3 times per week
Activity less than 3 hours per week.
Week 1 Days Logged In:Completed vs. Dropped Student
3.12.9
3.4
2.7 2.8
3.33.6
3.2
1.71.5
1.91.7
1.41.6
1.91.8
0
1
2
3
4
5
Business CompScience
English HealthSciences
Math Psych Science Soc
CompletedAvg. = 3.1
DroppedAvg. = 1.7
Avg. Days Logged In/Section by Department
Completed Student at Add/Drop Dropped Student at Add/Drop
Week 1 Activity:Completed vs. Dropped Student
2.93.3
3.8
2.52.8
3.3
4.0
3.0
1.4 1.3
1.9
1.3 1.2 1.2 1.21.5
0
1
2
3
4
5
Business CompScience
English HealthSciences
Math Psych Science Soc
CompletedAvg. = 3.1
DroppedAvg. = 1.3
Avg. Student Activity/Section by Department (hrs)
Completed Student at Add/Drop Dropped Student at Add/Drop
Action Taken
Hire Student Services Manager
Leverage LMS Data Warehouse
Identify / Track At-Risk Students
Provide Manual Intervention
Intervention Strategy
Personalized, Automated Welcome Email
Semi-Automated Intervention Emails
Weekly Activity/Progress Reports
Instructor Reporting
Sharing of Better Practices
FirstName
Course Last
Login Date
Activity Minutes
Activity Submission
Count
Course Points
Earned
Course Points
Possible to Date
Course Average
Grade To Date
Rocky 4/28/11 722 11 133 470 28.30%Melissa 5/8/11 3,053 35 479 530 90.38%Megan 5/5/11 2,981 30 557 556 100.18%Anneliese 5/7/11 5,633 66 509 585 87.01%Bradley 5/8/11 2,613 57 508 820 70.37%Alyssa 5/8/11 3,290 116 625 680 91.91%Lynsey 5/8/11 2,536 72 860 1,000 88.02%Jeremy 5/8/11 1,209 37 62 190 32.63%Angelica 5/7/11 731 24 789 900 87.67%Carrie 4/5/11 103 2 31 145 21.38%
Progress ReportProvided to College Advisors Weekly
Instructor ReportConveniently Located Within Course
Instructor ReportProvided to Advisors
Student Name
Instructor Name
Reason for At Risk
Other Explanations At Risk Date
Heather Lori Other Did not complete Unit 1. I did get an e-mail doesn't understand how class works, needs some help.
1/23/2011
Ann Steve Not completing assignments
missed first quiz and first lab assignment
1/26/2011
Courtney Rose Other Did not participate in Week 1. 1/26/2011
Donna Mark Stopped Participating
She participated through week 6, has since dropped off the map and I have not received any work from her. She’s not responded to e-mail.
3/16/2011
Results
Course success rate for at-risk students increased 9% over 2-year period.
IntermissionYou Can Do This
Questions?
In 2010:
84% of at-risk students completed their courses
Additional 2% improvement in 3 years
68% of successful at-risk students completed courses with a C or higher
Additional 2% improvement in 3 years
Now What?Results Today
60%
80%
100%
0 500 1,000 1,500 2,000 2,500 3,000 3,500 4,000
Avg. SectionCompletionRate = 77%
Department Term Start Enrollments
I
IIIII
IV
Median DeptEnrollments = 767
Dep
artm
ent
Co
mp
leti
on
Rat
es
– T
erm
Sta
rt t
o T
erm
En
d
English
Mathematics
Now What?Completion Rates vs. Enrollments
What Now?
Further improve Success in English Composition and Math.
Enhance At-Risk Case Management
Provide simple, visual Dashboard for instructors
Publish Methodology
Faculty Dashboard
Student Dashboard
Enrollments (at course end) 12Withdrawals 7Course Average Grade 55%
Noticeable Trends:Top scoring students show strong edges to professor in both directions.
Secondary networks forming between multiple high performing students
Questions to Research (Predictive)1. Do networks between students relate to successful completion?
2. How does professor interaction with at risk or low performing students impact student success?
MAT100 – Section A
Thread Interaction
Enrollments (at course end) 17Withdrawals 4Course Average Grade
83%
Noticeable Trends:Multiple directional edges to and from the professor node.
Multiple secondary networks forming.
Lamsen is a curiosity – no target interaction to that node, yet multiple source edges
Joshua not part of the forming subnetworks
Question to Research (Predictive)How important is multi node, multi directional interaction to course success?
MAT100 – Section B
Thread Interaction
ConclusionYou can do this too.
Questions?
Contacts
Steve Rheinschmidt, Director ICCOC
Steve Ast, Senior Account Exec. eCollege