Dr. Nimet Alpay & Dr. Natalya Koehler.

Successful Redesign of Online Statistical Concepts Course: Improved Student Retention,

Performance, and Satisfaction with the Course.  

THE NEED

College Graduation

1 According to the 2011 report from Complete College America

Part-time students have a 24.3% chance of graduating with a 4-year bachelor’s

degree within 8 years and adult students are far less likely to complete their

degrees than their traditional peers.1

Student PopulationStatistics of Undergraduate Population Attending Franklin University

Category Percentage/Years

Average Age 33.27 years

Race Caucasians – 66.7%African Americans – 21.6%Others – 11.7%

Student Status Full time – 33.6%Part time – 66.4%Non-traditional – 80%

Financial Aid Recipients 70%

Gender Males – 43.5%Females – 56.5%

MATH 215 Student Population

Gateway Course

General Education Course

FF and online formats

Pre-requisite

Levels of students’ motivation (the course serves as a pre-requisite or not)

MATH 215 Course Pre-requisites

• Learning Strategies• Introduction to Excel

Spreadsheets• Intermediate Algebra or

Introduction to Quantitative Reasoning (new course)

Student Performance and Attrition in Fall 2013

Redesign GoalsTo increase

Student Retention Performance Graduation Rates Instructional Support for Online Learners

Topics Coverage Based on ODHE Requirements

Below are the course topics based on ODHE requirements for MATH 215:

• sampling techniques• data types• experiments• measures of central tendency• measures of dispersion• graphical displays of data• basic probability concepts• binomial and normal probability distributions• sampling distributions and Central Limit Theorem• confidence intervals• hypothesis tests of a mean, or a proportion for one or two populations• hypothesis tests for qualitative data• linear regression

Topics that are excluded from the old course based on ODHE requirements:• ANOVA• Regression Analysis

THE ANALYSIS BEFORE REDESIGN

Data-driven Course RedesignThe course redesign was based on the analysis of:

Student course evaluation surveys

Instructor interviews

Program and department chair interviews

Student use of optional student academic support services

Data about student performance and attrition

Factors Impacting Student Retention

at the Course Level

Effective Instruction Effective Course Design

Student Retention

IMPACTIM

PACT

We Identified Need for Systematic Instruction (Fall 2013)

Class 1 – High Attrition Rate (44%)

Comments about the instructor:1. “Professor X seemed to genuinely want me

to succeed.“

Comments about the course:2. “I feel that this course was extremely

demanding. Between the homework, quizzes, group assignments, and test, it felt nearly impossible for someone who isn’t strong in math (like myself) to keep up”.

Class 2 – Low Attrition Rate (16%)

Comments about the instructor:1. “He did an excellent job.”2. Professor X was great, he always took the time to talk to all of us provided several examples that related the information to everyone.

Comments about the course:

2. I thought the program worked very well with my learning style.

• The instructor was helpful in providing “just in time” help.

• Students find the course very difficult.

• The instructor provided systematic instruction. • Students do not have negative comments about

the course.

Student Course Evaluation Survey (Fall 2013, old course design)

THE REDESIGN

The Redesign Elements

Weekly Assignment & Assessment

types

Textbook change

Web-based component

change

Systematic instruction

through interactive multimedia

lectures

Textbook ChangeOld Textbook• Triola, M. (2014) Elementary Statistics Using

the TI-83/84 Plus Calculator (3rd edition). Pearson.

New Textbook• Navidi W. & Monk B. (2014). Essential

Statistics. (Franklin University Custom Edition). McGraw-Hill

“Extremely confusing textbook.”-Rep. Student Comment “Textbook is easy to understand.”

-Rep. Student Comment

“I liked the book a lot as it broke down the concepts in a clear and

concise manner.”-Rep. Student Comment

“Content explanations and exercises don’t seem connected to each other”

-Rep. Instructor Comment

Web-Based Component Change• SmartBook – adaptive learning technology• Connect Math – assignment and assessment platform with integrated

media-rich ebook.*Both products are integrated under ConnectMath

SmartBook Connect Math

Weekly Assignment & Assessment Types

Systematic Instruction – Interactive Multimedia Lectures28 interactive multimedia lectures to help students understand

different statistical concepts without frustration.

The lectures include the following components:• New information delivery

• Includes examples• Includes both visual and auditory information

• Information summary• “Check your learning” comprehension question

• Multiple choice• Drag and drop• Fill in the blank• Enter answer• Other

http://cs.franklin.edu/~reid21/showcase/story.html

Click here to see the demo of multimedia lectures Click on the button to see the demo

of multimedia lectures

THE RESULTS

Data Collection Instruments

Attrition & Performance Data Evaluations• Course evaluation• Multimedia evaluation

Instructor Feedback• Course design feedback• Multimedia feedback

Academic Support Services• Attendance Rates

Triangulation

Results: D Below Rates & Attrition Rates

35.29%

64.71%

Fall 2014 Attrition Rate: 14.31%

36.49%

63.51%

MATH 215 Student Performance – Fall 2014

33.93%

66.07%

35%

65%

Overall Performance Face-to-face Classes Online Classes

Results: Student Performance RatesFALL 13 FALL 14

Total # of Students 335 255

Online Students 221 (66%) 127 (50%)

FF Students 114 (34%) 128 (50%)

Grade A 12.2% 21.2%

Grades: A – C 49.8% 64.3%

Grades: D & Below 50.2% 35.6%

Attrition Rates 26.11% 14.31%

Continuing Effect of Course Redesign

FALL 13 FALL 14 WINTER 15 SUMMER 15 FALL 15

Total # of Students 335 255 279 147 236

Online Students 221 (66%) 127 (50%) 170 (61%) 83 (56%) 147 (62%)

FF Students 114 (34%) 128 (50%) 109 (39%) 65 (44%) 89 (38%)

Grade A 12.2% 21.2% 19.4% 20% 23%

Grades: A – C 49.8% 64.3% 66.7% 68% 66%

Grades: D & Below 50.2% 35.6% 31.4% 32% 33%

Attrition Rates 26.11% 14.31% 11% 13% 17%

Course Recommend RatesTERMS

Percent Course Recommend

Percent Faculty Recommend

Fall 13 55.8% 82%

Winter 14 57.3% 88%

Summer 14 53% 76%

Fall 14 71% 88%

Winter 15 69% 85%

Summer 15 71% 92%

Fall 15 77% 88%

Results: Course SurveyThemes from student comments about effectiveness of the new design of the course over Fall 14 – Sum 15.

# The themes from students’ comments (composite responses) -- FALL 2014

Number of users commenting on the theme

1. The course is challenging and overwhelming. 33

2. The course is well designed and can be self-led. 19

4. The educational value of ConnectMath auto-graded learning activities is excellent because it contains examples and guided solutions.

13

4. LearnSmart auto-graded learning activities are redundant. 10

5. Need for additional resources for learning (Student Learning Center).

8

Use of Academic Support Services• The number of students who used optional support services in the

Fall 2013 – Fall 2015.

Types of optional support services Fall 2013

Winter 2014

Summer 2014 Fall 2014

Winter 2015

Summer 2015 Fall 2015

Tutoring Services (OL, FF, W-In)

37.8% * 9.50% 16.00% 12.90% 9.40% 16.20% 8.3%

Test Preparation Workshops22.40% 15.40% 23.00% 13.70% 20.20% 12.20% 17.40%

Structural Learning Assistance (SLA)

13.40% 13.20% 21.30% 9.40% 15.20% 14.20% 12.20%

Informal Student Feedback via Email

-Tina

I felt pretty good taking the test and am pleased with my final score! I must say that I am extremely impressed with the way Franklin does this class. I know Statistics is typically a universally feared class, but it hasn’t been too bad so far.

-Brandon

I enjoyed the class even with all of the work. Near the end, everything has fit together and it all makes sense. I have a better understanding of how this type of math helps businesses. All of the information from this class will help me with my future business.

-Dianne

THE RECENT UPDATES

Introduced a New Course as a Pre-requisite

Math 115: Introduction to

Quantitative Reasoning

It includes topics from Beginning

Algebra and some concepts from

Descriptive Statistics and Simple Probability.

Application based.

Excel is used for tech

assignments.

This is the pre-requisite for

students who will NOT take College Algebra for their

programs.

Conducting a Research Study on the Use of Interactive Multimedia

Winter 15 term: Too few participants• incentive: raffle for Ipad Mini

Fall 15 term: 86 students participated • incentive: bonus points (2% of the available points in the course)

Winter 16 term: study still going on.

Average Student Ratings of the Interactive Multimedia Lectures (86 participants; rating scale of 5; 5-strongly agree, 1-strongly disagree)

Survey Questions Average ratings 1. I liked the layout of the sections in the lecture. 4.72. I liked the navigation in the sections in the lecture. 4.73. I liked the explanation and the examples used in the informational slides of the lecture sections. 4.64. I liked the summary slides used thought the sections. 4.55. I found the transcript option useful. 3.66. The sections helped me understand this weeks’ material. 4.67. The section design helped me retain the new information. 4.48. The section design helped me maintain my attention. 4.59. I found the self-assessment feature (Check Your Learning questions) helpful 4.510. The answer feedback to the Check Your Learning questions (explanations of the correct answers) were helpful.

4.5

11. The answer feedback to the Check Your Learning questions was sufficient. 4.312. I liked the supplemental content (if any), such as calculator tips and topic videos. 3.9 13. The pace of the content was good. 4.614. I enjoyed my experience. 4.5

Themes from Student Comments about Effectiveness of Multimedia Design

Representing Student Comments about Effectiveness of Multimedia Design

“Narrator spoke slowly and clearly.”

“In the randomized experiment section, the colors of the coupons and the colors of flyers were color coordinated to make the student (me)

visualize how to think about the percentages of the different groups of people. This demonstration was a real-world scenario that I could relate to and therefore made the material easy to understand and

increased me chances of retaining the information.”

“What I liked most about this lecture was the flow. The flow moved from the description of percentiles and quartiles into showing examples of how to calculate these items.”

“I appreciate the fact that If I stopped a section I could resume where a left off.”

“It was also helpful having the tutorial to be long enough to explain the information, but not too long to lose interest.”

Themes from Student Comments about Helpfulness of Software Features

Representing Student Comments about Effectiveness of Software Features

“I did access the supplemental materials and it was helpful by providing some more in depth information.”

“I think the use of visual aids and animation help the learner adjust to the new material more quickly than the text.”

“The Summary is also important because it reiterates the key elements, which can also help the viewer retain the information.”

“I really liked the transcript option so you could keep volume off or go a little quicker if you read through a familiar topic.”

“I feel like the more I test myself on the subject material, the more comfortable I get with determining the correct answer and confirming my understanding.”

“it is nice to have graphs and tables involved in an online class.”

Effectiveness of Support for Cognitive Processing of Information Provided by the Multimedia Lectures

Sub category Number of Commends

Helped students remember the material 37

Helped students understand the material 32

Helped students maintain their attention 24

Helped students organize information for encoding into long-term memory

13

Representing Student Comments about Cognitive Support for Information Processing

“Useful tool to use for learning, definitely helped understand the material.”

“The interactive lectures during the beginning of a week to watch will most certainly help make you study and remember the material in the long run.”

“The in-depth discussion about the material is what made me pay attention and want to take down notes.”

“By using this as a review I feel I was able to better retain the information.”

“The explanations break the material down in terms I can understand and relate to therefore making it easier to retain the information.”

Overall Satisfaction with the Multimedia Lectures

# Theme Number of Comments

1 Reduces phobia of math 7

2 Recommendation to build similar lectures for other courses 10

“Overall, I plan to continue to use the interactive multimedia lecture until the course ends. The material and discussion is very clear and concise and to the point.”

“I had a terrible time in previous math classes and dreaded taking this one. As it turns out this class has not been bad at all. The interactive lectures make a difference for me.”

“I have enjoyed these interactive multimedia lectures since I began using them at the start of the course. I think they would be very useful in other classes where topics tend to need more explanation than just written text.”

Instructor Responses to Multimedia

I would often watch the multimedia pieces, first out of curiosity, then in order to get a sense as to how I could approach topics during my meet sessions. I found them to be a helpful suggestions about what I could do during my meet session to help students understand the concepts we were discussing.

I heard mostly positive comments from my students indicating that the interactive lectures helped to clarify the material.

These are highly recommended to view before class for my students. I ask them to take this little bit of time to view these lectures so they know what we will talk about in class as reading the text does take more time. I find these lectures hit the main topics of the content for the week and the explanations are very good. Keep these.

THANK YOU