using demographic data to predict students’ achievement in dspm courses daryl stephens, etsu...

Using Demographic Data to Predict Students’ Achievement in

DSPM Courses

Daryl Stephens, ETSUDaryl Stephens, ETSU

stephen@etsu.edustephen@etsu.edu

TNADE TNADE

October 27, 2005October 27, 2005

Students take developmental studies Students take developmental studies program (DSP) courses for many reasons:program (DSP) courses for many reasons:

Forgot material learned in high school by Forgot material learned in high school by the time they took entrance teststhe time they took entrance tests

Long gap between high school and Long gap between high school and matriculationmatriculation

Didn’t take high school seriouslyDidn’t take high school seriously

Didn’t consider going to college until laterDidn’t consider going to college until later

First generation college studentFirst generation college student

(Salter & Noblett, 1994)(Salter & Noblett, 1994)

Most students (~90%) at Most students (~90%) at ETSU take MATH 1530, ETSU take MATH 1530, Probability and Statistics, as Probability and Statistics, as their mathematics course to their mathematics course to satisfy core curriculum satisfy core curriculum requirements.requirements.

In previous years, this course In previous years, this course had a high failure rate.had a high failure rate.

The ProblemThe Problem

Placement procedures for TBR schools Placement procedures for TBR schools have changed.have changed.

Pre-2000:Pre-2000:ACT < 19 or age > 22 → take AAPPACT < 19 or age > 22 → take AAPP

2000-2002: COMPASS replaces AAPP2000-2002: COMPASS replaces AAPP

2002: ACT score alone determines 2002: ACT score alone determines placement for students with score < 3 placement for students with score < 3 years old; COMPASS alone for othersyears old; COMPASS alone for others

The ProblemThe Problem

State funding is static. State funding is static.

Placement decisions based on one test.Placement decisions based on one test.

Enrollment caps may happen in the future.Enrollment caps may happen in the future.

Is there a way to augment the placement Is there a way to augment the placement process by predicting a student’s chance process by predicting a student’s chance of success or failure in developmental or of success or failure in developmental or core math classes?core math classes?

PurposePurpose

Develop models to predict success of Develop models to predict success of students instudents in– DSPM 0800 (Elementary Algebra)DSPM 0800 (Elementary Algebra)– DSPM 0850 (Intermediate Algebra)DSPM 0850 (Intermediate Algebra)– MATH 1530 (Probability and Statistics)MATH 1530 (Probability and Statistics)

Use multiple regression to develop the Use multiple regression to develop the models using readily obtainable models using readily obtainable informationinformation

ImportanceImportance

MATH 1530 traditionally had high failure MATH 1530 traditionally had high failure raterate

Developmental math students are at a Developmental math students are at a greater risk of failure and dropping outgreater risk of failure and dropping out

Specific developmental studies program Specific developmental studies program advisors done away with in budget crunch advisors done away with in budget crunch of 2003of 2003

ImportanceImportance

Previous similar studies on developmental Previous similar studies on developmental students used additional instruments students used additional instruments which cost money.which cost money.

Very few universities require probability Very few universities require probability and statistics for math credit for and statistics for math credit for graduation. Most similar studies on core graduation. Most similar studies on core math deal with college algebra, math deal with college algebra, precalculus, or math survey courses.precalculus, or math survey courses.

AssumptionsAssumptions

Prediction is possiblePrediction is possible

Self-reported data are correctSelf-reported data are correct

Information in SIS is correctInformation in SIS is correct

LimitationsLimitations

Different demographics and Different demographics and course requirements from course requirements from other institutions, so work only other institutions, so work only applies to ETSUapplies to ETSU

Data only collected for fall; Data only collected for fall; spring and summer grades are spring and summer grades are probably differentprobably different

DefinitionsDefinitions

DSPM 0800: Elementary algebra DSPM 0800: Elementary algebra – Arithmetic review, algebraic representations, Arithmetic review, algebraic representations,

linear equations in one and two variableslinear equations in one and two variables

DSPM 0850: Intermediate algebraDSPM 0850: Intermediate algebra– Exponents, polynomials, factoring, Exponents, polynomials, factoring,

MATH 1530: Probability and statisticsMATH 1530: Probability and statistics– ““Stat mansion” and “stat cave”Stat mansion” and “stat cave”

DefinitionsDefinitions

COMPASS (Computerized Adaptive COMPASS (Computerized Adaptive Placement Assessment and Support Placement Assessment and Support SystemSystem– Prealgebra and algebra sectionsPrealgebra and algebra sections– Reading and writing sectionsReading and writing sections– Replaced AAPPReplaced AAPP

Previous ResearchPrevious Research

Developmental Studies HistoryDevelopmental Studies History

Special programs at Harvard in 17Special programs at Harvard in 17thth centurycentury

Preparatory department at University of Preparatory department at University of Wisconsin, 1849Wisconsin, 1849

Morrill Acts, late 19Morrill Acts, late 19thth century, establish century, establish land grant colleges and extend access to land grant colleges and extend access to higher education to more peoplehigher education to more people

Developmental Studies HistoryDevelopmental Studies History

Preparatory departments widespread in Preparatory departments widespread in early 20early 20thth century (350 in 1915) century (350 in 1915)

GI Bill of Rights brings in veterans with GI Bill of Rights brings in veterans with needs for auxiliary servicesneeds for auxiliary services

1960s-70s: Increase of women, students 1960s-70s: Increase of women, students of color, first-generation students, students of color, first-generation students, students with learning disabilities; open-access with learning disabilities; open-access community colleges establishedcommunity colleges established

Tennessee DSP HistoryTennessee DSP History

TBR establishes formal developmental TBR establishes formal developmental studies program in 1984.studies program in 1984.

Defining Our FutureDefining Our Future (2001): “operate more (2001): “operate more efficiently and with more limited resources”efficiently and with more limited resources”

Move 0700-level courses to community Move 0700-level courses to community collegescolleges

Related ResearchRelated Research

Developmental MathDevelopmental Math

Core Math (almost nothing on P&S)Core Math (almost nothing on P&S)

What factors are related to success in the What factors are related to success in the courses? Three broad categories:courses? Three broad categories:– AcademicAcademic– DemographicDemographic– AffectiveAffective

What variables predict course What variables predict course success in dev. math?success in dev. math?

High school GPA, at least for traditional High school GPA, at least for traditional studentsstudents

Scores on the ACT or SAT — sometimesScores on the ACT or SAT — sometimes

Placement tests (e.g. COMPASS, ASSET, Placement tests (e.g. COMPASS, ASSET, Accuplacer, CPT, PTT, CLAST) — Accuplacer, CPT, PTT, CLAST) — sometimessometimes

GED math scores — as adjunct to other GED math scores — as adjunct to other placement scoresplacement scores

Course success in dev. mathCourse success in dev. math

Number of high school mathematics Number of high school mathematics courses taken — sometimescourses taken — sometimes

High school math GPA – some coursesHigh school math GPA – some courses

College GPACollege GPA

AttendanceAttendance

Study habitsStudy habits

Age or length of time since last math classAge or length of time since last math class

Course success in dev. math (cont’d)Course success in dev. math (cont’d)

Gender (some studies)Gender (some studies)

Race Race

Math anxiety level Math anxiety level notnot related to grade! related to grade!

AttitudeAttitude

Perception of success, engagement in Perception of success, engagement in classclass

Paying attention and interacting with Paying attention and interacting with instructorinstructor

Success in core classesSuccess in core classes

Very little research on success in classes Very little research on success in classes like MATH 1530, so other courses like MATH 1530, so other courses examinedexamined

ACT/SAT math scores (usually)ACT/SAT math scores (usually)

COMPASS, TASP, local placement testsCOMPASS, TASP, local placement tests

High school GPA for college algebra High school GPA for college algebra (multiple studies) and calculus (but not (multiple studies) and calculus (but not precalculus)precalculus)

Success in core classes (contd.)Success in core classes (contd.)

HS math GPA – mixed resultsHS math GPA – mixed results

HS percentile rankHS percentile rank

Number and difficulty of HS math classes Number and difficulty of HS math classes takentaken

Whether a math class taken senior yearWhether a math class taken senior year

Students who didn’t take intermediate Students who didn’t take intermediate algebra scored sig. higher in college alg.algebra scored sig. higher in college alg.

Success in core classes (contd.)Success in core classes (contd.)

Age in some casesAge in some cases

Time since last math courseTime since last math course

Gender? Yes in 3, no in 5 studiesGender? Yes in 3, no in 5 studies

Full-time vs. part-time (1 study)Full-time vs. part-time (1 study)

Class meeting time (1 study)Class meeting time (1 study)

No difference in resident vs. commuter, No difference in resident vs. commuter, campus activitycampus activity

Success in core classes (contd.)Success in core classes (contd.)

AttitudeAttitude

Learning stylesLearning styles

Self-conceptSelf-concept

Research QuestionsResearch Questions

Relationship QuestionsRelationship QuestionsIs there a relation between course grade and Is there a relation between course grade and

……

ACT (DSPP) math scores?ACT (DSPP) math scores?

ACT (DSPP) reading scores?ACT (DSPP) reading scores?

COMPASS intermediate algebra scores?COMPASS intermediate algebra scores?

COMPASS reading scores?COMPASS reading scores?

Number of college preparatory math classes Number of college preparatory math classes taken in high school?taken in high school?

High school GPA?High school GPA?

Regression QuestionsRegression QuestionsCan a regression equation be found to predict final Can a regression equation be found to predict final

course grade based on these items?course grade based on these items?

From SIS:From SIS:ACT compositeACT compositeACT mathACT mathACT readingACT readingACT EnglishACT EnglishHigh school GPAHigh school GPAAge on first day of Age on first day of classclass

From Survey:From Survey:Number of high Number of high school math classes school math classes takentakenNumber of years Number of years since last HS math since last HS math classclass

Regression QuestionsRegression QuestionsCan a regression equation be found to predict final Can a regression equation be found to predict final

course grade based on these items?course grade based on these items?

From SIS:From SIS:COMPASS writingCOMPASS writingCOMPASS readingCOMPASS readingCOMPASS COMPASS prealgebraprealgebraCOMPASS COMPASS intermediate algebraintermediate algebraAge on first day of Age on first day of classclass

From Survey:From Survey:Number of high Number of high school math classes school math classes takentakenNumber of years Number of years since last HS math since last HS math classclass

MethodMethod

Collect information about courses taken in Collect information about courses taken in high school and year of last high school high school and year of last high school math class from students via surveymath class from students via survey

Obtain other information from SISObtain other information from SIS

Use Pearson correlation for relationship Use Pearson correlation for relationship questionsquestions

Use stepwise multiple regression for grade Use stepwise multiple regression for grade prediction questionsprediction questions

Initial PlacementInitial PlacementACT (DSPP) Math sectionACT (DSPP) Math section– << 17 (SAT 17 (SAT << 440): DSPM 0800 440): DSPM 0800– 18 (SAT 450): DSPM 085018 (SAT 450): DSPM 0850– >>19 (SAT > 450): college level math19 (SAT > 450): college level math

COMPASSCOMPASS– Prealgebra score Prealgebra score << 29: DSPM 0700 29: DSPM 0700– Prealgebra 30-99 and algebra 20-27: DSPM Prealgebra 30-99 and algebra 20-27: DSPM

08000800– Algebra 28-49: DSPM 0850Algebra 28-49: DSPM 0850– Algebra 50-99: college level mathAlgebra 50-99: college level math

Data considerationsData considerations

Not counted: grades of W, WF, INot counted: grades of W, WF, I

FN grade not counted to be consistent FN grade not counted to be consistent with some other ETSU studieswith some other ETSU studies

Online sections, RODP sections not Online sections, RODP sections not includedincluded

Surveys returnedSurveys returned

DSPM 0800: 149 / 304 (49%) DSPM 0800: 149 / 304 (49%) – No night, off-campus, or ITV sectionsNo night, off-campus, or ITV sections

DSPM 0850: 214 / 455 (47%)DSPM 0850: 214 / 455 (47%)– Included Kingsport, night, ITVIncluded Kingsport, night, ITV

MATH 1530: 631 / 1074 (59%)MATH 1530: 631 / 1074 (59%)

ResultsResults

Grade Grade DistributionDistribution

Table 3

* Grades of C-, D+, and D are not allowed in developmental studies courses.

**Not used in answering the research questions.

GradeDSPM 0800

DSPM 0850

MATH 1530

A 35 75 70

A- 17 18 48

B+ 14 17 35

B 21 21 98

B- 8 11 36

C+ 9 13 34

C 13 25 89

C- * * 35

D+ * * 42

D * * 28

F 19 19 49

FN** 10 8 58

I** 0 1 1

W** 3 6 5

WF** 0 0 3

Total 149 214 631

Mean 2.785 2.991 2.419

SD 1.2993 1.2075 1.1526

ACT Math vs. GradeACT Math vs. Grade

ACT Math

DSPM 0800 DSPM 0850 MATH 1530

r 0.025 .322** .481**

GPTS p 0.802 0 0

N 102 169 499

ACT Reading vs. GradeACT Reading vs. Grade

ACT Reading

DSPM 0800 DSPM

0850MATH

1530

r 0.047 0.164* 0.164*

GPTS p 0.642 0.033 0

N 102 169 497

COMPASS Int. Alg. vs. GradeCOMPASS Int. Alg. vs. Grade

COMPASS Intermediate Algebra Score

DSPM 0800

DSPM 0850

MATH 1530

r 0.107 0.326* 0.133

GPTS p 0.448 0.029 0.348

N 52 45 52

COMPASS Reading vs. GradeCOMPASS Reading vs. Grade

COMPASS Reading Score

DSPM 0800

DSPM 0850

MATH 1530

r 0.222 0.206 0.118

GPTS p 0.194 0.323 0.513

N 36 25 33

High High School School Math Math

CoursesCourses(Most (Most

Common)Common)

CourseCourse 08000800 08500850 15301530

PrealgebraPrealgebra 4747 3737 6767

Foundations 2Foundations 2 1616 2626 2828

Algebra 1Algebra 1 123123 181181 518518

Algebra 2Algebra 2 111111 182182 556556

GeometryGeometry 102102 176176 544544

PrecalculusPrecalculus 66 2727 211211

Trig/Adv. Alg.Trig/Adv. Alg. 44 2727 133133

Prob. & Stat.Prob. & Stat. 44 66 7272

CalculusCalculus 22 44 8888

NN 149149 214214 633633

Course Grade vs. Number of Course Grade vs. Number of College Prep Classes TakenCollege Prep Classes Taken

HSMATH (All students)

DSPM 0800

DSPM 0850

MATH 1530

r -0.068 0.191** 0.265**

GPTS p 0.434 0.007 .000

N 135 198 564

Mean # taken 2.52 3.11 3.73

SD taken 1.034 0.955 1.115

Course Grade vs. Number of Course Grade vs. Number of College Prep Classes TakenCollege Prep Classes Taken

HSMATH (Students with ACT Scores)

DSPM 0800

DSPM 0850

MATH 1530

r -0.237* 0.193* 0.298**

GPTS p 0.017 0.012 .000

N 102 168 499

Mean # taken 2.84 3.28 3.85

SD taken 1.2333 1.2 1.034

Course Grade vs. Number of Course Grade vs. Number of College Prep Classes TakenCollege Prep Classes Taken

HSMATH (Students with COMPASS scores)

DSPM 0800

DSPM 0850

MATH 1530

r -0.061 0.131 0.171

GPTS p 0.671 0.392 0.226

N 51 45 52

Mean # taken 1.88 2.53 2.85

SD taken 1.103 1.254 1.036

HSGPA vs. GradeHSGPA vs. Grade

HS Overall GPA

DSPM 0800 DSPM 0850 MATH 1530

r .389** .333** .445**

GPTS p .000 .000 .000

N 105 176 500

Regression Using ACTRegression Using ACT

DSPM 0800 DSPM 0800 (95 students)(95 students)

Model 1:Model 1:ŷ = –.589(HSMATH) + 4.599 (p = .001)

Model 2:ŷ  = –.765(HSMATH) + 1.009(HSOGPA)

+ 2.298 (p < .001)

Regression Using ACTRegression Using ACT

DSPM 0850DSPM 0850 (160 students) (160 students)

Model 1:Model 1:ŷ  = .364(ACTM) – 3.238 (p < .001)

Model 2:ŷ = .301(ACTM) + .662(HSOGPA) – 4.111

(p < .001)

Regression Using ACTRegression Using ACT

MATH 1530MATH 1530 (475 students) (475 students)Four models Four models ((p p < .001 in each case; < .001 in each case; rr22 ranging ranging

from .233 to .353):from .233 to .353):ŷ  = .134(ACTM) – .521ŷ  = .092(ACTM) + .771(HSOGPA) – 2.181ŷ  = .103(ACTM) + .910(HSOGPA) 

+  .108(AGE) –  4.953ŷ  = .089(ACTM) + .855(HSOGPA)  

+  .105(AGE) + .025(ACTE) – 4.9

Regression Using COMPASSRegression Using COMPASS

DSPM 0800DSPM 0800 (33 students) – no equation (33 students) – no equation possiblepossible

DSPM 0850DSPM 0850 (23 students with all sections) (23 students with all sections)

ŷ = .020 (COMPASS writing) + 1.374

(r2 = .221, p = .024)

DSPM 0850 (44 students with math scores)

ŷ = .023 (COMPASS arithmetic) + 1.730

(r2 = .190, p = .033)

Regression Using COMPASSRegression Using COMPASS

MATH 1530MATH 1530

22 students with all COMPASS sections: 22 students with all COMPASS sections: no equation possibleno equation possible

51 students with just math sections:51 students with just math sections:

ŷ = .027(COMPASS arithmetic) + .671

(r2 = .276, p < .001)

Observations, Conclusions, Observations, Conclusions, and Recommendationsand Recommendations

Observations: 0800Observations: 0800

Elementary algebra: only high school GPA Elementary algebra: only high school GPA showed a significant correlation with showed a significant correlation with course grade; math preparation in high course grade; math preparation in high school showed a little bit of predictive school showed a little bit of predictive value.value.

Finding agrees with Long (2003) – ACT Finding agrees with Long (2003) – ACT math score not significantly correlated with math score not significantly correlated with E.A. course gradeE.A. course grade

Observations: 0800 (cont’d)Observations: 0800 (cont’d)

Odd finding: Why a negative correlation Odd finding: Why a negative correlation between HS math prep and grade in between HS math prep and grade in 0800?0800?

ObservationsObservations

Intermediate algebra: course grade Intermediate algebra: course grade correlated with ACT math and reading, correlated with ACT math and reading, COMPASS intermediate algebra and COMPASS intermediate algebra and reading, high school math preparation, reading, high school math preparation, high school GPA. high school GPA.

ACT math correlation agrees with Lott ACT math correlation agrees with Lott (1990)(1990)

HS math preparation agrees with 4 studiesHS math preparation agrees with 4 studies

ObservationsObservations

Probability and statistics grade was Probability and statistics grade was significantly correlated with ACT math and significantly correlated with ACT math and reading and HSGPA, but regression used reading and HSGPA, but regression used ACT math and English, age, and HSGPA. ACT math and English, age, and HSGPA. Only COMPASS arithmetic score was in Only COMPASS arithmetic score was in regression for students without ACT regression for students without ACT scores.scores.Findings in line with 7 other studies of Findings in line with 7 other studies of other core college math courses.other core college math courses.

RecommendationsRecommendations

For students with borderline placement For students with borderline placement scores, use the regression equation to scores, use the regression equation to fine-tune the placement.fine-tune the placement.

If student’s regression equation predicts a If student’s regression equation predicts a failing grade, give extra attention to that failing grade, give extra attention to that student (watch attendance, recommend student (watch attendance, recommend tutoring, etc.), or consider placement in a tutoring, etc.), or consider placement in a lower level class if possible.lower level class if possible.

Future ResearchFuture Research

Repeat study with data from spring and Repeat study with data from spring and summer. Historically, grades in summer. Historically, grades in developmental math classes are developmental math classes are significantly higher in summer and lower in significantly higher in summer and lower in spring. (Unknown if true for 1530)spring. (Unknown if true for 1530)

Repeat treating repeaters separately, or Repeat treating repeaters separately, or include number of previous attempts as a include number of previous attempts as a variablevariable

Future ResearchFuture Research

Is grade in 1530 affected by other college-level Is grade in 1530 affected by other college-level or developmental math experience?or developmental math experience?

Characteristics of repeating students (mostly Characteristics of repeating students (mostly qualitative research)qualitative research)

What if core curriculum at ETSU changes What if core curriculum at ETSU changes allowing other math courses to count for allowing other math courses to count for graduation?graduation?

How would raising cutoff scores affect grades? How would raising cutoff scores affect grades? (TBR uses 19, ACT recommends 22)(TBR uses 19, ACT recommends 22)

Future ResearchFuture Research

Qualitative study (suggested Qualitative study (suggested by a student): What kind of by a student): What kind of guidance were students given guidance were students given in selecting high school math in selecting high school math classes?classes?

Why a negative correlation Why a negative correlation between HS math prep and between HS math prep and 0800 grade?0800 grade?

For more informationFor more information

Entire dissertation is available online:Entire dissertation is available online:http://faculty.etsu.edu/stephen/http://faculty.etsu.edu/stephen/

Will be online from UTK in Jan. or Feb., Will be online from UTK in Jan. or Feb., but will stay on my faculty web page but will stay on my faculty web page unless there’s a space crunch.unless there’s a space crunch.

This slide show is also on my faculty web This slide show is also on my faculty web page. Look for the “conference page. Look for the “conference presentations” link.presentations” link.