A Holistic Review of Gender Differences in Engineering Admissions and Early Retention

Beth M. HollowayP.K. Imbrie

Teri Reed-RhoadsSchool of Engineering Education

Purdue UniversityWest Lafayette, Indiana, USA

15th International Conference for Women Engineers and ScientistsAdelaide Australia, 19-22 July, 2011

Motivation

o Purdue’s College of Engineering (COE) has been working to increase the representation of women in its first year class for many years.

o Over the last 5 years, we have seen a 46% increase in the number of applications received from women, but only a 24% increase in the number of women admitted.

o At the same time, casual analysis seems to indicate that admitted women have higher metrics, on average, than admitted men.

Holistic Reviewo Subject matter expectationso Overall high school grade point average (GPA), on a 0 –

4.0 scaleo Core high school GPA (GPA of English, math, science,

foreign language, and speech classes only.), on a 0 – 4.0 scale

o High school class rank, in percentileo Standardized test scoreso Overall grades in academic courseworko Grades related to intended majoro Strength of student’s overall high school curriculumo Trends in achievemento Ability to be successful in intended majoro Personal background and experienceso Time of year student applieso Space availability in intended program

Research Questions

1. Are the metrics of women admitted to CoE statistically higher than those of men admitted to CoE?

2. To what extent do affective and cognitive measures from the Student Access and Success Instrument (SASI) model differences of success as measured by retention and graduation based on sex?

Admission Years: 2006, 2007, and 2008o Applicants to the College of Engineering

(resulting in 26,396 total records over the 3 cohort years)

o Applicants who are considered “Beginners”. (Transfer students, for example, were filtered out) (25,587 total records remaining)

o Applicants for the Fall semester ( 25,361 total records remaining)

o Applicants with complete applications (incomplete applications were filtered out)

o 23,068 total records remaining

Demographics of Fall 06, 07, and 08 Applicants to Eng.

Number % Number % Number % Number % Number % Number %

1632 5964 1652 6430 1369 6021

Caucasian, Non-Hispanic 1116 74.5% 4269 78.9% 1037 71.4% 4262 77.0% 926 74.3% 4023 76.8%

African American, Non-Hispanic 106 7.1% 214 4.0% 113 7.8% 206 3.7% 95 7.6% 259 4.9%

Hispanic American 73 4.9% 229 4.2% 76 5.2% 244 4.4% 66 5.3% 219 4.2%

Native American 15 1.0% 44 0.8% 6 0.4% 46 0.8% 5 0.4% 44 0.8%

Aisan American / Pacific Islander 160 10.7% 531 9.8% 178 12.3% 637 11.5% 135 10.8% 562 10.7%

Other 20 1.3% 93 1.7% 32 2.2% 92 1.7% 19 1.5% 89 1.7%

Not Reported 8 0.5% 29 0.5% 10 0.7% 46 0.8% 1 0.1% 43 0.8%

All Domestic 1498 91.8% 5409 90.7% 1452 87.9% 5533 86.0% 1247 91.1% 5239 87.0%

Indiana (% of Domestic) 334 22.3% 1510 27.9% 318 21.9% 1624 29.4% 272 21.8% 1715 32.7%

International 134 8.2% 555 9.3% 200 12.1% 897 14.0% 122 8.9% 782 13.0%

Race / Ethnicity

Residency

Total Number of Records

Demographics of Applicants2006

Women MenMenWomen2008 2007

Women Men

Analysis of Metric Medians for Applicant Pool

Women Men p-value

Median 3.9 3.7N 4457 17441

Median 3.74 3.48N 4603 18113

Median 93 86N 3029 11346

Median 620 600N 4611 18148

Median 670 680N 4611 18148

Median 1300 1280N 4611 18148

Overall GPA

Core GPA

Class Rank

SAT Verbal

SAT Math

SAT Total

0.0000

0.0000

0.0000

0.0000

0.0000

0.0000

All ApplicantsTotal

Boxplot of Overall GPA -Applicants

MenWomen

4.0

3.5

3.0

2.5

2.0

1.5

Overa

ll GPA

Boxplot of Overall GPA's for Men and WomenAll Applicants to Engineering

Boxplot of SAT Total Scores - Applicants

MenWomen

1750

1500

1250

1000

750

500

SAT

Tota

l Sco

re

Boxplot of SAT Total Scores for Men and WomenAll Applicants to Engineering

Analysis of Metric Medians for Admits to Engineering

Women Men p-value

Median 4.0 3.8N 3829 12790

Median 3.80 3.60N 3935 13201

Median 94 90N 2558 7963

Median 630 620N 3911 13127

Median 680 700N 3911 13127

Median 1320 1330N 3911 13127

SAT Math

SAT Total

All Admits to Engineering

Overall GPA

Core GPA

Class Rank

SAT Verbal

Total

0.0000

0.0000

0.0000

0.0000

0.0000

0.0100

Boxplot of Overall GPA - Admits

MenWomen

4.0

3.5

3.0

2.5

2.0

Overa

ll GPA

Boxplot of Overall GPA's for Men and WomenAll Admits to Engineering

Boxplot of SAT Total Scores - Admits

MenWomen

1700

1600

1500

1400

1300

1200

1100

1000

900

800

SA

T To

tal S

core

Boxplot of SAT Total Scores for Men and WomenAll Admits to Engineering

Analysis of Metric Medians for Denied Students

Women Men p-value

Median 3.4 3.2N 241 2071

Median 3.06 2.91N 255 2202

Median 75 66N 171 1485

Median 490 510N 277 2324

Median 550 590N 277 2324

Median 1050 1110N 277 2324

SAT Verbal

SAT Math

SAT Total

All Denies

Overall GPA

Core GPA

Class Rank

0.0000

Total

0.0000

0.0000

0.0000

0.0002

0.0000

Boxplot of Overall GPA - Denied

MenWomen

4.0

3.5

3.0

2.5

2.0

1.5

Overa

ll GPA

Boxplot of Men's and Women's Overall GPADenied Students

Boxplot of SAT Total Scores - Denied

MenWomen

1500

1250

1000

750

500

SAT

Tota

l Sco

re

Boxplot of Men's and Women's SAT Total ScoresDenied Students

Discussion❍ An unbiased process would result in no

statistical differences in the metrics of the admitted populations.

❍ SAT/ACT are intended to be a predictor of first year college grades, not academic achievement.

❍ Research shows that high school metrics are a better predictor of first year college grades than SAT (correlation coefficient of 0.42 vs. 0.36) Adding the two together gives a correlation coefficient of 0.52.

❍ 37 studies have shown a consistent gender bias in standardized tests. One study showed a 35 point bias in favor of males on the SAT math section.

Possible Conclusions

o Only the highest ability women are encouraged and/or self-select to apply to the College of Engineering, and men with a much wider range of academic ability are encouraged and/or self-select to do so.

o Women are held to a higher standard than men with regard to their high school performance.

o The admissions counselors put more weight on test scores than high school performance in the admissions process.

Bias at Work?

o According to Sevo & Chubin, “In situations where we evaluate the professional competence of men and women, and where there is much room for interpretation, men will have significant advantage due to unconscious assumptions. Our schema for males is a better fit for professional success, and especially for high-intensity scientific and engineering careers.”

Bias at Work?

o If a policy or tradition of an institution is to require a certain level of achievement on a test that is know to disadvantage a certain group, institutional bias exists.

1 2 3 4 5 6 7 8 9 10 11 120

10

20

30

40

50

60

70

80

90

100

Retention in Engineering for Several Large and Mid-Size Institutions

Male FemaleMinority

Semester

% R

eten

tio

n

o Understand the factors that impact students’ persistence in engineering. Such information could provide:

• provide a bases to assess the impact of program/institution-level decisions aimed at attracting students to engineering as well as student retention and success.

• evaluate the influences of current classroom pedagogical practices; and modify those deemed less effective; and

• more thoughtfully develop targeted interventions aimed at retaining students who otherwise have a propensity to leave engineering;

o Improve current retention modeling methods that are used to predict engineering students’ retention in engineering.

Motivation: Why study student retention and success?

Model of Student SuccessStudent Attitudinal Success Instrument (SASI)

( Imbrie, Lin & Malyscheff 2008; Reid 2009 )

Modified Model of Student SuccessStudent Attitudinal Success Instrument (SASI)

Methods

o Psychometric properties• Internal consistency (reliability)

− Cronbach’s coefficient alpha (α > 0.80)• Spearman-Brown formula used for subfactors with

< 10 items 2

o Exploratory Factor Analysis (EFA) • Used to establish subfactor structure or verify

structure if pre-defined• SAS proc factor, promax rotation

o Confirmatory Factor Analysis (CFA)• LISREL fit indices

−χ2

−Goodness of Fit (GFI > 0.90)−Comparative Fit Index (CFI > 0.95)−RMS Error Approximation (RMSEA < 0.08 for

acceptable fit)

Methods, continued

o Normative taxonomy: cluster analysis• McDermott’s 3-stage cluster analysis

−Standard cluster analysis of mutually exclusive groups

−Combining clusters from individual groups−Review to determine if individual data points

actually fit within a different cluster−Cattell’s between cluster similarity

coefficient – rp > 0.95 excellent similarity, rp < 0.7 poor

similarity 1

• Determines the number of groups based on normalized z-scores of overall constructs

Resultso Psychometric properties

• Cronbach’s coefficient alpha values for all constructs and subfactors > 0.80

− Spearman-Brown formula used to extrapolate subfactors to 10 items

− Exceptions: – Self-worth construct (0.69, 2007 cohort)– Team vs. Individual / Individual orientation subfactor

(0.74, 2006 cohort)o Exploratory Factor Analysis (EFA)

• Subfactor structure verified or defined for each construct

o Confirmatory Factor Analysis (CFA)• Subfactor structure verified for each construct• Fit indices 2,3,4 in all cases showed excellent fit*

− GFI>0.90, CFI>0.95

*RMSEA < 0.05 for excellent fit, <0.08 for acceptable fit

Results

o Normative taxonomy• 3 clusters indicated for each cohort (2004 –

2007)• 2004 – 2007 cohorts

−Visual inspection and −Values of Cattell’s between cluster

similarity coefficient again show three distinctly different clusters.

Cluster analysis results

Model of Student SuccessStudent Attitudinal Success Instrument (SASI)

( Imbrie, Lin & Malyscheff 2008; Reid 2009 )

Results: Ability to Identify At-Risk Students

Performance from New Model E’

Results: Important Factors by Different Methods

So What!!!!!

o Model results provide insight that can be used institutionally, programmatically, and individually to make informed decisions that will enhance undergraduate Engineering Education as well as provide a more personal learning experience for our students.

SAT_V SAT_M

SEM_ENGL

AVG_ENGL

SEM_MATH

AVG_MATH

SEM_SCI

AVG_SCIExpectMeta

Deep

Surface

Leader

Major

Motivation

Efficacy

TeamInd

0

0.5

1

Male (N=3852)

Female (N=823)

Institutional View2004 Cohort, 1 Year Retention

Institutional View

SAT_V SAT_M

SEM_ENGL

AVG_ENGL

SEM_MATH

AVG_MATH

SEM_SCI

AVG_SCIExpectMeta

Deep

Surface

Leader

Major

Motivation

Efficacy

TeamInd

0

0.5

1

Caucasian,Asi-Am,Other (N=4217)Underrepresented Minority (N=178)

Aggregated 2004-2006 Cohorts – 1 Year Retention

SAT_V SAT_MSEM_ENGL

AVG_ENGL

SEM_MATH

AVG_MATH

SEM_SCI

AVG_SCIExpectMeta

Deep

Surface

Leader

Major

Motivation

EfficacyTeamInd

0

0.5

1

SAT_V SAT_MSEM_ENGL

AVG_ENGL

SEM_MATH

AVG_MATH

SEM_SCI

AVG_SCIExpectMeta

Deep

Surface

Leader

Major

Motivation

EfficacyTeamInd

0

0.5

1SAT_V SAT_MSEM_ENGL

AVG_ENGL

SEM_MATH

AVG_MATH

SEM_SCI

AVG_SCIExpectMeta

Deep

Surface

Leader

Major

Motivation

EfficacyTeamInd

0

0.5

1

Male (N=1219)Female (N=289)

10 Semester Graduation

8 Semester Graduation

Institutional View

1 Year Retention

2004 Cohort, 1 Year Retention and 8, 10 Semester Graduation

Programmatic View

Individual View

Update

o Used this information for a discussion with the Admissions office staff

o For 2011 Admission process, • female applicants are up an additional

11% (Now 55% over the past 6 years)• Female admits are up 19%

o Also presented to Presidential Scholarship Committee prior to selections• Female awards up from 28 to 51%• Female yield is up 33% (Headcount of

489)

Acknowledgment

The researchers wish to acknowledge the support provided by a grant from the National Science Foundation, Division of Engineering Education and Centers (Award No. 0416113).

Discussion and Questions

Affective, Multidimensional Constructs o Motivation

• Control, challenge, curiosity, career outlook− Defined in terms of one’s pursuit of an activity for its

own sake » Pintrich & Schunk, 1996

o Metacognition• Planning, self-checking, cognitive strategy,

awareness− Strategies for planning, monitoring and modifying

one’s own cognitions.» Pintrich & DeGroot, 1990

o Propensity towards Deep and/or Surface Learning• Deep: Motive, strategy: Surface: Studying,

memorization− Propensity of a student within a learning environment

to adjust their learning style (deep or surface) to achieve the learning goal.

» Biggs, Kember and Leung, 2001

❍ Academic Self Efficacy− “Individuals’ beliefs of their competence affect

everything they do, and proposes that self-efficacy should prove to be an excellent predictor of their choice and direction of behavior. “

» Bandura, 1993− Studies have related self efficacy to retention

» Besterfield-Sacre et al., 1999; Pajares, 1996; House, et al., 1995; Bandura, 1986; Lent, Brown and Larkin, 1986

❍ Leadership• Motivation, planning, self-assessment, teammates

− The student’s self appraisal of their leadership abilities was identified as a noncognitive characteristic effecting student retention

» Tracy & Sedlacek, 1984; Hayden & Holloway, 1985; Ting, 2000

❍ Team vs. Individual Orientation• Individual, team dynamic

− Industry continues to seek graduates who can function as a team member and leader

» McMaster, 1996

Affective, Multidimensional Constructs

❍ Expectancy-Value• Community involvement, employment

opportunities, persistence, social engagement− Perception of the expectancy and value of

academic, social and employment expectancies» Wigfield & Eccles, 2000; Besterfield-Sacre

et al., 1999; Hayden & Holloway, 1985; Schaefers et al., 1997

❍ Major Decision• Certainty of decision, difficulty in decision,

personal issues, urgency of decision, independence

− Related to student success» Schaefers et al., 1997; Smith & Baker,

1987; Haislett & Hafer, 1990; Osipow, 1999

Affective, Multidimensional Constructs

Admissions Process (thru Fall ‘08)

Application arrives in ADMS data processing

Send request for more info. Code as “I”.

Yes

No Make recommendation and send to committee

Clear Admit?

Admit student to E. Code as “A”.

Send to designated ADMS counselor.

Clear Admit?

Admit student to E. Code as “A”.

Committee meets and agrees

Admit student to E. Code as “A”.

Admit student to 2nd choice. Code as “A”.

Hold decision. Code student as “E” or “P”

Yes

Yes

No

No

Deny admission to student. Code as “D”

App complete?