Assessing Gender Differences in Students' Understanding of Magnetism using Magnetism

Conceptual Survey

Jing LiDepartment of Physics and Astronomy

University of PittsburghCo-Author: Chandralekha Singh

• Several prior studies on gender effects in college introductory physics– K. Heller, APS April meeting Talk, Denver, CO, (2009).– M. Lorenzo, C. Crouch and E. Mazur, Am. J. Phys. 74(2), 118, (2006).– S. Pollock, N. Finkelstein and L. Kost, Phys. Rev. ST PER 3, 010107 (2007). – L. Kost et al., Phys. Rev. ST PER 5, 010101 (2009).– P. Kohl, H. Kuo, PERC Proceedings. 1179, 173 (2009).– L. Kost et al., PERC Proceedings, 1179, 177 (2009).– L. Kost et al., PhysRev: ST Phys Ed. Rsrch 6 (2), 020112 (2010) .

• Generally observed that in traditionally taught courses– Males outperform females on FCI/BEMA– Even if in some courses the final exam grades on traditional exams comparable (not

consistent across various studies)

• Especially designed courses (e.g., at Harvard course developed by the Mazur group) can reduce the gender difference

• Values affirmation and Self-efficacy issues and accumulation of bias of various kinds over time may play a role in gender differences

– Miyake et al. Science 330, 1234 (2010).

Background

• Difficult to pinpoint one reason for gender differences between males and females since issue is very complex

• Also, remedies suggested by various groups are different– For example, Z. Hazari, G. Sonnert & P. Sadler (AAPT meeing, Ann Arbor, 2009)

claim that presence of women role models is unlikely to bridge the gap between male and female performance because female students are unlikely to identify with the role models.

– Also see, Z. Hazari, R. Tai and P. Sadler, Science Education, 847, 2007.– Hazari and G. Potvin, E. Journal of Science Education, 10(1), 33, 1-33, 2005.– Hazari, P. sadler and R. Tai, Phys Teach, 46, 423, 2008.– They suggest that female students ‘ identity as a scientist and self-efficacy issues

are at the heart of gender difference in science/math

• Others emphasize the importance of role models– For example, S. Yennello (AAPT Meeting, Ann Arbor, 2009)

•

Background

• Here we investigate gender effects in traditionally taught introductory algebra-based and calculus-based introductory physics on magnetism concepts using Magnetism Conceptual Survey (MCS) we developed by administering it as a pre-test and post-test to large classes

• Many questions in magnetism require the ability to visualize in three dimensions (force on moving charges or current carrying wires, magnetic field produced by current carrying wires etc.)

• Some research suggests that females generally have a better verbal ability but worse spatial ability than males which can restrict their reasoning in 3D

– often there is a correlation between students’ spatial ability and their self-confidence

– D. Law, J. Pellegrino and E. Hunt, Psychological Science, 4(1), 35 (1993).– 12. M. Beth Casey, R. Nuttall and E. Pezaris, Journal for Research in Mathematics

Education, 32(1), 28 (2001).

•

Background

• Advantage of multiple-choice test– Objective, easy to grade, results for different groups can be compared

• Disadvantage is that thought process is not revealed by test answers alone

– However, research-based multiple-choice tests in which distracter choices are designed with input from students can be very effective

• After discussion of the development of Magnetism Conceptual Survey

– Gender effects– before and after instruction– Different types of classes such as algebra-based, calc-based, honors

students

Development of the MCS

• we developed a 30 item multiple-choice test on magnetism (up to Faraday’s law) to explore the difficulties students have in– interpreting these concepts– Correctly identifying & applying concepts in different situations

• To design good distractor choices, during test development students were

– given free-response questions asking them for reasoning for their responses – interviewed individually

• The magnetism test was developed by consulting with faculty members who teach introductory physics • What students should definitely be able to do– Three faculty members went over several versions of the test.– Several introductory students were asked to answer the test questions individually in

interviews.

Development of the MCS

• The test was administered as both a pre- and post-test to a large number of students at University of Pittsburgh. – three algebra-based classes and eight calculus-based classes. – Students were taking a traditional algebra-based or calculus-based E&M

course.

• Pre-tests were administered in the first lecture or recitation at the beginning of each semester.

• Post-tests were administered in the recitations after instructors finished all concepts on MCS– Students were asked to work on the test in one class period (40-50

minutes).

• We keep only the students who took both the pre- and post-test except those in one algebra class in which most students didn’t provide their names.– 267 algebra-based students took the pre-test, and 273 students took the

post-test. – 575 calculus-based students took both the pre- and post-tests.

Administration

Reliability of MCS

• Reliability is a measure of the self consistency of a whole test. It is high if students are not guessing

• KR-20 for MCS 0.83 for introductory physics– Good from standards of test design

• MCS administered to 42 graduate student to benchmark performance– KR-20 is 0.87– Average score is 83%

• A measure of consistency of a single test item with the whole test.

• The desired value should be greater than 0.2.• Point biserial coefficient for MCS is 0.42.

Point Biserial Coefficient

• A measure of the difficulty of a single test question. It is calculated by taking the ratio of the number correct responses on the question to the total number of students who attempted the question.

• The desired value is over 0.3. Average for MCS is 0.46.

Item Difficulty

• A measure of the discriminatory power of each item in a test.• Can be calculated as

• The desired value should be greater than 0.3.• Average item discrimination for MCS is 0.33.

Item Discrimination Index

2/N

NND LH

• For algebra-based course, there is no significant difference between men and women on the pre-test and post-test.

Assessing Gender Difference in the Performance in Algebra-based Courses

Algebra based Pre-test

Group Number Mean S.D. Sig.(2-tailed)

Men 91 7.3 2.51 0.940

Women 106 7.1 2.36

Algebra based Post-test

Group Number Mean S.D. Sig.(2-tailed)

Men 110 13.2 5.20 0.360

Women 121 12.3 5.57

• For calculus-based course, men and women don’t have significant different on the pre-test while men outperformed women on the post-test.

Assessing Gender Difference in the Performance in Calculus-based Courses

Calculus based Pre-test

Group Number Mean S.D. Sig.(2-tailed)

Men 403 8.5 3.40 0.490

Women 168 7.8 3.05

Calculus based Post-test

Group Number Mean S.D. Sig.(2-tailed)

Men 403 15.3 6.20 0.019

Women 168 13.0 5.38

Assessing Gender Difference in the Performance of the Honors Students

• For honour students, women do not perform as well on the post-test.

Honored students

Group Number Mean S.D. Sig.(2-tailed)

Men 75 17.4 5.89 0.030

Women 20 14.1 6.21

Gender Performance on individual Questions

• In algebra-based classes, males outperformed females on 20 questions• 4 questions have difference more than 10%.• Females outperformed males more than 10% only on

1 question.

• In calculus-based classes, males outperformed females on 28 questions • 9 questions have difference more than 10%. • Female s only performed slightly better on the other

two questions.

Possible Explanations on Gender Difference

• Other research using FCI/BEMA have shown gender difference in traditional introductory physics

• In addition, to societal factors and accumulated bias discussed in earlier studies

• In magnetism, most questions require ability to visualize in 3D– figure out the directions of magnetic field Lorentz force.

• Some prior research suggests that females generally worse spatial ability than males– restrict their reasoning in 3D – there is a correlation between students’ spatial ability and their self-confidence

• Pre-test were given at the beginning of the course – There is a statistical difference

• Post-test were given at the end of the course– Statistically significant difference

• Calculus-based courses do better than algebra-based courses

Algebra-based vs. Calculus-based course

Group Number Mean S.D. Sig. (2-tailed)

algebra 267 7.2 2.48 0.000

calculus 575 8.3 3.31

Group Number Mean S.D. Sig. (2-tailed)

algebra 273 12.2 5.34 0.010

calculus 575 14.6 6.05

Why should there be a difference between algebra-based and calc-based students when conceptual questions?

Possible explanations

• Calc-based students have better Math skills and Scientific reasoning skills

• Calc-based students are less likely to have cognitive overload while learning• Calc-based students are more likely to build a robust knowledge structure

Algebra-based vs. Calculus-based course

group Number

Mean S.D. Sig. (2-tailed)

Alge 108 7.43 2.36 0.042

Calc 284 8.24 3.89

group Number

Mean S.D. Sig. (2-tailed)

Alge 213 13.45 5.34 0.000

Calc 241 15.87 6.31