Replicating Physics Education Reforms:

How (and why) to keep a good thing going

Steven PollockNoah FinkelsteinPhysics Department

University of Colorado, Boulder

APSApr MeetingApril 2006

AcknowledgementsPhysics faculty:Physics faculty:

Noah FinkelsteinNoah FinkelsteinCarl WiemanCarl WiemanKathy PerkinsKathy PerkinsMichael DubsonMichael Dubson

Postdocs: Postdocs: Sam McKaganSam McKaganLinda Koch Linda Koch

Ph. D. students:Wendy AdamsJack BarberaKara GrayChris KellerPat KohlNoah PodolefskyChandra Turpen

School of Ed collaborators:Valerie OteroDerya CobanogluDanielle Harlow

QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.

QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.

Replicating successful curricula

What does it mean to replicate? • What is necessary, sufficient?

• What theoretical frameworks help us understand?

What does it mean to sustain?

CU Model of Course Transformation

Building from traditional courses• Bottom up reform• Peer instruction, Tutorials in

Introductory Physics• Learning Assistants

<g> = post-pre 100-preCU Fa03/

Sp04CU Fa98

red = trad, blue = interactive engagement

Force Concept Inventory

R. Hake, ”…A six-thousand-student survey…” AJP 66, 64-74 (‘98).

Departmental level

A “frames of context” perspective

Course culture (Physics I)

Situation (Tutorial)

Task (2-D drawing)

ConceptStudent

Finkelstein and Pollock, (2005). Physical Review: ST PER, 1,1.

Tasks

• Tutorials in Introductory Physics (UW)– Reconceptualize Recitation Section

– well documented, – easy to implement

Task

Proven CurriculaD.E. Trowbridge and L. C. McDermott, "Investigation of student understanding of

the concept of acceleration in one dimension," Am. J. Phys. 49 (3), 242 (1981).

D.E. Trowbridge and L. C. McDermott, "Investigation of student understanding of the concept of velocity in one dimension," Am. J. Phys. 48 (12), 1020 (1980)

R.A. Lawson and L.C. McDermott, "Student understanding of the work-energy and impulse-momentum theorems," Am. J. Phys. 55 (9), 811 (1987)

L.C. McDermott and P.S. Shaffer, "Research as a guide for curriculum development: An example from introductory electricity, Part I: Investigation of student understanding." Am. J. Phys. 60 (11), 994 (1992); Erratum to Part I, Am. J. Phys. 61 (1), 81 (1993).

P.S. Shaffer and L.C. McDermott, "Research as a guide for curriculum development: An example from introductory electricity, Part II: Design of instructional strategies." Am. J. Phys. 60 (11), 1003 (1992)

L.C.McDermott, P.S. Shaffer and M. Somers, "Research as a guide for curriculum development: An illustration in the context of the Atwood's machine," Am. J. Phys.62 (1) 46-55 (1994).

M.E. Loverude, C.H. Kautz, and P.R.L. Heron, ﾊ "Student understanding of the first law of thermodynamics”, Am.J. Phys. 70 (2) 137 (2002)

www.phys.washington.edu/groups/peg/pubsa.html

Task dataTOPIC PRE POST

(click)

POST (exam)

Direction of a at top of ramp

20

Direction of a on curved path

10

Task dataTOPIC PRE POST

(click)

POST (exam)

Direction of a at top of ramp

20 70 75

Direction of a on curved path

10 70 65

Tasks

Task alone insufficient (!)

Task

Situations

• Interactions with other students/TA’s

• Room/environment

Situat.

Classroom: Trad -- Tutorial

Situations

• Interactions with other students/TA’s• Room/environment• Grading norms

Situat.

Finkelstein and Pollock, (2005). Phys Rev ST PER, 1,1.010101

Trowbridge and McDermott," Am. J. Phys. 49 (3), 242 (1981).

0

10

20

30

40

50

60

70

80

90

100

Newton &constraints

Force diagrams Newton III CombineNewton's Laws

UW - No Tut UW - with Tut CU - with Tut

Impact and Reproducibility

Class Culture

• Critical, subtle!

• Ideas constructed by students

• Answers are justified and reasoned

• Tutorials connected to class (and world/your life)

• Enculturation over time

Culture

Broader layer(s)

• Support of institution

• $$$

• TA training

• Reformed classes

Depart.

What does replication mean?

• More than repeating activities/worksheets

• Appropriate activity, surrounding structures: – tasks, situations, norms, practices,

infrastructure…• New env’t => Adopt, adapt, reinvent

(Henderson)

intra-institutional replication

0

2

4

6

8

10

12

14

16

18

20

0 6 12 18 24 30 36 42 48 55 61 67 73 79 85 91 97Score

% of students

PreF04 PreS05

1120 BEMA pre/post

F04 (N=319) Pretest: 26% S05 (N=232): 27%

0

2

4

6

8

10

12

14

16

18

20

0 6 12 18 24 30 36 42 48 55 61 67 73 79 85 91 97Score

% of students

PreF04 PreS05 PostF04 PostS05

1120 BEMA pre/post

g(F04) = .44+/- .01 g(S05)=.43+/- .01

F04 (N=319) Post: 59% S05 (N=232): 59%

1120 BEMA LA’s

LA (pre)

LA (post)

TA (pre)

LA2 (post)TA

(post)

CU upper division (trad)

Replication? yes…

instructor effects?

Handoff to non-PER faculty

… everything looks the same…(except the instructor)

Hand-Off Results

g(F04) = .44+/- .01 g(S05)=.43+/- .01g(F05)=.33+/- .02

Post 50

Fall 2005:Pre: 25

Recitation (curriculum) effects?

The impact of recitation/pedagogy

Physics 1, 300+ students,

using Peer Instruction in lecture, and:

1: “Tutorials” (Sp04) Tutorials

2: “Workbook” (Fa04) Knight Workbook

3: “Traditional” (Sp05) Mostly traditional

Phys 1110 normalized gains

0

5

10

15

20

25

30

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

normalized gain

% of students

Tutorials

gain <g>= .66 +/-.02

0

5

10

15

20

25

30

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

normalized gain

% of students

Phys 1110 normalized gains

TutorialsWorkbooks

gain <g>= .66 +/-.02= .59 +/-.02

0

5

10

15

20

25

30

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

normalized gain

% of students

Phys 1110 normalized gains

TutorialsWorkbooks

Traditional

gain <g>= .66 +/-.02= .59 +/-.02= .45 +/-.02

Beyond the FMCE: Exam comparisons

0

10

20

30

40

50

60

70

80

Tut-related q's Trad/quant q's

Exam scores (%)

Tutorial course Workbook course Trad recitation

-7 -6-14 -10

N.B. 12 points is roughly 1 letter grade.

Beyond the FMCE: Exam comparisons

0

10

20

30

40

50

60

70

80

Tut-related q's Trad/quant q's

Exam scores (%)

Tutorial course Workbook course Trad recitation

-7 -6-14 -10

N.B. 12 points is roughly 1 letter grade.

Impact on different pretest populations:"high starters" 50<pre<93%

normalized gain for high pretest

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

S04 (13%) F04 (22%) S05 (14%)

Semester (% of class in this

g

pretest >50%

(% of class in this pool)

Course (1) (2) (3)

Tut course Trad Recit

Impact on different pretest populations:"low starters" pretest <=12.5%

normalized gain for low pretest

0

0.1

0.2

0.3

0.4

0.5

0.6

S04 (23%) F04 (20%) S05 (22%)

Semester (% of class in this

g

pretest <= 12.5%

Course (1) (2) (3)

(% of class in this pool)

Tut course Trad Recit

Ongoing studies

Currently:

4 non PER faculty running Tutorials - data at AAPT/PERC

Conclusions

• Replicating & Sustaining reforms is possible– theoretical framing helpful– curriculum matters– context matters

QUESTIONS Please!

per.colorado.edu (for this talk and more)

fin

A&B and learning gainsData from instructor attending (somewhat) to “hidden curriculum”)

Conceptual Understanding

35

45

55

65

75

g<=.25 0.25<g<=0.5 0.5<g<=0.75 0.75<g<=0.9 0.9<g<=1

Learning GainsLow learning gain <---------> high learning gain

Beliefs (pre / post) vs. learning gain

Adams et al. (2006) Physical Review, ST:PER, 010101Perkins et al (to appear). Proceedings of PERC

Connecting Tutorials to course as whole

0

20

40

60

80

100

0 20 40 60 80 100

Tutorial homework (%)

Course score (%)

Rem

aini

ng