Two Stage Exams: Learning Together?

Joss IvesDepartment of Physics & Astronomy and Vantage College,

University of British Columbia - Vancouver

Brett GilleyDepartment of Earth, Ocean and Atmospheric Sciences and Vantage College,

University of British Columbia - Vancouver

What do you notice?

from http://www.cwsei.ubc.ca/resources/SEI_video.html

Define two-stage exams

Discuss benefits and

drawbacks

A brief review of the

literature

How frequently do you use group work in your

courses?

On a scale of

• 0: No group work

• 5: I only interrupt group work to make

announcements

Have you used two stage exams or cooperative

exams?

Define two-stage exams

Discuss benefits and

drawbacks

A brief review of the

literature

Solo Phase

Students complete, then hand in the individual

exam

• Approximately 2/3rds of

the total exam time is

spent on the solo phase

* Our implementation

Group Phase

Then they get into groups of 3-4 to work on the

group exam

• A mix of group formation

styles at UBC-V: instructor

vs. student formed

• Group exam is nearly

identical to the individual

exam

• Groups get only one exam

sheet and must come to

consensus

* Our implementation

Group exams are in widespread use at UBC-V

Over 100 courses

• Physics, Chemistry, Biology, Math, Statistics, Earth, Ocean,

and Atmospheric Sciences, Computer Science, Forestry,

Pharmacy, Psychology, and Land and Food Systems, Poli

Sci,

Many class formats

• 450 student 1st year lectures

• 20 person laboratories

• 20 student 4th year seminars

• 5 student graduate classes

Define two-stage exams

Discuss benefits and

drawbacks

A brief review of the

literature

What are some possible benefits of group exams?

Student or instructor (Discuss with your neighbors)

• Simple mistakes are

cleared up in the moment

• Immediate feedback

• Explanations coming from

expert novices

• Learning through

articulation of ideas

• Increases problem solving

abilities and group work

skills

• Students like them

• Learning how to learn

• Stress reduction

• Get a good read on how

they did individually

• In-context language

practice

More benefits…

• Learning

• Articulating ideas deepens

understanding

• Building confidence

• Community belonging

• Diversity of ideas and

approaches

• Developing soft skills

• Immediate feedback

(right/wrong and learn

from mistakes)

• Reducing anxiety

• Solving very challenging

problems

• Reinforcing instructional

practices

• Boosting grades

• Intense discussions of your

favorite discipline

• Leaving the exam room on

a high note

Student surveys tell us they really like the format

Positive opinions Negative opinions

Rieger 2014 (Physics)

71% 10%

Fengler 2015(Mech Engineering)“prefer 2-stage”

76% 15%(all mild, no strong disagreement)

Rivas 2016(Nursing)

98%

Positive feedback from EOSC 114

Discussion 48

Learn why you were wrong 37

New Perspectives 29

Better grades 21

Instant feedback 16

Review 10

Build confidence 8

Understand questions better 6

Learn techniques from others 6

Other 27

Total 208

Negative feedback from EOSC 114

Coming to consensus 21

Time consuming 13

Unbalanced knowledge in group

6

Convinced of wrong answer 3

Realize did poorly individually 3

Worth too much 2

Other 8

Total 56

What are some possible drawbacks of group

exams? (Discuss with your neighbor)

• Being convinced of the

wrong answer

• Challenging to have

group-only questions that

require a lot of up-front

deep thinking

•

•

More drawbacks…

• Student anxiety surrounding

group work

• Differences in

communication skills (EAL)

• Dominating personalities

• Dead wood

• Being convinced of the

incorrect answer

• Students learning of their

own poor solo

performance

• Grade inflation

• Extra time

Define two-stage exams

Discuss benefits and

drawbacks

A brief review of the

literature

Group Questions

Individual Questions

All Questions(n=1008)

57.3 1.6%Correct(n=578)

97.8 0.6%Correct(n=565)

2.2 0.6%Inorrect(n=13)

42.7 1.6%Incorrect(n=430)

73.0 2.1%Correct(n=314)

27.0 2.1%Incorrect(n=116)

Correct individuals are infrequently lead astray

Data from Ives N = 37 (2011) Introductory E&M course

Groups outperform individuals and all performance

levels learn from the group

Reproduced from Jang 2017

0

10

20

30

40

50

60

70

Gro

up

qu

esti

on

co

rrec

t af

ter

all

ind

ivid

ual

s w

ere

inco

rrec

t (%

)

Groups composed of only incorrect individuals show good

evidence that learning takes place during the exam

Ives Singh Ives Gilley

[Singh 2005] [Gilley 2012]

32.9 ± 9.3%(𝑁 = 25)

29 ± 2%(𝑁 = 568)

50.0 ± 9.1%(𝑁 = 30)

56.4 ± 6.7%(𝑁 = 55)

The majority of studies point to evidence of learning in the

short-term…

Short-term (0-4 weeks)

N DisciplineEffect size(Cohen’s d)

p-value Notes

Gilley 2014 same 79Earth Science 0.54 0.0005Nearly 10% difference in scores

Gilley 2014 same 71Earth Science 0.39 0.0132

Rivaz 2016 same 84Nursing 1.66 0.001Treatment scored twice as high as control

Ives 2014near-transfer

679Physics 0.11 0.012

Ives 2015near-transfer

703Physics 0.2 0.001

Cao 2017 Expt 1

near-transfer

209Computer Science

0.26 0.003

Cao 2017 Expt 2

near-transfer

209Computer Science

0.19 0.026

Vogler 2016near-transfer

50, 37

Education Phsychology

No effect Two different years

Leight 2012near-transfer

200Biology No effect Two different tests

The majority of studies point to evidence of learning in the

short-term, but less so in the long-term

Long-term(> 4 weeks)

N DisciplineEffect size(Cohen’s d)

p-value Notes

Vogler 2016 same 50Education Phsychology

0.43 0.0412.5% better scores for treatement

Vogler 2016 same 37Education Phsychology

0.49 0.04

Ives 2014near-transfer

673Physics No effect

Ives 2015near-transfer

703Physics -0.11 0.012

Cao 2017near-transfer

209Computer Science

No effect Final exam

Fournier 2017

near-transfer

444Human Anatomy

No effect

There is evidence that the learning benefits are seen for all

performance-levels

Reproduced from Gilley 2014

In closing

Which barriers exist for you?

(Discuss with your neighbor)

• •

Group exam are the highest level of student engagement

that we have observed in any of our courses

VS.

Even the shyest students

participate

Biblography (Page 1)

[Azmitia 1993] Azmitia, M., & Montgomery, R. (1993).

Friendship, transactive dialogues, and the development of

scientific reasoning. Social Development, 2(3), 202–221.

http://doi.org/10.1111/j.1467-9507.1993.tb00014.x

[Cao 2017] Cao, Y., & Porter, L. (2017). Evaluating Student

Learning from Collaborative Group Tests in Introductory

Computing. (Accepted) SIGCSE 2017 - Proceedings of the 48th

ACM Technical Symposium on Computer Science Education.

[Dasgupta 2015] Dasgupta, N., McManus Scircle, M., &

Hunsinger, M. (2015). Female peers in small work groups

enhance women’s motivation, verbal participation, and career

aspirations in engineering. Proceedings of the National

Academy of Sciences, 112(16), 4988–4993.

http://doi.org/10.1073/pnas.1422822112

[Drouin 2010] Drouin, M. A., “Group–based formative

summative assessment relates to improved student performance

and satisfaction,” Teaching of Psychology, 37(2), 2010, 114–

118.

[Eddy 2015] Eddy, S. L., Brownell, S. E., Thummaphan, P., Lan,

M. C., & Wenderoth, M. P. (2015). Caution, student experience

may vary: Social identities impact a student’s experience in peer

discussions. CBE Life Sciences Education, 14(4), 1–17.

http://doi.org/10.1187/cbe.15-05-0108

[Epstein 2002] Epstein, M. L., Lazarus, A. D., Calvano, T. B., &

Matthews, K. A. (2002). Immediate feedback assessment

technique promotes learning and corrects inaccurate first

responses. The Psychological Record, 52(2), 187.

[Fengler 2015] Fengler, M., & Ostafichuk, P. M. (2015).

Successes with Two-Stage Exams in Mechanical Engineering.

2015 Canadian Engineering Education Association (CEEA15)

Conf., 1–5.

[Fournier 2017] Fournier, K. A., Couret, J., Ramsay, J. B., &

Caulkins, J. L. (2017). Using collaborative two-stage

examinations to address test anxiety in a large enrollment

gateway course. Anatomical Sciences Education, 1.

http://doi.org/10.1002/ase.1677

[Gilley 2014] Gilley, B., & Clarkston, B. (2014). Collaborative

Testing: Evidence of Learning in a Controlled In-Class Study of

Undergraduate Students. Journal of College Science Teaching,

43(3), 83–92.

[Heller 2092] Heller, P., & Hollabaugh, M. (1992). Teaching

problem solving through cooperative grouping. Part 2:

Designing problems and structuring groups. American Journal

of Physics.

[Ives 2014] Ives, J. (2014). Measuring the Learning from Two-

Stage Collaborative Group Exams, PERC Proceedings 2014.

http://doi.org/10.1119/perc.2014.pr.027

[Ives 2015] Ives, J. (2015). Further Investigation into the

Effectiveness of Collaborative Group Exams, Poster presented at

Physics Education Research Conference 2015.

[Ives 2016] Ives, J., van Lier, M., Sumah, N., Stang, J. (2016).

Examining Student Participation in Two-Phase Collaborative

Exams through Video Analysis, PERC 2016 Proceedings (arXiv:

1607.03960)

Biblography (Page 2)

[Jang 2017] Jang, H., Lasry, N., Miller, K., & Mazur, E. (2017).

Collaborative exams: Cheating? Or learning? American

Journal of Physics. http://doi.org/10.1119/1.4974744

[Leight 2012] Leight, H., Saunders, C., Calkins, R., & Withers,

M. (2012). Collaborative testing improves performance but not

content retention in a large-enrollment introductory biology

class. CBE Life Sciences Education, 11(4), 392–401.

http://doi.org/10.1187/cbe.12-04-0048

[Lou 1996] Lou, Y., Abrami, P. C., Spence, J. C., Poulsen, C.,

Chambers, B., Apollonia, S., … Apollonia, S. (2016). Within-

Class Grouping : A Meta-Analysis, 66(4), 423–458.

[Macpherson 2011] G. L. Macpherson, Y. J. Lee, and D.

Steeples. Group-examination improves learning for low-

achieving students. Journal of Geoscience Education, 59(1):41,

2011

[Rieger 2014] Rieger, G. W., & Heiner, C. E. (2014).

Examinations that support collaborative learning: The students’

perspective, 1–16.

[Rivaz 2016] Rivaz, M., Momennasab, M., & Shokrollahi, P.

(2015). Effect of collaborative testing on learning and retention

of course content in nursing students. Journal of Advances in

Medical Education & Professionalism, 3(4), 178–82.

[Singh 2005] Singh, C. (2005). Impact of peer interaction on

conceptual test performance. American Journal of Physics,

73(5), 446. http://doi.org/10.1119/1.1858450

Terry 2017] Mccurdy, T. T. R., Volterman, K., & Shiell, R. (2017).

Enhancing Two-stage Collaborative Exams by Incorporating

Immediate Feedback Enhancing Two-stage Collaborative Exams

by Incorporating Immediate, 1.

[Vogler 2016] Vogler JS, Robinson DH. Team-Based Testing

Improves Individual Learning. The Journal of Experimental

Education. 2016 Oct 1;84(4):787-803.

[Webb 1991] Webb, N. M. (1991). Task-related verbal

interaction and mathematics learning in small groups. Journal

for Research in Mathematics Education, 22(5), 366–389.

http://doi.org/10.2307/749186

[Yuretich 2001] R. F. Yuretich, S. a. Khan, R. M. Leckie, and J. J.

Clement. Active-learning methods to improve student

performance and scientific interest in a large introductory

oceanography course. Journal of Geoscience Education,

49:111–119, 2001.

[Zimbardo 2003] Zimbardo, P. G., Butler, L. D., & Wolfe, V. a.

(2003). Cooperative College Examinations: More Gain, Less

Pain When Students Share Information and Grades. The Journal

of Experimental Education, 71(2), 101–125.

http://doi.org/10.1080/00220970309602059

[Zipp 2007] Zipp, J. (2007). Learning by exams: The impact of

two-stage cooperative tests. Teaching Sociology. Retrieved from

http://tso.sagepub.com/content/35/1/62.short