· web viewtype of active learning treatment, categorized as cases (case studies completed in...
TRANSCRIPT
Table S4 : Catalog of studies analyzedtheir characteristics and effect size data
a. Exam/concept inventory data
(Ref#)Citation: Study
Disci-pline
Assess Type
Size
Studnt
Instr
Maj
Level
Trtmnt name
Sign?
OrigStN
Clust
AdjN
Hedgessg
S.E.
Lower limit
Upper limit
Z-value
p-value
(S53)Anderson et al. 20052
Chem
Exam
Sm
QRno
yes
Maj
Upper
PBL
Y
420
9
1.334
0.609
0.140
2.529
2.189
0.029
(S54)Armbruster et al. 20091
Bio
Exam
Lg
QRno
yes
Maj
Intro
Wrksht, Clicker
Y
520
13
0.186
0.540
-0.872
1.244
0.345
0.730
(S55)Armstrong & Hendricks 1999
Math
(grade)
na
QRacad
yes
Maj
Intro
Wrkshts
N
14
na
0.322
0.655
-0.961
1.605
0.491
0.623
(S56)Armstrong et al. 2007
Bio
Exam
Lg
QRpre
yes
Non
Intro
Clicker, Wrksht, Quiz
N
614
9
-0.090
0.593
-1.252
1.073
-0.151
0.880
(S57)Asiala et al. 1997a2
Math
Exam
na
QRacad
no
Maj
Intro
Wrksht
na
41
8
0.817
0.633
-0.423
2.058
1.291
0.197
(S58)Asiala et al. 1997b
Math
Exam
na
QRacad
no
Maj
na
Wrksht
na
37
8
1.143
0.645
-0.121
2.406
1.772
0.076
(S59)Austin 1995
Math
Exam
na
QRacad
yes
Maj
Upper
Wrksht
N
57
8
0.070
0.613
-1.130
1.271
0.115
0.908
(S60)Banerjee &Vidyapati 19972
Chem
Exam
Med
QRpre
yes
Maj
Intro
Wrksht
N
68
8
-0.099
0.606
-1.287
1.089
-0.164
0.870
(S61)Bardar& Brecher 2008
Phys
CI
na
Rand
yes
Non
Intro
Clicker, Wrksht
na
1,692
145
0.188
0.174
-0.153
0.529
1.080
0.280
(S62)Barg et al. 2000
CompSci
Exam
na
QRacad
no
Maj
Intro
PBL
N
66
8
0.058
0.607
-1.131
1.248
0.096
0.924
(S63)Barnard 19422
Bio
Exam
Sm
QRpre
yes
Non
Intro
Clicker, Wrksht
Y
63
8
0.097
0.609
-1.096
1.290
0.159
0.874
(S64)Basili& Sanford 1991
Chem
Exam
Sm
QRpre
yes
Maj
Intro
Wrksht
Y
49
8
1.217
0.631
-0.019
2.454
1.930
0.054
(S65)Beck &Chzhik 2008
CompSci
Exam
Sm
QRacad
no
Maj
Intro
Wksht
Y
64
8
0.720
0.611
-0.478
1.918
1.178
0.239
(S49)Beichner et al. 19991
Eng
CI
Sm
QRno
yes
Maj
Intro
Studio
Y
142
13
3.722
0.650
22.448
4.722
5.728
0.000
(S51)Beichner et al. 2007
Phys
CI
Med
QRpre
no
Maj
Intro
Studio
Y
19
NA
3.394
0.678
2.066
4.722
5.010
0.000
(S66)Bilgen 2006
Chem
Exam
Sm
QRpre
yes
Maj
Intro
Wrksht
Y
81
8
1.220
0.609
0.026
2.414
2.002
0.045
(S67)Bilgin et al. 20092
Chem
Exam
Sm
QRpre
yes
Maj
Intro
PBL
Y
78
8
0.351
0.603
-0.831
1.533
0.582
0.560
(Ref#)Citation: Study
Disci-pline
Assess Type
Size
Studnt
Instr
Maj
Level
Trtmnt name
Sign?
OrigStN
Clust
AdjN
Hedgessg
S.E.
Lower limit
Upper limit
Z-value
p-value
(S68)Blue 1997
Phys
CI
na
QRacad
no
na
na
Wrksht
na
34
7
0.975
0.628
-0.256
2.206
1.552
0.121
(S69)Bookman & Friedman 19942
Math
Exam
na
QRacad
yes
Maj
Intro
Wrksht
Y
605
9
0.413
0.598
-0.758
1.585
0.691
0.489
(S70)Booth & James 2001
Phys
CI
na
QRno
yes
Maj
Intro
Wrksht
N
60
8
-0.475
0.612
-1.675
0.725
-0.776
0.438
(S71)Born et al. 19722
Psych
Exam
Med
QRacad
yes
Maj
Upper
Wrksht, Quiz
Y
19
na
0.853
0.673
-0.467
2.173
1.267
0.205
(S72)Boyles et al. 19941,2
Psych
Exam
Sm
QRacad
no
Maj
intro
Wrksht
Y
69
20
0.499
0.542
-0.564
1.562
0.920
0.357
(S73)Bradley et al. 2002
Chem
(grade)
na
QRacad
yes
Maj
Upper
Wrksht
N
120
9
-0.017
0.607
-1.206
1.172
-0.028
0.978
(S74)Brown et al. 19723
Math
Exam
na
QRno
yes
Non
Intro
Quiz
N
37
8
0.086
0.630
-1.148
1.320
0.137
0.891
(S75)Buck & Wage 2005
Eng
CI
na
QRpre
no
Maj
Upper
Quiz, Wrksht
Y
473
9
0.716
0.597
-0.453
1.886
1.201
0.230
(S76)Bullard et al. 20082,3
Eng
Exam
Med
QRacad
no
Maj
Upper
Wrksht
Y
246
17
0.127
0.460
-0.774
1.029
0.276
0.782
(S77)Bunting &Cheville 2009
Eng
Exam
Sm
QRno
yes
Maj
Upper
Wrksht
Y
97
8
0.701
0.602
-0.479
1.881
1.164
0.244
(S78)Burrowes 20032
Bio
Exam
Lg
QRno
yes
Maj
Intro
Wrksht, Quiz
Y
200
9
0.302
0.600
-0.874
1.479
0.503
0.615
(S79)Cahyadi 20042
Phys
CI
Med
QRpre
no
Maj
Intro
Wrksht
Y
93
9
0.568
0.606
-0.620
1.756
0.937
0.349
(80)Carmichael 20092
Bio
Exam
Lg
QRpre
yes
na
Intro
Wksht
Y
336
9
0.178
0.596
-0.990
1.346
0.298
0.765
(S81)Chaplin 2009
Bio
Exam
Med
QRacad
yes
Maj
Intro
Cases
Y
95
8
0.139
0.598
-1.033
1.311
0.233
0.816
(S82)Cheng et al. 20043
Phys
CI
Med
QRpre
yes
Maj
Intro
Clickers
Y
419
18
0.271
0.455
-0.620
1.163
0.596
0.551
(S83)Christensen 2005
Phys
Exam
Med
Rand
yes
Non
Intro
Wksht
na
15
na
1.028
0.530
-0.012
2.067
1.938
0.053
(S50)Crider 20052
Phys
Exam
Sm
Rand
yes
na
Intro
Wrksht, Quiz
Y
106
na
2.831
0.358
2.128
3.533
7.898
0.000
(S84)Crouch & Mazur 2001
Phys
Exam
Lg
QRpre
no
Maj
Intro
Clickers
Y
333
9
0.445
0.596
-0.723
1.614
0.747
0.455
(S52)Cummings et al. 19992
Phys
CI
Sm
QRpre
no
Maj
Intro
IntDem
Y
12
na
2.882
0.993
0.936
4.827
2.903
0.004
(S85)Davis &Hult 19972
Psych
Exam
Med
Rand
yes
Maj
intro
Wrksht
Y
56
na
0.579
0.273
0.044
1.113
2.123
0.034
(Ref#)Citation: Study
Disci-pline
Assess Type
Size
Studnt
Instr
Maj
Level
Trtmnt name
Sign?
OrigStN
Clust
AdjN
Hedgessg
S.E.
Lower limit
Upper limit
Z-value
p-value
(S86)Day & Foley 2006
CompSci
(grade)
Sm
QRacad
yes
Maj
Upper
Wrksht
Y
46
8
1.141
0.634
-0.102
2.384
1.799
0.072
(S87)Dees 1991: Algebra
Math
Exam
na
Rand
yes
Non
Intro
Wrksht
Y
76
na
0.547
0.232
0.093
1.000
2.361
0.018
(S87)Dees 1991: Geometry
Math
Exam
na
Rand
yes
Non
Intro
Wrksht
Y
63
na
0.380
0.251
-0.112
0.873
1.513
0.130
(S88)DeMetry&Groccia 1997
Eng
Exam
Med
QRno
yes
Maj
Upper
Wrksht
N
308
9
0.142
0.597
-1.028
1.312
0.238
0.812
(S89)Dennis 20012
Math
Exam
na
QRpre
no
Non
intro
Wkrsht
Y
74
8
0.277
0.604
-0.907
1.462
0.459
0.646
(S90)Dollman 2005
Bio
(points)
Med
QRacad
yes
Maj
Upper
Wksht
Y
68
8
-0.502
0.608
-1.693
0.689
-0.826
0.409
(S91)Dougherty et al. 1995
Chem
Exam
Lg
QRpre
no
Maj
Intro
Quiz
Y
260
9
0.230
0.598
-0.941
1.402
0.385
0.700
(S92)Doymus 20072
Chem
Exam
Med
QRpre
yes
Maj
intro
Wrksht
Y
98
8
0.754
0.600
-0.422
1.930
1.257
0.209
(S93)Ellington 2005
Math
Exam
na
QRno
yes
Non
Intro
Wrksht
Y
502
9
0.326
0.594
-0.838
1.490
0.549
0.583
(S94)Enriquez 2008: CCC
Eng
(points)
Sm
QRno
no
Maj
Upper
Wrksht
Y
30
7
0.837
0.631
-0.400
2.074
1.326
0.185
(S94)Enriquez 2008: SFSU
Eng
(points)
Sm
QRpre
yes
Maj
Upper
Wrksht
Y
23
7
0.682
0.649
-0.590
1.954
1.051
0.293
(S95)Erwin &Rieppi 1999: AbnPsych
Psych
Exam
Med
QRacad
no
Maj
Upper
Clicker
Y
75
8
1.097
0.610
-0.100
2.293
1.797
0.072
(S95)Erwin &Rieppi 1999: HumDev
Psych
Exam
Lg
QRacad
no
Maj
Upper
Clicker
Y
133
9
0.763
0.611
-0.435
1.960
1.249
0.212
(S95)Erwin &Rieppi 1999: Stats
Psych
Exam
Med
QRacad
no
Maj
Upper
Clicker
Y
92
8
0.523
0.604
-0.661
1.706
0.865
0.387
(S96)Fallahi 20082
Psych
Exam
Sm
QRpre
yes
Maj
na
Wrksht
Y
82
8
0.829
0.608
-0.363
2.020
1.363
0.173
(S97)Felder et al. 1998
Eng
(grade)
Med
QRacad
yes
Maj
Upper
Wrksht
Y
369
9
0.328
0.595
-0.839
1.495
0.551
0.582
(S98)Forbes 1997
Math
(points)
na
QRpre
no
Maj
Intro
na
N
69
8
-0.102
0.612
-1.301
1.097
-0.166
0.868
(S99)Froelich et al. 2008
Math
Exam
na
QRacad
yes
Non
Intro
Wrksht
Y
33
7
0.872
0.626
-0.355
2.098
1.393
0.164
(S100)Gifford & Vicks 1982
Bio
Exam
Sm
QRacad
no
Maj
Intro
Wrksht
Y
80
8
0.700
0.604
-0.485
1.885
1.158
0.247
(S101)Giraud 1997
Math
Exam
Sm
QRpre
yes
Maj
Intro
Wrksht
Y
92
8
0.602
0.600
-0.575
1.778
1.002
0.316
(Ref#)Citation: Study
Disci-pline
Assess Type
Size
Studnt
Instr
Maj
Level
Trtmnt name
Sign?
OrigStN
Clust
AdjN
Hedgessg
S.E.
Lower limit
Upper limit
Z-value
p-value
(S102)Gonzalez 2006
CompSci
(grade)
Sm
QRno
no
Maj
Intro
Wrksht
Y
145
8
0.238
0.606
-0.950
1.426
0.393
0.694
(S103)Graham 1999
Psych
Exam
Sm
Rand
yes
Maj
Intro
Quiz
Y
130
na
0.532
0.178
0.184
0.880
2.995
0.003
(S104)Gruner 19972
Phys
Exam
Sm
Rand
no
Maj
intro
Studio
N
390
na
0.000
0.153
-0.300
0.300
0.003
0.998
(S105)Gutwill-Wise 2001: Berkeley2
Chem
Exam
Lg
QRpre
no
Maj
intro
Wrksht
Y
593
9
0.201
0.593
-0.962
1.364
0.339
0.735
(S105)Gutwill-Wise 2001: Grinnell2
Chem
Exam
Sm
QRpre
no
Maj
intro
Wrksht
Y
46
8
0.063
0.624
-1.160
1.285
0.100
0.920
(S106)Haberyan 20033
Bio
Exam
Med
QRacad
yes
Non
Intro
Quiz
N
111
9
0.067
0.608
-1.125
1.259
0.110
0.913
(S107)Hagen et al. 2000
Chem
Exam
Sm
QRno
yes
Maj
Upper
Wrksht
N
14
na
-0.088
0.506
-1.080
0.903
-0.174
0.862
(S108)Hake 1998
Phys
CI
na
Rand
yes
Maj
Intro
na
Y
62
na
1.970
0.348
1.288
2.653
5.658
0.000
(S109)Heron et al. 20032
Phys
Exam
Lg
QRno
no
Maj
Intro
Wrksht
na
336
9
-0.358
0.596
-1.526
0.810
-0.601
0.548
(S110)Hersam et al. 2004
Eng
(grade)
Sm
QRno
yes
Maj
Upper
Wrksht
Y
33
7
0.529
0.619
-0.685
1.743
0.854
0.393
(S111)Hoellworth et al. 20053
Phys
CI
Sm
QRacad
yes
Maj
Intro
Studio
Y
585
33
0.718
0.340
0.051
1.385
2.110
0.035
(S112)Hsieh & Knight 2008
Eng
Exam
Lg
QRpre
yes
Maj
Intro
PBL
Y
94
8
0.403
0.599
-0.771
1.576
0.672
0.501
(S113)Hurley 2002
CompSci
Exam
Med
QRpre
yes
Maj
intro
Clickers, Quiz
Y
181
9
1.146
0.604
-0.038
2.330
1.897
0.058
(S114)Jensen & Finley 19963
Bio
CI
na
QRpre
no
Non
Intro
Wrksht
Y
52
8
0.299
0.616
-0.908
1.507
0.486
0.627
(S115)Johnson &Fischbach 19922
Math
Exam
na
QRacad
yes
Maj
Intro
Wrksht
N
46
8
0.071
0.621
-1.145
1.288
0.115
0.909
(S116)Keeler &Voxman 1994
Math
(grade)
Sm
QRpre
yes
Non
Intro
Wrksht
N
62
8
0.292
0.610
-0.903
1.487
0.479
0.632
(S117)Kellum et al. 2001
Psych
Exam
Sm
Rand
yes
Maj
Intro
Clickers
na
80
na
1.479
0.250
0.989
1.970
5.909
0.000
(S118)Kitchen et al. 20032
Bio
Exam
Lg
QRno
no
Maj
Upper
Quiz, Wksht
Y
155
9
0.596
0.608
-0.597
1.788
0.979
0.328
(S119)Klionsky 2002
Bio
Exam
na
QRno
yes
Maj
Intro
Quiz, Wksht
na
48
8
0.788
0.623
-0.434
2.010
1.264
0.206
(Ref#)Citation: Study
Disci-pline
Assess Type
Size
Studnt
Instr
Maj
Level
Trtmnt name
Sign?
OrigStN
Clust
AdjN
Hedgessg
S.E.
Lower limit
Upper limit
Z-value
p-value
(S120)Knight & Wood 2005
Bio
Exam
Med
QRpre
yes
Maj
Upper
Clickers,Wrksht
Y
146
9
0.558
0.605
-0.627
1.743
0.923
0.356
(S219)Koch 1992
Math
Exam
Sm
QRpre
no
Non
Intro
Wrksht
Y
89
8
0.500
0.606
-0.688
1.688
0.825
0.409
(S122)Krause 19972,3
Phys
Exam
na
QRpre
no
Maj
Intro
Wrksht
N
242
9
-0.319
0.599
-1.492
0.854
-0.533
0.594
(S131)Krawiec et al. 2005
Bio
Exam
Sm
QRacad
yes
Maj
Intro
Wrksht
N
58
8
-0.083
0.611
-1.280
1.115
-0.135
0.892
(S124)Kurbanoglu et al. 2006
Chem
Exam
Lg
QRacad
no
Maj
Upper
Wrksht
Y
40
8
1.562
0.651
0.286
2.838
2.399
0.016
(S125)Lasry et al. 2008
Phys
CI
Sm
QRpre
no
Maj
Intro
Clicker
Y
91
8
0.507
0.608
-0.685
1.699
0.833
0.405
(S126)Lawson et al. 2006
Psych
Exam
Sm
QRpre
yes
na
Intro
Wksht
Y
127
9
0.496
0.607
-0.693
1.685
0.818
0.414
(S127)Lee 2009: E&M
Phys
CI
Lg
QRpre
no
Maj
Upper
Clickers
N
228
9
0.144
0.599
-1.030
1.318
0.240
0.810
(S127)Lee 2009: Mech
Phys
CI
Lg
QRpre
no
Maj
Upper
Clickers
N
760
18
0.239
0.451
-0.645
1.123
0.530
0.596
(S127)Lee 2009: Waves
Phys
Exam
Lg
QRpre
no
Maj
Upper
Clickers
Y
296
9
0.544
0.596
-0.624
1.712
0.913
0.361
(S128)LeJeaune 20023
CompSci
(grade)
na
QRacad
yes
Maj
Intro
PBL
N
16
13
-1.152
0.550
-2.229
-0.074
-2.095
0.036
(S129)Lenaerts et al. 2003
Phys
CI
Lg
QRno
no
Maj
Intro
Clickers
na
280
9
1.748
0.602
0.568
2.927
2.904
0.004
(S130)Lewis & Lewis 2008
Chem
Exam
Lg
QRacad
no
Maj
Intro
Wrksht
Y
1740
9
0.104
0.592
-1.056
1.264
0.176
0.861
(S131)Lindaman 20072
Math
Exam
na
QRacad
no
Maj
Intro
Wrksht
Y
54
8
0.369
0.615
-0.837
1.574
0.599
0.549
(S132)Linsey et al. 2009
Eng
Exam
Sm
QRpre
no
Maj
na
Wrksht
na
153
8
0.189
0.604
-0.995
1.372
0.313
0.755
(S133)Lohse et al. 2003
Bio
Exam
na
QRno
yes
Maj
Intro
PBL
N
25
7
-0.616
0.640
-1.870
0.637
-0.964
0.335
(S134)Lord 19972
Bio
Exam
Med
QRacad
yes
Non
Intro
Wrksht, Quiz
Y
178
9
0.360
0.602
-0.819
1.539
0.598
0.550
(S135)Lord 19992
Bio
Exam
na
QRacad
yes
na
Intro
Wrksht, Quiz
Y
178
9
0.719
0.603
-0.462
1.900
1.193
0.233
(S136)Lovelace & McKnight 1980
Math
Exam
Sm
Rand
yes
Maj
Intro
Wrksht
na
37
na
0.634
0.330
-0.013
1.281
1.920
0.055
(S137)Lucas 1999
Math
(points)
na
QRacad
yes
Non
Intro
Wrksht
Y
623
9
0.038
0.593
-1.124
1.201
0.064
0.949
(Ref#)Citation: Study
Disci-pline
Assess Type
Size
Studnt
Instr
Maj
Level
Trtmnt name
Sign?
OrigStN
Clust
AdjN
Hedgessg
S.E.
Lower limit
Upper limit
Z-value
p-value
(S138)Marbach-Ad &Sokolove 2000
Bio
(points)
Lg
QRno
yes
Maj
Intro
Wrksht
na
281
9
0.652
0.598
-0.521
1.824
1.089
0.276
(S139)Martin 20093
Phys
Exam
na
QRpre
yes
Non
Intro
Wrksht
N
48
7
-0.429
0.644
-1.691
0.832
-0.667
0.505
(S140)Mathew 2008
Phys
Exam
Sm
QRpre
no
Non
Intro
Wrksht
Y
42
8
0.911
0.632
-0.327
2.150
1.442
0.149
(S141)Mauk& Hingley 20052
Phys
Exam
Sm
Rand
no
Maj
Intro
Wrksht
Y
186
na
0.417
0.176
0.071
0.763
2.361
0.018
(S142)Mayer et al. 20091
Psych
Exam
Lg
QRacad
yes
Maj
Upper
Clickers
Y
358
13
0.150
0.542
-0.913
1.213
0.276
0.782
(S9)Mays pers. comm.
Math
(grade)
Lg
QRno
yes
Non
Intro
Clicker
na
350
9
0.049
0.596
-1.118
1.216
0.082
0.935
(S143)McConnell et al. 2005
Geo
(points)
Lg
QRpre
yes
Non
Intro
Wrksht
na
741
9
0.238
0.593
-0.923
1.400
0.402
0.688
(S144)McConnell et al. 2006
Geo
CI
na
QRno
yes
Non
Intro
Clicker
na
144
9
0.800
0.606
-0.388
1.988
1.320
0.187
(S145)McCormick 20001
Bio
Exam
Sm
QRpre
no
Maj
Intro
Studio
N
90
na
0.536
0.213
0.118
0.954
2.515
0.012
(S146)McDaniel et al. 2007
Bio
CI
na
QRpre
yes
Maj
Intro
Clickers
Y
137
9
0.635
0.607
-0.554
1.824
1.046
0.295
(S147)McFarlin 2008
Bio
Exam
na
QRno
yes
Maj
Upper
Quiz, Clickers
Y
658
9
0.848
0.593
-0.315
2.011
1.429
0.153
(S148)Meel 1998
Math
Exam
na
QRacad
yes
Maj
Intro
Wrksht
Y
26
7
-0.395
0.637
-1.643
0.853
-0.620
0.535
(S149)Miller 19992
Math
Exam
na
QRpre
no
Maj
Intro
Wrksht
N
46
8
-0.126
0.620
-1.341
1.089
-0.203
0.839
(S150)Mohamed 2008
Chem
Exam
Sm
QRno
yes
Maj
Intro
Wrksht
Y
42
8
0.510
0.626
-0.718
1.737
0.814
0.416
(S151)Montelone et al. 20082
Bio
Exam
Med
QRacad
yes
Maj
Intro
Studio
N
25
7
0.113
0.640
-1.142
1.368
0.176
0.860
(S152)Morling et al. 20082
Psych
Exam
Lg
Rand
yes
Maj
Intro
Clicker
Y
1,277
na
0.123
0.056
0.013
0.232
2.189
0.029
(S153)Nasr & Ramadan 2008
Eng
Exam
na
QRno
yes
Maj
Upper
PBL
na
79
8
0.999
0.617
-0.210
2.208
1.620
0.105
(S154)Nehm& Reilly 20072
Bio
(points)
Med
QRno
no
Maj
intro
Wrksht
Y
189
9
-0.323
0.601
-1.500
0.855
-0.537
0.591
(S155)Nelson et al. 20092
Bio
Exam
Lg
QRno
no
Maj
Upper
Wrksht
na
5,757
9
0.131
0.591
-1.028
1.289
0.221
0.825
(S156)Norwood 1995
Math
(grade)
na
QRacad
yes
Non
Intro
Wrksht
Y
390
9
0.532
0.595
-0.634
1.699
0.894
0.371
(Ref#)Citation: Study
Disci-pline
Assess Type
Size
Studnt
Instr
Maj
Level
Trtmnt name
Sign?
OrigStN
Clust
AdjN
Hedgessg
S.E.
Lower limit
Upper limit
Z-value
p-value
(S157)O'Brien 19932,3
Math
Exam
Sm
QRno
yes
Non
Intro
Wrksht
N
167
15
-0.012
0.482
-0.957
0.932
-0.025
0.980
(S158)Oliver-Hoyo et al. 20042
Chem
Exam
Med
QRno
yes
Maj
Intro
Studio
Y
167
9
0.105
0.605
-1.080
1.290
0.173
0.862
(S159)O'Sullivan & Copper 20033
Chem
Exam
Sm
QRacad
yes
Maj
Intro
Wrksht
N
4,048
36
0.023
0.326
-0.616
0.662
0.070
0.945
(S160)Overlock 1994
Phys
Exam
Sm
QRno
no
Maj
Intro
Wrksht
N
30
7
-0.077
0.624
-1.300
1.145
-0.124
0.901
(S161)Pandy et al. 20042
Eng
Exam
Sm
Rand
yes
Maj
Upper
Wrksht
Y
21
na
0.652
0.433
-0.197
1.502
1.505
0.132
(S162)Park & Travers 1996
Math
Exam
na
QRpre
no
Maj
Intro
Wrksht
Y
68
8
0.329
0.609
-0.864
1.521
0.540
0.589
(S163)Paschal 2002
Bio
Exam
Med
QRacad
yes
Maj
Upper
clickers
N
132
9
0.234
0.605
-0.953
1.421
0.386
0.699
(S164)Poirier & Feldman 2007
Psych
Exam
Lg
QRacad
yes
Maj
Intro
Clicker
Y
865
9
0.136
0.592
-1.024
1.296
0.230
0.818
(S165)Pollock & Finkelstein 20081
Phys
CI
Lg
QRno
no
Maj
Intro
Wrksht
na
2414
36
0.456
0.338
-0.206
1.118
1.349
0.177
(S166)Prather et al. 2009
Phys
CI
na
QRpre
yes
Non
Intro
na
Y
42
na
1.314
0.341
0.646
1.982
3.857
0.000
(S167)Priebe 19972
CompSci
CI
na
QRpre
yes
Maj
Intro
Wrksht
N
46
8
0.167
0.620
-1.049
1.383
0.269
0.788
(S168)Randolph et al. 1992
Bio
Exam
Sm
QRpre
yes
Non
intro
Wksht
N
76
8
-0.065
0.604
-1.248
1.118
-0.108
0.914
(S169)Redish et al. 19972
Phys
Exam
na
QRno
yes
Maj
Intro
Wrksht
na
10
na
1.919
0.860
0.234
3.605
2.232
0.026
(S170)Robinson &Niaz 19912
Chem
Exam
Med
QRacad
yes
Maj
Intro
Wrksht
Y
144
9
0.160
0.604
-1.024
1.344
0.265
0.791
(S171)Roddick 2001: ME
Math
(grade)
na
QRacad
yes
Maj
Intro
Wrksht
N
524
80
0.056
0.508
-0.939
1.052
0.111
0.912
(S171)Roddick 2001: Phys
Math
(grade)
na
QRacad
yes
Maj
Intro
Wrksht
Y
684
75
0.400
0.500
-0.580
1.379
0.800
0.424
(S172)Roselli&Brophy 2006
Eng
Exam
Med
Rand
no
Maj
Intro
PBL
N
87
na
0.138
0.213
-0.279
0.555
0.648
0.517
(S173)Ross & Fulton 1994
Chem
Exam
na
QRno
no
Maj
Upper
Wrksht
N
65
8
0.235
0.609
-0.959
1.429
0.385
0.700
(S174)Rutz et al. 2003
Eng
(grade)
Med
QRacad
no
Maj
Intro
Wrksht
Y
103
9
0.263
0.610
-0.932
1.459
0.431
0.666
(S175)Rybarczyk et al. 2007
Bio
Exam
Sm
QRpre
no
Maj
Upper
Cases
Y
120
9
0.817
0.610
-0.380
2.013
1.338
0.181
(Ref#)Citation: Study
Disci-pline
Assess Type
Size
Studnt
Instr
Maj
Level
Trtmnt name
Sign?
OrigStN
Clust
AdjN
Hedgessg
S.E.
Lower limit
Upper limit
Z-value
p-value
(S176)Sadler 20021
Bio
Exam
Med
QRno
no
Non
Intro
Wrksht
Y
349
26
0.331
0.514
-0.675
1.338
0.645
0.519
(S177)Schielack 19881
Math
Exam
na
QRno
yes
Maj
Intro
Wrkshts
N
54
11
-0.281
0.576
-1.409
0.848
-0.488
0.626
(S178)Shatila 20073
Chem
Exam
na
QRno
yes
Maj
Upper
Wrksht
N
100
16
-0.226
0.475
-1.158
0.706
-0.475
0.635
(S179)Sorenson et al. 20061
Phys
CI
Lg
QRpre
no
Non
Intro
Studio
na
547
18
0.781
0.526
-0.251
1.813
1.484
0.138
(S180)Steele 2003
Bio
Exam
na
QRacad
yes
maj
intro
Quiz
Y
1049
9
0.400
0.592
-0.760
1.560
0.676
0.499
(S181)Tanahoung et al. 2009
Phys
CI
Lg
QRpre
yes
Maj
Intro
IntDem
na
327
9
0.410
0.596
-0.759
1.578
0.687
0.492
(S182)Tessier 20071,2,3
Bio
Exam
Med
Rand
yes
Non
Intro
Wksht
Y
140
na
0.340
0.169
0.008
0.671
2.007
0.045
(S183)Tien et al. 2002
Chem
(points)
Sm
QRpre
yes
Maj
Upper
Wrksht
Y
1,809
521
0.630
0.113
0.410
0.851
5.599
0.000
(S184)Turns et al. 2009
Eng
Exam
na
QRno
yes
Maj
Upper
Wrksht
Y
na
9
0.333
0.609
-0.860
1.526
0.548
0.584
(S185)Udovic et al. 2002
Bio
Exam
na
QRpre
no
Non
Intro
Studio
Y
174
9
1.241
0.607
0.051
2.431
2.044
0.041
(S186)Usoh 20032
Eng
Exam
na
Rand
no
Maj
Intro
PBL
N
38
na
0.135
0.319
-0.491
0.761
0.423
0.672
(S187)Valentino 1988
Math
Exam
na
QRpre
no
Non
Intro
Wrksht
N
58
8
1.209
0.622
-0.011
2.428
1.943
0.052
(S188)Van Dijk et al. 20011,2
Eng
Exam
Sm
QRpre
yes
Maj
intro
Clickers
N
86
12
-0.230
0.567
-1.341
0.881
-0.405
0.685
(S189)Van Gorp& Grissom 2001
CompSci
Exam
Sm
QRno
yes
Maj
Intro
Wrksht
N
75
8
0.457
0.610
-0.740
1.653
0.748
0.454
(S190)Van Heuvelen 1991: Precalc
Phys
CI
na
QRacad
no
Maj
Intro
Wrksht, Quiz
na
94
8
0.744
0.601
-0.433
1.922
1.239
0.215
(S190)Van Heuvelen 1991: Eng
Phys
Exam
na
QRacad
no
Maj
Intro
Wrksht, Quiz
na
152
9
0.896
0.611
-0.301
2.093
1.467
0.142
(S191)Walker et al. 2008
Bio
Exam
Lg
QRacad
yes
Maj
Intro
Wrksht, Quiz
Y
503
9
0.190
0.594
-0.974
1.354
0.320
0.749
(S192)Weir 20042
Eng
Exam
na
QRacad
yes
Maj
Upper
Wrkshts
Y
76
8
0.062
0.603
-1.121
1.244
0.102
0.918
(S193)Wilke& Straits 2001
Bio
Exam
Lg
Rand
yes
na
Intro
Wrksht
Y
688
na
0.126
0.076
-0.024
0.275
1.650
0.099
(Ref#)Citation: Study
Disci-pline
Assess Type
Size
Studnt
Instr
Maj
Level
Trtmnt name
Sign?
OrigStN
Clust
AdjN
Hedgessg
S.E.
Lower limit
Upper limit
Z-value
p-value
(S194)Williamson & Rowe 2002
Chem
(grade)
Sm
QRacad
yes
Maj
Upper
Wrksht
N
45
8
0.252
0.630
-0.983
1.488
0.400
0.689
(S195)Yoder &Hochevar 20053
Psych
Exam
Sm
QRacad
yes
Maj
Upper
Wrksht
Y
79
17
0.710
0.469
-0.209
1.629
1.515
0.130
(S196)Zimrot& Ashkenazi 2007
Chem
Exam
Lg
QRacad
no
Maj
Intro
IntDem
Y
400
9
0.368
0.595
-0.798
1.534
0.619
0.536
Table S4, continued
b. Failure rate data
(Ref#)Citation: Study
Discipline
Metric
Size
Level
Odds ratio
Lower limit
Upper limit
Z-Value
p-value
(S197)Al-Holou et al. 1999: Rose-Hulman2
Eng
To2nd
na
Intro
2.213
1.541
3.179
4.298
0.000
(S197)Al-Holou et al. 1999: Florida
Eng
To3rd
na
Intro
1.481
0.947
2.318
1.719
0.086
(S197)Al-Holou et al. 1999: TAMU2
Eng
To2nd
na
Intro
1.442
0.656
3.173
0.911
0.363
(S197)Al-Holou et al. 1999: Alabama2
Eng
To2nd
na
Intro
1.224
0.670
2.234
0.657
0.511
(S198)Alsardary& Blumberg 2009
Math
DF
Sm
Upper
1.000
0.058
17.181
0.000
1.000
(S199)Barak et al. 20072
CompSci
F
Lg
Intro
2.749
0.979
7.724
1.919
0.055
(S200)Barbarick 1998
Geo
DF
Lg
Intro
1.465
0.994
2.160
1.928
0.054
(S201)Becvar et al. 2008
Chem
DF
na
Intro
2.598
2.326
2.900
16.961
0.000
(S49)Beichner et al. 19992
Eng
DFW
Sm
Intro
2.600
1.523
4.439
3.501
0.000
(S202)Bernold et al. 20002
Eng
DF
na
Upper
2.222
0.977
5.055
1.904
0.057
(S203)Biggers et al. 2009
CompSci
DFW
Med
Intro
1.797
1.124
2.875
2.446
0.014
(S68)Blue 1997
Phys
Drop
na
na
1.246
0.849
1.828
1.125
0.261
(S204)Borglund 2007
Eng
DF
na
Upper
2.364
0.850
6.572
1.649
0.099
(S205)Broyles 1999: Math
Math
Drop
na
Intro
0.816
0.441
1.510
-0.648
0.517
(S205)Broyles 1999: Physics
Phys
Drop
na
na
1.000
0.214
4.665
0.000
1.000
(S205)Broyles 1999: Eng
Eng
Drop
na
na
0.350
0.013
9.120
-0.631
0.528
(S76)Bullard et al. 20082
Eng
To3rd
Med
Upper
2.167
0.766
6.126
1.458
0.145
(S214)Buxeda& Moore 20002
Bio
DFW
Sm
Intro
1.114
0.190
6.540
0.119
0.905
(S9)Caldwell 20072
Bio
Drop
Lg
Intro
2.963
1.450
6.055
2.979
0.003
(S207)Casanova 1971
Chem
Drop
Med
Upper
1.714
0.520
5.657
0.885
0.376
(S208)Chambers 20082
Bio
DFW
Sm
Intro
8.260
3.862
17.666
5.444
0.000
(Ref#)Citation: Study
Disc
Metric
Size
Level
Odds ratio
Lower limit
Upper limit
Z-Value
p-value
(S81)Chaplin 2009
Bio
DF
Med
Intro
1.803
0.723
4.499
1.263
0.206
(S209)Chase & Okie 20002
CompSci
DFW
Sm
Intro
2.637
1.462
4.757
3.222
0.001
(S210)DePree 1998
Math
Drop
na
na
1.404
0.922
2.139
1.580
0.114
(S91)Dougherty et al. 1995
Chem
DFW
Lg
Intro
2.573
2.248
2.945
13.725
0.000
(S211)Farrell et al. 1999
Chem
DFW
Sm
Intro
2.645
1.785
3.919
4.847
0.000
(S97)Felder et al. 19982
Eng
DFW
Med
Upper
0.542
0.322
0.912
-2.307
0.021
(S212)Gautreau&Novemsky 1997: EngPhys
Phys
DFW
Med
Intro
2.052
1.150
3.663
2.432
0.015
(S212)Gautreau&Novemsky 1997: IntroPhys
Phys
DFW
Med
Intro
1.731
0.964
3.107
1.837
0.066
(S213)Gosser 2011: CCNY
Chem
DF
na
Intro
2.252
1.741
2.913
6.180
0.000
(S213)Gosser 2011: StX
Chem
DF
na
Upper
2.085
1.070
4.062
2.158
0.031
(S213)Gosser 2011: UPitt
Chem
DF
na
na
1.930
1.646
2.263
8.099
0.000
(S107)Hagen et al. 2000
Chem
DFW
Sm
Upper
2.280
1.591
3.268
4.488
0.000
(S214)Hoffman 2001
Bio
DF
Med
Intro
5.504
3.084
9.820
5.772
0.000
(S215)Howles 20092
CompSci
SubsCrse
Sm
Intro
2.082
0.593
7.305
1.145
0.252
(S216)Hoyt &Ohland 1998
Eng
To3rd
Lg
Intro
2.025
1.410
2.908
3.821
0.000
(S217)Kashy et al. 19982
Phys
Drop
Lg
Intro
1.793
1.096
2.933
2.324
0.020
(S116)Keeler &Steinhorst 19942
Math
DFW
na
Intro
7.091
3.474
14.473
5.380
0.000
(S218)Keeler &Voxman 1994
Math
F
Sm
Intro
5.905
1.216
28.674
2.202
0.028
(S219)Koch 1992
Math
DF
Sm
Intro
1.778
0.686
4.608
1.184
0.236
(S220)Kurki-Suonio&Hakola 2007
Phys
DFW
Lg
Upper
1.561
0.824
2.955
1.366
0.172
(S125)Lasry et al. 20072
Phys
Drop
Sm
Intro
5.079
1.465
17.606
2.562
0.010
(S221)Lasserre 20092
CompSci
Drop
na
Intro
2.823
1.458
5.466
3.079
0.002
(S222)Maggelakis&Lutzer 20072
Math
DFW
Lg
Intro
2.324
1.663
3.248
4.939
0.000
(S223)Maher 1998: Precalc
Math
FW
Med
Intro
0.977
0.770
1.240
-0.190
0.849
(S223)Maher 1998: Calc
Math
FW
Med
Intro
1.191
1.031
1.375
2.380
0.017
(Ref#)Citation: Study
Disc
Metric
Size
Level
Odds ratio
Lower limit
Upper limit
Z-Value
p-value
(S223)Maher 1998: Service math
Math
FW
Med
Intro
1.269
1.014
1.587
2.083
0.037
(S224)Marrs& Novak 2004
Bio
DFW
Lg
Intro
1.482
0.913
2.408
1.590
0.112
(S9)Mays pers. comm.
Math
DFW
Lg
Intro
1.202
0.785
1.839
0.847
0.397
(S225)McConnell et al. 20032
Geo
Drop
Lg
Intro
1.953
1.322
2.887
3.359
0.001
(S226)Mittag& Collins 2000
Math
DFW
na
Intro
1.016
0.598
1.726
0.060
0.952
(S227)Nirmalakhandan et al. 2007
Eng
DFW
Sm
Upper
2.786
1.511
5.135
3.283
0.001
(S228)Nuutila et al. 2005
CompSci
Drop
Sm
Intro
4.091
2.245
7.453
4.603
0.000
(S229)Paulson 19992
Chem
DFW
Med
Upper
4.838
2.446
9.570
4.531
0.000
(S230)Poulis et al. 1998
Phys
DFW
na
na
4.120
3.619
4.689
21.428
0.000
(S231)Preszler 2009
Bio
DFW
Lg
Intro
1.721
1.452
2.040
6.270
0.000
(S232)Quitadamo et al. 2009: Chem2
Chem
DFW
na
Upper
1.000
0.719
1.392
0.000
1.000
(S232)Quitadamo et al. 2009: Math107
Math
DFW
na
Upper
1.491
1.014
2.193
2.031
0.042
(S232)Quitadamo et al. 2009: Math2502
Math
DFW
na
Upper
1.583
0.562
4.461
0.870
0.385
(S233)Rogerson 2003
Chem
FW
Sm
Intro
2.627
1.173
5.883
2.348
0.019
(S178)Shatila 2007
Chem
DF
na
Upper
9.735
0.498
190.305
1.500
0.134
(S234)Steinberg & Donnelly 2003
Phys
Drop
Lg
Intro
1.440
0.720
2.881
1.031
0.303
(S183)Tien et al. 2002
Chem
DFW
Sm
Upper
1.714
1.418
2.073
5.563
0.000
(S187)Valentino 1988
Math
DFW
na
Intro
2.250
0.856
5.913
1.645
0.100
(S189)Van Gorp& Grissom 20012
CompSci
DFW
Sm
Intro
0.659
0.250
1.735
-0.844
0.399
(S190)Van Heuvelen 1991
Phys
Drop
na
Intro
4.833
1.339
17.441
2.406
0.016
(S194)Williamson & Rowe 2002
Chem
DFW
Sm
Upper
2.217
0.856
5.742
1.640
0.101
Definitions and annotations, focused on characteristics used in heterogeneity analyses:
Ref# The citations number in the list of references.
Citation Author(s) and year; note that full citations for each paper listed are given in the Supporting Information references.
Citation: Study Study name identifies independent studies reported in the same paper.
Citation/Study superscripted numeral Indicates that the effect size reported was calculated using data from:
1multiple treatment and/or control groups
2multiple outcomes (e.g. several midterm exams)
3multiple subgroups or replicates (paired treatments and controls)
Discipline Categorized as Bio (biology, natural resources, nutrition), Chem (chemistry), CompSi (computer science), Eng (engineering), Geo (geology), Math (mathematics and statistics), Phys (physics, astronomy), Psych (psychology, including statistics taught by psychology faculty for psychology majors).
Assess Type Type of assessment used to generate an effect size reflecting student performance or learning, as Exam (midterms, finals, quizzes), CI (concept inventory), grade (course grade, based on identical exams), points (scores on identical assessments that were not categorized as exams or concept inventories).
Size Class size, binned as Sm (small), Med (medium), Lg (large).
Studnt Control for student equivalence in the active learning vs. lecturing treatments, categorized as:
QRno (quasi-random design, no data on student equivalence);
QRpre (quasi-random design, no statistical difference on pre-test);
QRacad (quasi-random design, no statistical difference on measure(s) of academic ability/preparation);
Rand (randomized or crossover designs).
Instr Control for instructor identity, binned as yes (identical or >3 in each treatement) and no (no data, or different 1-2 instructors in the active learning vs. lecturing treatments).
Maj Student population, binned as Maj (majors) or Non (non-majors).
Level Course level, binned as Intro (introductory) and Upper (upper division).
Trtmntname Type of active learning treatment, categorized as Cases (case studies completed in class), Clicker (use of personal response devices), IntDem (interactive demonstrations), PBL (problem-based learning), Quiz (additional quizzes that replaced lecturing time), Studio (studio or workshop course designs), Wrksht (worksheets, tutorials, case studies, informal group problem-solving). Note that if a cell contains more than one type of treatment, the study was categorized as Combination in the heterogeneity analysis.
Sign? Original study reported a significant increase in student learning under active learning versus traditional lecturing; Y = yes, N = no, na = cannot be determined
Orig St N The total sample size in the original study.
ClustAdjN The sample size used in variance calculationsadjusted for cluster number in quasi-random studies.
Hedgessg Effect sizeestimate,reportedasHedgess g
S.E. Standard error of Hedgess g
Lower limit Lower bound of the 95% confidence interval
Upper limit Upper bound of the 95% confidence interval
Z-value Value returned from a Z-test for the significance of the effect size
p-value Probability of observing an effect size this large by chance
na Not applicable; either data are missing or cluster correction is not applicable
Metric Measurement used to indicate failure rate, categorized as the letter grades or letter grade combinations of DF, DFW (where W indicated withdrawal/drop), W, F, FW; or To2nd or To3rd (approval for progressing to the second year or third year in a program), or SubsCrse (approval for progressing to the subsequent course in a sequence).
Odds-ratio Effect size estimate, reported as the odds-ratio