· web view2014/05/08 · type of active learning treatment, categorized as cases (case studies...
TRANSCRIPT
Table S4 : Catalog of studies analyzed—their characteristics and effect size data
a. Exam/concept inventory data
(Ref#)Citation: Study
Disci-pline
Assess Type Size Studnt Instr Maj Level
Trtmnt name Sign?
OrigStN
ClustAdjN
Hedges’sg S.E.
Lower limit
Upper limit
Z-value
p-value
(S53)Anderson et al. 20052 Chem Exam Sm QRno yes Maj Upper PBL Y 420 9 1.334 0.609 0.140 2.529 2.189 0.029(S54)Armbruster et al. 20091 Bio Exam Lg QRno yes Maj Intro
Wrksht, Clicker Y 520 13 0.186 0.540 -0.872 1.244 0.345 0.730
(S55)Armstrong & Hendricks 1999 Math (grade) na QRacad yes Maj Intro Wrkshts N 14 na 0.322 0.655 -0.961 1.605 0.491 0.623
(S56)Armstrong et al. 2007 Bio Exam Lg QRpre yes Non Intro
Clicker, Wrksht, Quiz N 614 9 -0.090 0.593 -1.252 1.073 -0.151 0.880
(S57)Asiala et al. 1997a2 Math Exam na QRacad no Maj Intro Wrksht na 41 8 0.817 0.633 -0.423 2.058 1.291 0.197(S58)Asiala et al. 1997b Math Exam na QRacad no Maj na Wrksht na 37 8 1.143 0.645 -0.121 2.406 1.772 0.076(S59)Austin 1995 Math Exam na QRacad yes Maj Upper Wrksht N 57 8 0.070 0.613 -1.130 1.271 0.115 0.908(S60)Banerjee &Vidyapati 19972 Chem Exam Med QRpre yes Maj Intro Wrksht N 68 8 -0.099 0.606 -1.287 1.089 -0.164 0.870(S61)Bardar& Brecher 2008 Phys CI na Rand yes Non Intro
Clicker, Wrksht na 1,692 145 0.188 0.174 -0.153 0.529 1.080 0.280
(S62)Barg et al. 2000
CompSci Exam na QRacad no Maj Intro PBL N 66 8 0.058 0.607 -1.131 1.248 0.096 0.924
(S63)Barnard 19422 Bio Exam Sm QRpre yes Non Intro
Clicker, Wrksht Y 63 8 0.097 0.609 -1.096 1.290 0.159 0.874
(S64)Basili& Sanford 1991 Chem Exam Sm QRpre yes Maj Intro Wrksht Y 49 8 1.217 0.631 -0.019 2.454 1.930 0.054(S65)Beck &Chzhik 2008
CompSci Exam Sm QRacad no Maj Intro Wksht Y 64 8 0.720 0.611 -0.478 1.918 1.178 0.239
(S49)Beichner et al. 19991 Eng CI Sm QRno yes Maj Intro Studio Y 142 13 3.722 0.650 22.448 4.722 5.728 0.000(S51)Beichner et al. 2007 Phys CI Med QRpre no Maj Intro Studio Y 19 NA 3.394 0.678 2.066 4.722 5.010 0.000(S66)Bilgen 2006 Chem Exam Sm QRpre yes Maj Intro Wrksht Y 81 8 1.220 0.609 0.026 2.414 2.002 0.045(S67)Bilgin et al. 20092 Chem Exam Sm QRpre yes Maj Intro PBL Y 78 8 0.351 0.603 -0.831 1.533 0.582 0.560
(Ref#)Citation: Study
Disci-pline
Assess Type Size Studnt Instr Maj Level
Trtmnt name Sign?
OrigStN
ClustAdjN
Hedges’sg S.E.
Lower limit
Upper limit
Z-value
p-value
(S68)Blue 1997 Phys CI na QRacad no na na Wrksht na 34 7 0.975 0.628 -0.256 2.206 1.552 0.121(S69)Bookman & Friedman 19942 Math Exam na QRacad yes Maj Intro Wrksht Y 605 9 0.413 0.598 -0.758 1.585 0.691 0.489(S70)Booth & James 2001 Phys CI na QRno yes Maj Intro Wrksht N 60 8 -0.475 0.612 -1.675 0.725 -0.776 0.438(S71)Born et al. 19722 Psych Exam Med QRacad yes Maj Upper
Wrksht, Quiz Y 19 na 0.853 0.673 -0.467 2.173 1.267 0.205
(S72)Boyles et al. 19941,2 Psych Exam Sm QRacad no Maj intro Wrksht Y 69 20 0.499 0.542 -0.564 1.562 0.920 0.357(S73)Bradley et al. 2002 Chem (grade) na QRacad yes Maj Upper Wrksht N 120 9 -0.017 0.607 -1.206 1.172 -0.028 0.978(S74)Brown et al. 19723 Math Exam na QRno yes Non Intro Quiz N 37 8 0.086 0.630 -1.148 1.320 0.137 0.891(S75)Buck & Wage 2005 Eng CI na QRpre no Maj Upper
Quiz, Wrksht Y 473 9 0.716 0.597 -0.453 1.886 1.201 0.230
(S76)Bullard et al. 20082,3 Eng Exam Med QRacad no Maj Upper Wrksht Y 246 17 0.127 0.460 -0.774 1.029 0.276 0.782(S77)Bunting &Cheville 2009 Eng Exam Sm QRno yes Maj Upper Wrksht Y 97 8 0.701 0.602 -0.479 1.881 1.164 0.244(S78)Burrowes 20032 Bio Exam Lg QRno yes Maj Intro
Wrksht, Quiz Y 200 9 0.302 0.600 -0.874 1.479 0.503 0.615
(S79)Cahyadi 20042 Phys CI Med QRpre no Maj Intro Wrksht Y 93 9 0.568 0.606 -0.620 1.756 0.937 0.349(80)Carmichael 20092 Bio Exam Lg QRpre yes na Intro Wksht Y 336 9 0.178 0.596 -0.990 1.346 0.298 0.765(S81)Chaplin 2009 Bio Exam Med QRacad yes Maj Intro Cases Y 95 8 0.139 0.598 -1.033 1.311 0.233 0.816(S82)Cheng et al. 20043 Phys CI Med QRpre yes Maj Intro Clickers Y 419 18 0.271 0.455 -0.620 1.163 0.596 0.551(S83)Christensen 2005 Phys Exam Med Rand yes Non Intro Wksht na 15 na 1.028 0.530 -0.012 2.067 1.938 0.053(S50)Crider 20052 Phys Exam Sm Rand yes na Intro
Wrksht, Quiz Y 106 na 2.831 0.358 2.128 3.533 7.898 0.000
(S84)Crouch & Mazur 2001 Phys Exam Lg QRpre no Maj Intro Clickers Y 333 9 0.445 0.596 -0.723 1.614 0.747 0.455(S52)Cummings et al. 19992 Phys CI Sm QRpre no Maj Intro IntDem Y 12 na 2.882 0.993 0.936 4.827 2.903 0.004(S85)Davis &Hult 19972 Psych Exam Med Rand yes Maj intro Wrksht Y 56 na 0.579 0.273 0.044 1.113 2.123 0.034(Ref#)Citation: Disci- Assess Size Studnt Instr Maj Level Trtmnt Orig Clust Hedges’ S.E. Lower Upper Z- p-
Study pline Type name Sign? StN AdjN sg limit limit value value(S86)Day & Foley 2006
CompSci (grade) Sm QRacad yes Maj Upper Wrksht Y 46 8 1.141 0.634 -0.102 2.384 1.799 0.072
(S87)Dees 1991: Algebra Math Exam na Rand yes Non Intro Wrksht Y 76 na 0.547 0.232 0.093 1.000 2.361 0.018(S87)Dees 1991: Geometry Math Exam na Rand yes Non Intro Wrksht Y 63 na 0.380 0.251 -0.112 0.873 1.513 0.130
(S88)DeMetry&Groccia 1997 Eng Exam Med QRno yes Maj Upper Wrksht
N308 9 0.142 0.597 -1.028 1.312 0.238 0.812
(S89)Dennis 20012 Math Exam na QRpre no Non intro Wkrsht Y 74 8 0.277 0.604 -0.907 1.462 0.459 0.646(S90)Dollman 2005 Bio (points) Med QRacad yes Maj Upper Wksht Y 68 8 -0.502 0.608 -1.693 0.689 -0.826 0.409(S91)Dougherty et al. 1995 Chem Exam Lg QRpre no Maj Intro Quiz Y 260 9 0.230 0.598 -0.941 1.402 0.385 0.700(S92)Doymus 20072 Chem Exam Med QRpre yes Maj intro Wrksht Y 98 8 0.754 0.600 -0.422 1.930 1.257 0.209(S93)Ellington 2005 Math Exam na QRno yes Non Intro Wrksht Y 502 9 0.326 0.594 -0.838 1.490 0.549 0.583(S94)Enriquez 2008: CCC Eng (points) Sm QRno no Maj Upper Wrksht Y 30 7 0.837 0.631 -0.400 2.074 1.326 0.185(S94)Enriquez 2008: SFSU Eng (points) Sm QRpre yes Maj Upper Wrksht Y 23 7 0.682 0.649 -0.590 1.954 1.051 0.293(S95)Erwin &Rieppi 1999: AbnPsych Psych Exam Med QRacad no Maj Upper Clicker Y 75 8 1.097 0.610 -0.100 2.293 1.797 0.072(S95)Erwin &Rieppi 1999: HumDev Psych Exam Lg QRacad no Maj Upper Clicker Y 133 9 0.763 0.611 -0.435 1.960 1.249 0.212(S95)Erwin &Rieppi 1999: Stats Psych Exam Med QRacad no Maj Upper Clicker Y 92 8 0.523 0.604 -0.661 1.706 0.865 0.387(S96)Fallahi 20082 Psych Exam Sm QRpre yes Maj na Wrksht Y 82 8 0.829 0.608 -0.363 2.020 1.363 0.173(S97)Felder et al. 1998 Eng (grade) Med QRacad yes Maj Upper Wrksht Y 369 9 0.328 0.595 -0.839 1.495 0.551 0.582(S98)Forbes 1997 Math (points) na QRpre no Maj Intro na N 69 8 -0.102 0.612 -1.301 1.097 -0.166 0.868(S99)Froelich et al. 2008 Math Exam na QRacad yes Non Intro Wrksht Y 33 7 0.872 0.626 -0.355 2.098 1.393 0.164(S100)Gifford & Vicks 1982 Bio Exam Sm QRacad no Maj Intro Wrksht Y 80 8 0.700 0.604 -0.485 1.885 1.158 0.247
(S101)Giraud Math Exam Sm QRpre yes Maj Intro Wrksht 92 8 0.602 0.600 -0.575 1.778 1.002 0.316
1997 Y(Ref#)Citation: Study
Disci-pline
Assess Type Size Studnt Instr Maj Level
Trtmnt name Sign?
OrigStN
ClustAdjN
Hedges’sg S.E.
Lower limit
Upper limit
Z-value
p-value
(S102)Gonzalez 2006
CompSci (grade) Sm QRno no Maj Intro Wrksht Y 145 8 0.238 0.606 -0.950 1.426 0.393 0.694
(S103)Graham 1999 Psych Exam Sm Rand yes Maj Intro Quiz Y 130 na 0.532 0.178 0.184 0.880 2.995 0.003(S104)Gruner 19972 Phys Exam Sm Rand no Maj intro Studio N 390 na 0.000 0.153 -0.300 0.300 0.003 0.998(S105)Gutwill-Wise 2001: Berkeley2 Chem Exam Lg QRpre no Maj intro Wrksht Y 593 9 0.201 0.593 -0.962 1.364 0.339 0.735(S105)Gutwill-Wise 2001: Grinnell2 Chem Exam Sm QRpre no Maj intro Wrksht Y 46 8 0.063 0.624 -1.160 1.285 0.100 0.920(S106)Haberyan 20033 Bio Exam Med QRacad yes Non Intro Quiz N 111 9 0.067 0.608 -1.125 1.259 0.110 0.913(S107)Hagen et al. 2000 Chem Exam Sm QRno yes Maj Upper Wrksht N 14 na -0.088 0.506 -1.080 0.903 -0.174 0.862(S108)Hake 1998 Phys CI na Rand yes Maj Intro na Y 62 na 1.970 0.348 1.288 2.653 5.658 0.000(S109)Heron et al. 20032 Phys Exam Lg QRno no Maj Intro Wrksht na 336 9 -0.358 0.596 -1.526 0.810 -0.601 0.548(S110)Hersam et al. 2004 Eng (grade) Sm QRno yes Maj Upper Wrksht Y 33 7 0.529 0.619 -0.685 1.743 0.854 0.393(S111)Hoellworth et al. 20053 Phys CI Sm QRacad yes Maj Intro Studio Y 585 33 0.718 0.340 0.051 1.385 2.110 0.035(S112)Hsieh & Knight 2008 Eng Exam Lg QRpre yes Maj Intro PBL Y 94 8 0.403 0.599 -0.771 1.576 0.672 0.501(S113)Hurley 2002
CompSci Exam Med QRpre yes Maj intro
Clickers, Quiz Y 181 9 1.146 0.604 -0.038 2.330 1.897 0.058
(S114)Jensen & Finley 19963 Bio CI na QRpre no Non Intro Wrksht Y 52 8 0.299 0.616 -0.908 1.507 0.486 0.627(S115)Johnson &Fischbach 19922 Math Exam na QRacad yes Maj Intro Wrksht N 46 8 0.071 0.621 -1.145 1.288 0.115 0.909(S116)Keeler &Voxman 1994 Math (grade) Sm QRpre yes Non Intro Wrksht N 62 8 0.292 0.610 -0.903 1.487 0.479 0.632(S117)Kellum et al. 2001 Psych Exam Sm Rand yes Maj Intro Clickers na 80 na 1.479 0.250 0.989 1.970 5.909 0.000(S118)Kitchen et al. 20032 Bio Exam Lg QRno no Maj Upper
Quiz, Wksht Y 155 9 0.596 0.608 -0.597 1.788 0.979 0.328
(S119)Klionsky 2002 Bio Exam na QRno yes Maj Intro
Quiz, Wksht na 48 8 0.788 0.623 -0.434 2.010 1.264 0.206
(Ref#)Citation: Study
Disci-pline
Assess Type Size Studnt Instr Maj Level
Trtmnt name Sign?
OrigStN
ClustAdjN
Hedges’sg S.E.
Lower limit
Upper limit
Z-value
p-value
(S120)Knight & Wood 2005 Bio Exam Med QRpre yes Maj Upper
Clickers,Wrksht Y 146 9 0.558 0.605 -0.627 1.743 0.923 0.356
(S219)Koch 1992 Math Exam Sm QRpre no Non Intro Wrksht Y 89 8 0.500 0.606 -0.688 1.688 0.825 0.409(S122)Krause 19972,3 Phys Exam na QRpre no Maj Intro Wrksht N 242 9 -0.319 0.599 -1.492 0.854 -0.533 0.594(S131)Krawiec et al. 2005 Bio Exam Sm QRacad yes Maj Intro Wrksht N 58 8 -0.083 0.611 -1.280 1.115 -0.135 0.892(S124)Kurbanoglu et al. 2006 Chem Exam Lg QRacad no Maj Upper Wrksht Y 40 8 1.562 0.651 0.286 2.838 2.399 0.016(S125)Lasry et al. 2008 Phys CI Sm QRpre no Maj Intro Clicker Y 91 8 0.507 0.608 -0.685 1.699 0.833 0.405(S126)Lawson et al. 2006 Psych Exam Sm QRpre yes na Intro Wksht Y 127 9 0.496 0.607 -0.693 1.685 0.818 0.414(S127)Lee 2009: E&M Phys CI Lg QRpre no Maj Upper Clickers N 228 9 0.144 0.599 -1.030 1.318 0.240 0.810(S127)Lee 2009: Mech Phys CI Lg QRpre no Maj Upper Clickers N 760 18 0.239 0.451 -0.645 1.123 0.530 0.596(S127)Lee 2009: Waves Phys Exam Lg QRpre no Maj Upper Clickers Y 296 9 0.544 0.596 -0.624 1.712 0.913 0.361(S128)LeJeaune 20023
CompSci (grade) na QRacad yes Maj Intro PBL N 16 13 -1.152 0.550 -2.229 -0.074 -2.095 0.036
(S129)Lenaerts et al. 2003 Phys CI Lg QRno no Maj Intro Clickers na 280 9 1.748 0.602 0.568 2.927 2.904 0.004(S130)Lewis & Lewis 2008 Chem Exam Lg QRacad no Maj Intro Wrksht Y 1740 9 0.104 0.592 -1.056 1.264 0.176 0.861(S131)Lindaman 20072 Math Exam na QRacad no Maj Intro Wrksht Y 54 8 0.369 0.615 -0.837 1.574 0.599 0.549(S132)Linsey et al. 2009 Eng Exam Sm QRpre no Maj na Wrksht na 153 8 0.189 0.604 -0.995 1.372 0.313 0.755(S133)Lohse et al. 2003 Bio Exam na QRno yes Maj Intro PBL N 25 7 -0.616 0.640 -1.870 0.637 -0.964 0.335(S134)Lord 19972 Bio Exam Med QRacad yes Non Intro
Wrksht, Quiz Y 178 9 0.360 0.602 -0.819 1.539 0.598 0.550
(S135)Lord 19992 Bio Exam na QRacad yes na Intro
Wrksht, Quiz Y 178 9 0.719 0.603 -0.462 1.900 1.193 0.233
(S136)Lovelace & McKnight 1980 Math Exam Sm Rand yes Maj Intro Wrksht na 37 na 0.634 0.330 -0.013 1.281 1.920 0.055(S137)Lucas 1999 Math (points) na QRacad yes Non Intro Wrksht Y 623 9 0.038 0.593 -1.124 1.201 0.064 0.949(Ref#)Citation: Disci- Assess Size Studnt Instr Maj Level Trtmnt Orig Clust Hedges’ S.E. Lower Upper Z- p-
Study pline Type name Sign? StN AdjN sg limit limit value value
(S138)Marbach-Ad &Sokolove 2000 Bio (points) Lg QRno yes Maj Intro Wrksht
na281 9 0.652 0.598 -0.521 1.824 1.089 0.276
(S139)Martin 20093 Phys Exam na QRpre yes Non Intro Wrksht N 48 7 -0.429 0.644 -1.691 0.832 -0.667 0.505(S140)Mathew 2008 Phys Exam Sm QRpre no Non Intro Wrksht Y 42 8 0.911 0.632 -0.327 2.150 1.442 0.149(S141)Mauk& Hingley 20052 Phys Exam Sm Rand no Maj Intro Wrksht Y 186 na 0.417 0.176 0.071 0.763 2.361 0.018(S142)Mayer et al. 20091 Psych Exam Lg QRacad yes Maj Upper Clickers Y 358 13 0.150 0.542 -0.913 1.213 0.276 0.782(S9)Mays pers. comm. Math (grade) Lg QRno yes Non Intro Clicker na 350 9 0.049 0.596 -1.118 1.216 0.082 0.935(S143)McConnell et al. 2005 Geo (points) Lg QRpre yes Non Intro Wrksht na 741 9 0.238 0.593 -0.923 1.400 0.402 0.688(S144)McConnell et al. 2006 Geo CI na QRno yes Non Intro Clicker na 144 9 0.800 0.606 -0.388 1.988 1.320 0.187(S145)McCormick 20001 Bio Exam Sm QRpre no Maj Intro Studio N 90 na 0.536 0.213 0.118 0.954 2.515 0.012(S146)McDaniel et al. 2007 Bio CI na QRpre yes Maj Intro Clickers Y 137 9 0.635 0.607 -0.554 1.824 1.046 0.295(S147)McFarlin 2008 Bio Exam na QRno yes Maj Upper
Quiz, Clickers Y 658 9 0.848 0.593 -0.315 2.011 1.429 0.153
(S148)Meel 1998 Math Exam na QRacad yes Maj Intro Wrksht Y 26 7 -0.395 0.637 -1.643 0.853 -0.620 0.535(S149)Miller 19992 Math Exam na QRpre no Maj Intro Wrksht N 46 8 -0.126 0.620 -1.341 1.089 -0.203 0.839(S150)Mohamed 2008 Chem Exam Sm QRno yes Maj Intro Wrksht Y 42 8 0.510 0.626 -0.718 1.737 0.814 0.416(S151)Montelone et al. 20082 Bio Exam Med QRacad yes Maj Intro Studio N 25 7 0.113 0.640 -1.142 1.368 0.176 0.860(S152)Morling et al. 20082 Psych Exam Lg Rand yes Maj Intro Clicker Y 1,277 na 0.123 0.056 0.013 0.232 2.189 0.029(S153)Nasr & Ramadan 2008 Eng Exam na QRno yes Maj Upper PBL na 79 8 0.999 0.617 -0.210 2.208 1.620 0.105(S154)Nehm& Reilly 20072 Bio (points) Med QRno no Maj intro Wrksht Y 189 9 -0.323 0.601 -1.500 0.855 -0.537 0.591(S155)Nelson et al. 20092 Bio Exam Lg QRno no Maj Upper Wrksht na 5,757 9 0.131 0.591 -1.028 1.289 0.221 0.825
(S156)Norwood 1995 Math (grade) na QRacad yes Non Intro Wrksht
Y390 9 0.532 0.595 -0.634 1.699 0.894 0.371
(Ref#)Citation: Study
Disci-pline
Assess Type Size Studnt Instr Maj Level
Trtmnt name Sign?
OrigStN
ClustAdjN
Hedges’sg S.E.
Lower limit
Upper limit
Z-value
p-value
(S157)O'Brien 19932,3 Math Exam Sm QRno yes Non Intro Wrksht N 167 15 -0.012 0.482 -0.957 0.932 -0.025 0.980(S158)Oliver-Hoyo et al. 20042 Chem Exam Med QRno yes Maj Intro Studio Y 167 9 0.105 0.605 -1.080 1.290 0.173 0.862
(S159)O'Sullivan & Copper 20033 Chem Exam Sm QRacad yes Maj Intro Wrksht N 4,048 36 0.023 0.326 -0.616 0.662 0.070 0.945(S160)Overlock 1994 Phys Exam Sm QRno no Maj Intro Wrksht N 30 7 -0.077 0.624 -1.300 1.145 -0.124 0.901(S161)Pandy et al. 20042 Eng Exam Sm Rand yes Maj Upper Wrksht Y 21 na 0.652 0.433 -0.197 1.502 1.505 0.132(S162)Park & Travers 1996 Math Exam na QRpre no Maj Intro Wrksht Y 68 8 0.329 0.609 -0.864 1.521 0.540 0.589(S163)Paschal 2002 Bio Exam Med QRacad yes Maj Upper clickers N 132 9 0.234 0.605 -0.953 1.421 0.386 0.699(S164)Poirier & Feldman 2007 Psych Exam Lg QRacad yes Maj Intro Clicker Y 865 9 0.136 0.592 -1.024 1.296 0.230 0.818(S165)Pollock & Finkelstein 20081 Phys CI Lg QRno no Maj Intro Wrksht na 2414 36 0.456 0.338 -0.206 1.118 1.349 0.177(S166)Prather et al. 2009 Phys CI na QRpre yes Non Intro na Y 42 na 1.314 0.341 0.646 1.982 3.857 0.000(S167)Priebe 19972
CompSci CI na QRpre yes Maj Intro Wrksht N 46 8 0.167 0.620 -1.049 1.383 0.269 0.788
(S168)Randolph et al. 1992 Bio Exam Sm QRpre yes Non intro Wksht N 76 8 -0.065 0.604 -1.248 1.118 -0.108 0.914(S169)Redish et al. 19972 Phys Exam na QRno yes Maj Intro Wrksht na 10 na 1.919 0.860 0.234 3.605 2.232 0.026(S170)Robinson &Niaz 19912 Chem Exam Med QRacad yes Maj Intro Wrksht Y 144 9 0.160 0.604 -1.024 1.344 0.265 0.791(S171)Roddick 2001: ME Math (grade) na QRacad yes Maj Intro Wrksht N 524 80 0.056 0.508 -0.939 1.052 0.111 0.912(S171)Roddick 2001: Phys Math (grade) na QRacad yes Maj Intro Wrksht Y 684 75 0.400 0.500 -0.580 1.379 0.800 0.424
(S172)Roselli&Brophy 2006 Eng Exam Med Rand no Maj Intro PBL
N87 na 0.138 0.213 -0.279 0.555 0.648 0.517
(S173)Ross & Fulton 1994 Chem Exam na QRno no Maj Upper Wrksht N 65 8 0.235 0.609 -0.959 1.429 0.385 0.700(S174)Rutz et al. 2003 Eng (grade) Med QRacad no Maj Intro Wrksht Y 103 9 0.263 0.610 -0.932 1.459 0.431 0.666
(S175)Rybarczyk et al. 2007 Bio Exam Sm QRpre no Maj Upper Cases Y 120 9 0.817 0.610 -0.380 2.013 1.338 0.181(Ref#)Citation: Study
Disci-pline
Assess Type Size Studnt Instr Maj Level
Trtmnt name Sign?
OrigStN
ClustAdjN
Hedges’sg S.E.
Lower limit
Upper limit
Z-value
p-value
(S176)Sadler 20021 Bio Exam Med QRno no Non Intro Wrksht Y 349 26 0.331 0.514 -0.675 1.338 0.645 0.519(S177)Schielack 19881 Math Exam na QRno yes Maj Intro Wrkshts N 54 11 -0.281 0.576 -1.409 0.848 -0.488 0.626(S178)Shatila 20073 Chem Exam na QRno yes Maj Upper Wrksht N 100 16 -0.226 0.475 -1.158 0.706 -0.475 0.635(S179)Sorenson et al. 20061 Phys CI Lg QRpre no Non Intro Studio na 547 18 0.781 0.526 -0.251 1.813 1.484 0.138(S180)Steele 2003 Bio Exam na QRacad yes maj intro Quiz Y 1049 9 0.400 0.592 -0.760 1.560 0.676 0.499
(S181)Tanahoung et al. 2009 Phys CI Lg QRpre yes Maj Intro IntDem na 327 9 0.410 0.596 -0.759 1.578 0.687 0.492(S182)Tessier 20071,2,3 Bio Exam Med Rand yes Non Intro Wksht Y 140 na 0.340 0.169 0.008 0.671 2.007 0.045(S183)Tien et al. 2002 Chem (points) Sm QRpre yes Maj Upper Wrksht Y 1,809 521 0.630 0.113 0.410 0.851 5.599 0.000(S184)Turns et al. 2009 Eng Exam na QRno yes Maj Upper Wrksht Y na 9 0.333 0.609 -0.860 1.526 0.548 0.584(S185)Udovic et al. 2002 Bio Exam na QRpre no Non Intro Studio Y 174 9 1.241 0.607 0.051 2.431 2.044 0.041(S186)Usoh 20032 Eng Exam na Rand no Maj Intro PBL N 38 na 0.135 0.319 -0.491 0.761 0.423 0.672(S187)Valentino 1988 Math Exam na QRpre no Non Intro Wrksht N 58 8 1.209 0.622 -0.011 2.428 1.943 0.052(S188)Van Dijk et al. 20011,2 Eng Exam Sm QRpre yes Maj intro Clickers N 86 12 -0.230 0.567 -1.341 0.881 -0.405 0.685(S189)Van Gorp& Grissom 2001
CompSci Exam Sm QRno yes Maj Intro Wrksht N 75 8 0.457 0.610 -0.740 1.653 0.748 0.454
(S190)Van Heuvelen 1991: Precalc Phys CI na QRacad no Maj Intro
Wrksht, Quiz na 94 8 0.744 0.601 -0.433 1.922 1.239 0.215
(S190)Van Heuvelen 1991: Eng Phys Exam na QRacad no Maj Intro
Wrksht, Quiz na 152 9 0.896 0.611 -0.301 2.093 1.467 0.142
(S191)Walker et al. 2008 Bio Exam Lg QRacad yes Maj Intro
Wrksht, Quiz Y 503 9 0.190 0.594 -0.974 1.354 0.320 0.749
(S192)Weir 20042 Eng Exam na QRacad yes Maj Upper Wrkshts Y 76 8 0.062 0.603 -1.121 1.244 0.102 0.918
(S193)Wilke& Straits 2001 Bio Exam Lg Rand yes na Intro Wrksht Y 688 na 0.126 0.076 -0.024 0.275 1.650 0.099(Ref#)Citation: Study
Disci-pline
Assess Type Size Studnt Instr Maj Level
Trtmnt name Sign?
OrigStN
ClustAdjN
Hedges’sg S.E.
Lower limit
Upper limit
Z-value
p-value
(S194)Williamson & Rowe 2002 Chem (grade) Sm QRacad yes Maj Upper Wrksht N 45 8 0.252 0.630 -0.983 1.488 0.400 0.689(S195)Yoder &Hochevar 20053 Psych Exam Sm QRacad yes Maj Upper Wrksht
Y79 17 0.710 0.469 -0.209 1.629 1.515 0.130
(S196)Zimrot& Ashkenazi 2007 Chem Exam Lg QRacad no Maj Intro IntDem Y 400 9 0.368 0.595 -0.798 1.534 0.619 0.536
Table S4, continued
b. Failure rate data
(Ref#)Citation: Study Discipline Metric Size LevelOdds ratio
Lower limit
Upper limit
Z-Value p-value
(S197)Al-Holou et al. 1999: Rose-Hulman2 Eng To2nd na Intro 2.213 1.541 3.179 4.298 0.000(S197)Al-Holou et al. 1999: Florida Eng To3rd na Intro 1.481 0.947 2.318 1.719 0.086(S197)Al-Holou et al. 1999: TAMU2 Eng To2nd na Intro 1.442 0.656 3.173 0.911 0.363(S197)Al-Holou et al. 1999: Alabama2 Eng To2nd na Intro 1.224 0.670 2.234 0.657 0.511(S198)Alsardary& Blumberg 2009 Math DF Sm Upper 1.000 0.058 17.181 0.000 1.000(S199)Barak et al. 20072 CompSci F Lg Intro 2.749 0.979 7.724 1.919 0.055(S200)Barbarick 1998 Geo DF Lg Intro 1.465 0.994 2.160 1.928 0.054(S201)Becvar et al. 2008 Chem DF na Intro 2.598 2.326 2.900 16.961 0.000(S49)Beichner et al. 19992 Eng DFW Sm Intro 2.600 1.523 4.439 3.501 0.000(S202)Bernold et al. 20002 Eng DF na Upper 2.222 0.977 5.055 1.904 0.057(S203)Biggers et al. 2009 CompSci DFW Med Intro 1.797 1.124 2.875 2.446 0.014(S68)Blue 1997 Phys Drop na na 1.246 0.849 1.828 1.125 0.261(S204)Borglund 2007 Eng DF na Upper 2.364 0.850 6.572 1.649 0.099(S205)Broyles 1999: Math Math Drop na Intro 0.816 0.441 1.510 -0.648 0.517(S205)Broyles 1999: Physics Phys Drop na na 1.000 0.214 4.665 0.000 1.000(S205)Broyles 1999: Eng Eng Drop na na 0.350 0.013 9.120 -0.631 0.528(S76)Bullard et al. 20082 Eng To3rd Med Upper 2.167 0.766 6.126 1.458 0.145(S214)Buxeda& Moore 20002 Bio DFW Sm Intro 1.114 0.190 6.540 0.119 0.905(S9)Caldwell 20072 Bio Drop Lg Intro 2.963 1.450 6.055 2.979 0.003(S207)Casanova 1971 Chem Drop Med Upper 1.714 0.520 5.657 0.885 0.376(S208)Chambers 20082 Bio DFW Sm Intro 8.260 3.862 17.666 5.444 0.000(Ref#)Citation: Study Disc Metric Size Level Odds Lower Upper Z- p-value
ratio limit limit Value(S81)Chaplin 2009 Bio DF Med Intro 1.803 0.723 4.499 1.263 0.206(S209)Chase & Okie 20002 CompSci DFW Sm Intro 2.637 1.462 4.757 3.222 0.001(S210)DePree 1998 Math Drop na na 1.404 0.922 2.139 1.580 0.114(S91)Dougherty et al. 1995 Chem DFW Lg Intro 2.573 2.248 2.945 13.725 0.000(S211)Farrell et al. 1999 Chem DFW Sm Intro 2.645 1.785 3.919 4.847 0.000(S97)Felder et al. 19982 Eng DFW Med Upper 0.542 0.322 0.912 -2.307 0.021(S212)Gautreau&Novemsky 1997: EngPhys Phys DFW Med Intro 2.052 1.150 3.663 2.432 0.015(S212)Gautreau&Novemsky 1997: IntroPhys Phys DFW Med Intro 1.731 0.964 3.107 1.837 0.066(S213)Gosser 2011: CCNY Chem DF na Intro 2.252 1.741 2.913 6.180 0.000(S213)Gosser 2011: StX Chem DF na Upper 2.085 1.070 4.062 2.158 0.031(S213)Gosser 2011: UPitt Chem DF na na 1.930 1.646 2.263 8.099 0.000(S107)Hagen et al. 2000 Chem DFW Sm Upper 2.280 1.591 3.268 4.488 0.000(S214)Hoffman 2001 Bio DF Med Intro 5.504 3.084 9.820 5.772 0.000(S215)Howles 20092 CompSci SubsCrse Sm Intro 2.082 0.593 7.305 1.145 0.252(S216)Hoyt &Ohland 1998 Eng To3rd Lg Intro 2.025 1.410 2.908 3.821 0.000(S217)Kashy et al. 19982 Phys Drop Lg Intro 1.793 1.096 2.933 2.324 0.020(S116)Keeler &Steinhorst 19942 Math DFW na Intro 7.091 3.474 14.473 5.380 0.000(S218)Keeler &Voxman 1994 Math F Sm Intro 5.905 1.216 28.674 2.202 0.028(S219)Koch 1992 Math DF Sm Intro 1.778 0.686 4.608 1.184 0.236(S220)Kurki-Suonio&Hakola 2007 Phys DFW Lg Upper 1.561 0.824 2.955 1.366 0.172(S125)Lasry et al. 20072 Phys Drop Sm Intro 5.079 1.465 17.606 2.562 0.010(S221)Lasserre 20092 CompSci Drop na Intro 2.823 1.458 5.466 3.079 0.002(S222)Maggelakis&Lutzer 20072 Math DFW Lg Intro 2.324 1.663 3.248 4.939 0.000(S223)Maher 1998: Precalc Math FW Med Intro 0.977 0.770 1.240 -0.190 0.849(S223)Maher 1998: Calc Math FW Med Intro 1.191 1.031 1.375 2.380 0.017
(Ref#)Citation: Study Disc Metric Size LevelOdds ratio
Lower limit
Upper limit
Z-Value p-value
(S223)Maher 1998: Service math Math FW Med Intro 1.269 1.014 1.587 2.083 0.037(S224)Marrs& Novak 2004 Bio DFW Lg Intro 1.482 0.913 2.408 1.590 0.112(S9)Mays pers. comm. Math DFW Lg Intro 1.202 0.785 1.839 0.847 0.397(S225)McConnell et al. 20032 Geo Drop Lg Intro 1.953 1.322 2.887 3.359 0.001(S226)Mittag& Collins 2000 Math DFW na Intro 1.016 0.598 1.726 0.060 0.952(S227)Nirmalakhandan et al. 2007 Eng DFW Sm Upper 2.786 1.511 5.135 3.283 0.001(S228)Nuutila et al. 2005 CompSci Drop Sm Intro 4.091 2.245 7.453 4.603 0.000(S229)Paulson 19992 Chem DFW Med Upper 4.838 2.446 9.570 4.531 0.000(S230)Poulis et al. 1998 Phys DFW na na 4.120 3.619 4.689 21.428 0.000(S231)Preszler 2009 Bio DFW Lg Intro 1.721 1.452 2.040 6.270 0.000(S232)Quitadamo et al. 2009: Chem2 Chem DFW na Upper 1.000 0.719 1.392 0.000 1.000(S232)Quitadamo et al. 2009: Math107 Math DFW na Upper 1.491 1.014 2.193 2.031 0.042(S232)Quitadamo et al. 2009: Math2502 Math DFW na Upper 1.583 0.562 4.461 0.870 0.385(S233)Rogerson 2003 Chem FW Sm Intro 2.627 1.173 5.883 2.348 0.019(S178)Shatila 2007 Chem DF na Upper 9.735 0.498 190.305 1.500 0.134(S234)Steinberg & Donnelly 2003 Phys Drop Lg Intro 1.440 0.720 2.881 1.031 0.303(S183)Tien et al. 2002 Chem DFW Sm Upper 1.714 1.418 2.073 5.563 0.000(S187)Valentino 1988 Math DFW na Intro 2.250 0.856 5.913 1.645 0.100(S189)Van Gorp& Grissom 20012 CompSci DFW Sm Intro 0.659 0.250 1.735 -0.844 0.399(S190)Van Heuvelen 1991 Phys Drop na Intro 4.833 1.339 17.441 2.406 0.016(S194)Williamson & Rowe 2002 Chem DFW Sm Upper 2.217 0.856 5.742 1.640 0.101
Definitions and annotations, focused on characteristics used in heterogeneity analyses:
Ref# The citation’s number in the list of references.
Citation Author(s) and year; note that full citations for each paper listed are given in the Supporting Information references.
Citation: Study Study name identifies independent studies reported in the same paper.
Citation/Study superscripted numeral Indicates that the effect size reported was calculated using data from:1multiple treatment and/or control groups2multiple outcomes (e.g. several midterm exams)3multiple subgroups or replicates (paired treatments and controls)
Discipline Categorized as Bio (biology, natural resources, nutrition), Chem (chemistry), CompSi (computer science), Eng (engineering), Geo (geology), Math (mathematics and statistics), Phys (physics, astronomy), Psych (psychology, including statistics taught by psychology faculty for psychology majors).
Assess Type Type of assessment used to generate an effect size reflecting student performance or learning, as Exam (midterms, finals, quizzes), CI (concept inventory), grade (course grade, based on identical exams), points (scores on identical assessments that were not categorized as exams or concept inventories).
Size Class size, binned as Sm (small), Med (medium), Lg (large).
Studnt Control for student equivalence in the active learning vs. lecturing treatments, categorized as:
QRno (quasi-random design, no data on student equivalence);
QRpre (quasi-random design, no statistical difference on pre-test);
QRacad (quasi-random design, no statistical difference on measure(s) of academic ability/preparation);
Rand (randomized or crossover designs).
Instr Control for instructor identity, binned as yes (identical or >3 in each treatement) and no (no data, or different 1-2 instructors in the active learning vs. lecturing treatments).
Maj Student population, binned as Maj (majors) or Non (non-majors).
Level Course level, binned as Intro (introductory) and Upper (upper division).
Trtmntname Type of active learning treatment, categorized as Cases (case studies completed in class), Clicker (use of personal response devices), IntDem (interactive demonstrations), PBL (problem-based learning), Quiz (additional quizzes that replaced lecturing time), Studio (studio or workshop course designs), Wrksht (worksheets,
tutorials, case studies, informal group problem-solving). Note that if a cell contains more than one type of treatment, the study was categorized as “Combination” in the heterogeneity analysis.
Sign? Original study reported a significant increase in student learning under active learning versus traditional lecturing; Y = yes, N = no, na = cannot be determined
Orig St N The total sample size in the original study.
ClustAdjN The sample size used in variance calculations—adjusted for cluster number in quasi-random studies.
Hedges’sg Effect sizeestimate,reportedasHedges’s g
S.E. Standard error of Hedges’s g
Lower limit Lower bound of the 95% confidence interval
Upper limit Upper bound of the 95% confidence interval
Z-value Value returned from a Z-test for the significance of the effect size
p-value Probability of observing an effect size this large by chance
na Not applicable; either data are missing or cluster correction is not applicable
Metric Measurement used to indicate failure rate, categorized as the letter grades or letter grade combinations of DF, DFW (where W indicated withdrawal/drop), W, F, FW; or To2nd or To3rd (approval for progressing to the second year or third year in a program), or SubsCrse (approval for progressing to the subsequent course in a sequence).
Odds-ratio Effect size estimate, reported as the odds-ratio