interactive engagement strategies for large classes
TRANSCRIPT
Interactive Engagement Strategies For large and diverse classes (a choose-your-adventure workshop)
14-15 JUNE 2017 LIVERPOOL JMU TEACHING & LEARNING CONFERENCE “VISIONS FOR LEARNING”
Choose your own adventure
Lectures as 2-way conversations Interaction (in / out of class)
Assessments (student-generated) Assessments (exams)
Lectures
Context - class time (specifically lectures)
CC BY-NC 2.0 https://flic.kr/p/f3ynHx!
Derek Bruff: Class time reconsidered!http://prezi.com/donq036eunko/class-time-reconsidered/!
https://www.brookes.ac.uk/services/ocsld/resources/20reasons.html!
Sourcear)cle:Poh,M.Z.,Swenson,N.C.,Picard,R.W.,"AWearableSensorforUnobtrusive,Long-termAssessmentofElectrodermalAc)vity,"IEEETransac)onsonBiomedicalEngineering,vol.57,no.5,pp.1243-1252,May2010.doi:10.1109/TBME.2009.2038487Ac)vity:JaredStang,UBCPhysics
Weekly rhythm for our 1A class!
Poh,M.Z.,Swenson,N.C.,Picard,R.W.,"AWearableSensorforUnobtrusive,Long-termAssessmentofElectrodermalAc)vity,"IEEETransac)onsonBiomedicalEngineering,vol.57,no.5,pp.1243-1252,May2010.doi:10.1109/TBME.2009.2038487
Weekly rhythm for our 1A class!
Learning gains on PI !
1. Pre class material!
© Jorge Royan / http://www.royan.com.ar / CC-BY-SA-3.0, via Wikimedia Commons!
1. Pre class material - DIY media!
1. Pre class material - DIY media!
Learning gains on PI !
Reproduced from Eric Mazur (search “Confessions of a converted lecturer” on YouTube)
1!
2!
3!
A ball initially at rest is!thrown upwards, comes !back down & is caught!!Which of the following is a!plausible graph of the!acceleration of the ball !with time?!
Source activity: Simon Lancaster (UEA, UK) Ross Galloway (Edinburgh, UK)
A large truck collides head on with a small compact car. Which of the following statements is true? 1. The forces on the car is greater 2. The force on the truck is greater 3. The force on the car and truck are
equal 4. Can’t specify without knowing mass
and speed of vehicles
R R Hake !American Journal of Physics: Volume 66, Issue 1, Pages 64-74!http://dx.doi.org/10.1119/1.18809!
Freeman et al PNAS www.pnas.org/cgi/doi/10.1073/pnas.1319030111
Wieman commentary PNAS !http://www.pnas.org/cgi/doi/10.1073/pnas.1407304111
“This meta-analysis makes a powerful case that any college or university that is teaching its STEM courses by traditional lectures is providing an inferior education to its students”
1. Real (and perceived) workloads!
2. Dealing with resistance !
3. Suitability of teaching spaces!
4. About ‘covering content’!
5. Loss of (total) control !
!
… and the upsides too. !
Interaction / communication
FOSTERING INTERACTIONa case study
Simon Bates Senior Advisor, Teaching and Learning
Academic Director, CTLT
Professor of Teaching, Department of Physics and Astronomy
OVERVIEW Landscape of LT tools and applications
My course context
Tools matter!
Implementation
Features
ContentAuthoring ContentDelivery Publisher Simula4ons
AdobeCrea)veSuite Livescribe ArtsFileShare Kaltura(*) UBCBlogs/WordPress(Content)(*) Cengage(PIC)(*) GoogleCardboard
Ar)culateStudio MediaSite(*) CengageiLrn LearnDash(WordPress)(*) Macmillan(PIC)(*) Layar
Audacity Microso]Excel Connect(ContentManagement)(*)
LibraryOnlineCourseReserves(LOCR)(*)
UBCiTunes(*) McGraw-Hill(PIC)(*) MannequinSimula)ons
BbCollaborateVoiceAuthoring(*) Microso]PowerPoint DocumentCameras Lyryx UBCWiki(*) Pearson(PIC)(*) Mo)onControl
Camtasia(*) OnebuconStudio(*) DropBoxModernDigitalImageDatabase(MDID)
UBCYouTube(*) SaplingLearning(PIC)(*) NeoReality
Collabora)veLearning(*)Annota)onSystem(CLAS)
Panopto Drupal(*) Moodle(*) Vimeo Wiley(PIC)(*) PhetSimula)ons
ExplainEverything Dspace Omeka Vitalsource Praxis
FinalCutPro Prezi edX(*) Owncloud WebAssign VCER
Entrada Perusall Wikipedia
HotPotatoes RespondusStudymate(*) Evernote Piazza(*) Wis)a
Jing Snagit ExamPrepDatabase(*) Plone Workspace(*) PorLolios
Keynote TimelineJS Github Podcasts Zimbra ChalkandWire
Lectora GoogleDocs Reflector Zoomify Connect(ePorLolio)(*)
Lightboard(*) VideoScribe(*) HTML5FlashCards SourceTree Pep
iBooks SugarSync
Assessment ResponseSystem
Adap)veCompara)veJudgement(ACJ)(*) MechanicalTA Aplia
AutoMul)pleChoice(AMC) Moodle(*) iClicker(*)
CalibratedPeerReview(CPR)(*) OpenBadgesUBC(*) Kahoot!
CengageiLrn PearsonMyTest LearningCataly)cs
Crowdmark PeerScholar PollEverywhere
Entrada PeerWise(*) REEFpolling(*)
ExamPrepDB(*) Remark TopHat
ExamSo] RespondusQuiz(*)
FormBuilder(*) Scantron
GradeGrinder Turni)n(*)
iPeer(*) WebAssign
LearnDash(WordPress)(*) Webwork(*)
Analy4cs CourseEvalua4on CourseAdmin Other
ArtsDatamart BlackboardOutcomesAssessment(*) CengageiLrn 3Dprin)ng
BlackboardOutcomesAssessment(*) CoursEval(*) Connect(GradeCenter)(*) GoogleEarth
Connect(PerformanceCenter)(*) SEoTDatamart(*) Doodle iStudiezPro
GoogleAnaly)cs TeachEval(*) Entrada SCORM(*)
IBMSPSS GoogleCalendar TinCanAPI(ExperienceAPI)
JMP LearnDash(WordPress)(*)
Microso]Excel Moodle(*)
SEoTDatamart(*) One45
Stata Turni)n(*)
Tableau UBCBlogs/WordPress(*)
WebAssign
LEARNING TECHNOLOGY TOOLS - FUNCTIONAL MAP
CONTENT ASSESSMENT
COURSE MANAGEMENT & EVALUATIONINTERACTIONS
Discussion SurveyTool SocialMedia VC
Connect(Discussions)(*) Connect(EnterpriseSurveys)(*) Facebook AdobeConnect
Piazza(*) Connect(Surveys)(*) Figure1 BbCollaborateWebConf.(*)
PulsePress(*) FluidSurveys(*) Google+ BlueJeans(*)
Slack GravityForms(WordPress) LinkedIn GoogleHangouts
UBCBlogs/WordPress(Discussion)(*) LimeSurvey Twicer Lifesize
Qualtrics Skype
SurveyMonkey VC(MedIT)
Vovici WebEx
Bold = Integrated Tool(*) = Supported by LT Hub
CONTEXT First year intro Physics courses
P101 / P117
Non-majors
Flipped class design
Heterogeneous cohort
TOOLS MATTER
Connect discussions vs Piazza
TOOLS MATTER
Connect discussions vs Piazza
IMPLEMENTATION
“No email” policy
Give reasons to engage
Support TA: to monitor and respond
A BUNCH OF NICE FEATURES…..
Assessment: student Generated material
Selected results & analysis
Engagement - how do students use the system?
Benefits - what is the impact on learning?
Question quality - how good is what students produce?
Relevant publications:
Scaffolding student engagement via online peer learning - European Journal of Physics 35 (4), 045002 (2014)
Student-Generated Content: Enhancing learning through sharing multiple-choice questions. International Journal of Science Education, 1-15 (2014).
Assessing the quality of a student-generated question repository - Phys Rev ST PER (2014) 10, 020105
Student-generated assessment - Education in Chemistry (2013) 13 1
Typical implementation
Minimum participation requirements for each of two assessment exercises (PW1,
PW2) Write 1 Answer 5 Rate / comment 3
5% course credit
Physics 101, Energy & WavesWinter Semester: 3 sections, ~800 students
Not so typical implementation
Writing original questions is a demanding activity
Extensive scaffolding exercises
Revisited in subsequent tutorials
Engagement with PeerWise
Number Multiplier Number Multiplier
Questions 1105 [1.7] 998 [1.6]
Answers 11393 [17.2] 11807 [18.7]
Comments 4901 [7.4] 5509 [8.7]
PW 1 PW 2
Engagement with PeerWise
Engagement with PeerWise
Engagement with PeerWise
Engagement with PeerWise
Generally, students did
• Participate beyond minimum requirements • Engage in community learning, correcting errors • Create problems, not exercises • Provide positive feedback
Generally, students did not • Contribute trivial or irrelevant questions • Obviously plagiarize • Participate much beyond assessment periods • Leave it to the last minute (sort of….)
58
Correlation with learning
59
Correlation with learning
60
Correlation with learning
61
62
Question/Explanation Quality
Bloom’s Taxonomy of levels in the cognitive domain
Score Level Description
1 Remember Factual knowledge, trivial plugging in of numbers
2 Understand Basic understanding of content
3 Apply Implement, calculate / determine. Typically one-stage problem
4 Analyze Typical multi-step problem; requires identification of strategy
Evaluate Compare &assess various option possibilities; often conceptual
Synthesize Ideas and topics from disparate course sections combined. Significantly challenging problem.
Score Level Description
1 Remember Factual knowledge, trivial plugging in of numbers
2 Understand Basic understanding of content
3 Apply Implement, calculate / determine. Typically one-stage problem
4 Analyze Typical multi-step problem; requires identification of strategy
Evaluate Compare & assess various option possibilities; often conceptual
Synthesize Ideas and topics from disparate course sections combined. Significantly challenging problem.
63
Textp>0.05, NS
64
Question/Explanation Quality
Score Level Description
0 Missing No explanation provided or explanation incoherent/irrelevant
1 Inadequate Wrong reasoning and/or answer; trivial or flippant
2 Minimal Correct answer but with insufficient explanation/justification/ Some aspects may be unclear/incorrect/confused.
3 Good Clear and detailed exposition of correct method & answer.
4 ExcellentThorough description of relevant physics and solution strategy. Plausibility of all answers considered. Beyond normal expectation for a correct solution
65
66
Results (UoE 2010-11)
2 successive years of the same course (N=150, 350)!
‘High quality’ questions: 78%, 79%!
Over 90% (most likely) correct, and 3/5 of those wrong were !
identified by students. !
69% (2010) and 55% (2011) rated 3 or 4 for explanations!
Only 2% (2010) and 4% (2011) rated 1/ 6 for taxonomic level. !
67
Bottomley & Denny Biochem and Mol Biol Educ. 39(5) 352-361 (2011)
107 Year 2 biochem students 56 / 35 / 9 % of questions in lowest 3 levels.
Momsen et al CBE-Life Sci Educ 9, 436-440 (2010)
“9,713 assessment items submitted by 50 instructors in the United States reported that 93% of the questions asked on examinations in introductory biology courses were at the lowest two levels of the revised Bloom’s taxonomy”
Comparison with literature
68
69
Assessment: exams
IAD Course Organisers Forum – Edinburgh, Oct 2015
Two-stage exams!assessments of, for and as learning !
[email protected]@simonpbatesbit.ly/batestalks
Disclaimers!• Not ours; not new
• Similar to elements of well-established pedagogy e.g. TBL
• But….. Significant due to ease, effectiveness and take-up
Overview• Define two-stage exams (UBC style)
• Discuss advantages and disadvantages of two-stage exams
• Take your questions on logistics of administering them
• Describe some research done with two-stage exams at UBC
Two-stageExamsThe basics: Summative assessments During the exam (midterms / final / in-class test)
– Students complete then hand in individual exam – Get into groups of 4 to work on a group exam (for about ½
the time of the individual portion)
What they look (and sound!) like http://www.cwsei.ubc.ca/resources/SEI_video.html
Two-stageexams• The Group Exam is identical to the individual exam
(+/- one or two “difficult” questions)
• Students work on the group exam collaboratively Consensus
• The group only gets one exam sheet!
Two-stageexamsatUBCAt UBC: now well over 100 courses
Physics, Chemistry, Biology, Math, Statistics, Earth, Ocean, and Atmospheric Sciences, Computer Science, Forestry, Pharmacy, Psychology, and Land and Food Systems
Many class formats / assessment types
900 student final exams (that was me J ) 450 student 1st year lectures 20 person laboratories 20 student 4th year seminars 5 student graduate classes
Advantages• Immediate feedback • Models classroom behaviour • Engaged students • Teaching!
AdvantagesYour list: • Nearly instant feedback / reflection • More collaboration • Engagement • Responsibility • Different perspectives • Happier students learn better • Develops group work skills • Lower achieving students get extra explanation • Reduces anxiety
Advantages• Higher achieving students benefit from explaining and
reinforcing knowledge • ALL students participate! • Quieter students get a chance to contribute even in large
classes • Students like it and believe it helps their learning
SomeStudentCommentsPhys250andEOSC114
“Some problems, its a good way to find out what you did wrong on the individual exam
almost immediately.”
“Great idea! The group exams give you a chance to go over your answers to the exam
while you still care about the questions.”
SomeStudentCommentsPhys250andEOSC114
“Discussion over tricky questions facilitate learning immediately and the answer/
concept is stuck in your brain FOREVER!”
“You actually learn what you got wrong right away from a student
perspective”
DisadvantagesYour list Compressing the grades Group composition Loud students No agreement
PossibleDisadvantages• “Only certain subjects” • Social loafing • Dominant group members • Sidetracked by process • Assigning marks • A & D • Student inexperience with groups! • Time! Shorter exams • Getting “convinced” of wrong answers • Incompatible with curving as standard
SomeStudentCommentsPhys250andEOSC114
“… That said, there was the issue of excessive discussion in the group exam. That is, there were
several times where a part of a question was contentious within our group and the ensuing debate, frequently ended only by calling over
<instructor>, often took up so much time that doing the last few questions was hurried and messy.”
FeedbackfromEOSC114“Group Exams are…”
Posi)ve‘because…’comments
Discussion 48 Learn why you were wrong 37 New perspectives 29 Better grades 21 Instant feedback 16 Review 10 Build confidence 8 Understand questions better 6 Learn techniques from others 6 Other 27 Total 208
Nega)ve‘because…’comments
Coming to consensus 21 Time consuming 13 Unbalanced knowledge in group 6 Convinced of wrong answer 3 Realize did poorly individually 3 Worth too much 2 Other 8 Total 56
HowGroupsChoose(EOSC111)
Researchevidenceforeffec)veness
Gilley, B.H. and Clarkston, B. (2014) Collaborative Testing: Evidence of Learning in a Controlled In-Class Study of Undergraduate Students Journal of College Science Teaching Vol. 43 No. 3
Resources• Journal articles:
– Gilley & Clarkston (2014) Journal of College Science Teaching Vol. 43 No. 3
– Rieger & Heiner (2014) Journal of College Science Teaching Vol. 43 No. 4
– Wieman, Rieger & Heiner (2014) Phys. Teach. Vol 52
• For more information and videos visit: http://blogs.ubc.ca/eoassei/two-stage-exams/ • Misc articles:
http://www.macleans.ca/education/multiple-choice-multiple-students/ http://blogs.ubc.ca/catherinerawn/2014/07/22/two-stage-exam-introduction-and-resources/ https://teachingcommons.stanford.edu/teaching-talk/turn-exam-learning-experience-two-stage-exams http://www.artsrn.ualberta.ca/mengel/huco5002014/?p=356
Resources
www.cwsei.ubc.ca!
http://blog.peerinstruction.net/ !
http://flippedlab.learning.ubc.ca/!
http://diy.open.ubc.ca/!