ICMAs for Computing Courses Centre for Open Learning of Mathematics, Science, Computing and Technology (COLMSCT) Centre for Excellence in Teaching and.
Post on 31-Dec-2015
Embed Size (px)
iCMAs for Computing Courses Centre for Open Learning of Mathematics, Science, Computing and Technology (COLMSCT) Centre for Excellence in Teaching and Learning The Open University is incorporated by Royal Charter (RC 000391), an exempt charity in England & Wales and a charity registered in Scotland (SC 038302).
Michael IsherwoodAssociate Teaching Fellow, COLMSCT
Presentation SummaryProject AimsBackgroundThe Quizzes (including demonstration)Initial AnalysesStudent ViewsDeveloping Questions and QuizzesA Way ForwardA Vision
Project Aims - summarySupport students in their understanding of new and difficult parts of M150 (in effect, Block 2)Confirm understanding prior to tma feedbackInvestigate extension to other computing coursesThrough better understanding, aid retention
BackgroundStudents find getting to grips with programming difficultThis leads to frustration and dropping out of M150 (and therefore further computing courses)
Background 2006J TMAs
Background - TMA02/03 Marks
Question Types Selecting tick boxes, radio buttons Free text and expression entry Entering values Drag and drop
if constructs Q1 67% correct at 1st try further 21% at 2nd attempt 3% correct at 3rd attempt 10% remained wrong Most errors were < > instead of != = rather than ==
feedback for this Q is to indicate wrong answers. As always, correct answer is shown at the end
if constructs Q2 83% correct at 1st attempt Only 4% wrong after 3rd attemptNote Feedback after 1st attemptAfter 2nd attempt the correct answers remain, so student concentrates on incorrect answer(s)
if construct Q2 cont
if constructs Q375% of responses correct at 1st attempt 3% wrong after 3rd attempt. After a 2nd attempt, the correct answers were retained, as shown, for the final attempt.
The majority of the errors were at the boundary, as here, though several students reversed
if constructs Q6On 1st attempt the number of wrong answers is given along with a reminder of the OR (||) operator's effect. The 2nd attempt has feedback as above and the 3rd attempt starts with all the correct responses retained.
Conclusions on QuestionsMost students complete quizzes in 10 15 minutesMost/many students get questions right first time and few fail to be correct after third attempt there are exceptionsMost students find the quizzes valuable and the feedback helpfulOnly one question didnt work on 1st pilot - discardedVariants of a question generally seemed equivalent
iCMAs on Courses Philosophy is generally to use formativelyHowever Students need encouragement hence must form part of assessment M150 has 10-20% attempting as not part of assessmentS104 has similar number as attempt tma (effectively 100 in my TG)Students take seriously if part of assessmentCourses using iCMAs (some in pilot form) include MU120, M150, MST121, M256, S104, S342
Building a Quiz for AssessmentScope of QuizEach Question needs 5 variants must be equivalent3 attempts allowed for each questionEach Question requires feedback after each attemptfeedback targeted to answer giventherefore must anticipate wrong answersfree text (or symbol) entry much harder to anticipateCorrect answers may not be apparent, particularly with free text entry
Validating QuestionsInitial testing by SetterFurther testing by OthersReviewing Student Responses for variant equivalence for errors then zero rated
Learning from QuestionsWhat is understoodWhat is misunderstoodHow course material might be amended
Moodle and/or OpenMarkOpenMark questions very versatile It is intended to run OpenMark questions through Moodle work is procedingOpenMark feedback highly tailoredOpenmark requires specialist programming
Moodle has question typesMoodle feedback is available, though fairly rigidMoodle questions can be set up by anyone who is reasonably logical but there is a learning curve
Within Moodle a mixture of OpenMark and Moodle questions should soon be possible
A Way Forward (i)Identify Question Setter(s) for each iCMAAppoint Coordinator / iCMA expertWorkshop(s) on Question and Feedback settingSet Questions with FeedbackReview QuestionsProgram Qs (OpenMark and Moodle Qs separately)Test Quiz} iterativeMake Corrections}Review Quiz} iterativeFurther changes}
A Way Forward (ii)Coordinator could run workshopsReviewers may be others associated with BlockCoordinator reviewing could facilitate uniformity of approachPerhaps Coordinator could interface with developersOpenMark Developer would be neededMoodle Developer probably desirable (could be Coordinator)
A Vision for StudentStudiesSectionDoes QuizNextSectionPod +Ppt (IC)Tutor AdviceSpecialSession