icmas for computing courses centre for open learning of mathematics, science, computing and...

21
iCMAs for Computing Courses Centre for Open Learning of Mathematics, Science, Computing and Technology (COLMSCT) Centre for Excellence in Teaching and Learning The Open University is incorporated by Royal Charter (RC 000391), an exempt charity in England & Wales and a charity registered in Scotland (SC 038302). Michael Isherwood Associate Teaching Fellow, COLMSCT

Upload: erik-wells

Post on 31-Dec-2015

215 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: ICMAs for Computing Courses Centre for Open Learning of Mathematics, Science, Computing and Technology (COLMSCT) Centre for Excellence in Teaching and

iCMAs for Computing Courses

Centre for Open Learning of Mathematics, Science, Computing and Technology (COLMSCT) Centre for Excellence in Teaching and Learning The Open University is incorporated by Royal Charter (RC 000391), an exempt charity in England & Wales and a charity registered in Scotland (SC 038302).

Michael Isherwood

Associate Teaching Fellow, COLMSCT

Page 2: ICMAs for Computing Courses Centre for Open Learning of Mathematics, Science, Computing and Technology (COLMSCT) Centre for Excellence in Teaching and

Presentation Summary

• Project Aims• Background• The Quizzes (including demonstration)• Initial Analyses• Student Views• Developing Questions and Quizzes• A Way Forward• A Vision

Page 3: ICMAs for Computing Courses Centre for Open Learning of Mathematics, Science, Computing and Technology (COLMSCT) Centre for Excellence in Teaching and

Project Aims - summary

• Support students in their understanding of new and difficult parts of M150 (in effect, Block 2)

• Confirm understanding prior to tma feedback• Investigate extension to other computing courses• Through better understanding, aid retention

Page 4: ICMAs for Computing Courses Centre for Open Learning of Mathematics, Science, Computing and Technology (COLMSCT) Centre for Excellence in Teaching and

Background

• Students find getting to grips with programming difficult• This leads to frustration and dropping out of M150 (and

therefore further computing courses)

Page 5: ICMAs for Computing Courses Centre for Open Learning of Mathematics, Science, Computing and Technology (COLMSCT) Centre for Excellence in Teaching and

Background – 2006J TMAs

Page 6: ICMAs for Computing Courses Centre for Open Learning of Mathematics, Science, Computing and Technology (COLMSCT) Centre for Excellence in Teaching and

Background - TMA02/03 Marks

Page 7: ICMAs for Computing Courses Centre for Open Learning of Mathematics, Science, Computing and Technology (COLMSCT) Centre for Excellence in Teaching and

On-Line Quizzes• 6 Quizzes on 2008J and 2009B presentations

– Structured English– Conditions, truth and trace tables– Basic JavaScript– Selection - the if statement– Repetition - the while statement– Repetition - the for statement

• Question Types– Selecting tick boxes, radio buttons

– Free text and expression entry– Entering values– Drag and drop

Page 8: ICMAs for Computing Courses Centre for Open Learning of Mathematics, Science, Computing and Technology (COLMSCT) Centre for Excellence in Teaching and

if constructs – Q1

• 67% correct at 1st try • further 21% at 2nd attempt• 3% correct at 3rd attempt• 10% remained wrong

Most errors were

• < > instead of !=

• = rather than ==

feedback for this Q is to indicate wrong answers. As always, correct answer is shown at the end

Page 9: ICMAs for Computing Courses Centre for Open Learning of Mathematics, Science, Computing and Technology (COLMSCT) Centre for Excellence in Teaching and

if constructs – Q2• 83% correct at 1st attempt• Only 4% wrong after 3rd attempt

Note Feedback after 1st attempt

After 2nd attempt the correct answers remain, so student concentrates on incorrect answer(s)

Page 10: ICMAs for Computing Courses Centre for Open Learning of Mathematics, Science, Computing and Technology (COLMSCT) Centre for Excellence in Teaching and

if construct – Q2 cont

Page 11: ICMAs for Computing Courses Centre for Open Learning of Mathematics, Science, Computing and Technology (COLMSCT) Centre for Excellence in Teaching and

if constructs – Q3•75% of responses correct at 1st attempt• 3% wrong after 3rd attempt.

After a 2nd attempt, the correct answers were retained, as shown, for the final attempt.

The majority of the errors were at the boundary, as here, though several students reversed

Page 12: ICMAs for Computing Courses Centre for Open Learning of Mathematics, Science, Computing and Technology (COLMSCT) Centre for Excellence in Teaching and

if constructs – Q6

On 1st attempt the number of wrong answers is given along with a reminder of the OR (||) operator's effect. The 2nd attempt has feedback as above and the 3rd attempt starts with all the correct responses retained.

Page 13: ICMAs for Computing Courses Centre for Open Learning of Mathematics, Science, Computing and Technology (COLMSCT) Centre for Excellence in Teaching and

Conclusions on Questions• Most students complete quizzes in 10 – 15 minutes• Most/many students get questions right first time and

few fail to be correct after third attempt – there are exceptions

• Most students find the quizzes valuable and the feedback helpful

• Only one question “didn’t work” on 1st pilot - discarded• Variants of a question generally seemed equivalent

Page 14: ICMAs for Computing Courses Centre for Open Learning of Mathematics, Science, Computing and Technology (COLMSCT) Centre for Excellence in Teaching and

iCMAs on Courses • Philosophy is generally to use formatively• However Students need “encouragement”

– hence must form part of assessment • M150 has 10-20% attempting as not part of assessment• S104 has similar number as attempt tma (effectively 100 in

my TG)

• Students take seriously if part of assessment• Courses using iCMAs (some in pilot form) include

MU120, M150, MST121, M256, S104, S342

Page 15: ICMAs for Computing Courses Centre for Open Learning of Mathematics, Science, Computing and Technology (COLMSCT) Centre for Excellence in Teaching and

Building a Quiz for Assessment

• Scope of Quiz• Each Question needs 5 variants – must be equivalent• 3 attempts allowed for each question• Each Question requires feedback after each attempt

– feedback targeted to answer given• therefore must anticipate wrong answers• free text (or symbol) entry much harder to anticipate

• Correct answers may not be apparent, particularly with free text entry

Page 16: ICMAs for Computing Courses Centre for Open Learning of Mathematics, Science, Computing and Technology (COLMSCT) Centre for Excellence in Teaching and

Validating Questions

• Initial testing by Setter• Further testing by Others• Reviewing Student Responses

– for variant equivalence– for errors – then zero rated

Page 17: ICMAs for Computing Courses Centre for Open Learning of Mathematics, Science, Computing and Technology (COLMSCT) Centre for Excellence in Teaching and

Learning from Questions• What is understood• What is misunderstood• How course material might be amended

Page 18: ICMAs for Computing Courses Centre for Open Learning of Mathematics, Science, Computing and Technology (COLMSCT) Centre for Excellence in Teaching and

Moodle and/or OpenMark• OpenMark questions very versatile • It is intended to run OpenMark questions through

Moodle – work is proceding• OpenMark feedback highly tailored• Openmark requires specialist programming

• Moodle has question types• Moodle feedback is available, though fairly rigid• Moodle questions can be set up by anyone who is

reasonably logical – but there is a learning curve

• Within Moodle a mixture of OpenMark and Moodle questions should soon be possible

Page 19: ICMAs for Computing Courses Centre for Open Learning of Mathematics, Science, Computing and Technology (COLMSCT) Centre for Excellence in Teaching and

A Way Forward (i)• Identify Question Setter(s) for each iCMA• Appoint Coordinator / iCMA expert• Workshop(s) on Question and Feedback setting• Set Questions with Feedback• Review Questions• Program Qs (OpenMark and Moodle Qs separately)• Test Quiz } iterative• Make Corrections }• Review Quiz } iterative• Further changes }

Page 20: ICMAs for Computing Courses Centre for Open Learning of Mathematics, Science, Computing and Technology (COLMSCT) Centre for Excellence in Teaching and

A Way Forward (ii)• Coordinator could run workshops• Reviewers may be others associated with Block• Coordinator reviewing could facilitate uniformity of

approach• Perhaps Coordinator could interface with developers• OpenMark Developer would be needed• Moodle Developer probably desirable (could be

Coordinator)

Page 21: ICMAs for Computing Courses Centre for Open Learning of Mathematics, Science, Computing and Technology (COLMSCT) Centre for Excellence in Teaching and

A Vision for Student

Studies

Section

Does

Quiz

Next

Section

“Pod” +

Ppt (IC)

Tutor Advice Special

Session