Exploring Interactive Computer Marked Assessments for Logic Constructs in Level 1 Computing

Centre for Open Learning of Mathematics, Science, Computing and Technology (COLMSCT) Centre for Excellence in Teaching and Learning The Open University is incorporated by Royal Charter (RC 000391), an exempt charity in England & Wales and a charity registered in Scotland (SC 038302).

Michael Isherwood

Associate Teaching Fellow, COLMSCT

Presentation Summary

• Project Aims and Background• The Quizzes• Logical OR Analyses• Results and Student Views

Project Aims - summary• Support students in their understanding of new and

difficult parts of M150 (in effect, Block 2)• Confirm understanding prior to tma feedback• Investigate extension to other computing courses• Through better understanding, aid retention

Background• Students find getting to grips with programming difficult• This leads to frustration and dropping out of M150 (and

therefore further computing courses)

Background - TMA02/03 Marks

On-Line Quizzes• 6 Quizzes on 2008J and 2009B

presentations– Structured English– Conditions, truth and trace tables– Basic JavaScript– Selection - the if statement– Repetition - the while statement– Repetition - the for statement

Results following are from 2008J up to March

Example Question Quiz 4 Q2• 71% correct at 1st attempt• Only 4% wrong after 3rd attempt

Note Feedback after 1st attempt

After 2nd attempt the correct answers remain, so student concentrates on incorrect answer(s)

Logical OR

CP5D A person with body temperature outside the range 35oC to 37oC is viewed as requiring medical attention. Within this range the temperature is considered normal. Write a compound condition that evaluates to true for requiring medical attention, using temp for temperature (in oC)

(temp < 35) OR (temp > 37)

• 24% correct 1st attempt and 43% after 3rd attempt

• Some learning as earlier logical AND only 29% after 3rd attempt

Analysis of OROn 1st Attempt

AND and && & OR or || ¦¦ unit

22 5 1 0 56 12 2 0 12 %

• 70% correct in principle with 56% absolutely correct

• 28% incorrectly go for AND etc

•12% of responses included unit (g mm nm oC)

Again an improvement on earlier logical AND question where 62% and 19% are comparative results

No improvement in omitting unit (10% in logical AND) and no difference between unit depending on familiarity or difficulty

(temp < 35) OR (temp > 37)

Comparison Operator AnalysisThe 5 variants are :

(age<17) OR (age>=60) (WL<=400) OR (WL>=700)

(mass<53) OR (mass>=63) (temp<35) OR (temp>37)

(len<=219) OR (len>=221)

• 49% chose < when correct but only 14% chose <= when correct

• 69% chose > when correct but only 28% chose >= when correct

• Some 3% chose =< and/or =>

Interpreting boundary value operators (and values) creates an issue when having to apply from text –

“with body temperature outside the range 35oC to 37oC”

JavaScript Logical OR ExampleSY6D A shop waives address evidence on credit card sales when

the customer's status is that of existing customer or when the purchase amount is under £40.

In the following compound statement expressing this situation drag in the symbols required.

< <= = == > >= != AND OR & && ||

( amount 40 ) ( status 'existing' )

( amount < 40 ) || ( status == 'existing' )

Note that in JavaScript (as in other programming languages) many symbols differ from the pseudocode symbols eg || for OR and == for =

JavaScript Logical OR ConclusionsThis was the last question in this JavaScript quiz and all JavaScript

symbols had featured in earlier questions

• 74% chose “OR” with 65% correct with || and 4% with ¦¦

• not a great improvement on 70% earlier

• 25% chose && (and)

• thus virtually everyone used a logical operator

• < selected by 94%

• == correctly selected by 71% but 22% selected = which is the assignment operator

Are the Quizzes valuable for Learning?• Student Feedback says yes• Many students attempted more than once

– Quiz 2: 304 attempts by 211 students– Quiz 4: 195 attempts by 171 students

• As indicated there is improvement as students progress through question tries and through quizzes

• Ultimate proof in retention and throughput to further computing courses

Student Responses – Pilot 1

• 96% of students found the quiz enjoyable• 91% found the quiz useful• The majority of students didn’t suggest improvements or

make any other comment, however some useful suggestions and observations were made and one or two errors were identified

Student Views – feedback sample• makes it clear where to use capitals and put spaces in compound

statements• Excellent• ..quite short, but long enough to see if I had principles sorted• Very useful especially with the trace tables• It seemed to work very well• Question 7 does not work• Make question 3 less ambiguous• Should include more difficult questions – stage 1 and stage 2• Questions about repetition and selection could be less ambiguous

if construct – Q2 cont

if constructs – Q3•75% of responses correct at 1st attempt• 3% wrong after 3rd attempt.

After a 2nd attempt, the correct answers were retained, as shown, for the final attempt.

The majority of the errors were at the boundary, as here, though several students reversed

if constructs – Q6

On 1st attempt the number of wrong answers is given along with a reminder of the OR (||) operator's effect. The 2nd attempt has feedback as above and the 3rd attempt starts with all the correct responses retained.

A Vision for Student

Studies

Section

Does

Quiz

Next

Section

“Pod” +

Ppt (IC)

Tutor Advice Special

Session