Martin Greenhow & Abdulrahman Kamavi

Mathematical SciencesBrunel University

maths e.g. – a web assessment application for STEM and beyond

OutlineOutline1. CAA at Brunel University2. Characteristics of questions and variety

of disciplines3. Mathematics for Economics (Metal

Project)4. Students and CAA: perceptions and Students and CAA: perceptions and

realityreality

5. maths e.g.6. Conclusions

CAA at Brunel UniversityCAA at Brunel UniversityMathletics – not that Mathletics!Written within Question Mark Perception version 3.4Javascript, MathML, SVGQuestions database spans GCSE, A-level, undergraduate topicsQuestions database spans MC, NI, RNI, TFU, MR,

NI+confidence, Revealed MC etc (no free-form maths input … yet): PROS/CONS?

Hundreds of users take circa 30,00 tests pa. Students from Economics, Electrical and Electronic Engineering, Financial Computing, Foundations of Engineering,

Foundations of IT, PGCE and Sports Science. Much commonalityLow-stakes summative assessmentFormat: best-ever mark from their first 5 attempts counts

towards their module mark; not invigilated and group work is allowed/encouraged. Exam pass required!

TF(U) questiondecontextualisednumbers and

variable names (here d and S) randomised

Fonts & ColoursLow facility – high

discriminationSuccess largely

independent of question type

Computer science question

Pre-processing of input

Table generated on the fly

Required element positions randomised

Feedback – can we have too much?Engineering/built

env/maths question with schematic numbers change

ALL of the formative feedback

Feedback - contextualised

economics question part of the

formative feedbackSVG diagram

realised according to the random parameters in the question

related material button links to any web resource (via centrally-held lookup array)

Biosciences - decontextualised

Nurses - contextualised

The only question with Imperial units!

Realistic weights

Realistic weight loss!

Adult numeracy?

Pictogram changes

Scenario changes

Notice the key -> mal-rule

Mathematics for Economics(year-by-year changes in assessment)

2006-07, 2007-08: no CAA; A and non-A cohorts – results indistinguishable; two class tests, one 3-hour exam set by economics staff (10 days - times 2?)

2008-09: CAA introduced; 3-hour exam only (2 days - times 2?)

2009-10, 2010-11: one cohort, admission AS level mathematics; exam pass mandatory; 2-hour exam only (set by MG) (3 days!!!!!!)

Students’ perceptions 1Students’ perceptions 1 (147 returns)(147 returns)

79% of students use the CAA both for assessment and learning material, 21% for assessment only;

94% use all 5 attempts (first one or two attempts are often used to get feedback);

most students say they would use the assessment if not compulsory;

90% of students aim to achieve a mark of at least 80%;

84% plan to use the CAA as part of their revision;

Students’ perceptions 2Students’ perceptions 2 (147 returns)(147 returns)

failure to distinguish clearly between their enjoyment of maths and that of CAA per se;

the more they enjoy mathematics (or CAA) the more they use the feedback as a learning resource;

weaker students benefited relatively more from the CAA: 24% of ‘good’ students use the CAA only for marks in contrast to ‘weak’ students where this is 15%;

mathematics module is perceived as being of comparable difficulty to other modules maths being: very much easier (4%), easier (33%), about the same (31%), harder (29%), very much harder (3%).

Year-on-year results(% in grade/% mark)

0

10

20

30

40

50

60

70

F D C B A examav

overallav

2006-07

2007-08

2008-09

2009-10

2010-11

DeSTRESS JISC ProjectDeSTRESS JISC ProjectBuilds on the Metal question design methodology and

experiences to test basic stats for social sciencesReleased new material end August 2011Some trials have taken place alreadyHand calculation with realistically-sized data sets –

solution to link with external software such as Excel, SPSS

Real data? Data cannot be ‘designed’ to have certain features or not. Problems with keeping the data and answers current without accessing live web sites (problems of access and communication with the marking scheme).

Another challenging area is the interpretation of charts and graphs

What about literacy?

Some things can be done

Must disable Word

Should enable internet

VERY hard!!!

Javascript, MathML and SVG provide a rich environment for setting objective questions

Positive effects on students’ perceptions and on exam performances

Widely applicable database of questions

MathsE.G. Try it at:

http://www.mathcentre.ac.uk:8081/mathseg/

http://www.mathcentre.ac.uk:8081/mathsegteacher/

works on all browsers, mobile devices(?), link to VLEs