computer aided assessment (caa) for mathematics
DESCRIPTION
Chris Sangwin & Simon HammondTRANSCRIPT
![Page 1: Computer Aided Assessment (CAA) for mathematics](https://reader033.vdocuments.net/reader033/viewer/2022051514/54877b38b47959e70c8b54b3/html5/thumbnails/1.jpg)
Computer Aided Assessment(CAA) for mathematics
Chris Sangwin & Simon Hammond
Copyright c©Last Revision Date: June 1, 2009
![Page 2: Computer Aided Assessment (CAA) for mathematics](https://reader033.vdocuments.net/reader033/viewer/2022051514/54877b38b47959e70c8b54b3/html5/thumbnails/2.jpg)
2
Introduction.
... NOT multiple choice questions ...
• Computer aided assessment (CAA)
• CAA with computer algebra
• Practical issuesImplementations
• Pedagogical issues
• Future directions
![Page 3: Computer Aided Assessment (CAA) for mathematics](https://reader033.vdocuments.net/reader033/viewer/2022051514/54877b38b47959e70c8b54b3/html5/thumbnails/3.jpg)
3
JEM - Joining Educational Mathematics
eContentPlus Thematic Network
http://jem-thematic.net/
Founder members (15):
Universitat Politecnica de Catalunya, Helsingin Yliopisto, Tech-nical University, Jacobs University, Universiteit van Amster-dam, University of Birmingham, FernUniversitt Hagen, Mathsfor More, NAG Ltd, Liguori Editore, ISN Oldenburg GmbH,RWTH Aachen University, Univ. Nacional de Educacin a Dis-tancia, Universitat Oberta de Catalunya, Universidade de Lis-boa.
![Page 4: Computer Aided Assessment (CAA) for mathematics](https://reader033.vdocuments.net/reader033/viewer/2022051514/54877b38b47959e70c8b54b3/html5/thumbnails/4.jpg)
4
Use of objective tests
Consider the following question:
Example question 1Determine the following integral:∫
cos(x) sin(2x)dx.
As a multiple choice question:
◦ (2/3) cos3(x) + C◦ −(2/3) cos(x) + (2/3) sin3(x) + C◦ −(2/3) cos(x) + (1/3) sin(x) sin(2x) + C◦ Don’t know.
How do we know the students don’t differentiate thecandidate solutions to check?
![Page 5: Computer Aided Assessment (CAA) for mathematics](https://reader033.vdocuments.net/reader033/viewer/2022051514/54877b38b47959e70c8b54b3/html5/thumbnails/5.jpg)
5
Computer algebra marking
Computer algebra systems can be used to mark work.
This checks for algebraic equivalence.
(x + 1)2 ≡ x2 + 2x + 1
Useful for marking many routine problems.
![Page 6: Computer Aided Assessment (CAA) for mathematics](https://reader033.vdocuments.net/reader033/viewer/2022051514/54877b38b47959e70c8b54b3/html5/thumbnails/6.jpg)
6
Fundamental idea
if simplify(sa-ta) = 0 thenmark := 1 else mark := 0
![Page 7: Computer Aided Assessment (CAA) for mathematics](https://reader033.vdocuments.net/reader033/viewer/2022051514/54877b38b47959e70c8b54b3/html5/thumbnails/7.jpg)
7
STACK
System overview
The STACK system:
• internet based CAA system,
• uses very simpleMaxima (computer algebra), andLATEX (type setting)
• All components open source (e.g. GPL).
![Page 9: Computer Aided Assessment (CAA) for mathematics](https://reader033.vdocuments.net/reader033/viewer/2022051514/54877b38b47959e70c8b54b3/html5/thumbnails/9.jpg)
9
In learning and teaching
We are assessing a student provided answer.
This is an objective test.
This is
• not Multiple Choice Question;
• not string/regex match.
Other tests for the form of an answer.
![Page 10: Computer Aided Assessment (CAA) for mathematics](https://reader033.vdocuments.net/reader033/viewer/2022051514/54877b38b47959e70c8b54b3/html5/thumbnails/10.jpg)
10
Input of mathematics
This is a fundamental but unsolved problem.
There are a number of options
1. Strict CAS syntax. eg. 2*(x-1)*(x+1)
2. “informal” linear text syntax. eg. 2(x-1)(x+1)
x(t-1) ?
3. Graphical input tool. eg. equation editor.
4. (Pen-based input ?)
5. (Geometry applet ?)
Not all groups of students are equal.
![Page 11: Computer Aided Assessment (CAA) for mathematics](https://reader033.vdocuments.net/reader033/viewer/2022051514/54877b38b47959e70c8b54b3/html5/thumbnails/11.jpg)
11
Syntax innovations
Difficult to achieve!
Babbage 1830’s
“a profusion of notations [...] which threaten, if not duly cor-rected, to multiply our difficulties instead of promoting our progress”Babbage, C. (1827)
sin2(x) sin−1(x)
sin sin x = sin2 x
(composition)
![Page 12: Computer Aided Assessment (CAA) for mathematics](https://reader033.vdocuments.net/reader033/viewer/2022051514/54877b38b47959e70c8b54b3/html5/thumbnails/12.jpg)
12
Structure in random problem sets
In practice, the numbers often do not matter.
Tuckey, C. O., Examples in Algebra, Bell & Sons, London, (1904)
Too much randomization destroys structure.
An underlying question space.
![Page 13: Computer Aided Assessment (CAA) for mathematics](https://reader033.vdocuments.net/reader033/viewer/2022051514/54877b38b47959e70c8b54b3/html5/thumbnails/13.jpg)
13
Workshop task
Option A:Context: end of first calculus course. (Age 18)
Write 6 questions which test whether a student can differentiateelementary functions.
E.g. Differentiate cos(3x) with respect to x.
Option B:Context: age 11.
Write 6 questions which test whether a student can add frac-tions.
![Page 14: Computer Aided Assessment (CAA) for mathematics](https://reader033.vdocuments.net/reader033/viewer/2022051514/54877b38b47959e70c8b54b3/html5/thumbnails/14.jpg)
14
Randomization
1. What could you randomize?
2. What would you randomize?
3. What are some likely incorrect answers?
4. What feedback would you like to provide?
... with a view to implementing these questions live.
![Page 15: Computer Aided Assessment (CAA) for mathematics](https://reader033.vdocuments.net/reader033/viewer/2022051514/54877b38b47959e70c8b54b3/html5/thumbnails/15.jpg)
15
Issues
• Well-posed questions.
• Fair questions.
• Structure in question sets.Schemes of work, vs isolated questions.
• Algebraic form of answers as a goal.
![Page 16: Computer Aided Assessment (CAA) for mathematics](https://reader033.vdocuments.net/reader033/viewer/2022051514/54877b38b47959e70c8b54b3/html5/thumbnails/16.jpg)
16
Feedback
One third of feedback interventions decreased performance.
Kluger, A. N. and DeNisi, A., Psychological Bulletin (1996).
The nature of feedback determines its effectiveness.
![Page 17: Computer Aided Assessment (CAA) for mathematics](https://reader033.vdocuments.net/reader033/viewer/2022051514/54877b38b47959e70c8b54b3/html5/thumbnails/17.jpg)
17
Processing answers
Test for algebraic equivalence if simplify(sa-ta) = 0 thenmark := 1 else mark := 0
Using mainstream CAS
• Get a lot very quickly,Great for calculus and beyond.
• Elementary algebra can be a problem.
Maxima seems to be more suitable than most.
![Page 18: Computer Aided Assessment (CAA) for mathematics](https://reader033.vdocuments.net/reader033/viewer/2022051514/54877b38b47959e70c8b54b3/html5/thumbnails/18.jpg)
18
Every CAS is different!
Input Maple Maxima Axiom
(numbers)0.5-1/2 0.0 0.0 0.04^(1/2)
√4 2 2
4^(-1/2) 14
√4 1
212
-4^(1/2)√−4 2i 2
√−1
sqrt(-4) 2i 2i 2√−1
(indices)a^n*b*a^m anbam an+mb baman
(a^(1/2))^2 a a a(a^2)^(1/2)
√a2 |a|
√a2
(collecting terms)1+x^2-2*x x2 − 2x + 1 x2 − 2x + 1 x2 − 2x + 1
x/3+1.5*x+1/3 1.833x + .333 · · · 1.833x + 13
1.833x + 0.333 · · ·3*x/4+x/12 5
6x 5x
656x
3/(4*x)+1/(12*x) 56
1x
56x
56x
![Page 19: Computer Aided Assessment (CAA) for mathematics](https://reader033.vdocuments.net/reader033/viewer/2022051514/54877b38b47959e70c8b54b3/html5/thumbnails/19.jpg)
19
Input Maple Maxima Axiom
(brackets)
-1*(x+3) −x− 3 −x− 3 −x− 3
2*(x+3) 2x + 6 2(x + 3) 2x + 6
(2*x-1)/5+(x+3)/2 910
x + 1310
2x−15
+ x+32
910
x + 1310
(x-1)^3/(x-1) (x− 1)2 (x− 1)2 x2 − 2x + 1
(x^2-2*x+1)/(x-1) x2−2x+1x−1
x2−2x+1x−1
x− 1
(9*x^2+3*x)/(3*x) 13
9x2+3xx
9x2+3x3x
3x + 1
(other)
log(x^2) ln(x2
)2 log(x) log
(x2
)log(x^y) ln (xy) y log(x) log (xy)
log(exp(x)) ln (ex) x x
exp(log(x)) x x x
cos(-x) cos(x) cos(x) cos(x)
![Page 20: Computer Aided Assessment (CAA) for mathematics](https://reader033.vdocuments.net/reader033/viewer/2022051514/54877b38b47959e70c8b54b3/html5/thumbnails/20.jpg)
20
Issue: technical problems
• Mixed data types in polynomialsx/3 + 0.5?
• Unary minus (no simplification).
− 11− x
,−1
1− x, or
1x− 1
.
• Display,1. Implicit multiplication, (xy, x · y, x× y)2. i vs j,3.√
x vs x12 .
![Page 21: Computer Aided Assessment (CAA) for mathematics](https://reader033.vdocuments.net/reader033/viewer/2022051514/54877b38b47959e70c8b54b3/html5/thumbnails/21.jpg)
21
Language
Do we have a way to talk about these fine details?
Unhelpful phrases:
• simplify,e.g.221
= 4 or 221000= · · ·?
• “move over”
![Page 22: Computer Aided Assessment (CAA) for mathematics](https://reader033.vdocuments.net/reader033/viewer/2022051514/54877b38b47959e70c8b54b3/html5/thumbnails/22.jpg)
22
Checking for properties
To mark
Example question 2Give an odd function.
1. calculate f(x) + f(−x),2. simplify,3. check equality to zero.
![Page 23: Computer Aided Assessment (CAA) for mathematics](https://reader033.vdocuments.net/reader033/viewer/2022051514/54877b38b47959e70c8b54b3/html5/thumbnails/23.jpg)
23
Creating examples/instances
Some questions ask for examples of objects.
They require higher level thinking.
Such questions are rare. (11.5 questions from 486 ≈ 2.4%)Pointon and Sangwin, 2003
Perhaps because they are time consuming to mark.
STACK may mark some questions of this style.
Exemplar questions
![Page 24: Computer Aided Assessment (CAA) for mathematics](https://reader033.vdocuments.net/reader033/viewer/2022051514/54877b38b47959e70c8b54b3/html5/thumbnails/24.jpg)
24
Students’ answers
Students show great variety in their answer, and method.
For example, 190 students were asked for two functions thatsatisfy f ′(1) = 0.
Their answers were marked automatically.
![Page 25: Computer Aided Assessment (CAA) for mathematics](https://reader033.vdocuments.net/reader033/viewer/2022051514/54877b38b47959e70c8b54b3/html5/thumbnails/25.jpg)
25
The students (N = 190) gave 93 ‘different’ answers.
1st Answer Freq
uenc
y
2nd Answer Freq
uenc
y
x2 − 2x 45 x3 − 3x 29x2
2 − x 31 x2 − 2x 10x3
3 − x 11 x3
3 − x 9x2 − 2x + 1 7 (x− 1)2 8x2 − 2x + 3 7 x4
4 − x 8(x− 1)2 5 x4 − 4x 52x2 − 4x 5 ex−1 − x 1x3
3 −x2
2 5 ex−1 + e−x+1 10 4 ln(x)− x 1
![Page 26: Computer Aided Assessment (CAA) for mathematics](https://reader033.vdocuments.net/reader033/viewer/2022051514/54877b38b47959e70c8b54b3/html5/thumbnails/26.jpg)
26
Two strategies emerged:
JL: Ok, just take the parabola and shift it one.· · ·B: I said, x− 1 = 0, then integrated it.
![Page 27: Computer Aided Assessment (CAA) for mathematics](https://reader033.vdocuments.net/reader033/viewer/2022051514/54877b38b47959e70c8b54b3/html5/thumbnails/27.jpg)
27
These problems can be used to generate (short) discussions.
• sorting the data,
• methods used,
• ‘exotic’ examples.
f1(x) = 0, f2(x) = |x|(x− 2), f3(x) = e−1
(x−1)2 .
![Page 28: Computer Aided Assessment (CAA) for mathematics](https://reader033.vdocuments.net/reader033/viewer/2022051514/54877b38b47959e70c8b54b3/html5/thumbnails/28.jpg)
28
Automatic feedback
Sophisticated automatic feedback may be provided bycomputer algebra systems.
This
• is immediate,
• is based on properties of students’ answers,
• could be positive and encouraging,
• may be based on common mistakes,
• may be based on common misconceptions.
![Page 29: Computer Aided Assessment (CAA) for mathematics](https://reader033.vdocuments.net/reader033/viewer/2022051514/54877b38b47959e70c8b54b3/html5/thumbnails/29.jpg)
29
Common misconceptions
Computer algebra can also test for a type of incorrect answer.
Misconceptions may be identified by
• educational research,
• previous teaching experience,
• examining answers from previous students
![Page 30: Computer Aided Assessment (CAA) for mathematics](https://reader033.vdocuments.net/reader033/viewer/2022051514/54877b38b47959e70c8b54b3/html5/thumbnails/30.jpg)
30
Odd functions
On examining the odd functions given by students,
the majority of coefficients ( 6= 1) are odd,eg
3x5, 5x7, 7x5 − 3x.
Students’ concept image of an odd function requires odd coeffi-cients.
Furthermore, f(x) = 0 is odd, but was absent.
![Page 31: Computer Aided Assessment (CAA) for mathematics](https://reader033.vdocuments.net/reader033/viewer/2022051514/54877b38b47959e70c8b54b3/html5/thumbnails/31.jpg)
31
Functions that are odd and even.
When asked for a function that was both odd and even
35% gave the correct answer (eventually),35% failed to answer the question.
Incorrect answers revealed that 24% of the students added anodd and even function.
Examples include
x + x2, x2 + x3, x5 − x6.
The computer algebra system can test for these misconceptions.
![Page 32: Computer Aided Assessment (CAA) for mathematics](https://reader033.vdocuments.net/reader033/viewer/2022051514/54877b38b47959e70c8b54b3/html5/thumbnails/32.jpg)
32
Student feedback
What do you like about the system? Did you have any difficul-ties? If so please describe them.
Feedback & partial credit
i like the way that you are given credit if your an-swer is partially correct and also given guidance onachieving the full mark for that question.
I like the fact that feedback is immediate, but I donot like the fact that if I get an answer wrong I donot know where in my working I have made the error
![Page 33: Computer Aided Assessment (CAA) for mathematics](https://reader033.vdocuments.net/reader033/viewer/2022051514/54877b38b47959e70c8b54b3/html5/thumbnails/33.jpg)
33
Random questions
The questions are of the same style and want thesame things but they are subtly different which meansyou can talk to a friend about a certain question butthey cannot do it for you. You have to work it allout for yourself which is good.
Syntax problems
I feel the aim system is reasonably fair, however ihave lost a lot of marks in quiz 3 for simple syntaxerrors
![Page 34: Computer Aided Assessment (CAA) for mathematics](https://reader033.vdocuments.net/reader033/viewer/2022051514/54877b38b47959e70c8b54b3/html5/thumbnails/34.jpg)
34
Give me an example...
Recognising the turning points of the functions pro-duced in question 2 was impressive, as there are alot of functions with stationary points at x=1 andit would be difficult to simply input all possibilitiesto be recognised as answers.
![Page 35: Computer Aided Assessment (CAA) for mathematics](https://reader033.vdocuments.net/reader033/viewer/2022051514/54877b38b47959e70c8b54b3/html5/thumbnails/35.jpg)
35
Authoring questions
In authoring, there is tension:
1. Ability to use all features of CAS.
2. Ease of writing questions.Not making question authors into programmers.
![Page 36: Computer Aided Assessment (CAA) for mathematics](https://reader033.vdocuments.net/reader033/viewer/2022051514/54877b38b47959e70c8b54b3/html5/thumbnails/36.jpg)
36
Conclusion
Some important questions
• For what purposes is this tool useful?
• What properties do we want?– Not “looks correct”.– Not “select the correct answer”.
• What feedback should we give?