automatic assessment of student answers for geometric construction questions

Automatic Assessment of Student Answers for Geometric Construction Questions

B U D D H I M A W I J E W E E R A , G I H A N D I A S , S U R A N G I K A R A N AT H U N G A

DEPARTMENT OF COMPUTER SCIENCE AND ENGINEERING

UNIVERSITY OF MORATUWA, SRI LANKA

OutlineIntroduction

Motivation

Problem

Objectives

Literature Survey

System Functionality

System Evaluation

Demonstrating the System

Introduction“A Geometric Construction is an accurate drawing done by a compass and a straightedge following a set instructions to obtain a desired outcome.”

Motivation“Question 9, which was from Geometry, was the least attemptedquestion. Only 12% of the candidates had attempted this question”

H. Silva and B. Banneheka, “Factors Associated with Mathematics Results at GCE (OL) Examination,” Vidyodaya J. of Humanities and Social Sciences, vol. 3, pp. 23–43, 2011.

Motivation

The percentage of candidates that have obtained marks in the range 0 - 2 is 41%

The percentage of candidates that have obtained marks in the range 8 - 10 is 15%

(from candidates who attempted the question)

Source: G.C.E (OL) Examination Evaluation Report - 2011 Mathematics

ProblemStudents should be provided more opportunities to practice geometric constructions

Giving close attention to geometric constructions done by each student is a time consuming task

Teachers have to attend to the weak points of every student during a limited time period

ObjectivesMore engaging and interactive personal learning experience to the student

Automatically assess student answers and provide marks similar to Ordinary Level (Sri Lanka) examination

Provide feedback to improve geometric skills

Literature SurveyBuilding a geometric construction tool

◦ Facts need to consider when creating a geometric construction tool

Software for geometric constructions◦ Current implementations of geometric construction tools and evaluation

Geometric construction representations◦ Proposed representations for describing a geometric construction

Geometric construction validation◦ Validating a geometric construction

Software for Geometric Constructions

Existing software:

Geogebra

Cinderella2

C. a. R (Compass-and-Ruler)

Geometria

Software for Geometric ConstructionsAll of them contains high-level construction tools

Some are desktop-based and platform-specific

Some are not open-source

None of them provide platform for creating a geometric construction learning environment

Geometric Construction RepresentationsGeometric Construction is different from a drawing

Presenting a geometric construction that other software can understand

◦ Predrag Janičić ’s [25] perspective on a geometric construction language

◦ Pedro Quaresma et al. [26] work on creating a common geometric construction language

◦ The Intergeo Consortium’s effort to build I2G standard [28][29]

Geometric Construction ValidationGabriel and Andreas’s [32] introduction to “Compatibility Criterion”

◦ Compatibility Criterion: geometric construction should not violate the restrictions of constructing

In CleverPHL, Christian and Ulrich [33] have implemented the idea of ‘replay constructions’, to validate Geometric Constructions

Most of the above methods are semi-automatic

Our System

Solution

Composition of the system

Representing Marking SchemeMarking scheme (rubric) consists the solution and distribution of marks for the question

To represent the separation of sub-questions, alternative answers, a common format is used

To represent the geometric construction, I2G standard is used

Structure of a Rubric File

I2G FormatElements specify basic entities : Point, Line, Circle

Constraints specify the relation between elements:◦ free_point

◦ line_through_two_points

◦ line_perpendicular_to_line_through_point

◦ point_intersection_of_two_lines

◦ circle_by_center_and_point etc.

Marking Scheme Generation

Representing Student AnswerThe student’s answer is represented using SVG format

Answer can be constructed using any tool that outputs SVG representation

Student Answer Generation

Assessment SystemInfers the student’s answer

Validates student’s answer against rubric

Calculates the best-possible marks distribution

Generates feedback for the answer

Flow of the Assessment system

Rubric String, Answer String

Parse rubric string Parse answer string

Validates Answer

Result String

Inferring the Student’s Answer

Validation : Matching StepsValidation logic goes through each and every step mention in the rubric file

Pick the relevant, identifies elements from the answer (which are left from previous steps)

Identified elements in one step, help to identify elements in next steps

Complex constructions use other elements such as complex-arcs

Always goes through alternative answers (mark-sets), and selects the best one

Validation : Identify ElementsIdentification of Perpendicular Bisectors

Step Description

1 If already found during inferring, return it

2 Identify the possible perpendicular line in

diagram by checking perpendicularity

3 Identify Complex-Arcs which are having centers

similar to given 2 points

4 Among those, select Complex-Arcs which are

intersecting and having same radius

Validation : Identify ElementsOther Elements

Identification of Perpendicular to a Line

Identification of Angular Bisectors

Identification of Parallel Line to a Line◦ Angle-copy method

◦ Rhombus method

◦ Using perpendicular-lines method

Workflow

Sample QuestionQuestion:

Using only a straightedge with a cm/mm scale and a pair of compasses and showing construction lines clearly,

(i) Construct a line segment AB such that AB = 8 cm, and the angle ABC such that ABC = 60 degrees and BC =

10 cm. Complete ABC triangle.

(ii) Construct the perpendicular bisector of AB.

Question Number Answer Marks

(i) Drawing AB or BC 1

460 degree angle construction 1

Completion of triangle 2

(ii) Constructing perpendicular bisector 2 2

6

Sample AnswerCompleted within 16 milliseconds

Final results...

Marks for sub-question 1 :: 4.0 out of 4.0

STEP PASSED:: sub-question: 1, markset: 2, step: 1, marks: 1.0



Marks for sub-question 2 :: 0.0 out of 2.0

STEP FAILED:: sub-question: 2, markset: 1, step: 1

STEP ERROR:: There is an incorrect construction for bisector-line through points

A and B

STEP ERROR:: Please refer: http://www.mathopenref.com/constbisectline.html

STEP ERROR:: Place the compass point on A and B, and draw an arc on each side

of the line without changing the compass width

>> Total marks for the question:: 4.0 out of 6.0

UniquenessConstruction of geometric construction gives close-experience to paper-based environment

Validates student’s answer considering construction process

Gives partial marks and feedback based on student’s answer

Evaluates a large number of answers within a short period of time

Developed based on platform-independent & widely-accepted technologies and standards

System Evaluation

Data sources usedSource No. of Answer Scripts Purpose

Tuition Class 01 15 Testing

Tuition Class 02 10 Testing

Government School 01 – Grade 11 20 Testing



Government School 04 – Grade 11 7 Tuning

Tuition Class 03 5 Tuning

Government School 05 – Grade 10 10 Tuning

Evaluation CriteriaEvaluating Assessing Module

◦ Answers were manually graded according to a marking scheme

◦ Answers were graded using the system

◦ Two grades (manually and by system) were compared to evaluate the system performance

Experienced teachers’ instructions were considered to provide feedback and identify common errors

Answer GradingAccuracy of the system was 97% out of 100 answer scripts graded

Answer scripts belonged to 5 different questions

Marks distribution :◦ 17% students have scored 0 – 3 marks,

◦ 21% have scored 4 – 6 marks

◦ 62% have scored 7 – 10 marks.

◦ From these answer scripts, two students have scored zero marks and 43 students scored full marks.

Demohttps://youtu.be/-_tATk_-eGs

https://youtu.be/-_tATk_-eGs

Future WorkExtending the system to suite other examinations.

◦ This will require modifications such as support for more constraints in rubric file, support for more SVG elements (ellipse, polygon) and validating more construction types.

Feedback generation can be improved

Incorporate student-model, and machine learning techniques◦ This will help to select questions based on student’s skill level

References[1]S. Gulwani, V. A. Korthikanti, and A. Tiwari, “Synthesizing geometry constructions,” in ACM SIGPLAN Notices, 2011, vol. 46, no. 6, pp.

50–61.

[2]“The Math Forum.” [Online]. Available: http://mathforum.org/library/drmath/view/61335.html. [Accessed: 01-Nov-2016].

[3]H. Silva and B. Banneheka, “Factors Associated with Mathematics Results at GCE (OL) Examination,” 2015.

[4]H. P. Tam and Y.-L. Chen, “A regional survey of Taiwan students’ performance in geometric construction,” in Proc. 36th Conf. of the Int.

Group for the Psychology of Mathematics Education, 2012, vol. 4, pp. 131–38.

[5]S. Chakraborty, D. Roy, and A. Basu, “Development of knowledge based intelligent tutoring system,” Advanced Knowledge Based

Systems: Model, Applications and Research, vol. 1, pp. 74–100, 2010.

[6]“Geogebra.” [Online]. Available: https://www.geogebra.org/. [Accessed: 01-Nov-2016].

[7]“Cinderella.” [Online]. Available: http://www.cinderella.de/tiki-index.php. [Accessed: 01-Nov-2016].

[8]“C.a.R. - Compass and Ruler.” [Online]. Available: http://zul.rene-grothmann.de/doc_en/. [Accessed: 01-Nov-2016].

[9]B. B. Bantchev, “A brief tour to dynamic geometry software,” CaMSP February 2015 Newsletter, 2010.

[10]M. El-Demerdash and U. Kortenkamp, “The effectiveness of an enrichment program using dynamic geometry software in developing

mathematically gifted students’ geometric creativity in high schools,” 2010.

[11]M. L. Crowley, “The van Hiele model of the development of geometric thought,” Learning and teaching geometry, K-12, pp. 1–16,

1987.

References (cont.)[12]V. Santos and P. Quaresma, “Integrating DGSs and GATPs in an adaptative and collaborative blended-learning Web-environment,”

arXiv preprint arXiv:1202.4833, 2012.

[13]“Web Geometry Laboratory.” [Online]. Available: http://hilbert.mat.uc.pt/WebGeometryLab/. [Accessed: 01-Nov-2016].

[14]I. A. Elena and P. Manuela, “The Interactive Geometry Software Cinderella.”

[15]V. S. Cisuc and P. Quaresma, “Collaborative environment for geometry,” in Experiment@ International Conference (exp. at’13), 2013

2nd, 2013, pp. 42–46.

[16]G. R. Laigo, A. H. Bhatti, P. Lakshmi Kameswari, and H. M. GebreYohannes, “Revisiting Geometric Construction using Geogebra.”

[17]“Cinderella (software).” [Online]. Available: https://en.wikipedia.org/wiki/Cinderella_(software). [Accessed: 01-Nov-2016].

[18]“Geometria.” [Online]. Available: http://geocentral.net/geometria/. [Accessed: 01-Nov-2016].

[19]“Geometra.” [Online]. Available: https://sourceforge.net/projects/geometra/. [Accessed: 01-Nov-2016].

[20]“GCLC.” [Online]. Available: https://www.emis.de/misc/software/gclc/. [Accessed: 01-Nov-2016].

[21]“Emaths Geometry Toolbox.” [Online]. Available: http://www.emaths.co.uk/images/tutorials/geometrytoolbox.swf. [Accessed: 01-

Nov-2016].

[22]M. F. Inayat and S. N. Hamid, “Integrating New Technologies and Tools in Teaching and Learning of Mathematics: An Overview,”

Journal of Computer and Mathematical Sciences, vol. 7, no. 3, pp. 122–129, 2016.

[23]W3C-SVG-Working-Group, “Scalable Vector Graphics (SVG) 1.1,” 2011.

[24]J. Schiller, “Codedred - SVG Support.” [Online]. Available: http://www.codedread.com/svg-support.php. [Accessed: 01-Nov-2016].

References (cont.)[25]P. Janicic, “Geometry constructions language,” Journal of Automated Reasoning, vol. 44, no. 1–2, pp. 3–24, 2010.

[26]P. Quaresma, P. Janicic, J. Tomasevic, M. Vujosevic-Janicic, and D. Tosic, “XML-based Format for Descriptions of Geometrical Constructions and Geometrical

Proofs,” Communicating Mathematics in Digital Era, CMDE, pp. 15–18, 2006.

[27]S. Gulwani, “Dimensions in program synthesis,” in Proceedings of the 12th international ACM SIGPLAN symposium on Principles and practice of declarative

programming, 2010, pp. 13–24.

[28]S. Egido, M. Hendriks, Y. Kreis, U. Kortenkamp, and D. Marquès, “i2g Common File Format Final Version.”

[29]Intergeo-Consortium, “Interoperable Interactive Geometry for Europe.” [Online]. Available: http://i2geo.net/. [Accessed: 01-Nov-2016].

[30]M. Hendriks, U. Kortenkamp, Y. Kreis, and D. Marques, “Report on Implementations of the Common File Format,” 2009.

[31]P. Quaresma and N. Baeta, “Current Status of the I2GATP Common Format,” in International Workshop on Automated Deduction in Geometry, 2014, pp. 119–

128.

[32]G. J. Stylianides and A. J. Stylianides, “Validation of solutions of construction problems in dynamic geometry environments,” International Journal of

Computers for Mathematical Learning, vol. 10, no. 1, pp. 31–47, 2005.

[33]C. Spannagel and U. Kortenkamp, “Demonstrating, guiding, and analyzing processes in dynamic geometry systems,” in Proceedings of the 9th International

Conference on Technology in Mathematics Teaching. Metz, France: ICTMT, 2009, vol. 9.

[34]L. J. Blanco, “Errors in the teaching/learning of the basic concepts of geometry,” International Journal for Mathematics Teaching and Learning, vol. 24, 2001.

[35]A. Özerem, “Misconceptions in geometry and suggested solutions for seventh grade students,” Procedia-Social and Behavioral Sciences, vol. 55, pp. 720–729,

2012.

[36]C. Alvin, S. Gulwani, R. Majumdar, and S. Mukhopadhyay, “Synthesis of Geometry Proof Problems,” in AAAI, 2014, pp. 245–252.

Questions?

Thank you!