AnalyticalInk: An Interactive Learning Environment for Math Word Problem Solving

Bo Kang, Arun Kulshreshth, Joseph J. LaViola Jr.

University of Central Florida

OutlinePart 1:

● Motivation● Problem Statement● Challenges & Solutions

Part 2:● Semantic Representation, Constraint Solving, Scaffold Auto-Generation.● Tutoring Framework (Verification and Scaffold Adaptation)● Interactive System

Part 3:● Evaluation ● Discussion and Conclusion

Part 1

Motivation- An Example

What is the core difference between these two math geometry problems?

Find the midpoint of the line segment joining A(2,2) and B(4,6) ?

VS

Qualitative versus Quantitative

Thanks to Descartes!

Problem Statement

How could we build an intelligent tutoring system to help students understand analytical geometry math problems, solve them, and provide scaffolding explanations?

Spatial Reasoning(Geometric Concept)

Sequential Reasoning (Algebraic Skills)

An iterative Method(Rittle-Johnson et.al.)

Challenges & Solutions Challenge 1 How could we represent quantitative information from analytical geometry problems? Solution 1 Computational Semantic

Challenge 2 How could we perform quantitative reasoning using the semantic representation?Solution 2 Planning and Constraint Solving

Challenge 3 How could we generate context-sensitive tutoring scaffolds for the problem?Solution 3 Program Synthesis

Challenge 4 How could we analyze user input and give adaptive scaffold feedback?Solution 4 Model Tracing Tutor

Challenge 5 How could we let students interact with the tutoring engine as naturally as possible?Solution 5 Evidence based User Interface and Interactions

Part 2

Solution 1: Problem RepresentationSyntax Level: knowledge entity (such as a point, line or circle), knowledge attribute (y-coordinate of a point, slope of a line), knowledge relation (such as perpendicular lines, parallel lines, distance between two points). These three knowledge components can be related with a label.

Semantic Level: A math problem can be described using the above syntax as a directed acyclic graph.

Human Annotation: (4,3), (2,v), m=?, m*m1=-1, m1=½, v=?.

Solution 2: Quantitative Reasoning ScenariosScenario 1: Forward Reasoning

● Find the distance between two points A(2,3) and B(4,5).

Scenario 2: Backward Reasoning

● There are two points A(2,4) and B(5,v), the distance between A and B is 5. What is the value of v?

Scenario 3:Embedded Reasoning

● Line A passes through two points (4,3) and (2,v). The line is perpendicular to line B in which the slope of line B is 1/2. what is the value of v?

Solution 2: Planning & Constraint Solving

Unary Relation CheckingBinary Relation Checking

Solution 3: Tutoring Scaffold Auto-Generation Executing the solving procedure will fire logic rules. Each fired logic rule maps to a scaffold step, which records two statuses before and after the rule execution, the applied rule, and meta-rule.

Current Rule Types: Arithmetic Rules, Algebraic Rules, Algebraic Equation Rules, Geometry Pattern Match Rules, Reification Rule, Geometrical Unary/Binary Relation Rules.

Meta-Rule: A meta rule is the context-free version of the applied rule. For example, the meta rule for the algebraic distributive law is “Consider using the distributive law.”

Solution 4: Model Tracing Tutor

Scaffolding Graph:

The scaffolding trace is a linear sequence of geometric scaffoldings, where each scaffold could contain a linear sequence of algebraic scaffoldings. The scaffolding trace can be seen as a two-dimensional linear structure.

Solution 4: Stepwise Verification

The match-and-verify algorithm to check the user’s answer using the model-tracing tutor.

Match State on the Scaffolding Trace Graph

Match State on the Semantic DAG

Solution 4: Scaffold Adaptation

The current deterministic method:Status flag to track the current status in the scaffolding trace graph.

Solution 5: AnalyticalInk: User Interface

Solution 5: Keyword Highlighting and Dragging Interaction

There exists two points A(2,4) and B(5,v), the distance between A and B is 5. What is the value of v?

There exists two points A(2,4) and B(5,v), the distance between A and B is 5. What is the value of v?

What we got from dragging upon those visual highlight words?

Solution 5: Input and Reasoning

Reasoning: Inferential reasoning upon the sketch modality (Oviatt et.al.):

● Gesture-based question-mark.

Input (sketching): Algebraic expressions in the algebraic side. Geometric shapes in the geometric side.

Part 3

Preliminary Evaluation : Semantic & SolvingDescriptive Statistics:

● Geometry concept coverage: 15 concepts● Problem coverage: 30 problems

Evaluation Approach:

Two human-tutors graded 30 system-generated tutoring solutions.

Tutoring & User Interaction

1: Tutoring Performance (verification feedback and scaffold adaptation feedback).

2: UI keyword highlighting and dragging.

3: Overall usability.

Initial Evaluation:

For 2, we evaluated its effect using a repeated measure design. We evaluated tutoring performance and overall usability in both conditions. Ten participants solved 10 problems with the keyboard highlighting and dragging, and other 10 problems without it.

Evaluation FindingsTutoring (marginal of keyword highlighting and dragging independent variable):

● Participants used the system to check their answer step.● Participants used the system to ask the next step.● Participants made the error-rectification once during the study session.

UI keyword highlighting and dragging:

● Participants made less query for the next step using this technique. ● Participants made less error-rectification using this technique.● Participants preferred to use with the keyword highlighting and dragging UI.

Evaluation Findings

Overall usability (marginal of keyword independent variable):

● Dual Canvases Effect (log file analysis and post questionnaires)● Participants felt it was effective to use pen-and-gesture input-and-query flow

(5.6 out of 7).● Participants were satisfied with the current tutoring system (5.4 out of 7).

Discussion and Future WorkInteraction:

● Participants felt that the system has constraints to let them freely input and query the knowledge due to the sketch recognition errors in both canvases.

Scaffold Adaptation:

● Though the system showed both meta-scaffold and scaffold explanations, participants felt not that useful and effective toward their problem-solving due to:

Prediction accuracy, repeated scaffold explanations without considering the individualization (such as skipping certain scaffolds using the same KCs) and explanation verbal narratives.

ConclusionAnalyticalInk is an interactive tutoring system toward analytical geometry to facilitate the quantitative reasoning.

Semantic graph representation makes the connection between the algebraic reasoning and the geometric concept reasoning.

The system is useful in terms of interaction techniques and its user experience.

Thank you! Questions?

Back-end source code and data: https://github.com/buptkang/Behavior.Model

Front-end source code: https://bitbucket.org/buptkang/math-tutor-ui

Demo: https://www.youtube.com/watch?v=erDXTB7pHGY

Special thanks to Fei Liu, Pamela Wisniewski and all anonymous reviewers. This work is supported in part by NSF CAREER award IIS-0845921.