Search

This paper presents a framework for solving math problems stated in a natural language (NL) and applies the framework to develop algorithms for solving explicit arithmetic word problems and proving plane geometry theorems. We focus on problem understanding, that is, the transformation of a NL description of a math problem to a formal representation. We view this as a relation extraction problem, and adopt a greedy algorithm to extract the mathematical relations using a syntax-semantics model, which is a set of patterns describing how a syntactic pattern is mapped to its formal semantics. Our method yields a human readable solution that shows how the mathematical relations are extracted one at a time. We apply our framework to solve arithmetic word problems and prove plane geometry theorems. For arithmetic word problems, the extracted relations are transformed into a system of equations, and the equations are then solved to produce the solution. For plane geometry theorems, these extracted relations are input to an inference system to generate the proof. We evaluate our approach on a set of arithmetic word problems stated in Chinese, and two sets of plane geometry theorems stated in Chinese and English. Our algorithms achieve high accuracies on these datasets and they also show some desirable properties such as brevity of algorithm description and legibility of algorithm actions.

This paper presents an algorithm for proving plane geometry theorems stated by text and diagram in a complementary way. The problem of proving plane geometry theorems involves two challenging subtasks, being theorem understanding and theorem proving. This paper proposes to consider theorem understanding as a problem of extracting relations from text and diagram. A syntax–semantics (S2) model method is proposed to extract the geometric relations from theorem text, and a diagram mining method is proposed to extract geometry relations from diagram. Then, a procedure is developed to obtain a set of relations that is consistent with the given theorem with high confidence. Finally, theorem proving is conducted by using the existing proving methods which take the extracted geometric relations as input. The experimental results show that the proposed theorem proving algorithm can prove 86% of plane geometry theorems in the test dataset of 200 theorems, which is all the theorems in the popular textbook. The proposed algorithm outperforms the existing algorithms mainly because it can extract relations not only from text but also from diagram.

This paper presents an end-to-end algorithm for solving circuit problems in secondary physics. A key challenge in solving circuit problems is to automatically understand circuit problems over the modals of both text and schematic. Existing methods have a limited capacity in problem understanding due to the they cannot deal with the numerous expressions of problems in natural language and the various circuit diagrams. In fact that this paper, a batch of methods is proposed to work against the challenge of solving circuit problems. The problem understanding is modeled as a problem of relation extraction and a scheme is proposed to extract relations from both text and schematic. A syntax–semantics model is adopted to extract explicit relations from text, whereas a unit-theorem-based method is proposed to extract implicit relations. And a mesh search method is proposed to extract relations from schematic. Based on the result of problem understanding, an algorithm is proposed to produce the solutions of circuit problems, in which the solutions are presented in a readable way. The experimental results demonstrate the effectiveness of the proposed algorithm in solving circuit problems. To the best of our knowledge, this paper is the first literature which reports the quantitative results in understanding and solving circuit problems.

Automatically understanding natural language problems is a long-standing challenging research problem in automatic solving. This paper models the understanding of geometry problems as a problem of relation extraction, instead of as the problem of semantic understanding of natural language. Then it further proposes a supervised machine learning method to extract geometric relations, targeting to produce a group of relations to represent the given geometry problem. This method identifies the actual geometric relations from the relation candidates using a classifier trained from the labelled examples. The formalized geometric relations can then be transformed into the target system-native representations for manipulation in various tasks. Experiments conducted on the test problem dataset show that the proposed method can extract geometric relations at high F1 scores. The comparisons also demonstrate that the proposed method can achieve good performance against the baseline methods. Integrating the automatic understanding method with different geometry systems will greatly enhance the efficiency and intelligence in geometry tutoring.

This paper presents an algorithm for reading digital video clocks reliably and quickly. Reading digital clocks from videos is difficult due to the challenges such as color variety, font diversity, noise, and low resolution. The proposed algorithm overcomes these challenges by using the novel methods derived from the domain knowledge. This algorithm first localizes the digits of a digital video clock and then recognizes the digits representing the time of digital video clock. It is a robust three-step algorithm. The first step is an efficient procedure that directly identifies the region of the second digit at a very low computational cost, which replaces the traditional tedious image processing procedure of identifying the second digit region. The success of the first step mainly leverages on the novel second-pixel periodicity method. Using the acquired second digit region as input, the second step is a clock digit localization procedure. It first acquires the colors of the digits of the digital video clock and performs the color conversion. Then it localizes the remaining clock digits. Finally, the last step is a clock digit recognition procedure. It first employs an enhanced digit-sequence recognition method to robustly recognize the digits on the second; it then adopts a deep learning procedure to recognize the remaining digits. The proposed algorithm is tested on a prepared benchmark of 1000 videos that is publicly available and the experimental results show that it can read digital video clocks with a 100% accuracy at a low computational cost.