Search

Problem solving is a hot research area that develops algorithms for solving exercise problems like people. Hence, such algorithms are the crucial technology of developing digital twins such as digital teacher, digital tutor, and digital student. Solving chemistry problems is one of branch in problem solving. This paper proposes an algorithm for solving the organic chemistry problems described with text and chemical structure figure. The algorithm firstly recognizes chemical structure figure using an OSRA method, and then acquires chemical relation from problem text based on S2 method. Finally, it uses a knowledge conversion and matching procedure to solve the problem based on the knowledge obtained in first two steps. In the experiment, this work collects 373 organic chemistry problems described with text and chemical structure figure from People's Education Press (PEP) version exercise books. The experiment results show that the proposed algorithm in this paper is able to solve 86.33% of the problems.

Robust speaker identification of Internet of Things (IoT) becomes an active research problem in the field of biometrics, and it has wide applications in commercial fields and public security. In order to improve the robustness of speaker identification of IoT, we propose a novel Stacked Sparse Denoising Auto-encoder (SSDAE) for robust speaker identification in this paper. Firstly, the i-vector features are estimated based on the universal background model and total variability space model. For the second stage, sparsity constraints and corrupting operations are fused together into the general auto-encoder, and then the single sparse denoising auto-encoder is constructed. Then, the SSDAE is built with multi-hidden sparse denoising auto-encoder layers by layer wisely stacking operation. In the final layer of the designed deep neural network, the Softmax layer is included for classification purpose. The experimental results have shown that the proposed robust speaker identification method achieves better performance in robust speaker identification task than the state-of-the-art approaches for different kinds and levels of noisy inputs.

The development of intelligent technology has given birth to the emergence of new elements and relations in education, as well as the upgrading and transformation of the notification of old ones. An education model can help us better clarifying the core elements and their relations in education, so as to accurately discovering and grasping the primary contradictions and problems in the future work. This paper makes a literature review and finds that there are gaps between theory and technology, technology and industry, which make obstacles to the application of intelligent technology in education. The research first proposes a four-factor education reference model based on the perspective of embodied cognitive theory, and further elaborates the research status of each element in the model and its corresponding education problems. Secondly, the research and application of intelligent technology in solving educational problems are mainly discussed. Then, from three perspectives of teaching, learning, resources and space construction, this paper discusses how to solve the education problems with intelligent technology in recent years. Finally, this paper integrates the existing experience and the development trend of intelligent technology and proposes strategies for advancing the development of intelligent technology in the field of education from two aspects: dialogue between theory and technology, technology landing industry.

Zero Velocity Update (ZUPT) has played a key role in Pedestrian Dead Reckoning (PDR) with inertial measurement units (IMU). However, it is both crucial and difficult to determine ZUPT conditions given complex and varying motion types such as walking, fast walking or running, and different walking habits of distinct people, which have direct and significant impact on the tracking accuracy. In this research we proposed a model based on deep neural networks to determine moments when the ZUPT should be conducted. The proposed model ensures nearly identical performance regardless of different motion types. It has been demonstrated by extensive experiments conducted in three different scenarios that our model can work equally well with different pedestrians and walking patterns, enabling the wide use of PDR in real-world applications.

Many types of data exhibit characteristic of rotational symmetry. Chemical Organic Ring Structure(ORS) Symbol is such a case. In this paper, we focus on offline handwritten chemical ORS Symbols recognition using convolutional neural networks(CNNs), from application point of view, in order to relax the inconvenience and ineffectiveness of the traditional clickand-drag style of interaction when input chemical notations into electronic devices; from scientific point of view, to explore the capacity of rotation invariance of CNNs using data augmentation. We propose a VGGNet-based classifier for offline handwritten chemical ORS Symbols. To evaluate it, a new dataset of 3600 samples are collected of which 90% is for training while 10% is for test. The recognition accuracy is 84.3% with VGGNet-16 and 92.4% with VGGNet-19.

In this paper, we propose an end-to-end trainable system for recognizing handwritten chemical formulae. This system recognize once a time a chemical formula, instead of one chemical symbol or a whole chemical equation, which is in line with people's writing habits, at the same time could help to develop methods for the complicated chemical equations recognition. The proposed system adopts the CNN+RNN+CTC framework, which is one of state of the art methods in imagebased sequence labelling tasks. We extend the capability of the CNN+RNN+CTC framework to interpret 2D spatial relationships (such as 'subscript' existing in chemical formula) by introducing additional labels to represent them. The system evaluated on a self-collected data set of 12,224 samples, achieves the recognition rate of 94.98% at the chemical formula level.

This paper presents a novel approach for automatically extracting implicit quantity relations (IQRs) from Chinese algebra word problems (AWPs). Our algorithm first classifies AWPs into categories and analyzes each sentence in the problem to identify the variables and values, then maps them to equations according to the category, and enables its trivial as shown in Figure 1. Our algorithm is able to discover 15 kinds of IQRs in AWPs with an average accuracy of 88% when the training sample accounts for more than 3% of the total. Our data and code are publicly available.11Our data and code is available at: https://pan.baidu.com/s/1g1OI6EZTHCunrvXSzBuTew,Extraction code: zt1y

In order to analyze facial expression in human-computer interaction, real time face-tracking has become significant research problem. Traditional face tracking methods have achieved good results in some constrained environments (such as good illumination, no background interference, etc.) However, these methods require to design manual facial features depending on researcher’s experience. In addition, lacking ability for generalization problems is worthy of study. The robustness of face tracking in complex scenes is challenging due to fast moving, multi-scale changes, rotation and occlusion, illumination changes, etc. In view of the above considerations, this paper proposes an improved method based on Siamese-Net to optimize for face tracking tasks. Our work mainly includes four aspects. First, the first two convolutional layers of deeper VGG-16 are used to extract feature. So we call our method Siamese-VGG. Second, we report experiment on face tracking using a pre-trained VGG-Face model which is trained by 2.6M images for face recognition and then fine-tuning to acceleration convergence. Third, in this research the same size crops are input to two branches in the framework and then the inner smaller template feature maps are extracted during training. The proposed method in this paper reduce offset losses by this way. Finally, L2 regularization add to the loss function to improve the generalization ability of the model. The experiment results show better robustness and generalization performance over the original algorithm. In complex scenes, the proposed improved method have achieved the great improvements by almost 11% on average overlap. But, the frame rate of improved method is still 18.5fps on the Nvidia GTX1070Ti GPU.The improved method proposed in this paper is more practical in terms of speed and accuracy.

The development of a guide robot with explanatory features has always been a subject of concern. Based on NAO, a guide robot with Chinese and English voice recognition and expression is implemented, which enables visitors to interact with the robot in the exhibition hall. This paper designs interaction scheme, using the exhibition hall of the project team as experimental environment, focusing on how to make robots provide explanation service, especially on understanding the questions from visitors and the emotional expression of answers to visitors. This paper uses existing speech recognition methods and designs the explanation content and the dialog form, which allows the robot to recognize the questions from visitors, and combine speech representation with robot body movements so that the robot can express emotions to a certain extent. Finally, this paper verified the robot in the plane exhibition hall specializing in foreign propaganda, and the robot is able to perform their daily work on explanation, and greatly enhance the interactive experience of visitors.

The educational robotics is a novel technology for preschooler's companion and can be used for lower level education. This paper presents an AI-based robot system for achieving educational aims, such as metacognition tutoring and geometrical thinking training, with characteristics of contextual teaching by mining knowledge from the real world directly. For metacognition tutoring in our system, objects in real world are detected and a set of learning materials associated with the objects is presented for learners. For example, when a cat is detected, the robot will teach learners to pronounce the “Cat” in different languages, and more knowledge about cat will be pushed to the learners. For geometrical thinking training, an automatic questioning-and-answering section is employed to engage the learner to think, which is carried by a voice interaction between learners and robots. In our experiment, a set of specific object images are captured to validate the feasibility and efficiency of the proposed system. Our study indicated that the proposed system succeeded in captivating the children and parents in maximizing the children's desire to explore.

Positive samples of facial actions are much fewer than negative samples in natural conditions. The highly imbalanced class-distributions may cause very slow rate of convergence of error when using neural networks for facial action recognition. Traditional methods tackle this class-imbalance problem by changing data distributions, which is challenging for preventing the loss of useful information. In this paper we tackle this problem by using very deep (>10 layers) architectures to increase the chance that network training has acceptable rate of convergence using highly imbalanced data sets. Experimental results on EmotioNet Challenge data set show that the error rates of very deep covolutional networks converge to 40% after 90 epochs while shallower networks only converge to 60%. The results also show that very deep network outperforms shallower network by 0.2 on accuracy score. The proposed neural networks won the first place of the first track in the automatic detection of action units (AUs) of EmotioNet Challenge.

This paper presents two algorithms for understanding explicit arithmetic word problems (EAWPs) and explicit plane geometry problems (EPGPs) following the sharing approach, respectively. This approach proposed in this paper models understanding math problems as a problem of relation extraction, instead of as the problem of understanding the semantics of natural language. Then it further proposes a syntax-semantics (S2) model method to extract math relations. The S2 model method is very effective in that only 116 models can extract most of relations in EAWPs and that only 48 models can extract most of relations in EPGP texts. The experimental results show that the proposed algorithms can understand EAWPs and EPGPs very well.

The extraction of implicit quantity relations is one of critical steps of building the machine solver for solving arithmetic word problems. Extracting implicit quantity relations is a challenge research problem as partial problem of natural language understanding, which has no mature solution method. To address this challenge, the existing work in the extraction of quantity relations mainly relies on creating problem solving framework and extracting system of linear equations. However, these existing methods can solve the relatively simple problems due to they have no methods to extract implicit quality relations. This paper proposes a novel method for extracting implicit quantity relations, which is achieved by the process of Chinese phrase parse, classification and instantiation method of required general implicit quantity relations with semantic models. A set of experiments were conducted to classify the arithmetic word problems, and an example was taken to illustrate the proposed method of identifying general implicit quantity relations. The experimental results demonstrate that the proposed method is effective in extracting implicit quantity relations.

This paper develops a SDK for reconstructing plane geometry figures in documents, which can be used in multiple cases of the education. This SDK is an indispensable function of solving plane geometry problems automatically and it can save the dear time if users use it to replace the work of drawing plane geometry figures. This SDK consists of two main components which are the recognition and the drawing of plane geometry figures. In the component of recognizing plane geometry figures, it has a set of functions that recognize all the primitive shapes of plane geometry figures from image documents and describes the recognized primitive shapes in XML or latex. In the component of drawing plane geometry figures, it has a set of functions that draw all the primitive shapes described in XML or latex and generate a final figure in JPEG. Users can just edit the XML or latex document describing the original figure to add some primitive shapes into the original figure. As you can know, the developed SDK can be added into the document editing software to enhance its function. The test results demonstrate that our SDK can recognize and draw the plane geometry figures well.

This paper addresses the problem of solving arithmetic word problems that are directly stated in Chinese without any implicit quantity relations. This problem relates with the multiple challenging problems in artificial intelligent area. This paper proposes multiple new methods to overcome the challenges of the problem and forms a four-step algorithm. The first step is to parse the input math problem into phrases with property annotation. The second one is to use a pool of the semantic models to convert a math problem into a set of high-order equations. The third one is a procedure to solve the set of equations. The last step is to convert the machine solution into a humanoid solution. The experimental results on 140 arithmetic word problems from the text books show that the proposed method has a very good potential.

The availability of big data sets on the different learning modes has stimulated the applications of big data technology in education. This article is to introduce relevant application examples of education data mining to show how big data in education help to solve education issues. Firstly it lists the groups of target users and participants involved in education data mining, and the objectives to use data information of each group. It then goes on to illustrate the groups and their objectives further by given examples and finally conclude how big data help to solve education issues through reaching the objective of each group. The examples show that for students big data can generate personalized activities and offer academic advising, for teachers big data can reveal students' characteristics and predict future performance and offer assistance, for educational organizations big data can make scientific decisions.

Developing problem-solving humanoid machines has been being a hot research topic in the circle of developing educational technology due to many valuable and demanding learning services for K-12 education can developed based on such machines. The ultimate goal of our research team is to develop a machine to solve all kinds of problems of all main subjects of K-12 is a huge system, but this paper presents only the preliminary research and development in developing such a machine. First, we give the system design with the function description of main modules. Second, we identify the main research problems to develop such a machine following the process that the machine solves a given problem. Third, we present an approach towards an engine of problem solving. Finally, we discuss the education services that we can develop based on such machines.

This paper develops a HOG-based multistage approach for object detection and object pose recognition for service robots. This approach makes use of the merits of both multi-class and bi-class HOG-based detectors to form a three-stage algorithm at low computing cost. In the first stage, the multi-class classifier with coarse features is employed to estimate the orientation of a potential target object in the image; in the second stage, a bi-class detector corresponding to the detected orientation with intermediate level features is used to filter out most of false positives; and in the third stage, a bi-class detector corresponding to the detected orientation using fine features is used to achieve accurate detection with low rate of false positives. The training of multi-class and bi-class SVMs with their respective features in different levels is described. Experiments in real-world environments have shown that the proposed method is much more accurate than the detection method as it uses only multi-class detector. The proposed method is also much more efficient than the detection method as it uses a bi-class detector for each possible orientation. The approach works well on the scenarios where the SIFT-based detector may fail. The method can achieve real-time object detection, localization, and pose recognition on a P4 2.4GHz PC.

Head pose estimation is one of key steps for human and robot interaction because head pose is highly related with the attention of human to robot. Although head pose estimation has progressed in the past few decades, existing algorithms for visual imagery are not fast enough for smooth human and robot interaction. This paper proposes an algorithm to estimate head pose of a previously unseen head from thermal images, motivated by the fact that thermal imagery provides us the possibility to develop light-weight computing algorithm for estimating head pose. The proposed algorithm has multiple merits. Firstly, it is a fully-automatic algorithm. That does not require any manual initialization. Second, its operation is very simple. It recognizes the head pose from a single image. Third, it is a light-weight computing algorithm. This is mainly because it has a fast head localization procedure. The proposed algorithm can estimate head pose at 110 fps. Last, it is a very robust algorithm in head localization. The performance in head pose estimation is promising.

In this paper, we propose a novel Adaboost template to recognize human upper body poses from disparity images for natural human robot interaction (HRI). First, the upper body poses of standing persons are classified into seven categories of views. For each category, a mean template, variance template, and percentage template are generated. Then, the template region is divided into positive and negative regions, corresponding to the region of bodies and surrounding open space. A weak classifier is designed for each pixel in the template. A new EM-like Adaboost learning algorithm is designed to learn the Adaboost template. Different from existing Adaboost classifiers, we show that the Adaboost template can be used not only for recognition but also for adaptive top-down segmentation. By using Adaboost template, only a few positive samples for each category are required for learning. Comparison with conventional template matching techniques has been made. Experimental results show that significant improvements can be achieved in both cases. The method has been deployed in a social robot to estimate human attentions to the robot in real-time human robot interaction.

A novel stereo vision system for real-time human detection and tracking on a mobile service robot is presented in this paper. The system integrates the individually enhanced stereo-based human detection, HOG-based human detection, color-based tracking, and motion estimation for the robust detection and tracking of humans with large appearance and scale variations in real-world environments. A new framework of maximum likelihood based multi-model fusion is proposed to fuse these four human detection and tracking models according to the detection-track associations in 3D space, which is robust to the possible missed detections, false detections, and duplicated responses from the individual models. Multi-person tracking is implemented in a sequential near-to-far way, which well alleviates the difficulties caused by human-over-human occlusions. Extensive experimental results demonstrate the robustness of the proposed system under real-world scenarios with large variations in lighting conditions, cluttered backgrounds, human clothes and postures, and complex occlusion situations. Significant improvements in human detection and tracking have been achieved. The system has been deployed on six robot butlers to serve drinks, and showed encouraging performance in open ceremony events.

Lift operation is one of critical functions for mobile service robot to move across levels in buildings. Lift operation poses the problem of liftbutton detection and recognition for computer vision. This paper presents a framework for vision-based liftbutton detection and recognition. This problem is challenging due to reflection and complex background. To achieve the robustness in lift operation, we adopt multiple techniques to combat the challenges. First, we propose a multiple partial models method to increase the robustness of button panel detection. Second, we do button recognition by combining structural inference, Hough transform, and multi-symbol recognition techniques. Lastly, we use ultrasonic distance measure devices to aid the vision of the robot. The experimental results show that our framework can achieve the promising results in recognizing both the internal and external buttons of lift.

This paper presents an algorithm for robust time recognition of video clock. The existing OCR algorithms cannot recognize time properly due to digits of time are in very low resolution and blur. To confront the challenges of time recognition, our algorithm employs three techniques. The first one is a digit transition detection, which identifies SECOND transit frames. The second is a digit-sequence recognition, which uses the property that digits in clock appear in cycle of 0 to 9 to form digit sequence. The third is an on-the-fly template creation. Informally, the robustness of our algorithm benefits from the facts that both digit transition detection and digit-sequence recognition are more reliable than direct character recognition. Experimental results show that our algorithm can achieve a high accuracy in recognizing time.

Fall detection for elderly and patient has been an active research topic due to that the healthcare industry has a big demand for products and technology of fall detection. This paper gives a survey of fall detection for elderly and patient, focusing on identifying approaches and principles of the existing fall detection methods. To properly build the classification tree of the methods of fall detection we first study the characteristics of fall. Then according to what sensors and how sensors are used we first divide the methods of fall detection into three approaches: wearable device, ambience device, and camera-based. Further we divide each approach into two to three classes according to the used principles. For each class of algorithm we analyze their merits and demerits. We also give comments on how we can improve some algorithms.

The semantics of live sports video has big value in catering the services demanded by users (e.g. live event alert and on-the-fly language selection). This paper develops a tool-aided approach to acquire semantics for live soccer video. First, we develop a gamelog acquisition tool, which not only acquires accurate gamelogs but also gets rid of the heavy manual work of the text-way input. Then we develop algorithms to detect multiple boundaries of each event. The amount of manual work of our approach is very limited thanks to the good attributes of our gamelog acquisition tool: (a). it visualizes and symbolizes names of players, actions, etc; (b). it also requires fewer key strokes and mouse clicks for acquiring event description compared with text-way input. Experimental results show that we only need about 5% of mouse-clicks or key-strokes, 50% of input time, and 10% of file size, compared with the text-way input in English, and that our system can quickly obtain the event multiple boundaries of live soccer video.

This paper presents an enhanced trajectory-based ball detection and tracking algorithm, which acquires 2.5D ball position with the aid of camera motion recovery (CMR). Informally, CMR enhances the algorithm by obtaining better ball candidates and forming longer ball trajectories via computing 2.5D position of the ball. The algorithm in this paper comprises two phases. In the first phase, we achieve CMR by a procedure combining homography and global motion estimation. In the second phase, we employ the trajectory-based procedure to the sequence of transformed frames. The experimental results show that the algorithm presented in this paper achieves higher accuracy in identifying the ball than our previous trajectory-based algorithm. More importantly, the obtained ball position is 2.5D, i.e. the ball projection position.