Search

Stroke classification for online handwriting and sketches, aimed at grouping strokes into different semantic categories, has drawn considerable attention due to its wide applications. This task is challenging since individual strokes look similar and are easily confused with each other. The key is to consider both the individual strokes and the contextual information jointly for making prediction. Previous methods are insufficient in modeling and exploiting complex contextual information of strokes. To overcome this limitation, we propose a Transformer-based model for Online Handwriting and Sketches (T-OHS), with novel relation encoding schemes to take advantage of temporal and spatial information in stroke sequence. Particularly, we introduce a coarse-to-fine hierarchical encoding approach based on the polar coordinate system for precisely modeling spatial relations between strokes. Experiments on three types of handwriting data, including online handwritten documents, diagrams, and sketches, demonstrate that our method is universal and provides state-of-the-art performance. • We propose a novel stroke classification method for online handwriting and sketches. • We design an effective encoding scheme to represent the spatial relations among strokes. • Extensive experiments demonstrate that our method is universal and provides the SOTA performance. [ABSTRACT FROM AUTHOR]

• We propose a dynamic stroke classification approach, named DyGAT, for online handwritten documents and sketches. Different from previous methods, it can make real-time prediction during writers' writing process. • DyGAT is featured by a novel graph construction algorithm, which enables graph neural networks to better exploit the contextual information under the time constraint. • To evaluate the effectiveness and generality of DyGAT, we apply it to sketch semantic segmentation, document layout analysis, and diagram recognition. Extensive experiments have been conducted to analyze our method and demonstrate its superiority. Online handwriting is widely used in human-machine interface, education, office automation, and so on. Stroke classification for online handwritten documents and sketches aims to divide strokes into several semantic categories and is a necessary step for document recognition and understanding. Previous methods are essentially static in that they have to wait for the user to finish the whole sketch before making prediction. However, in practice, the more user-friendly way is to make real-time prediction as the user is writing. In this paper, we introduce Dynamic Graph ATtention network (DyGAT) to solve the dynamic stroke classification problem. The core of our method is to formalize a document/sketch into a multi-feature graph, in which nodes represent strokes, edges represent the relationships between strokes, and multiple nodes are applied to one stroke to control the information flow. The proposed method is general and is applicable to online handwritten data of many types. We conduct experiments on popular public datasets to perform sketch semantic segmentation, document layout analysis and diagram recognition, and experimental results show competitive performance. Particularly, the proposed method achieves stroke classification accuracies which are only slightly lower than those of static classification. [ABSTRACT FROM AUTHOR]

In this paper, we propose an adaptive spatial pooling method for enhancing the discriminability of feature representation for image classification. The core idea is to adopt a spatial distribution matrix to define how the image patches are pooled together. By formulating the pooling distribution learning and classifier training jointly, our method can extract multiple spatial layouts of arbitrary shapes rather than regular rectangular regions. By proper mathematical transformation, the distributions can be learned via a boosting-like algorithm, which improves the efficiency of learning especially for large distribution matrices. Further, our method allows category-specific pooling operations to take advantage of the different spatial layouts of different categories. Experimental results on three benchmark datasets UIUC-Sports, 21-Land-Use and Scene 15 demonstrate the effectiveness of our method. [ABSTRACT FROM AUTHOR]

Abstract: Semi-Markov conditional random fields (semi-CRFs) are usually trained with maximum a posteriori (MAP) criterion which adopts the 0/1 cost for measuring the loss of misclassification. In this paper, based on our previous work on handwritten Chinese/Japanese text recognition (HCTR) using semi-CRFs, we propose an alternative parameter learning method by minimizing the risk on the training set, which has unequal misclassification costs depending on the hypothesis and the ground-truth. Based on this framework, three non-uniform cost functions are compared with the conventional 0/1 cost, and training data selection is incorporated to reduce the computational complexity. In experiments of online handwriting recognition on databases CASIA-OLHWDB and TUAT Kondate, we compared the performances of the proposed method with several widely used learning criteria, including conditional log-likelihood (CLL), softmax-margin (SMM), minimum classification error (MCE), large-margin MCE (LM-MCE) and max-margin (MM). On the test set (online handwritten texts) of ICDAR 2011 Chinese handwriting recognition competition, the proposed method outperforms the best system in competition. [Copyright &y& Elsevier]

• A framework for joint text/non-text stroke classification and text line grouping in online handwritten documents is proposed. • Stroke classification and text line grouping problems are formulated as node classification/clustering problems in graph. • An edge pooling attention network (EPAT) is proposed to efficiently aggregate information of nodes and edges. • Superior performance of both stroke classification and text line grouping is achieved on public datasets. Stroke classification and text line grouping are important tasks in online handwritten document segmentation. In the past, the two tasks were usually performed using different models which are trained independently and perform sequentially. This cannot optimize the integration of contextual information and the system may suffer from error accumulation in stroke classification. In this paper, we propose a method for joint text/non-text stroke classification and text line grouping in online handwritten documents using attention based graph neural network. In our framework, the stroke classification and text line grouping problems are formulated as node classification and node clustering problems in a relational graph, which is constructed based on the temporal and spatial relationship between strokes. We propose a new graph network architecture, called edge pooling attention network (EPAT) to efficiently aggregate information between the features of neighboring nodes and edges. The proposed model is trained by multi-task learning with cross entropy loss for node classification and distance metric loss for node clustering. In experiments on two online handwritten document datasets IAMOnDo and Kondate, the proposed method is demonstrated effective, yielding superior performance in both stroke classification and text line grouping. [ABSTRACT FROM AUTHOR]