Search

Nowadays, with the development of electronic devices, more and more attention has been paid to camera-based text processing. Different from scene image, the recognition system of document image needs to sort out the recognition results and store them in the structured document for the subsequent data processing. However, in document images, the fusion of text lines largely depends on their semantic information rather than just the distance between the characters, which causes the problem of learning confusion in training. At the same time, for multi-directional printed characters in document images, it is necessary to use additional directional information to guide subsequent recognition tasks. In order to avoid learning confusion and get recognition-friendly detection results, we propose a character-level text detection framework, DetectGAN, based on the conditional generative adversarial networks (abbreviation cGAN used in the text). In the proposed framework, position regression and NMS process are removed, and the problem of text detection is directly transformed into an image-to-image generation problem. Experimental results show that our method has an excellent effect on text detection of camera-captured document images and outperforms the classical and state-of-the-art algorithms.

In ground-based remote sensing cloud image observation, images with the highest possible resolution are captured to obtain sufficient information about clouds. However, when features are extracted and classification is performed on the basis of the original images, a high-resolution probably means a high (or even more, unacceptable) computation cost. In practical application, a simple and commonly adopted method is to appropriately resize the original image to a version with a decreased resolution. An inevitable problem is whether useful information is lost in this resizing operation. This paper demonstrates that information loss is inevitable and poor classification results may be obtained from the analysis of local binary pattern (LBP) histogram features. However, this problem has been always neglected in previous studies, and the original image is arbitrarily resized without any criterion. In particular, the histogram features based on LBPs actually reflect the distribution of features. Thus, a criterion based on the Kullback–Leibler divergence between LBP histograms from the original and resized images and a penalty term imposed on the resolution are proposed to select the resolution of the resized image. The optimal resolution of the resized image can be selected by minimizing this criterion. Furthermore, experiments based on three ground-based remote sensing cloud image data sets with different original resolutions validate this criterion by analyzing the LBP histogram features.

Cloud top height (CTH) is an important parameter monitored in atmospheric observations, which has a significant impact on weather prediction, climate models, and flight services. CTH is typically obtained via three ways, namely, satellite, radiosonde, and ground-based radar, with their corresponding strengths and weaknesses. Traditionally, many studies have focused on independent comparison and analysis of CTHs retrieved from different observations. The researches on how to improve the reliability of the CTH by integrating multiple cloud measurements are rare in the literature despite the significance of this strategy to practical meteorological forecast and disaster prevention improvement. An integration technique of different CTHs retrieved from Fengyun 2 (FY-2) meteorological satellite, radiosonde, and ground-based millimeter wavelength cloud radar observations by using Bayesian decision theory is proposed in this study. A dataset is collected in Beijing, China for 12 months from June 2015 to May 2016 to validate the integration effect. Experimental results show that the integration observations improve the accuracy of single observations. Integration observations are more closely correlated with “true” CTH observations than the single observations. These all show the effectiveness of the proposed multiple source data integration strategy.

Feature selection and extraction is the first step in implementing pattern classification. The same is true for ground-based cloud classification. Histogram features based on local binary patterns (LBPs) are widely used to classify texture images. However, the conventional uniform LBP approach cannot capture all the dominant patterns in cloud texture images, thereby resulting in low classification performance. In this study, a robust feature extraction method by learning stable LBPs is proposed based on the averaged ranks of the occurrence frequencies of all rotation invariant patterns defined in the LBPs of cloud images. The proposed method is validated with a ground-based cloud classification database comprising five cloud types. Experimental results demonstrate that the proposed method achieves significantly higher classification accuracy than the uniform LBP, local texture patterns (LTP), dominant LBP (DLBP), completed LBP (CLTP) and salient LBP (SaLBP) methods in this cloud image database and under different noise conditions. And the performance of the proposed method is comparable with that of the popular deep convolutional neural network (DCNN) method, but with less computation complexity. Furthermore, the proposed method also achieves superior performance on an independent test data set.

Different layers of deep convolutional neural networks(CNNs) can encode different-level information. High-layer features always contain more semantic information, and low-layer features contain more detail information. However, low-layer features suffer from the background clutter and semantic ambiguity. During visual recognition, the feature combination of the low-layer and high-level features plays an important role in context modulation. If directly combining the high-layer and low-layer features, the background clutter and semantic ambiguity may be caused due to the introduction of detailed information. In this paper, we propose a general network architecture to concatenate CNN features of different layers in a simple and effective way, called Selective Feature Connection Mechanism (SFCM). Low-level features are selectively linked to high-level features with a feature selector which is generated by high-level features. The proposed connection mechanism can effectively overcome the above-mentioned drawbacks. We demonstrate the effectiveness, superiority, and universal applicability of this method on multiple challenging computer vision tasks, including image classification, scene text detection, and image-to-image translation., The paper is under consideration at Pattern Recognition Letters

In this paper, we propose a simple but effective semantic-based aggregation (SBA) method. The proposed SBA utilizes the discriminative filters of deep convolutional layers as semantic detectors. Moreover, we propose the effective unsupervised strategy to select some semantic detectors to generate the "probabilistic proposals", which highlight certain discriminative pattern of objects and suppress the noise of background. The final global SBA representation could then be acquired by aggregating the regional representations weighted by the selected "probabilistic proposals" corresponding to various semantic content. Our unsupervised SBA is easy to generalize and achieves excellent performance on various tasks. We conduct comprehensive experiments and show that our unsupervised SBA outperforms the state-of-the-art unsupervised and supervised aggregation methods on image retrieval, place recognition and cloud classification., Comment: 10 pages. arXiv admin note: text overlap with arXiv:1705.01247

Text information contained in scene images is very helpful for high-level image understanding. In this study, the authors propose to learn co-occurrence of local strokes for scene text recognition by using a spatiality embedded dictionary (SED). Unlike spatial pyramid partitioning images into grids to incorporate spatial information, the authors SED associates every codeword with a particular response region and introduces more precise spatial information for robust character recognition. After localised soft coding and max pooling of the first layer, a sparse dictionary is learned to model co-occurrence of several local strokes, which further improves classification performance. Experimental results on two scene character recognition datasets ICDAR2003 and CHARS74 K demonstrate that their character recognition method outperforms state-of-the-art methods. Besides, competitive word recognition results are also reported for four benchmark word recognition datasets ICDAR2003, ICDAR2011, ICDAR2013 and street view text when combining their character recognition method with a conditional random field language model.

Existing manifold learning methods are not appropriate for image retrieval task, because most of them are unable to process query image and they have much additional computational cost especially for large scale database. Therefore, we propose the iterative manifold embedding (IME) layer, of which the weights are learned off-line by unsupervised strategy, to explore the intrinsic manifolds by incomplete data. On the large scale database that contains 27000 images, IME layer is more than 120 times faster than other manifold learning methods to embed the original representations at query time. We embed the original descriptors of database images which lie on manifold in a high dimensional space into manifold-based representations iteratively to generate the IME representations in off-line learning stage. According to the original descriptors and the IME representations of database images, we estimate the weights of IME layer by ridge regression. In on-line retrieval stage, we employ the IME layer to map the original representation of query image with ignorable time cost (2 milliseconds). We experiment on five public standard datasets for image retrieval. The proposed IME layer significantly outperforms related dimension reduction methods and manifold learning methods. Without post-processing, Our IME layer achieves a boost in performance of state-of-the-art image retrieval methods with post-processing on most datasets, and needs less computational cost., 12 pages, 5 figures

Scene text recognition has inspired great interests from the computer vision community in recent years. In this paper, we propose a novel scene text-recognition method integrating structure-guided character detection and linguistic knowledge. We use part-based tree structure to model each category of characters so as to detect and recognize characters simultaneously. Since the character models make use of both the local appearance and global structure informations, the detection results are more reliable. For word recognition, we combine the detection scores and language model into the posterior probability of character sequence from the Bayesian decision view. The final word-recognition result is obtained by maximizing the character sequence posterior probability via Viterbi algorithm. Experimental results on a range of challenging public data sets (ICDAR 2003, ICDAR 2011, SVT) demonstrate that the proposed method achieves state-of-the-art performance both for character detection and word recognition.

Recognizing characters extracted from natural scene images is quite challenging due to the high degree of intraclass variation. In this paper, we propose a multi-scale graph-matching based kernel for scene character recognition. In order to capture the inherently distinctive structures of characters, each image is represented by several graphs associated with multi-scale image grids. The similarity between two images is thus defined as the optimum energy by matching two graphs (images), which finds the best match for each node in the graph while also preserving the spatial consistency across adjacent nodes. The computed similarity is suitable to construct a kernel for support vector machine (SVM). Multiple kernels acquired by matching graphs with multi-scale grids are combined so that the final kernel is more robust. Experimental results on challenging Chars74k and ICDAR03-CH datasets show that the proposed method performs better than the state of the art methods.

This paper tries to model spatial layout beyond the traditional spatial pyramid (SP) in the coding/pooling scheme for scene text character recognition. Specifically, we propose a novel method to build a dictionary called spatiality embedded dictionary (SED) in which each code-word represents a particular character stroke and is associated with a local response region. The promising results outperform other state-of-the-art algorithms.

In this paper, we propose a representation method based on local spatial strokes for scene character recognition. High-level semantic information, namely co-occurrence of several strokes is incorporated by learning a sparse dictionary, which can further restrain noise brought by single stroke detectors. The encouraging results outperform state-of-the-art algorithms.

Characters, which are man-made symbols composed of strokes arranged in a certain structure, could provide semantic information and play an indispensable role in our daily life. In this paper, we try to make use of the intrinsic characteristics of characters and explore the stroke and structure-based methods for character recognition. First, we introduce two existing part-based models to recognize characters by detecting the elastic strokelike parts. In order to utilize strokes of various scales, we propose to learn the discriminative multi-scale stroke detector-based representation (DMSDR) for characters. However, the part-based models and DMSDR need to manually label the parts or key points for training. In order to learn the discriminative stroke detectors automatically, we further propose the discriminative spatiality embedded dictionary learning-based representation (DSEDR) for character recognition. We make a comparative study of the performance of the tree-structured model (TSM), mixtures-of-parts TSM, DMSDR, and DSEDR for character recognition on three challenging scene character recognition (SCR) data sets as well as two handwritten digits recognition data sets. A series of experiments is done on these data sets with various experimental setup. The experimental results demonstrate the suitability of stroke detector-based models for recognizing characters with deformations and distortions, especially in the case of limited training samples.

In this paper, we propose a novel method for cross-view action recognition via a continuous virtual path which connects the source view and the target view. Each point on this virtual path is a virtual view which is obtained by a linear transformation of the action descriptor. All the virtual views are concatenated into an infinite-dimensional feature to characterize continuous changes from the source to the target view. However, these infinite-dimensional features cannot be used directly. Thus, we propose a virtual view kernel to compute the value of similarity between two infinite-dimensional features, which can be readily used to construct any kernelized classifiers. In addition, there are a lot of unlabeled samples from the target view, which can be utilized to improve the performance of classifiers. Thus, we present a constraint strategy to explore the information contained in the unlabeled samples. The rationality behind the constraint is that any action video belongs to only one class. Our method is verified on the IXMAS dataset, and the experimental results demonstrate that our method achieves better performance than the state-of-the-art methods.