Search

• A directed convolutional neural network model is proposed • Partial dilation-Global erosion generates resistance against information loss • Several recent approaches for the segmentation of graphemes/characters are discussed • The over-traces issue is addressed for the first time in the literature • MMOs can provide real solutions if a convenient strategy is adopted Due to the nature of Arabic handwriting, segmenting words into characters/graphemes is the most difficult and critical task of the recognition system. The present paper proposes an approach to segment handwritten Arabic words into graphemes based on a directed Convolutional Neural Network (CNN) and Mathematical Morphology Operations (MMO). Arabic script is cursive, which means that almost all graphemes are connected via horizontal links; therefore, a technique to remove links will facilitate the segmentation of graphemes. In general, an MMO such as erosion seems suitable for getting the job done, but since Arabic handwriting is difficult, MMOs cause information loss and suffer from many issues such as diacritics and over-traces, which lead to over/under/bad segmentations. To overcome limitations, the present paper addresses these issues in the following order: the over-traces issue is addressed for the first time in the literature; a robust algorithm for diacritics extraction is provided; and finally, the main segmentation algorithm adopts a strategy based on a Partial Dilation (PD)-Global Erosion (GE) technique to combat the information loss issue. The PD phase amplifies important regions, while GE eliminates links between graphemes. The complementarity between PD and GE facilitates the extraction of graphemes and creates resistance against information loss. To properly tackle these difficult problems, this article exploits the robustness of CNNs, so a new directed CNN model is suggested. The idea is to draw the model's attention to certain targeted features, which are selected according to the nature of the problem addressed. The proposed directed CNN is used in all phases of the segmentation process. The experimental results are very encouraging and show that the proposed directed CNN model outperformed basic CNN in many experiments. The results also reveal that the followed strategy improved the ability of MMOs to perform segmentation and to compete with other approaches in this research area. [ABSTRACT FROM AUTHOR]

Highlights • We design a semi-supervised feature representation framework for person Re-Identification which effectively utilizes both labeled and unlabeled training data to learn a discriminative representation so that person images across disjoint camera views can be reliably matched. • A multi-view clustering method is proposed to integrate features from multiple Convolutional Neural Networks for clustering, which can give more accurate label estimation for unlabeled data. • Each of our Convolutional Neural Networks utilizes a siamese network that simultaneously computes the identification loss and verification loss, which simultaneously learns a discriminative Convolutional Neural Network embedding and a similarity metric, and thus improving pedestrian retrieval accuracy. Extensive experiments on large-scale person Re-Id datasets demonstrate the effectiveness of our method. Abstract Person Re-Identification (Re-Id) is a challenging task focusing on identifying the same person among disjoint camera views. A number of deep learning algorithms have been reported for this task in fully-supervised fashion which requires a large amount of labeled training data, while obtaining high quality labels for Re-Id is extremely time consuming. To address this problem, we propose a semi-supervised Re-Id framework by using only a small portion of labeled data and some additional unlabeled samples. This paper approaches the problem by constructing a set of heterogeneous Convolutional Neural Networks (CNNs) fine-tuned using the labeled portion, and then propagating the labels to the unlabeled portion for further fine-tuning the overall system. In this work, label estimation is a key component during the propagation process. We propose a novel multi-view clustering method, which integrates features of multiple heterogeneous CNNs to cluster and generate pseudo labels for unlabeled samples. Then we fine-tune each of the multiple heterogeneous CNNs by minimizing an identification loss and a verification loss simultaneously, using training data with both true labels and pseudo labels. The procedure is iterated until the estimation of pseudo labels no longer changes. Extensive experiments on three large-scale person Re-Id datasets demonstrate the effectiveness of the proposed method. [ABSTRACT FROM AUTHOR]

Abstract In streaming data classification, most of the existing methods assume that all arrived evolving data are completely labeled. One challenge is that some applications where only small amount of labeled examples are available for training. Incremental semi-supervised learning algorithms have been proposed for regularizing neural networks by incorporating various side information, such as pairwise constraints or user-provided labels. However, it is hard to put them into practice, especially for non-stationary environments due to the effectiveness and parameter sensitivity of such algorithms. In this paper, we propose a novel incremental semi-supervised learning framework on streaming data. Each layer of model is comprised of a generative network, a discriminant structure and the bridge. The generative network uses dynamic feature learning based on autoencoders to learn generative features from streaming data which has been demonstrated its potential in learning latent feature representations. In addition, the discriminant structure regularizes the network construction via building pairwise similarity and dissimilarity constraints. It is also used for facilitating the parameter learning of the generative network. The network and structure are integrated into a joint learning framework and bridged by enforcing the correlation of their parameters, which balances the flexible incorporation of supervision information and numerical tractability for non-stationary environments as well as explores the intrinsic data structure. Moreover, an efficient algorithm is designed to solve the proposed optimization problem and we also give an ensemble method. Particularly, when multiple layers of model are stacked, the performance is significantly boosted. Finally, to validate the effectiveness of the proposed method, extensive experiments are conducted on synthetic and real-life datasets. The experimental results demonstrate that the performance of the proposed algorithms is superior to some state-of-the-art approaches. [ABSTRACT FROM AUTHOR]

Abstract In this paper, we propose the use of data symmetries, in the sense of equivalences under signal transformations, as priors for learning symmetry-adapted data representations, i.e., representations that are equivariant to these transformations. We rely on a group-theoretic definition of equivariance and provide conditions for enforcing a learned representation, for example the weights in a neural network layer or the atoms in a dictionary, to have the structure of a group and specifically the group structure in the distribution of the input. By reducing the analysis of generic group symmetries to permutation symmetries, we devise a regularization scheme for representation learning algorithm, using an unlabeled training set. The proposed regularization is aimed to be a conceptual, theoretical and computational proof of concept for symmetry-adapted representation learning, where the learned data representations are equivariant or invariant to transformations, without explicit knowledge of the underlying symmetries in the data. [ABSTRACT FROM AUTHOR]

Convolutional neural network (CNN) has demonstrated its superior ability to achieve amazing accuracy in computer vision field. Nevertheless, for practical domain-specific image recognition tasks, it still remains difficult to obtain massive high-quality labeled datasets due to the strong requirements for extensive, tedious manual processing. Inspired by the well-known observation that human brain can accurately recognize objects without relying on massive congeneric examples, we propose a novel deep variance network (DVN) to further enhance the generalization ability of CNN in this paper, which could still produce higher recognition accuracy even with unbalanced training datasets than original CNN. The key idea of our DVN is built upon the intrinsic exploitation of inter-class homogeneity and intra-class heterogeneity. Towards such goal, we make the first attempt to incorporate a hierarchical Bayesian model into the powerful CNN framework, which can transfer the joint feature distribution from certain object’s complete training dataset to other object’s incomplete training dataset in an iterative way. In each training cycle, the CNN-resulted features are clustered into discrimination-related subspaces to guide the learning and adaptive adjustment of homogeneity and heterogeneity over unbalanced training datasets. In practice, we furnish several state-of-the-art deep networks with our proposed DVN, and conduct extensive experiments and comprehensive evaluations over CIFAR-10, MNIST, and SVHN benchmarks. The experiments have shown that, most of the furnished deep networks can benefit from our DVN, wherein they gain at most 6.9% accuracy improvement over CIFAR-10 benchmark, 52.83% error reduction over MNIST benchmark, and an improvement of 6.2% over SVHN datasets. [ABSTRACT FROM AUTHOR]

Blind image quality assessment (BIQA) aims at precisely estimating human perceived image quality with no access to a reference. Recently, several attempts have been made to develop BIQA methods based on deep neural networks (DNNs). Although these methods obtained promising performance, they have some limitations: (1) their DNN models are actually ”shallow” in term of depth; and (2) these methods typically use the output of the last layer in the DNN model as the feature representation for quality prediction. Since the representation depth has been demonstrated beneficial for various vision tasks, it is significant to explore very deep networks for learning BIQA models. Besides, the information in the last layer may unduly generalize over local artifacts which are highly related to quality degradation. On the contrary, intermediate layers may be sensitive to local degradations but will not capture high-level semantics. Thus, reasoning at multiple levels of representation is necessary in the IQA task. In this paper, we propose to extract multi-level representations from a very deep DNN model for learning an effective BIQA model, and consequently present a simple but extraordinarily effective BIQA framework, codenamed BLINDER ( BLind Image quality predictioN via multi-level DEep Representations ). Thorough experiments have been conducted on five standard databases, which show that a significant improvement can be achieved by adopting multi-level deep representations. Besides, BLINDER considerably outperforms previous state-of-the-art BIQA methods for authentically distorted images. [ABSTRACT FROM AUTHOR]

Adaptive graph learning methods for clustering, which adjust a data similarity matrix while taking into account its clustering capability, have drawn increasing attention in recent years due to their promising clustering performance. Existing adaptive graph learning methods are based on either original data or linearly projected data and thus rely on the assumption that either representation is a good indicator of the underlying data structure. However, this assumption is sometimes not met in high dimensional data. Studies have shown that high-dimensional data in many problems tend to lie on an embedded nonlinear manifold structure. Motivated by this observation, in this paper, we develop dual data representations, i.e., original data and a nonlinear embedding of the data obtained via an Extreme Learning Machine (ELM)-based neural network, and propose to use them as the more reliable basis for graph learning. The resulting algorithm based on ELM and Constrained Laplacian Rank (ELM-CLR) further improves the clustering capability and robustness, while retaining the advantages of adaptive graph learning, such as not requiring any post-processing to extract cluster indicators. The empirical study shows that the proposed algorithm outperforms the state-of-the-art graph-based clustering methods on a broad range of benchmark datasets. [ABSTRACT FROM AUTHOR]

Utilizing unlabeled data to train deep neural networks (DNNs) is a crucial but challenging task. In this paper, we propose an end-to-end approach to tackle this problem with consistent inference of latent representations. Specifically, each unlabeled data point is considered as a seed to generate a set of latent labeled data points by adding various random disturbances or transformations. Under the expectation maximization framework, DNNs can be trained in an unsupervised way by minimizing the distances between the data points with the same latent representations. Furthermore, several variants of our approach can be derived by applying regularized and sparse constraints during optimization. Theoretically, the convergence of the proposed method and its variants are fully analyzed. Experimental results show that the proposed approach can significantly improve the performance on various tasks, including image classification and clustering. Such results also indicate that our method can guide DNNs to learn more invariant feature representations in comparison with traditional unsupervised methods. [ABSTRACT FROM AUTHOR]

Category-specific object segmentation has been a long-standing research topic in pattern recognition. This paper presents an unsupervised discriminant shape (UDS) to address category-specific object segmentation by incorporating the proposed shape prior into an intuitive energy minimization framework. Recently, based on the region proposal methods, deep Convolutional Neural Networks (CNNs) provide access to candidate segments in categories of interest from images. However, the segments obtained from bottom-up proposals tend to undershoot or overshoot objects and are easily classified into one specific class. To address this problem, we propose an unsupervised discriminant projection based clustering algorithm (UDC) to obtain more precise shape prior to guide the segmentation, and the class-specific proposals are clustered based on their projections onto the discriminant projection direction. Based on the set of proposals, we then obtain the prior information of foreground UDS with an easy voting scheme. The derived UDS prior is finally utilized in the subsequent energy minimizing formulation based figure-ground segmentation. We conduct extensive and comprehensive evaluations on the MSRC, Object Discovery, Fashionista and PASCAL-S datasets, demonstrating the effectiveness and robustness of the UDS based segmentation. [ABSTRACT FROM AUTHOR]

Recent advances in image classification mostly rely on the use of powerful local features combined with an adapted image representation. Although Convolutional Neural Network (CNN) features learned from ImageNet were shown to be generic and very efficient, they still lack of flexibility to take into account variations in the spatial layout of visual elements. In this paper, we investigate the use of structural representations on top of pretrained CNN features to improve image classification. Images are represented as strings of CNN features. Similarities between such representations are computed using two new edit distance variants adapted to the image classification domain. Our algorithms have been implemented and tested on several challenging datasets, 15Scenes, Caltech101, Pascal VOC 2007 and MIT indoor. The results show that our idea of using structural string representations and distances clearly improves the classification performance over standard approaches based on CNN and SVM with linear kernel, as well as other recognized methods of the literature. [ABSTRACT FROM AUTHOR]

• A novel explainable neural network is proposed for modeling main effects and structured interactions. • The GAMI-Net is a disentangled feedforward network with multiple additive subnetworks. • GAMI-Net takes into account three interpretability constraints: sparsity, heredity, marginal clarity. • An adaptive training algorithm is developed for training GAMI-Net efficiently. • GAMI-Net enjoys superior interpretability and outperforms benchmark methods. The lack of interpretability is an inevitable problem when using neural network models in real applications. In this paper, an explainable neural network based on generalized additive models with structured interactions (GAMI-Net) is proposed to pursue a good balance between prediction accuracy and model interpretability. GAMI-Net is a disentangled feedforward network with multiple additive subnetworks; each subnetwork consists of multiple hidden layers and is designed for capturing one main effect or one pairwise interaction. Three interpretability aspects are further considered, including a) sparsity, to select the most significant effects for parsimonious representations; b) heredity, a pairwise interaction could only be included when at least one of its parent main effects exists; and c) marginal clarity, to make main effects and pairwise interactions mutually distinguishable. An adaptive training algorithm is developed, where main effects are first trained and then pairwise interactions are fitted to the residuals. Numerical experiments on both synthetic functions and real-world datasets show that the proposed model enjoys superior interpretability and it maintains competitive prediction accuracy in comparison to the explainable boosting machine and other classic machine learning models. [ABSTRACT FROM AUTHOR]

• We adapt a traditional non-learnable GED algorithm to the novel paradigm of geometric deep learning. • Triplet network for learning graph distances by means of graph neural networks. • Learning distance in the graph domain without an embedding stage. • Graph-based keyword spotting application with state-of-the-art performance. The emergence of geometric deep learning as a novel framework to deal with graph-based representations has faded away traditional approaches in favor of completely new methodologies. In this paper, we propose a new framework able to combine the advances on deep metric learning with traditional approximations of the graph edit distance. Hence, we propose an efficient graph distance based on the novel field of geometric deep learning. Our method employs a message passing neural network to capture the graph structure, and thus, leveraging this information for its use on a distance computation. The performance of the proposed graph distance is validated on two different scenarios. On the one hand, in a graph retrieval of handwritten words i.e. keyword spotting, showing its superior performance when compared with (approximate) graph edit distance benchmarks. On the other hand, demonstrating competitive results for graph similarity learning when compared with the current state-of-the-art on a recent benchmark dataset. [ABSTRACT FROM AUTHOR]