Search

As one of standard approaches to train deep neural networks, dropout has been applied to regularize large models to avoid overfitting, and the improvement in performance by dropout has been explained as avoiding co-adaptation between nodes. However, when correlations between nodes are compared after training the networks with or without dropout, one question arises if co-adaptation avoidance explains the dropout effect completely. In this paper, we propose an additional explanation of why dropout works and propose a new technique to design better activation functions. First, we show that dropout can be explained as an optimization technique to push the input towards the saturation area of nonlinear activation function by accelerating gradient information flowing even in the saturation area in backpropagation. Based on this explanation, we propose a new technique for activation functions, gradient acceleration in activation function (GAAF) , that accelerates gradients to flow even in the saturation area. Then, input to the activation function can climb onto the saturation area which makes the network more robust because the model converges on a flat region. Experiment results support our explanation of dropout and confirm that the proposed GAAF technique improves image classification performance with expected properties. [ABSTRACT FROM AUTHOR]

Object detection is an important research problem in computer vision. Convolutional Neural Networks (CNN) based deep learning models could be used for this problem, but it would require a large number of manual annotated objects for training or fine-tuning. Unfortunately, fine-grained manually annotated objects are not available in many cases. Usually, it is possible to obtain imperfect initialized detections by some weak object detectors using some weak supervisions like the prior knowledge of shape, size or motion. In some real-world applications, objects have little inter-occlusions and split/merge difficulties, so the spatio-temporal consistency in object tracking are well preserved in the image sequences/videos. Starting from the imperfect initialization, this paper proposes a new easy-to-hard learning method to incrementally improve the object detection in image sequences/videos by an unsupervised spatio-temporal analysis which involves more complex examples that are hard for object detection for next-iteration training. The proposed method does not require manual annotations, but uses weak supervisions and spatio-temporal consistency in tracking to simulate the supervisions in the CNN training. Experimental results on three different tasks show significant improvements over the initialized detections by the weak object detectors. [ABSTRACT FROM AUTHOR]

With the development of multimedia technology, 3D model has been applied in many fields such as mechanical design, construction industry, entertainment industry, medical treatment and so on. The number of 3D model is becoming more and more in our lives. Therefore, effective automatic management and classification of 3D models become more and more important. In this paper, we propose a dual-meta-learner model based on LSTM to learn the exact optimization algorithm used to train another two learner neural network classifier in the few-shot regime. The parametrization of our model allows it to learn appropriate parameter updates specifically for the scenario where a set amount of updates will be made, while it can also achieve a general initialization of the learner (classifier) network that allows for quick convergence of training. Our method attains state-of-the-art performance by significant margins. [ABSTRACT FROM AUTHOR]

With the rapid growth of wind power penetration into modern power grids, wind speed forecasting (WSF) becomes an increasing important task in the planning and operation of electric power and energy systems. However, WSF is quite challengeable due to its highly varying and complex features. In this paper, a novel hybrid deep neural network forecasting method is constituted. A feature selection method based on mutual information is developed in the WSF problem. With the real-time big data from the wind farm running log, the deep neural network model for WSF is established using a stacked denoising auto-encoder and long short-term memory network. The effectiveness of the deep neural network is evaluated by 10-minutes-ahead WSF. Comparing with the traditional multi-layer perceptron network, conventional long short-term memory network and stacked auto-encoder, the resulting deep neural network significantly improves the forecasting accuracy. [ABSTRACT FROM AUTHOR]

The ever increasing proportion of photovoltaic (PV), which is, in effect, a random and intermittent energy source, makes PV power forecasting increasingly important for power grid stability. Artificial neural networks (ANN) have become one of the commonly utilized methods in PV power prediction. Since there is no ideal theoretical guidance as yet on the determination of the number of hidden layers and hidden units, there are always abundant neurons in traditional neural networks in order to learn as many data characteristics as possible, which often results in overfitting and high computational costs. The dendritic model proposed in recent years has the characteristics of simple structure, fast convergence and better fitting ability. This paper proposes a PV power forecasting model based on the dendritic neuron networks, which seeks to improve the computational efficiency and prediction accuracy. In order to better extract characteristics of different frequencies of the input data, the approach introduces a wavelet transform. Firstly, the data is decomposed into high-frequency and low-frequency components via a wavelet transform. Thereafter, the input data of different frequencies obtained by the decomposition are transmitted respectively to different sub-models. Finally, the results of sub-models are reconstructed to obtain the final output. The proposed PV power forecasting model was tested upon actual photovoltaic datasets. Results obtained through simulation demonstrate significant improvement in terms of accuracy and efficiency. [ABSTRACT FROM AUTHOR]

Solar power is an important renewable energy resource and acts as a major contributor to replacing fossil fuel generators and reducing carbon emissions. However, the intermittent power output due to the uncertain solar irradiance significantly challenges the economic integrations of solar generation within the existing power system, which calls for effective forecasting methods to improve the solar prediction accuracy. In this paper, a novel improved radial basis function neural network model is proposed and applied in forecasting the short-term solar power generation. A recent proposed meta-heuristic approach named competitive swarm optimization is adopted to train the non-linear and linear parameters of the radial basis function neural network model. The proposed model has been validated in nonlinear benchmark functions and then employed in forecasting the solar power generation of a real-world case study in the Netherlands. Numerical results demonstrate that the proposed competitive swarm optimized radial basis function neural network model could obtain higher accuracy compared to other counterparts and thus provides a useful tool for solar power forecasting. [ABSTRACT FROM AUTHOR]

A common drawback of convolutional features based trackers is the incapability of distinguishing tracking targets from distractors, even in the presence of distinct color difference. In this paper, we design a robust hybrid tracker that explicitly combines color features with convolutional features. We design our tracker on the basis of fully convolutional Siamese network (SiamFC) to emphasize the performance promotion by introducing color features instead of using a more advanced network architecture. A novel approach to integrate two score maps from different channels is proposed. Techniques include cropping out the ineffective area and denoising via Gaussian smoothing. Experiments conducted on OTB2015 and VOT2018 benchmarks show the superiority of our hybrid tracker over the original SiamFC. [ABSTRACT FROM AUTHOR]

Visual cortex is able to process information in multiple pathways and integrate various forms of representations. This paper proposed a bio-inspired method that utilizes the line-segment-based representation to perform a dedicated channel for the geometric feature learning process. The extracted geometric information can be integrated with the original pixel-based information and implemented on both the convolutional neural networks (SegCNN) and the stacked autoencoders (SegSAE). Segment-based operations such as segConvolve and segPooling are designed to further process the extracted geometric features. The proposed models are verified on the MNIST dataset, Caltech 101 dataset and QuickDraw dataset for image classification. According to the experimental results, the proposed models can facilitate the classification accuracies especially when the sizes of the training set are limited. Particularly, the method based on multiple representations is found to be effective for classifying the hand-drawn sketches. [ABSTRACT FROM AUTHOR]

We investigate in this paper the architecture of deep convolutional networks. Building on existing state of the art models, we propose a reconfiguration of the model parameters into several parallel branches at the global network level, with each branch being a standalone CNN. We show that this arrangement is an efficient way to significantly reduce the number of parameters while at the same time improving the performance. The use of branches brings an additional form of regularization. In addition to splitting the parameters into parallel branches, we propose a tighter coupling of these branches by averaging their log-probabilities. The tighter coupling favours the learning of better representations, even at the level of the individual branches, as compared to when each branch is trained independently. We refer to this branched architecture as "coupled ensembles". The approach is generic and can be applied to almost any neural network architecture. With coupled ensembles of DenseNet-BC and parameter budget of 25 M, we obtain error rates of 2.92%, 15.68% and 1.50% on CIFAR-10, CIFAR-100 and SVHN, respectively. For the same parameter budget, DenseNet-BC has an error rate of 3.46%, 17.18%, and 1.8%, respectively. With ensembles of coupled ensembles, of DenseNet-BC networks, with 50M total parameters, we obtain error rates of 2.72%, 15.13% and 1.42%, respectively on these tasks. [ABSTRACT FROM AUTHOR]

In this paper, the control objective is driving the output of a discrete-time switched constrained system with uncertainty to track a desired output of reference by an optimal manner. A new augmented switched system with discounted cost function is constructed based on the switched and reference dynamics, which converts the complex tracking problem to a stabilizing robust control optimization problem. Combining the two stage optimization and iteration learning technique, the overall optimal hybrid policy is first achieved for the constrained switched tracking control. Instead of the general critic-actor structure, only critic neural network (NN) is applied in the algorithm to simply the architecture and manner of implementation. As the main computational burden or load in iteration learning process comes from the information transmissions of tuning NNs, the designed critic-only structure can reduce computational load with less transmissions. Then the convergence of the iteration learning process is demonstrated by theorems and the tracking objective is achieved as the output tracking errors get converged to zero. Finally, the proposed robust tracking control scheme for constrained-input switched systems is applied in the simulation, and the tracking results proved the effectiveness and applicability of the designed method. [ABSTRACT FROM AUTHOR]

Semantic image segmentation requires large amounts of pixel-wise labeled training data. Creating such data generally requires labor-intensive human manual annotation. Thus, extracting training data from video games is a practical idea, and pixel-wise annotation can be automated from video games with near perfect accuracy. However, experiments show that models trained using raw video-game data cannot be directly applied to real-world scenes because of the domain shift problem. In this paper, we propose a domain-adaptive network based on CycleGAN that translates scenes from a virtual domain to a real domain in both the pixel and feature spaces. Our contributions are threefold: 1) we propose a dynamic perceptual network to improve the quality of the generated images in the feature spaces, making the translated images are more conducive to semantic segmentation; 2) we introduce a novel weighted self-regularization loss to prevent semantic changes in translated images; and 3) we design a discrimination mechanism to coordinate multiple subnetworks and improve the overall training efficiency. We devise a series of metrics to evaluate the quality of translated images during our experiments on the public GTA-V (a video game dataset, i.e., the virtual domain) and Cityscapes (a real-world dataset, i.e., the real domain) and achieved notably improved results, demonstrating the efficacy of the proposed model. [ABSTRACT FROM AUTHOR]

Single-image super-resolution (SISR) has been an important topic due to the demand for high-quality virtual images in the field of visual artificial intelligence. Methods based on deep learning have achieved great success based on the excellent capability of grasping complicated features of deep convolutional networks. The performance can be improved slightly but not obviously by simply widening or deepening the network. In this paper, we propose a merged convolutional network for super-resolution, which extracts more adequate details to restore high-resolution images. We used dense blocks for feature extraction to concatenate deep features with shallow features in depth. We also designed two sub-nets with distinct convolution kernels as different branches of the network, which can widen the network and improve the performance of the system. Finally, we employed sub-pixel layers to avoid feature distortion for up-sampling at the very end. Our method was evaluated using several standard benchmark datasets. The results demonstrate superior performance and good robustness compared with state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Regularization is a very effective algorithm to solve overfitting problem in neural network, which improves the generalization ability of the model. However, their working mechanisms and the impact on the model performance have not been fully explored. In this paper, we study and analyze them using information bottleneck theory and one theory from human brain sensory system. We propose a metric to characterise the encoding length of hidden layers, named as AEntry value. Then, we implement extensive experiments on MNIST and FashionMNIST datasets with several commonly used regularization algorithms, and calculate the corresponding AEntry values. We analyze these results and obtain three conclusions. (1) The introduction of regularization influences the encoding of relative features with prediction task in neural network. The early stopping technique avoids introducing unrelated information with the task into the model by stopping the training process as an appropriate iterations. Laplace, Gaussian and Sparse Response regularizations compress the related representation and improve the performance of neural network by introducing the prior information into the model. In contrast, Dropout, Batch Normalization, and Layer Normalization increase the encoding length of features by adopting redundant representation to improve the performance. (2) The encoding of neural network does not satisfy the data processing inequality of information theory, which is mainly caused by redundant coding of extracted features. (3) The overfitting is caused by introducing irrelative information with the target. These results can give us insight into building more efficient regularization algorithm to improve the performance of neural network model. [ABSTRACT FROM AUTHOR]

With the recent surge in popularity of Convolutional Neural Networks (CNNs), motivated by their significant performance in many classification and related tasks, a new challenge now needs to be addressed: how to accommodate CNNs in mobile devices, such as drones, smartphones, and similar low-power devices? In order to tackle this challenge we exploit the Vision Processing Unit (VPU) that combines dedicated CNN hardware blocks and very low power requirement. The lack of readily available training data and memory requirements are two of the factors hindering the training and accuracy performance of 3D CNNs. In this paper, we propose a method for generating synthetic 3D point-clouds from realistic CAD scene models to enrich the training process for volumetric CNNs. Furthermore, an efficient 3D volumetric object representation Volumetric Accelerator format (VOLA) is employed. VOLA is a sexaquaternary (power-of-four subdivision) tree-based representation which allows for significant memory saving for volumetric data. Multiple CNN models were trained and pruning techniques for the weights were applied to the trained 3D Volumetric Network in order to remove almost 70% of the parameters and outperform the existing state-of-the-art networks. The top performing and efficient model was ported to the Movidius™Neural Compute Stick (NCS). After deployment on the NCS, it takes 11 ms (∼ 90 frames per second) to perform inference on each input volume, with a reported power requirement of 1.2 W, which leads to 75.75 inferences per second per Watt. [ABSTRACT FROM AUTHOR]

Automated facial expression analysis has a variety of applications in human-computer interaction. Traditional methods mainly analyze prototypical facial expressions of no more than eight discrete emotions as a classification task. However, in practice, spontaneous facial expressions in a naturalistic environment can represent not only a wide range of emotions, but also different intensities within an emotion family. In such situations, these methods are not reliable or adequate. In this paper, we propose to train deep convolutional neural networks (CNNs) to analyze facial expressions explainable in a dimensional emotion model. The proposed method accommodates not only a set of basic emotion expressions, but also a full range of other emotions and subtle emotion intensities that we both feel in ourselves and perceive in others in our daily life. Specifically, we first mapped facial expressions into dimensional measures so that we transformed facial expression analysis from a classification problem to a regression one. We then tested our CNN-based methods for facial expression regression and these methods demonstrated promising performance. Moreover, we improved our method by a bilinear pooling which encodes second-order statistics of features. We showed that such bilinear-CNN models significantly outperformed their respective baseline models. [ABSTRACT FROM AUTHOR]

Cardiac diseases cause abnormal motion dynamics over the cardiac cycle, and can, therefore, be diagnosed by analyzing the myocardial motion of the left ventricle (LV). In this paper, a feature point descriptor using fully convolutional neural networks (FCNs) is proposed and applied to cardiac motion estimation based on graph matching. A fully convolutional network is trained to predict endocardial contours and extract features of points from short-axis cine magnetic resonance (MR) images. An LV graph is constructed using the extracted point features, and a convex graph matching cost function is defined to estimate the point correspondence between images in two given phases. The sparsity and double stochastic constraints are introduced into the cost function, which is optimized iteratively by the alternating direction method of multipliers (ADMM). Finally, the transformation using compact supported radial basis functions with sparsity constraint is employed to estimate the dense displacement field between two cardiac images in two phases based on the correspondence relationship. The performance of the proposed method was evaluated on two public cardiac databases, and the experimental results show that the FCN feature descriptor outperforms traditional feature descriptors in estimating the correspondence between endocardial contours of the LV. For LV motion estimation, the proposed method provides more accurate motion fields than existing graph matching algorithms. [ABSTRACT FROM AUTHOR]

High-resolution (HR) magnetic resonance (MR) imaging is an important diagnostic technique in clinical practice. However, hardware limitations and time constraints often result in the acquisition of anisotropic MR images. It is highly desirable but very challenging to enhance image spatial resolution in medical image analysis for disease diagnosis. Recently, studies have shown that deep convolutional neural networks (CNN) can significantly boost the performance of MR image super-resolution (SR) reconstruction. In this paper, we present a novel CNN-based anisotropic MR image reconstruction method based on residual learning with long and short skip connections. The proposed network can effectively alleviate the vanishing gradient problem of deep networks and learn to restore high-frequency details of MR images. To reduce computational complexity and memory usage, the proposed network utilizes cross-plane self-similarity of 3D T1-weighted (T1w) MR images. Based on experiments on simulated and clinical brain MR images, we demonstrate that the proposed network can significantly improve the spatial resolution of anisotropic MR images with high computational efficiency. The network trained on T1w MR images is able to effectively reconstruct both SR T1w and T2-weighted (T2w) images, exploiting image features for multi-modality reconstruction. Moreover, the experimental results show that the proposed method outperforms classical interpolation methods, non-local means method (NLM), and sparse coding based algorithm in terms of peak signal-to-noise-ratio, structural similarity image index, intensity profile, and small structures. The proposed method can be efficiently applied to SR reconstruction of thick-slice MR images in the out-of-plane views for radiological assessment and post-acquisition processing. [ABSTRACT FROM AUTHOR]

Automatic heart sound auscultation is one of the common used techniques for cardiovascular diseases detection. In this paper, a novel heart sound classification method based on deep learning technologies for cardiovascular disease prediction is introduced, which is mainly comprised three parts: pre-processing, 1-D waveform heart sound patches classification using a deep convolutional neural network (CNN) with attention mechanism, and majority voting for final prediction of heart sound recordings. In order to enhance the information flow of the CNNs, a block-stacked style architecture with clique blocks is employed, and in each clique block a bidirectional connection structure is introduced in the proposed CNN. By using the stacked clique and transition blocks, the proposed CNN achieves both spatial and channel attention leading a promising classification performance. Moreover, a novel separable convolution with inverted bottleneck is utilized to decouple the spatial and channel-wise relevancy of features efficiently. Experiments on PhysioNet/CinC 2016 show that the proposed method obtains a superior classification results and excels in consumption of parameter comparing to state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Deep learning technologies have played more and more important roles in Computer Aided Diagnosis (CAD) in medicine. In this paper, we tackled the problem of automatic prediction of Alzheimer's Disease (AD) based on Magnetic Resonance Imaging (MRI) images, and propose a fully unsupervised deep learning technology for AD diagnosis. We first implement the unsupervised Convolutional Neural Networks (CNNs) for feature extraction, and then utilize the unsupervised predictor to achieve the final diagnosis. In the proposed method, two kinds of data forms, one slice and three orthogonal panels (TOP) of MRI image, are employed as the input data respectively. Experimental results run on all the 1075 subjects in database of the Alzheimer's Disease Neuroimaging Initiative (ADNI 1 1.5T) show that the proposed method with one slice data yields the promising prediction results for AD vs. MCI (accuracy 95.52%) and MCI vs. NC (accuracy 90.63%), and the proposed methods with TOP data yields the best overall prediction results for AD vs. MCI (accuracy 97.01%) and MCI vs. NC (accuracy 92.6%). [ABSTRACT FROM AUTHOR]

Gliomas are the most frequent primary brain tumors, which have a high mortality. Surgery is the most commonly used treatment. Magnetic resonance imaging (MRI) is especially useful to assess gliomas and evaluate the success of the treatment in clinical practice. So accurately segmenting the brain tumor from MRI images is the key to clinical diagnostics and treatment planning. However, a large quantity of data produced by MRI prevents manual segmentation in a reasonable time. So automatic approaches are required for quick and effective segmentation. But the spatial and structural variability among brain tumors bring a challenge to automatically segment MRI images. Deep Convolutional Neural Network (DCNN) can achieve complex function mapping. This contributes to response to the challenge. Therefore, DCNN is the commonly used method in brain tumor segmentation. Although a large number of DCNN based methods have been proposed, these methods mainly aim to improve the quality of the image features extracted by DCNN. Actually, these methods usually ignore the prior knowledge in medical images, and the simple prior knowledge in brain tumor segmentation is that most of the tumor regions in images are left-right asymmetry. Based on this, in this paper, a novel deep convolutional neural network which combines symmetry have been proposed to automatically segment brain tumors. Our neural networks, called Deep Convolutional Symmetric Neural Network (DCSNN), extends DCNN based segmentation networks by adding symmetric masks in several layers. Our proposal was validated in the BRATS 2015 database, and we also give some baselines. The results are evaluated by dice similarity coefficient metric (DSC). And our proposed method achieved a competitive result with average DSC of 0.852. Furthermore, our method only takes about 10.8 s to segment a patient case. Although our method is not the best performance in BRATS 2015 challenge, as far as we know, our method outperforms other recent DCNN-based methods. [ABSTRACT FROM AUTHOR]

Retinal vessel segmentation (RVS) helps the diagnosis of diabetic retinopathy, which can cause visual impairment and even blindness. Some problems are hindering the application of automatic RVS, including accuracy, robustness and segmentation speed. In this paper, we propose a cross-connected convolutional neural network (CcNet) for the automatic segmentation of retinal vessel trees. In the CcNet, convolutional layers extract the features and predict the pixel classes according to those learned features. The CcNet is trained and tested with full green channel images directly. The cross connections between primary path and secondary path fuse the multi-level features. The experimental results on two publicly available datasets (DRIVE: Sn = 0.7625, Acc = 0.9528; STARE: Sn = 0.7709, Acc = 0.9633) are higher than those of most state-of-the-art methods. In the cross-training phase, CcNte's accuracy fluctuations (△ Accs) on DRIVE and STARE are 0.0042 and 0.007, respectively, which are relatively small compared with those of published methods. In addition, our algorithm has faster computing speed (0.063 s) than those listed algorithms using a GPU (graphics processing unit). These results reveal that our algorithm has potential in practical applications due to promising segmentation performances including advanced specificity, accuracy, robustness and fast processing speed. [ABSTRACT FROM AUTHOR]

Machine learning methods are a powerful tool to detect workload peaks and congestion in goods inspection facilities of seaports. In this paper, a time series data of freight inspection volume at the Border Inspections Posts in the Port of Algeciras Bay was used to construct 4 datasets based on different sizes of autoregressive window and several machine learning and ensemble models were used to aid decision-making in the inspection process. Moreover, an aggregation/disaggregation procedure to make predictions was proposed and compared to two different prediction horizons: daily (t + 1) and weekly (t + 7) predictions. In general, results showed that neural networks performed better than any other model independently of the size of the autoregressive window. The result obtained by a weighted average ensemble model was better and statistically significant than any other model. Moreover, the proposed aggregation/disaggregation procedure provided better performance results and more robust in terms of variance than considering daily or weekly predictions. [ABSTRACT FROM AUTHOR]

The optimal generation of initial connection weight parameters and dynamic updating strategies of connection weights are critical for adjusting the performance of back-propagation (BP) neural networks. This paper presents an adaptive fractional-order BP neural network abbreviated as PEO-FOBP for handwritten digit recognition problems by combining a competitive evolutionary algorithm called population extremal optimization and a fractional-order gradient descent learning mechanism. Population extremal optimization is introduced to optimize a large number of initial connection weight parameters and fractional-order gradient descent learning mechanism is designed to update these connection weight parameters adaptively during the evolutionary process of fractional-order BP neural network. The extensive experimental results for a well-known MNIST handwritten digits dataset have demonstrated that the proposed PEO-FOBP outperforms the original fractional-order BP neural network and the traditional integer-order BP neural network in terms of training and testing accuracies. [ABSTRACT FROM AUTHOR]

Data-driven segmentation has been a challenging task in automatic pulmonary nodules detection. Most methods for data-driven segmentation require training examples labeled with segmentation masks. This requirement makes it expensive to annotate new categories, especially in most cases, since only weak label information can be obtained from CT. This paper proposes a new semi-supervised 3D deep neural network to solve this problem. The model outputs all suspicious nodules for a subject by training this network using image-level tag annotations from the DSB dataset and mask annotations from the LUNA dataset. We evaluate our approach in a controlled study on the DSB test dataset. The over-fitting caused by the shortage of training data is alleviated by training two modules alternatively. This model can synchronously extract tag labels and mask labels in a sequence, and combine two objects through one weighted transfer function in the network learning process. The experimental results demonstrate the power of using the proposed method to produce significant improvements in the accuracy of pulmonary nodules detection. [ABSTRACT FROM AUTHOR]

The major disadvantage of supervised methods for action recognition is the need for a large amount of annotated data, where the data is matched to its label accurately. To address this issue, Zero-Shot Learning (ZSL) is introduced. Zero short learning primarily uses data that is synthesized to compensate for lack of training examples. In this paper, two different approaches are proposed for the synthesis of artificial examples for novel classes; namely, inverse autoregressive flow (IAF) based generative model and bi-directional adversarial GAN(Bi-dir GAN). A consequence of the proposed approach is a transductive setting using a semi-supervised variational autoencoder, where the unlabelled data from unseen classes are used to train the model. This enables the generation of novel class examples from textual descriptions. The proposed models perform well in the following settings, namely, i) Standard setting(ZSL), where the test data comes only from unseen classes, and ii) Generalized setting(GZSL), where the test data comes from both seen and unseen classes. In the case of the generalized setting, examples with pseudo labels are generated for unseen classes. Experiments are performed on three baseline datasets, UCF101, HMDB51, and Olympic. In comparison with state-of-the-art approaches, both the proposed models, IAF based generative model and Bi-dir GAN model outperform in UCF101, and Olympic datasets in all the settings and achieve comparative results in HMDB51. [ABSTRACT FROM AUTHOR]

State-of-the-art Handwritten Text Recognition (HTR) systems allow transcribers to speed-up the transcription of handwritten text images. These systems provide transcribers an initial draft transcription that can be corrected with less effort than transcribing the handwritten text images from scratch. Currently, even the draft transcriptions offered by the most advanced HTR systems contain errors. Therefore, the supervision of this draft by a human transcriber is still necessary to obtain the correct transcription of the handwritten text images. This supervision can be eased by using interactive and assistive transcription systems, where the transcriber and the automatic system cooperate in the amending process. In this paper, the draft transcription is provided by an HTR system based on Convolutional and Recurrent Neural Networks with Bidirectional Long-Short Term Memory units, and the assistive system is fed by lattices generated by using Weighted Finite State Transducers. The influence of the lexicon and language restrictions on the performance of our computer assisted transcription system is evaluated on three historical manuscripts. The transcriptions offered by the proposed HTR system present very low error rates for the studied historical manuscripts. However, our assistive transcription system without lexicon or language restrictions is able to provide an additional reduction on the human effort required to correct the transcriptions in more than 50% over the transcriptions offered by the HTR system. [ABSTRACT FROM AUTHOR]

This paper addresses the problem of composite adaptive learning and tracking control for discrete-time nonlinear uncertain systems in general normal form. To deal with the system's unstructured nonlinear uncertain dynamics, novel adaptive neural network (NN) learning control strategies are proposed by extending methodologies from the continuous-time deterministic learning theory. Both state-feedback and output-feedback cases are considered. The proposed control schemes are compelling in the sense that (i) they are capable of rendering not only stable tracking control, but also locally-accurate learning/identification of unknown system dynamics during stable closed-loop control; (ii) the learned knowledge can be effectively represented and stored as constant NN models, whose weights are guaranteed to partially converge to ideal/optimal values. Based on this, experience-based controllers are also developed to pursue improved tracking control performance without online adaptation. In particular for the output-feedback control case, a new observer-less adaptive NN learning control scheme is proposed without resorting to high-gain observers, followed by an observer-less output-feedback experience-based controller. Numerical simulations have been conducted to demonstrate the effectiveness of the proposed approaches. [ABSTRACT FROM AUTHOR]

Robot learning from demonstration (LfD) enables robots to be fast programmed. This paper presents a novel LfD framework involving a teaching phase, a learning phase and a reproduction phase, and proposes methods in each of these phases to guarantee the overall system performance. An adaptive admittance controller is developed to take into account the unknown human dynamics so that the human tutor can smoothly move the robot around in the teaching phase. The task model in this controller is formulated by the Gaussian mixture regression to extract the human-related motion characteristics. In the learning and reproduction phases, the dynamic movement primitive is employed to model a robotic motion that is generalizable. A neural network-based controller is designed for the robot to track the trajectories generated from the motion model, and a radial basis function neural network is used to compensate for the effect caused by the dynamic environments. Experiments have been performed using a Baxter robot and the results have confirmed the validity of the proposed robot learning framework. [ABSTRACT FROM AUTHOR]

Autonomous vehicles have become a hot spot of the automotive industry, many cities have claimed that autonomous vehicles should be capable of recognizing gestures used by traffic police. Traditional traffic police gesture recognition methods rely on depth-sensor or wearable-devices, which limits their availability in the domain of the intelligent vehicle. Vision-based methods have fewer requirements for distance, but the modeling process is challenging due to the complexity of the visual scenes. Inspired by the recent success in vision-based pose estimation networks such as Convolutional Pose Machine (CPM), in this paper, we propose a novel vision-based human-machine interface to recognize eight kinds of Chinese traffic police gestures and apply it in the real-time recognition tasks. This method integrates a modified CPM network and two kinds of handcrafted features: Relative Bone Length and Angle with Gravity as spatial domain features, and adopt a Long short-term memory (LSTM) network to extract temporal domain features. To train and validate our method, we create a gestures dataset with two hours of traffic police gesture videos, which has 3354 gesture instances. The experiment results show that the proposed method is capable of recognizing traffic police gestures, and is fast enough for online gesture prediction. [ABSTRACT FROM AUTHOR]

Indoor place recognition is a challenging problem because of the hard representation to complicated intra-class variations and inter-class similarities.This paper presents a new indoor place recognition scheme using deep neural network. Traditional representations of indoor place almost utilize image feature to retain the spatial structure without considering the object's semantic characteristics. However, we argue that the attributes, state and relationships of objects are much more helpful in indoor place recognition. In particular, we improve the recognition framework by utilizing Place Descriptors (PDs) in text from to connect different types of place information with their categories. Meanwhile, we analyse the ability of Long Short-Term Memory (LSTM) and Convolutional Neural Network (CNN) for classification in natural language, for which we use them to process the indoor place descriptions. In addition, we improve the robustness of the designed deep neural network by combining a number of effective strategies, i.e. L2-regularization, data normalization, and proper calibration of key parameters. Compared with existing state of the art, the proposed approach achieves well performance of 70.73%, 70.08% and 70.16% of accuracy, precision and recall on Visual Genome database respectively. Meanwhile, the accuracy becomes 98.6% after adding voting mechanics. [ABSTRACT FROM AUTHOR]

• We extend the successful RPN+BF framework to combine handcrafted features and CNN features for pedestrian detection. • RoI-pooling is used to extract features from handcrafted and CNN channels, and with larger output resolution for the former. • We explore several handcrafted channels such as HOG+LUV, Checkerboards, and RotatedFilters in our framework. • Experimental results on Caltech pedestrian dataset demonstrate the effectiveness of the proposed framework. • When using a more advanced RPN, our approach can be further improved and get competitive results on the benchmarks. Pedestrian detection has achieved great improvements with the help of Convolutional Neural Networks (CNNs). CNN can learn high-level features from input images, but the insufficient spatial resolution of CNN feature channels (feature maps) may cause a loss of information, which is harmful especially to small instances. In this paper, we propose a new pedestrian detection framework, which extends the successful RPN+BF framework to combine handcrafted features and CNN features. RoI-pooling is used to extract features from both handcrafted channels (e.g. HOG+LUV, CheckerBoards or RotatedFilters) and CNN channels. Since handcrafted channels always have higher spatial resolution than CNN channels, we apply RoI-pooling with larger output resolution to handcrafted channels to keep more detailed information. Our ablation experiments show that the developed handcrafted features can reach better detection accuracy than the CNN features extracted from the VGG-16 net, and a performance gain can be achieved by combining them. Experimental results on Caltech pedestrian dataset with the original annotations and the improved annotations demonstrate the effectiveness of the proposed approach. When using a more advanced RPN in our framework, our approach can be further improved and get competitive results on both benchmarks. [ABSTRACT FROM AUTHOR]

This paper proposes a new haze layer based single image dehazing algorithm. The residual images, which exists between the hazy images and the clear images, will be firstly obtained by haze layers through an end-to-end mapping from the original hazy images. Then the designed convolutional neural network can remove the residual image from the given hazy image to obtain a recovered dehazed image. In order to remove the halos and block artefacts in the dehazed images, a guided filter is applied before generating the final dehazed images, which will also increase the reality of the result images. Since the proposed haze layer based dehazing algorithm directly learns the residual images, it enjoys a relatively high learning rate and low computation. We tested and compared the proposed algorithm with other dehazing methods in various variables, such as the density of the fog and the testing scenes, e.g. real-world images or synthetic images. Experimental results showed the advancement of the proposed method in many metrics. [ABSTRACT FROM AUTHOR]

Estimating the 6D object pose is a very challenging task in computer vision. The main difficulty is mapping the object from RGB images to 3D space. In this paper, we present a novel two-stage method for estimating the 6D object pose by using the 2D keypoints of an object and its 2D bounding box. There are two stages in our method. The first stage detects the 2D keypoints and 2D bounding boxes of objects by a stable end-to-end framework. During the training phase, this framework uses viewpoint transformation information and object saliency regions to learn geometrically and semantically consistent keypoints. Then the 6D poses of objects are calculated by a series of geometric reasoning algorithms in the second stage. Experiments show that our method achieves accurate pose estimation and robust to occluded and cluttered scenes. [ABSTRACT FROM AUTHOR]

Applying machine learning methods to various modality medical images and clinical data for early diagnosis of Alzheimer's disease (AD) and its prodromal stage has many significant results. So far, the image data input to classifier mainly focus on 2D or 3D images. Although some functional imaging technologies, such as functional magnetic resonance imaging (fMRI), generate 4D data which contain both spatial and time-varying information of the brain, for the lack of suitable 4D image processing algorithm, these 4D data were always used by transforming them into functional connectivity or slicing them into 2D/3D pictures which causing apparent information loss. In this paper, we present a 4D deep learning model (C3d-LSTM) for AD discrimination, which is able to utilize the spatial and time-varying information simultaneously by dealing with 4D fMRI data directly. The proposed C3d-LSTM combines a series of 3D convolutional neural networks (CNNs) to extract spatial features from each volume of 3D static image in fMRI image sequence. Then the feature maps obtained were put into the long short-term memory (LSTM) network to capture the time-varying information contained in the data. Because of the design of structure, C3d-LSTM became an end-to-end data-driven model, which was more convenient for processing 4D fMRI data. The model proposed conducted on the AD Neuroimaging Initiative dataset for algorithm evaluation compared with controlled experiments. Results showed that using 4D fMRI data directly with the proposed method did make a far better result for AD detection than the methods using functional connectivity, 2D, or 3D fMRI data. It demonstrated our assumption that making the most of the natural spatial and temporal information preserved in 4D fMRI data is significant for AD detection and can increase the performance of the classifier. Meanwhile the result implied that the C3d-LSTM model proposed is a suitable model for processing 4D fMRI data and extracting the spatio-temporal property of fMRI data fully for diagnosis of AD. [ABSTRACT FROM AUTHOR]

Multi-loss-joint-optimization has been proven to be valid in computer vision literature. However, the learned deep sub-features usually fit their disjoint constraints, which yield confrontation and spatial inconsistency among the sub-features with nonshared FC layers. In this paper, we propose a Hybrid-loss supervision (HLS) framework in order to obtain smoother and more spatially consistent features with shared FC layers. First, we analyze the shortcomings of the monitoring with single-loss in the existing framework theoretically. Then, we selected two notable loss functions (e.g., Center loss and Weighted loss) to instantiate the HLS framework by linear combination. By instantiating the framework with two standard loss functions, the network has learned more compact intra-class deep features and uniform inter-class deep features. The HLS framework can significantly boost the efficiency of existing convolution networks for both image classification task and object detection task without increasing network parameters and computational complexity. Extensive experimental results on different vision tasks demonstrate consistent improvement can be achieved across a variety of datasets (e.g., CIFAR-10/100, ImageNet-2012, PASCAL VOC and MS-COCO) and different convolutional neural network architectures. [ABSTRACT FROM AUTHOR]

• An improved Cellular Neural Network (CellNN) by local-neighbor binary constraints. • A feature extraction framework based on the state and output feature maps from recurrent CellNN convolutions. • A feature map compression, joint-distribution pattern fusion and a multi-resolution scheme optimized by softmax. Texture recognition is one of the most important branches in image research. This paper mainly aims to develop a new solution to address texture recognition using a Cellular Neural Network (CellNN). Firstly, it proposes an improved model of CellNN by the binary constraints of local receptive fields, and then designs a recurrent convolution framework of such a model to generate two types of texture feature maps, including state feature maps and output feature maps. In order to obtain low-dimensional features, state feature maps are further compressed by the mapping of rotation-invariant patterns and the merging of low-frequency occurrence patterns. By the statistics of joint-distribution patterns, state feature maps and output feature maps are fused together to generate the features of single resolution. Moreover, a multi-resolution feature combination scheme is also designed by the optimization of softmax & variance and concatenation of multiple features. Finally, a fully-connected neural network is trained to work as a texture recognizer. The experimental comparisons of totally 15 algorithms on five benchmark datasets show that, on the dataset whose texture-class quantity is not beyond 30, such as Brodatz, our method could always acquire the highest recognition accuracy, outperforming any other compared ones. On the big dataset with huge texture-class quantity, such as ALOT, our method could also surpass any other non-deep-learning one, such as the state-of-the-art gLBP, only slightly falling behind the best two deep-learning ones, FV-Alex and FV-VGGVD. However, in terms of time cost, our method could always outperform any deep-learning one in feature extraction stage, and also surpass any compared one except original LBP in feature matching. [ABSTRACT FROM AUTHOR]

Video-based convolutional neural networks (CNNs) involve a large amount of training parameters, leading to the enormous computational complexity, which thereby delays the network convergence. Therefore, training successful CNN for action recognition rapidly is non-trivial. In this paper, a novel encoding called nonlinear gated channels unit (NGCU) is proposed to facilitate network training by encoding the global channel-level relationship. Based on this, nonlinear gated channels network (NGCN) is constructed for the end-to-end encoding, and the corresponding convergence performance is evaluated on the standard benchmarks UCF101 and HMDB51. Experimental results demonstrate that the proposed method is conducive to the convergence process of CNN based action recognition models. [ABSTRACT FROM AUTHOR]

Classifying visual information is an apparently simple and effortless task in our everyday routine, but can we automatically predict what we see from signals emitted by the brain? While other researchers have already attempted to answer this question, we are the first to show that a commercially available BCI could be effectively used for visual image classification in real-world scenarios – when testing takes place at a completely different time than training data collection. The task is difficult, as it requires relating the noisy and low-level EEG signals to complex and highly semantic visual categories. In this paper, we propose different learning approaches and show that simpler classifiers such as Ridge Regression with Gabor filtering of the input EEG signal could be more effective than the powerful Long Short Term Memory Networks and Convolutional Neural Networks in this case of limited and noisy training data. We analyzed the importance of each electrode for the visual classification task and noticed that the sensors with the highest accuracy were the ones that recorded brain activity from regions known to be correlated more with higher level recognition and cognitive processes and less to lower-level visual signal processing. The result is also in accordance with research in computer vision with deep neural networks, which shows that semantic visual features are learned only at higher levels of neural depth. While EEG signals are weaker by themselves for the task of visual classification, we demonstrate that they could be powerful when combined with deep visual features extracted from the image, improving performance from 91% to over 97% in a multi-class recognition setting. Our tests show that EEG input brings additional information that is not learned by artificial deep networks on the given image training set. Thus, a commercially available BCI could be effectively used in conjunction with a deep learning based vision system to form together a stronger visual recognition system that is suitable for real-world applications. [ABSTRACT FROM AUTHOR]

Unlike natural image captured with cameras, screen content image (SCI) is a composite image including textual and pictorial regions. The different characteristics lead to many difficulties in image quality assessment (IQA). Most existing models based on the convolutional neural network (CNN) divide large SCIs into image patches to increase training samples for CNN training. This brings two problems: (1) a single image patch can not represent the quality of the entire image, especially in IQA of SCI; (2) SCI patches of an entire image degraded by the same distortion type and strength may have drastically different quality. In addition, these models adopt the mean square error (MSE) between the predicted quality and the subjective differential mean opinion score (DMOS) to train the CNN, without considering quality ranking between different SCIs. In this paper, we propose a novel no-reference (NR) IQA model based on the convolutional neural network (CNN). The contributions of our algorithm can be concluded as follows: (1) considering a large difference exists in different regions in a SCI, the pseudo global features generated with multi-region local features are utilized for quality evaluation, which better reflect image quality than local features of each image patch; (2) the noise classification task is used as an auxiliary task which aids the quality score prediction task to improve the representation ability; (3) the Siamese networks are used to predict the quality scores of two different SCIs, and a new ranking loss is proposed to rank the predicted scores, aiming to enhance the ability of the model to rank image in terms of quality. Experimental results verify that our model outperforms all test NR IQA methods and full-reference (FR) IQA methods on the screen content image quality assessment database (SIQAD). [ABSTRACT FROM AUTHOR]

Osteoporosis makes bones weak and brittle, increasing the risk of fracture. In this paper, we designed a hybrid model to diagnose osteoporosis based on bone radiograph images. Two types of features were used to distinguish between the "healthy" and the "sick". One type of features was obtained from deep convolutional neural networks (CNNs), named CNN features, and the other was hand-crafted features containing a group of standard texture features such as local binary pattern and gray level co-occurrence matrix and a group of "encoded features" that have shown impressive discriminative capabilities. We used a minimum-redundancy maximum-relevance algorithm to reduce the high dimensionality of the features and a support vector machine was used as the recognizer. This is the first study to fuse the CNNs features with the state-of-the-art osteoporotic texture features for osteoporosis diagnosis. We explore if the fusion of the two types of powerful features will increase the performance or not. Comparative experiments show that considerable performance improvements can be made through the fusion of both types of features, and the fusion of AlexNet with encoded features or all the hand-crafted features achieved the highest accuracy among all the fusions. [ABSTRACT FROM AUTHOR]

• Unsupervised framework for depth estimation and camera motion prediction. • Depth CNN and pose CNN are trained jointly and can be used respectively. • The supervision signal with a left-right consistency is constructed by spatial and temporal geometry constraints. • Results outperform previous unsupervised methods and some supervised methods. • A model which is trained on the Euroc dataset is used to test the algorithm's generalization capability. Depth estimation from monocular video plays a crucial role in scene perception. The significant drawback of supervised learning models is the need for vast amounts of manually labeled data (ground truth) for training. To overcome this limitation, unsupervised learning strategies without the requirement for ground truth have achieved extensive attention from researchers in the past few years. This paper presents a novel unsupervised framework for estimating single-view depth and predicting camera motion jointly. Stereo image sequences are used to train the model while monocular images are required for inference. The presented framework is composed of two CNNs (depth CNN and pose CNN) which are trained concurrently and tested independently. The objective function is constructed on the basis of the epipolar geometry constraints between stereo image sequences. To improve the accuracy of the model, a left-right consistency loss is added to the objective function. The use of stereo image sequences enables us to utilize both spatial information between stereo images and temporal photometric warp error from image sequences. Experimental results on the KITTI and Cityscapes datasets show that our model not only outperforms prior unsupervised approaches but also achieving better results comparable with several supervised methods. Moreover, we also train our model on the Euroc dataset which is captured in an indoor environment. Experiments in indoor and outdoor scenes are conducted to test the generalization capability of the model. [ABSTRACT FROM AUTHOR]

Compressed sensing MRI (CS-MRI) is considered as a powerful technique for decreasing the scan time of MRI while ensuring the image quality. However, state of the art reconstruction algorithms are still subjected to two challenges including terrible parameters tuning and image details loss resulted from over-smoothing. In this paper, we propose a deep frequency-division network (DFDN) to face these two image reconstruction issues. The proposed DFDN approach applies a deep iterative reconstruction network (DIRN) to replace the regularization terms and the corresponding parameters by a stacked convolution neural network (CNN). And then multiple DIRN blocks are cascaded continuously as one deeper neural network. Data consistency (DC) layer is incorporated after each DIRN block to correct the k -space data of intermediate results. Image content loss is computed after each DC layer and frequency-division loss is gained by weighting the high frequency loss and low frequency loss after each DIRN block. The combination of image content loss and frequency-division loss is considered as the total loss for constraining the network training procedure. Validations of the proposed method have been performed on two brain datasets. Visual results and quantitative evaluations show that the proposed DFDN algorithm has better performance in sparse MRI reconstruction than other comparative methods. [ABSTRACT FROM AUTHOR]

• A novel object-adaptive LSTM network for real-time visual tracking is proposed. • The proposed LSTM network adaptively learns the object appearance variations. • A fast proposal selection strategy is presented for high computational efficiency. • A data augmentation technique using GAN is adopted for sample adequacy. • Experimental results show the state-of-the-art performance of our method. In recent years, deep learning based visual tracking methods have obtained great success owing to the powerful feature representation ability of Convolutional Neural Networks (CNNs). Among these methods, classification-based tracking methods exhibit excellent performance while their speeds are heavily limited by the expensive computation for massive proposal feature extraction. In contrast, matching-based tracking methods (such as Siamese networks) possess remarkable speed superiority. However, the absence of online updating renders these methods unadaptable to significant object appearance variations. In this paper, we propose a novel real-time visual tracking method, which adopts an object-adaptive LSTM network to effectively capture the video sequential dependencies and adaptively learn the object appearance variations. For high computational efficiency, we also present a fast proposal selection strategy, which utilizes the matching-based tracking method to pre-estimate dense proposals and selects high-quality ones to feed to the LSTM network for classification. This strategy efficiently filters out some irrelevant proposals and avoids the redundant computation for feature extraction, which enables our method to operate faster than conventional classification-based tracking methods. In addition, to handle the problems of sample inadequacy and class imbalance during online tracking, we adopt a data augmentation technique based on the Generative Adversarial Network (GAN) to facilitate the training of the LSTM network. Extensive experiments on four visual tracking benchmarks demonstrate the state-of-the-art performance of our method in terms of both tracking accuracy and speed, which exhibits great potentials of recurrent structures for visual tracking. [ABSTRACT FROM AUTHOR]

Ten years ago, linear models were applied in various domains. Before application of the algorithms, several current studies extracted features presumed to represent parochial markings from the data using engineering techniques. Recently the deep learning domain offered opportunities to directly feed data into the model without any extensive hand-crafted feature engineering techniques. In this paper, the proposed framework does the feature extraction in a non-supervised (i.e. self-supervised) manner using both Contextual Long Short-Term Memory (CLSTM) and Contextual Convolutional Neural Networks (CCNN). We can then concatenate data obtained from the CLSTM and CCNN blocks, feed it into the Attention block, pass it through the Multilayer Perceptron (MLP) block, ultimately passing it through a terminal layer for classification. The task involved here was non-trivial as there is a major challenge in implementing our model to solve the time series classification (TSC) problem: overfitting. We deal with this challenge as follows; firstly, we adjusted the number of neurons in each of the stages. Secondly we introduced dropouts after every layer in each stage of this model. Finally experiments regarding the University of California Riverside (UCR) dataset indicates the model's superiority. [ABSTRACT FROM AUTHOR]

The Person Re-identification (ReID) task aims to match persons across cameras in a surveillance system. In the past few years, many researches are devoted to ReID and its performance has gained significant improvement. ReID models are usually trained as a joint framework comprising a person feature extractor and a classifier. However, there exists co-adaptation between the feature extractor and the classifier, which prevents the feature extractor from making effective and sufficient optimization and results in inferior retrieval performance. In this paper, we propose a very simple and effective training method, called DeAda, to decouple this co-adaptation. Our main motivation is to construct a series of weak classifiers during training by randomization of parameters, so that optimization on the feature extractor could be strengthened in the training stage. DeAda is easy, effective, and efficient, and could serve as a plug-and-play optimization tool for ReID models, without additional memory and time cost. We also analyze the theoretical property of DeAda and show that it could produce identical features for the same person under some simple assumptions. We demonstrate its effectiveness on three public ReID datasets: Market1501, DukeMTMC-reID and CUHK03 over different ReID models. With DeAda optimization, we finally obtain state-of-the-art results on all the three datasets. [ABSTRACT FROM AUTHOR]

• A weakly supervised approach is proposed for ABDL in crowded scenes. The MISVM based framework not only can accurately find the optimal separate hyperplane between normal and abnormal behaviors, but also only requires the bag-level label information rather than the complete labels of all samples. • Benefit from the powerfulness of faster R-CNN, most objects can be accurately localized. By doing so, not only the abnormal behavior can be analyzed at object-wise, but also the influence from the background can be eliminated. Consequently, the robustness, generality, and the computational efficiency of our approach can be strengthened. • A histogram of large-scale optical flow (HSLOF) is proposed to describe the object behavior, which can extract the most significant optical flow, and is insensitive to the variations of the size of objects. In this paper, a weakly supervised framework is proposed for Abnormal Behavior Detection and Localization (ABDL) in the scenes. First, the objects in the scene such as pedestrians, vehicles, etc. are detected using the Faster Regional Conventional Neural Network (Faster R-CNN); then, the object behavior is described by a Histogram of Large Scale Optical Flow (HLSOF) descriptor; finally, the Multiple Instance Support Vector Machine (MISVM) is trained and then used to identify the testing behaviors as normal or abnormal. Summarily, the proposed approach has three main advantages: (1) Benefit from the Faster R-CNN, our approach can analyze the behavior at object-wise, which makes our approach has good generality and high computational efficiency; (2) The HLSOF descriptor can characterize the object behavior efficiently, and is insensitive to the variations of the size of objects; (3) As a weakly supervised learning framework, the MISVM only requires the labels at the bag level rather than instance level, which makes our approach has high accuracy as the supervised approaches but not requires completely labeled training samples, only the frame-level label is required. Experimental results analysis on different datasets validates the effectiveness of our approach. [ABSTRACT FROM AUTHOR]

Keywords can express the main content of an article or a sentence. Keywords extraction is a critical issue in many Natural Language Processing (NLP) applications and can improve the performance of many NLP systems. The traditional methods of keywords extraction are based on machine learning or graph model. The performance of these methods is influenced by the feature selection and the manually defined rules. In recent years, with the emergence of deep learning technology, learning features automatically with the deep learning algorithm can improve the performance of many tasks. In this paper, we propose a deep neural network model for the task of keywords extraction. We make two extensions on the basis of traditional LSTM model. First, to better utilize both the historic and following contextual information of the given target word, we propose a target center-based LSTM model (TC-LSTM), which learns to encode the target word by considering its contextual information. Second, on the basis of TC-LSTM model, we apply the self-attention mechanism, which enables our model has an ability to focus on informative parts of the associated text. In addition, we also introduce a two-stage training method, which takes advantage of large-scale pseudo training data. Experimental results show the advantage of our method, our model beats all the baseline systems all across the board. And also, the two-stage training method is of great significance for improving the effectiveness of the model. [ABSTRACT FROM AUTHOR]

To make ANNs have the ability of logical processing in order to fulfil the urgent requirement that computers can automatically judge according to numerous specific conditions, researches have been carried out to design novel neural network models for representing logical relations. Recently, a new ANN model for representing logical relations is proposed. In the model, six components are designed to simulate the operations of logic gates. The work provides a novel way for constructing logical relations running in a neural-like manner. However, the components are still complex and indirect for the representation since more extra neurons and links are needed to simulate logic gates. In order to represent logical relations more directly, this paper defines new neurons and multiple kinds of links to represent logic gates directly, and they can be combined to represent complex logical relations in a simpler neural network structures with fewer neurons. Additionally, this ANN model can dynamically create links on demand instead of the fixed full connections. It can constantly adjust its network structure when getting the data continuously. It can be used for the establishment of the rule library of the intelligent information system in the form of the neural network structure. [ABSTRACT FROM AUTHOR]

Pointer-generator networks have recently shown superior performance in Natural Language Generation (NLG) tasks, such as automatic generating descriptions for entities in Knowledge Graph (KG). In the absence of the introductory description about an entity we intend to know, the natural language text automatically generated by neural network model can greatly help people to better understand the entity. Entities in KG always have multiple property fields and corresponding values. The target generated description sentences should consist of these slot types and slot values in a consistent and high coverage manner. In order to cover the facts in an input KG, pointer-generator networks always copy certain segments from the input sequences via softmax pointing while generating novel words through the generator. But the challenges that when and where to integrate the copied information with the generated one, how to prevent duplicate generation and information loss are hard to conquer. In this paper, the KG2TEXT model based on adapted pointer-generator networks is proposed. In it, a varied coverage loss function is devised to cover attribute-value pairs as many as possible when generating natural language descriptions for entities in KG. Secondly, an attention mechanism named supervised attention mechanism is added to the model which aims to guide the soft switch transformation process (to generate or to copy). With empirical experiments on two kinds of realistic datasets, the KG2TEXT model achieves promising results and outperforms the state-of-the-art approaches. [ABSTRACT FROM AUTHOR]

In multi-label text classification, considering the correlation between labels is an important yet challenging task due to the combination possibility in the label space increasing exponentially. In recent years, neural network models have been widely applied and gradually achieved satisfactory performance in this field. However, existing methods either not model the fully internal correlations among labels or not capture the local and global semantic information of text simultaneously, which somewhat affects the classification results finally. In this paper, we implement a novel model for multi-label classification based on sequence-to-sequence learning, in which two different neural network modules are employed, named encoder and decoder respectively. The encoder uses the convolutional neural network to extract the high-level local sequential semantic, which is combined with the word vector to generate the final text representation through the recurrent neuron network and attention mechanism. The decoder, besides using a recurrent neural network to capture the global label correlation, employs an initialized fully connection layer to capture the correlation between any two different labels. When trained on RCV1-v2, AAPD and Ren-CECps datasets, the proposed model outperforms previous work in main evaluation metrics of hamming loss and micro-F1 score. [ABSTRACT FROM AUTHOR]