Search

With the advantages of low power consumption, flexibility, and high readability against bright ambiance, electrophoretic displays (EPDs) have wide application prospects in the fields of education, smart supermarkets, Internet of Things, smart homes, wearable devices, etc. EPDs suffer from a severe history dependence during grayscale switching, which results in annoying ghost images. However, currently, it is difficult to distinguish diverse types of ghost images automatically; thus, lookup-tables (LUTs) for multi-grayscale waveform design cannot be effectively generated but require cumbersome manual adjustment. In this article, we proposed to adopt a convolutional neural network (CNN) to automatically recognize ghost images, based on which, LUTs that could suppress ghost images and achieve accurate grayscales were automatically generated for waveform design. Moreover, the workforce for manual adjustment was significantly saved. The results suggest that the CNN is a powerful tool for EPDs to achieve better image quality, as well as less manual cost. [ABSTRACT FROM AUTHOR]

Data quality plays an important role during the training stage of machine/deep learning models. The annotation hinges on the experiences of domain experts. To acquire the expert’s knowledge in the context of machine learning, manual data labeling, a tedious and time-consuming task in supervised learning, should be given a top priority. However, the domain experts in the line of plentiful manual annotation may easily get distracted or fatigued after long-time work, causing judgment errors, mislabeling, etc. The pattern recognition of wafer defect map is investigated in this paper, the primary goal of which is to train the convolutional neural network (CNN) model through a very limited number of manually labeled data so that the trained model is capable of performing pseudo labeling. Subsequently, a self-assured adaptive ensemble learner in terms of a series of shallow neural networks is proposed to filter wafer map samples with untrusted pseudo-labels. In the result, the amount of human annotations is significantly reduced by 61% for training a highly accurate classifier. A minimum number of manually labeled data is suggested while the equally high classification performance of wafer defect pattern is maintained. For the evaluation purpose, the proposed self-assured learning is compared with the confidence learning. [ABSTRACT FROM AUTHOR]

This paper reports the development of a manipulation system for electric wires, implemented by means of a commercial gripper installed on an industrial manipulator and equipped with cameras and suitably designed tactile sensors. The purpose of this system is the execution of wire insertion on commercial electromechanical components. The synergy between computer vision and tactile sensing is necessary because, in a real environment, the tight spaces very often prevent the possibility to use the vision system, also when the same task is performed by a human being. A novel technique to speed up the generation of training data sets for convolutional neural networks (CNNs) is proposed. Therefore, this technique is used to train a CNN in order to detect small objects (such as wire terminals). Moreover, aiming to prevent faults during the task and to interact with the environment safely, several machine learning approaches are used to produce an affordable output from the tactile sensor. The proposed approach shows how a cheap sensor embedded with suitable intelligence can provide information comparable to a more expensive force sensor. Note to Practitioners—This paper was motivated by the lack of commercial solution for the automatic cabling of switchgears. Existing approaches to this problem are in some way limited to specific large-scale products or simple layouts. This paper investigated a robust and flexible solution, based on the exploitation of multiple sensors and machine learning algorithms, for wire detection, grasping, and connection. The proposed approach is characterized by simple design and self-tuning capabilities, and it can be easily employed on a wide range of switchgear layouts thanks to the large workspace of the manipulator. Experimental results show that the proposed system is able to achieve a 95% success rate within a realistic admissible region. In the future research, we will integrate the proposed solution with an electromechanical component localization module and a terminal fastening system to evaluate the performance on the real production line. [ABSTRACT FROM AUTHOR]

A safe and versatile interaction between humans and objects is based on tactile and visual information. In literature, visual sensing is widely explored compared to tactile sensing, despite showing significant limitations in environments with an obstructed view. Tactile perception does not depend on those factors. In this paper, a Machine Learning-based tactile object classification approach is presented. The hardware setup is composed of a 3-finger-gripper of a robotic manipulator mounted on the Doro robot of the Robot-Era project. This paper’s main contribution is the augmentation of the custom 20 class 2000 sample tactile time-series dataset using random jitter noise, scaling, magnitude, time warping, and cropping. The effect on the object recognition performance of the dataset expansion is investigated for the neural network architectures MLP, LSTM, CNN, CNNLSTM, ConvLSTM, and deep CNN (D-CNN). The data augmentation methods brought a statistically significant object classification accuracy increase compared to models trained on the original dataset. The best tactile object classification success rate of 72.58% is achieved for the D-CNN trained on an augmented dataset derived from scaling and time warping augmentation. [ABSTRACT FROM AUTHOR]

Convolutional neural networks (CNN) have demonstrated state-of-the-art classification results in image categorization, but have received comparatively little attention for classification of one-dimensional physiological signals. We design a deep CNN architecture for automated sleep stage classiffication of human sleep EEG and EOG signals. The CNN proposed in this paper amply outperforms recent work that uses a different CNN architecture over a single-EEG-channel version of the same dataset. We show that the performance gains achieved by our network rely mainly on network depth, and not on the use of several signal channels. Performance of our approach is on par with human expert inter-scorer agreement. By examining the internal activation levels of our CNN, we find that it spontaneously discovers signal features such as sleep spindles and slow waves that figure prominently in sleep stage categorization as performed by human experts. [ABSTRACT FROM AUTHOR]

Neural networks have achieved remarkable successes in machine learning tasks. This has recently been extended to graph learning using neural networks. However, there is limited theoretical work in understanding how and when they perform well, especially relative to established statistical learning techniques such as spectral embedding. In this short paper, we present a simple generative model where unsupervised graph convolutional network fails, while the adjacency spectral embedding succeeds. Specifically, unsupervised graph convolutional network is unable to look beyond the first eigenvector in certain approximately regular graphs, thus missing inference signals in non-leading eigenvectors. The phenomenon is demonstrated by visual illustrations and comprehensive simulations. [ABSTRACT FROM AUTHOR]

Deep convolutional neural networks (DCNNs) are currently the method of choice both for generative, as well as for discriminative learning in computer vision and machine learning. The success of DCNNs can be attributed to the careful selection of their building blocks (e.g., residual blocks, rectifiers, sophisticated normalization schemes, to mention but a few). In this paper, we propose $\Pi$ Π -Nets, a new class of function approximators based on polynomial expansions. $\Pi$ Π -Nets are polynomial neural networks, i.e., the output is a high-order polynomial of the input. The unknown parameters, which are naturally represented by high-order tensors, are estimated through a collective tensor factorization with factors sharing. We introduce three tensor decompositions that significantly reduce the number of parameters and show how they can be efficiently implemented by hierarchical neural networks. We empirically demonstrate that $\Pi$ Π -Nets are very expressive and they even produce good results without the use of non-linear activation functions in a large battery of tasks and signals, i.e., images, graphs, and audio. When used in conjunction with activation functions, $\Pi$ Π -Nets produce state-of-the-art results in three challenging tasks, i.e., image generation, face verification and 3D mesh representation learning. The source code is available at https://github.com/grigorisg9gr/polynomial_nets. [ABSTRACT FROM AUTHOR]

Due to the strong penetration ability and high transmission rate of ultra-wideband (UWB) signals, UWB technology plays a very significant role in the field of precise indoor positioning. However, the harsh and volatile indoor environment leads to non-line-of-sight (NLOS) propagation and severe attenuation of UWB signals, which may generate significant ranging and positioning errors. To mitigate NLOS effect and improve positioning accuracy, existing methods use ranging information and channel impulse response (CIR) to identify UWB signal propagation channels. However, these NLOS identification methods often require a priori knowledge and suitable thresholds, and most of them only perform a binary classification between LOS and NLOS in a particular scenario. To address these disadvantages, this paper proposes a novel multi-classification method to classify UWB signal propagation channels based on one-dimensional wavelet packet analysis (ODWPA) and convolutional neural network (CNN). This method first decomposes the CIR of known categories into colored coefficients images using a one-dimensional wavelet packet function, then uses these images to train CNNs with different structures, and finally selects CNN model with the best performance to identify the unknown category of UWB signal propagation channels. The experimental results show that the performance of the proposed method is always the best, regardless of the type of experimental scenario, and the identification accuracy is always higher than 90% and up to 100%. [ABSTRACT FROM AUTHOR]

This paper proposes energy-efficient approximate multipliers based on the Mitchell's log multiplication, optimized for performing inferences on convolutional neural networks (CNN). Various design techniques are applied to the log multiplier, including a fully-parallel LOD, efficient shift amount calculation, and exact zero computation. Additionally, the truncation of the operands is studied to create the customizable log multiplier that further reduces energy consumption. The paper also proposes using the one's complements to handle negative numbers, as an approximation of the two's complements that had been used in the prior works. The viability of the proposed designs is supported by the detailed formal analysis as well as the experimental results on CNNs. The experiments also provide insights into the effect of approximate multiplication in CNNs, identifying the importance of minimizing the range of error.The proposed customizable design at $w$w = 8 saves up to 88 percent energy compared to the exact fixed-point multiplier at 32 bits with just a performance degradation of 0.2 percent for the ImageNet ILSVRC2012 dataset. [ABSTRACT FROM AUTHOR]

We present VoxelMorph, a fast learning-based framework for deformable, pairwise medical image registration. Traditional registration methods optimize an objective function for each pair of images, which can be time-consuming for large datasets or rich deformation models. In contrast to this approach and building on recent learning-based methods, we formulate registration as a function that maps an input image pair to a deformation field that aligns these images. We parameterize the function via a convolutional neural network and optimize the parameters of the neural network on a set of images. Given a new pair of scans, VoxelMorph rapidly computes a deformation field by directly evaluating the function. In this paper, we explore two different training strategies. In the first (unsupervised) setting, we train the model to maximize standard image matching objective functions that are based on the image intensities. In the second setting, we leverage auxiliary segmentations available in the training data. We demonstrate that the unsupervised model’s accuracy is comparable to the state-of-the-art methods while operating orders of magnitude faster. We also show that VoxelMorph trained with auxiliary data improves registration accuracy at test time and evaluate the effect of training set size on registration. Our method promises to speed up medical image analysis and processing pipelines while facilitating novel directions in learning-based registration and its applications. Our code is freely available at https://github.com/voxelmorph/voxelmorph. [ABSTRACT FROM AUTHOR]

In this paper, we develop a new weakly supervised learning algorithm to learn to segment cancerous regions in histopathology images. This paper is under a multiple instance learning (MIL) framework with a new formulation, deep weak supervision (DWS); we also propose an effective way to introduce constraints to our neural networks to assist the learning process. The contributions of our algorithm are threefold: 1) we build an end-to-end learning system that segments cancerous regions with fully convolutional networks (FCNs) in which image-to-image weakly-supervised learning is performed; 2) we develop a DWS formulation to exploit multi-scale learning under weak supervision within FCNs; and 3) constraints about positive instances are introduced in our approach to effectively explore additional weakly supervised information that is easy to obtain and enjoy a significant boost to the learning process. The proposed algorithm, abbreviated as DWS-MIL, is easy to implement and can be trained efficiently. Our system demonstrates the state-of-the-art results on large-scale histopathology image data sets and can be applied to various applications in medical imaging beyond histopathology images, such as MRI, CT, and ultrasound images. [ABSTRACT FROM AUTHOR]

The goal of single-image deraining is to restore the rain-free background scenes of an image degraded by rain streaks and rain accumulation. The early single-image deraining methods employ a cost function, where various priors are developed to represent the properties of rain and background layers. Since 2017, single-image deraining methods step into a deep-learning era, and exploit various types of networks, i.e., convolutional neural networks, recurrent neural networks, generative adversarial networks, etc., demonstrating impressive performance. Given the current rapid development, in this paper, we provide a comprehensive survey of deraining methods over the last decade. We summarize the rain appearance models, and discuss two categories of deraining approaches: model-based and data-driven approaches. For the former, we organize the literature based on their basic models and priors. For the latter, we discuss the developed ideas related to architectures, constraints, loss functions, and training datasets. We present milestones of single-image deraining methods, review a broad selection of previous works in different categories, and provide insights on the historical development route from the model-based to data-driven methods. We also summarize performance comparisons quantitatively and qualitatively. Beyond discussing the technicality of deraining methods, we also discuss the future possible directions. [ABSTRACT FROM AUTHOR]

In this paper, we propose a channel-wise interaction based binary convolutional neural networks (CI-BCNN) approach for efficient inference. Conventional binary convolutional neural networks usually apply the xnor and bitcount operations in the binary convolution with notable quantization errors, which obtain opposite signs of pixels in binary feature maps compared to their full-precision counterparts and lead to significant information loss. In our proposed CI-BCNN method, we exploit the channel-wise interactions with the prior knowledge which aims to alleviate inconsistency of signs in binary feature maps and preserves the information of input samples during inference. Specifically, we mine the channel-wise interactions by using a reinforcement learning model, and impose channel-wise priors on the intermediate feature maps to correct inconsistent signs through the interacted bitcount. Since CI-BCNN mines the channel-wise interactions in a large search space where each channel may correlate with others, the search deficiency caused by sparse interactions obstacles the agent to obtain the optimal policy. To address this, we further present a hierarchical channel-wise interaction based binary convolutional neural networks (HCI-BCNN) method to shrink the search space via hierarchical reinforcement learning. Moreover, we propose a denoising interacted bitcount operation in binary convolution by smoothing the channel-wise interactions, so that noise in channel-wise priors can be alleviated. Extensive experimental results on the CIFAR-10 and ImageNet datasets demonstrate the effectiveness of the proposed CI-BCNN and HCI-BCNN. [ABSTRACT FROM AUTHOR]

Encrypted web traffic can reveal sensitive information of users, such as their browsing behaviors. Existing studies on encrypted traffic analysis focus on website fingerprinting. We claim that fine-grained webpage fingerprinting, which speculates specific webpages on a same website visited by a victim, allows exploiting more user private information, e.g., shopping interests in an online shopping mall. Since webpages from the same website usually have very similar traffic traces that make them indistinguishable, existing solutions may end up with low accuracy. In this paper, we propose FineWP, a novel fine-grained webpage fingerprinting method. We make an observation that the length information of packets in bidirectional client-server interactions can be distinctive features for webpage fingerprinting. The extracted features are then fed into traditional machine learning models to train classifiers, which achieve both high accuracy and low training overhead. We collect two real-world traffic datasets and construct closed- and open-world evaluations to verify the effectiveness of FineWP. The experimental results demonstrate that FineWP is superior to the state-of-the-art methods in terms of accuracy, time complexity and stability. [ABSTRACT FROM AUTHOR]

Pneumothorax is potentially a life-threatening disease that requires urgent diagnosis and treatment. The chest X-ray is the diagnostic modality of choice when pneumothorax is suspected. The computer-aided diagnosis of pneumothorax has received a dramatic boost in the last few years due to deep learning advances and the first public pneumothorax diagnosis competition with 15257 chest X-rays manually annotated by a team of 19 radiologists. This paper describes one of the top frameworks that participated in the competition. The framework investigates the benefits of combining the Unet convolutional neural network with various backbones, namely ResNet34, SE-ResNext50, SE-ResNext101, and DenseNet121. The paper presents a step-by-step instruction for the framework application, including data augmentation, and different pre- and post-processing steps. The performance of the framework was of 0.8574 measured in terms of the Dice coefficient. The second contribution of the paper is the comparison of the deep learning framework against three experienced radiologists on the pneumothorax detection and segmentation on challenging X-rays. We also evaluated how diagnostic confidence of radiologists affects the accuracy of the diagnosis and observed that the deep learning framework and radiologists find the same X-rays to be easy/difficult to analyze (p-value <1e4). Finally, the methodology of all top-performing teams from the competition leaderboard was analyzed to find the consistent methodological patterns of accurate pneumothorax detection and segmentation. [ABSTRACT FROM AUTHOR]

The core of many large-scale machine learning (ML) applications, such as neural networks (NN), support vector machine (SVM), and convolutional neural network (CNN), is the training algorithm that iteratively updates model parameters by processing massive datasets. From a plethora of studies aiming at accelerating ML, being data parallelization and parameter server, the prevalent assumption is that all data points are equivalently relevant to model parameter updating. In this article, we challenge this assumption by proposing a criterion to measure a data point's effect on model parameter updating, and experimentally demonstrate that the majority of data points are non-critical in the training process. We develop a slim learning framework, termed SlimML, which trains the ML models only on the critical data and thus significantly improves training performance. To such an end, SlimML efficiently leverages a small number of aggregated data points per iteration to approximate the criticalness of original input data instances. The proposed approach can be used by changing a few lines of code in a standard stochastic gradient descent (SGD) procedure, and we demonstrate experimentally, on NN regression, SVM classification, and CNN training, that for large datasets, it accelerates model training process by an average of 3.61 times while only incurring accuracy losses of 0.37 percent. [ABSTRACT FROM AUTHOR]

Enhancing the visibility in extreme low-light environments is a challenging task. Under nearly lightless condition, existing image denoising methods could easily break down due to significantly low SNR. In this paper, we systematically study the noise statistics in the imaging pipeline of CMOS photosensors, and formulate a comprehensive noise model that can accurately characterize the real noise structures. Our novel model considers the noise sources caused by digital camera electronics which are largely overlooked by existing methods yet have significant influence on raw measurement in the dark. It provides a way to decouple the intricate noise structure into different statistical distributions with physical interpretations. Moreover, our noise model can be used to synthesize realistic training data for learning-based low-light denoising algorithms. In this regard, although promising results have been shown recently with deep convolutional neural networks, the success heavily depends on abundant noisy-clean image pairs for training, which are tremendously difficult to obtain in practice. Generalizing their trained models to images from new devices is also problematic. Extensive experiments on multiple low-light denoising datasets – including a newly collected one in this work covering various devices – show that a deep neural network trained with our proposed noise formation model can reach surprisingly-high accuracy. The results are on par with or sometimes even outperform training with paired real data, opening a new door to real-world extreme low-light photography. [ABSTRACT FROM AUTHOR]

Time series classification is an important research topic in machine learning and data mining communities, since time series data exist in many application domains. Recent studies have shown that machine learning algorithms could benefit from good feature representation, explaining why deep learning has achieved breakthrough performance in many tasks. In deep learning, the convolutional neural network (CNN) is one of the most well-known approaches, since it incorporates feature learning and classification task in a unified network architecture. Although CNN has been successfully applied to image and text domains, it is still a challenge to apply CNN to time series data. This paper proposes a tensor scheme along with a novel deep learning architecture called multivariate convolutional neural network (MVCNN) for multivariate time series classification, in which the proposed architecture considers multivariate and lag-feature characteristics. We evaluate our proposed method with the prognostics and health management (PHM) 2015 challenge data, and compare with several algorithms. The experimental results indicate that the proposed method outperforms the other alternatives using the prediction score, which is the evaluation metric used by the PHM Society 2015 data challenge. Besides performance evaluation, we provide detailed analysis about the proposed method. [ABSTRACT FROM AUTHOR]

This paper proposes a novel intelligent fault diagnosis method to automatically identify different health conditions of wind turbine (WT) gearbox. Unlike traditional approaches, where feature extraction and classification are separately designed and performed, this paper aims to automatically learn effective fault features directly from raw vibration signals while classify the type of faults in a single framework, thus providing an end-to-end learning-based fault diagnosis system for WT gearbox without additional signal processing and diagnostic expertise. Considering the multiscale characteristics inherent in vibration signals of a gearbox, a new multiscale convolutional neural network (MSCNN) architecture is proposed to perform multiscale feature extraction and classification simultaneously. The proposed MSCNN incorporates multiscale learning into the traditional CNN architecture, which has two merits: 1) high-level fault features can be effectively learned by the hierarchical learning structure with multiple pairs of convolutional and pooling layers; and 2) multiscale learning scheme can capture complementary and rich diagnosis information at different scales. This greatly improves the feature learning ability and enables better diagnosis performance. The proposed MSCNN approach is evaluated through experiments on a WT gearbox test rig. Experimental results and comprehensive comparison analysis with respect to the traditional CNN and traditional multiscale feature extractors have demonstrated the superiority of the proposed method. [ABSTRACT FROM AUTHOR]

This paper proposes a novel method for voltage dip classification using deep convolutional neural networks. The main contributions of this paper include: 1) to propose a new effective deep convolutional neural network architecture for automatically learning voltage dip features, rather than extracting hand-crafted features; 2) to employ the deep learning in an effective two-dimensional (2-D) transform domain, under space-phasor model (SPM), for efficient learning of dip features; 3) to characterize voltage dips by 2-D SPM-based deep learning, which leads to voltage dip features independent of the duration and sampling frequency of dip recordings; and 4) to develop robust automatically-extracted features that are insensitive to training and test datasets measured from different countries/regions. Experiments were conducted on datasets containing about 6000 measured voltage dips spread over seven classes measured from several different countries. Results have shown good performance of the proposed method: average classification rate is about 97% and false alarm rate is about 0.50%. The test results from the proposed method are compared with the results from two existing dip classification methods. The proposed method is shown to outperform these existing methods. [ABSTRACT FROM AUTHOR]

A small ISO and a small exposure time are usually used to capture an image in back- or low-light condition which results in an image with negligible motion blur and small noise but looks dark. In this paper, a single image brightening algorithm is introduced to brighten such an image. The proposed algorithm includes a unique hybrid learning framework to generate two virtual images with large exposure times. The virtual images are first generated via intensity mapping functions (IMFs) which are computed using camera response functions (CRFs) and this is a model-driven approach. Both the virtual images are then enhanced by using a data-driven approach, i.e. a residual convolutional neural network to approach the ground truth images. The model-driven approach and the data-driven one compensate each other in the proposed hybrid learning framework. The final brightened image is obtained by fusing the original image and two virtual images via a multi-scale exposure fusion algorithm with properly defined weights. Experimental results show that the proposed brightening algorithm outperforms existing algorithms in terms of MEF-SSIM metric. [ABSTRACT FROM AUTHOR]

To protect underage people from accessing objectionable videos in the Internet, an effective objectionable video recognition algorithm is necessary for web filtering. Recently, the multi-instance learning has been introduced for objectionable video recognition and achieves impressive results. However, hand-crafted features as well as redundant and noisy frames in objectionable videos become an intractable problem that inevitably degrades the recognition performance. In this paper, we propose a novel representative prototype selection algorithm embedding deep multi-instance representation learning. In the proposed method, an improved convolutional neural network is designed for multimodal multi-instance feature learning and a self-expressive dictionary learning model based on sparse and low rank constraint is designed to select the representative prototypes from each subspace of instances. Then the bag-level feature is constructed via mapping the bag to the selected prototypes. Experiments on three objectionable video sets show the effectiveness of our method for objectionable video recognition. [ABSTRACT FROM AUTHOR]

Recently, a few image compressed sensing (CS) methods based on deep learning have been developed, which achieve remarkable reconstruction quality with low computational complexity. However, these existing deep learning-based image CS methods focus on exploring intraframe correlation while ignoring interframe cues, resulting in inefficiency when directly applied to video CS. In this paper, we propose a novel video CS framework based on a convolutional neural network (dubbed VCSNet) to explore both intraframe and interframe correlations. Specifically, VCSNet divides the video sequence into multiple groups of pictures (GOPs), of which the first frame is a keyframe that is sampled at a higher sampling ratio than the other nonkeyframes. In a GOP, the block-based framewise sampling by a convolution layer is proposed, which leads to the sampling matrix being automatically optimized. In the reconstruction process, the framewise initial reconstruction by using a linear convolutional neural network is first presented, which effectively utilizes the intraframe correlation. Then, the deep reconstruction with multilevel feature compensation is proposed, which compensates the nonkeyframes with the keyframe in a multilevel feature compensation manner. Such multilevel feature compensation allows the network to better explore both intraframe and interframe correlations. Extensive experiments on six benchmark videos show that VCSNet provides better performance over state-of-the-art video CS methods and deep learning-based image CS methods in both objective and subjective reconstruction quality. [ABSTRACT FROM AUTHOR]

Attention mechanisms have significantly boosted the performance of video classification neural networks thanks to the utilization of perspective contexts. However, the current research on video attention generally focuses on adopting a specific aspect of contexts (e.g., channel, spatial/temporal, or global context) to refine the features and neglects their underlying correlation when computing attentions. This leads to incomplete context utilization and hence bears the weakness of limited performance improvement. To tackle the problem, this paper proposes an efficient attention-in-attention (AIA) method for element-wise feature refinement, which investigates the feasibility of inserting the channel context into the spatio-temporal attention learning module, referred to as CinST, and also its reverse variant, referred to as STinC. Specifically, we instantiate the video feature contexts as dynamics aggregated along a specific axis with global average and max pooling operations. The workflow of an AIA module is that the first attention block uses one kind of context information to guide the gating weights calculation of the second attention that targets at the other context. Moreover, all the computational operations in attention units act on the pooled dimension, which results in quite few computational cost increase (< 0.02%). To verify our method, we densely integrate it into two classical video network backbones and conduct extensive experiments on several standard video classification benchmarks. The source code of our AIA is available at https://github.com/haoyanbin918/Attention-in-Attention. [ABSTRACT FROM AUTHOR]

With the fast development of various positioning techniques such as Global Position System (GPS), mobile devices and remote sensing, spatio-temporal data has become increasingly available nowadays. Mining valuable knowledge from spatio-temporal data is critically important to many real-world applications including human mobility understanding, smart transportation, urban planning, public safety, health care and environmental management. As the number, volume and resolution of spatio-temporal data increase rapidly, traditional data mining methods, especially statistics-based methods for dealing with such data are becoming overwhelmed. Recently deep learning models such as recurrent neural network (RNN) and convolutional neural network (CNN) have achieved remarkable success in many domains due to the powerful ability in automatic feature representation learning, and are also widely applied in various spatio-temporal data mining (STDM) tasks such as predictive learning, anomaly detection and classification. In this paper, we provide a comprehensive review of recent progress in applying deep learning techniques for STDM. We first categorize the spatio-temporal data into five different types, and then briefly introduce the deep learning models that are widely used in STDM. Next, we classify existing literature based on the types of spatio-temporal data, the data mining tasks, and the deep learning models, followed by the applications of deep learning for STDM in different domains including transportation, on-demand service, climate & weather analysis, human mobility, location-based social network, crime analysis, and neuroscience. Finally, we conclude the limitations of current research and point out future research directions. [ABSTRACT FROM AUTHOR]

Since the brain lesion detection and classification is a vital diagnosis task, in this paper, the problem of brain magnetic resonance imaging (MRI) classification is investigated. Recent advantages in machine learning and deep learning allows the researchers to develop the robust computer-aided diagnosis (CAD) tools for classification of brain lesions. Feature extraction is an essential step in any machine learning scheme. Time-frequency analysis methods provide localized information that makes them more attractive for image classification applications. Owing to the advantages of two-dimensional discrete orthonormal Stockwell transform (2D DOST), we propose to use it to extract the efficient features from brain MRIs and obtain the feature matrix. Since there are some irrelevant features, two-directional two-dimensional principal component analysis ((2D)2PCA) is used to reduce the dimension of the feature matrix. Finally, convolution neural networks (CNNs) are designed and trained for MRI classification. Simulation results indicate that the proposed CAD tool outperforms the recently introduced ones and can efficiently diagnose the MRI scans. [ABSTRACT FROM AUTHOR]

Deep learning algorithms have obtained great success in semantic segmentation of very high-resolution (VHR) remote sensing images. Nevertheless, training these models generally requires a large amount of accurate pixel-wise annotations, which is very laborious and time-consuming to collect. To reduce the annotation burden, this paper proposes a consistency-regularized region-growing network (CRGNet) to achieve semantic segmentation of VHR remote sensing images with point-level annotations. The key idea of CRGNet is to iteratively select unlabeled pixels with high confidence to expand the annotated area from the original sparse points. However, since there may exist some errors and noises in the expanded annotations, directly learning from them may mislead the training of the network. To this end, we further propose the consistency regularization strategy, where a base classifier and an expanded classifier are employed. Specifically, the base classifier is supervised by the original sparse annotations, while the expanded classifier aims to learn from the expanded annotations generated by the base classifier with the region-growing mechanism. The consistency regularization is thereby achieved by minimizing the discrepancy between the predictions from both the base and the expanded classifiers. We find such a simple regularization strategy is yet very useful to control the quality of the region-growing mechanism. Extensive experiments on two benchmark datasets demonstrate that the proposed CRGNet significantly outperforms the existing state-of-the-art methods. Codes and pre-trained models are available online (https://github.com/YonghaoXu/CRGNet). [ABSTRACT FROM AUTHOR]

In this paper, we focus on X-ray images (X-radiographs) of paintings with concealed sub-surface designs (e.g., deriving from reuse of the painting support or revision of a composition by the artist), which therefore include contributions from both the surface painting and the concealed features. In particular, we propose a self-supervised deep learning-based image separation approach that can be applied to the X-ray images from such paintings to separate them into two hypothetical X-ray images. One of these reconstructed images is related to the X-ray image of the concealed painting, while the second one contains only information related to the X-ray image of the visible painting. The proposed separation network consists of two components: the analysis and the synthesis sub-networks. The analysis sub-network is based on learned coupled iterative shrinkage thresholding algorithms (LCISTA) designed using algorithm unrolling techniques, and the synthesis sub-network consists of several linear mappings. The learning algorithm operates in a totally self-supervised fashion without requiring a sample set that contains both the mixed X-ray images and the separated ones. The proposed method is demonstrated on a real painting with concealed content, Do na Isabel de Porcel by Francisco de Goya, to show its effectiveness. [ABSTRACT FROM AUTHOR]

Monitoring the depth of unconsciousness during anesthesia is beneficial in both clinical settings and neuroscience investigations to understand brain mechanisms. Electroencephalogram (EEG) has been used as an objective means of characterizing brain altered arousal and/or cognition states induced by anesthetics in real-time. Different general anesthetics affect cerebral electrical activities in different ways. However, the performance of conventional machine learning models on EEG data is unsatisfactory due to the low Signal to Noise Ratio (SNR) in the EEG signals, especially in the office-based anesthesia EEG setting. Deep learning models have been used widely in the field of Brain Computer Interface (BCI) to perform classification and pattern recognition tasks due to their capability of good generalization and handling noises. Compared to other BCI applications, where deep learning has demonstrated encouraging results, the deep learning approach for classifying different brain consciousness states under anesthesia has been much less investigated. In this paper, we propose a new framework based on meta-learning using deep neural networks, named Anes-MetaNet, to classify brain states under anesthetics. The Anes-MetaNet is composed of Convolutional Neural Networks (CNN) to extract power spectrum features, and a time consequence model based on Long Short-Term Memory (LSTM) networks to capture the temporal dependencies, and a meta-learning framework to handle large cross-subject variability. We use a multi-stage training paradigm to improve the performance, which is justified by visualizing the high-level feature mapping. Experiments on the office-based anesthesia EEG dataset demonstrate the effectiveness of our proposed Anes-MetaNet by comparison of existing methods. [ABSTRACT FROM AUTHOR]

Pain is an integrative phenomenon coupled with dynamic interactions between sensory and contextual processes in the brain, often associated with detectable neurophysiological changes. Recent advances in brain activity recording tools and machine learning technologies have intrigued research and development of neurocomputing techniques for objective and neurophysiology-based pain detection. This paper proposes a pain detection framework based on Electroencephalogram (EEG) and deep convolutional neural networks (CNN). The feasibility of CNN is investigated for distinguishing induced pain state from resting state in the recruitment of 10 chronic back pain patients. The experimental study recorded EEG signals in two phases: 1. movement stimulation (MS), where induces back pain by executing predefined movement tasks; 2. video stimulation (VS), where induces back pain perception by watching a set of video clips. A multi-layer CNN classifies the EEG segments during the resting state and the pain state. The novel approach offers high and robust performance and hence is significant in building a powerful pain detection algorithm. The area under the receiver operating characteristic curve (AUC) of our approach is 0.83 ± 0.09 and 0.81 ± 0.15, in MS and VS, respectively, higher than the state-of-the-art approaches. The sub-brain-areas are also analyzed, to examine distinct brain topographies relevant for pain detection. The results indicate that MS-induced pain tends to evoke a generalized brain area, while the evoked area is relatively partial under VS-induced pain. This work may provide a new solution for researchers and clinical practitioners on pain detection. [ABSTRACT FROM AUTHOR]

The rapid development of Convolutional Neural Networks (CNNs) in recent years has triggered significant breakthroughs in many machine learning (ML) applications. The ability to understand and compare various CNN models available is thus essential. The conventional approach with visualizing each model's quantitative features, such as classification accuracy and computational complexity, is not sufficient for a deeper understanding and comparison of the behaviors of different models. Moreover, most of the existing tools for assessing CNN behaviors only support comparison between two models and lack the flexibility of customizing the analysis tasks according to user needs. This paper presents a visual analytics system, VAC-CNN (Visual Analytics for Comparing CNNs), that supports the in-depth inspection of a single CNN model as well as comparative studies of two or more models. The ability to compare a larger number of (e.g., tens of) models especially distinguishes our system from previous ones. With a carefully designed model visualization and explaining support, VAC-CNN facilitates a highly interactive workflow that promptly presents both quantitative and qualitative information at each analysis stage. We demonstrate VAC-CNN's effectiveness for assisting novice ML practitioners in evaluating and comparing multiple CNN models through two use cases and one preliminary evaluation study using the image classification tasks on the ImageNet dataset. [ABSTRACT FROM AUTHOR]

Along with the commercialization of evolved multimedia broadcast multicast services (eMBMS), the number of mobile broadcasting users is growing notably. Previous works reveal that the accuracy of mobile channel estimation will significantly impact the quality of broadcasting services. Motivated by this fact, we apply machine learning (ML) to the fifth-generation Radio Access Network (5G RAN) slicing in this paper for the estimation and the prediction of the channel status in mobile scenarios. More specifically, a cascaded convolutional neural network (CNN)-long short term memory network (LSTM) architecture is developed to achieve channel estimation for mobile broadcasting users. The energy efficiency of the base station (BS) is modeled mathematically, and the sub-optimal solution is achieved by deep Q-Network (DQN) based on the available channel status. Finally, we present the simulation results to justify the performance of our proposed schemes. [ABSTRACT FROM AUTHOR]

This paper presents a new solution that enables the use of transfer learning for cuff-less blood pressure (BP) monitoring via short duration of photoplethysmogram (PPG). The proposed method estimates BP with low computational budget by 1) creating images from segments of PPG via visibility graph (VG), hence, preserving the temporal information of the PPG waveform, 2) using pre-trained deep convolutional neural network (CNN) to extract feature vectors from VG images, and 3) solving for the weights and bias between the feature vectors and the reference BPs with ridge regression. Using the University of California Irvine (UCI) database consisting of 348 records, the proposed method achieves a best error performance of $0.00\pm 8.46$ mmHg for systolic blood pressure (SBP), and $-0.04\pm 5.36$ mmHg for diastolic blood pressure (DBP), respectively, in terms of the mean error (ME) and the standard deviation (SD) of error, ranking grade B for SBP and grade A for DBP under the British Hypertension Society (BHS) protocol. Our novel data-driven method offers a computationally-efficient end-to-end solution for rapid and user-friendly cuff-less PPG-based BP estimation. [ABSTRACT FROM AUTHOR]

Using acoustic waves to estimate fluid concentration is a promising technology due to its practicality and non-intrusive aspect, especially for medical applications. The existing approaches are exclusively based on the correlation between the reflection coefficient and the concentration. However, these techniques are limited by the high sensitivity of the reflection coefficient to environment conditions changes, even slight ones. This introduces inaccuracies that cannot be tolerated in medical applications. This paper proposed a deep learning model, Fluid Concentration Estimation Convolutional Neural Network (FCE-CNN), to estimate fluid concentration. Instead of using only the reflection coefficient, we train our model to detect concentration-related patterns based on the whole received acoustic signal. FCE-CNN shows promising results that overcome the state-of-the-art limitations. Specifically, our model that is able to estimate fluid concentration with 98.5% accuracy using ultra high frequency acoustic waves. [ABSTRACT FROM AUTHOR]

Convolutional operator learning is gaining attention in many signal processing and computer vision applications. Learning kernels has mostly relied on so-called patch-domain approaches that extract and store many overlapping patches across training signals. Due to memory demands, patch-domain methods have limitations when learning kernels from large datasets – particularly with multi-layered structures, e.g., convolutional neural networks – or when applying the learned kernels to high-dimensional signal recovery problems. The so-called convolution approach does not store many overlapping patches, and thus overcomes the memory problems particularly with careful algorithmic designs; it has been studied within the “synthesis” signal model, e.g., convolutional dictionary learning. This paper proposes a new convolutional analysis operator learning (CAOL) framework that learns an analysis sparsifying regularizer with the convolution perspective, and develops a new convergent Block Proximal Extrapolated Gradient method using a Majorizer (BPEG-M) to solve the corresponding block multi-nonconvex problems. To learn diverse filters within the CAOL framework, this paper introduces an orthogonality constraint that enforces a tight-frame filter condition, and a regularizer that promotes diversity between filters. Numerical experiments show that, with sharp majorizers, BPEG-M significantly accelerates the CAOL convergence rate compared to the state-of-the-art block proximal gradient (BPG) method. Numerical experiments for sparse-view computational tomography show that a convolutional sparsifying regularizer learned via CAOL significantly improves reconstruction quality compared to a conventional edge-preserving regularizer. Using more and wider kernels in a learned regularizer better preserves edges in reconstructed images. [ABSTRACT FROM AUTHOR]

Recently, deep learning, which are able to extract automatically features from data, has achieved state-of-the-art performance across a variety of sensor based human activity recognition (HAR) tasks. However, the existing deep neural networks are usually trained with a global loss, and all hidden layer weights have to be always kept in memory before the forward and backward pass has completed. The backward locking phenomenon prevents the reuse of memory, which is a crucial limitation for wearable activity recognition. In the paper, we proposed a layer-wise convolutional neural networks (CNN) with local loss for the use of HAR task. To our knowledge, this paper is the first that uses local loss based CNN for HAR in ubiquitous and wearable computing arena. We performed experiments on five public HAR datasets including UCI HAR dataset, OPPOTUNITY dataset, UniMib-SHAR dataset, PAMAP dataset, and WISDM dataset. The results show that local loss works better than global loss for tested baseline architectures. At no extra cost, the local loss can approach the state-of-the-arts on a variety of HAR datasets, even though the number of parameters was smaller. We believe that the layer-wise CNN with local loss can be used to update the existing deep HAR methods. [ABSTRACT FROM AUTHOR]

Surface defect detection has received increased attention in relation to the product quality and industry safety. This paper develops an image pyramid convolution neural network (IPCNN) model to detect surface defects in images. The IPCNN is an improvement of the Mask rcnn model. It combines image pyramid and deep convolution neural network to extract pyramid features for defect detections. First, it constructs an image pyramid. Then, images from the pyramid are processed with a deep residual neural network to extract features. Extracted features are then fused and processed with a feature pyramid network to generate pyramid features. A region proposal network then operates on the pyramid features to generate defect bounding boxes and make classifications. Finally, a fully convolutional neural network generates defect masks in the detected bounding boxes. The IPCNN was evaluated on the publicly available COCO dataset and improved the Mask rcnn by approximately one point. Additionally, it generated high precision and recall values on inspecting the oil leak defect on freight train with a small dataset. The comparison with state-of-the-art methods on freight train defect detection showed the promising results of IPCNN. The efficiency of the IPCNN is 0.243 seconds per frame. [ABSTRACT FROM AUTHOR]

The processing quality of laser cladding is a topic of interest to laser machine manufacturers. The management of various experimental data and process quality of the laser machine can effectively guide the customer to better adjust the processing parameters. This study finds that the processing quality of laser cladding is related to the signal of the coaxial image. Therefore, this study uses a machine learning method to establish a model of coaxial image and laser processing quality. The study does not merely implement a single machine learning method but also compares various machine learning algorithms. Convolutional neural networks and autoencoders are implemented as algorithms for the feature extraction phase. Linear regression, random forest, support vector machine, and SoftMax neural networks are implemented as algorithms for classification. The receiver operating characteristic curve and the accuracy rate are the result indicators of this paper. The experimental results show that there is indeed a correlation between the laser processing quality and the coaxial image, and the algorithm in this study can effectively supervise the processing quality of laser cladding. [ABSTRACT FROM AUTHOR]

This paper develops a hierarchical deep learning machine (HDLM) to efficiently achieve both quantitative and qualitative online transient stability prediction (TSP). For the sake of improving its online efficiency, multiple generators’ fault-on trajectories as well as the two closest data-points in pre-/post-fault stages are acquired by PMUs to form its raw inputs. An anti-noise graphical transient characterization technique is tactfully designed to transform multiplex trajectories into 2-D images, within which system-wide transients are concisely described. Then, following the divide-and-conquer philosophy, the HDLM trains a two-level convolutional neural network (CNN) based regression model. With stability margin regressions hierarchically refined, it manages to perform reliable and adaptive online TSP almost immediately after fault clearance. Test results on the IEEE 39-bus test system and the real-world Guangdong Power Grid in South China demonstrate the HDLM's superior performances on both stability status and stability margin predictions. [ABSTRACT FROM AUTHOR]

Fault diagnosis is vital for modern industry, and an increasing number of intelligent methods have been proposed for the fault diagnosis. However, most of the studies focus on distinguishing different fault patterns while ignoring fault deterioration. In this paper, a new hierarchical convolutional neural network (HCNN) is proposed as the two-level hierarchical diagnosis network, and it has two characteristics: 1) the fault pattern and fault severity are modeled as one hierarchical structure and 2) the fault pattern and fault severity can be estimated at the same time. Based on these, a new structure of HCNN is designed, which has two classifiers. Then, a two-stage training method is developed for HCNN to train these two classifiers at once training. The proposed HCNN is conducted on three case studies and has achieved state-of-the-art results. The results show that HCNN outperforms traditional two-layer hierarchical fault diagnosis network, and other machine learning and deep learning methods. [ABSTRACT FROM AUTHOR]

The purpose of this paper is the development of data science models for the detection of empty on-street parking spaces in urban road networks based on data provided by in-vehicle cameras that are already, or soon will be, a standard vehicle equipment. A rolling spatial interval is used to identify the existence of an on-street parking space and the properties of empty spaces are used to determine the availability of the parking space. Convolutional neural networks are developed, trained, and evaluated with the use of images from a moving vehicle camera. The images are preprocessed and converted to suitable matrices, so that only the useful information for the empty on-street parking space detection problem is preserved. The optimized convolutional networks, in terms of structural and learning parameters, provided predictions for the detection of empty on-street parking spaces with approximately 90% average accuracy. The proposed model performs better than the relatively complex SVMs, which supports its appropriateness as an approach. Finally, the implementation of a framework, which integrates the developed models to produce meaningful parking information for drivers in real time, is discussed. [ABSTRACT FROM AUTHOR]

In this paper, we focus on a defect analysis task that requires engineers to identify the causes of yield reduction from defect classification results. We organize the analysis work into three phases: defect classification, defect trend monitoring and detailed classification. To support the first and third engineer’s analytical work, we use a convolutional neural network based on the transfer learning method for automatic defect classification. We evaluated our proposed methods on real semiconductor fabrication data sets by performing a defect classification task using a scanning electron microscope image and thoroughly examining its performance. We concluded that the proposed method can classify defect images with high accuracy while lowering labor costs equivalent to one-third the labor required for manual inspection work. [ABSTRACT FROM AUTHOR]

Deep convolutional neural networks (DCNNs) are one of the most promising deep learning techniques and have been recognized as the dominant approach for almost all recognition and detection tasks. The computation of DCNNs is memory intensive due to large feature maps and neuron connections, and the performance highly depends on the capability of hardware resources. With the recent trend of wearable devices and Internet of Things, it becomes desirable to integrate the DCNNs onto embedded and portable devices that require low power and energy consumptions and small hardware footprints. Recently stochastic computing (SC)-DCNN demonstrated that SC as a low-cost substitute to binary-based computing radically simplifies the hardware implementation of arithmetic units and has the potential to satisfy the stringent power requirements in embedded devices. In SC, many arithmetic operations that are resource-consuming in binary designs can be implemented with very simple hardware logic, alleviating the extensive computational complexity. It offers a colossal design space for integration and optimization due to its reduced area and soft error resiliency. In this paper, we present HEIF, a highly efficient SC-based inference framework of the large-scale DCNNs, with broad applications including (but not limited to) LeNet-5 and AlexNet, that achieves high energy efficiency and low area/hardware cost. Compared to SC-DCNN, HEIF features: 1) the first (to the best of our knowledge) SC-based rectified linear unit activation function to catch up with the recent advances in software models and mitigate degradation in application-level accuracy; 2) the redesigned approximate parallel counter and optimized stochastic multiplication using transmission gates and inverse mirror adders; and 3) the new optimization of weight storage using clustering. Most importantly, to achieve maximum energy efficiency while maintaining acceptable accuracy, HEIF considers holistic optimizations on cascade connection of function blocks in DCNN, pipelining technique, and bit-stream length reduction. Experimental results show that in large-scale applications HEIF outperforms previous SC-DCNN by the throughput of $4.1\times $ , by area efficiency of up to $6.5\times $ , and achieves up to ${5.6\times }$ energy improvement. [ABSTRACT FROM AUTHOR]

Facial expression recognition is important for human–computer interaction and other applications. Several facial expression datasets have been published in recent decades and have enabled improvements in algorithms for classifying emotions. However, recognition of realistic expressions in real-world conditions is still challenging because of uncontrolled conditions, such as lighting, brightness, pose, and occlusion. In this paper, we propose a light attention embedding network based on the spatial attention mechanism (LAENet-SA), which can focus on locations in an image that are relevant to emotion. LAENet-SA allows a small number of attention modules to be embedded and can be constructed from typical convolutional neural networks. The performance of LAENet-SA on facial expression recognition has been validated on three facial expression datasets, including a lab-controlled dataset and two in-the-wild datasets. Experimental results show that LAENet-SA improved the performance on each dataset, compared with state-of-the-art methods, and achieved better generalization when tested on facial images with occlusion. [ABSTRACT FROM AUTHOR]

Human gait refers to the propulsion achieved by the effort of human limbs, a reflex progression resulting from the rhythmic reciprocal bursts of flexor and extensor activity. Several quantitative models are followed by health professionals to diagnose gait abnormality. Marker-based gait quantification is considered a gold standard by the research and health communities. It reconstructs motion in 3D and provides parameters to measure gait. But, it is an expensive and intrusive technique, limited to soft tissue artefact, prone to incorrect marker positioning, and skin sensitivity problems. Hence, markerless, swiftly deployable, non-intrusive, camera-less prototypes would be a game changing possibility, and an example is proposed here. This paper illustrates a 3D gait motion analyser employing impulse radio ultra-wide band (IR-UWB) wireless technology. The prototype can measure 3D motion and determine quantitative parameters considering anatomical reference planes. Knee angles have been calculated from the gait by applying vector algebra. Simultaneously, the model has been corroborated with the popular markerless camera based 3D motion capturing system, the Kinect sensor. Bland and Altman (B&A) statistics has been applied to the proposed prototype and Kinect sensor results to verify the measurement agreement. Finally, the proposed prototype has been incorporated with popular supervised machine learning such as, $k$ -nearest neighbour ($kNN$), support vector machine (SVM) and the deep learning technique deep neural multilayer perceptron (DMLP) network to automatically recognize gait abnormalities, with promising results presented. [ABSTRACT FROM AUTHOR]

Deep neural networks in deep learning have been widely demonstrated to have higher accuracy and distinct advantages over traditional machine learning methods in extracting data features. While convolutional neural networks (CNNs) have shown great success in feature extraction and audio classification, it is important to note that real-time audios are dependent on previous scenes. Also, the main drawback of deep learning algorithms is that they need a huge number of datasets to indicate their efficient performance. In this paper, a recurrent neural network (RNN) combined with CNN is proposed to address this problem. Moreover, a Deep Convolutional Generative Adversarial Network (DCGAN) is used for high-quality data augmentation. This data augmentation technique is applied to the UrbanSound8K dataset to improve the environmental sound classification. Batch normalization, transfer learning, and three feature representations map are used to improve the model accuracy. The results show that the generated images by DCGAN have similar features to the original training images and has the capability to generate spectrograms and improve the classification accuracy. Experimental results on UrbanSound8K datasets demonstrate that the proposed CNN-RNN architecture achieves better performance than the state-of-the-art classification models. [ABSTRACT FROM AUTHOR]

Human movement detection based on millimeter-wave radar sensors is a technology of interest in various areas such as for smart surveillance, security, behavioral biometrics, biomedical systems, robotics, etc. This paper shows the feasibility and effectiveness of using a compact 24 GHz Doppler radar with a built-in low-noise microwave amplifier (LNA) for detecting and extracting signals coming from human motion. Reliability and accuracy is assessed based on a comprehensive theoretical analysis, and on simulation results followed by experimental investigations. A continuous wavelet transform (CWT) is used to decompose in-phase and quadrature (${I/Q}$) baseband signals to extract information about the dynamics of the system. The dataset consists of 1000 recordings in 8 motion classes (standing, walking, sitting, etc.). We propose to apply a two-channel convolutional neural network (CNN), which is composed of two CNN channels, for learning high-level features from time-domain signals and CWT spectrograms. One channel has four one-dimensional (1D) convolutional and pooling layers, and the other channel is made of three two-dimensional (2D) convolutional and pooling layers. In addition, the ${I/Q}$ signals are denoised by using Savitzky-Golay filtering, and both noisy and denoised signals are used as input signals for deep learning data augmentation purpose. We achieved an overall classification accuracy rate of 98.85% in motion classification for a two-branch CNN architecture, and an accuracy rate of 95.3% for a one-branch 2D-CNN. Our results show that a dual-channel CNN model can greatly increase the classification capabilities of human motion recognition and classification, and the proposed method can be effectively used with various radar signal classifications. [ABSTRACT FROM AUTHOR]

Tracking-by-detection algorithms have considerably enhanced tracking performance with the introduction of recent convolutional neural networks (CNNs). However, most trackers directly exploit standard scalar-output CNN features, which may not capture enough feature encoding information, instead of aggregated CNN features of vector-output form. In this paper, we propose an end-to-end feature aggregation capsule framework. First, based on the existing CNN network, we aggregate a certain number of similar position-aware CNN features into a capsule to model the feature similarity. The acquired vector-level feature capsules (rather than previous scalar-level pointwise features) are utilized for differentiation learning. We then propose a group attention module to better model the entity representation between different capsule groups thus optimizes total discriminative capability. Third, to reduce the prediction interference resulted by the side effect of dimension rising within capsules, we propose a penalty attention module. Such strategy could dynamically adjust values of neurons by estimating whether they are beneficial or harmful to tracking. Experimental results on five representative benchmarks (UAVDT, DTB70, UAV123, VOT2016 and VOT2018) demonstrate the excellent tracking performance of our dual attention based capsule tracker (DACapT). Specially, it exceeds the previous top tracker by 4.6%/1.9% in precision/success evaluations on UAVDT. [ABSTRACT FROM AUTHOR]

Remote sensing scene classification (RSSC) is a hotspot and play very important role in the field of remote sensing image interpretation in recent years. With the recent development of the convolutional neural networks, a significant breakthrough has been made in the classification of remote sensing scenes. Many objects form complex and diverse scenes through spatial combination and association, which makes it difficult to classify remote sensing image scenes. The problem of insufficient differentiation of feature representations extracted by Convolutional Neural Networks (CNNs) still exists, which is mainly due to the characteristics of similarity for inter-class images and diversity for intra-class images. In this paper, we propose a remote sensing image scene classification method via Multi-Branch Local Attention Network (MBLANet), where Convolutional Local Attention Module (CLAM) is embedded into all down-sampling blocks and residual blocks of ResNet backbone. CLAM contains two submodules, Convolutional Channel Attention Module (CCAM) and Local Spatial Attention Module (LSAM). The two submodules are placed in parallel to obtain both channel and spatial attentions, which helps to emphasize the main target in the complex background and improve the ability of feature representation. Extensive experiments on three benchmark datasets show that our method is better than state-of-the-art methods. [ABSTRACT FROM AUTHOR]