Search

Estimating depth from RGB images is a long-standing ill-posed problem, which has been explored for decades by the computer vision, graphics, and machine learning communities. Among the existing techniques, stereo matching remains one of the most widely used in the literature due to its strong connection to the human binocular system. Traditionally, stereo-based depth estimation has been addressed through matching hand-crafted features across multiple images. Despite the extensive amount of research, these traditional techniques still suffer in the presence of highly textured areas, large uniform regions, and occlusions. Motivated by their growing success in solving various 2D and 3D vision problems, deep learning for stereo-based depth estimation has attracted a growing interest from the community, with more than 150 papers published in this area between 2014 and 2019. This new generation of methods has demonstrated a significant leap in performance, enabling applications such as autonomous driving and augmented reality. In this paper, we provide a comprehensive survey of this new and continuously growing field of research, summarize the most commonly used pipelines, and discuss their benefits and limitations. In retrospect of what has been achieved so far, we also conjecture what the future may hold for deep learning-based stereo for depth estimation research. [ABSTRACT FROM AUTHOR]

Radio signal recognition has a wide application in future communication systems and the vehicular communication, whose core is the extraction of signal features such as electromagnetic fingerprints. With the rapid development of artificial intelligence technology, deep learning has made amazing breakthroughs in image recognition, speech recognition and other fields. Deep learning is applied to electromagnetic fingerprint extraction in this paper. Firstly, thousands of the downlink aircraft communications addressing and reporting system (ACARS) signals used for communication between civil aircraft and airport tower are collected and generated. Then a pre-transformation network suitable for electromagnetic signals is constructed to convert one-dimensional signals into two-dimensional feature maps, and afterwards the feature maps are input into the convolutional neural network (CNN) for classification. By adopting the attention modules, the classification results were improved by a few percentage points over the baseline with a little cost. The method proposed in this paper achieves an accuracy rate of 94.1% and can obtain the aircraft type in a shorter time than traditional method. Moreover, the robustness of the proposed model in response to additive Gaussian white noise (AWGN) and phase deviation is studied and tested. [ABSTRACT FROM AUTHOR]

In this paper, simple methodologies of deep learning application to conventional multiple-input multiple-output (MIMO) communication systems are presented. The deep learning technologies with deep neural network (DNN) structure, emerging technologies in various engineering areas, have been actively investigated in the field of communication engineering as well. In the physical layer of conventional communication systems, there are practical challenges of application of DNN: calculating complex number in DNN and designing proper DNN structure for a specific communication system model. This paper proposes and verifies simple solutions for the difficulty. First, we apply a basic DNN structure for signal detection of one-tap MIMO channel. Second, convolutional neural network (CNN) and recurrent neural network (RNN) structures are presented for MIMO system with multipath fading channel. Our DNN structure for one-tap MIMO channel can achieve the optimal maximum likelihood detection performance, and furthermore, our CNN and RNN structures for multipath fading channel can detect the transmitted signal properly. [ABSTRACT FROM AUTHOR]

Convolutional neural networks (CNN) have demonstrated their ability to segment 2D cardiac ultrasound images. However, despite recent successes according to which the intra-observer variability on end-diastole and end-systole images has been reached, CNNs still struggle to leverage temporal information to provide accurate and temporally consistent segmentation maps across the whole cycle. Such consistency is required to accurately describe the cardiac function, a necessary step in diagnosing many cardiovascular diseases. In this paper, we propose a framework to learn the 2D+time apical long-axis cardiac shape such that the segmented sequences can benefit from temporal and anatomical consistency constraints. Our method is a post-processing that takes as input segmented echocardiographic sequences produced by any state-of-the-art method and processes it in two steps to (i) identify spatio-temporal inconsistencies according to the overall dynamics of the cardiac sequence and (ii) correct the inconsistencies. The identification and correction of cardiac inconsistencies relies on a constrained autoencoder trained to learn a physiologically interpretable embedding of cardiac shapes, where we can both detect and fix anomalies. We tested our framework on 98 full-cycle sequences from the CAMUS dataset, which are available alongside this paper. Our temporal regularization method not only improves the accuracy of the segmentation across the whole sequences, but also enforces temporal and anatomical consistency. [ABSTRACT FROM AUTHOR]

Due to the strong penetration ability and high transmission rate of ultra-wideband (UWB) signals, UWB technology plays a very significant role in the field of precise indoor positioning. However, the harsh and volatile indoor environment leads to non-line-of-sight (NLOS) propagation and severe attenuation of UWB signals, which may generate significant ranging and positioning errors. To mitigate NLOS effect and improve positioning accuracy, existing methods use ranging information and channel impulse response (CIR) to identify UWB signal propagation channels. However, these NLOS identification methods often require a priori knowledge and suitable thresholds, and most of them only perform a binary classification between LOS and NLOS in a particular scenario. To address these disadvantages, this paper proposes a novel multi-classification method to classify UWB signal propagation channels based on one-dimensional wavelet packet analysis (ODWPA) and convolutional neural network (CNN). This method first decomposes the CIR of known categories into colored coefficients images using a one-dimensional wavelet packet function, then uses these images to train CNNs with different structures, and finally selects CNN model with the best performance to identify the unknown category of UWB signal propagation channels. The experimental results show that the performance of the proposed method is always the best, regardless of the type of experimental scenario, and the identification accuracy is always higher than 90% and up to 100%. [ABSTRACT FROM AUTHOR]

For 360° video, the existing visual quality assessment (VQA) approaches are designed based on either the whole frames or the cropped patches, ignoring the fact that subjects can only access viewports. When watching 360° video, subjects select viewports through head movement (HM) and then fixate on attractive regions within the viewports through eye movement (EM). Therefore, this paper proposes a two-staged multi-task approach for viewport-based VQA on 360° video. Specifically, we first establish a large-scale VQA dataset of 360° video, called VQA-ODV, which collects the subjective quality scores and the HM and EM data on 600 video sequences. By mining our dataset, we find that the subjective quality of 360° video is related to camera motion, viewport positions and saliency within viewports. Accordingly, we propose a viewport-based convolutional neural network (V-CNN) approach for VQA on 360° video, which has a novel multi-task architecture composed of a viewport proposal network (VP-net) and viewport quality network (VQ-net). The VP-net handles the auxiliary tasks of camera motion detection and viewport proposal, while the VQ-net accomplishes the auxiliary task of viewport saliency prediction and the main task of VQA. The experiments validate that our V-CNN approach significantly advances state-of-the-art VQA performance on 360° video and it is also effective in the three auxiliary tasks. [ABSTRACT FROM AUTHOR]

As the extension of high efficiency video coding (HEVC) standard, three dimensional-HEVC (3D-HEVC) is the latest 3D video coding standard. 3D-HEVC adopts many complicated coding algorithms to generate additional intermediate views for 3D video representation, which result in extremely high coding complexity. Therefore, this paper proposed a fast depth intra coding approach to reduce the 3D-HEVC complexity, which is based on convolutional neural network (CNN). First, we established a database based on the independent view of the depth map, which includes coding unit (CU) partition data of the depth map. Second, we constructed a depth edge classification CNN (DEC-CNN) framework to classify the edges for the depth map and embedded the network into a 3D-HEVC test platform. Finally, we utilized the pixel value of the binarized depth image to correct the above classification results. The experimental results demonstrated that our approach can reduce the intra coding time by 72.5% on average under negligible degradation of coding performance. This result outperforms the other state-of-the-art methods to reduce the coding complexity of 3D-HEVC. [ABSTRACT FROM AUTHOR]

Vehicle type recognition (VTR) is a quite common requirement and one of the key challenges in real surveillance scenarios, such as intelligent traffic and unmanned driving. Usually coarse-grained and fine-grained VTRs are applied in different applications, and the challenge from multiple viewpoints is critical for both cases. In this paper, we propose a feedback-enhancement multi-branch CNN (FM-CNN) to solve the challenge in these two cases. The proposed FM-CNN takes three derivatives of an image as input and leverages the advantages of hierarchical details, feedback enhancement, model average, and stronger robustness to translation and mirroring. A single global cross-entropy loss is insufficient to train such a complex CNN and so we add extra branch losses to enhance feedbacks to each branch. Though reusing pre-trained parameters, we propose a novel parameter update method to adapt FM-CNN to task-specific local visual patterns and global information in new datasets. To test the effectiveness of FM-CNN, we create our own multi-view VTR (MVVTR) data set since there are no such data sets available. And, for fine-grained VTR, we use the CompCars data set. Compared with state-of-the-art classification solutions without special preprocessing, the proposed FM-CNN demonstrates better performance in both coarse-grained and fine-grained scenarios. For coarse-grained VTR, it achieves 94.9% Top-1 accuracy on the MVVTR data set. For fine-grained VTR, it achieves 91.0% Top-1 and 97.8% Top-5 accuracies on the CompCars data set. [ABSTRACT FROM AUTHOR]

The key role of spectrum sensing in cognitive radios attracted substantial research attention to improve the performance of detectors. We consider a general model for the receiver noise with the potential of generalization towards modeling noise in various environments. This general noise model describes accurately noise characteristics ranging from the Gaussian noise to the severe impulsive noise. However, many previous studies are based on the ideal Gaussian noise, and they cannot capture the non-Gaussian models. To provide a robust detector against different behaviors of the noise in various environments, we employ a convolutional neural network (CNN) compatible with different noise models. The proposed CNN detector is data-driven, and due to its single-dimensional input layer, it is consistent with the received signal and requires no pre-processing. The likelihood ratio test (LRT), the Wald, and the Rao tests for this problem are derived to enrich the paper with comparative evaluations of the proposed CNN and conventional model-based approaches and other neural networks. Although various simulated scenarios substantiate the general superiority and robustness of the CNN-based against impulsive noise and mismatch of parameters, it requires higher computational complexity than other discussed detectors. [ABSTRACT FROM AUTHOR]

Predicting the price of soybean meal futures has always been a hot topic in the financial market. It is influenced by many factors and has complex nonlinear characteristics, which makes it difficult to predict accurately. This paper proposes a prediction model for soybean meal futures price based on CNN-ILSTM-SA, which consists of CNN (Convolutional Neural Network), ILSTM (Improved Long Short-Term Memory), and SA (Self-Attention mechanism). CNN can well perform feature extraction on the input time series data. The ILSTM model presented in this paper removes the output gate of LSTM (Long Short-Term Memory) and improves the forget gate and input gate, thus reducing ILSTM’s complexity, shortening training time, and improving learning ability. SA can get the correlation between different input terms and predicted values, and assign different weight coefficients according to the correlation level to improve prediction accuracy. To verify the CNN-ILSTM-SA’s effectiveness, we compare it with eighteen baseline models, and the experimental results show that it is optimal. The MAE (Mean Absolute Error) of CNN-ILSTM-SA is 39.0441, the smallest of all models. As the data of the soybean meal futures price fluctuates in the range of 3500-4500 in experiments, the proportion of the MAE value of 39.0441 is 0.87%-1.15%, which is at an extremely low level; $\mathrm {R}^{2}$ (R-square) is 0.97487, which is closest to 1; The training time is 157.64s.

Ultra-wide Band (UWB) technology has emerged as a pivotal tool for human motion detection, finding applications in diverse areas ranging from smart homes to automotive safety. This paper presents a comprehensive survey of methodologies employed in UWB-based motion detection, elucidating their strengths, challenges, and performance metrics. While several methods, including Convolutional Neural Network (CNN) approaches, have been explored, challenges such as motion state overlaps, the necessity for enhanced spatial resolution, and background noise interference persist. Among the various methods analyzed, the SGWO-based RMDL technique emerges as a frontrunner, offering superior accuracy, reduced mean squared error, and impressive true negative and positive rates. Moreover, its computational efficiency sets a precedent in human motion detection. This paper provides insights into the state-of-the-art Through the wall imaging and human vital signs observation for future research and realtime applications.

Radar echo image sequence prediction is a spatial-temporal forecasting problem, which is one of the main challenges in precipitation nowcasting. In response to issues such as poor extraction of spatiotemporal features and blurred prediction of image sequence in previous models, this paper proposes a spatial-temporal dual discriminator precipitation nowcasting method using SNGAN with InceptionGRU (STD-SNGAN-InceptionGRU). The design of temporal and spatial dual discriminators constrains the generator’s prediction samples, effectively enhancing the spatiotemporal prediction capability of the generator through adversarial training. The generator utilizes multiscale convolution (Inception) and multiscale gate recurrent unit (InceptionGRU) to enhance its ability to extract spatiotemporal features, thereby improving its learning capability for the evolution of radar echo sequences. The comparative experimental results demonstrate that the proposed STD-SNGAN-InceptionGRU model outperforms other algorithms in terms of critical success index (CSI) and probability of detection (POD) at different echo intensities, especially in areas with long time series and high echo intensity. Furthermore, through ablation experiments, this paper proves that the temporal discriminator, spatial discriminator, and InceptionGRU all effectively improve the model’s prediction capability for radar echo images.

This study utilizes a new approach for short-term stock correlation forecasting using a combination of convolutional neural networks (CNN), bi-directional long and short-term memory (BiLSTM), and attention mechanisms to address the issue of information loss due to excessively long input time series data. The objective is to improve the accuracy of short-term equity correlation forecasts, which is essential for the efficient management of financial asset portfolios. The existing time series models usually only take into account the effect of stock historical data characteristics on stock correlation but ignore the fact that stock correlation is affected by multiple factors and undergoes an unknown unstable trend. More efficient methods or algorithms need to be devised to deal with financial data. This paper explores stock metrics and introduces a multi-factor model to quantify the factors affecting stock returns in order to calculate the correlation between stocks. Meanwhile, to cope with the problem of incomplete input data, this paper simultaneously incorporates stock return data that is directly related to the correlation data. In the CNN-BiLSTM-Attention (CLATT) model, the role of the CNN module is to obtain high-dimensional features from the input time series data, and the role of the BiLSTM module is to process the time series information to capture the time-series features in the stock return data and correlation data in the sample. In order to acquire the inherent correlation between the input stock return data and correlation data and to enhance the prediction accuracy of the model, the attention mechanism is finally implemented and used to optimize the weights of the BiLSTM output. In this paper, mean square error (MSE), root mean square error (RMSE), mean absolute error (MAE) metrics, mean absolute percentage of error (MAPE), and the coefficient of determination $R^{2}$ , as well as four benchmark models, are used to evaluate the performance of the proposed model. The result shows that the CLATT model has 57.32% and 33% improvement in the correlation prediction accuracy of different stock portfolios compared to the single LSTM model, and the model prediction accuracy has been improved. Compared with the benchmark CNN-BiLSTM, it also has 56.06% and 28.87% improvement.

As an essential metal, copper has the advantages of electrical conductivity and ductility, which is widely used in power transmission, electronics manufacturing and machining. The fluctuation of copper price has a great impact on the industry, especially on the development of the national economy, so predicting copper price has a great significance for economic development. However, traditional time series prediction models’ prediction accuracy is low. Therefore, this paper proposes a Shanghai copper price forecasting model based on Complete Ensemble Empirical Mode Decomposition with Adaptive Noise (CEEMDAN), Convolutional Neural Network (CNN), Self Attention Gated Unit (SAGU), named CEEMDAN-CNN-SAGU. CEEMDAN decomposes and reconstructs the Shanghai copper price data. In this paper, the zero-cross rate of the Intrinsic Mode Function (IMF) components is calculated, the IMF component with large noise is removed, and the remaining IMFs and Residual term (Res) are reorganized to obtain the high-frequency and low-frequency components. CNN performs convolution operations on the reconstructed components to extract time series features. SAGU is a new time series data prediction model proposed in this paper. SAGU includes two gated units (a forgetting gate and an input gate), and two data processing modules (Self Attention (SA) and Transition (Tra)). The SA module is responsible for processing the input data, redistributing the weight coefficients of different data, and improving the model’s attention to important information. The Tra module improves the low limit of the forgetting gate’s output so that the gated unit can keep the cell state from the previous moment to the current moment as much as possible. This paper uses Shanghai copper trading data from Shanghai Futures Exchange as experimental data. The comparison experiment with the other eleven prediction models shows that the CEEMDAN-CNN-SAGU model outperforms other models in all evaluation indexes.

Video forgery detection is a critical aspect of digital forensics, addressing the challenges posed by the manipulation of video content. This paper presents a novel approach for video forgery detection using Deep Convolutional Neural Networks (DCNN). Leveraging the power of deep learning, our method aims to improve the accuracy and efficiency of object-based forgery detection in advanced video sequences. In the proposed approach, we build upon the foundation of an existing method, which utilizes Convolutional Neural Networks, and introduce innovative modifications to the DCNN architecture. These modifications include data preprocessing, network architecture, and training strategies that enhance the model’s ability to detect tampered objects in video frames. We conduct experiments on the SYSU-OBJFORG dataset, the largest object-based forged video dataset to date, with advanced video encoding standards. Our DCNN-based approach is compared with the existing method, demonstrating superior performance. The results show increased accuracy and robustness in detecting object-based video forgery. This paper not only contributes to the field of video forgery detection but also underscores the potential of deep learning, particularly DCNN, in addressing the evolving challenges of digital video manipulation. The findings open avenues for future research in the localization of forged regions and the application of DCNN in lower bitrate or lower resolution video sequences.

A crucial component within any structural health monitoring system is a pipeline leak detection mechanism, vital for preventing avoidable water loss. Contemporary literature employs machine learning and deep learning for detecting pipeline leaks and cross-correlation for leak localization. The major drawbacks in the existing methodologies are that machine learning and deep learning methods need two different architectures for leak detection and localization, and the cross-correlation needs two sensors with a denoising technique. The primary objective of this paper is to deploy a unified architecture capable of executing both the detection and localization of a leak without any denoising technique and with a single sensor. The proposed technique utilizes the data collected using an Acousto-optic sensor with two different pressures. This paper proposes a novel custom one-dimensional time-series DenseNet for leak detection and localization. The proposed method gives better accuracies compared with the existing one-dimensional DenseNet-121, three different one-dimensional convolutional neural networks (1DCNN), and cross-correlation for two different pressure datasets. The proposed method’s processing time is thirteen times less than the existing one-dimensional DenseNet-121, with the observed average leak detection and localization accuracy of 99.08%. The results state that the proposed novel custom one-dimensional time-series DenseNet accurately detects and localizes the leak with less time.

This paper presents a novel approach for video-based person re-identification using multiple convolutional neural networks (CNNs). Unlike the previous work, we intend to extract a compact yet discriminative appearance representation from several frames rather than the whole sequence. Specifically, given a video, the representative frames are selected based on the walking profile of consecutive frames. A multiple CNN architecture incorporated with feature pooling is proposed to learn and compile the features of the selected representative frames into a compact description about the pedestrian for identification. Experiments are conducted on benchmark data sets to demonstrate the superiority of the proposed method over existing person re-identification approaches. [ABSTRACT FROM AUTHOR]

Low level image restoration is an integral component of modern artificial intelligence (AI) driven camera pipelines. Most of these frameworks are based on deep neural networks which present a massive computational overhead on resource constrained platform like a mobile phone. In this paper, we propose several lightweight low-level modules which can be used to create a computationally low cost variant of a given baseline model. Recent works for efficient neural networks design have mainly focused on classification. However, low-level image processing falls under the ‘image-to-image’ translation genre which requires some additional computational modules not present in classification. This paper seeks to bridge this gap by designing generic efficient modules which can replace essential components used in contemporary deep learning based image restoration networks. We also present and analyse our results highlighting the drawbacks of applying depthwise separable convolutional kernel (a popular method for efficient classification network) for sub-pixel convolution based upsampling (a popular upsampling strategy for low-level vision applications). This shows that concepts from domain of classification cannot always be seamlessly integrated into ‘image-to-image’ translation tasks. We extensively validate our findings on three popular tasks of image inpainting, denoising and super-resolution. Our results show that proposed networks consistently output visually similar reconstructions compared to full capacity baselines with significant reduction of parameters, memory footprint and execution speeds on contemporary mobile devices. [ABSTRACT FROM AUTHOR]

We focus on a fundamental task of detecting meaningful line structures, a.k.a., semantic line, in natural scenes. Many previous methods regard this problem as a special case of object detection and adjust existing object detectors for semantic line detection. However, these methods neglect the inherent characteristics of lines, leading to sub-optimal performance. Lines enjoy much simpler geometric property than complex objects and thus can be compactly parameterized by a few arguments. To better exploit the property of lines, in this paper, we incorporate the classical Hough transform technique into deeply learned representations and propose a one-shot end-to-end learning framework for line detection. By parameterizing lines with slopes and biases, we perform Hough transform to translate deep representations into the parametric domain, in which we perform line detection. Specifically, we aggregate features along candidate lines on the feature map plane and then assign the aggregated features to corresponding locations in the parametric domain. Consequently, the problem of detecting semantic lines in the spatial domain is transformed into spotting individual points in the parametric domain, making the post-processing steps, i.e., non-maximal suppression, more efficient. Furthermore, our method makes it easy to extract contextual line features that are critical for accurate line detection. In addition to the proposed method, we design an evaluation metric to assess the quality of line detection and construct a large scale dataset for the line detection task. Experimental results on our proposed dataset and another public dataset demonstrate the advantages of our method over previous state-of-the-art alternatives. The dataset and source code is available at https://mmcheng.net/dhtline/. [ABSTRACT FROM AUTHOR]

Brain-computer interface (BCI) based on motor imagery (MI) electroencephalogram (EEG) decoding helps motor-disabled patients to communicate with external devices directly, which can achieve the purpose of human-computer interaction and assisted living. MI EEG decoding has a core problem which is extracting as many multiple types of features as possible from the multi-channel time series of EEG to understand brain activity accurately. Recently, deep learning technology has been widely used in EEG decoding. However, the variability of the simple network framework is insufficient to satisfy the complex task of EEG decoding. A multi-scale fusion convolutional neural network based on the attention mechanism (MS-AMF) is proposed in this paper. The network extracts spatio temporal multi-scale features from multi-brain regions representation signals and is supplemented by a dense fusion strategy to retain the maximum information flow. The attention mechanism we added to the network has improved the sensitivity of the network. The experimental results show that the network has a better classification effect compared with the baseline method in the BCI Competition IV-2a dataset. We conducted visualization analysis in multiple parts of the network, and the results show that the attention mechanism is also convenient for analyzing the underlying information flow of EEG decoding, which verifies the effectiveness of the MS-AMF method. [ABSTRACT FROM AUTHOR]

Recognition and localization models can be generally decomposed into three components: encoder, decoder, and task head. In this paper, we rethink the necessity of decoder, as we observe that it brings additional computational and parametric burden. We thus propose to remove the decoder and present a bidirectional cross-scale architecture that is able to obtain rich semantic information and precise localization in a unified backbone. Extensive experiments demonstrate that, different from common encoder-decoder models and other down-sampling and up-sampling backbones, the proposed BiCSNet achieves improved performances compared to existing architectures for pixel-level tasks. In object detection, our BiCSNet brings significant performance improvement by ~ 3% AP at various scales with 13% – 23% fewer FLOPS, compared with ResNet-FPN models on COCO dataset. In Instance segmentation, the AP can be improved by 1% over SpineNet. BiCSNet is also promising for semantic segmentation tasks, as the proposed BiCSNet pre-trained on ImageNet alone significantly outperforms DeepLabv3 pre-trained on both ImageNet and COCO dataset by 1.3% in mIOU with 89% fewer FLOPs on PASCAL VOC 2012. [ABSTRACT FROM AUTHOR]

Sleep is a vital process of our daily life as we roughly spend one-third of our lives asleep. In order to evaluate sleep quality and potential sleep disorders, sleep stage classification is a gold standard method. In this paper, we introduce a novel fully convolutional neural network architecture (SleepFCN) to classify sleep stages into five classes using single-channel electroencephalograms (EEGs). The framework of SleepFCN includes two major parts for feature extraction and temporal sequence encoding namely multi-scale feature extraction (MSFE) and residual dilated causal convolutions (ResDC), respectively. These are then followed by convolutional layers of 1-sized kernels instead of dense layers to build the fully convolutional neural network. Due to the imbalance in the distribution of sleep stages, we incorporate a weight corresponding to the number of samples of each class in our loss function. We evaluated the performance of SleepFCN using the Sleep-EDF and SHHS datasets. Our experimental results show that the proposed method outperforms state-of-the-art works in both classification correctness and learning speed. [ABSTRACT FROM AUTHOR]

Pain is an integrative phenomenon coupled with dynamic interactions between sensory and contextual processes in the brain, often associated with detectable neurophysiological changes. Recent advances in brain activity recording tools and machine learning technologies have intrigued research and development of neurocomputing techniques for objective and neurophysiology-based pain detection. This paper proposes a pain detection framework based on Electroencephalogram (EEG) and deep convolutional neural networks (CNN). The feasibility of CNN is investigated for distinguishing induced pain state from resting state in the recruitment of 10 chronic back pain patients. The experimental study recorded EEG signals in two phases: 1. movement stimulation (MS), where induces back pain by executing predefined movement tasks; 2. video stimulation (VS), where induces back pain perception by watching a set of video clips. A multi-layer CNN classifies the EEG segments during the resting state and the pain state. The novel approach offers high and robust performance and hence is significant in building a powerful pain detection algorithm. The area under the receiver operating characteristic curve (AUC) of our approach is 0.83 ± 0.09 and 0.81 ± 0.15, in MS and VS, respectively, higher than the state-of-the-art approaches. The sub-brain-areas are also analyzed, to examine distinct brain topographies relevant for pain detection. The results indicate that MS-induced pain tends to evoke a generalized brain area, while the evoked area is relatively partial under VS-induced pain. This work may provide a new solution for researchers and clinical practitioners on pain detection. [ABSTRACT FROM AUTHOR]

Quasi-periodic time series (QTS) exists widely in the real world, and it is important to detect the anomalies of QTS. In this paper, we propose an automatic QTS anomaly detection framework (AQADF) consisting of a two-level clustering-based QTS segmentation algorithm (TCQSA) and a hybrid attentional LSTM-CNN model (HALCM). TCQSA first automatically splits the QTS into quasi-periods which are then classified by HALCM into normal periods or anomalies. Notably, TCQSA integrates a hierarchical clustering and the k-means technique, making itself highly universal and noise-resistant. HALCM hybridizes LSTM and CNN to simultaneously extract the overall variation trends and local features of QTS for modeling its fluctuation pattern. Furthermore, we embed a trend attention gate (TAG) into the LSTM, a feature attention mechanism (FAM) and a location attention mechanism (LAM) into the CNN to finely tune the extracted variation trends and local features according to their true importance to achieve a better representation of the fluctuation pattern of the QTS. On four public datasets, HALCM exceeds four state-of-the-art baselines and obtains at least 97.3 percent accuracy, TCQSA outperforms two cutting-edge QTS segmentation algorithms and can be applied to different types of QTSs. Additionally, the effectiveness of the attention mechanisms is quantitatively and qualitatively demonstrated. [ABSTRACT FROM AUTHOR]

In both H.264/AVC and HEVC, the angular prediction is adopted for intra coding, which only exploits the spatial correlation between the current block and its neighboring single line reference. This angular prediction can handle the main directional patterns well, however, lacks the ability to deal with other directions. In this paper, a multi-scale convolutional neural network based intra prediction is proposed to address this problem. Specifically, a predicted block is first generated by the angular prediction, then fed into the proposed network with its neighboring reconstructed $L$ -shape to generate a more accurate predicted block. On one hand, the $L$ -shape of multiple lines provides more reliable reconstructed pixels and more contextual information to get better prediction; on the other hand, the multi-scale feature extraction takes the advantage of the feature maps in different scales to further enhance the prediction. With this multi-scale structure, the $L$ -shape can be used to refine both left-above and right-bottom pixels in the predicted block during the convolution operation. Experimental results demonstrate that compared with HEVC reference software HM 16.9, the proposed intra prediction can achieve an average of 3.4% (up to 5.6%) bitrate saving with all intra configuration. [ABSTRACT FROM AUTHOR]

In this paper, the traditional model based variational methods and deep learning based algorithms are naturally integrated to address mixed noise removal, specially for Gaussian mixture noise and Gaussian-impulse noise removal problem. To be different from single type noise (e.g. Gaussian) removal, it is a challenge problem to accurately discriminate noise types and levels for each pixel. We propose a variational method to iteratively estimate the noise parameters, and then the algorithm can automatically classify the noise according to the different statistical parameters. The proposed variational problem can be separated into regularization, synthesis, parameters estimation and noise classification four steps with the operator splitting scheme. Each step is related to an optimization subproblem. To enforce the regularization, the deep learning method is employed to learn the natural images prior. Compared with some model based regularizations, the CNN regularizer can significantly improve the quality of the restored images. Compared with some learning based methods, the synthesis step can produce better reconstructions by analyzing the types and levels of the recognized noise. In our method, the convolution neutral network (CNN) can be regarded as an operator which associated to a variational functional. From this viewpoint, the proposed method can be extended to many image reconstruction and inverse problems. Numerical experiments in the paper show that our method can achieve some state-of-the-art results for Gaussian mixture noise and Gaussian-impulse noise removal. [ABSTRACT FROM AUTHOR]

In recent years, the image and video coding technologies have advanced by leaps and bounds. However, due to the popularization of image and video acquisition devices, the growth rate of image and video data is far beyond the improvement of the compression ratio. In particular, it has been widely recognized that there are increasing challenges of pursuing further coding performance improvement within the traditional hybrid coding framework. Deep convolution neural network which makes the neural network resurge in recent years and has achieved great success in both artificial intelligent and signal processing fields, also provides a novel and promising solution for image and video compression. In this paper, we provide a systematic, comprehensive and up-to-date review of neural network-based image and video compression techniques. The evolution and development of neural network-based compression methodologies are introduced for images and video respectively. More specifically, the cutting-edge video coding techniques by leveraging deep learning and HEVC framework are presented and discussed, which promote the state-of-the-art video coding performance substantially. Moreover, the end-to-end image and video coding frameworks based on neural networks are also reviewed, revealing interesting explorations on next generation image and video coding frameworks/standards. The most significant research works on the image and video coding related topics using neural networks are highlighted, and future trends are also envisioned. In particular, the joint compression on semantic and visual information is tentatively explored to formulate high efficiency signal representation structure for both human vision and machine vision, which are the two dominant signal receptors in the age of artificial intelligence. [ABSTRACT FROM AUTHOR]

Volumetric imaging is an essential diagnostic tool for medical practitioners. The use of popular techniques such as convolutional neural networks (CNN) for analysis of volumetric images is constrained by the availability of detailed (with local annotations) training data and GPU memory. In this paper, the volumetric image classification problem is posed as a multi-instance classification problem and a novel method is proposed to adaptively select positive instances from positive bags during the training phase. This method uses the extreme value theory to model the feature distribution of the images without a pathology and use it to identify positive instances of an imaged pathology. The experimental results, on three separate image classification tasks (i.e. classify retinal OCT images according to the presence or absence of fluid build-ups, emphysema detection in pulmonary 3D-CT images and detection of cancerous regions in 2D histopathology images) show that the proposed method produces classifiers that have similar performance to fully supervised methods and achieves the state of the art performance in all examined test cases. [ABSTRACT FROM AUTHOR]

Real-time detection of inshore ships plays an essential role in the efficient monitoring and management of maritime traffic and transportation for port management. Current ship detection methods which are mainly based on remote sensing images or radar images hardly meet real-time requirement due to the timeliness of image acquisition. In this paper, we propose to use visual images captured by an on-land surveillance camera network to achieve real-time detection. However, due to the complex background of visual images and the diversity of ship categories, the existing convolution neural network (CNN) based methods are either inaccurate or slow. To achieve high detection accuracy and real-time performance simultaneously, we propose a saliency-aware CNN framework for ship detection, comprising comprehensive ship discriminative features, such as deep feature, saliency map, and coastline prior. This model uses CNN to predict the category and the position of ships and uses the global contrast based salient region detection to correct the location. We also extract coastline information and respectively incorporate it into CNN and saliency detection to obtain more accurate ship locations. We implement our model on Darknet under CUDA 8.0 and CUDNN V5 and use a real-world visual image dataset for training and evaluation. The experimental results show that our model outperforms representative counterparts (Faster R-CNN, SSD, and YOLOv2) in terms of accuracy and speed. [ABSTRACT FROM AUTHOR]

With the improvement of sensor techniques, and the urgent requirement of automatic fault diagnosis technologies, the intelligent perception system on high speed train is more popular than ever before. It records the devices’ state information through a sensor network, and services for further analysis. However, Traditional machine learning algorithms are usually constrained by massive multi-sensor data and knowledge-based feature extraction in fault diagnosis. Therefore, this paper extended fault diagnosis methodology into tensor space to deal with multi-sensor monitoring data and take full use of available information. Moreover, the convolutional neural network (CNN) is used for automatic feature learning and classification without human intervention. The effectiveness and efficiency are validated by dataset of rolling element bearings obtained in lab and real-use case. Three features can be highlighted. First of all, the proposed model showed a good adaptability and high efficiency under various working condition by taking full use of the multi-sensor data. It has powerful ability in accuracy and convergence speed. Secondly, it is not as sensitive to data quantity as other deep learning algorithms do. Such superior characteristic made the model more suitable for practical application, because of the insufficient failure data. At last, it is an intelligent End-to-End model, performing automatic fault diagnosis without manual intervention and suitable for real-use case. [ABSTRACT FROM AUTHOR]

Siamese networks are prevalent in person re-identification (re-id) tasks to address the similarity and dissimilarity among video frames. It mainly focuses on the inter-video variation between spatio-temporal features extracted from different videos, while the variation between features of the same video has been rarely discussed. In this paper, we introduce the concept of “mean-body” and define an intra-video loss to address the variation between spatio-temporal features of the same video. A novel loss is presented to boost the training of the re-id networks by combining the proposed intra-video loss and the Siamese loss. Specifically, the intra-video loss uses the unique mean-body of each camera viewpoint to make the video sequence more clustered, while the Siamese loss is to make the wrong matching videos more separated. To train the whole network, we update the network and the mean-body in an iterative manner. As a result, the proposed loss is expected to improve the generalization capability of the re-id networks on the testing set. Extensive results demonstrate that the presented approach outperforms the state-of-the-art algorithms on the publicly available data sets, such as PRID2011, iLIDS-VID, and MARS, in terms of re-id accuracy. [ABSTRACT FROM AUTHOR]

In this paper, we present novel sharp attention networks by adaptively sampling feature maps from convolutional neural networks for person re-identification (re-ID) problems. Due to the introduction of sampling-based attention models, the proposed approach can adaptively generate sharper attention-aware feature masks. This greatly differs from the gating-based attention mechanism that relies on soft gating functions to select the relevant features for person re-ID. In contrast, the proposed sampling-based attention mechanism allows us to effectively trim irrelevant features by enforcing the resultant feature masks to focus on the most discriminative features. It can produce sharper attentions that is more assertive in localizing subtle features relevant to re-identifying people across cameras. For this purpose, a differentiable Gumbel-Softmax sampler is employed to approximate the Bernoulli sampling to train the sharp attention networks. Extensive experimental evaluations demonstrate the superiority of this new sharp attention model for person re-ID over other related existing, published state-of-the-art works on three challenging benchmarks, including CUHK03, Market-1501, and DukeMTMC-reID. [ABSTRACT FROM AUTHOR]

Spectrum sensing for the Digital Television Terrestrial Multimedia Broadcasting (DTMB) system is considered in this paper. A novel method, which contains a preprocessing method of partition and averaging, a convolutional neural network (CNN) and a threshold chosen scheme, is proposed to solve this problem. In the spectrum sensing process, the received signal is firstly preprocessed by a scheme of partition and averaging. Then it is fed into the well-trained CNN. The posterior probability of the signal being noise is output. Based on the Neyman-Pearson theorem, a threshold can be chosen by the Monte Carlo method. The result of spectrum sensing can be obtained by comparing the posterior probability with the threshold and the complexity of the proposed method is also analyzed. Through computer simulations, the advantage of the proposed method is demonstrated. At a low signal-to-noise ratio (SNR), the proposed method can achieve a satisfactory detection probability. Besides, the proposed method is robust to different SNRs and has good generalization performance under different noise assumptions. When the proposed method is transferred to sense DTMB signal under time dispersive channels, the performance does not deteriorate and even can be better, which shows the robustness of the proposed method to the time dispersive channels. [ABSTRACT FROM AUTHOR]

Most object detectors contain two important components: a feature extractor and an object classifier. The feature extractor has rapidly evolved with significant research efforts leading to better deep convolutional architectures. The object classifier, however, has not received much attention and many recent systems (like SPPnet and Fast/Faster R-CNN) use simple multi-layer perceptrons. This paper demonstrates that carefully designing deep networks for object classification is just as important. We experiment with region-wise classifier networks that use shared, region-independent convolutional features. We call them “Networks on Convolutional feature maps” (NoCs). We discover that aside from deep feature maps, a deep and convolutional per-region classifier is of particular importance for object detection, whereas latest superior image classification models (such as ResNets and GoogLeNets) do not directly lead to good detection accuracy without using such a per-region classifier. We show by experiments that despite the effective ResNets and Faster R-CNN systems, the design of NoCs is an essential element for the 1st-place winning entries in ImageNet and MS COCO challenges 2015. [ABSTRACT FROM PUBLISHER]

Human actions in video sequences are characterized by the complex interplay between spatial features and their temporal dynamics. In this paper, we propose novel tensor representations for compactly capturing such higher-order relationships between visual features for the task of action recognition. We propose two tensor-based feature representations, viz. (i) sequence compatibility kernel (SCK) and (ii) dynamics compatibility kernel (DCK). SCK builds on the spatio-temporal correlations between features, whereas DCK explicitly models the action dynamics of a sequence. We also explore generalization of SCK, coined SCK $\;\oplus$ ⊕ , that operates on subsequences to capture the local-global interplay of correlations, which can incorporate multi-modal inputs e.g., skeleton 3D body-joints and per-frame classifier scores obtained from deep learning models trained on videos. We introduce linearization of these kernels that lead to compact and fast descriptors. We provide experiments on (i) 3D skeleton action sequences, (ii) fine-grained video sequences, and (iii) standard non-fine-grained videos. As our final representations are tensors that capture higher-order relationships of features, they relate to co-occurrences for robust fine-grained recognition (Lin, 2017), (Koniusz, 2018). We use higher-order tensors and so-called Eigenvalue Power Normalization (EPN) which have been long speculated to perform spectral detection of higher-order occurrences (Koniusz, 2013), (Koniusz, 2017), thus detecting fine-grained relationships of features rather than merely count features in action sequences. We prove that a tensor of order $r$ r , built from $Z_*$ Z * dimensional features, coupled with EPN indeed detects if at least one higher-order occurrence is ‘projected’ into one of its $\binom{Z_*}{r}$ Z * r subspaces of dim. $r$ r represented by the tensor, thus forming a Tensor Power Normalization metric endowed with $\binom{Z_*}{r}$ Z * r such ‘detectors’. [ABSTRACT FROM AUTHOR]

Power Normalizations (PN) are useful non-linear operators which tackle feature imbalances in classification problems. We study PNs in the deep learning setup via a novel PN layer pooling feature maps. Our layer combines the feature vectors and their respective spatial locations in the feature maps produced by the last convolutional layer of CNN into a positive definite matrix with second-order statistics to which PN operators are applied, forming so-called Second-order Pooling (SOP). As the main goal of this paper is to study Power Normalizations, we investigate the role and meaning of MaxExp and Gamma, two popular PN functions. To this end, we provide probabilistic interpretations of such element-wise operators and discover surrogates with well-behaved derivatives for end-to-end training. Furthermore, we look at the spectral applicability of MaxExp and Gamma by studying Spectral Power Normalizations (SPN). We show that SPN on the autocorrelation/covariance matrix and the Heat Diffusion Process (HDP) on a graph Laplacian matrix are closely related, thus sharing their properties. Such a finding leads us to the culmination of our work, a fast spectral MaxExp which is a variant of HDP for covariances/autocorrelation matrices. We evaluate our ideas on fine-grained recognition, scene recognition, and material classification, as well as in few-shot learning and graph classification. [ABSTRACT FROM AUTHOR]

The integrated satellite-terrestrial network (ISN) is an emerging framework that complements the ground 5G network with global network access. The satellite layer consists of Low Earth Orbit (LEO) and Geosynchronous (GEO) satellites for long-distance and high-throughput data transmission with low latency. However, traditional ground routing strategies are not appropriate in ISN, considering the dynamic motion and different capabilities of satellites. Moreover, multi-type tasks need to be allocated with optimal inter-satellite transmission links (ISL) to ensure both the load balance of ISN and the quality of experience (QoE) of subscribers. In this paper, we combine the Software Defined Networking (SDN), AI technique, and fuzzy logic to optimize multi-task routing in ISN. The GEO satellites and the ground computing center (GCC) are set as the joint control plane. The load information of ISN at different times is collected by GEO controllers and made into a multi-D matrix. The GCC collects the historical traffic data from GEO controllers for convolutional neural network (CNN) model training and updating. The GEO satellites utilize the trained CNN model to make path allocation decisions and issue flow tables to LEO satellites. Further, considering the CNN judgment may contradict the user QoE, we use fuzzy logic to evaluate task requirements to improve the CNN output for optimal ISN path allocation. The simulation results demonstrate that the fuzzy-CNN based multi-task routing (FCMR) method has better traffic control performance and path finding flexibility under different ISN conditions. [ABSTRACT FROM AUTHOR]

In this paper, we present a novel hardware-friendly super-resolution (SR) method based on a convolutional neural network (CNN) and its dedicated hardware (HW) on field programmable gate array (FPGA). Although CNN-based SR methods have shown very promising results for SR, their computational complexities are prohibitive for hardware implementation. To the best of our knowledge, we are the first to implement a real-time CNN-based SR HW that upscales 2K full high-definition video to 4K ultra high-definition (UHD) video at 60 frames per second (fps). In our dedicated CNN-based SR HW, low-resolution input frames are processed line-by-line, and the number of convolutional filter parameters is reduced significantly by incorporating depth-wise separable convolutions with a residual connection. Our CNN-based SR HW incorporates a cascade of 1D convolutions having large receptive fields along horizontal lines while keeping vertical receptive fields minimal, which allows us to save required line memory space in achieving comparable SR performance against full 2D convolution operations. For efficient HW implementation, we use a simple and effective quantization method with little peak signal-to-noise ratio (PSNR) degradation. Also, we propose a compression method to efficiently store intermediate feature map data to reduce the number of line memories used in HW. Our HW implementation on the FPGA generates 4K UHD frames of higher PSNR values at 60 fps and shows better visual quality, compared with conventional CNN-based SR methods that are trained and tested in software. [ABSTRACT FROM AUTHOR]

With advanced image journaling tools, one can easily alter the semantic meaning of an image by exploiting certain manipulation techniques such as copy clone, object splicing, and removal, which mislead the viewers. In contrast, the identification of these manipulations becomes a very challenging task as manipulated regions are not visually apparent. This paper proposes a high-confidence manipulation localization architecture that utilizes resampling features, long short-term memory (LSTM) cells, and an encoder–decoder network to segment out manipulated regions from non-manipulated ones. Resampling features are used to capture artifacts, such as JPEG quality loss, upsampling, downsampling, rotation, and shearing. The proposed network exploits larger receptive fields (spatial maps) and frequency-domain correlation to analyze the discriminative characteristics between the manipulated and non-manipulated regions by incorporating the encoder and LSTM network. Finally, the decoder network learns the mapping from low-resolution feature maps to pixel-wise predictions for image tamper localization. With the predicted mask provided by the final layer (softmax) of the proposed architecture, end-to-end training is performed to learn the network parameters through back-propagation using the ground-truth masks. Furthermore, a large image splicing dataset is introduced to guide the training process. The proposed method is capable of localizing image manipulations at the pixel level with high precision, which is demonstrated through rigorous experimentation on three diverse datasets. [ABSTRACT FROM AUTHOR]

The GPS trajectories are rich with potential information that could be used to explore the regulation of traffic to serve the public. While that past approaches for short-term traffic prediction have existed for some time, emerging smart transportation technologies require the traffic prediction capability to be both fast and scalable to full urban networks. In this paper, we propose a novel neural network, named L-CNN based on CNN and LSTM, and develop an effective real-time prediction model to forecast the most likely potential passenger for taxi drivers. It is noteworthy that our model can be easily extended to other real-time traffic prediction problems, such as road traffic and flow prediction. Finally, we test our method based on GPS trajectories generated by Cheng Du taxi. The method presented provides passenger prediction over 15-min intervals for up to 1 h in advance and the results prove the efficiency of our predicting system. [ABSTRACT FROM AUTHOR]

Uncertainty sampling-based active learning has been well studied for selecting informative samples to improve the performance of a classifier. In batch-mode active learning, a batch of samples are selected for a query at the same time. The samples with top uncertainty are encouraged to be selected. However, this selection strategy ignores the relations among the samples, because the selected samples may have much redundant information with each other. This paper addresses this problem by proposing a novel method that combines uncertainty, diversity, and density via sparse modeling in the sample selection. We use sparse linear combination to represent the uncertainty of unlabeled pool data with Gaussian kernels, in which the diversity and density are well incorporated. The selective sampling method is proposed before optimization to reduce the representation error. To deal with ${l}_{0}$ norm constraint in the sparse problem, two approximated approaches are adopted for efficient optimization. Four image classification data sets are used for evaluation. Extensive experiments related to batch size, feature space, seed size, significant analysis, data transform, and time efficiency demonstrate the advantages of the proposed method. [ABSTRACT FROM AUTHOR]

Leveraging the abundant number of web data is a promising strategy in addressing the problem of data lacking when training convolutional neural networks (CNNs). However, the web images often contain incorrect tags, which may compromise the learned CNN model. To address this problem, this paper focuses on image classification and proposes to iterate between filtering out noisy web labels and fine-tuning the CNN model using the crawled web images. Overall, the proposed method benefits from the growing modeling capability of the learned model to correct labels for web images and learning from such new data to produce a more effective model. Our contribution is two-fold. First, we propose an iterative method that progressively improves the discriminative ability of CNNs and the accuracy of web image selection. This method is beneficial toward selecting high-quality web training images and expanding the training set as the model gets ameliorated. Second, since web images are usually complex and may not be accurately described by a single tag, we propose to assign a web image multiple labels to reduce the impact of hard label assignment. This labeling strategy mines more training samples to improve the CNN model. In the experiments, we crawl 0.5 million web images covering all categories of four public image classification data sets. Compared with the baseline which has no web images for training, we show that the proposed method brings notable improvement. We also report the competitive recognition accuracy compared with the state of the art. [ABSTRACT FROM AUTHOR]

This paper presents an end-to-end learning architecture for video-based person re-identification by integrating convolutional neural networks (CNNs) and bidirectional recurrent neural networks (BRNNs). Given a video with consecutive frames, features of each frame are extracted with CNN and then are fed into the BRNN to get a final spatio-temporal representation about the video. Specifically, CNN acts as a Spatial Feature Extractor, while BRNN is expected to capture the temporal cues of sequential frames in both forward and backward directions, simultaneously. The whole network is trained end-to-end with a joint identification and verification manner. Experimental results on benchmark data sets show that the proposed model can effectively learn spatio-temporal features relevant for re-identification and outperforms existing video-based person re-identification methods. [ABSTRACT FROM AUTHOR]

The visual cues from multiple support regions of different sizes and resolutions are complementary in classifying a candidate box in object detection. Effective integration of local and contextual visual cues from these regions has become a fundamental problem in object detection. In this paper, we propose a gated bi-directional CNN (GBD-Net) to pass messages among features from different support regions during both feature learning and feature extraction. Such message passing can be implemented through convolution between neighboring support regions in two directions and can be conducted in various layers. Therefore, local and contextual visual patterns can validate the existence of each other by learning their nonlinear relationships and their close interactions are modeled in a more complex way. It is also shown that message passing is not always helpful but dependent on individual samples. Gated functions are therefore needed to control message transmission, whose on-or-offs are controlled by extra visual evidence from the input sample. The effectiveness of GBD-Net is shown through experiments on three object detection datasets, ImageNet, Pascal VOC2007 and Microsoft COCO. Besides the GBD-Net, this paper also shows the details of our approach in winning the ImageNet object detection challenge of 2016, with source code provided on https://github.com/craftGBD/craftGBD. In this winning system, the modified GBD-Net, new pretraining scheme and better region proposal designs are provided. We also show the effectiveness of different network structures and existing techniques for object detection, such as multi-scale testing, left-right flip, bounding box voting, NMS, and context. [ABSTRACT FROM AUTHOR]

In this paper, a novel perceptual image hashing scheme based on convolutional neural network (CNN) with multiple constraints is proposed, in which our deep hashing network learns the process of features extraction automatically according to the training target and then generates the final hash sequence. The combination of convolutional and pooling layers is to reduce the size of input image while deepening the channels. Then, we construct two pairs of constraints and integrate them into an overall constraint function through a strategy of weight allocation. In order to guarantee the robustness and discrimination of deep hashing network simultaneously, a new training method is developed to adjust the training set structure dynamically according to the changes of constraint values. Experimental results show that the proposed deep hashing network can achieve a satisfactory balance between perceptual robustnzess and discrimination while maintaining security. Based on the large-scale test set, receiver operating characteristic (ROC) curves, ${F}_{1}$ scores and equal error rate (EER) demonstrate the superiority of our scheme in terms of content authentication compared with some state-of-the-art schemes. [ABSTRACT FROM AUTHOR]

In this paper, we tackle the problem of single image dehazing with a convolutional neural network. Within this network, we develop a multi-level fusion module to utilize both low-level and high-level features. The low-level features help to recover finer details, and the high-level features discover abstract semantics. They are complementary in the restoring of clear images. Moreover, a Residual Mixed-convolution Attention Module (RMAM) with an attention block is proposed to guide the network to focus on important features in the learning process. In this RMAM, group convolution, depth-wise convolution, and point-wise convolution are mixed, and thus it is much faster than its counterparts. With these two modules, we thus have an end-to-end network without explicitly estimating the atmospheric light intensity and the transmission map in the classical atmosphere scattering model. Both qualitative and quantitative experimental studies are carried out on public datasets including RESIDE, DCPDN-TestA, and the real-world dataset. The extensive results demonstrate both the effectiveness and efficiency of the proposed solution to single image dehazing. [ABSTRACT FROM AUTHOR]

Recently, convolutional neural network (CNN)-based channel estimation (CE) for massive multiple-input multiple-output communication systems has achieved remarkable success. However, complexity even needs to be reduced, and robustness can even be improved. Meanwhile, existing methods do not accurately explain which channel features help the denoising of CNNs. In this paper, we first compare the strengths and weaknesses of CNN-based CE in different domains. When complexity is limited, the channel sparsity in the angle-delay domain improves denoising and robustness whereas large noise power and pilot contamination are handled well in the spatial-frequency domain. Thus, we develop a novel network, called dual CNN, to exploit the advantages in the two domains. Furthermore, we introduce an extra neural network, called HyperNet, which learns to detect scenario changes from the same input as the dual CNN. HyperNet updates several parameters adaptively and combines the existing dual CNNs to improve robustness. Experimental results show improved estimation performance for the time-varying scenarios. To further exploit the correlation in the time domain, a recurrent neural network framework is developed, and training strategies are provided to ensure robustness to the changing of temporal correlation. This design improves channel estimation performance but its complexity is still low. [ABSTRACT FROM AUTHOR]