Search

Estimating depth from RGB images is a long-standing ill-posed problem, which has been explored for decades by the computer vision, graphics, and machine learning communities. Among the existing techniques, stereo matching remains one of the most widely used in the literature due to its strong connection to the human binocular system. Traditionally, stereo-based depth estimation has been addressed through matching hand-crafted features across multiple images. Despite the extensive amount of research, these traditional techniques still suffer in the presence of highly textured areas, large uniform regions, and occlusions. Motivated by their growing success in solving various 2D and 3D vision problems, deep learning for stereo-based depth estimation has attracted a growing interest from the community, with more than 150 papers published in this area between 2014 and 2019. This new generation of methods has demonstrated a significant leap in performance, enabling applications such as autonomous driving and augmented reality. In this paper, we provide a comprehensive survey of this new and continuously growing field of research, summarize the most commonly used pipelines, and discuss their benefits and limitations. In retrospect of what has been achieved so far, we also conjecture what the future may hold for deep learning-based stereo for depth estimation research. [ABSTRACT FROM AUTHOR]

Radio signal recognition has a wide application in future communication systems and the vehicular communication, whose core is the extraction of signal features such as electromagnetic fingerprints. With the rapid development of artificial intelligence technology, deep learning has made amazing breakthroughs in image recognition, speech recognition and other fields. Deep learning is applied to electromagnetic fingerprint extraction in this paper. Firstly, thousands of the downlink aircraft communications addressing and reporting system (ACARS) signals used for communication between civil aircraft and airport tower are collected and generated. Then a pre-transformation network suitable for electromagnetic signals is constructed to convert one-dimensional signals into two-dimensional feature maps, and afterwards the feature maps are input into the convolutional neural network (CNN) for classification. By adopting the attention modules, the classification results were improved by a few percentage points over the baseline with a little cost. The method proposed in this paper achieves an accuracy rate of 94.1% and can obtain the aircraft type in a shorter time than traditional method. Moreover, the robustness of the proposed model in response to additive Gaussian white noise (AWGN) and phase deviation is studied and tested. [ABSTRACT FROM AUTHOR]

In this paper, simple methodologies of deep learning application to conventional multiple-input multiple-output (MIMO) communication systems are presented. The deep learning technologies with deep neural network (DNN) structure, emerging technologies in various engineering areas, have been actively investigated in the field of communication engineering as well. In the physical layer of conventional communication systems, there are practical challenges of application of DNN: calculating complex number in DNN and designing proper DNN structure for a specific communication system model. This paper proposes and verifies simple solutions for the difficulty. First, we apply a basic DNN structure for signal detection of one-tap MIMO channel. Second, convolutional neural network (CNN) and recurrent neural network (RNN) structures are presented for MIMO system with multipath fading channel. Our DNN structure for one-tap MIMO channel can achieve the optimal maximum likelihood detection performance, and furthermore, our CNN and RNN structures for multipath fading channel can detect the transmitted signal properly. [ABSTRACT FROM AUTHOR]

In order to realize speaker recognition in flammable, explosive and strong electromagnetic interference environments, the speaker recognition technology of fiber-optic external Fabry-Perot interferometric (EFPI) microphone based on Deep learning is studied in this paper. The combination of convolutional neural network and long short-term memory (CNN-LSTM) model is proposed for speaker recognition in EFPI acoustic sensing system. The intensity self-compensation method is used to demodulate the voice signals of different speakers, then, the spectral features of voice signals are divided into training set and test set. The training set are input into the established CNN-LSTM model for model training, and the test set are input into the trained model for testing. The experimental results show that the CNN-LSTM model proposed in this paper has good training accuracy and test accuracy, reaching 100% and 94.0% respectively. Comparing with CNN and LSTM recurrent neural network (LSTM-RNN), the test accuracy of this model is 11.1% and 13.4% higher respectively. [ABSTRACT FROM AUTHOR]

Convolutional neural networks (CNN) have demonstrated their ability to segment 2D cardiac ultrasound images. However, despite recent successes according to which the intra-observer variability on end-diastole and end-systole images has been reached, CNNs still struggle to leverage temporal information to provide accurate and temporally consistent segmentation maps across the whole cycle. Such consistency is required to accurately describe the cardiac function, a necessary step in diagnosing many cardiovascular diseases. In this paper, we propose a framework to learn the 2D+time apical long-axis cardiac shape such that the segmented sequences can benefit from temporal and anatomical consistency constraints. Our method is a post-processing that takes as input segmented echocardiographic sequences produced by any state-of-the-art method and processes it in two steps to (i) identify spatio-temporal inconsistencies according to the overall dynamics of the cardiac sequence and (ii) correct the inconsistencies. The identification and correction of cardiac inconsistencies relies on a constrained autoencoder trained to learn a physiologically interpretable embedding of cardiac shapes, where we can both detect and fix anomalies. We tested our framework on 98 full-cycle sequences from the CAMUS dataset, which are available alongside this paper. Our temporal regularization method not only improves the accuracy of the segmentation across the whole sequences, but also enforces temporal and anatomical consistency. [ABSTRACT FROM AUTHOR]

Due to the strong penetration ability and high transmission rate of ultra-wideband (UWB) signals, UWB technology plays a very significant role in the field of precise indoor positioning. However, the harsh and volatile indoor environment leads to non-line-of-sight (NLOS) propagation and severe attenuation of UWB signals, which may generate significant ranging and positioning errors. To mitigate NLOS effect and improve positioning accuracy, existing methods use ranging information and channel impulse response (CIR) to identify UWB signal propagation channels. However, these NLOS identification methods often require a priori knowledge and suitable thresholds, and most of them only perform a binary classification between LOS and NLOS in a particular scenario. To address these disadvantages, this paper proposes a novel multi-classification method to classify UWB signal propagation channels based on one-dimensional wavelet packet analysis (ODWPA) and convolutional neural network (CNN). This method first decomposes the CIR of known categories into colored coefficients images using a one-dimensional wavelet packet function, then uses these images to train CNNs with different structures, and finally selects CNN model with the best performance to identify the unknown category of UWB signal propagation channels. The experimental results show that the performance of the proposed method is always the best, regardless of the type of experimental scenario, and the identification accuracy is always higher than 90% and up to 100%. [ABSTRACT FROM AUTHOR]

For 360° video, the existing visual quality assessment (VQA) approaches are designed based on either the whole frames or the cropped patches, ignoring the fact that subjects can only access viewports. When watching 360° video, subjects select viewports through head movement (HM) and then fixate on attractive regions within the viewports through eye movement (EM). Therefore, this paper proposes a two-staged multi-task approach for viewport-based VQA on 360° video. Specifically, we first establish a large-scale VQA dataset of 360° video, called VQA-ODV, which collects the subjective quality scores and the HM and EM data on 600 video sequences. By mining our dataset, we find that the subjective quality of 360° video is related to camera motion, viewport positions and saliency within viewports. Accordingly, we propose a viewport-based convolutional neural network (V-CNN) approach for VQA on 360° video, which has a novel multi-task architecture composed of a viewport proposal network (VP-net) and viewport quality network (VQ-net). The VP-net handles the auxiliary tasks of camera motion detection and viewport proposal, while the VQ-net accomplishes the auxiliary task of viewport saliency prediction and the main task of VQA. The experiments validate that our V-CNN approach significantly advances state-of-the-art VQA performance on 360° video and it is also effective in the three auxiliary tasks. [ABSTRACT FROM AUTHOR]

As the extension of high efficiency video coding (HEVC) standard, three dimensional-HEVC (3D-HEVC) is the latest 3D video coding standard. 3D-HEVC adopts many complicated coding algorithms to generate additional intermediate views for 3D video representation, which result in extremely high coding complexity. Therefore, this paper proposed a fast depth intra coding approach to reduce the 3D-HEVC complexity, which is based on convolutional neural network (CNN). First, we established a database based on the independent view of the depth map, which includes coding unit (CU) partition data of the depth map. Second, we constructed a depth edge classification CNN (DEC-CNN) framework to classify the edges for the depth map and embedded the network into a 3D-HEVC test platform. Finally, we utilized the pixel value of the binarized depth image to correct the above classification results. The experimental results demonstrated that our approach can reduce the intra coding time by 72.5% on average under negligible degradation of coding performance. This result outperforms the other state-of-the-art methods to reduce the coding complexity of 3D-HEVC. [ABSTRACT FROM AUTHOR]

The attractiveness of a property is one of the most interesting, yet challenging, categories to model. Image characteristics are used to describe certain attributes, and to examine the influence of visual factors on the price or timeframe of the listing. In this paper, we propose a set of techniques for the extraction of visual features for efficient numerical inclusion in modern-day predictive algorithms. We discuss techniques such as Shannon's entropy, calculating the center of gravity, employing image segmentation, and using Convolutional Neural Networks. After comparing these techniques as applied to a set of property-related images (indoor, outdoor, and satellite), we conclude the following: (i) the entropy is the most efficient single-digit visual measure for housing price prediction; (ii) image segmentation is the most important visual feature for the prediction of housing lifespan; and (iii) deep image features can be used to quantify interior characteristics and contribute to captivation modeling. The set of 40 image features selected here carries a significant amount of predictive power and outperforms some of the strongest metadata predictors. Without any need to replace a human expert in a real-estate appraisal process, we conclude that the techniques presented in this paper can efficiently describe visible characteristics, thus introducing perceived attractiveness as a quantitative measure into the predictive modeling of housing. [ABSTRACT FROM AUTHOR]

Vehicle type recognition (VTR) is a quite common requirement and one of the key challenges in real surveillance scenarios, such as intelligent traffic and unmanned driving. Usually coarse-grained and fine-grained VTRs are applied in different applications, and the challenge from multiple viewpoints is critical for both cases. In this paper, we propose a feedback-enhancement multi-branch CNN (FM-CNN) to solve the challenge in these two cases. The proposed FM-CNN takes three derivatives of an image as input and leverages the advantages of hierarchical details, feedback enhancement, model average, and stronger robustness to translation and mirroring. A single global cross-entropy loss is insufficient to train such a complex CNN and so we add extra branch losses to enhance feedbacks to each branch. Though reusing pre-trained parameters, we propose a novel parameter update method to adapt FM-CNN to task-specific local visual patterns and global information in new datasets. To test the effectiveness of FM-CNN, we create our own multi-view VTR (MVVTR) data set since there are no such data sets available. And, for fine-grained VTR, we use the CompCars data set. Compared with state-of-the-art classification solutions without special preprocessing, the proposed FM-CNN demonstrates better performance in both coarse-grained and fine-grained scenarios. For coarse-grained VTR, it achieves 94.9% Top-1 accuracy on the MVVTR data set. For fine-grained VTR, it achieves 91.0% Top-1 and 97.8% Top-5 accuracies on the CompCars data set. [ABSTRACT FROM AUTHOR]

Wireless Sensor Networks (WSN) are generally used for precision agriculture. However, reliability and cost are the key limitations of such approaches. In recent times, the application of Unmanned Aerial Vehicles (UAVs) in the agricultural field has become popular due to scalability, cost efficient and user friendly adaptations with the help of improved navigation algorithms. A Novel and cost effective Light Emitting Diode (LED) based wireless communication of field sensor data to server has been recently explored. This paper proposes a LEDNet framework that utilizes the LED pattern based sensor data encoding in an image with computer vision and deep learning based data extraction/communication of data. The LED pattern image can be captured using any decent resolution cameras that can be mounted on UAVs. The proposed framework generates LED sequences with the help of embedded boards and utilizes image processing and deep learning for the decoding of the sensor data from the LED pattern image. The experiments are carried out with the images taken under various lighting conditions from different heights in the field. Promising performance in terms of accuracy and power consumption is observed for the collection of sensor data using the proposed LEDNet framework for the LED bit sequence extraction from the dataset of images collected under various environmental conditions. [ABSTRACT FROM AUTHOR]

The key role of spectrum sensing in cognitive radios attracted substantial research attention to improve the performance of detectors. We consider a general model for the receiver noise with the potential of generalization towards modeling noise in various environments. This general noise model describes accurately noise characteristics ranging from the Gaussian noise to the severe impulsive noise. However, many previous studies are based on the ideal Gaussian noise, and they cannot capture the non-Gaussian models. To provide a robust detector against different behaviors of the noise in various environments, we employ a convolutional neural network (CNN) compatible with different noise models. The proposed CNN detector is data-driven, and due to its single-dimensional input layer, it is consistent with the received signal and requires no pre-processing. The likelihood ratio test (LRT), the Wald, and the Rao tests for this problem are derived to enrich the paper with comparative evaluations of the proposed CNN and conventional model-based approaches and other neural networks. Although various simulated scenarios substantiate the general superiority and robustness of the CNN-based against impulsive noise and mismatch of parameters, it requires higher computational complexity than other discussed detectors. [ABSTRACT FROM AUTHOR]

Predicting the price of soybean meal futures has always been a hot topic in the financial market. It is influenced by many factors and has complex nonlinear characteristics, which makes it difficult to predict accurately. This paper proposes a prediction model for soybean meal futures price based on CNN-ILSTM-SA, which consists of CNN (Convolutional Neural Network), ILSTM (Improved Long Short-Term Memory), and SA (Self-Attention mechanism). CNN can well perform feature extraction on the input time series data. The ILSTM model presented in this paper removes the output gate of LSTM (Long Short-Term Memory) and improves the forget gate and input gate, thus reducing ILSTM’s complexity, shortening training time, and improving learning ability. SA can get the correlation between different input terms and predicted values, and assign different weight coefficients according to the correlation level to improve prediction accuracy. To verify the CNN-ILSTM-SA’s effectiveness, we compare it with eighteen baseline models, and the experimental results show that it is optimal. The MAE (Mean Absolute Error) of CNN-ILSTM-SA is 39.0441, the smallest of all models. As the data of the soybean meal futures price fluctuates in the range of 3500-4500 in experiments, the proportion of the MAE value of 39.0441 is 0.87%-1.15%, which is at an extremely low level; $\mathrm {R}^{2}$ (R-square) is 0.97487, which is closest to 1; The training time is 157.64s.

Ultra-wide Band (UWB) technology has emerged as a pivotal tool for human motion detection, finding applications in diverse areas ranging from smart homes to automotive safety. This paper presents a comprehensive survey of methodologies employed in UWB-based motion detection, elucidating their strengths, challenges, and performance metrics. While several methods, including Convolutional Neural Network (CNN) approaches, have been explored, challenges such as motion state overlaps, the necessity for enhanced spatial resolution, and background noise interference persist. Among the various methods analyzed, the SGWO-based RMDL technique emerges as a frontrunner, offering superior accuracy, reduced mean squared error, and impressive true negative and positive rates. Moreover, its computational efficiency sets a precedent in human motion detection. This paper provides insights into the state-of-the-art Through the wall imaging and human vital signs observation for future research and realtime applications.

Radar echo image sequence prediction is a spatial-temporal forecasting problem, which is one of the main challenges in precipitation nowcasting. In response to issues such as poor extraction of spatiotemporal features and blurred prediction of image sequence in previous models, this paper proposes a spatial-temporal dual discriminator precipitation nowcasting method using SNGAN with InceptionGRU (STD-SNGAN-InceptionGRU). The design of temporal and spatial dual discriminators constrains the generator’s prediction samples, effectively enhancing the spatiotemporal prediction capability of the generator through adversarial training. The generator utilizes multiscale convolution (Inception) and multiscale gate recurrent unit (InceptionGRU) to enhance its ability to extract spatiotemporal features, thereby improving its learning capability for the evolution of radar echo sequences. The comparative experimental results demonstrate that the proposed STD-SNGAN-InceptionGRU model outperforms other algorithms in terms of critical success index (CSI) and probability of detection (POD) at different echo intensities, especially in areas with long time series and high echo intensity. Furthermore, through ablation experiments, this paper proves that the temporal discriminator, spatial discriminator, and InceptionGRU all effectively improve the model’s prediction capability for radar echo images.

This study utilizes a new approach for short-term stock correlation forecasting using a combination of convolutional neural networks (CNN), bi-directional long and short-term memory (BiLSTM), and attention mechanisms to address the issue of information loss due to excessively long input time series data. The objective is to improve the accuracy of short-term equity correlation forecasts, which is essential for the efficient management of financial asset portfolios. The existing time series models usually only take into account the effect of stock historical data characteristics on stock correlation but ignore the fact that stock correlation is affected by multiple factors and undergoes an unknown unstable trend. More efficient methods or algorithms need to be devised to deal with financial data. This paper explores stock metrics and introduces a multi-factor model to quantify the factors affecting stock returns in order to calculate the correlation between stocks. Meanwhile, to cope with the problem of incomplete input data, this paper simultaneously incorporates stock return data that is directly related to the correlation data. In the CNN-BiLSTM-Attention (CLATT) model, the role of the CNN module is to obtain high-dimensional features from the input time series data, and the role of the BiLSTM module is to process the time series information to capture the time-series features in the stock return data and correlation data in the sample. In order to acquire the inherent correlation between the input stock return data and correlation data and to enhance the prediction accuracy of the model, the attention mechanism is finally implemented and used to optimize the weights of the BiLSTM output. In this paper, mean square error (MSE), root mean square error (RMSE), mean absolute error (MAE) metrics, mean absolute percentage of error (MAPE), and the coefficient of determination $R^{2}$ , as well as four benchmark models, are used to evaluate the performance of the proposed model. The result shows that the CLATT model has 57.32% and 33% improvement in the correlation prediction accuracy of different stock portfolios compared to the single LSTM model, and the model prediction accuracy has been improved. Compared with the benchmark CNN-BiLSTM, it also has 56.06% and 28.87% improvement.

As an essential metal, copper has the advantages of electrical conductivity and ductility, which is widely used in power transmission, electronics manufacturing and machining. The fluctuation of copper price has a great impact on the industry, especially on the development of the national economy, so predicting copper price has a great significance for economic development. However, traditional time series prediction models’ prediction accuracy is low. Therefore, this paper proposes a Shanghai copper price forecasting model based on Complete Ensemble Empirical Mode Decomposition with Adaptive Noise (CEEMDAN), Convolutional Neural Network (CNN), Self Attention Gated Unit (SAGU), named CEEMDAN-CNN-SAGU. CEEMDAN decomposes and reconstructs the Shanghai copper price data. In this paper, the zero-cross rate of the Intrinsic Mode Function (IMF) components is calculated, the IMF component with large noise is removed, and the remaining IMFs and Residual term (Res) are reorganized to obtain the high-frequency and low-frequency components. CNN performs convolution operations on the reconstructed components to extract time series features. SAGU is a new time series data prediction model proposed in this paper. SAGU includes two gated units (a forgetting gate and an input gate), and two data processing modules (Self Attention (SA) and Transition (Tra)). The SA module is responsible for processing the input data, redistributing the weight coefficients of different data, and improving the model’s attention to important information. The Tra module improves the low limit of the forgetting gate’s output so that the gated unit can keep the cell state from the previous moment to the current moment as much as possible. This paper uses Shanghai copper trading data from Shanghai Futures Exchange as experimental data. The comparison experiment with the other eleven prediction models shows that the CEEMDAN-CNN-SAGU model outperforms other models in all evaluation indexes.

Video forgery detection is a critical aspect of digital forensics, addressing the challenges posed by the manipulation of video content. This paper presents a novel approach for video forgery detection using Deep Convolutional Neural Networks (DCNN). Leveraging the power of deep learning, our method aims to improve the accuracy and efficiency of object-based forgery detection in advanced video sequences. In the proposed approach, we build upon the foundation of an existing method, which utilizes Convolutional Neural Networks, and introduce innovative modifications to the DCNN architecture. These modifications include data preprocessing, network architecture, and training strategies that enhance the model’s ability to detect tampered objects in video frames. We conduct experiments on the SYSU-OBJFORG dataset, the largest object-based forged video dataset to date, with advanced video encoding standards. Our DCNN-based approach is compared with the existing method, demonstrating superior performance. The results show increased accuracy and robustness in detecting object-based video forgery. This paper not only contributes to the field of video forgery detection but also underscores the potential of deep learning, particularly DCNN, in addressing the evolving challenges of digital video manipulation. The findings open avenues for future research in the localization of forged regions and the application of DCNN in lower bitrate or lower resolution video sequences.

A crucial component within any structural health monitoring system is a pipeline leak detection mechanism, vital for preventing avoidable water loss. Contemporary literature employs machine learning and deep learning for detecting pipeline leaks and cross-correlation for leak localization. The major drawbacks in the existing methodologies are that machine learning and deep learning methods need two different architectures for leak detection and localization, and the cross-correlation needs two sensors with a denoising technique. The primary objective of this paper is to deploy a unified architecture capable of executing both the detection and localization of a leak without any denoising technique and with a single sensor. The proposed technique utilizes the data collected using an Acousto-optic sensor with two different pressures. This paper proposes a novel custom one-dimensional time-series DenseNet for leak detection and localization. The proposed method gives better accuracies compared with the existing one-dimensional DenseNet-121, three different one-dimensional convolutional neural networks (1DCNN), and cross-correlation for two different pressure datasets. The proposed method’s processing time is thirteen times less than the existing one-dimensional DenseNet-121, with the observed average leak detection and localization accuracy of 99.08%. The results state that the proposed novel custom one-dimensional time-series DenseNet accurately detects and localizes the leak with less time.

This paper presents a novel approach for video-based person re-identification using multiple convolutional neural networks (CNNs). Unlike the previous work, we intend to extract a compact yet discriminative appearance representation from several frames rather than the whole sequence. Specifically, given a video, the representative frames are selected based on the walking profile of consecutive frames. A multiple CNN architecture incorporated with feature pooling is proposed to learn and compile the features of the selected representative frames into a compact description about the pedestrian for identification. Experiments are conducted on benchmark data sets to demonstrate the superiority of the proposed method over existing person re-identification approaches. [ABSTRACT FROM AUTHOR]

Low level image restoration is an integral component of modern artificial intelligence (AI) driven camera pipelines. Most of these frameworks are based on deep neural networks which present a massive computational overhead on resource constrained platform like a mobile phone. In this paper, we propose several lightweight low-level modules which can be used to create a computationally low cost variant of a given baseline model. Recent works for efficient neural networks design have mainly focused on classification. However, low-level image processing falls under the ‘image-to-image’ translation genre which requires some additional computational modules not present in classification. This paper seeks to bridge this gap by designing generic efficient modules which can replace essential components used in contemporary deep learning based image restoration networks. We also present and analyse our results highlighting the drawbacks of applying depthwise separable convolutional kernel (a popular method for efficient classification network) for sub-pixel convolution based upsampling (a popular upsampling strategy for low-level vision applications). This shows that concepts from domain of classification cannot always be seamlessly integrated into ‘image-to-image’ translation tasks. We extensively validate our findings on three popular tasks of image inpainting, denoising and super-resolution. Our results show that proposed networks consistently output visually similar reconstructions compared to full capacity baselines with significant reduction of parameters, memory footprint and execution speeds on contemporary mobile devices. [ABSTRACT FROM AUTHOR]

Multiview Facial Expression Recognition (MFER) is a well-known interdisciplinary problem among computer science and related disciplines with promising and valuable applications. Recognizing the facial expression in pose variations, which is very common in real-world conditions, makes it very challenging. This paper aims to provide a comprehensive survey of the MFER progress, includingboth categories of traditional and deep approaches. In general, we sort each of these categories into three overall groups to meet the pose variations: Pose-Robust Features, Pose Normalization, and Pose-Specific Classification. While reviewing the traditional methods, a thorough study is proposed on the existing novel deep techniques. We also introduce the state-of-the-art and discuss the challenges, limitations, opportunities, and future trends that need to be addressed in this field. Moreover, we provide an extensive review of publicly available datasets for MFER, including the labs’ collections and the sets gathered from in the wild. Besides, we introduce the most popular protocols on each dataset to standardize comparisons in the future. [ABSTRACT FROM AUTHOR]

We focus on a fundamental task of detecting meaningful line structures, a.k.a., semantic line, in natural scenes. Many previous methods regard this problem as a special case of object detection and adjust existing object detectors for semantic line detection. However, these methods neglect the inherent characteristics of lines, leading to sub-optimal performance. Lines enjoy much simpler geometric property than complex objects and thus can be compactly parameterized by a few arguments. To better exploit the property of lines, in this paper, we incorporate the classical Hough transform technique into deeply learned representations and propose a one-shot end-to-end learning framework for line detection. By parameterizing lines with slopes and biases, we perform Hough transform to translate deep representations into the parametric domain, in which we perform line detection. Specifically, we aggregate features along candidate lines on the feature map plane and then assign the aggregated features to corresponding locations in the parametric domain. Consequently, the problem of detecting semantic lines in the spatial domain is transformed into spotting individual points in the parametric domain, making the post-processing steps, i.e., non-maximal suppression, more efficient. Furthermore, our method makes it easy to extract contextual line features that are critical for accurate line detection. In addition to the proposed method, we design an evaluation metric to assess the quality of line detection and construct a large scale dataset for the line detection task. Experimental results on our proposed dataset and another public dataset demonstrate the advantages of our method over previous state-of-the-art alternatives. The dataset and source code is available at https://mmcheng.net/dhtline/. [ABSTRACT FROM AUTHOR]

Brain-computer interface (BCI) based on motor imagery (MI) electroencephalogram (EEG) decoding helps motor-disabled patients to communicate with external devices directly, which can achieve the purpose of human-computer interaction and assisted living. MI EEG decoding has a core problem which is extracting as many multiple types of features as possible from the multi-channel time series of EEG to understand brain activity accurately. Recently, deep learning technology has been widely used in EEG decoding. However, the variability of the simple network framework is insufficient to satisfy the complex task of EEG decoding. A multi-scale fusion convolutional neural network based on the attention mechanism (MS-AMF) is proposed in this paper. The network extracts spatio temporal multi-scale features from multi-brain regions representation signals and is supplemented by a dense fusion strategy to retain the maximum information flow. The attention mechanism we added to the network has improved the sensitivity of the network. The experimental results show that the network has a better classification effect compared with the baseline method in the BCI Competition IV-2a dataset. We conducted visualization analysis in multiple parts of the network, and the results show that the attention mechanism is also convenient for analyzing the underlying information flow of EEG decoding, which verifies the effectiveness of the MS-AMF method. [ABSTRACT FROM AUTHOR]

Recent advances in deep learning (DL) allow for solving complex AI problems that used to be considered very hard. While this progress has advanced many fields, it is considered to be bad news for Completely Automated Public Turing tests to tell Computers and Humans Apart (CAPTCHAs), the security of which rests on the hardness of some learning problems. In this paper, we introduce DeepCAPTCHA, a new and secure CAPTCHA scheme based on adversarial examples, an inherit limitation of the current DL networks. These adversarial examples are constructed inputs, either synthesized from scratch or computed by adding a small and specific perturbation called adversarial noise to correctly classified items, causing the targeted DL network to misclassify them. We show that plain adversarial noise is insufficient to achieve secure CAPTCHA schemes, which leads us to introduce immutable adversarial noise—an adversarial noise that is resistant to removal attempts. In this paper, we implement a proof of concept system, and its analysis shows that the scheme offers high security and good usability compared with the best previously existing CAPTCHAs. [ABSTRACT FROM PUBLISHER]

Recognition and localization models can be generally decomposed into three components: encoder, decoder, and task head. In this paper, we rethink the necessity of decoder, as we observe that it brings additional computational and parametric burden. We thus propose to remove the decoder and present a bidirectional cross-scale architecture that is able to obtain rich semantic information and precise localization in a unified backbone. Extensive experiments demonstrate that, different from common encoder-decoder models and other down-sampling and up-sampling backbones, the proposed BiCSNet achieves improved performances compared to existing architectures for pixel-level tasks. In object detection, our BiCSNet brings significant performance improvement by ~ 3% AP at various scales with 13% – 23% fewer FLOPS, compared with ResNet-FPN models on COCO dataset. In Instance segmentation, the AP can be improved by 1% over SpineNet. BiCSNet is also promising for semantic segmentation tasks, as the proposed BiCSNet pre-trained on ImageNet alone significantly outperforms DeepLabv3 pre-trained on both ImageNet and COCO dataset by 1.3% in mIOU with 89% fewer FLOPs on PASCAL VOC 2012. [ABSTRACT FROM AUTHOR]

Deep learning based bridge damage diagnosis methods can successfully use labeled data to detect bridge damage. These successful applications usually need that the training samples (source domain) and test samples (target domain) obey the same probability distribution. However, it is difficult to acquire a large amount of labeled data with damage information from actual bridges. It is also difficult to apply a model trained with bridge A to diagnose bridge B because of the distribution discrepancy of data from different bridges or environments. Therefore, transferring a well-trained damage diagnosis model to another bridge with unlabeled data remains a major challenge. Motivated by transfer learning, this paper proposes a new intelligent damage diagnosis method for bridges, namely, sub-domain adaptive deep transfer learning network (SADTLN), to solve the feature generalization problem in different bridges. In our method, a multi-kernel local maximum mean discrepancy (MK-LMMD) based sub-domain adaptation module, including a domain classifier for aligning the global distribution and a sub-domain multi-layer adaptation for aligning local distribution, is proposed for transfer learning, so that the learned features are domain-invariant. Experiments prove the effectiveness and advancement of the proposed method. This exploration will promote the practical application of intelligent bridge structural damage diagnosis. [ABSTRACT FROM AUTHOR]

During the past two decades, epileptic seizure detection and prediction algorithms have evolved rapidly. However, despite significant performance improvements, their hardware implementation using conventional technologies, such as Complementary Metal–Oxide–Semiconductor (CMOS), in power and area-constrained settings remains a challenging task; especially when many recording channels are used. In this paper, we propose a novel low-latency parallel Convolutional Neural Network (CNN) architecture that has between 2-2,800x fewer network parameters compared to State-Of-The-Art (SOTA) CNN architectures and achieves 5-fold cross validation accuracy of 99.84% for epileptic seizure detection, and 99.01% and 97.54% for epileptic seizure prediction, when evaluated using the University of Bonn Electroencephalogram (EEG), CHB-MIT and SWEC-ETHZ seizure datasets, respectively. We subsequently implement our network onto analog crossbar arrays comprising Resistive Random-Access Memory (RRAM) devices, and provide a comprehensive benchmark by simulating, laying out, and determining hardware requirements of the CNN component of our system. We parallelize the execution of convolution layer kernels on separate analog crossbars to enable 2 orders of magnitude reduction in latency compared to SOTA hybrid Memristive-CMOS Deep Learning (DL) accelerators. Furthermore, we investigate the effects of non-idealities on our system and investigate Quantization Aware Training (QAT) to mitigate the performance degradation due to low Analog-to-Digital Converter (ADC)/Digital-to-Analog Converter (DAC) resolution. Finally, we propose a stuck weight offsetting methodology to mitigate performance degradation due to stuck $R_{\text{ON}}/R_{\text{OFF}}$ memristor weights, recovering up to 32% accuracy, without requiring retraining. The CNN component of our platform is estimated to consume approximately 2.791 W of power while occupying an area of 31.255 mm2 in a 22 nm FDSOI CMOS process. [ABSTRACT FROM AUTHOR]

Convolutional neural networks (CNNs) have gained significant popularity in orthopedic imaging in recent years due to their ability to solve fracture classification problems. A common criticism of CNNs is their opaque learning and reasoning process, making it difficult to trust machine diagnosis and the subsequent adoption of such algorithms in clinical setting. This is especially true when the CNN is trained with limited amount of medical data, which is a common issue as curating sufficiently large amount of annotated medical imaging data is a long and costly process. While interest has been devoted to explaining CNN learnt knowledge by visualizing network attention, the utilization of the visualized attention to improve network learning has been rarely investigated. This paper explores the effectiveness of regularizing CNN network with human-provided attention guidance on where in the image the network should look for answering clues. On two orthopedics radiographic fracture classification datasets, through extensive experiments we demonstrate that explicit human-guided attention indeed can direct correct network attention and consequently significantly improve classification performance. The development code for the proposed attention guidance is publicly available on https://github.com/zhibinliao89/fracture_attention_guidance. [ABSTRACT FROM AUTHOR]

Neuron tracing from optical image is critical in understanding brain function in diseases. A key problem is to trace discontinuous filamentary structures from noisy background, which is commonly encountered in neuronal and some medical images. Broken traces lead to cumulative topological errors, and current methods were hard to assemble various fragmentary traces for correct connection. In this paper, we propose a graph connectivity theoretical method for precise filamentary structure tracing in neuron image. First, we build the initial subgraphs of signals via a region-to-region based tracing method on CNN predicted probability. CNN technique removes noise interference, whereas its prediction for some elongated fragments is still incomplete. Second, we reformulate the global connection problem of individual or fragmented subgraphs under heuristic graph restrictions as a dynamic linear programming function via minimizing graph connectivity cost, where the connected cost of breakpoints are calculated using their probability strength via minimum cost path. Experimental results on challenging neuronal images proved that the proposed method outperformed existing methods and achieved similar results of manual tracing, even in some complex discontinuous issues. Performances on vessel images indicate the potential of the method for some other tubular objects tracing. [ABSTRACT FROM AUTHOR]

Breast cancer is one of the main causes behind cancer deaths in women worldwide. Yet, owing to the complexity of the histopathological images and the arduousness of manual analysis task, the entire diagnosis process becomes time-consuming and the results are often contingent on the pathologist's subjectivity. Thus developing an automated, precise histopathological image classification system is crucial. This paper presents a novel hybrid ensemble framework consisting of multiple fine-tuned convolutional neural network (CNN) architectures as supervised feature extractors and eXtreme gradient boosting trees (XGBoost) as a top-level classifier, for patch wise classification of high-resolution breast histopathology images. Due to the semantic complexity of the patch images, a single CNN architecture may not always extract high quality features, and the traditional Softmax classifier might not provide ideal results for classifying the CNN extracted features. Thus we aim to improve patch wise classification by proposing a hybrid ensemble model that incorporates different discriminating feature representations of the patches, coupled with XGBoost for robust classification. Experimental results show that our proposed method outperforms state-of-the-art methods to the best of our knowledge. [ABSTRACT FROM AUTHOR]

Sleep is a vital process of our daily life as we roughly spend one-third of our lives asleep. In order to evaluate sleep quality and potential sleep disorders, sleep stage classification is a gold standard method. In this paper, we introduce a novel fully convolutional neural network architecture (SleepFCN) to classify sleep stages into five classes using single-channel electroencephalograms (EEGs). The framework of SleepFCN includes two major parts for feature extraction and temporal sequence encoding namely multi-scale feature extraction (MSFE) and residual dilated causal convolutions (ResDC), respectively. These are then followed by convolutional layers of 1-sized kernels instead of dense layers to build the fully convolutional neural network. Due to the imbalance in the distribution of sleep stages, we incorporate a weight corresponding to the number of samples of each class in our loss function. We evaluated the performance of SleepFCN using the Sleep-EDF and SHHS datasets. Our experimental results show that the proposed method outperforms state-of-the-art works in both classification correctness and learning speed. [ABSTRACT FROM AUTHOR]

Pain is an integrative phenomenon coupled with dynamic interactions between sensory and contextual processes in the brain, often associated with detectable neurophysiological changes. Recent advances in brain activity recording tools and machine learning technologies have intrigued research and development of neurocomputing techniques for objective and neurophysiology-based pain detection. This paper proposes a pain detection framework based on Electroencephalogram (EEG) and deep convolutional neural networks (CNN). The feasibility of CNN is investigated for distinguishing induced pain state from resting state in the recruitment of 10 chronic back pain patients. The experimental study recorded EEG signals in two phases: 1. movement stimulation (MS), where induces back pain by executing predefined movement tasks; 2. video stimulation (VS), where induces back pain perception by watching a set of video clips. A multi-layer CNN classifies the EEG segments during the resting state and the pain state. The novel approach offers high and robust performance and hence is significant in building a powerful pain detection algorithm. The area under the receiver operating characteristic curve (AUC) of our approach is 0.83 ± 0.09 and 0.81 ± 0.15, in MS and VS, respectively, higher than the state-of-the-art approaches. The sub-brain-areas are also analyzed, to examine distinct brain topographies relevant for pain detection. The results indicate that MS-induced pain tends to evoke a generalized brain area, while the evoked area is relatively partial under VS-induced pain. This work may provide a new solution for researchers and clinical practitioners on pain detection. [ABSTRACT FROM AUTHOR]

Quasi-periodic time series (QTS) exists widely in the real world, and it is important to detect the anomalies of QTS. In this paper, we propose an automatic QTS anomaly detection framework (AQADF) consisting of a two-level clustering-based QTS segmentation algorithm (TCQSA) and a hybrid attentional LSTM-CNN model (HALCM). TCQSA first automatically splits the QTS into quasi-periods which are then classified by HALCM into normal periods or anomalies. Notably, TCQSA integrates a hierarchical clustering and the k-means technique, making itself highly universal and noise-resistant. HALCM hybridizes LSTM and CNN to simultaneously extract the overall variation trends and local features of QTS for modeling its fluctuation pattern. Furthermore, we embed a trend attention gate (TAG) into the LSTM, a feature attention mechanism (FAM) and a location attention mechanism (LAM) into the CNN to finely tune the extracted variation trends and local features according to their true importance to achieve a better representation of the fluctuation pattern of the QTS. On four public datasets, HALCM exceeds four state-of-the-art baselines and obtains at least 97.3 percent accuracy, TCQSA outperforms two cutting-edge QTS segmentation algorithms and can be applied to different types of QTSs. Additionally, the effectiveness of the attention mechanisms is quantitatively and qualitatively demonstrated. [ABSTRACT FROM AUTHOR]

Head pose estimation is one of the sensing systems needed for some intelligent surveillance, such as human behavior analysis, intelligent driver assistance, visual attention, and monitoring. These systems require accurate alignment and head movement direction prediction. The previous methods are greatly dependent on the facial landmarks and depth information. Usually, the head pose is measured by estimating several keypoints that require a correct head pose mapping to get accurate results. Moreover, facial landmarks have a detrimental effect on performance when the face is occluded or not adequately visualized. This paper proposes a method for head pose estimation of various facial conditions, such as occlusion and challenging viewpoints. We present a combination of coarse and fine feature maps classification to train a multi-loss deep Convolutional Neural Networks (CNN) to get precise Euler angles (yaw, pitch, roll) of the head position without keypoints and landmarks. Our proposed method uses more quantization units for angle classification to learn coarse and fine structure mapping for better spatial clustering features on an RGB image of a single camera. The experiments are performed on benchmark datasets and some head poses in real cases. The mean average error of prediction is 5.06°, 4.06°, and 2.96°, for the AFLW2000, AFLW, and BIWI datasets, which significantly improves the head pose estimation performance compared to the previous methods. Additionally, the proposed method outperforms previous approaches in computation time of 11 frames per second that is beneficial for real-life applications. [ABSTRACT FROM AUTHOR]

Convolutional neural networks (CNNs) are one of the most efficient deep learning techniques and have been widely used in motor fault diagnosis. However, most of them are implemented in desktop computers to process off-line signals. In this paper, an in situ motor fault diagnosis method is proposed by implementing an enhanced CNN model into a designed embedded system consisting of a Raspberry Pi and a signal acquisition and processing circuit. To the best of our knowledge, this topic has not been investigated yet in the literature. First, the hardware, algorithms, and heterogeneous computing framework are introduced in detail. Then, the method effectiveness and efficiency are validated on a motor test rig. In particular, as the resources in an embedded system are limited, the algorithm accuracy and execution time are investigated. The robustness of the designed system is further validated by analyzing the motor signals with different signal-to-noise ratios. The contributions of this paper include the following: 1) a heterogeneous computing framework is proposed and an integrated embedded system is designed; 2) the performance of the enhanced CNN in embedded system is validated; and 3) the proposed method provides a solution to realize in situ motor fault diagnosis on a small-size, flexible, and convenient handheld device by exploiting the artificial intelligence technique. [ABSTRACT FROM AUTHOR]

In both H.264/AVC and HEVC, the angular prediction is adopted for intra coding, which only exploits the spatial correlation between the current block and its neighboring single line reference. This angular prediction can handle the main directional patterns well, however, lacks the ability to deal with other directions. In this paper, a multi-scale convolutional neural network based intra prediction is proposed to address this problem. Specifically, a predicted block is first generated by the angular prediction, then fed into the proposed network with its neighboring reconstructed $L$ -shape to generate a more accurate predicted block. On one hand, the $L$ -shape of multiple lines provides more reliable reconstructed pixels and more contextual information to get better prediction; on the other hand, the multi-scale feature extraction takes the advantage of the feature maps in different scales to further enhance the prediction. With this multi-scale structure, the $L$ -shape can be used to refine both left-above and right-bottom pixels in the predicted block during the convolution operation. Experimental results demonstrate that compared with HEVC reference software HM 16.9, the proposed intra prediction can achieve an average of 3.4% (up to 5.6%) bitrate saving with all intra configuration. [ABSTRACT FROM AUTHOR]

In this paper, the traditional model based variational methods and deep learning based algorithms are naturally integrated to address mixed noise removal, specially for Gaussian mixture noise and Gaussian-impulse noise removal problem. To be different from single type noise (e.g. Gaussian) removal, it is a challenge problem to accurately discriminate noise types and levels for each pixel. We propose a variational method to iteratively estimate the noise parameters, and then the algorithm can automatically classify the noise according to the different statistical parameters. The proposed variational problem can be separated into regularization, synthesis, parameters estimation and noise classification four steps with the operator splitting scheme. Each step is related to an optimization subproblem. To enforce the regularization, the deep learning method is employed to learn the natural images prior. Compared with some model based regularizations, the CNN regularizer can significantly improve the quality of the restored images. Compared with some learning based methods, the synthesis step can produce better reconstructions by analyzing the types and levels of the recognized noise. In our method, the convolution neutral network (CNN) can be regarded as an operator which associated to a variational functional. From this viewpoint, the proposed method can be extended to many image reconstruction and inverse problems. Numerical experiments in the paper show that our method can achieve some state-of-the-art results for Gaussian mixture noise and Gaussian-impulse noise removal. [ABSTRACT FROM AUTHOR]

Text-based CAPTCHAs are the most widely used CAPTCHA scheme. Most text-based CAPTCHAs have been cracked. However, previous works have mostly relied on a series of preprocessing steps to attack text CAPTCHAs, which was complicated and inefficient. In this paper, we introduce a simple, generic, and effective end-to-end attack on text CAPTCHAs without any preprocessing. Through a convolutional neural network and an attention-based recurrent neural network, our attack broke a wide range of real-world text CAPTCHAs that are deployed by the top 50 most popular websites ranked by Alexa.com. In addition, this paper comprehensively analyzed the security of most resistance mechanisms of text-based CAPTCHAs through experiments. Experimental results prove that the anti-segmentation principle can be completely broken under deep learning attacks without any segmentation or preprocessing steps in contrast to commonly held beliefs. [ABSTRACT FROM AUTHOR]

In recent years, the image and video coding technologies have advanced by leaps and bounds. However, due to the popularization of image and video acquisition devices, the growth rate of image and video data is far beyond the improvement of the compression ratio. In particular, it has been widely recognized that there are increasing challenges of pursuing further coding performance improvement within the traditional hybrid coding framework. Deep convolution neural network which makes the neural network resurge in recent years and has achieved great success in both artificial intelligent and signal processing fields, also provides a novel and promising solution for image and video compression. In this paper, we provide a systematic, comprehensive and up-to-date review of neural network-based image and video compression techniques. The evolution and development of neural network-based compression methodologies are introduced for images and video respectively. More specifically, the cutting-edge video coding techniques by leveraging deep learning and HEVC framework are presented and discussed, which promote the state-of-the-art video coding performance substantially. Moreover, the end-to-end image and video coding frameworks based on neural networks are also reviewed, revealing interesting explorations on next generation image and video coding frameworks/standards. The most significant research works on the image and video coding related topics using neural networks are highlighted, and future trends are also envisioned. In particular, the joint compression on semantic and visual information is tentatively explored to formulate high efficiency signal representation structure for both human vision and machine vision, which are the two dominant signal receptors in the age of artificial intelligence. [ABSTRACT FROM AUTHOR]

The primary objective of this paper is to build classification models and strategies to identify breathing sound anomalies (wheeze, crackle) for automated diagnosis of respiratory and pulmonary diseases. In this work we propose a deep CNN-RNN model that classifies respiratory sounds based on Mel-spectrograms. We also implement a patient specific model tuning strategy that first screens respiratory patients and then builds patient specific classification models using limited patient data for reliable anomaly detection. Moreover, we devise a local log quantization strategy for model weights to reduce the memory footprint for deployment in memory constrained systems such as wearable devices. The proposed hybrid CNN-RNN model achieves a score of $\text{66.31}\%$ on four-class classification of breathing cycles for ICBHI’17 scientific challenge respiratory sound database. When the model is re-trained with patient specific data, it produces a score of $\text{71.81}\%$ for leave-one-out validation. The proposed weight quantization technique achieves ${\approx}4 \times$ reduction in total memory cost without loss of performance. The main contribution of the paper is as follows: Firstly, the proposed model is able to achieve state of the art score on the ICBHI’17 dataset. Secondly, deep learning models are shown to successfully learn domain specific knowledge when pre-trained with breathing data and produce significantly superior performance compared to generalized models. Finally, local log quantization of trained weights is shown to be able to reduce the memory requirement significantly. This type of patient-specific re-training strategy can be very useful in developing reliable long-term automated patient monitoring systems particularly in wearable healthcare solutions. [ABSTRACT FROM AUTHOR]

Volumetric imaging is an essential diagnostic tool for medical practitioners. The use of popular techniques such as convolutional neural networks (CNN) for analysis of volumetric images is constrained by the availability of detailed (with local annotations) training data and GPU memory. In this paper, the volumetric image classification problem is posed as a multi-instance classification problem and a novel method is proposed to adaptively select positive instances from positive bags during the training phase. This method uses the extreme value theory to model the feature distribution of the images without a pathology and use it to identify positive instances of an imaged pathology. The experimental results, on three separate image classification tasks (i.e. classify retinal OCT images according to the presence or absence of fluid build-ups, emphysema detection in pulmonary 3D-CT images and detection of cancerous regions in 2D histopathology images) show that the proposed method produces classifiers that have similar performance to fully supervised methods and achieves the state of the art performance in all examined test cases. [ABSTRACT FROM AUTHOR]

Real-time detection of inshore ships plays an essential role in the efficient monitoring and management of maritime traffic and transportation for port management. Current ship detection methods which are mainly based on remote sensing images or radar images hardly meet real-time requirement due to the timeliness of image acquisition. In this paper, we propose to use visual images captured by an on-land surveillance camera network to achieve real-time detection. However, due to the complex background of visual images and the diversity of ship categories, the existing convolution neural network (CNN) based methods are either inaccurate or slow. To achieve high detection accuracy and real-time performance simultaneously, we propose a saliency-aware CNN framework for ship detection, comprising comprehensive ship discriminative features, such as deep feature, saliency map, and coastline prior. This model uses CNN to predict the category and the position of ships and uses the global contrast based salient region detection to correct the location. We also extract coastline information and respectively incorporate it into CNN and saliency detection to obtain more accurate ship locations. We implement our model on Darknet under CUDA 8.0 and CUDNN V5 and use a real-world visual image dataset for training and evaluation. The experimental results show that our model outperforms representative counterparts (Faster R-CNN, SSD, and YOLOv2) in terms of accuracy and speed. [ABSTRACT FROM AUTHOR]

With the improvement of sensor techniques, and the urgent requirement of automatic fault diagnosis technologies, the intelligent perception system on high speed train is more popular than ever before. It records the devices’ state information through a sensor network, and services for further analysis. However, Traditional machine learning algorithms are usually constrained by massive multi-sensor data and knowledge-based feature extraction in fault diagnosis. Therefore, this paper extended fault diagnosis methodology into tensor space to deal with multi-sensor monitoring data and take full use of available information. Moreover, the convolutional neural network (CNN) is used for automatic feature learning and classification without human intervention. The effectiveness and efficiency are validated by dataset of rolling element bearings obtained in lab and real-use case. Three features can be highlighted. First of all, the proposed model showed a good adaptability and high efficiency under various working condition by taking full use of the multi-sensor data. It has powerful ability in accuracy and convergence speed. Secondly, it is not as sensitive to data quantity as other deep learning algorithms do. Such superior characteristic made the model more suitable for practical application, because of the insufficient failure data. At last, it is an intelligent End-to-End model, performing automatic fault diagnosis without manual intervention and suitable for real-use case. [ABSTRACT FROM AUTHOR]

Moving object detection (MOD) in videos is a challenging task. Estimation of accurate background is the key to extracting the foreground from video frames. In this paper, we have proposed a novel compact end-to-end convolutional neural network architecture, motion saliency foreground network (MSFgNet), to estimate the background and to extract the foreground from video frames. Initially, the long streaming video is divided into a number of small video streams (SVS). The proposed network takes the SVS as an input and estimates the background frame for each SVS. Second, the saliency map is extracted using the current video frame and estimated background. Furthermore, a compact encoder–decoder network is proposed to extract the foreground from the estimated saliency maps. The performance of the proposed MSFgNet is tested on three benchmark datasets (CDnet-2014, LASIESTA, and PTIS) for MOD. The computational complexity (handling of number of parameters and execution time) and the performance of the proposed MSFgNet are compared with the existing state-of-the-art methods for MOD in terms of precision, recall, and F-measure. Performance analysis shows that the proposed network is very compact and outperforms the existing state-of-the-art methods for MOD in videos. [ABSTRACT FROM AUTHOR]

Siamese networks are prevalent in person re-identification (re-id) tasks to address the similarity and dissimilarity among video frames. It mainly focuses on the inter-video variation between spatio-temporal features extracted from different videos, while the variation between features of the same video has been rarely discussed. In this paper, we introduce the concept of “mean-body” and define an intra-video loss to address the variation between spatio-temporal features of the same video. A novel loss is presented to boost the training of the re-id networks by combining the proposed intra-video loss and the Siamese loss. Specifically, the intra-video loss uses the unique mean-body of each camera viewpoint to make the video sequence more clustered, while the Siamese loss is to make the wrong matching videos more separated. To train the whole network, we update the network and the mean-body in an iterative manner. As a result, the proposed loss is expected to improve the generalization capability of the re-id networks on the testing set. Extensive results demonstrate that the presented approach outperforms the state-of-the-art algorithms on the publicly available data sets, such as PRID2011, iLIDS-VID, and MARS, in terms of re-id accuracy. [ABSTRACT FROM AUTHOR]

In this paper, we present novel sharp attention networks by adaptively sampling feature maps from convolutional neural networks for person re-identification (re-ID) problems. Due to the introduction of sampling-based attention models, the proposed approach can adaptively generate sharper attention-aware feature masks. This greatly differs from the gating-based attention mechanism that relies on soft gating functions to select the relevant features for person re-ID. In contrast, the proposed sampling-based attention mechanism allows us to effectively trim irrelevant features by enforcing the resultant feature masks to focus on the most discriminative features. It can produce sharper attentions that is more assertive in localizing subtle features relevant to re-identifying people across cameras. For this purpose, a differentiable Gumbel-Softmax sampler is employed to approximate the Bernoulli sampling to train the sharp attention networks. Extensive experimental evaluations demonstrate the superiority of this new sharp attention model for person re-ID over other related existing, published state-of-the-art works on three challenging benchmarks, including CUHK03, Market-1501, and DukeMTMC-reID. [ABSTRACT FROM AUTHOR]