Search

In this paper, a three-step model based on the integration of Deep Neural Networks (DNN) and Decision Models is introduced for image classification which is inspired by the human visual system. To make a decision about an object, many actions should be done in a hierarchical process in the brain. First, the retina receives visual stimuli and transfers them to the visual cortex in the brain. The information extracted in the visual cortex, is accumulated over time to select an appropriate response. Many of the current decision-making models do not show how each image is converted into useful information for the decision model. Some models have used neural networks to convert each image into the information needed in the decision-making model; however, the role of the retina is ignored among these models. In this paper, a combination of retina inspired filters, CNN-based description and accumulator-based decision model is used to classify images. This model's structure resembles the human brain due to the usage of the DoG filter bank as retina inspired filter in the first stage of it. This model shows a significant improvement in accuracy in comparison to other models; furthermore, its performance is acceptable even with the small sample training set. [ABSTRACT FROM AUTHOR]

In personal identity recognition systems, detecting a person's age, gender, and language using voice signal characteristics is a crucial issue, especially because of the importance of security considerations. Age, gender, and language classification problems are important in signal processing because they are used to analyze and understand human behavior, interactions, and preferences. This can be especially useful in the fields of human-computer interaction, psychology, and social science research. In this paper, a new system for detecting a speaker's age, gender, and language based on deep learning models is presented. Deep learning models have shown great efficacy in various fields of signal processing. For this paper, a range of deep models were tested, including convolutional neural networks (CNNs), recurrent neural network (RNN), and a fine-tuning ResNet34 architecture. Additionally, techniques such as transfer learning were applied to improve the efficiency of the proposed system. The input voice signals are preprocessed by applying the spectro-temporal transform to obtain additional features that can be fed to the ResNet34 model, which is designed specifically for the task of voice signal processing. The dataset used in this paper was sourced from the Mozilla common voice initiative, which is dedicated to advancing speech recognition and language identification technologies. The performance of the proposed algorithm was evaluated in the presence of Gaussian noise to determine its robustness. The experimental results demonstrated that the proposed algorithm significantly outperformed basic algorithms and other deep neural networks in terms of age and gender recognition from voice signals. [ABSTRACT FROM AUTHOR]

Filter pruning is proven to be an effective strategy in model compression. However, convolutional filter pruning methods usually pay all attention to evaluating filters' importance at a single layer, ignoring their collaborative relationship with corresponding filters of the next layer. In this paper, we propose novel consecutive layer collaborative filter similarity (CLCS) to make full use of the complete filter information and learn binary selection vectors to prune the redundant filters automatically. With learned selection vectors, the pruning ratio of each layer can be determined, and we can also calculate the FLOPs of the candidate pruned network at the current stage. Under the accuracy constraint and the FLOPs constraint, the selection vectors of each layer can be optimized to achieve a better trade-off between accuracy and efficiency. Extensive experiments on CIFAR-10 and ImageNet with multiple networks demonstrate the effectiveness of our proposed method. Specifically, we obtain 54.29% and 67.33% FLOPs reduction with 0.01% and 0.09% accuracy improvement for ResNet-56 and ResNet-110 on CIFAR-10, respectively. On ImageNet, we reduce FLOPs by nearly half compared to the ResNet-50 baseline with almost no loss of accuracy. Compared with state-of-the-art filter pruning methods, our approach also achieves superior results. [ABSTRACT FROM AUTHOR]

Modern and new integrated technologies have changed the traditional systems by using more advanced machine learning, artificial intelligence methods, new generation standards, and smart and intelligent devices. The new integrated networks like the Internet of Things (IoT) and 5G standards offer various benefits and services. However, these networks have suffered from multiple object detection, localization, and classification issues. Conventional Neural Networks (CNN) and their variants have been adopted for object detection, classification, and localization in IoT networks to create autonomous devices to make decisions and perform tasks without human intervention and helpful to learn in-depth features. Motivated by these facts, this paper investigates existing object detection and recognition techniques by using CNN models used in IoT networks. This paper presents a Conventional Neural Networks for 5G-Enabled Internet of Things Network (CNN-5GIoT) model for moving and static objects in IoT networks after a detailed comparison. The proposed model is evaluated with existing models to check the accuracy of real-time tracking. The proposed model is more efficient for real-time object detection and recognition than conventional methods. [ABSTRACT FROM AUTHOR]

Session-based recommendation aims to predict a user's next action based on a series of anonymous sequences and plays an essential role in various online applications, such as e-commerce and music applications. Recently, transformer-based models have obtained results that are competitive with or even surpass those of recurrent neural networks, because of the good performance of transformer models in capturing long-distance dependencies. However, a transformer has a limited ability to mine local contextual information, which can be regarded as collective features. Researchers are seeking to address this limitation by augmenting the contextual transition to boost session representation learning. Accordingly, in this paper, we enhance the capabilities of a transformer in a session-based recommendation task by introducing convolutional neural networks (CNNs) at the stage of aggregating the item features with long- and short-distance dependencies. We first borrow a self-attention module from the classic transformer model to explore the long-distance dependencies. We next propose horizontal and vertical convolutions for enhancing the local collective information and then obtain a session representation by integrating the two types of features. Extensive experiments on real-world datasets show that our method outperforms those that rely on a transformer or a CNN alone. [ABSTRACT FROM AUTHOR]

Active contour models (ACMs) are popular and widely used for many image segmentation applications and obtain promising results. However, these methods are unable to achieve the highest performance in the presence of intensity inhomogeneity. To address this issue, this paper presents an ACM based on the combination of a convolutional neural network (CNN) and a texture descriptor approach. This study uses a CNN model to generate parameter maps more effectively for ACM. Compared to conventional global techniques, these parameter maps increase the speed of movement of the contour into the target. In this approach, the Local Word Directional Pattern (LWDP) is applied as the texture descriptor. LWDP is a texture descriptor that uses the angle between two gradients for exploring the texture structure inside the image. In the proposed method both the original image and the new image obtained by the LWDP texture descriptor (encoded image) are provided as inputs to the CNN. The experimental outcomes show that the proposed strategy consistently outperforms the state-of-the-art in accuracy and robustness for segmenting images with fuzzy boundaries and intensity inhomogeneity. [ABSTRACT FROM AUTHOR]

Although there have been a lot of researches on convolutional neural networks (CNNs), still what happens in this black box remains a mystery. In this paper, we establish the connection between CNNs and signal modulation. From a signal modulation point of view, the forward-propagation process of CNNs can be explained as a process of modulating the input signals to the vicinity of a special energy spectrum distribution, and the back-propagation process is searching for the appropriate distribution which is better for classification or other tasks. Several experiments have been carried out to verify the modulated explanation of CNNs. Furthermore, we verify that modulating the signal to the appropriate energy spectrum distribution in advance can effectively improve the classification and segmentation accuracy. [ABSTRACT FROM AUTHOR]

Researchers have started to utilize the structures like Feature Pyramid Network (FPN) for crowd counting to extract multi-scale features, which can solve the scale variation problem in traditional density map regression-based methods. However, many downsampling operations in FPN inevitably induce loss of location information, resulting in pixel mismatch of the predicted density map. In this paper, we propose the Location Recovery Module (LRM) to recover the loss of location information. Unlike the basic feature fusion module in FPN, the proposed LRM builds an offset map to modify the position of the downsampled features under the guidance of low-level features. Meanwhile, we design a novel regression strategy to fully utilize the multi-scale information without upsampling operations in the fusion process. Considering that the highest-level feature cannot represent all density-level features well, different-scale features focus more on their related regions in our regression strategy. The proposed HARNet has been tested on four datasets and achieved state-of-the-art on two datasets. The performance metric such as MAE arrives at 52.4, 5.9 and 181.7 on ShanghaiTech A/B and UCF_CC_50 respectively. [ABSTRACT FROM AUTHOR]

[Display omitted] • In this paper, we present a RDRF-Net for depth estimation with high accuracy. • It introduces a dynamic adaptive selection method of receptive fields for higher accuracy in depth estimation. • It introduces a residual architecture of 3 × 3 kernels instead of parallel 3 × 3 and 5 × 5 kernels to simulate two receptive fields, which leads to model compression and less network parameters. • It introduces a bottleneck architecture to further compress the network without explicit accuracy loss. Image depth estimation is a challenging problem in computer vision, especially considering both high accuracy and low run time. To save run time and maintain high accuracy, we present a new light-weight model in this paper, i.e., a Residual-based Dynamic Receptive Field Network (RDRF-Net). This model can automatically select the receptive fields suitable for different image scales to generate the depth maps with higher fitting degrees. Residual design and bottleneck layers are used to compress the network for reducing run time. Three groups of experiments are performed on the KITTI dataset to test the accuracy, computation time, and the impact of dynamic receptive fields. Experimental results show that RDRF-Net has comparable accuracy with Godard's model and significantly outperforms it in terms of run time. In addition, it performs closely to Pyd-Net in terms of run time and beats Pyd-Net's accuracy. Experiments also demonstrate the beneficial impact of dynamic receptive fields on improving depth estimation accuracy. [ABSTRACT FROM AUTHOR]

Deep Learning (DL) has been recently employed to build smart systems that perform incredibly well in a wide range of tasks, such as image recognition, machine translation, and self-driving cars. In several fields the considerable improvement in the computing hardware and the increasing need for big data analytics has boosted DL work. In recent years physiological signal processing has strongly benefited from deep learning. In general, there is an exponential increase in the number of studies concerning the processing of electromyographic (EMG) signals using DL methods. This phenomenon is mostly explained by the current limitation of myoelectric controlled prostheses as well as the recent release of large EMG recording datasets, e.g. Ninapro. Such a growing trend has inspired us to seek and review recent papers focusing on processing EMG signals using DL methods. Referring to the Scopus database, a systematic literature search of papers published between January 2014 and March 2019 was carried out, and sixty-five papers were chosen for review after a full text analysis. The bibliometric research revealed that the reviewed papers can be grouped in four main categories according to the final application of the EMG signal analysis: Hand Gesture Classification, Speech and Emotion Classification, Sleep Stage Classification and Other Applications. The review process also confirmed the increasing trend in terms of published papers, the number of papers published in 2018 is indeed four times the amount of papers published the year before. As expected, most of the analyzed papers (≈ 60 %) concern the identification of hand gestures, thus supporting our hypothesis. Finally, it is worth reporting that the convolutional neural network (CNN) is the most used topology among the several involved DL architectures, in fact, the sixty percent approximately of the reviewed articles consider a CNN. [ABSTRACT FROM AUTHOR]

In this work, we propose a Fractional-order Residual Convolutional Neural Network (FRCNN) for Low-Dose CT (LDCT) denoising. As increasing the dose of radiation is harmful to the patient, how to trade off between reducing the radiation dose and improving the quality of the CT image has become a challenging problem. To this end, this paper proposes a new approach for LDCT image denoising using Convolutional Neural Network (CNN) with Fractional-order Total Variation (FTV) loss, as well as residual learning. Firstly, this paper introduced the FTV loss function for structural details enhancement. Secondly, skip connections were added to optimize the network. Thirdly, extensive experimental analysis was used to evaluate the capacity of this method in suppressing noise and preserving detailed information. The FTV loss can retain essential structural details while suppressing noise, generating high-quality CT images ready for interpretation by radiologists. Compared with state-of-the-art methods, our method obtained better results visually and numerically, especially in structural details preservation. These promising results will significantly improve the usability of LDCT images. [ABSTRACT FROM AUTHOR]

We address the task of establishing visual correspondences between two images depicting main objects of the same semantic category. This task encounters various challenges such as background clutter, intra-class variation, and viewpoint variations. Existing works are dominated by end-to-end training methods that rely on redundant calculation or large amounts of manual annotations, and cannot generalize to unseen object categories. In this paper, we propose to construct a weakly supervised object-aware convolutional neural network architecture for semantic feature matching, while being trainable end-to-end without the requirement for manual annotations. The main component of this architecture is a similarity filter module containing a trainable neural nearest neighbors network. Since training data for semantic feature matching is rather limited, we develop an object-aware matching mechanism to enable weakly-supervised learning and tackle the background clutter. Extensive experiments illustrate that the proposed approach outperforms the state-of-the-art methods for semantic feature matching on multiple public standard benchmark datasets. [Display omitted] We address the task of establishing visual correspondences between two images depicting main objects of the same semantic category. This task encounters various challenges such as background clutter, intra-class variation, and viewpoint variations. Existing works are dominated by end-to-end training methods that rely on redundant calculation or large amounts of manual annotations, and cannot generalize to unseen object categories. In this paper, we propose to construct a weakly supervised object-aware convolutional neural network architecture for semantic feature matching, while being trainable end-to-end without the requirement for manual annotations. The main component of this architecture is a similarity filter module containing a trainable neural nearest neighbors network. Since training data for semantic feature matching is rather limited, we introduce a simple and effective foreground selection strategy to produce the foreground masks. Using these masks as a form of weak supervision signal for correspondence task and tackle the background clutter. Extensive experiments illustrate that the proposed approach outperforms the state-of-the-art methods for semantic feature matching on multiple public standard benchmark datasets. [ABSTRACT FROM AUTHOR]

Recently, the successful applications of convolutional neural network (CNN) in computer vision have attracted considerable attentions. Particularly, deep learning models with attention mechanisms have shown impressive performance in hyperspectral (HS) pansharpening. However, most of these existing models follow an early/late-fusion strategy and do not take full advantage of the hierarchical features. In this paper, we specifically design a novel end-to-end multi-level feature fusion network with cross-layer guided attention for HS pansharpening, termed as HP-MFFN, which allows the network to extract the hierarchical features level by level. Specifically, as the different levels of the network have different receptive fields and contain different details, the hierarchical features extracted from the HS image and panchromatic (PAN) image are refined by the cross-layer guided attention fusion module (called CLGAF) to yield more effective spatial-spectral features with fine details and rich semantics. The experimental results conducted on widely-used datasets demonstrate that HP-MFFN provides high-quality pansharpened HS images in terms of perceptually and quantitatively. [ABSTRACT FROM AUTHOR]

• One new 1DCNN is established by adding a midpoint residual block before the middle layer. • A multiscale cascade structure is constructed to extract features from the original data. • The feature extraction capability is improved by multiscale cascade midpoint residual. • The anti-noise robustness is greatly improved by utilizing the ELU activation function. Convolutional Neural Network (CNN) has been widely used in mechanical fault diagnosis system, and has achieved satisfactory results. However, some limitations of the number of network layers and a single fixed convolution kernel are also exposed during performing the task of fault classification. To solve these problems, this paper proposes a multiscale cascade midpoint residual convolutional neural network (MSC-MpResCNN). Firstly, a new multiscale cascade structure is introduced to extract multi-resolution features contained in the original data. Secondly, the improved midpoint residual block is adopted in each branch of multiscale cascade structure to address deep network performance degradation. In addition, exponential linear unit (ELU) replaces the original linear rectification function, which makes the noise resistance of the model stronger and increasesthe robustness and generalization. L2 weight regularization and global average pooling (GAP) are applied to the model to avoid overfitting. The feasibility of the proposed method is validated by the experiments. The results indicates that the method can obtain higher fault recognition rate compared with previous methods by multiscale cascade midpoint residual block. Furthermore, the method has great anti-noise robustness under strong noise environment. [ABSTRACT FROM AUTHOR]

• We review artificial intelligence applications of brain image preprocessing. • We investigate several commonly seen brain diseases and survey their intelligence-based methods. • We discuss AI's future in brain disease studies. Brain diseases threaten hundreds of thousands of people over the world. Medical imaging techniques such as MRI and CT are employed for various brain disease studies. As artificial intelligence succeeded in image analysis, scientists employed artificial intelligence, especially deep learning technologies, to assist brain disease studies. The AI applications for brain disease studies can be divided into two categories. The first category is preprocessing, including denoising, registration, skull-stripping, intensity normalization, and data augmentation. The second category is the clinical application that contains lesion segmentation, disease detection, grade classification, and outcome prediction. In this survey, we reviewed over one hundred representative papers on how to apply AI to brain disease studies. We first introduced AI-based preprocessing for brain disease studies. Second, we reviewed the influential works of AI-based brain disease studies. At last, we also discussed three development trends in the future. We hope this survey will inspire both expert-level researchers and entry-level beginners. [ABSTRACT FROM AUTHOR]

In this work, we consider transferring global information from Transformer to Convolutional Neural Network (CNN) for medical semantic segmentation tasks. Previous network models for medical semantic segmentation tasks often suffer from difficulties in modeling global information or oversized model parameters. Here, to design a compact network with global and local information, we extract the global information modeling capability of Transformer into the CNN network and successfully apply it to the medical semantic segmentation tasks, called Global Information Distillation. In addition, the following two contributions are proposed to improve the effectiveness of distillation: i) We present an Information Transfer Module, which is based on a convolutional layer to prevent over-regularization and a Transformer layer to transfer global information; ii) For purpose of better transferring the teacher's soft targets, a Shrinking Result-Pixel distillation method is proposed in this paper. The effectiveness of our knowledge distillation approach is demonstrated by the experiments on multi-organ and cardiac segmentation tasks. [ABSTRACT FROM AUTHOR]

In this paper we present a framework for the design and implementation of offset equivariant networks, that is, neural networks that preserve in their output uniform increments in the input. In a suitable color space this kind of networks achieves equivariance with respect to the photometric transformations that characterize changes in the lighting conditions. We verified the framework on three different problems: image recognition, illuminant estimation, and image inpainting. Our experiments show that the performance of offset equivariant networks are comparable to those in the state of the art on regular data. Differently from conventional networks, however, equivariant networks do behave consistently well when the color of the illuminant changes. [ABSTRACT FROM AUTHOR]

Image deblurring is a fundamental and challenging low-level vision problem. Previous vision research indicates that edge structure in natural scenes is one of the most important factors to estimate the abilities of human visual perception. In this paper, we resort to human visual demands of sharp edges and propose a two-phase edge-aware deep network to improve deep image deblurring. An edge detection convolutional subnet is designed in the first phase and a residual fully convolutional deblur subnet is then used for generating deblur results. The introduction of the edge-aware network enables our model with the specific capacity of enhancing images with sharp edges. We successfully apply our framework on standard benchmarks and promising results are achieved by our proposed deblur model. [ABSTRACT FROM AUTHOR]

In modern convolutional neural networks (CNNs), the pooling layer is seen as one of the primary layers for building the CNN model, which effectively downscales the spatial size of feature maps to reduce memory consumption. Several types of pooling operations, such as average pooling, max pooling, and strided convolution, fail to capture the spatial dependence between the pooling region feature and its neighbour features. In this paper, we propose a simple but effective attention-based pooling method called Neighbour Feature Attention-Based Pooling (NFP), which integrates neighbour features of the pooling region to keep semantic continuity across multiple layers. NFP adopts attention weights encoding with neighbour features by depthwise convolution, which effectively directs local spatial pooling for learning discriminative features. Compared to other pooling methods, the proposed method generates more discriminative features directed by neighbour information of the pooling region. The experiments results show that it consistently improves the performance across various backbone architectures on image classification tasks. [ABSTRACT FROM AUTHOR]

Recent artificial intelligence-based methods have shown great promise in the use of neural networks for real-time detection of transmission line faults and estimation of their locations. The expansion of power systems including transmission lines with various lengths have made the fault detection, classification, and location estimation process more challenging. Transmission line datasets are stream data which are continuously collected by various sensors and hence, require generalized and fast fault diagnosis approaches. Newly collected datasets including voltages and currents for faulty and non-faulty situations might not have adequate and accurate labels that are useful to train neural networks. In this paper, a novel transfer learning framework based on a pre-trained LeNet-5 convolutional neural network is proposed. This method is able to diagnose faults for different transmission line lengths and impedances by transferring the knowledge from a source convolutional neural network to predict a dissimilar target dataset. By transferring this knowledge, faults from various transmission lines, even without sufficient data samples with labels, can be diagnosed faster and more efficiently than the existing methods. To prove the feasibility and effectiveness of this methodology, seven different datasets that include various lengths of transmission lines are used. The robustness of the proposed methodology against the generator voltage fluctuations, variations in fault locations, fault inception angle, fault resistance, and phase difference between the two generators are well studied to prove the reliability of this technique for fault diagnosis of transmission lines. [ABSTRACT FROM AUTHOR]

The evaluation of node importance is a critical research topic in network science, widely applied in social networks, transport systems, and computer networks. Prior works addressing this topic either consider a single metric or assign weights for multiple metrics or select features by handcraft, which exist one-sidedness and subjectivity issues. In this paper, to tackle these problems, we propose a new approach named CGNN to identify influential nodes based on deep learning methods, including Convolutional Neural Networks (CNNs) and Graph Neural Networks (GNNs). CGNN obtains the feature matrices by the contraction algorithm and gets the labels by the Susceptible-Infected-Recovered (SIR) model, which will be leveraged for learning the hidden representations of nodes without utilizing any network metrics as features. We adopt three evaluation criteria to verify CGNN concerning effectiveness and distinguishability, including Kendall's τ correlation coefficient, monotonicity index (MI), and ranking distribution function (RDF). Nine baselines are employed to compare with CGNN on thirty synthetic networks and twelve real-world networks from different domains. Simulation results demonstrate that CGNN manifests better performance than the baselines, in which the values of τ are large and significantly increase, the values of MI approach to 1, and the points in the RDF curves distribute more uniformly. These results may provide reference significance for controlling epidemic spreading and enhancing network robustness. [ABSTRACT FROM AUTHOR]

Temporal knowledge graphs (TKGs) are still far from completeness although they have benefited many artificial intelligence applications successfully. To alleviate the incompleteness of TKGs, temporal knowledge graph completion (TKGC) aims to perform link prediction and reason new time-sensitive facts from existing facts. Among the various TKGC methods, temporal knowledge graph embedding (TKGE) methods represent entities, relations, and timestamps in low-dimensional spaces, which are known for their high reasoning performance and robust scalability. Some previous researches on TKGE aim to endow the embedding quadruple with the translational characteristic by modeling the global relationship in the embedding quadruple. However, the weights of same dimensional entries of the embedding quadruple are fixed and unique, which limits the ability to capture diverse global relationships. Besides, most TKGE models view timestamps as a kind of general feature and overlook their interrelations and correlations with the queries. Moreover, the distinct characteristics of an entity as the subject entity and the object entity of facts have not been fully leveraged. In this paper, we propose a query-aware embedding model based on the convolutional neural network (CNN) named ConvTKG to perform TKGE, thus tackling the TKGC task. We design a new temporal information encoder based on the Gated Recurrent Unit (GRU) and attention mechanism to learn the query-aware representation of temporal information. More importantly, we utilize a CNN-based decoder where multiple one-dimensional convolution kernels are operated on the matrix composed of entity embedding, relation embedding, and temporal representation to capture global relationships among them. In addition, we assign two independent vectors for each entity and take advantage of the inverse relation to allow them to be learned dependently. Experiments show that ConvTKG achieves better link prediction performance than previous state-of-the-art baseline methods for TKGC on three benchmark datasets: ICEWS14, ICEWS05-15, and GDELT. • We propose the first weight-adaptive TKGE model with translational characteristic. • We introduce a new query-aware timestamp encoding method for TKGC. • Two dependent embedding vectors for each entity capture its distinct semantics. • Experiments demonstrate the effectiveness of ConvTKG on three TKGC benchmarks. [ABSTRACT FROM AUTHOR]

Capsule network is a promising network architecture in computer vision that integrates feature information and feature relationships of images. However, the original capsule network performs poorly with complex datasets, and the network is computationally intensive, which is not conducive to deployment in end-side devices. In this paper, a novel hardware-friendly capsule network named MRCapsNet is proposed to improve the performance of the capsule network and make it deployable in end-side devices. In the proposed network, the multi-level residual capsule block is constructed to extract features with multi-granularity from images. Moreover, a novel reconstruction subnetwork is designed to facilitate network training. Experiments show that MRCapsNet achieved competitive results on the CIFAR-10 (90.34 %) and SVHN (96.59 %) datasets compared with other variants of CapsNet. Further, the hardware implementation scheme of MRCapsNet based on memristor crossbars is designed to provide a nano-scale, low-power capsule network deployment scheme. Finally, the power consumption of the core computing unit built by memristor elements is analyzed and calculated, and the maximum power consumption of a memristive neuron is only 6.4 μ W during the inference of the network, which is significantly less than the CMOS-based circuit. [ABSTRACT FROM AUTHOR]

Convolutional neural networks (CNNs) have significantly advanced computational modelling for saliency prediction. However, accurately simulating the mechanisms of visual attention in the human cortex remains an academic challenge. It is critical to integrate properties of human vision into the design of CNN architectures, leading to perceptually more relevant saliency prediction. Due to the inherent inductive biases of CNN architectures, there is a lack of sufficient long-range contextual encoding capacity. This hinders CNN-based saliency models from capturing properties that emulate viewing behaviour of humans. Transformers have shown great potential in encoding long-range information by leveraging the self-attention mechanism. In this paper, we propose a novel saliency model that integrates transformer components to CNNs to capture the long-range contextual visual information. Experimental results show that the transformers provide added value to saliency prediction, enhancing its perceptual relevance in the performance. Our proposed saliency model using transformers has achieved superior results on public benchmarks and competitions for saliency prediction models. The source code of our proposed saliency model TranSalNet is available at: https://github.com/LJOVO/TranSalNet. [ABSTRACT FROM AUTHOR]

With the prevalence and the enormous societal consequence on health of Alzheimer's disease (AD), diagnosis of AD and its prodromal form, mild cognitive impairment (MCI) is essential for patient care, and has been a research hotspot in recent years. Existing studies have applied machine learning methods to perform AD early diagnosis by analyzing various biomarkers. However, the difficulty in extracting the low-dimensional high-level brain features that accurately reflect main AD-related variations of anatomical brain structures becomes a bottleneck of the diagnosis performance in most of the existing researches. To overcome this bottleneck, this paper proposes a novel three-component adversarial network-based AD detection method (brain slice generative adversarial network for Alzheimer's disease detection, BSGAN-ADD) to predict the disease category. BSGAN-ADD combines generative adversarial network (GAN)-based brain slice image enhancement and deep convolutional neural network (CNN)-based AD detection. In BSGAN-ADD, under the restriction of the discriminator, the generator learns to integrate the disease category feedbacks from classifier into 2D-brain slice image reconstruction process for image enhancement in the training phase. In the prediction phase, the stacked CNN layers in the generator are used to extract high-level brain features from category-enhanced 2D-brain slice images. And the classifier receives the extracted brain features to output the posterior probabilities of diseased states (Normal, AD and MCI). Experimental results on two real-world datasets (Alzheimer's disease neuroimaging initiative, ANDI, Open Access Series of Imaging Studies OASIS) demonstrate that the new feature extraction process used in BSGAN-ADD can extract more representative high-level brain features to achieve a significant diagnosis performance gain compared with several typical methods. [ABSTRACT FROM AUTHOR]

This paper proposes a distributed deep reinforcement learning algorithm for the traffic light control problem, which consists of local learning and global consensus. Firstly, the reinforcement learning environment for the traffic light control problem is built by defining the three key elements of state, action, and reward. Then, the CNN-based deep Q-network is designed to process the quantized traffic state information to obtain the state-action values. After locally optimizing the deep Q-networks of multiple traffic light agents based on their experience samples, the consensus algorithm is subsequently applied to globally update these agents that are connected over a decentralized communication topology. In this way, the distributed learning agents learn from their neighbors' experience to optimize the modeling process without actually sharing experience data samples. Lastly, homogeneous and heterogeneous traffic flow patterns on different intersections are simulated in SUMO to verify the superiority of the proposed distributed deep Q-networks, with the comparison to the fixed-time strategy, local learning and centralized learning algorithms. The simulation study demonstrates that the distributed learning algorithm without a central server shows comparable performance with centralized learning, which is much better than fixed-time strategy and the local learning method in both homogeneous and heterogeneous traffic scenarios. [ABSTRACT FROM AUTHOR]

Object detection is one significant field of computer vision. The imbalance problem exerts negative effects on achieving satisfactory performance. We reveal two sources of imbalance in existing object detection methods. Correspondingly, we propose our methods in terms of the model architecture and optimization target. Different from general object detection benchmarks, the location distribution of objects with different sizes is unbalanced in many practical applications. In addition, the representation information of different categories of objects is unbalanced. In this paper, we propose a location scale equilibrium module to utilize the prior location scale information and generate more balanced feature maps. More appropriate feature maps are selected and merged for different locations. After merging, feature maps become more consistent in terms of representation content, exerting positive effects on the following classification and regression tasks. For the imbalance caused by similar objects, we propose the repulsive loss to strengthen the punishment. Our method will not treat all categories of objects equally since we take the imbalance between them into consideration. With the enhanced supervision, the training will pay more attention to similar objects. Our proposed model is evaluated on the VisDrone benchmark and UAVDT benchmark. Sufficient experiments are conducted. Our model achieves the highest precision on most evaluation metrics, outperforming the other strong models. [ABSTRACT FROM AUTHOR]

Over the last years, advances in deep learning have brought huge developments to the studying of super-resolution reconstruction. However, most super-resolution methods only deal with simply down-sampled sharp images, which may lose efficacy when encountering severe blur. The severe motion blur caused by the rapid movement of an object or the large shake of the lens is common in video captured by cameras. However, existing super-resolution algorithms often bring a large amount of artifacts and are difficult to achieve satisfactory results when reconstructing such blurred video sequences. In this paper, a novel convolutional neural network is proposed that jointly processes video super-resolution (SR) and deblurring (DB) to deal with severe motion blur and recover sharp high-resolution (HR) frames. In particular, a pyramid optical flow module is introduced to estimate the sharp latent image in the blurred frame and generate HR optical flow in a coarse-to-fine way. Then, the frame-recurrent is used to warp the previous SR frame to achieve motion compensation and make full use of the previous sharp features and temporal information to help restoration of the subsequent frames. Next, to further overcome the destruction caused by motion blur in the final reconstruction, a parallel-fusion module was designed to extract and fuse the SR and DB features, finally reconstructing the output frame. Experimental results obtained in this study confirm that, compared with other advanced SR algorithms, the proposed method is both effective and efficient in dealing with videos that contain real motion blur. [ABSTRACT FROM AUTHOR]

Super-resolution (SR) and image enhancement (IE) are two common and important tasks in image processing. Due to poor equipment, illumination, and photographic skills, many low-resolution and low-quality (in terms of color, and contrasts) photos are captured in our daily life. To make the photos more visually pleasing, we usually enlarge and retouch them in two separate steps. In this paper, we propose a new Joint Image Super-resolution and Enhancement Network (JSENet), which is arguably the first end-to-end method based on deep learning for joint SR and IE. Compared with simply conducting these two tasks in parallel or sequentially, JSENet employs the bilateral learning framework to integrate the two tasks together seamlessly. Meanwhile, two lightweight modules are designed for restoring details and generating color transformation coefficients respectively, which ensures that our JSENet can be deployed in real-world applications. Extensive experiments are conducted to demonstrate that JSENet outperforms the state-of-the-art SR-IE methods. [ABSTRACT FROM AUTHOR]

In computer vision, edge and object contour detection is essential for higher-level vision tasks, such as shape matching, visual salience, image segmentation, and object recognition. It has attracted much attention during the past several decades, and many excellent methods have been proposed. In this paper, we make a comprehensive introduction to representative edge and object contour detection methods in the past two decades. Based on the development of these methods, we mainly classify them into two categories: traditional methods and learning-based methods. We further divide traditional methods into local pattern methods, edge grouping methods, active contour models, and bio-inspired methods. Further, we divide learning-based methods into classical learning-based methods and deep learning-based methods. At the same time, we introduce the most popular benchmarks and evaluation measures and quantitatively compare the performances of these promising methods. Moreover, we discuss current challenges in edge and object contour detection and suggest some future trends to bridge gaps with human vision. We believe that this overview will benefit newcomers and promote the development of edge and object contour detection. [ABSTRACT FROM AUTHOR]

Convolutional Neural Networks (CNNs) have won tremendous success in various applications, such as image recognition and natural language processing. Due to the computing-intensive and memory-intensive features of CNN models, it is challenging to deploy them on devices with limited resources and tight power budgets. To address this limitation, pruning, quantization, weights sharing, and other methods have been proposed as efficient solutions to compress CNN models. However, these works usually do not optimize the hardware design when compressing the CNNs. In this paper, we introduce Conv-Inheritance, a hardware-efficient compression method for CNNs by reducing the number of convolution operations to simultaneously reduce the inference time and hardware computing resources on chip of CNN edge devices. We also develop an extended version, called adaptive Conv-Inheritance, to further improve the inference performance by considering the similarity between pixels. We applied our Conv-Inheritance method to compressing various CNN models and performed extensive experiments on different image datasets. Experimental results demonstrate that Conv-Inheritance is both hardware-efficient and time-efficient on edge devices. Besides, Conv-Inheritance is a plug-and-play compression method for CNNs, and we can run Conv-Inheritance on top of other compression methods without any conflicts. [ABSTRACT FROM AUTHOR]

In autonomous vehicle technology, environmental perception is a very important part of the whole system. The robustness and accuracy of the perception determine the performance of the whole system. Compared with the camera, lidar can generate point cloud data which can provide accurate 3D information better. At present, more researches focus on environmental perception through point cloud. However, the point cloud has the characteristics of sparsity, unordered and non-uniform distribution, its processing method is also different from the general image-based method. In this paper, we proposed a method of semantic segmentation of point cloud which is realized by convolutional neural network. Through spherical mapping, the scattered point cloud is transformed into 2D spherical map with dense and uniform distribution, which is convenient for the processing of the network. To reduce the loss of point information, the spatial position, reflection intensity information and angle value of point are preserved during spherical mapping. Our work includes a lightweight convolutional neural network that fuses point cloud and image information. In addition, a spatial module is added to supplement the loss of spatial information and improve the performance of segmentation. Finally, a series of comparative experiments and performance evaluation are carried out, which show that the proposed method is not only much better than the original structure, but also better than another image and lidar fusion network in the segmentation of each category. [ABSTRACT FROM AUTHOR]

Diagnosis of non-tumorous facial pigmentation is an important and difficult issue in the field of dermatology (For example, chloasma and freckles are too similar to distinguish, but they often appear at the same time). In this paper, different from other studies, we used images with nine different forms (views) to comprehensively diagnose these diseases for the first time. In detail, firstly, we collected and published the first multi-view non-tumorous facial pigmentation (MVNFP) dataset provided as the benchmark for classification and segmentation problems: MVNFP, where each face consists of 9 images: 1) Standard Image, 2) Polarization Image, 3) Oil Secretion Image, 4) Sunshine Damage Image, 5) Pigmentation Image, 6) Physiological Aging Image, 7) Sensitive Image, 8) Brown Image, 9) Pigmentation Trend Image. MVNFP has a total of 10,719 images, 5 categories which brings great convenience for the follow-up research. Secondly, to make full use of the informations behind these images described from different views, we use multi-view convolutional neural network to diagnose these indistinguishable diseases. In addition, we introduced the attention mechanisms, and at the same time deeply studied the many influencing factors of multi-view non-tumorous facial pigmentation diagnosis. All experimental results show our method has achieved state-of-the-art results on the MVNFP dataset. Code and MVNFP dataset are released at: https://github.com/sunshiding/MVNFP.git. [ABSTRACT FROM AUTHOR]

The Covid-19 pandemic is the defining global health crisis of our time. Chest X-Rays (CXR) have been an important imaging modality for assisting in the diagnosis and management of hospitalised Covid-19 patients. However, their interpretation is time intensive for radiologists. Accurate computer aided systems can facilitate early diagnosis of Covid-19 and effective triaging. In this paper, we propose a fuzzy logic based deep learning (DL) approach to differentiate between CXR images of patients with Covid-19 pneumonia and with interstitial pneumonias not related to Covid-19. The developed model here, referred to as CovNNet , is used to extract some relevant features from CXR images, combined with fuzzy images generated by a fuzzy edge detection algorithm. Experimental results show that using a combination of CXR and fuzzy features, within a deep learning approach by developing a deep network inputed to a Multilayer Perceptron (MLP), results in a higher classification performance (accuracy rate up to 81%), compared to benchmark deep learning approaches. The approach has been validated through additional datasets which are continously generated due to the spread of the virus and would help triage patients in acute settings. A permutation analysis is carried out, and a simple occlusion methodology for explaining decisions is also proposed. The proposed pipeline can be easily embedded into present clinical decision support systems. [ABSTRACT FROM AUTHOR]

• A CNN for heavy rain removal is proposed based on two networks connected in series. • Novel attention modules are introduced to effectively remove heavy rain effect. • New heavy rain data is proposed with new haze map and various heavy rain models. In this paper, we propose a novel heavy rain removal algorithm using a deep neural network. Unlike most of the existing deraining methods, heavy rain removal is a more challenging task because it is necessary to remove both the rain marks and the haze effects, which are entangled in a complex manner. Motivated by this, we propose a new end-to-end two-stage attention network for single-shot heavy rain removal. The proposed network is connected serially with a removal network and a recovery network, which are based on a newly introduced dilation-wise attention block and skip attention block. Based on these attention techniques, the removal network predicts the heavy rain effect that needs to be removed from a given image, and the recovery network successfully predicts the details that need to be recovered, resulting in a clean image. We also introduce a new realistic RainCityscapes+ dataset, composed of synthesized outdoor images, and demonstrate extensive experiments, the results of which show our approach outperforms the state-of-the-art methods on both real and synthetic datasets quantitatively and qualitatively. [ABSTRACT FROM AUTHOR]

Cervical spondylosis (CS) recognition systems provide regular screening services outside of a hospital and promote early detection and treatment of CS. However, in this paper, we propose a deep learning-based CS recognition system. Concerning the state-of-the-art and state-of-the-practice systems, the innovations of our approaches and algorithms are as follows: First, to elevate the reliance upon the sample number required for training the high-quality model, we reduce sample dimension and find optimal neural network architectures to reduce the number of model parameters to fit. Second, we incorporate multi-stream parallel network architecture search with multi-view feature extraction by converting time series classification into an image classification task. Specifically, five feature extraction methods (time-domain, frequency-domain, time–frequency domain, model-based, nonlinear feature extraction) are firstly utilized to extract features from multiple perspectives and form low-dimensional data set with multi-properties. Third, we reorganize low-dimensional data into image one representing the spatio-temporal relationship of muscle activity pattern. Finally, a multi-stream parallel network architecture search is proposed to use a bypass mechanism for optimal neural network architecture, each of which processes a kind of features mentioned above with an idea of the sparse connection of convolution neural network. The results on the real-world data set show that our CS recognition system achieves the average accuracy of 95.54%, average sensitivity of 99.09%, and average specificity of 90.00%, outperforming the state-of-the-art ones. [ABSTRACT FROM AUTHOR]

Crowd counting and crowd density estimation methods are of great significance in the field of public security. Estimating crowd density and counting from single image or video frame has become an essential part of a computer vision system in various scenarios. In this paper, we comprehensively review the recent research advancement on crowd counting and density estimation. First of all, we introduce the background of crowd counting and crowd density estimation. Second, the traditional crowd counting methods are summarized. Third, we focus on reviewing the crowd counting and crowd density methods based on convolutional neural network (CNN) models. Next, we report and discuss the experimental results of a number of typical methods on benchmark datasets. Finally, we present the promising future directions of crowd counting and crowd density. [ABSTRACT FROM AUTHOR]

This paper proposes an automatic visual inspection method for the fracture detection of clevises in the catenary systems of high-speed railways, using images of catenaries captured by an inspection vehicle. First, the clevises are extracted from the catenary image using a convolutional neural network based algorithm, known as the faster region-based convolutional neural network. Because the structure of catenary systems does not have many variations and the contextual information near a catenary fitting may have strong correlation with its category, the architecture of the original faster region-based convolutional neural network is modified to make use of the contextual information of the regions of interest in the images for object recognition. A crack detection process is then used to recognize the fractures of clevises. To detect the cracks, the edge map of the clevis sub-image is generated using a region-scalable fitting model. Areas where the cracks are most likely to occur are projected from a standard clevis image to the clevis sub-image by shape context matching and affine transformation matrix computation. The cracks are then recognized by calculating the wavelet entropy inside these areas followed by morphological filtering. Experimental results show that the modified faster region-based convolutional neural network architecture achieves better results in clevis extraction than the original architecture as well as some other state-of-art object detection models. The detection is not affected by the scaling, texture and grayscale changes of the clevises caused by the variation of shooting distance, shooting angle and illumination variations. The fractures of the clevises can be accurately and reliably detected using the fracture detection method proposed in this paper and the performance of this visual inspection method meets the strict requirements for catenary system maintenance. [ABSTRACT FROM AUTHOR]

• This paper proposes a CNN-based segmentation framework multiscale dense CNN (MDCNN) to automatically segment cerebral vessel in DSA images. • In the training process, a patch selection strategy is put forward to get enough training data from the limited quantity of annotated cerebrovascular DSA images. • In order to verify the validity of the proposed method, we label a DSA cerebrovascular dataset DCVessel. Accurate cerebrovascular segmentation in Digital Subtraction Angiography (DSA) image, as an indispensable step, can help doctors appropriately estimate the degree of cerebrovascular lesions to avoid misdiagnosis. Because of the complexity of cerebrovascular structure and the uneven distribution of contrast media in DSA, automatic segmentation is a challenging task in clinical diagnosis. In recent years, deep convolutional neural networks (CNN) have outperformed the state-of-art methods and shown great potential for medical image segmentation. This paper proposes a CNN-based segmentation framework Multiscale Dense CNN (MDCNN) to automatically segment cerebral vessel in DSA images. Inspired by U-net, this proposed MDCNN structure is designed as encoder-decoder architecture. Considering that the diameters of blood vessel in cerebrovascular are various, we define a multiscale module to segment cerebral vessel with different diameters. Meanwhile, we redesign the skip connections between encoder and decoder stage to utilize more features from encoder stage. To improve the capability of extracting high-level features, improved dense blocks are introduced. In terms of simplifying parameters, we refer to the idea of deep supervision to make pruning possible. The proposed framework is tested on DCVessel (a DSA cerebrovascular dataset made by our lab). Our proposed method reaches 0.8813, 0.9784, 0.8775, 0.9886, 0.9944 in F1 score, accuracy (Acc), sensitivity (Sen), specificity (Spe) and AUC respectively, outperforming the state-of-art methods. Meanwhile, we evaluate our MDCNN framework on benchmark retinal vessel dataset DRIVE. The promising experiment results demonstrate that proposed MDCNN model is extensible for various vessel segmentation tasks. [ABSTRACT FROM AUTHOR]

• Neurobiological-inspired CNN models accurately predict sound location. • Models of binaural integration in human brainstem simulate human sound localization. • Objective functions minimizing angular or Euclidean distance lead to distinct errors. • Feature representations of shallow layers resemble brainstem neuronal spatial tuning. • Neurobiological-inspired CNN models generate testable predictions for neuroscience. The human brain effortlessly solves the complex computational task of sound localization using a mixture of spatial cues. How the brain performs this task in naturalistic listening environments (e.g. with reverberation) is not well understood. In the present paper, we build on the success of deep neural networks at solving complex and high-dimensional problems [1] to develop goal-driven, neurobiological-inspired convolutional neural network (CNN) models of human spatial hearing. After training, we visualize and quantify feature representations in intermediate layers to gain insights into the representational mechanisms underlying sound location encoding in CNNs. Our results show that neurobiological-inspired CNN models trained on real-life sounds spatialized with human binaural hearing characteristics can accurately predict sound location in the horizontal plane. CNN localization acuity across the azimuth resembles human sound localization acuity, but CNN models outperform human sound localization in the back. Training models with different objective functions - that is, minimizing either Euclidean or angular distance - modulates localization acuity in particular ways. Moreover, different implementations of binaural integration result in unique patterns of localization errors that resemble behavioral observations in humans. Finally, feature representations reveal a gradient of spatial selectivity across network layers, starting with broad spatial representations in early layers and progressing to sparse, highly selective spatial representations in deeper layers. In sum, our results show that neurobiological-inspired CNNs are a valid approach to modeling human spatial hearing. This work paves the way for future studies combining neural network models with empirical measurements of neural activity to unravel the complex computational mechanisms underlying neural sound location encoding in the human auditory pathway. [ABSTRACT FROM AUTHOR]

As MR Rician noise and CT low-dose perfusion noise have a complicated distribution, it is still a challenging task to automatically and accurately remove the noises existing in MR and CT images. Recently, convolutional neural networks have shown favorable performance on image denoising tasks. However, existing methods ignored the hierarchical features extracted from multi-supervision inner layers and estimated the denoised image just by the last single layer, which can not adequately reserve the details of the image. In this paper, we propose a cascaded multi-supervision convolutional neural network named CMSNet to remove the low-dose perfusion noise in CT images and the Rician noise exist in MR images. The CMSNet consists of a multi-supervision network (MSNet) followed with a Refinement network. MSNet is presented to predict the noise constrained by the supervisions from last three convolution layers, which can help acquire more accurate noise prediction and thus obtain the noise-free image. Refinement network is introduced to relief the details lost problem caused by the denoising operation. We employ a progressive training strategy, i.e. , MSNet is first trained independently to predict the preliminary noise and then jointly trained with Refinement network for more accurate noise estimating, which can boost the network performance. Experiments are conducted on clinic abdominal MR and CT images, and the results show that our proposed model achieved a promising performance in terms of unknown noise level, a specific noise level on peak signal to noise ratio (PSNR) and global structure similarity index measurement (SSIM). [ABSTRACT FROM AUTHOR]

The generalization capability of machine learning models, which refers to generalizing the knowledge for an "unseen" domain via learning from one or multiple seen domain(s), is of great importance to develop and deploy machine learning applications in the real-world conditions. Domain Generalization (DG) techniques aim to enhance such generalization capability of machine learning models, where the learnt feature representation and the classifier are two crucial factors to improve generalization and make decisions. In this paper, we propose Discriminative Adversarial Domain Generalization (DADG) with meta-learning-based cross-domain validation. Our proposed framework tries to learn a domain-invariant feature representation from source domains and generalize it to the unseen domains. It contains two main components that work synergistically to build a domain-generalized Deep Neural Network (DNN) model: (i) discriminative adversarial learning, which proactively learns a generalized feature representation on multiple "seen" domains, and (ii) meta-learning based cross domain validation, which simulates train/test domain shift via applying meta-learning techniques in the training process. In the experimental evaluation, a comprehensive comparison has been made among our proposed approach and other existing approaches on three benchmark datasets. The results shown that DADG consistently outperforms a strong baseline DeepAll, and outperforms the other existing DG algorithms in most of the evaluation cases. [ABSTRACT FROM AUTHOR]

As deep CNNs get larger, it becomes more challenging to deploy them on resource-restricted mobile devices. Filter-level pruning is one of the most popular methods to compress deep models for mobile deployment. It prunes unimportant filters in the pre-trained CNN to reduce its storage and computational cost, yielding a smaller and more efficient model. Though having made some progress, most filter-level pruning methods still suffer from at least one of the following two problems: 1) High redundancy: some methods pick out unimportant filters without considering their correlations, and thus many highly correlated filters are not pruned, yielding a model still with high redundancy. 2) Sub-optimal: all existing pruning methods achieve sub-optimal pruning plan because they neglect that a high parameters reduction ratio (PRR) does not always mean a high FLOPs reduction ratio (FRR). In this paper, we propose our customized correlation-based pruning (COP) to solve these problems. In particular, we observe redundant filters through their correlations. Moreover, to achieve the optimal pruning plan, PRR and FRR are considered when evaluating filters' importance. Besides, we also propose a new pruning pipeline, which improves the accuracy of the pruned model. Extensive experiments show that our proposed method has outperformed the state-of-the-art on several popular architectures and datasets. [ABSTRACT FROM AUTHOR]

Skin cancer is one of the most threatening diseases worldwide. However, diagnosing skin cancer correctly is challenging. Recently, deep learning algorithms have emerged to achieve excellent performance in various tasks. Particularly, they have been applied to the skin disease diagnosis tasks. In this paper, we present a review on deep learning methods and their applications in skin disease diagnosis. We first present a brief introduction to skin diseases and image acquisition methods in dermatology, and list several publicly available skin datasets. Then, we introduce the conception of deep learning, and review popular deep learning architectures and popular frameworks facilitating the implementation of deep learning algorithms. Thereafter, performance evaluation metrics are presented. As an important part of this article, we then review the literature involving deep learning methods for skin disease diagnosis from several aspects according to the specific tasks. Additionally, we discuss the challenges faced in the area and suggest possible future research directions. The major purpose of this article is to provide a conceptual and systematically review of the recent works on skin disease diagnosis with deep learning. Given the popularity of deep learning, there remains great challenges in the area, as well as opportunities that we can explore in the future. [ABSTRACT FROM AUTHOR]

To obtain more accurate saliency maps, current methods mainly focus on aggregating multi-level features with structures like U-Net and introducing edge information as auxiliary supervision. Different from the focus of existing methods, in the paper, we study the different roles of semantics and details in saliency detection. The task is decomposed into two parallel sub-tasks: internal semantic estimation and boundary detail prediction, and these sub-goals are optimized simultaneously via explicit constraints. Specifically, we propose a novel semantic and detail collaborative learning network (SDCLNet) for salient object detection. To this end, a backbone network (e.g., VGG-16) with an additional layer is first adopted as a shared encoder to extract features from each image. And then two asymmetric decoders without bells and whistles are designed, in which the semantic decoder generates a coarse semantic mask, and the detail decoder generates a fine-grained object boundary. Finally, a collaborative learning block with some meaningful design adaptively selects the discriminative features to undertake the task of saliency prediction. In this way, both the semantic and detailed information can effectively fused, and the final accurate and consistent saliency maps are generated. SDCLNet is easy to be trained in an end-to-end style, and it does not need any post-processing. Extensive experiments demonstrate the effectiveness and superiority of the proposed method in terms of both subjective visual perception and objective evaluation metrics on six benchmark datasets. SDCLNet achieves a real-time speed of 51 FPS when it is run on one GPU. [ABSTRACT FROM AUTHOR]

Scene classification of high-spatial resolution imagery (HSRI) includes various potential applications in various fields. Recently, deep convolutional neural networks (CNNs) have achieved competitive performance as a result of the powerful capability of feature extraction. In this paper, we propose a multi-level improved circle pooling (MICP) method with the pre-trained CNN-based model to enhance the discriminative power of CNN activations for scene classification. Specifically, an improved pooling strategy is presented to generate annular subregions without padding operations in traditional concentric circle pooling. Then, we extract the pooling features in these subregions under different levels and build a holistic representation by fusing these multi-level features. MICP is an effective and simple strategy enriching rotation insensitivity and multiscale spatial information. According to the experiments conducted on three challenging HRSI scene data sets, the proposed pooling method achieves similar or better classification accuracy compared to the other CNN-based scene classification methods. Moreover, a comprehensive discussion regarding the effect of data augmentation reveals that the proposed method can enhance the rotation insensitivity of CNNs for the HRSI scene classification. [ABSTRACT FROM AUTHOR]

Deep neural networks have been applied in many applications exhibiting extraordinary abilities in the field of computer vision. However, complex network architectures challenge efficient real-time deployment and require significant computation resources and energy costs. These challenges can be overcome through optimizations such as network compression. Network compression can often be realized with little loss of accuracy. In some cases accuracy may even improve. This paper provides a survey on two types of network compression: pruning and quantization. Pruning can be categorized as static if it is performed offline or dynamic if it is performed at run-time. We compare pruning techniques and describe criteria used to remove redundant computations. We discuss trade-offs in element-wise, channel-wise, shape-wise, filter-wise, layer-wise and even network-wise pruning. Quantization reduces computations by reducing the precision of the datatype. Weights, biases, and activations may be quantized typically to 8-bit integers although lower bit width implementations are also discussed including binary neural networks. Both pruning and quantization can be used independently or combined. We compare current techniques, analyze their strengths and weaknesses, present compressed network accuracy results on a number of frameworks, and provide practical guidance for compressing networks. [ABSTRACT FROM AUTHOR]

Domain adaptation has demonstrated promising performance to learn models in new environments. It aims to transfer the knowledge learned in labeled source domain to a new target domain, avoiding expensive efforts of label annotation. Many popular methods attempt to assign pseudo labels to unlabeled target samples and train the models as if they are true labels. However, the pseudo labels contain inevitable noises while their training strategies would enlarge and accumulate the errors, so that the model easily overfits to noisy data. Instead of focusing on assigning correct pseudo labels, in this paper, an unsupervised domain adaptation approach is proposed by preventing the model from overfitting false samples. Two different types of restrictions are considered on the training data to leverage the intra-class centralization and inter-class normalization. Each training sample and individual category are equally treated, where each point has the same opportunity to contribute to the model learning. A novel class restriction loss is proposed, which is further integrated into a teacher-student architecture, where the outputs from the teacher are treated as the pseudo labels for the student. The whole framework is trained in an end-to-end manner. Extensive experiments conducted on several image classification benchmarks demonstrate that the proposed method can significantly improve the performance, which outperforms the state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Convolutional neural networks (CNNs) have been widely used in end-to-end stereo matching networks in recent years. However, most stereo networks are not robust to variations in the environment and thus are difficult to be extended to practical applications. In this paper, an inherent factor that hinders the adaptive performance of stereo matching networks is first determined. Then we propose a domain-adaptive feature extractor (DAFE) that can extract the features of images on different domains and a feature normalization method to reduce the variances of features in across-domain situations. Moreover, the influence of various modules on the performance of the domain-adaptive network (DANet) is investigated. When trained on Sceneflow data and generalized to the real test sets, the method performs significantly better than state-of-the-art models and even better than some latest disparity networks fine-tuned on the target domain. [ABSTRACT FROM AUTHOR]

• Developing a new recursive convolutional layer (RCL) able to stop the iteration when the hidden state stabilize. • First network to have different depth for different batch of samples. • A comprehensive study on the best locations and numbers of RCL in a multilayer network called C-FRPN. To overcome problems with the design of large networks, particularly with respect to the depth of the network, this paper presents a new model of convolutional neural networks (CNN) which features fully recursive convolutional layers (RCLs). An RCL is a generalization of the classic one-stage feedforward convolutional layer (CL) to fully direct feedback connections between the outputs of the CL and its inputs. A traditional deep CNN consisting of many CLs, can then be generalized to include some CLs, and some RCLs in the intermediate stages. We call the corresponding network a Convolutional Neural Network with Fully Recursive Perceptron Network (C-FRPN). Through an analysis of results obtained from applications of the C-FRPN to three benchmark image classification datasets: CIFAR-10, SVHN, ISIC, it is found that (i) in general, the performance of a C-FRPN, even with only one RCL, is better than the performance of the corresponding deep CNN with all CLs, under the constraint of having the same number of unknown parameters; (ii) the performance of the C-FRPN varies with respect to (a) where the RCLs are located, and (b) the number of RCLs in the C-FRPN; and, (iii) the effectiveness of the RCLs depends on the size of the training dataset. The results suggest that: (a) it is advisable to use RCLs particularly when training very large sets of data, (b) it is best to prioritize placement of RCLs close to the input layer of the C-FRPN, and (c) it is advisable to increase the number of RCLs as long as the training dataset can sustain without overfitting being observed. [ABSTRACT FROM AUTHOR]