Search

Image dehazing is always a hot topic in the field of computer vision since haze has significant impact on the imaging quality of camera. Therefore, many image dehazing methods have been proposed for the past decades. To help researchers who are new to this field quickly figure out the development history as well as the current status of image dehazing, this review analyzes some representative dehazing methods, evaluates their advantages and disadvantages, and most importantly, points out the best dehazing method from different viewpoints. A large quantity of experiments show that AECR-Net can be generally considered to be the best dehazing algorithm and Tarel method can be regarded as the best real-time dehazing method. Besides, the mainstream benchmark, metrics, challenges and opportunities for image dehazing are also discussed in this review. [ABSTRACT FROM AUTHOR]

Most existing Deep Reinforcement Learning (DRL) algorithms solely apply to discrete action or continuous action spaces. However, the agent often has both continuous and discrete action space, named hybrid action space. This paper proposes an action-decoupled algorithm for hybrid action space. Specifically, the hybrid action is decoupled, and then the original agent in the hybrid action space is abstracted into two agents. Each agent contains only discrete or continuous action space. The discrete and continuous actions are independent of each other to be executed simultaneously. We use the Soft Actor-Critic (SAC) algorithm as the optimization method and name our proposed algorithm Action Decoupled SAC (AD-SAC). We handle multi-agent problems using a framework of Centralized Training Decentralized Execution (CTDE) and then reduce the concatenation of partial agent observations to avoid the interference of redundant observations. We design a hybrid action space environment for Unmanned Aerial Vehicles (UAVs) path planning and gimbal scanning using AirSim. The results show that our algorithm has better convergence and robustness than the discretization, relaxation, and the Parametrized Deep Q-Networks Learning (P-DQN) algorithms. Finally, we carried out a Hardware in the Loop (HITL) simulation experiment based on Pixhawk to verify the feasibility of our algorithm. [ABSTRACT FROM AUTHOR]

Recently, a massive number of deep learning-based approaches have been successfully applied to various remote sensing image (RSI) recognition tasks. However, most existing advances of deep learning methods in the RSI field heavily rely on the features extracted by the manually designed backbone network, which severely hinders the potential of deep learning models due to the complexity of RSI and the limitation of prior knowledge. In this paper, we research a new design paradigm for the backbone architecture in RSI recognition tasks, including scene classification, land-cover classification and object detection. A novel one-shot architecture search framework based on a weight-sharing strategy and an evolutionary algorithm is proposed, called RSBNet, which consists of three stages: Firstly, a supernet constructed in a layer-wise search space is pretrained on a self-assembled large-scale RSI dataset based on an ensemble single-path training strategy. Next, the pre-trained supernet is equipped with different recognition heads through the switchable recognition module and respectively fine-tuned on the target dataset to obtain task-specific supernet. Finally, we search for the optimal backbone architecture for different recognition tasks based on the evolutionary algorithm without any network training. Extensive experiments have been conducted on five benchmark datasets for different recognition tasks, the results show the effectiveness of the proposed search paradigm and demonstrate that the searched backbone is able to flexibly adapt different RSI recognition tasks and achieve impressive performance. [ABSTRACT FROM AUTHOR]

This paper presents an effective weakly supervised learning algorithm LLP-AAE to leverage the adversarial autoencoder (AAE) for learning from label proportions (LLP), in which only the bag-level proportional information is available. Our LLP-AAE utilizes an autoencoder backbone and performs adversarial training in latent space to match the aggregated posterior distribution of hidden coding with the prior distributions. In this way, apart from the reconstruction task, the encoder is also dedicated to producing fake samples, in order to deceive discriminators as far as possible. Ultimately, the encoder is employed as a competent label predictor for unseen data. In addition to the LLP classifier, our model can also achieve controllable samples generation by feeding the decoder with gradually changing latent code, which is proven to be useful for a better LLP performance. We also provide a panoramic explanation for LLP-AAE by regarding the LLP problem as an alternative learning procedure between proportion-based pseudo label generation and discriminative reconstruction. Experiments on six benchmark image datasets demonstrate the advantage of our method both in style manipulation with the latent feature representation and comparable multi-class LLP performance with the state-of-the-art models. [ABSTRACT FROM AUTHOR]

• This is a comprehensive review of Transformer for Skeleton-based Action Recognition. • This paper proposes a new taxonomy of transformer-style techniques. • This survey aims to help researchers systematically select promising future directions. Skeleton-based action recognition has rapidly become one of the most popular and essential research topics in computer vision. The task is to analyze the characteristics of human joints and accurately classify their behaviors through deep learning technology. Skeleton provides numerous unique advantages over other data modalities, such as robustness, compactness, noise immunity, etc. In particular, the skeleton modality is extremely lightweight, which is especially beneficial for deep learning research in low-resource environments. Due to the non-European nature of skeleton data, Graph Convolution Network (GCN) has become mainstream in the past few years, leveraging the benefits of processing topological information. However, with the explosive development of transformer methods in natural language processing and computer vision, many works have applied transformer into the field of skeleton action recognition, breaking the accuracy monopoly of GCN. Therefore, we conduct a survey using transformer method for skeleton-based action recognition, forming of a taxonomy on existing works. This paper gives a comprehensive overview of the recent transformer techniques for skeleton action recognition, proposes a taxonomy of transformer-style techniques for action recognition, conducts a detailed study on benchmark datasets, compares the algorithm accuracy of standard methods, and finally discusses the future research directions and trends. To the best of our knowledge, this study is the first to describe skeleton-based action recognition techniques in the style of transformers and to suggest novel recognition taxonomies in a review. We are confident that Transformer-based action recognition technology will become mainstream in the near future, so this survey aims to help researchers systematically learn core tasks, select appropriate datasets, understand current challenges, and select promising future directions. [ABSTRACT FROM AUTHOR]

Automatic identity recognition from ear images is an active research topic in the biometric community. The ability to secretly acquire images of the ear remotely and the stability of the ear shape over time make this technology a promising alternative for surveillance, authentication, and forensic applications. In recent years, significant research has been conducted in this area. Nevertheless, challenges remain that limit the commercial use of this technology. Several phases of the ear recognition system have been studied in the literature, from ear detection, normalization, and feature extraction to classification. This paper reviews the most recent methods used to describe and classify biometric features of the ear. We propose a first taxonomy to group existing approaches to ear recognition, including 2D, 3D, and combined 2D and 3D methods, as well as an overview of historical advances in this field. It is well known that data and algorithms are the essential components in biometrics, particularly in-ear recognition. However, early ear recognition datasets were very limited and collected in laboratory with controlled environments. With the wider use of deep neural networks, a considerable amount of training data has become necessary if acceptable ear recognition performance is to be achieved. As a consequence, current ear recognition datasets have increased significantly in size. This paper gives an overview of the chronological evolution of ear recognition datasets and compares the performance of conventional vs. deep learning methods on several datasets. We proposed a second taxonomy to classify the existing databases, including 2D, 3D, and video ear datasets. Finally, some open challenges and trends are debated for future research. [ABSTRACT FROM AUTHOR]

This paper presents an effective method for strengthening the discriminative ability of high-level deep features by enhancing and aggregating discriminative part-level features for the fine-grained vehicle recognition task. In general, the task of visual recognition concentrates more on the visual differences at the object level. However, for fine-grained object recognition, the visual differences between target objects typically exist in local discriminative areas, so it is more concerned about extracting fine-grained features from these part regions. In this context, we propose solving this issue with a novel feature extraction method from two perspectives: the generation of more feature descriptors of part regions through the learning process of deep networks and the aggregation of part-level discriminative features. This approach is designed to improve the backbone networks to generate finer-level part features through a part-level feature enhancement module and to investigate the intrinsic part-level features of the backbone networks with the help of a feature aggregation module. The enhancement module efficiently finds the finer features highly correlated to the part regions. Then the feature aggregation module builds correlations of similar part features through feature grouping and fusion. Moreover, our proposed method does not require additional parts annotations and achieves comparable performance on two widely-used benchmarks for recognizing fine-grained vehicle types. Experimental results and explainable visualizations demonstrate the effectiveness of the proposed method. [ABSTRACT FROM AUTHOR]

Mammography is an important screening criterion for breast cancer, one of the major diseases causing numerous deaths among female patients. Meanwhile, manual diagnosis of mammography is a time-consuming and labor-consuming job. Mammogram classification based on deep learning plays a vital role in computer-aided diagnosis (CAD) systems to mitigate the pressure on physicians. This paper proposes a learning-based multi-view mammogram classification model that captures long-distance dependence and extracts features of multiple receptive fields. Our model considers global and local features of mammography images using Transformer for global features and the proposed multiplex convolutions module for local features. We evaluate our proposed method on a dataset of mammography images obtained from a hospital in China. The proposed method achieves 90.57% accuracy and 94.86% AUC in benign or malignant classification tasks and outperforms other advanced methods for mammogram classification. It is worth noting that the proposed method only requires image-level labels and acts on the whole raw mammogram, which has clinical significance. [ABSTRACT FROM AUTHOR]

• Entangled representations bringing a negative influence on the performance of FAS. • Disentangle irrelevant and critical representations on face anti-spoofing task. • Refine critical representations by suppressing irrelevant representations. • Separate the spoof patterns and generate the live counterpart from spoof face. Face recognition systems have been widely applied in security-related areas of our daily life. However, they are vulnerable to face spoofing attacks. Specifically, an attacker can fool a face recognition system into making false decisions, by presenting spoof face information (such as printed photos, replayed videos, etc.), rather than live face, to the face recognition system. Therefore, Face Anti-Spoofing (FAS) is critical for the security operation of a face recognition system. Deep learning-based FAS approaches show the best performance among existing FAS approaches. The basic idea of deep learning-based FAS approaches is to learn statistical representations capable of distinguishing spoof faces from live ones, and then leverage the learned representations for live and spoof face classifications. Therefore, the learned representations play a key role in the performance of FAS. However, most existing approaches learn representations from representation-entangled spaces, in which critical and irrelevant representations for live and spoof face classifications are entangled with each other, thereby bringing a negative influence on the performance of a FAS system. To address the issue, we introduced a Twin Autoencoder Disentanglement (TAD) framework. Our TAD framework utilizes adversarial learning and a reconstruction strategy to disentangle both critical and irrelevant representations into two mutually independent representation spaces. In addition, to further suppress irrelevant representations that may remain in the critical representation space, we design a multi-branch supervision architecture (MSA) and embed it into TAD. MSA achieves the goal via imposing depth supervision and pattern supervision to the critical representation space. i.e., learning spatial representation (face depth information) and texture representation (face spoof pattern information). Experimental results on four typical public datasets, OULU-NPU, SiW, Replay-Attack, and CASIA-MFSD, demonstrate that our proposed TAD approach successfully disentangles critical and irrelevant representations, and the two disentangled representations are more interpretable than state-of-the-art FAS methods. The codes are available at https://github.com/TAD-FAS/TAD. [ABSTRACT FROM AUTHOR]

Recently, many efforts have been concentrated on land use land cover (LULC) classification due to rapid urbanization, environmental pollution, agriculture drought, frequent floods, and climate change. However, various aspects have attracted hyperspectral imaging due to there being informative discriminative features, such as spectral-spatial features. To this end, this paper is a comprehensive and systematic review of LULC classification using hyperspectral images by reviewing four significant research investigations. Moreover, the four investigations have addressed the following points: (1) the main components of the hyperspectral imaging, the modes of hyperspectral imaging with data acquisition methods, and the intrinsic differences between hyperspectral image and multispectral image, (2) the role of machine learning in LULC classification, and the standard deep learning methods: Convolution Neural Network (CNN), Stacked Autoencoder (SAE), Deep Belief Network (DBN), Recurrent Neural Network (RNN), and Generative Adversarial Network (GAN), (3) the standard benchmark hyperspectral datasets and the evaluation criteria, (4) the main challenges of LULC classification with the possible solutions for the limited training samples issue, the promising future directions, and finally the recent applications for LULC classification. [ABSTRACT FROM AUTHOR]

The performance of deep learning-based methods depends mainly on the availability of large-scale labeled learning data. However, obtaining precisely annotated examples is challenging in the medical domain. Although some semi-supervised deep learning methods have been proposed to train models with fewer labels, only a few studies have focused on the uncertainty caused by the low quality of the images and the lack of annotations. This paper addresses the above issues using Dempster-Shafer theory and deep learning: 1) a semi-supervised learning algorithm is proposed based on an image transformation strategy; 2) a probabilistic deep neural network and an evidential neural network are used in parallel to provide two sources of segmentation evidence; 3) Dempster's rule is used to combine the two pieces of evidence and reach a final segmentation result. Results from a series of experiments on the BraTS2019 brain tumor dataset show that our framework achieves promising results when only some training data are labeled. [ABSTRACT FROM AUTHOR]

[Display omitted] The contextual information is critical for various computer vision tasks, previous works commonly design plug-and-play modules and structural losses to effectively extract and aggregate the global context. These methods utilize fine-label to optimize the model but ignore that fine-trained features are also precious training resources, which can introduce preferable distribution to hard pixels (i.e., misclassified pixels). Inspired by contrastive learning in unsupervised paradigm, we apply the contrastive loss in a supervised manner and re-design the loss function to cast off the stereotype of unsupervised learning (e.g., imbalance of positives and negatives, confusion of anchors computing). To this end, we propose P ositive- N egative E qual contrastive loss (PNE loss), which increases the latent impact of positive embedding on the anchor and treats the positive as well as negative sample pairs equally. The PNE loss can be directly plugged right into existing semantic segmentation frameworks and leads to excellent performance with neglectable extra computational costs. We utilize a number of classic segmentation methods (e.g., DeepLabV3, HRNetV2, OCRNet, UperNet) and backbone (e.g., ResNet, HRNet, Swin Transformer) to conduct comprehensive experiments and achieve state-of-the-art performance on three benchmark datasets (e.g., Cityscapes, COCO-Stuff and ADE20K). Our code will be publicly available at https://github.com/jingw193/PNE_Loss. [ABSTRACT FROM AUTHOR]

Previous studies in facial age estimation can achieve promising performance when the training and test sets have a similar condition. However, these methods often fail to maintain performance and show significant degradation when encountering unseen domains. Therefore, we propose a novel method named Deep Domain-Invariant Learning (DDIL) to solve the Out-of-Distribution (OOD) generalization problem for facial age estimation. The proposed DDIL consists of the domain-invariant and style-invariant modules. The former extracts domain-specific features and trains a domain-invariant feature extractor by reducing the covariance discrepancy among features from different domains, while the latter leverages style randomization to overcome CNN's induction bias towards styles. Consolidating these two modules, our DDIL can effectively decrease the influence of domain discrepancy. Extensive experiments on multiple age benchmark datasets under the Leave-One-Domain-Out Cross-Validation setting indicate superior performance in tackling age estimation generalization. [ABSTRACT FROM AUTHOR]

For the semantic segmentation task, spatial information and the receptive field are indispensable. For semantic segmentation to be practically applicable, it must have real-time inference speed. However, most of today's methods almost choose to compromise the spatial resolution and low-level detail information, which leads to a significant decrease in accuracy. In this paper, we propose a new architecture based on Bilateral Segmentation Network (BiSeNet) called BiSeNet V3. It introduces a new feature refinement module to optimize the feature map and a feature fusion module to combine the features efficiently. An attention mechanism is introduced to assist the model in capturing contextual information. We also use edge detection to enhance features for boundaries. Extensive experiments on the Cityscapes dataset show that our proposed approach achieves an excellent performance between segmentation accuracy and inference speed. Specifically, for a 768 × 1536 input, BiSeNet V3 achieved 79.0% mIoU on the Cityscapes test set with a speed of 93.8 FPS on an NVIDIA GTX 1080Ti. For a 720 × 960 input, BiSeNet V3 achieved 76.6% mIoU on the CamVid dataset with a speed of 147.6 FPS on an NVIDIA GTX 1080Ti. The result is significantly better than the other methods. [ABSTRACT FROM AUTHOR]

Most recently, deep learning-based visual detection has attracted rapidly increasing attention paid to marine organisms, thereby expecting to significantly benefit ocean ecology. Suffering from underwater visual degradation including low contrast, color distortion and blur, etc. , both advances and challenges on visual detection of marine organisms (VDMO) co-exist in the literature. In this survey, deep learning-based VDMO techniques are comprehensively revisited from a systematic viewpoint covering advances in underwater image preprocessing, deep learning-based detection approaches, benchmark dataset and intensively quantitative comparisons. Furthermore, in terms of inherent features of marine organisms and complexity of underwater visual environments, underlying challenges are unfolded in depth. Such a self-contained survey is expected to exploit potential breakthroughs and explore probable trends in deep learning-based VDMO techniques. [ABSTRACT FROM AUTHOR]

Pedestrian Attribute Recognition (PAR) is currently an emerging research topic in the field of video surveillance. For PAR, it usually needs to analyze dozens of attributes simultaneously, e.g., age, gender and Clothing type. However, different attributes may focus on different image regions, which makes it difficult to concurrently extract exhaustive features over all attributes. Moreover, some of these attributes are highly correlated, which is the other challenge for pedestrian attribute recognition. To remedy the aforementioned two issues, we propose two novel modules, namely Attribute Localization Module (ALM) and Attribute Correlation Module (ACM). For ALM, it is constructed based on a multi-stream architecture with each stream processing a specific attribute individually. More specifically, an attention mechanism is employed to discover and enhance the attribute-related features while suppressing less important regions. For ACM, the Transformer structure is employed to effectively explore the correlations among different attributes. In particular, we place the Transformer blocks behind the ALM module, with regarding each attribute-specific feature as an input token. The ALM and ACM modules focus on different aspects, which exploits the interrelated and complementary information. We combine the proposed modules to form a unified network with Exploring Attribute Localization and Correlation (abbreviated as EALC). Our approach is validated on five large-scale pedestrian attribute datasets, including PETA, RAP, PA-100 K, Market-1501 and Duke attribute datasets. Experiments demonstrate the effectiveness and advancement of the proposed EALC. [ABSTRACT FROM AUTHOR]

This paper presents a federated learning approach based on utilizing computational resources of the IoT edge devices for training deep neural networks. In this approach, the edge devices and the cloud server collaborate in the training phase while preserving the privacy of the edge device data. Owing to the limited computational power and resources available to the edge devices, instead of the original neural network (NN), we suggest to use a smaller NN generated using a proposed heuristic method. In the proposed approach, the smaller model, which is trained on the edge device, is generated from the main NN model. By the exploiting Knowledge Distillation (K D) approach, the learned knowledge in the server and the edge devices can be exchanged, leading to lower required computation on the server and preserving data privacy of the edge devices. Also, to reduce the knowledge transfer overhead on the communication links between the server and the edge devices, a method for selecting the most valuable data to transfer the knowledge is introduced. The effectiveness of this method is assessed by comparing it to state-of-the-art methods. The results show that the proposed method lowers the communication traffic by up to 250 × and increases the learning accuracy by an average of 8.9 % in the cloud compared to the prior K D -based distributed training approaches in CIFAR-10 dataset. [ABSTRACT FROM AUTHOR]

The inception and rapid growth of the Web, social media, and other online forums have resulted in the continuous and rapid generation of opinionated textual data. Several real-world applications have been focusing on determining the sentiments expressed in these data. Owing to the multilinguistic nature of the generated data, there exists an increasing need to perform sentiment analysis on data in diverse languages. This study presents an overview of the methods used to perform sentiment analysis across languages. We primarily focus on multilingual and cross-lingual approaches. This survey covers the early approaches and current advancements that employ machine learning and deep learning models. We categorize these methods and techniques and provide new research directions. Our findings reveal that deep learning techniques have been widely used in both approaches and yield the best results. Additionally, the scarcity of multilingual annotated datasets limits the progress of multilingual and cross-lingual sentiment analyses, and therefore increases the complexity in comparing these techniques and determining the ones with the best performance. [ABSTRACT FROM AUTHOR]

• Identifies the current trends in deep learning based object detection task that can be run on devices with limited resources, by performing comprehensive analysis of existing literature. • Provides an insight of the techniques used to make the object detection models applicable on small devices. • Provides differentiation based on the input image types and applications for the object detection task • Guides other researchers willing to work on deep learning based object detection on constrained devices by identifying several research gaps and providing self-explanatory visualizations. Deep learning models are widely being employed for object detection due to their high performance. However, the majority of applications that require object detection are functioning on resource-constrained edge devices. In the present era, there is a need for deep learning-based object detectors that are lightweight and perform well on these constrained edge devices. Objective: The research aims to identify current trends in resource-constrained applications for deep learning-based object detectors in terms of the technique used to create the model, the type of input image involved, the type of device used, and the type of application addressed by the model. Method: To achieve the objective of our research, a systematic literature review was carried out that yielded 167 studies. The models or techniques employed in the studies were grouped to better understand the research problem at hand. This review carefully reports every decision and provides many visualizations of the final studies in order to draw clear conclusions. Conclusion: The conclusion discussed the gaps, possibilities, and future perspectives discovered throughout the research process, implying that this field of study has grown profoundly in the last decade. [ABSTRACT FROM AUTHOR]

Reconstructing natural image from its corresponding compressive sensing (CS) measurements is an ill-posed problem. Learning accurate prior of desirable image is essential to solve this inverse problem in high quality, especially for CS reconstruction from noisy measurements. The existing learning-based methods cannot effectively simulate whole potential noise in the testing data during external learning, and cannot effectively exploit the information from internal testing data. In this paper, we present an effective convolutional neural network (CNN) based method for CS reconstruction from noisy measurements, which learns the deep prior from an external dataset and internal noisy testing data with Stein's unbiased risk estimator (SURE). Specifically, we first pre-train an arbitrary CNN for CS reconstruction with an external dataset to learn a common prior. Then, we utilize meta learning to find a generic initial parameter that is suitable for fast internal learning and adaption to various noisy measurements. Finally, we customize the learned network to learn a specific prior for each internal testing data under Gaussian noise or more general mixed Poisson-Gaussian noise. Experimental results show that the proposed method outperforms the state-of-the-art methods under both comprehensively quantitative metrics and perceptive quality. [ABSTRACT FROM AUTHOR]

Filter pruning has drawn extensive attention due to its advantage in reducing computational costs and memory requirements of deep convolutional neural networks. However, most existing methods only prune filters based on their intrinsic properties or spatial feature maps, ignoring the correlation between filters. In this paper, we suggest the correlation is valuable and consider it from a novel view: the frequency domain. Specifically, we first transfer features to the frequency domain by Discrete Cosine Transform (DCT). Then, for each feature map, we compute a uniqueness score, which measures its probability of being replaced by others. This way allows to prune the filters corresponding to the low-uniqueness maps without significant performance degradation. Compared to the methods focusing on intrinsic properties, our proposed method introduces a more comprehensive criterion to prune filters, further improving the network compactness while preserving good performance. In addition, our method is more robust against noise than the spatial ones since the critical clues for pruning are more concentrated after DCT. Experimental results demonstrate the superiority of our method. To be specific, our method outperforms the baseline ResNet-56 by 0.38 % on CIFAR-10 while reducing the floating-point operations (FLOPs) by 47.4 %. In addition, a consistent improvement can be observed when pruning the baseline ResNet-110: 0.23 % performance increase and up to 71 % FLOPs drop. Finally, on ImageNet, our method reduces the FLOPs of the baseline ResNet-50 by 48.7 % with only 0.32 % accuracy loss. [ABSTRACT FROM AUTHOR]

The increasing availability of remote sensing data allows dealing with spatial-spectral limitations by means of pan-sharpening methods. However, fusing inter-sensor data poses important challenges, in terms of resolution differences, sensor-dependent deformations and ground-truth data availability, that demand more accurate pan-sharpening solutions. In response, this paper proposes a novel deep learning-based pan-sharpening model which is termed as the double-U network for self-supervised pan-sharpening (W-NetPan). In more details, the proposed architecture adopts an innovative W-shape that integrates two U-Net segments which sequentially work for spatially matching and fusing inter-sensor multi-modal data. In this way, a synergic effect is produced where the first segment resolves inter-sensor deviations while stimulating the second one to achieve a more accurate data fusion. Additionally, a joint loss formulation is proposed for effectively training the proposed model without external data supervision. The experimental comparison, conducted over four coupled Sentinel-2 and Sentinel-3 datasets, reveals the advantages of W-NetPan with respect to several of the most important state-of-the-art pan-sharpening methods available in the literature. The codes related to this paper will be available at https://github.com/rufernan/WNetPan. [ABSTRACT FROM AUTHOR]

Although reinforcement learning (RL) can solve complex tasks after training, it's difficult to extend trained agents to target environments with perturbations. This lack of generalization ability has caused the dilemma of large-scale application of RL. Rich neural networks have brought huge improvements to deep learning, but these aren't adapted to RL and even bring negative effects. Therefore, RL algorithms have few choices of neural network, which greatly limits the representation and generalization ability of RL. To overcome these limitations, we propose a deep semi-dense compression network (DSCN) to improve RL generalization ability. First, we perform a structural extension on a general network model for RL. Then based on the information theory, we propose a semi-dense connection to enhance the neural information flow (NIF) of initial features. Finally, drawing on the ideas of course learning, we propose a channel compression approach to filter the redundant information of the deep network. In addition, we innovatively extend the experimental environment and evaluation metrics of the existing platform, which can fully evaluate the performance of DSCN. The experimental results show that our model achieves stable and significant generalization performance improvement. [ABSTRACT FROM AUTHOR]

Fake news, which can be defined as intentionally and verifiably false news, has a strong influence on critical aspects of our society. Manual fact-checking is a widely adopted approach used to counteract the negative effects of fake news spreading. However, manual fact-checking is not sufficient when analysing the huge volume of newly created information. Moreover, the number of labeled datasets is limited, humans are not particularly reliable labelers and databases are mostly in English and focused on political news. To solve these issues state-of-the-art machine learning models have been used to automatically identify fake news. However, the high amount of models and the heterogeneity of features used in literature often represents a boundary for researchers trying to improve model performances. For this reason, in this systematic review, a taxonomy of machine learning and deep learning models and features adopted in Content-Based Fake News Detection is proposed and their performance is compared over the analysed works. To our knowledge, our contribution is the first attempt at identifying, on average, the best-performing models and features over multiple datasets/topics tested in all the reviewed works. Finally, challenges and opportunities in this research field are described with the aim of indicating areas where further research is needed. [ABSTRACT FROM AUTHOR]

Identifying influential nodes in social networks is a fundamental task. Due to the development of Graph Neural Networks, Graph Convolution Network (GCN) based model has been introduced to solve this problem. Compared to traditional methods, the existing GCN-based models are more accurate in identifying influential nodes because they can better aggregate the multi-dimension features. However, the GCN-based method treats this problem as a binary classification task rather than a regression task, making it less practical. To make the GCN-based model more practical, we treat identifying influential nodes as a regression task. Moreover, when aggregating neighbor features, GCN ignores the difference in neighbor importance, which will affect the prediction performance of the GCN-based models. This paper proposes a graph multi-head attention regression model to address these problems. Vast experiments on twelve real-world social networks demonstrate that the proposed model significantly outperforms baseline methods. To the best of our knowledge, this is the first work to introduce the multi-head attention mechanism to identify influential nodes in social networks. [ABSTRACT FROM AUTHOR]

• We focus on the data dilemma encountered in the field of human activity understanding, solve practical application problems from a new perspective, and use the proposed method to reduce the model's dependence on data. • We construct a human physical activity dataset containing 10 categories Human SA-10 for use in human activity understanding research. • A Core-Weight Entropy data information evaluation method that can be applied to human behavior recognition tasks is proposed. On Human SA-10, our method can reduce data usage by 50%. Compared to other methods, this method achieved state-of-the-art performance. • In addition, we propose a new unlabeled data redundancy information removal module, which effectively avoids introducing similar data into the training set. With the development of computer vision, the research on human activity understanding has been greatly promoted. The recognition algorithm based on vision transformer has made some achievements in a large number of computer vision tasks, but it still needs to be driven by a large amount of data. How to get rid of the constraints of large amounts of data is crucial for human behavior recognition based on vision transformer. This paper focuses on solving the dilemma of big data, and tries to achieve a high-performance model through a small amount of high information human activity data. The advantage of our work is that by studying feature distribution, we proposed a core weight entropy data information evaluation method for obtaining high information data, and through redundant information elimination strategy, we can avoid introducing similar data. A large number of experimental results show the effectiveness of the proposed method. Compared with existing methods, our method reduces the data consumption by 5% to 30%, and can achieve the performance of using only 50% of 100% data. More importantly, the data our method selected has no redundancy, which is not available in other methods. In addition, we carried out a large number of ablation experiments to prove the rationality of the method. The work of this paper solves the challenge of relying on a large amount of data when using the visual converter to recognize human behavior, which is of practical significance for realizing efficient human activity understanding research with low data. [ABSTRACT FROM AUTHOR]

• We propose a new graph convolution network with subgraph embedding. • It extracts local information with subgraphs partitioned by Girvan Newman algorithm. • Experiments confirm the superiority of predicting mutagenicity in aromatic hydrocarbons. • The accuracy is verified with comparison of nine SOTA deep learning models. • The proposed method prevents about 15 %p of GCN's information dilution. An aromatic hydrocarbon refers to an organic material having a carbon ring such as benzene and a functional group in the carbon ring. As the industry develops, natural pollution becomes harsh, new compounds emerge, and the exposure to aromatic hydrocarbons is continuously increasing. Predicting mutagenicity is one of the crucial issues in reducing the risk because these organisms may have properties that penetrate the DNA of living things to cause mutations. Recently, the accuracy of mutation prediction has improved due to the power of deep learning. However, most conventional methods do not consider the characteristics of molecular aromatic hydrocarbons, which dilutes local information and results in a severe deterioration of the prediction performance. In this paper, we propose a method of exploiting subgraph convolution neural networks that enables the extraction of local information of a graph by partitioning it to maintain the detailed information. For extracting the features of molecules, we use the Girvan Newman algorithm to partition the graph according to the carbon ring and functional group and obtain the embedding vectors of the subgraphs as well as the original graph with graph convolution network (GCN). The embedding vectors are combined to represent the whole graph information and predict mutagenicity. Experiments with MUTAG, NCI1 and NCI109, datasets for predicting mutagenicity of molecules in graph structure, confirm that we successfully segment carbon rings and functional groups from molecular graphs and predict mutations using the partitioned graphs, leading to a 2 %p performance improvement. In addition, the proposed method has prevented about 15 %p of information dilution in GCN, and an analysis of the latent space of graphs reveals that the subgraphs extracted maintain the local information appropriately. [ABSTRACT FROM AUTHOR]

Accurate and efficient object pose estimation holds the indispensable part of virtual/augmented reality (VR/AR) and many other applications. While previous works focus on directly regressing 6D pose from RGB and depth image and thus suffer from the non-linearity of rotation space, we propose an iterative 3D keypoints voting network, named as KVNet. Specifically, our method decouples the pose into separate translation and rotation branch, both estimated by Hough voting scheme. By treating the uncertainty of keypoints' vote as the Lipschitz continuous function of seed points' fused embedding feature, our method is able to adaptively select the optimal keypoints vote. In this way, we argue that KVNet bridges the gap between the non-linear rotation space and linear Euclidean space, which introduces inductive bias for our network to learn the intrinsic pattern and infer 6D pose from RGB and depth images. Furthermore, our model will refine the initial keypoints localization with iterative fashion. Experiments show that across three challenging benchmark datasets (LineMOD, YCB-Video and Occlusion LineMOD), our method exhibits excellent performance. [ABSTRACT FROM AUTHOR]

Monocular 3D object detection aims to localize objects in 3D space from a single image. This is a difficult problem due to the lack of accurate depth measurements. Many methods predict depths upfront using a pre-trained vision-based depth estimator to assist in 3D object detection. However, the methods show limited improved performances because of the depth inaccuracy and the neglect of depth confidence. In this paper, we propose a new end-to-end 3D object detection framework by combining a monocular camera with a cheap 4-beam LiDAR. The minimal LiDAR signal is leveraged as an additional input to predict high-quality and dense depth maps from monocular images. Meanwhile, several 3D proposals are generated by a keypoint-based detector. The key challenge is encoding the depth confidence to capture the depth estimation uncertainty. Therefore, we propose probabilistic instance shape reconstruction to exploit instance shape information for box refinement. Our method employs a fully differentiable end-to-end framework, making it simple and efficient. The experimental results on the KITTI dataset demonstrate that the proposed method achieves state-of-the-art performance, thus validating the effectiveness of the sparse LiDAR data and the probabilistic instance shape reconstruction. Code is available at https://github.com/xjtuwh/SparseLiDAR_fusion. [ABSTRACT FROM AUTHOR]

• A blind spot speckle suppression method based on a global awareness strategy is proposed to achieve self-supervised denoising of sonar images, which helps to reduce the noise interference to downstream tasks. • A hybrid contextual dynamic attention mechanism (HCDAM) is introduced to enhance and correct the locality map for salient region extraction to improve the reliability of segmented targets. • A co-training method for the joint speckle suppression task and the garbage instance segmentation task is presented, which facilitates the flexible execution of underwater garbage detection tasks. Sonar is widely used in marine water cleaning tasks, so sonar images have become an effective tool for garbage detection and underwater scene analysis. However, it is an extremely difficult task to achieve fully supervised denoising and garbage detection for sonar images. This is because sonar images are weakly annotated samples that are susceptible to noise interference and have no reference clean image. To this end, we propose a sonar image garbage instance segmentation model via global despeckling and dynamic attention graph optimization(GD-DAGO). Specifically, a self-supervised blind spot network denoising structure is presented in this paper. The proposed denoising model overcomes the defects of information loss in the traditional blind spot network structure and performs global awareness speckle suppression for the noise characteristics of sonar images themselves. In addition, a novel dynamic attention structure is employed to improve the target region estimation in the instance segmentation module and does not require supervision beyond image-level category labeling. Finally, in order to enhance the cooperative ability between the two tasks, we adopt a local perceptual loss strategy based on mask proposals guided by the downstream task, so that the whole model takes more into account the characteristics of sonar images and better serves the sonar garbage detection task. Experimental results on ARACATI 2017 and marine-debris-fls-datasets (MDFD) show that the proposed algorithm achieves a performance gain of 0.4218 and 4.2% in terms of denoising effect (ENL) and detection accuracy ( AP 25 ), respectively, compared with suboptimal algorithms. [ABSTRACT FROM AUTHOR]

In most Internet of Things (IoT) scenes, data missing can be unavoidable when huge number of smart devices are collecting data uninterruptedly. Therefore, data imputation can be an integral part of pre-processing before data mining. It is widely known that IoT time series show strong dependencies in both spatial and temporal dimension, and the spatial relation among the devices is in non-euclidean space. However, most machine-learning-based and deep-learning-based approaches either only take temporal features into account or only catch spatial features in euclidean space. In this paper, we propose a novel network as ST-VAE (Spatio-Temporal Variational Auto-Encoder) to address the problem above. Our architecture is mainly based on Variational Auto-Encoder (VAE). Specifically, two kinds of VAE are utilized. One is for calculating the adjacent matrix of device network which is the essential input of GCN, and the other is for data imputation task based on the spatial and temporal dependencies. Experiments conducted on different real-world and public datasets demonstrate that our ST-VAE can not only populate the missing spatio-temporal data accurately but also outperforms other state-of-art approaches from the whole. [ABSTRACT FROM AUTHOR]

[Display omitted] • We propose a black-box adversarial text generation scheme called TextGuise, which can be used for almost all text classification tasks. The experiments show that TextGuise is superior to the state-of-the-art text adversarial example generation methods in terms of its attack success rate and attack efficiency when the perturbation ratio does not exceed 0.2.The adversarial examples generated using TextGuise retain the semantic information and strong readability of the text. None of the current defense methods can effectively defend against TextGuise. Adversarial examples greatly compromise the security of deep learning models. The key to improving the robustness of a natural language processing (NLP) model is to study attacks and defenses involving adversarial text. However, the current adversarial attack methods still face problems, such as the low success rates of attacks on some datasets, and the existing defense methods can already successfully defend against some attack methods. As a result, such attacks are unable to dig deeper into the flaws of NLP models to inform further defense improvements. Hence, it is necessary to design an adversarial attack method with a wider attack range and stronger performance. Aiming at the advantages and disadvantages of existing methods, this paper proposes a new adaptive black-box text adversarial example generation scheme, TextGuise. First, we design a keyword selection method in which word scores are calculated by combining context semantics to select the appropriate keywords to modify. Second, to maintain semantics, new keyword substitution rules are designed in combination with the characteristics of text and popular text expressions. Finally, the best modification strategy is adaptively selected through a querying model to reduce the magnitudes of disturbances. TextGuise can automatically select replacement keywords and replacement strategies that efficiently generate adversarial examples with good readability for various text classification tasks. Attack experiments conducted with TextGuise on 5 datasets yield high attack success rates that can surpass 80% when the perturbation ratio does not exceed 0.2. In addition, we present and discuss experiments focusing on defense, text similarity, query times, time consumption, etc., to test the attack performance of TextGuise. The results show that our attack method can achieve a good balance among various metrics. [ABSTRACT FROM AUTHOR]

As a powerful data modeling tool, Heterogeneous Information Network (HIN) has been successfully used in auxiliary information exploitation to boost recommendation performance. For HIN based recommendation, it is challenging to extract and fuse useful features of user preferences and item attributes under different semantic paths in HINs. Existing methods leverage a pre-defined fusion function to integrate different semantics for recommendation, which cannot characterize the complex nonlinear interactions between users and items. In this paper, we present a general framework named MNRec, short for Meta-path fusion based Neural Recommendation, to extract and fuse user and item embeddings under different meta-paths for recommendation. Under the framework, we propose an instantiation of MNRec with Multi-Layer Perceptron (MLP) structure. It consists of two major steps, i.e., meta-path based heterogeneous network embedding and deep learning based rating prediction. Concretely, appropriate meta-paths are first designed according to domain knowledge. Then the embeddings of users and items are obtained through a meta-path and commuting matrix based heterogeneous network embedding method. Finally, in light of the powerful nonlinear modeling capabilities of deep neural networks, the learned embeddings under different meta-paths are integrated into a two-pathway MLP structure for rating prediction. Experimental results on three real-world datasets demonstrate the superiority and effectiveness of MNRec compared with state-of-the-art baselines in rating prediction. [ABSTRACT FROM AUTHOR]

[Display omitted] • The dense feature adaptive connection module improves the quality of depth map. • The distributed 3D convolution reduces the computational cost and memory space. • The joint loss function makes the network sensitive to small depth structures. The goal of Multi-View Stereo is to reconstruct the 3D point cloud model from multiple views. With the development of deep learning, more and more learning-based research has achieved remarkable results. However, existing methods ignore the fine-grained features of the bottom layer, which leads to the poor quality of model reconstruction, especially in terms of completeness. Besides, current methods still rely on a large amount of consumed memory resources because of the application of 3D convolution. To this end, this paper proposes a Multiple Granularities Feature Fusion Network for Multi-View Stereo, an end-to-end depth estimation network combining global and local features, which is characterized by fine-granularity multi-feature fusion. Firstly, we propose a dense feature adaptive connection module, which can adaptively fuse the global and local features in the scene, provide a more complete and effective feature map for inferring a more detailed depth map, and make the ultimate model more complete. Secondly, in order to further improve the accuracy and completeness of the reconstructed point cloud, we introduce normal and edge loss futead of only using depth loss functions as in the existing methods, which makes the network more sensitive to small depth structures. Finally, we propose distributed 3D convolution instead of traditional 3D convolution, which reduces memory consumption. The experimental results on the DTU and Tanks & Temples datasets demonstrate that the proposed method in this papaer achieves the state-of-the-art performance, which proves the accuracy and effectiveness of the MG-MVSNet proposed in this paper. [ABSTRACT FROM AUTHOR]

• Introduces the novel task of Sexual Facial Expression Recognition (SFER). • Triple-BigGAN; a conditional GAN with a generator, discriminator, and classifier. • SEA-Faces-30k; a new dataset with Erotic, Suggestive-Erotic, and Non-Erotic classes. • Evaluation using FER-2013, ExpW, MNIST, CIFAR-10, SVHN, and SEA-Faces-30k datasets. Automatic recognition of facial images showing erotic expressions can help to understand our social interaction and to detect non-appropriate images even when there is no nakedness present in them. This paper contemplates, for the first time, to exploit facial cues applied to automatic Sexual Facial Expression Recognition (SFER). With this goal, we introduce a new dataset named Sexual Expression and Activity Faces (SEA-Faces-30k) for SFER, which contains 30k manually labeled images under three categories: erotic, suggestive-erotic, and non-erotic. Deep Convolutional Neural Networks require large-scale annotated image datasets with diversity and variations to be properly trained. Unfortunately, gathering such a massive amount of data is not feasible in this area. Therefore, we present a new semi-supervised GAN framework named Triple-BigGAN, which learns a generative model and a classifier simultaneously. It learns both tasks in an end-to-end fashion while using unlabeled or partially labeled data. The Triple-BigGAN framework shows promising classification performance for the SFER task (i.e., 93.59 %) and other five benchmark datasets, i.e., FER-2013, CIFAR-10, Expression in-the-Wild (ExpW), Modified National Institute of Standards and Technology database (MNIST), and Street View House Numbers (SVHN). Next, we evaluated the quality of samples generated by Triple-BigGAN with a resolution of 256 × 256 pixels using Inception Score (IS) and Frechet Inception Distance (FID). Our approach obtained the best FID (i.e., 19.94 %) and IS (i.e., 97.98 %) scores on the SEA-Faces-30k dataset. Further, we empirically demonstrated that synthetic erotic face images generated by Triple-BigGAN could also help in improving the classification performance of deep supervised networks. [ABSTRACT FROM AUTHOR]

• We introduce the field of speech emotion recognition. • We review most popular datasets for validation. • We review current machine learning and neural networks models for SER. User emotional status recognition is becoming a key feature in advanced Human Computer Interfaces (HCI). A key source of emotional information is the spoken expression, which may be part of the interaction between the human and the machine. Speech emotion recognition (SER) is a very active area of research that involves the application of current machine learning and neural networks tools. This ongoing review covers recent and classical approaches to SER reported in the literature. [ABSTRACT FROM AUTHOR]

Class imbalance distribution widely exists in real-world engineering. However, the mainstream optimization algorithms that seek to minimize error will trap the deep learning model in sub-optimums when facing extreme class imbalance. It seriously harms the classification precision, especially in the minor classes. The essential reason is that the gradients of the classifier weights are imbalanced among the components from different classes. In this paper, we propose Attraction–Repulsion-Balanced Loss (ARB-Loss) to balance the different components of the gradients. We perform experiments on large-scale classification and segmentation datasets, and our ARB-Loss can achieve state-of-the-art performance via only one-stage training instead of 2-stage learning like nowadays SOTA works. [ABSTRACT FROM AUTHOR]

• Providing an in-depth and comprehensive review related to underwater object detection. • Extensively exploring the correlation between underwater image enhancement and underwater object detection. • Analyzing the impact and augmentation forms of underwater image enhancement in underwater object detection. • Discussing the challenges, future trends and applications of underwater object detection. Underwater object detection is one of the most challenging research topics in computer vision technology. The complex underwater environment makes underwater images suffer from high noise, low visibility, blurred edges, low contrast and color deviation, which brings significant challenges to underwater object detection tasks. In underwater object detection tasks, traditional object detection methods often perform poorly in terms of accuracy and generalization capabilities. Underwater object detection requires accurate, stable, generalizable, real-time and lightweight detection models, for which many researchers have proposed various underwater object detection techniques based on deep learning. Although many outstanding results have been achieved on underwater object detection over the years, the research status of underwater object detection techniques are still lack of unified induction, and some existing problems need to be further probed from the latest perspective. In addition, previous reviews lack analysis on the relationship between underwater image enhancement and object detection. Therefore, this paper provides a comprehensive review of the current research challenges, future development trends, and potential applications of underwater object detection techniques. More importantly, this paper has explored the internal relationship between underwater image enhancement and object detection, and analyzed the possible implementation manners of underwater image enhancement in the object detection task in order to further enhance its benefits. The experiments show the performances of current underwater image enhancement and state-of-the-art object detection algorithms, point out their limitations, and indicate that there is not a strict positive correlation between underwater image enhancement and the accuracy improvement of object detection. The domain shift caused by underwater image enhancement cannot be ignored. This paper can be regarded as a guide for future works on underwater object detection. [ABSTRACT FROM AUTHOR]

The brain is a unique organ that performs multiple processes simultaneously, such as sensory, motor, and cognitive function. However, several neurological diseases (ataxia, dystonia, Huntington's disease) or trauma affect the limb movement and there is no cure. Although brain-computer interfaces (BCIs) have been recently used to improve the quality of life for people with severe motor disabilities, anthropomorphic control of a prosthetic hand in upper limb rehabilitation still remains an unachieved goal. To this purpose, a hierarchical integration of neural commands to fingers was applied for execution of human hand grasping with better precision. For finger movement prediction and kinematics estimation, a neuromuscular approach was employed to establish a hierarchical synergy between electroencephalography (EEG) and electromyography (EMG). EEG, EMG and metacarpophalangeal (MCP) joint kinematics were acquired during five finger flexion movements of the human hand. EMG for five finger movements and kinematics were estimated from EEG using linear regression. A Long Short-Term Memory network (LSTM) and a random forest regressor were adjoined hierarchically for prediction of finger movements and estimation of finger kinematics from the estimated EMG. The results showed an average accuracy of 84.25 ± 0.61 % in predicting finger movements and an average minimum error of 0.318 ± 0.011 in terms of root mean squared error (RMSE) in predicting finger kinematics from EEG across six subjects and five fingers. These findings suggest the implementation of a hierarchical approach to develop anthropomorphic control for upper limb prostheses. [ABSTRACT FROM AUTHOR]

Infrared and visible image fusion aims to integrate complementary information from different types of images into one image. The existing image fusion methods are primarily based on convolutional neural network (CNN), which ignores long-range dependencies of images, resulting in the fusion network unable to generate images with good complementarity. Inspired by the importance of global information, we introduced the transformer technique into the CNN-based fusion network as a way to improve the entire image-level perception in complex fusion scenarios. In this paper, we propose an end-to-end image fusion framework based on transformer and hybrid feature extractor, which enables the network to focus on both global and local information, using the characteristics of transformer to compensate for the shortcomings of CNN itself. In our network, the dual-branch CNN module is used to extract the shallow features of images, and then the vision transformer module is used to obtain the global channel and spatial relationship in the features. Finally, the fusion results are obtained through the image reconstruction module. We calculate the loss in the features of different depths according to the different kinds of original images by using the pre-trained VGG19 network. The experimental results show the effectiveness of adding the vision transformer module. Compared with other traditional and deep learning methods, our method achieves state-of-the-art qualitative and quantitative experiments performance. [ABSTRACT FROM AUTHOR]

Analysis of point clouds through deep convolutional neural networks is an active area of research due to their massive real-world applications including autonomous driving, indoor navigation, robotics, virtual/augmented reality, unmanned aerial vehicles, and drone technology. However, capturing the fine-grained geometric and semantic properties for the underlying recognition task with raw unstructured point cloud is highly challenging due to the lack of explicit neighborhood relationship and sparsity among the points. In this paper, we have introduced a deep, hierarchical 3D point based architecture for object classification and part segmentation that is able to learn robust geometric features which remain invariant to both the geometry and orientation of the local patches. The proposed architecture consists of multiple layers of sampling, concentric annular convolution, pooling, and residual feature propagation blocks. In the skip connections of our deep residual design, we propose to use a combination of linear projection shortcut and nonlinear ReLU group normalization shortcut with batch normalization, to improve both the optimization landscape and the representational power. Our network achieves on par or even better than state-of-the-art results on synthetic and real-world object classification (i.e., ModelNet40 and ScanObjectNN) and part segmentation (i.e., ShapeNet-part) benchmark datasets. The implementation and results have been made available at ...https://github.com/Rabbia-Hassan/Deep_Annular_Residual_Feature_Learning_for_3dPointCloudsGitHub-link [ABSTRACT FROM AUTHOR]

• A crowd flow detection method is proposed for video sequences shot by a drone. • The method is based on a fully-convolutional network that performs crowd clustering. • The detected centroids of crowd-dense areas are tracked in consecutive frames. • The method has been tested on the Crowd Counting datasets of the VisDrone challenge. Crowd analysis from drones has attracted increasing attention in recent times due to the ease of use and affordable cost of these devices. However, how this technology can provide a solution to crowd flow detection is still an unexplored research question. To this end, we propose a crowd flow detection method for video sequences shot by a drone. The method is based on a fully-convolutional network that learns to perform crowd clustering in order to detect the centroids of crowd-dense areas and track their movement in consecutive frames. The proposed method proved effective and efficient when tested on the Crowd Counting datasets of the VisDrone challenge, characterized by video sequences rather than still images. The encouraging results show that the proposed method could open up new ways of analyzing high-level crowd behavior from drones. [ABSTRACT FROM AUTHOR]

With the introduction of spatial-spectral fusion and deep learning, the classification performance of hyperspectral imagery (HSI) has been promoted greatly. For some widely used datasets, the classification accuracy almost reaches 100%. However, for hyperspectral image classification, random sampling is still the most common strategy to collect the training and test samples. Because the training and test samples are randomly selected from the same image, so they have a high correlation and the classification results are overoptimistic. Besides, random sampling is not a good choice for practical applications because we cannot always collect training and test samples from the same region. Disjoint sampling selects training and testing samples from different local regions, which will provide a more objective performance evaluation for HSI classification models. In this paper, we first show the huge classification performance difference caused by different sampling strategies with a simple experiment, then we analyze the underlying reasons from the spectral information, spatial-spectral combination, sample overlapping and spatial distance, finally, a semi-supervised feature learning method is proposed for disjoint HSI classification, in which the spatial and spectral information are exploited effectively and reasonably. The experimental results based on three HSI datasets demonstrate the effectiveness of the proposed method. [ABSTRACT FROM AUTHOR]

• We develop a more robust long short-term memory network for short-term traffic flow forecasting. • We relax the distribution of prediction error every iteration to arbitrary distribution for the long short-term memory. • We demonstrate the superior performance of our model by extensive experiments. Timely and accurate traffic flow forecasting is open challenging. Canonical long short-term memory (LSTM) network is considered qualified to capture the long-term temporal dependencies in traffic flow. However, the training of LSTM networks is often guided by the mean square error (MSE) criterion. Such criterion depends on a strong assumption that the errors between the traffic flow and its predictions are Gaussian independent identically distributed. In this regard, the forecasting performance is seriously deteriorated by non-Gaussian noises inside the traffic flow sequences. To address this issue, we relax the assumption of the prediction errors to arbitrary distribution by a negative guided mixed correntropy criterion. Then, we formulate a robust loss function by the negative guided mixed correntropy criterion. We subsequently equip the loss function in an LSTM network, termed Δ free -LSTM, for short-term traffic flow forecasting. Extensive experiments on four benchmark datasets demonstrate that the Δ free -LSTM network outperforms the traditional parametric and nonparametric models, as well as state-of-the-art LSTM family models. The source code is available at https://github.com/541764418/Delta-free-LSTM. [ABSTRACT FROM AUTHOR]

• Sequence-to-sequence constituent parsers are lagging behind task-specific techniques. • We notably improve their accuracy by introducing a powerful neural architecture. • For the first time, we extend their coverage to deal with discontinuous constituent parsing. • Our text-to-parse transducer achieves a competitive performance on the main benchmarks. In order to achieve deep natural language understanding, syntactic constituent parsing is a vital step, highly demanded by many artificial intelligence systems to process both text and speech. One of the most recent proposals is the use of standard sequence-to-sequence models to perform constituent parsing as a machine translation task, instead of applying task-specific parsers. While they show a competitive performance, these text-to-parse transducers are still lagging behind classic techniques in terms of accuracy, coverage and speed. To close the gap, we here extend the framework of sequence-to-sequence models for constituent parsing, not only by providing a more powerful neural architecture for improving their performance, but also by enlarging their coverage to handle the most complex syntactic phenomena: discontinuous structures. To that end, we design several novel linearizations that can fully produce discontinuities and, for the first time, we test a sequence-to-sequence model on the main discontinuous benchmarks, obtaining competitive results on par with task-specific discontinuous constituent parsers and achieving state-of-the-art scores on the (discontinuous) English Penn Treebank. [ABSTRACT FROM AUTHOR]

[Display omitted] • We developed a novel internal learning process that finds the network architecture and trains the weights for the test image and blur kernel. • By designing the training sample for external and internal learning, the knowledge from the external and internal data can be reflected, leading to fast convergence. • We present a fractal network as an elegant super-network for an element-wise architecture search to adapt to the pixels from the test image. • The proposed method represents a generalized SR algorithm for various domain images with different kernels , and this ability is demonstrated experimentally. Fueled by the powerful learning ability of deep networks, generalized models have been proposed that use external datasets for single-image super-resolution tasks. However, a model trained only with external data may have difficulty in super-resolving images in a domain that differs from the training data. To solve this drawback, several methods have been proposed for internal learning approaches that learn the weights of the network accordance with the test image. Despite these attempts to adapt to specific images using internal learning, they suffer from poor performance due to lack of flexibility that comes from using a fixed architecture regardless of image domain. We thus propose a novel training process that includes external and internal learning. Our internal learning process finds a suitable network architecture and trains the weights for each unseen test image. The overall training process allows the network to learn obtain knowledge from external data and internal learning in a balanced manner. In blind and non-blind experiments, our proposed method outperforms state-of-the-art super-resolution algorithms in various image domains with different kernels. Our proposed approach obtains impressive results in terms of expressing detailed texture and accurate color in images from various domains. [ABSTRACT FROM AUTHOR]

A great many low-light image restoration methods have built their models according to Retinex theory. However, most of these methods cannot well achieve image detail enhancement. To achieve simultaneous restoration and enhancement, we study deep low-light image enhancement from a perspective of texture-structure decomposition, that is, learning image smoothing operator. Specifically, we design a low-light restoration and enhancement framework, in which a Deep Texture-Structure Decomposition (DTSD) network is introduced to estimate two complementary constituents: Fine-Texture (FT) and Prominent-Structure (PS) maps from low-light image. Since these two maps are leveraged to approximate FT and PS maps obtained from normal-light image, they can be combined as the restored image in a manner of pixel-wise addition. The DTSD network has three parts: U-attention block, Decomposition-Merger (DM) block, and Upsampling-Reconstruction (UR) block. To better explore multi-level informative features at different scales than U-Net, U-attention block is designed with intra group and inter group attentions. In the DM block, we extract high-frequency and low-frequency features in low-resolution space. After obtaining informative feature maps from these two blocks, these maps are fed into the UR block for the final prediction. Numerous experimental results have demonstrated that the proposed method can achieve simultaneous low-light image restoration and enhancement, and it has superior performance against many state-of-the-art approaches in terms of several objective and perceptual metrics. [ABSTRACT FROM AUTHOR]

Identification and detection of thin-cap fibroatheroma (TCFA) from intravascular optical coherence tomography (IVOCT) images is critical for treatment of coronary heart diseases. Recently, deep learning methods have shown promising successes in TCFA identification. However, most methods usually do not effectively utilize multi-view information or incorporate prior domain knowledge. In this paper, we propose a multi-view contour-constrained transformer network (MVCTN) for TCFA identification in IVOCT images. Inspired by the diagnosis process of cardiologists, we use contour constrained self-attention modules (CCSM) to emphasize features corresponding to salient regions (i.e., vessel walls) in an unsupervised manner and enhance the visual interpretability based on class activation mapping (CAM). Moreover, we exploit transformer modules (TM) to build global-range relations between two views (i.e., polar and Cartesian views) to effectively fuse features at multiple feature scales. Experimental results on a semi-public dataset and an in-house dataset demonstrate that the proposed MVCTN outperforms other single-view and multi-view methods. Lastly, the proposed MVCTN can also provide meaningful visualization for cardiologists via CAM. [ABSTRACT FROM AUTHOR]

Estimating 3D human poses from images is an ill-posed regression problem, which is usually tackled by viewpoint-invariant convolutional neural networks (CNNs). Recently, capsule networks (CapsNets) have been introduced as a viable alternative to CNNs, ensuring viewpoint-equivariance and drastically reducing both the dataset size and the network complexity, while retaining high output accuracy. We propose a real-time end-to-end human pose estimation (HPE) network which employs state-of-the-art matrix capsules [1] and a fast variational Bayesian capsule routing, without relying on pre-training, complex data augmentation or multiple datasets. We achieve comparable results to the HPE state-of-the-art, and the lowest error among methods using CapsNets, while at the same time achieving other desirable properties, namely greater generalization capabilities, stronger viewpoint equivariance and highly decreased data dependency, allowing for our network to be trained with only a fraction of the available datasets and without any data augmentation. [ABSTRACT FROM AUTHOR]

Segmentation of retinal vessel images is critical to the diagnosis of retinopathy. Recently, convolutional neural networks have shown significant ability to extract the blood vessel structure. However, it remains challenging to refined segmentation for the capillaries and the edges of retinal vessels due to thickness inconsistencies and blurry boundaries. In this paper, we propose a novel deep neural network for retinal vessel segmentation based on shared decoder and pyramid-like loss (SPNet) to address the above problems. Specifically, we introduce a decoder-sharing mechanism to capture multi-scale semantic information, where feature maps at diverse scales are decoded through a sequence of weight-sharing decoder modules. Also, to strengthen characterization on the capillaries and the edges of blood vessels, we define a residual pyramid architecture which decomposes the spatial information in the decoding phase. A pyramid-like loss function is designed to compensate possible segmentation errors progressively. Experimental results on public benchmarks show that the proposed method outperforms the backbone network and most state-of-the-art methods, especially in the regions of the capillaries and the vessel contours. In addition, performances on cross-datasets verify that SPNet shows stronger generalization ability. [ABSTRACT FROM AUTHOR]